--- /dev/null
+What: /sys/class/leds/<led>/hw_pattern
+Date: September 2018
+KernelVersion: 4.20
+Description:
+ Specify a hardware pattern for the SC27XX LED. For the SC27XX
+ LED controller, it only supports 4 stages to make a single
+ hardware pattern, which is used to configure the rise time,
+ high time, fall time and low time for the breathing mode.
+
+ For the breathing mode, the SC27XX LED only expects one brightness
+ for the high stage. To be compatible with the hardware pattern
+ format, we should set brightness as 0 for rise stage, fall
+ stage and low stage.
+
+ Min stage duration: 125 ms
+ Max stage duration: 31875 ms
+
+ Since the stage duration step is 125 ms, the duration should be
+ a multiplier of 125, like 125ms, 250ms, 375ms, 500ms ... 31875ms.
+
+ Thus the format of the hardware pattern values should be:
+ "0 rise_duration brightness high_duration 0 fall_duration 0 low_duration".
--- /dev/null
+What: /sys/class/leds/<led>/pattern
+Date: September 2018
+KernelVersion: 4.20
+Description:
+ Specify a software pattern for the LED, that supports altering
+ the brightness for the specified duration with one software
+ timer. It can do gradual dimming and step change of brightness.
+
+ The pattern is given by a series of tuples, of brightness and
+ duration (ms). The LED is expected to traverse the series and
+ each brightness value for the specified duration. Duration of
+ 0 means brightness should immediately change to new value, and
+ writing malformed pattern deactivates any active one.
+
+ 1. For gradual dimming, the dimming interval now is set as 50
+ milliseconds. So the tuple with duration less than dimming
+ interval (50ms) is treated as a step change of brightness,
+ i.e. the subsequent brightness will be applied without adding
+ intervening dimming intervals.
+
+ The gradual dimming format of the software pattern values should be:
+ "brightness_1 duration_1 brightness_2 duration_2 brightness_3
+ duration_3 ...". For example:
+
+ echo 0 1000 255 2000 > pattern
+
+ It will make the LED go gradually from zero-intensity to max (255)
+ intensity in 1000 milliseconds, then back to zero intensity in 2000
+ milliseconds:
+
+ LED brightness
+ ^
+ 255-| / \ / \ /
+ | / \ / \ /
+ | / \ / \ /
+ | / \ / \ /
+ 0-| / \/ \/
+ +---0----1----2----3----4----5----6------------> time (s)
+
+ 2. To make the LED go instantly from one brigntess value to another,
+ we should use use zero-time lengths (the brightness must be same as
+ the previous tuple's). So the format should be:
+ "brightness_1 duration_1 brightness_1 0 brightness_2 duration_2
+ brightness_2 0 ...". For example:
+
+ echo 0 1000 0 0 255 2000 255 0 > pattern
+
+ It will make the LED stay off for one second, then stay at max brightness
+ for two seconds:
+
+ LED brightness
+ ^
+ 255-| +---------+ +---------+
+ | | | | |
+ | | | | |
+ | | | | |
+ 0-| -----+ +----+ +----
+ +---0----1----2----3----4----5----6------------> time (s)
+
+What: /sys/class/leds/<led>/hw_pattern
+Date: September 2018
+KernelVersion: 4.20
+Description:
+ Specify a hardware pattern for the LED, for LED hardware that
+ supports autonomously controlling brightness over time, according
+ to some preprogrammed hardware patterns. It deactivates any active
+ software pattern.
+
+ Since different LED hardware can have different semantics of
+ hardware patterns, each driver is expected to provide its own
+ description for the hardware patterns in their ABI documentation
+ file.
+
+What: /sys/class/leds/<led>/repeat
+Date: September 2018
+KernelVersion: 4.20
+Description:
+ Specify a pattern repeat number. -1 means repeat indefinitely,
+ other negative numbers and number 0 are invalid.
+
+ This file will always return the originally written repeat
+ number.
this file, the suspend image will be as small as possible.
Reading from this file will display the current image size
- limit, which is set to 500 MB by default.
+ limit, which is set to around 2/5 of available RAM by default.
What: /sys/power/pm_trace
Date: August 2006
CPU enters the idle loop from process context.
</p><p>The <tt>->dynticks</tt> field counts the corresponding
-CPU's transitions to and from dyntick-idle mode, so that this counter
-has an even value when the CPU is in dyntick-idle mode and an odd
-value otherwise.
+CPU's transitions to and from either dyntick-idle or user mode, so
+that this counter has an even value when the CPU is in dyntick-idle
+mode or user mode and an odd value otherwise. The transitions to/from
+user mode need to be counted for user mode adaptive-ticks support
+(see timers/NO_HZ.txt).
</p><p>The <tt>->rcu_need_heavy_qs</tt> field is used
to record the fact that the RCU core code would really like to
Accessor Functions</a></h3>
<p>The following listing shows the
-<tt>rcu_get_root()</tt>, <tt>rcu_for_each_node_breadth_first</tt>,
-<tt>rcu_for_each_nonleaf_node_breadth_first()</tt>, and
+<tt>rcu_get_root()</tt>, <tt>rcu_for_each_node_breadth_first</tt> and
<tt>rcu_for_each_leaf_node()</tt> function and macros:
<pre>
7 for ((rnp) = &(rsp)->node[0]; \
8 (rnp) < &(rsp)->node[NUM_RCU_NODES]; (rnp)++)
9
- 10 #define rcu_for_each_nonleaf_node_breadth_first(rsp, rnp) \
- 11 for ((rnp) = &(rsp)->node[0]; \
- 12 (rnp) < (rsp)->level[NUM_RCU_LVLS - 1]; (rnp)++)
- 13
- 14 #define rcu_for_each_leaf_node(rsp, rnp) \
- 15 for ((rnp) = (rsp)->level[NUM_RCU_LVLS - 1]; \
- 16 (rnp) < &(rsp)->node[NUM_RCU_NODES]; (rnp)++)
+ 10 #define rcu_for_each_leaf_node(rsp, rnp) \
+ 11 for ((rnp) = (rsp)->level[NUM_RCU_LVLS - 1]; \
+ 12 (rnp) < &(rsp)->node[NUM_RCU_NODES]; (rnp)++)
</pre>
<p>The <tt>rcu_get_root()</tt> simply returns a pointer to the
structures in the <tt>rcu_state</tt> structure's
<tt>->node[]</tt> array, performing a breadth-first traversal by
simply traversing the array in order.
-The <tt>rcu_for_each_nonleaf_node_breadth_first()</tt> macro operates
-similarly, but traverses only the first part of the array, thus excluding
-the leaf <tt>rcu_node</tt> structures.
-Finally, the <tt>rcu_for_each_leaf_node()</tt> macro traverses only
+Similarly, the <tt>rcu_for_each_leaf_node()</tt> macro traverses only
the last part of the array, thus traversing only the leaf
<tt>rcu_node</tt> structures.
<tr><th> </th></tr>
<tr><th align="left">Quick Quiz:</th></tr>
<tr><td>
- What do <tt>rcu_for_each_nonleaf_node_breadth_first()</tt> and
+ What does
<tt>rcu_for_each_leaf_node()</tt> do if the <tt>rcu_node</tt> tree
contains only a single node?
</td></tr>
<tr><th align="left">Answer:</th></tr>
<tr><td bgcolor="#ffffff"><font color="ffffff">
In the single-node case,
- <tt>rcu_for_each_nonleaf_node_breadth_first()</tt> is a no-op
- and <tt>rcu_for_each_leaf_node()</tt> traverses the single node.
+ <tt>rcu_for_each_leaf_node()</tt> traverses the single node.
</font></td></tr>
<tr><td> </td></tr>
</table>
lower efficiency and significant disturbance to attain shorter latencies.
<p>
-There are three flavors of RCU (RCU-bh, RCU-preempt, and RCU-sched),
-but only two flavors of expedited grace periods because the RCU-bh
-expedited grace period maps onto the RCU-sched expedited grace period.
-Each of the remaining two implementations is covered in its own section.
+There are two flavors of RCU (RCU-preempt and RCU-sched), with an earlier
+third RCU-bh flavor having been implemented in terms of the other two.
+Each of the two implementations is covered in its own section.
<ol>
<li> <a href="#Expedited Grace Period Design">
The best that <tt>sync_sched_exp_handler()</tt> can do is to check
for idle, on the off-chance that the CPU went idle while the IPI
was in flight.
-If the CPU is idle, then tt>sync_sched_exp_handler()</tt> reports
+If the CPU is idle, then <tt>sync_sched_exp_handler()</tt> reports
the quiescent state.
<p>
<p>
This non-requirement appeared with preemptible RCU.
-If you need a grace period that waits on non-preemptible code regions, use
-<a href="#Sched Flavor">RCU-sched</a>.
<h2><a name="Parallelism Facts of Life">Parallelism Facts of Life</a></h2>
on what operations those callbacks could invoke.
<p>
-Perhaps surprisingly, <tt>synchronize_rcu()</tt>,
-<a href="#Bottom-Half Flavor"><tt>synchronize_rcu_bh()</tt></a>
-(<a href="#Bottom-Half Flavor">discussed below</a>),
-<a href="#Sched Flavor"><tt>synchronize_sched()</tt></a>,
+Perhaps surprisingly, <tt>synchronize_rcu()</tt> and
<tt>synchronize_rcu_expedited()</tt>,
-<tt>synchronize_rcu_bh_expedited()</tt>, and
-<tt>synchronize_sched_expedited()</tt>
-will all operate normally
+will operate normally
during very early boot, the reason being that there is only one CPU
and preemption is disabled.
This means that the call <tt>synchronize_rcu()</tt> (or friends)
The name notwithstanding, some Linux-kernel architectures
can have nested NMIs, which RCU must handle correctly.
Andy Lutomirski
-<a href="https://lkml.kernel.org/g/CALCETrXLq1y7e_dKFPgou-FKHB6Pu-r8+t-6Ds+8=va7anBWDA@mail.gmail.com">surprised me</a>
+<a href="https://lkml.kernel.org/r/CALCETrXLq1y7e_dKFPgou-FKHB6Pu-r8+t-6Ds+8=va7anBWDA@mail.gmail.com">surprised me</a>
with this requirement;
he also kindly surprised me with
-<a href="https://lkml.kernel.org/g/CALCETrXSY9JpW3uE6H8WYk81sg56qasA2aqmjMPsq5dOtzso=g@mail.gmail.com">an algorithm</a>
+<a href="https://lkml.kernel.org/r/CALCETrXSY9JpW3uE6H8WYk81sg56qasA2aqmjMPsq5dOtzso=g@mail.gmail.com">an algorithm</a>
that meets this requirement.
+<p>
+Furthermore, NMI handlers can be interrupted by what appear to RCU
+to be normal interrupts.
+One way that this can happen is for code that directly invokes
+<tt>rcu_irq_enter()</tt> and </tt>rcu_irq_exit()</tt> to be called
+from an NMI handler.
+This astonishing fact of life prompted the current code structure,
+which has <tt>rcu_irq_enter()</tt> invoking <tt>rcu_nmi_enter()</tt>
+and <tt>rcu_irq_exit()</tt> invoking <tt>rcu_nmi_exit()</tt>.
+And yes, I also learned of this requirement the hard way.
+
<h3><a name="Loadable Modules">Loadable Modules</a></h3>
<p>
<p>
RCU depends on the scheduler, and the scheduler uses RCU to
protect some of its data structures.
-This means the scheduler is forbidden from acquiring
-the runqueue locks and the priority-inheritance locks
-in the middle of an outermost RCU read-side critical section unless either
-(1) it releases them before exiting that same
-RCU read-side critical section, or
-(2) interrupts are disabled across
-that entire RCU read-side critical section.
-This same prohibition also applies (recursively!) to any lock that is acquired
-while holding any lock to which this prohibition applies.
-Adhering to this rule prevents preemptible RCU from invoking
-<tt>rcu_read_unlock_special()</tt> while either runqueue or
-priority-inheritance locks are held, thus avoiding deadlock.
-
-<p>
-Prior to v4.4, it was only necessary to disable preemption across
-RCU read-side critical sections that acquired scheduler locks.
-In v4.4, expedited grace periods started using IPIs, and these
-IPIs could force a <tt>rcu_read_unlock()</tt> to take the slowpath.
-Therefore, this expedited-grace-period change required disabling of
-interrupts, not just preemption.
-
-<p>
-For RCU's part, the preemptible-RCU <tt>rcu_read_unlock()</tt>
-implementation must be written carefully to avoid similar deadlocks.
+The preemptible-RCU <tt>rcu_read_unlock()</tt>
+implementation must therefore be written carefully to avoid deadlocks
+involving the scheduler's runqueue and priority-inheritance locks.
In particular, <tt>rcu_read_unlock()</tt> must tolerate an
interrupt where the interrupt handler invokes both
<tt>rcu_read_lock()</tt> and <tt>rcu_read_unlock()</tt>.
interrupt handler's use of RCU.
<p>
-This pair of mutual scheduler-RCU requirements came as a
+This scheduler-RCU requirement came as a
<a href="https://lwn.net/Articles/453002/">complete surprise</a>.
<p>
<tt>CONFIG_NO_HZ_FULL=y</tt>
<a href="http://www.rdrop.com/users/paulmck/scalability/paper/BareMetal.2015.01.15b.pdf">did come as a surprise [PDF]</a>.
RCU has made good progress towards meeting this requirement, even
-for context-switch-have <tt>CONFIG_NO_HZ_FULL=y</tt> workloads,
+for context-switch-heavy <tt>CONFIG_NO_HZ_FULL=y</tt> workloads,
but there is room for further improvement.
+<p>
+In the past, it was forbidden to disable interrupts across an
+<tt>rcu_read_unlock()</tt> unless that interrupt-disabled region
+of code also included the matching <tt>rcu_read_lock()</tt>.
+Violating this restriction could result in deadlocks involving the
+scheduler's runqueue and priority-inheritance spinlocks.
+This restriction was lifted when interrupt-disabled calls to
+<tt>rcu_read_unlock()</tt> started deferring the reporting of
+the resulting RCU-preempt quiescent state until the end of that
+interrupts-disabled region.
+This deferred reporting means that the scheduler's runqueue and
+priority-inheritance locks cannot be held while reporting an RCU-preempt
+quiescent state, which lifts the earlier restriction, at least from
+a deadlock perspective.
+Unfortunately, real-time systems using RCU priority boosting may
+need this restriction to remain in effect because deferred
+quiescent-state reporting also defers deboosting, which in turn
+degrades real-time latencies.
+
<h3><a name="Tracing and RCU">Tracing and RCU</a></h3>
<p>
described in a separate section.
<ol>
-<li> <a href="#Bottom-Half Flavor">Bottom-Half Flavor</a>
-<li> <a href="#Sched Flavor">Sched Flavor</a>
+<li> <a href="#Bottom-Half Flavor">Bottom-Half Flavor (Historical)</a>
+<li> <a href="#Sched Flavor">Sched Flavor (Historical)</a>
<li> <a href="#Sleepable RCU">Sleepable RCU</a>
<li> <a href="#Tasks RCU">Tasks RCU</a>
-<li> <a href="#Waiting for Multiple Grace Periods">
- Waiting for Multiple Grace Periods</a>
</ol>
-<h3><a name="Bottom-Half Flavor">Bottom-Half Flavor</a></h3>
+<h3><a name="Bottom-Half Flavor">Bottom-Half Flavor (Historical)</a></h3>
+
+<p>
+The RCU-bh flavor of RCU has since been expressed in terms of
+the other RCU flavors as part of a consolidation of the three
+flavors into a single flavor.
+The read-side API remains, and continues to disable softirq and to
+be accounted for by lockdep.
+Much of the material in this section is therefore strictly historical
+in nature.
<p>
The softirq-disable (AKA “bottom-half”,
<tt>call_rcu_bh()</tt>,
<tt>rcu_barrier_bh()</tt>, and
<tt>rcu_read_lock_bh_held()</tt>.
+However, the update-side APIs are now simple wrappers for other RCU
+flavors, namely RCU-sched in CONFIG_PREEMPT=n kernels and RCU-preempt
+otherwise.
+
+<h3><a name="Sched Flavor">Sched Flavor (Historical)</a></h3>
-<h3><a name="Sched Flavor">Sched Flavor</a></h3>
+<p>
+The RCU-sched flavor of RCU has since been expressed in terms of
+the other RCU flavors as part of a consolidation of the three
+flavors into a single flavor.
+The read-side API remains, and continues to disable preemption and to
+be accounted for by lockdep.
+Much of the material in this section is therefore strictly historical
+in nature.
<p>
Before preemptible RCU, waiting for an RCU grace period had the
<tt>call_rcu_tasks()</tt>,
<tt>synchronize_rcu_tasks()</tt>, and
<tt>rcu_barrier_tasks()</tt>.
-
-<h3><a name="Waiting for Multiple Grace Periods">
-Waiting for Multiple Grace Periods</a></h3>
-
-<p>
-Perhaps you have an RCU protected data structure that is accessed from
-RCU read-side critical sections, from softirq handlers, and from
-hardware interrupt handlers.
-That is three flavors of RCU, the normal flavor, the bottom-half flavor,
-and the sched flavor.
-How to wait for a compound grace period?
-
-<p>
-The best approach is usually to “just say no!” and
-insert <tt>rcu_read_lock()</tt> and <tt>rcu_read_unlock()</tt>
-around each RCU read-side critical section, regardless of what
-environment it happens to be in.
-But suppose that some of the RCU read-side critical sections are
-on extremely hot code paths, and that use of <tt>CONFIG_PREEMPT=n</tt>
-is not a viable option, so that <tt>rcu_read_lock()</tt> and
-<tt>rcu_read_unlock()</tt> are not free.
-What then?
-
-<p>
-You <i>could</i> wait on all three grace periods in succession, as follows:
-
-<blockquote>
-<pre>
- 1 synchronize_rcu();
- 2 synchronize_rcu_bh();
- 3 synchronize_sched();
-</pre>
-</blockquote>
-
-<p>
-This works, but triples the update-side latency penalty.
-In cases where this is not acceptable, <tt>synchronize_rcu_mult()</tt>
-may be used to wait on all three flavors of grace period concurrently:
-
-<blockquote>
-<pre>
- 1 synchronize_rcu_mult(call_rcu, call_rcu_bh, call_rcu_sched);
-</pre>
-</blockquote>
-
-<p>
-But what if it is necessary to also wait on SRCU?
-This can be done as follows:
-
-<blockquote>
-<pre>
- 1 static void call_my_srcu(struct rcu_head *head,
- 2 void (*func)(struct rcu_head *head))
- 3 {
- 4 call_srcu(&my_srcu, head, func);
- 5 }
- 6
- 7 synchronize_rcu_mult(call_rcu, call_rcu_bh, call_rcu_sched, call_my_srcu);
-</pre>
-</blockquote>
-
-<p>
-If you needed to wait on multiple different flavors of SRCU
-(but why???), you would need to create a wrapper function resembling
-<tt>call_my_srcu()</tt> for each SRCU flavor.
-
-<table>
-<tr><th> </th></tr>
-<tr><th align="left">Quick Quiz:</th></tr>
-<tr><td>
- But what if I need to wait for multiple RCU flavors, but I also need
- the grace periods to be expedited?
-</td></tr>
-<tr><th align="left">Answer:</th></tr>
-<tr><td bgcolor="#ffffff"><font color="ffffff">
- If you are using expedited grace periods, there should be less penalty
- for waiting on them in succession.
- But if that is nevertheless a problem, you can use workqueues
- or multiple kthreads to wait on the various expedited grace
- periods concurrently.
-</font></td></tr>
-<tr><td> </td></tr>
-</table>
-
-<p>
-Again, it is usually better to adjust the RCU read-side critical sections
-to use a single flavor of RCU, but when this is not feasible, you can use
-<tt>synchronize_rcu_mult()</tt>.
+In <tt>CONFIG_PREEMPT=n</tt> kernels, trampolines cannot be preempted,
+so these APIs map to
+<tt>call_rcu()</tt>,
+<tt>synchronize_rcu()</tt>, and
+<tt>rcu_barrier()</tt>, respectively.
+In <tt>CONFIG_PREEMPT=y</tt> kernels, trampolines can be preempted,
+and these three APIs are therefore implemented by separate functions
+that check for voluntary context switches.
<h2><a name="Possible Future Changes">Possible Future Changes</a></h2>
grace-period state machine so as to avoid the need for the additional
latency.
-<p>
-Expedited grace periods scan the CPUs, so their latency and overhead
-increases with increasing numbers of CPUs.
-If this becomes a serious problem on large systems, it will be necessary
-to do some redesign to avoid this scalability problem.
-
<p>
RCU disables CPU hotplug in a few places, perhaps most notably in the
<tt>rcu_barrier()</tt> operations.
require extremely good demonstration of need and full exploration of
alternatives.
-<p>
-There is an embarrassingly large number of flavors of RCU, and this
-number has been increasing over time.
-Perhaps it will be possible to combine some at some future date.
-
<p>
RCU's various kthreads are reasonably recent additions.
It is quite likely that adjustments will be required to more gracefully
o A CPU looping with interrupts disabled.
-o A CPU looping with preemption disabled. This condition can
- result in RCU-sched stalls and, if ksoftirqd is in use, RCU-bh
- stalls.
+o A CPU looping with preemption disabled.
-o A CPU looping with bottom halves disabled. This condition can
- result in RCU-sched and RCU-bh stalls.
+o A CPU looping with bottom halves disabled.
o For !CONFIG_PREEMPT kernels, a CPU looping anywhere in the kernel
without invoking schedule(). If the looping in the kernel is
This resulted in a series of RCU CPU stall warnings, eventually
leading the realization that the CPU had failed.
-The RCU, RCU-sched, RCU-bh, and RCU-tasks implementations have CPU stall
-warning. Note that SRCU does -not- have CPU stall warnings. Please note
-that RCU only detects CPU stalls when there is a grace period in progress.
+The RCU, RCU-sched, and RCU-tasks implementations have CPU stall warning.
+Note that SRCU does -not- have CPU stall warnings. Please note that
+RCU only detects CPU stalls when there is a grace period in progress.
No grace period, no CPU stall warnings.
To diagnose the cause of the stall, inspect the stack traces.
d. Do you need RCU grace periods to complete even in the face
of softirq monopolization of one or more of the CPUs? For
example, is your code subject to network-based denial-of-service
- attacks? If so, you need RCU-bh.
+ attacks? If so, you should disable softirq across your readers,
+ for example, by using rcu_read_lock_bh().
e. Is your workload too update-intensive for normal use of
RCU, but inappropriate for other synchronization mechanisms?
wbc_init_bio(@wbc, @bio)
Should be called for each bio carrying writeback data and
- associates the bio with the inode's owner cgroup. Can be
- called anytime between bio allocation and submission.
+ associates the bio with the inode's owner cgroup and the
+ corresponding request queue. This must be called after
+ a queue (device) has been associated with the bio and
+ before submission.
wbc_account_io(@wbc, @page, @bytes)
Should be called for each data segment being written out.
the writeback session is holding shared resources, e.g. a journal
entry, may lead to priority inversion. There is no one easy solution
for the problem. Filesystems can try to work around specific problem
-cases by skipping wbc_init_bio() or using bio_associate_blkcg()
+cases by skipping wbc_init_bio() or using bio_associate_create_blkg()
directly.
causing system reset or hang due to sending
INIT from AP to BSP.
+ disable_counter_freezing [HW]
+ Disable Intel PMU counter freezing feature.
+ The feature only exists starting from
+ Arch Perfmon v4 (Skylake and newer).
+
disable_ddw [PPC/PSERIES]
Disable Dynamic DMA Window support. Use this if
to workaround buggy firmware.
In kernels built with CONFIG_RCU_NOCB_CPU=y, set
the specified list of CPUs to be no-callback CPUs.
- Invocation of these CPUs' RCU callbacks will
- be offloaded to "rcuox/N" kthreads created for
- that purpose, where "x" is "b" for RCU-bh, "p"
- for RCU-preempt, and "s" for RCU-sched, and "N"
- is the CPU number. This reduces OS jitter on the
- offloaded CPUs, which can be useful for HPC and
- real-time workloads. It can also improve energy
- efficiency for asymmetric multiprocessors.
+ Invocation of these CPUs' RCU callbacks will be
+ offloaded to "rcuox/N" kthreads created for that
+ purpose, where "x" is "p" for RCU-preempt, and
+ "s" for RCU-sched, and "N" is the CPU number.
+ This reduces OS jitter on the offloaded CPUs,
+ which can be useful for HPC and real-time
+ workloads. It can also improve energy efficiency
+ for asymmetric multiprocessors.
rcu_nocb_poll [KNL]
Rather than requiring that offloaded CPUs
Set required age in jiffies for a
given grace period before RCU starts
soliciting quiescent-state help from
- rcu_note_context_switch().
+ rcu_note_context_switch(). If not specified, the
+ kernel will calculate a value based on the most
+ recent settings of rcutree.jiffies_till_first_fqs
+ and rcutree.jiffies_till_next_fqs.
+ This calculated value may be viewed in
+ rcutree.jiffies_to_sched_qs. Any attempt to
+ set rcutree.jiffies_to_sched_qs will be
+ cheerfully overwritten.
rcutree.jiffies_till_first_fqs= [KNL]
Set delay from grace-period initialization to
rcupdate.rcu_self_test= [KNL]
Run the RCU early boot self tests
- rcupdate.rcu_self_test_bh= [KNL]
- Run the RCU bh early boot self tests
-
- rcupdate.rcu_self_test_sched= [KNL]
- Run the RCU sched early boot self tests
-
rdinit= [KNL]
Format: <full_path>
Run specified binary instead of /init from the ramdisk,
policy for the time interval between the last two invocations of the
driver's utilization update callback by the CPU scheduler for that CPU.
+One more policy attribute is present if the `HWP feature is enabled in the
+processor <Active Mode With HWP_>`_:
+
+``base_frequency``
+ Shows the base frequency of the CPU. Any frequency above this will be
+ in the turbo frequency range.
+
The meaning of these attributes in the `passive mode <Passive Mode_>`_ is the
same as for other scaling drivers.
HWCAP_AES
- Functionality implied by ID_AA64ISAR1_EL1.AES == 0b0001.
+ Functionality implied by ID_AA64ISAR0_EL1.AES == 0b0001.
HWCAP_PMULL
- Functionality implied by ID_AA64ISAR1_EL1.AES == 0b0010.
+ Functionality implied by ID_AA64ISAR0_EL1.AES == 0b0010.
HWCAP_SHA1
HWCAP_SHA512
- Functionality implied by ID_AA64ISAR0_EL1.SHA2 == 0b0002.
+ Functionality implied by ID_AA64ISAR0_EL1.SHA2 == 0b0010.
HWCAP_SVE
HWCAP_ILRCPC
- Functionality implied by ID_AA64ISR1_EL1.LRCPC == 0b0002.
+ Functionality implied by ID_AA64ISAR1_EL1.LRCPC == 0b0010.
HWCAP_FLAGM
Functionality implied by ID_AA64ISAR0_EL1.TS == 0b0001.
+
+HWCAP_SSBS
+
+ Functionality implied by ID_AA64PFR1_EL1.SSBS == 0b0010.
--- /dev/null
+HugeTLBpage on ARM64
+====================
+
+Hugepage relies on making efficient use of TLBs to improve performance of
+address translations. The benefit depends on both -
+
+ - the size of hugepages
+ - size of entries supported by the TLBs
+
+The ARM64 port supports two flavours of hugepages.
+
+1) Block mappings at the pud/pmd level
+--------------------------------------
+
+These are regular hugepages where a pmd or a pud page table entry points to a
+block of memory. Regardless of the supported size of entries in TLB, block
+mappings reduce the depth of page table walk needed to translate hugepage
+addresses.
+
+2) Using the Contiguous bit
+---------------------------
+
+The architecture provides a contiguous bit in the translation table entries
+(D4.5.3, ARM DDI 0487C.a) that hints to the MMU to indicate that it is one of a
+contiguous set of entries that can be cached in a single TLB entry.
+
+The contiguous bit is used in Linux to increase the mapping size at the pmd and
+pte (last) level. The number of supported contiguous entries varies by page size
+and level of the page table.
+
+
+The following hugepage sizes are supported -
+
+ CONT PTE PMD CONT PMD PUD
+ -------- --- -------- ---
+ 4K: 64K 2M 32M 1G
+ 16K: 2M 32M 1G
+ 64K: 2M 512M 16G
| ARM | Cortex-A72 | #853709 | N/A |
| ARM | Cortex-A73 | #858921 | ARM64_ERRATUM_858921 |
| ARM | Cortex-A55 | #1024718 | ARM64_ERRATUM_1024718 |
+| ARM | Cortex-A76 | #1188873 | ARM64_ERRATUM_1188873 |
| ARM | MMU-500 | #841119,#826419 | N/A |
| | | | |
| Cavium | ThunderX ITS | #22375, #24313 | CAVIUM_ERRATUM_22375 |
+++ /dev/null
- Linux Driver for Mylex DAC960/AcceleRAID/eXtremeRAID PCI RAID Controllers
-
- Version 2.2.11 for Linux 2.2.19
- Version 2.4.11 for Linux 2.4.12
-
- PRODUCTION RELEASE
-
- 11 October 2001
-
- Leonard N. Zubkoff
- Dandelion Digital
- lnz@dandelion.com
-
- Copyright 1998-2001 by Leonard N. Zubkoff <lnz@dandelion.com>
-
-
- INTRODUCTION
-
-Mylex, Inc. designs and manufactures a variety of high performance PCI RAID
-controllers. Mylex Corporation is located at 34551 Ardenwood Blvd., Fremont,
-California 94555, USA and can be reached at 510.796.6100 or on the World Wide
-Web at http://www.mylex.com. Mylex Technical Support can be reached by
-electronic mail at mylexsup@us.ibm.com, by voice at 510.608.2400, or by FAX at
-510.745.7715. Contact information for offices in Europe and Japan is available
-on their Web site.
-
-The latest information on Linux support for DAC960 PCI RAID Controllers, as
-well as the most recent release of this driver, will always be available from
-my Linux Home Page at URL "http://www.dandelion.com/Linux/". The Linux DAC960
-driver supports all current Mylex PCI RAID controllers including the new
-eXtremeRAID 2000/3000 and AcceleRAID 352/170/160 models which have an entirely
-new firmware interface from the older eXtremeRAID 1100, AcceleRAID 150/200/250,
-and DAC960PJ/PG/PU/PD/PL. See below for a complete controller list as well as
-minimum firmware version requirements. For simplicity, in most places this
-documentation refers to DAC960 generically rather than explicitly listing all
-the supported models.
-
-Driver bug reports should be sent via electronic mail to "lnz@dandelion.com".
-Please include with the bug report the complete configuration messages reported
-by the driver at startup, along with any subsequent system messages relevant to
-the controller's operation, and a detailed description of your system's
-hardware configuration. Driver bugs are actually quite rare; if you encounter
-problems with disks being marked offline, for example, please contact Mylex
-Technical Support as the problem is related to the hardware configuration
-rather than the Linux driver.
-
-Please consult the RAID controller documentation for detailed information
-regarding installation and configuration of the controllers. This document
-primarily provides information specific to the Linux support.
-
-
- DRIVER FEATURES
-
-The DAC960 RAID controllers are supported solely as high performance RAID
-controllers, not as interfaces to arbitrary SCSI devices. The Linux DAC960
-driver operates at the block device level, the same level as the SCSI and IDE
-drivers. Unlike other RAID controllers currently supported on Linux, the
-DAC960 driver is not dependent on the SCSI subsystem, and hence avoids all the
-complexity and unnecessary code that would be associated with an implementation
-as a SCSI driver. The DAC960 driver is designed for as high a performance as
-possible with no compromises or extra code for compatibility with lower
-performance devices. The DAC960 driver includes extensive error logging and
-online configuration management capabilities. Except for initial configuration
-of the controller and adding new disk drives, most everything can be handled
-from Linux while the system is operational.
-
-The DAC960 driver is architected to support up to 8 controllers per system.
-Each DAC960 parallel SCSI controller can support up to 15 disk drives per
-channel, for a maximum of 60 drives on a four channel controller; the fibre
-channel eXtremeRAID 3000 controller supports up to 125 disk drives per loop for
-a total of 250 drives. The drives installed on a controller are divided into
-one or more "Drive Groups", and then each Drive Group is subdivided further
-into 1 to 32 "Logical Drives". Each Logical Drive has a specific RAID Level
-and caching policy associated with it, and it appears to Linux as a single
-block device. Logical Drives are further subdivided into up to 7 partitions
-through the normal Linux and PC disk partitioning schemes. Logical Drives are
-also known as "System Drives", and Drive Groups are also called "Packs". Both
-terms are in use in the Mylex documentation; I have chosen to standardize on
-the more generic "Logical Drive" and "Drive Group".
-
-DAC960 RAID disk devices are named in the style of the obsolete Device File
-System (DEVFS). The device corresponding to Logical Drive D on Controller C
-is referred to as /dev/rd/cCdD, and the partitions are called /dev/rd/cCdDp1
-through /dev/rd/cCdDp7. For example, partition 3 of Logical Drive 5 on
-Controller 2 is referred to as /dev/rd/c2d5p3. Note that unlike with SCSI
-disks the device names will not change in the event of a disk drive failure.
-The DAC960 driver is assigned major numbers 48 - 55 with one major number per
-controller. The 8 bits of minor number are divided into 5 bits for the Logical
-Drive and 3 bits for the partition.
-
-
- SUPPORTED DAC960/AcceleRAID/eXtremeRAID PCI RAID CONTROLLERS
-
-The following list comprises the supported DAC960, AcceleRAID, and eXtremeRAID
-PCI RAID Controllers as of the date of this document. It is recommended that
-anyone purchasing a Mylex PCI RAID Controller not in the following table
-contact the author beforehand to verify that it is or will be supported.
-
-eXtremeRAID 3000
- 1 Wide Ultra-2/LVD SCSI channel
- 2 External Fibre FC-AL channels
- 233MHz StrongARM SA 110 Processor
- 64 Bit 33MHz PCI (backward compatible with 32 Bit PCI slots)
- 32MB/64MB ECC SDRAM Memory
-
-eXtremeRAID 2000
- 4 Wide Ultra-160 LVD SCSI channels
- 233MHz StrongARM SA 110 Processor
- 64 Bit 33MHz PCI (backward compatible with 32 Bit PCI slots)
- 32MB/64MB ECC SDRAM Memory
-
-AcceleRAID 352
- 2 Wide Ultra-160 LVD SCSI channels
- 100MHz Intel i960RN RISC Processor
- 64 Bit 33MHz PCI (backward compatible with 32 Bit PCI slots)
- 32MB/64MB ECC SDRAM Memory
-
-AcceleRAID 170
- 1 Wide Ultra-160 LVD SCSI channel
- 100MHz Intel i960RM RISC Processor
- 16MB/32MB/64MB ECC SDRAM Memory
-
-AcceleRAID 160 (AcceleRAID 170LP)
- 1 Wide Ultra-160 LVD SCSI channel
- 100MHz Intel i960RS RISC Processor
- Built in 16M ECC SDRAM Memory
- PCI Low Profile Form Factor - fit for 2U height
-
-eXtremeRAID 1100 (DAC1164P)
- 3 Wide Ultra-2/LVD SCSI channels
- 233MHz StrongARM SA 110 Processor
- 64 Bit 33MHz PCI (backward compatible with 32 Bit PCI slots)
- 16MB/32MB/64MB Parity SDRAM Memory with Battery Backup
-
-AcceleRAID 250 (DAC960PTL1)
- Uses onboard Symbios SCSI chips on certain motherboards
- Also includes one onboard Wide Ultra-2/LVD SCSI Channel
- 66MHz Intel i960RD RISC Processor
- 4MB/8MB/16MB/32MB/64MB/128MB ECC EDO Memory
-
-AcceleRAID 200 (DAC960PTL0)
- Uses onboard Symbios SCSI chips on certain motherboards
- Includes no onboard SCSI Channels
- 66MHz Intel i960RD RISC Processor
- 4MB/8MB/16MB/32MB/64MB/128MB ECC EDO Memory
-
-AcceleRAID 150 (DAC960PRL)
- Uses onboard Symbios SCSI chips on certain motherboards
- Also includes one onboard Wide Ultra-2/LVD SCSI Channel
- 33MHz Intel i960RP RISC Processor
- 4MB Parity EDO Memory
-
-DAC960PJ 1/2/3 Wide Ultra SCSI-3 Channels
- 66MHz Intel i960RD RISC Processor
- 4MB/8MB/16MB/32MB/64MB/128MB ECC EDO Memory
-
-DAC960PG 1/2/3 Wide Ultra SCSI-3 Channels
- 33MHz Intel i960RP RISC Processor
- 4MB/8MB ECC EDO Memory
-
-DAC960PU 1/2/3 Wide Ultra SCSI-3 Channels
- Intel i960CF RISC Processor
- 4MB/8MB EDRAM or 2MB/4MB/8MB/16MB/32MB DRAM Memory
-
-DAC960PD 1/2/3 Wide Fast SCSI-2 Channels
- Intel i960CF RISC Processor
- 4MB/8MB EDRAM or 2MB/4MB/8MB/16MB/32MB DRAM Memory
-
-DAC960PL 1/2/3 Wide Fast SCSI-2 Channels
- Intel i960 RISC Processor
- 2MB/4MB/8MB/16MB/32MB DRAM Memory
-
-DAC960P 1/2/3 Wide Fast SCSI-2 Channels
- Intel i960 RISC Processor
- 2MB/4MB/8MB/16MB/32MB DRAM Memory
-
-For the eXtremeRAID 2000/3000 and AcceleRAID 352/170/160, firmware version
-6.00-01 or above is required.
-
-For the eXtremeRAID 1100, firmware version 5.06-0-52 or above is required.
-
-For the AcceleRAID 250, 200, and 150, firmware version 4.06-0-57 or above is
-required.
-
-For the DAC960PJ and DAC960PG, firmware version 4.06-0-00 or above is required.
-
-For the DAC960PU, DAC960PD, DAC960PL, and DAC960P, either firmware version
-3.51-0-04 or above is required (for dual Flash ROM controllers), or firmware
-version 2.73-0-00 or above is required (for single Flash ROM controllers)
-
-Please note that not all SCSI disk drives are suitable for use with DAC960
-controllers, and only particular firmware versions of any given model may
-actually function correctly. Similarly, not all motherboards have a BIOS that
-properly initializes the AcceleRAID 250, AcceleRAID 200, AcceleRAID 150,
-DAC960PJ, and DAC960PG because the Intel i960RD/RP is a multi-function device.
-If in doubt, contact Mylex RAID Technical Support (mylexsup@us.ibm.com) to
-verify compatibility. Mylex makes available a hard disk compatibility list at
-http://www.mylex.com/support/hdcomp/hd-lists.html.
-
-
- DRIVER INSTALLATION
-
-This distribution was prepared for Linux kernel version 2.2.19 or 2.4.12.
-
-To install the DAC960 RAID driver, you may use the following commands,
-replacing "/usr/src" with wherever you keep your Linux kernel source tree:
-
- cd /usr/src
- tar -xvzf DAC960-2.2.11.tar.gz (or DAC960-2.4.11.tar.gz)
- mv README.DAC960 linux/Documentation
- mv DAC960.[ch] linux/drivers/block
- patch -p0 < DAC960.patch (if DAC960.patch is included)
- cd linux
- make config
- make bzImage (or zImage)
-
-Then install "arch/x86/boot/bzImage" or "arch/x86/boot/zImage" as your
-standard kernel, run lilo if appropriate, and reboot.
-
-To create the necessary devices in /dev, the "make_rd" script included in
-"DAC960-Utilities.tar.gz" from http://www.dandelion.com/Linux/ may be used.
-LILO 21 and FDISK v2.9 include DAC960 support; also included in this archive
-are patches to LILO 20 and FDISK v2.8 that add DAC960 support, along with
-statically linked executables of LILO and FDISK. This modified version of LILO
-will allow booting from a DAC960 controller and/or mounting the root file
-system from a DAC960.
-
-Red Hat Linux 6.0 and SuSE Linux 6.1 include support for Mylex PCI RAID
-controllers. Installing directly onto a DAC960 may be problematic from other
-Linux distributions until their installation utilities are updated.
-
-
- INSTALLATION NOTES
-
-Before installing Linux or adding DAC960 logical drives to an existing Linux
-system, the controller must first be configured to provide one or more logical
-drives using the BIOS Configuration Utility or DACCF. Please note that since
-there are only at most 6 usable partitions on each logical drive, systems
-requiring more partitions should subdivide a drive group into multiple logical
-drives, each of which can have up to 6 usable partitions. Also, note that with
-large disk arrays it is advisable to enable the 8GB BIOS Geometry (255/63)
-rather than accepting the default 2GB BIOS Geometry (128/32); failing to so do
-will cause the logical drive geometry to have more than 65535 cylinders which
-will make it impossible for FDISK to be used properly. The 8GB BIOS Geometry
-can be enabled by configuring the DAC960 BIOS, which is accessible via Alt-M
-during the BIOS initialization sequence.
-
-For maximum performance and the most efficient E2FSCK performance, it is
-recommended that EXT2 file systems be built with a 4KB block size and 16 block
-stride to match the DAC960 controller's 64KB default stripe size. The command
-"mke2fs -b 4096 -R stride=16 <device>" is appropriate. Unless there will be a
-large number of small files on the file systems, it is also beneficial to add
-the "-i 16384" option to increase the bytes per inode parameter thereby
-reducing the file system metadata. Finally, on systems that will only be run
-with Linux 2.2 or later kernels it is beneficial to enable sparse superblocks
-with the "-s 1" option.
-
-
- DAC960 ANNOUNCEMENTS MAILING LIST
-
-The DAC960 Announcements Mailing List provides a forum for informing Linux
-users of new driver releases and other announcements regarding Linux support
-for DAC960 PCI RAID Controllers. To join the mailing list, send a message to
-"dac960-announce-request@dandelion.com" with the line "subscribe" in the
-message body.
-
-
- CONTROLLER CONFIGURATION AND STATUS MONITORING
-
-The DAC960 RAID controllers running firmware 4.06 or above include a Background
-Initialization facility so that system downtime is minimized both for initial
-installation and subsequent configuration of additional storage. The BIOS
-Configuration Utility (accessible via Alt-R during the BIOS initialization
-sequence) is used to quickly configure the controller, and then the logical
-drives that have been created are available for immediate use even while they
-are still being initialized by the controller. The primary need for online
-configuration and status monitoring is then to avoid system downtime when disk
-drives fail and must be replaced. Mylex's online monitoring and configuration
-utilities are being ported to Linux and will become available at some point in
-the future. Note that with a SAF-TE (SCSI Accessed Fault-Tolerant Enclosure)
-enclosure, the controller is able to rebuild failed drives automatically as
-soon as a drive replacement is made available.
-
-The primary interfaces for controller configuration and status monitoring are
-special files created in the /proc/rd/... hierarchy along with the normal
-system console logging mechanism. Whenever the system is operating, the DAC960
-driver queries each controller for status information every 10 seconds, and
-checks for additional conditions every 60 seconds. The initial status of each
-controller is always available for controller N in /proc/rd/cN/initial_status,
-and the current status as of the last status monitoring query is available in
-/proc/rd/cN/current_status. In addition, status changes are also logged by the
-driver to the system console and will appear in the log files maintained by
-syslog. The progress of asynchronous rebuild or consistency check operations
-is also available in /proc/rd/cN/current_status, and progress messages are
-logged to the system console at most every 60 seconds.
-
-Starting with the 2.2.3/2.0.3 versions of the driver, the status information
-available in /proc/rd/cN/initial_status and /proc/rd/cN/current_status has been
-augmented to include the vendor, model, revision, and serial number (if
-available) for each physical device found connected to the controller:
-
-***** DAC960 RAID Driver Version 2.2.3 of 19 August 1999 *****
-Copyright 1998-1999 by Leonard N. Zubkoff <lnz@dandelion.com>
-Configuring Mylex DAC960PRL PCI RAID Controller
- Firmware Version: 4.07-0-07, Channels: 1, Memory Size: 16MB
- PCI Bus: 1, Device: 4, Function: 1, I/O Address: Unassigned
- PCI Address: 0xFE300000 mapped at 0xA0800000, IRQ Channel: 21
- Controller Queue Depth: 128, Maximum Blocks per Command: 128
- Driver Queue Depth: 127, Maximum Scatter/Gather Segments: 33
- Stripe Size: 64KB, Segment Size: 8KB, BIOS Geometry: 255/63
- SAF-TE Enclosure Management Enabled
- Physical Devices:
- 0:0 Vendor: IBM Model: DRVS09D Revision: 0270
- Serial Number: 68016775HA
- Disk Status: Online, 17928192 blocks
- 0:1 Vendor: IBM Model: DRVS09D Revision: 0270
- Serial Number: 68004E53HA
- Disk Status: Online, 17928192 blocks
- 0:2 Vendor: IBM Model: DRVS09D Revision: 0270
- Serial Number: 13013935HA
- Disk Status: Online, 17928192 blocks
- 0:3 Vendor: IBM Model: DRVS09D Revision: 0270
- Serial Number: 13016897HA
- Disk Status: Online, 17928192 blocks
- 0:4 Vendor: IBM Model: DRVS09D Revision: 0270
- Serial Number: 68019905HA
- Disk Status: Online, 17928192 blocks
- 0:5 Vendor: IBM Model: DRVS09D Revision: 0270
- Serial Number: 68012753HA
- Disk Status: Online, 17928192 blocks
- 0:6 Vendor: ESG-SHV Model: SCA HSBP M6 Revision: 0.61
- Logical Drives:
- /dev/rd/c0d0: RAID-5, Online, 89640960 blocks, Write Thru
- No Rebuild or Consistency Check in Progress
-
-To simplify the monitoring process for custom software, the special file
-/proc/rd/status returns "OK" when all DAC960 controllers in the system are
-operating normally and no failures have occurred, or "ALERT" if any logical
-drives are offline or critical or any non-standby physical drives are dead.
-
-Configuration commands for controller N are available via the special file
-/proc/rd/cN/user_command. A human readable command can be written to this
-special file to initiate a configuration operation, and the results of the
-operation can then be read back from the special file in addition to being
-logged to the system console. The shell command sequence
-
- echo "<configuration-command>" > /proc/rd/c0/user_command
- cat /proc/rd/c0/user_command
-
-is typically used to execute configuration commands. The configuration
-commands are:
-
- flush-cache
-
- The "flush-cache" command flushes the controller's cache. The system
- automatically flushes the cache at shutdown or if the driver module is
- unloaded, so this command is only needed to be certain a write back cache
- is flushed to disk before the system is powered off by a command to a UPS.
- Note that the flush-cache command also stops an asynchronous rebuild or
- consistency check, so it should not be used except when the system is being
- halted.
-
- kill <channel>:<target-id>
-
- The "kill" command marks the physical drive <channel>:<target-id> as DEAD.
- This command is provided primarily for testing, and should not be used
- during normal system operation.
-
- make-online <channel>:<target-id>
-
- The "make-online" command changes the physical drive <channel>:<target-id>
- from status DEAD to status ONLINE. In cases where multiple physical drives
- have been killed simultaneously, this command may be used to bring all but
- one of them back online, after which a rebuild to the final drive is
- necessary.
-
- Warning: make-online should only be used on a dead physical drive that is
- an active part of a drive group, never on a standby drive. The command
- should never be used on a dead drive that is part of a critical logical
- drive; rebuild should be used if only a single drive is dead.
-
- make-standby <channel>:<target-id>
-
- The "make-standby" command changes physical drive <channel>:<target-id>
- from status DEAD to status STANDBY. It should only be used in cases where
- a dead drive was replaced after an automatic rebuild was performed onto a
- standby drive. It cannot be used to add a standby drive to the controller
- configuration if one was not created initially; the BIOS Configuration
- Utility must be used for that currently.
-
- rebuild <channel>:<target-id>
-
- The "rebuild" command initiates an asynchronous rebuild onto physical drive
- <channel>:<target-id>. It should only be used when a dead drive has been
- replaced.
-
- check-consistency <logical-drive-number>
-
- The "check-consistency" command initiates an asynchronous consistency check
- of <logical-drive-number> with automatic restoration. It can be used
- whenever it is desired to verify the consistency of the redundancy
- information.
-
- cancel-rebuild
- cancel-consistency-check
-
- The "cancel-rebuild" and "cancel-consistency-check" commands cancel any
- rebuild or consistency check operations previously initiated.
-
-
- EXAMPLE I - DRIVE FAILURE WITHOUT A STANDBY DRIVE
-
-The following annotated logs demonstrate the controller configuration and and
-online status monitoring capabilities of the Linux DAC960 Driver. The test
-configuration comprises 6 1GB Quantum Atlas I disk drives on two channels of a
-DAC960PJ controller. The physical drives are configured into a single drive
-group without a standby drive, and the drive group has been configured into two
-logical drives, one RAID-5 and one RAID-6. Note that these logs are from an
-earlier version of the driver and the messages have changed somewhat with newer
-releases, but the functionality remains similar. First, here is the current
-status of the RAID configuration:
-
-gwynedd:/u/lnz# cat /proc/rd/c0/current_status
-***** DAC960 RAID Driver Version 2.0.0 of 23 March 1999 *****
-Copyright 1998-1999 by Leonard N. Zubkoff <lnz@dandelion.com>
-Configuring Mylex DAC960PJ PCI RAID Controller
- Firmware Version: 4.06-0-08, Channels: 3, Memory Size: 8MB
- PCI Bus: 0, Device: 19, Function: 1, I/O Address: Unassigned
- PCI Address: 0xFD4FC000 mapped at 0x8807000, IRQ Channel: 9
- Controller Queue Depth: 128, Maximum Blocks per Command: 128
- Driver Queue Depth: 127, Maximum Scatter/Gather Segments: 33
- Stripe Size: 64KB, Segment Size: 8KB, BIOS Geometry: 255/63
- Physical Devices:
- 0:1 - Disk: Online, 2201600 blocks
- 0:2 - Disk: Online, 2201600 blocks
- 0:3 - Disk: Online, 2201600 blocks
- 1:1 - Disk: Online, 2201600 blocks
- 1:2 - Disk: Online, 2201600 blocks
- 1:3 - Disk: Online, 2201600 blocks
- Logical Drives:
- /dev/rd/c0d0: RAID-5, Online, 5498880 blocks, Write Thru
- /dev/rd/c0d1: RAID-6, Online, 3305472 blocks, Write Thru
- No Rebuild or Consistency Check in Progress
-
-gwynedd:/u/lnz# cat /proc/rd/status
-OK
-
-The above messages indicate that everything is healthy, and /proc/rd/status
-returns "OK" indicating that there are no problems with any DAC960 controller
-in the system. For demonstration purposes, while I/O is active Physical Drive
-1:1 is now disconnected, simulating a drive failure. The failure is noted by
-the driver within 10 seconds of the controller's having detected it, and the
-driver logs the following console status messages indicating that Logical
-Drives 0 and 1 are now CRITICAL as a result of Physical Drive 1:1 being DEAD:
-
-DAC960#0: Physical Drive 1:2 Error Log: Sense Key = 6, ASC = 29, ASCQ = 02
-DAC960#0: Physical Drive 1:3 Error Log: Sense Key = 6, ASC = 29, ASCQ = 02
-DAC960#0: Physical Drive 1:1 killed because of timeout on SCSI command
-DAC960#0: Physical Drive 1:1 is now DEAD
-DAC960#0: Logical Drive 0 (/dev/rd/c0d0) is now CRITICAL
-DAC960#0: Logical Drive 1 (/dev/rd/c0d1) is now CRITICAL
-
-The Sense Keys logged here are just Check Condition / Unit Attention conditions
-arising from a SCSI bus reset that is forced by the controller during its error
-recovery procedures. Concurrently with the above, the driver status available
-from /proc/rd also reflects the drive failure. The status message in
-/proc/rd/status has changed from "OK" to "ALERT":
-
-gwynedd:/u/lnz# cat /proc/rd/status
-ALERT
-
-and /proc/rd/c0/current_status has been updated:
-
-gwynedd:/u/lnz# cat /proc/rd/c0/current_status
- ...
- Physical Devices:
- 0:1 - Disk: Online, 2201600 blocks
- 0:2 - Disk: Online, 2201600 blocks
- 0:3 - Disk: Online, 2201600 blocks
- 1:1 - Disk: Dead, 2201600 blocks
- 1:2 - Disk: Online, 2201600 blocks
- 1:3 - Disk: Online, 2201600 blocks
- Logical Drives:
- /dev/rd/c0d0: RAID-5, Critical, 5498880 blocks, Write Thru
- /dev/rd/c0d1: RAID-6, Critical, 3305472 blocks, Write Thru
- No Rebuild or Consistency Check in Progress
-
-Since there are no standby drives configured, the system can continue to access
-the logical drives in a performance degraded mode until the failed drive is
-replaced and a rebuild operation completed to restore the redundancy of the
-logical drives. Once Physical Drive 1:1 is replaced with a properly
-functioning drive, or if the physical drive was killed without having failed
-(e.g., due to electrical problems on the SCSI bus), the user can instruct the
-controller to initiate a rebuild operation onto the newly replaced drive:
-
-gwynedd:/u/lnz# echo "rebuild 1:1" > /proc/rd/c0/user_command
-gwynedd:/u/lnz# cat /proc/rd/c0/user_command
-Rebuild of Physical Drive 1:1 Initiated
-
-The echo command instructs the controller to initiate an asynchronous rebuild
-operation onto Physical Drive 1:1, and the status message that results from the
-operation is then available for reading from /proc/rd/c0/user_command, as well
-as being logged to the console by the driver.
-
-Within 10 seconds of this command the driver logs the initiation of the
-asynchronous rebuild operation:
-
-DAC960#0: Rebuild of Physical Drive 1:1 Initiated
-DAC960#0: Physical Drive 1:1 Error Log: Sense Key = 6, ASC = 29, ASCQ = 01
-DAC960#0: Physical Drive 1:1 is now WRITE-ONLY
-DAC960#0: Rebuild in Progress: Logical Drive 0 (/dev/rd/c0d0) 1% completed
-
-and /proc/rd/c0/current_status is updated:
-
-gwynedd:/u/lnz# cat /proc/rd/c0/current_status
- ...
- Physical Devices:
- 0:1 - Disk: Online, 2201600 blocks
- 0:2 - Disk: Online, 2201600 blocks
- 0:3 - Disk: Online, 2201600 blocks
- 1:1 - Disk: Write-Only, 2201600 blocks
- 1:2 - Disk: Online, 2201600 blocks
- 1:3 - Disk: Online, 2201600 blocks
- Logical Drives:
- /dev/rd/c0d0: RAID-5, Critical, 5498880 blocks, Write Thru
- /dev/rd/c0d1: RAID-6, Critical, 3305472 blocks, Write Thru
- Rebuild in Progress: Logical Drive 0 (/dev/rd/c0d0) 6% completed
-
-As the rebuild progresses, the current status in /proc/rd/c0/current_status is
-updated every 10 seconds:
-
-gwynedd:/u/lnz# cat /proc/rd/c0/current_status
- ...
- Physical Devices:
- 0:1 - Disk: Online, 2201600 blocks
- 0:2 - Disk: Online, 2201600 blocks
- 0:3 - Disk: Online, 2201600 blocks
- 1:1 - Disk: Write-Only, 2201600 blocks
- 1:2 - Disk: Online, 2201600 blocks
- 1:3 - Disk: Online, 2201600 blocks
- Logical Drives:
- /dev/rd/c0d0: RAID-5, Critical, 5498880 blocks, Write Thru
- /dev/rd/c0d1: RAID-6, Critical, 3305472 blocks, Write Thru
- Rebuild in Progress: Logical Drive 0 (/dev/rd/c0d0) 15% completed
-
-and every minute a progress message is logged to the console by the driver:
-
-DAC960#0: Rebuild in Progress: Logical Drive 0 (/dev/rd/c0d0) 32% completed
-DAC960#0: Rebuild in Progress: Logical Drive 0 (/dev/rd/c0d0) 63% completed
-DAC960#0: Rebuild in Progress: Logical Drive 0 (/dev/rd/c0d0) 94% completed
-DAC960#0: Rebuild in Progress: Logical Drive 1 (/dev/rd/c0d1) 94% completed
-
-Finally, the rebuild completes successfully. The driver logs the status of the
-logical and physical drives and the rebuild completion:
-
-DAC960#0: Rebuild Completed Successfully
-DAC960#0: Physical Drive 1:1 is now ONLINE
-DAC960#0: Logical Drive 0 (/dev/rd/c0d0) is now ONLINE
-DAC960#0: Logical Drive 1 (/dev/rd/c0d1) is now ONLINE
-
-/proc/rd/c0/current_status is updated:
-
-gwynedd:/u/lnz# cat /proc/rd/c0/current_status
- ...
- Physical Devices:
- 0:1 - Disk: Online, 2201600 blocks
- 0:2 - Disk: Online, 2201600 blocks
- 0:3 - Disk: Online, 2201600 blocks
- 1:1 - Disk: Online, 2201600 blocks
- 1:2 - Disk: Online, 2201600 blocks
- 1:3 - Disk: Online, 2201600 blocks
- Logical Drives:
- /dev/rd/c0d0: RAID-5, Online, 5498880 blocks, Write Thru
- /dev/rd/c0d1: RAID-6, Online, 3305472 blocks, Write Thru
- Rebuild Completed Successfully
-
-and /proc/rd/status indicates that everything is healthy once again:
-
-gwynedd:/u/lnz# cat /proc/rd/status
-OK
-
-
- EXAMPLE II - DRIVE FAILURE WITH A STANDBY DRIVE
-
-The following annotated logs demonstrate the controller configuration and and
-online status monitoring capabilities of the Linux DAC960 Driver. The test
-configuration comprises 6 1GB Quantum Atlas I disk drives on two channels of a
-DAC960PJ controller. The physical drives are configured into a single drive
-group with a standby drive, and the drive group has been configured into two
-logical drives, one RAID-5 and one RAID-6. Note that these logs are from an
-earlier version of the driver and the messages have changed somewhat with newer
-releases, but the functionality remains similar. First, here is the current
-status of the RAID configuration:
-
-gwynedd:/u/lnz# cat /proc/rd/c0/current_status
-***** DAC960 RAID Driver Version 2.0.0 of 23 March 1999 *****
-Copyright 1998-1999 by Leonard N. Zubkoff <lnz@dandelion.com>
-Configuring Mylex DAC960PJ PCI RAID Controller
- Firmware Version: 4.06-0-08, Channels: 3, Memory Size: 8MB
- PCI Bus: 0, Device: 19, Function: 1, I/O Address: Unassigned
- PCI Address: 0xFD4FC000 mapped at 0x8807000, IRQ Channel: 9
- Controller Queue Depth: 128, Maximum Blocks per Command: 128
- Driver Queue Depth: 127, Maximum Scatter/Gather Segments: 33
- Stripe Size: 64KB, Segment Size: 8KB, BIOS Geometry: 255/63
- Physical Devices:
- 0:1 - Disk: Online, 2201600 blocks
- 0:2 - Disk: Online, 2201600 blocks
- 0:3 - Disk: Online, 2201600 blocks
- 1:1 - Disk: Online, 2201600 blocks
- 1:2 - Disk: Online, 2201600 blocks
- 1:3 - Disk: Standby, 2201600 blocks
- Logical Drives:
- /dev/rd/c0d0: RAID-5, Online, 4399104 blocks, Write Thru
- /dev/rd/c0d1: RAID-6, Online, 2754560 blocks, Write Thru
- No Rebuild or Consistency Check in Progress
-
-gwynedd:/u/lnz# cat /proc/rd/status
-OK
-
-The above messages indicate that everything is healthy, and /proc/rd/status
-returns "OK" indicating that there are no problems with any DAC960 controller
-in the system. For demonstration purposes, while I/O is active Physical Drive
-1:2 is now disconnected, simulating a drive failure. The failure is noted by
-the driver within 10 seconds of the controller's having detected it, and the
-driver logs the following console status messages:
-
-DAC960#0: Physical Drive 1:1 Error Log: Sense Key = 6, ASC = 29, ASCQ = 02
-DAC960#0: Physical Drive 1:3 Error Log: Sense Key = 6, ASC = 29, ASCQ = 02
-DAC960#0: Physical Drive 1:2 killed because of timeout on SCSI command
-DAC960#0: Physical Drive 1:2 is now DEAD
-DAC960#0: Physical Drive 1:2 killed because it was removed
-DAC960#0: Logical Drive 0 (/dev/rd/c0d0) is now CRITICAL
-DAC960#0: Logical Drive 1 (/dev/rd/c0d1) is now CRITICAL
-
-Since a standby drive is configured, the controller automatically begins
-rebuilding onto the standby drive:
-
-DAC960#0: Physical Drive 1:3 is now WRITE-ONLY
-DAC960#0: Rebuild in Progress: Logical Drive 0 (/dev/rd/c0d0) 4% completed
-
-Concurrently with the above, the driver status available from /proc/rd also
-reflects the drive failure and automatic rebuild. The status message in
-/proc/rd/status has changed from "OK" to "ALERT":
-
-gwynedd:/u/lnz# cat /proc/rd/status
-ALERT
-
-and /proc/rd/c0/current_status has been updated:
-
-gwynedd:/u/lnz# cat /proc/rd/c0/current_status
- ...
- Physical Devices:
- 0:1 - Disk: Online, 2201600 blocks
- 0:2 - Disk: Online, 2201600 blocks
- 0:3 - Disk: Online, 2201600 blocks
- 1:1 - Disk: Online, 2201600 blocks
- 1:2 - Disk: Dead, 2201600 blocks
- 1:3 - Disk: Write-Only, 2201600 blocks
- Logical Drives:
- /dev/rd/c0d0: RAID-5, Critical, 4399104 blocks, Write Thru
- /dev/rd/c0d1: RAID-6, Critical, 2754560 blocks, Write Thru
- Rebuild in Progress: Logical Drive 0 (/dev/rd/c0d0) 4% completed
-
-As the rebuild progresses, the current status in /proc/rd/c0/current_status is
-updated every 10 seconds:
-
-gwynedd:/u/lnz# cat /proc/rd/c0/current_status
- ...
- Physical Devices:
- 0:1 - Disk: Online, 2201600 blocks
- 0:2 - Disk: Online, 2201600 blocks
- 0:3 - Disk: Online, 2201600 blocks
- 1:1 - Disk: Online, 2201600 blocks
- 1:2 - Disk: Dead, 2201600 blocks
- 1:3 - Disk: Write-Only, 2201600 blocks
- Logical Drives:
- /dev/rd/c0d0: RAID-5, Critical, 4399104 blocks, Write Thru
- /dev/rd/c0d1: RAID-6, Critical, 2754560 blocks, Write Thru
- Rebuild in Progress: Logical Drive 0 (/dev/rd/c0d0) 40% completed
-
-and every minute a progress message is logged on the console by the driver:
-
-DAC960#0: Rebuild in Progress: Logical Drive 0 (/dev/rd/c0d0) 40% completed
-DAC960#0: Rebuild in Progress: Logical Drive 0 (/dev/rd/c0d0) 76% completed
-DAC960#0: Rebuild in Progress: Logical Drive 1 (/dev/rd/c0d1) 66% completed
-DAC960#0: Rebuild in Progress: Logical Drive 1 (/dev/rd/c0d1) 84% completed
-
-Finally, the rebuild completes successfully. The driver logs the status of the
-logical and physical drives and the rebuild completion:
-
-DAC960#0: Rebuild Completed Successfully
-DAC960#0: Physical Drive 1:3 is now ONLINE
-DAC960#0: Logical Drive 0 (/dev/rd/c0d0) is now ONLINE
-DAC960#0: Logical Drive 1 (/dev/rd/c0d1) is now ONLINE
-
-/proc/rd/c0/current_status is updated:
-
-***** DAC960 RAID Driver Version 2.0.0 of 23 March 1999 *****
-Copyright 1998-1999 by Leonard N. Zubkoff <lnz@dandelion.com>
-Configuring Mylex DAC960PJ PCI RAID Controller
- Firmware Version: 4.06-0-08, Channels: 3, Memory Size: 8MB
- PCI Bus: 0, Device: 19, Function: 1, I/O Address: Unassigned
- PCI Address: 0xFD4FC000 mapped at 0x8807000, IRQ Channel: 9
- Controller Queue Depth: 128, Maximum Blocks per Command: 128
- Driver Queue Depth: 127, Maximum Scatter/Gather Segments: 33
- Stripe Size: 64KB, Segment Size: 8KB, BIOS Geometry: 255/63
- Physical Devices:
- 0:1 - Disk: Online, 2201600 blocks
- 0:2 - Disk: Online, 2201600 blocks
- 0:3 - Disk: Online, 2201600 blocks
- 1:1 - Disk: Online, 2201600 blocks
- 1:2 - Disk: Dead, 2201600 blocks
- 1:3 - Disk: Online, 2201600 blocks
- Logical Drives:
- /dev/rd/c0d0: RAID-5, Online, 4399104 blocks, Write Thru
- /dev/rd/c0d1: RAID-6, Online, 2754560 blocks, Write Thru
- Rebuild Completed Successfully
-
-and /proc/rd/status indicates that everything is healthy once again:
-
-gwynedd:/u/lnz# cat /proc/rd/status
-OK
-
-Note that the absence of a viable standby drive does not create an "ALERT"
-status. Once dead Physical Drive 1:2 has been replaced, the controller must be
-told that this has occurred and that the newly replaced drive should become the
-new standby drive:
-
-gwynedd:/u/lnz# echo "make-standby 1:2" > /proc/rd/c0/user_command
-gwynedd:/u/lnz# cat /proc/rd/c0/user_command
-Make Standby of Physical Drive 1:2 Succeeded
-
-The echo command instructs the controller to make Physical Drive 1:2 into a
-standby drive, and the status message that results from the operation is then
-available for reading from /proc/rd/c0/user_command, as well as being logged to
-the console by the driver. Within 60 seconds of this command the driver logs:
-
-DAC960#0: Physical Drive 1:2 Error Log: Sense Key = 6, ASC = 29, ASCQ = 01
-DAC960#0: Physical Drive 1:2 is now STANDBY
-DAC960#0: Make Standby of Physical Drive 1:2 Succeeded
-
-and /proc/rd/c0/current_status is updated:
-
-gwynedd:/u/lnz# cat /proc/rd/c0/current_status
- ...
- Physical Devices:
- 0:1 - Disk: Online, 2201600 blocks
- 0:2 - Disk: Online, 2201600 blocks
- 0:3 - Disk: Online, 2201600 blocks
- 1:1 - Disk: Online, 2201600 blocks
- 1:2 - Disk: Standby, 2201600 blocks
- 1:3 - Disk: Online, 2201600 blocks
- Logical Drives:
- /dev/rd/c0d0: RAID-5, Online, 4399104 blocks, Write Thru
- /dev/rd/c0d1: RAID-6, Online, 2754560 blocks, Write Thru
- Rebuild Completed Successfully
notify_free Depending on device usage scenario it may account
a) the number of pages freed because of swap slot free
notifications or b) the number of pages freed because of
- REQ_DISCARD requests sent by bio. The former ones are
+ REQ_OP_DISCARD requests sent by bio. The former ones are
sent to a swap block device when a swap slot is freed,
which implies that this disk is being used as a swap disk.
The latter ones are sent by filesystem mounted with
-.. SPDX-License-Identifier: CC-BY-SA-4.0
+.. SPDX-License-Identifier: GPL-2.0+
=============
ID Allocation
Any REQ_FUA requests bypass this flushing mechanism and are logged as soon as
they complete as those requests will obviously bypass the device cache.
-Any REQ_DISCARD requests are treated like WRITE requests. Otherwise we would
+Any REQ_OP_DISCARD requests are treated like WRITE requests. Otherwise we would
have all the DISCARD requests, and then the WRITE requests and then the FLUSH
request. Consider the following example:
Required properties:
- compatible : compatible string, one of:
- "allwinner,sun4i-a10-ahci"
+ - "allwinner,sun8i-r40-ahci"
- "brcm,iproc-ahci"
- "hisilicon,hisi-ahci"
- "cavium,octeon-7130-ahci"
- clocks : a list of phandle + clock specifier pairs
- resets : a list of phandle + reset specifier pairs
- target-supply : regulator for SATA target power
+- phy-supply : regulator for PHY power
- phys : reference to the SATA PHY node
- phy-names : must be "sata-phy"
+- ahci-supply : regulator for AHCI controller
- ports-implemented : Mask that indicates which ports that the HBA supports
are available for software to use. Useful if PORTS_IMPL
is not programmed by the BIOS, which is true with
- #address-cells : number of cells to encode an address
- #size-cells : number of cells representing the size of an address
+For allwinner,sun8i-r40-ahci, the reset propertie must be present.
Sub-nodes required properties:
- reg : the port number
And at least one of the following properties:
- phys : reference to the SATA PHY node
-- target-supply : regulator for SATA target power
+- target-supply : regulator for SATA target power
Examples:
sata@ffe08000 {
"brcm,bcm7445-ahci"
"brcm,bcm-nsp-ahci"
"brcm,sata3-ahci"
+ "brcm,bcm63138-ahci"
- reg : register mappings for AHCI and SATA_TOP_CTRL
- reg-names : "ahci" and "top-ctrl"
- interrupts : interrupt mapping for SATA IRQ
Specifying GPIO information for devices
-============================================
+=======================================
1) gpios property
-----------------
-Nodes that makes use of GPIOs should specify them using one or more
-properties, each containing a 'gpio-list':
-
- gpio-list ::= <single-gpio> [gpio-list]
- single-gpio ::= <gpio-phandle> <gpio-specifier>
- gpio-phandle : phandle to gpio controller node
- gpio-specifier : Array of #gpio-cells specifying specific gpio
- (controller specific)
-
GPIO properties should be named "[<name>-]gpios", with <name> being the purpose
of this GPIO for the device. While a non-existent <name> is considered valid
for compatibility reasons (resolving to the "gpios" property), it is not allowed
and bit-banged data signals:
gpio1: gpio1 {
- gpio-controller
- #gpio-cells = <2>;
- };
- gpio2: gpio2 {
- gpio-controller
- #gpio-cells = <1>;
+ gpio-controller;
+ #gpio-cells = <2>;
};
[...]
- enable-gpios = <&gpio2 2>;
data-gpios = <&gpio1 12 0>,
<&gpio1 13 0>,
<&gpio1 14 0>,
<&gpio1 15 0>;
-Note that gpio-specifier length is controller dependent. In the
-above example, &gpio1 uses 2 cells to specify a gpio, while &gpio2
-only uses one.
+In the above example, &gpio1 uses 2 cells to specify a gpio. The first cell is
+a local offset to the GPIO line and the second cell represent consumer flags,
+such as if the consumer desire the line to be active low (inverted) or open
+drain. This is the recommended practice.
-gpio-specifier may encode: bank, pin position inside the bank,
-whether pin is open-drain and whether pin is logically inverted.
+The exact meaning of each specifier cell is controller specific, and must be
+documented in the device tree binding for the device, but it is strongly
+recommended to use the two-cell approach.
-Exact meaning of each specifier cell is controller specific, and must
-be documented in the device tree binding for the device.
-
-Most controllers are however specifying a generic flag bitfield
-in the last cell, so for these, use the macros defined in
+Most controllers are specifying a generic flag bitfield in the last cell, so
+for these, use the macros defined in
include/dt-bindings/gpio/gpio.h whenever possible:
Example of a node using GPIOs:
Some or all of the GPIOs provided by a GPIO controller may be routed to pins
on the package via a pin controller. This allows muxing those pins between
-GPIO and other functions.
+GPIO and other functions. It is a fairly common practice among silicon
+engineers.
+
+2.2) Ordinary (numerical) GPIO ranges
+-------------------------------------
It is useful to represent which GPIOs correspond to which pins on which pin
-controllers. The gpio-ranges property described below represents this, and
-contains information structures as follows:
-
- gpio-range-list ::= <single-gpio-range> [gpio-range-list]
- single-gpio-range ::= <numeric-gpio-range> | <named-gpio-range>
- numeric-gpio-range ::=
- <pinctrl-phandle> <gpio-base> <pinctrl-base> <count>
- named-gpio-range ::= <pinctrl-phandle> <gpio-base> '<0 0>'
- pinctrl-phandle : phandle to pin controller node
- gpio-base : Base GPIO ID in the GPIO controller
- pinctrl-base : Base pinctrl pin ID in the pin controller
- count : The number of GPIOs/pins in this range
-
-The "pin controller node" mentioned above must conform to the bindings
-described in ../pinctrl/pinctrl-bindings.txt.
-
-In case named gpio ranges are used (ranges with both <pinctrl-base> and
-<count> set to 0), the property gpio-ranges-group-names contains one string
-for every single-gpio-range in gpio-ranges:
- gpiorange-names-list ::= <gpiorange-name> [gpiorange-names-list]
- gpiorange-name : Name of the pingroup associated to the GPIO range in
- the respective pin controller.
-
-Elements of gpiorange-names-list corresponding to numeric ranges contain
-the empty string. Elements of gpiorange-names-list corresponding to named
-ranges contain the name of a pin group defined in the respective pin
-controller. The number of pins/GPIOs in the range is the number of pins in
-that pin group.
+controllers. The gpio-ranges property described below represents this with
+a discrete set of ranges mapping pins from the pin controller local number space
+to pins in the GPIO controller local number space.
-Previous versions of this binding required all pin controller nodes that
-were referenced by any gpio-ranges property to contain a property named
-#gpio-range-cells with value <3>. This requirement is now deprecated.
-However, that property may still exist in older device trees for
-compatibility reasons, and would still be required even in new device
-trees that need to be compatible with older software.
+The format is: <[pin controller phandle], [GPIO controller offset],
+ [pin controller offset], [number of pins]>;
+
+The GPIO controller offset pertains to the GPIO controller node containing the
+range definition.
+
+The pin controller node referenced by the phandle must conform to the bindings
+described in pinctrl/pinctrl-bindings.txt.
+
+Each offset runs from 0 to N. It is perfectly fine to pile any number of
+ranges with just one pin-to-GPIO line mapping if the ranges are concocted, but
+in practice these ranges are often lumped in discrete sets.
+
+Example:
+
+ gpio-ranges = <&foo 0 20 10>, <&bar 10 50 20>;
-Example 1:
+This means:
+- pins 20..29 on pin controller "foo" is mapped to GPIO line 0..9 and
+- pins 50..69 on pin controller "bar" is mapped to GPIO line 10..29
+
+
+Verbose example:
qe_pio_e: gpio-controller@1460 {
#gpio-cells = <2>;
pinctrl1's pins 20..29, and GPIOs 10..29 routed to pin controller pinctrl2's
pins 50..69.
-Example 2:
+
+2.3) GPIO ranges from named pin groups
+--------------------------------------
+
+It is also possible to use pin groups for gpio ranges when pin groups are the
+easiest and most convenient mapping.
+
+Both both <pinctrl-base> and <count> must set to 0 when using named pin groups
+names.
+
+The property gpio-ranges-group-names must contain exactly one string for each
+range.
+
+Elements of gpio-ranges-group-names must contain the name of a pin group
+defined in the respective pin controller. The number of pins/GPIO lines in the
+range is the number of pins in that pin group. The number of pins of that
+group is defined int the implementation and not in the device tree.
+
+If numerical and named pin groups are mixed, the string corresponding to a
+numerical pin range in gpio-ranges-group-names must be empty.
+
+Example:
gpio_pio_i: gpio-controller@14b0 {
#gpio-cells = <2>;
"bar";
};
-Here, three GPIO ranges are defined wrt. two pin controllers. pinctrl1 GPIO
-ranges are defined using pin numbers whereas the GPIO ranges wrt. pinctrl2
-are named "foo" and "bar".
+Here, three GPIO ranges are defined referring to two pin controllers.
+
+pinctrl1 GPIO ranges are defined using pin numbers whereas the GPIO ranges
+in pinctrl2 are defined using the pin groups named "foo" and "bar".
+
+Previous versions of this binding required all pin controller nodes that
+were referenced by any gpio-ranges property to contain a property named
+#gpio-range-cells with value <3>. This requirement is now deprecated.
+However, that property may still exist in older device trees for
+compatibility reasons, and would still be required even in new device
+trees that need to be compatible with older software.
+++ /dev/null
-Ingenic jz47xx GPIO controller
-
-That the Ingenic GPIO driver node must be a sub-node of the Ingenic pinctrl
-driver node.
-
-Required properties:
---------------------
-
- - compatible: Must contain one of:
- - "ingenic,jz4740-gpio"
- - "ingenic,jz4770-gpio"
- - "ingenic,jz4780-gpio"
- - reg: The GPIO bank number.
- - interrupt-controller: Marks the device node as an interrupt controller.
- - interrupts: Interrupt specifier for the controllers interrupt.
- - #interrupt-cells: Should be 2. Refer to
- ../interrupt-controller/interrupts.txt for more details.
- - gpio-controller: Marks the device node as a GPIO controller.
- - #gpio-cells: Should be 2. The first cell is the GPIO number and the second
- cell specifies GPIO flags, as defined in <dt-bindings/gpio/gpio.h>. Only the
- GPIO_ACTIVE_HIGH and GPIO_ACTIVE_LOW flags are supported.
- - gpio-ranges: Range of pins managed by the GPIO controller. Refer to
- 'gpio.txt' in this directory for more details.
-
-Example:
---------
-
-&pinctrl {
- #address-cells = <1>;
- #size-cells = <0>;
-
- gpa: gpio@0 {
- compatible = "ingenic,jz4740-gpio";
- reg = <0>;
-
- gpio-controller;
- gpio-ranges = <&pinctrl 0 0 32>;
- #gpio-cells = <2>;
-
- interrupt-controller;
- #interrupt-cells = <2>;
-
- interrupt-parent = <&intc>;
- interrupts = <28>;
- };
-};
- compatible: should contain one or more of the following:
- "renesas,gpio-r8a7743": for R8A7743 (RZ/G1M) compatible GPIO controller.
+ - "renesas,gpio-r8a7744": for R8A7744 (RZ/G1N) compatible GPIO controller.
- "renesas,gpio-r8a7745": for R8A7745 (RZ/G1E) compatible GPIO controller.
- "renesas,gpio-r8a77470": for R8A77470 (RZ/G1C) compatible GPIO controller.
+ - "renesas,gpio-r8a774a1": for R8A774A1 (RZ/G2M) compatible GPIO controller.
- "renesas,gpio-r8a7778": for R8A7778 (R-Car M1) compatible GPIO controller.
- "renesas,gpio-r8a7779": for R8A7779 (R-Car H1) compatible GPIO controller.
- "renesas,gpio-r8a7790": for R8A7790 (R-Car H2) compatible GPIO controller.
- "renesas,gpio-r8a77995": for R8A77995 (R-Car D3) compatible GPIO controller.
- "renesas,rcar-gen1-gpio": for a generic R-Car Gen1 GPIO controller.
- "renesas,rcar-gen2-gpio": for a generic R-Car Gen2 or RZ/G1 GPIO controller.
- - "renesas,rcar-gen3-gpio": for a generic R-Car Gen3 GPIO controller.
+ - "renesas,rcar-gen3-gpio": for a generic R-Car Gen3 or RZ/G2 GPIO controller.
- "renesas,gpio-rcar": deprecated.
When compatible with the generic version nodes must list the
- #gpio-cells: Should be 2. The first cell is the GPIO number and the second
cell specifies GPIO flags, as defined in <dt-bindings/gpio/gpio.h>. Only the
GPIO_ACTIVE_HIGH and GPIO_ACTIVE_LOW flags are supported.
- - gpio-ranges: Range of pins managed by the GPIO controller.
+ - gpio-ranges: See gpio.txt.
Optional properties:
mandatory if the hardware implements a controllable functional clock for
the GPIO instance.
-Please refer to gpio.txt in this directory for details of gpio-ranges property
-and the common GPIO bindings used by client devices.
+ - gpio-reserved-ranges: See gpio.txt.
+
+Please refer to gpio.txt in this directory for the common GPIO bindings used by
+client devices.
The GPIO controller also acts as an interrupt controller. It uses the default
two cells specifier as described in Documentation/devicetree/bindings/
interrupt-controller/interrupts.txt.
-Example: R8A7779 (R-Car H1) GPIO controller nodes
+Example: R8A77470 (RZ/G1C) GPIO controller nodes
- gpio0: gpio@ffc40000 {
- compatible = "renesas,gpio-r8a7779", "renesas,rcar-gen1-gpio";
- reg = <0xffc40000 0x2c>;
- interrupt-parent = <&gic>;
- interrupts = <0 141 0x4>;
- #gpio-cells = <2>;
- gpio-controller;
- gpio-ranges = <&pfc 0 0 32>;
- interrupt-controller;
- #interrupt-cells = <2>;
- };
+ gpio0: gpio@e6050000 {
+ compatible = "renesas,gpio-r8a77470",
+ "renesas,rcar-gen2-gpio";
+ reg = <0 0xe6050000 0 0x50>;
+ interrupts = <GIC_SPI 4 IRQ_TYPE_LEVEL_HIGH>;
+ #gpio-cells = <2>;
+ gpio-controller;
+ gpio-ranges = <&pfc 0 0 23>;
+ #interrupt-cells = <2>;
+ interrupt-controller;
+ clocks = <&cpg CPG_MOD 912>;
+ power-domains = <&sysc R8A77470_PD_ALWAYS_ON>;
+ resets = <&cpg 912>;
+ };
...
- gpio6: gpio@ffc46000 {
- compatible = "renesas,gpio-r8a7779", "renesas,rcar-gen1-gpio";
- reg = <0xffc46000 0x2c>;
- interrupt-parent = <&gic>;
- interrupts = <0 147 0x4>;
- #gpio-cells = <2>;
- gpio-controller;
- gpio-ranges = <&pfc 0 192 9>;
- interrupt-controller;
- #interrupt-cells = <2>;
- };
+ gpio3: gpio@e6053000 {
+ compatible = "renesas,gpio-r8a77470",
+ "renesas,rcar-gen2-gpio";
+ reg = <0 0xe6053000 0 0x50>;
+ interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>;
+ #gpio-cells = <2>;
+ gpio-controller;
+ gpio-ranges = <&pfc 0 96 30>;
+ gpio-reserved-ranges = <17 10>;
+ #interrupt-cells = <2>;
+ interrupt-controller;
+ clocks = <&cpg CPG_MOD 909>;
+ power-domains = <&sysc R8A77470_PD_ALWAYS_ON>;
+ resets = <&cpg 909>;
+ };
--- /dev/null
+Synopsys GPIO via CREG (Control REGisters) driver
+
+Required properties:
+- compatible : "snps,creg-gpio-hsdk" or "snps,creg-gpio-axs10x".
+- reg : Exactly one register range with length 0x4.
+- #gpio-cells : Since the generic GPIO binding is used, the
+ amount of cells must be specified as 2. The first cell is the
+ pin number, the second cell is used to specify optional parameters:
+ See "gpio-specifier" in .../devicetree/bindings/gpio/gpio.txt.
+- gpio-controller : Marks the device node as a GPIO controller.
+- ngpios: Number of GPIO pins.
+
+Example:
+
+gpio: gpio@f00014b0 {
+ compatible = "snps,creg-gpio-hsdk";
+ reg = <0xf00014b0 0x4>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ ngpios = <2>;
+};
--- /dev/null
+Texas Instruments INA3221 Device Tree Bindings
+
+1) ina3221 node
+ Required properties:
+ - compatible: Must be "ti,ina3221"
+ - reg: I2C address
+
+ Optional properties:
+ = The node contains optional child nodes for three channels =
+ = Each child node describes the information of input source =
+
+ - #address-cells: Required only if a child node is present. Must be 1.
+ - #size-cells: Required only if a child node is present. Must be 0.
+
+2) child nodes
+ Required properties:
+ - reg: Must be 0, 1 or 2, corresponding to IN1, IN2 or IN3 port of INA3221
+
+ Optional properties:
+ - label: Name of the input source
+ - shunt-resistor-micro-ohms: Shunt resistor value in micro-Ohm
+
+Example:
+
+ina3221@40 {
+ compatible = "ti,ina3221";
+ reg = <0x40>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ input@0 {
+ reg = <0x0>;
+ status = "disabled";
+ };
+ input@1 {
+ reg = <0x1>;
+ shunt-resistor-micro-ohms = <5000>;
+ };
+ input@2 {
+ reg = <0x2>;
+ label = "VDD_5V";
+ shunt-resistor-micro-ohms = <5000>;
+ };
+};
* "lltc,ltm2987"
* "lltc,ltm4675"
* "lltc,ltm4676"
+ * "lltc,ltm4686"
- reg: I2C slave address
Optional properties:
* ltc3880, ltc3882, ltc3886 : vout0 - vout1
* ltc3883 : vout0
* ltm4676 : vout0 - vout1
+ * ltm4686 : vout0 - vout1
Example:
ltc2978@5e {
--- /dev/null
+* Panasonic AN30259A 3-channel LED driver
+
+The AN30259A is a LED controller capable of driving three LEDs independently. It supports
+constant current output and sloping current output modes. The chip is connected over I2C.
+
+Required properties:
+ - compatible: Must be "panasonic,an30259a".
+ - reg: I2C slave address.
+ - #address-cells: Must be 1.
+ - #size-cells: Must be 0.
+
+Each LED is represented as a sub-node of the panasonic,an30259a node.
+
+Required sub-node properties:
+ - reg: Pin that the LED is connected to. Must be 1, 2, or 3.
+
+Optional sub-node properties:
+ - label: see Documentation/devicetree/bindings/leds/common.txt
+ - linux,default-trigger: see Documentation/devicetree/bindings/leds/common.txt
+
+Example:
+led-controller@30 {
+ compatible = "panasonic,an30259a";
+ reg = <0x30>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ led@1 {
+ reg = <1>;
+ linux,default-trigger = "heartbeat";
+ label = "red:indicator";
+ };
+
+ led@2 {
+ reg = <2>;
+ label = "green:indicator";
+ };
+
+ led@3 {
+ reg = <3>;
+ label = "blue:indicator";
+ };
+};
-* ROHM BD71837 Power Management Integrated Circuit bindings
+* ROHM BD71837 and BD71847 Power Management Integrated Circuit bindings
-BD71837MWV is a programmable Power Management IC for powering single-core,
-dual-core, and quad-core SoCs such as NXP-i.MX 8M. It is optimized for
-low BOM cost and compact solution footprint. It integrates 8 Buck
-egulators and 7 LDOs to provide all the power rails required by the SoC and
-the commonly used peripherals.
+BD71837MWV and BD71847MWV are programmable Power Management ICs for powering
+single-core, dual-core, and quad-core SoCs such as NXP-i.MX 8M. They are
+optimized for low BOM cost and compact solution footprint. BD71837MWV
+integrates 8 Buck regulators and 7 LDOs. BD71847MWV contains 6 Buck regulators
+and 6 LDOs.
-Datasheet for PMIC is available at:
+Datasheet for BD71837 is available at:
https://www.rohm.com/datasheet/BD71837MWV/bd71837mwv-e
Required properties:
- - compatible : Should be "rohm,bd71837".
+ - compatible : Should be "rohm,bd71837" for bd71837
+ "rohm,bd71847" for bd71847.
- reg : I2C slave address.
- interrupt-parent : Phandle to the parent interrupt controller.
- interrupts : The interrupt line the device is connected to.
- "arasan,sdhci-5.1": generic Arasan SDHCI 5.1 PHY
- "rockchip,rk3399-sdhci-5.1", "arasan,sdhci-5.1": rk3399 eMMC PHY
For this device it is strongly suggested to include arasan,soc-ctl-syscon.
+ - "ti,am654-sdhci-5.1", "arasan,sdhci-5.1": TI AM654 MMC PHY
- reg: From mmc bindings: Register location and length.
- clocks: From clock bindings: Handles to clock inputs.
- clock-names: From clock bindings: Tuple including "clk_xin" and "clk_ahb"
Required properties:
- compatible: Should be one of the following:
- "ingenic,jz4740-mmc" for the JZ4740
+ - "ingenic,jz4725b-mmc" for the JZ4725B
- "ingenic,jz4780-mmc" for the JZ4780
- reg: Should contain the MMC controller registers location and length.
- interrupts: Should contain the interrupt specifier of the MMC controller.
Optional properties:
- arm,primecell-periphid : contains the PrimeCell Peripheral ID, it overrides
the ID provided by the HW
+- resets : phandle to internal reset line.
+ Should be defined for sdmmc variant.
- vqmmc-supply : phandle to the regulator device tree node, mentioned
as the VCCQ/VDD_IO supply in the eMMC/SD specs.
+specific for ux500 variant:
- st,sig-dir-dat0 : bus signal direction pin used for DAT[0].
- st,sig-dir-dat2 : bus signal direction pin used for DAT[2].
- st,sig-dir-dat31 : bus signal direction pin used for DAT[3] and DAT[1].
- st,sig-dir-cmd : cmd signal direction pin used for CMD.
- st,sig-pin-fbclk : feedback clock signal pin used.
+specific for sdmmc variant:
+- st,sig-dir : signal direction polarity used for cmd, dat0 dat123.
+- st,neg-edge : data & command phase relation, generated on
+ sd clock falling edge.
+- st,use-ckin : use ckin pin from an external driver to sample
+ the receive data (example: with voltage
+ switch transceiver).
+
Deprecated properties:
- mmc-cap-mmc-highspeed : indicates whether MMC is high speed capable.
- mmc-cap-sd-highspeed : indicates whether SD is high speed capable.
- compatible: value should be either of the following.
"mediatek,mt8135-mmc": for mmc host ip compatible with mt8135
"mediatek,mt8173-mmc": for mmc host ip compatible with mt8173
+ "mediatek,mt8183-mmc": for mmc host ip compatible with mt8183
"mediatek,mt2701-mmc": for mmc host ip compatible with mt2701
"mediatek,mt2712-mmc": for mmc host ip compatible with mt2712
"mediatek,mt7622-mmc": for MT7622 SoC
"source" - source clock (required)
"hclk" - HCLK which used for host (required)
"source_cg" - independent source clock gate (required for MT2712)
+ "bus_clk" - bus clock used for internal register access (required for MT2712 MSDC0/3)
- pinctrl-names: should be "default", "state_uhs"
- pinctrl-0: should contain default/high speed pin ctrl
- pinctrl-1: should contain uhs mode pin ctrl
power-gpios = <&gpio 155 0>; /* gpio PT3 */
bus-width = <8>;
};
+
+Optional properties for Tegra210 and Tegra186:
+- pinctrl-names, pinctrl-0, pinctrl-1 : Specify pad voltage
+ configurations. Valid pinctrl-names are "sdmmc-3v3" and "sdmmc-1v8"
+ for controllers supporting multiple voltage levels. The order of names
+ should correspond to the pin configuration states in pinctrl-0 and
+ pinctrl-1.
+- nvidia,only-1-8-v : The presence of this property indicates that the
+ controller operates at a 1.8 V fixed I/O voltage.
+- nvidia,pad-autocal-pull-up-offset-3v3,
+ nvidia,pad-autocal-pull-down-offset-3v3 : Specify drive strength
+ calibration offsets for 3.3 V signaling modes.
+- nvidia,pad-autocal-pull-up-offset-1v8,
+ nvidia,pad-autocal-pull-down-offset-1v8 : Specify drive strength
+ calibration offsets for 1.8 V signaling modes.
+- nvidia,pad-autocal-pull-up-offset-3v3-timeout,
+ nvidia,pad-autocal-pull-down-offset-3v3-timeout : Specify drive
+ strength used as a fallback in case the automatic calibration times
+ out on a 3.3 V signaling mode.
+- nvidia,pad-autocal-pull-up-offset-1v8-timeout,
+ nvidia,pad-autocal-pull-down-offset-1v8-timeout : Specify drive
+ strength used as a fallback in case the automatic calibration times
+ out on a 1.8 V signaling mode.
+- nvidia,pad-autocal-pull-up-offset-sdr104,
+ nvidia,pad-autocal-pull-down-offset-sdr104 : Specify drive strength
+ calibration offsets for SDR104 mode.
+- nvidia,pad-autocal-pull-up-offset-hs400,
+ nvidia,pad-autocal-pull-down-offset-hs400 : Specify drive strength
+ calibration offsets for HS400 mode.
+- nvidia,default-tap : Specify the default inbound sampling clock
+ trimmer value for non-tunable modes.
+- nvidia,default-trim : Specify the default outbound clock trimmer
+ value.
+- nvidia,dqs-trim : Specify DQS trim value for HS400 timing
+
+ Notes on the pad calibration pull up and pulldown offset values:
+ - The property values are drive codes which are programmed into the
+ PD_OFFSET and PU_OFFSET sections of the
+ SDHCI_TEGRA_AUTO_CAL_CONFIG register.
+ - A higher value corresponds to higher drive strength. Please refer
+ to the reference manual of the SoC for correct values.
+ - The SDR104 and HS400 timing specific values are used in
+ corresponding modes if specified.
+
+ Notes on tap and trim values:
+ - The values are used for compensating trace length differences
+ by adjusting the sampling point.
+ - The values are programmed to the Vendor Clock Control Register.
+ Please refer to the reference manual of the SoC for correct
+ values.
+ - The DQS trim values are only used on controllers which support
+ HS400 timing. Only SDMMC4 on Tegra210 and Tegra 186 supports
+ HS400.
+
+Example:
+sdhci@700b0000 {
+ compatible = "nvidia,tegra210-sdhci", "nvidia,tegra124-sdhci";
+ reg = <0x0 0x700b0000 0x0 0x200>;
+ interrupts = <GIC_SPI 14 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&tegra_car TEGRA210_CLK_SDMMC1>;
+ clock-names = "sdhci";
+ resets = <&tegra_car 14>;
+ reset-names = "sdhci";
+ pinctrl-names = "sdmmc-3v3", "sdmmc-1v8";
+ pinctrl-0 = <&sdmmc1_3v3>;
+ pinctrl-1 = <&sdmmc1_1v8>;
+ nvidia,pad-autocal-pull-up-offset-3v3 = <0x00>;
+ nvidia,pad-autocal-pull-down-offset-3v3 = <0x7d>;
+ nvidia,pad-autocal-pull-up-offset-1v8 = <0x7b>;
+ nvidia,pad-autocal-pull-down-offset-1v8 = <0x7b>;
+ status = "disabled";
+};
- "renesas,mmcif-r8a73a4" for the MMCIF found in r8a73a4 SoCs
- "renesas,mmcif-r8a7740" for the MMCIF found in r8a7740 SoCs
- "renesas,mmcif-r8a7743" for the MMCIF found in r8a7743 SoCs
+ - "renesas,mmcif-r8a7744" for the MMCIF found in r8a7744 SoCs
- "renesas,mmcif-r8a7745" for the MMCIF found in r8a7745 SoCs
- "renesas,mmcif-r8a7778" for the MMCIF found in r8a7778 SoCs
- "renesas,mmcif-r8a7790" for the MMCIF found in r8a7790 SoCs
- interrupts: Some SoCs have only 1 shared interrupt, while others have either
2 or 3 individual interrupts (error, int, card detect). Below is the number
of interrupts for each SoC:
- 1: r8a73a4, r8a7743, r8a7745, r8a7778, r8a7790, r8a7791, r8a7793, r8a7794
+ 1: r8a73a4, r8a7743, r8a7744, r8a7745, r8a7778, r8a7790, r8a7791, r8a7793,
+ r8a7794
2: r8a7740, sh73a0
3: r7s72100
--- /dev/null
+* Spreadtrum SDHCI controller (sdhci-sprd)
+
+The Secure Digital (SD) Host controller on Spreadtrum SoCs provides an interface
+for MMC, SD and SDIO types of cards.
+
+This file documents differences between the core properties in mmc.txt
+and the properties used by the sdhci-sprd driver.
+
+Required properties:
+- compatible: Should contain "sprd,sdhci-r11".
+- reg: physical base address of the controller and length.
+- interrupts: Interrupts used by the SDHCI controller.
+- clocks: Should contain phandle for the clock feeding the SDHCI controller
+- clock-names: Should contain the following:
+ "sdio" - SDIO source clock (required)
+ "enable" - gate clock which used for enabling/disabling the device (required)
+
+Optional properties:
+- assigned-clocks: the same with "sdio" clock
+- assigned-clock-parents: the default parent of "sdio" clock
+
+Examples:
+
+sdio0: sdio@20600000 {
+ compatible = "sprd,sdhci-r11";
+ reg = <0 0x20600000 0 0x1000>;
+ interrupts = <GIC_SPI 60 IRQ_TYPE_LEVEL_HIGH>;
+
+ clock-names = "sdio", "enable";
+ clocks = <&ap_clk CLK_EMMC_2X>,
+ <&apahb_gate CLK_EMMC_EB>;
+ assigned-clocks = <&ap_clk CLK_EMMC_2X>;
+ assigned-clock-parents = <&rpll CLK_RPLL_390M>;
+
+ bus-width = <8>;
+ non-removable;
+ no-sdio;
+ no-sd;
+ cap-mmc-hw-reset;
+ status = "okay";
+};
"renesas,sdhi-r8a73a4" - SDHI IP on R8A73A4 SoC
"renesas,sdhi-r8a7740" - SDHI IP on R8A7740 SoC
"renesas,sdhi-r8a7743" - SDHI IP on R8A7743 SoC
+ "renesas,sdhi-r8a7744" - SDHI IP on R8A7744 SoC
"renesas,sdhi-r8a7745" - SDHI IP on R8A7745 SoC
+ "renesas,sdhi-r8a774a1" - SDHI IP on R8A774A1 SoC
+ "renesas,sdhi-r8a77470" - SDHI IP on R8A77470 SoC
+ "renesas,sdhi-mmc-r8a77470" - SDHI/MMC IP on R8A77470 SoC
"renesas,sdhi-r8a7778" - SDHI IP on R8A7778 SoC
"renesas,sdhi-r8a7779" - SDHI IP on R8A7779 SoC
"renesas,sdhi-r8a7790" - SDHI IP on R8A7790 SoC
"renesas,sdhi-r8a7795" - SDHI IP on R8A7795 SoC
"renesas,sdhi-r8a7796" - SDHI IP on R8A7796 SoC
"renesas,sdhi-r8a77965" - SDHI IP on R8A77965 SoC
+ "renesas,sdhi-r8a77970" - SDHI IP on R8A77970 SoC
"renesas,sdhi-r8a77980" - SDHI IP on R8A77980 SoC
"renesas,sdhi-r8a77990" - SDHI IP on R8A77990 SoC
"renesas,sdhi-r8a77995" - SDHI IP on R8A77995 SoC
"renesas,sdhi-shmobile" - a generic sh-mobile SDHI controller
"renesas,rcar-gen1-sdhi" - a generic R-Car Gen1 SDHI controller
- "renesas,rcar-gen2-sdhi" - a generic R-Car Gen2 or RZ/G1
+ "renesas,rcar-gen2-sdhi" - a generic R-Car Gen2 and RZ/G1 SDHI
+ (not SDHI/MMC) controller
+ "renesas,rcar-gen3-sdhi" - a generic R-Car Gen3 or RZ/G2
SDHI controller
- "renesas,rcar-gen3-sdhi" - a generic R-Car Gen3 SDHI controller
When compatible with the generic version, nodes must list
--- /dev/null
+UniPhier SD/eMMC controller
+
+Required properties:
+- compatible: should be one of the following:
+ "socionext,uniphier-sd-v2.91" - IP version 2.91
+ "socionext,uniphier-sd-v3.1" - IP version 3.1
+ "socionext,uniphier-sd-v3.1.1" - IP version 3.1.1
+- reg: offset and length of the register set for the device.
+- interrupts: a single interrupt specifier.
+- clocks: a single clock specifier of the controller clock.
+- reset-names: should contain the following:
+ "host" - mandatory for all versions
+ "bridge" - should exist only for "socionext,uniphier-sd-v2.91"
+ "hw" - should exist if eMMC hw reset line is available
+- resets: a list of reset specifiers, corresponding to the reset-names
+
+Optional properties:
+- pinctrl-names: if present, should contain the following:
+ "default" - should exist for all instances
+ "uhs" - should exist for SD instance with UHS support
+- pinctrl-0: pin control state for the default mode
+- pinctrl-1: pin control state for the UHS mode
+- dma-names: should be "rx-tx" if present.
+ This property can exist only for "socionext,uniphier-sd-v2.91".
+- dmas: a single DMA channel specifier
+ This property can exist only for "socionext,uniphier-sd-v2.91".
+- bus-width: see mmc.txt
+- cap-sd-highspeed: see mmc.txt
+- cap-mmc-highspeed: see mmc.txt
+- sd-uhs-sdr12: see mmc.txt
+- sd-uhs-sdr25: see mmc.txt
+- sd-uhs-sdr50: see mmc.txt
+- cap-mmc-hw-reset: should exist if reset-names contains "hw". see mmc.txt
+- non-removable: see mmc.txt
+
+Example:
+
+ sd: sdhc@5a400000 {
+ compatible = "socionext,uniphier-sd-v2.91";
+ reg = <0x5a400000 0x200>;
+ interrupts = <0 76 4>;
+ pinctrl-names = "default", "uhs";
+ pinctrl-0 = <&pinctrl_sd>;
+ pinctrl-1 = <&pinctrl_sd_uhs>;
+ clocks = <&mio_clk 0>;
+ reset-names = "host", "bridge";
+ resets = <&mio_rst 0>, <&mio_rst 3>;
+ dma-names = "rx-tx";
+ dmas = <&dmac 4>;
+ bus-width = <4>;
+ cap-sd-highspeed;
+ sd-uhs-sdr12;
+ sd-uhs-sdr25;
+ sd-uhs-sdr50;
+ };
--- /dev/null
+Broadcom Northstar pins mux controller
+
+Some of Northstar SoCs's pins can be used for various purposes thanks to the mux
+controller. This binding allows describing mux controller and listing available
+functions. They can be referenced later by other bindings to let system
+configure controller correctly.
+
+A list of pins varies across chipsets so few bindings are available.
+
+Required properties:
+- compatible: must be one of:
+ "brcm,bcm4708-pinmux"
+ "brcm,bcm4709-pinmux"
+ "brcm,bcm53012-pinmux"
+- reg: iomem address range of CRU (Central Resource Unit) pin registers
+- reg-names: "cru_gpio_control" - the only needed & supported reg right now
+
+Functions and their groups available for all chipsets:
+- "spi": "spi_grp"
+- "i2c": "i2c_grp"
+- "pwm": "pwm0_grp", "pwm1_grp", "pwm2_grp", "pwm3_grp"
+- "uart1": "uart1_grp"
+
+Additionally available on BCM4709 and BCM53012:
+- "mdio": "mdio_grp"
+- "uart2": "uart2_grp"
+- "sdio": "sdio_pwr_grp", "sdio_1p8v_grp"
+
+For documentation of subnodes see:
+Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt
+
+Example:
+ dmu@1800c000 {
+ compatible = "simple-bus";
+ ranges = <0 0x1800c000 0x1000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ cru@100 {
+ compatible = "simple-bus";
+ reg = <0x100 0x1a4>;
+ ranges;
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ pin-controller@1c0 {
+ compatible = "brcm,bcm4708-pinmux";
+ reg = <0x1c0 0x24>;
+ reg-names = "cru_gpio_control";
+
+ spi-pins {
+ function = "spi";
+ groups = "spi_grp";
+ };
+ };
+ };
+ };
- compatible: One of:
- "ingenic,jz4740-pinctrl"
+ - "ingenic,jz4725b-pinctrl"
- "ingenic,jz4770-pinctrl"
- "ingenic,jz4780-pinctrl"
- reg: Address range of the pinctrl registers.
-GPIO sub-nodes
---------------
+Required properties for sub-nodes (GPIO chips):
+-----------------------------------------------
-The pinctrl node can have optional sub-nodes for the Ingenic GPIO driver;
-please refer to ../gpio/ingenic,gpio.txt.
+ - compatible: Must contain one of:
+ - "ingenic,jz4740-gpio"
+ - "ingenic,jz4770-gpio"
+ - "ingenic,jz4780-gpio"
+ - reg: The GPIO bank number.
+ - interrupt-controller: Marks the device node as an interrupt controller.
+ - interrupts: Interrupt specifier for the controllers interrupt.
+ - #interrupt-cells: Should be 2. Refer to
+ ../interrupt-controller/interrupts.txt for more details.
+ - gpio-controller: Marks the device node as a GPIO controller.
+ - #gpio-cells: Should be 2. The first cell is the GPIO number and the second
+ cell specifies GPIO flags, as defined in <dt-bindings/gpio/gpio.h>. Only the
+ GPIO_ACTIVE_HIGH and GPIO_ACTIVE_LOW flags are supported.
+ - gpio-ranges: Range of pins managed by the GPIO controller. Refer to
+ ../gpio/gpio.txt for more details.
Example:
pinctrl: pin-controller@10010000 {
compatible = "ingenic,jz4740-pinctrl";
reg = <0x10010000 0x400>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ gpa: gpio@0 {
+ compatible = "ingenic,jz4740-gpio";
+ reg = <0>;
+
+ gpio-controller;
+ gpio-ranges = <&pinctrl 0 0 32>;
+ #gpio-cells = <2>;
+
+ interrupt-controller;
+ #interrupt-cells = <2>;
+
+ interrupt-parent = <&intc>;
+ interrupts = <28>;
+ };
};
"amlogic,meson-gxl-aobus-pinctrl"
"amlogic,meson-axg-periphs-pinctrl"
"amlogic,meson-axg-aobus-pinctrl"
+ "amlogic,meson-g12a-periphs-pinctrl"
+ "amlogic,meson-g12a-aobus-pinctrl"
- reg: address and size of registers controlling irq functionality
=== GPIO sub-nodes ===
--- /dev/null
+Nuvoton NPCM7XX Pin Controllers
+
+The Nuvoton BMC NPCM7XX Pin Controller multi-function routed through
+the multiplexing block, Each pin supports GPIO functionality (GPIOx)
+and multiple functions that directly connect the pin to different
+hardware blocks.
+
+Required properties:
+- #address-cells : should be 1.
+- #size-cells : should be 1.
+- compatible : "nuvoton,npcm750-pinctrl" for Poleg NPCM7XX.
+- ranges : defines mapping ranges between pin controller node (parent)
+ to GPIO bank node (children).
+
+=== GPIO Bank Subnode ===
+
+The NPCM7XX has 8 GPIO Banks each GPIO bank supports 32 GPIO.
+
+Required GPIO Bank subnode-properties:
+- reg : specifies physical base address and size of the GPIO
+ bank registers.
+- gpio-controller : Marks the device node as a GPIO controller.
+- #gpio-cells : Must be <2>. The first cell is the gpio pin number
+ and the second cell is used for optional parameters.
+- interrupts : contain the GPIO bank interrupt with flags for falling edge.
+- gpio-ranges : defines the range of pins managed by the GPIO bank controller.
+
+For example, GPIO bank subnodes like the following:
+ gpio0: gpio@f0010000 {
+ gpio-controller;
+ #gpio-cells = <2>;
+ reg = <0x0 0x80>;
+ interrupts = <GIC_SPI 116 IRQ_TYPE_LEVEL_HIGH>;
+ gpio-ranges = <&pinctrl 0 0 32>;
+ };
+
+=== Pin Mux Subnode ===
+
+- pin: A string containing the name of the pin
+ An array of strings, each string containing the name of a pin.
+ These pin are used for selecting pin configuration.
+
+The following are the list of pins available:
+ "GPIO0/IOX1DI", "GPIO1/IOX1LD", "GPIO2/IOX1CK", "GPIO3/IOX1D0",
+ "GPIO4/IOX2DI/SMB1DSDA", "GPIO5/IOX2LD/SMB1DSCL", "GPIO6/IOX2CK/SMB2DSDA",
+ "GPIO7/IOX2D0/SMB2DSCL", "GPIO8/LKGPO1", "GPIO9/LKGPO2", "GPIO10/IOXHLD",
+ "GPIO11/IOXHCK", "GPIO12/GSPICK/SMB5BSCL", "GPIO13/GSPIDO/SMB5BSDA",
+ "GPIO14/GSPIDI/SMB5CSCL", "GPIO15/GSPICS/SMB5CSDA", "GPIO16/LKGPO0",
+ "GPIO17/PSPI2DI/SMB4DEN","GPIO18/PSPI2D0/SMB4BSDA", "GPIO19/PSPI2CK/SMB4BSCL",
+ "GPIO20/SMB4CSDA/SMB15SDA", "GPIO21/SMB4CSCL/SMB15SCL", "GPIO22/SMB4DSDA/SMB14SDA",
+ "GPIO23/SMB4DSCL/SMB14SCL", "GPIO24/IOXHDO", "GPIO25/IOXHDI", "GPIO26/SMB5SDA",
+ "GPIO27/SMB5SCL", "GPIO28/SMB4SDA", "GPIO29/SMB4SCL", "GPIO30/SMB3SDA",
+ "GPIO31/SMB3SCL", "GPIO32/nSPI0CS1","SPI0D2", "SPI0D3", "GPIO37/SMB3CSDA",
+ "GPIO38/SMB3CSCL", "GPIO39/SMB3BSDA", "GPIO40/SMB3BSCL", "GPIO41/BSPRXD",
+ "GPO42/BSPTXD/STRAP11", "GPIO43/RXD1/JTMS2/BU1RXD", "GPIO44/nCTS1/JTDI2/BU1CTS",
+ "GPIO45/nDCD1/JTDO2", "GPIO46/nDSR1/JTCK2", "GPIO47/nRI1/JCP_RDY2",
+ "GPIO48/TXD2/BSPTXD", "GPIO49/RXD2/BSPRXD", "GPIO50/nCTS2", "GPO51/nRTS2/STRAP2",
+ "GPIO52/nDCD2", "GPO53/nDTR2_BOUT2/STRAP1", "GPIO54/nDSR2", "GPIO55/nRI2",
+ "GPIO56/R1RXERR", "GPIO57/R1MDC", "GPIO58/R1MDIO", "GPIO59/SMB3DSDA",
+ "GPIO60/SMB3DSCL", "GPO61/nDTR1_BOUT1/STRAP6", "GPO62/nRTST1/STRAP5",
+ "GPO63/TXD1/STRAP4", "GPIO64/FANIN0", "GPIO65/FANIN1", "GPIO66/FANIN2",
+ "GPIO67/FANIN3", "GPIO68/FANIN4", "GPIO69/FANIN5", "GPIO70/FANIN6", "GPIO71/FANIN7",
+ "GPIO72/FANIN8", "GPIO73/FANIN9", "GPIO74/FANIN10", "GPIO75/FANIN11",
+ "GPIO76/FANIN12", "GPIO77/FANIN13","GPIO78/FANIN14", "GPIO79/FANIN15",
+ "GPIO80/PWM0", "GPIO81/PWM1", "GPIO82/PWM2", "GPIO83/PWM3", "GPIO84/R2TXD0",
+ "GPIO85/R2TXD1", "GPIO86/R2TXEN", "GPIO87/R2RXD0", "GPIO88/R2RXD1", "GPIO89/R2CRSDV",
+ "GPIO90/R2RXERR", "GPIO91/R2MDC", "GPIO92/R2MDIO", "GPIO93/GA20/SMB5DSCL",
+ "GPIO94/nKBRST/SMB5DSDA", "GPIO95/nLRESET/nESPIRST", "GPIO96/RG1TXD0",
+ "GPIO97/RG1TXD1", "GPIO98/RG1TXD2", "GPIO99/RG1TXD3","GPIO100/RG1TXC",
+ "GPIO101/RG1TXCTL", "GPIO102/RG1RXD0", "GPIO103/RG1RXD1", "GPIO104/RG1RXD2",
+ "GPIO105/RG1RXD3", "GPIO106/RG1RXC", "GPIO107/RG1RXCTL", "GPIO108/RG1MDC",
+ "GPIO109/RG1MDIO", "GPIO110/RG2TXD0/DDRV0", "GPIO111/RG2TXD1/DDRV1",
+ "GPIO112/RG2TXD2/DDRV2", "GPIO113/RG2TXD3/DDRV3", "GPIO114/SMB0SCL",
+ "GPIO115/SMB0SDA", "GPIO116/SMB1SCL", "GPIO117/SMB1SDA", "GPIO118/SMB2SCL",
+ "GPIO119/SMB2SDA", "GPIO120/SMB2CSDA", "GPIO121/SMB2CSCL", "GPIO122/SMB2BSDA",
+ "GPIO123/SMB2BSCL", "GPIO124/SMB1CSDA", "GPIO125/SMB1CSCL","GPIO126/SMB1BSDA",
+ "GPIO127/SMB1BSCL", "GPIO128/SMB8SCL", "GPIO129/SMB8SDA", "GPIO130/SMB9SCL",
+ "GPIO131/SMB9SDA", "GPIO132/SMB10SCL", "GPIO133/SMB10SDA","GPIO134/SMB11SCL",
+ "GPIO135/SMB11SDA", "GPIO136/SD1DT0", "GPIO137/SD1DT1", "GPIO138/SD1DT2",
+ "GPIO139/SD1DT3", "GPIO140/SD1CLK", "GPIO141/SD1WP", "GPIO142/SD1CMD",
+ "GPIO143/SD1CD/SD1PWR", "GPIO144/PWM4", "GPIO145/PWM5", "GPIO146/PWM6",
+ "GPIO147/PWM7", "GPIO148/MMCDT4", "GPIO149/MMCDT5", "GPIO150/MMCDT6",
+ "GPIO151/MMCDT7", "GPIO152/MMCCLK", "GPIO153/MMCWP", "GPIO154/MMCCMD",
+ "GPIO155/nMMCCD/nMMCRST", "GPIO156/MMCDT0", "GPIO157/MMCDT1", "GPIO158/MMCDT2",
+ "GPIO159/MMCDT3", "GPIO160/CLKOUT/RNGOSCOUT", "GPIO161/nLFRAME/nESPICS",
+ "GPIO162/SERIRQ", "GPIO163/LCLK/ESPICLK", "GPIO164/LAD0/ESPI_IO0",
+ "GPIO165/LAD1/ESPI_IO1", "GPIO166/LAD2/ESPI_IO2", "GPIO167/LAD3/ESPI_IO3",
+ "GPIO168/nCLKRUN/nESPIALERT", "GPIO169/nSCIPME", "GPIO170/nSMI", "GPIO171/SMB6SCL",
+ "GPIO172/SMB6SDA", "GPIO173/SMB7SCL", "GPIO174/SMB7SDA", "GPIO175/PSPI1CK/FANIN19",
+ "GPIO176/PSPI1DO/FANIN18", "GPIO177/PSPI1DI/FANIN17", "GPIO178/R1TXD0",
+ "GPIO179/R1TXD1", "GPIO180/R1TXEN", "GPIO181/R1RXD0", "GPIO182/R1RXD1",
+ "GPIO183/SPI3CK", "GPO184/SPI3D0/STRAP9", "GPO185/SPI3D1/STRAP10",
+ "GPIO186/nSPI3CS0", "GPIO187/nSPI3CS1", "GPIO188/SPI3D2/nSPI3CS2",
+ "GPIO189/SPI3D3/nSPI3CS3", "GPIO190/nPRD_SMI", "GPIO191", "GPIO192", "GPIO193/R1CRSDV",
+ "GPIO194/SMB0BSCL", "GPIO195/SMB0BSDA", "GPIO196/SMB0CSCL", "GPIO197/SMB0DEN",
+ "GPIO198/SMB0DSDA", "GPIO199/SMB0DSCL", "GPIO200/R2CK", "GPIO201/R1CK",
+ "GPIO202/SMB0CSDA", "GPIO203/FANIN16", "GPIO204/DDC2SCL", "GPIO205/DDC2SDA",
+ "GPIO206/HSYNC2", "GPIO207/VSYNC2", "GPIO208/RG2TXC/DVCK", "GPIO209/RG2TXCTL/DDRV4",
+ "GPIO210/RG2RXD0/DDRV5", "GPIO211/RG2RXD1/DDRV6", "GPIO212/RG2RXD2/DDRV7",
+ "GPIO213/RG2RXD3/DDRV8", "GPIO214/RG2RXC/DDRV9", "GPIO215/RG2RXCTL/DDRV10",
+ "GPIO216/RG2MDC/DDRV11", "GPIO217/RG2MDIO/DVHSYNC", "GPIO218/nWDO1",
+ "GPIO219/nWDO2", "GPIO220/SMB12SCL", "GPIO221/SMB12SDA", "GPIO222/SMB13SCL",
+ "GPIO223/SMB13SDA", "GPIO224/SPIXCK", "GPO225/SPIXD0/STRAP12", "GPO226/SPIXD1/STRAP13",
+ "GPIO227/nSPIXCS0", "GPIO228/nSPIXCS1", "GPO229/SPIXD2/STRAP3", "GPIO230/SPIXD3",
+ "GPIO231/nCLKREQ", "GPI255/DACOSEL"
+
+Optional Properties:
+ bias-disable, bias-pull-down, bias-pull-up, input-enable,
+ input-disable, output-high, output-low, drive-push-pull,
+ drive-open-drain, input-debounce, slew-rate, drive-strength
+
+ slew-rate valid arguments are:
+ <0> - slow
+ <1> - fast
+ drive-strength valid arguments are:
+ <2> - 2mA
+ <4> - 4mA
+ <8> - 8mA
+ <12> - 12mA
+ <16> - 16mA
+ <24> - 24mA
+
+For example, pinctrl might have pinmux subnodes like the following:
+
+ gpio0_iox1d1_pin: gpio0-iox1d1-pin {
+ pins = "GPIO0/IOX1DI";
+ output-high;
+ };
+ gpio0_iox1ck_pin: gpio0-iox1ck-pin {
+ pins = "GPIO2/IOX1CK";
+ output_high;
+ };
+
+=== Pin Group Subnode ===
+
+Required pin group subnode-properties:
+- groups : A string containing the name of the group to mux.
+- function: A string containing the name of the function to mux to the
+ group.
+
+The following are the list of the available groups and functions :
+ smb0, smb0b, smb0c, smb0d, smb0den, smb1, smb1b, smb1c, smb1d,
+ smb2, smb2b, smb2c, smb2d, smb3, smb3b, smb3c, smb3d, smb4, smb4b,
+ smb4c, smb4d, smb4den, smb5, smb5b, smb5c, smb5d, ga20kbc, smb6,
+ smb7, smb8, smb9, smb10, smb11, smb12, smb13, smb14, smb15, fanin0,
+ fanin1, fanin2, fanin3, fanin4, fanin5, fanin6, fanin7, fanin8,
+ fanin9, fanin10, fanin11 fanin12 fanin13, fanin14, fanin15, faninx,
+ pwm0, pwm1, pwm2, pwm3, pwm4, pwm5, pwm6, pwm7, rg1, rg1mdio, rg2,
+ rg2mdio, ddr, uart1, uart2, bmcuart0a, bmcuart0b, bmcuart1, iox1,
+ iox2, ioxh, gspi, mmc, mmcwp, mmccd, mmcrst, mmc8, r1, r1err, r1md,
+ r2, r2err, r2md, sd1, sd1pwr, wdog1, wdog2, scipme, sci, serirq,
+ jtag2, spix, spixcs1, pspi1, pspi2, ddc, clkreq, clkout, spi3, spi3cs1,
+ spi3quad, spi3cs2, spi3cs3, spi0cs1, lpc, lpcclk, espi, lkgpo0, lkgpo1,
+ lkgpo2, nprd_smi
+
+For example, pinctrl might have group subnodes like the following:
+ r1err_pins: r1err-pins {
+ groups = "r1err";
+ function = "r1err";
+ };
+ r1md_pins: r1md-pins {
+ groups = "r1md";
+ function = "r1md";
+ };
+ r1_pins: r1-pins {
+ groups = "r1";
+ function = "r1";
+ };
+
+Examples
+========
+pinctrl: pinctrl@f0800000 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "nuvoton,npcm750-pinctrl";
+ ranges = <0 0xf0010000 0x8000>;
+
+ gpio0: gpio@f0010000 {
+ gpio-controller;
+ #gpio-cells = <2>;
+ reg = <0x0 0x80>;
+ interrupts = <GIC_SPI 116 IRQ_TYPE_LEVEL_HIGH>;
+ gpio-ranges = <&pinctrl 0 0 32>;
+ };
+
+ ....
+
+ gpio7: gpio@f0017000 {
+ gpio-controller;
+ #gpio-cells = <2>;
+ reg = <0x7000 0x80>;
+ interrupts = <GIC_SPI 123 IRQ_TYPE_LEVEL_HIGH>;
+ gpio-ranges = <&pinctrl 0 224 32>;
+ };
+
+ gpio0_iox1d1_pin: gpio0-iox1d1-pin {
+ pins = "GPIO0/IOX1DI";
+ output-high;
+ };
+
+ iox1_pins: iox1-pins {
+ groups = "iox1";
+ function = "iox1";
+ };
+ iox2_pins: iox2-pins {
+ groups = "iox2";
+ function = "iox2";
+ };
+
+ ....
+
+ clkreq_pins: clkreq-pins {
+ groups = "clkreq";
+ function = "clkreq";
+ };
+};
\ No newline at end of file
"qcom,pm8998-gpio"
"qcom,pma8084-gpio"
"qcom,pmi8994-gpio"
+ "qcom,pms405-gpio"
And must contain either "qcom,spmi-gpio" or "qcom,ssbi-gpio"
if the device is on an spmi bus or an ssbi bus respectively
gpio1-gpio26 for pm8998
gpio1-gpio22 for pma8084
gpio1-gpio10 for pmi8994
+ gpio1-gpio11 for pms405
- function:
Usage: required
--- /dev/null
+Qualcomm QCS404 TLMM block
+
+This binding describes the Top Level Mode Multiplexer block found in the
+QCS404 platform.
+
+- compatible:
+ Usage: required
+ Value type: <string>
+ Definition: must be "qcom,qcs404-pinctrl"
+
+- reg:
+ Usage: required
+ Value type: <prop-encoded-array>
+ Definition: the base address and size of the north, south and east TLMM
+ tiles.
+
+- reg-names:
+ Usage: required
+ Value type: <stringlist>
+ Defintiion: names for the cells of reg, must contain "north", "south"
+ and "east".
+
+- interrupts:
+ Usage: required
+ Value type: <prop-encoded-array>
+ Definition: should specify the TLMM summary IRQ.
+
+- interrupt-controller:
+ Usage: required
+ Value type: <none>
+ Definition: identifies this node as an interrupt controller
+
+- #interrupt-cells:
+ Usage: required
+ Value type: <u32>
+ Definition: must be 2. Specifying the pin number and flags, as defined
+ in <dt-bindings/interrupt-controller/irq.h>
+
+- gpio-controller:
+ Usage: required
+ Value type: <none>
+ Definition: identifies this node as a gpio controller
+
+- #gpio-cells:
+ Usage: required
+ Value type: <u32>
+ Definition: must be 2. Specifying the pin number and flags, as defined
+ in <dt-bindings/gpio/gpio.h>
+
+- gpio-ranges:
+ Usage: required
+ Definition: see ../gpio/gpio.txt
+
+Please refer to ../gpio/gpio.txt and ../interrupt-controller/interrupts.txt for
+a general description of GPIO and interrupt bindings.
+
+Please refer to pinctrl-bindings.txt in this directory for details of the
+common pinctrl bindings used by client devices, including the meaning of the
+phrase "pin configuration node".
+
+The pin configuration nodes act as a container for an arbitrary number of
+subnodes. Each of these subnodes represents some desired configuration for a
+pin, a group, or a list of pins or groups. This configuration can include the
+mux function to select on those pin(s)/group(s), and various pin configuration
+parameters, such as pull-up, drive strength, etc.
+
+
+PIN CONFIGURATION NODES:
+
+The name of each subnode is not important; all subnodes should be enumerated
+and processed purely based on their content.
+
+Each subnode only affects those parameters that are explicitly listed. In
+other words, a subnode that lists a mux function but no pin configuration
+parameters implies no information about any pin configuration parameters.
+Similarly, a pin subnode that describes a pullup parameter implies no
+information about e.g. the mux function.
+
+
+The following generic properties as defined in pinctrl-bindings.txt are valid
+to specify in a pin configuration subnode:
+
+- pins:
+ Usage: required
+ Value type: <string-array>
+ Definition: List of gpio pins affected by the properties specified in
+ this subnode.
+
+ Valid pins are:
+ gpio0-gpio119
+ Supports mux, bias and drive-strength
+
+ sdc1_clk, sdc1_cmd, sdc1_data, sdc2_clk, sdc2_cmd,
+ sdc2_data
+ Supports bias and drive-strength
+
+ ufs_reset
+ Supports bias and drive-strength
+
+- function:
+ Usage: required
+ Value type: <string>
+ Definition: Specify the alternative function to be configured for the
+ specified pins. Functions are only valid for gpio pins.
+ Valid values are:
+
+ gpio, hdmi_tx, hdmi_ddc, blsp_uart_tx_a2, blsp_spi2, m_voc,
+ qdss_cti_trig_in_a0, blsp_uart_rx_a2, qdss_tracectl_a,
+ blsp_uart2, aud_cdc, blsp_i2c_sda_a2, qdss_tracedata_a,
+ blsp_i2c_scl_a2, qdss_tracectl_b, qdss_cti_trig_in_b0,
+ blsp_uart1, blsp_spi_mosi_a1, blsp_spi_miso_a1,
+ qdss_tracedata_b, blsp_i2c1, blsp_spi_cs_n_a1, gcc_plltest,
+ blsp_spi_clk_a1, rgb_data0, blsp_uart5, blsp_spi5,
+ adsp_ext, rgb_data1, prng_rosc, rgb_data2, blsp_i2c5,
+ gcc_gp1_clk_b, rgb_data3, gcc_gp2_clk_b, blsp_spi0,
+ blsp_uart0, gcc_gp3_clk_b, blsp_i2c0, qdss_traceclk_b,
+ pcie_clk, nfc_irq, blsp_spi4, nfc_dwl, audio_ts, rgb_data4,
+ spi_lcd, blsp_uart_tx_b2, gcc_gp3_clk_a, rgb_data5,
+ blsp_uart_rx_b2, blsp_i2c_sda_b2, blsp_i2c_scl_b2,
+ pwm_led11, i2s_3_data0_a, ebi2_lcd, i2s_3_data1_a,
+ i2s_3_data2_a, atest_char, pwm_led3, i2s_3_data3_a,
+ pwm_led4, i2s_4, ebi2_a, dsd_clk_b, pwm_led5, pwm_led6,
+ pwm_led7, pwm_led8, pwm_led24, spkr_dac0, blsp_i2c4,
+ pwm_led9, pwm_led10, spdifrx_opt, pwm_led12, pwm_led13,
+ pwm_led14, wlan1_adc1, rgb_data_b0, pwm_led15,
+ blsp_spi_mosi_b1, wlan1_adc0, rgb_data_b1, pwm_led16,
+ blsp_spi_miso_b1, qdss_cti_trig_out_b0, wlan2_adc1,
+ rgb_data_b2, pwm_led17, blsp_spi_cs_n_b1, wlan2_adc0,
+ rgb_data_b3, pwm_led18, blsp_spi_clk_b1, rgb_data_b4,
+ pwm_led19, ext_mclk1_b, qdss_traceclk_a, rgb_data_b5,
+ pwm_led20, atest_char3, i2s_3_sck_b, ldo_update, bimc_dte0,
+ rgb_hsync, pwm_led21, i2s_3_ws_b, dbg_out, rgb_vsync,
+ i2s_3_data0_b, ldo_en, hdmi_dtest, rgb_de, i2s_3_data1_b,
+ hdmi_lbk9, rgb_clk, atest_char1, i2s_3_data2_b, ebi_cdc,
+ hdmi_lbk8, rgb_mdp, atest_char0, i2s_3_data3_b, hdmi_lbk7,
+ rgb_data_b6, rgb_data_b7, hdmi_lbk6, rgmii_int, cri_trng1,
+ rgmii_wol, cri_trng0, gcc_tlmm, rgmii_ck, rgmii_tx,
+ hdmi_lbk5, hdmi_pixel, hdmi_rcv, hdmi_lbk4, rgmii_ctl,
+ ext_lpass, rgmii_rx, cri_trng, hdmi_lbk3, hdmi_lbk2,
+ qdss_cti_trig_out_b1, rgmii_mdio, hdmi_lbk1, rgmii_mdc,
+ hdmi_lbk0, ir_in, wsa_en, rgb_data6, rgb_data7,
+ atest_char2, ebi_ch0, blsp_uart3, blsp_spi3, sd_write,
+ blsp_i2c3, gcc_gp1_clk_a, qdss_cti_trig_in_b1,
+ gcc_gp2_clk_a, ext_mclk0, mclk_in1, i2s_1, dsd_clk_a,
+ qdss_cti_trig_in_a1, rgmi_dll1, pwm_led22, pwm_led23,
+ qdss_cti_trig_out_a0, rgmi_dll2, pwm_led1,
+ qdss_cti_trig_out_a1, pwm_led2, i2s_2, pll_bist,
+ ext_mclk1_a, mclk_in2, bimc_dte1, i2s_3_sck_a, i2s_3_ws_a
+
+- bias-disable:
+ Usage: optional
+ Value type: <none>
+ Definition: The specified pins should be configued as no pull.
+
+- bias-pull-down:
+ Usage: optional
+ Value type: <none>
+ Definition: The specified pins should be configued as pull down.
+
+- bias-pull-up:
+ Usage: optional
+ Value type: <none>
+ Definition: The specified pins should be configued as pull up.
+
+- output-high:
+ Usage: optional
+ Value type: <none>
+ Definition: The specified pins are configured in output mode, driven
+ high.
+ Not valid for sdc pins.
+
+- output-low:
+ Usage: optional
+ Value type: <none>
+ Definition: The specified pins are configured in output mode, driven
+ low.
+ Not valid for sdc pins.
+
+- drive-strength:
+ Usage: optional
+ Value type: <u32>
+ Definition: Selects the drive strength for the specified pins, in mA.
+ Valid values are: 2, 4, 6, 8, 10, 12, 14 and 16
+
+Example:
+
+ tlmm: pinctrl@1000000 {
+ compatible = "qcom,qcs404-pinctrl";
+ reg = <0x01000000 0x200000>,
+ <0x01300000 0x200000>,
+ <0x07b00000 0x200000>;
+ reg-names = "south", "north", "east";
+ interrupts = <GIC_SPI 208 IRQ_TYPE_LEVEL_HIGH>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-ranges = <&tlmm 0 0 120>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ };
--- /dev/null
+Qualcomm Technologies, Inc. SDM660 TLMM block
+
+This binding describes the Top Level Mode Multiplexer block found in the
+SDM660 platform.
+
+- compatible:
+ Usage: required
+ Value type: <string>
+ Definition: must be "qcom,sdm660-pinctrl" or
+ "qcom,sdm630-pinctrl".
+
+- reg:
+ Usage: required
+ Value type: <prop-encoded-array>
+ Definition: the base address and size of the north, center and south
+ TLMM tiles.
+
+- reg-names:
+ Usage: required
+ Value type: <stringlist>
+ Definition: names for the cells of reg, must contain "north", "center"
+ and "south".
+
+- interrupts:
+ Usage: required
+ Value type: <prop-encoded-array>
+ Definition: should specify the TLMM summary IRQ.
+
+- interrupt-controller:
+ Usage: required
+ Value type: <none>
+ Definition: identifies this node as an interrupt controller
+
+- #interrupt-cells:
+ Usage: required
+ Value type: <u32>
+ Definition: must be 2. Specifying the pin number and flags, as defined
+ in <dt-bindings/interrupt-controller/irq.h>
+
+- gpio-controller:
+ Usage: required
+ Value type: <none>
+ Definition: identifies this node as a gpio controller
+
+- gpio-ranges:
+ Usage: required
+ Value type: <prop-encoded-array>
+ Definition: Specifies the mapping between gpio controller and
+ pin-controller pins.
+
+- #gpio-cells:
+ Usage: required
+ Value type: <u32>
+ Definition: must be 2. Specifying the pin number and flags, as defined
+ in <dt-bindings/gpio/gpio.h>
+
+Please refer to ../gpio/gpio.txt and ../interrupt-controller/interrupts.txt for
+a general description of GPIO and interrupt bindings.
+
+Please refer to pinctrl-bindings.txt in this directory for details of the
+common pinctrl bindings used by client devices, including the meaning of the
+phrase "pin configuration node".
+
+The pin configuration nodes act as a container for an arbitrary number of
+subnodes. Each of these subnodes represents some desired configuration for a
+pin, a group, or a list of pins or groups. This configuration can include the
+mux function to select on those pin(s)/group(s), and various pin configuration
+parameters, such as pull-up, drive strength, etc.
+
+
+PIN CONFIGURATION NODES:
+
+The name of each subnode is not important; all subnodes should be enumerated
+and processed purely based on their content.
+
+Each subnode only affects those parameters that are explicitly listed. In
+other words, a subnode that lists a mux function but no pin configuration
+parameters implies no information about any pin configuration parameters.
+Similarly, a pin subnode that describes a pullup parameter implies no
+information about e.g. the mux function.
+
+
+The following generic properties as defined in pinctrl-bindings.txt are valid
+to specify in a pin configuration subnode:
+
+- pins:
+ Usage: required
+ Value type: <string-array>
+ Definition: List of gpio pins affected by the properties specified in
+ this subnode. Valid pins are:
+ gpio0-gpio113,
+ Supports mux, bias and drive-strength
+ sdc1_clk, sdc1_cmd, sdc1_data sdc2_clk, sdc2_cmd, sdc2_data sdc1_rclk,
+ Supports bias and drive-strength
+
+- function:
+ Usage: required
+ Value type: <string>
+ Definition: Specify the alternative function to be configured for the
+ specified pins. Functions are only valid for gpio pins.
+ Valid values are:
+ adsp_ext, agera_pll, atest_char, atest_char0, atest_char1,
+ atest_char2, atest_char3, atest_gpsadc0, atest_gpsadc1,
+ atest_tsens, atest_tsens2, atest_usb1, atest_usb10,
+ atest_usb11, atest_usb12, atest_usb13, atest_usb2,
+ atest_usb20, atest_usb21, atest_usb22, atest_usb23,
+ audio_ref, bimc_dte0, bimc_dte1, blsp_i2c1, blsp_i2c2,
+ blsp_i2c3, blsp_i2c4, blsp_i2c5, blsp_i2c6, blsp_i2c7,
+ blsp_i2c8_a, blsp_i2c8_b, blsp_spi1, blsp_spi2, blsp_spi3,
+ blsp_spi3_cs1, blsp_spi3_cs2, blsp_spi4, blsp_spi5,
+ blsp_spi6, blsp_spi7, blsp_spi8_a, blsp_spi8_b,
+ blsp_spi8_cs1, blsp_spi8_cs2, blsp_uart1, blsp_uart2,
+ blsp_uart5, blsp_uart6_a, blsp_uart6_b, blsp_uim1,
+ blsp_uim2, blsp_uim5, blsp_uim6, cam_mclk, cci_async,
+ cci_i2c, cri_trng, cri_trng0, cri_trng1, dbg_out, ddr_bist,
+ gcc_gp1, gcc_gp2, gcc_gp3, gpio, gps_tx_a, gps_tx_b, gps_tx_c,
+ isense_dbg, jitter_bist, ldo_en, ldo_update, m_voc, mdp_vsync,
+ mdss_vsync0, mdss_vsync1, mdss_vsync2, mdss_vsync3, mss_lte,
+ nav_pps_a, nav_pps_b, nav_pps_c, pa_indicator, phase_flag0,
+ phase_flag1, phase_flag10, phase_flag11, phase_flag12,
+ phase_flag13, phase_flag14, phase_flag15, phase_flag16,
+ phase_flag17, phase_flag18, phase_flag19, phase_flag2,
+ phase_flag20, phase_flag21, phase_flag22, phase_flag23,
+ phase_flag24, phase_flag25, phase_flag26, phase_flag27,
+ phase_flag28, phase_flag29, phase_flag3, phase_flag30,
+ phase_flag31, phase_flag4, phase_flag5, phase_flag6,
+ phase_flag7, phase_flag8, phase_flag9, pll_bypassnl,
+ pll_reset, pri_mi2s, pri_mi2s_ws, prng_rosc, pwr_crypto,
+ pwr_modem, pwr_nav, qdss_cti0_a, qdss_cti0_b, qdss_cti1_a,
+ qdss_cti1_b, qdss_gpio, qdss_gpio0, qdss_gpio1, qdss_gpio10,
+ qdss_gpio11, qdss_gpio12, qdss_gpio13, qdss_gpio14, qdss_gpio15,
+ qdss_gpio2, qdss_gpio3, qdss_gpio4, qdss_gpio5, qdss_gpio6,
+ qdss_gpio7, qdss_gpio8, qdss_gpio9, qlink_enable, qlink_request,
+ qspi_clk, qspi_cs, qspi_data0, qspi_data1, qspi_data2,
+ qspi_data3, qspi_resetn, sec_mi2s, sndwire_clk, sndwire_data,
+ sp_cmu, ssc_irq, tgu_ch0, tgu_ch1, tsense_pwm1, tsense_pwm2,
+ uim1_clk, uim1_data, uim1_present, uim1_reset, uim2_clk,
+ uim2_data, uim2_present, uim2_reset, uim_batt, vfr_1,
+ vsense_clkout, vsense_data0, vsense_data1, vsense_mode,
+ wlan1_adc0, wlan1_adc1, wlan2_adc0, wlan2_adc1
+
+- bias-disable:
+ Usage: optional
+ Value type: <none>
+ Definition: The specified pins should be configued as no pull.
+
+- bias-pull-down:
+ Usage: optional
+ Value type: <none>
+ Definition: The specified pins should be configued as pull down.
+
+- bias-pull-up:
+ Usage: optional
+ Value type: <none>
+ Definition: The specified pins should be configued as pull up.
+
+- output-high:
+ Usage: optional
+ Value type: <none>
+ Definition: The specified pins are configured in output mode, driven
+ high.
+ Not valid for sdc pins.
+
+- output-low:
+ Usage: optional
+ Value type: <none>
+ Definition: The specified pins are configured in output mode, driven
+ low.
+ Not valid for sdc pins.
+
+- drive-strength:
+ Usage: optional
+ Value type: <u32>
+ Definition: Selects the drive strength for the specified pins, in mA.
+ Valid values are: 2, 4, 6, 8, 10, 12, 14 and 16
+
+Example:
+
+ tlmm: pinctrl@3100000 {
+ compatible = "qcom,sdm660-pinctrl";
+ reg = <0x3100000 0x200000>,
+ <0x3500000 0x200000>,
+ <0x3900000 0x200000>;
+ reg-names = "south", "center", "north";
+ interrupts = <GIC_SPI 208 IRQ_TYPE_LEVEL_HIGH>;
+ gpio-controller;
+ gpio-ranges = <&tlmm 0 0 114>;
+ #gpio-cells = <2>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ };
- "renesas,pfc-r8a73a4": for R8A73A4 (R-Mobile APE6) compatible pin-controller.
- "renesas,pfc-r8a7740": for R8A7740 (R-Mobile A1) compatible pin-controller.
- "renesas,pfc-r8a7743": for R8A7743 (RZ/G1M) compatible pin-controller.
+ - "renesas,pfc-r8a7744": for R8A7744 (RZ/G1N) compatible pin-controller.
- "renesas,pfc-r8a7745": for R8A7745 (RZ/G1E) compatible pin-controller.
- "renesas,pfc-r8a77470": for R8A77470 (RZ/G1C) compatible pin-controller.
+ - "renesas,pfc-r8a774a1": for R8A774A1 (RZ/G2M) compatible pin-controller.
+ - "renesas,pfc-r8a774c0": for R8A774C0 (RZ/G2E) compatible pin-controller.
- "renesas,pfc-r8a7778": for R8A7778 (R-Car M1) compatible pin-controller.
- "renesas,pfc-r8a7779": for R8A7779 (R-Car H1) compatible pin-controller.
- "renesas,pfc-r8a7790": for R8A7790 (R-Car H2) compatible pin-controller.
--- /dev/null
+Renesas RZ/N1 SoC Pinctrl node description.
+
+Pin controller node
+-------------------
+Required properties:
+- compatible: SoC-specific compatible string "renesas,<soc-specific>-pinctrl"
+ followed by "renesas,rzn1-pinctrl" as fallback. The SoC-specific compatible
+ strings must be one of:
+ "renesas,r9a06g032-pinctrl" for RZ/N1D
+ "renesas,r9a06g033-pinctrl" for RZ/N1S
+- reg: Address base and length of the memory area where the pin controller
+ hardware is mapped to.
+- clocks: phandle for the clock, see the description of clock-names below.
+- clock-names: Contains the name of the clock:
+ "bus", the bus clock, sometimes described as pclk, for register accesses.
+
+Example:
+ pinctrl: pin-controller@40067000 {
+ compatible = "renesas,r9a06g032-pinctrl", "renesas,rzn1-pinctrl";
+ reg = <0x40067000 0x1000>, <0x51000000 0x480>;
+ clocks = <&sysctrl R9A06G032_HCLK_PINCONFIG>;
+ clock-names = "bus";
+ };
+
+Sub-nodes
+---------
+
+The child nodes of the pin controller node describe a pin multiplexing
+function.
+
+- Pin multiplexing sub-nodes:
+ A pin multiplexing sub-node describes how to configure a set of
+ (or a single) pin in some desired alternate function mode.
+ A single sub-node may define several pin configurations.
+ Please refer to pinctrl-bindings.txt to get to know more on generic
+ pin properties usage.
+
+ The allowed generic formats for a pin multiplexing sub-node are the
+ following ones:
+
+ node-1 {
+ pinmux = <PIN_ID_AND_MUX>, <PIN_ID_AND_MUX>, ... ;
+ GENERIC_PINCONFIG;
+ };
+
+ node-2 {
+ sub-node-1 {
+ pinmux = <PIN_ID_AND_MUX>, <PIN_ID_AND_MUX>, ... ;
+ GENERIC_PINCONFIG;
+ };
+
+ sub-node-2 {
+ pinmux = <PIN_ID_AND_MUX>, <PIN_ID_AND_MUX>, ... ;
+ GENERIC_PINCONFIG;
+ };
+
+ ...
+
+ sub-node-n {
+ pinmux = <PIN_ID_AND_MUX>, <PIN_ID_AND_MUX>, ... ;
+ GENERIC_PINCONFIG;
+ };
+ };
+
+ node-3 {
+ pinmux = <PIN_ID_AND_MUX>, <PIN_ID_AND_MUX>, ... ;
+ GENERIC_PINCONFIG;
+
+ sub-node-1 {
+ pinmux = <PIN_ID_AND_MUX>, <PIN_ID_AND_MUX>, ... ;
+ GENERIC_PINCONFIG;
+ };
+
+ ...
+
+ sub-node-n {
+ pinmux = <PIN_ID_AND_MUX>, <PIN_ID_AND_MUX>, ... ;
+ GENERIC_PINCONFIG;
+ };
+ };
+
+ Use the latter two formats when pins part of the same logical group need to
+ have different generic pin configuration flags applied. Note that the generic
+ pinconfig in node-3 does not apply to the sub-nodes.
+
+ Client sub-nodes shall refer to pin multiplexing sub-nodes using the phandle
+ of the most external one.
+
+ Eg.
+
+ client-1 {
+ ...
+ pinctrl-0 = <&node-1>;
+ ...
+ };
+
+ client-2 {
+ ...
+ pinctrl-0 = <&node-2>;
+ ...
+ };
+
+ Required properties:
+ - pinmux:
+ integer array representing pin number and pin multiplexing configuration.
+ When a pin has to be configured in alternate function mode, use this
+ property to identify the pin by its global index, and provide its
+ alternate function configuration number along with it.
+ When multiple pins are required to be configured as part of the same
+ alternate function they shall be specified as members of the same
+ argument list of a single "pinmux" property.
+ Integers values in the "pinmux" argument list are assembled as:
+ (PIN | MUX_FUNC << 8)
+ where PIN directly corresponds to the pl_gpio pin number and MUX_FUNC is
+ one of the alternate function identifiers defined in:
+ <include/dt-bindings/pinctrl/rzn1-pinctrl.h>
+ These identifiers collapse the IO Multiplex Configuration Level 1 and
+ Level 2 numbers that are detailed in the hardware reference manual into a
+ single number. The identifiers for Level 2 are simply offset by 10.
+ Additional identifiers are provided to specify the MDIO source peripheral.
+
+ Optional generic pinconf properties:
+ - bias-disable - disable any pin bias
+ - bias-pull-up - pull up the pin with 50 KOhm
+ - bias-pull-down - pull down the pin with 50 KOhm
+ - bias-high-impedance - high impedance mode
+ - drive-strength - sink or source at most 4, 6, 8 or 12 mA
+
+ Example:
+ A serial communication interface with a TX output pin and an RX input pin.
+
+ &pinctrl {
+ pins_uart0: pins_uart0 {
+ pinmux = <
+ RZN1_PINMUX(103, RZN1_FUNC_UART0_I) /* UART0_TXD */
+ RZN1_PINMUX(104, RZN1_FUNC_UART0_I) /* UART0_RXD */
+ >;
+ };
+ };
+
+ Example 2:
+ Here we set the pull up on the RXD pin of the UART.
+
+ &pinctrl {
+ pins_uart0: pins_uart0 {
+ pinmux = <RZN1_PINMUX(103, RZN1_FUNC_UART0_I)>; /* TXD */
+
+ pins_uart6_rx {
+ pinmux = <RZN1_PINMUX(104, RZN1_FUNC_UART0_I)>; /* RXD */
+ bias-pull-up;
+ };
+ };
+ };
disabled. This binding is a workaround to keep backward compatibility with
old dtb's which rely on the fact that the switched regulators are always on
and don't mark them explicit as "regulator-always-on".
+- fsl,pmic-stby-poweroff: if present, configure the PMIC to shutdown all
+ power rails when PMIC_STBY_REQ line is asserted during the power off sequence.
+ Use this option if the SoC should be powered off by external power
+ management IC (PMIC) on PMIC_STBY_REQ signal.
+ As opposite to PMIC_STBY_REQ boards can implement PMIC_ON_REQ signal.
Required child node:
- regulators: This is the list of child nodes that specify the regulator
"qcom,rpm-pm8998-regulators"
"qcom,rpm-pma8084-regulators"
"qcom,rpm-pmi8998-regulators"
+ "qcom,rpm-pms405-regulators"
- vdd_s1-supply:
- vdd_s2-supply:
Definition: reference to regulator supplying the input pin, as
described in the data sheet
+- vdd_s1-supply:
+- vdd_s2-supply:
+- vdd_s3-supply:
+- vdd_s4-supply:
+- vdd_s5-supply:
+- vdd_l1_l2-supply:
+- vdd_l3_l8-supply:
+- vdd_l4-supply:
+- vdd_l5_l6-supply:
+- vdd_l7-supply:
+- vdd_l3_l8-supply:
+- vdd_l9-supply:
+- vdd_l10_l11_l12_l13-supply:
+ Usage: optional (pms405 only)
+ Value type: <phandle>
+ Definition: reference to regulator supplying the input pin, as
+ described in the data sheet
+
The regulator node houses sub-nodes for each regulator within the device. Each
sub-node is identified using the node's name, with valid values listed for each
of the pmics below.
pmi8998:
bob
+pms405:
+ s1, s2, s3, s4, s5, l1, l2, l3, l4, l5, l6, l7, l8, l9, l10, l11, l12,
+ l13
+
The content of each sub-node is defined by the standard binding for regulators -
see regulator.txt.
-ROHM BD71837 Power Management Integrated Circuit (PMIC) regulator bindings
+ROHM BD71837 and BD71847 Power Management Integrated Circuit regulator bindings
Required properties:
- - regulator-name: should be "buck1", ..., "buck8" and "ldo1", ..., "ldo7"
+ - regulator-name: should be "buck1", ..., "buck8" and "ldo1", ..., "ldo7" for
+ BD71837. For BD71847 names should be "buck1", ..., "buck6"
+ and "ldo1", ..., "ldo6"
List of regulators provided by this controller. BD71837 regulators node
should be sub node of the BD71837 MFD node. See BD71837 MFD bindings at
if they are disabled at startup the voltage monitoring for LDO5/LDO6 will
cause PMIC to reset.
-The valid names for regulator nodes are:
+The valid names for BD71837 regulator nodes are:
BUCK1, BUCK2, BUCK3, BUCK4, BUCK5, BUCK6, BUCK7, BUCK8
LDO1, LDO2, LDO3, LDO4, LDO5, LDO6, LDO7
+The valid names for BD71847 regulator nodes are:
+BUCK1, BUCK2, BUCK3, BUCK4, BUCK5, BUCK6
+LDO1, LDO2, LDO3, LDO4, LDO5, LDO6
+
Optional properties:
- Any optional property defined in bindings/regulator/regulator.txt
--- /dev/null
+STMicroelectronics STPMIC1 Voltage regulators
+
+Regulator Nodes are optional depending on needs.
+
+Available Regulators in STPMIC1 device are:
+ - buck1 for Buck BUCK1
+ - buck2 for Buck BUCK2
+ - buck3 for Buck BUCK3
+ - buck4 for Buck BUCK4
+ - ldo1 for LDO LDO1
+ - ldo2 for LDO LDO2
+ - ldo3 for LDO LDO3
+ - ldo4 for LDO LDO4
+ - ldo5 for LDO LDO5
+ - ldo6 for LDO LDO6
+ - vref_ddr for LDO Vref DDR
+ - boost for Buck BOOST
+ - pwr_sw1 for VBUS_OTG switch
+ - pwr_sw2 for SW_OUT switch
+
+Switches are fixed voltage regulators with only enable/disable capability.
+
+Optional properties:
+- st,mask-reset: mask reset for this regulator: the regulator configuration
+ is maintained during pmic reset.
+- regulator-pull-down: enable high pull down
+ if not specified light pull down is used
+- regulator-over-current-protection:
+ if set, all regulators are switched off in case of over-current detection
+ on this regulator,
+ if not set, the driver only sends an over-current event.
+- interrupt-parent: phandle to the parent interrupt controller
+- interrupts: index of current limit detection interrupt
+- <regulator>-supply: phandle to the parent supply/regulator node
+ each regulator supply can be described except vref_ddr.
+
+Example:
+regulators {
+ compatible = "st,stpmic1-regulators";
+
+ ldo6-supply = <&v3v3>;
+
+ vdd_core: buck1 {
+ regulator-name = "vdd_core";
+ interrupts = <IT_CURLIM_BUCK1 0>;
+ interrupt-parent = <&pmic>;
+ st,mask-reset;
+ regulator-pull-down;
+ regulator-min-microvolt = <700000>;
+ regulator-max-microvolt = <1200000>;
+ };
+
+ v3v3: buck4 {
+ regulator-name = "v3v3";
+ interrupts = <IT_CURLIM_BUCK4 0>;
+ interrupt-parent = <&mypmic>;
+
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ v1v8: ldo6 {
+ regulator-name = "v1v8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ regulator-over-current-protection;
+ };
+};
- clocks: Serial engine core clock needed by the device.
Qualcomm Technologies Inc. GENI Serial Engine based SPI Controller
-
-Required properties:
-- compatible: Must contain "qcom,geni-spi".
-- reg: Must contain SPI register location and length.
-- interrupts: Must contain SPI controller interrupts.
-- clock-names: Must contain "se".
-- clocks: Serial engine core clock needed by the device.
-- spi-max-frequency: Specifies maximum SPI clock frequency, units - Hz.
-- #address-cells: Must be <1> to define a chip select address on
- the SPI bus.
-- #size-cells: Must be <0>.
-
-SPI slave nodes must be children of the SPI master node and conform to SPI bus
-binding as described in Documentation/devicetree/bindings/spi/spi-bus.txt.
+node binding is described in
+Documentation/devicetree/bindings/spi/qcom,spi-geni-qcom.txt.
Example:
geniqup@8c0000 {
pinctrl-1 = <&qup_1_uart_3_sleep>;
};
- spi0: spi@a84000 {
- compatible = "qcom,geni-spi";
- reg = <0xa84000 0x4000>;
- interrupts = <GIC_SPI 354 IRQ_TYPE_LEVEL_HIGH>;
- clock-names = "se";
- clocks = <&clock_gcc GCC_QUPV3_WRAP0_S0_CLK>;
- pinctrl-names = "default", "sleep";
- pinctrl-0 = <&qup_1_spi_2_active>;
- pinctrl-1 = <&qup_1_spi_2_sleep>;
- spi-max-frequency = <19200000>;
- #address-cells = <1>;
- #size-cells = <0>;
- };
}
--- /dev/null
+GENI based Qualcomm Universal Peripheral (QUP) Serial Peripheral Interface (SPI)
+
+The QUP v3 core is a GENI based AHB slave that provides a common data path
+(an output FIFO and an input FIFO) for serial peripheral interface (SPI)
+mini-core.
+
+SPI in master mode supports up to 50MHz, up to four chip selects, programmable
+data path from 4 bits to 32 bits and numerous protocol variants.
+
+Required properties:
+- compatible: Must contain "qcom,geni-spi".
+- reg: Must contain SPI register location and length.
+- interrupts: Must contain SPI controller interrupts.
+- clock-names: Must contain "se".
+- clocks: Serial engine core clock needed by the device.
+- #address-cells: Must be <1> to define a chip select address on
+ the SPI bus.
+- #size-cells: Must be <0>.
+
+SPI Controller nodes must be child of GENI based Qualcomm Universal
+Peripharal. Please refer GENI based QUP wrapper controller node bindings
+described in Documentation/devicetree/bindings/soc/qcom/qcom,geni-se.txt.
+
+SPI slave nodes must be children of the SPI master node and conform to SPI bus
+binding as described in Documentation/devicetree/bindings/spi/spi-bus.txt.
+
+Example:
+ spi0: spi@a84000 {
+ compatible = "qcom,geni-spi";
+ reg = <0xa84000 0x4000>;
+ interrupts = <GIC_SPI 354 IRQ_TYPE_LEVEL_HIGH>;
+ clock-names = "se";
+ clocks = <&clock_gcc GCC_QUPV3_WRAP0_S0_CLK>;
+ pinctrl-names = "default", "sleep";
+ pinctrl-0 = <&qup_1_spi_2_active>;
+ pinctrl-1 = <&qup_1_spi_2_sleep>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ };
--- /dev/null
+Qualcomm Quad Serial Peripheral Interface (QSPI)
+
+The QSPI controller allows SPI protocol communication in single, dual, or quad
+wire transmission modes for read/write access to slaves such as NOR flash.
+
+Required properties:
+- compatible: An SoC specific identifier followed by "qcom,qspi-v1", such as
+ "qcom,sdm845-qspi", "qcom,qspi-v1"
+- reg: Should contain the base register location and length.
+- interrupts: Interrupt number used by the controller.
+- clocks: Should contain the core and AHB clock.
+- clock-names: Should be "core" for core clock and "iface" for AHB clock.
+
+SPI slave nodes must be children of the SPI master node and can contain
+properties described in Documentation/devicetree/bindings/spi/spi-bus.txt
+
+Example:
+
+ qspi: spi@88df000 {
+ compatible = "qcom,sdm845-qspi", "qcom,qspi-v1";
+ reg = <0x88df000 0x600>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ interrupts = <GIC_SPI 82 IRQ_TYPE_LEVEL_HIGH>;
+ clock-names = "iface", "core";
+ clocks = <&gcc GCC_QSPI_CNOC_PERIPH_AHB_CLK>,
+ <&gcc GCC_QSPI_CORE_CLK>;
+
+ flash@0 {
+ compatible = "jedec,spi-nor";
+ reg = <0>;
+ spi-max-frequency = <25000000>;
+ spi-tx-bus-width = <2>;
+ spi-rx-bus-width = <2>;
+ };
+ };
Required properties:
- compatible : "renesas,msiof-r8a7743" (RZ/G1M)
+ "renesas,msiof-r8a7744" (RZ/G1N)
"renesas,msiof-r8a7745" (RZ/G1E)
+ "renesas,msiof-r8a774a1" (RZ/G2M)
"renesas,msiof-r8a7790" (R-Car H2)
"renesas,msiof-r8a7791" (R-Car M2-W)
"renesas,msiof-r8a7792" (R-Car V2H)
"renesas,msiof-r8a7795" (R-Car H3)
"renesas,msiof-r8a7796" (R-Car M3-W)
"renesas,msiof-r8a77965" (R-Car M3-N)
+ "renesas,msiof-r8a77970" (R-Car V3M)
+ "renesas,msiof-r8a77980" (R-Car V3H)
+ "renesas,msiof-r8a77990" (R-Car E3)
+ "renesas,msiof-r8a77995" (R-Car D3)
"renesas,msiof-sh73a0" (SH-Mobile AG5)
"renesas,sh-mobile-msiof" (generic SH-Mobile compatibile device)
"renesas,rcar-gen2-msiof" (generic R-Car Gen2 and RZ/G1 compatible device)
- "renesas,rcar-gen3-msiof" (generic R-Car Gen3 compatible device)
+ "renesas,rcar-gen3-msiof" (generic R-Car Gen3 and RZ/G2 compatible device)
"renesas,sh-msiof" (deprecated)
When compatible with the generic version, nodes
Required properties:
- compatible : "snps,dw-apb-ssi" or "mscc,<soc>-spi", where soc is "ocelot" or
- "jaguar2"
+ "jaguar2", or "amazon,alpine-dw-apb-ssi"
- reg : The register base for the controller. For "mscc,<soc>-spi", a second
register set is required (named ICPU_CFG:SPI_MST)
- interrupts : One interrupt, used by the controller.
Required properties:
- compatible :
- "fsl,imx7ulp-spi" for LPSPI compatible with the one integrated on i.MX7ULP soc
+ - "fsl,imx8qxp-spi" for LPSPI compatible with the one integrated on i.MX8QXP soc
- reg : address and length of the lpspi master registers
- interrupts : lpspi interrupt
- clocks : lpspi clock specifier
--- /dev/null
+PXA2xx SSP SPI Controller
+
+Required properties:
+- compatible: Must be "marvell,mmp2-ssp".
+- reg: Offset and length of the device's register set.
+- interrupts: Should be the interrupt number.
+- clocks: Should contain a single entry describing the clock input.
+- #address-cells: Number of cells required to define a chip select address.
+- #size-cells: Should be zero.
+
+Optional properties:
+- cs-gpios: list of GPIO chip selects. See the SPI bus bindings,
+ Documentation/devicetree/bindings/spi/spi-bus.txt
+
+Child nodes represent devices on the SPI bus
+ See ../spi/spi-bus.txt
+
+Example:
+ ssp1: spi@d4035000 {
+ compatible = "marvell,mmp2-ssp";
+ reg = <0xd4035000 0x1000>;
+ clocks = <&soc_clocks MMP2_CLK_SSP0>;
+ interrupts = <0>;
+ };
Required properties:
- compatible : For Renesas Serial Peripheral Interface on legacy SH:
"renesas,rspi-<soctype>", "renesas,rspi" as fallback.
- For Renesas Serial Peripheral Interface on RZ/A1H:
+ For Renesas Serial Peripheral Interface on RZ/A:
"renesas,rspi-<soctype>", "renesas,rspi-rz" as fallback.
For Quad Serial Peripheral Interface on R-Car Gen2 and
RZ/G1 devices:
Examples with soctypes are:
- "renesas,rspi-sh7757" (SH)
- "renesas,rspi-r7s72100" (RZ/A1H)
+ - "renesas,rspi-r7s9210" (RZ/A2)
- "renesas,qspi-r8a7743" (RZ/G1M)
+ - "renesas,qspi-r8a7744" (RZ/G1N)
- "renesas,qspi-r8a7745" (RZ/G1E)
- "renesas,qspi-r8a7790" (R-Car H2)
- "renesas,qspi-r8a7791" (R-Car M2-W)
--- /dev/null
+Binding for MTK SPI Slave controller
+
+Required properties:
+- compatible: should be one of the following.
+ - mediatek,mt2712-spi-slave: for mt2712 platforms
+- reg: Address and length of the register set for the device.
+- interrupts: Should contain spi interrupt.
+- clocks: phandles to input clocks.
+ It's clock gate, and should be <&infracfg CLK_INFRA_AO_SPI1>.
+- clock-names: should be "spi" for the clock gate.
+
+Optional properties:
+- assigned-clocks: it's mux clock, should be <&topckgen CLK_TOP_SPISLV_SEL>.
+- assigned-clock-parents: parent of mux clock.
+ It's PLL, and should be one of the following.
+ - <&topckgen CLK_TOP_UNIVPLL1_D2>: specify parent clock 312MHZ.
+ It's the default one.
+ - <&topckgen CLK_TOP_UNIVPLL1_D4>: specify parent clock 156MHZ.
+ - <&topckgen CLK_TOP_UNIVPLL2_D4>: specify parent clock 104MHZ.
+ - <&topckgen CLK_TOP_UNIVPLL1_D8>: specify parent clock 78MHZ.
+
+Example:
+- SoC Specific Portion:
+spis1: spi@10013000 {
+ compatible = "mediatek,mt2712-spi-slave";
+ reg = <0 0x10013000 0 0x100>;
+ interrupts = <GIC_SPI 283 IRQ_TYPE_LEVEL_LOW>;
+ clocks = <&infracfg CLK_INFRA_AO_SPI1>;
+ clock-names = "spi";
+ assigned-clocks = <&topckgen CLK_TOP_SPISLV_SEL>;
+ assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL1_D2>;
+};
--- /dev/null
+Spreadtrum SPI Controller
+
+Required properties:
+- compatible: Should be "sprd,sc9860-spi".
+- reg: Offset and length of SPI controller register space.
+- interrupts: Should contain SPI interrupt.
+- clock-names: Should contain following entries:
+ "spi" for SPI clock,
+ "source" for SPI source (parent) clock,
+ "enable" for SPI module enable clock.
+- clocks: List of clock input name strings sorted in the same order
+ as the clock-names property.
+- #address-cells: The number of cells required to define a chip select
+ address on the SPI bus. Should be set to 1.
+- #size-cells: Should be set to 0.
+
+Example:
+spi0: spi@70a00000{
+ compatible = "sprd,sc9860-spi";
+ reg = <0 0x70a00000 0 0x1000>;
+ interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>;
+ clock-names = "spi", "source","enable";
+ clocks = <&clk_spi0>, <&ext_26m>, <&clk_ap_apb_gates 5>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+};
--- /dev/null
+* STMicroelectronics Quad Serial Peripheral Interface(QSPI)
+
+Required properties:
+- compatible: should be "st,stm32f469-qspi"
+- reg: the first contains the register location and length.
+ the second contains the memory mapping address and length
+- reg-names: should contain the reg names "qspi" "qspi_mm"
+- interrupts: should contain the interrupt for the device
+- clocks: the phandle of the clock needed by the QSPI controller
+- A pinctrl must be defined to set pins in mode of operation for QSPI transfer
+
+Optional properties:
+- resets: must contain the phandle to the reset controller.
+
+A spi flash (NOR/NAND) must be a child of spi node and could have some
+properties. Also see jedec,spi-nor.txt.
+
+Required properties:
+- reg: chip-Select number (QSPI controller may connect 2 flashes)
+- spi-max-frequency: max frequency of spi bus
+
+Optional property:
+- spi-rx-bus-width: see ./spi-bus.txt for the description
+
+Example:
+
+qspi: spi@a0001000 {
+ compatible = "st,stm32f469-qspi";
+ reg = <0xa0001000 0x1000>, <0x90000000 0x10000000>;
+ reg-names = "qspi", "qspi_mm";
+ interrupts = <91>;
+ resets = <&rcc STM32F4_AHB3_RESET(QSPI)>;
+ clocks = <&rcc 0 STM32F4_AHB3_CLOCK(QSPI)>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_qspi0>;
+
+ flash@0 {
+ compatible = "jedec,spi-nor";
+ reg = <0>;
+ spi-rx-bus-width = <4>;
+ spi-max-frequency = <108000000>;
+ ...
+ };
+};
The line will be hogged as soon as the gpiochip is created or - in case the
chip was created earlier - when the hog table is registered.
+
+Arrays of pins
+--------------
+In addition to requesting pins belonging to a function one by one, a device may
+also request an array of pins assigned to the function. The way those pins are
+mapped to the device determines if the array qualifies for fast bitmap
+processing. If yes, a bitmap is passed over get/set array functions directly
+between a caller and a respective .get/set_multiple() callback of a GPIO chip.
+
+In order to qualify for fast bitmap processing, the array must meet the
+following requirements:
+- pin hardware number of array member 0 must also be 0,
+- pin hardware numbers of consecutive array members which belong to the same
+ chip as member 0 does must also match their array indexes.
+
+Otherwise fast bitmap processing path is not used in order to avoid consecutive
+pins which belong to the same chip but are not in hardware order being processed
+separately.
+
+If the array applies for fast bitmap processing path, pins which belong to
+different chips than member 0 does, as well as those with indexes different from
+their hardware pin numbers, are excluded from the fast path, both input and
+output. Moreover, open drain and open source pins are excluded from fast bitmap
+output processing.
enum gpiod_flags flags)
This function returns a struct gpio_descs which contains an array of
-descriptors::
+descriptors. It also contains a pointer to a gpiolib private structure which,
+if passed back to get/set array functions, may speed up I/O proocessing::
struct gpio_descs {
+ struct gpio_array *info;
unsigned int ndescs;
struct gpio_desc *desc[];
}
int gpiod_get_array_value(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array);
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
int gpiod_get_raw_array_value(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array);
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
int gpiod_get_array_value_cansleep(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array);
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
int gpiod_get_raw_array_value_cansleep(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array);
-
- void gpiod_set_array_value(unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array)
- void gpiod_set_raw_array_value(unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array)
- void gpiod_set_array_value_cansleep(unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array)
- void gpiod_set_raw_array_value_cansleep(unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array)
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
+
+ int gpiod_set_array_value(unsigned int array_size,
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
+ int gpiod_set_raw_array_value(unsigned int array_size,
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
+ int gpiod_set_array_value_cansleep(unsigned int array_size,
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
+ int gpiod_set_raw_array_value_cansleep(unsigned int array_size,
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
The array can be an arbitrary set of GPIOs. The functions will try to access
GPIOs belonging to the same bank or chip simultaneously if supported by the
The functions take three arguments:
* array_size - the number of array elements
* desc_array - an array of GPIO descriptors
- * value_array - an array to store the GPIOs' values (get) or
- an array of values to assign to the GPIOs (set)
+ * array_info - optional information obtained from gpiod_array_get()
+ * value_bitmap - a bitmap to store the GPIOs' values (get) or
+ a bitmap of values to assign to the GPIOs (set)
The descriptor array can be obtained using the gpiod_get_array() function
or one of its variants. If the group of descriptors returned by that function
struct gpio_descs *my_gpio_descs = gpiod_get_array(...);
gpiod_set_array_value(my_gpio_descs->ndescs, my_gpio_descs->desc,
- my_gpio_values);
+ my_gpio_descs->info, my_gpio_value_bitmap);
It is also possible to access a completely arbitrary array of descriptors. The
descriptors may be obtained using any combination of gpiod_get() and
gpiod_get_array(). Afterwards the array of descriptors has to be setup
-manually before it can be passed to one of the above functions.
+manually before it can be passed to one of the above functions. In that case,
+array_info should be set to NULL.
Note that for optimal performance GPIOs belonging to the same chip should be
contiguous within the array of descriptors.
+Still better performance may be achieved if array indexes of the descriptors
+match hardware pin numbers of a single chip. If an array passed to a get/set
+array function matches the one obtained from gpiod_get_array() and array_info
+associated with the array is also passed, the function may take a fast bitmap
+processing path, passing the value_bitmap argument directly to the respective
+.get/set_multiple() callback of the chip. That allows for utilization of GPIO
+banks as data I/O ports without much loss of performance.
+
The return value of gpiod_get_array_value() and its variants is 0 on success
or negative on error. Note the difference to gpiod_get_value(), which returns
0 or 1 on success to convey the GPIO value. With the array functions, the GPIO
typically be called in the .startup() and .shutdown() callbacks from the
irqchip.
-When using the gpiolib irqchip helpers, these callback are automatically
+When using the gpiolib irqchip helpers, these callbacks are automatically
+assigned.
+
+
+Disabling and enabling IRQs
+---------------------------
+When a GPIO is used as an IRQ signal, then gpiolib also needs to know if
+the IRQ is enabled or disabled. In order to inform gpiolib about this,
+a driver should call::
+
+ void gpiochip_disable_irq(struct gpio_chip *chip, unsigned int offset)
+
+This allows drivers to drive the GPIO as an output while the IRQ is
+disabled. When the IRQ is enabled again, a driver should call::
+
+ void gpiochip_enable_irq(struct gpio_chip *chip, unsigned int offset)
+
+When implementing an irqchip inside a GPIO driver, these two functions should
+typically be called in the .irq_disable() and .irq_enable() callbacks from the
+irqchip.
+
+When using the gpiolib irqchip helpers, these callbacks are automatically
assigned.
Real-Time compliance for GPIO IRQ chips
Device-managed API
==================
-.. kernel-doc:: drivers/gpio/devres.c
+.. kernel-doc:: drivers/gpio/gpiolib-devres.c
:export:
sysfs helpers
{
struct nand_chip *this = mtd_to_nand(mtd);
switch(cmd){
- case NAND_CTL_SETCLE: this->IO_ADDR_W |= CLE_ADRR_BIT; break;
- case NAND_CTL_CLRCLE: this->IO_ADDR_W &= ~CLE_ADRR_BIT; break;
- case NAND_CTL_SETALE: this->IO_ADDR_W |= ALE_ADRR_BIT; break;
- case NAND_CTL_CLRALE: this->IO_ADDR_W &= ~ALE_ADRR_BIT; break;
+ case NAND_CTL_SETCLE: this->legacy.IO_ADDR_W |= CLE_ADRR_BIT; break;
+ case NAND_CTL_CLRCLE: this->legacy.IO_ADDR_W &= ~CLE_ADRR_BIT; break;
+ case NAND_CTL_SETALE: this->legacy.IO_ADDR_W |= ALE_ADRR_BIT; break;
+ case NAND_CTL_CLRALE: this->legacy.IO_ADDR_W &= ~ALE_ADRR_BIT; break;
}
}
should return 0, if the device is busy (R/B pin is low) and 1, if the
device is ready (R/B pin is high). If the hardware interface does not
give access to the ready busy pin, then the function must not be defined
-and the function pointer this->dev_ready is set to NULL.
+and the function pointer this->legacy.dev_ready is set to NULL.
Init function
-------------
}
/* Set address of NAND IO lines */
- this->IO_ADDR_R = baseaddr;
- this->IO_ADDR_W = baseaddr;
+ this->legacy.IO_ADDR_R = baseaddr;
+ this->legacy.IO_ADDR_W = baseaddr;
/* Reference hardware control function */
this->hwcontrol = board_hwcontrol;
/* Set command delay time, see datasheet for correct value */
- this->chip_delay = CHIP_DEPENDEND_COMMAND_DELAY;
+ this->legacy.chip_delay = CHIP_DEPENDEND_COMMAND_DELAY;
/* Assign the device ready function, if available */
- this->dev_ready = board_dev_ready;
+ this->legacy.dev_ready = board_dev_ready;
this->eccmode = NAND_ECC_SOFT;
/* Scan to find existence of the device */
- if (nand_scan (board_mtd, 1)) {
+ if (nand_scan (this, 1)) {
err = -ENXIO;
goto out_ior;
}
static void __exit board_cleanup (void)
{
/* Release resources, unregister device */
- nand_release (board_mtd);
+ nand_release (mtd_to_nand(board_mtd));
/* unmap physical address */
iounmap(baseaddr);
struct nand_chip *this = mtd_to_nand(mtd);
/* Deselect all chips */
- this->IO_ADDR_R &= ~BOARD_NAND_ADDR_MASK;
- this->IO_ADDR_W &= ~BOARD_NAND_ADDR_MASK;
+ this->legacy.IO_ADDR_R &= ~BOARD_NAND_ADDR_MASK;
+ this->legacy.IO_ADDR_W &= ~BOARD_NAND_ADDR_MASK;
switch (chip) {
case 0:
- this->IO_ADDR_R |= BOARD_NAND_ADDR_CHIP0;
- this->IO_ADDR_W |= BOARD_NAND_ADDR_CHIP0;
+ this->legacy.IO_ADDR_R |= BOARD_NAND_ADDR_CHIP0;
+ this->legacy.IO_ADDR_W |= BOARD_NAND_ADDR_CHIP0;
break;
....
case n:
- this->IO_ADDR_R |= BOARD_NAND_ADDR_CHIPn;
- this->IO_ADDR_W |= BOARD_NAND_ADDR_CHIPn;
+ this->legacy.IO_ADDR_R |= BOARD_NAND_ADDR_CHIPn;
+ this->legacy.IO_ADDR_W |= BOARD_NAND_ADDR_CHIPn;
break;
}
}
Sysfs entries
-------------
+in[123]_label Voltage channel labels
+in[123]_enable Voltage channel enable controls
in[123]_input Bus voltage(mV) channels
curr[123]_input Current(mA) measurement channels
shunt[123]_resistor Shunt resistance(uOhm) channels
Addresses scanned: none
Datasheet: Publicly available at the Maxim website
http://www.maximintegrated.com/
- * Maxim MAX6625, MAX6626
- Prefixes: 'max6625', 'max6626'
+ * Maxim MAX6625, MAX6626, MAX31725, MAX31726
+ Prefixes: 'max6625', 'max6626', 'max31725', 'max31726'
Addresses scanned: none
Datasheet: Publicly available at the Maxim website
http://www.maxim-ic.com/
LM75 clones not listed here, with or without various enhancements,
that are supported. The clones are not detected by the driver, unless
they reproduce the exact register tricks of the original LM75, and must
-therefore be instantiated explicitly. Higher resolution up to 12-bit
+therefore be instantiated explicitly. Higher resolution up to 16-bit
is supported by this driver, other specific enhancements are not.
The LM77 is not supported, contrary to what we pretended for a long time.
Prefix: 'ltm4676'
Addresses scanned: -
Datasheet: http://www.linear.com/product/ltm4676
+ * Analog Devices LTM4686
+ Prefix: 'ltm4686'
+ Addresses scanned: -
+ Datasheet: http://www.analog.com/ltm4686
Author: Guenter Roeck <linux@roeck-us.net>
as two separate chips on two different I2C bus addresses.
LTM4675 is a dual 9A or single 18A μModule regulator
LTM4676 is a dual 13A or single 26A uModule regulator.
+LTM4686 is a dual 10A or single 20A uModule regulator.
Usage Notes
=========================
Supported chips:
- * Freescale Atlas MC13783
+ * Freescale MC13783
Prefix: 'mc13783'
- Datasheet: http://www.freescale.com/files/rf_if/doc/data_sheet/MC13783.pdf?fsrch=1
- * Freescale Atlas MC13892
+ Datasheet: https://www.nxp.com/docs/en/data-sheet/MC13783.pdf
+ * Freescale MC13892
Prefix: 'mc13892'
- Datasheet: http://cache.freescale.com/files/analog/doc/data_sheet/MC13892.pdf?fsrch=1&sr=1
+ Datasheet: https://www.nxp.com/docs/en/data-sheet/MC13892.pdf
Authors:
Sascha Hauer <s.hauer@pengutronix.de>
to do.
Name:
- rcuob/%d, rcuop/%d, and rcuos/%d
+ rcuop/%d and rcuos/%d
Purpose:
Offload RCU callbacks from the corresponding CPU.
07 &mm->mmap_sem-R: 37 100 1.31 299502.61 325629.52 3256.30 212344 34316685 0.10 7744.91 95016910.20 2.77
08 ---------------
09 &mm->mmap_sem 1 [<ffffffff811502a7>] khugepaged_scan_mm_slot+0x57/0x280
-19 &mm->mmap_sem 96 [<ffffffff815351c4>] __do_page_fault+0x1d4/0x510
+10 &mm->mmap_sem 96 [<ffffffff815351c4>] __do_page_fault+0x1d4/0x510
11 &mm->mmap_sem 34 [<ffffffff81113d77>] vm_mmap_pgoff+0x87/0xd0
12 &mm->mmap_sem 17 [<ffffffff81127e71>] vm_munmap+0x41/0x80
13 ---------------
operations after the ACQUIRE operation will appear to happen after the
ACQUIRE operation with respect to the other components of the system.
ACQUIRE operations include LOCK operations and both smp_load_acquire()
- and smp_cond_acquire() operations. The later builds the necessary ACQUIRE
- semantics from relying on a control dependency and smp_rmb().
+ and smp_cond_load_acquire() operations.
Memory operations that occur before an ACQUIRE operation may appear to
happen after it completes.
+++ /dev/null
-
-About this document
-===================
-
-Some notes about Marvell's NAND controller available in PXA and Armada 370/XP
-SoC (aka NFCv1 and NFCv2), with an emphasis on the latter.
-
-NFCv2 controller background
-===========================
-
-The controller has a 2176 bytes FIFO buffer. Therefore, in order to support
-larger pages, I/O operations on 4 KiB and 8 KiB pages is done with a set of
-chunked transfers.
-
-For instance, if we choose a 2048 data chunk and set "BCH" ECC (see below)
-we'll have this layout in the pages:
-
- ------------------------------------------------------------------------------
- | 2048B data | 32B spare | 30B ECC || 2048B data | 32B spare | 30B ECC | ... |
- ------------------------------------------------------------------------------
-
-The driver reads the data and spare portions independently and builds an internal
-buffer with this layout (in the 4 KiB page case):
-
- ------------------------------------------
- | 4096B data | 64B spare |
- ------------------------------------------
-
-Also, for the READOOB command the driver disables the ECC and reads a 'spare + ECC'
-OOB, one per chunk read.
-
- -------------------------------------------------------------------
- | 4096B data | 32B spare | 30B ECC | 32B spare | 30B ECC |
- -------------------------------------------------------------------
-
-So, in order to achieve reading (for instance), we issue several READ0 commands
-(with some additional controller-specific magic) and read two chunks of 2080B
-(2048 data + 32 spare) each.
-The driver accommodates this data to expose the NAND core a contiguous buffer
-(4096 data + spare) or (4096 + spare + ECC + spare + ECC).
-
-ECC
-===
-
-The controller has built-in hardware ECC capabilities. In addition it is
-configurable between two modes: 1) Hamming, 2) BCH.
-
-Note that the actual BCH mode: BCH-4 or BCH-8 will depend on the way
-the controller is configured to transfer the data.
-
-In the BCH mode the ECC code will be calculated for each transferred chunk
-and expected to be located (when reading/programming) right after the spare
-bytes as the figure above shows.
-
-So, repeating the above scheme, a 2048B data chunk will be followed by 32B
-spare, and then the ECC controller will read/write the ECC code (30B in
-this case):
-
- ------------------------------------
- | 2048B data | 32B spare | 30B ECC |
- ------------------------------------
-
-If the ECC mode is 'BCH' then the ECC is *always* 30 bytes long.
-If the ECC mode is 'Hamming' the ECC is 6 bytes long, for each 512B block.
-So in Hamming mode, a 2048B page will have a 24B ECC.
-
-Despite all of the above, the controller requires the driver to only read or
-write in multiples of 8-bytes, because the data buffer is 64-bits.
-
-OOB
-===
-
-Because of the above scheme, and because the "spare" OOB is really located in
-the middle of a page, spare OOB cannot be read or write independently of the
-data area. In other words, in order to read the OOB (aka READOOB), the entire
-page (aka READ0) has to be read.
-
-In the same sense, in order to write to the spare OOB the driver has to write
-an *entire* page.
-
-Factory bad blocks handling
-===========================
-
-Given the ECC BCH requires to layout the device's pages in a split
-data/OOB/data/OOB way, the controller has a view of the flash page that's
-different from the specified (aka the manufacturer's) view. In other words,
-
-Factory view:
-
- -----------------------------------------------
- | Data |x OOB |
- -----------------------------------------------
-
-Driver's view:
-
- -----------------------------------------------
- | Data | OOB | Data x | OOB |
- -----------------------------------------------
-
-It can be seen from the above, that the factory bad block marker must be
-searched within the 'data' region, and not in the usual OOB region.
-
-In addition, this means under regular usage the driver will write such
-position (since it belongs to the data region) and every used block is
-likely to be marked as bad.
-
-For this reason, marking the block as bad in the OOB is explicitly
-disabled by using the NAND_BBT_NO_OOB_BBM option in the driver. The rationale
-for this is that there's no point in marking a block as bad, because good
-blocks are also 'marked as bad' (in the OOB BBM sense) under normal usage.
-
-Instead, the driver relies on the bad block table alone, and should only perform
-the bad block scan on the very first time (when the device hasn't been used).
echo N > /sys/power/image_size
-before suspend (it is limited to 500 MB by default).
+before suspend (it is limited to around 2/5 of available RAM by default).
. The resume process checks for the presence of the resume device,
if found, it then checks the contents for the hibernation image signature.
--- /dev/null
+.. _code_of_conduct_interpretation:
+
+Linux Kernel Contributor Covenant Code of Conduct Interpretation
+================================================================
+
+The :ref:`code_of_conduct` is a general document meant to
+provide a set of rules for almost any open source community. Every
+open-source community is unique and the Linux kernel is no exception.
+Because of this, this document describes how we in the Linux kernel
+community will interpret it. We also do not expect this interpretation
+to be static over time, and will adjust it as needed.
+
+The Linux kernel development effort is a very personal process compared
+to "traditional" ways of developing software. Your contributions and
+ideas behind them will be carefully reviewed, often resulting in
+critique and criticism. The review will almost always require
+improvements before the material can be included in the
+kernel. Know that this happens because everyone involved wants to see
+the best possible solution for the overall success of Linux. This
+development process has been proven to create the most robust operating
+system kernel ever, and we do not want to do anything to cause the
+quality of submission and eventual result to ever decrease.
+
+Maintainers
+-----------
+
+The Code of Conduct uses the term "maintainers" numerous times. In the
+kernel community, a "maintainer" is anyone who is responsible for a
+subsystem, driver, or file, and is listed in the MAINTAINERS file in the
+kernel source tree.
+
+Responsibilities
+----------------
+
+The Code of Conduct mentions rights and responsibilities for
+maintainers, and this needs some further clarifications.
+
+First and foremost, it is a reasonable expectation to have maintainers
+lead by example.
+
+That being said, our community is vast and broad, and there is no new
+requirement for maintainers to unilaterally handle how other people
+behave in the parts of the community where they are active. That
+responsibility is upon all of us, and ultimately the Code of Conduct
+documents final escalation paths in case of unresolved concerns
+regarding conduct issues.
+
+Maintainers should be willing to help when problems occur, and work with
+others in the community when needed. Do not be afraid to reach out to
+the Technical Advisory Board (TAB) or other maintainers if you're
+uncertain how to handle situations that come up. It will not be
+considered a violation report unless you want it to be. If you are
+uncertain about approaching the TAB or any other maintainers, please
+reach out to our conflict mediator, Mishi Choudhary <mishi@linux.com>.
+
+In the end, "be kind to each other" is really what the end goal is for
+everybody. We know everyone is human and we all fail at times, but the
+primary goal for all of us should be to work toward amicable resolutions
+of problems. Enforcement of the code of conduct will only be a last
+resort option.
+
+Our goal of creating a robust and technically advanced operating system
+and the technical complexity involved naturally require expertise and
+decision-making.
+
+The required expertise varies depending on the area of contribution. It
+is determined mainly by context and technical complexity and only
+secondary by the expectations of contributors and maintainers.
+
+Both the expertise expectations and decision-making are subject to
+discussion, but at the very end there is a basic necessity to be able to
+make decisions in order to make progress. This prerogative is in the
+hands of maintainers and project's leadership and is expected to be used
+in good faith.
+
+As a consequence, setting expertise expectations, making decisions and
+rejecting unsuitable contributions are not viewed as a violation of the
+Code of Conduct.
+
+While maintainers are in general welcoming to newcomers, their capacity
+of helping contributors overcome the entry hurdles is limited, so they
+have to set priorities. This, also, is not to be seen as a violation of
+the Code of Conduct. The kernel community is aware of that and provides
+entry level programs in various forms like kernelnewbies.org.
+
+Scope
+-----
+
+The Linux kernel community primarily interacts on a set of public email
+lists distributed around a number of different servers controlled by a
+number of different companies or individuals. All of these lists are
+defined in the MAINTAINERS file in the kernel source tree. Any emails
+sent to those mailing lists are considered covered by the Code of
+Conduct.
+
+Developers who use the kernel.org bugzilla, and other subsystem bugzilla
+or bug tracking tools should follow the guidelines of the Code of
+Conduct. The Linux kernel community does not have an "official" project
+email address, or "official" social media address. Any activity
+performed using a kernel.org email account must follow the Code of
+Conduct as published for kernel.org, just as any individual using a
+corporate email account must follow the specific rules of that
+corporation.
+
+The Code of Conduct does not prohibit continuing to include names, email
+addresses, and associated comments in mailing list messages, kernel
+change log messages, or code comments.
+
+Interaction in other forums is covered by whatever rules apply to said
+forums and is in general not covered by the Code of Conduct. Exceptions
+may be considered for extreme circumstances.
+
+Contributions submitted for the kernel should use appropriate language.
+Content that already exists predating the Code of Conduct will not be
+addressed now as a violation. Inappropriate language can be seen as a
+bug, though; such bugs will be fixed more quickly if any interested
+parties submit patches to that effect. Expressions that are currently
+part of the user/kernel API, or reflect terminology used in published
+standards or specifications, are not considered bugs.
+
+Enforcement
+-----------
+
+The address listed in the Code of Conduct goes to the Code of Conduct
+Committee. The exact members receiving these emails at any given time
+are listed at https://kernel.org/code-of-conduct.html. Members can not
+access reports made before they joined or after they have left the
+committee.
+
+The initial Code of Conduct Committee consists of volunteer members of
+the TAB, as well as a professional mediator acting as a neutral third
+party. The first task of the committee is to establish documented
+processes, which will be made public.
+
+Any member of the committee, including the mediator, can be contacted
+directly if a reporter does not wish to include the full committee in a
+complaint or concern.
+
+The Code of Conduct Committee reviews the cases according to the
+processes (see above) and consults with the TAB as needed and
+appropriate, for instance to request and receive information about the
+kernel community.
+
+Any decisions by the committee will be brought to the TAB, for
+implementation of enforcement with the relevant maintainers if needed.
+A decision by the Code of Conduct Committee can be overturned by the TAB
+by a two-thirds vote.
+
+At quarterly intervals, the Code of Conduct Committee and TAB will
+provide a report summarizing the anonymised reports that the Code of
+Conduct committee has received and their status, as well details of any
+overridden decisions including complete and identifiable voting details.
+
+We expect to establish a different process for Code of Conduct Committee
+staffing beyond the bootstrap period. This document will be updated
+with that information when this occurs.
+.. _code_of_conduct:
+
Contributor Covenant Code of Conduct
++++++++++++++++++++++++++++++++++++
===========
Instances of abusive, harassing, or otherwise unacceptable behavior may be
-reported by contacting the Technical Advisory Board (TAB) at
-<tab@lists.linux-foundation.org>. All complaints will be reviewed and
-investigated and will result in a response that is deemed necessary and
-appropriate to the circumstances. The TAB is obligated to maintain
-confidentiality with regard to the reporter of an incident. Further details of
-specific enforcement policies may be posted separately.
-
-Maintainers who do not follow or enforce the Code of Conduct in good faith may
-face temporary or permanent repercussions as determined by other members of the
-project’s leadership.
+reported by contacting the Code of Conduct Committee at
+<conduct@kernel.org>. All complaints will be reviewed and investigated
+and will result in a response that is deemed necessary and appropriate
+to the circumstances. The Code of Conduct Committee is obligated to
+maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted
+separately.
Attribution
===========
This Code of Conduct is adapted from the Contributor Covenant, version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+Interpretation
+==============
+
+See the :ref:`code_of_conduct_interpretation` document for how the Linux
+kernel community will be interpreting this document.
howto
code-of-conduct
+ code-of-conduct-interpretation
development-process
submitting-patches
coding-style
-completions - wait for completion handling
-==========================================
-
-This document was originally written based on 3.18.0 (linux-next)
+Completions - "wait for completion" barrier APIs
+================================================
Introduction:
-------------
-If you have one or more threads of execution that must wait for some process
+If you have one or more threads that must wait for some kernel activity
to have reached a point or a specific state, completions can provide a
race-free solution to this problem. Semantically they are somewhat like a
-pthread_barrier and have similar use-cases.
+pthread_barrier() and have similar use-cases.
Completions are a code synchronization mechanism which is preferable to any
-misuse of locks. Any time you think of using yield() or some quirky
-msleep(1) loop to allow something else to proceed, you probably want to
-look into using one of the wait_for_completion*() calls instead. The
-advantage of using completions is clear intent of the code, but also more
-efficient code as both threads can continue until the result is actually
-needed.
-
-Completions are built on top of the generic event infrastructure in Linux,
-with the event reduced to a simple flag (appropriately called "done") in
-struct completion that tells the waiting threads of execution if they
-can continue safely.
-
-As completions are scheduling related, the code is found in
+misuse of locks/semaphores and busy-loops. Any time you think of using
+yield() or some quirky msleep(1) loop to allow something else to proceed,
+you probably want to look into using one of the wait_for_completion*()
+calls and complete() instead.
+
+The advantage of using completions is that they have a well defined, focused
+purpose which makes it very easy to see the intent of the code, but they
+also result in more efficient code as all threads can continue execution
+until the result is actually needed, and both the waiting and the signalling
+is highly efficient using low level scheduler sleep/wakeup facilities.
+
+Completions are built on top of the waitqueue and wakeup infrastructure of
+the Linux scheduler. The event the threads on the waitqueue are waiting for
+is reduced to a simple flag in 'struct completion', appropriately called "done".
+
+As completions are scheduling related, the code can be found in
kernel/sched/completion.c.
Usage:
------
-There are three parts to using completions, the initialization of the
-struct completion, the waiting part through a call to one of the variants of
-wait_for_completion() and the signaling side through a call to complete()
-or complete_all(). Further there are some helper functions for checking the
-state of completions.
+There are three main parts to using completions:
+
+ - the initialization of the 'struct completion' synchronization object
+ - the waiting part through a call to one of the variants of wait_for_completion(),
+ - the signaling side through a call to complete() or complete_all().
+
+There are also some helper functions for checking the state of completions.
+Note that while initialization must happen first, the waiting and signaling
+part can happen in any order. I.e. it's entirely normal for a thread
+to have marked a completion as 'done' before another thread checks whether
+it has to wait for it.
-To use completions one needs to include <linux/completion.h> and
-create a variable of type struct completion. The structure used for
-handling of completions is:
+To use completions you need to #include <linux/completion.h> and
+create a static or dynamic variable of type 'struct completion',
+which has only two fields:
struct completion {
unsigned int done;
wait_queue_head_t wait;
};
-providing the wait queue to place tasks on for waiting and the flag for
-indicating the state of affairs.
+This provides the ->wait waitqueue to place tasks on for waiting (if any), and
+the ->done completion flag for indicating whether it's completed or not.
-Completions should be named to convey the intent of the waiter. A good
-example is:
+Completions should be named to refer to the event that is being synchronized on.
+A good example is:
wait_for_completion(&early_console_added);
complete(&early_console_added);
-Good naming (as always) helps code readability.
+Good, intuitive naming (as always) helps code readability. Naming a completion
+'complete' is not helpful unless the purpose is super obvious...
Initializing completions:
-------------------------
-Initialization of dynamically allocated completions, often embedded in
-other structures, is done with:
+Dynamically allocated completion objects should preferably be embedded in data
+structures that are assured to be alive for the life-time of the function/driver,
+to prevent races with asynchronous complete() calls from occurring.
+
+Particular care should be taken when using the _timeout() or _killable()/_interruptible()
+variants of wait_for_completion(), as it must be assured that memory de-allocation
+does not happen until all related activities (complete() or reinit_completion())
+have taken place, even if these wait functions return prematurely due to a timeout
+or a signal triggering.
+
+Initializing of dynamically allocated completion objects is done via a call to
+init_completion():
- void init_completion(&done);
+ init_completion(&dynamic_object->done);
-Initialization is accomplished by initializing the wait queue and setting
-the default state to "not available", that is, "done" is set to 0.
+In this call we initialize the waitqueue and set ->done to 0, i.e. "not completed"
+or "not done".
The re-initialization function, reinit_completion(), simply resets the
-done element to "not available", thus again to 0, without touching the
-wait queue. Calling init_completion() twice on the same completion object is
+->done field to 0 ("not done"), without touching the waitqueue.
+Callers of this function must make sure that there are no racy
+wait_for_completion() calls going on in parallel.
+
+Calling init_completion() on the same completion object twice is
most likely a bug as it re-initializes the queue to an empty queue and
-enqueued tasks could get "lost" - use reinit_completion() in that case.
+enqueued tasks could get "lost" - use reinit_completion() in that case,
+but be aware of other races.
+
+For static declaration and initialization, macros are available.
+
+For static (or global) declarations in file scope you can use DECLARE_COMPLETION():
-For static declaration and initialization, macros are available. These are:
+ static DECLARE_COMPLETION(setup_done);
+ DECLARE_COMPLETION(setup_done);
- static DECLARE_COMPLETION(setup_done)
+Note that in this case the completion is boot time (or module load time)
+initialized to 'not done' and doesn't require an init_completion() call.
-used for static declarations in file scope. Within functions the static
-initialization should always use:
+When a completion is declared as a local variable within a function,
+then the initialization should always use DECLARE_COMPLETION_ONSTACK()
+explicitly, not just to make lockdep happy, but also to make it clear
+that limited scope had been considered and is intentional:
DECLARE_COMPLETION_ONSTACK(setup_done)
-suitable for automatic/local variables on the stack and will make lockdep
-happy. Note also that one needs to make *sure* the completion passed to
-work threads remains in-scope, and no references remain to on-stack data
-when the initiating function returns.
+Note that when using completion objects as local variables you must be
+acutely aware of the short life time of the function stack: the function
+must not return to a calling context until all activities (such as waiting
+threads) have ceased and the completion object is completely unused.
-Using on-stack completions for code that calls any of the _timeout or
-_interruptible/_killable variants is not advisable as they will require
-additional synchronization to prevent the on-stack completion object in
-the timeout/signal cases from going out of scope. Consider using dynamically
-allocated completions when intending to use the _interruptible/_killable
-or _timeout variants of wait_for_completion().
+To emphasise this again: in particular when using some of the waiting API variants
+with more complex outcomes, such as the timeout or signalling (_timeout(),
+_killable() and _interruptible()) variants, the wait might complete
+prematurely while the object might still be in use by another thread - and a return
+from the wait_on_completion*() caller function will deallocate the function
+stack and cause subtle data corruption if a complete() is done in some
+other thread. Simple testing might not trigger these kinds of races.
+If unsure, use dynamically allocated completion objects, preferably embedded
+in some other long lived object that has a boringly long life time which
+exceeds the life time of any helper threads using the completion object,
+or has a lock or other synchronization mechanism to make sure complete()
+is not called on a freed object.
+
+A naive DECLARE_COMPLETION() on the stack triggers a lockdep warning.
Waiting for completions:
------------------------
-For a thread of execution to wait for some concurrent work to finish, it
-calls wait_for_completion() on the initialized completion structure.
+For a thread to wait for some concurrent activity to finish, it
+calls wait_for_completion() on the initialized completion structure:
+
+ void wait_for_completion(struct completion *done)
+
A typical usage scenario is:
+ CPU#1 CPU#2
+
struct completion setup_done;
+
init_completion(&setup_done);
- initialize_work(...,&setup_done,...)
+ initialize_work(...,&setup_done,...);
- /* run non-dependent code */ /* do setup */
+ /* run non-dependent code */ /* do setup */
- wait_for_completion(&setup_done); complete(setup_done)
+ wait_for_completion(&setup_done); complete(setup_done);
-This is not implying any temporal order on wait_for_completion() and the
-call to complete() - if the call to complete() happened before the call
+This is not implying any particular order between wait_for_completion() and
+the call to complete() - if the call to complete() happened before the call
to wait_for_completion() then the waiting side simply will continue
-immediately as all dependencies are satisfied if not it will block until
+immediately as all dependencies are satisfied; if not, it will block until
completion is signaled by complete().
Note that wait_for_completion() is calling spin_lock_irq()/spin_unlock_irq(),
so it can only be called safely when you know that interrupts are enabled.
-Calling it from hard-irq or irqs-off atomic contexts will result in
-hard-to-detect spurious enabling of interrupts.
-
-wait_for_completion():
-
- void wait_for_completion(struct completion *done):
+Calling it from IRQs-off atomic contexts will result in hard-to-detect
+spurious enabling of interrupts.
The default behavior is to wait without a timeout and to mark the task as
uninterruptible. wait_for_completion() and its variants are only safe
in process context (as they can sleep) but not in atomic context,
-interrupt context, with disabled irqs. or preemption is disabled - see also
+interrupt context, with disabled IRQs, or preemption is disabled - see also
try_wait_for_completion() below for handling completion in atomic/interrupt
context.
As all variants of wait_for_completion() can (obviously) block for a long
-time, you probably don't want to call this with held mutexes.
+time depending on the nature of the activity they are waiting for, so in
+most cases you probably don't want to call this with held mutexes.
-Variants available:
--------------------
+wait_for_completion*() variants available:
+------------------------------------------
The below variants all return status and this status should be checked in
most(/all) cases - in cases where the status is deliberately not checked you
arch/arm/kernel/smp.c:__cpu_up()).
A common problem that occurs is to have unclean assignment of return types,
-so care should be taken with assigning return-values to variables of proper
-type. Checking for the specific meaning of return values also has been found
-to be quite inaccurate e.g. constructs like
-if (!wait_for_completion_interruptible_timeout(...)) would execute the same
-code path for successful completion and for the interrupted case - which is
-probably not what you want.
+so take care to assign return-values to variables of the proper type.
+
+Checking for the specific meaning of return values also has been found
+to be quite inaccurate, e.g. constructs like:
+
+ if (!wait_for_completion_interruptible_timeout(...))
+
+... would execute the same code path for successful completion and for the
+interrupted case - which is probably not what you want.
int wait_for_completion_interruptible(struct completion *done)
-This function marks the task TASK_INTERRUPTIBLE. If a signal was received
-while waiting it will return -ERESTARTSYS; 0 otherwise.
+This function marks the task TASK_INTERRUPTIBLE while it is waiting.
+If a signal was received while waiting it will return -ERESTARTSYS; 0 otherwise.
- unsigned long wait_for_completion_timeout(struct completion *done,
- unsigned long timeout)
+ unsigned long wait_for_completion_timeout(struct completion *done, unsigned long timeout)
The task is marked as TASK_UNINTERRUPTIBLE and will wait at most 'timeout'
-(in jiffies). If timeout occurs it returns 0 else the remaining time in
-jiffies (but at least 1). Timeouts are preferably calculated with
-msecs_to_jiffies() or usecs_to_jiffies(). If the returned timeout value is
-deliberately ignored a comment should probably explain why (e.g. see
-drivers/mfd/wm8350-core.c wm8350_read_auxadc())
+jiffies. If a timeout occurs it returns 0, else the remaining time in
+jiffies (but at least 1).
+
+Timeouts are preferably calculated with msecs_to_jiffies() or usecs_to_jiffies(),
+to make the code largely HZ-invariant.
+
+If the returned timeout value is deliberately ignored a comment should probably explain
+why (e.g. see drivers/mfd/wm8350-core.c wm8350_read_auxadc()).
- long wait_for_completion_interruptible_timeout(
- struct completion *done, unsigned long timeout)
+ long wait_for_completion_interruptible_timeout(struct completion *done, unsigned long timeout)
This function passes a timeout in jiffies and marks the task as
TASK_INTERRUPTIBLE. If a signal was received it will return -ERESTARTSYS;
-otherwise it returns 0 if the completion timed out or the remaining time in
+otherwise it returns 0 if the completion timed out, or the remaining time in
jiffies if completion occurred.
Further variants include _killable which uses TASK_KILLABLE as the
-designated tasks state and will return -ERESTARTSYS if it is interrupted or
-else 0 if completion was achieved. There is a _timeout variant as well:
+designated tasks state and will return -ERESTARTSYS if it is interrupted,
+or 0 if completion was achieved. There is a _timeout variant as well:
long wait_for_completion_killable(struct completion *done)
- long wait_for_completion_killable_timeout(struct completion *done,
- unsigned long timeout)
+ long wait_for_completion_killable_timeout(struct completion *done, unsigned long timeout)
The _io variants wait_for_completion_io() behave the same as the non-_io
-variants, except for accounting waiting time as waiting on IO, which has
-an impact on how the task is accounted in scheduling stats.
+variants, except for accounting waiting time as 'waiting on IO', which has
+an impact on how the task is accounted in scheduling/IO stats:
void wait_for_completion_io(struct completion *done)
- unsigned long wait_for_completion_io_timeout(struct completion *done
- unsigned long timeout)
+ unsigned long wait_for_completion_io_timeout(struct completion *done, unsigned long timeout)
Signaling completions:
A thread that wants to signal that the conditions for continuation have been
achieved calls complete() to signal exactly one of the waiters that it can
-continue.
+continue:
void complete(struct completion *done)
-or calls complete_all() to signal all current and future waiters.
+... or calls complete_all() to signal all current and future waiters:
void complete_all(struct completion *done)
The signaling will work as expected even if completions are signaled before
a thread starts waiting. This is achieved by the waiter "consuming"
-(decrementing) the done element of struct completion. Waiting threads
+(decrementing) the done field of 'struct completion'. Waiting threads
wakeup order is the same in which they were enqueued (FIFO order).
If complete() is called multiple times then this will allow for that number
of waiters to continue - each call to complete() will simply increment the
-done element. Calling complete_all() multiple times is a bug though. Both
-complete() and complete_all() can be called in hard-irq/atomic context safely.
+done field. Calling complete_all() multiple times is a bug though. Both
+complete() and complete_all() can be called in IRQ/atomic context safely.
-There only can be one thread calling complete() or complete_all() on a
-particular struct completion at any time - serialized through the wait
+There can only be one thread calling complete() or complete_all() on a
+particular 'struct completion' at any time - serialized through the wait
queue spinlock. Any such concurrent calls to complete() or complete_all()
probably are a design bug.
-Signaling completion from hard-irq context is fine as it will appropriately
-lock with spin_lock_irqsave/spin_unlock_irqrestore and it will never sleep.
+Signaling completion from IRQ context is fine as it will appropriately
+lock with spin_lock_irqsave()/spin_unlock_irqrestore() and it will never sleep.
try_wait_for_completion()/completion_done():
bool try_wait_for_completion(struct completion *done)
-Finally, to check the state of a completion without changing it in any way,
+Finally, to check the state of a completion without changing it in any way,
call completion_done(), which returns false if there are no posted
completions that were not yet consumed by waiters (implying that there are
waiters) and true otherwise;
bool completion_done(struct completion *done)
Both try_wait_for_completion() and completion_done() are safe to be called in
-hard-irq or atomic context.
+IRQ or atomic context.
2) Cache hit and miss measurements using model specific precision counters if
available. Depending on the levels of cache on the system the pseudo_lock_l2
and pseudo_lock_l3 tracepoints are available.
- WARNING: triggering this measurement uses from two (for just L2
- measurements) to four (for L2 and L3 measurements) precision counters on
- the system, if any other measurements are in progress the counters and
- their corresponding event registers will be clobbered.
When a pseudo-locked region is created a new debugfs directory is created for
it in debugfs as /sys/kernel/debug/resctrl/<newdir>. A single
write-only file, pseudo_lock_measure, is present in this directory. The
-measurement on the pseudo-locked region depends on the number, 1 or 2,
-written to this debugfs file. Since the measurements are recorded with the
-tracing infrastructure the relevant tracepoints need to be enabled before the
-measurement is triggered.
+measurement of the pseudo-locked region depends on the number written to this
+debugfs file:
+1 - writing "1" to the pseudo_lock_measure file will trigger the latency
+ measurement captured in the pseudo_lock_mem_latency tracepoint. See
+ example below.
+2 - writing "2" to the pseudo_lock_measure file will trigger the L2 cache
+ residency (cache hits and misses) measurement captured in the
+ pseudo_lock_l2 tracepoint. See example below.
+3 - writing "3" to the pseudo_lock_measure file will trigger the L3 cache
+ residency (cache hits and misses) measurement captured in the
+ pseudo_lock_l3 tracepoint.
+
+All measurements are recorded with the tracing infrastructure. This requires
+the relevant tracepoints to be enabled before the measurement is triggered.
Example of latency debugging interface:
In this example a pseudo-locked region named "newlock" was created. Here is
+++ /dev/null
-Valid-License-Identifier: CC-BY-SA-4.0
-SPDX-URL: https://spdx.org/licenses/CC-BY-SA-4.0
-Usage-Guide:
- To use the Creative Commons Attribution Share Alike 4.0 International
- license put the following SPDX tag/value pair into a comment according to
- the placement guidelines in the licensing rules documentation:
- SPDX-License-Identifier: CC-BY-SA-4.0
-License-Text:
-
-Creative Commons Attribution-ShareAlike 4.0 International
-
-Creative Commons Corporation ("Creative Commons") is not a law firm and
-does not provide legal services or legal advice. Distribution of Creative
-Commons public licenses does not create a lawyer-client or other
-relationship. Creative Commons makes its licenses and related information
-available on an "as-is" basis. Creative Commons gives no warranties
-regarding its licenses, any material licensed under their terms and
-conditions, or any related information. Creative Commons disclaims all
-liability for damages resulting from their use to the fullest extent
-possible.
-
-Using Creative Commons Public Licenses
-
-Creative Commons public licenses provide a standard set of terms and
-conditions that creators and other rights holders may use to share original
-works of authorship and other material subject to copyright and certain
-other rights specified in the public license below. The following
-considerations are for informational purposes only, are not exhaustive, and
-do not form part of our licenses.
-
-Considerations for licensors: Our public licenses are intended for use by
-those authorized to give the public permission to use material in ways
-otherwise restricted by copyright and certain other rights. Our licenses
-are irrevocable. Licensors should read and understand the terms and
-conditions of the license they choose before applying it. Licensors should
-also secure all rights necessary before applying our licenses so that the
-public can reuse the material as expected. Licensors should clearly mark
-any material not subject to the license. This includes other CC-licensed
-material, or material used under an exception or limitation to
-copyright. More considerations for licensors :
-wiki.creativecommons.org/Considerations_for_licensors
-
-Considerations for the public: By using one of our public licenses, a
-licensor grants the public permission to use the licensed material under
-specified terms and conditions. If the licensor's permission is not
-necessary for any reason - for example, because of any applicable exception
-or limitation to copyright - then that use is not regulated by the
-license. Our licenses grant only permissions under copyright and certain
-other rights that a licensor has authority to grant. Use of the licensed
-material may still be restricted for other reasons, including because
-others have copyright or other rights in the material. A licensor may make
-special requests, such as asking that all changes be marked or described.
-
-Although not required by our licenses, you are encouraged to respect those
-requests where reasonable. More considerations for the public :
-wiki.creativecommons.org/Considerations_for_licensees
-
-Creative Commons Attribution-ShareAlike 4.0 International Public License
-
-By exercising the Licensed Rights (defined below), You accept and agree to
-be bound by the terms and conditions of this Creative Commons
-Attribution-ShareAlike 4.0 International Public License ("Public
-License"). To the extent this Public License may be interpreted as a
-contract, You are granted the Licensed Rights in consideration of Your
-acceptance of these terms and conditions, and the Licensor grants You such
-rights in consideration of benefits the Licensor receives from making the
-Licensed Material available under these terms and conditions.
-
-Section 1 - Definitions.
-
- a. Adapted Material means material subject to Copyright and Similar
- Rights that is derived from or based upon the Licensed Material and
- in which the Licensed Material is translated, altered, arranged,
- transformed, or otherwise modified in a manner requiring permission
- under the Copyright and Similar Rights held by the Licensor. For
- purposes of this Public License, where the Licensed Material is a
- musical work, performance, or sound recording, Adapted Material is
- always produced where the Licensed Material is synched in timed
- relation with a moving image.
-
- b. Adapter's License means the license You apply to Your Copyright and
- Similar Rights in Your contributions to Adapted Material in
- accordance with the terms and conditions of this Public License.
-
- c. BY-SA Compatible License means a license listed at
- creativecommons.org/compatiblelicenses, approved by Creative Commons
- as essentially the equivalent of this Public License.
-
- d. Copyright and Similar Rights means copyright and/or similar rights
- closely related to copyright including, without limitation,
- performance, broadcast, sound recording, and Sui Generis Database
- Rights, without regard to how the rights are labeled or
- categorized. For purposes of this Public License, the rights
- specified in Section 2(b)(1)-(2) are not Copyright and Similar
- Rights.
-
- e. Effective Technological Measures means those measures that, in the
- absence of proper authority, may not be circumvented under laws
- fulfilling obligations under Article 11 of the WIPO Copyright Treaty
- adopted on December 20, 1996, and/or similar international
- agreements.
-
- f. Exceptions and Limitations means fair use, fair dealing, and/or any
- other exception or limitation to Copyright and Similar Rights that
- applies to Your use of the Licensed Material.
-
- g. License Elements means the license attributes listed in the name of
- a Creative Commons Public License. The License Elements of this
- Public License are Attribution and ShareAlike.
-
- h. Licensed Material means the artistic or literary work, database, or
- other material to which the Licensor applied this Public License.
-
- i. Licensed Rights means the rights granted to You subject to the terms
- and conditions of this Public License, which are limited to all
- Copyright and Similar Rights that apply to Your use of the Licensed
- Material and that the Licensor has authority to license.
-
- j. Licensor means the individual(s) or entity(ies) granting rights
- under this Public License.
-
- k. Share means to provide material to the public by any means or
- process that requires permission under the Licensed Rights, such as
- reproduction, public display, public performance, distribution,
- dissemination, communication, or importation, and to make material
- available to the public including in ways that members of the public
- may access the material from a place and at a time individually
- chosen by them.
-
- l. Sui Generis Database Rights means rights other than copyright
- resulting from Directive 96/9/EC of the European Parliament and of
- the Council of 11 March 1996 on the legal protection of databases,
- as amended and/or succeeded, as well as other essentially equivalent
- rights anywhere in the world. m. You means the individual or entity
- exercising the Licensed Rights under this Public License. Your has a
- corresponding meaning.
-
-Section 2 - Scope.
-
- a. License grant.
-
- 1. Subject to the terms and conditions of this Public License, the
- Licensor hereby grants You a worldwide, royalty-free,
- non-sublicensable, non-exclusive, irrevocable license to
- exercise the Licensed Rights in the Licensed Material to:
-
- A. reproduce and Share the Licensed Material, in whole or in part; and
-
- B. produce, reproduce, and Share Adapted Material.
-
- 2. Exceptions and Limitations. For the avoidance of doubt, where
- Exceptions and Limitations apply to Your use, this Public
- License does not apply, and You do not need to comply with its
- terms and conditions.
-
- 3. Term. The term of this Public License is specified in Section 6(a).
-
- 4. Media and formats; technical modifications allowed. The Licensor
- authorizes You to exercise the Licensed Rights in all media and
- formats whether now known or hereafter created, and to make
- technical modifications necessary to do so. The Licensor waives
- and/or agrees not to assert any right or authority to forbid You
- from making technical modifications necessary to exercise the
- Licensed Rights, including technical modifications necessary to
- circumvent Effective Technological Measures. For purposes of
- this Public License, simply making modifications authorized by
- this Section 2(a)(4) never produces Adapted Material.
-
- 5. Downstream recipients.
-
- A. Offer from the Licensor - Licensed Material. Every recipient
- of the Licensed Material automatically receives an offer
- from the Licensor to exercise the Licensed Rights under the
- terms and conditions of this Public License.
-
- B. Additional offer from the Licensor - Adapted Material. Every
- recipient of Adapted Material from You automatically
- receives an offer from the Licensor to exercise the Licensed
- Rights in the Adapted Material under the conditions of the
- Adapter's License You apply.
-
- C. No downstream restrictions. You may not offer or impose any
- additional or different terms or conditions on, or apply any
- Effective Technological Measures to, the Licensed Material
- if doing so restricts exercise of the Licensed Rights by any
- recipient of the Licensed Material.
-
- 6. No endorsement. Nothing in this Public License constitutes or
- may be construed as permission to assert or imply that You are,
- or that Your use of the Licensed Material is, connected with, or
- sponsored, endorsed, or granted official status by, the Licensor
- or others designated to receive attribution as provided in
- Section 3(a)(1)(A)(i).
-
- b. Other rights.
-
- 1. Moral rights, such as the right of integrity, are not licensed
- under this Public License, nor are publicity, privacy, and/or
- other similar personality rights; however, to the extent
- possible, the Licensor waives and/or agrees not to assert any
- such rights held by the Licensor to the limited extent necessary
- to allow You to exercise the Licensed Rights, but not otherwise.
-
- 2. Patent and trademark rights are not licensed under this Public
- License.
-
- 3. To the extent possible, the Licensor waives any right to collect
- royalties from You for the exercise of the Licensed Rights,
- whether directly or through a collecting society under any
- voluntary or waivable statutory or compulsory licensing
- scheme. In all other cases the Licensor expressly reserves any
- right to collect such royalties.
-
-Section 3 - License Conditions.
-
-Your exercise of the Licensed Rights is expressly made subject to the
-following conditions.
-
- a. Attribution.
-
- 1. If You Share the Licensed Material (including in modified form),
- You must:
-
- A. retain the following if it is supplied by the Licensor with
- the Licensed Material:
-
- i. identification of the creator(s) of the Licensed
- Material and any others designated to receive
- attribution, in any reasonable manner requested by the
- Licensor (including by pseudonym if designated);
-
- ii. a copyright notice;
-
- iii. a notice that refers to this Public License;
-
- iv. a notice that refers to the disclaimer of warranties;
-
- v. a URI or hyperlink to the Licensed Material to the extent reasonably practicable;
-
- B. indicate if You modified the Licensed Material and retain an
- indication of any previous modifications; and
-
- C. indicate the Licensed Material is licensed under this Public
- License, and include the text of, or the URI or hyperlink to,
- this Public License.
-
- 2. You may satisfy the conditions in Section 3(a)(1) in any
- reasonable manner based on the medium, means, and context in
- which You Share the Licensed Material. For example, it may be
- reasonable to satisfy the conditions by providing a URI or
- hyperlink to a resource that includes the required information.
-
- 3. If requested by the Licensor, You must remove any of the
- information required by Section 3(a)(1)(A) to the extent
- reasonably practicable. b. ShareAlike.In addition to the
- conditions in Section 3(a), if You Share Adapted Material You
- produce, the following conditions also apply.
-
- 1. The Adapter's License You apply must be a Creative Commons
- license with the same License Elements, this version or
- later, or a BY-SA Compatible License.
-
- 2. You must include the text of, or the URI or hyperlink to, the
- Adapter's License You apply. You may satisfy this condition
- in any reasonable manner based on the medium, means, and
- context in which You Share Adapted Material.
-
- 3. You may not offer or impose any additional or different terms
- or conditions on, or apply any Effective Technological
- Measures to, Adapted Material that restrict exercise of the
- rights granted under the Adapter's License You apply.
-
-Section 4 - Sui Generis Database Rights.
-
-Where the Licensed Rights include Sui Generis Database Rights that apply to
-Your use of the Licensed Material:
-
- a. for the avoidance of doubt, Section 2(a)(1) grants You the right to
- extract, reuse, reproduce, and Share all or a substantial portion of
- the contents of the database;
-
- b. if You include all or a substantial portion of the database contents
- in a database in which You have Sui Generis Database Rights, then
- the database in which You have Sui Generis Database Rights (but not
- its individual contents) is Adapted Material, including for purposes
- of Section 3(b); and
-
- c. You must comply with the conditions in Section 3(a) if You Share all
- or a substantial portion of the contents of the database.
-
- For the avoidance of doubt, this Section 4 supplements and does not
- replace Your obligations under this Public License where the Licensed
- Rights include other Copyright and Similar Rights.
-
-Section 5 - Disclaimer of Warranties and Limitation of Liability.
-
- a. Unless otherwise separately undertaken by the Licensor, to the
- extent possible, the Licensor offers the Licensed Material as-is and
- as-available, and makes no representations or warranties of any kind
- concerning the Licensed Material, whether express, implied,
- statutory, or other. This includes, without limitation, warranties
- of title, merchantability, fitness for a particular purpose,
- non-infringement, absence of latent or other defects, accuracy, or
- the presence or absence of errors, whether or not known or
- discoverable. Where disclaimers of warranties are not allowed in
- full or in part, this disclaimer may not apply to You.
-
- b. To the extent possible, in no event will the Licensor be liable to
- You on any legal theory (including, without limitation, negligence)
- or otherwise for any direct, special, indirect, incidental,
- consequential, punitive, exemplary, or other losses, costs,
- expenses, or damages arising out of this Public License or use of
- the Licensed Material, even if the Licensor has been advised of the
- possibility of such losses, costs, expenses, or damages. Where a
- limitation of liability is not allowed in full or in part, this
- limitation may not apply to You.
-
- c. The disclaimer of warranties and limitation of liability provided
- above shall be interpreted in a manner that, to the extent possible,
- most closely approximates an absolute disclaimer and waiver of all
- liability.
-
-Section 6 - Term and Termination.
-
- a. This Public License applies for the term of the Copyright and
- Similar Rights licensed here. However, if You fail to comply with
- this Public License, then Your rights under this Public License
- terminate automatically.
-
- b. Where Your right to use the Licensed Material has terminated under
- Section 6(a), it reinstates:
-
- 1. automatically as of the date the violation is cured, provided it
- is cured within 30 days of Your discovery of the violation; or
-
- 2. upon express reinstatement by the Licensor.
-
- c. For the avoidance of doubt, this Section 6(b) does not affect any
- right the Licensor may have to seek remedies for Your violations of
- this Public License.
-
- d. For the avoidance of doubt, the Licensor may also offer the Licensed
- Material under separate terms or conditions or stop distributing the
- Licensed Material at any time; however, doing so will not terminate
- this Public License.
-
- e. Sections 1, 5, 6, 7, and 8 survive termination of this Public License.
-
-Section 7 - Other Terms and Conditions.
-
- a. The Licensor shall not be bound by any additional or different terms
- or conditions communicated by You unless expressly agreed.
-
- b. Any arrangements, understandings, or agreements regarding the
- Licensed Material not stated herein are separate from and
- independent of the terms and conditions of this Public License.
-
-Section 8 - Interpretation.
-
- a. For the avoidance of doubt, this Public License does not, and shall
- not be interpreted to, reduce, limit, restrict, or impose conditions
- on any use of the Licensed Material that could lawfully be made
- without permission under this Public License.
-
- b. To the extent possible, if any provision of this Public License is
- deemed unenforceable, it shall be automatically reformed to the
- minimum extent necessary to make it enforceable. If the provision
- cannot be reformed, it shall be severed from this Public License
- without affecting the enforceability of the remaining terms and
- conditions.
-
- c. No term or condition of this Public License will be waived and no
- failure to comply consented to unless expressly agreed to by the
- Licensor.
-
- d. Nothing in this Public License constitutes or may be interpreted as
- a limitation upon, or waiver of, any privileges and immunities that
- apply to the Licensor or You, including from the legal processes of
- any jurisdiction or authority.
-
-Creative Commons is not a party to its public licenses. Notwithstanding,
-Creative Commons may elect to apply one of its public licenses to material
-it publishes and in those instances will be considered the "Licensor." The
-text of the Creative Commons public licenses is dedicated to the public
-domain under the CC0 Public Domain Dedication. Except for the limited
-purpose of indicating that material is shared under a Creative Commons
-public license or as otherwise permitted by the Creative Commons policies
-published at creativecommons.org/policies, Creative Commons does not
-authorize the use of the trademark "Creative Commons" or any other
-trademark or logo of Creative Commons without its prior written consent
-including, without limitation, in connection with any unauthorized
-modifications to any of its public licenses or any other arrangements,
-understandings, or agreements concerning use of licensed material. For the
-avoidance of doubt, this paragraph does not form part of the public
-licenses.
-
-Creative Commons may be contacted at creativecommons.org.
F: drivers/gpio/gpio-uniphier.c
F: drivers/i2c/busses/i2c-uniphier*
F: drivers/irqchip/irq-uniphier-aidet.c
+F: drivers/mmc/host/uniphier-sd.c
F: drivers/pinctrl/uniphier/
F: drivers/reset/reset-uniphier.c
F: drivers/tty/serial/8250/8250_uniphier.c
F: drivers/gpio/gpio-brcmstb.c
F: Documentation/devicetree/bindings/gpio/brcm,brcmstb-gpio.txt
+BROADCOM BRCMSTB I2C DRIVER
+M: Kamal Dasu <kdasu.kdev@gmail.com>
+L: linux-i2c@vger.kernel.org
+L: bcm-kernel-feedback-list@broadcom.com
+S: Supported
+F: drivers/i2c/busses/i2c-brcmstb.c
+F: Documentation/devicetree/bindings/i2c/i2c-brcmstb.txt
+
BROADCOM BRCMSTB USB2 and USB3 PHY DRIVER
M: Al Cooper <alcooperx@gmail.com>
L: linux-kernel@vger.kernel.org
F: Documentation/devicetree/bindings/memory-controllers/brcm,dpfe-cpu.txt
F: drivers/memory/brcmstb_dpfe.c
+BROADCOM SPI DRIVER
+M: Kamal Dasu <kdasu.kdev@gmail.com>
+M: bcm-kernel-feedback-list@broadcom.com
+S: Maintained
+F: Documentation/devicetree/bindings/spi/brcm,spi-bcm-qspi.txt
+F: drivers/spi/spi-bcm-qspi.*
+F: drivers/spi/spi-brcmstb-qspi.c
+F: drivers/spi/spi-iproc-qspi.c
+
BROADCOM SYSTEMPORT ETHERNET DRIVER
M: Florian Fainelli <f.fainelli@gmail.com>
L: netdev@vger.kernel.org
F: Documentation/devicetree/bindings/media/coda.txt
F: drivers/media/platform/coda/
+CODE OF CONDUCT
+M: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
+S: Supported
+F: Documentation/process/code-of-conduct.rst
+F: Documentation/process/code-of-conduct-interpretation.rst
+
COMMON CLK FRAMEWORK
M: Michael Turquette <mturquette@baylibre.com>
M: Stephen Boyd <sboyd@kernel.org>
F: Documentation/hwmon/
F: drivers/hwmon/
F: include/linux/hwmon*.h
+F: include/trace/events/hwmon*.h
HARDWARE RANDOM NUMBER GENERATOR CORE
M: Matt Mackall <mpm@selenic.com>
F: mm/memory-failure.c
F: mm/hwpoison-inject.c
+HYGON PROCESSOR SUPPORT
+M: Pu Wen <puwen@hygon.cn>
+L: linux-kernel@vger.kernel.org
+S: Maintained
+F: arch/x86/kernel/cpu/hygon.c
+
Hyper-V CORE AND DRIVERS
M: "K. Y. Srinivasan" <kys@microsoft.com>
M: Haiyang Zhang <haiyangz@microsoft.com>
S: Supported
F: drivers/gpu/drm/i915/gvt/
+INTEL PMIC GPIO DRIVER
+R: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
+S: Maintained
+F: drivers/gpio/gpio-*cove.c
+F: drivers/gpio/gpio-msic.c
+
INTEL HID EVENT DRIVER
M: Alex Hung <alex.hung@canonical.com>
L: platform-driver-x86@vger.kernel.org
L: openipmi-developer@lists.sourceforge.net (moderated for non-subscribers)
W: http://openipmi.sourceforge.net/
S: Supported
+F: Documentation/devicetree/bindings/ipmi/
F: Documentation/IPMI.txt
F: drivers/char/ipmi/
F: include/linux/ipmi*
F: Documentation/hwmon/max16065
F: drivers/hwmon/max16065.c
-MAX20751 HARDWARE MONITOR DRIVER
-M: Guenter Roeck <linux@roeck-us.net>
-L: linux-hwmon@vger.kernel.org
-S: Maintained
-F: Documentation/hwmon/max20751
-F: drivers/hwmon/max20751.c
-
MAX2175 SDR TUNER DRIVER
M: Ramesh Shanmugasundaram <ramesh.shanmugasundaram@bp.renesas.com>
L: linux-media@vger.kernel.org
M: Jiaxun Yang <jiaxun.yang@flygoat.com>
L: linux-mips@linux-mips.org
S: Maintained
-F: arch/mips/loongson64/*{2e/2f}*
+F: arch/mips/loongson64/fuloong-2e/
+F: arch/mips/loongson64/lemote-2f/
F: arch/mips/include/asm/mach-loongson64/
F: drivers/*/*loongson2*
F: drivers/*/*/*loongson2*
F: arch/arm/boot/dts/mmp*
F: arch/arm/mach-mmp/
+MMU GATHER AND TLB INVALIDATION
+M: Will Deacon <will.deacon@arm.com>
+M: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
+M: Andrew Morton <akpm@linux-foundation.org>
+M: Nick Piggin <npiggin@gmail.com>
+M: Peter Zijlstra <peterz@infradead.org>
+L: linux-arch@vger.kernel.org
+L: linux-mm@kvack.org
+S: Maintained
+F: arch/*/include/asm/tlb.h
+F: include/asm-generic/tlb.h
+F: mm/mmu_gather.c
+
MN88472 MEDIA DRIVER
M: Antti Palosaari <crope@iki.fi>
L: linux-media@vger.kernel.org
S: Maintained
F: Documentation/ABI/testing/sysfs-class-mux*
F: Documentation/devicetree/bindings/mux/
-F: include/linux/dt-bindings/mux/
+F: include/dt-bindings/mux/
F: include/linux/mux/
F: drivers/mux/
T: git git://git.kernel.org/pub/scm/linux/kernel/git/klassert/ipsec.git
T: git git://git.kernel.org/pub/scm/linux/kernel/git/klassert/ipsec-next.git
S: Maintained
-F: net/core/flow.c
F: net/xfrm/
F: net/key/
F: net/ipv4/xfrm*
F: drivers/pinctrl/intel/
PIN CONTROLLER - MEDIATEK
-M: Sean Wang <sean.wang@mediatek.com>
+M: Sean Wang <sean.wang@kernel.org>
L: linux-mediatek@lists.infradead.org (moderated for non-subscribers)
S: Maintained
F: Documentation/devicetree/bindings/pinctrl/pinctrl-mt65xx.txt
F: Documentation/devicetree/bindings/pinctrl/pinctrl-mt7622.txt
-F: drivers/pinctrl/mediatek/mtk-eint.*
-F: drivers/pinctrl/mediatek/pinctrl-mtk-common.*
-F: drivers/pinctrl/mediatek/pinctrl-mt2701.c
-F: drivers/pinctrl/mediatek/pinctrl-mt7622.c
+F: drivers/pinctrl/mediatek/
PIN CONTROLLER - QUALCOMM
M: Bjorn Andersson <bjorn.andersson@linaro.org>
W: http://www.roeck-us.net/linux/drivers/
T: git git://git.kernel.org/pub/scm/linux/kernel/git/groeck/linux-staging.git
S: Maintained
+F: Documentation/devicetree/bindings/hwmon/ibm,cffps1.txt
+F: Documentation/devicetree/bindings/hwmon/max31785.txt
+F: Documentation/devicetree/bindings/hwmon/ltc2978.txt
+F: Documentation/hwmon/adm1275
+F: Documentation/hwmon/ibm-cffps
+F: Documentation/hwmon/ir35221
+F: Documentation/hwmon/lm25066
+F: Documentation/hwmon/ltc2978
+F: Documentation/hwmon/ltc3815
+F: Documentation/hwmon/max16064
+F: Documentation/hwmon/max20751
+F: Documentation/hwmon/max31785
+F: Documentation/hwmon/max34440
+F: Documentation/hwmon/max8688
F: Documentation/hwmon/pmbus
+F: Documentation/hwmon/pmbus-core
+F: Documentation/hwmon/tps40422
+F: Documentation/hwmon/ucd9000
+F: Documentation/hwmon/ucd9200
+F: Documentation/hwmon/zl6100
F: drivers/hwmon/pmbus/
F: include/linux/pmbus.h
M: Paul Moore <paul@paul-moore.com>
M: Stephen Smalley <sds@tycho.nsa.gov>
M: Eric Paris <eparis@parisplace.org>
-L: selinux@tycho.nsa.gov (moderated for non-subscribers)
+L: selinux@vger.kernel.org
W: https://selinuxproject.org
W: https://github.com/SELinuxProject
T: git git://git.kernel.org/pub/scm/linux/kernel/git/pcmoore/selinux.git
R: Pengutronix Kernel Team <kernel@pengutronix.de>
S: Supported
F: drivers/siox/*
+F: drivers/gpio/gpio-siox.c
F: include/trace/events/siox.h
SIS 190 ETHERNET DRIVER
S: Maintained
F: Documentation/devicetree/bindings/arm/firmware/sdei.txt
F: drivers/firmware/arm_sdei.c
-F: include/linux/sdei.h
-F: include/uapi/linux/sdei.h
+F: include/linux/arm_sdei.h
+F: include/uapi/linux/arm_sdei.h
SOFTWARE RAID (Multiple Disks) SUPPORT
M: Shaohua Li <shli@kernel.org>
F: drivers/reset/reset-axs10x.c
F: Documentation/devicetree/bindings/reset/snps,axs10x-reset.txt
+SYNOPSYS CREG GPIO DRIVER
+M: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
+S: Maintained
+F: drivers/gpio/gpio-creg-snps.c
+F: Documentation/devicetree/bindings/gpio/snps,creg-gpio.txt
+
SYNOPSYS DESIGNWARE 8250 UART DRIVER
R: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
S: Maintained
VERSION = 4
PATCHLEVEL = 19
SUBLEVEL = 0
-EXTRAVERSION = -rc7
-NAME = Merciless Moray
+EXTRAVERSION =
+NAME = "People's Front"
# *DOCUMENTATION*
# To see a list of typical targets execute "make help"
ifeq ($(cc-name),clang)
ifneq ($(CROSS_COMPILE),)
CLANG_TARGET := --target=$(notdir $(CROSS_COMPILE:%-=%))
-GCC_TOOLCHAIN := $(realpath $(dir $(shell which $(LD)))/..)
+GCC_TOOLCHAIN_DIR := $(dir $(shell which $(LD)))
+CLANG_PREFIX := --prefix=$(GCC_TOOLCHAIN_DIR)
+GCC_TOOLCHAIN := $(realpath $(GCC_TOOLCHAIN_DIR)/..)
endif
ifneq ($(GCC_TOOLCHAIN),)
CLANG_GCC_TC := --gcc-toolchain=$(GCC_TOOLCHAIN)
endif
-KBUILD_CFLAGS += $(CLANG_TARGET) $(CLANG_GCC_TC)
-KBUILD_AFLAGS += $(CLANG_TARGET) $(CLANG_GCC_TC)
+KBUILD_CFLAGS += $(CLANG_TARGET) $(CLANG_GCC_TC) $(CLANG_PREFIX)
+KBUILD_AFLAGS += $(CLANG_TARGET) $(CLANG_GCC_TC) $(CLANG_PREFIX)
KBUILD_CFLAGS += $(call cc-option, -no-integrated-as)
KBUILD_AFLAGS += $(call cc-option, -no-integrated-as)
endif
# version.h and scripts_basic is processed / created.
# Listed in dependency order
-PHONY += prepare archprepare prepare0 prepare1 prepare2 prepare3
+PHONY += prepare archprepare macroprepare prepare0 prepare1 prepare2 prepare3
# prepare3 is used to check if we are building in a separate output directory,
# and if so do:
prepare1: prepare2 $(version_h) $(autoksyms_h) include/generated/utsrelease.h
$(cmd_crmodverdir)
-archprepare: archheaders archscripts prepare1 scripts_basic
+macroprepare: prepare1 archmacros
+
+archprepare: archheaders archscripts macroprepare scripts_basic
prepare0: archprepare gcc-plugins
$(Q)$(MAKE) $(build)=.
PHONY += archscripts
archscripts:
+PHONY += archmacros
+archmacros:
+
PHONY += __headers
__headers: $(version_h) scripts_basic uapi-asm-generic archheaders archscripts
$(Q)$(MAKE) $(build)=scripts build_unifdef
config HAVE_ARCH_JUMP_LABEL
bool
+config HAVE_ARCH_JUMP_LABEL_RELATIVE
+ bool
+
config HAVE_RCU_TABLE_FREE
bool
config ARC
def_bool y
select ARC_TIMERS
+ select ARCH_HAS_DMA_COHERENT_TO_PFN
select ARCH_HAS_PTE_SPECIAL
select ARCH_HAS_SYNC_DMA_FOR_CPU
select ARCH_HAS_SYNC_DMA_FOR_DEVICE
select BUILDTIME_EXTABLE_SORT
select CLONE_BACKWARDS
select COMMON_CLK
- select DMA_NONCOHERENT_OPS
- select DMA_NONCOHERENT_MMAP
+ select DMA_DIRECT_OPS
select GENERIC_ATOMIC64 if !ISA_ARCV2 || !(ARC_HAS_LL64 && ARC_HAS_LLSC)
select GENERIC_CLOCKEVENTS
select GENERIC_FIND_FIRST_BIT
Support for ARC770 core introduced with Rel 4.10 (Summer 2011)
This core has a bunch of cool new features:
-MMU-v3: Variable Page Sz (4k, 8k, 16k), bigger J-TLB (128x4)
- Shared Address Spaces (for sharing TLB entires in MMU)
+ Shared Address Spaces (for sharing TLB entries in MMU)
-Caches: New Prog Model, Region Flush
-Insns: endian swap, load-locked/store-conditional, time-stamp-ctr
# published by the Free Software Foundation.
#
-ifeq ($(CROSS_COMPILE),)
-ifndef CONFIG_CPU_BIG_ENDIAN
-CROSS_COMPILE := arc-linux-
-else
-CROSS_COMPILE := arceb-linux-
-endif
-endif
-
KBUILD_DEFCONFIG := nsim_700_defconfig
cflags-y += -fno-common -pipe -fno-builtin -mmedium-calls -D__linux__
cflags-$(CONFIG_ISA_ARCOMPACT) += -mA7
-cflags-$(CONFIG_ISA_ARCV2) += -mcpu=archs
-
-is_700 = $(shell $(CC) -dM -E - < /dev/null | grep -q "ARC700" && echo 1 || echo 0)
-
-ifdef CONFIG_ISA_ARCOMPACT
-ifeq ($(is_700), 0)
- $(error Toolchain not configured for ARCompact builds)
-endif
-endif
-
-ifdef CONFIG_ISA_ARCV2
-ifeq ($(is_700), 1)
- $(error Toolchain not configured for ARCv2 builds)
-endif
-endif
+cflags-$(CONFIG_ISA_ARCV2) += -mcpu=hs38
ifdef CONFIG_ARC_CURR_IN_REG
# For a global register defintion, make sure it gets passed to every file
cflags-$(CONFIG_CPU_BIG_ENDIAN) += -mbig-endian
ldflags-$(CONFIG_CPU_BIG_ENDIAN) += -EB
-LIBGCC := $(shell $(CC) $(cflags-y) --print-libgcc-file-name)
+LIBGCC = $(shell $(CC) $(cflags-y) --print-libgcc-file-name)
# Modules with short calls might break for calls into builtin-kernel
KBUILD_CFLAGS_MODULE += -mlong-calls -mno-millicode
task_thread_info(current)->thr_ptr;
}
+
+ /*
+ * setup usermode thread pointer #1:
+ * when child is picked by scheduler, __switch_to() uses @c_callee to
+ * populate usermode callee regs: this works (despite being in a kernel
+ * function) since special return path for child @ret_from_fork()
+ * ensures those regs are not clobbered all the way to RTIE to usermode
+ */
+ c_callee->r25 = task_thread_info(p)->thr_ptr;
+
+#ifdef CONFIG_ARC_CURR_IN_REG
+ /*
+ * setup usermode thread pointer #2:
+ * however for this special use of r25 in kernel, __switch_to() sets
+ * r25 for kernel needs and only in the final return path is usermode
+ * r25 setup, from pt_regs->user_r25. So set that up as well
+ */
+ c_regs->user_r25 = c_callee->r25;
+#endif
+
return 0;
}
__free_pages(page, get_order(size));
}
-int arch_dma_mmap(struct device *dev, struct vm_area_struct *vma,
- void *cpu_addr, dma_addr_t dma_addr, size_t size,
- unsigned long attrs)
+long arch_dma_coherent_to_pfn(struct device *dev, void *cpu_addr,
+ dma_addr_t dma_addr)
{
- unsigned long user_count = vma_pages(vma);
- unsigned long count = PAGE_ALIGN(size) >> PAGE_SHIFT;
- unsigned long pfn = __phys_to_pfn(dma_addr);
- unsigned long off = vma->vm_pgoff;
- int ret = -ENXIO;
-
- vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
-
- if (dma_mmap_from_dev_coherent(dev, vma, cpu_addr, size, &ret))
- return ret;
-
- if (off < count && user_count <= (count - off)) {
- ret = remap_pfn_range(vma, vma->vm_start,
- pfn + off,
- user_count << PAGE_SHIFT,
- vma->vm_page_prot);
- }
-
- return ret;
+ return __phys_to_pfn(dma_addr);
}
/*
}
/*
- * Plug in coherent or noncoherent dma ops
+ * Plug in direct dma map ops.
*/
void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
const struct iommu_ops *iommu, bool coherent)
/*
* IOC hardware snoops all DMA traffic keeping the caches consistent
* with memory - eliding need for any explicit cache maintenance of
- * DMA buffers - so we can use dma_direct cache ops.
+ * DMA buffers.
*/
- if (is_isa_arcv2() && ioc_enable && coherent) {
- set_dma_ops(dev, &dma_direct_ops);
- dev_info(dev, "use dma_direct_ops cache ops\n");
- } else {
- set_dma_ops(dev, &dma_noncoherent_ops);
- dev_info(dev, "use dma_noncoherent_ops cache ops\n");
- }
+ if (is_isa_arcv2() && ioc_enable && coherent)
+ dev->dma_coherent = true;
+
+ dev_info(dev, "use %sncoherent DMA ops\n",
+ dev->dma_coherent ? "" : "non");
}
};
};
+&cpu0 {
+ /* CPU rated to 1GHz, not 1.2GHz as per the default settings */
+ operating-points = <
+ /* kHz uV */
+ 166666 850000
+ 400000 900000
+ 800000 1050000
+ 1000000 1200000
+ >;
+};
+
&esdhc1 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_esdhc1>;
extern void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
const struct iommu_ops *iommu, bool coherent);
+#ifdef CONFIG_MMU
#define arch_teardown_dma_ops arch_teardown_dma_ops
extern void arch_teardown_dma_ops(struct device *dev);
+#endif
/* do not use this function in a driver */
static inline bool is_device_dma_coherent(struct device *dev)
#include <asm/byteorder.h>
#include <asm/memory.h>
#include <asm-generic/pci_iomap.h>
-#include <xen/xen.h>
/*
* ISA I/O bus memory addresses are 1:1 with the physical address.
#include <asm-generic/io.h>
-/*
- * can the hardware map this into one segment or not, given no other
- * constraints.
- */
-#define BIOVEC_MERGEABLE(vec1, vec2) \
- ((bvec_to_phys((vec1)) + (vec1)->bv_len) == bvec_to_phys((vec2)))
-
-struct bio_vec;
-extern bool xen_biovec_phys_mergeable(const struct bio_vec *vec1,
- const struct bio_vec *vec2);
-#define BIOVEC_PHYS_MERGEABLE(vec1, vec2) \
- (__BIOVEC_PHYS_MERGEABLE(vec1, vec2) && \
- (!xen_domain() || xen_biovec_phys_mergeable(vec1, vec2)))
-
#ifdef CONFIG_MMU
#define ARCH_HAS_VALID_PHYS_ADDR_RANGE
extern int valid_phys_addr_range(phys_addr_t addr, size_t size);
#else
#define VTTBR_X (5 - KVM_T0SZ)
#endif
+#define VTTBR_CNP_BIT _AC(1, UL)
#define VTTBR_BADDR_MASK (((_AC(1, ULL) << (40 - VTTBR_X)) - 1) << VTTBR_X)
#define VTTBR_VMID_SHIFT _AC(48, ULL)
#define VTTBR_VMID_MASK(size) (_AT(u64, (1 << size) - 1) << VTTBR_VMID_SHIFT)
#define kvm_phys_to_vttbr(addr) (addr)
+static inline bool kvm_cpu_has_cnp(void)
+{
+ return false;
+}
+
#endif /* !__ASSEMBLY__ */
#endif /* __ARM_KVM_MMU_H__ */
/* Replace task scheduler's default cpu-invariant accounting */
#define arch_scale_cpu_capacity topology_get_cpu_scale
+/* Enable topology flag updates */
+#define arch_update_cpu_topology topology_update_cpu_topology
+
#else
static inline void init_cpu_topology(void) { }
#define ARM_DISCARD \
*(.ARM.exidx.exit.text) \
*(.ARM.extab.exit.text) \
+ *(.ARM.exidx.text.exit) \
+ *(.ARM.extab.text.exit) \
ARM_CPU_DISCARD(*(.ARM.exidx.cpuexit.text)) \
ARM_CPU_DISCARD(*(.ARM.extab.cpuexit.text)) \
ARM_EXIT_DISCARD(EXIT_TEXT) \
/* ICC_SGI1R */
{ CRm64(12), Op1( 0), is64, access_gic_sgi},
- /* ICC_ASGI1R */
- { CRm64(12), Op1( 1), is64, access_gic_sgi},
- /* ICC_SGI0R */
- { CRm64(12), Op1( 2), is64, access_gic_sgi},
/* VBAR: swapped by interrupt.S. */
{ CRn(12), CRm( 0), Op1( 0), Op2( 0), is32,
NULL, reset_val, c12_VBAR, 0x00000000 },
+ /* ICC_ASGI1R */
+ { CRm64(12), Op1( 1), is64, access_gic_sgi},
+ /* ICC_SGI0R */
+ { CRm64(12), Op1( 2), is64, access_gic_sgi},
/* ICC_SRE */
{ CRn(12), CRm(12), Op1( 0), Op2(5), is32, access_gic_sre },
};
static const struct gpio_led ntosd2_leds[] = {
- { .name = "led1_green", .gpio = GPIO(10), },
- { .name = "led1_red", .gpio = GPIO(11), },
- { .name = "led2_green", .gpio = GPIO(12), },
- { .name = "led2_red", .gpio = GPIO(13), },
+ { .name = "led1_green", .gpio = 10, },
+ { .name = "led1_red", .gpio = 11, },
+ { .name = "led2_green", .gpio = 12, },
+ { .name = "led2_red", .gpio = 13, },
};
static struct gpio_led_platform_data ntosd2_leds_data = {
*************************************************************************/
static struct resource ep93xx_gpio_resource[] = {
DEFINE_RES_MEM(EP93XX_GPIO_PHYS_BASE, 0xcc),
+ DEFINE_RES_IRQ(IRQ_EP93XX_GPIO_AB),
+ DEFINE_RES_IRQ(IRQ_EP93XX_GPIO0MUX),
+ DEFINE_RES_IRQ(IRQ_EP93XX_GPIO1MUX),
+ DEFINE_RES_IRQ(IRQ_EP93XX_GPIO2MUX),
+ DEFINE_RES_IRQ(IRQ_EP93XX_GPIO3MUX),
+ DEFINE_RES_IRQ(IRQ_EP93XX_GPIO4MUX),
+ DEFINE_RES_IRQ(IRQ_EP93XX_GPIO5MUX),
+ DEFINE_RES_IRQ(IRQ_EP93XX_GPIO6MUX),
+ DEFINE_RES_IRQ(IRQ_EP93XX_GPIO7MUX),
};
static struct platform_device ep93xx_gpio_device = {
#include <linux/i2c.h>
#include <linux/fb.h>
-#include <linux/mtd/partitions.h>
-#include <linux/mtd/rawnand.h>
+#include <linux/mtd/platnand.h>
#include <mach/hardware.h>
#include <linux/platform_data/video-ep93xx.h>
#define SNAPPERCL15_NAND_CEN (1 << 11) /* Chip enable (active low) */
#define SNAPPERCL15_NAND_RDY (1 << 14) /* Device ready */
-#define NAND_CTRL_ADDR(chip) (chip->IO_ADDR_W + 0x40)
+#define NAND_CTRL_ADDR(chip) (chip->legacy.IO_ADDR_W + 0x40)
-static void snappercl15_nand_cmd_ctrl(struct mtd_info *mtd, int cmd,
+static void snappercl15_nand_cmd_ctrl(struct nand_chip *chip, int cmd,
unsigned int ctrl)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
static u16 nand_state = SNAPPERCL15_NAND_WPN;
u16 set;
}
if (cmd != NAND_CMD_NONE)
- __raw_writew((cmd & 0xff) | nand_state, chip->IO_ADDR_W);
+ __raw_writew((cmd & 0xff) | nand_state,
+ chip->legacy.IO_ADDR_W);
}
-static int snappercl15_nand_dev_ready(struct mtd_info *mtd)
+static int snappercl15_nand_dev_ready(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
return !!(__raw_readw(NAND_CTRL_ADDR(chip)) & SNAPPERCL15_NAND_RDY);
}
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/io.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <linux/spi/spi.h>
#include <linux/spi/flash.h>
#include <linux/spi/mmc_spi.h>
#define TS72XX_NAND_CONTROL_ADDR_LINE 22 /* 0xN0400000 */
#define TS72XX_NAND_BUSY_ADDR_LINE 23 /* 0xN0800000 */
-static void ts72xx_nand_hwcontrol(struct mtd_info *mtd,
+static void ts72xx_nand_hwcontrol(struct nand_chip *chip,
int cmd, unsigned int ctrl)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
if (ctrl & NAND_CTRL_CHANGE) {
- void __iomem *addr = chip->IO_ADDR_R;
+ void __iomem *addr = chip->legacy.IO_ADDR_R;
unsigned char bits;
addr += (1 << TS72XX_NAND_CONTROL_ADDR_LINE);
}
if (cmd != NAND_CMD_NONE)
- __raw_writeb(cmd, chip->IO_ADDR_W);
+ __raw_writeb(cmd, chip->legacy.IO_ADDR_W);
}
-static int ts72xx_nand_device_ready(struct mtd_info *mtd)
+static int ts72xx_nand_device_ready(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
- void __iomem *addr = chip->IO_ADDR_R;
+ void __iomem *addr = chip->legacy.IO_ADDR_R;
addr += (1 << TS72XX_NAND_BUSY_ADDR_LINE);
#include <linux/mtd/mtd.h>
#include <linux/mtd/physmap.h>
#include <linux/gpio/driver.h>
+#include <linux/gpio/machine.h>
#include <linux/gpio.h>
#include <linux/regulator/fixed.h>
#include <linux/regulator/machine.h>
DEFINE_RES_MEM_NAMED(MX21ADS_IO_REG, SZ_2, "dat");
static struct bgpio_pdata mx21ads_mmgpio_pdata = {
+ .label = "mx21ads-mmgpio",
.base = MX21ADS_MMGPIO_BASE,
.ngpio = 16,
};
static struct fixed_voltage_config mx21ads_lcd_regulator_pdata = {
.supply_name = "LCD",
.microvolts = 3300000,
- .gpio = MX21ADS_IO_LCDON,
.enable_high = 1,
.init_data = &mx21ads_lcd_regulator_init_data,
};
},
};
+static struct gpiod_lookup_table mx21ads_lcd_regulator_gpiod_table = {
+ .dev_id = "reg-fixed-voltage.0", /* Let's hope ID 0 is what we get */
+ .table = {
+ GPIO_LOOKUP("mx21ads-mmgpio", 9, NULL, GPIO_ACTIVE_HIGH),
+ { },
+ },
+};
+
/*
* Connected is a portrait Sharp-QVGA display
* of type: LQ035Q7DB02
{
imx21_add_mxc_mmc(0, &mx21ads_sdhc_pdata);
+ gpiod_add_lookup_table(&mx21ads_lcd_regulator_gpiod_table);
platform_add_devices(platform_devices, ARRAY_SIZE(platform_devices));
mx21ads_cs8900_resources[1].start =
#include <linux/gpio/driver.h>
/* Needed for gpio_to_irq() */
#include <linux/gpio.h>
+#include <linux/gpio/machine.h>
#include <linux/platform_device.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/map.h>
static struct fixed_voltage_config mx27ads_lcd_regulator_pdata = {
.supply_name = "LCD",
.microvolts = 3300000,
- .gpio = MX27ADS_LCD_GPIO,
.init_data = &mx27ads_lcd_regulator_init_data,
};
+static struct gpiod_lookup_table mx27ads_lcd_regulator_gpiod_table = {
+ .dev_id = "reg-fixed-voltage.0", /* Let's hope ID 0 is what we get */
+ .table = {
+ GPIO_LOOKUP("LCD", 0, NULL, GPIO_ACTIVE_HIGH),
+ { },
+ },
+};
+
static void __init mx27ads_regulator_init(void)
{
struct gpio_chip *vchip;
vchip->set = vgpio_set;
gpiochip_add_data(vchip, NULL);
+ gpiod_add_lookup_table(&mx27ads_lcd_regulator_gpiod_table);
+
platform_device_register_data(NULL, "reg-fixed-voltage",
PLATFORM_DEVID_AUTO,
&mx27ads_lcd_regulator_pdata,
#include <linux/memory.h>
#include <linux/platform_device.h>
#include <linux/mtd/physmap.h>
-#include <linux/mtd/rawnand.h>
+#include <linux/mtd/platnand.h>
#include <linux/gpio.h>
#include <asm/mach-types.h>
/*
* Hardware specific access to control-lines
*/
-static void qong_nand_cmd_ctrl(struct mtd_info *mtd, int cmd, unsigned int ctrl)
+static void qong_nand_cmd_ctrl(struct nand_chip *nand_chip, int cmd,
+ unsigned int ctrl)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
-
if (cmd == NAND_CMD_NONE)
return;
if (ctrl & NAND_CLE)
- writeb(cmd, nand_chip->IO_ADDR_W + (1 << 24));
+ writeb(cmd, nand_chip->legacy.IO_ADDR_W + (1 << 24));
else
- writeb(cmd, nand_chip->IO_ADDR_W + (1 << 23));
+ writeb(cmd, nand_chip->legacy.IO_ADDR_W + (1 << 23));
}
/*
* Read the Device Ready pin.
*/
-static int qong_nand_device_ready(struct mtd_info *mtd)
+static int qong_nand_device_ready(struct nand_chip *chip)
{
return gpio_get_value(IOMUX_TO_GPIO(MX31_PIN_NFRB));
}
-static void qong_nand_select_chip(struct mtd_info *mtd, int chip)
+static void qong_nand_select_chip(struct nand_chip *chip, int cs)
{
- if (chip >= 0)
+ if (cs >= 0)
gpio_set_value(IOMUX_TO_GPIO(MX31_PIN_NFCE_B), 0);
else
gpio_set_value(IOMUX_TO_GPIO(MX31_PIN_NFCE_B), 1);
static struct mmci_platform_data mmc_data = {
.ocr_mask = MMC_VDD_32_33|MMC_VDD_33_34,
.status = mmc_status,
- .gpio_wp = -1,
- .gpio_cd = -1,
};
static u64 notrace intcp_read_sched_clock(void)
#include <linux/mtd/mtd.h>
#include <linux/mtd/rawnand.h>
#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <linux/delay.h>
#include <linux/gpio.h>
#include <asm/types.h>
};
static void
-ixdp425_flash_nand_cmd_ctrl(struct mtd_info *mtd, int cmd, unsigned int ctrl)
+ixdp425_flash_nand_cmd_ctrl(struct nand_chip *this, int cmd, unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
int offset = (int)nand_get_controller_data(this);
if (ctrl & NAND_CTRL_CHANGE) {
}
if (cmd != NAND_CMD_NONE)
- writeb(cmd, this->IO_ADDR_W + offset);
+ writeb(cmd, this->legacy.IO_ADDR_W + offset);
}
static struct platform_nand_data ixdp425_flash_nand_data = {
#include <linux/platform_device.h>
#include <linux/io.h>
#include <linux/gpio-pxa.h>
+#include <linux/gpio/machine.h>
#include <linux/regulator/machine.h>
#include <linux/regulator/max8649.h>
#include <linux/regulator/fixed.h>
static struct fixed_voltage_config brownstone_v_5vp = {
.supply_name = "v_5vp",
.microvolts = 5000000,
- .gpio = GPIO_5V_ENABLE,
.enable_high = 1,
.enabled_at_boot = 1,
.init_data = &brownstone_v_5vp_data,
},
};
+static struct gpiod_lookup_table brownstone_v_5vp_gpiod_table = {
+ .dev_id = "reg-fixed-voltage.1", /* .id set to 1 above */
+ .table = {
+ GPIO_LOOKUP("gpio-pxa", GPIO_5V_ENABLE,
+ NULL, GPIO_ACTIVE_HIGH),
+ { },
+ },
+};
+
static struct max8925_platform_data brownstone_max8925_info = {
.irq_base = MMP_NR_IRQS,
};
mmp2_add_isram(&mmp2_isram_platdata);
/* enable 5v regulator */
+ gpiod_add_lookup_table(&brownstone_v_5vp_gpiod_table);
platform_device_register(&brownstone_v_5vp_device);
}
static struct fixed_voltage_config modem_nreset_config = {
.supply_name = "modem_nreset",
.microvolts = 3300000,
- .gpio = AMS_DELTA_GPIO_PIN_MODEM_NRESET,
.startup_delay = 25000,
.enable_high = 1,
.enabled_at_boot = 1,
},
};
+static struct gpiod_lookup_table ams_delta_nreset_gpiod_table = {
+ .dev_id = "reg-fixed-voltage",
+ .table = {
+ GPIO_LOOKUP(LATCH2_LABEL, LATCH2_PIN_MODEM_NRESET,
+ NULL, GPIO_ACTIVE_HIGH),
+ { },
+ },
+};
+
struct modem_private_data {
struct regulator *regulator;
};
static struct fixed_voltage_config keybrd_pwr_config = {
.supply_name = "keybrd_pwr",
.microvolts = 5000000,
- .gpio = AMS_DELTA_GPIO_PIN_KEYBRD_PWR,
.enable_high = 1,
.init_data = &keybrd_pwr_initdata,
};
};
static struct gpiod_lookup_table *ams_delta_gpio_tables[] __initdata = {
+ &ams_delta_nreset_gpiod_table,
&ams_delta_audio_gpio_table,
&keybrd_pwr_gpio_table,
&ams_delta_lcd_gpio_table,
#include <linux/platform_device.h>
#include <linux/delay.h>
#include <linux/mtd/mtd.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <linux/mtd/physmap.h>
#include <linux/input.h>
#include <linux/smc91x.h>
#define FSAMPLE_NAND_RB_GPIO_PIN 62
-static int nand_dev_ready(struct mtd_info *mtd)
+static int nand_dev_ready(struct nand_chip *chip)
{
return gpio_get_value(FSAMPLE_NAND_RB_GPIO_PIN);
}
#include <linux/delay.h>
#include <linux/i2c.h>
#include <linux/mtd/mtd.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <linux/mtd/physmap.h>
#include <linux/input.h>
#include <linux/mfd/tps65010.h>
#define H2_NAND_RB_GPIO_PIN 62
-static int h2_nand_dev_ready(struct mtd_info *mtd)
+static int h2_nand_dev_ready(struct nand_chip *chip)
{
return gpio_get_value(H2_NAND_RB_GPIO_PIN);
}
#include <linux/workqueue.h>
#include <linux/i2c.h>
#include <linux/mtd/mtd.h>
-#include <linux/mtd/rawnand.h>
+#include <linux/mtd/platnand.h>
#include <linux/mtd/partitions.h>
#include <linux/mtd/physmap.h>
#include <linux/input.h>
#define H3_NAND_RB_GPIO_PIN 10
-static int nand_dev_ready(struct mtd_info *mtd)
+static int nand_dev_ready(struct nand_chip *chip)
{
return gpio_get_value(H3_NAND_RB_GPIO_PIN);
}
#include "common.h"
-void omap1_nand_cmd_ctl(struct mtd_info *mtd, int cmd, unsigned int ctrl)
+void omap1_nand_cmd_ctl(struct nand_chip *this, int cmd, unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
unsigned long mask;
if (cmd == NAND_CMD_NONE)
if (ctrl & NAND_ALE)
mask |= 0x04;
- writeb(cmd, this->IO_ADDR_W + mask);
+ writeb(cmd, this->legacy.IO_ADDR_W + mask);
}
#include <linux/platform_device.h>
#include <linux/delay.h>
#include <linux/mtd/mtd.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <linux/mtd/physmap.h>
#include <linux/input.h>
#include <linux/smc91x.h>
#define P2_NAND_RB_GPIO_PIN 62
-static int nand_dev_ready(struct mtd_info *mtd)
+static int nand_dev_ready(struct nand_chip *chip)
{
return gpio_get_value(P2_NAND_RB_GPIO_PIN);
}
#ifndef __ARCH_ARM_MACH_OMAP1_COMMON_H
#define __ARCH_ARM_MACH_OMAP1_COMMON_H
-#include <linux/mtd/mtd.h>
#include <linux/platform_data/i2c-omap.h>
#include <linux/reboot.h>
extern void __init omap_check_revision(void);
-extern void omap1_nand_cmd_ctl(struct mtd_info *mtd, int cmd,
+struct nand_chip;
+extern void omap1_nand_cmd_ctl(struct nand_chip *this, int cmd,
unsigned int ctrl);
extern void omap1_timer_init(void);
u8 mmc; /* controller 1/2/3 */
u32 caps; /* 4/8 wires and any additional host
* capabilities OR'd (ref. linux/mmc/host.h) */
- int gpio_cd; /* or -EINVAL */
- int gpio_wp; /* or -EINVAL */
struct platform_device *pdev; /* mmc controller instance */
/* init some special card */
void (*init_card)(struct mmc_card *card);
#include <linux/clk.h>
#include <linux/davinci_emac.h>
#include <linux/gpio.h>
+#include <linux/gpio/machine.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/of_platform.h>
static struct fixed_voltage_config pandora_vwlan = {
.supply_name = "vwlan",
.microvolts = 1800000, /* 1.8V */
- .gpio = PANDORA_WIFI_NRESET_GPIO,
.startup_delay = 50000, /* 50ms */
.enable_high = 1,
.init_data = &pandora_vmmc3,
},
};
+static struct gpiod_lookup_table pandora_vwlan_gpiod_table = {
+ .dev_id = "reg-fixed-voltage.1",
+ .table = {
+ /*
+ * As this is a low GPIO number it should be at the first
+ * GPIO bank.
+ */
+ GPIO_LOOKUP("gpio-0-31", PANDORA_WIFI_NRESET_GPIO,
+ NULL, GPIO_ACTIVE_HIGH),
+ { },
+ },
+};
+
static void pandora_wl1251_init_card(struct mmc_card *card)
{
/*
{
.mmc = 3,
.caps = MMC_CAP_4_BIT_DATA | MMC_CAP_POWER_OFF_CARD,
- .gpio_cd = -EINVAL,
- .gpio_wp = -EINVAL,
.init_card = pandora_wl1251_init_card,
},
{} /* Terminator */
static void __init omap3_pandora_legacy_init(void)
{
platform_device_register(&pandora_backlight);
+ gpiod_add_lookup_table(&pandora_vwlan_gpiod_table);
platform_device_register(&pandora_vwlan_device);
omap_hsmmc_init(pandora_mmc3);
omap_hsmmc_late_init(pandora_mmc3);
* published by the Free Software Foundation.
*/
+#include <linux/cpu_pm.h>
#include <linux/suspend.h>
#include <linux/sched.h>
#include <linux/proc_fs.h>
#include <linux/clk-provider.h>
#include <linux/irq.h>
#include <linux/time.h>
-#include <linux/gpio.h>
-#include <linux/platform_data/gpio-omap.h>
#include <asm/fncpy.h>
l = omap_ctrl_readl(OMAP2_CONTROL_DEVCONF0) | OMAP24XX_USBSTANDBYCTRL;
omap_ctrl_writel(l, OMAP2_CONTROL_DEVCONF0);
- omap2_gpio_prepare_for_idle(0);
+ cpu_cluster_pm_enter();
/* One last check for pending IRQs to avoid extra latency due
* to sleeping unnecessarily. */
OMAP_SDRC_REGADDR(SDRC_POWER));
no_sleep:
- omap2_gpio_resume_after_idle();
+ cpu_cluster_pm_exit();
clk_enable(osc_ck);
* published by the Free Software Foundation.
*/
+#include <linux/cpu_pm.h>
#include <linux/pm.h>
#include <linux/suspend.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/list.h>
#include <linux/err.h>
-#include <linux/gpio.h>
#include <linux/clk.h>
#include <linux/delay.h>
#include <linux/slab.h>
#include <linux/omap-dma.h>
#include <linux/omap-gpmc.h>
-#include <linux/platform_data/gpio-omap.h>
#include <trace/events/power.h>
int mpu_next_state = PWRDM_POWER_ON;
int per_next_state = PWRDM_POWER_ON;
int core_next_state = PWRDM_POWER_ON;
- int per_going_off;
u32 sdrc_pwr = 0;
mpu_next_state = pwrdm_read_next_pwrst(mpu_pwrdm);
pwrdm_pre_transition(NULL);
/* PER */
- if (per_next_state < PWRDM_POWER_ON) {
- per_going_off = (per_next_state == PWRDM_POWER_OFF) ? 1 : 0;
- omap2_gpio_prepare_for_idle(per_going_off);
- }
+ if (per_next_state == PWRDM_POWER_OFF)
+ cpu_cluster_pm_enter();
/* CORE */
if (core_next_state < PWRDM_POWER_ON) {
pwrdm_post_transition(NULL);
/* PER */
- if (per_next_state < PWRDM_POWER_ON)
- omap2_gpio_resume_after_idle();
+ if (per_next_state == PWRDM_POWER_OFF)
+ cpu_cluster_pm_exit();
}
static void omap3_pm_idle(void)
#include <linux/platform_device.h>
#include <linux/mv643xx_eth.h>
#include <linux/ata_platform.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <linux/timeriomem-rng.h>
#include <asm/mach-types.h>
#include <asm/mach/arch.h>
* NAND_CLE: bit 1 -> bit 1
* NAND_ALE: bit 2 -> bit 0
*/
-static void ts78xx_ts_nand_cmd_ctrl(struct mtd_info *mtd, int cmd,
- unsigned int ctrl)
+static void ts78xx_ts_nand_cmd_ctrl(struct nand_chip *this, int cmd,
+ unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
-
if (ctrl & NAND_CTRL_CHANGE) {
unsigned char bits;
}
if (cmd != NAND_CMD_NONE)
- writeb(cmd, this->IO_ADDR_W);
+ writeb(cmd, this->legacy.IO_ADDR_W);
}
-static int ts78xx_ts_nand_dev_ready(struct mtd_info *mtd)
+static int ts78xx_ts_nand_dev_ready(struct nand_chip *chip)
{
return readb(TS_NAND_CTRL) & 0x20;
}
-static void ts78xx_ts_nand_write_buf(struct mtd_info *mtd,
- const uint8_t *buf, int len)
+static void ts78xx_ts_nand_write_buf(struct nand_chip *chip,
+ const uint8_t *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
- void __iomem *io_base = chip->IO_ADDR_W;
+ void __iomem *io_base = chip->legacy.IO_ADDR_W;
unsigned long off = ((unsigned long)buf & 3);
int sz;
writesb(io_base, buf, len);
}
-static void ts78xx_ts_nand_read_buf(struct mtd_info *mtd,
- uint8_t *buf, int len)
+static void ts78xx_ts_nand_read_buf(struct nand_chip *chip,
+ uint8_t *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
- void __iomem *io_base = chip->IO_ADDR_R;
+ void __iomem *io_base = chip->legacy.IO_ADDR_R;
unsigned long off = ((unsigned long)buf & 3);
int sz;
#include <linux/ioport.h>
#include <linux/ucb1400.h>
#include <linux/mtd/mtd.h>
-#include <linux/mtd/partitions.h>
#include <linux/types.h>
#include <linux/platform_data/pcf857x.h>
#include <linux/platform_data/i2c-pxa.h>
-#include <linux/mtd/rawnand.h>
+#include <linux/mtd/platnand.h>
#include <linux/mtd/physmap.h>
#include <linux/regulator/max1586.h>
* NAND
******************************************************************************/
#if defined(CONFIG_MTD_NAND_PLATFORM)||defined(CONFIG_MTD_NAND_PLATFORM_MODULE)
-static void balloon3_nand_cmd_ctl(struct mtd_info *mtd, int cmd, unsigned int ctrl)
+static void balloon3_nand_cmd_ctl(struct nand_chip *this, int cmd,
+ unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
uint8_t balloon3_ctl_set = 0, balloon3_ctl_clr = 0;
if (ctrl & NAND_CTRL_CHANGE) {
}
if (cmd != NAND_CMD_NONE)
- writeb(cmd, this->IO_ADDR_W);
+ writeb(cmd, this->legacy.IO_ADDR_W);
}
-static void balloon3_nand_select_chip(struct mtd_info *mtd, int chip)
+static void balloon3_nand_select_chip(struct nand_chip *this, int chip)
{
if (chip < 0 || chip > 3)
return;
BALLOON3_NAND_CONTROL_REG);
}
-static int balloon3_nand_dev_ready(struct mtd_info *mtd)
+static int balloon3_nand_dev_ready(struct nand_chip *this)
{
return __raw_readl(BALLOON3_NAND_STAT_REG) & BALLOON3_NAND_STAT_RNB;
}
#include <linux/dm9000.h>
#include <linux/platform_data/rtc-v3020.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <linux/mtd/physmap.h>
#include <linux/input.h>
#include <linux/gpio_keys.h>
}
/* hardware specific access to control-lines */
-static void em_x270_nand_cmd_ctl(struct mtd_info *mtd, int dat,
+static void em_x270_nand_cmd_ctl(struct nand_chip *this, int dat,
unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- unsigned long nandaddr = (unsigned long)this->IO_ADDR_W;
+ unsigned long nandaddr = (unsigned long)this->legacy.IO_ADDR_W;
dsb();
}
dsb();
- this->IO_ADDR_W = (void __iomem *)nandaddr;
+ this->legacy.IO_ADDR_W = (void __iomem *)nandaddr;
if (dat != NAND_CMD_NONE)
- writel(dat, this->IO_ADDR_W);
+ writel(dat, this->legacy.IO_ADDR_W);
dsb();
}
/* read device ready pin */
-static int em_x270_nand_device_ready(struct mtd_info *mtd)
+static int em_x270_nand_device_ready(struct nand_chip *this)
{
dsb();
.supply_name = "camera_vdd",
.input_supply = "vcc cam",
.microvolts = 2800000,
- .gpio = -1,
.enable_high = 0,
.init_data = &camera_dummy_initdata,
};
#include <linux/regulator/fixed.h>
#include <linux/input.h>
#include <linux/gpio.h>
+#include <linux/gpio/machine.h>
#include <linux/gpio_keys.h>
#include <linux/leds-lp3944.h>
#include <linux/platform_data/i2c-pxa.h>
#if defined(CONFIG_MACH_EZX_A780) || defined(CONFIG_MACH_EZX_A910)
/* camera */
-static struct regulator_consumer_supply camera_dummy_supplies[] = {
+static struct regulator_consumer_supply camera_regulator_supplies[] = {
REGULATOR_SUPPLY("vdd", "0-005d"),
};
-static struct regulator_init_data camera_dummy_initdata = {
- .consumer_supplies = camera_dummy_supplies,
- .num_consumer_supplies = ARRAY_SIZE(camera_dummy_supplies),
+static struct regulator_init_data camera_regulator_initdata = {
+ .consumer_supplies = camera_regulator_supplies,
+ .num_consumer_supplies = ARRAY_SIZE(camera_regulator_supplies),
.constraints = {
.valid_ops_mask = REGULATOR_CHANGE_STATUS,
},
};
-static struct fixed_voltage_config camera_dummy_config = {
+static struct fixed_voltage_config camera_regulator_config = {
.supply_name = "camera_vdd",
.microvolts = 2800000,
- .gpio = GPIO50_nCAM_EN,
.enable_high = 0,
- .init_data = &camera_dummy_initdata,
+ .init_data = &camera_regulator_initdata,
};
-static struct platform_device camera_supply_dummy_device = {
+static struct platform_device camera_supply_regulator_device = {
.name = "reg-fixed-voltage",
.id = 1,
.dev = {
- .platform_data = &camera_dummy_config,
+ .platform_data = &camera_regulator_config,
+ },
+};
+
+static struct gpiod_lookup_table camera_supply_gpiod_table = {
+ .dev_id = "reg-fixed-voltage.1",
+ .table = {
+ GPIO_LOOKUP("gpio-pxa", GPIO50_nCAM_EN,
+ NULL, GPIO_ACTIVE_HIGH),
+ { },
},
};
#endif
static struct platform_device *a780_devices[] __initdata = {
&a780_gpio_keys,
- &camera_supply_dummy_device,
+ &camera_supply_regulator_device,
};
static void __init a780_init(void)
if (a780_camera_init() == 0)
pxa_set_camera_info(&a780_pxacamera_platform_data);
+ gpiod_add_lookup_table(&camera_supply_gpiod_table);
pwm_add_table(ezx_pwm_lookup, ARRAY_SIZE(ezx_pwm_lookup));
platform_add_devices(ARRAY_AND_SIZE(ezx_devices));
platform_add_devices(ARRAY_AND_SIZE(a780_devices));
static struct platform_device *a910_devices[] __initdata = {
&a910_gpio_keys,
- &camera_supply_dummy_device,
+ &camera_supply_regulator_device,
};
static void __init a910_init(void)
if (a910_camera_init() == 0)
pxa_set_camera_info(&a910_pxacamera_platform_data);
+ gpiod_add_lookup_table(&camera_supply_gpiod_table);
pwm_add_table(ezx_pwm_lookup, ARRAY_SIZE(ezx_pwm_lookup));
platform_add_devices(ARRAY_AND_SIZE(ezx_devices));
platform_add_devices(ARRAY_AND_SIZE(a910_devices));
#include <linux/platform_device.h>
#include <linux/delay.h>
#include <linux/gpio.h>
+#include <linux/gpio/machine.h>
#include <linux/gpio_keys.h>
#include <linux/input.h>
#include <linux/mfd/htc-pasic3.h>
static struct fixed_voltage_config vads7846 = {
.supply_name = "vads7846",
.microvolts = 3300000, /* probably */
- .gpio = -EINVAL,
.startup_delay = 0,
.init_data = &vads7846_regulator,
};
platform_device_register(&palmtreo_leds);
}
-/******************************************************************************
- * diskonchip docg4 flash
- ******************************************************************************/
-#if defined(CONFIG_MACH_TREO680)
-/* REVISIT: does the centro have this device also? */
-#if IS_ENABLED(CONFIG_MTD_NAND_DOCG4)
-static struct resource docg4_resources[] = {
- {
- .start = 0x00000000,
- .end = 0x00001FFF,
- .flags = IORESOURCE_MEM,
- },
-};
-
-static struct platform_device treo680_docg4_flash = {
- .name = "docg4",
- .id = -1,
- .resource = docg4_resources,
- .num_resources = ARRAY_SIZE(docg4_resources),
-};
-
-static void __init treo680_docg4_flash_init(void)
-{
- platform_device_register(&treo680_docg4_flash);
-}
-#else
-static inline void treo680_docg4_flash_init(void) {}
-#endif
-#endif
-
/******************************************************************************
* Machine init
******************************************************************************/
treo680_gpio_init();
palm27x_mmc_init(GPIO_NR_TREO_SD_DETECT_N, GPIO_NR_TREO680_SD_READONLY,
GPIO_NR_TREO680_SD_POWER, 0);
- treo680_docg4_flash_init();
}
#endif
#include <linux/wm97xx.h>
#include <linux/power_supply.h>
#include <linux/usb/gpio_vbus.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/physmap.h>
******************************************************************************/
#if defined(CONFIG_MTD_NAND_PLATFORM) || \
defined(CONFIG_MTD_NAND_PLATFORM_MODULE)
-static void palmtx_nand_cmd_ctl(struct mtd_info *mtd, int cmd,
- unsigned int ctrl)
+static void palmtx_nand_cmd_ctl(struct nand_chip *this, int cmd,
+ unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- char __iomem *nandaddr = this->IO_ADDR_W;
+ char __iomem *nandaddr = this->legacy.IO_ADDR_W;
if (cmd == NAND_CMD_NONE)
return;
static struct fixed_voltage_config audio_va_config = {
.supply_name = "audio_va",
.microvolts = 5000000,
- .gpio = GPIO_AUDIO_VA_ENABLE,
.enable_high = 1,
.enabled_at_boot = 0,
.init_data = &audio_va_initdata,
},
};
+static struct gpiod_lookup_table audio_va_gpiod_table = {
+ .dev_id = "reg-fixed-voltage.0",
+ .table = {
+ GPIO_LOOKUP("gpio-pxa", GPIO_AUDIO_VA_ENABLE,
+ NULL, GPIO_ACTIVE_HIGH),
+ { },
+ },
+};
+
/* Dummy supplies for Codec's VD/VLC */
static struct regulator_consumer_supply audio_dummy_supplies[] = {
static struct fixed_voltage_config audio_dummy_config = {
.supply_name = "audio_vd",
.microvolts = 3300000,
- .gpio = -1,
.init_data = &audio_dummy_initdata,
};
else
gpio_direction_output(GPIO_MCLK_RESET, 1);
+ gpiod_add_lookup_table(&audio_va_gpiod_table);
platform_add_devices(ARRAY_AND_SIZE(audio_regulator_devices));
}
#include <linux/irq.h>
#include <linux/pm.h>
#include <linux/gpio.h>
+#include <linux/gpio/machine.h>
#include <linux/serial_8250.h>
#include <linux/dm9000.h>
#include <linux/mmc/host.h>
static struct fixed_voltage_config can_regulator_pdata = {
.supply_name = "CAN_SHDN",
.microvolts = 3300000,
- .gpio = ZEUS_CAN_SHDN_GPIO,
.init_data = &can_regulator_init_data,
};
},
};
+static struct gpiod_lookup_table can_regulator_gpiod_table = {
+ .dev_id = "reg-fixed-voltage.0",
+ .table = {
+ GPIO_LOOKUP("gpio-pxa", ZEUS_CAN_SHDN_GPIO,
+ NULL, GPIO_ACTIVE_HIGH),
+ { },
+ },
+};
+
static struct mcp251x_platform_data zeus_mcp2515_pdata = {
.oscillator_frequency = 16*1000*1000,
};
static struct fixed_voltage_config zeus_ohci_regulator_config = {
.supply_name = "vbus2",
.microvolts = 5000000, /* 5.0V */
- .gpio = ZEUS_USB2_PWREN_GPIO,
.enable_high = 1,
.startup_delay = 0,
.init_data = &zeus_ohci_regulator_data,
},
};
+static struct gpiod_lookup_table zeus_ohci_regulator_gpiod_table = {
+ .dev_id = "reg-fixed-voltage.0",
+ .table = {
+ GPIO_LOOKUP("gpio-pxa", ZEUS_USB2_PWREN_GPIO,
+ NULL, GPIO_ACTIVE_HIGH),
+ { },
+ },
+};
+
static struct pxaohci_platform_data zeus_ohci_platform_data = {
.port_mode = PMM_NPS_MODE,
/* Clear Power Control Polarity Low and set Power Sense
pxa2xx_mfp_config(ARRAY_AND_SIZE(zeus_pin_config));
+ gpiod_add_lookup_table(&can_regulator_gpiod_table);
+ gpiod_add_lookup_table(&zeus_ohci_regulator_gpiod_table);
platform_add_devices(zeus_devices, ARRAY_SIZE(zeus_devices));
zeus_register_ohci();
.supply_name = "WALLVDD",
.microvolts = 5000000,
.init_data = &wallvdd_data,
- .gpio = -EINVAL,
};
static struct platform_device wallvdd_device = {
.supply_name = "B_PWR_5V",
.microvolts = 5000000,
.init_data = &smdk6410_b_pwr_5v_data,
- .gpio = -EINVAL,
};
static struct platform_device smdk6410_b_pwr_5v = {
assabet_bcr_gc = gc;
- return gc->base;
+ return 0;
}
/*
.enable_high = 1,
};
+static struct gpiod_lookup_table assabet_cf_vcc_gpio_table = {
+ .dev_id = "reg-fixed-voltage.0",
+ .table = {
+ GPIO_LOOKUP("assabet", 0, NULL, GPIO_ACTIVE_HIGH),
+ { },
+ },
+};
+
static void __init assabet_init(void)
{
/*
neponset_resources, ARRAY_SIZE(neponset_resources));
#endif
} else {
+ gpiod_add_lookup_table(&assabet_cf_vcc_gpio_table);
sa11x0_register_fixed_regulator(0, &assabet_cf_vcc_pdata,
- assabet_cf_vcc_consumers,
- ARRAY_SIZE(assabet_cf_vcc_consumers));
+ assabet_cf_vcc_consumers,
+ ARRAY_SIZE(assabet_cf_vcc_consumers),
+ true);
}
void __init assabet_init_irq(void)
{
- unsigned int assabet_gpio_base;
u32 def_val;
sa1100_init_irq();
*
* This must precede any driver calls to BCR_set() or BCR_clear().
*/
- assabet_gpio_base = assabet_init_gpio((void *)&ASSABET_BCR, def_val);
-
- assabet_cf_vcc_pdata.gpio = assabet_gpio_base + 0;
+ assabet_init_gpio((void *)&ASSABET_BCR, def_val);
}
MACHINE_START(ASSABET, "Intel-Assabet")
int __init sa11x0_register_fixed_regulator(int n,
struct fixed_voltage_config *cfg,
- struct regulator_consumer_supply *supplies, unsigned num_supplies)
+ struct regulator_consumer_supply *supplies, unsigned num_supplies,
+ bool uses_gpio)
{
struct regulator_init_data *id;
if (!cfg->init_data)
return -ENOMEM;
- if (cfg->gpio < 0)
+ if (!uses_gpio)
id->constraints.always_on = 1;
id->constraints.name = cfg->supply_name;
id->constraints.min_uV = cfg->microvolts;
struct fixed_voltage_config;
struct regulator_consumer_supply;
int sa11x0_register_fixed_regulator(int n, struct fixed_voltage_config *cfg,
- struct regulator_consumer_supply *supplies, unsigned num_supplies);
+ struct regulator_consumer_supply *supplies, unsigned num_supplies,
+ bool uses_gpio);
.supply_name = "cf-power",
.microvolts = 3300000,
.enabled_at_boot = 1,
- .gpio = -EINVAL,
};
static void __init shannon_init(void)
{
sa11x0_register_fixed_regulator(0, &shannon_cf_vcc_pdata,
shannon_cf_vcc_consumers,
- ARRAY_SIZE(shannon_cf_vcc_consumers));
+ ARRAY_SIZE(shannon_cf_vcc_consumers),
+ false);
sa11x0_register_pcmcia(0, &shannon_pcmcia0_gpio_table);
sa11x0_register_pcmcia(1, &shannon_pcmcia1_gpio_table);
sa11x0_ppc_configure_mcp();
static struct mmci_platform_data mmc0_plat_data = {
.ocr_mask = MMC_VDD_32_33|MMC_VDD_33_34,
.status = mmc_status,
- .gpio_wp = -1,
- .gpio_cd = -1,
};
static struct mmci_platform_data mmc1_plat_data = {
.ocr_mask = MMC_VDD_32_33|MMC_VDD_33_34,
.status = mmc_status,
- .gpio_wp = -1,
- .gpio_cd = -1,
};
/*
*/
if (attrs & DMA_ATTR_NON_CONSISTENT)
- return dma_direct_alloc(dev, size, dma_handle, gfp, attrs);
+ return dma_direct_alloc_pages(dev, size, dma_handle, gfp,
+ attrs);
ret = dma_alloc_from_global_coherent(size, dma_handle);
unsigned long attrs)
{
if (attrs & DMA_ATTR_NON_CONSISTENT) {
- dma_direct_free(dev, size, cpu_addr, dma_addr, attrs);
+ dma_direct_free_pages(dev, size, cpu_addr, dma_addr, attrs);
} else {
int ret = dma_release_from_global_coherent(get_order(size),
cpu_addr);
if (dma_mmap_from_global_coherent(vma, cpu_addr, size, &ret))
return ret;
- return dma_common_mmap(dev, vma, cpu_addr, dma_addr, size);
+ return dma_common_mmap(dev, vma, cpu_addr, dma_addr, size, attrs);
}
set_dma_ops(dev, dma_ops);
}
-
-void arch_teardown_dma_ops(struct device *dev)
-{
-}
select CLONE_BACKWARDS
select COMMON_CLK
select CPU_PM if (SUSPEND || CPU_IDLE)
+ select CRC32
select DCACHE_WORD_ACCESS
select DMA_DIRECT_OPS
select EDAC_SUPPORT
select HAVE_ARCH_BITREVERSE
select HAVE_ARCH_HUGE_VMAP
select HAVE_ARCH_JUMP_LABEL
+ select HAVE_ARCH_JUMP_LABEL_RELATIVE
select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48)
select HAVE_ARCH_KGDB
select HAVE_ARCH_MMAP_RND_BITS
select HAVE_PERF_USER_STACK_DUMP
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_RCU_TABLE_FREE
+ select HAVE_RCU_TABLE_INVALIDATE
select HAVE_RSEQ
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
If unsure, say Y.
+config ARM64_ERRATUM_1188873
+ bool "Cortex-A76: MRC read following MRRC read of specific Generic Timer in AArch32 might give incorrect result"
+ default y
+ select ARM_ARCH_TIMER_OOL_WORKAROUND
+ help
+ This option adds work arounds for ARM Cortex-A76 erratum 1188873
+
+ Affected Cortex-A76 cores (r0p0, r1p0, r2p0) could cause
+ register corruption when accessing the timer registers from
+ AArch32 userspace.
+
+ If unsure, say Y.
+
config CAVIUM_ERRATUM_22375
bool "Cavium erratum 22375, 24313"
default y
config ARCH_SUPPORTS_DEBUG_PAGEALLOC
def_bool y
-config ARCH_HAS_HOLES_MEMORYMODEL
- def_bool y if SPARSEMEM
-
config ARCH_SPARSEMEM_ENABLE
def_bool y
select SPARSEMEM_VMEMMAP_ENABLE
def_bool !NUMA
config HAVE_ARCH_PFN_VALID
- def_bool ARCH_HAS_HOLES_MEMORYMODEL || !SPARSEMEM
+ def_bool y
config HW_PERF_EVENTS
def_bool y
and access the new registers if the system supports the extension.
Platform RAS features may additionally depend on firmware support.
+config ARM64_CNP
+ bool "Enable support for Common Not Private (CNP) translations"
+ default y
+ depends on ARM64_PAN || !ARM64_SW_TTBR0_PAN
+ help
+ Common Not Private (CNP) allows translation table entries to
+ be shared between different PEs in the same inner shareable
+ domain, so the hardware can use this fact to optimise the
+ caching of such entries in the TLB.
+
+ Selecting this option allows the CNP feature to be detected
+ at runtime, and does not affect PEs that do not implement
+ this feature.
+
endmenu
config ARM64_SVE
ldr \rd, [\rn, #MM_CONTEXT_ID]
.endm
/*
- * read_ctr - read CTR_EL0. If the system has mismatched
- * cache line sizes, provide the system wide safe value
- * from arm64_ftr_reg_ctrel0.sys_val
+ * read_ctr - read CTR_EL0. If the system has mismatched register fields,
+ * provide the system wide safe value from arm64_ftr_reg_ctrel0.sys_val
*/
.macro read_ctr, reg
-alternative_if_not ARM64_MISMATCHED_CACHE_LINE_SIZE
+alternative_if_not ARM64_MISMATCHED_CACHE_TYPE
mrs \reg, ctr_el0 // read CTR
nop
alternative_else
#define L1_CACHE_SHIFT (6)
#define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT)
+
+#define CLIDR_LOUU_SHIFT 27
+#define CLIDR_LOC_SHIFT 24
+#define CLIDR_LOUIS_SHIFT 21
+
+#define CLIDR_LOUU(clidr) (((clidr) >> CLIDR_LOUU_SHIFT) & 0x7)
+#define CLIDR_LOC(clidr) (((clidr) >> CLIDR_LOC_SHIFT) & 0x7)
+#define CLIDR_LOUIS(clidr) (((clidr) >> CLIDR_LOUIS_SHIFT) & 0x7)
+
/*
* Memory returned by kmalloc() may be used for DMA, so we must make
* sure that all such allocations are cache aligned. Otherwise,
return cwg ? 4 << cwg : ARCH_DMA_MINALIGN;
}
+/*
+ * Read the effective value of CTR_EL0.
+ *
+ * According to ARM ARM for ARMv8-A (ARM DDI 0487C.a),
+ * section D10.2.33 "CTR_EL0, Cache Type Register" :
+ *
+ * CTR_EL0.IDC reports the data cache clean requirements for
+ * instruction to data coherence.
+ *
+ * 0 - dcache clean to PoU is required unless :
+ * (CLIDR_EL1.LoC == 0) || (CLIDR_EL1.LoUIS == 0 && CLIDR_EL1.LoUU == 0)
+ * 1 - dcache clean to PoU is not required for i-to-d coherence.
+ *
+ * This routine provides the CTR_EL0 with the IDC field updated to the
+ * effective state.
+ */
+static inline u32 __attribute_const__ read_cpuid_effective_cachetype(void)
+{
+ u32 ctr = read_cpuid_cachetype();
+
+ if (!(ctr & BIT(CTR_IDC_SHIFT))) {
+ u64 clidr = read_sysreg(clidr_el1);
+
+ if (CLIDR_LOC(clidr) == 0 ||
+ (CLIDR_LOUIS(clidr) == 0 && CLIDR_LOUU(clidr) == 0))
+ ctr |= BIT(CTR_IDC_SHIFT);
+ }
+
+ return ctr;
+}
+
#endif /* __ASSEMBLY__ */
#endif
}
#define compat_user_stack_pointer() (user_stack_pointer(task_pt_regs(current)))
+#define COMPAT_MINSIGSTKSZ 2048
static inline void __user *arch_compat_alloc_user_space(long len)
{
+++ /dev/null
-/*
- * Based on arch/arm/include/asm/compiler.h
- *
- * Copyright (C) 2012 ARM Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-#ifndef __ASM_COMPILER_H
-#define __ASM_COMPILER_H
-
-/*
- * This is used to ensure the compiler did actually allocate the register we
- * asked it for some inline assembly sequences. Apparently we can't trust the
- * compiler from one version to another so a bit of paranoia won't hurt. This
- * string is meant to be concatenated with the inline asm string and will
- * cause compilation to stop on mismatch. (for details, see gcc PR 15089)
- */
-#define __asmeq(x, y) ".ifnc " x "," y " ; .err ; .endif\n\t"
-
-#endif /* __ASM_COMPILER_H */
#define ARM64_WORKAROUND_CAVIUM_27456 12
#define ARM64_HAS_32BIT_EL0 13
#define ARM64_HARDEN_EL2_VECTORS 14
-#define ARM64_MISMATCHED_CACHE_LINE_SIZE 15
+#define ARM64_HAS_CNP 15
#define ARM64_HAS_NO_FPSIMD 16
#define ARM64_WORKAROUND_REPEAT_TLBI 17
#define ARM64_WORKAROUND_QCOM_FALKOR_E1003 18
#define ARM64_SSBD 30
#define ARM64_MISMATCHED_CACHE_TYPE 31
#define ARM64_HAS_STAGE2_FWB 32
+#define ARM64_HAS_CRC32 33
+#define ARM64_SSBS 34
+#define ARM64_WORKAROUND_1188873 35
-#define ARM64_NCAPS 33
+#define ARM64_NCAPS 36
#endif /* __ASM_CPUCAPS_H */
/*
* CPU feature detected at boot time based on system-wide value of a
* feature. It is safe for a late CPU to have this feature even though
- * the system hasn't enabled it, although the featuer will not be used
+ * the system hasn't enabled it, although the feature will not be used
* by Linux in this case. If the system has enabled this feature already,
* then every late CPU must have it.
*/
cpus_have_const_cap(ARM64_SVE);
}
+static inline bool system_supports_cnp(void)
+{
+ return IS_ENABLED(CONFIG_ARM64_CNP) &&
+ cpus_have_const_cap(ARM64_HAS_CNP);
+}
+
#define ARM64_SSBD_UNKNOWN -1
#define ARM64_SSBD_FORCE_DISABLE 0
#define ARM64_SSBD_KERNEL 1
static inline void arm64_set_ssbd_mitigation(bool state) {}
#endif
+extern int do_emulate_mrs(struct pt_regs *regs, u32 sys_reg, u32 rt);
#endif /* __ASSEMBLY__ */
#endif
#define ARM_CPU_PART_CORTEX_A75 0xD0A
#define ARM_CPU_PART_CORTEX_A35 0xD04
#define ARM_CPU_PART_CORTEX_A55 0xD05
+#define ARM_CPU_PART_CORTEX_A76 0xD0B
#define APM_CPU_PART_POTENZA 0x000
#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75)
#define MIDR_CORTEX_A35 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A35)
#define MIDR_CORTEX_A55 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A55)
+#define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A76)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
{
unsigned long flags;
- asm volatile(
- "mrs %0, daif // local_daif_save\n"
- : "=r" (flags)
- :
- : "memory");
+ flags = arch_local_save_flags();
+
local_daif_mask();
return flags;
{
if (!arch_irqs_disabled_flags(flags))
trace_hardirqs_on();
- asm volatile(
- "msr daif, %0 // local_daif_restore"
- :
- : "r" (flags)
- : "memory");
+
+ arch_local_irq_restore(flags);
+
if (arch_irqs_disabled_flags(flags))
trace_hardirqs_off();
}
#define ESR_ELx_CV (UL(1) << 24)
#define ESR_ELx_COND_SHIFT (20)
#define ESR_ELx_COND_MASK (UL(0xF) << ESR_ELx_COND_SHIFT)
+#define ESR_ELx_WFx_ISS_TI (UL(1) << 0)
+#define ESR_ELx_WFx_ISS_WFI (UL(0) << 0)
#define ESR_ELx_WFx_ISS_WFE (UL(1) << 0)
#define ESR_ELx_xVC_IMM_MASK ((1UL << 16) - 1)
#define DISR_EL1_ESR_MASK (ESR_ELx_AET | ESR_ELx_EA | ESR_ELx_FSC)
/* ESR value templates for specific events */
+#define ESR_ELx_WFx_MASK (ESR_ELx_EC_MASK | ESR_ELx_WFx_ISS_TI)
+#define ESR_ELx_WFx_WFI_VAL ((ESR_ELx_EC_WFx << ESR_ELx_EC_SHIFT) | \
+ ESR_ELx_WFx_ISS_WFI)
/* BRK instruction trap from AArch64 state */
#define ESR_ELx_VAL_BRK64(imm) \
#define ESR_ELx_SYS64_ISS_SYS_OP_MASK (ESR_ELx_SYS64_ISS_SYS_MASK | \
ESR_ELx_SYS64_ISS_DIR_MASK)
+#define ESR_ELx_SYS64_ISS_RT(esr) \
+ (((esr) & ESR_ELx_SYS64_ISS_RT_MASK) >> ESR_ELx_SYS64_ISS_RT_SHIFT)
/*
* User space cache operations have the following sysreg encoding
* in System instructions.
#define ESR_ELx_SYS64_ISS_EL0_CACHE_OP_VAL \
(ESR_ELx_SYS64_ISS_SYS_VAL(1, 3, 1, 7, 0) | \
ESR_ELx_SYS64_ISS_DIR_WRITE)
+/*
+ * User space MRS operations which are supported for emulation
+ * have the following sysreg encoding in System instructions.
+ * op0 = 3, op1= 0, crn = 0, {crm = 0, 4-7}, READ (L = 1)
+ */
+#define ESR_ELx_SYS64_ISS_SYS_MRS_OP_MASK (ESR_ELx_SYS64_ISS_OP0_MASK | \
+ ESR_ELx_SYS64_ISS_OP1_MASK | \
+ ESR_ELx_SYS64_ISS_CRN_MASK | \
+ ESR_ELx_SYS64_ISS_DIR_MASK)
+#define ESR_ELx_SYS64_ISS_SYS_MRS_OP_VAL \
+ (ESR_ELx_SYS64_ISS_SYS_VAL(3, 0, 0, 0, 0) | \
+ ESR_ELx_SYS64_ISS_DIR_READ)
#define ESR_ELx_SYS64_ISS_SYS_CTR ESR_ELx_SYS64_ISS_SYS_VAL(3, 3, 1, 0, 0)
#define ESR_ELx_SYS64_ISS_SYS_CTR_READ (ESR_ELx_SYS64_ISS_SYS_CTR | \
#define ESR_ELx_FP_EXC_TFV (UL(1) << 23)
+/*
+ * ISS field definitions for CP15 accesses
+ */
+#define ESR_ELx_CP15_32_ISS_DIR_MASK 0x1
+#define ESR_ELx_CP15_32_ISS_DIR_READ 0x1
+#define ESR_ELx_CP15_32_ISS_DIR_WRITE 0x0
+
+#define ESR_ELx_CP15_32_ISS_RT_SHIFT 5
+#define ESR_ELx_CP15_32_ISS_RT_MASK (UL(0x1f) << ESR_ELx_CP15_32_ISS_RT_SHIFT)
+#define ESR_ELx_CP15_32_ISS_CRM_SHIFT 1
+#define ESR_ELx_CP15_32_ISS_CRM_MASK (UL(0xf) << ESR_ELx_CP15_32_ISS_CRM_SHIFT)
+#define ESR_ELx_CP15_32_ISS_CRN_SHIFT 10
+#define ESR_ELx_CP15_32_ISS_CRN_MASK (UL(0xf) << ESR_ELx_CP15_32_ISS_CRN_SHIFT)
+#define ESR_ELx_CP15_32_ISS_OP1_SHIFT 14
+#define ESR_ELx_CP15_32_ISS_OP1_MASK (UL(0x7) << ESR_ELx_CP15_32_ISS_OP1_SHIFT)
+#define ESR_ELx_CP15_32_ISS_OP2_SHIFT 17
+#define ESR_ELx_CP15_32_ISS_OP2_MASK (UL(0x7) << ESR_ELx_CP15_32_ISS_OP2_SHIFT)
+
+#define ESR_ELx_CP15_32_ISS_SYS_MASK (ESR_ELx_CP15_32_ISS_OP1_MASK | \
+ ESR_ELx_CP15_32_ISS_OP2_MASK | \
+ ESR_ELx_CP15_32_ISS_CRN_MASK | \
+ ESR_ELx_CP15_32_ISS_CRM_MASK | \
+ ESR_ELx_CP15_32_ISS_DIR_MASK)
+#define ESR_ELx_CP15_32_ISS_SYS_VAL(op1, op2, crn, crm) \
+ (((op1) << ESR_ELx_CP15_32_ISS_OP1_SHIFT) | \
+ ((op2) << ESR_ELx_CP15_32_ISS_OP2_SHIFT) | \
+ ((crn) << ESR_ELx_CP15_32_ISS_CRN_SHIFT) | \
+ ((crm) << ESR_ELx_CP15_32_ISS_CRM_SHIFT))
+
+#define ESR_ELx_CP15_64_ISS_DIR_MASK 0x1
+#define ESR_ELx_CP15_64_ISS_DIR_READ 0x1
+#define ESR_ELx_CP15_64_ISS_DIR_WRITE 0x0
+
+#define ESR_ELx_CP15_64_ISS_RT_SHIFT 5
+#define ESR_ELx_CP15_64_ISS_RT_MASK (UL(0x1f) << ESR_ELx_CP15_64_ISS_RT_SHIFT)
+
+#define ESR_ELx_CP15_64_ISS_RT2_SHIFT 10
+#define ESR_ELx_CP15_64_ISS_RT2_MASK (UL(0x1f) << ESR_ELx_CP15_64_ISS_RT2_SHIFT)
+
+#define ESR_ELx_CP15_64_ISS_OP1_SHIFT 16
+#define ESR_ELx_CP15_64_ISS_OP1_MASK (UL(0xf) << ESR_ELx_CP15_64_ISS_OP1_SHIFT)
+#define ESR_ELx_CP15_64_ISS_CRM_SHIFT 1
+#define ESR_ELx_CP15_64_ISS_CRM_MASK (UL(0xf) << ESR_ELx_CP15_64_ISS_CRM_SHIFT)
+
+#define ESR_ELx_CP15_64_ISS_SYS_VAL(op1, crm) \
+ (((op1) << ESR_ELx_CP15_64_ISS_OP1_SHIFT) | \
+ ((crm) << ESR_ELx_CP15_64_ISS_CRM_SHIFT))
+
+#define ESR_ELx_CP15_64_ISS_SYS_MASK (ESR_ELx_CP15_64_ISS_OP1_MASK | \
+ ESR_ELx_CP15_64_ISS_CRM_MASK | \
+ ESR_ELx_CP15_64_ISS_DIR_MASK)
+
+#define ESR_ELx_CP15_64_ISS_SYS_CNTVCT (ESR_ELx_CP15_64_ISS_SYS_VAL(1, 14) | \
+ ESR_ELx_CP15_64_ISS_DIR_READ)
+
+#define ESR_ELx_CP15_32_ISS_SYS_CNTFRQ (ESR_ELx_CP15_32_ISS_SYS_VAL(0, 0, 14, 0) |\
+ ESR_ELx_CP15_32_ISS_DIR_READ)
+
#ifndef __ASSEMBLY__
#include <asm/types.h>
#include <asm/alternative.h>
#include <asm/cpufeature.h>
-#include <xen/xen.h>
-
/*
* Generic IO read/write. These perform native-endian accesses.
*/
extern int devmem_is_allowed(unsigned long pfn);
-struct bio_vec;
-extern bool xen_biovec_phys_mergeable(const struct bio_vec *vec1,
- const struct bio_vec *vec2);
-#define BIOVEC_PHYS_MERGEABLE(vec1, vec2) \
- (__BIOVEC_PHYS_MERGEABLE(vec1, vec2) && \
- (!xen_domain() || xen_biovec_phys_mergeable(vec1, vec2)))
-
#endif /* __KERNEL__ */
#endif /* __ASM_IO_H */
#define JUMP_LABEL_NOP_SIZE AARCH64_INSN_SIZE
-static __always_inline bool arch_static_branch(struct static_key *key, bool branch)
+static __always_inline bool arch_static_branch(struct static_key *key,
+ bool branch)
{
- asm_volatile_goto("1: nop\n\t"
- ".pushsection __jump_table, \"aw\"\n\t"
- ".align 3\n\t"
- ".quad 1b, %l[l_yes], %c0\n\t"
- ".popsection\n\t"
+ asm_volatile_goto(
+ "1: nop \n\t"
+ " .pushsection __jump_table, \"aw\" \n\t"
+ " .align 3 \n\t"
+ " .long 1b - ., %l[l_yes] - . \n\t"
+ " .quad %c0 - . \n\t"
+ " .popsection \n\t"
: : "i"(&((char *)key)[branch]) : : l_yes);
return false;
return true;
}
-static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch)
+static __always_inline bool arch_static_branch_jump(struct static_key *key,
+ bool branch)
{
- asm_volatile_goto("1: b %l[l_yes]\n\t"
- ".pushsection __jump_table, \"aw\"\n\t"
- ".align 3\n\t"
- ".quad 1b, %l[l_yes], %c0\n\t"
- ".popsection\n\t"
+ asm_volatile_goto(
+ "1: b %l[l_yes] \n\t"
+ " .pushsection __jump_table, \"aw\" \n\t"
+ " .align 3 \n\t"
+ " .long 1b - ., %l[l_yes] - . \n\t"
+ " .quad %c0 - . \n\t"
+ " .popsection \n\t"
: : "i"(&((char *)key)[branch]) : : l_yes);
return false;
return true;
}
-typedef u64 jump_label_t;
-
-struct jump_entry {
- jump_label_t code;
- jump_label_t target;
- jump_label_t key;
-};
-
#endif /* __ASSEMBLY__ */
#endif /* __ASM_JUMP_LABEL_H */
+ EARLY_PGDS((vstart), (vend)) /* each PGDIR needs a next level page table */ \
+ EARLY_PUDS((vstart), (vend)) /* each PUD needs a next level page table */ \
+ EARLY_PMDS((vstart), (vend))) /* each PMD needs a next level page table */
-#define SWAPPER_DIR_SIZE (PAGE_SIZE * EARLY_PAGES(KIMAGE_VADDR + TEXT_OFFSET, _end))
+#define INIT_DIR_SIZE (PAGE_SIZE * EARLY_PAGES(KIMAGE_VADDR + TEXT_OFFSET, _end))
#define IDMAP_DIR_SIZE (IDMAP_PGTABLE_LEVELS * PAGE_SIZE)
#ifdef CONFIG_ARM64_SW_TTBR0_PAN
#define VTCR_EL2_FLAGS (VTCR_EL2_COMMON_BITS | VTCR_EL2_TGRAN_FLAGS)
#define VTTBR_X (VTTBR_X_TGRAN_MAGIC - VTCR_EL2_T0SZ_IPA)
+#define VTTBR_CNP_BIT (UL(1))
#define VTTBR_BADDR_MASK (((UL(1) << (PHYS_MASK_SHIFT - VTTBR_X)) - 1) << VTTBR_X)
#define VTTBR_VMID_SHIFT (UL(48))
#define VTTBR_VMID_MASK(size) (_AT(u64, (1 << size) - 1) << VTTBR_VMID_SHIFT)
static inline int kvm_vcpu_sys_get_rt(struct kvm_vcpu *vcpu)
{
u32 esr = kvm_vcpu_get_hsr(vcpu);
- return (esr & ESR_ELx_SYS64_ISS_RT_MASK) >> ESR_ELx_SYS64_ISS_RT_SHIFT;
+ return ESR_ELx_SYS64_ISS_RT(esr);
}
static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu)
DECLARE_PER_CPU(kvm_cpu_context_t, kvm_host_cpu_state);
+void __kvm_enable_ssbs(void);
+
static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr,
unsigned long hyp_stack_ptr,
unsigned long vector_ptr)
*/
BUG_ON(!static_branch_likely(&arm64_const_caps_ready));
__kvm_call_hyp((void *)pgd_ptr, hyp_stack_ptr, vector_ptr, tpidr_el2);
+
+ /*
+ * Disabling SSBD on a non-VHE system requires us to enable SSBS
+ * at EL2.
+ */
+ if (!has_vhe() && this_cpu_has_cap(ARM64_SSBS) &&
+ arm64_get_ssbd_state() == ARM64_SSBD_FORCE_DISABLE) {
+ kvm_call_hyp(__kvm_enable_ssbs);
+ }
}
static inline bool kvm_arch_check_sve_has_vhe(void)
#define kvm_phys_to_vttbr(addr) phys_to_ttbr(addr)
+static inline bool kvm_cpu_has_cnp(void)
+{
+ return system_supports_cnp();
+}
+
#endif /* __ASSEMBLY__ */
#endif /* __ARM64_KVM_MMU_H__ */
extern void *fixmap_remap_fdt(phys_addr_t dt_phys);
extern void mark_linear_text_alias_ro(void);
+#define INIT_MM_CONTEXT(name) \
+ .pgd = init_pg_dir,
+
#endif /* !__ASSEMBLY__ */
#endif
extern ttbr_replace_func idmap_cpu_replace_ttbr1;
ttbr_replace_func *replace_phys;
- phys_addr_t pgd_phys = virt_to_phys(pgdp);
+ /* phys_to_ttbr() zeros lower 2 bits of ttbr with 52-bit PA */
+ phys_addr_t ttbr1 = phys_to_ttbr(virt_to_phys(pgdp));
+
+ if (system_supports_cnp() && !WARN_ON(pgdp != lm_alias(swapper_pg_dir))) {
+ /*
+ * cpu_replace_ttbr1() is used when there's a boot CPU
+ * up (i.e. cpufeature framework is not up yet) and
+ * latter only when we enable CNP via cpufeature's
+ * enable() callback.
+ * Also we rely on the cpu_hwcap bit being set before
+ * calling the enable() function.
+ */
+ ttbr1 |= TTBR_CNP_BIT;
+ }
replace_phys = (void *)__pa_symbol(idmap_cpu_replace_ttbr1);
cpu_install_idmap();
- replace_phys(pgd_phys);
+ replace_phys(ttbr1);
cpu_uninstall_idmap();
}
typedef struct page *pgtable_t;
-#ifdef CONFIG_HAVE_ARCH_PFN_VALID
extern int pfn_valid(unsigned long);
-#endif
#include <asm/memory.h>
#define PHYS_MASK_SHIFT (CONFIG_ARM64_PA_BITS)
#define PHYS_MASK ((UL(1) << PHYS_MASK_SHIFT) - 1)
+#define TTBR_CNP_BIT (UL(1) << 0)
+
/*
* TCR flags.
*/
#define pmd_present(pmd) pte_present(pmd_pte(pmd))
#define pmd_dirty(pmd) pte_dirty(pmd_pte(pmd))
#define pmd_young(pmd) pte_young(pmd_pte(pmd))
+#define pmd_valid(pmd) pte_valid(pmd_pte(pmd))
#define pmd_wrprotect(pmd) pte_pmd(pte_wrprotect(pmd_pte(pmd)))
#define pmd_mkold(pmd) pte_pmd(pte_mkold(pmd_pte(pmd)))
#define pmd_mkwrite(pmd) pte_pmd(pte_mkwrite(pmd_pte(pmd)))
PUD_TYPE_TABLE)
#endif
+extern pgd_t init_pg_dir[PTRS_PER_PGD];
+extern pgd_t init_pg_end[];
+extern pgd_t swapper_pg_dir[PTRS_PER_PGD];
+extern pgd_t idmap_pg_dir[PTRS_PER_PGD];
+extern pgd_t tramp_pg_dir[PTRS_PER_PGD];
+
+extern void set_swapper_pgd(pgd_t *pgdp, pgd_t pgd);
+
+static inline bool in_swapper_pgdir(void *addr)
+{
+ return ((unsigned long)addr & PAGE_MASK) ==
+ ((unsigned long)swapper_pg_dir & PAGE_MASK);
+}
+
static inline void set_pmd(pmd_t *pmdp, pmd_t pmd)
{
+#ifdef __PAGETABLE_PMD_FOLDED
+ if (in_swapper_pgdir(pmdp)) {
+ set_swapper_pgd((pgd_t *)pmdp, __pgd(pmd_val(pmd)));
+ return;
+ }
+#endif /* __PAGETABLE_PMD_FOLDED */
+
WRITE_ONCE(*pmdp, pmd);
- dsb(ishst);
+
+ if (pmd_valid(pmd))
+ dsb(ishst);
}
static inline void pmd_clear(pmd_t *pmdp)
#define pud_none(pud) (!pud_val(pud))
#define pud_bad(pud) (!(pud_val(pud) & PUD_TABLE_BIT))
#define pud_present(pud) pte_present(pud_pte(pud))
+#define pud_valid(pud) pte_valid(pud_pte(pud))
static inline void set_pud(pud_t *pudp, pud_t pud)
{
+#ifdef __PAGETABLE_PUD_FOLDED
+ if (in_swapper_pgdir(pudp)) {
+ set_swapper_pgd((pgd_t *)pudp, __pgd(pud_val(pud)));
+ return;
+ }
+#endif /* __PAGETABLE_PUD_FOLDED */
+
WRITE_ONCE(*pudp, pud);
- dsb(ishst);
+
+ if (pud_valid(pud))
+ dsb(ishst);
}
static inline void pud_clear(pud_t *pudp)
static inline void set_pgd(pgd_t *pgdp, pgd_t pgd)
{
+ if (in_swapper_pgdir(pgdp)) {
+ set_swapper_pgd(pgdp, pgd);
+ return;
+ }
+
WRITE_ONCE(*pgdp, pgd);
dsb(ishst);
}
}
#endif
-extern pgd_t swapper_pg_dir[PTRS_PER_PGD];
-extern pgd_t swapper_pg_end[];
-extern pgd_t idmap_pg_dir[PTRS_PER_PGD];
-extern pgd_t tramp_pg_dir[PTRS_PER_PGD];
-
/*
* Encode and decode a swap entry:
* bits 0-1: present (must be zero)
{
start_thread_common(regs, pc);
regs->pstate = PSR_MODE_EL0t;
+
+ if (arm64_get_ssbd_state() != ARM64_SSBD_FORCE_ENABLE)
+ regs->pstate |= PSR_SSBS_BIT;
+
regs->sp = sp;
}
regs->pstate |= PSR_AA32_E_BIT;
#endif
+ if (arm64_get_ssbd_state() != ARM64_SSBD_FORCE_ENABLE)
+ regs->pstate |= PSR_AA32_SSBS_BIT;
+
regs->compat_sp = sp;
}
#endif
#endif
-void cpu_enable_pan(const struct arm64_cpu_capabilities *__unused);
-void cpu_enable_cache_maint_trap(const struct arm64_cpu_capabilities *__unused);
-void cpu_clear_disr(const struct arm64_cpu_capabilities *__unused);
-
extern unsigned long __ro_after_init signal_minsigstksz; /* sigframe size */
extern void __init minsigstksz_setup(void);
#define PSR_AA32_I_BIT 0x00000080
#define PSR_AA32_A_BIT 0x00000100
#define PSR_AA32_E_BIT 0x00000200
+#define PSR_AA32_SSBS_BIT 0x00800000
#define PSR_AA32_DIT_BIT 0x01000000
#define PSR_AA32_Q_BIT 0x08000000
#define PSR_AA32_V_BIT 0x10000000
#ifndef __ASM_SYSREG_H
#define __ASM_SYSREG_H
-#include <asm/compiler.h>
#include <linux/stringify.h>
/*
#endif /* CONFIG_BROKEN_GAS_INST */
-#define REG_PSTATE_PAN_IMM sys_reg(0, 0, 4, 0, 4)
-#define REG_PSTATE_UAO_IMM sys_reg(0, 0, 4, 0, 3)
+/*
+ * Instructions for modifying PSTATE fields.
+ * As per Arm ARM for v8-A, Section "C.5.1.3 op0 == 0b00, architectural hints,
+ * barriers and CLREX, and PSTATE access", ARM DDI 0487 C.a, system instructions
+ * for accessing PSTATE fields have the following encoding:
+ * Op0 = 0, CRn = 4
+ * Op1, Op2 encodes the PSTATE field modified and defines the constraints.
+ * CRm = Imm4 for the instruction.
+ * Rt = 0x1f
+ */
+#define pstate_field(op1, op2) ((op1) << Op1_shift | (op2) << Op2_shift)
+#define PSTATE_Imm_shift CRm_shift
+
+#define PSTATE_PAN pstate_field(0, 4)
+#define PSTATE_UAO pstate_field(0, 3)
+#define PSTATE_SSBS pstate_field(3, 1)
-#define SET_PSTATE_PAN(x) __emit_inst(0xd5000000 | REG_PSTATE_PAN_IMM | \
- (!!x)<<8 | 0x1f)
-#define SET_PSTATE_UAO(x) __emit_inst(0xd5000000 | REG_PSTATE_UAO_IMM | \
- (!!x)<<8 | 0x1f)
+#define SET_PSTATE_PAN(x) __emit_inst(0xd500401f | PSTATE_PAN | ((!!x) << PSTATE_Imm_shift))
+#define SET_PSTATE_UAO(x) __emit_inst(0xd500401f | PSTATE_UAO | ((!!x) << PSTATE_Imm_shift))
+#define SET_PSTATE_SSBS(x) __emit_inst(0xd500401f | PSTATE_SSBS | ((!!x) << PSTATE_Imm_shift))
#define SYS_DC_ISW sys_insn(1, 0, 7, 6, 2)
#define SYS_DC_CSW sys_insn(1, 0, 7, 10, 2)
#define SYS_ICH_LR15_EL2 __SYS__LR8_EL2(7)
/* Common SCTLR_ELx flags. */
+#define SCTLR_ELx_DSSBS (1UL << 44)
#define SCTLR_ELx_EE (1 << 25)
#define SCTLR_ELx_IESB (1 << 21)
#define SCTLR_ELx_WXN (1 << 19)
(1 << 10) | (1 << 13) | (1 << 14) | (1 << 15) | \
(1 << 17) | (1 << 20) | (1 << 24) | (1 << 26) | \
(1 << 27) | (1 << 30) | (1 << 31) | \
- (0xffffffffUL << 32))
+ (0xffffefffUL << 32))
#ifdef CONFIG_CPU_BIG_ENDIAN
#define ENDIAN_SET_EL2 SCTLR_ELx_EE
#define SCTLR_EL2_SET (SCTLR_ELx_IESB | ENDIAN_SET_EL2 | SCTLR_EL2_RES1)
#define SCTLR_EL2_CLEAR (SCTLR_ELx_M | SCTLR_ELx_A | SCTLR_ELx_C | \
SCTLR_ELx_SA | SCTLR_ELx_I | SCTLR_ELx_WXN | \
- ENDIAN_CLEAR_EL2 | SCTLR_EL2_RES0)
+ SCTLR_ELx_DSSBS | ENDIAN_CLEAR_EL2 | SCTLR_EL2_RES0)
#if (SCTLR_EL2_SET ^ SCTLR_EL2_CLEAR) != 0xffffffffffffffff
#error "Inconsistent SCTLR_EL2 set/clear bits"
(1 << 29))
#define SCTLR_EL1_RES0 ((1 << 6) | (1 << 10) | (1 << 13) | (1 << 17) | \
(1 << 27) | (1 << 30) | (1 << 31) | \
- (0xffffffffUL << 32))
+ (0xffffefffUL << 32))
#ifdef CONFIG_CPU_BIG_ENDIAN
#define ENDIAN_SET_EL1 (SCTLR_EL1_E0E | SCTLR_ELx_EE)
#define SCTLR_EL1_SET (SCTLR_ELx_M | SCTLR_ELx_C | SCTLR_ELx_SA |\
SCTLR_EL1_SA0 | SCTLR_EL1_SED | SCTLR_ELx_I |\
- SCTLR_EL1_DZE | SCTLR_EL1_UCT | SCTLR_EL1_NTWI |\
+ SCTLR_EL1_DZE | SCTLR_EL1_UCT |\
SCTLR_EL1_NTWE | SCTLR_ELx_IESB | SCTLR_EL1_SPAN |\
ENDIAN_SET_EL1 | SCTLR_EL1_UCI | SCTLR_EL1_RES1)
#define SCTLR_EL1_CLEAR (SCTLR_ELx_A | SCTLR_EL1_CP15BEN | SCTLR_EL1_ITD |\
SCTLR_EL1_UMA | SCTLR_ELx_WXN | ENDIAN_CLEAR_EL1 |\
- SCTLR_EL1_RES0)
+ SCTLR_ELx_DSSBS | SCTLR_EL1_NTWI | SCTLR_EL1_RES0)
#if (SCTLR_EL1_SET ^ SCTLR_EL1_CLEAR) != 0xffffffffffffffff
#error "Inconsistent SCTLR_EL1 set/clear bits"
#define ID_AA64PFR0_EL0_64BIT_ONLY 0x1
#define ID_AA64PFR0_EL0_32BIT_64BIT 0x2
+/* id_aa64pfr1 */
+#define ID_AA64PFR1_SSBS_SHIFT 4
+
+#define ID_AA64PFR1_SSBS_PSTATE_NI 0
+#define ID_AA64PFR1_SSBS_PSTATE_ONLY 1
+#define ID_AA64PFR1_SSBS_PSTATE_INSNS 2
+
/* id_aa64mmfr0 */
#define ID_AA64MMFR0_TGRAN4_SHIFT 28
#define ID_AA64MMFR0_TGRAN64_SHIFT 24
#include <linux/pagemap.h>
#include <linux/swap.h>
-#ifdef CONFIG_HAVE_RCU_TABLE_FREE
-
-#define tlb_remove_entry(tlb, entry) tlb_remove_table(tlb, entry)
static inline void __tlb_remove_table(void *_table)
{
free_page_and_swap_cache((struct page *)_table);
}
-#else
-#define tlb_remove_entry(tlb, entry) tlb_remove_page(tlb, entry)
-#endif /* CONFIG_HAVE_RCU_TABLE_FREE */
static void tlb_flush(struct mmu_gather *tlb);
static inline void tlb_flush(struct mmu_gather *tlb)
{
struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0);
+ bool last_level = !tlb->freed_tables;
+ unsigned long stride = tlb_get_unmap_size(tlb);
/*
- * The ASID allocator will either invalidate the ASID or mark
- * it as used.
+ * If we're tearing down the address space then we only care about
+ * invalidating the walk-cache, since the ASID allocator won't
+ * reallocate our ASID without invalidating the entire TLB.
*/
- if (tlb->fullmm)
+ if (tlb->fullmm) {
+ if (!last_level)
+ flush_tlb_mm(tlb->mm);
return;
+ }
- /*
- * The intermediate page table levels are already handled by
- * the __(pte|pmd|pud)_free_tlb() functions, so last level
- * TLBI is sufficient here.
- */
- __flush_tlb_range(&vma, tlb->start, tlb->end, true);
+ __flush_tlb_range(&vma, tlb->start, tlb->end, stride, last_level);
}
static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte,
unsigned long addr)
{
- __flush_tlb_pgtable(tlb->mm, addr);
pgtable_page_dtor(pte);
- tlb_remove_entry(tlb, pte);
+ tlb_remove_table(tlb, pte);
}
#if CONFIG_PGTABLE_LEVELS > 2
static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmdp,
unsigned long addr)
{
- __flush_tlb_pgtable(tlb->mm, addr);
- tlb_remove_entry(tlb, virt_to_page(pmdp));
+ tlb_remove_table(tlb, virt_to_page(pmdp));
}
#endif
static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pudp,
unsigned long addr)
{
- __flush_tlb_pgtable(tlb->mm, addr);
- tlb_remove_entry(tlb, virt_to_page(pudp));
+ tlb_remove_table(tlb, virt_to_page(pudp));
}
#endif
})
/*
- * TLB Management
- * ==============
+ * TLB Invalidation
+ * ================
*
- * The TLB specific code is expected to perform whatever tests it needs
- * to determine if it should invalidate the TLB for each call. Start
- * addresses are inclusive and end addresses are exclusive; it is safe to
- * round these addresses down.
+ * This header file implements the low-level TLB invalidation routines
+ * (sometimes referred to as "flushing" in the kernel) for arm64.
*
- * flush_tlb_all()
+ * Every invalidation operation uses the following template:
+ *
+ * DSB ISHST // Ensure prior page-table updates have completed
+ * TLBI ... // Invalidate the TLB
+ * DSB ISH // Ensure the TLB invalidation has completed
+ * if (invalidated kernel mappings)
+ * ISB // Discard any instructions fetched from the old mapping
+ *
+ *
+ * The following functions form part of the "core" TLB invalidation API,
+ * as documented in Documentation/core-api/cachetlb.rst:
*
- * Invalidate the entire TLB.
+ * flush_tlb_all()
+ * Invalidate the entire TLB (kernel + user) on all CPUs
*
* flush_tlb_mm(mm)
+ * Invalidate an entire user address space on all CPUs.
+ * The 'mm' argument identifies the ASID to invalidate.
+ *
+ * flush_tlb_range(vma, start, end)
+ * Invalidate the virtual-address range '[start, end)' on all
+ * CPUs for the user address space corresponding to 'vma->mm'.
+ * Note that this operation also invalidates any walk-cache
+ * entries associated with translations for the specified address
+ * range.
+ *
+ * flush_tlb_kernel_range(start, end)
+ * Same as flush_tlb_range(..., start, end), but applies to
+ * kernel mappings rather than a particular user address space.
+ * Whilst not explicitly documented, this function is used when
+ * unmapping pages from vmalloc/io space.
+ *
+ * flush_tlb_page(vma, addr)
+ * Invalidate a single user mapping for address 'addr' in the
+ * address space corresponding to 'vma->mm'. Note that this
+ * operation only invalidates a single, last-level page-table
+ * entry and therefore does not affect any walk-caches.
*
- * Invalidate all TLB entries in a particular address space.
- * - mm - mm_struct describing address space
*
- * flush_tlb_range(mm,start,end)
+ * Next, we have some undocumented invalidation routines that you probably
+ * don't want to call unless you know what you're doing:
*
- * Invalidate a range of TLB entries in the specified address
- * space.
- * - mm - mm_struct describing address space
- * - start - start address (may not be aligned)
- * - end - end address (exclusive, may not be aligned)
+ * local_flush_tlb_all()
+ * Same as flush_tlb_all(), but only applies to the calling CPU.
*
- * flush_tlb_page(vaddr,vma)
+ * __flush_tlb_kernel_pgtable(addr)
+ * Invalidate a single kernel mapping for address 'addr' on all
+ * CPUs, ensuring that any walk-cache entries associated with the
+ * translation are also invalidated.
*
- * Invalidate the specified page in the specified address range.
- * - vaddr - virtual address (may not be aligned)
- * - vma - vma_struct describing address range
+ * __flush_tlb_range(vma, start, end, stride, last_level)
+ * Invalidate the virtual-address range '[start, end)' on all
+ * CPUs for the user address space corresponding to 'vma->mm'.
+ * The invalidation operations are issued at a granularity
+ * determined by 'stride' and only affect any walk-cache entries
+ * if 'last_level' is equal to false.
*
- * flush_kern_tlb_page(kaddr)
*
- * Invalidate the TLB entry for the specified page. The address
- * will be in the kernels virtual memory space. Current uses
- * only require the D-TLB to be invalidated.
- * - kaddr - Kernel virtual memory address
+ * Finally, take a look at asm/tlb.h to see how tlb_flush() is implemented
+ * on top of these routines, since that is our interface to the mmu_gather
+ * API as used by munmap() and friends.
*/
static inline void local_flush_tlb_all(void)
{
* This is meant to avoid soft lock-ups on large TLB flushing ranges and not
* necessarily a performance improvement.
*/
-#define MAX_TLB_RANGE (1024UL << PAGE_SHIFT)
+#define MAX_TLBI_OPS 1024UL
static inline void __flush_tlb_range(struct vm_area_struct *vma,
unsigned long start, unsigned long end,
- bool last_level)
+ unsigned long stride, bool last_level)
{
unsigned long asid = ASID(vma->vm_mm);
unsigned long addr;
- if ((end - start) > MAX_TLB_RANGE) {
+ if ((end - start) > (MAX_TLBI_OPS * stride)) {
flush_tlb_mm(vma->vm_mm);
return;
}
+ /* Convert the stride into units of 4k */
+ stride >>= 12;
+
start = __TLBI_VADDR(start, asid);
end = __TLBI_VADDR(end, asid);
dsb(ishst);
- for (addr = start; addr < end; addr += 1 << (PAGE_SHIFT - 12)) {
+ for (addr = start; addr < end; addr += stride) {
if (last_level) {
__tlbi(vale1is, addr);
__tlbi_user(vale1is, addr);
static inline void flush_tlb_range(struct vm_area_struct *vma,
unsigned long start, unsigned long end)
{
- __flush_tlb_range(vma, start, end, false);
+ /*
+ * We cannot use leaf-only invalidation here, since we may be invalidating
+ * table entries as part of collapsing hugepages or moving page tables.
+ */
+ __flush_tlb_range(vma, start, end, PAGE_SIZE, false);
}
static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end)
{
unsigned long addr;
- if ((end - start) > MAX_TLB_RANGE) {
+ if ((end - start) > (MAX_TLBI_OPS * PAGE_SIZE)) {
flush_tlb_all();
return;
}
dsb(ishst);
for (addr = start; addr < end; addr += 1 << (PAGE_SHIFT - 12))
- __tlbi(vaae1is, addr);
+ __tlbi(vaale1is, addr);
dsb(ish);
isb();
}
* Used to invalidate the TLB (walk caches) corresponding to intermediate page
* table levels (pgd/pud/pmd).
*/
-static inline void __flush_tlb_pgtable(struct mm_struct *mm,
- unsigned long uaddr)
-{
- unsigned long addr = __TLBI_VADDR(uaddr, ASID(mm));
-
- __tlbi(vae1is, addr);
- __tlbi_user(vae1is, addr);
- dsb(ish);
-}
-
static inline void __flush_tlb_kernel_pgtable(unsigned long kaddr)
{
unsigned long addr = __TLBI_VADDR(kaddr, 0);
+ dsb(ishst);
__tlbi(vaae1is, addr);
dsb(ish);
}
/* Replace task scheduler's default cpu-invariant accounting */
#define arch_scale_cpu_capacity topology_get_cpu_scale
+/* Enable topology flag updates */
+#define arch_update_cpu_topology topology_update_cpu_topology
+
#include <asm-generic/topology.h>
#endif /* _ASM_ARM_TOPOLOGY_H */
#include <asm/cpufeature.h>
#include <asm/ptrace.h>
#include <asm/memory.h>
-#include <asm/compiler.h>
#include <asm/extable.h>
#define get_ds() (KERNEL_DS)
static inline int xen_irqs_disabled(struct pt_regs *regs)
{
- return raw_irqs_disabled_flags((unsigned long) regs->pstate);
+ return !interrupts_enabled(regs);
}
#define xchg_xen_ulong(ptr, val) xchg((ptr), (val))
#define HWCAP_USCAT (1 << 25)
#define HWCAP_ILRCPC (1 << 26)
#define HWCAP_FLAGM (1 << 27)
+#define HWCAP_SSBS (1 << 28)
#endif /* _UAPI__ASM_HWCAP_H */
#define PSR_I_BIT 0x00000080
#define PSR_A_BIT 0x00000100
#define PSR_D_BIT 0x00000200
+#define PSR_SSBS_BIT 0x00001000
#define PSR_PAN_BIT 0x00400000
#define PSR_UAO_BIT 0x00800000
#define PSR_V_BIT 0x10000000
has_mismatched_cache_type(const struct arm64_cpu_capabilities *entry,
int scope)
{
- u64 mask = CTR_CACHE_MINLINE_MASK;
-
- /* Skip matching the min line sizes for cache type check */
- if (entry->capability == ARM64_MISMATCHED_CACHE_TYPE)
- mask ^= arm64_ftr_reg_ctrel0.strict_mask;
+ u64 mask = arm64_ftr_reg_ctrel0.strict_mask;
+ u64 sys = arm64_ftr_reg_ctrel0.sys_val & mask;
+ u64 ctr_raw, ctr_real;
WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
- return (read_cpuid_cachetype() & mask) !=
- (arm64_ftr_reg_ctrel0.sys_val & mask);
+
+ /*
+ * We want to make sure that all the CPUs in the system expose
+ * a consistent CTR_EL0 to make sure that applications behaves
+ * correctly with migration.
+ *
+ * If a CPU has CTR_EL0.IDC but does not advertise it via CTR_EL0 :
+ *
+ * 1) It is safe if the system doesn't support IDC, as CPU anyway
+ * reports IDC = 0, consistent with the rest.
+ *
+ * 2) If the system has IDC, it is still safe as we trap CTR_EL0
+ * access on this CPU via the ARM64_HAS_CACHE_IDC capability.
+ *
+ * So, we need to make sure either the raw CTR_EL0 or the effective
+ * CTR_EL0 matches the system's copy to allow a secondary CPU to boot.
+ */
+ ctr_raw = read_cpuid_cachetype() & mask;
+ ctr_real = read_cpuid_effective_cachetype() & mask;
+
+ return (ctr_real != sys) && (ctr_raw != sys);
}
static void
cpu_enable_trap_ctr_access(const struct arm64_cpu_capabilities *__unused)
{
- sysreg_clear_set(sctlr_el1, SCTLR_EL1_UCT, 0);
+ u64 mask = arm64_ftr_reg_ctrel0.strict_mask;
+
+ /* Trap CTR_EL0 access on this CPU, only if it has a mismatch */
+ if ((read_cpuid_cachetype() & mask) !=
+ (arm64_ftr_reg_ctrel0.sys_val & mask))
+ sysreg_clear_set(sctlr_el1, SCTLR_EL1_UCT, 0);
}
atomic_t arm64_el2_vector_last_slot = ATOMIC_INIT(-1);
static DEFINE_SPINLOCK(bp_lock);
int cpu, slot = -1;
+ /*
+ * enable_smccc_arch_workaround_1() passes NULL for the hyp_vecs
+ * start/end if we're a guest. Skip the hyp-vectors work.
+ */
+ if (!hyp_vecs_start) {
+ __this_cpu_write(bp_hardening_data.fn, fn);
+ return;
+ }
+
spin_lock(&bp_lock);
for_each_possible_cpu(cpu) {
if (per_cpu(bp_hardening_data.fn, cpu) == fn) {
void arm64_set_ssbd_mitigation(bool state)
{
+ if (this_cpu_has_cap(ARM64_SSBS)) {
+ if (state)
+ asm volatile(SET_PSTATE_SSBS(0));
+ else
+ asm volatile(SET_PSTATE_SSBS(1));
+ return;
+ }
+
switch (psci_ops.conduit) {
case PSCI_CONDUIT_HVC:
arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_WORKAROUND_2, state, NULL);
WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
+ if (this_cpu_has_cap(ARM64_SSBS)) {
+ required = false;
+ goto out_printmsg;
+ }
+
if (psci_ops.smccc_version == SMCCC_VERSION_1_0) {
ssbd_state = ARM64_SSBD_UNKNOWN;
return false;
switch (ssbd_state) {
case ARM64_SSBD_FORCE_DISABLE:
- pr_info_once("%s disabled from command-line\n", entry->desc);
arm64_set_ssbd_mitigation(false);
required = false;
break;
break;
case ARM64_SSBD_FORCE_ENABLE:
- pr_info_once("%s forced from command-line\n", entry->desc);
arm64_set_ssbd_mitigation(true);
required = true;
break;
break;
}
+out_printmsg:
+ switch (ssbd_state) {
+ case ARM64_SSBD_FORCE_DISABLE:
+ pr_info_once("%s disabled from command-line\n", entry->desc);
+ break;
+
+ case ARM64_SSBD_FORCE_ENABLE:
+ pr_info_once("%s forced from command-line\n", entry->desc);
+ break;
+ }
+
return required;
}
#endif /* CONFIG_ARM64_SSBD */
+static void __maybe_unused
+cpu_enable_cache_maint_trap(const struct arm64_cpu_capabilities *__unused)
+{
+ sysreg_clear_set(sctlr_el1, SCTLR_EL1_UCI, 0);
+}
+
#define CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
.matches = is_affected_midr_range, \
.midr_range = MIDR_RANGE(model, v_min, r_min, v_max, r_max)
},
#endif
{
- .desc = "Mismatched cache line size",
- .capability = ARM64_MISMATCHED_CACHE_LINE_SIZE,
- .matches = has_mismatched_cache_type,
- .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
- .cpu_enable = cpu_enable_trap_ctr_access,
- },
- {
- .desc = "Mismatched cache type",
+ .desc = "Mismatched cache type (CTR_EL0)",
.capability = ARM64_MISMATCHED_CACHE_TYPE,
.matches = has_mismatched_cache_type,
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
.matches = has_ssbd_mitigation,
},
+#endif
+#ifdef CONFIG_ARM64_ERRATUM_1188873
+ {
+ /* Cortex-A76 r0p0 to r2p0 */
+ .desc = "ARM erratum 1188873",
+ .capability = ARM64_WORKAROUND_1188873,
+ ERRATA_MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0),
+ },
#endif
{
}
#include <linux/bsearch.h>
#include <linux/cpumask.h>
+#include <linux/crash_dump.h>
#include <linux/sort.h>
#include <linux/stop_machine.h>
#include <linux/types.h>
static bool __maybe_unused
cpufeature_pan_not_uao(const struct arm64_cpu_capabilities *entry, int __unused);
+static void cpu_enable_cnp(struct arm64_cpu_capabilities const *cap);
/*
* NOTE: Any changes to the visibility of features should be kept in
ARM64_FTR_END,
};
+static const struct arm64_ftr_bits ftr_id_aa64pfr1[] = {
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_SSBS_SHIFT, 4, ID_AA64PFR1_SSBS_PSTATE_NI),
+ ARM64_FTR_END,
+};
+
static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = {
S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN4_SHIFT, 4, ID_AA64MMFR0_TGRAN4_NI),
S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_TGRAN64_SHIFT, 4, ID_AA64MMFR0_TGRAN64_NI),
/* Op1 = 0, CRn = 0, CRm = 4 */
ARM64_FTR_REG(SYS_ID_AA64PFR0_EL1, ftr_id_aa64pfr0),
- ARM64_FTR_REG(SYS_ID_AA64PFR1_EL1, ftr_raz),
+ ARM64_FTR_REG(SYS_ID_AA64PFR1_EL1, ftr_id_aa64pfr1),
ARM64_FTR_REG(SYS_ID_AA64ZFR0_EL1, ftr_raz),
/* Op1 = 0, CRn = 0, CRm = 5 */
/*
* EL3 is not our concern.
- * ID_AA64PFR1 is currently RES0.
*/
taint |= check_update_ftr_reg(SYS_ID_AA64PFR0_EL1, cpu,
info->reg_id_aa64pfr0, boot->reg_id_aa64pfr0);
}
static bool has_cache_idc(const struct arm64_cpu_capabilities *entry,
- int __unused)
+ int scope)
{
- return read_sanitised_ftr_reg(SYS_CTR_EL0) & BIT(CTR_IDC_SHIFT);
+ u64 ctr;
+
+ if (scope == SCOPE_SYSTEM)
+ ctr = arm64_ftr_reg_ctrel0.sys_val;
+ else
+ ctr = read_cpuid_effective_cachetype();
+
+ return ctr & BIT(CTR_IDC_SHIFT);
+}
+
+static void cpu_emulate_effective_ctr(const struct arm64_cpu_capabilities *__unused)
+{
+ /*
+ * If the CPU exposes raw CTR_EL0.IDC = 0, while effectively
+ * CTR_EL0.IDC = 1 (from CLIDR values), we need to trap accesses
+ * to the CTR_EL0 on this CPU and emulate it with the real/safe
+ * value.
+ */
+ if (!(read_cpuid_cachetype() & BIT(CTR_IDC_SHIFT)))
+ sysreg_clear_set(sctlr_el1, SCTLR_EL1_UCT, 0);
}
static bool has_cache_dic(const struct arm64_cpu_capabilities *entry,
- int __unused)
+ int scope)
{
- return read_sanitised_ftr_reg(SYS_CTR_EL0) & BIT(CTR_DIC_SHIFT);
+ u64 ctr;
+
+ if (scope == SCOPE_SYSTEM)
+ ctr = arm64_ftr_reg_ctrel0.sys_val;
+ else
+ ctr = read_cpuid_cachetype();
+
+ return ctr & BIT(CTR_DIC_SHIFT);
+}
+
+static bool __maybe_unused
+has_useable_cnp(const struct arm64_cpu_capabilities *entry, int scope)
+{
+ /*
+ * Kdump isn't guaranteed to power-off all secondary CPUs, CNP
+ * may share TLB entries with a CPU stuck in the crashed
+ * kernel.
+ */
+ if (is_kdump_kernel())
+ return false;
+
+ return has_cpuid_feature(entry, scope);
}
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
WARN_ON(val & (7 << 27 | 7 << 21));
}
+#ifdef CONFIG_ARM64_SSBD
+static int ssbs_emulation_handler(struct pt_regs *regs, u32 instr)
+{
+ if (user_mode(regs))
+ return 1;
+
+ if (instr & BIT(PSTATE_Imm_shift))
+ regs->pstate |= PSR_SSBS_BIT;
+ else
+ regs->pstate &= ~PSR_SSBS_BIT;
+
+ arm64_skip_faulting_instruction(regs, 4);
+ return 0;
+}
+
+static struct undef_hook ssbs_emulation_hook = {
+ .instr_mask = ~(1U << PSTATE_Imm_shift),
+ .instr_val = 0xd500401f | PSTATE_SSBS,
+ .fn = ssbs_emulation_handler,
+};
+
+static void cpu_enable_ssbs(const struct arm64_cpu_capabilities *__unused)
+{
+ static bool undef_hook_registered = false;
+ static DEFINE_SPINLOCK(hook_lock);
+
+ spin_lock(&hook_lock);
+ if (!undef_hook_registered) {
+ register_undef_hook(&ssbs_emulation_hook);
+ undef_hook_registered = true;
+ }
+ spin_unlock(&hook_lock);
+
+ if (arm64_get_ssbd_state() == ARM64_SSBD_FORCE_DISABLE) {
+ sysreg_clear_set(sctlr_el1, 0, SCTLR_ELx_DSSBS);
+ arm64_set_ssbd_mitigation(false);
+ } else {
+ arm64_set_ssbd_mitigation(true);
+ }
+}
+#endif /* CONFIG_ARM64_SSBD */
+
+#ifdef CONFIG_ARM64_PAN
+static void cpu_enable_pan(const struct arm64_cpu_capabilities *__unused)
+{
+ /*
+ * We modify PSTATE. This won't work from irq context as the PSTATE
+ * is discarded once we return from the exception.
+ */
+ WARN_ON_ONCE(in_interrupt());
+
+ sysreg_clear_set(sctlr_el1, SCTLR_EL1_SPAN, 0);
+ asm(SET_PSTATE_PAN(1));
+}
+#endif /* CONFIG_ARM64_PAN */
+
+#ifdef CONFIG_ARM64_RAS_EXTN
+static void cpu_clear_disr(const struct arm64_cpu_capabilities *__unused)
+{
+ /* Firmware may have left a deferred SError in this register. */
+ write_sysreg_s(0, SYS_DISR_EL1);
+}
+#endif /* CONFIG_ARM64_RAS_EXTN */
+
static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "GIC system register CPU interface",
.capability = ARM64_HAS_CACHE_IDC,
.type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cache_idc,
+ .cpu_enable = cpu_emulate_effective_ctr,
},
{
.desc = "Instruction cache invalidation not required for I/D coherence",
.matches = has_hw_dbm,
.cpu_enable = cpu_enable_hw_dbm,
},
+#endif
+#ifdef CONFIG_ARM64_SSBD
+ {
+ .desc = "CRC32 instructions",
+ .capability = ARM64_HAS_CRC32,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .matches = has_cpuid_feature,
+ .sys_reg = SYS_ID_AA64ISAR0_EL1,
+ .field_pos = ID_AA64ISAR0_CRC32_SHIFT,
+ .min_field_value = 1,
+ },
+ {
+ .desc = "Speculative Store Bypassing Safe (SSBS)",
+ .capability = ARM64_SSBS,
+ .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE,
+ .matches = has_cpuid_feature,
+ .sys_reg = SYS_ID_AA64PFR1_EL1,
+ .field_pos = ID_AA64PFR1_SSBS_SHIFT,
+ .sign = FTR_UNSIGNED,
+ .min_field_value = ID_AA64PFR1_SSBS_PSTATE_ONLY,
+ .cpu_enable = cpu_enable_ssbs,
+ },
+#endif
+#ifdef CONFIG_ARM64_CNP
+ {
+ .desc = "Common not Private translations",
+ .capability = ARM64_HAS_CNP,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .matches = has_useable_cnp,
+ .sys_reg = SYS_ID_AA64MMFR2_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64MMFR2_CNP_SHIFT,
+ .min_field_value = 1,
+ .cpu_enable = cpu_enable_cnp,
+ },
#endif
{},
};
#ifdef CONFIG_ARM64_SVE
HWCAP_CAP(SYS_ID_AA64PFR0_EL1, ID_AA64PFR0_SVE_SHIFT, FTR_UNSIGNED, ID_AA64PFR0_SVE, CAP_HWCAP, HWCAP_SVE),
#endif
+ HWCAP_CAP(SYS_ID_AA64PFR1_EL1, ID_AA64PFR1_SSBS_SHIFT, FTR_UNSIGNED, ID_AA64PFR1_SSBS_PSTATE_INSNS, CAP_HWCAP, HWCAP_SSBS),
{},
};
return (cpus_have_const_cap(ARM64_HAS_PAN) && !cpus_have_const_cap(ARM64_HAS_UAO));
}
+static void __maybe_unused cpu_enable_cnp(struct arm64_cpu_capabilities const *cap)
+{
+ cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
+}
+
/*
* We emulate only the following system register space.
* Op0 = 0x3, CRn = 0x0, Op1 = 0x0, CRm = [0, 4 - 7]
return 0;
}
-static int emulate_mrs(struct pt_regs *regs, u32 insn)
+int do_emulate_mrs(struct pt_regs *regs, u32 sys_reg, u32 rt)
{
int rc;
- u32 sys_reg, dst;
u64 val;
- /*
- * sys_reg values are defined as used in mrs/msr instruction.
- * shift the imm value to get the encoding.
- */
- sys_reg = (u32)aarch64_insn_decode_immediate(AARCH64_INSN_IMM_16, insn) << 5;
rc = emulate_sys_reg(sys_reg, &val);
if (!rc) {
- dst = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RT, insn);
- pt_regs_write_reg(regs, dst, val);
+ pt_regs_write_reg(regs, rt, val);
arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE);
}
-
return rc;
}
+static int emulate_mrs(struct pt_regs *regs, u32 insn)
+{
+ u32 sys_reg, rt;
+
+ /*
+ * sys_reg values are defined as used in mrs/msr instruction.
+ * shift the imm value to get the encoding.
+ */
+ sys_reg = (u32)aarch64_insn_decode_immediate(AARCH64_INSN_IMM_16, insn) << 5;
+ rt = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RT, insn);
+ return do_emulate_mrs(regs, sys_reg, rt);
+}
+
static struct undef_hook mrs_hook = {
.instr_mask = 0xfff00000,
.instr_val = 0xd5300000,
}
core_initcall(enable_mrs_emulation);
-
-void cpu_clear_disr(const struct arm64_cpu_capabilities *__unused)
-{
- /* Firmware may have left a deferred SError in this register. */
- write_sysreg_s(0, SYS_DISR_EL1);
-}
"uscat",
"ilrcpc",
"flagm",
+ "ssbs",
NULL
};
static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
{
info->reg_cntfrq = arch_timer_get_cntfrq();
- info->reg_ctr = read_cpuid_cachetype();
+ /*
+ * Use the effective value of the CTR_EL0 than the raw value
+ * exposed by the CPU. CTR_E0.IDC field value must be interpreted
+ * with the CLIDR_EL1 fields to avoid triggering false warnings
+ * when there is a mismatch across the CPUs. Keep track of the
+ * effective value of the CTR_EL0 in our internal records for
+ * acurate sanity check and feature enablement.
+ */
+ info->reg_ctr = read_cpuid_effective_cachetype();
info->reg_dczid = read_cpuid(DCZID_EL0);
info->reg_midr = read_cpuid_id();
info->reg_revidr = read_cpuid(REVIDR_EL1);
inherit_daif pstate=x23, tmp=x2
mov x0, sp
bl do_undefinstr
- ASM_BUG()
+ kernel_exit 1
el1_dbg:
/*
* Debug exception handling
cmp x24, #ESR_ELx_EC_FP_EXC64 // FP/ASIMD exception
b.eq el0_fpsimd_exc
cmp x24, #ESR_ELx_EC_SYS64 // configurable trap
+ ccmp x24, #ESR_ELx_EC_WFx, #4, ne
b.eq el0_sys
cmp x24, #ESR_ELx_EC_SP_ALIGN // stack alignment exception
b.eq el0_sp_pc
cmp x24, #ESR_ELx_EC_UNKNOWN // unknown exception in EL0
b.eq el0_undef
cmp x24, #ESR_ELx_EC_CP15_32 // CP15 MRC/MCR trap
- b.eq el0_undef
+ b.eq el0_cp15
cmp x24, #ESR_ELx_EC_CP15_64 // CP15 MRRC/MCRR trap
- b.eq el0_undef
+ b.eq el0_cp15
cmp x24, #ESR_ELx_EC_CP14_MR // CP14 MRC/MCR trap
b.eq el0_undef
cmp x24, #ESR_ELx_EC_CP14_LS // CP14 LDC/STC trap
el0_error_compat:
kernel_entry 0, 32
b el0_error_naked
+
+el0_cp15:
+ /*
+ * Trapped CP15 (MRC, MCR, MRRC, MCRR) instructions
+ */
+ enable_daif
+ ct_user_exit
+ mov x0, x25
+ mov x1, sp
+ bl do_cp15instr
+ b ret_to_user
#endif
el0_da:
mov x28, lr
/*
- * Invalidate the idmap and swapper page tables to avoid potential
- * dirty cache lines being evicted.
+ * Invalidate the init page tables to avoid potential dirty cache lines
+ * being evicted. Other page tables are allocated in rodata as part of
+ * the kernel image, and thus are clean to the PoC per the boot
+ * protocol.
*/
- adrp x0, idmap_pg_dir
- adrp x1, swapper_pg_end
+ adrp x0, init_pg_dir
+ adrp x1, init_pg_end
sub x1, x1, x0
bl __inval_dcache_area
/*
- * Clear the idmap and swapper page tables.
+ * Clear the init page tables.
*/
- adrp x0, idmap_pg_dir
- adrp x1, swapper_pg_end
+ adrp x0, init_pg_dir
+ adrp x1, init_pg_end
sub x1, x1, x0
1: stp xzr, xzr, [x0], #16
stp xzr, xzr, [x0], #16
/*
* Map the kernel image (starting with PHYS_OFFSET).
*/
- adrp x0, swapper_pg_dir
+ adrp x0, init_pg_dir
mov_q x5, KIMAGE_VADDR + TEXT_OFFSET // compile time __va(_text)
add x5, x5, x23 // add KASLR displacement
mov x4, PTRS_PER_PGD
* tables again to remove any speculatively loaded cache lines.
*/
adrp x0, idmap_pg_dir
- adrp x1, swapper_pg_end
+ adrp x1, init_pg_end
sub x1, x1, x0
dmb sy
bl __inval_dcache_area
* Common entry point for secondary CPUs.
*/
bl __cpu_setup // initialise processor
+ adrp x1, swapper_pg_dir
bl __enable_mmu
ldr x8, =__secondary_switched
br x8
* Enable the MMU.
*
* x0 = SCTLR_EL1 value for turning on the MMU.
+ * x1 = TTBR1_EL1 value
*
* Returns to the caller via x30/lr. This requires the caller to be covered
* by the .idmap.text section.
* If it isn't, park the CPU
*/
ENTRY(__enable_mmu)
- mrs x1, ID_AA64MMFR0_EL1
- ubfx x2, x1, #ID_AA64MMFR0_TGRAN_SHIFT, 4
+ mrs x2, ID_AA64MMFR0_EL1
+ ubfx x2, x2, #ID_AA64MMFR0_TGRAN_SHIFT, 4
cmp x2, #ID_AA64MMFR0_TGRAN_SUPPORTED
b.ne __no_granule_support
- update_early_cpu_boot_status 0, x1, x2
- adrp x1, idmap_pg_dir
- adrp x2, swapper_pg_dir
- phys_to_ttbr x3, x1
- phys_to_ttbr x4, x2
- msr ttbr0_el1, x3 // load TTBR0
- msr ttbr1_el1, x4 // load TTBR1
+ update_early_cpu_boot_status 0, x2, x3
+ adrp x2, idmap_pg_dir
+ phys_to_ttbr x1, x1
+ phys_to_ttbr x2, x2
+ msr ttbr0_el1, x2 // load TTBR0
+ msr ttbr1_el1, x1 // load TTBR1
isb
msr sctlr_el1, x0
isb
mrs x20, sctlr_el1 // preserve old SCTLR_EL1 value
#endif
+ adrp x1, init_pg_dir
bl __enable_mmu
#ifdef CONFIG_RELOCATABLE
bl __relocate_kernel
void arch_jump_label_transform(struct jump_entry *entry,
enum jump_label_type type)
{
- void *addr = (void *)entry->code;
+ void *addr = (void *)jump_entry_code(entry);
u32 insn;
if (type == JUMP_LABEL_JMP) {
- insn = aarch64_insn_gen_branch_imm(entry->code,
- entry->target,
+ insn = aarch64_insn_gen_branch_imm(jump_entry_code(entry),
+ jump_entry_target(entry),
AARCH64_INSN_BRANCH_NOLINK);
} else {
insn = aarch64_insn_gen_nop();
return 0;
}
+static int armv8pmu_filter_match(struct perf_event *event)
+{
+ unsigned long evtype = event->hw.config_base & ARMV8_PMU_EVTYPE_EVENT;
+ return evtype != ARMV8_PMUV3_PERFCTR_CHAIN;
+}
+
static void armv8pmu_reset(void *info)
{
struct arm_pmu *cpu_pmu = (struct arm_pmu *)info;
cpu_pmu->stop = armv8pmu_stop,
cpu_pmu->reset = armv8pmu_reset,
cpu_pmu->set_event_filter = armv8pmu_set_event_filter;
+ cpu_pmu->filter_match = armv8pmu_filter_match;
return 0;
}
if (!p->ainsn.api.insn)
return -ENOMEM;
break;
- };
+ }
/* prepare the instruction */
if (p->ainsn.api.insn)
if (IS_ENABLED(CONFIG_ARM64_UAO) &&
cpus_have_const_cap(ARM64_HAS_UAO))
childregs->pstate |= PSR_UAO_BIT;
+
+ if (arm64_get_ssbd_state() == ARM64_SSBD_FORCE_DISABLE)
+ childregs->pstate |= PSR_SSBS_BIT;
+
p->thread.cpu_context.x19 = stack_start;
p->thread.cpu_context.x20 = stk_sz;
}
#include <uapi/linux/psci.h>
-#include <asm/compiler.h>
#include <asm/cpu_ops.h>
#include <asm/errno.h>
#include <asm/smp_plat.h>
#include <asm/xen/hypervisor.h>
#include <asm/mmu_context.h>
+static int num_standard_resources;
+static struct resource *standard_resources;
+
phys_addr_t __fdt_pointer __initdata;
/*
{
struct memblock_region *region;
struct resource *res;
+ unsigned long i = 0;
kernel_code.start = __pa_symbol(_text);
kernel_code.end = __pa_symbol(__init_begin - 1);
kernel_data.start = __pa_symbol(_sdata);
kernel_data.end = __pa_symbol(_end - 1);
+ num_standard_resources = memblock.memory.cnt;
+ standard_resources = alloc_bootmem_low(num_standard_resources *
+ sizeof(*standard_resources));
+
for_each_memblock(memory, region) {
- res = alloc_bootmem_low(sizeof(*res));
+ res = &standard_resources[i++];
if (memblock_is_nomap(region)) {
res->name = "reserved";
res->flags = IORESOURCE_MEM;
static int __init reserve_memblock_reserved_regions(void)
{
- phys_addr_t start, end, roundup_end = 0;
- struct resource *mem, *res;
- u64 i;
-
- for_each_reserved_mem_region(i, &start, &end) {
- if (end <= roundup_end)
- continue; /* done already */
-
- start = __pfn_to_phys(PFN_DOWN(start));
- end = __pfn_to_phys(PFN_UP(end)) - 1;
- roundup_end = end;
-
- res = kzalloc(sizeof(*res), GFP_ATOMIC);
- if (WARN_ON(!res))
- return -ENOMEM;
- res->start = start;
- res->end = end;
- res->name = "reserved";
- res->flags = IORESOURCE_MEM;
-
- mem = request_resource_conflict(&iomem_resource, res);
- /*
- * We expected memblock_reserve() regions to conflict with
- * memory created by request_standard_resources().
- */
- if (WARN_ON_ONCE(!mem))
+ u64 i, j;
+
+ for (i = 0; i < num_standard_resources; ++i) {
+ struct resource *mem = &standard_resources[i];
+ phys_addr_t r_start, r_end, mem_size = resource_size(mem);
+
+ if (!memblock_is_region_reserved(mem->start, mem_size))
continue;
- kfree(res);
- reserve_region_with_split(mem, start, end, "reserved");
+ for_each_reserved_mem_region(j, &r_start, &r_end) {
+ resource_size_t start, end;
+
+ start = max(PFN_PHYS(PFN_DOWN(r_start)), mem->start);
+ end = min(PFN_PHYS(PFN_UP(r_end)) - 1, mem->end);
+
+ if (start > mem->end || end < mem->start)
+ continue;
+
+ reserve_region_with_split(mem, start, end, "reserved");
+ }
}
return 0;
#endif
#ifdef CONFIG_VT
-#if defined(CONFIG_VGA_CONSOLE)
- conswitchp = &vga_con;
-#elif defined(CONFIG_DUMMY_CONSOLE)
conswitchp = &dummy_con;
-#endif
#endif
if (boot_args[1] || boot_args[2] || boot_args[3]) {
pr_err("WARNING: x1-x3 nonzero in violation of boot protocol:\n"
bl el2_setup // if in EL2 drop to EL1 cleanly
bl __cpu_setup
/* enable the MMU early - so we can access sleep_save_stash by va */
+ adrp x1, swapper_pg_dir
bl __enable_mmu
ldr x8, =_cpu_resume
br x8
* Copyright (C) 2018 ARM Ltd, All Rights Reserved.
*/
+#include <linux/compat.h>
#include <linux/errno.h>
#include <linux/sched.h>
+#include <linux/sched/task_stack.h>
#include <linux/thread_info.h>
#include <asm/cpufeature.h>
+static void ssbd_ssbs_enable(struct task_struct *task)
+{
+ u64 val = is_compat_thread(task_thread_info(task)) ?
+ PSR_AA32_SSBS_BIT : PSR_SSBS_BIT;
+
+ task_pt_regs(task)->pstate |= val;
+}
+
+static void ssbd_ssbs_disable(struct task_struct *task)
+{
+ u64 val = is_compat_thread(task_thread_info(task)) ?
+ PSR_AA32_SSBS_BIT : PSR_SSBS_BIT;
+
+ task_pt_regs(task)->pstate &= ~val;
+}
+
/*
* prctl interface for SSBD
- * FIXME: Drop the below ifdefery once merged in 4.18.
*/
-#ifdef PR_SPEC_STORE_BYPASS
static int ssbd_prctl_set(struct task_struct *task, unsigned long ctrl)
{
int state = arm64_get_ssbd_state();
return -EPERM;
task_clear_spec_ssb_disable(task);
clear_tsk_thread_flag(task, TIF_SSBD);
+ ssbd_ssbs_enable(task);
break;
case PR_SPEC_DISABLE:
if (state == ARM64_SSBD_FORCE_DISABLE)
return -EPERM;
task_set_spec_ssb_disable(task);
set_tsk_thread_flag(task, TIF_SSBD);
+ ssbd_ssbs_disable(task);
break;
case PR_SPEC_FORCE_DISABLE:
if (state == ARM64_SSBD_FORCE_DISABLE)
task_set_spec_ssb_disable(task);
task_set_spec_ssb_force_disable(task);
set_tsk_thread_flag(task, TIF_SSBD);
+ ssbd_ssbs_disable(task);
break;
default:
return -ERANGE;
return -ENODEV;
}
}
-#endif /* PR_SPEC_STORE_BYPASS */
*/
cpu_uninstall_idmap();
+ /* Restore CnP bit in TTBR1_EL1 */
+ if (system_supports_cnp())
+ cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
+
/*
* PSTATE was not saved over suspend/resume, re-enable any detected
* features that might not have been set correctly.
int (*fn)(struct pt_regs *regs, u32 instr) = NULL;
void __user *pc = (void __user *)instruction_pointer(regs);
- if (!user_mode(regs))
- return 1;
-
- if (compat_thumb_mode(regs)) {
+ if (!user_mode(regs)) {
+ __le32 instr_le;
+ if (probe_kernel_address((__force __le32 *)pc, instr_le))
+ goto exit;
+ instr = le32_to_cpu(instr_le);
+ } else if (compat_thumb_mode(regs)) {
/* 16-bit Thumb instruction */
__le16 instr_le;
if (get_user(instr_le, (__le16 __user *)pc))
const char *desc;
struct pt_regs *regs = current_pt_regs();
+ if (WARN_ON(!user_mode(regs)))
+ return;
+
clear_siginfo(&info);
switch (signal) {
if (call_undef_hook(regs) == 0)
return;
+ BUG_ON(!user_mode(regs));
force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc);
}
-void cpu_enable_cache_maint_trap(const struct arm64_cpu_capabilities *__unused)
-{
- sysreg_clear_set(sctlr_el1, SCTLR_EL1_UCI, 0);
-}
-
#define __user_cache_maint(insn, address, res) \
if (address >= user_addr_max()) { \
res = -EFAULT; \
static void user_cache_maint_handler(unsigned int esr, struct pt_regs *regs)
{
unsigned long address;
- int rt = (esr & ESR_ELx_SYS64_ISS_RT_MASK) >> ESR_ELx_SYS64_ISS_RT_SHIFT;
+ int rt = ESR_ELx_SYS64_ISS_RT(esr);
int crm = (esr & ESR_ELx_SYS64_ISS_CRM_MASK) >> ESR_ELx_SYS64_ISS_CRM_SHIFT;
int ret = 0;
static void ctr_read_handler(unsigned int esr, struct pt_regs *regs)
{
- int rt = (esr & ESR_ELx_SYS64_ISS_RT_MASK) >> ESR_ELx_SYS64_ISS_RT_SHIFT;
+ int rt = ESR_ELx_SYS64_ISS_RT(esr);
unsigned long val = arm64_ftr_reg_user_value(&arm64_ftr_reg_ctrel0);
pt_regs_write_reg(regs, rt, val);
static void cntvct_read_handler(unsigned int esr, struct pt_regs *regs)
{
- int rt = (esr & ESR_ELx_SYS64_ISS_RT_MASK) >> ESR_ELx_SYS64_ISS_RT_SHIFT;
+ int rt = ESR_ELx_SYS64_ISS_RT(esr);
pt_regs_write_reg(regs, rt, arch_counter_get_cntvct());
arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE);
static void cntfrq_read_handler(unsigned int esr, struct pt_regs *regs)
{
- int rt = (esr & ESR_ELx_SYS64_ISS_RT_MASK) >> ESR_ELx_SYS64_ISS_RT_SHIFT;
+ int rt = ESR_ELx_SYS64_ISS_RT(esr);
pt_regs_write_reg(regs, rt, arch_timer_get_rate());
arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE);
}
+static void mrs_handler(unsigned int esr, struct pt_regs *regs)
+{
+ u32 sysreg, rt;
+
+ rt = ESR_ELx_SYS64_ISS_RT(esr);
+ sysreg = esr_sys64_to_sysreg(esr);
+
+ if (do_emulate_mrs(regs, sysreg, rt) != 0)
+ force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc);
+}
+
+static void wfi_handler(unsigned int esr, struct pt_regs *regs)
+{
+ arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE);
+}
+
struct sys64_hook {
unsigned int esr_mask;
unsigned int esr_val;
.esr_val = ESR_ELx_SYS64_ISS_SYS_CNTFRQ,
.handler = cntfrq_read_handler,
},
+ {
+ /* Trap read access to CPUID registers */
+ .esr_mask = ESR_ELx_SYS64_ISS_SYS_MRS_OP_MASK,
+ .esr_val = ESR_ELx_SYS64_ISS_SYS_MRS_OP_VAL,
+ .handler = mrs_handler,
+ },
+ {
+ /* Trap WFI instructions executed in userspace */
+ .esr_mask = ESR_ELx_WFx_MASK,
+ .esr_val = ESR_ELx_WFx_WFI_VAL,
+ .handler = wfi_handler,
+ },
{},
};
+
+#ifdef CONFIG_COMPAT
+#define PSTATE_IT_1_0_SHIFT 25
+#define PSTATE_IT_1_0_MASK (0x3 << PSTATE_IT_1_0_SHIFT)
+#define PSTATE_IT_7_2_SHIFT 10
+#define PSTATE_IT_7_2_MASK (0x3f << PSTATE_IT_7_2_SHIFT)
+
+static u32 compat_get_it_state(struct pt_regs *regs)
+{
+ u32 it, pstate = regs->pstate;
+
+ it = (pstate & PSTATE_IT_1_0_MASK) >> PSTATE_IT_1_0_SHIFT;
+ it |= ((pstate & PSTATE_IT_7_2_MASK) >> PSTATE_IT_7_2_SHIFT) << 2;
+
+ return it;
+}
+
+static void compat_set_it_state(struct pt_regs *regs, u32 it)
+{
+ u32 pstate_it;
+
+ pstate_it = (it << PSTATE_IT_1_0_SHIFT) & PSTATE_IT_1_0_MASK;
+ pstate_it |= ((it >> 2) << PSTATE_IT_7_2_SHIFT) & PSTATE_IT_7_2_MASK;
+
+ regs->pstate &= ~PSR_AA32_IT_MASK;
+ regs->pstate |= pstate_it;
+}
+
+static bool cp15_cond_valid(unsigned int esr, struct pt_regs *regs)
+{
+ int cond;
+
+ /* Only a T32 instruction can trap without CV being set */
+ if (!(esr & ESR_ELx_CV)) {
+ u32 it;
+
+ it = compat_get_it_state(regs);
+ if (!it)
+ return true;
+
+ cond = it >> 4;
+ } else {
+ cond = (esr & ESR_ELx_COND_MASK) >> ESR_ELx_COND_SHIFT;
+ }
+
+ return aarch32_opcode_cond_checks[cond](regs->pstate);
+}
+
+static void advance_itstate(struct pt_regs *regs)
+{
+ u32 it;
+
+ /* ARM mode */
+ if (!(regs->pstate & PSR_AA32_T_BIT) ||
+ !(regs->pstate & PSR_AA32_IT_MASK))
+ return;
+
+ it = compat_get_it_state(regs);
+
+ /*
+ * If this is the last instruction of the block, wipe the IT
+ * state. Otherwise advance it.
+ */
+ if (!(it & 7))
+ it = 0;
+ else
+ it = (it & 0xe0) | ((it << 1) & 0x1f);
+
+ compat_set_it_state(regs, it);
+}
+
+static void arm64_compat_skip_faulting_instruction(struct pt_regs *regs,
+ unsigned int sz)
+{
+ advance_itstate(regs);
+ arm64_skip_faulting_instruction(regs, sz);
+}
+
+static void compat_cntfrq_read_handler(unsigned int esr, struct pt_regs *regs)
+{
+ int reg = (esr & ESR_ELx_CP15_32_ISS_RT_MASK) >> ESR_ELx_CP15_32_ISS_RT_SHIFT;
+
+ pt_regs_write_reg(regs, reg, arch_timer_get_rate());
+ arm64_compat_skip_faulting_instruction(regs, 4);
+}
+
+static struct sys64_hook cp15_32_hooks[] = {
+ {
+ .esr_mask = ESR_ELx_CP15_32_ISS_SYS_MASK,
+ .esr_val = ESR_ELx_CP15_32_ISS_SYS_CNTFRQ,
+ .handler = compat_cntfrq_read_handler,
+ },
+ {},
+};
+
+static void compat_cntvct_read_handler(unsigned int esr, struct pt_regs *regs)
+{
+ int rt = (esr & ESR_ELx_CP15_64_ISS_RT_MASK) >> ESR_ELx_CP15_64_ISS_RT_SHIFT;
+ int rt2 = (esr & ESR_ELx_CP15_64_ISS_RT2_MASK) >> ESR_ELx_CP15_64_ISS_RT2_SHIFT;
+ u64 val = arch_counter_get_cntvct();
+
+ pt_regs_write_reg(regs, rt, lower_32_bits(val));
+ pt_regs_write_reg(regs, rt2, upper_32_bits(val));
+ arm64_compat_skip_faulting_instruction(regs, 4);
+}
+
+static struct sys64_hook cp15_64_hooks[] = {
+ {
+ .esr_mask = ESR_ELx_CP15_64_ISS_SYS_MASK,
+ .esr_val = ESR_ELx_CP15_64_ISS_SYS_CNTVCT,
+ .handler = compat_cntvct_read_handler,
+ },
+ {},
+};
+
+asmlinkage void __exception do_cp15instr(unsigned int esr, struct pt_regs *regs)
+{
+ struct sys64_hook *hook, *hook_base;
+
+ if (!cp15_cond_valid(esr, regs)) {
+ /*
+ * There is no T16 variant of a CP access, so we
+ * always advance PC by 4 bytes.
+ */
+ arm64_compat_skip_faulting_instruction(regs, 4);
+ return;
+ }
+
+ switch (ESR_ELx_EC(esr)) {
+ case ESR_ELx_EC_CP15_32:
+ hook_base = cp15_32_hooks;
+ break;
+ case ESR_ELx_EC_CP15_64:
+ hook_base = cp15_64_hooks;
+ break;
+ default:
+ do_undefinstr(regs);
+ return;
+ }
+
+ for (hook = hook_base; hook->handler; hook++)
+ if ((hook->esr_mask & esr) == hook->esr_val) {
+ hook->handler(esr, regs);
+ return;
+ }
+
+ /*
+ * New cp15 instructions may previously have been undefined at
+ * EL0. Fall back to our usual undefined instruction handler
+ * so that we handle these consistently.
+ */
+ do_undefinstr(regs);
+}
+#endif
+
asmlinkage void __exception do_sysinstr(unsigned int esr, struct pt_regs *regs)
{
struct sys64_hook *hook;
handler[reason], smp_processor_id(), esr,
esr_get_class_string(esr));
- die("Oops - bad mode", regs, 0);
local_daif_mask();
panic("bad mode");
}
EXCEPTION_TABLE(8) /* __init_begin will be marked RO NX */
NOTES
+ . = ALIGN(PAGE_SIZE);
+ idmap_pg_dir = .;
+ . += IDMAP_DIR_SIZE;
+
+#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
+ tramp_pg_dir = .;
+ . += PAGE_SIZE;
+#endif
+
+#ifdef CONFIG_ARM64_SW_TTBR0_PAN
+ reserved_ttbr0 = .;
+ . += RESERVED_TTBR0_SIZE;
+#endif
+ swapper_pg_dir = .;
+ . += PAGE_SIZE;
+ swapper_pg_end = .;
+
. = ALIGN(SEGMENT_ALIGN);
__init_begin = .;
__inittext_begin = .;
BSS_SECTION(0, 0, 0)
. = ALIGN(PAGE_SIZE);
- idmap_pg_dir = .;
- . += IDMAP_DIR_SIZE;
-
-#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
- tramp_pg_dir = .;
- . += PAGE_SIZE;
-#endif
-
-#ifdef CONFIG_ARM64_SW_TTBR0_PAN
- reserved_ttbr0 = .;
- . += RESERVED_TTBR0_SIZE;
-#endif
- swapper_pg_dir = .;
- . += SWAPPER_DIR_SIZE;
- swapper_pg_end = .;
+ init_pg_dir = .;
+ . += INIT_DIR_SIZE;
+ init_pg_end = .;
__pecoff_data_size = ABSOLUTE(. - __initdata_begin);
_end = .;
b.lo __kvm_handle_stub_hvc
phys_to_ttbr x4, x0
+alternative_if ARM64_HAS_CNP
+ orr x4, x4, #TTBR_CNP_BIT
+alternative_else_nop_endif
msr ttbr0_el2, x4
mrs x4, tcr_el1
vcpu->arch.sysregs_loaded_on_cpu = false;
}
+
+void __hyp_text __kvm_enable_ssbs(void)
+{
+ u64 tmp;
+
+ asm volatile(
+ "mrs %0, sctlr_el2\n"
+ "orr %0, %0, %1\n"
+ "msr sctlr_el2, %0"
+ : "=&r" (tmp) : "L" (SCTLR_ELx_DSSBS));
+}
# when supported by the CPU. Result and argument registers are handled
# correctly, based on the function prototype.
lib-$(CONFIG_ARM64_LSE_ATOMICS) += atomic_ll_sc.o
-CFLAGS_atomic_ll_sc.o := -fcall-used-x0 -ffixed-x1 -ffixed-x2 \
+CFLAGS_atomic_ll_sc.o := -ffixed-x1 -ffixed-x2 \
-ffixed-x3 -ffixed-x4 -ffixed-x5 -ffixed-x6 \
-ffixed-x7 -fcall-saved-x8 -fcall-saved-x9 \
-fcall-saved-x10 -fcall-saved-x11 -fcall-saved-x12 \
UBSAN_SANITIZE_atomic_ll_sc.o := n
lib-$(CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE) += uaccess_flushcache.o
+
+obj-$(CONFIG_CRC32) += crc32.o
--- /dev/null
+/*
+ * Accelerated CRC32(C) using AArch64 CRC instructions
+ *
+ * Copyright (C) 2016 - 2018 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/linkage.h>
+#include <asm/alternative.h>
+#include <asm/assembler.h>
+
+ .cpu generic+crc
+
+ .macro __crc32, c
+0: subs x2, x2, #16
+ b.mi 8f
+ ldp x3, x4, [x1], #16
+CPU_BE( rev x3, x3 )
+CPU_BE( rev x4, x4 )
+ crc32\c\()x w0, w0, x3
+ crc32\c\()x w0, w0, x4
+ b.ne 0b
+ ret
+
+8: tbz x2, #3, 4f
+ ldr x3, [x1], #8
+CPU_BE( rev x3, x3 )
+ crc32\c\()x w0, w0, x3
+4: tbz x2, #2, 2f
+ ldr w3, [x1], #4
+CPU_BE( rev w3, w3 )
+ crc32\c\()w w0, w0, w3
+2: tbz x2, #1, 1f
+ ldrh w3, [x1], #2
+CPU_BE( rev16 w3, w3 )
+ crc32\c\()h w0, w0, w3
+1: tbz x2, #0, 0f
+ ldrb w3, [x1]
+ crc32\c\()b w0, w0, w3
+0: ret
+ .endm
+
+ .align 5
+ENTRY(crc32_le)
+alternative_if_not ARM64_HAS_CRC32
+ b crc32_le_base
+alternative_else_nop_endif
+ __crc32
+ENDPROC(crc32_le)
+
+ .align 5
+ENTRY(__crc32c_le)
+alternative_if_not ARM64_HAS_CRC32
+ b __crc32c_le_base
+alternative_else_nop_endif
+ __crc32 c
+ENDPROC(__crc32c_le)
}
}
-static void flush_context(unsigned int cpu)
+static void flush_context(void)
{
int i;
u64 asid;
return hit;
}
-static u64 new_context(struct mm_struct *mm, unsigned int cpu)
+static u64 new_context(struct mm_struct *mm)
{
static u32 cur_idx = 1;
u64 asid = atomic64_read(&mm->context.id);
/* We're out of ASIDs, so increment the global generation count */
generation = atomic64_add_return_relaxed(ASID_FIRST_VERSION,
&asid_generation);
- flush_context(cpu);
+ flush_context();
/* We have more ASIDs than CPUs, so this will always succeed */
asid = find_next_zero_bit(asid_map, NUM_USER_ASIDS, 1);
unsigned long flags;
u64 asid, old_active_asid;
+ if (system_supports_cnp())
+ cpu_set_reserved_ttbr0();
+
asid = atomic64_read(&mm->context.id);
/*
/* Check that our ASID belongs to the current generation. */
asid = atomic64_read(&mm->context.id);
if ((asid ^ atomic64_read(&asid_generation)) >> asid_bits) {
- asid = new_context(mm, cpu);
+ asid = new_context(mm);
atomic64_set(&mm->context.id, asid);
}
#endif
{ MODULES_VADDR, "Modules start" },
{ MODULES_END, "Modules end" },
- { VMALLOC_START, "vmalloc() Area" },
- { VMALLOC_END, "vmalloc() End" },
+ { VMALLOC_START, "vmalloc() area" },
+ { VMALLOC_END, "vmalloc() end" },
{ FIXADDR_START, "Fixmap start" },
{ FIXADDR_TOP, "Fixmap end" },
{ PCI_IO_START, "PCI I/O start" },
{ VMEMMAP_START, "vmemmap start" },
{ VMEMMAP_START + VMEMMAP_SIZE, "vmemmap end" },
#endif
- { PAGE_OFFSET, "Linear Mapping" },
+ { PAGE_OFFSET, "Linear mapping" },
{ -1, NULL },
};
#include <asm/cmpxchg.h>
#include <asm/cpufeature.h>
#include <asm/exception.h>
+#include <asm/daifflags.h>
#include <asm/debug-monitors.h>
#include <asm/esr.h>
#include <asm/sysreg.h>
};
static const struct fault_info fault_info[];
+static struct fault_info debug_fault_info[];
static inline const struct fault_info *esr_to_fault_info(unsigned int esr)
{
- return fault_info + (esr & 63);
+ return fault_info + (esr & ESR_ELx_FSC);
+}
+
+static inline const struct fault_info *esr_to_debug_fault_info(unsigned int esr)
+{
+ return debug_fault_info + DBG_ESR_EVT(esr);
}
#ifdef CONFIG_KPROBES
return ESR_ELx_EC(esr) == ESR_ELx_EC_IABT_CUR;
}
-static inline bool is_el1_permission_fault(unsigned int esr,
- struct pt_regs *regs,
- unsigned long addr)
+static inline bool is_el1_permission_fault(unsigned long addr, unsigned int esr,
+ struct pt_regs *regs)
{
unsigned int ec = ESR_ELx_EC(esr);
unsigned int fsc_type = esr & ESR_ELx_FSC_TYPE;
if (!is_el1_instruction_abort(esr) && fixup_exception(regs))
return;
- if (is_el1_permission_fault(esr, regs, addr)) {
+ if (is_el1_permission_fault(addr, esr, regs)) {
if (esr & ESR_ELx_WNR)
msg = "write to read-only memory";
else
mm_flags |= FAULT_FLAG_WRITE;
}
- if (addr < TASK_SIZE && is_el1_permission_fault(esr, regs, addr)) {
+ if (addr < TASK_SIZE && is_el1_permission_fault(addr, esr, regs)) {
/* regs->orig_addr_limit may be 0 if we entered from EL0 */
if (regs->orig_addr_limit == KERNEL_DS)
die_kernel_fault("access to user memory with fs=KERNEL_DS",
if (addr > TASK_SIZE)
arm64_apply_bp_hardening();
- local_irq_enable();
+ local_daif_restore(DAIF_PROCCTX);
do_mem_abort(addr, esr, regs);
}
if (user_mode(regs)) {
if (instruction_pointer(regs) > TASK_SIZE)
arm64_apply_bp_hardening();
- local_irq_enable();
+ local_daif_restore(DAIF_PROCCTX);
}
clear_siginfo(&info);
unsigned int esr,
struct pt_regs *regs)
{
- const struct fault_info *inf = debug_fault_info + DBG_ESR_EVT(esr);
+ const struct fault_info *inf = esr_to_debug_fault_info(esr);
int rv;
/*
return rv;
}
NOKPROBE_SYMBOL(do_debug_exception);
-
-#ifdef CONFIG_ARM64_PAN
-void cpu_enable_pan(const struct arm64_cpu_capabilities *__unused)
-{
- /*
- * We modify PSTATE. This won't work from irq context as the PSTATE
- * is discarded once we return from the exception.
- */
- WARN_ON_ONCE(in_interrupt());
-
- sysreg_clear_set(sctlr_el1, SCTLR_EL1_SPAN, 0);
- asm(SET_PSTATE_PAN(1));
-}
-#endif /* CONFIG_ARM64_PAN */
#endif /* CONFIG_NUMA */
-#ifdef CONFIG_HAVE_ARCH_PFN_VALID
int pfn_valid(unsigned long pfn)
{
phys_addr_t addr = pfn << PAGE_SHIFT;
return memblock_is_map_memory(addr);
}
EXPORT_SYMBOL(pfn_valid);
-#endif
#ifndef CONFIG_SPARSEMEM
static void __init arm64_memory_present(void)
/*
* We are going to perform proper setup of shadow memory.
- * At first we should unmap early shadow (clear_pgds() call bellow).
+ * At first we should unmap early shadow (clear_pgds() call below).
* However, instrumented code couldn't execute without shadow memory.
* tmp_pg_dir used to keep early shadow mapped until full shadow
* setup will be finished.
static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss __maybe_unused;
static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss __maybe_unused;
+static DEFINE_SPINLOCK(swapper_pgdir_lock);
+
+void set_swapper_pgd(pgd_t *pgdp, pgd_t pgd)
+{
+ pgd_t *fixmap_pgdp;
+
+ spin_lock(&swapper_pgdir_lock);
+ fixmap_pgdp = pgd_set_fixmap(__pa_symbol(pgdp));
+ WRITE_ONCE(*fixmap_pgdp, pgd);
+ /*
+ * We need dsb(ishst) here to ensure the page-table-walker sees
+ * our new entry before set_p?d() returns. The fixmap's
+ * flush_tlb_kernel_range() via clear_fixmap() does this for us.
+ */
+ pgd_clear_fixmap();
+ spin_unlock(&swapper_pgdir_lock);
+}
+
pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
unsigned long size, pgprot_t vma_prot)
{
*/
void __init paging_init(void)
{
- phys_addr_t pgd_phys = early_pgtable_alloc();
- pgd_t *pgdp = pgd_set_fixmap(pgd_phys);
+ pgd_t *pgdp = pgd_set_fixmap(__pa_symbol(swapper_pg_dir));
map_kernel(pgdp);
map_mem(pgdp);
- /*
- * We want to reuse the original swapper_pg_dir so we don't have to
- * communicate the new address to non-coherent secondaries in
- * secondary_entry, and so cpu_switch_mm can generate the address with
- * adrp+add rather than a load from some global variable.
- *
- * To do this we need to go via a temporary pgd.
- */
- cpu_replace_ttbr1(__va(pgd_phys));
- memcpy(swapper_pg_dir, pgdp, PGD_SIZE);
- cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
-
pgd_clear_fixmap();
- memblock_free(pgd_phys, PAGE_SIZE);
- /*
- * We only reuse the PGD from the swapper_pg_dir, not the pud + pmd
- * allocated with it.
- */
- memblock_free(__pa_symbol(swapper_pg_dir) + PAGE_SIZE,
- __pa_symbol(swapper_pg_end) - __pa_symbol(swapper_pg_dir)
- - PAGE_SIZE);
+ cpu_replace_ttbr1(lm_alias(swapper_pg_dir));
+ init_mm.pgd = swapper_pg_dir;
+
+ memblock_free(__pa_symbol(init_pg_dir),
+ __pa_symbol(init_pg_end) - __pa_symbol(init_pg_dir));
}
/*
nodes_clear(numa_nodes_parsed);
nodes_clear(node_possible_map);
nodes_clear(node_online_map);
- numa_free_distance();
ret = numa_alloc_distance();
if (ret < 0)
ret = init_func();
if (ret < 0)
- return ret;
+ goto out_free_distance;
if (nodes_empty(numa_nodes_parsed)) {
pr_info("No NUMA configuration found\n");
- return -EINVAL;
+ ret = -EINVAL;
+ goto out_free_distance;
}
ret = numa_register_nodes();
if (ret < 0)
- return ret;
+ goto out_free_distance;
setup_node_to_cpumask_map();
return 0;
+out_free_distance:
+ numa_free_distance();
+ return ret;
}
/**
if (numa_off)
pr_info("NUMA disabled\n"); /* Forced off on command line. */
pr_info("Faking a node at [mem %#018Lx-%#018Lx]\n",
- 0LLU, PFN_PHYS(max_pfn) - 1);
+ memblock_start_of_DRAM(), memblock_end_of_DRAM() - 1);
for_each_memblock(memory, mblk) {
ret = numa_add_memblk(0, mblk->base, mblk->base + mblk->size);
mrs x2, ttbr1_el1
mmid x1, x1 // get mm->context.id
phys_to_ttbr x3, x0
+
+alternative_if ARM64_HAS_CNP
+ cbz x1, 1f // skip CNP for reserved ASID
+ orr x3, x3, #TTBR_CNP_BIT
+1:
+alternative_else_nop_endif
#ifdef CONFIG_ARM64_SW_TTBR0_PAN
bfi x3, x1, #48, #16 // set the ASID field in TTBR0
#endif
.endm
/*
- * void idmap_cpu_replace_ttbr1(phys_addr_t new_pgd)
+ * void idmap_cpu_replace_ttbr1(phys_addr_t ttbr1)
*
* This is the low-level counterpart to cpu_replace_ttbr1, and should not be
* called by anything else. It can only be executed from a TTBR0 mapping.
__idmap_cpu_set_reserved_ttbr1 x1, x3
- phys_to_ttbr x3, x0
- msr ttbr1_el1, x3
+ msr ttbr1_el1, x0
isb
restore_daif x2
select ARCH_HAS_SYNC_DMA_FOR_CPU
select ARCH_HAS_SYNC_DMA_FOR_DEVICE
select CLKDEV_LOOKUP
- select DMA_NONCOHERENT_OPS
+ select DMA_DIRECT_OPS
select GENERIC_ATOMIC64
select GENERIC_IRQ_SHOW
select HAVE_ARCH_TRACEHOOK
config HEXAGON
def_bool y
+ select ARCH_HAS_SYNC_DMA_FOR_DEVICE
select ARCH_NO_PREEMPT
select HAVE_OPROFILE
# Other pending projects/to-do items.
select GENERIC_CLOCKEVENTS_BROADCAST
select MODULES_USE_ELF_RELA
select GENERIC_CPU_DEVICES
+ select DMA_DIRECT_OPS
---help---
Qualcomm Hexagon is a processor architecture designed for high
performance and low power across a wide variety of applications.
generic-y += current.h
generic-y += device.h
generic-y += div64.h
+generic-y += dma-mapping.h
generic-y += emergency-restart.h
generic-y += extable.h
generic-y += fb.h
+++ /dev/null
-/*
- * DMA operations for the Hexagon architecture
- *
- * Copyright (c) 2010-2011, The Linux Foundation. All rights reserved.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 and
- * only version 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
- * 02110-1301, USA.
- */
-
-#ifndef _ASM_DMA_MAPPING_H
-#define _ASM_DMA_MAPPING_H
-
-#include <linux/types.h>
-#include <linux/cache.h>
-#include <linux/mm.h>
-#include <linux/scatterlist.h>
-#include <linux/dma-debug.h>
-#include <asm/io.h>
-
-struct device;
-
-extern const struct dma_map_ops *dma_ops;
-
-static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
-{
- return dma_ops;
-}
-
-#endif
* 02110-1301, USA.
*/
-#include <linux/dma-mapping.h>
-#include <linux/dma-direct.h>
+#include <linux/dma-noncoherent.h>
#include <linux/bootmem.h>
#include <linux/genalloc.h>
-#include <asm/dma-mapping.h>
#include <linux/module.h>
#include <asm/page.h>
-#define HEXAGON_MAPPING_ERROR 0
-
-const struct dma_map_ops *dma_ops;
-EXPORT_SYMBOL(dma_ops);
-
-static inline void *dma_addr_to_virt(dma_addr_t dma_addr)
-{
- return phys_to_virt((unsigned long) dma_addr);
-}
-
static struct gen_pool *coherent_pool;
/* Allocates from a pool of uncached memory that was reserved at boot time */
-static void *hexagon_dma_alloc_coherent(struct device *dev, size_t size,
- dma_addr_t *dma_addr, gfp_t flag,
- unsigned long attrs)
+void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_addr,
+ gfp_t flag, unsigned long attrs)
{
void *ret;
return ret;
}
-static void hexagon_free_coherent(struct device *dev, size_t size, void *vaddr,
- dma_addr_t dma_addr, unsigned long attrs)
+void arch_dma_free(struct device *dev, size_t size, void *vaddr,
+ dma_addr_t dma_addr, unsigned long attrs)
{
gen_pool_free(coherent_pool, (unsigned long) vaddr, size);
}
-static int check_addr(const char *name, struct device *hwdev,
- dma_addr_t bus, size_t size)
-{
- if (hwdev && hwdev->dma_mask && !dma_capable(hwdev, bus, size)) {
- if (*hwdev->dma_mask >= DMA_BIT_MASK(32))
- printk(KERN_ERR
- "%s: overflow %Lx+%zu of device mask %Lx\n",
- name, (long long)bus, size,
- (long long)*hwdev->dma_mask);
- return 0;
- }
- return 1;
-}
-
-static int hexagon_map_sg(struct device *hwdev, struct scatterlist *sg,
- int nents, enum dma_data_direction dir,
- unsigned long attrs)
+void arch_sync_dma_for_device(struct device *dev, phys_addr_t paddr,
+ size_t size, enum dma_data_direction dir)
{
- struct scatterlist *s;
- int i;
-
- WARN_ON(nents == 0 || sg[0].length == 0);
-
- for_each_sg(sg, s, nents, i) {
- s->dma_address = sg_phys(s);
- if (!check_addr("map_sg", hwdev, s->dma_address, s->length))
- return 0;
-
- s->dma_length = s->length;
+ void *addr = phys_to_virt(paddr);
- if (attrs & DMA_ATTR_SKIP_CPU_SYNC)
- continue;
-
- flush_dcache_range(dma_addr_to_virt(s->dma_address),
- dma_addr_to_virt(s->dma_address + s->length));
- }
-
- return nents;
-}
-
-/*
- * address is virtual
- */
-static inline void dma_sync(void *addr, size_t size,
- enum dma_data_direction dir)
-{
switch (dir) {
case DMA_TO_DEVICE:
hexagon_clean_dcache_range((unsigned long) addr,
BUG();
}
}
-
-/**
- * hexagon_map_page() - maps an address for device DMA
- * @dev: pointer to DMA device
- * @page: pointer to page struct of DMA memory
- * @offset: offset within page
- * @size: size of memory to map
- * @dir: transfer direction
- * @attrs: pointer to DMA attrs (not used)
- *
- * Called to map a memory address to a DMA address prior
- * to accesses to/from device.
- *
- * We don't particularly have many hoops to jump through
- * so far. Straight translation between phys and virtual.
- *
- * DMA is not cache coherent so sync is necessary; this
- * seems to be a convenient place to do it.
- *
- */
-static dma_addr_t hexagon_map_page(struct device *dev, struct page *page,
- unsigned long offset, size_t size,
- enum dma_data_direction dir,
- unsigned long attrs)
-{
- dma_addr_t bus = page_to_phys(page) + offset;
- WARN_ON(size == 0);
-
- if (!check_addr("map_single", dev, bus, size))
- return HEXAGON_MAPPING_ERROR;
-
- if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
- dma_sync(dma_addr_to_virt(bus), size, dir);
-
- return bus;
-}
-
-static void hexagon_sync_single_for_cpu(struct device *dev,
- dma_addr_t dma_handle, size_t size,
- enum dma_data_direction dir)
-{
- dma_sync(dma_addr_to_virt(dma_handle), size, dir);
-}
-
-static void hexagon_sync_single_for_device(struct device *dev,
- dma_addr_t dma_handle, size_t size,
- enum dma_data_direction dir)
-{
- dma_sync(dma_addr_to_virt(dma_handle), size, dir);
-}
-
-static int hexagon_mapping_error(struct device *dev, dma_addr_t dma_addr)
-{
- return dma_addr == HEXAGON_MAPPING_ERROR;
-}
-
-const struct dma_map_ops hexagon_dma_ops = {
- .alloc = hexagon_dma_alloc_coherent,
- .free = hexagon_free_coherent,
- .map_sg = hexagon_map_sg,
- .map_page = hexagon_map_page,
- .sync_single_for_cpu = hexagon_sync_single_for_cpu,
- .sync_single_for_device = hexagon_sync_single_for_device,
- .mapping_error = hexagon_mapping_error,
-};
-
-void __init hexagon_dma_init(void)
-{
- if (dma_ops)
- return;
-
- dma_ops = &hexagon_dma_ops;
-}
.unmap_page = sba_unmap_page,
.map_sg = sba_map_sg_attrs,
.unmap_sg = sba_unmap_sg_attrs,
- .sync_single_for_cpu = machvec_dma_sync_single,
- .sync_sg_for_cpu = machvec_dma_sync_sg,
- .sync_single_for_device = machvec_dma_sync_single,
- .sync_sg_for_device = machvec_dma_sync_sg,
.dma_supported = sba_dma_supported,
.mapping_error = sba_dma_mapping_error,
};
#include <linux/scatterlist.h>
#include <linux/dma-debug.h>
-#define ARCH_HAS_DMA_GET_REQUIRED_MASK
-
extern const struct dma_map_ops *dma_ops;
extern struct ia64_machine_vector ia64_mv;
extern void set_iommu_machvec(void);
-extern void machvec_dma_sync_single(struct device *, dma_addr_t, size_t,
- enum dma_data_direction);
-extern void machvec_dma_sync_sg(struct device *, struct scatterlist *, int,
- enum dma_data_direction);
-
static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return platform_dma_get_ops(NULL);
/* 10 seconds */
#define DMAR_OPERATION_TIMEOUT (((cycles_t) local_cpu_data->itc_freq)*10)
-extern void pci_iommu_shutdown(void);
extern void no_iommu_init(void);
#ifdef CONFIG_INTEL_IOMMU
extern int force_iommu, no_iommu;
#define no_iommu (1)
#define iommu_detected (0)
#endif
-extern void iommu_dma_init(void);
extern void machvec_init(const char *name);
#endif
/* DMA-mapping interface: */
typedef void ia64_mv_dma_init (void);
-typedef u64 ia64_mv_dma_get_required_mask (struct device *);
typedef const struct dma_map_ops *ia64_mv_dma_get_ops(struct device *);
/*
# define platform_global_tlb_purge ia64_mv.global_tlb_purge
# define platform_tlb_migrate_finish ia64_mv.tlb_migrate_finish
# define platform_dma_init ia64_mv.dma_init
-# define platform_dma_get_required_mask ia64_mv.dma_get_required_mask
# define platform_dma_get_ops ia64_mv.dma_get_ops
# define platform_irq_to_vector ia64_mv.irq_to_vector
# define platform_local_vector_to_irq ia64_mv.local_vector_to_irq
ia64_mv_global_tlb_purge_t *global_tlb_purge;
ia64_mv_tlb_migrate_finish_t *tlb_migrate_finish;
ia64_mv_dma_init *dma_init;
- ia64_mv_dma_get_required_mask *dma_get_required_mask;
ia64_mv_dma_get_ops *dma_get_ops;
ia64_mv_irq_to_vector *irq_to_vector;
ia64_mv_local_vector_to_irq *local_vector_to_irq;
platform_global_tlb_purge, \
platform_tlb_migrate_finish, \
platform_dma_init, \
- platform_dma_get_required_mask, \
platform_dma_get_ops, \
platform_irq_to_vector, \
platform_local_vector_to_irq, \
#ifndef platform_dma_get_ops
# define platform_dma_get_ops dma_get_ops
#endif
-#ifndef platform_dma_get_required_mask
-# define platform_dma_get_required_mask ia64_dma_get_required_mask
-#endif
#ifndef platform_irq_to_vector
# define platform_irq_to_vector __ia64_irq_to_vector
#endif
extern ia64_mv_send_ipi_t ia64_send_ipi;
extern ia64_mv_global_tlb_purge_t ia64_global_tlb_purge;
-extern ia64_mv_dma_get_required_mask ia64_dma_get_required_mask;
extern ia64_mv_irq_to_vector __ia64_irq_to_vector;
extern ia64_mv_local_vector_to_irq __ia64_local_vector_to_irq;
extern ia64_mv_pci_get_legacy_mem_t ia64_pci_get_legacy_mem;
extern ia64_mv_readw_t __sn_readw_relaxed;
extern ia64_mv_readl_t __sn_readl_relaxed;
extern ia64_mv_readq_t __sn_readq_relaxed;
-extern ia64_mv_dma_get_required_mask sn_dma_get_required_mask;
extern ia64_mv_dma_init sn_dma_init;
extern ia64_mv_migrate_t sn_migrate;
extern ia64_mv_kernel_launch_event_t sn_kernel_launch_event;
#define platform_pci_get_legacy_mem sn_pci_get_legacy_mem
#define platform_pci_legacy_read sn_pci_legacy_read
#define platform_pci_legacy_write sn_pci_legacy_write
-#define platform_dma_get_required_mask sn_dma_get_required_mask
#define platform_dma_init sn_dma_init
#define platform_migrate sn_migrate
#define platform_kernel_launch_event sn_kernel_launch_event
} while (md);
return 0; /* never reached */
}
-EXPORT_SYMBOL(kern_mem_attribute);
int
valid_phys_addr_range (phys_addr_t phys_addr, unsigned long size)
{
}
EXPORT_SYMBOL(machvec_timer_interrupt);
-
-void
-machvec_dma_sync_single(struct device *hwdev, dma_addr_t dma_handle, size_t size,
- enum dma_data_direction dir)
-{
- mb();
-}
-EXPORT_SYMBOL(machvec_dma_sync_single);
-
-void
-machvec_dma_sync_sg(struct device *hwdev, struct scatterlist *sg, int n,
- enum dma_data_direction dir)
-{
- mb();
-}
-EXPORT_SYMBOL(machvec_dma_sync_sg);
#include <linux/kernel.h>
#include <asm/page.h>
-dma_addr_t bad_dma_address __read_mostly;
-EXPORT_SYMBOL(bad_dma_address);
-
-static int iommu_sac_force __read_mostly;
-
int no_iommu __read_mostly;
#ifdef CONFIG_IOMMU_DEBUG
int force_iommu __read_mostly = 1;
int iommu_pass_through;
-extern struct dma_map_ops intel_dma_ops;
-
static int __init pci_iommu_init(void)
{
if (iommu_detected)
/* Must execute after PCI subsystem */
fs_initcall(pci_iommu_init);
-void pci_iommu_shutdown(void)
-{
- return;
-}
-
-void __init
-iommu_dma_init(void)
-{
- return;
-}
-
-int iommu_dma_supported(struct device *dev, u64 mask)
-{
- /* Copied from i386. Doesn't make much sense, because it will
- only work for pci_alloc_coherent.
- The caller just has to use GFP_DMA in this case. */
- if (mask < DMA_BIT_MASK(24))
- return 0;
-
- /* Tell the device to use SAC when IOMMU force is on. This
- allows the driver to use cheaper accesses in some cases.
-
- Problem with this is that if we overflow the IOMMU area and
- return DAC as fallback address the device may not handle it
- correctly.
-
- As a special case some controllers have a 39bit address
- mode that is as efficient as 32bit (aic79xx). Don't force
- SAC for these. Assume all masks <= 40 bits are of this
- type. Normally this doesn't make any difference, but gives
- more gentle handling of IOMMU overflow. */
- if (iommu_sac_force && (mask >= DMA_BIT_MASK(40))) {
- dev_info(dev, "Force SAC with mask %llx\n", mask);
- return 0;
- }
-
- return 1;
-}
-EXPORT_SYMBOL(iommu_dma_supported);
-
void __init pci_iommu_alloc(void)
{
- dma_ops = &intel_dma_ops;
-
- intel_dma_ops.sync_single_for_cpu = machvec_dma_sync_single;
- intel_dma_ops.sync_sg_for_cpu = machvec_dma_sync_sg;
- intel_dma_ops.sync_single_for_device = machvec_dma_sync_single;
- intel_dma_ops.sync_sg_for_device = machvec_dma_sync_sg;
- intel_dma_ops.dma_supported = iommu_dma_supported;
-
/*
* The order of these functions is important for
* fall-back/fail-over reasons
pci_dfl_cache_line_size = (1 << cci.pcci_line_size) / 4;
}
-u64 ia64_dma_get_required_mask(struct device *dev)
-{
- u32 low_totalram = ((max_pfn - 1) << PAGE_SHIFT);
- u32 high_totalram = ((max_pfn - 1) >> (32 - PAGE_SHIFT));
- u64 mask;
-
- if (!high_totalram) {
- /* convert to mask just covering totalram */
- low_totalram = (1 << (fls(low_totalram) - 1));
- low_totalram += low_totalram - 1;
- mask = low_totalram;
- } else {
- high_totalram = (1 << (fls(high_totalram) - 1));
- high_totalram += high_totalram - 1;
- mask = (((u64)high_totalram) << 32) + 0xffffffff;
- }
- return mask;
-}
-EXPORT_SYMBOL_GPL(ia64_dma_get_required_mask);
-
-u64 dma_get_required_mask(struct device *dev)
-{
- return platform_dma_get_required_mask(dev);
-}
-EXPORT_SYMBOL_GPL(dma_get_required_mask);
-
static int __init pcibios_init(void)
{
set_pci_dfl_cacheline_size();
return nhwentries;
}
-static void sn_dma_sync_single_for_cpu(struct device *dev, dma_addr_t dma_handle,
- size_t size, enum dma_data_direction dir)
-{
- BUG_ON(!dev_is_pci(dev));
-}
-
-static void sn_dma_sync_single_for_device(struct device *dev, dma_addr_t dma_handle,
- size_t size,
- enum dma_data_direction dir)
-{
- BUG_ON(!dev_is_pci(dev));
-}
-
-static void sn_dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg,
- int nelems, enum dma_data_direction dir)
-{
- BUG_ON(!dev_is_pci(dev));
-}
-
-static void sn_dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
- int nelems, enum dma_data_direction dir)
-{
- BUG_ON(!dev_is_pci(dev));
-}
-
static int sn_dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
{
return 0;
}
-u64 sn_dma_get_required_mask(struct device *dev)
+static u64 sn_dma_get_required_mask(struct device *dev)
{
return DMA_BIT_MASK(64);
}
-EXPORT_SYMBOL_GPL(sn_dma_get_required_mask);
char *sn_pci_get_legacy_mem(struct pci_bus *bus)
{
.unmap_page = sn_dma_unmap_page,
.map_sg = sn_dma_map_sg,
.unmap_sg = sn_dma_unmap_sg,
- .sync_single_for_cpu = sn_dma_sync_single_for_cpu,
- .sync_sg_for_cpu = sn_dma_sync_sg_for_cpu,
- .sync_single_for_device = sn_dma_sync_single_for_device,
- .sync_sg_for_device = sn_dma_sync_sg_for_device,
.mapping_error = sn_dma_mapping_error,
.dma_supported = sn_dma_supported,
+ .get_required_mask = sn_dma_get_required_mask,
};
void sn_dma_init(void)
select MODULES_USE_ELF_RELA
select OLD_SIGSUSPEND3
select OLD_SIGACTION
- select DMA_NONCOHERENT_OPS if HAS_DMA
+ select DMA_DIRECT_OPS if HAS_DMA
select HAVE_MEMBLOCK
select ARCH_DISCARD_MEMBLOCK
select NO_BOOTMEM
len = bvec.bv_len;
len >>= 9;
nfhd_read_write(dev->id, 0, dir, sec >> shift, len >> shift,
- bvec_to_phys(&bvec));
+ page_to_phys(bvec.bv_page) + bvec.bv_offset);
sec += len;
}
bio_endio(bio);
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _ASM_M68K_FD_H
-#define _ASM_M68K_FD_H
-
-/* Definitions for the Atari Floppy driver */
-
-struct atari_format_descr {
- int track; /* to be formatted */
- int head; /* "" "" */
- int sect_offset; /* offset of first sector */
-};
-
-#endif
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _LINUX_FDREG_H
-#define _LINUX_FDREG_H
-
-/*
-** WD1772 stuff
- */
-
-/* register codes */
-
-#define FDCSELREG_STP (0x80) /* command/status register */
-#define FDCSELREG_TRA (0x82) /* track register */
-#define FDCSELREG_SEC (0x84) /* sector register */
-#define FDCSELREG_DTA (0x86) /* data register */
-
-/* register names for FDC_READ/WRITE macros */
-
-#define FDCREG_CMD 0
-#define FDCREG_STATUS 0
-#define FDCREG_TRACK 2
-#define FDCREG_SECTOR 4
-#define FDCREG_DATA 6
-
-/* command opcodes */
-
-#define FDCCMD_RESTORE (0x00) /* - */
-#define FDCCMD_SEEK (0x10) /* | */
-#define FDCCMD_STEP (0x20) /* | TYP 1 Commands */
-#define FDCCMD_STIN (0x40) /* | */
-#define FDCCMD_STOT (0x60) /* - */
-#define FDCCMD_RDSEC (0x80) /* - TYP 2 Commands */
-#define FDCCMD_WRSEC (0xa0) /* - " */
-#define FDCCMD_RDADR (0xc0) /* - */
-#define FDCCMD_RDTRA (0xe0) /* | TYP 3 Commands */
-#define FDCCMD_WRTRA (0xf0) /* - */
-#define FDCCMD_FORCI (0xd0) /* - TYP 4 Command */
-
-/* command modifier bits */
-
-#define FDCCMDADD_SR6 (0x00) /* step rate settings */
-#define FDCCMDADD_SR12 (0x01)
-#define FDCCMDADD_SR2 (0x02)
-#define FDCCMDADD_SR3 (0x03)
-#define FDCCMDADD_V (0x04) /* verify */
-#define FDCCMDADD_H (0x08) /* wait for spin-up */
-#define FDCCMDADD_U (0x10) /* update track register */
-#define FDCCMDADD_M (0x10) /* multiple sector access */
-#define FDCCMDADD_E (0x04) /* head settling flag */
-#define FDCCMDADD_P (0x02) /* precompensation off */
-#define FDCCMDADD_A0 (0x01) /* DAM flag */
-
-/* status register bits */
-
-#define FDCSTAT_MOTORON (0x80) /* motor on */
-#define FDCSTAT_WPROT (0x40) /* write protected (FDCCMD_WR*) */
-#define FDCSTAT_SPINUP (0x20) /* motor speed stable (Type I) */
-#define FDCSTAT_DELDAM (0x20) /* sector has deleted DAM (Type II+III) */
-#define FDCSTAT_RECNF (0x10) /* record not found */
-#define FDCSTAT_CRC (0x08) /* CRC error */
-#define FDCSTAT_TR00 (0x04) /* Track 00 flag (Type I) */
-#define FDCSTAT_LOST (0x04) /* Lost Data (Type II+III) */
-#define FDCSTAT_IDX (0x02) /* Index status (Type I) */
-#define FDCSTAT_DRQ (0x02) /* DRQ status (Type II+III) */
-#define FDCSTAT_BUSY (0x01) /* FDC is busy */
-
-
-/* PSG Port A Bit Nr 0 .. Side Sel .. 0 -> Side 1 1 -> Side 2 */
-#define DSKSIDE (0x01)
-
-#define DSKDRVNONE (0x06)
-#define DSKDRV0 (0x02)
-#define DSKDRV1 (0x04)
-
-/* step rates */
-#define FDCSTEP_6 0x00
-#define FDCSTEP_12 0x01
-#define FDCSTEP_2 0x02
-#define FDCSTEP_3 0x03
-
-#endif
config MICROBLAZE
def_bool y
select ARCH_NO_SWAP
+ select ARCH_HAS_DMA_COHERENT_TO_PFN if MMU
select ARCH_HAS_GCOV_PROFILE_ALL
select ARCH_HAS_SYNC_DMA_FOR_CPU
select ARCH_HAS_SYNC_DMA_FOR_DEVICE
select TIMER_OF
select CLONE_BACKWARDS3
select COMMON_CLK
- select DMA_NONCOHERENT_OPS
- select DMA_NONCOHERENT_MMAP
+ select DMA_DIRECT_OPS
select GENERIC_ATOMIC64
select GENERIC_CLOCKEVENTS
select GENERIC_CPU_DEVICES
extern unsigned long ioremap_bot, ioremap_base;
-unsigned long consistent_virt_to_pfn(void *vaddr);
-
void setup_memory(void);
#endif /* __ASSEMBLY__ */
{
__dma_sync(dev, paddr, size, dir);
}
-
-int arch_dma_mmap(struct device *dev, struct vm_area_struct *vma,
- void *cpu_addr, dma_addr_t handle, size_t size,
- unsigned long attrs)
-{
-#ifdef CONFIG_MMU
- unsigned long user_count = vma_pages(vma);
- unsigned long count = PAGE_ALIGN(size) >> PAGE_SHIFT;
- unsigned long off = vma->vm_pgoff;
- unsigned long pfn;
-
- if (off >= count || user_count > (count - off))
- return -ENXIO;
-
- vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
- pfn = consistent_virt_to_pfn(cpu_addr);
- return remap_pfn_range(vma, vma->vm_start, pfn + off,
- vma->vm_end - vma->vm_start, vma->vm_page_prot);
-#else
- return -ENXIO;
-#endif
-}
return pte_offset_kernel(pmd_offset(pgd_offset_k(addr), addr), addr);
}
-unsigned long consistent_virt_to_pfn(void *vaddr)
+long arch_dma_coherent_to_pfn(struct device *dev, void *vaddr,
+ dma_addr_t dma_addr)
{
pte_t *ptep = consistent_virt_to_pte(vaddr);
bool
config DMA_MAYBE_COHERENT
+ select ARCH_HAS_DMA_COHERENCE_H
select DMA_NONCOHERENT
bool
config DMA_PERDEV_COHERENT
bool
- select DMA_MAYBE_COHERENT
+ select DMA_NONCOHERENT
config DMA_NONCOHERENT
bool
+ select ARCH_HAS_DMA_MMAP_PGPROT
select ARCH_HAS_SYNC_DMA_FOR_DEVICE
select ARCH_HAS_SYNC_DMA_FOR_CPU
select NEED_DMA_MAP_STATE
- select DMA_NONCOHERENT_MMAP
+ select ARCH_HAS_DMA_COHERENT_TO_PFN
select DMA_NONCOHERENT_CACHE_SYNC
- select DMA_NONCOHERENT_OPS
config SYS_HAS_EARLY_PRINTK
bool
#include <linux/leds.h>
#include <linux/mmc/host.h>
#include <linux/mtd/mtd.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <linux/platform_device.h>
#include <linux/serial_8250.h>
#include <linux/spi/spi.h>
/**********************************************************************/
-static void au1200_nand_cmd_ctrl(struct mtd_info *mtd, int cmd,
+static void au1200_nand_cmd_ctrl(struct nand_chip *this, int cmd,
unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- unsigned long ioaddr = (unsigned long)this->IO_ADDR_W;
+ unsigned long ioaddr = (unsigned long)this->legacy.IO_ADDR_W;
ioaddr &= 0xffffff00;
/* assume we want to r/w real data by default */
ioaddr += MEM_STNAND_DATA;
}
- this->IO_ADDR_R = this->IO_ADDR_W = (void __iomem *)ioaddr;
+ this->legacy.IO_ADDR_R = this->legacy.IO_ADDR_W = (void __iomem *)ioaddr;
if (cmd != NAND_CMD_NONE) {
- __raw_writeb(cmd, this->IO_ADDR_W);
+ __raw_writeb(cmd, this->legacy.IO_ADDR_W);
wmb();
}
}
-static int au1200_nand_device_ready(struct mtd_info *mtd)
+static int au1200_nand_device_ready(struct nand_chip *this)
{
return alchemy_rdsmem(AU1000_MEM_STSTAT) & 1;
}
#include <linux/mmc/host.h>
#include <linux/module.h>
#include <linux/mtd/mtd.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <linux/platform_device.h>
#include <linux/smsc911x.h>
#include <linux/wm97xx.h>
/**********************************************************************/
-static void au1300_nand_cmd_ctrl(struct mtd_info *mtd, int cmd,
+static void au1300_nand_cmd_ctrl(struct nand_chip *this, int cmd,
unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- unsigned long ioaddr = (unsigned long)this->IO_ADDR_W;
+ unsigned long ioaddr = (unsigned long)this->legacy.IO_ADDR_W;
ioaddr &= 0xffffff00;
/* assume we want to r/w real data by default */
ioaddr += MEM_STNAND_DATA;
}
- this->IO_ADDR_R = this->IO_ADDR_W = (void __iomem *)ioaddr;
+ this->legacy.IO_ADDR_R = this->legacy.IO_ADDR_W = (void __iomem *)ioaddr;
if (cmd != NAND_CMD_NONE) {
- __raw_writeb(cmd, this->IO_ADDR_W);
+ __raw_writeb(cmd, this->legacy.IO_ADDR_W);
wmb();
}
}
-static int au1300_nand_device_ready(struct mtd_info *mtd)
+static int au1300_nand_device_ready(struct nand_chip *this)
{
return alchemy_rdsmem(AU1000_MEM_STSTAT) & 1;
}
#include <linux/io.h>
#include <linux/interrupt.h>
#include <linux/mtd/mtd.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <linux/platform_device.h>
#include <linux/pm.h>
#include <linux/spi/spi.h>
/**********************************************************************/
-static void au1550_nand_cmd_ctrl(struct mtd_info *mtd, int cmd,
+static void au1550_nand_cmd_ctrl(struct nand_chip *this, int cmd,
unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- unsigned long ioaddr = (unsigned long)this->IO_ADDR_W;
+ unsigned long ioaddr = (unsigned long)this->legacy.IO_ADDR_W;
ioaddr &= 0xffffff00;
/* assume we want to r/w real data by default */
ioaddr += MEM_STNAND_DATA;
}
- this->IO_ADDR_R = this->IO_ADDR_W = (void __iomem *)ioaddr;
+ this->legacy.IO_ADDR_R = this->legacy.IO_ADDR_W = (void __iomem *)ioaddr;
if (cmd != NAND_CMD_NONE) {
- __raw_writeb(cmd, this->IO_ADDR_W);
+ __raw_writeb(cmd, this->legacy.IO_ADDR_W);
wmb();
}
}
-static int au1550_nand_device_ready(struct mtd_info *mtd)
+static int au1550_nand_device_ready(struct nand_chip *this)
{
return alchemy_rdsmem(AU1000_MEM_STSTAT) & 1;
}
# MIPS headers
generic-(CONFIG_GENERIC_CSUM) += checksum.h
generic-y += current.h
+generic-y += device.h
generic-y += dma-contiguous.h
generic-y += emergency-restart.h
generic-y += export.h
+++ /dev/null
-/*
- * Arch specific extensions to struct device
- *
- * This file is released under the GPLv2
- */
-#ifndef _ASM_MIPS_DEVICE_H
-#define _ASM_MIPS_DEVICE_H
-
-struct dev_archdata {
-#ifdef CONFIG_DMA_PERDEV_COHERENT
- /* Non-zero if DMA is coherent with CPU caches */
- bool dma_coherent;
-#endif
-};
-
-struct pdev_archdata {
-};
-
-#endif /* _ASM_MIPS_DEVICE_H*/
#elif defined(CONFIG_DMA_MAYBE_COHERENT)
extern enum coherent_io_user_state coherentio;
extern int hw_coherentio;
+
+static inline bool dev_is_dma_coherent(struct device *dev)
+{
+ return coherentio == IO_COHERENCE_ENABLED ||
+ (coherentio == IO_COHERENCE_DEFAULT && hw_coherentio);
+}
#else
#ifdef CONFIG_DMA_NONCOHERENT
#define coherentio IO_COHERENCE_DISABLED
return &jazz_dma_ops;
#elif defined(CONFIG_SWIOTLB)
return &swiotlb_dma_ops;
-#elif defined(CONFIG_DMA_NONCOHERENT_OPS)
- return &dma_noncoherent_ops;
#else
return &dma_direct_ops;
#endif
bool coherent)
{
#ifdef CONFIG_DMA_PERDEV_COHERENT
- dev->archdata.dma_coherent = coherent;
+ dev->dma_coherent = coherent;
#endif
}
#include <linux/atomic.h>
#include <linux/cpumask.h>
+#include <linux/sizes.h>
#include <linux/threads.h>
#include <asm/cachectl.h>
#endif
-/*
- * One page above the stack is used for branch delay slot "emulation".
- * See dsemul.c for details.
- */
-#define STACK_TOP ((TASK_SIZE & PAGE_MASK) - PAGE_SIZE)
+#define VDSO_RANDOMIZE_SIZE (TASK_IS_32BIT_ADDR ? SZ_1M : SZ_256M)
+
+extern unsigned long mips_stack_top(void);
+#define STACK_TOP mips_stack_top()
/*
* This decides where the kernel will search for a free chunk of vm
extern void vr41xx_set_irq_level(unsigned int pin, irq_level_t level);
-typedef enum {
- GPIO_PULL_DOWN,
- GPIO_PULL_UP,
- GPIO_PULL_DISABLE,
-} gpio_pull_t;
-
-extern int vr41xx_gpio_pullupdown(unsigned int pin, gpio_pull_t pull);
-
#endif /* __NEC_VR41XX_GIU_H */
{
void *ret;
- ret = dma_direct_alloc(dev, size, dma_handle, gfp, attrs);
+ ret = dma_direct_alloc_pages(dev, size, dma_handle, gfp, attrs);
if (!ret)
return NULL;
*dma_handle = vdma_alloc(virt_to_phys(ret), size);
if (*dma_handle == VDMA_ERROR) {
- dma_direct_free(dev, size, ret, *dma_handle, attrs);
+ dma_direct_free_pages(dev, size, ret, *dma_handle, attrs);
return NULL;
}
vdma_free(dma_handle);
if (!(attrs & DMA_ATTR_NON_CONSISTENT))
vaddr = (void *)CAC_ADDR((unsigned long)vaddr);
- return dma_direct_free(dev, size, vaddr, dma_handle, attrs);
+ dma_direct_free_pages(dev, size, vaddr, dma_handle, attrs);
}
static dma_addr_t jazz_dma_map_page(struct device *dev, struct page *page,
const struct dma_map_ops jazz_dma_ops = {
.alloc = jazz_dma_alloc,
.free = jazz_dma_free,
- .mmap = arch_dma_mmap,
.map_page = jazz_dma_map_page,
.unmap_page = jazz_dma_unmap_page,
.map_sg = jazz_dma_map_sg,
#include <linux/nmi.h>
#include <linux/cpu.h>
+#include <asm/abi.h>
#include <asm/asm.h>
#include <asm/bootinfo.h>
#include <asm/cpu.h>
#include <asm/dsp.h>
#include <asm/fpu.h>
#include <asm/irq.h>
+#include <asm/mips-cps.h>
#include <asm/msa.h>
#include <asm/pgtable.h>
#include <asm/mipsregs.h>
return pc;
}
+unsigned long mips_stack_top(void)
+{
+ unsigned long top = TASK_SIZE & PAGE_MASK;
+
+ /* One page for branch delay slot "emulation" */
+ top -= PAGE_SIZE;
+
+ /* Space for the VDSO, data page & GIC user page */
+ top -= PAGE_ALIGN(current->thread.abi->vdso->size);
+ top -= PAGE_SIZE;
+ top -= mips_gic_present() ? PAGE_SIZE : 0;
+
+ /* Space for cache colour alignment */
+ if (cpu_has_dc_aliases)
+ top -= shm_align_mask + 1;
+
+ /* Space to randomize the VDSO base */
+ if (current->flags & PF_RANDOMIZE)
+ top -= VDSO_RANDOMIZE_SIZE;
+
+ return top;
+}
+
/*
* Don't forget that the stack pointer must be aligned on a 8 bytes
* boundary for 32-bits ABI and 16 bytes for 64-bits ABI.
struct memblock_region *reg;
extern void plat_mem_setup(void);
+ /*
+ * Initialize boot_command_line to an innocuous but non-empty string in
+ * order to prevent early_init_dt_scan_chosen() from copying
+ * CONFIG_CMDLINE into it without our knowledge. We handle
+ * CONFIG_CMDLINE ourselves below & don't want to duplicate its
+ * content because repeating arguments can be problematic.
+ */
+ strlcpy(boot_command_line, " ", COMMAND_LINE_SIZE);
+
+ /* call board setup routine */
+ plat_mem_setup();
+
+ /*
+ * Make sure all kernel memory is in the maps. The "UP" and
+ * "DOWN" are opposite for initdata since if it crosses over
+ * into another memory section you don't want that to be
+ * freed when the initdata is freed.
+ */
+ arch_mem_addpart(PFN_DOWN(__pa_symbol(&_text)) << PAGE_SHIFT,
+ PFN_UP(__pa_symbol(&_edata)) << PAGE_SHIFT,
+ BOOT_MEM_RAM);
+ arch_mem_addpart(PFN_UP(__pa_symbol(&__init_begin)) << PAGE_SHIFT,
+ PFN_DOWN(__pa_symbol(&__init_end)) << PAGE_SHIFT,
+ BOOT_MEM_INIT_RAM);
+
+ pr_info("Determined physical RAM map:\n");
+ print_memory_map();
+
#if defined(CONFIG_CMDLINE_BOOL) && defined(CONFIG_CMDLINE_OVERRIDE)
strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
#else
}
#endif
#endif
-
- /* call board setup routine */
- plat_mem_setup();
-
- /*
- * Make sure all kernel memory is in the maps. The "UP" and
- * "DOWN" are opposite for initdata since if it crosses over
- * into another memory section you don't want that to be
- * freed when the initdata is freed.
- */
- arch_mem_addpart(PFN_DOWN(__pa_symbol(&_text)) << PAGE_SHIFT,
- PFN_UP(__pa_symbol(&_edata)) << PAGE_SHIFT,
- BOOT_MEM_RAM);
- arch_mem_addpart(PFN_UP(__pa_symbol(&__init_begin)) << PAGE_SHIFT,
- PFN_DOWN(__pa_symbol(&__init_end)) << PAGE_SHIFT,
- BOOT_MEM_INIT_RAM);
-
- pr_info("Determined physical RAM map:\n");
- print_memory_map();
-
strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
*cmdline_p = command_line;
arch_initcall(debugfs_mips);
#endif
-#if defined(CONFIG_DMA_MAYBE_COHERENT) && !defined(CONFIG_DMA_PERDEV_COHERENT)
+#ifdef CONFIG_DMA_MAYBE_COHERENT
/* User defined DMA coherency from command line. */
enum coherent_io_user_state coherentio = IO_COHERENCE_DEFAULT;
EXPORT_SYMBOL_GPL(coherentio);
#include <linux/ioport.h>
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/random.h>
#include <linux/sched.h>
#include <linux/slab.h>
#include <linux/timekeeper_internal.h>
}
}
+static unsigned long vdso_base(void)
+{
+ unsigned long base;
+
+ /* Skip the delay slot emulation page */
+ base = STACK_TOP + PAGE_SIZE;
+
+ if (current->flags & PF_RANDOMIZE) {
+ base += get_random_int() & (VDSO_RANDOMIZE_SIZE - 1);
+ base = PAGE_ALIGN(base);
+ }
+
+ return base;
+}
+
int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
{
struct mips_vdso_image *image = current->thread.abi->vdso;
if (cpu_has_dc_aliases)
size += shm_align_mask + 1;
- base = get_unmapped_area(NULL, 0, size, 0, 0);
+ base = get_unmapped_area(NULL, vdso_base(), size, 0, 0);
if (IS_ERR_VALUE(base)) {
ret = base;
goto out;
* unset_bytes = end_addr - current_addr + 1
* a2 = t1 - a0 + 1
*/
+ .set reorder
PTR_SUBU a2, t1, a0
+ PTR_ADDIU a2, 1
jr ra
- PTR_ADDIU a2, 1
+ .set noreorder
.endm
__flush_icache_user_range = r4k_flush_icache_user_range;
__local_flush_icache_user_range = local_r4k_flush_icache_user_range;
-#if defined(CONFIG_DMA_NONCOHERENT) || defined(CONFIG_DMA_MAYBE_COHERENT)
-# if defined(CONFIG_DMA_PERDEV_COHERENT)
- if (0) {
-# else
- if ((coherentio == IO_COHERENCE_ENABLED) ||
- ((coherentio == IO_COHERENCE_DEFAULT) && hw_coherentio)) {
-# endif
+#ifdef CONFIG_DMA_NONCOHERENT
+#ifdef CONFIG_DMA_MAYBE_COHERENT
+ if (coherentio == IO_COHERENCE_ENABLED ||
+ (coherentio == IO_COHERENCE_DEFAULT && hw_coherentio)) {
_dma_cache_wback_inv = (void *)cache_noop;
_dma_cache_wback = (void *)cache_noop;
_dma_cache_inv = (void *)cache_noop;
- } else {
+ } else
+#endif /* CONFIG_DMA_MAYBE_COHERENT */
+ {
_dma_cache_wback_inv = r4k_dma_cache_wback_inv;
_dma_cache_wback = r4k_dma_cache_wback_inv;
_dma_cache_inv = r4k_dma_cache_inv;
}
-#endif
+#endif /* CONFIG_DMA_NONCOHERENT */
build_clear_page();
build_copy_page();
#include <asm/dma-coherence.h>
#include <asm/io.h>
-#ifdef CONFIG_DMA_PERDEV_COHERENT
-static inline int dev_is_coherent(struct device *dev)
-{
- return dev->archdata.dma_coherent;
-}
-#else
-static inline int dev_is_coherent(struct device *dev)
-{
- switch (coherentio) {
- default:
- case IO_COHERENCE_DEFAULT:
- return hw_coherentio;
- case IO_COHERENCE_ENABLED:
- return 1;
- case IO_COHERENCE_DISABLED:
- return 0;
- }
-}
-#endif /* CONFIG_DMA_PERDEV_COHERENT */
-
/*
* The affected CPUs below in 'cpu_needs_post_dma_flush()' can speculatively
* fill random cachelines with stale data at any time, requiring an extra
*/
static inline bool cpu_needs_post_dma_flush(struct device *dev)
{
- if (dev_is_coherent(dev))
- return false;
-
switch (boot_cpu_type()) {
case CPU_R10000:
case CPU_R12000:
{
void *ret;
- ret = dma_direct_alloc(dev, size, dma_handle, gfp, attrs);
- if (!ret)
- return NULL;
-
- if (!dev_is_coherent(dev) && !(attrs & DMA_ATTR_NON_CONSISTENT)) {
+ ret = dma_direct_alloc_pages(dev, size, dma_handle, gfp, attrs);
+ if (!ret && !(attrs & DMA_ATTR_NON_CONSISTENT)) {
dma_cache_wback_inv((unsigned long) ret, size);
ret = (void *)UNCAC_ADDR(ret);
}
void arch_dma_free(struct device *dev, size_t size, void *cpu_addr,
dma_addr_t dma_addr, unsigned long attrs)
{
- if (!(attrs & DMA_ATTR_NON_CONSISTENT) && !dev_is_coherent(dev))
+ if (!(attrs & DMA_ATTR_NON_CONSISTENT))
cpu_addr = (void *)CAC_ADDR((unsigned long)cpu_addr);
- dma_direct_free(dev, size, cpu_addr, dma_addr, attrs);
+ dma_direct_free_pages(dev, size, cpu_addr, dma_addr, attrs);
}
-int arch_dma_mmap(struct device *dev, struct vm_area_struct *vma,
- void *cpu_addr, dma_addr_t dma_addr, size_t size,
- unsigned long attrs)
+long arch_dma_coherent_to_pfn(struct device *dev, void *cpu_addr,
+ dma_addr_t dma_addr)
{
- unsigned long user_count = vma_pages(vma);
- unsigned long count = PAGE_ALIGN(size) >> PAGE_SHIFT;
- unsigned long addr = (unsigned long)cpu_addr;
- unsigned long off = vma->vm_pgoff;
- unsigned long pfn;
- int ret = -ENXIO;
-
- if (!dev_is_coherent(dev))
- addr = CAC_ADDR(addr);
-
- pfn = page_to_pfn(virt_to_page((void *)addr));
+ unsigned long addr = CAC_ADDR((unsigned long)cpu_addr);
+ return page_to_pfn(virt_to_page((void *)addr));
+}
+pgprot_t arch_dma_mmap_pgprot(struct device *dev, pgprot_t prot,
+ unsigned long attrs)
+{
if (attrs & DMA_ATTR_WRITE_COMBINE)
- vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
- else
- vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
-
- if (dma_mmap_from_dev_coherent(dev, vma, cpu_addr, size, &ret))
- return ret;
-
- if (off < count && user_count <= (count - off)) {
- ret = remap_pfn_range(vma, vma->vm_start,
- pfn + off,
- user_count << PAGE_SHIFT,
- vma->vm_page_prot);
- }
-
- return ret;
+ return pgprot_writecombine(prot);
+ return pgprot_noncached(prot);
}
static inline void dma_sync_virt(void *addr, size_t size,
void arch_sync_dma_for_device(struct device *dev, phys_addr_t paddr,
size_t size, enum dma_data_direction dir)
{
- if (!dev_is_coherent(dev))
- dma_sync_phys(paddr, size, dir);
+ dma_sync_phys(paddr, size, dir);
}
void arch_sync_dma_for_cpu(struct device *dev, phys_addr_t paddr,
{
BUG_ON(direction == DMA_NONE);
- if (!dev_is_coherent(dev))
- dma_sync_virt(vaddr, size, direction);
+ dma_sync_virt(vaddr, size, direction);
}
#include <linux/mtd/mtd.h>
#include <linux/mtd/physmap.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <asm/netlogic/haldefs.h>
#include <asm/netlogic/xlr/iomap.h>
static struct xlr_nand_flash_priv nand_priv;
-static void xlr_nand_ctrl(struct mtd_info *mtd, int cmd,
- unsigned int ctrl)
+static void xlr_nand_ctrl(struct nand_chip *chip, int cmd,
+ unsigned int ctrl)
{
if (ctrl & NAND_CLE)
nlm_write_reg(nand_priv.flash_mmio,
#include <linux/resource.h>
#include <linux/serial.h>
#include <linux/serial_pnx8xxx.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
#include <irq.h>
#include <irq-mapping.h>
};
static void
-pnx833x_flash_nand_cmd_ctrl(struct mtd_info *mtd, int cmd, unsigned int ctrl)
+pnx833x_flash_nand_cmd_ctrl(struct nand_chip *this, int cmd, unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- unsigned long nandaddr = (unsigned long)this->IO_ADDR_W;
+ unsigned long nandaddr = (unsigned long)this->legacy.IO_ADDR_W;
if (cmd == NAND_CMD_NONE)
return;
#include <linux/ctype.h>
#include <linux/string.h>
#include <linux/platform_device.h>
-#include <linux/mtd/rawnand.h>
+#include <linux/mtd/platnand.h>
#include <linux/mtd/mtd.h>
-#include <linux/mtd/partitions.h>
#include <linux/gpio.h>
#include <linux/gpio_keys.h>
#include <linux/input.h>
};
/* Resources and device for NAND */
-static int rb532_dev_ready(struct mtd_info *mtd)
+static int rb532_dev_ready(struct nand_chip *chip)
{
return gpio_get_value(GPIO_RDY);
}
-static void rb532_cmd_ctrl(struct mtd_info *mtd, int cmd, unsigned int ctrl)
+static void rb532_cmd_ctrl(struct nand_chip *chip, int cmd, unsigned int ctrl)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
unsigned char orbits, nandbits;
if (ctrl & NAND_CTRL_CHANGE) {
set_latch_u5(orbits, nandbits);
}
if (cmd != NAND_CMD_NONE)
- writeb(cmd, chip->IO_ADDR_W);
+ writeb(cmd, chip->legacy.IO_ADDR_W);
}
static struct resource nand_slot0_res[] = {
select CLKSRC_MMIO
select CLONE_BACKWARDS
select COMMON_CLK
- select DMA_NONCOHERENT_OPS
+ select DMA_DIRECT_OPS
select GENERIC_ATOMIC64
select GENERIC_CPU_DEVICES
select GENERIC_CLOCKEVENTS
select ARCH_HAS_SYNC_DMA_FOR_CPU
select ARCH_HAS_SYNC_DMA_FOR_DEVICE
select ARCH_NO_SWAP
- select DMA_NONCOHERENT_OPS
+ select DMA_DIRECT_OPS
select TIMER_OF
select GENERIC_ATOMIC64
select GENERIC_CLOCKEVENTS
config OPENRISC
def_bool y
select ARCH_HAS_SYNC_DMA_FOR_DEVICE
- select DMA_NONCOHERENT_OPS
+ select DMA_DIRECT_OPS
select OF
select OF_EARLY_FLATTREE
select IRQ_DOMAIN
depends on PA7000 || PA7100LC || PA7200 || PA7300LC
select ARCH_HAS_SYNC_DMA_FOR_CPU
select ARCH_HAS_SYNC_DMA_FOR_DEVICE
- select DMA_NONCOHERENT_OPS
+ select DMA_DIRECT_OPS
select DMA_NONCOHERENT_CACHE_SYNC
config PREFETCH
case pcxl: /* falls through */
case pcxs:
case pcxt:
- hppa_dma_ops = &dma_noncoherent_ops;
+ hppa_dma_ops = &dma_direct_ops;
break;
default:
break;
r.gr[30] = get_parisc_stackpointer();
regs = &r;
}
- unwind_frame_init(info, task, &r);
+ unwind_frame_init(info, task, regs);
} else {
unwind_frame_init_from_blocked_task(info, task);
}
*/
#define _HPAGE_CHG_MASK (PTE_RPN_MASK | _PAGE_HPTEFLAGS | _PAGE_DIRTY | \
_PAGE_ACCESSED | H_PAGE_THP_HUGE | _PAGE_PTE | \
- _PAGE_SOFT_DIRTY)
+ _PAGE_SOFT_DIRTY | _PAGE_DEVMAP)
/*
* user access blocked by key
*/
*/
#define _PAGE_CHG_MASK (PTE_RPN_MASK | _PAGE_HPTEFLAGS | _PAGE_DIRTY | \
_PAGE_ACCESSED | _PAGE_SPECIAL | _PAGE_PTE | \
- _PAGE_SOFT_DIRTY)
+ _PAGE_SOFT_DIRTY | _PAGE_DEVMAP)
#define H_PTE_PKEY (H_PTE_PKEY_BIT0 | H_PTE_PKEY_BIT1 | H_PTE_PKEY_BIT2 | \
H_PTE_PKEY_BIT3 | H_PTE_PKEY_BIT4)
config ARCH_SUPPORTS_UPROBES
def_bool y
+config KASAN_SHADOW_OFFSET
+ hex
+ depends on KASAN
+ default 0x18000000000000 if KASAN_S390_4_LEVEL_PAGING
+ default 0x30000000000
+
config S390
def_bool y
select ARCH_BINFMT_ELF_STATE
select HAVE_ALIGNED_STRUCT_PAGE if SLUB
select HAVE_ARCH_AUDITSYSCALL
select HAVE_ARCH_JUMP_LABEL
+ select HAVE_ARCH_JUMP_LABEL_RELATIVE
+ select HAVE_ARCH_KASAN
select CPU_NO_EFFICIENT_FFS if !HAVE_MARCH_Z9_109_FEATURES
select HAVE_ARCH_SECCOMP_FILTER
select HAVE_ARCH_SOFT_DIRTY
select HAVE_ARCH_TRACEHOOK
select HAVE_ARCH_TRANSPARENT_HUGEPAGE
+ select HAVE_ARCH_VMAP_STACK
select HAVE_EBPF_JIT if PACK_STACK && HAVE_MARCH_Z196_FEATURES
select HAVE_CMPXCHG_DOUBLE
select HAVE_CMPXCHG_LOCAL
config CHECK_STACK
def_bool y
+ depends on !VMAP_STACK
prompt "Detect kernel stack overflow"
help
This option enables the compiler option -mstack-guard and
KBUILD_CFLAGS_DECOMPRESSOR += $(if $(CONFIG_DEBUG_INFO),-g)
KBUILD_CFLAGS_DECOMPRESSOR += $(if $(CONFIG_DEBUG_INFO_DWARF4), $(call cc-option, -gdwarf-4,))
UTS_MACHINE := s390x
-STACK_SIZE := 16384
+STACK_SIZE := $(if $(CONFIG_KASAN),32768,16384)
CHECKFLAGS += -D__s390__ -D__s390x__
export LD_BFD
mutex_unlock(&appldata_ops_mutex);
}
+static struct appldata_product_id appldata_id = {
+ .prod_nr = {0xD3, 0xC9, 0xD5, 0xE4,
+ 0xE7, 0xD2, 0xD9}, /* "LINUXKR" */
+ .prod_fn = 0xD5D3, /* "NL" */
+ .version_nr = 0xF2F6, /* "26" */
+ .release_nr = 0xF0F1, /* "01" */
+};
+
/*
* appldata_diag()
*
int appldata_diag(char record_nr, u16 function, unsigned long buffer,
u16 length, char *mod_lvl)
{
- struct appldata_product_id id = {
- .prod_nr = {0xD3, 0xC9, 0xD5, 0xE4,
- 0xE7, 0xD2, 0xD9}, /* "LINUXKR" */
- .prod_fn = 0xD5D3, /* "NL" */
- .version_nr = 0xF2F6, /* "26" */
- .release_nr = 0xF0F1, /* "01" */
- };
+ struct appldata_parameter_list *parm_list;
+ struct appldata_product_id *id;
+ int rc;
- id.record_nr = record_nr;
- id.mod_lvl = (mod_lvl[0]) << 8 | mod_lvl[1];
- return appldata_asm(&id, function, (void *) buffer, length);
+ parm_list = kmalloc(sizeof(*parm_list), GFP_KERNEL);
+ id = kmemdup(&appldata_id, sizeof(appldata_id), GFP_KERNEL);
+ rc = -ENOMEM;
+ if (parm_list && id) {
+ id->record_nr = record_nr;
+ id->mod_lvl = (mod_lvl[0]) << 8 | mod_lvl[1];
+ rc = appldata_asm(parm_list, id, function,
+ (void *) buffer, length);
+ }
+ kfree(id);
+ kfree(parm_list);
+ return rc;
}
/************************ timer, work, DIAG <END> ****************************/
image
bzImage
+section_cmp.*
KCOV_INSTRUMENT := n
GCOV_PROFILE := n
UBSAN_SANITIZE := n
+KASAN_SANITIZE := n
KBUILD_AFLAGS := $(KBUILD_AFLAGS_DECOMPRESSOR)
KBUILD_CFLAGS := $(KBUILD_CFLAGS_DECOMPRESSOR)
CFLAGS_sclp_early_core.o += -I$(srctree)/drivers/s390/char
-obj-y := head.o als.o ebcdic.o sclp_early_core.o mem.o
-targets := bzImage startup.a $(obj-y)
+obj-y := head.o als.o startup.o mem_detect.o ipl_parm.o string.o ebcdic.o
+obj-y += sclp_early_core.o mem.o ipl_vmparm.o cmdline.o ctype.o
+targets := bzImage startup.a section_cmp.boot.data $(obj-y)
subdir- := compressed
OBJECTS := $(addprefix $(obj)/,$(obj-y))
-$(obj)/bzImage: $(obj)/compressed/vmlinux FORCE
+quiet_cmd_section_cmp = SECTCMP $*
+define cmd_section_cmp
+ s1=`$(OBJDUMP) -t -j "$*" "$<" | sort | \
+ sed -n "/0000000000000000/! s/.*\s$*\s\+//p" | sha256sum`; \
+ s2=`$(OBJDUMP) -t -j "$*" "$(word 2,$^)" | sort | \
+ sed -n "/0000000000000000/! s/.*\s$*\s\+//p" | sha256sum`; \
+ if [ "$$s1" != "$$s2" ]; then \
+ echo "error: section $* differs between $< and $(word 2,$^)" >&2; \
+ exit 1; \
+ fi; \
+ touch $@
+endef
+
+$(obj)/bzImage: $(obj)/compressed/vmlinux $(obj)/section_cmp.boot.data FORCE
$(call if_changed,objcopy)
+$(obj)/section_cmp%: vmlinux $(obj)/compressed/vmlinux FORCE
+ $(call if_changed,section_cmp)
+
$(obj)/compressed/vmlinux: $(obj)/startup.a FORCE
$(Q)$(MAKE) $(build)=$(obj)/compressed $@
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef BOOT_BOOT_H
+#define BOOT_BOOT_H
+
+void startup_kernel(void);
+void detect_memory(void);
+void store_ipl_parmblock(void);
+void setup_boot_command_line(void);
+void setup_memory_end(void);
+
+#endif /* BOOT_BOOT_H */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include "../../../lib/cmdline.c"
KCOV_INSTRUMENT := n
GCOV_PROFILE := n
UBSAN_SANITIZE := n
+KASAN_SANITIZE := n
-obj-y := $(if $(CONFIG_KERNEL_UNCOMPRESSED),,head.o misc.o) piggy.o
+obj-y := $(if $(CONFIG_KERNEL_UNCOMPRESSED),,decompressor.o) piggy.o info.o
targets := vmlinux.lds vmlinux vmlinux.bin vmlinux.bin.gz vmlinux.bin.bz2
targets += vmlinux.bin.xz vmlinux.bin.lzma vmlinux.bin.lzo vmlinux.bin.lz4
-targets += vmlinux.scr.lds $(obj-y) $(if $(CONFIG_KERNEL_UNCOMPRESSED),,sizes.h)
+targets += info.bin $(obj-y)
KBUILD_AFLAGS := $(KBUILD_AFLAGS_DECOMPRESSOR)
KBUILD_CFLAGS := $(KBUILD_CFLAGS_DECOMPRESSOR)
+OBJCOPYFLAGS :=
OBJECTS := $(addprefix $(obj)/,$(obj-y))
$(obj)/vmlinux: $(obj)/vmlinux.lds $(objtree)/arch/s390/boot/startup.a $(OBJECTS)
$(call if_changed,ld)
-# extract required uncompressed vmlinux symbols and adjust them to reflect offsets inside vmlinux.bin
-sed-sizes := -e 's/^\([0-9a-fA-F]*\) . \(__bss_start\|_end\)$$/\#define SZ\2 (0x\1 - 0x100000)/p'
-
-quiet_cmd_sizes = GEN $@
- cmd_sizes = $(NM) $< | sed -n $(sed-sizes) > $@
-
-$(obj)/sizes.h: vmlinux
- $(call if_changed,sizes)
-
-AFLAGS_head.o += -I$(objtree)/$(obj)
-$(obj)/head.o: $(obj)/sizes.h
+OBJCOPYFLAGS_info.bin := -O binary --only-section=.vmlinux.info
+$(obj)/info.bin: vmlinux FORCE
+ $(call if_changed,objcopy)
-CFLAGS_misc.o += -I$(objtree)/$(obj)
-$(obj)/misc.o: $(obj)/sizes.h
+OBJCOPYFLAGS_info.o := -I binary -O elf64-s390 -B s390:64-bit --rename-section .data=.vmlinux.info
+$(obj)/info.o: $(obj)/info.bin FORCE
+ $(call if_changed,objcopy)
-OBJCOPYFLAGS_vmlinux.bin := -R .comment -S
-$(obj)/vmlinux.bin: vmlinux
+OBJCOPYFLAGS_vmlinux.bin := -O binary --remove-section=.comment --remove-section=.vmlinux.info -S
+$(obj)/vmlinux.bin: vmlinux FORCE
$(call if_changed,objcopy)
vmlinux.bin.all-y := $(obj)/vmlinux.bin
$(obj)/vmlinux.bin.xz: $(vmlinux.bin.all-y)
$(call if_changed,xzkern)
-LDFLAGS_piggy.o := -r --format binary --oformat $(LD_BFD) -T
-$(obj)/piggy.o: $(obj)/vmlinux.scr.lds $(obj)/vmlinux.bin$(suffix-y)
- $(call if_changed,ld)
+OBJCOPYFLAGS_piggy.o := -I binary -O elf64-s390 -B s390:64-bit --rename-section .data=.vmlinux.bin.compressed
+$(obj)/piggy.o: $(obj)/vmlinux.bin$(suffix-y) FORCE
+ $(call if_changed,objcopy)
-chkbss := $(filter-out $(obj)/misc.o $(obj)/piggy.o,$(OBJECTS))
+chkbss := $(filter-out $(obj)/piggy.o $(obj)/info.o,$(OBJECTS))
chkbss-target := $(obj)/vmlinux.bin
include $(srctree)/arch/s390/scripts/Makefile.chkbss
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Definitions and wrapper functions for kernel decompressor
+ *
+ * Copyright IBM Corp. 2010
+ *
+ * Author(s): Martin Schwidefsky <schwidefsky@de.ibm.com>
+ */
+
+#include <linux/kernel.h>
+#include <linux/string.h>
+#include <asm/page.h>
+#include "decompressor.h"
+
+/*
+ * gzip declarations
+ */
+#define STATIC static
+#define STATIC_RW_DATA static __section(.data)
+
+#undef memset
+#undef memcpy
+#undef memmove
+#define memmove memmove
+#define memzero(s, n) memset((s), 0, (n))
+
+/* Symbols defined by linker scripts */
+extern char _end[];
+extern unsigned char _compressed_start[];
+extern unsigned char _compressed_end[];
+
+#ifdef CONFIG_HAVE_KERNEL_BZIP2
+#define HEAP_SIZE 0x400000
+#else
+#define HEAP_SIZE 0x10000
+#endif
+
+static unsigned long free_mem_ptr = (unsigned long) _end;
+static unsigned long free_mem_end_ptr = (unsigned long) _end + HEAP_SIZE;
+
+#ifdef CONFIG_KERNEL_GZIP
+#include "../../../../lib/decompress_inflate.c"
+#endif
+
+#ifdef CONFIG_KERNEL_BZIP2
+#include "../../../../lib/decompress_bunzip2.c"
+#endif
+
+#ifdef CONFIG_KERNEL_LZ4
+#include "../../../../lib/decompress_unlz4.c"
+#endif
+
+#ifdef CONFIG_KERNEL_LZMA
+#include "../../../../lib/decompress_unlzma.c"
+#endif
+
+#ifdef CONFIG_KERNEL_LZO
+#include "../../../../lib/decompress_unlzo.c"
+#endif
+
+#ifdef CONFIG_KERNEL_XZ
+#include "../../../../lib/decompress_unxz.c"
+#endif
+
+#define decompress_offset ALIGN((unsigned long)_end + HEAP_SIZE, PAGE_SIZE)
+
+unsigned long mem_safe_offset(void)
+{
+ /*
+ * due to 4MB HEAD_SIZE for bzip2
+ * 'decompress_offset + vmlinux.image_size' could be larger than
+ * kernel at final position + its .bss, so take the larger of two
+ */
+ return max(decompress_offset + vmlinux.image_size,
+ vmlinux.default_lma + vmlinux.image_size + vmlinux.bss_size);
+}
+
+void *decompress_kernel(void)
+{
+ void *output = (void *)decompress_offset;
+
+ __decompress(_compressed_start, _compressed_end - _compressed_start,
+ NULL, NULL, output, 0, NULL, error);
+ return output;
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef BOOT_COMPRESSED_DECOMPRESSOR_H
+#define BOOT_COMPRESSED_DECOMPRESSOR_H
+
+#ifdef CONFIG_KERNEL_UNCOMPRESSED
+static inline void *decompress_kernel(void) {}
+#else
+void *decompress_kernel(void);
+#endif
+unsigned long mem_safe_offset(void);
+void error(char *m);
+
+struct vmlinux_info {
+ unsigned long default_lma;
+ void (*entry)(void);
+ unsigned long image_size; /* does not include .bss */
+ unsigned long bss_size; /* uncompressed image .bss size */
+ unsigned long bootdata_off;
+ unsigned long bootdata_size;
+};
+
+extern char _vmlinux_info[];
+#define vmlinux (*(struct vmlinux_info *)_vmlinux_info)
+
+#endif /* BOOT_COMPRESSED_DECOMPRESSOR_H */
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * Startup glue code to uncompress the kernel
- *
- * Copyright IBM Corp. 2010
- *
- * Author(s): Martin Schwidefsky <schwidefsky@de.ibm.com>
- */
-
-#include <linux/init.h>
-#include <linux/linkage.h>
-#include <asm/asm-offsets.h>
-#include <asm/thread_info.h>
-#include <asm/page.h>
-#include "sizes.h"
-
-__HEAD
-ENTRY(startup_decompressor)
- basr %r13,0 # get base
-.LPG1:
- # setup stack
- lg %r15,.Lstack-.LPG1(%r13)
- aghi %r15,-160
- brasl %r14,decompress_kernel
- # Set up registers for memory mover. We move the decompressed image to
- # 0x100000, where startup_continue of the decompressed image is supposed
- # to be.
- lgr %r4,%r2
- lg %r2,.Loffset-.LPG1(%r13)
- lg %r3,.Lmvsize-.LPG1(%r13)
- lgr %r5,%r3
- # Move the memory mover someplace safe so it doesn't overwrite itself.
- la %r1,0x200
- mvc 0(mover_end-mover,%r1),mover-.LPG1(%r13)
- # When the memory mover is done we pass control to
- # arch/s390/kernel/head64.S:startup_continue which lives at 0x100000 in
- # the decompressed image.
- lgr %r6,%r2
- br %r1
-mover:
- mvcle %r2,%r4,0
- jo mover
- br %r6
-mover_end:
-
- .align 8
-.Lstack:
- .quad 0x8000 + (1<<(PAGE_SHIFT+THREAD_SIZE_ORDER))
-.Loffset:
- .quad 0x100000
-.Lmvsize:
- .quad SZ__bss_start
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Definitions and wrapper functions for kernel decompressor
- *
- * Copyright IBM Corp. 2010
- *
- * Author(s): Martin Schwidefsky <schwidefsky@de.ibm.com>
- */
-
-#include <linux/uaccess.h>
-#include <asm/page.h>
-#include <asm/sclp.h>
-#include <asm/ipl.h>
-#include "sizes.h"
-
-/*
- * gzip declarations
- */
-#define STATIC static
-
-#undef memset
-#undef memcpy
-#undef memmove
-#define memmove memmove
-#define memzero(s, n) memset((s), 0, (n))
-
-/* Symbols defined by linker scripts */
-extern char input_data[];
-extern int input_len;
-extern char _end[];
-extern char _bss[], _ebss[];
-
-static void error(char *m);
-
-static unsigned long free_mem_ptr;
-static unsigned long free_mem_end_ptr;
-
-#ifdef CONFIG_HAVE_KERNEL_BZIP2
-#define HEAP_SIZE 0x400000
-#else
-#define HEAP_SIZE 0x10000
-#endif
-
-#ifdef CONFIG_KERNEL_GZIP
-#include "../../../../lib/decompress_inflate.c"
-#endif
-
-#ifdef CONFIG_KERNEL_BZIP2
-#include "../../../../lib/decompress_bunzip2.c"
-#endif
-
-#ifdef CONFIG_KERNEL_LZ4
-#include "../../../../lib/decompress_unlz4.c"
-#endif
-
-#ifdef CONFIG_KERNEL_LZMA
-#include "../../../../lib/decompress_unlzma.c"
-#endif
-
-#ifdef CONFIG_KERNEL_LZO
-#include "../../../../lib/decompress_unlzo.c"
-#endif
-
-#ifdef CONFIG_KERNEL_XZ
-#include "../../../../lib/decompress_unxz.c"
-#endif
-
-static int puts(const char *s)
-{
- sclp_early_printk(s);
- return 0;
-}
-
-static void error(char *x)
-{
- unsigned long long psw = 0x000a0000deadbeefULL;
-
- puts("\n\n");
- puts(x);
- puts("\n\n -- System halted");
-
- asm volatile("lpsw %0" : : "Q" (psw));
-}
-
-unsigned long decompress_kernel(void)
-{
- void *output, *kernel_end;
-
- output = (void *) ALIGN((unsigned long) _end + HEAP_SIZE, PAGE_SIZE);
- kernel_end = output + SZ__bss_start;
-
-#ifdef CONFIG_BLK_DEV_INITRD
- /*
- * Move the initrd right behind the end of the decompressed
- * kernel image. This also prevents initrd corruption caused by
- * bss clearing since kernel_end will always be located behind the
- * current bss section..
- */
- if (INITRD_START && INITRD_SIZE && kernel_end > (void *) INITRD_START) {
- memmove(kernel_end, (void *) INITRD_START, INITRD_SIZE);
- INITRD_START = (unsigned long) kernel_end;
- }
-#endif
-
- /*
- * Clear bss section. free_mem_ptr and free_mem_end_ptr need to be
- * initialized afterwards since they reside in bss.
- */
- memset(_bss, 0, _ebss - _bss);
- free_mem_ptr = (unsigned long) _end;
- free_mem_end_ptr = free_mem_ptr + HEAP_SIZE;
-
- __decompress(input_data, input_len, NULL, NULL, output, 0, NULL, error);
- return (unsigned long) output;
-}
-
/* SPDX-License-Identifier: GPL-2.0 */
#include <asm-generic/vmlinux.lds.h>
+#include <asm/vmlinux.lds.h>
OUTPUT_FORMAT("elf64-s390", "elf64-s390", "elf64-s390")
OUTPUT_ARCH(s390:64-bit)
SECTIONS
{
- /* Be careful parts of head_64.S assume startup_32 is at
- * address 0.
- */
. = 0;
.head.text : {
_head = . ;
.rodata : {
_rodata = . ;
*(.rodata) /* read-only data */
- *(EXCLUDE_FILE (*piggy.o) .rodata.compressed)
+ *(.rodata.*)
_erodata = . ;
}
.data : {
*(.data.*)
_edata = . ;
}
- startup_continue = 0x100000;
+ BOOT_DATA
+
+ /*
+ * uncompressed image info used by the decompressor it should match
+ * struct vmlinux_info. It comes from .vmlinux.info section of
+ * uncompressed vmlinux in a form of info.o
+ */
+ . = ALIGN(8);
+ .vmlinux.info : {
+ _vmlinux_info = .;
+ *(.vmlinux.info)
+ }
+
#ifdef CONFIG_KERNEL_UNCOMPRESSED
. = 0x100000;
#else
. = ALIGN(8);
#endif
.rodata.compressed : {
- *(.rodata.compressed)
+ _compressed_start = .;
+ *(.vmlinux.bin.compressed)
+ _compressed_end = .;
}
. = ALIGN(256);
.bss : {
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 */
-SECTIONS
-{
- .rodata.compressed : {
-#ifndef CONFIG_KERNEL_UNCOMPRESSED
- input_len = .;
- LONG(input_data_end - input_data) input_data = .;
-#endif
- *(.data)
-#ifndef CONFIG_KERNEL_UNCOMPRESSED
- output_len = . - 4;
- input_data_end = .;
-#endif
- }
-}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include "../../../lib/ctype.c"
.long 0x02000690,0x60000050
.long 0x020006e0,0x20000050
+ .org 0x1a0
+ .quad 0,iplstart
+
.org 0x200
#
spt 6f-.LPG0(%r13)
mvc __LC_LAST_UPDATE_TIMER(8),6f-.LPG0(%r13)
l %r15,.Lstack-.LPG0(%r13)
- ahi %r15,-STACK_FRAME_OVERHEAD
brasl %r14,verify_facilities
-#ifdef CONFIG_KERNEL_UNCOMPRESSED
- jg startup_continue
-#else
- jg startup_decompressor
-#endif
+ brasl %r14,startup_kernel
.Lstack:
- .long 0x8000 + (1<<(PAGE_SHIFT+THREAD_SIZE_ORDER))
+ .long 0x8000 + (1<<(PAGE_SHIFT+BOOT_STACK_ORDER)) - STACK_FRAME_OVERHEAD
.align 8
6: .long 0x7fffffff,0xffffffff
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/ctype.h>
+#include <asm/ebcdic.h>
+#include <asm/sclp.h>
+#include <asm/sections.h>
+#include <asm/boot_data.h>
+#include "boot.h"
+
+char __bootdata(early_command_line)[COMMAND_LINE_SIZE];
+struct ipl_parameter_block __bootdata(early_ipl_block);
+int __bootdata(early_ipl_block_valid);
+
+unsigned long __bootdata(memory_end);
+int __bootdata(memory_end_set);
+int __bootdata(noexec_disabled);
+
+static inline int __diag308(unsigned long subcode, void *addr)
+{
+ register unsigned long _addr asm("0") = (unsigned long)addr;
+ register unsigned long _rc asm("1") = 0;
+ unsigned long reg1, reg2;
+ psw_t old = S390_lowcore.program_new_psw;
+
+ asm volatile(
+ " epsw %0,%1\n"
+ " st %0,%[psw_pgm]\n"
+ " st %1,%[psw_pgm]+4\n"
+ " larl %0,1f\n"
+ " stg %0,%[psw_pgm]+8\n"
+ " diag %[addr],%[subcode],0x308\n"
+ "1: nopr %%r7\n"
+ : "=&d" (reg1), "=&a" (reg2),
+ [psw_pgm] "=Q" (S390_lowcore.program_new_psw),
+ [addr] "+d" (_addr), "+d" (_rc)
+ : [subcode] "d" (subcode)
+ : "cc", "memory");
+ S390_lowcore.program_new_psw = old;
+ return _rc;
+}
+
+void store_ipl_parmblock(void)
+{
+ int rc;
+
+ rc = __diag308(DIAG308_STORE, &early_ipl_block);
+ if (rc == DIAG308_RC_OK &&
+ early_ipl_block.hdr.version <= IPL_MAX_SUPPORTED_VERSION)
+ early_ipl_block_valid = 1;
+}
+
+static size_t scpdata_length(const char *buf, size_t count)
+{
+ while (count) {
+ if (buf[count - 1] != '\0' && buf[count - 1] != ' ')
+ break;
+ count--;
+ }
+ return count;
+}
+
+static size_t ipl_block_get_ascii_scpdata(char *dest, size_t size,
+ const struct ipl_parameter_block *ipb)
+{
+ size_t count;
+ size_t i;
+ int has_lowercase;
+
+ count = min(size - 1, scpdata_length(ipb->ipl_info.fcp.scp_data,
+ ipb->ipl_info.fcp.scp_data_len));
+ if (!count)
+ goto out;
+
+ has_lowercase = 0;
+ for (i = 0; i < count; i++) {
+ if (!isascii(ipb->ipl_info.fcp.scp_data[i])) {
+ count = 0;
+ goto out;
+ }
+ if (!has_lowercase && islower(ipb->ipl_info.fcp.scp_data[i]))
+ has_lowercase = 1;
+ }
+
+ if (has_lowercase)
+ memcpy(dest, ipb->ipl_info.fcp.scp_data, count);
+ else
+ for (i = 0; i < count; i++)
+ dest[i] = tolower(ipb->ipl_info.fcp.scp_data[i]);
+out:
+ dest[count] = '\0';
+ return count;
+}
+
+static void append_ipl_block_parm(void)
+{
+ char *parm, *delim;
+ size_t len, rc = 0;
+
+ len = strlen(early_command_line);
+
+ delim = early_command_line + len; /* '\0' character position */
+ parm = early_command_line + len + 1; /* append right after '\0' */
+
+ switch (early_ipl_block.hdr.pbt) {
+ case DIAG308_IPL_TYPE_CCW:
+ rc = ipl_block_get_ascii_vmparm(
+ parm, COMMAND_LINE_SIZE - len - 1, &early_ipl_block);
+ break;
+ case DIAG308_IPL_TYPE_FCP:
+ rc = ipl_block_get_ascii_scpdata(
+ parm, COMMAND_LINE_SIZE - len - 1, &early_ipl_block);
+ break;
+ }
+ if (rc) {
+ if (*parm == '=')
+ memmove(early_command_line, parm + 1, rc);
+ else
+ *delim = ' '; /* replace '\0' with space */
+ }
+}
+
+static inline int has_ebcdic_char(const char *str)
+{
+ int i;
+
+ for (i = 0; str[i]; i++)
+ if (str[i] & 0x80)
+ return 1;
+ return 0;
+}
+
+void setup_boot_command_line(void)
+{
+ COMMAND_LINE[ARCH_COMMAND_LINE_SIZE - 1] = 0;
+ /* convert arch command line to ascii if necessary */
+ if (has_ebcdic_char(COMMAND_LINE))
+ EBCASC(COMMAND_LINE, ARCH_COMMAND_LINE_SIZE);
+ /* copy arch command line */
+ strcpy(early_command_line, strim(COMMAND_LINE));
+
+ /* append IPL PARM data to the boot command line */
+ if (early_ipl_block_valid)
+ append_ipl_block_parm();
+}
+
+static char command_line_buf[COMMAND_LINE_SIZE] __section(.data);
+static void parse_mem_opt(void)
+{
+ char *param, *val;
+ bool enabled;
+ char *args;
+ int rc;
+
+ args = strcpy(command_line_buf, early_command_line);
+ while (*args) {
+ args = next_arg(args, ¶m, &val);
+
+ if (!strcmp(param, "mem")) {
+ memory_end = memparse(val, NULL);
+ memory_end_set = 1;
+ }
+
+ if (!strcmp(param, "noexec")) {
+ rc = kstrtobool(val, &enabled);
+ if (!rc && !enabled)
+ noexec_disabled = 1;
+ }
+ }
+}
+
+void setup_memory_end(void)
+{
+ parse_mem_opt();
+#ifdef CONFIG_CRASH_DUMP
+ if (!OLDMEM_BASE && early_ipl_block_valid &&
+ early_ipl_block.hdr.pbt == DIAG308_IPL_TYPE_FCP &&
+ early_ipl_block.ipl_info.fcp.opt == DIAG308_IPL_OPT_DUMP) {
+ if (!sclp_early_get_hsa_size(&memory_end) && memory_end)
+ memory_end_set = 1;
+ }
+#endif
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include "../kernel/ipl_vmparm.c"
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/errno.h>
+#include <linux/init.h>
+#include <asm/sclp.h>
+#include <asm/sections.h>
+#include <asm/mem_detect.h>
+#include <asm/sparsemem.h>
+#include "compressed/decompressor.h"
+#include "boot.h"
+
+unsigned long __bootdata(max_physmem_end);
+struct mem_detect_info __bootdata(mem_detect);
+
+/* up to 256 storage elements, 1020 subincrements each */
+#define ENTRIES_EXTENDED_MAX \
+ (256 * (1020 / 2) * sizeof(struct mem_detect_block))
+
+/*
+ * To avoid corrupting old kernel memory during dump, find lowest memory
+ * chunk possible either right after the kernel end (decompressed kernel) or
+ * after initrd (if it is present and there is no hole between the kernel end
+ * and initrd)
+ */
+static void *mem_detect_alloc_extended(void)
+{
+ unsigned long offset = ALIGN(mem_safe_offset(), sizeof(u64));
+
+ if (IS_ENABLED(BLK_DEV_INITRD) && INITRD_START && INITRD_SIZE &&
+ INITRD_START < offset + ENTRIES_EXTENDED_MAX)
+ offset = ALIGN(INITRD_START + INITRD_SIZE, sizeof(u64));
+
+ return (void *)offset;
+}
+
+static struct mem_detect_block *__get_mem_detect_block_ptr(u32 n)
+{
+ if (n < MEM_INLINED_ENTRIES)
+ return &mem_detect.entries[n];
+ if (unlikely(!mem_detect.entries_extended))
+ mem_detect.entries_extended = mem_detect_alloc_extended();
+ return &mem_detect.entries_extended[n - MEM_INLINED_ENTRIES];
+}
+
+/*
+ * sequential calls to add_mem_detect_block with adjacent memory areas
+ * are merged together into single memory block.
+ */
+void add_mem_detect_block(u64 start, u64 end)
+{
+ struct mem_detect_block *block;
+
+ if (mem_detect.count) {
+ block = __get_mem_detect_block_ptr(mem_detect.count - 1);
+ if (block->end == start) {
+ block->end = end;
+ return;
+ }
+ }
+
+ block = __get_mem_detect_block_ptr(mem_detect.count);
+ block->start = start;
+ block->end = end;
+ mem_detect.count++;
+}
+
+static unsigned long get_mem_detect_end(void)
+{
+ if (mem_detect.count)
+ return __get_mem_detect_block_ptr(mem_detect.count - 1)->end;
+ return 0;
+}
+
+static int __diag260(unsigned long rx1, unsigned long rx2)
+{
+ register unsigned long _rx1 asm("2") = rx1;
+ register unsigned long _rx2 asm("3") = rx2;
+ register unsigned long _ry asm("4") = 0x10; /* storage configuration */
+ int rc = -1; /* fail */
+ unsigned long reg1, reg2;
+ psw_t old = S390_lowcore.program_new_psw;
+
+ asm volatile(
+ " epsw %0,%1\n"
+ " st %0,%[psw_pgm]\n"
+ " st %1,%[psw_pgm]+4\n"
+ " larl %0,1f\n"
+ " stg %0,%[psw_pgm]+8\n"
+ " diag %[rx],%[ry],0x260\n"
+ " ipm %[rc]\n"
+ " srl %[rc],28\n"
+ "1:\n"
+ : "=&d" (reg1), "=&a" (reg2),
+ [psw_pgm] "=Q" (S390_lowcore.program_new_psw),
+ [rc] "+&d" (rc), [ry] "+d" (_ry)
+ : [rx] "d" (_rx1), "d" (_rx2)
+ : "cc", "memory");
+ S390_lowcore.program_new_psw = old;
+ return rc == 0 ? _ry : -1;
+}
+
+static int diag260(void)
+{
+ int rc, i;
+
+ struct {
+ unsigned long start;
+ unsigned long end;
+ } storage_extents[8] __aligned(16); /* VM supports up to 8 extends */
+
+ memset(storage_extents, 0, sizeof(storage_extents));
+ rc = __diag260((unsigned long)storage_extents, sizeof(storage_extents));
+ if (rc == -1)
+ return -1;
+
+ for (i = 0; i < min_t(int, rc, ARRAY_SIZE(storage_extents)); i++)
+ add_mem_detect_block(storage_extents[i].start, storage_extents[i].end + 1);
+ return 0;
+}
+
+static int tprot(unsigned long addr)
+{
+ unsigned long pgm_addr;
+ int rc = -EFAULT;
+ psw_t old = S390_lowcore.program_new_psw;
+
+ S390_lowcore.program_new_psw.mask = __extract_psw();
+ asm volatile(
+ " larl %[pgm_addr],1f\n"
+ " stg %[pgm_addr],%[psw_pgm_addr]\n"
+ " tprot 0(%[addr]),0\n"
+ " ipm %[rc]\n"
+ " srl %[rc],28\n"
+ "1:\n"
+ : [pgm_addr] "=&d"(pgm_addr),
+ [psw_pgm_addr] "=Q"(S390_lowcore.program_new_psw.addr),
+ [rc] "+&d"(rc)
+ : [addr] "a"(addr)
+ : "cc", "memory");
+ S390_lowcore.program_new_psw = old;
+ return rc;
+}
+
+static void search_mem_end(void)
+{
+ unsigned long range = 1 << (MAX_PHYSMEM_BITS - 20); /* in 1MB blocks */
+ unsigned long offset = 0;
+ unsigned long pivot;
+
+ while (range > 1) {
+ range >>= 1;
+ pivot = offset + range;
+ if (!tprot(pivot << 20))
+ offset = pivot;
+ }
+
+ add_mem_detect_block(0, (offset + 1) << 20);
+}
+
+void detect_memory(void)
+{
+ sclp_early_get_memsize(&max_physmem_end);
+
+ if (!sclp_early_read_storage_info()) {
+ mem_detect.info_source = MEM_DETECT_SCLP_STOR_INFO;
+ return;
+ }
+
+ if (!diag260()) {
+ mem_detect.info_source = MEM_DETECT_DIAG260;
+ return;
+ }
+
+ if (max_physmem_end) {
+ add_mem_detect_block(0, max_physmem_end);
+ mem_detect.info_source = MEM_DETECT_SCLP_READ_INFO;
+ return;
+ }
+
+ search_mem_end();
+ mem_detect.info_source = MEM_DETECT_BIN_SEARCH;
+ max_physmem_end = get_mem_detect_end();
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/string.h>
+#include <asm/setup.h>
+#include <asm/sclp.h>
+#include "compressed/decompressor.h"
+#include "boot.h"
+
+extern char __boot_data_start[], __boot_data_end[];
+
+void error(char *x)
+{
+ sclp_early_printk("\n\n");
+ sclp_early_printk(x);
+ sclp_early_printk("\n\n -- System halted");
+
+ disabled_wait(0xdeadbeef);
+}
+
+#ifdef CONFIG_KERNEL_UNCOMPRESSED
+unsigned long mem_safe_offset(void)
+{
+ return vmlinux.default_lma + vmlinux.image_size + vmlinux.bss_size;
+}
+#endif
+
+static void rescue_initrd(void)
+{
+ unsigned long min_initrd_addr;
+
+ if (!IS_ENABLED(CONFIG_BLK_DEV_INITRD))
+ return;
+ if (!INITRD_START || !INITRD_SIZE)
+ return;
+ min_initrd_addr = mem_safe_offset();
+ if (min_initrd_addr <= INITRD_START)
+ return;
+ memmove((void *)min_initrd_addr, (void *)INITRD_START, INITRD_SIZE);
+ INITRD_START = min_initrd_addr;
+}
+
+static void copy_bootdata(void)
+{
+ if (__boot_data_end - __boot_data_start != vmlinux.bootdata_size)
+ error(".boot.data section size mismatch");
+ memcpy((void *)vmlinux.bootdata_off, __boot_data_start, vmlinux.bootdata_size);
+}
+
+void startup_kernel(void)
+{
+ void *img;
+
+ rescue_initrd();
+ sclp_early_read_info();
+ store_ipl_parmblock();
+ setup_boot_command_line();
+ setup_memory_end();
+ detect_memory();
+ if (!IS_ENABLED(CONFIG_KERNEL_UNCOMPRESSED)) {
+ img = decompress_kernel();
+ memmove((void *)vmlinux.default_lma, img, vmlinux.image_size);
+ }
+ copy_bootdata();
+ vmlinux.entry();
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ctype.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include "../lib/string.c"
+
+int strncmp(const char *cs, const char *ct, size_t count)
+{
+ unsigned char c1, c2;
+
+ while (count) {
+ c1 = *cs++;
+ c2 = *ct++;
+ if (c1 != c2)
+ return c1 < c2 ? -1 : 1;
+ if (!c1)
+ break;
+ count--;
+ }
+ return 0;
+}
+
+char *skip_spaces(const char *str)
+{
+ while (isspace(*str))
+ ++str;
+ return (char *)str;
+}
+
+char *strim(char *s)
+{
+ size_t size;
+ char *end;
+
+ size = strlen(s);
+ if (!size)
+ return s;
+
+ end = s + size - 1;
+ while (end >= s && isspace(*end))
+ end--;
+ *(end + 1) = '\0';
+
+ return skip_spaces(s);
+}
+
+/* Works only for digits and letters, but small and fast */
+#define TOLOWER(x) ((x) | 0x20)
+
+static unsigned int simple_guess_base(const char *cp)
+{
+ if (cp[0] == '0') {
+ if (TOLOWER(cp[1]) == 'x' && isxdigit(cp[2]))
+ return 16;
+ else
+ return 8;
+ } else {
+ return 10;
+ }
+}
+
+/**
+ * simple_strtoull - convert a string to an unsigned long long
+ * @cp: The start of the string
+ * @endp: A pointer to the end of the parsed string will be placed here
+ * @base: The number base to use
+ */
+
+unsigned long long simple_strtoull(const char *cp, char **endp,
+ unsigned int base)
+{
+ unsigned long long result = 0;
+
+ if (!base)
+ base = simple_guess_base(cp);
+
+ if (base == 16 && cp[0] == '0' && TOLOWER(cp[1]) == 'x')
+ cp += 2;
+
+ while (isxdigit(*cp)) {
+ unsigned int value;
+
+ value = isdigit(*cp) ? *cp - '0' : TOLOWER(*cp) - 'a' + 10;
+ if (value >= base)
+ break;
+ result = result * base + value;
+ cp++;
+ }
+ if (endp)
+ *endp = (char *)cp;
+
+ return result;
+}
+
+long simple_strtol(const char *cp, char **endp, unsigned int base)
+{
+ if (*cp == '-')
+ return -simple_strtoull(cp + 1, endp, base);
+
+ return simple_strtoull(cp, endp, base);
+}
+
+int kstrtobool(const char *s, bool *res)
+{
+ if (!s)
+ return -EINVAL;
+
+ switch (s[0]) {
+ case 'y':
+ case 'Y':
+ case '1':
+ *res = true;
+ return 0;
+ case 'n':
+ case 'N':
+ case '0':
+ *res = false;
+ return 0;
+ case 'o':
+ case 'O':
+ switch (s[1]) {
+ case 'n':
+ case 'N':
+ *res = true;
+ return 0;
+ case 'f':
+ case 'F':
+ *res = false;
+ return 0;
+ default:
+ break;
+ }
+ default:
+ break;
+ }
+
+ return -EINVAL;
+}
static cpacf_mask_t km_functions, kmc_functions, kmctr_functions;
+struct key_blob {
+ __u8 key[MAXKEYBLOBSIZE];
+ unsigned int keylen;
+};
+
struct s390_paes_ctx {
- struct pkey_seckey sk;
+ struct key_blob kb;
struct pkey_protkey pk;
unsigned long fc;
};
struct s390_pxts_ctx {
- struct pkey_seckey sk[2];
+ struct key_blob kb[2];
struct pkey_protkey pk[2];
unsigned long fc;
};
-static inline int __paes_convert_key(struct pkey_seckey *sk,
+static inline int __paes_convert_key(struct key_blob *kb,
struct pkey_protkey *pk)
{
int i, ret;
/* try three times in case of failure */
for (i = 0; i < 3; i++) {
- ret = pkey_skey2pkey(sk, pk);
+ ret = pkey_keyblob2pkey(kb->key, kb->keylen, pk);
if (ret == 0)
break;
}
{
unsigned long fc;
- if (__paes_convert_key(&ctx->sk, &ctx->pk))
+ if (__paes_convert_key(&ctx->kb, &ctx->pk))
return -EINVAL;
/* Pick the correct function code based on the protected key type */
{
struct s390_paes_ctx *ctx = crypto_tfm_ctx(tfm);
- if (key_len != SECKEYBLOBSIZE)
- return -EINVAL;
-
- memcpy(ctx->sk.seckey, in_key, SECKEYBLOBSIZE);
+ memcpy(ctx->kb.key, in_key, key_len);
+ ctx->kb.keylen = key_len;
if (__paes_set_key(ctx)) {
tfm->crt_flags |= CRYPTO_TFM_RES_BAD_KEY_LEN;
return -EINVAL;
.cra_list = LIST_HEAD_INIT(ecb_paes_alg.cra_list),
.cra_u = {
.blkcipher = {
- .min_keysize = SECKEYBLOBSIZE,
- .max_keysize = SECKEYBLOBSIZE,
+ .min_keysize = MINKEYBLOBSIZE,
+ .max_keysize = MAXKEYBLOBSIZE,
.setkey = ecb_paes_set_key,
.encrypt = ecb_paes_encrypt,
.decrypt = ecb_paes_decrypt,
{
unsigned long fc;
- if (__paes_convert_key(&ctx->sk, &ctx->pk))
+ if (__paes_convert_key(&ctx->kb, &ctx->pk))
return -EINVAL;
/* Pick the correct function code based on the protected key type */
{
struct s390_paes_ctx *ctx = crypto_tfm_ctx(tfm);
- memcpy(ctx->sk.seckey, in_key, SECKEYBLOBSIZE);
+ memcpy(ctx->kb.key, in_key, key_len);
+ ctx->kb.keylen = key_len;
if (__cbc_paes_set_key(ctx)) {
tfm->crt_flags |= CRYPTO_TFM_RES_BAD_KEY_LEN;
return -EINVAL;
.cra_list = LIST_HEAD_INIT(cbc_paes_alg.cra_list),
.cra_u = {
.blkcipher = {
- .min_keysize = SECKEYBLOBSIZE,
- .max_keysize = SECKEYBLOBSIZE,
+ .min_keysize = MINKEYBLOBSIZE,
+ .max_keysize = MAXKEYBLOBSIZE,
.ivsize = AES_BLOCK_SIZE,
.setkey = cbc_paes_set_key,
.encrypt = cbc_paes_encrypt,
{
unsigned long fc;
- if (__paes_convert_key(&ctx->sk[0], &ctx->pk[0]) ||
- __paes_convert_key(&ctx->sk[1], &ctx->pk[1]))
+ if (__paes_convert_key(&ctx->kb[0], &ctx->pk[0]) ||
+ __paes_convert_key(&ctx->kb[1], &ctx->pk[1]))
return -EINVAL;
if (ctx->pk[0].type != ctx->pk[1].type)
{
struct s390_pxts_ctx *ctx = crypto_tfm_ctx(tfm);
u8 ckey[2 * AES_MAX_KEY_SIZE];
- unsigned int ckey_len;
+ unsigned int ckey_len, keytok_len;
+
+ if (key_len % 2)
+ return -EINVAL;
- memcpy(ctx->sk[0].seckey, in_key, SECKEYBLOBSIZE);
- memcpy(ctx->sk[1].seckey, in_key + SECKEYBLOBSIZE, SECKEYBLOBSIZE);
+ keytok_len = key_len / 2;
+ memcpy(ctx->kb[0].key, in_key, keytok_len);
+ ctx->kb[0].keylen = keytok_len;
+ memcpy(ctx->kb[1].key, in_key + keytok_len, keytok_len);
+ ctx->kb[1].keylen = keytok_len;
if (__xts_paes_set_key(ctx)) {
tfm->crt_flags |= CRYPTO_TFM_RES_BAD_KEY_LEN;
return -EINVAL;
.cra_list = LIST_HEAD_INIT(xts_paes_alg.cra_list),
.cra_u = {
.blkcipher = {
- .min_keysize = 2 * SECKEYBLOBSIZE,
- .max_keysize = 2 * SECKEYBLOBSIZE,
+ .min_keysize = 2 * MINKEYBLOBSIZE,
+ .max_keysize = 2 * MAXKEYBLOBSIZE,
.ivsize = AES_BLOCK_SIZE,
.setkey = xts_paes_set_key,
.encrypt = xts_paes_encrypt,
{
unsigned long fc;
- if (__paes_convert_key(&ctx->sk, &ctx->pk))
+ if (__paes_convert_key(&ctx->kb, &ctx->pk))
return -EINVAL;
/* Pick the correct function code based on the protected key type */
{
struct s390_paes_ctx *ctx = crypto_tfm_ctx(tfm);
- memcpy(ctx->sk.seckey, in_key, key_len);
+ memcpy(ctx->kb.key, in_key, key_len);
+ ctx->kb.keylen = key_len;
if (__ctr_paes_set_key(ctx)) {
tfm->crt_flags |= CRYPTO_TFM_RES_BAD_KEY_LEN;
return -EINVAL;
.cra_list = LIST_HEAD_INIT(ctr_paes_alg.cra_list),
.cra_u = {
.blkcipher = {
- .min_keysize = SECKEYBLOBSIZE,
- .max_keysize = SECKEYBLOBSIZE,
+ .min_keysize = MINKEYBLOBSIZE,
+ .max_keysize = MAXKEYBLOBSIZE,
.ivsize = AES_BLOCK_SIZE,
.setkey = ctr_paes_set_key,
.encrypt = ctr_paes_encrypt,
CONFIG_CRYPTO_USER_API_SKCIPHER=m
CONFIG_CRYPTO_USER_API_RNG=m
CONFIG_ZCRYPT=m
+CONFIG_ZCRYPT_MULTIDEVNODES=y
CONFIG_PKEY=m
CONFIG_CRYPTO_PAES_S390=m
CONFIG_CRYPTO_SHA1_S390=m
static int __hypfs_sprp_ioctl(void __user *user_area)
{
- struct hypfs_diag304 diag304;
+ struct hypfs_diag304 *diag304;
unsigned long cmd;
void __user *udata;
void *data;
int rc;
- if (copy_from_user(&diag304, user_area, sizeof(diag304)))
- return -EFAULT;
- if ((diag304.args[0] >> 8) != 0 || diag304.args[1] > DIAG304_CMD_MAX)
- return -EINVAL;
-
+ rc = -ENOMEM;
data = (void *) get_zeroed_page(GFP_KERNEL | GFP_DMA);
- if (!data)
- return -ENOMEM;
-
- udata = (void __user *)(unsigned long) diag304.data;
- if (diag304.args[1] == DIAG304_SET_WEIGHTS ||
- diag304.args[1] == DIAG304_SET_CAPPING)
- if (copy_from_user(data, udata, PAGE_SIZE)) {
- rc = -EFAULT;
+ diag304 = kzalloc(sizeof(*diag304), GFP_KERNEL);
+ if (!data || !diag304)
+ goto out;
+
+ rc = -EFAULT;
+ if (copy_from_user(diag304, user_area, sizeof(*diag304)))
+ goto out;
+ rc = -EINVAL;
+ if ((diag304->args[0] >> 8) != 0 || diag304->args[1] > DIAG304_CMD_MAX)
+ goto out;
+
+ rc = -EFAULT;
+ udata = (void __user *)(unsigned long) diag304->data;
+ if (diag304->args[1] == DIAG304_SET_WEIGHTS ||
+ diag304->args[1] == DIAG304_SET_CAPPING)
+ if (copy_from_user(data, udata, PAGE_SIZE))
goto out;
- }
- cmd = *(unsigned long *) &diag304.args[0];
- diag304.rc = hypfs_sprp_diag304(data, cmd);
+ cmd = *(unsigned long *) &diag304->args[0];
+ diag304->rc = hypfs_sprp_diag304(data, cmd);
- if (diag304.args[1] == DIAG304_QUERY_PRP)
+ if (diag304->args[1] == DIAG304_QUERY_PRP)
if (copy_to_user(udata, data, PAGE_SIZE)) {
rc = -EFAULT;
goto out;
}
- rc = copy_to_user(user_area, &diag304, sizeof(diag304)) ? -EFAULT : 0;
+ rc = copy_to_user(user_area, diag304, sizeof(*diag304)) ? -EFAULT : 0;
out:
+ kfree(diag304);
free_page((unsigned long) data);
return rc;
}
u16 mod_lvl; /* modification level */
} __attribute__ ((packed));
-static inline int appldata_asm(struct appldata_product_id *id,
+
+static inline int appldata_asm(struct appldata_parameter_list *parm_list,
+ struct appldata_product_id *id,
unsigned short fn, void *buffer,
unsigned short length)
{
- struct appldata_parameter_list parm_list;
int ry;
if (!MACHINE_IS_VM)
return -EOPNOTSUPP;
- parm_list.diag = 0xdc;
- parm_list.function = fn;
- parm_list.parlist_length = sizeof(parm_list);
- parm_list.buffer_length = length;
- parm_list.product_id_addr = (unsigned long) id;
- parm_list.buffer_addr = virt_to_phys(buffer);
+ parm_list->diag = 0xdc;
+ parm_list->function = fn;
+ parm_list->parlist_length = sizeof(*parm_list);
+ parm_list->buffer_length = length;
+ parm_list->product_id_addr = (unsigned long) id;
+ parm_list->buffer_addr = virt_to_phys(buffer);
diag_stat_inc(DIAG_STAT_X0DC);
asm volatile(
" diag %1,%0,0xdc"
: "=d" (ry)
- : "d" (&parm_list), "m" (parm_list), "m" (*id)
+ : "d" (parm_list), "m" (*parm_list), "m" (*id)
: "cc");
return ry;
}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_S390_BOOT_DATA_H
+
+#include <asm/setup.h>
+#include <asm/ipl.h>
+
+extern char early_command_line[COMMAND_LINE_SIZE];
+extern struct ipl_parameter_block early_ipl_block;
+extern int early_ipl_block_valid;
+
+#endif /* _ASM_S390_BOOT_DATA_H */
extern void ccwgroup_driver_unregister (struct ccwgroup_driver *cdriver);
int ccwgroup_create_dev(struct device *root, struct ccwgroup_driver *gdrv,
int num_devices, const char *buf);
+struct ccwgroup_device *get_ccwgroupdev_by_busid(struct ccwgroup_driver *gdrv,
+ char *bus_id);
extern int ccwgroup_set_online(struct ccwgroup_device *gdev);
extern int ccwgroup_set_offline(struct ccwgroup_device *gdev);
* @stfle_fac_list: array where facility list can be stored
* @size: size of passed in array in double words
*/
-static inline void stfle(u64 *stfle_fac_list, int size)
+static inline void __stfle(u64 *stfle_fac_list, int size)
{
unsigned long nr;
- preempt_disable();
asm volatile(
" stfl 0(0)\n"
: "=m" (S390_lowcore.stfl_fac_list));
nr = (reg0 + 1) * 8; /* # bytes stored by stfle */
}
memset((char *) stfle_fac_list + nr, 0, size * 8 - nr);
+}
+
+static inline void stfle(u64 *stfle_fac_list, int size)
+{
+ preempt_disable();
+ __stfle(stfle_fac_list, size);
preempt_enable();
}
extern void s390_reset_system(void);
extern void ipl_store_parameters(void);
-extern size_t append_ipl_vmparm(char *, size_t);
-extern size_t append_ipl_scpdata(char *, size_t);
+extern size_t ipl_block_get_ascii_vmparm(char *dest, size_t size,
+ const struct ipl_parameter_block *ipb);
enum ipl_type {
IPL_TYPE_UNKNOWN = 1,
* We use a brcl 0,2 instruction for jump labels at compile time so it
* can be easily distinguished from a hotpatch generated instruction.
*/
-static __always_inline bool arch_static_branch(struct static_key *key, bool branch)
+static inline bool arch_static_branch(struct static_key *key, bool branch)
{
- asm_volatile_goto("0: brcl 0,"__stringify(JUMP_LABEL_NOP_OFFSET)"\n"
- ".pushsection __jump_table, \"aw\"\n"
- ".balign 8\n"
- ".quad 0b, %l[label], %0\n"
- ".popsection\n"
- : : "X" (&((char *)key)[branch]) : : label);
-
+ asm_volatile_goto("0: brcl 0,"__stringify(JUMP_LABEL_NOP_OFFSET)"\n"
+ ".pushsection __jump_table,\"aw\"\n"
+ ".balign 8\n"
+ ".long 0b-.,%l[label]-.\n"
+ ".quad %0-.\n"
+ ".popsection\n"
+ : : "X" (&((char *)key)[branch]) : : label);
return false;
label:
return true;
}
-static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch)
+static inline bool arch_static_branch_jump(struct static_key *key, bool branch)
{
- asm_volatile_goto("0: brcl 15, %l[label]\n"
- ".pushsection __jump_table, \"aw\"\n"
- ".balign 8\n"
- ".quad 0b, %l[label], %0\n"
- ".popsection\n"
- : : "X" (&((char *)key)[branch]) : : label);
-
+ asm_volatile_goto("0: brcl 15,%l[label]\n"
+ ".pushsection __jump_table,\"aw\"\n"
+ ".balign 8\n"
+ ".long 0b-.,%l[label]-.\n"
+ ".quad %0-.\n"
+ ".popsection\n"
+ : : "X" (&((char *)key)[branch]) : : label);
return false;
label:
return true;
}
-typedef unsigned long jump_label_t;
-
-struct jump_entry {
- jump_label_t code;
- jump_label_t target;
- jump_label_t key;
-};
-
#endif /* __ASSEMBLY__ */
#endif
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ASM_KASAN_H
+#define __ASM_KASAN_H
+
+#include <asm/pgtable.h>
+
+#ifdef CONFIG_KASAN
+
+#define KASAN_SHADOW_SCALE_SHIFT 3
+#ifdef CONFIG_KASAN_S390_4_LEVEL_PAGING
+#define KASAN_SHADOW_SIZE \
+ (_AC(1, UL) << (_REGION1_SHIFT - KASAN_SHADOW_SCALE_SHIFT))
+#else
+#define KASAN_SHADOW_SIZE \
+ (_AC(1, UL) << (_REGION2_SHIFT - KASAN_SHADOW_SCALE_SHIFT))
+#endif
+#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL)
+#define KASAN_SHADOW_START KASAN_SHADOW_OFFSET
+#define KASAN_SHADOW_END (KASAN_SHADOW_START + KASAN_SHADOW_SIZE)
+
+extern void kasan_early_init(void);
+extern void kasan_copy_shadow(pgd_t *dst);
+extern void kasan_free_early_identity(void);
+#else
+static inline void kasan_early_init(void) { }
+static inline void kasan_copy_shadow(pgd_t *dst) { }
+static inline void kasan_free_early_identity(void) { }
+#endif
+
+#endif
__u64 current_task; /* 0x0338 */
__u64 kernel_stack; /* 0x0340 */
- /* Interrupt, panic and restart stack. */
+ /* Interrupt, DAT-off and restartstack. */
__u64 async_stack; /* 0x0348 */
- __u64 panic_stack; /* 0x0350 */
+ __u64 nodat_stack; /* 0x0350 */
__u64 restart_stack; /* 0x0358 */
/* Restart function and parameter. */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_S390_MEM_DETECT_H
+#define _ASM_S390_MEM_DETECT_H
+
+#include <linux/types.h>
+
+enum mem_info_source {
+ MEM_DETECT_NONE = 0,
+ MEM_DETECT_SCLP_STOR_INFO,
+ MEM_DETECT_DIAG260,
+ MEM_DETECT_SCLP_READ_INFO,
+ MEM_DETECT_BIN_SEARCH
+};
+
+struct mem_detect_block {
+ u64 start;
+ u64 end;
+};
+
+/*
+ * Storage element id is defined as 1 byte (up to 256 storage elements).
+ * In practise only storage element id 0 and 1 are used).
+ * According to architecture one storage element could have as much as
+ * 1020 subincrements. 255 mem_detect_blocks are embedded in mem_detect_info.
+ * If more mem_detect_blocks are required, a block of memory from already
+ * known mem_detect_block is taken (entries_extended points to it).
+ */
+#define MEM_INLINED_ENTRIES 255 /* (PAGE_SIZE - 16) / 16 */
+
+struct mem_detect_info {
+ u32 count;
+ u8 info_source;
+ struct mem_detect_block entries[MEM_INLINED_ENTRIES];
+ struct mem_detect_block *entries_extended;
+};
+extern struct mem_detect_info mem_detect;
+
+void add_mem_detect_block(u64 start, u64 end);
+
+static inline int __get_mem_detect_block(u32 n, unsigned long *start,
+ unsigned long *end)
+{
+ if (n >= mem_detect.count) {
+ *start = 0;
+ *end = 0;
+ return -1;
+ }
+
+ if (n < MEM_INLINED_ENTRIES) {
+ *start = (unsigned long)mem_detect.entries[n].start;
+ *end = (unsigned long)mem_detect.entries[n].end;
+ } else {
+ *start = (unsigned long)mem_detect.entries_extended[n - MEM_INLINED_ENTRIES].start;
+ *end = (unsigned long)mem_detect.entries_extended[n - MEM_INLINED_ENTRIES].end;
+ }
+ return 0;
+}
+
+/**
+ * for_each_mem_detect_block - early online memory range iterator
+ * @i: an integer used as loop variable
+ * @p_start: ptr to unsigned long for start address of the range
+ * @p_end: ptr to unsigned long for end address of the range
+ *
+ * Walks over detected online memory ranges.
+ */
+#define for_each_mem_detect_block(i, p_start, p_end) \
+ for (i = 0, __get_mem_detect_block(i, p_start, p_end); \
+ i < mem_detect.count; \
+ i++, __get_mem_detect_block(i, p_start, p_end))
+
+static inline void get_mem_detect_reserved(unsigned long *start,
+ unsigned long *size)
+{
+ *start = (unsigned long)mem_detect.entries_extended;
+ if (mem_detect.count > MEM_INLINED_ENTRIES)
+ *size = (mem_detect.count - MEM_INLINED_ENTRIES) * sizeof(struct mem_detect_block);
+ else
+ *size = 0;
+}
+
+#endif
unsigned int uses_cmm:1;
/* The gmaps associated with this context are allowed to use huge pages. */
unsigned int allow_gmap_hpage_1m:1;
+ /* The mmu context is for compat task */
+ unsigned int compat_mm:1;
} mm_context_t;
#define INIT_MM_CONTEXT(name) \
atomic_set(&mm->context.flush_count, 0);
mm->context.gmap_asce = 0;
mm->context.flush_mm = 0;
+ mm->context.compat_mm = 0;
#ifdef CONFIG_PGSTE
mm->context.alloc_pgste = page_table_allocate_pgste ||
test_thread_flag(TIF_PGSTE) ||
#define virt_to_pfn(kaddr) (__pa(kaddr) >> PAGE_SHIFT)
#define pfn_to_virt(pfn) __va((pfn) << PAGE_SHIFT)
+#define pfn_to_kaddr(pfn) pfn_to_virt(pfn)
#define virt_to_page(kaddr) pfn_to_page(virt_to_pfn(kaddr))
#define page_to_virt(page) pfn_to_virt(page_to_pfn(page))
#define PTRS_PER_P4D _CRST_ENTRIES
#define PTRS_PER_PGD _CRST_ENTRIES
+#define MAX_PTRS_PER_P4D PTRS_PER_P4D
+
/*
* Segment table and region3 table entry encoding
* (R = read-only, I = invalid, y = young bit):
_SEGMENT_ENTRY_YOUNG | \
_SEGMENT_ENTRY_PROTECT | \
_SEGMENT_ENTRY_NOEXEC)
+#define SEGMENT_KERNEL_EXEC __pgprot(_SEGMENT_ENTRY | \
+ _SEGMENT_ENTRY_LARGE | \
+ _SEGMENT_ENTRY_READ | \
+ _SEGMENT_ENTRY_WRITE | \
+ _SEGMENT_ENTRY_YOUNG | \
+ _SEGMENT_ENTRY_DIRTY)
/*
* Region3 entry (large page) protection definitions.
return (pgd_val(pgd) & mask) != 0;
}
+static inline unsigned long pgd_pfn(pgd_t pgd)
+{
+ unsigned long origin_mask;
+
+ origin_mask = _REGION_ENTRY_ORIGIN;
+ return (pgd_val(pgd) & origin_mask) >> PAGE_SHIFT;
+}
+
static inline int p4d_folded(p4d_t p4d)
{
return (p4d_val(p4d) & _REGION_ENTRY_TYPE_MASK) < _REGION_ENTRY_TYPE_R2;
#define pgd_offset(mm, address) ((mm)->pgd + pgd_index(address))
#define pgd_offset_k(address) pgd_offset(&init_mm, address)
+#define pgd_offset_raw(pgd, addr) ((pgd) + pgd_index(addr))
#define pmd_deref(pmd) (pmd_val(pmd) & _SEGMENT_ENTRY_ORIGIN)
#define pud_deref(pud) (pud_val(pud) & _REGION_ENTRY_ORIGIN)
#define pmd_page(pmd) pfn_to_page(pmd_pfn(pmd))
#define pud_page(pud) pfn_to_page(pud_pfn(pud))
-#define p4d_page(pud) pfn_to_page(p4d_pfn(p4d))
+#define p4d_page(p4d) pfn_to_page(p4d_pfn(p4d))
+#define pgd_page(pgd) pfn_to_page(pgd_pfn(pgd))
/* Find an entry in the lowest level page table.. */
#define pte_offset(pmd, addr) ((pte_t *) pmd_deref(*(pmd)) + pte_index(addr))
u16 *pcardnr, u16 *pdomain,
u16 *pkeysize, u32 *pattributes);
+/*
+ * In-kernel API: Generate (AES) random protected key.
+ * @param keytype one of the PKEY_KEYTYPE values
+ * @param protkey pointer to buffer receiving the protected key
+ * @return 0 on success, negative errno value on failure
+ */
+int pkey_genprotkey(__u32 keytype, struct pkey_protkey *protkey);
+
+/*
+ * In-kernel API: Verify an (AES) protected key.
+ * @param protkey pointer to buffer containing the protected key to verify
+ * @return 0 on success, negative errno value on failure. In case the protected
+ * key is not valid -EKEYREJECTED is returned
+ */
+int pkey_verifyprotkey(const struct pkey_protkey *protkey);
+
+/*
+ * In-kernel API: Transform an key blob (of any type) into a protected key.
+ * @param key pointer to a buffer containing the key blob
+ * @param keylen size of the key blob in bytes
+ * @param protkey pointer to buffer receiving the protected key
+ * @return 0 on success, negative errno value on failure
+ */
+int pkey_keyblob2pkey(const __u8 *key, __u32 keylen,
+ struct pkey_protkey *protkey);
+
#endif /* _KAPI_PKEY_H */
return sp;
}
-static inline unsigned short stap(void)
+static __no_sanitize_address_or_inline unsigned short stap(void)
{
unsigned short cpu_address;
return cpu_address;
}
+#define CALL_ARGS_0() \
+ register unsigned long r2 asm("2")
+#define CALL_ARGS_1(arg1) \
+ register unsigned long r2 asm("2") = (unsigned long)(arg1)
+#define CALL_ARGS_2(arg1, arg2) \
+ CALL_ARGS_1(arg1); \
+ register unsigned long r3 asm("3") = (unsigned long)(arg2)
+#define CALL_ARGS_3(arg1, arg2, arg3) \
+ CALL_ARGS_2(arg1, arg2); \
+ register unsigned long r4 asm("4") = (unsigned long)(arg3)
+#define CALL_ARGS_4(arg1, arg2, arg3, arg4) \
+ CALL_ARGS_3(arg1, arg2, arg3); \
+ register unsigned long r4 asm("5") = (unsigned long)(arg4)
+#define CALL_ARGS_5(arg1, arg2, arg3, arg4, arg5) \
+ CALL_ARGS_4(arg1, arg2, arg3, arg4); \
+ register unsigned long r4 asm("6") = (unsigned long)(arg5)
+
+#define CALL_FMT_0
+#define CALL_FMT_1 CALL_FMT_0, "0" (r2)
+#define CALL_FMT_2 CALL_FMT_1, "d" (r3)
+#define CALL_FMT_3 CALL_FMT_2, "d" (r4)
+#define CALL_FMT_4 CALL_FMT_3, "d" (r5)
+#define CALL_FMT_5 CALL_FMT_4, "d" (r6)
+
+#define CALL_CLOBBER_5 "0", "1", "14", "cc", "memory"
+#define CALL_CLOBBER_4 CALL_CLOBBER_5
+#define CALL_CLOBBER_3 CALL_CLOBBER_4, "5"
+#define CALL_CLOBBER_2 CALL_CLOBBER_3, "4"
+#define CALL_CLOBBER_1 CALL_CLOBBER_2, "3"
+#define CALL_CLOBBER_0 CALL_CLOBBER_1
+
+#define CALL_ON_STACK(fn, stack, nr, args...) \
+({ \
+ CALL_ARGS_##nr(args); \
+ unsigned long prev; \
+ \
+ asm volatile( \
+ " la %[_prev],0(15)\n" \
+ " la 15,0(%[_stack])\n" \
+ " stg %[_prev],%[_bc](15)\n" \
+ " brasl 14,%[_fn]\n" \
+ " la 15,0(%[_prev])\n" \
+ : "+&d" (r2), [_prev] "=&a" (prev) \
+ : [_stack] "a" (stack), \
+ [_bc] "i" (offsetof(struct stack_frame, back_chain)), \
+ [_fn] "X" (fn) CALL_FMT_##nr : CALL_CLOBBER_##nr); \
+ r2; \
+})
+
/*
* Give up the time slice of the virtual PU.
*/
* Set PSW mask to specified value, while leaving the
* PSW addr pointing to the next instruction.
*/
-static inline void __load_psw_mask(unsigned long mask)
+static __no_sanitize_address_or_inline void __load_psw_mask(unsigned long mask)
{
unsigned long addr;
psw_t psw;
* (for communication with upper layer programs)
* (only required for use with completion queues)
* @flags: flags indicating state of buffer
- * @aob: pointer to QAOB used for the particular SBAL
* @user: pointer to upper layer program's state information related to SBAL
* (stored in user1 data of QAOB)
*/
struct qdio_outbuf_state {
u8 flags;
- struct qaob *aob;
void *user;
};
struct zpci_report_error_header {
u8 version; /* Interface version byte */
u8 action; /* Action qualifier byte
+ * 0: Adapter Reset Request
* 1: Deconfigure and repair action requested
* (OpenCrypto Problem Call Home)
* 2: Informational Report
u8 data[0]; /* Subsequent Data passed verbatim to SCLP ET 24 */
} __packed;
+int sclp_early_read_info(void);
+int sclp_early_read_storage_info(void);
int sclp_early_get_core_info(struct sclp_core_info *info);
void sclp_early_get_ipl_info(struct sclp_ipl_info *info);
void sclp_early_detect(void);
void sclp_early_printk(const char *s);
-void __sclp_early_printk(const char *s, unsigned int len);
+void sclp_early_printk_force(const char *s);
+void __sclp_early_printk(const char *s, unsigned int len, unsigned int force);
+int sclp_early_get_memsize(unsigned long *mem);
+int sclp_early_get_hsa_size(unsigned long *hsa_size);
int _sclp_get_core_info(struct sclp_core_info *info);
int sclp_core_configure(u8 core);
int sclp_core_deconfigure(u8 core);
#include <asm-generic/sections.h>
+/*
+ * .boot.data section contains variables "shared" between the decompressor and
+ * the decompressed kernel. The decompressor will store values in them, and
+ * copy over to the decompressed image before starting it.
+ *
+ * Each variable end up in its own intermediate section .boot.data.<var name>,
+ * those sections are later sorted by alignment + name and merged together into
+ * final .boot.data section, which should be identical in the decompressor and
+ * the decompressed kernel (that is checked during the build).
+ */
+#define __bootdata(var) __section(.boot.data.var) var
+
#endif
#define OLDMEM_SIZE (*(unsigned long *) (OLDMEM_SIZE_OFFSET))
#define COMMAND_LINE ((char *) (COMMAND_LINE_OFFSET))
+extern int noexec_disabled;
extern int memory_end_set;
extern unsigned long memory_end;
extern unsigned long max_physmem_end;
-extern void detect_memory_memblock(void);
-
#define MACHINE_IS_VM (S390_lowcore.machine_flags & MACHINE_FLAG_VM)
#define MACHINE_IS_KVM (S390_lowcore.machine_flags & MACHINE_FLAG_KVM)
#define MACHINE_IS_LPAR (S390_lowcore.machine_flags & MACHINE_FLAG_LPAR)
#undef __HAVE_ARCH_STRSEP
#undef __HAVE_ARCH_STRSPN
+#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)
+
+extern void *__memcpy(void *dest, const void *src, size_t n);
+extern void *__memset(void *s, int c, size_t n);
+extern void *__memmove(void *dest, const void *src, size_t n);
+
+/*
+ * For files that are not instrumented (e.g. mm/slub.c) we
+ * should use not instrumented version of mem* functions.
+ */
+
+#define memcpy(dst, src, len) __memcpy(dst, src, len)
+#define memmove(dst, src, len) __memmove(dst, src, len)
+#define memset(s, c, n) __memset(s, c, n)
+
+#ifndef __NO_FORTIFY
+#define __NO_FORTIFY /* FORTIFY_SOURCE uses __builtin_memcpy, etc. */
+#endif
+
+#endif /* defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__) */
+
void *__memset16(uint16_t *s, uint16_t v, size_t count);
void *__memset32(uint32_t *s, uint32_t v, size_t count);
void *__memset64(uint64_t *s, uint64_t v, size_t count);
#include <linux/const.h>
/*
- * Size of kernel stack for each process
+ * General size of kernel stacks
*/
+#ifdef CONFIG_KASAN
+#define THREAD_SIZE_ORDER 3
+#else
#define THREAD_SIZE_ORDER 2
-#define ASYNC_ORDER 2
-
+#endif
+#define BOOT_STACK_ORDER 2
#define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER)
-#define ASYNC_SIZE (PAGE_SIZE << ASYNC_ORDER)
#ifndef __ASSEMBLY__
#include <asm/lowcore.h>
#include <asm/page.h>
#include <asm/processor.h>
+#define STACK_INIT_OFFSET \
+ (THREAD_SIZE - STACK_FRAME_OVERHEAD - sizeof(struct pt_regs))
+
/*
* low level task data that entry.S needs immediate access to
* - this struct should fit entirely inside of one cache line
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/page.h>
+
+/*
+ * .boot.data section is shared between the decompressor code and the
+ * decompressed kernel. The decompressor will store values in it, and copy
+ * over to the decompressed image before starting it.
+ *
+ * .boot.data variables are kept in separate .boot.data.<var name> sections,
+ * which are sorted by alignment first, then by name before being merged
+ * into single .boot.data section. This way big holes cased by page aligned
+ * structs are avoided and linker produces consistent result.
+ */
+#define BOOT_DATA \
+ . = ALIGN(PAGE_SIZE); \
+ .boot.data : { \
+ __boot_data_start = .; \
+ *(SORT_BY_ALIGNMENT(SORT_BY_NAME(.boot.data*))) \
+ __boot_data_end = .; \
+ }
#define PKEY_IOCTL_MAGIC 'p'
#define SECKEYBLOBSIZE 64 /* secure key blob size is always 64 bytes */
+#define PROTKEYBLOBSIZE 80 /* protected key blob size is always 80 bytes */
#define MAXPROTKEYSIZE 64 /* a protected key blob may be up to 64 bytes */
#define MAXCLRKEYSIZE 32 /* a clear key value may be up to 32 bytes */
+#define MINKEYBLOBSIZE SECKEYBLOBSIZE /* Minimum size of a key blob */
+#define MAXKEYBLOBSIZE PROTKEYBLOBSIZE /* Maximum size of a key blob */
+
/* defines for the type field within the pkey_protkey struct */
#define PKEY_KEYTYPE_AES_128 1
#define PKEY_KEYTYPE_AES_192 2
#define PKEY_VERIFY_ATTR_AES 0x00000001 /* key is an AES key */
#define PKEY_VERIFY_ATTR_OLD_MKVP 0x00000100 /* key has old MKVP value */
+/*
+ * Generate (AES) random protected key.
+ */
+struct pkey_genprotk {
+ __u32 keytype; /* in: key type to generate */
+ struct pkey_protkey protkey; /* out: the protected key */
+};
+
+#define PKEY_GENPROTK _IOWR(PKEY_IOCTL_MAGIC, 0x08, struct pkey_genprotk)
+
+/*
+ * Verify an (AES) protected key.
+ */
+struct pkey_verifyprotk {
+ struct pkey_protkey protkey; /* in: the protected key to verify */
+};
+
+#define PKEY_VERIFYPROTK _IOW(PKEY_IOCTL_MAGIC, 0x09, struct pkey_verifyprotk)
+
+/*
+ * Transform an key blob (of any type) into a protected key
+ */
+struct pkey_kblob2pkey {
+ __u8 __user *key; /* in: the key blob */
+ __u32 keylen; /* in: the key blob length */
+ struct pkey_protkey protkey; /* out: the protected key */
+};
+
+#define PKEY_KBLOB2PROTK _IOWR(PKEY_IOCTL_MAGIC, 0x0A, struct pkey_kblob2pkey)
+
#endif /* _UAPI_PKEY_H */
/*
* include/asm-s390/zcrypt.h
*
- * zcrypt 2.1.0 (user-visible header)
+ * zcrypt 2.2.1 (user-visible header)
*
- * Copyright IBM Corp. 2001, 2006
+ * Copyright IBM Corp. 2001, 2018
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
*
#define __ASM_S390_ZCRYPT_H
#define ZCRYPT_VERSION 2
-#define ZCRYPT_RELEASE 1
+#define ZCRYPT_RELEASE 2
#define ZCRYPT_VARIANT 1
#include <linux/ioctl.h>
#include <linux/compiler.h>
+/* Name of the zcrypt device driver. */
+#define ZCRYPT_NAME "zcrypt"
+
/**
* struct ica_rsa_modexpo
*
#define ZCRYPT_QDEPTH_MASK _IOR(ZCRYPT_IOCTL_MAGIC, 0x59, char[MAX_ZDEV_CARDIDS_EXT])
#define ZCRYPT_PERDEV_REQCNT _IOR(ZCRYPT_IOCTL_MAGIC, 0x5a, int[MAX_ZDEV_CARDIDS_EXT])
+/*
+ * Support for multiple zcrypt device nodes.
+ */
+
+/* Nr of minor device node numbers to allocate. */
+#define ZCRYPT_MAX_MINOR_NODES 256
+
+/* Max amount of possible ioctls */
+#define MAX_ZDEV_IOCTLS (1 << _IOC_NRBITS)
+
/*
* Only deprecated defines, structs and ioctls below this line.
*/
UBSAN_SANITIZE_early.o := n
UBSAN_SANITIZE_early_nobss.o := n
+KASAN_SANITIZE_early_nobss.o := n
+KASAN_SANITIZE_ipl.o := n
+KASAN_SANITIZE_machine_kexec.o := n
+
#
# Passing null pointers is ok for smp code, since we access the lowcore here.
#
obj-y += sysinfo.o jump_label.o lgr.o os_info.o machine_kexec.o pgm_check.o
obj-y += runtime_instr.o cache.o fpu.o dumpstack.o guarded_storage.o sthyi.o
obj-y += entry.o reipl.o relocate_kernel.o kdebugfs.o alternative.o
-obj-y += nospec-branch.o
+obj-y += nospec-branch.o ipl_vmparm.o
extra-y += head64.o vmlinux.lds
OFFSET(__LC_CURRENT, lowcore, current_task);
OFFSET(__LC_KERNEL_STACK, lowcore, kernel_stack);
OFFSET(__LC_ASYNC_STACK, lowcore, async_stack);
- OFFSET(__LC_PANIC_STACK, lowcore, panic_stack);
+ OFFSET(__LC_NODAT_STACK, lowcore, nodat_stack);
OFFSET(__LC_RESTART_STACK, lowcore, restart_stack);
OFFSET(__LC_RESTART_FN, lowcore, restart_fn);
OFFSET(__LC_RESTART_DATA, lowcore, restart_data);
ENTRY(s390_base_mcck_handler)
basr %r13,0
-0: lg %r15,__LC_PANIC_STACK # load panic stack
+0: lg %r15,__LC_NODAT_STACK # load panic stack
aghi %r15,-STACK_FRAME_OVERHEAD
larl %r1,s390_base_mcck_handler_fn
lg %r9,0(%r1)
* The stack trace can start at any of the three stacks and can potentially
* touch all of them. The order is: panic stack, async stack, sync stack.
*/
-static unsigned long
+static unsigned long __no_sanitize_address
__dump_trace(dump_trace_func_t func, void *data, unsigned long sp,
unsigned long low, unsigned long high)
{
frame_size = STACK_FRAME_OVERHEAD + sizeof(struct pt_regs);
#ifdef CONFIG_CHECK_STACK
sp = __dump_trace(func, data, sp,
- S390_lowcore.panic_stack + frame_size - PAGE_SIZE,
- S390_lowcore.panic_stack + frame_size);
+ S390_lowcore.nodat_stack + frame_size - THREAD_SIZE,
+ S390_lowcore.nodat_stack + frame_size);
#endif
sp = __dump_trace(func, data, sp,
- S390_lowcore.async_stack + frame_size - ASYNC_SIZE,
+ S390_lowcore.async_stack + frame_size - THREAD_SIZE,
S390_lowcore.async_stack + frame_size);
task = task ?: current;
__dump_trace(func, data, sp,
char *mode;
mode = user_mode(regs) ? "User" : "Krnl";
- printk("%s PSW : %p %p", mode, (void *)regs->psw.mask, (void *)regs->psw.addr);
+ printk("%s PSW : %px %px", mode, (void *)regs->psw.mask, (void *)regs->psw.addr);
if (!user_mode(regs))
pr_cont(" (%pSR)", (void *)regs->psw.addr);
pr_cont("\n");
#include <asm/cpcmd.h>
#include <asm/sclp.h>
#include <asm/facility.h>
+#include <asm/boot_data.h>
#include "entry.h"
-static void __init setup_boot_command_line(void);
-
/*
* Initialize storage key for kernel pages
*/
}
early_param("cad", cad_setup);
-/* Set up boot command line */
-static void __init append_to_cmdline(size_t (*ipl_data)(char *, size_t))
-{
- char *parm, *delim;
- size_t rc, len;
-
- len = strlen(boot_command_line);
-
- delim = boot_command_line + len; /* '\0' character position */
- parm = boot_command_line + len + 1; /* append right after '\0' */
-
- rc = ipl_data(parm, COMMAND_LINE_SIZE - len - 1);
- if (rc) {
- if (*parm == '=')
- memmove(boot_command_line, parm + 1, rc);
- else
- *delim = ' '; /* replace '\0' with space */
- }
-}
-
-static inline int has_ebcdic_char(const char *str)
-{
- int i;
-
- for (i = 0; str[i]; i++)
- if (str[i] & 0x80)
- return 1;
- return 0;
-}
-
+char __bootdata(early_command_line)[COMMAND_LINE_SIZE];
static void __init setup_boot_command_line(void)
{
- COMMAND_LINE[ARCH_COMMAND_LINE_SIZE - 1] = 0;
- /* convert arch command line to ascii if necessary */
- if (has_ebcdic_char(COMMAND_LINE))
- EBCASC(COMMAND_LINE, ARCH_COMMAND_LINE_SIZE);
/* copy arch command line */
- strlcpy(boot_command_line, strstrip(COMMAND_LINE),
- ARCH_COMMAND_LINE_SIZE);
-
- /* append IPL PARM data to the boot command line */
- if (MACHINE_IS_VM)
- append_to_cmdline(append_ipl_vmparm);
-
- append_to_cmdline(append_ipl_scpdata);
+ strlcpy(boot_command_line, early_command_line, ARCH_COMMAND_LINE_SIZE);
}
static void __init check_image_bootable(void)
#include <linux/string.h>
#include <asm/sections.h>
#include <asm/lowcore.h>
-#include <asm/setup.h>
#include <asm/timex.h>
+#include <asm/kasan.h>
#include "entry.h"
static void __init reset_tod_clock(void)
S390_lowcore.last_update_clock = TOD_UNIX_EPOCH;
}
-static void __init rescue_initrd(void)
-{
- unsigned long min_initrd_addr = (unsigned long) _end + (4UL << 20);
-
- /*
- * Just like in case of IPL from VM reader we make sure there is a
- * gap of 4MB between end of kernel and start of initrd.
- * That way we can also be sure that saving an NSS will succeed,
- * which however only requires different segments.
- */
- if (!IS_ENABLED(CONFIG_BLK_DEV_INITRD))
- return;
- if (!INITRD_START || !INITRD_SIZE)
- return;
- if (INITRD_START >= min_initrd_addr)
- return;
- memmove((void *) min_initrd_addr, (void *) INITRD_START, INITRD_SIZE);
- INITRD_START = min_initrd_addr;
-}
-
static void __init clear_bss_section(void)
{
memset(__bss_start, 0, __bss_stop - __bss_start);
void __init startup_init_nobss(void)
{
reset_tod_clock();
- rescue_initrd();
clear_bss_section();
+ kasan_early_init();
}
static void sclp_early_write(struct console *con, const char *s, unsigned int len)
{
- __sclp_early_printk(s, len);
+ __sclp_early_printk(s, len, 0);
}
static struct console sclp_early_console = {
#endif
.endm
- .macro CHECK_STACK stacksize,savearea
+ .macro CHECK_STACK savearea
#ifdef CONFIG_CHECK_STACK
- tml %r15,\stacksize - CONFIG_STACK_GUARD
+ tml %r15,STACK_SIZE - CONFIG_STACK_GUARD
lghi %r14,\savearea
jz stack_overflow
#endif
.endm
+ .macro CHECK_VMAP_STACK savearea,oklabel
+#ifdef CONFIG_VMAP_STACK
+ lgr %r14,%r15
+ nill %r14,0x10000 - STACK_SIZE
+ oill %r14,STACK_INIT
+ clg %r14,__LC_KERNEL_STACK
+ je \oklabel
+ clg %r14,__LC_ASYNC_STACK
+ je \oklabel
+ clg %r14,__LC_NODAT_STACK
+ je \oklabel
+ clg %r14,__LC_RESTART_STACK
+ je \oklabel
+ lghi %r14,\savearea
+ j stack_overflow
+#else
+ j \oklabel
+#endif
+ .endm
+
.macro SWITCH_ASYNC savearea,timer
tmhh %r8,0x0001 # interrupting from user ?
jnz 1f
brasl %r14,cleanup_critical
tmhh %r8,0x0001 # retest problem state after cleanup
jnz 1f
-0: lg %r14,__LC_ASYNC_STACK # are we already on the async stack?
+0: lg %r14,__LC_ASYNC_STACK # are we already on the target stack?
slgr %r14,%r15
srag %r14,%r14,STACK_SHIFT
jnz 2f
- CHECK_STACK 1<<STACK_SHIFT,\savearea
+ CHECK_STACK \savearea
aghi %r15,-(STACK_FRAME_OVERHEAD + __PT_SIZE)
j 3f
1: UPDATE_VTIME %r14,%r15,\timer
jnz 1f # -> enabled, can't be a double fault
tm __LC_PGM_ILC+3,0x80 # check for per exception
jnz .Lpgm_svcper # -> single stepped svc
-1: CHECK_STACK STACK_SIZE,__LC_SAVE_AREA_SYNC
+1: CHECK_STACK __LC_SAVE_AREA_SYNC
aghi %r15,-(STACK_FRAME_OVERHEAD + __PT_SIZE)
- j 4f
+ # CHECK_VMAP_STACK branches to stack_overflow or 4f
+ CHECK_VMAP_STACK __LC_SAVE_AREA_SYNC,4f
2: UPDATE_VTIME %r14,%r15,__LC_SYNC_ENTER_TIMER
BPENTER __TI_flags(%r12),_TIF_ISOLATE_BP
lg %r15,__LC_KERNEL_STACK
jnz 4f
TSTMSK __LC_MCCK_CODE,MCCK_CODE_PSW_IA_VALID
jno .Lmcck_panic
-4: SWITCH_ASYNC __LC_GPREGS_SAVE_AREA+64,__LC_MCCK_ENTER_TIMER
+4: ssm __LC_PGM_NEW_PSW # turn dat on, keep irqs off
+ SWITCH_ASYNC __LC_GPREGS_SAVE_AREA+64,__LC_MCCK_ENTER_TIMER
.Lmcck_skip:
lghi %r14,__LC_GPREGS_SAVE_AREA+64
stmg %r0,%r7,__PT_R0(%r11)
xc __SF_BACKCHAIN(8,%r1),__SF_BACKCHAIN(%r1)
la %r11,STACK_FRAME_OVERHEAD(%r1)
lgr %r15,%r1
- ssm __LC_PGM_NEW_PSW # turn dat on, keep irqs off
TSTMSK __LC_CPU_FLAGS,_CIF_MCCK_PENDING
jno .Lmcck_return
TRACE_IRQS_OFF
lpswe __LC_RETURN_MCCK_PSW
.Lmcck_panic:
- lg %r15,__LC_PANIC_STACK
+ lg %r15,__LC_NODAT_STACK
la %r11,STACK_FRAME_OVERHEAD(%r15)
j .Lmcck_skip
ALTERNATIVE "", ".insn s,0xb2800000,_LPP_OFFSET", 40
stg %r15,__LC_SAVE_AREA_RESTART
lg %r15,__LC_RESTART_STACK
- aghi %r15,-__PT_SIZE # create pt_regs on stack
- xc 0(__PT_SIZE,%r15),0(%r15)
- stmg %r0,%r14,__PT_R0(%r15)
- mvc __PT_R15(8,%r15),__LC_SAVE_AREA_RESTART
- mvc __PT_PSW(16,%r15),__LC_RST_OLD_PSW # store restart old psw
- aghi %r15,-STACK_FRAME_OVERHEAD # create stack frame on stack
+ xc STACK_FRAME_OVERHEAD(__PT_SIZE,%r15),STACK_FRAME_OVERHEAD(%r15)
+ stmg %r0,%r14,STACK_FRAME_OVERHEAD+__PT_R0(%r15)
+ mvc STACK_FRAME_OVERHEAD+__PT_R15(8,%r15),__LC_SAVE_AREA_RESTART
+ mvc STACK_FRAME_OVERHEAD+__PT_PSW(16,%r15),__LC_RST_OLD_PSW
xc 0(STACK_FRAME_OVERHEAD,%r15),0(%r15)
lg %r1,__LC_RESTART_FN # load fn, parm & source cpu
lg %r2,__LC_RESTART_DATA
.section .kprobes.text, "ax"
-#ifdef CONFIG_CHECK_STACK
+#if defined(CONFIG_CHECK_STACK) || defined(CONFIG_VMAP_STACK)
/*
* The synchronous or the asynchronous stack overflowed. We are dead.
* No need to properly save the registers, we are going to panic anyway.
* Setup a pt_regs so that show_trace can provide a good call trace.
*/
stack_overflow:
- lg %r15,__LC_PANIC_STACK # change to panic stack
+ lg %r15,__LC_NODAT_STACK # change to panic stack
la %r11,STACK_FRAME_OVERHEAD(%r15)
stmg %r0,%r7,__PT_R0(%r11)
stmg %r8,%r9,__PT_PSW(%r11)
void gs_load_bc_cb(struct pt_regs *regs);
void set_fs_fixup(void);
+unsigned long stack_alloc(void);
+void stack_free(unsigned long stack);
+
#endif /* _ENTRY_H */
#include <asm/asm-offsets.h>
#include <asm/thread_info.h>
#include <asm/page.h>
+#include <asm/ptrace.h>
__HEAD
ENTRY(startup_continue)
#
larl %r14,init_task
stg %r14,__LC_CURRENT
- larl %r15,init_thread_union
- aghi %r15,1<<(PAGE_SHIFT+THREAD_SIZE_ORDER) # init_task_union + THREAD_SIZE
- stg %r15,__LC_KERNEL_STACK # set end of kernel stack
- aghi %r15,-160
+ larl %r15,init_thread_union+THREAD_SIZE-STACK_FRAME_OVERHEAD
#
# Early setup functions that may not rely on an initialized bss section,
# like moving the initrd. Returns with an initialized bss section.
#include <asm/checksum.h>
#include <asm/debug.h>
#include <asm/os_info.h>
+#include <asm/sections.h>
+#include <asm/boot_data.h>
#include "entry.h"
#define IPL_PARM_BLOCK_VERSION 0
}
}
+struct ipl_parameter_block __bootdata(early_ipl_block);
+int __bootdata(early_ipl_block_valid);
+
static int ipl_block_valid;
static struct ipl_parameter_block ipl_block;
int diag308(unsigned long subcode, void *addr)
{
+ if (IS_ENABLED(CONFIG_KASAN))
+ __arch_local_irq_stosm(0x04); /* enable DAT */
diag_stat_inc(DIAG_STAT_X308);
return __diag308(subcode, addr);
}
static struct kobj_attribute sys_ipl_type_attr = __ATTR_RO(ipl_type);
-/* VM IPL PARM routines */
-static size_t reipl_get_ascii_vmparm(char *dest, size_t size,
- const struct ipl_parameter_block *ipb)
-{
- int i;
- size_t len;
- char has_lowercase = 0;
-
- len = 0;
- if ((ipb->ipl_info.ccw.vm_flags & DIAG308_VM_FLAGS_VP_VALID) &&
- (ipb->ipl_info.ccw.vm_parm_len > 0)) {
-
- len = min_t(size_t, size - 1, ipb->ipl_info.ccw.vm_parm_len);
- memcpy(dest, ipb->ipl_info.ccw.vm_parm, len);
- /* If at least one character is lowercase, we assume mixed
- * case; otherwise we convert everything to lowercase.
- */
- for (i = 0; i < len; i++)
- if ((dest[i] > 0x80 && dest[i] < 0x8a) || /* a-i */
- (dest[i] > 0x90 && dest[i] < 0x9a) || /* j-r */
- (dest[i] > 0xa1 && dest[i] < 0xaa)) { /* s-z */
- has_lowercase = 1;
- break;
- }
- if (!has_lowercase)
- EBC_TOLOWER(dest, len);
- EBCASC(dest, len);
- }
- dest[len] = 0;
-
- return len;
-}
-
-size_t append_ipl_vmparm(char *dest, size_t size)
-{
- size_t rc;
-
- rc = 0;
- if (ipl_block_valid && ipl_block.hdr.pbt == DIAG308_IPL_TYPE_CCW)
- rc = reipl_get_ascii_vmparm(dest, size, &ipl_block);
- else
- dest[0] = 0;
- return rc;
-}
-
static ssize_t ipl_vm_parm_show(struct kobject *kobj,
struct kobj_attribute *attr, char *page)
{
char parm[DIAG308_VMPARM_SIZE + 1] = {};
- append_ipl_vmparm(parm, sizeof(parm));
+ if (ipl_block_valid && (ipl_block.hdr.pbt == DIAG308_IPL_TYPE_CCW))
+ ipl_block_get_ascii_vmparm(parm, sizeof(parm), &ipl_block);
return sprintf(page, "%s\n", parm);
}
-static size_t scpdata_length(const char* buf, size_t count)
-{
- while (count) {
- if (buf[count - 1] != '\0' && buf[count - 1] != ' ')
- break;
- count--;
- }
- return count;
-}
-
-static size_t reipl_append_ascii_scpdata(char *dest, size_t size,
- const struct ipl_parameter_block *ipb)
-{
- size_t count;
- size_t i;
- int has_lowercase;
-
- count = min(size - 1, scpdata_length(ipb->ipl_info.fcp.scp_data,
- ipb->ipl_info.fcp.scp_data_len));
- if (!count)
- goto out;
-
- has_lowercase = 0;
- for (i = 0; i < count; i++) {
- if (!isascii(ipb->ipl_info.fcp.scp_data[i])) {
- count = 0;
- goto out;
- }
- if (!has_lowercase && islower(ipb->ipl_info.fcp.scp_data[i]))
- has_lowercase = 1;
- }
-
- if (has_lowercase)
- memcpy(dest, ipb->ipl_info.fcp.scp_data, count);
- else
- for (i = 0; i < count; i++)
- dest[i] = tolower(ipb->ipl_info.fcp.scp_data[i]);
-out:
- dest[count] = '\0';
- return count;
-}
-
-size_t append_ipl_scpdata(char *dest, size_t len)
-{
- size_t rc;
-
- rc = 0;
- if (ipl_block_valid && ipl_block.hdr.pbt == DIAG308_IPL_TYPE_FCP)
- rc = reipl_append_ascii_scpdata(dest, len, &ipl_block);
- else
- dest[0] = 0;
- return rc;
-}
-
-
static struct kobj_attribute sys_ipl_vm_parm_attr =
__ATTR(parm, S_IRUGO, ipl_vm_parm_show, NULL);
{
char vmparm[DIAG308_VMPARM_SIZE + 1] = {};
- reipl_get_ascii_vmparm(vmparm, sizeof(vmparm), ipb);
+ ipl_block_get_ascii_vmparm(vmparm, sizeof(vmparm), ipb);
return sprintf(page, "%s\n", vmparm);
}
void __init ipl_store_parameters(void)
{
- int rc;
-
- rc = diag308(DIAG308_STORE, &ipl_block);
- if (rc == DIAG308_RC_OK && ipl_block.hdr.version <= IPL_MAX_SUPPORTED_VERSION)
+ if (early_ipl_block_valid) {
+ memcpy(&ipl_block, &early_ipl_block, sizeof(ipl_block));
ipl_block_valid = 1;
+ }
}
void s390_reset_system(void)
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <asm/ebcdic.h>
+#include <asm/ipl.h>
+
+/* VM IPL PARM routines */
+size_t ipl_block_get_ascii_vmparm(char *dest, size_t size,
+ const struct ipl_parameter_block *ipb)
+{
+ int i;
+ size_t len;
+ char has_lowercase = 0;
+
+ len = 0;
+ if ((ipb->ipl_info.ccw.vm_flags & DIAG308_VM_FLAGS_VP_VALID) &&
+ (ipb->ipl_info.ccw.vm_parm_len > 0)) {
+
+ len = min_t(size_t, size - 1, ipb->ipl_info.ccw.vm_parm_len);
+ memcpy(dest, ipb->ipl_info.ccw.vm_parm, len);
+ /* If at least one character is lowercase, we assume mixed
+ * case; otherwise we convert everything to lowercase.
+ */
+ for (i = 0; i < len; i++)
+ if ((dest[i] > 0x80 && dest[i] < 0x8a) || /* a-i */
+ (dest[i] > 0x90 && dest[i] < 0x9a) || /* j-r */
+ (dest[i] > 0xa1 && dest[i] < 0xaa)) { /* s-z */
+ has_lowercase = 1;
+ break;
+ }
+ if (!has_lowercase)
+ EBC_TOLOWER(dest, len);
+ EBCASC(dest, len);
+ }
+ dest[len] = 0;
+
+ return len;
+}
/* Check against async. stack address range. */
new = S390_lowcore.async_stack;
if (((new - old) >> (PAGE_SHIFT + THREAD_SIZE_ORDER)) != 0) {
- /* Need to switch to the async. stack. */
- new -= STACK_FRAME_OVERHEAD;
- ((struct stack_frame *) new)->back_chain = old;
- asm volatile(" la 15,0(%0)\n"
- " brasl 14,__do_softirq\n"
- " la 15,0(%1)\n"
- : : "a" (new), "a" (old)
- : "0", "1", "2", "3", "4", "5", "14",
- "cc", "memory" );
+ CALL_ON_STACK(__do_softirq, new, 0);
} else {
/* We are already on the async stack. */
__do_softirq();
{
/* brcl 15,offset */
insn->opcode = 0xc0f4;
- insn->offset = (entry->target - entry->code) >> 1;
+ insn->offset = (jump_entry_target(entry) - jump_entry_code(entry)) >> 1;
}
static void jump_label_bug(struct jump_entry *entry, struct insn *expected,
struct insn *new)
{
- unsigned char *ipc = (unsigned char *)entry->code;
+ unsigned char *ipc = (unsigned char *)jump_entry_code(entry);
unsigned char *ipe = (unsigned char *)expected;
unsigned char *ipn = (unsigned char *)new;
enum jump_label_type type,
int init)
{
+ void *code = (void *)jump_entry_code(entry);
struct insn old, new;
if (type == JUMP_LABEL_JMP) {
jump_label_make_nop(entry, &new);
}
if (init) {
- if (memcmp((void *)entry->code, &orignop, sizeof(orignop)))
+ if (memcmp(code, &orignop, sizeof(orignop)))
jump_label_bug(entry, &orignop, &new);
} else {
- if (memcmp((void *)entry->code, &old, sizeof(old)))
+ if (memcmp(code, &old, sizeof(old)))
jump_label_bug(entry, &old, &new);
}
- s390_kernel_write((void *)entry->code, &new, sizeof(new));
+ s390_kernel_write(code, &new, sizeof(new));
}
static int __sm_arch_jump_label_transform(void *data)
}
#endif
+static unsigned long do_start_kdump(unsigned long addr)
+{
+ struct kimage *image = (struct kimage *) addr;
+ int (*start_kdump)(int) = (void *)image->start;
+ int rc;
+
+ __arch_local_irq_stnsm(0xfb); /* disable DAT */
+ rc = start_kdump(0);
+ __arch_local_irq_stosm(0x04); /* enable DAT */
+ return rc;
+}
+
/*
* Check if kdump checksums are valid: We call purgatory with parameter "0"
*/
static bool kdump_csum_valid(struct kimage *image)
{
#ifdef CONFIG_CRASH_DUMP
- int (*start_kdump)(int) = (void *)image->start;
int rc;
- __arch_local_irq_stnsm(0xfb); /* disable DAT */
- rc = start_kdump(0);
- __arch_local_irq_stosm(0x04); /* enable DAT */
+ rc = CALL_ON_STACK(do_start_kdump, S390_lowcore.nodat_stack, 1, image);
return rc == 0;
#else
return false;
#include <linux/fs.h>
#include <linux/string.h>
#include <linux/kernel.h>
+#include <linux/kasan.h>
#include <linux/moduleloader.h>
#include <linux/bug.h>
#include <asm/alternative.h>
void *module_alloc(unsigned long size)
{
+ void *p;
+
if (PAGE_ALIGN(size) > MODULES_LEN)
return NULL;
- return __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END,
- GFP_KERNEL, PAGE_KERNEL_EXEC,
- 0, NUMA_NO_NODE,
- __builtin_return_address(0));
+ p = __vmalloc_node_range(size, MODULE_ALIGN, MODULES_VADDR, MODULES_END,
+ GFP_KERNEL, PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE,
+ __builtin_return_address(0));
+ if (p && (kasan_module_alloc(p, size) < 0)) {
+ vfree(p);
+ return NULL;
+ }
+ return p;
}
void module_arch_freeing_init(struct module *mod)
}
sfdbg = debug_register(KMSG_COMPONENT, 2, 1, 80);
- if (!sfdbg)
+ if (!sfdbg) {
pr_err("Registering for s390dbf failed\n");
+ return -ENOMEM;
+ }
debug_register_view(sfdbg, &debug_sprintf_view);
err = register_external_irq(EXT_IRQ_MEASURE_ALERT,
cpumf_measurement_alert);
if (err) {
pr_cpumsf_err(RS_INIT_FAILURE_ALRT);
+ debug_unregister(sfdbg);
goto out;
}
pr_cpumsf_err(RS_INIT_FAILURE_PERF);
unregister_external_irq(EXT_IRQ_MEASURE_ALERT,
cpumf_measurement_alert);
+ debug_unregister(sfdbg);
goto out;
}
#include <linux/crash_dump.h>
#include <linux/memory.h>
#include <linux/compat.h>
+#include <linux/start_kernel.h>
#include <asm/ipl.h>
#include <asm/facility.h>
#include <asm/numa.h>
#include <asm/alternative.h>
#include <asm/nospec-branch.h>
+#include <asm/mem_detect.h>
#include "entry.h"
/*
unsigned long int_hwcap = 0;
-int __initdata memory_end_set;
-unsigned long __initdata memory_end;
-unsigned long __initdata max_physmem_end;
+int __bootdata(noexec_disabled);
+int __bootdata(memory_end_set);
+unsigned long __bootdata(memory_end);
+unsigned long __bootdata(max_physmem_end);
+struct mem_detect_info __bootdata(mem_detect);
unsigned long VMALLOC_START;
EXPORT_SYMBOL(VMALLOC_START);
void (*pm_power_off)(void) = machine_power_off;
EXPORT_SYMBOL_GPL(pm_power_off);
-static int __init early_parse_mem(char *p)
-{
- memory_end = memparse(p, &p);
- memory_end &= PAGE_MASK;
- memory_end_set = 1;
- return 0;
-}
-early_param("mem", early_parse_mem);
-
static int __init parse_vmalloc(char *arg)
{
if (!arg)
void *restart_stack __section(.data);
+unsigned long stack_alloc(void)
+{
+#ifdef CONFIG_VMAP_STACK
+ return (unsigned long)
+ __vmalloc_node_range(THREAD_SIZE, THREAD_SIZE,
+ VMALLOC_START, VMALLOC_END,
+ THREADINFO_GFP,
+ PAGE_KERNEL, 0, NUMA_NO_NODE,
+ __builtin_return_address(0));
+#else
+ return __get_free_pages(GFP_KERNEL, THREAD_SIZE_ORDER);
+#endif
+}
+
+void stack_free(unsigned long stack)
+{
+#ifdef CONFIG_VMAP_STACK
+ vfree((void *) stack);
+#else
+ free_pages(stack, THREAD_SIZE_ORDER);
+#endif
+}
+
+int __init arch_early_irq_init(void)
+{
+ unsigned long stack;
+
+ stack = __get_free_pages(GFP_KERNEL, THREAD_SIZE_ORDER);
+ if (!stack)
+ panic("Couldn't allocate async stack");
+ S390_lowcore.async_stack = stack + STACK_INIT_OFFSET;
+ return 0;
+}
+
+static int __init async_stack_realloc(void)
+{
+ unsigned long old, new;
+
+ old = S390_lowcore.async_stack - STACK_INIT_OFFSET;
+ new = stack_alloc();
+ if (!new)
+ panic("Couldn't allocate async stack");
+ S390_lowcore.async_stack = new + STACK_INIT_OFFSET;
+ free_pages(old, THREAD_SIZE_ORDER);
+ return 0;
+}
+early_initcall(async_stack_realloc);
+
+void __init arch_call_rest_init(void)
+{
+ struct stack_frame *frame;
+ unsigned long stack;
+
+ stack = stack_alloc();
+ if (!stack)
+ panic("Couldn't allocate kernel stack");
+ current->stack = (void *) stack;
+#ifdef CONFIG_VMAP_STACK
+ current->stack_vm_area = (void *) stack;
+#endif
+ set_task_stack_end_magic(current);
+ stack += STACK_INIT_OFFSET;
+ S390_lowcore.kernel_stack = stack;
+ frame = (struct stack_frame *) stack;
+ memset(frame, 0, sizeof(*frame));
+ /* Branch to rest_init on the new stack, never returns */
+ asm volatile(
+ " la 15,0(%[_frame])\n"
+ " jg rest_init\n"
+ : : [_frame] "a" (frame));
+}
+
static void __init setup_lowcore(void)
{
struct lowcore *lc;
PSW_MASK_DAT | PSW_MASK_MCHECK;
lc->io_new_psw.addr = (unsigned long) io_int_handler;
lc->clock_comparator = clock_comparator_max;
- lc->kernel_stack = ((unsigned long) &init_thread_union)
+ lc->nodat_stack = ((unsigned long) &init_thread_union)
+ THREAD_SIZE - STACK_FRAME_OVERHEAD - sizeof(struct pt_regs);
- lc->async_stack = (unsigned long)
- memblock_virt_alloc(ASYNC_SIZE, ASYNC_SIZE)
- + ASYNC_SIZE - STACK_FRAME_OVERHEAD - sizeof(struct pt_regs);
- lc->panic_stack = (unsigned long)
- memblock_virt_alloc(PAGE_SIZE, PAGE_SIZE)
- + PAGE_SIZE - STACK_FRAME_OVERHEAD - sizeof(struct pt_regs);
lc->current_task = (unsigned long)&init_task;
lc->lpp = LPP_MAGIC;
lc->machine_flags = S390_lowcore.machine_flags;
lc->last_update_timer = S390_lowcore.last_update_timer;
lc->last_update_clock = S390_lowcore.last_update_clock;
- restart_stack = memblock_virt_alloc(ASYNC_SIZE, ASYNC_SIZE);
- restart_stack += ASYNC_SIZE;
+ /*
+ * Allocate the global restart stack which is the same for
+ * all CPUs in cast *one* of them does a PSW restart.
+ */
+ restart_stack = memblock_virt_alloc(THREAD_SIZE, THREAD_SIZE);
+ restart_stack += STACK_INIT_OFFSET;
/*
* Set up PSW restart to call ipl.c:do_restart(). Copy the relevant
{
unsigned long vmax, vmalloc_size, tmp;
- /* Choose kernel address space layout: 2, 3, or 4 levels. */
+ /* Choose kernel address space layout: 3 or 4 levels. */
vmalloc_size = VMALLOC_END ?: (128UL << 30) - MODULES_LEN;
- tmp = (memory_end ?: max_physmem_end) / PAGE_SIZE;
- tmp = tmp * (sizeof(struct page) + PAGE_SIZE);
- if (tmp + vmalloc_size + MODULES_LEN <= _REGION2_SIZE)
- vmax = _REGION2_SIZE; /* 3-level kernel page table */
- else
- vmax = _REGION1_SIZE; /* 4-level kernel page table */
+ if (IS_ENABLED(CONFIG_KASAN)) {
+ vmax = IS_ENABLED(CONFIG_KASAN_S390_4_LEVEL_PAGING)
+ ? _REGION1_SIZE
+ : _REGION2_SIZE;
+ } else {
+ tmp = (memory_end ?: max_physmem_end) / PAGE_SIZE;
+ tmp = tmp * (sizeof(struct page) + PAGE_SIZE);
+ if (tmp + vmalloc_size + MODULES_LEN <= _REGION2_SIZE)
+ vmax = _REGION2_SIZE; /* 3-level kernel page table */
+ else
+ vmax = _REGION1_SIZE; /* 4-level kernel page table */
+ }
+
/* module area is at the end of the kernel address space. */
MODULES_END = vmax;
MODULES_VADDR = MODULES_END - MODULES_LEN;
VMALLOC_END = MODULES_VADDR;
- VMALLOC_START = vmax - vmalloc_size;
+ VMALLOC_START = VMALLOC_END - vmalloc_size;
/* Split remaining virtual space between 1:1 mapping & vmemmap array */
tmp = VMALLOC_START / (PAGE_SIZE + sizeof(struct page));
vmemmap = (struct page *) tmp;
/* Take care that memory_end is set and <= vmemmap */
- memory_end = min(memory_end ?: max_physmem_end, tmp);
+ memory_end = min(memory_end ?: max_physmem_end, (unsigned long)vmemmap);
+#ifdef CONFIG_KASAN
+ /* fit in kasan shadow memory region between 1:1 and vmemmap */
+ memory_end = min(memory_end, KASAN_SHADOW_START);
+ vmemmap = max(vmemmap, (struct page *)KASAN_SHADOW_END);
+#endif
max_pfn = max_low_pfn = PFN_DOWN(memory_end);
memblock_remove(memory_end, ULONG_MAX);
*/
static void reserve_memory_end(void)
{
-#ifdef CONFIG_CRASH_DUMP
- if (ipl_info.type == IPL_TYPE_FCP_DUMP &&
- !OLDMEM_BASE && sclp.hsa_size) {
- memory_end = sclp.hsa_size;
- memory_end &= PAGE_MASK;
- memory_end_set = 1;
- }
-#endif
- if (!memory_end_set)
- return;
- memblock_reserve(memory_end, ULONG_MAX);
+ if (memory_end_set)
+ memblock_reserve(memory_end, ULONG_MAX);
}
/*
#endif
}
+static void __init reserve_mem_detect_info(void)
+{
+ unsigned long start, size;
+
+ get_mem_detect_reserved(&start, &size);
+ if (size)
+ memblock_reserve(start, size);
+}
+
+static void __init free_mem_detect_info(void)
+{
+ unsigned long start, size;
+
+ get_mem_detect_reserved(&start, &size);
+ if (size)
+ memblock_free(start, size);
+}
+
+static void __init memblock_physmem_add(phys_addr_t start, phys_addr_t size)
+{
+ memblock_dbg("memblock_physmem_add: [%#016llx-%#016llx]\n",
+ start, start + size - 1);
+ memblock_add_range(&memblock.memory, start, size, 0, 0);
+ memblock_add_range(&memblock.physmem, start, size, 0, 0);
+}
+
+static const char * __init get_mem_info_source(void)
+{
+ switch (mem_detect.info_source) {
+ case MEM_DETECT_SCLP_STOR_INFO:
+ return "sclp storage info";
+ case MEM_DETECT_DIAG260:
+ return "diag260";
+ case MEM_DETECT_SCLP_READ_INFO:
+ return "sclp read info";
+ case MEM_DETECT_BIN_SEARCH:
+ return "binary search";
+ }
+ return "none";
+}
+
+static void __init memblock_add_mem_detect_info(void)
+{
+ unsigned long start, end;
+ int i;
+
+ memblock_dbg("physmem info source: %s (%hhd)\n",
+ get_mem_info_source(), mem_detect.info_source);
+ /* keep memblock lists close to the kernel */
+ memblock_set_bottom_up(true);
+ for_each_mem_detect_block(i, &start, &end)
+ memblock_physmem_add(start, end - start);
+ memblock_set_bottom_up(false);
+ memblock_dump_all();
+}
+
/*
* Check for initrd being in usable memory
*/
reserve_oldmem();
reserve_kernel();
reserve_initrd();
+ reserve_mem_detect_info();
memblock_allow_resize();
/* Get information about *all* installed memory */
- detect_memory_memblock();
+ memblock_add_mem_detect_info();
+ free_mem_detect_info();
remove_oldmem();
/*
pcpu_sigp_retry(pcpu, order, 0);
}
-#define ASYNC_FRAME_OFFSET (ASYNC_SIZE - STACK_FRAME_OVERHEAD - __PT_SIZE)
-#define PANIC_FRAME_OFFSET (PAGE_SIZE - STACK_FRAME_OVERHEAD - __PT_SIZE)
-
static int pcpu_alloc_lowcore(struct pcpu *pcpu, int cpu)
{
- unsigned long async_stack, panic_stack;
+ unsigned long async_stack, nodat_stack;
struct lowcore *lc;
if (pcpu != &pcpu_devices[0]) {
pcpu->lowcore = (struct lowcore *)
__get_free_pages(GFP_KERNEL | GFP_DMA, LC_ORDER);
- async_stack = __get_free_pages(GFP_KERNEL, ASYNC_ORDER);
- panic_stack = __get_free_page(GFP_KERNEL);
- if (!pcpu->lowcore || !panic_stack || !async_stack)
+ nodat_stack = __get_free_pages(GFP_KERNEL, THREAD_SIZE_ORDER);
+ if (!pcpu->lowcore || !nodat_stack)
goto out;
} else {
- async_stack = pcpu->lowcore->async_stack - ASYNC_FRAME_OFFSET;
- panic_stack = pcpu->lowcore->panic_stack - PANIC_FRAME_OFFSET;
+ nodat_stack = pcpu->lowcore->nodat_stack - STACK_INIT_OFFSET;
}
+ async_stack = stack_alloc();
+ if (!async_stack)
+ goto out;
lc = pcpu->lowcore;
memcpy(lc, &S390_lowcore, 512);
memset((char *) lc + 512, 0, sizeof(*lc) - 512);
- lc->async_stack = async_stack + ASYNC_FRAME_OFFSET;
- lc->panic_stack = panic_stack + PANIC_FRAME_OFFSET;
+ lc->async_stack = async_stack + STACK_INIT_OFFSET;
+ lc->nodat_stack = nodat_stack + STACK_INIT_OFFSET;
lc->cpu_nr = cpu;
lc->spinlock_lockval = arch_spin_lockval(cpu);
lc->spinlock_index = 0;
lc->br_r1_trampoline = 0x07f1; /* br %r1 */
if (nmi_alloc_per_cpu(lc))
- goto out;
+ goto out_async;
if (vdso_alloc_per_cpu(lc))
goto out_mcesa;
lowcore_ptr[cpu] = lc;
out_mcesa:
nmi_free_per_cpu(lc);
+out_async:
+ stack_free(async_stack);
out:
if (pcpu != &pcpu_devices[0]) {
- free_page(panic_stack);
- free_pages(async_stack, ASYNC_ORDER);
+ free_pages(nodat_stack, THREAD_SIZE_ORDER);
free_pages((unsigned long) pcpu->lowcore, LC_ORDER);
}
return -ENOMEM;
static void pcpu_free_lowcore(struct pcpu *pcpu)
{
+ unsigned long async_stack, nodat_stack, lowcore;
+
+ nodat_stack = pcpu->lowcore->nodat_stack - STACK_INIT_OFFSET;
+ async_stack = pcpu->lowcore->async_stack - STACK_INIT_OFFSET;
+ lowcore = (unsigned long) pcpu->lowcore;
+
pcpu_sigp_retry(pcpu, SIGP_SET_PREFIX, 0);
lowcore_ptr[pcpu - pcpu_devices] = NULL;
vdso_free_per_cpu(pcpu->lowcore);
nmi_free_per_cpu(pcpu->lowcore);
+ stack_free(async_stack);
if (pcpu == &pcpu_devices[0])
return;
- free_page(pcpu->lowcore->panic_stack-PANIC_FRAME_OFFSET);
- free_pages(pcpu->lowcore->async_stack-ASYNC_FRAME_OFFSET, ASYNC_ORDER);
- free_pages((unsigned long) pcpu->lowcore, LC_ORDER);
+ free_pages(nodat_stack, THREAD_SIZE_ORDER);
+ free_pages(lowcore, LC_ORDER);
}
#endif /* CONFIG_HOTPLUG_CPU */
{
struct lowcore *lc = pcpu->lowcore;
- lc->restart_stack = lc->kernel_stack;
+ lc->restart_stack = lc->nodat_stack;
lc->restart_fn = (unsigned long) func;
lc->restart_data = (unsigned long) data;
lc->restart_source = -1UL;
/*
* Call function via PSW restart on pcpu and stop the current cpu.
*/
-static void pcpu_delegate(struct pcpu *pcpu, void (*func)(void *),
- void *data, unsigned long stack)
+static void __pcpu_delegate(void (*func)(void*), void *data)
+{
+ func(data); /* should not return */
+}
+
+static void __no_sanitize_address pcpu_delegate(struct pcpu *pcpu,
+ void (*func)(void *),
+ void *data, unsigned long stack)
{
struct lowcore *lc = lowcore_ptr[pcpu - pcpu_devices];
unsigned long source_cpu = stap();
- __load_psw_mask(PSW_KERNEL_BITS);
+ __load_psw_mask(PSW_KERNEL_BITS | PSW_MASK_DAT);
if (pcpu->address == source_cpu)
- func(data); /* should not return */
+ CALL_ON_STACK(__pcpu_delegate, stack, 2, func, data);
/* Stop target cpu (if func returns this stops the current cpu). */
pcpu_sigp_retry(pcpu, SIGP_STOP, 0);
/* Restart func on the target cpu and stop the current cpu. */
void smp_call_ipl_cpu(void (*func)(void *), void *data)
{
pcpu_delegate(&pcpu_devices[0], func, data,
- pcpu_devices->lowcore->panic_stack -
- PANIC_FRAME_OFFSET + PAGE_SIZE);
+ pcpu_devices->lowcore->nodat_stack);
}
int smp_find_processor_id(u16 address)
memblock_free_early((unsigned long)info, sizeof(*info));
}
-/*
- * Activate a secondary processor.
- */
-static void smp_start_secondary(void *cpuvoid)
+static void smp_init_secondary(void)
{
int cpu = smp_processor_id();
S390_lowcore.last_update_clock = get_tod_clock();
- S390_lowcore.restart_stack = (unsigned long) restart_stack;
- S390_lowcore.restart_fn = (unsigned long) do_restart;
- S390_lowcore.restart_data = 0;
- S390_lowcore.restart_source = -1UL;
restore_access_regs(S390_lowcore.access_regs_save_area);
- __ctl_load(S390_lowcore.cregs_save_area, 0, 15);
- __load_psw_mask(PSW_KERNEL_BITS | PSW_MASK_DAT);
cpu_init();
preempt_disable();
init_cpu_timer();
vtime_init();
pfault_init();
- notify_cpu_starting(cpu);
+ notify_cpu_starting(smp_processor_id());
if (topology_cpu_dedicated(cpu))
set_cpu_flag(CIF_DEDICATED_CPU);
else
clear_cpu_flag(CIF_DEDICATED_CPU);
- set_cpu_online(cpu, true);
+ set_cpu_online(smp_processor_id(), true);
inc_irq_stat(CPU_RST);
local_irq_enable();
cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
}
+/*
+ * Activate a secondary processor.
+ */
+static void __no_sanitize_address smp_start_secondary(void *cpuvoid)
+{
+ S390_lowcore.restart_stack = (unsigned long) restart_stack;
+ S390_lowcore.restart_fn = (unsigned long) do_restart;
+ S390_lowcore.restart_data = 0;
+ S390_lowcore.restart_source = -1UL;
+ __ctl_load(S390_lowcore.cregs_save_area, 0, 15);
+ __load_psw_mask(PSW_KERNEL_BITS | PSW_MASK_DAT);
+ CALL_ON_STACK(smp_init_secondary, S390_lowcore.kernel_stack, 0);
+}
+
/* Upping and downing of CPUs */
int __cpu_up(unsigned int cpu, struct task_struct *tidle)
{
static void fill_stsi_mac(struct sthyi_sctns *sctns,
struct sysinfo_1_1_1 *sysinfo)
{
+ sclp_ocf_cpc_name_copy(sctns->mac.infmname);
+ if (*(u64 *)sctns->mac.infmname != 0)
+ sctns->mac.infmval1 |= MAC_NAME_VLD;
+
if (stsi(sysinfo, 1, 1, 1))
return;
- sclp_ocf_cpc_name_copy(sctns->mac.infmname);
-
memcpy(sctns->mac.infmtype, sysinfo->type, sizeof(sctns->mac.infmtype));
memcpy(sctns->mac.infmmanu, sysinfo->manufacturer, sizeof(sctns->mac.infmmanu));
memcpy(sctns->mac.infmpman, sysinfo->plant, sizeof(sctns->mac.infmpman));
memcpy(sctns->mac.infmseq, sysinfo->sequence, sizeof(sctns->mac.infmseq));
- sctns->mac.infmval1 |= MAC_ID_VLD | MAC_NAME_VLD;
+ sctns->mac.infmval1 |= MAC_ID_VLD;
}
static void fill_stsi_par(struct sthyi_sctns *sctns,
.section .text
ENTRY(swsusp_arch_suspend)
- stmg %r6,%r15,__SF_GPRS(%r15)
+ lg %r1,__LC_NODAT_STACK
+ aghi %r1,-STACK_FRAME_OVERHEAD
+ stmg %r6,%r15,__SF_GPRS(%r1)
+ stg %r15,__SF_BACKCHAIN(%r1)
lgr %r1,%r15
- aghi %r15,-STACK_FRAME_OVERHEAD
- stg %r1,__SF_BACKCHAIN(%r15)
/* Store FPU registers */
brasl %r14,save_fpu_regs
brc 2,3b /* busy, try again */
/* Suspend CPU not available -> panic */
- larl %r15,init_thread_union
- ahi %r15,1<<(PAGE_SHIFT+THREAD_SIZE_ORDER)
+ larl %r15,init_thread_union+THREAD_SIZE-STACK_FRAME_OVERHEAD
larl %r2,.Lpanic_string
- lghi %r1,0
- sam31
- sigp %r1,%r0,SIGP_SET_ARCHITECTURE
- brasl %r14,sclp_early_printk
+ brasl %r14,sclp_early_printk_force
larl %r3,.Ldisabled_wait_31
lpsw 0(%r3)
4:
vdso_pagelist = vdso64_pagelist;
vdso_pages = vdso64_pages;
#ifdef CONFIG_COMPAT
- if (is_compat_task()) {
+ if (vma->vm_mm->context.compat_mm) {
vdso_pagelist = vdso32_pagelist;
vdso_pages = vdso32_pages;
}
vdso_pages = vdso64_pages;
#ifdef CONFIG_COMPAT
- if (is_compat_task())
+ if (vma->vm_mm->context.compat_mm)
vdso_pages = vdso32_pages;
#endif
vdso_pages = vdso64_pages;
#ifdef CONFIG_COMPAT
- if (is_compat_task())
+ if (is_compat_task()) {
vdso_pages = vdso32_pages;
+ mm->context.compat_mm = 1;
+ }
#endif
/*
* vDSO has a problem and was disabled, just don't "enable" it for
extra-y += vdso32.lds
CPPFLAGS_vdso32.lds += -P -C -U$(ARCH)
-# Disable gcov profiling and ubsan for VDSO code
+# Disable gcov profiling, ubsan and kasan for VDSO code
GCOV_PROFILE := n
UBSAN_SANITIZE := n
+KASAN_SANITIZE := n
# Force dependency (incbin is bad)
$(obj)/vdso32_wrapper.o : $(obj)/vdso32.so
#include <asm/asm-offsets.h>
#include <asm/unistd.h>
#include <asm/dwarf.h>
+#include <asm/ptrace.h>
.text
.align 4
__kernel_clock_gettime:
CFI_STARTPROC
ahi %r15,-16
- CFI_DEF_CFA_OFFSET 176
- CFI_VAL_OFFSET 15, -160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD+16
+ CFI_VAL_OFFSET 15, -STACK_FRAME_OVERHEAD
basr %r5,0
0: al %r5,21f-0b(%r5) /* get &_vdso_data */
chi %r2,__CLOCK_REALTIME_COARSE
st %r1,4(%r3) /* store tp->tv_nsec */
lhi %r2,0
ahi %r15,16
- CFI_DEF_CFA_OFFSET 160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD
CFI_RESTORE 15
br %r14
/* CLOCK_MONOTONIC_COARSE */
- CFI_DEF_CFA_OFFSET 176
- CFI_VAL_OFFSET 15, -160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD+16
+ CFI_VAL_OFFSET 15, -STACK_FRAME_OVERHEAD
9: l %r4,__VDSO_UPD_COUNT+4(%r5) /* load update counter */
tml %r4,0x0001 /* pending update ? loop */
jnz 9b
st %r1,4(%r3) /* store tp->tv_nsec */
lhi %r2,0
ahi %r15,16
- CFI_DEF_CFA_OFFSET 160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD
CFI_RESTORE 15
br %r14
/* Fallback to system call */
- CFI_DEF_CFA_OFFSET 176
- CFI_VAL_OFFSET 15, -160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD+16
+ CFI_VAL_OFFSET 15, -STACK_FRAME_OVERHEAD
19: lhi %r1,__NR_clock_gettime
svc 0
ahi %r15,16
- CFI_DEF_CFA_OFFSET 160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD
CFI_RESTORE 15
br %r14
CFI_ENDPROC
#include <asm/asm-offsets.h>
#include <asm/unistd.h>
#include <asm/dwarf.h>
+#include <asm/ptrace.h>
.text
.align 4
CFI_STARTPROC
ahi %r15,-16
CFI_ADJUST_CFA_OFFSET 16
- CFI_VAL_OFFSET 15, -160
+ CFI_VAL_OFFSET 15, -STACK_FRAME_OVERHEAD
basr %r5,0
0: al %r5,13f-0b(%r5) /* get &_vdso_data */
1: ltr %r3,%r3 /* check if tz is NULL */
extra-y += vdso64.lds
CPPFLAGS_vdso64.lds += -P -C -U$(ARCH)
-# Disable gcov profiling and ubsan for VDSO code
+# Disable gcov profiling, ubsan and kasan for VDSO code
GCOV_PROFILE := n
UBSAN_SANITIZE := n
+KASAN_SANITIZE := n
# Force dependency (incbin is bad)
$(obj)/vdso64_wrapper.o : $(obj)/vdso64.so
#include <asm/asm-offsets.h>
#include <asm/unistd.h>
#include <asm/dwarf.h>
+#include <asm/ptrace.h>
.text
.align 4
__kernel_clock_gettime:
CFI_STARTPROC
aghi %r15,-16
- CFI_DEF_CFA_OFFSET 176
- CFI_VAL_OFFSET 15, -160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD+16
+ CFI_VAL_OFFSET 15, -STACK_FRAME_OVERHEAD
larl %r5,_vdso_data
cghi %r2,__CLOCK_REALTIME_COARSE
je 4f
stg %r1,8(%r3) /* store tp->tv_nsec */
lghi %r2,0
aghi %r15,16
- CFI_DEF_CFA_OFFSET 160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD
CFI_RESTORE 15
br %r14
/* CLOCK_MONOTONIC_COARSE */
- CFI_DEF_CFA_OFFSET 176
- CFI_VAL_OFFSET 15, -160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD+16
+ CFI_VAL_OFFSET 15, -STACK_FRAME_OVERHEAD
3: lg %r4,__VDSO_UPD_COUNT(%r5) /* load update counter */
tmll %r4,0x0001 /* pending update ? loop */
jnz 3b
stg %r1,8(%r3) /* store tp->tv_nsec */
lghi %r2,0
aghi %r15,16
- CFI_DEF_CFA_OFFSET 160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD
CFI_RESTORE 15
br %r14
/* CPUCLOCK_VIRT for this thread */
- CFI_DEF_CFA_OFFSET 176
- CFI_VAL_OFFSET 15, -160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD+16
+ CFI_VAL_OFFSET 15, -STACK_FRAME_OVERHEAD
9: lghi %r4,0
icm %r0,15,__VDSO_ECTG_OK(%r5)
jz 12f
stg %r4,8(%r3)
lghi %r2,0
aghi %r15,16
- CFI_DEF_CFA_OFFSET 160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD
CFI_RESTORE 15
br %r14
/* Fallback to system call */
- CFI_DEF_CFA_OFFSET 176
- CFI_VAL_OFFSET 15, -160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD+16
+ CFI_VAL_OFFSET 15, -STACK_FRAME_OVERHEAD
12: lghi %r1,__NR_clock_gettime
svc 0
aghi %r15,16
- CFI_DEF_CFA_OFFSET 160
+ CFI_DEF_CFA_OFFSET STACK_FRAME_OVERHEAD
CFI_RESTORE 15
br %r14
CFI_ENDPROC
#include <asm/asm-offsets.h>
#include <asm/unistd.h>
#include <asm/dwarf.h>
+#include <asm/ptrace.h>
.text
.align 4
CFI_STARTPROC
aghi %r15,-16
CFI_ADJUST_CFA_OFFSET 16
- CFI_VAL_OFFSET 15, -160
+ CFI_VAL_OFFSET 15, -STACK_FRAME_OVERHEAD
larl %r5,_vdso_data
0: ltgr %r3,%r3 /* check if tz is NULL */
je 1f
#define RO_AFTER_INIT_DATA
#include <asm-generic/vmlinux.lds.h>
+#include <asm/vmlinux.lds.h>
OUTPUT_FORMAT("elf64-s390", "elf64-s390", "elf64-s390")
OUTPUT_ARCH(s390:64-bit)
__start_ro_after_init = .;
.data..ro_after_init : {
*(.data..ro_after_init)
+ JUMP_TABLE_DATA
}
EXCEPTION_TABLE(16)
. = ALIGN(PAGE_SIZE);
__nospec_return_end = . ;
}
+ BOOT_DATA
+
/* early.c uses stsi, which requires page aligned data. */
. = ALIGN(PAGE_SIZE);
INIT_DATA_SECTION(0x100)
_end = . ;
+ /*
+ * uncompressed image info used by the decompressor
+ * it should match struct vmlinux_info
+ */
+ .vmlinux.info 0 : {
+ QUAD(_stext) /* default_lma */
+ QUAD(startup_continue) /* entry */
+ QUAD(__bss_start - _stext) /* image_size */
+ QUAD(__bss_stop - __bss_start) /* bss_size */
+ QUAD(__boot_data_start) /* bootdata_off */
+ QUAD(__boot_data_end - __boot_data_start) /* bootdata_size */
+ }
+
/* Debugging sections. */
STABS_DEBUG
DWARF_DEBUG
lib-$(CONFIG_KPROBES) += probes.o
lib-$(CONFIG_UPROBES) += probes.o
+# Instrumenting memory accesses to __user data (in different address space)
+# produce false positives
+KASAN_SANITIZE_uaccess.o := n
+
chkbss := mem.o
include $(srctree)/arch/s390/scripts/Makefile.chkbss
/*
* void *memmove(void *dest, const void *src, size_t n)
*/
-ENTRY(memmove)
+WEAK(memmove)
+ENTRY(__memmove)
ltgr %r4,%r4
lgr %r1,%r2
jz .Lmemmove_exit
BR_EX %r14
.Lmemmove_mvc:
mvc 0(1,%r1),0(%r3)
+ENDPROC(__memmove)
EXPORT_SYMBOL(memmove)
/*
* return __builtin_memset(s, c, n);
* }
*/
-ENTRY(memset)
+WEAK(memset)
+ENTRY(__memset)
ltgr %r4,%r4
jz .Lmemset_exit
ltgr %r3,%r3
xc 0(1,%r1),0(%r1)
.Lmemset_mvc:
mvc 1(1,%r1),0(%r1)
+ENDPROC(__memset)
EXPORT_SYMBOL(memset)
/*
*
* void *memcpy(void *dest, const void *src, size_t n)
*/
-ENTRY(memcpy)
+WEAK(memcpy)
+ENTRY(__memcpy)
ltgr %r4,%r4
jz .Lmemcpy_exit
aghi %r4,-1
j .Lmemcpy_remainder
.Lmemcpy_mvc:
mvc 0(1,%r1),0(%r3)
+ENDPROC(__memcpy)
EXPORT_SYMBOL(memcpy)
/*
#
obj-y := init.o fault.o extmem.o mmap.o vmem.o maccess.o
-obj-y += page-states.o gup.o pageattr.o mem_detect.o
-obj-y += pgtable.o pgalloc.o
+obj-y += page-states.o gup.o pageattr.o pgtable.o pgalloc.o
obj-$(CONFIG_CMM) += cmm.o
obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
obj-$(CONFIG_S390_PTDUMP) += dump_pagetables.o
obj-$(CONFIG_PGSTE) += gmap.o
+
+KASAN_SANITIZE_kasan_init.o := n
+obj-$(CONFIG_KASAN) += kasan_init.o
#include <linux/debugfs.h>
#include <linux/sched.h>
#include <linux/mm.h>
+#include <linux/kasan.h>
+#include <asm/kasan.h>
#include <asm/sections.h>
#include <asm/pgtable.h>
IDENTITY_NR = 0,
KERNEL_START_NR,
KERNEL_END_NR,
+#ifdef CONFIG_KASAN
+ KASAN_SHADOW_START_NR,
+ KASAN_SHADOW_END_NR,
+#endif
VMEMMAP_NR,
VMALLOC_NR,
MODULES_NR,
};
static struct addr_marker address_markers[] = {
- [IDENTITY_NR] = {0, "Identity Mapping"},
- [KERNEL_START_NR] = {(unsigned long)_stext, "Kernel Image Start"},
- [KERNEL_END_NR] = {(unsigned long)_end, "Kernel Image End"},
- [VMEMMAP_NR] = {0, "vmemmap Area"},
- [VMALLOC_NR] = {0, "vmalloc Area"},
- [MODULES_NR] = {0, "Modules Area"},
+ [IDENTITY_NR] = {0, "Identity Mapping"},
+ [KERNEL_START_NR] = {(unsigned long)_stext, "Kernel Image Start"},
+ [KERNEL_END_NR] = {(unsigned long)_end, "Kernel Image End"},
+#ifdef CONFIG_KASAN
+ [KASAN_SHADOW_START_NR] = {KASAN_SHADOW_START, "Kasan Shadow Start"},
+ [KASAN_SHADOW_END_NR] = {KASAN_SHADOW_END, "Kasan Shadow End"},
+#endif
+ [VMEMMAP_NR] = {0, "vmemmap Area"},
+ [VMALLOC_NR] = {0, "vmalloc Area"},
+ [MODULES_NR] = {0, "Modules Area"},
{ -1, NULL }
};
} else if (prot != cur || level != st->level ||
st->current_address >= st->marker[1].start_address) {
/* Print the actual finished series */
- seq_printf(m, "0x%0*lx-0x%0*lx",
+ seq_printf(m, "0x%0*lx-0x%0*lx ",
width, st->start_address,
width, st->current_address);
delta = (st->current_address - st->start_address) >> 10;
}
seq_printf(m, "%9lu%c ", delta, *unit);
print_prot(m, st->current_prot, st->level);
- if (st->current_address >= st->marker[1].start_address) {
+ while (st->current_address >= st->marker[1].start_address) {
st->marker++;
seq_printf(m, "---[ %s ]---\n", st->marker->name);
}
}
}
+#ifdef CONFIG_KASAN
+static void note_kasan_zero_page(struct seq_file *m, struct pg_state *st)
+{
+ unsigned int prot;
+
+ prot = pte_val(*kasan_zero_pte) &
+ (_PAGE_PROTECT | _PAGE_INVALID | _PAGE_NOEXEC);
+ note_page(m, st, prot, 4);
+}
+#endif
+
/*
* The actual page table walker functions. In order to keep the
* implementation of print_prot() short, we only check and pass
pmd_t *pmd;
int i;
+#ifdef CONFIG_KASAN
+ if ((pud_val(*pud) & PAGE_MASK) == __pa(kasan_zero_pmd)) {
+ note_kasan_zero_page(m, st);
+ return;
+ }
+#endif
+
for (i = 0; i < PTRS_PER_PMD && addr < max_addr; i++) {
st->current_address = addr;
pmd = pmd_offset(pud, addr);
pud_t *pud;
int i;
+#ifdef CONFIG_KASAN
+ if ((p4d_val(*p4d) & PAGE_MASK) == __pa(kasan_zero_pud)) {
+ note_kasan_zero_page(m, st);
+ return;
+ }
+#endif
+
for (i = 0; i < PTRS_PER_PUD && addr < max_addr; i++) {
st->current_address = addr;
pud = pud_offset(p4d, addr);
p4d_t *p4d;
int i;
+#ifdef CONFIG_KASAN
+ if ((pgd_val(*pgd) & PAGE_MASK) == __pa(kasan_zero_p4d)) {
+ note_kasan_zero_page(m, st);
+ return;
+ }
+#endif
+
for (i = 0; i < PTRS_PER_P4D && addr < max_addr; i++) {
st->current_address = addr;
p4d = p4d_offset(pgd, addr);
u64 reserved;
} __attribute__ ((packed, aligned(8)));
+static struct pfault_refbk pfault_init_refbk = {
+ .refdiagc = 0x258,
+ .reffcode = 0,
+ .refdwlen = 5,
+ .refversn = 2,
+ .refgaddr = __LC_LPP,
+ .refselmk = 1ULL << 48,
+ .refcmpmk = 1ULL << 48,
+ .reserved = __PF_RES_FIELD
+};
+
int pfault_init(void)
{
- struct pfault_refbk refbk = {
- .refdiagc = 0x258,
- .reffcode = 0,
- .refdwlen = 5,
- .refversn = 2,
- .refgaddr = __LC_LPP,
- .refselmk = 1ULL << 48,
- .refcmpmk = 1ULL << 48,
- .reserved = __PF_RES_FIELD };
int rc;
if (pfault_disable)
"1: la %0,8\n"
"2:\n"
EX_TABLE(0b,1b)
- : "=d" (rc) : "a" (&refbk), "m" (refbk) : "cc");
+ : "=d" (rc)
+ : "a" (&pfault_init_refbk), "m" (pfault_init_refbk) : "cc");
return rc;
}
+static struct pfault_refbk pfault_fini_refbk = {
+ .refdiagc = 0x258,
+ .reffcode = 1,
+ .refdwlen = 5,
+ .refversn = 2,
+};
+
void pfault_fini(void)
{
- struct pfault_refbk refbk = {
- .refdiagc = 0x258,
- .reffcode = 1,
- .refdwlen = 5,
- .refversn = 2,
- };
if (pfault_disable)
return;
" diag %0,0,0x258\n"
"0: nopr %%r7\n"
EX_TABLE(0b,0b)
- : : "a" (&refbk), "m" (refbk) : "cc");
+ : : "a" (&pfault_fini_refbk), "m" (pfault_fini_refbk) : "cc");
}
static DEFINE_SPINLOCK(pfault_lock);
#include <asm/ctl_reg.h>
#include <asm/sclp.h>
#include <asm/set_memory.h>
+#include <asm/kasan.h>
pgd_t swapper_pg_dir[PTRS_PER_PGD] __section(.bss..swapper_pg_dir);
S390_lowcore.user_asce = S390_lowcore.kernel_asce;
crst_table_init((unsigned long *) init_mm.pgd, pgd_type);
vmem_map_init();
+ kasan_copy_shadow(init_mm.pgd);
- /* enable virtual mapping in kernel mode */
+ /* enable virtual mapping in kernel mode */
__ctl_load(S390_lowcore.kernel_asce, 1, 1);
__ctl_load(S390_lowcore.kernel_asce, 7, 7);
__ctl_load(S390_lowcore.kernel_asce, 13, 13);
psw_bits(psw).dat = 1;
psw_bits(psw).as = PSW_BITS_AS_HOME;
__load_psw_mask(psw.mask);
+ kasan_free_early_identity();
sparse_memory_present_with_active_regions(MAX_NUMNODES);
sparse_init();
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kasan.h>
+#include <linux/sched/task.h>
+#include <linux/memblock.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+#include <asm/kasan.h>
+#include <asm/mem_detect.h>
+#include <asm/processor.h>
+#include <asm/sclp.h>
+#include <asm/facility.h>
+#include <asm/sections.h>
+#include <asm/setup.h>
+
+static unsigned long segment_pos __initdata;
+static unsigned long segment_low __initdata;
+static unsigned long pgalloc_pos __initdata;
+static unsigned long pgalloc_low __initdata;
+static unsigned long pgalloc_freeable __initdata;
+static bool has_edat __initdata;
+static bool has_nx __initdata;
+
+#define __sha(x) ((unsigned long)kasan_mem_to_shadow((void *)x))
+
+static pgd_t early_pg_dir[PTRS_PER_PGD] __initdata __aligned(PAGE_SIZE);
+
+static void __init kasan_early_panic(const char *reason)
+{
+ sclp_early_printk("The Linux kernel failed to boot with the KernelAddressSanitizer:\n");
+ sclp_early_printk(reason);
+ disabled_wait(0);
+}
+
+static void * __init kasan_early_alloc_segment(void)
+{
+ segment_pos -= _SEGMENT_SIZE;
+
+ if (segment_pos < segment_low)
+ kasan_early_panic("out of memory during initialisation\n");
+
+ return (void *)segment_pos;
+}
+
+static void * __init kasan_early_alloc_pages(unsigned int order)
+{
+ pgalloc_pos -= (PAGE_SIZE << order);
+
+ if (pgalloc_pos < pgalloc_low)
+ kasan_early_panic("out of memory during initialisation\n");
+
+ return (void *)pgalloc_pos;
+}
+
+static void * __init kasan_early_crst_alloc(unsigned long val)
+{
+ unsigned long *table;
+
+ table = kasan_early_alloc_pages(CRST_ALLOC_ORDER);
+ if (table)
+ crst_table_init(table, val);
+ return table;
+}
+
+static pte_t * __init kasan_early_pte_alloc(void)
+{
+ static void *pte_leftover;
+ pte_t *pte;
+
+ BUILD_BUG_ON(_PAGE_TABLE_SIZE * 2 != PAGE_SIZE);
+
+ if (!pte_leftover) {
+ pte_leftover = kasan_early_alloc_pages(0);
+ pte = pte_leftover + _PAGE_TABLE_SIZE;
+ } else {
+ pte = pte_leftover;
+ pte_leftover = NULL;
+ }
+ memset64((u64 *)pte, _PAGE_INVALID, PTRS_PER_PTE);
+ return pte;
+}
+
+enum populate_mode {
+ POPULATE_ONE2ONE,
+ POPULATE_MAP,
+ POPULATE_ZERO_SHADOW
+};
+static void __init kasan_early_vmemmap_populate(unsigned long address,
+ unsigned long end,
+ enum populate_mode mode)
+{
+ unsigned long pgt_prot_zero, pgt_prot, sgt_prot;
+ pgd_t *pg_dir;
+ p4d_t *p4_dir;
+ pud_t *pu_dir;
+ pmd_t *pm_dir;
+ pte_t *pt_dir;
+
+ pgt_prot_zero = pgprot_val(PAGE_KERNEL_RO);
+ if (!has_nx)
+ pgt_prot_zero &= ~_PAGE_NOEXEC;
+ pgt_prot = pgprot_val(PAGE_KERNEL_EXEC);
+ sgt_prot = pgprot_val(SEGMENT_KERNEL_EXEC);
+
+ while (address < end) {
+ pg_dir = pgd_offset_k(address);
+ if (pgd_none(*pg_dir)) {
+ if (mode == POPULATE_ZERO_SHADOW &&
+ IS_ALIGNED(address, PGDIR_SIZE) &&
+ end - address >= PGDIR_SIZE) {
+ pgd_populate(&init_mm, pg_dir, kasan_zero_p4d);
+ address = (address + PGDIR_SIZE) & PGDIR_MASK;
+ continue;
+ }
+ p4_dir = kasan_early_crst_alloc(_REGION2_ENTRY_EMPTY);
+ pgd_populate(&init_mm, pg_dir, p4_dir);
+ }
+
+ p4_dir = p4d_offset(pg_dir, address);
+ if (p4d_none(*p4_dir)) {
+ if (mode == POPULATE_ZERO_SHADOW &&
+ IS_ALIGNED(address, P4D_SIZE) &&
+ end - address >= P4D_SIZE) {
+ p4d_populate(&init_mm, p4_dir, kasan_zero_pud);
+ address = (address + P4D_SIZE) & P4D_MASK;
+ continue;
+ }
+ pu_dir = kasan_early_crst_alloc(_REGION3_ENTRY_EMPTY);
+ p4d_populate(&init_mm, p4_dir, pu_dir);
+ }
+
+ pu_dir = pud_offset(p4_dir, address);
+ if (pud_none(*pu_dir)) {
+ if (mode == POPULATE_ZERO_SHADOW &&
+ IS_ALIGNED(address, PUD_SIZE) &&
+ end - address >= PUD_SIZE) {
+ pud_populate(&init_mm, pu_dir, kasan_zero_pmd);
+ address = (address + PUD_SIZE) & PUD_MASK;
+ continue;
+ }
+ pm_dir = kasan_early_crst_alloc(_SEGMENT_ENTRY_EMPTY);
+ pud_populate(&init_mm, pu_dir, pm_dir);
+ }
+
+ pm_dir = pmd_offset(pu_dir, address);
+ if (pmd_none(*pm_dir)) {
+ if (mode == POPULATE_ZERO_SHADOW &&
+ IS_ALIGNED(address, PMD_SIZE) &&
+ end - address >= PMD_SIZE) {
+ pmd_populate(&init_mm, pm_dir, kasan_zero_pte);
+ address = (address + PMD_SIZE) & PMD_MASK;
+ continue;
+ }
+ /* the first megabyte of 1:1 is mapped with 4k pages */
+ if (has_edat && address && end - address >= PMD_SIZE &&
+ mode != POPULATE_ZERO_SHADOW) {
+ void *page;
+
+ if (mode == POPULATE_ONE2ONE) {
+ page = (void *)address;
+ } else {
+ page = kasan_early_alloc_segment();
+ memset(page, 0, _SEGMENT_SIZE);
+ }
+ pmd_val(*pm_dir) = __pa(page) | sgt_prot;
+ address = (address + PMD_SIZE) & PMD_MASK;
+ continue;
+ }
+
+ pt_dir = kasan_early_pte_alloc();
+ pmd_populate(&init_mm, pm_dir, pt_dir);
+ } else if (pmd_large(*pm_dir)) {
+ address = (address + PMD_SIZE) & PMD_MASK;
+ continue;
+ }
+
+ pt_dir = pte_offset_kernel(pm_dir, address);
+ if (pte_none(*pt_dir)) {
+ void *page;
+
+ switch (mode) {
+ case POPULATE_ONE2ONE:
+ page = (void *)address;
+ pte_val(*pt_dir) = __pa(page) | pgt_prot;
+ break;
+ case POPULATE_MAP:
+ page = kasan_early_alloc_pages(0);
+ memset(page, 0, PAGE_SIZE);
+ pte_val(*pt_dir) = __pa(page) | pgt_prot;
+ break;
+ case POPULATE_ZERO_SHADOW:
+ page = kasan_zero_page;
+ pte_val(*pt_dir) = __pa(page) | pgt_prot_zero;
+ break;
+ }
+ }
+ address += PAGE_SIZE;
+ }
+}
+
+static void __init kasan_set_pgd(pgd_t *pgd, unsigned long asce_type)
+{
+ unsigned long asce_bits;
+
+ asce_bits = asce_type | _ASCE_TABLE_LENGTH;
+ S390_lowcore.kernel_asce = (__pa(pgd) & PAGE_MASK) | asce_bits;
+ S390_lowcore.user_asce = S390_lowcore.kernel_asce;
+
+ __ctl_load(S390_lowcore.kernel_asce, 1, 1);
+ __ctl_load(S390_lowcore.kernel_asce, 7, 7);
+ __ctl_load(S390_lowcore.kernel_asce, 13, 13);
+}
+
+static void __init kasan_enable_dat(void)
+{
+ psw_t psw;
+
+ psw.mask = __extract_psw();
+ psw_bits(psw).dat = 1;
+ psw_bits(psw).as = PSW_BITS_AS_HOME;
+ __load_psw_mask(psw.mask);
+}
+
+static void __init kasan_early_detect_facilities(void)
+{
+ __stfle(S390_lowcore.stfle_fac_list,
+ ARRAY_SIZE(S390_lowcore.stfle_fac_list));
+ if (test_facility(8)) {
+ has_edat = true;
+ __ctl_set_bit(0, 23);
+ }
+ if (!noexec_disabled && test_facility(130)) {
+ has_nx = true;
+ __ctl_set_bit(0, 20);
+ }
+}
+
+static unsigned long __init get_mem_detect_end(void)
+{
+ unsigned long start;
+ unsigned long end;
+
+ if (mem_detect.count) {
+ __get_mem_detect_block(mem_detect.count - 1, &start, &end);
+ return end;
+ }
+ return 0;
+}
+
+void __init kasan_early_init(void)
+{
+ unsigned long untracked_mem_end;
+ unsigned long shadow_alloc_size;
+ unsigned long initrd_end;
+ unsigned long asce_type;
+ unsigned long memsize;
+ unsigned long vmax;
+ unsigned long pgt_prot = pgprot_val(PAGE_KERNEL_RO);
+ pte_t pte_z;
+ pmd_t pmd_z = __pmd(__pa(kasan_zero_pte) | _SEGMENT_ENTRY);
+ pud_t pud_z = __pud(__pa(kasan_zero_pmd) | _REGION3_ENTRY);
+ p4d_t p4d_z = __p4d(__pa(kasan_zero_pud) | _REGION2_ENTRY);
+
+ kasan_early_detect_facilities();
+ if (!has_nx)
+ pgt_prot &= ~_PAGE_NOEXEC;
+ pte_z = __pte(__pa(kasan_zero_page) | pgt_prot);
+
+ memsize = get_mem_detect_end();
+ if (!memsize)
+ kasan_early_panic("cannot detect physical memory size\n");
+ /* respect mem= cmdline parameter */
+ if (memory_end_set && memsize > memory_end)
+ memsize = memory_end;
+ memsize = min(memsize, KASAN_SHADOW_START);
+
+ if (IS_ENABLED(CONFIG_KASAN_S390_4_LEVEL_PAGING)) {
+ /* 4 level paging */
+ BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_START, P4D_SIZE));
+ BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_END, P4D_SIZE));
+ crst_table_init((unsigned long *)early_pg_dir,
+ _REGION2_ENTRY_EMPTY);
+ untracked_mem_end = vmax = _REGION1_SIZE;
+ asce_type = _ASCE_TYPE_REGION2;
+ } else {
+ /* 3 level paging */
+ BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_START, PUD_SIZE));
+ BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_END, PUD_SIZE));
+ crst_table_init((unsigned long *)early_pg_dir,
+ _REGION3_ENTRY_EMPTY);
+ untracked_mem_end = vmax = _REGION2_SIZE;
+ asce_type = _ASCE_TYPE_REGION3;
+ }
+
+ /* init kasan zero shadow */
+ crst_table_init((unsigned long *)kasan_zero_p4d, p4d_val(p4d_z));
+ crst_table_init((unsigned long *)kasan_zero_pud, pud_val(pud_z));
+ crst_table_init((unsigned long *)kasan_zero_pmd, pmd_val(pmd_z));
+ memset64((u64 *)kasan_zero_pte, pte_val(pte_z), PTRS_PER_PTE);
+
+ shadow_alloc_size = memsize >> KASAN_SHADOW_SCALE_SHIFT;
+ pgalloc_low = round_up((unsigned long)_end, _SEGMENT_SIZE);
+ if (IS_ENABLED(CONFIG_BLK_DEV_INITRD)) {
+ initrd_end =
+ round_up(INITRD_START + INITRD_SIZE, _SEGMENT_SIZE);
+ pgalloc_low = max(pgalloc_low, initrd_end);
+ }
+
+ if (pgalloc_low + shadow_alloc_size > memsize)
+ kasan_early_panic("out of memory during initialisation\n");
+
+ if (has_edat) {
+ segment_pos = round_down(memsize, _SEGMENT_SIZE);
+ segment_low = segment_pos - shadow_alloc_size;
+ pgalloc_pos = segment_low;
+ } else {
+ pgalloc_pos = memsize;
+ }
+ init_mm.pgd = early_pg_dir;
+ /*
+ * Current memory layout:
+ * +- 0 -------------+ +- shadow start -+
+ * | 1:1 ram mapping | /| 1/8 ram |
+ * +- end of ram ----+ / +----------------+
+ * | ... gap ... |/ | kasan |
+ * +- shadow start --+ | zero |
+ * | 1/8 addr space | | page |
+ * +- shadow end -+ | mapping |
+ * | ... gap ... |\ | (untracked) |
+ * +- modules vaddr -+ \ +----------------+
+ * | 2Gb | \| unmapped | allocated per module
+ * +-----------------+ +- shadow end ---+
+ */
+ /* populate kasan shadow (for identity mapping and zero page mapping) */
+ kasan_early_vmemmap_populate(__sha(0), __sha(memsize), POPULATE_MAP);
+ if (IS_ENABLED(CONFIG_MODULES))
+ untracked_mem_end = vmax - MODULES_LEN;
+ kasan_early_vmemmap_populate(__sha(max_physmem_end),
+ __sha(untracked_mem_end),
+ POPULATE_ZERO_SHADOW);
+ /* memory allocated for identity mapping structs will be freed later */
+ pgalloc_freeable = pgalloc_pos;
+ /* populate identity mapping */
+ kasan_early_vmemmap_populate(0, memsize, POPULATE_ONE2ONE);
+ kasan_set_pgd(early_pg_dir, asce_type);
+ kasan_enable_dat();
+ /* enable kasan */
+ init_task.kasan_depth = 0;
+ memblock_reserve(pgalloc_pos, memsize - pgalloc_pos);
+ sclp_early_printk("KernelAddressSanitizer initialized\n");
+}
+
+void __init kasan_copy_shadow(pgd_t *pg_dir)
+{
+ /*
+ * At this point we are still running on early pages setup early_pg_dir,
+ * while swapper_pg_dir has just been initialized with identity mapping.
+ * Carry over shadow memory region from early_pg_dir to swapper_pg_dir.
+ */
+
+ pgd_t *pg_dir_src;
+ pgd_t *pg_dir_dst;
+ p4d_t *p4_dir_src;
+ p4d_t *p4_dir_dst;
+ pud_t *pu_dir_src;
+ pud_t *pu_dir_dst;
+
+ pg_dir_src = pgd_offset_raw(early_pg_dir, KASAN_SHADOW_START);
+ pg_dir_dst = pgd_offset_raw(pg_dir, KASAN_SHADOW_START);
+ p4_dir_src = p4d_offset(pg_dir_src, KASAN_SHADOW_START);
+ p4_dir_dst = p4d_offset(pg_dir_dst, KASAN_SHADOW_START);
+ if (!p4d_folded(*p4_dir_src)) {
+ /* 4 level paging */
+ memcpy(p4_dir_dst, p4_dir_src,
+ (KASAN_SHADOW_SIZE >> P4D_SHIFT) * sizeof(p4d_t));
+ return;
+ }
+ /* 3 level paging */
+ pu_dir_src = pud_offset(p4_dir_src, KASAN_SHADOW_START);
+ pu_dir_dst = pud_offset(p4_dir_dst, KASAN_SHADOW_START);
+ memcpy(pu_dir_dst, pu_dir_src,
+ (KASAN_SHADOW_SIZE >> PUD_SHIFT) * sizeof(pud_t));
+}
+
+void __init kasan_free_early_identity(void)
+{
+ memblock_free(pgalloc_pos, pgalloc_freeable - pgalloc_pos);
+}
return rc;
}
-/*
- * Copy memory in real mode (kernel to kernel)
- */
-int memcpy_real(void *dest, void *src, size_t count)
+static unsigned long _memcpy_real(unsigned long dest, unsigned long src,
+ unsigned long count)
{
int irqs_disabled, rc;
unsigned long flags;
irqs_disabled = arch_irqs_disabled_flags(flags);
if (!irqs_disabled)
trace_hardirqs_off();
- rc = __memcpy_real(dest, src, count);
+ rc = __memcpy_real((void *) dest, (void *) src, (size_t) count);
if (!irqs_disabled)
trace_hardirqs_on();
__arch_local_irq_ssm(flags);
return rc;
}
+/*
+ * Copy memory in real mode (kernel to kernel)
+ */
+int memcpy_real(void *dest, void *src, size_t count)
+{
+ if (S390_lowcore.nodat_stack != 0)
+ return CALL_ON_STACK(_memcpy_real, S390_lowcore.nodat_stack,
+ 3, dest, src, count);
+ /*
+ * This is a really early memcpy_real call, the stacks are
+ * not set up yet. Just call _memcpy_real on the early boot
+ * stack
+ */
+ return _memcpy_real((unsigned long) dest,(unsigned long) src,
+ (unsigned long) count);
+}
+
/*
* Copy memory in absolute mode (kernel to kernel)
*/
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Copyright IBM Corp. 2008, 2009
- *
- * Author(s): Heiko Carstens <heiko.carstens@de.ibm.com>
- */
-
-#include <linux/kernel.h>
-#include <linux/memblock.h>
-#include <linux/init.h>
-#include <linux/debugfs.h>
-#include <linux/seq_file.h>
-#include <asm/ipl.h>
-#include <asm/sclp.h>
-#include <asm/setup.h>
-
-#define CHUNK_READ_WRITE 0
-#define CHUNK_READ_ONLY 1
-
-static inline void memblock_physmem_add(phys_addr_t start, phys_addr_t size)
-{
- memblock_dbg("memblock_physmem_add: [%#016llx-%#016llx]\n",
- start, start + size - 1);
- memblock_add_range(&memblock.memory, start, size, 0, 0);
- memblock_add_range(&memblock.physmem, start, size, 0, 0);
-}
-
-void __init detect_memory_memblock(void)
-{
- unsigned long memsize, rnmax, rzm, addr, size;
- int type;
-
- rzm = sclp.rzm;
- rnmax = sclp.rnmax;
- memsize = rzm * rnmax;
- if (!rzm)
- rzm = 1UL << 17;
- max_physmem_end = memsize;
- addr = 0;
- /* keep memblock lists close to the kernel */
- memblock_set_bottom_up(true);
- do {
- size = 0;
- /* assume lowcore is writable */
- type = addr ? tprot(addr) : CHUNK_READ_WRITE;
- do {
- size += rzm;
- if (max_physmem_end && addr + size >= max_physmem_end)
- break;
- } while (type == tprot(addr + size));
- if (type == CHUNK_READ_WRITE || type == CHUNK_READ_ONLY) {
- if (max_physmem_end && (addr + size > max_physmem_end))
- size = max_physmem_end - addr;
- memblock_physmem_add(addr, size);
- }
- addr += size;
- } while (addr < max_physmem_end);
- memblock_set_bottom_up(false);
- if (!max_physmem_end)
- max_physmem_end = memblock_end_of_DRAM();
- memblock_dump_all();
-}
#include <asm/asm-offsets.h>
#include <asm/page.h>
#include <asm/sigp.h>
+#include <asm/ptrace.h>
/* The purgatory is the code running between two kernels. It's main purpose
* is to verify that the next kernel was not corrupted after load and to
.base_crash:
/* Setup stack */
- larl %r15,purgatory_end
- aghi %r15,-160
+ larl %r15,purgatory_end-STACK_FRAME_OVERHEAD
/* If the next kernel is KEXEC_TYPE_CRASH the purgatory is called
* directly with a flag passed in %r2 whether the purgatory shall do
select ARCH_NO_COHERENT_DMA_MMAP if !MMU
select HAVE_PATA_PLATFORM
select CLKDEV_LOOKUP
+ select DMA_DIRECT_OPS
select HAVE_IDE if HAS_IOPORT_MAP
select HAVE_MEMBLOCK
select HAVE_MEMBLOCK_NODE_MAP
bool
config DMA_COHERENT
- select DMA_DIRECT_OPS
bool
config DMA_NONCOHERENT
def_bool !DMA_COHERENT
select ARCH_HAS_SYNC_DMA_FOR_DEVICE
- select DMA_NONCOHERENT_OPS
config PGTABLE_LEVELS
default 3 if X2TLB
static struct fixed_voltage_config cn12_power_info = {
.supply_name = "CN12 SD/MMC Vdd",
.microvolts = 3300000,
- .gpio = GPIO_PTB7,
.enable_high = 1,
.init_data = &cn12_power_init_data,
};
},
};
+static struct gpiod_lookup_table cn12_power_gpiod_table = {
+ .dev_id = "reg-fixed-voltage.0",
+ .table = {
+ /* Offset 7 on port B */
+ GPIO_LOOKUP("sh7724_pfc", GPIO_PTB7,
+ NULL, GPIO_ACTIVE_HIGH),
+ { },
+ },
+};
+
#if defined(CONFIG_MMC_SDHI) || defined(CONFIG_MMC_SDHI_MODULE)
/* SDHI0 */
static struct regulator_consumer_supply sdhi0_power_consumers[] =
static struct fixed_voltage_config sdhi0_power_info = {
.supply_name = "CN11 SD/MMC Vdd",
.microvolts = 3300000,
- .gpio = GPIO_PTB6,
.enable_high = 1,
.init_data = &sdhi0_power_init_data,
};
},
};
+static struct gpiod_lookup_table sdhi0_power_gpiod_table = {
+ .dev_id = "reg-fixed-voltage.1",
+ .table = {
+ /* Offset 6 on port B */
+ GPIO_LOOKUP("sh7724_pfc", GPIO_PTB6,
+ NULL, GPIO_ACTIVE_HIGH),
+ { },
+ },
+};
+
static struct tmio_mmc_data sdhi0_info = {
.chan_priv_tx = (void *)SHDMA_SLAVE_SDHI0_TX,
.chan_priv_rx = (void *)SHDMA_SLAVE_SDHI0_RX,
DMA_MEMORY_EXCLUSIVE);
platform_device_add(ecovec_ceu_devices[1]);
+ gpiod_add_lookup_table(&cn12_power_gpiod_table);
+#if defined(CONFIG_MMC_SDHI) || defined(CONFIG_MMC_SDHI_MODULE)
+ gpiod_add_lookup_table(&sdhi0_power_gpiod_table);
+#endif
+
return platform_add_devices(ecovec_devices,
ARRAY_SIZE(ecovec_devices));
}
#include <linux/mmc/host.h>
#include <linux/mtd/physmap.h>
#include <linux/mfd/tmio.h>
-#include <linux/mtd/rawnand.h>
+#include <linux/mtd/platnand.h>
#include <linux/i2c.h>
#include <linux/regulator/fixed.h>
#include <linux/regulator/machine.h>
},
};
-static void migor_nand_flash_cmd_ctl(struct mtd_info *mtd, int cmd,
+static void migor_nand_flash_cmd_ctl(struct nand_chip *chip, int cmd,
unsigned int ctrl)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
if (cmd == NAND_CMD_NONE)
return;
if (ctrl & NAND_CLE)
- writeb(cmd, chip->IO_ADDR_W + 0x00400000);
+ writeb(cmd, chip->legacy.IO_ADDR_W + 0x00400000);
else if (ctrl & NAND_ALE)
- writeb(cmd, chip->IO_ADDR_W + 0x00800000);
+ writeb(cmd, chip->legacy.IO_ADDR_W + 0x00800000);
else
- writeb(cmd, chip->IO_ADDR_W);
+ writeb(cmd, chip->legacy.IO_ADDR_W);
}
-static int migor_nand_flash_ready(struct mtd_info *mtd)
+static int migor_nand_flash_ready(struct nand_chip *chip)
{
return gpio_get_value(GPIO_PTA1); /* NAND_RBn */
}
config SPARC32
def_bool !64BIT
select ARCH_HAS_SYNC_DMA_FOR_CPU
- select DMA_NONCOHERENT_OPS
+ select DMA_DIRECT_OPS
select GENERIC_ATOMIC64
select CLZ_TAB
select HAVE_UID16
unsigned short sock_id; /* physical package */
unsigned short core_id;
unsigned short max_cache_id; /* groupings of highest shared cache */
- unsigned short proc_id; /* strand (aka HW thread) id */
+ signed short proc_id; /* strand (aka HW thread) id */
} cpuinfo_sparc;
DECLARE_PER_CPU(cpuinfo_sparc, __cpu_data);
{
#ifdef CONFIG_SPARC_LEON
if (sparc_cpu_model == sparc_leon)
- return &dma_noncoherent_ops;
+ return &dma_direct_ops;
#endif
#if defined(CONFIG_SPARC32) && defined(CONFIG_PCI)
if (bus == &pci_bus_type)
- return &dma_noncoherent_ops;
+ return &dma_direct_ops;
#endif
return dma_ops;
}
#define __NR_preadv2 358
#define __NR_pwritev2 359
#define __NR_statx 360
+#define __NR_io_pgetevents 361
-#define NR_syscalls 361
+#define NR_syscalls 362
/* Bitmask values returned from kern_features system call. */
#define KERN_FEATURE_MIXED_MODE_STACK 0x00000001
linux_regs->pc = addr;
linux_regs->npc = addr + 4;
}
- /* fallthru */
+ /* fall through */
case 'D':
case 'k':
linux_regs->tpc = addr;
linux_regs->tnpc = addr + 4;
}
- /* fallthru */
+ /* fall through */
case 'D':
case 'k':
#include <asm/cpudata.h>
#include <linux/uaccess.h>
#include <linux/atomic.h>
+#include <linux/sched/clock.h>
#include <asm/nmi.h>
#include <asm/pcr.h>
#include <asm/cacheflush.h>
sparc_perf_event_update(cp, &cp->hw,
cpuc->current_idx[i]);
cpuc->current_idx[i] = PIC_NO_INDEX;
+ if (cp->hw.state & PERF_HES_STOPPED)
+ cp->hw.state |= PERF_HES_ARCH;
}
}
}
enc = perf_event_get_enc(cpuc->events[i]);
cpuc->pcr[0] &= ~mask_for_index(idx);
- if (hwc->state & PERF_HES_STOPPED)
+ if (hwc->state & PERF_HES_ARCH) {
cpuc->pcr[0] |= nop_for_index(idx);
- else
+ } else {
cpuc->pcr[0] |= event_encoding(enc, idx);
+ hwc->state = 0;
+ }
}
out:
cpuc->pcr[0] |= cpuc->event[0]->hw.config_base;
cpuc->current_idx[i] = idx;
+ if (cp->hw.state & PERF_HES_ARCH)
+ continue;
+
sparc_pmu_start(cp, PERF_EF_RELOAD);
}
out:
event->hw.state = 0;
sparc_pmu_enable_event(cpuc, &event->hw, idx);
+
+ perf_event_update_userpage(event);
}
static void sparc_pmu_stop(struct perf_event *event, int flags)
cpuc->events[n0] = event->hw.event_base;
cpuc->current_idx[n0] = PIC_NO_INDEX;
- event->hw.state = PERF_HES_UPTODATE;
+ event->hw.state = PERF_HES_UPTODATE | PERF_HES_STOPPED;
if (!(ef_flags & PERF_EF_START))
- event->hw.state |= PERF_HES_STOPPED;
+ event->hw.state |= PERF_HES_ARCH;
/*
* If group events scheduling transaction was started,
struct perf_sample_data data;
struct cpu_hw_events *cpuc;
struct pt_regs *regs;
+ u64 finish_clock;
+ u64 start_clock;
int i;
if (!atomic_read(&active_events))
return NOTIFY_DONE;
}
+ start_clock = sched_clock();
+
regs = args->regs;
cpuc = this_cpu_ptr(&cpu_hw_events);
sparc_pmu_stop(event, 0);
}
+ finish_clock = sched_clock();
+
+ perf_sample_event_took(finish_clock - start_clock);
+
return NOTIFY_STOP;
}
ldx [%sp + PTREGS_OFF + PT_V9_TSTATE], %l1
sethi %hi(0xf << 20), %l4
and %l1, %l4, %l4
+ andn %l1, %l4, %l1
ba,pt %xcc, __handle_preemption_continue
- andn %l1, %l4, %l1
+ srl %l4, 20, %l4
/* When returning from a NMI (%pil==15) interrupt we want to
* avoid running softirqs, doing IRQ tracing, preempting, etc.
/*345*/ .long sys_renameat2, sys_seccomp, sys_getrandom, sys_memfd_create, sys_bpf
/*350*/ .long sys_execveat, sys_membarrier, sys_userfaultfd, sys_bind, sys_listen
/*355*/ .long sys_setsockopt, sys_mlock2, sys_copy_file_range, sys_preadv2, sys_pwritev2
-/*360*/ .long sys_statx
+/*360*/ .long sys_statx, sys_io_pgetevents
.word sys_renameat2, sys_seccomp, sys_getrandom, sys_memfd_create, sys_bpf
/*350*/ .word sys32_execveat, sys_membarrier, sys_userfaultfd, sys_bind, sys_listen
.word compat_sys_setsockopt, sys_mlock2, sys_copy_file_range, compat_sys_preadv2, compat_sys_pwritev2
-/*360*/ .word sys_statx
+/*360*/ .word sys_statx, compat_sys_io_pgetevents
#endif /* CONFIG_COMPAT */
.word sys_renameat2, sys_seccomp, sys_getrandom, sys_memfd_create, sys_bpf
/*350*/ .word sys64_execveat, sys_membarrier, sys_userfaultfd, sys_bind, sys_listen
.word sys_setsockopt, sys_mlock2, sys_copy_file_range, sys_preadv2, sys_pwritev2
-/*360*/ .word sys_statx
+/*360*/ .word sys_statx, sys_io_pgetevents
struct vio_dring_register pkt;
char all[sizeof(struct vio_dring_register) +
(sizeof(struct ldc_trans_cookie) *
- dr->ncookies)];
+ VIO_MAX_RING_COOKIES)];
} u;
+ size_t bytes = sizeof(struct vio_dring_register) +
+ (sizeof(struct ldc_trans_cookie) *
+ dr->ncookies);
int i;
- memset(&u, 0, sizeof(u));
+ if (WARN_ON(bytes > sizeof(u)))
+ return -EINVAL;
+
+ memset(&u, 0, bytes);
init_tag(&u.pkt.tag, VIO_TYPE_CTRL, VIO_SUBTYPE_INFO, VIO_DRING_REG);
u.pkt.dring_ident = 0;
u.pkt.num_descr = dr->num_entries;
(unsigned long long) u.pkt.cookies[i].cookie_size);
}
- return send_ctrl(vio, &u.pkt.tag, sizeof(u));
+ return send_ctrl(vio, &u.pkt.tag, bytes);
}
static int send_rdx(struct vio_driver_state *vio)
targets += $(vdso_img_cfiles)
targets += $(vdso_img_sodbg) $(vdso_img-y:%=vdso%.so)
-export CPPFLAGS_vdso.lds += -P -C
+CPPFLAGS_vdso.lds += -P -C
VDSO_LDFLAGS_vdso.lds = -m64 -Wl,-soname=linux-vdso.so.1 \
-Wl,--no-undefined \
-Wl,-z,max-page-size=8192 -Wl,-z,common-page-size=8192 \
$(DISABLE_LTO)
-$(obj)/vdso64.so.dbg: $(src)/vdso.lds $(vobjs) FORCE
+$(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE
$(call if_changed,vdso)
HOST_EXTRACFLAGS += -I$(srctree)/tools/include
hostprogs-y += vdso2c
quiet_cmd_vdso2c = VDSO2C $@
-define cmd_vdso2c
- $(obj)/vdso2c $< $(<:%.dbg=%) $@
-endef
+ cmd_vdso2c = $(obj)/vdso2c $< $(<:%.dbg=%) $@
$(obj)/vdso-image-%.c: $(obj)/vdso%.so.dbg $(obj)/vdso%.so $(obj)/vdso2c FORCE
$(call if_changed,vdso2c)
#define TICK_PRIV_BIT (1ULL << 63)
#endif
+#ifdef CONFIG_SPARC64
#define SYSCALL_STRING \
"ta 0x6d;" \
- "sub %%g0, %%o0, %%o0;" \
+ "bcs,a 1f;" \
+ " sub %%g0, %%o0, %%o0;" \
+ "1:"
+#else
+#define SYSCALL_STRING \
+ "ta 0x10;" \
+ "bcs,a 1f;" \
+ " sub %%g0, %%o0, %%o0;" \
+ "1:"
+#endif
#define SYSCALL_CLOBBERS \
"f0", "f1", "f2", "f3", "f4", "f5", "f6", "f7", \
unsigned long val;
err = kstrtoul(s, 10, &val);
+ if (err)
+ return err;
vdso_enabled = val;
- return err;
+ return 0;
}
__setup("vdso=", vdso_setup);
#include <linux/module.h>
#include <linux/init.h>
#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/ata.h>
#include <linux/hdreg.h>
#include <linux/cdrom.h>
#define MAX_SG 64
struct ubd {
- struct list_head restart;
/* name (and fd, below) of the file opened for writing, either the
* backing or the cow file. */
char *file;
struct cow cow;
struct platform_device pdev;
struct request_queue *queue;
+ struct blk_mq_tag_set tag_set;
spinlock_t lock;
- struct scatterlist sg[MAX_SG];
- struct request *request;
- int start_sg, end_sg;
- sector_t rq_pos;
};
#define DEFAULT_COW { \
.shared = 0, \
.cow = DEFAULT_COW, \
.lock = __SPIN_LOCK_UNLOCKED(ubd_devs.lock), \
- .request = NULL, \
- .start_sg = 0, \
- .end_sg = 0, \
- .rq_pos = 0, \
}
/* Protected by ubd_lock */
static struct proc_dir_entry *proc_ide_root = NULL;
static struct proc_dir_entry *proc_ide = NULL;
+static blk_status_t ubd_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd);
+
static void make_proc_ide(void)
{
proc_ide_root = proc_mkdir("ide", NULL);
" in the boot output.\n\n"
);
-static void do_ubd_request(struct request_queue * q);
-
/* Only changed by ubd_init, which is an initcall. */
static int thread_fd = -1;
-static LIST_HEAD(restart);
/* Function to read several request pointers at a time
* handling fractional reads if (and as) needed
/* Called without dev->lock held, and only in interrupt context. */
static void ubd_handler(void)
{
- struct ubd *ubd;
- struct list_head *list, *next_ele;
- unsigned long flags;
int n;
int count;
return;
}
for (count = 0; count < n/sizeof(struct io_thread_req *); count++) {
- blk_end_request(
- (*irq_req_buffer)[count]->req,
- BLK_STS_OK,
- (*irq_req_buffer)[count]->length
- );
- kfree((*irq_req_buffer)[count]);
+ struct io_thread_req *io_req = (*irq_req_buffer)[count];
+ int err = io_req->error ? BLK_STS_IOERR : BLK_STS_OK;
+
+ if (!blk_update_request(io_req->req, err, io_req->length))
+ __blk_mq_end_request(io_req->req, err);
+
+ kfree(io_req);
}
}
- reactivate_fd(thread_fd, UBD_IRQ);
- list_for_each_safe(list, next_ele, &restart){
- ubd = container_of(list, struct ubd, restart);
- list_del_init(&ubd->restart);
- spin_lock_irqsave(&ubd->lock, flags);
- do_ubd_request(ubd->queue);
- spin_unlock_irqrestore(&ubd->lock, flags);
- }
+ reactivate_fd(thread_fd, UBD_IRQ);
}
static irqreturn_t ubd_intr(int irq, void *dev)
struct ubd *ubd_dev = dev_get_drvdata(dev);
blk_cleanup_queue(ubd_dev->queue);
+ blk_mq_free_tag_set(&ubd_dev->tag_set);
*ubd_dev = ((struct ubd) DEFAULT_UBD);
}
disk->private_data = &ubd_devs[unit];
disk->queue = ubd_devs[unit].queue;
- device_add_disk(parent, disk);
+ device_add_disk(parent, disk, NULL);
*disk_out = disk;
return 0;
#define ROUND_BLOCK(n) ((n + ((1 << 9) - 1)) & (-1 << 9))
+static const struct blk_mq_ops ubd_mq_ops = {
+ .queue_rq = ubd_queue_rq,
+};
+
static int ubd_add(int n, char **error_out)
{
struct ubd *ubd_dev = &ubd_devs[n];
ubd_dev->size = ROUND_BLOCK(ubd_dev->size);
- INIT_LIST_HEAD(&ubd_dev->restart);
- sg_init_table(ubd_dev->sg, MAX_SG);
+ ubd_dev->tag_set.ops = &ubd_mq_ops;
+ ubd_dev->tag_set.queue_depth = 64;
+ ubd_dev->tag_set.numa_node = NUMA_NO_NODE;
+ ubd_dev->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
+ ubd_dev->tag_set.driver_data = ubd_dev;
+ ubd_dev->tag_set.nr_hw_queues = 1;
- err = -ENOMEM;
- ubd_dev->queue = blk_init_queue(do_ubd_request, &ubd_dev->lock);
- if (ubd_dev->queue == NULL) {
- *error_out = "Failed to initialize device queue";
+ err = blk_mq_alloc_tag_set(&ubd_dev->tag_set);
+ if (err)
goto out;
+
+ ubd_dev->queue = blk_mq_init_queue(&ubd_dev->tag_set);
+ if (IS_ERR(ubd_dev->queue)) {
+ err = PTR_ERR(ubd_dev->queue);
+ goto out_cleanup;
}
+
ubd_dev->queue->queuedata = ubd_dev;
blk_queue_write_cache(ubd_dev->queue, true, false);
err = ubd_disk_register(UBD_MAJOR, ubd_dev->size, n, &ubd_gendisk[n]);
if(err){
*error_out = "Failed to register device";
- goto out_cleanup;
+ goto out_cleanup_tags;
}
if (fake_major != UBD_MAJOR)
out:
return err;
+out_cleanup_tags:
+ blk_mq_free_tag_set(&ubd_dev->tag_set);
out_cleanup:
blk_cleanup_queue(ubd_dev->queue);
goto out;
req->bitmap_words, bitmap_len);
}
-/* Called with dev->lock held */
-static void prepare_request(struct request *req, struct io_thread_req *io_req,
- unsigned long long offset, int page_offset,
- int len, struct page *page)
+static int ubd_queue_one_vec(struct blk_mq_hw_ctx *hctx, struct request *req,
+ u64 off, struct bio_vec *bvec)
{
- struct gendisk *disk = req->rq_disk;
- struct ubd *ubd_dev = disk->private_data;
-
- io_req->req = req;
- io_req->fds[0] = (ubd_dev->cow.file != NULL) ? ubd_dev->cow.fd :
- ubd_dev->fd;
- io_req->fds[1] = ubd_dev->fd;
- io_req->cow_offset = -1;
- io_req->offset = offset;
- io_req->length = len;
- io_req->error = 0;
- io_req->sector_mask = 0;
-
- io_req->op = (rq_data_dir(req) == READ) ? UBD_READ : UBD_WRITE;
- io_req->offsets[0] = 0;
- io_req->offsets[1] = ubd_dev->cow.data_offset;
- io_req->buffer = page_address(page) + page_offset;
- io_req->sectorsize = 1 << 9;
-
- if(ubd_dev->cow.file != NULL)
- cowify_req(io_req, ubd_dev->cow.bitmap,
- ubd_dev->cow.bitmap_offset, ubd_dev->cow.bitmap_len);
-
-}
+ struct ubd *dev = hctx->queue->queuedata;
+ struct io_thread_req *io_req;
+ int ret;
-/* Called with dev->lock held */
-static void prepare_flush_request(struct request *req,
- struct io_thread_req *io_req)
-{
- struct gendisk *disk = req->rq_disk;
- struct ubd *ubd_dev = disk->private_data;
+ io_req = kmalloc(sizeof(struct io_thread_req), GFP_ATOMIC);
+ if (!io_req)
+ return -ENOMEM;
io_req->req = req;
- io_req->fds[0] = (ubd_dev->cow.file != NULL) ? ubd_dev->cow.fd :
- ubd_dev->fd;
- io_req->op = UBD_FLUSH;
-}
+ if (dev->cow.file)
+ io_req->fds[0] = dev->cow.fd;
+ else
+ io_req->fds[0] = dev->fd;
-static bool submit_request(struct io_thread_req *io_req, struct ubd *dev)
-{
- int n = os_write_file(thread_fd, &io_req,
- sizeof(io_req));
- if (n != sizeof(io_req)) {
- if (n != -EAGAIN)
- printk("write to io thread failed, "
- "errno = %d\n", -n);
- else if (list_empty(&dev->restart))
- list_add(&dev->restart, &restart);
+ if (req_op(req) == REQ_OP_FLUSH) {
+ io_req->op = UBD_FLUSH;
+ } else {
+ io_req->fds[1] = dev->fd;
+ io_req->cow_offset = -1;
+ io_req->offset = off;
+ io_req->length = bvec->bv_len;
+ io_req->error = 0;
+ io_req->sector_mask = 0;
+
+ io_req->op = rq_data_dir(req) == READ ? UBD_READ : UBD_WRITE;
+ io_req->offsets[0] = 0;
+ io_req->offsets[1] = dev->cow.data_offset;
+ io_req->buffer = page_address(bvec->bv_page) + bvec->bv_offset;
+ io_req->sectorsize = 1 << 9;
+
+ if (dev->cow.file) {
+ cowify_req(io_req, dev->cow.bitmap,
+ dev->cow.bitmap_offset, dev->cow.bitmap_len);
+ }
+ }
+ ret = os_write_file(thread_fd, &io_req, sizeof(io_req));
+ if (ret != sizeof(io_req)) {
+ if (ret != -EAGAIN)
+ pr_err("write to io thread failed: %d\n", -ret);
kfree(io_req);
- return false;
}
- return true;
+
+ return ret;
}
-/* Called with dev->lock held */
-static void do_ubd_request(struct request_queue *q)
+static blk_status_t ubd_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
- struct io_thread_req *io_req;
- struct request *req;
-
- while(1){
- struct ubd *dev = q->queuedata;
- if(dev->request == NULL){
- struct request *req = blk_fetch_request(q);
- if(req == NULL)
- return;
-
- dev->request = req;
- dev->rq_pos = blk_rq_pos(req);
- dev->start_sg = 0;
- dev->end_sg = blk_rq_map_sg(q, req, dev->sg);
- }
-
- req = dev->request;
+ struct request *req = bd->rq;
+ int ret = 0;
- if (req_op(req) == REQ_OP_FLUSH) {
- io_req = kmalloc(sizeof(struct io_thread_req),
- GFP_ATOMIC);
- if (io_req == NULL) {
- if (list_empty(&dev->restart))
- list_add(&dev->restart, &restart);
- return;
- }
- prepare_flush_request(req, io_req);
- if (submit_request(io_req, dev) == false)
- return;
- }
+ blk_mq_start_request(req);
- while(dev->start_sg < dev->end_sg){
- struct scatterlist *sg = &dev->sg[dev->start_sg];
-
- io_req = kmalloc(sizeof(struct io_thread_req),
- GFP_ATOMIC);
- if(io_req == NULL){
- if(list_empty(&dev->restart))
- list_add(&dev->restart, &restart);
- return;
- }
- prepare_request(req, io_req,
- (unsigned long long)dev->rq_pos << 9,
- sg->offset, sg->length, sg_page(sg));
-
- if (submit_request(io_req, dev) == false)
- return;
-
- dev->rq_pos += sg->length >> 9;
- dev->start_sg++;
+ if (req_op(req) == REQ_OP_FLUSH) {
+ ret = ubd_queue_one_vec(hctx, req, 0, NULL);
+ } else {
+ struct req_iterator iter;
+ struct bio_vec bvec;
+ u64 off = (u64)blk_rq_pos(req) << 9;
+
+ rq_for_each_segment(bvec, req, iter) {
+ ret = ubd_queue_one_vec(hctx, req, off, &bvec);
+ if (ret < 0)
+ goto out;
+ off += bvec.bv_len;
}
- dev->end_sg = 0;
- dev->request = NULL;
}
+out:
+ if (ret < 0) {
+ blk_mq_requeue_request(req, true);
+ }
+ return BLK_STS_OK;
}
static int ubd_getgeo(struct block_device *bdev, struct hd_geometry *geo)
select ARCH_HAS_DEVMEM_IS_ALLOWED
select ARCH_MIGHT_HAVE_PC_PARPORT
select ARCH_MIGHT_HAVE_PC_SERIO
+ select DMA_DIRECT_OPS
select HAVE_MEMBLOCK
select HAVE_GENERIC_DMA_COHERENT
select HAVE_KERNEL_GZIP
select GENERIC_IOMAP
select MODULES_USE_ELF_REL
select NEED_DMA_MAP_STATE
- select SWIOTLB
help
UniCore-32 is 32-bit Instruction Set Architecture,
including a series of low-power-consumption RISC chip
generic-y += current.h
generic-y += device.h
generic-y += div64.h
+generic-y += dma-mapping.h
generic-y += emergency-restart.h
generic-y += exec.h
generic-y += extable.h
+++ /dev/null
-/*
- * linux/arch/unicore32/include/asm/dma-mapping.h
- *
- * Code specific to PKUnity SoC and UniCore ISA
- *
- * Copyright (C) 2001-2010 GUAN Xue-tao
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-#ifndef __UNICORE_DMA_MAPPING_H__
-#define __UNICORE_DMA_MAPPING_H__
-
-#include <linux/swiotlb.h>
-
-static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
-{
- return &swiotlb_dma_ops;
-}
-
-#endif
uc32_bootmem_init(min, max_low);
-#ifdef CONFIG_SWIOTLB
- swiotlb_init(1);
-#endif
/*
* Sparsemem tries to allocate bootmem in memory_present(),
* so must be done after the fixed reservations
select HAVE_ARCH_AUDITSYSCALL
select HAVE_ARCH_HUGE_VMAP if X86_64 || X86_PAE
select HAVE_ARCH_JUMP_LABEL
+ select HAVE_ARCH_JUMP_LABEL_RELATIVE
select HAVE_ARCH_KASAN if X86_64
select HAVE_ARCH_KGDB
select HAVE_ARCH_MMAP_RND_BITS if MMU
config INTEL_RDT
bool "Intel Resource Director Technology support"
- default n
depends on X86 && CPU_SUP_INTEL
select KERNFS
help
select SWIOTLB
select MFD_STA2X11
select GPIOLIB
- default n
---help---
This adds support for boards based on the STA2X11 IO-Hub,
a.k.a. "ConneXt". The chip is used in place of the standard
config KVM_DEBUG_FS
bool "Enable debug information for KVM Guests in debugfs"
depends on KVM_GUEST && DEBUG_FS
- default n
---help---
This option enables collection of various statistics for KVM guest.
Statistics are displayed in debugfs filesystem. Enabling this option
config PARAVIRT_TIME_ACCOUNTING
bool "Paravirtual steal time accounting"
depends on PARAVIRT
- default n
---help---
Select this option to enable fine granularity task steal time
accounting. Time spent executing other tasks in parallel with
config X86_LEGACY_VM86
bool "Legacy VM86 support"
- default n
depends on X86_32
---help---
This option allows user programs to put the CPU into V8086
config BOOTPARAM_HOTPLUG_CPU0
bool "Set default setting of cpu0_hotpluggable"
- default n
depends on HOTPLUG_CPU
---help---
Set whether default state of cpu0_hotpluggable is on or off.
config ARCH_HIBERNATION_HEADER
def_bool y
- depends on X86_64 && HIBERNATION
+ depends on HIBERNATION
source "kernel/power/Kconfig"
config RAPIDIO
tristate "RapidIO support"
depends on PCI
- default n
help
If enabled this option will include drivers and the core
infrastructure code to support RapidIO interconnect devices.
If unsure, say N.
+config CPU_SUP_HYGON
+ default y
+ bool "Support Hygon processors" if PROCESSOR_SELECT
+ select CPU_SUP_AMD
+ help
+ This enables detection, tunings and quirks for Hygon processors
+
+ You need this enabled if you want your kernel to run on an
+ Hygon CPU. Disabling this option on other types of CPUs
+ makes the kernel a tiny bit smaller. Disabling it on an Hygon
+ CPU might render the kernel unbootable.
+
+ If unsure, say N.
+
config CPU_SUP_CENTAUR
default y
bool "Support Centaur processors" if PROCESSOR_SELECT
config DEBUG_IMR_SELFTEST
bool "Isolated Memory Region self test"
- default n
depends on INTEL_IMR
---help---
This option enables automated sanity testing of the IMR code.
# does binutils support specific instructions?
asinstr := $(call as-instr,fxsaveq (%rax),-DCONFIG_AS_FXSAVEQ=1)
asinstr += $(call as-instr,pshufb %xmm0$(comma)%xmm0,-DCONFIG_AS_SSSE3=1)
-asinstr += $(call as-instr,crc32l %eax$(comma)%eax,-DCONFIG_AS_CRC32=1)
avx_instr := $(call as-instr,vxorps %ymm0$(comma)%ymm1$(comma)%ymm2,-DCONFIG_AS_AVX=1)
avx2_instr :=$(call as-instr,vpbroadcastb %xmm0$(comma)%ymm1,-DCONFIG_AS_AVX2=1)
avx512_instr :=$(call as-instr,vpmovm2b %k1$(comma)%zmm5,-DCONFIG_AS_AVX512=1)
archheaders:
$(Q)$(MAKE) $(build)=arch/x86/entry/syscalls all
+archmacros:
+ $(Q)$(MAKE) $(build)=arch/x86/kernel arch/x86/kernel/macros.s
+
+ASM_MACRO_FLAGS = -Wa,arch/x86/kernel/macros.s -Wa,-
+export ASM_MACRO_FLAGS
+KBUILD_CFLAGS += $(ASM_MACRO_FLAGS)
+
###
# Kernel objects
KBUILD_CFLAGS += $(call cc-option,-fno-stack-protector)
KBUILD_CFLAGS += $(call cc-disable-warning, address-of-packed-member)
KBUILD_CFLAGS += $(call cc-disable-warning, gnu)
+KBUILD_CFLAGS += -Wno-pointer-sign
KBUILD_AFLAGS := $(KBUILD_CFLAGS) -D__ASSEMBLY__
GCOV_PROFILE := n
struct desc_struct *desc;
void *handle;
efi_system_table_t *_table;
+ unsigned long cmdline_paddr;
efi_early = c;
else
setup_boot_services32(efi_early);
+ /*
+ * make_boot_params() may have been called before efi_main(), in which
+ * case this is the second time we parse the cmdline. This is ok,
+ * parsing the cmdline multiple times does not have side-effects.
+ */
+ cmdline_paddr = ((u64)hdr->cmd_line_ptr |
+ ((u64)boot_params->ext_cmd_line_ptr << 32));
+ efi_parse_options((char *)cmdline_paddr);
+
/*
* If the boot loader gave us a value for secure_boot then we use that,
* otherwise we ask the BIOS.
}
-static int handle_mem_options(void)
+static void handle_mem_options(void)
{
char *args = (char *)get_cmd_line_ptr();
size_t len = strlen((char *)args);
if (!strstr(args, "memmap=") && !strstr(args, "mem=") &&
!strstr(args, "hugepages"))
- return 0;
+ return;
tmp_cmdline = malloc(len + 1);
if (!tmp_cmdline)
/* Stop at -- */
if (!val && strcmp(param, "--") == 0) {
warn("Only '--' specified in cmdline");
- free(tmp_cmdline);
- return -1;
+ goto out;
}
if (!strcmp(param, "memmap")) {
if (!strcmp(p, "nopentium"))
continue;
mem_size = memparse(p, &p);
- if (mem_size == 0) {
- free(tmp_cmdline);
- return -EINVAL;
- }
+ if (mem_size == 0)
+ goto out;
+
mem_limit = mem_size;
}
}
+out:
free(tmp_cmdline);
- return 0;
+ return;
}
/*
unsigned long image_size)
{
struct mem_vector region, overlap;
- struct slot_area slot_area;
unsigned long start_orig, end;
struct mem_vector cur_entry;
die("Unable to mmap '%s': %m", argv[2]);
/* Number of 16-byte paragraphs, including space for a 4-byte CRC */
sys_size = (sz + 15 + 4) / 16;
+#ifdef CONFIG_EFI_STUB
+ /*
+ * COFF requires minimum 32-byte alignment of sections, and
+ * adding a signature is problematic without that alignment.
+ */
+ sys_size = (sys_size + 1) & ~1;
+#endif
/* Patch the setup code with the appropriate size parameters */
buf[0x1f1] = setup_sectors-1;
CONFIG_USB=y
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
CONFIG_USB_MON=y
+CONFIG_USB_XHCI_HCD=y
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_EHCI_TT_NEWSCHED=y
CONFIG_USB_OHCI_HCD=y
CONFIG_USB=y
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
CONFIG_USB_MON=y
+CONFIG_USB_XHCI_HCD=y
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_EHCI_TT_NEWSCHED=y
CONFIG_USB_OHCI_HCD=y
.macro CALL_enter_from_user_mode
#ifdef CONFIG_CONTEXT_TRACKING
#ifdef HAVE_JUMP_LABEL
- STATIC_JUMP_IF_FALSE .Lafter_call_\@, context_tracking_enabled, def=0
+ STATIC_BRANCH_JMP l_yes=.Lafter_call_\@, key=context_tracking_enabled, branch=1
#endif
call enter_from_user_mode
.Lafter_call_\@:
* that register for the time this macro runs
*/
+ /*
+ * The high bits of the CS dword (__csh) are used for
+ * CS_FROM_ENTRY_STACK and CS_FROM_USER_CR3. Clear them in case
+ * hardware didn't do this for us.
+ */
+ andl $(0x0000ffff), PT_CS(%esp)
+
/* Are we on the entry stack? Bail out if not! */
movl PER_CPU_VAR(cpu_entry_area), %ecx
addl $CPU_ENTRY_AREA_entry_stack + SIZEOF_entry_stack, %ecx
/* Load top of task-stack into %edi */
movl TSS_entry2task_stack(%edi), %edi
- /*
- * Clear unused upper bits of the dword containing the word-sized CS
- * slot in pt_regs in case hardware didn't clear it for us.
- */
- andl $(0x0000ffff), PT_CS(%esp)
-
/* Special case - entry from kernel mode via entry stack */
#ifdef CONFIG_VM86
movl PT_EFLAGS(%esp), %ecx # mix EFLAGS and CS
xorl %ebx, %ebx
1:
+ /*
+ * Always stash CR3 in %r14. This value will be restored,
+ * verbatim, at exit. Needed if paranoid_entry interrupted
+ * another entry that already switched to the user CR3 value
+ * but has not yet returned to userspace.
+ *
+ * This is also why CS (stashed in the "iret frame" by the
+ * hardware at entry) can not be used: this may be a return
+ * to kernel code, but with a user CR3 value.
+ */
SAVE_AND_SWITCH_TO_KERNEL_CR3 scratch_reg=%rax save_reg=%r14
ret
testl %ebx, %ebx /* swapgs needed? */
jnz .Lparanoid_exit_no_swapgs
TRACE_IRQS_IRETQ
+ /* Always restore stashed CR3 value (see paranoid_entry) */
RESTORE_CR3 scratch_reg=%rbx save_reg=%r14
SWAPGS_UNSAFE_STACK
jmp .Lparanoid_exit_restore
.Lparanoid_exit_no_swapgs:
TRACE_IRQS_IRETQ_DEBUG
+ /* Always restore stashed CR3 value (see paranoid_entry) */
RESTORE_CR3 scratch_reg=%rbx save_reg=%r14
.Lparanoid_exit_restore:
jmp restore_regs_and_return_to_kernel
movq $-1, %rsi
call do_nmi
+ /* Always restore stashed CR3 value (see paranoid_entry) */
RESTORE_CR3 scratch_reg=%r15 save_reg=%r14
testl %ebx, %ebx /* swapgs needed? */
notrace long
__vdso_getcpu(unsigned *cpu, unsigned *node, struct getcpu_cache *unused)
{
- unsigned int p;
+ vdso_read_cpunode(cpu, node);
- p = __getcpu();
-
- if (cpu)
- *cpu = p & VGETCPU_CPU_MASK;
- if (node)
- *node = p >> 12;
return 0;
}
return 0;
}
__setup("vdso=", vdso_setup);
-#endif
-
-#ifdef CONFIG_X86_64
-static void vgetcpu_cpu_init(void *arg)
-{
- int cpu = smp_processor_id();
- struct desc_struct d = { };
- unsigned long node = 0;
-#ifdef CONFIG_NUMA
- node = cpu_to_node(cpu);
-#endif
- if (static_cpu_has(X86_FEATURE_RDTSCP))
- write_rdtscp_aux((node << 12) | cpu);
-
- /*
- * Store cpu number in limit so that it can be loaded
- * quickly in user space in vgetcpu. (12 bits for the CPU
- * and 8 bits for the node)
- */
- d.limit0 = cpu | ((node & 0xf) << 12);
- d.limit1 = node >> 4;
- d.type = 5; /* RO data, expand down, accessed */
- d.dpl = 3; /* Visible to user code */
- d.s = 1; /* Not a system segment */
- d.p = 1; /* Present */
- d.d = 1; /* 32-bit */
-
- write_gdt_entry(get_cpu_gdt_rw(cpu), GDT_ENTRY_PER_CPU, &d, DESCTYPE_S);
-}
-
-static int vgetcpu_online(unsigned int cpu)
-{
- return smp_call_function_single(cpu, vgetcpu_cpu_init, NULL, 1);
-}
static int __init init_vdso(void)
{
init_vdso_image(&vdso_image_x32);
#endif
- /* notifier priority > KVM */
- return cpuhp_setup_state(CPUHP_AP_X86_VDSO_VMA_ONLINE,
- "x86/vdso/vma:online", vgetcpu_online, NULL);
+ return 0;
}
subsys_initcall(init_vdso);
#endif /* CONFIG_X86_64 */
* We fallback to using default amd_get_event_constraints.
*/
break;
+ case 0x18:
+ pr_cont("Fam18h ");
+ /* Using default amd_get_event_constraints. */
+ break;
default:
pr_err("core perfctr but no constraints; unknown hardware!\n");
return -ENODEV;
{
int ret = -ENODEV;
- if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD)
+ if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD &&
+ boot_cpu_data.x86_vendor != X86_VENDOR_HYGON)
return -ENODEV;
if (!boot_cpu_has(X86_FEATURE_TOPOEXT))
return -ENODEV;
- if (boot_cpu_data.x86 == 0x17) {
+ if (boot_cpu_data.x86 == 0x17 || boot_cpu_data.x86 == 0x18) {
/*
- * For F17h, the Northbridge counters are repurposed as Data
- * Fabric counters. Also, L3 counters are supported too. The PMUs
- * are exported based on family as either L2 or L3 and NB or DF.
+ * For F17h or F18h, the Northbridge counters are
+ * repurposed as Data Fabric counters. Also, L3
+ * counters are supported too. The PMUs are exported
+ * based on family as either L2 or L3 and NB or DF.
*/
num_counters_nb = NUM_COUNTERS_NB;
num_counters_llc = NUM_COUNTERS_L3;
if (ret)
goto fail_nb;
- pr_info("AMD NB counters detected\n");
+ pr_info("%s NB counters detected\n",
+ boot_cpu_data.x86_vendor == X86_VENDOR_HYGON ?
+ "HYGON" : "AMD");
ret = 0;
}
if (ret)
goto fail_llc;
- pr_info("AMD LLC counters detected\n");
+ pr_info("%s LLC counters detected\n",
+ boot_cpu_data.x86_vendor == X86_VENDOR_HYGON ?
+ "HYGON" : "AMD");
ret = 0;
}
}
}
+/**
+ * x86_perf_rdpmc_index - Return PMC counter used for event
+ * @event: the perf_event to which the PMC counter was assigned
+ *
+ * The counter assigned to this performance event may change if interrupts
+ * are enabled. This counter should thus never be used while interrupts are
+ * enabled. Before this function is used to obtain the assigned counter the
+ * event should be checked for validity using, for example,
+ * perf_event_read_local(), within the same interrupt disabled section in
+ * which this counter is planned to be used.
+ *
+ * Return: The index of the performance monitoring counter assigned to
+ * @perf_event.
+ */
+int x86_perf_rdpmc_index(struct perf_event *event)
+{
+ lockdep_assert_irqs_disabled();
+
+ return event->hw.event_base_rdpmc;
+}
+
static inline int match_prev_assignment(struct hw_perf_event *hwc,
struct cpu_hw_events *cpuc,
int i)
}
-static struct attribute_group x86_pmu_format_group = {
+static struct attribute_group x86_pmu_format_group __ro_after_init = {
.name = "format",
.attrs = NULL,
};
struct attribute **new;
int j, i;
- for (j = 0; a[j]; j++)
+ for (j = 0; a && a[j]; j++)
;
- for (i = 0; b[i]; i++)
+ for (i = 0; b && b[i]; i++)
j++;
j++;
return NULL;
j = 0;
- for (i = 0; a[i]; i++)
+ for (i = 0; a && a[i]; i++)
new[j++] = a[i];
- for (i = 0; b[i]; i++)
+ for (i = 0; b && b[i]; i++)
new[j++] = b[i];
new[j] = NULL;
NULL,
};
-static struct attribute_group x86_pmu_events_group = {
+static struct attribute_group x86_pmu_events_group __ro_after_init = {
.name = "events",
.attrs = events_attr,
};
case X86_VENDOR_AMD:
err = amd_pmu_init();
break;
+ case X86_VENDOR_HYGON:
+ err = amd_pmu_init();
+ x86_pmu.name = "HYGON";
+ break;
default:
err = -ENOTSUPP;
}
NULL,
};
-static struct attribute_group x86_pmu_attr_group = {
+static struct attribute_group x86_pmu_attr_group __ro_after_init = {
.attrs = x86_pmu_attrs,
};
NULL
};
-static struct attribute_group x86_pmu_caps_group = {
+static struct attribute_group x86_pmu_caps_group __ro_after_init = {
.name = "caps",
.attrs = x86_pmu_caps_attrs,
};
EVENT_ATTR_STR(mem-loads, mem_ld_snb, "event=0xcd,umask=0x1,ldlat=3");
EVENT_ATTR_STR(mem-stores, mem_st_snb, "event=0xcd,umask=0x2");
-static struct attribute *nhm_events_attrs[] = {
+static struct attribute *nhm_mem_events_attrs[] = {
EVENT_PTR(mem_ld_nhm),
NULL,
};
"4", "2");
static struct attribute *snb_events_attrs[] = {
- EVENT_PTR(mem_ld_snb),
- EVENT_PTR(mem_st_snb),
EVENT_PTR(td_slots_issued),
EVENT_PTR(td_slots_retired),
EVENT_PTR(td_fetch_bubbles),
NULL,
};
+static struct attribute *snb_mem_events_attrs[] = {
+ EVENT_PTR(mem_ld_snb),
+ EVENT_PTR(mem_st_snb),
+ NULL,
+};
+
static struct event_constraint intel_hsw_event_constraints[] = {
FIXED_EVENT_CONSTRAINT(0x00c0, 0), /* INST_RETIRED.ANY */
FIXED_EVENT_CONSTRAINT(0x003c, 1), /* CPU_CLK_UNHALTED.CORE */
intel_pmu_enable_all(added);
}
+static void enable_counter_freeze(void)
+{
+ update_debugctlmsr(get_debugctlmsr() |
+ DEBUGCTLMSR_FREEZE_PERFMON_ON_PMI);
+}
+
+static void disable_counter_freeze(void)
+{
+ update_debugctlmsr(get_debugctlmsr() &
+ ~DEBUGCTLMSR_FREEZE_PERFMON_ON_PMI);
+}
+
static inline u64 intel_pmu_get_status(void)
{
u64 status;
local_irq_restore(flags);
}
-/*
- * This handler is triggered by the local APIC, so the APIC IRQ handling
- * rules apply:
- */
-static int intel_pmu_handle_irq(struct pt_regs *regs)
+static int handle_pmi_common(struct pt_regs *regs, u64 status)
{
struct perf_sample_data data;
- struct cpu_hw_events *cpuc;
- int bit, loops;
- u64 status;
- int handled;
- int pmu_enabled;
-
- cpuc = this_cpu_ptr(&cpu_hw_events);
-
- /*
- * Save the PMU state.
- * It needs to be restored when leaving the handler.
- */
- pmu_enabled = cpuc->enabled;
- /*
- * No known reason to not always do late ACK,
- * but just in case do it opt-in.
- */
- if (!x86_pmu.late_ack)
- apic_write(APIC_LVTPC, APIC_DM_NMI);
- intel_bts_disable_local();
- cpuc->enabled = 0;
- __intel_pmu_disable_all();
- handled = intel_pmu_drain_bts_buffer();
- handled += intel_bts_interrupt();
- status = intel_pmu_get_status();
- if (!status)
- goto done;
-
- loops = 0;
-again:
- intel_pmu_lbr_read();
- intel_pmu_ack_status(status);
- if (++loops > 100) {
- static bool warned = false;
- if (!warned) {
- WARN(1, "perfevents: irq loop stuck!\n");
- perf_event_print_debug();
- warned = true;
- }
- intel_pmu_reset();
- goto done;
- }
+ struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
+ int bit;
+ int handled = 0;
inc_irq_stat(apic_perf_irqs);
-
/*
* Ignore a range of extra bits in status that do not indicate
* overflow by themselves.
GLOBAL_STATUS_ASIF |
GLOBAL_STATUS_LBRS_FROZEN);
if (!status)
- goto done;
+ return 0;
/*
* In case multiple PEBS events are sampled at the same time,
* it is possible to have GLOBAL_STATUS bit 62 set indicating
x86_pmu_stop(event, 0);
}
+ return handled;
+}
+
+static bool disable_counter_freezing;
+static int __init intel_perf_counter_freezing_setup(char *s)
+{
+ disable_counter_freezing = true;
+ pr_info("Intel PMU Counter freezing feature disabled\n");
+ return 1;
+}
+__setup("disable_counter_freezing", intel_perf_counter_freezing_setup);
+
+/*
+ * Simplified handler for Arch Perfmon v4:
+ * - We rely on counter freezing/unfreezing to enable/disable the PMU.
+ * This is done automatically on PMU ack.
+ * - Ack the PMU only after the APIC.
+ */
+
+static int intel_pmu_handle_irq_v4(struct pt_regs *regs)
+{
+ struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
+ int handled = 0;
+ bool bts = false;
+ u64 status;
+ int pmu_enabled = cpuc->enabled;
+ int loops = 0;
+
+ /* PMU has been disabled because of counter freezing */
+ cpuc->enabled = 0;
+ if (test_bit(INTEL_PMC_IDX_FIXED_BTS, cpuc->active_mask)) {
+ bts = true;
+ intel_bts_disable_local();
+ handled = intel_pmu_drain_bts_buffer();
+ handled += intel_bts_interrupt();
+ }
+ status = intel_pmu_get_status();
+ if (!status)
+ goto done;
+again:
+ intel_pmu_lbr_read();
+ if (++loops > 100) {
+ static bool warned;
+
+ if (!warned) {
+ WARN(1, "perfevents: irq loop stuck!\n");
+ perf_event_print_debug();
+ warned = true;
+ }
+ intel_pmu_reset();
+ goto done;
+ }
+
+
+ handled += handle_pmi_common(regs, status);
+done:
+ /* Ack the PMI in the APIC */
+ apic_write(APIC_LVTPC, APIC_DM_NMI);
+
+ /*
+ * The counters start counting immediately while ack the status.
+ * Make it as close as possible to IRET. This avoids bogus
+ * freezing on Skylake CPUs.
+ */
+ if (status) {
+ intel_pmu_ack_status(status);
+ } else {
+ /*
+ * CPU may issues two PMIs very close to each other.
+ * When the PMI handler services the first one, the
+ * GLOBAL_STATUS is already updated to reflect both.
+ * When it IRETs, the second PMI is immediately
+ * handled and it sees clear status. At the meantime,
+ * there may be a third PMI, because the freezing bit
+ * isn't set since the ack in first PMI handlers.
+ * Double check if there is more work to be done.
+ */
+ status = intel_pmu_get_status();
+ if (status)
+ goto again;
+ }
+
+ if (bts)
+ intel_bts_enable_local();
+ cpuc->enabled = pmu_enabled;
+ return handled;
+}
+
+/*
+ * This handler is triggered by the local APIC, so the APIC IRQ handling
+ * rules apply:
+ */
+static int intel_pmu_handle_irq(struct pt_regs *regs)
+{
+ struct cpu_hw_events *cpuc;
+ int loops;
+ u64 status;
+ int handled;
+ int pmu_enabled;
+
+ cpuc = this_cpu_ptr(&cpu_hw_events);
+
+ /*
+ * Save the PMU state.
+ * It needs to be restored when leaving the handler.
+ */
+ pmu_enabled = cpuc->enabled;
+ /*
+ * No known reason to not always do late ACK,
+ * but just in case do it opt-in.
+ */
+ if (!x86_pmu.late_ack)
+ apic_write(APIC_LVTPC, APIC_DM_NMI);
+ intel_bts_disable_local();
+ cpuc->enabled = 0;
+ __intel_pmu_disable_all();
+ handled = intel_pmu_drain_bts_buffer();
+ handled += intel_bts_interrupt();
+ status = intel_pmu_get_status();
+ if (!status)
+ goto done;
+
+ loops = 0;
+again:
+ intel_pmu_lbr_read();
+ intel_pmu_ack_status(status);
+ if (++loops > 100) {
+ static bool warned;
+
+ if (!warned) {
+ WARN(1, "perfevents: irq loop stuck!\n");
+ perf_event_print_debug();
+ warned = true;
+ }
+ intel_pmu_reset();
+ goto done;
+ }
+
+ handled += handle_pmi_common(regs, status);
+
/*
* Repeat if there is more work to be done:
*/
if (x86_pmu.version > 1)
flip_smm_bit(&x86_pmu.attr_freeze_on_smi);
+ if (x86_pmu.counter_freezing)
+ enable_counter_freeze();
+
if (!cpuc->shared_regs)
return;
free_excl_cntrs(cpu);
fini_debug_store_on_cpu(cpu);
+
+ if (x86_pmu.counter_freezing)
+ disable_counter_freeze();
}
static void intel_pmu_sched_task(struct perf_event_context *ctx,
}
}
+static bool intel_glp_counter_freezing_broken(int cpu)
+{
+ u32 rev = UINT_MAX; /* default to broken for unknown stepping */
+
+ switch (cpu_data(cpu).x86_stepping) {
+ case 1:
+ rev = 0x28;
+ break;
+ case 8:
+ rev = 0x6;
+ break;
+ }
+
+ return (cpu_data(cpu).microcode < rev);
+}
+
+static __init void intel_glp_counter_freezing_quirk(void)
+{
+ /* Check if it's already disabled */
+ if (disable_counter_freezing)
+ return;
+
+ /*
+ * If the system starts with the wrong ucode, leave the
+ * counter-freezing feature permanently disabled.
+ */
+ if (intel_glp_counter_freezing_broken(raw_smp_processor_id())) {
+ pr_info("PMU counter freezing disabled due to CPU errata,"
+ "please upgrade microcode\n");
+ x86_pmu.counter_freezing = false;
+ x86_pmu.handle_irq = intel_pmu_handle_irq;
+ }
+}
+
/*
* enable software workaround for errata:
* SNB: BJ122
EVENT_ATTR_STR(cycles-ct, cycles_ct, "event=0x3c,in_tx=1,in_tx_cp=1");
static struct attribute *hsw_events_attrs[] = {
- EVENT_PTR(mem_ld_hsw),
- EVENT_PTR(mem_st_hsw),
EVENT_PTR(td_slots_issued),
EVENT_PTR(td_slots_retired),
EVENT_PTR(td_fetch_bubbles),
NULL
};
+static struct attribute *hsw_mem_events_attrs[] = {
+ EVENT_PTR(mem_ld_hsw),
+ EVENT_PTR(mem_st_hsw),
+ NULL,
+};
+
static struct attribute *hsw_tsx_events_attrs[] = {
EVENT_PTR(tx_start),
EVENT_PTR(tx_commit),
NULL
};
-static __init struct attribute **get_hsw_events_attrs(void)
-{
- return boot_cpu_has(X86_FEATURE_RTM) ?
- merge_attr(hsw_events_attrs, hsw_tsx_events_attrs) :
- hsw_events_attrs;
-}
-
static ssize_t freeze_on_smi_show(struct device *cdev,
struct device_attribute *attr,
char *buf)
NULL,
};
+static __init struct attribute **
+get_events_attrs(struct attribute **base,
+ struct attribute **mem,
+ struct attribute **tsx)
+{
+ struct attribute **attrs = base;
+ struct attribute **old;
+
+ if (mem && x86_pmu.pebs)
+ attrs = merge_attr(attrs, mem);
+
+ if (tsx && boot_cpu_has(X86_FEATURE_RTM)) {
+ old = attrs;
+ attrs = merge_attr(attrs, tsx);
+ if (old != base)
+ kfree(old);
+ }
+
+ return attrs;
+}
+
__init int intel_pmu_init(void)
{
struct attribute **extra_attr = NULL;
+ struct attribute **mem_attr = NULL;
+ struct attribute **tsx_attr = NULL;
struct attribute **to_free = NULL;
union cpuid10_edx edx;
union cpuid10_eax eax;
max((int)edx.split.num_counters_fixed, assume);
}
+ if (version >= 4)
+ x86_pmu.counter_freezing = !disable_counter_freezing;
+
if (boot_cpu_has(X86_FEATURE_PDCM)) {
u64 capabilities;
x86_pmu.enable_all = intel_pmu_nhm_enable_all;
x86_pmu.extra_regs = intel_nehalem_extra_regs;
- x86_pmu.cpu_events = nhm_events_attrs;
+ mem_attr = nhm_mem_events_attrs;
/* UOPS_ISSUED.STALLED_CYCLES */
intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
name = "nehalem";
break;
- case INTEL_FAM6_ATOM_PINEVIEW:
- case INTEL_FAM6_ATOM_LINCROFT:
- case INTEL_FAM6_ATOM_PENWELL:
- case INTEL_FAM6_ATOM_CLOVERVIEW:
- case INTEL_FAM6_ATOM_CEDARVIEW:
+ case INTEL_FAM6_ATOM_BONNELL:
+ case INTEL_FAM6_ATOM_BONNELL_MID:
+ case INTEL_FAM6_ATOM_SALTWELL:
+ case INTEL_FAM6_ATOM_SALTWELL_MID:
+ case INTEL_FAM6_ATOM_SALTWELL_TABLET:
memcpy(hw_cache_event_ids, atom_hw_cache_event_ids,
sizeof(hw_cache_event_ids));
name = "bonnell";
break;
- case INTEL_FAM6_ATOM_SILVERMONT1:
- case INTEL_FAM6_ATOM_SILVERMONT2:
+ case INTEL_FAM6_ATOM_SILVERMONT:
+ case INTEL_FAM6_ATOM_SILVERMONT_X:
+ case INTEL_FAM6_ATOM_SILVERMONT_MID:
case INTEL_FAM6_ATOM_AIRMONT:
+ case INTEL_FAM6_ATOM_AIRMONT_MID:
memcpy(hw_cache_event_ids, slm_hw_cache_event_ids,
sizeof(hw_cache_event_ids));
memcpy(hw_cache_extra_regs, slm_hw_cache_extra_regs,
break;
case INTEL_FAM6_ATOM_GOLDMONT:
- case INTEL_FAM6_ATOM_DENVERTON:
+ case INTEL_FAM6_ATOM_GOLDMONT_X:
memcpy(hw_cache_event_ids, glm_hw_cache_event_ids,
sizeof(hw_cache_event_ids));
memcpy(hw_cache_extra_regs, glm_hw_cache_extra_regs,
name = "goldmont";
break;
- case INTEL_FAM6_ATOM_GEMINI_LAKE:
+ case INTEL_FAM6_ATOM_GOLDMONT_PLUS:
+ x86_add_quirk(intel_glp_counter_freezing_quirk);
memcpy(hw_cache_event_ids, glp_hw_cache_event_ids,
sizeof(hw_cache_event_ids));
memcpy(hw_cache_extra_regs, glp_hw_cache_extra_regs,
x86_pmu.extra_regs = intel_westmere_extra_regs;
x86_pmu.flags |= PMU_FL_HAS_RSP_1;
- x86_pmu.cpu_events = nhm_events_attrs;
+ mem_attr = nhm_mem_events_attrs;
/* UOPS_ISSUED.STALLED_CYCLES */
intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
x86_pmu.flags |= PMU_FL_NO_HT_SHARING;
x86_pmu.cpu_events = snb_events_attrs;
+ mem_attr = snb_mem_events_attrs;
/* UOPS_ISSUED.ANY,c=1,i=1 to count stall cycles */
intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
x86_pmu.flags |= PMU_FL_NO_HT_SHARING;
x86_pmu.cpu_events = snb_events_attrs;
+ mem_attr = snb_mem_events_attrs;
/* UOPS_ISSUED.ANY,c=1,i=1 to count stall cycles */
intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
x86_pmu.hw_config = hsw_hw_config;
x86_pmu.get_event_constraints = hsw_get_event_constraints;
- x86_pmu.cpu_events = get_hsw_events_attrs();
+ x86_pmu.cpu_events = hsw_events_attrs;
x86_pmu.lbr_double_abort = true;
extra_attr = boot_cpu_has(X86_FEATURE_RTM) ?
hsw_format_attr : nhm_format_attr;
+ mem_attr = hsw_mem_events_attrs;
+ tsx_attr = hsw_tsx_events_attrs;
pr_cont("Haswell events, ");
name = "haswell";
break;
x86_pmu.hw_config = hsw_hw_config;
x86_pmu.get_event_constraints = hsw_get_event_constraints;
- x86_pmu.cpu_events = get_hsw_events_attrs();
+ x86_pmu.cpu_events = hsw_events_attrs;
x86_pmu.limit_period = bdw_limit_period;
extra_attr = boot_cpu_has(X86_FEATURE_RTM) ?
hsw_format_attr : nhm_format_attr;
+ mem_attr = hsw_mem_events_attrs;
+ tsx_attr = hsw_tsx_events_attrs;
pr_cont("Broadwell events, ");
name = "broadwell";
break;
hsw_format_attr : nhm_format_attr;
extra_attr = merge_attr(extra_attr, skl_format_attr);
to_free = extra_attr;
- x86_pmu.cpu_events = get_hsw_events_attrs();
+ x86_pmu.cpu_events = hsw_events_attrs;
+ mem_attr = hsw_mem_events_attrs;
+ tsx_attr = hsw_tsx_events_attrs;
intel_pmu_pebs_data_source_skl(
boot_cpu_data.x86_model == INTEL_FAM6_SKYLAKE_X);
pr_cont("Skylake events, ");
WARN_ON(!x86_pmu.format_attrs);
}
+ x86_pmu.cpu_events = get_events_attrs(x86_pmu.cpu_events,
+ mem_attr, tsx_attr);
+
if (x86_pmu.num_counters > INTEL_PMC_MAX_GENERIC) {
WARN(1, KERN_ERR "hw perf events %d > max(%d), clipping!",
x86_pmu.num_counters, INTEL_PMC_MAX_GENERIC);
pr_cont("full-width counters, ");
}
+ /*
+ * For arch perfmon 4 use counter freezing to avoid
+ * several MSR accesses in the PMI.
+ */
+ if (x86_pmu.counter_freezing)
+ x86_pmu.handle_irq = intel_pmu_handle_irq_v4;
+
kfree(to_free);
return 0;
}
X86_CSTATES_MODEL(INTEL_FAM6_HASWELL_ULT, hswult_cstates),
- X86_CSTATES_MODEL(INTEL_FAM6_ATOM_SILVERMONT1, slm_cstates),
- X86_CSTATES_MODEL(INTEL_FAM6_ATOM_SILVERMONT2, slm_cstates),
+ X86_CSTATES_MODEL(INTEL_FAM6_ATOM_SILVERMONT, slm_cstates),
+ X86_CSTATES_MODEL(INTEL_FAM6_ATOM_SILVERMONT_X, slm_cstates),
X86_CSTATES_MODEL(INTEL_FAM6_ATOM_AIRMONT, slm_cstates),
X86_CSTATES_MODEL(INTEL_FAM6_BROADWELL_CORE, snb_cstates),
X86_CSTATES_MODEL(INTEL_FAM6_XEON_PHI_KNM, knl_cstates),
X86_CSTATES_MODEL(INTEL_FAM6_ATOM_GOLDMONT, glm_cstates),
- X86_CSTATES_MODEL(INTEL_FAM6_ATOM_DENVERTON, glm_cstates),
+ X86_CSTATES_MODEL(INTEL_FAM6_ATOM_GOLDMONT_X, glm_cstates),
- X86_CSTATES_MODEL(INTEL_FAM6_ATOM_GEMINI_LAKE, glm_cstates),
+ X86_CSTATES_MODEL(INTEL_FAM6_ATOM_GOLDMONT_PLUS, glm_cstates),
{ },
};
MODULE_DEVICE_TABLE(x86cpu, intel_cstates_match);
return snprintf(buf, PAGE_SIZE, "%x\n", pt_cap_get(cap));
}
-static struct attribute_group pt_cap_group = {
+static struct attribute_group pt_cap_group __ro_after_init = {
.name = "caps",
};
X86_RAPL_MODEL_MATCH(INTEL_FAM6_CANNONLAKE_MOBILE, skl_rapl_init),
X86_RAPL_MODEL_MATCH(INTEL_FAM6_ATOM_GOLDMONT, hsw_rapl_init),
- X86_RAPL_MODEL_MATCH(INTEL_FAM6_ATOM_DENVERTON, hsw_rapl_init),
+ X86_RAPL_MODEL_MATCH(INTEL_FAM6_ATOM_GOLDMONT_X, hsw_rapl_init),
- X86_RAPL_MODEL_MATCH(INTEL_FAM6_ATOM_GEMINI_LAKE, hsw_rapl_init),
+ X86_RAPL_MODEL_MATCH(INTEL_FAM6_ATOM_GOLDMONT_PLUS, hsw_rapl_init),
{},
};
case INTEL_FAM6_BROADWELL_GT3E:
case INTEL_FAM6_BROADWELL_X:
- case INTEL_FAM6_ATOM_SILVERMONT1:
- case INTEL_FAM6_ATOM_SILVERMONT2:
+ case INTEL_FAM6_ATOM_SILVERMONT:
+ case INTEL_FAM6_ATOM_SILVERMONT_X:
case INTEL_FAM6_ATOM_AIRMONT:
case INTEL_FAM6_ATOM_GOLDMONT:
- case INTEL_FAM6_ATOM_DENVERTON:
+ case INTEL_FAM6_ATOM_GOLDMONT_X:
- case INTEL_FAM6_ATOM_GEMINI_LAKE:
+ case INTEL_FAM6_ATOM_GOLDMONT_PLUS:
case INTEL_FAM6_XEON_PHI_KNL:
case INTEL_FAM6_XEON_PHI_KNM:
struct event_constraint *event_constraints;
struct x86_pmu_quirk *quirks;
int perfctr_second_write;
- bool late_ack;
u64 (*limit_period)(struct perf_event *event, u64 l);
+ /* PMI handler bits */
+ unsigned int late_ack :1,
+ counter_freezing :1;
/*
* sysfs attrs
*/
#include <asm/asm.h>
#ifdef CONFIG_SMP
- .macro LOCK_PREFIX
-672: lock
+.macro LOCK_PREFIX_HERE
.pushsection .smp_locks,"a"
.balign 4
- .long 672b - .
+ .long 671f - . # offset
.popsection
- .endm
+671:
+.endm
+
+.macro LOCK_PREFIX insn:vararg
+ LOCK_PREFIX_HERE
+ lock \insn
+.endm
#else
- .macro LOCK_PREFIX
- .endm
+.macro LOCK_PREFIX_HERE
+.endm
+
+.macro LOCK_PREFIX insn:vararg
+.endm
#endif
/*
*/
#ifdef CONFIG_SMP
-#define LOCK_PREFIX_HERE \
- ".pushsection .smp_locks,\"a\"\n" \
- ".balign 4\n" \
- ".long 671f - .\n" /* offset */ \
- ".popsection\n" \
- "671:"
-
-#define LOCK_PREFIX LOCK_PREFIX_HERE "\n\tlock; "
-
+#define LOCK_PREFIX_HERE "LOCK_PREFIX_HERE\n\t"
+#define LOCK_PREFIX "LOCK_PREFIX "
#else /* ! CONFIG_SMP */
#define LOCK_PREFIX_HERE ""
#define LOCK_PREFIX ""
static inline bool amd_gart_present(void)
{
+ if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD)
+ return false;
+
/* GART present only on Fam15h, upto model 0fh */
if (boot_cpu_data.x86 == 0xf || boot_cpu_data.x86 == 0x10 ||
(boot_cpu_data.x86 == 0x15 && boot_cpu_data.x86_model < 0x10))
/* Exception table entry */
#ifdef __ASSEMBLY__
# define _ASM_EXTABLE_HANDLE(from, to, handler) \
- .pushsection "__ex_table","a" ; \
- .balign 4 ; \
- .long (from) - . ; \
- .long (to) - . ; \
- .long (handler) - . ; \
+ ASM_EXTABLE_HANDLE from to handler
+
+.macro ASM_EXTABLE_HANDLE from:req to:req handler:req
+ .pushsection "__ex_table","a"
+ .balign 4
+ .long (\from) - .
+ .long (\to) - .
+ .long (\handler) - .
.popsection
+.endm
+#else /* __ASSEMBLY__ */
+
+# define _ASM_EXTABLE_HANDLE(from, to, handler) \
+ "ASM_EXTABLE_HANDLE from=" #from " to=" #to \
+ " handler=\"" #handler "\"\n\t"
+
+/* For C file, we already have NOKPROBE_SYMBOL macro */
+
+#endif /* __ASSEMBLY__ */
# define _ASM_EXTABLE(from, to) \
_ASM_EXTABLE_HANDLE(from, to, ex_handler_default)
+# define _ASM_EXTABLE_UA(from, to) \
+ _ASM_EXTABLE_HANDLE(from, to, ex_handler_uaccess)
+
# define _ASM_EXTABLE_FAULT(from, to) \
_ASM_EXTABLE_HANDLE(from, to, ex_handler_fault)
_ASM_PTR (entry); \
.popsection
+#ifdef __ASSEMBLY__
.macro ALIGN_DESTINATION
/* check for bad alignment of destination */
movl %edi,%ecx
jmp copy_user_handle_tail
.previous
- _ASM_EXTABLE(100b,103b)
- _ASM_EXTABLE(101b,103b)
+ _ASM_EXTABLE_UA(100b, 103b)
+ _ASM_EXTABLE_UA(101b, 103b)
.endm
-
-#else
-# define _EXPAND_EXTABLE_HANDLE(x) #x
-# define _ASM_EXTABLE_HANDLE(from, to, handler) \
- " .pushsection \"__ex_table\",\"a\"\n" \
- " .balign 4\n" \
- " .long (" #from ") - .\n" \
- " .long (" #to ") - .\n" \
- " .long (" _EXPAND_EXTABLE_HANDLE(handler) ") - .\n" \
- " .popsection\n"
-
-# define _ASM_EXTABLE(from, to) \
- _ASM_EXTABLE_HANDLE(from, to, ex_handler_default)
-
-# define _ASM_EXTABLE_FAULT(from, to) \
- _ASM_EXTABLE_HANDLE(from, to, ex_handler_fault)
-
-# define _ASM_EXTABLE_EX(from, to) \
- _ASM_EXTABLE_HANDLE(from, to, ex_handler_ext)
-
-# define _ASM_EXTABLE_REFCOUNT(from, to) \
- _ASM_EXTABLE_HANDLE(from, to, ex_handler_refcount)
-
-/* For C file, we already have NOKPROBE_SYMBOL macro */
-#endif
+#endif /* __ASSEMBLY__ */
#ifndef __ASSEMBLY__
/*
*/
static __always_inline bool arch_atomic_sub_and_test(int i, atomic_t *v)
{
- GEN_BINARY_RMWcc(LOCK_PREFIX "subl", v->counter, "er", i, "%0", e);
+ return GEN_BINARY_RMWcc(LOCK_PREFIX "subl", v->counter, e, "er", i);
}
#define arch_atomic_sub_and_test arch_atomic_sub_and_test
*/
static __always_inline bool arch_atomic_dec_and_test(atomic_t *v)
{
- GEN_UNARY_RMWcc(LOCK_PREFIX "decl", v->counter, "%0", e);
+ return GEN_UNARY_RMWcc(LOCK_PREFIX "decl", v->counter, e);
}
#define arch_atomic_dec_and_test arch_atomic_dec_and_test
*/
static __always_inline bool arch_atomic_inc_and_test(atomic_t *v)
{
- GEN_UNARY_RMWcc(LOCK_PREFIX "incl", v->counter, "%0", e);
+ return GEN_UNARY_RMWcc(LOCK_PREFIX "incl", v->counter, e);
}
#define arch_atomic_inc_and_test arch_atomic_inc_and_test
*/
static __always_inline bool arch_atomic_add_negative(int i, atomic_t *v)
{
- GEN_BINARY_RMWcc(LOCK_PREFIX "addl", v->counter, "er", i, "%0", s);
+ return GEN_BINARY_RMWcc(LOCK_PREFIX "addl", v->counter, s, "er", i);
}
#define arch_atomic_add_negative arch_atomic_add_negative
*/
static inline bool arch_atomic64_sub_and_test(long i, atomic64_t *v)
{
- GEN_BINARY_RMWcc(LOCK_PREFIX "subq", v->counter, "er", i, "%0", e);
+ return GEN_BINARY_RMWcc(LOCK_PREFIX "subq", v->counter, e, "er", i);
}
#define arch_atomic64_sub_and_test arch_atomic64_sub_and_test
*/
static inline bool arch_atomic64_dec_and_test(atomic64_t *v)
{
- GEN_UNARY_RMWcc(LOCK_PREFIX "decq", v->counter, "%0", e);
+ return GEN_UNARY_RMWcc(LOCK_PREFIX "decq", v->counter, e);
}
#define arch_atomic64_dec_and_test arch_atomic64_dec_and_test
*/
static inline bool arch_atomic64_inc_and_test(atomic64_t *v)
{
- GEN_UNARY_RMWcc(LOCK_PREFIX "incq", v->counter, "%0", e);
+ return GEN_UNARY_RMWcc(LOCK_PREFIX "incq", v->counter, e);
}
#define arch_atomic64_inc_and_test arch_atomic64_inc_and_test
*/
static inline bool arch_atomic64_add_negative(long i, atomic64_t *v)
{
- GEN_BINARY_RMWcc(LOCK_PREFIX "addq", v->counter, "er", i, "%0", s);
+ return GEN_BINARY_RMWcc(LOCK_PREFIX "addq", v->counter, s, "er", i);
}
#define arch_atomic64_add_negative arch_atomic64_add_negative
*/
static __always_inline bool test_and_set_bit(long nr, volatile unsigned long *addr)
{
- GEN_BINARY_RMWcc(LOCK_PREFIX __ASM_SIZE(bts),
- *addr, "Ir", nr, "%0", c);
+ return GEN_BINARY_RMWcc(LOCK_PREFIX __ASM_SIZE(bts), *addr, c, "Ir", nr);
}
/**
*/
static __always_inline bool test_and_clear_bit(long nr, volatile unsigned long *addr)
{
- GEN_BINARY_RMWcc(LOCK_PREFIX __ASM_SIZE(btr),
- *addr, "Ir", nr, "%0", c);
+ return GEN_BINARY_RMWcc(LOCK_PREFIX __ASM_SIZE(btr), *addr, c, "Ir", nr);
}
/**
*/
static __always_inline bool test_and_change_bit(long nr, volatile unsigned long *addr)
{
- GEN_BINARY_RMWcc(LOCK_PREFIX __ASM_SIZE(btc),
- *addr, "Ir", nr, "%0", c);
+ return GEN_BINARY_RMWcc(LOCK_PREFIX __ASM_SIZE(btc), *addr, c, "Ir", nr);
}
static __always_inline bool constant_test_bit(long nr, const volatile unsigned long *addr)
#include <linux/stringify.h>
+#ifndef __ASSEMBLY__
+
/*
* Despite that some emulators terminate on UD2, we use it for WARN().
*
#define LEN_UD2 2
-#ifdef CONFIG_GENERIC_BUG
-
-#ifdef CONFIG_X86_32
-# define __BUG_REL(val) ".long " __stringify(val)
-#else
-# define __BUG_REL(val) ".long " __stringify(val) " - 2b"
-#endif
-
-#ifdef CONFIG_DEBUG_BUGVERBOSE
-
-#define _BUG_FLAGS(ins, flags) \
-do { \
- asm volatile("1:\t" ins "\n" \
- ".pushsection __bug_table,\"aw\"\n" \
- "2:\t" __BUG_REL(1b) "\t# bug_entry::bug_addr\n" \
- "\t" __BUG_REL(%c0) "\t# bug_entry::file\n" \
- "\t.word %c1" "\t# bug_entry::line\n" \
- "\t.word %c2" "\t# bug_entry::flags\n" \
- "\t.org 2b+%c3\n" \
- ".popsection" \
- : : "i" (__FILE__), "i" (__LINE__), \
- "i" (flags), \
- "i" (sizeof(struct bug_entry))); \
-} while (0)
-
-#else /* !CONFIG_DEBUG_BUGVERBOSE */
-
#define _BUG_FLAGS(ins, flags) \
do { \
- asm volatile("1:\t" ins "\n" \
- ".pushsection __bug_table,\"aw\"\n" \
- "2:\t" __BUG_REL(1b) "\t# bug_entry::bug_addr\n" \
- "\t.word %c0" "\t# bug_entry::flags\n" \
- "\t.org 2b+%c1\n" \
- ".popsection" \
- : : "i" (flags), \
+ asm volatile("ASM_BUG ins=\"" ins "\" file=%c0 line=%c1 " \
+ "flags=%c2 size=%c3" \
+ : : "i" (__FILE__), "i" (__LINE__), \
+ "i" (flags), \
"i" (sizeof(struct bug_entry))); \
} while (0)
-#endif /* CONFIG_DEBUG_BUGVERBOSE */
-
-#else
-
-#define _BUG_FLAGS(ins, flags) asm volatile(ins)
-
-#endif /* CONFIG_GENERIC_BUG */
-
#define HAVE_ARCH_BUG
#define BUG() \
do { \
#include <asm-generic/bug.h>
+#else /* __ASSEMBLY__ */
+
+#ifdef CONFIG_GENERIC_BUG
+
+#ifdef CONFIG_X86_32
+.macro __BUG_REL val:req
+ .long \val
+.endm
+#else
+.macro __BUG_REL val:req
+ .long \val - 2b
+.endm
+#endif
+
+#ifdef CONFIG_DEBUG_BUGVERBOSE
+
+.macro ASM_BUG ins:req file:req line:req flags:req size:req
+1: \ins
+ .pushsection __bug_table,"aw"
+2: __BUG_REL val=1b # bug_entry::bug_addr
+ __BUG_REL val=\file # bug_entry::file
+ .word \line # bug_entry::line
+ .word \flags # bug_entry::flags
+ .org 2b+\size
+ .popsection
+.endm
+
+#else /* !CONFIG_DEBUG_BUGVERBOSE */
+
+.macro ASM_BUG ins:req file:req line:req flags:req size:req
+1: \ins
+ .pushsection __bug_table,"aw"
+2: __BUG_REL val=1b # bug_entry::bug_addr
+ .word \flags # bug_entry::flags
+ .org 2b+\size
+ .popsection
+.endm
+
+#endif /* CONFIG_DEBUG_BUGVERBOSE */
+
+#else /* CONFIG_GENERIC_BUG */
+
+.macro ASM_BUG ins:req file:req line:req flags:req size:req
+ \ins
+.endm
+
+#endif /* CONFIG_GENERIC_BUG */
+
+#endif /* __ASSEMBLY__ */
+
#endif /* _ASM_X86_BUG_H */
#define _ASM_X86_CACHEINFO_H
void cacheinfo_amd_init_llc_id(struct cpuinfo_x86 *c, int cpu, u8 node_id);
+void cacheinfo_hygon_init_llc_id(struct cpuinfo_x86 *c, int cpu, u8 node_id);
#endif /* _ASM_X86_CACHEINFO_H */
BUILD_BUG_ON(sizeof(*(p2)) != sizeof(long)); \
VM_BUG_ON((unsigned long)(p1) % (2 * sizeof(long))); \
VM_BUG_ON((unsigned long)((p1) + 1) != (unsigned long)(p2)); \
- asm volatile(pfx "cmpxchg%c4b %2; sete %0" \
- : "=a" (__ret), "+d" (__old2), \
- "+m" (*(p1)), "+m" (*(p2)) \
- : "i" (2 * sizeof(long)), "a" (__old1), \
+ asm volatile(pfx "cmpxchg%c5b %1" \
+ CC_SET(e) \
+ : CC_OUT(e) (__ret), \
+ "+m" (*(p1)), "+m" (*(p2)), \
+ "+a" (__old1), "+d" (__old2) \
+ : "i" (2 * sizeof(long)), \
"b" (__new1), "c" (__new2)); \
__ret; \
})
#ifndef _ASM_X86_CPUFEATURE_H
#define _ASM_X86_CPUFEATURE_H
-#include <asm/processor.h>
-
-#if defined(__KERNEL__) && !defined(__ASSEMBLY__)
+#ifdef __KERNEL__
+#ifndef __ASSEMBLY__
+#include <asm/processor.h>
#include <asm/asm.h>
#include <linux/bitops.h>
*/
static __always_inline __pure bool _static_cpu_has(u16 bit)
{
- asm_volatile_goto("1: jmp 6f\n"
- "2:\n"
- ".skip -(((5f-4f) - (2b-1b)) > 0) * "
- "((5f-4f) - (2b-1b)),0x90\n"
- "3:\n"
- ".section .altinstructions,\"a\"\n"
- " .long 1b - .\n" /* src offset */
- " .long 4f - .\n" /* repl offset */
- " .word %P[always]\n" /* always replace */
- " .byte 3b - 1b\n" /* src len */
- " .byte 5f - 4f\n" /* repl len */
- " .byte 3b - 2b\n" /* pad len */
- ".previous\n"
- ".section .altinstr_replacement,\"ax\"\n"
- "4: jmp %l[t_no]\n"
- "5:\n"
- ".previous\n"
- ".section .altinstructions,\"a\"\n"
- " .long 1b - .\n" /* src offset */
- " .long 0\n" /* no replacement */
- " .word %P[feature]\n" /* feature bit */
- " .byte 3b - 1b\n" /* src len */
- " .byte 0\n" /* repl len */
- " .byte 0\n" /* pad len */
- ".previous\n"
- ".section .altinstr_aux,\"ax\"\n"
- "6:\n"
- " testb %[bitnum],%[cap_byte]\n"
- " jnz %l[t_yes]\n"
- " jmp %l[t_no]\n"
- ".previous\n"
+ asm_volatile_goto("STATIC_CPU_HAS bitnum=%[bitnum] "
+ "cap_byte=\"%[cap_byte]\" "
+ "feature=%P[feature] t_yes=%l[t_yes] "
+ "t_no=%l[t_no] always=%P[always]"
: : [feature] "i" (bit),
[always] "i" (X86_FEATURE_ALWAYS),
[bitnum] "i" (1 << (bit & 7)),
#define CPU_FEATURE_TYPEVAL boot_cpu_data.x86_vendor, boot_cpu_data.x86, \
boot_cpu_data.x86_model
-#endif /* defined(__KERNEL__) && !defined(__ASSEMBLY__) */
+#else /* __ASSEMBLY__ */
+
+.macro STATIC_CPU_HAS bitnum:req cap_byte:req feature:req t_yes:req t_no:req always:req
+1:
+ jmp 6f
+2:
+ .skip -(((5f-4f) - (2b-1b)) > 0) * ((5f-4f) - (2b-1b)),0x90
+3:
+ .section .altinstructions,"a"
+ .long 1b - . /* src offset */
+ .long 4f - . /* repl offset */
+ .word \always /* always replace */
+ .byte 3b - 1b /* src len */
+ .byte 5f - 4f /* repl len */
+ .byte 3b - 2b /* pad len */
+ .previous
+ .section .altinstr_replacement,"ax"
+4:
+ jmp \t_no
+5:
+ .previous
+ .section .altinstructions,"a"
+ .long 1b - . /* src offset */
+ .long 0 /* no replacement */
+ .word \feature /* feature bit */
+ .byte 3b - 1b /* src len */
+ .byte 0 /* repl len */
+ .byte 0 /* pad len */
+ .previous
+ .section .altinstr_aux,"ax"
+6:
+ testb \bitnum,\cap_byte
+ jnz \t_yes
+ jmp \t_no
+ .previous
+.endm
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
#endif /* _ASM_X86_CPUFEATURE_H */
extern int __init efi_reuse_config(u64 tables, int nr_tables);
extern void efi_delete_dummy_variable(void);
extern void efi_switch_mm(struct mm_struct *mm);
+extern void efi_recover_from_page_fault(unsigned long phys_addr);
struct efi_setup_data {
u64 fw_vendor;
#include <asm/ptrace.h>
#include <asm/user.h>
#include <asm/auxvec.h>
+#include <asm/fsgsbase.h>
typedef unsigned long elf_greg_t;
#define R_X86_64_PC16 13 /* 16 bit sign extended pc relative */
#define R_X86_64_8 14 /* Direct 8 bit sign extended */
#define R_X86_64_PC8 15 /* 8 bit sign extended pc relative */
-
-#define R_X86_64_NUM 16
+#define R_X86_64_PC64 24 /* Place relative 64-bit signed */
/*
* These are used to set parameters in the core dumps.
#define ELF_CORE_COPY_REGS(pr_reg, regs) \
do { \
- unsigned long base; \
unsigned v; \
(pr_reg)[0] = (regs)->r15; \
(pr_reg)[1] = (regs)->r14; \
(pr_reg)[18] = (regs)->flags; \
(pr_reg)[19] = (regs)->sp; \
(pr_reg)[20] = (regs)->ss; \
- rdmsrl(MSR_FS_BASE, base); (pr_reg)[21] = base; \
- rdmsrl(MSR_KERNEL_GS_BASE, base); (pr_reg)[22] = base; \
+ (pr_reg)[21] = x86_fsbase_read_cpu(); \
+ (pr_reg)[22] = x86_gsbase_read_cpu_inactive(); \
asm("movl %%ds,%0" : "=r" (v)); (pr_reg)[23] = v; \
asm("movl %%es,%0" : "=r" (v)); (pr_reg)[24] = v; \
asm("movl %%fs,%0" : "=r" (v)); (pr_reg)[25] = v; \
(b)->handler = (tmp).handler - (delta); \
} while (0)
-extern int fixup_exception(struct pt_regs *regs, int trapnr);
+extern int fixup_exception(struct pt_regs *regs, int trapnr,
+ unsigned long error_code, unsigned long fault_addr);
extern int fixup_bug(struct pt_regs *regs, int trapnr);
extern bool ex_has_fault_handler(unsigned long ip);
extern void early_fixup_exception(struct pt_regs *regs, int trapnr);
"3: movl $-2,%[err]\n\t" \
"jmp 2b\n\t" \
".popsection\n\t" \
- _ASM_EXTABLE(1b, 3b) \
+ _ASM_EXTABLE_UA(1b, 3b) \
: [err] "=r" (err) \
: "D" (st), "m" (*st), "a" (lmask), "d" (hmask) \
: "memory")
static inline void
switch_fpu_prepare(struct fpu *old_fpu, int cpu)
{
- if (old_fpu->initialized) {
+ if (static_cpu_has(X86_FEATURE_FPU) && old_fpu->initialized) {
if (!copy_fpregs_to_fpstate(old_fpu))
old_fpu->last_cpu = -1;
else
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_FSGSBASE_H
+#define _ASM_FSGSBASE_H
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_X86_64
+
+#include <asm/msr-index.h>
+
+/*
+ * Read/write a task's FSBASE or GSBASE. This returns the value that
+ * the FS/GS base would have (if the task were to be resumed). These
+ * work on the current task or on a non-running (typically stopped
+ * ptrace child) task.
+ */
+extern unsigned long x86_fsbase_read_task(struct task_struct *task);
+extern unsigned long x86_gsbase_read_task(struct task_struct *task);
+extern int x86_fsbase_write_task(struct task_struct *task, unsigned long fsbase);
+extern int x86_gsbase_write_task(struct task_struct *task, unsigned long gsbase);
+
+/* Helper functions for reading/writing FS/GS base */
+
+static inline unsigned long x86_fsbase_read_cpu(void)
+{
+ unsigned long fsbase;
+
+ rdmsrl(MSR_FS_BASE, fsbase);
+
+ return fsbase;
+}
+
+static inline unsigned long x86_gsbase_read_cpu_inactive(void)
+{
+ unsigned long gsbase;
+
+ rdmsrl(MSR_KERNEL_GS_BASE, gsbase);
+
+ return gsbase;
+}
+
+extern void x86_fsbase_write_cpu(unsigned long fsbase);
+extern void x86_gsbase_write_cpu_inactive(unsigned long gsbase);
+
+#endif /* CONFIG_X86_64 */
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ASM_FSGSBASE_H */
"3:\tmov\t%3, %1\n" \
"\tjmp\t2b\n" \
"\t.previous\n" \
- _ASM_EXTABLE(1b, 3b) \
+ _ASM_EXTABLE_UA(1b, 3b) \
: "=r" (oldval), "=r" (ret), "+m" (*uaddr) \
: "i" (-EFAULT), "0" (oparg), "1" (0))
"4:\tmov\t%5, %1\n" \
"\tjmp\t3b\n" \
"\t.previous\n" \
- _ASM_EXTABLE(1b, 4b) \
- _ASM_EXTABLE(2b, 4b) \
+ _ASM_EXTABLE_UA(1b, 4b) \
+ _ASM_EXTABLE_UA(2b, 4b) \
: "=&a" (oldval), "=&r" (ret), \
"+m" (*uaddr), "=&r" (tem) \
: "r" (oparg), "i" (-EFAULT), "1" (0))
* The "_X" parts are generally the EP and EX Xeons, or the
* "Extreme" ones, like Broadwell-E.
*
- * Things ending in "2" are usually because we have no better
- * name for them. There's no processor called "SILVERMONT2".
- *
* While adding a new CPUID for a new microarchitecture, add a new
* group to keep logically sorted out in chronological order. Within
* that group keep the CPUID for the variants sorted by model number.
/* "Small Core" Processors (Atom) */
-#define INTEL_FAM6_ATOM_PINEVIEW 0x1C
-#define INTEL_FAM6_ATOM_LINCROFT 0x26
-#define INTEL_FAM6_ATOM_PENWELL 0x27
-#define INTEL_FAM6_ATOM_CLOVERVIEW 0x35
-#define INTEL_FAM6_ATOM_CEDARVIEW 0x36
-#define INTEL_FAM6_ATOM_SILVERMONT1 0x37 /* BayTrail/BYT / Valleyview */
-#define INTEL_FAM6_ATOM_SILVERMONT2 0x4D /* Avaton/Rangely */
-#define INTEL_FAM6_ATOM_AIRMONT 0x4C /* CherryTrail / Braswell */
-#define INTEL_FAM6_ATOM_MERRIFIELD 0x4A /* Tangier */
-#define INTEL_FAM6_ATOM_MOOREFIELD 0x5A /* Anniedale */
-#define INTEL_FAM6_ATOM_GOLDMONT 0x5C
-#define INTEL_FAM6_ATOM_DENVERTON 0x5F /* Goldmont Microserver */
-#define INTEL_FAM6_ATOM_GEMINI_LAKE 0x7A
+#define INTEL_FAM6_ATOM_BONNELL 0x1C /* Diamondville, Pineview */
+#define INTEL_FAM6_ATOM_BONNELL_MID 0x26 /* Silverthorne, Lincroft */
+
+#define INTEL_FAM6_ATOM_SALTWELL 0x36 /* Cedarview */
+#define INTEL_FAM6_ATOM_SALTWELL_MID 0x27 /* Penwell */
+#define INTEL_FAM6_ATOM_SALTWELL_TABLET 0x35 /* Cloverview */
+
+#define INTEL_FAM6_ATOM_SILVERMONT 0x37 /* Bay Trail, Valleyview */
+#define INTEL_FAM6_ATOM_SILVERMONT_X 0x4D /* Avaton, Rangely */
+#define INTEL_FAM6_ATOM_SILVERMONT_MID 0x4A /* Merriefield */
+
+#define INTEL_FAM6_ATOM_AIRMONT 0x4C /* Cherry Trail, Braswell */
+#define INTEL_FAM6_ATOM_AIRMONT_MID 0x5A /* Moorefield */
+
+#define INTEL_FAM6_ATOM_GOLDMONT 0x5C /* Apollo Lake */
+#define INTEL_FAM6_ATOM_GOLDMONT_X 0x5F /* Denverton */
+#define INTEL_FAM6_ATOM_GOLDMONT_PLUS 0x7A /* Gemini Lake */
/* Xeon Phi */
extern bool is_early_ioremap_ptep(pte_t *ptep);
-#ifdef CONFIG_XEN
-#include <xen/xen.h>
-struct bio_vec;
-
-extern bool xen_biovec_phys_mergeable(const struct bio_vec *vec1,
- const struct bio_vec *vec2);
-
-#define BIOVEC_PHYS_MERGEABLE(vec1, vec2) \
- (__BIOVEC_PHYS_MERGEABLE(vec1, vec2) && \
- (!xen_domain() || xen_biovec_phys_mergeable(vec1, vec2)))
-#endif /* CONFIG_XEN */
-
#define IO_SPACE_LIMIT 0xffff
#include <asm-generic/io.h>
#ifndef _ASM_X86_JUMP_LABEL_H
#define _ASM_X86_JUMP_LABEL_H
-#ifndef HAVE_JUMP_LABEL
-/*
- * For better or for worse, if jump labels (the gcc extension) are missing,
- * then the entire static branch patching infrastructure is compiled out.
- * If that happens, the code in here will malfunction. Raise a compiler
- * error instead.
- *
- * In theory, jump labels and the static branch patching infrastructure
- * could be decoupled to fix this.
- */
-#error asm/jump_label.h included on a non-jump-label kernel
-#endif
-
#define JUMP_LABEL_NOP_SIZE 5
#ifdef CONFIG_X86_64
static __always_inline bool arch_static_branch(struct static_key *key, bool branch)
{
- asm_volatile_goto("1:"
- ".byte " __stringify(STATIC_KEY_INIT_NOP) "\n\t"
- ".pushsection __jump_table, \"aw\" \n\t"
- _ASM_ALIGN "\n\t"
- _ASM_PTR "1b, %l[l_yes], %c0 + %c1 \n\t"
- ".popsection \n\t"
- : : "i" (key), "i" (branch) : : l_yes);
-
+ asm_volatile_goto("STATIC_BRANCH_NOP l_yes=\"%l[l_yes]\" key=\"%c0\" "
+ "branch=\"%c1\""
+ : : "i" (key), "i" (branch) : : l_yes);
return false;
l_yes:
return true;
static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch)
{
- asm_volatile_goto("1:"
- ".byte 0xe9\n\t .long %l[l_yes] - 2f\n\t"
- "2:\n\t"
- ".pushsection __jump_table, \"aw\" \n\t"
- _ASM_ALIGN "\n\t"
- _ASM_PTR "1b, %l[l_yes], %c0 + %c1 \n\t"
- ".popsection \n\t"
+ asm_volatile_goto("STATIC_BRANCH_JMP l_yes=\"%l[l_yes]\" key=\"%c0\" "
+ "branch=\"%c1\""
: : "i" (key), "i" (branch) : : l_yes);
return false;
return true;
}
-#ifdef CONFIG_X86_64
-typedef u64 jump_label_t;
-#else
-typedef u32 jump_label_t;
-#endif
-
-struct jump_entry {
- jump_label_t code;
- jump_label_t target;
- jump_label_t key;
-};
-
#else /* __ASSEMBLY__ */
-.macro STATIC_JUMP_IF_TRUE target, key, def
-.Lstatic_jump_\@:
- .if \def
- /* Equivalent to "jmp.d32 \target" */
- .byte 0xe9
- .long \target - .Lstatic_jump_after_\@
-.Lstatic_jump_after_\@:
- .else
- .byte STATIC_KEY_INIT_NOP
- .endif
+.macro STATIC_BRANCH_NOP l_yes:req key:req branch:req
+.Lstatic_branch_nop_\@:
+ .byte STATIC_KEY_INIT_NOP
+.Lstatic_branch_no_after_\@:
.pushsection __jump_table, "aw"
_ASM_ALIGN
- _ASM_PTR .Lstatic_jump_\@, \target, \key
+ .long .Lstatic_branch_nop_\@ - ., \l_yes - .
+ _ASM_PTR \key + \branch - .
.popsection
.endm
-.macro STATIC_JUMP_IF_FALSE target, key, def
-.Lstatic_jump_\@:
- .if \def
- .byte STATIC_KEY_INIT_NOP
- .else
- /* Equivalent to "jmp.d32 \target" */
- .byte 0xe9
- .long \target - .Lstatic_jump_after_\@
-.Lstatic_jump_after_\@:
- .endif
+.macro STATIC_BRANCH_JMP l_yes:req key:req branch:req
+.Lstatic_branch_jmp_\@:
+ .byte 0xe9
+ .long \l_yes - .Lstatic_branch_jmp_after_\@
+.Lstatic_branch_jmp_after_\@:
.pushsection __jump_table, "aw"
_ASM_ALIGN
- _ASM_PTR .Lstatic_jump_\@, \target, \key + 1
+ .long .Lstatic_branch_jmp_\@ - ., \l_yes - .
+ _ASM_PTR \key + \branch - .
.popsection
.endm
#define X86EMUL_CPUID_VENDOR_AMDisbetterI_ecx 0x21726574
#define X86EMUL_CPUID_VENDOR_AMDisbetterI_edx 0x74656273
+#define X86EMUL_CPUID_VENDOR_HygonGenuine_ebx 0x6f677948
+#define X86EMUL_CPUID_VENDOR_HygonGenuine_ecx 0x656e6975
+#define X86EMUL_CPUID_VENDOR_HygonGenuine_edx 0x6e65476e
+
#define X86EMUL_CPUID_VENDOR_GenuineIntel_ebx 0x756e6547
#define X86EMUL_CPUID_VENDOR_GenuineIntel_ecx 0x6c65746e
#define X86EMUL_CPUID_VENDOR_GenuineIntel_edx 0x49656e69
*/
static inline bool local_sub_and_test(long i, local_t *l)
{
- GEN_BINARY_RMWcc(_ASM_SUB, l->a.counter, "er", i, "%0", e);
+ return GEN_BINARY_RMWcc(_ASM_SUB, l->a.counter, e, "er", i);
}
/**
*/
static inline bool local_dec_and_test(local_t *l)
{
- GEN_UNARY_RMWcc(_ASM_DEC, l->a.counter, "%0", e);
+ return GEN_UNARY_RMWcc(_ASM_DEC, l->a.counter, e);
}
/**
*/
static inline bool local_inc_and_test(local_t *l)
{
- GEN_UNARY_RMWcc(_ASM_INC, l->a.counter, "%0", e);
+ return GEN_UNARY_RMWcc(_ASM_INC, l->a.counter, e);
}
/**
*/
static inline bool local_add_negative(long i, local_t *l)
{
- GEN_BINARY_RMWcc(_ASM_ADD, l->a.counter, "er", i, "%0", s);
+ return GEN_BINARY_RMWcc(_ASM_ADD, l->a.counter, s, "er", i);
}
/**
/* MCG_CAP register defines */
#define MCG_BANKCNT_MASK 0xff /* Number of Banks */
-#define MCG_CTL_P (1ULL<<8) /* MCG_CTL register available */
-#define MCG_EXT_P (1ULL<<9) /* Extended registers available */
-#define MCG_CMCI_P (1ULL<<10) /* CMCI supported */
+#define MCG_CTL_P BIT_ULL(8) /* MCG_CTL register available */
+#define MCG_EXT_P BIT_ULL(9) /* Extended registers available */
+#define MCG_CMCI_P BIT_ULL(10) /* CMCI supported */
#define MCG_EXT_CNT_MASK 0xff0000 /* Number of Extended registers */
#define MCG_EXT_CNT_SHIFT 16
#define MCG_EXT_CNT(c) (((c) & MCG_EXT_CNT_MASK) >> MCG_EXT_CNT_SHIFT)
-#define MCG_SER_P (1ULL<<24) /* MCA recovery/new status bits */
-#define MCG_ELOG_P (1ULL<<26) /* Extended error log supported */
-#define MCG_LMCE_P (1ULL<<27) /* Local machine check supported */
+#define MCG_SER_P BIT_ULL(24) /* MCA recovery/new status bits */
+#define MCG_ELOG_P BIT_ULL(26) /* Extended error log supported */
+#define MCG_LMCE_P BIT_ULL(27) /* Local machine check supported */
/* MCG_STATUS register defines */
-#define MCG_STATUS_RIPV (1ULL<<0) /* restart ip valid */
-#define MCG_STATUS_EIPV (1ULL<<1) /* ip points to correct instruction */
-#define MCG_STATUS_MCIP (1ULL<<2) /* machine check in progress */
-#define MCG_STATUS_LMCES (1ULL<<3) /* LMCE signaled */
+#define MCG_STATUS_RIPV BIT_ULL(0) /* restart ip valid */
+#define MCG_STATUS_EIPV BIT_ULL(1) /* ip points to correct instruction */
+#define MCG_STATUS_MCIP BIT_ULL(2) /* machine check in progress */
+#define MCG_STATUS_LMCES BIT_ULL(3) /* LMCE signaled */
/* MCG_EXT_CTL register defines */
-#define MCG_EXT_CTL_LMCE_EN (1ULL<<0) /* Enable LMCE */
+#define MCG_EXT_CTL_LMCE_EN BIT_ULL(0) /* Enable LMCE */
/* MCi_STATUS register defines */
-#define MCI_STATUS_VAL (1ULL<<63) /* valid error */
-#define MCI_STATUS_OVER (1ULL<<62) /* previous errors lost */
-#define MCI_STATUS_UC (1ULL<<61) /* uncorrected error */
-#define MCI_STATUS_EN (1ULL<<60) /* error enabled */
-#define MCI_STATUS_MISCV (1ULL<<59) /* misc error reg. valid */
-#define MCI_STATUS_ADDRV (1ULL<<58) /* addr reg. valid */
-#define MCI_STATUS_PCC (1ULL<<57) /* processor context corrupt */
-#define MCI_STATUS_S (1ULL<<56) /* Signaled machine check */
-#define MCI_STATUS_AR (1ULL<<55) /* Action required */
+#define MCI_STATUS_VAL BIT_ULL(63) /* valid error */
+#define MCI_STATUS_OVER BIT_ULL(62) /* previous errors lost */
+#define MCI_STATUS_UC BIT_ULL(61) /* uncorrected error */
+#define MCI_STATUS_EN BIT_ULL(60) /* error enabled */
+#define MCI_STATUS_MISCV BIT_ULL(59) /* misc error reg. valid */
+#define MCI_STATUS_ADDRV BIT_ULL(58) /* addr reg. valid */
+#define MCI_STATUS_PCC BIT_ULL(57) /* processor context corrupt */
+#define MCI_STATUS_S BIT_ULL(56) /* Signaled machine check */
+#define MCI_STATUS_AR BIT_ULL(55) /* Action required */
+#define MCI_STATUS_CEC_SHIFT 38 /* Corrected Error Count */
+#define MCI_STATUS_CEC_MASK GENMASK_ULL(52,38)
+#define MCI_STATUS_CEC(c) (((c) & MCI_STATUS_CEC_MASK) >> MCI_STATUS_CEC_SHIFT)
/* AMD-specific bits */
-#define MCI_STATUS_TCC (1ULL<<55) /* Task context corrupt */
-#define MCI_STATUS_SYNDV (1ULL<<53) /* synd reg. valid */
-#define MCI_STATUS_DEFERRED (1ULL<<44) /* uncorrected error, deferred exception */
-#define MCI_STATUS_POISON (1ULL<<43) /* access poisonous data */
+#define MCI_STATUS_TCC BIT_ULL(55) /* Task context corrupt */
+#define MCI_STATUS_SYNDV BIT_ULL(53) /* synd reg. valid */
+#define MCI_STATUS_DEFERRED BIT_ULL(44) /* uncorrected error, deferred exception */
+#define MCI_STATUS_POISON BIT_ULL(43) /* access poisonous data */
/*
* McaX field if set indicates a given bank supports MCA extensions:
#define MCI_MISC_ADDR_GENERIC 7 /* generic */
/* CTL2 register defines */
-#define MCI_CTL2_CMCI_EN (1ULL << 30)
+#define MCI_CTL2_CMCI_EN BIT_ULL(30)
#define MCI_CTL2_CMCI_THRESHOLD_MASK 0x7fffULL
#define MCJ_CTX_MASK 3
static inline int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr) { return -EINVAL; };
#endif
+static inline void mce_hygon_feature_init(struct cpuinfo_x86 *c) { return mce_amd_feature_init(c); }
+
int mce_available(struct cpuinfo_x86 *c);
bool mce_is_memory_error(struct mce *m);
#define DEBUGCTLMSR_BTS_OFF_OS (1UL << 9)
#define DEBUGCTLMSR_BTS_OFF_USR (1UL << 10)
#define DEBUGCTLMSR_FREEZE_LBRS_ON_PMI (1UL << 11)
+#define DEBUGCTLMSR_FREEZE_PERFMON_ON_PMI (1UL << 12)
#define DEBUGCTLMSR_FREEZE_IN_SMM_BIT 14
#define DEBUGCTLMSR_FREEZE_IN_SMM (1UL << DEBUGCTLMSR_FREEZE_IN_SMM_BIT)
#define paravirt_clobber(clobber) \
[paravirt_clobber] "i" (clobber)
-/*
- * Generate some code, and mark it as patchable by the
- * apply_paravirt() alternate instruction patcher.
- */
-#define _paravirt_alt(insn_string, type, clobber) \
- "771:\n\t" insn_string "\n" "772:\n" \
- ".pushsection .parainstructions,\"a\"\n" \
- _ASM_ALIGN "\n" \
- _ASM_PTR " 771b\n" \
- " .byte " type "\n" \
- " .byte 772b-771b\n" \
- " .short " clobber "\n" \
- ".popsection\n"
-
/* Generate patchable code, with the default asm parameters. */
-#define paravirt_alt(insn_string) \
- _paravirt_alt(insn_string, "%c[paravirt_typenum]", "%c[paravirt_clobber]")
+#define paravirt_call \
+ "PARAVIRT_CALL type=\"%c[paravirt_typenum]\"" \
+ " clobber=\"%c[paravirt_clobber]\"" \
+ " pv_opptr=\"%c[paravirt_opptr]\";"
/* Simple instruction patching code. */
#define NATIVE_LABEL(a,x,b) "\n\t.globl " a #x "_" #b "\n" a #x "_" #b ":\n\t"
int paravirt_disable_iospace(void);
-/*
- * This generates an indirect call based on the operation type number.
- * The type number, computed in PARAVIRT_PATCH, is derived from the
- * offset into the paravirt_patch_template structure, and can therefore be
- * freely converted back into a structure offset.
- */
-#define PARAVIRT_CALL \
- ANNOTATE_RETPOLINE_SAFE \
- "call *%c[paravirt_opptr];"
-
/*
* These macros are intended to wrap calls through one of the paravirt
* ops structs, so that they can be later identified and patched at
/* since this condition will never hold */ \
if (sizeof(rettype) > sizeof(unsigned long)) { \
asm volatile(pre \
- paravirt_alt(PARAVIRT_CALL) \
+ paravirt_call \
post \
: call_clbr, ASM_CALL_CONSTRAINT \
: paravirt_type(op), \
__ret = (rettype)((((u64)__edx) << 32) | __eax); \
} else { \
asm volatile(pre \
- paravirt_alt(PARAVIRT_CALL) \
+ paravirt_call \
post \
: call_clbr, ASM_CALL_CONSTRAINT \
: paravirt_type(op), \
PVOP_VCALL_ARGS; \
PVOP_TEST_NULL(op); \
asm volatile(pre \
- paravirt_alt(PARAVIRT_CALL) \
+ paravirt_call \
post \
: call_clbr, ASM_CALL_CONSTRAINT \
: paravirt_type(op), \
extern struct paravirt_patch_site __parainstructions[],
__parainstructions_end[];
+#else /* __ASSEMBLY__ */
+
+/*
+ * This generates an indirect call based on the operation type number.
+ * The type number, computed in PARAVIRT_PATCH, is derived from the
+ * offset into the paravirt_patch_template structure, and can therefore be
+ * freely converted back into a structure offset.
+ */
+.macro PARAVIRT_CALL type:req clobber:req pv_opptr:req
+771: ANNOTATE_RETPOLINE_SAFE
+ call *\pv_opptr
+772: .pushsection .parainstructions,"a"
+ _ASM_ALIGN
+ _ASM_PTR 771b
+ .byte \type
+ .byte 772b-771b
+ .short \clobber
+ .popsection
+.endm
+
#endif /* __ASSEMBLY__ */
#endif /* _ASM_X86_PARAVIRT_TYPES_H */
typeof(var) pfo_ret__; \
switch (sizeof(var)) { \
case 1: \
- asm(op "b "__percpu_arg(1)",%0" \
+ asm volatile(op "b "__percpu_arg(1)",%0"\
: "=q" (pfo_ret__) \
: "m" (var)); \
break; \
case 2: \
- asm(op "w "__percpu_arg(1)",%0" \
+ asm volatile(op "w "__percpu_arg(1)",%0"\
: "=r" (pfo_ret__) \
: "m" (var)); \
break; \
case 4: \
- asm(op "l "__percpu_arg(1)",%0" \
+ asm volatile(op "l "__percpu_arg(1)",%0"\
: "=r" (pfo_ret__) \
: "m" (var)); \
break; \
case 8: \
- asm(op "q "__percpu_arg(1)",%0" \
+ asm volatile(op "q "__percpu_arg(1)",%0"\
: "=r" (pfo_ret__) \
: "m" (var)); \
break; \
extern struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr);
extern void perf_get_x86_pmu_capability(struct x86_pmu_capability *cap);
extern void perf_check_microcode(void);
+extern int x86_perf_rdpmc_index(struct perf_event *event);
#else
static inline struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr)
{
*/
#define _PAGE_CHG_MASK (PTE_PFN_MASK | _PAGE_PCD | _PAGE_PWT | \
_PAGE_SPECIAL | _PAGE_ACCESSED | _PAGE_DIRTY | \
- _PAGE_SOFT_DIRTY)
+ _PAGE_SOFT_DIRTY | _PAGE_DEVMAP)
#define _HPAGE_CHG_MASK (_PAGE_CHG_MASK | _PAGE_PSE)
/*
*/
static __always_inline bool __preempt_count_dec_and_test(void)
{
- GEN_UNARY_RMWcc("decl", __preempt_count, __percpu_arg(0), e);
+ return GEN_UNARY_RMWcc("decl", __preempt_count, e, __percpu_arg([var]));
}
/*
#define X86_VENDOR_CENTAUR 5
#define X86_VENDOR_TRANSMETA 7
#define X86_VENDOR_NSC 8
-#define X86_VENDOR_NUM 9
+#define X86_VENDOR_HYGON 9
+#define X86_VENDOR_NUM 10
#define X86_VENDOR_UNKNOWN 0xff
unsigned short __esh;
unsigned short fs;
unsigned short __fsh;
+ /* On interrupt, gs and __gsh store the vector number. */
unsigned short gs;
unsigned short __gsh;
+ /* On interrupt, this is the error code. */
unsigned long orig_ax;
unsigned long ip;
unsigned short cs;
(kernel_stack_pointer(regs) & ~(THREAD_SIZE - 1)));
}
+/**
+ * regs_get_kernel_stack_nth_addr() - get the address of the Nth entry on stack
+ * @regs: pt_regs which contains kernel stack pointer.
+ * @n: stack entry number.
+ *
+ * regs_get_kernel_stack_nth() returns the address of the @n th entry of the
+ * kernel stack which is specified by @regs. If the @n th entry is NOT in
+ * the kernel stack, this returns NULL.
+ */
+static inline unsigned long *regs_get_kernel_stack_nth_addr(struct pt_regs *regs, unsigned int n)
+{
+ unsigned long *addr = (unsigned long *)kernel_stack_pointer(regs);
+
+ addr += n;
+ if (regs_within_kernel_stack(regs, (unsigned long)addr))
+ return addr;
+ else
+ return NULL;
+}
+
+/* To avoid include hell, we can't include uaccess.h */
+extern long probe_kernel_read(void *dst, const void *src, size_t size);
+
/**
* regs_get_kernel_stack_nth() - get Nth entry of the stack
* @regs: pt_regs which contains kernel stack pointer.
* @n: stack entry number.
*
* regs_get_kernel_stack_nth() returns @n th entry of the kernel stack which
- * is specified by @regs. If the @n th entry is NOT in the kernel stack,
+ * is specified by @regs. If the @n th entry is NOT in the kernel stack
* this returns 0.
*/
static inline unsigned long regs_get_kernel_stack_nth(struct pt_regs *regs,
unsigned int n)
{
- unsigned long *addr = (unsigned long *)kernel_stack_pointer(regs);
- addr += n;
- if (regs_within_kernel_stack(regs, (unsigned long)addr))
- return *addr;
- else
- return 0;
+ unsigned long *addr;
+ unsigned long val;
+ long ret;
+
+ addr = regs_get_kernel_stack_nth_addr(regs, n);
+ if (addr) {
+ ret = probe_kernel_read(&val, addr, sizeof(val));
+ if (!ret)
+ return val;
+ }
+ return 0;
}
#define arch_has_single_step() (1)
#include <asm/cpufeature.h>
#include <asm-generic/qspinlock_types.h>
#include <asm/paravirt.h>
+#include <asm/rmwcc.h>
#define _Q_PENDING_LOOPS (1 << 9)
+#define queued_fetch_set_pending_acquire queued_fetch_set_pending_acquire
+static __always_inline u32 queued_fetch_set_pending_acquire(struct qspinlock *lock)
+{
+ u32 val = 0;
+
+ if (GEN_BINARY_RMWcc(LOCK_PREFIX "btsl", lock->val.counter, c,
+ "I", _Q_PENDING_OFFSET))
+ val |= _Q_PENDING_VAL;
+
+ val |= atomic_read(&lock->val) & ~_Q_PENDING_MASK;
+
+ return val;
+}
+
#ifdef CONFIG_PARAVIRT_SPINLOCKS
extern void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
extern void __pv_init_lock_hash(void);
* x86-specific implementation of refcount_t. Based on PAX_REFCOUNT from
* PaX/grsecurity.
*/
+
+#ifdef __ASSEMBLY__
+
+#include <asm/asm.h>
+#include <asm/bug.h>
+
+.macro REFCOUNT_EXCEPTION counter:req
+ .pushsection .text..refcount
+111: lea \counter, %_ASM_CX
+112: ud2
+ ASM_UNREACHABLE
+ .popsection
+113: _ASM_EXTABLE_REFCOUNT(112b, 113b)
+.endm
+
+/* Trigger refcount exception if refcount result is negative. */
+.macro REFCOUNT_CHECK_LT_ZERO counter:req
+ js 111f
+ REFCOUNT_EXCEPTION counter="\counter"
+.endm
+
+/* Trigger refcount exception if refcount result is zero or negative. */
+.macro REFCOUNT_CHECK_LE_ZERO counter:req
+ jz 111f
+ REFCOUNT_CHECK_LT_ZERO counter="\counter"
+.endm
+
+/* Trigger refcount exception unconditionally. */
+.macro REFCOUNT_ERROR counter:req
+ jmp 111f
+ REFCOUNT_EXCEPTION counter="\counter"
+.endm
+
+#else /* __ASSEMBLY__ */
+
#include <linux/refcount.h>
#include <asm/bug.h>
* central refcount exception. The fixup address for the exception points
* back to the regular execution flow in .text.
*/
-#define _REFCOUNT_EXCEPTION \
- ".pushsection .text..refcount\n" \
- "111:\tlea %[counter], %%" _ASM_CX "\n" \
- "112:\t" ASM_UD2 "\n" \
- ASM_UNREACHABLE \
- ".popsection\n" \
- "113:\n" \
- _ASM_EXTABLE_REFCOUNT(112b, 113b)
-
-/* Trigger refcount exception if refcount result is negative. */
-#define REFCOUNT_CHECK_LT_ZERO \
- "js 111f\n\t" \
- _REFCOUNT_EXCEPTION
-
-/* Trigger refcount exception if refcount result is zero or negative. */
-#define REFCOUNT_CHECK_LE_ZERO \
- "jz 111f\n\t" \
- REFCOUNT_CHECK_LT_ZERO
-
-/* Trigger refcount exception unconditionally. */
-#define REFCOUNT_ERROR \
- "jmp 111f\n\t" \
- _REFCOUNT_EXCEPTION
static __always_inline void refcount_add(unsigned int i, refcount_t *r)
{
asm volatile(LOCK_PREFIX "addl %1,%0\n\t"
- REFCOUNT_CHECK_LT_ZERO
+ "REFCOUNT_CHECK_LT_ZERO counter=\"%[counter]\""
: [counter] "+m" (r->refs.counter)
: "ir" (i)
: "cc", "cx");
static __always_inline void refcount_inc(refcount_t *r)
{
asm volatile(LOCK_PREFIX "incl %0\n\t"
- REFCOUNT_CHECK_LT_ZERO
+ "REFCOUNT_CHECK_LT_ZERO counter=\"%[counter]\""
: [counter] "+m" (r->refs.counter)
: : "cc", "cx");
}
static __always_inline void refcount_dec(refcount_t *r)
{
asm volatile(LOCK_PREFIX "decl %0\n\t"
- REFCOUNT_CHECK_LE_ZERO
+ "REFCOUNT_CHECK_LE_ZERO counter=\"%[counter]\""
: [counter] "+m" (r->refs.counter)
: : "cc", "cx");
}
static __always_inline __must_check
bool refcount_sub_and_test(unsigned int i, refcount_t *r)
{
- GEN_BINARY_SUFFIXED_RMWcc(LOCK_PREFIX "subl", REFCOUNT_CHECK_LT_ZERO,
- r->refs.counter, "er", i, "%0", e, "cx");
+
+ return GEN_BINARY_SUFFIXED_RMWcc(LOCK_PREFIX "subl",
+ "REFCOUNT_CHECK_LT_ZERO counter=\"%[var]\"",
+ r->refs.counter, e, "er", i, "cx");
}
static __always_inline __must_check bool refcount_dec_and_test(refcount_t *r)
{
- GEN_UNARY_SUFFIXED_RMWcc(LOCK_PREFIX "decl", REFCOUNT_CHECK_LT_ZERO,
- r->refs.counter, "%0", e, "cx");
+ return GEN_UNARY_SUFFIXED_RMWcc(LOCK_PREFIX "decl",
+ "REFCOUNT_CHECK_LT_ZERO counter=\"%[var]\"",
+ r->refs.counter, e, "cx");
}
static __always_inline __must_check
/* Did we try to increment from/to an undesirable state? */
if (unlikely(c < 0 || c == INT_MAX || result < c)) {
- asm volatile(REFCOUNT_ERROR
+ asm volatile("REFCOUNT_ERROR counter=\"%[counter]\""
: : [counter] "m" (r->refs.counter)
: "cc", "cx");
break;
return refcount_add_not_zero(1, r);
}
+#endif /* __ASSEMBLY__ */
+
#endif
#ifndef _ASM_X86_RMWcc
#define _ASM_X86_RMWcc
+/* This counts to 12. Any more, it will return 13th argument. */
+#define __RMWcc_ARGS(_0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _n, X...) _n
+#define RMWcc_ARGS(X...) __RMWcc_ARGS(, ##X, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0)
+
+#define __RMWcc_CONCAT(a, b) a ## b
+#define RMWcc_CONCAT(a, b) __RMWcc_CONCAT(a, b)
+
#define __CLOBBERS_MEM(clb...) "memory", ## clb
#if !defined(__GCC_ASM_FLAG_OUTPUTS__) && defined(CC_HAVE_ASM_GOTO)
/* Use asm goto */
-#define __GEN_RMWcc(fullop, var, cc, clobbers, ...) \
-do { \
+#define __GEN_RMWcc(fullop, _var, cc, clobbers, ...) \
+({ \
+ bool c = false; \
asm_volatile_goto (fullop "; j" #cc " %l[cc_label]" \
- : : [counter] "m" (var), ## __VA_ARGS__ \
+ : : [var] "m" (_var), ## __VA_ARGS__ \
: clobbers : cc_label); \
- return 0; \
-cc_label: \
- return 1; \
-} while (0)
-
-#define __BINARY_RMWcc_ARG " %1, "
-
+ if (0) { \
+cc_label: c = true; \
+ } \
+ c; \
+})
#else /* defined(__GCC_ASM_FLAG_OUTPUTS__) || !defined(CC_HAVE_ASM_GOTO) */
/* Use flags output or a set instruction */
-#define __GEN_RMWcc(fullop, var, cc, clobbers, ...) \
-do { \
+#define __GEN_RMWcc(fullop, _var, cc, clobbers, ...) \
+({ \
bool c; \
asm volatile (fullop CC_SET(cc) \
- : [counter] "+m" (var), CC_OUT(cc) (c) \
+ : [var] "+m" (_var), CC_OUT(cc) (c) \
: __VA_ARGS__ : clobbers); \
- return c; \
-} while (0)
-
-#define __BINARY_RMWcc_ARG " %2, "
+ c; \
+})
#endif /* defined(__GCC_ASM_FLAG_OUTPUTS__) || !defined(CC_HAVE_ASM_GOTO) */
-#define GEN_UNARY_RMWcc(op, var, arg0, cc) \
+#define GEN_UNARY_RMWcc_4(op, var, cc, arg0) \
__GEN_RMWcc(op " " arg0, var, cc, __CLOBBERS_MEM())
-#define GEN_UNARY_SUFFIXED_RMWcc(op, suffix, var, arg0, cc, clobbers...)\
- __GEN_RMWcc(op " " arg0 "\n\t" suffix, var, cc, \
- __CLOBBERS_MEM(clobbers))
+#define GEN_UNARY_RMWcc_3(op, var, cc) \
+ GEN_UNARY_RMWcc_4(op, var, cc, "%[var]")
-#define GEN_BINARY_RMWcc(op, var, vcon, val, arg0, cc) \
- __GEN_RMWcc(op __BINARY_RMWcc_ARG arg0, var, cc, \
- __CLOBBERS_MEM(), vcon (val))
+#define GEN_UNARY_RMWcc(X...) RMWcc_CONCAT(GEN_UNARY_RMWcc_, RMWcc_ARGS(X))(X)
+
+#define GEN_BINARY_RMWcc_6(op, var, cc, vcon, _val, arg0) \
+ __GEN_RMWcc(op " %[val], " arg0, var, cc, \
+ __CLOBBERS_MEM(), [val] vcon (_val))
+
+#define GEN_BINARY_RMWcc_5(op, var, cc, vcon, val) \
+ GEN_BINARY_RMWcc_6(op, var, cc, vcon, val, "%[var]")
+
+#define GEN_BINARY_RMWcc(X...) RMWcc_CONCAT(GEN_BINARY_RMWcc_, RMWcc_ARGS(X))(X)
+
+#define GEN_UNARY_SUFFIXED_RMWcc(op, suffix, var, cc, clobbers...) \
+ __GEN_RMWcc(op " %[var]\n\t" suffix, var, cc, \
+ __CLOBBERS_MEM(clobbers))
-#define GEN_BINARY_SUFFIXED_RMWcc(op, suffix, var, vcon, val, arg0, cc, \
- clobbers...) \
- __GEN_RMWcc(op __BINARY_RMWcc_ARG arg0 "\n\t" suffix, var, cc, \
- __CLOBBERS_MEM(clobbers), vcon (val))
+#define GEN_BINARY_SUFFIXED_RMWcc(op, suffix, var, cc, vcon, _val, clobbers...)\
+ __GEN_RMWcc(op " %[val], %[var]\n\t" suffix, var, cc, \
+ __CLOBBERS_MEM(clobbers), [val] vcon (_val))
#endif /* _ASM_X86_RMWcc */
#define GDT_ENTRY_TLS_MIN 12
#define GDT_ENTRY_TLS_MAX 14
-/* Abused to load per CPU data from limit */
-#define GDT_ENTRY_PER_CPU 15
+#define GDT_ENTRY_CPUNODE 15
/*
* Number of entries in the GDT table:
#define __USER_DS (GDT_ENTRY_DEFAULT_USER_DS*8 + 3)
#define __USER32_DS __USER_DS
#define __USER_CS (GDT_ENTRY_DEFAULT_USER_CS*8 + 3)
-#define __PER_CPU_SEG (GDT_ENTRY_PER_CPU*8 + 3)
+#define __CPUNODE_SEG (GDT_ENTRY_CPUNODE*8 + 3)
#endif
#define GDT_ENTRY_TLS_ENTRIES 3
#define TLS_SIZE (GDT_ENTRY_TLS_ENTRIES* 8)
+#ifdef CONFIG_X86_64
+
+/* Bit size and mask of CPU number stored in the per CPU data (and TSC_AUX) */
+#define VDSO_CPUNODE_BITS 12
+#define VDSO_CPUNODE_MASK 0xfff
+
+#ifndef __ASSEMBLY__
+
+/* Helper functions to store/load CPU and node numbers */
+
+static inline unsigned long vdso_encode_cpunode(int cpu, unsigned long node)
+{
+ return (node << VDSO_CPUNODE_BITS) | cpu;
+}
+
+static inline void vdso_read_cpunode(unsigned *cpu, unsigned *node)
+{
+ unsigned int p;
+
+ /*
+ * Load CPU and node number from the GDT. LSL is faster than RDTSCP
+ * and works on all CPUs. This is volatile so that it orders
+ * correctly with respect to barrier() and to keep GCC from cleverly
+ * hoisting it out of the calling function.
+ *
+ * If RDPID is available, use it.
+ */
+ alternative_io ("lsl %[seg],%[p]",
+ ".byte 0xf3,0x0f,0xc7,0xf8", /* RDPID %eax/rax */
+ X86_FEATURE_RDPID,
+ [p] "=a" (p), [seg] "r" (__CPUNODE_SEG));
+
+ if (cpu)
+ *cpu = (p & VDSO_CPUNODE_MASK);
+ if (node)
+ *node = (p >> VDSO_CPUNODE_BITS);
+}
+
+#endif /* !__ASSEMBLY__ */
+#endif /* CONFIG_X86_64 */
+
#ifdef __KERNEL__
/*
#ifdef CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE
#define __HAVE_ARCH_MEMCPY_FLUSHCACHE 1
-void memcpy_flushcache(void *dst, const void *src, size_t cnt);
+void __memcpy_flushcache(void *dst, const void *src, size_t cnt);
+static __always_inline void memcpy_flushcache(void *dst, const void *src, size_t cnt)
+{
+ if (__builtin_constant_p(cnt)) {
+ switch (cnt) {
+ case 4:
+ asm ("movntil %1, %0" : "=m"(*(u32 *)dst) : "r"(*(u32 *)src));
+ return;
+ case 8:
+ asm ("movntiq %1, %0" : "=m"(*(u64 *)dst) : "r"(*(u64 *)src));
+ return;
+ case 16:
+ asm ("movntiq %1, %0" : "=m"(*(u64 *)dst) : "r"(*(u64 *)src));
+ asm ("movntiq %1, %0" : "=m"(*(u64 *)(dst + 8)) : "r"(*(u64 *)(src + 8)));
+ return;
+ }
+ }
+ __memcpy_flushcache(dst, src, cnt);
+}
#endif
#endif /* __KERNEL__ */
#else
# include <asm/suspend_64.h>
#endif
+extern unsigned long restore_jump_address __visible;
+extern unsigned long jump_address_phys;
+extern unsigned long restore_cr3 __visible;
+extern unsigned long temp_pgt __visible;
+extern unsigned long relocated_restore_code __visible;
+extern int relocate_restore_code(void);
+/* Defined in hibernate_asm_32/64.S */
+extern asmlinkage __visible int restore_image(void);
unsigned long return_address;
} __attribute__((packed));
+/* routines for saving/restoring kernel state */
+extern char core_restore_code[];
+extern char restore_registers[];
+
#endif /* _ASM_X86_SUSPEND_32_H */
"4: movl %3,%0\n" \
" jmp 3b\n" \
".previous\n" \
- _ASM_EXTABLE(1b, 4b) \
- _ASM_EXTABLE(2b, 4b) \
+ _ASM_EXTABLE_UA(1b, 4b) \
+ _ASM_EXTABLE_UA(2b, 4b) \
: "=r" (err) \
: "A" (x), "r" (addr), "i" (errret), "0" (err))
" xorl %%edx,%%edx\n" \
" jmp 3b\n" \
".previous\n" \
- _ASM_EXTABLE(1b, 4b) \
- _ASM_EXTABLE(2b, 4b) \
+ _ASM_EXTABLE_UA(1b, 4b) \
+ _ASM_EXTABLE_UA(2b, 4b) \
: "=r" (retval), "=&A"(x) \
: "m" (__m(__ptr)), "m" __m(((u32 __user *)(__ptr)) + 1), \
"i" (errret), "0" (retval)); \
" xor"itype" %"rtype"1,%"rtype"1\n" \
" jmp 2b\n" \
".previous\n" \
- _ASM_EXTABLE(1b, 3b) \
+ _ASM_EXTABLE_UA(1b, 3b) \
: "=r" (err), ltype(x) \
: "m" (__m(addr)), "i" (errret), "0" (err))
"3: mov %3,%0\n" \
" jmp 2b\n" \
".previous\n" \
- _ASM_EXTABLE(1b, 3b) \
+ _ASM_EXTABLE_UA(1b, 3b) \
: "=r" (err), ltype(x) \
: "m" (__m(addr)), "i" (errret), "0" (err))
"3: mov %3,%0\n" \
" jmp 2b\n" \
".previous\n" \
- _ASM_EXTABLE(1b, 3b) \
+ _ASM_EXTABLE_UA(1b, 3b) \
: "=r"(err) \
: ltype(x), "m" (__m(addr)), "i" (errret), "0" (err))
"3:\tmov %3, %0\n" \
"\tjmp 2b\n" \
"\t.previous\n" \
- _ASM_EXTABLE(1b, 3b) \
+ _ASM_EXTABLE_UA(1b, 3b) \
: "+r" (__ret), "=a" (__old), "+m" (*(ptr)) \
: "i" (-EFAULT), "q" (__new), "1" (__old) \
: "memory" \
"3:\tmov %3, %0\n" \
"\tjmp 2b\n" \
"\t.previous\n" \
- _ASM_EXTABLE(1b, 3b) \
+ _ASM_EXTABLE_UA(1b, 3b) \
: "+r" (__ret), "=a" (__old), "+m" (*(ptr)) \
: "i" (-EFAULT), "r" (__new), "1" (__old) \
: "memory" \
"3:\tmov %3, %0\n" \
"\tjmp 2b\n" \
"\t.previous\n" \
- _ASM_EXTABLE(1b, 3b) \
+ _ASM_EXTABLE_UA(1b, 3b) \
: "+r" (__ret), "=a" (__old), "+m" (*(ptr)) \
: "i" (-EFAULT), "r" (__new), "1" (__old) \
: "memory" \
"3:\tmov %3, %0\n" \
"\tjmp 2b\n" \
"\t.previous\n" \
- _ASM_EXTABLE(1b, 3b) \
+ _ASM_EXTABLE_UA(1b, 3b) \
: "+r" (__ret), "=a" (__old), "+m" (*(ptr)) \
: "i" (-EFAULT), "r" (__new), "1" (__old) \
: "memory" \
++s->seq;
}
-#ifdef CONFIG_X86_64
-
-#define VGETCPU_CPU_MASK 0xfff
-
-static inline unsigned int __getcpu(void)
-{
- unsigned int p;
-
- /*
- * Load per CPU data from GDT. LSL is faster than RDTSCP and
- * works on all CPUs. This is volatile so that it orders
- * correctly wrt barrier() and to keep gcc from cleverly
- * hoisting it out of the calling function.
- *
- * If RDPID is available, use it.
- */
- alternative_io ("lsl %[seg],%[p]",
- ".byte 0xf3,0x0f,0xc7,0xf8", /* RDPID %eax/rax */
- X86_FEATURE_RDPID,
- [p] "=a" (p), [seg] "r" (__PER_CPU_SEG));
-
- return p;
-}
-
-#endif /* CONFIG_X86_64 */
-
#endif /* _ASM_X86_VGTOD_H */
*/
static inline int cpu_has_svm(const char **msg)
{
- if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD) {
+ if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD &&
+ boot_cpu_data.x86_vendor != X86_VENDOR_HYGON) {
if (msg)
- *msg = "not amd";
+ *msg = "not amd or hygon";
return 0;
}
#ifndef _ASM_X86_XEN_EVENTS_H
#define _ASM_X86_XEN_EVENTS_H
+#include <xen/xen.h>
+
enum ipi_vector {
XEN_RESCHEDULE_VECTOR,
XEN_CALL_FUNCTION_VECTOR,
}
break;
+ case X86_VENDOR_HYGON:
+ ideal_nops = p6_nops;
+ return;
+
case X86_VENDOR_AMD:
if (boot_cpu_data.x86 > 0xf) {
ideal_nops = p6_nops;
{
void *vaddr;
- vaddr = dma_direct_alloc(dev, size, dma_addr, flag, attrs);
+ vaddr = dma_direct_alloc_pages(dev, size, dma_addr, flag, attrs);
if (!vaddr ||
!force_iommu || dev->coherent_dma_mask <= DMA_BIT_MASK(24))
return vaddr;
goto out_free;
return vaddr;
out_free:
- dma_direct_free(dev, size, vaddr, *dma_addr, attrs);
+ dma_direct_free_pages(dev, size, vaddr, *dma_addr, attrs);
return NULL;
}
dma_addr_t dma_addr, unsigned long attrs)
{
gart_unmap_page(dev, dma_addr, size, DMA_BIDIRECTIONAL, 0);
- dma_direct_free(dev, size, vaddr, dma_addr, attrs);
+ dma_direct_free_pages(dev, size, vaddr, dma_addr, attrs);
}
static int gart_mapping_error(struct device *dev, dma_addr_t dma_addr)
{}
};
+static const struct pci_device_id hygon_root_ids[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_HYGON, PCI_DEVICE_ID_AMD_17H_ROOT) },
+ {}
+};
+
+const struct pci_device_id hygon_nb_misc_ids[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_HYGON, PCI_DEVICE_ID_AMD_17H_DF_F3) },
+ {}
+};
+
+static const struct pci_device_id hygon_nb_link_ids[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_HYGON, PCI_DEVICE_ID_AMD_17H_DF_F4) },
+ {}
+};
+
const struct amd_nb_bus_dev_range amd_nb_bus_dev_ranges[] __initconst = {
{ 0x00, 0x18, 0x20 },
{ 0xff, 0x00, 0x20 },
int amd_cache_northbridges(void)
{
- u16 i = 0;
- struct amd_northbridge *nb;
+ const struct pci_device_id *misc_ids = amd_nb_misc_ids;
+ const struct pci_device_id *link_ids = amd_nb_link_ids;
+ const struct pci_device_id *root_ids = amd_root_ids;
struct pci_dev *root, *misc, *link;
+ struct amd_northbridge *nb;
+ u16 i = 0;
if (amd_northbridges.num)
return 0;
+ if (boot_cpu_data.x86_vendor == X86_VENDOR_HYGON) {
+ root_ids = hygon_root_ids;
+ misc_ids = hygon_nb_misc_ids;
+ link_ids = hygon_nb_link_ids;
+ }
+
misc = NULL;
- while ((misc = next_northbridge(misc, amd_nb_misc_ids)) != NULL)
+ while ((misc = next_northbridge(misc, misc_ids)) != NULL)
i++;
if (!i)
link = misc = root = NULL;
for (i = 0; i != amd_northbridges.num; i++) {
node_to_amd_nb(i)->root = root =
- next_northbridge(root, amd_root_ids);
+ next_northbridge(root, root_ids);
node_to_amd_nb(i)->misc = misc =
- next_northbridge(misc, amd_nb_misc_ids);
+ next_northbridge(misc, misc_ids);
node_to_amd_nb(i)->link = link =
- next_northbridge(link, amd_nb_link_ids);
+ next_northbridge(link, link_ids);
}
if (amd_gart_present())
*/
bool __init early_is_amd_nb(u32 device)
{
+ const struct pci_device_id *misc_ids = amd_nb_misc_ids;
const struct pci_device_id *id;
u32 vendor = device & 0xffff;
+ if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD &&
+ boot_cpu_data.x86_vendor != X86_VENDOR_HYGON)
+ return false;
+
+ if (boot_cpu_data.x86_vendor == X86_VENDOR_HYGON)
+ misc_ids = hygon_nb_misc_ids;
+
device >>= 16;
- for (id = amd_nb_misc_ids; id->vendor; id++)
+ for (id = misc_ids; id->vendor; id++)
if (vendor == id->vendor && device == id->device)
return true;
return false;
u64 base, msr;
unsigned int segn_busn_bits;
- if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD)
+ if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD &&
+ boot_cpu_data.x86_vendor != X86_VENDOR_HYGON)
return NULL;
/* assume all cpus from fam10h have mmconfig */
if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD &&
boot_cpu_data.x86 >= 0xf)
return 1;
+
+ /* Hygon systems use modern APIC */
+ if (boot_cpu_data.x86_vendor == X86_VENDOR_HYGON)
+ return 1;
+
return lapic_get_version() >= 0x14;
}
(boot_cpu_data.x86 >= 15))
break;
goto no_apic;
+ case X86_VENDOR_HYGON:
+ break;
case X86_VENDOR_INTEL:
if (boot_cpu_data.x86 == 6 || boot_cpu_data.x86 == 15 ||
(boot_cpu_data.x86 == 5 && boot_cpu_has(X86_FEATURE_APIC)))
break;
}
/* If P4 and above fall through */
+ case X86_VENDOR_HYGON:
case X86_VENDOR_AMD:
def_to_bigsmp = 1;
}
struct apic_chip_data *apicd = apic_chip_data(irqd);
int vector, cpu;
- cpumask_and(vector_searchmask, vector_searchmask, affmsk);
- cpu = cpumask_first(vector_searchmask);
- if (cpu >= nr_cpu_ids)
- return -EINVAL;
+ cpumask_and(vector_searchmask, dest, affmsk);
+
/* set_affinity might call here for nothing */
if (apicd->vector && cpumask_test_cpu(apicd->cpu, vector_searchmask))
return 0;
- vector = irq_matrix_alloc_managed(vector_matrix, cpu);
+ vector = irq_matrix_alloc_managed(vector_matrix, vector_searchmask,
+ &cpu);
trace_vector_alloc_managed(irqd->irq, vector, vector);
if (vector < 0)
return vector;
// SPDX-License-Identifier: GPL-2.0
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
#include <linux/init.h>
#include <linux/sched.h>
#include <linux/kthread.h>
ssize_t ret;
unsigned long val;
+ if (!arg) {
+ pr_err("memory_corruption_check config string not provided\n");
+ return -EINVAL;
+ }
+
ret = kstrtoul(arg, 10, &val);
if (ret)
return ret;
memory_corruption_check = val;
+
return 0;
}
early_param("memory_corruption_check", set_corruption_check);
ssize_t ret;
unsigned long val;
+ if (!arg) {
+ pr_err("memory_corruption_check_period config string not provided\n");
+ return -EINVAL;
+ }
+
ret = kstrtoul(arg, 10, &val);
if (ret)
return ret;
char *end;
unsigned size;
+ if (!arg) {
+ pr_err("memory_corruption_check_size config string not provided\n");
+ return -EINVAL;
+ }
+
size = memparse(arg, &end);
if (*end == '\0')
}
if (num_scan_areas)
- printk(KERN_INFO "Scanning %d areas for low memory corruption\n", num_scan_areas);
+ pr_info("Scanning %d areas for low memory corruption\n", num_scan_areas);
}
for (; size; addr++, size -= sizeof(unsigned long)) {
if (!*addr)
continue;
- printk(KERN_ERR "Corrupted low memory at %p (%lx phys) = %08lx\n",
- addr, __pa(addr), *addr);
+ pr_err("Corrupted low memory at %p (%lx phys) = %08lx\n", addr, __pa(addr), *addr);
corruption = 1;
*addr = 0;
}
if (!num_scan_areas || !memory_corruption_check || corruption_check_period == 0)
return 0;
- printk(KERN_INFO "Scanning for low memory corruption every %d seconds\n",
- corruption_check_period);
+ pr_info("Scanning for low memory corruption every %d seconds\n", corruption_check_period);
/* First time we run the checks right away */
schedule_delayed_work(&bios_check_work, 0);
+
return 0;
}
device_initcall(start_periodic_check_for_corruption);
obj-$(CONFIG_CPU_SUP_INTEL) += intel.o intel_pconfig.o
obj-$(CONFIG_CPU_SUP_AMD) += amd.o
+obj-$(CONFIG_CPU_SUP_HYGON) += hygon.o
obj-$(CONFIG_CPU_SUP_CYRIX_32) += cyrix.o
obj-$(CONFIG_CPU_SUP_CENTAUR) += centaur.o
obj-$(CONFIG_CPU_SUP_TRANSMETA_32) += transmeta.o
}
if (cmd == SPECTRE_V2_CMD_RETPOLINE_AMD &&
+ boot_cpu_data.x86_vendor != X86_VENDOR_HYGON &&
boot_cpu_data.x86_vendor != X86_VENDOR_AMD) {
pr_err("retpoline,amd selected but CPU is not AMD. Switching to AUTO select\n");
return SPECTRE_V2_CMD_AUTO;
return;
retpoline_auto:
- if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD) {
+ if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD ||
+ boot_cpu_data.x86_vendor == X86_VENDOR_HYGON) {
retpoline_amd:
if (!boot_cpu_has(X86_FEATURE_LFENCE_RDTSC)) {
pr_err("Spectre mitigation: LFENCE not serializing, switching to generic retpoline\n");
else
amd_cpuid4(index, &eax, &ebx, &ecx);
amd_init_l3_cache(this_leaf, index);
+ } else if (boot_cpu_data.x86_vendor == X86_VENDOR_HYGON) {
+ cpuid_count(0x8000001d, index, &eax.full,
+ &ebx.full, &ecx.full, &edx);
+ amd_init_l3_cache(this_leaf, index);
} else {
cpuid_count(4, index, &eax.full, &ebx.full, &ecx.full, &edx);
}
union _cpuid4_leaf_eax cache_eax;
int i = -1;
- if (c->x86_vendor == X86_VENDOR_AMD)
+ if (c->x86_vendor == X86_VENDOR_AMD ||
+ c->x86_vendor == X86_VENDOR_HYGON)
op = 0x8000001d;
else
op = 4;
}
}
+void cacheinfo_hygon_init_llc_id(struct cpuinfo_x86 *c, int cpu, u8 node_id)
+{
+ /*
+ * We may have multiple LLCs if L3 caches exist, so check if we
+ * have an L3 cache by looking at the L3 cache CPUID leaf.
+ */
+ if (!cpuid_edx(0x80000006))
+ return;
+
+ /*
+ * LLC is at the core complex level.
+ * Core complex ID is ApicId[3] for these processors.
+ */
+ per_cpu(cpu_llc_id, cpu) = c->apicid >> 3;
+}
+
void init_amd_cacheinfo(struct cpuinfo_x86 *c)
{
}
}
+void init_hygon_cacheinfo(struct cpuinfo_x86 *c)
+{
+ num_cache_leaves = find_num_cache_leaves(c);
+}
+
void init_intel_cacheinfo(struct cpuinfo_x86 *c)
{
/* Cache sizes */
int index_msb, i;
struct cpuinfo_x86 *c = &cpu_data(cpu);
- if (c->x86_vendor == X86_VENDOR_AMD) {
+ if (c->x86_vendor == X86_VENDOR_AMD ||
+ c->x86_vendor == X86_VENDOR_HYGON) {
if (__cache_amd_cpumap_setup(cpu, index, base))
return;
}
}
static const __initconst struct x86_cpu_id cpu_no_speculation[] = {
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_CEDARVIEW, X86_FEATURE_ANY },
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_CLOVERVIEW, X86_FEATURE_ANY },
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_LINCROFT, X86_FEATURE_ANY },
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_PENWELL, X86_FEATURE_ANY },
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_PINEVIEW, X86_FEATURE_ANY },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SALTWELL, X86_FEATURE_ANY },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SALTWELL_TABLET, X86_FEATURE_ANY },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_BONNELL_MID, X86_FEATURE_ANY },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SALTWELL_MID, X86_FEATURE_ANY },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_BONNELL, X86_FEATURE_ANY },
{ X86_VENDOR_CENTAUR, 5 },
{ X86_VENDOR_INTEL, 5 },
{ X86_VENDOR_NSC, 5 },
static const __initconst struct x86_cpu_id cpu_no_meltdown[] = {
{ X86_VENDOR_AMD },
+ { X86_VENDOR_HYGON },
{}
};
/* Only list CPUs which speculate but are non susceptible to SSB */
static const __initconst struct x86_cpu_id cpu_no_spec_store_bypass[] = {
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT1 },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT },
{ X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_AIRMONT },
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT2 },
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_MERRIFIELD },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT_X },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT_MID },
{ X86_VENDOR_INTEL, 6, INTEL_FAM6_CORE_YONAH },
{ X86_VENDOR_INTEL, 6, INTEL_FAM6_XEON_PHI_KNL },
{ X86_VENDOR_INTEL, 6, INTEL_FAM6_XEON_PHI_KNM },
static const __initconst struct x86_cpu_id cpu_no_l1tf[] = {
/* in addition to cpu_no_speculation */
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT1 },
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT2 },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT_X },
{ X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_AIRMONT },
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_MERRIFIELD },
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_MOOREFIELD },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT_MID },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_AIRMONT_MID },
{ X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_GOLDMONT },
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_DENVERTON },
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_GEMINI_LAKE },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_GOLDMONT_X },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_GOLDMONT_PLUS },
{ X86_VENDOR_INTEL, 6, INTEL_FAM6_XEON_PHI_KNL },
{ X86_VENDOR_INTEL, 6, INTEL_FAM6_XEON_PHI_KNM },
{}
memset(&c->x86_capability, 0, sizeof c->x86_capability);
c->extended_cpuid_level = 0;
+ if (!have_cpuid_p())
+ identify_cpu_without_cpuid(c);
+
/* cyrix could have cpuid enabled via c_identify()*/
if (have_cpuid_p()) {
cpu_detect(c);
if (this_cpu->c_bsp_init)
this_cpu->c_bsp_init(c);
} else {
- identify_cpu_without_cpuid(c);
setup_clear_cpu_cap(X86_FEATURE_CPUID);
}
#endif
}
+#ifdef CONFIG_X86_64
+static void setup_getcpu(int cpu)
+{
+ unsigned long cpudata = vdso_encode_cpunode(cpu, early_cpu_to_node(cpu));
+ struct desc_struct d = { };
+
+ if (static_cpu_has(X86_FEATURE_RDTSCP))
+ write_rdtscp_aux(cpudata);
+
+ /* Store CPU and node number in limit. */
+ d.limit0 = cpudata;
+ d.limit1 = cpudata >> 16;
+
+ d.type = 5; /* RO data, expand down, accessed */
+ d.dpl = 3; /* Visible to user code */
+ d.s = 1; /* Not a system segment */
+ d.p = 1; /* Present */
+ d.d = 1; /* 32-bit */
+
+ write_gdt_entry(get_cpu_gdt_rw(cpu), GDT_ENTRY_CPUNODE, &d, DESCTYPE_S);
+}
+#endif
+
/*
* cpu_init() initializes state that is per-CPU. Some data is already
* initialized (naturally) in the bootstrap process, such as the GDT
early_cpu_to_node(cpu) != NUMA_NO_NODE)
set_numa_node(early_cpu_to_node(cpu));
#endif
+ setup_getcpu(cpu);
me = current;
enum cpuid_regs_idx reg);
extern void init_intel_cacheinfo(struct cpuinfo_x86 *c);
extern void init_amd_cacheinfo(struct cpuinfo_x86 *c);
+extern void init_hygon_cacheinfo(struct cpuinfo_x86 *c);
extern void detect_num_cpu_cores(struct cpuinfo_x86 *c);
extern int detect_extended_topology_early(struct cpuinfo_x86 *c);
/* enable MAPEN */
setCx86(CX86_CCR3, (ccr3 & 0x0f) | 0x10);
/* enable cpuid */
- setCx86_old(CX86_CCR4, getCx86_old(CX86_CCR4) | 0x80);
+ setCx86(CX86_CCR4, getCx86(CX86_CCR4) | 0x80);
/* disable MAPEN */
setCx86(CX86_CCR3, ccr3);
local_irq_restore(flags);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Hygon Processor Support for Linux
+ *
+ * Copyright (C) 2018 Chengdu Haiguang IC Design Co., Ltd.
+ *
+ * Author: Pu Wen <puwen@hygon.cn>
+ */
+#include <linux/io.h>
+
+#include <asm/cpu.h>
+#include <asm/smp.h>
+#include <asm/cacheinfo.h>
+#include <asm/spec-ctrl.h>
+#include <asm/delay.h>
+#ifdef CONFIG_X86_64
+# include <asm/set_memory.h>
+#endif
+
+#include "cpu.h"
+
+/*
+ * nodes_per_socket: Stores the number of nodes per socket.
+ * Refer to CPUID Fn8000_001E_ECX Node Identifiers[10:8]
+ */
+static u32 nodes_per_socket = 1;
+
+#ifdef CONFIG_NUMA
+/*
+ * To workaround broken NUMA config. Read the comment in
+ * srat_detect_node().
+ */
+static int nearby_node(int apicid)
+{
+ int i, node;
+
+ for (i = apicid - 1; i >= 0; i--) {
+ node = __apicid_to_node[i];
+ if (node != NUMA_NO_NODE && node_online(node))
+ return node;
+ }
+ for (i = apicid + 1; i < MAX_LOCAL_APIC; i++) {
+ node = __apicid_to_node[i];
+ if (node != NUMA_NO_NODE && node_online(node))
+ return node;
+ }
+ return first_node(node_online_map); /* Shouldn't happen */
+}
+#endif
+
+static void hygon_get_topology_early(struct cpuinfo_x86 *c)
+{
+ if (cpu_has(c, X86_FEATURE_TOPOEXT))
+ smp_num_siblings = ((cpuid_ebx(0x8000001e) >> 8) & 0xff) + 1;
+}
+
+/*
+ * Fixup core topology information for
+ * (1) Hygon multi-node processors
+ * Assumption: Number of cores in each internal node is the same.
+ * (2) Hygon processors supporting compute units
+ */
+static void hygon_get_topology(struct cpuinfo_x86 *c)
+{
+ u8 node_id;
+ int cpu = smp_processor_id();
+
+ /* get information required for multi-node processors */
+ if (boot_cpu_has(X86_FEATURE_TOPOEXT)) {
+ int err;
+ u32 eax, ebx, ecx, edx;
+
+ cpuid(0x8000001e, &eax, &ebx, &ecx, &edx);
+
+ node_id = ecx & 0xff;
+
+ c->cpu_core_id = ebx & 0xff;
+
+ if (smp_num_siblings > 1)
+ c->x86_max_cores /= smp_num_siblings;
+
+ /*
+ * In case leaf B is available, use it to derive
+ * topology information.
+ */
+ err = detect_extended_topology(c);
+ if (!err)
+ c->x86_coreid_bits = get_count_order(c->x86_max_cores);
+
+ cacheinfo_hygon_init_llc_id(c, cpu, node_id);
+ } else if (cpu_has(c, X86_FEATURE_NODEID_MSR)) {
+ u64 value;
+
+ rdmsrl(MSR_FAM10H_NODE_ID, value);
+ node_id = value & 7;
+
+ per_cpu(cpu_llc_id, cpu) = node_id;
+ } else
+ return;
+
+ if (nodes_per_socket > 1)
+ set_cpu_cap(c, X86_FEATURE_AMD_DCM);
+}
+
+/*
+ * On Hygon setup the lower bits of the APIC id distinguish the cores.
+ * Assumes number of cores is a power of two.
+ */
+static void hygon_detect_cmp(struct cpuinfo_x86 *c)
+{
+ unsigned int bits;
+ int cpu = smp_processor_id();
+
+ bits = c->x86_coreid_bits;
+ /* Low order bits define the core id (index of core in socket) */
+ c->cpu_core_id = c->initial_apicid & ((1 << bits)-1);
+ /* Convert the initial APIC ID into the socket ID */
+ c->phys_proc_id = c->initial_apicid >> bits;
+ /* use socket ID also for last level cache */
+ per_cpu(cpu_llc_id, cpu) = c->phys_proc_id;
+}
+
+static void srat_detect_node(struct cpuinfo_x86 *c)
+{
+#ifdef CONFIG_NUMA
+ int cpu = smp_processor_id();
+ int node;
+ unsigned int apicid = c->apicid;
+
+ node = numa_cpu_node(cpu);
+ if (node == NUMA_NO_NODE)
+ node = per_cpu(cpu_llc_id, cpu);
+
+ /*
+ * On multi-fabric platform (e.g. Numascale NumaChip) a
+ * platform-specific handler needs to be called to fixup some
+ * IDs of the CPU.
+ */
+ if (x86_cpuinit.fixup_cpu_id)
+ x86_cpuinit.fixup_cpu_id(c, node);
+
+ if (!node_online(node)) {
+ /*
+ * Two possibilities here:
+ *
+ * - The CPU is missing memory and no node was created. In
+ * that case try picking one from a nearby CPU.
+ *
+ * - The APIC IDs differ from the HyperTransport node IDs.
+ * Assume they are all increased by a constant offset, but
+ * in the same order as the HT nodeids. If that doesn't
+ * result in a usable node fall back to the path for the
+ * previous case.
+ *
+ * This workaround operates directly on the mapping between
+ * APIC ID and NUMA node, assuming certain relationship
+ * between APIC ID, HT node ID and NUMA topology. As going
+ * through CPU mapping may alter the outcome, directly
+ * access __apicid_to_node[].
+ */
+ int ht_nodeid = c->initial_apicid;
+
+ if (__apicid_to_node[ht_nodeid] != NUMA_NO_NODE)
+ node = __apicid_to_node[ht_nodeid];
+ /* Pick a nearby node */
+ if (!node_online(node))
+ node = nearby_node(apicid);
+ }
+ numa_set_node(cpu, node);
+#endif
+}
+
+static void early_init_hygon_mc(struct cpuinfo_x86 *c)
+{
+#ifdef CONFIG_SMP
+ unsigned int bits, ecx;
+
+ /* Multi core CPU? */
+ if (c->extended_cpuid_level < 0x80000008)
+ return;
+
+ ecx = cpuid_ecx(0x80000008);
+
+ c->x86_max_cores = (ecx & 0xff) + 1;
+
+ /* CPU telling us the core id bits shift? */
+ bits = (ecx >> 12) & 0xF;
+
+ /* Otherwise recompute */
+ if (bits == 0) {
+ while ((1 << bits) < c->x86_max_cores)
+ bits++;
+ }
+
+ c->x86_coreid_bits = bits;
+#endif
+}
+
+static void bsp_init_hygon(struct cpuinfo_x86 *c)
+{
+#ifdef CONFIG_X86_64
+ unsigned long long tseg;
+
+ /*
+ * Split up direct mapping around the TSEG SMM area.
+ * Don't do it for gbpages because there seems very little
+ * benefit in doing so.
+ */
+ if (!rdmsrl_safe(MSR_K8_TSEG_ADDR, &tseg)) {
+ unsigned long pfn = tseg >> PAGE_SHIFT;
+
+ pr_debug("tseg: %010llx\n", tseg);
+ if (pfn_range_is_mapped(pfn, pfn + 1))
+ set_memory_4k((unsigned long)__va(tseg), 1);
+ }
+#endif
+
+ if (cpu_has(c, X86_FEATURE_CONSTANT_TSC)) {
+ u64 val;
+
+ rdmsrl(MSR_K7_HWCR, val);
+ if (!(val & BIT(24)))
+ pr_warn(FW_BUG "TSC doesn't count with P0 frequency!\n");
+ }
+
+ if (cpu_has(c, X86_FEATURE_MWAITX))
+ use_mwaitx_delay();
+
+ if (boot_cpu_has(X86_FEATURE_TOPOEXT)) {
+ u32 ecx;
+
+ ecx = cpuid_ecx(0x8000001e);
+ nodes_per_socket = ((ecx >> 8) & 7) + 1;
+ } else if (boot_cpu_has(X86_FEATURE_NODEID_MSR)) {
+ u64 value;
+
+ rdmsrl(MSR_FAM10H_NODE_ID, value);
+ nodes_per_socket = ((value >> 3) & 7) + 1;
+ }
+
+ if (!boot_cpu_has(X86_FEATURE_AMD_SSBD) &&
+ !boot_cpu_has(X86_FEATURE_VIRT_SSBD)) {
+ /*
+ * Try to cache the base value so further operations can
+ * avoid RMW. If that faults, do not enable SSBD.
+ */
+ if (!rdmsrl_safe(MSR_AMD64_LS_CFG, &x86_amd_ls_cfg_base)) {
+ setup_force_cpu_cap(X86_FEATURE_LS_CFG_SSBD);
+ setup_force_cpu_cap(X86_FEATURE_SSBD);
+ x86_amd_ls_cfg_ssbd_mask = 1ULL << 10;
+ }
+ }
+}
+
+static void early_init_hygon(struct cpuinfo_x86 *c)
+{
+ u32 dummy;
+
+ early_init_hygon_mc(c);
+
+ set_cpu_cap(c, X86_FEATURE_K8);
+
+ rdmsr_safe(MSR_AMD64_PATCH_LEVEL, &c->microcode, &dummy);
+
+ /*
+ * c->x86_power is 8000_0007 edx. Bit 8 is TSC runs at constant rate
+ * with P/T states and does not stop in deep C-states
+ */
+ if (c->x86_power & (1 << 8)) {
+ set_cpu_cap(c, X86_FEATURE_CONSTANT_TSC);
+ set_cpu_cap(c, X86_FEATURE_NONSTOP_TSC);
+ }
+
+ /* Bit 12 of 8000_0007 edx is accumulated power mechanism. */
+ if (c->x86_power & BIT(12))
+ set_cpu_cap(c, X86_FEATURE_ACC_POWER);
+
+#ifdef CONFIG_X86_64
+ set_cpu_cap(c, X86_FEATURE_SYSCALL32);
+#endif
+
+#if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_PCI)
+ /*
+ * ApicID can always be treated as an 8-bit value for Hygon APIC So, we
+ * can safely set X86_FEATURE_EXTD_APICID unconditionally.
+ */
+ if (boot_cpu_has(X86_FEATURE_APIC))
+ set_cpu_cap(c, X86_FEATURE_EXTD_APICID);
+#endif
+
+ /*
+ * This is only needed to tell the kernel whether to use VMCALL
+ * and VMMCALL. VMMCALL is never executed except under virt, so
+ * we can set it unconditionally.
+ */
+ set_cpu_cap(c, X86_FEATURE_VMMCALL);
+
+ hygon_get_topology_early(c);
+}
+
+static void init_hygon(struct cpuinfo_x86 *c)
+{
+ early_init_hygon(c);
+
+ /*
+ * Bit 31 in normal CPUID used for nonstandard 3DNow ID;
+ * 3DNow is IDd by bit 31 in extended CPUID (1*32+31) anyway
+ */
+ clear_cpu_cap(c, 0*32+31);
+
+ set_cpu_cap(c, X86_FEATURE_REP_GOOD);
+
+ /* get apicid instead of initial apic id from cpuid */
+ c->apicid = hard_smp_processor_id();
+
+ set_cpu_cap(c, X86_FEATURE_ZEN);
+ set_cpu_cap(c, X86_FEATURE_CPB);
+
+ cpu_detect_cache_sizes(c);
+
+ hygon_detect_cmp(c);
+ hygon_get_topology(c);
+ srat_detect_node(c);
+
+ init_hygon_cacheinfo(c);
+
+ if (cpu_has(c, X86_FEATURE_XMM2)) {
+ unsigned long long val;
+ int ret;
+
+ /*
+ * A serializing LFENCE has less overhead than MFENCE, so
+ * use it for execution serialization. On families which
+ * don't have that MSR, LFENCE is already serializing.
+ * msr_set_bit() uses the safe accessors, too, even if the MSR
+ * is not present.
+ */
+ msr_set_bit(MSR_F10H_DECFG,
+ MSR_F10H_DECFG_LFENCE_SERIALIZE_BIT);
+
+ /*
+ * Verify that the MSR write was successful (could be running
+ * under a hypervisor) and only then assume that LFENCE is
+ * serializing.
+ */
+ ret = rdmsrl_safe(MSR_F10H_DECFG, &val);
+ if (!ret && (val & MSR_F10H_DECFG_LFENCE_SERIALIZE)) {
+ /* A serializing LFENCE stops RDTSC speculation */
+ set_cpu_cap(c, X86_FEATURE_LFENCE_RDTSC);
+ } else {
+ /* MFENCE stops RDTSC speculation */
+ set_cpu_cap(c, X86_FEATURE_MFENCE_RDTSC);
+ }
+ }
+
+ /*
+ * Hygon processors have APIC timer running in deep C states.
+ */
+ set_cpu_cap(c, X86_FEATURE_ARAT);
+
+ /* Hygon CPUs don't reset SS attributes on SYSRET, Xen does. */
+ if (!cpu_has(c, X86_FEATURE_XENPV))
+ set_cpu_bug(c, X86_BUG_SYSRET_SS_ATTRS);
+}
+
+static void cpu_detect_tlb_hygon(struct cpuinfo_x86 *c)
+{
+ u32 ebx, eax, ecx, edx;
+ u16 mask = 0xfff;
+
+ if (c->extended_cpuid_level < 0x80000006)
+ return;
+
+ cpuid(0x80000006, &eax, &ebx, &ecx, &edx);
+
+ tlb_lld_4k[ENTRIES] = (ebx >> 16) & mask;
+ tlb_lli_4k[ENTRIES] = ebx & mask;
+
+ /* Handle DTLB 2M and 4M sizes, fall back to L1 if L2 is disabled */
+ if (!((eax >> 16) & mask))
+ tlb_lld_2m[ENTRIES] = (cpuid_eax(0x80000005) >> 16) & 0xff;
+ else
+ tlb_lld_2m[ENTRIES] = (eax >> 16) & mask;
+
+ /* a 4M entry uses two 2M entries */
+ tlb_lld_4m[ENTRIES] = tlb_lld_2m[ENTRIES] >> 1;
+
+ /* Handle ITLB 2M and 4M sizes, fall back to L1 if L2 is disabled */
+ if (!(eax & mask)) {
+ cpuid(0x80000005, &eax, &ebx, &ecx, &edx);
+ tlb_lli_2m[ENTRIES] = eax & 0xff;
+ } else
+ tlb_lli_2m[ENTRIES] = eax & mask;
+
+ tlb_lli_4m[ENTRIES] = tlb_lli_2m[ENTRIES] >> 1;
+}
+
+static const struct cpu_dev hygon_cpu_dev = {
+ .c_vendor = "Hygon",
+ .c_ident = { "HygonGenuine" },
+ .c_early_init = early_init_hygon,
+ .c_detect_tlb = cpu_detect_tlb_hygon,
+ .c_bsp_init = bsp_init_hygon,
+ .c_init = init_hygon,
+ .c_x86_vendor = X86_VENDOR_HYGON,
+};
+
+cpu_dev_register(hygon_cpu_dev);
size_t tsize;
if (is_llc_occupancy_enabled()) {
- d->rmid_busy_llc = kcalloc(BITS_TO_LONGS(r->num_rmid),
- sizeof(unsigned long),
- GFP_KERNEL);
+ d->rmid_busy_llc = bitmap_zalloc(r->num_rmid, GFP_KERNEL);
if (!d->rmid_busy_llc)
return -ENOMEM;
INIT_DELAYED_WORK(&d->cqm_limbo, cqm_handle_limbo);
tsize = sizeof(*d->mbm_total);
d->mbm_total = kcalloc(r->num_rmid, tsize, GFP_KERNEL);
if (!d->mbm_total) {
- kfree(d->rmid_busy_llc);
+ bitmap_free(d->rmid_busy_llc);
return -ENOMEM;
}
}
tsize = sizeof(*d->mbm_local);
d->mbm_local = kcalloc(r->num_rmid, tsize, GFP_KERNEL);
if (!d->mbm_local) {
- kfree(d->rmid_busy_llc);
+ bitmap_free(d->rmid_busy_llc);
kfree(d->mbm_total);
return -ENOMEM;
}
cancel_delayed_work(&d->cqm_limbo);
}
+ /*
+ * rdt_domain "d" is going to be freed below, so clear
+ * its pointer from pseudo_lock_region struct.
+ */
+ if (d->plr)
+ d->plr->d = NULL;
+
kfree(d->ctrl_val);
kfree(d->mbps_val);
- kfree(d->rmid_busy_llc);
+ bitmap_free(d->rmid_busy_llc);
kfree(d->mbm_total);
kfree(d->mbm_local);
kfree(d);
int rdtgroup_schemata_show(struct kernfs_open_file *of,
struct seq_file *s, void *v);
bool rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d,
- u32 _cbm, int closid, bool exclusive);
+ unsigned long cbm, int closid, bool exclusive);
unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, struct rdt_domain *d,
- u32 cbm);
+ unsigned long cbm);
enum rdtgrp_mode rdtgroup_mode_by_closid(int closid);
int rdtgroup_tasks_assigned(struct rdtgroup *r);
int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);
int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp);
-bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, u32 _cbm);
+bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm);
bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d);
int rdt_pseudo_lock_init(void);
void rdt_pseudo_lock_release(void);
for_each_alloc_enabled_rdt_resource(r)
seq_printf(s, "%s:uninitialized\n", r->name);
} else if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
- seq_printf(s, "%s:%d=%x\n", rdtgrp->plr->r->name,
- rdtgrp->plr->d->id, rdtgrp->plr->cbm);
+ if (!rdtgrp->plr->d) {
+ rdt_last_cmd_clear();
+ rdt_last_cmd_puts("Cache domain offline\n");
+ ret = -ENODEV;
+ } else {
+ seq_printf(s, "%s:%d=%x\n",
+ rdtgrp->plr->r->name,
+ rdtgrp->plr->d->id,
+ rdtgrp->plr->cbm);
+ }
} else {
closid = rdtgrp->closid;
for_each_alloc_enabled_rdt_resource(r) {
#include <linux/debugfs.h>
#include <linux/kthread.h>
#include <linux/mman.h>
+#include <linux/perf_event.h>
#include <linux/pm_qos.h>
#include <linux/slab.h>
#include <linux/uaccess.h>
#include <asm/intel_rdt_sched.h>
#include <asm/perf_event.h>
+#include "../../events/perf_event.h" /* For X86_CONFIG() */
#include "intel_rdt.h"
#define CREATE_TRACE_POINTS
*/
return 0xF;
case INTEL_FAM6_ATOM_GOLDMONT:
- case INTEL_FAM6_ATOM_GEMINI_LAKE:
+ case INTEL_FAM6_ATOM_GOLDMONT_PLUS:
/*
* SDM defines bits of MSR_MISC_FEATURE_CONTROL register
* as:
return 0;
}
-/*
- * Helper to write 64bit value to MSR without tracing. Used when
- * use of the cache should be restricted and use of registers used
- * for local variables avoided.
- */
-static inline void pseudo_wrmsrl_notrace(unsigned int msr, u64 val)
-{
- __wrmsr(msr, (u32)(val & 0xffffffffULL), (u32)(val >> 32));
-}
-
/**
* pseudo_lock_minor_get - Obtain available minor number
* @minor: Pointer to where new minor number will be stored
/**
* rdtgroup_cbm_overlaps_pseudo_locked - Test if CBM or portion is pseudo-locked
* @d: RDT domain
- * @_cbm: CBM to test
+ * @cbm: CBM to test
*
- * @d represents a cache instance and @_cbm a capacity bitmask that is
- * considered for it. Determine if @_cbm overlaps with any existing
+ * @d represents a cache instance and @cbm a capacity bitmask that is
+ * considered for it. Determine if @cbm overlaps with any existing
* pseudo-locked region on @d.
*
- * Return: true if @_cbm overlaps with pseudo-locked region on @d, false
+ * @cbm is unsigned long, even if only 32 bits are used, to make the
+ * bitmap functions work correctly.
+ *
+ * Return: true if @cbm overlaps with pseudo-locked region on @d, false
* otherwise.
*/
-bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, u32 _cbm)
+bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm)
{
- unsigned long *cbm = (unsigned long *)&_cbm;
- unsigned long *cbm_b;
unsigned int cbm_len;
+ unsigned long cbm_b;
if (d->plr) {
cbm_len = d->plr->r->cache.cbm_len;
- cbm_b = (unsigned long *)&d->plr->cbm;
- if (bitmap_intersects(cbm, cbm_b, cbm_len))
+ cbm_b = d->plr->cbm;
+ if (bitmap_intersects(&cbm, &cbm_b, cbm_len))
return true;
}
return false;
struct pseudo_lock_region *plr = _plr;
unsigned long i;
u64 start, end;
-#ifdef CONFIG_KASAN
- /*
- * The registers used for local register variables are also used
- * when KASAN is active. When KASAN is active we use a regular
- * variable to ensure we always use a valid pointer to access memory.
- * The cost is that accessing this pointer, which could be in
- * cache, will be included in the measurement of memory read latency.
- */
void *mem_r;
-#else
-#ifdef CONFIG_X86_64
- register void *mem_r asm("rbx");
-#else
- register void *mem_r asm("ebx");
-#endif /* CONFIG_X86_64 */
-#endif /* CONFIG_KASAN */
local_irq_disable();
/*
- * The wrmsr call may be reordered with the assignment below it.
- * Call wrmsr as directly as possible to avoid tracing clobbering
- * local register variable used for memory pointer.
+ * Disable hardware prefetchers.
*/
- __wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0);
- mem_r = plr->kmem;
+ wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0);
+ mem_r = READ_ONCE(plr->kmem);
/*
* Dummy execute of the time measurement to load the needed
* instructions into the L1 instruction cache.
return 0;
}
-static int measure_cycles_perf_fn(void *_plr)
+/*
+ * Create a perf_event_attr for the hit and miss perf events that will
+ * be used during the performance measurement. A perf_event maintains
+ * a pointer to its perf_event_attr so a unique attribute structure is
+ * created for each perf_event.
+ *
+ * The actual configuration of the event is set right before use in order
+ * to use the X86_CONFIG macro.
+ */
+static struct perf_event_attr perf_miss_attr = {
+ .type = PERF_TYPE_RAW,
+ .size = sizeof(struct perf_event_attr),
+ .pinned = 1,
+ .disabled = 0,
+ .exclude_user = 1,
+};
+
+static struct perf_event_attr perf_hit_attr = {
+ .type = PERF_TYPE_RAW,
+ .size = sizeof(struct perf_event_attr),
+ .pinned = 1,
+ .disabled = 0,
+ .exclude_user = 1,
+};
+
+struct residency_counts {
+ u64 miss_before, hits_before;
+ u64 miss_after, hits_after;
+};
+
+static int measure_residency_fn(struct perf_event_attr *miss_attr,
+ struct perf_event_attr *hit_attr,
+ struct pseudo_lock_region *plr,
+ struct residency_counts *counts)
{
- unsigned long long l3_hits = 0, l3_miss = 0;
- u64 l3_hit_bits = 0, l3_miss_bits = 0;
- struct pseudo_lock_region *plr = _plr;
- unsigned long long l2_hits, l2_miss;
- u64 l2_hit_bits, l2_miss_bits;
- unsigned long i;
-#ifdef CONFIG_KASAN
- /*
- * The registers used for local register variables are also used
- * when KASAN is active. When KASAN is active we use regular variables
- * at the cost of including cache access latency to these variables
- * in the measurements.
- */
+ u64 hits_before = 0, hits_after = 0, miss_before = 0, miss_after = 0;
+ struct perf_event *miss_event, *hit_event;
+ int hit_pmcnum, miss_pmcnum;
unsigned int line_size;
unsigned int size;
+ unsigned long i;
void *mem_r;
-#else
- register unsigned int line_size asm("esi");
- register unsigned int size asm("edi");
-#ifdef CONFIG_X86_64
- register void *mem_r asm("rbx");
-#else
- register void *mem_r asm("ebx");
-#endif /* CONFIG_X86_64 */
-#endif /* CONFIG_KASAN */
+ u64 tmp;
+
+ miss_event = perf_event_create_kernel_counter(miss_attr, plr->cpu,
+ NULL, NULL, NULL);
+ if (IS_ERR(miss_event))
+ goto out;
+
+ hit_event = perf_event_create_kernel_counter(hit_attr, plr->cpu,
+ NULL, NULL, NULL);
+ if (IS_ERR(hit_event))
+ goto out_miss;
+
+ local_irq_disable();
+ /*
+ * Check any possible error state of events used by performing
+ * one local read.
+ */
+ if (perf_event_read_local(miss_event, &tmp, NULL, NULL)) {
+ local_irq_enable();
+ goto out_hit;
+ }
+ if (perf_event_read_local(hit_event, &tmp, NULL, NULL)) {
+ local_irq_enable();
+ goto out_hit;
+ }
+
+ /*
+ * Disable hardware prefetchers.
+ */
+ wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0);
+
+ /* Initialize rest of local variables */
+ /*
+ * Performance event has been validated right before this with
+ * interrupts disabled - it is thus safe to read the counter index.
+ */
+ miss_pmcnum = x86_perf_rdpmc_index(miss_event);
+ hit_pmcnum = x86_perf_rdpmc_index(hit_event);
+ line_size = READ_ONCE(plr->line_size);
+ mem_r = READ_ONCE(plr->kmem);
+ size = READ_ONCE(plr->size);
+
+ /*
+ * Read counter variables twice - first to load the instructions
+ * used in L1 cache, second to capture accurate value that does not
+ * include cache misses incurred because of instruction loads.
+ */
+ rdpmcl(hit_pmcnum, hits_before);
+ rdpmcl(miss_pmcnum, miss_before);
+ /*
+ * From SDM: Performing back-to-back fast reads are not guaranteed
+ * to be monotonic.
+ * Use LFENCE to ensure all previous instructions are retired
+ * before proceeding.
+ */
+ rmb();
+ rdpmcl(hit_pmcnum, hits_before);
+ rdpmcl(miss_pmcnum, miss_before);
+ /*
+ * Use LFENCE to ensure all previous instructions are retired
+ * before proceeding.
+ */
+ rmb();
+ for (i = 0; i < size; i += line_size) {
+ /*
+ * Add a barrier to prevent speculative execution of this
+ * loop reading beyond the end of the buffer.
+ */
+ rmb();
+ asm volatile("mov (%0,%1,1), %%eax\n\t"
+ :
+ : "r" (mem_r), "r" (i)
+ : "%eax", "memory");
+ }
+ /*
+ * Use LFENCE to ensure all previous instructions are retired
+ * before proceeding.
+ */
+ rmb();
+ rdpmcl(hit_pmcnum, hits_after);
+ rdpmcl(miss_pmcnum, miss_after);
+ /*
+ * Use LFENCE to ensure all previous instructions are retired
+ * before proceeding.
+ */
+ rmb();
+ /* Re-enable hardware prefetchers */
+ wrmsr(MSR_MISC_FEATURE_CONTROL, 0x0, 0x0);
+ local_irq_enable();
+out_hit:
+ perf_event_release_kernel(hit_event);
+out_miss:
+ perf_event_release_kernel(miss_event);
+out:
+ /*
+ * All counts will be zero on failure.
+ */
+ counts->miss_before = miss_before;
+ counts->hits_before = hits_before;
+ counts->miss_after = miss_after;
+ counts->hits_after = hits_after;
+ return 0;
+}
+
+static int measure_l2_residency(void *_plr)
+{
+ struct pseudo_lock_region *plr = _plr;
+ struct residency_counts counts = {0};
/*
* Non-architectural event for the Goldmont Microarchitecture
* from Intel x86 Architecture Software Developer Manual (SDM):
* MEM_LOAD_UOPS_RETIRED D1H (event number)
* Umask values:
- * L1_HIT 01H
* L2_HIT 02H
- * L1_MISS 08H
* L2_MISS 10H
- *
- * On Broadwell Microarchitecture the MEM_LOAD_UOPS_RETIRED event
- * has two "no fix" errata associated with it: BDM35 and BDM100. On
- * this platform we use the following events instead:
- * L2_RQSTS 24H (Documented in https://download.01.org/perfmon/BDW/)
- * REFERENCES FFH
- * MISS 3FH
- * LONGEST_LAT_CACHE 2EH (Documented in SDM)
- * REFERENCE 4FH
- * MISS 41H
*/
-
- /*
- * Start by setting flags for IA32_PERFEVTSELx:
- * OS (Operating system mode) 0x2
- * INT (APIC interrupt enable) 0x10
- * EN (Enable counter) 0x40
- *
- * Then add the Umask value and event number to select performance
- * event.
- */
-
switch (boot_cpu_data.x86_model) {
case INTEL_FAM6_ATOM_GOLDMONT:
- case INTEL_FAM6_ATOM_GEMINI_LAKE:
- l2_hit_bits = (0x52ULL << 16) | (0x2 << 8) | 0xd1;
- l2_miss_bits = (0x52ULL << 16) | (0x10 << 8) | 0xd1;
- break;
- case INTEL_FAM6_BROADWELL_X:
- /* On BDW the l2_hit_bits count references, not hits */
- l2_hit_bits = (0x52ULL << 16) | (0xff << 8) | 0x24;
- l2_miss_bits = (0x52ULL << 16) | (0x3f << 8) | 0x24;
- /* On BDW the l3_hit_bits count references, not hits */
- l3_hit_bits = (0x52ULL << 16) | (0x4f << 8) | 0x2e;
- l3_miss_bits = (0x52ULL << 16) | (0x41 << 8) | 0x2e;
+ case INTEL_FAM6_ATOM_GOLDMONT_PLUS:
+ perf_miss_attr.config = X86_CONFIG(.event = 0xd1,
+ .umask = 0x10);
+ perf_hit_attr.config = X86_CONFIG(.event = 0xd1,
+ .umask = 0x2);
break;
default:
goto out;
}
- local_irq_disable();
+ measure_residency_fn(&perf_miss_attr, &perf_hit_attr, plr, &counts);
/*
- * Call wrmsr direcly to avoid the local register variables from
- * being overwritten due to reordering of their assignment with
- * the wrmsr calls.
+ * If a failure prevented the measurements from succeeding
+ * tracepoints will still be written and all counts will be zero.
*/
- __wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0);
- /* Disable events and reset counters */
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_EVENTSEL0, 0x0);
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_EVENTSEL0 + 1, 0x0);
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_PERFCTR0, 0x0);
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_PERFCTR0 + 1, 0x0);
- if (l3_hit_bits > 0) {
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_EVENTSEL0 + 2, 0x0);
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_EVENTSEL0 + 3, 0x0);
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_PERFCTR0 + 2, 0x0);
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_PERFCTR0 + 3, 0x0);
- }
- /* Set and enable the L2 counters */
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_EVENTSEL0, l2_hit_bits);
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_EVENTSEL0 + 1, l2_miss_bits);
- if (l3_hit_bits > 0) {
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_EVENTSEL0 + 2,
- l3_hit_bits);
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_EVENTSEL0 + 3,
- l3_miss_bits);
- }
- mem_r = plr->kmem;
- size = plr->size;
- line_size = plr->line_size;
- for (i = 0; i < size; i += line_size) {
- asm volatile("mov (%0,%1,1), %%eax\n\t"
- :
- : "r" (mem_r), "r" (i)
- : "%eax", "memory");
- }
+ trace_pseudo_lock_l2(counts.hits_after - counts.hits_before,
+ counts.miss_after - counts.miss_before);
+out:
+ plr->thread_done = 1;
+ wake_up_interruptible(&plr->lock_thread_wq);
+ return 0;
+}
+
+static int measure_l3_residency(void *_plr)
+{
+ struct pseudo_lock_region *plr = _plr;
+ struct residency_counts counts = {0};
+
/*
- * Call wrmsr directly (no tracing) to not influence
- * the cache access counters as they are disabled.
+ * On Broadwell Microarchitecture the MEM_LOAD_UOPS_RETIRED event
+ * has two "no fix" errata associated with it: BDM35 and BDM100. On
+ * this platform the following events are used instead:
+ * LONGEST_LAT_CACHE 2EH (Documented in SDM)
+ * REFERENCE 4FH
+ * MISS 41H
*/
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_EVENTSEL0,
- l2_hit_bits & ~(0x40ULL << 16));
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_EVENTSEL0 + 1,
- l2_miss_bits & ~(0x40ULL << 16));
- if (l3_hit_bits > 0) {
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_EVENTSEL0 + 2,
- l3_hit_bits & ~(0x40ULL << 16));
- pseudo_wrmsrl_notrace(MSR_ARCH_PERFMON_EVENTSEL0 + 3,
- l3_miss_bits & ~(0x40ULL << 16));
- }
- l2_hits = native_read_pmc(0);
- l2_miss = native_read_pmc(1);
- if (l3_hit_bits > 0) {
- l3_hits = native_read_pmc(2);
- l3_miss = native_read_pmc(3);
+
+ switch (boot_cpu_data.x86_model) {
+ case INTEL_FAM6_BROADWELL_X:
+ /* On BDW the hit event counts references, not hits */
+ perf_hit_attr.config = X86_CONFIG(.event = 0x2e,
+ .umask = 0x4f);
+ perf_miss_attr.config = X86_CONFIG(.event = 0x2e,
+ .umask = 0x41);
+ break;
+ default:
+ goto out;
}
- wrmsr(MSR_MISC_FEATURE_CONTROL, 0x0, 0x0);
- local_irq_enable();
+
+ measure_residency_fn(&perf_miss_attr, &perf_hit_attr, plr, &counts);
/*
- * On BDW we count references and misses, need to adjust. Sometimes
- * the "hits" counter is a bit more than the references, for
- * example, x references but x + 1 hits. To not report invalid
- * hit values in this case we treat that as misses eaqual to
- * references.
+ * If a failure prevented the measurements from succeeding
+ * tracepoints will still be written and all counts will be zero.
*/
- if (boot_cpu_data.x86_model == INTEL_FAM6_BROADWELL_X)
- l2_hits -= (l2_miss > l2_hits ? l2_hits : l2_miss);
- trace_pseudo_lock_l2(l2_hits, l2_miss);
- if (l3_hit_bits > 0) {
- if (boot_cpu_data.x86_model == INTEL_FAM6_BROADWELL_X)
- l3_hits -= (l3_miss > l3_hits ? l3_hits : l3_miss);
- trace_pseudo_lock_l3(l3_hits, l3_miss);
+
+ counts.miss_after -= counts.miss_before;
+ if (boot_cpu_data.x86_model == INTEL_FAM6_BROADWELL_X) {
+ /*
+ * On BDW references and misses are counted, need to adjust.
+ * Sometimes the "hits" counter is a bit more than the
+ * references, for example, x references but x + 1 hits.
+ * To not report invalid hit values in this case we treat
+ * that as misses equal to references.
+ */
+ /* First compute the number of cache references measured */
+ counts.hits_after -= counts.hits_before;
+ /* Next convert references to cache hits */
+ counts.hits_after -= min(counts.miss_after, counts.hits_after);
+ } else {
+ counts.hits_after -= counts.hits_before;
}
+ trace_pseudo_lock_l3(counts.hits_after, counts.miss_after);
out:
plr->thread_done = 1;
wake_up_interruptible(&plr->lock_thread_wq);
goto out;
}
+ if (!plr->d) {
+ ret = -ENODEV;
+ goto out;
+ }
+
plr->thread_done = 0;
cpu = cpumask_first(&plr->d->cpu_mask);
if (!cpu_online(cpu)) {
goto out;
}
+ plr->cpu = cpu;
+
if (sel == 1)
thread = kthread_create_on_node(measure_cycles_lat_fn, plr,
cpu_to_node(cpu),
"pseudo_lock_measure/%u",
cpu);
else if (sel == 2)
- thread = kthread_create_on_node(measure_cycles_perf_fn, plr,
+ thread = kthread_create_on_node(measure_l2_residency, plr,
+ cpu_to_node(cpu),
+ "pseudo_lock_measure/%u",
+ cpu);
+ else if (sel == 3)
+ thread = kthread_create_on_node(measure_l3_residency, plr,
cpu_to_node(cpu),
"pseudo_lock_measure/%u",
cpu);
buf[buf_size] = '\0';
ret = kstrtoint(buf, 10, &sel);
if (ret == 0) {
- if (sel != 1)
+ if (sel != 1 && sel != 2 && sel != 3)
return -EINVAL;
ret = debugfs_file_get(file->f_path.dentry);
if (ret)
plr = rdtgrp->plr;
+ if (!plr->d) {
+ mutex_unlock(&rdtgroup_mutex);
+ return -ENODEV;
+ }
+
/*
* Task is required to run with affinity to the cpus associated
* with the pseudo-locked region. If this is not the case the task
struct seq_file *s, void *v)
{
struct rdtgroup *rdtgrp;
+ struct cpumask *mask;
int ret = 0;
rdtgrp = rdtgroup_kn_lock_live(of->kn);
if (rdtgrp) {
- if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED)
- seq_printf(s, is_cpu_list(of) ? "%*pbl\n" : "%*pb\n",
- cpumask_pr_args(&rdtgrp->plr->d->cpu_mask));
- else
+ if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
+ if (!rdtgrp->plr->d) {
+ rdt_last_cmd_clear();
+ rdt_last_cmd_puts("Cache domain offline\n");
+ ret = -ENODEV;
+ } else {
+ mask = &rdtgrp->plr->d->cpu_mask;
+ seq_printf(s, is_cpu_list(of) ?
+ "%*pbl\n" : "%*pb\n",
+ cpumask_pr_args(mask));
+ }
+ } else {
seq_printf(s, is_cpu_list(of) ? "%*pbl\n" : "%*pb\n",
cpumask_pr_args(&rdtgrp->cpu_mask));
+ }
} else {
ret = -ENOENT;
}
}
/**
- * rdtgroup_cbm_overlaps - Does CBM for intended closid overlap with other
+ * rdt_cdp_peer_get - Retrieve CDP peer if it exists
+ * @r: RDT resource to which RDT domain @d belongs
+ * @d: Cache instance for which a CDP peer is requested
+ * @r_cdp: RDT resource that shares hardware with @r (RDT resource peer)
+ * Used to return the result.
+ * @d_cdp: RDT domain that shares hardware with @d (RDT domain peer)
+ * Used to return the result.
+ *
+ * RDT resources are managed independently and by extension the RDT domains
+ * (RDT resource instances) are managed independently also. The Code and
+ * Data Prioritization (CDP) RDT resources, while managed independently,
+ * could refer to the same underlying hardware. For example,
+ * RDT_RESOURCE_L2CODE and RDT_RESOURCE_L2DATA both refer to the L2 cache.
+ *
+ * When provided with an RDT resource @r and an instance of that RDT
+ * resource @d rdt_cdp_peer_get() will return if there is a peer RDT
+ * resource and the exact instance that shares the same hardware.
+ *
+ * Return: 0 if a CDP peer was found, <0 on error or if no CDP peer exists.
+ * If a CDP peer was found, @r_cdp will point to the peer RDT resource
+ * and @d_cdp will point to the peer RDT domain.
+ */
+static int rdt_cdp_peer_get(struct rdt_resource *r, struct rdt_domain *d,
+ struct rdt_resource **r_cdp,
+ struct rdt_domain **d_cdp)
+{
+ struct rdt_resource *_r_cdp = NULL;
+ struct rdt_domain *_d_cdp = NULL;
+ int ret = 0;
+
+ switch (r->rid) {
+ case RDT_RESOURCE_L3DATA:
+ _r_cdp = &rdt_resources_all[RDT_RESOURCE_L3CODE];
+ break;
+ case RDT_RESOURCE_L3CODE:
+ _r_cdp = &rdt_resources_all[RDT_RESOURCE_L3DATA];
+ break;
+ case RDT_RESOURCE_L2DATA:
+ _r_cdp = &rdt_resources_all[RDT_RESOURCE_L2CODE];
+ break;
+ case RDT_RESOURCE_L2CODE:
+ _r_cdp = &rdt_resources_all[RDT_RESOURCE_L2DATA];
+ break;
+ default:
+ ret = -ENOENT;
+ goto out;
+ }
+
+ /*
+ * When a new CPU comes online and CDP is enabled then the new
+ * RDT domains (if any) associated with both CDP RDT resources
+ * are added in the same CPU online routine while the
+ * rdtgroup_mutex is held. It should thus not happen for one
+ * RDT domain to exist and be associated with its RDT CDP
+ * resource but there is no RDT domain associated with the
+ * peer RDT CDP resource. Hence the WARN.
+ */
+ _d_cdp = rdt_find_domain(_r_cdp, d->id, NULL);
+ if (WARN_ON(!_d_cdp)) {
+ _r_cdp = NULL;
+ ret = -EINVAL;
+ }
+
+out:
+ *r_cdp = _r_cdp;
+ *d_cdp = _d_cdp;
+
+ return ret;
+}
+
+/**
+ * __rdtgroup_cbm_overlaps - Does CBM for intended closid overlap with other
* @r: Resource to which domain instance @d belongs.
* @d: The domain instance for which @closid is being tested.
* @cbm: Capacity bitmask being tested.
* is false then overlaps with any resource group or hardware entities
* will be considered.
*
+ * @cbm is unsigned long, even if only 32 bits are used, to make the
+ * bitmap functions work correctly.
+ *
* Return: false if CBM does not overlap, true if it does.
*/
-bool rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d,
- u32 _cbm, int closid, bool exclusive)
+static bool __rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d,
+ unsigned long cbm, int closid, bool exclusive)
{
- unsigned long *cbm = (unsigned long *)&_cbm;
- unsigned long *ctrl_b;
enum rdtgrp_mode mode;
+ unsigned long ctrl_b;
u32 *ctrl;
int i;
/* Check for any overlap with regions used by hardware directly */
if (!exclusive) {
- if (bitmap_intersects(cbm,
- (unsigned long *)&r->cache.shareable_bits,
- r->cache.cbm_len))
+ ctrl_b = r->cache.shareable_bits;
+ if (bitmap_intersects(&cbm, &ctrl_b, r->cache.cbm_len))
return true;
}
/* Check for overlap with other resource groups */
ctrl = d->ctrl_val;
for (i = 0; i < closids_supported(); i++, ctrl++) {
- ctrl_b = (unsigned long *)ctrl;
+ ctrl_b = *ctrl;
mode = rdtgroup_mode_by_closid(i);
if (closid_allocated(i) && i != closid &&
mode != RDT_MODE_PSEUDO_LOCKSETUP) {
- if (bitmap_intersects(cbm, ctrl_b, r->cache.cbm_len)) {
+ if (bitmap_intersects(&cbm, &ctrl_b, r->cache.cbm_len)) {
if (exclusive) {
if (mode == RDT_MODE_EXCLUSIVE)
return true;
return false;
}
+/**
+ * rdtgroup_cbm_overlaps - Does CBM overlap with other use of hardware
+ * @r: Resource to which domain instance @d belongs.
+ * @d: The domain instance for which @closid is being tested.
+ * @cbm: Capacity bitmask being tested.
+ * @closid: Intended closid for @cbm.
+ * @exclusive: Only check if overlaps with exclusive resource groups
+ *
+ * Resources that can be allocated using a CBM can use the CBM to control
+ * the overlap of these allocations. rdtgroup_cmb_overlaps() is the test
+ * for overlap. Overlap test is not limited to the specific resource for
+ * which the CBM is intended though - when dealing with CDP resources that
+ * share the underlying hardware the overlap check should be performed on
+ * the CDP resource sharing the hardware also.
+ *
+ * Refer to description of __rdtgroup_cbm_overlaps() for the details of the
+ * overlap test.
+ *
+ * Return: true if CBM overlap detected, false if there is no overlap
+ */
+bool rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d,
+ unsigned long cbm, int closid, bool exclusive)
+{
+ struct rdt_resource *r_cdp;
+ struct rdt_domain *d_cdp;
+
+ if (__rdtgroup_cbm_overlaps(r, d, cbm, closid, exclusive))
+ return true;
+
+ if (rdt_cdp_peer_get(r, d, &r_cdp, &d_cdp) < 0)
+ return false;
+
+ return __rdtgroup_cbm_overlaps(r_cdp, d_cdp, cbm, closid, exclusive);
+}
+
/**
* rdtgroup_mode_test_exclusive - Test if this resource group can be exclusive
*
* computed by first dividing the total cache size by the CBM length to
* determine how many bytes each bit in the bitmask represents. The result
* is multiplied with the number of bits set in the bitmask.
+ *
+ * @cbm is unsigned long, even if only 32 bits are used to make the
+ * bitmap functions work correctly.
*/
unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r,
- struct rdt_domain *d, u32 cbm)
+ struct rdt_domain *d, unsigned long cbm)
{
struct cpu_cacheinfo *ci;
unsigned int size = 0;
int num_b, i;
- num_b = bitmap_weight((unsigned long *)&cbm, r->cache.cbm_len);
+ num_b = bitmap_weight(&cbm, r->cache.cbm_len);
ci = get_cpu_cacheinfo(cpumask_any(&d->cpu_mask));
for (i = 0; i < ci->num_leaves; i++) {
if (ci->info_list[i].level == r->cache_level) {
struct rdt_resource *r;
struct rdt_domain *d;
unsigned int size;
+ int ret = 0;
bool sep;
u32 ctrl;
}
if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKED) {
- seq_printf(s, "%*s:", max_name_width, rdtgrp->plr->r->name);
- size = rdtgroup_cbm_to_size(rdtgrp->plr->r,
- rdtgrp->plr->d,
- rdtgrp->plr->cbm);
- seq_printf(s, "%d=%u\n", rdtgrp->plr->d->id, size);
+ if (!rdtgrp->plr->d) {
+ rdt_last_cmd_clear();
+ rdt_last_cmd_puts("Cache domain offline\n");
+ ret = -ENODEV;
+ } else {
+ seq_printf(s, "%*s:", max_name_width,
+ rdtgrp->plr->r->name);
+ size = rdtgroup_cbm_to_size(rdtgrp->plr->r,
+ rdtgrp->plr->d,
+ rdtgrp->plr->cbm);
+ seq_printf(s, "%d=%u\n", rdtgrp->plr->d->id, size);
+ }
goto out;
}
out:
rdtgroup_kn_unlock(of->kn);
- return 0;
+ return ret;
}
/* rdtgroup information files for one cache resource. */
*/
static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
{
+ struct rdt_resource *r_cdp = NULL;
+ struct rdt_domain *d_cdp = NULL;
u32 used_b = 0, unused_b = 0;
u32 closid = rdtgrp->closid;
struct rdt_resource *r;
+ unsigned long tmp_cbm;
enum rdtgrp_mode mode;
struct rdt_domain *d;
+ u32 peer_ctl, *ctrl;
int i, ret;
- u32 *ctrl;
for_each_alloc_enabled_rdt_resource(r) {
/*
if (r->rid == RDT_RESOURCE_MBA)
continue;
list_for_each_entry(d, &r->domains, list) {
+ rdt_cdp_peer_get(r, d, &r_cdp, &d_cdp);
d->have_new_ctrl = false;
d->new_ctrl = r->cache.shareable_bits;
used_b = r->cache.shareable_bits;
mode = rdtgroup_mode_by_closid(i);
if (mode == RDT_MODE_PSEUDO_LOCKSETUP)
break;
- used_b |= *ctrl;
+ /*
+ * If CDP is active include peer
+ * domain's usage to ensure there
+ * is no overlap with an exclusive
+ * group.
+ */
+ if (d_cdp)
+ peer_ctl = d_cdp->ctrl_val[i];
+ else
+ peer_ctl = 0;
+ used_b |= *ctrl | peer_ctl;
if (mode == RDT_MODE_SHAREABLE)
- d->new_ctrl |= *ctrl;
+ d->new_ctrl |= *ctrl | peer_ctl;
}
}
if (d->plr && d->plr->cbm > 0)
* modify the CBM based on system availability.
*/
cbm_ensure_valid(&d->new_ctrl, r);
- if (bitmap_weight((unsigned long *) &d->new_ctrl,
- r->cache.cbm_len) <
- r->cache.min_cbm_bits) {
+ /*
+ * Assign the u32 CBM to an unsigned long to ensure
+ * that bitmap_weight() does not access out-of-bound
+ * memory.
+ */
+ tmp_cbm = d->new_ctrl;
+ if (bitmap_weight(&tmp_cbm, r->cache.cbm_len) <
+ r->cache.min_cbm_bits) {
rdt_last_cmd_printf("no space on %s:%d\n",
r->name, d->id);
return -ENOSPC;
{
if (rdt_resources_all[RDT_RESOURCE_L3DATA].alloc_enabled)
seq_puts(seq, ",cdp");
+
+ if (rdt_resources_all[RDT_RESOURCE_L2DATA].alloc_enabled)
+ seq_puts(seq, ",cdpl2");
+
+ if (is_mba_sc(&rdt_resources_all[RDT_RESOURCE_MBA]))
+ seq_puts(seq, ",mba_MBps");
+
return 0;
}
static DECLARE_WAIT_QUEUE_HEAD(mce_chrdev_wait);
-/* User mode helper program triggered by machine check event */
-extern char mce_helper[128];
-
static int dev_mce_log(struct notifier_block *nb, unsigned long val,
void *data)
{
memset(m, 0, sizeof(struct mce));
m->cpuvendor = boot_cpu_data.x86_vendor;
+ m->time = ktime_get_real_seconds();
+ m->cpuid = cpuid_eax(1);
+ m->microcode = boot_cpu_data.microcode;
}
/* Update fake mce registers on current CPU. */
m->bank = val;
do_inject();
+ /* Reset injection struct */
+ setup_inj_struct(&i_mce);
+
return 0;
}
void __init mcheck_vendor_init_severity(void)
{
- if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD)
+ if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD ||
+ boot_cpu_data.x86_vendor == X86_VENDOR_HYGON)
mce_severity = mce_severity_amd;
}
{
__print_mce(m);
- if (m->cpuvendor != X86_VENDOR_AMD)
+ if (m->cpuvendor != X86_VENDOR_AMD && m->cpuvendor != X86_VENDOR_HYGON)
pr_emerg_ratelimited(HW_ERR "Run the above through 'mcelog --ascii'\n");
}
bool mce_is_memory_error(struct mce *m)
{
- if (m->cpuvendor == X86_VENDOR_AMD) {
+ if (m->cpuvendor == X86_VENDOR_AMD ||
+ m->cpuvendor == X86_VENDOR_HYGON) {
return amd_mce_is_memory_error(m);
-
} else if (m->cpuvendor == X86_VENDOR_INTEL) {
/*
* Intel SDM Volume 3B - 15.9.2 Compound Error Codes
if (m->cpuvendor == X86_VENDOR_AMD && m->status & MCI_STATUS_DEFERRED)
return false;
+ if (m->cpuvendor == X86_VENDOR_HYGON && m->status & MCI_STATUS_DEFERRED)
+ return false;
+
if (m->status & MCI_STATUS_UC)
return false;
local_irq_disable();
ist_end_non_atomic();
} else {
- if (!fixup_exception(regs, X86_TRAP_MC))
+ if (!fixup_exception(regs, X86_TRAP_MC, error_code, 0))
mce_panic("Failed kernel mode recovery", &m, NULL);
}
*/
static void __mcheck_cpu_init_early(struct cpuinfo_x86 *c)
{
- if (c->x86_vendor == X86_VENDOR_AMD) {
+ if (c->x86_vendor == X86_VENDOR_AMD || c->x86_vendor == X86_VENDOR_HYGON) {
mce_flags.overflow_recov = !!cpu_has(c, X86_FEATURE_OVERFLOW_RECOV);
mce_flags.succor = !!cpu_has(c, X86_FEATURE_SUCCOR);
mce_flags.smca = !!cpu_has(c, X86_FEATURE_SMCA);
mce_amd_feature_init(c);
break;
}
+
+ case X86_VENDOR_HYGON:
+ mce_hygon_feature_init(c);
+ break;
+
case X86_VENDOR_CENTAUR:
mce_centaur_feature_init(c);
break;
static void vendor_disable_error_reporting(void)
{
/*
- * Don't clear on Intel or AMD CPUs. Some of these MSRs are socket-wide.
+ * Don't clear on Intel or AMD or Hygon CPUs. Some of these MSRs
+ * are socket-wide.
* Disabling them for just a single offlined CPU is bad, since it will
* inhibit reporting for all shared resources on the socket like the
* last level cache (LLC), the integrated memory controller (iMC), etc.
*/
if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL ||
+ boot_cpu_data.x86_vendor == X86_VENDOR_HYGON ||
boot_cpu_data.x86_vendor == X86_VENDOR_AMD)
return;
{
u32 l, h;
- if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD)
+ if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD &&
+ boot_cpu_data.x86_vendor != X86_VENDOR_HYGON)
return 0;
if (boot_cpu_data.x86 < 0xf)
return 0;
if (use_intel())
rdmsr(MSR_MTRRcap, config, dummy);
- else if (is_cpu(AMD))
+ else if (is_cpu(AMD) || is_cpu(HYGON))
config = 2;
else if (is_cpu(CYRIX) || is_cpu(CENTAUR))
config = 8;
{
/* returns the bit offset of the performance counter register */
switch (boot_cpu_data.x86_vendor) {
+ case X86_VENDOR_HYGON:
case X86_VENDOR_AMD:
if (msr >= MSR_F15H_PERF_CTR)
return (msr - MSR_F15H_PERF_CTR) >> 1;
{
/* returns the bit offset of the event selection register */
switch (boot_cpu_data.x86_vendor) {
+ case X86_VENDOR_HYGON:
case X86_VENDOR_AMD:
if (msr >= MSR_F15H_PERF_CTL)
return (msr - MSR_F15H_PERF_CTL) >> 1;
* thread's fpu state, reconstruct fxstate from the fsave
* header. Validate and sanitize the copied state.
*/
- struct fpu *fpu = &tsk->thread.fpu;
struct user_i387_ia32_struct env;
int err = 0;
void *(*poker)(void *, const void *, size_t),
int init)
{
- union jump_code_union code;
+ union jump_code_union jmp;
const unsigned char default_nop[] = { STATIC_KEY_INIT_NOP };
const unsigned char *ideal_nop = ideal_nops[NOP_ATOMIC5];
+ const void *expect, *code;
+ int line;
+
+ jmp.jump = 0xe9;
+ jmp.offset = jump_entry_target(entry) -
+ (jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE);
if (early_boot_irqs_disabled)
poker = text_poke_early;
if (type == JUMP_LABEL_JMP) {
if (init) {
- /*
- * Jump label is enabled for the first time.
- * So we expect a default_nop...
- */
- if (unlikely(memcmp((void *)entry->code, default_nop, 5)
- != 0))
- bug_at((void *)entry->code, __LINE__);
+ expect = default_nop; line = __LINE__;
} else {
- /*
- * ...otherwise expect an ideal_nop. Otherwise
- * something went horribly wrong.
- */
- if (unlikely(memcmp((void *)entry->code, ideal_nop, 5)
- != 0))
- bug_at((void *)entry->code, __LINE__);
+ expect = ideal_nop; line = __LINE__;
}
- code.jump = 0xe9;
- code.offset = entry->target -
- (entry->code + JUMP_LABEL_NOP_SIZE);
+ code = &jmp.code;
} else {
- /*
- * We are disabling this jump label. If it is not what
- * we think it is, then something must have gone wrong.
- * If this is the first initialization call, then we
- * are converting the default nop to the ideal nop.
- */
if (init) {
- if (unlikely(memcmp((void *)entry->code, default_nop, 5) != 0))
- bug_at((void *)entry->code, __LINE__);
+ expect = default_nop; line = __LINE__;
} else {
- code.jump = 0xe9;
- code.offset = entry->target -
- (entry->code + JUMP_LABEL_NOP_SIZE);
- if (unlikely(memcmp((void *)entry->code, &code, 5) != 0))
- bug_at((void *)entry->code, __LINE__);
+ expect = &jmp.code; line = __LINE__;
}
- memcpy(&code, ideal_nops[NOP_ATOMIC5], JUMP_LABEL_NOP_SIZE);
+
+ code = ideal_nop;
}
+ if (memcmp((void *)jump_entry_code(entry), expect, JUMP_LABEL_NOP_SIZE))
+ bug_at((void *)jump_entry_code(entry), line);
+
/*
* Make text_poke_bp() a default fallback poker.
*
* always nop being the 'currently valid' instruction
*
*/
- if (poker)
- (*poker)((void *)entry->code, &code, JUMP_LABEL_NOP_SIZE);
- else
- text_poke_bp((void *)entry->code, &code, JUMP_LABEL_NOP_SIZE,
- (void *)entry->code + JUMP_LABEL_NOP_SIZE);
+ if (poker) {
+ (*poker)((void *)jump_entry_code(entry), code,
+ JUMP_LABEL_NOP_SIZE);
+ return;
+ }
+
+ text_poke_bp((void *)jump_entry_code(entry), code, JUMP_LABEL_NOP_SIZE,
+ (void *)jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE);
}
void arch_jump_label_transform(struct jump_entry *entry,
*/
if (cur->fault_handler && cur->fault_handler(cur, regs, trapnr))
return 1;
-
- /*
- * In case the user-specified fault handler returned
- * zero, try to fix up.
- */
- if (fixup_exception(regs, trapnr))
- return 1;
-
- /*
- * fixup routine could not handle it,
- * Let do_page_fault() fix it.
- */
}
return 0;
}
NOKPROBE_SYMBOL(kprobe_fault_handler);
-/*
- * Wrapper routine for handling exceptions.
- */
-int kprobe_exceptions_notify(struct notifier_block *self, unsigned long val,
- void *data)
-{
- struct die_args *args = data;
- int ret = NOTIFY_DONE;
-
- if (args->regs && user_mode(args->regs))
- return ret;
-
- if (val == DIE_GPF) {
- /*
- * To be potentially processing a kprobe fault and to
- * trust the result from kprobe_running(), we have
- * be non-preemptible.
- */
- if (!preemptible() && kprobe_running() &&
- kprobe_fault_handler(args->regs, args->trapnr))
- ret = NOTIFY_STOP;
- }
- return ret;
-}
-NOKPROBE_SYMBOL(kprobe_exceptions_notify);
-
bool arch_within_kprobe_blacklist(unsigned long addr)
{
bool is_in_entry_trampoline_section = false;
opt_pre_handler(&op->kp, regs);
__this_cpu_write(current_kprobe, NULL);
}
- preempt_enable_no_resched();
+ preempt_enable();
}
NOKPROBE_SYMBOL(optimized_callback);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * This file includes headers whose assembly part includes macros which are
+ * commonly used. The macros are precompiled into assmebly file which is later
+ * assembled together with each compiled file.
+ */
+
+#include <linux/compiler.h>
+#include <asm/refcount.h>
+#include <asm/alternative-asm.h>
+#include <asm/bug.h>
+#include <asm/paravirt.h>
+#include <asm/asm.h>
+#include <asm/cpufeature.h>
+#include <asm/jump_label.h>
goto overflow;
#endif
break;
+ case R_X86_64_PC64:
+ if (*(u64 *)loc != 0)
+ goto invalid_relocation;
+ val -= (u64)loc;
+ *(u64 *)loc = val;
+ break;
default:
pr_err("%s: Unknown rela relocation: %llu\n",
me->name, ELF64_R_TYPE(rel[i].r_info));
int __init pci_swiotlb_detect_4gb(void)
{
/* don't initialize swiotlb if iommu=off (no_iommu=1) */
-#ifdef CONFIG_X86_64
if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
swiotlb = 1;
-#endif
/*
* If SME is active then swiotlb will be set to 1 so that bounce
#include <asm/vdso.h>
#include <asm/intel_rdt_sched.h>
#include <asm/unistd.h>
+#include <asm/fsgsbase.h>
#ifdef CONFIG_IA32_EMULATION
/* Not included via unistd.h */
#include <asm/unistd_32_ia32.h>
}
}
+static __always_inline void x86_fsgsbase_load(struct thread_struct *prev,
+ struct thread_struct *next)
+{
+ load_seg_legacy(prev->fsindex, prev->fsbase,
+ next->fsindex, next->fsbase, FS);
+ load_seg_legacy(prev->gsindex, prev->gsbase,
+ next->gsindex, next->gsbase, GS);
+}
+
+static unsigned long x86_fsgsbase_read_task(struct task_struct *task,
+ unsigned short selector)
+{
+ unsigned short idx = selector >> 3;
+ unsigned long base;
+
+ if (likely((selector & SEGMENT_TI_MASK) == 0)) {
+ if (unlikely(idx >= GDT_ENTRIES))
+ return 0;
+
+ /*
+ * There are no user segments in the GDT with nonzero bases
+ * other than the TLS segments.
+ */
+ if (idx < GDT_ENTRY_TLS_MIN || idx > GDT_ENTRY_TLS_MAX)
+ return 0;
+
+ idx -= GDT_ENTRY_TLS_MIN;
+ base = get_desc_base(&task->thread.tls_array[idx]);
+ } else {
+#ifdef CONFIG_MODIFY_LDT_SYSCALL
+ struct ldt_struct *ldt;
+
+ /*
+ * If performance here mattered, we could protect the LDT
+ * with RCU. This is a slow path, though, so we can just
+ * take the mutex.
+ */
+ mutex_lock(&task->mm->context.lock);
+ ldt = task->mm->context.ldt;
+ if (unlikely(idx >= ldt->nr_entries))
+ base = 0;
+ else
+ base = get_desc_base(ldt->entries + idx);
+ mutex_unlock(&task->mm->context.lock);
+#else
+ base = 0;
+#endif
+ }
+
+ return base;
+}
+
+void x86_fsbase_write_cpu(unsigned long fsbase)
+{
+ /*
+ * Set the selector to 0 as a notion, that the segment base is
+ * overwritten, which will be checked for skipping the segment load
+ * during context switch.
+ */
+ loadseg(FS, 0);
+ wrmsrl(MSR_FS_BASE, fsbase);
+}
+
+void x86_gsbase_write_cpu_inactive(unsigned long gsbase)
+{
+ /* Set the selector to 0 for the same reason as %fs above. */
+ loadseg(GS, 0);
+ wrmsrl(MSR_KERNEL_GS_BASE, gsbase);
+}
+
+unsigned long x86_fsbase_read_task(struct task_struct *task)
+{
+ unsigned long fsbase;
+
+ if (task == current)
+ fsbase = x86_fsbase_read_cpu();
+ else if (task->thread.fsindex == 0)
+ fsbase = task->thread.fsbase;
+ else
+ fsbase = x86_fsgsbase_read_task(task, task->thread.fsindex);
+
+ return fsbase;
+}
+
+unsigned long x86_gsbase_read_task(struct task_struct *task)
+{
+ unsigned long gsbase;
+
+ if (task == current)
+ gsbase = x86_gsbase_read_cpu_inactive();
+ else if (task->thread.gsindex == 0)
+ gsbase = task->thread.gsbase;
+ else
+ gsbase = x86_fsgsbase_read_task(task, task->thread.gsindex);
+
+ return gsbase;
+}
+
+int x86_fsbase_write_task(struct task_struct *task, unsigned long fsbase)
+{
+ /*
+ * Not strictly needed for %fs, but do it for symmetry
+ * with %gs
+ */
+ if (unlikely(fsbase >= TASK_SIZE_MAX))
+ return -EPERM;
+
+ preempt_disable();
+ task->thread.fsbase = fsbase;
+ if (task == current)
+ x86_fsbase_write_cpu(fsbase);
+ task->thread.fsindex = 0;
+ preempt_enable();
+
+ return 0;
+}
+
+int x86_gsbase_write_task(struct task_struct *task, unsigned long gsbase)
+{
+ if (unlikely(gsbase >= TASK_SIZE_MAX))
+ return -EPERM;
+
+ preempt_disable();
+ task->thread.gsbase = gsbase;
+ if (task == current)
+ x86_gsbase_write_cpu_inactive(gsbase);
+ task->thread.gsindex = 0;
+ preempt_enable();
+
+ return 0;
+}
+
int copy_thread_tls(unsigned long clone_flags, unsigned long sp,
unsigned long arg, struct task_struct *p, unsigned long tls)
{
if (unlikely(next->ds | prev->ds))
loadsegment(ds, next->ds);
- load_seg_legacy(prev->fsindex, prev->fsbase,
- next->fsindex, next->fsbase, FS);
- load_seg_legacy(prev->gsindex, prev->gsbase,
- next->gsindex, next->gsbase, GS);
+ x86_fsgsbase_load(prev, next);
switch_fpu_finish(next_fpu, cpu);
long do_arch_prctl_64(struct task_struct *task, int option, unsigned long arg2)
{
int ret = 0;
- int doit = task == current;
- int cpu;
switch (option) {
- case ARCH_SET_GS:
- if (arg2 >= TASK_SIZE_MAX)
- return -EPERM;
- cpu = get_cpu();
- task->thread.gsindex = 0;
- task->thread.gsbase = arg2;
- if (doit) {
- load_gs_index(0);
- ret = wrmsrl_safe(MSR_KERNEL_GS_BASE, arg2);
- }
- put_cpu();
+ case ARCH_SET_GS: {
+ ret = x86_gsbase_write_task(task, arg2);
break;
- case ARCH_SET_FS:
- /* Not strictly needed for fs, but do it for symmetry
- with gs */
- if (arg2 >= TASK_SIZE_MAX)
- return -EPERM;
- cpu = get_cpu();
- task->thread.fsindex = 0;
- task->thread.fsbase = arg2;
- if (doit) {
- /* set the selector to 0 to not confuse __switch_to */
- loadsegment(fs, 0);
- ret = wrmsrl_safe(MSR_FS_BASE, arg2);
- }
- put_cpu();
+ }
+ case ARCH_SET_FS: {
+ ret = x86_fsbase_write_task(task, arg2);
break;
+ }
case ARCH_GET_FS: {
- unsigned long base;
+ unsigned long base = x86_fsbase_read_task(task);
- if (doit)
- rdmsrl(MSR_FS_BASE, base);
- else
- base = task->thread.fsbase;
ret = put_user(base, (unsigned long __user *)arg2);
break;
}
case ARCH_GET_GS: {
- unsigned long base;
+ unsigned long base = x86_gsbase_read_task(task);
- if (doit)
- rdmsrl(MSR_KERNEL_GS_BASE, base);
- else
- base = task->thread.gsbase;
ret = put_user(base, (unsigned long __user *)arg2);
break;
}
#include <asm/hw_breakpoint.h>
#include <asm/traps.h>
#include <asm/syscall.h>
+#include <asm/fsgsbase.h>
#include "tls.h"
if (value >= TASK_SIZE_MAX)
return -EIO;
/*
- * When changing the segment base, use do_arch_prctl_64
- * to set either thread.fs or thread.fsindex and the
- * corresponding GDT slot.
+ * When changing the FS base, use the same
+ * mechanism as for do_arch_prctl_64().
*/
if (child->thread.fsbase != value)
- return do_arch_prctl_64(child, ARCH_SET_FS, value);
+ return x86_fsbase_write_task(child, value);
return 0;
case offsetof(struct user_regs_struct,gs_base):
/*
if (value >= TASK_SIZE_MAX)
return -EIO;
if (child->thread.gsbase != value)
- return do_arch_prctl_64(child, ARCH_SET_GS, value);
+ return x86_gsbase_write_task(child, value);
return 0;
#endif
}
return get_flags(task);
#ifdef CONFIG_X86_64
- case offsetof(struct user_regs_struct, fs_base): {
- /*
- * XXX: This will not behave as expected if called on
- * current or if fsindex != 0.
- */
- return task->thread.fsbase;
- }
- case offsetof(struct user_regs_struct, gs_base): {
- /*
- * XXX: This will not behave as expected if called on
- * current or if fsindex != 0.
- */
- return task->thread.gsbase;
- }
+ case offsetof(struct user_regs_struct, fs_base):
+ return x86_fsbase_read_task(task);
+ case offsetof(struct user_regs_struct, gs_base):
+ return x86_gsbase_read_task(task);
#endif
}
x86_init.hyper.guest_late_init();
e820__reserve_resources();
- e820__register_nosave_regions(max_low_pfn);
+ e820__register_nosave_regions(max_pfn);
x86_init.resources.reserve_resources();
/* if modern processor, use no delay */
if (((boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) && (boot_cpu_data.x86 == 6)) ||
+ ((boot_cpu_data.x86_vendor == X86_VENDOR_HYGON) && (boot_cpu_data.x86 >= 0x18)) ||
((boot_cpu_data.x86_vendor == X86_VENDOR_AMD) && (boot_cpu_data.x86 >= 0xF))) {
init_udelay = 0;
return;
void *mwait_ptr;
int i;
- if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD)
+ if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD ||
+ boot_cpu_data.x86_vendor == X86_VENDOR_HYGON)
return;
if (!this_cpu_has(X86_FEATURE_MWAIT))
return;
#include <asm/time.h>
#ifdef CONFIG_X86_64
-__visible volatile unsigned long jiffies __cacheline_aligned = INITIAL_JIFFIES;
+__visible volatile unsigned long jiffies __cacheline_aligned_in_smp = INITIAL_JIFFIES;
#endif
unsigned long profile_pc(struct pt_regs *regs)
}
if (!user_mode(regs)) {
- if (fixup_exception(regs, trapnr))
+ if (fixup_exception(regs, trapnr, error_code, 0))
return 0;
tsk->thread.error_code = error_code;
tsk = current;
if (!user_mode(regs)) {
- if (fixup_exception(regs, X86_TRAP_GP))
+ if (fixup_exception(regs, X86_TRAP_GP, error_code, 0))
return;
tsk->thread.error_code = error_code;
tsk->thread.trap_nr = X86_TRAP_GP;
+
+ /*
+ * To be potentially processing a kprobe fault and to
+ * trust the result from kprobe_running(), we have to
+ * be non-preemptible.
+ */
+ if (!preemptible() && kprobe_running() &&
+ kprobe_fault_handler(regs, X86_TRAP_GP))
+ return;
+
if (notify_die(DIE_GPF, "general protection fault", regs, error_code,
X86_TRAP_GP, SIGSEGV) != NOTIFY_STOP)
die("general protection fault", regs, error_code);
cond_local_irq_enable(regs);
if (!user_mode(regs)) {
- if (fixup_exception(regs, trapnr))
+ if (fixup_exception(regs, trapnr, error_code, 0))
return;
task->thread.error_code = error_code;
static DEFINE_PER_CPU_ALIGNED(struct cyc2ns, cyc2ns);
-void cyc2ns_read_begin(struct cyc2ns_data *data)
+void __always_inline cyc2ns_read_begin(struct cyc2ns_data *data)
{
int seq, idx;
} while (unlikely(seq != this_cpu_read(cyc2ns.seq.sequence)));
}
-void cyc2ns_read_end(void)
+void __always_inline cyc2ns_read_end(void)
{
preempt_enable_notrace();
}
* -johnstul@us.ibm.com "math is hard, lets go shopping!"
*/
-static inline unsigned long long cycles_2_ns(unsigned long long cyc)
+static __always_inline unsigned long long cycles_2_ns(unsigned long long cyc)
{
struct cyc2ns_data data;
unsigned long long ns;
case INTEL_FAM6_KABYLAKE_DESKTOP:
crystal_khz = 24000; /* 24.0 MHz */
break;
- case INTEL_FAM6_ATOM_DENVERTON:
+ case INTEL_FAM6_ATOM_GOLDMONT_X:
crystal_khz = 25000; /* 25.0 MHz */
break;
case INTEL_FAM6_ATOM_GOLDMONT:
};
static const struct x86_cpu_id tsc_msr_cpu_ids[] = {
- INTEL_CPU_FAM6(ATOM_PENWELL, freq_desc_pnw),
- INTEL_CPU_FAM6(ATOM_CLOVERVIEW, freq_desc_clv),
- INTEL_CPU_FAM6(ATOM_SILVERMONT1, freq_desc_byt),
+ INTEL_CPU_FAM6(ATOM_SALTWELL_MID, freq_desc_pnw),
+ INTEL_CPU_FAM6(ATOM_SALTWELL_TABLET, freq_desc_clv),
+ INTEL_CPU_FAM6(ATOM_SILVERMONT, freq_desc_byt),
+ INTEL_CPU_FAM6(ATOM_SILVERMONT_MID, freq_desc_tng),
INTEL_CPU_FAM6(ATOM_AIRMONT, freq_desc_cht),
- INTEL_CPU_FAM6(ATOM_MERRIFIELD, freq_desc_tng),
- INTEL_CPU_FAM6(ATOM_MOOREFIELD, freq_desc_ann),
+ INTEL_CPU_FAM6(ATOM_AIRMONT_MID, freq_desc_ann),
{}
};
edx == X86EMUL_CPUID_VENDOR_AMDisbetterI_edx)
return true;
- /* default: (not Intel, not AMD), apply Intel's stricter rules... */
+ /* Hygon ("HygonGenuine") */
+ if (ebx == X86EMUL_CPUID_VENDOR_HygonGenuine_ebx &&
+ ecx == X86EMUL_CPUID_VENDOR_HygonGenuine_ecx &&
+ edx == X86EMUL_CPUID_VENDOR_HygonGenuine_edx)
+ return true;
+
+ /*
+ * default: (not Intel, not AMD, not Hygon), apply Intel's
+ * stricter rules...
+ */
return false;
}
static inline bool svm_sev_enabled(void)
{
- return max_sev_asid;
+ return IS_ENABLED(CONFIG_KVM_AMD_SEV) ? max_sev_asid : 0;
}
static inline bool sev_guest(struct kvm *kvm)
{
+#ifdef CONFIG_KVM_AMD_SEV
struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
return sev->active;
+#else
+ return false;
+#endif
}
static inline int sev_get_asid(struct kvm *kvm)
goto out;
}
+ /*
+ * FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE hypercall needs the address of the
+ * base of EPT PML4 table, strip off EPT configuration information.
+ */
ret = hyperv_flush_guest_mapping(
- to_vmx(kvm_get_vcpu(kvm, 0))->ept_pointer);
+ to_vmx(kvm_get_vcpu(kvm, 0))->ept_pointer & PAGE_MASK);
out:
spin_unlock(&to_kvm_vmx(kvm)->ept_pointer_lock);
#define SRC(y...) \
9999: y; \
- _ASM_EXTABLE(9999b, 6001f)
+ _ASM_EXTABLE_UA(9999b, 6001f)
#define DST(y...) \
9999: y; \
- _ASM_EXTABLE(9999b, 6002f)
+ _ASM_EXTABLE_UA(9999b, 6002f)
#ifndef CONFIG_X86_USE_PPRO_CHECKSUM
60: jmp copy_user_handle_tail /* ecx is zerorest also */
.previous
- _ASM_EXTABLE(1b,30b)
- _ASM_EXTABLE(2b,30b)
- _ASM_EXTABLE(3b,30b)
- _ASM_EXTABLE(4b,30b)
- _ASM_EXTABLE(5b,30b)
- _ASM_EXTABLE(6b,30b)
- _ASM_EXTABLE(7b,30b)
- _ASM_EXTABLE(8b,30b)
- _ASM_EXTABLE(9b,30b)
- _ASM_EXTABLE(10b,30b)
- _ASM_EXTABLE(11b,30b)
- _ASM_EXTABLE(12b,30b)
- _ASM_EXTABLE(13b,30b)
- _ASM_EXTABLE(14b,30b)
- _ASM_EXTABLE(15b,30b)
- _ASM_EXTABLE(16b,30b)
- _ASM_EXTABLE(18b,40b)
- _ASM_EXTABLE(19b,40b)
- _ASM_EXTABLE(21b,50b)
- _ASM_EXTABLE(22b,50b)
+ _ASM_EXTABLE_UA(1b, 30b)
+ _ASM_EXTABLE_UA(2b, 30b)
+ _ASM_EXTABLE_UA(3b, 30b)
+ _ASM_EXTABLE_UA(4b, 30b)
+ _ASM_EXTABLE_UA(5b, 30b)
+ _ASM_EXTABLE_UA(6b, 30b)
+ _ASM_EXTABLE_UA(7b, 30b)
+ _ASM_EXTABLE_UA(8b, 30b)
+ _ASM_EXTABLE_UA(9b, 30b)
+ _ASM_EXTABLE_UA(10b, 30b)
+ _ASM_EXTABLE_UA(11b, 30b)
+ _ASM_EXTABLE_UA(12b, 30b)
+ _ASM_EXTABLE_UA(13b, 30b)
+ _ASM_EXTABLE_UA(14b, 30b)
+ _ASM_EXTABLE_UA(15b, 30b)
+ _ASM_EXTABLE_UA(16b, 30b)
+ _ASM_EXTABLE_UA(18b, 40b)
+ _ASM_EXTABLE_UA(19b, 40b)
+ _ASM_EXTABLE_UA(21b, 50b)
+ _ASM_EXTABLE_UA(22b, 50b)
ENDPROC(copy_user_generic_unrolled)
EXPORT_SYMBOL(copy_user_generic_unrolled)
jmp copy_user_handle_tail
.previous
- _ASM_EXTABLE(1b,11b)
- _ASM_EXTABLE(3b,12b)
+ _ASM_EXTABLE_UA(1b, 11b)
+ _ASM_EXTABLE_UA(3b, 12b)
ENDPROC(copy_user_generic_string)
EXPORT_SYMBOL(copy_user_generic_string)
jmp copy_user_handle_tail
.previous
- _ASM_EXTABLE(1b,12b)
+ _ASM_EXTABLE_UA(1b, 12b)
ENDPROC(copy_user_enhanced_fast_string)
EXPORT_SYMBOL(copy_user_enhanced_fast_string)
jmp copy_user_handle_tail
.previous
- _ASM_EXTABLE(1b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(2b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(3b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(4b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(5b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(6b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(7b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(8b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(9b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(10b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(11b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(12b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(13b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(14b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(15b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(16b,.L_fixup_4x8b_copy)
- _ASM_EXTABLE(20b,.L_fixup_8b_copy)
- _ASM_EXTABLE(21b,.L_fixup_8b_copy)
- _ASM_EXTABLE(30b,.L_fixup_4b_copy)
- _ASM_EXTABLE(31b,.L_fixup_4b_copy)
- _ASM_EXTABLE(40b,.L_fixup_1b_copy)
- _ASM_EXTABLE(41b,.L_fixup_1b_copy)
+ _ASM_EXTABLE_UA(1b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(2b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(3b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(4b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(5b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(6b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(7b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(8b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(9b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(10b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(11b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(12b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(13b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(14b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(15b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(16b, .L_fixup_4x8b_copy)
+ _ASM_EXTABLE_UA(20b, .L_fixup_8b_copy)
+ _ASM_EXTABLE_UA(21b, .L_fixup_8b_copy)
+ _ASM_EXTABLE_UA(30b, .L_fixup_4b_copy)
+ _ASM_EXTABLE_UA(31b, .L_fixup_4b_copy)
+ _ASM_EXTABLE_UA(40b, .L_fixup_1b_copy)
+ _ASM_EXTABLE_UA(41b, .L_fixup_1b_copy)
ENDPROC(__copy_user_nocache)
EXPORT_SYMBOL(__copy_user_nocache)
.macro source
10:
- _ASM_EXTABLE(10b, .Lbad_source)
+ _ASM_EXTABLE_UA(10b, .Lbad_source)
.endm
.macro dest
20:
- _ASM_EXTABLE(20b, .Lbad_dest)
+ _ASM_EXTABLE_UA(20b, .Lbad_dest)
.endm
+ /*
+ * No _ASM_EXTABLE_UA; this is used for intentional prefetch on a
+ * potentially unmapped kernel address.
+ */
.macro ignore L=.Lignore
30:
_ASM_EXTABLE(30b, \L)
END(bad_get_user_8)
#endif
- _ASM_EXTABLE(1b,bad_get_user)
- _ASM_EXTABLE(2b,bad_get_user)
- _ASM_EXTABLE(3b,bad_get_user)
+ _ASM_EXTABLE_UA(1b, bad_get_user)
+ _ASM_EXTABLE_UA(2b, bad_get_user)
+ _ASM_EXTABLE_UA(3b, bad_get_user)
#ifdef CONFIG_X86_64
- _ASM_EXTABLE(4b,bad_get_user)
+ _ASM_EXTABLE_UA(4b, bad_get_user)
#else
- _ASM_EXTABLE(4b,bad_get_user_8)
- _ASM_EXTABLE(5b,bad_get_user_8)
+ _ASM_EXTABLE_UA(4b, bad_get_user_8)
+ _ASM_EXTABLE_UA(5b, bad_get_user_8)
#endif
EXIT
END(bad_put_user)
- _ASM_EXTABLE(1b,bad_put_user)
- _ASM_EXTABLE(2b,bad_put_user)
- _ASM_EXTABLE(3b,bad_put_user)
- _ASM_EXTABLE(4b,bad_put_user)
+ _ASM_EXTABLE_UA(1b, bad_put_user)
+ _ASM_EXTABLE_UA(2b, bad_put_user)
+ _ASM_EXTABLE_UA(3b, bad_put_user)
+ _ASM_EXTABLE_UA(4b, bad_put_user)
#ifdef CONFIG_X86_32
- _ASM_EXTABLE(5b,bad_put_user)
+ _ASM_EXTABLE_UA(5b, bad_put_user)
#endif
"3: lea 0(%2,%0,4),%0\n" \
" jmp 2b\n" \
".previous\n" \
- _ASM_EXTABLE(0b,3b) \
- _ASM_EXTABLE(1b,2b) \
+ _ASM_EXTABLE_UA(0b, 3b) \
+ _ASM_EXTABLE_UA(1b, 2b) \
: "=&c"(size), "=&D" (__d0) \
: "r"(size & 3), "0"(size / 4), "1"(addr), "a"(0)); \
} while (0)
"101: lea 0(%%eax,%0,4),%0\n"
" jmp 100b\n"
".previous\n"
- _ASM_EXTABLE(1b,100b)
- _ASM_EXTABLE(2b,100b)
- _ASM_EXTABLE(3b,100b)
- _ASM_EXTABLE(4b,100b)
- _ASM_EXTABLE(5b,100b)
- _ASM_EXTABLE(6b,100b)
- _ASM_EXTABLE(7b,100b)
- _ASM_EXTABLE(8b,100b)
- _ASM_EXTABLE(9b,100b)
- _ASM_EXTABLE(10b,100b)
- _ASM_EXTABLE(11b,100b)
- _ASM_EXTABLE(12b,100b)
- _ASM_EXTABLE(13b,100b)
- _ASM_EXTABLE(14b,100b)
- _ASM_EXTABLE(15b,100b)
- _ASM_EXTABLE(16b,100b)
- _ASM_EXTABLE(17b,100b)
- _ASM_EXTABLE(18b,100b)
- _ASM_EXTABLE(19b,100b)
- _ASM_EXTABLE(20b,100b)
- _ASM_EXTABLE(21b,100b)
- _ASM_EXTABLE(22b,100b)
- _ASM_EXTABLE(23b,100b)
- _ASM_EXTABLE(24b,100b)
- _ASM_EXTABLE(25b,100b)
- _ASM_EXTABLE(26b,100b)
- _ASM_EXTABLE(27b,100b)
- _ASM_EXTABLE(28b,100b)
- _ASM_EXTABLE(29b,100b)
- _ASM_EXTABLE(30b,100b)
- _ASM_EXTABLE(31b,100b)
- _ASM_EXTABLE(32b,100b)
- _ASM_EXTABLE(33b,100b)
- _ASM_EXTABLE(34b,100b)
- _ASM_EXTABLE(35b,100b)
- _ASM_EXTABLE(36b,100b)
- _ASM_EXTABLE(37b,100b)
- _ASM_EXTABLE(99b,101b)
+ _ASM_EXTABLE_UA(1b, 100b)
+ _ASM_EXTABLE_UA(2b, 100b)
+ _ASM_EXTABLE_UA(3b, 100b)
+ _ASM_EXTABLE_UA(4b, 100b)
+ _ASM_EXTABLE_UA(5b, 100b)
+ _ASM_EXTABLE_UA(6b, 100b)
+ _ASM_EXTABLE_UA(7b, 100b)
+ _ASM_EXTABLE_UA(8b, 100b)
+ _ASM_EXTABLE_UA(9b, 100b)
+ _ASM_EXTABLE_UA(10b, 100b)
+ _ASM_EXTABLE_UA(11b, 100b)
+ _ASM_EXTABLE_UA(12b, 100b)
+ _ASM_EXTABLE_UA(13b, 100b)
+ _ASM_EXTABLE_UA(14b, 100b)
+ _ASM_EXTABLE_UA(15b, 100b)
+ _ASM_EXTABLE_UA(16b, 100b)
+ _ASM_EXTABLE_UA(17b, 100b)
+ _ASM_EXTABLE_UA(18b, 100b)
+ _ASM_EXTABLE_UA(19b, 100b)
+ _ASM_EXTABLE_UA(20b, 100b)
+ _ASM_EXTABLE_UA(21b, 100b)
+ _ASM_EXTABLE_UA(22b, 100b)
+ _ASM_EXTABLE_UA(23b, 100b)
+ _ASM_EXTABLE_UA(24b, 100b)
+ _ASM_EXTABLE_UA(25b, 100b)
+ _ASM_EXTABLE_UA(26b, 100b)
+ _ASM_EXTABLE_UA(27b, 100b)
+ _ASM_EXTABLE_UA(28b, 100b)
+ _ASM_EXTABLE_UA(29b, 100b)
+ _ASM_EXTABLE_UA(30b, 100b)
+ _ASM_EXTABLE_UA(31b, 100b)
+ _ASM_EXTABLE_UA(32b, 100b)
+ _ASM_EXTABLE_UA(33b, 100b)
+ _ASM_EXTABLE_UA(34b, 100b)
+ _ASM_EXTABLE_UA(35b, 100b)
+ _ASM_EXTABLE_UA(36b, 100b)
+ _ASM_EXTABLE_UA(37b, 100b)
+ _ASM_EXTABLE_UA(99b, 101b)
: "=&c"(size), "=&D" (d0), "=&S" (d1)
: "1"(to), "2"(from), "0"(size)
: "eax", "edx", "memory");
"9: lea 0(%%eax,%0,4),%0\n"
"16: jmp 8b\n"
".previous\n"
- _ASM_EXTABLE(0b,16b)
- _ASM_EXTABLE(1b,16b)
- _ASM_EXTABLE(2b,16b)
- _ASM_EXTABLE(21b,16b)
- _ASM_EXTABLE(3b,16b)
- _ASM_EXTABLE(31b,16b)
- _ASM_EXTABLE(4b,16b)
- _ASM_EXTABLE(41b,16b)
- _ASM_EXTABLE(10b,16b)
- _ASM_EXTABLE(51b,16b)
- _ASM_EXTABLE(11b,16b)
- _ASM_EXTABLE(61b,16b)
- _ASM_EXTABLE(12b,16b)
- _ASM_EXTABLE(71b,16b)
- _ASM_EXTABLE(13b,16b)
- _ASM_EXTABLE(81b,16b)
- _ASM_EXTABLE(14b,16b)
- _ASM_EXTABLE(91b,16b)
- _ASM_EXTABLE(6b,9b)
- _ASM_EXTABLE(7b,16b)
+ _ASM_EXTABLE_UA(0b, 16b)
+ _ASM_EXTABLE_UA(1b, 16b)
+ _ASM_EXTABLE_UA(2b, 16b)
+ _ASM_EXTABLE_UA(21b, 16b)
+ _ASM_EXTABLE_UA(3b, 16b)
+ _ASM_EXTABLE_UA(31b, 16b)
+ _ASM_EXTABLE_UA(4b, 16b)
+ _ASM_EXTABLE_UA(41b, 16b)
+ _ASM_EXTABLE_UA(10b, 16b)
+ _ASM_EXTABLE_UA(51b, 16b)
+ _ASM_EXTABLE_UA(11b, 16b)
+ _ASM_EXTABLE_UA(61b, 16b)
+ _ASM_EXTABLE_UA(12b, 16b)
+ _ASM_EXTABLE_UA(71b, 16b)
+ _ASM_EXTABLE_UA(13b, 16b)
+ _ASM_EXTABLE_UA(81b, 16b)
+ _ASM_EXTABLE_UA(14b, 16b)
+ _ASM_EXTABLE_UA(91b, 16b)
+ _ASM_EXTABLE_UA(6b, 9b)
+ _ASM_EXTABLE_UA(7b, 16b)
: "=&c"(size), "=&D" (d0), "=&S" (d1)
: "1"(to), "2"(from), "0"(size)
: "eax", "edx", "memory");
"3: lea 0(%3,%0,4),%0\n" \
" jmp 2b\n" \
".previous\n" \
- _ASM_EXTABLE(4b,5b) \
- _ASM_EXTABLE(0b,3b) \
- _ASM_EXTABLE(1b,2b) \
+ _ASM_EXTABLE_UA(4b, 5b) \
+ _ASM_EXTABLE_UA(0b, 3b) \
+ _ASM_EXTABLE_UA(1b, 2b) \
: "=&c"(size), "=&D" (__d0), "=&S" (__d1), "=r"(__d2) \
: "3"(size), "0"(size), "1"(to), "2"(from) \
: "memory"); \
"3: lea 0(%[size1],%[size8],8),%[size8]\n"
" jmp 2b\n"
".previous\n"
- _ASM_EXTABLE(0b,3b)
- _ASM_EXTABLE(1b,2b)
+ _ASM_EXTABLE_UA(0b, 3b)
+ _ASM_EXTABLE_UA(1b, 2b)
: [size8] "=&c"(size), [dst] "=&D" (__d0)
: [size1] "r"(size & 7), "[size8]" (size / 8), "[dst]"(addr));
clac();
return rc;
}
-void memcpy_flushcache(void *_dst, const void *_src, size_t size)
+void __memcpy_flushcache(void *_dst, const void *_src, size_t size)
{
unsigned long dest = (unsigned long) _dst;
unsigned long source = (unsigned long) _src;
clean_cache_range((void *) dest, size);
}
}
-EXPORT_SYMBOL_GPL(memcpy_flushcache);
+EXPORT_SYMBOL_GPL(__memcpy_flushcache);
void memcpy_page_flushcache(char *to, struct page *page, size_t offset,
size_t len)
#include <asm/kdebug.h>
typedef bool (*ex_handler_t)(const struct exception_table_entry *,
- struct pt_regs *, int);
+ struct pt_regs *, int, unsigned long,
+ unsigned long);
static inline unsigned long
ex_fixup_addr(const struct exception_table_entry *x)
}
__visible bool ex_handler_default(const struct exception_table_entry *fixup,
- struct pt_regs *regs, int trapnr)
+ struct pt_regs *regs, int trapnr,
+ unsigned long error_code,
+ unsigned long fault_addr)
{
regs->ip = ex_fixup_addr(fixup);
return true;
EXPORT_SYMBOL(ex_handler_default);
__visible bool ex_handler_fault(const struct exception_table_entry *fixup,
- struct pt_regs *regs, int trapnr)
+ struct pt_regs *regs, int trapnr,
+ unsigned long error_code,
+ unsigned long fault_addr)
{
regs->ip = ex_fixup_addr(fixup);
regs->ax = trapnr;
* result of a refcount inc/dec/add/sub.
*/
__visible bool ex_handler_refcount(const struct exception_table_entry *fixup,
- struct pt_regs *regs, int trapnr)
+ struct pt_regs *regs, int trapnr,
+ unsigned long error_code,
+ unsigned long fault_addr)
{
/* First unconditionally saturate the refcount. */
*(int *)regs->cx = INT_MIN / 2;
* out all the FPU registers) if we can't restore from the task's FPU state.
*/
__visible bool ex_handler_fprestore(const struct exception_table_entry *fixup,
- struct pt_regs *regs, int trapnr)
+ struct pt_regs *regs, int trapnr,
+ unsigned long error_code,
+ unsigned long fault_addr)
{
regs->ip = ex_fixup_addr(fixup);
}
EXPORT_SYMBOL_GPL(ex_handler_fprestore);
+/* Helper to check whether a uaccess fault indicates a kernel bug. */
+static bool bogus_uaccess(struct pt_regs *regs, int trapnr,
+ unsigned long fault_addr)
+{
+ /* This is the normal case: #PF with a fault address in userspace. */
+ if (trapnr == X86_TRAP_PF && fault_addr < TASK_SIZE_MAX)
+ return false;
+
+ /*
+ * This code can be reached for machine checks, but only if the #MC
+ * handler has already decided that it looks like a candidate for fixup.
+ * This e.g. happens when attempting to access userspace memory which
+ * the CPU can't access because of uncorrectable bad memory.
+ */
+ if (trapnr == X86_TRAP_MC)
+ return false;
+
+ /*
+ * There are two remaining exception types we might encounter here:
+ * - #PF for faulting accesses to kernel addresses
+ * - #GP for faulting accesses to noncanonical addresses
+ * Complain about anything else.
+ */
+ if (trapnr != X86_TRAP_PF && trapnr != X86_TRAP_GP) {
+ WARN(1, "unexpected trap %d in uaccess\n", trapnr);
+ return false;
+ }
+
+ /*
+ * This is a faulting memory access in kernel space, on a kernel
+ * address, in a usercopy function. This can e.g. be caused by improper
+ * use of helpers like __put_user and by improper attempts to access
+ * userspace addresses in KERNEL_DS regions.
+ * The one (semi-)legitimate exception are probe_kernel_{read,write}(),
+ * which can be invoked from places like kgdb, /dev/mem (for reading)
+ * and privileged BPF code (for reading).
+ * The probe_kernel_*() functions set the kernel_uaccess_faults_ok flag
+ * to tell us that faulting on kernel addresses, and even noncanonical
+ * addresses, in a userspace accessor does not necessarily imply a
+ * kernel bug, root might just be doing weird stuff.
+ */
+ if (current->kernel_uaccess_faults_ok)
+ return false;
+
+ /* This is bad. Refuse the fixup so that we go into die(). */
+ if (trapnr == X86_TRAP_PF) {
+ pr_emerg("BUG: pagefault on kernel address 0x%lx in non-whitelisted uaccess\n",
+ fault_addr);
+ } else {
+ pr_emerg("BUG: GPF in non-whitelisted uaccess (non-canonical address?)\n");
+ }
+ return true;
+}
+
+__visible bool ex_handler_uaccess(const struct exception_table_entry *fixup,
+ struct pt_regs *regs, int trapnr,
+ unsigned long error_code,
+ unsigned long fault_addr)
+{
+ if (bogus_uaccess(regs, trapnr, fault_addr))
+ return false;
+ regs->ip = ex_fixup_addr(fixup);
+ return true;
+}
+EXPORT_SYMBOL(ex_handler_uaccess);
+
__visible bool ex_handler_ext(const struct exception_table_entry *fixup,
- struct pt_regs *regs, int trapnr)
+ struct pt_regs *regs, int trapnr,
+ unsigned long error_code,
+ unsigned long fault_addr)
{
+ if (bogus_uaccess(regs, trapnr, fault_addr))
+ return false;
/* Special hack for uaccess_err */
current->thread.uaccess_err = 1;
regs->ip = ex_fixup_addr(fixup);
EXPORT_SYMBOL(ex_handler_ext);
__visible bool ex_handler_rdmsr_unsafe(const struct exception_table_entry *fixup,
- struct pt_regs *regs, int trapnr)
+ struct pt_regs *regs, int trapnr,
+ unsigned long error_code,
+ unsigned long fault_addr)
{
if (pr_warn_once("unchecked MSR access error: RDMSR from 0x%x at rIP: 0x%lx (%pF)\n",
(unsigned int)regs->cx, regs->ip, (void *)regs->ip))
EXPORT_SYMBOL(ex_handler_rdmsr_unsafe);
__visible bool ex_handler_wrmsr_unsafe(const struct exception_table_entry *fixup,
- struct pt_regs *regs, int trapnr)
+ struct pt_regs *regs, int trapnr,
+ unsigned long error_code,
+ unsigned long fault_addr)
{
if (pr_warn_once("unchecked MSR access error: WRMSR to 0x%x (tried to write 0x%08x%08x) at rIP: 0x%lx (%pF)\n",
(unsigned int)regs->cx, (unsigned int)regs->dx,
EXPORT_SYMBOL(ex_handler_wrmsr_unsafe);
__visible bool ex_handler_clear_fs(const struct exception_table_entry *fixup,
- struct pt_regs *regs, int trapnr)
+ struct pt_regs *regs, int trapnr,
+ unsigned long error_code,
+ unsigned long fault_addr)
{
if (static_cpu_has(X86_BUG_NULL_SEG))
asm volatile ("mov %0, %%fs" : : "rm" (__USER_DS));
asm volatile ("mov %0, %%fs" : : "rm" (0));
- return ex_handler_default(fixup, regs, trapnr);
+ return ex_handler_default(fixup, regs, trapnr, error_code, fault_addr);
}
EXPORT_SYMBOL(ex_handler_clear_fs);
return handler == ex_handler_fault;
}
-int fixup_exception(struct pt_regs *regs, int trapnr)
+int fixup_exception(struct pt_regs *regs, int trapnr, unsigned long error_code,
+ unsigned long fault_addr)
{
const struct exception_table_entry *e;
ex_handler_t handler;
return 0;
handler = ex_fixup_handler(e);
- return handler(e, regs, trapnr);
+ return handler(e, regs, trapnr, error_code, fault_addr);
}
extern unsigned int early_recursion_flag;
* result in a hard-to-debug panic.
*
* Keep in mind that not all vectors actually get here. Early
- * fage faults, for example, are special.
+ * page faults, for example, are special.
*/
- if (fixup_exception(regs, trapnr))
+ if (fixup_exception(regs, trapnr, regs->orig_ax, 0))
return;
if (fixup_bug(regs, trapnr))
#include <linux/prefetch.h> /* prefetchw */
#include <linux/context_tracking.h> /* exception_enter(), ... */
#include <linux/uaccess.h> /* faulthandler_disabled() */
+#include <linux/efi.h> /* efi_recover_from_page_fault()*/
#include <linux/mm_types.h>
#include <asm/cpufeature.h> /* boot_cpu_has, ... */
#include <asm/vsyscall.h> /* emulate_vsyscall */
#include <asm/vm86.h> /* struct vm86 */
#include <asm/mmu_context.h> /* vma_pkey() */
+#include <asm/efi.h> /* efi_recover_from_page_fault()*/
#define CREATE_TRACE_POINTS
#include <asm/trace/exceptions.h>
static nokprobe_inline int kprobes_fault(struct pt_regs *regs)
{
- int ret = 0;
-
- /* kprobe_running() needs smp_processor_id() */
- if (kprobes_built_in() && !user_mode(regs)) {
- preempt_disable();
- if (kprobe_running() && kprobe_fault_handler(regs, 14))
- ret = 1;
- preempt_enable();
- }
-
- return ret;
+ if (!kprobes_built_in())
+ return 0;
+ if (user_mode(regs))
+ return 0;
+ /*
+ * To be potentially processing a kprobe fault and to be allowed to call
+ * kprobe_running(), we have to be non-preemptible.
+ */
+ if (preemptible())
+ return 0;
+ if (!kprobe_running())
+ return 0;
+ return kprobe_fault_handler(regs, X86_TRAP_PF);
}
/*
int sig;
/* Are we prepared to handle this kernel fault? */
- if (fixup_exception(regs, X86_TRAP_PF)) {
+ if (fixup_exception(regs, X86_TRAP_PF, error_code, address)) {
/*
* Any interrupt that takes a fault gets the fixup. This makes
* the below recursive fault logic only apply to a faults from
if (is_errata93(regs, address))
return;
+ /*
+ * Buggy firmware could access regions which might page fault, try to
+ * recover from such faults.
+ */
+ if (IS_ENABLED(CONFIG_EFI))
+ efi_recover_from_page_fault(address);
+
/*
* Oops. The kernel tried to access some bad page. We'll have to
* terminate things with extreme prejudice:
#define UNSHARED_PTRS_PER_PGD \
(SHARED_KERNEL_PMD ? KERNEL_PGD_BOUNDARY : PTRS_PER_PGD)
+#define MAX_UNSHARED_PTRS_PER_PGD \
+ max_t(size_t, KERNEL_PGD_BOUNDARY, PTRS_PER_PGD)
static void pgd_set_mm(pgd_t *pgd, struct mm_struct *mm)
* and initialize the kernel pmds here.
*/
#define PREALLOCATED_PMDS UNSHARED_PTRS_PER_PGD
+#define MAX_PREALLOCATED_PMDS MAX_UNSHARED_PTRS_PER_PGD
/*
* We allocate separate PMDs for the kernel part of the user page-table
*/
#define PREALLOCATED_USER_PMDS (static_cpu_has(X86_FEATURE_PTI) ? \
KERNEL_PGD_PTRS : 0)
+#define MAX_PREALLOCATED_USER_PMDS KERNEL_PGD_PTRS
void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmd)
{
/* No need to prepopulate any pagetable entries in non-PAE modes. */
#define PREALLOCATED_PMDS 0
+#define MAX_PREALLOCATED_PMDS 0
#define PREALLOCATED_USER_PMDS 0
+#define MAX_PREALLOCATED_USER_PMDS 0
#endif /* CONFIG_X86_PAE */
static void free_pmds(struct mm_struct *mm, pmd_t *pmds[], int count)
pgd_t *pgd_alloc(struct mm_struct *mm)
{
pgd_t *pgd;
- pmd_t *u_pmds[PREALLOCATED_USER_PMDS];
- pmd_t *pmds[PREALLOCATED_PMDS];
+ pmd_t *u_pmds[MAX_PREALLOCATED_USER_PMDS];
+ pmd_t *pmds[MAX_PREALLOCATED_PMDS];
pgd = _pgd_alloc();
vendor = id & 0xffff;
device = (id>>16) & 0xffff;
- if (vendor != PCI_VENDOR_ID_AMD)
+ if (vendor != PCI_VENDOR_ID_AMD &&
+ vendor != PCI_VENDOR_ID_HYGON)
continue;
if (hb_probes[i].device == device) {
static int __init amd_postcore_init(void)
{
- if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD)
+ if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD &&
+ boot_cpu_data.x86_vendor != X86_VENDOR_HYGON)
return 0;
early_root_info_init();
static int punit_dbgfs_register(struct punit_device *punit_device)
{
- static struct dentry *dev_state;
+ struct dentry *dev_state;
punit_dbg_file = debugfs_create_dir("punit_atom", NULL);
if (!punit_dbg_file)
(kernel_ulong_t)&drv_data }
static const struct x86_cpu_id intel_punit_cpu_ids[] = {
- ICPU(INTEL_FAM6_ATOM_SILVERMONT1, punit_device_byt),
- ICPU(INTEL_FAM6_ATOM_MERRIFIELD, punit_device_tng),
+ ICPU(INTEL_FAM6_ATOM_SILVERMONT, punit_device_byt),
+ ICPU(INTEL_FAM6_ATOM_SILVERMONT_MID, punit_device_tng),
ICPU(INTEL_FAM6_ATOM_AIRMONT, punit_device_cht),
{}
};
*/
static __init int early_efi_map_fb(void)
{
- unsigned long base, size;
+ u64 base, size;
if (!early_efi_keep)
return 0;
base = boot_params.screen_info.lfb_base;
+ if (boot_params.screen_info.capabilities & VIDEO_CAPABILITY_64BIT_BASE)
+ base |= (u64)boot_params.screen_info.ext_lfb_base << 32;
size = boot_params.screen_info.lfb_size;
efi_fb = ioremap(base, size);
*/
static __ref void *early_efi_map(unsigned long start, unsigned long len)
{
- unsigned long base;
+ u64 base;
base = boot_params.screen_info.lfb_base;
+ if (boot_params.screen_info.capabilities & VIDEO_CAPABILITY_64BIT_BASE)
+ base |= (u64)boot_params.screen_info.ext_lfb_base << 32;
if (efi_fb)
return (efi_fb + start);
/*
* Makes the calling thread switch to/from efi_mm context. Can be used
- * for SetVirtualAddressMap() i.e. current->active_mm == init_mm as well
- * as during efi runtime calls i.e current->active_mm == current_mm.
- * We are not mm_dropping()/mm_grabbing() any mm, because we are not
- * losing/creating any references.
+ * in a kernel thread and user context. Preemption needs to remain disabled
+ * while the EFI-mm is borrowed. mmgrab()/mmdrop() is not used because the mm
+ * can not change under us.
+ * It should be ensured that there are no concurent calls to this function.
*/
void efi_switch_mm(struct mm_struct *mm)
{
- task_lock(current);
efi_scratch.prev_mm = current->active_mm;
current->active_mm = mm;
switch_mm(efi_scratch.prev_mm, mm, NULL);
- task_unlock(current);
}
#ifdef CONFIG_EFI_MIXED
#include <asm/efi.h>
#include <asm/uv/uv.h>
#include <asm/cpu_device_id.h>
+#include <asm/reboot.h>
#define EFI_MIN_RESERVE 5120
}
#endif
+
+/*
+ * If any access by any efi runtime service causes a page fault, then,
+ * 1. If it's efi_reset_system(), reboot through BIOS.
+ * 2. If any other efi runtime service, then
+ * a. Return error status to the efi caller process.
+ * b. Disable EFI Runtime Services forever and
+ * c. Freeze efi_rts_wq and schedule new process.
+ *
+ * @return: Returns, if the page fault is not handled. This function
+ * will never return if the page fault is handled successfully.
+ */
+void efi_recover_from_page_fault(unsigned long phys_addr)
+{
+ if (!IS_ENABLED(CONFIG_X86_64))
+ return;
+
+ /*
+ * Make sure that an efi runtime service caused the page fault.
+ * "efi_mm" cannot be used to check if the page fault had occurred
+ * in the firmware context because efi=old_map doesn't use efi_pgd.
+ */
+ if (efi_rts_work.efi_rts_id == NONE)
+ return;
+
+ /*
+ * Address range 0x0000 - 0x0fff is always mapped in the efi_pgd, so
+ * page faulting on these addresses isn't expected.
+ */
+ if (phys_addr >= 0x0000 && phys_addr <= 0x0fff)
+ return;
+
+ /*
+ * Print stack trace as it might be useful to know which EFI Runtime
+ * Service is buggy.
+ */
+ WARN(1, FW_BUG "Page fault caused by firmware at PA: 0x%lx\n",
+ phys_addr);
+
+ /*
+ * Buggy efi_reset_system() is handled differently from other EFI
+ * Runtime Services as it doesn't use efi_rts_wq. Although,
+ * native_machine_emergency_restart() says that machine_real_restart()
+ * could fail, it's better not to compilcate this fault handler
+ * because this case occurs *very* rarely and hence could be improved
+ * on a need by basis.
+ */
+ if (efi_rts_work.efi_rts_id == RESET_SYSTEM) {
+ pr_info("efi_reset_system() buggy! Reboot through BIOS\n");
+ machine_real_restart(MRR_BIOS);
+ return;
+ }
+
+ /*
+ * Before calling EFI Runtime Service, the kernel has switched the
+ * calling process to efi_mm. Hence, switch back to task_mm.
+ */
+ arch_efi_call_virt_teardown();
+
+ /* Signal error status to the efi caller process */
+ efi_rts_work.status = EFI_ABORTED;
+ complete(&efi_rts_work.efi_rts_comp);
+
+ clear_bit(EFI_RUNTIME_SERVICES, &efi.flags);
+ pr_info("Froze efi_rts_wq and disabled EFI Runtime Services\n");
+
+ /*
+ * Call schedule() in an infinite loop, so that any spurious wake ups
+ * will never run efi_rts_wq again.
+ */
+ for (;;) {
+ set_current_state(TASK_IDLE);
+ schedule();
+ }
+
+ return;
+}
* of the License.
*/
-#include <linux/gpio.h>
+#include <linux/gpio/machine.h>
#include <linux/platform_device.h>
#include <linux/regulator/machine.h>
#include <linux/regulator/fixed.h>
* real voltage and signaling are still 1.8V.
*/
.microvolts = 2000000, /* 1.8V */
- .gpio = -EINVAL,
.startup_delay = 250 * 1000, /* 250ms */
.enable_high = 1, /* active high */
.enabled_at_boot = 0, /* disabled at boot */
},
};
+static struct gpiod_lookup_table bcm43xx_vmmc_gpio_table = {
+ .dev_id = "reg-fixed-voltage.0",
+ .table = {
+ GPIO_LOOKUP("0000:00:0c.0", -1, NULL, GPIO_ACTIVE_LOW),
+ {}
+ },
+};
+
static int __init bcm43xx_regulator_register(void)
{
+ struct gpiod_lookup_table *table = &bcm43xx_vmmc_gpio_table;
+ struct gpiod_lookup *lookup = table->table;
int ret;
- bcm43xx_vmmc.gpio = get_gpio_by_name(WLAN_SFI_GPIO_ENABLE_NAME);
+ lookup[0].chip_hwnum = get_gpio_by_name(WLAN_SFI_GPIO_ENABLE_NAME);
+ gpiod_add_lookup_table(table);
+
ret = platform_device_register(&bcm43xx_vmmc_regulator);
if (ret) {
pr_err("%s: vmmc regulator register failed\n", __func__);
{ X86_VENDOR_INTEL, 6, model, X86_FEATURE_ANY, (kernel_ulong_t)&ddata }
static const struct x86_cpu_id bt_sfi_cpu_ids[] = {
- ICPU(INTEL_FAM6_ATOM_MERRIFIELD, tng_bt_sfi_data),
+ ICPU(INTEL_FAM6_ATOM_SILVERMONT_MID, tng_bt_sfi_data),
{}
};
#include <linux/kernel.h>
#include <linux/leds.h>
#include <linux/init.h>
-#include <linux/platform_data/gpio-ts5500.h>
#include <linux/platform_data/max197.h>
#include <linux/platform_device.h>
#include <linux/slab.h>
CFLAGS_cpu.o := $(nostackp)
obj-$(CONFIG_PM_SLEEP) += cpu.o
-obj-$(CONFIG_HIBERNATION) += hibernate_$(BITS).o hibernate_asm_$(BITS).o
+obj-$(CONFIG_HIBERNATION) += hibernate_$(BITS).o hibernate_asm_$(BITS).o hibernate.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hibernation support for x86
+ *
+ * Copyright (c) 2007 Rafael J. Wysocki <rjw@sisk.pl>
+ * Copyright (c) 2002 Pavel Machek <pavel@ucw.cz>
+ * Copyright (c) 2001 Patrick Mochel <mochel@osdl.org>
+ */
+#include <linux/gfp.h>
+#include <linux/smp.h>
+#include <linux/suspend.h>
+#include <linux/scatterlist.h>
+#include <linux/kdebug.h>
+
+#include <crypto/hash.h>
+
+#include <asm/e820/api.h>
+#include <asm/init.h>
+#include <asm/proto.h>
+#include <asm/page.h>
+#include <asm/pgtable.h>
+#include <asm/mtrr.h>
+#include <asm/sections.h>
+#include <asm/suspend.h>
+#include <asm/tlbflush.h>
+
+/*
+ * Address to jump to in the last phase of restore in order to get to the image
+ * kernel's text (this value is passed in the image header).
+ */
+unsigned long restore_jump_address __visible;
+unsigned long jump_address_phys;
+
+/*
+ * Value of the cr3 register from before the hibernation (this value is passed
+ * in the image header).
+ */
+unsigned long restore_cr3 __visible;
+unsigned long temp_pgt __visible;
+unsigned long relocated_restore_code __visible;
+
+/**
+ * pfn_is_nosave - check if given pfn is in the 'nosave' section
+ */
+int pfn_is_nosave(unsigned long pfn)
+{
+ unsigned long nosave_begin_pfn;
+ unsigned long nosave_end_pfn;
+
+ nosave_begin_pfn = __pa_symbol(&__nosave_begin) >> PAGE_SHIFT;
+ nosave_end_pfn = PAGE_ALIGN(__pa_symbol(&__nosave_end)) >> PAGE_SHIFT;
+
+ return pfn >= nosave_begin_pfn && pfn < nosave_end_pfn;
+}
+
+
+#define MD5_DIGEST_SIZE 16
+
+struct restore_data_record {
+ unsigned long jump_address;
+ unsigned long jump_address_phys;
+ unsigned long cr3;
+ unsigned long magic;
+ u8 e820_digest[MD5_DIGEST_SIZE];
+};
+
+#if IS_BUILTIN(CONFIG_CRYPTO_MD5)
+/**
+ * get_e820_md5 - calculate md5 according to given e820 table
+ *
+ * @table: the e820 table to be calculated
+ * @buf: the md5 result to be stored to
+ */
+static int get_e820_md5(struct e820_table *table, void *buf)
+{
+ struct crypto_shash *tfm;
+ struct shash_desc *desc;
+ int size;
+ int ret = 0;
+
+ tfm = crypto_alloc_shash("md5", 0, 0);
+ if (IS_ERR(tfm))
+ return -ENOMEM;
+
+ desc = kmalloc(sizeof(struct shash_desc) + crypto_shash_descsize(tfm),
+ GFP_KERNEL);
+ if (!desc) {
+ ret = -ENOMEM;
+ goto free_tfm;
+ }
+
+ desc->tfm = tfm;
+ desc->flags = 0;
+
+ size = offsetof(struct e820_table, entries) +
+ sizeof(struct e820_entry) * table->nr_entries;
+
+ if (crypto_shash_digest(desc, (u8 *)table, size, buf))
+ ret = -EINVAL;
+
+ kzfree(desc);
+
+free_tfm:
+ crypto_free_shash(tfm);
+ return ret;
+}
+
+static int hibernation_e820_save(void *buf)
+{
+ return get_e820_md5(e820_table_firmware, buf);
+}
+
+static bool hibernation_e820_mismatch(void *buf)
+{
+ int ret;
+ u8 result[MD5_DIGEST_SIZE];
+
+ memset(result, 0, MD5_DIGEST_SIZE);
+ /* If there is no digest in suspend kernel, let it go. */
+ if (!memcmp(result, buf, MD5_DIGEST_SIZE))
+ return false;
+
+ ret = get_e820_md5(e820_table_firmware, result);
+ if (ret)
+ return true;
+
+ return memcmp(result, buf, MD5_DIGEST_SIZE) ? true : false;
+}
+#else
+static int hibernation_e820_save(void *buf)
+{
+ return 0;
+}
+
+static bool hibernation_e820_mismatch(void *buf)
+{
+ /* If md5 is not builtin for restore kernel, let it go. */
+ return false;
+}
+#endif
+
+#ifdef CONFIG_X86_64
+#define RESTORE_MAGIC 0x23456789ABCDEF01UL
+#else
+#define RESTORE_MAGIC 0x12345678UL
+#endif
+
+/**
+ * arch_hibernation_header_save - populate the architecture specific part
+ * of a hibernation image header
+ * @addr: address to save the data at
+ */
+int arch_hibernation_header_save(void *addr, unsigned int max_size)
+{
+ struct restore_data_record *rdr = addr;
+
+ if (max_size < sizeof(struct restore_data_record))
+ return -EOVERFLOW;
+ rdr->magic = RESTORE_MAGIC;
+ rdr->jump_address = (unsigned long)restore_registers;
+ rdr->jump_address_phys = __pa_symbol(restore_registers);
+
+ /*
+ * The restore code fixes up CR3 and CR4 in the following sequence:
+ *
+ * [in hibernation asm]
+ * 1. CR3 <= temporary page tables
+ * 2. CR4 <= mmu_cr4_features (from the kernel that restores us)
+ * 3. CR3 <= rdr->cr3
+ * 4. CR4 <= mmu_cr4_features (from us, i.e. the image kernel)
+ * [in restore_processor_state()]
+ * 5. CR4 <= saved CR4
+ * 6. CR3 <= saved CR3
+ *
+ * Our mmu_cr4_features has CR4.PCIDE=0, and toggling
+ * CR4.PCIDE while CR3's PCID bits are nonzero is illegal, so
+ * rdr->cr3 needs to point to valid page tables but must not
+ * have any of the PCID bits set.
+ */
+ rdr->cr3 = restore_cr3 & ~CR3_PCID_MASK;
+
+ return hibernation_e820_save(rdr->e820_digest);
+}
+
+/**
+ * arch_hibernation_header_restore - read the architecture specific data
+ * from the hibernation image header
+ * @addr: address to read the data from
+ */
+int arch_hibernation_header_restore(void *addr)
+{
+ struct restore_data_record *rdr = addr;
+
+ if (rdr->magic != RESTORE_MAGIC) {
+ pr_crit("Unrecognized hibernate image header format!\n");
+ return -EINVAL;
+ }
+
+ restore_jump_address = rdr->jump_address;
+ jump_address_phys = rdr->jump_address_phys;
+ restore_cr3 = rdr->cr3;
+
+ if (hibernation_e820_mismatch(rdr->e820_digest)) {
+ pr_crit("Hibernate inconsistent memory map detected!\n");
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+int relocate_restore_code(void)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+ pte_t *pte;
+
+ relocated_restore_code = get_safe_page(GFP_ATOMIC);
+ if (!relocated_restore_code)
+ return -ENOMEM;
+
+ memcpy((void *)relocated_restore_code, core_restore_code, PAGE_SIZE);
+
+ /* Make the page containing the relocated code executable */
+ pgd = (pgd_t *)__va(read_cr3_pa()) +
+ pgd_index(relocated_restore_code);
+ p4d = p4d_offset(pgd, relocated_restore_code);
+ if (p4d_large(*p4d)) {
+ set_p4d(p4d, __p4d(p4d_val(*p4d) & ~_PAGE_NX));
+ goto out;
+ }
+ pud = pud_offset(p4d, relocated_restore_code);
+ if (pud_large(*pud)) {
+ set_pud(pud, __pud(pud_val(*pud) & ~_PAGE_NX));
+ goto out;
+ }
+ pmd = pmd_offset(pud, relocated_restore_code);
+ if (pmd_large(*pmd)) {
+ set_pmd(pmd, __pmd(pmd_val(*pmd) & ~_PAGE_NX));
+ goto out;
+ }
+ pte = pte_offset_kernel(pmd, relocated_restore_code);
+ set_pte(pte, __pte(pte_val(*pte) & ~_PAGE_NX));
+out:
+ __flush_tlb_all();
+ return 0;
+}
#include <asm/pgtable.h>
#include <asm/mmzone.h>
#include <asm/sections.h>
-
-/* Defined in hibernate_asm_32.S */
-extern int restore_image(void);
+#include <asm/suspend.h>
/* Pointer to the temporary resume page tables */
pgd_t *resume_pg_dir;
#endif
}
+static int set_up_temporary_text_mapping(pgd_t *pgd_base)
+{
+ pgd_t *pgd;
+ pmd_t *pmd;
+ pte_t *pte;
+
+ pgd = pgd_base + pgd_index(restore_jump_address);
+
+ pmd = resume_one_md_table_init(pgd);
+ if (!pmd)
+ return -ENOMEM;
+
+ if (boot_cpu_has(X86_FEATURE_PSE)) {
+ set_pmd(pmd + pmd_index(restore_jump_address),
+ __pmd((jump_address_phys & PMD_MASK) | pgprot_val(PAGE_KERNEL_LARGE_EXEC)));
+ } else {
+ pte = resume_one_page_table_init(pmd);
+ if (!pte)
+ return -ENOMEM;
+ set_pte(pte + pte_index(restore_jump_address),
+ __pte((jump_address_phys & PAGE_MASK) | pgprot_val(PAGE_KERNEL_EXEC)));
+ }
+
+ return 0;
+}
+
asmlinkage int swsusp_arch_resume(void)
{
int error;
return -ENOMEM;
resume_init_first_level_page_table(resume_pg_dir);
+
+ error = set_up_temporary_text_mapping(resume_pg_dir);
+ if (error)
+ return error;
+
error = resume_physical_mapping_init(resume_pg_dir);
if (error)
return error;
+ temp_pgt = __pa(resume_pg_dir);
+
+ error = relocate_restore_code();
+ if (error)
+ return error;
+
/* We have got enough memory and from now on we cannot recover */
restore_image();
return 0;
}
-
-/*
- * pfn_is_nosave - check if given pfn is in the 'nosave' section
- */
-
-int pfn_is_nosave(unsigned long pfn)
-{
- unsigned long nosave_begin_pfn = __pa_symbol(&__nosave_begin) >> PAGE_SHIFT;
- unsigned long nosave_end_pfn = PAGE_ALIGN(__pa_symbol(&__nosave_end)) >> PAGE_SHIFT;
- return (pfn >= nosave_begin_pfn) && (pfn < nosave_end_pfn);
-}
#include <asm/suspend.h>
#include <asm/tlbflush.h>
-/* Defined in hibernate_asm_64.S */
-extern asmlinkage __visible int restore_image(void);
-
-/*
- * Address to jump to in the last phase of restore in order to get to the image
- * kernel's text (this value is passed in the image header).
- */
-unsigned long restore_jump_address __visible;
-unsigned long jump_address_phys;
-
-/*
- * Value of the cr3 register from before the hibernation (this value is passed
- * in the image header).
- */
-unsigned long restore_cr3 __visible;
-
-unsigned long temp_level4_pgt __visible;
-
-unsigned long relocated_restore_code __visible;
-
static int set_up_temporary_text_mapping(pgd_t *pgd)
{
pmd_t *pmd;
return result;
}
- temp_level4_pgt = __pa(pgd);
- return 0;
-}
-
-static int relocate_restore_code(void)
-{
- pgd_t *pgd;
- p4d_t *p4d;
- pud_t *pud;
- pmd_t *pmd;
- pte_t *pte;
-
- relocated_restore_code = get_safe_page(GFP_ATOMIC);
- if (!relocated_restore_code)
- return -ENOMEM;
-
- memcpy((void *)relocated_restore_code, core_restore_code, PAGE_SIZE);
-
- /* Make the page containing the relocated code executable */
- pgd = (pgd_t *)__va(read_cr3_pa()) +
- pgd_index(relocated_restore_code);
- p4d = p4d_offset(pgd, relocated_restore_code);
- if (p4d_large(*p4d)) {
- set_p4d(p4d, __p4d(p4d_val(*p4d) & ~_PAGE_NX));
- goto out;
- }
- pud = pud_offset(p4d, relocated_restore_code);
- if (pud_large(*pud)) {
- set_pud(pud, __pud(pud_val(*pud) & ~_PAGE_NX));
- goto out;
- }
- pmd = pmd_offset(pud, relocated_restore_code);
- if (pmd_large(*pmd)) {
- set_pmd(pmd, __pmd(pmd_val(*pmd) & ~_PAGE_NX));
- goto out;
- }
- pte = pte_offset_kernel(pmd, relocated_restore_code);
- set_pte(pte, __pte(pte_val(*pte) & ~_PAGE_NX));
-out:
- __flush_tlb_all();
+ temp_pgt = __pa(pgd);
return 0;
}
restore_image();
return 0;
}
-
-/*
- * pfn_is_nosave - check if given pfn is in the 'nosave' section
- */
-
-int pfn_is_nosave(unsigned long pfn)
-{
- unsigned long nosave_begin_pfn = __pa_symbol(&__nosave_begin) >> PAGE_SHIFT;
- unsigned long nosave_end_pfn = PAGE_ALIGN(__pa_symbol(&__nosave_end)) >> PAGE_SHIFT;
- return (pfn >= nosave_begin_pfn) && (pfn < nosave_end_pfn);
-}
-
-#define MD5_DIGEST_SIZE 16
-
-struct restore_data_record {
- unsigned long jump_address;
- unsigned long jump_address_phys;
- unsigned long cr3;
- unsigned long magic;
- u8 e820_digest[MD5_DIGEST_SIZE];
-};
-
-#define RESTORE_MAGIC 0x23456789ABCDEF01UL
-
-#if IS_BUILTIN(CONFIG_CRYPTO_MD5)
-/**
- * get_e820_md5 - calculate md5 according to given e820 table
- *
- * @table: the e820 table to be calculated
- * @buf: the md5 result to be stored to
- */
-static int get_e820_md5(struct e820_table *table, void *buf)
-{
- struct crypto_shash *tfm;
- struct shash_desc *desc;
- int size;
- int ret = 0;
-
- tfm = crypto_alloc_shash("md5", 0, 0);
- if (IS_ERR(tfm))
- return -ENOMEM;
-
- desc = kmalloc(sizeof(struct shash_desc) + crypto_shash_descsize(tfm),
- GFP_KERNEL);
- if (!desc) {
- ret = -ENOMEM;
- goto free_tfm;
- }
-
- desc->tfm = tfm;
- desc->flags = 0;
-
- size = offsetof(struct e820_table, entries) +
- sizeof(struct e820_entry) * table->nr_entries;
-
- if (crypto_shash_digest(desc, (u8 *)table, size, buf))
- ret = -EINVAL;
-
- kzfree(desc);
-
-free_tfm:
- crypto_free_shash(tfm);
- return ret;
-}
-
-static void hibernation_e820_save(void *buf)
-{
- get_e820_md5(e820_table_firmware, buf);
-}
-
-static bool hibernation_e820_mismatch(void *buf)
-{
- int ret;
- u8 result[MD5_DIGEST_SIZE];
-
- memset(result, 0, MD5_DIGEST_SIZE);
- /* If there is no digest in suspend kernel, let it go. */
- if (!memcmp(result, buf, MD5_DIGEST_SIZE))
- return false;
-
- ret = get_e820_md5(e820_table_firmware, result);
- if (ret)
- return true;
-
- return memcmp(result, buf, MD5_DIGEST_SIZE) ? true : false;
-}
-#else
-static void hibernation_e820_save(void *buf)
-{
-}
-
-static bool hibernation_e820_mismatch(void *buf)
-{
- /* If md5 is not builtin for restore kernel, let it go. */
- return false;
-}
-#endif
-
-/**
- * arch_hibernation_header_save - populate the architecture specific part
- * of a hibernation image header
- * @addr: address to save the data at
- */
-int arch_hibernation_header_save(void *addr, unsigned int max_size)
-{
- struct restore_data_record *rdr = addr;
-
- if (max_size < sizeof(struct restore_data_record))
- return -EOVERFLOW;
- rdr->jump_address = (unsigned long)restore_registers;
- rdr->jump_address_phys = __pa_symbol(restore_registers);
-
- /*
- * The restore code fixes up CR3 and CR4 in the following sequence:
- *
- * [in hibernation asm]
- * 1. CR3 <= temporary page tables
- * 2. CR4 <= mmu_cr4_features (from the kernel that restores us)
- * 3. CR3 <= rdr->cr3
- * 4. CR4 <= mmu_cr4_features (from us, i.e. the image kernel)
- * [in restore_processor_state()]
- * 5. CR4 <= saved CR4
- * 6. CR3 <= saved CR3
- *
- * Our mmu_cr4_features has CR4.PCIDE=0, and toggling
- * CR4.PCIDE while CR3's PCID bits are nonzero is illegal, so
- * rdr->cr3 needs to point to valid page tables but must not
- * have any of the PCID bits set.
- */
- rdr->cr3 = restore_cr3 & ~CR3_PCID_MASK;
-
- rdr->magic = RESTORE_MAGIC;
-
- hibernation_e820_save(rdr->e820_digest);
-
- return 0;
-}
-
-/**
- * arch_hibernation_header_restore - read the architecture specific data
- * from the hibernation image header
- * @addr: address to read the data from
- */
-int arch_hibernation_header_restore(void *addr)
-{
- struct restore_data_record *rdr = addr;
-
- restore_jump_address = rdr->jump_address;
- jump_address_phys = rdr->jump_address_phys;
- restore_cr3 = rdr->cr3;
-
- if (rdr->magic != RESTORE_MAGIC) {
- pr_crit("Unrecognized hibernate image header format!\n");
- return -EINVAL;
- }
-
- if (hibernation_e820_mismatch(rdr->e820_digest)) {
- pr_crit("Hibernate inconsistent memory map detected!\n");
- return -ENODEV;
- }
-
- return 0;
-}
#include <asm/page_types.h>
#include <asm/asm-offsets.h>
#include <asm/processor-flags.h>
+#include <asm/frame.h>
.text
pushfl
popl saved_context_eflags
+ /* save cr3 */
+ movl %cr3, %eax
+ movl %eax, restore_cr3
+
+ FRAME_BEGIN
call swsusp_save
+ FRAME_END
ret
+ENDPROC(swsusp_arch_suspend)
ENTRY(restore_image)
+ /* prepare to jump to the image kernel */
+ movl restore_jump_address, %ebx
+ movl restore_cr3, %ebp
+
movl mmu_cr4_features, %ecx
- movl resume_pg_dir, %eax
- subl $__PAGE_OFFSET, %eax
+
+ /* jump to relocated restore code */
+ movl relocated_restore_code, %eax
+ jmpl *%eax
+
+/* code below has been relocated to a safe page */
+ENTRY(core_restore_code)
+ movl temp_pgt, %eax
movl %eax, %cr3
jecxz 1f # cr4 Pentium and higher, skip if zero
movl pbe_address(%edx), %esi
movl pbe_orig_address(%edx), %edi
- movl $1024, %ecx
+ movl $(PAGE_SIZE >> 2), %ecx
rep
movsl
.p2align 4,,7
done:
+ jmpl *%ebx
+
+ /* code below belongs to the image kernel */
+ .align PAGE_SIZE
+ENTRY(restore_registers)
/* go back to the original page tables */
- movl $swapper_pg_dir, %eax
- subl $__PAGE_OFFSET, %eax
- movl %eax, %cr3
+ movl %ebp, %cr3
movl mmu_cr4_features, %ecx
jecxz 1f # cr4 Pentium and higher, skip if zero
movl %ecx, %cr4; # turn PGE back on
xorl %eax, %eax
+ /* tell the hibernation core that we've just restored the memory */
+ movl %eax, in_suspend
+
ret
+ENDPROC(restore_registers)
movq restore_cr3(%rip), %r9
/* prepare to switch to temporary page tables */
- movq temp_level4_pgt(%rip), %rax
+ movq temp_pgt(%rip), %rax
movq mmu_cr4_features(%rip), %rbx
/* prepare to copy image data to their original locations */
#if ELF_BITS == 64
REL_TYPE(R_X86_64_NONE),
REL_TYPE(R_X86_64_64),
+ REL_TYPE(R_X86_64_PC64),
REL_TYPE(R_X86_64_PC32),
REL_TYPE(R_X86_64_GOT32),
REL_TYPE(R_X86_64_PLT32),
add_reloc(&relocs32neg, offset);
break;
+ case R_X86_64_PC64:
+ /*
+ * Only used by jump labels
+ */
+ if (is_percpu_sym(sym, symname))
+ die("Invalid R_X86_64_PC64 relocation against per-CPU symbol %s\n",
+ symname);
+ break;
+
case R_X86_64_32:
case R_X86_64_32S:
case R_X86_64_64:
#define R_X86_64_PC16 13 /* 16 bit sign extended pc relative */
#define R_X86_64_8 14 /* Direct 8 bit sign extended */
#define R_X86_64_PC8 15 /* 8 bit sign extended pc relative */
-
-#define R_X86_64_NUM 16
+#define R_X86_64_PC64 24 /* Place relative 64-bit signed */
/*
* This is used to ensure we don't load something for the wrong architecture.
config XEN_DEBUG_FS
bool "Enable Xen debug and tuning parameters in debugfs"
depends on XEN && DEBUG_FS
- default n
help
Enable statistics output and various tuning options in debugfs.
Enabling this option may incur a significant performance overhead.
#include <linux/kexec.h>
#include <linux/slab.h>
+#include <xen/xen.h>
#include <xen/features.h>
#include <xen/page.h>
#include <xen/interface/memory.h>
#include <asm/xen/interface.h>
#include <asm/xen/hypercall.h>
+#include <xen/xen.h>
#include <xen/interface/memory.h>
#include <xen/interface/hvm/start_info.h>
#include <linux/io.h>
#include <linux/export.h>
+#include <xen/xen.h>
#include <xen/platform_pci.h>
#include "xen-ops.h"
#include <linux/interrupt.h>
#include <asm/xen/hypercall.h>
+#include <xen/xen.h>
#include <xen/page.h>
#include <xen/interface/xen.h>
#include <xen/interface/vcpu.h>
k7_counters_mirrored = 0;
break;
}
+ } else if (boot_cpu_data.x86_vendor == X86_VENDOR_HYGON) {
+ amd_num_counters = F10H_NUM_COUNTERS;
+ amd_counters_base = MSR_K7_PERFCTR0;
+ amd_ctrls_base = MSR_K7_EVNTSEL0;
+ amd_msr_step = 1;
+ k7_counters_mirrored = 0;
} else {
uint32_t eax, ebx, ecx, edx;
bool pmu_msr_read(unsigned int msr, uint64_t *val, int *err)
{
- if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD) {
+ if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) {
if (is_amd_pmu_msr(msr)) {
if (!xen_amd_pmu_emulate(msr, val, 1))
*val = native_read_msr_safe(msr, err);
{
uint64_t val = ((uint64_t)high << 32) | low;
- if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD) {
+ if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) {
if (is_amd_pmu_msr(msr)) {
if (!xen_amd_pmu_emulate(msr, &val, 0))
*err = native_write_msr_safe(msr, low, high);
unsigned long long xen_read_pmc(int counter)
{
- if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD)
+ if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL)
return xen_amd_read_pmc(counter);
else
return xen_intel_read_pmc(counter);
select BUILDTIME_EXTABLE_SORT
select CLONE_BACKWARDS
select COMMON_CLK
- select DMA_NONCOHERENT_OPS
+ select DMA_DIRECT_OPS
select GENERIC_ATOMIC64
select GENERIC_CLOCKEVENTS
select GENERIC_IRQ_SHOW
-e 's/\.{text}/.text/g'
quiet_cmd__cpp_lds_S = LDS $@
-cmd__cpp_lds_S = $(CPP) $(cpp_flags) -P -C -Uxtensa -D__ASSEMBLY__ $< \
- | sed $(sed-y) >$@
+cmd__cpp_lds_S = $(CPP) $(cpp_flags) -P -C -Uxtensa -D__ASSEMBLY__ \
+ -DLINKER_SCRIPT $< | sed $(sed-y) >$@
$(obj)/vmlinux.lds: $(src)/vmlinux.lds.S FORCE
$(call if_changed_dep,_cpp_lds_S)
config BLK_DEV_BSGLIB
bool "Block layer SG support v4 helper lib"
- default n
select BLK_DEV_BSG
select BLK_SCSI_REQUEST
help
config BLK_DEV_THROTTLING
bool "Block layer bio throttling support"
depends on BLK_CGROUP=y
- default n
---help---
Block layer bio throttling support. It can be used to limit
the IO rate to a device. IO rate policies are per cgroup and
config BLK_DEV_THROTTLING_LOW
bool "Block throttling .low limit interface support (EXPERIMENTAL)"
depends on BLK_DEV_THROTTLING
- default n
---help---
Add .low limit interface for block throttling. The low limit is a best
effort limit to prioritize cgroups. Depending on the setting, the limit
config BLK_CMDLINE_PARSER
bool "Block device command line partition parser"
- default n
---help---
Enabling this option allows you to specify the partition layout from
the kernel boot args. This is typically of use for embedded devices
config BLK_WBT
bool "Enable support for block device writeback throttling"
- default n
---help---
Enabling this option enables the block layer to throttle buffered
background writeback from the VM, making it more smooth and having
config BLK_CGROUP_IOLATENCY
bool "Enable support for latency based cgroup IO protection"
depends on BLK_CGROUP=y
- default n
---help---
Enabling this option enables the .latency interface for IO throttling.
The IO controller will attempt to maintain average IO latencies below
config BLK_WBT_SQ
bool "Single queue writeback throttling"
- default n
depends on BLK_WBT
---help---
Enable writeback throttling by default on legacy single queue devices
depends on BLOCK && INFINIBAND
default y
+config BLK_PM
+ def_bool BLOCK && PM
+
source block/Kconfig.iosched
config CFQ_GROUP_IOSCHED
bool "CFQ Group Scheduling support"
depends on IOSCHED_CFQ && BLK_CGROUP
- default n
---help---
Enable group IO scheduling in CFQ.
config IOSCHED_BFQ
tristate "BFQ I/O scheduler"
- default n
---help---
BFQ I/O scheduler for BLK-MQ. BFQ distributes the bandwidth of
of the device among all processes according to their weights,
config BFQ_GROUP_IOSCHED
bool "BFQ hierarchical scheduling support"
depends on IOSCHED_BFQ && BLK_CGROUP
- default n
---help---
Enable hierarchical scheduling in BFQ, using the blkio
obj-$(CONFIG_BLK_DEBUG_FS) += blk-mq-debugfs.o
obj-$(CONFIG_BLK_DEBUG_FS_ZONED)+= blk-mq-debugfs-zoned.o
obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o
+obj-$(CONFIG_BLK_PM) += blk-pm.o
uint64_t serial_nr;
rcu_read_lock();
- serial_nr = bio_blkcg(bio)->css.serial_nr;
+ serial_nr = __bio_blkcg(bio)->css.serial_nr;
/*
* Check whether blkcg has changed. The condition may trigger
if (unlikely(!bfqd) || likely(bic->blkcg_serial_nr == serial_nr))
goto out;
- bfqg = __bfq_bic_change_cgroup(bfqd, bic, bio_blkcg(bio));
+ bfqg = __bfq_bic_change_cgroup(bfqd, bic, __bio_blkcg(bio));
/*
* Update blkg_path for bfq_log_* functions. We cache this
* path, and update it here, for the following
}
/*
- * Tell whether there are active queues or groups with differentiated weights.
+ * Tell whether there are active queues with different weights or
+ * active groups.
*/
-static bool bfq_differentiated_weights(struct bfq_data *bfqd)
+static bool bfq_varied_queue_weights_or_active_groups(struct bfq_data *bfqd)
{
/*
- * For weights to differ, at least one of the trees must contain
+ * For queue weights to differ, queue_weights_tree must contain
* at least two nodes.
*/
return (!RB_EMPTY_ROOT(&bfqd->queue_weights_tree) &&
bfqd->queue_weights_tree.rb_node->rb_right)
#ifdef CONFIG_BFQ_GROUP_IOSCHED
) ||
- (!RB_EMPTY_ROOT(&bfqd->group_weights_tree) &&
- (bfqd->group_weights_tree.rb_node->rb_left ||
- bfqd->group_weights_tree.rb_node->rb_right)
+ (bfqd->num_active_groups > 0
#endif
);
}
* 3) all active groups at the same level in the groups tree have the same
* number of children.
*
- * Unfortunately, keeping the necessary state for evaluating exactly the
- * above symmetry conditions would be quite complex and time-consuming.
- * Therefore this function evaluates, instead, the following stronger
- * sub-conditions, for which it is much easier to maintain the needed
- * state:
+ * Unfortunately, keeping the necessary state for evaluating exactly
+ * the last two symmetry sub-conditions above would be quite complex
+ * and time consuming. Therefore this function evaluates, instead,
+ * only the following stronger two sub-conditions, for which it is
+ * much easier to maintain the needed state:
* 1) all active queues have the same weight,
- * 2) all active groups have the same weight,
- * 3) all active groups have at most one active child each.
- * In particular, the last two conditions are always true if hierarchical
- * support and the cgroups interface are not enabled, thus no state needs
- * to be maintained in this case.
+ * 2) there are no active groups.
+ * In particular, the last condition is always true if hierarchical
+ * support or the cgroups interface are not enabled, thus no state
+ * needs to be maintained in this case.
*/
static bool bfq_symmetric_scenario(struct bfq_data *bfqd)
{
- return !bfq_differentiated_weights(bfqd);
+ return !bfq_varied_queue_weights_or_active_groups(bfqd);
}
/*
* If the weight-counter tree passed as input contains no counter for
- * the weight of the input entity, then add that counter; otherwise just
+ * the weight of the input queue, then add that counter; otherwise just
* increment the existing counter.
*
* Note that weight-counter trees contain few nodes in mostly symmetric
* In most scenarios, the rate at which nodes are created/destroyed
* should be low too.
*/
-void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_entity *entity,
+void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_queue *bfqq,
struct rb_root *root)
{
+ struct bfq_entity *entity = &bfqq->entity;
struct rb_node **new = &(root->rb_node), *parent = NULL;
/*
- * Do not insert if the entity is already associated with a
+ * Do not insert if the queue is already associated with a
* counter, which happens if:
- * 1) the entity is associated with a queue,
- * 2) a request arrival has caused the queue to become both
+ * 1) a request arrival has caused the queue to become both
* non-weight-raised, and hence change its weight, and
* backlogged; in this respect, each of the two events
* causes an invocation of this function,
- * 3) this is the invocation of this function caused by the
+ * 2) this is the invocation of this function caused by the
* second event. This second invocation is actually useless,
* and we handle this fact by exiting immediately. More
* efficient or clearer solutions might possibly be adopted.
*/
- if (entity->weight_counter)
+ if (bfqq->weight_counter)
return;
while (*new) {
parent = *new;
if (entity->weight == __counter->weight) {
- entity->weight_counter = __counter;
+ bfqq->weight_counter = __counter;
goto inc_counter;
}
if (entity->weight < __counter->weight)
new = &((*new)->rb_right);
}
- entity->weight_counter = kzalloc(sizeof(struct bfq_weight_counter),
- GFP_ATOMIC);
+ bfqq->weight_counter = kzalloc(sizeof(struct bfq_weight_counter),
+ GFP_ATOMIC);
/*
* In the unlucky event of an allocation failure, we just
- * exit. This will cause the weight of entity to not be
- * considered in bfq_differentiated_weights, which, in its
- * turn, causes the scenario to be deemed wrongly symmetric in
- * case entity's weight would have been the only weight making
- * the scenario asymmetric. On the bright side, no unbalance
- * will however occur when entity becomes inactive again (the
- * invocation of this function is triggered by an activation
- * of entity). In fact, bfq_weights_tree_remove does nothing
- * if !entity->weight_counter.
+ * exit. This will cause the weight of queue to not be
+ * considered in bfq_varied_queue_weights_or_active_groups,
+ * which, in its turn, causes the scenario to be deemed
+ * wrongly symmetric in case bfqq's weight would have been
+ * the only weight making the scenario asymmetric. On the
+ * bright side, no unbalance will however occur when bfqq
+ * becomes inactive again (the invocation of this function
+ * is triggered by an activation of queue). In fact,
+ * bfq_weights_tree_remove does nothing if
+ * !bfqq->weight_counter.
*/
- if (unlikely(!entity->weight_counter))
+ if (unlikely(!bfqq->weight_counter))
return;
- entity->weight_counter->weight = entity->weight;
- rb_link_node(&entity->weight_counter->weights_node, parent, new);
- rb_insert_color(&entity->weight_counter->weights_node, root);
+ bfqq->weight_counter->weight = entity->weight;
+ rb_link_node(&bfqq->weight_counter->weights_node, parent, new);
+ rb_insert_color(&bfqq->weight_counter->weights_node, root);
inc_counter:
- entity->weight_counter->num_active++;
+ bfqq->weight_counter->num_active++;
}
/*
- * Decrement the weight counter associated with the entity, and, if the
+ * Decrement the weight counter associated with the queue, and, if the
* counter reaches 0, remove the counter from the tree.
* See the comments to the function bfq_weights_tree_add() for considerations
* about overhead.
*/
void __bfq_weights_tree_remove(struct bfq_data *bfqd,
- struct bfq_entity *entity,
+ struct bfq_queue *bfqq,
struct rb_root *root)
{
- if (!entity->weight_counter)
+ if (!bfqq->weight_counter)
return;
- entity->weight_counter->num_active--;
- if (entity->weight_counter->num_active > 0)
+ bfqq->weight_counter->num_active--;
+ if (bfqq->weight_counter->num_active > 0)
goto reset_entity_pointer;
- rb_erase(&entity->weight_counter->weights_node, root);
- kfree(entity->weight_counter);
+ rb_erase(&bfqq->weight_counter->weights_node, root);
+ kfree(bfqq->weight_counter);
reset_entity_pointer:
- entity->weight_counter = NULL;
+ bfqq->weight_counter = NULL;
}
/*
- * Invoke __bfq_weights_tree_remove on bfqq and all its inactive
- * parent entities.
+ * Invoke __bfq_weights_tree_remove on bfqq and decrement the number
+ * of active groups for each queue's inactive parent entity.
*/
void bfq_weights_tree_remove(struct bfq_data *bfqd,
struct bfq_queue *bfqq)
{
struct bfq_entity *entity = bfqq->entity.parent;
- __bfq_weights_tree_remove(bfqd, &bfqq->entity,
+ __bfq_weights_tree_remove(bfqd, bfqq,
&bfqd->queue_weights_tree);
for_each_entity(entity) {
* next_in_service for details on why
* in_service_entity must be checked too).
*
- * As a consequence, the weight of entity is
- * not to be removed. In addition, if entity
- * is active, then its parent entities are
- * active as well, and thus their weights are
- * not to be removed either. In the end, this
- * loop must stop here.
+ * As a consequence, its parent entities are
+ * active as well, and thus this loop must
+ * stop here.
*/
break;
}
- __bfq_weights_tree_remove(bfqd, entity,
- &bfqd->group_weights_tree);
+ bfqd->num_active_groups--;
}
}
jiffies + nsecs_to_jiffies(bfqq->bfqd->bfq_slice_idle) + 4);
}
+static bool bfq_bfqq_injectable(struct bfq_queue *bfqq)
+{
+ return BFQQ_SEEKY(bfqq) && bfqq->wr_coeff == 1 &&
+ blk_queue_nonrot(bfqq->bfqd->queue) &&
+ bfqq->bfqd->hw_tag;
+}
+
/**
* bfq_bfqq_expire - expire a queue.
* @bfqd: device owning the queue.
if (ref == 1) /* bfqq is gone, no more actions on it */
return;
+ bfqq->injected_service = 0;
+
/* mark bfqq as waiting a request only if a bic still points to it */
if (!bfq_bfqq_busy(bfqq) &&
reason != BFQQE_BUDGET_TIMEOUT &&
* symmetric scenario where:
* (i) each of these processes must get the same throughput as
* the others;
- * (ii) all these processes have the same I/O pattern
- (either sequential or random).
- * In fact, in such a scenario, the drive will tend to treat
+ * (ii) the I/O of each process has the same properties, in
+ * terms of locality (sequential or random), direction
+ * (reads or writes), request sizes, greediness
+ * (from I/O-bound to sporadic), and so on.
+ * In fact, in such a scenario, the drive tends to treat
* the requests of each of these processes in about the same
* way as the requests of the others, and thus to provide
* each of these processes with about the same throughput
* certainly needed to guarantee that bfqq receives its
* assigned fraction of the device throughput (see [1] for
* details).
+ * The problem is that idling may significantly reduce
+ * throughput with certain combinations of types of I/O and
+ * devices. An important example is sync random I/O, on flash
+ * storage with command queueing. So, unless bfqq falls in the
+ * above cases where idling also boosts throughput, it would
+ * be important to check conditions (i) and (ii) accurately,
+ * so as to avoid idling when not strictly needed for service
+ * guarantees.
+ *
+ * Unfortunately, it is extremely difficult to thoroughly
+ * check condition (ii). And, in case there are active groups,
+ * it becomes very difficult to check condition (i) too. In
+ * fact, if there are active groups, then, for condition (i)
+ * to become false, it is enough that an active group contains
+ * more active processes or sub-groups than some other active
+ * group. We address this issue with the following bi-modal
+ * behavior, implemented in the function
+ * bfq_symmetric_scenario().
*
- * We address this issue by controlling, actually, only the
- * symmetry sub-condition (i), i.e., provided that
- * sub-condition (i) holds, idling is not performed,
- * regardless of whether sub-condition (ii) holds. In other
- * words, only if sub-condition (i) holds, then idling is
+ * If there are active groups, then the scenario is tagged as
+ * asymmetric, conservatively, without checking any of the
+ * conditions (i) and (ii). So the device is idled for bfqq.
+ * This behavior matches also the fact that groups are created
+ * exactly if controlling I/O (to preserve bandwidth and
+ * latency guarantees) is a primary concern.
+ *
+ * On the opposite end, if there are no active groups, then
+ * only condition (i) is actually controlled, i.e., provided
+ * that condition (i) holds, idling is not performed,
+ * regardless of whether condition (ii) holds. In other words,
+ * only if condition (i) does not hold, then idling is
* allowed, and the device tends to be prevented from queueing
- * many requests, possibly of several processes. The reason
- * for not controlling also sub-condition (ii) is that we
- * exploit preemption to preserve guarantees in case of
- * symmetric scenarios, even if (ii) does not hold, as
- * explained in the next two paragraphs.
+ * many requests, possibly of several processes. Since there
+ * are no active groups, then, to control condition (i) it is
+ * enough to check whether all active queues have the same
+ * weight.
+ *
+ * Not checking condition (ii) evidently exposes bfqq to the
+ * risk of getting less throughput than its fair share.
+ * However, for queues with the same weight, a further
+ * mechanism, preemption, mitigates or even eliminates this
+ * problem. And it does so without consequences on overall
+ * throughput. This mechanism and its benefits are explained
+ * in the next three paragraphs.
*
* Even if a queue, say Q, is expired when it remains idle, Q
* can still preempt the new in-service queue if the next
* idling allows the internal queues of the device to contain
* many requests, and thus to reorder requests, we can rather
* safely assume that the internal scheduler still preserves a
- * minimum of mid-term fairness. The motivation for using
- * preemption instead of idling is that, by not idling,
- * service guarantees are preserved without minimally
- * sacrificing throughput. In other words, both a high
- * throughput and its desired distribution are obtained.
+ * minimum of mid-term fairness.
*
* More precisely, this preemption-based, idleless approach
* provides fairness in terms of IOPS, and not sectors per
* 1024/8 times as high as the service received by the other
* queue.
*
- * On the other hand, device idling is performed, and thus
- * pure sector-domain guarantees are provided, for the
- * following queues, which are likely to need stronger
- * throughput guarantees: weight-raised queues, and queues
- * with a higher weight than other queues. When such queues
- * are active, sub-condition (i) is false, which triggers
- * device idling.
+ * The motivation for using preemption instead of idling (for
+ * queues with the same weight) is that, by not idling,
+ * service guarantees are preserved (completely or at least in
+ * part) without minimally sacrificing throughput. And, if
+ * there is no active group, then the primary expectation for
+ * this device is probably a high throughput.
*
- * According to the above considerations, the next variable is
- * true (only) if sub-condition (i) holds. To compute the
- * value of this variable, we not only use the return value of
- * the function bfq_symmetric_scenario(), but also check
- * whether bfqq is being weight-raised, because
- * bfq_symmetric_scenario() does not take into account also
- * weight-raised queues (see comments on
- * bfq_weights_tree_add()).
+ * We are now left only with explaining the additional
+ * compound condition that is checked below for deciding
+ * whether the scenario is asymmetric. To explain this
+ * compound condition, we need to add that the function
+ * bfq_symmetric_scenario checks the weights of only
+ * non-weight-raised queues, for efficiency reasons (see
+ * comments on bfq_weights_tree_add()). Then the fact that
+ * bfqq is weight-raised is checked explicitly here. More
+ * precisely, the compound condition below takes into account
+ * also the fact that, even if bfqq is being weight-raised,
+ * the scenario is still symmetric if all active queues happen
+ * to be weight-raised. Actually, we should be even more
+ * precise here, and differentiate between interactive weight
+ * raising and soft real-time weight raising.
*
* As a side note, it is worth considering that the above
* device-idling countermeasures may however fail in the
* to let requests be served in the desired order until all
* the requests already queued in the device have been served.
*/
- asymmetric_scenario = bfqq->wr_coeff > 1 ||
+ asymmetric_scenario = (bfqq->wr_coeff > 1 &&
+ bfqd->wr_busy_queues < bfqd->busy_queues) ||
!bfq_symmetric_scenario(bfqd);
/*
return RB_EMPTY_ROOT(&bfqq->sort_list) && bfq_better_to_idle(bfqq);
}
+static struct bfq_queue *bfq_choose_bfqq_for_injection(struct bfq_data *bfqd)
+{
+ struct bfq_queue *bfqq;
+
+ /*
+ * A linear search; but, with a high probability, very few
+ * steps are needed to find a candidate queue, i.e., a queue
+ * with enough budget left for its next request. In fact:
+ * - BFQ dynamically updates the budget of every queue so as
+ * to accommodate the expected backlog of the queue;
+ * - if a queue gets all its requests dispatched as injected
+ * service, then the queue is removed from the active list
+ * (and re-added only if it gets new requests, but with
+ * enough budget for its new backlog).
+ */
+ list_for_each_entry(bfqq, &bfqd->active_list, bfqq_list)
+ if (!RB_EMPTY_ROOT(&bfqq->sort_list) &&
+ bfq_serv_to_charge(bfqq->next_rq, bfqq) <=
+ bfq_bfqq_budget_left(bfqq))
+ return bfqq;
+
+ return NULL;
+}
+
/*
* Select a queue for service. If we have a current queue in service,
* check whether to continue servicing it, or retrieve and set a new one.
* No requests pending. However, if the in-service queue is idling
* for a new request, or has requests waiting for a completion and
* may idle after their completion, then keep it anyway.
+ *
+ * Yet, to boost throughput, inject service from other queues if
+ * possible.
*/
if (bfq_bfqq_wait_request(bfqq) ||
(bfqq->dispatched != 0 && bfq_better_to_idle(bfqq))) {
- bfqq = NULL;
+ if (bfq_bfqq_injectable(bfqq) &&
+ bfqq->injected_service * bfqq->inject_coeff <
+ bfqq->entity.service * 10)
+ bfqq = bfq_choose_bfqq_for_injection(bfqd);
+ else
+ bfqq = NULL;
+
goto keep_queue;
}
bfq_dispatch_remove(bfqd->queue, rq);
+ if (bfqq != bfqd->in_service_queue) {
+ if (likely(bfqd->in_service_queue))
+ bfqd->in_service_queue->injected_service +=
+ bfq_serv_to_charge(rq, bfqq);
+
+ goto return_rq;
+ }
+
/*
* If weight raising has to terminate for bfqq, then next
* function causes an immediate update of bfqq's weight,
* belongs to CLASS_IDLE and other queues are waiting for
* service.
*/
- if (bfqd->busy_queues > 1 && bfq_class_idle(bfqq))
- goto expire;
-
- return rq;
+ if (!(bfqd->busy_queues > 1 && bfq_class_idle(bfqq)))
+ goto return_rq;
-expire:
bfq_bfqq_expire(bfqd, bfqq, false, BFQQE_BUDGET_EXHAUSTED);
+
+return_rq:
return rq;
}
bfq_mark_bfqq_has_short_ttime(bfqq);
bfq_mark_bfqq_sync(bfqq);
bfq_mark_bfqq_just_created(bfqq);
+ /*
+ * Aggressively inject a lot of service: up to 90%.
+ * This coefficient remains constant during bfqq life,
+ * but this behavior might be changed, after enough
+ * testing and tuning.
+ */
+ bfqq->inject_coeff = 1;
} else
bfq_clear_bfqq_sync(bfqq);
rcu_read_lock();
- bfqg = bfq_find_set_group(bfqd, bio_blkcg(bio));
+ bfqg = bfq_find_set_group(bfqd, __bio_blkcg(bio));
if (!bfqg) {
bfqq = &bfqd->oom_bfqq;
goto out;
bfqd->idle_slice_timer.function = bfq_idle_slice_timer;
bfqd->queue_weights_tree = RB_ROOT;
- bfqd->group_weights_tree = RB_ROOT;
+ bfqd->num_active_groups = 0;
INIT_LIST_HEAD(&bfqd->active_list);
INIT_LIST_HEAD(&bfqd->idle_list);
};
/**
- * struct bfq_weight_counter - counter of the number of all active entities
+ * struct bfq_weight_counter - counter of the number of all active queues
* with a given weight.
*/
struct bfq_weight_counter {
- unsigned int weight; /* weight of the entities this counter refers to */
- unsigned int num_active; /* nr of active entities with this weight */
+ unsigned int weight; /* weight of the queues this counter refers to */
+ unsigned int num_active; /* nr of active queues with this weight */
/*
- * Weights tree member (see bfq_data's @queue_weights_tree and
- * @group_weights_tree)
+ * Weights tree member (see bfq_data's @queue_weights_tree)
*/
struct rb_node weights_node;
};
struct bfq_entity {
/* service_tree member */
struct rb_node rb_node;
- /* pointer to the weight counter associated with this entity */
- struct bfq_weight_counter *weight_counter;
/*
* Flag, true if the entity is on a tree (either the active or
/* entity representing this queue in the scheduler */
struct bfq_entity entity;
+ /* pointer to the weight counter associated with this entity */
+ struct bfq_weight_counter *weight_counter;
+
/* maximum budget allowed from the feedback mechanism */
int max_budget;
/* budget expiration (in jiffies) */
unsigned long split_time; /* time of last split */
unsigned long first_IO_time; /* time of first I/O for this queue */
+
+ /* max service rate measured so far */
+ u32 max_service_rate;
+ /*
+ * Ratio between the service received by bfqq while it is in
+ * service, and the cumulative service (of requests of other
+ * queues) that may be injected while bfqq is empty but still
+ * in service. To increase precision, the coefficient is
+ * measured in tenths of unit. Here are some example of (1)
+ * ratios, (2) resulting percentages of service injected
+ * w.r.t. to the total service dispatched while bfqq is in
+ * service, and (3) corresponding values of the coefficient:
+ * 1 (50%) -> 10
+ * 2 (33%) -> 20
+ * 10 (9%) -> 100
+ * 9.9 (9%) -> 99
+ * 1.5 (40%) -> 15
+ * 0.5 (66%) -> 5
+ * 0.1 (90%) -> 1
+ *
+ * So, if the coefficient is lower than 10, then
+ * injected service is more than bfqq service.
+ */
+ unsigned int inject_coeff;
+ /* amount of service injected in current service slot */
+ unsigned int injected_service;
};
/**
*/
struct rb_root queue_weights_tree;
/*
- * rbtree of non-queue @bfq_entity weight counters, sorted by
- * weight. Used to keep track of whether all @bfq_groups have
- * the same weight. The tree contains one counter for each
- * distinct weight associated to some active @bfq_group (see
- * the comments to the functions bfq_weights_tree_[add|remove]
- * for further details).
+ * number of groups with requests still waiting for completion
*/
- struct rb_root group_weights_tree;
+ unsigned int num_active_groups;
/*
* Number of bfq_queues containing requests (including the
void bic_set_bfqq(struct bfq_io_cq *bic, struct bfq_queue *bfqq, bool is_sync);
struct bfq_data *bic_to_bfqd(struct bfq_io_cq *bic);
void bfq_pos_tree_add_move(struct bfq_data *bfqd, struct bfq_queue *bfqq);
-void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_entity *entity,
+void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_queue *bfqq,
struct rb_root *root);
void __bfq_weights_tree_remove(struct bfq_data *bfqd,
- struct bfq_entity *entity,
+ struct bfq_queue *bfqq,
struct rb_root *root);
void bfq_weights_tree_remove(struct bfq_data *bfqd,
struct bfq_queue *bfqq);
new_weight = entity->orig_weight *
(bfqq ? bfqq->wr_coeff : 1);
/*
- * If the weight of the entity changes, remove the entity
- * from its old weight counter (if there is a counter
- * associated with the entity), and add it to the counter
- * associated with its new weight.
+ * If the weight of the entity changes, and the entity is a
+ * queue, remove the entity from its old weight counter (if
+ * there is a counter associated with the entity).
*/
if (prev_weight != new_weight) {
- root = bfqq ? &bfqd->queue_weights_tree :
- &bfqd->group_weights_tree;
- __bfq_weights_tree_remove(bfqd, entity, root);
+ if (bfqq) {
+ root = &bfqd->queue_weights_tree;
+ __bfq_weights_tree_remove(bfqd, bfqq, root);
+ } else
+ bfqd->num_active_groups--;
}
entity->weight = new_weight;
/*
- * Add the entity to its weights tree only if it is
- * not associated with a weight-raised queue.
+ * Add the entity, if it is not a weight-raised queue,
+ * to the counter associated with its new weight.
*/
- if (prev_weight != new_weight &&
- (bfqq ? bfqq->wr_coeff == 1 : 1))
- /* If we get here, root has been initialized. */
- bfq_weights_tree_add(bfqd, entity, root);
+ if (prev_weight != new_weight) {
+ if (bfqq && bfqq->wr_coeff == 1) {
+ /* If we get here, root has been initialized. */
+ bfq_weights_tree_add(bfqd, bfqq, root);
+ } else
+ bfqd->num_active_groups++;
+ }
new_st->wsum += entity->weight;
if (!bfq_entity_to_bfqq(entity)) { /* bfq_group */
struct bfq_group *bfqg =
container_of(entity, struct bfq_group, entity);
+ struct bfq_data *bfqd = bfqg->bfqd;
- bfq_weights_tree_add(bfqg->bfqd, entity,
- &bfqd->group_weights_tree);
+ bfqd->num_active_groups++;
}
#endif
st = bfq_entity_service_tree(entity);
is_in_service = entity == sd->in_service_entity;
- if (is_in_service) {
- bfq_calc_finish(entity, entity->service);
+ bfq_calc_finish(entity, entity->service);
+
+ if (is_in_service)
sd->in_service_entity = NULL;
- }
+ else
+ /*
+ * Non in-service entity: nobody will take care of
+ * resetting its service counter on expiration. Do it
+ * now.
+ */
+ entity->service = 0;
if (entity->tree == &st->active)
bfq_active_extract(st, entity);
if (!bfqq->dispatched)
if (bfqq->wr_coeff == 1)
- bfq_weights_tree_add(bfqd, &bfqq->entity,
+ bfq_weights_tree_add(bfqd, bfqq,
&bfqd->queue_weights_tree);
if (bfqq->wr_coeff > 1)
if (bio_data_dir(bio) == WRITE) {
bio_integrity_process(bio, &bio->bi_iter,
bi->profile->generate_fn);
+ } else {
+ bip->bio_iter = bio->bi_iter;
}
return true;
container_of(work, struct bio_integrity_payload, bip_work);
struct bio *bio = bip->bip_bio;
struct blk_integrity *bi = blk_get_integrity(bio->bi_disk);
- struct bvec_iter iter = bio->bi_iter;
/*
* At the moment verify is called bio's iterator was advanced
* during split and completion, we need to rewind iterator to
* it's original position.
*/
- if (bio_rewind_iter(bio, &iter, iter.bi_done)) {
- bio->bi_status = bio_integrity_process(bio, &iter,
- bi->profile->verify_fn);
- } else {
- bio->bi_status = BLK_STS_IOERR;
- }
-
+ bio->bi_status = bio_integrity_process(bio, &bip->bio_iter,
+ bi->profile->verify_fn);
bio_integrity_free(bio);
bio_endio(bio);
}
bio->bi_iter = bio_src->bi_iter;
bio->bi_io_vec = bio_src->bi_io_vec;
- bio_clone_blkcg_association(bio, bio_src);
+ bio_clone_blkg_association(bio, bio_src);
+
+ blkcg_bio_issue_init(bio);
}
EXPORT_SYMBOL(__bio_clone_fast);
}
/* If we may be able to merge these biovecs, force a recount */
- if (bio->bi_vcnt > 1 && (BIOVEC_PHYS_MERGEABLE(bvec-1, bvec)))
+ if (bio->bi_vcnt > 1 && biovec_phys_mergeable(q, bvec - 1, bvec))
bio_clear_flag(bio, BIO_SEG_VALID);
done:
}
EXPORT_SYMBOL(bio_add_page);
+#define PAGE_PTRS_PER_BVEC (sizeof(struct bio_vec) / sizeof(struct page *))
+
/**
* __bio_iov_iter_get_pages - pin user or kernel pages and add them to a bio
* @bio: bio to add pages to
*/
static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)
{
- unsigned short nr_pages = bio->bi_max_vecs - bio->bi_vcnt, idx;
+ unsigned short nr_pages = bio->bi_max_vecs - bio->bi_vcnt;
+ unsigned short entries_left = bio->bi_max_vecs - bio->bi_vcnt;
struct bio_vec *bv = bio->bi_io_vec + bio->bi_vcnt;
struct page **pages = (struct page **)bv;
+ ssize_t size, left;
+ unsigned len, i;
size_t offset;
- ssize_t size;
+
+ /*
+ * Move page array up in the allocated memory for the bio vecs as far as
+ * possible so that we can start filling biovecs from the beginning
+ * without overwriting the temporary page array.
+ */
+ BUILD_BUG_ON(PAGE_PTRS_PER_BVEC < 2);
+ pages += entries_left * (PAGE_PTRS_PER_BVEC - 1);
size = iov_iter_get_pages(iter, pages, LONG_MAX, nr_pages, &offset);
if (unlikely(size <= 0))
return size ? size : -EFAULT;
- idx = nr_pages = (size + offset + PAGE_SIZE - 1) / PAGE_SIZE;
- /*
- * Deep magic below: We need to walk the pinned pages backwards
- * because we are abusing the space allocated for the bio_vecs
- * for the page array. Because the bio_vecs are larger than the
- * page pointers by definition this will always work. But it also
- * means we can't use bio_add_page, so any changes to it's semantics
- * need to be reflected here as well.
- */
- bio->bi_iter.bi_size += size;
- bio->bi_vcnt += nr_pages;
+ for (left = size, i = 0; left > 0; left -= len, i++) {
+ struct page *page = pages[i];
- while (idx--) {
- bv[idx].bv_page = pages[idx];
- bv[idx].bv_len = PAGE_SIZE;
- bv[idx].bv_offset = 0;
+ len = min_t(size_t, PAGE_SIZE - offset, left);
+ if (WARN_ON_ONCE(bio_add_page(bio, page, len, offset) != len))
+ return -EINVAL;
+ offset = 0;
}
- bv[0].bv_offset += offset;
- bv[0].bv_len -= offset;
- bv[nr_pages - 1].bv_len -= nr_pages * PAGE_SIZE - offset - size;
-
iov_iter_advance(iter, size);
return 0;
}
bio_integrity_trim(split);
bio_advance(bio, split->bi_iter.bi_size);
- bio->bi_iter.bi_done = 0;
if (bio_flagged(bio, BIO_TRACE_COMPLETION))
bio_set_flag(split, BIO_TRACE_COMPLETION);
#ifdef CONFIG_BLK_CGROUP
+/**
+ * bio_associate_blkg - associate a bio with the a blkg
+ * @bio: target bio
+ * @blkg: the blkg to associate
+ *
+ * This tries to associate @bio with the specified blkg. Association failure
+ * is handled by walking up the blkg tree. Therefore, the blkg associated can
+ * be anything between @blkg and the root_blkg. This situation only happens
+ * when a cgroup is dying and then the remaining bios will spill to the closest
+ * alive blkg.
+ *
+ * A reference will be taken on the @blkg and will be released when @bio is
+ * freed.
+ */
+int bio_associate_blkg(struct bio *bio, struct blkcg_gq *blkg)
+{
+ if (unlikely(bio->bi_blkg))
+ return -EBUSY;
+ bio->bi_blkg = blkg_tryget_closest(blkg);
+ return 0;
+}
+
+/**
+ * __bio_associate_blkg_from_css - internal blkg association function
+ *
+ * This in the core association function that all association paths rely on.
+ * A blkg reference is taken which is released upon freeing of the bio.
+ */
+static int __bio_associate_blkg_from_css(struct bio *bio,
+ struct cgroup_subsys_state *css)
+{
+ struct request_queue *q = bio->bi_disk->queue;
+ struct blkcg_gq *blkg;
+ int ret;
+
+ rcu_read_lock();
+
+ if (!css || !css->parent)
+ blkg = q->root_blkg;
+ else
+ blkg = blkg_lookup_create(css_to_blkcg(css), q);
+
+ ret = bio_associate_blkg(bio, blkg);
+
+ rcu_read_unlock();
+ return ret;
+}
+
+/**
+ * bio_associate_blkg_from_css - associate a bio with a specified css
+ * @bio: target bio
+ * @css: target css
+ *
+ * Associate @bio with the blkg found by combining the css's blkg and the
+ * request_queue of the @bio. This falls back to the queue's root_blkg if
+ * the association fails with the css.
+ */
+int bio_associate_blkg_from_css(struct bio *bio,
+ struct cgroup_subsys_state *css)
+{
+ if (unlikely(bio->bi_blkg))
+ return -EBUSY;
+ return __bio_associate_blkg_from_css(bio, css);
+}
+EXPORT_SYMBOL_GPL(bio_associate_blkg_from_css);
+
#ifdef CONFIG_MEMCG
/**
- * bio_associate_blkcg_from_page - associate a bio with the page's blkcg
+ * bio_associate_blkg_from_page - associate a bio with the page's blkg
* @bio: target bio
* @page: the page to lookup the blkcg from
*
- * Associate @bio with the blkcg from @page's owning memcg. This works like
- * every other associate function wrt references.
+ * Associate @bio with the blkg from @page's owning memcg and the respective
+ * request_queue. If cgroup_e_css returns NULL, fall back to the queue's
+ * root_blkg.
+ *
+ * Note: this must be called after bio has an associated device.
*/
-int bio_associate_blkcg_from_page(struct bio *bio, struct page *page)
+int bio_associate_blkg_from_page(struct bio *bio, struct page *page)
{
- struct cgroup_subsys_state *blkcg_css;
+ struct cgroup_subsys_state *css;
+ int ret;
- if (unlikely(bio->bi_css))
+ if (unlikely(bio->bi_blkg))
return -EBUSY;
if (!page->mem_cgroup)
return 0;
- blkcg_css = cgroup_get_e_css(page->mem_cgroup->css.cgroup,
- &io_cgrp_subsys);
- bio->bi_css = blkcg_css;
- return 0;
+
+ rcu_read_lock();
+
+ css = cgroup_e_css(page->mem_cgroup->css.cgroup, &io_cgrp_subsys);
+
+ ret = __bio_associate_blkg_from_css(bio, css);
+
+ rcu_read_unlock();
+ return ret;
}
#endif /* CONFIG_MEMCG */
/**
- * bio_associate_blkcg - associate a bio with the specified blkcg
+ * bio_associate_create_blkg - associate a bio with a blkg from q
+ * @q: request_queue where bio is going
* @bio: target bio
- * @blkcg_css: css of the blkcg to associate
- *
- * Associate @bio with the blkcg specified by @blkcg_css. Block layer will
- * treat @bio as if it were issued by a task which belongs to the blkcg.
*
- * This function takes an extra reference of @blkcg_css which will be put
- * when @bio is released. The caller must own @bio and is responsible for
- * synchronizing calls to this function.
+ * Associate @bio with the blkg found from the bio's css and the request_queue.
+ * If one is not found, bio_lookup_blkg creates the blkg. This falls back to
+ * the queue's root_blkg if association fails.
*/
-int bio_associate_blkcg(struct bio *bio, struct cgroup_subsys_state *blkcg_css)
+int bio_associate_create_blkg(struct request_queue *q, struct bio *bio)
{
- if (unlikely(bio->bi_css))
- return -EBUSY;
- css_get(blkcg_css);
- bio->bi_css = blkcg_css;
- return 0;
+ struct cgroup_subsys_state *css;
+ int ret = 0;
+
+ /* someone has already associated this bio with a blkg */
+ if (bio->bi_blkg)
+ return ret;
+
+ rcu_read_lock();
+
+ css = blkcg_css();
+
+ ret = __bio_associate_blkg_from_css(bio, css);
+
+ rcu_read_unlock();
+ return ret;
}
-EXPORT_SYMBOL_GPL(bio_associate_blkcg);
/**
- * bio_associate_blkg - associate a bio with the specified blkg
+ * bio_reassociate_blkg - reassociate a bio with a blkg from q
+ * @q: request_queue where bio is going
* @bio: target bio
- * @blkg: the blkg to associate
*
- * Associate @bio with the blkg specified by @blkg. This is the queue specific
- * blkcg information associated with the @bio, a reference will be taken on the
- * @blkg and will be freed when the bio is freed.
+ * When submitting a bio, multiple recursive calls to make_request() may occur.
+ * This causes the initial associate done in blkcg_bio_issue_check() to be
+ * incorrect and reference the prior request_queue. This performs reassociation
+ * when this situation happens.
*/
-int bio_associate_blkg(struct bio *bio, struct blkcg_gq *blkg)
+int bio_reassociate_blkg(struct request_queue *q, struct bio *bio)
{
- if (unlikely(bio->bi_blkg))
- return -EBUSY;
- if (!blkg_try_get(blkg))
- return -ENODEV;
- bio->bi_blkg = blkg;
- return 0;
+ if (bio->bi_blkg) {
+ blkg_put(bio->bi_blkg);
+ bio->bi_blkg = NULL;
+ }
+
+ return bio_associate_create_blkg(q, bio);
}
/**
put_io_context(bio->bi_ioc);
bio->bi_ioc = NULL;
}
- if (bio->bi_css) {
- css_put(bio->bi_css);
- bio->bi_css = NULL;
- }
if (bio->bi_blkg) {
blkg_put(bio->bi_blkg);
bio->bi_blkg = NULL;
}
/**
- * bio_clone_blkcg_association - clone blkcg association from src to dst bio
+ * bio_clone_blkg_association - clone blkg association from src to dst bio
* @dst: destination bio
* @src: source bio
*/
-void bio_clone_blkcg_association(struct bio *dst, struct bio *src)
+void bio_clone_blkg_association(struct bio *dst, struct bio *src)
{
- if (src->bi_css)
- WARN_ON(bio_associate_blkcg(dst, src->bi_css));
+ if (src->bi_blkg)
+ bio_associate_blkg(dst, src->bi_blkg);
}
-EXPORT_SYMBOL_GPL(bio_clone_blkcg_association);
+EXPORT_SYMBOL_GPL(bio_clone_blkg_association);
#endif /* CONFIG_BLK_CGROUP */
static void __init biovec_init_slabs(void)
kfree(blkg);
}
+static void __blkg_release(struct rcu_head *rcu)
+{
+ struct blkcg_gq *blkg = container_of(rcu, struct blkcg_gq, rcu_head);
+
+ percpu_ref_exit(&blkg->refcnt);
+
+ /* release the blkcg and parent blkg refs this blkg has been holding */
+ css_put(&blkg->blkcg->css);
+ if (blkg->parent)
+ blkg_put(blkg->parent);
+
+ wb_congested_put(blkg->wb_congested);
+
+ blkg_free(blkg);
+}
+
+/*
+ * A group is RCU protected, but having an rcu lock does not mean that one
+ * can access all the fields of blkg and assume these are valid. For
+ * example, don't try to follow throtl_data and request queue links.
+ *
+ * Having a reference to blkg under an rcu allows accesses to only values
+ * local to groups like group stats and group rate limits.
+ */
+static void blkg_release(struct percpu_ref *ref)
+{
+ struct blkcg_gq *blkg = container_of(ref, struct blkcg_gq, refcnt);
+
+ call_rcu(&blkg->rcu_head, __blkg_release);
+}
+
/**
* blkg_alloc - allocate a blkg
* @blkcg: block cgroup the new blkg is associated with
blkg->q = q;
INIT_LIST_HEAD(&blkg->q_node);
blkg->blkcg = blkcg;
- atomic_set(&blkg->refcnt, 1);
/* root blkg uses @q->root_rl, init rl only for !root blkgs */
if (blkcg != &blkcg_root) {
blkg_get(blkg->parent);
}
+ ret = percpu_ref_init(&blkg->refcnt, blkg_release, 0,
+ GFP_NOWAIT | __GFP_NOWARN);
+ if (ret)
+ goto err_cancel_ref;
+
/* invoke per-policy init */
for (i = 0; i < BLKCG_MAX_POLS; i++) {
struct blkcg_policy *pol = blkcg_policy[i];
blkg_put(blkg);
return ERR_PTR(ret);
+err_cancel_ref:
+ percpu_ref_exit(&blkg->refcnt);
err_put_congested:
wb_congested_put(wb_congested);
err_put_css:
}
/**
- * blkg_lookup_create - lookup blkg, try to create one if not there
+ * __blkg_lookup_create - lookup blkg, try to create one if not there
* @blkcg: blkcg of interest
* @q: request_queue of interest
*
* that all non-root blkg's have access to the parent blkg. This function
* should be called under RCU read lock and @q->queue_lock.
*
- * Returns pointer to the looked up or created blkg on success, ERR_PTR()
- * value on error. If @q is dead, returns ERR_PTR(-EINVAL). If @q is not
- * dead and bypassing, returns ERR_PTR(-EBUSY).
+ * Returns the blkg or the closest blkg if blkg_create fails as it walks
+ * down from root.
*/
-struct blkcg_gq *blkg_lookup_create(struct blkcg *blkcg,
- struct request_queue *q)
+struct blkcg_gq *__blkg_lookup_create(struct blkcg *blkcg,
+ struct request_queue *q)
{
struct blkcg_gq *blkg;
* we shouldn't allow anything to go through for a bypassing queue.
*/
if (unlikely(blk_queue_bypass(q)))
- return ERR_PTR(blk_queue_dying(q) ? -ENODEV : -EBUSY);
+ return q->root_blkg;
blkg = __blkg_lookup(blkcg, q, true);
if (blkg)
/*
* Create blkgs walking down from blkcg_root to @blkcg, so that all
- * non-root blkgs have access to their parents.
+ * non-root blkgs have access to their parents. Returns the closest
+ * blkg to the intended blkg should blkg_create() fail.
*/
while (true) {
struct blkcg *pos = blkcg;
struct blkcg *parent = blkcg_parent(blkcg);
-
- while (parent && !__blkg_lookup(parent, q, false)) {
+ struct blkcg_gq *ret_blkg = q->root_blkg;
+
+ while (parent) {
+ blkg = __blkg_lookup(parent, q, false);
+ if (blkg) {
+ /* remember closest blkg */
+ ret_blkg = blkg;
+ break;
+ }
pos = parent;
parent = blkcg_parent(parent);
}
blkg = blkg_create(pos, q, NULL);
- if (pos == blkcg || IS_ERR(blkg))
+ if (IS_ERR(blkg))
+ return ret_blkg;
+ if (pos == blkcg)
return blkg;
}
}
+/**
+ * blkg_lookup_create - find or create a blkg
+ * @blkcg: target block cgroup
+ * @q: target request_queue
+ *
+ * This looks up or creates the blkg representing the unique pair
+ * of the blkcg and the request_queue.
+ */
+struct blkcg_gq *blkg_lookup_create(struct blkcg *blkcg,
+ struct request_queue *q)
+{
+ struct blkcg_gq *blkg = blkg_lookup(blkcg, q);
+ unsigned long flags;
+
+ if (unlikely(!blkg)) {
+ spin_lock_irqsave(q->queue_lock, flags);
+
+ blkg = __blkg_lookup_create(blkcg, q);
+
+ spin_unlock_irqrestore(q->queue_lock, flags);
+ }
+
+ return blkg;
+}
+
static void blkg_destroy(struct blkcg_gq *blkg)
{
struct blkcg *blkcg = blkg->blkcg;
* Put the reference taken at the time of creation so that when all
* queues are gone, group can be destroyed.
*/
- blkg_put(blkg);
+ percpu_ref_kill(&blkg->refcnt);
}
/**
q->root_rl.blkg = NULL;
}
-/*
- * A group is RCU protected, but having an rcu lock does not mean that one
- * can access all the fields of blkg and assume these are valid. For
- * example, don't try to follow throtl_data and request queue links.
- *
- * Having a reference to blkg under an rcu allows accesses to only values
- * local to groups like group stats and group rate limits.
- */
-void __blkg_release_rcu(struct rcu_head *rcu_head)
-{
- struct blkcg_gq *blkg = container_of(rcu_head, struct blkcg_gq, rcu_head);
-
- /* release the blkcg and parent blkg refs this blkg has been holding */
- css_put(&blkg->blkcg->css);
- if (blkg->parent)
- blkg_put(blkg->parent);
-
- wb_congested_put(blkg->wb_congested);
-
- blkg_free(blkg);
-}
-EXPORT_SYMBOL_GPL(__blkg_release_rcu);
-
/*
* The next function used by blk_queue_for_each_rl(). It's a bit tricky
* because the root blkg uses @q->root_rl instead of its own rl.
blkg = blkg_lookup(blkcg, q);
if (!blkg)
goto out;
- blkg = blkg_try_get(blkg);
- if (!blkg)
+ if (!blkg_tryget(blkg))
goto out;
rcu_read_unlock();
#include "blk.h"
#include "blk-mq.h"
#include "blk-mq-sched.h"
+#include "blk-pm.h"
#include "blk-rq-qos.h"
#ifdef CONFIG_DEBUG_FS
EXPORT_SYMBOL(blk_sync_queue);
/**
- * blk_set_preempt_only - set QUEUE_FLAG_PREEMPT_ONLY
+ * blk_set_pm_only - increment pm_only counter
* @q: request queue pointer
- *
- * Returns the previous value of the PREEMPT_ONLY flag - 0 if the flag was not
- * set and 1 if the flag was already set.
*/
-int blk_set_preempt_only(struct request_queue *q)
+void blk_set_pm_only(struct request_queue *q)
{
- return blk_queue_flag_test_and_set(QUEUE_FLAG_PREEMPT_ONLY, q);
+ atomic_inc(&q->pm_only);
}
-EXPORT_SYMBOL_GPL(blk_set_preempt_only);
+EXPORT_SYMBOL_GPL(blk_set_pm_only);
-void blk_clear_preempt_only(struct request_queue *q)
+void blk_clear_pm_only(struct request_queue *q)
{
- blk_queue_flag_clear(QUEUE_FLAG_PREEMPT_ONLY, q);
- wake_up_all(&q->mq_freeze_wq);
+ int pm_only;
+
+ pm_only = atomic_dec_return(&q->pm_only);
+ WARN_ON_ONCE(pm_only < 0);
+ if (pm_only == 0)
+ wake_up_all(&q->mq_freeze_wq);
}
-EXPORT_SYMBOL_GPL(blk_clear_preempt_only);
+EXPORT_SYMBOL_GPL(blk_clear_pm_only);
/**
* __blk_run_queue_uncond - run a queue whether or not it has been stopped
*/
int blk_queue_enter(struct request_queue *q, blk_mq_req_flags_t flags)
{
- const bool preempt = flags & BLK_MQ_REQ_PREEMPT;
+ const bool pm = flags & BLK_MQ_REQ_PREEMPT;
while (true) {
bool success = false;
rcu_read_lock();
if (percpu_ref_tryget_live(&q->q_usage_counter)) {
/*
- * The code that sets the PREEMPT_ONLY flag is
- * responsible for ensuring that that flag is globally
- * visible before the queue is unfrozen.
+ * The code that increments the pm_only counter is
+ * responsible for ensuring that that counter is
+ * globally visible before the queue is unfrozen.
*/
- if (preempt || !blk_queue_preempt_only(q)) {
+ if (pm || !blk_queue_pm_only(q)) {
success = true;
} else {
percpu_ref_put(&q->q_usage_counter);
wait_event(q->mq_freeze_wq,
(atomic_read(&q->mq_freeze_depth) == 0 &&
- (preempt || !blk_queue_preempt_only(q))) ||
+ (pm || (blk_pm_request_resume(q),
+ !blk_queue_pm_only(q)))) ||
blk_queue_dying(q));
if (blk_queue_dying(q))
return -ENODEV;
mutex_init(&q->sysfs_lock);
spin_lock_init(&q->__queue_lock);
- if (!q->mq_ops)
- q->queue_lock = lock ? : &q->__queue_lock;
+ q->queue_lock = lock ? : &q->__queue_lock;
/*
* A queue starts its life with bypass turned on to avoid
{
WARN_ON_ONCE(q->mq_ops);
- q->fq = blk_alloc_flush_queue(q, NUMA_NO_NODE, q->cmd_size);
+ q->fq = blk_alloc_flush_queue(q, NUMA_NO_NODE, q->cmd_size, GFP_KERNEL);
if (!q->fq)
return -ENOMEM;
}
EXPORT_SYMBOL_GPL(part_round_stats);
-#ifdef CONFIG_PM
-static void blk_pm_put_request(struct request *rq)
-{
- if (rq->q->dev && !(rq->rq_flags & RQF_PM) && !--rq->q->nr_pending)
- pm_runtime_mark_last_busy(rq->q->dev);
-}
-#else
-static inline void blk_pm_put_request(struct request *rq) {}
-#endif
-
void __blk_put_request(struct request_queue *q, struct request *req)
{
req_flags_t rq_flags = req->rq_flags;
blk_req_zone_write_unlock(req);
blk_pm_put_request(req);
+ blk_pm_mark_last_busy(req);
elv_completed_request(q, req);
if (q)
blk_queue_exit(q);
q = bio->bi_disk->queue;
+ bio_reassociate_blkg(q, bio);
flags = 0;
if (bio->bi_opf & REQ_NOWAIT)
flags = BLK_MQ_REQ_NOWAIT;
}
}
-#ifdef CONFIG_PM
-/*
- * Don't process normal requests when queue is suspended
- * or in the process of suspending/resuming
- */
-static bool blk_pm_allow_request(struct request *rq)
-{
- switch (rq->q->rpm_status) {
- case RPM_RESUMING:
- case RPM_SUSPENDING:
- return rq->rq_flags & RQF_PM;
- case RPM_SUSPENDED:
- return false;
- default:
- return true;
- }
-}
-#else
-static bool blk_pm_allow_request(struct request *rq)
-{
- return true;
-}
-#endif
-
void blk_account_io_start(struct request *rq, bool new_io)
{
struct hd_struct *part;
while (1) {
list_for_each_entry(rq, &q->queue_head, queuelist) {
- if (blk_pm_allow_request(rq))
- return rq;
-
- if (rq->rq_flags & RQF_SOFTBARRIER)
- break;
+#ifdef CONFIG_PM
+ /*
+ * If a request gets queued in state RPM_SUSPENDED
+ * then that's a kernel bug.
+ */
+ WARN_ON_ONCE(q->rpm_status == RPM_SUSPENDED);
+#endif
+ return rq;
}
/*
}
EXPORT_SYMBOL(blk_finish_plug);
-#ifdef CONFIG_PM
-/**
- * blk_pm_runtime_init - Block layer runtime PM initialization routine
- * @q: the queue of the device
- * @dev: the device the queue belongs to
- *
- * Description:
- * Initialize runtime-PM-related fields for @q and start auto suspend for
- * @dev. Drivers that want to take advantage of request-based runtime PM
- * should call this function after @dev has been initialized, and its
- * request queue @q has been allocated, and runtime PM for it can not happen
- * yet(either due to disabled/forbidden or its usage_count > 0). In most
- * cases, driver should call this function before any I/O has taken place.
- *
- * This function takes care of setting up using auto suspend for the device,
- * the autosuspend delay is set to -1 to make runtime suspend impossible
- * until an updated value is either set by user or by driver. Drivers do
- * not need to touch other autosuspend settings.
- *
- * The block layer runtime PM is request based, so only works for drivers
- * that use request as their IO unit instead of those directly use bio's.
- */
-void blk_pm_runtime_init(struct request_queue *q, struct device *dev)
-{
- /* Don't enable runtime PM for blk-mq until it is ready */
- if (q->mq_ops) {
- pm_runtime_disable(dev);
- return;
- }
-
- q->dev = dev;
- q->rpm_status = RPM_ACTIVE;
- pm_runtime_set_autosuspend_delay(q->dev, -1);
- pm_runtime_use_autosuspend(q->dev);
-}
-EXPORT_SYMBOL(blk_pm_runtime_init);
-
-/**
- * blk_pre_runtime_suspend - Pre runtime suspend check
- * @q: the queue of the device
- *
- * Description:
- * This function will check if runtime suspend is allowed for the device
- * by examining if there are any requests pending in the queue. If there
- * are requests pending, the device can not be runtime suspended; otherwise,
- * the queue's status will be updated to SUSPENDING and the driver can
- * proceed to suspend the device.
- *
- * For the not allowed case, we mark last busy for the device so that
- * runtime PM core will try to autosuspend it some time later.
- *
- * This function should be called near the start of the device's
- * runtime_suspend callback.
- *
- * Return:
- * 0 - OK to runtime suspend the device
- * -EBUSY - Device should not be runtime suspended
- */
-int blk_pre_runtime_suspend(struct request_queue *q)
-{
- int ret = 0;
-
- if (!q->dev)
- return ret;
-
- spin_lock_irq(q->queue_lock);
- if (q->nr_pending) {
- ret = -EBUSY;
- pm_runtime_mark_last_busy(q->dev);
- } else {
- q->rpm_status = RPM_SUSPENDING;
- }
- spin_unlock_irq(q->queue_lock);
- return ret;
-}
-EXPORT_SYMBOL(blk_pre_runtime_suspend);
-
-/**
- * blk_post_runtime_suspend - Post runtime suspend processing
- * @q: the queue of the device
- * @err: return value of the device's runtime_suspend function
- *
- * Description:
- * Update the queue's runtime status according to the return value of the
- * device's runtime suspend function and mark last busy for the device so
- * that PM core will try to auto suspend the device at a later time.
- *
- * This function should be called near the end of the device's
- * runtime_suspend callback.
- */
-void blk_post_runtime_suspend(struct request_queue *q, int err)
-{
- if (!q->dev)
- return;
-
- spin_lock_irq(q->queue_lock);
- if (!err) {
- q->rpm_status = RPM_SUSPENDED;
- } else {
- q->rpm_status = RPM_ACTIVE;
- pm_runtime_mark_last_busy(q->dev);
- }
- spin_unlock_irq(q->queue_lock);
-}
-EXPORT_SYMBOL(blk_post_runtime_suspend);
-
-/**
- * blk_pre_runtime_resume - Pre runtime resume processing
- * @q: the queue of the device
- *
- * Description:
- * Update the queue's runtime status to RESUMING in preparation for the
- * runtime resume of the device.
- *
- * This function should be called near the start of the device's
- * runtime_resume callback.
- */
-void blk_pre_runtime_resume(struct request_queue *q)
-{
- if (!q->dev)
- return;
-
- spin_lock_irq(q->queue_lock);
- q->rpm_status = RPM_RESUMING;
- spin_unlock_irq(q->queue_lock);
-}
-EXPORT_SYMBOL(blk_pre_runtime_resume);
-
-/**
- * blk_post_runtime_resume - Post runtime resume processing
- * @q: the queue of the device
- * @err: return value of the device's runtime_resume function
- *
- * Description:
- * Update the queue's runtime status according to the return value of the
- * device's runtime_resume function. If it is successfully resumed, process
- * the requests that are queued into the device's queue when it is resuming
- * and then mark last busy and initiate autosuspend for it.
- *
- * This function should be called near the end of the device's
- * runtime_resume callback.
- */
-void blk_post_runtime_resume(struct request_queue *q, int err)
-{
- if (!q->dev)
- return;
-
- spin_lock_irq(q->queue_lock);
- if (!err) {
- q->rpm_status = RPM_ACTIVE;
- __blk_run_queue(q);
- pm_runtime_mark_last_busy(q->dev);
- pm_request_autosuspend(q->dev);
- } else {
- q->rpm_status = RPM_SUSPENDED;
- }
- spin_unlock_irq(q->queue_lock);
-}
-EXPORT_SYMBOL(blk_post_runtime_resume);
-
-/**
- * blk_set_runtime_active - Force runtime status of the queue to be active
- * @q: the queue of the device
- *
- * If the device is left runtime suspended during system suspend the resume
- * hook typically resumes the device and corrects runtime status
- * accordingly. However, that does not affect the queue runtime PM status
- * which is still "suspended". This prevents processing requests from the
- * queue.
- *
- * This function can be used in driver's resume hook to correct queue
- * runtime PM status and re-enable peeking requests from the queue. It
- * should be called before first request is added to the queue.
- */
-void blk_set_runtime_active(struct request_queue *q)
-{
- spin_lock_irq(q->queue_lock);
- q->rpm_status = RPM_ACTIVE;
- pm_runtime_mark_last_busy(q->dev);
- pm_request_autosuspend(q->dev);
- spin_unlock_irq(q->queue_lock);
-}
-EXPORT_SYMBOL(blk_set_runtime_active);
-#endif
-
int __init blk_dev_init(void)
{
BUILD_BUG_ON(REQ_OP_LAST >= (1 << REQ_OP_BITS));
EXPORT_SYMBOL(blkdev_issue_flush);
struct blk_flush_queue *blk_alloc_flush_queue(struct request_queue *q,
- int node, int cmd_size)
+ int node, int cmd_size, gfp_t flags)
{
struct blk_flush_queue *fq;
int rq_sz = sizeof(struct request);
- fq = kzalloc_node(sizeof(*fq), GFP_KERNEL, node);
+ fq = kzalloc_node(sizeof(*fq), flags, node);
if (!fq)
goto fail;
spin_lock_init(&fq->mq_flush_lock);
rq_sz = round_up(rq_sz + cmd_size, cache_line_size());
- fq->flush_rq = kzalloc_node(rq_sz, GFP_KERNEL, node);
+ fq->flush_rq = kzalloc_node(rq_sz, flags, node);
if (!fq->flush_rq)
goto fail_rq;
bio_for_each_integrity_vec(iv, bio, iter) {
if (prev) {
- if (!BIOVEC_PHYS_MERGEABLE(&ivprv, &iv))
+ if (!biovec_phys_mergeable(q, &ivprv, &iv))
goto new_segment;
-
- if (!BIOVEC_SEG_BOUNDARY(q, &ivprv, &iv))
- goto new_segment;
-
if (seg_size + iv.bv_len > queue_max_segment_size(q))
goto new_segment;
bio_for_each_integrity_vec(iv, bio, iter) {
if (prev) {
- if (!BIOVEC_PHYS_MERGEABLE(&ivprv, &iv))
+ if (!biovec_phys_mergeable(q, &ivprv, &iv))
goto new_segment;
-
- if (!BIOVEC_SEG_BOUNDARY(q, &ivprv, &iv))
- goto new_segment;
-
if (sg->length + iv.bv_len > queue_max_segment_size(q))
goto new_segment;
atomic_t scale_cookie;
};
+struct percentile_stats {
+ u64 total;
+ u64 missed;
+};
+
+struct latency_stat {
+ union {
+ struct percentile_stats ps;
+ struct blk_rq_stat rqs;
+ };
+};
+
struct iolatency_grp {
struct blkg_policy_data pd;
- struct blk_rq_stat __percpu *stats;
+ struct latency_stat __percpu *stats;
+ struct latency_stat cur_stat;
struct blk_iolatency *blkiolat;
struct rq_depth rq_depth;
struct rq_wait rq_wait;
/* Our current number of IO's for the last summation. */
u64 nr_samples;
+ bool ssd;
struct child_latency_info child_lat;
};
return pd_to_blkg(&iolat->pd);
}
+static inline void latency_stat_init(struct iolatency_grp *iolat,
+ struct latency_stat *stat)
+{
+ if (iolat->ssd) {
+ stat->ps.total = 0;
+ stat->ps.missed = 0;
+ } else
+ blk_rq_stat_init(&stat->rqs);
+}
+
+static inline void latency_stat_sum(struct iolatency_grp *iolat,
+ struct latency_stat *sum,
+ struct latency_stat *stat)
+{
+ if (iolat->ssd) {
+ sum->ps.total += stat->ps.total;
+ sum->ps.missed += stat->ps.missed;
+ } else
+ blk_rq_stat_sum(&sum->rqs, &stat->rqs);
+}
+
+static inline void latency_stat_record_time(struct iolatency_grp *iolat,
+ u64 req_time)
+{
+ struct latency_stat *stat = get_cpu_ptr(iolat->stats);
+ if (iolat->ssd) {
+ if (req_time >= iolat->min_lat_nsec)
+ stat->ps.missed++;
+ stat->ps.total++;
+ } else
+ blk_rq_stat_add(&stat->rqs, req_time);
+ put_cpu_ptr(stat);
+}
+
+static inline bool latency_sum_ok(struct iolatency_grp *iolat,
+ struct latency_stat *stat)
+{
+ if (iolat->ssd) {
+ u64 thresh = div64_u64(stat->ps.total, 10);
+ thresh = max(thresh, 1ULL);
+ return stat->ps.missed < thresh;
+ }
+ return stat->rqs.mean <= iolat->min_lat_nsec;
+}
+
+static inline u64 latency_stat_samples(struct iolatency_grp *iolat,
+ struct latency_stat *stat)
+{
+ if (iolat->ssd)
+ return stat->ps.total;
+ return stat->rqs.nr_samples;
+}
+
+static inline void iolat_update_total_lat_avg(struct iolatency_grp *iolat,
+ struct latency_stat *stat)
+{
+ int exp_idx;
+
+ if (iolat->ssd)
+ return;
+
+ /*
+ * CALC_LOAD takes in a number stored in fixed point representation.
+ * Because we are using this for IO time in ns, the values stored
+ * are significantly larger than the FIXED_1 denominator (2048).
+ * Therefore, rounding errors in the calculation are negligible and
+ * can be ignored.
+ */
+ exp_idx = min_t(int, BLKIOLATENCY_NR_EXP_FACTORS - 1,
+ div64_u64(iolat->cur_win_nsec,
+ BLKIOLATENCY_EXP_BUCKET_SIZE));
+ CALC_LOAD(iolat->lat_avg, iolatency_exp_factors[exp_idx], stat->rqs.mean);
+}
+
static inline bool iolatency_may_queue(struct iolatency_grp *iolat,
wait_queue_entry_t *wait,
bool first_block)
struct child_latency_info *lat_info,
bool up)
{
- unsigned long qd = blk_queue_depth(blkiolat->rqos.q);
+ unsigned long qd = blkiolat->rqos.q->nr_requests;
unsigned long scale = scale_amount(qd, up);
unsigned long old = atomic_read(&lat_info->scale_cookie);
unsigned long max_scale = qd << 1;
*/
static void scale_change(struct iolatency_grp *iolat, bool up)
{
- unsigned long qd = blk_queue_depth(iolat->blkiolat->rqos.q);
+ unsigned long qd = iolat->blkiolat->rqos.q->nr_requests;
unsigned long scale = scale_amount(qd, up);
unsigned long old = iolat->rq_depth.max_depth;
- bool changed = false;
if (old > qd)
old = qd;
return;
if (old < qd) {
- changed = true;
old += scale;
old = min(old, qd);
iolat->rq_depth.max_depth = old;
wake_up_all(&iolat->rq_wait.wait);
}
- } else if (old > 1) {
+ } else {
old >>= 1;
- changed = true;
iolat->rq_depth.max_depth = max(old, 1UL);
}
}
* scale down event.
*/
samples_thresh = lat_info->nr_samples * 5;
- samples_thresh = div64_u64(samples_thresh, 100);
+ samples_thresh = max(1ULL, div64_u64(samples_thresh, 100));
if (iolat->nr_samples <= samples_thresh)
return;
}
spinlock_t *lock)
{
struct blk_iolatency *blkiolat = BLKIOLATENCY(rqos);
- struct blkcg *blkcg;
- struct blkcg_gq *blkg;
- struct request_queue *q = rqos->q;
+ struct blkcg_gq *blkg = bio->bi_blkg;
bool issue_as_root = bio_issue_as_root_blkg(bio);
if (!blk_iolatency_enabled(blkiolat))
return;
- rcu_read_lock();
- blkcg = bio_blkcg(bio);
- bio_associate_blkcg(bio, &blkcg->css);
- blkg = blkg_lookup(blkcg, q);
- if (unlikely(!blkg)) {
- if (!lock)
- spin_lock_irq(q->queue_lock);
- blkg = blkg_lookup_create(blkcg, q);
- if (IS_ERR(blkg))
- blkg = NULL;
- if (!lock)
- spin_unlock_irq(q->queue_lock);
- }
- if (!blkg)
- goto out;
-
- bio_issue_init(&bio->bi_issue, bio_sectors(bio));
- bio_associate_blkg(bio, blkg);
-out:
- rcu_read_unlock();
while (blkg && blkg->parent) {
struct iolatency_grp *iolat = blkg_to_lat(blkg);
if (!iolat) {
struct bio_issue *issue, u64 now,
bool issue_as_root)
{
- struct blk_rq_stat *rq_stat;
u64 start = bio_issue_time(issue);
u64 req_time;
return;
}
- rq_stat = get_cpu_ptr(iolat->stats);
- blk_rq_stat_add(rq_stat, req_time);
- put_cpu_ptr(rq_stat);
+ latency_stat_record_time(iolat, req_time);
}
#define BLKIOLATENCY_MIN_ADJUST_TIME (500 * NSEC_PER_MSEC)
struct blkcg_gq *blkg = lat_to_blkg(iolat);
struct iolatency_grp *parent;
struct child_latency_info *lat_info;
- struct blk_rq_stat stat;
+ struct latency_stat stat;
unsigned long flags;
- int cpu, exp_idx;
+ int cpu;
- blk_rq_stat_init(&stat);
+ latency_stat_init(iolat, &stat);
preempt_disable();
for_each_online_cpu(cpu) {
- struct blk_rq_stat *s;
+ struct latency_stat *s;
s = per_cpu_ptr(iolat->stats, cpu);
- blk_rq_stat_sum(&stat, s);
- blk_rq_stat_init(s);
+ latency_stat_sum(iolat, &stat, s);
+ latency_stat_init(iolat, s);
}
preempt_enable();
lat_info = &parent->child_lat;
- /*
- * CALC_LOAD takes in a number stored in fixed point representation.
- * Because we are using this for IO time in ns, the values stored
- * are significantly larger than the FIXED_1 denominator (2048).
- * Therefore, rounding errors in the calculation are negligible and
- * can be ignored.
- */
- exp_idx = min_t(int, BLKIOLATENCY_NR_EXP_FACTORS - 1,
- div64_u64(iolat->cur_win_nsec,
- BLKIOLATENCY_EXP_BUCKET_SIZE));
- CALC_LOAD(iolat->lat_avg, iolatency_exp_factors[exp_idx], stat.mean);
+ iolat_update_total_lat_avg(iolat, &stat);
/* Everything is ok and we don't need to adjust the scale. */
- if (stat.mean <= iolat->min_lat_nsec &&
+ if (latency_sum_ok(iolat, &stat) &&
atomic_read(&lat_info->scale_cookie) == DEFAULT_SCALE_COOKIE)
return;
/* Somebody beat us to the punch, just bail. */
spin_lock_irqsave(&lat_info->lock, flags);
+
+ latency_stat_sum(iolat, &iolat->cur_stat, &stat);
lat_info->nr_samples -= iolat->nr_samples;
- lat_info->nr_samples += stat.nr_samples;
- iolat->nr_samples = stat.nr_samples;
+ lat_info->nr_samples += latency_stat_samples(iolat, &iolat->cur_stat);
+ iolat->nr_samples = latency_stat_samples(iolat, &iolat->cur_stat);
if ((lat_info->last_scale_event >= now ||
- now - lat_info->last_scale_event < BLKIOLATENCY_MIN_ADJUST_TIME) &&
- lat_info->scale_lat <= iolat->min_lat_nsec)
+ now - lat_info->last_scale_event < BLKIOLATENCY_MIN_ADJUST_TIME))
goto out;
- if (stat.mean <= iolat->min_lat_nsec &&
- stat.nr_samples >= BLKIOLATENCY_MIN_GOOD_SAMPLES) {
+ if (latency_sum_ok(iolat, &iolat->cur_stat) &&
+ latency_sum_ok(iolat, &stat)) {
+ if (latency_stat_samples(iolat, &iolat->cur_stat) <
+ BLKIOLATENCY_MIN_GOOD_SAMPLES)
+ goto out;
if (lat_info->scale_grp == iolat) {
lat_info->last_scale_event = now;
scale_cookie_change(iolat->blkiolat, lat_info, true);
}
- } else if (stat.mean > iolat->min_lat_nsec) {
+ } else if (lat_info->scale_lat == 0 ||
+ lat_info->scale_lat >= iolat->min_lat_nsec) {
lat_info->last_scale_event = now;
if (!lat_info->scale_grp ||
lat_info->scale_lat > iolat->min_lat_nsec) {
}
scale_cookie_change(iolat->blkiolat, lat_info, false);
}
+ latency_stat_init(iolat, &iolat->cur_stat);
out:
spin_unlock_irqrestore(&lat_info->lock, flags);
}
* We could be exiting, don't access the pd unless we have a
* ref on the blkg.
*/
- if (!blkg_try_get(blkg))
+ if (!blkg_tryget(blkg))
continue;
iolat = blkg_to_lat(blkg);
{
struct blkcg *blkcg = css_to_blkcg(of_css(of));
struct blkcg_gq *blkg;
- struct blk_iolatency *blkiolat;
struct blkg_conf_ctx ctx;
struct iolatency_grp *iolat;
char *p, *tok;
return ret;
iolat = blkg_to_lat(ctx.blkg);
- blkiolat = iolat->blkiolat;
p = ctx.body;
ret = -EINVAL;
return 0;
}
+static size_t iolatency_ssd_stat(struct iolatency_grp *iolat, char *buf,
+ size_t size)
+{
+ struct latency_stat stat;
+ int cpu;
+
+ latency_stat_init(iolat, &stat);
+ preempt_disable();
+ for_each_online_cpu(cpu) {
+ struct latency_stat *s;
+ s = per_cpu_ptr(iolat->stats, cpu);
+ latency_stat_sum(iolat, &stat, s);
+ }
+ preempt_enable();
+
+ if (iolat->rq_depth.max_depth == UINT_MAX)
+ return scnprintf(buf, size, " missed=%llu total=%llu depth=max",
+ (unsigned long long)stat.ps.missed,
+ (unsigned long long)stat.ps.total);
+ return scnprintf(buf, size, " missed=%llu total=%llu depth=%u",
+ (unsigned long long)stat.ps.missed,
+ (unsigned long long)stat.ps.total,
+ iolat->rq_depth.max_depth);
+}
+
static size_t iolatency_pd_stat(struct blkg_policy_data *pd, char *buf,
size_t size)
{
struct iolatency_grp *iolat = pd_to_lat(pd);
- unsigned long long avg_lat = div64_u64(iolat->lat_avg, NSEC_PER_USEC);
- unsigned long long cur_win = div64_u64(iolat->cur_win_nsec, NSEC_PER_MSEC);
+ unsigned long long avg_lat;
+ unsigned long long cur_win;
+
+ if (iolat->ssd)
+ return iolatency_ssd_stat(iolat, buf, size);
+ avg_lat = div64_u64(iolat->lat_avg, NSEC_PER_USEC);
+ cur_win = div64_u64(iolat->cur_win_nsec, NSEC_PER_MSEC);
if (iolat->rq_depth.max_depth == UINT_MAX)
return scnprintf(buf, size, " depth=max avg_lat=%llu win=%llu",
avg_lat, cur_win);
iolat = kzalloc_node(sizeof(*iolat), gfp, node);
if (!iolat)
return NULL;
- iolat->stats = __alloc_percpu_gfp(sizeof(struct blk_rq_stat),
- __alignof__(struct blk_rq_stat), gfp);
+ iolat->stats = __alloc_percpu_gfp(sizeof(struct latency_stat),
+ __alignof__(struct latency_stat), gfp);
if (!iolat->stats) {
kfree(iolat);
return NULL;
u64 now = ktime_to_ns(ktime_get());
int cpu;
+ if (blk_queue_nonrot(blkg->q))
+ iolat->ssd = true;
+ else
+ iolat->ssd = false;
+
for_each_possible_cpu(cpu) {
- struct blk_rq_stat *stat;
+ struct latency_stat *stat;
stat = per_cpu_ptr(iolat->stats, cpu);
- blk_rq_stat_init(stat);
+ latency_stat_init(iolat, stat);
}
+ latency_stat_init(iolat, &iolat->cur_stat);
rq_wait_init(&iolat->rq_wait);
spin_lock_init(&iolat->child_lat.lock);
- iolat->rq_depth.queue_depth = blk_queue_depth(blkg->q);
+ iolat->rq_depth.queue_depth = blkg->q->nr_requests;
iolat->rq_depth.max_depth = UINT_MAX;
iolat->rq_depth.default_depth = iolat->rq_depth.queue_depth;
iolat->blkiolat = blkiolat;
{
struct request_queue *q = bdev_get_queue(bdev);
struct bio *bio = *biop;
- unsigned int granularity;
unsigned int op;
- int alignment;
sector_t bs_mask;
if (!q)
if ((sector | nr_sects) & bs_mask)
return -EINVAL;
- /* Zero-sector (unknown) and one-sector granularities are the same. */
- granularity = max(q->limits.discard_granularity >> 9, 1U);
- alignment = (bdev_discard_alignment(bdev) >> 9) % granularity;
-
while (nr_sects) {
- unsigned int req_sects;
- sector_t end_sect, tmp;
+ unsigned int req_sects = nr_sects;
+ sector_t end_sect;
- /*
- * Issue in chunks of the user defined max discard setting,
- * ensuring that bi_size doesn't overflow
- */
- req_sects = min_t(sector_t, nr_sects,
- q->limits.max_discard_sectors);
if (!req_sects)
goto fail;
if (req_sects > UINT_MAX >> 9)
req_sects = UINT_MAX >> 9;
- /*
- * If splitting a request, and the next starting sector would be
- * misaligned, stop the discard at the previous aligned sector.
- */
end_sect = sector + req_sects;
- tmp = end_sect;
- if (req_sects < nr_sects &&
- sector_div(tmp, granularity) != alignment) {
- end_sect = end_sect - alignment;
- sector_div(end_sect, granularity);
- end_sect = end_sect * granularity + alignment;
- req_sects = end_sect - sector;
- }
bio = next_bio(bio, 0, gfp_mask);
bio->bi_iter.bi_sector = sector;
#include "blk.h"
+/*
+ * Check if the two bvecs from two bios can be merged to one segment. If yes,
+ * no need to check gap between the two bios since the 1st bio and the 1st bvec
+ * in the 2nd bio can be handled in one segment.
+ */
+static inline bool bios_segs_mergeable(struct request_queue *q,
+ struct bio *prev, struct bio_vec *prev_last_bv,
+ struct bio_vec *next_first_bv)
+{
+ if (!biovec_phys_mergeable(q, prev_last_bv, next_first_bv))
+ return false;
+ if (prev->bi_seg_back_size + next_first_bv->bv_len >
+ queue_max_segment_size(q))
+ return false;
+ return true;
+}
+
+static inline bool bio_will_gap(struct request_queue *q,
+ struct request *prev_rq, struct bio *prev, struct bio *next)
+{
+ struct bio_vec pb, nb;
+
+ if (!bio_has_data(prev) || !queue_virt_boundary(q))
+ return false;
+
+ /*
+ * Don't merge if the 1st bio starts with non-zero offset, otherwise it
+ * is quite difficult to respect the sg gap limit. We work hard to
+ * merge a huge number of small single bios in case of mkfs.
+ */
+ if (prev_rq)
+ bio_get_first_bvec(prev_rq->bio, &pb);
+ else
+ bio_get_first_bvec(prev, &pb);
+ if (pb.bv_offset)
+ return true;
+
+ /*
+ * We don't need to worry about the situation that the merged segment
+ * ends in unaligned virt boundary:
+ *
+ * - if 'pb' ends aligned, the merged segment ends aligned
+ * - if 'pb' ends unaligned, the next bio must include
+ * one single bvec of 'nb', otherwise the 'nb' can't
+ * merge with 'pb'
+ */
+ bio_get_last_bvec(prev, &pb);
+ bio_get_first_bvec(next, &nb);
+ if (bios_segs_mergeable(q, prev, &pb, &nb))
+ return false;
+ return __bvec_gap_to_prev(q, &pb, nb.bv_offset);
+}
+
+static inline bool req_gap_back_merge(struct request *req, struct bio *bio)
+{
+ return bio_will_gap(req->q, req, req->biotail, bio);
+}
+
+static inline bool req_gap_front_merge(struct request *req, struct bio *bio)
+{
+ return bio_will_gap(req->q, NULL, bio, req->bio);
+}
+
static struct bio *blk_bio_discard_split(struct request_queue *q,
struct bio *bio,
struct bio_set *bs,
if (bvprvp && blk_queue_cluster(q)) {
if (seg_size + bv.bv_len > queue_max_segment_size(q))
goto new_segment;
- if (!BIOVEC_PHYS_MERGEABLE(bvprvp, &bv))
- goto new_segment;
- if (!BIOVEC_SEG_BOUNDARY(q, bvprvp, &bv))
+ if (!biovec_phys_mergeable(q, bvprvp, &bv))
goto new_segment;
seg_size += bv.bv_len;
if (seg_size + bv.bv_len
> queue_max_segment_size(q))
goto new_segment;
- if (!BIOVEC_PHYS_MERGEABLE(&bvprv, &bv))
- goto new_segment;
- if (!BIOVEC_SEG_BOUNDARY(q, &bvprv, &bv))
+ if (!biovec_phys_mergeable(q, &bvprv, &bv))
goto new_segment;
seg_size += bv.bv_len;
bio_get_last_bvec(bio, &end_bv);
bio_get_first_bvec(nxt, &nxt_bv);
- if (!BIOVEC_PHYS_MERGEABLE(&end_bv, &nxt_bv))
- return 0;
-
- /*
- * bio and nxt are contiguous in memory; check if the queue allows
- * these two to be merged into one
- */
- if (BIOVEC_SEG_BOUNDARY(q, &end_bv, &nxt_bv))
- return 1;
-
- return 0;
+ return biovec_phys_mergeable(q, &end_bv, &nxt_bv);
}
static inline void
if (*sg && *cluster) {
if ((*sg)->length + nbytes > queue_max_segment_size(q))
goto new_segment;
-
- if (!BIOVEC_PHYS_MERGEABLE(bvprv, bvec))
- goto new_segment;
- if (!BIOVEC_SEG_BOUNDARY(q, bvprv, bvec))
+ if (!biovec_phys_mergeable(q, bvprv, bvec))
goto new_segment;
(*sg)->length += nbytes;
return 0;
}
+static int queue_pm_only_show(void *data, struct seq_file *m)
+{
+ struct request_queue *q = data;
+
+ seq_printf(m, "%d\n", atomic_read(&q->pm_only));
+ return 0;
+}
+
#define QUEUE_FLAG_NAME(name) [QUEUE_FLAG_##name] = #name
static const char *const blk_queue_flag_name[] = {
QUEUE_FLAG_NAME(QUEUED),
QUEUE_FLAG_NAME(REGISTERED),
QUEUE_FLAG_NAME(SCSI_PASSTHROUGH),
QUEUE_FLAG_NAME(QUIESCED),
- QUEUE_FLAG_NAME(PREEMPT_ONLY),
};
#undef QUEUE_FLAG_NAME
static const struct blk_mq_debugfs_attr blk_mq_debugfs_queue_attrs[] = {
{ "poll_stat", 0400, queue_poll_stat_show },
{ "requeue_list", 0400, .seq_ops = &queue_requeue_list_seq_ops },
+ { "pm_only", 0600, queue_pm_only_show, NULL },
{ "state", 0600, queue_state_show, queue_state_write },
{ "write_hints", 0600, queue_write_hint_show, queue_write_hint_store },
{ "zone_wlock", 0400, queue_zone_wlock_show, NULL },
{
const struct show_busy_params *params = data;
- if (blk_mq_map_queue(rq->q, rq->mq_ctx->cpu) == params->hctx &&
- blk_mq_rq_state(rq) != MQ_RQ_IDLE)
+ if (blk_mq_map_queue(rq->q, rq->mq_ctx->cpu) == params->hctx)
__blk_mq_debugfs_rq_show(params->m,
list_entry_rq(&rq->queuelist));
}
return true;
}
-static inline void blk_mq_sched_completed_request(struct request *rq)
+static inline void blk_mq_sched_completed_request(struct request *rq, u64 now)
{
struct elevator_queue *e = rq->q->elevator;
if (e && e->type->ops.mq.completed_request)
- e->type->ops.mq.completed_request(rq);
+ e->type->ops.mq.completed_request(rq, now);
}
static inline void blk_mq_sched_started_request(struct request *rq)
/*
* We can hit rq == NULL here, because the tagging functions
- * test and set the bit before assining ->rqs[].
+ * test and set the bit before assigning ->rqs[].
*/
if (rq && rq->q == hctx->queue)
iter_data->fn(hctx, rq, iter_data->data, reserved);
return true;
}
+/**
+ * bt_for_each - iterate over the requests associated with a hardware queue
+ * @hctx: Hardware queue to examine.
+ * @bt: sbitmap to examine. This is either the breserved_tags member
+ * or the bitmap_tags member of struct blk_mq_tags.
+ * @fn: Pointer to the function that will be called for each request
+ * associated with @hctx that has been assigned a driver tag.
+ * @fn will be called as follows: @fn(@hctx, rq, @data, @reserved)
+ * where rq is a pointer to a request.
+ * @data: Will be passed as third argument to @fn.
+ * @reserved: Indicates whether @bt is the breserved_tags member or the
+ * bitmap_tags member of struct blk_mq_tags.
+ */
static void bt_for_each(struct blk_mq_hw_ctx *hctx, struct sbitmap_queue *bt,
busy_iter_fn *fn, void *data, bool reserved)
{
return true;
}
+/**
+ * bt_tags_for_each - iterate over the requests in a tag map
+ * @tags: Tag map to iterate over.
+ * @bt: sbitmap to examine. This is either the breserved_tags member
+ * or the bitmap_tags member of struct blk_mq_tags.
+ * @fn: Pointer to the function that will be called for each started
+ * request. @fn will be called as follows: @fn(rq, @data,
+ * @reserved) where rq is a pointer to a request.
+ * @data: Will be passed as second argument to @fn.
+ * @reserved: Indicates whether @bt is the breserved_tags member or the
+ * bitmap_tags member of struct blk_mq_tags.
+ */
static void bt_tags_for_each(struct blk_mq_tags *tags, struct sbitmap_queue *bt,
busy_tag_iter_fn *fn, void *data, bool reserved)
{
sbitmap_for_each_set(&bt->sb, bt_tags_iter, &iter_data);
}
+/**
+ * blk_mq_all_tag_busy_iter - iterate over all started requests in a tag map
+ * @tags: Tag map to iterate over.
+ * @fn: Pointer to the function that will be called for each started
+ * request. @fn will be called as follows: @fn(rq, @priv,
+ * reserved) where rq is a pointer to a request. 'reserved'
+ * indicates whether or not @rq is a reserved request.
+ * @priv: Will be passed as second argument to @fn.
+ */
static void blk_mq_all_tag_busy_iter(struct blk_mq_tags *tags,
busy_tag_iter_fn *fn, void *priv)
{
bt_tags_for_each(tags, &tags->bitmap_tags, fn, priv, false);
}
+/**
+ * blk_mq_tagset_busy_iter - iterate over all started requests in a tag set
+ * @tagset: Tag set to iterate over.
+ * @fn: Pointer to the function that will be called for each started
+ * request. @fn will be called as follows: @fn(rq, @priv,
+ * reserved) where rq is a pointer to a request. 'reserved'
+ * indicates whether or not @rq is a reserved request.
+ * @priv: Will be passed as second argument to @fn.
+ */
void blk_mq_tagset_busy_iter(struct blk_mq_tag_set *tagset,
busy_tag_iter_fn *fn, void *priv)
{
}
EXPORT_SYMBOL(blk_mq_tagset_busy_iter);
+/**
+ * blk_mq_queue_tag_busy_iter - iterate over all requests with a driver tag
+ * @q: Request queue to examine.
+ * @fn: Pointer to the function that will be called for each request
+ * on @q. @fn will be called as follows: @fn(hctx, rq, @priv,
+ * reserved) where rq is a pointer to a request and hctx points
+ * to the hardware queue associated with the request. 'reserved'
+ * indicates whether or not @rq is a reserved request.
+ * @priv: Will be passed as third argument to @fn.
+ *
+ * Note: if @q->tag_set is shared with other request queues then @fn will be
+ * called for all requests on all queues that share that tag set and not only
+ * for requests associated with @q.
+ */
void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn,
void *priv)
{
int i;
/*
- * __blk_mq_update_nr_hw_queues will update the nr_hw_queues and
- * queue_hw_ctx after freeze the queue, so we use q_usage_counter
- * to avoid race with it.
+ * __blk_mq_update_nr_hw_queues() updates nr_hw_queues and queue_hw_ctx
+ * while the queue is frozen. So we can use q_usage_counter to avoid
+ * racing with it. __blk_mq_update_nr_hw_queues() uses
+ * synchronize_rcu() to ensure this function left the critical section
+ * below.
*/
if (!percpu_ref_tryget(&q->q_usage_counter))
return;
struct blk_mq_tags *tags = hctx->tags;
/*
- * If not software queues are currently mapped to this
+ * If no software queues are currently mapped to this
* hardware queue, there's nothing to check
*/
if (!blk_mq_hw_queue_mapped(hctx))
#include "blk-mq.h"
#include "blk-mq-debugfs.h"
#include "blk-mq-tag.h"
+#include "blk-pm.h"
#include "blk-stat.h"
#include "blk-mq-sched.h"
#include "blk-rq-qos.h"
freeze_depth = atomic_dec_return(&q->mq_freeze_depth);
WARN_ON_ONCE(freeze_depth < 0);
if (!freeze_depth) {
- percpu_ref_reinit(&q->q_usage_counter);
+ percpu_ref_resurrect(&q->q_usage_counter);
wake_up_all(&q->mq_freeze_wq);
}
}
struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(q, ctx->cpu);
const int sched_tag = rq->internal_tag;
+ blk_pm_mark_last_busy(rq);
if (rq->tag != -1)
blk_mq_put_tag(hctx, hctx->tags, ctx, rq->tag);
if (sched_tag != -1)
blk_stat_add(rq, now);
}
+ if (rq->internal_tag != -1)
+ blk_mq_sched_completed_request(rq, now);
+
blk_account_io_done(rq, now);
if (rq->end_io) {
if (!blk_mq_mark_complete(rq))
return;
- if (rq->internal_tag != -1)
- blk_mq_sched_completed_request(rq);
+
+ /*
+ * Most of single queue controllers, there is only one irq vector
+ * for handling IO completion, and the only irq's affinity is set
+ * as all possible CPUs. On most of ARCHs, this affinity means the
+ * irq is handled on one specific CPU.
+ *
+ * So complete IO reqeust in softirq context in case of single queue
+ * for not degrading IO performance by irqsoff latency.
+ */
+ if (rq->q->nr_hw_queues == 1) {
+ __blk_complete_request(rq);
+ return;
+ }
if (!test_bit(QUEUE_FLAG_SAME_COMP, &rq->q->queue_flags)) {
rq->q->softirq_done_fn(rq);
struct blk_mq_tag_set *set,
struct blk_mq_hw_ctx *hctx, unsigned int hctx_idx)
{
- blk_mq_debugfs_unregister_hctx(hctx);
-
if (blk_mq_hw_queue_mapped(hctx))
blk_mq_tag_idle(hctx);
queue_for_each_hw_ctx(q, hctx, i) {
if (i == nr_queue)
break;
+ blk_mq_debugfs_unregister_hctx(hctx);
blk_mq_exit_hctx(q, set, hctx, i);
}
}
* runtime
*/
hctx->ctxs = kmalloc_array_node(nr_cpu_ids, sizeof(void *),
- GFP_KERNEL, node);
+ GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY, node);
if (!hctx->ctxs)
goto unregister_cpu_notifier;
- if (sbitmap_init_node(&hctx->ctx_map, nr_cpu_ids, ilog2(8), GFP_KERNEL,
- node))
+ if (sbitmap_init_node(&hctx->ctx_map, nr_cpu_ids, ilog2(8),
+ GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY, node))
goto free_ctxs;
hctx->nr_ctx = 0;
set->ops->init_hctx(hctx, set->driver_data, hctx_idx))
goto free_bitmap;
- hctx->fq = blk_alloc_flush_queue(q, hctx->numa_node, set->cmd_size);
+ hctx->fq = blk_alloc_flush_queue(q, hctx->numa_node, set->cmd_size,
+ GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY);
if (!hctx->fq)
goto exit_hctx;
if (hctx->flags & BLK_MQ_F_BLOCKING)
init_srcu_struct(hctx->srcu);
- blk_mq_debugfs_register_hctx(q, hctx);
-
return 0;
free_fq:
}
EXPORT_SYMBOL(blk_mq_init_queue);
+/*
+ * Helper for setting up a queue with mq ops, given queue depth, and
+ * the passed in mq ops flags.
+ */
+struct request_queue *blk_mq_init_sq_queue(struct blk_mq_tag_set *set,
+ const struct blk_mq_ops *ops,
+ unsigned int queue_depth,
+ unsigned int set_flags)
+{
+ struct request_queue *q;
+ int ret;
+
+ memset(set, 0, sizeof(*set));
+ set->ops = ops;
+ set->nr_hw_queues = 1;
+ set->queue_depth = queue_depth;
+ set->numa_node = NUMA_NO_NODE;
+ set->flags = set_flags;
+
+ ret = blk_mq_alloc_tag_set(set);
+ if (ret)
+ return ERR_PTR(ret);
+
+ q = blk_mq_init_queue(set);
+ if (IS_ERR(q)) {
+ blk_mq_free_tag_set(set);
+ return q;
+ }
+
+ return q;
+}
+EXPORT_SYMBOL(blk_mq_init_sq_queue);
+
static int blk_mq_hw_ctx_size(struct blk_mq_tag_set *tag_set)
{
int hw_ctx_size = sizeof(struct blk_mq_hw_ctx);
return hw_ctx_size;
}
+static struct blk_mq_hw_ctx *blk_mq_alloc_and_init_hctx(
+ struct blk_mq_tag_set *set, struct request_queue *q,
+ int hctx_idx, int node)
+{
+ struct blk_mq_hw_ctx *hctx;
+
+ hctx = kzalloc_node(blk_mq_hw_ctx_size(set),
+ GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY,
+ node);
+ if (!hctx)
+ return NULL;
+
+ if (!zalloc_cpumask_var_node(&hctx->cpumask,
+ GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY,
+ node)) {
+ kfree(hctx);
+ return NULL;
+ }
+
+ atomic_set(&hctx->nr_active, 0);
+ hctx->numa_node = node;
+ hctx->queue_num = hctx_idx;
+
+ if (blk_mq_init_hctx(q, set, hctx, hctx_idx)) {
+ free_cpumask_var(hctx->cpumask);
+ kfree(hctx);
+ return NULL;
+ }
+ blk_mq_hctx_kobj_init(hctx);
+
+ return hctx;
+}
+
static void blk_mq_realloc_hw_ctxs(struct blk_mq_tag_set *set,
struct request_queue *q)
{
- int i, j;
+ int i, j, end;
struct blk_mq_hw_ctx **hctxs = q->queue_hw_ctx;
- blk_mq_sysfs_unregister(q);
-
/* protect against switching io scheduler */
mutex_lock(&q->sysfs_lock);
for (i = 0; i < set->nr_hw_queues; i++) {
int node;
-
- if (hctxs[i])
- continue;
+ struct blk_mq_hw_ctx *hctx;
node = blk_mq_hw_queue_to_node(q->mq_map, i);
- hctxs[i] = kzalloc_node(blk_mq_hw_ctx_size(set),
- GFP_KERNEL, node);
- if (!hctxs[i])
- break;
-
- if (!zalloc_cpumask_var_node(&hctxs[i]->cpumask, GFP_KERNEL,
- node)) {
- kfree(hctxs[i]);
- hctxs[i] = NULL;
- break;
- }
-
- atomic_set(&hctxs[i]->nr_active, 0);
- hctxs[i]->numa_node = node;
- hctxs[i]->queue_num = i;
+ /*
+ * If the hw queue has been mapped to another numa node,
+ * we need to realloc the hctx. If allocation fails, fallback
+ * to use the previous one.
+ */
+ if (hctxs[i] && (hctxs[i]->numa_node == node))
+ continue;
- if (blk_mq_init_hctx(q, set, hctxs[i], i)) {
- free_cpumask_var(hctxs[i]->cpumask);
- kfree(hctxs[i]);
- hctxs[i] = NULL;
- break;
+ hctx = blk_mq_alloc_and_init_hctx(set, q, i, node);
+ if (hctx) {
+ if (hctxs[i]) {
+ blk_mq_exit_hctx(q, set, hctxs[i], i);
+ kobject_put(&hctxs[i]->kobj);
+ }
+ hctxs[i] = hctx;
+ } else {
+ if (hctxs[i])
+ pr_warn("Allocate new hctx on node %d fails,\
+ fallback to previous one on node %d\n",
+ node, hctxs[i]->numa_node);
+ else
+ break;
}
- blk_mq_hctx_kobj_init(hctxs[i]);
}
- for (j = i; j < q->nr_hw_queues; j++) {
+ /*
+ * Increasing nr_hw_queues fails. Free the newly allocated
+ * hctxs and keep the previous q->nr_hw_queues.
+ */
+ if (i != set->nr_hw_queues) {
+ j = q->nr_hw_queues;
+ end = i;
+ } else {
+ j = i;
+ end = q->nr_hw_queues;
+ q->nr_hw_queues = set->nr_hw_queues;
+ }
+
+ for (; j < end; j++) {
struct blk_mq_hw_ctx *hctx = hctxs[j];
if (hctx) {
}
}
- q->nr_hw_queues = i;
mutex_unlock(&q->sysfs_lock);
- blk_mq_sysfs_register(q);
}
struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set,
blk_mq_exit_hw_queues(q, set, set->nr_hw_queues);
}
-/* Basically redo blk_mq_init_queue with queue frozen */
-static void blk_mq_queue_reinit(struct request_queue *q)
-{
- WARN_ON_ONCE(!atomic_read(&q->mq_freeze_depth));
-
- blk_mq_debugfs_unregister_hctxs(q);
- blk_mq_sysfs_unregister(q);
-
- /*
- * redo blk_mq_init_cpu_queues and blk_mq_init_hw_queues. FIXME: maybe
- * we should change hctx numa_node according to the new topology (this
- * involves freeing and re-allocating memory, worth doing?)
- */
- blk_mq_map_swqueue(q);
-
- blk_mq_sysfs_register(q);
- blk_mq_debugfs_register_hctxs(q);
-}
-
static int __blk_mq_alloc_rq_maps(struct blk_mq_tag_set *set)
{
int i;
{
struct request_queue *q;
LIST_HEAD(head);
+ int prev_nr_hw_queues;
lockdep_assert_held(&set->tag_list_lock);
if (!blk_mq_elv_switch_none(&head, q))
goto switch_back;
+ list_for_each_entry(q, &set->tag_list, tag_set_list) {
+ blk_mq_debugfs_unregister_hctxs(q);
+ blk_mq_sysfs_unregister(q);
+ }
+
+ prev_nr_hw_queues = set->nr_hw_queues;
set->nr_hw_queues = nr_hw_queues;
blk_mq_update_queue_map(set);
+fallback:
list_for_each_entry(q, &set->tag_list, tag_set_list) {
blk_mq_realloc_hw_ctxs(set, q);
- blk_mq_queue_reinit(q);
+ if (q->nr_hw_queues != set->nr_hw_queues) {
+ pr_warn("Increasing nr_hw_queues to %d fails, fallback to %d\n",
+ nr_hw_queues, prev_nr_hw_queues);
+ set->nr_hw_queues = prev_nr_hw_queues;
+ blk_mq_map_queues(set);
+ goto fallback;
+ }
+ blk_mq_map_swqueue(q);
+ }
+
+ list_for_each_entry(q, &set->tag_list, tag_set_list) {
+ blk_mq_sysfs_register(q);
+ blk_mq_debugfs_register_hctxs(q);
}
switch_back:
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/blk-mq.h>
+#include <linux/blk-pm.h>
+#include <linux/blkdev.h>
+#include <linux/pm_runtime.h>
+#include "blk-mq.h"
+#include "blk-mq-tag.h"
+
+/**
+ * blk_pm_runtime_init - Block layer runtime PM initialization routine
+ * @q: the queue of the device
+ * @dev: the device the queue belongs to
+ *
+ * Description:
+ * Initialize runtime-PM-related fields for @q and start auto suspend for
+ * @dev. Drivers that want to take advantage of request-based runtime PM
+ * should call this function after @dev has been initialized, and its
+ * request queue @q has been allocated, and runtime PM for it can not happen
+ * yet(either due to disabled/forbidden or its usage_count > 0). In most
+ * cases, driver should call this function before any I/O has taken place.
+ *
+ * This function takes care of setting up using auto suspend for the device,
+ * the autosuspend delay is set to -1 to make runtime suspend impossible
+ * until an updated value is either set by user or by driver. Drivers do
+ * not need to touch other autosuspend settings.
+ *
+ * The block layer runtime PM is request based, so only works for drivers
+ * that use request as their IO unit instead of those directly use bio's.
+ */
+void blk_pm_runtime_init(struct request_queue *q, struct device *dev)
+{
+ q->dev = dev;
+ q->rpm_status = RPM_ACTIVE;
+ pm_runtime_set_autosuspend_delay(q->dev, -1);
+ pm_runtime_use_autosuspend(q->dev);
+}
+EXPORT_SYMBOL(blk_pm_runtime_init);
+
+/**
+ * blk_pre_runtime_suspend - Pre runtime suspend check
+ * @q: the queue of the device
+ *
+ * Description:
+ * This function will check if runtime suspend is allowed for the device
+ * by examining if there are any requests pending in the queue. If there
+ * are requests pending, the device can not be runtime suspended; otherwise,
+ * the queue's status will be updated to SUSPENDING and the driver can
+ * proceed to suspend the device.
+ *
+ * For the not allowed case, we mark last busy for the device so that
+ * runtime PM core will try to autosuspend it some time later.
+ *
+ * This function should be called near the start of the device's
+ * runtime_suspend callback.
+ *
+ * Return:
+ * 0 - OK to runtime suspend the device
+ * -EBUSY - Device should not be runtime suspended
+ */
+int blk_pre_runtime_suspend(struct request_queue *q)
+{
+ int ret = 0;
+
+ if (!q->dev)
+ return ret;
+
+ WARN_ON_ONCE(q->rpm_status != RPM_ACTIVE);
+
+ /*
+ * Increase the pm_only counter before checking whether any
+ * non-PM blk_queue_enter() calls are in progress to avoid that any
+ * new non-PM blk_queue_enter() calls succeed before the pm_only
+ * counter is decreased again.
+ */
+ blk_set_pm_only(q);
+ ret = -EBUSY;
+ /* Switch q_usage_counter from per-cpu to atomic mode. */
+ blk_freeze_queue_start(q);
+ /*
+ * Wait until atomic mode has been reached. Since that
+ * involves calling call_rcu(), it is guaranteed that later
+ * blk_queue_enter() calls see the pm-only state. See also
+ * http://lwn.net/Articles/573497/.
+ */
+ percpu_ref_switch_to_atomic_sync(&q->q_usage_counter);
+ if (percpu_ref_is_zero(&q->q_usage_counter))
+ ret = 0;
+ /* Switch q_usage_counter back to per-cpu mode. */
+ blk_mq_unfreeze_queue(q);
+
+ spin_lock_irq(q->queue_lock);
+ if (ret < 0)
+ pm_runtime_mark_last_busy(q->dev);
+ else
+ q->rpm_status = RPM_SUSPENDING;
+ spin_unlock_irq(q->queue_lock);
+
+ if (ret)
+ blk_clear_pm_only(q);
+
+ return ret;
+}
+EXPORT_SYMBOL(blk_pre_runtime_suspend);
+
+/**
+ * blk_post_runtime_suspend - Post runtime suspend processing
+ * @q: the queue of the device
+ * @err: return value of the device's runtime_suspend function
+ *
+ * Description:
+ * Update the queue's runtime status according to the return value of the
+ * device's runtime suspend function and mark last busy for the device so
+ * that PM core will try to auto suspend the device at a later time.
+ *
+ * This function should be called near the end of the device's
+ * runtime_suspend callback.
+ */
+void blk_post_runtime_suspend(struct request_queue *q, int err)
+{
+ if (!q->dev)
+ return;
+
+ spin_lock_irq(q->queue_lock);
+ if (!err) {
+ q->rpm_status = RPM_SUSPENDED;
+ } else {
+ q->rpm_status = RPM_ACTIVE;
+ pm_runtime_mark_last_busy(q->dev);
+ }
+ spin_unlock_irq(q->queue_lock);
+
+ if (err)
+ blk_clear_pm_only(q);
+}
+EXPORT_SYMBOL(blk_post_runtime_suspend);
+
+/**
+ * blk_pre_runtime_resume - Pre runtime resume processing
+ * @q: the queue of the device
+ *
+ * Description:
+ * Update the queue's runtime status to RESUMING in preparation for the
+ * runtime resume of the device.
+ *
+ * This function should be called near the start of the device's
+ * runtime_resume callback.
+ */
+void blk_pre_runtime_resume(struct request_queue *q)
+{
+ if (!q->dev)
+ return;
+
+ spin_lock_irq(q->queue_lock);
+ q->rpm_status = RPM_RESUMING;
+ spin_unlock_irq(q->queue_lock);
+}
+EXPORT_SYMBOL(blk_pre_runtime_resume);
+
+/**
+ * blk_post_runtime_resume - Post runtime resume processing
+ * @q: the queue of the device
+ * @err: return value of the device's runtime_resume function
+ *
+ * Description:
+ * Update the queue's runtime status according to the return value of the
+ * device's runtime_resume function. If it is successfully resumed, process
+ * the requests that are queued into the device's queue when it is resuming
+ * and then mark last busy and initiate autosuspend for it.
+ *
+ * This function should be called near the end of the device's
+ * runtime_resume callback.
+ */
+void blk_post_runtime_resume(struct request_queue *q, int err)
+{
+ if (!q->dev)
+ return;
+
+ spin_lock_irq(q->queue_lock);
+ if (!err) {
+ q->rpm_status = RPM_ACTIVE;
+ pm_runtime_mark_last_busy(q->dev);
+ pm_request_autosuspend(q->dev);
+ } else {
+ q->rpm_status = RPM_SUSPENDED;
+ }
+ spin_unlock_irq(q->queue_lock);
+
+ if (!err)
+ blk_clear_pm_only(q);
+}
+EXPORT_SYMBOL(blk_post_runtime_resume);
+
+/**
+ * blk_set_runtime_active - Force runtime status of the queue to be active
+ * @q: the queue of the device
+ *
+ * If the device is left runtime suspended during system suspend the resume
+ * hook typically resumes the device and corrects runtime status
+ * accordingly. However, that does not affect the queue runtime PM status
+ * which is still "suspended". This prevents processing requests from the
+ * queue.
+ *
+ * This function can be used in driver's resume hook to correct queue
+ * runtime PM status and re-enable peeking requests from the queue. It
+ * should be called before first request is added to the queue.
+ */
+void blk_set_runtime_active(struct request_queue *q)
+{
+ spin_lock_irq(q->queue_lock);
+ q->rpm_status = RPM_ACTIVE;
+ pm_runtime_mark_last_busy(q->dev);
+ pm_request_autosuspend(q->dev);
+ spin_unlock_irq(q->queue_lock);
+}
+EXPORT_SYMBOL(blk_set_runtime_active);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _BLOCK_BLK_PM_H_
+#define _BLOCK_BLK_PM_H_
+
+#include <linux/pm_runtime.h>
+
+#ifdef CONFIG_PM
+static inline void blk_pm_request_resume(struct request_queue *q)
+{
+ if (q->dev && (q->rpm_status == RPM_SUSPENDED ||
+ q->rpm_status == RPM_SUSPENDING))
+ pm_request_resume(q->dev);
+}
+
+static inline void blk_pm_mark_last_busy(struct request *rq)
+{
+ if (rq->q->dev && !(rq->rq_flags & RQF_PM))
+ pm_runtime_mark_last_busy(rq->q->dev);
+}
+
+static inline void blk_pm_requeue_request(struct request *rq)
+{
+ lockdep_assert_held(rq->q->queue_lock);
+
+ if (rq->q->dev && !(rq->rq_flags & RQF_PM))
+ rq->q->nr_pending--;
+}
+
+static inline void blk_pm_add_request(struct request_queue *q,
+ struct request *rq)
+{
+ lockdep_assert_held(q->queue_lock);
+
+ if (q->dev && !(rq->rq_flags & RQF_PM))
+ q->nr_pending++;
+}
+
+static inline void blk_pm_put_request(struct request *rq)
+{
+ lockdep_assert_held(rq->q->queue_lock);
+
+ if (rq->q->dev && !(rq->rq_flags & RQF_PM))
+ --rq->q->nr_pending;
+}
+#else
+static inline void blk_pm_request_resume(struct request_queue *q)
+{
+}
+
+static inline void blk_pm_mark_last_busy(struct request *rq)
+{
+}
+
+static inline void blk_pm_requeue_request(struct request *rq)
+{
+}
+
+static inline void blk_pm_add_request(struct request_queue *q,
+ struct request *rq)
+{
+}
+
+static inline void blk_pm_put_request(struct request *rq)
+{
+}
+#endif
+
+#endif /* _BLOCK_BLK_PM_H_ */
void __blk_complete_request(struct request *req)
{
- int ccpu, cpu;
struct request_queue *q = req->q;
+ int cpu, ccpu = q->mq_ops ? req->mq_ctx->cpu : req->cpu;
unsigned long flags;
bool shared = false;
/*
* Select completion CPU
*/
- if (req->cpu != -1) {
- ccpu = req->cpu;
+ if (test_bit(QUEUE_FLAG_SAME_COMP, &q->queue_flags) && ccpu != -1) {
if (!test_bit(QUEUE_FLAG_SAME_FORCE, &q->queue_flags))
shared = cpus_share_cache(cpu, ccpu);
} else
blk_queue_flag_set(QUEUE_FLAG_STATS, q);
spin_unlock(&q->stats->lock);
}
+EXPORT_SYMBOL_GPL(blk_stat_enable_accounting);
struct blk_queue_stats *blk_alloc_queue_stats(void)
{
* RB tree of active children throtl_grp's, which are sorted by
* their ->disptime.
*/
- struct rb_root pending_tree; /* RB tree of active tgs */
- struct rb_node *first_pending; /* first node in the tree */
+ struct rb_root_cached pending_tree; /* RB tree of active tgs */
unsigned int nr_pending; /* # queued in the tree */
unsigned long first_pending_disptime; /* disptime of the first tg */
struct timer_list pending_timer; /* fires on first_pending_disptime */
{
INIT_LIST_HEAD(&sq->queued[0]);
INIT_LIST_HEAD(&sq->queued[1]);
- sq->pending_tree = RB_ROOT;
+ sq->pending_tree = RB_ROOT_CACHED;
timer_setup(&sq->pending_timer, throtl_pending_timer_fn, 0);
}
static struct throtl_grp *
throtl_rb_first(struct throtl_service_queue *parent_sq)
{
+ struct rb_node *n;
/* Service tree is empty */
if (!parent_sq->nr_pending)
return NULL;
- if (!parent_sq->first_pending)
- parent_sq->first_pending = rb_first(&parent_sq->pending_tree);
-
- if (parent_sq->first_pending)
- return rb_entry_tg(parent_sq->first_pending);
-
- return NULL;
-}
-
-static void rb_erase_init(struct rb_node *n, struct rb_root *root)
-{
- rb_erase(n, root);
- RB_CLEAR_NODE(n);
+ n = rb_first_cached(&parent_sq->pending_tree);
+ WARN_ON_ONCE(!n);
+ if (!n)
+ return NULL;
+ return rb_entry_tg(n);
}
static void throtl_rb_erase(struct rb_node *n,
struct throtl_service_queue *parent_sq)
{
- if (parent_sq->first_pending == n)
- parent_sq->first_pending = NULL;
- rb_erase_init(n, &parent_sq->pending_tree);
+ rb_erase_cached(n, &parent_sq->pending_tree);
+ RB_CLEAR_NODE(n);
--parent_sq->nr_pending;
}
static void tg_service_queue_add(struct throtl_grp *tg)
{
struct throtl_service_queue *parent_sq = tg->service_queue.parent_sq;
- struct rb_node **node = &parent_sq->pending_tree.rb_node;
+ struct rb_node **node = &parent_sq->pending_tree.rb_root.rb_node;
struct rb_node *parent = NULL;
struct throtl_grp *__tg;
unsigned long key = tg->disptime;
- int left = 1;
+ bool leftmost = true;
while (*node != NULL) {
parent = *node;
node = &parent->rb_left;
else {
node = &parent->rb_right;
- left = 0;
+ leftmost = false;
}
}
- if (left)
- parent_sq->first_pending = &tg->rb_node;
-
rb_link_node(&tg->rb_node, parent, node);
- rb_insert_color(&tg->rb_node, &parent_sq->pending_tree);
+ rb_insert_color_cached(&tg->rb_node, &parent_sq->pending_tree,
+ leftmost);
}
static void __throtl_enqueue_tg(struct throtl_grp *tg)
}
#endif
-static void blk_throtl_assoc_bio(struct throtl_grp *tg, struct bio *bio)
-{
-#ifdef CONFIG_BLK_DEV_THROTTLING_LOW
- /* fallback to root_blkg if we fail to get a blkg ref */
- if (bio->bi_css && (bio_associate_blkg(bio, tg_to_blkg(tg)) == -ENODEV))
- bio_associate_blkg(bio, bio->bi_disk->queue->root_blkg);
- bio_issue_init(&bio->bi_issue, bio_sectors(bio));
-#endif
-}
-
bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
struct bio *bio)
{
struct throtl_qnode *qn = NULL;
- struct throtl_grp *tg = blkg_to_tg(blkg ?: q->root_blkg);
+ struct throtl_grp *tg = blkg_to_tg(blkg);
struct throtl_service_queue *sq;
bool rw = bio_data_dir(bio);
bool throttled = false;
if (unlikely(blk_queue_bypass(q)))
goto out_unlock;
- blk_throtl_assoc_bio(tg, bio);
blk_throtl_update_idletime(tg);
sq = &tg->service_queue;
rq_depth_scale_up(&rwb->rq_depth);
calc_wb_limits(rwb);
rwb->unknown_cnt = 0;
+ rwb_wake_all(rwb);
rwb_trace_step(rwb, "scale up");
}
rq_depth_scale_down(&rwb->rq_depth, hard_throttle);
calc_wb_limits(rwb);
rwb->unknown_cnt = 0;
- rwb_wake_all(rwb);
rwb_trace_step(rwb, "scale down");
}
#include <linux/idr.h>
#include <linux/blk-mq.h>
+#include <xen/xen.h>
#include "blk-mq.h"
/* Amount of time in which a process may batch requests */
}
struct blk_flush_queue *blk_alloc_flush_queue(struct request_queue *q,
- int node, int cmd_size);
+ int node, int cmd_size, gfp_t flags);
void blk_free_flush_queue(struct blk_flush_queue *q);
int blk_init_rl(struct request_list *rl, struct request_queue *q,
percpu_ref_get(&q->q_usage_counter);
}
+static inline bool biovec_phys_mergeable(struct request_queue *q,
+ struct bio_vec *vec1, struct bio_vec *vec2)
+{
+ unsigned long mask = queue_segment_boundary(q);
+ phys_addr_t addr1 = page_to_phys(vec1->bv_page) + vec1->bv_offset;
+ phys_addr_t addr2 = page_to_phys(vec2->bv_page) + vec2->bv_offset;
+
+ if (addr1 + vec1->bv_len != addr2)
+ return false;
+ if (xen_domain() && !xen_biovec_phys_mergeable(vec1, vec2))
+ return false;
+ if ((addr1 | mask) != ((addr2 + vec2->bv_len - 1) | mask))
+ return false;
+ return true;
+}
+
+static inline bool __bvec_gap_to_prev(struct request_queue *q,
+ struct bio_vec *bprv, unsigned int offset)
+{
+ return offset ||
+ ((bprv->bv_offset + bprv->bv_len) & queue_virt_boundary(q));
+}
+
+/*
+ * Check if adding a bio_vec after bprv with offset would create a gap in
+ * the SG list. Most drivers don't care about this, but some do.
+ */
+static inline bool bvec_gap_to_prev(struct request_queue *q,
+ struct bio_vec *bprv, unsigned int offset)
+{
+ if (!queue_virt_boundary(q))
+ return false;
+ return __bvec_gap_to_prev(q, bprv, offset);
+}
+
#ifdef CONFIG_BLK_DEV_INTEGRITY
void blk_flush_integrity(void);
bool __bio_integrity_endio(struct bio *);
return __bio_integrity_endio(bio);
return true;
}
-#else
+
+static inline bool integrity_req_gap_back_merge(struct request *req,
+ struct bio *next)
+{
+ struct bio_integrity_payload *bip = bio_integrity(req->bio);
+ struct bio_integrity_payload *bip_next = bio_integrity(next);
+
+ return bvec_gap_to_prev(req->q, &bip->bip_vec[bip->bip_vcnt - 1],
+ bip_next->bip_vec[0].bv_offset);
+}
+
+static inline bool integrity_req_gap_front_merge(struct request *req,
+ struct bio *bio)
+{
+ struct bio_integrity_payload *bip = bio_integrity(bio);
+ struct bio_integrity_payload *bip_next = bio_integrity(req->bio);
+
+ return bvec_gap_to_prev(req->q, &bip->bip_vec[bip->bip_vcnt - 1],
+ bip_next->bip_vec[0].bv_offset);
+}
+#else /* CONFIG_BLK_DEV_INTEGRITY */
+static inline bool integrity_req_gap_back_merge(struct request *req,
+ struct bio *next)
+{
+ return false;
+}
+static inline bool integrity_req_gap_front_merge(struct request *req,
+ struct bio *bio)
+{
+ return false;
+}
+
static inline void blk_flush_integrity(void)
{
}
{
return true;
}
-#endif
+#endif /* CONFIG_BLK_DEV_INTEGRITY */
void blk_timeout_work(struct work_struct *work);
unsigned long blk_rq_timeout(unsigned long timeout);
static struct bio_set bounce_bio_set, bounce_bio_split;
static mempool_t page_pool, isa_page_pool;
+static void init_bounce_bioset(void)
+{
+ static bool bounce_bs_setup;
+ int ret;
+
+ if (bounce_bs_setup)
+ return;
+
+ ret = bioset_init(&bounce_bio_set, BIO_POOL_SIZE, 0, BIOSET_NEED_BVECS);
+ BUG_ON(ret);
+ if (bioset_integrity_create(&bounce_bio_set, BIO_POOL_SIZE))
+ BUG_ON(1);
+
+ ret = bioset_init(&bounce_bio_split, BIO_POOL_SIZE, 0, 0);
+ BUG_ON(ret);
+ bounce_bs_setup = true;
+}
+
#if defined(CONFIG_HIGHMEM)
static __init int init_emergency_pool(void)
{
BUG_ON(ret);
pr_info("pool size: %d pages\n", POOL_SIZE);
- ret = bioset_init(&bounce_bio_set, BIO_POOL_SIZE, 0, BIOSET_NEED_BVECS);
- BUG_ON(ret);
- if (bioset_integrity_create(&bounce_bio_set, BIO_POOL_SIZE))
- BUG_ON(1);
-
- ret = bioset_init(&bounce_bio_split, BIO_POOL_SIZE, 0, 0);
- BUG_ON(ret);
-
+ init_bounce_bioset();
return 0;
}
return mempool_alloc_pages(gfp_mask | GFP_DMA, data);
}
+static DEFINE_MUTEX(isa_mutex);
+
/*
* gets called "every" time someone init's a queue with BLK_BOUNCE_ISA
* as the max address, so check if the pool has already been created.
{
int ret;
- if (mempool_initialized(&isa_page_pool))
+ mutex_lock(&isa_mutex);
+
+ if (mempool_initialized(&isa_page_pool)) {
+ mutex_unlock(&isa_mutex);
return 0;
+ }
ret = mempool_init(&isa_page_pool, ISA_POOL_SIZE, mempool_alloc_pages_isa,
mempool_free_pages, (void *) 0);
BUG_ON(ret);
pr_info("isa pool size: %d pages\n", ISA_POOL_SIZE);
+ init_bounce_bioset();
+ mutex_unlock(&isa_mutex);
return 0;
}
}
}
- bio_clone_blkcg_association(bio, bio_src);
+ bio_clone_blkg_association(bio, bio_src);
+
+ blkcg_bio_issue_init(bio);
return bio;
}
int i;
for (i = 0; i < IOPRIO_BE_NR; i++) {
- if (cfqg->async_cfqq[0][i])
+ if (cfqg->async_cfqq[0][i]) {
cfq_put_queue(cfqg->async_cfqq[0][i]);
- if (cfqg->async_cfqq[1][i])
+ cfqg->async_cfqq[0][i] = NULL;
+ }
+ if (cfqg->async_cfqq[1][i]) {
cfq_put_queue(cfqg->async_cfqq[1][i]);
+ cfqg->async_cfqq[1][i] = NULL;
+ }
}
- if (cfqg->async_idle_cfqq)
+ if (cfqg->async_idle_cfqq) {
cfq_put_queue(cfqg->async_idle_cfqq);
+ cfqg->async_idle_cfqq = NULL;
+ }
/*
* @blkg is going offline and will be ignored by
uint64_t serial_nr;
rcu_read_lock();
- serial_nr = bio_blkcg(bio)->css.serial_nr;
+ serial_nr = __bio_blkcg(bio)->css.serial_nr;
rcu_read_unlock();
/*
struct cfq_group *cfqg;
rcu_read_lock();
- cfqg = cfq_lookup_cfqg(cfqd, bio_blkcg(bio));
+ cfqg = cfq_lookup_cfqg(cfqd, __bio_blkcg(bio));
if (!cfqg) {
cfqq = &cfqd->oom_cfqq;
goto out;
#include "blk.h"
#include "blk-mq-sched.h"
+#include "blk-pm.h"
#include "blk-wbt.h"
static DEFINE_SPINLOCK(elv_list_lock);
e->type->ops.sq.elevator_bio_merged_fn(q, rq, bio);
}
-#ifdef CONFIG_PM
-static void blk_pm_requeue_request(struct request *rq)
-{
- if (rq->q->dev && !(rq->rq_flags & RQF_PM))
- rq->q->nr_pending--;
-}
-
-static void blk_pm_add_request(struct request_queue *q, struct request *rq)
-{
- if (q->dev && !(rq->rq_flags & RQF_PM) && q->nr_pending++ == 0 &&
- (q->rpm_status == RPM_SUSPENDED || q->rpm_status == RPM_SUSPENDING))
- pm_request_resume(q->dev);
-}
-#else
-static inline void blk_pm_requeue_request(struct request *rq) {}
-static inline void blk_pm_add_request(struct request_queue *q,
- struct request *rq)
-{
-}
-#endif
-
void elv_requeue_request(struct request_queue *q, struct request *rq)
{
/*
return 0;
}
-static void register_disk(struct device *parent, struct gendisk *disk)
+static void register_disk(struct device *parent, struct gendisk *disk,
+ const struct attribute_group **groups)
{
struct device *ddev = disk_to_dev(disk);
struct block_device *bdev;
/* delay uevents, until we scanned partition table */
dev_set_uevent_suppress(ddev, 1);
+ if (groups) {
+ WARN_ON(ddev->groups);
+ ddev->groups = groups;
+ }
if (device_add(ddev))
return;
if (!sysfs_deprecated) {
* __device_add_disk - add disk information to kernel list
* @parent: parent device for the disk
* @disk: per-device partitioning information
+ * @groups: Additional per-device sysfs groups
* @register_queue: register the queue if set to true
*
* This function registers the partitioning information in @disk
* FIXME: error handling
*/
static void __device_add_disk(struct device *parent, struct gendisk *disk,
+ const struct attribute_group **groups,
bool register_queue)
{
dev_t devt;
blk_register_region(disk_devt(disk), disk->minors, NULL,
exact_match, exact_lock, disk);
}
- register_disk(parent, disk);
+ register_disk(parent, disk, groups);
if (register_queue)
blk_register_queue(disk);
blk_integrity_add(disk);
}
-void device_add_disk(struct device *parent, struct gendisk *disk)
+void device_add_disk(struct device *parent, struct gendisk *disk,
+ const struct attribute_group **groups)
+
{
- __device_add_disk(parent, disk, true);
+ __device_add_disk(parent, disk, groups, true);
}
EXPORT_SYMBOL(device_add_disk);
void device_add_disk_no_queue_reg(struct device *parent, struct gendisk *disk)
{
- __device_add_disk(parent, disk, false);
+ __device_add_disk(parent, disk, NULL, false);
}
EXPORT_SYMBOL(device_add_disk_no_queue_reg);
#include "blk-mq-debugfs.h"
#include "blk-mq-sched.h"
#include "blk-mq-tag.h"
-#include "blk-stat.h"
-/* Scheduling domains. */
+#define CREATE_TRACE_POINTS
+#include <trace/events/kyber.h>
+
+/*
+ * Scheduling domains: the device is divided into multiple domains based on the
+ * request type.
+ */
enum {
KYBER_READ,
- KYBER_SYNC_WRITE,
- KYBER_OTHER, /* Async writes, discard, etc. */
+ KYBER_WRITE,
+ KYBER_DISCARD,
+ KYBER_OTHER,
KYBER_NUM_DOMAINS,
};
-enum {
- KYBER_MIN_DEPTH = 256,
+static const char *kyber_domain_names[] = {
+ [KYBER_READ] = "READ",
+ [KYBER_WRITE] = "WRITE",
+ [KYBER_DISCARD] = "DISCARD",
+ [KYBER_OTHER] = "OTHER",
+};
+enum {
/*
* In order to prevent starvation of synchronous requests by a flood of
* asynchronous requests, we reserve 25% of requests for synchronous
};
/*
- * Initial device-wide depths for each scheduling domain.
+ * Maximum device-wide depth for each scheduling domain.
*
- * Even for fast devices with lots of tags like NVMe, you can saturate
- * the device with only a fraction of the maximum possible queue depth.
- * So, we cap these to a reasonable value.
+ * Even for fast devices with lots of tags like NVMe, you can saturate the
+ * device with only a fraction of the maximum possible queue depth. So, we cap
+ * these to a reasonable value.
*/
static const unsigned int kyber_depth[] = {
[KYBER_READ] = 256,
- [KYBER_SYNC_WRITE] = 128,
- [KYBER_OTHER] = 64,
+ [KYBER_WRITE] = 128,
+ [KYBER_DISCARD] = 64,
+ [KYBER_OTHER] = 16,
};
/*
- * Scheduling domain batch sizes. We favor reads.
+ * Default latency targets for each scheduling domain.
+ */
+static const u64 kyber_latency_targets[] = {
+ [KYBER_READ] = 2ULL * NSEC_PER_MSEC,
+ [KYBER_WRITE] = 10ULL * NSEC_PER_MSEC,
+ [KYBER_DISCARD] = 5ULL * NSEC_PER_SEC,
+};
+
+/*
+ * Batch size (number of requests we'll dispatch in a row) for each scheduling
+ * domain.
*/
static const unsigned int kyber_batch_size[] = {
[KYBER_READ] = 16,
- [KYBER_SYNC_WRITE] = 8,
- [KYBER_OTHER] = 8,
+ [KYBER_WRITE] = 8,
+ [KYBER_DISCARD] = 1,
+ [KYBER_OTHER] = 1,
+};
+
+/*
+ * Requests latencies are recorded in a histogram with buckets defined relative
+ * to the target latency:
+ *
+ * <= 1/4 * target latency
+ * <= 1/2 * target latency
+ * <= 3/4 * target latency
+ * <= target latency
+ * <= 1 1/4 * target latency
+ * <= 1 1/2 * target latency
+ * <= 1 3/4 * target latency
+ * > 1 3/4 * target latency
+ */
+enum {
+ /*
+ * The width of the latency histogram buckets is
+ * 1 / (1 << KYBER_LATENCY_SHIFT) * target latency.
+ */
+ KYBER_LATENCY_SHIFT = 2,
+ /*
+ * The first (1 << KYBER_LATENCY_SHIFT) buckets are <= target latency,
+ * thus, "good".
+ */
+ KYBER_GOOD_BUCKETS = 1 << KYBER_LATENCY_SHIFT,
+ /* There are also (1 << KYBER_LATENCY_SHIFT) "bad" buckets. */
+ KYBER_LATENCY_BUCKETS = 2 << KYBER_LATENCY_SHIFT,
+};
+
+/*
+ * We measure both the total latency and the I/O latency (i.e., latency after
+ * submitting to the device).
+ */
+enum {
+ KYBER_TOTAL_LATENCY,
+ KYBER_IO_LATENCY,
+};
+
+static const char *kyber_latency_type_names[] = {
+ [KYBER_TOTAL_LATENCY] = "total",
+ [KYBER_IO_LATENCY] = "I/O",
+};
+
+/*
+ * Per-cpu latency histograms: total latency and I/O latency for each scheduling
+ * domain except for KYBER_OTHER.
+ */
+struct kyber_cpu_latency {
+ atomic_t buckets[KYBER_OTHER][2][KYBER_LATENCY_BUCKETS];
};
/*
struct kyber_queue_data {
struct request_queue *q;
- struct blk_stat_callback *cb;
-
/*
- * The device is divided into multiple scheduling domains based on the
- * request type. Each domain has a fixed number of in-flight requests of
- * that type device-wide, limited by these tokens.
+ * Each scheduling domain has a limited number of in-flight requests
+ * device-wide, limited by these tokens.
*/
struct sbitmap_queue domain_tokens[KYBER_NUM_DOMAINS];
*/
unsigned int async_depth;
+ struct kyber_cpu_latency __percpu *cpu_latency;
+
+ /* Timer for stats aggregation and adjusting domain tokens. */
+ struct timer_list timer;
+
+ unsigned int latency_buckets[KYBER_OTHER][2][KYBER_LATENCY_BUCKETS];
+
+ unsigned long latency_timeout[KYBER_OTHER];
+
+ int domain_p99[KYBER_OTHER];
+
/* Target latencies in nanoseconds. */
- u64 read_lat_nsec, write_lat_nsec;
+ u64 latency_targets[KYBER_OTHER];
};
struct kyber_hctx_data {
static unsigned int kyber_sched_domain(unsigned int op)
{
- if ((op & REQ_OP_MASK) == REQ_OP_READ)
+ switch (op & REQ_OP_MASK) {
+ case REQ_OP_READ:
return KYBER_READ;
- else if ((op & REQ_OP_MASK) == REQ_OP_WRITE && op_is_sync(op))
- return KYBER_SYNC_WRITE;
- else
+ case REQ_OP_WRITE:
+ return KYBER_WRITE;
+ case REQ_OP_DISCARD:
+ return KYBER_DISCARD;
+ default:
return KYBER_OTHER;
+ }
}
-enum {
- NONE = 0,
- GOOD = 1,
- GREAT = 2,
- BAD = -1,
- AWFUL = -2,
-};
-
-#define IS_GOOD(status) ((status) > 0)
-#define IS_BAD(status) ((status) < 0)
-
-static int kyber_lat_status(struct blk_stat_callback *cb,
- unsigned int sched_domain, u64 target)
+static void flush_latency_buckets(struct kyber_queue_data *kqd,
+ struct kyber_cpu_latency *cpu_latency,
+ unsigned int sched_domain, unsigned int type)
{
- u64 latency;
-
- if (!cb->stat[sched_domain].nr_samples)
- return NONE;
+ unsigned int *buckets = kqd->latency_buckets[sched_domain][type];
+ atomic_t *cpu_buckets = cpu_latency->buckets[sched_domain][type];
+ unsigned int bucket;
- latency = cb->stat[sched_domain].mean;
- if (latency >= 2 * target)
- return AWFUL;
- else if (latency > target)
- return BAD;
- else if (latency <= target / 2)
- return GREAT;
- else /* (latency <= target) */
- return GOOD;
+ for (bucket = 0; bucket < KYBER_LATENCY_BUCKETS; bucket++)
+ buckets[bucket] += atomic_xchg(&cpu_buckets[bucket], 0);
}
/*
- * Adjust the read or synchronous write depth given the status of reads and
- * writes. The goal is that the latencies of the two domains are fair (i.e., if
- * one is good, then the other is good).
+ * Calculate the histogram bucket with the given percentile rank, or -1 if there
+ * aren't enough samples yet.
*/
-static void kyber_adjust_rw_depth(struct kyber_queue_data *kqd,
- unsigned int sched_domain, int this_status,
- int other_status)
+static int calculate_percentile(struct kyber_queue_data *kqd,
+ unsigned int sched_domain, unsigned int type,
+ unsigned int percentile)
{
- unsigned int orig_depth, depth;
+ unsigned int *buckets = kqd->latency_buckets[sched_domain][type];
+ unsigned int bucket, samples = 0, percentile_samples;
+
+ for (bucket = 0; bucket < KYBER_LATENCY_BUCKETS; bucket++)
+ samples += buckets[bucket];
+
+ if (!samples)
+ return -1;
/*
- * If this domain had no samples, or reads and writes are both good or
- * both bad, don't adjust the depth.
+ * We do the calculation once we have 500 samples or one second passes
+ * since the first sample was recorded, whichever comes first.
*/
- if (this_status == NONE ||
- (IS_GOOD(this_status) && IS_GOOD(other_status)) ||
- (IS_BAD(this_status) && IS_BAD(other_status)))
- return;
-
- orig_depth = depth = kqd->domain_tokens[sched_domain].sb.depth;
+ if (!kqd->latency_timeout[sched_domain])
+ kqd->latency_timeout[sched_domain] = max(jiffies + HZ, 1UL);
+ if (samples < 500 &&
+ time_is_after_jiffies(kqd->latency_timeout[sched_domain])) {
+ return -1;
+ }
+ kqd->latency_timeout[sched_domain] = 0;
- if (other_status == NONE) {
- depth++;
- } else {
- switch (this_status) {
- case GOOD:
- if (other_status == AWFUL)
- depth -= max(depth / 4, 1U);
- else
- depth -= max(depth / 8, 1U);
- break;
- case GREAT:
- if (other_status == AWFUL)
- depth /= 2;
- else
- depth -= max(depth / 4, 1U);
+ percentile_samples = DIV_ROUND_UP(samples * percentile, 100);
+ for (bucket = 0; bucket < KYBER_LATENCY_BUCKETS - 1; bucket++) {
+ if (buckets[bucket] >= percentile_samples)
break;
- case BAD:
- depth++;
- break;
- case AWFUL:
- if (other_status == GREAT)
- depth += 2;
- else
- depth++;
- break;
- }
+ percentile_samples -= buckets[bucket];
}
+ memset(buckets, 0, sizeof(kqd->latency_buckets[sched_domain][type]));
- depth = clamp(depth, 1U, kyber_depth[sched_domain]);
- if (depth != orig_depth)
- sbitmap_queue_resize(&kqd->domain_tokens[sched_domain], depth);
+ trace_kyber_latency(kqd->q, kyber_domain_names[sched_domain],
+ kyber_latency_type_names[type], percentile,
+ bucket + 1, 1 << KYBER_LATENCY_SHIFT, samples);
+
+ return bucket;
}
-/*
- * Adjust the depth of other requests given the status of reads and synchronous
- * writes. As long as either domain is doing fine, we don't throttle, but if
- * both domains are doing badly, we throttle heavily.
- */
-static void kyber_adjust_other_depth(struct kyber_queue_data *kqd,
- int read_status, int write_status,
- bool have_samples)
-{
- unsigned int orig_depth, depth;
- int status;
-
- orig_depth = depth = kqd->domain_tokens[KYBER_OTHER].sb.depth;
-
- if (read_status == NONE && write_status == NONE) {
- depth += 2;
- } else if (have_samples) {
- if (read_status == NONE)
- status = write_status;
- else if (write_status == NONE)
- status = read_status;
- else
- status = max(read_status, write_status);
- switch (status) {
- case GREAT:
- depth += 2;
- break;
- case GOOD:
- depth++;
- break;
- case BAD:
- depth -= max(depth / 4, 1U);
- break;
- case AWFUL:
- depth /= 2;
- break;
- }
+static void kyber_resize_domain(struct kyber_queue_data *kqd,
+ unsigned int sched_domain, unsigned int depth)
+{
+ depth = clamp(depth, 1U, kyber_depth[sched_domain]);
+ if (depth != kqd->domain_tokens[sched_domain].sb.depth) {
+ sbitmap_queue_resize(&kqd->domain_tokens[sched_domain], depth);
+ trace_kyber_adjust(kqd->q, kyber_domain_names[sched_domain],
+ depth);
}
-
- depth = clamp(depth, 1U, kyber_depth[KYBER_OTHER]);
- if (depth != orig_depth)
- sbitmap_queue_resize(&kqd->domain_tokens[KYBER_OTHER], depth);
}
-/*
- * Apply heuristics for limiting queue depths based on gathered latency
- * statistics.
- */
-static void kyber_stat_timer_fn(struct blk_stat_callback *cb)
+static void kyber_timer_fn(struct timer_list *t)
{
- struct kyber_queue_data *kqd = cb->data;
- int read_status, write_status;
+ struct kyber_queue_data *kqd = from_timer(kqd, t, timer);
+ unsigned int sched_domain;
+ int cpu;
+ bool bad = false;
+
+ /* Sum all of the per-cpu latency histograms. */
+ for_each_online_cpu(cpu) {
+ struct kyber_cpu_latency *cpu_latency;
+
+ cpu_latency = per_cpu_ptr(kqd->cpu_latency, cpu);
+ for (sched_domain = 0; sched_domain < KYBER_OTHER; sched_domain++) {
+ flush_latency_buckets(kqd, cpu_latency, sched_domain,
+ KYBER_TOTAL_LATENCY);
+ flush_latency_buckets(kqd, cpu_latency, sched_domain,
+ KYBER_IO_LATENCY);
+ }
+ }
- read_status = kyber_lat_status(cb, KYBER_READ, kqd->read_lat_nsec);
- write_status = kyber_lat_status(cb, KYBER_SYNC_WRITE, kqd->write_lat_nsec);
+ /*
+ * Check if any domains have a high I/O latency, which might indicate
+ * congestion in the device. Note that we use the p90; we don't want to
+ * be too sensitive to outliers here.
+ */
+ for (sched_domain = 0; sched_domain < KYBER_OTHER; sched_domain++) {
+ int p90;
- kyber_adjust_rw_depth(kqd, KYBER_READ, read_status, write_status);
- kyber_adjust_rw_depth(kqd, KYBER_SYNC_WRITE, write_status, read_status);
- kyber_adjust_other_depth(kqd, read_status, write_status,
- cb->stat[KYBER_OTHER].nr_samples != 0);
+ p90 = calculate_percentile(kqd, sched_domain, KYBER_IO_LATENCY,
+ 90);
+ if (p90 >= KYBER_GOOD_BUCKETS)
+ bad = true;
+ }
/*
- * Continue monitoring latencies if we aren't hitting the targets or
- * we're still throttling other requests.
+ * Adjust the scheduling domain depths. If we determined that there was
+ * congestion, we throttle all domains with good latencies. Either way,
+ * we ease up on throttling domains with bad latencies.
*/
- if (!blk_stat_is_active(kqd->cb) &&
- ((IS_BAD(read_status) || IS_BAD(write_status) ||
- kqd->domain_tokens[KYBER_OTHER].sb.depth < kyber_depth[KYBER_OTHER])))
- blk_stat_activate_msecs(kqd->cb, 100);
+ for (sched_domain = 0; sched_domain < KYBER_OTHER; sched_domain++) {
+ unsigned int orig_depth, depth;
+ int p99;
+
+ p99 = calculate_percentile(kqd, sched_domain,
+ KYBER_TOTAL_LATENCY, 99);
+ /*
+ * This is kind of subtle: different domains will not
+ * necessarily have enough samples to calculate the latency
+ * percentiles during the same window, so we have to remember
+ * the p99 for the next time we observe congestion; once we do,
+ * we don't want to throttle again until we get more data, so we
+ * reset it to -1.
+ */
+ if (bad) {
+ if (p99 < 0)
+ p99 = kqd->domain_p99[sched_domain];
+ kqd->domain_p99[sched_domain] = -1;
+ } else if (p99 >= 0) {
+ kqd->domain_p99[sched_domain] = p99;
+ }
+ if (p99 < 0)
+ continue;
+
+ /*
+ * If this domain has bad latency, throttle less. Otherwise,
+ * throttle more iff we determined that there is congestion.
+ *
+ * The new depth is scaled linearly with the p99 latency vs the
+ * latency target. E.g., if the p99 is 3/4 of the target, then
+ * we throttle down to 3/4 of the current depth, and if the p99
+ * is 2x the target, then we double the depth.
+ */
+ if (bad || p99 >= KYBER_GOOD_BUCKETS) {
+ orig_depth = kqd->domain_tokens[sched_domain].sb.depth;
+ depth = (orig_depth * (p99 + 1)) >> KYBER_LATENCY_SHIFT;
+ kyber_resize_domain(kqd, sched_domain, depth);
+ }
+ }
}
-static unsigned int kyber_sched_tags_shift(struct kyber_queue_data *kqd)
+static unsigned int kyber_sched_tags_shift(struct request_queue *q)
{
/*
* All of the hardware queues have the same depth, so we can just grab
* the shift of the first one.
*/
- return kqd->q->queue_hw_ctx[0]->sched_tags->bitmap_tags.sb.shift;
-}
-
-static int kyber_bucket_fn(const struct request *rq)
-{
- return kyber_sched_domain(rq->cmd_flags);
+ return q->queue_hw_ctx[0]->sched_tags->bitmap_tags.sb.shift;
}
static struct kyber_queue_data *kyber_queue_data_alloc(struct request_queue *q)
{
struct kyber_queue_data *kqd;
- unsigned int max_tokens;
unsigned int shift;
int ret = -ENOMEM;
int i;
- kqd = kmalloc_node(sizeof(*kqd), GFP_KERNEL, q->node);
+ kqd = kzalloc_node(sizeof(*kqd), GFP_KERNEL, q->node);
if (!kqd)
goto err;
+
kqd->q = q;
- kqd->cb = blk_stat_alloc_callback(kyber_stat_timer_fn, kyber_bucket_fn,
- KYBER_NUM_DOMAINS, kqd);
- if (!kqd->cb)
+ kqd->cpu_latency = alloc_percpu_gfp(struct kyber_cpu_latency,
+ GFP_KERNEL | __GFP_ZERO);
+ if (!kqd->cpu_latency)
goto err_kqd;
- /*
- * The maximum number of tokens for any scheduling domain is at least
- * the queue depth of a single hardware queue. If the hardware doesn't
- * have many tags, still provide a reasonable number.
- */
- max_tokens = max_t(unsigned int, q->tag_set->queue_depth,
- KYBER_MIN_DEPTH);
+ timer_setup(&kqd->timer, kyber_timer_fn, 0);
+
for (i = 0; i < KYBER_NUM_DOMAINS; i++) {
WARN_ON(!kyber_depth[i]);
WARN_ON(!kyber_batch_size[i]);
ret = sbitmap_queue_init_node(&kqd->domain_tokens[i],
- max_tokens, -1, false, GFP_KERNEL,
- q->node);
+ kyber_depth[i], -1, false,
+ GFP_KERNEL, q->node);
if (ret) {
while (--i >= 0)
sbitmap_queue_free(&kqd->domain_tokens[i]);
- goto err_cb;
+ goto err_buckets;
}
- sbitmap_queue_resize(&kqd->domain_tokens[i], kyber_depth[i]);
}
- shift = kyber_sched_tags_shift(kqd);
- kqd->async_depth = (1U << shift) * KYBER_ASYNC_PERCENT / 100U;
+ for (i = 0; i < KYBER_OTHER; i++) {
+ kqd->domain_p99[i] = -1;
+ kqd->latency_targets[i] = kyber_latency_targets[i];
+ }
- kqd->read_lat_nsec = 2000000ULL;
- kqd->write_lat_nsec = 10000000ULL;
+ shift = kyber_sched_tags_shift(q);
+ kqd->async_depth = (1U << shift) * KYBER_ASYNC_PERCENT / 100U;
return kqd;
-err_cb:
- blk_stat_free_callback(kqd->cb);
+err_buckets:
+ free_percpu(kqd->cpu_latency);
err_kqd:
kfree(kqd);
err:
return PTR_ERR(kqd);
}
+ blk_stat_enable_accounting(q);
+
eq->elevator_data = kqd;
q->elevator = eq;
- blk_stat_add_callback(q, kqd->cb);
-
return 0;
}
static void kyber_exit_sched(struct elevator_queue *e)
{
struct kyber_queue_data *kqd = e->elevator_data;
- struct request_queue *q = kqd->q;
int i;
- blk_stat_remove_callback(q, kqd->cb);
+ del_timer_sync(&kqd->timer);
for (i = 0; i < KYBER_NUM_DOMAINS; i++)
sbitmap_queue_free(&kqd->domain_tokens[i]);
- blk_stat_free_callback(kqd->cb);
+ free_percpu(kqd->cpu_latency);
kfree(kqd);
}
rq_clear_domain_token(kqd, rq);
}
-static void kyber_completed_request(struct request *rq)
+static void add_latency_sample(struct kyber_cpu_latency *cpu_latency,
+ unsigned int sched_domain, unsigned int type,
+ u64 target, u64 latency)
{
- struct request_queue *q = rq->q;
- struct kyber_queue_data *kqd = q->elevator->elevator_data;
- unsigned int sched_domain;
- u64 now, latency, target;
+ unsigned int bucket;
+ u64 divisor;
- /*
- * Check if this request met our latency goal. If not, quickly gather
- * some statistics and start throttling.
- */
- sched_domain = kyber_sched_domain(rq->cmd_flags);
- switch (sched_domain) {
- case KYBER_READ:
- target = kqd->read_lat_nsec;
- break;
- case KYBER_SYNC_WRITE:
- target = kqd->write_lat_nsec;
- break;
- default:
- return;
+ if (latency > 0) {
+ divisor = max_t(u64, target >> KYBER_LATENCY_SHIFT, 1);
+ bucket = min_t(unsigned int, div64_u64(latency - 1, divisor),
+ KYBER_LATENCY_BUCKETS - 1);
+ } else {
+ bucket = 0;
}
- /* If we are already monitoring latencies, don't check again. */
- if (blk_stat_is_active(kqd->cb))
- return;
+ atomic_inc(&cpu_latency->buckets[sched_domain][type][bucket]);
+}
- now = ktime_get_ns();
- if (now < rq->io_start_time_ns)
+static void kyber_completed_request(struct request *rq, u64 now)
+{
+ struct kyber_queue_data *kqd = rq->q->elevator->elevator_data;
+ struct kyber_cpu_latency *cpu_latency;
+ unsigned int sched_domain;
+ u64 target;
+
+ sched_domain = kyber_sched_domain(rq->cmd_flags);
+ if (sched_domain == KYBER_OTHER)
return;
- latency = now - rq->io_start_time_ns;
+ cpu_latency = get_cpu_ptr(kqd->cpu_latency);
+ target = kqd->latency_targets[sched_domain];
+ add_latency_sample(cpu_latency, sched_domain, KYBER_TOTAL_LATENCY,
+ target, now - rq->start_time_ns);
+ add_latency_sample(cpu_latency, sched_domain, KYBER_IO_LATENCY, target,
+ now - rq->io_start_time_ns);
+ put_cpu_ptr(kqd->cpu_latency);
- if (latency > target)
- blk_stat_activate_msecs(kqd->cb, 10);
+ timer_reduce(&kqd->timer, jiffies + HZ / 10);
}
struct flush_kcq_data {
rq_set_domain_token(rq, nr);
list_del_init(&rq->queuelist);
return rq;
+ } else {
+ trace_kyber_throttled(kqd->q,
+ kyber_domain_names[khd->cur_domain]);
}
} else if (sbitmap_any_bit_set(&khd->kcq_map[khd->cur_domain])) {
nr = kyber_get_domain_token(kqd, khd, hctx);
rq_set_domain_token(rq, nr);
list_del_init(&rq->queuelist);
return rq;
+ } else {
+ trace_kyber_throttled(kqd->q,
+ kyber_domain_names[khd->cur_domain]);
}
}
return false;
}
-#define KYBER_LAT_SHOW_STORE(op) \
-static ssize_t kyber_##op##_lat_show(struct elevator_queue *e, \
- char *page) \
+#define KYBER_LAT_SHOW_STORE(domain, name) \
+static ssize_t kyber_##name##_lat_show(struct elevator_queue *e, \
+ char *page) \
{ \
struct kyber_queue_data *kqd = e->elevator_data; \
\
- return sprintf(page, "%llu\n", kqd->op##_lat_nsec); \
+ return sprintf(page, "%llu\n", kqd->latency_targets[domain]); \
} \
\
-static ssize_t kyber_##op##_lat_store(struct elevator_queue *e, \
- const char *page, size_t count) \
+static ssize_t kyber_##name##_lat_store(struct elevator_queue *e, \
+ const char *page, size_t count) \
{ \
struct kyber_queue_data *kqd = e->elevator_data; \
unsigned long long nsec; \
if (ret) \
return ret; \
\
- kqd->op##_lat_nsec = nsec; \
+ kqd->latency_targets[domain] = nsec; \
\
return count; \
}
-KYBER_LAT_SHOW_STORE(read);
-KYBER_LAT_SHOW_STORE(write);
+KYBER_LAT_SHOW_STORE(KYBER_READ, read);
+KYBER_LAT_SHOW_STORE(KYBER_WRITE, write);
#undef KYBER_LAT_SHOW_STORE
#define KYBER_LAT_ATTR(op) __ATTR(op##_lat_nsec, 0644, kyber_##op##_lat_show, kyber_##op##_lat_store)
return 0; \
}
KYBER_DEBUGFS_DOMAIN_ATTRS(KYBER_READ, read)
-KYBER_DEBUGFS_DOMAIN_ATTRS(KYBER_SYNC_WRITE, sync_write)
+KYBER_DEBUGFS_DOMAIN_ATTRS(KYBER_WRITE, write)
+KYBER_DEBUGFS_DOMAIN_ATTRS(KYBER_DISCARD, discard)
KYBER_DEBUGFS_DOMAIN_ATTRS(KYBER_OTHER, other)
#undef KYBER_DEBUGFS_DOMAIN_ATTRS
struct blk_mq_hw_ctx *hctx = data;
struct kyber_hctx_data *khd = hctx->sched_data;
- switch (khd->cur_domain) {
- case KYBER_READ:
- seq_puts(m, "READ\n");
- break;
- case KYBER_SYNC_WRITE:
- seq_puts(m, "SYNC_WRITE\n");
- break;
- case KYBER_OTHER:
- seq_puts(m, "OTHER\n");
- break;
- default:
- seq_printf(m, "%u\n", khd->cur_domain);
- break;
- }
+ seq_printf(m, "%s\n", kyber_domain_names[khd->cur_domain]);
return 0;
}
{#name "_tokens", 0400, kyber_##name##_tokens_show}
static const struct blk_mq_debugfs_attr kyber_queue_debugfs_attrs[] = {
KYBER_QUEUE_DOMAIN_ATTRS(read),
- KYBER_QUEUE_DOMAIN_ATTRS(sync_write),
+ KYBER_QUEUE_DOMAIN_ATTRS(write),
+ KYBER_QUEUE_DOMAIN_ATTRS(discard),
KYBER_QUEUE_DOMAIN_ATTRS(other),
{"async_depth", 0400, kyber_async_depth_show},
{},
{#name "_waiting", 0400, kyber_##name##_waiting_show}
static const struct blk_mq_debugfs_attr kyber_hctx_debugfs_attrs[] = {
KYBER_HCTX_DOMAIN_ATTRS(read),
- KYBER_HCTX_DOMAIN_ATTRS(sync_write),
+ KYBER_HCTX_DOMAIN_ATTRS(write),
+ KYBER_HCTX_DOMAIN_ATTRS(discard),
KYBER_HCTX_DOMAIN_ATTRS(other),
{"cur_domain", 0400, kyber_cur_domain_show},
{"batching", 0400, kyber_batching_show},
config ACPI_EC_DEBUGFS
tristate "EC read/write access through /sys/kernel/debug/ec"
- default n
help
Say N to disable Embedded Controller /sys/kernel/debug interface
config ACPI_IPMI
tristate "IPMI"
depends on IPMI_HANDLER
- default n
help
This driver enables the ACPI to access the BMC controller. And it
uses the IPMI request/response message to communicate with BMC
config ACPI_DEBUG
bool "Debug Statements"
- default n
help
The ACPI subsystem can produce debug output. Saying Y enables this
output and increases the kernel size by around 50K.
config ACPI_PCI_SLOT
bool "PCI slot detection driver"
depends on SYSFS
- default n
help
This driver creates entries in /sys/bus/pci/slots/ for all PCI
slots in the system. This can help correlate PCI bus addresses,
config ACPI_CUSTOM_METHOD
tristate "Allow ACPI methods to be inserted/replaced at run time"
depends on DEBUG_FS
- default n
help
This debug facility allows ACPI AML methods to be inserted and/or
replaced without rebooting the system. For details refer to:
tristate "Extended Error Log support"
depends on X86_MCE && X86_LOCAL_APIC && EDAC
select UEFI_CPER
- default n
help
Certain usages such as Predictive Failure Analysis (PFA) require
more information about the error than what can be described in
spinlock_t tx_msg_lock;
acpi_handle handle;
struct device *dev;
- ipmi_user_t user_interface;
+ struct ipmi_user *user_interface;
int ipmi_ifnum; /* IPMI interface number */
long curr_msgid;
bool dead;
{
struct acpi_ipmi_device *ipmi_device;
int err;
- ipmi_user_t user;
+ struct ipmi_user *user;
ipmi_device = kzalloc(sizeof(*ipmi_device), GFP_KERNEL);
if (!ipmi_device)
if (!info->iomem_addr)
return;
+ if (!(acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0))
+ return;
+
/* Silently fail, if cpuidle attribute group is not present */
sysfs_add_file_to_group(&cpu_subsys.dev_root->kobj,
&dev_attr_low_power_idle_system_residency_us.attr,
"cpuidle");
} else if (info->gaddr.space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) {
+ if (!(acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0))
+ return;
+
/* Silently fail, if cpuidle attribute group is not present */
sysfs_add_file_to_group(&cpu_subsys.dev_root->kobj,
&dev_attr_low_power_idle_cpu_residency_us.attr,
#include <linux/err.h>
#include <linux/io.h>
#include <linux/mutex.h>
+#include <linux/pci.h>
#include <linux/platform_device.h>
#include <linux/platform_data/clk-lpss.h>
#include <linux/platform_data/x86/pmc_atom.h>
size_t prv_size_override;
struct property_entry *properties;
void (*setup)(struct lpss_private_data *pdata);
+ bool resume_from_noirq;
};
static const struct lpss_device_desc lpss_dma_desc = {
u32 prv_reg_ctx[LPSS_PRV_REG_COUNT];
};
+/* Devices which need to be in D3 before lpss_iosf_enter_d3_state() proceeds */
+static u32 pmc_atom_d3_mask = 0xfe000ffe;
+
/* LPSS run time quirks */
static unsigned int lpss_quirks;
static void byt_i2c_setup(struct lpss_private_data *pdata)
{
+ const char *uid_str = acpi_device_uid(pdata->adev);
+ acpi_handle handle = pdata->adev->handle;
+ unsigned long long shared_host = 0;
+ acpi_status status;
+ long uid = 0;
+
+ /* Expected to always be true, but better safe then sorry */
+ if (uid_str)
+ uid = simple_strtol(uid_str, NULL, 10);
+
+ /* Detect I2C bus shared with PUNIT and ignore its d3 status */
+ status = acpi_evaluate_integer(handle, "_SEM", NULL, &shared_host);
+ if (ACPI_SUCCESS(status) && shared_host && uid)
+ pmc_atom_d3_mask &= ~(BIT_LPSS2_F1_I2C1 << (uid - 1));
+
lpss_deassert_reset(pdata);
if (readl(pdata->mmio_base + pdata->dev_desc->prv_offset))
.flags = LPSS_CLK | LPSS_SAVE_CTX,
.prv_offset = 0x800,
.setup = byt_i2c_setup,
+ .resume_from_noirq = true,
};
static const struct lpss_device_desc bsw_i2c_dev_desc = {
.flags = LPSS_CLK | LPSS_SAVE_CTX | LPSS_NO_D3_DELAY,
.prv_offset = 0x800,
.setup = byt_i2c_setup,
+ .resume_from_noirq = true,
};
static const struct lpss_device_desc bsw_spi_dev_desc = {
#define ICPU(model) { X86_VENDOR_INTEL, 6, model, X86_FEATURE_ANY, }
static const struct x86_cpu_id lpss_cpu_ids[] = {
- ICPU(INTEL_FAM6_ATOM_SILVERMONT1), /* Valleyview, Bay Trail */
+ ICPU(INTEL_FAM6_ATOM_SILVERMONT), /* Valleyview, Bay Trail */
ICPU(INTEL_FAM6_ATOM_AIRMONT), /* Braswell, Cherry Trail */
{}
};
{ "INT33FC", },
/* Braswell LPSS devices */
+ { "80862286", LPSS_ADDR(lpss_dma_desc) },
{ "80862288", LPSS_ADDR(bsw_pwm_dev_desc) },
{ "8086228A", LPSS_ADDR(bsw_uart_dev_desc) },
{ "8086228E", LPSS_ADDR(bsw_spi_dev_desc) },
+ { "808622C0", LPSS_ADDR(lpss_dma_desc) },
{ "808622C1", LPSS_ADDR(bsw_i2c_dev_desc) },
/* Broadwell LPSS devices */
*/
static const struct lpss_device_links lpss_device_links[] = {
{"808622C1", "7", "80860F14", "3", DL_FLAG_PM_RUNTIME},
+ {"808622C1", "7", "LNXVIDEO", NULL, DL_FLAG_PM_RUNTIME},
+ {"80860F41", "5", "LNXVIDEO", NULL, DL_FLAG_PM_RUNTIME},
};
-static bool hid_uid_match(const char *hid1, const char *uid1,
+static bool hid_uid_match(struct acpi_device *adev,
const char *hid2, const char *uid2)
{
- return !strcmp(hid1, hid2) && uid1 && uid2 && !strcmp(uid1, uid2);
+ const char *hid1 = acpi_device_hid(adev);
+ const char *uid1 = acpi_device_uid(adev);
+
+ if (strcmp(hid1, hid2))
+ return false;
+
+ if (!uid2)
+ return true;
+
+ return uid1 && !strcmp(uid1, uid2);
}
static bool acpi_lpss_is_supplier(struct acpi_device *adev,
const struct lpss_device_links *link)
{
- return hid_uid_match(acpi_device_hid(adev), acpi_device_uid(adev),
- link->supplier_hid, link->supplier_uid);
+ return hid_uid_match(adev, link->supplier_hid, link->supplier_uid);
}
static bool acpi_lpss_is_consumer(struct acpi_device *adev,
const struct lpss_device_links *link)
{
- return hid_uid_match(acpi_device_hid(adev), acpi_device_uid(adev),
- link->consumer_hid, link->consumer_uid);
+ return hid_uid_match(adev, link->consumer_hid, link->consumer_uid);
}
struct hid_uid {
if (!adev)
return 0;
- return hid_uid_match(acpi_device_hid(adev), acpi_device_uid(adev),
- id->hid, id->uid);
+ return hid_uid_match(adev, id->hid, id->uid);
}
static struct device *acpi_lpss_find_device(const char *hid, const char *uid)
{
+ struct device *dev;
+
struct hid_uid data = {
.hid = hid,
.uid = uid,
};
- return bus_find_device(&platform_bus_type, NULL, &data, match_hid_uid);
+ dev = bus_find_device(&platform_bus_type, NULL, &data, match_hid_uid);
+ if (dev)
+ return dev;
+
+ return bus_find_device(&pci_bus_type, NULL, &data, match_hid_uid);
}
static bool acpi_lpss_dep(struct acpi_device *adev, acpi_handle handle)
* Here we read the values related to LPSS power island, i.e. LPSS
* devices, excluding both LPSS DMA controllers, along with SCC domain.
*/
- u32 func_dis, d3_sts_0, pmc_status, pmc_mask = 0xfe000ffe;
+ u32 func_dis, d3_sts_0, pmc_status;
int ret;
ret = pmc_atom_read(PMC_FUNC_DIS, &func_dis);
* Shutdown both LPSS DMA controllers if and only if all other devices
* are already in D3hot.
*/
- pmc_status = (~(d3_sts_0 | func_dis)) & pmc_mask;
+ pmc_status = (~(d3_sts_0 | func_dis)) & pmc_atom_d3_mask;
if (pmc_status)
goto exit;
}
#ifdef CONFIG_PM_SLEEP
-static int acpi_lpss_suspend_late(struct device *dev)
+static int acpi_lpss_do_suspend_late(struct device *dev)
{
int ret;
return ret ? ret : acpi_lpss_suspend(dev, device_may_wakeup(dev));
}
-static int acpi_lpss_resume_early(struct device *dev)
+static int acpi_lpss_suspend_late(struct device *dev)
+{
+ struct lpss_private_data *pdata = acpi_driver_data(ACPI_COMPANION(dev));
+
+ if (pdata->dev_desc->resume_from_noirq)
+ return 0;
+
+ return acpi_lpss_do_suspend_late(dev);
+}
+
+static int acpi_lpss_suspend_noirq(struct device *dev)
+{
+ struct lpss_private_data *pdata = acpi_driver_data(ACPI_COMPANION(dev));
+ int ret;
+
+ if (pdata->dev_desc->resume_from_noirq) {
+ ret = acpi_lpss_do_suspend_late(dev);
+ if (ret)
+ return ret;
+ }
+
+ return acpi_subsys_suspend_noirq(dev);
+}
+
+static int acpi_lpss_do_resume_early(struct device *dev)
{
int ret = acpi_lpss_resume(dev);
return ret ? ret : pm_generic_resume_early(dev);
}
+
+static int acpi_lpss_resume_early(struct device *dev)
+{
+ struct lpss_private_data *pdata = acpi_driver_data(ACPI_COMPANION(dev));
+
+ if (pdata->dev_desc->resume_from_noirq)
+ return 0;
+
+ return acpi_lpss_do_resume_early(dev);
+}
+
+static int acpi_lpss_resume_noirq(struct device *dev)
+{
+ struct lpss_private_data *pdata = acpi_driver_data(ACPI_COMPANION(dev));
+ int ret;
+
+ ret = acpi_subsys_resume_noirq(dev);
+ if (ret)
+ return ret;
+
+ if (!dev_pm_may_skip_resume(dev) && pdata->dev_desc->resume_from_noirq)
+ ret = acpi_lpss_do_resume_early(dev);
+
+ return ret;
+}
+
#endif /* CONFIG_PM_SLEEP */
static int acpi_lpss_runtime_suspend(struct device *dev)
.complete = acpi_subsys_complete,
.suspend = acpi_subsys_suspend,
.suspend_late = acpi_lpss_suspend_late,
- .suspend_noirq = acpi_subsys_suspend_noirq,
- .resume_noirq = acpi_subsys_resume_noirq,
+ .suspend_noirq = acpi_lpss_suspend_noirq,
+ .resume_noirq = acpi_lpss_resume_noirq,
.resume_early = acpi_lpss_resume_early,
.freeze = acpi_subsys_freeze,
.freeze_late = acpi_subsys_freeze_late,
#if defined(CONFIG_X86)
switch (boot_cpu_data.x86_vendor) {
+ case X86_VENDOR_HYGON:
case X86_VENDOR_AMD:
case X86_VENDOR_INTEL:
/*
status = acpi_get_type(handle, &acpi_type);
if (ACPI_FAILURE(status))
- return false;
+ return status;
switch (acpi_type) {
case ACPI_TYPE_PROCESSOR:
}
processor_validated_ids_update(uid);
- return true;
+ return AE_OK;
err:
+ /* Exit on error, but don't abort the namespace walk */
acpi_handle_info(handle, "Invalid processor object\n");
- return false;
+ return AE_OK;
}
u32 capabilities;
};
+struct acpi_tad_rt {
+ u16 year; /* 1900 - 9999 */
+ u8 month; /* 1 - 12 */
+ u8 day; /* 1 - 31 */
+ u8 hour; /* 0 - 23 */
+ u8 minute; /* 0 - 59 */
+ u8 second; /* 0 - 59 */
+ u8 valid; /* 0 (failed) or 1 (success) for reads, 0 for writes */
+ u16 msec; /* 1 - 1000 */
+ s16 tz; /* -1440 to 1440 or 2047 (unspecified) */
+ u8 daylight;
+ u8 padding[3]; /* must be 0 */
+} __packed;
+
+static int acpi_tad_set_real_time(struct device *dev, struct acpi_tad_rt *rt)
+{
+ acpi_handle handle = ACPI_HANDLE(dev);
+ union acpi_object args[] = {
+ { .type = ACPI_TYPE_BUFFER, },
+ };
+ struct acpi_object_list arg_list = {
+ .pointer = args,
+ .count = ARRAY_SIZE(args),
+ };
+ unsigned long long retval;
+ acpi_status status;
+
+ if (rt->year < 1900 || rt->year > 9999 ||
+ rt->month < 1 || rt->month > 12 ||
+ rt->hour > 23 || rt->minute > 59 || rt->second > 59 ||
+ rt->tz < -1440 || (rt->tz > 1440 && rt->tz != 2047) ||
+ rt->daylight > 3)
+ return -ERANGE;
+
+ args[0].buffer.pointer = (u8 *)rt;
+ args[0].buffer.length = sizeof(*rt);
+
+ pm_runtime_get_sync(dev);
+
+ status = acpi_evaluate_integer(handle, "_SRT", &arg_list, &retval);
+
+ pm_runtime_put_sync(dev);
+
+ if (ACPI_FAILURE(status) || retval)
+ return -EIO;
+
+ return 0;
+}
+
+static int acpi_tad_get_real_time(struct device *dev, struct acpi_tad_rt *rt)
+{
+ acpi_handle handle = ACPI_HANDLE(dev);
+ struct acpi_buffer output = { ACPI_ALLOCATE_BUFFER };
+ union acpi_object *out_obj;
+ struct acpi_tad_rt *data;
+ acpi_status status;
+ int ret = -EIO;
+
+ pm_runtime_get_sync(dev);
+
+ status = acpi_evaluate_object(handle, "_GRT", NULL, &output);
+
+ pm_runtime_put_sync(dev);
+
+ if (ACPI_FAILURE(status))
+ goto out_free;
+
+ out_obj = output.pointer;
+ if (out_obj->type != ACPI_TYPE_BUFFER)
+ goto out_free;
+
+ if (out_obj->buffer.length != sizeof(*rt))
+ goto out_free;
+
+ data = (struct acpi_tad_rt *)(out_obj->buffer.pointer);
+ if (!data->valid)
+ goto out_free;
+
+ memcpy(rt, data, sizeof(*rt));
+ ret = 0;
+
+out_free:
+ ACPI_FREE(output.pointer);
+ return ret;
+}
+
+static char *acpi_tad_rt_next_field(char *s, int *val)
+{
+ char *p;
+
+ p = strchr(s, ':');
+ if (!p)
+ return NULL;
+
+ *p = '\0';
+ if (kstrtoint(s, 10, val))
+ return NULL;
+
+ return p + 1;
+}
+
+static ssize_t time_store(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct acpi_tad_rt rt;
+ char *str, *s;
+ int val, ret = -ENODATA;
+
+ str = kmemdup_nul(buf, count, GFP_KERNEL);
+ if (!str)
+ return -ENOMEM;
+
+ s = acpi_tad_rt_next_field(str, &val);
+ if (!s)
+ goto out_free;
+
+ rt.year = val;
+
+ s = acpi_tad_rt_next_field(s, &val);
+ if (!s)
+ goto out_free;
+
+ rt.month = val;
+
+ s = acpi_tad_rt_next_field(s, &val);
+ if (!s)
+ goto out_free;
+
+ rt.day = val;
+
+ s = acpi_tad_rt_next_field(s, &val);
+ if (!s)
+ goto out_free;
+
+ rt.hour = val;
+
+ s = acpi_tad_rt_next_field(s, &val);
+ if (!s)
+ goto out_free;
+
+ rt.minute = val;
+
+ s = acpi_tad_rt_next_field(s, &val);
+ if (!s)
+ goto out_free;
+
+ rt.second = val;
+
+ s = acpi_tad_rt_next_field(s, &val);
+ if (!s)
+ goto out_free;
+
+ rt.tz = val;
+
+ if (kstrtoint(s, 10, &val))
+ goto out_free;
+
+ rt.daylight = val;
+
+ rt.valid = 0;
+ rt.msec = 0;
+ memset(rt.padding, 0, 3);
+
+ ret = acpi_tad_set_real_time(dev, &rt);
+
+out_free:
+ kfree(str);
+ return ret ? ret : count;
+}
+
+static ssize_t time_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct acpi_tad_rt rt;
+ int ret;
+
+ ret = acpi_tad_get_real_time(dev, &rt);
+ if (ret)
+ return ret;
+
+ return sprintf(buf, "%u:%u:%u:%u:%u:%u:%d:%u\n",
+ rt.year, rt.month, rt.day, rt.hour, rt.minute, rt.second,
+ rt.tz, rt.daylight);
+}
+
+static DEVICE_ATTR(time, S_IRUSR | S_IWUSR, time_show, time_store);
+
+static struct attribute *acpi_tad_time_attrs[] = {
+ &dev_attr_time.attr,
+ NULL,
+};
+static const struct attribute_group acpi_tad_time_attr_group = {
+ .attrs = acpi_tad_time_attrs,
+};
+
static int acpi_tad_wake_set(struct device *dev, char *method, u32 timer_id,
u32 value)
{
goto fail;
}
+ if (caps & ACPI_TAD_RT) {
+ ret = sysfs_create_group(&dev->kobj, &acpi_tad_time_attr_group);
+ if (ret)
+ goto fail;
+ }
+
return 0;
fail:
exresnte.o \
exresolv.o \
exresop.o \
+ exserial.o \
exstore.o \
exstoren.o \
exstorob.o \
acpi_status acpi_ev_initialize_region(union acpi_operand_object *region_obj);
+u8 acpi_ev_is_pci_root_bridge(struct acpi_namespace_node *node);
+
/*
* evsci - SCI (System Control Interrupt) handling/dispatch
*/
/*
* exfield - ACPI AML (p-code) execution - field manipulation
*/
+acpi_status
+acpi_ex_get_protocol_buffer_length(u32 protocol_id, u32 *return_length);
+
acpi_status
acpi_ex_common_buffer_setup(union acpi_operand_object *obj_desc,
u32 buffer_length, u32 * datum_count);
acpi_status acpi_ex_prep_field_value(struct acpi_create_field_info *info);
+/*
+ * exserial - field_unit support for serial address spaces
+ */
+acpi_status
+acpi_ex_read_serial_bus(union acpi_operand_object *obj_desc,
+ union acpi_operand_object **return_buffer);
+
+acpi_status
+acpi_ex_write_serial_bus(union acpi_operand_object *source_desc,
+ union acpi_operand_object *obj_desc,
+ union acpi_operand_object **return_buffer);
+
+acpi_status
+acpi_ex_read_gpio(union acpi_operand_object *obj_desc, void *buffer);
+
+acpi_status
+acpi_ex_write_gpio(union acpi_operand_object *source_desc,
+ union acpi_operand_object *obj_desc,
+ union acpi_operand_object **return_buffer);
+
/*
* exsystem - Interface to OS services
*/
/* Info for running the _REG methods */
struct acpi_reg_walk_info {
- acpi_adr_space_type space_id;
u32 function;
u32 reg_run_count;
+ acpi_adr_space_type space_id;
};
/*****************************************************************************
*/
typedef enum {
AML_FIELD_ATTRIB_QUICK = 0x02,
- AML_FIELD_ATTRIB_SEND_RCV = 0x04,
+ AML_FIELD_ATTRIB_SEND_RECEIVE = 0x04,
AML_FIELD_ATTRIB_BYTE = 0x06,
AML_FIELD_ATTRIB_WORD = 0x08,
AML_FIELD_ATTRIB_BLOCK = 0x0A,
- AML_FIELD_ATTRIB_MULTIBYTE = 0x0B,
- AML_FIELD_ATTRIB_WORD_CALL = 0x0C,
- AML_FIELD_ATTRIB_BLOCK_CALL = 0x0D,
+ AML_FIELD_ATTRIB_BYTES = 0x0B,
+ AML_FIELD_ATTRIB_PROCESS_CALL = 0x0C,
+ AML_FIELD_ATTRIB_BLOCK_PROCESS_CALL = 0x0D,
AML_FIELD_ATTRIB_RAW_BYTES = 0x0E,
- AML_FIELD_ATTRIB_RAW_PROCESS = 0x0F
+ AML_FIELD_ATTRIB_RAW_PROCESS_BYTES = 0x0F
} AML_ACCESS_ATTRIBUTE;
/* Bit fields in the AML method_flags byte */
ACPI_FORMAT_UINT64(obj_desc->region.address),
obj_desc->region.length));
+ status = acpi_ut_add_address_range(obj_desc->region.space_id,
+ obj_desc->region.address,
+ obj_desc->region.length, node);
+
/* Now the address and length are valid for this opregion */
obj_desc->region.flags |= AOPOBJ_DATA_VALID;
ACPI_FUNCTION_TRACE(ev_execute_reg_methods);
+ /*
+ * These address spaces do not need a call to _REG, since the ACPI
+ * specification defines them as: "must always be accessible". Since
+ * they never change state (never become unavailable), no need to ever
+ * call _REG on them. Also, a data_table is not a "real" address space,
+ * so do not call _REG. September 2018.
+ */
+ if ((space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY) ||
+ (space_id == ACPI_ADR_SPACE_SYSTEM_IO) ||
+ (space_id == ACPI_ADR_SPACE_DATA_TABLE)) {
+ return_VOID;
+ }
+
info.space_id = space_id;
info.function = function;
info.reg_run_count = 0;
}
/*
- * We only care about regions.and objects that are allowed to have address
- * space handlers
+ * We only care about regions and objects that are allowed to have
+ * address space handlers
*/
if ((node->type != ACPI_TYPE_REGION) && (node != acpi_gbl_root_node)) {
return (AE_OK);
#define _COMPONENT ACPI_EVENTS
ACPI_MODULE_NAME("evrgnini")
-/* Local prototypes */
-static u8 acpi_ev_is_pci_root_bridge(struct acpi_namespace_node *node);
-
/*******************************************************************************
*
* FUNCTION: acpi_ev_system_memory_region_setup
* DESCRIPTION: Setup a system_memory operation region
*
******************************************************************************/
-
acpi_status
acpi_ev_system_memory_region_setup(acpi_handle handle,
u32 function,
*
******************************************************************************/
-static u8 acpi_ev_is_pci_root_bridge(struct acpi_namespace_node *node)
+u8 acpi_ev_is_pci_root_bridge(struct acpi_namespace_node *node)
{
acpi_status status;
struct acpi_pnp_device_id *hid;
*/
region_obj =
handler_obj->address_space.region_list;
-
}
/* Remove this Handler object from the list */
// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0
/******************************************************************************
*
- * Module Name: exfield - ACPI AML (p-code) execution - field manipulation
+ * Module Name: exfield - AML execution - field_unit read/write
*
* Copyright (C) 2000 - 2018, Intel Corp.
*
#define _COMPONENT ACPI_EXECUTER
ACPI_MODULE_NAME("exfield")
-/* Local prototypes */
-static u32
-acpi_ex_get_serial_access_length(u32 accessor_type, u32 access_length);
+/*
+ * This table maps the various Attrib protocols to the byte transfer
+ * length. Used for the generic serial bus.
+ */
+#define ACPI_INVALID_PROTOCOL_ID 0x80
+#define ACPI_MAX_PROTOCOL_ID 0x0F
+const u8 acpi_protocol_lengths[] = {
+ ACPI_INVALID_PROTOCOL_ID, /* 0 - reserved */
+ ACPI_INVALID_PROTOCOL_ID, /* 1 - reserved */
+ 0x00, /* 2 - ATTRIB_QUICK */
+ ACPI_INVALID_PROTOCOL_ID, /* 3 - reserved */
+ 0x01, /* 4 - ATTRIB_SEND_RECEIVE */
+ ACPI_INVALID_PROTOCOL_ID, /* 5 - reserved */
+ 0x01, /* 6 - ATTRIB_BYTE */
+ ACPI_INVALID_PROTOCOL_ID, /* 7 - reserved */
+ 0x02, /* 8 - ATTRIB_WORD */
+ ACPI_INVALID_PROTOCOL_ID, /* 9 - reserved */
+ 0xFF, /* A - ATTRIB_BLOCK */
+ 0xFF, /* B - ATTRIB_BYTES */
+ 0x02, /* C - ATTRIB_PROCESS_CALL */
+ 0xFF, /* D - ATTRIB_BLOCK_PROCESS_CALL */
+ 0xFF, /* E - ATTRIB_RAW_BYTES */
+ 0xFF /* F - ATTRIB_RAW_PROCESS_BYTES */
+};
/*******************************************************************************
*
- * FUNCTION: acpi_ex_get_serial_access_length
+ * FUNCTION: acpi_ex_get_protocol_buffer_length
*
- * PARAMETERS: accessor_type - The type of the protocol indicated by region
+ * PARAMETERS: protocol_id - The type of the protocol indicated by region
* field access attributes
- * access_length - The access length of the region field
+ * return_length - Where the protocol byte transfer length is
+ * returned
*
- * RETURN: Decoded access length
+ * RETURN: Status and decoded byte transfer length
*
* DESCRIPTION: This routine returns the length of the generic_serial_bus
* protocol bytes
*
******************************************************************************/
-static u32
-acpi_ex_get_serial_access_length(u32 accessor_type, u32 access_length)
+acpi_status
+acpi_ex_get_protocol_buffer_length(u32 protocol_id, u32 *return_length)
{
- u32 length;
-
- switch (accessor_type) {
- case AML_FIELD_ATTRIB_QUICK:
-
- length = 0;
- break;
-
- case AML_FIELD_ATTRIB_SEND_RCV:
- case AML_FIELD_ATTRIB_BYTE:
-
- length = 1;
- break;
-
- case AML_FIELD_ATTRIB_WORD:
- case AML_FIELD_ATTRIB_WORD_CALL:
-
- length = 2;
- break;
- case AML_FIELD_ATTRIB_MULTIBYTE:
- case AML_FIELD_ATTRIB_RAW_BYTES:
- case AML_FIELD_ATTRIB_RAW_PROCESS:
+ if ((protocol_id > ACPI_MAX_PROTOCOL_ID) ||
+ (acpi_protocol_lengths[protocol_id] == ACPI_INVALID_PROTOCOL_ID)) {
+ ACPI_ERROR((AE_INFO,
+ "Invalid Field/AccessAs protocol ID: 0x%4.4X",
+ protocol_id));
- length = access_length;
- break;
-
- case AML_FIELD_ATTRIB_BLOCK:
- case AML_FIELD_ATTRIB_BLOCK_CALL:
- default:
-
- length = ACPI_GSBUS_BUFFER_SIZE - 2;
- break;
+ return (AE_AML_PROTOCOL);
}
- return (length);
+ *return_length = acpi_protocol_lengths[protocol_id];
+ return (AE_OK);
}
/*******************************************************************************
{
acpi_status status;
union acpi_operand_object *buffer_desc;
- acpi_size length;
void *buffer;
- u32 function;
- u16 accessor_type;
+ u32 buffer_length;
ACPI_FUNCTION_TRACE_PTR(ex_read_data_from_field, obj_desc);
ACPI_ADR_SPACE_GSBUS
|| obj_desc->field.region_obj->region.space_id ==
ACPI_ADR_SPACE_IPMI)) {
- /*
- * This is an SMBus, GSBus or IPMI read. We must create a buffer to
- * hold the data and then directly access the region handler.
- *
- * Note: SMBus and GSBus protocol value is passed in upper 16-bits
- * of Function
- */
- if (obj_desc->field.region_obj->region.space_id ==
- ACPI_ADR_SPACE_SMBUS) {
- length = ACPI_SMBUS_BUFFER_SIZE;
- function =
- ACPI_READ | (obj_desc->field.attribute << 16);
- } else if (obj_desc->field.region_obj->region.space_id ==
- ACPI_ADR_SPACE_GSBUS) {
- accessor_type = obj_desc->field.attribute;
- length =
- acpi_ex_get_serial_access_length(accessor_type,
- obj_desc->field.
- access_length);
-
- /*
- * Add additional 2 bytes for the generic_serial_bus data buffer:
- *
- * Status; (Byte 0 of the data buffer)
- * Length; (Byte 1 of the data buffer)
- * Data[x-1]: (Bytes 2-x of the arbitrary length data buffer)
- */
- length += 2;
- function = ACPI_READ | (accessor_type << 16);
- } else { /* IPMI */
-
- length = ACPI_IPMI_BUFFER_SIZE;
- function = ACPI_READ;
- }
-
- buffer_desc = acpi_ut_create_buffer_object(length);
- if (!buffer_desc) {
- return_ACPI_STATUS(AE_NO_MEMORY);
- }
-
- /* Lock entire transaction if requested */
-
- acpi_ex_acquire_global_lock(obj_desc->common_field.field_flags);
-
- /* Call the region handler for the read */
- status = acpi_ex_access_region(obj_desc, 0,
- ACPI_CAST_PTR(u64,
- buffer_desc->
- buffer.pointer),
- function);
+ /* SMBus, GSBus, IPMI serial */
- acpi_ex_release_global_lock(obj_desc->common_field.field_flags);
- goto exit;
+ status = acpi_ex_read_serial_bus(obj_desc, ret_buffer_desc);
+ return_ACPI_STATUS(status);
}
/*
*
* Note: Field.length is in bits.
*/
- length =
+ buffer_length =
(acpi_size)ACPI_ROUND_BITS_UP_TO_BYTES(obj_desc->field.bit_length);
- if (length > acpi_gbl_integer_byte_width) {
+ if (buffer_length > acpi_gbl_integer_byte_width) {
/* Field is too large for an Integer, create a Buffer instead */
- buffer_desc = acpi_ut_create_buffer_object(length);
+ buffer_desc = acpi_ut_create_buffer_object(buffer_length);
if (!buffer_desc) {
return_ACPI_STATUS(AE_NO_MEMORY);
}
return_ACPI_STATUS(AE_NO_MEMORY);
}
- length = acpi_gbl_integer_byte_width;
+ buffer_length = acpi_gbl_integer_byte_width;
buffer = &buffer_desc->integer.value;
}
if ((obj_desc->common.type == ACPI_TYPE_LOCAL_REGION_FIELD) &&
(obj_desc->field.region_obj->region.space_id ==
ACPI_ADR_SPACE_GPIO)) {
- /*
- * For GPIO (general_purpose_io), the Address will be the bit offset
- * from the previous Connection() operator, making it effectively a
- * pin number index. The bit_length is the length of the field, which
- * is thus the number of pins.
- */
- ACPI_DEBUG_PRINT((ACPI_DB_BFIELD,
- "GPIO FieldRead [FROM]: Pin %u Bits %u\n",
- obj_desc->field.pin_number_index,
- obj_desc->field.bit_length));
-
- /* Lock entire transaction if requested */
- acpi_ex_acquire_global_lock(obj_desc->common_field.field_flags);
+ /* General Purpose I/O */
- /* Perform the write */
-
- status =
- acpi_ex_access_region(obj_desc, 0, (u64 *)buffer,
- ACPI_READ);
-
- acpi_ex_release_global_lock(obj_desc->common_field.field_flags);
- if (ACPI_FAILURE(status)) {
- acpi_ut_remove_reference(buffer_desc);
- } else {
- *ret_buffer_desc = buffer_desc;
- }
- return_ACPI_STATUS(status);
+ status = acpi_ex_read_gpio(obj_desc, buffer);
+ goto exit;
}
ACPI_DEBUG_PRINT((ACPI_DB_BFIELD,
"FieldRead [TO]: Obj %p, Type %X, Buf %p, ByteLen %X\n",
obj_desc, obj_desc->common.type, buffer,
- (u32) length));
+ buffer_length));
ACPI_DEBUG_PRINT((ACPI_DB_BFIELD,
"FieldRead [FROM]: BitLen %X, BitOff %X, ByteOff %X\n",
obj_desc->common_field.bit_length,
/* Read from the field */
- status = acpi_ex_extract_from_field(obj_desc, buffer, (u32) length);
+ status = acpi_ex_extract_from_field(obj_desc, buffer, buffer_length);
acpi_ex_release_global_lock(obj_desc->common_field.field_flags);
exit:
union acpi_operand_object **result_desc)
{
acpi_status status;
- u32 length;
+ u32 buffer_length;
void *buffer;
- union acpi_operand_object *buffer_desc;
- u32 function;
- u16 accessor_type;
ACPI_FUNCTION_TRACE_PTR(ex_write_data_to_field, obj_desc);
}
} else if ((obj_desc->common.type == ACPI_TYPE_LOCAL_REGION_FIELD) &&
(obj_desc->field.region_obj->region.space_id ==
- ACPI_ADR_SPACE_SMBUS
- || obj_desc->field.region_obj->region.space_id ==
- ACPI_ADR_SPACE_GSBUS
- || obj_desc->field.region_obj->region.space_id ==
- ACPI_ADR_SPACE_IPMI)) {
- /*
- * This is an SMBus, GSBus or IPMI write. We will bypass the entire
- * field mechanism and handoff the buffer directly to the handler.
- * For these address spaces, the buffer is bi-directional; on a
- * write, return data is returned in the same buffer.
- *
- * Source must be a buffer of sufficient size:
- * ACPI_SMBUS_BUFFER_SIZE, ACPI_GSBUS_BUFFER_SIZE, or
- * ACPI_IPMI_BUFFER_SIZE.
- *
- * Note: SMBus and GSBus protocol type is passed in upper 16-bits
- * of Function
- */
- if (source_desc->common.type != ACPI_TYPE_BUFFER) {
- ACPI_ERROR((AE_INFO,
- "SMBus/IPMI/GenericSerialBus write requires "
- "Buffer, found type %s",
- acpi_ut_get_object_type_name(source_desc)));
-
- return_ACPI_STATUS(AE_AML_OPERAND_TYPE);
- }
-
- if (obj_desc->field.region_obj->region.space_id ==
- ACPI_ADR_SPACE_SMBUS) {
- length = ACPI_SMBUS_BUFFER_SIZE;
- function =
- ACPI_WRITE | (obj_desc->field.attribute << 16);
- } else if (obj_desc->field.region_obj->region.space_id ==
- ACPI_ADR_SPACE_GSBUS) {
- accessor_type = obj_desc->field.attribute;
- length =
- acpi_ex_get_serial_access_length(accessor_type,
- obj_desc->field.
- access_length);
-
- /*
- * Add additional 2 bytes for the generic_serial_bus data buffer:
- *
- * Status; (Byte 0 of the data buffer)
- * Length; (Byte 1 of the data buffer)
- * Data[x-1]: (Bytes 2-x of the arbitrary length data buffer)
- */
- length += 2;
- function = ACPI_WRITE | (accessor_type << 16);
- } else { /* IPMI */
-
- length = ACPI_IPMI_BUFFER_SIZE;
- function = ACPI_WRITE;
- }
-
- if (source_desc->buffer.length < length) {
- ACPI_ERROR((AE_INFO,
- "SMBus/IPMI/GenericSerialBus write requires "
- "Buffer of length %u, found length %u",
- length, source_desc->buffer.length));
-
- return_ACPI_STATUS(AE_AML_BUFFER_LIMIT);
- }
-
- /* Create the bi-directional buffer */
-
- buffer_desc = acpi_ut_create_buffer_object(length);
- if (!buffer_desc) {
- return_ACPI_STATUS(AE_NO_MEMORY);
- }
-
- buffer = buffer_desc->buffer.pointer;
- memcpy(buffer, source_desc->buffer.pointer, length);
-
- /* Lock entire transaction if requested */
+ ACPI_ADR_SPACE_GPIO)) {
- acpi_ex_acquire_global_lock(obj_desc->common_field.field_flags);
+ /* General Purpose I/O */
- /*
- * Perform the write (returns status and perhaps data in the
- * same buffer)
- */
- status =
- acpi_ex_access_region(obj_desc, 0, (u64 *)buffer, function);
- acpi_ex_release_global_lock(obj_desc->common_field.field_flags);
-
- *result_desc = buffer_desc;
+ status = acpi_ex_write_gpio(source_desc, obj_desc, result_desc);
return_ACPI_STATUS(status);
} else if ((obj_desc->common.type == ACPI_TYPE_LOCAL_REGION_FIELD) &&
(obj_desc->field.region_obj->region.space_id ==
- ACPI_ADR_SPACE_GPIO)) {
- /*
- * For GPIO (general_purpose_io), we will bypass the entire field
- * mechanism and handoff the bit address and bit width directly to
- * the handler. The Address will be the bit offset
- * from the previous Connection() operator, making it effectively a
- * pin number index. The bit_length is the length of the field, which
- * is thus the number of pins.
- */
- if (source_desc->common.type != ACPI_TYPE_INTEGER) {
- return_ACPI_STATUS(AE_AML_OPERAND_TYPE);
- }
-
- ACPI_DEBUG_PRINT((ACPI_DB_BFIELD,
- "GPIO FieldWrite [FROM]: (%s:%X), Val %.8X [TO]: Pin %u Bits %u\n",
- acpi_ut_get_type_name(source_desc->common.
- type),
- source_desc->common.type,
- (u32)source_desc->integer.value,
- obj_desc->field.pin_number_index,
- obj_desc->field.bit_length));
-
- buffer = &source_desc->integer.value;
-
- /* Lock entire transaction if requested */
-
- acpi_ex_acquire_global_lock(obj_desc->common_field.field_flags);
+ ACPI_ADR_SPACE_SMBUS
+ || obj_desc->field.region_obj->region.space_id ==
+ ACPI_ADR_SPACE_GSBUS
+ || obj_desc->field.region_obj->region.space_id ==
+ ACPI_ADR_SPACE_IPMI)) {
- /* Perform the write */
+ /* SMBus, GSBus, IPMI serial */
status =
- acpi_ex_access_region(obj_desc, 0, (u64 *)buffer,
- ACPI_WRITE);
- acpi_ex_release_global_lock(obj_desc->common_field.field_flags);
+ acpi_ex_write_serial_bus(source_desc, obj_desc,
+ result_desc);
return_ACPI_STATUS(status);
}
case ACPI_TYPE_INTEGER:
buffer = &source_desc->integer.value;
- length = sizeof(source_desc->integer.value);
+ buffer_length = sizeof(source_desc->integer.value);
break;
case ACPI_TYPE_BUFFER:
buffer = source_desc->buffer.pointer;
- length = source_desc->buffer.length;
+ buffer_length = source_desc->buffer.length;
break;
case ACPI_TYPE_STRING:
buffer = source_desc->string.pointer;
- length = source_desc->string.length;
+ buffer_length = source_desc->string.length;
break;
default:
-
return_ACPI_STATUS(AE_AML_OPERAND_TYPE);
}
"FieldWrite [FROM]: Obj %p (%s:%X), Buf %p, ByteLen %X\n",
source_desc,
acpi_ut_get_type_name(source_desc->common.type),
- source_desc->common.type, buffer, length));
+ source_desc->common.type, buffer, buffer_length));
ACPI_DEBUG_PRINT((ACPI_DB_BFIELD,
"FieldWrite [TO]: Obj %p (%s:%X), BitLen %X, BitOff %X, ByteOff %X\n",
/* Write to the field */
- status = acpi_ex_insert_into_field(obj_desc, buffer, length);
+ status = acpi_ex_insert_into_field(obj_desc, buffer, buffer_length);
acpi_ex_release_global_lock(obj_desc->common_field.field_flags);
-
return_ACPI_STATUS(status);
}
--- /dev/null
+// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0
+/******************************************************************************
+ *
+ * Module Name: exserial - field_unit support for serial address spaces
+ *
+ * Copyright (C) 2000 - 2018, Intel Corp.
+ *
+ *****************************************************************************/
+
+#include <acpi/acpi.h>
+#include "accommon.h"
+#include "acdispat.h"
+#include "acinterp.h"
+#include "amlcode.h"
+
+#define _COMPONENT ACPI_EXECUTER
+ACPI_MODULE_NAME("exserial")
+
+/*******************************************************************************
+ *
+ * FUNCTION: acpi_ex_read_gpio
+ *
+ * PARAMETERS: obj_desc - The named field to read
+ * buffer - Where the return data is returnd
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Read from a named field that references a Generic Serial Bus
+ * field
+ *
+ ******************************************************************************/
+acpi_status acpi_ex_read_gpio(union acpi_operand_object *obj_desc, void *buffer)
+{
+ acpi_status status;
+
+ ACPI_FUNCTION_TRACE_PTR(ex_read_gpio, obj_desc);
+
+ /*
+ * For GPIO (general_purpose_io), the Address will be the bit offset
+ * from the previous Connection() operator, making it effectively a
+ * pin number index. The bit_length is the length of the field, which
+ * is thus the number of pins.
+ */
+ ACPI_DEBUG_PRINT((ACPI_DB_BFIELD,
+ "GPIO FieldRead [FROM]: Pin %u Bits %u\n",
+ obj_desc->field.pin_number_index,
+ obj_desc->field.bit_length));
+
+ /* Lock entire transaction if requested */
+
+ acpi_ex_acquire_global_lock(obj_desc->common_field.field_flags);
+
+ /* Perform the read */
+
+ status = acpi_ex_access_region(obj_desc, 0, (u64 *)buffer, ACPI_READ);
+
+ acpi_ex_release_global_lock(obj_desc->common_field.field_flags);
+ return_ACPI_STATUS(status);
+}
+
+/*******************************************************************************
+ *
+ * FUNCTION: acpi_ex_write_gpio
+ *
+ * PARAMETERS: source_desc - Contains data to write. Expect to be
+ * an Integer object.
+ * obj_desc - The named field
+ * result_desc - Where the return value is returned, if any
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Write to a named field that references a General Purpose I/O
+ * field.
+ *
+ ******************************************************************************/
+
+acpi_status
+acpi_ex_write_gpio(union acpi_operand_object *source_desc,
+ union acpi_operand_object *obj_desc,
+ union acpi_operand_object **return_buffer)
+{
+ acpi_status status;
+ void *buffer;
+
+ ACPI_FUNCTION_TRACE_PTR(ex_write_gpio, obj_desc);
+
+ /*
+ * For GPIO (general_purpose_io), we will bypass the entire field
+ * mechanism and handoff the bit address and bit width directly to
+ * the handler. The Address will be the bit offset
+ * from the previous Connection() operator, making it effectively a
+ * pin number index. The bit_length is the length of the field, which
+ * is thus the number of pins.
+ */
+ if (source_desc->common.type != ACPI_TYPE_INTEGER) {
+ return_ACPI_STATUS(AE_AML_OPERAND_TYPE);
+ }
+
+ ACPI_DEBUG_PRINT((ACPI_DB_BFIELD,
+ "GPIO FieldWrite [FROM]: (%s:%X), Value %.8X [TO]: Pin %u Bits %u\n",
+ acpi_ut_get_type_name(source_desc->common.type),
+ source_desc->common.type,
+ (u32)source_desc->integer.value,
+ obj_desc->field.pin_number_index,
+ obj_desc->field.bit_length));
+
+ buffer = &source_desc->integer.value;
+
+ /* Lock entire transaction if requested */
+
+ acpi_ex_acquire_global_lock(obj_desc->common_field.field_flags);
+
+ /* Perform the write */
+
+ status = acpi_ex_access_region(obj_desc, 0, (u64 *)buffer, ACPI_WRITE);
+ acpi_ex_release_global_lock(obj_desc->common_field.field_flags);
+ return_ACPI_STATUS(status);
+}
+
+/*******************************************************************************
+ *
+ * FUNCTION: acpi_ex_read_serial_bus
+ *
+ * PARAMETERS: obj_desc - The named field to read
+ * return_buffer - Where the return value is returned, if any
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Read from a named field that references a serial bus
+ * (SMBus, IPMI, or GSBus).
+ *
+ ******************************************************************************/
+
+acpi_status
+acpi_ex_read_serial_bus(union acpi_operand_object *obj_desc,
+ union acpi_operand_object **return_buffer)
+{
+ acpi_status status;
+ u32 buffer_length;
+ union acpi_operand_object *buffer_desc;
+ u32 function;
+ u16 accessor_type;
+
+ ACPI_FUNCTION_TRACE_PTR(ex_read_serial_bus, obj_desc);
+
+ /*
+ * This is an SMBus, GSBus or IPMI read. We must create a buffer to
+ * hold the data and then directly access the region handler.
+ *
+ * Note: SMBus and GSBus protocol value is passed in upper 16-bits
+ * of Function
+ *
+ * Common buffer format:
+ * Status; (Byte 0 of the data buffer)
+ * Length; (Byte 1 of the data buffer)
+ * Data[x-1]: (Bytes 2-x of the arbitrary length data buffer)
+ */
+ switch (obj_desc->field.region_obj->region.space_id) {
+ case ACPI_ADR_SPACE_SMBUS:
+
+ buffer_length = ACPI_SMBUS_BUFFER_SIZE;
+ function = ACPI_READ | (obj_desc->field.attribute << 16);
+ break;
+
+ case ACPI_ADR_SPACE_IPMI:
+
+ buffer_length = ACPI_IPMI_BUFFER_SIZE;
+ function = ACPI_READ;
+ break;
+
+ case ACPI_ADR_SPACE_GSBUS:
+
+ accessor_type = obj_desc->field.attribute;
+ if (accessor_type == AML_FIELD_ATTRIB_RAW_PROCESS_BYTES) {
+ ACPI_ERROR((AE_INFO,
+ "Invalid direct read using bidirectional write-then-read protocol"));
+
+ return_ACPI_STATUS(AE_AML_PROTOCOL);
+ }
+
+ status =
+ acpi_ex_get_protocol_buffer_length(accessor_type,
+ &buffer_length);
+ if (ACPI_FAILURE(status)) {
+ ACPI_ERROR((AE_INFO,
+ "Invalid protocol ID for GSBus: 0x%4.4X",
+ accessor_type));
+
+ return_ACPI_STATUS(status);
+ }
+
+ /* Add header length to get the full size of the buffer */
+
+ buffer_length += ACPI_SERIAL_HEADER_SIZE;
+ function = ACPI_READ | (accessor_type << 16);
+ break;
+
+ default:
+ return_ACPI_STATUS(AE_AML_INVALID_SPACE_ID);
+ }
+
+ /* Create the local transfer buffer that is returned to the caller */
+
+ buffer_desc = acpi_ut_create_buffer_object(buffer_length);
+ if (!buffer_desc) {
+ return_ACPI_STATUS(AE_NO_MEMORY);
+ }
+
+ /* Lock entire transaction if requested */
+
+ acpi_ex_acquire_global_lock(obj_desc->common_field.field_flags);
+
+ /* Call the region handler for the write-then-read */
+
+ status = acpi_ex_access_region(obj_desc, 0,
+ ACPI_CAST_PTR(u64,
+ buffer_desc->buffer.
+ pointer), function);
+ acpi_ex_release_global_lock(obj_desc->common_field.field_flags);
+
+ *return_buffer = buffer_desc;
+ return_ACPI_STATUS(status);
+}
+
+/*******************************************************************************
+ *
+ * FUNCTION: acpi_ex_write_serial_bus
+ *
+ * PARAMETERS: source_desc - Contains data to write
+ * obj_desc - The named field
+ * return_buffer - Where the return value is returned, if any
+ *
+ * RETURN: Status
+ *
+ * DESCRIPTION: Write to a named field that references a serial bus
+ * (SMBus, IPMI, GSBus).
+ *
+ ******************************************************************************/
+
+acpi_status
+acpi_ex_write_serial_bus(union acpi_operand_object *source_desc,
+ union acpi_operand_object *obj_desc,
+ union acpi_operand_object **return_buffer)
+{
+ acpi_status status;
+ u32 buffer_length;
+ u32 data_length;
+ void *buffer;
+ union acpi_operand_object *buffer_desc;
+ u32 function;
+ u16 accessor_type;
+
+ ACPI_FUNCTION_TRACE_PTR(ex_write_serial_bus, obj_desc);
+
+ /*
+ * This is an SMBus, GSBus or IPMI write. We will bypass the entire
+ * field mechanism and handoff the buffer directly to the handler.
+ * For these address spaces, the buffer is bidirectional; on a
+ * write, return data is returned in the same buffer.
+ *
+ * Source must be a buffer of sufficient size, these are fixed size:
+ * ACPI_SMBUS_BUFFER_SIZE, or ACPI_IPMI_BUFFER_SIZE.
+ *
+ * Note: SMBus and GSBus protocol type is passed in upper 16-bits
+ * of Function
+ *
+ * Common buffer format:
+ * Status; (Byte 0 of the data buffer)
+ * Length; (Byte 1 of the data buffer)
+ * Data[x-1]: (Bytes 2-x of the arbitrary length data buffer)
+ */
+ if (source_desc->common.type != ACPI_TYPE_BUFFER) {
+ ACPI_ERROR((AE_INFO,
+ "SMBus/IPMI/GenericSerialBus write requires "
+ "Buffer, found type %s",
+ acpi_ut_get_object_type_name(source_desc)));
+
+ return_ACPI_STATUS(AE_AML_OPERAND_TYPE);
+ }
+
+ switch (obj_desc->field.region_obj->region.space_id) {
+ case ACPI_ADR_SPACE_SMBUS:
+
+ buffer_length = ACPI_SMBUS_BUFFER_SIZE;
+ data_length = ACPI_SMBUS_DATA_SIZE;
+ function = ACPI_WRITE | (obj_desc->field.attribute << 16);
+ break;
+
+ case ACPI_ADR_SPACE_IPMI:
+
+ buffer_length = ACPI_IPMI_BUFFER_SIZE;
+ data_length = ACPI_IPMI_DATA_SIZE;
+ function = ACPI_WRITE;
+ break;
+
+ case ACPI_ADR_SPACE_GSBUS:
+
+ accessor_type = obj_desc->field.attribute;
+ status =
+ acpi_ex_get_protocol_buffer_length(accessor_type,
+ &buffer_length);
+ if (ACPI_FAILURE(status)) {
+ ACPI_ERROR((AE_INFO,
+ "Invalid protocol ID for GSBus: 0x%4.4X",
+ accessor_type));
+
+ return_ACPI_STATUS(status);
+ }
+
+ /* Add header length to get the full size of the buffer */
+
+ buffer_length += ACPI_SERIAL_HEADER_SIZE;
+ data_length = source_desc->buffer.pointer[1];
+ function = ACPI_WRITE | (accessor_type << 16);
+ break;
+
+ default:
+ return_ACPI_STATUS(AE_AML_INVALID_SPACE_ID);
+ }
+
+#if 0
+ OBSOLETE ?
+ /* Check for possible buffer overflow */
+ if (data_length > source_desc->buffer.length) {
+ ACPI_ERROR((AE_INFO,
+ "Length in buffer header (%u)(%u) is greater than "
+ "the physical buffer length (%u) and will overflow",
+ data_length, buffer_length,
+ source_desc->buffer.length));
+
+ return_ACPI_STATUS(AE_AML_BUFFER_LIMIT);
+ }
+#endif
+
+ /* Create the transfer/bidirectional/return buffer */
+
+ buffer_desc = acpi_ut_create_buffer_object(buffer_length);
+ if (!buffer_desc) {
+ return_ACPI_STATUS(AE_NO_MEMORY);
+ }
+
+ /* Copy the input buffer data to the transfer buffer */
+
+ buffer = buffer_desc->buffer.pointer;
+ memcpy(buffer, source_desc->buffer.pointer, data_length);
+
+ /* Lock entire transaction if requested */
+
+ acpi_ex_acquire_global_lock(obj_desc->common_field.field_flags);
+
+ /*
+ * Perform the write (returns status and perhaps data in the
+ * same buffer)
+ */
+ status = acpi_ex_access_region(obj_desc, 0, (u64 *)buffer, function);
+ acpi_ex_release_global_lock(obj_desc->common_field.field_flags);
+
+ *return_buffer = buffer_desc;
+ return_ACPI_STATUS(status);
+}
* future. Use of this option can cause problems with AML code that
* depends upon in-order immediate execution of module-level code.
*/
- if (acpi_gbl_group_module_level_code &&
+ if (!acpi_gbl_execute_tables_as_methods &&
(walk_state->pass_number <= ACPI_IMODE_LOAD_PASS2) &&
((walk_state->parse_flags & ACPI_PARSE_DISASSEMBLE) == 0)) {
/*
union acpi_parse_object *op = NULL; /* current op */
struct acpi_parse_state *parser_state;
u8 *aml_op_start = NULL;
+ u8 opcode_length;
ACPI_FUNCTION_TRACE_PTR(ps_parse_loop, walk_state);
"Skip parsing opcode %s",
acpi_ps_get_opcode_name
(walk_state->opcode)));
+
+ /*
+ * Determine the opcode length before skipping the opcode.
+ * An opcode can be 1 byte or 2 bytes in length.
+ */
+ opcode_length = 1;
+ if ((walk_state->opcode & 0xFF00) ==
+ AML_EXTENDED_OPCODE) {
+ opcode_length = 2;
+ }
walk_state->parser_state.aml =
- walk_state->aml + 1;
+ walk_state->aml + opcode_length;
+
walk_state->parser_state.aml =
acpi_ps_get_next_package_end
(&walk_state->parser_state);
"While loading namespace from ACPI tables"));
}
- if (acpi_gbl_execute_tables_as_methods
- || !acpi_gbl_group_module_level_code) {
+ if (acpi_gbl_execute_tables_as_methods) {
/*
* If the module-level code support is enabled, initialize the objects
* in the namespace that remain uninitialized. This runs the executable
return 0;
dma_deconfigure:
- acpi_dma_deconfigure(&pdev->dev);
+ arch_teardown_dma_ops(&pdev->dev);
dev_put:
platform_device_put(pdev);
goto error0;
}
- if (!acpi_gbl_execute_tables_as_methods &&
- acpi_gbl_group_module_level_code) {
- status = acpi_load_tables();
- if (ACPI_FAILURE(status)) {
- printk(KERN_ERR PREFIX
- "Unable to load the System Description Tables\n");
- goto error0;
- }
- }
+ /*
+ * ACPI 2.0 requires the EC driver to be loaded and work before
+ * the EC device is found in the namespace (i.e. before
+ * acpi_load_tables() is called).
+ *
+ * This is accomplished by looking for the ECDT table, and getting
+ * the EC parameters out of that.
+ *
+ * Ignore the result. Not having an ECDT is not fatal.
+ */
+ status = acpi_ec_ecdt_probe();
#ifdef CONFIG_X86
if (!acpi_ioapic) {
acpi_os_initialize1();
- /*
- * ACPI 2.0 requires the EC driver to be loaded and work before
- * the EC device is found in the namespace (i.e. before
- * acpi_load_tables() is called).
- *
- * This is accomplished by looking for the ECDT table, and getting
- * the EC parameters out of that.
- */
- status = acpi_ec_ecdt_probe();
- /* Ignore result. Not having an ECDT is not fatal. */
-
- if (acpi_gbl_execute_tables_as_methods ||
- !acpi_gbl_group_module_level_code) {
- status = acpi_load_tables();
- if (ACPI_FAILURE(status)) {
- printk(KERN_ERR PREFIX
- "Unable to load the System Description Tables\n");
- goto error1;
- }
+ status = acpi_load_tables();
+ if (ACPI_FAILURE(status)) {
+ printk(KERN_ERR PREFIX
+ "Unable to load the System Description Tables\n");
+ goto error1;
}
status = acpi_enable_subsystem(ACPI_NO_ACPI_ENABLE);
{
struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpunum);
struct cpc_register_resource *highest_reg, *lowest_reg,
- *lowest_non_linear_reg, *nominal_reg,
+ *lowest_non_linear_reg, *nominal_reg, *guaranteed_reg,
*low_freq_reg = NULL, *nom_freq_reg = NULL;
- u64 high, low, nom, min_nonlinear, low_f = 0, nom_f = 0;
+ u64 high, low, guaranteed, nom, min_nonlinear, low_f = 0, nom_f = 0;
int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, cpunum);
struct cppc_pcc_data *pcc_ss_data = NULL;
int ret = 0, regs_in_pcc = 0;
nominal_reg = &cpc_desc->cpc_regs[NOMINAL_PERF];
low_freq_reg = &cpc_desc->cpc_regs[LOWEST_FREQ];
nom_freq_reg = &cpc_desc->cpc_regs[NOMINAL_FREQ];
+ guaranteed_reg = &cpc_desc->cpc_regs[GUARANTEED_PERF];
/* Are any of the regs PCC ?*/
if (CPC_IN_PCC(highest_reg) || CPC_IN_PCC(lowest_reg) ||
cpc_read(cpunum, nominal_reg, &nom);
perf_caps->nominal_perf = nom;
+ cpc_read(cpunum, guaranteed_reg, &guaranteed);
+ perf_caps->guaranteed_perf = guaranteed;
+
cpc_read(cpunum, lowest_non_linear_reg, &min_nonlinear);
perf_caps->lowest_nonlinear_perf = min_nonlinear;
static void __exit acpi_custom_method_exit(void)
{
- if (cm_dentry)
- debugfs_remove(cm_dentry);
+ debugfs_remove(cm_dentry);
}
module_init(acpi_custom_method_init);
if (!adev)
goto out;
- if (dev->bus == &platform_bus_type)
+ if (dev_is_platform(dev))
acpi_configure_pmsi_domain(dev);
if (type && type->setup)
}
/*
- * Support ACPI 3.0 AML Timer operand
- * Returns 64-bit free-running, monotonically increasing timer
- * with 100ns granularity
+ * Support ACPI 3.0 AML Timer operand. Returns a 64-bit free-running,
+ * monotonically increasing timer with 100ns granularity. Do not use
+ * ktime_get() to implement this function because this function may get
+ * called after timekeeping has been suspended. Note: calling this function
+ * after timekeeping has been suspended may lead to unexpected results
+ * because when timekeeping is suspended the jiffies counter is not
+ * incremented. See also timekeeping_suspend().
*/
u64 acpi_os_get_timer(void)
{
- u64 time_ns = ktime_to_ns(ktime_get());
- do_div(time_ns, 100);
- return time_ns;
+ return (get_jiffies_64() - INITIAL_JIFFIES) *
+ (ACPI_100NSEC_PER_SEC / HZ);
}
acpi_status acpi_os_read_port(acpi_io_address port, u32 * value, u32 width)
flush_workqueue(kacpid_wq);
flush_workqueue(kacpi_notify_wq);
}
+EXPORT_SYMBOL(acpi_os_wait_events_complete);
struct acpi_hp_work {
struct work_struct work;
+// SPDX-License-Identifier: GPL-2.0
/*
- * intel_pmic_bxtwc.c - Intel BXT WhiskeyCove PMIC operation region driver
+ * Intel BXT WhiskeyCove PMIC operation region driver
*
* Copyright (C) 2015 Intel Corporation. All rights reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License version
- * 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
#include <linux/init.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* Dollar Cove TI PMIC operation region driver
* Copyright (C) 2014 Intel Corporation. All rights reserved.
+// SPDX-License-Identifier: GPL-2.0
/*
* Intel CHT Whiskey Cove PMIC operation region driver
* Copyright (C) 2017 Hans de Goede <hdegoede@redhat.com>
*
* Based on various non upstream patches to support the CHT Whiskey Cove PMIC:
* Copyright (C) 2013-2015 Intel Corporation. All rights reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License version
- * 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
#include <linux/acpi.h>
+// SPDX-License-Identifier: GPL-2.0
/*
- * intel_pmic_crc.c - Intel CrystalCove PMIC operation region driver
+ * Intel CrystalCove PMIC operation region driver
*
* Copyright (C) 2014 Intel Corporation. All rights reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License version
- * 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
-#include <linux/init.h>
#include <linux/acpi.h>
+#include <linux/init.h>
#include <linux/mfd/intel_soc_pmic.h>
-#include <linux/regmap.h>
#include <linux/platform_device.h>
+#include <linux/regmap.h>
#include "intel_pmic.h"
#define PWR_SOURCE_SELECT BIT(1)
+// SPDX-License-Identifier: GPL-2.0
/*
- * intel_pmic_xpower.c - XPower AXP288 PMIC operation region driver
+ * XPower AXP288 PMIC operation region driver
*
* Copyright (C) 2014 Intel Corporation. All rights reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License version
- * 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
-#include <linux/init.h>
#include <linux/acpi.h>
+#include <linux/init.h>
#include <linux/mfd/axp20x.h>
-#include <linux/regmap.h>
#include <linux/platform_device.h>
+#include <linux/regmap.h>
#include "intel_pmic.h"
#define XPOWER_GPADC_LOW 0x5b
*/
#include <linux/acpi.h>
-#include <linux/mfd/tps68470.h>
#include <linux/init.h>
+#include <linux/mfd/tps68470.h>
#include <linux/platform_device.h>
#include <linux/regmap.h>
return found;
}
-/* total number of attributes checked by the properties code */
-#define PPTT_CHECKED_ATTRIBUTES 4
-
/**
* update_cache_properties() - Update cacheinfo for the given processor
* @this_leaf: Kernel cache info structure being updated
struct acpi_pptt_cache *found_cache,
struct acpi_pptt_processor *cpu_node)
{
- int valid_flags = 0;
-
this_leaf->fw_token = cpu_node;
- if (found_cache->flags & ACPI_PPTT_SIZE_PROPERTY_VALID) {
+ if (found_cache->flags & ACPI_PPTT_SIZE_PROPERTY_VALID)
this_leaf->size = found_cache->size;
- valid_flags++;
- }
- if (found_cache->flags & ACPI_PPTT_LINE_SIZE_VALID) {
+ if (found_cache->flags & ACPI_PPTT_LINE_SIZE_VALID)
this_leaf->coherency_line_size = found_cache->line_size;
- valid_flags++;
- }
- if (found_cache->flags & ACPI_PPTT_NUMBER_OF_SETS_VALID) {
+ if (found_cache->flags & ACPI_PPTT_NUMBER_OF_SETS_VALID)
this_leaf->number_of_sets = found_cache->number_of_sets;
- valid_flags++;
- }
- if (found_cache->flags & ACPI_PPTT_ASSOCIATIVITY_VALID) {
+ if (found_cache->flags & ACPI_PPTT_ASSOCIATIVITY_VALID)
this_leaf->ways_of_associativity = found_cache->associativity;
- valid_flags++;
- }
if (found_cache->flags & ACPI_PPTT_WRITE_POLICY_VALID) {
switch (found_cache->attributes & ACPI_PPTT_MASK_WRITE_POLICY) {
case ACPI_PPTT_CACHE_POLICY_WT:
}
}
/*
- * If the above flags are valid, and the cache type is NOCACHE
- * update the cache type as well.
+ * If cache type is NOCACHE, then the cache hasn't been specified
+ * via other mechanisms. Update the type if a cache type has been
+ * provided.
+ *
+ * Note, we assume such caches are unified based on conventional system
+ * design and known examples. Significant work is required elsewhere to
+ * fully support data/instruction only type caches which are only
+ * specified in PPTT.
*/
if (this_leaf->type == CACHE_TYPE_NOCACHE &&
- valid_flags == PPTT_CHECKED_ATTRIBUTES)
+ found_cache->flags & ACPI_PPTT_CACHE_TYPE_VALID)
this_leaf->type = CACHE_TYPE_UNIFIED;
}
static void tsc_check_state(int state)
{
switch (boot_cpu_data.x86_vendor) {
+ case X86_VENDOR_HYGON:
case X86_VENDOR_AMD:
case X86_VENDOR_INTEL:
case X86_VENDOR_CENTAUR:
/*
* The spec requires that bit 4 always be 1. If it's not set, assume
- * that the implementation doesn't support an SBS charger
+ * that the implementation doesn't support an SBS charger.
+ *
+ * And on some MacBooks a status of 0xffff is always returned, no
+ * matter whether the charger is plugged in or not, which is also
+ * wrong, so ignore the SBS charger for those too.
*/
- if (!((status >> 4) & 0x1))
+ if (!((status >> 4) & 0x1) || status == 0xffff)
return -ENODEV;
sbs->charger_present = (status >> 15) & 0x1;
hc->callback = NULL;
hc->context = NULL;
mutex_unlock(&hc->lock);
+ acpi_os_wait_events_complete();
return 0;
}
hc = acpi_driver_data(device);
acpi_ec_remove_query_handler(hc->ec, hc->query_bit);
+ acpi_os_wait_events_complete();
kfree(hc);
device->driver_data = NULL;
return 0;
}
EXPORT_SYMBOL_GPL(acpi_dma_configure);
-/**
- * acpi_dma_deconfigure - Tear-down DMA configuration for the device.
- * @dev: The pointer to the device
- */
-void acpi_dma_deconfigure(struct device *dev)
-{
- arch_teardown_dma_ops(dev);
-}
-EXPORT_SYMBOL_GPL(acpi_dma_deconfigure);
-
static void acpi_init_coherency(struct acpi_device *adev)
{
unsigned long long cca = 0;
*/
static const struct acpi_device_id i2c_multi_instantiate_ids[] = {
{"BSG1160", },
+ {"INT33FE", },
{}
};
if (!numprops)
goto out_free;
- valid = kcalloc(BITS_TO_LONGS(numprops), sizeof(long), GFP_KERNEL);
+ valid = bitmap_zalloc(numprops, GFP_KERNEL);
if (!valid)
goto out_free;
out_free:
ACPI_FREE(props);
- kfree(valid);
+ bitmap_free(valid);
}
* Bay / Cherry Trail PWM directly poked by GPU driver in win10,
* but Linux uses a separate PWM driver, harmless if not used.
*/
- ENTRY("80860F09", "1", ICPU(INTEL_FAM6_ATOM_SILVERMONT1), {}),
+ ENTRY("80860F09", "1", ICPU(INTEL_FAM6_ATOM_SILVERMONT), {}),
ENTRY("80862288", "1", ICPU(INTEL_FAM6_ATOM_AIRMONT), {}),
/*
* The INT0002 device is necessary to clear wakeup interrupt sources
config ATA_NONSTANDARD
bool
- default n
config ATA_VERBOSE_ERROR
bool "Verbose ATA error reporting"
config SATA_ZPODD
bool "SATA Zero Power Optical Disc Drive (ZPODD) support"
depends on ATA_ACPI && PM
- default n
help
This option adds support for SATA Zero Power Optical Disc
Drive (ZPODD). It requires both the ODD and the platform
config AHCI_BRCM
tristate "Broadcom AHCI SATA support"
- depends on ARCH_BRCMSTB || BMIPS_GENERIC || ARCH_BCM_NSP
+ depends on ARCH_BRCMSTB || BMIPS_GENERIC || ARCH_BCM_NSP || \
+ ARCH_BCM_63XX
help
This option enables support for the AHCI SATA3 controller found on
Broadcom SoC's.
struct clk *clks[AHCI_MAX_CLKS]; /* Optional */
struct reset_control *rsts; /* Optional */
struct regulator **target_pwrs; /* Optional */
+ struct regulator *ahci_regulator;/* Optional */
+ struct regulator *phy_regulator;/* Optional */
/*
* If platform uses PHYs. There is a 1:1 relation between the port number and
* the PHY position in this array.
#include <linux/module.h>
#include <linux/of.h>
#include <linux/platform_device.h>
+#include <linux/reset.h>
#include <linux/string.h>
#include "ahci.h"
u32 port_mask;
u32 quirks;
enum brcm_ahci_version version;
+ struct reset_control *rcdev;
};
static inline u32 brcm_sata_readreg(void __iomem *addr)
static const struct of_device_id ahci_of_match[] = {
{.compatible = "brcm,bcm7425-ahci", .data = (void *)BRCM_SATA_BCM7425},
{.compatible = "brcm,bcm7445-ahci", .data = (void *)BRCM_SATA_BCM7445},
+ {.compatible = "brcm,bcm63138-ahci", .data = (void *)BRCM_SATA_BCM7445},
{.compatible = "brcm,bcm-nsp-ahci", .data = (void *)BRCM_SATA_NSP},
{},
};
if (IS_ERR(priv->top_ctrl))
return PTR_ERR(priv->top_ctrl);
+ /* Reset is optional depending on platform */
+ priv->rcdev = devm_reset_control_get(&pdev->dev, "ahci");
+ if (!IS_ERR_OR_NULL(priv->rcdev))
+ reset_control_deassert(priv->rcdev);
+
if ((priv->version == BRCM_SATA_BCM7425) ||
(priv->version == BRCM_SATA_NSP)) {
priv->quirks |= BRCM_AHCI_QUIRK_NO_NCQ;
.port_ops = &ahci_platform_ops,
};
+static const struct ata_port_info ahci_port_info_nolpm = {
+ .flags = AHCI_FLAG_COMMON | ATA_FLAG_NO_LPM,
+ .pio_mask = ATA_PIO4,
+ .udma_mask = ATA_UDMA6,
+ .port_ops = &ahci_platform_ops,
+};
+
static struct scsi_host_template ahci_platform_sht = {
AHCI_SHT(DRV_NAME),
};
{
struct device *dev = &pdev->dev;
struct ahci_host_priv *hpriv;
+ const struct ata_port_info *port;
int rc;
hpriv = ahci_platform_get_resources(pdev,
if (of_device_is_compatible(dev->of_node, "hisilicon,hisi-ahci"))
hpriv->flags |= AHCI_HFLAG_NO_FBS | AHCI_HFLAG_NO_NCQ;
- rc = ahci_platform_init_host(pdev, hpriv, &ahci_port_info,
+ port = acpi_device_get_match_data(dev);
+ if (!port)
+ port = &ahci_port_info;
+
+ rc = ahci_platform_init_host(pdev, hpriv, port,
&ahci_platform_sht);
if (rc)
goto disable_resources;
MODULE_DEVICE_TABLE(of, ahci_of_match);
static const struct acpi_device_id ahci_acpi_match[] = {
+ { "APMC0D33", (unsigned long)&ahci_port_info_nolpm },
{ ACPI_DEVICE_CLASS(PCI_CLASS_STORAGE_SATA_AHCI, 0xffffff) },
{},
};
struct ahci_host_priv *hpriv;
int rc;
- hpriv = ahci_platform_get_resources(pdev, 0);
+ hpriv = ahci_platform_get_resources(pdev, AHCI_PLATFORM_GET_RESETS);
if (IS_ERR(hpriv))
return PTR_ERR(hpriv);
static const struct of_device_id ahci_sunxi_of_match[] = {
{ .compatible = "allwinner,sun4i-a10-ahci", },
+ { .compatible = "allwinner,sun8i-r40-ahci", },
{ },
};
MODULE_DEVICE_TABLE(of, ahci_sunxi_of_match);
* ahci_platform_enable_regulators - Enable regulators
* @hpriv: host private area to store config values
*
- * This function enables all the regulators found in
+ * This function enables all the regulators found in controller and
* hpriv->target_pwrs, if any. If a regulator fails to be enabled, it
* disables all the regulators already enabled in reverse order and
* returns an error.
{
int rc, i;
+ if (hpriv->ahci_regulator) {
+ rc = regulator_enable(hpriv->ahci_regulator);
+ if (rc)
+ return rc;
+ }
+
+ if (hpriv->phy_regulator) {
+ rc = regulator_enable(hpriv->phy_regulator);
+ if (rc)
+ goto disable_ahci_pwrs;
+ }
+
for (i = 0; i < hpriv->nports; i++) {
if (!hpriv->target_pwrs[i])
continue;
if (hpriv->target_pwrs[i])
regulator_disable(hpriv->target_pwrs[i]);
+ if (hpriv->phy_regulator)
+ regulator_disable(hpriv->phy_regulator);
+disable_ahci_pwrs:
+ if (hpriv->ahci_regulator)
+ regulator_disable(hpriv->ahci_regulator);
return rc;
}
EXPORT_SYMBOL_GPL(ahci_platform_enable_regulators);
* ahci_platform_disable_regulators - Disable regulators
* @hpriv: host private area to store config values
*
- * This function disables all regulators found in hpriv->target_pwrs.
+ * This function disables all regulators found in hpriv->target_pwrs and
+ * AHCI controller.
*/
void ahci_platform_disable_regulators(struct ahci_host_priv *hpriv)
{
continue;
regulator_disable(hpriv->target_pwrs[i]);
}
+
+ if (hpriv->ahci_regulator)
+ regulator_disable(hpriv->ahci_regulator);
+ if (hpriv->phy_regulator)
+ regulator_disable(hpriv->phy_regulator);
}
EXPORT_SYMBOL_GPL(ahci_platform_disable_regulators);
/**
/* No PHY support. Check if PHY is required. */
if (of_find_property(node, "phys", NULL)) {
dev_err(dev,
- "couldn't get PHY in node %s: ENOSYS\n",
- node->name);
+ "couldn't get PHY in node %pOFn: ENOSYS\n",
+ node);
break;
}
/* fall through */
default:
dev_err(dev,
- "couldn't get PHY in node %s: %d\n",
- node->name, rc);
+ "couldn't get PHY in node %pOFn: %d\n",
+ node, rc);
break;
}
*
* 1) mmio registers (IORESOURCE_MEM 0, mandatory)
* 2) regulator for controlling the targets power (optional)
+ * regulator for controlling the AHCI controller (optional)
* 3) 0 - AHCI_MAX_CLKS clocks, as specified in the devs devicetree node,
* or for non devicetree enabled platforms a single clock
* 4) resets, if flags has AHCI_PLATFORM_GET_RESETS (optional)
hpriv->clks[i] = clk;
}
+ hpriv->ahci_regulator = devm_regulator_get_optional(dev, "ahci");
+ if (IS_ERR(hpriv->ahci_regulator)) {
+ rc = PTR_ERR(hpriv->ahci_regulator);
+ if (rc == -EPROBE_DEFER)
+ goto err_out;
+ rc = 0;
+ hpriv->ahci_regulator = NULL;
+ }
+
+ hpriv->phy_regulator = devm_regulator_get_optional(dev, "phy");
+ if (IS_ERR(hpriv->phy_regulator)) {
+ rc = PTR_ERR(hpriv->phy_regulator);
+ if (rc == -EPROBE_DEFER)
+ goto err_out;
+ rc = 0;
+ hpriv->phy_regulator = NULL;
+ }
+
if (flags & AHCI_PLATFORM_GET_RESETS) {
hpriv->rsts = devm_reset_control_array_get_optional_shared(dev);
if (IS_ERR(hpriv->rsts)) {
if (args[0] == ATA_CMD_SMART) { /* hack -- ide driver does this too */
scsi_cmd[6] = args[3];
scsi_cmd[8] = args[1];
- scsi_cmd[10] = 0x4f;
- scsi_cmd[12] = 0xc2;
+ scsi_cmd[10] = ATA_SMART_LBAM_PASS;
+ scsi_cmd[12] = ATA_SMART_LBAH_PASS;
} else {
scsi_cmd[6] = args[1];
}
const struct ata_port_info *ppi[] = { &info, &info };
/* SB600 doesn't have secondary port wired */
- if((pdev->device == PCI_DEVICE_ID_ATI_IXP600_IDE))
+ if (pdev->device == PCI_DEVICE_ID_ATI_IXP600_IDE)
ppi[1] = &ata_dummy_port_info;
return ata_pci_bmdma_init_one(pdev, ppi, &atiixp_sht, NULL,
* start of new transfer.
*/
drv_data->dma_rx_data.port = EP93XX_DMA_IDE;
- drv_data->dma_rx_data.direction = DMA_FROM_DEVICE;
+ drv_data->dma_rx_data.direction = DMA_DEV_TO_MEM;
drv_data->dma_rx_data.name = "ep93xx-pata-rx";
drv_data->dma_rx_channel = dma_request_channel(mask,
ep93xx_pata_dma_filter, &drv_data->dma_rx_data);
return;
drv_data->dma_tx_data.port = EP93XX_DMA_IDE;
- drv_data->dma_tx_data.direction = DMA_TO_DEVICE;
+ drv_data->dma_tx_data.direction = DMA_MEM_TO_DEV;
drv_data->dma_tx_data.name = "ep93xx-pata-tx";
drv_data->dma_tx_channel = dma_request_channel(mask,
ep93xx_pata_dma_filter, &drv_data->dma_tx_data);
/* Configure receive channel direction and source address */
memset(&conf, 0, sizeof(conf));
- conf.direction = DMA_FROM_DEVICE;
+ conf.direction = DMA_DEV_TO_MEM;
conf.src_addr = drv_data->udma_in_phys;
conf.src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
if (dmaengine_slave_config(drv_data->dma_rx_channel, &conf)) {
/* Configure transmit channel direction and destination address */
memset(&conf, 0, sizeof(conf));
- conf.direction = DMA_TO_DEVICE;
+ conf.direction = DMA_MEM_TO_DEV;
conf.dst_addr = drv_data->udma_out_phys;
conf.dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
if (dmaengine_slave_config(drv_data->dma_tx_channel, &conf)) {
/* write to an LCD panel register in 8 bit GPIO mode */
static void hd44780_write_gpio8(struct hd44780 *hd, u8 val, unsigned int rs)
{
- int values[10]; /* for DATA[0-7], RS, RW */
- unsigned int i, n;
-
- for (i = 0; i < 8; i++)
- values[PIN_DATA0 + i] = !!(val & BIT(i));
- values[PIN_CTRL_RS] = rs;
- n = 9;
- if (hd->pins[PIN_CTRL_RW]) {
- values[PIN_CTRL_RW] = 0;
- n++;
- }
+ DECLARE_BITMAP(values, 10); /* for DATA[0-7], RS, RW */
+ unsigned int n;
+
+ values[0] = val;
+ __assign_bit(8, values, rs);
+ n = hd->pins[PIN_CTRL_RW] ? 10 : 9;
/* Present the data to the port */
- gpiod_set_array_value_cansleep(n, &hd->pins[PIN_DATA0], values);
+ gpiod_set_array_value_cansleep(n, &hd->pins[PIN_DATA0], NULL, values);
hd44780_strobe_gpio(hd);
}
/* write to an LCD panel register in 4 bit GPIO mode */
static void hd44780_write_gpio4(struct hd44780 *hd, u8 val, unsigned int rs)
{
- int values[10]; /* for DATA[0-7], RS, RW, but DATA[0-3] is unused */
- unsigned int i, n;
+ DECLARE_BITMAP(values, 6); /* for DATA[4-7], RS, RW */
+ unsigned int n;
/* High nibble + RS, RW */
- for (i = 4; i < 8; i++)
- values[PIN_DATA0 + i] = !!(val & BIT(i));
- values[PIN_CTRL_RS] = rs;
- n = 5;
- if (hd->pins[PIN_CTRL_RW]) {
- values[PIN_CTRL_RW] = 0;
- n++;
- }
+ values[0] = val >> 4;
+ __assign_bit(4, values, rs);
+ n = hd->pins[PIN_CTRL_RW] ? 6 : 5;
/* Present the data to the port */
- gpiod_set_array_value_cansleep(n, &hd->pins[PIN_DATA4],
- &values[PIN_DATA4]);
+ gpiod_set_array_value_cansleep(n, &hd->pins[PIN_DATA4], NULL, values);
hd44780_strobe_gpio(hd);
/* Low nibble */
- for (i = 0; i < 4; i++)
- values[PIN_DATA4 + i] = !!(val & BIT(i));
+ values[0] &= ~0x0fUL;
+ values[0] |= val & 0x0f;
/* Present the data to the port */
- gpiod_set_array_value_cansleep(n, &hd->pins[PIN_DATA4],
- &values[PIN_DATA4]);
+ gpiod_set_array_value_cansleep(n, &hd->pins[PIN_DATA4], NULL, values);
hd44780_strobe_gpio(hd);
}
/* Send 4-bits of a command to the LCD panel in raw 4 bit GPIO mode */
static void hd44780_write_cmd_raw_gpio4(struct charlcd *lcd, int cmd)
{
- int values[10]; /* for DATA[0-7], RS, RW, but DATA[0-3] is unused */
+ DECLARE_BITMAP(values, 6); /* for DATA[4-7], RS, RW */
struct hd44780 *hd = lcd->drvdata;
- unsigned int i, n;
+ unsigned int n;
/* Command nibble + RS, RW */
- for (i = 0; i < 4; i++)
- values[PIN_DATA4 + i] = !!(cmd & BIT(i));
- values[PIN_CTRL_RS] = 0;
- n = 5;
- if (hd->pins[PIN_CTRL_RW]) {
- values[PIN_CTRL_RW] = 0;
- n++;
- }
+ values[0] = cmd & 0x0f;
+ n = hd->pins[PIN_CTRL_RW] ? 6 : 5;
/* Present the data to the port */
- gpiod_set_array_value_cansleep(n, &hd->pins[PIN_DATA4],
- &values[PIN_DATA4]);
+ gpiod_set_array_value_cansleep(n, &hd->pins[PIN_DATA4], NULL, values);
hd44780_strobe_gpio(hd);
}
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/sched/topology.h>
+#include <linux/cpuset.h>
DEFINE_PER_CPU(unsigned long, freq_scale) = SCHED_CAPACITY_SCALE;
return sprintf(buf, "%lu\n", topology_get_cpu_scale(NULL, cpu->dev.id));
}
+static void update_topology_flags_workfn(struct work_struct *work);
+static DECLARE_WORK(update_topology_flags_work, update_topology_flags_workfn);
+
static ssize_t cpu_capacity_store(struct device *dev,
struct device_attribute *attr,
const char *buf,
topology_set_cpu_scale(i, new_capacity);
mutex_unlock(&cpu_scale_mutex);
+ schedule_work(&update_topology_flags_work);
+
return count;
}
}
subsys_initcall(register_cpu_capacity_sysctl);
+static int update_topology;
+
+int topology_update_cpu_topology(void)
+{
+ return update_topology;
+}
+
+/*
+ * Updating the sched_domains can't be done directly from cpufreq callbacks
+ * due to locking, so queue the work for later.
+ */
+static void update_topology_flags_workfn(struct work_struct *work)
+{
+ update_topology = 1;
+ rebuild_sched_domains();
+ pr_debug("sched_domain hierarchy rebuilt, flags updated\n");
+ update_topology = 0;
+}
+
static u32 capacity_scale;
static u32 *raw_capacity;
if (cpumask_empty(cpus_to_visit)) {
topology_normalize_cpu_scale();
+ schedule_work(&update_topology_flags_work);
free_raw_capacity();
pr_debug("cpu_capacity: parsing done\n");
schedule_work(&parsing_done_work);
this_leaf = this_cpu_ci->info_list + i;
if (this_leaf->disable_sysfs)
continue;
+ if (this_leaf->type == CACHE_TYPE_NOCACHE)
+ break;
cache_groups = cache_get_attribute_groups(this_leaf);
ci_dev = cpu_device_create(parent, this_leaf, cache_groups,
"index%1u", i);
if (ret)
goto pinctrl_bind_failed;
- ret = dma_configure(dev);
- if (ret)
- goto dma_failed;
+ if (dev->bus->dma_configure) {
+ ret = dev->bus->dma_configure(dev);
+ if (ret)
+ goto dma_failed;
+ }
if (driver_sysfs_add(dev)) {
printk(KERN_ERR "%s: driver_sysfs_add(%s) failed\n",
goto done;
probe_failed:
- dma_deconfigure(dev);
+ arch_teardown_dma_ops(dev);
dma_failed:
if (dev->bus)
blocking_notifier_call_chain(&dev->bus->p->bus_notifier,
drv->remove(dev);
device_links_driver_cleanup(dev);
- dma_deconfigure(dev);
+ arch_teardown_dma_ops(dev);
devres_release_all(dev);
dev->driver = NULL;
}
#ifndef ARCH_HAS_DMA_GET_REQUIRED_MASK
-u64 dma_get_required_mask(struct device *dev)
+static u64 dma_default_get_required_mask(struct device *dev)
{
u32 low_totalram = ((max_pfn - 1) << PAGE_SHIFT);
u32 high_totalram = ((max_pfn - 1) >> (32 - PAGE_SHIFT));
}
return mask;
}
+
+u64 dma_get_required_mask(struct device *dev)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+
+ if (ops->get_required_mask)
+ return ops->get_required_mask(dev);
+ return dma_default_get_required_mask(dev);
+}
EXPORT_SYMBOL_GPL(dma_get_required_mask);
#endif
return -EAGAIN;
}
+ /* Default to shallowest state. */
+ if (!genpd->gov)
+ genpd->state_idx = 0;
+
if (genpd->power_off) {
int ret;
ret = genpd_set_default_power_state(genpd);
if (ret)
return ret;
+ } else if (!gov) {
+ pr_warn("%s : no governor for states\n", genpd->name);
}
device_initialize(&genpd->dev);
*
* Returns the device states parsed from the OF node. The memory for the states
* is allocated by this function and is the responsibility of the caller to
- * free the memory after use. If no domain idle states is found it returns
- * -EINVAL and in case of errors, a negative error code.
+ * free the memory after use. If any or zero compatible domain idle states is
+ * found it returns 0 and in case of errors, a negative error code is returned.
*/
int of_genpd_parse_idle_states(struct device_node *dn,
struct genpd_power_state **states, int *n)
int ret;
ret = genpd_iterate_idle_states(dn, NULL);
- if (ret <= 0)
- return ret < 0 ? ret : -EINVAL;
+ if (ret < 0)
+ return ret;
+
+ if (!ret) {
+ *states = NULL;
+ *n = 0;
+ return 0;
+ }
st = kcalloc(ret, sizeof(*st), GFP_KERNEL);
if (!st)
bool (*readable_reg)(struct device *dev, unsigned int reg);
bool (*volatile_reg)(struct device *dev, unsigned int reg);
bool (*precious_reg)(struct device *dev, unsigned int reg);
+ bool (*writeable_noinc_reg)(struct device *dev, unsigned int reg);
bool (*readable_noinc_reg)(struct device *dev, unsigned int reg);
const struct regmap_access_table *wr_table;
const struct regmap_access_table *rd_table;
const struct regmap_access_table *volatile_table;
const struct regmap_access_table *precious_table;
+ const struct regmap_access_table *wr_noinc_table;
const struct regmap_access_table *rd_noinc_table;
int (*reg_read)(void *context, unsigned int reg, unsigned int *val);
/* if set, converts bulk read to single read */
bool use_single_read;
- /* if set, converts bulk read to single read */
+ /* if set, converts bulk write to single write */
bool use_single_write;
/* if set, the device supports multi write mode */
bool can_multi_write;
bool regmap_readable(struct regmap *map, unsigned int reg);
bool regmap_volatile(struct regmap *map, unsigned int reg);
bool regmap_precious(struct regmap *map, unsigned int reg);
+bool regmap_writeable_noinc(struct regmap *map, unsigned int reg);
bool regmap_readable_noinc(struct regmap *map, unsigned int reg);
int _regmap_write(struct regmap *map, unsigned int reg,
*/
#undef LOG_DEVICE
+#ifdef LOG_DEVICE
+static inline bool regmap_should_log(struct regmap *map)
+{
+ return (map->dev && strcmp(dev_name(map->dev), LOG_DEVICE) == 0);
+}
+#else
+static inline bool regmap_should_log(struct regmap *map) { return false; }
+#endif
+
+
static int _regmap_update_bits(struct regmap *map, unsigned int reg,
unsigned int mask, unsigned int val,
bool *change, bool force_write);
return false;
}
+bool regmap_writeable_noinc(struct regmap *map, unsigned int reg)
+{
+ if (map->writeable_noinc_reg)
+ return map->writeable_noinc_reg(map->dev, reg);
+
+ if (map->wr_noinc_table)
+ return regmap_check_range_table(map, reg, map->wr_noinc_table);
+
+ return true;
+}
+
bool regmap_readable_noinc(struct regmap *map, unsigned int reg)
{
if (map->readable_noinc_reg)
map->reg_stride_order = ilog2(map->reg_stride);
else
map->reg_stride_order = -1;
- map->use_single_read = config->use_single_rw || !bus || !bus->read;
- map->use_single_write = config->use_single_rw || !bus || !bus->write;
+ map->use_single_read = config->use_single_read || !bus || !bus->read;
+ map->use_single_write = config->use_single_write || !bus || !bus->write;
map->can_multi_write = config->can_multi_write && bus && bus->write;
if (bus) {
map->max_raw_read = bus->max_raw_read;
map->rd_table = config->rd_table;
map->volatile_table = config->volatile_table;
map->precious_table = config->precious_table;
+ map->wr_noinc_table = config->wr_noinc_table;
map->rd_noinc_table = config->rd_noinc_table;
map->writeable_reg = config->writeable_reg;
map->readable_reg = config->readable_reg;
map->volatile_reg = config->volatile_reg;
map->precious_reg = config->precious_reg;
+ map->writeable_noinc_reg = config->writeable_noinc_reg;
map->readable_noinc_reg = config->readable_noinc_reg;
map->cache_type = config->cache_type;
map->readable_reg = config->readable_reg;
map->volatile_reg = config->volatile_reg;
map->precious_reg = config->precious_reg;
+ map->writeable_noinc_reg = config->writeable_noinc_reg;
map->readable_noinc_reg = config->readable_noinc_reg;
map->cache_type = config->cache_type;
}
}
-#ifdef LOG_DEVICE
- if (map->dev && strcmp(dev_name(map->dev), LOG_DEVICE) == 0)
+ if (regmap_should_log(map))
dev_info(map->dev, "%x <= %x\n", reg, val);
-#endif
trace_regmap_reg_write(map, reg, val);
}
EXPORT_SYMBOL_GPL(regmap_raw_write);
+/**
+ * regmap_noinc_write(): Write data from a register without incrementing the
+ * register number
+ *
+ * @map: Register map to write to
+ * @reg: Register to write to
+ * @val: Pointer to data buffer
+ * @val_len: Length of output buffer in bytes.
+ *
+ * The regmap API usually assumes that bulk bus write operations will write a
+ * range of registers. Some devices have certain registers for which a write
+ * operation can write to an internal FIFO.
+ *
+ * The target register must be volatile but registers after it can be
+ * completely unrelated cacheable registers.
+ *
+ * This will attempt multiple writes as required to write val_len bytes.
+ *
+ * A value of zero will be returned on success, a negative errno will be
+ * returned in error cases.
+ */
+int regmap_noinc_write(struct regmap *map, unsigned int reg,
+ const void *val, size_t val_len)
+{
+ size_t write_len;
+ int ret;
+
+ if (!map->bus)
+ return -EINVAL;
+ if (!map->bus->write)
+ return -ENOTSUPP;
+ if (val_len % map->format.val_bytes)
+ return -EINVAL;
+ if (!IS_ALIGNED(reg, map->reg_stride))
+ return -EINVAL;
+ if (val_len == 0)
+ return -EINVAL;
+
+ map->lock(map->lock_arg);
+
+ if (!regmap_volatile(map, reg) || !regmap_writeable_noinc(map, reg)) {
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+
+ while (val_len) {
+ if (map->max_raw_write && map->max_raw_write < val_len)
+ write_len = map->max_raw_write;
+ else
+ write_len = val_len;
+ ret = _regmap_raw_write(map, reg, val, write_len);
+ if (ret)
+ goto out_unlock;
+ val = ((u8 *)val) + write_len;
+ val_len -= write_len;
+ }
+
+out_unlock:
+ map->unlock(map->lock_arg);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(regmap_noinc_write);
+
/**
* regmap_field_update_bits_base() - Perform a read/modify/write cycle a
* register field.
ret = map->reg_read(context, reg, val);
if (ret == 0) {
-#ifdef LOG_DEVICE
- if (map->dev && strcmp(dev_name(map->dev), LOG_DEVICE) == 0)
+ if (regmap_should_log(map))
dev_info(map->dev, "%x => %x\n", reg, *val);
-#endif
trace_regmap_reg_read(map, reg, *val);
+++ /dev/null
-/*
-
- Linux Driver for Mylex DAC960/AcceleRAID/eXtremeRAID PCI RAID Controllers
-
- Copyright 1998-2001 by Leonard N. Zubkoff <lnz@dandelion.com>
- Portions Copyright 2002 by Mylex (An IBM Business Unit)
-
- This program is free software; you may redistribute and/or modify it under
- the terms of the GNU General Public License Version 2 as published by the
- Free Software Foundation.
-
- This program is distributed in the hope that it will be useful, but
- WITHOUT ANY WARRANTY, without even the implied warranty of MERCHANTABILITY
- or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- for complete details.
-
-*/
-
-
-#define DAC960_DriverVersion "2.5.49"
-#define DAC960_DriverDate "21 Aug 2007"
-
-
-#include <linux/compiler.h>
-#include <linux/module.h>
-#include <linux/types.h>
-#include <linux/miscdevice.h>
-#include <linux/blkdev.h>
-#include <linux/bio.h>
-#include <linux/completion.h>
-#include <linux/delay.h>
-#include <linux/genhd.h>
-#include <linux/hdreg.h>
-#include <linux/blkpg.h>
-#include <linux/dma-mapping.h>
-#include <linux/interrupt.h>
-#include <linux/ioport.h>
-#include <linux/mm.h>
-#include <linux/slab.h>
-#include <linux/mutex.h>
-#include <linux/proc_fs.h>
-#include <linux/seq_file.h>
-#include <linux/reboot.h>
-#include <linux/spinlock.h>
-#include <linux/timer.h>
-#include <linux/pci.h>
-#include <linux/init.h>
-#include <linux/jiffies.h>
-#include <linux/random.h>
-#include <linux/scatterlist.h>
-#include <asm/io.h>
-#include <linux/uaccess.h>
-#include "DAC960.h"
-
-#define DAC960_GAM_MINOR 252
-
-
-static DEFINE_MUTEX(DAC960_mutex);
-static DAC960_Controller_T *DAC960_Controllers[DAC960_MaxControllers];
-static int DAC960_ControllerCount;
-static struct proc_dir_entry *DAC960_ProcDirectoryEntry;
-
-static long disk_size(DAC960_Controller_T *p, int drive_nr)
-{
- if (p->FirmwareType == DAC960_V1_Controller) {
- if (drive_nr >= p->LogicalDriveCount)
- return 0;
- return p->V1.LogicalDriveInformation[drive_nr].
- LogicalDriveSize;
- } else {
- DAC960_V2_LogicalDeviceInfo_T *i =
- p->V2.LogicalDeviceInformation[drive_nr];
- if (i == NULL)
- return 0;
- return i->ConfigurableDeviceSize;
- }
-}
-
-static int DAC960_open(struct block_device *bdev, fmode_t mode)
-{
- struct gendisk *disk = bdev->bd_disk;
- DAC960_Controller_T *p = disk->queue->queuedata;
- int drive_nr = (long)disk->private_data;
- int ret = -ENXIO;
-
- mutex_lock(&DAC960_mutex);
- if (p->FirmwareType == DAC960_V1_Controller) {
- if (p->V1.LogicalDriveInformation[drive_nr].
- LogicalDriveState == DAC960_V1_LogicalDrive_Offline)
- goto out;
- } else {
- DAC960_V2_LogicalDeviceInfo_T *i =
- p->V2.LogicalDeviceInformation[drive_nr];
- if (!i || i->LogicalDeviceState == DAC960_V2_LogicalDevice_Offline)
- goto out;
- }
-
- check_disk_change(bdev);
-
- if (!get_capacity(p->disks[drive_nr]))
- goto out;
- ret = 0;
-out:
- mutex_unlock(&DAC960_mutex);
- return ret;
-}
-
-static int DAC960_getgeo(struct block_device *bdev, struct hd_geometry *geo)
-{
- struct gendisk *disk = bdev->bd_disk;
- DAC960_Controller_T *p = disk->queue->queuedata;
- int drive_nr = (long)disk->private_data;
-
- if (p->FirmwareType == DAC960_V1_Controller) {
- geo->heads = p->V1.GeometryTranslationHeads;
- geo->sectors = p->V1.GeometryTranslationSectors;
- geo->cylinders = p->V1.LogicalDriveInformation[drive_nr].
- LogicalDriveSize / (geo->heads * geo->sectors);
- } else {
- DAC960_V2_LogicalDeviceInfo_T *i =
- p->V2.LogicalDeviceInformation[drive_nr];
- switch (i->DriveGeometry) {
- case DAC960_V2_Geometry_128_32:
- geo->heads = 128;
- geo->sectors = 32;
- break;
- case DAC960_V2_Geometry_255_63:
- geo->heads = 255;
- geo->sectors = 63;
- break;
- default:
- DAC960_Error("Illegal Logical Device Geometry %d\n",
- p, i->DriveGeometry);
- return -EINVAL;
- }
-
- geo->cylinders = i->ConfigurableDeviceSize /
- (geo->heads * geo->sectors);
- }
-
- return 0;
-}
-
-static unsigned int DAC960_check_events(struct gendisk *disk,
- unsigned int clearing)
-{
- DAC960_Controller_T *p = disk->queue->queuedata;
- int drive_nr = (long)disk->private_data;
-
- if (!p->LogicalDriveInitiallyAccessible[drive_nr])
- return DISK_EVENT_MEDIA_CHANGE;
- return 0;
-}
-
-static int DAC960_revalidate_disk(struct gendisk *disk)
-{
- DAC960_Controller_T *p = disk->queue->queuedata;
- int unit = (long)disk->private_data;
-
- set_capacity(disk, disk_size(p, unit));
- return 0;
-}
-
-static const struct block_device_operations DAC960_BlockDeviceOperations = {
- .owner = THIS_MODULE,
- .open = DAC960_open,
- .getgeo = DAC960_getgeo,
- .check_events = DAC960_check_events,
- .revalidate_disk = DAC960_revalidate_disk,
-};
-
-
-/*
- DAC960_AnnounceDriver announces the Driver Version and Date, Author's Name,
- Copyright Notice, and Electronic Mail Address.
-*/
-
-static void DAC960_AnnounceDriver(DAC960_Controller_T *Controller)
-{
- DAC960_Announce("***** DAC960 RAID Driver Version "
- DAC960_DriverVersion " of "
- DAC960_DriverDate " *****\n", Controller);
- DAC960_Announce("Copyright 1998-2001 by Leonard N. Zubkoff "
- "<lnz@dandelion.com>\n", Controller);
-}
-
-
-/*
- DAC960_Failure prints a standardized error message, and then returns false.
-*/
-
-static bool DAC960_Failure(DAC960_Controller_T *Controller,
- unsigned char *ErrorMessage)
-{
- DAC960_Error("While configuring DAC960 PCI RAID Controller at\n",
- Controller);
- if (Controller->IO_Address == 0)
- DAC960_Error("PCI Bus %d Device %d Function %d I/O Address N/A "
- "PCI Address 0x%X\n", Controller,
- Controller->Bus, Controller->Device,
- Controller->Function, Controller->PCI_Address);
- else DAC960_Error("PCI Bus %d Device %d Function %d I/O Address "
- "0x%X PCI Address 0x%X\n", Controller,
- Controller->Bus, Controller->Device,
- Controller->Function, Controller->IO_Address,
- Controller->PCI_Address);
- DAC960_Error("%s FAILED - DETACHING\n", Controller, ErrorMessage);
- return false;
-}
-
-/*
- init_dma_loaf() and slice_dma_loaf() are helper functions for
- aggregating the dma-mapped memory for a well-known collection of
- data structures that are of different lengths.
-
- These routines don't guarantee any alignment. The caller must
- include any space needed for alignment in the sizes of the structures
- that are passed in.
- */
-
-static bool init_dma_loaf(struct pci_dev *dev, struct dma_loaf *loaf,
- size_t len)
-{
- void *cpu_addr;
- dma_addr_t dma_handle;
-
- cpu_addr = pci_alloc_consistent(dev, len, &dma_handle);
- if (cpu_addr == NULL)
- return false;
-
- loaf->cpu_free = loaf->cpu_base = cpu_addr;
- loaf->dma_free =loaf->dma_base = dma_handle;
- loaf->length = len;
- memset(cpu_addr, 0, len);
- return true;
-}
-
-static void *slice_dma_loaf(struct dma_loaf *loaf, size_t len,
- dma_addr_t *dma_handle)
-{
- void *cpu_end = loaf->cpu_free + len;
- void *cpu_addr = loaf->cpu_free;
-
- BUG_ON(cpu_end > loaf->cpu_base + loaf->length);
- *dma_handle = loaf->dma_free;
- loaf->cpu_free = cpu_end;
- loaf->dma_free += len;
- return cpu_addr;
-}
-
-static void free_dma_loaf(struct pci_dev *dev, struct dma_loaf *loaf_handle)
-{
- if (loaf_handle->cpu_base != NULL)
- pci_free_consistent(dev, loaf_handle->length,
- loaf_handle->cpu_base, loaf_handle->dma_base);
-}
-
-
-/*
- DAC960_CreateAuxiliaryStructures allocates and initializes the auxiliary
- data structures for Controller. It returns true on success and false on
- failure.
-*/
-
-static bool DAC960_CreateAuxiliaryStructures(DAC960_Controller_T *Controller)
-{
- int CommandAllocationLength, CommandAllocationGroupSize;
- int CommandsRemaining = 0, CommandIdentifier, CommandGroupByteCount;
- void *AllocationPointer = NULL;
- void *ScatterGatherCPU = NULL;
- dma_addr_t ScatterGatherDMA;
- struct dma_pool *ScatterGatherPool;
- void *RequestSenseCPU = NULL;
- dma_addr_t RequestSenseDMA;
- struct dma_pool *RequestSensePool = NULL;
-
- if (Controller->FirmwareType == DAC960_V1_Controller)
- {
- CommandAllocationLength = offsetof(DAC960_Command_T, V1.EndMarker);
- CommandAllocationGroupSize = DAC960_V1_CommandAllocationGroupSize;
- ScatterGatherPool = dma_pool_create("DAC960_V1_ScatterGather",
- &Controller->PCIDevice->dev,
- DAC960_V1_ScatterGatherLimit * sizeof(DAC960_V1_ScatterGatherSegment_T),
- sizeof(DAC960_V1_ScatterGatherSegment_T), 0);
- if (ScatterGatherPool == NULL)
- return DAC960_Failure(Controller,
- "AUXILIARY STRUCTURE CREATION (SG)");
- Controller->ScatterGatherPool = ScatterGatherPool;
- }
- else
- {
- CommandAllocationLength = offsetof(DAC960_Command_T, V2.EndMarker);
- CommandAllocationGroupSize = DAC960_V2_CommandAllocationGroupSize;
- ScatterGatherPool = dma_pool_create("DAC960_V2_ScatterGather",
- &Controller->PCIDevice->dev,
- DAC960_V2_ScatterGatherLimit * sizeof(DAC960_V2_ScatterGatherSegment_T),
- sizeof(DAC960_V2_ScatterGatherSegment_T), 0);
- if (ScatterGatherPool == NULL)
- return DAC960_Failure(Controller,
- "AUXILIARY STRUCTURE CREATION (SG)");
- RequestSensePool = dma_pool_create("DAC960_V2_RequestSense",
- &Controller->PCIDevice->dev, sizeof(DAC960_SCSI_RequestSense_T),
- sizeof(int), 0);
- if (RequestSensePool == NULL) {
- dma_pool_destroy(ScatterGatherPool);
- return DAC960_Failure(Controller,
- "AUXILIARY STRUCTURE CREATION (SG)");
- }
- Controller->ScatterGatherPool = ScatterGatherPool;
- Controller->V2.RequestSensePool = RequestSensePool;
- }
- Controller->CommandAllocationGroupSize = CommandAllocationGroupSize;
- Controller->FreeCommands = NULL;
- for (CommandIdentifier = 1;
- CommandIdentifier <= Controller->DriverQueueDepth;
- CommandIdentifier++)
- {
- DAC960_Command_T *Command;
- if (--CommandsRemaining <= 0)
- {
- CommandsRemaining =
- Controller->DriverQueueDepth - CommandIdentifier + 1;
- if (CommandsRemaining > CommandAllocationGroupSize)
- CommandsRemaining = CommandAllocationGroupSize;
- CommandGroupByteCount =
- CommandsRemaining * CommandAllocationLength;
- AllocationPointer = kzalloc(CommandGroupByteCount, GFP_ATOMIC);
- if (AllocationPointer == NULL)
- return DAC960_Failure(Controller,
- "AUXILIARY STRUCTURE CREATION");
- }
- Command = (DAC960_Command_T *) AllocationPointer;
- AllocationPointer += CommandAllocationLength;
- Command->CommandIdentifier = CommandIdentifier;
- Command->Controller = Controller;
- Command->Next = Controller->FreeCommands;
- Controller->FreeCommands = Command;
- Controller->Commands[CommandIdentifier-1] = Command;
- ScatterGatherCPU = dma_pool_alloc(ScatterGatherPool, GFP_ATOMIC,
- &ScatterGatherDMA);
- if (ScatterGatherCPU == NULL)
- return DAC960_Failure(Controller, "AUXILIARY STRUCTURE CREATION");
-
- if (RequestSensePool != NULL) {
- RequestSenseCPU = dma_pool_alloc(RequestSensePool, GFP_ATOMIC,
- &RequestSenseDMA);
- if (RequestSenseCPU == NULL) {
- dma_pool_free(ScatterGatherPool, ScatterGatherCPU,
- ScatterGatherDMA);
- return DAC960_Failure(Controller,
- "AUXILIARY STRUCTURE CREATION");
- }
- }
- if (Controller->FirmwareType == DAC960_V1_Controller) {
- Command->cmd_sglist = Command->V1.ScatterList;
- Command->V1.ScatterGatherList =
- (DAC960_V1_ScatterGatherSegment_T *)ScatterGatherCPU;
- Command->V1.ScatterGatherListDMA = ScatterGatherDMA;
- sg_init_table(Command->cmd_sglist, DAC960_V1_ScatterGatherLimit);
- } else {
- Command->cmd_sglist = Command->V2.ScatterList;
- Command->V2.ScatterGatherList =
- (DAC960_V2_ScatterGatherSegment_T *)ScatterGatherCPU;
- Command->V2.ScatterGatherListDMA = ScatterGatherDMA;
- Command->V2.RequestSense =
- (DAC960_SCSI_RequestSense_T *)RequestSenseCPU;
- Command->V2.RequestSenseDMA = RequestSenseDMA;
- sg_init_table(Command->cmd_sglist, DAC960_V2_ScatterGatherLimit);
- }
- }
- return true;
-}
-
-
-/*
- DAC960_DestroyAuxiliaryStructures deallocates the auxiliary data
- structures for Controller.
-*/
-
-static void DAC960_DestroyAuxiliaryStructures(DAC960_Controller_T *Controller)
-{
- int i;
- struct dma_pool *ScatterGatherPool = Controller->ScatterGatherPool;
- struct dma_pool *RequestSensePool = NULL;
- void *ScatterGatherCPU;
- dma_addr_t ScatterGatherDMA;
- void *RequestSenseCPU;
- dma_addr_t RequestSenseDMA;
- DAC960_Command_T *CommandGroup = NULL;
-
-
- if (Controller->FirmwareType == DAC960_V2_Controller)
- RequestSensePool = Controller->V2.RequestSensePool;
-
- Controller->FreeCommands = NULL;
- for (i = 0; i < Controller->DriverQueueDepth; i++)
- {
- DAC960_Command_T *Command = Controller->Commands[i];
-
- if (Command == NULL)
- continue;
-
- if (Controller->FirmwareType == DAC960_V1_Controller) {
- ScatterGatherCPU = (void *)Command->V1.ScatterGatherList;
- ScatterGatherDMA = Command->V1.ScatterGatherListDMA;
- RequestSenseCPU = NULL;
- RequestSenseDMA = (dma_addr_t)0;
- } else {
- ScatterGatherCPU = (void *)Command->V2.ScatterGatherList;
- ScatterGatherDMA = Command->V2.ScatterGatherListDMA;
- RequestSenseCPU = (void *)Command->V2.RequestSense;
- RequestSenseDMA = Command->V2.RequestSenseDMA;
- }
- if (ScatterGatherCPU != NULL)
- dma_pool_free(ScatterGatherPool, ScatterGatherCPU, ScatterGatherDMA);
- if (RequestSenseCPU != NULL)
- dma_pool_free(RequestSensePool, RequestSenseCPU, RequestSenseDMA);
-
- if ((Command->CommandIdentifier
- % Controller->CommandAllocationGroupSize) == 1) {
- /*
- * We can't free the group of commands until all of the
- * request sense and scatter gather dma structures are free.
- * Remember the beginning of the group, but don't free it
- * until we've reached the beginning of the next group.
- */
- kfree(CommandGroup);
- CommandGroup = Command;
- }
- Controller->Commands[i] = NULL;
- }
- kfree(CommandGroup);
-
- if (Controller->CombinedStatusBuffer != NULL)
- {
- kfree(Controller->CombinedStatusBuffer);
- Controller->CombinedStatusBuffer = NULL;
- Controller->CurrentStatusBuffer = NULL;
- }
-
- dma_pool_destroy(ScatterGatherPool);
- if (Controller->FirmwareType == DAC960_V1_Controller)
- return;
-
- dma_pool_destroy(RequestSensePool);
-
- for (i = 0; i < DAC960_MaxLogicalDrives; i++) {
- kfree(Controller->V2.LogicalDeviceInformation[i]);
- Controller->V2.LogicalDeviceInformation[i] = NULL;
- }
-
- for (i = 0; i < DAC960_V2_MaxPhysicalDevices; i++)
- {
- kfree(Controller->V2.PhysicalDeviceInformation[i]);
- Controller->V2.PhysicalDeviceInformation[i] = NULL;
- kfree(Controller->V2.InquiryUnitSerialNumber[i]);
- Controller->V2.InquiryUnitSerialNumber[i] = NULL;
- }
-}
-
-
-/*
- DAC960_V1_ClearCommand clears critical fields of Command for DAC960 V1
- Firmware Controllers.
-*/
-
-static inline void DAC960_V1_ClearCommand(DAC960_Command_T *Command)
-{
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- memset(CommandMailbox, 0, sizeof(DAC960_V1_CommandMailbox_T));
- Command->V1.CommandStatus = 0;
-}
-
-
-/*
- DAC960_V2_ClearCommand clears critical fields of Command for DAC960 V2
- Firmware Controllers.
-*/
-
-static inline void DAC960_V2_ClearCommand(DAC960_Command_T *Command)
-{
- DAC960_V2_CommandMailbox_T *CommandMailbox = &Command->V2.CommandMailbox;
- memset(CommandMailbox, 0, sizeof(DAC960_V2_CommandMailbox_T));
- Command->V2.CommandStatus = 0;
-}
-
-
-/*
- DAC960_AllocateCommand allocates a Command structure from Controller's
- free list. During driver initialization, a special initialization command
- has been placed on the free list to guarantee that command allocation can
- never fail.
-*/
-
-static inline DAC960_Command_T *DAC960_AllocateCommand(DAC960_Controller_T
- *Controller)
-{
- DAC960_Command_T *Command = Controller->FreeCommands;
- if (Command == NULL) return NULL;
- Controller->FreeCommands = Command->Next;
- Command->Next = NULL;
- return Command;
-}
-
-
-/*
- DAC960_DeallocateCommand deallocates Command, returning it to Controller's
- free list.
-*/
-
-static inline void DAC960_DeallocateCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
-
- Command->Request = NULL;
- Command->Next = Controller->FreeCommands;
- Controller->FreeCommands = Command;
-}
-
-
-/*
- DAC960_WaitForCommand waits for a wake_up on Controller's Command Wait Queue.
-*/
-
-static void DAC960_WaitForCommand(DAC960_Controller_T *Controller)
-{
- spin_unlock_irq(&Controller->queue_lock);
- __wait_event(Controller->CommandWaitQueue, Controller->FreeCommands);
- spin_lock_irq(&Controller->queue_lock);
-}
-
-/*
- DAC960_GEM_QueueCommand queues Command for DAC960 GEM Series Controllers.
-*/
-
-static void DAC960_GEM_QueueCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V2_CommandMailbox_T *CommandMailbox = &Command->V2.CommandMailbox;
- DAC960_V2_CommandMailbox_T *NextCommandMailbox =
- Controller->V2.NextCommandMailbox;
-
- CommandMailbox->Common.CommandIdentifier = Command->CommandIdentifier;
- DAC960_GEM_WriteCommandMailbox(NextCommandMailbox, CommandMailbox);
-
- if (Controller->V2.PreviousCommandMailbox1->Words[0] == 0 ||
- Controller->V2.PreviousCommandMailbox2->Words[0] == 0)
- DAC960_GEM_MemoryMailboxNewCommand(ControllerBaseAddress);
-
- Controller->V2.PreviousCommandMailbox2 =
- Controller->V2.PreviousCommandMailbox1;
- Controller->V2.PreviousCommandMailbox1 = NextCommandMailbox;
-
- if (++NextCommandMailbox > Controller->V2.LastCommandMailbox)
- NextCommandMailbox = Controller->V2.FirstCommandMailbox;
-
- Controller->V2.NextCommandMailbox = NextCommandMailbox;
-}
-
-/*
- DAC960_BA_QueueCommand queues Command for DAC960 BA Series Controllers.
-*/
-
-static void DAC960_BA_QueueCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V2_CommandMailbox_T *CommandMailbox = &Command->V2.CommandMailbox;
- DAC960_V2_CommandMailbox_T *NextCommandMailbox =
- Controller->V2.NextCommandMailbox;
- CommandMailbox->Common.CommandIdentifier = Command->CommandIdentifier;
- DAC960_BA_WriteCommandMailbox(NextCommandMailbox, CommandMailbox);
- if (Controller->V2.PreviousCommandMailbox1->Words[0] == 0 ||
- Controller->V2.PreviousCommandMailbox2->Words[0] == 0)
- DAC960_BA_MemoryMailboxNewCommand(ControllerBaseAddress);
- Controller->V2.PreviousCommandMailbox2 =
- Controller->V2.PreviousCommandMailbox1;
- Controller->V2.PreviousCommandMailbox1 = NextCommandMailbox;
- if (++NextCommandMailbox > Controller->V2.LastCommandMailbox)
- NextCommandMailbox = Controller->V2.FirstCommandMailbox;
- Controller->V2.NextCommandMailbox = NextCommandMailbox;
-}
-
-
-/*
- DAC960_LP_QueueCommand queues Command for DAC960 LP Series Controllers.
-*/
-
-static void DAC960_LP_QueueCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V2_CommandMailbox_T *CommandMailbox = &Command->V2.CommandMailbox;
- DAC960_V2_CommandMailbox_T *NextCommandMailbox =
- Controller->V2.NextCommandMailbox;
- CommandMailbox->Common.CommandIdentifier = Command->CommandIdentifier;
- DAC960_LP_WriteCommandMailbox(NextCommandMailbox, CommandMailbox);
- if (Controller->V2.PreviousCommandMailbox1->Words[0] == 0 ||
- Controller->V2.PreviousCommandMailbox2->Words[0] == 0)
- DAC960_LP_MemoryMailboxNewCommand(ControllerBaseAddress);
- Controller->V2.PreviousCommandMailbox2 =
- Controller->V2.PreviousCommandMailbox1;
- Controller->V2.PreviousCommandMailbox1 = NextCommandMailbox;
- if (++NextCommandMailbox > Controller->V2.LastCommandMailbox)
- NextCommandMailbox = Controller->V2.FirstCommandMailbox;
- Controller->V2.NextCommandMailbox = NextCommandMailbox;
-}
-
-
-/*
- DAC960_LA_QueueCommandDualMode queues Command for DAC960 LA Series
- Controllers with Dual Mode Firmware.
-*/
-
-static void DAC960_LA_QueueCommandDualMode(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- DAC960_V1_CommandMailbox_T *NextCommandMailbox =
- Controller->V1.NextCommandMailbox;
- CommandMailbox->Common.CommandIdentifier = Command->CommandIdentifier;
- DAC960_LA_WriteCommandMailbox(NextCommandMailbox, CommandMailbox);
- if (Controller->V1.PreviousCommandMailbox1->Words[0] == 0 ||
- Controller->V1.PreviousCommandMailbox2->Words[0] == 0)
- DAC960_LA_MemoryMailboxNewCommand(ControllerBaseAddress);
- Controller->V1.PreviousCommandMailbox2 =
- Controller->V1.PreviousCommandMailbox1;
- Controller->V1.PreviousCommandMailbox1 = NextCommandMailbox;
- if (++NextCommandMailbox > Controller->V1.LastCommandMailbox)
- NextCommandMailbox = Controller->V1.FirstCommandMailbox;
- Controller->V1.NextCommandMailbox = NextCommandMailbox;
-}
-
-
-/*
- DAC960_LA_QueueCommandSingleMode queues Command for DAC960 LA Series
- Controllers with Single Mode Firmware.
-*/
-
-static void DAC960_LA_QueueCommandSingleMode(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- DAC960_V1_CommandMailbox_T *NextCommandMailbox =
- Controller->V1.NextCommandMailbox;
- CommandMailbox->Common.CommandIdentifier = Command->CommandIdentifier;
- DAC960_LA_WriteCommandMailbox(NextCommandMailbox, CommandMailbox);
- if (Controller->V1.PreviousCommandMailbox1->Words[0] == 0 ||
- Controller->V1.PreviousCommandMailbox2->Words[0] == 0)
- DAC960_LA_HardwareMailboxNewCommand(ControllerBaseAddress);
- Controller->V1.PreviousCommandMailbox2 =
- Controller->V1.PreviousCommandMailbox1;
- Controller->V1.PreviousCommandMailbox1 = NextCommandMailbox;
- if (++NextCommandMailbox > Controller->V1.LastCommandMailbox)
- NextCommandMailbox = Controller->V1.FirstCommandMailbox;
- Controller->V1.NextCommandMailbox = NextCommandMailbox;
-}
-
-
-/*
- DAC960_PG_QueueCommandDualMode queues Command for DAC960 PG Series
- Controllers with Dual Mode Firmware.
-*/
-
-static void DAC960_PG_QueueCommandDualMode(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- DAC960_V1_CommandMailbox_T *NextCommandMailbox =
- Controller->V1.NextCommandMailbox;
- CommandMailbox->Common.CommandIdentifier = Command->CommandIdentifier;
- DAC960_PG_WriteCommandMailbox(NextCommandMailbox, CommandMailbox);
- if (Controller->V1.PreviousCommandMailbox1->Words[0] == 0 ||
- Controller->V1.PreviousCommandMailbox2->Words[0] == 0)
- DAC960_PG_MemoryMailboxNewCommand(ControllerBaseAddress);
- Controller->V1.PreviousCommandMailbox2 =
- Controller->V1.PreviousCommandMailbox1;
- Controller->V1.PreviousCommandMailbox1 = NextCommandMailbox;
- if (++NextCommandMailbox > Controller->V1.LastCommandMailbox)
- NextCommandMailbox = Controller->V1.FirstCommandMailbox;
- Controller->V1.NextCommandMailbox = NextCommandMailbox;
-}
-
-
-/*
- DAC960_PG_QueueCommandSingleMode queues Command for DAC960 PG Series
- Controllers with Single Mode Firmware.
-*/
-
-static void DAC960_PG_QueueCommandSingleMode(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- DAC960_V1_CommandMailbox_T *NextCommandMailbox =
- Controller->V1.NextCommandMailbox;
- CommandMailbox->Common.CommandIdentifier = Command->CommandIdentifier;
- DAC960_PG_WriteCommandMailbox(NextCommandMailbox, CommandMailbox);
- if (Controller->V1.PreviousCommandMailbox1->Words[0] == 0 ||
- Controller->V1.PreviousCommandMailbox2->Words[0] == 0)
- DAC960_PG_HardwareMailboxNewCommand(ControllerBaseAddress);
- Controller->V1.PreviousCommandMailbox2 =
- Controller->V1.PreviousCommandMailbox1;
- Controller->V1.PreviousCommandMailbox1 = NextCommandMailbox;
- if (++NextCommandMailbox > Controller->V1.LastCommandMailbox)
- NextCommandMailbox = Controller->V1.FirstCommandMailbox;
- Controller->V1.NextCommandMailbox = NextCommandMailbox;
-}
-
-
-/*
- DAC960_PD_QueueCommand queues Command for DAC960 PD Series Controllers.
-*/
-
-static void DAC960_PD_QueueCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- CommandMailbox->Common.CommandIdentifier = Command->CommandIdentifier;
- while (DAC960_PD_MailboxFullP(ControllerBaseAddress))
- udelay(1);
- DAC960_PD_WriteCommandMailbox(ControllerBaseAddress, CommandMailbox);
- DAC960_PD_NewCommand(ControllerBaseAddress);
-}
-
-
-/*
- DAC960_P_QueueCommand queues Command for DAC960 P Series Controllers.
-*/
-
-static void DAC960_P_QueueCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- CommandMailbox->Common.CommandIdentifier = Command->CommandIdentifier;
- switch (CommandMailbox->Common.CommandOpcode)
- {
- case DAC960_V1_Enquiry:
- CommandMailbox->Common.CommandOpcode = DAC960_V1_Enquiry_Old;
- break;
- case DAC960_V1_GetDeviceState:
- CommandMailbox->Common.CommandOpcode = DAC960_V1_GetDeviceState_Old;
- break;
- case DAC960_V1_Read:
- CommandMailbox->Common.CommandOpcode = DAC960_V1_Read_Old;
- DAC960_PD_To_P_TranslateReadWriteCommand(CommandMailbox);
- break;
- case DAC960_V1_Write:
- CommandMailbox->Common.CommandOpcode = DAC960_V1_Write_Old;
- DAC960_PD_To_P_TranslateReadWriteCommand(CommandMailbox);
- break;
- case DAC960_V1_ReadWithScatterGather:
- CommandMailbox->Common.CommandOpcode =
- DAC960_V1_ReadWithScatterGather_Old;
- DAC960_PD_To_P_TranslateReadWriteCommand(CommandMailbox);
- break;
- case DAC960_V1_WriteWithScatterGather:
- CommandMailbox->Common.CommandOpcode =
- DAC960_V1_WriteWithScatterGather_Old;
- DAC960_PD_To_P_TranslateReadWriteCommand(CommandMailbox);
- break;
- default:
- break;
- }
- while (DAC960_PD_MailboxFullP(ControllerBaseAddress))
- udelay(1);
- DAC960_PD_WriteCommandMailbox(ControllerBaseAddress, CommandMailbox);
- DAC960_PD_NewCommand(ControllerBaseAddress);
-}
-
-
-/*
- DAC960_ExecuteCommand executes Command and waits for completion.
-*/
-
-static void DAC960_ExecuteCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- DECLARE_COMPLETION_ONSTACK(Completion);
- unsigned long flags;
- Command->Completion = &Completion;
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- DAC960_QueueCommand(Command);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
-
- if (in_interrupt())
- return;
- wait_for_completion(&Completion);
-}
-
-
-/*
- DAC960_V1_ExecuteType3 executes a DAC960 V1 Firmware Controller Type 3
- Command and waits for completion. It returns true on success and false
- on failure.
-*/
-
-static bool DAC960_V1_ExecuteType3(DAC960_Controller_T *Controller,
- DAC960_V1_CommandOpcode_T CommandOpcode,
- dma_addr_t DataDMA)
-{
- DAC960_Command_T *Command = DAC960_AllocateCommand(Controller);
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- DAC960_V1_CommandStatus_T CommandStatus;
- DAC960_V1_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- CommandMailbox->Type3.CommandOpcode = CommandOpcode;
- CommandMailbox->Type3.BusAddress = DataDMA;
- DAC960_ExecuteCommand(Command);
- CommandStatus = Command->V1.CommandStatus;
- DAC960_DeallocateCommand(Command);
- return (CommandStatus == DAC960_V1_NormalCompletion);
-}
-
-
-/*
- DAC960_V1_ExecuteTypeB executes a DAC960 V1 Firmware Controller Type 3B
- Command and waits for completion. It returns true on success and false
- on failure.
-*/
-
-static bool DAC960_V1_ExecuteType3B(DAC960_Controller_T *Controller,
- DAC960_V1_CommandOpcode_T CommandOpcode,
- unsigned char CommandOpcode2,
- dma_addr_t DataDMA)
-{
- DAC960_Command_T *Command = DAC960_AllocateCommand(Controller);
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- DAC960_V1_CommandStatus_T CommandStatus;
- DAC960_V1_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- CommandMailbox->Type3B.CommandOpcode = CommandOpcode;
- CommandMailbox->Type3B.CommandOpcode2 = CommandOpcode2;
- CommandMailbox->Type3B.BusAddress = DataDMA;
- DAC960_ExecuteCommand(Command);
- CommandStatus = Command->V1.CommandStatus;
- DAC960_DeallocateCommand(Command);
- return (CommandStatus == DAC960_V1_NormalCompletion);
-}
-
-
-/*
- DAC960_V1_ExecuteType3D executes a DAC960 V1 Firmware Controller Type 3D
- Command and waits for completion. It returns true on success and false
- on failure.
-*/
-
-static bool DAC960_V1_ExecuteType3D(DAC960_Controller_T *Controller,
- DAC960_V1_CommandOpcode_T CommandOpcode,
- unsigned char Channel,
- unsigned char TargetID,
- dma_addr_t DataDMA)
-{
- DAC960_Command_T *Command = DAC960_AllocateCommand(Controller);
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- DAC960_V1_CommandStatus_T CommandStatus;
- DAC960_V1_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- CommandMailbox->Type3D.CommandOpcode = CommandOpcode;
- CommandMailbox->Type3D.Channel = Channel;
- CommandMailbox->Type3D.TargetID = TargetID;
- CommandMailbox->Type3D.BusAddress = DataDMA;
- DAC960_ExecuteCommand(Command);
- CommandStatus = Command->V1.CommandStatus;
- DAC960_DeallocateCommand(Command);
- return (CommandStatus == DAC960_V1_NormalCompletion);
-}
-
-
-/*
- DAC960_V2_GeneralInfo executes a DAC960 V2 Firmware General Information
- Reading IOCTL Command and waits for completion. It returns true on success
- and false on failure.
-
- Return data in The controller's HealthStatusBuffer, which is dma-able memory
-*/
-
-static bool DAC960_V2_GeneralInfo(DAC960_Controller_T *Controller)
-{
- DAC960_Command_T *Command = DAC960_AllocateCommand(Controller);
- DAC960_V2_CommandMailbox_T *CommandMailbox = &Command->V2.CommandMailbox;
- DAC960_V2_CommandStatus_T CommandStatus;
- DAC960_V2_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- CommandMailbox->Common.CommandOpcode = DAC960_V2_IOCTL;
- CommandMailbox->Common.CommandControlBits
- .DataTransferControllerToHost = true;
- CommandMailbox->Common.CommandControlBits
- .NoAutoRequestSense = true;
- CommandMailbox->Common.DataTransferSize = sizeof(DAC960_V2_HealthStatusBuffer_T);
- CommandMailbox->Common.IOCTL_Opcode = DAC960_V2_GetHealthStatus;
- CommandMailbox->Common.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer =
- Controller->V2.HealthStatusBufferDMA;
- CommandMailbox->Common.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->Common.DataTransferSize;
- DAC960_ExecuteCommand(Command);
- CommandStatus = Command->V2.CommandStatus;
- DAC960_DeallocateCommand(Command);
- return (CommandStatus == DAC960_V2_NormalCompletion);
-}
-
-
-/*
- DAC960_V2_ControllerInfo executes a DAC960 V2 Firmware Controller
- Information Reading IOCTL Command and waits for completion. It returns
- true on success and false on failure.
-
- Data is returned in the controller's V2.NewControllerInformation dma-able
- memory buffer.
-*/
-
-static bool DAC960_V2_NewControllerInfo(DAC960_Controller_T *Controller)
-{
- DAC960_Command_T *Command = DAC960_AllocateCommand(Controller);
- DAC960_V2_CommandMailbox_T *CommandMailbox = &Command->V2.CommandMailbox;
- DAC960_V2_CommandStatus_T CommandStatus;
- DAC960_V2_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- CommandMailbox->ControllerInfo.CommandOpcode = DAC960_V2_IOCTL;
- CommandMailbox->ControllerInfo.CommandControlBits
- .DataTransferControllerToHost = true;
- CommandMailbox->ControllerInfo.CommandControlBits
- .NoAutoRequestSense = true;
- CommandMailbox->ControllerInfo.DataTransferSize = sizeof(DAC960_V2_ControllerInfo_T);
- CommandMailbox->ControllerInfo.ControllerNumber = 0;
- CommandMailbox->ControllerInfo.IOCTL_Opcode = DAC960_V2_GetControllerInfo;
- CommandMailbox->ControllerInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer =
- Controller->V2.NewControllerInformationDMA;
- CommandMailbox->ControllerInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->ControllerInfo.DataTransferSize;
- DAC960_ExecuteCommand(Command);
- CommandStatus = Command->V2.CommandStatus;
- DAC960_DeallocateCommand(Command);
- return (CommandStatus == DAC960_V2_NormalCompletion);
-}
-
-
-/*
- DAC960_V2_LogicalDeviceInfo executes a DAC960 V2 Firmware Controller Logical
- Device Information Reading IOCTL Command and waits for completion. It
- returns true on success and false on failure.
-
- Data is returned in the controller's V2.NewLogicalDeviceInformation
-*/
-
-static bool DAC960_V2_NewLogicalDeviceInfo(DAC960_Controller_T *Controller,
- unsigned short LogicalDeviceNumber)
-{
- DAC960_Command_T *Command = DAC960_AllocateCommand(Controller);
- DAC960_V2_CommandMailbox_T *CommandMailbox = &Command->V2.CommandMailbox;
- DAC960_V2_CommandStatus_T CommandStatus;
-
- DAC960_V2_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- CommandMailbox->LogicalDeviceInfo.CommandOpcode =
- DAC960_V2_IOCTL;
- CommandMailbox->LogicalDeviceInfo.CommandControlBits
- .DataTransferControllerToHost = true;
- CommandMailbox->LogicalDeviceInfo.CommandControlBits
- .NoAutoRequestSense = true;
- CommandMailbox->LogicalDeviceInfo.DataTransferSize =
- sizeof(DAC960_V2_LogicalDeviceInfo_T);
- CommandMailbox->LogicalDeviceInfo.LogicalDevice.LogicalDeviceNumber =
- LogicalDeviceNumber;
- CommandMailbox->LogicalDeviceInfo.IOCTL_Opcode = DAC960_V2_GetLogicalDeviceInfoValid;
- CommandMailbox->LogicalDeviceInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer =
- Controller->V2.NewLogicalDeviceInformationDMA;
- CommandMailbox->LogicalDeviceInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->LogicalDeviceInfo.DataTransferSize;
- DAC960_ExecuteCommand(Command);
- CommandStatus = Command->V2.CommandStatus;
- DAC960_DeallocateCommand(Command);
- return (CommandStatus == DAC960_V2_NormalCompletion);
-}
-
-
-/*
- DAC960_V2_PhysicalDeviceInfo executes a DAC960 V2 Firmware Controller "Read
- Physical Device Information" IOCTL Command and waits for completion. It
- returns true on success and false on failure.
-
- The Channel, TargetID, LogicalUnit arguments should be 0 the first time
- this function is called for a given controller. This will return data
- for the "first" device on that controller. The returned data includes a
- Channel, TargetID, LogicalUnit that can be passed in to this routine to
- get data for the NEXT device on that controller.
-
- Data is stored in the controller's V2.NewPhysicalDeviceInfo dma-able
- memory buffer.
-
-*/
-
-static bool DAC960_V2_NewPhysicalDeviceInfo(DAC960_Controller_T *Controller,
- unsigned char Channel,
- unsigned char TargetID,
- unsigned char LogicalUnit)
-{
- DAC960_Command_T *Command = DAC960_AllocateCommand(Controller);
- DAC960_V2_CommandMailbox_T *CommandMailbox = &Command->V2.CommandMailbox;
- DAC960_V2_CommandStatus_T CommandStatus;
-
- DAC960_V2_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- CommandMailbox->PhysicalDeviceInfo.CommandOpcode = DAC960_V2_IOCTL;
- CommandMailbox->PhysicalDeviceInfo.CommandControlBits
- .DataTransferControllerToHost = true;
- CommandMailbox->PhysicalDeviceInfo.CommandControlBits
- .NoAutoRequestSense = true;
- CommandMailbox->PhysicalDeviceInfo.DataTransferSize =
- sizeof(DAC960_V2_PhysicalDeviceInfo_T);
- CommandMailbox->PhysicalDeviceInfo.PhysicalDevice.LogicalUnit = LogicalUnit;
- CommandMailbox->PhysicalDeviceInfo.PhysicalDevice.TargetID = TargetID;
- CommandMailbox->PhysicalDeviceInfo.PhysicalDevice.Channel = Channel;
- CommandMailbox->PhysicalDeviceInfo.IOCTL_Opcode =
- DAC960_V2_GetPhysicalDeviceInfoValid;
- CommandMailbox->PhysicalDeviceInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer =
- Controller->V2.NewPhysicalDeviceInformationDMA;
- CommandMailbox->PhysicalDeviceInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->PhysicalDeviceInfo.DataTransferSize;
- DAC960_ExecuteCommand(Command);
- CommandStatus = Command->V2.CommandStatus;
- DAC960_DeallocateCommand(Command);
- return (CommandStatus == DAC960_V2_NormalCompletion);
-}
-
-
-static void DAC960_V2_ConstructNewUnitSerialNumber(
- DAC960_Controller_T *Controller,
- DAC960_V2_CommandMailbox_T *CommandMailbox, int Channel, int TargetID,
- int LogicalUnit)
-{
- CommandMailbox->SCSI_10.CommandOpcode = DAC960_V2_SCSI_10_Passthru;
- CommandMailbox->SCSI_10.CommandControlBits
- .DataTransferControllerToHost = true;
- CommandMailbox->SCSI_10.CommandControlBits
- .NoAutoRequestSense = true;
- CommandMailbox->SCSI_10.DataTransferSize =
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T);
- CommandMailbox->SCSI_10.PhysicalDevice.LogicalUnit = LogicalUnit;
- CommandMailbox->SCSI_10.PhysicalDevice.TargetID = TargetID;
- CommandMailbox->SCSI_10.PhysicalDevice.Channel = Channel;
- CommandMailbox->SCSI_10.CDBLength = 6;
- CommandMailbox->SCSI_10.SCSI_CDB[0] = 0x12; /* INQUIRY */
- CommandMailbox->SCSI_10.SCSI_CDB[1] = 1; /* EVPD = 1 */
- CommandMailbox->SCSI_10.SCSI_CDB[2] = 0x80; /* Page Code */
- CommandMailbox->SCSI_10.SCSI_CDB[3] = 0; /* Reserved */
- CommandMailbox->SCSI_10.SCSI_CDB[4] =
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T);
- CommandMailbox->SCSI_10.SCSI_CDB[5] = 0; /* Control */
- CommandMailbox->SCSI_10.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer =
- Controller->V2.NewInquiryUnitSerialNumberDMA;
- CommandMailbox->SCSI_10.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->SCSI_10.DataTransferSize;
-}
-
-
-/*
- DAC960_V2_NewUnitSerialNumber executes an SCSI pass-through
- Inquiry command to a SCSI device identified by Channel number,
- Target id, Logical Unit Number. This function Waits for completion
- of the command.
-
- The return data includes Unit Serial Number information for the
- specified device.
-
- Data is stored in the controller's V2.NewPhysicalDeviceInfo dma-able
- memory buffer.
-*/
-
-static bool DAC960_V2_NewInquiryUnitSerialNumber(DAC960_Controller_T *Controller,
- int Channel, int TargetID, int LogicalUnit)
-{
- DAC960_Command_T *Command;
- DAC960_V2_CommandMailbox_T *CommandMailbox;
- DAC960_V2_CommandStatus_T CommandStatus;
-
- Command = DAC960_AllocateCommand(Controller);
- CommandMailbox = &Command->V2.CommandMailbox;
- DAC960_V2_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
-
- DAC960_V2_ConstructNewUnitSerialNumber(Controller, CommandMailbox,
- Channel, TargetID, LogicalUnit);
-
- DAC960_ExecuteCommand(Command);
- CommandStatus = Command->V2.CommandStatus;
- DAC960_DeallocateCommand(Command);
- return (CommandStatus == DAC960_V2_NormalCompletion);
-}
-
-
-/*
- DAC960_V2_DeviceOperation executes a DAC960 V2 Firmware Controller Device
- Operation IOCTL Command and waits for completion. It returns true on
- success and false on failure.
-*/
-
-static bool DAC960_V2_DeviceOperation(DAC960_Controller_T *Controller,
- DAC960_V2_IOCTL_Opcode_T IOCTL_Opcode,
- DAC960_V2_OperationDevice_T
- OperationDevice)
-{
- DAC960_Command_T *Command = DAC960_AllocateCommand(Controller);
- DAC960_V2_CommandMailbox_T *CommandMailbox = &Command->V2.CommandMailbox;
- DAC960_V2_CommandStatus_T CommandStatus;
- DAC960_V2_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- CommandMailbox->DeviceOperation.CommandOpcode = DAC960_V2_IOCTL;
- CommandMailbox->DeviceOperation.CommandControlBits
- .DataTransferControllerToHost = true;
- CommandMailbox->DeviceOperation.CommandControlBits
- .NoAutoRequestSense = true;
- CommandMailbox->DeviceOperation.IOCTL_Opcode = IOCTL_Opcode;
- CommandMailbox->DeviceOperation.OperationDevice = OperationDevice;
- DAC960_ExecuteCommand(Command);
- CommandStatus = Command->V2.CommandStatus;
- DAC960_DeallocateCommand(Command);
- return (CommandStatus == DAC960_V2_NormalCompletion);
-}
-
-
-/*
- DAC960_V1_EnableMemoryMailboxInterface enables the Memory Mailbox Interface
- for DAC960 V1 Firmware Controllers.
-
- PD and P controller types have no memory mailbox, but still need the
- other dma mapped memory.
-*/
-
-static bool DAC960_V1_EnableMemoryMailboxInterface(DAC960_Controller_T
- *Controller)
-{
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_HardwareType_T hw_type = Controller->HardwareType;
- struct pci_dev *PCI_Device = Controller->PCIDevice;
- struct dma_loaf *DmaPages = &Controller->DmaPages;
- size_t DmaPagesSize;
- size_t CommandMailboxesSize;
- size_t StatusMailboxesSize;
-
- DAC960_V1_CommandMailbox_T *CommandMailboxesMemory;
- dma_addr_t CommandMailboxesMemoryDMA;
-
- DAC960_V1_StatusMailbox_T *StatusMailboxesMemory;
- dma_addr_t StatusMailboxesMemoryDMA;
-
- DAC960_V1_CommandMailbox_T CommandMailbox;
- DAC960_V1_CommandStatus_T CommandStatus;
- int TimeoutCounter;
- int i;
-
- memset(&CommandMailbox, 0, sizeof(DAC960_V1_CommandMailbox_T));
-
- if (pci_set_dma_mask(Controller->PCIDevice, DMA_BIT_MASK(32)))
- return DAC960_Failure(Controller, "DMA mask out of range");
-
- if ((hw_type == DAC960_PD_Controller) || (hw_type == DAC960_P_Controller)) {
- CommandMailboxesSize = 0;
- StatusMailboxesSize = 0;
- } else {
- CommandMailboxesSize = DAC960_V1_CommandMailboxCount * sizeof(DAC960_V1_CommandMailbox_T);
- StatusMailboxesSize = DAC960_V1_StatusMailboxCount * sizeof(DAC960_V1_StatusMailbox_T);
- }
- DmaPagesSize = CommandMailboxesSize + StatusMailboxesSize +
- sizeof(DAC960_V1_DCDB_T) + sizeof(DAC960_V1_Enquiry_T) +
- sizeof(DAC960_V1_ErrorTable_T) + sizeof(DAC960_V1_EventLogEntry_T) +
- sizeof(DAC960_V1_RebuildProgress_T) +
- sizeof(DAC960_V1_LogicalDriveInformationArray_T) +
- sizeof(DAC960_V1_BackgroundInitializationStatus_T) +
- sizeof(DAC960_V1_DeviceState_T) + sizeof(DAC960_SCSI_Inquiry_T) +
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T);
-
- if (!init_dma_loaf(PCI_Device, DmaPages, DmaPagesSize))
- return false;
-
-
- if ((hw_type == DAC960_PD_Controller) || (hw_type == DAC960_P_Controller))
- goto skip_mailboxes;
-
- CommandMailboxesMemory = slice_dma_loaf(DmaPages,
- CommandMailboxesSize, &CommandMailboxesMemoryDMA);
-
- /* These are the base addresses for the command memory mailbox array */
- Controller->V1.FirstCommandMailbox = CommandMailboxesMemory;
- Controller->V1.FirstCommandMailboxDMA = CommandMailboxesMemoryDMA;
-
- CommandMailboxesMemory += DAC960_V1_CommandMailboxCount - 1;
- Controller->V1.LastCommandMailbox = CommandMailboxesMemory;
- Controller->V1.NextCommandMailbox = Controller->V1.FirstCommandMailbox;
- Controller->V1.PreviousCommandMailbox1 = Controller->V1.LastCommandMailbox;
- Controller->V1.PreviousCommandMailbox2 =
- Controller->V1.LastCommandMailbox - 1;
-
- /* These are the base addresses for the status memory mailbox array */
- StatusMailboxesMemory = slice_dma_loaf(DmaPages,
- StatusMailboxesSize, &StatusMailboxesMemoryDMA);
-
- Controller->V1.FirstStatusMailbox = StatusMailboxesMemory;
- Controller->V1.FirstStatusMailboxDMA = StatusMailboxesMemoryDMA;
- StatusMailboxesMemory += DAC960_V1_StatusMailboxCount - 1;
- Controller->V1.LastStatusMailbox = StatusMailboxesMemory;
- Controller->V1.NextStatusMailbox = Controller->V1.FirstStatusMailbox;
-
-skip_mailboxes:
- Controller->V1.MonitoringDCDB = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V1_DCDB_T),
- &Controller->V1.MonitoringDCDB_DMA);
-
- Controller->V1.NewEnquiry = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V1_Enquiry_T),
- &Controller->V1.NewEnquiryDMA);
-
- Controller->V1.NewErrorTable = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V1_ErrorTable_T),
- &Controller->V1.NewErrorTableDMA);
-
- Controller->V1.EventLogEntry = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V1_EventLogEntry_T),
- &Controller->V1.EventLogEntryDMA);
-
- Controller->V1.RebuildProgress = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V1_RebuildProgress_T),
- &Controller->V1.RebuildProgressDMA);
-
- Controller->V1.NewLogicalDriveInformation = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V1_LogicalDriveInformationArray_T),
- &Controller->V1.NewLogicalDriveInformationDMA);
-
- Controller->V1.BackgroundInitializationStatus = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V1_BackgroundInitializationStatus_T),
- &Controller->V1.BackgroundInitializationStatusDMA);
-
- Controller->V1.NewDeviceState = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V1_DeviceState_T),
- &Controller->V1.NewDeviceStateDMA);
-
- Controller->V1.NewInquiryStandardData = slice_dma_loaf(DmaPages,
- sizeof(DAC960_SCSI_Inquiry_T),
- &Controller->V1.NewInquiryStandardDataDMA);
-
- Controller->V1.NewInquiryUnitSerialNumber = slice_dma_loaf(DmaPages,
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T),
- &Controller->V1.NewInquiryUnitSerialNumberDMA);
-
- if ((hw_type == DAC960_PD_Controller) || (hw_type == DAC960_P_Controller))
- return true;
-
- /* Enable the Memory Mailbox Interface. */
- Controller->V1.DualModeMemoryMailboxInterface = true;
- CommandMailbox.TypeX.CommandOpcode = 0x2B;
- CommandMailbox.TypeX.CommandIdentifier = 0;
- CommandMailbox.TypeX.CommandOpcode2 = 0x14;
- CommandMailbox.TypeX.CommandMailboxesBusAddress =
- Controller->V1.FirstCommandMailboxDMA;
- CommandMailbox.TypeX.StatusMailboxesBusAddress =
- Controller->V1.FirstStatusMailboxDMA;
-#define TIMEOUT_COUNT 1000000
-
- for (i = 0; i < 2; i++)
- switch (Controller->HardwareType)
- {
- case DAC960_LA_Controller:
- TimeoutCounter = TIMEOUT_COUNT;
- while (--TimeoutCounter >= 0)
- {
- if (!DAC960_LA_HardwareMailboxFullP(ControllerBaseAddress))
- break;
- udelay(10);
- }
- if (TimeoutCounter < 0) return false;
- DAC960_LA_WriteHardwareMailbox(ControllerBaseAddress, &CommandMailbox);
- DAC960_LA_HardwareMailboxNewCommand(ControllerBaseAddress);
- TimeoutCounter = TIMEOUT_COUNT;
- while (--TimeoutCounter >= 0)
- {
- if (DAC960_LA_HardwareMailboxStatusAvailableP(
- ControllerBaseAddress))
- break;
- udelay(10);
- }
- if (TimeoutCounter < 0) return false;
- CommandStatus = DAC960_LA_ReadStatusRegister(ControllerBaseAddress);
- DAC960_LA_AcknowledgeHardwareMailboxInterrupt(ControllerBaseAddress);
- DAC960_LA_AcknowledgeHardwareMailboxStatus(ControllerBaseAddress);
- if (CommandStatus == DAC960_V1_NormalCompletion) return true;
- Controller->V1.DualModeMemoryMailboxInterface = false;
- CommandMailbox.TypeX.CommandOpcode2 = 0x10;
- break;
- case DAC960_PG_Controller:
- TimeoutCounter = TIMEOUT_COUNT;
- while (--TimeoutCounter >= 0)
- {
- if (!DAC960_PG_HardwareMailboxFullP(ControllerBaseAddress))
- break;
- udelay(10);
- }
- if (TimeoutCounter < 0) return false;
- DAC960_PG_WriteHardwareMailbox(ControllerBaseAddress, &CommandMailbox);
- DAC960_PG_HardwareMailboxNewCommand(ControllerBaseAddress);
-
- TimeoutCounter = TIMEOUT_COUNT;
- while (--TimeoutCounter >= 0)
- {
- if (DAC960_PG_HardwareMailboxStatusAvailableP(
- ControllerBaseAddress))
- break;
- udelay(10);
- }
- if (TimeoutCounter < 0) return false;
- CommandStatus = DAC960_PG_ReadStatusRegister(ControllerBaseAddress);
- DAC960_PG_AcknowledgeHardwareMailboxInterrupt(ControllerBaseAddress);
- DAC960_PG_AcknowledgeHardwareMailboxStatus(ControllerBaseAddress);
- if (CommandStatus == DAC960_V1_NormalCompletion) return true;
- Controller->V1.DualModeMemoryMailboxInterface = false;
- CommandMailbox.TypeX.CommandOpcode2 = 0x10;
- break;
- default:
- DAC960_Failure(Controller, "Unknown Controller Type\n");
- break;
- }
- return false;
-}
-
-
-/*
- DAC960_V2_EnableMemoryMailboxInterface enables the Memory Mailbox Interface
- for DAC960 V2 Firmware Controllers.
-
- Aggregate the space needed for the controller's memory mailbox and
- the other data structures that will be targets of dma transfers with
- the controller. Allocate a dma-mapped region of memory to hold these
- structures. Then, save CPU pointers and dma_addr_t values to reference
- the structures that are contained in that region.
-*/
-
-static bool DAC960_V2_EnableMemoryMailboxInterface(DAC960_Controller_T
- *Controller)
-{
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- struct pci_dev *PCI_Device = Controller->PCIDevice;
- struct dma_loaf *DmaPages = &Controller->DmaPages;
- size_t DmaPagesSize;
- size_t CommandMailboxesSize;
- size_t StatusMailboxesSize;
-
- DAC960_V2_CommandMailbox_T *CommandMailboxesMemory;
- dma_addr_t CommandMailboxesMemoryDMA;
-
- DAC960_V2_StatusMailbox_T *StatusMailboxesMemory;
- dma_addr_t StatusMailboxesMemoryDMA;
-
- DAC960_V2_CommandMailbox_T *CommandMailbox;
- dma_addr_t CommandMailboxDMA;
- DAC960_V2_CommandStatus_T CommandStatus;
-
- if (pci_set_dma_mask(Controller->PCIDevice, DMA_BIT_MASK(64)) &&
- pci_set_dma_mask(Controller->PCIDevice, DMA_BIT_MASK(32)))
- return DAC960_Failure(Controller, "DMA mask out of range");
-
- /* This is a temporary dma mapping, used only in the scope of this function */
- CommandMailbox = pci_alloc_consistent(PCI_Device,
- sizeof(DAC960_V2_CommandMailbox_T), &CommandMailboxDMA);
- if (CommandMailbox == NULL)
- return false;
-
- CommandMailboxesSize = DAC960_V2_CommandMailboxCount * sizeof(DAC960_V2_CommandMailbox_T);
- StatusMailboxesSize = DAC960_V2_StatusMailboxCount * sizeof(DAC960_V2_StatusMailbox_T);
- DmaPagesSize =
- CommandMailboxesSize + StatusMailboxesSize +
- sizeof(DAC960_V2_HealthStatusBuffer_T) +
- sizeof(DAC960_V2_ControllerInfo_T) +
- sizeof(DAC960_V2_LogicalDeviceInfo_T) +
- sizeof(DAC960_V2_PhysicalDeviceInfo_T) +
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T) +
- sizeof(DAC960_V2_Event_T) +
- sizeof(DAC960_V2_PhysicalToLogicalDevice_T);
-
- if (!init_dma_loaf(PCI_Device, DmaPages, DmaPagesSize)) {
- pci_free_consistent(PCI_Device, sizeof(DAC960_V2_CommandMailbox_T),
- CommandMailbox, CommandMailboxDMA);
- return false;
- }
-
- CommandMailboxesMemory = slice_dma_loaf(DmaPages,
- CommandMailboxesSize, &CommandMailboxesMemoryDMA);
-
- /* These are the base addresses for the command memory mailbox array */
- Controller->V2.FirstCommandMailbox = CommandMailboxesMemory;
- Controller->V2.FirstCommandMailboxDMA = CommandMailboxesMemoryDMA;
-
- CommandMailboxesMemory += DAC960_V2_CommandMailboxCount - 1;
- Controller->V2.LastCommandMailbox = CommandMailboxesMemory;
- Controller->V2.NextCommandMailbox = Controller->V2.FirstCommandMailbox;
- Controller->V2.PreviousCommandMailbox1 = Controller->V2.LastCommandMailbox;
- Controller->V2.PreviousCommandMailbox2 =
- Controller->V2.LastCommandMailbox - 1;
-
- /* These are the base addresses for the status memory mailbox array */
- StatusMailboxesMemory = slice_dma_loaf(DmaPages,
- StatusMailboxesSize, &StatusMailboxesMemoryDMA);
-
- Controller->V2.FirstStatusMailbox = StatusMailboxesMemory;
- Controller->V2.FirstStatusMailboxDMA = StatusMailboxesMemoryDMA;
- StatusMailboxesMemory += DAC960_V2_StatusMailboxCount - 1;
- Controller->V2.LastStatusMailbox = StatusMailboxesMemory;
- Controller->V2.NextStatusMailbox = Controller->V2.FirstStatusMailbox;
-
- Controller->V2.HealthStatusBuffer = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V2_HealthStatusBuffer_T),
- &Controller->V2.HealthStatusBufferDMA);
-
- Controller->V2.NewControllerInformation = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V2_ControllerInfo_T),
- &Controller->V2.NewControllerInformationDMA);
-
- Controller->V2.NewLogicalDeviceInformation = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V2_LogicalDeviceInfo_T),
- &Controller->V2.NewLogicalDeviceInformationDMA);
-
- Controller->V2.NewPhysicalDeviceInformation = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V2_PhysicalDeviceInfo_T),
- &Controller->V2.NewPhysicalDeviceInformationDMA);
-
- Controller->V2.NewInquiryUnitSerialNumber = slice_dma_loaf(DmaPages,
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T),
- &Controller->V2.NewInquiryUnitSerialNumberDMA);
-
- Controller->V2.Event = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V2_Event_T),
- &Controller->V2.EventDMA);
-
- Controller->V2.PhysicalToLogicalDevice = slice_dma_loaf(DmaPages,
- sizeof(DAC960_V2_PhysicalToLogicalDevice_T),
- &Controller->V2.PhysicalToLogicalDeviceDMA);
-
- /*
- Enable the Memory Mailbox Interface.
-
- I don't know why we can't just use one of the memory mailboxes
- we just allocated to do this, instead of using this temporary one.
- Try this change later.
- */
- memset(CommandMailbox, 0, sizeof(DAC960_V2_CommandMailbox_T));
- CommandMailbox->SetMemoryMailbox.CommandIdentifier = 1;
- CommandMailbox->SetMemoryMailbox.CommandOpcode = DAC960_V2_IOCTL;
- CommandMailbox->SetMemoryMailbox.CommandControlBits.NoAutoRequestSense = true;
- CommandMailbox->SetMemoryMailbox.FirstCommandMailboxSizeKB =
- (DAC960_V2_CommandMailboxCount * sizeof(DAC960_V2_CommandMailbox_T)) >> 10;
- CommandMailbox->SetMemoryMailbox.FirstStatusMailboxSizeKB =
- (DAC960_V2_StatusMailboxCount * sizeof(DAC960_V2_StatusMailbox_T)) >> 10;
- CommandMailbox->SetMemoryMailbox.SecondCommandMailboxSizeKB = 0;
- CommandMailbox->SetMemoryMailbox.SecondStatusMailboxSizeKB = 0;
- CommandMailbox->SetMemoryMailbox.RequestSenseSize = 0;
- CommandMailbox->SetMemoryMailbox.IOCTL_Opcode = DAC960_V2_SetMemoryMailbox;
- CommandMailbox->SetMemoryMailbox.HealthStatusBufferSizeKB = 1;
- CommandMailbox->SetMemoryMailbox.HealthStatusBufferBusAddress =
- Controller->V2.HealthStatusBufferDMA;
- CommandMailbox->SetMemoryMailbox.FirstCommandMailboxBusAddress =
- Controller->V2.FirstCommandMailboxDMA;
- CommandMailbox->SetMemoryMailbox.FirstStatusMailboxBusAddress =
- Controller->V2.FirstStatusMailboxDMA;
- switch (Controller->HardwareType)
- {
- case DAC960_GEM_Controller:
- while (DAC960_GEM_HardwareMailboxFullP(ControllerBaseAddress))
- udelay(1);
- DAC960_GEM_WriteHardwareMailbox(ControllerBaseAddress, CommandMailboxDMA);
- DAC960_GEM_HardwareMailboxNewCommand(ControllerBaseAddress);
- while (!DAC960_GEM_HardwareMailboxStatusAvailableP(ControllerBaseAddress))
- udelay(1);
- CommandStatus = DAC960_GEM_ReadCommandStatus(ControllerBaseAddress);
- DAC960_GEM_AcknowledgeHardwareMailboxInterrupt(ControllerBaseAddress);
- DAC960_GEM_AcknowledgeHardwareMailboxStatus(ControllerBaseAddress);
- break;
- case DAC960_BA_Controller:
- while (DAC960_BA_HardwareMailboxFullP(ControllerBaseAddress))
- udelay(1);
- DAC960_BA_WriteHardwareMailbox(ControllerBaseAddress, CommandMailboxDMA);
- DAC960_BA_HardwareMailboxNewCommand(ControllerBaseAddress);
- while (!DAC960_BA_HardwareMailboxStatusAvailableP(ControllerBaseAddress))
- udelay(1);
- CommandStatus = DAC960_BA_ReadCommandStatus(ControllerBaseAddress);
- DAC960_BA_AcknowledgeHardwareMailboxInterrupt(ControllerBaseAddress);
- DAC960_BA_AcknowledgeHardwareMailboxStatus(ControllerBaseAddress);
- break;
- case DAC960_LP_Controller:
- while (DAC960_LP_HardwareMailboxFullP(ControllerBaseAddress))
- udelay(1);
- DAC960_LP_WriteHardwareMailbox(ControllerBaseAddress, CommandMailboxDMA);
- DAC960_LP_HardwareMailboxNewCommand(ControllerBaseAddress);
- while (!DAC960_LP_HardwareMailboxStatusAvailableP(ControllerBaseAddress))
- udelay(1);
- CommandStatus = DAC960_LP_ReadCommandStatus(ControllerBaseAddress);
- DAC960_LP_AcknowledgeHardwareMailboxInterrupt(ControllerBaseAddress);
- DAC960_LP_AcknowledgeHardwareMailboxStatus(ControllerBaseAddress);
- break;
- default:
- DAC960_Failure(Controller, "Unknown Controller Type\n");
- CommandStatus = DAC960_V2_AbormalCompletion;
- break;
- }
- pci_free_consistent(PCI_Device, sizeof(DAC960_V2_CommandMailbox_T),
- CommandMailbox, CommandMailboxDMA);
- return (CommandStatus == DAC960_V2_NormalCompletion);
-}
-
-
-/*
- DAC960_V1_ReadControllerConfiguration reads the Configuration Information
- from DAC960 V1 Firmware Controllers and initializes the Controller structure.
-*/
-
-static bool DAC960_V1_ReadControllerConfiguration(DAC960_Controller_T
- *Controller)
-{
- DAC960_V1_Enquiry2_T *Enquiry2;
- dma_addr_t Enquiry2DMA;
- DAC960_V1_Config2_T *Config2;
- dma_addr_t Config2DMA;
- int LogicalDriveNumber, Channel, TargetID;
- struct dma_loaf local_dma;
-
- if (!init_dma_loaf(Controller->PCIDevice, &local_dma,
- sizeof(DAC960_V1_Enquiry2_T) + sizeof(DAC960_V1_Config2_T)))
- return DAC960_Failure(Controller, "LOGICAL DEVICE ALLOCATION");
-
- Enquiry2 = slice_dma_loaf(&local_dma, sizeof(DAC960_V1_Enquiry2_T), &Enquiry2DMA);
- Config2 = slice_dma_loaf(&local_dma, sizeof(DAC960_V1_Config2_T), &Config2DMA);
-
- if (!DAC960_V1_ExecuteType3(Controller, DAC960_V1_Enquiry,
- Controller->V1.NewEnquiryDMA)) {
- free_dma_loaf(Controller->PCIDevice, &local_dma);
- return DAC960_Failure(Controller, "ENQUIRY");
- }
- memcpy(&Controller->V1.Enquiry, Controller->V1.NewEnquiry,
- sizeof(DAC960_V1_Enquiry_T));
-
- if (!DAC960_V1_ExecuteType3(Controller, DAC960_V1_Enquiry2, Enquiry2DMA)) {
- free_dma_loaf(Controller->PCIDevice, &local_dma);
- return DAC960_Failure(Controller, "ENQUIRY2");
- }
-
- if (!DAC960_V1_ExecuteType3(Controller, DAC960_V1_ReadConfig2, Config2DMA)) {
- free_dma_loaf(Controller->PCIDevice, &local_dma);
- return DAC960_Failure(Controller, "READ CONFIG2");
- }
-
- if (!DAC960_V1_ExecuteType3(Controller, DAC960_V1_GetLogicalDriveInformation,
- Controller->V1.NewLogicalDriveInformationDMA)) {
- free_dma_loaf(Controller->PCIDevice, &local_dma);
- return DAC960_Failure(Controller, "GET LOGICAL DRIVE INFORMATION");
- }
- memcpy(&Controller->V1.LogicalDriveInformation,
- Controller->V1.NewLogicalDriveInformation,
- sizeof(DAC960_V1_LogicalDriveInformationArray_T));
-
- for (Channel = 0; Channel < Enquiry2->ActualChannels; Channel++)
- for (TargetID = 0; TargetID < Enquiry2->MaxTargets; TargetID++) {
- if (!DAC960_V1_ExecuteType3D(Controller, DAC960_V1_GetDeviceState,
- Channel, TargetID,
- Controller->V1.NewDeviceStateDMA)) {
- free_dma_loaf(Controller->PCIDevice, &local_dma);
- return DAC960_Failure(Controller, "GET DEVICE STATE");
- }
- memcpy(&Controller->V1.DeviceState[Channel][TargetID],
- Controller->V1.NewDeviceState, sizeof(DAC960_V1_DeviceState_T));
- }
- /*
- Initialize the Controller Model Name and Full Model Name fields.
- */
- switch (Enquiry2->HardwareID.SubModel)
- {
- case DAC960_V1_P_PD_PU:
- if (Enquiry2->SCSICapability.BusSpeed == DAC960_V1_Ultra)
- strcpy(Controller->ModelName, "DAC960PU");
- else strcpy(Controller->ModelName, "DAC960PD");
- break;
- case DAC960_V1_PL:
- strcpy(Controller->ModelName, "DAC960PL");
- break;
- case DAC960_V1_PG:
- strcpy(Controller->ModelName, "DAC960PG");
- break;
- case DAC960_V1_PJ:
- strcpy(Controller->ModelName, "DAC960PJ");
- break;
- case DAC960_V1_PR:
- strcpy(Controller->ModelName, "DAC960PR");
- break;
- case DAC960_V1_PT:
- strcpy(Controller->ModelName, "DAC960PT");
- break;
- case DAC960_V1_PTL0:
- strcpy(Controller->ModelName, "DAC960PTL0");
- break;
- case DAC960_V1_PRL:
- strcpy(Controller->ModelName, "DAC960PRL");
- break;
- case DAC960_V1_PTL1:
- strcpy(Controller->ModelName, "DAC960PTL1");
- break;
- case DAC960_V1_1164P:
- strcpy(Controller->ModelName, "DAC1164P");
- break;
- default:
- free_dma_loaf(Controller->PCIDevice, &local_dma);
- return DAC960_Failure(Controller, "MODEL VERIFICATION");
- }
- strcpy(Controller->FullModelName, "Mylex ");
- strcat(Controller->FullModelName, Controller->ModelName);
- /*
- Initialize the Controller Firmware Version field and verify that it
- is a supported firmware version. The supported firmware versions are:
-
- DAC1164P 5.06 and above
- DAC960PTL/PRL/PJ/PG 4.06 and above
- DAC960PU/PD/PL 3.51 and above
- DAC960PU/PD/PL/P 2.73 and above
- */
-#if defined(CONFIG_ALPHA)
- /*
- DEC Alpha machines were often equipped with DAC960 cards that were
- OEMed from Mylex, and had their own custom firmware. Version 2.70,
- the last custom FW revision to be released by DEC for these older
- controllers, appears to work quite well with this driver.
-
- Cards tested successfully were several versions each of the PD and
- PU, called by DEC the KZPSC and KZPAC, respectively, and having
- the Manufacturer Numbers (from Mylex), usually on a sticker on the
- back of the board, of:
-
- KZPSC: D040347 (1-channel) or D040348 (2-channel) or D040349 (3-channel)
- KZPAC: D040395 (1-channel) or D040396 (2-channel) or D040397 (3-channel)
- */
-# define FIRMWARE_27X "2.70"
-#else
-# define FIRMWARE_27X "2.73"
-#endif
-
- if (Enquiry2->FirmwareID.MajorVersion == 0)
- {
- Enquiry2->FirmwareID.MajorVersion =
- Controller->V1.Enquiry.MajorFirmwareVersion;
- Enquiry2->FirmwareID.MinorVersion =
- Controller->V1.Enquiry.MinorFirmwareVersion;
- Enquiry2->FirmwareID.FirmwareType = '0';
- Enquiry2->FirmwareID.TurnID = 0;
- }
- snprintf(Controller->FirmwareVersion, sizeof(Controller->FirmwareVersion),
- "%d.%02d-%c-%02d",
- Enquiry2->FirmwareID.MajorVersion,
- Enquiry2->FirmwareID.MinorVersion,
- Enquiry2->FirmwareID.FirmwareType,
- Enquiry2->FirmwareID.TurnID);
- if (!((Controller->FirmwareVersion[0] == '5' &&
- strcmp(Controller->FirmwareVersion, "5.06") >= 0) ||
- (Controller->FirmwareVersion[0] == '4' &&
- strcmp(Controller->FirmwareVersion, "4.06") >= 0) ||
- (Controller->FirmwareVersion[0] == '3' &&
- strcmp(Controller->FirmwareVersion, "3.51") >= 0) ||
- (Controller->FirmwareVersion[0] == '2' &&
- strcmp(Controller->FirmwareVersion, FIRMWARE_27X) >= 0)))
- {
- DAC960_Failure(Controller, "FIRMWARE VERSION VERIFICATION");
- DAC960_Error("Firmware Version = '%s'\n", Controller,
- Controller->FirmwareVersion);
- free_dma_loaf(Controller->PCIDevice, &local_dma);
- return false;
- }
- /*
- Initialize the Controller Channels, Targets, Memory Size, and SAF-TE
- Enclosure Management Enabled fields.
- */
- Controller->Channels = Enquiry2->ActualChannels;
- Controller->Targets = Enquiry2->MaxTargets;
- Controller->MemorySize = Enquiry2->MemorySize >> 20;
- Controller->V1.SAFTE_EnclosureManagementEnabled =
- (Enquiry2->FaultManagementType == DAC960_V1_SAFTE);
- /*
- Initialize the Controller Queue Depth, Driver Queue Depth, Logical Drive
- Count, Maximum Blocks per Command, Controller Scatter/Gather Limit, and
- Driver Scatter/Gather Limit. The Driver Queue Depth must be at most one
- less than the Controller Queue Depth to allow for an automatic drive
- rebuild operation.
- */
- Controller->ControllerQueueDepth = Controller->V1.Enquiry.MaxCommands;
- Controller->DriverQueueDepth = Controller->ControllerQueueDepth - 1;
- if (Controller->DriverQueueDepth > DAC960_MaxDriverQueueDepth)
- Controller->DriverQueueDepth = DAC960_MaxDriverQueueDepth;
- Controller->LogicalDriveCount =
- Controller->V1.Enquiry.NumberOfLogicalDrives;
- Controller->MaxBlocksPerCommand = Enquiry2->MaxBlocksPerCommand;
- Controller->ControllerScatterGatherLimit = Enquiry2->MaxScatterGatherEntries;
- Controller->DriverScatterGatherLimit =
- Controller->ControllerScatterGatherLimit;
- if (Controller->DriverScatterGatherLimit > DAC960_V1_ScatterGatherLimit)
- Controller->DriverScatterGatherLimit = DAC960_V1_ScatterGatherLimit;
- /*
- Initialize the Stripe Size, Segment Size, and Geometry Translation.
- */
- Controller->V1.StripeSize = Config2->BlocksPerStripe * Config2->BlockFactor
- >> (10 - DAC960_BlockSizeBits);
- Controller->V1.SegmentSize = Config2->BlocksPerCacheLine * Config2->BlockFactor
- >> (10 - DAC960_BlockSizeBits);
- switch (Config2->DriveGeometry)
- {
- case DAC960_V1_Geometry_128_32:
- Controller->V1.GeometryTranslationHeads = 128;
- Controller->V1.GeometryTranslationSectors = 32;
- break;
- case DAC960_V1_Geometry_255_63:
- Controller->V1.GeometryTranslationHeads = 255;
- Controller->V1.GeometryTranslationSectors = 63;
- break;
- default:
- free_dma_loaf(Controller->PCIDevice, &local_dma);
- return DAC960_Failure(Controller, "CONFIG2 DRIVE GEOMETRY");
- }
- /*
- Initialize the Background Initialization Status.
- */
- if ((Controller->FirmwareVersion[0] == '4' &&
- strcmp(Controller->FirmwareVersion, "4.08") >= 0) ||
- (Controller->FirmwareVersion[0] == '5' &&
- strcmp(Controller->FirmwareVersion, "5.08") >= 0))
- {
- Controller->V1.BackgroundInitializationStatusSupported = true;
- DAC960_V1_ExecuteType3B(Controller,
- DAC960_V1_BackgroundInitializationControl, 0x20,
- Controller->
- V1.BackgroundInitializationStatusDMA);
- memcpy(&Controller->V1.LastBackgroundInitializationStatus,
- Controller->V1.BackgroundInitializationStatus,
- sizeof(DAC960_V1_BackgroundInitializationStatus_T));
- }
- /*
- Initialize the Logical Drive Initially Accessible flag.
- */
- for (LogicalDriveNumber = 0;
- LogicalDriveNumber < Controller->LogicalDriveCount;
- LogicalDriveNumber++)
- if (Controller->V1.LogicalDriveInformation
- [LogicalDriveNumber].LogicalDriveState !=
- DAC960_V1_LogicalDrive_Offline)
- Controller->LogicalDriveInitiallyAccessible[LogicalDriveNumber] = true;
- Controller->V1.LastRebuildStatus = DAC960_V1_NoRebuildOrCheckInProgress;
- free_dma_loaf(Controller->PCIDevice, &local_dma);
- return true;
-}
-
-
-/*
- DAC960_V2_ReadControllerConfiguration reads the Configuration Information
- from DAC960 V2 Firmware Controllers and initializes the Controller structure.
-*/
-
-static bool DAC960_V2_ReadControllerConfiguration(DAC960_Controller_T
- *Controller)
-{
- DAC960_V2_ControllerInfo_T *ControllerInfo =
- &Controller->V2.ControllerInformation;
- unsigned short LogicalDeviceNumber = 0;
- int ModelNameLength;
-
- /* Get data into dma-able area, then copy into permanent location */
- if (!DAC960_V2_NewControllerInfo(Controller))
- return DAC960_Failure(Controller, "GET CONTROLLER INFO");
- memcpy(ControllerInfo, Controller->V2.NewControllerInformation,
- sizeof(DAC960_V2_ControllerInfo_T));
-
-
- if (!DAC960_V2_GeneralInfo(Controller))
- return DAC960_Failure(Controller, "GET HEALTH STATUS");
-
- /*
- Initialize the Controller Model Name and Full Model Name fields.
- */
- ModelNameLength = sizeof(ControllerInfo->ControllerName);
- if (ModelNameLength > sizeof(Controller->ModelName)-1)
- ModelNameLength = sizeof(Controller->ModelName)-1;
- memcpy(Controller->ModelName, ControllerInfo->ControllerName,
- ModelNameLength);
- ModelNameLength--;
- while (Controller->ModelName[ModelNameLength] == ' ' ||
- Controller->ModelName[ModelNameLength] == '\0')
- ModelNameLength--;
- Controller->ModelName[++ModelNameLength] = '\0';
- strcpy(Controller->FullModelName, "Mylex ");
- strcat(Controller->FullModelName, Controller->ModelName);
- /*
- Initialize the Controller Firmware Version field.
- */
- sprintf(Controller->FirmwareVersion, "%d.%02d-%02d",
- ControllerInfo->FirmwareMajorVersion,
- ControllerInfo->FirmwareMinorVersion,
- ControllerInfo->FirmwareTurnNumber);
- if (ControllerInfo->FirmwareMajorVersion == 6 &&
- ControllerInfo->FirmwareMinorVersion == 0 &&
- ControllerInfo->FirmwareTurnNumber < 1)
- {
- DAC960_Info("FIRMWARE VERSION %s DOES NOT PROVIDE THE CONTROLLER\n",
- Controller, Controller->FirmwareVersion);
- DAC960_Info("STATUS MONITORING FUNCTIONALITY NEEDED BY THIS DRIVER.\n",
- Controller);
- DAC960_Info("PLEASE UPGRADE TO VERSION 6.00-01 OR ABOVE.\n",
- Controller);
- }
- /*
- Initialize the Controller Channels, Targets, and Memory Size.
- */
- Controller->Channels = ControllerInfo->NumberOfPhysicalChannelsPresent;
- Controller->Targets =
- ControllerInfo->MaximumTargetsPerChannel
- [ControllerInfo->NumberOfPhysicalChannelsPresent-1];
- Controller->MemorySize = ControllerInfo->MemorySizeMB;
- /*
- Initialize the Controller Queue Depth, Driver Queue Depth, Logical Drive
- Count, Maximum Blocks per Command, Controller Scatter/Gather Limit, and
- Driver Scatter/Gather Limit. The Driver Queue Depth must be at most one
- less than the Controller Queue Depth to allow for an automatic drive
- rebuild operation.
- */
- Controller->ControllerQueueDepth = ControllerInfo->MaximumParallelCommands;
- Controller->DriverQueueDepth = Controller->ControllerQueueDepth - 1;
- if (Controller->DriverQueueDepth > DAC960_MaxDriverQueueDepth)
- Controller->DriverQueueDepth = DAC960_MaxDriverQueueDepth;
- Controller->LogicalDriveCount = ControllerInfo->LogicalDevicesPresent;
- Controller->MaxBlocksPerCommand =
- ControllerInfo->MaximumDataTransferSizeInBlocks;
- Controller->ControllerScatterGatherLimit =
- ControllerInfo->MaximumScatterGatherEntries;
- Controller->DriverScatterGatherLimit =
- Controller->ControllerScatterGatherLimit;
- if (Controller->DriverScatterGatherLimit > DAC960_V2_ScatterGatherLimit)
- Controller->DriverScatterGatherLimit = DAC960_V2_ScatterGatherLimit;
- /*
- Initialize the Logical Device Information.
- */
- while (true)
- {
- DAC960_V2_LogicalDeviceInfo_T *NewLogicalDeviceInfo =
- Controller->V2.NewLogicalDeviceInformation;
- DAC960_V2_LogicalDeviceInfo_T *LogicalDeviceInfo;
- DAC960_V2_PhysicalDevice_T PhysicalDevice;
-
- if (!DAC960_V2_NewLogicalDeviceInfo(Controller, LogicalDeviceNumber))
- break;
- LogicalDeviceNumber = NewLogicalDeviceInfo->LogicalDeviceNumber;
- if (LogicalDeviceNumber >= DAC960_MaxLogicalDrives) {
- DAC960_Error("DAC960: Logical Drive Number %d not supported\n",
- Controller, LogicalDeviceNumber);
- break;
- }
- if (NewLogicalDeviceInfo->DeviceBlockSizeInBytes != DAC960_BlockSize) {
- DAC960_Error("DAC960: Logical Drive Block Size %d not supported\n",
- Controller, NewLogicalDeviceInfo->DeviceBlockSizeInBytes);
- LogicalDeviceNumber++;
- continue;
- }
- PhysicalDevice.Controller = 0;
- PhysicalDevice.Channel = NewLogicalDeviceInfo->Channel;
- PhysicalDevice.TargetID = NewLogicalDeviceInfo->TargetID;
- PhysicalDevice.LogicalUnit = NewLogicalDeviceInfo->LogicalUnit;
- Controller->V2.LogicalDriveToVirtualDevice[LogicalDeviceNumber] =
- PhysicalDevice;
- if (NewLogicalDeviceInfo->LogicalDeviceState !=
- DAC960_V2_LogicalDevice_Offline)
- Controller->LogicalDriveInitiallyAccessible[LogicalDeviceNumber] = true;
- LogicalDeviceInfo = kmalloc(sizeof(DAC960_V2_LogicalDeviceInfo_T),
- GFP_ATOMIC);
- if (LogicalDeviceInfo == NULL)
- return DAC960_Failure(Controller, "LOGICAL DEVICE ALLOCATION");
- Controller->V2.LogicalDeviceInformation[LogicalDeviceNumber] =
- LogicalDeviceInfo;
- memcpy(LogicalDeviceInfo, NewLogicalDeviceInfo,
- sizeof(DAC960_V2_LogicalDeviceInfo_T));
- LogicalDeviceNumber++;
- }
- return true;
-}
-
-
-/*
- DAC960_ReportControllerConfiguration reports the Configuration Information
- for Controller.
-*/
-
-static bool DAC960_ReportControllerConfiguration(DAC960_Controller_T
- *Controller)
-{
- DAC960_Info("Configuring Mylex %s PCI RAID Controller\n",
- Controller, Controller->ModelName);
- DAC960_Info(" Firmware Version: %s, Channels: %d, Memory Size: %dMB\n",
- Controller, Controller->FirmwareVersion,
- Controller->Channels, Controller->MemorySize);
- DAC960_Info(" PCI Bus: %d, Device: %d, Function: %d, I/O Address: ",
- Controller, Controller->Bus,
- Controller->Device, Controller->Function);
- if (Controller->IO_Address == 0)
- DAC960_Info("Unassigned\n", Controller);
- else DAC960_Info("0x%X\n", Controller, Controller->IO_Address);
- DAC960_Info(" PCI Address: 0x%X mapped at 0x%lX, IRQ Channel: %d\n",
- Controller, Controller->PCI_Address,
- (unsigned long) Controller->BaseAddress,
- Controller->IRQ_Channel);
- DAC960_Info(" Controller Queue Depth: %d, "
- "Maximum Blocks per Command: %d\n",
- Controller, Controller->ControllerQueueDepth,
- Controller->MaxBlocksPerCommand);
- DAC960_Info(" Driver Queue Depth: %d, "
- "Scatter/Gather Limit: %d of %d Segments\n",
- Controller, Controller->DriverQueueDepth,
- Controller->DriverScatterGatherLimit,
- Controller->ControllerScatterGatherLimit);
- if (Controller->FirmwareType == DAC960_V1_Controller)
- {
- DAC960_Info(" Stripe Size: %dKB, Segment Size: %dKB, "
- "BIOS Geometry: %d/%d\n", Controller,
- Controller->V1.StripeSize,
- Controller->V1.SegmentSize,
- Controller->V1.GeometryTranslationHeads,
- Controller->V1.GeometryTranslationSectors);
- if (Controller->V1.SAFTE_EnclosureManagementEnabled)
- DAC960_Info(" SAF-TE Enclosure Management Enabled\n", Controller);
- }
- return true;
-}
-
-
-/*
- DAC960_V1_ReadDeviceConfiguration reads the Device Configuration Information
- for DAC960 V1 Firmware Controllers by requesting the SCSI Inquiry and SCSI
- Inquiry Unit Serial Number information for each device connected to
- Controller.
-*/
-
-static bool DAC960_V1_ReadDeviceConfiguration(DAC960_Controller_T
- *Controller)
-{
- struct dma_loaf local_dma;
-
- dma_addr_t DCDBs_dma[DAC960_V1_MaxChannels];
- DAC960_V1_DCDB_T *DCDBs_cpu[DAC960_V1_MaxChannels];
-
- dma_addr_t SCSI_Inquiry_dma[DAC960_V1_MaxChannels];
- DAC960_SCSI_Inquiry_T *SCSI_Inquiry_cpu[DAC960_V1_MaxChannels];
-
- dma_addr_t SCSI_NewInquiryUnitSerialNumberDMA[DAC960_V1_MaxChannels];
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *SCSI_NewInquiryUnitSerialNumberCPU[DAC960_V1_MaxChannels];
-
- struct completion Completions[DAC960_V1_MaxChannels];
- unsigned long flags;
- int Channel, TargetID;
-
- if (!init_dma_loaf(Controller->PCIDevice, &local_dma,
- DAC960_V1_MaxChannels*(sizeof(DAC960_V1_DCDB_T) +
- sizeof(DAC960_SCSI_Inquiry_T) +
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T))))
- return DAC960_Failure(Controller,
- "DMA ALLOCATION FAILED IN ReadDeviceConfiguration");
-
- for (Channel = 0; Channel < Controller->Channels; Channel++) {
- DCDBs_cpu[Channel] = slice_dma_loaf(&local_dma,
- sizeof(DAC960_V1_DCDB_T), DCDBs_dma + Channel);
- SCSI_Inquiry_cpu[Channel] = slice_dma_loaf(&local_dma,
- sizeof(DAC960_SCSI_Inquiry_T),
- SCSI_Inquiry_dma + Channel);
- SCSI_NewInquiryUnitSerialNumberCPU[Channel] = slice_dma_loaf(&local_dma,
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T),
- SCSI_NewInquiryUnitSerialNumberDMA + Channel);
- }
-
- for (TargetID = 0; TargetID < Controller->Targets; TargetID++)
- {
- /*
- * For each channel, submit a probe for a device on that channel.
- * The timeout interval for a device that is present is 10 seconds.
- * With this approach, the timeout periods can elapse in parallel
- * on each channel.
- */
- for (Channel = 0; Channel < Controller->Channels; Channel++)
- {
- dma_addr_t NewInquiryStandardDataDMA = SCSI_Inquiry_dma[Channel];
- DAC960_V1_DCDB_T *DCDB = DCDBs_cpu[Channel];
- dma_addr_t DCDB_dma = DCDBs_dma[Channel];
- DAC960_Command_T *Command = Controller->Commands[Channel];
- struct completion *Completion = &Completions[Channel];
-
- init_completion(Completion);
- DAC960_V1_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- Command->Completion = Completion;
- Command->V1.CommandMailbox.Type3.CommandOpcode = DAC960_V1_DCDB;
- Command->V1.CommandMailbox.Type3.BusAddress = DCDB_dma;
- DCDB->Channel = Channel;
- DCDB->TargetID = TargetID;
- DCDB->Direction = DAC960_V1_DCDB_DataTransferDeviceToSystem;
- DCDB->EarlyStatus = false;
- DCDB->Timeout = DAC960_V1_DCDB_Timeout_10_seconds;
- DCDB->NoAutomaticRequestSense = false;
- DCDB->DisconnectPermitted = true;
- DCDB->TransferLength = sizeof(DAC960_SCSI_Inquiry_T);
- DCDB->BusAddress = NewInquiryStandardDataDMA;
- DCDB->CDBLength = 6;
- DCDB->TransferLengthHigh4 = 0;
- DCDB->SenseLength = sizeof(DCDB->SenseData);
- DCDB->CDB[0] = 0x12; /* INQUIRY */
- DCDB->CDB[1] = 0; /* EVPD = 0 */
- DCDB->CDB[2] = 0; /* Page Code */
- DCDB->CDB[3] = 0; /* Reserved */
- DCDB->CDB[4] = sizeof(DAC960_SCSI_Inquiry_T);
- DCDB->CDB[5] = 0; /* Control */
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- DAC960_QueueCommand(Command);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- }
- /*
- * Wait for the problems submitted in the previous loop
- * to complete. On the probes that are successful,
- * get the serial number of the device that was found.
- */
- for (Channel = 0; Channel < Controller->Channels; Channel++)
- {
- DAC960_SCSI_Inquiry_T *InquiryStandardData =
- &Controller->V1.InquiryStandardData[Channel][TargetID];
- DAC960_SCSI_Inquiry_T *NewInquiryStandardData = SCSI_Inquiry_cpu[Channel];
- dma_addr_t NewInquiryUnitSerialNumberDMA =
- SCSI_NewInquiryUnitSerialNumberDMA[Channel];
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *NewInquiryUnitSerialNumber =
- SCSI_NewInquiryUnitSerialNumberCPU[Channel];
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *InquiryUnitSerialNumber =
- &Controller->V1.InquiryUnitSerialNumber[Channel][TargetID];
- DAC960_Command_T *Command = Controller->Commands[Channel];
- DAC960_V1_DCDB_T *DCDB = DCDBs_cpu[Channel];
- struct completion *Completion = &Completions[Channel];
-
- wait_for_completion(Completion);
-
- if (Command->V1.CommandStatus != DAC960_V1_NormalCompletion) {
- memset(InquiryStandardData, 0, sizeof(DAC960_SCSI_Inquiry_T));
- InquiryStandardData->PeripheralDeviceType = 0x1F;
- continue;
- } else
- memcpy(InquiryStandardData, NewInquiryStandardData, sizeof(DAC960_SCSI_Inquiry_T));
-
- /* Preserve Channel and TargetID values from the previous loop */
- Command->Completion = Completion;
- DCDB->TransferLength = sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T);
- DCDB->BusAddress = NewInquiryUnitSerialNumberDMA;
- DCDB->SenseLength = sizeof(DCDB->SenseData);
- DCDB->CDB[0] = 0x12; /* INQUIRY */
- DCDB->CDB[1] = 1; /* EVPD = 1 */
- DCDB->CDB[2] = 0x80; /* Page Code */
- DCDB->CDB[3] = 0; /* Reserved */
- DCDB->CDB[4] = sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T);
- DCDB->CDB[5] = 0; /* Control */
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- DAC960_QueueCommand(Command);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- wait_for_completion(Completion);
-
- if (Command->V1.CommandStatus != DAC960_V1_NormalCompletion) {
- memset(InquiryUnitSerialNumber, 0,
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T));
- InquiryUnitSerialNumber->PeripheralDeviceType = 0x1F;
- } else
- memcpy(InquiryUnitSerialNumber, NewInquiryUnitSerialNumber,
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T));
- }
- }
- free_dma_loaf(Controller->PCIDevice, &local_dma);
- return true;
-}
-
-
-/*
- DAC960_V2_ReadDeviceConfiguration reads the Device Configuration Information
- for DAC960 V2 Firmware Controllers by requesting the Physical Device
- Information and SCSI Inquiry Unit Serial Number information for each
- device connected to Controller.
-*/
-
-static bool DAC960_V2_ReadDeviceConfiguration(DAC960_Controller_T
- *Controller)
-{
- unsigned char Channel = 0, TargetID = 0, LogicalUnit = 0;
- unsigned short PhysicalDeviceIndex = 0;
-
- while (true)
- {
- DAC960_V2_PhysicalDeviceInfo_T *NewPhysicalDeviceInfo =
- Controller->V2.NewPhysicalDeviceInformation;
- DAC960_V2_PhysicalDeviceInfo_T *PhysicalDeviceInfo;
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *NewInquiryUnitSerialNumber =
- Controller->V2.NewInquiryUnitSerialNumber;
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *InquiryUnitSerialNumber;
-
- if (!DAC960_V2_NewPhysicalDeviceInfo(Controller, Channel, TargetID, LogicalUnit))
- break;
-
- PhysicalDeviceInfo = kmalloc(sizeof(DAC960_V2_PhysicalDeviceInfo_T),
- GFP_ATOMIC);
- if (PhysicalDeviceInfo == NULL)
- return DAC960_Failure(Controller, "PHYSICAL DEVICE ALLOCATION");
- Controller->V2.PhysicalDeviceInformation[PhysicalDeviceIndex] =
- PhysicalDeviceInfo;
- memcpy(PhysicalDeviceInfo, NewPhysicalDeviceInfo,
- sizeof(DAC960_V2_PhysicalDeviceInfo_T));
-
- InquiryUnitSerialNumber = kmalloc(
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T), GFP_ATOMIC);
- if (InquiryUnitSerialNumber == NULL) {
- kfree(PhysicalDeviceInfo);
- return DAC960_Failure(Controller, "SERIAL NUMBER ALLOCATION");
- }
- Controller->V2.InquiryUnitSerialNumber[PhysicalDeviceIndex] =
- InquiryUnitSerialNumber;
-
- Channel = NewPhysicalDeviceInfo->Channel;
- TargetID = NewPhysicalDeviceInfo->TargetID;
- LogicalUnit = NewPhysicalDeviceInfo->LogicalUnit;
-
- /*
- Some devices do NOT have Unit Serial Numbers.
- This command fails for them. But, we still want to
- remember those devices are there. Construct a
- UnitSerialNumber structure for the failure case.
- */
- if (!DAC960_V2_NewInquiryUnitSerialNumber(Controller, Channel, TargetID, LogicalUnit)) {
- memset(InquiryUnitSerialNumber, 0,
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T));
- InquiryUnitSerialNumber->PeripheralDeviceType = 0x1F;
- } else
- memcpy(InquiryUnitSerialNumber, NewInquiryUnitSerialNumber,
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T));
-
- PhysicalDeviceIndex++;
- LogicalUnit++;
- }
- return true;
-}
-
-
-/*
- DAC960_SanitizeInquiryData sanitizes the Vendor, Model, Revision, and
- Product Serial Number fields of the Inquiry Standard Data and Inquiry
- Unit Serial Number structures.
-*/
-
-static void DAC960_SanitizeInquiryData(DAC960_SCSI_Inquiry_T
- *InquiryStandardData,
- DAC960_SCSI_Inquiry_UnitSerialNumber_T
- *InquiryUnitSerialNumber,
- unsigned char *Vendor,
- unsigned char *Model,
- unsigned char *Revision,
- unsigned char *SerialNumber)
-{
- int SerialNumberLength, i;
- if (InquiryStandardData->PeripheralDeviceType == 0x1F) return;
- for (i = 0; i < sizeof(InquiryStandardData->VendorIdentification); i++)
- {
- unsigned char VendorCharacter =
- InquiryStandardData->VendorIdentification[i];
- Vendor[i] = (VendorCharacter >= ' ' && VendorCharacter <= '~'
- ? VendorCharacter : ' ');
- }
- Vendor[sizeof(InquiryStandardData->VendorIdentification)] = '\0';
- for (i = 0; i < sizeof(InquiryStandardData->ProductIdentification); i++)
- {
- unsigned char ModelCharacter =
- InquiryStandardData->ProductIdentification[i];
- Model[i] = (ModelCharacter >= ' ' && ModelCharacter <= '~'
- ? ModelCharacter : ' ');
- }
- Model[sizeof(InquiryStandardData->ProductIdentification)] = '\0';
- for (i = 0; i < sizeof(InquiryStandardData->ProductRevisionLevel); i++)
- {
- unsigned char RevisionCharacter =
- InquiryStandardData->ProductRevisionLevel[i];
- Revision[i] = (RevisionCharacter >= ' ' && RevisionCharacter <= '~'
- ? RevisionCharacter : ' ');
- }
- Revision[sizeof(InquiryStandardData->ProductRevisionLevel)] = '\0';
- if (InquiryUnitSerialNumber->PeripheralDeviceType == 0x1F) return;
- SerialNumberLength = InquiryUnitSerialNumber->PageLength;
- if (SerialNumberLength >
- sizeof(InquiryUnitSerialNumber->ProductSerialNumber))
- SerialNumberLength = sizeof(InquiryUnitSerialNumber->ProductSerialNumber);
- for (i = 0; i < SerialNumberLength; i++)
- {
- unsigned char SerialNumberCharacter =
- InquiryUnitSerialNumber->ProductSerialNumber[i];
- SerialNumber[i] =
- (SerialNumberCharacter >= ' ' && SerialNumberCharacter <= '~'
- ? SerialNumberCharacter : ' ');
- }
- SerialNumber[SerialNumberLength] = '\0';
-}
-
-
-/*
- DAC960_V1_ReportDeviceConfiguration reports the Device Configuration
- Information for DAC960 V1 Firmware Controllers.
-*/
-
-static bool DAC960_V1_ReportDeviceConfiguration(DAC960_Controller_T
- *Controller)
-{
- int LogicalDriveNumber, Channel, TargetID;
- DAC960_Info(" Physical Devices:\n", Controller);
- for (Channel = 0; Channel < Controller->Channels; Channel++)
- for (TargetID = 0; TargetID < Controller->Targets; TargetID++)
- {
- DAC960_SCSI_Inquiry_T *InquiryStandardData =
- &Controller->V1.InquiryStandardData[Channel][TargetID];
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *InquiryUnitSerialNumber =
- &Controller->V1.InquiryUnitSerialNumber[Channel][TargetID];
- DAC960_V1_DeviceState_T *DeviceState =
- &Controller->V1.DeviceState[Channel][TargetID];
- DAC960_V1_ErrorTableEntry_T *ErrorEntry =
- &Controller->V1.ErrorTable.ErrorTableEntries[Channel][TargetID];
- char Vendor[1+sizeof(InquiryStandardData->VendorIdentification)];
- char Model[1+sizeof(InquiryStandardData->ProductIdentification)];
- char Revision[1+sizeof(InquiryStandardData->ProductRevisionLevel)];
- char SerialNumber[1+sizeof(InquiryUnitSerialNumber
- ->ProductSerialNumber)];
- if (InquiryStandardData->PeripheralDeviceType == 0x1F) continue;
- DAC960_SanitizeInquiryData(InquiryStandardData, InquiryUnitSerialNumber,
- Vendor, Model, Revision, SerialNumber);
- DAC960_Info(" %d:%d%s Vendor: %s Model: %s Revision: %s\n",
- Controller, Channel, TargetID, (TargetID < 10 ? " " : ""),
- Vendor, Model, Revision);
- if (InquiryUnitSerialNumber->PeripheralDeviceType != 0x1F)
- DAC960_Info(" Serial Number: %s\n", Controller, SerialNumber);
- if (DeviceState->Present &&
- DeviceState->DeviceType == DAC960_V1_DiskType)
- {
- if (Controller->V1.DeviceResetCount[Channel][TargetID] > 0)
- DAC960_Info(" Disk Status: %s, %u blocks, %d resets\n",
- Controller,
- (DeviceState->DeviceState == DAC960_V1_Device_Dead
- ? "Dead"
- : DeviceState->DeviceState
- == DAC960_V1_Device_WriteOnly
- ? "Write-Only"
- : DeviceState->DeviceState
- == DAC960_V1_Device_Online
- ? "Online" : "Standby"),
- DeviceState->DiskSize,
- Controller->V1.DeviceResetCount[Channel][TargetID]);
- else
- DAC960_Info(" Disk Status: %s, %u blocks\n", Controller,
- (DeviceState->DeviceState == DAC960_V1_Device_Dead
- ? "Dead"
- : DeviceState->DeviceState
- == DAC960_V1_Device_WriteOnly
- ? "Write-Only"
- : DeviceState->DeviceState
- == DAC960_V1_Device_Online
- ? "Online" : "Standby"),
- DeviceState->DiskSize);
- }
- if (ErrorEntry->ParityErrorCount > 0 ||
- ErrorEntry->SoftErrorCount > 0 ||
- ErrorEntry->HardErrorCount > 0 ||
- ErrorEntry->MiscErrorCount > 0)
- DAC960_Info(" Errors - Parity: %d, Soft: %d, "
- "Hard: %d, Misc: %d\n", Controller,
- ErrorEntry->ParityErrorCount,
- ErrorEntry->SoftErrorCount,
- ErrorEntry->HardErrorCount,
- ErrorEntry->MiscErrorCount);
- }
- DAC960_Info(" Logical Drives:\n", Controller);
- for (LogicalDriveNumber = 0;
- LogicalDriveNumber < Controller->LogicalDriveCount;
- LogicalDriveNumber++)
- {
- DAC960_V1_LogicalDriveInformation_T *LogicalDriveInformation =
- &Controller->V1.LogicalDriveInformation[LogicalDriveNumber];
- DAC960_Info(" /dev/rd/c%dd%d: RAID-%d, %s, %u blocks, %s\n",
- Controller, Controller->ControllerNumber, LogicalDriveNumber,
- LogicalDriveInformation->RAIDLevel,
- (LogicalDriveInformation->LogicalDriveState
- == DAC960_V1_LogicalDrive_Online
- ? "Online"
- : LogicalDriveInformation->LogicalDriveState
- == DAC960_V1_LogicalDrive_Critical
- ? "Critical" : "Offline"),
- LogicalDriveInformation->LogicalDriveSize,
- (LogicalDriveInformation->WriteBack
- ? "Write Back" : "Write Thru"));
- }
- return true;
-}
-
-
-/*
- DAC960_V2_ReportDeviceConfiguration reports the Device Configuration
- Information for DAC960 V2 Firmware Controllers.
-*/
-
-static bool DAC960_V2_ReportDeviceConfiguration(DAC960_Controller_T
- *Controller)
-{
- int PhysicalDeviceIndex, LogicalDriveNumber;
- DAC960_Info(" Physical Devices:\n", Controller);
- for (PhysicalDeviceIndex = 0;
- PhysicalDeviceIndex < DAC960_V2_MaxPhysicalDevices;
- PhysicalDeviceIndex++)
- {
- DAC960_V2_PhysicalDeviceInfo_T *PhysicalDeviceInfo =
- Controller->V2.PhysicalDeviceInformation[PhysicalDeviceIndex];
- DAC960_SCSI_Inquiry_T *InquiryStandardData =
- (DAC960_SCSI_Inquiry_T *) &PhysicalDeviceInfo->SCSI_InquiryData;
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *InquiryUnitSerialNumber =
- Controller->V2.InquiryUnitSerialNumber[PhysicalDeviceIndex];
- char Vendor[1+sizeof(InquiryStandardData->VendorIdentification)];
- char Model[1+sizeof(InquiryStandardData->ProductIdentification)];
- char Revision[1+sizeof(InquiryStandardData->ProductRevisionLevel)];
- char SerialNumber[1+sizeof(InquiryUnitSerialNumber->ProductSerialNumber)];
- if (PhysicalDeviceInfo == NULL) break;
- DAC960_SanitizeInquiryData(InquiryStandardData, InquiryUnitSerialNumber,
- Vendor, Model, Revision, SerialNumber);
- DAC960_Info(" %d:%d%s Vendor: %s Model: %s Revision: %s\n",
- Controller,
- PhysicalDeviceInfo->Channel,
- PhysicalDeviceInfo->TargetID,
- (PhysicalDeviceInfo->TargetID < 10 ? " " : ""),
- Vendor, Model, Revision);
- if (PhysicalDeviceInfo->NegotiatedSynchronousMegaTransfers == 0)
- DAC960_Info(" %sAsynchronous\n", Controller,
- (PhysicalDeviceInfo->NegotiatedDataWidthBits == 16
- ? "Wide " :""));
- else
- DAC960_Info(" %sSynchronous at %d MB/sec\n", Controller,
- (PhysicalDeviceInfo->NegotiatedDataWidthBits == 16
- ? "Wide " :""),
- (PhysicalDeviceInfo->NegotiatedSynchronousMegaTransfers
- * PhysicalDeviceInfo->NegotiatedDataWidthBits/8));
- if (InquiryUnitSerialNumber->PeripheralDeviceType != 0x1F)
- DAC960_Info(" Serial Number: %s\n", Controller, SerialNumber);
- if (PhysicalDeviceInfo->PhysicalDeviceState ==
- DAC960_V2_Device_Unconfigured)
- continue;
- DAC960_Info(" Disk Status: %s, %u blocks\n", Controller,
- (PhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Online
- ? "Online"
- : PhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Rebuild
- ? "Rebuild"
- : PhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Missing
- ? "Missing"
- : PhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Critical
- ? "Critical"
- : PhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Dead
- ? "Dead"
- : PhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_SuspectedDead
- ? "Suspected-Dead"
- : PhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_CommandedOffline
- ? "Commanded-Offline"
- : PhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Standby
- ? "Standby" : "Unknown"),
- PhysicalDeviceInfo->ConfigurableDeviceSize);
- if (PhysicalDeviceInfo->ParityErrors == 0 &&
- PhysicalDeviceInfo->SoftErrors == 0 &&
- PhysicalDeviceInfo->HardErrors == 0 &&
- PhysicalDeviceInfo->MiscellaneousErrors == 0 &&
- PhysicalDeviceInfo->CommandTimeouts == 0 &&
- PhysicalDeviceInfo->Retries == 0 &&
- PhysicalDeviceInfo->Aborts == 0 &&
- PhysicalDeviceInfo->PredictedFailuresDetected == 0)
- continue;
- DAC960_Info(" Errors - Parity: %d, Soft: %d, "
- "Hard: %d, Misc: %d\n", Controller,
- PhysicalDeviceInfo->ParityErrors,
- PhysicalDeviceInfo->SoftErrors,
- PhysicalDeviceInfo->HardErrors,
- PhysicalDeviceInfo->MiscellaneousErrors);
- DAC960_Info(" Timeouts: %d, Retries: %d, "
- "Aborts: %d, Predicted: %d\n", Controller,
- PhysicalDeviceInfo->CommandTimeouts,
- PhysicalDeviceInfo->Retries,
- PhysicalDeviceInfo->Aborts,
- PhysicalDeviceInfo->PredictedFailuresDetected);
- }
- DAC960_Info(" Logical Drives:\n", Controller);
- for (LogicalDriveNumber = 0;
- LogicalDriveNumber < DAC960_MaxLogicalDrives;
- LogicalDriveNumber++)
- {
- DAC960_V2_LogicalDeviceInfo_T *LogicalDeviceInfo =
- Controller->V2.LogicalDeviceInformation[LogicalDriveNumber];
- static const unsigned char *ReadCacheStatus[] = {
- "Read Cache Disabled",
- "Read Cache Enabled",
- "Read Ahead Enabled",
- "Intelligent Read Ahead Enabled",
- "-", "-", "-", "-"
- };
- static const unsigned char *WriteCacheStatus[] = {
- "Write Cache Disabled",
- "Logical Device Read Only",
- "Write Cache Enabled",
- "Intelligent Write Cache Enabled",
- "-", "-", "-", "-"
- };
- unsigned char *GeometryTranslation;
- if (LogicalDeviceInfo == NULL) continue;
- switch (LogicalDeviceInfo->DriveGeometry)
- {
- case DAC960_V2_Geometry_128_32:
- GeometryTranslation = "128/32";
- break;
- case DAC960_V2_Geometry_255_63:
- GeometryTranslation = "255/63";
- break;
- default:
- GeometryTranslation = "Invalid";
- DAC960_Error("Illegal Logical Device Geometry %d\n",
- Controller, LogicalDeviceInfo->DriveGeometry);
- break;
- }
- DAC960_Info(" /dev/rd/c%dd%d: RAID-%d, %s, %u blocks\n",
- Controller, Controller->ControllerNumber, LogicalDriveNumber,
- LogicalDeviceInfo->RAIDLevel,
- (LogicalDeviceInfo->LogicalDeviceState
- == DAC960_V2_LogicalDevice_Online
- ? "Online"
- : LogicalDeviceInfo->LogicalDeviceState
- == DAC960_V2_LogicalDevice_Critical
- ? "Critical" : "Offline"),
- LogicalDeviceInfo->ConfigurableDeviceSize);
- DAC960_Info(" Logical Device %s, BIOS Geometry: %s\n",
- Controller,
- (LogicalDeviceInfo->LogicalDeviceControl
- .LogicalDeviceInitialized
- ? "Initialized" : "Uninitialized"),
- GeometryTranslation);
- if (LogicalDeviceInfo->StripeSize == 0)
- {
- if (LogicalDeviceInfo->CacheLineSize == 0)
- DAC960_Info(" Stripe Size: N/A, "
- "Segment Size: N/A\n", Controller);
- else
- DAC960_Info(" Stripe Size: N/A, "
- "Segment Size: %dKB\n", Controller,
- 1 << (LogicalDeviceInfo->CacheLineSize - 2));
- }
- else
- {
- if (LogicalDeviceInfo->CacheLineSize == 0)
- DAC960_Info(" Stripe Size: %dKB, "
- "Segment Size: N/A\n", Controller,
- 1 << (LogicalDeviceInfo->StripeSize - 2));
- else
- DAC960_Info(" Stripe Size: %dKB, "
- "Segment Size: %dKB\n", Controller,
- 1 << (LogicalDeviceInfo->StripeSize - 2),
- 1 << (LogicalDeviceInfo->CacheLineSize - 2));
- }
- DAC960_Info(" %s, %s\n", Controller,
- ReadCacheStatus[
- LogicalDeviceInfo->LogicalDeviceControl.ReadCache],
- WriteCacheStatus[
- LogicalDeviceInfo->LogicalDeviceControl.WriteCache]);
- if (LogicalDeviceInfo->SoftErrors > 0 ||
- LogicalDeviceInfo->CommandsFailed > 0 ||
- LogicalDeviceInfo->DeferredWriteErrors)
- DAC960_Info(" Errors - Soft: %d, Failed: %d, "
- "Deferred Write: %d\n", Controller,
- LogicalDeviceInfo->SoftErrors,
- LogicalDeviceInfo->CommandsFailed,
- LogicalDeviceInfo->DeferredWriteErrors);
-
- }
- return true;
-}
-
-/*
- DAC960_RegisterBlockDevice registers the Block Device structures
- associated with Controller.
-*/
-
-static bool DAC960_RegisterBlockDevice(DAC960_Controller_T *Controller)
-{
- int MajorNumber = DAC960_MAJOR + Controller->ControllerNumber;
- int n;
-
- /*
- Register the Block Device Major Number for this DAC960 Controller.
- */
- if (register_blkdev(MajorNumber, "dac960") < 0)
- return false;
-
- for (n = 0; n < DAC960_MaxLogicalDrives; n++) {
- struct gendisk *disk = Controller->disks[n];
- struct request_queue *RequestQueue;
-
- /* for now, let all request queues share controller's lock */
- RequestQueue = blk_init_queue(DAC960_RequestFunction,&Controller->queue_lock);
- if (!RequestQueue) {
- printk("DAC960: failure to allocate request queue\n");
- continue;
- }
- Controller->RequestQueue[n] = RequestQueue;
- RequestQueue->queuedata = Controller;
- blk_queue_max_segments(RequestQueue, Controller->DriverScatterGatherLimit);
- blk_queue_max_hw_sectors(RequestQueue, Controller->MaxBlocksPerCommand);
- disk->queue = RequestQueue;
- sprintf(disk->disk_name, "rd/c%dd%d", Controller->ControllerNumber, n);
- disk->major = MajorNumber;
- disk->first_minor = n << DAC960_MaxPartitionsBits;
- disk->fops = &DAC960_BlockDeviceOperations;
- }
- /*
- Indicate the Block Device Registration completed successfully,
- */
- return true;
-}
-
-
-/*
- DAC960_UnregisterBlockDevice unregisters the Block Device structures
- associated with Controller.
-*/
-
-static void DAC960_UnregisterBlockDevice(DAC960_Controller_T *Controller)
-{
- int MajorNumber = DAC960_MAJOR + Controller->ControllerNumber;
- int disk;
-
- /* does order matter when deleting gendisk and cleanup in request queue? */
- for (disk = 0; disk < DAC960_MaxLogicalDrives; disk++) {
- del_gendisk(Controller->disks[disk]);
- blk_cleanup_queue(Controller->RequestQueue[disk]);
- Controller->RequestQueue[disk] = NULL;
- }
-
- /*
- Unregister the Block Device Major Number for this DAC960 Controller.
- */
- unregister_blkdev(MajorNumber, "dac960");
-}
-
-/*
- DAC960_ComputeGenericDiskInfo computes the values for the Generic Disk
- Information Partition Sector Counts and Block Sizes.
-*/
-
-static void DAC960_ComputeGenericDiskInfo(DAC960_Controller_T *Controller)
-{
- int disk;
- for (disk = 0; disk < DAC960_MaxLogicalDrives; disk++)
- set_capacity(Controller->disks[disk], disk_size(Controller, disk));
-}
-
-/*
- DAC960_ReportErrorStatus reports Controller BIOS Messages passed through
- the Error Status Register when the driver performs the BIOS handshaking.
- It returns true for fatal errors and false otherwise.
-*/
-
-static bool DAC960_ReportErrorStatus(DAC960_Controller_T *Controller,
- unsigned char ErrorStatus,
- unsigned char Parameter0,
- unsigned char Parameter1)
-{
- switch (ErrorStatus)
- {
- case 0x00:
- DAC960_Notice("Physical Device %d:%d Not Responding\n",
- Controller, Parameter1, Parameter0);
- break;
- case 0x08:
- if (Controller->DriveSpinUpMessageDisplayed) break;
- DAC960_Notice("Spinning Up Drives\n", Controller);
- Controller->DriveSpinUpMessageDisplayed = true;
- break;
- case 0x30:
- DAC960_Notice("Configuration Checksum Error\n", Controller);
- break;
- case 0x60:
- DAC960_Notice("Mirror Race Recovery Failed\n", Controller);
- break;
- case 0x70:
- DAC960_Notice("Mirror Race Recovery In Progress\n", Controller);
- break;
- case 0x90:
- DAC960_Notice("Physical Device %d:%d COD Mismatch\n",
- Controller, Parameter1, Parameter0);
- break;
- case 0xA0:
- DAC960_Notice("Logical Drive Installation Aborted\n", Controller);
- break;
- case 0xB0:
- DAC960_Notice("Mirror Race On A Critical Logical Drive\n", Controller);
- break;
- case 0xD0:
- DAC960_Notice("New Controller Configuration Found\n", Controller);
- break;
- case 0xF0:
- DAC960_Error("Fatal Memory Parity Error for Controller at\n", Controller);
- return true;
- default:
- DAC960_Error("Unknown Initialization Error %02X for Controller at\n",
- Controller, ErrorStatus);
- return true;
- }
- return false;
-}
-
-
-/*
- * DAC960_DetectCleanup releases the resources that were allocated
- * during DAC960_DetectController(). DAC960_DetectController can
- * has several internal failure points, so not ALL resources may
- * have been allocated. It's important to free only
- * resources that HAVE been allocated. The code below always
- * tests that the resource has been allocated before attempting to
- * free it.
- */
-static void DAC960_DetectCleanup(DAC960_Controller_T *Controller)
-{
- int i;
-
- /* Free the memory mailbox, status, and related structures */
- free_dma_loaf(Controller->PCIDevice, &Controller->DmaPages);
- if (Controller->MemoryMappedAddress) {
- switch(Controller->HardwareType)
- {
- case DAC960_GEM_Controller:
- DAC960_GEM_DisableInterrupts(Controller->BaseAddress);
- break;
- case DAC960_BA_Controller:
- DAC960_BA_DisableInterrupts(Controller->BaseAddress);
- break;
- case DAC960_LP_Controller:
- DAC960_LP_DisableInterrupts(Controller->BaseAddress);
- break;
- case DAC960_LA_Controller:
- DAC960_LA_DisableInterrupts(Controller->BaseAddress);
- break;
- case DAC960_PG_Controller:
- DAC960_PG_DisableInterrupts(Controller->BaseAddress);
- break;
- case DAC960_PD_Controller:
- DAC960_PD_DisableInterrupts(Controller->BaseAddress);
- break;
- case DAC960_P_Controller:
- DAC960_PD_DisableInterrupts(Controller->BaseAddress);
- break;
- }
- iounmap(Controller->MemoryMappedAddress);
- }
- if (Controller->IRQ_Channel)
- free_irq(Controller->IRQ_Channel, Controller);
- if (Controller->IO_Address)
- release_region(Controller->IO_Address, 0x80);
- pci_disable_device(Controller->PCIDevice);
- for (i = 0; (i < DAC960_MaxLogicalDrives) && Controller->disks[i]; i++)
- put_disk(Controller->disks[i]);
- DAC960_Controllers[Controller->ControllerNumber] = NULL;
- kfree(Controller);
-}
-
-
-/*
- DAC960_DetectController detects Mylex DAC960/AcceleRAID/eXtremeRAID
- PCI RAID Controllers by interrogating the PCI Configuration Space for
- Controller Type.
-*/
-
-static DAC960_Controller_T *
-DAC960_DetectController(struct pci_dev *PCI_Device,
- const struct pci_device_id *entry)
-{
- struct DAC960_privdata *privdata =
- (struct DAC960_privdata *)entry->driver_data;
- irq_handler_t InterruptHandler = privdata->InterruptHandler;
- unsigned int MemoryWindowSize = privdata->MemoryWindowSize;
- DAC960_Controller_T *Controller = NULL;
- unsigned char DeviceFunction = PCI_Device->devfn;
- unsigned char ErrorStatus, Parameter0, Parameter1;
- unsigned int IRQ_Channel;
- void __iomem *BaseAddress;
- int i;
-
- Controller = kzalloc(sizeof(DAC960_Controller_T), GFP_ATOMIC);
- if (Controller == NULL) {
- DAC960_Error("Unable to allocate Controller structure for "
- "Controller at\n", NULL);
- return NULL;
- }
- Controller->ControllerNumber = DAC960_ControllerCount;
- DAC960_Controllers[DAC960_ControllerCount++] = Controller;
- Controller->Bus = PCI_Device->bus->number;
- Controller->FirmwareType = privdata->FirmwareType;
- Controller->HardwareType = privdata->HardwareType;
- Controller->Device = DeviceFunction >> 3;
- Controller->Function = DeviceFunction & 0x7;
- Controller->PCIDevice = PCI_Device;
- strcpy(Controller->FullModelName, "DAC960");
-
- if (pci_enable_device(PCI_Device))
- goto Failure;
-
- switch (Controller->HardwareType)
- {
- case DAC960_GEM_Controller:
- Controller->PCI_Address = pci_resource_start(PCI_Device, 0);
- break;
- case DAC960_BA_Controller:
- Controller->PCI_Address = pci_resource_start(PCI_Device, 0);
- break;
- case DAC960_LP_Controller:
- Controller->PCI_Address = pci_resource_start(PCI_Device, 0);
- break;
- case DAC960_LA_Controller:
- Controller->PCI_Address = pci_resource_start(PCI_Device, 0);
- break;
- case DAC960_PG_Controller:
- Controller->PCI_Address = pci_resource_start(PCI_Device, 0);
- break;
- case DAC960_PD_Controller:
- Controller->IO_Address = pci_resource_start(PCI_Device, 0);
- Controller->PCI_Address = pci_resource_start(PCI_Device, 1);
- break;
- case DAC960_P_Controller:
- Controller->IO_Address = pci_resource_start(PCI_Device, 0);
- Controller->PCI_Address = pci_resource_start(PCI_Device, 1);
- break;
- }
-
- pci_set_drvdata(PCI_Device, (void *)((long)Controller->ControllerNumber));
- for (i = 0; i < DAC960_MaxLogicalDrives; i++) {
- Controller->disks[i] = alloc_disk(1<<DAC960_MaxPartitionsBits);
- if (!Controller->disks[i])
- goto Failure;
- Controller->disks[i]->private_data = (void *)((long)i);
- }
- init_waitqueue_head(&Controller->CommandWaitQueue);
- init_waitqueue_head(&Controller->HealthStatusWaitQueue);
- spin_lock_init(&Controller->queue_lock);
- DAC960_AnnounceDriver(Controller);
- /*
- Map the Controller Register Window.
- */
- if (MemoryWindowSize < PAGE_SIZE)
- MemoryWindowSize = PAGE_SIZE;
- Controller->MemoryMappedAddress =
- ioremap_nocache(Controller->PCI_Address & PAGE_MASK, MemoryWindowSize);
- Controller->BaseAddress =
- Controller->MemoryMappedAddress + (Controller->PCI_Address & ~PAGE_MASK);
- if (Controller->MemoryMappedAddress == NULL)
- {
- DAC960_Error("Unable to map Controller Register Window for "
- "Controller at\n", Controller);
- goto Failure;
- }
- BaseAddress = Controller->BaseAddress;
- switch (Controller->HardwareType)
- {
- case DAC960_GEM_Controller:
- DAC960_GEM_DisableInterrupts(BaseAddress);
- DAC960_GEM_AcknowledgeHardwareMailboxStatus(BaseAddress);
- udelay(1000);
- while (DAC960_GEM_InitializationInProgressP(BaseAddress))
- {
- if (DAC960_GEM_ReadErrorStatus(BaseAddress, &ErrorStatus,
- &Parameter0, &Parameter1) &&
- DAC960_ReportErrorStatus(Controller, ErrorStatus,
- Parameter0, Parameter1))
- goto Failure;
- udelay(10);
- }
- if (!DAC960_V2_EnableMemoryMailboxInterface(Controller))
- {
- DAC960_Error("Unable to Enable Memory Mailbox Interface "
- "for Controller at\n", Controller);
- goto Failure;
- }
- DAC960_GEM_EnableInterrupts(BaseAddress);
- Controller->QueueCommand = DAC960_GEM_QueueCommand;
- Controller->ReadControllerConfiguration =
- DAC960_V2_ReadControllerConfiguration;
- Controller->ReadDeviceConfiguration =
- DAC960_V2_ReadDeviceConfiguration;
- Controller->ReportDeviceConfiguration =
- DAC960_V2_ReportDeviceConfiguration;
- Controller->QueueReadWriteCommand =
- DAC960_V2_QueueReadWriteCommand;
- break;
- case DAC960_BA_Controller:
- DAC960_BA_DisableInterrupts(BaseAddress);
- DAC960_BA_AcknowledgeHardwareMailboxStatus(BaseAddress);
- udelay(1000);
- while (DAC960_BA_InitializationInProgressP(BaseAddress))
- {
- if (DAC960_BA_ReadErrorStatus(BaseAddress, &ErrorStatus,
- &Parameter0, &Parameter1) &&
- DAC960_ReportErrorStatus(Controller, ErrorStatus,
- Parameter0, Parameter1))
- goto Failure;
- udelay(10);
- }
- if (!DAC960_V2_EnableMemoryMailboxInterface(Controller))
- {
- DAC960_Error("Unable to Enable Memory Mailbox Interface "
- "for Controller at\n", Controller);
- goto Failure;
- }
- DAC960_BA_EnableInterrupts(BaseAddress);
- Controller->QueueCommand = DAC960_BA_QueueCommand;
- Controller->ReadControllerConfiguration =
- DAC960_V2_ReadControllerConfiguration;
- Controller->ReadDeviceConfiguration =
- DAC960_V2_ReadDeviceConfiguration;
- Controller->ReportDeviceConfiguration =
- DAC960_V2_ReportDeviceConfiguration;
- Controller->QueueReadWriteCommand =
- DAC960_V2_QueueReadWriteCommand;
- break;
- case DAC960_LP_Controller:
- DAC960_LP_DisableInterrupts(BaseAddress);
- DAC960_LP_AcknowledgeHardwareMailboxStatus(BaseAddress);
- udelay(1000);
- while (DAC960_LP_InitializationInProgressP(BaseAddress))
- {
- if (DAC960_LP_ReadErrorStatus(BaseAddress, &ErrorStatus,
- &Parameter0, &Parameter1) &&
- DAC960_ReportErrorStatus(Controller, ErrorStatus,
- Parameter0, Parameter1))
- goto Failure;
- udelay(10);
- }
- if (!DAC960_V2_EnableMemoryMailboxInterface(Controller))
- {
- DAC960_Error("Unable to Enable Memory Mailbox Interface "
- "for Controller at\n", Controller);
- goto Failure;
- }
- DAC960_LP_EnableInterrupts(BaseAddress);
- Controller->QueueCommand = DAC960_LP_QueueCommand;
- Controller->ReadControllerConfiguration =
- DAC960_V2_ReadControllerConfiguration;
- Controller->ReadDeviceConfiguration =
- DAC960_V2_ReadDeviceConfiguration;
- Controller->ReportDeviceConfiguration =
- DAC960_V2_ReportDeviceConfiguration;
- Controller->QueueReadWriteCommand =
- DAC960_V2_QueueReadWriteCommand;
- break;
- case DAC960_LA_Controller:
- DAC960_LA_DisableInterrupts(BaseAddress);
- DAC960_LA_AcknowledgeHardwareMailboxStatus(BaseAddress);
- udelay(1000);
- while (DAC960_LA_InitializationInProgressP(BaseAddress))
- {
- if (DAC960_LA_ReadErrorStatus(BaseAddress, &ErrorStatus,
- &Parameter0, &Parameter1) &&
- DAC960_ReportErrorStatus(Controller, ErrorStatus,
- Parameter0, Parameter1))
- goto Failure;
- udelay(10);
- }
- if (!DAC960_V1_EnableMemoryMailboxInterface(Controller))
- {
- DAC960_Error("Unable to Enable Memory Mailbox Interface "
- "for Controller at\n", Controller);
- goto Failure;
- }
- DAC960_LA_EnableInterrupts(BaseAddress);
- if (Controller->V1.DualModeMemoryMailboxInterface)
- Controller->QueueCommand = DAC960_LA_QueueCommandDualMode;
- else Controller->QueueCommand = DAC960_LA_QueueCommandSingleMode;
- Controller->ReadControllerConfiguration =
- DAC960_V1_ReadControllerConfiguration;
- Controller->ReadDeviceConfiguration =
- DAC960_V1_ReadDeviceConfiguration;
- Controller->ReportDeviceConfiguration =
- DAC960_V1_ReportDeviceConfiguration;
- Controller->QueueReadWriteCommand =
- DAC960_V1_QueueReadWriteCommand;
- break;
- case DAC960_PG_Controller:
- DAC960_PG_DisableInterrupts(BaseAddress);
- DAC960_PG_AcknowledgeHardwareMailboxStatus(BaseAddress);
- udelay(1000);
- while (DAC960_PG_InitializationInProgressP(BaseAddress))
- {
- if (DAC960_PG_ReadErrorStatus(BaseAddress, &ErrorStatus,
- &Parameter0, &Parameter1) &&
- DAC960_ReportErrorStatus(Controller, ErrorStatus,
- Parameter0, Parameter1))
- goto Failure;
- udelay(10);
- }
- if (!DAC960_V1_EnableMemoryMailboxInterface(Controller))
- {
- DAC960_Error("Unable to Enable Memory Mailbox Interface "
- "for Controller at\n", Controller);
- goto Failure;
- }
- DAC960_PG_EnableInterrupts(BaseAddress);
- if (Controller->V1.DualModeMemoryMailboxInterface)
- Controller->QueueCommand = DAC960_PG_QueueCommandDualMode;
- else Controller->QueueCommand = DAC960_PG_QueueCommandSingleMode;
- Controller->ReadControllerConfiguration =
- DAC960_V1_ReadControllerConfiguration;
- Controller->ReadDeviceConfiguration =
- DAC960_V1_ReadDeviceConfiguration;
- Controller->ReportDeviceConfiguration =
- DAC960_V1_ReportDeviceConfiguration;
- Controller->QueueReadWriteCommand =
- DAC960_V1_QueueReadWriteCommand;
- break;
- case DAC960_PD_Controller:
- if (!request_region(Controller->IO_Address, 0x80,
- Controller->FullModelName)) {
- DAC960_Error("IO port 0x%lx busy for Controller at\n",
- Controller, Controller->IO_Address);
- goto Failure;
- }
- DAC960_PD_DisableInterrupts(BaseAddress);
- DAC960_PD_AcknowledgeStatus(BaseAddress);
- udelay(1000);
- while (DAC960_PD_InitializationInProgressP(BaseAddress))
- {
- if (DAC960_PD_ReadErrorStatus(BaseAddress, &ErrorStatus,
- &Parameter0, &Parameter1) &&
- DAC960_ReportErrorStatus(Controller, ErrorStatus,
- Parameter0, Parameter1))
- goto Failure;
- udelay(10);
- }
- if (!DAC960_V1_EnableMemoryMailboxInterface(Controller))
- {
- DAC960_Error("Unable to allocate DMA mapped memory "
- "for Controller at\n", Controller);
- goto Failure;
- }
- DAC960_PD_EnableInterrupts(BaseAddress);
- Controller->QueueCommand = DAC960_PD_QueueCommand;
- Controller->ReadControllerConfiguration =
- DAC960_V1_ReadControllerConfiguration;
- Controller->ReadDeviceConfiguration =
- DAC960_V1_ReadDeviceConfiguration;
- Controller->ReportDeviceConfiguration =
- DAC960_V1_ReportDeviceConfiguration;
- Controller->QueueReadWriteCommand =
- DAC960_V1_QueueReadWriteCommand;
- break;
- case DAC960_P_Controller:
- if (!request_region(Controller->IO_Address, 0x80,
- Controller->FullModelName)){
- DAC960_Error("IO port 0x%lx busy for Controller at\n",
- Controller, Controller->IO_Address);
- goto Failure;
- }
- DAC960_PD_DisableInterrupts(BaseAddress);
- DAC960_PD_AcknowledgeStatus(BaseAddress);
- udelay(1000);
- while (DAC960_PD_InitializationInProgressP(BaseAddress))
- {
- if (DAC960_PD_ReadErrorStatus(BaseAddress, &ErrorStatus,
- &Parameter0, &Parameter1) &&
- DAC960_ReportErrorStatus(Controller, ErrorStatus,
- Parameter0, Parameter1))
- goto Failure;
- udelay(10);
- }
- if (!DAC960_V1_EnableMemoryMailboxInterface(Controller))
- {
- DAC960_Error("Unable to allocate DMA mapped memory"
- "for Controller at\n", Controller);
- goto Failure;
- }
- DAC960_PD_EnableInterrupts(BaseAddress);
- Controller->QueueCommand = DAC960_P_QueueCommand;
- Controller->ReadControllerConfiguration =
- DAC960_V1_ReadControllerConfiguration;
- Controller->ReadDeviceConfiguration =
- DAC960_V1_ReadDeviceConfiguration;
- Controller->ReportDeviceConfiguration =
- DAC960_V1_ReportDeviceConfiguration;
- Controller->QueueReadWriteCommand =
- DAC960_V1_QueueReadWriteCommand;
- break;
- }
- /*
- Acquire shared access to the IRQ Channel.
- */
- IRQ_Channel = PCI_Device->irq;
- if (request_irq(IRQ_Channel, InterruptHandler, IRQF_SHARED,
- Controller->FullModelName, Controller) < 0)
- {
- DAC960_Error("Unable to acquire IRQ Channel %d for Controller at\n",
- Controller, Controller->IRQ_Channel);
- goto Failure;
- }
- Controller->IRQ_Channel = IRQ_Channel;
- Controller->InitialCommand.CommandIdentifier = 1;
- Controller->InitialCommand.Controller = Controller;
- Controller->Commands[0] = &Controller->InitialCommand;
- Controller->FreeCommands = &Controller->InitialCommand;
- return Controller;
-
-Failure:
- if (Controller->IO_Address == 0)
- DAC960_Error("PCI Bus %d Device %d Function %d I/O Address N/A "
- "PCI Address 0x%X\n", Controller,
- Controller->Bus, Controller->Device,
- Controller->Function, Controller->PCI_Address);
- else
- DAC960_Error("PCI Bus %d Device %d Function %d I/O Address "
- "0x%X PCI Address 0x%X\n", Controller,
- Controller->Bus, Controller->Device,
- Controller->Function, Controller->IO_Address,
- Controller->PCI_Address);
- DAC960_DetectCleanup(Controller);
- DAC960_ControllerCount--;
- return NULL;
-}
-
-/*
- DAC960_InitializeController initializes Controller.
-*/
-
-static bool
-DAC960_InitializeController(DAC960_Controller_T *Controller)
-{
- if (DAC960_ReadControllerConfiguration(Controller) &&
- DAC960_ReportControllerConfiguration(Controller) &&
- DAC960_CreateAuxiliaryStructures(Controller) &&
- DAC960_ReadDeviceConfiguration(Controller) &&
- DAC960_ReportDeviceConfiguration(Controller) &&
- DAC960_RegisterBlockDevice(Controller))
- {
- /*
- Initialize the Monitoring Timer.
- */
- timer_setup(&Controller->MonitoringTimer,
- DAC960_MonitoringTimerFunction, 0);
- Controller->MonitoringTimer.expires =
- jiffies + DAC960_MonitoringTimerInterval;
- add_timer(&Controller->MonitoringTimer);
- Controller->ControllerInitialized = true;
- return true;
- }
- return false;
-}
-
-
-/*
- DAC960_FinalizeController finalizes Controller.
-*/
-
-static void DAC960_FinalizeController(DAC960_Controller_T *Controller)
-{
- if (Controller->ControllerInitialized)
- {
- unsigned long flags;
-
- /*
- * Acquiring and releasing lock here eliminates
- * a very low probability race.
- *
- * The code below allocates controller command structures
- * from the free list without holding the controller lock.
- * This is safe assuming there is no other activity on
- * the controller at the time.
- *
- * But, there might be a monitoring command still
- * in progress. Setting the Shutdown flag while holding
- * the lock ensures that there is no monitoring command
- * in the interrupt handler currently, and any monitoring
- * commands that complete from this time on will NOT return
- * their command structure to the free list.
- */
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- Controller->ShutdownMonitoringTimer = 1;
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
-
- del_timer_sync(&Controller->MonitoringTimer);
- if (Controller->FirmwareType == DAC960_V1_Controller)
- {
- DAC960_Notice("Flushing Cache...", Controller);
- DAC960_V1_ExecuteType3(Controller, DAC960_V1_Flush, 0);
- DAC960_Notice("done\n", Controller);
-
- if (Controller->HardwareType == DAC960_PD_Controller)
- release_region(Controller->IO_Address, 0x80);
- }
- else
- {
- DAC960_Notice("Flushing Cache...", Controller);
- DAC960_V2_DeviceOperation(Controller, DAC960_V2_PauseDevice,
- DAC960_V2_RAID_Controller);
- DAC960_Notice("done\n", Controller);
- }
- }
- DAC960_UnregisterBlockDevice(Controller);
- DAC960_DestroyAuxiliaryStructures(Controller);
- DAC960_DestroyProcEntries(Controller);
- DAC960_DetectCleanup(Controller);
-}
-
-
-/*
- DAC960_Probe verifies controller's existence and
- initializes the DAC960 Driver for that controller.
-*/
-
-static int
-DAC960_Probe(struct pci_dev *dev, const struct pci_device_id *entry)
-{
- int disk;
- DAC960_Controller_T *Controller;
-
- if (DAC960_ControllerCount == DAC960_MaxControllers)
- {
- DAC960_Error("More than %d DAC960 Controllers detected - "
- "ignoring from Controller at\n",
- NULL, DAC960_MaxControllers);
- return -ENODEV;
- }
-
- Controller = DAC960_DetectController(dev, entry);
- if (!Controller)
- return -ENODEV;
-
- if (!DAC960_InitializeController(Controller)) {
- DAC960_FinalizeController(Controller);
- return -ENODEV;
- }
-
- for (disk = 0; disk < DAC960_MaxLogicalDrives; disk++) {
- set_capacity(Controller->disks[disk], disk_size(Controller, disk));
- add_disk(Controller->disks[disk]);
- }
- DAC960_CreateProcEntries(Controller);
- return 0;
-}
-
-
-/*
- DAC960_Finalize finalizes the DAC960 Driver.
-*/
-
-static void DAC960_Remove(struct pci_dev *PCI_Device)
-{
- int Controller_Number = (long)pci_get_drvdata(PCI_Device);
- DAC960_Controller_T *Controller = DAC960_Controllers[Controller_Number];
- if (Controller != NULL)
- DAC960_FinalizeController(Controller);
-}
-
-
-/*
- DAC960_V1_QueueReadWriteCommand prepares and queues a Read/Write Command for
- DAC960 V1 Firmware Controllers.
-*/
-
-static void DAC960_V1_QueueReadWriteCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- DAC960_V1_ScatterGatherSegment_T *ScatterGatherList =
- Command->V1.ScatterGatherList;
- struct scatterlist *ScatterList = Command->V1.ScatterList;
-
- DAC960_V1_ClearCommand(Command);
-
- if (Command->SegmentCount == 1)
- {
- if (Command->DmaDirection == PCI_DMA_FROMDEVICE)
- CommandMailbox->Type5.CommandOpcode = DAC960_V1_Read;
- else
- CommandMailbox->Type5.CommandOpcode = DAC960_V1_Write;
-
- CommandMailbox->Type5.LD.TransferLength = Command->BlockCount;
- CommandMailbox->Type5.LD.LogicalDriveNumber = Command->LogicalDriveNumber;
- CommandMailbox->Type5.LogicalBlockAddress = Command->BlockNumber;
- CommandMailbox->Type5.BusAddress =
- (DAC960_BusAddress32_T)sg_dma_address(ScatterList);
- }
- else
- {
- int i;
-
- if (Command->DmaDirection == PCI_DMA_FROMDEVICE)
- CommandMailbox->Type5.CommandOpcode = DAC960_V1_ReadWithScatterGather;
- else
- CommandMailbox->Type5.CommandOpcode = DAC960_V1_WriteWithScatterGather;
-
- CommandMailbox->Type5.LD.TransferLength = Command->BlockCount;
- CommandMailbox->Type5.LD.LogicalDriveNumber = Command->LogicalDriveNumber;
- CommandMailbox->Type5.LogicalBlockAddress = Command->BlockNumber;
- CommandMailbox->Type5.BusAddress = Command->V1.ScatterGatherListDMA;
-
- CommandMailbox->Type5.ScatterGatherCount = Command->SegmentCount;
-
- for (i = 0; i < Command->SegmentCount; i++, ScatterList++, ScatterGatherList++) {
- ScatterGatherList->SegmentDataPointer =
- (DAC960_BusAddress32_T)sg_dma_address(ScatterList);
- ScatterGatherList->SegmentByteCount =
- (DAC960_ByteCount32_T)sg_dma_len(ScatterList);
- }
- }
- DAC960_QueueCommand(Command);
-}
-
-
-/*
- DAC960_V2_QueueReadWriteCommand prepares and queues a Read/Write Command for
- DAC960 V2 Firmware Controllers.
-*/
-
-static void DAC960_V2_QueueReadWriteCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- DAC960_V2_CommandMailbox_T *CommandMailbox = &Command->V2.CommandMailbox;
- struct scatterlist *ScatterList = Command->V2.ScatterList;
-
- DAC960_V2_ClearCommand(Command);
-
- CommandMailbox->SCSI_10.CommandOpcode = DAC960_V2_SCSI_10;
- CommandMailbox->SCSI_10.CommandControlBits.DataTransferControllerToHost =
- (Command->DmaDirection == PCI_DMA_FROMDEVICE);
- CommandMailbox->SCSI_10.DataTransferSize =
- Command->BlockCount << DAC960_BlockSizeBits;
- CommandMailbox->SCSI_10.RequestSenseBusAddress = Command->V2.RequestSenseDMA;
- CommandMailbox->SCSI_10.PhysicalDevice =
- Controller->V2.LogicalDriveToVirtualDevice[Command->LogicalDriveNumber];
- CommandMailbox->SCSI_10.RequestSenseSize = sizeof(DAC960_SCSI_RequestSense_T);
- CommandMailbox->SCSI_10.CDBLength = 10;
- CommandMailbox->SCSI_10.SCSI_CDB[0] =
- (Command->DmaDirection == PCI_DMA_FROMDEVICE ? 0x28 : 0x2A);
- CommandMailbox->SCSI_10.SCSI_CDB[2] = Command->BlockNumber >> 24;
- CommandMailbox->SCSI_10.SCSI_CDB[3] = Command->BlockNumber >> 16;
- CommandMailbox->SCSI_10.SCSI_CDB[4] = Command->BlockNumber >> 8;
- CommandMailbox->SCSI_10.SCSI_CDB[5] = Command->BlockNumber;
- CommandMailbox->SCSI_10.SCSI_CDB[7] = Command->BlockCount >> 8;
- CommandMailbox->SCSI_10.SCSI_CDB[8] = Command->BlockCount;
-
- if (Command->SegmentCount == 1)
- {
- CommandMailbox->SCSI_10.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer =
- (DAC960_BusAddress64_T)sg_dma_address(ScatterList);
- CommandMailbox->SCSI_10.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->SCSI_10.DataTransferSize;
- }
- else
- {
- DAC960_V2_ScatterGatherSegment_T *ScatterGatherList;
- int i;
-
- if (Command->SegmentCount > 2)
- {
- ScatterGatherList = Command->V2.ScatterGatherList;
- CommandMailbox->SCSI_10.CommandControlBits
- .AdditionalScatterGatherListMemory = true;
- CommandMailbox->SCSI_10.DataTransferMemoryAddress
- .ExtendedScatterGather.ScatterGatherList0Length = Command->SegmentCount;
- CommandMailbox->SCSI_10.DataTransferMemoryAddress
- .ExtendedScatterGather.ScatterGatherList0Address =
- Command->V2.ScatterGatherListDMA;
- }
- else
- ScatterGatherList = CommandMailbox->SCSI_10.DataTransferMemoryAddress
- .ScatterGatherSegments;
-
- for (i = 0; i < Command->SegmentCount; i++, ScatterList++, ScatterGatherList++) {
- ScatterGatherList->SegmentDataPointer =
- (DAC960_BusAddress64_T)sg_dma_address(ScatterList);
- ScatterGatherList->SegmentByteCount =
- (DAC960_ByteCount64_T)sg_dma_len(ScatterList);
- }
- }
- DAC960_QueueCommand(Command);
-}
-
-
-static int DAC960_process_queue(DAC960_Controller_T *Controller, struct request_queue *req_q)
-{
- struct request *Request;
- DAC960_Command_T *Command;
-
- while(1) {
- Request = blk_peek_request(req_q);
- if (!Request)
- return 1;
-
- Command = DAC960_AllocateCommand(Controller);
- if (Command == NULL)
- return 0;
-
- if (rq_data_dir(Request) == READ) {
- Command->DmaDirection = PCI_DMA_FROMDEVICE;
- Command->CommandType = DAC960_ReadCommand;
- } else {
- Command->DmaDirection = PCI_DMA_TODEVICE;
- Command->CommandType = DAC960_WriteCommand;
- }
- Command->Completion = Request->end_io_data;
- Command->LogicalDriveNumber = (long)Request->rq_disk->private_data;
- Command->BlockNumber = blk_rq_pos(Request);
- Command->BlockCount = blk_rq_sectors(Request);
- Command->Request = Request;
- blk_start_request(Request);
- Command->SegmentCount = blk_rq_map_sg(req_q,
- Command->Request, Command->cmd_sglist);
- /* pci_map_sg MAY change the value of SegCount */
- Command->SegmentCount = pci_map_sg(Controller->PCIDevice, Command->cmd_sglist,
- Command->SegmentCount, Command->DmaDirection);
-
- DAC960_QueueReadWriteCommand(Command);
- }
-}
-
-/*
- DAC960_ProcessRequest attempts to remove one I/O Request from Controller's
- I/O Request Queue and queues it to the Controller. WaitForCommand is true if
- this function should wait for a Command to become available if necessary.
- This function returns true if an I/O Request was queued and false otherwise.
-*/
-static void DAC960_ProcessRequest(DAC960_Controller_T *controller)
-{
- int i;
-
- if (!controller->ControllerInitialized)
- return;
-
- /* Do this better later! */
- for (i = controller->req_q_index; i < DAC960_MaxLogicalDrives; i++) {
- struct request_queue *req_q = controller->RequestQueue[i];
-
- if (req_q == NULL)
- continue;
-
- if (!DAC960_process_queue(controller, req_q)) {
- controller->req_q_index = i;
- return;
- }
- }
-
- if (controller->req_q_index == 0)
- return;
-
- for (i = 0; i < controller->req_q_index; i++) {
- struct request_queue *req_q = controller->RequestQueue[i];
-
- if (req_q == NULL)
- continue;
-
- if (!DAC960_process_queue(controller, req_q)) {
- controller->req_q_index = i;
- return;
- }
- }
-}
-
-
-/*
- DAC960_queue_partial_rw extracts one bio from the request already
- associated with argument command, and construct a new command block to retry I/O
- only on that bio. Queue that command to the controller.
-
- This function re-uses a previously-allocated Command,
- there is no failure mode from trying to allocate a command.
-*/
-
-static void DAC960_queue_partial_rw(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- struct request *Request = Command->Request;
- struct request_queue *req_q = Controller->RequestQueue[Command->LogicalDriveNumber];
-
- if (Command->DmaDirection == PCI_DMA_FROMDEVICE)
- Command->CommandType = DAC960_ReadRetryCommand;
- else
- Command->CommandType = DAC960_WriteRetryCommand;
-
- /*
- * We could be more efficient with these mapping requests
- * and map only the portions that we need. But since this
- * code should almost never be called, just go with a
- * simple coding.
- */
- (void)blk_rq_map_sg(req_q, Command->Request, Command->cmd_sglist);
-
- (void)pci_map_sg(Controller->PCIDevice, Command->cmd_sglist, 1, Command->DmaDirection);
- /*
- * Resubmitting the request sector at a time is really tedious.
- * But, this should almost never happen. So, we're willing to pay
- * this price so that in the end, as much of the transfer is completed
- * successfully as possible.
- */
- Command->SegmentCount = 1;
- Command->BlockNumber = blk_rq_pos(Request);
- Command->BlockCount = 1;
- DAC960_QueueReadWriteCommand(Command);
- return;
-}
-
-/*
- DAC960_RequestFunction is the I/O Request Function for DAC960 Controllers.
-*/
-
-static void DAC960_RequestFunction(struct request_queue *RequestQueue)
-{
- DAC960_ProcessRequest(RequestQueue->queuedata);
-}
-
-/*
- DAC960_ProcessCompletedBuffer performs completion processing for an
- individual Buffer.
-*/
-
-static inline bool DAC960_ProcessCompletedRequest(DAC960_Command_T *Command,
- bool SuccessfulIO)
-{
- struct request *Request = Command->Request;
- blk_status_t Error = SuccessfulIO ? BLK_STS_OK : BLK_STS_IOERR;
-
- pci_unmap_sg(Command->Controller->PCIDevice, Command->cmd_sglist,
- Command->SegmentCount, Command->DmaDirection);
-
- if (!__blk_end_request(Request, Error, Command->BlockCount << 9)) {
- if (Command->Completion) {
- complete(Command->Completion);
- Command->Completion = NULL;
- }
- return true;
- }
- return false;
-}
-
-/*
- DAC960_V1_ReadWriteError prints an appropriate error message for Command
- when an error occurs on a Read or Write operation.
-*/
-
-static void DAC960_V1_ReadWriteError(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- unsigned char *CommandName = "UNKNOWN";
- switch (Command->CommandType)
- {
- case DAC960_ReadCommand:
- case DAC960_ReadRetryCommand:
- CommandName = "READ";
- break;
- case DAC960_WriteCommand:
- case DAC960_WriteRetryCommand:
- CommandName = "WRITE";
- break;
- case DAC960_MonitoringCommand:
- case DAC960_ImmediateCommand:
- case DAC960_QueuedCommand:
- break;
- }
- switch (Command->V1.CommandStatus)
- {
- case DAC960_V1_IrrecoverableDataError:
- DAC960_Error("Irrecoverable Data Error on %s:\n",
- Controller, CommandName);
- break;
- case DAC960_V1_LogicalDriveNonexistentOrOffline:
- DAC960_Error("Logical Drive Nonexistent or Offline on %s:\n",
- Controller, CommandName);
- break;
- case DAC960_V1_AccessBeyondEndOfLogicalDrive:
- DAC960_Error("Attempt to Access Beyond End of Logical Drive "
- "on %s:\n", Controller, CommandName);
- break;
- case DAC960_V1_BadDataEncountered:
- DAC960_Error("Bad Data Encountered on %s:\n", Controller, CommandName);
- break;
- default:
- DAC960_Error("Unexpected Error Status %04X on %s:\n",
- Controller, Command->V1.CommandStatus, CommandName);
- break;
- }
- DAC960_Error(" /dev/rd/c%dd%d: absolute blocks %u..%u\n",
- Controller, Controller->ControllerNumber,
- Command->LogicalDriveNumber, Command->BlockNumber,
- Command->BlockNumber + Command->BlockCount - 1);
-}
-
-
-/*
- DAC960_V1_ProcessCompletedCommand performs completion processing for Command
- for DAC960 V1 Firmware Controllers.
-*/
-
-static void DAC960_V1_ProcessCompletedCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- DAC960_CommandType_T CommandType = Command->CommandType;
- DAC960_V1_CommandOpcode_T CommandOpcode =
- Command->V1.CommandMailbox.Common.CommandOpcode;
- DAC960_V1_CommandStatus_T CommandStatus = Command->V1.CommandStatus;
-
- if (CommandType == DAC960_ReadCommand ||
- CommandType == DAC960_WriteCommand)
- {
-
-#ifdef FORCE_RETRY_DEBUG
- CommandStatus = DAC960_V1_IrrecoverableDataError;
-#endif
-
- if (CommandStatus == DAC960_V1_NormalCompletion) {
-
- if (!DAC960_ProcessCompletedRequest(Command, true))
- BUG();
-
- } else if (CommandStatus == DAC960_V1_IrrecoverableDataError ||
- CommandStatus == DAC960_V1_BadDataEncountered)
- {
- /*
- * break the command down into pieces and resubmit each
- * piece, hoping that some of them will succeed.
- */
- DAC960_queue_partial_rw(Command);
- return;
- }
- else
- {
- if (CommandStatus != DAC960_V1_LogicalDriveNonexistentOrOffline)
- DAC960_V1_ReadWriteError(Command);
-
- if (!DAC960_ProcessCompletedRequest(Command, false))
- BUG();
- }
- }
- else if (CommandType == DAC960_ReadRetryCommand ||
- CommandType == DAC960_WriteRetryCommand)
- {
- bool normal_completion;
-#ifdef FORCE_RETRY_FAILURE_DEBUG
- static int retry_count = 1;
-#endif
- /*
- Perform completion processing for the portion that was
- retried, and submit the next portion, if any.
- */
- normal_completion = true;
- if (CommandStatus != DAC960_V1_NormalCompletion) {
- normal_completion = false;
- if (CommandStatus != DAC960_V1_LogicalDriveNonexistentOrOffline)
- DAC960_V1_ReadWriteError(Command);
- }
-
-#ifdef FORCE_RETRY_FAILURE_DEBUG
- if (!(++retry_count % 10000)) {
- printk("V1 error retry failure test\n");
- normal_completion = false;
- DAC960_V1_ReadWriteError(Command);
- }
-#endif
-
- if (!DAC960_ProcessCompletedRequest(Command, normal_completion)) {
- DAC960_queue_partial_rw(Command);
- return;
- }
- }
-
- else if (CommandType == DAC960_MonitoringCommand)
- {
- if (Controller->ShutdownMonitoringTimer)
- return;
- if (CommandOpcode == DAC960_V1_Enquiry)
- {
- DAC960_V1_Enquiry_T *OldEnquiry = &Controller->V1.Enquiry;
- DAC960_V1_Enquiry_T *NewEnquiry = Controller->V1.NewEnquiry;
- unsigned int OldCriticalLogicalDriveCount =
- OldEnquiry->CriticalLogicalDriveCount;
- unsigned int NewCriticalLogicalDriveCount =
- NewEnquiry->CriticalLogicalDriveCount;
- if (NewEnquiry->NumberOfLogicalDrives > Controller->LogicalDriveCount)
- {
- int LogicalDriveNumber = Controller->LogicalDriveCount - 1;
- while (++LogicalDriveNumber < NewEnquiry->NumberOfLogicalDrives)
- DAC960_Critical("Logical Drive %d (/dev/rd/c%dd%d) "
- "Now Exists\n", Controller,
- LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber);
- Controller->LogicalDriveCount = NewEnquiry->NumberOfLogicalDrives;
- DAC960_ComputeGenericDiskInfo(Controller);
- }
- if (NewEnquiry->NumberOfLogicalDrives < Controller->LogicalDriveCount)
- {
- int LogicalDriveNumber = NewEnquiry->NumberOfLogicalDrives - 1;
- while (++LogicalDriveNumber < Controller->LogicalDriveCount)
- DAC960_Critical("Logical Drive %d (/dev/rd/c%dd%d) "
- "No Longer Exists\n", Controller,
- LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber);
- Controller->LogicalDriveCount = NewEnquiry->NumberOfLogicalDrives;
- DAC960_ComputeGenericDiskInfo(Controller);
- }
- if (NewEnquiry->StatusFlags.DeferredWriteError !=
- OldEnquiry->StatusFlags.DeferredWriteError)
- DAC960_Critical("Deferred Write Error Flag is now %s\n", Controller,
- (NewEnquiry->StatusFlags.DeferredWriteError
- ? "TRUE" : "FALSE"));
- if ((NewCriticalLogicalDriveCount > 0 ||
- NewCriticalLogicalDriveCount != OldCriticalLogicalDriveCount) ||
- (NewEnquiry->OfflineLogicalDriveCount > 0 ||
- NewEnquiry->OfflineLogicalDriveCount !=
- OldEnquiry->OfflineLogicalDriveCount) ||
- (NewEnquiry->DeadDriveCount > 0 ||
- NewEnquiry->DeadDriveCount !=
- OldEnquiry->DeadDriveCount) ||
- (NewEnquiry->EventLogSequenceNumber !=
- OldEnquiry->EventLogSequenceNumber) ||
- Controller->MonitoringTimerCount == 0 ||
- time_after_eq(jiffies, Controller->SecondaryMonitoringTime
- + DAC960_SecondaryMonitoringInterval))
- {
- Controller->V1.NeedLogicalDriveInformation = true;
- Controller->V1.NewEventLogSequenceNumber =
- NewEnquiry->EventLogSequenceNumber;
- Controller->V1.NeedErrorTableInformation = true;
- Controller->V1.NeedDeviceStateInformation = true;
- Controller->V1.StartDeviceStateScan = true;
- Controller->V1.NeedBackgroundInitializationStatus =
- Controller->V1.BackgroundInitializationStatusSupported;
- Controller->SecondaryMonitoringTime = jiffies;
- }
- if (NewEnquiry->RebuildFlag == DAC960_V1_StandbyRebuildInProgress ||
- NewEnquiry->RebuildFlag
- == DAC960_V1_BackgroundRebuildInProgress ||
- OldEnquiry->RebuildFlag == DAC960_V1_StandbyRebuildInProgress ||
- OldEnquiry->RebuildFlag == DAC960_V1_BackgroundRebuildInProgress)
- {
- Controller->V1.NeedRebuildProgress = true;
- Controller->V1.RebuildProgressFirst =
- (NewEnquiry->CriticalLogicalDriveCount <
- OldEnquiry->CriticalLogicalDriveCount);
- }
- if (OldEnquiry->RebuildFlag == DAC960_V1_BackgroundCheckInProgress)
- switch (NewEnquiry->RebuildFlag)
- {
- case DAC960_V1_NoStandbyRebuildOrCheckInProgress:
- DAC960_Progress("Consistency Check Completed Successfully\n",
- Controller);
- break;
- case DAC960_V1_StandbyRebuildInProgress:
- case DAC960_V1_BackgroundRebuildInProgress:
- break;
- case DAC960_V1_BackgroundCheckInProgress:
- Controller->V1.NeedConsistencyCheckProgress = true;
- break;
- case DAC960_V1_StandbyRebuildCompletedWithError:
- DAC960_Progress("Consistency Check Completed with Error\n",
- Controller);
- break;
- case DAC960_V1_BackgroundRebuildOrCheckFailed_DriveFailed:
- DAC960_Progress("Consistency Check Failed - "
- "Physical Device Failed\n", Controller);
- break;
- case DAC960_V1_BackgroundRebuildOrCheckFailed_LogicalDriveFailed:
- DAC960_Progress("Consistency Check Failed - "
- "Logical Drive Failed\n", Controller);
- break;
- case DAC960_V1_BackgroundRebuildOrCheckFailed_OtherCauses:
- DAC960_Progress("Consistency Check Failed - Other Causes\n",
- Controller);
- break;
- case DAC960_V1_BackgroundRebuildOrCheckSuccessfullyTerminated:
- DAC960_Progress("Consistency Check Successfully Terminated\n",
- Controller);
- break;
- }
- else if (NewEnquiry->RebuildFlag
- == DAC960_V1_BackgroundCheckInProgress)
- Controller->V1.NeedConsistencyCheckProgress = true;
- Controller->MonitoringAlertMode =
- (NewEnquiry->CriticalLogicalDriveCount > 0 ||
- NewEnquiry->OfflineLogicalDriveCount > 0 ||
- NewEnquiry->DeadDriveCount > 0);
- if (NewEnquiry->RebuildFlag > DAC960_V1_BackgroundCheckInProgress)
- {
- Controller->V1.PendingRebuildFlag = NewEnquiry->RebuildFlag;
- Controller->V1.RebuildFlagPending = true;
- }
- memcpy(&Controller->V1.Enquiry, &Controller->V1.NewEnquiry,
- sizeof(DAC960_V1_Enquiry_T));
- }
- else if (CommandOpcode == DAC960_V1_PerformEventLogOperation)
- {
- static char
- *DAC960_EventMessages[] =
- { "killed because write recovery failed",
- "killed because of SCSI bus reset failure",
- "killed because of double check condition",
- "killed because it was removed",
- "killed because of gross error on SCSI chip",
- "killed because of bad tag returned from drive",
- "killed because of timeout on SCSI command",
- "killed because of reset SCSI command issued from system",
- "killed because busy or parity error count exceeded limit",
- "killed because of 'kill drive' command from system",
- "killed because of selection timeout",
- "killed due to SCSI phase sequence error",
- "killed due to unknown status" };
- DAC960_V1_EventLogEntry_T *EventLogEntry =
- Controller->V1.EventLogEntry;
- if (EventLogEntry->SequenceNumber ==
- Controller->V1.OldEventLogSequenceNumber)
- {
- unsigned char SenseKey = EventLogEntry->SenseKey;
- unsigned char AdditionalSenseCode =
- EventLogEntry->AdditionalSenseCode;
- unsigned char AdditionalSenseCodeQualifier =
- EventLogEntry->AdditionalSenseCodeQualifier;
- if (SenseKey == DAC960_SenseKey_VendorSpecific &&
- AdditionalSenseCode == 0x80 &&
- AdditionalSenseCodeQualifier <
- ARRAY_SIZE(DAC960_EventMessages))
- DAC960_Critical("Physical Device %d:%d %s\n", Controller,
- EventLogEntry->Channel,
- EventLogEntry->TargetID,
- DAC960_EventMessages[
- AdditionalSenseCodeQualifier]);
- else if (SenseKey == DAC960_SenseKey_UnitAttention &&
- AdditionalSenseCode == 0x29)
- {
- if (Controller->MonitoringTimerCount > 0)
- Controller->V1.DeviceResetCount[EventLogEntry->Channel]
- [EventLogEntry->TargetID]++;
- }
- else if (!(SenseKey == DAC960_SenseKey_NoSense ||
- (SenseKey == DAC960_SenseKey_NotReady &&
- AdditionalSenseCode == 0x04 &&
- (AdditionalSenseCodeQualifier == 0x01 ||
- AdditionalSenseCodeQualifier == 0x02))))
- {
- DAC960_Critical("Physical Device %d:%d Error Log: "
- "Sense Key = %X, ASC = %02X, ASCQ = %02X\n",
- Controller,
- EventLogEntry->Channel,
- EventLogEntry->TargetID,
- SenseKey,
- AdditionalSenseCode,
- AdditionalSenseCodeQualifier);
- DAC960_Critical("Physical Device %d:%d Error Log: "
- "Information = %02X%02X%02X%02X "
- "%02X%02X%02X%02X\n",
- Controller,
- EventLogEntry->Channel,
- EventLogEntry->TargetID,
- EventLogEntry->Information[0],
- EventLogEntry->Information[1],
- EventLogEntry->Information[2],
- EventLogEntry->Information[3],
- EventLogEntry->CommandSpecificInformation[0],
- EventLogEntry->CommandSpecificInformation[1],
- EventLogEntry->CommandSpecificInformation[2],
- EventLogEntry->CommandSpecificInformation[3]);
- }
- }
- Controller->V1.OldEventLogSequenceNumber++;
- }
- else if (CommandOpcode == DAC960_V1_GetErrorTable)
- {
- DAC960_V1_ErrorTable_T *OldErrorTable = &Controller->V1.ErrorTable;
- DAC960_V1_ErrorTable_T *NewErrorTable = Controller->V1.NewErrorTable;
- int Channel, TargetID;
- for (Channel = 0; Channel < Controller->Channels; Channel++)
- for (TargetID = 0; TargetID < Controller->Targets; TargetID++)
- {
- DAC960_V1_ErrorTableEntry_T *NewErrorEntry =
- &NewErrorTable->ErrorTableEntries[Channel][TargetID];
- DAC960_V1_ErrorTableEntry_T *OldErrorEntry =
- &OldErrorTable->ErrorTableEntries[Channel][TargetID];
- if ((NewErrorEntry->ParityErrorCount !=
- OldErrorEntry->ParityErrorCount) ||
- (NewErrorEntry->SoftErrorCount !=
- OldErrorEntry->SoftErrorCount) ||
- (NewErrorEntry->HardErrorCount !=
- OldErrorEntry->HardErrorCount) ||
- (NewErrorEntry->MiscErrorCount !=
- OldErrorEntry->MiscErrorCount))
- DAC960_Critical("Physical Device %d:%d Errors: "
- "Parity = %d, Soft = %d, "
- "Hard = %d, Misc = %d\n",
- Controller, Channel, TargetID,
- NewErrorEntry->ParityErrorCount,
- NewErrorEntry->SoftErrorCount,
- NewErrorEntry->HardErrorCount,
- NewErrorEntry->MiscErrorCount);
- }
- memcpy(&Controller->V1.ErrorTable, Controller->V1.NewErrorTable,
- sizeof(DAC960_V1_ErrorTable_T));
- }
- else if (CommandOpcode == DAC960_V1_GetDeviceState)
- {
- DAC960_V1_DeviceState_T *OldDeviceState =
- &Controller->V1.DeviceState[Controller->V1.DeviceStateChannel]
- [Controller->V1.DeviceStateTargetID];
- DAC960_V1_DeviceState_T *NewDeviceState =
- Controller->V1.NewDeviceState;
- if (NewDeviceState->DeviceState != OldDeviceState->DeviceState)
- DAC960_Critical("Physical Device %d:%d is now %s\n", Controller,
- Controller->V1.DeviceStateChannel,
- Controller->V1.DeviceStateTargetID,
- (NewDeviceState->DeviceState
- == DAC960_V1_Device_Dead
- ? "DEAD"
- : NewDeviceState->DeviceState
- == DAC960_V1_Device_WriteOnly
- ? "WRITE-ONLY"
- : NewDeviceState->DeviceState
- == DAC960_V1_Device_Online
- ? "ONLINE" : "STANDBY"));
- if (OldDeviceState->DeviceState == DAC960_V1_Device_Dead &&
- NewDeviceState->DeviceState != DAC960_V1_Device_Dead)
- {
- Controller->V1.NeedDeviceInquiryInformation = true;
- Controller->V1.NeedDeviceSerialNumberInformation = true;
- Controller->V1.DeviceResetCount
- [Controller->V1.DeviceStateChannel]
- [Controller->V1.DeviceStateTargetID] = 0;
- }
- memcpy(OldDeviceState, NewDeviceState,
- sizeof(DAC960_V1_DeviceState_T));
- }
- else if (CommandOpcode == DAC960_V1_GetLogicalDriveInformation)
- {
- int LogicalDriveNumber;
- for (LogicalDriveNumber = 0;
- LogicalDriveNumber < Controller->LogicalDriveCount;
- LogicalDriveNumber++)
- {
- DAC960_V1_LogicalDriveInformation_T *OldLogicalDriveInformation =
- &Controller->V1.LogicalDriveInformation[LogicalDriveNumber];
- DAC960_V1_LogicalDriveInformation_T *NewLogicalDriveInformation =
- &(*Controller->V1.NewLogicalDriveInformation)[LogicalDriveNumber];
- if (NewLogicalDriveInformation->LogicalDriveState !=
- OldLogicalDriveInformation->LogicalDriveState)
- DAC960_Critical("Logical Drive %d (/dev/rd/c%dd%d) "
- "is now %s\n", Controller,
- LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber,
- (NewLogicalDriveInformation->LogicalDriveState
- == DAC960_V1_LogicalDrive_Online
- ? "ONLINE"
- : NewLogicalDriveInformation->LogicalDriveState
- == DAC960_V1_LogicalDrive_Critical
- ? "CRITICAL" : "OFFLINE"));
- if (NewLogicalDriveInformation->WriteBack !=
- OldLogicalDriveInformation->WriteBack)
- DAC960_Critical("Logical Drive %d (/dev/rd/c%dd%d) "
- "is now %s\n", Controller,
- LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber,
- (NewLogicalDriveInformation->WriteBack
- ? "WRITE BACK" : "WRITE THRU"));
- }
- memcpy(&Controller->V1.LogicalDriveInformation,
- Controller->V1.NewLogicalDriveInformation,
- sizeof(DAC960_V1_LogicalDriveInformationArray_T));
- }
- else if (CommandOpcode == DAC960_V1_GetRebuildProgress)
- {
- unsigned int LogicalDriveNumber =
- Controller->V1.RebuildProgress->LogicalDriveNumber;
- unsigned int LogicalDriveSize =
- Controller->V1.RebuildProgress->LogicalDriveSize;
- unsigned int BlocksCompleted =
- LogicalDriveSize - Controller->V1.RebuildProgress->RemainingBlocks;
- if (CommandStatus == DAC960_V1_NoRebuildOrCheckInProgress &&
- Controller->V1.LastRebuildStatus == DAC960_V1_NormalCompletion)
- CommandStatus = DAC960_V1_RebuildSuccessful;
- switch (CommandStatus)
- {
- case DAC960_V1_NormalCompletion:
- Controller->EphemeralProgressMessage = true;
- DAC960_Progress("Rebuild in Progress: "
- "Logical Drive %d (/dev/rd/c%dd%d) "
- "%d%% completed\n",
- Controller, LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber,
- (100 * (BlocksCompleted >> 7))
- / (LogicalDriveSize >> 7));
- Controller->EphemeralProgressMessage = false;
- break;
- case DAC960_V1_RebuildFailed_LogicalDriveFailure:
- DAC960_Progress("Rebuild Failed due to "
- "Logical Drive Failure\n", Controller);
- break;
- case DAC960_V1_RebuildFailed_BadBlocksOnOther:
- DAC960_Progress("Rebuild Failed due to "
- "Bad Blocks on Other Drives\n", Controller);
- break;
- case DAC960_V1_RebuildFailed_NewDriveFailed:
- DAC960_Progress("Rebuild Failed due to "
- "Failure of Drive Being Rebuilt\n", Controller);
- break;
- case DAC960_V1_NoRebuildOrCheckInProgress:
- break;
- case DAC960_V1_RebuildSuccessful:
- DAC960_Progress("Rebuild Completed Successfully\n", Controller);
- break;
- case DAC960_V1_RebuildSuccessfullyTerminated:
- DAC960_Progress("Rebuild Successfully Terminated\n", Controller);
- break;
- }
- Controller->V1.LastRebuildStatus = CommandStatus;
- if (CommandType != DAC960_MonitoringCommand &&
- Controller->V1.RebuildStatusPending)
- {
- Command->V1.CommandStatus = Controller->V1.PendingRebuildStatus;
- Controller->V1.RebuildStatusPending = false;
- }
- else if (CommandType == DAC960_MonitoringCommand &&
- CommandStatus != DAC960_V1_NormalCompletion &&
- CommandStatus != DAC960_V1_NoRebuildOrCheckInProgress)
- {
- Controller->V1.PendingRebuildStatus = CommandStatus;
- Controller->V1.RebuildStatusPending = true;
- }
- }
- else if (CommandOpcode == DAC960_V1_RebuildStat)
- {
- unsigned int LogicalDriveNumber =
- Controller->V1.RebuildProgress->LogicalDriveNumber;
- unsigned int LogicalDriveSize =
- Controller->V1.RebuildProgress->LogicalDriveSize;
- unsigned int BlocksCompleted =
- LogicalDriveSize - Controller->V1.RebuildProgress->RemainingBlocks;
- if (CommandStatus == DAC960_V1_NormalCompletion)
- {
- Controller->EphemeralProgressMessage = true;
- DAC960_Progress("Consistency Check in Progress: "
- "Logical Drive %d (/dev/rd/c%dd%d) "
- "%d%% completed\n",
- Controller, LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber,
- (100 * (BlocksCompleted >> 7))
- / (LogicalDriveSize >> 7));
- Controller->EphemeralProgressMessage = false;
- }
- }
- else if (CommandOpcode == DAC960_V1_BackgroundInitializationControl)
- {
- unsigned int LogicalDriveNumber =
- Controller->V1.BackgroundInitializationStatus->LogicalDriveNumber;
- unsigned int LogicalDriveSize =
- Controller->V1.BackgroundInitializationStatus->LogicalDriveSize;
- unsigned int BlocksCompleted =
- Controller->V1.BackgroundInitializationStatus->BlocksCompleted;
- switch (CommandStatus)
- {
- case DAC960_V1_NormalCompletion:
- switch (Controller->V1.BackgroundInitializationStatus->Status)
- {
- case DAC960_V1_BackgroundInitializationInvalid:
- break;
- case DAC960_V1_BackgroundInitializationStarted:
- DAC960_Progress("Background Initialization Started\n",
- Controller);
- break;
- case DAC960_V1_BackgroundInitializationInProgress:
- if (BlocksCompleted ==
- Controller->V1.LastBackgroundInitializationStatus.
- BlocksCompleted &&
- LogicalDriveNumber ==
- Controller->V1.LastBackgroundInitializationStatus.
- LogicalDriveNumber)
- break;
- Controller->EphemeralProgressMessage = true;
- DAC960_Progress("Background Initialization in Progress: "
- "Logical Drive %d (/dev/rd/c%dd%d) "
- "%d%% completed\n",
- Controller, LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber,
- (100 * (BlocksCompleted >> 7))
- / (LogicalDriveSize >> 7));
- Controller->EphemeralProgressMessage = false;
- break;
- case DAC960_V1_BackgroundInitializationSuspended:
- DAC960_Progress("Background Initialization Suspended\n",
- Controller);
- break;
- case DAC960_V1_BackgroundInitializationCancelled:
- DAC960_Progress("Background Initialization Cancelled\n",
- Controller);
- break;
- }
- memcpy(&Controller->V1.LastBackgroundInitializationStatus,
- Controller->V1.BackgroundInitializationStatus,
- sizeof(DAC960_V1_BackgroundInitializationStatus_T));
- break;
- case DAC960_V1_BackgroundInitSuccessful:
- if (Controller->V1.BackgroundInitializationStatus->Status ==
- DAC960_V1_BackgroundInitializationInProgress)
- DAC960_Progress("Background Initialization "
- "Completed Successfully\n", Controller);
- Controller->V1.BackgroundInitializationStatus->Status =
- DAC960_V1_BackgroundInitializationInvalid;
- break;
- case DAC960_V1_BackgroundInitAborted:
- if (Controller->V1.BackgroundInitializationStatus->Status ==
- DAC960_V1_BackgroundInitializationInProgress)
- DAC960_Progress("Background Initialization Aborted\n",
- Controller);
- Controller->V1.BackgroundInitializationStatus->Status =
- DAC960_V1_BackgroundInitializationInvalid;
- break;
- case DAC960_V1_NoBackgroundInitInProgress:
- break;
- }
- }
- else if (CommandOpcode == DAC960_V1_DCDB)
- {
- /*
- This is a bit ugly.
-
- The InquiryStandardData and
- the InquiryUntitSerialNumber information
- retrieval operations BOTH use the DAC960_V1_DCDB
- commands. the test above can't distinguish between
- these two cases.
-
- Instead, we rely on the order of code later in this
- function to ensure that DeviceInquiryInformation commands
- are submitted before DeviceSerialNumber commands.
- */
- if (Controller->V1.NeedDeviceInquiryInformation)
- {
- DAC960_SCSI_Inquiry_T *InquiryStandardData =
- &Controller->V1.InquiryStandardData
- [Controller->V1.DeviceStateChannel]
- [Controller->V1.DeviceStateTargetID];
- if (CommandStatus != DAC960_V1_NormalCompletion)
- {
- memset(InquiryStandardData, 0,
- sizeof(DAC960_SCSI_Inquiry_T));
- InquiryStandardData->PeripheralDeviceType = 0x1F;
- }
- else
- memcpy(InquiryStandardData,
- Controller->V1.NewInquiryStandardData,
- sizeof(DAC960_SCSI_Inquiry_T));
- Controller->V1.NeedDeviceInquiryInformation = false;
- }
- else if (Controller->V1.NeedDeviceSerialNumberInformation)
- {
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *InquiryUnitSerialNumber =
- &Controller->V1.InquiryUnitSerialNumber
- [Controller->V1.DeviceStateChannel]
- [Controller->V1.DeviceStateTargetID];
- if (CommandStatus != DAC960_V1_NormalCompletion)
- {
- memset(InquiryUnitSerialNumber, 0,
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T));
- InquiryUnitSerialNumber->PeripheralDeviceType = 0x1F;
- }
- else
- memcpy(InquiryUnitSerialNumber,
- Controller->V1.NewInquiryUnitSerialNumber,
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T));
- Controller->V1.NeedDeviceSerialNumberInformation = false;
- }
- }
- /*
- Begin submitting new monitoring commands.
- */
- if (Controller->V1.NewEventLogSequenceNumber
- - Controller->V1.OldEventLogSequenceNumber > 0)
- {
- Command->V1.CommandMailbox.Type3E.CommandOpcode =
- DAC960_V1_PerformEventLogOperation;
- Command->V1.CommandMailbox.Type3E.OperationType =
- DAC960_V1_GetEventLogEntry;
- Command->V1.CommandMailbox.Type3E.OperationQualifier = 1;
- Command->V1.CommandMailbox.Type3E.SequenceNumber =
- Controller->V1.OldEventLogSequenceNumber;
- Command->V1.CommandMailbox.Type3E.BusAddress =
- Controller->V1.EventLogEntryDMA;
- DAC960_QueueCommand(Command);
- return;
- }
- if (Controller->V1.NeedErrorTableInformation)
- {
- Controller->V1.NeedErrorTableInformation = false;
- Command->V1.CommandMailbox.Type3.CommandOpcode =
- DAC960_V1_GetErrorTable;
- Command->V1.CommandMailbox.Type3.BusAddress =
- Controller->V1.NewErrorTableDMA;
- DAC960_QueueCommand(Command);
- return;
- }
- if (Controller->V1.NeedRebuildProgress &&
- Controller->V1.RebuildProgressFirst)
- {
- Controller->V1.NeedRebuildProgress = false;
- Command->V1.CommandMailbox.Type3.CommandOpcode =
- DAC960_V1_GetRebuildProgress;
- Command->V1.CommandMailbox.Type3.BusAddress =
- Controller->V1.RebuildProgressDMA;
- DAC960_QueueCommand(Command);
- return;
- }
- if (Controller->V1.NeedDeviceStateInformation)
- {
- if (Controller->V1.NeedDeviceInquiryInformation)
- {
- DAC960_V1_DCDB_T *DCDB = Controller->V1.MonitoringDCDB;
- dma_addr_t DCDB_DMA = Controller->V1.MonitoringDCDB_DMA;
-
- dma_addr_t NewInquiryStandardDataDMA =
- Controller->V1.NewInquiryStandardDataDMA;
-
- Command->V1.CommandMailbox.Type3.CommandOpcode = DAC960_V1_DCDB;
- Command->V1.CommandMailbox.Type3.BusAddress = DCDB_DMA;
- DCDB->Channel = Controller->V1.DeviceStateChannel;
- DCDB->TargetID = Controller->V1.DeviceStateTargetID;
- DCDB->Direction = DAC960_V1_DCDB_DataTransferDeviceToSystem;
- DCDB->EarlyStatus = false;
- DCDB->Timeout = DAC960_V1_DCDB_Timeout_10_seconds;
- DCDB->NoAutomaticRequestSense = false;
- DCDB->DisconnectPermitted = true;
- DCDB->TransferLength = sizeof(DAC960_SCSI_Inquiry_T);
- DCDB->BusAddress = NewInquiryStandardDataDMA;
- DCDB->CDBLength = 6;
- DCDB->TransferLengthHigh4 = 0;
- DCDB->SenseLength = sizeof(DCDB->SenseData);
- DCDB->CDB[0] = 0x12; /* INQUIRY */
- DCDB->CDB[1] = 0; /* EVPD = 0 */
- DCDB->CDB[2] = 0; /* Page Code */
- DCDB->CDB[3] = 0; /* Reserved */
- DCDB->CDB[4] = sizeof(DAC960_SCSI_Inquiry_T);
- DCDB->CDB[5] = 0; /* Control */
- DAC960_QueueCommand(Command);
- return;
- }
- if (Controller->V1.NeedDeviceSerialNumberInformation)
- {
- DAC960_V1_DCDB_T *DCDB = Controller->V1.MonitoringDCDB;
- dma_addr_t DCDB_DMA = Controller->V1.MonitoringDCDB_DMA;
- dma_addr_t NewInquiryUnitSerialNumberDMA =
- Controller->V1.NewInquiryUnitSerialNumberDMA;
-
- Command->V1.CommandMailbox.Type3.CommandOpcode = DAC960_V1_DCDB;
- Command->V1.CommandMailbox.Type3.BusAddress = DCDB_DMA;
- DCDB->Channel = Controller->V1.DeviceStateChannel;
- DCDB->TargetID = Controller->V1.DeviceStateTargetID;
- DCDB->Direction = DAC960_V1_DCDB_DataTransferDeviceToSystem;
- DCDB->EarlyStatus = false;
- DCDB->Timeout = DAC960_V1_DCDB_Timeout_10_seconds;
- DCDB->NoAutomaticRequestSense = false;
- DCDB->DisconnectPermitted = true;
- DCDB->TransferLength =
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T);
- DCDB->BusAddress = NewInquiryUnitSerialNumberDMA;
- DCDB->CDBLength = 6;
- DCDB->TransferLengthHigh4 = 0;
- DCDB->SenseLength = sizeof(DCDB->SenseData);
- DCDB->CDB[0] = 0x12; /* INQUIRY */
- DCDB->CDB[1] = 1; /* EVPD = 1 */
- DCDB->CDB[2] = 0x80; /* Page Code */
- DCDB->CDB[3] = 0; /* Reserved */
- DCDB->CDB[4] = sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T);
- DCDB->CDB[5] = 0; /* Control */
- DAC960_QueueCommand(Command);
- return;
- }
- if (Controller->V1.StartDeviceStateScan)
- {
- Controller->V1.DeviceStateChannel = 0;
- Controller->V1.DeviceStateTargetID = 0;
- Controller->V1.StartDeviceStateScan = false;
- }
- else if (++Controller->V1.DeviceStateTargetID == Controller->Targets)
- {
- Controller->V1.DeviceStateChannel++;
- Controller->V1.DeviceStateTargetID = 0;
- }
- if (Controller->V1.DeviceStateChannel < Controller->Channels)
- {
- Controller->V1.NewDeviceState->DeviceState =
- DAC960_V1_Device_Dead;
- Command->V1.CommandMailbox.Type3D.CommandOpcode =
- DAC960_V1_GetDeviceState;
- Command->V1.CommandMailbox.Type3D.Channel =
- Controller->V1.DeviceStateChannel;
- Command->V1.CommandMailbox.Type3D.TargetID =
- Controller->V1.DeviceStateTargetID;
- Command->V1.CommandMailbox.Type3D.BusAddress =
- Controller->V1.NewDeviceStateDMA;
- DAC960_QueueCommand(Command);
- return;
- }
- Controller->V1.NeedDeviceStateInformation = false;
- }
- if (Controller->V1.NeedLogicalDriveInformation)
- {
- Controller->V1.NeedLogicalDriveInformation = false;
- Command->V1.CommandMailbox.Type3.CommandOpcode =
- DAC960_V1_GetLogicalDriveInformation;
- Command->V1.CommandMailbox.Type3.BusAddress =
- Controller->V1.NewLogicalDriveInformationDMA;
- DAC960_QueueCommand(Command);
- return;
- }
- if (Controller->V1.NeedRebuildProgress)
- {
- Controller->V1.NeedRebuildProgress = false;
- Command->V1.CommandMailbox.Type3.CommandOpcode =
- DAC960_V1_GetRebuildProgress;
- Command->V1.CommandMailbox.Type3.BusAddress =
- Controller->V1.RebuildProgressDMA;
- DAC960_QueueCommand(Command);
- return;
- }
- if (Controller->V1.NeedConsistencyCheckProgress)
- {
- Controller->V1.NeedConsistencyCheckProgress = false;
- Command->V1.CommandMailbox.Type3.CommandOpcode =
- DAC960_V1_RebuildStat;
- Command->V1.CommandMailbox.Type3.BusAddress =
- Controller->V1.RebuildProgressDMA;
- DAC960_QueueCommand(Command);
- return;
- }
- if (Controller->V1.NeedBackgroundInitializationStatus)
- {
- Controller->V1.NeedBackgroundInitializationStatus = false;
- Command->V1.CommandMailbox.Type3B.CommandOpcode =
- DAC960_V1_BackgroundInitializationControl;
- Command->V1.CommandMailbox.Type3B.CommandOpcode2 = 0x20;
- Command->V1.CommandMailbox.Type3B.BusAddress =
- Controller->V1.BackgroundInitializationStatusDMA;
- DAC960_QueueCommand(Command);
- return;
- }
- Controller->MonitoringTimerCount++;
- Controller->MonitoringTimer.expires =
- jiffies + DAC960_MonitoringTimerInterval;
- add_timer(&Controller->MonitoringTimer);
- }
- if (CommandType == DAC960_ImmediateCommand)
- {
- complete(Command->Completion);
- Command->Completion = NULL;
- return;
- }
- if (CommandType == DAC960_QueuedCommand)
- {
- DAC960_V1_KernelCommand_T *KernelCommand = Command->V1.KernelCommand;
- KernelCommand->CommandStatus = Command->V1.CommandStatus;
- Command->V1.KernelCommand = NULL;
- if (CommandOpcode == DAC960_V1_DCDB)
- Controller->V1.DirectCommandActive[KernelCommand->DCDB->Channel]
- [KernelCommand->DCDB->TargetID] =
- false;
- DAC960_DeallocateCommand(Command);
- KernelCommand->CompletionFunction(KernelCommand);
- return;
- }
- /*
- Queue a Status Monitoring Command to the Controller using the just
- completed Command if one was deferred previously due to lack of a
- free Command when the Monitoring Timer Function was called.
- */
- if (Controller->MonitoringCommandDeferred)
- {
- Controller->MonitoringCommandDeferred = false;
- DAC960_V1_QueueMonitoringCommand(Command);
- return;
- }
- /*
- Deallocate the Command.
- */
- DAC960_DeallocateCommand(Command);
- /*
- Wake up any processes waiting on a free Command.
- */
- wake_up(&Controller->CommandWaitQueue);
-}
-
-
-/*
- DAC960_V2_ReadWriteError prints an appropriate error message for Command
- when an error occurs on a Read or Write operation.
-*/
-
-static void DAC960_V2_ReadWriteError(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- static const unsigned char *SenseErrors[] = {
- "NO SENSE", "RECOVERED ERROR",
- "NOT READY", "MEDIUM ERROR",
- "HARDWARE ERROR", "ILLEGAL REQUEST",
- "UNIT ATTENTION", "DATA PROTECT",
- "BLANK CHECK", "VENDOR-SPECIFIC",
- "COPY ABORTED", "ABORTED COMMAND",
- "EQUAL", "VOLUME OVERFLOW",
- "MISCOMPARE", "RESERVED"
- };
- unsigned char *CommandName = "UNKNOWN";
- switch (Command->CommandType)
- {
- case DAC960_ReadCommand:
- case DAC960_ReadRetryCommand:
- CommandName = "READ";
- break;
- case DAC960_WriteCommand:
- case DAC960_WriteRetryCommand:
- CommandName = "WRITE";
- break;
- case DAC960_MonitoringCommand:
- case DAC960_ImmediateCommand:
- case DAC960_QueuedCommand:
- break;
- }
- DAC960_Error("Error Condition %s on %s:\n", Controller,
- SenseErrors[Command->V2.RequestSense->SenseKey], CommandName);
- DAC960_Error(" /dev/rd/c%dd%d: absolute blocks %u..%u\n",
- Controller, Controller->ControllerNumber,
- Command->LogicalDriveNumber, Command->BlockNumber,
- Command->BlockNumber + Command->BlockCount - 1);
-}
-
-
-/*
- DAC960_V2_ReportEvent prints an appropriate message when a Controller Event
- occurs.
-*/
-
-static void DAC960_V2_ReportEvent(DAC960_Controller_T *Controller,
- DAC960_V2_Event_T *Event)
-{
- DAC960_SCSI_RequestSense_T *RequestSense =
- (DAC960_SCSI_RequestSense_T *) &Event->RequestSenseData;
- unsigned char MessageBuffer[DAC960_LineBufferSize];
- static struct { int EventCode; unsigned char *EventMessage; } EventList[] =
- { /* Physical Device Events (0x0000 - 0x007F) */
- { 0x0001, "P Online" },
- { 0x0002, "P Standby" },
- { 0x0005, "P Automatic Rebuild Started" },
- { 0x0006, "P Manual Rebuild Started" },
- { 0x0007, "P Rebuild Completed" },
- { 0x0008, "P Rebuild Cancelled" },
- { 0x0009, "P Rebuild Failed for Unknown Reasons" },
- { 0x000A, "P Rebuild Failed due to New Physical Device" },
- { 0x000B, "P Rebuild Failed due to Logical Drive Failure" },
- { 0x000C, "S Offline" },
- { 0x000D, "P Found" },
- { 0x000E, "P Removed" },
- { 0x000F, "P Unconfigured" },
- { 0x0010, "P Expand Capacity Started" },
- { 0x0011, "P Expand Capacity Completed" },
- { 0x0012, "P Expand Capacity Failed" },
- { 0x0013, "P Command Timed Out" },
- { 0x0014, "P Command Aborted" },
- { 0x0015, "P Command Retried" },
- { 0x0016, "P Parity Error" },
- { 0x0017, "P Soft Error" },
- { 0x0018, "P Miscellaneous Error" },
- { 0x0019, "P Reset" },
- { 0x001A, "P Active Spare Found" },
- { 0x001B, "P Warm Spare Found" },
- { 0x001C, "S Sense Data Received" },
- { 0x001D, "P Initialization Started" },
- { 0x001E, "P Initialization Completed" },
- { 0x001F, "P Initialization Failed" },
- { 0x0020, "P Initialization Cancelled" },
- { 0x0021, "P Failed because Write Recovery Failed" },
- { 0x0022, "P Failed because SCSI Bus Reset Failed" },
- { 0x0023, "P Failed because of Double Check Condition" },
- { 0x0024, "P Failed because Device Cannot Be Accessed" },
- { 0x0025, "P Failed because of Gross Error on SCSI Processor" },
- { 0x0026, "P Failed because of Bad Tag from Device" },
- { 0x0027, "P Failed because of Command Timeout" },
- { 0x0028, "P Failed because of System Reset" },
- { 0x0029, "P Failed because of Busy Status or Parity Error" },
- { 0x002A, "P Failed because Host Set Device to Failed State" },
- { 0x002B, "P Failed because of Selection Timeout" },
- { 0x002C, "P Failed because of SCSI Bus Phase Error" },
- { 0x002D, "P Failed because Device Returned Unknown Status" },
- { 0x002E, "P Failed because Device Not Ready" },
- { 0x002F, "P Failed because Device Not Found at Startup" },
- { 0x0030, "P Failed because COD Write Operation Failed" },
- { 0x0031, "P Failed because BDT Write Operation Failed" },
- { 0x0039, "P Missing at Startup" },
- { 0x003A, "P Start Rebuild Failed due to Physical Drive Too Small" },
- { 0x003C, "P Temporarily Offline Device Automatically Made Online" },
- { 0x003D, "P Standby Rebuild Started" },
- /* Logical Device Events (0x0080 - 0x00FF) */
- { 0x0080, "M Consistency Check Started" },
- { 0x0081, "M Consistency Check Completed" },
- { 0x0082, "M Consistency Check Cancelled" },
- { 0x0083, "M Consistency Check Completed With Errors" },
- { 0x0084, "M Consistency Check Failed due to Logical Drive Failure" },
- { 0x0085, "M Consistency Check Failed due to Physical Device Failure" },
- { 0x0086, "L Offline" },
- { 0x0087, "L Critical" },
- { 0x0088, "L Online" },
- { 0x0089, "M Automatic Rebuild Started" },
- { 0x008A, "M Manual Rebuild Started" },
- { 0x008B, "M Rebuild Completed" },
- { 0x008C, "M Rebuild Cancelled" },
- { 0x008D, "M Rebuild Failed for Unknown Reasons" },
- { 0x008E, "M Rebuild Failed due to New Physical Device" },
- { 0x008F, "M Rebuild Failed due to Logical Drive Failure" },
- { 0x0090, "M Initialization Started" },
- { 0x0091, "M Initialization Completed" },
- { 0x0092, "M Initialization Cancelled" },
- { 0x0093, "M Initialization Failed" },
- { 0x0094, "L Found" },
- { 0x0095, "L Deleted" },
- { 0x0096, "M Expand Capacity Started" },
- { 0x0097, "M Expand Capacity Completed" },
- { 0x0098, "M Expand Capacity Failed" },
- { 0x0099, "L Bad Block Found" },
- { 0x009A, "L Size Changed" },
- { 0x009B, "L Type Changed" },
- { 0x009C, "L Bad Data Block Found" },
- { 0x009E, "L Read of Data Block in BDT" },
- { 0x009F, "L Write Back Data for Disk Block Lost" },
- { 0x00A0, "L Temporarily Offline RAID-5/3 Drive Made Online" },
- { 0x00A1, "L Temporarily Offline RAID-6/1/0/7 Drive Made Online" },
- { 0x00A2, "L Standby Rebuild Started" },
- /* Fault Management Events (0x0100 - 0x017F) */
- { 0x0140, "E Fan %d Failed" },
- { 0x0141, "E Fan %d OK" },
- { 0x0142, "E Fan %d Not Present" },
- { 0x0143, "E Power Supply %d Failed" },
- { 0x0144, "E Power Supply %d OK" },
- { 0x0145, "E Power Supply %d Not Present" },
- { 0x0146, "E Temperature Sensor %d Temperature Exceeds Safe Limit" },
- { 0x0147, "E Temperature Sensor %d Temperature Exceeds Working Limit" },
- { 0x0148, "E Temperature Sensor %d Temperature Normal" },
- { 0x0149, "E Temperature Sensor %d Not Present" },
- { 0x014A, "E Enclosure Management Unit %d Access Critical" },
- { 0x014B, "E Enclosure Management Unit %d Access OK" },
- { 0x014C, "E Enclosure Management Unit %d Access Offline" },
- /* Controller Events (0x0180 - 0x01FF) */
- { 0x0181, "C Cache Write Back Error" },
- { 0x0188, "C Battery Backup Unit Found" },
- { 0x0189, "C Battery Backup Unit Charge Level Low" },
- { 0x018A, "C Battery Backup Unit Charge Level OK" },
- { 0x0193, "C Installation Aborted" },
- { 0x0195, "C Battery Backup Unit Physically Removed" },
- { 0x0196, "C Memory Error During Warm Boot" },
- { 0x019E, "C Memory Soft ECC Error Corrected" },
- { 0x019F, "C Memory Hard ECC Error Corrected" },
- { 0x01A2, "C Battery Backup Unit Failed" },
- { 0x01AB, "C Mirror Race Recovery Failed" },
- { 0x01AC, "C Mirror Race on Critical Drive" },
- /* Controller Internal Processor Events */
- { 0x0380, "C Internal Controller Hung" },
- { 0x0381, "C Internal Controller Firmware Breakpoint" },
- { 0x0390, "C Internal Controller i960 Processor Specific Error" },
- { 0x03A0, "C Internal Controller StrongARM Processor Specific Error" },
- { 0, "" } };
- int EventListIndex = 0, EventCode;
- unsigned char EventType, *EventMessage;
- if (Event->EventCode == 0x1C &&
- RequestSense->SenseKey == DAC960_SenseKey_VendorSpecific &&
- (RequestSense->AdditionalSenseCode == 0x80 ||
- RequestSense->AdditionalSenseCode == 0x81))
- Event->EventCode = ((RequestSense->AdditionalSenseCode - 0x80) << 8) |
- RequestSense->AdditionalSenseCodeQualifier;
- while (true)
- {
- EventCode = EventList[EventListIndex].EventCode;
- if (EventCode == Event->EventCode || EventCode == 0) break;
- EventListIndex++;
- }
- EventType = EventList[EventListIndex].EventMessage[0];
- EventMessage = &EventList[EventListIndex].EventMessage[2];
- if (EventCode == 0)
- {
- DAC960_Critical("Unknown Controller Event Code %04X\n",
- Controller, Event->EventCode);
- return;
- }
- switch (EventType)
- {
- case 'P':
- DAC960_Critical("Physical Device %d:%d %s\n", Controller,
- Event->Channel, Event->TargetID, EventMessage);
- break;
- case 'L':
- DAC960_Critical("Logical Drive %d (/dev/rd/c%dd%d) %s\n", Controller,
- Event->LogicalUnit, Controller->ControllerNumber,
- Event->LogicalUnit, EventMessage);
- break;
- case 'M':
- DAC960_Progress("Logical Drive %d (/dev/rd/c%dd%d) %s\n", Controller,
- Event->LogicalUnit, Controller->ControllerNumber,
- Event->LogicalUnit, EventMessage);
- break;
- case 'S':
- if (RequestSense->SenseKey == DAC960_SenseKey_NoSense ||
- (RequestSense->SenseKey == DAC960_SenseKey_NotReady &&
- RequestSense->AdditionalSenseCode == 0x04 &&
- (RequestSense->AdditionalSenseCodeQualifier == 0x01 ||
- RequestSense->AdditionalSenseCodeQualifier == 0x02)))
- break;
- DAC960_Critical("Physical Device %d:%d %s\n", Controller,
- Event->Channel, Event->TargetID, EventMessage);
- DAC960_Critical("Physical Device %d:%d Request Sense: "
- "Sense Key = %X, ASC = %02X, ASCQ = %02X\n",
- Controller,
- Event->Channel,
- Event->TargetID,
- RequestSense->SenseKey,
- RequestSense->AdditionalSenseCode,
- RequestSense->AdditionalSenseCodeQualifier);
- DAC960_Critical("Physical Device %d:%d Request Sense: "
- "Information = %02X%02X%02X%02X "
- "%02X%02X%02X%02X\n",
- Controller,
- Event->Channel,
- Event->TargetID,
- RequestSense->Information[0],
- RequestSense->Information[1],
- RequestSense->Information[2],
- RequestSense->Information[3],
- RequestSense->CommandSpecificInformation[0],
- RequestSense->CommandSpecificInformation[1],
- RequestSense->CommandSpecificInformation[2],
- RequestSense->CommandSpecificInformation[3]);
- break;
- case 'E':
- if (Controller->SuppressEnclosureMessages) break;
- sprintf(MessageBuffer, EventMessage, Event->LogicalUnit);
- DAC960_Critical("Enclosure %d %s\n", Controller,
- Event->TargetID, MessageBuffer);
- break;
- case 'C':
- DAC960_Critical("Controller %s\n", Controller, EventMessage);
- break;
- default:
- DAC960_Critical("Unknown Controller Event Code %04X\n",
- Controller, Event->EventCode);
- break;
- }
-}
-
-
-/*
- DAC960_V2_ReportProgress prints an appropriate progress message for
- Logical Device Long Operations.
-*/
-
-static void DAC960_V2_ReportProgress(DAC960_Controller_T *Controller,
- unsigned char *MessageString,
- unsigned int LogicalDeviceNumber,
- unsigned long BlocksCompleted,
- unsigned long LogicalDeviceSize)
-{
- Controller->EphemeralProgressMessage = true;
- DAC960_Progress("%s in Progress: Logical Drive %d (/dev/rd/c%dd%d) "
- "%d%% completed\n", Controller,
- MessageString,
- LogicalDeviceNumber,
- Controller->ControllerNumber,
- LogicalDeviceNumber,
- (100 * (BlocksCompleted >> 7)) / (LogicalDeviceSize >> 7));
- Controller->EphemeralProgressMessage = false;
-}
-
-
-/*
- DAC960_V2_ProcessCompletedCommand performs completion processing for Command
- for DAC960 V2 Firmware Controllers.
-*/
-
-static void DAC960_V2_ProcessCompletedCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- DAC960_CommandType_T CommandType = Command->CommandType;
- DAC960_V2_CommandMailbox_T *CommandMailbox = &Command->V2.CommandMailbox;
- DAC960_V2_IOCTL_Opcode_T IOCTLOpcode = CommandMailbox->Common.IOCTL_Opcode;
- DAC960_V2_CommandOpcode_T CommandOpcode = CommandMailbox->SCSI_10.CommandOpcode;
- DAC960_V2_CommandStatus_T CommandStatus = Command->V2.CommandStatus;
-
- if (CommandType == DAC960_ReadCommand ||
- CommandType == DAC960_WriteCommand)
- {
-
-#ifdef FORCE_RETRY_DEBUG
- CommandStatus = DAC960_V2_AbormalCompletion;
-#endif
- Command->V2.RequestSense->SenseKey = DAC960_SenseKey_MediumError;
-
- if (CommandStatus == DAC960_V2_NormalCompletion) {
-
- if (!DAC960_ProcessCompletedRequest(Command, true))
- BUG();
-
- } else if (Command->V2.RequestSense->SenseKey == DAC960_SenseKey_MediumError)
- {
- /*
- * break the command down into pieces and resubmit each
- * piece, hoping that some of them will succeed.
- */
- DAC960_queue_partial_rw(Command);
- return;
- }
- else
- {
- if (Command->V2.RequestSense->SenseKey != DAC960_SenseKey_NotReady)
- DAC960_V2_ReadWriteError(Command);
- /*
- Perform completion processing for all buffers in this I/O Request.
- */
- (void)DAC960_ProcessCompletedRequest(Command, false);
- }
- }
- else if (CommandType == DAC960_ReadRetryCommand ||
- CommandType == DAC960_WriteRetryCommand)
- {
- bool normal_completion;
-
-#ifdef FORCE_RETRY_FAILURE_DEBUG
- static int retry_count = 1;
-#endif
- /*
- Perform completion processing for the portion that was
- retried, and submit the next portion, if any.
- */
- normal_completion = true;
- if (CommandStatus != DAC960_V2_NormalCompletion) {
- normal_completion = false;
- if (Command->V2.RequestSense->SenseKey != DAC960_SenseKey_NotReady)
- DAC960_V2_ReadWriteError(Command);
- }
-
-#ifdef FORCE_RETRY_FAILURE_DEBUG
- if (!(++retry_count % 10000)) {
- printk("V2 error retry failure test\n");
- normal_completion = false;
- DAC960_V2_ReadWriteError(Command);
- }
-#endif
-
- if (!DAC960_ProcessCompletedRequest(Command, normal_completion)) {
- DAC960_queue_partial_rw(Command);
- return;
- }
- }
- else if (CommandType == DAC960_MonitoringCommand)
- {
- if (Controller->ShutdownMonitoringTimer)
- return;
- if (IOCTLOpcode == DAC960_V2_GetControllerInfo)
- {
- DAC960_V2_ControllerInfo_T *NewControllerInfo =
- Controller->V2.NewControllerInformation;
- DAC960_V2_ControllerInfo_T *ControllerInfo =
- &Controller->V2.ControllerInformation;
- Controller->LogicalDriveCount =
- NewControllerInfo->LogicalDevicesPresent;
- Controller->V2.NeedLogicalDeviceInformation = true;
- Controller->V2.NeedPhysicalDeviceInformation = true;
- Controller->V2.StartLogicalDeviceInformationScan = true;
- Controller->V2.StartPhysicalDeviceInformationScan = true;
- Controller->MonitoringAlertMode =
- (NewControllerInfo->LogicalDevicesCritical > 0 ||
- NewControllerInfo->LogicalDevicesOffline > 0 ||
- NewControllerInfo->PhysicalDisksCritical > 0 ||
- NewControllerInfo->PhysicalDisksOffline > 0);
- memcpy(ControllerInfo, NewControllerInfo,
- sizeof(DAC960_V2_ControllerInfo_T));
- }
- else if (IOCTLOpcode == DAC960_V2_GetEvent)
- {
- if (CommandStatus == DAC960_V2_NormalCompletion) {
- DAC960_V2_ReportEvent(Controller, Controller->V2.Event);
- }
- Controller->V2.NextEventSequenceNumber++;
- }
- else if (IOCTLOpcode == DAC960_V2_GetPhysicalDeviceInfoValid &&
- CommandStatus == DAC960_V2_NormalCompletion)
- {
- DAC960_V2_PhysicalDeviceInfo_T *NewPhysicalDeviceInfo =
- Controller->V2.NewPhysicalDeviceInformation;
- unsigned int PhysicalDeviceIndex = Controller->V2.PhysicalDeviceIndex;
- DAC960_V2_PhysicalDeviceInfo_T *PhysicalDeviceInfo =
- Controller->V2.PhysicalDeviceInformation[PhysicalDeviceIndex];
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *InquiryUnitSerialNumber =
- Controller->V2.InquiryUnitSerialNumber[PhysicalDeviceIndex];
- unsigned int DeviceIndex;
- while (PhysicalDeviceInfo != NULL &&
- (NewPhysicalDeviceInfo->Channel >
- PhysicalDeviceInfo->Channel ||
- (NewPhysicalDeviceInfo->Channel ==
- PhysicalDeviceInfo->Channel &&
- (NewPhysicalDeviceInfo->TargetID >
- PhysicalDeviceInfo->TargetID ||
- (NewPhysicalDeviceInfo->TargetID ==
- PhysicalDeviceInfo->TargetID &&
- NewPhysicalDeviceInfo->LogicalUnit >
- PhysicalDeviceInfo->LogicalUnit)))))
- {
- DAC960_Critical("Physical Device %d:%d No Longer Exists\n",
- Controller,
- PhysicalDeviceInfo->Channel,
- PhysicalDeviceInfo->TargetID);
- Controller->V2.PhysicalDeviceInformation
- [PhysicalDeviceIndex] = NULL;
- Controller->V2.InquiryUnitSerialNumber
- [PhysicalDeviceIndex] = NULL;
- kfree(PhysicalDeviceInfo);
- kfree(InquiryUnitSerialNumber);
- for (DeviceIndex = PhysicalDeviceIndex;
- DeviceIndex < DAC960_V2_MaxPhysicalDevices - 1;
- DeviceIndex++)
- {
- Controller->V2.PhysicalDeviceInformation[DeviceIndex] =
- Controller->V2.PhysicalDeviceInformation[DeviceIndex+1];
- Controller->V2.InquiryUnitSerialNumber[DeviceIndex] =
- Controller->V2.InquiryUnitSerialNumber[DeviceIndex+1];
- }
- Controller->V2.PhysicalDeviceInformation
- [DAC960_V2_MaxPhysicalDevices-1] = NULL;
- Controller->V2.InquiryUnitSerialNumber
- [DAC960_V2_MaxPhysicalDevices-1] = NULL;
- PhysicalDeviceInfo =
- Controller->V2.PhysicalDeviceInformation[PhysicalDeviceIndex];
- InquiryUnitSerialNumber =
- Controller->V2.InquiryUnitSerialNumber[PhysicalDeviceIndex];
- }
- if (PhysicalDeviceInfo == NULL ||
- (NewPhysicalDeviceInfo->Channel !=
- PhysicalDeviceInfo->Channel) ||
- (NewPhysicalDeviceInfo->TargetID !=
- PhysicalDeviceInfo->TargetID) ||
- (NewPhysicalDeviceInfo->LogicalUnit !=
- PhysicalDeviceInfo->LogicalUnit))
- {
- PhysicalDeviceInfo =
- kmalloc(sizeof(DAC960_V2_PhysicalDeviceInfo_T), GFP_ATOMIC);
- InquiryUnitSerialNumber =
- kmalloc(sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T),
- GFP_ATOMIC);
- if (InquiryUnitSerialNumber == NULL ||
- PhysicalDeviceInfo == NULL)
- {
- kfree(InquiryUnitSerialNumber);
- InquiryUnitSerialNumber = NULL;
- kfree(PhysicalDeviceInfo);
- PhysicalDeviceInfo = NULL;
- }
- DAC960_Critical("Physical Device %d:%d Now Exists%s\n",
- Controller,
- NewPhysicalDeviceInfo->Channel,
- NewPhysicalDeviceInfo->TargetID,
- (PhysicalDeviceInfo != NULL
- ? "" : " - Allocation Failed"));
- if (PhysicalDeviceInfo != NULL)
- {
- memset(PhysicalDeviceInfo, 0,
- sizeof(DAC960_V2_PhysicalDeviceInfo_T));
- PhysicalDeviceInfo->PhysicalDeviceState =
- DAC960_V2_Device_InvalidState;
- memset(InquiryUnitSerialNumber, 0,
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T));
- InquiryUnitSerialNumber->PeripheralDeviceType = 0x1F;
- for (DeviceIndex = DAC960_V2_MaxPhysicalDevices - 1;
- DeviceIndex > PhysicalDeviceIndex;
- DeviceIndex--)
- {
- Controller->V2.PhysicalDeviceInformation[DeviceIndex] =
- Controller->V2.PhysicalDeviceInformation[DeviceIndex-1];
- Controller->V2.InquiryUnitSerialNumber[DeviceIndex] =
- Controller->V2.InquiryUnitSerialNumber[DeviceIndex-1];
- }
- Controller->V2.PhysicalDeviceInformation
- [PhysicalDeviceIndex] =
- PhysicalDeviceInfo;
- Controller->V2.InquiryUnitSerialNumber
- [PhysicalDeviceIndex] =
- InquiryUnitSerialNumber;
- Controller->V2.NeedDeviceSerialNumberInformation = true;
- }
- }
- if (PhysicalDeviceInfo != NULL)
- {
- if (NewPhysicalDeviceInfo->PhysicalDeviceState !=
- PhysicalDeviceInfo->PhysicalDeviceState)
- DAC960_Critical(
- "Physical Device %d:%d is now %s\n", Controller,
- NewPhysicalDeviceInfo->Channel,
- NewPhysicalDeviceInfo->TargetID,
- (NewPhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Online
- ? "ONLINE"
- : NewPhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Rebuild
- ? "REBUILD"
- : NewPhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Missing
- ? "MISSING"
- : NewPhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Critical
- ? "CRITICAL"
- : NewPhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Dead
- ? "DEAD"
- : NewPhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_SuspectedDead
- ? "SUSPECTED-DEAD"
- : NewPhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_CommandedOffline
- ? "COMMANDED-OFFLINE"
- : NewPhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Standby
- ? "STANDBY" : "UNKNOWN"));
- if ((NewPhysicalDeviceInfo->ParityErrors !=
- PhysicalDeviceInfo->ParityErrors) ||
- (NewPhysicalDeviceInfo->SoftErrors !=
- PhysicalDeviceInfo->SoftErrors) ||
- (NewPhysicalDeviceInfo->HardErrors !=
- PhysicalDeviceInfo->HardErrors) ||
- (NewPhysicalDeviceInfo->MiscellaneousErrors !=
- PhysicalDeviceInfo->MiscellaneousErrors) ||
- (NewPhysicalDeviceInfo->CommandTimeouts !=
- PhysicalDeviceInfo->CommandTimeouts) ||
- (NewPhysicalDeviceInfo->Retries !=
- PhysicalDeviceInfo->Retries) ||
- (NewPhysicalDeviceInfo->Aborts !=
- PhysicalDeviceInfo->Aborts) ||
- (NewPhysicalDeviceInfo->PredictedFailuresDetected !=
- PhysicalDeviceInfo->PredictedFailuresDetected))
- {
- DAC960_Critical("Physical Device %d:%d Errors: "
- "Parity = %d, Soft = %d, "
- "Hard = %d, Misc = %d\n",
- Controller,
- NewPhysicalDeviceInfo->Channel,
- NewPhysicalDeviceInfo->TargetID,
- NewPhysicalDeviceInfo->ParityErrors,
- NewPhysicalDeviceInfo->SoftErrors,
- NewPhysicalDeviceInfo->HardErrors,
- NewPhysicalDeviceInfo->MiscellaneousErrors);
- DAC960_Critical("Physical Device %d:%d Errors: "
- "Timeouts = %d, Retries = %d, "
- "Aborts = %d, Predicted = %d\n",
- Controller,
- NewPhysicalDeviceInfo->Channel,
- NewPhysicalDeviceInfo->TargetID,
- NewPhysicalDeviceInfo->CommandTimeouts,
- NewPhysicalDeviceInfo->Retries,
- NewPhysicalDeviceInfo->Aborts,
- NewPhysicalDeviceInfo
- ->PredictedFailuresDetected);
- }
- if ((PhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_Dead ||
- PhysicalDeviceInfo->PhysicalDeviceState
- == DAC960_V2_Device_InvalidState) &&
- NewPhysicalDeviceInfo->PhysicalDeviceState
- != DAC960_V2_Device_Dead)
- Controller->V2.NeedDeviceSerialNumberInformation = true;
- memcpy(PhysicalDeviceInfo, NewPhysicalDeviceInfo,
- sizeof(DAC960_V2_PhysicalDeviceInfo_T));
- }
- NewPhysicalDeviceInfo->LogicalUnit++;
- Controller->V2.PhysicalDeviceIndex++;
- }
- else if (IOCTLOpcode == DAC960_V2_GetPhysicalDeviceInfoValid)
- {
- unsigned int DeviceIndex;
- for (DeviceIndex = Controller->V2.PhysicalDeviceIndex;
- DeviceIndex < DAC960_V2_MaxPhysicalDevices;
- DeviceIndex++)
- {
- DAC960_V2_PhysicalDeviceInfo_T *PhysicalDeviceInfo =
- Controller->V2.PhysicalDeviceInformation[DeviceIndex];
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *InquiryUnitSerialNumber =
- Controller->V2.InquiryUnitSerialNumber[DeviceIndex];
- if (PhysicalDeviceInfo == NULL) break;
- DAC960_Critical("Physical Device %d:%d No Longer Exists\n",
- Controller,
- PhysicalDeviceInfo->Channel,
- PhysicalDeviceInfo->TargetID);
- Controller->V2.PhysicalDeviceInformation[DeviceIndex] = NULL;
- Controller->V2.InquiryUnitSerialNumber[DeviceIndex] = NULL;
- kfree(PhysicalDeviceInfo);
- kfree(InquiryUnitSerialNumber);
- }
- Controller->V2.NeedPhysicalDeviceInformation = false;
- }
- else if (IOCTLOpcode == DAC960_V2_GetLogicalDeviceInfoValid &&
- CommandStatus == DAC960_V2_NormalCompletion)
- {
- DAC960_V2_LogicalDeviceInfo_T *NewLogicalDeviceInfo =
- Controller->V2.NewLogicalDeviceInformation;
- unsigned short LogicalDeviceNumber =
- NewLogicalDeviceInfo->LogicalDeviceNumber;
- DAC960_V2_LogicalDeviceInfo_T *LogicalDeviceInfo =
- Controller->V2.LogicalDeviceInformation[LogicalDeviceNumber];
- if (LogicalDeviceInfo == NULL)
- {
- DAC960_V2_PhysicalDevice_T PhysicalDevice;
- PhysicalDevice.Controller = 0;
- PhysicalDevice.Channel = NewLogicalDeviceInfo->Channel;
- PhysicalDevice.TargetID = NewLogicalDeviceInfo->TargetID;
- PhysicalDevice.LogicalUnit = NewLogicalDeviceInfo->LogicalUnit;
- Controller->V2.LogicalDriveToVirtualDevice[LogicalDeviceNumber] =
- PhysicalDevice;
- LogicalDeviceInfo = kmalloc(sizeof(DAC960_V2_LogicalDeviceInfo_T),
- GFP_ATOMIC);
- Controller->V2.LogicalDeviceInformation[LogicalDeviceNumber] =
- LogicalDeviceInfo;
- DAC960_Critical("Logical Drive %d (/dev/rd/c%dd%d) "
- "Now Exists%s\n", Controller,
- LogicalDeviceNumber,
- Controller->ControllerNumber,
- LogicalDeviceNumber,
- (LogicalDeviceInfo != NULL
- ? "" : " - Allocation Failed"));
- if (LogicalDeviceInfo != NULL)
- {
- memset(LogicalDeviceInfo, 0,
- sizeof(DAC960_V2_LogicalDeviceInfo_T));
- DAC960_ComputeGenericDiskInfo(Controller);
- }
- }
- if (LogicalDeviceInfo != NULL)
- {
- unsigned long LogicalDeviceSize =
- NewLogicalDeviceInfo->ConfigurableDeviceSize;
- if (NewLogicalDeviceInfo->LogicalDeviceState !=
- LogicalDeviceInfo->LogicalDeviceState)
- DAC960_Critical("Logical Drive %d (/dev/rd/c%dd%d) "
- "is now %s\n", Controller,
- LogicalDeviceNumber,
- Controller->ControllerNumber,
- LogicalDeviceNumber,
- (NewLogicalDeviceInfo->LogicalDeviceState
- == DAC960_V2_LogicalDevice_Online
- ? "ONLINE"
- : NewLogicalDeviceInfo->LogicalDeviceState
- == DAC960_V2_LogicalDevice_Critical
- ? "CRITICAL" : "OFFLINE"));
- if ((NewLogicalDeviceInfo->SoftErrors !=
- LogicalDeviceInfo->SoftErrors) ||
- (NewLogicalDeviceInfo->CommandsFailed !=
- LogicalDeviceInfo->CommandsFailed) ||
- (NewLogicalDeviceInfo->DeferredWriteErrors !=
- LogicalDeviceInfo->DeferredWriteErrors))
- DAC960_Critical("Logical Drive %d (/dev/rd/c%dd%d) Errors: "
- "Soft = %d, Failed = %d, Deferred Write = %d\n",
- Controller, LogicalDeviceNumber,
- Controller->ControllerNumber,
- LogicalDeviceNumber,
- NewLogicalDeviceInfo->SoftErrors,
- NewLogicalDeviceInfo->CommandsFailed,
- NewLogicalDeviceInfo->DeferredWriteErrors);
- if (NewLogicalDeviceInfo->ConsistencyCheckInProgress)
- DAC960_V2_ReportProgress(Controller,
- "Consistency Check",
- LogicalDeviceNumber,
- NewLogicalDeviceInfo
- ->ConsistencyCheckBlockNumber,
- LogicalDeviceSize);
- else if (NewLogicalDeviceInfo->RebuildInProgress)
- DAC960_V2_ReportProgress(Controller,
- "Rebuild",
- LogicalDeviceNumber,
- NewLogicalDeviceInfo
- ->RebuildBlockNumber,
- LogicalDeviceSize);
- else if (NewLogicalDeviceInfo->BackgroundInitializationInProgress)
- DAC960_V2_ReportProgress(Controller,
- "Background Initialization",
- LogicalDeviceNumber,
- NewLogicalDeviceInfo
- ->BackgroundInitializationBlockNumber,
- LogicalDeviceSize);
- else if (NewLogicalDeviceInfo->ForegroundInitializationInProgress)
- DAC960_V2_ReportProgress(Controller,
- "Foreground Initialization",
- LogicalDeviceNumber,
- NewLogicalDeviceInfo
- ->ForegroundInitializationBlockNumber,
- LogicalDeviceSize);
- else if (NewLogicalDeviceInfo->DataMigrationInProgress)
- DAC960_V2_ReportProgress(Controller,
- "Data Migration",
- LogicalDeviceNumber,
- NewLogicalDeviceInfo
- ->DataMigrationBlockNumber,
- LogicalDeviceSize);
- else if (NewLogicalDeviceInfo->PatrolOperationInProgress)
- DAC960_V2_ReportProgress(Controller,
- "Patrol Operation",
- LogicalDeviceNumber,
- NewLogicalDeviceInfo
- ->PatrolOperationBlockNumber,
- LogicalDeviceSize);
- if (LogicalDeviceInfo->BackgroundInitializationInProgress &&
- !NewLogicalDeviceInfo->BackgroundInitializationInProgress)
- DAC960_Progress("Logical Drive %d (/dev/rd/c%dd%d) "
- "Background Initialization %s\n",
- Controller,
- LogicalDeviceNumber,
- Controller->ControllerNumber,
- LogicalDeviceNumber,
- (NewLogicalDeviceInfo->LogicalDeviceControl
- .LogicalDeviceInitialized
- ? "Completed" : "Failed"));
- memcpy(LogicalDeviceInfo, NewLogicalDeviceInfo,
- sizeof(DAC960_V2_LogicalDeviceInfo_T));
- }
- Controller->V2.LogicalDriveFoundDuringScan
- [LogicalDeviceNumber] = true;
- NewLogicalDeviceInfo->LogicalDeviceNumber++;
- }
- else if (IOCTLOpcode == DAC960_V2_GetLogicalDeviceInfoValid)
- {
- int LogicalDriveNumber;
- for (LogicalDriveNumber = 0;
- LogicalDriveNumber < DAC960_MaxLogicalDrives;
- LogicalDriveNumber++)
- {
- DAC960_V2_LogicalDeviceInfo_T *LogicalDeviceInfo =
- Controller->V2.LogicalDeviceInformation[LogicalDriveNumber];
- if (LogicalDeviceInfo == NULL ||
- Controller->V2.LogicalDriveFoundDuringScan
- [LogicalDriveNumber])
- continue;
- DAC960_Critical("Logical Drive %d (/dev/rd/c%dd%d) "
- "No Longer Exists\n", Controller,
- LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber);
- Controller->V2.LogicalDeviceInformation
- [LogicalDriveNumber] = NULL;
- kfree(LogicalDeviceInfo);
- Controller->LogicalDriveInitiallyAccessible
- [LogicalDriveNumber] = false;
- DAC960_ComputeGenericDiskInfo(Controller);
- }
- Controller->V2.NeedLogicalDeviceInformation = false;
- }
- else if (CommandOpcode == DAC960_V2_SCSI_10_Passthru)
- {
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *InquiryUnitSerialNumber =
- Controller->V2.InquiryUnitSerialNumber[Controller->V2.PhysicalDeviceIndex - 1];
-
- if (CommandStatus != DAC960_V2_NormalCompletion) {
- memset(InquiryUnitSerialNumber,
- 0, sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T));
- InquiryUnitSerialNumber->PeripheralDeviceType = 0x1F;
- } else
- memcpy(InquiryUnitSerialNumber,
- Controller->V2.NewInquiryUnitSerialNumber,
- sizeof(DAC960_SCSI_Inquiry_UnitSerialNumber_T));
-
- Controller->V2.NeedDeviceSerialNumberInformation = false;
- }
-
- if (Controller->V2.HealthStatusBuffer->NextEventSequenceNumber
- - Controller->V2.NextEventSequenceNumber > 0)
- {
- CommandMailbox->GetEvent.CommandOpcode = DAC960_V2_IOCTL;
- CommandMailbox->GetEvent.DataTransferSize = sizeof(DAC960_V2_Event_T);
- CommandMailbox->GetEvent.EventSequenceNumberHigh16 =
- Controller->V2.NextEventSequenceNumber >> 16;
- CommandMailbox->GetEvent.ControllerNumber = 0;
- CommandMailbox->GetEvent.IOCTL_Opcode =
- DAC960_V2_GetEvent;
- CommandMailbox->GetEvent.EventSequenceNumberLow16 =
- Controller->V2.NextEventSequenceNumber & 0xFFFF;
- CommandMailbox->GetEvent.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer =
- Controller->V2.EventDMA;
- CommandMailbox->GetEvent.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->GetEvent.DataTransferSize;
- DAC960_QueueCommand(Command);
- return;
- }
- if (Controller->V2.NeedPhysicalDeviceInformation)
- {
- if (Controller->V2.NeedDeviceSerialNumberInformation)
- {
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *InquiryUnitSerialNumber =
- Controller->V2.NewInquiryUnitSerialNumber;
- InquiryUnitSerialNumber->PeripheralDeviceType = 0x1F;
-
- DAC960_V2_ConstructNewUnitSerialNumber(Controller, CommandMailbox,
- Controller->V2.NewPhysicalDeviceInformation->Channel,
- Controller->V2.NewPhysicalDeviceInformation->TargetID,
- Controller->V2.NewPhysicalDeviceInformation->LogicalUnit - 1);
-
-
- DAC960_QueueCommand(Command);
- return;
- }
- if (Controller->V2.StartPhysicalDeviceInformationScan)
- {
- Controller->V2.PhysicalDeviceIndex = 0;
- Controller->V2.NewPhysicalDeviceInformation->Channel = 0;
- Controller->V2.NewPhysicalDeviceInformation->TargetID = 0;
- Controller->V2.NewPhysicalDeviceInformation->LogicalUnit = 0;
- Controller->V2.StartPhysicalDeviceInformationScan = false;
- }
- CommandMailbox->PhysicalDeviceInfo.CommandOpcode = DAC960_V2_IOCTL;
- CommandMailbox->PhysicalDeviceInfo.DataTransferSize =
- sizeof(DAC960_V2_PhysicalDeviceInfo_T);
- CommandMailbox->PhysicalDeviceInfo.PhysicalDevice.LogicalUnit =
- Controller->V2.NewPhysicalDeviceInformation->LogicalUnit;
- CommandMailbox->PhysicalDeviceInfo.PhysicalDevice.TargetID =
- Controller->V2.NewPhysicalDeviceInformation->TargetID;
- CommandMailbox->PhysicalDeviceInfo.PhysicalDevice.Channel =
- Controller->V2.NewPhysicalDeviceInformation->Channel;
- CommandMailbox->PhysicalDeviceInfo.IOCTL_Opcode =
- DAC960_V2_GetPhysicalDeviceInfoValid;
- CommandMailbox->PhysicalDeviceInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer =
- Controller->V2.NewPhysicalDeviceInformationDMA;
- CommandMailbox->PhysicalDeviceInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->PhysicalDeviceInfo.DataTransferSize;
- DAC960_QueueCommand(Command);
- return;
- }
- if (Controller->V2.NeedLogicalDeviceInformation)
- {
- if (Controller->V2.StartLogicalDeviceInformationScan)
- {
- int LogicalDriveNumber;
- for (LogicalDriveNumber = 0;
- LogicalDriveNumber < DAC960_MaxLogicalDrives;
- LogicalDriveNumber++)
- Controller->V2.LogicalDriveFoundDuringScan
- [LogicalDriveNumber] = false;
- Controller->V2.NewLogicalDeviceInformation->LogicalDeviceNumber = 0;
- Controller->V2.StartLogicalDeviceInformationScan = false;
- }
- CommandMailbox->LogicalDeviceInfo.CommandOpcode = DAC960_V2_IOCTL;
- CommandMailbox->LogicalDeviceInfo.DataTransferSize =
- sizeof(DAC960_V2_LogicalDeviceInfo_T);
- CommandMailbox->LogicalDeviceInfo.LogicalDevice.LogicalDeviceNumber =
- Controller->V2.NewLogicalDeviceInformation->LogicalDeviceNumber;
- CommandMailbox->LogicalDeviceInfo.IOCTL_Opcode =
- DAC960_V2_GetLogicalDeviceInfoValid;
- CommandMailbox->LogicalDeviceInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer =
- Controller->V2.NewLogicalDeviceInformationDMA;
- CommandMailbox->LogicalDeviceInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->LogicalDeviceInfo.DataTransferSize;
- DAC960_QueueCommand(Command);
- return;
- }
- Controller->MonitoringTimerCount++;
- Controller->MonitoringTimer.expires =
- jiffies + DAC960_HealthStatusMonitoringInterval;
- add_timer(&Controller->MonitoringTimer);
- }
- if (CommandType == DAC960_ImmediateCommand)
- {
- complete(Command->Completion);
- Command->Completion = NULL;
- return;
- }
- if (CommandType == DAC960_QueuedCommand)
- {
- DAC960_V2_KernelCommand_T *KernelCommand = Command->V2.KernelCommand;
- KernelCommand->CommandStatus = CommandStatus;
- KernelCommand->RequestSenseLength = Command->V2.RequestSenseLength;
- KernelCommand->DataTransferLength = Command->V2.DataTransferResidue;
- Command->V2.KernelCommand = NULL;
- DAC960_DeallocateCommand(Command);
- KernelCommand->CompletionFunction(KernelCommand);
- return;
- }
- /*
- Queue a Status Monitoring Command to the Controller using the just
- completed Command if one was deferred previously due to lack of a
- free Command when the Monitoring Timer Function was called.
- */
- if (Controller->MonitoringCommandDeferred)
- {
- Controller->MonitoringCommandDeferred = false;
- DAC960_V2_QueueMonitoringCommand(Command);
- return;
- }
- /*
- Deallocate the Command.
- */
- DAC960_DeallocateCommand(Command);
- /*
- Wake up any processes waiting on a free Command.
- */
- wake_up(&Controller->CommandWaitQueue);
-}
-
-/*
- DAC960_GEM_InterruptHandler handles hardware interrupts from DAC960 GEM Series
- Controllers.
-*/
-
-static irqreturn_t DAC960_GEM_InterruptHandler(int IRQ_Channel,
- void *DeviceIdentifier)
-{
- DAC960_Controller_T *Controller = DeviceIdentifier;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V2_StatusMailbox_T *NextStatusMailbox;
- unsigned long flags;
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- DAC960_GEM_AcknowledgeInterrupt(ControllerBaseAddress);
- NextStatusMailbox = Controller->V2.NextStatusMailbox;
- while (NextStatusMailbox->Fields.CommandIdentifier > 0)
- {
- DAC960_V2_CommandIdentifier_T CommandIdentifier =
- NextStatusMailbox->Fields.CommandIdentifier;
- DAC960_Command_T *Command = Controller->Commands[CommandIdentifier-1];
- Command->V2.CommandStatus = NextStatusMailbox->Fields.CommandStatus;
- Command->V2.RequestSenseLength =
- NextStatusMailbox->Fields.RequestSenseLength;
- Command->V2.DataTransferResidue =
- NextStatusMailbox->Fields.DataTransferResidue;
- NextStatusMailbox->Words[0] = 0;
- if (++NextStatusMailbox > Controller->V2.LastStatusMailbox)
- NextStatusMailbox = Controller->V2.FirstStatusMailbox;
- DAC960_V2_ProcessCompletedCommand(Command);
- }
- Controller->V2.NextStatusMailbox = NextStatusMailbox;
- /*
- Attempt to remove additional I/O Requests from the Controller's
- I/O Request Queue and queue them to the Controller.
- */
- DAC960_ProcessRequest(Controller);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- return IRQ_HANDLED;
-}
-
-/*
- DAC960_BA_InterruptHandler handles hardware interrupts from DAC960 BA Series
- Controllers.
-*/
-
-static irqreturn_t DAC960_BA_InterruptHandler(int IRQ_Channel,
- void *DeviceIdentifier)
-{
- DAC960_Controller_T *Controller = DeviceIdentifier;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V2_StatusMailbox_T *NextStatusMailbox;
- unsigned long flags;
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- DAC960_BA_AcknowledgeInterrupt(ControllerBaseAddress);
- NextStatusMailbox = Controller->V2.NextStatusMailbox;
- while (NextStatusMailbox->Fields.CommandIdentifier > 0)
- {
- DAC960_V2_CommandIdentifier_T CommandIdentifier =
- NextStatusMailbox->Fields.CommandIdentifier;
- DAC960_Command_T *Command = Controller->Commands[CommandIdentifier-1];
- Command->V2.CommandStatus = NextStatusMailbox->Fields.CommandStatus;
- Command->V2.RequestSenseLength =
- NextStatusMailbox->Fields.RequestSenseLength;
- Command->V2.DataTransferResidue =
- NextStatusMailbox->Fields.DataTransferResidue;
- NextStatusMailbox->Words[0] = 0;
- if (++NextStatusMailbox > Controller->V2.LastStatusMailbox)
- NextStatusMailbox = Controller->V2.FirstStatusMailbox;
- DAC960_V2_ProcessCompletedCommand(Command);
- }
- Controller->V2.NextStatusMailbox = NextStatusMailbox;
- /*
- Attempt to remove additional I/O Requests from the Controller's
- I/O Request Queue and queue them to the Controller.
- */
- DAC960_ProcessRequest(Controller);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- return IRQ_HANDLED;
-}
-
-
-/*
- DAC960_LP_InterruptHandler handles hardware interrupts from DAC960 LP Series
- Controllers.
-*/
-
-static irqreturn_t DAC960_LP_InterruptHandler(int IRQ_Channel,
- void *DeviceIdentifier)
-{
- DAC960_Controller_T *Controller = DeviceIdentifier;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V2_StatusMailbox_T *NextStatusMailbox;
- unsigned long flags;
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- DAC960_LP_AcknowledgeInterrupt(ControllerBaseAddress);
- NextStatusMailbox = Controller->V2.NextStatusMailbox;
- while (NextStatusMailbox->Fields.CommandIdentifier > 0)
- {
- DAC960_V2_CommandIdentifier_T CommandIdentifier =
- NextStatusMailbox->Fields.CommandIdentifier;
- DAC960_Command_T *Command = Controller->Commands[CommandIdentifier-1];
- Command->V2.CommandStatus = NextStatusMailbox->Fields.CommandStatus;
- Command->V2.RequestSenseLength =
- NextStatusMailbox->Fields.RequestSenseLength;
- Command->V2.DataTransferResidue =
- NextStatusMailbox->Fields.DataTransferResidue;
- NextStatusMailbox->Words[0] = 0;
- if (++NextStatusMailbox > Controller->V2.LastStatusMailbox)
- NextStatusMailbox = Controller->V2.FirstStatusMailbox;
- DAC960_V2_ProcessCompletedCommand(Command);
- }
- Controller->V2.NextStatusMailbox = NextStatusMailbox;
- /*
- Attempt to remove additional I/O Requests from the Controller's
- I/O Request Queue and queue them to the Controller.
- */
- DAC960_ProcessRequest(Controller);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- return IRQ_HANDLED;
-}
-
-
-/*
- DAC960_LA_InterruptHandler handles hardware interrupts from DAC960 LA Series
- Controllers.
-*/
-
-static irqreturn_t DAC960_LA_InterruptHandler(int IRQ_Channel,
- void *DeviceIdentifier)
-{
- DAC960_Controller_T *Controller = DeviceIdentifier;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V1_StatusMailbox_T *NextStatusMailbox;
- unsigned long flags;
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- DAC960_LA_AcknowledgeInterrupt(ControllerBaseAddress);
- NextStatusMailbox = Controller->V1.NextStatusMailbox;
- while (NextStatusMailbox->Fields.Valid)
- {
- DAC960_V1_CommandIdentifier_T CommandIdentifier =
- NextStatusMailbox->Fields.CommandIdentifier;
- DAC960_Command_T *Command = Controller->Commands[CommandIdentifier-1];
- Command->V1.CommandStatus = NextStatusMailbox->Fields.CommandStatus;
- NextStatusMailbox->Word = 0;
- if (++NextStatusMailbox > Controller->V1.LastStatusMailbox)
- NextStatusMailbox = Controller->V1.FirstStatusMailbox;
- DAC960_V1_ProcessCompletedCommand(Command);
- }
- Controller->V1.NextStatusMailbox = NextStatusMailbox;
- /*
- Attempt to remove additional I/O Requests from the Controller's
- I/O Request Queue and queue them to the Controller.
- */
- DAC960_ProcessRequest(Controller);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- return IRQ_HANDLED;
-}
-
-
-/*
- DAC960_PG_InterruptHandler handles hardware interrupts from DAC960 PG Series
- Controllers.
-*/
-
-static irqreturn_t DAC960_PG_InterruptHandler(int IRQ_Channel,
- void *DeviceIdentifier)
-{
- DAC960_Controller_T *Controller = DeviceIdentifier;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- DAC960_V1_StatusMailbox_T *NextStatusMailbox;
- unsigned long flags;
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- DAC960_PG_AcknowledgeInterrupt(ControllerBaseAddress);
- NextStatusMailbox = Controller->V1.NextStatusMailbox;
- while (NextStatusMailbox->Fields.Valid)
- {
- DAC960_V1_CommandIdentifier_T CommandIdentifier =
- NextStatusMailbox->Fields.CommandIdentifier;
- DAC960_Command_T *Command = Controller->Commands[CommandIdentifier-1];
- Command->V1.CommandStatus = NextStatusMailbox->Fields.CommandStatus;
- NextStatusMailbox->Word = 0;
- if (++NextStatusMailbox > Controller->V1.LastStatusMailbox)
- NextStatusMailbox = Controller->V1.FirstStatusMailbox;
- DAC960_V1_ProcessCompletedCommand(Command);
- }
- Controller->V1.NextStatusMailbox = NextStatusMailbox;
- /*
- Attempt to remove additional I/O Requests from the Controller's
- I/O Request Queue and queue them to the Controller.
- */
- DAC960_ProcessRequest(Controller);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- return IRQ_HANDLED;
-}
-
-
-/*
- DAC960_PD_InterruptHandler handles hardware interrupts from DAC960 PD Series
- Controllers.
-*/
-
-static irqreturn_t DAC960_PD_InterruptHandler(int IRQ_Channel,
- void *DeviceIdentifier)
-{
- DAC960_Controller_T *Controller = DeviceIdentifier;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- unsigned long flags;
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- while (DAC960_PD_StatusAvailableP(ControllerBaseAddress))
- {
- DAC960_V1_CommandIdentifier_T CommandIdentifier =
- DAC960_PD_ReadStatusCommandIdentifier(ControllerBaseAddress);
- DAC960_Command_T *Command = Controller->Commands[CommandIdentifier-1];
- Command->V1.CommandStatus =
- DAC960_PD_ReadStatusRegister(ControllerBaseAddress);
- DAC960_PD_AcknowledgeInterrupt(ControllerBaseAddress);
- DAC960_PD_AcknowledgeStatus(ControllerBaseAddress);
- DAC960_V1_ProcessCompletedCommand(Command);
- }
- /*
- Attempt to remove additional I/O Requests from the Controller's
- I/O Request Queue and queue them to the Controller.
- */
- DAC960_ProcessRequest(Controller);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- return IRQ_HANDLED;
-}
-
-
-/*
- DAC960_P_InterruptHandler handles hardware interrupts from DAC960 P Series
- Controllers.
-
- Translations of DAC960_V1_Enquiry and DAC960_V1_GetDeviceState rely
- on the data having been placed into DAC960_Controller_T, rather than
- an arbitrary buffer.
-*/
-
-static irqreturn_t DAC960_P_InterruptHandler(int IRQ_Channel,
- void *DeviceIdentifier)
-{
- DAC960_Controller_T *Controller = DeviceIdentifier;
- void __iomem *ControllerBaseAddress = Controller->BaseAddress;
- unsigned long flags;
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- while (DAC960_PD_StatusAvailableP(ControllerBaseAddress))
- {
- DAC960_V1_CommandIdentifier_T CommandIdentifier =
- DAC960_PD_ReadStatusCommandIdentifier(ControllerBaseAddress);
- DAC960_Command_T *Command = Controller->Commands[CommandIdentifier-1];
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- DAC960_V1_CommandOpcode_T CommandOpcode =
- CommandMailbox->Common.CommandOpcode;
- Command->V1.CommandStatus =
- DAC960_PD_ReadStatusRegister(ControllerBaseAddress);
- DAC960_PD_AcknowledgeInterrupt(ControllerBaseAddress);
- DAC960_PD_AcknowledgeStatus(ControllerBaseAddress);
- switch (CommandOpcode)
- {
- case DAC960_V1_Enquiry_Old:
- Command->V1.CommandMailbox.Common.CommandOpcode = DAC960_V1_Enquiry;
- DAC960_P_To_PD_TranslateEnquiry(Controller->V1.NewEnquiry);
- break;
- case DAC960_V1_GetDeviceState_Old:
- Command->V1.CommandMailbox.Common.CommandOpcode =
- DAC960_V1_GetDeviceState;
- DAC960_P_To_PD_TranslateDeviceState(Controller->V1.NewDeviceState);
- break;
- case DAC960_V1_Read_Old:
- Command->V1.CommandMailbox.Common.CommandOpcode = DAC960_V1_Read;
- DAC960_P_To_PD_TranslateReadWriteCommand(CommandMailbox);
- break;
- case DAC960_V1_Write_Old:
- Command->V1.CommandMailbox.Common.CommandOpcode = DAC960_V1_Write;
- DAC960_P_To_PD_TranslateReadWriteCommand(CommandMailbox);
- break;
- case DAC960_V1_ReadWithScatterGather_Old:
- Command->V1.CommandMailbox.Common.CommandOpcode =
- DAC960_V1_ReadWithScatterGather;
- DAC960_P_To_PD_TranslateReadWriteCommand(CommandMailbox);
- break;
- case DAC960_V1_WriteWithScatterGather_Old:
- Command->V1.CommandMailbox.Common.CommandOpcode =
- DAC960_V1_WriteWithScatterGather;
- DAC960_P_To_PD_TranslateReadWriteCommand(CommandMailbox);
- break;
- default:
- break;
- }
- DAC960_V1_ProcessCompletedCommand(Command);
- }
- /*
- Attempt to remove additional I/O Requests from the Controller's
- I/O Request Queue and queue them to the Controller.
- */
- DAC960_ProcessRequest(Controller);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- return IRQ_HANDLED;
-}
-
-
-/*
- DAC960_V1_QueueMonitoringCommand queues a Monitoring Command to DAC960 V1
- Firmware Controllers.
-*/
-
-static void DAC960_V1_QueueMonitoringCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- DAC960_V1_ClearCommand(Command);
- Command->CommandType = DAC960_MonitoringCommand;
- CommandMailbox->Type3.CommandOpcode = DAC960_V1_Enquiry;
- CommandMailbox->Type3.BusAddress = Controller->V1.NewEnquiryDMA;
- DAC960_QueueCommand(Command);
-}
-
-
-/*
- DAC960_V2_QueueMonitoringCommand queues a Monitoring Command to DAC960 V2
- Firmware Controllers.
-*/
-
-static void DAC960_V2_QueueMonitoringCommand(DAC960_Command_T *Command)
-{
- DAC960_Controller_T *Controller = Command->Controller;
- DAC960_V2_CommandMailbox_T *CommandMailbox = &Command->V2.CommandMailbox;
- DAC960_V2_ClearCommand(Command);
- Command->CommandType = DAC960_MonitoringCommand;
- CommandMailbox->ControllerInfo.CommandOpcode = DAC960_V2_IOCTL;
- CommandMailbox->ControllerInfo.CommandControlBits
- .DataTransferControllerToHost = true;
- CommandMailbox->ControllerInfo.CommandControlBits
- .NoAutoRequestSense = true;
- CommandMailbox->ControllerInfo.DataTransferSize =
- sizeof(DAC960_V2_ControllerInfo_T);
- CommandMailbox->ControllerInfo.ControllerNumber = 0;
- CommandMailbox->ControllerInfo.IOCTL_Opcode = DAC960_V2_GetControllerInfo;
- CommandMailbox->ControllerInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer =
- Controller->V2.NewControllerInformationDMA;
- CommandMailbox->ControllerInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->ControllerInfo.DataTransferSize;
- DAC960_QueueCommand(Command);
-}
-
-
-/*
- DAC960_MonitoringTimerFunction is the timer function for monitoring
- the status of DAC960 Controllers.
-*/
-
-static void DAC960_MonitoringTimerFunction(struct timer_list *t)
-{
- DAC960_Controller_T *Controller = from_timer(Controller, t, MonitoringTimer);
- DAC960_Command_T *Command;
- unsigned long flags;
-
- if (Controller->FirmwareType == DAC960_V1_Controller)
- {
- spin_lock_irqsave(&Controller->queue_lock, flags);
- /*
- Queue a Status Monitoring Command to Controller.
- */
- Command = DAC960_AllocateCommand(Controller);
- if (Command != NULL)
- DAC960_V1_QueueMonitoringCommand(Command);
- else Controller->MonitoringCommandDeferred = true;
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- }
- else
- {
- DAC960_V2_ControllerInfo_T *ControllerInfo =
- &Controller->V2.ControllerInformation;
- unsigned int StatusChangeCounter =
- Controller->V2.HealthStatusBuffer->StatusChangeCounter;
- bool ForceMonitoringCommand = false;
- if (time_after(jiffies, Controller->SecondaryMonitoringTime
- + DAC960_SecondaryMonitoringInterval))
- {
- int LogicalDriveNumber;
- for (LogicalDriveNumber = 0;
- LogicalDriveNumber < DAC960_MaxLogicalDrives;
- LogicalDriveNumber++)
- {
- DAC960_V2_LogicalDeviceInfo_T *LogicalDeviceInfo =
- Controller->V2.LogicalDeviceInformation[LogicalDriveNumber];
- if (LogicalDeviceInfo == NULL) continue;
- if (!LogicalDeviceInfo->LogicalDeviceControl
- .LogicalDeviceInitialized)
- {
- ForceMonitoringCommand = true;
- break;
- }
- }
- Controller->SecondaryMonitoringTime = jiffies;
- }
- if (StatusChangeCounter == Controller->V2.StatusChangeCounter &&
- Controller->V2.HealthStatusBuffer->NextEventSequenceNumber
- == Controller->V2.NextEventSequenceNumber &&
- (ControllerInfo->BackgroundInitializationsActive +
- ControllerInfo->LogicalDeviceInitializationsActive +
- ControllerInfo->PhysicalDeviceInitializationsActive +
- ControllerInfo->ConsistencyChecksActive +
- ControllerInfo->RebuildsActive +
- ControllerInfo->OnlineExpansionsActive == 0 ||
- time_before(jiffies, Controller->PrimaryMonitoringTime
- + DAC960_MonitoringTimerInterval)) &&
- !ForceMonitoringCommand)
- {
- Controller->MonitoringTimer.expires =
- jiffies + DAC960_HealthStatusMonitoringInterval;
- add_timer(&Controller->MonitoringTimer);
- return;
- }
- Controller->V2.StatusChangeCounter = StatusChangeCounter;
- Controller->PrimaryMonitoringTime = jiffies;
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- /*
- Queue a Status Monitoring Command to Controller.
- */
- Command = DAC960_AllocateCommand(Controller);
- if (Command != NULL)
- DAC960_V2_QueueMonitoringCommand(Command);
- else Controller->MonitoringCommandDeferred = true;
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- /*
- Wake up any processes waiting on a Health Status Buffer change.
- */
- wake_up(&Controller->HealthStatusWaitQueue);
- }
-}
-
-/*
- DAC960_CheckStatusBuffer verifies that there is room to hold ByteCount
- additional bytes in the Combined Status Buffer and grows the buffer if
- necessary. It returns true if there is enough room and false otherwise.
-*/
-
-static bool DAC960_CheckStatusBuffer(DAC960_Controller_T *Controller,
- unsigned int ByteCount)
-{
- unsigned char *NewStatusBuffer;
- if (Controller->InitialStatusLength + 1 +
- Controller->CurrentStatusLength + ByteCount + 1 <=
- Controller->CombinedStatusBufferLength)
- return true;
- if (Controller->CombinedStatusBufferLength == 0)
- {
- unsigned int NewStatusBufferLength = DAC960_InitialStatusBufferSize;
- while (NewStatusBufferLength < ByteCount)
- NewStatusBufferLength *= 2;
- Controller->CombinedStatusBuffer = kmalloc(NewStatusBufferLength,
- GFP_ATOMIC);
- if (Controller->CombinedStatusBuffer == NULL) return false;
- Controller->CombinedStatusBufferLength = NewStatusBufferLength;
- return true;
- }
- NewStatusBuffer = kmalloc_array(2, Controller->CombinedStatusBufferLength,
- GFP_ATOMIC);
- if (NewStatusBuffer == NULL)
- {
- DAC960_Warning("Unable to expand Combined Status Buffer - Truncating\n",
- Controller);
- return false;
- }
- memcpy(NewStatusBuffer, Controller->CombinedStatusBuffer,
- Controller->CombinedStatusBufferLength);
- kfree(Controller->CombinedStatusBuffer);
- Controller->CombinedStatusBuffer = NewStatusBuffer;
- Controller->CombinedStatusBufferLength *= 2;
- Controller->CurrentStatusBuffer =
- &NewStatusBuffer[Controller->InitialStatusLength + 1];
- return true;
-}
-
-
-/*
- DAC960_Message prints Driver Messages.
-*/
-
-static void DAC960_Message(DAC960_MessageLevel_T MessageLevel,
- unsigned char *Format,
- DAC960_Controller_T *Controller,
- ...)
-{
- static unsigned char Buffer[DAC960_LineBufferSize];
- static bool BeginningOfLine = true;
- va_list Arguments;
- int Length = 0;
- va_start(Arguments, Controller);
- Length = vsprintf(Buffer, Format, Arguments);
- va_end(Arguments);
- if (Controller == NULL)
- printk("%sDAC960#%d: %s", DAC960_MessageLevelMap[MessageLevel],
- DAC960_ControllerCount, Buffer);
- else if (MessageLevel == DAC960_AnnounceLevel ||
- MessageLevel == DAC960_InfoLevel)
- {
- if (!Controller->ControllerInitialized)
- {
- if (DAC960_CheckStatusBuffer(Controller, Length))
- {
- strcpy(&Controller->CombinedStatusBuffer
- [Controller->InitialStatusLength],
- Buffer);
- Controller->InitialStatusLength += Length;
- Controller->CurrentStatusBuffer =
- &Controller->CombinedStatusBuffer
- [Controller->InitialStatusLength + 1];
- }
- if (MessageLevel == DAC960_AnnounceLevel)
- {
- static int AnnouncementLines = 0;
- if (++AnnouncementLines <= 2)
- printk("%sDAC960: %s", DAC960_MessageLevelMap[MessageLevel],
- Buffer);
- }
- else
- {
- if (BeginningOfLine)
- {
- if (Buffer[0] != '\n' || Length > 1)
- printk("%sDAC960#%d: %s",
- DAC960_MessageLevelMap[MessageLevel],
- Controller->ControllerNumber, Buffer);
- }
- else printk("%s", Buffer);
- }
- }
- else if (DAC960_CheckStatusBuffer(Controller, Length))
- {
- strcpy(&Controller->CurrentStatusBuffer[
- Controller->CurrentStatusLength], Buffer);
- Controller->CurrentStatusLength += Length;
- }
- }
- else if (MessageLevel == DAC960_ProgressLevel)
- {
- strcpy(Controller->ProgressBuffer, Buffer);
- Controller->ProgressBufferLength = Length;
- if (Controller->EphemeralProgressMessage)
- {
- if (time_after_eq(jiffies, Controller->LastProgressReportTime
- + DAC960_ProgressReportingInterval))
- {
- printk("%sDAC960#%d: %s", DAC960_MessageLevelMap[MessageLevel],
- Controller->ControllerNumber, Buffer);
- Controller->LastProgressReportTime = jiffies;
- }
- }
- else printk("%sDAC960#%d: %s", DAC960_MessageLevelMap[MessageLevel],
- Controller->ControllerNumber, Buffer);
- }
- else if (MessageLevel == DAC960_UserCriticalLevel)
- {
- strcpy(&Controller->UserStatusBuffer[Controller->UserStatusLength],
- Buffer);
- Controller->UserStatusLength += Length;
- if (Buffer[0] != '\n' || Length > 1)
- printk("%sDAC960#%d: %s", DAC960_MessageLevelMap[MessageLevel],
- Controller->ControllerNumber, Buffer);
- }
- else
- {
- if (BeginningOfLine)
- printk("%sDAC960#%d: %s", DAC960_MessageLevelMap[MessageLevel],
- Controller->ControllerNumber, Buffer);
- else printk("%s", Buffer);
- }
- BeginningOfLine = (Buffer[Length-1] == '\n');
-}
-
-
-/*
- DAC960_ParsePhysicalDevice parses spaces followed by a Physical Device
- Channel:TargetID specification from a User Command string. It updates
- Channel and TargetID and returns true on success and false on failure.
-*/
-
-static bool DAC960_ParsePhysicalDevice(DAC960_Controller_T *Controller,
- char *UserCommandString,
- unsigned char *Channel,
- unsigned char *TargetID)
-{
- char *NewUserCommandString = UserCommandString;
- unsigned long XChannel, XTargetID;
- while (*UserCommandString == ' ') UserCommandString++;
- if (UserCommandString == NewUserCommandString)
- return false;
- XChannel = simple_strtoul(UserCommandString, &NewUserCommandString, 10);
- if (NewUserCommandString == UserCommandString ||
- *NewUserCommandString != ':' ||
- XChannel >= Controller->Channels)
- return false;
- UserCommandString = ++NewUserCommandString;
- XTargetID = simple_strtoul(UserCommandString, &NewUserCommandString, 10);
- if (NewUserCommandString == UserCommandString ||
- *NewUserCommandString != '\0' ||
- XTargetID >= Controller->Targets)
- return false;
- *Channel = XChannel;
- *TargetID = XTargetID;
- return true;
-}
-
-
-/*
- DAC960_ParseLogicalDrive parses spaces followed by a Logical Drive Number
- specification from a User Command string. It updates LogicalDriveNumber and
- returns true on success and false on failure.
-*/
-
-static bool DAC960_ParseLogicalDrive(DAC960_Controller_T *Controller,
- char *UserCommandString,
- unsigned char *LogicalDriveNumber)
-{
- char *NewUserCommandString = UserCommandString;
- unsigned long XLogicalDriveNumber;
- while (*UserCommandString == ' ') UserCommandString++;
- if (UserCommandString == NewUserCommandString)
- return false;
- XLogicalDriveNumber =
- simple_strtoul(UserCommandString, &NewUserCommandString, 10);
- if (NewUserCommandString == UserCommandString ||
- *NewUserCommandString != '\0' ||
- XLogicalDriveNumber > DAC960_MaxLogicalDrives - 1)
- return false;
- *LogicalDriveNumber = XLogicalDriveNumber;
- return true;
-}
-
-
-/*
- DAC960_V1_SetDeviceState sets the Device State for a Physical Device for
- DAC960 V1 Firmware Controllers.
-*/
-
-static void DAC960_V1_SetDeviceState(DAC960_Controller_T *Controller,
- DAC960_Command_T *Command,
- unsigned char Channel,
- unsigned char TargetID,
- DAC960_V1_PhysicalDeviceState_T
- DeviceState,
- const unsigned char *DeviceStateString)
-{
- DAC960_V1_CommandMailbox_T *CommandMailbox = &Command->V1.CommandMailbox;
- CommandMailbox->Type3D.CommandOpcode = DAC960_V1_StartDevice;
- CommandMailbox->Type3D.Channel = Channel;
- CommandMailbox->Type3D.TargetID = TargetID;
- CommandMailbox->Type3D.DeviceState = DeviceState;
- CommandMailbox->Type3D.Modifier = 0;
- DAC960_ExecuteCommand(Command);
- switch (Command->V1.CommandStatus)
- {
- case DAC960_V1_NormalCompletion:
- DAC960_UserCritical("%s of Physical Device %d:%d Succeeded\n", Controller,
- DeviceStateString, Channel, TargetID);
- break;
- case DAC960_V1_UnableToStartDevice:
- DAC960_UserCritical("%s of Physical Device %d:%d Failed - "
- "Unable to Start Device\n", Controller,
- DeviceStateString, Channel, TargetID);
- break;
- case DAC960_V1_NoDeviceAtAddress:
- DAC960_UserCritical("%s of Physical Device %d:%d Failed - "
- "No Device at Address\n", Controller,
- DeviceStateString, Channel, TargetID);
- break;
- case DAC960_V1_InvalidChannelOrTargetOrModifier:
- DAC960_UserCritical("%s of Physical Device %d:%d Failed - "
- "Invalid Channel or Target or Modifier\n",
- Controller, DeviceStateString, Channel, TargetID);
- break;
- case DAC960_V1_ChannelBusy:
- DAC960_UserCritical("%s of Physical Device %d:%d Failed - "
- "Channel Busy\n", Controller,
- DeviceStateString, Channel, TargetID);
- break;
- default:
- DAC960_UserCritical("%s of Physical Device %d:%d Failed - "
- "Unexpected Status %04X\n", Controller,
- DeviceStateString, Channel, TargetID,
- Command->V1.CommandStatus);
- break;
- }
-}
-
-
-/*
- DAC960_V1_ExecuteUserCommand executes a User Command for DAC960 V1 Firmware
- Controllers.
-*/
-
-static bool DAC960_V1_ExecuteUserCommand(DAC960_Controller_T *Controller,
- unsigned char *UserCommand)
-{
- DAC960_Command_T *Command;
- DAC960_V1_CommandMailbox_T *CommandMailbox;
- unsigned long flags;
- unsigned char Channel, TargetID, LogicalDriveNumber;
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- while ((Command = DAC960_AllocateCommand(Controller)) == NULL)
- DAC960_WaitForCommand(Controller);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- Controller->UserStatusLength = 0;
- DAC960_V1_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- CommandMailbox = &Command->V1.CommandMailbox;
- if (strcmp(UserCommand, "flush-cache") == 0)
- {
- CommandMailbox->Type3.CommandOpcode = DAC960_V1_Flush;
- DAC960_ExecuteCommand(Command);
- DAC960_UserCritical("Cache Flush Completed\n", Controller);
- }
- else if (strncmp(UserCommand, "kill", 4) == 0 &&
- DAC960_ParsePhysicalDevice(Controller, &UserCommand[4],
- &Channel, &TargetID))
- {
- DAC960_V1_DeviceState_T *DeviceState =
- &Controller->V1.DeviceState[Channel][TargetID];
- if (DeviceState->Present &&
- DeviceState->DeviceType == DAC960_V1_DiskType &&
- DeviceState->DeviceState != DAC960_V1_Device_Dead)
- DAC960_V1_SetDeviceState(Controller, Command, Channel, TargetID,
- DAC960_V1_Device_Dead, "Kill");
- else DAC960_UserCritical("Kill of Physical Device %d:%d Illegal\n",
- Controller, Channel, TargetID);
- }
- else if (strncmp(UserCommand, "make-online", 11) == 0 &&
- DAC960_ParsePhysicalDevice(Controller, &UserCommand[11],
- &Channel, &TargetID))
- {
- DAC960_V1_DeviceState_T *DeviceState =
- &Controller->V1.DeviceState[Channel][TargetID];
- if (DeviceState->Present &&
- DeviceState->DeviceType == DAC960_V1_DiskType &&
- DeviceState->DeviceState == DAC960_V1_Device_Dead)
- DAC960_V1_SetDeviceState(Controller, Command, Channel, TargetID,
- DAC960_V1_Device_Online, "Make Online");
- else DAC960_UserCritical("Make Online of Physical Device %d:%d Illegal\n",
- Controller, Channel, TargetID);
-
- }
- else if (strncmp(UserCommand, "make-standby", 12) == 0 &&
- DAC960_ParsePhysicalDevice(Controller, &UserCommand[12],
- &Channel, &TargetID))
- {
- DAC960_V1_DeviceState_T *DeviceState =
- &Controller->V1.DeviceState[Channel][TargetID];
- if (DeviceState->Present &&
- DeviceState->DeviceType == DAC960_V1_DiskType &&
- DeviceState->DeviceState == DAC960_V1_Device_Dead)
- DAC960_V1_SetDeviceState(Controller, Command, Channel, TargetID,
- DAC960_V1_Device_Standby, "Make Standby");
- else DAC960_UserCritical("Make Standby of Physical "
- "Device %d:%d Illegal\n",
- Controller, Channel, TargetID);
- }
- else if (strncmp(UserCommand, "rebuild", 7) == 0 &&
- DAC960_ParsePhysicalDevice(Controller, &UserCommand[7],
- &Channel, &TargetID))
- {
- CommandMailbox->Type3D.CommandOpcode = DAC960_V1_RebuildAsync;
- CommandMailbox->Type3D.Channel = Channel;
- CommandMailbox->Type3D.TargetID = TargetID;
- DAC960_ExecuteCommand(Command);
- switch (Command->V1.CommandStatus)
- {
- case DAC960_V1_NormalCompletion:
- DAC960_UserCritical("Rebuild of Physical Device %d:%d Initiated\n",
- Controller, Channel, TargetID);
- break;
- case DAC960_V1_AttemptToRebuildOnlineDrive:
- DAC960_UserCritical("Rebuild of Physical Device %d:%d Failed - "
- "Attempt to Rebuild Online or "
- "Unresponsive Drive\n",
- Controller, Channel, TargetID);
- break;
- case DAC960_V1_NewDiskFailedDuringRebuild:
- DAC960_UserCritical("Rebuild of Physical Device %d:%d Failed - "
- "New Disk Failed During Rebuild\n",
- Controller, Channel, TargetID);
- break;
- case DAC960_V1_InvalidDeviceAddress:
- DAC960_UserCritical("Rebuild of Physical Device %d:%d Failed - "
- "Invalid Device Address\n",
- Controller, Channel, TargetID);
- break;
- case DAC960_V1_RebuildOrCheckAlreadyInProgress:
- DAC960_UserCritical("Rebuild of Physical Device %d:%d Failed - "
- "Rebuild or Consistency Check Already "
- "in Progress\n", Controller, Channel, TargetID);
- break;
- default:
- DAC960_UserCritical("Rebuild of Physical Device %d:%d Failed - "
- "Unexpected Status %04X\n", Controller,
- Channel, TargetID, Command->V1.CommandStatus);
- break;
- }
- }
- else if (strncmp(UserCommand, "check-consistency", 17) == 0 &&
- DAC960_ParseLogicalDrive(Controller, &UserCommand[17],
- &LogicalDriveNumber))
- {
- CommandMailbox->Type3C.CommandOpcode = DAC960_V1_CheckConsistencyAsync;
- CommandMailbox->Type3C.LogicalDriveNumber = LogicalDriveNumber;
- CommandMailbox->Type3C.AutoRestore = true;
- DAC960_ExecuteCommand(Command);
- switch (Command->V1.CommandStatus)
- {
- case DAC960_V1_NormalCompletion:
- DAC960_UserCritical("Consistency Check of Logical Drive %d "
- "(/dev/rd/c%dd%d) Initiated\n",
- Controller, LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber);
- break;
- case DAC960_V1_DependentDiskIsDead:
- DAC960_UserCritical("Consistency Check of Logical Drive %d "
- "(/dev/rd/c%dd%d) Failed - "
- "Dependent Physical Device is DEAD\n",
- Controller, LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber);
- break;
- case DAC960_V1_InvalidOrNonredundantLogicalDrive:
- DAC960_UserCritical("Consistency Check of Logical Drive %d "
- "(/dev/rd/c%dd%d) Failed - "
- "Invalid or Nonredundant Logical Drive\n",
- Controller, LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber);
- break;
- case DAC960_V1_RebuildOrCheckAlreadyInProgress:
- DAC960_UserCritical("Consistency Check of Logical Drive %d "
- "(/dev/rd/c%dd%d) Failed - Rebuild or "
- "Consistency Check Already in Progress\n",
- Controller, LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber);
- break;
- default:
- DAC960_UserCritical("Consistency Check of Logical Drive %d "
- "(/dev/rd/c%dd%d) Failed - "
- "Unexpected Status %04X\n",
- Controller, LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber, Command->V1.CommandStatus);
- break;
- }
- }
- else if (strcmp(UserCommand, "cancel-rebuild") == 0 ||
- strcmp(UserCommand, "cancel-consistency-check") == 0)
- {
- /*
- the OldRebuildRateConstant is never actually used
- once its value is retrieved from the controller.
- */
- unsigned char *OldRebuildRateConstant;
- dma_addr_t OldRebuildRateConstantDMA;
-
- OldRebuildRateConstant = pci_alloc_consistent( Controller->PCIDevice,
- sizeof(char), &OldRebuildRateConstantDMA);
- if (OldRebuildRateConstant == NULL) {
- DAC960_UserCritical("Cancellation of Rebuild or "
- "Consistency Check Failed - "
- "Out of Memory",
- Controller);
- goto failure;
- }
- CommandMailbox->Type3R.CommandOpcode = DAC960_V1_RebuildControl;
- CommandMailbox->Type3R.RebuildRateConstant = 0xFF;
- CommandMailbox->Type3R.BusAddress = OldRebuildRateConstantDMA;
- DAC960_ExecuteCommand(Command);
- switch (Command->V1.CommandStatus)
- {
- case DAC960_V1_NormalCompletion:
- DAC960_UserCritical("Rebuild or Consistency Check Cancelled\n",
- Controller);
- break;
- default:
- DAC960_UserCritical("Cancellation of Rebuild or "
- "Consistency Check Failed - "
- "Unexpected Status %04X\n",
- Controller, Command->V1.CommandStatus);
- break;
- }
-failure:
- pci_free_consistent(Controller->PCIDevice, sizeof(char),
- OldRebuildRateConstant, OldRebuildRateConstantDMA);
- }
- else DAC960_UserCritical("Illegal User Command: '%s'\n",
- Controller, UserCommand);
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- DAC960_DeallocateCommand(Command);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- return true;
-}
-
-
-/*
- DAC960_V2_TranslatePhysicalDevice translates a Physical Device Channel and
- TargetID into a Logical Device. It returns true on success and false
- on failure.
-*/
-
-static bool DAC960_V2_TranslatePhysicalDevice(DAC960_Command_T *Command,
- unsigned char Channel,
- unsigned char TargetID,
- unsigned short
- *LogicalDeviceNumber)
-{
- DAC960_V2_CommandMailbox_T SavedCommandMailbox, *CommandMailbox;
- DAC960_Controller_T *Controller = Command->Controller;
-
- CommandMailbox = &Command->V2.CommandMailbox;
- memcpy(&SavedCommandMailbox, CommandMailbox,
- sizeof(DAC960_V2_CommandMailbox_T));
-
- CommandMailbox->PhysicalDeviceInfo.CommandOpcode = DAC960_V2_IOCTL;
- CommandMailbox->PhysicalDeviceInfo.CommandControlBits
- .DataTransferControllerToHost = true;
- CommandMailbox->PhysicalDeviceInfo.CommandControlBits
- .NoAutoRequestSense = true;
- CommandMailbox->PhysicalDeviceInfo.DataTransferSize =
- sizeof(DAC960_V2_PhysicalToLogicalDevice_T);
- CommandMailbox->PhysicalDeviceInfo.PhysicalDevice.TargetID = TargetID;
- CommandMailbox->PhysicalDeviceInfo.PhysicalDevice.Channel = Channel;
- CommandMailbox->PhysicalDeviceInfo.IOCTL_Opcode =
- DAC960_V2_TranslatePhysicalToLogicalDevice;
- CommandMailbox->Common.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer =
- Controller->V2.PhysicalToLogicalDeviceDMA;
- CommandMailbox->Common.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->Common.DataTransferSize;
-
- DAC960_ExecuteCommand(Command);
- *LogicalDeviceNumber = Controller->V2.PhysicalToLogicalDevice->LogicalDeviceNumber;
-
- memcpy(CommandMailbox, &SavedCommandMailbox,
- sizeof(DAC960_V2_CommandMailbox_T));
- return (Command->V2.CommandStatus == DAC960_V2_NormalCompletion);
-}
-
-
-/*
- DAC960_V2_ExecuteUserCommand executes a User Command for DAC960 V2 Firmware
- Controllers.
-*/
-
-static bool DAC960_V2_ExecuteUserCommand(DAC960_Controller_T *Controller,
- unsigned char *UserCommand)
-{
- DAC960_Command_T *Command;
- DAC960_V2_CommandMailbox_T *CommandMailbox;
- unsigned long flags;
- unsigned char Channel, TargetID, LogicalDriveNumber;
- unsigned short LogicalDeviceNumber;
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- while ((Command = DAC960_AllocateCommand(Controller)) == NULL)
- DAC960_WaitForCommand(Controller);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- Controller->UserStatusLength = 0;
- DAC960_V2_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- CommandMailbox = &Command->V2.CommandMailbox;
- CommandMailbox->Common.CommandOpcode = DAC960_V2_IOCTL;
- CommandMailbox->Common.CommandControlBits.DataTransferControllerToHost = true;
- CommandMailbox->Common.CommandControlBits.NoAutoRequestSense = true;
- if (strcmp(UserCommand, "flush-cache") == 0)
- {
- CommandMailbox->DeviceOperation.IOCTL_Opcode = DAC960_V2_PauseDevice;
- CommandMailbox->DeviceOperation.OperationDevice =
- DAC960_V2_RAID_Controller;
- DAC960_ExecuteCommand(Command);
- DAC960_UserCritical("Cache Flush Completed\n", Controller);
- }
- else if (strncmp(UserCommand, "kill", 4) == 0 &&
- DAC960_ParsePhysicalDevice(Controller, &UserCommand[4],
- &Channel, &TargetID) &&
- DAC960_V2_TranslatePhysicalDevice(Command, Channel, TargetID,
- &LogicalDeviceNumber))
- {
- CommandMailbox->SetDeviceState.LogicalDevice.LogicalDeviceNumber =
- LogicalDeviceNumber;
- CommandMailbox->SetDeviceState.IOCTL_Opcode =
- DAC960_V2_SetDeviceState;
- CommandMailbox->SetDeviceState.DeviceState.PhysicalDeviceState =
- DAC960_V2_Device_Dead;
- DAC960_ExecuteCommand(Command);
- DAC960_UserCritical("Kill of Physical Device %d:%d %s\n",
- Controller, Channel, TargetID,
- (Command->V2.CommandStatus
- == DAC960_V2_NormalCompletion
- ? "Succeeded" : "Failed"));
- }
- else if (strncmp(UserCommand, "make-online", 11) == 0 &&
- DAC960_ParsePhysicalDevice(Controller, &UserCommand[11],
- &Channel, &TargetID) &&
- DAC960_V2_TranslatePhysicalDevice(Command, Channel, TargetID,
- &LogicalDeviceNumber))
- {
- CommandMailbox->SetDeviceState.LogicalDevice.LogicalDeviceNumber =
- LogicalDeviceNumber;
- CommandMailbox->SetDeviceState.IOCTL_Opcode =
- DAC960_V2_SetDeviceState;
- CommandMailbox->SetDeviceState.DeviceState.PhysicalDeviceState =
- DAC960_V2_Device_Online;
- DAC960_ExecuteCommand(Command);
- DAC960_UserCritical("Make Online of Physical Device %d:%d %s\n",
- Controller, Channel, TargetID,
- (Command->V2.CommandStatus
- == DAC960_V2_NormalCompletion
- ? "Succeeded" : "Failed"));
- }
- else if (strncmp(UserCommand, "make-standby", 12) == 0 &&
- DAC960_ParsePhysicalDevice(Controller, &UserCommand[12],
- &Channel, &TargetID) &&
- DAC960_V2_TranslatePhysicalDevice(Command, Channel, TargetID,
- &LogicalDeviceNumber))
- {
- CommandMailbox->SetDeviceState.LogicalDevice.LogicalDeviceNumber =
- LogicalDeviceNumber;
- CommandMailbox->SetDeviceState.IOCTL_Opcode =
- DAC960_V2_SetDeviceState;
- CommandMailbox->SetDeviceState.DeviceState.PhysicalDeviceState =
- DAC960_V2_Device_Standby;
- DAC960_ExecuteCommand(Command);
- DAC960_UserCritical("Make Standby of Physical Device %d:%d %s\n",
- Controller, Channel, TargetID,
- (Command->V2.CommandStatus
- == DAC960_V2_NormalCompletion
- ? "Succeeded" : "Failed"));
- }
- else if (strncmp(UserCommand, "rebuild", 7) == 0 &&
- DAC960_ParsePhysicalDevice(Controller, &UserCommand[7],
- &Channel, &TargetID) &&
- DAC960_V2_TranslatePhysicalDevice(Command, Channel, TargetID,
- &LogicalDeviceNumber))
- {
- CommandMailbox->LogicalDeviceInfo.LogicalDevice.LogicalDeviceNumber =
- LogicalDeviceNumber;
- CommandMailbox->LogicalDeviceInfo.IOCTL_Opcode =
- DAC960_V2_RebuildDeviceStart;
- DAC960_ExecuteCommand(Command);
- DAC960_UserCritical("Rebuild of Physical Device %d:%d %s\n",
- Controller, Channel, TargetID,
- (Command->V2.CommandStatus
- == DAC960_V2_NormalCompletion
- ? "Initiated" : "Not Initiated"));
- }
- else if (strncmp(UserCommand, "cancel-rebuild", 14) == 0 &&
- DAC960_ParsePhysicalDevice(Controller, &UserCommand[14],
- &Channel, &TargetID) &&
- DAC960_V2_TranslatePhysicalDevice(Command, Channel, TargetID,
- &LogicalDeviceNumber))
- {
- CommandMailbox->LogicalDeviceInfo.LogicalDevice.LogicalDeviceNumber =
- LogicalDeviceNumber;
- CommandMailbox->LogicalDeviceInfo.IOCTL_Opcode =
- DAC960_V2_RebuildDeviceStop;
- DAC960_ExecuteCommand(Command);
- DAC960_UserCritical("Rebuild of Physical Device %d:%d %s\n",
- Controller, Channel, TargetID,
- (Command->V2.CommandStatus
- == DAC960_V2_NormalCompletion
- ? "Cancelled" : "Not Cancelled"));
- }
- else if (strncmp(UserCommand, "check-consistency", 17) == 0 &&
- DAC960_ParseLogicalDrive(Controller, &UserCommand[17],
- &LogicalDriveNumber))
- {
- CommandMailbox->ConsistencyCheck.LogicalDevice.LogicalDeviceNumber =
- LogicalDriveNumber;
- CommandMailbox->ConsistencyCheck.IOCTL_Opcode =
- DAC960_V2_ConsistencyCheckStart;
- CommandMailbox->ConsistencyCheck.RestoreConsistency = true;
- CommandMailbox->ConsistencyCheck.InitializedAreaOnly = false;
- DAC960_ExecuteCommand(Command);
- DAC960_UserCritical("Consistency Check of Logical Drive %d "
- "(/dev/rd/c%dd%d) %s\n",
- Controller, LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber,
- (Command->V2.CommandStatus
- == DAC960_V2_NormalCompletion
- ? "Initiated" : "Not Initiated"));
- }
- else if (strncmp(UserCommand, "cancel-consistency-check", 24) == 0 &&
- DAC960_ParseLogicalDrive(Controller, &UserCommand[24],
- &LogicalDriveNumber))
- {
- CommandMailbox->ConsistencyCheck.LogicalDevice.LogicalDeviceNumber =
- LogicalDriveNumber;
- CommandMailbox->ConsistencyCheck.IOCTL_Opcode =
- DAC960_V2_ConsistencyCheckStop;
- DAC960_ExecuteCommand(Command);
- DAC960_UserCritical("Consistency Check of Logical Drive %d "
- "(/dev/rd/c%dd%d) %s\n",
- Controller, LogicalDriveNumber,
- Controller->ControllerNumber,
- LogicalDriveNumber,
- (Command->V2.CommandStatus
- == DAC960_V2_NormalCompletion
- ? "Cancelled" : "Not Cancelled"));
- }
- else if (strcmp(UserCommand, "perform-discovery") == 0)
- {
- CommandMailbox->Common.IOCTL_Opcode = DAC960_V2_StartDiscovery;
- DAC960_ExecuteCommand(Command);
- DAC960_UserCritical("Discovery %s\n", Controller,
- (Command->V2.CommandStatus
- == DAC960_V2_NormalCompletion
- ? "Initiated" : "Not Initiated"));
- if (Command->V2.CommandStatus == DAC960_V2_NormalCompletion)
- {
- CommandMailbox->ControllerInfo.CommandOpcode = DAC960_V2_IOCTL;
- CommandMailbox->ControllerInfo.CommandControlBits
- .DataTransferControllerToHost = true;
- CommandMailbox->ControllerInfo.CommandControlBits
- .NoAutoRequestSense = true;
- CommandMailbox->ControllerInfo.DataTransferSize =
- sizeof(DAC960_V2_ControllerInfo_T);
- CommandMailbox->ControllerInfo.ControllerNumber = 0;
- CommandMailbox->ControllerInfo.IOCTL_Opcode =
- DAC960_V2_GetControllerInfo;
- /*
- * How does this NOT race with the queued Monitoring
- * usage of this structure?
- */
- CommandMailbox->ControllerInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer =
- Controller->V2.NewControllerInformationDMA;
- CommandMailbox->ControllerInfo.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->ControllerInfo.DataTransferSize;
- while (1) {
- DAC960_ExecuteCommand(Command);
- if (!Controller->V2.NewControllerInformation->PhysicalScanActive)
- break;
- msleep(1000);
- }
- DAC960_UserCritical("Discovery Completed\n", Controller);
- }
- }
- else if (strcmp(UserCommand, "suppress-enclosure-messages") == 0)
- Controller->SuppressEnclosureMessages = true;
- else DAC960_UserCritical("Illegal User Command: '%s'\n",
- Controller, UserCommand);
-
- spin_lock_irqsave(&Controller->queue_lock, flags);
- DAC960_DeallocateCommand(Command);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- return true;
-}
-
-static int __maybe_unused dac960_proc_show(struct seq_file *m, void *v)
-{
- unsigned char *StatusMessage = "OK\n";
- int ControllerNumber;
- for (ControllerNumber = 0;
- ControllerNumber < DAC960_ControllerCount;
- ControllerNumber++)
- {
- DAC960_Controller_T *Controller = DAC960_Controllers[ControllerNumber];
- if (Controller == NULL) continue;
- if (Controller->MonitoringAlertMode)
- {
- StatusMessage = "ALERT\n";
- break;
- }
- }
- seq_puts(m, StatusMessage);
- return 0;
-}
-
-static int __maybe_unused dac960_initial_status_proc_show(struct seq_file *m,
- void *v)
-{
- DAC960_Controller_T *Controller = (DAC960_Controller_T *)m->private;
- seq_printf(m, "%.*s", Controller->InitialStatusLength, Controller->CombinedStatusBuffer);
- return 0;
-}
-
-static int __maybe_unused dac960_current_status_proc_show(struct seq_file *m,
- void *v)
-{
- DAC960_Controller_T *Controller = (DAC960_Controller_T *) m->private;
- unsigned char *StatusMessage =
- "No Rebuild or Consistency Check in Progress\n";
- int ProgressMessageLength = strlen(StatusMessage);
- if (jiffies != Controller->LastCurrentStatusTime)
- {
- Controller->CurrentStatusLength = 0;
- DAC960_AnnounceDriver(Controller);
- DAC960_ReportControllerConfiguration(Controller);
- DAC960_ReportDeviceConfiguration(Controller);
- if (Controller->ProgressBufferLength > 0)
- ProgressMessageLength = Controller->ProgressBufferLength;
- if (DAC960_CheckStatusBuffer(Controller, 2 + ProgressMessageLength))
- {
- unsigned char *CurrentStatusBuffer = Controller->CurrentStatusBuffer;
- CurrentStatusBuffer[Controller->CurrentStatusLength++] = ' ';
- CurrentStatusBuffer[Controller->CurrentStatusLength++] = ' ';
- if (Controller->ProgressBufferLength > 0)
- strcpy(&CurrentStatusBuffer[Controller->CurrentStatusLength],
- Controller->ProgressBuffer);
- else
- strcpy(&CurrentStatusBuffer[Controller->CurrentStatusLength],
- StatusMessage);
- Controller->CurrentStatusLength += ProgressMessageLength;
- }
- Controller->LastCurrentStatusTime = jiffies;
- }
- seq_printf(m, "%.*s", Controller->CurrentStatusLength, Controller->CurrentStatusBuffer);
- return 0;
-}
-
-static int dac960_user_command_proc_show(struct seq_file *m, void *v)
-{
- DAC960_Controller_T *Controller = (DAC960_Controller_T *)m->private;
-
- seq_printf(m, "%.*s", Controller->UserStatusLength, Controller->UserStatusBuffer);
- return 0;
-}
-
-static int dac960_user_command_proc_open(struct inode *inode, struct file *file)
-{
- return single_open(file, dac960_user_command_proc_show, PDE_DATA(inode));
-}
-
-static ssize_t dac960_user_command_proc_write(struct file *file,
- const char __user *Buffer,
- size_t Count, loff_t *pos)
-{
- DAC960_Controller_T *Controller = PDE_DATA(file_inode(file));
- unsigned char CommandBuffer[80];
- int Length;
- if (Count > sizeof(CommandBuffer)-1) return -EINVAL;
- if (copy_from_user(CommandBuffer, Buffer, Count)) return -EFAULT;
- CommandBuffer[Count] = '\0';
- Length = strlen(CommandBuffer);
- if (Length > 0 && CommandBuffer[Length-1] == '\n')
- CommandBuffer[--Length] = '\0';
- if (Controller->FirmwareType == DAC960_V1_Controller)
- return (DAC960_V1_ExecuteUserCommand(Controller, CommandBuffer)
- ? Count : -EBUSY);
- else
- return (DAC960_V2_ExecuteUserCommand(Controller, CommandBuffer)
- ? Count : -EBUSY);
-}
-
-static const struct file_operations dac960_user_command_proc_fops = {
- .owner = THIS_MODULE,
- .open = dac960_user_command_proc_open,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
- .write = dac960_user_command_proc_write,
-};
-
-/*
- DAC960_CreateProcEntries creates the /proc/rd/... entries for the
- DAC960 Driver.
-*/
-
-static void DAC960_CreateProcEntries(DAC960_Controller_T *Controller)
-{
- struct proc_dir_entry *ControllerProcEntry;
-
- if (DAC960_ProcDirectoryEntry == NULL) {
- DAC960_ProcDirectoryEntry = proc_mkdir("rd", NULL);
- proc_create_single("status", 0, DAC960_ProcDirectoryEntry,
- dac960_proc_show);
- }
-
- snprintf(Controller->ControllerName, sizeof(Controller->ControllerName),
- "c%d", Controller->ControllerNumber);
- ControllerProcEntry = proc_mkdir(Controller->ControllerName,
- DAC960_ProcDirectoryEntry);
- proc_create_single_data("initial_status", 0, ControllerProcEntry,
- dac960_initial_status_proc_show, Controller);
- proc_create_single_data("current_status", 0, ControllerProcEntry,
- dac960_current_status_proc_show, Controller);
- proc_create_data("user_command", 0600, ControllerProcEntry, &dac960_user_command_proc_fops, Controller);
- Controller->ControllerProcEntry = ControllerProcEntry;
-}
-
-
-/*
- DAC960_DestroyProcEntries destroys the /proc/rd/... entries for the
- DAC960 Driver.
-*/
-
-static void DAC960_DestroyProcEntries(DAC960_Controller_T *Controller)
-{
- if (Controller->ControllerProcEntry == NULL)
- return;
- remove_proc_entry("initial_status", Controller->ControllerProcEntry);
- remove_proc_entry("current_status", Controller->ControllerProcEntry);
- remove_proc_entry("user_command", Controller->ControllerProcEntry);
- remove_proc_entry(Controller->ControllerName, DAC960_ProcDirectoryEntry);
- Controller->ControllerProcEntry = NULL;
-}
-
-#ifdef DAC960_GAM_MINOR
-
-static long DAC960_gam_get_controller_info(DAC960_ControllerInfo_T __user *UserSpaceControllerInfo)
-{
- DAC960_ControllerInfo_T ControllerInfo;
- DAC960_Controller_T *Controller;
- int ControllerNumber;
- long ErrorCode;
-
- if (UserSpaceControllerInfo == NULL)
- ErrorCode = -EINVAL;
- else ErrorCode = get_user(ControllerNumber,
- &UserSpaceControllerInfo->ControllerNumber);
- if (ErrorCode != 0)
- goto out;
- ErrorCode = -ENXIO;
- if (ControllerNumber < 0 ||
- ControllerNumber > DAC960_ControllerCount - 1) {
- goto out;
- }
- Controller = DAC960_Controllers[ControllerNumber];
- if (Controller == NULL)
- goto out;
- memset(&ControllerInfo, 0, sizeof(DAC960_ControllerInfo_T));
- ControllerInfo.ControllerNumber = ControllerNumber;
- ControllerInfo.FirmwareType = Controller->FirmwareType;
- ControllerInfo.Channels = Controller->Channels;
- ControllerInfo.Targets = Controller->Targets;
- ControllerInfo.PCI_Bus = Controller->Bus;
- ControllerInfo.PCI_Device = Controller->Device;
- ControllerInfo.PCI_Function = Controller->Function;
- ControllerInfo.IRQ_Channel = Controller->IRQ_Channel;
- ControllerInfo.PCI_Address = Controller->PCI_Address;
- strcpy(ControllerInfo.ModelName, Controller->ModelName);
- strcpy(ControllerInfo.FirmwareVersion, Controller->FirmwareVersion);
- ErrorCode = (copy_to_user(UserSpaceControllerInfo, &ControllerInfo,
- sizeof(DAC960_ControllerInfo_T)) ? -EFAULT : 0);
-out:
- return ErrorCode;
-}
-
-static long DAC960_gam_v1_execute_command(DAC960_V1_UserCommand_T __user *UserSpaceUserCommand)
-{
- DAC960_V1_UserCommand_T UserCommand;
- DAC960_Controller_T *Controller;
- DAC960_Command_T *Command = NULL;
- DAC960_V1_CommandOpcode_T CommandOpcode;
- DAC960_V1_CommandStatus_T CommandStatus;
- DAC960_V1_DCDB_T DCDB;
- DAC960_V1_DCDB_T *DCDB_IOBUF = NULL;
- dma_addr_t DCDB_IOBUFDMA;
- unsigned long flags;
- int ControllerNumber, DataTransferLength;
- unsigned char *DataTransferBuffer = NULL;
- dma_addr_t DataTransferBufferDMA;
- long ErrorCode;
-
- if (UserSpaceUserCommand == NULL) {
- ErrorCode = -EINVAL;
- goto out;
- }
- if (copy_from_user(&UserCommand, UserSpaceUserCommand,
- sizeof(DAC960_V1_UserCommand_T))) {
- ErrorCode = -EFAULT;
- goto out;
- }
- ControllerNumber = UserCommand.ControllerNumber;
- ErrorCode = -ENXIO;
- if (ControllerNumber < 0 ||
- ControllerNumber > DAC960_ControllerCount - 1)
- goto out;
- Controller = DAC960_Controllers[ControllerNumber];
- if (Controller == NULL)
- goto out;
- ErrorCode = -EINVAL;
- if (Controller->FirmwareType != DAC960_V1_Controller)
- goto out;
- CommandOpcode = UserCommand.CommandMailbox.Common.CommandOpcode;
- DataTransferLength = UserCommand.DataTransferLength;
- if (CommandOpcode & 0x80)
- goto out;
- if (CommandOpcode == DAC960_V1_DCDB)
- {
- if (copy_from_user(&DCDB, UserCommand.DCDB,
- sizeof(DAC960_V1_DCDB_T))) {
- ErrorCode = -EFAULT;
- goto out;
- }
- if (DCDB.Channel >= DAC960_V1_MaxChannels)
- goto out;
- if (!((DataTransferLength == 0 &&
- DCDB.Direction
- == DAC960_V1_DCDB_NoDataTransfer) ||
- (DataTransferLength > 0 &&
- DCDB.Direction
- == DAC960_V1_DCDB_DataTransferDeviceToSystem) ||
- (DataTransferLength < 0 &&
- DCDB.Direction
- == DAC960_V1_DCDB_DataTransferSystemToDevice)))
- goto out;
- if (((DCDB.TransferLengthHigh4 << 16) | DCDB.TransferLength)
- != abs(DataTransferLength))
- goto out;
- DCDB_IOBUF = pci_alloc_consistent(Controller->PCIDevice,
- sizeof(DAC960_V1_DCDB_T), &DCDB_IOBUFDMA);
- if (DCDB_IOBUF == NULL) {
- ErrorCode = -ENOMEM;
- goto out;
- }
- }
- ErrorCode = -ENOMEM;
- if (DataTransferLength > 0)
- {
- DataTransferBuffer = pci_zalloc_consistent(Controller->PCIDevice,
- DataTransferLength,
- &DataTransferBufferDMA);
- if (DataTransferBuffer == NULL)
- goto out;
- }
- else if (DataTransferLength < 0)
- {
- DataTransferBuffer = pci_alloc_consistent(Controller->PCIDevice,
- -DataTransferLength, &DataTransferBufferDMA);
- if (DataTransferBuffer == NULL)
- goto out;
- if (copy_from_user(DataTransferBuffer,
- UserCommand.DataTransferBuffer,
- -DataTransferLength)) {
- ErrorCode = -EFAULT;
- goto out;
- }
- }
- if (CommandOpcode == DAC960_V1_DCDB)
- {
- spin_lock_irqsave(&Controller->queue_lock, flags);
- while ((Command = DAC960_AllocateCommand(Controller)) == NULL)
- DAC960_WaitForCommand(Controller);
- while (Controller->V1.DirectCommandActive[DCDB.Channel]
- [DCDB.TargetID])
- {
- spin_unlock_irq(&Controller->queue_lock);
- __wait_event(Controller->CommandWaitQueue,
- !Controller->V1.DirectCommandActive
- [DCDB.Channel][DCDB.TargetID]);
- spin_lock_irq(&Controller->queue_lock);
- }
- Controller->V1.DirectCommandActive[DCDB.Channel]
- [DCDB.TargetID] = true;
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- DAC960_V1_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- memcpy(&Command->V1.CommandMailbox, &UserCommand.CommandMailbox,
- sizeof(DAC960_V1_CommandMailbox_T));
- Command->V1.CommandMailbox.Type3.BusAddress = DCDB_IOBUFDMA;
- DCDB.BusAddress = DataTransferBufferDMA;
- memcpy(DCDB_IOBUF, &DCDB, sizeof(DAC960_V1_DCDB_T));
- }
- else
- {
- spin_lock_irqsave(&Controller->queue_lock, flags);
- while ((Command = DAC960_AllocateCommand(Controller)) == NULL)
- DAC960_WaitForCommand(Controller);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- DAC960_V1_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- memcpy(&Command->V1.CommandMailbox, &UserCommand.CommandMailbox,
- sizeof(DAC960_V1_CommandMailbox_T));
- if (DataTransferBuffer != NULL)
- Command->V1.CommandMailbox.Type3.BusAddress =
- DataTransferBufferDMA;
- }
- DAC960_ExecuteCommand(Command);
- CommandStatus = Command->V1.CommandStatus;
- spin_lock_irqsave(&Controller->queue_lock, flags);
- DAC960_DeallocateCommand(Command);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- if (DataTransferLength > 0)
- {
- if (copy_to_user(UserCommand.DataTransferBuffer,
- DataTransferBuffer, DataTransferLength)) {
- ErrorCode = -EFAULT;
- goto Failure1;
- }
- }
- if (CommandOpcode == DAC960_V1_DCDB)
- {
- /*
- I don't believe Target or Channel in the DCDB_IOBUF
- should be any different from the contents of DCDB.
- */
- Controller->V1.DirectCommandActive[DCDB.Channel]
- [DCDB.TargetID] = false;
- if (copy_to_user(UserCommand.DCDB, DCDB_IOBUF,
- sizeof(DAC960_V1_DCDB_T))) {
- ErrorCode = -EFAULT;
- goto Failure1;
- }
- }
- ErrorCode = CommandStatus;
- Failure1:
- if (DataTransferBuffer != NULL)
- pci_free_consistent(Controller->PCIDevice, abs(DataTransferLength),
- DataTransferBuffer, DataTransferBufferDMA);
- if (DCDB_IOBUF != NULL)
- pci_free_consistent(Controller->PCIDevice, sizeof(DAC960_V1_DCDB_T),
- DCDB_IOBUF, DCDB_IOBUFDMA);
- out:
- return ErrorCode;
-}
-
-static long DAC960_gam_v2_execute_command(DAC960_V2_UserCommand_T __user *UserSpaceUserCommand)
-{
- DAC960_V2_UserCommand_T UserCommand;
- DAC960_Controller_T *Controller;
- DAC960_Command_T *Command = NULL;
- DAC960_V2_CommandMailbox_T *CommandMailbox;
- DAC960_V2_CommandStatus_T CommandStatus;
- unsigned long flags;
- int ControllerNumber, DataTransferLength;
- int DataTransferResidue, RequestSenseLength;
- unsigned char *DataTransferBuffer = NULL;
- dma_addr_t DataTransferBufferDMA;
- unsigned char *RequestSenseBuffer = NULL;
- dma_addr_t RequestSenseBufferDMA;
- long ErrorCode = -EINVAL;
-
- if (UserSpaceUserCommand == NULL)
- goto out;
- if (copy_from_user(&UserCommand, UserSpaceUserCommand,
- sizeof(DAC960_V2_UserCommand_T))) {
- ErrorCode = -EFAULT;
- goto out;
- }
- ErrorCode = -ENXIO;
- ControllerNumber = UserCommand.ControllerNumber;
- if (ControllerNumber < 0 ||
- ControllerNumber > DAC960_ControllerCount - 1)
- goto out;
- Controller = DAC960_Controllers[ControllerNumber];
- if (Controller == NULL)
- goto out;
- if (Controller->FirmwareType != DAC960_V2_Controller){
- ErrorCode = -EINVAL;
- goto out;
- }
- DataTransferLength = UserCommand.DataTransferLength;
- ErrorCode = -ENOMEM;
- if (DataTransferLength > 0)
- {
- DataTransferBuffer = pci_zalloc_consistent(Controller->PCIDevice,
- DataTransferLength,
- &DataTransferBufferDMA);
- if (DataTransferBuffer == NULL)
- goto out;
- }
- else if (DataTransferLength < 0)
- {
- DataTransferBuffer = pci_alloc_consistent(Controller->PCIDevice,
- -DataTransferLength, &DataTransferBufferDMA);
- if (DataTransferBuffer == NULL)
- goto out;
- if (copy_from_user(DataTransferBuffer,
- UserCommand.DataTransferBuffer,
- -DataTransferLength)) {
- ErrorCode = -EFAULT;
- goto Failure2;
- }
- }
- RequestSenseLength = UserCommand.RequestSenseLength;
- if (RequestSenseLength > 0)
- {
- RequestSenseBuffer = pci_zalloc_consistent(Controller->PCIDevice,
- RequestSenseLength,
- &RequestSenseBufferDMA);
- if (RequestSenseBuffer == NULL)
- {
- ErrorCode = -ENOMEM;
- goto Failure2;
- }
- }
- spin_lock_irqsave(&Controller->queue_lock, flags);
- while ((Command = DAC960_AllocateCommand(Controller)) == NULL)
- DAC960_WaitForCommand(Controller);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- DAC960_V2_ClearCommand(Command);
- Command->CommandType = DAC960_ImmediateCommand;
- CommandMailbox = &Command->V2.CommandMailbox;
- memcpy(CommandMailbox, &UserCommand.CommandMailbox,
- sizeof(DAC960_V2_CommandMailbox_T));
- CommandMailbox->Common.CommandControlBits
- .AdditionalScatterGatherListMemory = false;
- CommandMailbox->Common.CommandControlBits
- .NoAutoRequestSense = true;
- CommandMailbox->Common.DataTransferSize = 0;
- CommandMailbox->Common.DataTransferPageNumber = 0;
- memset(&CommandMailbox->Common.DataTransferMemoryAddress, 0,
- sizeof(DAC960_V2_DataTransferMemoryAddress_T));
- if (DataTransferLength != 0)
- {
- if (DataTransferLength > 0)
- {
- CommandMailbox->Common.CommandControlBits
- .DataTransferControllerToHost = true;
- CommandMailbox->Common.DataTransferSize = DataTransferLength;
- }
- else
- {
- CommandMailbox->Common.CommandControlBits
- .DataTransferControllerToHost = false;
- CommandMailbox->Common.DataTransferSize = -DataTransferLength;
- }
- CommandMailbox->Common.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentDataPointer = DataTransferBufferDMA;
- CommandMailbox->Common.DataTransferMemoryAddress
- .ScatterGatherSegments[0]
- .SegmentByteCount =
- CommandMailbox->Common.DataTransferSize;
- }
- if (RequestSenseLength > 0)
- {
- CommandMailbox->Common.CommandControlBits
- .NoAutoRequestSense = false;
- CommandMailbox->Common.RequestSenseSize = RequestSenseLength;
- CommandMailbox->Common.RequestSenseBusAddress =
- RequestSenseBufferDMA;
- }
- DAC960_ExecuteCommand(Command);
- CommandStatus = Command->V2.CommandStatus;
- RequestSenseLength = Command->V2.RequestSenseLength;
- DataTransferResidue = Command->V2.DataTransferResidue;
- spin_lock_irqsave(&Controller->queue_lock, flags);
- DAC960_DeallocateCommand(Command);
- spin_unlock_irqrestore(&Controller->queue_lock, flags);
- if (RequestSenseLength > UserCommand.RequestSenseLength)
- RequestSenseLength = UserCommand.RequestSenseLength;
- if (copy_to_user(&UserSpaceUserCommand->DataTransferLength,
- &DataTransferResidue,
- sizeof(DataTransferResidue))) {
- ErrorCode = -EFAULT;
- goto Failure2;
- }
- if (copy_to_user(&UserSpaceUserCommand->RequestSenseLength,
- &RequestSenseLength, sizeof(RequestSenseLength))) {
- ErrorCode = -EFAULT;
- goto Failure2;
- }
- if (DataTransferLength > 0)
- {
- if (copy_to_user(UserCommand.DataTransferBuffer,
- DataTransferBuffer, DataTransferLength)) {
- ErrorCode = -EFAULT;
- goto Failure2;
- }
- }
- if (RequestSenseLength > 0)
- {
- if (copy_to_user(UserCommand.RequestSenseBuffer,
- RequestSenseBuffer, RequestSenseLength)) {
- ErrorCode = -EFAULT;
- goto Failure2;
- }
- }
- ErrorCode = CommandStatus;
- Failure2:
- pci_free_consistent(Controller->PCIDevice, abs(DataTransferLength),
- DataTransferBuffer, DataTransferBufferDMA);
- if (RequestSenseBuffer != NULL)
- pci_free_consistent(Controller->PCIDevice, RequestSenseLength,
- RequestSenseBuffer, RequestSenseBufferDMA);
-out:
- return ErrorCode;
-}
-
-static long DAC960_gam_v2_get_health_status(DAC960_V2_GetHealthStatus_T __user *UserSpaceGetHealthStatus)
-{
- DAC960_V2_GetHealthStatus_T GetHealthStatus;
- DAC960_V2_HealthStatusBuffer_T HealthStatusBuffer;
- DAC960_Controller_T *Controller;
- int ControllerNumber;
- long ErrorCode;
-
- if (UserSpaceGetHealthStatus == NULL) {
- ErrorCode = -EINVAL;
- goto out;
- }
- if (copy_from_user(&GetHealthStatus, UserSpaceGetHealthStatus,
- sizeof(DAC960_V2_GetHealthStatus_T))) {
- ErrorCode = -EFAULT;
- goto out;
- }
- ErrorCode = -ENXIO;
- ControllerNumber = GetHealthStatus.ControllerNumber;
- if (ControllerNumber < 0 ||
- ControllerNumber > DAC960_ControllerCount - 1)
- goto out;
- Controller = DAC960_Controllers[ControllerNumber];
- if (Controller == NULL)
- goto out;
- if (Controller->FirmwareType != DAC960_V2_Controller) {
- ErrorCode = -EINVAL;
- goto out;
- }
- if (copy_from_user(&HealthStatusBuffer,
- GetHealthStatus.HealthStatusBuffer,
- sizeof(DAC960_V2_HealthStatusBuffer_T))) {
- ErrorCode = -EFAULT;
- goto out;
- }
- ErrorCode = wait_event_interruptible_timeout(Controller->HealthStatusWaitQueue,
- !(Controller->V2.HealthStatusBuffer->StatusChangeCounter
- == HealthStatusBuffer.StatusChangeCounter &&
- Controller->V2.HealthStatusBuffer->NextEventSequenceNumber
- == HealthStatusBuffer.NextEventSequenceNumber),
- DAC960_MonitoringTimerInterval);
- if (ErrorCode == -ERESTARTSYS) {
- ErrorCode = -EINTR;
- goto out;
- }
- if (copy_to_user(GetHealthStatus.HealthStatusBuffer,
- Controller->V2.HealthStatusBuffer,
- sizeof(DAC960_V2_HealthStatusBuffer_T)))
- ErrorCode = -EFAULT;
- else
- ErrorCode = 0;
-
-out:
- return ErrorCode;
-}
-
-/*
- * DAC960_gam_ioctl is the ioctl function for performing RAID operations.
-*/
-
-static long DAC960_gam_ioctl(struct file *file, unsigned int Request,
- unsigned long Argument)
-{
- long ErrorCode = 0;
- void __user *argp = (void __user *)Argument;
- if (!capable(CAP_SYS_ADMIN)) return -EACCES;
-
- mutex_lock(&DAC960_mutex);
- switch (Request)
- {
- case DAC960_IOCTL_GET_CONTROLLER_COUNT:
- ErrorCode = DAC960_ControllerCount;
- break;
- case DAC960_IOCTL_GET_CONTROLLER_INFO:
- ErrorCode = DAC960_gam_get_controller_info(argp);
- break;
- case DAC960_IOCTL_V1_EXECUTE_COMMAND:
- ErrorCode = DAC960_gam_v1_execute_command(argp);
- break;
- case DAC960_IOCTL_V2_EXECUTE_COMMAND:
- ErrorCode = DAC960_gam_v2_execute_command(argp);
- break;
- case DAC960_IOCTL_V2_GET_HEALTH_STATUS:
- ErrorCode = DAC960_gam_v2_get_health_status(argp);
- break;
- default:
- ErrorCode = -ENOTTY;
- }
- mutex_unlock(&DAC960_mutex);
- return ErrorCode;
-}
-
-static const struct file_operations DAC960_gam_fops = {
- .owner = THIS_MODULE,
- .unlocked_ioctl = DAC960_gam_ioctl,
- .llseek = noop_llseek,
-};
-
-static struct miscdevice DAC960_gam_dev = {
- DAC960_GAM_MINOR,
- "dac960_gam",
- &DAC960_gam_fops
-};
-
-static int DAC960_gam_init(void)
-{
- int ret;
-
- ret = misc_register(&DAC960_gam_dev);
- if (ret)
- printk(KERN_ERR "DAC960_gam: can't misc_register on minor %d\n", DAC960_GAM_MINOR);
- return ret;
-}
-
-static void DAC960_gam_cleanup(void)
-{
- misc_deregister(&DAC960_gam_dev);
-}
-
-#endif /* DAC960_GAM_MINOR */
-
-static struct DAC960_privdata DAC960_GEM_privdata = {
- .HardwareType = DAC960_GEM_Controller,
- .FirmwareType = DAC960_V2_Controller,
- .InterruptHandler = DAC960_GEM_InterruptHandler,
- .MemoryWindowSize = DAC960_GEM_RegisterWindowSize,
-};
-
-
-static struct DAC960_privdata DAC960_BA_privdata = {
- .HardwareType = DAC960_BA_Controller,
- .FirmwareType = DAC960_V2_Controller,
- .InterruptHandler = DAC960_BA_InterruptHandler,
- .MemoryWindowSize = DAC960_BA_RegisterWindowSize,
-};
-
-static struct DAC960_privdata DAC960_LP_privdata = {
- .HardwareType = DAC960_LP_Controller,
- .FirmwareType = DAC960_V2_Controller,
- .InterruptHandler = DAC960_LP_InterruptHandler,
- .MemoryWindowSize = DAC960_LP_RegisterWindowSize,
-};
-
-static struct DAC960_privdata DAC960_LA_privdata = {
- .HardwareType = DAC960_LA_Controller,
- .FirmwareType = DAC960_V1_Controller,
- .InterruptHandler = DAC960_LA_InterruptHandler,
- .MemoryWindowSize = DAC960_LA_RegisterWindowSize,
-};
-
-static struct DAC960_privdata DAC960_PG_privdata = {
- .HardwareType = DAC960_PG_Controller,
- .FirmwareType = DAC960_V1_Controller,
- .InterruptHandler = DAC960_PG_InterruptHandler,
- .MemoryWindowSize = DAC960_PG_RegisterWindowSize,
-};
-
-static struct DAC960_privdata DAC960_PD_privdata = {
- .HardwareType = DAC960_PD_Controller,
- .FirmwareType = DAC960_V1_Controller,
- .InterruptHandler = DAC960_PD_InterruptHandler,
- .MemoryWindowSize = DAC960_PD_RegisterWindowSize,
-};
-
-static struct DAC960_privdata DAC960_P_privdata = {
- .HardwareType = DAC960_P_Controller,
- .FirmwareType = DAC960_V1_Controller,
- .InterruptHandler = DAC960_P_InterruptHandler,
- .MemoryWindowSize = DAC960_PD_RegisterWindowSize,
-};
-
-static const struct pci_device_id DAC960_id_table[] = {
- {
- .vendor = PCI_VENDOR_ID_MYLEX,
- .device = PCI_DEVICE_ID_MYLEX_DAC960_GEM,
- .subvendor = PCI_VENDOR_ID_MYLEX,
- .subdevice = PCI_ANY_ID,
- .driver_data = (unsigned long) &DAC960_GEM_privdata,
- },
- {
- .vendor = PCI_VENDOR_ID_MYLEX,
- .device = PCI_DEVICE_ID_MYLEX_DAC960_BA,
- .subvendor = PCI_ANY_ID,
- .subdevice = PCI_ANY_ID,
- .driver_data = (unsigned long) &DAC960_BA_privdata,
- },
- {
- .vendor = PCI_VENDOR_ID_MYLEX,
- .device = PCI_DEVICE_ID_MYLEX_DAC960_LP,
- .subvendor = PCI_ANY_ID,
- .subdevice = PCI_ANY_ID,
- .driver_data = (unsigned long) &DAC960_LP_privdata,
- },
- {
- .vendor = PCI_VENDOR_ID_DEC,
- .device = PCI_DEVICE_ID_DEC_21285,
- .subvendor = PCI_VENDOR_ID_MYLEX,
- .subdevice = PCI_DEVICE_ID_MYLEX_DAC960_LA,
- .driver_data = (unsigned long) &DAC960_LA_privdata,
- },
- {
- .vendor = PCI_VENDOR_ID_MYLEX,
- .device = PCI_DEVICE_ID_MYLEX_DAC960_PG,
- .subvendor = PCI_ANY_ID,
- .subdevice = PCI_ANY_ID,
- .driver_data = (unsigned long) &DAC960_PG_privdata,
- },
- {
- .vendor = PCI_VENDOR_ID_MYLEX,
- .device = PCI_DEVICE_ID_MYLEX_DAC960_PD,
- .subvendor = PCI_ANY_ID,
- .subdevice = PCI_ANY_ID,
- .driver_data = (unsigned long) &DAC960_PD_privdata,
- },
- {
- .vendor = PCI_VENDOR_ID_MYLEX,
- .device = PCI_DEVICE_ID_MYLEX_DAC960_P,
- .subvendor = PCI_ANY_ID,
- .subdevice = PCI_ANY_ID,
- .driver_data = (unsigned long) &DAC960_P_privdata,
- },
- {0, },
-};
-
-MODULE_DEVICE_TABLE(pci, DAC960_id_table);
-
-static struct pci_driver DAC960_pci_driver = {
- .name = "DAC960",
- .id_table = DAC960_id_table,
- .probe = DAC960_Probe,
- .remove = DAC960_Remove,
-};
-
-static int __init DAC960_init_module(void)
-{
- int ret;
-
- ret = pci_register_driver(&DAC960_pci_driver);
-#ifdef DAC960_GAM_MINOR
- if (!ret)
- DAC960_gam_init();
-#endif
- return ret;
-}
-
-static void __exit DAC960_cleanup_module(void)
-{
- int i;
-
-#ifdef DAC960_GAM_MINOR
- DAC960_gam_cleanup();
-#endif
-
- for (i = 0; i < DAC960_ControllerCount; i++) {
- DAC960_Controller_T *Controller = DAC960_Controllers[i];
- if (Controller == NULL)
- continue;
- DAC960_FinalizeController(Controller);
- }
- if (DAC960_ProcDirectoryEntry != NULL) {
- remove_proc_entry("rd/status", NULL);
- remove_proc_entry("rd", NULL);
- }
- DAC960_ControllerCount = 0;
- pci_unregister_driver(&DAC960_pci_driver);
-}
-
-module_init(DAC960_init_module);
-module_exit(DAC960_cleanup_module);
-
-MODULE_LICENSE("GPL");
+++ /dev/null
-/*
-
- Linux Driver for Mylex DAC960/AcceleRAID/eXtremeRAID PCI RAID Controllers
-
- Copyright 1998-2001 by Leonard N. Zubkoff <lnz@dandelion.com>
-
- This program is free software; you may redistribute and/or modify it under
- the terms of the GNU General Public License Version 2 as published by the
- Free Software Foundation.
-
- This program is distributed in the hope that it will be useful, but
- WITHOUT ANY WARRANTY, without even the implied warranty of MERCHANTABILITY
- or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- for complete details.
-
- The author respectfully requests that any modifications to this software be
- sent directly to him for evaluation and testing.
-
-*/
-
-
-/*
- Define the maximum number of DAC960 Controllers supported by this driver.
-*/
-
-#define DAC960_MaxControllers 8
-
-
-/*
- Define the maximum number of Controller Channels supported by DAC960
- V1 and V2 Firmware Controllers.
-*/
-
-#define DAC960_V1_MaxChannels 3
-#define DAC960_V2_MaxChannels 4
-
-
-/*
- Define the maximum number of Targets per Channel supported by DAC960
- V1 and V2 Firmware Controllers.
-*/
-
-#define DAC960_V1_MaxTargets 16
-#define DAC960_V2_MaxTargets 128
-
-
-/*
- Define the maximum number of Logical Drives supported by DAC960
- V1 and V2 Firmware Controllers.
-*/
-
-#define DAC960_MaxLogicalDrives 32
-
-
-/*
- Define the maximum number of Physical Devices supported by DAC960
- V1 and V2 Firmware Controllers.
-*/
-
-#define DAC960_V1_MaxPhysicalDevices 45
-#define DAC960_V2_MaxPhysicalDevices 272
-
-/*
- Define a 32/64 bit I/O Address data type.
-*/
-
-typedef unsigned long DAC960_IO_Address_T;
-
-
-/*
- Define a 32/64 bit PCI Bus Address data type.
-*/
-
-typedef unsigned long DAC960_PCI_Address_T;
-
-
-/*
- Define a 32 bit Bus Address data type.
-*/
-
-typedef unsigned int DAC960_BusAddress32_T;
-
-
-/*
- Define a 64 bit Bus Address data type.
-*/
-
-typedef unsigned long long DAC960_BusAddress64_T;
-
-
-/*
- Define a 32 bit Byte Count data type.
-*/
-
-typedef unsigned int DAC960_ByteCount32_T;
-
-
-/*
- Define a 64 bit Byte Count data type.
-*/
-
-typedef unsigned long long DAC960_ByteCount64_T;
-
-
-/*
- dma_loaf is used by helper routines to divide a region of
- dma mapped memory into smaller pieces, where those pieces
- are not of uniform size.
- */
-
-struct dma_loaf {
- void *cpu_base;
- dma_addr_t dma_base;
- size_t length;
- void *cpu_free;
- dma_addr_t dma_free;
-};
-
-/*
- Define the SCSI INQUIRY Standard Data structure.
-*/
-
-typedef struct DAC960_SCSI_Inquiry
-{
- unsigned char PeripheralDeviceType:5; /* Byte 0 Bits 0-4 */
- unsigned char PeripheralQualifier:3; /* Byte 0 Bits 5-7 */
- unsigned char DeviceTypeModifier:7; /* Byte 1 Bits 0-6 */
- bool RMB:1; /* Byte 1 Bit 7 */
- unsigned char ANSI_ApprovedVersion:3; /* Byte 2 Bits 0-2 */
- unsigned char ECMA_Version:3; /* Byte 2 Bits 3-5 */
- unsigned char ISO_Version:2; /* Byte 2 Bits 6-7 */
- unsigned char ResponseDataFormat:4; /* Byte 3 Bits 0-3 */
- unsigned char :2; /* Byte 3 Bits 4-5 */
- bool TrmIOP:1; /* Byte 3 Bit 6 */
- bool AENC:1; /* Byte 3 Bit 7 */
- unsigned char AdditionalLength; /* Byte 4 */
- unsigned char :8; /* Byte 5 */
- unsigned char :8; /* Byte 6 */
- bool SftRe:1; /* Byte 7 Bit 0 */
- bool CmdQue:1; /* Byte 7 Bit 1 */
- bool :1; /* Byte 7 Bit 2 */
- bool Linked:1; /* Byte 7 Bit 3 */
- bool Sync:1; /* Byte 7 Bit 4 */
- bool WBus16:1; /* Byte 7 Bit 5 */
- bool WBus32:1; /* Byte 7 Bit 6 */
- bool RelAdr:1; /* Byte 7 Bit 7 */
- unsigned char VendorIdentification[8]; /* Bytes 8-15 */
- unsigned char ProductIdentification[16]; /* Bytes 16-31 */
- unsigned char ProductRevisionLevel[4]; /* Bytes 32-35 */
-}
-DAC960_SCSI_Inquiry_T;
-
-
-/*
- Define the SCSI INQUIRY Unit Serial Number structure.
-*/
-
-typedef struct DAC960_SCSI_Inquiry_UnitSerialNumber
-{
- unsigned char PeripheralDeviceType:5; /* Byte 0 Bits 0-4 */
- unsigned char PeripheralQualifier:3; /* Byte 0 Bits 5-7 */
- unsigned char PageCode; /* Byte 1 */
- unsigned char :8; /* Byte 2 */
- unsigned char PageLength; /* Byte 3 */
- unsigned char ProductSerialNumber[28]; /* Bytes 4-31 */
-}
-DAC960_SCSI_Inquiry_UnitSerialNumber_T;
-
-
-/*
- Define the SCSI REQUEST SENSE Sense Key type.
-*/
-
-typedef enum
-{
- DAC960_SenseKey_NoSense = 0x0,
- DAC960_SenseKey_RecoveredError = 0x1,
- DAC960_SenseKey_NotReady = 0x2,
- DAC960_SenseKey_MediumError = 0x3,
- DAC960_SenseKey_HardwareError = 0x4,
- DAC960_SenseKey_IllegalRequest = 0x5,
- DAC960_SenseKey_UnitAttention = 0x6,
- DAC960_SenseKey_DataProtect = 0x7,
- DAC960_SenseKey_BlankCheck = 0x8,
- DAC960_SenseKey_VendorSpecific = 0x9,
- DAC960_SenseKey_CopyAborted = 0xA,
- DAC960_SenseKey_AbortedCommand = 0xB,
- DAC960_SenseKey_Equal = 0xC,
- DAC960_SenseKey_VolumeOverflow = 0xD,
- DAC960_SenseKey_Miscompare = 0xE,
- DAC960_SenseKey_Reserved = 0xF
-}
-__attribute__ ((packed))
-DAC960_SCSI_RequestSenseKey_T;
-
-
-/*
- Define the SCSI REQUEST SENSE structure.
-*/
-
-typedef struct DAC960_SCSI_RequestSense
-{
- unsigned char ErrorCode:7; /* Byte 0 Bits 0-6 */
- bool Valid:1; /* Byte 0 Bit 7 */
- unsigned char SegmentNumber; /* Byte 1 */
- DAC960_SCSI_RequestSenseKey_T SenseKey:4; /* Byte 2 Bits 0-3 */
- unsigned char :1; /* Byte 2 Bit 4 */
- bool ILI:1; /* Byte 2 Bit 5 */
- bool EOM:1; /* Byte 2 Bit 6 */
- bool Filemark:1; /* Byte 2 Bit 7 */
- unsigned char Information[4]; /* Bytes 3-6 */
- unsigned char AdditionalSenseLength; /* Byte 7 */
- unsigned char CommandSpecificInformation[4]; /* Bytes 8-11 */
- unsigned char AdditionalSenseCode; /* Byte 12 */
- unsigned char AdditionalSenseCodeQualifier; /* Byte 13 */
-}
-DAC960_SCSI_RequestSense_T;
-
-
-/*
- Define the DAC960 V1 Firmware Command Opcodes.
-*/
-
-typedef enum
-{
- /* I/O Commands */
- DAC960_V1_ReadExtended = 0x33,
- DAC960_V1_WriteExtended = 0x34,
- DAC960_V1_ReadAheadExtended = 0x35,
- DAC960_V1_ReadExtendedWithScatterGather = 0xB3,
- DAC960_V1_WriteExtendedWithScatterGather = 0xB4,
- DAC960_V1_Read = 0x36,
- DAC960_V1_ReadWithScatterGather = 0xB6,
- DAC960_V1_Write = 0x37,
- DAC960_V1_WriteWithScatterGather = 0xB7,
- DAC960_V1_DCDB = 0x04,
- DAC960_V1_DCDBWithScatterGather = 0x84,
- DAC960_V1_Flush = 0x0A,
- /* Controller Status Related Commands */
- DAC960_V1_Enquiry = 0x53,
- DAC960_V1_Enquiry2 = 0x1C,
- DAC960_V1_GetLogicalDriveElement = 0x55,
- DAC960_V1_GetLogicalDriveInformation = 0x19,
- DAC960_V1_IOPortRead = 0x39,
- DAC960_V1_IOPortWrite = 0x3A,
- DAC960_V1_GetSDStats = 0x3E,
- DAC960_V1_GetPDStats = 0x3F,
- DAC960_V1_PerformEventLogOperation = 0x72,
- /* Device Related Commands */
- DAC960_V1_StartDevice = 0x10,
- DAC960_V1_GetDeviceState = 0x50,
- DAC960_V1_StopChannel = 0x13,
- DAC960_V1_StartChannel = 0x12,
- DAC960_V1_ResetChannel = 0x1A,
- /* Commands Associated with Data Consistency and Errors */
- DAC960_V1_Rebuild = 0x09,
- DAC960_V1_RebuildAsync = 0x16,
- DAC960_V1_CheckConsistency = 0x0F,
- DAC960_V1_CheckConsistencyAsync = 0x1E,
- DAC960_V1_RebuildStat = 0x0C,
- DAC960_V1_GetRebuildProgress = 0x27,
- DAC960_V1_RebuildControl = 0x1F,
- DAC960_V1_ReadBadBlockTable = 0x0B,
- DAC960_V1_ReadBadDataTable = 0x25,
- DAC960_V1_ClearBadDataTable = 0x26,
- DAC960_V1_GetErrorTable = 0x17,
- DAC960_V1_AddCapacityAsync = 0x2A,
- DAC960_V1_BackgroundInitializationControl = 0x2B,
- /* Configuration Related Commands */
- DAC960_V1_ReadConfig2 = 0x3D,
- DAC960_V1_WriteConfig2 = 0x3C,
- DAC960_V1_ReadConfigurationOnDisk = 0x4A,
- DAC960_V1_WriteConfigurationOnDisk = 0x4B,
- DAC960_V1_ReadConfiguration = 0x4E,
- DAC960_V1_ReadBackupConfiguration = 0x4D,
- DAC960_V1_WriteConfiguration = 0x4F,
- DAC960_V1_AddConfiguration = 0x4C,
- DAC960_V1_ReadConfigurationLabel = 0x48,
- DAC960_V1_WriteConfigurationLabel = 0x49,
- /* Firmware Upgrade Related Commands */
- DAC960_V1_LoadImage = 0x20,
- DAC960_V1_StoreImage = 0x21,
- DAC960_V1_ProgramImage = 0x22,
- /* Diagnostic Commands */
- DAC960_V1_SetDiagnosticMode = 0x31,
- DAC960_V1_RunDiagnostic = 0x32,
- /* Subsystem Service Commands */
- DAC960_V1_GetSubsystemData = 0x70,
- DAC960_V1_SetSubsystemParameters = 0x71,
- /* Version 2.xx Firmware Commands */
- DAC960_V1_Enquiry_Old = 0x05,
- DAC960_V1_GetDeviceState_Old = 0x14,
- DAC960_V1_Read_Old = 0x02,
- DAC960_V1_Write_Old = 0x03,
- DAC960_V1_ReadWithScatterGather_Old = 0x82,
- DAC960_V1_WriteWithScatterGather_Old = 0x83
-}
-__attribute__ ((packed))
-DAC960_V1_CommandOpcode_T;
-
-
-/*
- Define the DAC960 V1 Firmware Command Identifier type.
-*/
-
-typedef unsigned char DAC960_V1_CommandIdentifier_T;
-
-
-/*
- Define the DAC960 V1 Firmware Command Status Codes.
-*/
-
-#define DAC960_V1_NormalCompletion 0x0000 /* Common */
-#define DAC960_V1_CheckConditionReceived 0x0002 /* Common */
-#define DAC960_V1_NoDeviceAtAddress 0x0102 /* Common */
-#define DAC960_V1_InvalidDeviceAddress 0x0105 /* Common */
-#define DAC960_V1_InvalidParameter 0x0105 /* Common */
-#define DAC960_V1_IrrecoverableDataError 0x0001 /* I/O */
-#define DAC960_V1_LogicalDriveNonexistentOrOffline 0x0002 /* I/O */
-#define DAC960_V1_AccessBeyondEndOfLogicalDrive 0x0105 /* I/O */
-#define DAC960_V1_BadDataEncountered 0x010C /* I/O */
-#define DAC960_V1_DeviceBusy 0x0008 /* DCDB */
-#define DAC960_V1_DeviceNonresponsive 0x000E /* DCDB */
-#define DAC960_V1_CommandTerminatedAbnormally 0x000F /* DCDB */
-#define DAC960_V1_UnableToStartDevice 0x0002 /* Device */
-#define DAC960_V1_InvalidChannelOrTargetOrModifier 0x0105 /* Device */
-#define DAC960_V1_ChannelBusy 0x0106 /* Device */
-#define DAC960_V1_ChannelNotStopped 0x0002 /* Device */
-#define DAC960_V1_AttemptToRebuildOnlineDrive 0x0002 /* Consistency */
-#define DAC960_V1_RebuildBadBlocksEncountered 0x0003 /* Consistency */
-#define DAC960_V1_NewDiskFailedDuringRebuild 0x0004 /* Consistency */
-#define DAC960_V1_RebuildOrCheckAlreadyInProgress 0x0106 /* Consistency */
-#define DAC960_V1_DependentDiskIsDead 0x0002 /* Consistency */
-#define DAC960_V1_InconsistentBlocksFound 0x0003 /* Consistency */
-#define DAC960_V1_InvalidOrNonredundantLogicalDrive 0x0105 /* Consistency */
-#define DAC960_V1_NoRebuildOrCheckInProgress 0x0105 /* Consistency */
-#define DAC960_V1_RebuildInProgress_DataValid 0x0000 /* Consistency */
-#define DAC960_V1_RebuildFailed_LogicalDriveFailure 0x0002 /* Consistency */
-#define DAC960_V1_RebuildFailed_BadBlocksOnOther 0x0003 /* Consistency */
-#define DAC960_V1_RebuildFailed_NewDriveFailed 0x0004 /* Consistency */
-#define DAC960_V1_RebuildSuccessful 0x0100 /* Consistency */
-#define DAC960_V1_RebuildSuccessfullyTerminated 0x0107 /* Consistency */
-#define DAC960_V1_BackgroundInitSuccessful 0x0100 /* Consistency */
-#define DAC960_V1_BackgroundInitAborted 0x0005 /* Consistency */
-#define DAC960_V1_NoBackgroundInitInProgress 0x0105 /* Consistency */
-#define DAC960_V1_AddCapacityInProgress 0x0004 /* Consistency */
-#define DAC960_V1_AddCapacityFailedOrSuspended 0x00F4 /* Consistency */
-#define DAC960_V1_Config2ChecksumError 0x0002 /* Configuration */
-#define DAC960_V1_ConfigurationSuspended 0x0106 /* Configuration */
-#define DAC960_V1_FailedToConfigureNVRAM 0x0105 /* Configuration */
-#define DAC960_V1_ConfigurationNotSavedStateChange 0x0106 /* Configuration */
-#define DAC960_V1_SubsystemNotInstalled 0x0001 /* Subsystem */
-#define DAC960_V1_SubsystemFailed 0x0002 /* Subsystem */
-#define DAC960_V1_SubsystemBusy 0x0106 /* Subsystem */
-
-typedef unsigned short DAC960_V1_CommandStatus_T;
-
-
-/*
- Define the DAC960 V1 Firmware Enquiry Command reply structure.
-*/
-
-typedef struct DAC960_V1_Enquiry
-{
- unsigned char NumberOfLogicalDrives; /* Byte 0 */
- unsigned int :24; /* Bytes 1-3 */
- unsigned int LogicalDriveSizes[32]; /* Bytes 4-131 */
- unsigned short FlashAge; /* Bytes 132-133 */
- struct {
- bool DeferredWriteError:1; /* Byte 134 Bit 0 */
- bool BatteryLow:1; /* Byte 134 Bit 1 */
- unsigned char :6; /* Byte 134 Bits 2-7 */
- } StatusFlags;
- unsigned char :8; /* Byte 135 */
- unsigned char MinorFirmwareVersion; /* Byte 136 */
- unsigned char MajorFirmwareVersion; /* Byte 137 */
- enum {
- DAC960_V1_NoStandbyRebuildOrCheckInProgress = 0x00,
- DAC960_V1_StandbyRebuildInProgress = 0x01,
- DAC960_V1_BackgroundRebuildInProgress = 0x02,
- DAC960_V1_BackgroundCheckInProgress = 0x03,
- DAC960_V1_StandbyRebuildCompletedWithError = 0xFF,
- DAC960_V1_BackgroundRebuildOrCheckFailed_DriveFailed = 0xF0,
- DAC960_V1_BackgroundRebuildOrCheckFailed_LogicalDriveFailed = 0xF1,
- DAC960_V1_BackgroundRebuildOrCheckFailed_OtherCauses = 0xF2,
- DAC960_V1_BackgroundRebuildOrCheckSuccessfullyTerminated = 0xF3
- } __attribute__ ((packed)) RebuildFlag; /* Byte 138 */
- unsigned char MaxCommands; /* Byte 139 */
- unsigned char OfflineLogicalDriveCount; /* Byte 140 */
- unsigned char :8; /* Byte 141 */
- unsigned short EventLogSequenceNumber; /* Bytes 142-143 */
- unsigned char CriticalLogicalDriveCount; /* Byte 144 */
- unsigned int :24; /* Bytes 145-147 */
- unsigned char DeadDriveCount; /* Byte 148 */
- unsigned char :8; /* Byte 149 */
- unsigned char RebuildCount; /* Byte 150 */
- struct {
- unsigned char :3; /* Byte 151 Bits 0-2 */
- bool BatteryBackupUnitPresent:1; /* Byte 151 Bit 3 */
- unsigned char :3; /* Byte 151 Bits 4-6 */
- unsigned char :1; /* Byte 151 Bit 7 */
- } MiscFlags;
- struct {
- unsigned char TargetID;
- unsigned char Channel;
- } DeadDrives[21]; /* Bytes 152-194 */
- unsigned char Reserved[62]; /* Bytes 195-255 */
-}
-__attribute__ ((packed))
-DAC960_V1_Enquiry_T;
-
-
-/*
- Define the DAC960 V1 Firmware Enquiry2 Command reply structure.
-*/
-
-typedef struct DAC960_V1_Enquiry2
-{
- struct {
- enum {
- DAC960_V1_P_PD_PU = 0x01,
- DAC960_V1_PL = 0x02,
- DAC960_V1_PG = 0x10,
- DAC960_V1_PJ = 0x11,
- DAC960_V1_PR = 0x12,
- DAC960_V1_PT = 0x13,
- DAC960_V1_PTL0 = 0x14,
- DAC960_V1_PRL = 0x15,
- DAC960_V1_PTL1 = 0x16,
- DAC960_V1_1164P = 0x20
- } __attribute__ ((packed)) SubModel; /* Byte 0 */
- unsigned char ActualChannels; /* Byte 1 */
- enum {
- DAC960_V1_FiveChannelBoard = 0x01,
- DAC960_V1_ThreeChannelBoard = 0x02,
- DAC960_V1_TwoChannelBoard = 0x03,
- DAC960_V1_ThreeChannelASIC_DAC = 0x04
- } __attribute__ ((packed)) Model; /* Byte 2 */
- enum {
- DAC960_V1_EISA_Controller = 0x01,
- DAC960_V1_MicroChannel_Controller = 0x02,
- DAC960_V1_PCI_Controller = 0x03,
- DAC960_V1_SCSItoSCSI_Controller = 0x08
- } __attribute__ ((packed)) ProductFamily; /* Byte 3 */
- } HardwareID; /* Bytes 0-3 */
- /* MajorVersion.MinorVersion-FirmwareType-TurnID */
- struct {
- unsigned char MajorVersion; /* Byte 4 */
- unsigned char MinorVersion; /* Byte 5 */
- unsigned char TurnID; /* Byte 6 */
- char FirmwareType; /* Byte 7 */
- } FirmwareID; /* Bytes 4-7 */
- unsigned char :8; /* Byte 8 */
- unsigned int :24; /* Bytes 9-11 */
- unsigned char ConfiguredChannels; /* Byte 12 */
- unsigned char ActualChannels; /* Byte 13 */
- unsigned char MaxTargets; /* Byte 14 */
- unsigned char MaxTags; /* Byte 15 */
- unsigned char MaxLogicalDrives; /* Byte 16 */
- unsigned char MaxArms; /* Byte 17 */
- unsigned char MaxSpans; /* Byte 18 */
- unsigned char :8; /* Byte 19 */
- unsigned int :32; /* Bytes 20-23 */
- unsigned int MemorySize; /* Bytes 24-27 */
- unsigned int CacheSize; /* Bytes 28-31 */
- unsigned int FlashMemorySize; /* Bytes 32-35 */
- unsigned int NonVolatileMemorySize; /* Bytes 36-39 */
- struct {
- enum {
- DAC960_V1_RamType_DRAM = 0x0,
- DAC960_V1_RamType_EDO = 0x1,
- DAC960_V1_RamType_SDRAM = 0x2,
- DAC960_V1_RamType_Last = 0x7
- } __attribute__ ((packed)) RamType:3; /* Byte 40 Bits 0-2 */
- enum {
- DAC960_V1_ErrorCorrection_None = 0x0,
- DAC960_V1_ErrorCorrection_Parity = 0x1,
- DAC960_V1_ErrorCorrection_ECC = 0x2,
- DAC960_V1_ErrorCorrection_Last = 0x7
- } __attribute__ ((packed)) ErrorCorrection:3; /* Byte 40 Bits 3-5 */
- bool FastPageMode:1; /* Byte 40 Bit 6 */
- bool LowPowerMemory:1; /* Byte 40 Bit 7 */
- unsigned char :8; /* Bytes 41 */
- } MemoryType;
- unsigned short ClockSpeed; /* Bytes 42-43 */
- unsigned short MemorySpeed; /* Bytes 44-45 */
- unsigned short HardwareSpeed; /* Bytes 46-47 */
- unsigned int :32; /* Bytes 48-51 */
- unsigned int :32; /* Bytes 52-55 */
- unsigned char :8; /* Byte 56 */
- unsigned char :8; /* Byte 57 */
- unsigned short :16; /* Bytes 58-59 */
- unsigned short MaxCommands; /* Bytes 60-61 */
- unsigned short MaxScatterGatherEntries; /* Bytes 62-63 */
- unsigned short MaxDriveCommands; /* Bytes 64-65 */
- unsigned short MaxIODescriptors; /* Bytes 66-67 */
- unsigned short MaxCombinedSectors; /* Bytes 68-69 */
- unsigned char Latency; /* Byte 70 */
- unsigned char :8; /* Byte 71 */
- unsigned char SCSITimeout; /* Byte 72 */
- unsigned char :8; /* Byte 73 */
- unsigned short MinFreeLines; /* Bytes 74-75 */
- unsigned int :32; /* Bytes 76-79 */
- unsigned int :32; /* Bytes 80-83 */
- unsigned char RebuildRateConstant; /* Byte 84 */
- unsigned char :8; /* Byte 85 */
- unsigned char :8; /* Byte 86 */
- unsigned char :8; /* Byte 87 */
- unsigned int :32; /* Bytes 88-91 */
- unsigned int :32; /* Bytes 92-95 */
- unsigned short PhysicalDriveBlockSize; /* Bytes 96-97 */
- unsigned short LogicalDriveBlockSize; /* Bytes 98-99 */
- unsigned short MaxBlocksPerCommand; /* Bytes 100-101 */
- unsigned short BlockFactor; /* Bytes 102-103 */
- unsigned short CacheLineSize; /* Bytes 104-105 */
- struct {
- enum {
- DAC960_V1_Narrow_8bit = 0x0,
- DAC960_V1_Wide_16bit = 0x1,
- DAC960_V1_Wide_32bit = 0x2
- } __attribute__ ((packed)) BusWidth:2; /* Byte 106 Bits 0-1 */
- enum {
- DAC960_V1_Fast = 0x0,
- DAC960_V1_Ultra = 0x1,
- DAC960_V1_Ultra2 = 0x2
- } __attribute__ ((packed)) BusSpeed:2; /* Byte 106 Bits 2-3 */
- bool Differential:1; /* Byte 106 Bit 4 */
- unsigned char :3; /* Byte 106 Bits 5-7 */
- } SCSICapability;
- unsigned char :8; /* Byte 107 */
- unsigned int :32; /* Bytes 108-111 */
- unsigned short FirmwareBuildNumber; /* Bytes 112-113 */
- enum {
- DAC960_V1_AEMI = 0x01,
- DAC960_V1_OEM1 = 0x02,
- DAC960_V1_OEM2 = 0x04,
- DAC960_V1_OEM3 = 0x08,
- DAC960_V1_Conner = 0x10,
- DAC960_V1_SAFTE = 0x20
- } __attribute__ ((packed)) FaultManagementType; /* Byte 114 */
- unsigned char :8; /* Byte 115 */
- struct {
- bool Clustering:1; /* Byte 116 Bit 0 */
- bool MylexOnlineRAIDExpansion:1; /* Byte 116 Bit 1 */
- bool ReadAhead:1; /* Byte 116 Bit 2 */
- bool BackgroundInitialization:1; /* Byte 116 Bit 3 */
- unsigned int :28; /* Bytes 116-119 */
- } FirmwareFeatures;
- unsigned int :32; /* Bytes 120-123 */
- unsigned int :32; /* Bytes 124-127 */
-}
-DAC960_V1_Enquiry2_T;
-
-
-/*
- Define the DAC960 V1 Firmware Logical Drive State type.
-*/
-
-typedef enum
-{
- DAC960_V1_LogicalDrive_Online = 0x03,
- DAC960_V1_LogicalDrive_Critical = 0x04,
- DAC960_V1_LogicalDrive_Offline = 0xFF
-}
-__attribute__ ((packed))
-DAC960_V1_LogicalDriveState_T;
-
-
-/*
- Define the DAC960 V1 Firmware Logical Drive Information structure.
-*/
-
-typedef struct DAC960_V1_LogicalDriveInformation
-{
- unsigned int LogicalDriveSize; /* Bytes 0-3 */
- DAC960_V1_LogicalDriveState_T LogicalDriveState; /* Byte 4 */
- unsigned char RAIDLevel:7; /* Byte 5 Bits 0-6 */
- bool WriteBack:1; /* Byte 5 Bit 7 */
- unsigned short :16; /* Bytes 6-7 */
-}
-DAC960_V1_LogicalDriveInformation_T;
-
-
-/*
- Define the DAC960 V1 Firmware Get Logical Drive Information Command
- reply structure.
-*/
-
-typedef DAC960_V1_LogicalDriveInformation_T
- DAC960_V1_LogicalDriveInformationArray_T[DAC960_MaxLogicalDrives];
-
-
-/*
- Define the DAC960 V1 Firmware Perform Event Log Operation Types.
-*/
-
-typedef enum
-{
- DAC960_V1_GetEventLogEntry = 0x00
-}
-__attribute__ ((packed))
-DAC960_V1_PerformEventLogOpType_T;
-
-
-/*
- Define the DAC960 V1 Firmware Get Event Log Entry Command reply structure.
-*/
-
-typedef struct DAC960_V1_EventLogEntry
-{
- unsigned char MessageType; /* Byte 0 */
- unsigned char MessageLength; /* Byte 1 */
- unsigned char TargetID:5; /* Byte 2 Bits 0-4 */
- unsigned char Channel:3; /* Byte 2 Bits 5-7 */
- unsigned char LogicalUnit:6; /* Byte 3 Bits 0-5 */
- unsigned char :2; /* Byte 3 Bits 6-7 */
- unsigned short SequenceNumber; /* Bytes 4-5 */
- unsigned char ErrorCode:7; /* Byte 6 Bits 0-6 */
- bool Valid:1; /* Byte 6 Bit 7 */
- unsigned char SegmentNumber; /* Byte 7 */
- DAC960_SCSI_RequestSenseKey_T SenseKey:4; /* Byte 8 Bits 0-3 */
- unsigned char :1; /* Byte 8 Bit 4 */
- bool ILI:1; /* Byte 8 Bit 5 */
- bool EOM:1; /* Byte 8 Bit 6 */
- bool Filemark:1; /* Byte 8 Bit 7 */
- unsigned char Information[4]; /* Bytes 9-12 */
- unsigned char AdditionalSenseLength; /* Byte 13 */
- unsigned char CommandSpecificInformation[4]; /* Bytes 14-17 */
- unsigned char AdditionalSenseCode; /* Byte 18 */
- unsigned char AdditionalSenseCodeQualifier; /* Byte 19 */
- unsigned char Dummy[12]; /* Bytes 20-31 */
-}
-DAC960_V1_EventLogEntry_T;
-
-
-/*
- Define the DAC960 V1 Firmware Physical Device State type.
-*/
-
-typedef enum
-{
- DAC960_V1_Device_Dead = 0x00,
- DAC960_V1_Device_WriteOnly = 0x02,
- DAC960_V1_Device_Online = 0x03,
- DAC960_V1_Device_Standby = 0x10
-}
-__attribute__ ((packed))
-DAC960_V1_PhysicalDeviceState_T;
-
-
-/*
- Define the DAC960 V1 Firmware Get Device State Command reply structure.
- The structure is padded by 2 bytes for compatibility with Version 2.xx
- Firmware.
-*/
-
-typedef struct DAC960_V1_DeviceState
-{
- bool Present:1; /* Byte 0 Bit 0 */
- unsigned char :7; /* Byte 0 Bits 1-7 */
- enum {
- DAC960_V1_OtherType = 0x0,
- DAC960_V1_DiskType = 0x1,
- DAC960_V1_SequentialType = 0x2,
- DAC960_V1_CDROM_or_WORM_Type = 0x3
- } __attribute__ ((packed)) DeviceType:2; /* Byte 1 Bits 0-1 */
- bool :1; /* Byte 1 Bit 2 */
- bool Fast20:1; /* Byte 1 Bit 3 */
- bool Sync:1; /* Byte 1 Bit 4 */
- bool Fast:1; /* Byte 1 Bit 5 */
- bool Wide:1; /* Byte 1 Bit 6 */
- bool TaggedQueuingSupported:1; /* Byte 1 Bit 7 */
- DAC960_V1_PhysicalDeviceState_T DeviceState; /* Byte 2 */
- unsigned char :8; /* Byte 3 */
- unsigned char SynchronousMultiplier; /* Byte 4 */
- unsigned char SynchronousOffset:5; /* Byte 5 Bits 0-4 */
- unsigned char :3; /* Byte 5 Bits 5-7 */
- unsigned int DiskSize __attribute__ ((packed)); /* Bytes 6-9 */
- unsigned short :16; /* Bytes 10-11 */
-}
-DAC960_V1_DeviceState_T;
-
-
-/*
- Define the DAC960 V1 Firmware Get Rebuild Progress Command reply structure.
-*/
-
-typedef struct DAC960_V1_RebuildProgress
-{
- unsigned int LogicalDriveNumber; /* Bytes 0-3 */
- unsigned int LogicalDriveSize; /* Bytes 4-7 */
- unsigned int RemainingBlocks; /* Bytes 8-11 */
-}
-DAC960_V1_RebuildProgress_T;
-
-
-/*
- Define the DAC960 V1 Firmware Background Initialization Status Command
- reply structure.
-*/
-
-typedef struct DAC960_V1_BackgroundInitializationStatus
-{
- unsigned int LogicalDriveSize; /* Bytes 0-3 */
- unsigned int BlocksCompleted; /* Bytes 4-7 */
- unsigned char Reserved1[12]; /* Bytes 8-19 */
- unsigned int LogicalDriveNumber; /* Bytes 20-23 */
- unsigned char RAIDLevel; /* Byte 24 */
- enum {
- DAC960_V1_BackgroundInitializationInvalid = 0x00,
- DAC960_V1_BackgroundInitializationStarted = 0x02,
- DAC960_V1_BackgroundInitializationInProgress = 0x04,
- DAC960_V1_BackgroundInitializationSuspended = 0x05,
- DAC960_V1_BackgroundInitializationCancelled = 0x06
- } __attribute__ ((packed)) Status; /* Byte 25 */
- unsigned char Reserved2[6]; /* Bytes 26-31 */
-}
-DAC960_V1_BackgroundInitializationStatus_T;
-
-
-/*
- Define the DAC960 V1 Firmware Error Table Entry structure.
-*/
-
-typedef struct DAC960_V1_ErrorTableEntry
-{
- unsigned char ParityErrorCount; /* Byte 0 */
- unsigned char SoftErrorCount; /* Byte 1 */
- unsigned char HardErrorCount; /* Byte 2 */
- unsigned char MiscErrorCount; /* Byte 3 */
-}
-DAC960_V1_ErrorTableEntry_T;
-
-
-/*
- Define the DAC960 V1 Firmware Get Error Table Command reply structure.
-*/
-
-typedef struct DAC960_V1_ErrorTable
-{
- DAC960_V1_ErrorTableEntry_T
- ErrorTableEntries[DAC960_V1_MaxChannels][DAC960_V1_MaxTargets];
-}
-DAC960_V1_ErrorTable_T;
-
-
-/*
- Define the DAC960 V1 Firmware Read Config2 Command reply structure.
-*/
-
-typedef struct DAC960_V1_Config2
-{
- unsigned char :1; /* Byte 0 Bit 0 */
- bool ActiveNegationEnabled:1; /* Byte 0 Bit 1 */
- unsigned char :5; /* Byte 0 Bits 2-6 */
- bool NoRescanIfResetReceivedDuringScan:1; /* Byte 0 Bit 7 */
- bool StorageWorksSupportEnabled:1; /* Byte 1 Bit 0 */
- bool HewlettPackardSupportEnabled:1; /* Byte 1 Bit 1 */
- bool NoDisconnectOnFirstCommand:1; /* Byte 1 Bit 2 */
- unsigned char :2; /* Byte 1 Bits 3-4 */
- bool AEMI_ARM:1; /* Byte 1 Bit 5 */
- bool AEMI_OFM:1; /* Byte 1 Bit 6 */
- unsigned char :1; /* Byte 1 Bit 7 */
- enum {
- DAC960_V1_OEMID_Mylex = 0x00,
- DAC960_V1_OEMID_IBM = 0x08,
- DAC960_V1_OEMID_HP = 0x0A,
- DAC960_V1_OEMID_DEC = 0x0C,
- DAC960_V1_OEMID_Siemens = 0x10,
- DAC960_V1_OEMID_Intel = 0x12
- } __attribute__ ((packed)) OEMID; /* Byte 2 */
- unsigned char OEMModelNumber; /* Byte 3 */
- unsigned char PhysicalSector; /* Byte 4 */
- unsigned char LogicalSector; /* Byte 5 */
- unsigned char BlockFactor; /* Byte 6 */
- bool ReadAheadEnabled:1; /* Byte 7 Bit 0 */
- bool LowBIOSDelay:1; /* Byte 7 Bit 1 */
- unsigned char :2; /* Byte 7 Bits 2-3 */
- bool ReassignRestrictedToOneSector:1; /* Byte 7 Bit 4 */
- unsigned char :1; /* Byte 7 Bit 5 */
- bool ForceUnitAccessDuringWriteRecovery:1; /* Byte 7 Bit 6 */
- bool EnableLeftSymmetricRAID5Algorithm:1; /* Byte 7 Bit 7 */
- unsigned char DefaultRebuildRate; /* Byte 8 */
- unsigned char :8; /* Byte 9 */
- unsigned char BlocksPerCacheLine; /* Byte 10 */
- unsigned char BlocksPerStripe; /* Byte 11 */
- struct {
- enum {
- DAC960_V1_Async = 0x0,
- DAC960_V1_Sync_8MHz = 0x1,
- DAC960_V1_Sync_5MHz = 0x2,
- DAC960_V1_Sync_10or20MHz = 0x3 /* Byte 11 Bits 0-1 */
- } __attribute__ ((packed)) Speed:2;
- bool Force8Bit:1; /* Byte 11 Bit 2 */
- bool DisableFast20:1; /* Byte 11 Bit 3 */
- unsigned char :3; /* Byte 11 Bits 4-6 */
- bool EnableTaggedQueuing:1; /* Byte 11 Bit 7 */
- } __attribute__ ((packed)) ChannelParameters[6]; /* Bytes 12-17 */
- unsigned char SCSIInitiatorID; /* Byte 18 */
- unsigned char :8; /* Byte 19 */
- enum {
- DAC960_V1_StartupMode_ControllerSpinUp = 0x00,
- DAC960_V1_StartupMode_PowerOnSpinUp = 0x01
- } __attribute__ ((packed)) StartupMode; /* Byte 20 */
- unsigned char SimultaneousDeviceSpinUpCount; /* Byte 21 */
- unsigned char SecondsDelayBetweenSpinUps; /* Byte 22 */
- unsigned char Reserved1[29]; /* Bytes 23-51 */
- bool BIOSDisabled:1; /* Byte 52 Bit 0 */
- bool CDROMBootEnabled:1; /* Byte 52 Bit 1 */
- unsigned char :3; /* Byte 52 Bits 2-4 */
- enum {
- DAC960_V1_Geometry_128_32 = 0x0,
- DAC960_V1_Geometry_255_63 = 0x1,
- DAC960_V1_Geometry_Reserved1 = 0x2,
- DAC960_V1_Geometry_Reserved2 = 0x3
- } __attribute__ ((packed)) DriveGeometry:2; /* Byte 52 Bits 5-6 */
- unsigned char :1; /* Byte 52 Bit 7 */
- unsigned char Reserved2[9]; /* Bytes 53-61 */
- unsigned short Checksum; /* Bytes 62-63 */
-}
-DAC960_V1_Config2_T;
-
-
-/*
- Define the DAC960 V1 Firmware DCDB request structure.
-*/
-
-typedef struct DAC960_V1_DCDB
-{
- unsigned char TargetID:4; /* Byte 0 Bits 0-3 */
- unsigned char Channel:4; /* Byte 0 Bits 4-7 */
- enum {
- DAC960_V1_DCDB_NoDataTransfer = 0,
- DAC960_V1_DCDB_DataTransferDeviceToSystem = 1,
- DAC960_V1_DCDB_DataTransferSystemToDevice = 2,
- DAC960_V1_DCDB_IllegalDataTransfer = 3
- } __attribute__ ((packed)) Direction:2; /* Byte 1 Bits 0-1 */
- bool EarlyStatus:1; /* Byte 1 Bit 2 */
- unsigned char :1; /* Byte 1 Bit 3 */
- enum {
- DAC960_V1_DCDB_Timeout_24_hours = 0,
- DAC960_V1_DCDB_Timeout_10_seconds = 1,
- DAC960_V1_DCDB_Timeout_60_seconds = 2,
- DAC960_V1_DCDB_Timeout_10_minutes = 3
- } __attribute__ ((packed)) Timeout:2; /* Byte 1 Bits 4-5 */
- bool NoAutomaticRequestSense:1; /* Byte 1 Bit 6 */
- bool DisconnectPermitted:1; /* Byte 1 Bit 7 */
- unsigned short TransferLength; /* Bytes 2-3 */
- DAC960_BusAddress32_T BusAddress; /* Bytes 4-7 */
- unsigned char CDBLength:4; /* Byte 8 Bits 0-3 */
- unsigned char TransferLengthHigh4:4; /* Byte 8 Bits 4-7 */
- unsigned char SenseLength; /* Byte 9 */
- unsigned char CDB[12]; /* Bytes 10-21 */
- unsigned char SenseData[64]; /* Bytes 22-85 */
- unsigned char Status; /* Byte 86 */
- unsigned char :8; /* Byte 87 */
-}
-DAC960_V1_DCDB_T;
-
-
-/*
- Define the DAC960 V1 Firmware Scatter/Gather List Type 1 32 Bit Address
- 32 Bit Byte Count structure.
-*/
-
-typedef struct DAC960_V1_ScatterGatherSegment
-{
- DAC960_BusAddress32_T SegmentDataPointer; /* Bytes 0-3 */
- DAC960_ByteCount32_T SegmentByteCount; /* Bytes 4-7 */
-}
-DAC960_V1_ScatterGatherSegment_T;
-
-
-/*
- Define the 13 Byte DAC960 V1 Firmware Command Mailbox structure. Bytes 13-15
- are not used. The Command Mailbox structure is padded to 16 bytes for
- efficient access.
-*/
-
-typedef union DAC960_V1_CommandMailbox
-{
- unsigned int Words[4]; /* Words 0-3 */
- unsigned char Bytes[16]; /* Bytes 0-15 */
- struct {
- DAC960_V1_CommandOpcode_T CommandOpcode; /* Byte 0 */
- DAC960_V1_CommandIdentifier_T CommandIdentifier; /* Byte 1 */
- unsigned char Dummy[14]; /* Bytes 2-15 */
- } __attribute__ ((packed)) Common;
- struct {
- DAC960_V1_CommandOpcode_T CommandOpcode; /* Byte 0 */
- DAC960_V1_CommandIdentifier_T CommandIdentifier; /* Byte 1 */
- unsigned char Dummy1[6]; /* Bytes 2-7 */
- DAC960_BusAddress32_T BusAddress; /* Bytes 8-11 */
- unsigned char Dummy2[4]; /* Bytes 12-15 */
- } __attribute__ ((packed)) Type3;
- struct {
- DAC960_V1_CommandOpcode_T CommandOpcode; /* Byte 0 */
- DAC960_V1_CommandIdentifier_T CommandIdentifier; /* Byte 1 */
- unsigned char CommandOpcode2; /* Byte 2 */
- unsigned char Dummy1[5]; /* Bytes 3-7 */
- DAC960_BusAddress32_T BusAddress; /* Bytes 8-11 */
- unsigned char Dummy2[4]; /* Bytes 12-15 */
- } __attribute__ ((packed)) Type3B;
- struct {
- DAC960_V1_CommandOpcode_T CommandOpcode; /* Byte 0 */
- DAC960_V1_CommandIdentifier_T CommandIdentifier; /* Byte 1 */
- unsigned char Dummy1[5]; /* Bytes 2-6 */
- unsigned char LogicalDriveNumber:6; /* Byte 7 Bits 0-6 */
- bool AutoRestore:1; /* Byte 7 Bit 7 */
- unsigned char Dummy2[8]; /* Bytes 8-15 */
- } __attribute__ ((packed)) Type3C;
- struct {
- DAC960_V1_CommandOpcode_T CommandOpcode; /* Byte 0 */
- DAC960_V1_CommandIdentifier_T CommandIdentifier; /* Byte 1 */
- unsigned char Channel; /* Byte 2 */
- unsigned char TargetID; /* Byte 3 */
- DAC960_V1_PhysicalDeviceState_T DeviceState:5; /* Byte 4 Bits 0-4 */
- unsigned char Modifier:3; /* Byte 4 Bits 5-7 */
- unsigned char Dummy1[3]; /* Bytes 5-7 */
- DAC960_BusAddress32_T BusAddress; /* Bytes 8-11 */
- unsigned char Dummy2[4]; /* Bytes 12-15 */
- } __attribute__ ((packed)) Type3D;
- struct {
- DAC960_V1_CommandOpcode_T CommandOpcode; /* Byte 0 */
- DAC960_V1_CommandIdentifier_T CommandIdentifier; /* Byte 1 */
- DAC960_V1_PerformEventLogOpType_T OperationType; /* Byte 2 */
- unsigned char OperationQualifier; /* Byte 3 */
- unsigned short SequenceNumber; /* Bytes 4-5 */
- unsigned char Dummy1[2]; /* Bytes 6-7 */
- DAC960_BusAddress32_T BusAddress; /* Bytes 8-11 */
- unsigned char Dummy2[4]; /* Bytes 12-15 */
- } __attribute__ ((packed)) Type3E;
- struct {
- DAC960_V1_CommandOpcode_T CommandOpcode; /* Byte 0 */
- DAC960_V1_CommandIdentifier_T CommandIdentifier; /* Byte 1 */
- unsigned char Dummy1[2]; /* Bytes 2-3 */
- unsigned char RebuildRateConstant; /* Byte 4 */
- unsigned char Dummy2[3]; /* Bytes 5-7 */
- DAC960_BusAddress32_T BusAddress; /* Bytes 8-11 */
- unsigned char Dummy3[4]; /* Bytes 12-15 */
- } __attribute__ ((packed)) Type3R;
- struct {
- DAC960_V1_CommandOpcode_T CommandOpcode; /* Byte 0 */
- DAC960_V1_CommandIdentifier_T CommandIdentifier; /* Byte 1 */
- unsigned short TransferLength; /* Bytes 2-3 */
- unsigned int LogicalBlockAddress; /* Bytes 4-7 */
- DAC960_BusAddress32_T BusAddress; /* Bytes 8-11 */
- unsigned char LogicalDriveNumber; /* Byte 12 */
- unsigned char Dummy[3]; /* Bytes 13-15 */
- } __attribute__ ((packed)) Type4;
- struct {
- DAC960_V1_CommandOpcode_T CommandOpcode; /* Byte 0 */
- DAC960_V1_CommandIdentifier_T CommandIdentifier; /* Byte 1 */
- struct {
- unsigned short TransferLength:11; /* Bytes 2-3 */
- unsigned char LogicalDriveNumber:5; /* Byte 3 Bits 3-7 */
- } __attribute__ ((packed)) LD;
- unsigned int LogicalBlockAddress; /* Bytes 4-7 */
- DAC960_BusAddress32_T BusAddress; /* Bytes 8-11 */
- unsigned char ScatterGatherCount:6; /* Byte 12 Bits 0-5 */
- enum {
- DAC960_V1_ScatterGather_32BitAddress_32BitByteCount = 0x0,
- DAC960_V1_ScatterGather_32BitAddress_16BitByteCount = 0x1,
- DAC960_V1_ScatterGather_32BitByteCount_32BitAddress = 0x2,
- DAC960_V1_ScatterGather_16BitByteCount_32BitAddress = 0x3
- } __attribute__ ((packed)) ScatterGatherType:2; /* Byte 12 Bits 6-7 */
- unsigned char Dummy[3]; /* Bytes 13-15 */
- } __attribute__ ((packed)) Type5;
- struct {
- DAC960_V1_CommandOpcode_T CommandOpcode; /* Byte 0 */
- DAC960_V1_CommandIdentifier_T CommandIdentifier; /* Byte 1 */
- unsigned char CommandOpcode2; /* Byte 2 */
- unsigned char :8; /* Byte 3 */
- DAC960_BusAddress32_T CommandMailboxesBusAddress; /* Bytes 4-7 */
- DAC960_BusAddress32_T StatusMailboxesBusAddress; /* Bytes 8-11 */
- unsigned char Dummy[4]; /* Bytes 12-15 */
- } __attribute__ ((packed)) TypeX;
-}
-DAC960_V1_CommandMailbox_T;
-
-
-/*
- Define the DAC960 V2 Firmware Command Opcodes.
-*/
-
-typedef enum
-{
- DAC960_V2_MemCopy = 0x01,
- DAC960_V2_SCSI_10_Passthru = 0x02,
- DAC960_V2_SCSI_255_Passthru = 0x03,
- DAC960_V2_SCSI_10 = 0x04,
- DAC960_V2_SCSI_256 = 0x05,
- DAC960_V2_IOCTL = 0x20
-}
-__attribute__ ((packed))
-DAC960_V2_CommandOpcode_T;
-
-
-/*
- Define the DAC960 V2 Firmware IOCTL Opcodes.
-*/
-
-typedef enum
-{
- DAC960_V2_GetControllerInfo = 0x01,
- DAC960_V2_GetLogicalDeviceInfoValid = 0x03,
- DAC960_V2_GetPhysicalDeviceInfoValid = 0x05,
- DAC960_V2_GetHealthStatus = 0x11,
- DAC960_V2_GetEvent = 0x15,
- DAC960_V2_StartDiscovery = 0x81,
- DAC960_V2_SetDeviceState = 0x82,
- DAC960_V2_RebuildDeviceStart = 0x88,
- DAC960_V2_RebuildDeviceStop = 0x89,
- DAC960_V2_ConsistencyCheckStart = 0x8C,
- DAC960_V2_ConsistencyCheckStop = 0x8D,
- DAC960_V2_SetMemoryMailbox = 0x8E,
- DAC960_V2_PauseDevice = 0x92,
- DAC960_V2_TranslatePhysicalToLogicalDevice = 0xC5
-}
-__attribute__ ((packed))
-DAC960_V2_IOCTL_Opcode_T;
-
-
-/*
- Define the DAC960 V2 Firmware Command Identifier type.
-*/
-
-typedef unsigned short DAC960_V2_CommandIdentifier_T;
-
-
-/*
- Define the DAC960 V2 Firmware Command Status Codes.
-*/
-
-#define DAC960_V2_NormalCompletion 0x00
-#define DAC960_V2_AbormalCompletion 0x02
-#define DAC960_V2_DeviceBusy 0x08
-#define DAC960_V2_DeviceNonresponsive 0x0E
-#define DAC960_V2_DeviceNonresponsive2 0x0F
-#define DAC960_V2_DeviceRevervationConflict 0x18
-
-typedef unsigned char DAC960_V2_CommandStatus_T;
-
-
-/*
- Define the DAC960 V2 Firmware Memory Type structure.
-*/
-
-typedef struct DAC960_V2_MemoryType
-{
- enum {
- DAC960_V2_MemoryType_Reserved = 0x00,
- DAC960_V2_MemoryType_DRAM = 0x01,
- DAC960_V2_MemoryType_EDRAM = 0x02,
- DAC960_V2_MemoryType_EDO = 0x03,
- DAC960_V2_MemoryType_SDRAM = 0x04,
- DAC960_V2_MemoryType_Last = 0x1F
- } __attribute__ ((packed)) MemoryType:5; /* Byte 0 Bits 0-4 */
- bool :1; /* Byte 0 Bit 5 */
- bool MemoryParity:1; /* Byte 0 Bit 6 */
- bool MemoryECC:1; /* Byte 0 Bit 7 */
-}
-DAC960_V2_MemoryType_T;
-
-
-/*
- Define the DAC960 V2 Firmware Processor Type structure.
-*/
-
-typedef enum
-{
- DAC960_V2_ProcessorType_i960CA = 0x01,
- DAC960_V2_ProcessorType_i960RD = 0x02,
- DAC960_V2_ProcessorType_i960RN = 0x03,
- DAC960_V2_ProcessorType_i960RP = 0x04,
- DAC960_V2_ProcessorType_NorthBay = 0x05,
- DAC960_V2_ProcessorType_StrongArm = 0x06,
- DAC960_V2_ProcessorType_i960RM = 0x07
-}
-__attribute__ ((packed))
-DAC960_V2_ProcessorType_T;
-
-
-/*
- Define the DAC960 V2 Firmware Get Controller Info reply structure.
-*/
-
-typedef struct DAC960_V2_ControllerInfo
-{
- unsigned char :8; /* Byte 0 */
- enum {
- DAC960_V2_SCSI_Bus = 0x00,
- DAC960_V2_Fibre_Bus = 0x01,
- DAC960_V2_PCI_Bus = 0x03
- } __attribute__ ((packed)) BusInterfaceType; /* Byte 1 */
- enum {
- DAC960_V2_DAC960E = 0x01,
- DAC960_V2_DAC960M = 0x08,
- DAC960_V2_DAC960PD = 0x10,
- DAC960_V2_DAC960PL = 0x11,
- DAC960_V2_DAC960PU = 0x12,
- DAC960_V2_DAC960PE = 0x13,
- DAC960_V2_DAC960PG = 0x14,
- DAC960_V2_DAC960PJ = 0x15,
- DAC960_V2_DAC960PTL0 = 0x16,
- DAC960_V2_DAC960PR = 0x17,
- DAC960_V2_DAC960PRL = 0x18,
- DAC960_V2_DAC960PT = 0x19,
- DAC960_V2_DAC1164P = 0x1A,
- DAC960_V2_DAC960PTL1 = 0x1B,
- DAC960_V2_EXR2000P = 0x1C,
- DAC960_V2_EXR3000P = 0x1D,
- DAC960_V2_AcceleRAID352 = 0x1E,
- DAC960_V2_AcceleRAID170 = 0x1F,
- DAC960_V2_AcceleRAID160 = 0x20,
- DAC960_V2_DAC960S = 0x60,
- DAC960_V2_DAC960SU = 0x61,
- DAC960_V2_DAC960SX = 0x62,
- DAC960_V2_DAC960SF = 0x63,
- DAC960_V2_DAC960SS = 0x64,
- DAC960_V2_DAC960FL = 0x65,
- DAC960_V2_DAC960LL = 0x66,
- DAC960_V2_DAC960FF = 0x67,
- DAC960_V2_DAC960HP = 0x68,
- DAC960_V2_RAIDBRICK = 0x69,
- DAC960_V2_METEOR_FL = 0x6A,
- DAC960_V2_METEOR_FF = 0x6B
- } __attribute__ ((packed)) ControllerType; /* Byte 2 */
- unsigned char :8; /* Byte 3 */
- unsigned short BusInterfaceSpeedMHz; /* Bytes 4-5 */
- unsigned char BusWidthBits; /* Byte 6 */
- unsigned char FlashCodeTypeOrProductID; /* Byte 7 */
- unsigned char NumberOfHostPortsPresent; /* Byte 8 */
- unsigned char Reserved1[7]; /* Bytes 9-15 */
- unsigned char BusInterfaceName[16]; /* Bytes 16-31 */
- unsigned char ControllerName[16]; /* Bytes 32-47 */
- unsigned char Reserved2[16]; /* Bytes 48-63 */
- /* Firmware Release Information */
- unsigned char FirmwareMajorVersion; /* Byte 64 */
- unsigned char FirmwareMinorVersion; /* Byte 65 */
- unsigned char FirmwareTurnNumber; /* Byte 66 */
- unsigned char FirmwareBuildNumber; /* Byte 67 */
- unsigned char FirmwareReleaseDay; /* Byte 68 */
- unsigned char FirmwareReleaseMonth; /* Byte 69 */
- unsigned char FirmwareReleaseYearHigh2Digits; /* Byte 70 */
- unsigned char FirmwareReleaseYearLow2Digits; /* Byte 71 */
- /* Hardware Release Information */
- unsigned char HardwareRevision; /* Byte 72 */
- unsigned int :24; /* Bytes 73-75 */
- unsigned char HardwareReleaseDay; /* Byte 76 */
- unsigned char HardwareReleaseMonth; /* Byte 77 */
- unsigned char HardwareReleaseYearHigh2Digits; /* Byte 78 */
- unsigned char HardwareReleaseYearLow2Digits; /* Byte 79 */
- /* Hardware Manufacturing Information */
- unsigned char ManufacturingBatchNumber; /* Byte 80 */
- unsigned char :8; /* Byte 81 */
- unsigned char ManufacturingPlantNumber; /* Byte 82 */
- unsigned char :8; /* Byte 83 */
- unsigned char HardwareManufacturingDay; /* Byte 84 */
- unsigned char HardwareManufacturingMonth; /* Byte 85 */
- unsigned char HardwareManufacturingYearHigh2Digits; /* Byte 86 */
- unsigned char HardwareManufacturingYearLow2Digits; /* Byte 87 */
- unsigned char MaximumNumberOfPDDperXLD; /* Byte 88 */
- unsigned char MaximumNumberOfILDperXLD; /* Byte 89 */
- unsigned short NonvolatileMemorySizeKB; /* Bytes 90-91 */
- unsigned char MaximumNumberOfXLD; /* Byte 92 */
- unsigned int :24; /* Bytes 93-95 */
- /* Unique Information per Controller */
- unsigned char ControllerSerialNumber[16]; /* Bytes 96-111 */
- unsigned char Reserved3[16]; /* Bytes 112-127 */
- /* Vendor Information */
- unsigned int :24; /* Bytes 128-130 */
- unsigned char OEM_Code; /* Byte 131 */
- unsigned char VendorName[16]; /* Bytes 132-147 */
- /* Other Physical/Controller/Operation Information */
- bool BBU_Present:1; /* Byte 148 Bit 0 */
- bool ActiveActiveClusteringMode:1; /* Byte 148 Bit 1 */
- unsigned char :6; /* Byte 148 Bits 2-7 */
- unsigned char :8; /* Byte 149 */
- unsigned short :16; /* Bytes 150-151 */
- /* Physical Device Scan Information */
- bool PhysicalScanActive:1; /* Byte 152 Bit 0 */
- unsigned char :7; /* Byte 152 Bits 1-7 */
- unsigned char PhysicalDeviceChannelNumber; /* Byte 153 */
- unsigned char PhysicalDeviceTargetID; /* Byte 154 */
- unsigned char PhysicalDeviceLogicalUnit; /* Byte 155 */
- /* Maximum Command Data Transfer Sizes */
- unsigned short MaximumDataTransferSizeInBlocks; /* Bytes 156-157 */
- unsigned short MaximumScatterGatherEntries; /* Bytes 158-159 */
- /* Logical/Physical Device Counts */
- unsigned short LogicalDevicesPresent; /* Bytes 160-161 */
- unsigned short LogicalDevicesCritical; /* Bytes 162-163 */
- unsigned short LogicalDevicesOffline; /* Bytes 164-165 */
- unsigned short PhysicalDevicesPresent; /* Bytes 166-167 */
- unsigned short PhysicalDisksPresent; /* Bytes 168-169 */
- unsigned short PhysicalDisksCritical; /* Bytes 170-171 */
- unsigned short PhysicalDisksOffline; /* Bytes 172-173 */
- unsigned short MaximumParallelCommands; /* Bytes 174-175 */
- /* Channel and Target ID Information */
- unsigned char NumberOfPhysicalChannelsPresent; /* Byte 176 */
- unsigned char NumberOfVirtualChannelsPresent; /* Byte 177 */
- unsigned char NumberOfPhysicalChannelsPossible; /* Byte 178 */
- unsigned char NumberOfVirtualChannelsPossible; /* Byte 179 */
- unsigned char MaximumTargetsPerChannel[16]; /* Bytes 180-195 */
- unsigned char Reserved4[12]; /* Bytes 196-207 */
- /* Memory/Cache Information */
- unsigned short MemorySizeMB; /* Bytes 208-209 */
- unsigned short CacheSizeMB; /* Bytes 210-211 */
- unsigned int ValidCacheSizeInBytes; /* Bytes 212-215 */
- unsigned int DirtyCacheSizeInBytes; /* Bytes 216-219 */
- unsigned short MemorySpeedMHz; /* Bytes 220-221 */
- unsigned char MemoryDataWidthBits; /* Byte 222 */
- DAC960_V2_MemoryType_T MemoryType; /* Byte 223 */
- unsigned char CacheMemoryTypeName[16]; /* Bytes 224-239 */
- /* Execution Memory Information */
- unsigned short ExecutionMemorySizeMB; /* Bytes 240-241 */
- unsigned short ExecutionL2CacheSizeMB; /* Bytes 242-243 */
- unsigned char Reserved5[8]; /* Bytes 244-251 */
- unsigned short ExecutionMemorySpeedMHz; /* Bytes 252-253 */
- unsigned char ExecutionMemoryDataWidthBits; /* Byte 254 */
- DAC960_V2_MemoryType_T ExecutionMemoryType; /* Byte 255 */
- unsigned char ExecutionMemoryTypeName[16]; /* Bytes 256-271 */
- /* First CPU Type Information */
- unsigned short FirstProcessorSpeedMHz; /* Bytes 272-273 */
- DAC960_V2_ProcessorType_T FirstProcessorType; /* Byte 274 */
- unsigned char FirstProcessorCount; /* Byte 275 */
- unsigned char Reserved6[12]; /* Bytes 276-287 */
- unsigned char FirstProcessorName[16]; /* Bytes 288-303 */
- /* Second CPU Type Information */
- unsigned short SecondProcessorSpeedMHz; /* Bytes 304-305 */
- DAC960_V2_ProcessorType_T SecondProcessorType; /* Byte 306 */
- unsigned char SecondProcessorCount; /* Byte 307 */
- unsigned char Reserved7[12]; /* Bytes 308-319 */
- unsigned char SecondProcessorName[16]; /* Bytes 320-335 */
- /* Debugging/Profiling/Command Time Tracing Information */
- unsigned short CurrentProfilingDataPageNumber; /* Bytes 336-337 */
- unsigned short ProgramsAwaitingProfilingData; /* Bytes 338-339 */
- unsigned short CurrentCommandTimeTraceDataPageNumber; /* Bytes 340-341 */
- unsigned short ProgramsAwaitingCommandTimeTraceData; /* Bytes 342-343 */
- unsigned char Reserved8[8]; /* Bytes 344-351 */
- /* Error Counters on Physical Devices */
- unsigned short PhysicalDeviceBusResets; /* Bytes 352-353 */
- unsigned short PhysicalDeviceParityErrors; /* Bytes 355-355 */
- unsigned short PhysicalDeviceSoftErrors; /* Bytes 356-357 */
- unsigned short PhysicalDeviceCommandsFailed; /* Bytes 358-359 */
- unsigned short PhysicalDeviceMiscellaneousErrors; /* Bytes 360-361 */
- unsigned short PhysicalDeviceCommandTimeouts; /* Bytes 362-363 */
- unsigned short PhysicalDeviceSelectionTimeouts; /* Bytes 364-365 */
- unsigned short PhysicalDeviceRetriesDone; /* Bytes 366-367 */
- unsigned short PhysicalDeviceAbortsDone; /* Bytes 368-369 */
- unsigned short PhysicalDeviceHostCommandAbortsDone; /* Bytes 370-371 */
- unsigned short PhysicalDevicePredictedFailuresDetected; /* Bytes 372-373 */
- unsigned short PhysicalDeviceHostCommandsFailed; /* Bytes 374-375 */
- unsigned short PhysicalDeviceHardErrors; /* Bytes 376-377 */
- unsigned char Reserved9[6]; /* Bytes 378-383 */
- /* Error Counters on Logical Devices */
- unsigned short LogicalDeviceSoftErrors; /* Bytes 384-385 */
- unsigned short LogicalDeviceCommandsFailed; /* Bytes 386-387 */
- unsigned short LogicalDeviceHostCommandAbortsDone; /* Bytes 388-389 */
- unsigned short :16; /* Bytes 390-391 */
- /* Error Counters on Controller */
- unsigned short ControllerMemoryErrors; /* Bytes 392-393 */
- unsigned short ControllerHostCommandAbortsDone; /* Bytes 394-395 */
- unsigned int :32; /* Bytes 396-399 */
- /* Long Duration Activity Information */
- unsigned short BackgroundInitializationsActive; /* Bytes 400-401 */
- unsigned short LogicalDeviceInitializationsActive; /* Bytes 402-403 */
- unsigned short PhysicalDeviceInitializationsActive; /* Bytes 404-405 */
- unsigned short ConsistencyChecksActive; /* Bytes 406-407 */
- unsigned short RebuildsActive; /* Bytes 408-409 */
- unsigned short OnlineExpansionsActive; /* Bytes 410-411 */
- unsigned short PatrolActivitiesActive; /* Bytes 412-413 */
- unsigned short :16; /* Bytes 414-415 */
- /* Flash ROM Information */
- unsigned char FlashType; /* Byte 416 */
- unsigned char :8; /* Byte 417 */
- unsigned short FlashSizeMB; /* Bytes 418-419 */
- unsigned int FlashLimit; /* Bytes 420-423 */
- unsigned int FlashCount; /* Bytes 424-427 */
- unsigned int :32; /* Bytes 428-431 */
- unsigned char FlashTypeName[16]; /* Bytes 432-447 */
- /* Firmware Run Time Information */
- unsigned char RebuildRate; /* Byte 448 */
- unsigned char BackgroundInitializationRate; /* Byte 449 */
- unsigned char ForegroundInitializationRate; /* Byte 450 */
- unsigned char ConsistencyCheckRate; /* Byte 451 */
- unsigned int :32; /* Bytes 452-455 */
- unsigned int MaximumDP; /* Bytes 456-459 */
- unsigned int FreeDP; /* Bytes 460-463 */
- unsigned int MaximumIOP; /* Bytes 464-467 */
- unsigned int FreeIOP; /* Bytes 468-471 */
- unsigned short MaximumCombLengthInBlocks; /* Bytes 472-473 */
- unsigned short NumberOfConfigurationGroups; /* Bytes 474-475 */
- bool InstallationAbortStatus:1; /* Byte 476 Bit 0 */
- bool MaintenanceModeStatus:1; /* Byte 476 Bit 1 */
- unsigned int :24; /* Bytes 476-479 */
- unsigned char Reserved10[32]; /* Bytes 480-511 */
- unsigned char Reserved11[512]; /* Bytes 512-1023 */
-}
-DAC960_V2_ControllerInfo_T;
-
-
-/*
- Define the DAC960 V2 Firmware Logical Device State type.
-*/
-
-typedef enum
-{
- DAC960_V2_LogicalDevice_Online = 0x01,
- DAC960_V2_LogicalDevice_Offline = 0x08,
- DAC960_V2_LogicalDevice_Critical = 0x09
-}
-__attribute__ ((packed))
-DAC960_V2_LogicalDeviceState_T;
-
-
-/*
- Define the DAC960 V2 Firmware Get Logical Device Info reply structure.
-*/
-
-typedef struct DAC960_V2_LogicalDeviceInfo
-{
- unsigned char :8; /* Byte 0 */
- unsigned char Channel; /* Byte 1 */
- unsigned char TargetID; /* Byte 2 */
- unsigned char LogicalUnit; /* Byte 3 */
- DAC960_V2_LogicalDeviceState_T LogicalDeviceState; /* Byte 4 */
- unsigned char RAIDLevel; /* Byte 5 */
- unsigned char StripeSize; /* Byte 6 */
- unsigned char CacheLineSize; /* Byte 7 */
- struct {
- enum {
- DAC960_V2_ReadCacheDisabled = 0x0,
- DAC960_V2_ReadCacheEnabled = 0x1,
- DAC960_V2_ReadAheadEnabled = 0x2,
- DAC960_V2_IntelligentReadAheadEnabled = 0x3,
- DAC960_V2_ReadCache_Last = 0x7
- } __attribute__ ((packed)) ReadCache:3; /* Byte 8 Bits 0-2 */
- enum {
- DAC960_V2_WriteCacheDisabled = 0x0,
- DAC960_V2_LogicalDeviceReadOnly = 0x1,
- DAC960_V2_WriteCacheEnabled = 0x2,
- DAC960_V2_IntelligentWriteCacheEnabled = 0x3,
- DAC960_V2_WriteCache_Last = 0x7
- } __attribute__ ((packed)) WriteCache:3; /* Byte 8 Bits 3-5 */
- bool :1; /* Byte 8 Bit 6 */
- bool LogicalDeviceInitialized:1; /* Byte 8 Bit 7 */
- } LogicalDeviceControl; /* Byte 8 */
- /* Logical Device Operations Status */
- bool ConsistencyCheckInProgress:1; /* Byte 9 Bit 0 */
- bool RebuildInProgress:1; /* Byte 9 Bit 1 */
- bool BackgroundInitializationInProgress:1; /* Byte 9 Bit 2 */
- bool ForegroundInitializationInProgress:1; /* Byte 9 Bit 3 */
- bool DataMigrationInProgress:1; /* Byte 9 Bit 4 */
- bool PatrolOperationInProgress:1; /* Byte 9 Bit 5 */
- unsigned char :2; /* Byte 9 Bits 6-7 */
- unsigned char RAID5WriteUpdate; /* Byte 10 */
- unsigned char RAID5Algorithm; /* Byte 11 */
- unsigned short LogicalDeviceNumber; /* Bytes 12-13 */
- /* BIOS Info */
- bool BIOSDisabled:1; /* Byte 14 Bit 0 */
- bool CDROMBootEnabled:1; /* Byte 14 Bit 1 */
- bool DriveCoercionEnabled:1; /* Byte 14 Bit 2 */
- bool WriteSameDisabled:1; /* Byte 14 Bit 3 */
- bool HBA_ModeEnabled:1; /* Byte 14 Bit 4 */
- enum {
- DAC960_V2_Geometry_128_32 = 0x0,
- DAC960_V2_Geometry_255_63 = 0x1,
- DAC960_V2_Geometry_Reserved1 = 0x2,
- DAC960_V2_Geometry_Reserved2 = 0x3
- } __attribute__ ((packed)) DriveGeometry:2; /* Byte 14 Bits 5-6 */
- bool SuperReadAheadEnabled:1; /* Byte 14 Bit 7 */
- unsigned char :8; /* Byte 15 */
- /* Error Counters */
- unsigned short SoftErrors; /* Bytes 16-17 */
- unsigned short CommandsFailed; /* Bytes 18-19 */
- unsigned short HostCommandAbortsDone; /* Bytes 20-21 */
- unsigned short DeferredWriteErrors; /* Bytes 22-23 */
- unsigned int :32; /* Bytes 24-27 */
- unsigned int :32; /* Bytes 28-31 */
- /* Device Size Information */
- unsigned short :16; /* Bytes 32-33 */
- unsigned short DeviceBlockSizeInBytes; /* Bytes 34-35 */
- unsigned int OriginalDeviceSize; /* Bytes 36-39 */
- unsigned int ConfigurableDeviceSize; /* Bytes 40-43 */
- unsigned int :32; /* Bytes 44-47 */
- unsigned char LogicalDeviceName[32]; /* Bytes 48-79 */
- unsigned char SCSI_InquiryData[36]; /* Bytes 80-115 */
- unsigned char Reserved1[12]; /* Bytes 116-127 */
- DAC960_ByteCount64_T LastReadBlockNumber; /* Bytes 128-135 */
- DAC960_ByteCount64_T LastWrittenBlockNumber; /* Bytes 136-143 */
- DAC960_ByteCount64_T ConsistencyCheckBlockNumber; /* Bytes 144-151 */
- DAC960_ByteCount64_T RebuildBlockNumber; /* Bytes 152-159 */
- DAC960_ByteCount64_T BackgroundInitializationBlockNumber; /* Bytes 160-167 */
- DAC960_ByteCount64_T ForegroundInitializationBlockNumber; /* Bytes 168-175 */
- DAC960_ByteCount64_T DataMigrationBlockNumber; /* Bytes 176-183 */
- DAC960_ByteCount64_T PatrolOperationBlockNumber; /* Bytes 184-191 */
- unsigned char Reserved2[64]; /* Bytes 192-255 */
-}
-DAC960_V2_LogicalDeviceInfo_T;
-
-
-/*
- Define the DAC960 V2 Firmware Physical Device State type.
-*/
-
-typedef enum
-{
- DAC960_V2_Device_Unconfigured = 0x00,
- DAC960_V2_Device_Online = 0x01,
- DAC960_V2_Device_Rebuild = 0x03,
- DAC960_V2_Device_Missing = 0x04,
- DAC960_V2_Device_Critical = 0x05,
- DAC960_V2_Device_Dead = 0x08,
- DAC960_V2_Device_SuspectedDead = 0x0C,
- DAC960_V2_Device_CommandedOffline = 0x10,
- DAC960_V2_Device_Standby = 0x21,
- DAC960_V2_Device_InvalidState = 0xFF
-}
-__attribute__ ((packed))
-DAC960_V2_PhysicalDeviceState_T;
-
-
-/*
- Define the DAC960 V2 Firmware Get Physical Device Info reply structure.
-*/
-
-typedef struct DAC960_V2_PhysicalDeviceInfo
-{
- unsigned char :8; /* Byte 0 */
- unsigned char Channel; /* Byte 1 */
- unsigned char TargetID; /* Byte 2 */
- unsigned char LogicalUnit; /* Byte 3 */
- /* Configuration Status Bits */
- bool PhysicalDeviceFaultTolerant:1; /* Byte 4 Bit 0 */
- bool PhysicalDeviceConnected:1; /* Byte 4 Bit 1 */
- bool PhysicalDeviceLocalToController:1; /* Byte 4 Bit 2 */
- unsigned char :5; /* Byte 4 Bits 3-7 */
- /* Multiple Host/Controller Status Bits */
- bool RemoteHostSystemDead:1; /* Byte 5 Bit 0 */
- bool RemoteControllerDead:1; /* Byte 5 Bit 1 */
- unsigned char :6; /* Byte 5 Bits 2-7 */
- DAC960_V2_PhysicalDeviceState_T PhysicalDeviceState; /* Byte 6 */
- unsigned char NegotiatedDataWidthBits; /* Byte 7 */
- unsigned short NegotiatedSynchronousMegaTransfers; /* Bytes 8-9 */
- /* Multiported Physical Device Information */
- unsigned char NumberOfPortConnections; /* Byte 10 */
- unsigned char DriveAccessibilityBitmap; /* Byte 11 */
- unsigned int :32; /* Bytes 12-15 */
- unsigned char NetworkAddress[16]; /* Bytes 16-31 */
- unsigned short MaximumTags; /* Bytes 32-33 */
- /* Physical Device Operations Status */
- bool ConsistencyCheckInProgress:1; /* Byte 34 Bit 0 */
- bool RebuildInProgress:1; /* Byte 34 Bit 1 */
- bool MakingDataConsistentInProgress:1; /* Byte 34 Bit 2 */
- bool PhysicalDeviceInitializationInProgress:1; /* Byte 34 Bit 3 */
- bool DataMigrationInProgress:1; /* Byte 34 Bit 4 */
- bool PatrolOperationInProgress:1; /* Byte 34 Bit 5 */
- unsigned char :2; /* Byte 34 Bits 6-7 */
- unsigned char LongOperationStatus; /* Byte 35 */
- unsigned char ParityErrors; /* Byte 36 */
- unsigned char SoftErrors; /* Byte 37 */
- unsigned char HardErrors; /* Byte 38 */
- unsigned char MiscellaneousErrors; /* Byte 39 */
- unsigned char CommandTimeouts; /* Byte 40 */
- unsigned char Retries; /* Byte 41 */
- unsigned char Aborts; /* Byte 42 */
- unsigned char PredictedFailuresDetected; /* Byte 43 */
- unsigned int :32; /* Bytes 44-47 */
- unsigned short :16; /* Bytes 48-49 */
- unsigned short DeviceBlockSizeInBytes; /* Bytes 50-51 */
- unsigned int OriginalDeviceSize; /* Bytes 52-55 */
- unsigned int ConfigurableDeviceSize; /* Bytes 56-59 */
- unsigned int :32; /* Bytes 60-63 */
- unsigned char PhysicalDeviceName[16]; /* Bytes 64-79 */
- unsigned char Reserved1[16]; /* Bytes 80-95 */
- unsigned char Reserved2[32]; /* Bytes 96-127 */
- unsigned char SCSI_InquiryData[36]; /* Bytes 128-163 */
- unsigned char Reserved3[20]; /* Bytes 164-183 */
- unsigned char Reserved4[8]; /* Bytes 184-191 */
- DAC960_ByteCount64_T LastReadBlockNumber; /* Bytes 192-199 */
- DAC960_ByteCount64_T LastWrittenBlockNumber; /* Bytes 200-207 */
- DAC960_ByteCount64_T ConsistencyCheckBlockNumber; /* Bytes 208-215 */
- DAC960_ByteCount64_T RebuildBlockNumber; /* Bytes 216-223 */
- DAC960_ByteCount64_T MakingDataConsistentBlockNumber; /* Bytes 224-231 */
- DAC960_ByteCount64_T DeviceInitializationBlockNumber; /* Bytes 232-239 */
- DAC960_ByteCount64_T DataMigrationBlockNumber; /* Bytes 240-247 */
- DAC960_ByteCount64_T PatrolOperationBlockNumber; /* Bytes 248-255 */
- unsigned char Reserved5[256]; /* Bytes 256-511 */
-}
-DAC960_V2_PhysicalDeviceInfo_T;
-
-
-/*
- Define the DAC960 V2 Firmware Health Status Buffer structure.
-*/
-
-typedef struct DAC960_V2_HealthStatusBuffer
-{
- unsigned int MicrosecondsFromControllerStartTime; /* Bytes 0-3 */
- unsigned int MillisecondsFromControllerStartTime; /* Bytes 4-7 */
- unsigned int SecondsFrom1January1970; /* Bytes 8-11 */
- unsigned int :32; /* Bytes 12-15 */
- unsigned int StatusChangeCounter; /* Bytes 16-19 */
- unsigned int :32; /* Bytes 20-23 */
- unsigned int DebugOutputMessageBufferIndex; /* Bytes 24-27 */
- unsigned int CodedMessageBufferIndex; /* Bytes 28-31 */
- unsigned int CurrentTimeTracePageNumber; /* Bytes 32-35 */
- unsigned int CurrentProfilerPageNumber; /* Bytes 36-39 */
- unsigned int NextEventSequenceNumber; /* Bytes 40-43 */
- unsigned int :32; /* Bytes 44-47 */
- unsigned char Reserved1[16]; /* Bytes 48-63 */
- unsigned char Reserved2[64]; /* Bytes 64-127 */
-}
-DAC960_V2_HealthStatusBuffer_T;
-
-
-/*
- Define the DAC960 V2 Firmware Get Event reply structure.
-*/
-
-typedef struct DAC960_V2_Event
-{
- unsigned int EventSequenceNumber; /* Bytes 0-3 */
- unsigned int EventTime; /* Bytes 4-7 */
- unsigned int EventCode; /* Bytes 8-11 */
- unsigned char :8; /* Byte 12 */
- unsigned char Channel; /* Byte 13 */
- unsigned char TargetID; /* Byte 14 */
- unsigned char LogicalUnit; /* Byte 15 */
- unsigned int :32; /* Bytes 16-19 */
- unsigned int EventSpecificParameter; /* Bytes 20-23 */
- unsigned char RequestSenseData[40]; /* Bytes 24-63 */
-}
-DAC960_V2_Event_T;
-
-
-/*
- Define the DAC960 V2 Firmware Command Control Bits structure.
-*/
-
-typedef struct DAC960_V2_CommandControlBits
-{
- bool ForceUnitAccess:1; /* Byte 0 Bit 0 */
- bool DisablePageOut:1; /* Byte 0 Bit 1 */
- bool :1; /* Byte 0 Bit 2 */
- bool AdditionalScatterGatherListMemory:1; /* Byte 0 Bit 3 */
- bool DataTransferControllerToHost:1; /* Byte 0 Bit 4 */
- bool :1; /* Byte 0 Bit 5 */
- bool NoAutoRequestSense:1; /* Byte 0 Bit 6 */
- bool DisconnectProhibited:1; /* Byte 0 Bit 7 */
-}
-DAC960_V2_CommandControlBits_T;
-
-
-/*
- Define the DAC960 V2 Firmware Command Timeout structure.
-*/
-
-typedef struct DAC960_V2_CommandTimeout
-{
- unsigned char TimeoutValue:6; /* Byte 0 Bits 0-5 */
- enum {
- DAC960_V2_TimeoutScale_Seconds = 0,
- DAC960_V2_TimeoutScale_Minutes = 1,
- DAC960_V2_TimeoutScale_Hours = 2,
- DAC960_V2_TimeoutScale_Reserved = 3
- } __attribute__ ((packed)) TimeoutScale:2; /* Byte 0 Bits 6-7 */
-}
-DAC960_V2_CommandTimeout_T;
-
-
-/*
- Define the DAC960 V2 Firmware Physical Device structure.
-*/
-
-typedef struct DAC960_V2_PhysicalDevice
-{
- unsigned char LogicalUnit; /* Byte 0 */
- unsigned char TargetID; /* Byte 1 */
- unsigned char Channel:3; /* Byte 2 Bits 0-2 */
- unsigned char Controller:5; /* Byte 2 Bits 3-7 */
-}
-__attribute__ ((packed))
-DAC960_V2_PhysicalDevice_T;
-
-
-/*
- Define the DAC960 V2 Firmware Logical Device structure.
-*/
-
-typedef struct DAC960_V2_LogicalDevice
-{
- unsigned short LogicalDeviceNumber; /* Bytes 0-1 */
- unsigned char :3; /* Byte 2 Bits 0-2 */
- unsigned char Controller:5; /* Byte 2 Bits 3-7 */
-}
-__attribute__ ((packed))
-DAC960_V2_LogicalDevice_T;
-
-
-/*
- Define the DAC960 V2 Firmware Operation Device type.
-*/
-
-typedef enum
-{
- DAC960_V2_Physical_Device = 0x00,
- DAC960_V2_RAID_Device = 0x01,
- DAC960_V2_Physical_Channel = 0x02,
- DAC960_V2_RAID_Channel = 0x03,
- DAC960_V2_Physical_Controller = 0x04,
- DAC960_V2_RAID_Controller = 0x05,
- DAC960_V2_Configuration_Group = 0x10,
- DAC960_V2_Enclosure = 0x11
-}
-__attribute__ ((packed))
-DAC960_V2_OperationDevice_T;
-
-
-/*
- Define the DAC960 V2 Firmware Translate Physical To Logical Device structure.
-*/
-
-typedef struct DAC960_V2_PhysicalToLogicalDevice
-{
- unsigned short LogicalDeviceNumber; /* Bytes 0-1 */
- unsigned short :16; /* Bytes 2-3 */
- unsigned char PreviousBootController; /* Byte 4 */
- unsigned char PreviousBootChannel; /* Byte 5 */
- unsigned char PreviousBootTargetID; /* Byte 6 */
- unsigned char PreviousBootLogicalUnit; /* Byte 7 */
-}
-DAC960_V2_PhysicalToLogicalDevice_T;
-
-
-
-/*
- Define the DAC960 V2 Firmware Scatter/Gather List Entry structure.
-*/
-
-typedef struct DAC960_V2_ScatterGatherSegment
-{
- DAC960_BusAddress64_T SegmentDataPointer; /* Bytes 0-7 */
- DAC960_ByteCount64_T SegmentByteCount; /* Bytes 8-15 */
-}
-DAC960_V2_ScatterGatherSegment_T;
-
-
-/*
- Define the DAC960 V2 Firmware Data Transfer Memory Address structure.
-*/
-
-typedef union DAC960_V2_DataTransferMemoryAddress
-{
- DAC960_V2_ScatterGatherSegment_T ScatterGatherSegments[2]; /* Bytes 0-31 */
- struct {
- unsigned short ScatterGatherList0Length; /* Bytes 0-1 */
- unsigned short ScatterGatherList1Length; /* Bytes 2-3 */
- unsigned short ScatterGatherList2Length; /* Bytes 4-5 */
- unsigned short :16; /* Bytes 6-7 */
- DAC960_BusAddress64_T ScatterGatherList0Address; /* Bytes 8-15 */
- DAC960_BusAddress64_T ScatterGatherList1Address; /* Bytes 16-23 */
- DAC960_BusAddress64_T ScatterGatherList2Address; /* Bytes 24-31 */
- } ExtendedScatterGather;
-}
-DAC960_V2_DataTransferMemoryAddress_T;
-
-
-/*
- Define the 64 Byte DAC960 V2 Firmware Command Mailbox structure.
-*/
-
-typedef union DAC960_V2_CommandMailbox
-{
- unsigned int Words[16]; /* Words 0-15 */
- struct {
- DAC960_V2_CommandIdentifier_T CommandIdentifier; /* Bytes 0-1 */
- DAC960_V2_CommandOpcode_T CommandOpcode; /* Byte 2 */
- DAC960_V2_CommandControlBits_T CommandControlBits; /* Byte 3 */
- DAC960_ByteCount32_T DataTransferSize:24; /* Bytes 4-6 */
- unsigned char DataTransferPageNumber; /* Byte 7 */
- DAC960_BusAddress64_T RequestSenseBusAddress; /* Bytes 8-15 */
- unsigned int :24; /* Bytes 16-18 */
- DAC960_V2_CommandTimeout_T CommandTimeout; /* Byte 19 */
- unsigned char RequestSenseSize; /* Byte 20 */
- unsigned char IOCTL_Opcode; /* Byte 21 */
- unsigned char Reserved[10]; /* Bytes 22-31 */
- DAC960_V2_DataTransferMemoryAddress_T
- DataTransferMemoryAddress; /* Bytes 32-63 */
- } Common;
- struct {
- DAC960_V2_CommandIdentifier_T CommandIdentifier; /* Bytes 0-1 */
- DAC960_V2_CommandOpcode_T CommandOpcode; /* Byte 2 */
- DAC960_V2_CommandControlBits_T CommandControlBits; /* Byte 3 */
- DAC960_ByteCount32_T DataTransferSize; /* Bytes 4-7 */
- DAC960_BusAddress64_T RequestSenseBusAddress; /* Bytes 8-15 */
- DAC960_V2_PhysicalDevice_T PhysicalDevice; /* Bytes 16-18 */
- DAC960_V2_CommandTimeout_T CommandTimeout; /* Byte 19 */
- unsigned char RequestSenseSize; /* Byte 20 */
- unsigned char CDBLength; /* Byte 21 */
- unsigned char SCSI_CDB[10]; /* Bytes 22-31 */
- DAC960_V2_DataTransferMemoryAddress_T
- DataTransferMemoryAddress; /* Bytes 32-63 */
- } SCSI_10;
- struct {
- DAC960_V2_CommandIdentifier_T CommandIdentifier; /* Bytes 0-1 */
- DAC960_V2_CommandOpcode_T CommandOpcode; /* Byte 2 */
- DAC960_V2_CommandControlBits_T CommandControlBits; /* Byte 3 */
- DAC960_ByteCount32_T DataTransferSize; /* Bytes 4-7 */
- DAC960_BusAddress64_T RequestSenseBusAddress; /* Bytes 8-15 */
- DAC960_V2_PhysicalDevice_T PhysicalDevice; /* Bytes 16-18 */
- DAC960_V2_CommandTimeout_T CommandTimeout; /* Byte 19 */
- unsigned char RequestSenseSize; /* Byte 20 */
- unsigned char CDBLength; /* Byte 21 */
- unsigned short :16; /* Bytes 22-23 */
- DAC960_BusAddress64_T SCSI_CDB_BusAddress; /* Bytes 24-31 */
- DAC960_V2_DataTransferMemoryAddress_T
- DataTransferMemoryAddress; /* Bytes 32-63 */
- } SCSI_255;
- struct {
- DAC960_V2_CommandIdentifier_T CommandIdentifier; /* Bytes 0-1 */
- DAC960_V2_CommandOpcode_T CommandOpcode; /* Byte 2 */
- DAC960_V2_CommandControlBits_T CommandControlBits; /* Byte 3 */
- DAC960_ByteCount32_T DataTransferSize:24; /* Bytes 4-6 */
- unsigned char DataTransferPageNumber; /* Byte 7 */
- DAC960_BusAddress64_T RequestSenseBusAddress; /* Bytes 8-15 */
- unsigned short :16; /* Bytes 16-17 */
- unsigned char ControllerNumber; /* Byte 18 */
- DAC960_V2_CommandTimeout_T CommandTimeout; /* Byte 19 */
- unsigned char RequestSenseSize; /* Byte 20 */
- unsigned char IOCTL_Opcode; /* Byte 21 */
- unsigned char Reserved[10]; /* Bytes 22-31 */
- DAC960_V2_DataTransferMemoryAddress_T
- DataTransferMemoryAddress; /* Bytes 32-63 */
- } ControllerInfo;
- struct {
- DAC960_V2_CommandIdentifier_T CommandIdentifier; /* Bytes 0-1 */
- DAC960_V2_CommandOpcode_T CommandOpcode; /* Byte 2 */
- DAC960_V2_CommandControlBits_T CommandControlBits; /* Byte 3 */
- DAC960_ByteCount32_T DataTransferSize:24; /* Bytes 4-6 */
- unsigned char DataTransferPageNumber; /* Byte 7 */
- DAC960_BusAddress64_T RequestSenseBusAddress; /* Bytes 8-15 */
- DAC960_V2_LogicalDevice_T LogicalDevice; /* Bytes 16-18 */
- DAC960_V2_CommandTimeout_T CommandTimeout; /* Byte 19 */
- unsigned char RequestSenseSize; /* Byte 20 */
- unsigned char IOCTL_Opcode; /* Byte 21 */
- unsigned char Reserved[10]; /* Bytes 22-31 */
- DAC960_V2_DataTransferMemoryAddress_T
- DataTransferMemoryAddress; /* Bytes 32-63 */
- } LogicalDeviceInfo;
- struct {
- DAC960_V2_CommandIdentifier_T CommandIdentifier; /* Bytes 0-1 */
- DAC960_V2_CommandOpcode_T CommandOpcode; /* Byte 2 */
- DAC960_V2_CommandControlBits_T CommandControlBits; /* Byte 3 */
- DAC960_ByteCount32_T DataTransferSize:24; /* Bytes 4-6 */
- unsigned char DataTransferPageNumber; /* Byte 7 */
- DAC960_BusAddress64_T RequestSenseBusAddress; /* Bytes 8-15 */
- DAC960_V2_PhysicalDevice_T PhysicalDevice; /* Bytes 16-18 */
- DAC960_V2_CommandTimeout_T CommandTimeout; /* Byte 19 */
- unsigned char RequestSenseSize; /* Byte 20 */
- unsigned char IOCTL_Opcode; /* Byte 21 */
- unsigned char Reserved[10]; /* Bytes 22-31 */
- DAC960_V2_DataTransferMemoryAddress_T
- DataTransferMemoryAddress; /* Bytes 32-63 */
- } PhysicalDeviceInfo;
- struct {
- DAC960_V2_CommandIdentifier_T CommandIdentifier; /* Bytes 0-1 */
- DAC960_V2_CommandOpcode_T CommandOpcode; /* Byte 2 */
- DAC960_V2_CommandControlBits_T CommandControlBits; /* Byte 3 */
- DAC960_ByteCount32_T DataTransferSize:24; /* Bytes 4-6 */
- unsigned char DataTransferPageNumber; /* Byte 7 */
- DAC960_BusAddress64_T RequestSenseBusAddress; /* Bytes 8-15 */
- unsigned short EventSequenceNumberHigh16; /* Bytes 16-17 */
- unsigned char ControllerNumber; /* Byte 18 */
- DAC960_V2_CommandTimeout_T CommandTimeout; /* Byte 19 */
- unsigned char RequestSenseSize; /* Byte 20 */
- unsigned char IOCTL_Opcode; /* Byte 21 */
- unsigned short EventSequenceNumberLow16; /* Bytes 22-23 */
- unsigned char Reserved[8]; /* Bytes 24-31 */
- DAC960_V2_DataTransferMemoryAddress_T
- DataTransferMemoryAddress; /* Bytes 32-63 */
- } GetEvent;
- struct {
- DAC960_V2_CommandIdentifier_T CommandIdentifier; /* Bytes 0-1 */
- DAC960_V2_CommandOpcode_T CommandOpcode; /* Byte 2 */
- DAC960_V2_CommandControlBits_T CommandControlBits; /* Byte 3 */
- DAC960_ByteCount32_T DataTransferSize:24; /* Bytes 4-6 */
- unsigned char DataTransferPageNumber; /* Byte 7 */
- DAC960_BusAddress64_T RequestSenseBusAddress; /* Bytes 8-15 */
- DAC960_V2_LogicalDevice_T LogicalDevice; /* Bytes 16-18 */
- DAC960_V2_CommandTimeout_T CommandTimeout; /* Byte 19 */
- unsigned char RequestSenseSize; /* Byte 20 */
- unsigned char IOCTL_Opcode; /* Byte 21 */
- union {
- DAC960_V2_LogicalDeviceState_T LogicalDeviceState;
- DAC960_V2_PhysicalDeviceState_T PhysicalDeviceState;
- } DeviceState; /* Byte 22 */
- unsigned char Reserved[9]; /* Bytes 23-31 */
- DAC960_V2_DataTransferMemoryAddress_T
- DataTransferMemoryAddress; /* Bytes 32-63 */
- } SetDeviceState;
- struct {
- DAC960_V2_CommandIdentifier_T CommandIdentifier; /* Bytes 0-1 */
- DAC960_V2_CommandOpcode_T CommandOpcode; /* Byte 2 */
- DAC960_V2_CommandControlBits_T CommandControlBits; /* Byte 3 */
- DAC960_ByteCount32_T DataTransferSize:24; /* Bytes 4-6 */
- unsigned char DataTransferPageNumber; /* Byte 7 */
- DAC960_BusAddress64_T RequestSenseBusAddress; /* Bytes 8-15 */
- DAC960_V2_LogicalDevice_T LogicalDevice; /* Bytes 16-18 */
- DAC960_V2_CommandTimeout_T CommandTimeout; /* Byte 19 */
- unsigned char RequestSenseSize; /* Byte 20 */
- unsigned char IOCTL_Opcode; /* Byte 21 */
- bool RestoreConsistency:1; /* Byte 22 Bit 0 */
- bool InitializedAreaOnly:1; /* Byte 22 Bit 1 */
- unsigned char :6; /* Byte 22 Bits 2-7 */
- unsigned char Reserved[9]; /* Bytes 23-31 */
- DAC960_V2_DataTransferMemoryAddress_T
- DataTransferMemoryAddress; /* Bytes 32-63 */
- } ConsistencyCheck;
- struct {
- DAC960_V2_CommandIdentifier_T CommandIdentifier; /* Bytes 0-1 */
- DAC960_V2_CommandOpcode_T CommandOpcode; /* Byte 2 */
- DAC960_V2_CommandControlBits_T CommandControlBits; /* Byte 3 */
- unsigned char FirstCommandMailboxSizeKB; /* Byte 4 */
- unsigned char FirstStatusMailboxSizeKB; /* Byte 5 */
- unsigned char SecondCommandMailboxSizeKB; /* Byte 6 */
- unsigned char SecondStatusMailboxSizeKB; /* Byte 7 */
- DAC960_BusAddress64_T RequestSenseBusAddress; /* Bytes 8-15 */
- unsigned int :24; /* Bytes 16-18 */
- DAC960_V2_CommandTimeout_T CommandTimeout; /* Byte 19 */
- unsigned char RequestSenseSize; /* Byte 20 */
- unsigned char IOCTL_Opcode; /* Byte 21 */
- unsigned char HealthStatusBufferSizeKB; /* Byte 22 */
- unsigned char :8; /* Byte 23 */
- DAC960_BusAddress64_T HealthStatusBufferBusAddress; /* Bytes 24-31 */
- DAC960_BusAddress64_T FirstCommandMailboxBusAddress; /* Bytes 32-39 */
- DAC960_BusAddress64_T FirstStatusMailboxBusAddress; /* Bytes 40-47 */
- DAC960_BusAddress64_T SecondCommandMailboxBusAddress; /* Bytes 48-55 */
- DAC960_BusAddress64_T SecondStatusMailboxBusAddress; /* Bytes 56-63 */
- } SetMemoryMailbox;
- struct {
- DAC960_V2_CommandIdentifier_T CommandIdentifier; /* Bytes 0-1 */
- DAC960_V2_CommandOpcode_T CommandOpcode; /* Byte 2 */
- DAC960_V2_CommandControlBits_T CommandControlBits; /* Byte 3 */
- DAC960_ByteCount32_T DataTransferSize:24; /* Bytes 4-6 */
- unsigned char DataTransferPageNumber; /* Byte 7 */
- DAC960_BusAddress64_T RequestSenseBusAddress; /* Bytes 8-15 */
- DAC960_V2_PhysicalDevice_T PhysicalDevice; /* Bytes 16-18 */
- DAC960_V2_CommandTimeout_T CommandTimeout; /* Byte 19 */
- unsigned char RequestSenseSize; /* Byte 20 */
- unsigned char IOCTL_Opcode; /* Byte 21 */
- DAC960_V2_OperationDevice_T OperationDevice; /* Byte 22 */
- unsigned char Reserved[9]; /* Bytes 23-31 */
- DAC960_V2_DataTransferMemoryAddress_T
- DataTransferMemoryAddress; /* Bytes 32-63 */
- } DeviceOperation;
-}
-DAC960_V2_CommandMailbox_T;
-
-
-/*
- Define the DAC960 Driver IOCTL requests.
-*/
-
-#define DAC960_IOCTL_GET_CONTROLLER_COUNT 0xDAC001
-#define DAC960_IOCTL_GET_CONTROLLER_INFO 0xDAC002
-#define DAC960_IOCTL_V1_EXECUTE_COMMAND 0xDAC003
-#define DAC960_IOCTL_V2_EXECUTE_COMMAND 0xDAC004
-#define DAC960_IOCTL_V2_GET_HEALTH_STATUS 0xDAC005
-
-
-/*
- Define the DAC960_IOCTL_GET_CONTROLLER_INFO reply structure.
-*/
-
-typedef struct DAC960_ControllerInfo
-{
- unsigned char ControllerNumber;
- unsigned char FirmwareType;
- unsigned char Channels;
- unsigned char Targets;
- unsigned char PCI_Bus;
- unsigned char PCI_Device;
- unsigned char PCI_Function;
- unsigned char IRQ_Channel;
- DAC960_PCI_Address_T PCI_Address;
- unsigned char ModelName[20];
- unsigned char FirmwareVersion[12];
-}
-DAC960_ControllerInfo_T;
-
-
-/*
- Define the User Mode DAC960_IOCTL_V1_EXECUTE_COMMAND request structure.
-*/
-
-typedef struct DAC960_V1_UserCommand
-{
- unsigned char ControllerNumber;
- DAC960_V1_CommandMailbox_T CommandMailbox;
- int DataTransferLength;
- void __user *DataTransferBuffer;
- DAC960_V1_DCDB_T __user *DCDB;
-}
-DAC960_V1_UserCommand_T;
-
-
-/*
- Define the Kernel Mode DAC960_IOCTL_V1_EXECUTE_COMMAND request structure.
-*/
-
-typedef struct DAC960_V1_KernelCommand
-{
- unsigned char ControllerNumber;
- DAC960_V1_CommandMailbox_T CommandMailbox;
- int DataTransferLength;
- void *DataTransferBuffer;
- DAC960_V1_DCDB_T *DCDB;
- DAC960_V1_CommandStatus_T CommandStatus;
- void (*CompletionFunction)(struct DAC960_V1_KernelCommand *);
- void *CompletionData;
-}
-DAC960_V1_KernelCommand_T;
-
-
-/*
- Define the User Mode DAC960_IOCTL_V2_EXECUTE_COMMAND request structure.
-*/
-
-typedef struct DAC960_V2_UserCommand
-{
- unsigned char ControllerNumber;
- DAC960_V2_CommandMailbox_T CommandMailbox;
- int DataTransferLength;
- int RequestSenseLength;
- void __user *DataTransferBuffer;
- void __user *RequestSenseBuffer;
-}
-DAC960_V2_UserCommand_T;
-
-
-/*
- Define the Kernel Mode DAC960_IOCTL_V2_EXECUTE_COMMAND request structure.
-*/
-
-typedef struct DAC960_V2_KernelCommand
-{
- unsigned char ControllerNumber;
- DAC960_V2_CommandMailbox_T CommandMailbox;
- int DataTransferLength;
- int RequestSenseLength;
- void *DataTransferBuffer;
- void *RequestSenseBuffer;
- DAC960_V2_CommandStatus_T CommandStatus;
- void (*CompletionFunction)(struct DAC960_V2_KernelCommand *);
- void *CompletionData;
-}
-DAC960_V2_KernelCommand_T;
-
-
-/*
- Define the User Mode DAC960_IOCTL_V2_GET_HEALTH_STATUS request structure.
-*/
-
-typedef struct DAC960_V2_GetHealthStatus
-{
- unsigned char ControllerNumber;
- DAC960_V2_HealthStatusBuffer_T __user *HealthStatusBuffer;
-}
-DAC960_V2_GetHealthStatus_T;
-
-
-/*
- Import the Kernel Mode IOCTL interface.
-*/
-
-extern int DAC960_KernelIOCTL(unsigned int Request, void *Argument);
-
-
-/*
- DAC960_DriverVersion protects the private portion of this file.
-*/
-
-#ifdef DAC960_DriverVersion
-
-
-/*
- Define the maximum Driver Queue Depth and Controller Queue Depth supported
- by DAC960 V1 and V2 Firmware Controllers.
-*/
-
-#define DAC960_MaxDriverQueueDepth 511
-#define DAC960_MaxControllerQueueDepth 512
-
-
-/*
- Define the maximum number of Scatter/Gather Segments supported for any
- DAC960 V1 and V2 Firmware controller.
-*/
-
-#define DAC960_V1_ScatterGatherLimit 33
-#define DAC960_V2_ScatterGatherLimit 128
-
-
-/*
- Define the number of Command Mailboxes and Status Mailboxes used by the
- DAC960 V1 and V2 Firmware Memory Mailbox Interface.
-*/
-
-#define DAC960_V1_CommandMailboxCount 256
-#define DAC960_V1_StatusMailboxCount 1024
-#define DAC960_V2_CommandMailboxCount 512
-#define DAC960_V2_StatusMailboxCount 512
-
-
-/*
- Define the DAC960 Controller Monitoring Timer Interval.
-*/
-
-#define DAC960_MonitoringTimerInterval (10 * HZ)
-
-
-/*
- Define the DAC960 Controller Secondary Monitoring Interval.
-*/
-
-#define DAC960_SecondaryMonitoringInterval (60 * HZ)
-
-
-/*
- Define the DAC960 Controller Health Status Monitoring Interval.
-*/
-
-#define DAC960_HealthStatusMonitoringInterval (1 * HZ)
-
-
-/*
- Define the DAC960 Controller Progress Reporting Interval.
-*/
-
-#define DAC960_ProgressReportingInterval (60 * HZ)
-
-
-/*
- Define the maximum number of Partitions allowed for each Logical Drive.
-*/
-
-#define DAC960_MaxPartitions 8
-#define DAC960_MaxPartitionsBits 3
-
-/*
- Define the DAC960 Controller fixed Block Size and Block Size Bits.
-*/
-
-#define DAC960_BlockSize 512
-#define DAC960_BlockSizeBits 9
-
-
-/*
- Define the number of Command structures that should be allocated as a
- group to optimize kernel memory allocation.
-*/
-
-#define DAC960_V1_CommandAllocationGroupSize 11
-#define DAC960_V2_CommandAllocationGroupSize 29
-
-
-/*
- Define the Controller Line Buffer, Progress Buffer, User Message, and
- Initial Status Buffer sizes.
-*/
-
-#define DAC960_LineBufferSize 100
-#define DAC960_ProgressBufferSize 200
-#define DAC960_UserMessageSize 200
-#define DAC960_InitialStatusBufferSize (8192-32)
-
-
-/*
- Define the DAC960 Controller Firmware Types.
-*/
-
-typedef enum
-{
- DAC960_V1_Controller = 1,
- DAC960_V2_Controller = 2
-}
-DAC960_FirmwareType_T;
-
-
-/*
- Define the DAC960 Controller Hardware Types.
-*/
-
-typedef enum
-{
- DAC960_BA_Controller = 1, /* eXtremeRAID 2000 */
- DAC960_LP_Controller = 2, /* AcceleRAID 352 */
- DAC960_LA_Controller = 3, /* DAC1164P */
- DAC960_PG_Controller = 4, /* DAC960PTL/PJ/PG */
- DAC960_PD_Controller = 5, /* DAC960PU/PD/PL/P */
- DAC960_P_Controller = 6, /* DAC960PU/PD/PL/P */
- DAC960_GEM_Controller = 7, /* AcceleRAID 4/5/600 */
-}
-DAC960_HardwareType_T;
-
-
-/*
- Define the Driver Message Levels.
-*/
-
-typedef enum DAC960_MessageLevel
-{
- DAC960_AnnounceLevel = 0,
- DAC960_InfoLevel = 1,
- DAC960_NoticeLevel = 2,
- DAC960_WarningLevel = 3,
- DAC960_ErrorLevel = 4,
- DAC960_ProgressLevel = 5,
- DAC960_CriticalLevel = 6,
- DAC960_UserCriticalLevel = 7
-}
-DAC960_MessageLevel_T;
-
-static char
- *DAC960_MessageLevelMap[] =
- { KERN_NOTICE, KERN_NOTICE, KERN_NOTICE, KERN_WARNING,
- KERN_ERR, KERN_CRIT, KERN_CRIT, KERN_CRIT };
-
-
-/*
- Define Driver Message macros.
-*/
-
-#define DAC960_Announce(Format, Arguments...) \
- DAC960_Message(DAC960_AnnounceLevel, Format, ##Arguments)
-
-#define DAC960_Info(Format, Arguments...) \
- DAC960_Message(DAC960_InfoLevel, Format, ##Arguments)
-
-#define DAC960_Notice(Format, Arguments...) \
- DAC960_Message(DAC960_NoticeLevel, Format, ##Arguments)
-
-#define DAC960_Warning(Format, Arguments...) \
- DAC960_Message(DAC960_WarningLevel, Format, ##Arguments)
-
-#define DAC960_Error(Format, Arguments...) \
- DAC960_Message(DAC960_ErrorLevel, Format, ##Arguments)
-
-#define DAC960_Progress(Format, Arguments...) \
- DAC960_Message(DAC960_ProgressLevel, Format, ##Arguments)
-
-#define DAC960_Critical(Format, Arguments...) \
- DAC960_Message(DAC960_CriticalLevel, Format, ##Arguments)
-
-#define DAC960_UserCritical(Format, Arguments...) \
- DAC960_Message(DAC960_UserCriticalLevel, Format, ##Arguments)
-
-
-struct DAC960_privdata {
- DAC960_HardwareType_T HardwareType;
- DAC960_FirmwareType_T FirmwareType;
- irq_handler_t InterruptHandler;
- unsigned int MemoryWindowSize;
-};
-
-
-/*
- Define the DAC960 V1 Firmware Controller Status Mailbox structure.
-*/
-
-typedef union DAC960_V1_StatusMailbox
-{
- unsigned int Word; /* Word 0 */
- struct {
- DAC960_V1_CommandIdentifier_T CommandIdentifier; /* Byte 0 */
- unsigned char :7; /* Byte 1 Bits 0-6 */
- bool Valid:1; /* Byte 1 Bit 7 */
- DAC960_V1_CommandStatus_T CommandStatus; /* Bytes 2-3 */
- } Fields;
-}
-DAC960_V1_StatusMailbox_T;
-
-
-/*
- Define the DAC960 V2 Firmware Controller Status Mailbox structure.
-*/
-
-typedef union DAC960_V2_StatusMailbox
-{
- unsigned int Words[2]; /* Words 0-1 */
- struct {
- DAC960_V2_CommandIdentifier_T CommandIdentifier; /* Bytes 0-1 */
- DAC960_V2_CommandStatus_T CommandStatus; /* Byte 2 */
- unsigned char RequestSenseLength; /* Byte 3 */
- int DataTransferResidue; /* Bytes 4-7 */
- } Fields;
-}
-DAC960_V2_StatusMailbox_T;
-
-
-/*
- Define the DAC960 Driver Command Types.
-*/
-
-typedef enum
-{
- DAC960_ReadCommand = 1,
- DAC960_WriteCommand = 2,
- DAC960_ReadRetryCommand = 3,
- DAC960_WriteRetryCommand = 4,
- DAC960_MonitoringCommand = 5,
- DAC960_ImmediateCommand = 6,
- DAC960_QueuedCommand = 7
-}
-DAC960_CommandType_T;
-
-
-/*
- Define the DAC960 Driver Command structure.
-*/
-
-typedef struct DAC960_Command
-{
- int CommandIdentifier;
- DAC960_CommandType_T CommandType;
- struct DAC960_Controller *Controller;
- struct DAC960_Command *Next;
- struct completion *Completion;
- unsigned int LogicalDriveNumber;
- unsigned int BlockNumber;
- unsigned int BlockCount;
- unsigned int SegmentCount;
- int DmaDirection;
- struct scatterlist *cmd_sglist;
- struct request *Request;
- union {
- struct {
- DAC960_V1_CommandMailbox_T CommandMailbox;
- DAC960_V1_KernelCommand_T *KernelCommand;
- DAC960_V1_CommandStatus_T CommandStatus;
- DAC960_V1_ScatterGatherSegment_T *ScatterGatherList;
- dma_addr_t ScatterGatherListDMA;
- struct scatterlist ScatterList[DAC960_V1_ScatterGatherLimit];
- unsigned int EndMarker[0];
- } V1;
- struct {
- DAC960_V2_CommandMailbox_T CommandMailbox;
- DAC960_V2_KernelCommand_T *KernelCommand;
- DAC960_V2_CommandStatus_T CommandStatus;
- unsigned char RequestSenseLength;
- int DataTransferResidue;
- DAC960_V2_ScatterGatherSegment_T *ScatterGatherList;
- dma_addr_t ScatterGatherListDMA;
- DAC960_SCSI_RequestSense_T *RequestSense;
- dma_addr_t RequestSenseDMA;
- struct scatterlist ScatterList[DAC960_V2_ScatterGatherLimit];
- unsigned int EndMarker[0];
- } V2;
- } FW;
-}
-DAC960_Command_T;
-
-
-/*
- Define the DAC960 Driver Controller structure.
-*/
-
-typedef struct DAC960_Controller
-{
- void __iomem *BaseAddress;
- void __iomem *MemoryMappedAddress;
- DAC960_FirmwareType_T FirmwareType;
- DAC960_HardwareType_T HardwareType;
- DAC960_IO_Address_T IO_Address;
- DAC960_PCI_Address_T PCI_Address;
- struct pci_dev *PCIDevice;
- unsigned char ControllerNumber;
- unsigned char ControllerName[4];
- unsigned char ModelName[20];
- unsigned char FullModelName[28];
- unsigned char FirmwareVersion[12];
- unsigned char Bus;
- unsigned char Device;
- unsigned char Function;
- unsigned char IRQ_Channel;
- unsigned char Channels;
- unsigned char Targets;
- unsigned char MemorySize;
- unsigned char LogicalDriveCount;
- unsigned short CommandAllocationGroupSize;
- unsigned short ControllerQueueDepth;
- unsigned short DriverQueueDepth;
- unsigned short MaxBlocksPerCommand;
- unsigned short ControllerScatterGatherLimit;
- unsigned short DriverScatterGatherLimit;
- unsigned int CombinedStatusBufferLength;
- unsigned int InitialStatusLength;
- unsigned int CurrentStatusLength;
- unsigned int ProgressBufferLength;
- unsigned int UserStatusLength;
- struct dma_loaf DmaPages;
- unsigned long MonitoringTimerCount;
- unsigned long PrimaryMonitoringTime;
- unsigned long SecondaryMonitoringTime;
- unsigned long ShutdownMonitoringTimer;
- unsigned long LastProgressReportTime;
- unsigned long LastCurrentStatusTime;
- bool ControllerInitialized;
- bool MonitoringCommandDeferred;
- bool EphemeralProgressMessage;
- bool DriveSpinUpMessageDisplayed;
- bool MonitoringAlertMode;
- bool SuppressEnclosureMessages;
- struct timer_list MonitoringTimer;
- struct gendisk *disks[DAC960_MaxLogicalDrives];
- struct dma_pool *ScatterGatherPool;
- DAC960_Command_T *FreeCommands;
- unsigned char *CombinedStatusBuffer;
- unsigned char *CurrentStatusBuffer;
- struct request_queue *RequestQueue[DAC960_MaxLogicalDrives];
- int req_q_index;
- spinlock_t queue_lock;
- wait_queue_head_t CommandWaitQueue;
- wait_queue_head_t HealthStatusWaitQueue;
- DAC960_Command_T InitialCommand;
- DAC960_Command_T *Commands[DAC960_MaxDriverQueueDepth];
- struct proc_dir_entry *ControllerProcEntry;
- bool LogicalDriveInitiallyAccessible[DAC960_MaxLogicalDrives];
- void (*QueueCommand)(DAC960_Command_T *Command);
- bool (*ReadControllerConfiguration)(struct DAC960_Controller *);
- bool (*ReadDeviceConfiguration)(struct DAC960_Controller *);
- bool (*ReportDeviceConfiguration)(struct DAC960_Controller *);
- void (*QueueReadWriteCommand)(DAC960_Command_T *Command);
- union {
- struct {
- unsigned char GeometryTranslationHeads;
- unsigned char GeometryTranslationSectors;
- unsigned char PendingRebuildFlag;
- unsigned short StripeSize;
- unsigned short SegmentSize;
- unsigned short NewEventLogSequenceNumber;
- unsigned short OldEventLogSequenceNumber;
- unsigned short DeviceStateChannel;
- unsigned short DeviceStateTargetID;
- bool DualModeMemoryMailboxInterface;
- bool BackgroundInitializationStatusSupported;
- bool SAFTE_EnclosureManagementEnabled;
- bool NeedLogicalDriveInformation;
- bool NeedErrorTableInformation;
- bool NeedDeviceStateInformation;
- bool NeedDeviceInquiryInformation;
- bool NeedDeviceSerialNumberInformation;
- bool NeedRebuildProgress;
- bool NeedConsistencyCheckProgress;
- bool NeedBackgroundInitializationStatus;
- bool StartDeviceStateScan;
- bool RebuildProgressFirst;
- bool RebuildFlagPending;
- bool RebuildStatusPending;
-
- dma_addr_t FirstCommandMailboxDMA;
- DAC960_V1_CommandMailbox_T *FirstCommandMailbox;
- DAC960_V1_CommandMailbox_T *LastCommandMailbox;
- DAC960_V1_CommandMailbox_T *NextCommandMailbox;
- DAC960_V1_CommandMailbox_T *PreviousCommandMailbox1;
- DAC960_V1_CommandMailbox_T *PreviousCommandMailbox2;
-
- dma_addr_t FirstStatusMailboxDMA;
- DAC960_V1_StatusMailbox_T *FirstStatusMailbox;
- DAC960_V1_StatusMailbox_T *LastStatusMailbox;
- DAC960_V1_StatusMailbox_T *NextStatusMailbox;
-
- DAC960_V1_DCDB_T *MonitoringDCDB;
- dma_addr_t MonitoringDCDB_DMA;
-
- DAC960_V1_Enquiry_T Enquiry;
- DAC960_V1_Enquiry_T *NewEnquiry;
- dma_addr_t NewEnquiryDMA;
-
- DAC960_V1_ErrorTable_T ErrorTable;
- DAC960_V1_ErrorTable_T *NewErrorTable;
- dma_addr_t NewErrorTableDMA;
-
- DAC960_V1_EventLogEntry_T *EventLogEntry;
- dma_addr_t EventLogEntryDMA;
-
- DAC960_V1_RebuildProgress_T *RebuildProgress;
- dma_addr_t RebuildProgressDMA;
- DAC960_V1_CommandStatus_T LastRebuildStatus;
- DAC960_V1_CommandStatus_T PendingRebuildStatus;
-
- DAC960_V1_LogicalDriveInformationArray_T LogicalDriveInformation;
- DAC960_V1_LogicalDriveInformationArray_T *NewLogicalDriveInformation;
- dma_addr_t NewLogicalDriveInformationDMA;
-
- DAC960_V1_BackgroundInitializationStatus_T
- *BackgroundInitializationStatus;
- dma_addr_t BackgroundInitializationStatusDMA;
- DAC960_V1_BackgroundInitializationStatus_T
- LastBackgroundInitializationStatus;
-
- DAC960_V1_DeviceState_T
- DeviceState[DAC960_V1_MaxChannels][DAC960_V1_MaxTargets];
- DAC960_V1_DeviceState_T *NewDeviceState;
- dma_addr_t NewDeviceStateDMA;
-
- DAC960_SCSI_Inquiry_T
- InquiryStandardData[DAC960_V1_MaxChannels][DAC960_V1_MaxTargets];
- DAC960_SCSI_Inquiry_T *NewInquiryStandardData;
- dma_addr_t NewInquiryStandardDataDMA;
-
- DAC960_SCSI_Inquiry_UnitSerialNumber_T
- InquiryUnitSerialNumber[DAC960_V1_MaxChannels][DAC960_V1_MaxTargets];
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *NewInquiryUnitSerialNumber;
- dma_addr_t NewInquiryUnitSerialNumberDMA;
-
- int DeviceResetCount[DAC960_V1_MaxChannels][DAC960_V1_MaxTargets];
- bool DirectCommandActive[DAC960_V1_MaxChannels][DAC960_V1_MaxTargets];
- } V1;
- struct {
- unsigned int StatusChangeCounter;
- unsigned int NextEventSequenceNumber;
- unsigned int PhysicalDeviceIndex;
- bool NeedLogicalDeviceInformation;
- bool NeedPhysicalDeviceInformation;
- bool NeedDeviceSerialNumberInformation;
- bool StartLogicalDeviceInformationScan;
- bool StartPhysicalDeviceInformationScan;
- struct dma_pool *RequestSensePool;
-
- dma_addr_t FirstCommandMailboxDMA;
- DAC960_V2_CommandMailbox_T *FirstCommandMailbox;
- DAC960_V2_CommandMailbox_T *LastCommandMailbox;
- DAC960_V2_CommandMailbox_T *NextCommandMailbox;
- DAC960_V2_CommandMailbox_T *PreviousCommandMailbox1;
- DAC960_V2_CommandMailbox_T *PreviousCommandMailbox2;
-
- dma_addr_t FirstStatusMailboxDMA;
- DAC960_V2_StatusMailbox_T *FirstStatusMailbox;
- DAC960_V2_StatusMailbox_T *LastStatusMailbox;
- DAC960_V2_StatusMailbox_T *NextStatusMailbox;
-
- dma_addr_t HealthStatusBufferDMA;
- DAC960_V2_HealthStatusBuffer_T *HealthStatusBuffer;
-
- DAC960_V2_ControllerInfo_T ControllerInformation;
- DAC960_V2_ControllerInfo_T *NewControllerInformation;
- dma_addr_t NewControllerInformationDMA;
-
- DAC960_V2_LogicalDeviceInfo_T
- *LogicalDeviceInformation[DAC960_MaxLogicalDrives];
- DAC960_V2_LogicalDeviceInfo_T *NewLogicalDeviceInformation;
- dma_addr_t NewLogicalDeviceInformationDMA;
-
- DAC960_V2_PhysicalDeviceInfo_T
- *PhysicalDeviceInformation[DAC960_V2_MaxPhysicalDevices];
- DAC960_V2_PhysicalDeviceInfo_T *NewPhysicalDeviceInformation;
- dma_addr_t NewPhysicalDeviceInformationDMA;
-
- DAC960_SCSI_Inquiry_UnitSerialNumber_T *NewInquiryUnitSerialNumber;
- dma_addr_t NewInquiryUnitSerialNumberDMA;
- DAC960_SCSI_Inquiry_UnitSerialNumber_T
- *InquiryUnitSerialNumber[DAC960_V2_MaxPhysicalDevices];
-
- DAC960_V2_Event_T *Event;
- dma_addr_t EventDMA;
-
- DAC960_V2_PhysicalToLogicalDevice_T *PhysicalToLogicalDevice;
- dma_addr_t PhysicalToLogicalDeviceDMA;
-
- DAC960_V2_PhysicalDevice_T
- LogicalDriveToVirtualDevice[DAC960_MaxLogicalDrives];
- bool LogicalDriveFoundDuringScan[DAC960_MaxLogicalDrives];
- } V2;
- } FW;
- unsigned char ProgressBuffer[DAC960_ProgressBufferSize];
- unsigned char UserStatusBuffer[DAC960_UserMessageSize];
-}
-DAC960_Controller_T;
-
-
-/*
- Simplify access to Firmware Version Dependent Data Structure Components
- and Functions.
-*/
-
-#define V1 FW.V1
-#define V2 FW.V2
-#define DAC960_QueueCommand(Command) \
- (Controller->QueueCommand)(Command)
-#define DAC960_ReadControllerConfiguration(Controller) \
- (Controller->ReadControllerConfiguration)(Controller)
-#define DAC960_ReadDeviceConfiguration(Controller) \
- (Controller->ReadDeviceConfiguration)(Controller)
-#define DAC960_ReportDeviceConfiguration(Controller) \
- (Controller->ReportDeviceConfiguration)(Controller)
-#define DAC960_QueueReadWriteCommand(Command) \
- (Controller->QueueReadWriteCommand)(Command)
-
-/*
- * dma_addr_writeql is provided to write dma_addr_t types
- * to a 64-bit pci address space register. The controller
- * will accept having the register written as two 32-bit
- * values.
- *
- * In HIGHMEM kernels, dma_addr_t is a 64-bit value.
- * without HIGHMEM, dma_addr_t is a 32-bit value.
- *
- * The compiler should always fix up the assignment
- * to u.wq appropriately, depending upon the size of
- * dma_addr_t.
- */
-static inline
-void dma_addr_writeql(dma_addr_t addr, void __iomem *write_address)
-{
- union {
- u64 wq;
- uint wl[2];
- } u;
-
- u.wq = addr;
-
- writel(u.wl[0], write_address);
- writel(u.wl[1], write_address + 4);
-}
-
-/*
- Define the DAC960 GEM Series Controller Interface Register Offsets.
- */
-
-#define DAC960_GEM_RegisterWindowSize 0x600
-
-typedef enum
-{
- DAC960_GEM_InboundDoorBellRegisterReadSetOffset = 0x214,
- DAC960_GEM_InboundDoorBellRegisterClearOffset = 0x218,
- DAC960_GEM_OutboundDoorBellRegisterReadSetOffset = 0x224,
- DAC960_GEM_OutboundDoorBellRegisterClearOffset = 0x228,
- DAC960_GEM_InterruptStatusRegisterOffset = 0x208,
- DAC960_GEM_InterruptMaskRegisterReadSetOffset = 0x22C,
- DAC960_GEM_InterruptMaskRegisterClearOffset = 0x230,
- DAC960_GEM_CommandMailboxBusAddressOffset = 0x510,
- DAC960_GEM_CommandStatusOffset = 0x518,
- DAC960_GEM_ErrorStatusRegisterReadSetOffset = 0x224,
- DAC960_GEM_ErrorStatusRegisterClearOffset = 0x228,
-}
-DAC960_GEM_RegisterOffsets_T;
-
-/*
- Define the structure of the DAC960 GEM Series Inbound Door Bell
- */
-
-typedef union DAC960_GEM_InboundDoorBellRegister
-{
- unsigned int All;
- struct {
- unsigned int :24;
- bool HardwareMailboxNewCommand:1;
- bool AcknowledgeHardwareMailboxStatus:1;
- bool GenerateInterrupt:1;
- bool ControllerReset:1;
- bool MemoryMailboxNewCommand:1;
- unsigned int :3;
- } Write;
- struct {
- unsigned int :24;
- bool HardwareMailboxFull:1;
- bool InitializationInProgress:1;
- unsigned int :6;
- } Read;
-}
-DAC960_GEM_InboundDoorBellRegister_T;
-
-/*
- Define the structure of the DAC960 GEM Series Outbound Door Bell Register.
- */
-typedef union DAC960_GEM_OutboundDoorBellRegister
-{
- unsigned int All;
- struct {
- unsigned int :24;
- bool AcknowledgeHardwareMailboxInterrupt:1;
- bool AcknowledgeMemoryMailboxInterrupt:1;
- unsigned int :6;
- } Write;
- struct {
- unsigned int :24;
- bool HardwareMailboxStatusAvailable:1;
- bool MemoryMailboxStatusAvailable:1;
- unsigned int :6;
- } Read;
-}
-DAC960_GEM_OutboundDoorBellRegister_T;
-
-/*
- Define the structure of the DAC960 GEM Series Interrupt Mask Register.
- */
-typedef union DAC960_GEM_InterruptMaskRegister
-{
- unsigned int All;
- struct {
- unsigned int :16;
- unsigned int :8;
- unsigned int HardwareMailboxInterrupt:1;
- unsigned int MemoryMailboxInterrupt:1;
- unsigned int :6;
- } Bits;
-}
-DAC960_GEM_InterruptMaskRegister_T;
-
-/*
- Define the structure of the DAC960 GEM Series Error Status Register.
- */
-
-typedef union DAC960_GEM_ErrorStatusRegister
-{
- unsigned int All;
- struct {
- unsigned int :24;
- unsigned int :5;
- bool ErrorStatusPending:1;
- unsigned int :2;
- } Bits;
-}
-DAC960_GEM_ErrorStatusRegister_T;
-
-/*
- Define inline functions to provide an abstraction for reading and writing the
- DAC960 GEM Series Controller Interface Registers.
-*/
-
-static inline
-void DAC960_GEM_HardwareMailboxNewCommand(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.HardwareMailboxNewCommand = true;
- writel(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_GEM_InboundDoorBellRegisterReadSetOffset);
-}
-
-static inline
-void DAC960_GEM_AcknowledgeHardwareMailboxStatus(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.AcknowledgeHardwareMailboxStatus = true;
- writel(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_GEM_InboundDoorBellRegisterClearOffset);
-}
-
-static inline
-void DAC960_GEM_GenerateInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.GenerateInterrupt = true;
- writel(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_GEM_InboundDoorBellRegisterReadSetOffset);
-}
-
-static inline
-void DAC960_GEM_ControllerReset(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.ControllerReset = true;
- writel(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_GEM_InboundDoorBellRegisterReadSetOffset);
-}
-
-static inline
-void DAC960_GEM_MemoryMailboxNewCommand(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.MemoryMailboxNewCommand = true;
- writel(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_GEM_InboundDoorBellRegisterReadSetOffset);
-}
-
-static inline
-bool DAC960_GEM_HardwareMailboxFullP(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All =
- readl(ControllerBaseAddress +
- DAC960_GEM_InboundDoorBellRegisterReadSetOffset);
- return InboundDoorBellRegister.Read.HardwareMailboxFull;
-}
-
-static inline
-bool DAC960_GEM_InitializationInProgressP(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All =
- readl(ControllerBaseAddress +
- DAC960_GEM_InboundDoorBellRegisterReadSetOffset);
- return InboundDoorBellRegister.Read.InitializationInProgress;
-}
-
-static inline
-void DAC960_GEM_AcknowledgeHardwareMailboxInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeHardwareMailboxInterrupt = true;
- writel(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_GEM_OutboundDoorBellRegisterClearOffset);
-}
-
-static inline
-void DAC960_GEM_AcknowledgeMemoryMailboxInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeMemoryMailboxInterrupt = true;
- writel(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_GEM_OutboundDoorBellRegisterClearOffset);
-}
-
-static inline
-void DAC960_GEM_AcknowledgeInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeHardwareMailboxInterrupt = true;
- OutboundDoorBellRegister.Write.AcknowledgeMemoryMailboxInterrupt = true;
- writel(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_GEM_OutboundDoorBellRegisterClearOffset);
-}
-
-static inline
-bool DAC960_GEM_HardwareMailboxStatusAvailableP(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All =
- readl(ControllerBaseAddress +
- DAC960_GEM_OutboundDoorBellRegisterReadSetOffset);
- return OutboundDoorBellRegister.Read.HardwareMailboxStatusAvailable;
-}
-
-static inline
-bool DAC960_GEM_MemoryMailboxStatusAvailableP(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All =
- readl(ControllerBaseAddress +
- DAC960_GEM_OutboundDoorBellRegisterReadSetOffset);
- return OutboundDoorBellRegister.Read.MemoryMailboxStatusAvailable;
-}
-
-static inline
-void DAC960_GEM_EnableInterrupts(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All = 0;
- InterruptMaskRegister.Bits.HardwareMailboxInterrupt = true;
- InterruptMaskRegister.Bits.MemoryMailboxInterrupt = true;
- writel(InterruptMaskRegister.All,
- ControllerBaseAddress + DAC960_GEM_InterruptMaskRegisterClearOffset);
-}
-
-static inline
-void DAC960_GEM_DisableInterrupts(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All = 0;
- InterruptMaskRegister.Bits.HardwareMailboxInterrupt = true;
- InterruptMaskRegister.Bits.MemoryMailboxInterrupt = true;
- writel(InterruptMaskRegister.All,
- ControllerBaseAddress + DAC960_GEM_InterruptMaskRegisterReadSetOffset);
-}
-
-static inline
-bool DAC960_GEM_InterruptsEnabledP(void __iomem *ControllerBaseAddress)
-{
- DAC960_GEM_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All =
- readl(ControllerBaseAddress +
- DAC960_GEM_InterruptMaskRegisterReadSetOffset);
- return !(InterruptMaskRegister.Bits.HardwareMailboxInterrupt ||
- InterruptMaskRegister.Bits.MemoryMailboxInterrupt);
-}
-
-static inline
-void DAC960_GEM_WriteCommandMailbox(DAC960_V2_CommandMailbox_T
- *MemoryCommandMailbox,
- DAC960_V2_CommandMailbox_T
- *CommandMailbox)
-{
- memcpy(&MemoryCommandMailbox->Words[1], &CommandMailbox->Words[1],
- sizeof(DAC960_V2_CommandMailbox_T) - sizeof(unsigned int));
- wmb();
- MemoryCommandMailbox->Words[0] = CommandMailbox->Words[0];
- mb();
-}
-
-static inline
-void DAC960_GEM_WriteHardwareMailbox(void __iomem *ControllerBaseAddress,
- dma_addr_t CommandMailboxDMA)
-{
- dma_addr_writeql(CommandMailboxDMA,
- ControllerBaseAddress +
- DAC960_GEM_CommandMailboxBusAddressOffset);
-}
-
-static inline DAC960_V2_CommandIdentifier_T
-DAC960_GEM_ReadCommandIdentifier(void __iomem *ControllerBaseAddress)
-{
- return readw(ControllerBaseAddress + DAC960_GEM_CommandStatusOffset);
-}
-
-static inline DAC960_V2_CommandStatus_T
-DAC960_GEM_ReadCommandStatus(void __iomem *ControllerBaseAddress)
-{
- return readw(ControllerBaseAddress + DAC960_GEM_CommandStatusOffset + 2);
-}
-
-static inline bool
-DAC960_GEM_ReadErrorStatus(void __iomem *ControllerBaseAddress,
- unsigned char *ErrorStatus,
- unsigned char *Parameter0,
- unsigned char *Parameter1)
-{
- DAC960_GEM_ErrorStatusRegister_T ErrorStatusRegister;
- ErrorStatusRegister.All =
- readl(ControllerBaseAddress + DAC960_GEM_ErrorStatusRegisterReadSetOffset);
- if (!ErrorStatusRegister.Bits.ErrorStatusPending) return false;
- ErrorStatusRegister.Bits.ErrorStatusPending = false;
- *ErrorStatus = ErrorStatusRegister.All;
- *Parameter0 =
- readb(ControllerBaseAddress + DAC960_GEM_CommandMailboxBusAddressOffset + 0);
- *Parameter1 =
- readb(ControllerBaseAddress + DAC960_GEM_CommandMailboxBusAddressOffset + 1);
- writel(0x03000000, ControllerBaseAddress +
- DAC960_GEM_ErrorStatusRegisterClearOffset);
- return true;
-}
-
-/*
- Define the DAC960 BA Series Controller Interface Register Offsets.
-*/
-
-#define DAC960_BA_RegisterWindowSize 0x80
-
-typedef enum
-{
- DAC960_BA_InboundDoorBellRegisterOffset = 0x60,
- DAC960_BA_OutboundDoorBellRegisterOffset = 0x61,
- DAC960_BA_InterruptStatusRegisterOffset = 0x30,
- DAC960_BA_InterruptMaskRegisterOffset = 0x34,
- DAC960_BA_CommandMailboxBusAddressOffset = 0x50,
- DAC960_BA_CommandStatusOffset = 0x58,
- DAC960_BA_ErrorStatusRegisterOffset = 0x63
-}
-DAC960_BA_RegisterOffsets_T;
-
-
-/*
- Define the structure of the DAC960 BA Series Inbound Door Bell Register.
-*/
-
-typedef union DAC960_BA_InboundDoorBellRegister
-{
- unsigned char All;
- struct {
- bool HardwareMailboxNewCommand:1; /* Bit 0 */
- bool AcknowledgeHardwareMailboxStatus:1; /* Bit 1 */
- bool GenerateInterrupt:1; /* Bit 2 */
- bool ControllerReset:1; /* Bit 3 */
- bool MemoryMailboxNewCommand:1; /* Bit 4 */
- unsigned char :3; /* Bits 5-7 */
- } Write;
- struct {
- bool HardwareMailboxEmpty:1; /* Bit 0 */
- bool InitializationNotInProgress:1; /* Bit 1 */
- unsigned char :6; /* Bits 2-7 */
- } Read;
-}
-DAC960_BA_InboundDoorBellRegister_T;
-
-
-/*
- Define the structure of the DAC960 BA Series Outbound Door Bell Register.
-*/
-
-typedef union DAC960_BA_OutboundDoorBellRegister
-{
- unsigned char All;
- struct {
- bool AcknowledgeHardwareMailboxInterrupt:1; /* Bit 0 */
- bool AcknowledgeMemoryMailboxInterrupt:1; /* Bit 1 */
- unsigned char :6; /* Bits 2-7 */
- } Write;
- struct {
- bool HardwareMailboxStatusAvailable:1; /* Bit 0 */
- bool MemoryMailboxStatusAvailable:1; /* Bit 1 */
- unsigned char :6; /* Bits 2-7 */
- } Read;
-}
-DAC960_BA_OutboundDoorBellRegister_T;
-
-
-/*
- Define the structure of the DAC960 BA Series Interrupt Mask Register.
-*/
-
-typedef union DAC960_BA_InterruptMaskRegister
-{
- unsigned char All;
- struct {
- unsigned int :2; /* Bits 0-1 */
- bool DisableInterrupts:1; /* Bit 2 */
- bool DisableInterruptsI2O:1; /* Bit 3 */
- unsigned int :4; /* Bits 4-7 */
- } Bits;
-}
-DAC960_BA_InterruptMaskRegister_T;
-
-
-/*
- Define the structure of the DAC960 BA Series Error Status Register.
-*/
-
-typedef union DAC960_BA_ErrorStatusRegister
-{
- unsigned char All;
- struct {
- unsigned int :2; /* Bits 0-1 */
- bool ErrorStatusPending:1; /* Bit 2 */
- unsigned int :5; /* Bits 3-7 */
- } Bits;
-}
-DAC960_BA_ErrorStatusRegister_T;
-
-
-/*
- Define inline functions to provide an abstraction for reading and writing the
- DAC960 BA Series Controller Interface Registers.
-*/
-
-static inline
-void DAC960_BA_HardwareMailboxNewCommand(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.HardwareMailboxNewCommand = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_BA_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_BA_AcknowledgeHardwareMailboxStatus(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.AcknowledgeHardwareMailboxStatus = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_BA_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_BA_GenerateInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.GenerateInterrupt = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_BA_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_BA_ControllerReset(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.ControllerReset = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_BA_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_BA_MemoryMailboxNewCommand(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.MemoryMailboxNewCommand = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_BA_InboundDoorBellRegisterOffset);
-}
-
-static inline
-bool DAC960_BA_HardwareMailboxFullP(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_BA_InboundDoorBellRegisterOffset);
- return !InboundDoorBellRegister.Read.HardwareMailboxEmpty;
-}
-
-static inline
-bool DAC960_BA_InitializationInProgressP(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_BA_InboundDoorBellRegisterOffset);
- return !InboundDoorBellRegister.Read.InitializationNotInProgress;
-}
-
-static inline
-void DAC960_BA_AcknowledgeHardwareMailboxInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeHardwareMailboxInterrupt = true;
- writeb(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_BA_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_BA_AcknowledgeMemoryMailboxInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeMemoryMailboxInterrupt = true;
- writeb(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_BA_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_BA_AcknowledgeInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeHardwareMailboxInterrupt = true;
- OutboundDoorBellRegister.Write.AcknowledgeMemoryMailboxInterrupt = true;
- writeb(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_BA_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-bool DAC960_BA_HardwareMailboxStatusAvailableP(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_BA_OutboundDoorBellRegisterOffset);
- return OutboundDoorBellRegister.Read.HardwareMailboxStatusAvailable;
-}
-
-static inline
-bool DAC960_BA_MemoryMailboxStatusAvailableP(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_BA_OutboundDoorBellRegisterOffset);
- return OutboundDoorBellRegister.Read.MemoryMailboxStatusAvailable;
-}
-
-static inline
-void DAC960_BA_EnableInterrupts(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All = 0xFF;
- InterruptMaskRegister.Bits.DisableInterrupts = false;
- InterruptMaskRegister.Bits.DisableInterruptsI2O = true;
- writeb(InterruptMaskRegister.All,
- ControllerBaseAddress + DAC960_BA_InterruptMaskRegisterOffset);
-}
-
-static inline
-void DAC960_BA_DisableInterrupts(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All = 0xFF;
- InterruptMaskRegister.Bits.DisableInterrupts = true;
- InterruptMaskRegister.Bits.DisableInterruptsI2O = true;
- writeb(InterruptMaskRegister.All,
- ControllerBaseAddress + DAC960_BA_InterruptMaskRegisterOffset);
-}
-
-static inline
-bool DAC960_BA_InterruptsEnabledP(void __iomem *ControllerBaseAddress)
-{
- DAC960_BA_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All =
- readb(ControllerBaseAddress + DAC960_BA_InterruptMaskRegisterOffset);
- return !InterruptMaskRegister.Bits.DisableInterrupts;
-}
-
-static inline
-void DAC960_BA_WriteCommandMailbox(DAC960_V2_CommandMailbox_T
- *MemoryCommandMailbox,
- DAC960_V2_CommandMailbox_T
- *CommandMailbox)
-{
- memcpy(&MemoryCommandMailbox->Words[1], &CommandMailbox->Words[1],
- sizeof(DAC960_V2_CommandMailbox_T) - sizeof(unsigned int));
- wmb();
- MemoryCommandMailbox->Words[0] = CommandMailbox->Words[0];
- mb();
-}
-
-
-static inline
-void DAC960_BA_WriteHardwareMailbox(void __iomem *ControllerBaseAddress,
- dma_addr_t CommandMailboxDMA)
-{
- dma_addr_writeql(CommandMailboxDMA,
- ControllerBaseAddress +
- DAC960_BA_CommandMailboxBusAddressOffset);
-}
-
-static inline DAC960_V2_CommandIdentifier_T
-DAC960_BA_ReadCommandIdentifier(void __iomem *ControllerBaseAddress)
-{
- return readw(ControllerBaseAddress + DAC960_BA_CommandStatusOffset);
-}
-
-static inline DAC960_V2_CommandStatus_T
-DAC960_BA_ReadCommandStatus(void __iomem *ControllerBaseAddress)
-{
- return readw(ControllerBaseAddress + DAC960_BA_CommandStatusOffset + 2);
-}
-
-static inline bool
-DAC960_BA_ReadErrorStatus(void __iomem *ControllerBaseAddress,
- unsigned char *ErrorStatus,
- unsigned char *Parameter0,
- unsigned char *Parameter1)
-{
- DAC960_BA_ErrorStatusRegister_T ErrorStatusRegister;
- ErrorStatusRegister.All =
- readb(ControllerBaseAddress + DAC960_BA_ErrorStatusRegisterOffset);
- if (!ErrorStatusRegister.Bits.ErrorStatusPending) return false;
- ErrorStatusRegister.Bits.ErrorStatusPending = false;
- *ErrorStatus = ErrorStatusRegister.All;
- *Parameter0 =
- readb(ControllerBaseAddress + DAC960_BA_CommandMailboxBusAddressOffset + 0);
- *Parameter1 =
- readb(ControllerBaseAddress + DAC960_BA_CommandMailboxBusAddressOffset + 1);
- writeb(0xFF, ControllerBaseAddress + DAC960_BA_ErrorStatusRegisterOffset);
- return true;
-}
-
-
-/*
- Define the DAC960 LP Series Controller Interface Register Offsets.
-*/
-
-#define DAC960_LP_RegisterWindowSize 0x80
-
-typedef enum
-{
- DAC960_LP_InboundDoorBellRegisterOffset = 0x20,
- DAC960_LP_OutboundDoorBellRegisterOffset = 0x2C,
- DAC960_LP_InterruptStatusRegisterOffset = 0x30,
- DAC960_LP_InterruptMaskRegisterOffset = 0x34,
- DAC960_LP_CommandMailboxBusAddressOffset = 0x10,
- DAC960_LP_CommandStatusOffset = 0x18,
- DAC960_LP_ErrorStatusRegisterOffset = 0x2E
-}
-DAC960_LP_RegisterOffsets_T;
-
-
-/*
- Define the structure of the DAC960 LP Series Inbound Door Bell Register.
-*/
-
-typedef union DAC960_LP_InboundDoorBellRegister
-{
- unsigned char All;
- struct {
- bool HardwareMailboxNewCommand:1; /* Bit 0 */
- bool AcknowledgeHardwareMailboxStatus:1; /* Bit 1 */
- bool GenerateInterrupt:1; /* Bit 2 */
- bool ControllerReset:1; /* Bit 3 */
- bool MemoryMailboxNewCommand:1; /* Bit 4 */
- unsigned char :3; /* Bits 5-7 */
- } Write;
- struct {
- bool HardwareMailboxFull:1; /* Bit 0 */
- bool InitializationInProgress:1; /* Bit 1 */
- unsigned char :6; /* Bits 2-7 */
- } Read;
-}
-DAC960_LP_InboundDoorBellRegister_T;
-
-
-/*
- Define the structure of the DAC960 LP Series Outbound Door Bell Register.
-*/
-
-typedef union DAC960_LP_OutboundDoorBellRegister
-{
- unsigned char All;
- struct {
- bool AcknowledgeHardwareMailboxInterrupt:1; /* Bit 0 */
- bool AcknowledgeMemoryMailboxInterrupt:1; /* Bit 1 */
- unsigned char :6; /* Bits 2-7 */
- } Write;
- struct {
- bool HardwareMailboxStatusAvailable:1; /* Bit 0 */
- bool MemoryMailboxStatusAvailable:1; /* Bit 1 */
- unsigned char :6; /* Bits 2-7 */
- } Read;
-}
-DAC960_LP_OutboundDoorBellRegister_T;
-
-
-/*
- Define the structure of the DAC960 LP Series Interrupt Mask Register.
-*/
-
-typedef union DAC960_LP_InterruptMaskRegister
-{
- unsigned char All;
- struct {
- unsigned int :2; /* Bits 0-1 */
- bool DisableInterrupts:1; /* Bit 2 */
- unsigned int :5; /* Bits 3-7 */
- } Bits;
-}
-DAC960_LP_InterruptMaskRegister_T;
-
-
-/*
- Define the structure of the DAC960 LP Series Error Status Register.
-*/
-
-typedef union DAC960_LP_ErrorStatusRegister
-{
- unsigned char All;
- struct {
- unsigned int :2; /* Bits 0-1 */
- bool ErrorStatusPending:1; /* Bit 2 */
- unsigned int :5; /* Bits 3-7 */
- } Bits;
-}
-DAC960_LP_ErrorStatusRegister_T;
-
-
-/*
- Define inline functions to provide an abstraction for reading and writing the
- DAC960 LP Series Controller Interface Registers.
-*/
-
-static inline
-void DAC960_LP_HardwareMailboxNewCommand(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.HardwareMailboxNewCommand = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LP_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_LP_AcknowledgeHardwareMailboxStatus(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.AcknowledgeHardwareMailboxStatus = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LP_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_LP_GenerateInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.GenerateInterrupt = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LP_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_LP_ControllerReset(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.ControllerReset = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LP_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_LP_MemoryMailboxNewCommand(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.MemoryMailboxNewCommand = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LP_InboundDoorBellRegisterOffset);
-}
-
-static inline
-bool DAC960_LP_HardwareMailboxFullP(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_LP_InboundDoorBellRegisterOffset);
- return InboundDoorBellRegister.Read.HardwareMailboxFull;
-}
-
-static inline
-bool DAC960_LP_InitializationInProgressP(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_LP_InboundDoorBellRegisterOffset);
- return InboundDoorBellRegister.Read.InitializationInProgress;
-}
-
-static inline
-void DAC960_LP_AcknowledgeHardwareMailboxInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeHardwareMailboxInterrupt = true;
- writeb(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LP_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_LP_AcknowledgeMemoryMailboxInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeMemoryMailboxInterrupt = true;
- writeb(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LP_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_LP_AcknowledgeInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeHardwareMailboxInterrupt = true;
- OutboundDoorBellRegister.Write.AcknowledgeMemoryMailboxInterrupt = true;
- writeb(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LP_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-bool DAC960_LP_HardwareMailboxStatusAvailableP(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_LP_OutboundDoorBellRegisterOffset);
- return OutboundDoorBellRegister.Read.HardwareMailboxStatusAvailable;
-}
-
-static inline
-bool DAC960_LP_MemoryMailboxStatusAvailableP(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_LP_OutboundDoorBellRegisterOffset);
- return OutboundDoorBellRegister.Read.MemoryMailboxStatusAvailable;
-}
-
-static inline
-void DAC960_LP_EnableInterrupts(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All = 0xFF;
- InterruptMaskRegister.Bits.DisableInterrupts = false;
- writeb(InterruptMaskRegister.All,
- ControllerBaseAddress + DAC960_LP_InterruptMaskRegisterOffset);
-}
-
-static inline
-void DAC960_LP_DisableInterrupts(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All = 0xFF;
- InterruptMaskRegister.Bits.DisableInterrupts = true;
- writeb(InterruptMaskRegister.All,
- ControllerBaseAddress + DAC960_LP_InterruptMaskRegisterOffset);
-}
-
-static inline
-bool DAC960_LP_InterruptsEnabledP(void __iomem *ControllerBaseAddress)
-{
- DAC960_LP_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All =
- readb(ControllerBaseAddress + DAC960_LP_InterruptMaskRegisterOffset);
- return !InterruptMaskRegister.Bits.DisableInterrupts;
-}
-
-static inline
-void DAC960_LP_WriteCommandMailbox(DAC960_V2_CommandMailbox_T
- *MemoryCommandMailbox,
- DAC960_V2_CommandMailbox_T
- *CommandMailbox)
-{
- memcpy(&MemoryCommandMailbox->Words[1], &CommandMailbox->Words[1],
- sizeof(DAC960_V2_CommandMailbox_T) - sizeof(unsigned int));
- wmb();
- MemoryCommandMailbox->Words[0] = CommandMailbox->Words[0];
- mb();
-}
-
-static inline
-void DAC960_LP_WriteHardwareMailbox(void __iomem *ControllerBaseAddress,
- dma_addr_t CommandMailboxDMA)
-{
- dma_addr_writeql(CommandMailboxDMA,
- ControllerBaseAddress +
- DAC960_LP_CommandMailboxBusAddressOffset);
-}
-
-static inline DAC960_V2_CommandIdentifier_T
-DAC960_LP_ReadCommandIdentifier(void __iomem *ControllerBaseAddress)
-{
- return readw(ControllerBaseAddress + DAC960_LP_CommandStatusOffset);
-}
-
-static inline DAC960_V2_CommandStatus_T
-DAC960_LP_ReadCommandStatus(void __iomem *ControllerBaseAddress)
-{
- return readw(ControllerBaseAddress + DAC960_LP_CommandStatusOffset + 2);
-}
-
-static inline bool
-DAC960_LP_ReadErrorStatus(void __iomem *ControllerBaseAddress,
- unsigned char *ErrorStatus,
- unsigned char *Parameter0,
- unsigned char *Parameter1)
-{
- DAC960_LP_ErrorStatusRegister_T ErrorStatusRegister;
- ErrorStatusRegister.All =
- readb(ControllerBaseAddress + DAC960_LP_ErrorStatusRegisterOffset);
- if (!ErrorStatusRegister.Bits.ErrorStatusPending) return false;
- ErrorStatusRegister.Bits.ErrorStatusPending = false;
- *ErrorStatus = ErrorStatusRegister.All;
- *Parameter0 =
- readb(ControllerBaseAddress + DAC960_LP_CommandMailboxBusAddressOffset + 0);
- *Parameter1 =
- readb(ControllerBaseAddress + DAC960_LP_CommandMailboxBusAddressOffset + 1);
- writeb(0xFF, ControllerBaseAddress + DAC960_LP_ErrorStatusRegisterOffset);
- return true;
-}
-
-
-/*
- Define the DAC960 LA Series Controller Interface Register Offsets.
-*/
-
-#define DAC960_LA_RegisterWindowSize 0x80
-
-typedef enum
-{
- DAC960_LA_InboundDoorBellRegisterOffset = 0x60,
- DAC960_LA_OutboundDoorBellRegisterOffset = 0x61,
- DAC960_LA_InterruptMaskRegisterOffset = 0x34,
- DAC960_LA_CommandOpcodeRegisterOffset = 0x50,
- DAC960_LA_CommandIdentifierRegisterOffset = 0x51,
- DAC960_LA_MailboxRegister2Offset = 0x52,
- DAC960_LA_MailboxRegister3Offset = 0x53,
- DAC960_LA_MailboxRegister4Offset = 0x54,
- DAC960_LA_MailboxRegister5Offset = 0x55,
- DAC960_LA_MailboxRegister6Offset = 0x56,
- DAC960_LA_MailboxRegister7Offset = 0x57,
- DAC960_LA_MailboxRegister8Offset = 0x58,
- DAC960_LA_MailboxRegister9Offset = 0x59,
- DAC960_LA_MailboxRegister10Offset = 0x5A,
- DAC960_LA_MailboxRegister11Offset = 0x5B,
- DAC960_LA_MailboxRegister12Offset = 0x5C,
- DAC960_LA_StatusCommandIdentifierRegOffset = 0x5D,
- DAC960_LA_StatusRegisterOffset = 0x5E,
- DAC960_LA_ErrorStatusRegisterOffset = 0x63
-}
-DAC960_LA_RegisterOffsets_T;
-
-
-/*
- Define the structure of the DAC960 LA Series Inbound Door Bell Register.
-*/
-
-typedef union DAC960_LA_InboundDoorBellRegister
-{
- unsigned char All;
- struct {
- bool HardwareMailboxNewCommand:1; /* Bit 0 */
- bool AcknowledgeHardwareMailboxStatus:1; /* Bit 1 */
- bool GenerateInterrupt:1; /* Bit 2 */
- bool ControllerReset:1; /* Bit 3 */
- bool MemoryMailboxNewCommand:1; /* Bit 4 */
- unsigned char :3; /* Bits 5-7 */
- } Write;
- struct {
- bool HardwareMailboxEmpty:1; /* Bit 0 */
- bool InitializationNotInProgress:1; /* Bit 1 */
- unsigned char :6; /* Bits 2-7 */
- } Read;
-}
-DAC960_LA_InboundDoorBellRegister_T;
-
-
-/*
- Define the structure of the DAC960 LA Series Outbound Door Bell Register.
-*/
-
-typedef union DAC960_LA_OutboundDoorBellRegister
-{
- unsigned char All;
- struct {
- bool AcknowledgeHardwareMailboxInterrupt:1; /* Bit 0 */
- bool AcknowledgeMemoryMailboxInterrupt:1; /* Bit 1 */
- unsigned char :6; /* Bits 2-7 */
- } Write;
- struct {
- bool HardwareMailboxStatusAvailable:1; /* Bit 0 */
- bool MemoryMailboxStatusAvailable:1; /* Bit 1 */
- unsigned char :6; /* Bits 2-7 */
- } Read;
-}
-DAC960_LA_OutboundDoorBellRegister_T;
-
-
-/*
- Define the structure of the DAC960 LA Series Interrupt Mask Register.
-*/
-
-typedef union DAC960_LA_InterruptMaskRegister
-{
- unsigned char All;
- struct {
- unsigned char :2; /* Bits 0-1 */
- bool DisableInterrupts:1; /* Bit 2 */
- unsigned char :5; /* Bits 3-7 */
- } Bits;
-}
-DAC960_LA_InterruptMaskRegister_T;
-
-
-/*
- Define the structure of the DAC960 LA Series Error Status Register.
-*/
-
-typedef union DAC960_LA_ErrorStatusRegister
-{
- unsigned char All;
- struct {
- unsigned int :2; /* Bits 0-1 */
- bool ErrorStatusPending:1; /* Bit 2 */
- unsigned int :5; /* Bits 3-7 */
- } Bits;
-}
-DAC960_LA_ErrorStatusRegister_T;
-
-
-/*
- Define inline functions to provide an abstraction for reading and writing the
- DAC960 LA Series Controller Interface Registers.
-*/
-
-static inline
-void DAC960_LA_HardwareMailboxNewCommand(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.HardwareMailboxNewCommand = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LA_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_LA_AcknowledgeHardwareMailboxStatus(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.AcknowledgeHardwareMailboxStatus = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LA_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_LA_GenerateInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.GenerateInterrupt = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LA_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_LA_ControllerReset(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.ControllerReset = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LA_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_LA_MemoryMailboxNewCommand(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.MemoryMailboxNewCommand = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LA_InboundDoorBellRegisterOffset);
-}
-
-static inline
-bool DAC960_LA_HardwareMailboxFullP(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_LA_InboundDoorBellRegisterOffset);
- return !InboundDoorBellRegister.Read.HardwareMailboxEmpty;
-}
-
-static inline
-bool DAC960_LA_InitializationInProgressP(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_LA_InboundDoorBellRegisterOffset);
- return !InboundDoorBellRegister.Read.InitializationNotInProgress;
-}
-
-static inline
-void DAC960_LA_AcknowledgeHardwareMailboxInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeHardwareMailboxInterrupt = true;
- writeb(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LA_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_LA_AcknowledgeMemoryMailboxInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeMemoryMailboxInterrupt = true;
- writeb(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LA_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_LA_AcknowledgeInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeHardwareMailboxInterrupt = true;
- OutboundDoorBellRegister.Write.AcknowledgeMemoryMailboxInterrupt = true;
- writeb(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_LA_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-bool DAC960_LA_HardwareMailboxStatusAvailableP(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_LA_OutboundDoorBellRegisterOffset);
- return OutboundDoorBellRegister.Read.HardwareMailboxStatusAvailable;
-}
-
-static inline
-bool DAC960_LA_MemoryMailboxStatusAvailableP(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_LA_OutboundDoorBellRegisterOffset);
- return OutboundDoorBellRegister.Read.MemoryMailboxStatusAvailable;
-}
-
-static inline
-void DAC960_LA_EnableInterrupts(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All = 0xFF;
- InterruptMaskRegister.Bits.DisableInterrupts = false;
- writeb(InterruptMaskRegister.All,
- ControllerBaseAddress + DAC960_LA_InterruptMaskRegisterOffset);
-}
-
-static inline
-void DAC960_LA_DisableInterrupts(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All = 0xFF;
- InterruptMaskRegister.Bits.DisableInterrupts = true;
- writeb(InterruptMaskRegister.All,
- ControllerBaseAddress + DAC960_LA_InterruptMaskRegisterOffset);
-}
-
-static inline
-bool DAC960_LA_InterruptsEnabledP(void __iomem *ControllerBaseAddress)
-{
- DAC960_LA_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All =
- readb(ControllerBaseAddress + DAC960_LA_InterruptMaskRegisterOffset);
- return !InterruptMaskRegister.Bits.DisableInterrupts;
-}
-
-static inline
-void DAC960_LA_WriteCommandMailbox(DAC960_V1_CommandMailbox_T
- *MemoryCommandMailbox,
- DAC960_V1_CommandMailbox_T
- *CommandMailbox)
-{
- MemoryCommandMailbox->Words[1] = CommandMailbox->Words[1];
- MemoryCommandMailbox->Words[2] = CommandMailbox->Words[2];
- MemoryCommandMailbox->Words[3] = CommandMailbox->Words[3];
- wmb();
- MemoryCommandMailbox->Words[0] = CommandMailbox->Words[0];
- mb();
-}
-
-static inline
-void DAC960_LA_WriteHardwareMailbox(void __iomem *ControllerBaseAddress,
- DAC960_V1_CommandMailbox_T *CommandMailbox)
-{
- writel(CommandMailbox->Words[0],
- ControllerBaseAddress + DAC960_LA_CommandOpcodeRegisterOffset);
- writel(CommandMailbox->Words[1],
- ControllerBaseAddress + DAC960_LA_MailboxRegister4Offset);
- writel(CommandMailbox->Words[2],
- ControllerBaseAddress + DAC960_LA_MailboxRegister8Offset);
- writeb(CommandMailbox->Bytes[12],
- ControllerBaseAddress + DAC960_LA_MailboxRegister12Offset);
-}
-
-static inline DAC960_V1_CommandIdentifier_T
-DAC960_LA_ReadStatusCommandIdentifier(void __iomem *ControllerBaseAddress)
-{
- return readb(ControllerBaseAddress
- + DAC960_LA_StatusCommandIdentifierRegOffset);
-}
-
-static inline DAC960_V1_CommandStatus_T
-DAC960_LA_ReadStatusRegister(void __iomem *ControllerBaseAddress)
-{
- return readw(ControllerBaseAddress + DAC960_LA_StatusRegisterOffset);
-}
-
-static inline bool
-DAC960_LA_ReadErrorStatus(void __iomem *ControllerBaseAddress,
- unsigned char *ErrorStatus,
- unsigned char *Parameter0,
- unsigned char *Parameter1)
-{
- DAC960_LA_ErrorStatusRegister_T ErrorStatusRegister;
- ErrorStatusRegister.All =
- readb(ControllerBaseAddress + DAC960_LA_ErrorStatusRegisterOffset);
- if (!ErrorStatusRegister.Bits.ErrorStatusPending) return false;
- ErrorStatusRegister.Bits.ErrorStatusPending = false;
- *ErrorStatus = ErrorStatusRegister.All;
- *Parameter0 =
- readb(ControllerBaseAddress + DAC960_LA_CommandOpcodeRegisterOffset);
- *Parameter1 =
- readb(ControllerBaseAddress + DAC960_LA_CommandIdentifierRegisterOffset);
- writeb(0xFF, ControllerBaseAddress + DAC960_LA_ErrorStatusRegisterOffset);
- return true;
-}
-
-/*
- Define the DAC960 PG Series Controller Interface Register Offsets.
-*/
-
-#define DAC960_PG_RegisterWindowSize 0x2000
-
-typedef enum
-{
- DAC960_PG_InboundDoorBellRegisterOffset = 0x0020,
- DAC960_PG_OutboundDoorBellRegisterOffset = 0x002C,
- DAC960_PG_InterruptMaskRegisterOffset = 0x0034,
- DAC960_PG_CommandOpcodeRegisterOffset = 0x1000,
- DAC960_PG_CommandIdentifierRegisterOffset = 0x1001,
- DAC960_PG_MailboxRegister2Offset = 0x1002,
- DAC960_PG_MailboxRegister3Offset = 0x1003,
- DAC960_PG_MailboxRegister4Offset = 0x1004,
- DAC960_PG_MailboxRegister5Offset = 0x1005,
- DAC960_PG_MailboxRegister6Offset = 0x1006,
- DAC960_PG_MailboxRegister7Offset = 0x1007,
- DAC960_PG_MailboxRegister8Offset = 0x1008,
- DAC960_PG_MailboxRegister9Offset = 0x1009,
- DAC960_PG_MailboxRegister10Offset = 0x100A,
- DAC960_PG_MailboxRegister11Offset = 0x100B,
- DAC960_PG_MailboxRegister12Offset = 0x100C,
- DAC960_PG_StatusCommandIdentifierRegOffset = 0x1018,
- DAC960_PG_StatusRegisterOffset = 0x101A,
- DAC960_PG_ErrorStatusRegisterOffset = 0x103F
-}
-DAC960_PG_RegisterOffsets_T;
-
-
-/*
- Define the structure of the DAC960 PG Series Inbound Door Bell Register.
-*/
-
-typedef union DAC960_PG_InboundDoorBellRegister
-{
- unsigned int All;
- struct {
- bool HardwareMailboxNewCommand:1; /* Bit 0 */
- bool AcknowledgeHardwareMailboxStatus:1; /* Bit 1 */
- bool GenerateInterrupt:1; /* Bit 2 */
- bool ControllerReset:1; /* Bit 3 */
- bool MemoryMailboxNewCommand:1; /* Bit 4 */
- unsigned int :27; /* Bits 5-31 */
- } Write;
- struct {
- bool HardwareMailboxFull:1; /* Bit 0 */
- bool InitializationInProgress:1; /* Bit 1 */
- unsigned int :30; /* Bits 2-31 */
- } Read;
-}
-DAC960_PG_InboundDoorBellRegister_T;
-
-
-/*
- Define the structure of the DAC960 PG Series Outbound Door Bell Register.
-*/
-
-typedef union DAC960_PG_OutboundDoorBellRegister
-{
- unsigned int All;
- struct {
- bool AcknowledgeHardwareMailboxInterrupt:1; /* Bit 0 */
- bool AcknowledgeMemoryMailboxInterrupt:1; /* Bit 1 */
- unsigned int :30; /* Bits 2-31 */
- } Write;
- struct {
- bool HardwareMailboxStatusAvailable:1; /* Bit 0 */
- bool MemoryMailboxStatusAvailable:1; /* Bit 1 */
- unsigned int :30; /* Bits 2-31 */
- } Read;
-}
-DAC960_PG_OutboundDoorBellRegister_T;
-
-
-/*
- Define the structure of the DAC960 PG Series Interrupt Mask Register.
-*/
-
-typedef union DAC960_PG_InterruptMaskRegister
-{
- unsigned int All;
- struct {
- unsigned int MessageUnitInterruptMask1:2; /* Bits 0-1 */
- bool DisableInterrupts:1; /* Bit 2 */
- unsigned int MessageUnitInterruptMask2:5; /* Bits 3-7 */
- unsigned int Reserved0:24; /* Bits 8-31 */
- } Bits;
-}
-DAC960_PG_InterruptMaskRegister_T;
-
-
-/*
- Define the structure of the DAC960 PG Series Error Status Register.
-*/
-
-typedef union DAC960_PG_ErrorStatusRegister
-{
- unsigned char All;
- struct {
- unsigned int :2; /* Bits 0-1 */
- bool ErrorStatusPending:1; /* Bit 2 */
- unsigned int :5; /* Bits 3-7 */
- } Bits;
-}
-DAC960_PG_ErrorStatusRegister_T;
-
-
-/*
- Define inline functions to provide an abstraction for reading and writing the
- DAC960 PG Series Controller Interface Registers.
-*/
-
-static inline
-void DAC960_PG_HardwareMailboxNewCommand(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.HardwareMailboxNewCommand = true;
- writel(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PG_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_PG_AcknowledgeHardwareMailboxStatus(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.AcknowledgeHardwareMailboxStatus = true;
- writel(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PG_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_PG_GenerateInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.GenerateInterrupt = true;
- writel(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PG_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_PG_ControllerReset(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.ControllerReset = true;
- writel(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PG_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_PG_MemoryMailboxNewCommand(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.MemoryMailboxNewCommand = true;
- writel(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PG_InboundDoorBellRegisterOffset);
-}
-
-static inline
-bool DAC960_PG_HardwareMailboxFullP(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All =
- readl(ControllerBaseAddress + DAC960_PG_InboundDoorBellRegisterOffset);
- return InboundDoorBellRegister.Read.HardwareMailboxFull;
-}
-
-static inline
-bool DAC960_PG_InitializationInProgressP(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All =
- readl(ControllerBaseAddress + DAC960_PG_InboundDoorBellRegisterOffset);
- return InboundDoorBellRegister.Read.InitializationInProgress;
-}
-
-static inline
-void DAC960_PG_AcknowledgeHardwareMailboxInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeHardwareMailboxInterrupt = true;
- writel(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PG_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_PG_AcknowledgeMemoryMailboxInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeMemoryMailboxInterrupt = true;
- writel(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PG_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_PG_AcknowledgeInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeHardwareMailboxInterrupt = true;
- OutboundDoorBellRegister.Write.AcknowledgeMemoryMailboxInterrupt = true;
- writel(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PG_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-bool DAC960_PG_HardwareMailboxStatusAvailableP(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All =
- readl(ControllerBaseAddress + DAC960_PG_OutboundDoorBellRegisterOffset);
- return OutboundDoorBellRegister.Read.HardwareMailboxStatusAvailable;
-}
-
-static inline
-bool DAC960_PG_MemoryMailboxStatusAvailableP(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All =
- readl(ControllerBaseAddress + DAC960_PG_OutboundDoorBellRegisterOffset);
- return OutboundDoorBellRegister.Read.MemoryMailboxStatusAvailable;
-}
-
-static inline
-void DAC960_PG_EnableInterrupts(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All = 0;
- InterruptMaskRegister.Bits.MessageUnitInterruptMask1 = 0x3;
- InterruptMaskRegister.Bits.DisableInterrupts = false;
- InterruptMaskRegister.Bits.MessageUnitInterruptMask2 = 0x1F;
- writel(InterruptMaskRegister.All,
- ControllerBaseAddress + DAC960_PG_InterruptMaskRegisterOffset);
-}
-
-static inline
-void DAC960_PG_DisableInterrupts(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All = 0;
- InterruptMaskRegister.Bits.MessageUnitInterruptMask1 = 0x3;
- InterruptMaskRegister.Bits.DisableInterrupts = true;
- InterruptMaskRegister.Bits.MessageUnitInterruptMask2 = 0x1F;
- writel(InterruptMaskRegister.All,
- ControllerBaseAddress + DAC960_PG_InterruptMaskRegisterOffset);
-}
-
-static inline
-bool DAC960_PG_InterruptsEnabledP(void __iomem *ControllerBaseAddress)
-{
- DAC960_PG_InterruptMaskRegister_T InterruptMaskRegister;
- InterruptMaskRegister.All =
- readl(ControllerBaseAddress + DAC960_PG_InterruptMaskRegisterOffset);
- return !InterruptMaskRegister.Bits.DisableInterrupts;
-}
-
-static inline
-void DAC960_PG_WriteCommandMailbox(DAC960_V1_CommandMailbox_T
- *MemoryCommandMailbox,
- DAC960_V1_CommandMailbox_T
- *CommandMailbox)
-{
- MemoryCommandMailbox->Words[1] = CommandMailbox->Words[1];
- MemoryCommandMailbox->Words[2] = CommandMailbox->Words[2];
- MemoryCommandMailbox->Words[3] = CommandMailbox->Words[3];
- wmb();
- MemoryCommandMailbox->Words[0] = CommandMailbox->Words[0];
- mb();
-}
-
-static inline
-void DAC960_PG_WriteHardwareMailbox(void __iomem *ControllerBaseAddress,
- DAC960_V1_CommandMailbox_T *CommandMailbox)
-{
- writel(CommandMailbox->Words[0],
- ControllerBaseAddress + DAC960_PG_CommandOpcodeRegisterOffset);
- writel(CommandMailbox->Words[1],
- ControllerBaseAddress + DAC960_PG_MailboxRegister4Offset);
- writel(CommandMailbox->Words[2],
- ControllerBaseAddress + DAC960_PG_MailboxRegister8Offset);
- writeb(CommandMailbox->Bytes[12],
- ControllerBaseAddress + DAC960_PG_MailboxRegister12Offset);
-}
-
-static inline DAC960_V1_CommandIdentifier_T
-DAC960_PG_ReadStatusCommandIdentifier(void __iomem *ControllerBaseAddress)
-{
- return readb(ControllerBaseAddress
- + DAC960_PG_StatusCommandIdentifierRegOffset);
-}
-
-static inline DAC960_V1_CommandStatus_T
-DAC960_PG_ReadStatusRegister(void __iomem *ControllerBaseAddress)
-{
- return readw(ControllerBaseAddress + DAC960_PG_StatusRegisterOffset);
-}
-
-static inline bool
-DAC960_PG_ReadErrorStatus(void __iomem *ControllerBaseAddress,
- unsigned char *ErrorStatus,
- unsigned char *Parameter0,
- unsigned char *Parameter1)
-{
- DAC960_PG_ErrorStatusRegister_T ErrorStatusRegister;
- ErrorStatusRegister.All =
- readb(ControllerBaseAddress + DAC960_PG_ErrorStatusRegisterOffset);
- if (!ErrorStatusRegister.Bits.ErrorStatusPending) return false;
- ErrorStatusRegister.Bits.ErrorStatusPending = false;
- *ErrorStatus = ErrorStatusRegister.All;
- *Parameter0 =
- readb(ControllerBaseAddress + DAC960_PG_CommandOpcodeRegisterOffset);
- *Parameter1 =
- readb(ControllerBaseAddress + DAC960_PG_CommandIdentifierRegisterOffset);
- writeb(0, ControllerBaseAddress + DAC960_PG_ErrorStatusRegisterOffset);
- return true;
-}
-
-/*
- Define the DAC960 PD Series Controller Interface Register Offsets.
-*/
-
-#define DAC960_PD_RegisterWindowSize 0x80
-
-typedef enum
-{
- DAC960_PD_CommandOpcodeRegisterOffset = 0x00,
- DAC960_PD_CommandIdentifierRegisterOffset = 0x01,
- DAC960_PD_MailboxRegister2Offset = 0x02,
- DAC960_PD_MailboxRegister3Offset = 0x03,
- DAC960_PD_MailboxRegister4Offset = 0x04,
- DAC960_PD_MailboxRegister5Offset = 0x05,
- DAC960_PD_MailboxRegister6Offset = 0x06,
- DAC960_PD_MailboxRegister7Offset = 0x07,
- DAC960_PD_MailboxRegister8Offset = 0x08,
- DAC960_PD_MailboxRegister9Offset = 0x09,
- DAC960_PD_MailboxRegister10Offset = 0x0A,
- DAC960_PD_MailboxRegister11Offset = 0x0B,
- DAC960_PD_MailboxRegister12Offset = 0x0C,
- DAC960_PD_StatusCommandIdentifierRegOffset = 0x0D,
- DAC960_PD_StatusRegisterOffset = 0x0E,
- DAC960_PD_ErrorStatusRegisterOffset = 0x3F,
- DAC960_PD_InboundDoorBellRegisterOffset = 0x40,
- DAC960_PD_OutboundDoorBellRegisterOffset = 0x41,
- DAC960_PD_InterruptEnableRegisterOffset = 0x43
-}
-DAC960_PD_RegisterOffsets_T;
-
-
-/*
- Define the structure of the DAC960 PD Series Inbound Door Bell Register.
-*/
-
-typedef union DAC960_PD_InboundDoorBellRegister
-{
- unsigned char All;
- struct {
- bool NewCommand:1; /* Bit 0 */
- bool AcknowledgeStatus:1; /* Bit 1 */
- bool GenerateInterrupt:1; /* Bit 2 */
- bool ControllerReset:1; /* Bit 3 */
- unsigned char :4; /* Bits 4-7 */
- } Write;
- struct {
- bool MailboxFull:1; /* Bit 0 */
- bool InitializationInProgress:1; /* Bit 1 */
- unsigned char :6; /* Bits 2-7 */
- } Read;
-}
-DAC960_PD_InboundDoorBellRegister_T;
-
-
-/*
- Define the structure of the DAC960 PD Series Outbound Door Bell Register.
-*/
-
-typedef union DAC960_PD_OutboundDoorBellRegister
-{
- unsigned char All;
- struct {
- bool AcknowledgeInterrupt:1; /* Bit 0 */
- unsigned char :7; /* Bits 1-7 */
- } Write;
- struct {
- bool StatusAvailable:1; /* Bit 0 */
- unsigned char :7; /* Bits 1-7 */
- } Read;
-}
-DAC960_PD_OutboundDoorBellRegister_T;
-
-
-/*
- Define the structure of the DAC960 PD Series Interrupt Enable Register.
-*/
-
-typedef union DAC960_PD_InterruptEnableRegister
-{
- unsigned char All;
- struct {
- bool EnableInterrupts:1; /* Bit 0 */
- unsigned char :7; /* Bits 1-7 */
- } Bits;
-}
-DAC960_PD_InterruptEnableRegister_T;
-
-
-/*
- Define the structure of the DAC960 PD Series Error Status Register.
-*/
-
-typedef union DAC960_PD_ErrorStatusRegister
-{
- unsigned char All;
- struct {
- unsigned int :2; /* Bits 0-1 */
- bool ErrorStatusPending:1; /* Bit 2 */
- unsigned int :5; /* Bits 3-7 */
- } Bits;
-}
-DAC960_PD_ErrorStatusRegister_T;
-
-
-/*
- Define inline functions to provide an abstraction for reading and writing the
- DAC960 PD Series Controller Interface Registers.
-*/
-
-static inline
-void DAC960_PD_NewCommand(void __iomem *ControllerBaseAddress)
-{
- DAC960_PD_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.NewCommand = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PD_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_PD_AcknowledgeStatus(void __iomem *ControllerBaseAddress)
-{
- DAC960_PD_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.AcknowledgeStatus = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PD_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_PD_GenerateInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_PD_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.GenerateInterrupt = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PD_InboundDoorBellRegisterOffset);
-}
-
-static inline
-void DAC960_PD_ControllerReset(void __iomem *ControllerBaseAddress)
-{
- DAC960_PD_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All = 0;
- InboundDoorBellRegister.Write.ControllerReset = true;
- writeb(InboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PD_InboundDoorBellRegisterOffset);
-}
-
-static inline
-bool DAC960_PD_MailboxFullP(void __iomem *ControllerBaseAddress)
-{
- DAC960_PD_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_PD_InboundDoorBellRegisterOffset);
- return InboundDoorBellRegister.Read.MailboxFull;
-}
-
-static inline
-bool DAC960_PD_InitializationInProgressP(void __iomem *ControllerBaseAddress)
-{
- DAC960_PD_InboundDoorBellRegister_T InboundDoorBellRegister;
- InboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_PD_InboundDoorBellRegisterOffset);
- return InboundDoorBellRegister.Read.InitializationInProgress;
-}
-
-static inline
-void DAC960_PD_AcknowledgeInterrupt(void __iomem *ControllerBaseAddress)
-{
- DAC960_PD_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All = 0;
- OutboundDoorBellRegister.Write.AcknowledgeInterrupt = true;
- writeb(OutboundDoorBellRegister.All,
- ControllerBaseAddress + DAC960_PD_OutboundDoorBellRegisterOffset);
-}
-
-static inline
-bool DAC960_PD_StatusAvailableP(void __iomem *ControllerBaseAddress)
-{
- DAC960_PD_OutboundDoorBellRegister_T OutboundDoorBellRegister;
- OutboundDoorBellRegister.All =
- readb(ControllerBaseAddress + DAC960_PD_OutboundDoorBellRegisterOffset);
- return OutboundDoorBellRegister.Read.StatusAvailable;
-}
-
-static inline
-void DAC960_PD_EnableInterrupts(void __iomem *ControllerBaseAddress)
-{
- DAC960_PD_InterruptEnableRegister_T InterruptEnableRegister;
- InterruptEnableRegister.All = 0;
- InterruptEnableRegister.Bits.EnableInterrupts = true;
- writeb(InterruptEnableRegister.All,
- ControllerBaseAddress + DAC960_PD_InterruptEnableRegisterOffset);
-}
-
-static inline
-void DAC960_PD_DisableInterrupts(void __iomem *ControllerBaseAddress)
-{
- DAC960_PD_InterruptEnableRegister_T InterruptEnableRegister;
- InterruptEnableRegister.All = 0;
- InterruptEnableRegister.Bits.EnableInterrupts = false;
- writeb(InterruptEnableRegister.All,
- ControllerBaseAddress + DAC960_PD_InterruptEnableRegisterOffset);
-}
-
-static inline
-bool DAC960_PD_InterruptsEnabledP(void __iomem *ControllerBaseAddress)
-{
- DAC960_PD_InterruptEnableRegister_T InterruptEnableRegister;
- InterruptEnableRegister.All =
- readb(ControllerBaseAddress + DAC960_PD_InterruptEnableRegisterOffset);
- return InterruptEnableRegister.Bits.EnableInterrupts;
-}
-
-static inline
-void DAC960_PD_WriteCommandMailbox(void __iomem *ControllerBaseAddress,
- DAC960_V1_CommandMailbox_T *CommandMailbox)
-{
- writel(CommandMailbox->Words[0],
- ControllerBaseAddress + DAC960_PD_CommandOpcodeRegisterOffset);
- writel(CommandMailbox->Words[1],
- ControllerBaseAddress + DAC960_PD_MailboxRegister4Offset);
- writel(CommandMailbox->Words[2],
- ControllerBaseAddress + DAC960_PD_MailboxRegister8Offset);
- writeb(CommandMailbox->Bytes[12],
- ControllerBaseAddress + DAC960_PD_MailboxRegister12Offset);
-}
-
-static inline DAC960_V1_CommandIdentifier_T
-DAC960_PD_ReadStatusCommandIdentifier(void __iomem *ControllerBaseAddress)
-{
- return readb(ControllerBaseAddress
- + DAC960_PD_StatusCommandIdentifierRegOffset);
-}
-
-static inline DAC960_V1_CommandStatus_T
-DAC960_PD_ReadStatusRegister(void __iomem *ControllerBaseAddress)
-{
- return readw(ControllerBaseAddress + DAC960_PD_StatusRegisterOffset);
-}
-
-static inline bool
-DAC960_PD_ReadErrorStatus(void __iomem *ControllerBaseAddress,
- unsigned char *ErrorStatus,
- unsigned char *Parameter0,
- unsigned char *Parameter1)
-{
- DAC960_PD_ErrorStatusRegister_T ErrorStatusRegister;
- ErrorStatusRegister.All =
- readb(ControllerBaseAddress + DAC960_PD_ErrorStatusRegisterOffset);
- if (!ErrorStatusRegister.Bits.ErrorStatusPending) return false;
- ErrorStatusRegister.Bits.ErrorStatusPending = false;
- *ErrorStatus = ErrorStatusRegister.All;
- *Parameter0 =
- readb(ControllerBaseAddress + DAC960_PD_CommandOpcodeRegisterOffset);
- *Parameter1 =
- readb(ControllerBaseAddress + DAC960_PD_CommandIdentifierRegisterOffset);
- writeb(0, ControllerBaseAddress + DAC960_PD_ErrorStatusRegisterOffset);
- return true;
-}
-
-static inline void DAC960_P_To_PD_TranslateEnquiry(void *Enquiry)
-{
- memcpy(Enquiry + 132, Enquiry + 36, 64);
- memset(Enquiry + 36, 0, 96);
-}
-
-static inline void DAC960_P_To_PD_TranslateDeviceState(void *DeviceState)
-{
- memcpy(DeviceState + 2, DeviceState + 3, 1);
- memmove(DeviceState + 4, DeviceState + 5, 2);
- memmove(DeviceState + 6, DeviceState + 8, 4);
-}
-
-static inline
-void DAC960_PD_To_P_TranslateReadWriteCommand(DAC960_V1_CommandMailbox_T
- *CommandMailbox)
-{
- int LogicalDriveNumber = CommandMailbox->Type5.LD.LogicalDriveNumber;
- CommandMailbox->Bytes[3] &= 0x7;
- CommandMailbox->Bytes[3] |= CommandMailbox->Bytes[7] << 6;
- CommandMailbox->Bytes[7] = LogicalDriveNumber;
-}
-
-static inline
-void DAC960_P_To_PD_TranslateReadWriteCommand(DAC960_V1_CommandMailbox_T
- *CommandMailbox)
-{
- int LogicalDriveNumber = CommandMailbox->Bytes[7];
- CommandMailbox->Bytes[7] = CommandMailbox->Bytes[3] >> 6;
- CommandMailbox->Bytes[3] &= 0x7;
- CommandMailbox->Bytes[3] |= LogicalDriveNumber << 3;
-}
-
-
-/*
- Define prototypes for the forward referenced DAC960 Driver Internal Functions.
-*/
-
-static void DAC960_FinalizeController(DAC960_Controller_T *);
-static void DAC960_V1_QueueReadWriteCommand(DAC960_Command_T *);
-static void DAC960_V2_QueueReadWriteCommand(DAC960_Command_T *);
-static void DAC960_RequestFunction(struct request_queue *);
-static irqreturn_t DAC960_BA_InterruptHandler(int, void *);
-static irqreturn_t DAC960_LP_InterruptHandler(int, void *);
-static irqreturn_t DAC960_LA_InterruptHandler(int, void *);
-static irqreturn_t DAC960_PG_InterruptHandler(int, void *);
-static irqreturn_t DAC960_PD_InterruptHandler(int, void *);
-static irqreturn_t DAC960_P_InterruptHandler(int, void *);
-static void DAC960_V1_QueueMonitoringCommand(DAC960_Command_T *);
-static void DAC960_V2_QueueMonitoringCommand(DAC960_Command_T *);
-static void DAC960_MonitoringTimerFunction(struct timer_list *);
-static void DAC960_Message(DAC960_MessageLevel_T, unsigned char *,
- DAC960_Controller_T *, ...);
-static void DAC960_CreateProcEntries(DAC960_Controller_T *);
-static void DAC960_DestroyProcEntries(DAC960_Controller_T *);
-
-#endif /* DAC960_DriverVersion */
source "drivers/block/zram/Kconfig"
-config BLK_DEV_DAC960
- tristate "Mylex DAC960/DAC1100 PCI RAID Controller support"
- depends on PCI
- help
- This driver adds support for the Mylex DAC960, AcceleRAID, and
- eXtremeRAID PCI RAID controllers. See the file
- <file:Documentation/blockdev/README.DAC960> for further information
- about this driver.
-
- To compile this driver as a module, choose M here: the
- module will be called DAC960.
-
config BLK_DEV_UMEM
tristate "Micro Memory MM5415 Battery Backed RAM support"
depends on PCI
select LIBCRC32C
select CRYPTO_AES
select CRYPTO
- default n
help
Say Y here if you want include the Rados block device, which stripes
a block device over objects stored in the Ceph distributed object
obj-$(CONFIG_AMIGA_Z2RAM) += z2ram.o
obj-$(CONFIG_BLK_DEV_RAM) += brd.o
obj-$(CONFIG_BLK_DEV_LOOP) += loop.o
-obj-$(CONFIG_BLK_DEV_DAC960) += DAC960.o
obj-$(CONFIG_XILINX_SYSACE) += xsysace.o
obj-$(CONFIG_CDROM_PKTCDVD) += pktcdvd.o
obj-$(CONFIG_SUNVDC) += sunvdc.o
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/mutex.h>
-#include <linux/amifdreg.h>
-#include <linux/amifd.h>
#include <linux/fs.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/elevator.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
* Defines
*/
+/*
+ * CIAAPRA bits (read only)
+ */
+
+#define DSKRDY (0x1<<5) /* disk ready when low */
+#define DSKTRACK0 (0x1<<4) /* head at track zero when low */
+#define DSKPROT (0x1<<3) /* disk protected when low */
+#define DSKCHANGE (0x1<<2) /* low when disk removed */
+
+/*
+ * CIAAPRB bits (read/write)
+ */
+
+#define DSKMOTOR (0x1<<7) /* motor on when low */
+#define DSKSEL3 (0x1<<6) /* select drive 3 when low */
+#define DSKSEL2 (0x1<<5) /* select drive 2 when low */
+#define DSKSEL1 (0x1<<4) /* select drive 1 when low */
+#define DSKSEL0 (0x1<<3) /* select drive 0 when low */
+#define DSKSIDE (0x1<<2) /* side selection: 0 = upper, 1 = lower */
+#define DSKDIREC (0x1<<1) /* step direction: 0=in, 1=out (to trk 0) */
+#define DSKSTEP (0x1) /* pulse low to step head 1 track */
+
+/*
+ * DSKBYTR bits (read only)
+ */
+
+#define DSKBYT (1<<15) /* register contains valid byte when set */
+#define DMAON (1<<14) /* disk DMA enabled */
+#define DISKWRITE (1<<13) /* disk write bit in DSKLEN enabled */
+#define WORDEQUAL (1<<12) /* DSKSYNC register match when true */
+/* bits 7-0 are data */
+
+/*
+ * ADKCON/ADKCONR bits
+ */
+
+#ifndef SETCLR
+#define ADK_SETCLR (1<<15) /* control bit */
+#endif
+#define ADK_PRECOMP1 (1<<14) /* precompensation selection */
+#define ADK_PRECOMP0 (1<<13) /* 00=none, 01=140ns, 10=280ns, 11=500ns */
+#define ADK_MFMPREC (1<<12) /* 0=GCR precomp., 1=MFM precomp. */
+#define ADK_WORDSYNC (1<<10) /* enable DSKSYNC auto DMA */
+#define ADK_MSBSYNC (1<<9) /* when 1, enable sync on MSbit (for GCR) */
+#define ADK_FAST (1<<8) /* bit cell: 0=2us (GCR), 1=1us (MFM) */
+
+/*
+ * DSKLEN bits
+ */
+
+#define DSKLEN_DMAEN (1<<15)
+#define DSKLEN_WRITE (1<<14)
+
+/*
+ * INTENA/INTREQ bits
+ */
+
+#define DSKINDEX (0x1<<4) /* DSKINDEX bit */
+
+/*
+ * Misc
+ */
+
+#define MFM_SYNC 0x4489 /* standard MFM sync value */
+
+/* Values for FD_COMMAND */
+#define FD_RECALIBRATE 0x07 /* move to track 0 */
+#define FD_SEEK 0x0F /* seek track */
+#define FD_READ 0xE6 /* read with MT, MFM, SKip deleted */
+#define FD_WRITE 0xC5 /* write with MT, MFM */
+#define FD_SENSEI 0x08 /* Sense Interrupt Status */
+#define FD_SPECIFY 0x03 /* specify HUT etc */
+#define FD_FORMAT 0x4D /* format one track */
+#define FD_VERSION 0x10 /* get version code */
+#define FD_CONFIGURE 0x13 /* configure FIFO operation */
+#define FD_PERPENDICULAR 0x12 /* perpendicular r/w mode */
+
+#define FD_MAX_UNITS 4 /* Max. Number of drives */
+#define FLOPPY_MAX_SECTORS 22 /* Max. Number of sectors per track */
+
+struct fd_data_type {
+ char *name; /* description of data type */
+ int sects; /* sectors per track */
+ int (*read_fkt)(int); /* read whole track */
+ void (*write_fkt)(int); /* write whole track */
+};
+
+struct fd_drive_type {
+ unsigned long code; /* code returned from drive */
+ char *name; /* description of drive */
+ unsigned int tracks; /* number of tracks */
+ unsigned int heads; /* number of heads */
+ unsigned int read_size; /* raw read size for one track */
+ unsigned int write_size; /* raw write size for one track */
+ unsigned int sect_mult; /* sectors and gap multiplier (HD = 2) */
+ unsigned int precomp1; /* start track for precomp 1 */
+ unsigned int precomp2; /* start track for precomp 2 */
+ unsigned int step_delay; /* time (in ms) for delay after step */
+ unsigned int settle_time; /* time to settle after dir change */
+ unsigned int side_time; /* time needed to change sides */
+};
+
+struct amiga_floppy_struct {
+ struct fd_drive_type *type; /* type of floppy for this unit */
+ struct fd_data_type *dtype; /* type of floppy for this unit */
+ int track; /* current track (-1 == unknown) */
+ unsigned char *trackbuf; /* current track (kmaloc()'d */
+
+ int blocks; /* total # blocks on disk */
+
+ int changed; /* true when not known */
+ int disk; /* disk in drive (-1 == unknown) */
+ int motor; /* true when motor is at speed */
+ int busy; /* true when drive is active */
+ int dirty; /* true when trackbuf is not on disk */
+ int status; /* current error code for unit */
+ struct gendisk *gendisk;
+ struct blk_mq_tag_set tag_set;
+};
+
/*
* Error codes
*/
static int writepending;
static int writefromint;
static char *raw_buf;
-static int fdc_queue;
static DEFINE_SPINLOCK(amiflop_lock);
return -1;
}
-/*
- * Round-robin between our available drives, doing one request from each
- */
-static struct request *set_next_request(void)
-{
- struct request_queue *q;
- int cnt = FD_MAX_UNITS;
- struct request *rq = NULL;
-
- /* Find next queue we can dispatch from */
- fdc_queue = fdc_queue + 1;
- if (fdc_queue == FD_MAX_UNITS)
- fdc_queue = 0;
-
- for(cnt = FD_MAX_UNITS; cnt > 0; cnt--) {
-
- if (unit[fdc_queue].type->code == FD_NODRIVE) {
- if (++fdc_queue == FD_MAX_UNITS)
- fdc_queue = 0;
- continue;
- }
-
- q = unit[fdc_queue].gendisk->queue;
- if (q) {
- rq = blk_fetch_request(q);
- if (rq)
- break;
- }
-
- if (++fdc_queue == FD_MAX_UNITS)
- fdc_queue = 0;
- }
-
- return rq;
-}
-
-static void redo_fd_request(void)
+static blk_status_t amiflop_rw_cur_segment(struct amiga_floppy_struct *floppy,
+ struct request *rq)
{
- struct request *rq;
+ int drive = floppy - unit;
unsigned int cnt, block, track, sector;
- int drive;
- struct amiga_floppy_struct *floppy;
char *data;
- unsigned long flags;
- blk_status_t err;
-
-next_req:
- rq = set_next_request();
- if (!rq) {
- /* Nothing left to do */
- return;
- }
-
- floppy = rq->rq_disk->private_data;
- drive = floppy - unit;
-next_segment:
- /* Here someone could investigate to be more efficient */
- for (cnt = 0, err = BLK_STS_OK; cnt < blk_rq_cur_sectors(rq); cnt++) {
+ for (cnt = 0; cnt < blk_rq_cur_sectors(rq); cnt++) {
#ifdef DEBUG
printk("fd: sector %ld + %d requested for %s\n",
blk_rq_pos(rq), cnt,
(rq_data_dir(rq) == READ) ? "read" : "write");
#endif
block = blk_rq_pos(rq) + cnt;
- if ((int)block > floppy->blocks) {
- err = BLK_STS_IOERR;
- break;
- }
-
track = block / (floppy->dtype->sects * floppy->type->sect_mult);
sector = block % (floppy->dtype->sects * floppy->type->sect_mult);
data = bio_data(rq->bio) + 512 * cnt;
"0x%08lx\n", track, sector, data);
#endif
- if (get_track(drive, track) == -1) {
- err = BLK_STS_IOERR;
- break;
- }
+ if (get_track(drive, track) == -1)
+ return BLK_STS_IOERR;
if (rq_data_dir(rq) == READ) {
memcpy(data, floppy->trackbuf + sector * 512, 512);
memcpy(floppy->trackbuf + sector * 512, data, 512);
/* keep the drive spinning while writes are scheduled */
- if (!fd_motor_on(drive)) {
- err = BLK_STS_IOERR;
- break;
- }
+ if (!fd_motor_on(drive))
+ return BLK_STS_IOERR;
/*
* setup a callback to write the track buffer
* after a short (1 tick) delay.
*/
- local_irq_save(flags);
-
floppy->dirty = 1;
/* reset the timer */
mod_timer (flush_track_timer + drive, jiffies + 1);
- local_irq_restore(flags);
}
}
- if (__blk_end_request_cur(rq, err))
- goto next_segment;
- goto next_req;
+ return BLK_STS_OK;
}
-static void do_fd_request(struct request_queue * q)
+static blk_status_t amiflop_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
- redo_fd_request();
+ struct request *rq = bd->rq;
+ struct amiga_floppy_struct *floppy = rq->rq_disk->private_data;
+ blk_status_t err;
+
+ if (!spin_trylock_irq(&amiflop_lock))
+ return BLK_STS_DEV_RESOURCE;
+
+ blk_mq_start_request(rq);
+
+ do {
+ err = amiflop_rw_cur_segment(floppy, rq);
+ } while (blk_update_request(rq, err, blk_rq_cur_bytes(rq)));
+ blk_mq_end_request(rq, err);
+
+ spin_unlock_irq(&amiflop_lock);
+ return BLK_STS_OK;
}
static int fd_getgeo(struct block_device *bdev, struct hd_geometry *geo)
.check_events = amiga_check_events,
};
+static const struct blk_mq_ops amiflop_mq_ops = {
+ .queue_rq = amiflop_queue_rq,
+};
+
+static struct gendisk *fd_alloc_disk(int drive)
+{
+ struct gendisk *disk;
+
+ disk = alloc_disk(1);
+ if (!disk)
+ goto out;
+
+ disk->queue = blk_mq_init_sq_queue(&unit[drive].tag_set, &amiflop_mq_ops,
+ 2, BLK_MQ_F_SHOULD_MERGE);
+ if (IS_ERR(disk->queue)) {
+ disk->queue = NULL;
+ goto out_put_disk;
+ }
+
+ unit[drive].trackbuf = kmalloc(FLOPPY_MAX_SECTORS * 512, GFP_KERNEL);
+ if (!unit[drive].trackbuf)
+ goto out_cleanup_queue;
+
+ return disk;
+
+out_cleanup_queue:
+ blk_cleanup_queue(disk->queue);
+ disk->queue = NULL;
+ blk_mq_free_tag_set(&unit[drive].tag_set);
+out_put_disk:
+ put_disk(disk);
+out:
+ unit[drive].type->code = FD_NODRIVE;
+ return NULL;
+}
+
static int __init fd_probe_drives(void)
{
int drive,drives,nomem;
- printk(KERN_INFO "FD: probing units\nfound ");
+ pr_info("FD: probing units\nfound");
drives=0;
nomem=0;
for(drive=0;drive<FD_MAX_UNITS;drive++) {
fd_probe(drive);
if (unit[drive].type->code == FD_NODRIVE)
continue;
- disk = alloc_disk(1);
+
+ disk = fd_alloc_disk(drive);
if (!disk) {
- unit[drive].type->code = FD_NODRIVE;
+ pr_cont(" no mem for fd%d", drive);
+ nomem = 1;
continue;
}
unit[drive].gendisk = disk;
-
- disk->queue = blk_init_queue(do_fd_request, &amiflop_lock);
- if (!disk->queue) {
- unit[drive].type->code = FD_NODRIVE;
- continue;
- }
-
drives++;
- if ((unit[drive].trackbuf = kmalloc(FLOPPY_MAX_SECTORS * 512, GFP_KERNEL)) == NULL) {
- printk("no mem for ");
- unit[drive].type = &drive_types[num_dr_types - 1]; /* FD_NODRIVE */
- drives--;
- nomem = 1;
- }
- printk("fd%d ",drive);
+
+ pr_cont(" fd%d",drive);
disk->major = FLOPPY_MAJOR;
disk->first_minor = drive;
disk->fops = &floppy_fops;
}
if ((drives > 0) || (nomem == 0)) {
if (drives == 0)
- printk("no drives");
- printk("\n");
+ pr_cont(" no drives");
+ pr_cont("\n");
return drives;
}
- printk("\n");
+ pr_cont("\n");
return -ENOMEM;
}
return ret;
}
-#if 0 /* not safe to unload */
-static int __exit amiga_floppy_remove(struct platform_device *pdev)
-{
- int i;
-
- for( i = 0; i < FD_MAX_UNITS; i++) {
- if (unit[i].type->code != FD_NODRIVE) {
- struct request_queue *q = unit[i].gendisk->queue;
- del_gendisk(unit[i].gendisk);
- put_disk(unit[i].gendisk);
- kfree(unit[i].trackbuf);
- if (q)
- blk_cleanup_queue(q);
- }
- }
- blk_unregister_region(MKDEV(FLOPPY_MAJOR, 0), 256);
- free_irq(IRQ_AMIGA_CIAA_TB, NULL);
- free_irq(IRQ_AMIGA_DSKBLK, NULL);
- custom.dmacon = DMAF_DISK; /* disable DMA */
- amiga_chip_free(raw_buf);
- unregister_blkdev(FLOPPY_MAJOR, "fd");
-}
-#endif
-
static struct platform_driver amiga_floppy_driver = {
.driver = {
.name = "amiga-floppy",
/* Copyright (c) 2013 Coraid, Inc. See COPYING for GPL terms. */
+#include <linux/blk-mq.h>
+
#define VERSION "85"
#define AOE_MAJOR 152
#define DEVICE_NAME "aoe"
struct gendisk *gd;
struct dentry *debugfs;
struct request_queue *blkq;
+ struct list_head rq_list;
+ struct blk_mq_tag_set tag_set;
struct hd_geometry geo;
sector_t ssize;
struct timer_list timer;
void aoeblk_exit(void);
void aoeblk_gdalloc(void *);
void aoedisk_rm_debugfs(struct aoedev *d);
-void aoedisk_rm_sysfs(struct aoedev *d);
int aoechr_init(void);
void aoechr_exit(void);
#include <linux/kernel.h>
#include <linux/hdreg.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/backing-dev.h>
#include <linux/fs.h>
#include <linux/ioctl.h>
NULL,
};
-static const struct attribute_group attr_group = {
+static const struct attribute_group aoe_attr_group = {
.attrs = aoe_attrs,
};
+static const struct attribute_group *aoe_attr_groups[] = {
+ &aoe_attr_group,
+ NULL,
+};
+
static const struct file_operations aoe_debugfs_fops = {
.open = aoe_debugfs_open,
.read = seq_read,
d->debugfs = NULL;
}
-static int
-aoedisk_add_sysfs(struct aoedev *d)
-{
- return sysfs_create_group(&disk_to_dev(d->gd)->kobj, &attr_group);
-}
-void
-aoedisk_rm_sysfs(struct aoedev *d)
-{
- sysfs_remove_group(&disk_to_dev(d->gd)->kobj, &attr_group);
-}
-
static int
aoeblk_open(struct block_device *bdev, fmode_t mode)
{
spin_unlock_irqrestore(&d->lock, flags);
}
-static void
-aoeblk_request(struct request_queue *q)
+static blk_status_t aoeblk_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
- struct aoedev *d;
- struct request *rq;
+ struct aoedev *d = hctx->queue->queuedata;
+
+ spin_lock_irq(&d->lock);
- d = q->queuedata;
if ((d->flags & DEVFL_UP) == 0) {
pr_info_ratelimited("aoe: device %ld.%d is not up\n",
d->aoemajor, d->aoeminor);
- while ((rq = blk_peek_request(q))) {
- blk_start_request(rq);
- aoe_end_request(d, rq, 1);
- }
- return;
+ spin_unlock_irq(&d->lock);
+ blk_mq_start_request(bd->rq);
+ return BLK_STS_IOERR;
}
+
+ list_add_tail(&bd->rq->queuelist, &d->rq_list);
aoecmd_work(d);
+ spin_unlock_irq(&d->lock);
+ return BLK_STS_OK;
}
static int
.owner = THIS_MODULE,
};
+static const struct blk_mq_ops aoeblk_mq_ops = {
+ .queue_rq = aoeblk_queue_rq,
+};
+
/* alloc_disk and add_disk can sleep */
void
aoeblk_gdalloc(void *vp)
struct gendisk *gd;
mempool_t *mp;
struct request_queue *q;
+ struct blk_mq_tag_set *set;
enum { KB = 1024, MB = KB * KB, READ_AHEAD = 2 * MB, };
ulong flags;
int late = 0;
+ int err;
spin_lock_irqsave(&d->lock, flags);
if (d->flags & DEVFL_GDALLOC
d->aoemajor, d->aoeminor);
goto err_disk;
}
- q = blk_init_queue(aoeblk_request, &d->lock);
- if (q == NULL) {
+
+ set = &d->tag_set;
+ set->ops = &aoeblk_mq_ops;
+ set->nr_hw_queues = 1;
+ set->queue_depth = 128;
+ set->numa_node = NUMA_NO_NODE;
+ set->flags = BLK_MQ_F_SHOULD_MERGE;
+ err = blk_mq_alloc_tag_set(set);
+ if (err) {
+ pr_err("aoe: cannot allocate tag set for %ld.%d\n",
+ d->aoemajor, d->aoeminor);
+ goto err_mempool;
+ }
+
+ q = blk_mq_init_queue(set);
+ if (IS_ERR(q)) {
pr_err("aoe: cannot allocate block queue for %ld.%d\n",
d->aoemajor, d->aoeminor);
+ blk_mq_free_tag_set(set);
goto err_mempool;
}
spin_unlock_irqrestore(&d->lock, flags);
- add_disk(gd);
- aoedisk_add_sysfs(d);
+ device_add_disk(NULL, gd, aoe_attr_groups);
aoedisk_add_debugfs(d);
spin_lock_irqsave(&d->lock, flags);
#include <linux/ata.h>
#include <linux/slab.h>
#include <linux/hdreg.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
#include <linux/genhd.h>
out:
if ((d->flags & DEVFL_KICKME) && d->blkq) {
d->flags &= ~DEVFL_KICKME;
- d->blkq->request_fn(d->blkq);
+ blk_mq_run_hw_queues(d->blkq, true);
}
d->timer.expires = jiffies + TIMERTICK;
return d->ip.buf;
rq = d->ip.rq;
if (rq == NULL) {
- rq = blk_peek_request(q);
+ rq = list_first_entry_or_null(&d->rq_list, struct request,
+ queuelist);
if (rq == NULL)
return NULL;
- blk_start_request(rq);
+ list_del_init(&rq->queuelist);
+ blk_mq_start_request(rq);
d->ip.rq = rq;
d->ip.nxbio = rq->bio;
rq->special = (void *) rqbiocnt(rq);
struct bio *bio;
int bok;
struct request_queue *q;
+ blk_status_t err = BLK_STS_OK;
q = d->blkq;
if (rq == d->ip.rq)
do {
bio = rq->bio;
bok = !fastfail && !bio->bi_status;
- } while (__blk_end_request(rq, bok ? BLK_STS_OK : BLK_STS_IOERR, bio->bi_iter.bi_size));
+ if (!bok)
+ err = BLK_STS_IOERR;
+ } while (blk_update_request(rq, bok ? BLK_STS_OK : BLK_STS_IOERR, bio->bi_iter.bi_size));
+
+ __blk_mq_end_request(rq, err);
/* cf. http://lkml.org/lkml/2006/10/31/28 */
if (!fastfail)
- __blk_run_queue(q);
+ blk_mq_run_hw_queues(q, true);
}
static void
*/
#include <linux/hdreg.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/netdevice.h>
#include <linux/delay.h>
#include <linux/slab.h>
{
struct aoetgt *t, **tt, **te;
struct list_head *head, *pos, *nx;
- struct request *rq;
int i;
d->flags &= ~DEVFL_UP;
/* fast fail all pending I/O */
if (d->blkq) {
- while ((rq = blk_peek_request(d->blkq))) {
- blk_start_request(rq);
- aoe_end_request(d, rq, 1);
- }
+ /* UP is cleared, freeze+quiesce to insure all are errored */
+ blk_mq_freeze_queue(d->blkq);
+ blk_mq_quiesce_queue(d->blkq);
+ blk_mq_unquiesce_queue(d->blkq);
+ blk_mq_unfreeze_queue(d->blkq);
}
if (d->gd)
del_timer_sync(&d->timer);
if (d->gd) {
aoedisk_rm_debugfs(d);
- aoedisk_rm_sysfs(d);
del_gendisk(d->gd);
put_disk(d->gd);
+ blk_mq_free_tag_set(&d->tag_set);
blk_cleanup_queue(d->blkq);
}
t = d->targets;
d->ntargets = NTARGETS;
INIT_WORK(&d->work, aoecmd_sleepwork);
spin_lock_init(&d->lock);
+ INIT_LIST_HEAD(&d->rq_list);
skb_queue_head_init(&d->skbpool);
timer_setup(&d->timer, dummy_timer, 0);
d->timer.expires = jiffies + HZ;
#include <linux/fd.h>
#include <linux/delay.h>
#include <linux/init.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/mutex.h>
#include <linux/completion.h>
#include <linux/wait.h>
-#include <asm/atafd.h>
-#include <asm/atafdreg.h>
#include <asm/atariints.h>
#include <asm/atari_stdma.h>
#include <asm/atari_stram.h>
static DEFINE_MUTEX(ataflop_mutex);
static struct request *fd_request;
-static int fdc_queue;
+
+/*
+ * WD1772 stuff
+ */
+
+/* register codes */
+
+#define FDCSELREG_STP (0x80) /* command/status register */
+#define FDCSELREG_TRA (0x82) /* track register */
+#define FDCSELREG_SEC (0x84) /* sector register */
+#define FDCSELREG_DTA (0x86) /* data register */
+
+/* register names for FDC_READ/WRITE macros */
+
+#define FDCREG_CMD 0
+#define FDCREG_STATUS 0
+#define FDCREG_TRACK 2
+#define FDCREG_SECTOR 4
+#define FDCREG_DATA 6
+
+/* command opcodes */
+
+#define FDCCMD_RESTORE (0x00) /* - */
+#define FDCCMD_SEEK (0x10) /* | */
+#define FDCCMD_STEP (0x20) /* | TYP 1 Commands */
+#define FDCCMD_STIN (0x40) /* | */
+#define FDCCMD_STOT (0x60) /* - */
+#define FDCCMD_RDSEC (0x80) /* - TYP 2 Commands */
+#define FDCCMD_WRSEC (0xa0) /* - " */
+#define FDCCMD_RDADR (0xc0) /* - */
+#define FDCCMD_RDTRA (0xe0) /* | TYP 3 Commands */
+#define FDCCMD_WRTRA (0xf0) /* - */
+#define FDCCMD_FORCI (0xd0) /* - TYP 4 Command */
+
+/* command modifier bits */
+
+#define FDCCMDADD_SR6 (0x00) /* step rate settings */
+#define FDCCMDADD_SR12 (0x01)
+#define FDCCMDADD_SR2 (0x02)
+#define FDCCMDADD_SR3 (0x03)
+#define FDCCMDADD_V (0x04) /* verify */
+#define FDCCMDADD_H (0x08) /* wait for spin-up */
+#define FDCCMDADD_U (0x10) /* update track register */
+#define FDCCMDADD_M (0x10) /* multiple sector access */
+#define FDCCMDADD_E (0x04) /* head settling flag */
+#define FDCCMDADD_P (0x02) /* precompensation off */
+#define FDCCMDADD_A0 (0x01) /* DAM flag */
+
+/* status register bits */
+
+#define FDCSTAT_MOTORON (0x80) /* motor on */
+#define FDCSTAT_WPROT (0x40) /* write protected (FDCCMD_WR*) */
+#define FDCSTAT_SPINUP (0x20) /* motor speed stable (Type I) */
+#define FDCSTAT_DELDAM (0x20) /* sector has deleted DAM (Type II+III) */
+#define FDCSTAT_RECNF (0x10) /* record not found */
+#define FDCSTAT_CRC (0x08) /* CRC error */
+#define FDCSTAT_TR00 (0x04) /* Track 00 flag (Type I) */
+#define FDCSTAT_LOST (0x04) /* Lost Data (Type II+III) */
+#define FDCSTAT_IDX (0x02) /* Index status (Type I) */
+#define FDCSTAT_DRQ (0x02) /* DRQ status (Type II+III) */
+#define FDCSTAT_BUSY (0x01) /* FDC is busy */
+
+
+/* PSG Port A Bit Nr 0 .. Side Sel .. 0 -> Side 1 1 -> Side 2 */
+#define DSKSIDE (0x01)
+
+#define DSKDRVNONE (0x06)
+#define DSKDRV0 (0x02)
+#define DSKDRV1 (0x04)
+
+/* step rates */
+#define FDCSTEP_6 0x00
+#define FDCSTEP_12 0x01
+#define FDCSTEP_2 0x02
+#define FDCSTEP_3 0x03
+
+struct atari_format_descr {
+ int track; /* to be formatted */
+ int head; /* "" "" */
+ int sect_offset; /* offset of first sector */
+};
/* Disk types: DD, HD, ED */
static struct atari_disk_type {
struct gendisk *disk;
int ref;
int type;
+ struct blk_mq_tag_set tag_set;
} unit[FD_MAX_UNITS];
#define UD unit[drive]
static int UserSteprate[FD_MAX_UNITS] = { -1, -1 };
module_param_array(UserSteprate, int, NULL, 0);
-/* Synchronization of FDC access. */
-static volatile int fdc_busy = 0;
-static DECLARE_WAIT_QUEUE_HEAD(fdc_wait);
static DECLARE_COMPLETION(format_wait);
static unsigned long changed_floppies = 0xff, fake_change = 0;
static void finish_fdc( void );
static void finish_fdc_done( int dummy );
static void setup_req_params( int drive );
-static void redo_fd_request( void);
static int fd_locked_ioctl(struct block_device *bdev, fmode_t mode, unsigned int
cmd, unsigned long param);
static void fd_probe( int drive );
static void fd_end_request_cur(blk_status_t err)
{
- if (!__blk_end_request_cur(fd_request, err))
+ if (!blk_update_request(fd_request, err,
+ blk_rq_cur_bytes(fd_request))) {
+ __blk_mq_end_request(fd_request, err);
fd_request = NULL;
+ }
}
static inline void start_motor_off_timer(void)
if (SelectedDrive != -1)
SUD.track = -1;
}
- redo_fd_request();
}
static int do_format(int drive, int type, struct atari_format_descr *desc)
{
+ struct request_queue *q = unit[drive].disk->queue;
unsigned char *p;
int sect, nsect;
unsigned long flags;
+ int ret;
- DPRINT(("do_format( dr=%d tr=%d he=%d offs=%d )\n",
- drive, desc->track, desc->head, desc->sect_offset ));
+ blk_mq_freeze_queue(q);
+ blk_mq_quiesce_queue(q);
- wait_event(fdc_wait, cmpxchg(&fdc_busy, 0, 1) == 0);
local_irq_save(flags);
stdma_lock(floppy_irq, NULL);
atari_turnon_irq( IRQ_MFP_FDC ); /* should be already, just to be sure */
if (type) {
if (--type >= NUM_DISK_MINORS ||
minor2disktype[type].drive_types > DriveType) {
- redo_fd_request();
- return -EINVAL;
+ ret = -EINVAL;
+ goto out;
}
type = minor2disktype[type].index;
UDT = &atari_disk_type[type];
}
if (!UDT || desc->track >= UDT->blocks/UDT->spt/2 || desc->head >= 2) {
- redo_fd_request();
- return -EINVAL;
+ ret = -EINVAL;
+ goto out;
}
nsect = UDT->spt;
wait_for_completion(&format_wait);
- redo_fd_request();
- return( FormatError ? -EIO : 0 );
+ ret = FormatError ? -EIO : 0;
+out:
+ blk_mq_unquiesce_queue(q);
+ blk_mq_unfreeze_queue(q);
+ return ret;
}
else {
/* all sectors finished */
fd_end_request_cur(BLK_STS_OK);
- redo_fd_request();
return;
}
}
else {
/* all sectors finished */
fd_end_request_cur(BLK_STS_OK);
- redo_fd_request();
}
return;
local_irq_save(flags);
stdma_release();
- fdc_busy = 0;
- wake_up( &fdc_wait );
local_irq_restore(flags);
DPRINT(("finish_fdc() finished\n"));
ReqTrack, ReqSector, (unsigned long)ReqData ));
}
-/*
- * Round-robin between our available drives, doing one request from each
- */
-static struct request *set_next_request(void)
+static blk_status_t ataflop_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
- struct request_queue *q;
- int old_pos = fdc_queue;
- struct request *rq = NULL;
-
- do {
- q = unit[fdc_queue].disk->queue;
- if (++fdc_queue == FD_MAX_UNITS)
- fdc_queue = 0;
- if (q) {
- rq = blk_fetch_request(q);
- if (rq) {
- rq->error_count = 0;
- break;
- }
- }
- } while (fdc_queue != old_pos);
-
- return rq;
-}
-
+ struct atari_floppy_struct *floppy = bd->rq->rq_disk->private_data;
+ int drive = floppy - unit;
+ int type = floppy->type;
-static void redo_fd_request(void)
-{
- int drive, type;
- struct atari_floppy_struct *floppy;
+ spin_lock_irq(&ataflop_lock);
+ if (fd_request) {
+ spin_unlock_irq(&ataflop_lock);
+ return BLK_STS_DEV_RESOURCE;
+ }
+ if (!stdma_try_lock(floppy_irq, NULL)) {
+ spin_unlock_irq(&ataflop_lock);
+ return BLK_STS_RESOURCE;
+ }
+ fd_request = bd->rq;
+ blk_mq_start_request(fd_request);
- DPRINT(("redo_fd_request: fd_request=%p dev=%s fd_request->sector=%ld\n",
- fd_request, fd_request ? fd_request->rq_disk->disk_name : "",
- fd_request ? blk_rq_pos(fd_request) : 0 ));
+ atari_disable_irq( IRQ_MFP_FDC );
IsFormatting = 0;
-repeat:
- if (!fd_request) {
- fd_request = set_next_request();
- if (!fd_request)
- goto the_end;
- }
-
- floppy = fd_request->rq_disk->private_data;
- drive = floppy - unit;
- type = floppy->type;
-
if (!UD.connected) {
/* drive not connected */
printk(KERN_ERR "Unknown Device: fd%d\n", drive );
fd_end_request_cur(BLK_STS_IOERR);
- goto repeat;
+ goto out;
}
if (type == 0) {
if (--type >= NUM_DISK_MINORS) {
printk(KERN_WARNING "fd%d: invalid disk format", drive );
fd_end_request_cur(BLK_STS_IOERR);
- goto repeat;
+ goto out;
}
if (minor2disktype[type].drive_types > DriveType) {
printk(KERN_WARNING "fd%d: unsupported disk format", drive );
fd_end_request_cur(BLK_STS_IOERR);
- goto repeat;
+ goto out;
}
type = minor2disktype[type].index;
UDT = &atari_disk_type[type];
set_capacity(floppy->disk, UDT->blocks);
UD.autoprobe = 0;
}
-
- if (blk_rq_pos(fd_request) + 1 > UDT->blocks) {
- fd_end_request_cur(BLK_STS_IOERR);
- goto repeat;
- }
/* stop deselect timer */
del_timer( &motor_off_timer );
setup_req_params( drive );
do_fd_action( drive );
- return;
-
- the_end:
- finish_fdc();
-}
-
-
-void do_fd_request(struct request_queue * q)
-{
- DPRINT(("do_fd_request for pid %d\n",current->pid));
- wait_event(fdc_wait, cmpxchg(&fdc_busy, 0, 1) == 0);
- stdma_lock(floppy_irq, NULL);
-
- atari_disable_irq( IRQ_MFP_FDC );
- redo_fd_request();
+ if (bd->last)
+ finish_fdc();
atari_enable_irq( IRQ_MFP_FDC );
+
+out:
+ spin_unlock_irq(&ataflop_lock);
+ return BLK_STS_OK;
}
static int fd_locked_ioctl(struct block_device *bdev, fmode_t mode,
/* what if type > 0 here? Overwrite specified entry ? */
if (type) {
/* refuse to re-set a predefined type for now */
- redo_fd_request();
return -EINVAL;
}
/* sanity check */
if (setprm.track != dtp->blocks/dtp->spt/2 ||
- setprm.head != 2) {
- redo_fd_request();
+ setprm.head != 2)
return -EINVAL;
- }
UDT = dtp;
set_capacity(floppy->disk, UDT->blocks);
.revalidate_disk= floppy_revalidate,
};
+static const struct blk_mq_ops ataflop_mq_ops = {
+ .queue_rq = ataflop_queue_rq,
+};
+
static struct kobject *floppy_find(dev_t dev, int *part, void *data)
{
int drive = *part & 3;
static int __init atari_floppy_init (void)
{
int i;
+ int ret;
if (!MACH_IS_ATARI)
/* Amiga, Mac, ... don't have Atari-compatible floppy :-) */
for (i = 0; i < FD_MAX_UNITS; i++) {
unit[i].disk = alloc_disk(1);
- if (!unit[i].disk)
- goto Enomem;
+ if (!unit[i].disk) {
+ ret = -ENOMEM;
+ goto err;
+ }
+
+ unit[i].disk->queue = blk_mq_init_sq_queue(&unit[i].tag_set,
+ &ataflop_mq_ops, 2,
+ BLK_MQ_F_SHOULD_MERGE);
+ if (IS_ERR(unit[i].disk->queue)) {
+ ret = PTR_ERR(unit[i].disk->queue);
+ unit[i].disk->queue = NULL;
+ goto err;
+ }
}
if (UseTrackbuffer < 0)
DMABuffer = atari_stram_alloc(BUFFER_SIZE+512, "ataflop");
if (!DMABuffer) {
printk(KERN_ERR "atari_floppy_init: cannot get dma buffer\n");
- goto Enomem;
+ ret = -ENOMEM;
+ goto err;
}
TrackBuffer = DMABuffer + 512;
PhysDMABuffer = atari_stram_to_phys(DMABuffer);
sprintf(unit[i].disk->disk_name, "fd%d", i);
unit[i].disk->fops = &floppy_fops;
unit[i].disk->private_data = &unit[i];
- unit[i].disk->queue = blk_init_queue(do_fd_request,
- &ataflop_lock);
- if (!unit[i].disk->queue)
- goto Enomem;
set_capacity(unit[i].disk, MAX_DISK_SIZE * 2);
add_disk(unit[i].disk);
}
config_types();
return 0;
-Enomem:
- while (i--) {
- struct request_queue *q = unit[i].disk->queue;
- put_disk(unit[i].disk);
- if (q)
- blk_cleanup_queue(q);
- }
+err:
+ do {
+ struct gendisk *disk = unit[i].disk;
+
+ if (disk) {
+ if (disk->queue) {
+ blk_cleanup_queue(disk->queue);
+ disk->queue = NULL;
+ }
+ blk_mq_free_tag_set(&unit[i].tag_set);
+ put_disk(unit[i].disk);
+ }
+ } while (i--);
unregister_blkdev(FLOPPY_MAJOR, "fd");
- return -ENOMEM;
+ return ret;
}
#ifndef MODULE
int i;
blk_unregister_region(MKDEV(FLOPPY_MAJOR, 0), 256);
for (i = 0; i < FD_MAX_UNITS; i++) {
- struct request_queue *q = unit[i].disk->queue;
-
del_gendisk(unit[i].disk);
+ blk_cleanup_queue(unit[i].disk->queue);
+ blk_mq_free_tag_set(&unit[i].tag_set);
put_disk(unit[i].disk);
- blk_cleanup_queue(q);
}
unregister_blkdev(FLOPPY_MAJOR, "fd");
depends on PROC_FS && INET
select LRU_CACHE
select LIBCRC32C
- default n
help
NOTE: In order to authenticate connections you have to select
__EE_CALL_AL_COMPLETE_IO,
__EE_MAY_SET_IN_SYNC,
- /* is this a TRIM aka REQ_DISCARD? */
+ /* is this a TRIM aka REQ_OP_DISCARD? */
__EE_IS_TRIM,
/* In case a barrier failed,
struct list_head transfer_log; /* all requests not yet fully processed */
struct crypto_shash *cram_hmac_tfm;
- struct crypto_ahash *integrity_tfm; /* checksums we compute, updates protected by connection->data->mutex */
- struct crypto_ahash *peer_integrity_tfm; /* checksums we verify, only accessed from receiver thread */
- struct crypto_ahash *csums_tfm;
- struct crypto_ahash *verify_tfm;
+ struct crypto_shash *integrity_tfm; /* checksums we compute, updates protected by connection->data->mutex */
+ struct crypto_shash *peer_integrity_tfm; /* checksums we verify, only accessed from receiver thread */
+ struct crypto_shash *csums_tfm;
+ struct crypto_shash *verify_tfm;
void *int_dig_in;
void *int_dig_vv;
}
-extern void drbd_csum_bio(struct crypto_ahash *, struct bio *, void *);
-extern void drbd_csum_ee(struct crypto_ahash *, struct drbd_peer_request *, void *);
+extern void drbd_csum_bio(struct crypto_shash *, struct bio *, void *);
+extern void drbd_csum_ee(struct crypto_shash *, struct drbd_peer_request *,
+ void *);
/* worker callbacks */
extern int w_e_end_data_req(struct drbd_work *, int);
extern int w_e_end_rsdata_req(struct drbd_work *, int);
struct p_data *dp, int data_size)
{
if (peer_device->connection->peer_integrity_tfm)
- data_size -= crypto_ahash_digestsize(peer_device->connection->peer_integrity_tfm);
+ data_size -= crypto_shash_digestsize(peer_device->connection->peer_integrity_tfm);
_drbd_send_ack(peer_device, cmd, dp->sector, cpu_to_be32(data_size),
dp->block_id);
}
return bio->bi_opf & REQ_SYNC ? DP_RW_SYNC : 0;
}
-/* Used to send write or TRIM aka REQ_DISCARD requests
+/* Used to send write or TRIM aka REQ_OP_DISCARD requests
* R_PRIMARY -> Peer (P_DATA, P_TRIM)
*/
int drbd_send_dblock(struct drbd_peer_device *peer_device, struct drbd_request *req)
sock = &peer_device->connection->data;
p = drbd_prepare_command(peer_device, sock);
digest_size = peer_device->connection->integrity_tfm ?
- crypto_ahash_digestsize(peer_device->connection->integrity_tfm) : 0;
+ crypto_shash_digestsize(peer_device->connection->integrity_tfm) : 0;
if (!p)
return -EIO;
p = drbd_prepare_command(peer_device, sock);
digest_size = peer_device->connection->integrity_tfm ?
- crypto_ahash_digestsize(peer_device->connection->integrity_tfm) : 0;
+ crypto_shash_digestsize(peer_device->connection->integrity_tfm) : 0;
if (!p)
return -EIO;
{
drbd_free_sock(connection);
- crypto_free_ahash(connection->csums_tfm);
- crypto_free_ahash(connection->verify_tfm);
+ crypto_free_shash(connection->csums_tfm);
+ crypto_free_shash(connection->verify_tfm);
crypto_free_shash(connection->cram_hmac_tfm);
- crypto_free_ahash(connection->integrity_tfm);
- crypto_free_ahash(connection->peer_integrity_tfm);
+ crypto_free_shash(connection->integrity_tfm);
+ crypto_free_shash(connection->peer_integrity_tfm);
kfree(connection->int_dig_in);
kfree(connection->int_dig_vv);
}
struct crypto {
- struct crypto_ahash *verify_tfm;
- struct crypto_ahash *csums_tfm;
+ struct crypto_shash *verify_tfm;
+ struct crypto_shash *csums_tfm;
struct crypto_shash *cram_hmac_tfm;
- struct crypto_ahash *integrity_tfm;
+ struct crypto_shash *integrity_tfm;
};
static int
return NO_ERROR;
}
-static int
-alloc_ahash(struct crypto_ahash **tfm, char *tfm_name, int err_alg)
-{
- if (!tfm_name[0])
- return NO_ERROR;
-
- *tfm = crypto_alloc_ahash(tfm_name, 0, CRYPTO_ALG_ASYNC);
- if (IS_ERR(*tfm)) {
- *tfm = NULL;
- return err_alg;
- }
-
- return NO_ERROR;
-}
-
static enum drbd_ret_code
alloc_crypto(struct crypto *crypto, struct net_conf *new_net_conf)
{
char hmac_name[CRYPTO_MAX_ALG_NAME];
enum drbd_ret_code rv;
- rv = alloc_ahash(&crypto->csums_tfm, new_net_conf->csums_alg,
+ rv = alloc_shash(&crypto->csums_tfm, new_net_conf->csums_alg,
ERR_CSUMS_ALG);
if (rv != NO_ERROR)
return rv;
- rv = alloc_ahash(&crypto->verify_tfm, new_net_conf->verify_alg,
+ rv = alloc_shash(&crypto->verify_tfm, new_net_conf->verify_alg,
ERR_VERIFY_ALG);
if (rv != NO_ERROR)
return rv;
- rv = alloc_ahash(&crypto->integrity_tfm, new_net_conf->integrity_alg,
+ rv = alloc_shash(&crypto->integrity_tfm, new_net_conf->integrity_alg,
ERR_INTEGRITY_ALG);
if (rv != NO_ERROR)
return rv;
static void free_crypto(struct crypto *crypto)
{
crypto_free_shash(crypto->cram_hmac_tfm);
- crypto_free_ahash(crypto->integrity_tfm);
- crypto_free_ahash(crypto->csums_tfm);
- crypto_free_ahash(crypto->verify_tfm);
+ crypto_free_shash(crypto->integrity_tfm);
+ crypto_free_shash(crypto->csums_tfm);
+ crypto_free_shash(crypto->verify_tfm);
}
int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
rcu_assign_pointer(connection->net_conf, new_net_conf);
if (!rsr) {
- crypto_free_ahash(connection->csums_tfm);
+ crypto_free_shash(connection->csums_tfm);
connection->csums_tfm = crypto.csums_tfm;
crypto.csums_tfm = NULL;
}
if (!ovr) {
- crypto_free_ahash(connection->verify_tfm);
+ crypto_free_shash(connection->verify_tfm);
connection->verify_tfm = crypto.verify_tfm;
crypto.verify_tfm = NULL;
}
- crypto_free_ahash(connection->integrity_tfm);
+ crypto_free_shash(connection->integrity_tfm);
connection->integrity_tfm = crypto.integrity_tfm;
if (connection->cstate >= C_WF_REPORT_PARAMS && connection->agreed_pro_version >= 100)
/* Do this without trying to take connection->data.mutex again. */
P_PROTOCOL_UPDATE = 0x2d, /* data sock: is used in established connections */
/* 0x2e to 0x30 reserved, used in drbd 9 */
- /* REQ_DISCARD. We used "discard" in different contexts before,
+ /* REQ_OP_DISCARD. We used "discard" in different contexts before,
* which is why I chose TRIM here, to disambiguate. */
P_TRIM = 0x31,
#define DP_UNPLUG 8 /* not used anymore */
#define DP_FUA 16 /* equals REQ_FUA */
#define DP_FLUSH 32 /* equals REQ_PREFLUSH */
-#define DP_DISCARD 64 /* equals REQ_DISCARD */
+#define DP_DISCARD 64 /* equals REQ_OP_DISCARD */
#define DP_SEND_RECEIVE_ACK 128 /* This is a proto B write request */
#define DP_SEND_WRITE_ACK 256 /* This is a proto C write request */
#define DP_WSAME 512 /* equiv. REQ_WRITE_SAME */
}
/* quick wrapper in case payload size != request_size (write same) */
-static void drbd_csum_ee_size(struct crypto_ahash *h,
+static void drbd_csum_ee_size(struct crypto_shash *h,
struct drbd_peer_request *r, void *d,
unsigned int payload_size)
{
digest_size = 0;
if (!trim && peer_device->connection->peer_integrity_tfm) {
- digest_size = crypto_ahash_digestsize(peer_device->connection->peer_integrity_tfm);
+ digest_size = crypto_shash_digestsize(peer_device->connection->peer_integrity_tfm);
/*
* FIXME: Receive the incoming digest into the receive buffer
* here, together with its struct p_data?
digest_size = 0;
if (peer_device->connection->peer_integrity_tfm) {
- digest_size = crypto_ahash_digestsize(peer_device->connection->peer_integrity_tfm);
+ digest_size = crypto_shash_digestsize(peer_device->connection->peer_integrity_tfm);
err = drbd_recv_all_warn(peer_device->connection, dig_in, digest_size);
if (err)
return err;
int p_proto, p_discard_my_data, p_two_primaries, cf;
struct net_conf *nc, *old_net_conf, *new_net_conf = NULL;
char integrity_alg[SHARED_SECRET_MAX] = "";
- struct crypto_ahash *peer_integrity_tfm = NULL;
+ struct crypto_shash *peer_integrity_tfm = NULL;
void *int_dig_in = NULL, *int_dig_vv = NULL;
p_proto = be32_to_cpu(p->protocol);
* change.
*/
- peer_integrity_tfm = crypto_alloc_ahash(integrity_alg, 0, CRYPTO_ALG_ASYNC);
+ peer_integrity_tfm = crypto_alloc_shash(integrity_alg, 0, CRYPTO_ALG_ASYNC);
if (IS_ERR(peer_integrity_tfm)) {
peer_integrity_tfm = NULL;
drbd_err(connection, "peer data-integrity-alg %s not supported\n",
goto disconnect;
}
- hash_size = crypto_ahash_digestsize(peer_integrity_tfm);
+ hash_size = crypto_shash_digestsize(peer_integrity_tfm);
int_dig_in = kmalloc(hash_size, GFP_KERNEL);
int_dig_vv = kmalloc(hash_size, GFP_KERNEL);
if (!(int_dig_in && int_dig_vv)) {
mutex_unlock(&connection->resource->conf_update);
mutex_unlock(&connection->data.mutex);
- crypto_free_ahash(connection->peer_integrity_tfm);
+ crypto_free_shash(connection->peer_integrity_tfm);
kfree(connection->int_dig_in);
kfree(connection->int_dig_vv);
connection->peer_integrity_tfm = peer_integrity_tfm;
disconnect_rcu_unlock:
rcu_read_unlock();
disconnect:
- crypto_free_ahash(peer_integrity_tfm);
+ crypto_free_shash(peer_integrity_tfm);
kfree(int_dig_in);
kfree(int_dig_vv);
conn_request_state(connection, NS(conn, C_DISCONNECTING), CS_HARD);
* return: NULL (alg name was "")
* ERR_PTR(error) if something goes wrong
* or the crypto hash ptr, if it worked out ok. */
-static struct crypto_ahash *drbd_crypto_alloc_digest_safe(const struct drbd_device *device,
+static struct crypto_shash *drbd_crypto_alloc_digest_safe(
+ const struct drbd_device *device,
const char *alg, const char *name)
{
- struct crypto_ahash *tfm;
+ struct crypto_shash *tfm;
if (!alg[0])
return NULL;
- tfm = crypto_alloc_ahash(alg, 0, CRYPTO_ALG_ASYNC);
+ tfm = crypto_alloc_shash(alg, 0, 0);
if (IS_ERR(tfm)) {
drbd_err(device, "Can not allocate \"%s\" as %s (reason: %ld)\n",
alg, name, PTR_ERR(tfm));
struct drbd_device *device;
struct p_rs_param_95 *p;
unsigned int header_size, data_size, exp_max_sz;
- struct crypto_ahash *verify_tfm = NULL;
- struct crypto_ahash *csums_tfm = NULL;
+ struct crypto_shash *verify_tfm = NULL;
+ struct crypto_shash *csums_tfm = NULL;
struct net_conf *old_net_conf, *new_net_conf = NULL;
struct disk_conf *old_disk_conf = NULL, *new_disk_conf = NULL;
const int apv = connection->agreed_pro_version;
if (verify_tfm) {
strcpy(new_net_conf->verify_alg, p->verify_alg);
new_net_conf->verify_alg_len = strlen(p->verify_alg) + 1;
- crypto_free_ahash(peer_device->connection->verify_tfm);
+ crypto_free_shash(peer_device->connection->verify_tfm);
peer_device->connection->verify_tfm = verify_tfm;
drbd_info(device, "using verify-alg: \"%s\"\n", p->verify_alg);
}
if (csums_tfm) {
strcpy(new_net_conf->csums_alg, p->csums_alg);
new_net_conf->csums_alg_len = strlen(p->csums_alg) + 1;
- crypto_free_ahash(peer_device->connection->csums_tfm);
+ crypto_free_shash(peer_device->connection->csums_tfm);
peer_device->connection->csums_tfm = csums_tfm;
drbd_info(device, "using csums-alg: \"%s\"\n", p->csums_alg);
}
mutex_unlock(&connection->resource->conf_update);
/* just for completeness: actually not needed,
* as this is not reached if csums_tfm was ok. */
- crypto_free_ahash(csums_tfm);
+ crypto_free_shash(csums_tfm);
/* but free the verify_tfm again, if csums_tfm did not work out */
- crypto_free_ahash(verify_tfm);
+ crypto_free_shash(verify_tfm);
conn_request_state(peer_device->connection, NS(conn, C_DISCONNECTING), CS_HARD);
return -EIO;
}
case DISCARD_COMPLETED_NOTSUPP:
case DISCARD_COMPLETED_WITH_ERROR:
/* I'd rather not detach from local disk just because it
- * failed a REQ_DISCARD. */
+ * failed a REQ_OP_DISCARD. */
mod_rq_state(req, m, RQ_LOCAL_PENDING, RQ_LOCAL_COMPLETED);
break;
do_wake = list_empty(block_id == ID_SYNCER ? &device->sync_ee : &device->active_ee);
- /* FIXME do we want to detach for failed REQ_DISCARD?
+ /* FIXME do we want to detach for failed REQ_OP_DISCARD?
* ((peer_req->flags & (EE_WAS_ERROR|EE_IS_TRIM)) == EE_WAS_ERROR) */
if (peer_req->flags & EE_WAS_ERROR)
__drbd_chk_io_error(device, DRBD_WRITE_ERROR);
complete_master_bio(device, &m);
}
-void drbd_csum_ee(struct crypto_ahash *tfm, struct drbd_peer_request *peer_req, void *digest)
+void drbd_csum_ee(struct crypto_shash *tfm, struct drbd_peer_request *peer_req, void *digest)
{
- AHASH_REQUEST_ON_STACK(req, tfm);
- struct scatterlist sg;
+ SHASH_DESC_ON_STACK(desc, tfm);
struct page *page = peer_req->pages;
struct page *tmp;
unsigned len;
+ void *src;
- ahash_request_set_tfm(req, tfm);
- ahash_request_set_callback(req, 0, NULL, NULL);
+ desc->tfm = tfm;
+ desc->flags = 0;
- sg_init_table(&sg, 1);
- crypto_ahash_init(req);
+ crypto_shash_init(desc);
+ src = kmap_atomic(page);
while ((tmp = page_chain_next(page))) {
/* all but the last page will be fully used */
- sg_set_page(&sg, page, PAGE_SIZE, 0);
- ahash_request_set_crypt(req, &sg, NULL, sg.length);
- crypto_ahash_update(req);
+ crypto_shash_update(desc, src, PAGE_SIZE);
+ kunmap_atomic(src);
page = tmp;
+ src = kmap_atomic(page);
}
/* and now the last, possibly only partially used page */
len = peer_req->i.size & (PAGE_SIZE - 1);
- sg_set_page(&sg, page, len ?: PAGE_SIZE, 0);
- ahash_request_set_crypt(req, &sg, digest, sg.length);
- crypto_ahash_finup(req);
- ahash_request_zero(req);
+ crypto_shash_update(desc, src, len ?: PAGE_SIZE);
+ kunmap_atomic(src);
+
+ crypto_shash_final(desc, digest);
+ shash_desc_zero(desc);
}
-void drbd_csum_bio(struct crypto_ahash *tfm, struct bio *bio, void *digest)
+void drbd_csum_bio(struct crypto_shash *tfm, struct bio *bio, void *digest)
{
- AHASH_REQUEST_ON_STACK(req, tfm);
- struct scatterlist sg;
+ SHASH_DESC_ON_STACK(desc, tfm);
struct bio_vec bvec;
struct bvec_iter iter;
- ahash_request_set_tfm(req, tfm);
- ahash_request_set_callback(req, 0, NULL, NULL);
+ desc->tfm = tfm;
+ desc->flags = 0;
- sg_init_table(&sg, 1);
- crypto_ahash_init(req);
+ crypto_shash_init(desc);
bio_for_each_segment(bvec, bio, iter) {
- sg_set_page(&sg, bvec.bv_page, bvec.bv_len, bvec.bv_offset);
- ahash_request_set_crypt(req, &sg, NULL, sg.length);
- crypto_ahash_update(req);
+ u8 *src;
+
+ src = kmap_atomic(bvec.bv_page);
+ crypto_shash_update(desc, src + bvec.bv_offset, bvec.bv_len);
+ kunmap_atomic(src);
+
/* REQ_OP_WRITE_SAME has only one segment,
* checksum the payload only once. */
if (bio_op(bio) == REQ_OP_WRITE_SAME)
break;
}
- ahash_request_set_crypt(req, NULL, digest, 0);
- crypto_ahash_final(req);
- ahash_request_zero(req);
+ crypto_shash_final(desc, digest);
+ shash_desc_zero(desc);
}
/* MAYBE merge common code with w_e_end_ov_req */
if (unlikely((peer_req->flags & EE_WAS_ERROR) != 0))
goto out;
- digest_size = crypto_ahash_digestsize(peer_device->connection->csums_tfm);
+ digest_size = crypto_shash_digestsize(peer_device->connection->csums_tfm);
digest = kmalloc(digest_size, GFP_NOIO);
if (digest) {
sector_t sector = peer_req->i.sector;
* a real fix would be much more involved,
* introducing more locking mechanisms */
if (peer_device->connection->csums_tfm) {
- digest_size = crypto_ahash_digestsize(peer_device->connection->csums_tfm);
+ digest_size = crypto_shash_digestsize(peer_device->connection->csums_tfm);
D_ASSERT(device, digest_size == di->digest_size);
digest = kmalloc(digest_size, GFP_NOIO);
}
if (unlikely(cancel))
goto out;
- digest_size = crypto_ahash_digestsize(peer_device->connection->verify_tfm);
+ digest_size = crypto_shash_digestsize(peer_device->connection->verify_tfm);
digest = kmalloc(digest_size, GFP_NOIO);
if (!digest) {
err = 1; /* terminate the connection in case the allocation failed */
di = peer_req->digest;
if (likely((peer_req->flags & EE_WAS_ERROR) == 0)) {
- digest_size = crypto_ahash_digestsize(peer_device->connection->verify_tfm);
+ digest_size = crypto_shash_digestsize(peer_device->connection->verify_tfm);
digest = kmalloc(digest_size, GFP_NOIO);
if (digest) {
drbd_csum_ee(peer_device->connection->verify_tfm, peer_req, digest);
static int irqdma_allocated;
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/blkpg.h>
#include <linux/cdrom.h> /* for the compatibility eject ioctl */
#include <linux/completion.h>
+static LIST_HEAD(floppy_reqs);
static struct request *current_req;
-static void do_fd_request(struct request_queue *q);
static int set_next_request(void);
#ifndef fd_get_dma_residue
static struct floppy_write_errors write_errors[N_DRIVE];
static struct timer_list motor_off_timer[N_DRIVE];
static struct gendisk *disks[N_DRIVE];
+static struct blk_mq_tag_set tag_sets[N_DRIVE];
static struct block_device *opened_bdev[N_DRIVE];
static DEFINE_MUTEX(open_lock);
static struct floppy_raw_cmd *raw_cmd, default_raw_cmd;
-static int fdc_queue;
/*
* This struct defines the different floppy types.
/* current_count_sectors can be zero if transfer failed */
if (error)
nr_sectors = blk_rq_cur_sectors(req);
- if (__blk_end_request(req, error, nr_sectors << 9))
+ if (blk_update_request(req, error, nr_sectors << 9))
return;
+ __blk_mq_end_request(req, error);
/* We're done with the request */
floppy_off(drive);
return 2;
}
-/*
- * Round-robin between our available drives, doing one request from each
- */
static int set_next_request(void)
{
- struct request_queue *q;
- int old_pos = fdc_queue;
-
- do {
- q = disks[fdc_queue]->queue;
- if (++fdc_queue == N_DRIVE)
- fdc_queue = 0;
- if (q) {
- current_req = blk_fetch_request(q);
- if (current_req) {
- current_req->error_count = 0;
- break;
- }
- }
- } while (fdc_queue != old_pos);
-
+ current_req = list_first_entry_or_null(&floppy_reqs, struct request,
+ queuelist);
+ if (current_req) {
+ current_req->error_count = 0;
+ list_del_init(¤t_req->queuelist);
+ }
return current_req != NULL;
}
schedule_bh(redo_fd_request);
}
-static void do_fd_request(struct request_queue *q)
+static blk_status_t floppy_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
+ blk_mq_start_request(bd->rq);
+
if (WARN(max_buffer_sectors == 0,
"VFS: %s called on non-open device\n", __func__))
- return;
+ return BLK_STS_IOERR;
if (WARN(atomic_read(&usage_count) == 0,
"warning: usage count=0, current_req=%p sect=%ld flags=%llx\n",
current_req, (long)blk_rq_pos(current_req),
(unsigned long long) current_req->cmd_flags))
- return;
+ return BLK_STS_IOERR;
+
+ spin_lock_irq(&floppy_lock);
+ list_add_tail(&bd->rq->queuelist, &floppy_reqs);
+ spin_unlock_irq(&floppy_lock);
if (test_and_set_bit(0, &fdc_busy)) {
/* fdc busy, this new request will be treated when the
current one is done */
is_alive(__func__, "old request running");
- return;
+ return BLK_STS_OK;
}
+
command_status = FD_COMMAND_NONE;
__reschedule_timeout(MAXTIMEOUT, "fd_request");
set_fdc(0);
process_fd_request();
is_alive(__func__, "");
+ return BLK_STS_OK;
}
static const struct cont_t poll_cont = {
},
};
+static const struct blk_mq_ops floppy_mq_ops = {
+ .queue_rq = floppy_queue_rq,
+};
+
static struct platform_device floppy_device[N_DRIVE];
static bool floppy_available(int drive)
goto out_put_disk;
}
- disks[drive]->queue = blk_init_queue(do_fd_request, &floppy_lock);
- if (!disks[drive]->queue) {
- err = -ENOMEM;
+ disks[drive]->queue = blk_mq_init_sq_queue(&tag_sets[drive],
+ &floppy_mq_ops, 2,
+ BLK_MQ_F_SHOULD_MERGE);
+ if (IS_ERR(disks[drive]->queue)) {
+ err = PTR_ERR(disks[drive]->queue);
+ disks[drive]->queue = NULL;
goto out_put_disk;
}
/* to be cleaned up... */
disks[drive]->private_data = (void *)(long)drive;
disks[drive]->flags |= GENHD_FL_REMOVABLE;
- device_add_disk(&floppy_device[drive].dev, disks[drive]);
+ device_add_disk(&floppy_device[drive].dev, disks[drive], NULL);
}
return 0;
del_timer_sync(&motor_off_timer[drive]);
blk_cleanup_queue(disks[drive]->queue);
disks[drive]->queue = NULL;
+ blk_mq_free_tag_set(&tag_sets[drive]);
}
put_disk(disks[drive]);
}
platform_device_unregister(&floppy_device[drive]);
}
blk_cleanup_queue(disks[drive]->queue);
+ blk_mq_free_tag_set(&tag_sets[drive]);
/*
* These disks have not called add_disk(). Don't put down
#include <linux/falloc.h>
#include <linux/uio.h>
#include <linux/ioprio.h>
+#include <linux/blk-cgroup.h>
#include "loop.h"
/* always use the first bio's css */
#ifdef CONFIG_BLK_CGROUP
- if (cmd->use_aio && rq->bio && rq->bio->bi_css) {
- cmd->css = rq->bio->bi_css;
+ if (cmd->use_aio && rq->bio && rq->bio->bi_blkg) {
+ cmd->css = &bio_blkcg(rq->bio)->css;
css_get(cmd->css);
} else
#endif
if (IS_ERR(outbuf))
return PTR_ERR(outbuf);
- outbuf_dma = pci_map_single(dd->pdev,
- outbuf,
- taskout,
- DMA_TO_DEVICE);
- if (pci_dma_mapping_error(dd->pdev, outbuf_dma)) {
+ outbuf_dma = dma_map_single(&dd->pdev->dev, outbuf,
+ taskout, DMA_TO_DEVICE);
+ if (dma_mapping_error(&dd->pdev->dev, outbuf_dma)) {
err = -ENOMEM;
goto abort;
}
inbuf = NULL;
goto abort;
}
- inbuf_dma = pci_map_single(dd->pdev,
- inbuf,
- taskin, DMA_FROM_DEVICE);
- if (pci_dma_mapping_error(dd->pdev, inbuf_dma)) {
+ inbuf_dma = dma_map_single(&dd->pdev->dev, inbuf,
+ taskin, DMA_FROM_DEVICE);
+ if (dma_mapping_error(&dd->pdev->dev, inbuf_dma)) {
err = -ENOMEM;
goto abort;
}
/* reclaim the DMA buffers.*/
if (inbuf_dma)
- pci_unmap_single(dd->pdev, inbuf_dma,
- taskin, DMA_FROM_DEVICE);
+ dma_unmap_single(&dd->pdev->dev, inbuf_dma, taskin,
+ DMA_FROM_DEVICE);
if (outbuf_dma)
- pci_unmap_single(dd->pdev, outbuf_dma,
- taskout, DMA_TO_DEVICE);
+ dma_unmap_single(&dd->pdev->dev, outbuf_dma, taskout,
+ DMA_TO_DEVICE);
inbuf_dma = 0;
outbuf_dma = 0;
}
abort:
if (inbuf_dma)
- pci_unmap_single(dd->pdev, inbuf_dma,
- taskin, DMA_FROM_DEVICE);
+ dma_unmap_single(&dd->pdev->dev, inbuf_dma, taskin,
+ DMA_FROM_DEVICE);
if (outbuf_dma)
- pci_unmap_single(dd->pdev, outbuf_dma,
- taskout, DMA_TO_DEVICE);
+ dma_unmap_single(&dd->pdev->dev, outbuf_dma, taskout,
+ DMA_TO_DEVICE);
kfree(outbuf);
kfree(inbuf);
set_capacity(dd->disk, capacity);
/* Enable the block device and add it to /dev */
- device_add_disk(&dd->pdev->dev, dd->disk);
+ device_add_disk(&dd->pdev->dev, dd->disk, NULL);
dd->bdev = bdget_disk(dd->disk, 0);
/*
goto iomap_err;
}
- if (!pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
- rv = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
-
- if (rv) {
- rv = pci_set_consistent_dma_mask(pdev,
- DMA_BIT_MASK(32));
- if (rv) {
- dev_warn(&pdev->dev,
- "64-bit DMA enable failed\n");
- goto setmask_err;
- }
- }
+ rv = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (rv) {
+ dev_warn(&pdev->dev, "64-bit DMA enable failed\n");
+ goto setmask_err;
}
/* Copy the info we may need later into the private data structure. */
static void end_cmd(struct nullb_cmd *cmd)
{
- struct request_queue *q = NULL;
int queue_mode = cmd->nq->dev->queue_mode;
- if (cmd->rq)
- q = cmd->rq->q;
-
switch (queue_mode) {
case NULL_Q_MQ:
blk_mq_end_request(cmd->rq, cmd->error);
return;
- case NULL_Q_RQ:
- INIT_LIST_HEAD(&cmd->rq->queuelist);
- blk_end_request_all(cmd->rq, cmd->error);
- break;
case NULL_Q_BIO:
cmd->bio->bi_status = cmd->error;
bio_endio(cmd->bio);
}
free_cmd(cmd);
-
- /* Restart queue if needed, as we are freeing a tag */
- if (queue_mode == NULL_Q_RQ && blk_queue_stopped(q)) {
- unsigned long flags;
-
- spin_lock_irqsave(q->queue_lock, flags);
- blk_start_queue_async(q);
- spin_unlock_irqrestore(q->queue_lock, flags);
- }
}
static enum hrtimer_restart null_cmd_timer_expired(struct hrtimer *timer)
if (nullb->dev->queue_mode == NULL_Q_MQ)
blk_mq_stop_hw_queues(q);
- else {
- spin_lock_irq(q->queue_lock);
- blk_stop_queue(q);
- spin_unlock_irq(q->queue_lock);
- }
}
static void null_restart_queue_async(struct nullb *nullb)
{
struct request_queue *q = nullb->q;
- unsigned long flags;
if (nullb->dev->queue_mode == NULL_Q_MQ)
blk_mq_start_stopped_hw_queues(q, true);
- else {
- spin_lock_irqsave(q->queue_lock, flags);
- blk_start_queue_async(q);
- spin_unlock_irqrestore(q->queue_lock, flags);
- }
}
static bool cmd_report_zone(struct nullb *nullb, struct nullb_cmd *cmd)
/* race with timer */
if (atomic_long_read(&nullb->cur_bytes) > 0)
null_restart_queue_async(nullb);
- if (dev->queue_mode == NULL_Q_RQ) {
- struct request_queue *q = nullb->q;
-
- spin_lock_irq(q->queue_lock);
- rq->rq_flags |= RQF_DONTPREP;
- blk_requeue_request(q, rq);
- spin_unlock_irq(q->queue_lock);
- return BLK_STS_OK;
- } else
- /* requeue request */
- return BLK_STS_DEV_RESOURCE;
+ /* requeue request */
+ return BLK_STS_DEV_RESOURCE;
}
}
case NULL_Q_MQ:
blk_mq_complete_request(cmd->rq);
break;
- case NULL_Q_RQ:
- blk_complete_request(cmd->rq);
- break;
case NULL_Q_BIO:
/*
* XXX: no proper submitting cpu information available.
return BLK_QC_T_NONE;
}
-static enum blk_eh_timer_return null_rq_timed_out_fn(struct request *rq)
-{
- pr_info("null: rq %p timed out\n", rq);
- __blk_complete_request(rq);
- return BLK_EH_DONE;
-}
-
-static int null_rq_prep_fn(struct request_queue *q, struct request *req)
-{
- struct nullb *nullb = q->queuedata;
- struct nullb_queue *nq = nullb_to_queue(nullb);
- struct nullb_cmd *cmd;
-
- cmd = alloc_cmd(nq, 0);
- if (cmd) {
- cmd->rq = req;
- req->special = cmd;
- return BLKPREP_OK;
- }
- blk_stop_queue(q);
-
- return BLKPREP_DEFER;
-}
-
static bool should_timeout_request(struct request *rq)
{
#ifdef CONFIG_BLK_DEV_NULL_BLK_FAULT_INJECTION
return false;
}
-static void null_request_fn(struct request_queue *q)
-{
- struct request *rq;
-
- while ((rq = blk_fetch_request(q)) != NULL) {
- struct nullb_cmd *cmd = rq->special;
-
- /* just ignore the request */
- if (should_timeout_request(rq))
- continue;
- if (should_requeue_request(rq)) {
- blk_requeue_request(q, rq);
- continue;
- }
-
- spin_unlock_irq(q->queue_lock);
- null_handle_cmd(cmd);
- spin_lock_irq(q->queue_lock);
- }
-}
-
static enum blk_eh_timer_return null_timeout_rq(struct request *rq, bool res)
{
pr_info("null: rq %p timed out\n", rq);
rv = init_driver_queues(nullb);
if (rv)
goto out_cleanup_blk_queue;
- } else {
- nullb->q = blk_init_queue_node(null_request_fn, &nullb->lock,
- dev->home_node);
- if (!nullb->q) {
- rv = -ENOMEM;
- goto out_cleanup_queues;
- }
-
- if (!null_setup_fault())
- goto out_cleanup_blk_queue;
-
- blk_queue_prep_rq(nullb->q, null_rq_prep_fn);
- blk_queue_softirq_done(nullb->q, null_softirq_done_fn);
- blk_queue_rq_timed_out(nullb->q, null_rq_timed_out_fn);
- nullb->q->rq_timeout = 5 * HZ;
- rv = init_driver_queues(nullb);
- if (rv)
- goto out_cleanup_blk_queue;
}
if (dev->mbps) {
return -EINVAL;
}
+ if (g_queue_mode == NULL_Q_RQ) {
+ pr_err("null_blk: legacy IO path no longer available\n");
+ return -EINVAL;
+ }
if (g_queue_mode == NULL_Q_MQ && g_use_per_node_hctx) {
if (g_submit_queues != nr_online_nodes) {
pr_warn("null_blk: submit_queues param is set to %u.\n",
#include <linux/delay.h>
#include <linux/cdrom.h>
#include <linux/spinlock.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/mutex.h>
#include <linux/uaccess.h>
static int pcd_detect(void);
static void pcd_probe_capabilities(void);
static void do_pcd_read_drq(void);
-static void do_pcd_request(struct request_queue * q);
+static blk_status_t pcd_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd);
static void do_pcd_read(void);
struct pcd_unit {
char *name; /* pcd0, pcd1, etc */
struct cdrom_device_info info; /* uniform cdrom interface */
struct gendisk *disk;
+ struct blk_mq_tag_set tag_set;
+ struct list_head rq_list;
};
static struct pcd_unit pcd[PCD_UNITS];
CDC_CD_RW,
};
+static const struct blk_mq_ops pcd_mq_ops = {
+ .queue_rq = pcd_queue_rq,
+};
+
static void pcd_init_units(void)
{
struct pcd_unit *cd;
pcd_drive_count = 0;
for (unit = 0, cd = pcd; unit < PCD_UNITS; unit++, cd++) {
struct gendisk *disk = alloc_disk(1);
+
if (!disk)
continue;
- disk->queue = blk_init_queue(do_pcd_request, &pcd_lock);
- if (!disk->queue) {
- put_disk(disk);
+
+ disk->queue = blk_mq_init_sq_queue(&cd->tag_set, &pcd_mq_ops,
+ 1, BLK_MQ_F_SHOULD_MERGE);
+ if (IS_ERR(disk->queue)) {
+ disk->queue = NULL;
continue;
}
+
+ INIT_LIST_HEAD(&cd->rq_list);
+ disk->queue->queuedata = cd;
blk_queue_bounce_limit(disk->queue, BLK_BOUNCE_HIGH);
cd->disk = disk;
cd->pi = &cd->pia;
static int set_next_request(void)
{
struct pcd_unit *cd;
- struct request_queue *q;
int old_pos = pcd_queue;
do {
cd = &pcd[pcd_queue];
- q = cd->present ? cd->disk->queue : NULL;
if (++pcd_queue == PCD_UNITS)
pcd_queue = 0;
- if (q) {
- pcd_req = blk_fetch_request(q);
- if (pcd_req)
- break;
+ if (cd->present && !list_empty(&cd->rq_list)) {
+ pcd_req = list_first_entry(&cd->rq_list, struct request,
+ queuelist);
+ list_del_init(&pcd_req->queuelist);
+ blk_mq_start_request(pcd_req);
+ break;
}
} while (pcd_queue != old_pos);
static void pcd_request(void)
{
+ struct pcd_unit *cd;
+
if (pcd_busy)
return;
- while (1) {
- if (!pcd_req && !set_next_request())
- return;
- if (rq_data_dir(pcd_req) == READ) {
- struct pcd_unit *cd = pcd_req->rq_disk->private_data;
- if (cd != pcd_current)
- pcd_bufblk = -1;
- pcd_current = cd;
- pcd_sector = blk_rq_pos(pcd_req);
- pcd_count = blk_rq_cur_sectors(pcd_req);
- pcd_buf = bio_data(pcd_req->bio);
- pcd_busy = 1;
- ps_set_intr(do_pcd_read, NULL, 0, nice);
- return;
- } else {
- __blk_end_request_all(pcd_req, BLK_STS_IOERR);
- pcd_req = NULL;
- }
- }
+ if (!pcd_req && !set_next_request())
+ return;
+
+ cd = pcd_req->rq_disk->private_data;
+ if (cd != pcd_current)
+ pcd_bufblk = -1;
+ pcd_current = cd;
+ pcd_sector = blk_rq_pos(pcd_req);
+ pcd_count = blk_rq_cur_sectors(pcd_req);
+ pcd_buf = bio_data(pcd_req->bio);
+ pcd_busy = 1;
+ ps_set_intr(do_pcd_read, NULL, 0, nice);
}
-static void do_pcd_request(struct request_queue *q)
+static blk_status_t pcd_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
+ struct pcd_unit *cd = hctx->queue->queuedata;
+
+ if (rq_data_dir(bd->rq) != READ) {
+ blk_mq_start_request(bd->rq);
+ return BLK_STS_IOERR;
+ }
+
+ spin_lock_irq(&pcd_lock);
+ list_add_tail(&bd->rq->queuelist, &cd->rq_list);
pcd_request();
+ spin_unlock_irq(&pcd_lock);
+
+ return BLK_STS_OK;
}
static inline void next_request(blk_status_t err)
unsigned long saved_flags;
spin_lock_irqsave(&pcd_lock, saved_flags);
- if (!__blk_end_request_cur(pcd_req, err))
+ if (!blk_update_request(pcd_req, err, blk_rq_cur_bytes(pcd_req))) {
+ __blk_mq_end_request(pcd_req, err);
pcd_req = NULL;
+ }
pcd_busy = 0;
pcd_request();
spin_unlock_irqrestore(&pcd_lock, saved_flags);
unregister_cdrom(&cd->info);
}
blk_cleanup_queue(cd->disk->queue);
+ blk_mq_free_tag_set(&cd->tag_set);
put_disk(cd->disk);
}
unregister_blkdev(major, name);
#include <linux/delay.h>
#include <linux/hdreg.h>
#include <linux/cdrom.h> /* for the eject ioctl */
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/blkpg.h>
#include <linux/kernel.h>
#include <linux/mutex.h>
int alt_geom;
char name[PD_NAMELEN]; /* pda, pdb, etc ... */
struct gendisk *gd;
+ struct blk_mq_tag_set tag_set;
+ struct list_head rq_list;
};
static struct pd_unit pd[PD_UNITS];
if (++pd_queue == PD_UNITS)
pd_queue = 0;
if (q) {
- pd_req = blk_fetch_request(q);
- if (pd_req)
- break;
+ struct pd_unit *disk = q->queuedata;
+
+ if (list_empty(&disk->rq_list))
+ continue;
+
+ pd_req = list_first_entry(&disk->rq_list,
+ struct request,
+ queuelist);
+ list_del_init(&pd_req->queuelist);
+ blk_mq_start_request(pd_req);
+ break;
}
} while (pd_queue != old_pos);
{
while (1) {
enum action res;
- unsigned long saved_flags;
int stop = 0;
if (!phase) {
}
switch(res = phase()) {
- case Ok: case Fail:
+ case Ok: case Fail: {
+ blk_status_t err;
+
+ err = res == Ok ? 0 : BLK_STS_IOERR;
pi_disconnect(pi_current);
pd_claimed = 0;
phase = NULL;
- spin_lock_irqsave(&pd_lock, saved_flags);
- if (!__blk_end_request_cur(pd_req,
- res == Ok ? 0 : BLK_STS_IOERR)) {
- if (!set_next_request())
- stop = 1;
+ spin_lock_irq(&pd_lock);
+ if (!blk_update_request(pd_req, err,
+ blk_rq_cur_bytes(pd_req))) {
+ __blk_mq_end_request(pd_req, err);
+ pd_req = NULL;
+ stop = !set_next_request();
}
- spin_unlock_irqrestore(&pd_lock, saved_flags);
+ spin_unlock_irq(&pd_lock);
if (stop)
return;
+ }
/* fall through */
case Hold:
schedule_fsm();
if (pd_count)
return 0;
spin_lock_irqsave(&pd_lock, saved_flags);
- __blk_end_request_cur(pd_req, 0);
- pd_count = blk_rq_cur_sectors(pd_req);
- pd_buf = bio_data(pd_req->bio);
+ if (!blk_update_request(pd_req, 0, blk_rq_cur_bytes(pd_req))) {
+ __blk_mq_end_request(pd_req, 0);
+ pd_req = NULL;
+ pd_count = 0;
+ pd_buf = NULL;
+ } else {
+ pd_count = blk_rq_cur_sectors(pd_req);
+ pd_buf = bio_data(pd_req->bio);
+ }
spin_unlock_irqrestore(&pd_lock, saved_flags);
- return 0;
+ return !pd_count;
}
static unsigned long pd_timeout;
/* end of io request engine */
-static void do_pd_request(struct request_queue * q)
+static blk_status_t pd_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
- if (pd_req)
- return;
- pd_req = blk_fetch_request(q);
- if (!pd_req)
- return;
+ struct pd_unit *disk = hctx->queue->queuedata;
+
+ spin_lock_irq(&pd_lock);
+ if (!pd_req) {
+ pd_req = bd->rq;
+ blk_mq_start_request(pd_req);
+ } else
+ list_add_tail(&bd->rq->queuelist, &disk->rq_list);
+ spin_unlock_irq(&pd_lock);
- schedule_fsm();
+ run_fsm();
+ return BLK_STS_OK;
}
static int pd_special_command(struct pd_unit *disk,
/* probing */
+static const struct blk_mq_ops pd_mq_ops = {
+ .queue_rq = pd_queue_rq,
+};
+
static void pd_probe_drive(struct pd_unit *disk)
{
- struct gendisk *p = alloc_disk(1 << PD_BITS);
+ struct gendisk *p;
+
+ p = alloc_disk(1 << PD_BITS);
if (!p)
return;
+
strcpy(p->disk_name, disk->name);
p->fops = &pd_fops;
p->major = major;
p->first_minor = (disk - pd) << PD_BITS;
disk->gd = p;
p->private_data = disk;
- p->queue = blk_init_queue(do_pd_request, &pd_lock);
- if (!p->queue) {
- disk->gd = NULL;
- put_disk(p);
+
+ p->queue = blk_mq_init_sq_queue(&disk->tag_set, &pd_mq_ops, 2,
+ BLK_MQ_F_SHOULD_MERGE | BLK_MQ_F_BLOCKING);
+ if (IS_ERR(p->queue)) {
+ p->queue = NULL;
return;
}
+
+ p->queue->queuedata = disk;
blk_queue_max_hw_sectors(p->queue, cluster);
blk_queue_bounce_limit(p->queue, BLK_BOUNCE_HIGH);
disk->standby = parm[D_SBY];
if (parm[D_PRT])
pd_drive_count++;
+ INIT_LIST_HEAD(&disk->rq_list);
}
par_drv = pi_register_driver(name);
disk->gd = NULL;
del_gendisk(p);
blk_cleanup_queue(p->queue);
+ blk_mq_free_tag_set(&disk->tag_set);
put_disk(p);
pi_release(disk->pi);
}
#include <linux/hdreg.h>
#include <linux/cdrom.h>
#include <linux/spinlock.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/blkpg.h>
#include <linux/mutex.h>
#include <linux/uaccess.h>
#define ATAPI_WRITE_10 0x2a
static int pf_open(struct block_device *bdev, fmode_t mode);
-static void do_pf_request(struct request_queue * q);
+static blk_status_t pf_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd);
static int pf_ioctl(struct block_device *bdev, fmode_t mode,
unsigned int cmd, unsigned long arg);
static int pf_getgeo(struct block_device *bdev, struct hd_geometry *geo);
int present; /* device present ? */
char name[PF_NAMELEN]; /* pf0, pf1, ... */
struct gendisk *disk;
+ struct blk_mq_tag_set tag_set;
+ struct list_head rq_list;
};
static struct pf_unit units[PF_UNITS];
.check_events = pf_check_events,
};
+static const struct blk_mq_ops pf_mq_ops = {
+ .queue_rq = pf_queue_rq,
+};
+
static void __init pf_init_units(void)
{
struct pf_unit *pf;
pf_drive_count = 0;
for (unit = 0, pf = units; unit < PF_UNITS; unit++, pf++) {
- struct gendisk *disk = alloc_disk(1);
+ struct gendisk *disk;
+
+ disk = alloc_disk(1);
if (!disk)
continue;
- disk->queue = blk_init_queue(do_pf_request, &pf_spin_lock);
- if (!disk->queue) {
+
+ disk->queue = blk_mq_init_sq_queue(&pf->tag_set, &pf_mq_ops,
+ 1, BLK_MQ_F_SHOULD_MERGE);
+ if (IS_ERR(disk->queue)) {
put_disk(disk);
- return;
+ disk->queue = NULL;
+ continue;
}
+
+ INIT_LIST_HEAD(&pf->rq_list);
+ disk->queue->queuedata = pf;
blk_queue_max_segments(disk->queue, cluster);
blk_queue_bounce_limit(disk->queue, BLK_BOUNCE_HIGH);
pf->disk = disk;
static int set_next_request(void)
{
struct pf_unit *pf;
- struct request_queue *q;
int old_pos = pf_queue;
do {
pf = &units[pf_queue];
- q = pf->present ? pf->disk->queue : NULL;
if (++pf_queue == PF_UNITS)
pf_queue = 0;
- if (q) {
- pf_req = blk_fetch_request(q);
- if (pf_req)
- break;
+ if (pf->present && !list_empty(&pf->rq_list)) {
+ pf_req = list_first_entry(&pf->rq_list, struct request,
+ queuelist);
+ list_del_init(&pf_req->queuelist);
+ blk_mq_start_request(pf_req);
+ break;
}
} while (pf_queue != old_pos);
static void pf_end_request(blk_status_t err)
{
- if (pf_req && !__blk_end_request_cur(pf_req, err))
+ if (!pf_req)
+ return;
+ if (!blk_update_request(pf_req, err, blk_rq_cur_bytes(pf_req))) {
+ __blk_mq_end_request(pf_req, err);
pf_req = NULL;
+ }
}
static void pf_request(void)
}
}
-static void do_pf_request(struct request_queue *q)
+static blk_status_t pf_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
+ struct pf_unit *pf = hctx->queue->queuedata;
+
+ spin_lock_irq(&pf_spin_lock);
+ list_add_tail(&bd->rq->queuelist, &pf->rq_list);
pf_request();
+ spin_unlock_irq(&pf_spin_lock);
+
+ return BLK_STS_OK;
}
static int pf_next_buf(void)
continue;
del_gendisk(pf->disk);
blk_cleanup_queue(pf->disk->queue);
+ blk_mq_free_tag_set(&pf->tag_set);
put_disk(pf->disk);
pi_release(pf->pi);
}
*/
if (pd->refcnt == 1)
pkt_lock_door(pd, 0);
- /* fallthru */
+ /* fall through */
/*
* forward selected CDROM ioctls to CD-ROM, for UDF
*/
*/
#include <linux/ata.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/slab.h>
#include <linux/module.h>
struct ps3disk_private {
spinlock_t lock; /* Request queue spinlock */
struct request_queue *queue;
+ struct blk_mq_tag_set tag_set;
struct gendisk *gendisk;
unsigned int blocking_factor;
struct request *req;
}
}
-static int ps3disk_submit_request_sg(struct ps3_storage_device *dev,
- struct request *req)
+static blk_status_t ps3disk_submit_request_sg(struct ps3_storage_device *dev,
+ struct request *req)
{
struct ps3disk_private *priv = ps3_system_bus_get_drvdata(&dev->sbd);
int write = rq_data_dir(req), res;
if (res) {
dev_err(&dev->sbd.core, "%s:%u: %s failed %d\n", __func__,
__LINE__, op, res);
- __blk_end_request_all(req, BLK_STS_IOERR);
- return 0;
+ return BLK_STS_IOERR;
}
priv->req = req;
- return 1;
+ return BLK_STS_OK;
}
-static int ps3disk_submit_flush_request(struct ps3_storage_device *dev,
- struct request *req)
+static blk_status_t ps3disk_submit_flush_request(struct ps3_storage_device *dev,
+ struct request *req)
{
struct ps3disk_private *priv = ps3_system_bus_get_drvdata(&dev->sbd);
u64 res;
if (res) {
dev_err(&dev->sbd.core, "%s:%u: sync cache failed 0x%llx\n",
__func__, __LINE__, res);
- __blk_end_request_all(req, BLK_STS_IOERR);
- return 0;
+ return BLK_STS_IOERR;
}
priv->req = req;
- return 1;
+ return BLK_STS_OK;
}
-static void ps3disk_do_request(struct ps3_storage_device *dev,
- struct request_queue *q)
+static blk_status_t ps3disk_do_request(struct ps3_storage_device *dev,
+ struct request *req)
{
- struct request *req;
-
dev_dbg(&dev->sbd.core, "%s:%u\n", __func__, __LINE__);
- while ((req = blk_fetch_request(q))) {
- switch (req_op(req)) {
- case REQ_OP_FLUSH:
- if (ps3disk_submit_flush_request(dev, req))
- return;
- break;
- case REQ_OP_READ:
- case REQ_OP_WRITE:
- if (ps3disk_submit_request_sg(dev, req))
- return;
- break;
- default:
- blk_dump_rq_flags(req, DEVICE_NAME " bad request");
- __blk_end_request_all(req, BLK_STS_IOERR);
- }
+ switch (req_op(req)) {
+ case REQ_OP_FLUSH:
+ return ps3disk_submit_flush_request(dev, req);
+ case REQ_OP_READ:
+ case REQ_OP_WRITE:
+ return ps3disk_submit_request_sg(dev, req);
+ default:
+ blk_dump_rq_flags(req, DEVICE_NAME " bad request");
+ return BLK_STS_IOERR;
}
}
-static void ps3disk_request(struct request_queue *q)
+static blk_status_t ps3disk_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
+ struct request_queue *q = hctx->queue;
struct ps3_storage_device *dev = q->queuedata;
struct ps3disk_private *priv = ps3_system_bus_get_drvdata(&dev->sbd);
+ blk_status_t ret;
- if (priv->req) {
- dev_dbg(&dev->sbd.core, "%s:%u busy\n", __func__, __LINE__);
- return;
- }
+ blk_mq_start_request(bd->rq);
+
+ spin_lock_irq(&priv->lock);
+ ret = ps3disk_do_request(dev, bd->rq);
+ spin_unlock_irq(&priv->lock);
- ps3disk_do_request(dev, q);
+ return ret;
}
static irqreturn_t ps3disk_interrupt(int irq, void *data)
}
spin_lock(&priv->lock);
- __blk_end_request_all(req, error);
priv->req = NULL;
- ps3disk_do_request(dev, priv->queue);
+ blk_mq_end_request(req, error);
spin_unlock(&priv->lock);
+ blk_mq_run_hw_queues(priv->queue, true);
return IRQ_HANDLED;
}
static DEFINE_MUTEX(ps3disk_mask_mutex);
+static const struct blk_mq_ops ps3disk_mq_ops = {
+ .queue_rq = ps3disk_queue_rq,
+};
+
static int ps3disk_probe(struct ps3_system_bus_device *_dev)
{
struct ps3_storage_device *dev = to_ps3_storage_device(&_dev->core);
ps3disk_identify(dev);
- queue = blk_init_queue(ps3disk_request, &priv->lock);
- if (!queue) {
- dev_err(&dev->sbd.core, "%s:%u: blk_init_queue failed\n",
+ queue = blk_mq_init_sq_queue(&priv->tag_set, &ps3disk_mq_ops, 1,
+ BLK_MQ_F_SHOULD_MERGE);
+ if (IS_ERR(queue)) {
+ dev_err(&dev->sbd.core, "%s:%u: blk_mq_init_queue failed\n",
__func__, __LINE__);
- error = -ENOMEM;
+ error = PTR_ERR(queue);
goto fail_teardown;
}
gendisk->disk_name, priv->model, priv->raw_capacity >> 11,
get_capacity(gendisk) >> 11);
- device_add_disk(&dev->sbd.core, gendisk);
+ device_add_disk(&dev->sbd.core, gendisk, NULL);
return 0;
fail_cleanup_queue:
blk_cleanup_queue(queue);
+ blk_mq_free_tag_set(&priv->tag_set);
fail_teardown:
ps3stor_teardown(dev);
fail_free_bounce:
mutex_unlock(&ps3disk_mask_mutex);
del_gendisk(priv->gendisk);
blk_cleanup_queue(priv->queue);
+ blk_mq_free_tag_set(&priv->tag_set);
put_disk(priv->gendisk);
dev_notice(&dev->sbd.core, "Synchronizing disk cache\n");
ps3disk_sync_cache(dev);
dev_info(&dev->core, "%s: Using %lu MiB of GPU memory\n",
gendisk->disk_name, get_capacity(gendisk) >> 11);
- device_add_disk(&dev->core, gendisk);
+ device_add_disk(&dev->core, gendisk, NULL);
return 0;
fail_cleanup_queue:
pci_set_master(dev);
pci_set_dma_max_seg_size(dev, RSXX_HW_BLK_SIZE);
- st = pci_set_dma_mask(dev, DMA_BIT_MASK(64));
+ st = dma_set_mask(&dev->dev, DMA_BIT_MASK(64));
if (st) {
dev_err(CARD_TO_DEV(card),
"No usable DMA configuration,aborting\n");
st = -EIO;
}
- if ((cmd->op == CREG_OP_READ)) {
+ if (cmd->op == CREG_OP_READ) {
unsigned int cnt8 = ioread32(card->regmap + CREG_CNT);
/* Paranoid Sanity Checks */
set_capacity(card->gendisk, card->size8 >> 9);
else
set_capacity(card->gendisk, 0);
- device_add_disk(CARD_TO_DEV(card), card->gendisk);
+ device_add_disk(CARD_TO_DEV(card), card->gendisk, NULL);
card->bdev_attached = 1;
}
static void rsxx_free_dma(struct rsxx_dma_ctrl *ctrl, struct rsxx_dma *dma)
{
if (dma->cmd != HW_CMD_BLK_DISCARD) {
- if (!pci_dma_mapping_error(ctrl->card->dev, dma->dma_addr)) {
- pci_unmap_page(ctrl->card->dev, dma->dma_addr,
+ if (!dma_mapping_error(&ctrl->card->dev->dev, dma->dma_addr)) {
+ dma_unmap_page(&ctrl->card->dev->dev, dma->dma_addr,
get_dma_size(dma),
dma->cmd == HW_CMD_BLK_WRITE ?
- PCI_DMA_TODEVICE :
- PCI_DMA_FROMDEVICE);
+ DMA_TO_DEVICE :
+ DMA_FROM_DEVICE);
}
}
if (dma->cmd != HW_CMD_BLK_DISCARD) {
if (dma->cmd == HW_CMD_BLK_WRITE)
- dir = PCI_DMA_TODEVICE;
+ dir = DMA_TO_DEVICE;
else
- dir = PCI_DMA_FROMDEVICE;
+ dir = DMA_FROM_DEVICE;
/*
- * The function pci_map_page is placed here because we
+ * The function dma_map_page is placed here because we
* can only, by design, issue up to 255 commands to the
* hardware at one time per DMA channel. So the maximum
* amount of mapped memory would be 255 * 4 channels *
* 4096 Bytes which is less than 2GB, the limit of a x8
- * Non-HWWD PCIe slot. This way the pci_map_page
+ * Non-HWWD PCIe slot. This way the dma_map_page
* function should never fail because of a lack of
* mappable memory.
*/
- dma->dma_addr = pci_map_page(ctrl->card->dev, dma->page,
+ dma->dma_addr = dma_map_page(&ctrl->card->dev->dev, dma->page,
dma->pg_off, dma->sub_page.cnt << 9, dir);
- if (pci_dma_mapping_error(ctrl->card->dev, dma->dma_addr)) {
+ if (dma_mapping_error(&ctrl->card->dev->dev, dma->dma_addr)) {
push_tracker(ctrl->trackers, tag);
rsxx_complete_dma(ctrl, dma, DMA_CANCELLED);
continue;
/*----------------- DMA Engine Initialization & Setup -------------------*/
int rsxx_hw_buffers_init(struct pci_dev *dev, struct rsxx_dma_ctrl *ctrl)
{
- ctrl->status.buf = pci_alloc_consistent(dev, STATUS_BUFFER_SIZE8,
- &ctrl->status.dma_addr);
- ctrl->cmd.buf = pci_alloc_consistent(dev, COMMAND_BUFFER_SIZE8,
- &ctrl->cmd.dma_addr);
+ ctrl->status.buf = dma_alloc_coherent(&dev->dev, STATUS_BUFFER_SIZE8,
+ &ctrl->status.dma_addr, GFP_KERNEL);
+ ctrl->cmd.buf = dma_alloc_coherent(&dev->dev, COMMAND_BUFFER_SIZE8,
+ &ctrl->cmd.dma_addr, GFP_KERNEL);
if (ctrl->status.buf == NULL || ctrl->cmd.buf == NULL)
return -ENOMEM;
vfree(ctrl->trackers);
if (ctrl->status.buf)
- pci_free_consistent(card->dev, STATUS_BUFFER_SIZE8,
- ctrl->status.buf,
- ctrl->status.dma_addr);
+ dma_free_coherent(&card->dev->dev, STATUS_BUFFER_SIZE8,
+ ctrl->status.buf,
+ ctrl->status.dma_addr);
if (ctrl->cmd.buf)
- pci_free_consistent(card->dev, COMMAND_BUFFER_SIZE8,
- ctrl->cmd.buf, ctrl->cmd.dma_addr);
+ dma_free_coherent(&card->dev->dev, COMMAND_BUFFER_SIZE8,
+ ctrl->cmd.buf, ctrl->cmd.dma_addr);
}
return st;
vfree(ctrl->trackers);
- pci_free_consistent(card->dev, STATUS_BUFFER_SIZE8,
- ctrl->status.buf, ctrl->status.dma_addr);
- pci_free_consistent(card->dev, COMMAND_BUFFER_SIZE8,
- ctrl->cmd.buf, ctrl->cmd.dma_addr);
+ dma_free_coherent(&card->dev->dev, STATUS_BUFFER_SIZE8,
+ ctrl->status.buf, ctrl->status.dma_addr);
+ dma_free_coherent(&card->dev->dev, COMMAND_BUFFER_SIZE8,
+ ctrl->cmd.buf, ctrl->cmd.dma_addr);
}
}
card->ctrl[i].stats.reads_issued--;
if (dma->cmd != HW_CMD_BLK_DISCARD) {
- pci_unmap_page(card->dev, dma->dma_addr,
+ dma_unmap_page(&card->dev->dev, dma->dma_addr,
get_dma_size(dma),
dma->cmd == HW_CMD_BLK_WRITE ?
- PCI_DMA_TODEVICE :
- PCI_DMA_FROMDEVICE);
+ DMA_TO_DEVICE :
+ DMA_FROM_DEVICE);
}
list_add_tail(&dma->list, &issued_dmas[i]);
* Map scatterlist to PCI bus addresses.
* Note PCI might change the number of entries.
*/
- n_sg = pci_map_sg(skdev->pdev, sgl, n_sg, skreq->data_dir);
+ n_sg = dma_map_sg(&skdev->pdev->dev, sgl, n_sg, skreq->data_dir);
if (n_sg <= 0)
return false;
skreq->sksg_list[skreq->n_sg - 1].next_desc_ptr =
skreq->sksg_dma_address +
((skreq->n_sg) * sizeof(struct fit_sg_descriptor));
- pci_unmap_sg(skdev->pdev, &skreq->sg[0], skreq->n_sg, skreq->data_dir);
+ dma_unmap_sg(&skdev->pdev->dev, &skreq->sg[0], skreq->n_sg,
+ skreq->data_dir);
}
/*
case SKD_CHECK_STATUS_BUSY_IMMINENT:
skd_log_skreq(skdev, skreq, "retry(busy)");
- blk_requeue_request(skdev->queue, req);
+ blk_mq_requeue_request(req, true);
dev_info(&skdev->pdev->dev, "drive BUSY imminent\n");
skdev->state = SKD_DRVR_STATE_BUSY_IMMINENT;
skdev->timer_countdown = SKD_TIMER_MINUTES(20);
case SKD_CHECK_STATUS_REQUEUE_REQUEST:
if ((unsigned long) ++req->special < SKD_MAX_RETRIES) {
skd_log_skreq(skdev, skreq, "retry");
- blk_requeue_request(skdev->queue, req);
+ blk_mq_requeue_request(req, true);
break;
}
/* fall through */
"comp pci_alloc, total bytes %zd entries %d\n",
SKD_SKCOMP_SIZE, SKD_N_COMPLETION_ENTRY);
- skcomp = pci_zalloc_consistent(skdev->pdev, SKD_SKCOMP_SIZE,
- &skdev->cq_dma_address);
+ skcomp = dma_zalloc_coherent(&skdev->pdev->dev, SKD_SKCOMP_SIZE,
+ &skdev->cq_dma_address, GFP_KERNEL);
if (skcomp == NULL) {
rc = -ENOMEM;
skmsg->id = i + SKD_ID_FIT_MSG;
- skmsg->msg_buf = pci_alloc_consistent(skdev->pdev,
- SKD_N_FITMSG_BYTES,
- &skmsg->mb_dma_address);
-
+ skmsg->msg_buf = dma_alloc_coherent(&skdev->pdev->dev,
+ SKD_N_FITMSG_BYTES,
+ &skmsg->mb_dma_address,
+ GFP_KERNEL);
if (skmsg->msg_buf == NULL) {
rc = -ENOMEM;
goto err_out;
static void skd_free_skcomp(struct skd_device *skdev)
{
if (skdev->skcomp_table)
- pci_free_consistent(skdev->pdev, SKD_SKCOMP_SIZE,
- skdev->skcomp_table, skdev->cq_dma_address);
+ dma_free_coherent(&skdev->pdev->dev, SKD_SKCOMP_SIZE,
+ skdev->skcomp_table, skdev->cq_dma_address);
skdev->skcomp_table = NULL;
skdev->cq_dma_address = 0;
skmsg = &skdev->skmsg_table[i];
if (skmsg->msg_buf != NULL) {
- pci_free_consistent(skdev->pdev, SKD_N_FITMSG_BYTES,
- skmsg->msg_buf,
+ dma_free_coherent(&skdev->pdev->dev, SKD_N_FITMSG_BYTES,
+ skmsg->msg_buf,
skmsg->mb_dma_address);
}
skmsg->msg_buf = NULL;
static int skd_bdev_attach(struct device *parent, struct skd_device *skdev)
{
dev_dbg(&skdev->pdev->dev, "add_disk\n");
- device_add_disk(parent, skdev->disk);
+ device_add_disk(parent, skdev->disk, NULL);
return 0;
}
rc = pci_request_regions(pdev, DRV_NAME);
if (rc)
goto err_out;
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
- if (!rc) {
- if (pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64))) {
- dev_err(&pdev->dev, "consistent DMA mask error %d\n",
- rc);
- }
- } else {
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
- if (rc) {
- dev_err(&pdev->dev, "DMA mask error %d\n", rc);
- goto err_out_regions;
- }
+ rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (rc)
+ dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ if (rc) {
+ dev_err(&pdev->dev, "DMA mask error %d\n", rc);
+ goto err_out_regions;
}
if (!skd_major) {
rc = pci_request_regions(pdev, DRV_NAME);
if (rc)
goto err_out;
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
- if (!rc) {
- if (pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64))) {
-
- dev_err(&pdev->dev, "consistent DMA mask error %d\n",
- rc);
- }
- } else {
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
- if (rc) {
-
- dev_err(&pdev->dev, "DMA mask error %d\n", rc);
- goto err_out_regions;
- }
+ rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (rc)
+ dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ if (rc) {
+ dev_err(&pdev->dev, "DMA mask error %d\n", rc);
+ goto err_out_regions;
}
pci_set_master(pdev);
#define VDC_TX_RING_SIZE 512
#define VDC_DEFAULT_BLK_SIZE 512
+#define MAX_XFER_BLKS (128 * 1024)
+#define MAX_XFER_SIZE (MAX_XFER_BLKS / VDC_DEFAULT_BLK_SIZE)
+#define MAX_RING_COOKIES ((MAX_XFER_BLKS / PAGE_SIZE) + 2)
+
#define WAITING_FOR_LINK_UP 0x01
#define WAITING_FOR_TX_SPACE 0x02
#define WAITING_FOR_GEN_CMD 0x04
{
struct vdc_port *port = req->rq_disk->private_data;
struct vio_dring_state *dr = &port->vio.drings[VIO_DRIVER_TX_RING];
- struct scatterlist sg[port->ring_cookies];
+ struct scatterlist sg[MAX_RING_COOKIES];
struct vdc_req_entry *rqe;
struct vio_disk_desc *desc;
unsigned int map_perm;
u64 len;
u8 op;
+ if (WARN_ON(port->ring_cookies > MAX_RING_COOKIES))
+ return -EINVAL;
+
map_perm = LDC_MAP_SHADOW | LDC_MAP_DIRECT | LDC_MAP_IO;
if (rq_data_dir(req) == READ) {
port->vdisk_size, (port->vdisk_size >> (20 - 9)),
port->vio.ver.major, port->vio.ver.minor);
- device_add_disk(&port->vio.vdev->dev, g);
+ device_add_disk(&port->vio.vdev->dev, g, NULL);
return 0;
}
goto err_out_free_port;
port->vdisk_block_size = VDC_DEFAULT_BLK_SIZE;
- port->max_xfer_size = ((128 * 1024) / port->vdisk_block_size);
- port->ring_cookies = ((port->max_xfer_size *
- port->vdisk_block_size) / PAGE_SIZE) + 2;
+ port->max_xfer_size = MAX_XFER_SIZE;
+ port->ring_cookies = MAX_RING_COOKIES;
err = vio_ldc_alloc(&port->vio, &vdc_ldc_cfg, port);
if (err)
#include <linux/module.h>
#include <linux/fd.h>
#include <linux/slab.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/mutex.h>
#include <linux/hdreg.h>
#include <linux/kernel.h>
int ref_count;
struct gendisk *disk;
+ struct blk_mq_tag_set tag_set;
/* parent controller */
struct swim_priv {
struct swim __iomem *base;
spinlock_t lock;
- int fdc_queue;
int floppy_count;
struct floppy_state unit[FD_MAX_UNIT];
};
return 0;
}
-static struct request *swim_next_request(struct swim_priv *swd)
+static blk_status_t swim_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
- struct request_queue *q;
- struct request *rq;
- int old_pos = swd->fdc_queue;
+ struct floppy_state *fs = hctx->queue->queuedata;
+ struct swim_priv *swd = fs->swd;
+ struct request *req = bd->rq;
+ blk_status_t err;
- do {
- q = swd->unit[swd->fdc_queue].disk->queue;
- if (++swd->fdc_queue == swd->floppy_count)
- swd->fdc_queue = 0;
- if (q) {
- rq = blk_fetch_request(q);
- if (rq)
- return rq;
- }
- } while (swd->fdc_queue != old_pos);
+ if (!spin_trylock_irq(&swd->lock))
+ return BLK_STS_DEV_RESOURCE;
- return NULL;
-}
+ blk_mq_start_request(req);
-static void do_fd_request(struct request_queue *q)
-{
- struct swim_priv *swd = q->queuedata;
- struct request *req;
- struct floppy_state *fs;
+ if (!fs->disk_in || rq_data_dir(req) == WRITE) {
+ err = BLK_STS_IOERR;
+ goto out;
+ }
- req = swim_next_request(swd);
- while (req) {
- blk_status_t err = BLK_STS_IOERR;
+ do {
+ err = floppy_read_sectors(fs, blk_rq_pos(req),
+ blk_rq_cur_sectors(req),
+ bio_data(req->bio));
+ } while (blk_update_request(req, err, blk_rq_cur_bytes(req)));
+ __blk_mq_end_request(req, err);
- fs = req->rq_disk->private_data;
- if (blk_rq_pos(req) >= fs->total_secs)
- goto done;
- if (!fs->disk_in)
- goto done;
- if (rq_data_dir(req) == WRITE && fs->write_protected)
- goto done;
+ err = BLK_STS_OK;
+out:
+ spin_unlock_irq(&swd->lock);
+ return err;
- switch (rq_data_dir(req)) {
- case WRITE:
- /* NOT IMPLEMENTED */
- break;
- case READ:
- err = floppy_read_sectors(fs, blk_rq_pos(req),
- blk_rq_cur_sectors(req),
- bio_data(req->bio));
- break;
- }
- done:
- if (!__blk_end_request_cur(req, err))
- req = swim_next_request(swd);
- }
}
static struct floppy_struct floppy_type[4] = {
return 0;
}
+static const struct blk_mq_ops swim_mq_ops = {
+ .queue_rq = swim_queue_rq,
+};
+
static int swim_floppy_init(struct swim_priv *swd)
{
int err;
spin_lock_init(&swd->lock);
for (drive = 0; drive < swd->floppy_count; drive++) {
+ struct request_queue *q;
+
swd->unit[drive].disk = alloc_disk(1);
if (swd->unit[drive].disk == NULL) {
err = -ENOMEM;
goto exit_put_disks;
}
- swd->unit[drive].disk->queue = blk_init_queue(do_fd_request,
- &swd->lock);
- if (!swd->unit[drive].disk->queue) {
- err = -ENOMEM;
+
+ q = blk_mq_init_sq_queue(&swd->unit[drive].tag_set, &swim_mq_ops,
+ 2, BLK_MQ_F_SHOULD_MERGE);
+ if (IS_ERR(q)) {
+ err = PTR_ERR(q);
goto exit_put_disks;
}
+
+ swd->unit[drive].disk->queue = q;
blk_queue_bounce_limit(swd->unit[drive].disk->queue,
BLK_BOUNCE_HIGH);
- swd->unit[drive].disk->queue->queuedata = swd;
+ swd->unit[drive].disk->queue->queuedata = &swd->unit[drive];
swd->unit[drive].swd = swd;
}
exit_put_disks:
unregister_blkdev(FLOPPY_MAJOR, "fd");
- while (drive--)
- put_disk(swd->unit[drive].disk);
+ do {
+ struct gendisk *disk = swd->unit[drive].disk;
+
+ if (disk) {
+ if (disk->queue) {
+ blk_cleanup_queue(disk->queue);
+ disk->queue = NULL;
+ }
+ blk_mq_free_tag_set(&swd->unit[drive].tag_set);
+ put_disk(disk);
+ }
+ } while (drive--);
return err;
}
for (drive = 0; drive < swd->floppy_count; drive++) {
del_gendisk(swd->unit[drive].disk);
blk_cleanup_queue(swd->unit[drive].disk->queue);
+ blk_mq_free_tag_set(&swd->unit[drive].tag_set);
put_disk(swd->unit[drive].disk);
}
#include <linux/delay.h>
#include <linux/fd.h>
#include <linux/ioctl.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/interrupt.h>
#include <linux/mutex.h>
#include <linux/module.h>
char dbdma_cmd_space[5 * sizeof(struct dbdma_cmd)];
int index;
struct request *cur_req;
+ struct blk_mq_tag_set tag_set;
};
#define swim3_err(fmt, arg...) dev_err(&fs->mdev->ofdev.dev, "[fd%d] " fmt, fs->index, arg)
static bool swim3_end_request(struct floppy_state *fs, blk_status_t err, unsigned int nr_bytes)
{
struct request *req = fs->cur_req;
- int rc;
swim3_dbg(" end request, err=%d nr_bytes=%d, cur_req=%p\n",
err, nr_bytes, req);
if (err)
nr_bytes = blk_rq_cur_bytes(req);
- rc = __blk_end_request(req, err, nr_bytes);
- if (rc)
+ if (blk_update_request(req, err, nr_bytes))
return true;
+ __blk_mq_end_request(req, err);
fs->cur_req = NULL;
return false;
}
return (stat & DATA) == 0;
}
-static void start_request(struct floppy_state *fs)
+static blk_status_t swim3_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
- struct request *req;
+ struct floppy_state *fs = hctx->queue->queuedata;
+ struct request *req = bd->rq;
unsigned long x;
- swim3_dbg("start request, initial state=%d\n", fs->state);
-
- if (fs->state == idle && fs->wanted) {
- fs->state = available;
- wake_up(&fs->wait);
- return;
+ spin_lock_irq(&swim3_lock);
+ if (fs->cur_req || fs->state != idle) {
+ spin_unlock_irq(&swim3_lock);
+ return BLK_STS_DEV_RESOURCE;
}
- while (fs->state == idle) {
- swim3_dbg("start request, idle loop, cur_req=%p\n", fs->cur_req);
- if (!fs->cur_req) {
- fs->cur_req = blk_fetch_request(disks[fs->index]->queue);
- swim3_dbg(" fetched request %p\n", fs->cur_req);
- if (!fs->cur_req)
- break;
- }
- req = fs->cur_req;
-
- if (fs->mdev->media_bay &&
- check_media_bay(fs->mdev->media_bay) != MB_FD) {
- swim3_dbg("%s", " media bay absent, dropping req\n");
- swim3_end_request(fs, BLK_STS_IOERR, 0);
- continue;
- }
-
-#if 0 /* This is really too verbose */
- swim3_dbg("do_fd_req: dev=%s cmd=%d sec=%ld nr_sec=%u buf=%p\n",
- req->rq_disk->disk_name, req->cmd,
- (long)blk_rq_pos(req), blk_rq_sectors(req),
- bio_data(req->bio));
- swim3_dbg(" current_nr_sectors=%u\n",
- blk_rq_cur_sectors(req));
-#endif
-
- if (blk_rq_pos(req) >= fs->total_secs) {
- swim3_dbg(" pos out of bounds (%ld, max is %ld)\n",
- (long)blk_rq_pos(req), (long)fs->total_secs);
- swim3_end_request(fs, BLK_STS_IOERR, 0);
- continue;
- }
- if (fs->ejected) {
- swim3_dbg("%s", " disk ejected\n");
+ blk_mq_start_request(req);
+ fs->cur_req = req;
+ if (fs->mdev->media_bay &&
+ check_media_bay(fs->mdev->media_bay) != MB_FD) {
+ swim3_dbg("%s", " media bay absent, dropping req\n");
+ swim3_end_request(fs, BLK_STS_IOERR, 0);
+ goto out;
+ }
+ if (fs->ejected) {
+ swim3_dbg("%s", " disk ejected\n");
+ swim3_end_request(fs, BLK_STS_IOERR, 0);
+ goto out;
+ }
+ if (rq_data_dir(req) == WRITE) {
+ if (fs->write_prot < 0)
+ fs->write_prot = swim3_readbit(fs, WRITE_PROT);
+ if (fs->write_prot) {
+ swim3_dbg("%s", " try to write, disk write protected\n");
swim3_end_request(fs, BLK_STS_IOERR, 0);
- continue;
+ goto out;
}
-
- if (rq_data_dir(req) == WRITE) {
- if (fs->write_prot < 0)
- fs->write_prot = swim3_readbit(fs, WRITE_PROT);
- if (fs->write_prot) {
- swim3_dbg("%s", " try to write, disk write protected\n");
- swim3_end_request(fs, BLK_STS_IOERR, 0);
- continue;
- }
- }
-
- /* Do not remove the cast. blk_rq_pos(req) is now a
- * sector_t and can be 64 bits, but it will never go
- * past 32 bits for this driver anyway, so we can
- * safely cast it down and not have to do a 64/32
- * division
- */
- fs->req_cyl = ((long)blk_rq_pos(req)) / fs->secpercyl;
- x = ((long)blk_rq_pos(req)) % fs->secpercyl;
- fs->head = x / fs->secpertrack;
- fs->req_sector = x % fs->secpertrack + 1;
- fs->state = do_transfer;
- fs->retries = 0;
-
- act(fs);
}
-}
-static void do_fd_request(struct request_queue * q)
-{
- start_request(q->queuedata);
+ /*
+ * Do not remove the cast. blk_rq_pos(req) is now a sector_t and can be
+ * 64 bits, but it will never go past 32 bits for this driver anyway, so
+ * we can safely cast it down and not have to do a 64/32 division
+ */
+ fs->req_cyl = ((long)blk_rq_pos(req)) / fs->secpercyl;
+ x = ((long)blk_rq_pos(req)) % fs->secpercyl;
+ fs->head = x / fs->secpertrack;
+ fs->req_sector = x % fs->secpertrack + 1;
+ fs->state = do_transfer;
+ fs->retries = 0;
+
+ act(fs);
+
+out:
+ spin_unlock_irq(&swim3_lock);
+ return BLK_STS_OK;
}
static void set_timeout(struct floppy_state *fs, int nticks,
if (fs->retries > 5) {
swim3_end_request(fs, BLK_STS_IOERR, 0);
fs->state = idle;
- start_request(fs);
} else {
fs->state = jogging;
act(fs);
swim3_err("%s", "Seek timeout\n");
swim3_end_request(fs, BLK_STS_IOERR, 0);
fs->state = idle;
- start_request(fs);
spin_unlock_irqrestore(&swim3_lock, flags);
}
swim3_err("%s", "Seek settle timeout\n");
swim3_end_request(fs, BLK_STS_IOERR, 0);
fs->state = idle;
- start_request(fs);
unlock:
spin_unlock_irqrestore(&swim3_lock, flags);
}
(long)blk_rq_pos(fs->cur_req));
swim3_end_request(fs, BLK_STS_IOERR, 0);
fs->state = idle;
- start_request(fs);
spin_unlock_irqrestore(&swim3_lock, flags);
}
if (fs->retries > 5) {
swim3_end_request(fs, BLK_STS_IOERR, 0);
fs->state = idle;
- start_request(fs);
} else {
fs->state = jogging;
act(fs);
fs->state, rq_data_dir(req), intr, err);
swim3_end_request(fs, BLK_STS_IOERR, 0);
fs->state = idle;
- start_request(fs);
break;
}
fs->retries = 0;
} else
fs->state = idle;
}
- if (fs->state == idle)
- start_request(fs);
break;
default:
swim3_err("Don't know what to do in state %d\n", fs->state);
static void release_drive(struct floppy_state *fs)
{
+ struct request_queue *q = disks[fs->index]->queue;
unsigned long flags;
swim3_dbg("%s", "-> release drive\n");
spin_lock_irqsave(&swim3_lock, flags);
fs->state = idle;
- start_request(fs);
spin_unlock_irqrestore(&swim3_lock, flags);
+
+ blk_mq_freeze_queue(q);
+ blk_mq_quiesce_queue(q);
+ blk_mq_unquiesce_queue(q);
+ blk_mq_unfreeze_queue(q);
}
static int fd_eject(struct floppy_state *fs)
.revalidate_disk= floppy_revalidate,
};
+static const struct blk_mq_ops swim3_mq_ops = {
+ .queue_rq = swim3_queue_rq,
+};
+
static void swim3_mb_event(struct macio_dev* mdev, int mb_state)
{
struct floppy_state *fs = macio_get_drvdata(mdev);
static int swim3_attach(struct macio_dev *mdev,
const struct of_device_id *match)
{
+ struct floppy_state *fs;
struct gendisk *disk;
- int index, rc;
+ int rc;
- index = floppy_count++;
- if (index >= MAX_FLOPPIES)
+ if (floppy_count >= MAX_FLOPPIES)
return -ENXIO;
- /* Add the drive */
- rc = swim3_add_device(mdev, index);
- if (rc)
- return rc;
- /* Now register that disk. Same comment about failure handling */
- disk = disks[index] = alloc_disk(1);
- if (disk == NULL)
- return -ENOMEM;
- disk->queue = blk_init_queue(do_fd_request, &swim3_lock);
- if (disk->queue == NULL) {
- put_disk(disk);
- return -ENOMEM;
+ if (floppy_count == 0) {
+ rc = register_blkdev(FLOPPY_MAJOR, "fd");
+ if (rc)
+ return rc;
}
- blk_queue_bounce_limit(disk->queue, BLK_BOUNCE_HIGH);
- disk->queue->queuedata = &floppy_states[index];
- if (index == 0) {
- /* If we failed, there isn't much we can do as the driver is still
- * too dumb to remove the device, just bail out
- */
- if (register_blkdev(FLOPPY_MAJOR, "fd"))
- return 0;
+ fs = &floppy_states[floppy_count];
+
+ disk = alloc_disk(1);
+ if (disk == NULL) {
+ rc = -ENOMEM;
+ goto out_unregister;
+ }
+
+ disk->queue = blk_mq_init_sq_queue(&fs->tag_set, &swim3_mq_ops, 2,
+ BLK_MQ_F_SHOULD_MERGE);
+ if (IS_ERR(disk->queue)) {
+ rc = PTR_ERR(disk->queue);
+ disk->queue = NULL;
+ goto out_put_disk;
}
+ blk_queue_bounce_limit(disk->queue, BLK_BOUNCE_HIGH);
+ disk->queue->queuedata = fs;
+
+ rc = swim3_add_device(mdev, floppy_count);
+ if (rc)
+ goto out_cleanup_queue;
disk->major = FLOPPY_MAJOR;
- disk->first_minor = index;
+ disk->first_minor = floppy_count;
disk->fops = &floppy_fops;
- disk->private_data = &floppy_states[index];
+ disk->private_data = fs;
disk->flags |= GENHD_FL_REMOVABLE;
- sprintf(disk->disk_name, "fd%d", index);
+ sprintf(disk->disk_name, "fd%d", floppy_count);
set_capacity(disk, 2880);
add_disk(disk);
+ disks[floppy_count++] = disk;
return 0;
+
+out_cleanup_queue:
+ blk_cleanup_queue(disk->queue);
+ disk->queue = NULL;
+ blk_mq_free_tag_set(&fs->tag_set);
+out_put_disk:
+ put_disk(disk);
+out_unregister:
+ if (floppy_count == 0)
+ unregister_blkdev(FLOPPY_MAJOR, "fd");
+ return rc;
}
static const struct of_device_id swim3_match[] =
#include <linux/pci.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/sched.h>
#include <linux/interrupt.h>
#include <linux/compiler.h>
FL_NON_RAID = FW_VER_NON_RAID,
FL_4PORT = FW_VER_4PORT,
FL_FW_VER_MASK = (FW_VER_NON_RAID | FW_VER_4PORT),
- FL_DAC = (1 << 16),
FL_DYN_MAJOR = (1 << 17),
};
unsigned int port_no;
struct gendisk *disk;
struct carm_host *host;
+ struct blk_mq_tag_set tag_set;
/* attached device characteristics */
u64 capacity;
unsigned int state;
u32 fw_ver;
+ struct blk_mq_tag_set tag_set;
struct request_queue *oob_q;
unsigned int n_oob;
struct request *req = crq->rq;
int rc;
- __blk_end_request_all(req, error);
+ blk_mq_end_request(req, error);
rc = carm_put_request(host, crq);
assert(rc == 0);
{
unsigned int idx = host->wait_q_prod % CARM_MAX_WAIT_Q;
- blk_stop_queue(q);
+ blk_mq_stop_hw_queues(q);
VPRINTK("STOPPED QUEUE %p\n", q);
host->wait_q[idx] = q;
{
struct request_queue *q = carm_pop_q(host);
if (q) {
- blk_start_queue(q);
+ blk_mq_start_hw_queues(q);
VPRINTK("STARTED QUEUE %p\n", q);
}
}
}
}
-static void carm_oob_rq_fn(struct request_queue *q)
+static blk_status_t carm_oob_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
+ struct request_queue *q = hctx->queue;
struct carm_host *host = q->queuedata;
struct carm_request *crq;
- struct request *rq;
int rc;
- while (1) {
- DPRINTK("get req\n");
- rq = blk_fetch_request(q);
- if (!rq)
- break;
+ blk_mq_start_request(bd->rq);
- crq = rq->special;
- assert(crq != NULL);
- assert(crq->rq == rq);
+ spin_lock_irq(&host->lock);
- crq->n_elem = 0;
+ crq = bd->rq->special;
+ assert(crq != NULL);
+ assert(crq->rq == bd->rq);
- DPRINTK("send req\n");
- rc = carm_send_msg(host, crq);
- if (rc) {
- blk_requeue_request(q, rq);
- carm_push_q(host, q);
- return; /* call us again later, eventually */
- }
+ crq->n_elem = 0;
+
+ DPRINTK("send req\n");
+ rc = carm_send_msg(host, crq);
+ if (rc) {
+ carm_push_q(host, q);
+ spin_unlock_irq(&host->lock);
+ return BLK_STS_DEV_RESOURCE;
}
+
+ spin_unlock_irq(&host->lock);
+ return BLK_STS_OK;
}
-static void carm_rq_fn(struct request_queue *q)
+static blk_status_t carm_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
+ struct request_queue *q = hctx->queue;
struct carm_port *port = q->queuedata;
struct carm_host *host = port->host;
struct carm_msg_rw *msg;
struct carm_request *crq;
- struct request *rq;
+ struct request *rq = bd->rq;
struct scatterlist *sg;
int writing = 0, pci_dir, i, n_elem, rc;
u32 tmp;
unsigned int msg_size;
-queue_one_request:
- VPRINTK("get req\n");
- rq = blk_peek_request(q);
- if (!rq)
- return;
+ blk_mq_start_request(rq);
+
+ spin_lock_irq(&host->lock);
crq = carm_get_request(host);
if (!crq) {
carm_push_q(host, q);
- return; /* call us again later, eventually */
+ spin_unlock_irq(&host->lock);
+ return BLK_STS_DEV_RESOURCE;
}
crq->rq = rq;
- blk_start_request(rq);
-
if (rq_data_dir(rq) == WRITE) {
writing = 1;
- pci_dir = PCI_DMA_TODEVICE;
+ pci_dir = DMA_TO_DEVICE;
} else {
- pci_dir = PCI_DMA_FROMDEVICE;
+ pci_dir = DMA_FROM_DEVICE;
}
/* get scatterlist from block layer */
sg = &crq->sg[0];
n_elem = blk_rq_map_sg(q, rq, sg);
if (n_elem <= 0) {
+ /* request with no s/g entries? */
carm_end_rq(host, crq, BLK_STS_IOERR);
- return; /* request with no s/g entries? */
+ spin_unlock_irq(&host->lock);
+ return BLK_STS_IOERR;
}
/* map scatterlist to PCI bus addresses */
- n_elem = pci_map_sg(host->pdev, sg, n_elem, pci_dir);
+ n_elem = dma_map_sg(&host->pdev->dev, sg, n_elem, pci_dir);
if (n_elem <= 0) {
+ /* request with no s/g entries? */
carm_end_rq(host, crq, BLK_STS_IOERR);
- return; /* request with no s/g entries? */
+ spin_unlock_irq(&host->lock);
+ return BLK_STS_IOERR;
}
crq->n_elem = n_elem;
crq->port = port;
rc = carm_send_msg(host, crq);
if (rc) {
carm_put_request(host, crq);
- blk_requeue_request(q, rq);
carm_push_q(host, q);
- return; /* call us again later, eventually */
+ spin_unlock_irq(&host->lock);
+ return BLK_STS_DEV_RESOURCE;
}
- goto queue_one_request;
+ spin_unlock_irq(&host->lock);
+ return BLK_STS_OK;
}
static void carm_handle_array_info(struct carm_host *host,
VPRINTK("ENTER\n");
if (rq_data_dir(crq->rq) == WRITE)
- pci_dir = PCI_DMA_TODEVICE;
+ pci_dir = DMA_TO_DEVICE;
else
- pci_dir = PCI_DMA_FROMDEVICE;
+ pci_dir = DMA_FROM_DEVICE;
- pci_unmap_sg(host->pdev, &crq->sg[0], crq->n_elem, pci_dir);
+ dma_unmap_sg(&host->pdev->dev, &crq->sg[0], crq->n_elem, pci_dir);
carm_end_rq(host, crq, error);
}
return 0;
}
+static const struct blk_mq_ops carm_oob_mq_ops = {
+ .queue_rq = carm_oob_queue_rq,
+};
+
+static const struct blk_mq_ops carm_mq_ops = {
+ .queue_rq = carm_queue_rq,
+};
+
static int carm_init_disks(struct carm_host *host)
{
unsigned int i;
disk->fops = &carm_bd_ops;
disk->private_data = port;
- q = blk_init_queue(carm_rq_fn, &host->lock);
- if (!q) {
- rc = -ENOMEM;
+ q = blk_mq_init_sq_queue(&port->tag_set, &carm_mq_ops,
+ max_queue, BLK_MQ_F_SHOULD_MERGE);
+ if (IS_ERR(q)) {
+ rc = PTR_ERR(q);
break;
}
disk->queue = q;
unsigned int i;
for (i = 0; i < CARM_MAX_PORTS; i++) {
- struct gendisk *disk = host->port[i].disk;
+ struct carm_port *port = &host->port[i];
+ struct gendisk *disk = port->disk;
+
if (disk) {
struct request_queue *q = disk->queue;
if (disk->flags & GENHD_FL_UP)
del_gendisk(disk);
- if (q)
+ if (q) {
+ blk_mq_free_tag_set(&port->tag_set);
blk_cleanup_queue(q);
+ }
put_disk(disk);
}
}
static int carm_init_shm(struct carm_host *host)
{
- host->shm = pci_alloc_consistent(host->pdev, CARM_SHM_SIZE,
- &host->shm_dma);
+ host->shm = dma_alloc_coherent(&host->pdev->dev, CARM_SHM_SIZE,
+ &host->shm_dma, GFP_KERNEL);
if (!host->shm)
return -ENOMEM;
static int carm_init_one (struct pci_dev *pdev, const struct pci_device_id *ent)
{
struct carm_host *host;
- unsigned int pci_dac;
int rc;
struct request_queue *q;
unsigned int i;
if (rc)
goto err_out;
-#ifdef IF_64BIT_DMA_IS_POSSIBLE /* grrrr... */
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
- if (!rc) {
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
- if (rc) {
- printk(KERN_ERR DRV_NAME "(%s): consistent DMA mask failure\n",
- pci_name(pdev));
- goto err_out_regions;
- }
- pci_dac = 1;
- } else {
-#endif
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
- if (rc) {
- printk(KERN_ERR DRV_NAME "(%s): DMA mask failure\n",
- pci_name(pdev));
- goto err_out_regions;
- }
- pci_dac = 0;
-#ifdef IF_64BIT_DMA_IS_POSSIBLE /* grrrr... */
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
+ if (rc) {
+ printk(KERN_ERR DRV_NAME "(%s): DMA mask failure\n",
+ pci_name(pdev));
+ goto err_out_regions;
}
-#endif
host = kzalloc(sizeof(*host), GFP_KERNEL);
if (!host) {
}
host->pdev = pdev;
- host->flags = pci_dac ? FL_DAC : 0;
spin_lock_init(&host->lock);
INIT_WORK(&host->fsm_task, carm_fsm_task);
init_completion(&host->probe_comp);
goto err_out_iounmap;
}
- q = blk_init_queue(carm_oob_rq_fn, &host->lock);
- if (!q) {
+ q = blk_mq_init_sq_queue(&host->tag_set, &carm_oob_mq_ops, 1,
+ BLK_MQ_F_NO_SCHED);
+ if (IS_ERR(q)) {
printk(KERN_ERR DRV_NAME "(%s): OOB queue alloc failure\n",
pci_name(pdev));
- rc = -ENOMEM;
- goto err_out_pci_free;
+ rc = PTR_ERR(q);
+ goto err_out_dma_free;
}
host->oob_q = q;
q->queuedata = host;
else if (host->major == 161)
clear_bit(1, &carm_major_alloc);
blk_cleanup_queue(host->oob_q);
-err_out_pci_free:
- pci_free_consistent(pdev, CARM_SHM_SIZE, host->shm, host->shm_dma);
+ blk_mq_free_tag_set(&host->tag_set);
+err_out_dma_free:
+ dma_free_coherent(&pdev->dev, CARM_SHM_SIZE, host->shm, host->shm_dma);
err_out_iounmap:
iounmap(host->mmio);
err_out_kfree:
else if (host->major == 161)
clear_bit(1, &carm_major_alloc);
blk_cleanup_queue(host->oob_q);
- pci_free_consistent(pdev, CARM_SHM_SIZE, host->shm, host->shm_dma);
+ blk_mq_free_tag_set(&host->tag_set);
+ dma_free_coherent(&pdev->dev, CARM_SHM_SIZE, host->shm, host->shm_dma);
iounmap(host->mmio);
kfree(host);
pci_release_regions(pdev);
vec = bio_iter_iovec(bio, card->current_iter);
- dma_handle = pci_map_page(card->dev,
+ dma_handle = dma_map_page(&card->dev->dev,
vec.bv_page,
vec.bv_offset,
vec.bv_len,
bio_op(bio) == REQ_OP_READ ?
- PCI_DMA_FROMDEVICE : PCI_DMA_TODEVICE);
+ DMA_FROM_DEVICE : DMA_TO_DEVICE);
p = &card->mm_pages[card->Ready];
desc = &p->desc[p->cnt];
struct cardinfo *card = (struct cardinfo *)data;
unsigned int dma_status = card->dma_status;
- spin_lock_bh(&card->lock);
+ spin_lock(&card->lock);
if (card->Active < 0)
goto out_unlock;
page = &card->mm_pages[card->Active];
page->iter = page->bio->bi_iter;
}
- pci_unmap_page(card->dev, desc->data_dma_handle,
+ dma_unmap_page(&card->dev->dev, desc->data_dma_handle,
vec.bv_len,
(control & DMASCR_TRANSFER_READ) ?
- PCI_DMA_TODEVICE : PCI_DMA_FROMDEVICE);
+ DMA_TO_DEVICE : DMA_FROM_DEVICE);
if (control & DMASCR_HARD_ERROR) {
/* error */
bio->bi_status = BLK_STS_IOERR;
mm_start_io(card);
}
out_unlock:
- spin_unlock_bh(&card->lock);
+ spin_unlock(&card->lock);
while (return_bio) {
struct bio *bio = return_bio;
dev_printk(KERN_INFO, &dev->dev,
"Micro Memory(tm) controller found (PCI Mem Module (Battery Backup))\n");
- if (pci_set_dma_mask(dev, DMA_BIT_MASK(64)) &&
- pci_set_dma_mask(dev, DMA_BIT_MASK(32))) {
+ if (dma_set_mask(&dev->dev, DMA_BIT_MASK(64)) &&
+ dma_set_mask(&dev->dev, DMA_BIT_MASK(32))) {
dev_printk(KERN_WARNING, &dev->dev, "NO suitable DMA found\n");
return -ENOMEM;
}
goto failed_magic;
}
- card->mm_pages[0].desc = pci_alloc_consistent(card->dev,
- PAGE_SIZE * 2,
- &card->mm_pages[0].page_dma);
- card->mm_pages[1].desc = pci_alloc_consistent(card->dev,
- PAGE_SIZE * 2,
- &card->mm_pages[1].page_dma);
+ card->mm_pages[0].desc = dma_alloc_coherent(&card->dev->dev,
+ PAGE_SIZE * 2, &card->mm_pages[0].page_dma, GFP_KERNEL);
+ card->mm_pages[1].desc = dma_alloc_coherent(&card->dev->dev,
+ PAGE_SIZE * 2, &card->mm_pages[1].page_dma, GFP_KERNEL);
if (card->mm_pages[0].desc == NULL ||
card->mm_pages[1].desc == NULL) {
dev_printk(KERN_ERR, &card->dev->dev, "alloc failed\n");
failed_req_irq:
failed_alloc:
if (card->mm_pages[0].desc)
- pci_free_consistent(card->dev, PAGE_SIZE*2,
- card->mm_pages[0].desc,
- card->mm_pages[0].page_dma);
+ dma_free_coherent(&card->dev->dev, PAGE_SIZE * 2,
+ card->mm_pages[0].desc,
+ card->mm_pages[0].page_dma);
if (card->mm_pages[1].desc)
- pci_free_consistent(card->dev, PAGE_SIZE*2,
- card->mm_pages[1].desc,
- card->mm_pages[1].page_dma);
+ dma_free_coherent(&card->dev->dev, PAGE_SIZE * 2,
+ card->mm_pages[1].desc,
+ card->mm_pages[1].page_dma);
failed_magic:
iounmap(card->csr_remap);
failed_remap_csr:
iounmap(card->csr_remap);
if (card->mm_pages[0].desc)
- pci_free_consistent(card->dev, PAGE_SIZE*2,
+ dma_free_coherent(&card->dev->dev, PAGE_SIZE * 2,
card->mm_pages[0].desc,
card->mm_pages[0].page_dma);
if (card->mm_pages[1].desc)
- pci_free_consistent(card->dev, PAGE_SIZE*2,
+ dma_free_coherent(&card->dev->dev, PAGE_SIZE * 2,
card->mm_pages[1].desc,
card->mm_pages[1].page_dma);
blk_cleanup_queue(card->queue);
return minor >> PART_BITS;
}
-static ssize_t virtblk_serial_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t serial_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
{
struct gendisk *disk = dev_to_disk(dev);
int err;
return err;
}
-static DEVICE_ATTR(serial, 0444, virtblk_serial_show, NULL);
+static DEVICE_ATTR_RO(serial);
/* The queue's logical block size must be set before calling this */
static void virtblk_update_capacity(struct virtio_blk *vblk, bool resize)
};
static ssize_t
-virtblk_cache_type_store(struct device *dev, struct device_attribute *attr,
- const char *buf, size_t count)
+cache_type_store(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t count)
{
struct gendisk *disk = dev_to_disk(dev);
struct virtio_blk *vblk = disk->private_data;
}
static ssize_t
-virtblk_cache_type_show(struct device *dev, struct device_attribute *attr,
- char *buf)
+cache_type_show(struct device *dev, struct device_attribute *attr, char *buf)
{
struct gendisk *disk = dev_to_disk(dev);
struct virtio_blk *vblk = disk->private_data;
return snprintf(buf, 40, "%s\n", virtblk_cache_types[writeback]);
}
-static const struct device_attribute dev_attr_cache_type_ro =
- __ATTR(cache_type, 0444,
- virtblk_cache_type_show, NULL);
-static const struct device_attribute dev_attr_cache_type_rw =
- __ATTR(cache_type, 0644,
- virtblk_cache_type_show, virtblk_cache_type_store);
+static DEVICE_ATTR_RW(cache_type);
+
+static struct attribute *virtblk_attrs[] = {
+ &dev_attr_serial.attr,
+ &dev_attr_cache_type.attr,
+ NULL,
+};
+
+static umode_t virtblk_attrs_are_visible(struct kobject *kobj,
+ struct attribute *a, int n)
+{
+ struct device *dev = container_of(kobj, struct device, kobj);
+ struct gendisk *disk = dev_to_disk(dev);
+ struct virtio_blk *vblk = disk->private_data;
+ struct virtio_device *vdev = vblk->vdev;
+
+ if (a == &dev_attr_cache_type.attr &&
+ !virtio_has_feature(vdev, VIRTIO_BLK_F_CONFIG_WCE))
+ return S_IRUGO;
+
+ return a->mode;
+}
+
+static const struct attribute_group virtblk_attr_group = {
+ .attrs = virtblk_attrs,
+ .is_visible = virtblk_attrs_are_visible,
+};
+
+static const struct attribute_group *virtblk_attr_groups[] = {
+ &virtblk_attr_group,
+ NULL,
+};
static int virtblk_init_request(struct blk_mq_tag_set *set, struct request *rq,
unsigned int hctx_idx, unsigned int numa_node)
virtblk_update_capacity(vblk, false);
virtio_device_ready(vdev);
- device_add_disk(&vdev->dev, vblk->disk);
- err = device_create_file(disk_to_dev(vblk->disk), &dev_attr_serial);
- if (err)
- goto out_del_disk;
-
- if (virtio_has_feature(vdev, VIRTIO_BLK_F_CONFIG_WCE))
- err = device_create_file(disk_to_dev(vblk->disk),
- &dev_attr_cache_type_rw);
- else
- err = device_create_file(disk_to_dev(vblk->disk),
- &dev_attr_cache_type_ro);
- if (err)
- goto out_del_disk;
+ device_add_disk(&vdev->dev, vblk->disk, virtblk_attr_groups);
return 0;
-out_del_disk:
- del_gendisk(vblk->disk);
- blk_cleanup_queue(vblk->disk->queue);
out_free_tags:
blk_mq_free_tag_set(&vblk->tag_set);
out_put_disk:
for (i = 0; i < info->nr_rings; i++)
kick_pending_request_queues(&info->rinfo[i]);
- device_add_disk(&info->xbdev->dev, info->gd);
+ device_add_disk(&info->xbdev->dev, info->gd, NULL);
info->is_ready = 1;
return;
#include <linux/kernel.h>
#include <linux/delay.h>
#include <linux/slab.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/mutex.h>
#include <linux/ata.h>
#include <linux/hdreg.h>
struct device *dev;
struct request_queue *queue;
struct gendisk *gd;
+ struct blk_mq_tag_set tag_set;
+ struct list_head rq_list;
/* Inserted CF card parameters */
u16 cf_id[ATA_ID_WORDS];
ace->fsm_continue_flag = 0;
}
+static bool ace_has_next_request(struct request_queue *q)
+{
+ struct ace_device *ace = q->queuedata;
+
+ return !list_empty(&ace->rq_list);
+}
+
/* Get the next read/write request; ending requests that we don't handle */
static struct request *ace_get_next_request(struct request_queue *q)
{
- struct request *req;
+ struct ace_device *ace = q->queuedata;
+ struct request *rq;
- while ((req = blk_peek_request(q)) != NULL) {
- if (!blk_rq_is_passthrough(req))
- break;
- blk_start_request(req);
- __blk_end_request_all(req, BLK_STS_IOERR);
+ rq = list_first_entry_or_null(&ace->rq_list, struct request, queuelist);
+ if (rq) {
+ list_del_init(&rq->queuelist);
+ blk_mq_start_request(rq);
}
- return req;
+
+ return NULL;
}
static void ace_fsm_dostate(struct ace_device *ace)
/* Drop all in-flight and pending requests */
if (ace->req) {
- __blk_end_request_all(ace->req, BLK_STS_IOERR);
+ blk_mq_end_request(ace->req, BLK_STS_IOERR);
ace->req = NULL;
}
- while ((req = blk_fetch_request(ace->queue)) != NULL)
- __blk_end_request_all(req, BLK_STS_IOERR);
+ while ((req = ace_get_next_request(ace->queue)) != NULL)
+ blk_mq_end_request(req, BLK_STS_IOERR);
/* Drop back to IDLE state and notify waiters */
ace->fsm_state = ACE_FSM_STATE_IDLE;
switch (ace->fsm_state) {
case ACE_FSM_STATE_IDLE:
/* See if there is anything to do */
- if (ace->id_req_count || ace_get_next_request(ace->queue)) {
+ if (ace->id_req_count || ace_has_next_request(ace->queue)) {
ace->fsm_iter_num++;
ace->fsm_state = ACE_FSM_STATE_REQ_LOCK;
mod_timer(&ace->stall_timer, jiffies + HZ);
ace->fsm_state = ACE_FSM_STATE_IDLE;
break;
}
- blk_start_request(req);
/* Okay, it's a data request, set it up for transfer */
dev_dbg(ace->dev,
}
/* bio finished; is there another one? */
- if (__blk_end_request_cur(ace->req, BLK_STS_OK)) {
+ if (blk_update_request(ace->req, BLK_STS_OK,
+ blk_rq_cur_bytes(ace->req))) {
/* dev_dbg(ace->dev, "next block; h=%u c=%u\n",
* blk_rq_sectors(ace->req),
* blk_rq_cur_sectors(ace->req));
/* ---------------------------------------------------------------------
* Block ops
*/
-static void ace_request(struct request_queue * q)
+static blk_status_t ace_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
- struct request *req;
- struct ace_device *ace;
-
- req = ace_get_next_request(q);
+ struct ace_device *ace = hctx->queue->queuedata;
+ struct request *req = bd->rq;
- if (req) {
- ace = req->rq_disk->private_data;
- tasklet_schedule(&ace->fsm_tasklet);
+ if (blk_rq_is_passthrough(req)) {
+ blk_mq_start_request(req);
+ return BLK_STS_IOERR;
}
+
+ spin_lock_irq(&ace->lock);
+ list_add_tail(&req->queuelist, &ace->rq_list);
+ spin_unlock_irq(&ace->lock);
+
+ tasklet_schedule(&ace->fsm_tasklet);
+ return BLK_STS_OK;
}
static unsigned int ace_check_events(struct gendisk *gd, unsigned int clearing)
.getgeo = ace_getgeo,
};
+static const struct blk_mq_ops ace_mq_ops = {
+ .queue_rq = ace_queue_rq,
+};
+
/* --------------------------------------------------------------------
* SystemACE device setup/teardown code
*/
spin_lock_init(&ace->lock);
init_completion(&ace->id_completion);
+ INIT_LIST_HEAD(&ace->rq_list);
/*
* Map the device
/*
* Initialize the request queue
*/
- ace->queue = blk_init_queue(ace_request, &ace->lock);
- if (ace->queue == NULL)
+ ace->queue = blk_mq_init_sq_queue(&ace->tag_set, &ace_mq_ops, 2,
+ BLK_MQ_F_SHOULD_MERGE);
+ if (IS_ERR(ace->queue)) {
+ rc = PTR_ERR(ace->queue);
+ ace->queue = NULL;
goto err_blk_initq;
+ }
+ ace->queue->queuedata = ace;
+
blk_queue_logical_block_size(ace->queue, 512);
blk_queue_bounce_limit(ace->queue, BLK_BOUNCE_HIGH);
put_disk(ace->gd);
err_alloc_disk:
blk_cleanup_queue(ace->queue);
+ blk_mq_free_tag_set(&ace->tag_set);
err_blk_initq:
iounmap(ace->baseaddr);
err_ioremap:
put_disk(ace->gd);
}
- if (ace->queue)
+ if (ace->queue) {
blk_cleanup_queue(ace->queue);
+ blk_mq_free_tag_set(&ace->tag_set);
+ }
tasklet_kill(&ace->fsm_tasklet);
#include <linux/vmalloc.h>
#include <linux/init.h>
#include <linux/module.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/bitops.h>
#include <linux/mutex.h>
#include <linux/slab.h>
static struct gendisk *z2ram_gendisk;
-static void do_z2_request(struct request_queue *q)
+static blk_status_t z2_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
- struct request *req;
-
- req = blk_fetch_request(q);
- while (req) {
- unsigned long start = blk_rq_pos(req) << 9;
- unsigned long len = blk_rq_cur_bytes(req);
- blk_status_t err = BLK_STS_OK;
-
- if (start + len > z2ram_size) {
- pr_err(DEVICE_NAME ": bad access: block=%llu, "
- "count=%u\n",
- (unsigned long long)blk_rq_pos(req),
- blk_rq_cur_sectors(req));
- err = BLK_STS_IOERR;
- goto done;
- }
- while (len) {
- unsigned long addr = start & Z2RAM_CHUNKMASK;
- unsigned long size = Z2RAM_CHUNKSIZE - addr;
- void *buffer = bio_data(req->bio);
-
- if (len < size)
- size = len;
- addr += z2ram_map[ start >> Z2RAM_CHUNKSHIFT ];
- if (rq_data_dir(req) == READ)
- memcpy(buffer, (char *)addr, size);
- else
- memcpy((char *)addr, buffer, size);
- start += size;
- len -= size;
- }
- done:
- if (!__blk_end_request_cur(req, err))
- req = blk_fetch_request(q);
+ struct request *req = bd->rq;
+ unsigned long start = blk_rq_pos(req) << 9;
+ unsigned long len = blk_rq_cur_bytes(req);
+
+ blk_mq_start_request(req);
+
+ if (start + len > z2ram_size) {
+ pr_err(DEVICE_NAME ": bad access: block=%llu, "
+ "count=%u\n",
+ (unsigned long long)blk_rq_pos(req),
+ blk_rq_cur_sectors(req));
+ return BLK_STS_IOERR;
+ }
+
+ spin_lock_irq(&z2ram_lock);
+
+ while (len) {
+ unsigned long addr = start & Z2RAM_CHUNKMASK;
+ unsigned long size = Z2RAM_CHUNKSIZE - addr;
+ void *buffer = bio_data(req->bio);
+
+ if (len < size)
+ size = len;
+ addr += z2ram_map[ start >> Z2RAM_CHUNKSHIFT ];
+ if (rq_data_dir(req) == READ)
+ memcpy(buffer, (char *)addr, size);
+ else
+ memcpy((char *)addr, buffer, size);
+ start += size;
+ len -= size;
}
+
+ spin_unlock_irq(&z2ram_lock);
+ blk_mq_end_request(req, BLK_STS_OK);
+ return BLK_STS_OK;
}
static void
}
static struct request_queue *z2_queue;
+static struct blk_mq_tag_set tag_set;
+
+static const struct blk_mq_ops z2_mq_ops = {
+ .queue_rq = z2_queue_rq,
+};
static int __init
z2_init(void)
if (!z2ram_gendisk)
goto out_disk;
- z2_queue = blk_init_queue(do_z2_request, &z2ram_lock);
- if (!z2_queue)
+ z2_queue = blk_mq_init_sq_queue(&tag_set, &z2_mq_ops, 16,
+ BLK_MQ_F_SHOULD_MERGE);
+ if (IS_ERR(z2_queue)) {
+ ret = PTR_ERR(z2_queue);
+ z2_queue = NULL;
goto out_queue;
+ }
z2ram_gendisk->major = Z2RAM_MAJOR;
z2ram_gendisk->first_minor = 0;
del_gendisk(z2ram_gendisk);
put_disk(z2ram_gendisk);
blk_cleanup_queue(z2_queue);
+ blk_mq_free_tag_set(&tag_set);
if ( current_device != -1 )
{
tristate "Compressed RAM block device support"
depends on BLOCK && SYSFS && ZSMALLOC && CRYPTO
select CRYPTO_LZO
- default n
help
Creates virtual block devices called /dev/zramX (X = 0, 1, ...).
Pages written to these disks are compressed and stored in memory
config ZRAM_WRITEBACK
bool "Write back incompressible page to backing device"
depends on ZRAM
- default n
help
With incompressible page, there is no memory saving to keep it
in memory. Instead, write it out to backing device.
.attrs = zram_disk_attrs,
};
+static const struct attribute_group *zram_disk_attr_groups[] = {
+ &zram_disk_attr_group,
+ NULL,
+};
+
/*
* Allocate and initialize new zram device. the function returns
* '>= 0' device_id upon success, and negative value otherwise.
zram->disk->queue->backing_dev_info->capabilities |=
(BDI_CAP_STABLE_WRITES | BDI_CAP_SYNCHRONOUS_IO);
- add_disk(zram->disk);
-
- ret = sysfs_create_group(&disk_to_dev(zram->disk)->kobj,
- &zram_disk_attr_group);
- if (ret < 0) {
- pr_err("Error creating sysfs group for device %d\n",
- device_id);
- goto out_free_disk;
- }
+ device_add_disk(NULL, zram->disk, zram_disk_attr_groups);
+
strlcpy(zram->compressor, default_compressor, sizeof(zram->compressor));
zram_debugfs_register(zram);
pr_info("Added device: %s\n", zram->disk->disk_name);
return device_id;
-out_free_disk:
- del_gendisk(zram->disk);
- put_disk(zram->disk);
out_free_queue:
blk_cleanup_queue(queue);
out_free_idr:
mutex_unlock(&bdev->bd_mutex);
zram_debugfs_unregister(zram);
- /*
- * Remove sysfs first, so no one will perform a disksize
- * store while we destroy the devices. This also helps during
- * hot_remove -- zram_reset_device() is the last holder of
- * ->init_lock, no later/concurrent disksize_store() or any
- * other sysfs handlers are possible.
- */
- sysfs_remove_group(&disk_to_dev(zram->disk)->kobj,
- &zram_disk_attr_group);
/* Make sure all the pending I/O are finished */
fsync_bdev(bdev);
{
int i;
- qca->vreg_bulk = devm_kzalloc(qca->dev, num_vregs *
+ qca->vreg_bulk = devm_kcalloc(qca->dev, num_vregs,
sizeof(struct regulator_bulk_data),
GFP_KERNEL);
if (!qca->vreg_bulk)
*/
static void ts_nbus_reset_bus(struct ts_nbus *ts_nbus)
{
- int i;
- int values[8];
+ DECLARE_BITMAP(values, 8);
- for (i = 0; i < 8; i++)
- values[i] = 0;
+ values[0] = 0;
- gpiod_set_array_value_cansleep(8, ts_nbus->data->desc, values);
+ gpiod_set_array_value_cansleep(8, ts_nbus->data->desc,
+ ts_nbus->data->info, values);
gpiod_set_value_cansleep(ts_nbus->csn, 0);
gpiod_set_value_cansleep(ts_nbus->strobe, 0);
gpiod_set_value_cansleep(ts_nbus->ale, 0);
static void ts_nbus_write_byte(struct ts_nbus *ts_nbus, u8 byte)
{
struct gpio_descs *gpios = ts_nbus->data;
- int i;
- int values[8];
+ DECLARE_BITMAP(values, 8);
- for (i = 0; i < 8; i++)
- if (byte & BIT(i))
- values[i] = 1;
- else
- values[i] = 0;
+ values[0] = byte;
- gpiod_set_array_value_cansleep(8, gpios->desc, values);
+ gpiod_set_array_value_cansleep(8, gpios->desc, gpios->info, values);
}
/*
* hack to have the capability flags defined const, while we can still
* change it here without gcc complaining at every line.
*/
-#define ENSURE(call, bits) \
-do { \
- if (cdo->call == NULL) \
- *change_capability &= ~(bits); \
+#define ENSURE(cdo, call, bits) \
+do { \
+ if (cdo->call == NULL) \
+ WARN_ON_ONCE((cdo)->capability & (bits)); \
} while (0)
/*
{
static char banner_printed;
const struct cdrom_device_ops *cdo = cdi->ops;
- int *change_capability = (int *)&cdo->capability; /* hack */
cd_dbg(CD_OPEN, "entering register_cdrom\n");
cdrom_sysctl_register();
}
- ENSURE(drive_status, CDC_DRIVE_STATUS);
+ ENSURE(cdo, drive_status, CDC_DRIVE_STATUS);
if (cdo->check_events == NULL && cdo->media_changed == NULL)
- *change_capability = ~(CDC_MEDIA_CHANGED | CDC_SELECT_DISC);
- ENSURE(tray_move, CDC_CLOSE_TRAY | CDC_OPEN_TRAY);
- ENSURE(lock_door, CDC_LOCK);
- ENSURE(select_speed, CDC_SELECT_SPEED);
- ENSURE(get_last_session, CDC_MULTI_SESSION);
- ENSURE(get_mcn, CDC_MCN);
- ENSURE(reset, CDC_RESET);
- ENSURE(generic_packet, CDC_GENERIC_PACKET);
+ WARN_ON_ONCE(cdo->capability & (CDC_MEDIA_CHANGED | CDC_SELECT_DISC));
+ ENSURE(cdo, tray_move, CDC_CLOSE_TRAY | CDC_OPEN_TRAY);
+ ENSURE(cdo, lock_door, CDC_LOCK);
+ ENSURE(cdo, select_speed, CDC_SELECT_SPEED);
+ ENSURE(cdo, get_last_session, CDC_MULTI_SESSION);
+ ENSURE(cdo, get_mcn, CDC_MCN);
+ ENSURE(cdo, reset, CDC_RESET);
+ ENSURE(cdo, generic_packet, CDC_GENERIC_PACKET);
cdi->mc_flags = 0;
cdi->options = CDO_USE_FFLAGS;
return -ENOSYS;
if (arg != CDSL_CURRENT && arg != CDSL_NONE) {
- if ((int)arg >= cdi->capacity)
+ if (arg >= cdi->capacity)
return -EINVAL;
}
#include <linux/cdrom.h>
#include <linux/genhd.h>
#include <linux/bio.h>
-#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/interrupt.h>
#include <linux/device.h>
#include <linux/mutex.h>
#include <linux/wait.h>
-#include <linux/workqueue.h>
#include <linux/platform_device.h>
#include <scsi/scsi.h>
#include <asm/io.h>
static DECLARE_WAIT_QUEUE_HEAD(command_queue);
static DECLARE_WAIT_QUEUE_HEAD(request_queue);
-static DEFINE_SPINLOCK(gdrom_lock);
-static void gdrom_readdisk_dma(struct work_struct *work);
-static DECLARE_WORK(work, gdrom_readdisk_dma);
-static LIST_HEAD(gdrom_deferred);
-
struct gdromtoc {
unsigned int entry[99];
unsigned int first, last;
char disk_type;
struct gdromtoc *toc;
struct request_queue *gdrom_rq;
+ struct blk_mq_tag_set tag_set;
} gd;
struct gdrom_id {
* 9 -> sectors >> 8
* 10 -> sectors
*/
-static void gdrom_readdisk_dma(struct work_struct *work)
+static blk_status_t gdrom_readdisk_dma(struct request *req)
{
int block, block_cnt;
blk_status_t err;
struct packet_command *read_command;
- struct list_head *elem, *next;
- struct request *req;
unsigned long timeout;
- if (list_empty(&gdrom_deferred))
- return;
read_command = kzalloc(sizeof(struct packet_command), GFP_KERNEL);
if (!read_command)
- return; /* get more memory later? */
+ return BLK_STS_RESOURCE;
+
read_command->cmd[0] = 0x30;
read_command->cmd[1] = 0x20;
- spin_lock(&gdrom_lock);
- list_for_each_safe(elem, next, &gdrom_deferred) {
- req = list_entry(elem, struct request, queuelist);
- spin_unlock(&gdrom_lock);
- block = blk_rq_pos(req)/GD_TO_BLK + GD_SESSION_OFFSET;
- block_cnt = blk_rq_sectors(req)/GD_TO_BLK;
- __raw_writel(virt_to_phys(bio_data(req->bio)), GDROM_DMA_STARTADDR_REG);
- __raw_writel(block_cnt * GDROM_HARD_SECTOR, GDROM_DMA_LENGTH_REG);
- __raw_writel(1, GDROM_DMA_DIRECTION_REG);
- __raw_writel(1, GDROM_DMA_ENABLE_REG);
- read_command->cmd[2] = (block >> 16) & 0xFF;
- read_command->cmd[3] = (block >> 8) & 0xFF;
- read_command->cmd[4] = block & 0xFF;
- read_command->cmd[8] = (block_cnt >> 16) & 0xFF;
- read_command->cmd[9] = (block_cnt >> 8) & 0xFF;
- read_command->cmd[10] = block_cnt & 0xFF;
- /* set for DMA */
- __raw_writeb(1, GDROM_ERROR_REG);
- /* other registers */
- __raw_writeb(0, GDROM_SECNUM_REG);
- __raw_writeb(0, GDROM_BCL_REG);
- __raw_writeb(0, GDROM_BCH_REG);
- __raw_writeb(0, GDROM_DSEL_REG);
- __raw_writeb(0, GDROM_INTSEC_REG);
- /* Wait for registers to reset after any previous activity */
- timeout = jiffies + HZ / 2;
- while (gdrom_is_busy() && time_before(jiffies, timeout))
- cpu_relax();
- __raw_writeb(GDROM_COM_PACKET, GDROM_STATUSCOMMAND_REG);
- timeout = jiffies + HZ / 2;
- /* Wait for packet command to finish */
- while (gdrom_is_busy() && time_before(jiffies, timeout))
- cpu_relax();
- gd.pending = 1;
- gd.transfer = 1;
- outsw(GDROM_DATA_REG, &read_command->cmd, 6);
- timeout = jiffies + HZ / 2;
- /* Wait for any pending DMA to finish */
- while (__raw_readb(GDROM_DMA_STATUS_REG) &&
- time_before(jiffies, timeout))
- cpu_relax();
- /* start transfer */
- __raw_writeb(1, GDROM_DMA_STATUS_REG);
- wait_event_interruptible_timeout(request_queue,
- gd.transfer == 0, GDROM_DEFAULT_TIMEOUT);
- err = gd.transfer ? BLK_STS_IOERR : BLK_STS_OK;
- gd.transfer = 0;
- gd.pending = 0;
- /* now seek to take the request spinlock
- * before handling ending the request */
- spin_lock(&gdrom_lock);
- list_del_init(&req->queuelist);
- __blk_end_request_all(req, err);
- }
- spin_unlock(&gdrom_lock);
+ block = blk_rq_pos(req)/GD_TO_BLK + GD_SESSION_OFFSET;
+ block_cnt = blk_rq_sectors(req)/GD_TO_BLK;
+ __raw_writel(virt_to_phys(bio_data(req->bio)), GDROM_DMA_STARTADDR_REG);
+ __raw_writel(block_cnt * GDROM_HARD_SECTOR, GDROM_DMA_LENGTH_REG);
+ __raw_writel(1, GDROM_DMA_DIRECTION_REG);
+ __raw_writel(1, GDROM_DMA_ENABLE_REG);
+ read_command->cmd[2] = (block >> 16) & 0xFF;
+ read_command->cmd[3] = (block >> 8) & 0xFF;
+ read_command->cmd[4] = block & 0xFF;
+ read_command->cmd[8] = (block_cnt >> 16) & 0xFF;
+ read_command->cmd[9] = (block_cnt >> 8) & 0xFF;
+ read_command->cmd[10] = block_cnt & 0xFF;
+ /* set for DMA */
+ __raw_writeb(1, GDROM_ERROR_REG);
+ /* other registers */
+ __raw_writeb(0, GDROM_SECNUM_REG);
+ __raw_writeb(0, GDROM_BCL_REG);
+ __raw_writeb(0, GDROM_BCH_REG);
+ __raw_writeb(0, GDROM_DSEL_REG);
+ __raw_writeb(0, GDROM_INTSEC_REG);
+ /* Wait for registers to reset after any previous activity */
+ timeout = jiffies + HZ / 2;
+ while (gdrom_is_busy() && time_before(jiffies, timeout))
+ cpu_relax();
+ __raw_writeb(GDROM_COM_PACKET, GDROM_STATUSCOMMAND_REG);
+ timeout = jiffies + HZ / 2;
+ /* Wait for packet command to finish */
+ while (gdrom_is_busy() && time_before(jiffies, timeout))
+ cpu_relax();
+ gd.pending = 1;
+ gd.transfer = 1;
+ outsw(GDROM_DATA_REG, &read_command->cmd, 6);
+ timeout = jiffies + HZ / 2;
+ /* Wait for any pending DMA to finish */
+ while (__raw_readb(GDROM_DMA_STATUS_REG) &&
+ time_before(jiffies, timeout))
+ cpu_relax();
+ /* start transfer */
+ __raw_writeb(1, GDROM_DMA_STATUS_REG);
+ wait_event_interruptible_timeout(request_queue,
+ gd.transfer == 0, GDROM_DEFAULT_TIMEOUT);
+ err = gd.transfer ? BLK_STS_IOERR : BLK_STS_OK;
+ gd.transfer = 0;
+ gd.pending = 0;
+
+ blk_mq_end_request(req, err);
kfree(read_command);
+ return BLK_STS_OK;
}
-static void gdrom_request(struct request_queue *rq)
-{
- struct request *req;
-
- while ((req = blk_fetch_request(rq)) != NULL) {
- switch (req_op(req)) {
- case REQ_OP_READ:
- /*
- * Add to list of deferred work and then schedule
- * workqueue.
- */
- list_add_tail(&req->queuelist, &gdrom_deferred);
- schedule_work(&work);
- break;
- case REQ_OP_WRITE:
- pr_notice("Read only device - write request ignored\n");
- __blk_end_request_all(req, BLK_STS_IOERR);
- break;
- default:
- printk(KERN_DEBUG "gdrom: Non-fs request ignored\n");
- __blk_end_request_all(req, BLK_STS_IOERR);
- break;
- }
+static blk_status_t gdrom_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
+{
+ blk_mq_start_request(bd->rq);
+
+ switch (req_op(bd->rq)) {
+ case REQ_OP_READ:
+ return gdrom_readdisk_dma(bd->rq);
+ case REQ_OP_WRITE:
+ pr_notice("Read only device - write request ignored\n");
+ return BLK_STS_IOERR;
+ default:
+ printk(KERN_DEBUG "gdrom: Non-fs request ignored\n");
+ return BLK_STS_IOERR;
}
}
return gdrom_init_dma_mode();
}
+static const struct blk_mq_ops gdrom_mq_ops = {
+ .queue_rq = gdrom_queue_rq,
+};
+
/*
* register this as a block device and as compliant with the
* universal CD Rom driver interface
err = gdrom_set_interrupt_handlers();
if (err)
goto probe_fail_cmdirq_register;
- gd.gdrom_rq = blk_init_queue(gdrom_request, &gdrom_lock);
- if (!gd.gdrom_rq) {
- err = -ENOMEM;
+
+ gd.gdrom_rq = blk_mq_init_sq_queue(&gd.tag_set, &gdrom_mq_ops, 1,
+ BLK_MQ_F_SHOULD_MERGE | BLK_MQ_F_BLOCKING);
+ if (IS_ERR(gd.gdrom_rq)) {
+ rc = PTR_ERR(gd.gdrom_rq);
+ gd.gdrom_rq = NULL;
goto probe_fail_requestq;
}
+
blk_queue_bounce_limit(gd.gdrom_rq, BLK_BOUNCE_HIGH);
err = probe_gdrom_setupqueue();
probe_fail_toc:
blk_cleanup_queue(gd.gdrom_rq);
+ blk_mq_free_tag_set(&gd.tag_set);
probe_fail_requestq:
free_irq(HW_EVENT_GDROM_DMA, &gd);
free_irq(HW_EVENT_GDROM_CMD, &gd);
static int remove_gdrom(struct platform_device *devptr)
{
- flush_work(&work);
blk_cleanup_queue(gd.gdrom_rq);
+ blk_mq_free_tag_set(&gd.tag_set);
free_irq(HW_EVENT_GDROM_CMD, &gd);
free_irq(HW_EVENT_GDROM_DMA, &gd);
del_gendisk(gd.disk);
* Author: Rocky Craig <first.last@hp.com>
*/
+#define DEBUG /* So dev_dbg() is always available. */
+
#include <linux/kernel.h> /* For printk. */
#include <linux/string.h>
#include <linux/module.h>
return IPMI_NOT_IN_MY_STATE_ERR;
if (bt_debug & BT_DEBUG_MSG) {
- printk(KERN_WARNING "BT: +++++++++++++++++ New command\n");
- printk(KERN_WARNING "BT: NetFn/LUN CMD [%d data]:", size - 2);
+ dev_dbg(bt->io->dev, "+++++++++++++++++ New command\n");
+ dev_dbg(bt->io->dev, "NetFn/LUN CMD [%d data]:", size - 2);
for (i = 0; i < size; i ++)
- printk(" %02x", data[i]);
- printk("\n");
+ pr_cont(" %02x", data[i]);
+ pr_cont("\n");
}
bt->write_data[0] = size + 1; /* all data plus seq byte */
bt->write_data[1] = *data; /* NetFn/LUN */
memcpy(data + 2, bt->read_data + 4, msg_len - 2);
if (bt_debug & BT_DEBUG_MSG) {
- printk(KERN_WARNING "BT: result %d bytes:", msg_len);
+ dev_dbg(bt->io->dev, "result %d bytes:", msg_len);
for (i = 0; i < msg_len; i++)
- printk(" %02x", data[i]);
- printk("\n");
+ pr_cont(" %02x", data[i]);
+ pr_cont("\n");
}
return msg_len;
}
static void reset_flags(struct si_sm_data *bt)
{
if (bt_debug)
- printk(KERN_WARNING "IPMI BT: flag reset %s\n",
- status2txt(BT_STATUS));
+ dev_dbg(bt->io->dev, "flag reset %s\n", status2txt(BT_STATUS));
if (BT_STATUS & BT_H_BUSY)
BT_CONTROL(BT_H_BUSY); /* force clear */
BT_CONTROL(BT_CLR_WR_PTR); /* always reset */
BT_CONTROL(BT_B2H_ATN); /* some BMCs are stubborn */
BT_CONTROL(BT_CLR_RD_PTR); /* always reset */
if (bt_debug)
- printk(KERN_WARNING "IPMI BT: stale response %s; ",
+ dev_dbg(bt->io->dev, "stale response %s; ",
status2txt(BT_STATUS));
size = BMC2HOST;
for (i = 0; i < size ; i++)
BMC2HOST;
BT_CONTROL(BT_H_BUSY); /* now clear */
if (bt_debug)
- printk("drained %d bytes\n", size + 1);
+ pr_cont("drained %d bytes\n", size + 1);
}
static inline void write_all_bytes(struct si_sm_data *bt)
int i;
if (bt_debug & BT_DEBUG_MSG) {
- printk(KERN_WARNING "BT: write %d bytes seq=0x%02X",
+ dev_dbg(bt->io->dev, "write %d bytes seq=0x%02X",
bt->write_count, bt->seq);
for (i = 0; i < bt->write_count; i++)
- printk(" %02x", bt->write_data[i]);
- printk("\n");
+ pr_cont(" %02x", bt->write_data[i]);
+ pr_cont("\n");
}
for (i = 0; i < bt->write_count; i++)
HOST2BMC(bt->write_data[i]);
if (bt->read_count < 4 || bt->read_count >= IPMI_MAX_MSG_LENGTH) {
if (bt_debug & BT_DEBUG_MSG)
- printk(KERN_WARNING "BT: bad raw rsp len=%d\n",
- bt->read_count);
+ dev_dbg(bt->io->dev,
+ "bad raw rsp len=%d\n", bt->read_count);
bt->truncated = 1;
return 1; /* let next XACTION START clean it up */
}
if (bt_debug & BT_DEBUG_MSG) {
int max = bt->read_count;
- printk(KERN_WARNING "BT: got %d bytes seq=0x%02X",
- max, bt->read_data[2]);
+ dev_dbg(bt->io->dev,
+ "got %d bytes seq=0x%02X", max, bt->read_data[2]);
if (max > 16)
max = 16;
for (i = 0; i < max; i++)
- printk(KERN_CONT " %02x", bt->read_data[i]);
- printk(KERN_CONT "%s\n", bt->read_count == max ? "" : " ...");
+ pr_cont(" %02x", bt->read_data[i]);
+ pr_cont("%s\n", bt->read_count == max ? "" : " ...");
}
/* per the spec, the (NetFn[1], Seq[2], Cmd[3]) tuples must match */
return 1;
if (bt_debug & BT_DEBUG_MSG)
- printk(KERN_WARNING "IPMI BT: bad packet: "
- "want 0x(%02X, %02X, %02X) got (%02X, %02X, %02X)\n",
- bt->write_data[1] | 0x04, bt->write_data[2], bt->write_data[3],
- bt->read_data[1], bt->read_data[2], bt->read_data[3]);
+ dev_dbg(bt->io->dev,
+ "IPMI BT: bad packet: want 0x(%02X, %02X, %02X) got (%02X, %02X, %02X)\n",
+ bt->write_data[1] | 0x04, bt->write_data[2],
+ bt->write_data[3],
+ bt->read_data[1], bt->read_data[2], bt->read_data[3]);
return 0;
}
break;
}
- printk(KERN_WARNING "IPMI BT: %s in %s %s ", /* open-ended line */
- reason, STATE2TXT, STATUS2TXT);
+ dev_warn(bt->io->dev, "IPMI BT: %s in %s %s ", /* open-ended line */
+ reason, STATE2TXT, STATUS2TXT);
/*
* Per the IPMI spec, retries are based on the sequence number
*/
(bt->error_retries)++;
if (bt->error_retries < bt->BT_CAP_retries) {
- printk("%d retries left\n",
+ pr_cont("%d retries left\n",
bt->BT_CAP_retries - bt->error_retries);
bt->state = BT_STATE_RESTART;
return SI_SM_CALL_WITHOUT_DELAY;
}
- printk(KERN_WARNING "failed %d retries, sending error response\n",
- bt->BT_CAP_retries);
+ dev_warn(bt->io->dev, "failed %d retries, sending error response\n",
+ bt->BT_CAP_retries);
if (!bt->nonzero_status)
- printk(KERN_ERR "IPMI BT: stuck, try power cycle\n");
+ dev_err(bt->io->dev, "stuck, try power cycle\n");
/* this is most likely during insmod */
else if (bt->seq <= (unsigned char)(bt->BT_CAP_retries & 0xFF)) {
- printk(KERN_WARNING "IPMI: BT reset (takes 5 secs)\n");
+ dev_warn(bt->io->dev, "BT reset (takes 5 secs)\n");
bt->state = BT_STATE_RESET1;
return SI_SM_CALL_WITHOUT_DELAY;
}
status = BT_STATUS;
bt->nonzero_status |= status;
if ((bt_debug & BT_DEBUG_STATES) && (bt->state != last_printed)) {
- printk(KERN_WARNING "BT: %s %s TO=%ld - %ld \n",
+ dev_dbg(bt->io->dev, "BT: %s %s TO=%ld - %ld\n",
STATE2TXT,
STATUS2TXT,
bt->timeout,
entry = kmalloc(sizeof(*entry), GFP_KERNEL);
if (!entry) {
- printk(KERN_ERR "ipmi_devintf: Unable to create the"
- " ipmi class device link\n");
+ pr_err("ipmi_devintf: Unable to create the ipmi class device link\n");
return;
}
entry->dev = dev;
if (ipmi_major < 0)
return -EINVAL;
- printk(KERN_INFO "ipmi device interface\n");
+ pr_info("ipmi device interface\n");
ipmi_class = class_create(THIS_MODULE, "ipmi");
if (IS_ERR(ipmi_class)) {
- printk(KERN_ERR "ipmi: can't register device class\n");
+ pr_err("ipmi: can't register device class\n");
return PTR_ERR(ipmi_class);
}
rv = register_chrdev(ipmi_major, DEVICE_NAME, &ipmi_fops);
if (rv < 0) {
class_destroy(ipmi_class);
- printk(KERN_ERR "ipmi: can't get major %d\n", ipmi_major);
+ pr_err("ipmi: can't get major %d\n", ipmi_major);
return rv;
}
if (rv) {
unregister_chrdev(ipmi_major, DEVICE_NAME);
class_destroy(ipmi_class);
- printk(KERN_WARNING "ipmi: can't register smi watcher\n");
+ pr_warn("ipmi: can't register smi watcher\n");
return rv;
}
* allow autoloading of the IPMI drive based on SMBIOS entries.
*/
+#define pr_fmt(fmt) "%s" fmt, "ipmi:dmi: "
+#define dev_fmt pr_fmt
+
#include <linux/ipmi.h>
#include <linux/init.h>
#include <linux/dmi.h>
unsigned int num_r = 1, size;
struct property_entry p[5];
unsigned int pidx = 0;
- char *name, *override;
+ char *name;
int rv;
enum si_type si_type;
struct ipmi_dmi_info *info;
memset(p, 0, sizeof(p));
name = "dmi-ipmi-si";
- override = "ipmi_si";
switch (type) {
case IPMI_DMI_TYPE_SSIF:
name = "dmi-ipmi-ssif";
- override = "ipmi_ssif";
offset = 1;
size = 1;
si_type = SI_TYPE_INVALID;
si_type = SI_SMIC;
break;
default:
- pr_err("ipmi:dmi: Invalid IPMI type: %d\n", type);
+ pr_err("Invalid IPMI type: %d\n", type);
return;
}
info = kmalloc(sizeof(*info), GFP_KERNEL);
if (!info) {
- pr_warn("ipmi:dmi: Could not allocate dmi info\n");
+ pr_warn("Could not allocate dmi info\n");
} else {
info->si_type = si_type;
info->flags = flags;
pdev = platform_device_alloc(name, ipmi_dmi_nr);
if (!pdev) {
- pr_err("ipmi:dmi: Error allocation IPMI platform device\n");
+ pr_err("Error allocation IPMI platform device\n");
return;
}
- pdev->driver_override = kasprintf(GFP_KERNEL, "%s",
- override);
- if (!pdev->driver_override)
- goto err;
if (type == IPMI_DMI_TYPE_SSIF) {
p[pidx++] = PROPERTY_ENTRY_U16("i2c-addr", base_addr);
rv = platform_device_add_resources(pdev, r, num_r);
if (rv) {
- dev_err(&pdev->dev,
- "ipmi:dmi: Unable to add resources: %d\n", rv);
+ dev_err(&pdev->dev, "Unable to add resources: %d\n", rv);
goto err;
}
add_properties:
rv = platform_device_add_properties(pdev, p);
if (rv) {
- dev_err(&pdev->dev,
- "ipmi:dmi: Unable to add properties: %d\n", rv);
+ dev_err(&pdev->dev, "Unable to add properties: %d\n", rv);
goto err;
}
rv = platform_device_add(pdev);
if (rv) {
- dev_err(&pdev->dev, "ipmi:dmi: Unable to add device: %d\n", rv);
+ dev_err(&pdev->dev, "Unable to add device: %d\n", rv);
goto err;
}
slave_addr = data[DMI_IPMI_SLAVEADDR];
memcpy(&base_addr, data + DMI_IPMI_ADDR, sizeof(unsigned long));
+ if (!base_addr) {
+ pr_err("Base address is zero, assuming no IPMI interface\n");
+ return;
+ }
if (len >= DMI_IPMI_VER2_LENGTH) {
if (type == IPMI_DMI_TYPE_SSIF) {
offset = 0;
offset = 16;
break;
default:
- pr_err("ipmi:dmi: Invalid offset: 0\n");
+ pr_err("Invalid offset: 0\n");
return;
}
}
if (kcs_debug & KCS_DEBUG_MSG) {
printk(KERN_DEBUG "start_kcs_transaction -");
for (i = 0; i < size; i++)
- printk(" %02x", (unsigned char) (data [i]));
- printk("\n");
+ pr_cont(" %02x", data[i]);
+ pr_cont("\n");
}
kcs->error_retries = 0;
memcpy(kcs->write_data, data, size);
* Copyright 2002 MontaVista Software Inc.
*/
+#define pr_fmt(fmt) "%s" fmt, "IPMI message handler: "
+#define dev_fmt pr_fmt
+
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/poll.h>
#include <linux/workqueue.h>
#include <linux/uuid.h>
-#define PFX "IPMI message handler: "
-
#define IPMI_DRIVER_VERSION "39.2"
static struct ipmi_recv_msg *ipmi_alloc_recv_msg(void);
user->intf->addrinfo[channel].lun = LUN & 0x3;
release_ipmi_user(user, index);
- return 0;
+ return rv;
}
EXPORT_SYMBOL(ipmi_set_my_LUN);
list_move_tail(&msg->link, &msgs);
intf->waiting_events_count = 0;
if (intf->event_msg_printed) {
- dev_warn(intf->si_dev,
- PFX "Event queue no longer full\n");
+ dev_warn(intf->si_dev, "Event queue no longer full\n");
intf->event_msg_printed = 0;
}
|| (msg->msg.netfn != IPMI_NETFN_APP_RESPONSE)
|| (msg->msg.cmd != IPMI_GET_DEVICE_ID_CMD)) {
dev_warn(intf->si_dev,
- PFX "invalid device_id msg: addr_type=%d netfn=%x cmd=%x\n",
- msg->addr.addr_type, msg->msg.netfn, msg->msg.cmd);
+ "invalid device_id msg: addr_type=%d netfn=%x cmd=%x\n",
+ msg->addr.addr_type, msg->msg.netfn, msg->msg.cmd);
return;
}
rv = ipmi_demangle_device_id(msg->msg.netfn, msg->msg.cmd,
msg->msg.data, msg->msg.data_len, &intf->bmc->fetch_id);
if (rv) {
- dev_warn(intf->si_dev,
- PFX "device id demangle failed: %d\n", rv);
+ dev_warn(intf->si_dev, "device id demangle failed: %d\n", rv);
intf->bmc->dyn_id_set = 0;
} else {
/*
mutex_unlock(&bmc->dyn_mutex);
dev_info(intf->si_dev,
- "ipmi: interfacing existing BMC (man_id: 0x%6.6x,"
- " prod_id: 0x%4.4x, dev_id: 0x%2.2x)\n",
+ "interfacing existing BMC (man_id: 0x%6.6x, prod_id: 0x%4.4x, dev_id: 0x%2.2x)\n",
bmc->id.manufacturer_id,
bmc->id.product_id,
bmc->id.device_id);
rv = platform_device_register(&bmc->pdev);
if (rv) {
dev_err(intf->si_dev,
- PFX " Unable to register bmc device: %d\n",
+ "Unable to register bmc device: %d\n",
rv);
goto out_list_del;
}
*/
rv = sysfs_create_link(&intf->si_dev->kobj, &bmc->pdev.dev.kobj, "bmc");
if (rv) {
- dev_err(intf->si_dev,
- PFX "Unable to create bmc symlink: %d\n", rv);
+ dev_err(intf->si_dev, "Unable to create bmc symlink: %d\n", rv);
goto out_put_bmc;
}
intf->my_dev_name = kasprintf(GFP_KERNEL, "ipmi%d", intf_num);
if (!intf->my_dev_name) {
rv = -ENOMEM;
- dev_err(intf->si_dev,
- PFX "Unable to allocate link from BMC: %d\n", rv);
+ dev_err(intf->si_dev, "Unable to allocate link from BMC: %d\n",
+ rv);
goto out_unlink1;
}
if (rv) {
kfree(intf->my_dev_name);
intf->my_dev_name = NULL;
- dev_err(intf->si_dev,
- PFX "Unable to create symlink to bmc: %d\n", rv);
+ dev_err(intf->si_dev, "Unable to create symlink to bmc: %d\n",
+ rv);
goto out_free_my_dev_name;
}
if (msg->msg.data_len < 17) {
bmc->dyn_guid_set = 0;
dev_warn(intf->si_dev,
- PFX "The GUID response from the BMC was too short, it was %d but should have been 17. Assuming GUID is not available.\n",
+ "The GUID response from the BMC was too short, it was %d but should have been 17. Assuming GUID is not available.\n",
msg->msg.data_len);
goto out;
}
if (rv) {
/* Got an error somehow, just give up. */
dev_warn(intf->si_dev,
- PFX "Error sending channel information for channel %d: %d\n",
+ "Error sending channel information for channel %d: %d\n",
intf->curr_channel, rv);
intf->channel_list = intf->wchannels + set;
* message.
*/
dev_warn(intf->si_dev,
- PFX "Event queue full, discarding incoming events\n");
+ "Event queue full, discarding incoming events\n");
intf->event_msg_printed = 1;
}
recv_msg = (struct ipmi_recv_msg *) msg->user_data;
if (recv_msg == NULL) {
dev_warn(intf->si_dev,
- "IPMI message received with no owner. This could be because of a malformed message, or because of a hardware error. Contact your hardware vender for assistance\n");
+ "IPMI message received with no owner. This could be because of a malformed message, or because of a hardware error. Contact your hardware vendor for assistance.\n");
return 0;
}
if (msg->rsp_size < 2) {
/* Message is too small to be correct. */
dev_warn(intf->si_dev,
- PFX "BMC returned to small a message for netfn %x cmd %x, got %d bytes\n",
+ "BMC returned too small a message for netfn %x cmd %x, got %d bytes\n",
(msg->data[0] >> 2) | 1, msg->data[1], msg->rsp_size);
/* Generate an error response for the message. */
* marginally correct.
*/
dev_warn(intf->si_dev,
- PFX "BMC returned incorrect response, expected netfn %x cmd %x, got netfn %x cmd %x\n",
+ "BMC returned incorrect response, expected netfn %x cmd %x, got netfn %x cmd %x\n",
(msg->data[0] >> 2) | 1, msg->data[1],
msg->rsp[0] >> 2, msg->rsp[1]);
rv = driver_register(&ipmidriver.driver);
if (rv) {
- pr_err(PFX "Could not register IPMI driver\n");
+ pr_err("Could not register IPMI driver\n");
return rv;
}
- pr_info("ipmi message handler version " IPMI_DRIVER_VERSION "\n");
+ pr_info("version " IPMI_DRIVER_VERSION "\n");
timer_setup(&ipmi_timer, ipmi_timeout, 0);
mod_timer(&ipmi_timer, jiffies + IPMI_TIMEOUT_JIFFIES);
/* Check for buffer leaks. */
count = atomic_read(&smi_msg_inuse_count);
if (count != 0)
- pr_warn(PFX "SMI message count %d at exit\n", count);
+ pr_warn("SMI message count %d at exit\n", count);
count = atomic_read(&recv_msg_inuse_count);
if (count != 0)
- pr_warn(PFX "recv message count %d at exit\n", count);
+ pr_warn("recv message count %d at exit\n", count);
}
module_exit(cleanup_ipmi);
struct ipmi_smi_powernv {
u64 interface_id;
- ipmi_smi_t intf;
+ struct ipmi_smi *intf;
unsigned int irq;
/**
struct opal_ipmi_msg *opal_msg;
};
-static int ipmi_powernv_start_processing(void *send_info, ipmi_smi_t intf)
+static int ipmi_powernv_start_processing(void *send_info, struct ipmi_smi *intf)
{
struct ipmi_smi_powernv *smi = send_info;
*
* Copyright 2002,2004 MontaVista Software Inc.
*/
+
+#define pr_fmt(fmt) "IPMI poweroff: " fmt
+
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/proc_fs.h>
#include <linux/ipmi.h>
#include <linux/ipmi_smi.h>
-#define PFX "IPMI poweroff: "
-
static void ipmi_po_smi_gone(int if_num);
static void ipmi_po_new_smi(int if_num, struct device *device);
smi_addr.channel = IPMI_BMC_CHANNEL;
smi_addr.lun = 0;
- printk(KERN_INFO PFX "PPS powerdown hook used");
+ pr_info("PPS powerdown hook used\n");
send_msg.netfn = IPMI_NETFN_OEM;
send_msg.cmd = IPMI_ATCA_PPS_GRACEFUL_RESTART;
rv = ipmi_request_in_rc_mode(user,
(struct ipmi_addr *) &smi_addr,
&send_msg);
- if (rv && rv != IPMI_UNKNOWN_ERR_COMPLETION_CODE) {
- printk(KERN_ERR PFX "Unable to send ATCA ,"
- " IPMI error 0x%x\n", rv);
- }
+ if (rv && rv != IPMI_UNKNOWN_ERR_COMPLETION_CODE)
+ pr_err("Unable to send ATCA, IPMI error 0x%x\n", rv);
+
return;
}
(struct ipmi_addr *) &smi_addr,
&send_msg);
- printk(KERN_INFO PFX "ATCA Detect mfg 0x%X prod 0x%X\n",
- mfg_id, prod_id);
+ pr_info("ATCA Detect mfg 0x%X prod 0x%X\n", mfg_id, prod_id);
if ((mfg_id == IPMI_MOTOROLA_MANUFACTURER_ID)
&& (prod_id == IPMI_MOTOROLA_PPS_IPMC_PRODUCT_ID)) {
- printk(KERN_INFO PFX
- "Installing Pigeon Point Systems Poweroff Hook\n");
+ pr_info("Installing Pigeon Point Systems Poweroff Hook\n");
atca_oem_poweroff_hook = pps_poweroff_atca;
}
return !rv;
smi_addr.channel = IPMI_BMC_CHANNEL;
smi_addr.lun = 0;
- printk(KERN_INFO PFX "Powering down via ATCA power command\n");
+ pr_info("Powering down via ATCA power command\n");
/*
* Power down
* return code
*/
if (rv && rv != IPMI_UNKNOWN_ERR_COMPLETION_CODE) {
- printk(KERN_ERR PFX "Unable to send ATCA powerdown message,"
- " IPMI error 0x%x\n", rv);
+ pr_err("Unable to send ATCA powerdown message, IPMI error 0x%x\n",
+ rv);
goto out;
}
smi_addr.channel = IPMI_BMC_CHANNEL;
smi_addr.lun = 0;
- printk(KERN_INFO PFX "Powering down via CPI1 power command\n");
+ pr_info("Powering down via CPI1 power command\n");
/*
* Get IPMI ipmb address
smi_addr.lun = 0;
powercyclefailed:
- printk(KERN_INFO PFX "Powering %s via IPMI chassis control command\n",
+ pr_info("Powering %s via IPMI chassis control command\n",
(poweroff_powercycle ? "cycle" : "down"));
/*
if (rv) {
if (poweroff_powercycle) {
/* power cycle failed, default to power down */
- printk(KERN_ERR PFX "Unable to send chassis power " \
- "cycle message, IPMI error 0x%x\n", rv);
+ pr_err("Unable to send chassis power cycle message, IPMI error 0x%x\n",
+ rv);
poweroff_powercycle = 0;
goto powercyclefailed;
}
- printk(KERN_ERR PFX "Unable to send chassis power " \
- "down message, IPMI error 0x%x\n", rv);
+ pr_err("Unable to send chassis power down message, IPMI error 0x%x\n",
+ rv);
}
}
rv = ipmi_create_user(if_num, &ipmi_poweroff_handler, NULL,
&ipmi_user);
if (rv) {
- printk(KERN_ERR PFX "could not create IPMI user, error %d\n",
- rv);
+ pr_err("could not create IPMI user, error %d\n", rv);
return;
}
(struct ipmi_addr *) &smi_addr,
&send_msg);
if (rv) {
- printk(KERN_ERR PFX "Unable to send IPMI get device id info,"
- " IPMI error 0x%x\n", rv);
+ pr_err("Unable to send IPMI get device id info, IPMI error 0x%x\n",
+ rv);
goto out_err;
}
if (halt_recv_msg.msg.data_len < 12) {
- printk(KERN_ERR PFX "(chassis) IPMI get device id info too,"
- " short, was %d bytes, needed %d bytes\n",
+ pr_err("(chassis) IPMI get device id info too short, was %d bytes, needed %d bytes\n",
halt_recv_msg.msg.data_len, 12);
goto out_err;
}
}
out_err:
- printk(KERN_ERR PFX "Unable to find a poweroff function that"
- " will work, giving up\n");
+ pr_err("Unable to find a poweroff function that will work, giving up\n");
ipmi_destroy_user(ipmi_user);
return;
found:
- printk(KERN_INFO PFX "Found a %s style poweroff function\n",
- poweroff_functions[i].platform_type);
+ pr_info("Found a %s style poweroff function\n",
+ poweroff_functions[i].platform_type);
specific_poweroff_func = poweroff_functions[i].poweroff_func;
old_poweroff_func = pm_power_off;
pm_power_off = ipmi_poweroff_function;
{
int rv;
- printk(KERN_INFO "Copyright (C) 2004 MontaVista Software -"
- " IPMI Powerdown via sys_reboot.\n");
+ pr_info("Copyright (C) 2004 MontaVista Software - IPMI Powerdown via sys_reboot\n");
if (poweroff_powercycle)
- printk(KERN_INFO PFX "Power cycle is enabled.\n");
+ pr_info("Power cycle is enabled\n");
#ifdef CONFIG_PROC_FS
ipmi_table_header = register_sysctl_table(ipmi_root_table);
if (!ipmi_table_header) {
- printk(KERN_ERR PFX "Unable to register powercycle sysctl\n");
+ pr_err("Unable to register powercycle sysctl\n");
rv = -ENOMEM;
goto out_err;
}
#ifdef CONFIG_PROC_FS
if (rv) {
unregister_sysctl_table(ipmi_table_header);
- printk(KERN_ERR PFX "Unable to register SMI watcher: %d\n", rv);
+ pr_err("Unable to register SMI watcher: %d\n", rv);
goto out_err;
}
if (ready) {
rv = ipmi_destroy_user(ipmi_user);
if (rv)
- printk(KERN_ERR PFX "could not cleanup the IPMI"
- " user: 0x%x\n", rv);
+ pr_err("could not cleanup the IPMI user: 0x%x\n", rv);
pm_power_off = old_poweroff_func;
}
}
// SPDX-License-Identifier: GPL-2.0+
+#define pr_fmt(fmt) "ipmi_hardcode: " fmt
+
#include <linux/moduleparam.h>
#include "ipmi_si.h"
-#define PFX "ipmi_hardcode: "
/*
* There can be 4 IO ports passed in (with or without IRQs), 4 addresses,
* a default IO port, and 1 ACPI/SPMI address. That sets SI_MAX_DRIVERS.
continue;
io.addr_source = SI_HARDCODED;
- pr_info(PFX "probing via hardcoded address\n");
+ pr_info("probing via hardcoded address\n");
if (!si_type[i] || strcmp(si_type[i], "kcs") == 0) {
io.si_type = SI_KCS;
} else if (strcmp(si_type[i], "bt") == 0) {
io.si_type = SI_BT;
} else {
- pr_warn(PFX "Interface type specified for interface %d, was invalid: %s\n",
+ pr_warn("Interface type specified for interface %d, was invalid: %s\n",
i, si_type[i]);
continue;
}
io.addr_data = addrs[i];
io.addr_type = IPMI_MEM_ADDR_SPACE;
} else {
- pr_warn(PFX "Interface type specified for interface %d, but port and address were not set or set to zero.\n",
+ pr_warn("Interface type specified for interface %d, but port and address were not set or set to zero\n",
i);
continue;
}
* Handling for dynamically adding/removing IPMI devices through
* a module parameter (and thus sysfs).
*/
+
+#define pr_fmt(fmt) "ipmi_hotmod: " fmt
+
#include <linux/moduleparam.h>
#include <linux/ipmi.h>
#include "ipmi_si.h"
-#define PFX "ipmi_hotmod: "
-
static int hotmod_handler(const char *val, const struct kernel_param *kp);
module_param_call(hotmod, hotmod_handler, NULL, NULL, 0200);
s = strchr(*curr, ',');
if (!s) {
- pr_warn(PFX "No hotmod %s given.\n", name);
+ pr_warn("No hotmod %s given\n", name);
return -EINVAL;
}
*s = '\0';
}
}
- pr_warn(PFX "Invalid hotmod %s '%s'\n", name, *curr);
+ pr_warn("Invalid hotmod %s '%s'\n", name, *curr);
return -EINVAL;
}
if (strcmp(curr, name) == 0) {
if (!option) {
- pr_warn(PFX "No option given for '%s'\n", curr);
+ pr_warn("No option given for '%s'\n", curr);
return -EINVAL;
}
*val = simple_strtoul(option, &n, 0);
if ((*n != '\0') || (*option == '\0')) {
- pr_warn(PFX "Bad option given for '%s'\n", curr);
+ pr_warn("Bad option given for '%s'\n", curr);
return -EINVAL;
}
return 1;
}
addr = simple_strtoul(curr, &n, 0);
if ((*n != '\0') || (*curr == '\0')) {
- pr_warn(PFX "Invalid hotmod address '%s'\n", curr);
+ pr_warn("Invalid hotmod address '%s'\n", curr);
break;
}
continue;
rv = -EINVAL;
- pr_warn(PFX "Invalid hotmod option '%s'\n", curr);
+ pr_warn("Invalid hotmod option '%s'\n", curr);
goto out;
}
* and drives the real SMI state machine.
*/
+#define pr_fmt(fmt) "ipmi_si: " fmt
+
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/sched.h>
#include <linux/string.h>
#include <linux/ctype.h>
-#define PFX "ipmi_si: "
-
/* Measure times between events in the driver. */
#undef DEBUG_TIMING
{
struct timespec64 t;
- getnstimeofday64(&t);
+ ktime_get_ts64(&t);
pr_debug("**%s: %lld.%9.9ld\n", msg, (long long) t.tv_sec, t.tv_nsec);
}
#else
if (max_busy_us == 0 || smi_result != SI_SM_CALL_WITH_DELAY)
ipmi_si_set_not_busy(busy_until);
else if (!ipmi_si_is_busy(busy_until)) {
- getnstimeofday64(busy_until);
+ ktime_get_ts64(busy_until);
timespec64_add_ns(busy_until, max_busy_us*NSEC_PER_USEC);
} else {
struct timespec64 now;
- getnstimeofday64(&now);
+ ktime_get_ts64(&now);
if (unlikely(timespec64_compare(&now, busy_until) > 0)) {
ipmi_si_set_not_busy(busy_until);
return 0;
rv = wait_for_msg_done(smi_info);
if (rv) {
- pr_warn(PFX "Error getting response from get global enables command, the event buffer is not enabled.\n");
+ pr_warn("Error getting response from get global enables command, the event buffer is not enabled\n");
goto out;
}
resp[0] != (IPMI_NETFN_APP_REQUEST | 1) << 2 ||
resp[1] != IPMI_GET_BMC_GLOBAL_ENABLES_CMD ||
resp[2] != 0) {
- pr_warn(PFX "Invalid return from get global enables command, cannot enable the event buffer.\n");
+ pr_warn("Invalid return from get global enables command, cannot enable the event buffer\n");
rv = -EINVAL;
goto out;
}
rv = wait_for_msg_done(smi_info);
if (rv) {
- pr_warn(PFX "Error getting response from set global, enables command, the event buffer is not enabled.\n");
+ pr_warn("Error getting response from set global, enables command, the event buffer is not enabled\n");
goto out;
}
if (resp_len < 3 ||
resp[0] != (IPMI_NETFN_APP_REQUEST | 1) << 2 ||
resp[1] != IPMI_SET_BMC_GLOBAL_ENABLES_CMD) {
- pr_warn(PFX "Invalid return from get global, enables command, not enable the event buffer.\n");
+ pr_warn("Invalid return from get global, enables command, not enable the event buffer\n");
rv = -EINVAL;
goto out;
}
}
}
- pr_info(PFX "Adding %s-specified %s state machine\n",
+ pr_info("Adding %s-specified %s state machine\n",
ipmi_addr_src_to_str(new_smi->io.addr_source),
si_to_str[new_smi->io.si_type]);
int i;
char *init_name = NULL;
- pr_info(PFX "Trying %s-specified %s state machine at %s address 0x%lx, slave address 0x%x, irq %d\n",
+ pr_info("Trying %s-specified %s state machine at %s address 0x%lx, slave address 0x%x, irq %d\n",
ipmi_addr_src_to_str(new_smi->io.addr_source),
si_to_str[new_smi->io.si_type],
addr_space_to_str[new_smi->io.addr_type],
new_smi->pdev = platform_device_alloc("ipmi_si",
new_smi->si_num);
if (!new_smi->pdev) {
- pr_err(PFX "Unable to allocate platform device\n");
+ pr_err("Unable to allocate platform device\n");
rv = -ENOMEM;
goto out_err;
}
if (initialized)
return 0;
- pr_info("IPMI System Interface driver.\n");
+ pr_info("IPMI System Interface driver\n");
/* If the user gave us a device, they presumably want us to use it */
if (!ipmi_si_hardcode_find_bmc())
if (unload_when_empty && list_empty(&smi_infos)) {
mutex_unlock(&smi_infos_lock);
cleanup_ipmi_si();
- pr_warn(PFX "Unable to find any System Interface(s)\n");
+ pr_warn("Unable to find any System Interface(s)\n");
return -ENODEV;
} else {
mutex_unlock(&smi_infos_lock);
static void mem_outq(const struct si_sm_io *io, unsigned int offset,
unsigned char b)
{
- writeq(b << io->regshift, (io->addr)+(offset * io->regspacing));
+ writeq((u64)b << io->regshift, (io->addr)+(offset * io->regspacing));
}
#endif
*
* Handling for IPMI devices on the PCI bus.
*/
+
+#define pr_fmt(fmt) "ipmi_pci: " fmt
+
#include <linux/module.h>
#include <linux/pci.h>
#include "ipmi_si.h"
-#define PFX "ipmi_pci: "
-
static bool pci_registered;
static bool si_trypci = true;
MODULE_PARM_DESC(trypci, "Setting this to zero will disable the"
" default scan of the interfaces identified via pci");
-#define PCI_CLASS_SERIAL_IPMI 0x0c07
-#define PCI_CLASS_SERIAL_IPMI_SMIC 0x0c0700
-#define PCI_CLASS_SERIAL_IPMI_KCS 0x0c0701
-#define PCI_CLASS_SERIAL_IPMI_BT 0x0c0702
-
#define PCI_DEVICE_ID_HP_MMC 0x121A
static void ipmi_pci_cleanup(struct si_sm_io *io)
for (regspacing = DEFAULT_REGSPACING; regspacing <= 16;) {
io->regspacing = regspacing;
if (io->io_setup(io)) {
- dev_err(io->dev,
- "Could not setup I/O space\n");
+ dev_err(io->dev, "Could not setup I/O space\n");
return DEFAULT_REGSPACING;
}
/* write invalid cmd */
}
io.addr_data = pci_resource_start(pdev, 0);
+ io.dev = &pdev->dev;
+
io.regspacing = ipmi_pci_probe_regspacing(&io);
io.regsize = DEFAULT_REGSIZE;
io.regshift = 0;
if (io.irq)
io.irq_setup = ipmi_std_irq_setup;
- io.dev = &pdev->dev;
-
dev_info(&pdev->dev, "%pR regsize %d spacing %d irq %d\n",
- &pdev->resource[0], io.regsize, io.regspacing, io.irq);
+ &pdev->resource[0], io.regsize, io.regspacing, io.irq);
rv = ipmi_si_add_smi(&io);
if (rv)
if (si_trypci) {
int rv = pci_register_driver(&ipmi_pci_driver);
if (rv)
- pr_err(PFX "Unable to register PCI driver: %d\n", rv);
+ pr_err("Unable to register PCI driver: %d\n", rv);
else
pci_registered = true;
}
* Handling for platform devices in IPMI (ACPI, OF, and things
* coming from the platform.
*/
+
+#define pr_fmt(fmt) "ipmi_platform: " fmt
+#define dev_fmt pr_fmt
+
#include <linux/types.h>
#include <linux/module.h>
#include <linux/of_device.h>
#include "ipmi_si.h"
#include "ipmi_dmi.h"
-#define PFX "ipmi_platform: "
-
static bool si_tryplatform = true;
#ifdef CONFIG_ACPI
static bool si_tryacpi = true;
memset(&io, 0, sizeof(io));
io.addr_source = addr_source;
- dev_info(&pdev->dev, PFX "probing via %s\n",
+ dev_info(&pdev->dev, "probing via %s\n",
ipmi_addr_src_to_str(addr_source));
switch (type) {
ret = of_address_to_resource(np, 0, &resource);
if (ret) {
- dev_warn(&pdev->dev, PFX "invalid address from OF\n");
+ dev_warn(&pdev->dev, "invalid address from OF\n");
return ret;
}
regsize = of_get_property(np, "reg-size", &proplen);
if (regsize && proplen != 4) {
- dev_warn(&pdev->dev, PFX "invalid regsize from OF\n");
+ dev_warn(&pdev->dev, "invalid regsize from OF\n");
return -EINVAL;
}
regspacing = of_get_property(np, "reg-spacing", &proplen);
if (regspacing && proplen != 4) {
- dev_warn(&pdev->dev, PFX "invalid regspacing from OF\n");
+ dev_warn(&pdev->dev, "invalid regspacing from OF\n");
return -EINVAL;
}
regshift = of_get_property(np, "reg-shift", &proplen);
if (regshift && proplen != 4) {
- dev_warn(&pdev->dev, PFX "invalid regshift from OF\n");
+ dev_warn(&pdev->dev, "invalid regshift from OF\n");
return -EINVAL;
}
memset(&io, 0, sizeof(io));
io.addr_source = SI_ACPI;
- dev_info(&pdev->dev, PFX "probing via ACPI\n");
+ dev_info(&pdev->dev, "probing via ACPI\n");
io.addr_info.acpi_info.acpi_handle = handle;
return ipmi_si_remove_by_dev(&pdev->dev);
}
+static const struct platform_device_id si_plat_ids[] = {
+ { "dmi-ipmi-si", 0 },
+ { }
+};
+
struct platform_driver ipmi_platform_driver = {
.driver = {
.name = DEVICE_NAME,
},
.probe = ipmi_probe,
.remove = ipmi_remove,
+ .id_table = si_plat_ids
};
void ipmi_si_platform_init(void)
{
int rv = platform_driver_register(&ipmi_platform_driver);
if (rv)
- pr_err(PFX "Unable to register driver: %d\n", rv);
+ pr_err("Unable to register driver: %d\n", rv);
}
void ipmi_si_platform_shutdown(void)
if (smic_debug & SMIC_DEBUG_MSG) {
printk(KERN_DEBUG "start_smic_transaction -");
for (i = 0; i < size; i++)
- printk(" %02x", (unsigned char) data[i]);
- printk("\n");
+ pr_cont(" %02x", data[i]);
+ pr_cont("\n");
}
smic->error_retries = 0;
memcpy(smic->write_data, data, size);
if (smic_debug & SMIC_DEBUG_MSG) {
printk(KERN_DEBUG "smic_get result -");
for (i = 0; i < smic->read_pos; i++)
- printk(" %02x", smic->read_data[i]);
- printk("\n");
+ pr_cont(" %02x", smic->read_data[i]);
+ pr_cont("\n");
}
if (length < smic->read_pos) {
smic->read_pos = length;
(smic->error_retries)++;
if (smic->error_retries > SMIC_MAX_ERROR_RETRIES) {
if (smic_debug & SMIC_DEBUG_ENABLE)
- printk(KERN_WARNING
- "ipmi_smic_drv: smic hosed: %s\n", reason);
+ pr_warn("ipmi_smic_drv: smic hosed: %s\n", reason);
smic->state = SMIC_HOSED;
} else {
smic->write_count = smic->orig_write_count;
if (smic->state != SMIC_IDLE) {
if (smic_debug & SMIC_DEBUG_STATES)
printk(KERN_DEBUG
- "smic_event - smic->smic_timeout = %ld,"
- " time = %ld\n",
+ "smic_event - smic->smic_timeout = %ld, time = %ld\n",
smic->smic_timeout, time);
/*
* FIXME: smic_event is sometimes called with time >
status = read_smic_status(smic);
if (smic_debug & SMIC_DEBUG_STATES)
- printk(KERN_DEBUG
- "smic_event - state = %d, flags = 0x%02x,"
- " status = 0x%02x\n",
+ printk(KERN_DEBUG "smic_event - state = %d, flags = 0x%02x, status = 0x%02x\n",
smic->state, flags, status);
switch (smic->state) {
data = read_smic_data(smic);
if (data != 0) {
if (smic_debug & SMIC_DEBUG_ENABLE)
- printk(KERN_DEBUG
- "SMIC_WRITE_END: data = %02x\n", data);
+ printk(KERN_DEBUG "SMIC_WRITE_END: data = %02x\n",
+ data);
start_error_recovery(smic,
"state = SMIC_WRITE_END, "
"data != SUCCESS");
/* data register holds an error code */
if (data != 0) {
if (smic_debug & SMIC_DEBUG_ENABLE)
- printk(KERN_DEBUG
- "SMIC_READ_END: data = %02x\n", data);
+ printk(KERN_DEBUG "SMIC_READ_END: data = %02x\n",
+ data);
start_error_recovery(smic,
"state = SMIC_READ_END, "
"data != SUCCESS");
* interface into the I2C driver, I believe.
*/
+#define pr_fmt(fmt) "ipmi_ssif: " fmt
+
#if defined(MODVERSIONS)
#include <linux/modversions.h>
#endif
#include "ipmi_si_sm.h"
#include "ipmi_dmi.h"
-#define PFX "ipmi_ssif: "
#define DEVICE_NAME "ipmi_ssif"
#define IPMI_GET_SYSTEM_INTERFACE_CAPABILITIES_CMD 0x57
#define SSIF_IPMI_REQUEST 2
#define SSIF_IPMI_MULTI_PART_REQUEST_START 6
#define SSIF_IPMI_MULTI_PART_REQUEST_MIDDLE 7
+#define SSIF_IPMI_MULTI_PART_REQUEST_END 8
#define SSIF_IPMI_RESPONSE 3
#define SSIF_IPMI_MULTI_PART_RESPONSE_MIDDLE 9
/* Info from SSIF cmd */
unsigned char max_xmit_msg_size;
unsigned char max_recv_msg_size;
+ bool cmd8_works; /* See test_multipart_messages() for details. */
unsigned int multi_support;
int supports_pec;
{
if (msg->rsp_size < 0) {
return_hosed_msg(ssif_info, msg);
- pr_err(PFX
- "Malformed message in deliver_recv_msg: rsp_size = %d\n",
- msg->rsp_size);
+ pr_err("%s: Malformed message: rsp_size = %d\n",
+ __func__, msg->rsp_size);
} else {
ipmi_smi_msg_received(ssif_info->intf, msg);
}
flags = ipmi_ssif_lock_cond(ssif_info, &oflags);
ssif_info->waiting_alert = true;
ssif_info->rtc_us_timer = SSIF_MSG_USEC;
- mod_timer(&ssif_info->retry_timer,
- jiffies + SSIF_MSG_JIFFIES);
+ if (!ssif_info->stopping)
+ mod_timer(&ssif_info->retry_timer,
+ jiffies + SSIF_MSG_JIFFIES);
ipmi_ssif_unlock_cond(ssif_info, flags);
return;
}
if (len == 0) {
result = -EIO;
if (ssif_info->ssif_debug & SSIF_DEBUG_MSG)
- pr_info(PFX "Middle message with no data\n");
+ pr_info("Middle message with no data\n");
goto continue_op;
}
I2C_SMBUS_BLOCK_DATA);
if (rv < 0) {
if (ssif_info->ssif_debug & SSIF_DEBUG_MSG)
- pr_info(PFX
- "Error from ssif_i2c_send\n");
+ pr_info("Error from ssif_i2c_send\n");
result = -EIO;
} else
continue_op:
if (ssif_info->ssif_debug & SSIF_DEBUG_STATE)
- pr_info(PFX "DONE 1: state = %d, result=%d.\n",
+ pr_info("DONE 1: state = %d, result=%d\n",
ssif_info->ssif_state, result);
flags = ipmi_ssif_lock_cond(ssif_info, &oflags);
*/
ssif_info->ssif_state = SSIF_NORMAL;
ipmi_ssif_unlock_cond(ssif_info, flags);
- pr_warn(PFX "Error getting flags: %d %d, %x\n",
- result, len, (len >= 3) ? data[2] : 0);
+ pr_warn("Error getting flags: %d %d, %x\n",
+ result, len, (len >= 3) ? data[2] : 0);
} else if (data[0] != (IPMI_NETFN_APP_REQUEST | 1) << 2
|| data[1] != IPMI_GET_MSG_FLAGS_CMD) {
/*
* response to a previous command.
*/
ipmi_ssif_unlock_cond(ssif_info, flags);
- pr_warn(PFX "Invalid response getting flags: %x %x\n",
+ pr_warn("Invalid response getting flags: %x %x\n",
data[0], data[1]);
} else {
ssif_inc_stat(ssif_info, flag_fetches);
/* We cleared the flags. */
if ((result < 0) || (len < 3) || (data[2] != 0)) {
/* Error clearing flags */
- pr_warn(PFX "Error clearing flags: %d %d, %x\n",
- result, len, (len >= 3) ? data[2] : 0);
+ pr_warn("Error clearing flags: %d %d, %x\n",
+ result, len, (len >= 3) ? data[2] : 0);
} else if (data[0] != (IPMI_NETFN_APP_REQUEST | 1) << 2
|| data[1] != IPMI_CLEAR_MSG_FLAGS_CMD) {
- pr_warn(PFX "Invalid response clearing flags: %x %x\n",
+ pr_warn("Invalid response clearing flags: %x %x\n",
data[0], data[1]);
}
ssif_info->ssif_state = SSIF_NORMAL;
handle_flags(ssif_info, flags);
} else if (msg->rsp[0] != (IPMI_NETFN_APP_REQUEST | 1) << 2
|| msg->rsp[1] != IPMI_READ_EVENT_MSG_BUFFER_CMD) {
- pr_warn(PFX "Invalid response getting events: %x %x\n",
+ pr_warn("Invalid response getting events: %x %x\n",
msg->rsp[0], msg->rsp[1]);
msg->done(msg);
/* Take off the event flag. */
handle_flags(ssif_info, flags);
} else if (msg->rsp[0] != (IPMI_NETFN_APP_REQUEST | 1) << 2
|| msg->rsp[1] != IPMI_GET_MSG_CMD) {
- pr_warn(PFX "Invalid response clearing flags: %x %x\n",
+ pr_warn("Invalid response clearing flags: %x %x\n",
msg->rsp[0], msg->rsp[1]);
msg->done(msg);
ipmi_ssif_unlock_cond(ssif_info, flags);
if (ssif_info->ssif_debug & SSIF_DEBUG_STATE)
- pr_info(PFX "DONE 2: state = %d.\n", ssif_info->ssif_state);
+ pr_info("DONE 2: state = %d.\n", ssif_info->ssif_state);
}
static void msg_written_handler(struct ssif_info *ssif_info, int result,
ssif_inc_stat(ssif_info, send_errors);
if (ssif_info->ssif_debug & SSIF_DEBUG_MSG)
- pr_info(PFX
- "Out of retries in msg_written_handler\n");
+ pr_info("%s: Out of retries\n", __func__);
msg_done_handler(ssif_info, -EIO, NULL, 0);
return;
}
* in the SSIF_MULTI_n_PART case in the probe function
* for details on the intricacies of this.
*/
- int left;
+ int left, to_write;
unsigned char *data_to_send;
+ unsigned char cmd;
ssif_inc_stat(ssif_info, sent_messages_parts);
left = ssif_info->multi_len - ssif_info->multi_pos;
- if (left > 32)
- left = 32;
+ to_write = left;
+ if (to_write > 32)
+ to_write = 32;
/* Length byte. */
- ssif_info->multi_data[ssif_info->multi_pos] = left;
+ ssif_info->multi_data[ssif_info->multi_pos] = to_write;
data_to_send = ssif_info->multi_data + ssif_info->multi_pos;
- ssif_info->multi_pos += left;
- if (left < 32)
- /*
- * Write is finished. Note that we must end
- * with a write of less than 32 bytes to
- * complete the transaction, even if it is
- * zero bytes.
- */
+ ssif_info->multi_pos += to_write;
+ cmd = SSIF_IPMI_MULTI_PART_REQUEST_MIDDLE;
+ if (ssif_info->cmd8_works) {
+ if (left == to_write) {
+ cmd = SSIF_IPMI_MULTI_PART_REQUEST_END;
+ ssif_info->multi_data = NULL;
+ }
+ } else if (to_write < 32) {
ssif_info->multi_data = NULL;
+ }
rv = ssif_i2c_send(ssif_info, msg_written_handler,
- I2C_SMBUS_WRITE,
- SSIF_IPMI_MULTI_PART_REQUEST_MIDDLE,
- data_to_send,
- I2C_SMBUS_BLOCK_DATA);
+ I2C_SMBUS_WRITE, cmd,
+ data_to_send, I2C_SMBUS_BLOCK_DATA);
if (rv < 0) {
/* request failed, just return the error. */
ssif_inc_stat(ssif_info, send_errors);
ssif_info->waiting_alert = true;
ssif_info->retries_left = SSIF_RECV_RETRIES;
ssif_info->rtc_us_timer = SSIF_MSG_PART_USEC;
- mod_timer(&ssif_info->retry_timer,
- jiffies + SSIF_MSG_PART_JIFFIES);
+ if (!ssif_info->stopping)
+ mod_timer(&ssif_info->retry_timer,
+ jiffies + SSIF_MSG_PART_JIFFIES);
ipmi_ssif_unlock_cond(ssif_info, flags);
}
}
ktime_get_real_ts64(&t);
pr_info("**Enqueue %02x %02x: %lld.%6.6ld\n",
- msg->data[0], msg->data[1],
- (long long) t.tv_sec, (long) t.tv_nsec / NSEC_PER_USEC);
+ msg->data[0], msg->data[1],
+ (long long)t.tv_sec, (long)t.tv_nsec / NSEC_PER_USEC);
}
}
return 0;
}
+static int read_response(struct i2c_client *client, unsigned char *resp)
+{
+ int ret = -ENODEV, retry_cnt = SSIF_RECV_RETRIES;
+
+ while (retry_cnt > 0) {
+ ret = i2c_smbus_read_block_data(client, SSIF_IPMI_RESPONSE,
+ resp);
+ if (ret > 0)
+ break;
+ msleep(SSIF_MSG_MSEC);
+ retry_cnt--;
+ if (retry_cnt <= 0)
+ break;
+ }
+
+ return ret;
+}
+
static int do_cmd(struct i2c_client *client, int len, unsigned char *msg,
int *resp_len, unsigned char *resp)
{
return -ENODEV;
}
- ret = -ENODEV;
- retry_cnt = SSIF_RECV_RETRIES;
- while (retry_cnt > 0) {
- ret = i2c_smbus_read_block_data(client, SSIF_IPMI_RESPONSE,
- resp);
- if (ret > 0)
- break;
- msleep(SSIF_MSG_MSEC);
- retry_cnt--;
- if (retry_cnt <= 0)
- break;
- }
-
+ ret = read_response(client, resp);
if (ret > 0) {
/* Validate that the response is correct. */
if (ret < 3 ||
(resp[0] != (msg[0] | (1 << 2))) ||
(resp[1] != msg[1]))
ret = -EINVAL;
- else {
+ else if (ret > IPMI_MAX_MSG_LENGTH) {
+ ret = -E2BIG;
+ } else {
*resp_len = ret;
ret = 0;
}
return slave_addr;
}
+static int start_multipart_test(struct i2c_client *client,
+ unsigned char *msg, bool do_middle)
+{
+ int retry_cnt = SSIF_SEND_RETRIES, ret;
+
+retry_write:
+ ret = i2c_smbus_write_block_data(client,
+ SSIF_IPMI_MULTI_PART_REQUEST_START,
+ 32, msg);
+ if (ret) {
+ retry_cnt--;
+ if (retry_cnt > 0)
+ goto retry_write;
+ dev_err(&client->dev, "Could not write multi-part start, though the BMC said it could handle it. Just limit sends to one part.\n");
+ return ret;
+ }
+
+ if (!do_middle)
+ return 0;
+
+ ret = i2c_smbus_write_block_data(client,
+ SSIF_IPMI_MULTI_PART_REQUEST_MIDDLE,
+ 32, msg + 32);
+ if (ret) {
+ dev_err(&client->dev, "Could not write multi-part middle, though the BMC said it could handle it. Just limit sends to one part.\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static void test_multipart_messages(struct i2c_client *client,
+ struct ssif_info *ssif_info,
+ unsigned char *resp)
+{
+ unsigned char msg[65];
+ int ret;
+ bool do_middle;
+
+ if (ssif_info->max_xmit_msg_size <= 32)
+ return;
+
+ do_middle = ssif_info->max_xmit_msg_size > 63;
+
+ memset(msg, 0, sizeof(msg));
+ msg[0] = IPMI_NETFN_APP_REQUEST << 2;
+ msg[1] = IPMI_GET_DEVICE_ID_CMD;
+
+ /*
+ * The specification is all messed up dealing with sending
+ * multi-part messages. Per what the specification says, it
+ * is impossible to send a message that is a multiple of 32
+ * bytes, except for 32 itself. It talks about a "start"
+ * transaction (cmd=6) that must be 32 bytes, "middle"
+ * transaction (cmd=7) that must be 32 bytes, and an "end"
+ * transaction. The "end" transaction is shown as cmd=7 in
+ * the text, but if that's the case there is no way to
+ * differentiate between a middle and end part except the
+ * length being less than 32. But there is a table at the far
+ * end of the section (that I had never noticed until someone
+ * pointed it out to me) that mentions it as cmd=8.
+ *
+ * After some thought, I think the example is wrong and the
+ * end transaction should be cmd=8. But some systems don't
+ * implement cmd=8, they use a zero-length end transaction,
+ * even though that violates the SMBus specification.
+ *
+ * So, to work around this, this code tests if cmd=8 works.
+ * If it does, then we use that. If not, it tests zero-
+ * byte end transactions. If that works, good. If not,
+ * we only allow 63-byte transactions max.
+ */
+
+ ret = start_multipart_test(client, msg, do_middle);
+ if (ret)
+ goto out_no_multi_part;
+
+ ret = i2c_smbus_write_block_data(client,
+ SSIF_IPMI_MULTI_PART_REQUEST_END,
+ 1, msg + 64);
+
+ if (!ret)
+ ret = read_response(client, resp);
+
+ if (ret > 0) {
+ /* End transactions work, we are good. */
+ ssif_info->cmd8_works = true;
+ return;
+ }
+
+ ret = start_multipart_test(client, msg, do_middle);
+ if (ret) {
+ dev_err(&client->dev, "Second multipart test failed.\n");
+ goto out_no_multi_part;
+ }
+
+ ret = i2c_smbus_write_block_data(client,
+ SSIF_IPMI_MULTI_PART_REQUEST_MIDDLE,
+ 0, msg + 64);
+ if (!ret)
+ ret = read_response(client, resp);
+ if (ret > 0)
+ /* Zero-size end parts work, use those. */
+ return;
+
+ /* Limit to 63 bytes and use a short middle command to mark the end. */
+ if (ssif_info->max_xmit_msg_size > 63)
+ ssif_info->max_xmit_msg_size = 63;
+ return;
+
+out_no_multi_part:
+ ssif_info->max_xmit_msg_size = 32;
+ return;
+}
+
/*
* Global enables we care about.
*/
slave_addr = find_slave_address(client, slave_addr);
- pr_info(PFX "Trying %s-specified SSIF interface at i2c address 0x%x, adapter %s, slave address 0x%x\n",
- ipmi_addr_src_to_str(ssif_info->addr_source),
- client->addr, client->adapter->name, slave_addr);
+ pr_info("Trying %s-specified SSIF interface at i2c address 0x%x, adapter %s, slave address 0x%x\n",
+ ipmi_addr_src_to_str(ssif_info->addr_source),
+ client->addr, client->adapter->name, slave_addr);
ssif_info->client = client;
i2c_set_clientdata(client, ssif_info);
if (!rv && (len >= 3) && (resp[2] == 0)) {
if (len < 7) {
if (ssif_dbg_probe)
- pr_info(PFX "SSIF info too short: %d\n", len);
+ pr_info("SSIF info too short: %d\n", len);
goto no_support;
}
break;
case SSIF_MULTI_n_PART:
- /*
- * The specification is rather confusing at
- * this point, but I think I understand what
- * is meant. At least I have a workable
- * solution. With multi-part messages, you
- * cannot send a message that is a multiple of
- * 32-bytes in length, because the start and
- * middle messages are 32-bytes and the end
- * message must be at least one byte. You
- * can't fudge on an extra byte, that would
- * screw up things like fru data writes. So
- * we limit the length to 63 bytes. That way
- * a 32-byte message gets sent as a single
- * part. A larger message will be a 32-byte
- * start and the next message is always going
- * to be 1-31 bytes in length. Not ideal, but
- * it should work.
- */
- if (ssif_info->max_xmit_msg_size > 63)
- ssif_info->max_xmit_msg_size = 63;
+ /* We take whatever size given, but do some testing. */
break;
default:
} else {
no_support:
/* Assume no multi-part or PEC support */
- pr_info(PFX "Error fetching SSIF: %d %d %2.2x, your system probably doesn't support this command so using defaults\n",
- rv, len, resp[2]);
+ pr_info("Error fetching SSIF: %d %d %2.2x, your system probably doesn't support this command so using defaults\n",
+ rv, len, resp[2]);
ssif_info->max_xmit_msg_size = 32;
ssif_info->max_recv_msg_size = 32;
ssif_info->supports_pec = 0;
}
+ test_multipart_messages(client, ssif_info, resp);
+
/* Make sure the NMI timeout is cleared. */
msg[0] = IPMI_NETFN_APP_REQUEST << 2;
msg[1] = IPMI_CLEAR_MSG_FLAGS_CMD;
msg[2] = WDT_PRE_TIMEOUT_INT;
rv = do_cmd(client, 3, msg, &len, resp);
if (rv || (len < 3) || (resp[2] != 0))
- pr_warn(PFX "Unable to clear message flags: %d %d %2.2x\n",
+ pr_warn("Unable to clear message flags: %d %d %2.2x\n",
rv, len, resp[2]);
/* Attempt to enable the event buffer. */
msg[1] = IPMI_GET_BMC_GLOBAL_ENABLES_CMD;
rv = do_cmd(client, 2, msg, &len, resp);
if (rv || (len < 4) || (resp[2] != 0)) {
- pr_warn(PFX "Error getting global enables: %d %d %2.2x\n",
+ pr_warn("Error getting global enables: %d %d %2.2x\n",
rv, len, resp[2]);
rv = 0; /* Not fatal */
goto found;
msg[2] = ssif_info->global_enables | IPMI_BMC_EVT_MSG_BUFF;
rv = do_cmd(client, 3, msg, &len, resp);
if (rv || (len < 2)) {
- pr_warn(PFX "Error setting global enables: %d %d %2.2x\n",
+ pr_warn("Error setting global enables: %d %d %2.2x\n",
rv, len, resp[2]);
rv = 0; /* Not fatal */
goto found;
msg[2] = ssif_info->global_enables | IPMI_BMC_RCV_MSG_INTR;
rv = do_cmd(client, 3, msg, &len, resp);
if (rv || (len < 2)) {
- pr_warn(PFX "Error setting global enables: %d %d %2.2x\n",
+ pr_warn("Error setting global enables: %d %d %2.2x\n",
rv, len, resp[2]);
rv = 0; /* Not fatal */
goto found;
&ssif_info->client->dev,
slave_addr);
if (rv) {
- pr_err(PFX "Unable to register device: error %d\n", rv);
+ pr_err("Unable to register device: error %d\n", rv);
goto out_remove_attr;
}
static unsigned short *ssif_address_list(void)
{
struct ssif_addr_info *info;
- unsigned int count = 0, i;
+ unsigned int count = 0, i = 0;
unsigned short *address_list;
list_for_each_entry(info, &ssif_infos, link)
if (!address_list)
return NULL;
- i = 0;
list_for_each_entry(info, &ssif_infos, link) {
unsigned short addr = info->binfo.addr;
int j;
for (j = 0; j < i; j++) {
if (address_list[j] == addr)
- goto skip_addr;
+ /* Found a dup. */
+ break;
}
- address_list[i] = addr;
-skip_addr:
- i++;
+ if (j == i) /* Didn't find it in the list. */
+ address_list[i++] = addr;
}
address_list[i] = I2C_CLIENT_END;
rv = device_property_read_u16(&pdev->dev, "i2c-addr", &i2c_addr);
if (rv) {
- dev_warn(&pdev->dev, PFX "No i2c-addr property\n");
+ dev_warn(&pdev->dev, "No i2c-addr property\n");
return -ENODEV;
}
return 0;
}
+static const struct platform_device_id ssif_plat_ids[] = {
+ { "dmi-ipmi-ssif", 0 },
+ { }
+};
+
static struct platform_driver ipmi_driver = {
.driver = {
.name = DEVICE_NAME,
},
.probe = ssif_platform_probe,
.remove = ssif_platform_remove,
+ .id_table = ssif_plat_ids
};
static int init_ipmi_ssif(void)
dbg[i], slave_addrs[i],
SI_HARDCODED, NULL);
if (rv)
- pr_err(PFX
- "Couldn't add hardcoded device at addr 0x%x\n",
+ pr_err("Couldn't add hardcoded device at addr 0x%x\n",
addr[i]);
}
if (ssif_trydmi) {
rv = platform_driver_register(&ipmi_driver);
if (rv)
- pr_err(PFX "Unable to register driver: %d\n", rv);
+ pr_err("Unable to register driver: %d\n", rv);
}
ssif_i2c_driver.address_list = ssif_address_list();
i2c_del_driver(&ssif_i2c_driver);
+ kfree(ssif_i2c_driver.address_list);
+
platform_driver_unregister(&ipmi_driver);
free_ssif_clients();
* Copyright 2002 MontaVista Software Inc.
*/
+#define pr_fmt(fmt) "IPMI Watchdog: " fmt
+
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/ipmi.h>
#define HAVE_DIE_NMI
#endif
-#define PFX "IPMI Watchdog: "
-
/*
* The IPMI command/response information for the watchdog timer.
*/
recv_msg,
1);
if (rv)
- pr_warn(PFX "set timeout error: %d\n", rv);
+ pr_warn("set timeout error: %d\n", rv);
else if (send_heartbeat_now)
*send_heartbeat_now = hbnow;
&send_heartbeat_now);
if (rv) {
atomic_sub(1, &panic_done_count);
- pr_warn(PFX "Unable to extend the watchdog timeout.");
+ pr_warn("Unable to extend the watchdog timeout\n");
} else {
if (send_heartbeat_now)
panic_halt_ipmi_heartbeat();
&recv_msg,
1);
if (rv) {
- pr_warn(PFX "heartbeat send failure: %d\n", rv);
+ pr_warn("heartbeat send failure: %d\n", rv);
return rv;
}
if (recv_msg.msg.data[0] == IPMI_WDOG_TIMER_NOT_INIT_RESP) {
timeout_retries++;
if (timeout_retries > 3) {
- pr_err(PFX ": Unable to restore the IPMI watchdog's settings, giving up.\n");
+ pr_err("Unable to restore the IPMI watchdog's settings, giving up\n");
rv = -EIO;
goto out;
}
*/
rv = _ipmi_set_timeout(IPMI_SET_TIMEOUT_NO_HB);
if (rv) {
- pr_err(PFX ": Unable to send the command to set the watchdog's settings, giving up.\n");
+ pr_err("Unable to send the command to set the watchdog's settings, giving up\n");
goto out;
}
_ipmi_set_timeout(IPMI_SET_TIMEOUT_NO_HB);
mutex_unlock(&ipmi_watchdog_mutex);
} else {
- pr_crit(PFX
- "Unexpected close, not stopping watchdog!\n");
+ pr_crit("Unexpected close, not stopping watchdog!\n");
ipmi_heartbeat();
}
clear_bit(0, &ipmi_wdog_open);
{
if (msg->msg.cmd == IPMI_WDOG_RESET_TIMER &&
msg->msg.data[0] == IPMI_WDOG_TIMER_NOT_INIT_RESP)
- pr_info(PFX "response: The IPMI controller appears to have been reset, will attempt to reinitialize the watchdog timer\n");
+ pr_info("response: The IPMI controller appears to have been reset, will attempt to reinitialize the watchdog timer\n");
else if (msg->msg.data[0] != 0)
- pr_err(PFX "response: Error %x on cmd %x\n",
+ pr_err("response: Error %x on cmd %x\n",
msg->msg.data[0],
msg->msg.cmd);
rv = ipmi_create_user(ipmi_intf, &ipmi_hndlrs, NULL, &watchdog_user);
if (rv < 0) {
- pr_crit(PFX "Unable to register with ipmi\n");
+ pr_crit("Unable to register with ipmi\n");
goto out;
}
&ipmi_version_major,
&ipmi_version_minor);
if (rv) {
- pr_warn(PFX "Unable to get IPMI version, assuming 1.0\n");
+ pr_warn("Unable to get IPMI version, assuming 1.0\n");
ipmi_version_major = 1;
ipmi_version_minor = 0;
}
if (rv < 0) {
ipmi_destroy_user(watchdog_user);
watchdog_user = NULL;
- pr_crit(PFX "Unable to register misc device\n");
+ pr_crit("Unable to register misc device\n");
}
#ifdef HAVE_DIE_NMI
rv = ipmi_set_timeout(IPMI_SET_TIMEOUT_FORCE_HB);
if (rv) {
- pr_warn(PFX "Error starting timer to test NMI: 0x%x. The NMI pretimeout will likely not work\n",
+ pr_warn("Error starting timer to test NMI: 0x%x. The NMI pretimeout will likely not work\n",
rv);
rv = 0;
goto out_restore;
msleep(1500);
if (testing_nmi != 2) {
- pr_warn(PFX "IPMI NMI didn't seem to occur. The NMI pretimeout will likely not work\n");
+ pr_warn("IPMI NMI didn't seem to occur. The NMI pretimeout will likely not work\n");
}
out_restore:
testing_nmi = 0;
start_now = 0; /* Disable this function after first startup. */
ipmi_watchdog_state = action_val;
ipmi_set_timeout(IPMI_SET_TIMEOUT_FORCE_HB);
- pr_info(PFX "Starting now!\n");
+ pr_info("Starting now!\n");
} else {
/* Stop the timer now. */
ipmi_watchdog_state = WDOG_TIMEOUT_NONE;
/* Disconnect from IPMI. */
rv = ipmi_destroy_user(loc_user);
if (rv)
- pr_warn(PFX "error unlinking from IPMI: %d\n", rv);
+ pr_warn("error unlinking from IPMI: %d\n", rv);
/* If it comes back, restart it properly. */
ipmi_start_timer_on_heartbeat = 1;
the timer. So do so. */
atomic_set(&pretimeout_since_last_heartbeat, 1);
if (atomic_inc_and_test(&preop_panic_excl))
- nmi_panic(regs, PFX "pre-timeout");
+ nmi_panic(regs, "pre-timeout");
}
return NMI_HANDLED;
if (preaction_val == WDOG_PRETIMEOUT_NMI) {
do_nmi = 1;
if (preop_val == WDOG_PREOP_GIVE_DATA) {
- pr_warn(PFX "Pretimeout op is to give data but NMI pretimeout is enabled, setting pretimeout op to none\n");
+ pr_warn("Pretimeout op is to give data but NMI pretimeout is enabled, setting pretimeout op to none\n");
preop_op("preop_none", NULL);
do_nmi = 0;
}
rv = register_nmi_handler(NMI_UNKNOWN, ipmi_nmi, 0,
"ipmi");
if (rv) {
- pr_warn(PFX "Can't register nmi handler\n");
+ pr_warn("Can't register nmi handler\n");
return;
} else
nmi_handler_registered = 1;
if (action_op(action, NULL)) {
action_op("reset", NULL);
- pr_info(PFX "Unknown action '%s', defaulting to reset\n",
- action);
+ pr_info("Unknown action '%s', defaulting to reset\n", action);
}
if (preaction_op(preaction, NULL)) {
preaction_op("pre_none", NULL);
- pr_info(PFX "Unknown preaction '%s', defaulting to none\n",
+ pr_info("Unknown preaction '%s', defaulting to none\n",
preaction);
}
if (preop_op(preop, NULL)) {
preop_op("preop_none", NULL);
- pr_info(PFX "Unknown preop '%s', defaulting to none\n", preop);
+ pr_info("Unknown preop '%s', defaulting to none\n", preop);
}
check_parms();
unregister_nmi_handler(NMI_UNKNOWN, "ipmi");
#endif
unregister_reboot_notifier(&wdog_reboot_notifier);
- pr_warn(PFX "can't register smi watcher\n");
+ pr_warn("can't register smi watcher\n");
return rv;
}
- pr_info(PFX "driver initialized\n");
+ pr_info("driver initialized\n");
return 0;
}
DEBUGP(5, dev, "NumRecBytes is valid\n");
break;
}
- mdelay(10);
+ usleep_range(10000, 11000);
}
if (i == 100) {
DEBUGP(5, dev, "Timeout waiting for NumRecBytes getting "
DEBUGP(2, dev, "NumRecBytes = %i\n", num_bytes_read);
break;
}
- mdelay(10);
+ usleep_range(10000, 11000);
}
/* check whether it is a short PTS reply? */
DEBUGP(3, dev, "-> cm4040_reader_release\n");
while (link->open) {
- DEBUGP(3, dev, KERN_INFO MODULE_NAME ": delaying release "
+ DEBUGP(3, dev, MODULE_NAME ": delaying release "
"until process has terminated\n");
wait_event(dev->devq, (link->open == 0));
}
return;
}
- /* Force the PLL-Audio-1x divider to 1 */
val = readl(reg + SUN4I_PLL_AUDIO_REG);
+
+ /*
+ * Force VCO and PLL bias current to lowest setting. Higher
+ * settings interfere with sigma-delta modulation and result
+ * in audible noise and distortions when using SPDIF or I2S.
+ */
+ val &= ~GENMASK(25, 16);
+
+ /* Force the PLL-Audio-1x divider to 1 */
val &= ~GENMASK(29, 26);
writel(val | (1 << 26), reg + SUN4I_PLL_AUDIO_REG);
}
#endif
+#ifdef CONFIG_ARM64_ERRATUM_1188873
+static u64 notrace arm64_1188873_read_cntvct_el0(void)
+{
+ return read_sysreg(cntvct_el0);
+}
+#endif
+
#ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
DEFINE_PER_CPU(const struct arch_timer_erratum_workaround *, timer_unstable_counter_workaround);
EXPORT_SYMBOL_GPL(timer_unstable_counter_workaround);
.read_cntvct_el0 = arm64_858921_read_cntvct_el0,
},
#endif
+#ifdef CONFIG_ARM64_ERRATUM_1188873
+ {
+ .match_type = ate_match_local_cap_id,
+ .id = (void *)ARM64_WORKAROUND_1188873,
+ .desc = "ARM erratum 1188873",
+ .read_cntvct_el0 = arm64_1188873_read_cntvct_el0,
+ },
+#endif
};
typedef bool (*ate_match_fn_t)(const struct arch_timer_erratum_workaround *,
#define INTEL_MSR_RANGE (0xffff)
#define AMD_MSR_RANGE (0x7)
+#define HYGON_MSR_RANGE (0x7)
#define MSR_K7_HWCR_CPB_DIS (1ULL << 25)
rdmsr_on_cpu(cpu, MSR_IA32_MISC_ENABLE, &lo, &hi);
msr = lo | ((u64)hi << 32);
return !(msr & MSR_IA32_MISC_ENABLE_TURBO_DISABLE);
+ case X86_VENDOR_HYGON:
case X86_VENDOR_AMD:
rdmsr_on_cpu(cpu, MSR_K7_HWCR, &lo, &hi);
msr = lo | ((u64)hi << 32);
msr_addr = MSR_IA32_MISC_ENABLE;
msr_mask = MSR_IA32_MISC_ENABLE_TURBO_DISABLE;
break;
+ case X86_VENDOR_HYGON:
case X86_VENDOR_AMD:
msr_addr = MSR_K7_HWCR;
msr_mask = MSR_K7_HWCR_CPB_DIS;
if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD)
msr &= AMD_MSR_RANGE;
+ else if (boot_cpu_data.x86_vendor == X86_VENDOR_HYGON)
+ msr &= HYGON_MSR_RANGE;
else
msr &= INTEL_MSR_RANGE;
{
u64 val;
struct pci_dev *pcidev;
+ unsigned int pci_vendor;
- if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD)
+ if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD)
+ pci_vendor = PCI_VENDOR_ID_AMD;
+ else if (boot_cpu_data.x86_vendor == X86_VENDOR_HYGON)
+ pci_vendor = PCI_VENDOR_ID_HYGON;
+ else
return -ENODEV;
- pcidev = pci_get_device(PCI_VENDOR_ID_AMD,
+ pcidev = pci_get_device(pci_vendor,
PCI_DEVICE_ID_AMD_KERNCZ_SMBUS, NULL);
if (!pcidev) {
late_initcall(cppc_cpufreq_init);
-static const struct acpi_device_id cppc_acpi_ids[] = {
+static const struct acpi_device_id cppc_acpi_ids[] __used = {
{ACPI_PROCESSOR_DEVICE_HID, },
{}
};
{ .compatible = "renesas,r8a73a4", },
{ .compatible = "renesas,r8a7740", },
{ .compatible = "renesas,r8a7743", },
+ { .compatible = "renesas,r8a7744", },
{ .compatible = "renesas,r8a7745", },
{ .compatible = "renesas,r8a7778", },
{ .compatible = "renesas,r8a7779", },
{ .compatible = "rockchip,rk3328", },
{ .compatible = "rockchip,rk3366", },
{ .compatible = "rockchip,rk3368", },
- { .compatible = "rockchip,rk3399", },
+ { .compatible = "rockchip,rk3399",
+ .data = &(struct cpufreq_dt_platform_data)
+ { .have_governor_per_policy = true, },
+ },
{ .compatible = "st-ericsson,u8500", },
{ .compatible = "st-ericsson,u8540", },
struct device *cpu_dev;
struct thermal_cooling_device *cdev;
const char *reg_name;
+ bool have_static_opps;
};
static struct freq_attr *cpufreq_dt_attr[] = {
}
}
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv) {
+ ret = -ENOMEM;
+ goto out_put_regulator;
+ }
+
+ priv->reg_name = name;
+ priv->opp_table = opp_table;
+
/*
* Initialize OPP tables for all policy->cpus. They will be shared by
* all CPUs which have marked their CPUs shared with OPP bindings.
*
* OPPs might be populated at runtime, don't check for error here
*/
- dev_pm_opp_of_cpumask_add_table(policy->cpus);
+ if (!dev_pm_opp_of_cpumask_add_table(policy->cpus))
+ priv->have_static_opps = true;
/*
* But we need OPP table to function so if it is not there let's
__func__, ret);
}
- priv = kzalloc(sizeof(*priv), GFP_KERNEL);
- if (!priv) {
- ret = -ENOMEM;
- goto out_free_opp;
- }
-
- priv->reg_name = name;
- priv->opp_table = opp_table;
-
ret = dev_pm_opp_init_cpufreq_table(cpu_dev, &freq_table);
if (ret) {
dev_err(cpu_dev, "failed to init cpufreq table: %d\n", ret);
- goto out_free_priv;
+ goto out_free_opp;
}
priv->cpu_dev = cpu_dev;
out_free_cpufreq_table:
dev_pm_opp_free_cpufreq_table(cpu_dev, &freq_table);
-out_free_priv:
- kfree(priv);
out_free_opp:
- dev_pm_opp_of_cpumask_remove_table(policy->cpus);
+ if (priv->have_static_opps)
+ dev_pm_opp_of_cpumask_remove_table(policy->cpus);
+ kfree(priv);
+out_put_regulator:
if (name)
dev_pm_opp_put_regulators(opp_table);
out_put_clk:
cpufreq_cooling_unregister(priv->cdev);
dev_pm_opp_free_cpufreq_table(priv->cpu_dev, &policy->freq_table);
- dev_pm_opp_of_cpumask_remove_table(policy->related_cpus);
+ if (priv->have_static_opps)
+ dev_pm_opp_of_cpumask_remove_table(policy->related_cpus);
if (priv->reg_name)
dev_pm_opp_put_regulators(priv->opp_table);
void cpufreq_freq_transition_end(struct cpufreq_policy *policy,
struct cpufreq_freqs *freqs, int transition_failed)
{
- if (unlikely(WARN_ON(!policy->transition_ongoing)))
+ if (WARN_ON(!policy->transition_ongoing))
return;
cpufreq_notify_post_transition(policy, freqs, transition_failed);
* changed in the meantime, so fall back to current frequency in that
* case.
*/
- if (requested_freq > policy->max || requested_freq < policy->min)
+ if (requested_freq > policy->max || requested_freq < policy->min) {
requested_freq = policy->cur;
+ dbs_info->requested_freq = requested_freq;
+ }
freq_step = get_freq_step(cs_tuners, policy);
if (policy_dbs->idle_periods < UINT_MAX) {
unsigned int freq_steps = policy_dbs->idle_periods * freq_step;
- if (requested_freq > freq_steps)
+ if (requested_freq > policy->min + freq_steps)
requested_freq -= freq_steps;
else
requested_freq = policy->min;
#include <linux/cpu_cooling.h>
#include <linux/err.h>
#include <linux/module.h>
+#include <linux/nvmem-consumer.h>
#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/pm_opp.h>
#define OCOTP_CFG3_6ULL_SPEED_792MHZ 0x2
#define OCOTP_CFG3_6ULL_SPEED_900MHZ 0x3
-static void imx6ul_opp_check_speed_grading(struct device *dev)
+static int imx6ul_opp_check_speed_grading(struct device *dev)
{
- struct device_node *np;
- void __iomem *base;
u32 val;
+ int ret = 0;
- np = of_find_compatible_node(NULL, NULL, "fsl,imx6ul-ocotp");
- if (!np)
- return;
+ if (of_find_property(dev->of_node, "nvmem-cells", NULL)) {
+ ret = nvmem_cell_read_u32(dev, "speed_grade", &val);
+ if (ret)
+ return ret;
+ } else {
+ struct device_node *np;
+ void __iomem *base;
+
+ np = of_find_compatible_node(NULL, NULL, "fsl,imx6ul-ocotp");
+ if (!np)
+ return -ENOENT;
+
+ base = of_iomap(np, 0);
+ of_node_put(np);
+ if (!base) {
+ dev_err(dev, "failed to map ocotp\n");
+ return -EFAULT;
+ }
- base = of_iomap(np, 0);
- if (!base) {
- dev_err(dev, "failed to map ocotp\n");
- goto put_node;
+ val = readl_relaxed(base + OCOTP_CFG3);
+ iounmap(base);
}
/*
* 2b'11: 900000000Hz on i.MX6ULL only;
* We need to set the max speed of ARM according to fuse map.
*/
- val = readl_relaxed(base + OCOTP_CFG3);
val >>= OCOTP_CFG3_SPEED_SHIFT;
val &= 0x3;
dev_warn(dev, "failed to disable 900MHz OPP\n");
}
- iounmap(base);
-put_node:
- of_node_put(np);
+ return ret;
}
static int imx6q_cpufreq_probe(struct platform_device *pdev)
}
if (of_machine_is_compatible("fsl,imx6ul") ||
- of_machine_is_compatible("fsl,imx6ull"))
- imx6ul_opp_check_speed_grading(cpu_dev);
- else
+ of_machine_is_compatible("fsl,imx6ull")) {
+ ret = imx6ul_opp_check_speed_grading(cpu_dev);
+ if (ret == -EPROBE_DEFER)
+ return ret;
+ if (ret) {
+ dev_err(cpu_dev, "failed to read ocotp: %d\n",
+ ret);
+ return ret;
+ }
+ } else {
imx6q_opp_check_speed_grading(cpu_dev);
+ }
/* Because we have added the OPPs here, we must free them */
free_opp = true;
}
}
}
+
+static int intel_pstate_get_cppc_guranteed(int cpu)
+{
+ struct cppc_perf_caps cppc_perf;
+ int ret;
+
+ ret = cppc_get_perf_caps(cpu, &cppc_perf);
+ if (ret)
+ return ret;
+
+ return cppc_perf.guaranteed_perf;
+}
+
#else
static void intel_pstate_set_itmt_prio(int cpu)
{
}
+
+static int intel_pstate_get_cppc_guranteed(int cpu)
+{
+ return -ENOTSUPP;
+}
#endif
static void intel_pstate_init_acpi_perf_limits(struct cpufreq_policy *policy)
cpufreq_freq_attr_rw(energy_performance_preference);
+static ssize_t show_base_frequency(struct cpufreq_policy *policy, char *buf)
+{
+ struct cpudata *cpu;
+ u64 cap;
+ int ratio;
+
+ ratio = intel_pstate_get_cppc_guranteed(policy->cpu);
+ if (ratio <= 0) {
+ rdmsrl_on_cpu(policy->cpu, MSR_HWP_CAPABILITIES, &cap);
+ ratio = HWP_GUARANTEED_PERF(cap);
+ }
+
+ cpu = all_cpu_data[policy->cpu];
+
+ return sprintf(buf, "%d\n", ratio * cpu->pstate.scaling);
+}
+
+cpufreq_freq_attr_ro(base_frequency);
+
static struct freq_attr *hwp_cpufreq_attrs[] = {
&energy_performance_preference,
&energy_performance_available_preferences,
+ &base_frequency,
NULL,
};
static const struct x86_cpu_id intel_pstate_cpu_ids[] = {
ICPU(INTEL_FAM6_SANDYBRIDGE, core_funcs),
ICPU(INTEL_FAM6_SANDYBRIDGE_X, core_funcs),
- ICPU(INTEL_FAM6_ATOM_SILVERMONT1, silvermont_funcs),
+ ICPU(INTEL_FAM6_ATOM_SILVERMONT, silvermont_funcs),
ICPU(INTEL_FAM6_IVYBRIDGE, core_funcs),
ICPU(INTEL_FAM6_HASWELL_CORE, core_funcs),
ICPU(INTEL_FAM6_BROADWELL_CORE, core_funcs),
ICPU(INTEL_FAM6_XEON_PHI_KNL, knl_funcs),
ICPU(INTEL_FAM6_XEON_PHI_KNM, knl_funcs),
ICPU(INTEL_FAM6_ATOM_GOLDMONT, core_funcs),
- ICPU(INTEL_FAM6_ATOM_GEMINI_LAKE, core_funcs),
+ ICPU(INTEL_FAM6_ATOM_GOLDMONT_PLUS, core_funcs),
ICPU(INTEL_FAM6_SKYLAKE_X, core_funcs),
{}
};
ret = dev_pm_opp_add(cpu_dev, clk_get_rate(clk) / 2, 0);
if (ret) {
+ dev_pm_opp_remove(cpu_dev, clk_get_rate(clk));
clk_put(clk);
dev_err(cpu_dev, "Failed to register OPPs\n");
- goto opp_register_failed;
+ return ret;
}
ret = dev_pm_opp_set_sharing_cpus(cpu_dev,
platform_device_register_simple("cpufreq-dt", -1, NULL, 0);
return 0;
-
-opp_register_failed:
- /* As registering has failed remove all the opp for all cpus */
- dev_pm_opp_cpumask_remove_table(cpu_possible_mask);
-
- return ret;
}
device_initcall(armada_xp_pmsu_cpufreq_init);
for_each_compatible_node(np, NULL, "samsung,s5pv210-dmc") {
id = of_alias_get_id(np, "dmc");
if (id < 0 || id >= ARRAY_SIZE(dmc_base)) {
- pr_err("%s: failed to get alias of dmc node '%s'\n",
- __func__, np->name);
+ pr_err("%s: failed to get alias of dmc node '%pOFn'\n",
+ __func__, np);
of_node_put(np);
return id;
}
void *virt;
virt = dma_alloc_coherent(bpmp->dev, sizeof(*data), &phys,
- GFP_KERNEL | GFP_DMA32);
+ GFP_KERNEL);
if (!virt)
return ERR_PTR(-ENOMEM);
if (!cpuidle_state_is_coupled(drv, index))
local_irq_enable();
- diff = ktime_us_delta(time_end, time_start);
- if (diff > INT_MAX)
- diff = INT_MAX;
-
- dev->last_residency = (int) diff;
-
if (entered_state >= 0) {
- /* Update cpuidle counters */
- /* This can be moved to within driver enter routine
+ /*
+ * Update cpuidle counters
+ * This can be moved to within driver enter routine,
* but that results in multiple copies of same code.
*/
+ diff = ktime_us_delta(time_end, time_start);
+ if (diff > INT_MAX)
+ diff = INT_MAX;
+
+ dev->last_residency = (int)diff;
dev->states_usage[entered_state].time += dev->last_residency;
dev->states_usage[entered_state].usage++;
} else {
last_state = &ldev->states[last_idx];
- last_residency = cpuidle_get_last_residency(dev) - drv->states[last_idx].exit_latency;
+ last_residency = dev->last_residency - drv->states[last_idx].exit_latency;
/* consider promotion */
if (last_idx < drv->state_count - 1 &&
int tick_wakeup;
unsigned int next_timer_us;
- unsigned int predicted_us;
unsigned int bucket;
unsigned int correction_factor[BUCKETS];
unsigned int intervals[INTERVALS];
* of points is below a threshold. If it is... then use the
* average of these 8 points as the estimated value.
*/
-static unsigned int get_typical_interval(struct menu_device *data)
+static unsigned int get_typical_interval(struct menu_device *data,
+ unsigned int predicted_us)
{
int i, divisor;
- unsigned int max, thresh, avg;
+ unsigned int min, max, thresh, avg;
uint64_t sum, variance;
thresh = UINT_MAX; /* Discard outliers above this value */
again:
/* First calculate the average of past intervals */
+ min = UINT_MAX;
max = 0;
sum = 0;
divisor = 0;
divisor++;
if (value > max)
max = value;
+
+ if (value < min)
+ min = value;
}
}
+
+ /*
+ * If the result of the computation is going to be discarded anyway,
+ * avoid the computation altogether.
+ */
+ if (min >= predicted_us)
+ return UINT_MAX;
+
if (divisor == INTERVALS)
avg = sum >> INTERVAL_SHIFT;
else
struct menu_device *data = this_cpu_ptr(&menu_devices);
int latency_req = cpuidle_governor_latency_req(dev->cpu);
int i;
- int first_idx;
int idx;
unsigned int interactivity_req;
- unsigned int expected_interval;
+ unsigned int predicted_us;
unsigned long nr_iowaiters, cpu_load;
ktime_t delta_next;
data->needs_update = 0;
}
- /* Special case when user has set very strict latency requirement */
- if (unlikely(latency_req == 0)) {
- *stop_tick = false;
- return 0;
- }
-
/* determine the expected residency time, round up */
data->next_timer_us = ktime_to_us(tick_nohz_get_sleep_length(&delta_next));
get_iowait_load(&nr_iowaiters, &cpu_load);
data->bucket = which_bucket(data->next_timer_us, nr_iowaiters);
+ if (unlikely(drv->state_count <= 1 || latency_req == 0) ||
+ ((data->next_timer_us < drv->states[1].target_residency ||
+ latency_req < drv->states[1].exit_latency) &&
+ !drv->states[0].disabled && !dev->states_usage[0].disable)) {
+ /*
+ * In this case state[0] will be used no matter what, so return
+ * it right away and keep the tick running.
+ */
+ *stop_tick = false;
+ return 0;
+ }
+
/*
* Force the result of multiplication to be 64 bits even if both
* operands are 32 bits.
* Make sure to round up for half microseconds.
*/
- data->predicted_us = DIV_ROUND_CLOSEST_ULL((uint64_t)data->next_timer_us *
+ predicted_us = DIV_ROUND_CLOSEST_ULL((uint64_t)data->next_timer_us *
data->correction_factor[data->bucket],
RESOLUTION * DECAY);
-
- expected_interval = get_typical_interval(data);
- expected_interval = min(expected_interval, data->next_timer_us);
-
- first_idx = 0;
- if (drv->states[0].flags & CPUIDLE_FLAG_POLLING) {
- struct cpuidle_state *s = &drv->states[1];
- unsigned int polling_threshold;
-
- /*
- * Default to a physical idle state, not to busy polling, unless
- * a timer is going to trigger really really soon.
- */
- polling_threshold = max_t(unsigned int, 20, s->target_residency);
- if (data->next_timer_us > polling_threshold &&
- latency_req > s->exit_latency && !s->disabled &&
- !dev->states_usage[1].disable)
- first_idx = 1;
- }
-
/*
* Use the lowest expected idle interval to pick the idle state.
*/
- data->predicted_us = min(data->predicted_us, expected_interval);
+ predicted_us = min(predicted_us, get_typical_interval(data, predicted_us));
if (tick_nohz_tick_stopped()) {
/*
* the known time till the closest timer event for the idle
* state selection.
*/
- if (data->predicted_us < TICK_USEC)
- data->predicted_us = ktime_to_us(delta_next);
+ if (predicted_us < TICK_USEC)
+ predicted_us = ktime_to_us(delta_next);
} else {
/*
* Use the performance multiplier and the user-configurable
* latency_req to determine the maximum exit latency.
*/
- interactivity_req = data->predicted_us / performance_multiplier(nr_iowaiters, cpu_load);
+ interactivity_req = predicted_us / performance_multiplier(nr_iowaiters, cpu_load);
if (latency_req > interactivity_req)
latency_req = interactivity_req;
}
- expected_interval = data->predicted_us;
/*
* Find the idle state with the lowest power while satisfying
* our constraints.
*/
idx = -1;
- for (i = first_idx; i < drv->state_count; i++) {
+ for (i = 0; i < drv->state_count; i++) {
struct cpuidle_state *s = &drv->states[i];
struct cpuidle_state_usage *su = &dev->states_usage[i];
if (s->disabled || su->disable)
continue;
+
if (idx == -1)
idx = i; /* first enabled state */
- if (s->target_residency > data->predicted_us) {
- if (data->predicted_us < TICK_USEC)
+
+ if (s->target_residency > predicted_us) {
+ /*
+ * Use a physical idle state, not busy polling, unless
+ * a timer is going to trigger soon enough.
+ */
+ if ((drv->states[idx].flags & CPUIDLE_FLAG_POLLING) &&
+ s->exit_latency <= latency_req &&
+ s->target_residency <= data->next_timer_us) {
+ predicted_us = s->target_residency;
+ idx = i;
+ break;
+ }
+ if (predicted_us < TICK_USEC)
break;
if (!tick_nohz_tick_stopped()) {
* tick in that case and let the governor run
* again in the next iteration of the loop.
*/
- expected_interval = drv->states[idx].target_residency;
+ predicted_us = drv->states[idx].target_residency;
break;
}
s->target_residency <= ktime_to_us(delta_next))
idx = i;
- goto out;
+ return idx;
}
if (s->exit_latency > latency_req) {
/*
* expected idle duration so that the tick is retained
* as long as that target residency is low enough.
*/
- expected_interval = drv->states[idx].target_residency;
+ predicted_us = drv->states[idx].target_residency;
break;
}
idx = i;
* expected idle duration is shorter than the tick period length.
*/
if (((drv->states[idx].flags & CPUIDLE_FLAG_POLLING) ||
- expected_interval < TICK_USEC) && !tick_nohz_tick_stopped()) {
+ predicted_us < TICK_USEC) && !tick_nohz_tick_stopped()) {
unsigned int delta_next_us = ktime_to_us(delta_next);
*stop_tick = false;
}
}
-out:
- data->last_state_idx = idx;
-
- return data->last_state_idx;
+ return idx;
}
/**
* duration predictor do a better job next time.
*/
measured_us = 9 * MAX_INTERESTING / 10;
+ } else if ((drv->states[last_idx].flags & CPUIDLE_FLAG_POLLING) &&
+ dev->poll_time_limit) {
+ /*
+ * The CPU exited the "polling" state due to a time limit, so
+ * the idle duration prediction leading to the selection of that
+ * state was inaccurate. If a better prediction had been made,
+ * the CPU might have been woken up from idle by the next timer.
+ * Assume that to be the case.
+ */
+ measured_us = data->next_timer_us;
} else {
/* measured value */
- measured_us = cpuidle_get_last_residency(dev);
+ measured_us = dev->last_residency;
/* Deduct exit latency */
if (measured_us > 2 * target->exit_latency)
#include <linux/sched/clock.h>
#include <linux/sched/idle.h>
-#define POLL_IDLE_TIME_LIMIT (TICK_NSEC / 16)
#define POLL_IDLE_RELAX_COUNT 200
static int __cpuidle poll_idle(struct cpuidle_device *dev,
{
u64 time_start = local_clock();
+ dev->poll_time_limit = false;
+
local_irq_enable();
if (!current_set_polling_and_test()) {
+ u64 limit = (u64)drv->states[1].target_residency * NSEC_PER_USEC;
unsigned int loop_count = 0;
while (!need_resched()) {
continue;
loop_count = 0;
- if (local_clock() - time_start > POLL_IDLE_TIME_LIMIT)
+ if (local_clock() - time_start > limit) {
+ dev->poll_time_limit = true;
break;
+ }
}
}
current_clr_polling();
+ Crypto Express 2,3,4 or 5 Accelerator (CEXxA)
+ Crypto Express 4 or 5 EP11 Coprocessor (CEXxP)
+config ZCRYPT_MULTIDEVNODES
+ bool "Support for multiple zcrypt device nodes"
+ default y
+ depends on S390
+ depends on ZCRYPT
+ help
+ With this option enabled the zcrypt device driver can
+ provide multiple devices nodes in /dev. Each device
+ node can get customized to limit access and narrow
+ down the use of the available crypto hardware.
+
config PKEY
tristate "Kernel API for protected key handling"
depends on S390
safexcel_configure(priv);
- priv->ring = devm_kzalloc(dev, priv->config.rings * sizeof(*priv->ring),
+ priv->ring = devm_kcalloc(dev, priv->config.rings,
+ sizeof(*priv->ring),
GFP_KERNEL);
if (!priv->ring) {
ret = -ENOMEM;
if (ret)
goto err_reg_clk;
- priv->ring[i].rdr_req = devm_kzalloc(dev,
- sizeof(priv->ring[i].rdr_req) * EIP197_DEFAULT_RING_SIZE,
+ priv->ring[i].rdr_req = devm_kcalloc(dev,
+ EIP197_DEFAULT_RING_SIZE,
+ sizeof(priv->ring[i].rdr_req),
GFP_KERNEL);
if (!priv->ring[i].rdr_req) {
ret = -ENOMEM;
*/
#include <linux/kernel.h>
+#include <linux/kmod.h>
#include <linux/sched.h>
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/of.h>
#include "governor.h"
-#define MAX(a,b) ((a > b) ? a : b)
-#define MIN(a,b) ((a < b) ? a : b)
-
static struct class *devfreq_class;
/*
return ERR_PTR(-ENODEV);
}
+/**
+ * try_then_request_governor() - Try to find the governor and request the
+ * module if is not found.
+ * @name: name of the governor
+ *
+ * Search the list of devfreq governors and request the module and try again
+ * if is not found. This can happen when both drivers (the governor driver
+ * and the driver that call devfreq_add_device) are built as modules.
+ * devfreq_list_lock should be held by the caller. Returns the matched
+ * governor's pointer.
+ */
+static struct devfreq_governor *try_then_request_governor(const char *name)
+{
+ struct devfreq_governor *governor;
+ int err = 0;
+
+ if (IS_ERR_OR_NULL(name)) {
+ pr_err("DEVFREQ: %s: Invalid parameters\n", __func__);
+ return ERR_PTR(-EINVAL);
+ }
+ WARN(!mutex_is_locked(&devfreq_list_lock),
+ "devfreq_list_lock must be locked.");
+
+ governor = find_devfreq_governor(name);
+ if (IS_ERR(governor)) {
+ mutex_unlock(&devfreq_list_lock);
+
+ if (!strncmp(name, DEVFREQ_GOV_SIMPLE_ONDEMAND,
+ DEVFREQ_NAME_LEN))
+ err = request_module("governor_%s", "simpleondemand");
+ else
+ err = request_module("governor_%s", name);
+ /* Restore previous state before return */
+ mutex_lock(&devfreq_list_lock);
+ if (err)
+ return NULL;
+
+ governor = find_devfreq_governor(name);
+ }
+
+ return governor;
+}
+
static int devfreq_notify_transition(struct devfreq *devfreq,
struct devfreq_freqs *freqs, unsigned int state)
{
* max_freq
* min_freq
*/
- max_freq = MIN(devfreq->scaling_max_freq, devfreq->max_freq);
- min_freq = MAX(devfreq->scaling_min_freq, devfreq->min_freq);
+ max_freq = min(devfreq->scaling_max_freq, devfreq->max_freq);
+ min_freq = max(devfreq->scaling_min_freq, devfreq->min_freq);
- if (min_freq && freq < min_freq) {
+ if (freq < min_freq) {
freq = min_freq;
flags &= ~DEVFREQ_FLAG_LEAST_UPPER_BOUND; /* Use GLB */
}
- if (max_freq && freq > max_freq) {
+ if (freq > max_freq) {
freq = max_freq;
flags |= DEVFREQ_FLAG_LEAST_UPPER_BOUND; /* Use LUB */
}
list_del(&devfreq->node);
mutex_unlock(&devfreq_list_lock);
- if (devfreq->governor)
- devfreq->governor->event_handler(devfreq,
- DEVFREQ_GOV_STOP, NULL);
-
if (devfreq->profile->exit)
devfreq->profile->exit(devfreq->dev.parent);
mutex_unlock(&devfreq->lock);
mutex_lock(&devfreq_list_lock);
- list_add(&devfreq->node, &devfreq_list);
- governor = find_devfreq_governor(devfreq->governor_name);
+ governor = try_then_request_governor(devfreq->governor_name);
if (IS_ERR(governor)) {
dev_err(dev, "%s: Unable to find governor for the device\n",
__func__);
__func__);
goto err_init;
}
+
+ list_add(&devfreq->node, &devfreq_list);
+
mutex_unlock(&devfreq_list_lock);
return devfreq;
err_init:
- list_del(&devfreq->node);
mutex_unlock(&devfreq_list_lock);
- device_unregister(&devfreq->dev);
+ devfreq_remove_device(devfreq);
devfreq = NULL;
err_dev:
- if (devfreq)
- kfree(devfreq);
+ kfree(devfreq);
err_out:
return ERR_PTR(err);
}
if (!devfreq)
return -EINVAL;
+ if (devfreq->governor)
+ devfreq->governor->event_handler(devfreq,
+ DEVFREQ_GOV_STOP, NULL);
device_unregister(&devfreq->dev);
return 0;
return -EINVAL;
mutex_lock(&devfreq_list_lock);
- governor = find_devfreq_governor(str_governor);
+ governor = try_then_request_governor(str_governor);
if (IS_ERR(governor)) {
ret = PTR_ERR(governor);
goto out;
struct devfreq *df = to_devfreq(dev);
unsigned long value;
int ret;
- unsigned long max;
ret = sscanf(buf, "%lu", &value);
if (ret != 1)
return -EINVAL;
mutex_lock(&df->lock);
- max = df->max_freq;
- if (value && max && value > max) {
- ret = -EINVAL;
- goto unlock;
+
+ if (value) {
+ if (value > df->max_freq) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+ } else {
+ unsigned long *freq_table = df->profile->freq_table;
+
+ /* Get minimum frequency according to sorting order */
+ if (freq_table[0] < freq_table[df->profile->max_state - 1])
+ value = freq_table[0];
+ else
+ value = freq_table[df->profile->max_state - 1];
}
df->min_freq = value;
{
struct devfreq *df = to_devfreq(dev);
- return sprintf(buf, "%lu\n", MAX(df->scaling_min_freq, df->min_freq));
+ return sprintf(buf, "%lu\n", max(df->scaling_min_freq, df->min_freq));
}
static ssize_t max_freq_store(struct device *dev, struct device_attribute *attr,
struct devfreq *df = to_devfreq(dev);
unsigned long value;
int ret;
- unsigned long min;
ret = sscanf(buf, "%lu", &value);
if (ret != 1)
return -EINVAL;
mutex_lock(&df->lock);
- min = df->min_freq;
- if (value && min && value < min) {
- ret = -EINVAL;
- goto unlock;
+
+ if (value) {
+ if (value < df->min_freq) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+ } else {
+ unsigned long *freq_table = df->profile->freq_table;
+
+ /* Get maximum frequency according to sorting order */
+ if (freq_table[0] < freq_table[df->profile->max_state - 1])
+ value = freq_table[df->profile->max_state - 1];
+ else
+ value = freq_table[0];
}
df->max_freq = value;
{
struct devfreq *df = to_devfreq(dev);
- return sprintf(buf, "%lu\n", MIN(df->scaling_max_freq, df->max_freq));
+ return sprintf(buf, "%lu\n", min(df->scaling_max_freq, df->max_freq));
}
static DEVICE_ATTR_RW(max_freq);
if (i == ARRAY_SIZE(ppmu_events)) {
dev_warn(dev,
- "don't know how to configure events : %s\n",
- node->name);
+ "don't know how to configure events : %pOFn\n",
+ node);
continue;
}
#define DEVFREQ_GOV_SUSPEND 0x4
#define DEVFREQ_GOV_RESUME 0x5
+#define DEVFREQ_MIN_FREQ 0
+#define DEVFREQ_MAX_FREQ ULONG_MAX
+
/**
* struct devfreq_governor - Devfreq policy governor
* @node: list node - contains registered devfreq governors
unsigned int event, void *data);
};
-/* Caution: devfreq->lock must be locked before calling update_devfreq */
-extern int update_devfreq(struct devfreq *devfreq);
-
extern void devfreq_monitor_start(struct devfreq *devfreq);
extern void devfreq_monitor_stop(struct devfreq *devfreq);
extern void devfreq_monitor_suspend(struct devfreq *devfreq);
* target callback should be able to get floor value as
* said in devfreq.h
*/
- if (!df->max_freq)
- *freq = UINT_MAX;
- else
- *freq = df->max_freq;
+ *freq = DEVFREQ_MAX_FREQ;
return 0;
}
* target callback should be able to get ceiling value as
* said in devfreq.h
*/
- *freq = df->min_freq;
+ *freq = DEVFREQ_MIN_FREQ;
return 0;
}
unsigned int dfso_upthreshold = DFSO_UPTHRESHOLD;
unsigned int dfso_downdifferential = DFSO_DOWNDIFFERENCTIAL;
struct devfreq_simple_ondemand_data *data = df->data;
- unsigned long max = (df->max_freq) ? df->max_freq : UINT_MAX;
err = devfreq_update_stats(df);
if (err)
/* Assume MAX if it is going to be divided by zero */
if (stat->total_time == 0) {
- *freq = max;
+ *freq = DEVFREQ_MAX_FREQ;
return 0;
}
/* Set MAX if it's busy enough */
if (stat->busy_time * 100 >
stat->total_time * dfso_upthreshold) {
- *freq = max;
+ *freq = DEVFREQ_MAX_FREQ;
return 0;
}
/* Set MAX if we do not know the initial frequency */
if (stat->current_frequency == 0) {
- *freq = max;
+ *freq = DEVFREQ_MAX_FREQ;
return 0;
}
b = div_u64(b, (dfso_upthreshold - dfso_downdifferential / 2));
*freq = (unsigned long) b;
- if (df->min_freq && *freq < df->min_freq)
- *freq = df->min_freq;
- if (df->max_freq && *freq > df->max_freq)
- *freq = df->max_freq;
-
return 0;
}
{
struct userspace_data *data = df->data;
- if (data->valid) {
- unsigned long adjusted_freq = data->user_frequency;
-
- if (df->max_freq && adjusted_freq > df->max_freq)
- adjusted_freq = df->max_freq;
-
- if (df->min_freq && adjusted_freq < df->min_freq)
- adjusted_freq = df->min_freq;
-
- *freq = adjusted_freq;
- } else {
+ if (data->valid)
+ *freq = data->user_frequency;
+ else
*freq = df->previous_freq; /* No user freq specified yet */
- }
+
return 0;
}
.volatile_reg = s10_sdram_volatile_reg,
.reg_read = s10_protected_reg_read,
.reg_write = s10_protected_reg_write,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
static int altr_s10_sdram_probe(struct platform_device *pdev)
static const struct x86_cpu_id pnd2_cpuids[] = {
{ X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_GOLDMONT, 0, (kernel_ulong_t)&apl_ops },
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_DENVERTON, 0, (kernel_ulong_t)&dnv_ops },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_GOLDMONT_X, 0, (kernel_ulong_t)&dnv_ops },
{ }
};
MODULE_DEVICE_TABLE(x86cpu, pnd2_cpuids);
.properties_table = EFI_INVALID_TABLE_ADDR,
.mem_attr_table = EFI_INVALID_TABLE_ADDR,
.rng_seed = EFI_INVALID_TABLE_ADDR,
- .tpm_log = EFI_INVALID_TABLE_ADDR
+ .tpm_log = EFI_INVALID_TABLE_ADDR,
+ .mem_reserve = EFI_INVALID_TABLE_ADDR,
};
EXPORT_SYMBOL(efi);
{EFI_MEMORY_ATTRIBUTES_TABLE_GUID, "MEMATTR", &efi.mem_attr_table},
{LINUX_EFI_RANDOM_SEED_TABLE_GUID, "RNG", &efi.rng_seed},
{LINUX_EFI_TPM_EVENT_LOG_GUID, "TPMEventLog", &efi.tpm_log},
+ {LINUX_EFI_MEMRESERVE_TABLE_GUID, "MEMRESERVE", &efi.mem_reserve},
{NULL_GUID, NULL, NULL},
};
early_memunmap(tbl, sizeof(*tbl));
}
+ if (efi.mem_reserve != EFI_INVALID_TABLE_ADDR) {
+ unsigned long prsv = efi.mem_reserve;
+
+ while (prsv) {
+ struct linux_efi_memreserve *rsv;
+
+ /* reserve the entry itself */
+ memblock_reserve(prsv, sizeof(*rsv));
+
+ rsv = early_memremap(prsv, sizeof(*rsv));
+ if (rsv == NULL) {
+ pr_err("Could not map UEFI memreserve entry!\n");
+ return -ENOMEM;
+ }
+
+ if (rsv->size)
+ memblock_reserve(rsv->base, rsv->size);
+
+ prsv = rsv->next;
+ early_memunmap(rsv, sizeof(*rsv));
+ }
+ }
+
return 0;
}
return false;
}
+static DEFINE_SPINLOCK(efi_mem_reserve_persistent_lock);
+
+int efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
+{
+ struct linux_efi_memreserve *rsv, *parent;
+
+ if (efi.mem_reserve == EFI_INVALID_TABLE_ADDR)
+ return -ENODEV;
+
+ rsv = kmalloc(sizeof(*rsv), GFP_KERNEL);
+ if (!rsv)
+ return -ENOMEM;
+
+ parent = memremap(efi.mem_reserve, sizeof(*rsv), MEMREMAP_WB);
+ if (!parent) {
+ kfree(rsv);
+ return -ENOMEM;
+ }
+
+ rsv->base = addr;
+ rsv->size = size;
+
+ spin_lock(&efi_mem_reserve_persistent_lock);
+ rsv->next = parent->next;
+ parent->next = __pa(rsv);
+ spin_unlock(&efi_mem_reserve_persistent_lock);
+
+ memunmap(parent);
+
+ return 0;
+}
+
#ifdef CONFIG_KEXEC
static int update_efi_random_seed(struct notifier_block *nb,
unsigned long code, void *unused)
cflags-$(CONFIG_ARM64) := $(subst -pg,,$(KBUILD_CFLAGS)) -fpie \
$(DISABLE_STACKLEAK_PLUGIN)
cflags-$(CONFIG_ARM) := $(subst -pg,,$(KBUILD_CFLAGS)) \
- -fno-builtin -fpic -mno-single-pic-base
+ -fno-builtin -fpic \
+ $(call cc-option,-mno-single-pic-base)
cflags-$(CONFIG_EFI_ARMSTUB) += -I$(srctree)/scripts/dtc/libfdt
return si;
}
+void install_memreserve_table(efi_system_table_t *sys_table_arg)
+{
+ struct linux_efi_memreserve *rsv;
+ efi_guid_t memreserve_table_guid = LINUX_EFI_MEMRESERVE_TABLE_GUID;
+ efi_status_t status;
+
+ status = efi_call_early(allocate_pool, EFI_LOADER_DATA, sizeof(*rsv),
+ (void **)&rsv);
+ if (status != EFI_SUCCESS) {
+ pr_efi_err(sys_table_arg, "Failed to allocate memreserve entry!\n");
+ return;
+ }
+
+ rsv->next = 0;
+ rsv->base = 0;
+ rsv->size = 0;
+
+ status = efi_call_early(install_configuration_table,
+ &memreserve_table_guid,
+ rsv);
+ if (status != EFI_SUCCESS)
+ pr_efi_err(sys_table_arg, "Failed to install memreserve config table!\n");
+}
+
+
/*
* This function handles the architcture specific differences between arm and
* arm64 regarding where the kernel image must be loaded and any memory that
}
}
+ install_memreserve_table(sys_table);
+
new_fdt_addr = fdt_addr;
status = allocate_new_fdt_and_exit_boot(sys_table, handle,
&new_fdt_addr, efi_get_max_fdt_addr(dram_base),
#define __efi_call_virt(f, args...) \
__efi_call_virt_pointer(efi.systab->runtime, f, args)
-/* efi_runtime_service() function identifiers */
-enum efi_rts_ids {
- GET_TIME,
- SET_TIME,
- GET_WAKEUP_TIME,
- SET_WAKEUP_TIME,
- GET_VARIABLE,
- GET_NEXT_VARIABLE,
- SET_VARIABLE,
- QUERY_VARIABLE_INFO,
- GET_NEXT_HIGH_MONO_COUNT,
- UPDATE_CAPSULE,
- QUERY_CAPSULE_CAPS,
-};
-
-/*
- * efi_runtime_work: Details of EFI Runtime Service work
- * @arg<1-5>: EFI Runtime Service function arguments
- * @status: Status of executing EFI Runtime Service
- * @efi_rts_id: EFI Runtime Service function identifier
- * @efi_rts_comp: Struct used for handling completions
- */
-struct efi_runtime_work {
- void *arg1;
- void *arg2;
- void *arg3;
- void *arg4;
- void *arg5;
- efi_status_t status;
- struct work_struct work;
- enum efi_rts_ids efi_rts_id;
- struct completion efi_rts_comp;
-};
+struct efi_runtime_work efi_rts_work;
/*
* efi_queue_work: Queue efi_runtime_service() and wait until it's done
*/
#define efi_queue_work(_rts, _arg1, _arg2, _arg3, _arg4, _arg5) \
({ \
- struct efi_runtime_work efi_rts_work; \
efi_rts_work.status = EFI_ABORTED; \
\
+ if (!efi_enabled(EFI_RUNTIME_SERVICES)) { \
+ pr_warn_once("EFI Runtime Services are disabled!\n"); \
+ goto exit; \
+ } \
+ \
init_completion(&efi_rts_work.efi_rts_comp); \
INIT_WORK_ONSTACK(&efi_rts_work.work, efi_call_rts); \
efi_rts_work.arg1 = _arg1; \
else \
pr_err("Failed to queue work to efi_rts_wq.\n"); \
\
+exit: \
+ efi_rts_work.efi_rts_id = NONE; \
efi_rts_work.status; \
})
*/
static void efi_call_rts(struct work_struct *work)
{
- struct efi_runtime_work *efi_rts_work;
void *arg1, *arg2, *arg3, *arg4, *arg5;
efi_status_t status = EFI_NOT_FOUND;
- efi_rts_work = container_of(work, struct efi_runtime_work, work);
- arg1 = efi_rts_work->arg1;
- arg2 = efi_rts_work->arg2;
- arg3 = efi_rts_work->arg3;
- arg4 = efi_rts_work->arg4;
- arg5 = efi_rts_work->arg5;
+ arg1 = efi_rts_work.arg1;
+ arg2 = efi_rts_work.arg2;
+ arg3 = efi_rts_work.arg3;
+ arg4 = efi_rts_work.arg4;
+ arg5 = efi_rts_work.arg5;
- switch (efi_rts_work->efi_rts_id) {
+ switch (efi_rts_work.efi_rts_id) {
case GET_TIME:
status = efi_call_virt(get_time, (efi_time_t *)arg1,
(efi_time_cap_t *)arg2);
*/
pr_err("Requested executing invalid EFI Runtime Service.\n");
}
- efi_rts_work->status = status;
- complete(&efi_rts_work->efi_rts_comp);
+ efi_rts_work.status = status;
+ complete(&efi_rts_work.efi_rts_comp);
}
static efi_status_t virt_efi_get_time(efi_time_t *tm, efi_time_cap_t *tc)
"could not get exclusive access to the firmware\n");
return;
}
+ efi_rts_work.efi_rts_id = RESET_SYSTEM;
__efi_call_virt(reset_system, reset_type, status, data_size, data);
up(&efi_runtime_lock);
}
return 0;
}
+static long efi_runtime_reset_system(unsigned long arg)
+{
+ struct efi_resetsystem __user *resetsystem_user;
+ struct efi_resetsystem resetsystem;
+ void *data = NULL;
+
+ resetsystem_user = (struct efi_resetsystem __user *)arg;
+ if (copy_from_user(&resetsystem, resetsystem_user,
+ sizeof(resetsystem)))
+ return -EFAULT;
+ if (resetsystem.data_size != 0) {
+ data = memdup_user((void *)resetsystem.data,
+ resetsystem.data_size);
+ if (IS_ERR(data))
+ return PTR_ERR(data);
+ }
+
+ efi.reset_system(resetsystem.reset_type, resetsystem.status,
+ resetsystem.data_size, (efi_char16_t *)data);
+
+ kfree(data);
+ return 0;
+}
+
static long efi_runtime_query_variableinfo(unsigned long arg)
{
struct efi_queryvariableinfo __user *queryvariableinfo_user;
case EFI_RUNTIME_QUERY_CAPSULECAPABILITIES:
return efi_runtime_query_capsulecaps(arg);
+
+ case EFI_RUNTIME_RESET_SYSTEM:
+ return efi_runtime_reset_system(arg);
}
return -ENOTTY;
efi_status_t *status;
} __packed;
+struct efi_resetsystem {
+ int reset_type;
+ efi_status_t status;
+ unsigned long data_size;
+ efi_char16_t *data;
+} __packed;
+
#define EFI_RUNTIME_GET_VARIABLE \
_IOWR('p', 0x01, struct efi_getvariable)
#define EFI_RUNTIME_SET_VARIABLE \
#define EFI_RUNTIME_QUERY_CAPSULECAPABILITIES \
_IOR('p', 0x0A, struct efi_querycapsulecapabilities)
+#define EFI_RUNTIME_RESET_SYSTEM \
+ _IOW('p', 0x0B, struct efi_resetsystem)
+
#endif /* _DRIVERS_FIRMWARE_EFI_TEST_H_ */
def_bool y
depends on ARCH_EP93XX
select GPIO_GENERIC
+ select GPIOLIB_IRQCHIP
config GPIO_EXAR
tristate "Support for GPIO pins on XR17V352/354/358"
If unsure, say N.
-config GPIO_INGENIC
- tristate "Ingenic JZ47xx SoCs GPIO support"
- depends on OF
- depends on MACH_INGENIC || COMPILE_TEST
- select GPIOLIB_IRQCHIP
- help
- Say yes here to support the GPIO functionality present on the
- JZ4740 and JZ4780 SoCs from Ingenic.
-
- If unsure, say N.
-
config GPIO_IOP
tristate "Intel IOP GPIO"
depends on ARCH_IOP32X || ARCH_IOP33X || COMPILE_TEST
A 32-bit single register GPIO fixed in/out implementation. This
can be used to represent any register as a set of GPIO signals.
+config GPIO_SIOX
+ tristate "SIOX GPIO support"
+ depends on SIOX
+ select GPIOLIB_IRQCHIP
+ help
+ Say yes here to support SIOX I/O devices. These are units connected
+ via a SIOX bus and have a number of fixed-direction I/O lines.
+
+config GPIO_SNPS_CREG
+ bool "Synopsys GPIO via CREG (Control REGisters) driver"
+ depends on ARC || COMPILE_TEST
+ depends on OF_GPIO
+ help
+ This driver supports GPIOs via CREG on various Synopsys SoCs.
+ This is a single-register MMIO GPIO driver for complex cases
+ where only several fields in register belong to GPIO lines and
+ each GPIO line owns a field with different length and on/off value.
+
config GPIO_SPEAR_SPICS
bool "ST SPEAr13xx SPI Chip Select as GPIO support"
depends on PLAT_SPEAR
config GPIO_TB10X
bool
+ select GPIO_GENERIC
select GENERIC_IRQ_CHIP
select OF_GPIO
ccflags-$(CONFIG_DEBUG_GPIO) += -DDEBUG
-obj-$(CONFIG_GPIOLIB) += devres.o
obj-$(CONFIG_GPIOLIB) += gpiolib.o
+obj-$(CONFIG_GPIOLIB) += gpiolib-devres.o
obj-$(CONFIG_GPIOLIB) += gpiolib-legacy.o
obj-$(CONFIG_GPIOLIB) += gpiolib-devprop.o
obj-$(CONFIG_OF_GPIO) += gpiolib-of.o
obj-$(CONFIG_GPIO_HLWD) += gpio-hlwd.o
obj-$(CONFIG_HTC_EGPIO) += gpio-htc-egpio.o
obj-$(CONFIG_GPIO_ICH) += gpio-ich.o
-obj-$(CONFIG_GPIO_INGENIC) += gpio-ingenic.o
obj-$(CONFIG_GPIO_IOP) += gpio-iop.o
obj-$(CONFIG_GPIO_IT87) += gpio-it87.o
obj-$(CONFIG_GPIO_JANZ_TTL) += gpio-janz-ttl.o
obj-$(CONFIG_ARCH_SA1100) += gpio-sa1100.o
obj-$(CONFIG_GPIO_SCH) += gpio-sch.o
obj-$(CONFIG_GPIO_SCH311X) += gpio-sch311x.o
+obj-$(CONFIG_GPIO_SNPS_CREG) += gpio-creg-snps.o
obj-$(CONFIG_GPIO_SODAVILLE) += gpio-sodaville.o
obj-$(CONFIG_GPIO_SPEAR_SPICS) += gpio-spear-spics.o
obj-$(CONFIG_GPIO_SPRD) += gpio-sprd.o
obj-$(CONFIG_GPIO_THUNDERX) += gpio-thunderx.o
obj-$(CONFIG_GPIO_TIMBERDALE) += gpio-timberdale.o
obj-$(CONFIG_GPIO_PALMAS) += gpio-palmas.o
+obj-$(CONFIG_GPIO_SIOX) += gpio-siox.o
obj-$(CONFIG_GPIO_TPIC2810) += gpio-tpic2810.o
obj-$(CONFIG_GPIO_TPS65086) += gpio-tps65086.o
obj-$(CONFIG_GPIO_TPS65218) += gpio-tps65218.o
module_platform_driver(adp5520_gpio_driver);
-MODULE_AUTHOR("Michael Hennerich <hennerich@blackfin.uclinux.org>");
+MODULE_AUTHOR("Michael Hennerich <michael.hennerich@analog.com>");
MODULE_DESCRIPTION("GPIO ADP5520 Driver");
MODULE_LICENSE("GPL");
MODULE_ALIAS("platform:adp5520-gpio");
module_i2c_driver(adp5588_gpio_driver);
-MODULE_AUTHOR("Michael Hennerich <hennerich@blackfin.uclinux.org>");
+MODULE_AUTHOR("Michael Hennerich <michael.hennerich@analog.com>");
MODULE_DESCRIPTION("GPIO ADP5588 Driver");
MODULE_LICENSE("GPL");
val = readl(reg_base + GPIO_INT_MASK(bank_id));
val |= BIT(bit);
writel(val, reg_base + GPIO_INT_MASK(bank_id));
+ gpiochip_disable_irq(&kona_gpio->gpio_chip, gpio);
raw_spin_unlock_irqrestore(&kona_gpio->lock, flags);
}
val = readl(reg_base + GPIO_INT_MSKCLR(bank_id));
val |= BIT(bit);
writel(val, reg_base + GPIO_INT_MSKCLR(bank_id));
+ gpiochip_enable_irq(&kona_gpio->gpio_chip, gpio);
raw_spin_unlock_irqrestore(&kona_gpio->lock, flags);
}
static int bcm_kona_gpio_irq_reqres(struct irq_data *d)
{
struct bcm_kona_gpio *kona_gpio = irq_data_get_irq_chip_data(d);
- int ret;
- ret = gpiochip_lock_as_irq(&kona_gpio->gpio_chip, d->hwirq);
- if (ret) {
- dev_err(kona_gpio->gpio_chip.parent,
- "unable to lock HW IRQ %lu for IRQ\n",
- d->hwirq);
- return ret;
- }
- return 0;
+ return gpiochip_reqres_irq(&kona_gpio->gpio_chip, d->hwirq);
}
static void bcm_kona_gpio_irq_relres(struct irq_data *d)
{
struct bcm_kona_gpio *kona_gpio = irq_data_get_irq_chip_data(d);
- gpiochip_unlock_as_irq(&kona_gpio->gpio_chip, d->hwirq);
+ gpiochip_relres_irq(&kona_gpio->gpio_chip, d->hwirq);
}
static struct irq_chip bcm_gpio_irq_chip = {
struct brcmstb_gpio_bank *bank;
struct gpio_chip *gc;
+ /*
+ * If bank_width is 0, then there is an empty bank in the
+ * register block. Special handling for this case.
+ */
+ if (bank_width == 0) {
+ dev_dbg(dev, "Width 0 found: Empty bank @ %d\n",
+ num_banks);
+ num_banks++;
+ gpio_base += MAX_GPIO_PER_BANK;
+ continue;
+ }
+
bank = devm_kzalloc(dev, sizeof(*bank), GFP_KERNEL);
if (!bank) {
err = -ENOMEM;
goto fail;
}
- dev_info(dev, "Registered %d banks (GPIO(s): %d-%d)\n",
- num_banks, priv->gpio_base, gpio_base - 1);
-
if (priv->parent_wake_irq && need_wakeup_event)
pm_wakeup_event(dev, 0);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0+
+//
+// Synopsys CREG (Control REGisters) GPIO driver
+//
+// Copyright (C) 2018 Synopsys
+// Author: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
+
+#include <linux/gpio/driver.h>
+#include <linux/io.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+
+#define MAX_GPIO 32
+
+struct creg_layout {
+ u8 ngpio;
+ u8 shift[MAX_GPIO];
+ u8 on[MAX_GPIO];
+ u8 off[MAX_GPIO];
+ u8 bit_per_gpio[MAX_GPIO];
+};
+
+struct creg_gpio {
+ struct gpio_chip gc;
+ void __iomem *regs;
+ spinlock_t lock;
+ const struct creg_layout *layout;
+};
+
+static void creg_gpio_set(struct gpio_chip *gc, unsigned int offset, int val)
+{
+ struct creg_gpio *hcg = gpiochip_get_data(gc);
+ const struct creg_layout *layout = hcg->layout;
+ u32 reg, reg_shift, value;
+ unsigned long flags;
+ int i;
+
+ value = val ? hcg->layout->on[offset] : hcg->layout->off[offset];
+
+ reg_shift = layout->shift[offset];
+ for (i = 0; i < offset; i++)
+ reg_shift += layout->bit_per_gpio[i] + layout->shift[i];
+
+ spin_lock_irqsave(&hcg->lock, flags);
+ reg = readl(hcg->regs);
+ reg &= ~(GENMASK(layout->bit_per_gpio[i] - 1, 0) << reg_shift);
+ reg |= (value << reg_shift);
+ writel(reg, hcg->regs);
+ spin_unlock_irqrestore(&hcg->lock, flags);
+}
+
+static int creg_gpio_dir_out(struct gpio_chip *gc, unsigned int offset, int val)
+{
+ creg_gpio_set(gc, offset, val);
+
+ return 0;
+}
+
+static int creg_gpio_validate_pg(struct device *dev, struct creg_gpio *hcg,
+ int i)
+{
+ const struct creg_layout *layout = hcg->layout;
+
+ if (layout->bit_per_gpio[i] < 1 || layout->bit_per_gpio[i] > 8)
+ return -EINVAL;
+
+ /* Check that on valiue fits it's placeholder */
+ if (GENMASK(31, layout->bit_per_gpio[i]) & layout->on[i])
+ return -EINVAL;
+
+ /* Check that off valiue fits it's placeholder */
+ if (GENMASK(31, layout->bit_per_gpio[i]) & layout->off[i])
+ return -EINVAL;
+
+ if (layout->on[i] == layout->off[i])
+ return -EINVAL;
+
+ return 0;
+}
+
+static int creg_gpio_validate(struct device *dev, struct creg_gpio *hcg,
+ u32 ngpios)
+{
+ u32 reg_len = 0;
+ int i;
+
+ if (hcg->layout->ngpio < 1 || hcg->layout->ngpio > MAX_GPIO)
+ return -EINVAL;
+
+ if (ngpios < 1 || ngpios > hcg->layout->ngpio) {
+ dev_err(dev, "ngpios must be in [1:%u]\n", hcg->layout->ngpio);
+ return -EINVAL;
+ }
+
+ for (i = 0; i < hcg->layout->ngpio; i++) {
+ if (creg_gpio_validate_pg(dev, hcg, i))
+ return -EINVAL;
+
+ reg_len += hcg->layout->shift[i] + hcg->layout->bit_per_gpio[i];
+ }
+
+ /* Check that we fit in 32 bit register */
+ if (reg_len > 32)
+ return -EINVAL;
+
+ return 0;
+}
+
+static const struct creg_layout hsdk_cs_ctl = {
+ .ngpio = 10,
+ .shift = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ .off = { 2, 2, 2, 2, 2, 2, 2, 2, 2, 2 },
+ .on = { 3, 3, 3, 3, 3, 3, 3, 3, 3, 3 },
+ .bit_per_gpio = { 2, 2, 2, 2, 2, 2, 2, 2, 2, 2 }
+};
+
+static const struct creg_layout axs10x_flsh_cs_ctl = {
+ .ngpio = 1,
+ .shift = { 0 },
+ .off = { 1 },
+ .on = { 3 },
+ .bit_per_gpio = { 2 }
+};
+
+static const struct of_device_id creg_gpio_ids[] = {
+ {
+ .compatible = "snps,creg-gpio-axs10x",
+ .data = &axs10x_flsh_cs_ctl
+ }, {
+ .compatible = "snps,creg-gpio-hsdk",
+ .data = &hsdk_cs_ctl
+ }, { /* sentinel */ }
+};
+
+static int creg_gpio_probe(struct platform_device *pdev)
+{
+ const struct of_device_id *match;
+ struct device *dev = &pdev->dev;
+ struct creg_gpio *hcg;
+ struct resource *mem;
+ u32 ngpios;
+ int ret;
+
+ hcg = devm_kzalloc(dev, sizeof(struct creg_gpio), GFP_KERNEL);
+ if (!hcg)
+ return -ENOMEM;
+
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ hcg->regs = devm_ioremap_resource(dev, mem);
+ if (IS_ERR(hcg->regs))
+ return PTR_ERR(hcg->regs);
+
+ match = of_match_node(creg_gpio_ids, pdev->dev.of_node);
+ hcg->layout = match->data;
+ if (!hcg->layout)
+ return -EINVAL;
+
+ ret = of_property_read_u32(dev->of_node, "ngpios", &ngpios);
+ if (ret)
+ return ret;
+
+ ret = creg_gpio_validate(dev, hcg, ngpios);
+ if (ret)
+ return ret;
+
+ spin_lock_init(&hcg->lock);
+
+ hcg->gc.label = dev_name(dev);
+ hcg->gc.base = -1;
+ hcg->gc.ngpio = ngpios;
+ hcg->gc.set = creg_gpio_set;
+ hcg->gc.direction_output = creg_gpio_dir_out;
+ hcg->gc.of_node = dev->of_node;
+
+ ret = devm_gpiochip_add_data(dev, &hcg->gc, hcg);
+ if (ret)
+ return ret;
+
+ dev_info(dev, "GPIO controller with %d gpios probed\n", ngpios);
+
+ return 0;
+}
+
+static struct platform_driver creg_gpio_snps_driver = {
+ .driver = {
+ .name = "snps-creg-gpio",
+ .of_match_table = creg_gpio_ids,
+ },
+ .probe = creg_gpio_probe,
+};
+builtin_platform_driver(creg_gpio_snps_driver);
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*/
+
#include <linux/gpio/driver.h>
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
#include <linux/platform_data/gpio-davinci.h>
#include <linux/irqchip/chained_irq.h>
+#include <linux/spinlock.h>
+
+#include <asm-generic/gpio.h>
+
+#define MAX_REGS_BANKS 5
+#define MAX_INT_PER_BANK 32
struct davinci_gpio_regs {
u32 dir;
typedef struct irq_chip *(*gpio_get_irq_chip_cb_t)(unsigned int irq);
#define BINTEN 0x8 /* GPIO Interrupt Per-Bank Enable Register */
-#define MAX_LABEL_SIZE 20
static void __iomem *gpio_base;
static unsigned int offset_array[5] = {0x10, 0x38, 0x60, 0x88, 0xb0};
+struct davinci_gpio_irq_data {
+ void __iomem *regs;
+ struct davinci_gpio_controller *chip;
+ int bank_num;
+};
+
+struct davinci_gpio_controller {
+ struct gpio_chip chip;
+ struct irq_domain *irq_domain;
+ /* Serialize access to GPIO registers */
+ spinlock_t lock;
+ void __iomem *regs[MAX_REGS_BANKS];
+ int gpio_unbanked;
+ int irqs[MAX_INT_PER_BANK];
+};
+
+static inline u32 __gpio_mask(unsigned gpio)
+{
+ return 1 << (gpio % 32);
+}
+
static inline struct davinci_gpio_regs __iomem *irq2regs(struct irq_data *d)
{
struct davinci_gpio_regs __iomem *g;
static int davinci_gpio_probe(struct platform_device *pdev)
{
- static int ctrl_num, bank_base;
- int gpio, bank, i, ret = 0;
+ int bank, i, ret = 0;
unsigned int ngpio, nbank, nirq;
struct davinci_gpio_controller *chips;
struct davinci_gpio_platform_data *pdata;
struct device *dev = &pdev->dev;
struct resource *res;
- char label[MAX_LABEL_SIZE];
pdata = davinci_gpio_get_pdata(pdev);
if (!pdata) {
else
nirq = DIV_ROUND_UP(ngpio, 16);
- nbank = DIV_ROUND_UP(ngpio, 32);
- chips = devm_kcalloc(dev,
- nbank, sizeof(struct davinci_gpio_controller),
- GFP_KERNEL);
+ chips = devm_kzalloc(dev, sizeof(*chips), GFP_KERNEL);
if (!chips)
return -ENOMEM;
}
}
- snprintf(label, MAX_LABEL_SIZE, "davinci_gpio.%d", ctrl_num++);
- chips->chip.label = devm_kstrdup(dev, label, GFP_KERNEL);
- if (!chips->chip.label)
- return -ENOMEM;
+ chips->chip.label = dev_name(dev);
chips->chip.direction_input = davinci_direction_in;
chips->chip.get = davinci_gpio_get;
chips->chip.set = davinci_gpio_set;
chips->chip.ngpio = ngpio;
- chips->chip.base = bank_base;
+ chips->chip.base = -1;
#ifdef CONFIG_OF_GPIO
chips->chip.of_gpio_n_cells = 2;
}
#endif
spin_lock_init(&chips->lock);
- bank_base += ngpio;
- for (gpio = 0, bank = 0; gpio < ngpio; gpio += 32, bank++)
+ nbank = DIV_ROUND_UP(ngpio, 32);
+ for (bank = 0; bank < nbank; bank++)
chips->regs[bank] = gpio_base + offset_array[bank];
ret = devm_gpiochip_add_data(dev, &chips->chip, chips);
if (ret)
- goto err;
+ return ret;
platform_set_drvdata(pdev, chips);
ret = davinci_gpio_irq_setup(pdev);
if (ret)
- goto err;
+ return ret;
return 0;
-
-err:
- /* Revert the static variable increments */
- ctrl_num--;
- bank_base -= ngpio;
-
- return ret;
}
/*--------------------------------------------------------------------------*/
+// SPDX-License-Identifier: GPL-2.0
/*
* Generic EP93xx GPIO handling
*
*
* Based on code originally from:
* linux/arch/arm/mach-ep93xx/core.c
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
*/
#include <linux/init.h>
#include <linux/irq.h>
#include <linux/slab.h>
#include <linux/gpio/driver.h>
-/* FIXME: this is here for gpio_to_irq() - get rid of this! */
-#include <linux/gpio.h>
+#include <linux/bitops.h>
+
+#define EP93XX_GPIO_F_INT_STATUS 0x5c
+#define EP93XX_GPIO_A_INT_STATUS 0xa0
+#define EP93XX_GPIO_B_INT_STATUS 0xbc
+
+/* Maximum value for gpio line identifiers */
+#define EP93XX_GPIO_LINE_MAX 63
-#include <mach/hardware.h>
-#include <mach/gpio-ep93xx.h>
+/* Maximum value for irq capable line identifiers */
+#define EP93XX_GPIO_LINE_MAX_IRQ 23
-#define irq_to_gpio(irq) ((irq) - gpio_to_irq(0))
+/*
+ * Static mapping of GPIO bank F IRQS:
+ * F0..F7 (16..24) to irq 80..87.
+ */
+#define EP93XX_GPIO_F_IRQ_BASE 80
struct ep93xx_gpio {
- void __iomem *mmio_base;
+ void __iomem *base;
struct gpio_chip gc[8];
};
static const u8 int_en_register_offset[3] = { 0x9c, 0xb8, 0x58 };
static const u8 int_debounce_register_offset[3] = { 0xa8, 0xc4, 0x64 };
-static void ep93xx_gpio_update_int_params(unsigned port)
+static void ep93xx_gpio_update_int_params(struct ep93xx_gpio *epg, unsigned port)
{
BUG_ON(port > 2);
- writeb_relaxed(0, EP93XX_GPIO_REG(int_en_register_offset[port]));
+ writeb_relaxed(0, epg->base + int_en_register_offset[port]);
writeb_relaxed(gpio_int_type2[port],
- EP93XX_GPIO_REG(int_type2_register_offset[port]));
+ epg->base + int_type2_register_offset[port]);
writeb_relaxed(gpio_int_type1[port],
- EP93XX_GPIO_REG(int_type1_register_offset[port]));
+ epg->base + int_type1_register_offset[port]);
writeb(gpio_int_unmasked[port] & gpio_int_enabled[port],
- EP93XX_GPIO_REG(int_en_register_offset[port]));
+ epg->base + int_en_register_offset[port]);
+}
+
+static int ep93xx_gpio_port(struct gpio_chip *gc)
+{
+ struct ep93xx_gpio *epg = gpiochip_get_data(gc);
+ int port = 0;
+
+ while (port < ARRAY_SIZE(epg->gc) && gc != &epg->gc[port])
+ port++;
+
+ /* This should not happen but is there as a last safeguard */
+ if (port == ARRAY_SIZE(epg->gc)) {
+ pr_crit("can't find the GPIO port\n");
+ return 0;
+ }
+
+ return port;
}
-static void ep93xx_gpio_int_debounce(unsigned int irq, bool enable)
+static void ep93xx_gpio_int_debounce(struct gpio_chip *gc,
+ unsigned int offset, bool enable)
{
- int line = irq_to_gpio(irq);
- int port = line >> 3;
- int port_mask = 1 << (line & 7);
+ struct ep93xx_gpio *epg = gpiochip_get_data(gc);
+ int port = ep93xx_gpio_port(gc);
+ int port_mask = BIT(offset);
if (enable)
gpio_int_debounce[port] |= port_mask;
gpio_int_debounce[port] &= ~port_mask;
writeb(gpio_int_debounce[port],
- EP93XX_GPIO_REG(int_debounce_register_offset[port]));
+ epg->base + int_debounce_register_offset[port]);
}
static void ep93xx_gpio_ab_irq_handler(struct irq_desc *desc)
{
- unsigned char status;
- int i;
+ struct gpio_chip *gc = irq_desc_get_handler_data(desc);
+ struct ep93xx_gpio *epg = gpiochip_get_data(gc);
+ struct irq_chip *irqchip = irq_desc_get_chip(desc);
+ unsigned long stat;
+ int offset;
- status = readb(EP93XX_GPIO_A_INT_STATUS);
- for (i = 0; i < 8; i++) {
- if (status & (1 << i)) {
- int gpio_irq = gpio_to_irq(EP93XX_GPIO_LINE_A(0)) + i;
- generic_handle_irq(gpio_irq);
- }
- }
+ chained_irq_enter(irqchip, desc);
- status = readb(EP93XX_GPIO_B_INT_STATUS);
- for (i = 0; i < 8; i++) {
- if (status & (1 << i)) {
- int gpio_irq = gpio_to_irq(EP93XX_GPIO_LINE_B(0)) + i;
- generic_handle_irq(gpio_irq);
- }
- }
+ /*
+ * Dispatch the IRQs to the irqdomain of each A and B
+ * gpiochip irqdomains depending on what has fired.
+ * The tricky part is that the IRQ line is shared
+ * between bank A and B and each has their own gpiochip.
+ */
+ stat = readb(epg->base + EP93XX_GPIO_A_INT_STATUS);
+ for_each_set_bit(offset, &stat, 8)
+ generic_handle_irq(irq_find_mapping(epg->gc[0].irq.domain,
+ offset));
+
+ stat = readb(epg->base + EP93XX_GPIO_B_INT_STATUS);
+ for_each_set_bit(offset, &stat, 8)
+ generic_handle_irq(irq_find_mapping(epg->gc[1].irq.domain,
+ offset));
+
+ chained_irq_exit(irqchip, desc);
}
static void ep93xx_gpio_f_irq_handler(struct irq_desc *desc)
/*
* map discontiguous hw irq range to continuous sw irq range:
*
- * IRQ_EP93XX_GPIO{0..7}MUX -> gpio_to_irq(EP93XX_GPIO_LINE_F({0..7})
+ * IRQ_EP93XX_GPIO{0..7}MUX -> EP93XX_GPIO_LINE_F{0..7}
*/
+ struct irq_chip *irqchip = irq_desc_get_chip(desc);
unsigned int irq = irq_desc_get_irq(desc);
int port_f_idx = ((irq + 1) & 7) ^ 4; /* {19..22,47..50} -> {0..7} */
- int gpio_irq = gpio_to_irq(EP93XX_GPIO_LINE_F(0)) + port_f_idx;
+ int gpio_irq = EP93XX_GPIO_F_IRQ_BASE + port_f_idx;
+ chained_irq_enter(irqchip, desc);
generic_handle_irq(gpio_irq);
+ chained_irq_exit(irqchip, desc);
}
static void ep93xx_gpio_irq_ack(struct irq_data *d)
{
- int line = irq_to_gpio(d->irq);
- int port = line >> 3;
- int port_mask = 1 << (line & 7);
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct ep93xx_gpio *epg = gpiochip_get_data(gc);
+ int port = ep93xx_gpio_port(gc);
+ int port_mask = BIT(d->irq & 7);
if (irqd_get_trigger_type(d) == IRQ_TYPE_EDGE_BOTH) {
gpio_int_type2[port] ^= port_mask; /* switch edge direction */
- ep93xx_gpio_update_int_params(port);
+ ep93xx_gpio_update_int_params(epg, port);
}
- writeb(port_mask, EP93XX_GPIO_REG(eoi_register_offset[port]));
+ writeb(port_mask, epg->base + eoi_register_offset[port]);
}
static void ep93xx_gpio_irq_mask_ack(struct irq_data *d)
{
- int line = irq_to_gpio(d->irq);
- int port = line >> 3;
- int port_mask = 1 << (line & 7);
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct ep93xx_gpio *epg = gpiochip_get_data(gc);
+ int port = ep93xx_gpio_port(gc);
+ int port_mask = BIT(d->irq & 7);
if (irqd_get_trigger_type(d) == IRQ_TYPE_EDGE_BOTH)
gpio_int_type2[port] ^= port_mask; /* switch edge direction */
gpio_int_unmasked[port] &= ~port_mask;
- ep93xx_gpio_update_int_params(port);
+ ep93xx_gpio_update_int_params(epg, port);
- writeb(port_mask, EP93XX_GPIO_REG(eoi_register_offset[port]));
+ writeb(port_mask, epg->base + eoi_register_offset[port]);
}
static void ep93xx_gpio_irq_mask(struct irq_data *d)
{
- int line = irq_to_gpio(d->irq);
- int port = line >> 3;
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct ep93xx_gpio *epg = gpiochip_get_data(gc);
+ int port = ep93xx_gpio_port(gc);
- gpio_int_unmasked[port] &= ~(1 << (line & 7));
- ep93xx_gpio_update_int_params(port);
+ gpio_int_unmasked[port] &= ~BIT(d->irq & 7);
+ ep93xx_gpio_update_int_params(epg, port);
}
static void ep93xx_gpio_irq_unmask(struct irq_data *d)
{
- int line = irq_to_gpio(d->irq);
- int port = line >> 3;
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct ep93xx_gpio *epg = gpiochip_get_data(gc);
+ int port = ep93xx_gpio_port(gc);
- gpio_int_unmasked[port] |= 1 << (line & 7);
- ep93xx_gpio_update_int_params(port);
+ gpio_int_unmasked[port] |= BIT(d->irq & 7);
+ ep93xx_gpio_update_int_params(epg, port);
}
/*
*/
static int ep93xx_gpio_irq_type(struct irq_data *d, unsigned int type)
{
- const int gpio = irq_to_gpio(d->irq);
- const int port = gpio >> 3;
- const int port_mask = 1 << (gpio & 7);
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct ep93xx_gpio *epg = gpiochip_get_data(gc);
+ int port = ep93xx_gpio_port(gc);
+ int offset = d->irq & 7;
+ int port_mask = BIT(offset);
irq_flow_handler_t handler;
- gpio_direction_input(gpio);
+ gc->direction_input(gc, offset);
switch (type) {
case IRQ_TYPE_EDGE_RISING:
case IRQ_TYPE_EDGE_BOTH:
gpio_int_type1[port] |= port_mask;
/* set initial polarity based on current input level */
- if (gpio_get_value(gpio))
+ if (gc->get(gc, offset))
gpio_int_type2[port] &= ~port_mask; /* falling */
else
gpio_int_type2[port] |= port_mask; /* rising */
gpio_int_enabled[port] |= port_mask;
- ep93xx_gpio_update_int_params(port);
+ ep93xx_gpio_update_int_params(epg, port);
return 0;
}
.irq_set_type = ep93xx_gpio_irq_type,
};
-static void ep93xx_gpio_init_irq(void)
+static int ep93xx_gpio_init_irq(struct platform_device *pdev,
+ struct ep93xx_gpio *epg)
{
+ int ab_parent_irq = platform_get_irq(pdev, 0);
+ struct device *dev = &pdev->dev;
int gpio_irq;
+ int ret;
+ int i;
- for (gpio_irq = gpio_to_irq(0);
- gpio_irq <= gpio_to_irq(EP93XX_GPIO_LINE_MAX_IRQ); ++gpio_irq) {
+ /* The A bank */
+ ret = gpiochip_irqchip_add(&epg->gc[0], &ep93xx_gpio_irq_chip,
+ 64, handle_level_irq,
+ IRQ_TYPE_NONE);
+ if (ret) {
+ dev_err(dev, "Could not add irqchip 0\n");
+ return ret;
+ }
+ gpiochip_set_chained_irqchip(&epg->gc[0], &ep93xx_gpio_irq_chip,
+ ab_parent_irq,
+ ep93xx_gpio_ab_irq_handler);
+
+ /* The B bank */
+ ret = gpiochip_irqchip_add(&epg->gc[1], &ep93xx_gpio_irq_chip,
+ 72, handle_level_irq,
+ IRQ_TYPE_NONE);
+ if (ret) {
+ dev_err(dev, "Could not add irqchip 1\n");
+ return ret;
+ }
+ gpiochip_set_chained_irqchip(&epg->gc[1], &ep93xx_gpio_irq_chip,
+ ab_parent_irq,
+ ep93xx_gpio_ab_irq_handler);
+
+ /* The F bank */
+ for (i = 0; i < 8; i++) {
+ gpio_irq = EP93XX_GPIO_F_IRQ_BASE + i;
+ irq_set_chip_data(gpio_irq, &epg->gc[5]);
irq_set_chip_and_handler(gpio_irq, &ep93xx_gpio_irq_chip,
handle_level_irq);
irq_clear_status_flags(gpio_irq, IRQ_NOREQUEST);
}
- irq_set_chained_handler(IRQ_EP93XX_GPIO_AB,
- ep93xx_gpio_ab_irq_handler);
- irq_set_chained_handler(IRQ_EP93XX_GPIO0MUX,
- ep93xx_gpio_f_irq_handler);
- irq_set_chained_handler(IRQ_EP93XX_GPIO1MUX,
- ep93xx_gpio_f_irq_handler);
- irq_set_chained_handler(IRQ_EP93XX_GPIO2MUX,
- ep93xx_gpio_f_irq_handler);
- irq_set_chained_handler(IRQ_EP93XX_GPIO3MUX,
- ep93xx_gpio_f_irq_handler);
- irq_set_chained_handler(IRQ_EP93XX_GPIO4MUX,
- ep93xx_gpio_f_irq_handler);
- irq_set_chained_handler(IRQ_EP93XX_GPIO5MUX,
- ep93xx_gpio_f_irq_handler);
- irq_set_chained_handler(IRQ_EP93XX_GPIO6MUX,
- ep93xx_gpio_f_irq_handler);
- irq_set_chained_handler(IRQ_EP93XX_GPIO7MUX,
- ep93xx_gpio_f_irq_handler);
+ for (i = 1; i <= 8; i++)
+ irq_set_chained_handler_and_data(platform_get_irq(pdev, i),
+ ep93xx_gpio_f_irq_handler,
+ &epg->gc[i]);
+ return 0;
}
int data;
int dir;
int base;
- bool has_debounce;
+ bool has_irq;
};
-#define EP93XX_GPIO_BANK(_label, _data, _dir, _base, _debounce) \
+#define EP93XX_GPIO_BANK(_label, _data, _dir, _base, _has_irq) \
{ \
.label = _label, \
.data = _data, \
.dir = _dir, \
.base = _base, \
- .has_debounce = _debounce, \
+ .has_irq = _has_irq, \
}
static struct ep93xx_gpio_bank ep93xx_gpio_banks[] = {
- EP93XX_GPIO_BANK("A", 0x00, 0x10, 0, true),
- EP93XX_GPIO_BANK("B", 0x04, 0x14, 8, true),
+ EP93XX_GPIO_BANK("A", 0x00, 0x10, 0, true), /* Bank A has 8 IRQs */
+ EP93XX_GPIO_BANK("B", 0x04, 0x14, 8, true), /* Bank B has 8 IRQs */
EP93XX_GPIO_BANK("C", 0x08, 0x18, 40, false),
EP93XX_GPIO_BANK("D", 0x0c, 0x1c, 24, false),
EP93XX_GPIO_BANK("E", 0x20, 0x24, 32, false),
- EP93XX_GPIO_BANK("F", 0x30, 0x34, 16, true),
+ EP93XX_GPIO_BANK("F", 0x30, 0x34, 16, true), /* Bank F has 8 IRQs */
EP93XX_GPIO_BANK("G", 0x38, 0x3c, 48, false),
EP93XX_GPIO_BANK("H", 0x40, 0x44, 56, false),
};
-static int ep93xx_gpio_set_config(struct gpio_chip *chip, unsigned offset,
+static int ep93xx_gpio_set_config(struct gpio_chip *gc, unsigned offset,
unsigned long config)
{
- int gpio = chip->base + offset;
- int irq = gpio_to_irq(gpio);
u32 debounce;
if (pinconf_to_config_param(config) != PIN_CONFIG_INPUT_DEBOUNCE)
return -ENOTSUPP;
- if (irq < 0)
- return -EINVAL;
-
debounce = pinconf_to_config_argument(config);
- ep93xx_gpio_int_debounce(irq, debounce ? true : false);
+ ep93xx_gpio_int_debounce(gc, offset, debounce ? true : false);
return 0;
}
-/*
- * Map GPIO A0..A7 (0..7) to irq 64..71,
- * B0..B7 (7..15) to irq 72..79, and
- * F0..F7 (16..24) to irq 80..87.
- */
-static int ep93xx_gpio_to_irq(struct gpio_chip *chip, unsigned offset)
+static int ep93xx_gpio_f_to_irq(struct gpio_chip *gc, unsigned offset)
{
- int gpio = chip->base + offset;
-
- if (gpio > EP93XX_GPIO_LINE_MAX_IRQ)
- return -EINVAL;
-
- return 64 + gpio;
+ return EP93XX_GPIO_F_IRQ_BASE + offset;
}
static int ep93xx_gpio_add_bank(struct gpio_chip *gc, struct device *dev,
- void __iomem *mmio_base, struct ep93xx_gpio_bank *bank)
+ struct ep93xx_gpio *epg,
+ struct ep93xx_gpio_bank *bank)
{
- void __iomem *data = mmio_base + bank->data;
- void __iomem *dir = mmio_base + bank->dir;
+ void __iomem *data = epg->base + bank->data;
+ void __iomem *dir = epg->base + bank->dir;
int err;
err = bgpio_init(gc, dev, 1, data, NULL, NULL, dir, NULL, 0);
gc->label = bank->label;
gc->base = bank->base;
- if (bank->has_debounce) {
+ if (bank->has_irq)
gc->set_config = ep93xx_gpio_set_config;
- gc->to_irq = ep93xx_gpio_to_irq;
- }
- return devm_gpiochip_add_data(dev, gc, NULL);
+ return devm_gpiochip_add_data(dev, gc, epg);
}
static int ep93xx_gpio_probe(struct platform_device *pdev)
{
- struct ep93xx_gpio *ep93xx_gpio;
+ struct ep93xx_gpio *epg;
struct resource *res;
int i;
struct device *dev = &pdev->dev;
- ep93xx_gpio = devm_kzalloc(dev, sizeof(struct ep93xx_gpio), GFP_KERNEL);
- if (!ep93xx_gpio)
+ epg = devm_kzalloc(dev, sizeof(*epg), GFP_KERNEL);
+ if (!epg)
return -ENOMEM;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- ep93xx_gpio->mmio_base = devm_ioremap_resource(dev, res);
- if (IS_ERR(ep93xx_gpio->mmio_base))
- return PTR_ERR(ep93xx_gpio->mmio_base);
+ epg->base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(epg->base))
+ return PTR_ERR(epg->base);
for (i = 0; i < ARRAY_SIZE(ep93xx_gpio_banks); i++) {
- struct gpio_chip *gc = &ep93xx_gpio->gc[i];
+ struct gpio_chip *gc = &epg->gc[i];
struct ep93xx_gpio_bank *bank = &ep93xx_gpio_banks[i];
- if (ep93xx_gpio_add_bank(gc, &pdev->dev,
- ep93xx_gpio->mmio_base, bank))
+ if (ep93xx_gpio_add_bank(gc, &pdev->dev, epg, bank))
dev_warn(&pdev->dev, "Unable to add gpio bank %s\n",
- bank->label);
+ bank->label);
+ /* Only bank F has especially funky IRQ handling */
+ if (i == 5)
+ gc->to_irq = ep93xx_gpio_f_to_irq;
}
- ep93xx_gpio_init_irq();
+ ep93xx_gpio_init_irq(pdev, epg);
return 0;
}
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/bitops.h>
+#include <linux/clk.h>
/* GPIO registers definition */
#define GPIO_DATA_OUT 0x00
* struct ftgpio_gpio - Gemini GPIO state container
* @dev: containing device for this instance
* @gc: gpiochip for this instance
+ * @base: remapped I/O-memory base
+ * @clk: silicon clock
*/
struct ftgpio_gpio {
struct device *dev;
struct gpio_chip gc;
void __iomem *base;
+ struct clk *clk;
};
static void ftgpio_gpio_ack_irq(struct irq_data *d)
chained_irq_exit(irqchip, desc);
}
+static int ftgpio_gpio_set_config(struct gpio_chip *gc, unsigned int offset,
+ unsigned long config)
+{
+ enum pin_config_param param = pinconf_to_config_param(config);
+ u32 arg = pinconf_to_config_argument(config);
+ struct ftgpio_gpio *g = gpiochip_get_data(gc);
+ unsigned long pclk_freq;
+ u32 deb_div;
+ u32 val;
+
+ if (param != PIN_CONFIG_INPUT_DEBOUNCE)
+ return -ENOTSUPP;
+
+ /*
+ * Debounce only works if interrupts are enabled. The manual
+ * states that if PCLK is 66 MHz, and this is set to 0x7D0, then
+ * PCLK is divided down to 33 kHz for the debounce timer. 0x7D0 is
+ * 2000 decimal, so what they mean is simply that the PCLK is
+ * divided by this value.
+ *
+ * As we get a debounce setting in microseconds, we calculate the
+ * desired period time and see if we can get a suitable debounce
+ * time.
+ */
+ pclk_freq = clk_get_rate(g->clk);
+ deb_div = DIV_ROUND_CLOSEST(pclk_freq, arg);
+
+ /* This register is only 24 bits wide */
+ if (deb_div > (1 << 24))
+ return -ENOTSUPP;
+
+ dev_dbg(g->dev, "prescale divisor: %08x, resulting frequency %lu Hz\n",
+ deb_div, (pclk_freq/deb_div));
+
+ val = readl(g->base + GPIO_DEBOUNCE_PRESCALE);
+ if (val == deb_div) {
+ /*
+ * The debounce timer happens to already be set to the
+ * desireable value, what a coincidence! We can just enable
+ * debounce on this GPIO line and return. This happens more
+ * often than you think, for example when all GPIO keys
+ * on a system are requesting the same debounce interval.
+ */
+ val = readl(g->base + GPIO_DEBOUNCE_EN);
+ val |= BIT(offset);
+ writel(val, g->base + GPIO_DEBOUNCE_EN);
+ return 0;
+ }
+
+ val = readl(g->base + GPIO_DEBOUNCE_EN);
+ if (val) {
+ /*
+ * Oh no! Someone is already using the debounce with
+ * another setting than what we need. Bummer.
+ */
+ return -ENOTSUPP;
+ }
+
+ /* First come, first serve */
+ writel(deb_div, g->base + GPIO_DEBOUNCE_PRESCALE);
+ /* Enable debounce */
+ val |= BIT(offset);
+ writel(val, g->base + GPIO_DEBOUNCE_EN);
+
+ return 0;
+}
+
static int ftgpio_gpio_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
if (irq <= 0)
return irq ? irq : -EINVAL;
+ g->clk = devm_clk_get(dev, NULL);
+ if (!IS_ERR(g->clk)) {
+ ret = clk_prepare_enable(g->clk);
+ if (ret)
+ return ret;
+ } else if (PTR_ERR(g->clk) == -EPROBE_DEFER) {
+ /*
+ * Percolate deferrals, for anything else,
+ * just live without the clocking.
+ */
+ return PTR_ERR(g->clk);
+ }
+
ret = bgpio_init(&g->gc, dev, 4,
g->base + GPIO_DATA_IN,
g->base + GPIO_DATA_SET,
0);
if (ret) {
dev_err(dev, "unable to init generic GPIO\n");
- return ret;
+ goto dis_clk;
}
g->gc.label = "FTGPIO010";
g->gc.base = -1;
g->gc.owner = THIS_MODULE;
/* ngpio is set by bgpio_init() */
+ /* We need a silicon clock to do debounce */
+ if (!IS_ERR(g->clk))
+ g->gc.set_config = ftgpio_gpio_set_config;
+
ret = devm_gpiochip_add_data(dev, &g->gc, g);
if (ret)
- return ret;
+ goto dis_clk;
/* Disable, unmask and clear all interrupts */
writel(0x0, g->base + GPIO_INT_EN);
writel(0x0, g->base + GPIO_INT_MASK);
writel(~0x0, g->base + GPIO_INT_CLR);
+ /* Clear any use of debounce */
+ writel(0x0, g->base + GPIO_DEBOUNCE_EN);
+
ret = gpiochip_irqchip_add(&g->gc, &ftgpio_gpio_irqchip,
0, handle_bad_irq,
IRQ_TYPE_NONE);
if (ret) {
dev_info(dev, "could not add irqchip\n");
- return ret;
+ goto dis_clk;
}
gpiochip_set_chained_irqchip(&g->gc, &ftgpio_gpio_irqchip,
irq, ftgpio_gpio_irq_handler);
+ platform_set_drvdata(pdev, g);
dev_info(dev, "FTGPIO010 @%p registered\n", g->base);
return 0;
+
+dis_clk:
+ if (!IS_ERR(g->clk))
+ clk_disable_unprepare(g->clk);
+ return ret;
+}
+
+static int ftgpio_gpio_remove(struct platform_device *pdev)
+{
+ struct ftgpio_gpio *g = platform_get_drvdata(pdev);
+
+ if (!IS_ERR(g->clk))
+ clk_disable_unprepare(g->clk);
+ return 0;
}
static const struct of_device_id ftgpio_gpio_of_match[] = {
.name = "ftgpio010-gpio",
.of_match_table = of_match_ptr(ftgpio_gpio_of_match),
},
- .probe = ftgpio_gpio_probe,
+ .probe = ftgpio_gpio_probe,
+ .remove = ftgpio_gpio_remove,
};
builtin_platform_driver(ftgpio_gpio_driver);
unsigned long flag;
struct egpio_chip *egpio;
struct egpio_info *ei;
- unsigned bit;
int pos;
int reg;
int shift;
egpio = gpiochip_get_data(chip);
ei = dev_get_drvdata(egpio->dev);
- bit = egpio_bit(ei, offset);
pos = egpio_pos(ei, offset);
reg = egpio->reg_start + pos;
shift = pos << ei->reg_shift;
ei->chip[i].is_out = pdata->chip[i].direction;
ei->chip[i].dev = &(pdev->dev);
chip = &(ei->chip[i].chip);
- chip->label = "htc-egpio";
+ chip->label = devm_kasprintf(&pdev->dev, GFP_KERNEL,
+ "htc-egpio-%d",
+ i);
+ if (!chip->label) {
+ ret = -ENOMEM;
+ goto fail;
+ }
chip->parent = &pdev->dev;
chip->owner = THIS_MODULE;
chip->get = egpio_get;
+++ /dev/null
-/*
- * Ingenic JZ47xx GPIO driver
- *
- * Copyright (c) 2017 Paul Cercueil <paul@crapouillou.net>
- *
- * License terms: GNU General Public License (GPL) version 2
- */
-
-#include <linux/gpio/driver.h>
-#include <linux/interrupt.h>
-#include <linux/io.h>
-#include <linux/module.h>
-#include <linux/of_address.h>
-#include <linux/of_device.h>
-#include <linux/of_irq.h>
-#include <linux/pinctrl/consumer.h>
-#include <linux/regmap.h>
-
-#define GPIO_PIN 0x00
-#define GPIO_MSK 0x20
-
-#define JZ4740_GPIO_DATA 0x10
-#define JZ4740_GPIO_SELECT 0x50
-#define JZ4740_GPIO_DIR 0x60
-#define JZ4740_GPIO_TRIG 0x70
-#define JZ4740_GPIO_FLAG 0x80
-
-#define JZ4770_GPIO_INT 0x10
-#define JZ4770_GPIO_PAT1 0x30
-#define JZ4770_GPIO_PAT0 0x40
-#define JZ4770_GPIO_FLAG 0x50
-
-#define REG_SET(x) ((x) + 0x4)
-#define REG_CLEAR(x) ((x) + 0x8)
-
-enum jz_version {
- ID_JZ4740,
- ID_JZ4770,
- ID_JZ4780,
-};
-
-struct ingenic_gpio_chip {
- struct regmap *map;
- struct gpio_chip gc;
- struct irq_chip irq_chip;
- unsigned int irq, reg_base;
- enum jz_version version;
-};
-
-static u32 gpio_ingenic_read_reg(struct ingenic_gpio_chip *jzgc, u8 reg)
-{
- unsigned int val;
-
- regmap_read(jzgc->map, jzgc->reg_base + reg, &val);
-
- return (u32) val;
-}
-
-static void gpio_ingenic_set_bit(struct ingenic_gpio_chip *jzgc,
- u8 reg, u8 offset, bool set)
-{
- if (set)
- reg = REG_SET(reg);
- else
- reg = REG_CLEAR(reg);
-
- regmap_write(jzgc->map, jzgc->reg_base + reg, BIT(offset));
-}
-
-static inline bool gpio_get_value(struct ingenic_gpio_chip *jzgc, u8 offset)
-{
- unsigned int val = gpio_ingenic_read_reg(jzgc, GPIO_PIN);
-
- return !!(val & BIT(offset));
-}
-
-static void gpio_set_value(struct ingenic_gpio_chip *jzgc, u8 offset, int value)
-{
- if (jzgc->version >= ID_JZ4770)
- gpio_ingenic_set_bit(jzgc, JZ4770_GPIO_PAT0, offset, !!value);
- else
- gpio_ingenic_set_bit(jzgc, JZ4740_GPIO_DATA, offset, !!value);
-}
-
-static void irq_set_type(struct ingenic_gpio_chip *jzgc,
- u8 offset, unsigned int type)
-{
- u8 reg1, reg2;
-
- if (jzgc->version >= ID_JZ4770) {
- reg1 = JZ4770_GPIO_PAT1;
- reg2 = JZ4770_GPIO_PAT0;
- } else {
- reg1 = JZ4740_GPIO_TRIG;
- reg2 = JZ4740_GPIO_DIR;
- }
-
- switch (type) {
- case IRQ_TYPE_EDGE_RISING:
- gpio_ingenic_set_bit(jzgc, reg2, offset, true);
- gpio_ingenic_set_bit(jzgc, reg1, offset, true);
- break;
- case IRQ_TYPE_EDGE_FALLING:
- gpio_ingenic_set_bit(jzgc, reg2, offset, false);
- gpio_ingenic_set_bit(jzgc, reg1, offset, true);
- break;
- case IRQ_TYPE_LEVEL_HIGH:
- gpio_ingenic_set_bit(jzgc, reg2, offset, true);
- gpio_ingenic_set_bit(jzgc, reg1, offset, false);
- break;
- case IRQ_TYPE_LEVEL_LOW:
- default:
- gpio_ingenic_set_bit(jzgc, reg2, offset, false);
- gpio_ingenic_set_bit(jzgc, reg1, offset, false);
- break;
- }
-}
-
-static void ingenic_gpio_irq_mask(struct irq_data *irqd)
-{
- struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
- struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
-
- gpio_ingenic_set_bit(jzgc, GPIO_MSK, irqd->hwirq, true);
-}
-
-static void ingenic_gpio_irq_unmask(struct irq_data *irqd)
-{
- struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
- struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
-
- gpio_ingenic_set_bit(jzgc, GPIO_MSK, irqd->hwirq, false);
-}
-
-static void ingenic_gpio_irq_enable(struct irq_data *irqd)
-{
- struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
- struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
- int irq = irqd->hwirq;
-
- if (jzgc->version >= ID_JZ4770)
- gpio_ingenic_set_bit(jzgc, JZ4770_GPIO_INT, irq, true);
- else
- gpio_ingenic_set_bit(jzgc, JZ4740_GPIO_SELECT, irq, true);
-
- ingenic_gpio_irq_unmask(irqd);
-}
-
-static void ingenic_gpio_irq_disable(struct irq_data *irqd)
-{
- struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
- struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
- int irq = irqd->hwirq;
-
- ingenic_gpio_irq_mask(irqd);
-
- if (jzgc->version >= ID_JZ4770)
- gpio_ingenic_set_bit(jzgc, JZ4770_GPIO_INT, irq, false);
- else
- gpio_ingenic_set_bit(jzgc, JZ4740_GPIO_SELECT, irq, false);
-}
-
-static void ingenic_gpio_irq_ack(struct irq_data *irqd)
-{
- struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
- struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
- int irq = irqd->hwirq;
- bool high;
-
- if (irqd_get_trigger_type(irqd) == IRQ_TYPE_EDGE_BOTH) {
- /*
- * Switch to an interrupt for the opposite edge to the one that
- * triggered the interrupt being ACKed.
- */
- high = gpio_get_value(jzgc, irq);
- if (high)
- irq_set_type(jzgc, irq, IRQ_TYPE_EDGE_FALLING);
- else
- irq_set_type(jzgc, irq, IRQ_TYPE_EDGE_RISING);
- }
-
- if (jzgc->version >= ID_JZ4770)
- gpio_ingenic_set_bit(jzgc, JZ4770_GPIO_FLAG, irq, false);
- else
- gpio_ingenic_set_bit(jzgc, JZ4740_GPIO_DATA, irq, true);
-}
-
-static int ingenic_gpio_irq_set_type(struct irq_data *irqd, unsigned int type)
-{
- struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
- struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
-
- switch (type) {
- case IRQ_TYPE_EDGE_BOTH:
- case IRQ_TYPE_EDGE_RISING:
- case IRQ_TYPE_EDGE_FALLING:
- irq_set_handler_locked(irqd, handle_edge_irq);
- break;
- case IRQ_TYPE_LEVEL_HIGH:
- case IRQ_TYPE_LEVEL_LOW:
- irq_set_handler_locked(irqd, handle_level_irq);
- break;
- default:
- irq_set_handler_locked(irqd, handle_bad_irq);
- }
-
- if (type == IRQ_TYPE_EDGE_BOTH) {
- /*
- * The hardware does not support interrupts on both edges. The
- * best we can do is to set up a single-edge interrupt and then
- * switch to the opposing edge when ACKing the interrupt.
- */
- bool high = gpio_get_value(jzgc, irqd->hwirq);
-
- type = high ? IRQ_TYPE_EDGE_FALLING : IRQ_TYPE_EDGE_RISING;
- }
-
- irq_set_type(jzgc, irqd->hwirq, type);
- return 0;
-}
-
-static int ingenic_gpio_irq_set_wake(struct irq_data *irqd, unsigned int on)
-{
- struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
- struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
-
- return irq_set_irq_wake(jzgc->irq, on);
-}
-
-static void ingenic_gpio_irq_handler(struct irq_desc *desc)
-{
- struct gpio_chip *gc = irq_desc_get_handler_data(desc);
- struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
- struct irq_chip *irq_chip = irq_data_get_irq_chip(&desc->irq_data);
- unsigned long flag, i;
-
- chained_irq_enter(irq_chip, desc);
-
- if (jzgc->version >= ID_JZ4770)
- flag = gpio_ingenic_read_reg(jzgc, JZ4770_GPIO_FLAG);
- else
- flag = gpio_ingenic_read_reg(jzgc, JZ4740_GPIO_FLAG);
-
- for_each_set_bit(i, &flag, 32)
- generic_handle_irq(irq_linear_revmap(gc->irq.domain, i));
- chained_irq_exit(irq_chip, desc);
-}
-
-static void ingenic_gpio_set(struct gpio_chip *gc,
- unsigned int offset, int value)
-{
- struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
-
- gpio_set_value(jzgc, offset, value);
-}
-
-static int ingenic_gpio_get(struct gpio_chip *gc, unsigned int offset)
-{
- struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
-
- return (int) gpio_get_value(jzgc, offset);
-}
-
-static int ingenic_gpio_direction_input(struct gpio_chip *gc,
- unsigned int offset)
-{
- return pinctrl_gpio_direction_input(gc->base + offset);
-}
-
-static int ingenic_gpio_direction_output(struct gpio_chip *gc,
- unsigned int offset, int value)
-{
- ingenic_gpio_set(gc, offset, value);
- return pinctrl_gpio_direction_output(gc->base + offset);
-}
-
-static const struct of_device_id ingenic_gpio_of_match[] = {
- { .compatible = "ingenic,jz4740-gpio", .data = (void *)ID_JZ4740 },
- { .compatible = "ingenic,jz4770-gpio", .data = (void *)ID_JZ4770 },
- { .compatible = "ingenic,jz4780-gpio", .data = (void *)ID_JZ4780 },
- {},
-};
-MODULE_DEVICE_TABLE(of, ingenic_gpio_of_match);
-
-static int ingenic_gpio_probe(struct platform_device *pdev)
-{
- struct device *dev = &pdev->dev;
- struct ingenic_gpio_chip *jzgc;
- u32 bank;
- int err;
-
- jzgc = devm_kzalloc(dev, sizeof(*jzgc), GFP_KERNEL);
- if (!jzgc)
- return -ENOMEM;
-
- jzgc->map = dev_get_drvdata(dev->parent);
- if (!jzgc->map) {
- dev_err(dev, "Cannot get parent regmap\n");
- return -ENXIO;
- }
-
- err = of_property_read_u32(dev->of_node, "reg", &bank);
- if (err) {
- dev_err(dev, "Cannot read \"reg\" property: %i\n", err);
- return err;
- }
-
- jzgc->reg_base = bank * 0x100;
-
- jzgc->gc.label = devm_kasprintf(dev, GFP_KERNEL, "GPIO%c", 'A' + bank);
- if (!jzgc->gc.label)
- return -ENOMEM;
-
- /* DO NOT EXPAND THIS: FOR BACKWARD GPIO NUMBERSPACE COMPATIBIBILITY
- * ONLY: WORK TO TRANSITION CONSUMERS TO USE THE GPIO DESCRIPTOR API IN
- * <linux/gpio/consumer.h> INSTEAD.
- */
- jzgc->gc.base = bank * 32;
-
- jzgc->gc.ngpio = 32;
- jzgc->gc.parent = dev;
- jzgc->gc.of_node = dev->of_node;
- jzgc->gc.owner = THIS_MODULE;
- jzgc->version = (enum jz_version)of_device_get_match_data(dev);
-
- jzgc->gc.set = ingenic_gpio_set;
- jzgc->gc.get = ingenic_gpio_get;
- jzgc->gc.direction_input = ingenic_gpio_direction_input;
- jzgc->gc.direction_output = ingenic_gpio_direction_output;
-
- if (of_property_read_bool(dev->of_node, "gpio-ranges")) {
- jzgc->gc.request = gpiochip_generic_request;
- jzgc->gc.free = gpiochip_generic_free;
- }
-
- err = devm_gpiochip_add_data(dev, &jzgc->gc, jzgc);
- if (err)
- return err;
-
- jzgc->irq = irq_of_parse_and_map(dev->of_node, 0);
- if (!jzgc->irq)
- return -EINVAL;
-
- jzgc->irq_chip.name = jzgc->gc.label;
- jzgc->irq_chip.irq_enable = ingenic_gpio_irq_enable;
- jzgc->irq_chip.irq_disable = ingenic_gpio_irq_disable;
- jzgc->irq_chip.irq_unmask = ingenic_gpio_irq_unmask;
- jzgc->irq_chip.irq_mask = ingenic_gpio_irq_mask;
- jzgc->irq_chip.irq_ack = ingenic_gpio_irq_ack;
- jzgc->irq_chip.irq_set_type = ingenic_gpio_irq_set_type;
- jzgc->irq_chip.irq_set_wake = ingenic_gpio_irq_set_wake;
- jzgc->irq_chip.flags = IRQCHIP_MASK_ON_SUSPEND;
-
- err = gpiochip_irqchip_add(&jzgc->gc, &jzgc->irq_chip, 0,
- handle_level_irq, IRQ_TYPE_NONE);
- if (err)
- return err;
-
- gpiochip_set_chained_irqchip(&jzgc->gc, &jzgc->irq_chip,
- jzgc->irq, ingenic_gpio_irq_handler);
- return 0;
-}
-
-static int ingenic_gpio_remove(struct platform_device *pdev)
-{
- return 0;
-}
-
-static struct platform_driver ingenic_gpio_driver = {
- .driver = {
- .name = "gpio-ingenic",
- .of_match_table = of_match_ptr(ingenic_gpio_of_match),
- },
- .probe = ingenic_gpio_probe,
- .remove = ingenic_gpio_remove,
-};
-
-static int __init ingenic_gpio_drv_register(void)
-{
- return platform_driver_register(&ingenic_gpio_driver);
-}
-subsys_initcall(ingenic_gpio_drv_register);
-
-static void __exit ingenic_gpio_drv_unregister(void)
-{
- platform_driver_unregister(&ingenic_gpio_driver);
-}
-module_exit(ingenic_gpio_drv_unregister);
-
-MODULE_AUTHOR("Paul Cercueil <paul@crapouillou.net>");
-MODULE_DESCRIPTION("Ingenic JZ47xx GPIO driver");
-MODULE_LICENSE("GPL");
static void gpiod_set_array_single_value_cansleep(unsigned int ndescs,
struct gpio_desc **desc,
+ struct gpio_array *info,
int value)
{
- int i, *values;
+ unsigned long *values;
- values = kmalloc_array(ndescs, sizeof(*values), GFP_KERNEL);
+ values = bitmap_alloc(ndescs, GFP_KERNEL);
if (!values)
return;
- for (i = 0; i < ndescs; i++)
- values[i] = value;
+ if (value)
+ bitmap_fill(values, ndescs);
+ else
+ bitmap_zero(values, ndescs);
- gpiod_set_array_value_cansleep(ndescs, desc, values);
+ gpiod_set_array_value_cansleep(ndescs, desc, info, values);
kfree(values);
}
if (max3191x->modesel_pins)
gpiod_set_array_single_value_cansleep(
max3191x->modesel_pins->ndescs,
- max3191x->modesel_pins->desc, max3191x->mode);
+ max3191x->modesel_pins->desc,
+ max3191x->modesel_pins->info, max3191x->mode);
max3191x->ignore_uv = device_property_read_bool(dev,
"maxim,ignore-undervoltage");
+// SPDX-License-Identifier: GPL-2.0+
/*
* Generic driver for memory-mapped GPIO controllers.
*
* Copyright 2008 MontaVista Software, Inc.
* Copyright 2008,2010 Anton Vorontsov <cbouatmailru@gmail.com>
*
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- *
* ....``.```~~~~````.`.`.`.`.```````'',,,.........`````......`.......
* ...`` ```````..
* ..The simplest form of a GPIO controller that the driver supports is``
#include <linux/irq_sim.h>
#include <linux/debugfs.h>
#include <linux/uaccess.h>
+#include <linux/property.h>
#include "gpiolib.h"
* of GPIO lines.
*/
#define GPIO_MOCKUP_MAX_RANGES (GPIO_MOCKUP_MAX_GC * 2)
+/* Maximum of three properties + the sentinel. */
+#define GPIO_MOCKUP_MAX_PROP 4
#define gpio_mockup_err(...) pr_err(GPIO_MOCKUP_NAME ": " __VA_ARGS__)
int offset;
};
-struct gpio_mockup_platform_data {
- int base;
- int ngpio;
- int index;
- bool named_lines;
-};
-
static int gpio_mockup_ranges[GPIO_MOCKUP_MAX_RANGES];
static int gpio_mockup_num_ranges;
module_param_array(gpio_mockup_ranges, int, &gpio_mockup_num_ranges, 0400);
static int gpio_mockup_probe(struct platform_device *pdev)
{
- struct gpio_mockup_platform_data *pdata;
struct gpio_mockup_chip *chip;
struct gpio_chip *gc;
- int rv, base, ngpio;
struct device *dev;
- char *name;
+ const char *name;
+ int rv, base;
+ u16 ngpio;
dev = &pdev->dev;
- pdata = dev_get_platdata(dev);
- base = pdata->base;
- ngpio = pdata->ngpio;
+
+ rv = device_property_read_u32(dev, "gpio-base", &base);
+ if (rv)
+ base = -1;
+
+ rv = device_property_read_u16(dev, "nr-gpios", &ngpio);
+ if (rv)
+ return rv;
+
+ rv = device_property_read_string(dev, "chip-name", &name);
+ if (rv)
+ name = NULL;
chip = devm_kzalloc(dev, sizeof(*chip), GFP_KERNEL);
if (!chip)
return -ENOMEM;
- name = devm_kasprintf(dev, GFP_KERNEL, "%s-%c",
- pdev->name, pdata->index);
- if (!name)
- return -ENOMEM;
+ if (!name) {
+ name = devm_kasprintf(dev, GFP_KERNEL,
+ "%s-%c", pdev->name, pdev->id + 'A');
+ if (!name)
+ return -ENOMEM;
+ }
gc = &chip->gc;
gc->base = base;
if (!chip->lines)
return -ENOMEM;
- if (pdata->named_lines) {
+ if (device_property_read_bool(dev, "named-gpio-lines")) {
rv = gpio_mockup_name_lines(dev, chip);
if (rv)
return rv;
static int __init gpio_mockup_init(void)
{
- int i, num_chips, err = 0, index = 'A';
- struct gpio_mockup_platform_data pdata;
+ struct property_entry properties[GPIO_MOCKUP_MAX_PROP];
+ int i, prop, num_chips, err = 0, base;
+ struct platform_device_info pdevinfo;
struct platform_device *pdev;
+ u16 ngpio;
if ((gpio_mockup_num_ranges < 2) ||
(gpio_mockup_num_ranges % 2) ||
}
for (i = 0; i < num_chips; i++) {
- pdata.index = index++;
- pdata.base = gpio_mockup_range_base(i);
- pdata.ngpio = pdata.base < 0
- ? gpio_mockup_range_ngpio(i)
- : gpio_mockup_range_ngpio(i) - pdata.base;
- pdata.named_lines = gpio_mockup_named_lines;
-
- pdev = platform_device_register_resndata(NULL,
- GPIO_MOCKUP_NAME,
- i, NULL, 0, &pdata,
- sizeof(pdata));
+ memset(properties, 0, sizeof(properties));
+ memset(&pdevinfo, 0, sizeof(pdevinfo));
+ prop = 0;
+
+ base = gpio_mockup_range_base(i);
+ if (base >= 0)
+ properties[prop++] = PROPERTY_ENTRY_U32("gpio-base",
+ base);
+
+ ngpio = base < 0 ? gpio_mockup_range_ngpio(i)
+ : gpio_mockup_range_ngpio(i) - base;
+ properties[prop++] = PROPERTY_ENTRY_U16("nr-gpios", ngpio);
+
+ if (gpio_mockup_named_lines)
+ properties[prop++] = PROPERTY_ENTRY_BOOL(
+ "named-gpio-lines");
+
+ pdevinfo.name = GPIO_MOCKUP_NAME;
+ pdevinfo.id = i;
+ pdevinfo.properties = properties;
+
+ pdev = platform_device_register_full(&pdevinfo);
if (IS_ERR(pdev)) {
gpio_mockup_err("error registering device");
platform_driver_unregister(&gpio_mockup_driver);
#include <linux/platform_device.h>
#include <linux/slab.h>
#include <linux/gpio/driver.h>
-/* FIXME: for gpio_get_value(), replace this by direct register read */
-#include <linux/gpio.h>
#include <linux/module.h>
#define MXS_SET 0x4
port->both_edges &= ~pin_mask;
switch (type) {
case IRQ_TYPE_EDGE_BOTH:
- val = gpio_get_value(port->gc.base + d->hwirq);
+ val = port->gc.get(&port->gc, d->hwirq);
if (val)
edge = GPIO_INT_FALL_EDGE;
else
#include <linux/err.h>
#include <linux/clk.h>
#include <linux/io.h>
+#include <linux/cpu_pm.h>
#include <linux/device.h>
#include <linux/pm_runtime.h>
#include <linux/pm.h>
#include <linux/bitops.h>
#include <linux/platform_data/gpio-omap.h>
-#define OFF_MODE 1
#define OMAP4_GPIO_DEBOUNCINGTIME_MASK 0xFF
-static LIST_HEAD(omap_gpio_list);
+#define OMAP_GPIO_QUIRK_IDLE_REMOVE_TRIGGER BIT(2)
+#define OMAP_GPIO_QUIRK_DEFERRED_WKUP_EN BIT(1)
struct gpio_regs {
u32 irqenable1;
u32 debounce_en;
};
+struct gpio_bank;
+
+struct gpio_omap_funcs {
+ void (*idle_enable_level_quirk)(struct gpio_bank *bank);
+ void (*idle_disable_level_quirk)(struct gpio_bank *bank);
+};
+
struct gpio_bank {
struct list_head node;
void __iomem *base;
u32 non_wakeup_gpios;
u32 enabled_non_wakeup_gpios;
struct gpio_regs context;
+ struct gpio_omap_funcs funcs;
u32 saved_datain;
u32 level_mask;
u32 toggle_mask;
raw_spinlock_t wa_lock;
struct gpio_chip chip;
struct clk *dbck;
+ struct notifier_block nb;
+ unsigned int is_suspended:1;
u32 mod_usage;
u32 irq_usage;
u32 dbck_enable_mask;
int stride;
u32 width;
int context_loss_count;
- int power_mode;
bool workaround_enabled;
+ u32 quirks;
void (*set_dataout)(struct gpio_bank *bank, unsigned gpio, int enable);
void (*set_dataout_multiple)(struct gpio_bank *bank,
readl_relaxed(bank->base + bank->regs->fallingdetect);
if (likely(!(bank->non_wakeup_gpios & gpio_bit))) {
- omap_gpio_rmw(base, bank->regs->wkup_en, gpio_bit, trigger != 0);
- bank->context.wake_en =
- readl_relaxed(bank->base + bank->regs->wkup_en);
+ /* Defer wkup_en register update until we idle? */
+ if (bank->quirks & OMAP_GPIO_QUIRK_DEFERRED_WKUP_EN) {
+ if (trigger)
+ bank->context.wake_en |= gpio_bit;
+ else
+ bank->context.wake_en &= ~gpio_bit;
+ } else {
+ omap_gpio_rmw(base, bank->regs->wkup_en, gpio_bit,
+ trigger != 0);
+ bank->context.wake_en =
+ readl_relaxed(bank->base + bank->regs->wkup_en);
+ }
}
/* This part needs to be executed always for OMAP{34xx, 44xx} */
struct gpio_bank *bank = gpiochip_get_data(chip);
unsigned long flags;
- /*
- * If this is the first gpio_request for the bank,
- * enable the bank module.
- */
- if (!BANK_USED(bank))
- pm_runtime_get_sync(chip->parent);
+ pm_runtime_get_sync(chip->parent);
raw_spin_lock_irqsave(&bank->lock, flags);
omap_enable_gpio_module(bank, offset);
omap_disable_gpio_module(bank, offset);
raw_spin_unlock_irqrestore(&bank->lock, flags);
- /*
- * If this is the last gpio to be freed in the bank,
- * disable the bank module.
- */
- if (!BANK_USED(bank))
- pm_runtime_put(chip->parent);
+ pm_runtime_put(chip->parent);
}
/*
if (WARN_ON(!isr_reg))
goto exit;
- pm_runtime_get_sync(bank->chip.parent);
+ if (WARN_ONCE(!pm_runtime_active(bank->chip.parent),
+ "gpio irq%i while runtime suspended?\n", irq))
+ return IRQ_NONE;
while (1) {
raw_spin_lock_irqsave(&bank->lock, lock_flags);
}
}
exit:
- pm_runtime_put(bank->chip.parent);
return IRQ_HANDLED;
}
{
struct gpio_bank *bank = omap_irq_data_get_bank(data);
- if (!BANK_USED(bank))
- pm_runtime_get_sync(bank->chip.parent);
+ pm_runtime_get_sync(bank->chip.parent);
}
static void gpio_irq_bus_sync_unlock(struct irq_data *data)
{
struct gpio_bank *bank = omap_irq_data_get_bank(data);
- /*
- * If this is the last IRQ to be freed in the bank,
- * disable the bank module.
- */
- if (!BANK_USED(bank))
- pm_runtime_put(bank->chip.parent);
+ pm_runtime_put(bank->chip.parent);
}
static void omap_gpio_ack_irq(struct irq_data *d)
raw_spin_unlock_irqrestore(&bank->lock, flags);
}
+/*
+ * Only edges can generate a wakeup event to the PRCM.
+ *
+ * Therefore, ensure any wake-up capable GPIOs have
+ * edge-detection enabled before going idle to ensure a wakeup
+ * to the PRCM is generated on a GPIO transition. (c.f. 34xx
+ * NDA TRM 25.5.3.1)
+ *
+ * The normal values will be restored upon ->runtime_resume()
+ * by writing back the values saved in bank->context.
+ */
+static void __maybe_unused
+omap2_gpio_enable_level_quirk(struct gpio_bank *bank)
+{
+ u32 wake_low, wake_hi;
+
+ /* Enable additional edge detection for level gpios for idle */
+ wake_low = bank->context.leveldetect0 & bank->context.wake_en;
+ if (wake_low)
+ writel_relaxed(wake_low | bank->context.fallingdetect,
+ bank->base + bank->regs->fallingdetect);
+
+ wake_hi = bank->context.leveldetect1 & bank->context.wake_en;
+ if (wake_hi)
+ writel_relaxed(wake_hi | bank->context.risingdetect,
+ bank->base + bank->regs->risingdetect);
+}
+
+static void __maybe_unused
+omap2_gpio_disable_level_quirk(struct gpio_bank *bank)
+{
+ /* Disable edge detection for level gpios after idle */
+ writel_relaxed(bank->context.fallingdetect,
+ bank->base + bank->regs->fallingdetect);
+ writel_relaxed(bank->context.risingdetect,
+ bank->base + bank->regs->risingdetect);
+}
+
+/*
+ * On omap4 and later SoC variants a level interrupt with wkup_en
+ * enabled blocks the GPIO functional clock from idling until the GPIO
+ * instance has been reset. To avoid that, we must set wkup_en only for
+ * idle for level interrupts, and clear level registers for the duration
+ * of idle. The level interrupts will be still there on wakeup by their
+ * nature.
+ */
+static void __maybe_unused
+omap4_gpio_enable_level_quirk(struct gpio_bank *bank)
+{
+ /* Update wake register for idle, edge bits might be already set */
+ writel_relaxed(bank->context.wake_en,
+ bank->base + bank->regs->wkup_en);
+
+ /* Clear level registers for idle */
+ writel_relaxed(0, bank->base + bank->regs->leveldetect0);
+ writel_relaxed(0, bank->base + bank->regs->leveldetect1);
+}
+
+static void __maybe_unused
+omap4_gpio_disable_level_quirk(struct gpio_bank *bank)
+{
+ /* Restore level registers after idle */
+ writel_relaxed(bank->context.leveldetect0,
+ bank->base + bank->regs->leveldetect0);
+ writel_relaxed(bank->context.leveldetect1,
+ bank->base + bank->regs->leveldetect1);
+
+ /* Clear saved wkup_en for level, it will be set for next idle again */
+ bank->context.wake_en &= ~(bank->context.leveldetect0 |
+ bank->context.leveldetect1);
+
+ /* Update wake with only edge configuration */
+ writel_relaxed(bank->context.wake_en,
+ bank->base + bank->regs->wkup_en);
+}
+
/*---------------------------------------------------------------------*/
static int omap_mpuio_suspend_noirq(struct device *dev)
return ret;
}
+static void omap_gpio_idle(struct gpio_bank *bank, bool may_lose_context);
+static void omap_gpio_unidle(struct gpio_bank *bank);
+
+static int gpio_omap_cpu_notifier(struct notifier_block *nb,
+ unsigned long cmd, void *v)
+{
+ struct gpio_bank *bank;
+ unsigned long flags;
+
+ bank = container_of(nb, struct gpio_bank, nb);
+
+ raw_spin_lock_irqsave(&bank->lock, flags);
+ switch (cmd) {
+ case CPU_CLUSTER_PM_ENTER:
+ if (bank->is_suspended)
+ break;
+ omap_gpio_idle(bank, true);
+ break;
+ case CPU_CLUSTER_PM_ENTER_FAILED:
+ case CPU_CLUSTER_PM_EXIT:
+ if (bank->is_suspended)
+ break;
+ omap_gpio_unidle(bank);
+ break;
+ }
+ raw_spin_unlock_irqrestore(&bank->lock, flags);
+
+ return NOTIFY_OK;
+}
+
static const struct of_device_id omap_gpio_match[];
static int omap_gpio_probe(struct platform_device *pdev)
irqc->irq_bus_sync_unlock = gpio_irq_bus_sync_unlock,
irqc->name = dev_name(&pdev->dev);
irqc->flags = IRQCHIP_MASK_ON_SUSPEND;
+ irqc->parent_device = dev;
bank->irq = platform_get_irq(pdev, 0);
if (bank->irq <= 0) {
bank->chip.parent = dev;
bank->chip.owner = THIS_MODULE;
bank->dbck_flag = pdata->dbck_flag;
+ bank->quirks = pdata->quirks;
bank->stride = pdata->bank_stride;
bank->width = pdata->bank_width;
bank->is_mpuio = pdata->is_mpuio;
#ifdef CONFIG_OF_GPIO
bank->chip.of_node = of_node_get(node);
#endif
+
if (node) {
if (!of_property_read_bool(node, "ti,gpio-always-on"))
bank->loses_context = true;
omap_set_gpio_dataout_mask_multiple;
}
+ if (bank->quirks & OMAP_GPIO_QUIRK_DEFERRED_WKUP_EN) {
+ bank->funcs.idle_enable_level_quirk =
+ omap4_gpio_enable_level_quirk;
+ bank->funcs.idle_disable_level_quirk =
+ omap4_gpio_disable_level_quirk;
+ } else if (bank->quirks & OMAP_GPIO_QUIRK_IDLE_REMOVE_TRIGGER) {
+ bank->funcs.idle_enable_level_quirk =
+ omap2_gpio_enable_level_quirk;
+ bank->funcs.idle_disable_level_quirk =
+ omap2_gpio_disable_level_quirk;
+ }
+
raw_spin_lock_init(&bank->lock);
raw_spin_lock_init(&bank->wa_lock);
platform_set_drvdata(pdev, bank);
pm_runtime_enable(dev);
- pm_runtime_irq_safe(dev);
pm_runtime_get_sync(dev);
if (bank->is_mpuio)
omap_gpio_show_rev(bank);
- pm_runtime_put(dev);
+ if (bank->funcs.idle_enable_level_quirk &&
+ bank->funcs.idle_disable_level_quirk) {
+ bank->nb.notifier_call = gpio_omap_cpu_notifier;
+ cpu_pm_register_notifier(&bank->nb);
+ }
- list_add_tail(&bank->node, &omap_gpio_list);
+ pm_runtime_put(dev);
return 0;
}
{
struct gpio_bank *bank = platform_get_drvdata(pdev);
+ if (bank->nb.notifier_call)
+ cpu_pm_unregister_notifier(&bank->nb);
list_del(&bank->node);
gpiochip_remove(&bank->chip);
pm_runtime_disable(&pdev->dev);
return 0;
}
-#ifdef CONFIG_ARCH_OMAP2PLUS
-
-#if defined(CONFIG_PM)
static void omap_gpio_restore_context(struct gpio_bank *bank);
-static int omap_gpio_runtime_suspend(struct device *dev)
+static void omap_gpio_idle(struct gpio_bank *bank, bool may_lose_context)
{
- struct platform_device *pdev = to_platform_device(dev);
- struct gpio_bank *bank = platform_get_drvdata(pdev);
+ struct device *dev = bank->chip.parent;
u32 l1 = 0, l2 = 0;
- unsigned long flags;
- u32 wake_low, wake_hi;
- raw_spin_lock_irqsave(&bank->lock, flags);
-
- /*
- * Only edges can generate a wakeup event to the PRCM.
- *
- * Therefore, ensure any wake-up capable GPIOs have
- * edge-detection enabled before going idle to ensure a wakeup
- * to the PRCM is generated on a GPIO transition. (c.f. 34xx
- * NDA TRM 25.5.3.1)
- *
- * The normal values will be restored upon ->runtime_resume()
- * by writing back the values saved in bank->context.
- */
- wake_low = bank->context.leveldetect0 & bank->context.wake_en;
- if (wake_low)
- writel_relaxed(wake_low | bank->context.fallingdetect,
- bank->base + bank->regs->fallingdetect);
- wake_hi = bank->context.leveldetect1 & bank->context.wake_en;
- if (wake_hi)
- writel_relaxed(wake_hi | bank->context.risingdetect,
- bank->base + bank->regs->risingdetect);
+ if (bank->funcs.idle_enable_level_quirk)
+ bank->funcs.idle_enable_level_quirk(bank);
if (!bank->enabled_non_wakeup_gpios)
goto update_gpio_context_count;
- if (bank->power_mode != OFF_MODE) {
- bank->power_mode = 0;
+ if (!may_lose_context)
goto update_gpio_context_count;
- }
+
/*
* If going to OFF, remove triggering for all
* non-wakeup GPIOs. Otherwise spurious IRQs will be
bank->get_context_loss_count(dev);
omap_gpio_dbck_disable(bank);
- raw_spin_unlock_irqrestore(&bank->lock, flags);
-
- return 0;
}
static void omap_gpio_init_context(struct gpio_bank *p);
-static int omap_gpio_runtime_resume(struct device *dev)
+static void omap_gpio_unidle(struct gpio_bank *bank)
{
- struct platform_device *pdev = to_platform_device(dev);
- struct gpio_bank *bank = platform_get_drvdata(pdev);
+ struct device *dev = bank->chip.parent;
u32 l = 0, gen, gen0, gen1;
- unsigned long flags;
int c;
- raw_spin_lock_irqsave(&bank->lock, flags);
-
/*
* On the first resume during the probe, the context has not
* been initialised and so initialise it now. Also initialise
omap_gpio_dbck_enable(bank);
- /*
- * In ->runtime_suspend(), level-triggered, wakeup-enabled
- * GPIOs were set to edge trigger also in order to be able to
- * generate a PRCM wakeup. Here we restore the
- * pre-runtime_suspend() values for edge triggering.
- */
- writel_relaxed(bank->context.fallingdetect,
- bank->base + bank->regs->fallingdetect);
- writel_relaxed(bank->context.risingdetect,
- bank->base + bank->regs->risingdetect);
+ if (bank->funcs.idle_disable_level_quirk)
+ bank->funcs.idle_disable_level_quirk(bank);
if (bank->loses_context) {
if (!bank->get_context_loss_count) {
if (c != bank->context_loss_count) {
omap_gpio_restore_context(bank);
} else {
- raw_spin_unlock_irqrestore(&bank->lock, flags);
- return 0;
+ return;
}
}
}
- if (!bank->workaround_enabled) {
- raw_spin_unlock_irqrestore(&bank->lock, flags);
- return 0;
- }
+ if (!bank->workaround_enabled)
+ return;
l = readl_relaxed(bank->base + bank->regs->datain);
}
bank->workaround_enabled = false;
- raw_spin_unlock_irqrestore(&bank->lock, flags);
-
- return 0;
-}
-#endif /* CONFIG_PM */
-
-#if IS_BUILTIN(CONFIG_GPIO_OMAP)
-void omap2_gpio_prepare_for_idle(int pwr_mode)
-{
- struct gpio_bank *bank;
-
- list_for_each_entry(bank, &omap_gpio_list, node) {
- if (!BANK_USED(bank) || !bank->loses_context)
- continue;
-
- bank->power_mode = pwr_mode;
-
- pm_runtime_put_sync_suspend(bank->chip.parent);
- }
-}
-
-void omap2_gpio_resume_after_idle(void)
-{
- struct gpio_bank *bank;
-
- list_for_each_entry(bank, &omap_gpio_list, node) {
- if (!BANK_USED(bank) || !bank->loses_context)
- continue;
-
- pm_runtime_get_sync(bank->chip.parent);
- }
}
-#endif
-#if defined(CONFIG_PM)
static void omap_gpio_init_context(struct gpio_bank *p)
{
struct omap_gpio_reg_offs *regs = p->regs;
writel_relaxed(bank->context.irqenable2,
bank->base + bank->regs->irqenable2);
}
-#endif /* CONFIG_PM */
-#else
-#define omap_gpio_runtime_suspend NULL
-#define omap_gpio_runtime_resume NULL
-static inline void omap_gpio_init_context(struct gpio_bank *p) {}
-#endif
+static int __maybe_unused omap_gpio_runtime_suspend(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct gpio_bank *bank = platform_get_drvdata(pdev);
+ unsigned long flags;
+ int error = 0;
+
+ raw_spin_lock_irqsave(&bank->lock, flags);
+ /* Must be idled only by CPU_CLUSTER_PM_ENTER? */
+ if (bank->irq_usage) {
+ error = -EBUSY;
+ goto unlock;
+ }
+ omap_gpio_idle(bank, true);
+ bank->is_suspended = true;
+unlock:
+ raw_spin_unlock_irqrestore(&bank->lock, flags);
+
+ return error;
+}
+
+static int __maybe_unused omap_gpio_runtime_resume(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct gpio_bank *bank = platform_get_drvdata(pdev);
+ unsigned long flags;
+ int error = 0;
+
+ raw_spin_lock_irqsave(&bank->lock, flags);
+ /* Must be unidled only by CPU_CLUSTER_PM_ENTER? */
+ if (bank->irq_usage) {
+ error = -EBUSY;
+ goto unlock;
+ }
+ omap_gpio_unidle(bank);
+ bank->is_suspended = false;
+unlock:
+ raw_spin_unlock_irqrestore(&bank->lock, flags);
+
+ return error;
+}
+
+#ifdef CONFIG_ARCH_OMAP2PLUS
static const struct dev_pm_ops gpio_pm_ops = {
SET_RUNTIME_PM_OPS(omap_gpio_runtime_suspend, omap_gpio_runtime_resume,
NULL)
};
+#else
+static const struct dev_pm_ops gpio_pm_ops;
+#endif /* CONFIG_ARCH_OMAP2PLUS */
#if defined(CONFIG_OF)
static struct omap_gpio_reg_offs omap2_gpio_regs = {
.fallingdetect = OMAP4_GPIO_FALLINGDETECT,
};
+/*
+ * Note that omap2 does not currently support idle modes with context loss so
+ * no need to add OMAP_GPIO_QUIRK_IDLE_REMOVE_TRIGGER quirk flag to save
+ * and restore context.
+ */
static const struct omap_gpio_platform_data omap2_pdata = {
.regs = &omap2_gpio_regs,
.bank_width = 32,
.regs = &omap2_gpio_regs,
.bank_width = 32,
.dbck_flag = true,
+ .quirks = OMAP_GPIO_QUIRK_IDLE_REMOVE_TRIGGER,
};
static const struct omap_gpio_platform_data omap4_pdata = {
.regs = &omap4_gpio_regs,
.bank_width = 32,
.dbck_flag = true,
+ .quirks = OMAP_GPIO_QUIRK_IDLE_REMOVE_TRIGGER |
+ OMAP_GPIO_QUIRK_DEFERRED_WKUP_EN,
};
static const struct of_device_id omap_gpio_match[] = {
struct pxa_gpio_bank *c;
int gpio;
+ if (!pchip)
+ return 0;
+
for_each_gpio_bank(gpio, c, pchip) {
c->saved_gplr = readl_relaxed(c->regbase + GPLR_OFFSET);
c->saved_gpdr = readl_relaxed(c->regbase + GPDR_OFFSET);
struct pxa_gpio_bank *c;
int gpio;
+ if (!pchip)
+ return;
+
for_each_gpio_bank(gpio, c, pchip) {
/* restore level with set/clear */
writel_relaxed(c->saved_gplr, c->regbase + GPSR_OFFSET);
u32 val, bankmask;
bankmask = mask[0] & GENMASK(chip->ngpio - 1, 0);
+ if (chip->valid_mask)
+ bankmask &= chip->valid_mask[0];
+
if (!bankmask)
return;
u32 mask;
for (offset = 0; offset < p->gpio_chip.ngpio; offset++) {
+ if (!gpiochip_line_is_valid(&p->gpio_chip, offset))
+ continue;
+
mask = BIT(offset);
/* I/O pin */
if (!(p->bank_info.iointsel & mask)) {
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2015-2018 Pengutronix, Uwe Kleine-König <kernel@pengutronix.de>
+ */
+
+#include <linux/module.h>
+#include <linux/siox.h>
+#include <linux/gpio/driver.h>
+#include <linux/of.h>
+
+struct gpio_siox_ddata {
+ struct gpio_chip gchip;
+ struct irq_chip ichip;
+ struct mutex lock;
+ u8 setdata[1];
+ u8 getdata[3];
+
+ spinlock_t irqlock;
+ u32 irq_enable;
+ u32 irq_status;
+ u32 irq_type[20];
+};
+
+/*
+ * Note that this callback only sets the value that is clocked out in the next
+ * cycle.
+ */
+static int gpio_siox_set_data(struct siox_device *sdevice, u8 status, u8 buf[])
+{
+ struct gpio_siox_ddata *ddata = dev_get_drvdata(&sdevice->dev);
+
+ mutex_lock(&ddata->lock);
+ buf[0] = ddata->setdata[0];
+ mutex_unlock(&ddata->lock);
+
+ return 0;
+}
+
+static int gpio_siox_get_data(struct siox_device *sdevice, const u8 buf[])
+{
+ struct gpio_siox_ddata *ddata = dev_get_drvdata(&sdevice->dev);
+ size_t offset;
+ u32 trigger;
+
+ mutex_lock(&ddata->lock);
+
+ spin_lock_irq(&ddata->irqlock);
+
+ for (offset = 0; offset < 12; ++offset) {
+ unsigned int bitpos = 11 - offset;
+ unsigned int gpiolevel = buf[bitpos / 8] & (1 << bitpos % 8);
+ unsigned int prev_level =
+ ddata->getdata[bitpos / 8] & (1 << (bitpos % 8));
+ u32 irq_type = ddata->irq_type[offset];
+
+ if (gpiolevel) {
+ if ((irq_type & IRQ_TYPE_LEVEL_HIGH) ||
+ ((irq_type & IRQ_TYPE_EDGE_RISING) && !prev_level))
+ ddata->irq_status |= 1 << offset;
+ } else {
+ if ((irq_type & IRQ_TYPE_LEVEL_LOW) ||
+ ((irq_type & IRQ_TYPE_EDGE_FALLING) && prev_level))
+ ddata->irq_status |= 1 << offset;
+ }
+ }
+
+ trigger = ddata->irq_status & ddata->irq_enable;
+
+ spin_unlock_irq(&ddata->irqlock);
+
+ ddata->getdata[0] = buf[0];
+ ddata->getdata[1] = buf[1];
+ ddata->getdata[2] = buf[2];
+
+ mutex_unlock(&ddata->lock);
+
+ for (offset = 0; offset < 12; ++offset) {
+ if (trigger & (1 << offset)) {
+ struct irq_domain *irqdomain = ddata->gchip.irq.domain;
+ unsigned int irq = irq_find_mapping(irqdomain, offset);
+
+ /*
+ * Conceptually handle_nested_irq should call the flow
+ * handler of the irq chip. But it doesn't, so we have
+ * to clean the irq_status here.
+ */
+ spin_lock_irq(&ddata->irqlock);
+ ddata->irq_status &= ~(1 << offset);
+ spin_unlock_irq(&ddata->irqlock);
+
+ handle_nested_irq(irq);
+ }
+ }
+
+ return 0;
+}
+
+static void gpio_siox_irq_ack(struct irq_data *d)
+{
+ struct irq_chip *ic = irq_data_get_irq_chip(d);
+ struct gpio_siox_ddata *ddata =
+ container_of(ic, struct gpio_siox_ddata, ichip);
+
+ spin_lock_irq(&ddata->irqlock);
+ ddata->irq_status &= ~(1 << d->hwirq);
+ spin_unlock_irq(&ddata->irqlock);
+}
+
+static void gpio_siox_irq_mask(struct irq_data *d)
+{
+ struct irq_chip *ic = irq_data_get_irq_chip(d);
+ struct gpio_siox_ddata *ddata =
+ container_of(ic, struct gpio_siox_ddata, ichip);
+
+ spin_lock_irq(&ddata->irqlock);
+ ddata->irq_enable &= ~(1 << d->hwirq);
+ spin_unlock_irq(&ddata->irqlock);
+}
+
+static void gpio_siox_irq_unmask(struct irq_data *d)
+{
+ struct irq_chip *ic = irq_data_get_irq_chip(d);
+ struct gpio_siox_ddata *ddata =
+ container_of(ic, struct gpio_siox_ddata, ichip);
+
+ spin_lock_irq(&ddata->irqlock);
+ ddata->irq_enable |= 1 << d->hwirq;
+ spin_unlock_irq(&ddata->irqlock);
+}
+
+static int gpio_siox_irq_set_type(struct irq_data *d, u32 type)
+{
+ struct irq_chip *ic = irq_data_get_irq_chip(d);
+ struct gpio_siox_ddata *ddata =
+ container_of(ic, struct gpio_siox_ddata, ichip);
+
+ spin_lock_irq(&ddata->irqlock);
+ ddata->irq_type[d->hwirq] = type;
+ spin_unlock_irq(&ddata->irqlock);
+
+ return 0;
+}
+
+static int gpio_siox_get(struct gpio_chip *chip, unsigned int offset)
+{
+ struct gpio_siox_ddata *ddata =
+ container_of(chip, struct gpio_siox_ddata, gchip);
+ int ret;
+
+ mutex_lock(&ddata->lock);
+
+ if (offset >= 12) {
+ unsigned int bitpos = 19 - offset;
+
+ ret = ddata->setdata[0] & (1 << bitpos);
+ } else {
+ unsigned int bitpos = 11 - offset;
+
+ ret = ddata->getdata[bitpos / 8] & (1 << (bitpos % 8));
+ }
+
+ mutex_unlock(&ddata->lock);
+
+ return ret;
+}
+
+static void gpio_siox_set(struct gpio_chip *chip,
+ unsigned int offset, int value)
+{
+ struct gpio_siox_ddata *ddata =
+ container_of(chip, struct gpio_siox_ddata, gchip);
+ u8 mask = 1 << (19 - offset);
+
+ mutex_lock(&ddata->lock);
+
+ if (value)
+ ddata->setdata[0] |= mask;
+ else
+ ddata->setdata[0] &= ~mask;
+
+ mutex_unlock(&ddata->lock);
+}
+
+static int gpio_siox_direction_input(struct gpio_chip *chip,
+ unsigned int offset)
+{
+ if (offset >= 12)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int gpio_siox_direction_output(struct gpio_chip *chip,
+ unsigned int offset, int value)
+{
+ if (offset < 12)
+ return -EINVAL;
+
+ gpio_siox_set(chip, offset, value);
+ return 0;
+}
+
+static int gpio_siox_get_direction(struct gpio_chip *chip, unsigned int offset)
+{
+ if (offset < 12)
+ return 1; /* input */
+ else
+ return 0; /* output */
+}
+
+static int gpio_siox_probe(struct siox_device *sdevice)
+{
+ struct gpio_siox_ddata *ddata;
+ int ret;
+
+ ddata = devm_kzalloc(&sdevice->dev, sizeof(*ddata), GFP_KERNEL);
+ if (!ddata)
+ return -ENOMEM;
+
+ dev_set_drvdata(&sdevice->dev, ddata);
+
+ mutex_init(&ddata->lock);
+ spin_lock_init(&ddata->irqlock);
+
+ ddata->gchip.base = -1;
+ ddata->gchip.can_sleep = 1;
+ ddata->gchip.parent = &sdevice->dev;
+ ddata->gchip.owner = THIS_MODULE;
+ ddata->gchip.get = gpio_siox_get;
+ ddata->gchip.set = gpio_siox_set;
+ ddata->gchip.direction_input = gpio_siox_direction_input;
+ ddata->gchip.direction_output = gpio_siox_direction_output;
+ ddata->gchip.get_direction = gpio_siox_get_direction;
+ ddata->gchip.ngpio = 20;
+
+ ddata->ichip.name = "siox-gpio";
+ ddata->ichip.irq_ack = gpio_siox_irq_ack;
+ ddata->ichip.irq_mask = gpio_siox_irq_mask;
+ ddata->ichip.irq_unmask = gpio_siox_irq_unmask;
+ ddata->ichip.irq_set_type = gpio_siox_irq_set_type;
+
+ ret = gpiochip_add(&ddata->gchip);
+ if (ret) {
+ dev_err(&sdevice->dev,
+ "Failed to register gpio chip (%d)\n", ret);
+ goto err_gpiochip;
+ }
+
+ ret = gpiochip_irqchip_add(&ddata->gchip, &ddata->ichip,
+ 0, handle_level_irq, IRQ_TYPE_EDGE_RISING);
+ if (ret) {
+ dev_err(&sdevice->dev,
+ "Failed to register irq chip (%d)\n", ret);
+err_gpiochip:
+ gpiochip_remove(&ddata->gchip);
+ }
+
+ return ret;
+}
+
+static int gpio_siox_remove(struct siox_device *sdevice)
+{
+ struct gpio_siox_ddata *ddata = dev_get_drvdata(&sdevice->dev);
+
+ gpiochip_remove(&ddata->gchip);
+ return 0;
+}
+
+static struct siox_driver gpio_siox_driver = {
+ .probe = gpio_siox_probe,
+ .remove = gpio_siox_remove,
+ .set_data = gpio_siox_set_data,
+ .get_data = gpio_siox_get_data,
+ .driver = {
+ .name = "gpio-siox",
+ },
+};
+
+static int __init gpio_siox_init(void)
+{
+ return siox_driver_register(&gpio_siox_driver);
+}
+module_init(gpio_siox_init);
+
+static void __exit gpio_siox_exit(void)
+{
+ siox_driver_unregister(&gpio_siox_driver);
+}
+module_exit(gpio_siox_exit);
+
+MODULE_AUTHOR("Uwe Kleine-Koenig <u.kleine-koenig@pengutronix.de>");
+MODULE_DESCRIPTION("SIOX gpio driver");
+MODULE_LICENSE("GPL v2");
BIT(offs % SYSCON_REG_BITS));
}
- priv->data->set(chip, offset, val);
+ chip->set(chip, offset, val);
return 0;
}
/**
- * @spinlock: used for atomic read/modify/write of registers
* @base: register base address
* @domain: IRQ domain of GPIO generated interrupts managed by this controller
* @irq: Interrupt line of parent interrupt controller
* @gc: gpio_chip structure associated to this GPIO controller
*/
struct tb10x_gpio {
- spinlock_t spinlock;
void __iomem *base;
struct irq_domain *domain;
int irq;
u32 r;
unsigned long flags;
- spin_lock_irqsave(&gpio->spinlock, flags);
+ spin_lock_irqsave(&gpio->gc.bgpio_lock, flags);
r = tb10x_reg_read(gpio, offs);
r = (r & ~mask) | (val & mask);
tb10x_reg_write(gpio, offs, r);
- spin_unlock_irqrestore(&gpio->spinlock, flags);
-}
-
-static int tb10x_gpio_direction_in(struct gpio_chip *chip, unsigned offset)
-{
- struct tb10x_gpio *tb10x_gpio = gpiochip_get_data(chip);
- int mask = BIT(offset);
- int val = TB10X_GPIO_DIR_IN << offset;
-
- tb10x_set_bits(tb10x_gpio, OFFSET_TO_REG_DDR, mask, val);
-
- return 0;
-}
-
-static int tb10x_gpio_get(struct gpio_chip *chip, unsigned offset)
-{
- struct tb10x_gpio *tb10x_gpio = gpiochip_get_data(chip);
- int val;
-
- val = tb10x_reg_read(tb10x_gpio, OFFSET_TO_REG_DATA);
-
- if (val & BIT(offset))
- return 1;
- else
- return 0;
-}
-
-static void tb10x_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
-{
- struct tb10x_gpio *tb10x_gpio = gpiochip_get_data(chip);
- int mask = BIT(offset);
- int val = value << offset;
-
- tb10x_set_bits(tb10x_gpio, OFFSET_TO_REG_DATA, mask, val);
-}
-
-static int tb10x_gpio_direction_out(struct gpio_chip *chip,
- unsigned offset, int value)
-{
- struct tb10x_gpio *tb10x_gpio = gpiochip_get_data(chip);
- int mask = BIT(offset);
- int val = TB10X_GPIO_DIR_OUT << offset;
-
- tb10x_gpio_set(chip, offset, value);
- tb10x_set_bits(tb10x_gpio, OFFSET_TO_REG_DDR, mask, val);
-
- return 0;
+ spin_unlock_irqrestore(&gpio->gc.bgpio_lock, flags);
}
static int tb10x_gpio_to_irq(struct gpio_chip *chip, unsigned offset)
{
struct tb10x_gpio *tb10x_gpio;
struct resource *mem;
- struct device_node *dn = pdev->dev.of_node;
+ struct device *dev = &pdev->dev;
+ struct device_node *np = dev->of_node;
int ret = -EBUSY;
u32 ngpio;
- if (!dn)
+ if (!np)
return -EINVAL;
- if (of_property_read_u32(dn, "abilis,ngpio", &ngpio))
+ if (of_property_read_u32(np, "abilis,ngpio", &ngpio))
return -EINVAL;
- tb10x_gpio = devm_kzalloc(&pdev->dev, sizeof(*tb10x_gpio), GFP_KERNEL);
+ tb10x_gpio = devm_kzalloc(dev, sizeof(*tb10x_gpio), GFP_KERNEL);
if (tb10x_gpio == NULL)
return -ENOMEM;
- spin_lock_init(&tb10x_gpio->spinlock);
-
mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- tb10x_gpio->base = devm_ioremap_resource(&pdev->dev, mem);
+ tb10x_gpio->base = devm_ioremap_resource(dev, mem);
if (IS_ERR(tb10x_gpio->base))
return PTR_ERR(tb10x_gpio->base);
- tb10x_gpio->gc.label =
- devm_kasprintf(&pdev->dev, GFP_KERNEL, "%pOF", pdev->dev.of_node);
+ tb10x_gpio->gc.label =
+ devm_kasprintf(dev, GFP_KERNEL, "%pOF", pdev->dev.of_node);
if (!tb10x_gpio->gc.label)
return -ENOMEM;
- tb10x_gpio->gc.parent = &pdev->dev;
- tb10x_gpio->gc.owner = THIS_MODULE;
- tb10x_gpio->gc.direction_input = tb10x_gpio_direction_in;
- tb10x_gpio->gc.get = tb10x_gpio_get;
- tb10x_gpio->gc.direction_output = tb10x_gpio_direction_out;
- tb10x_gpio->gc.set = tb10x_gpio_set;
- tb10x_gpio->gc.request = gpiochip_generic_request;
- tb10x_gpio->gc.free = gpiochip_generic_free;
- tb10x_gpio->gc.base = -1;
- tb10x_gpio->gc.ngpio = ngpio;
- tb10x_gpio->gc.can_sleep = false;
-
-
- ret = devm_gpiochip_add_data(&pdev->dev, &tb10x_gpio->gc, tb10x_gpio);
+ /*
+ * Initialize generic GPIO with one single register for reading and setting
+ * the lines, no special set or clear registers and a data direction register
+ * wher 1 means "output".
+ */
+ ret = bgpio_init(&tb10x_gpio->gc, dev, 4,
+ tb10x_gpio->base + OFFSET_TO_REG_DATA,
+ NULL,
+ NULL,
+ tb10x_gpio->base + OFFSET_TO_REG_DDR,
+ NULL,
+ 0);
+ if (ret) {
+ dev_err(dev, "unable to init generic GPIO\n");
+ return ret;
+ }
+ tb10x_gpio->gc.base = -1;
+ tb10x_gpio->gc.parent = dev;
+ tb10x_gpio->gc.owner = THIS_MODULE;
+ /*
+ * ngpio is set by bgpio_init() but we override it, this .request()
+ * callback also overrides the one set up by generic GPIO.
+ */
+ tb10x_gpio->gc.ngpio = ngpio;
+ tb10x_gpio->gc.request = gpiochip_generic_request;
+ tb10x_gpio->gc.free = gpiochip_generic_free;
+
+ ret = devm_gpiochip_add_data(dev, &tb10x_gpio->gc, tb10x_gpio);
if (ret < 0) {
- dev_err(&pdev->dev, "Could not add gpiochip.\n");
+ dev_err(dev, "Could not add gpiochip.\n");
return ret;
}
platform_set_drvdata(pdev, tb10x_gpio);
- if (of_find_property(dn, "interrupt-controller", NULL)) {
+ if (of_find_property(np, "interrupt-controller", NULL)) {
struct irq_chip_generic *gc;
ret = platform_get_irq(pdev, 0);
if (ret < 0) {
- dev_err(&pdev->dev, "No interrupt specified.\n");
+ dev_err(dev, "No interrupt specified.\n");
return ret;
}
tb10x_gpio->gc.to_irq = tb10x_gpio_to_irq;
tb10x_gpio->irq = ret;
- ret = devm_request_irq(&pdev->dev, ret, tb10x_gpio_irq_cascade,
+ ret = devm_request_irq(dev, ret, tb10x_gpio_irq_cascade,
IRQF_TRIGGER_NONE | IRQF_SHARED,
- dev_name(&pdev->dev), tb10x_gpio);
+ dev_name(dev), tb10x_gpio);
if (ret != 0)
return ret;
- tb10x_gpio->domain = irq_domain_add_linear(dn,
+ tb10x_gpio->domain = irq_domain_add_linear(np,
tb10x_gpio->gc.ngpio,
&irq_generic_chip_ops, NULL);
if (!tb10x_gpio->domain) {
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2015 Texas Instruments Incorporated - http://www.ti.com/
* Andrew F. Davis <afd@ti.com>
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed "as is" WITHOUT ANY WARRANTY of any
- * kind, whether expressed or implied; without even the implied warranty
- * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License version 2 for more details.
- *
* Based on the TPS65912 driver
*/
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* TI TPS6586x GPIO driver
*
* Based on tps6586x.c
* Copyright (c) 2010 CompuLab Ltd.
* Mike Rapoport <mike@compulab.co.il>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <linux/errno.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/mfd/tps6586x.h>
+// SPDX-License-Identifier: GPL-2.0+
/*
* TI TPS6591x GPIO driver
*
*
* Author: Graeme Gregory <gg@slimlogic.co.uk>
* Author: Jorge Eduardo Candelaria <jedu@slimlogic.co.uk>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- *
*/
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/errno.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/mfd/tps65910.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* GPIO driver for TI TPS65912x PMICs
*
* Copyright (C) 2015 Texas Instruments Incorporated - http://www.ti.com/
* Andrew F. Davis <afd@ti.com>
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed "as is" WITHOUT ANY WARRANTY of any
- * kind, whether expressed or implied; without even the implied warranty
- * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License version 2 for more details.
- *
* Based on the Arizona GPIO driver and the previous TPS65912 driver by
* Margarita Olaya Cabrera <magi@slimlogic.co.uk>
*/
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/module.h>
#include <linux/platform_device.h>
return ret;
if (val & GPIO_CFG_MASK)
- return GPIOF_DIR_OUT;
+ return 0;
else
- return GPIOF_DIR_IN;
+ return 1;
}
static int tps65912_gpio_direction_input(struct gpio_chip *gc, unsigned offset)
+// SPDX-License-Identifier: GPL-2.0
/*
* Digital I/O driver for Technologic Systems TS-5500
*
* TS-5600:
* Documentation: http://wiki.embeddedarm.com/wiki/TS-5600
* Blocks: LCD port (identical to TS-5500 LCD).
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
*/
#include <linux/bitops.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/io.h>
#include <linux/module.h>
-#include <linux/platform_data/gpio-ts5500.h>
#include <linux/platform_device.h>
#include <linux/slab.h>
static int ts5500_dio_probe(struct platform_device *pdev)
{
enum ts5500_blocks block = platform_get_device_id(pdev)->driver_data;
- struct ts5500_dio_platform_data *pdata = dev_get_platdata(&pdev->dev);
struct device *dev = &pdev->dev;
const char *name = dev_name(dev);
struct ts5500_priv *priv;
priv->gpio_chip.set = ts5500_gpio_set;
priv->gpio_chip.to_irq = ts5500_gpio_to_irq;
priv->gpio_chip.base = -1;
- if (pdata) {
- priv->gpio_chip.base = pdata->base;
- priv->strap = pdata->strap;
- }
switch (block) {
case TS5500_DIO1:
+// SPDX-License-Identifier: GPL-2.0+
/*
* Access to GPIOs on TWL4030/TPS659x0 chips
*
*
* Initial Code:
* Andy Lowe / Nishanth Menon
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/module.h>
#include <linux/interrupt.h>
#include <linux/kthread.h>
#include <linux/irq.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/platform_device.h>
#include <linux/of.h>
#include <linux/irqdomain.h>
return ret;
}
+static int twl4030_get_gpio_direction(int gpio)
+{
+ u8 d_bnk = gpio >> 3;
+ u8 d_msk = BIT(gpio & 0x7);
+ u8 base = REG_GPIODATADIR1 + d_bnk;
+ int ret = 0;
+
+ ret = gpio_twl4030_read(base);
+ if (ret < 0)
+ return ret;
+
+ /* 1 = output, but gpiolib semantics are inverse so invert */
+ ret = !(ret & d_msk);
+
+ return ret;
+}
+
static int twl4030_set_gpio_dataout(int gpio, int enable)
{
u8 d_bnk = gpio >> 3;
return ret;
}
+static int twl_get_direction(struct gpio_chip *chip, unsigned offset)
+{
+ struct gpio_twl4030_priv *priv = gpiochip_get_data(chip);
+ /*
+ * Default 0 = output
+ * LED GPIOs >= TWL4030_GPIO_MAX are always output
+ */
+ int ret = 0;
+
+ mutex_lock(&priv->mutex);
+ if (offset < TWL4030_GPIO_MAX) {
+ ret = twl4030_get_gpio_direction(offset);
+ if (ret) {
+ mutex_unlock(&priv->mutex);
+ return ret;
+ }
+ }
+ mutex_unlock(&priv->mutex);
+
+ return ret;
+}
+
static int twl_to_irq(struct gpio_chip *chip, unsigned offset)
{
struct gpio_twl4030_priv *priv = gpiochip_get_data(chip);
.request = twl_request,
.free = twl_free,
.direction_input = twl_direction_in,
- .get = twl_get,
.direction_output = twl_direction_out,
+ .get_direction = twl_get_direction,
+ .get = twl_get,
.set = twl_set,
.to_irq = twl_to_irq,
.can_sleep = true,
+// SPDX-License-Identifier: GPL-2.0+
/*
* Access to GPOs on TWL6040 chip
*
* Authors:
* Sergio Aguirre <saaguirre@ti.com>
* Peter Ujfalusi <peter.ujfalusi@ti.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/module.h>
#include <linux/init.h>
#include <linux/kthread.h>
#include <linux/irq.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/platform_device.h>
+#include <linux/bitops.h>
#include <linux/of.h>
#include <linux/mfd/twl6040.h>
if (ret < 0)
return ret;
- return (ret >> offset) & 1;
+ return !!(ret & BIT(offset));
+}
+
+static int twl6040gpo_get_direction(struct gpio_chip *chip, unsigned offset)
+{
+ /* This means "out" */
+ return 0;
}
static int twl6040gpo_direction_out(struct gpio_chip *chip, unsigned offset,
return;
if (value)
- gpoctl = ret | (1 << offset);
+ gpoctl = ret | BIT(offset);
else
- gpoctl = ret & ~(1 << offset);
+ gpoctl = ret & ~BIT(offset);
twl6040_reg_write(twl6040, TWL6040_REG_GPOCTL, gpoctl);
}
.owner = THIS_MODULE,
.get = twl6040gpo_get,
.direction_output = twl6040gpo_direction_out,
+ .get_direction = twl6040gpo_get_direction,
.set = twl6040gpo_set,
.can_sleep = true,
};
* GNU General Public License for more details.
*/
-#include <linux/bitops.h>
+#include <linux/bits.h>
#include <linux/gpio/driver.h>
#include <linux/irq.h>
#include <linux/irqdomain.h>
+// SPDX-License-Identifier: GPL-2.0+
/*
* Freescale vf610 GPIO support through PORT and GPIO
*
* Copyright (c) 2014 Toradex AG.
*
* Author: Stefan Agner <stefan@agner.ch>.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version 2
- * of the License, or (at your option) any later version.
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
-
#include <linux/bitops.h>
#include <linux/err.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/io.h>
+// SPDX-License-Identifier: GPL-2.0+
/*
* Nano River Technologies viperboard GPIO lib driver
*
* (C) 2012 by Lemonage GmbH
* Author: Lars Poeschel <poeschel@lemonage.de>
* All rights reserved.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- *
*/
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/mutex.h>
#include <linux/platform_device.h>
-
#include <linux/usb.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/mfd/viperboard.h>
+// SPDX-License-Identifier: GPL-2.0+
/*
* Driver for NEC VR4100 series General-purpose I/O Unit.
*
* Copyright (C) 2002 MontaVista Software Inc.
* Author: Yoichi Yuasa <source@mvista.com>
* Copyright (C) 2003-2009 Yoichi Yuasa <yuasa@linux-mips.org>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/errno.h>
#include <linux/fs.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/io.h>
return 0;
}
-int vr41xx_gpio_pullupdown(unsigned int pin, gpio_pull_t pull)
-{
- u16 reg, mask;
- unsigned long flags;
-
- if ((giu_flags & GPIO_HAS_PULLUPDOWN_IO) != GPIO_HAS_PULLUPDOWN_IO)
- return -EPERM;
-
- if (pin >= 15)
- return -EINVAL;
-
- mask = 1 << pin;
-
- spin_lock_irqsave(&giu_lock, flags);
-
- if (pull == GPIO_PULL_UP || pull == GPIO_PULL_DOWN) {
- reg = giu_read(GIUTERMUPDN);
- if (pull == GPIO_PULL_UP)
- reg |= mask;
- else
- reg &= ~mask;
- giu_write(GIUTERMUPDN, reg);
-
- reg = giu_read(GIUUSEUPDN);
- reg |= mask;
- giu_write(GIUUSEUPDN, reg);
- } else {
- reg = giu_read(GIUUSEUPDN);
- reg &= ~mask;
- giu_write(GIUUSEUPDN, reg);
- }
-
- spin_unlock_irqrestore(&giu_lock, flags);
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(vr41xx_gpio_pullupdown);
-
static int vr41xx_gpio_get(struct gpio_chip *chip, unsigned pin)
{
u16 reg, mask;
+// SPDX-License-Identifier: GPL-2.0+
/*
* Linux GPIOlib driver for the VIA VX855 integrated southbridge GPIO
*
* Copyright (C) 2010 One Laptop per Child
* Author: Harald Welte <HaraldWelte@viatech.com>
* All rights reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of
- * the License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston,
- * MA 02111-1307 USA
- *
*/
-
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/slab.h>
#include <linux/device.h>
#include <linux/platform_device.h>
+// SPDX-License-Identifier: GPL-2.0+
/*
* gpiolib support for Wolfson WM831x PMICs
*
*
* Author: Mark Brown <broonie@opensource.wolfsonmicro.com>
*
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- *
*/
#include <linux/kernel.h>
#include <linux/slab.h>
#include <linux/module.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/mfd/core.h>
#include <linux/platform_device.h>
#include <linux/seq_file.h>
+// SPDX-License-Identifier: GPL-2.0+
/*
* gpiolib support for Wolfson WM835x PMICs
*
*
* Author: Mark Brown <broonie@opensource.wolfsonmicro.com>
*
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- *
*/
#include <linux/kernel.h>
#include <linux/slab.h>
#include <linux/module.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/mfd/core.h>
#include <linux/platform_device.h>
#include <linux/seq_file.h>
+// SPDX-License-Identifier: GPL-2.0+
/*
* gpiolib support for Wolfson WM8994
*
*
* Author: Mark Brown <broonie@opensource.wolfsonmicro.com>
*
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- *
*/
#include <linux/kernel.h>
#include <linux/slab.h>
#include <linux/module.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/mfd/core.h>
#include <linux/platform_device.h>
#include <linux/seq_file.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2003-2015 Broadcom Corporation
* All Rights Reserved
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/platform_device.h>
#include <linux/of_device.h>
#include <linux/module.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2013 TangoTec Ltd.
* Author: Baruch Siach <baruch@tkos.co.il>
*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
* Driver for the Xtensa LX4 GPIO32 Option
*
* Documentation: Xtensa LX4 Microprocessor Data Book, Section 2.22
#include <linux/err.h>
#include <linux/module.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/bitops.h>
#include <linux/platform_device.h>
#include <linux/of_device.h>
#include <linux/of_gpio.h>
#include <linux/slab.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
/*
* Memory layout:
+// SPDX-License-Identifier: GPL-2.0
/*
* ACPI helpers for GPIO API
*
* Copyright (C) 2012, Intel Corporation
* Authors: Mathias Nyman <mathias.nyman@linux.intel.com>
* Mika Westerberg <mika.westerberg@linux.intel.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
*/
#include <linux/errno.h>
-#include <linux/gpio.h>
#include <linux/gpio/consumer.h>
#include <linux/gpio/driver.h>
#include <linux/gpio/machine.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* Device property helpers for GPIO chips.
*
* Copyright (C) 2016, Intel Corporation
* Author: Mika Westerberg <mika.westerberg@linux.intel.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
*/
#include <linux/property.h>
struct gpio_device *gdev = chip->gpiodev;
const char **names;
int ret, i;
+ int count;
- ret = fwnode_property_read_string_array(fwnode, "gpio-line-names",
- NULL, 0);
- if (ret < 0)
+ count = fwnode_property_read_string_array(fwnode, "gpio-line-names",
+ NULL, 0);
+ if (count < 0)
return;
- if (ret != gdev->ngpio) {
- dev_warn(&gdev->dev,
- "names %d do not match number of GPIOs %d\n", ret,
- gdev->ngpio);
- return;
- }
+ if (count > gdev->ngpio)
+ count = gdev->ngpio;
- names = kcalloc(gdev->ngpio, sizeof(*names), GFP_KERNEL);
+ names = kcalloc(count, sizeof(*names), GFP_KERNEL);
if (!names)
return;
ret = fwnode_property_read_string_array(fwnode, "gpio-line-names",
- names, gdev->ngpio);
+ names, count);
if (ret < 0) {
dev_warn(&gdev->dev, "failed to read GPIO line names\n");
kfree(names);
return;
}
- for (i = 0; i < gdev->ngpio; i++)
+ for (i = 0; i < count; i++)
gdev->descs[i].name = names[i];
kfree(names);
+/* SPDX-License-Identifier: GPL-2.0 */
/*
- * drivers/gpio/devres.c - managed gpio resources
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2
- * as published by the Free Software Foundation.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
+ * devres.c - managed gpio resources
* This file is based on kernel/irq/devres.c
*
* Copyright (c) 2011 John Crispin <john@phrozen.org>
+// SPDX-License-Identifier: GPL-2.0
#include <linux/gpio/consumer.h>
#include <linux/gpio/driver.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* OF helpers for the GPIO API
*
* Copyright (c) 2007-2008 MontaVista Software, Inc.
*
* Author: Anton Vorontsov <avorontsov@ru.mvista.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
*/
#include <linux/device.h>
}
static void of_gpio_flags_quirks(struct device_node *np,
- enum of_gpio_flags *flags)
+ enum of_gpio_flags *flags,
+ int index)
{
/*
* Some GPIO fixed regulator quirks.
pr_info("%s uses legacy open drain flag - update the DTS if you can\n",
of_node_full_name(np));
}
+
+ /*
+ * Legacy handling of SPI active high chip select. If we have a
+ * property named "cs-gpios" we need to inspect the child node
+ * to determine if the flags should have inverted semantics.
+ */
+ if (IS_ENABLED(CONFIG_SPI_MASTER) &&
+ of_property_read_bool(np, "cs-gpios")) {
+ struct device_node *child;
+ u32 cs;
+ int ret;
+
+ for_each_child_of_node(np, child) {
+ ret = of_property_read_u32(child, "reg", &cs);
+ if (!ret)
+ continue;
+ if (cs == index) {
+ /*
+ * SPI children have active low chip selects
+ * by default. This can be specified negatively
+ * by just omitting "spi-cs-high" in the
+ * device node, or actively by tagging on
+ * GPIO_ACTIVE_LOW as flag in the device
+ * tree. If the line is simultaneously
+ * tagged as active low in the device tree
+ * and has the "spi-cs-high" set, we get a
+ * conflict and the "spi-cs-high" flag will
+ * take precedence.
+ */
+ if (of_property_read_bool(np, "spi-cs-high")) {
+ if (*flags & OF_GPIO_ACTIVE_LOW) {
+ pr_warn("%s GPIO handle specifies active low - ignored\n",
+ of_node_full_name(np));
+ *flags &= ~OF_GPIO_ACTIVE_LOW;
+ }
+ } else {
+ if (!(*flags & OF_GPIO_ACTIVE_LOW))
+ pr_info("%s enforce active low on chipselect handle\n",
+ of_node_full_name(np));
+ *flags |= OF_GPIO_ACTIVE_LOW;
+ }
+ break;
+ }
+ }
+ }
}
/**
goto out;
if (flags)
- of_gpio_flags_quirks(np, flags);
+ of_gpio_flags_quirks(np, flags, index);
pr_debug("%s: parsed '%s' property of node '%pOF[%d]' - status (%d)\n",
__func__, propname, np, index,
else if (of_property_read_bool(np, "output-high"))
*dflags |= GPIOD_OUT_HIGH;
else {
- pr_warn("GPIO line %d (%s): no hogging state specified, bailing out\n",
- desc_to_gpio(desc), np->name);
+ pr_warn("GPIO line %d (%pOFn): no hogging state specified, bailing out\n",
+ desc_to_gpio(desc), np);
return ERR_PTR(-EINVAL);
}
+// SPDX-License-Identifier: GPL-2.0
#include <linux/idr.h>
#include <linux/mutex.h>
#include <linux/device.h>
#include <linux/sysfs.h>
-#include <linux/gpio.h>
#include <linux/gpio/consumer.h>
#include <linux/gpio/driver.h>
#include <linux/interrupt.h>
};
ATTRIBUTE_GROUPS(gpiochip);
-static struct gpio_desc *gpio_to_valid_desc(int gpio)
-{
- return gpio_is_valid(gpio) ? gpio_to_desc(gpio) : NULL;
-}
-
/*
* /sys/class/gpio/export ... write-only
* integer N ... number of GPIO to export (full access)
if (status < 0)
goto done;
- desc = gpio_to_valid_desc(gpio);
+ desc = gpio_to_desc(gpio);
/* reject invalid GPIOs */
if (!desc) {
pr_warn("%s: invalid GPIO %ld\n", __func__, gpio);
if (status < 0)
goto done;
- desc = gpio_to_valid_desc(gpio);
+ desc = gpio_to_desc(gpio);
/* reject bogus commands (gpio_unexport ignores them) */
if (!desc) {
pr_warn("%s: invalid GPIO %ld\n", __func__, gpio);
+// SPDX-License-Identifier: GPL-2.0
#include <linux/bitmap.h>
#include <linux/kernel.h>
#include <linux/module.h>
*/
int gpiod_get_direction(struct gpio_desc *desc)
{
- struct gpio_chip *chip;
- unsigned offset;
- int status = -EINVAL;
+ struct gpio_chip *chip;
+ unsigned offset;
+ int status;
chip = gpiod_to_chip(desc);
offset = gpio_chip_hwgpio(desc);
if (!chip->get_direction)
- return status;
+ return -ENOTSUPP;
status = chip->get_direction(chip, offset);
if (status > 0) {
return p;
}
-static int gpiochip_init_valid_mask(struct gpio_chip *gpiochip)
+static int gpiochip_alloc_valid_mask(struct gpio_chip *gpiochip)
{
#ifdef CONFIG_OF_GPIO
int size;
return 0;
}
+static int gpiochip_init_valid_mask(struct gpio_chip *gpiochip)
+{
+ if (gpiochip->init_valid_mask)
+ return gpiochip->init_valid_mask(gpiochip);
+
+ return 0;
+}
+
static void gpiochip_free_valid_mask(struct gpio_chip *gpiochip)
{
kfree(gpiochip->valid_mask);
struct linehandle_state *lh = filep->private_data;
void __user *ip = (void __user *)arg;
struct gpiohandle_data ghd;
- int vals[GPIOHANDLES_MAX];
+ DECLARE_BITMAP(vals, GPIOHANDLES_MAX);
int i;
if (cmd == GPIOHANDLE_GET_LINE_VALUES_IOCTL) {
true,
lh->numdescs,
lh->descs,
+ NULL,
vals);
if (ret)
return ret;
memset(&ghd, 0, sizeof(ghd));
for (i = 0; i < lh->numdescs; i++)
- ghd.values[i] = vals[i];
+ ghd.values[i] = test_bit(i, vals);
if (copy_to_user(ip, &ghd, sizeof(ghd)))
return -EFAULT;
/* Clamp all values to [0,1] */
for (i = 0; i < lh->numdescs; i++)
- vals[i] = !!ghd.values[i];
+ __assign_bit(i, vals, ghd.values[i]);
/* Reuse the array setting function */
return gpiod_set_array_value_complex(false,
true,
lh->numdescs,
lh->descs,
+ NULL,
vals);
}
return -EINVAL;
{
struct lineevent_state *le = p;
struct gpioevent_data ge;
- int ret, level;
+ int ret;
/* Do not leak kernel stack to userspace */
memset(&ge, 0, sizeof(ge));
ge.timestamp = le->timestamp;
- level = gpiod_get_value_cansleep(le->desc);
if (le->eflags & GPIOEVENT_REQUEST_RISING_EDGE
&& le->eflags & GPIOEVENT_REQUEST_FALLING_EDGE) {
+ int level = gpiod_get_value_cansleep(le->desc);
if (level)
/* Emit low-to-high event */
ge.id = GPIOEVENT_EVENT_RISING_EDGE;
else
/* Emit high-to-low event */
ge.id = GPIOEVENT_EVENT_FALLING_EDGE;
- } else if (le->eflags & GPIOEVENT_REQUEST_RISING_EDGE && level) {
+ } else if (le->eflags & GPIOEVENT_REQUEST_RISING_EDGE) {
/* Emit low-to-high event */
ge.id = GPIOEVENT_EVENT_RISING_EDGE;
- } else if (le->eflags & GPIOEVENT_REQUEST_FALLING_EDGE && !level) {
+ } else if (le->eflags & GPIOEVENT_REQUEST_FALLING_EDGE) {
/* Emit high-to-low event */
ge.id = GPIOEVENT_EVENT_FALLING_EDGE;
} else {
if (eflags & GPIOEVENT_REQUEST_FALLING_EDGE)
irqflags |= IRQF_TRIGGER_FALLING;
irqflags |= IRQF_ONESHOT;
- irqflags |= IRQF_SHARED;
INIT_KFIFO(le->events);
init_waitqueue_head(&le->wait);
spin_unlock_irqrestore(&gpio_lock, flags);
- for (i = 0; i < chip->ngpio; i++) {
- struct gpio_desc *desc = &gdev->descs[i];
-
- desc->gdev = gdev;
-
- /* REVISIT: most hardware initializes GPIOs as inputs (often
- * with pullups enabled) so power usage is minimized. Linux
- * code should set the gpio direction first thing; but until
- * it does, and in case chip->get_direction is not set, we may
- * expose the wrong direction in sysfs.
- */
- desc->flags = !chip->direction_input ? (1 << FLAG_IS_OUT) : 0;
- }
+ for (i = 0; i < chip->ngpio; i++)
+ gdev->descs[i].gdev = gdev;
#ifdef CONFIG_PINCTRL
INIT_LIST_HEAD(&gdev->pin_ranges);
if (status)
goto err_remove_from_list;
- status = gpiochip_init_valid_mask(chip);
+ status = gpiochip_alloc_valid_mask(chip);
if (status)
goto err_remove_irqchip_mask;
if (status)
goto err_remove_chip;
+ status = gpiochip_init_valid_mask(chip);
+ if (status)
+ goto err_remove_chip;
+
+ for (i = 0; i < chip->ngpio; i++) {
+ struct gpio_desc *desc = &gdev->descs[i];
+
+ if (chip->get_direction && gpiochip_line_is_valid(chip, i))
+ desc->flags = !chip->get_direction(chip, i) ?
+ (1 << FLAG_IS_OUT) : 0;
+ else
+ desc->flags = !chip->direction_input ?
+ (1 << FLAG_IS_OUT) : 0;
+ }
+
acpi_gpiochip_add(chip);
machine_gpiochip_add(chip);
/**
* devm_gpiochip_add_data() - Resource manager gpiochip_add_data()
- * @dev: the device pointer on which irq_chip belongs to.
+ * @dev: pointer to the device that gpio_chip belongs to.
* @chip: the chip to register, with chip->base initialized
* @data: driver-private data associated with this chip
*
/**
* gpiochip_set_cascaded_irqchip() - connects a cascaded irqchip to a gpiochip
* @gpiochip: the gpiochip to set the irqchip chain to
- * @irqchip: the irqchip to chain to the gpiochip
* @parent_irq: the irq number corresponding to the parent IRQ for this
* chained irqchip
* @parent_handler: the parent interrupt handler for the accumulated IRQ
* cascaded, pass NULL in this handler argument
*/
static void gpiochip_set_cascaded_irqchip(struct gpio_chip *gpiochip,
- struct irq_chip *irqchip,
unsigned int parent_irq,
irq_flow_handler_t parent_handler)
{
- unsigned int offset;
-
if (!gpiochip->irq.domain) {
chip_err(gpiochip, "called %s before setting up irqchip\n",
__func__);
irq_set_chained_handler_and_data(parent_irq, parent_handler,
gpiochip);
- gpiochip->irq.parents = &parent_irq;
+ gpiochip->irq.parent_irq = parent_irq;
+ gpiochip->irq.parents = &gpiochip->irq.parent_irq;
gpiochip->irq.num_parents = 1;
}
-
- /* Set the parent IRQ for all affected IRQs */
- for (offset = 0; offset < gpiochip->ngpio; offset++) {
- if (!gpiochip_irqchip_irq_valid(gpiochip, offset))
- continue;
- irq_set_parent(irq_find_mapping(gpiochip->irq.domain, offset),
- parent_irq);
- }
}
/**
* @parent_irq: the irq number corresponding to the parent IRQ for this
* chained irqchip
* @parent_handler: the parent interrupt handler for the accumulated IRQ
- * coming out of the gpiochip. If the interrupt is nested rather than
- * cascaded, pass NULL in this handler argument
+ * coming out of the gpiochip.
*/
void gpiochip_set_chained_irqchip(struct gpio_chip *gpiochip,
struct irq_chip *irqchip,
return;
}
- gpiochip_set_cascaded_irqchip(gpiochip, irqchip, parent_irq,
- parent_handler);
+ gpiochip_set_cascaded_irqchip(gpiochip, parent_irq, parent_handler);
}
EXPORT_SYMBOL_GPL(gpiochip_set_chained_irqchip);
struct irq_chip *irqchip,
unsigned int parent_irq)
{
- gpiochip_set_cascaded_irqchip(gpiochip, irqchip, parent_irq,
- NULL);
+ gpiochip_set_cascaded_irqchip(gpiochip, parent_irq, NULL);
}
EXPORT_SYMBOL_GPL(gpiochip_set_nested_irqchip);
.xlate = irq_domain_xlate_twocell,
};
+static int gpiochip_to_irq(struct gpio_chip *chip, unsigned offset)
+{
+ if (!gpiochip_irqchip_irq_valid(chip, offset))
+ return -ENXIO;
+
+ return irq_create_mapping(chip->irq.domain, offset);
+}
+
static int gpiochip_irq_reqres(struct irq_data *d)
{
struct gpio_chip *chip = irq_data_get_irq_chip_data(d);
- int ret;
-
- if (!try_module_get(chip->gpiodev->owner))
- return -ENODEV;
- ret = gpiochip_lock_as_irq(chip, d->hwirq);
- if (ret) {
- chip_err(chip,
- "unable to lock HW IRQ %lu for IRQ\n",
- d->hwirq);
- module_put(chip->gpiodev->owner);
- return ret;
- }
- return 0;
+ return gpiochip_reqres_irq(chip, d->hwirq);
}
static void gpiochip_irq_relres(struct irq_data *d)
{
struct gpio_chip *chip = irq_data_get_irq_chip_data(d);
- gpiochip_unlock_as_irq(chip, d->hwirq);
- module_put(chip->gpiodev->owner);
+ gpiochip_relres_irq(chip, d->hwirq);
}
-static int gpiochip_to_irq(struct gpio_chip *chip, unsigned offset)
+static void gpiochip_irq_enable(struct irq_data *d)
{
- if (!gpiochip_irqchip_irq_valid(chip, offset))
- return -ENXIO;
+ struct gpio_chip *chip = irq_data_get_irq_chip_data(d);
- return irq_create_mapping(chip->irq.domain, offset);
+ gpiochip_enable_irq(chip, d->hwirq);
+ if (chip->irq.irq_enable)
+ chip->irq.irq_enable(d);
+ else
+ chip->irq.chip->irq_unmask(d);
+}
+
+static void gpiochip_irq_disable(struct irq_data *d)
+{
+ struct gpio_chip *chip = irq_data_get_irq_chip_data(d);
+
+ if (chip->irq.irq_disable)
+ chip->irq.irq_disable(d);
+ else
+ chip->irq.chip->irq_mask(d);
+ gpiochip_disable_irq(chip, d->hwirq);
+}
+
+static void gpiochip_set_irq_hooks(struct gpio_chip *gpiochip)
+{
+ struct irq_chip *irqchip = gpiochip->irq.chip;
+
+ if (!irqchip->irq_request_resources &&
+ !irqchip->irq_release_resources) {
+ irqchip->irq_request_resources = gpiochip_irq_reqres;
+ irqchip->irq_release_resources = gpiochip_irq_relres;
+ }
+ if (WARN_ON(gpiochip->irq.irq_enable))
+ return;
+ /* Check if the irqchip already has this hook... */
+ if (irqchip->irq_enable == gpiochip_irq_enable) {
+ /*
+ * ...and if so, give a gentle warning that this is bad
+ * practice.
+ */
+ chip_info(gpiochip,
+ "detected irqchip that is shared with multiple gpiochips: please fix the driver.\n");
+ return;
+ }
+ gpiochip->irq.irq_enable = irqchip->irq_enable;
+ gpiochip->irq.irq_disable = irqchip->irq_disable;
+ irqchip->irq_enable = gpiochip_irq_enable;
+ irqchip->irq_disable = gpiochip_irq_disable;
}
/**
if (!gpiochip->irq.domain)
return -EINVAL;
- /*
- * It is possible for a driver to override this, but only if the
- * alternative functions are both implemented.
- */
- if (!irqchip->irq_request_resources &&
- !irqchip->irq_release_resources) {
- irqchip->irq_request_resources = gpiochip_irq_reqres;
- irqchip->irq_release_resources = gpiochip_irq_relres;
- }
-
if (gpiochip->irq.parent_handler) {
void *data = gpiochip->irq.parent_handler_data ?: gpiochip;
}
}
+ gpiochip_set_irq_hooks(gpiochip);
+
acpi_gpiochip_request_interrupts(gpiochip);
return 0;
*/
static void gpiochip_irqchip_remove(struct gpio_chip *gpiochip)
{
+ struct irq_chip *irqchip = gpiochip->irq.chip;
unsigned int offset;
acpi_gpiochip_free_interrupts(gpiochip);
- if (gpiochip->irq.chip && gpiochip->irq.parent_handler) {
+ if (irqchip && gpiochip->irq.parent_handler) {
struct gpio_irq_chip *irq = &gpiochip->irq;
unsigned int i;
irq_domain_remove(gpiochip->irq.domain);
}
- if (gpiochip->irq.chip) {
- gpiochip->irq.chip->irq_request_resources = NULL;
- gpiochip->irq.chip->irq_release_resources = NULL;
- gpiochip->irq.chip = NULL;
+ if (irqchip) {
+ if (irqchip->irq_request_resources == gpiochip_irq_reqres) {
+ irqchip->irq_request_resources = NULL;
+ irqchip->irq_release_resources = NULL;
+ }
+ if (irqchip->irq_enable == gpiochip_irq_enable) {
+ irqchip->irq_enable = gpiochip->irq.irq_enable;
+ irqchip->irq_disable = gpiochip->irq.irq_disable;
+ }
}
+ gpiochip->irq.irq_enable = NULL;
+ gpiochip->irq.irq_disable = NULL;
+ gpiochip->irq.chip = NULL;
gpiochip_irqchip_free_valid_mask(gpiochip);
}
return -EINVAL;
}
- /*
- * It is possible for a driver to override this, but only if the
- * alternative functions are both implemented.
- */
- if (!irqchip->irq_request_resources &&
- !irqchip->irq_release_resources) {
- irqchip->irq_request_resources = gpiochip_irq_reqres;
- irqchip->irq_release_resources = gpiochip_irq_relres;
- }
+ gpiochip_set_irq_hooks(gpiochip);
acpi_gpiochip_request_interrupts(gpiochip);
int gpiod_direction_input(struct gpio_desc *desc)
{
struct gpio_chip *chip;
- int status = -EINVAL;
+ int status = 0;
VALIDATE_DESC(desc);
chip = desc->gdev->chip;
- if (!chip->get || !chip->direction_input) {
+ /*
+ * It is legal to have no .get() and .direction_input() specified if
+ * the chip is output-only, but you can't specify .direction_input()
+ * and not support the .get() operation, that doesn't make sense.
+ */
+ if (!chip->get && chip->direction_input) {
gpiod_warn(desc,
- "%s: missing get() or direction_input() operations\n",
- __func__);
+ "%s: missing get() but have direction_input()\n",
+ __func__);
return -EIO;
}
- status = chip->direction_input(chip, gpio_chip_hwgpio(desc));
+ /*
+ * If we have a .direction_input() callback, things are simple,
+ * just call it. Else we are some input-only chip so try to check the
+ * direction (if .get_direction() is supported) else we silently
+ * assume we are in input mode after this.
+ */
+ if (chip->direction_input) {
+ status = chip->direction_input(chip, gpio_chip_hwgpio(desc));
+ } else if (chip->get_direction &&
+ (chip->get_direction(chip, gpio_chip_hwgpio(desc)) != 1)) {
+ gpiod_warn(desc,
+ "%s: missing direction_input() operation and line is output\n",
+ __func__);
+ return -EIO;
+ }
if (status == 0)
clear_bit(FLAG_IS_OUT, &desc->flags);
{
struct gpio_chip *gc = desc->gdev->chip;
int val = !!value;
- int ret;
+ int ret = 0;
- if (!gc->set || !gc->direction_output) {
+ /*
+ * It's OK not to specify .direction_output() if the gpiochip is
+ * output-only, but if there is then not even a .set() operation it
+ * is pretty tricky to drive the output line.
+ */
+ if (!gc->set && !gc->direction_output) {
gpiod_warn(desc,
- "%s: missing set() or direction_output() operations\n",
- __func__);
+ "%s: missing set() and direction_output() operations\n",
+ __func__);
return -EIO;
}
- ret = gc->direction_output(gc, gpio_chip_hwgpio(desc), val);
+ if (gc->direction_output) {
+ ret = gc->direction_output(gc, gpio_chip_hwgpio(desc), val);
+ } else {
+ /* Check that we are in output mode if we can */
+ if (gc->get_direction &&
+ gc->get_direction(gc, gpio_chip_hwgpio(desc))) {
+ gpiod_warn(desc,
+ "%s: missing direction_output() operation\n",
+ __func__);
+ return -EIO;
+ }
+ /*
+ * If we can't actively set the direction, we are some
+ * output-only chip, so just drive the output as desired.
+ */
+ gc->set(gc, gpio_chip_hwgpio(desc), val);
+ }
+
if (!ret)
set_bit(FLAG_IS_OUT, &desc->flags);
trace_gpio_value(desc_to_gpio(desc), 0, val);
else
value = !!value;
- /* GPIOs used for IRQs shall not be set as output */
- if (test_bit(FLAG_USED_AS_IRQ, &desc->flags)) {
+ /* GPIOs used for enabled IRQs shall not be set as output */
+ if (test_bit(FLAG_USED_AS_IRQ, &desc->flags) &&
+ test_bit(FLAG_IRQ_IS_ENABLED, &desc->flags)) {
gpiod_err(desc,
"%s: tried to set a GPIO tied to an IRQ as output\n",
__func__);
int gpiod_get_array_value_complex(bool raw, bool can_sleep,
unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array)
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
- int i = 0;
+ int err, i = 0;
+
+ /*
+ * Validate array_info against desc_array and its size.
+ * It should immediately follow desc_array if both
+ * have been obtained from the same gpiod_get_array() call.
+ */
+ if (array_info && array_info->desc == desc_array &&
+ array_size <= array_info->size &&
+ (void *)array_info == desc_array + array_info->size) {
+ if (!can_sleep)
+ WARN_ON(array_info->chip->can_sleep);
+
+ err = gpio_chip_get_multiple(array_info->chip,
+ array_info->get_mask,
+ value_bitmap);
+ if (err)
+ return err;
+
+ if (!raw && !bitmap_empty(array_info->invert_mask, array_size))
+ bitmap_xor(value_bitmap, value_bitmap,
+ array_info->invert_mask, array_size);
+
+ if (bitmap_full(array_info->get_mask, array_size))
+ return 0;
+
+ i = find_first_zero_bit(array_info->get_mask, array_size);
+ } else {
+ array_info = NULL;
+ }
while (i < array_size) {
struct gpio_chip *chip = desc_array[i]->gdev->chip;
__set_bit(hwgpio, mask);
i++;
+
+ if (array_info)
+ i = find_next_zero_bit(array_info->get_mask,
+ array_size, i);
} while ((i < array_size) &&
(desc_array[i]->gdev->chip == chip));
return ret;
}
- for (j = first; j < i; j++) {
+ for (j = first; j < i; ) {
const struct gpio_desc *desc = desc_array[j];
int hwgpio = gpio_chip_hwgpio(desc);
int value = test_bit(hwgpio, bits);
if (!raw && test_bit(FLAG_ACTIVE_LOW, &desc->flags))
value = !value;
- value_array[j] = value;
+ __assign_bit(j, value_bitmap, value);
trace_gpio_value(desc_to_gpio(desc), 1, value);
+ j++;
+
+ if (array_info)
+ j = find_next_zero_bit(array_info->get_mask, i,
+ j);
}
if (mask != fastpath)
/**
* gpiod_get_raw_array_value() - read raw values from an array of GPIOs
- * @array_size: number of elements in the descriptor / value arrays
+ * @array_size: number of elements in the descriptor array / value bitmap
* @desc_array: array of GPIO descriptors whose values will be read
- * @value_array: array to store the read values
+ * @array_info: information on applicability of fast bitmap processing path
+ * @value_bitmap: bitmap to store the read values
*
* Read the raw values of the GPIOs, i.e. the values of the physical lines
* without regard for their ACTIVE_LOW status. Return 0 in case of success,
* and it will complain if the GPIO chip functions potentially sleep.
*/
int gpiod_get_raw_array_value(unsigned int array_size,
- struct gpio_desc **desc_array, int *value_array)
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
if (!desc_array)
return -EINVAL;
return gpiod_get_array_value_complex(true, false, array_size,
- desc_array, value_array);
+ desc_array, array_info,
+ value_bitmap);
}
EXPORT_SYMBOL_GPL(gpiod_get_raw_array_value);
/**
* gpiod_get_array_value() - read values from an array of GPIOs
- * @array_size: number of elements in the descriptor / value arrays
+ * @array_size: number of elements in the descriptor array / value bitmap
* @desc_array: array of GPIO descriptors whose values will be read
- * @value_array: array to store the read values
+ * @array_info: information on applicability of fast bitmap processing path
+ * @value_bitmap: bitmap to store the read values
*
* Read the logical values of the GPIOs, i.e. taking their ACTIVE_LOW status
* into account. Return 0 in case of success, else an error code.
* and it will complain if the GPIO chip functions potentially sleep.
*/
int gpiod_get_array_value(unsigned int array_size,
- struct gpio_desc **desc_array, int *value_array)
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
if (!desc_array)
return -EINVAL;
return gpiod_get_array_value_complex(false, false, array_size,
- desc_array, value_array);
+ desc_array, array_info,
+ value_bitmap);
}
EXPORT_SYMBOL_GPL(gpiod_get_array_value);
}
int gpiod_set_array_value_complex(bool raw, bool can_sleep,
- unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array)
+ unsigned int array_size,
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
int i = 0;
+ /*
+ * Validate array_info against desc_array and its size.
+ * It should immediately follow desc_array if both
+ * have been obtained from the same gpiod_get_array() call.
+ */
+ if (array_info && array_info->desc == desc_array &&
+ array_size <= array_info->size &&
+ (void *)array_info == desc_array + array_info->size) {
+ if (!can_sleep)
+ WARN_ON(array_info->chip->can_sleep);
+
+ if (!raw && !bitmap_empty(array_info->invert_mask, array_size))
+ bitmap_xor(value_bitmap, value_bitmap,
+ array_info->invert_mask, array_size);
+
+ gpio_chip_set_multiple(array_info->chip, array_info->set_mask,
+ value_bitmap);
+
+ if (bitmap_full(array_info->set_mask, array_size))
+ return 0;
+
+ i = find_first_zero_bit(array_info->set_mask, array_size);
+ } else {
+ array_info = NULL;
+ }
+
while (i < array_size) {
struct gpio_chip *chip = desc_array[i]->gdev->chip;
unsigned long fastpath[2 * BITS_TO_LONGS(FASTPATH_NGPIO)];
do {
struct gpio_desc *desc = desc_array[i];
int hwgpio = gpio_chip_hwgpio(desc);
- int value = value_array[i];
+ int value = test_bit(i, value_bitmap);
- if (!raw && test_bit(FLAG_ACTIVE_LOW, &desc->flags))
+ /*
+ * Pins applicable for fast input but not for
+ * fast output processing may have been already
+ * inverted inside the fast path, skip them.
+ */
+ if (!raw && !(array_info &&
+ test_bit(i, array_info->invert_mask)) &&
+ test_bit(FLAG_ACTIVE_LOW, &desc->flags))
value = !value;
trace_gpio_value(desc_to_gpio(desc), 0, value);
/*
count++;
}
i++;
+
+ if (array_info)
+ i = find_next_zero_bit(array_info->set_mask,
+ array_size, i);
} while ((i < array_size) &&
(desc_array[i]->gdev->chip == chip));
/* push collected bits to outputs */
/**
* gpiod_set_raw_array_value() - assign values to an array of GPIOs
- * @array_size: number of elements in the descriptor / value arrays
+ * @array_size: number of elements in the descriptor array / value bitmap
* @desc_array: array of GPIO descriptors whose values will be assigned
- * @value_array: array of values to assign
+ * @array_info: information on applicability of fast bitmap processing path
+ * @value_bitmap: bitmap of values to assign
*
* Set the raw values of the GPIOs, i.e. the values of the physical lines
* without regard for their ACTIVE_LOW status.
* complain if the GPIO chip functions potentially sleep.
*/
int gpiod_set_raw_array_value(unsigned int array_size,
- struct gpio_desc **desc_array, int *value_array)
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
if (!desc_array)
return -EINVAL;
return gpiod_set_array_value_complex(true, false, array_size,
- desc_array, value_array);
+ desc_array, array_info, value_bitmap);
}
EXPORT_SYMBOL_GPL(gpiod_set_raw_array_value);
/**
* gpiod_set_array_value() - assign values to an array of GPIOs
- * @array_size: number of elements in the descriptor / value arrays
+ * @array_size: number of elements in the descriptor array / value bitmap
* @desc_array: array of GPIO descriptors whose values will be assigned
- * @value_array: array of values to assign
+ * @array_info: information on applicability of fast bitmap processing path
+ * @value_bitmap: bitmap of values to assign
*
* Set the logical values of the GPIOs, i.e. taking their ACTIVE_LOW status
* into account.
* This function should be called from contexts where we cannot sleep, and will
* complain if the GPIO chip functions potentially sleep.
*/
-void gpiod_set_array_value(unsigned int array_size,
- struct gpio_desc **desc_array, int *value_array)
+int gpiod_set_array_value(unsigned int array_size,
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
if (!desc_array)
- return;
- gpiod_set_array_value_complex(false, false, array_size, desc_array,
- value_array);
+ return -EINVAL;
+ return gpiod_set_array_value_complex(false, false, array_size,
+ desc_array, array_info,
+ value_bitmap);
}
EXPORT_SYMBOL_GPL(gpiod_set_array_value);
}
set_bit(FLAG_USED_AS_IRQ, &desc->flags);
+ set_bit(FLAG_IRQ_IS_ENABLED, &desc->flags);
/*
* If the consumer has not set up a label (such as when the
return;
clear_bit(FLAG_USED_AS_IRQ, &desc->flags);
+ clear_bit(FLAG_IRQ_IS_ENABLED, &desc->flags);
/* If we only had this marking, erase it */
if (desc->label && !strcmp(desc->label, "interrupt"))
}
EXPORT_SYMBOL_GPL(gpiochip_unlock_as_irq);
+void gpiochip_disable_irq(struct gpio_chip *chip, unsigned int offset)
+{
+ struct gpio_desc *desc = gpiochip_get_desc(chip, offset);
+
+ if (!IS_ERR(desc) &&
+ !WARN_ON(!test_bit(FLAG_USED_AS_IRQ, &desc->flags)))
+ clear_bit(FLAG_IRQ_IS_ENABLED, &desc->flags);
+}
+EXPORT_SYMBOL_GPL(gpiochip_disable_irq);
+
+void gpiochip_enable_irq(struct gpio_chip *chip, unsigned int offset)
+{
+ struct gpio_desc *desc = gpiochip_get_desc(chip, offset);
+
+ if (!IS_ERR(desc) &&
+ !WARN_ON(!test_bit(FLAG_USED_AS_IRQ, &desc->flags))) {
+ WARN_ON(test_bit(FLAG_IS_OUT, &desc->flags));
+ set_bit(FLAG_IRQ_IS_ENABLED, &desc->flags);
+ }
+}
+EXPORT_SYMBOL_GPL(gpiochip_enable_irq);
+
bool gpiochip_line_is_irq(struct gpio_chip *chip, unsigned int offset)
{
if (offset >= chip->ngpio)
}
EXPORT_SYMBOL_GPL(gpiochip_line_is_irq);
+int gpiochip_reqres_irq(struct gpio_chip *chip, unsigned int offset)
+{
+ int ret;
+
+ if (!try_module_get(chip->gpiodev->owner))
+ return -ENODEV;
+
+ ret = gpiochip_lock_as_irq(chip, offset);
+ if (ret) {
+ chip_err(chip, "unable to lock HW IRQ %u for IRQ\n", offset);
+ module_put(chip->gpiodev->owner);
+ return ret;
+ }
+ return 0;
+}
+EXPORT_SYMBOL_GPL(gpiochip_reqres_irq);
+
+void gpiochip_relres_irq(struct gpio_chip *chip, unsigned int offset)
+{
+ gpiochip_unlock_as_irq(chip, offset);
+ module_put(chip->gpiodev->owner);
+}
+EXPORT_SYMBOL_GPL(gpiochip_relres_irq);
+
bool gpiochip_line_is_open_drain(struct gpio_chip *chip, unsigned int offset)
{
if (offset >= chip->ngpio)
/**
* gpiod_get_raw_array_value_cansleep() - read raw values from an array of GPIOs
- * @array_size: number of elements in the descriptor / value arrays
+ * @array_size: number of elements in the descriptor array / value bitmap
* @desc_array: array of GPIO descriptors whose values will be read
- * @value_array: array to store the read values
+ * @array_info: information on applicability of fast bitmap processing path
+ * @value_bitmap: bitmap to store the read values
*
* Read the raw values of the GPIOs, i.e. the values of the physical lines
* without regard for their ACTIVE_LOW status. Return 0 in case of success,
*/
int gpiod_get_raw_array_value_cansleep(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array)
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
might_sleep_if(extra_checks);
if (!desc_array)
return -EINVAL;
return gpiod_get_array_value_complex(true, true, array_size,
- desc_array, value_array);
+ desc_array, array_info,
+ value_bitmap);
}
EXPORT_SYMBOL_GPL(gpiod_get_raw_array_value_cansleep);
/**
* gpiod_get_array_value_cansleep() - read values from an array of GPIOs
- * @array_size: number of elements in the descriptor / value arrays
+ * @array_size: number of elements in the descriptor array / value bitmap
* @desc_array: array of GPIO descriptors whose values will be read
- * @value_array: array to store the read values
+ * @array_info: information on applicability of fast bitmap processing path
+ * @value_bitmap: bitmap to store the read values
*
* Read the logical values of the GPIOs, i.e. taking their ACTIVE_LOW status
* into account. Return 0 in case of success, else an error code.
*/
int gpiod_get_array_value_cansleep(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array)
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
might_sleep_if(extra_checks);
if (!desc_array)
return -EINVAL;
return gpiod_get_array_value_complex(false, true, array_size,
- desc_array, value_array);
+ desc_array, array_info,
+ value_bitmap);
}
EXPORT_SYMBOL_GPL(gpiod_get_array_value_cansleep);
/**
* gpiod_set_raw_array_value_cansleep() - assign values to an array of GPIOs
- * @array_size: number of elements in the descriptor / value arrays
+ * @array_size: number of elements in the descriptor array / value bitmap
* @desc_array: array of GPIO descriptors whose values will be assigned
- * @value_array: array of values to assign
+ * @array_info: information on applicability of fast bitmap processing path
+ * @value_bitmap: bitmap of values to assign
*
* Set the raw values of the GPIOs, i.e. the values of the physical lines
* without regard for their ACTIVE_LOW status.
* This function is to be called from contexts that can sleep.
*/
int gpiod_set_raw_array_value_cansleep(unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array)
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
might_sleep_if(extra_checks);
if (!desc_array)
return -EINVAL;
return gpiod_set_array_value_complex(true, true, array_size, desc_array,
- value_array);
+ array_info, value_bitmap);
}
EXPORT_SYMBOL_GPL(gpiod_set_raw_array_value_cansleep);
/**
* gpiod_set_array_value_cansleep() - assign values to an array of GPIOs
- * @array_size: number of elements in the descriptor / value arrays
+ * @array_size: number of elements in the descriptor array / value bitmap
* @desc_array: array of GPIO descriptors whose values will be assigned
- * @value_array: array of values to assign
+ * @array_info: information on applicability of fast bitmap processing path
+ * @value_bitmap: bitmap of values to assign
*
* Set the logical values of the GPIOs, i.e. taking their ACTIVE_LOW status
* into account.
*
* This function is to be called from contexts that can sleep.
*/
-void gpiod_set_array_value_cansleep(unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array)
+int gpiod_set_array_value_cansleep(unsigned int array_size,
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
might_sleep_if(extra_checks);
if (!desc_array)
- return;
- gpiod_set_array_value_complex(false, true, array_size, desc_array,
- value_array);
+ return -EINVAL;
+ return gpiod_set_array_value_complex(false, true, array_size,
+ desc_array, array_info,
+ value_bitmap);
}
EXPORT_SYMBOL_GPL(gpiod_set_array_value_cansleep);
* the device name as label
*/
status = gpiod_request(desc, con_id ? con_id : devname);
- if (status < 0)
- return ERR_PTR(status);
+ if (status < 0) {
+ if (status == -EBUSY && flags & GPIOD_FLAGS_BIT_NONEXCLUSIVE) {
+ /*
+ * This happens when there are several consumers for
+ * the same GPIO line: we just return here without
+ * further initialization. It is a bit if a hack.
+ * This is necessary to support fixed regulators.
+ *
+ * FIXME: Make this more sane and safe.
+ */
+ dev_info(dev, "nonexclusive access to GPIO for %s\n",
+ con_id ? con_id : devname);
+ return desc;
+ } else {
+ return ERR_PTR(status);
+ }
+ }
status = gpiod_configure_flags(desc, con_id, lookupflags, flags);
if (status < 0) {
{
struct gpio_desc *desc;
struct gpio_descs *descs;
- int count;
+ struct gpio_array *array_info = NULL;
+ struct gpio_chip *chip;
+ int count, bitmap_size;
count = gpiod_count(dev, con_id);
if (count < 0)
gpiod_put_array(descs);
return ERR_CAST(desc);
}
+
descs->desc[descs->ndescs] = desc;
+
+ chip = gpiod_to_chip(desc);
+ /*
+ * If pin hardware number of array member 0 is also 0, select
+ * its chip as a candidate for fast bitmap processing path.
+ */
+ if (descs->ndescs == 0 && gpio_chip_hwgpio(desc) == 0) {
+ struct gpio_descs *array;
+
+ bitmap_size = BITS_TO_LONGS(chip->ngpio > count ?
+ chip->ngpio : count);
+
+ array = kzalloc(struct_size(descs, desc, count) +
+ struct_size(array_info, invert_mask,
+ 3 * bitmap_size), GFP_KERNEL);
+ if (!array) {
+ gpiod_put_array(descs);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ memcpy(array, descs,
+ struct_size(descs, desc, descs->ndescs + 1));
+ kfree(descs);
+
+ descs = array;
+ array_info = (void *)(descs->desc + count);
+ array_info->get_mask = array_info->invert_mask +
+ bitmap_size;
+ array_info->set_mask = array_info->get_mask +
+ bitmap_size;
+
+ array_info->desc = descs->desc;
+ array_info->size = count;
+ array_info->chip = chip;
+ bitmap_set(array_info->get_mask, descs->ndescs,
+ count - descs->ndescs);
+ bitmap_set(array_info->set_mask, descs->ndescs,
+ count - descs->ndescs);
+ descs->info = array_info;
+ }
+ /* Unmark array members which don't belong to the 'fast' chip */
+ if (array_info && array_info->chip != chip) {
+ __clear_bit(descs->ndescs, array_info->get_mask);
+ __clear_bit(descs->ndescs, array_info->set_mask);
+ }
+ /*
+ * Detect array members which belong to the 'fast' chip
+ * but their pins are not in hardware order.
+ */
+ else if (array_info &&
+ gpio_chip_hwgpio(desc) != descs->ndescs) {
+ /*
+ * Don't use fast path if all array members processed so
+ * far belong to the same chip as this one but its pin
+ * hardware number is different from its array index.
+ */
+ if (bitmap_full(array_info->get_mask, descs->ndescs)) {
+ array_info = NULL;
+ } else {
+ __clear_bit(descs->ndescs,
+ array_info->get_mask);
+ __clear_bit(descs->ndescs,
+ array_info->set_mask);
+ }
+ } else if (array_info) {
+ /* Exclude open drain or open source from fast output */
+ if (gpiochip_line_is_open_drain(chip, descs->ndescs) ||
+ gpiochip_line_is_open_source(chip, descs->ndescs))
+ __clear_bit(descs->ndescs,
+ array_info->set_mask);
+ /* Identify 'fast' pins which require invertion */
+ if (gpiod_is_active_low(desc))
+ __set_bit(descs->ndescs,
+ array_info->invert_mask);
+ }
+
descs->ndescs++;
}
+ if (array_info)
+ dev_dbg(dev,
+ "GPIO array info: chip=%s, size=%d, get_mask=%lx, set_mask=%lx, invert_mask=%lx\n",
+ array_info->chip->label, array_info->size,
+ *array_info->get_mask, *array_info->set_mask,
+ *array_info->invert_mask);
return descs;
}
EXPORT_SYMBOL_GPL(gpiod_get_array);
struct gpio_chip *chip = gdev->chip;
unsigned gpio = gdev->base;
struct gpio_desc *gdesc = &gdev->descs[0];
- int is_out;
- int is_irq;
+ bool is_out;
+ bool is_irq;
+ bool active_low;
for (i = 0; i < gdev->ngpio; i++, gpio++, gdesc++) {
if (!test_bit(FLAG_REQUESTED, &gdesc->flags)) {
gpiod_get_direction(gdesc);
is_out = test_bit(FLAG_IS_OUT, &gdesc->flags);
is_irq = test_bit(FLAG_USED_AS_IRQ, &gdesc->flags);
- seq_printf(s, " gpio-%-3d (%-20.20s|%-20.20s) %s %s %s",
+ active_low = test_bit(FLAG_ACTIVE_LOW, &gdesc->flags);
+ seq_printf(s, " gpio-%-3d (%-20.20s|%-20.20s) %s %s %s%s",
gpio, gdesc->name ? gdesc->name : "", gdesc->label,
is_out ? "out" : "in ",
chip->get ? (chip->get(chip, i) ? "hi" : "lo") : "? ",
- is_irq ? "IRQ" : " ");
+ is_irq ? "IRQ " : "",
+ active_low ? "ACTIVE LOW" : "");
seq_printf(s, "\n");
}
}
+/* SPDX-License-Identifier: GPL-2.0 */
/*
* Internal GPIO functions.
*
* Copyright (C) 2013, Intel Corporation
* Author: Mika Westerberg <mika.westerberg@linux.intel.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
*/
#ifndef GPIOLIB_H
}
#endif
+struct gpio_array {
+ struct gpio_desc **desc;
+ unsigned int size;
+ struct gpio_chip *chip;
+ unsigned long *get_mask;
+ unsigned long *set_mask;
+ unsigned long invert_mask[];
+};
+
struct gpio_desc *gpiochip_get_desc(struct gpio_chip *chip, u16 hwnum);
int gpiod_get_array_value_complex(bool raw, bool can_sleep,
unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array);
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
int gpiod_set_array_value_complex(bool raw, bool can_sleep,
- unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array);
+ unsigned int array_size,
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
/* This is just passed between gpiolib and devres */
struct gpio_desc *gpiod_get_from_of_node(struct device_node *node,
#define FLAG_OPEN_DRAIN 7 /* Gpio is open drain type */
#define FLAG_OPEN_SOURCE 8 /* Gpio is open source type */
#define FLAG_USED_AS_IRQ 9 /* GPIO is connected to an IRQ */
+#define FLAG_IRQ_IS_ENABLED 10 /* GPIO is connected to an enabled IRQ */
#define FLAG_IS_HOGGED 11 /* GPIO is hogged */
#define FLAG_TRANSITORY 12 /* GPIO may lose value in sleep or reset */
state->crtcs[i].state = NULL;
state->crtcs[i].old_state = NULL;
state->crtcs[i].new_state = NULL;
+
+ if (state->crtcs[i].commit) {
+ drm_crtc_commit_put(state->crtcs[i].commit);
+ state->crtcs[i].commit = NULL;
+ }
}
for (i = 0; i < config->num_total_plane; i++) {
void drm_atomic_helper_wait_for_flip_done(struct drm_device *dev,
struct drm_atomic_state *old_state)
{
- struct drm_crtc_state *new_crtc_state;
struct drm_crtc *crtc;
int i;
- for_each_new_crtc_in_state(old_state, crtc, new_crtc_state, i) {
- struct drm_crtc_commit *commit = new_crtc_state->commit;
+ for (i = 0; i < dev->mode_config.num_crtc; i++) {
+ struct drm_crtc_commit *commit = old_state->crtcs[i].commit;
int ret;
- if (!commit)
+ crtc = old_state->crtcs[i].ptr;
+
+ if (!crtc || !commit)
continue;
ret = wait_for_completion_timeout(&commit->flip_done, 10 * HZ);
drm_crtc_commit_get(commit);
commit->abort_completion = true;
+
+ state->crtcs[i].commit = commit;
+ drm_crtc_commit_get(commit);
}
for_each_oldnew_connector_in_state(state, conn, old_conn_state, new_conn_state, i) {
struct drm_mode_crtc *crtc_req = data;
struct drm_crtc *crtc;
struct drm_plane *plane;
- struct drm_connector **connector_set = NULL, *connector;
- struct drm_framebuffer *fb = NULL;
- struct drm_display_mode *mode = NULL;
+ struct drm_connector **connector_set, *connector;
+ struct drm_framebuffer *fb;
+ struct drm_display_mode *mode;
struct drm_mode_set set;
uint32_t __user *set_connectors_ptr;
struct drm_modeset_acquire_ctx ctx;
mutex_lock(&crtc->dev->mode_config.mutex);
drm_modeset_acquire_init(&ctx, DRM_MODESET_ACQUIRE_INTERRUPTIBLE);
retry:
+ connector_set = NULL;
+ fb = NULL;
+ mode = NULL;
+
ret = drm_modeset_lock_all_ctx(crtc->dev, &ctx);
if (ret)
goto out;
/* AEO model 0 reports 8 bpc, but is a 6 bpc panel */
{ "AEO", 0, EDID_QUIRK_FORCE_6BPC },
+ /* BOE model on HP Pavilion 15-n233sl reports 8 bpc, but is a 6 bpc panel */
+ { "BOE", 0x78b, EDID_QUIRK_FORCE_6BPC },
+
/* CPT panel of Asus UX303LA reports 8 bpc, but is a 6 bpc panel */
{ "CPT", 0x17df, EDID_QUIRK_FORCE_6BPC },
struct drm_hdmi_info *hdmi = &connector->display_info.hdmi;
dc_mask = db[7] & DRM_EDID_YCBCR420_DC_MASK;
- hdmi->y420_dc_modes |= dc_mask;
+ hdmi->y420_dc_modes = dc_mask;
}
static void drm_parse_hdmi_forum_vsdb(struct drm_connector *connector,
}
EXPORT_SYMBOL(drm_fb_helper_ioctl);
+static bool drm_fb_pixel_format_equal(const struct fb_var_screeninfo *var_1,
+ const struct fb_var_screeninfo *var_2)
+{
+ return var_1->bits_per_pixel == var_2->bits_per_pixel &&
+ var_1->grayscale == var_2->grayscale &&
+ var_1->red.offset == var_2->red.offset &&
+ var_1->red.length == var_2->red.length &&
+ var_1->red.msb_right == var_2->red.msb_right &&
+ var_1->green.offset == var_2->green.offset &&
+ var_1->green.length == var_2->green.length &&
+ var_1->green.msb_right == var_2->green.msb_right &&
+ var_1->blue.offset == var_2->blue.offset &&
+ var_1->blue.length == var_2->blue.length &&
+ var_1->blue.msb_right == var_2->blue.msb_right &&
+ var_1->transp.offset == var_2->transp.offset &&
+ var_1->transp.length == var_2->transp.length &&
+ var_1->transp.msb_right == var_2->transp.msb_right;
+}
+
/**
* drm_fb_helper_check_var - implementation for &fb_ops.fb_check_var
* @var: screeninfo to check
{
struct drm_fb_helper *fb_helper = info->par;
struct drm_framebuffer *fb = fb_helper->fb;
- int depth;
if (var->pixclock != 0 || in_dbg_master())
return -EINVAL;
return -EINVAL;
}
- switch (var->bits_per_pixel) {
- case 16:
- depth = (var->green.length == 6) ? 16 : 15;
- break;
- case 32:
- depth = (var->transp.length > 0) ? 32 : 24;
- break;
- default:
- depth = var->bits_per_pixel;
- break;
- }
-
- switch (depth) {
- case 8:
- var->red.offset = 0;
- var->green.offset = 0;
- var->blue.offset = 0;
- var->red.length = 8;
- var->green.length = 8;
- var->blue.length = 8;
- var->transp.length = 0;
- var->transp.offset = 0;
- break;
- case 15:
- var->red.offset = 10;
- var->green.offset = 5;
- var->blue.offset = 0;
- var->red.length = 5;
- var->green.length = 5;
- var->blue.length = 5;
- var->transp.length = 1;
- var->transp.offset = 15;
- break;
- case 16:
- var->red.offset = 11;
- var->green.offset = 5;
- var->blue.offset = 0;
- var->red.length = 5;
- var->green.length = 6;
- var->blue.length = 5;
- var->transp.length = 0;
- var->transp.offset = 0;
- break;
- case 24:
- var->red.offset = 16;
- var->green.offset = 8;
- var->blue.offset = 0;
- var->red.length = 8;
- var->green.length = 8;
- var->blue.length = 8;
- var->transp.length = 0;
- var->transp.offset = 0;
- break;
- case 32:
- var->red.offset = 16;
- var->green.offset = 8;
- var->blue.offset = 0;
- var->red.length = 8;
- var->green.length = 8;
- var->blue.length = 8;
- var->transp.length = 8;
- var->transp.offset = 24;
- break;
- default:
+ /*
+ * drm fbdev emulation doesn't support changing the pixel format at all,
+ * so reject all pixel format changing requests.
+ */
+ if (!drm_fb_pixel_format_equal(var, &info->var)) {
+ DRM_DEBUG("fbdev emulation doesn't support changing the pixel format\n");
return -EINVAL;
}
+
return 0;
}
EXPORT_SYMBOL(drm_fb_helper_check_var);
}
mtk_crtc->layer_nr = mtk_ddp_comp_layer_nr(mtk_crtc->ddp_comp[0]);
- mtk_crtc->planes = devm_kzalloc(dev, mtk_crtc->layer_nr *
+ mtk_crtc->planes = devm_kcalloc(dev, mtk_crtc->layer_nr,
sizeof(struct drm_plane),
GFP_KERNEL);
return 0;
}
- mp->clk_config = devm_kzalloc(&pdev->dev,
- sizeof(struct dss_clk) * num_clk,
+ mp->clk_config = devm_kcalloc(&pdev->dev,
+ num_clk, sizeof(struct dss_clk),
GFP_KERNEL);
if (!mp->clk_config)
return -ENOMEM;
nv50_mstc_detect(struct drm_connector *connector, bool force)
{
struct nv50_mstc *mstc = nv50_mstc(connector);
+ enum drm_connector_status conn_status;
+ int ret;
+
if (!mstc->port)
return connector_status_disconnected;
- return drm_dp_mst_detect_port(connector, mstc->port->mgr, mstc->port);
+
+ ret = pm_runtime_get_sync(connector->dev->dev);
+ if (ret < 0 && ret != -EACCES)
+ return connector_status_disconnected;
+
+ conn_status = drm_dp_mst_detect_port(connector, mstc->port->mgr,
+ mstc->port);
+
+ pm_runtime_mark_last_busy(connector->dev->dev);
+ pm_runtime_put_autosuspend(connector->dev->dev);
+ return conn_status;
}
static void
int i;
for (i = tcon->dclk_min_div; i <= tcon->dclk_max_div; i++) {
- unsigned long ideal = rate * i;
+ u64 ideal = (u64)rate * i;
unsigned long rounded;
+ /*
+ * ideal has overflowed the max value that can be stored in an
+ * unsigned long, and every clk operation we might do on a
+ * truncated u64 value will give us incorrect results.
+ * Let's just stop there since bigger dividers will result in
+ * the same overflow issue.
+ */
+ if (ideal > ULONG_MAX)
+ goto out;
+
rounded = clk_hw_round_rate(clk_hw_get_parent(hw),
ideal);
dev_err(dev, "Property 'cooling-levels' cannot be read.\n");
return ret;
}
- snprintf(cdev->name, MAX_CDEV_NAME_LEN, "%s%d", child->name, pwm_port);
+ snprintf(cdev->name, MAX_CDEV_NAME_LEN, "%pOFn%d", child, pwm_port);
cdev->tcdev = thermal_of_cooling_device_register(child,
cdev->name,
data->hwmon_dev = hwmon_device_register_with_groups(dev, "atk0110",
data,
data->attr_groups);
- if (IS_ERR(data->hwmon_dev))
- return PTR_ERR(data->hwmon_dev);
- return 0;
+ return PTR_ERR_OR_ZERO(data->hwmon_dev);
}
static int atk_probe_if(struct atk_data *data)
#include <linux/string.h>
#include <linux/thermal.h>
+#define CREATE_TRACE_POINTS
+#include <trace/events/hwmon.h>
+
#define HWMON_ID_PREFIX "hwmon"
#define HWMON_ID_FORMAT HWMON_ID_PREFIX "%d"
}
#endif /* IS_REACHABLE(CONFIG_THERMAL) && ... */
+static int hwmon_attr_base(enum hwmon_sensor_types type)
+{
+ if (type == hwmon_in)
+ return 0;
+ return 1;
+}
+
/* sysfs attribute management */
static ssize_t hwmon_attr_show(struct device *dev,
if (ret < 0)
return ret;
+ trace_hwmon_attr_show(hattr->index + hwmon_attr_base(hattr->type),
+ hattr->name, val);
+
return sprintf(buf, "%ld\n", val);
}
char *buf)
{
struct hwmon_device_attribute *hattr = to_hwmon_attr(devattr);
+ enum hwmon_sensor_types type = hattr->type;
const char *s;
int ret;
if (ret < 0)
return ret;
+ trace_hwmon_attr_show_string(hattr->index + hwmon_attr_base(type),
+ hattr->name, s);
+
return sprintf(buf, "%s\n", s);
}
if (ret < 0)
return ret;
- return count;
-}
+ trace_hwmon_attr_store(hattr->index + hwmon_attr_base(hattr->type),
+ hattr->name, val);
-static int hwmon_attr_base(enum hwmon_sensor_types type)
-{
- if (type == hwmon_in)
- return 0;
- return 1;
+ return count;
}
static bool is_string_attr(enum hwmon_sensor_types type, u32 attr)
[hwmon_in_max_alarm] = "in%d_max_alarm",
[hwmon_in_lcrit_alarm] = "in%d_lcrit_alarm",
[hwmon_in_crit_alarm] = "in%d_crit_alarm",
+ [hwmon_in_enable] = "in%d_enable",
};
static const char * const hwmon_curr_attr_templates[] = {
struct aem_ipmi_data {
struct completion read_complete;
struct ipmi_addr address;
- ipmi_user_t user;
+ struct ipmi_user *user;
int interface;
struct kernel_ipmi_msg tx_message;
struct ipmi_addr address;
struct completion read_complete;
- ipmi_user_t user;
+ struct ipmi_user *user;
int interface;
struct kernel_ipmi_msg tx_message;
for_each_child_of_node(opal, np) {
const char *label;
- if (np->name == NULL)
- continue;
-
type = get_sensor_type(np);
if (type == MAX_SENSOR_TYPE)
continue;
const char *label;
enum sensors type;
- if (np->name == NULL)
- continue;
-
type = get_sensor_type(np);
if (type == MAX_SENSOR_TYPE)
continue;
if (of_property_read_u32(np, "sensor-id", &sensor_id) &&
of_property_read_u32(np, "sensor-data", &sensor_id)) {
dev_info(&pdev->dev,
- "'sensor-id' missing in the node '%s'\n",
- np->name);
+ "'sensor-id' missing in the node '%pOFn'\n",
+ np);
continue;
}
int in_i = 1, temp_i = 1, curr_i = 1, humidity_i = 1;
enum iio_chan_type type;
struct iio_channel *channels;
- const char *name = "iio_hwmon";
struct device *hwmon_dev;
char *sname;
- if (dev->of_node && dev->of_node->name)
- name = dev->of_node->name;
-
channels = devm_iio_channel_get_all(dev);
if (IS_ERR(channels)) {
if (PTR_ERR(channels) == -ENODEV)
st->attr_group.attrs = st->attrs;
st->groups[0] = &st->attr_group;
- sname = devm_kstrdup(dev, name, GFP_KERNEL);
- if (!sname)
- return -ENOMEM;
+ if (dev->of_node) {
+ sname = devm_kasprintf(dev, GFP_KERNEL, "%pOFn", dev->of_node);
+ if (!sname)
+ return -ENOMEM;
+ strreplace(sname, '-', '_');
+ } else {
+ sname = "iio_hwmon";
+ }
- strreplace(sname, '-', '_');
hwmon_dev = devm_hwmon_device_register_with_groups(dev, sname, st,
st->groups);
return PTR_ERR_OR_ZERO(hwmon_dev);
#define INA3221_WARN3 0x0c
#define INA3221_MASK_ENABLE 0x0f
-#define INA3221_CONFIG_MODE_SHUNT BIT(1)
-#define INA3221_CONFIG_MODE_BUS BIT(2)
-#define INA3221_CONFIG_MODE_CONTINUOUS BIT(3)
+#define INA3221_CONFIG_MODE_MASK GENMASK(2, 0)
+#define INA3221_CONFIG_MODE_POWERDOWN 0
+#define INA3221_CONFIG_MODE_SHUNT BIT(0)
+#define INA3221_CONFIG_MODE_BUS BIT(1)
+#define INA3221_CONFIG_MODE_CONTINUOUS BIT(2)
+#define INA3221_CONFIG_CHx_EN(x) BIT(14 - (x))
#define INA3221_RSHUNT_DEFAULT 10000
INA3221_NUM_CHANNELS
};
-static const unsigned int register_channel[] = {
- [INA3221_SHUNT1] = INA3221_CHANNEL1,
- [INA3221_SHUNT2] = INA3221_CHANNEL2,
- [INA3221_SHUNT3] = INA3221_CHANNEL3,
- [INA3221_CRIT1] = INA3221_CHANNEL1,
- [INA3221_CRIT2] = INA3221_CHANNEL2,
- [INA3221_CRIT3] = INA3221_CHANNEL3,
- [INA3221_WARN1] = INA3221_CHANNEL1,
- [INA3221_WARN2] = INA3221_CHANNEL2,
- [INA3221_WARN3] = INA3221_CHANNEL3,
+/**
+ * struct ina3221_input - channel input source specific information
+ * @label: label of channel input source
+ * @shunt_resistor: shunt resistor value of channel input source
+ * @disconnected: connection status of channel input source
+ */
+struct ina3221_input {
+ const char *label;
+ int shunt_resistor;
+ bool disconnected;
};
/**
* struct ina3221_data - device specific information
* @regmap: Register map of the device
* @fields: Register fields of the device
- * @shunt_resistors: Array of resistor values per channel
+ * @inputs: Array of channel input source specific structures
+ * @reg_config: Register value of INA3221_CONFIG
*/
struct ina3221_data {
struct regmap *regmap;
struct regmap_field *fields[F_MAX_FIELDS];
- int shunt_resistors[INA3221_NUM_CHANNELS];
+ struct ina3221_input inputs[INA3221_NUM_CHANNELS];
+ u32 reg_config;
};
+static inline bool ina3221_is_enabled(struct ina3221_data *ina, int channel)
+{
+ return ina->reg_config & INA3221_CONFIG_CHx_EN(channel);
+}
+
static int ina3221_read_value(struct ina3221_data *ina, unsigned int reg,
int *val)
{
return 0;
}
-static ssize_t ina3221_show_bus_voltage(struct device *dev,
- struct device_attribute *attr,
- char *buf)
+static const u8 ina3221_in_reg[] = {
+ INA3221_BUS1,
+ INA3221_BUS2,
+ INA3221_BUS3,
+ INA3221_SHUNT1,
+ INA3221_SHUNT2,
+ INA3221_SHUNT3,
+};
+
+static int ina3221_read_in(struct device *dev, u32 attr, int channel, long *val)
{
- struct sensor_device_attribute *sd_attr = to_sensor_dev_attr(attr);
+ const bool is_shunt = channel > INA3221_CHANNEL3;
struct ina3221_data *ina = dev_get_drvdata(dev);
- unsigned int reg = sd_attr->index;
- int val, voltage_mv, ret;
-
- ret = ina3221_read_value(ina, reg, &val);
- if (ret)
- return ret;
+ u8 reg = ina3221_in_reg[channel];
+ int regval, ret;
+
+ /* Translate shunt channel index to sensor channel index */
+ channel %= INA3221_NUM_CHANNELS;
+
+ switch (attr) {
+ case hwmon_in_input:
+ if (!ina3221_is_enabled(ina, channel))
+ return -ENODATA;
+
+ ret = ina3221_read_value(ina, reg, ®val);
+ if (ret)
+ return ret;
+
+ /*
+ * Scale of shunt voltage (uV): LSB is 40uV
+ * Scale of bus voltage (mV): LSB is 8mV
+ */
+ *val = regval * (is_shunt ? 40 : 8);
+ return 0;
+ case hwmon_in_enable:
+ *val = ina3221_is_enabled(ina, channel);
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+}
- voltage_mv = val * 8;
+static const u8 ina3221_curr_reg[][INA3221_NUM_CHANNELS] = {
+ [hwmon_curr_input] = { INA3221_SHUNT1, INA3221_SHUNT2, INA3221_SHUNT3 },
+ [hwmon_curr_max] = { INA3221_WARN1, INA3221_WARN2, INA3221_WARN3 },
+ [hwmon_curr_crit] = { INA3221_CRIT1, INA3221_CRIT2, INA3221_CRIT3 },
+ [hwmon_curr_max_alarm] = { F_WF1, F_WF2, F_WF3 },
+ [hwmon_curr_crit_alarm] = { F_CF1, F_CF2, F_CF3 },
+};
- return snprintf(buf, PAGE_SIZE, "%d\n", voltage_mv);
+static int ina3221_read_curr(struct device *dev, u32 attr,
+ int channel, long *val)
+{
+ struct ina3221_data *ina = dev_get_drvdata(dev);
+ struct ina3221_input *input = &ina->inputs[channel];
+ int resistance_uo = input->shunt_resistor;
+ u8 reg = ina3221_curr_reg[attr][channel];
+ int regval, voltage_nv, ret;
+
+ switch (attr) {
+ case hwmon_curr_input:
+ if (!ina3221_is_enabled(ina, channel))
+ return -ENODATA;
+ /* fall through */
+ case hwmon_curr_crit:
+ case hwmon_curr_max:
+ ret = ina3221_read_value(ina, reg, ®val);
+ if (ret)
+ return ret;
+
+ /* Scale of shunt voltage: LSB is 40uV (40000nV) */
+ voltage_nv = regval * 40000;
+ /* Return current in mA */
+ *val = DIV_ROUND_CLOSEST(voltage_nv, resistance_uo);
+ return 0;
+ case hwmon_curr_crit_alarm:
+ case hwmon_curr_max_alarm:
+ ret = regmap_field_read(ina->fields[reg], ®val);
+ if (ret)
+ return ret;
+ *val = regval;
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
}
-static ssize_t ina3221_show_shunt_voltage(struct device *dev,
- struct device_attribute *attr,
- char *buf)
+static int ina3221_write_curr(struct device *dev, u32 attr,
+ int channel, long val)
{
- struct sensor_device_attribute *sd_attr = to_sensor_dev_attr(attr);
struct ina3221_data *ina = dev_get_drvdata(dev);
- unsigned int reg = sd_attr->index;
- int val, voltage_uv, ret;
+ struct ina3221_input *input = &ina->inputs[channel];
+ int resistance_uo = input->shunt_resistor;
+ u8 reg = ina3221_curr_reg[attr][channel];
+ int regval, current_ma, voltage_uv;
- ret = ina3221_read_value(ina, reg, &val);
- if (ret)
- return ret;
- voltage_uv = val * 40;
+ /* clamp current */
+ current_ma = clamp_val(val,
+ INT_MIN / resistance_uo,
+ INT_MAX / resistance_uo);
+
+ voltage_uv = DIV_ROUND_CLOSEST(current_ma * resistance_uo, 1000);
- return snprintf(buf, PAGE_SIZE, "%d\n", voltage_uv);
+ /* clamp voltage */
+ voltage_uv = clamp_val(voltage_uv, -163800, 163800);
+
+ /* 1 / 40uV(scale) << 3(register shift) = 5 */
+ regval = DIV_ROUND_CLOSEST(voltage_uv, 5) & 0xfff8;
+
+ return regmap_write(ina->regmap, reg, regval);
}
-static ssize_t ina3221_show_current(struct device *dev,
- struct device_attribute *attr, char *buf)
+static int ina3221_write_enable(struct device *dev, int channel, bool enable)
{
- struct sensor_device_attribute *sd_attr = to_sensor_dev_attr(attr);
struct ina3221_data *ina = dev_get_drvdata(dev);
- unsigned int reg = sd_attr->index;
- unsigned int channel = register_channel[reg];
- int resistance_uo = ina->shunt_resistors[channel];
- int val, current_ma, voltage_nv, ret;
+ u16 config, mask = INA3221_CONFIG_CHx_EN(channel);
+ int ret;
- ret = ina3221_read_value(ina, reg, &val);
+ config = enable ? mask : 0;
+
+ /* Enable or disable the channel */
+ ret = regmap_update_bits(ina->regmap, INA3221_CONFIG, mask, config);
if (ret)
return ret;
- voltage_nv = val * 40000;
- current_ma = DIV_ROUND_CLOSEST(voltage_nv, resistance_uo);
+ /* Cache the latest config register value */
+ ret = regmap_read(ina->regmap, INA3221_CONFIG, &ina->reg_config);
+ if (ret)
+ return ret;
- return snprintf(buf, PAGE_SIZE, "%d\n", current_ma);
+ return 0;
}
-static ssize_t ina3221_set_current(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t count)
+static int ina3221_read(struct device *dev, enum hwmon_sensor_types type,
+ u32 attr, int channel, long *val)
+{
+ switch (type) {
+ case hwmon_in:
+ /* 0-align channel ID */
+ return ina3221_read_in(dev, attr, channel - 1, val);
+ case hwmon_curr:
+ return ina3221_read_curr(dev, attr, channel, val);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static int ina3221_write(struct device *dev, enum hwmon_sensor_types type,
+ u32 attr, int channel, long val)
+{
+ switch (type) {
+ case hwmon_in:
+ /* 0-align channel ID */
+ return ina3221_write_enable(dev, channel - 1, val);
+ case hwmon_curr:
+ return ina3221_write_curr(dev, attr, channel, val);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static int ina3221_read_string(struct device *dev, enum hwmon_sensor_types type,
+ u32 attr, int channel, const char **str)
{
- struct sensor_device_attribute *sd_attr = to_sensor_dev_attr(attr);
struct ina3221_data *ina = dev_get_drvdata(dev);
- unsigned int reg = sd_attr->index;
- unsigned int channel = register_channel[reg];
- int resistance_uo = ina->shunt_resistors[channel];
- int val, current_ma, voltage_uv, ret;
+ int index = channel - 1;
- ret = kstrtoint(buf, 0, ¤t_ma);
- if (ret)
- return ret;
+ *str = ina->inputs[index].label;
- /* clamp current */
- current_ma = clamp_val(current_ma,
- INT_MIN / resistance_uo,
- INT_MAX / resistance_uo);
+ return 0;
+}
- voltage_uv = DIV_ROUND_CLOSEST(current_ma * resistance_uo, 1000);
+static umode_t ina3221_is_visible(const void *drvdata,
+ enum hwmon_sensor_types type,
+ u32 attr, int channel)
+{
+ const struct ina3221_data *ina = drvdata;
+ const struct ina3221_input *input = NULL;
+
+ switch (type) {
+ case hwmon_in:
+ /* Ignore in0_ */
+ if (channel == 0)
+ return 0;
+
+ switch (attr) {
+ case hwmon_in_label:
+ if (channel - 1 <= INA3221_CHANNEL3)
+ input = &ina->inputs[channel - 1];
+ /* Hide label node if label is not provided */
+ return (input && input->label) ? 0444 : 0;
+ case hwmon_in_input:
+ return 0444;
+ case hwmon_in_enable:
+ return 0644;
+ default:
+ return 0;
+ }
+ case hwmon_curr:
+ switch (attr) {
+ case hwmon_curr_input:
+ case hwmon_curr_crit_alarm:
+ case hwmon_curr_max_alarm:
+ return 0444;
+ case hwmon_curr_crit:
+ case hwmon_curr_max:
+ return 0644;
+ default:
+ return 0;
+ }
+ default:
+ return 0;
+ }
+}
- /* clamp voltage */
- voltage_uv = clamp_val(voltage_uv, -163800, 163800);
+static const u32 ina3221_in_config[] = {
+ /* 0: dummy, skipped in is_visible */
+ HWMON_I_INPUT,
+ /* 1-3: input voltage Channels */
+ HWMON_I_INPUT | HWMON_I_ENABLE | HWMON_I_LABEL,
+ HWMON_I_INPUT | HWMON_I_ENABLE | HWMON_I_LABEL,
+ HWMON_I_INPUT | HWMON_I_ENABLE | HWMON_I_LABEL,
+ /* 4-6: shunt voltage Channels */
+ HWMON_I_INPUT,
+ HWMON_I_INPUT,
+ HWMON_I_INPUT,
+ 0
+};
- /* 1 / 40uV(scale) << 3(register shift) = 5 */
- val = DIV_ROUND_CLOSEST(voltage_uv, 5) & 0xfff8;
+static const struct hwmon_channel_info ina3221_in = {
+ .type = hwmon_in,
+ .config = ina3221_in_config,
+};
- ret = regmap_write(ina->regmap, reg, val);
- if (ret)
- return ret;
+#define INA3221_HWMON_CURR_CONFIG (HWMON_C_INPUT | \
+ HWMON_C_CRIT | HWMON_C_CRIT_ALARM | \
+ HWMON_C_MAX | HWMON_C_MAX_ALARM)
- return count;
-}
+static const u32 ina3221_curr_config[] = {
+ INA3221_HWMON_CURR_CONFIG,
+ INA3221_HWMON_CURR_CONFIG,
+ INA3221_HWMON_CURR_CONFIG,
+ 0
+};
+
+static const struct hwmon_channel_info ina3221_curr = {
+ .type = hwmon_curr,
+ .config = ina3221_curr_config,
+};
+
+static const struct hwmon_channel_info *ina3221_info[] = {
+ &ina3221_in,
+ &ina3221_curr,
+ NULL
+};
+
+static const struct hwmon_ops ina3221_hwmon_ops = {
+ .is_visible = ina3221_is_visible,
+ .read_string = ina3221_read_string,
+ .read = ina3221_read,
+ .write = ina3221_write,
+};
+static const struct hwmon_chip_info ina3221_chip_info = {
+ .ops = &ina3221_hwmon_ops,
+ .info = ina3221_info,
+};
+
+/* Extra attribute groups */
static ssize_t ina3221_show_shunt(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct sensor_device_attribute *sd_attr = to_sensor_dev_attr(attr);
struct ina3221_data *ina = dev_get_drvdata(dev);
unsigned int channel = sd_attr->index;
- unsigned int resistance_uo;
-
- resistance_uo = ina->shunt_resistors[channel];
+ struct ina3221_input *input = &ina->inputs[channel];
- return snprintf(buf, PAGE_SIZE, "%d\n", resistance_uo);
+ return snprintf(buf, PAGE_SIZE, "%d\n", input->shunt_resistor);
}
static ssize_t ina3221_set_shunt(struct device *dev,
struct sensor_device_attribute *sd_attr = to_sensor_dev_attr(attr);
struct ina3221_data *ina = dev_get_drvdata(dev);
unsigned int channel = sd_attr->index;
+ struct ina3221_input *input = &ina->inputs[channel];
int val;
int ret;
val = clamp_val(val, 1, INT_MAX);
- ina->shunt_resistors[channel] = val;
+ input->shunt_resistor = val;
return count;
}
-static ssize_t ina3221_show_alert(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct sensor_device_attribute *sd_attr = to_sensor_dev_attr(attr);
- struct ina3221_data *ina = dev_get_drvdata(dev);
- unsigned int field = sd_attr->index;
- unsigned int regval;
- int ret;
-
- ret = regmap_field_read(ina->fields[field], ®val);
- if (ret)
- return ret;
-
- return snprintf(buf, PAGE_SIZE, "%d\n", regval);
-}
-
-/* bus voltage */
-static SENSOR_DEVICE_ATTR(in1_input, S_IRUGO,
- ina3221_show_bus_voltage, NULL, INA3221_BUS1);
-static SENSOR_DEVICE_ATTR(in2_input, S_IRUGO,
- ina3221_show_bus_voltage, NULL, INA3221_BUS2);
-static SENSOR_DEVICE_ATTR(in3_input, S_IRUGO,
- ina3221_show_bus_voltage, NULL, INA3221_BUS3);
-
-/* calculated current */
-static SENSOR_DEVICE_ATTR(curr1_input, S_IRUGO,
- ina3221_show_current, NULL, INA3221_SHUNT1);
-static SENSOR_DEVICE_ATTR(curr2_input, S_IRUGO,
- ina3221_show_current, NULL, INA3221_SHUNT2);
-static SENSOR_DEVICE_ATTR(curr3_input, S_IRUGO,
- ina3221_show_current, NULL, INA3221_SHUNT3);
-
/* shunt resistance */
static SENSOR_DEVICE_ATTR(shunt1_resistor, S_IRUGO | S_IWUSR,
ina3221_show_shunt, ina3221_set_shunt, INA3221_CHANNEL1);
static SENSOR_DEVICE_ATTR(shunt3_resistor, S_IRUGO | S_IWUSR,
ina3221_show_shunt, ina3221_set_shunt, INA3221_CHANNEL3);
-/* critical current */
-static SENSOR_DEVICE_ATTR(curr1_crit, S_IRUGO | S_IWUSR,
- ina3221_show_current, ina3221_set_current, INA3221_CRIT1);
-static SENSOR_DEVICE_ATTR(curr2_crit, S_IRUGO | S_IWUSR,
- ina3221_show_current, ina3221_set_current, INA3221_CRIT2);
-static SENSOR_DEVICE_ATTR(curr3_crit, S_IRUGO | S_IWUSR,
- ina3221_show_current, ina3221_set_current, INA3221_CRIT3);
-
-/* critical current alert */
-static SENSOR_DEVICE_ATTR(curr1_crit_alarm, S_IRUGO,
- ina3221_show_alert, NULL, F_CF1);
-static SENSOR_DEVICE_ATTR(curr2_crit_alarm, S_IRUGO,
- ina3221_show_alert, NULL, F_CF2);
-static SENSOR_DEVICE_ATTR(curr3_crit_alarm, S_IRUGO,
- ina3221_show_alert, NULL, F_CF3);
-
-/* warning current */
-static SENSOR_DEVICE_ATTR(curr1_max, S_IRUGO | S_IWUSR,
- ina3221_show_current, ina3221_set_current, INA3221_WARN1);
-static SENSOR_DEVICE_ATTR(curr2_max, S_IRUGO | S_IWUSR,
- ina3221_show_current, ina3221_set_current, INA3221_WARN2);
-static SENSOR_DEVICE_ATTR(curr3_max, S_IRUGO | S_IWUSR,
- ina3221_show_current, ina3221_set_current, INA3221_WARN3);
-
-/* warning current alert */
-static SENSOR_DEVICE_ATTR(curr1_max_alarm, S_IRUGO,
- ina3221_show_alert, NULL, F_WF1);
-static SENSOR_DEVICE_ATTR(curr2_max_alarm, S_IRUGO,
- ina3221_show_alert, NULL, F_WF2);
-static SENSOR_DEVICE_ATTR(curr3_max_alarm, S_IRUGO,
- ina3221_show_alert, NULL, F_WF3);
-
-/* shunt voltage */
-static SENSOR_DEVICE_ATTR(in4_input, S_IRUGO,
- ina3221_show_shunt_voltage, NULL, INA3221_SHUNT1);
-static SENSOR_DEVICE_ATTR(in5_input, S_IRUGO,
- ina3221_show_shunt_voltage, NULL, INA3221_SHUNT2);
-static SENSOR_DEVICE_ATTR(in6_input, S_IRUGO,
- ina3221_show_shunt_voltage, NULL, INA3221_SHUNT3);
-
static struct attribute *ina3221_attrs[] = {
- /* channel 1 */
- &sensor_dev_attr_in1_input.dev_attr.attr,
- &sensor_dev_attr_curr1_input.dev_attr.attr,
&sensor_dev_attr_shunt1_resistor.dev_attr.attr,
- &sensor_dev_attr_curr1_crit.dev_attr.attr,
- &sensor_dev_attr_curr1_crit_alarm.dev_attr.attr,
- &sensor_dev_attr_curr1_max.dev_attr.attr,
- &sensor_dev_attr_curr1_max_alarm.dev_attr.attr,
- &sensor_dev_attr_in4_input.dev_attr.attr,
-
- /* channel 2 */
- &sensor_dev_attr_in2_input.dev_attr.attr,
- &sensor_dev_attr_curr2_input.dev_attr.attr,
&sensor_dev_attr_shunt2_resistor.dev_attr.attr,
- &sensor_dev_attr_curr2_crit.dev_attr.attr,
- &sensor_dev_attr_curr2_crit_alarm.dev_attr.attr,
- &sensor_dev_attr_curr2_max.dev_attr.attr,
- &sensor_dev_attr_curr2_max_alarm.dev_attr.attr,
- &sensor_dev_attr_in5_input.dev_attr.attr,
-
- /* channel 3 */
- &sensor_dev_attr_in3_input.dev_attr.attr,
- &sensor_dev_attr_curr3_input.dev_attr.attr,
&sensor_dev_attr_shunt3_resistor.dev_attr.attr,
- &sensor_dev_attr_curr3_crit.dev_attr.attr,
- &sensor_dev_attr_curr3_crit_alarm.dev_attr.attr,
- &sensor_dev_attr_curr3_max.dev_attr.attr,
- &sensor_dev_attr_curr3_max_alarm.dev_attr.attr,
- &sensor_dev_attr_in6_input.dev_attr.attr,
-
NULL,
};
ATTRIBUTE_GROUPS(ina3221);
static const struct regmap_range ina3221_yes_ranges[] = {
- regmap_reg_range(INA3221_SHUNT1, INA3221_BUS3),
+ regmap_reg_range(INA3221_CONFIG, INA3221_BUS3),
regmap_reg_range(INA3221_MASK_ENABLE, INA3221_MASK_ENABLE),
};
.volatile_table = &ina3221_volatile_table,
};
+static int ina3221_probe_child_from_dt(struct device *dev,
+ struct device_node *child,
+ struct ina3221_data *ina)
+{
+ struct ina3221_input *input;
+ u32 val;
+ int ret;
+
+ ret = of_property_read_u32(child, "reg", &val);
+ if (ret) {
+ dev_err(dev, "missing reg property of %s\n", child->name);
+ return ret;
+ } else if (val > INA3221_CHANNEL3) {
+ dev_err(dev, "invalid reg %d of %s\n", val, child->name);
+ return ret;
+ }
+
+ input = &ina->inputs[val];
+
+ /* Log the disconnected channel input */
+ if (!of_device_is_available(child)) {
+ input->disconnected = true;
+ return 0;
+ }
+
+ /* Save the connected input label if available */
+ of_property_read_string(child, "label", &input->label);
+
+ /* Overwrite default shunt resistor value optionally */
+ if (!of_property_read_u32(child, "shunt-resistor-micro-ohms", &val)) {
+ if (val < 1 || val > INT_MAX) {
+ dev_err(dev, "invalid shunt resistor value %u of %s\n",
+ val, child->name);
+ return -EINVAL;
+ }
+ input->shunt_resistor = val;
+ }
+
+ return 0;
+}
+
+static int ina3221_probe_from_dt(struct device *dev, struct ina3221_data *ina)
+{
+ const struct device_node *np = dev->of_node;
+ struct device_node *child;
+ int ret;
+
+ /* Compatible with non-DT platforms */
+ if (!np)
+ return 0;
+
+ for_each_child_of_node(np, child) {
+ ret = ina3221_probe_child_from_dt(dev, child, ina);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
static int ina3221_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
}
for (i = 0; i < INA3221_NUM_CHANNELS; i++)
- ina->shunt_resistors[i] = INA3221_RSHUNT_DEFAULT;
+ ina->inputs[i].shunt_resistor = INA3221_RSHUNT_DEFAULT;
+
+ ret = ina3221_probe_from_dt(dev, ina);
+ if (ret) {
+ dev_err(dev, "Unable to probe from device tree\n");
+ return ret;
+ }
ret = regmap_field_write(ina->fields[F_RST], true);
if (ret) {
return ret;
}
- hwmon_dev = devm_hwmon_device_register_with_groups(dev,
- client->name,
- ina, ina3221_groups);
+ /* Sync config register after reset */
+ ret = regmap_read(ina->regmap, INA3221_CONFIG, &ina->reg_config);
+ if (ret)
+ return ret;
+
+ /* Disable channels if their inputs are disconnected */
+ for (i = 0; i < INA3221_NUM_CHANNELS; i++) {
+ if (ina->inputs[i].disconnected)
+ ina->reg_config &= ~INA3221_CONFIG_CHx_EN(i);
+ }
+ ret = regmap_write(ina->regmap, INA3221_CONFIG, ina->reg_config);
+ if (ret)
+ return ret;
+
+ dev_set_drvdata(dev, ina);
+
+ hwmon_dev = devm_hwmon_device_register_with_info(dev, client->name, ina,
+ &ina3221_chip_info,
+ ina3221_groups);
if (IS_ERR(hwmon_dev)) {
dev_err(dev, "Unable to register hwmon device\n");
return PTR_ERR(hwmon_dev);
return 0;
}
+static int __maybe_unused ina3221_suspend(struct device *dev)
+{
+ struct ina3221_data *ina = dev_get_drvdata(dev);
+ int ret;
+
+ /* Save config register value and enable cache-only */
+ ret = regmap_read(ina->regmap, INA3221_CONFIG, &ina->reg_config);
+ if (ret)
+ return ret;
+
+ /* Set to power-down mode for power saving */
+ ret = regmap_update_bits(ina->regmap, INA3221_CONFIG,
+ INA3221_CONFIG_MODE_MASK,
+ INA3221_CONFIG_MODE_POWERDOWN);
+ if (ret)
+ return ret;
+
+ regcache_cache_only(ina->regmap, true);
+ regcache_mark_dirty(ina->regmap);
+
+ return 0;
+}
+
+static int __maybe_unused ina3221_resume(struct device *dev)
+{
+ struct ina3221_data *ina = dev_get_drvdata(dev);
+ int ret;
+
+ regcache_cache_only(ina->regmap, false);
+
+ /* Software reset the chip */
+ ret = regmap_field_write(ina->fields[F_RST], true);
+ if (ret) {
+ dev_err(dev, "Unable to reset device\n");
+ return ret;
+ }
+
+ /* Restore cached register values to hardware */
+ ret = regcache_sync(ina->regmap);
+ if (ret)
+ return ret;
+
+ /* Restore config register value to hardware */
+ ret = regmap_write(ina->regmap, INA3221_CONFIG, ina->reg_config);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static const struct dev_pm_ops ina3221_pm = {
+ SET_SYSTEM_SLEEP_PM_OPS(ina3221_suspend, ina3221_resume)
+};
+
static const struct of_device_id ina3221_of_match_table[] = {
{ .compatible = "ti,ina3221", },
{ /* sentinel */ }
.driver = {
.name = INA3221_DRIVER_NAME,
.of_match_table = ina3221_of_match_table,
+ .pm = &ina3221_pm,
},
.id_table = ina3221_ids,
};
data->pdev = pdev;
- if (boot_cpu_data.x86 == 0x15 && (boot_cpu_data.x86_model == 0x60 ||
- boot_cpu_data.x86_model == 0x70)) {
+ if (boot_cpu_data.x86 == 0x15 &&
+ ((boot_cpu_data.x86_model & 0xf0) == 0x60 ||
+ (boot_cpu_data.x86_model & 0xf0) == 0x70)) {
data->read_htcreg = read_htcreg_nb_f15;
data->read_tempreg = read_tempreg_nb_f15;
} else if (boot_cpu_data.x86 == 0x17) {
lm75b,
max6625,
max6626,
+ max31725,
mcp980x,
stds75,
tcn75,
static const unsigned short normal_i2c[] = { 0x48, 0x49, 0x4a, 0x4b, 0x4c,
0x4d, 0x4e, 0x4f, I2C_CLIENT_END };
-
/* The LM75 registers */
#define LM75_REG_TEMP 0x00
#define LM75_REG_CONF 0x01
struct i2c_client *client;
struct regmap *regmap;
u8 orig_conf;
- u8 resolution; /* In bits, between 9 and 12 */
+ u8 resolution; /* In bits, between 9 and 16 */
u8 resolution_limits;
unsigned int sample_time; /* In ms */
};
.volatile_reg = lm75_is_volatile_reg,
.val_format_endian = REGMAP_ENDIAN_BIG,
.cache_type = REGCACHE_RBTREE,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
static void lm75_remove(void *data)
data->resolution_limits = 9;
data->sample_time = MSEC_PER_SEC / 4;
break;
+ case max31725:
+ data->resolution = 16;
+ data->sample_time = MSEC_PER_SEC / 8;
+ break;
case tcn75:
data->resolution = 9;
data->sample_time = MSEC_PER_SEC / 8;
{ "lm75b", lm75b, },
{ "max6625", max6625, },
{ "max6626", max6626, },
+ { "max31725", max31725, },
+ { "max31726", max31725, },
{ "mcp980x", mcp980x, },
{ "stds75", stds75, },
{ "tcn75", tcn75, },
.compatible = "maxim,max6626",
.data = (void *)max6626
},
+ {
+ .compatible = "maxim,max31725",
+ .data = (void *)max31725
+ },
+ {
+ .compatible = "maxim,max31726",
+ .data = (void *)max31725
+ },
{
.compatible = "maxim,mcp980x",
.data = (void *)mcp980x
mutex_lock(&data->update_lock);
- if (time_after(jiffies, data->last_updated + HZ)
- || !data->valid) {
+ if (time_after(jiffies, data->last_updated + HZ) ||
+ !data->valid) {
dev_dbg(&client->dev, "Updating lm92 data\n");
for (i = 0; i < t_num_regs; i++) {
data->temp[i] =
}
static ssize_t set_temp(struct device *dev, struct device_attribute *devattr,
- const char *buf, size_t count)
+ const char *buf, size_t count)
{
struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr);
struct lm92_data *data = dev_get_drvdata(dev);
int nr = attr->index;
long val;
int err;
-
+
err = kstrtol(buf, 10, &val);
if (err)
return err;
{
struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr);
struct lm92_data *data = lm92_update_device(dev);
+
return sprintf(buf, "%d\n", TEMP_FROM_REG(data->temp[attr->index])
- TEMP_FROM_REG(data->temp[t_hyst]));
}
struct device_attribute *attr, char *buf)
{
struct lm92_data *data = lm92_update_device(dev);
+
return sprintf(buf, "%d\n", TEMP_FROM_REG(data->temp[t_min])
+ TEMP_FROM_REG(data->temp[t_hyst]));
}
val = clamp_val(val, -120000, 220000);
mutex_lock(&data->update_lock);
- data->temp[t_hyst] =
+ data->temp[t_hyst] =
TEMP_TO_REG(TEMP_FROM_REG(data->temp[attr->index]) - val);
i2c_smbus_write_word_swapped(client, LM92_REG_TEMP_HYST,
data->temp[t_hyst]);
char *buf)
{
struct lm92_data *data = lm92_update_device(dev);
+
return sprintf(buf, "%d\n", ALARMS_FROM_REG(data->temp[t_input]));
}
return PTR_ERR_OR_ZERO(hwmon_dev);
}
-
/*
* Module and driver stuff
*/
.writeable_reg = lm95245_is_writeable_reg,
.volatile_reg = lm95245_is_volatile_reg,
.cache_type = REGCACHE_RBTREE,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
static const u32 lm95245_chip_config[] = {
+// SPDX-License-Identifier: GPL-2.0+
/*
* Driver for the ADC on Freescale Semiconductor MC13783 and MC13892 PMICs.
*
* Copyright 2004-2007 Freescale Semiconductor, Inc. All Rights Reserved.
* Copyright (C) 2009 Sascha Hauer, Pengutronix
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version 2
- * of the License, or (at your option) any later version.
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc., 51
- * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
#include <linux/mfd/mc13xxx.h>
* nct6793d 15 6 6 2+6 0xd120 0xc1 0x5ca3
* nct6795d 14 6 6 2+6 0xd350 0xc1 0x5ca3
* nct6796d 14 7 7 2+6 0xd420 0xc1 0x5ca3
+ * nct6797d 14 7 7 2+6 0xd450 0xc1 0x5ca3
+ * (0xd451)
+ * nct6798d 14 7 7 2+6 0xd458 0xc1 0x5ca3
+ * (0xd459)
*
* #temp lists the number of monitored temperature sources (first value) plus
* the number of directly connectable temperature sensors (second value).
#define USE_ALTERNATE
enum kinds { nct6106, nct6775, nct6776, nct6779, nct6791, nct6792, nct6793,
- nct6795, nct6796 };
+ nct6795, nct6796, nct6797, nct6798 };
/* used to set data->name = nct6775_device_names[data->sio_kind] */
static const char * const nct6775_device_names[] = {
"nct6793",
"nct6795",
"nct6796",
+ "nct6797",
+ "nct6798",
};
static const char * const nct6775_sio_names[] __initconst = {
"NCT6793D",
"NCT6795D",
"NCT6796D",
+ "NCT6797D",
+ "NCT6798D",
};
static unsigned short force_id;
#define SIO_NCT6793_ID 0xd120
#define SIO_NCT6795_ID 0xd350
#define SIO_NCT6796_ID 0xd420
-#define SIO_ID_MASK 0xFFF0
+#define SIO_NCT6797_ID 0xd450
+#define SIO_NCT6798_ID 0xd458
+#define SIO_ID_MASK 0xFFF8
enum pwm_enable { off, manual, thermal_cruise, speed_cruise, sf3, sf4 };
static const u16 NCT6779_REG_FAN[] = {
0x4c0, 0x4c2, 0x4c4, 0x4c6, 0x4c8, 0x4ca, 0x4ce };
static const u16 NCT6779_REG_FAN_PULSES[NUM_FAN] = {
- 0x644, 0x645, 0x646, 0x647, 0x648, 0x649 };
+ 0x644, 0x645, 0x646, 0x647, 0x648, 0x649, 0x64f };
static const u16 NCT6779_REG_CRITICAL_PWM_ENABLE[] = {
0x136, 0x236, 0x336, 0x836, 0x936, 0xa36, 0xb36 };
"PCH_CHIP_TEMP",
"PCH_CPU_TEMP",
"PCH_MCH_TEMP",
- "PCH_DIM0_TEMP",
- "PCH_DIM1_TEMP",
- "PCH_DIM2_TEMP",
- "PCH_DIM3_TEMP",
+ "Agent0 Dimm0",
+ "Agent0 Dimm1",
+ "Agent1 Dimm0",
+ "Agent1 Dimm1",
"BYTE_TEMP0",
"BYTE_TEMP1",
"PECI Agent 0 Calibration",
"PCH_CHIP_TEMP",
"PCH_CPU_TEMP",
"PCH_MCH_TEMP",
- "PCH_DIM0_TEMP",
- "PCH_DIM1_TEMP",
- "PCH_DIM2_TEMP",
- "PCH_DIM3_TEMP",
+ "Agent0 Dimm0",
+ "Agent0 Dimm1",
+ "Agent1 Dimm0",
+ "Agent1 Dimm1",
"BYTE_TEMP0",
"BYTE_TEMP1",
"PECI Agent 0 Calibration",
#define NCT6796_TEMP_MASK 0xbfff0ffe
#define NCT6796_VIRT_TEMP_MASK 0x80000c00
+static const char *const nct6798_temp_label[] = {
+ "",
+ "SYSTIN",
+ "CPUTIN",
+ "AUXTIN0",
+ "AUXTIN1",
+ "AUXTIN2",
+ "AUXTIN3",
+ "AUXTIN4",
+ "SMBUSMASTER 0",
+ "SMBUSMASTER 1",
+ "Virtual_TEMP",
+ "Virtual_TEMP",
+ "",
+ "",
+ "",
+ "",
+ "PECI Agent 0",
+ "PECI Agent 1",
+ "PCH_CHIP_CPU_MAX_TEMP",
+ "PCH_CHIP_TEMP",
+ "PCH_CPU_TEMP",
+ "PCH_MCH_TEMP",
+ "Agent0 Dimm0",
+ "Agent0 Dimm1",
+ "Agent1 Dimm0",
+ "Agent1 Dimm1",
+ "BYTE_TEMP0",
+ "BYTE_TEMP1",
+ "",
+ "",
+ "",
+ "Virtual_TEMP"
+};
+
+#define NCT6798_TEMP_MASK 0x8fff0ffe
+#define NCT6798_VIRT_TEMP_MASK 0x80000c00
+
/* NCT6102D/NCT6106D specific data */
#define NCT6106_REG_VBAT 0x318
case nct6793:
case nct6795:
case nct6796:
+ case nct6797:
+ case nct6798:
return reg == 0x150 || reg == 0x153 || reg == 0x155 ||
(reg & 0xfff0) == 0x4c0 ||
reg == 0x402 ||
case nct6793:
case nct6795:
case nct6796:
+ case nct6797:
+ case nct6798:
reg = nct6775_read_value(data,
data->REG_CRITICAL_PWM_ENABLE[i]);
if (reg & data->CRITICAL_PWM_ENABLE_MASK)
* Fan speed tolerance is a tricky beast, since the associated register is
* a tick counter, but the value is reported and configured as rpm.
* Compute resulting low and high rpm values and report the difference.
+ * A fan speed tolerance only makes sense if a fan target speed has been
+ * configured, so only display values other than 0 if that is the case.
*/
static ssize_t
show_speed_tolerance(struct device *dev, struct device_attribute *attr,
struct nct6775_data *data = nct6775_update_device(dev);
struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr);
int nr = sattr->index;
- int low = data->target_speed[nr] - data->target_speed_tolerance[nr];
- int high = data->target_speed[nr] + data->target_speed_tolerance[nr];
- int tolerance;
-
- if (low <= 0)
- low = 1;
- if (high > 0xffff)
- high = 0xffff;
- if (high < low)
- high = low;
-
- tolerance = (fan_from_reg16(low, data->fan_div[nr])
- - fan_from_reg16(high, data->fan_div[nr])) / 2;
+ int target = data->target_speed[nr];
+ int tolerance = 0;
+
+ if (target) {
+ int low = target - data->target_speed_tolerance[nr];
+ int high = target + data->target_speed_tolerance[nr];
+
+ if (low <= 0)
+ low = 1;
+ if (high > 0xffff)
+ high = 0xffff;
+ if (high < low)
+ high = low;
+
+ tolerance = (fan_from_reg16(low, data->fan_div[nr])
+ - fan_from_reg16(high, data->fan_div[nr])) / 2;
+ }
return sprintf(buf, "%d\n", tolerance);
}
case nct6793:
case nct6795:
case nct6796:
+ case nct6797:
+ case nct6798:
nct6775_write_value(data, data->REG_CRITICAL_PWM[nr],
val);
reg = nct6775_read_value(data,
bool pwm3pin = false, pwm4pin = false, pwm5pin = false;
bool pwm6pin = false, pwm7pin = false;
int sioreg = data->sioreg;
- int regval;
/* Store SIO_REG_ENABLE for use during resume */
superio_select(sioreg, NCT6775_LD_HWM);
/* fan4 and fan5 share some pins with the GPIO and serial flash */
if (data->kind == nct6775) {
- regval = superio_inb(sioreg, 0x2c);
+ int cr2c = superio_inb(sioreg, 0x2c);
- fan3pin = regval & BIT(6);
- pwm3pin = regval & BIT(7);
+ fan3pin = cr2c & BIT(6);
+ pwm3pin = cr2c & BIT(7);
/* On NCT6775, fan4 shares pins with the fdc interface */
fan4pin = !(superio_inb(sioreg, 0x2A) & 0x80);
fan4min = fan4pin;
pwm3pin = fan3pin;
} else if (data->kind == nct6106) {
- regval = superio_inb(sioreg, 0x24);
- fan3pin = !(regval & 0x80);
- pwm3pin = regval & 0x08;
- } else {
- /* NCT6779D, NCT6791D, NCT6792D, NCT6793D, NCT6795D, NCT6796D */
- int regval_1b, regval_2a, regval_2f;
- bool dsw_en;
-
- regval = superio_inb(sioreg, 0x1c);
+ int cr24 = superio_inb(sioreg, 0x24);
- fan3pin = !(regval & BIT(5));
- fan4pin = !(regval & BIT(6));
- fan5pin = !(regval & BIT(7));
-
- pwm3pin = !(regval & BIT(0));
- pwm4pin = !(regval & BIT(1));
- pwm5pin = !(regval & BIT(2));
+ fan3pin = !(cr24 & 0x80);
+ pwm3pin = cr24 & 0x08;
+ } else {
+ /*
+ * NCT6779D, NCT6791D, NCT6792D, NCT6793D, NCT6795D, NCT6796D,
+ * NCT6797D, NCT6798D
+ */
+ int cr1a = superio_inb(sioreg, 0x1a);
+ int cr1b = superio_inb(sioreg, 0x1b);
+ int cr1c = superio_inb(sioreg, 0x1c);
+ int cr1d = superio_inb(sioreg, 0x1d);
+ int cr2a = superio_inb(sioreg, 0x2a);
+ int cr2b = superio_inb(sioreg, 0x2b);
+ int cr2d = superio_inb(sioreg, 0x2d);
+ int cr2f = superio_inb(sioreg, 0x2f);
+ bool dsw_en = cr2f & BIT(3);
+ bool ddr4_en = cr2f & BIT(4);
+ int cre0;
+ int creb;
+ int cred;
+
+ superio_select(sioreg, NCT6775_LD_12);
+ cre0 = superio_inb(sioreg, 0xe0);
+ creb = superio_inb(sioreg, 0xeb);
+ cred = superio_inb(sioreg, 0xed);
+
+ fan3pin = !(cr1c & BIT(5));
+ fan4pin = !(cr1c & BIT(6));
+ fan5pin = !(cr1c & BIT(7));
+
+ pwm3pin = !(cr1c & BIT(0));
+ pwm4pin = !(cr1c & BIT(1));
+ pwm5pin = !(cr1c & BIT(2));
- regval = superio_inb(sioreg, 0x2d);
switch (data->kind) {
case nct6791:
+ fan6pin = cr2d & BIT(1);
+ pwm6pin = cr2d & BIT(0);
+ break;
case nct6792:
- fan6pin = regval & BIT(1);
- pwm6pin = regval & BIT(0);
+ fan6pin = !dsw_en && (cr2d & BIT(1));
+ pwm6pin = !dsw_en && (cr2d & BIT(0));
break;
case nct6793:
+ fan5pin |= cr1b & BIT(5);
+ fan5pin |= creb & BIT(5);
+
+ fan6pin = creb & BIT(3);
+
+ pwm5pin |= cr2d & BIT(7);
+ pwm5pin |= (creb & BIT(4)) && !(cr2a & BIT(0));
+
+ pwm6pin = !dsw_en && (cr2d & BIT(0));
+ pwm6pin |= creb & BIT(2);
+ break;
case nct6795:
+ fan5pin |= cr1b & BIT(5);
+ fan5pin |= creb & BIT(5);
+
+ fan6pin = (cr2a & BIT(4)) &&
+ (!dsw_en || (cred & BIT(4)));
+ fan6pin |= creb & BIT(3);
+
+ pwm5pin |= cr2d & BIT(7);
+ pwm5pin |= (creb & BIT(4)) && !(cr2a & BIT(0));
+
+ pwm6pin = (cr2a & BIT(3)) && (cred & BIT(2));
+ pwm6pin |= creb & BIT(2);
+ break;
case nct6796:
- regval_1b = superio_inb(sioreg, 0x1b);
- regval_2a = superio_inb(sioreg, 0x2a);
- regval_2f = superio_inb(sioreg, 0x2f);
- dsw_en = regval_2f & BIT(3);
+ fan5pin |= cr1b & BIT(5);
+ fan5pin |= (cre0 & BIT(3)) && !(cr1b & BIT(0));
+ fan5pin |= creb & BIT(5);
- if (!pwm5pin)
- pwm5pin = regval & BIT(7);
+ fan6pin = (cr2a & BIT(4)) &&
+ (!dsw_en || (cred & BIT(4)));
+ fan6pin |= creb & BIT(3);
- if (!fan5pin)
- fan5pin = regval_1b & BIT(5);
+ fan7pin = !(cr2b & BIT(2));
- superio_select(sioreg, NCT6775_LD_12);
- if (data->kind != nct6796) {
- int regval_eb = superio_inb(sioreg, 0xeb);
+ pwm5pin |= cr2d & BIT(7);
+ pwm5pin |= (cre0 & BIT(4)) && !(cr1b & BIT(0));
+ pwm5pin |= (creb & BIT(4)) && !(cr2a & BIT(0));
- if (!dsw_en) {
- fan6pin = regval & BIT(1);
- pwm6pin = regval & BIT(0);
- }
+ pwm6pin = (cr2a & BIT(3)) && (cred & BIT(2));
+ pwm6pin |= creb & BIT(2);
- if (!fan5pin)
- fan5pin = regval_eb & BIT(5);
- if (!pwm5pin)
- pwm5pin = (regval_eb & BIT(4)) &&
- !(regval_2a & BIT(0));
- if (!fan6pin)
- fan6pin = regval_eb & BIT(3);
- if (!pwm6pin)
- pwm6pin = regval_eb & BIT(2);
- }
+ pwm7pin = !(cr1d & (BIT(2) | BIT(3)));
+ break;
+ case nct6797:
+ fan5pin |= !ddr4_en && (cr1b & BIT(5));
+ fan5pin |= creb & BIT(5);
- if (data->kind == nct6795 || data->kind == nct6796) {
- int regval_ed = superio_inb(sioreg, 0xed);
+ fan6pin = cr2a & BIT(4);
+ fan6pin |= creb & BIT(3);
- if (!fan6pin)
- fan6pin = (regval_2a & BIT(4)) &&
- (!dsw_en ||
- (dsw_en && (regval_ed & BIT(4))));
- if (!pwm6pin)
- pwm6pin = (regval_2a & BIT(3)) &&
- (regval_ed & BIT(2));
- }
+ fan7pin = cr1a & BIT(1);
- if (data->kind == nct6796) {
- int regval_1d = superio_inb(sioreg, 0x1d);
- int regval_2b = superio_inb(sioreg, 0x2b);
+ pwm5pin |= (creb & BIT(4)) && !(cr2a & BIT(0));
+ pwm5pin |= !ddr4_en && (cr2d & BIT(7));
- fan7pin = !(regval_2b & BIT(2));
- pwm7pin = !(regval_1d & (BIT(2) | BIT(3)));
- }
+ pwm6pin = creb & BIT(2);
+ pwm6pin |= cred & BIT(2);
+ pwm7pin = cr1d & BIT(4);
+ break;
+ case nct6798:
+ fan6pin = !(cr1b & BIT(0)) && (cre0 & BIT(3));
+ fan6pin |= cr2a & BIT(4);
+ fan6pin |= creb & BIT(5);
+
+ fan7pin = cr1b & BIT(5);
+ fan7pin |= !(cr2b & BIT(2));
+ fan7pin |= creb & BIT(3);
+
+ pwm6pin = !(cr1b & BIT(0)) && (cre0 & BIT(4));
+ pwm6pin |= !(cred & BIT(2)) && (cr2a & BIT(3));
+ pwm6pin |= (creb & BIT(4)) && !(cr2a & BIT(0));
+
+ pwm7pin = !(cr1d & (BIT(2) | BIT(3)));
+ pwm7pin |= cr2d & BIT(7);
+ pwm7pin |= creb & BIT(2);
break;
default: /* NCT6779D */
break;
case nct6793:
case nct6795:
case nct6796:
+ case nct6797:
+ case nct6798:
data->in_num = 15;
- data->pwm_num = (data->kind == nct6796) ? 7 : 6;
+ data->pwm_num = (data->kind == nct6796 ||
+ data->kind == nct6797 ||
+ data->kind == nct6798) ? 7 : 6;
data->auto_pwm_num = 4;
data->has_fan_div = false;
data->temp_fixed_num = 6;
data->virt_temp_mask = NCT6793_VIRT_TEMP_MASK;
break;
case nct6795:
+ case nct6797:
data->temp_label = nct6795_temp_label;
data->temp_mask = NCT6795_TEMP_MASK;
data->virt_temp_mask = NCT6795_VIRT_TEMP_MASK;
data->temp_mask = NCT6796_TEMP_MASK;
data->virt_temp_mask = NCT6796_VIRT_TEMP_MASK;
break;
+ case nct6798:
+ data->temp_label = nct6798_temp_label;
+ data->temp_mask = NCT6798_TEMP_MASK;
+ data->virt_temp_mask = NCT6798_VIRT_TEMP_MASK;
+ break;
}
data->REG_CONFIG = NCT6775_REG_CONFIG;
case nct6793:
case nct6795:
case nct6796:
+ case nct6797:
+ case nct6798:
break;
}
case nct6793:
case nct6795:
case nct6796:
+ case nct6797:
+ case nct6798:
tmp |= 0x7e;
break;
}
case SIO_NCT6796_ID:
sio_data->kind = nct6796;
break;
+ case SIO_NCT6797_ID:
+ sio_data->kind = nct6797;
+ break;
+ case SIO_NCT6798_ID:
+ sio_data->kind = nct6798;
+ break;
default:
if (val != 0xffff)
pr_debug("unsupported chip ID: 0x%04x\n", val);
/* Define the Counter Register, value = 100 for match 100% */
#define NPCM7XX_PWM_COUNTER_DEFAULT_NUM 255
-#define NPCM7XX_PWM_CMR_DEFAULT_NUM 127
+#define NPCM7XX_PWM_CMR_DEFAULT_NUM 255
#define NPCM7XX_PWM_CMR_MAX 255
/* default all PWM channels PRESCALE2 = 1 */
dev_err(dev, "Property 'cooling-levels' cannot be read.\n");
return ret;
}
- snprintf(cdev->name, THERMAL_NAME_LENGTH, "%s%d", child->name,
+ snprintf(cdev->name, THERMAL_NAME_LENGTH, "%pOFn%d", child,
pwm_port);
cdev->tcdev = thermal_of_cooling_device_register(child,
if (fan_cnt < 1)
return -EINVAL;
- fan_ch = devm_kzalloc(dev, sizeof(*fan_ch) * fan_cnt, GFP_KERNEL);
+ fan_ch = devm_kcalloc(dev, fan_cnt, sizeof(*fan_ch), GFP_KERNEL);
if (!fan_ch)
return -ENOMEM;
menuconfig PMBUS
tristate "PMBus support"
depends on I2C
- default n
help
Say yes here if you want to enable PMBus support.
config SENSORS_ADM1275
tristate "Analog Devices ADM1275 and compatibles"
- default n
help
If you say yes here you get hardware monitoring support for Analog
Devices ADM1075, ADM1272, ADM1275, ADM1276, ADM1278, ADM1293,
config SENSORS_IR35221
tristate "Infineon IR35221"
- default n
help
If you say yes here you get hardware monitoring support for the
Infineon IR35221 controller.
config SENSORS_LM25066
tristate "National Semiconductor LM25066 and compatibles"
- default n
help
If you say yes here you get hardware monitoring support for National
Semiconductor LM25056, LM25066, LM5064, and LM5066.
config SENSORS_LTC2978
tristate "Linear Technologies LTC2978 and compatibles"
- default n
help
If you say yes here you get hardware monitoring support for Linear
Technology LTC2974, LTC2975, LTC2977, LTC2978, LTC2980, LTC3880,
depends on SENSORS_LTC2978 && REGULATOR
help
If you say yes here you get regulator support for Linear
- Technology LTC2974, LTC2977, LTC2978, LTC3880, LTC3883, and LTM4676.
+ Technology LTC2974, LTC2977, LTC2978, LTC3880, LTC3883, LTM4676
+ and LTM4686.
config SENSORS_LTC3815
tristate "Linear Technologies LTC3815"
- default n
help
If you say yes here you get hardware monitoring support for Linear
Technology LTC3815.
config SENSORS_MAX16064
tristate "Maxim MAX16064"
- default n
help
If you say yes here you get hardware monitoring support for Maxim
MAX16064.
config SENSORS_MAX20751
tristate "Maxim MAX20751"
- default n
help
If you say yes here you get hardware monitoring support for Maxim
MAX20751.
config SENSORS_MAX31785
tristate "Maxim MAX31785 and compatibles"
- default n
help
If you say yes here you get hardware monitoring support for Maxim
MAX31785.
config SENSORS_MAX34440
tristate "Maxim MAX34440 and compatibles"
- default n
help
If you say yes here you get hardware monitoring support for Maxim
MAX34440, MAX34441, MAX34446, MAX34451, MAX34460, and MAX34461.
config SENSORS_MAX8688
tristate "Maxim MAX8688"
- default n
help
If you say yes here you get hardware monitoring support for Maxim
MAX8688.
config SENSORS_TPS40422
tristate "TI TPS40422"
- default n
help
If you say yes here you get hardware monitoring support for TI
TPS40422.
config SENSORS_UCD9000
tristate "TI UCD90120, UCD90124, UCD90160, UCD9090, UCD90910"
- default n
help
If you say yes here you get hardware monitoring support for TI
UCD90120, UCD90124, UCD90160, UCD9090, UCD90910, Sequencer and System
config SENSORS_UCD9200
tristate "TI UCD9220, UCD9222, UCD9224, UCD9240, UCD9244, UCD9246, UCD9248"
- default n
help
If you say yes here you get hardware monitoring support for TI
UCD9220, UCD9222, UCD9224, UCD9240, UCD9244, UCD9246, and UCD9248
config SENSORS_ZL6100
tristate "Intersil ZL6100 and compatibles"
- default n
help
If you say yes here you get hardware monitoring support for Intersil
ZL2004, ZL2005, ZL2006, ZL2008, ZL2105, ZL2106, ZL6100, ZL6105,
* Copyright (c) 2011 Ericsson AB.
* Copyright (c) 2013, 2014, 2015 Guenter Roeck
* Copyright (c) 2015 Linear Technology
+ * Copyright (c) 2018 Analog Devices Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
#include "pmbus.h"
enum chips { ltc2974, ltc2975, ltc2977, ltc2978, ltc2980, ltc3880, ltc3882,
- ltc3883, ltc3886, ltc3887, ltm2987, ltm4675, ltm4676 };
+ ltc3883, ltc3886, ltc3887, ltm2987, ltm4675, ltm4676, ltm4686 };
/* Common for all chips */
#define LTC2978_MFR_VOUT_PEAK 0xdd
#define LTM4676_ID_REV1 0x4400
#define LTM4676_ID_REV2 0x4480
#define LTM4676A_ID 0x47e0
+#define LTM4686_ID 0x4770
#define LTC2974_NUM_PAGES 4
#define LTC2978_NUM_PAGES 8
{"ltm2987", ltm2987},
{"ltm4675", ltm4675},
{"ltm4676", ltm4676},
+ {"ltm4686", ltm4686},
{}
};
MODULE_DEVICE_TABLE(i2c, ltc2978_id);
else if (chip_id == LTM4676_ID_REV1 || chip_id == LTM4676_ID_REV2 ||
chip_id == LTM4676A_ID)
return ltm4676;
+ else if (chip_id == LTM4686_ID)
+ return ltm4686;
dev_err(&client->dev, "Unsupported chip ID 0x%x\n", chip_id);
return -ENODEV;
case ltc3887:
case ltm4675:
case ltm4676:
+ case ltm4686:
data->features |= FEAT_CLEAR_PEAKS | FEAT_NEEDS_POLLING;
info->read_word_data = ltc3880_read_word_data;
info->pages = LTC3880_NUM_PAGES;
{ .compatible = "lltc,ltm2987" },
{ .compatible = "lltc,ltm4675" },
{ .compatible = "lltc,ltm4676" },
+ { .compatible = "lltc,ltm4686" },
{ }
};
MODULE_DEVICE_TABLE(of, ltc2978_of_match);
} else {
info->pages = 1;
}
+
+ pmbus_clear_faults(client);
}
if (pmbus_check_byte_register(client, 0, PMBUS_VOUT_MODE)) {
if (ret >= 0 && (ret & PB_CAPABILITY_ERROR_CHECK))
client->flags |= I2C_CLIENT_PEC;
- pmbus_clear_faults(client);
+ if (data->info->pages)
+ pmbus_clear_faults(client);
+ else
+ pmbus_clear_fault_page(client, -1);
if (info->identify) {
ret = (*info->identify)(client, info);
ctx->pwm = devm_of_pwm_get(&pdev->dev, pdev->dev.of_node, NULL);
if (IS_ERR(ctx->pwm)) {
- dev_err(&pdev->dev, "Could not get PWM\n");
- return PTR_ERR(ctx->pwm);
+ ret = PTR_ERR(ctx->pwm);
+
+ if (ret != -EPROBE_DEFER)
+ dev_err(&pdev->dev, "Could not get PWM: %d\n", ret);
+
+ return ret;
}
platform_set_drvdata(pdev, ctx);
static int pwm_fan_suspend(struct device *dev)
{
struct pwm_fan_ctx *ctx = dev_get_drvdata(dev);
+ struct pwm_args args;
+ int ret;
+
+ pwm_get_args(ctx->pwm, &args);
+
+ if (ctx->pwm_value) {
+ ret = pwm_config(ctx->pwm, 0, args.period);
+ if (ret < 0)
+ return ret;
- if (ctx->pwm_value)
pwm_disable(ctx->pwm);
+ }
+
return 0;
}
const struct scmi_sensors *scmi_sensors = drvdata;
sensor = *(scmi_sensors->info[type] + channel);
- if (sensor && sensor->name)
+ if (sensor)
return S_IRUGO;
return 0;
* any thermal zones or if the thermal subsystem is
* not configured.
*/
- if (IS_ERR(z)) {
+ if (IS_ERR(z))
devm_kfree(dev, zone);
- continue;
- }
}
return 0;
+// SPDX-License-Identifier: GPL-2.0
/*
* sht15.c - support for the SHT15 Temperature and Humidity Sensor
*
*
* Copyright (c) 2007 Wouter Horre
*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
* For further information, see the Documentation/hwmon/sht15 file.
*/
.volatile_reg = tmp102_is_volatile_reg,
.val_format_endian = REGMAP_ENDIAN_BIG,
.cache_type = REGCACHE_RBTREE,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
static int tmp102_probe(struct i2c_client *client,
.volatile_reg = tmp108_is_volatile_reg,
.val_format_endian = REGMAP_ENDIAN_BIG,
.cache_type = REGCACHE_RBTREE,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
static int tmp108_probe(struct i2c_client *client,
{
enum chips kind;
struct i2c_adapter *adapter = client->adapter;
- const char * const names[] = { "TMP421", "TMP422", "TMP423",
- "TMP441", "TMP442" };
+ static const char * const names[] = {
+ "TMP421", "TMP422", "TMP423",
+ "TMP441", "TMP442"
+ };
int addr = client->addr;
u8 reg;
time_left = wait_event_timeout(priv->wait, priv->flags & ID_DONE,
num * adap->timeout);
- if (!time_left) {
+
+ /* cleanup DMA if it couldn't complete properly due to an error */
+ if (priv->dma_direction != DMA_NONE)
rcar_i2c_cleanup_dma(priv);
+
+ if (!time_left) {
rcar_i2c_init(priv);
ret = -ETIMEDOUT;
} else if (priv->flags & ID_NACK) {
*
* Return: NULL if a DMA safe buffer was not obtained. Use msg->buf with PIO.
* Or a valid pointer to be used with DMA. After use, release it by
- * calling i2c_release_dma_safe_msg_buf().
+ * calling i2c_put_dma_safe_msg_buf().
*
* This function must only be called from process context!
*/
struct i2c_mux_gpio_platform_data data;
unsigned gpio_base;
struct gpio_desc **gpios;
- int *values;
};
static void i2c_mux_gpio_set(const struct gpiomux *mux, unsigned val)
{
- int i;
+ DECLARE_BITMAP(values, BITS_PER_TYPE(val));
- for (i = 0; i < mux->data.n_gpios; i++)
- mux->values[i] = (val >> i) & 1;
+ values[0] = val;
- gpiod_set_array_value_cansleep(mux->data.n_gpios,
- mux->gpios, mux->values);
+ gpiod_set_array_value_cansleep(mux->data.n_gpios, mux->gpios, NULL,
+ values);
}
static int i2c_mux_gpio_select(struct i2c_mux_core *muxc, u32 chan)
return -EPROBE_DEFER;
muxc = i2c_mux_alloc(parent, &pdev->dev, mux->data.n_values,
- mux->data.n_gpios * sizeof(*mux->gpios) +
- mux->data.n_gpios * sizeof(*mux->values), 0,
+ mux->data.n_gpios * sizeof(*mux->gpios), 0,
i2c_mux_gpio_select, NULL);
if (!muxc) {
ret = -ENOMEM;
goto alloc_failed;
}
mux->gpios = muxc->priv;
- mux->values = (int *)(mux->gpios + mux->data.n_gpios);
muxc->priv = mux;
platform_set_drvdata(pdev, muxc);
ide_cd_read_toc(drive);
g->fops = &idecd_ops;
g->flags |= GENHD_FL_REMOVABLE | GENHD_FL_BLOCK_EVENTS_ON_EXCL_WRITE;
- device_add_disk(&drive->gendev, g);
+ device_add_disk(&drive->gendev, g, NULL);
return 0;
out_free_disk:
if (drive->dev_flags & IDE_DFLAG_REMOVABLE)
g->flags = GENHD_FL_REMOVABLE;
g->fops = &ide_gd_ops;
- device_add_disk(&drive->gendev, g);
+ device_add_disk(&drive->gendev, g, NULL);
return 0;
out_free_disk:
.disable_promotion_to_c1e = true,
};
-#define ICPU(model, cpu) \
- { X86_VENDOR_INTEL, 6, model, X86_FEATURE_ANY, (unsigned long)&cpu }
-
static const struct x86_cpu_id intel_idle_ids[] __initconst = {
- ICPU(INTEL_FAM6_NEHALEM_EP, idle_cpu_nehalem),
- ICPU(INTEL_FAM6_NEHALEM, idle_cpu_nehalem),
- ICPU(INTEL_FAM6_NEHALEM_G, idle_cpu_nehalem),
- ICPU(INTEL_FAM6_WESTMERE, idle_cpu_nehalem),
- ICPU(INTEL_FAM6_WESTMERE_EP, idle_cpu_nehalem),
- ICPU(INTEL_FAM6_NEHALEM_EX, idle_cpu_nehalem),
- ICPU(INTEL_FAM6_ATOM_PINEVIEW, idle_cpu_atom),
- ICPU(INTEL_FAM6_ATOM_LINCROFT, idle_cpu_lincroft),
- ICPU(INTEL_FAM6_WESTMERE_EX, idle_cpu_nehalem),
- ICPU(INTEL_FAM6_SANDYBRIDGE, idle_cpu_snb),
- ICPU(INTEL_FAM6_SANDYBRIDGE_X, idle_cpu_snb),
- ICPU(INTEL_FAM6_ATOM_CEDARVIEW, idle_cpu_atom),
- ICPU(INTEL_FAM6_ATOM_SILVERMONT1, idle_cpu_byt),
- ICPU(INTEL_FAM6_ATOM_MERRIFIELD, idle_cpu_tangier),
- ICPU(INTEL_FAM6_ATOM_AIRMONT, idle_cpu_cht),
- ICPU(INTEL_FAM6_IVYBRIDGE, idle_cpu_ivb),
- ICPU(INTEL_FAM6_IVYBRIDGE_X, idle_cpu_ivt),
- ICPU(INTEL_FAM6_HASWELL_CORE, idle_cpu_hsw),
- ICPU(INTEL_FAM6_HASWELL_X, idle_cpu_hsw),
- ICPU(INTEL_FAM6_HASWELL_ULT, idle_cpu_hsw),
- ICPU(INTEL_FAM6_HASWELL_GT3E, idle_cpu_hsw),
- ICPU(INTEL_FAM6_ATOM_SILVERMONT2, idle_cpu_avn),
- ICPU(INTEL_FAM6_BROADWELL_CORE, idle_cpu_bdw),
- ICPU(INTEL_FAM6_BROADWELL_GT3E, idle_cpu_bdw),
- ICPU(INTEL_FAM6_BROADWELL_X, idle_cpu_bdw),
- ICPU(INTEL_FAM6_BROADWELL_XEON_D, idle_cpu_bdw),
- ICPU(INTEL_FAM6_SKYLAKE_MOBILE, idle_cpu_skl),
- ICPU(INTEL_FAM6_SKYLAKE_DESKTOP, idle_cpu_skl),
- ICPU(INTEL_FAM6_KABYLAKE_MOBILE, idle_cpu_skl),
- ICPU(INTEL_FAM6_KABYLAKE_DESKTOP, idle_cpu_skl),
- ICPU(INTEL_FAM6_SKYLAKE_X, idle_cpu_skx),
- ICPU(INTEL_FAM6_XEON_PHI_KNL, idle_cpu_knl),
- ICPU(INTEL_FAM6_XEON_PHI_KNM, idle_cpu_knl),
- ICPU(INTEL_FAM6_ATOM_GOLDMONT, idle_cpu_bxt),
- ICPU(INTEL_FAM6_ATOM_GEMINI_LAKE, idle_cpu_bxt),
- ICPU(INTEL_FAM6_ATOM_DENVERTON, idle_cpu_dnv),
+ INTEL_CPU_FAM6(NEHALEM_EP, idle_cpu_nehalem),
+ INTEL_CPU_FAM6(NEHALEM, idle_cpu_nehalem),
+ INTEL_CPU_FAM6(NEHALEM_G, idle_cpu_nehalem),
+ INTEL_CPU_FAM6(WESTMERE, idle_cpu_nehalem),
+ INTEL_CPU_FAM6(WESTMERE_EP, idle_cpu_nehalem),
+ INTEL_CPU_FAM6(NEHALEM_EX, idle_cpu_nehalem),
+ INTEL_CPU_FAM6(ATOM_BONNELL, idle_cpu_atom),
+ INTEL_CPU_FAM6(ATOM_BONNELL_MID, idle_cpu_lincroft),
+ INTEL_CPU_FAM6(WESTMERE_EX, idle_cpu_nehalem),
+ INTEL_CPU_FAM6(SANDYBRIDGE, idle_cpu_snb),
+ INTEL_CPU_FAM6(SANDYBRIDGE_X, idle_cpu_snb),
+ INTEL_CPU_FAM6(ATOM_SALTWELL, idle_cpu_atom),
+ INTEL_CPU_FAM6(ATOM_SILVERMONT, idle_cpu_byt),
+ INTEL_CPU_FAM6(ATOM_SILVERMONT_MID, idle_cpu_tangier),
+ INTEL_CPU_FAM6(ATOM_AIRMONT, idle_cpu_cht),
+ INTEL_CPU_FAM6(IVYBRIDGE, idle_cpu_ivb),
+ INTEL_CPU_FAM6(IVYBRIDGE_X, idle_cpu_ivt),
+ INTEL_CPU_FAM6(HASWELL_CORE, idle_cpu_hsw),
+ INTEL_CPU_FAM6(HASWELL_X, idle_cpu_hsw),
+ INTEL_CPU_FAM6(HASWELL_ULT, idle_cpu_hsw),
+ INTEL_CPU_FAM6(HASWELL_GT3E, idle_cpu_hsw),
+ INTEL_CPU_FAM6(ATOM_SILVERMONT_X, idle_cpu_avn),
+ INTEL_CPU_FAM6(BROADWELL_CORE, idle_cpu_bdw),
+ INTEL_CPU_FAM6(BROADWELL_GT3E, idle_cpu_bdw),
+ INTEL_CPU_FAM6(BROADWELL_X, idle_cpu_bdw),
+ INTEL_CPU_FAM6(BROADWELL_XEON_D, idle_cpu_bdw),
+ INTEL_CPU_FAM6(SKYLAKE_MOBILE, idle_cpu_skl),
+ INTEL_CPU_FAM6(SKYLAKE_DESKTOP, idle_cpu_skl),
+ INTEL_CPU_FAM6(KABYLAKE_MOBILE, idle_cpu_skl),
+ INTEL_CPU_FAM6(KABYLAKE_DESKTOP, idle_cpu_skl),
+ INTEL_CPU_FAM6(SKYLAKE_X, idle_cpu_skx),
+ INTEL_CPU_FAM6(XEON_PHI_KNL, idle_cpu_knl),
+ INTEL_CPU_FAM6(XEON_PHI_KNM, idle_cpu_knl),
+ INTEL_CPU_FAM6(ATOM_GOLDMONT, idle_cpu_bxt),
+ INTEL_CPU_FAM6(ATOM_GOLDMONT_PLUS, idle_cpu_bxt),
+ INTEL_CPU_FAM6(ATOM_GOLDMONT_X, idle_cpu_dnv),
{}
};
ivt_idle_state_table_update();
break;
case INTEL_FAM6_ATOM_GOLDMONT:
- case INTEL_FAM6_ATOM_GEMINI_LAKE:
+ case INTEL_FAM6_ATOM_GOLDMONT_PLUS:
bxt_idle_state_table_update();
break;
case INTEL_FAM6_SKYLAKE_DESKTOP:
.name = APDS9960_REGMAP_NAME,
.reg_bits = 8,
.val_bits = 8,
- .use_single_rw = 1,
+ .use_single_read = true,
+ .use_single_write = true,
.volatile_table = &apds9960_volatile_table,
.precious_table = &apds9960_precious_table,
}
static const struct regmap_config max44000_regmap_config = {
- .reg_bits = 8,
- .val_bits = 8,
-
- .max_register = MAX44000_REG_PRX_DATA,
- .readable_reg = max44000_readable_reg,
- .writeable_reg = max44000_writeable_reg,
- .volatile_reg = max44000_volatile_reg,
- .precious_reg = max44000_precious_reg,
-
- .use_single_rw = 1,
- .cache_type = REGCACHE_RBTREE,
+ .reg_bits = 8,
+ .val_bits = 8,
+
+ .max_register = MAX44000_REG_PRX_DATA,
+ .readable_reg = max44000_readable_reg,
+ .writeable_reg = max44000_writeable_reg,
+ .volatile_reg = max44000_volatile_reg,
+ .precious_reg = max44000_precious_reg,
+
+ .use_single_read = true,
+ .use_single_write = true,
+ .cache_type = REGCACHE_RBTREE,
};
static irqreturn_t max44000_trigger_handler(int irq, void *p)
.rd_table = &mlx90632_readable_regs_tbl,
.wr_table = &mlx90632_writeable_regs_tbl,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.reg_format_endian = REGMAP_ENDIAN_BIG,
.val_format_endian = REGMAP_ENDIAN_BIG,
.cache_type = REGCACHE_RBTREE,
#include <linux/mutex.h>
#include <linux/slab.h>
+#include <linux/nospec.h>
+
#include <linux/uaccess.h>
#include <rdma/ib.h>
if (hdr.cmd >= ARRAY_SIZE(ucm_cmd_table))
return -EINVAL;
+ hdr.cmd = array_index_nospec(hdr.cmd, ARRAY_SIZE(ucm_cmd_table));
if (hdr.in + sizeof(hdr) > len)
return -EINVAL;
#include <linux/module.h>
#include <linux/nsproxy.h>
+#include <linux/nospec.h>
+
#include <rdma/rdma_user_cm.h>
#include <rdma/ib_marshall.h>
#include <rdma/rdma_cm.h>
if (hdr.cmd >= ARRAY_SIZE(ucma_cmd_table))
return -EINVAL;
+ hdr.cmd = array_index_nospec(hdr.cmd, ARRAY_SIZE(ucma_cmd_table));
if (hdr.in + sizeof(hdr) > len)
return -EINVAL;
int shrink = 0;
int c;
+ if (!mr->allocated_from_cache)
+ return;
+
c = order2idx(dev, mr->order);
if (c < 0 || c >= MAX_MR_CACHE_ENTRIES) {
mlx5_ib_warn(dev, "order %d, cache index %d\n", mr->order, c);
umem = NULL;
}
#endif
-
clean_mr(dev, mr);
+ /*
+ * We should unregister the DMA address from the HCA before
+ * remove the DMA mapping.
+ */
+ mlx5_mr_cache_free(dev, mr);
if (umem) {
ib_umem_release(umem);
atomic_sub(npages, &dev->mdev->priv.reg_pages);
}
-
if (!mr->allocated_from_cache)
kfree(mr);
- else
- mlx5_mr_cache_free(dev, mr);
}
int mlx5_ib_dereg_mr(struct ib_mr *ibmr)
input_inject_event(&evdev->handle,
event.type, event.code, event.value);
+ cond_resched();
}
out:
{ 0x0e6f, 0x0246, "Rock Candy Gamepad for Xbox One 2015", 0, XTYPE_XBOXONE },
{ 0x0e6f, 0x02ab, "PDP Controller for Xbox One", 0, XTYPE_XBOXONE },
{ 0x0e6f, 0x02a4, "PDP Wired Controller for Xbox One - Stealth Series", 0, XTYPE_XBOXONE },
+ { 0x0e6f, 0x02a6, "PDP Wired Controller for Xbox One - Camo Series", 0, XTYPE_XBOXONE },
{ 0x0e6f, 0x0301, "Logic3 Controller", 0, XTYPE_XBOX360 },
{ 0x0e6f, 0x0346, "Rock Candy Gamepad for Xbox One 2016", 0, XTYPE_XBOXONE },
{ 0x0e6f, 0x0401, "Logic3 Controller", 0, XTYPE_XBOX360 },
XBOXONE_INIT_PKT(0x0e6f, 0x02ab, xboxone_pdp_init2),
XBOXONE_INIT_PKT(0x0e6f, 0x02a4, xboxone_pdp_init1),
XBOXONE_INIT_PKT(0x0e6f, 0x02a4, xboxone_pdp_init2),
+ XBOXONE_INIT_PKT(0x0e6f, 0x02a6, xboxone_pdp_init1),
+ XBOXONE_INIT_PKT(0x0e6f, 0x02a6, xboxone_pdp_init2),
XBOXONE_INIT_PKT(0x24c6, 0x541a, xboxone_rumblebegin_init),
XBOXONE_INIT_PKT(0x24c6, 0x542a, xboxone_rumblebegin_init),
XBOXONE_INIT_PKT(0x24c6, 0x543a, xboxone_rumblebegin_init),
input_event(udev->dev, ev.type, ev.code, ev.value);
bytes += input_event_size();
+ cond_resched();
}
return bytes;
{ "ELAN0611", 0 },
{ "ELAN0612", 0 },
{ "ELAN0618", 0 },
+ { "ELAN061C", 0 },
{ "ELAN061D", 0 },
{ "ELAN0622", 0 },
{ "ELAN1000", 0 },
mousedev_generate_response(client, c);
spin_unlock_irq(&client->packet_lock);
+ cond_resched();
}
kill_fasync(&client->fasync, SIGIO, POLL_IN);
for (i = 0; i < I8042_NUM_PORTS; i++) {
struct serio *serio = i8042_ports[i].serio;
- if (serio) {
- printk(KERN_INFO "serio: %s at %#lx,%#lx irq %d\n",
- serio->name,
- (unsigned long) I8042_DATA_REG,
- (unsigned long) I8042_COMMAND_REG,
- i8042_ports[i].irq);
- serio_register_port(serio);
- device_set_wakeup_capable(&serio->dev, true);
- }
+ if (!serio)
+ continue;
+
+ printk(KERN_INFO "serio: %s at %#lx,%#lx irq %d\n",
+ serio->name,
+ (unsigned long) I8042_DATA_REG,
+ (unsigned long) I8042_COMMAND_REG,
+ i8042_ports[i].irq);
+ serio_register_port(serio);
+ device_set_wakeup_capable(&serio->dev, true);
+
+ /*
+ * On platforms using suspend-to-idle, allow the keyboard to
+ * wake up the system from sleep by enabling keyboard wakeups
+ * by default. This is consistent with keyboard wakeup
+ * behavior on many platforms using suspend-to-RAM (ACPI S3)
+ * by default.
+ */
+ if (pm_suspend_via_s2idle() && i == I8042_KBD_PORT_NO)
+ device_set_wakeup_enable(&serio->dev, true);
}
}
.read_flag_mask = TSC200X_REG_READ,
.write_flag_mask = TSC200X_REG_PND0,
.wr_table = &tsc200x_writable_table,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
EXPORT_SYMBOL_GPL(tsc200x_regmap_config);
return !dma_addr;
}
-const struct dma_map_ops intel_dma_ops = {
+static const struct dma_map_ops intel_dma_ops = {
.alloc = intel_alloc_coherent,
.free = intel_free_coherent,
.map_sg = intel_map_sg,
.map_page = intel_map_page,
.unmap_page = intel_unmap_page,
.mapping_error = intel_mapping_error,
-#ifdef CONFIG_X86
.dma_supported = dma_direct_supported,
-#endif
};
static inline int iommu_domain_cache_init(void)
help
This option enables support for the LEDs on the AAT1290.
+config LEDS_AN30259A
+ tristate "LED support for Panasonic AN30259A"
+ depends on LEDS_CLASS && I2C && OF
+ help
+ This option enables support for the AN30259A 3-channel
+ LED driver.
+
+ To compile this driver as a module, choose M here: the module
+ will be called leds-an30259a.
+
config LEDS_APU
tristate "Front panel LED support for PC Engines APU/APU2/APU3 boards"
depends on LEDS_CLASS
obj-$(CONFIG_LEDS_AAT1290) += leds-aat1290.o
obj-$(CONFIG_LEDS_APU) += leds-apu.o
obj-$(CONFIG_LEDS_AS3645A) += leds-as3645a.o
+obj-$(CONFIG_LEDS_AN30259A) += leds-an30259a.o
obj-$(CONFIG_LEDS_BCM6328) += leds-bcm6328.o
obj-$(CONFIG_LEDS_BCM6358) += leds-bcm6358.o
obj-$(CONFIG_LEDS_BD2802) += leds-bd2802.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0+
+//
+// Driver for Panasonic AN30259A 3-channel LED driver
+//
+// Copyright (c) 2018 Simon Shields <simon@lineageos.org>
+//
+// Datasheet:
+// https://www.alliedelec.com/m/d/a9d2b3ee87c2d1a535a41dd747b1c247.pdf
+
+#include <linux/i2c.h>
+#include <linux/leds.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/regmap.h>
+#include <uapi/linux/uleds.h>
+
+#define AN30259A_MAX_LEDS 3
+
+#define AN30259A_REG_SRESET 0x00
+#define AN30259A_LED_SRESET BIT(0)
+
+/* LED power registers */
+#define AN30259A_REG_LED_ON 0x01
+#define AN30259A_LED_EN(x) BIT((x) - 1)
+#define AN30259A_LED_SLOPE(x) BIT(((x) - 1) + 4)
+
+#define AN30259A_REG_LEDCC(x) (0x03 + ((x) - 1))
+
+/* slope control registers */
+#define AN30259A_REG_SLOPE(x) (0x06 + ((x) - 1))
+#define AN30259A_LED_SLOPETIME1(x) (x)
+#define AN30259A_LED_SLOPETIME2(x) ((x) << 4)
+
+#define AN30259A_REG_LEDCNT1(x) (0x09 + (4 * ((x) - 1)))
+#define AN30259A_LED_DUTYMAX(x) ((x) << 4)
+#define AN30259A_LED_DUTYMID(x) (x)
+
+#define AN30259A_REG_LEDCNT2(x) (0x0A + (4 * ((x) - 1)))
+#define AN30259A_LED_DELAY(x) ((x) << 4)
+#define AN30259A_LED_DUTYMIN(x) (x)
+
+/* detention time control (length of each slope step) */
+#define AN30259A_REG_LEDCNT3(x) (0x0B + (4 * ((x) - 1)))
+#define AN30259A_LED_DT1(x) (x)
+#define AN30259A_LED_DT2(x) ((x) << 4)
+
+#define AN30259A_REG_LEDCNT4(x) (0x0C + (4 * ((x) - 1)))
+#define AN30259A_LED_DT3(x) (x)
+#define AN30259A_LED_DT4(x) ((x) << 4)
+
+#define AN30259A_REG_MAX 0x14
+
+#define AN30259A_BLINK_MAX_TIME 7500 /* ms */
+#define AN30259A_SLOPE_RESOLUTION 500 /* ms */
+
+#define STATE_OFF 0
+#define STATE_KEEP 1
+#define STATE_ON 2
+
+struct an30259a;
+
+struct an30259a_led {
+ struct an30259a *chip;
+ struct led_classdev cdev;
+ u32 num;
+ u32 default_state;
+ bool sloping;
+ char label[LED_MAX_NAME_SIZE];
+};
+
+struct an30259a {
+ struct mutex mutex; /* held when writing to registers */
+ struct i2c_client *client;
+ struct an30259a_led leds[AN30259A_MAX_LEDS];
+ struct regmap *regmap;
+ int num_leds;
+};
+
+static int an30259a_brightness_set(struct led_classdev *cdev,
+ enum led_brightness brightness)
+{
+ struct an30259a_led *led;
+ int ret;
+ unsigned int led_on;
+
+ led = container_of(cdev, struct an30259a_led, cdev);
+ mutex_lock(&led->chip->mutex);
+
+ ret = regmap_read(led->chip->regmap, AN30259A_REG_LED_ON, &led_on);
+ if (ret)
+ goto error;
+
+ switch (brightness) {
+ case LED_OFF:
+ led_on &= ~AN30259A_LED_EN(led->num);
+ led_on &= ~AN30259A_LED_SLOPE(led->num);
+ led->sloping = false;
+ break;
+ default:
+ led_on |= AN30259A_LED_EN(led->num);
+ if (led->sloping)
+ led_on |= AN30259A_LED_SLOPE(led->num);
+ ret = regmap_write(led->chip->regmap,
+ AN30259A_REG_LEDCNT1(led->num),
+ AN30259A_LED_DUTYMAX(0xf) |
+ AN30259A_LED_DUTYMID(0xf));
+ if (ret)
+ goto error;
+ break;
+ }
+
+ ret = regmap_write(led->chip->regmap, AN30259A_REG_LED_ON, led_on);
+ if (ret)
+ goto error;
+
+ ret = regmap_write(led->chip->regmap, AN30259A_REG_LEDCC(led->num),
+ brightness);
+
+error:
+ mutex_unlock(&led->chip->mutex);
+
+ return ret;
+}
+
+static int an30259a_blink_set(struct led_classdev *cdev,
+ unsigned long *delay_off, unsigned long *delay_on)
+{
+ struct an30259a_led *led;
+ int ret, num;
+ unsigned int led_on;
+ unsigned long off = *delay_off, on = *delay_on;
+
+ led = container_of(cdev, struct an30259a_led, cdev);
+
+ mutex_lock(&led->chip->mutex);
+ num = led->num;
+
+ /* slope time can only be a multiple of 500ms. */
+ if (off % AN30259A_SLOPE_RESOLUTION || on % AN30259A_SLOPE_RESOLUTION) {
+ ret = -EINVAL;
+ goto error;
+ }
+
+ /* up to a maximum of 7500ms. */
+ if (off > AN30259A_BLINK_MAX_TIME || on > AN30259A_BLINK_MAX_TIME) {
+ ret = -EINVAL;
+ goto error;
+ }
+
+ /* if no blink specified, default to 1 Hz. */
+ if (!off && !on) {
+ *delay_off = off = 500;
+ *delay_on = on = 500;
+ }
+
+ /* convert into values the HW will understand. */
+ off /= AN30259A_SLOPE_RESOLUTION;
+ on /= AN30259A_SLOPE_RESOLUTION;
+
+ /* duty min should be zero (=off), delay should be zero. */
+ ret = regmap_write(led->chip->regmap, AN30259A_REG_LEDCNT2(num),
+ AN30259A_LED_DELAY(0) | AN30259A_LED_DUTYMIN(0));
+ if (ret)
+ goto error;
+
+ /* reset detention time (no "breathing" effect). */
+ ret = regmap_write(led->chip->regmap, AN30259A_REG_LEDCNT3(num),
+ AN30259A_LED_DT1(0) | AN30259A_LED_DT2(0));
+ if (ret)
+ goto error;
+ ret = regmap_write(led->chip->regmap, AN30259A_REG_LEDCNT4(num),
+ AN30259A_LED_DT3(0) | AN30259A_LED_DT4(0));
+ if (ret)
+ goto error;
+
+ /* slope time controls on/off cycle length. */
+ ret = regmap_write(led->chip->regmap, AN30259A_REG_SLOPE(num),
+ AN30259A_LED_SLOPETIME1(off) |
+ AN30259A_LED_SLOPETIME2(on));
+ if (ret)
+ goto error;
+
+ /* Finally, enable slope mode. */
+ ret = regmap_read(led->chip->regmap, AN30259A_REG_LED_ON, &led_on);
+ if (ret)
+ goto error;
+
+ led_on |= AN30259A_LED_SLOPE(num) | AN30259A_LED_EN(led->num);
+
+ ret = regmap_write(led->chip->regmap, AN30259A_REG_LED_ON, led_on);
+
+ if (!ret)
+ led->sloping = true;
+error:
+ mutex_unlock(&led->chip->mutex);
+
+ return ret;
+}
+
+static int an30259a_dt_init(struct i2c_client *client,
+ struct an30259a *chip)
+{
+ struct device_node *np = client->dev.of_node, *child;
+ int count, ret;
+ int i = 0;
+ const char *str;
+ struct an30259a_led *led;
+
+ count = of_get_child_count(np);
+ if (!count || count > AN30259A_MAX_LEDS)
+ return -EINVAL;
+
+ for_each_available_child_of_node(np, child) {
+ u32 source;
+
+ ret = of_property_read_u32(child, "reg", &source);
+ if (ret != 0 || !source || source > AN30259A_MAX_LEDS) {
+ dev_err(&client->dev, "Couldn't read LED address: %d\n",
+ ret);
+ count--;
+ continue;
+ }
+
+ led = &chip->leds[i];
+
+ led->num = source;
+ led->chip = chip;
+
+ if (of_property_read_string(child, "label", &str))
+ snprintf(led->label, sizeof(led->label), "an30259a::");
+ else
+ snprintf(led->label, sizeof(led->label), "an30259a:%s",
+ str);
+
+ led->cdev.name = led->label;
+
+ if (!of_property_read_string(child, "default-state", &str)) {
+ if (!strcmp(str, "on"))
+ led->default_state = STATE_ON;
+ else if (!strcmp(str, "keep"))
+ led->default_state = STATE_KEEP;
+ else
+ led->default_state = STATE_OFF;
+ }
+
+ of_property_read_string(child, "linux,default-trigger",
+ &led->cdev.default_trigger);
+
+ i++;
+ }
+
+ if (!count)
+ return -EINVAL;
+
+ chip->num_leds = i;
+
+ return 0;
+}
+
+static const struct regmap_config an30259a_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+ .max_register = AN30259A_REG_MAX,
+};
+
+static void an30259a_init_default_state(struct an30259a_led *led)
+{
+ struct an30259a *chip = led->chip;
+ int led_on, err;
+
+ switch (led->default_state) {
+ case STATE_ON:
+ led->cdev.brightness = LED_FULL;
+ break;
+ case STATE_KEEP:
+ err = regmap_read(chip->regmap, AN30259A_REG_LED_ON, &led_on);
+ if (err)
+ break;
+
+ if (!(led_on & AN30259A_LED_EN(led->num))) {
+ led->cdev.brightness = LED_OFF;
+ break;
+ }
+ regmap_read(chip->regmap, AN30259A_REG_LEDCC(led->num),
+ &led->cdev.brightness);
+ break;
+ default:
+ led->cdev.brightness = LED_OFF;
+ }
+
+ an30259a_brightness_set(&led->cdev, led->cdev.brightness);
+}
+
+static int an30259a_probe(struct i2c_client *client)
+{
+ struct an30259a *chip;
+ int i, err;
+
+ chip = devm_kzalloc(&client->dev, sizeof(*chip), GFP_KERNEL);
+ if (!chip)
+ return -ENOMEM;
+
+ err = an30259a_dt_init(client, chip);
+ if (err < 0)
+ return err;
+
+ mutex_init(&chip->mutex);
+ chip->client = client;
+ i2c_set_clientdata(client, chip);
+
+ chip->regmap = devm_regmap_init_i2c(client, &an30259a_regmap_config);
+
+ for (i = 0; i < chip->num_leds; i++) {
+ an30259a_init_default_state(&chip->leds[i]);
+ chip->leds[i].cdev.brightness_set_blocking =
+ an30259a_brightness_set;
+ chip->leds[i].cdev.blink_set = an30259a_blink_set;
+
+ err = devm_led_classdev_register(&client->dev,
+ &chip->leds[i].cdev);
+ if (err < 0)
+ goto exit;
+ }
+ return 0;
+
+exit:
+ mutex_destroy(&chip->mutex);
+ return err;
+}
+
+static int an30259a_remove(struct i2c_client *client)
+{
+ struct an30259a *chip = i2c_get_clientdata(client);
+
+ mutex_destroy(&chip->mutex);
+
+ return 0;
+}
+
+static const struct of_device_id an30259a_match_table[] = {
+ { .compatible = "panasonic,an30259a", },
+ { /* sentinel */ },
+};
+
+MODULE_DEVICE_TABLE(of, an30259a_match_table);
+
+static const struct i2c_device_id an30259a_id[] = {
+ { "an30259a", 0 },
+ { /* sentinel */ },
+};
+MODULE_DEVICE_TABLE(i2c, an30259a_id);
+
+static struct i2c_driver an30259a_driver = {
+ .driver = {
+ .name = "leds-an32059a",
+ .of_match_table = of_match_ptr(an30259a_match_table),
+ },
+ .probe_new = an30259a_probe,
+ .remove = an30259a_remove,
+ .id_table = an30259a_id,
+};
+
+module_i2c_driver(an30259a_driver);
+
+MODULE_AUTHOR("Simon Shields <simon@lineageos.org>");
+MODULE_DESCRIPTION("AN32059A LED driver");
+MODULE_LICENSE("GPL v2");
strlcpy(names->flash, name, sizeof(names->flash));
else
snprintf(names->flash, sizeof(names->flash),
- "%s:flash", node->name);
+ "%pOFn:flash", node);
rval = of_property_read_u32(flash->flash_node, "flash-timeout-us",
&cfg->flash_timeout_us);
strlcpy(names->indicator, name, sizeof(names->indicator));
else
snprintf(names->indicator, sizeof(names->indicator),
- "%s:indicator", node->name);
+ "%pOFn:indicator", node);
rval = of_property_read_u32(flash->indicator_node, "led-max-microamp",
&cfg->indicator_max_ua);
{
int ret, state;
- led_dat->gpiod = template->gpiod;
- if (!led_dat->gpiod) {
- /*
- * This is the legacy code path for platform code that
- * still uses GPIO numbers. Ultimately we would like to get
- * rid of this block completely.
- */
- unsigned long flags = GPIOF_OUT_INIT_LOW;
-
- /* skip leds that aren't available */
- if (!gpio_is_valid(template->gpio)) {
- dev_info(parent, "Skipping unavailable LED gpio %d (%s)\n",
- template->gpio, template->name);
- return 0;
- }
-
- if (template->active_low)
- flags |= GPIOF_ACTIVE_LOW;
-
- ret = devm_gpio_request_one(parent, template->gpio, flags,
- template->name);
- if (ret < 0)
- return ret;
-
- led_dat->gpiod = gpio_to_desc(template->gpio);
- if (!led_dat->gpiod)
- return -EINVAL;
- }
-
led_dat->cdev.name = template->name;
led_dat->cdev.default_trigger = template->default_trigger;
led_dat->can_sleep = gpiod_cansleep(led_dat->gpiod);
MODULE_DEVICE_TABLE(of, of_gpio_leds_match);
+static struct gpio_desc *gpio_led_get_gpiod(struct device *dev, int idx,
+ const struct gpio_led *template)
+{
+ struct gpio_desc *gpiod;
+ unsigned long flags = GPIOF_OUT_INIT_LOW;
+ int ret;
+
+ /*
+ * This means the LED does not come from the device tree
+ * or ACPI, so let's try just getting it by index from the
+ * device, this will hit the board file, if any and get
+ * the GPIO from there.
+ */
+ gpiod = devm_gpiod_get_index(dev, NULL, idx, flags);
+ if (!IS_ERR(gpiod)) {
+ gpiod_set_consumer_name(gpiod, template->name);
+ return gpiod;
+ }
+ if (PTR_ERR(gpiod) != -ENOENT)
+ return gpiod;
+
+ /*
+ * This is the legacy code path for platform code that
+ * still uses GPIO numbers. Ultimately we would like to get
+ * rid of this block completely.
+ */
+
+ /* skip leds that aren't available */
+ if (!gpio_is_valid(template->gpio))
+ return ERR_PTR(-ENOENT);
+
+ if (template->active_low)
+ flags |= GPIOF_ACTIVE_LOW;
+
+ ret = devm_gpio_request_one(dev, template->gpio, flags,
+ template->name);
+ if (ret < 0)
+ return ERR_PTR(ret);
+
+ gpiod = gpio_to_desc(template->gpio);
+ if (!gpiod)
+ return ERR_PTR(-EINVAL);
+
+ return gpiod;
+}
+
static int gpio_led_probe(struct platform_device *pdev)
{
struct gpio_led_platform_data *pdata = dev_get_platdata(&pdev->dev);
priv->num_leds = pdata->num_leds;
for (i = 0; i < priv->num_leds; i++) {
- ret = create_gpio_led(&pdata->leds[i], &priv->leds[i],
+ const struct gpio_led *template = &pdata->leds[i];
+ struct gpio_led_data *led_dat = &priv->leds[i];
+
+ if (template->gpiod)
+ led_dat->gpiod = template->gpiod;
+ else
+ led_dat->gpiod =
+ gpio_led_get_gpiod(&pdev->dev,
+ i, template);
+ if (IS_ERR(led_dat->gpiod)) {
+ dev_info(&pdev->dev, "Skipping unavailable LED gpio %d (%s)\n",
+ template->gpio, template->name);
+ continue;
+ }
+
+ ret = create_gpio_led(template, led_dat,
&pdev->dev, NULL,
pdata->gpio_blink_set);
if (ret < 0)
led_data->pwm = devm_pwm_get(dev, led->name);
if (IS_ERR(led_data->pwm)) {
ret = PTR_ERR(led_data->pwm);
- dev_err(dev, "unable to request PWM for %s: %d\n",
- led->name, ret);
+ if (ret != -EPROBE_DEFER)
+ dev_err(dev, "unable to request PWM for %s: %d\n",
+ led->name, ret);
return ret;
}
#define SC27XX_DUTY_MASK GENMASK(15, 0)
#define SC27XX_MOD_MASK GENMASK(7, 0)
+#define SC27XX_CURVE_SHIFT 8
+#define SC27XX_CURVE_L_MASK GENMASK(7, 0)
+#define SC27XX_CURVE_H_MASK GENMASK(15, 8)
+
#define SC27XX_LEDS_OFFSET 0x10
#define SC27XX_LEDS_MAX 3
+#define SC27XX_LEDS_PATTERN_CNT 4
+/* Stage duration step, in milliseconds */
+#define SC27XX_LEDS_STEP 125
+/* Minimum and maximum duration, in milliseconds */
+#define SC27XX_DELTA_T_MIN SC27XX_LEDS_STEP
+#define SC27XX_DELTA_T_MAX (SC27XX_LEDS_STEP * 255)
struct sc27xx_led {
char name[LED_MAX_NAME_SIZE];
return err;
}
+static void sc27xx_led_clamp_align_delta_t(u32 *delta_t)
+{
+ u32 v, offset, t = *delta_t;
+
+ v = t + SC27XX_LEDS_STEP / 2;
+ v = clamp_t(u32, v, SC27XX_DELTA_T_MIN, SC27XX_DELTA_T_MAX);
+ offset = v - SC27XX_DELTA_T_MIN;
+ offset = SC27XX_LEDS_STEP * (offset / SC27XX_LEDS_STEP);
+
+ *delta_t = SC27XX_DELTA_T_MIN + offset;
+}
+
+static int sc27xx_led_pattern_clear(struct led_classdev *ldev)
+{
+ struct sc27xx_led *leds = to_sc27xx_led(ldev);
+ struct regmap *regmap = leds->priv->regmap;
+ u32 base = sc27xx_led_get_offset(leds);
+ u32 ctrl_base = leds->priv->base + SC27XX_LEDS_CTRL;
+ u8 ctrl_shift = SC27XX_CTRL_SHIFT * leds->line;
+ int err;
+
+ mutex_lock(&leds->priv->lock);
+
+ /* Reset the rise, high, fall and low time to zero. */
+ regmap_write(regmap, base + SC27XX_LEDS_CURVE0, 0);
+ regmap_write(regmap, base + SC27XX_LEDS_CURVE1, 0);
+
+ err = regmap_update_bits(regmap, ctrl_base,
+ (SC27XX_LED_RUN | SC27XX_LED_TYPE) << ctrl_shift, 0);
+
+ ldev->brightness = LED_OFF;
+
+ mutex_unlock(&leds->priv->lock);
+
+ return err;
+}
+
+static int sc27xx_led_pattern_set(struct led_classdev *ldev,
+ struct led_pattern *pattern,
+ u32 len, int repeat)
+{
+ struct sc27xx_led *leds = to_sc27xx_led(ldev);
+ u32 base = sc27xx_led_get_offset(leds);
+ u32 ctrl_base = leds->priv->base + SC27XX_LEDS_CTRL;
+ u8 ctrl_shift = SC27XX_CTRL_SHIFT * leds->line;
+ struct regmap *regmap = leds->priv->regmap;
+ int err;
+
+ /*
+ * Must contain 4 tuples to configure the rise time, high time, fall
+ * time and low time to enable the breathing mode.
+ */
+ if (len != SC27XX_LEDS_PATTERN_CNT)
+ return -EINVAL;
+
+ mutex_lock(&leds->priv->lock);
+
+ sc27xx_led_clamp_align_delta_t(&pattern[0].delta_t);
+ err = regmap_update_bits(regmap, base + SC27XX_LEDS_CURVE0,
+ SC27XX_CURVE_L_MASK,
+ pattern[0].delta_t / SC27XX_LEDS_STEP);
+ if (err)
+ goto out;
+
+ sc27xx_led_clamp_align_delta_t(&pattern[1].delta_t);
+ err = regmap_update_bits(regmap, base + SC27XX_LEDS_CURVE1,
+ SC27XX_CURVE_L_MASK,
+ pattern[1].delta_t / SC27XX_LEDS_STEP);
+ if (err)
+ goto out;
+
+ sc27xx_led_clamp_align_delta_t(&pattern[2].delta_t);
+ err = regmap_update_bits(regmap, base + SC27XX_LEDS_CURVE0,
+ SC27XX_CURVE_H_MASK,
+ (pattern[2].delta_t / SC27XX_LEDS_STEP) <<
+ SC27XX_CURVE_SHIFT);
+ if (err)
+ goto out;
+
+ sc27xx_led_clamp_align_delta_t(&pattern[3].delta_t);
+ err = regmap_update_bits(regmap, base + SC27XX_LEDS_CURVE1,
+ SC27XX_CURVE_H_MASK,
+ (pattern[3].delta_t / SC27XX_LEDS_STEP) <<
+ SC27XX_CURVE_SHIFT);
+ if (err)
+ goto out;
+
+ err = regmap_update_bits(regmap, base + SC27XX_LEDS_DUTY,
+ SC27XX_DUTY_MASK,
+ (pattern[1].brightness << SC27XX_DUTY_SHIFT) |
+ SC27XX_MOD_MASK);
+ if (err)
+ goto out;
+
+ /* Enable the LED breathing mode */
+ err = regmap_update_bits(regmap, ctrl_base,
+ SC27XX_LED_RUN << ctrl_shift,
+ SC27XX_LED_RUN << ctrl_shift);
+ if (!err)
+ ldev->brightness = pattern[1].brightness;
+
+out:
+ mutex_unlock(&leds->priv->lock);
+
+ return err;
+}
+
static int sc27xx_led_register(struct device *dev, struct sc27xx_led_priv *priv)
{
int i, err;
led->priv = priv;
led->ldev.name = led->name;
led->ldev.brightness_set_blocking = sc27xx_led_set;
+ led->ldev.pattern_set = sc27xx_led_pattern_set;
+ led->ldev.pattern_clear = sc27xx_led_pattern_clear;
+ led->ldev.default_trigger = "pattern";
err = devm_led_classdev_register(dev, &led->ldev);
if (err)
MODULE_DESCRIPTION("Spreadtrum SC27xx breathing light controller driver");
MODULE_AUTHOR("Xiaotong Lu <xiaotong.lu@spreadtrum.com>");
+MODULE_AUTHOR("Baolin Wang <baolin.wang@linaro.org>");
MODULE_LICENSE("GPL v2");
This allows LEDs to be controlled by network device activity.
If unsure, say Y.
+config LEDS_TRIGGER_PATTERN
+ tristate "LED Pattern Trigger"
+ help
+ This allows LEDs to be controlled by a software or hardware pattern
+ which is a series of tuples, of brightness and duration (ms).
+ If unsure, say N
+
endif # LEDS_TRIGGERS
obj-$(CONFIG_LEDS_TRIGGER_CAMERA) += ledtrig-camera.o
obj-$(CONFIG_LEDS_TRIGGER_PANIC) += ledtrig-panic.o
obj-$(CONFIG_LEDS_TRIGGER_NETDEV) += ledtrig-netdev.o
+obj-$(CONFIG_LEDS_TRIGGER_PATTERN) += ledtrig-pattern.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * LED pattern trigger
+ *
+ * Idea discussed with Pavel Machek. Raphael Teysseyre implemented
+ * the first version, Baolin Wang simplified and improved the approach.
+ */
+
+#include <linux/kernel.h>
+#include <linux/leds.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+#include <linux/timer.h>
+
+#define MAX_PATTERNS 1024
+/*
+ * When doing gradual dimming, the led brightness will be updated
+ * every 50 milliseconds.
+ */
+#define UPDATE_INTERVAL 50
+
+struct pattern_trig_data {
+ struct led_classdev *led_cdev;
+ struct led_pattern patterns[MAX_PATTERNS];
+ struct led_pattern *curr;
+ struct led_pattern *next;
+ struct mutex lock;
+ u32 npatterns;
+ int repeat;
+ int last_repeat;
+ int delta_t;
+ bool is_indefinite;
+ bool is_hw_pattern;
+ struct timer_list timer;
+};
+
+static void pattern_trig_update_patterns(struct pattern_trig_data *data)
+{
+ data->curr = data->next;
+ if (!data->is_indefinite && data->curr == data->patterns)
+ data->repeat--;
+
+ if (data->next == data->patterns + data->npatterns - 1)
+ data->next = data->patterns;
+ else
+ data->next++;
+
+ data->delta_t = 0;
+}
+
+static int pattern_trig_compute_brightness(struct pattern_trig_data *data)
+{
+ int step_brightness;
+
+ /*
+ * If current tuple's duration is less than the dimming interval,
+ * we should treat it as a step change of brightness instead of
+ * doing gradual dimming.
+ */
+ if (data->delta_t == 0 || data->curr->delta_t < UPDATE_INTERVAL)
+ return data->curr->brightness;
+
+ step_brightness = abs(data->next->brightness - data->curr->brightness);
+ step_brightness = data->delta_t * step_brightness / data->curr->delta_t;
+
+ if (data->next->brightness > data->curr->brightness)
+ return data->curr->brightness + step_brightness;
+ else
+ return data->curr->brightness - step_brightness;
+}
+
+static void pattern_trig_timer_function(struct timer_list *t)
+{
+ struct pattern_trig_data *data = from_timer(data, t, timer);
+
+ mutex_lock(&data->lock);
+
+ for (;;) {
+ if (!data->is_indefinite && !data->repeat)
+ break;
+
+ if (data->curr->brightness == data->next->brightness) {
+ /* Step change of brightness */
+ led_set_brightness(data->led_cdev,
+ data->curr->brightness);
+ mod_timer(&data->timer,
+ jiffies + msecs_to_jiffies(data->curr->delta_t));
+
+ /* Skip the tuple with zero duration */
+ pattern_trig_update_patterns(data);
+ /* Select next tuple */
+ pattern_trig_update_patterns(data);
+ } else {
+ /* Gradual dimming */
+
+ /*
+ * If the accumulation time is larger than current
+ * tuple's duration, we should go next one and re-check
+ * if we repeated done.
+ */
+ if (data->delta_t > data->curr->delta_t) {
+ pattern_trig_update_patterns(data);
+ continue;
+ }
+
+ led_set_brightness(data->led_cdev,
+ pattern_trig_compute_brightness(data));
+ mod_timer(&data->timer,
+ jiffies + msecs_to_jiffies(UPDATE_INTERVAL));
+
+ /* Accumulate the gradual dimming time */
+ data->delta_t += UPDATE_INTERVAL;
+ }
+
+ break;
+ }
+
+ mutex_unlock(&data->lock);
+}
+
+static int pattern_trig_start_pattern(struct led_classdev *led_cdev)
+{
+ struct pattern_trig_data *data = led_cdev->trigger_data;
+
+ if (!data->npatterns)
+ return 0;
+
+ if (data->is_hw_pattern) {
+ return led_cdev->pattern_set(led_cdev, data->patterns,
+ data->npatterns, data->repeat);
+ }
+
+ /* At least 2 tuples for software pattern. */
+ if (data->npatterns < 2)
+ return -EINVAL;
+
+ data->delta_t = 0;
+ data->curr = data->patterns;
+ data->next = data->patterns + 1;
+ data->timer.expires = jiffies;
+ add_timer(&data->timer);
+
+ return 0;
+}
+
+static ssize_t repeat_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct led_classdev *led_cdev = dev_get_drvdata(dev);
+ struct pattern_trig_data *data = led_cdev->trigger_data;
+ int repeat;
+
+ mutex_lock(&data->lock);
+
+ repeat = data->last_repeat;
+
+ mutex_unlock(&data->lock);
+
+ return scnprintf(buf, PAGE_SIZE, "%d\n", repeat);
+}
+
+static ssize_t repeat_store(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct led_classdev *led_cdev = dev_get_drvdata(dev);
+ struct pattern_trig_data *data = led_cdev->trigger_data;
+ int err, res;
+
+ err = kstrtos32(buf, 10, &res);
+ if (err)
+ return err;
+
+ /* Number 0 and negative numbers except -1 are invalid. */
+ if (res < -1 || res == 0)
+ return -EINVAL;
+
+ /*
+ * Clear previous patterns' performence firstly, and remove the timer
+ * without mutex lock to avoid dead lock.
+ */
+ del_timer_sync(&data->timer);
+
+ mutex_lock(&data->lock);
+
+ if (data->is_hw_pattern)
+ led_cdev->pattern_clear(led_cdev);
+
+ data->last_repeat = data->repeat = res;
+ /* -1 means repeat indefinitely */
+ if (data->repeat == -1)
+ data->is_indefinite = true;
+ else
+ data->is_indefinite = false;
+
+ err = pattern_trig_start_pattern(led_cdev);
+
+ mutex_unlock(&data->lock);
+ return err < 0 ? err : count;
+}
+
+static DEVICE_ATTR_RW(repeat);
+
+static ssize_t pattern_trig_show_patterns(struct pattern_trig_data *data,
+ char *buf, bool hw_pattern)
+{
+ ssize_t count = 0;
+ int i;
+
+ mutex_lock(&data->lock);
+
+ if (!data->npatterns || (data->is_hw_pattern ^ hw_pattern))
+ goto out;
+
+ for (i = 0; i < data->npatterns; i++) {
+ count += scnprintf(buf + count, PAGE_SIZE - count,
+ "%d %u ",
+ data->patterns[i].brightness,
+ data->patterns[i].delta_t);
+ }
+
+ buf[count - 1] = '\n';
+
+out:
+ mutex_unlock(&data->lock);
+ return count;
+}
+
+static ssize_t pattern_trig_store_patterns(struct led_classdev *led_cdev,
+ const char *buf, size_t count,
+ bool hw_pattern)
+{
+ struct pattern_trig_data *data = led_cdev->trigger_data;
+ int ccount, cr, offset = 0, err = 0;
+
+ /*
+ * Clear previous patterns' performence firstly, and remove the timer
+ * without mutex lock to avoid dead lock.
+ */
+ del_timer_sync(&data->timer);
+
+ mutex_lock(&data->lock);
+
+ if (data->is_hw_pattern)
+ led_cdev->pattern_clear(led_cdev);
+
+ data->is_hw_pattern = hw_pattern;
+ data->npatterns = 0;
+
+ while (offset < count - 1 && data->npatterns < MAX_PATTERNS) {
+ cr = 0;
+ ccount = sscanf(buf + offset, "%d %u %n",
+ &data->patterns[data->npatterns].brightness,
+ &data->patterns[data->npatterns].delta_t, &cr);
+ if (ccount != 2) {
+ data->npatterns = 0;
+ err = -EINVAL;
+ goto out;
+ }
+
+ offset += cr;
+ data->npatterns++;
+ }
+
+ err = pattern_trig_start_pattern(led_cdev);
+ if (err)
+ data->npatterns = 0;
+
+out:
+ mutex_unlock(&data->lock);
+ return err < 0 ? err : count;
+}
+
+static ssize_t pattern_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct led_classdev *led_cdev = dev_get_drvdata(dev);
+ struct pattern_trig_data *data = led_cdev->trigger_data;
+
+ return pattern_trig_show_patterns(data, buf, false);
+}
+
+static ssize_t pattern_store(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct led_classdev *led_cdev = dev_get_drvdata(dev);
+
+ return pattern_trig_store_patterns(led_cdev, buf, count, false);
+}
+
+static DEVICE_ATTR_RW(pattern);
+
+static ssize_t hw_pattern_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct led_classdev *led_cdev = dev_get_drvdata(dev);
+ struct pattern_trig_data *data = led_cdev->trigger_data;
+
+ return pattern_trig_show_patterns(data, buf, true);
+}
+
+static ssize_t hw_pattern_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct led_classdev *led_cdev = dev_get_drvdata(dev);
+
+ return pattern_trig_store_patterns(led_cdev, buf, count, true);
+}
+
+static DEVICE_ATTR_RW(hw_pattern);
+
+static umode_t pattern_trig_attrs_mode(struct kobject *kobj,
+ struct attribute *attr, int index)
+{
+ struct device *dev = container_of(kobj, struct device, kobj);
+ struct led_classdev *led_cdev = dev_get_drvdata(dev);
+
+ if (attr == &dev_attr_repeat.attr || attr == &dev_attr_pattern.attr)
+ return attr->mode;
+ else if (attr == &dev_attr_hw_pattern.attr && led_cdev->pattern_set)
+ return attr->mode;
+
+ return 0;
+}
+
+static struct attribute *pattern_trig_attrs[] = {
+ &dev_attr_pattern.attr,
+ &dev_attr_hw_pattern.attr,
+ &dev_attr_repeat.attr,
+ NULL
+};
+
+static const struct attribute_group pattern_trig_group = {
+ .attrs = pattern_trig_attrs,
+ .is_visible = pattern_trig_attrs_mode,
+};
+
+static const struct attribute_group *pattern_trig_groups[] = {
+ &pattern_trig_group,
+ NULL,
+};
+
+static int pattern_trig_activate(struct led_classdev *led_cdev)
+{
+ struct pattern_trig_data *data;
+
+ data = kzalloc(sizeof(*data), GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
+
+ if (!!led_cdev->pattern_set ^ !!led_cdev->pattern_clear) {
+ dev_warn(led_cdev->dev,
+ "Hardware pattern ops validation failed\n");
+ led_cdev->pattern_set = NULL;
+ led_cdev->pattern_clear = NULL;
+ }
+
+ data->is_indefinite = true;
+ data->last_repeat = -1;
+ mutex_init(&data->lock);
+ data->led_cdev = led_cdev;
+ led_set_trigger_data(led_cdev, data);
+ timer_setup(&data->timer, pattern_trig_timer_function, 0);
+ led_cdev->activated = true;
+
+ return 0;
+}
+
+static void pattern_trig_deactivate(struct led_classdev *led_cdev)
+{
+ struct pattern_trig_data *data = led_cdev->trigger_data;
+
+ if (!led_cdev->activated)
+ return;
+
+ if (led_cdev->pattern_clear)
+ led_cdev->pattern_clear(led_cdev);
+
+ del_timer_sync(&data->timer);
+
+ led_set_brightness(led_cdev, LED_OFF);
+ kfree(data);
+ led_cdev->activated = false;
+}
+
+static struct led_trigger pattern_led_trigger = {
+ .name = "pattern",
+ .activate = pattern_trig_activate,
+ .deactivate = pattern_trig_deactivate,
+ .groups = pattern_trig_groups,
+};
+
+static int __init pattern_trig_init(void)
+{
+ return led_trigger_register(&pattern_led_trigger);
+}
+
+static void __exit pattern_trig_exit(void)
+{
+ led_trigger_unregister(&pattern_led_trigger);
+}
+
+module_init(pattern_trig_init);
+module_exit(pattern_trig_exit);
+
+MODULE_AUTHOR("Raphael Teysseyre <rteysseyre@gmail.com");
+MODULE_AUTHOR("Baolin Wang <baolin.wang@linaro.org");
+MODULE_DESCRIPTION("LED Pattern trigger");
+MODULE_LICENSE("GPL v2");
menuconfig NVM
bool "Open-Channel SSD target support"
- depends on BLOCK && PCI
- select BLK_DEV_NVME
+ depends on BLOCK
help
Say Y here to get to enable Open-channel SSDs.
return -EINVAL;
}
+ if ((tt->flags & NVM_TGT_F_HOST_L2P) != (dev->geo.dom & NVM_RSP_L2P)) {
+ pr_err("nvm: device is incompatible with target L2P type.\n");
+ return -EINVAL;
+ }
+
if (nvm_target_exists(create->tgtname)) {
pr_err("nvm: target name already exists (%s)\n",
create->tgtname);
static void nvm_rq_tgt_to_dev(struct nvm_tgt_dev *tgt_dev, struct nvm_rq *rqd)
{
- if (rqd->nr_ppas == 1) {
- nvm_ppa_tgt_to_dev(tgt_dev, &rqd->ppa_addr, 1);
- return;
- }
+ struct ppa_addr *ppa_list = nvm_rq_to_ppa_list(rqd);
- nvm_ppa_tgt_to_dev(tgt_dev, rqd->ppa_list, rqd->nr_ppas);
+ nvm_ppa_tgt_to_dev(tgt_dev, ppa_list, rqd->nr_ppas);
}
static void nvm_rq_dev_to_tgt(struct nvm_tgt_dev *tgt_dev, struct nvm_rq *rqd)
{
- if (rqd->nr_ppas == 1) {
- nvm_ppa_dev_to_tgt(tgt_dev, &rqd->ppa_addr, 1);
- return;
- }
+ struct ppa_addr *ppa_list = nvm_rq_to_ppa_list(rqd);
- nvm_ppa_dev_to_tgt(tgt_dev, rqd->ppa_list, rqd->nr_ppas);
+ nvm_ppa_dev_to_tgt(tgt_dev, ppa_list, rqd->nr_ppas);
}
int nvm_register_tgt_type(struct nvm_tgt_type *tt)
nvm_dev_dma_free(tgt_dev->parent, rqd->ppa_list, rqd->dma_ppa_list);
}
-int nvm_get_chunk_meta(struct nvm_tgt_dev *tgt_dev, struct nvm_chk_meta *meta,
- struct ppa_addr ppa, int nchks)
+static int nvm_set_flags(struct nvm_geo *geo, struct nvm_rq *rqd)
{
- struct nvm_dev *dev = tgt_dev->parent;
-
- nvm_ppa_tgt_to_dev(tgt_dev, &ppa, 1);
-
- return dev->ops->get_chk_meta(tgt_dev->parent, meta,
- (sector_t)ppa.ppa, nchks);
-}
-EXPORT_SYMBOL(nvm_get_chunk_meta);
-
-int nvm_set_tgt_bb_tbl(struct nvm_tgt_dev *tgt_dev, struct ppa_addr *ppas,
- int nr_ppas, int type)
-{
- struct nvm_dev *dev = tgt_dev->parent;
- struct nvm_rq rqd;
- int ret;
+ int flags = 0;
- if (nr_ppas > NVM_MAX_VLBA) {
- pr_err("nvm: unable to update all blocks atomically\n");
- return -EINVAL;
- }
+ if (geo->version == NVM_OCSSD_SPEC_20)
+ return 0;
- memset(&rqd, 0, sizeof(struct nvm_rq));
+ if (rqd->is_seq)
+ flags |= geo->pln_mode >> 1;
- nvm_set_rqd_ppalist(tgt_dev, &rqd, ppas, nr_ppas);
- nvm_rq_tgt_to_dev(tgt_dev, &rqd);
+ if (rqd->opcode == NVM_OP_PREAD)
+ flags |= (NVM_IO_SCRAMBLE_ENABLE | NVM_IO_SUSPEND);
+ else if (rqd->opcode == NVM_OP_PWRITE)
+ flags |= NVM_IO_SCRAMBLE_ENABLE;
- ret = dev->ops->set_bb_tbl(dev, &rqd.ppa_addr, rqd.nr_ppas, type);
- nvm_free_rqd_ppalist(tgt_dev, &rqd);
- if (ret) {
- pr_err("nvm: failed bb mark\n");
- return -EINVAL;
- }
-
- return 0;
+ return flags;
}
-EXPORT_SYMBOL(nvm_set_tgt_bb_tbl);
int nvm_submit_io(struct nvm_tgt_dev *tgt_dev, struct nvm_rq *rqd)
{
nvm_rq_tgt_to_dev(tgt_dev, rqd);
rqd->dev = tgt_dev;
+ rqd->flags = nvm_set_flags(&tgt_dev->geo, rqd);
/* In case of error, fail with right address format */
ret = dev->ops->submit_io(dev, rqd);
nvm_rq_tgt_to_dev(tgt_dev, rqd);
rqd->dev = tgt_dev;
+ rqd->flags = nvm_set_flags(&tgt_dev->geo, rqd);
/* In case of error, fail with right address format */
ret = dev->ops->submit_io_sync(dev, rqd);
}
EXPORT_SYMBOL(nvm_end_io);
+static int nvm_submit_io_sync_raw(struct nvm_dev *dev, struct nvm_rq *rqd)
+{
+ if (!dev->ops->submit_io_sync)
+ return -ENODEV;
+
+ rqd->flags = nvm_set_flags(&dev->geo, rqd);
+
+ return dev->ops->submit_io_sync(dev, rqd);
+}
+
+static int nvm_bb_chunk_sense(struct nvm_dev *dev, struct ppa_addr ppa)
+{
+ struct nvm_rq rqd = { NULL };
+ struct bio bio;
+ struct bio_vec bio_vec;
+ struct page *page;
+ int ret;
+
+ page = alloc_page(GFP_KERNEL);
+ if (!page)
+ return -ENOMEM;
+
+ bio_init(&bio, &bio_vec, 1);
+ bio_add_page(&bio, page, PAGE_SIZE, 0);
+ bio_set_op_attrs(&bio, REQ_OP_READ, 0);
+
+ rqd.bio = &bio;
+ rqd.opcode = NVM_OP_PREAD;
+ rqd.is_seq = 1;
+ rqd.nr_ppas = 1;
+ rqd.ppa_addr = generic_to_dev_addr(dev, ppa);
+
+ ret = nvm_submit_io_sync_raw(dev, &rqd);
+ if (ret)
+ return ret;
+
+ __free_page(page);
+
+ return rqd.error;
+}
+
/*
- * folds a bad block list from its plane representation to its virtual
- * block representation. The fold is done in place and reduced size is
- * returned.
- *
- * If any of the planes status are bad or grown bad block, the virtual block
- * is marked bad. If not bad, the first plane state acts as the block state.
+ * Scans a 1.2 chunk first and last page to determine if its state.
+ * If the chunk is found to be open, also scan it to update the write
+ * pointer.
*/
-int nvm_bb_tbl_fold(struct nvm_dev *dev, u8 *blks, int nr_blks)
+static int nvm_bb_chunk_scan(struct nvm_dev *dev, struct ppa_addr ppa,
+ struct nvm_chk_meta *meta)
{
struct nvm_geo *geo = &dev->geo;
- int blk, offset, pl, blktype;
+ int ret, pg, pl;
- if (nr_blks != geo->num_chk * geo->pln_mode)
- return -EINVAL;
+ /* sense first page */
+ ret = nvm_bb_chunk_sense(dev, ppa);
+ if (ret < 0) /* io error */
+ return ret;
+ else if (ret == 0) /* valid data */
+ meta->state = NVM_CHK_ST_OPEN;
+ else if (ret > 0) {
+ /*
+ * If empty page, the chunk is free, else it is an
+ * actual io error. In that case, mark it offline.
+ */
+ switch (ret) {
+ case NVM_RSP_ERR_EMPTYPAGE:
+ meta->state = NVM_CHK_ST_FREE;
+ return 0;
+ case NVM_RSP_ERR_FAILCRC:
+ case NVM_RSP_ERR_FAILECC:
+ case NVM_RSP_WARN_HIGHECC:
+ meta->state = NVM_CHK_ST_OPEN;
+ goto scan;
+ default:
+ return -ret; /* other io error */
+ }
+ }
+
+ /* sense last page */
+ ppa.g.pg = geo->num_pg - 1;
+ ppa.g.pl = geo->num_pln - 1;
+
+ ret = nvm_bb_chunk_sense(dev, ppa);
+ if (ret < 0) /* io error */
+ return ret;
+ else if (ret == 0) { /* Chunk fully written */
+ meta->state = NVM_CHK_ST_CLOSED;
+ meta->wp = geo->clba;
+ return 0;
+ } else if (ret > 0) {
+ switch (ret) {
+ case NVM_RSP_ERR_EMPTYPAGE:
+ case NVM_RSP_ERR_FAILCRC:
+ case NVM_RSP_ERR_FAILECC:
+ case NVM_RSP_WARN_HIGHECC:
+ meta->state = NVM_CHK_ST_OPEN;
+ break;
+ default:
+ return -ret; /* other io error */
+ }
+ }
+
+scan:
+ /*
+ * chunk is open, we scan sequentially to update the write pointer.
+ * We make the assumption that targets write data across all planes
+ * before moving to the next page.
+ */
+ for (pg = 0; pg < geo->num_pg; pg++) {
+ for (pl = 0; pl < geo->num_pln; pl++) {
+ ppa.g.pg = pg;
+ ppa.g.pl = pl;
+
+ ret = nvm_bb_chunk_sense(dev, ppa);
+ if (ret < 0) /* io error */
+ return ret;
+ else if (ret == 0) {
+ meta->wp += geo->ws_min;
+ } else if (ret > 0) {
+ switch (ret) {
+ case NVM_RSP_ERR_EMPTYPAGE:
+ return 0;
+ case NVM_RSP_ERR_FAILCRC:
+ case NVM_RSP_ERR_FAILECC:
+ case NVM_RSP_WARN_HIGHECC:
+ meta->wp += geo->ws_min;
+ break;
+ default:
+ return -ret; /* other io error */
+ }
+ }
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * folds a bad block list from its plane representation to its
+ * chunk representation.
+ *
+ * If any of the planes status are bad or grown bad, the chunk is marked
+ * offline. If not bad, the first plane state acts as the chunk state.
+ */
+static int nvm_bb_to_chunk(struct nvm_dev *dev, struct ppa_addr ppa,
+ u8 *blks, int nr_blks, struct nvm_chk_meta *meta)
+{
+ struct nvm_geo *geo = &dev->geo;
+ int ret, blk, pl, offset, blktype;
for (blk = 0; blk < geo->num_chk; blk++) {
offset = blk * geo->pln_mode;
blktype = blks[offset];
- /* Bad blocks on any planes take precedence over other types */
for (pl = 0; pl < geo->pln_mode; pl++) {
if (blks[offset + pl] &
(NVM_BLK_T_BAD|NVM_BLK_T_GRWN_BAD)) {
}
}
- blks[blk] = blktype;
+ ppa.g.blk = blk;
+
+ meta->wp = 0;
+ meta->type = NVM_CHK_TP_W_SEQ;
+ meta->wi = 0;
+ meta->slba = generic_to_dev_addr(dev, ppa).ppa;
+ meta->cnlb = dev->geo.clba;
+
+ if (blktype == NVM_BLK_T_FREE) {
+ ret = nvm_bb_chunk_scan(dev, ppa, meta);
+ if (ret)
+ return ret;
+ } else {
+ meta->state = NVM_CHK_ST_OFFLINE;
+ }
+
+ meta++;
}
- return geo->num_chk;
+ return 0;
+}
+
+static int nvm_get_bb_meta(struct nvm_dev *dev, sector_t slba,
+ int nchks, struct nvm_chk_meta *meta)
+{
+ struct nvm_geo *geo = &dev->geo;
+ struct ppa_addr ppa;
+ u8 *blks;
+ int ch, lun, nr_blks;
+ int ret;
+
+ ppa.ppa = slba;
+ ppa = dev_to_generic_addr(dev, ppa);
+
+ if (ppa.g.blk != 0)
+ return -EINVAL;
+
+ if ((nchks % geo->num_chk) != 0)
+ return -EINVAL;
+
+ nr_blks = geo->num_chk * geo->pln_mode;
+
+ blks = kmalloc(nr_blks, GFP_KERNEL);
+ if (!blks)
+ return -ENOMEM;
+
+ for (ch = ppa.g.ch; ch < geo->num_ch; ch++) {
+ for (lun = ppa.g.lun; lun < geo->num_lun; lun++) {
+ struct ppa_addr ppa_gen, ppa_dev;
+
+ if (!nchks)
+ goto done;
+
+ ppa_gen.ppa = 0;
+ ppa_gen.g.ch = ch;
+ ppa_gen.g.lun = lun;
+ ppa_dev = generic_to_dev_addr(dev, ppa_gen);
+
+ ret = dev->ops->get_bb_tbl(dev, ppa_dev, blks);
+ if (ret)
+ goto done;
+
+ ret = nvm_bb_to_chunk(dev, ppa_gen, blks, nr_blks,
+ meta);
+ if (ret)
+ goto done;
+
+ meta += geo->num_chk;
+ nchks -= geo->num_chk;
+ }
+ }
+done:
+ kfree(blks);
+ return ret;
}
-EXPORT_SYMBOL(nvm_bb_tbl_fold);
-int nvm_get_tgt_bb_tbl(struct nvm_tgt_dev *tgt_dev, struct ppa_addr ppa,
- u8 *blks)
+int nvm_get_chunk_meta(struct nvm_tgt_dev *tgt_dev, struct ppa_addr ppa,
+ int nchks, struct nvm_chk_meta *meta)
{
struct nvm_dev *dev = tgt_dev->parent;
nvm_ppa_tgt_to_dev(tgt_dev, &ppa, 1);
- return dev->ops->get_bb_tbl(dev, ppa, blks);
+ if (dev->geo.version == NVM_OCSSD_SPEC_12)
+ return nvm_get_bb_meta(dev, (sector_t)ppa.ppa, nchks, meta);
+
+ return dev->ops->get_chk_meta(dev, (sector_t)ppa.ppa, nchks, meta);
+}
+EXPORT_SYMBOL_GPL(nvm_get_chunk_meta);
+
+int nvm_set_chunk_meta(struct nvm_tgt_dev *tgt_dev, struct ppa_addr *ppas,
+ int nr_ppas, int type)
+{
+ struct nvm_dev *dev = tgt_dev->parent;
+ struct nvm_rq rqd;
+ int ret;
+
+ if (dev->geo.version == NVM_OCSSD_SPEC_20)
+ return 0;
+
+ if (nr_ppas > NVM_MAX_VLBA) {
+ pr_err("nvm: unable to update all blocks atomically\n");
+ return -EINVAL;
+ }
+
+ memset(&rqd, 0, sizeof(struct nvm_rq));
+
+ nvm_set_rqd_ppalist(tgt_dev, &rqd, ppas, nr_ppas);
+ nvm_rq_tgt_to_dev(tgt_dev, &rqd);
+
+ ret = dev->ops->set_bb_tbl(dev, &rqd.ppa_addr, rqd.nr_ppas, type);
+ nvm_free_rqd_ppalist(tgt_dev, &rqd);
+ if (ret)
+ return -EINVAL;
+
+ return 0;
}
-EXPORT_SYMBOL(nvm_get_tgt_bb_tbl);
+EXPORT_SYMBOL_GPL(nvm_set_chunk_meta);
static int nvm_core_init(struct nvm_dev *dev)
{
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2016 CNEX Labs
* Initial release: Javier Gonzalez <javier@cnexlabs.com>
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2016 CNEX Labs
* Initial release: Javier Gonzalez <javier@cnexlabs.com>
*
*/
+#define CREATE_TRACE_POINTS
+
#include "pblk.h"
+#include "pblk-trace.h"
static void pblk_line_mark_bb(struct work_struct *work)
{
struct ppa_addr *ppa = line_ws->priv;
int ret;
- ret = nvm_set_tgt_bb_tbl(dev, ppa, 1, NVM_BLK_T_GRWN_BAD);
+ ret = nvm_set_chunk_meta(dev, ppa, 1, NVM_BLK_T_GRWN_BAD);
if (ret) {
struct pblk_line *line;
int pos;
- line = &pblk->lines[pblk_ppa_to_line(*ppa)];
+ line = pblk_ppa_to_line(pblk, *ppa);
pos = pblk_ppa_to_pos(&dev->geo, *ppa);
pblk_err(pblk, "failed to mark bb, line:%d, pos:%d\n",
struct pblk_line *line;
int pos;
- line = &pblk->lines[pblk_ppa_to_line(rqd->ppa_addr)];
+ line = pblk_ppa_to_line(pblk, rqd->ppa_addr);
pos = pblk_ppa_to_pos(geo, rqd->ppa_addr);
chunk = &line->chks[pos];
atomic_dec(&line->left_seblks);
if (rqd->error) {
+ trace_pblk_chunk_reset(pblk_disk_name(pblk),
+ &rqd->ppa_addr, PBLK_CHUNK_RESET_FAILED);
+
chunk->state = NVM_CHK_ST_OFFLINE;
pblk_mark_bb(pblk, line, rqd->ppa_addr);
} else {
+ trace_pblk_chunk_reset(pblk_disk_name(pblk),
+ &rqd->ppa_addr, PBLK_CHUNK_RESET_DONE);
+
chunk->state = NVM_CHK_ST_FREE;
}
+ trace_pblk_chunk_state(pblk_disk_name(pblk), &rqd->ppa_addr,
+ chunk->state);
+
atomic_dec(&pblk->inflight_io);
}
/*
* Get information for all chunks from the device.
*
- * The caller is responsible for freeing the returned structure
+ * The caller is responsible for freeing (vmalloc) the returned structure
*/
-struct nvm_chk_meta *pblk_chunk_get_info(struct pblk *pblk)
+struct nvm_chk_meta *pblk_get_chunk_meta(struct pblk *pblk)
{
struct nvm_tgt_dev *dev = pblk->dev;
struct nvm_geo *geo = &dev->geo;
ppa.ppa = 0;
len = geo->all_chunks * sizeof(*meta);
- meta = kzalloc(len, GFP_KERNEL);
+ meta = vzalloc(len);
if (!meta)
return ERR_PTR(-ENOMEM);
- ret = nvm_get_chunk_meta(dev, meta, ppa, geo->all_chunks);
+ ret = nvm_get_chunk_meta(dev, ppa, geo->all_chunks, meta);
if (ret) {
kfree(meta);
return ERR_PTR(-EIO);
{
struct pblk_line *line;
u64 paddr;
- int line_id;
#ifdef CONFIG_NVM_PBLK_DEBUG
/* Callers must ensure that the ppa points to a device address */
BUG_ON(pblk_ppa_empty(ppa));
#endif
- line_id = pblk_ppa_to_line(ppa);
- line = &pblk->lines[line_id];
+ line = pblk_ppa_to_line(pblk, ppa);
paddr = pblk_dev_ppa_to_line_addr(pblk, ppa);
__pblk_map_invalidate(pblk, line, paddr);
spin_unlock(&pblk->trans_lock);
}
+int pblk_alloc_rqd_meta(struct pblk *pblk, struct nvm_rq *rqd)
+{
+ struct nvm_tgt_dev *dev = pblk->dev;
+
+ rqd->meta_list = nvm_dev_dma_alloc(dev->parent, GFP_KERNEL,
+ &rqd->dma_meta_list);
+ if (!rqd->meta_list)
+ return -ENOMEM;
+
+ if (rqd->nr_ppas == 1)
+ return 0;
+
+ rqd->ppa_list = rqd->meta_list + pblk_dma_meta_size;
+ rqd->dma_ppa_list = rqd->dma_meta_list + pblk_dma_meta_size;
+
+ return 0;
+}
+
+void pblk_free_rqd_meta(struct pblk *pblk, struct nvm_rq *rqd)
+{
+ struct nvm_tgt_dev *dev = pblk->dev;
+
+ if (rqd->meta_list)
+ nvm_dev_dma_free(dev->parent, rqd->meta_list,
+ rqd->dma_meta_list);
+}
+
/* Caller must guarantee that the request is a valid type */
struct nvm_rq *pblk_alloc_rqd(struct pblk *pblk, int type)
{
/* Typically used on completion path. Cannot guarantee request consistency */
void pblk_free_rqd(struct pblk *pblk, struct nvm_rq *rqd, int type)
{
- struct nvm_tgt_dev *dev = pblk->dev;
mempool_t *pool;
switch (type) {
return;
}
- if (rqd->meta_list)
- nvm_dev_dma_free(dev->parent, rqd->meta_list,
- rqd->dma_meta_list);
+ pblk_free_rqd_meta(pblk, rqd);
mempool_free(rqd, pool);
}
}
} else {
line->state = PBLK_LINESTATE_CORRUPT;
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
+
line->gc_group = PBLK_LINEGC_NONE;
move_list = &l_mg->corrupt_list;
pblk_err(pblk, "corrupted vsc for line %d, vsc:%d (%d/%d/%d)\n",
return nvm_submit_io(dev, rqd);
}
+void pblk_check_chunk_state_update(struct pblk *pblk, struct nvm_rq *rqd)
+{
+ struct ppa_addr *ppa_list = nvm_rq_to_ppa_list(rqd);
+
+ int i;
+
+ for (i = 0; i < rqd->nr_ppas; i++) {
+ struct ppa_addr *ppa = &ppa_list[i];
+ struct nvm_chk_meta *chunk = pblk_dev_ppa_to_chunk(pblk, *ppa);
+ u64 caddr = pblk_dev_ppa_to_chunk_addr(pblk, *ppa);
+
+ if (caddr == 0)
+ trace_pblk_chunk_state(pblk_disk_name(pblk),
+ ppa, NVM_CHK_ST_OPEN);
+ else if (caddr == chunk->cnlb)
+ trace_pblk_chunk_state(pblk_disk_name(pblk),
+ ppa, NVM_CHK_ST_CLOSED);
+ }
+}
+
int pblk_submit_io_sync(struct pblk *pblk, struct nvm_rq *rqd)
{
struct nvm_tgt_dev *dev = pblk->dev;
+ int ret;
atomic_inc(&pblk->inflight_io);
return NVM_IO_ERR;
#endif
- return nvm_submit_io_sync(dev, rqd);
+ ret = nvm_submit_io_sync(dev, rqd);
+
+ if (trace_pblk_chunk_state_enabled() && !ret &&
+ rqd->opcode == NVM_OP_PWRITE)
+ pblk_check_chunk_state_update(pblk, rqd);
+
+ return ret;
+}
+
+int pblk_submit_io_sync_sem(struct pblk *pblk, struct nvm_rq *rqd)
+{
+ struct ppa_addr *ppa_list;
+ int ret;
+
+ ppa_list = (rqd->nr_ppas > 1) ? rqd->ppa_list : &rqd->ppa_addr;
+
+ pblk_down_chunk(pblk, ppa_list[0]);
+ ret = pblk_submit_io_sync(pblk, rqd);
+ pblk_up_chunk(pblk, ppa_list[0]);
+
+ return ret;
}
static void pblk_bio_map_addr_endio(struct bio *bio)
return paddr;
}
-/*
- * Submit emeta to one LUN in the raid line at the time to avoid a deadlock when
- * taking the per LUN semaphore.
- */
-static int pblk_line_submit_emeta_io(struct pblk *pblk, struct pblk_line *line,
- void *emeta_buf, u64 paddr, int dir)
+u64 pblk_line_smeta_start(struct pblk *pblk, struct pblk_line *line)
{
struct nvm_tgt_dev *dev = pblk->dev;
struct nvm_geo *geo = &dev->geo;
- struct pblk_line_mgmt *l_mg = &pblk->l_mg;
struct pblk_line_meta *lm = &pblk->lm;
- void *ppa_list, *meta_list;
- struct bio *bio;
- struct nvm_rq rqd;
- dma_addr_t dma_ppa_list, dma_meta_list;
- int min = pblk->min_write_pgs;
- int left_ppas = lm->emeta_sec[0];
- int id = line->id;
- int rq_ppas, rq_len;
- int cmd_op, bio_op;
- int i, j;
- int ret;
+ int bit;
- if (dir == PBLK_WRITE) {
- bio_op = REQ_OP_WRITE;
- cmd_op = NVM_OP_PWRITE;
- } else if (dir == PBLK_READ) {
- bio_op = REQ_OP_READ;
- cmd_op = NVM_OP_PREAD;
- } else
- return -EINVAL;
+ /* This usually only happens on bad lines */
+ bit = find_first_zero_bit(line->blk_bitmap, lm->blk_per_line);
+ if (bit >= lm->blk_per_line)
+ return -1;
- meta_list = nvm_dev_dma_alloc(dev->parent, GFP_KERNEL,
- &dma_meta_list);
- if (!meta_list)
- return -ENOMEM;
+ return bit * geo->ws_opt;
+}
- ppa_list = meta_list + pblk_dma_meta_size;
- dma_ppa_list = dma_meta_list + pblk_dma_meta_size;
+int pblk_line_smeta_read(struct pblk *pblk, struct pblk_line *line)
+{
+ struct nvm_tgt_dev *dev = pblk->dev;
+ struct pblk_line_meta *lm = &pblk->lm;
+ struct bio *bio;
+ struct nvm_rq rqd;
+ u64 paddr = pblk_line_smeta_start(pblk, line);
+ int i, ret;
-next_rq:
memset(&rqd, 0, sizeof(struct nvm_rq));
- rq_ppas = pblk_calc_secs(pblk, left_ppas, 0);
- rq_len = rq_ppas * geo->csecs;
+ ret = pblk_alloc_rqd_meta(pblk, &rqd);
+ if (ret)
+ return ret;
- bio = pblk_bio_map_addr(pblk, emeta_buf, rq_ppas, rq_len,
- l_mg->emeta_alloc_type, GFP_KERNEL);
+ bio = bio_map_kern(dev->q, line->smeta, lm->smeta_len, GFP_KERNEL);
if (IS_ERR(bio)) {
ret = PTR_ERR(bio);
- goto free_rqd_dma;
+ goto clear_rqd;
}
bio->bi_iter.bi_sector = 0; /* internal bio */
- bio_set_op_attrs(bio, bio_op, 0);
+ bio_set_op_attrs(bio, REQ_OP_READ, 0);
rqd.bio = bio;
- rqd.meta_list = meta_list;
- rqd.ppa_list = ppa_list;
- rqd.dma_meta_list = dma_meta_list;
- rqd.dma_ppa_list = dma_ppa_list;
- rqd.opcode = cmd_op;
- rqd.nr_ppas = rq_ppas;
-
- if (dir == PBLK_WRITE) {
- struct pblk_sec_meta *meta_list = rqd.meta_list;
-
- rqd.flags = pblk_set_progr_mode(pblk, PBLK_WRITE);
- for (i = 0; i < rqd.nr_ppas; ) {
- spin_lock(&line->lock);
- paddr = __pblk_alloc_page(pblk, line, min);
- spin_unlock(&line->lock);
- for (j = 0; j < min; j++, i++, paddr++) {
- meta_list[i].lba = cpu_to_le64(ADDR_EMPTY);
- rqd.ppa_list[i] =
- addr_to_gen_ppa(pblk, paddr, id);
- }
- }
- } else {
- for (i = 0; i < rqd.nr_ppas; ) {
- struct ppa_addr ppa = addr_to_gen_ppa(pblk, paddr, id);
- int pos = pblk_ppa_to_pos(geo, ppa);
- int read_type = PBLK_READ_RANDOM;
-
- if (pblk_io_aligned(pblk, rq_ppas))
- read_type = PBLK_READ_SEQUENTIAL;
- rqd.flags = pblk_set_read_mode(pblk, read_type);
-
- while (test_bit(pos, line->blk_bitmap)) {
- paddr += min;
- if (pblk_boundary_paddr_checks(pblk, paddr)) {
- pblk_err(pblk, "corrupt emeta line:%d\n",
- line->id);
- bio_put(bio);
- ret = -EINTR;
- goto free_rqd_dma;
- }
-
- ppa = addr_to_gen_ppa(pblk, paddr, id);
- pos = pblk_ppa_to_pos(geo, ppa);
- }
-
- if (pblk_boundary_paddr_checks(pblk, paddr + min)) {
- pblk_err(pblk, "corrupt emeta line:%d\n",
- line->id);
- bio_put(bio);
- ret = -EINTR;
- goto free_rqd_dma;
- }
+ rqd.opcode = NVM_OP_PREAD;
+ rqd.nr_ppas = lm->smeta_sec;
+ rqd.is_seq = 1;
- for (j = 0; j < min; j++, i++, paddr++)
- rqd.ppa_list[i] =
- addr_to_gen_ppa(pblk, paddr, line->id);
- }
- }
+ for (i = 0; i < lm->smeta_sec; i++, paddr++)
+ rqd.ppa_list[i] = addr_to_gen_ppa(pblk, paddr, line->id);
ret = pblk_submit_io_sync(pblk, &rqd);
if (ret) {
- pblk_err(pblk, "emeta I/O submission failed: %d\n", ret);
+ pblk_err(pblk, "smeta I/O submission failed: %d\n", ret);
bio_put(bio);
- goto free_rqd_dma;
+ goto clear_rqd;
}
atomic_dec(&pblk->inflight_io);
- if (rqd.error) {
- if (dir == PBLK_WRITE)
- pblk_log_write_err(pblk, &rqd);
- else
- pblk_log_read_err(pblk, &rqd);
- }
+ if (rqd.error)
+ pblk_log_read_err(pblk, &rqd);
- emeta_buf += rq_len;
- left_ppas -= rq_ppas;
- if (left_ppas)
- goto next_rq;
-free_rqd_dma:
- nvm_dev_dma_free(dev->parent, rqd.meta_list, rqd.dma_meta_list);
+clear_rqd:
+ pblk_free_rqd_meta(pblk, &rqd);
return ret;
}
-u64 pblk_line_smeta_start(struct pblk *pblk, struct pblk_line *line)
-{
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
- struct pblk_line_meta *lm = &pblk->lm;
- int bit;
-
- /* This usually only happens on bad lines */
- bit = find_first_zero_bit(line->blk_bitmap, lm->blk_per_line);
- if (bit >= lm->blk_per_line)
- return -1;
-
- return bit * geo->ws_opt;
-}
-
-static int pblk_line_submit_smeta_io(struct pblk *pblk, struct pblk_line *line,
- u64 paddr, int dir)
+static int pblk_line_smeta_write(struct pblk *pblk, struct pblk_line *line,
+ u64 paddr)
{
struct nvm_tgt_dev *dev = pblk->dev;
struct pblk_line_meta *lm = &pblk->lm;
struct bio *bio;
struct nvm_rq rqd;
- __le64 *lba_list = NULL;
+ __le64 *lba_list = emeta_to_lbas(pblk, line->emeta->buf);
+ __le64 addr_empty = cpu_to_le64(ADDR_EMPTY);
int i, ret;
- int cmd_op, bio_op;
- int flags;
-
- if (dir == PBLK_WRITE) {
- bio_op = REQ_OP_WRITE;
- cmd_op = NVM_OP_PWRITE;
- flags = pblk_set_progr_mode(pblk, PBLK_WRITE);
- lba_list = emeta_to_lbas(pblk, line->emeta->buf);
- } else if (dir == PBLK_READ_RECOV || dir == PBLK_READ) {
- bio_op = REQ_OP_READ;
- cmd_op = NVM_OP_PREAD;
- flags = pblk_set_read_mode(pblk, PBLK_READ_SEQUENTIAL);
- } else
- return -EINVAL;
memset(&rqd, 0, sizeof(struct nvm_rq));
- rqd.meta_list = nvm_dev_dma_alloc(dev->parent, GFP_KERNEL,
- &rqd.dma_meta_list);
- if (!rqd.meta_list)
- return -ENOMEM;
-
- rqd.ppa_list = rqd.meta_list + pblk_dma_meta_size;
- rqd.dma_ppa_list = rqd.dma_meta_list + pblk_dma_meta_size;
+ ret = pblk_alloc_rqd_meta(pblk, &rqd);
+ if (ret)
+ return ret;
bio = bio_map_kern(dev->q, line->smeta, lm->smeta_len, GFP_KERNEL);
if (IS_ERR(bio)) {
ret = PTR_ERR(bio);
- goto free_ppa_list;
+ goto clear_rqd;
}
bio->bi_iter.bi_sector = 0; /* internal bio */
- bio_set_op_attrs(bio, bio_op, 0);
+ bio_set_op_attrs(bio, REQ_OP_WRITE, 0);
rqd.bio = bio;
- rqd.opcode = cmd_op;
- rqd.flags = flags;
+ rqd.opcode = NVM_OP_PWRITE;
rqd.nr_ppas = lm->smeta_sec;
+ rqd.is_seq = 1;
for (i = 0; i < lm->smeta_sec; i++, paddr++) {
struct pblk_sec_meta *meta_list = rqd.meta_list;
rqd.ppa_list[i] = addr_to_gen_ppa(pblk, paddr, line->id);
-
- if (dir == PBLK_WRITE) {
- __le64 addr_empty = cpu_to_le64(ADDR_EMPTY);
-
- meta_list[i].lba = lba_list[paddr] = addr_empty;
- }
+ meta_list[i].lba = lba_list[paddr] = addr_empty;
}
- /*
- * This I/O is sent by the write thread when a line is replace. Since
- * the write thread is the only one sending write and erase commands,
- * there is no need to take the LUN semaphore.
- */
- ret = pblk_submit_io_sync(pblk, &rqd);
+ ret = pblk_submit_io_sync_sem(pblk, &rqd);
if (ret) {
pblk_err(pblk, "smeta I/O submission failed: %d\n", ret);
bio_put(bio);
- goto free_ppa_list;
+ goto clear_rqd;
}
atomic_dec(&pblk->inflight_io);
if (rqd.error) {
- if (dir == PBLK_WRITE) {
- pblk_log_write_err(pblk, &rqd);
- ret = 1;
- } else if (dir == PBLK_READ)
- pblk_log_read_err(pblk, &rqd);
+ pblk_log_write_err(pblk, &rqd);
+ ret = -EIO;
}
-free_ppa_list:
- nvm_dev_dma_free(dev->parent, rqd.meta_list, rqd.dma_meta_list);
-
+clear_rqd:
+ pblk_free_rqd_meta(pblk, &rqd);
return ret;
}
-int pblk_line_read_smeta(struct pblk *pblk, struct pblk_line *line)
+int pblk_line_emeta_read(struct pblk *pblk, struct pblk_line *line,
+ void *emeta_buf)
{
- u64 bpaddr = pblk_line_smeta_start(pblk, line);
+ struct nvm_tgt_dev *dev = pblk->dev;
+ struct nvm_geo *geo = &dev->geo;
+ struct pblk_line_mgmt *l_mg = &pblk->l_mg;
+ struct pblk_line_meta *lm = &pblk->lm;
+ void *ppa_list, *meta_list;
+ struct bio *bio;
+ struct nvm_rq rqd;
+ u64 paddr = line->emeta_ssec;
+ dma_addr_t dma_ppa_list, dma_meta_list;
+ int min = pblk->min_write_pgs;
+ int left_ppas = lm->emeta_sec[0];
+ int line_id = line->id;
+ int rq_ppas, rq_len;
+ int i, j;
+ int ret;
- return pblk_line_submit_smeta_io(pblk, line, bpaddr, PBLK_READ_RECOV);
-}
+ meta_list = nvm_dev_dma_alloc(dev->parent, GFP_KERNEL,
+ &dma_meta_list);
+ if (!meta_list)
+ return -ENOMEM;
-int pblk_line_read_emeta(struct pblk *pblk, struct pblk_line *line,
- void *emeta_buf)
-{
- return pblk_line_submit_emeta_io(pblk, line, emeta_buf,
- line->emeta_ssec, PBLK_READ);
+ ppa_list = meta_list + pblk_dma_meta_size;
+ dma_ppa_list = dma_meta_list + pblk_dma_meta_size;
+
+next_rq:
+ memset(&rqd, 0, sizeof(struct nvm_rq));
+
+ rq_ppas = pblk_calc_secs(pblk, left_ppas, 0);
+ rq_len = rq_ppas * geo->csecs;
+
+ bio = pblk_bio_map_addr(pblk, emeta_buf, rq_ppas, rq_len,
+ l_mg->emeta_alloc_type, GFP_KERNEL);
+ if (IS_ERR(bio)) {
+ ret = PTR_ERR(bio);
+ goto free_rqd_dma;
+ }
+
+ bio->bi_iter.bi_sector = 0; /* internal bio */
+ bio_set_op_attrs(bio, REQ_OP_READ, 0);
+
+ rqd.bio = bio;
+ rqd.meta_list = meta_list;
+ rqd.ppa_list = ppa_list;
+ rqd.dma_meta_list = dma_meta_list;
+ rqd.dma_ppa_list = dma_ppa_list;
+ rqd.opcode = NVM_OP_PREAD;
+ rqd.nr_ppas = rq_ppas;
+
+ for (i = 0; i < rqd.nr_ppas; ) {
+ struct ppa_addr ppa = addr_to_gen_ppa(pblk, paddr, line_id);
+ int pos = pblk_ppa_to_pos(geo, ppa);
+
+ if (pblk_io_aligned(pblk, rq_ppas))
+ rqd.is_seq = 1;
+
+ while (test_bit(pos, line->blk_bitmap)) {
+ paddr += min;
+ if (pblk_boundary_paddr_checks(pblk, paddr)) {
+ bio_put(bio);
+ ret = -EINTR;
+ goto free_rqd_dma;
+ }
+
+ ppa = addr_to_gen_ppa(pblk, paddr, line_id);
+ pos = pblk_ppa_to_pos(geo, ppa);
+ }
+
+ if (pblk_boundary_paddr_checks(pblk, paddr + min)) {
+ bio_put(bio);
+ ret = -EINTR;
+ goto free_rqd_dma;
+ }
+
+ for (j = 0; j < min; j++, i++, paddr++)
+ rqd.ppa_list[i] = addr_to_gen_ppa(pblk, paddr, line_id);
+ }
+
+ ret = pblk_submit_io_sync(pblk, &rqd);
+ if (ret) {
+ pblk_err(pblk, "emeta I/O submission failed: %d\n", ret);
+ bio_put(bio);
+ goto free_rqd_dma;
+ }
+
+ atomic_dec(&pblk->inflight_io);
+
+ if (rqd.error)
+ pblk_log_read_err(pblk, &rqd);
+
+ emeta_buf += rq_len;
+ left_ppas -= rq_ppas;
+ if (left_ppas)
+ goto next_rq;
+
+free_rqd_dma:
+ nvm_dev_dma_free(dev->parent, rqd.meta_list, rqd.dma_meta_list);
+ return ret;
}
static void pblk_setup_e_rq(struct pblk *pblk, struct nvm_rq *rqd,
rqd->opcode = NVM_OP_ERASE;
rqd->ppa_addr = ppa;
rqd->nr_ppas = 1;
- rqd->flags = pblk_set_progr_mode(pblk, PBLK_ERASE);
+ rqd->is_seq = 1;
rqd->bio = NULL;
}
static int pblk_blk_erase_sync(struct pblk *pblk, struct ppa_addr ppa)
{
- struct nvm_rq rqd;
- int ret = 0;
+ struct nvm_rq rqd = {NULL};
+ int ret;
- memset(&rqd, 0, sizeof(struct nvm_rq));
+ trace_pblk_chunk_reset(pblk_disk_name(pblk), &ppa,
+ PBLK_CHUNK_RESET_START);
pblk_setup_e_rq(pblk, &rqd, ppa);
* with writes. Thus, there is no need to take the LUN semaphore.
*/
ret = pblk_submit_io_sync(pblk, &rqd);
- if (ret) {
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
-
- pblk_err(pblk, "could not sync erase line:%d,blk:%d\n",
- pblk_ppa_to_line(ppa),
- pblk_ppa_to_pos(geo, ppa));
-
- rqd.error = ret;
- goto out;
- }
-
-out:
rqd.private = pblk;
__pblk_end_io_erase(pblk, &rqd);
spin_lock(&l_mg->free_lock);
spin_lock(&line->lock);
line->state = PBLK_LINESTATE_BAD;
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
spin_unlock(&line->lock);
list_add_tail(&line->list, &l_mg->bad_list);
static int pblk_line_alloc_bitmaps(struct pblk *pblk, struct pblk_line *line)
{
struct pblk_line_meta *lm = &pblk->lm;
+ struct pblk_line_mgmt *l_mg = &pblk->l_mg;
- line->map_bitmap = kzalloc(lm->sec_bitmap_len, GFP_KERNEL);
+ line->map_bitmap = mempool_alloc(l_mg->bitmap_pool, GFP_KERNEL);
if (!line->map_bitmap)
return -ENOMEM;
+ memset(line->map_bitmap, 0, lm->sec_bitmap_len);
+
/* will be initialized using bb info from map_bitmap */
- line->invalid_bitmap = kmalloc(lm->sec_bitmap_len, GFP_KERNEL);
+ line->invalid_bitmap = mempool_alloc(l_mg->bitmap_pool, GFP_KERNEL);
if (!line->invalid_bitmap) {
- kfree(line->map_bitmap);
+ mempool_free(line->map_bitmap, l_mg->bitmap_pool);
line->map_bitmap = NULL;
return -ENOMEM;
}
line->smeta_ssec = off;
line->cur_sec = off + lm->smeta_sec;
- if (init && pblk_line_submit_smeta_io(pblk, line, off, PBLK_WRITE)) {
+ if (init && pblk_line_smeta_write(pblk, line, off)) {
pblk_debug(pblk, "line smeta I/O failed. Retry\n");
return 0;
}
bitmap_weight(line->invalid_bitmap, lm->sec_per_line)) {
spin_lock(&line->lock);
line->state = PBLK_LINESTATE_BAD;
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
spin_unlock(&line->lock);
list_add_tail(&line->list, &l_mg->bad_list);
if (line->state == PBLK_LINESTATE_NEW) {
blk_to_erase = pblk_prepare_new_line(pblk, line);
line->state = PBLK_LINESTATE_FREE;
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
} else {
blk_to_erase = blk_in_line;
}
}
line->state = PBLK_LINESTATE_OPEN;
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
atomic_set(&line->left_eblks, blk_to_erase);
atomic_set(&line->left_seblks, blk_to_erase);
void pblk_line_recov_close(struct pblk *pblk, struct pblk_line *line)
{
- kfree(line->map_bitmap);
+ struct pblk_line_mgmt *l_mg = &pblk->l_mg;
+
+ mempool_free(line->map_bitmap, l_mg->bitmap_pool);
line->map_bitmap = NULL;
line->smeta = NULL;
line->emeta = NULL;
void pblk_line_free(struct pblk_line *line)
{
- kfree(line->map_bitmap);
- kfree(line->invalid_bitmap);
+ struct pblk *pblk = line->pblk;
+ struct pblk_line_mgmt *l_mg = &pblk->l_mg;
+
+ mempool_free(line->map_bitmap, l_mg->bitmap_pool);
+ mempool_free(line->invalid_bitmap, l_mg->bitmap_pool);
pblk_line_reinit(line);
}
if (unlikely(bit >= lm->blk_per_line)) {
spin_lock(&line->lock);
line->state = PBLK_LINESTATE_BAD;
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
spin_unlock(&line->lock);
list_add_tail(&line->list, &l_mg->bad_list);
return line;
}
+void pblk_ppa_to_line_put(struct pblk *pblk, struct ppa_addr ppa)
+{
+ struct pblk_line *line;
+
+ line = pblk_ppa_to_line(pblk, ppa);
+ kref_put(&line->ref, pblk_line_put_wq);
+}
+
+void pblk_rq_to_line_put(struct pblk *pblk, struct nvm_rq *rqd)
+{
+ struct ppa_addr *ppa_list;
+ int i;
+
+ ppa_list = (rqd->nr_ppas > 1) ? rqd->ppa_list : &rqd->ppa_addr;
+
+ for (i = 0; i < rqd->nr_ppas; i++)
+ pblk_ppa_to_line_put(pblk, ppa_list[i]);
+}
+
static void pblk_stop_writes(struct pblk *pblk, struct pblk_line *line)
{
lockdep_assert_held(&pblk->l_mg.free_lock);
pblk_set_space_limit(pblk);
pblk->state = PBLK_STATE_STOPPING;
+ trace_pblk_state(pblk_disk_name(pblk), pblk->state);
}
static void pblk_line_close_meta_sync(struct pblk *pblk)
return;
}
pblk->state = PBLK_STATE_RECOVERING;
+ trace_pblk_state(pblk_disk_name(pblk), pblk->state);
spin_unlock(&l_mg->free_lock);
pblk_flush_writer(pblk);
spin_lock(&l_mg->free_lock);
pblk->state = PBLK_STATE_STOPPED;
+ trace_pblk_state(pblk_disk_name(pblk), pblk->state);
l_mg->data_line = NULL;
l_mg->data_next = NULL;
spin_unlock(&l_mg->free_lock);
struct pblk_line *cur, *new = NULL;
unsigned int left_seblks;
- cur = l_mg->data_line;
new = l_mg->data_next;
if (!new)
goto out;
- l_mg->data_line = new;
spin_lock(&l_mg->free_lock);
+ cur = l_mg->data_line;
+ l_mg->data_line = new;
+
pblk_line_setup_metadata(new, l_mg, &pblk->lm);
spin_unlock(&l_mg->free_lock);
spin_lock(&line->lock);
WARN_ON(line->state != PBLK_LINESTATE_GC);
line->state = PBLK_LINESTATE_FREE;
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
line->gc_group = PBLK_LINEGC_NONE;
pblk_line_free(line);
rqd->end_io = pblk_end_io_erase;
rqd->private = pblk;
+ trace_pblk_chunk_reset(pblk_disk_name(pblk),
+ &ppa, PBLK_CHUNK_RESET_START);
+
/* The write thread schedules erases so that it minimizes disturbances
* with writes. Thus, there is no need to take the LUN semaphore.
*/
struct nvm_geo *geo = &dev->geo;
pblk_err(pblk, "could not async erase line:%d,blk:%d\n",
- pblk_ppa_to_line(ppa),
+ pblk_ppa_to_line_id(ppa),
pblk_ppa_to_pos(geo, ppa));
}
WARN_ON(line->state != PBLK_LINESTATE_OPEN);
line->state = PBLK_LINESTATE_CLOSED;
move_list = pblk_line_gc_list(pblk, line);
-
list_add_tail(&line->list, move_list);
- kfree(line->map_bitmap);
+ mempool_free(line->map_bitmap, l_mg->bitmap_pool);
line->map_bitmap = NULL;
line->smeta = NULL;
line->emeta = NULL;
spin_unlock(&line->lock);
spin_unlock(&l_mg->gc_lock);
+
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
}
void pblk_line_close_meta(struct pblk *pblk, struct pblk_line *line)
wa->pad = cpu_to_le64(atomic64_read(&pblk->pad_wa));
wa->gc = cpu_to_le64(atomic64_read(&pblk->gc_wa));
+ if (le32_to_cpu(emeta_buf->header.identifier) != PBLK_MAGIC) {
+ emeta_buf->header.identifier = cpu_to_le32(PBLK_MAGIC);
+ memcpy(emeta_buf->header.uuid, pblk->instance_uuid, 16);
+ emeta_buf->header.id = cpu_to_le32(line->id);
+ emeta_buf->header.type = cpu_to_le16(line->type);
+ emeta_buf->header.version_major = EMETA_VERSION_MAJOR;
+ emeta_buf->header.version_minor = EMETA_VERSION_MINOR;
+ emeta_buf->header.crc = cpu_to_le32(
+ pblk_calc_meta_header_crc(pblk, &emeta_buf->header));
+ }
+
emeta_buf->nr_valid_lbas = cpu_to_le64(line->nr_valid_lbas);
emeta_buf->crc = cpu_to_le32(pblk_calc_emeta_crc(pblk, emeta_buf));
spin_unlock(&l_mg->close_lock);
pblk_line_should_sync_meta(pblk);
-
-
}
static void pblk_save_lba_list(struct pblk *pblk, struct pblk_line *line)
queue_work(wq, &line_ws->ws);
}
-static void __pblk_down_page(struct pblk *pblk, struct ppa_addr *ppa_list,
- int nr_ppas, int pos)
+static void __pblk_down_chunk(struct pblk *pblk, int pos)
{
struct pblk_lun *rlun = &pblk->luns[pos];
int ret;
* Only send one inflight I/O per LUN. Since we map at a page
* granurality, all ppas in the I/O will map to the same LUN
*/
-#ifdef CONFIG_NVM_PBLK_DEBUG
- int i;
-
- for (i = 1; i < nr_ppas; i++)
- WARN_ON(ppa_list[0].a.lun != ppa_list[i].a.lun ||
- ppa_list[0].a.ch != ppa_list[i].a.ch);
-#endif
ret = down_timeout(&rlun->wr_sem, msecs_to_jiffies(30000));
if (ret == -ETIME || ret == -EINTR)
-ret);
}
-void pblk_down_page(struct pblk *pblk, struct ppa_addr *ppa_list, int nr_ppas)
+void pblk_down_chunk(struct pblk *pblk, struct ppa_addr ppa)
{
struct nvm_tgt_dev *dev = pblk->dev;
struct nvm_geo *geo = &dev->geo;
- int pos = pblk_ppa_to_pos(geo, ppa_list[0]);
+ int pos = pblk_ppa_to_pos(geo, ppa);
- __pblk_down_page(pblk, ppa_list, nr_ppas, pos);
+ __pblk_down_chunk(pblk, pos);
}
-void pblk_down_rq(struct pblk *pblk, struct ppa_addr *ppa_list, int nr_ppas,
+void pblk_down_rq(struct pblk *pblk, struct ppa_addr ppa,
unsigned long *lun_bitmap)
{
struct nvm_tgt_dev *dev = pblk->dev;
struct nvm_geo *geo = &dev->geo;
- int pos = pblk_ppa_to_pos(geo, ppa_list[0]);
+ int pos = pblk_ppa_to_pos(geo, ppa);
/* If the LUN has been locked for this same request, do no attempt to
* lock it again
if (test_and_set_bit(pos, lun_bitmap))
return;
- __pblk_down_page(pblk, ppa_list, nr_ppas, pos);
+ __pblk_down_chunk(pblk, pos);
}
-void pblk_up_page(struct pblk *pblk, struct ppa_addr *ppa_list, int nr_ppas)
+void pblk_up_chunk(struct pblk *pblk, struct ppa_addr ppa)
{
struct nvm_tgt_dev *dev = pblk->dev;
struct nvm_geo *geo = &dev->geo;
struct pblk_lun *rlun;
- int pos = pblk_ppa_to_pos(geo, ppa_list[0]);
-
-#ifdef CONFIG_NVM_PBLK_DEBUG
- int i;
-
- for (i = 1; i < nr_ppas; i++)
- WARN_ON(ppa_list[0].a.lun != ppa_list[i].a.lun ||
- ppa_list[0].a.ch != ppa_list[i].a.ch);
-#endif
+ int pos = pblk_ppa_to_pos(geo, ppa);
rlun = &pblk->luns[pos];
up(&rlun->wr_sem);
}
-void pblk_up_rq(struct pblk *pblk, struct ppa_addr *ppa_list, int nr_ppas,
- unsigned long *lun_bitmap)
+void pblk_up_rq(struct pblk *pblk, unsigned long *lun_bitmap)
{
struct nvm_tgt_dev *dev = pblk->dev;
struct nvm_geo *geo = &dev->geo;
/* If the L2P entry maps to a line, the reference is valid */
if (!pblk_ppa_empty(ppa) && !pblk_addr_in_cache(ppa)) {
- int line_id = pblk_ppa_to_line(ppa);
- struct pblk_line *line = &pblk->lines[line_id];
+ struct pblk_line *line = pblk_ppa_to_line(pblk, ppa);
kref_get(&line->ref);
}
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2016 CNEX Labs
* Initial release: Javier Gonzalez <javier@cnexlabs.com>
*/
#include "pblk.h"
+#include "pblk-trace.h"
#include <linux/delay.h>
+
static void pblk_gc_free_gc_rq(struct pblk_gc_rq *gc_rq)
{
if (gc_rq->data)
spin_lock(&line->lock);
WARN_ON(line->state != PBLK_LINESTATE_GC);
line->state = PBLK_LINESTATE_CLOSED;
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
move_list = pblk_line_gc_list(pblk, line);
spin_unlock(&line->lock);
if (!emeta_buf)
return NULL;
- ret = pblk_line_read_emeta(pblk, line, emeta_buf);
+ ret = pblk_line_emeta_read(pblk, line, emeta_buf);
if (ret) {
pblk_err(pblk, "line %d read emeta failed (%d)\n",
line->id, ret);
spin_lock(&line->lock);
WARN_ON(line->state != PBLK_LINESTATE_CLOSED);
line->state = PBLK_LINESTATE_GC;
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
spin_unlock(&line->lock);
list_del(&line->list);
spin_lock(&line->lock);
WARN_ON(line->state != PBLK_LINESTATE_CLOSED);
line->state = PBLK_LINESTATE_GC;
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
spin_unlock(&line->lock);
list_del(&line->list);
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2015 IT University of Copenhagen (rrpc.c)
* Copyright (C) 2016 CNEX Labs
*/
#include "pblk.h"
+#include "pblk-trace.h"
static unsigned int write_buffer_size;
module_param(write_buffer_size, uint, 0644);
MODULE_PARM_DESC(write_buffer_size, "number of entries in a write buffer");
-static struct kmem_cache *pblk_ws_cache, *pblk_rec_cache, *pblk_g_rq_cache,
- *pblk_w_rq_cache;
-static DECLARE_RWSEM(pblk_lock);
+struct pblk_global_caches {
+ struct kmem_cache *ws;
+ struct kmem_cache *rec;
+ struct kmem_cache *g_rq;
+ struct kmem_cache *w_rq;
+
+ struct kref kref;
+
+ struct mutex mutex; /* Ensures consistency between
+ * caches and kref
+ */
+};
+
+static struct pblk_global_caches pblk_caches = {
+ .mutex = __MUTEX_INITIALIZER(pblk_caches.mutex),
+ .kref = KREF_INIT(0),
+};
+
struct bio_set pblk_bio_set;
static int pblk_rw_io(struct request_queue *q, struct pblk *pblk,
if (pblk_rb_tear_down_check(&pblk->rwb))
pblk_err(pblk, "write buffer error on tear down\n");
- pblk_rb_data_free(&pblk->rwb);
- vfree(pblk_rb_entries_ref(&pblk->rwb));
+ pblk_rb_free(&pblk->rwb);
}
static int pblk_rwb_init(struct pblk *pblk)
{
struct nvm_tgt_dev *dev = pblk->dev;
struct nvm_geo *geo = &dev->geo;
- struct pblk_rb_entry *entries;
- unsigned long nr_entries, buffer_size;
- unsigned int power_size, power_seg_sz;
- int pgs_in_buffer;
+ unsigned long buffer_size;
+ int pgs_in_buffer, threshold;
- pgs_in_buffer = max(geo->mw_cunits, geo->ws_opt) * geo->all_luns;
+ threshold = geo->mw_cunits * geo->all_luns;
+ pgs_in_buffer = (max(geo->mw_cunits, geo->ws_opt) + geo->ws_opt)
+ * geo->all_luns;
if (write_buffer_size && (write_buffer_size > pgs_in_buffer))
buffer_size = write_buffer_size;
else
buffer_size = pgs_in_buffer;
- nr_entries = pblk_rb_calculate_size(buffer_size);
-
- entries = vzalloc(array_size(nr_entries, sizeof(struct pblk_rb_entry)));
- if (!entries)
- return -ENOMEM;
-
- power_size = get_count_order(nr_entries);
- power_seg_sz = get_count_order(geo->csecs);
-
- return pblk_rb_init(&pblk->rwb, entries, power_size, power_seg_sz);
+ return pblk_rb_init(&pblk->rwb, buffer_size, threshold, geo->csecs);
}
/* Minimum pages needed within a lun */
return 0;
}
-static int pblk_init_global_caches(struct pblk *pblk)
+static int pblk_create_global_caches(void)
{
- down_write(&pblk_lock);
- pblk_ws_cache = kmem_cache_create("pblk_blk_ws",
+
+ pblk_caches.ws = kmem_cache_create("pblk_blk_ws",
sizeof(struct pblk_line_ws), 0, 0, NULL);
- if (!pblk_ws_cache) {
- up_write(&pblk_lock);
+ if (!pblk_caches.ws)
return -ENOMEM;
- }
- pblk_rec_cache = kmem_cache_create("pblk_rec",
+ pblk_caches.rec = kmem_cache_create("pblk_rec",
sizeof(struct pblk_rec_ctx), 0, 0, NULL);
- if (!pblk_rec_cache) {
- kmem_cache_destroy(pblk_ws_cache);
- up_write(&pblk_lock);
- return -ENOMEM;
- }
+ if (!pblk_caches.rec)
+ goto fail_destroy_ws;
- pblk_g_rq_cache = kmem_cache_create("pblk_g_rq", pblk_g_rq_size,
+ pblk_caches.g_rq = kmem_cache_create("pblk_g_rq", pblk_g_rq_size,
0, 0, NULL);
- if (!pblk_g_rq_cache) {
- kmem_cache_destroy(pblk_ws_cache);
- kmem_cache_destroy(pblk_rec_cache);
- up_write(&pblk_lock);
- return -ENOMEM;
- }
+ if (!pblk_caches.g_rq)
+ goto fail_destroy_rec;
- pblk_w_rq_cache = kmem_cache_create("pblk_w_rq", pblk_w_rq_size,
+ pblk_caches.w_rq = kmem_cache_create("pblk_w_rq", pblk_w_rq_size,
0, 0, NULL);
- if (!pblk_w_rq_cache) {
- kmem_cache_destroy(pblk_ws_cache);
- kmem_cache_destroy(pblk_rec_cache);
- kmem_cache_destroy(pblk_g_rq_cache);
- up_write(&pblk_lock);
- return -ENOMEM;
- }
- up_write(&pblk_lock);
+ if (!pblk_caches.w_rq)
+ goto fail_destroy_g_rq;
return 0;
+
+fail_destroy_g_rq:
+ kmem_cache_destroy(pblk_caches.g_rq);
+fail_destroy_rec:
+ kmem_cache_destroy(pblk_caches.rec);
+fail_destroy_ws:
+ kmem_cache_destroy(pblk_caches.ws);
+
+ return -ENOMEM;
}
-static void pblk_free_global_caches(struct pblk *pblk)
+static int pblk_get_global_caches(void)
{
- kmem_cache_destroy(pblk_ws_cache);
- kmem_cache_destroy(pblk_rec_cache);
- kmem_cache_destroy(pblk_g_rq_cache);
- kmem_cache_destroy(pblk_w_rq_cache);
+ int ret;
+
+ mutex_lock(&pblk_caches.mutex);
+
+ if (kref_read(&pblk_caches.kref) > 0) {
+ kref_get(&pblk_caches.kref);
+ mutex_unlock(&pblk_caches.mutex);
+ return 0;
+ }
+
+ ret = pblk_create_global_caches();
+
+ if (!ret)
+ kref_get(&pblk_caches.kref);
+
+ mutex_unlock(&pblk_caches.mutex);
+
+ return ret;
+}
+
+static void pblk_destroy_global_caches(struct kref *ref)
+{
+ struct pblk_global_caches *c;
+
+ c = container_of(ref, struct pblk_global_caches, kref);
+
+ kmem_cache_destroy(c->ws);
+ kmem_cache_destroy(c->rec);
+ kmem_cache_destroy(c->g_rq);
+ kmem_cache_destroy(c->w_rq);
+}
+
+static void pblk_put_global_caches(void)
+{
+ mutex_lock(&pblk_caches.mutex);
+ kref_put(&pblk_caches.kref, pblk_destroy_global_caches);
+ mutex_unlock(&pblk_caches.mutex);
}
static int pblk_core_init(struct pblk *pblk)
atomic64_set(&pblk->nr_flush, 0);
pblk->nr_flush_rst = 0;
- pblk->min_write_pgs = geo->ws_opt * (geo->csecs / PAGE_SIZE);
+ pblk->min_write_pgs = geo->ws_opt;
max_write_ppas = pblk->min_write_pgs * geo->all_luns;
pblk->max_write_pgs = min_t(int, max_write_ppas, NVM_MAX_VLBA);
+ pblk->max_write_pgs = min_t(int, pblk->max_write_pgs,
+ queue_max_hw_sectors(dev->q) / (geo->csecs >> SECTOR_SHIFT));
pblk_set_sec_per_write(pblk, pblk->min_write_pgs);
- if (pblk->max_write_pgs > PBLK_MAX_REQ_ADDRS) {
- pblk_err(pblk, "vector list too big(%u > %u)\n",
- pblk->max_write_pgs, PBLK_MAX_REQ_ADDRS);
- return -EINVAL;
- }
-
pblk->pad_dist = kcalloc(pblk->min_write_pgs - 1, sizeof(atomic64_t),
GFP_KERNEL);
if (!pblk->pad_dist)
return -ENOMEM;
- if (pblk_init_global_caches(pblk))
+ if (pblk_get_global_caches())
goto fail_free_pad_dist;
/* Internal bios can be at most the sectors signaled by the device. */
goto free_global_caches;
ret = mempool_init_slab_pool(&pblk->gen_ws_pool, PBLK_GEN_WS_POOL_SIZE,
- pblk_ws_cache);
+ pblk_caches.ws);
if (ret)
goto free_page_bio_pool;
ret = mempool_init_slab_pool(&pblk->rec_pool, geo->all_luns,
- pblk_rec_cache);
+ pblk_caches.rec);
if (ret)
goto free_gen_ws_pool;
ret = mempool_init_slab_pool(&pblk->r_rq_pool, geo->all_luns,
- pblk_g_rq_cache);
+ pblk_caches.g_rq);
if (ret)
goto free_rec_pool;
ret = mempool_init_slab_pool(&pblk->e_rq_pool, geo->all_luns,
- pblk_g_rq_cache);
+ pblk_caches.g_rq);
if (ret)
goto free_r_rq_pool;
ret = mempool_init_slab_pool(&pblk->w_rq_pool, geo->all_luns,
- pblk_w_rq_cache);
+ pblk_caches.w_rq);
if (ret)
goto free_e_rq_pool;
free_page_bio_pool:
mempool_exit(&pblk->page_bio_pool);
free_global_caches:
- pblk_free_global_caches(pblk);
+ pblk_put_global_caches();
fail_free_pad_dist:
kfree(pblk->pad_dist);
return -ENOMEM;
mempool_exit(&pblk->e_rq_pool);
mempool_exit(&pblk->w_rq_pool);
- pblk_free_global_caches(pblk);
+ pblk_put_global_caches();
kfree(pblk->pad_dist);
}
pblk_mfree(l_mg->eline_meta[i]->buf, l_mg->emeta_alloc_type);
kfree(l_mg->eline_meta[i]);
}
+
+ mempool_destroy(l_mg->bitmap_pool);
+ kmem_cache_destroy(l_mg->bitmap_cache);
}
static void pblk_line_meta_free(struct pblk_line_mgmt *l_mg,
kfree(pblk->lines);
}
-static int pblk_bb_get_tbl(struct nvm_tgt_dev *dev, struct pblk_lun *rlun,
- u8 *blks, int nr_blks)
-{
- struct ppa_addr ppa;
- int ret;
-
- ppa.ppa = 0;
- ppa.g.ch = rlun->bppa.g.ch;
- ppa.g.lun = rlun->bppa.g.lun;
-
- ret = nvm_get_tgt_bb_tbl(dev, ppa, blks);
- if (ret)
- return ret;
-
- nr_blks = nvm_bb_tbl_fold(dev->parent, blks, nr_blks);
- if (nr_blks < 0)
- return -EIO;
-
- return 0;
-}
-
-static void *pblk_bb_get_meta(struct pblk *pblk)
-{
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
- u8 *meta;
- int i, nr_blks, blk_per_lun;
- int ret;
-
- blk_per_lun = geo->num_chk * geo->pln_mode;
- nr_blks = blk_per_lun * geo->all_luns;
-
- meta = kmalloc(nr_blks, GFP_KERNEL);
- if (!meta)
- return ERR_PTR(-ENOMEM);
-
- for (i = 0; i < geo->all_luns; i++) {
- struct pblk_lun *rlun = &pblk->luns[i];
- u8 *meta_pos = meta + i * blk_per_lun;
-
- ret = pblk_bb_get_tbl(dev, rlun, meta_pos, blk_per_lun);
- if (ret) {
- kfree(meta);
- return ERR_PTR(-EIO);
- }
- }
-
- return meta;
-}
-
-static void *pblk_chunk_get_meta(struct pblk *pblk)
-{
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
-
- if (geo->version == NVM_OCSSD_SPEC_12)
- return pblk_bb_get_meta(pblk);
- else
- return pblk_chunk_get_info(pblk);
-}
-
static int pblk_luns_init(struct pblk *pblk)
{
struct nvm_tgt_dev *dev = pblk->dev;
atomic_set(&pblk->rl.free_user_blocks, nr_free_blks);
}
-static int pblk_setup_line_meta_12(struct pblk *pblk, struct pblk_line *line,
- void *chunk_meta)
-{
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
- struct pblk_line_meta *lm = &pblk->lm;
- int i, chk_per_lun, nr_bad_chks = 0;
-
- chk_per_lun = geo->num_chk * geo->pln_mode;
-
- for (i = 0; i < lm->blk_per_line; i++) {
- struct pblk_lun *rlun = &pblk->luns[i];
- struct nvm_chk_meta *chunk;
- int pos = pblk_ppa_to_pos(geo, rlun->bppa);
- u8 *lun_bb_meta = chunk_meta + pos * chk_per_lun;
-
- chunk = &line->chks[pos];
-
- /*
- * In 1.2 spec. chunk state is not persisted by the device. Thus
- * some of the values are reset each time pblk is instantiated,
- * so we have to assume that the block is closed.
- */
- if (lun_bb_meta[line->id] == NVM_BLK_T_FREE)
- chunk->state = NVM_CHK_ST_CLOSED;
- else
- chunk->state = NVM_CHK_ST_OFFLINE;
-
- chunk->type = NVM_CHK_TP_W_SEQ;
- chunk->wi = 0;
- chunk->slba = -1;
- chunk->cnlb = geo->clba;
- chunk->wp = 0;
-
- if (!(chunk->state & NVM_CHK_ST_OFFLINE))
- continue;
-
- set_bit(pos, line->blk_bitmap);
- nr_bad_chks++;
- }
-
- return nr_bad_chks;
-}
-
-static int pblk_setup_line_meta_20(struct pblk *pblk, struct pblk_line *line,
+static int pblk_setup_line_meta_chk(struct pblk *pblk, struct pblk_line *line,
struct nvm_chk_meta *meta)
{
struct nvm_tgt_dev *dev = pblk->dev;
chunk->cnlb = chunk_meta->cnlb;
chunk->wp = chunk_meta->wp;
+ trace_pblk_chunk_state(pblk_disk_name(pblk), &ppa,
+ chunk->state);
+
if (chunk->type & NVM_CHK_TP_SZ_SPEC) {
WARN_ONCE(1, "pblk: custom-sized chunks unsupported\n");
continue;
static long pblk_setup_line_meta(struct pblk *pblk, struct pblk_line *line,
void *chunk_meta, int line_id)
{
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
struct pblk_line_mgmt *l_mg = &pblk->l_mg;
struct pblk_line_meta *lm = &pblk->lm;
long nr_bad_chks, chk_in_line;
line->vsc = &l_mg->vsc_list[line_id];
spin_lock_init(&line->lock);
- if (geo->version == NVM_OCSSD_SPEC_12)
- nr_bad_chks = pblk_setup_line_meta_12(pblk, line, chunk_meta);
- else
- nr_bad_chks = pblk_setup_line_meta_20(pblk, line, chunk_meta);
+ nr_bad_chks = pblk_setup_line_meta_chk(pblk, line, chunk_meta);
chk_in_line = lm->blk_per_line - nr_bad_chks;
if (nr_bad_chks < 0 || nr_bad_chks > lm->blk_per_line ||
goto fail_free_smeta;
}
+ l_mg->bitmap_cache = kmem_cache_create("pblk_lm_bitmap",
+ lm->sec_bitmap_len, 0, 0, NULL);
+ if (!l_mg->bitmap_cache)
+ goto fail_free_smeta;
+
+ /* the bitmap pool is used for both valid and map bitmaps */
+ l_mg->bitmap_pool = mempool_create_slab_pool(PBLK_DATA_LINES * 2,
+ l_mg->bitmap_cache);
+ if (!l_mg->bitmap_pool)
+ goto fail_destroy_bitmap_cache;
+
/* emeta allocates three different buffers for managing metadata with
* in-memory and in-media layouts
*/
kfree(l_mg->eline_meta[i]->buf);
kfree(l_mg->eline_meta[i]);
}
+
+ mempool_destroy(l_mg->bitmap_pool);
+fail_destroy_bitmap_cache:
+ kmem_cache_destroy(l_mg->bitmap_cache);
fail_free_smeta:
for (i = 0; i < PBLK_DATA_LINES; i++)
kfree(l_mg->sline_meta[i]);
if (ret)
goto fail_free_meta;
- chunk_meta = pblk_chunk_get_meta(pblk);
+ chunk_meta = pblk_get_chunk_meta(pblk);
if (IS_ERR(chunk_meta)) {
ret = PTR_ERR(chunk_meta);
goto fail_free_luns;
goto fail_free_lines;
nr_free_chks += pblk_setup_line_meta(pblk, line, chunk_meta, i);
+
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
}
if (!nr_free_chks) {
pblk_err(pblk, "too many bad blocks prevent for sane instance\n");
- return -EINTR;
+ ret = -EINTR;
+ goto fail_free_lines;
}
pblk_set_provision(pblk, nr_free_chks);
- kfree(chunk_meta);
+ vfree(chunk_meta);
return 0;
fail_free_lines:
{
struct pblk *pblk = private;
- down_write(&pblk_lock);
pblk_gc_exit(pblk, graceful);
pblk_tear_down(pblk, graceful);
#endif
pblk_free(pblk);
- up_write(&pblk_lock);
}
static sector_t pblk_capacity(void *private)
pblk->dev = dev;
pblk->disk = tdisk;
pblk->state = PBLK_STATE_RUNNING;
+ trace_pblk_state(pblk_disk_name(pblk), pblk->state);
pblk->gc.gc_enabled = 0;
if (!(geo->version == NVM_OCSSD_SPEC_12 ||
return ERR_PTR(-EINVAL);
}
- if (geo->version == NVM_OCSSD_SPEC_12 && geo->dom & NVM_RSP_L2P) {
- pblk_err(pblk, "host-side L2P table not supported. (%x)\n",
- geo->dom);
- kfree(pblk);
- return ERR_PTR(-EINVAL);
- }
-
spin_lock_init(&pblk->resubmit_lock);
spin_lock_init(&pblk->trans_lock);
spin_lock_init(&pblk->lock);
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2016 CNEX Labs
* Initial release: Javier Gonzalez <javier@cnexlabs.com>
}
}
- pblk_down_rq(pblk, ppa_list, nr_secs, lun_bitmap);
+ pblk_down_rq(pblk, ppa_list[0], lun_bitmap);
return 0;
}
unsigned int off)
{
struct pblk_sec_meta *meta_list = rqd->meta_list;
+ struct ppa_addr *ppa_list = nvm_rq_to_ppa_list(rqd);
unsigned int map_secs;
int min = pblk->min_write_pgs;
int i;
for (i = off; i < rqd->nr_ppas; i += min) {
map_secs = (i + min > valid_secs) ? (valid_secs % min) : min;
- if (pblk_map_page_data(pblk, sentry + i, &rqd->ppa_list[i],
+ if (pblk_map_page_data(pblk, sentry + i, &ppa_list[i],
lun_bitmap, &meta_list[i], map_secs)) {
bio_put(rqd->bio);
pblk_free_rqd(pblk, rqd, PBLK_WRITE);
struct nvm_geo *geo = &dev->geo;
struct pblk_line_meta *lm = &pblk->lm;
struct pblk_sec_meta *meta_list = rqd->meta_list;
+ struct ppa_addr *ppa_list = nvm_rq_to_ppa_list(rqd);
struct pblk_line *e_line, *d_line;
unsigned int map_secs;
int min = pblk->min_write_pgs;
for (i = 0; i < rqd->nr_ppas; i += min) {
map_secs = (i + min > valid_secs) ? (valid_secs % min) : min;
- if (pblk_map_page_data(pblk, sentry + i, &rqd->ppa_list[i],
+ if (pblk_map_page_data(pblk, sentry + i, &ppa_list[i],
lun_bitmap, &meta_list[i], map_secs)) {
bio_put(rqd->bio);
pblk_free_rqd(pblk, rqd, PBLK_WRITE);
pblk_pipeline_stop(pblk);
}
- erase_lun = pblk_ppa_to_pos(geo, rqd->ppa_list[i]);
+ erase_lun = pblk_ppa_to_pos(geo, ppa_list[i]);
/* line can change after page map. We might also be writing the
* last line.
set_bit(erase_lun, e_line->erase_bitmap);
atomic_dec(&e_line->left_eblks);
- *erase_ppa = rqd->ppa_list[i];
+ *erase_ppa = ppa_list[i];
erase_ppa->a.blk = e_line->id;
spin_unlock(&e_line->lock);
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2016 CNEX Labs
* Initial release: Javier Gonzalez <javier@cnexlabs.com>
static DECLARE_RWSEM(pblk_rb_lock);
-void pblk_rb_data_free(struct pblk_rb *rb)
+static void pblk_rb_data_free(struct pblk_rb *rb)
{
struct pblk_rb_pages *p, *t;
up_write(&pblk_rb_lock);
}
+void pblk_rb_free(struct pblk_rb *rb)
+{
+ pblk_rb_data_free(rb);
+ vfree(rb->entries);
+}
+
+/*
+ * pblk_rb_calculate_size -- calculate the size of the write buffer
+ */
+static unsigned int pblk_rb_calculate_size(unsigned int nr_entries)
+{
+ /* Alloc a write buffer that can at least fit 128 entries */
+ return (1 << max(get_count_order(nr_entries), 7));
+}
+
/*
* Initialize ring buffer. The data and metadata buffers must be previously
* allocated and their size must be a power of two
* (Documentation/core-api/circular-buffers.rst)
*/
-int pblk_rb_init(struct pblk_rb *rb, struct pblk_rb_entry *rb_entry_base,
- unsigned int power_size, unsigned int power_seg_sz)
+int pblk_rb_init(struct pblk_rb *rb, unsigned int size, unsigned int threshold,
+ unsigned int seg_size)
{
struct pblk *pblk = container_of(rb, struct pblk, rwb);
+ struct pblk_rb_entry *entries;
unsigned int init_entry = 0;
- unsigned int alloc_order = power_size;
unsigned int max_order = MAX_ORDER - 1;
- unsigned int order, iter;
+ unsigned int power_size, power_seg_sz;
+ unsigned int alloc_order, order, iter;
+ unsigned int nr_entries;
+
+ nr_entries = pblk_rb_calculate_size(size);
+ entries = vzalloc(array_size(nr_entries, sizeof(struct pblk_rb_entry)));
+ if (!entries)
+ return -ENOMEM;
+
+ power_size = get_count_order(size);
+ power_seg_sz = get_count_order(seg_size);
down_write(&pblk_rb_lock);
- rb->entries = rb_entry_base;
+ rb->entries = entries;
rb->seg_size = (1 << power_seg_sz);
rb->nr_entries = (1 << power_size);
rb->mem = rb->subm = rb->sync = rb->l2p_update = 0;
+ rb->back_thres = threshold;
rb->flush_point = EMPTY_ENTRY;
spin_lock_init(&rb->w_lock);
INIT_LIST_HEAD(&rb->pages);
+ alloc_order = power_size;
if (alloc_order >= max_order) {
order = max_order;
iter = (1 << (alloc_order - max_order));
page_set = kmalloc(sizeof(struct pblk_rb_pages), GFP_KERNEL);
if (!page_set) {
up_write(&pblk_rb_lock);
+ vfree(entries);
return -ENOMEM;
}
kfree(page_set);
pblk_rb_data_free(rb);
up_write(&pblk_rb_lock);
+ vfree(entries);
return -ENOMEM;
}
kaddr = page_address(page_set->pages);
return 0;
}
-/*
- * pblk_rb_calculate_size -- calculate the size of the write buffer
- */
-unsigned int pblk_rb_calculate_size(unsigned int nr_entries)
-{
- /* Alloc a write buffer that can at least fit 128 entries */
- return (1 << max(get_count_order(nr_entries), 7));
-}
-
-void *pblk_rb_entries_ref(struct pblk_rb *rb)
-{
- return rb->entries;
-}
-
static void clean_wctx(struct pblk_w_ctx *w_ctx)
{
int flags;
return pblk_rb_ring_space(rb, mem, sync, rb->nr_entries);
}
+unsigned int pblk_rb_ptr_wrap(struct pblk_rb *rb, unsigned int p,
+ unsigned int nr_entries)
+{
+ return (p + nr_entries) & (rb->nr_entries - 1);
+}
+
/*
* Buffer count is calculated with respect to the submission entry signaling the
* entries that are available to send to the media
subm = READ_ONCE(rb->subm);
/* Commit read means updating submission pointer */
- smp_store_release(&rb->subm,
- (subm + nr_entries) & (rb->nr_entries - 1));
+ smp_store_release(&rb->subm, pblk_rb_ptr_wrap(rb, subm, nr_entries));
return subm;
}
pblk_update_map_dev(pblk, w_ctx->lba, w_ctx->ppa,
entry->cacheline);
- line = &pblk->lines[pblk_ppa_to_line(w_ctx->ppa)];
+ line = pblk_ppa_to_line(pblk, w_ctx->ppa);
kref_put(&line->ref, pblk_line_put);
clean_wctx(w_ctx);
- rb->l2p_update = (rb->l2p_update + 1) & (rb->nr_entries - 1);
+ rb->l2p_update = pblk_rb_ptr_wrap(rb, rb->l2p_update, 1);
}
pblk_rl_out(&pblk->rl, user_io, gc_io);
{
unsigned int mem;
unsigned int sync;
+ unsigned int threshold;
sync = READ_ONCE(rb->sync);
mem = READ_ONCE(rb->mem);
- if (pblk_rb_ring_space(rb, mem, sync, rb->nr_entries) < nr_entries)
+ threshold = nr_entries + rb->back_thres;
+
+ if (pblk_rb_ring_space(rb, mem, sync, rb->nr_entries) < threshold)
return 0;
if (pblk_rb_update_l2p(rb, nr_entries, mem, sync))
return 0;
/* Protect from read count */
- smp_store_release(&rb->mem, (*pos + nr_entries) & (rb->nr_entries - 1));
+ smp_store_release(&rb->mem, pblk_rb_ptr_wrap(rb, *pos, nr_entries));
return 1;
}
if (!__pblk_rb_may_write(rb, nr_entries, pos))
return 0;
- mem = (*pos + nr_entries) & (rb->nr_entries - 1);
+ mem = pblk_rb_ptr_wrap(rb, *pos, nr_entries);
*io_ret = NVM_IO_DONE;
if (bio->bi_opf & REQ_PREFLUSH) {
/* Release flags on context. Protect from writes */
smp_store_release(&entry->w_ctx.flags, flags);
- pos = (pos + 1) & (rb->nr_entries - 1);
+ pos = pblk_rb_ptr_wrap(rb, pos, 1);
}
if (pad) {
struct pblk_w_ctx *pblk_rb_w_ctx(struct pblk_rb *rb, unsigned int pos)
{
- unsigned int entry = pos & (rb->nr_entries - 1);
+ unsigned int entry = pblk_rb_ptr_wrap(rb, pos, 0);
return &rb->entries[entry].w_ctx;
}
}
}
- sync = (sync + nr_entries) & (rb->nr_entries - 1);
+ sync = pblk_rb_ptr_wrap(rb, sync, nr_entries);
/* Protect from counts */
smp_store_release(&rb->sync, sync);
return (submitted < to_flush) ? (to_flush - submitted) : 0;
}
-/*
- * Scan from the current position of the sync pointer to find the entry that
- * corresponds to the given ppa. This is necessary since write requests can be
- * completed out of order. The assumption is that the ppa is close to the sync
- * pointer thus the search will not take long.
- *
- * The caller of this function must guarantee that the sync pointer will no
- * reach the entry while it is using the metadata associated with it. With this
- * assumption in mind, there is no need to take the sync lock.
- */
-struct pblk_rb_entry *pblk_rb_sync_scan_entry(struct pblk_rb *rb,
- struct ppa_addr *ppa)
-{
- unsigned int sync, subm, count;
- unsigned int i;
-
- sync = READ_ONCE(rb->sync);
- subm = READ_ONCE(rb->subm);
- count = pblk_rb_ring_count(subm, sync, rb->nr_entries);
-
- for (i = 0; i < count; i++)
- sync = (sync + 1) & (rb->nr_entries - 1);
-
- return NULL;
-}
-
int pblk_rb_tear_down_check(struct pblk_rb *rb)
{
struct pblk_rb_entry *entry;
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2016 CNEX Labs
* Initial release: Javier Gonzalez <javier@cnexlabs.com>
unsigned long *read_bitmap)
{
struct pblk_sec_meta *meta_list = rqd->meta_list;
- struct ppa_addr ppas[PBLK_MAX_REQ_ADDRS];
+ struct ppa_addr ppas[NVM_MAX_VLBA];
int nr_secs = rqd->nr_ppas;
bool advanced_bio = false;
int i, j = 0;
}
if (pblk_io_aligned(pblk, nr_secs))
- rqd->flags = pblk_set_read_mode(pblk, PBLK_READ_SEQUENTIAL);
- else
- rqd->flags = pblk_set_read_mode(pblk, PBLK_READ_RANDOM);
+ rqd->is_seq = 1;
#ifdef CONFIG_NVM_PBLK_DEBUG
atomic_long_add(nr_secs, &pblk->inflight_reads);
if (lba != blba + i) {
#ifdef CONFIG_NVM_PBLK_DEBUG
- struct ppa_addr *p;
+ struct ppa_addr *ppa_list = nvm_rq_to_ppa_list(rqd);
- p = (nr_lbas == 1) ? &rqd->ppa_list[i] : &rqd->ppa_addr;
- print_ppa(pblk, p, "seq", i);
+ print_ppa(pblk, &ppa_list[i], "seq", i);
#endif
pblk_err(pblk, "corrupted read LBA (%llu/%llu)\n",
lba, (u64)blba + i);
if (lba != meta_lba) {
#ifdef CONFIG_NVM_PBLK_DEBUG
- struct ppa_addr *p;
- int nr_ppas = rqd->nr_ppas;
+ struct ppa_addr *ppa_list = nvm_rq_to_ppa_list(rqd);
- p = (nr_ppas == 1) ? &rqd->ppa_list[j] : &rqd->ppa_addr;
- print_ppa(pblk, p, "seq", j);
+ print_ppa(pblk, &ppa_list[j], "rnd", j);
#endif
pblk_err(pblk, "corrupted read LBA (%llu/%llu)\n",
- lba, meta_lba);
+ meta_lba, lba);
WARN_ON(1);
}
WARN_ONCE(j != rqd->nr_ppas, "pblk: corrupted random request\n");
}
-static void pblk_read_put_rqd_kref(struct pblk *pblk, struct nvm_rq *rqd)
-{
- struct ppa_addr *ppa_list;
- int i;
-
- ppa_list = (rqd->nr_ppas > 1) ? rqd->ppa_list : &rqd->ppa_addr;
-
- for (i = 0; i < rqd->nr_ppas; i++) {
- struct ppa_addr ppa = ppa_list[i];
- struct pblk_line *line;
-
- line = &pblk->lines[pblk_ppa_to_line(ppa)];
- kref_put(&line->ref, pblk_line_put_wq);
- }
-}
-
static void pblk_end_user_read(struct bio *bio)
{
#ifdef CONFIG_NVM_PBLK_DEBUG
bio_put(int_bio);
if (put_line)
- pblk_read_put_rqd_kref(pblk, rqd);
+ pblk_rq_to_line_put(pblk, rqd);
#ifdef CONFIG_NVM_PBLK_DEBUG
atomic_long_add(rqd->nr_ppas, &pblk->sync_reads);
i = 0;
hole = find_first_zero_bit(read_bitmap, nr_secs);
do {
- int line_id = pblk_ppa_to_line(rqd->ppa_list[i]);
- struct pblk_line *line = &pblk->lines[line_id];
+ struct pblk_line *line;
+ line = pblk_ppa_to_line(pblk, rqd->ppa_list[i]);
kref_put(&line->ref, pblk_line_put);
meta_list[hole].lba = lba_list_media[i];
rqd->bio = new_bio;
rqd->nr_ppas = nr_holes;
- rqd->flags = pblk_set_read_mode(pblk, PBLK_READ_RANDOM);
pr_ctx->ppa_ptr = NULL;
pr_ctx->orig_bio = bio;
} else {
rqd->ppa_addr = ppa;
}
-
- rqd->flags = pblk_set_read_mode(pblk, PBLK_READ_RANDOM);
}
int pblk_submit_read(struct pblk *pblk, struct bio *bio)
DECLARE_BITMAP(read_bitmap, NVM_MAX_VLBA);
int ret = NVM_IO_ERR;
- /* logic error: lba out-of-bounds. Ignore read request */
- if (blba >= pblk->rl.nr_secs || nr_secs > PBLK_MAX_REQ_ADDRS) {
- WARN(1, "pblk: read lba out of bounds (lba:%llu, nr:%d)\n",
- (unsigned long long)blba, nr_secs);
- return NVM_IO_ERR;
- }
-
generic_start_io_acct(q, REQ_OP_READ, bio_sectors(bio),
&pblk->disk->part0);
*/
bio_init_idx = pblk_get_bi_idx(bio);
- rqd->meta_list = nvm_dev_dma_alloc(dev->parent, GFP_KERNEL,
- &rqd->dma_meta_list);
- if (!rqd->meta_list) {
- pblk_err(pblk, "not able to allocate ppa list\n");
+ if (pblk_alloc_rqd_meta(pblk, rqd))
goto fail_rqd_free;
- }
-
- if (nr_secs > 1) {
- rqd->ppa_list = rqd->meta_list + pblk_dma_meta_size;
- rqd->dma_ppa_list = rqd->dma_meta_list + pblk_dma_meta_size;
+ if (nr_secs > 1)
pblk_read_ppalist_rq(pblk, rqd, bio, blba, read_bitmap);
- } else {
+ else
pblk_read_rq(pblk, rqd, bio, blba, read_bitmap);
- }
if (bitmap_full(read_bitmap, nr_secs)) {
atomic_inc(&pblk->inflight_io);
struct pblk_line *line, u64 *lba_list,
u64 *paddr_list_gc, unsigned int nr_secs)
{
- struct ppa_addr ppa_list_l2p[PBLK_MAX_REQ_ADDRS];
+ struct ppa_addr ppa_list_l2p[NVM_MAX_VLBA];
struct ppa_addr ppa_gc;
int valid_secs = 0;
int i;
memset(&rqd, 0, sizeof(struct nvm_rq));
- rqd.meta_list = nvm_dev_dma_alloc(dev->parent, GFP_KERNEL,
- &rqd.dma_meta_list);
- if (!rqd.meta_list)
- return -ENOMEM;
+ ret = pblk_alloc_rqd_meta(pblk, &rqd);
+ if (ret)
+ return ret;
if (gc_rq->nr_secs > 1) {
- rqd.ppa_list = rqd.meta_list + pblk_dma_meta_size;
- rqd.dma_ppa_list = rqd.dma_meta_list + pblk_dma_meta_size;
-
gc_rq->secs_to_gc = read_ppalist_rq_gc(pblk, &rqd, gc_rq->line,
gc_rq->lba_list,
gc_rq->paddr_list,
PBLK_VMALLOC_META, GFP_KERNEL);
if (IS_ERR(bio)) {
pblk_err(pblk, "could not allocate GC bio (%lu)\n",
- PTR_ERR(bio));
+ PTR_ERR(bio));
+ ret = PTR_ERR(bio);
goto err_free_dma;
}
rqd.opcode = NVM_OP_PREAD;
rqd.nr_ppas = gc_rq->secs_to_gc;
- rqd.flags = pblk_set_read_mode(pblk, PBLK_READ_RANDOM);
rqd.bio = bio;
if (pblk_submit_io_sync(pblk, &rqd)) {
#endif
out:
- nvm_dev_dma_free(dev->parent, rqd.meta_list, rqd.dma_meta_list);
+ pblk_free_rqd_meta(pblk, &rqd);
return ret;
err_free_bio:
bio_put(bio);
err_free_dma:
- nvm_dev_dma_free(dev->parent, rqd.meta_list, rqd.dma_meta_list);
+ pblk_free_rqd_meta(pblk, &rqd);
return ret;
}
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2016 CNEX Labs
* Initial: Javier Gonzalez <javier@cnexlabs.com>
*/
#include "pblk.h"
+#include "pblk-trace.h"
int pblk_recov_check_emeta(struct pblk *pblk, struct line_emeta *emeta_buf)
{
return 0;
}
-static int pblk_calc_sec_in_line(struct pblk *pblk, struct pblk_line *line)
+static void pblk_update_line_wp(struct pblk *pblk, struct pblk_line *line,
+ u64 written_secs)
+{
+ int i;
+
+ for (i = 0; i < written_secs; i += pblk->min_write_pgs)
+ pblk_alloc_page(pblk, line, pblk->min_write_pgs);
+}
+
+static u64 pblk_sec_in_open_line(struct pblk *pblk, struct pblk_line *line)
{
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
struct pblk_line_meta *lm = &pblk->lm;
int nr_bb = bitmap_weight(line->blk_bitmap, lm->blk_per_line);
+ u64 written_secs = 0;
+ int valid_chunks = 0;
+ int i;
+
+ for (i = 0; i < lm->blk_per_line; i++) {
+ struct nvm_chk_meta *chunk = &line->chks[i];
+
+ if (chunk->state & NVM_CHK_ST_OFFLINE)
+ continue;
+
+ written_secs += chunk->wp;
+ valid_chunks++;
+ }
+
+ if (lm->blk_per_line - nr_bb != valid_chunks)
+ pblk_err(pblk, "recovery line %d is bad\n", line->id);
- return lm->sec_per_line - lm->smeta_sec - lm->emeta_sec[0] -
- nr_bb * geo->clba;
+ pblk_update_line_wp(pblk, line, written_secs - lm->smeta_sec);
+
+ return written_secs;
}
struct pblk_recov_alloc {
dma_addr_t dma_meta_list;
};
-static int pblk_recov_read_oob(struct pblk *pblk, struct pblk_line *line,
- struct pblk_recov_alloc p, u64 r_ptr)
-{
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
- struct ppa_addr *ppa_list;
- struct pblk_sec_meta *meta_list;
- struct nvm_rq *rqd;
- struct bio *bio;
- void *data;
- dma_addr_t dma_ppa_list, dma_meta_list;
- u64 r_ptr_int;
- int left_ppas;
- int rq_ppas, rq_len;
- int i, j;
- int ret = 0;
-
- ppa_list = p.ppa_list;
- meta_list = p.meta_list;
- rqd = p.rqd;
- data = p.data;
- dma_ppa_list = p.dma_ppa_list;
- dma_meta_list = p.dma_meta_list;
-
- left_ppas = line->cur_sec - r_ptr;
- if (!left_ppas)
- return 0;
-
- r_ptr_int = r_ptr;
-
-next_read_rq:
- memset(rqd, 0, pblk_g_rq_size);
-
- rq_ppas = pblk_calc_secs(pblk, left_ppas, 0);
- if (!rq_ppas)
- rq_ppas = pblk->min_write_pgs;
- rq_len = rq_ppas * geo->csecs;
-
- bio = bio_map_kern(dev->q, data, rq_len, GFP_KERNEL);
- if (IS_ERR(bio))
- return PTR_ERR(bio);
-
- bio->bi_iter.bi_sector = 0; /* internal bio */
- bio_set_op_attrs(bio, REQ_OP_READ, 0);
-
- rqd->bio = bio;
- rqd->opcode = NVM_OP_PREAD;
- rqd->meta_list = meta_list;
- rqd->nr_ppas = rq_ppas;
- rqd->ppa_list = ppa_list;
- rqd->dma_ppa_list = dma_ppa_list;
- rqd->dma_meta_list = dma_meta_list;
-
- if (pblk_io_aligned(pblk, rq_ppas))
- rqd->flags = pblk_set_read_mode(pblk, PBLK_READ_SEQUENTIAL);
- else
- rqd->flags = pblk_set_read_mode(pblk, PBLK_READ_RANDOM);
-
- for (i = 0; i < rqd->nr_ppas; ) {
- struct ppa_addr ppa;
- int pos;
-
- ppa = addr_to_gen_ppa(pblk, r_ptr_int, line->id);
- pos = pblk_ppa_to_pos(geo, ppa);
-
- while (test_bit(pos, line->blk_bitmap)) {
- r_ptr_int += pblk->min_write_pgs;
- ppa = addr_to_gen_ppa(pblk, r_ptr_int, line->id);
- pos = pblk_ppa_to_pos(geo, ppa);
- }
-
- for (j = 0; j < pblk->min_write_pgs; j++, i++, r_ptr_int++)
- rqd->ppa_list[i] =
- addr_to_gen_ppa(pblk, r_ptr_int, line->id);
- }
-
- /* If read fails, more padding is needed */
- ret = pblk_submit_io_sync(pblk, rqd);
- if (ret) {
- pblk_err(pblk, "I/O submission failed: %d\n", ret);
- return ret;
- }
-
- atomic_dec(&pblk->inflight_io);
-
- /* At this point, the read should not fail. If it does, it is a problem
- * we cannot recover from here. Need FTL log.
- */
- if (rqd->error && rqd->error != NVM_RSP_WARN_HIGHECC) {
- pblk_err(pblk, "L2P recovery failed (%d)\n", rqd->error);
- return -EINTR;
- }
-
- for (i = 0; i < rqd->nr_ppas; i++) {
- u64 lba = le64_to_cpu(meta_list[i].lba);
-
- if (lba == ADDR_EMPTY || lba > pblk->rl.nr_secs)
- continue;
-
- pblk_update_map(pblk, lba, rqd->ppa_list[i]);
- }
-
- left_ppas -= rq_ppas;
- if (left_ppas > 0)
- goto next_read_rq;
-
- return 0;
-}
-
static void pblk_recov_complete(struct kref *ref)
{
struct pblk_pad_rq *pad_rq = container_of(ref, struct pblk_pad_rq, ref);
static void pblk_end_io_recov(struct nvm_rq *rqd)
{
+ struct ppa_addr *ppa_list = nvm_rq_to_ppa_list(rqd);
struct pblk_pad_rq *pad_rq = rqd->private;
struct pblk *pblk = pad_rq->pblk;
- pblk_up_page(pblk, rqd->ppa_list, rqd->nr_ppas);
+ pblk_up_chunk(pblk, ppa_list[0]);
pblk_free_rqd(pblk, rqd, PBLK_WRITE_INT);
kref_put(&pad_rq->ref, pblk_recov_complete);
}
-static int pblk_recov_pad_oob(struct pblk *pblk, struct pblk_line *line,
- int left_ppas)
+/* pad line using line bitmap. */
+static int pblk_recov_pad_line(struct pblk *pblk, struct pblk_line *line,
+ int left_ppas)
{
struct nvm_tgt_dev *dev = pblk->dev;
struct nvm_geo *geo = &dev->geo;
- struct ppa_addr *ppa_list;
struct pblk_sec_meta *meta_list;
struct pblk_pad_rq *pad_rq;
struct nvm_rq *rqd;
struct bio *bio;
void *data;
- dma_addr_t dma_ppa_list, dma_meta_list;
__le64 *lba_list = emeta_to_lbas(pblk, line->emeta->buf);
u64 w_ptr = line->cur_sec;
int left_line_ppas, rq_ppas, rq_len;
rq_len = rq_ppas * geo->csecs;
- meta_list = nvm_dev_dma_alloc(dev->parent, GFP_KERNEL, &dma_meta_list);
- if (!meta_list) {
- ret = -ENOMEM;
- goto fail_free_pad;
- }
-
- ppa_list = (void *)(meta_list) + pblk_dma_meta_size;
- dma_ppa_list = dma_meta_list + pblk_dma_meta_size;
-
bio = pblk_bio_map_addr(pblk, data, rq_ppas, rq_len,
PBLK_VMALLOC_META, GFP_KERNEL);
if (IS_ERR(bio)) {
ret = PTR_ERR(bio);
- goto fail_free_meta;
+ goto fail_free_pad;
}
bio->bi_iter.bi_sector = 0; /* internal bio */
rqd = pblk_alloc_rqd(pblk, PBLK_WRITE_INT);
+ ret = pblk_alloc_rqd_meta(pblk, rqd);
+ if (ret)
+ goto fail_free_rqd;
+
rqd->bio = bio;
rqd->opcode = NVM_OP_PWRITE;
- rqd->flags = pblk_set_progr_mode(pblk, PBLK_WRITE);
- rqd->meta_list = meta_list;
+ rqd->is_seq = 1;
rqd->nr_ppas = rq_ppas;
- rqd->ppa_list = ppa_list;
- rqd->dma_ppa_list = dma_ppa_list;
- rqd->dma_meta_list = dma_meta_list;
rqd->end_io = pblk_end_io_recov;
rqd->private = pad_rq;
+ meta_list = rqd->meta_list;
+
for (i = 0; i < rqd->nr_ppas; ) {
struct ppa_addr ppa;
int pos;
}
kref_get(&pad_rq->ref);
- pblk_down_page(pblk, rqd->ppa_list, rqd->nr_ppas);
+ pblk_down_chunk(pblk, rqd->ppa_list[0]);
ret = pblk_submit_io(pblk, rqd);
if (ret) {
pblk_err(pblk, "I/O submission failed: %d\n", ret);
- pblk_up_page(pblk, rqd->ppa_list, rqd->nr_ppas);
- goto fail_free_bio;
+ pblk_up_chunk(pblk, rqd->ppa_list[0]);
+ goto fail_free_rqd;
}
left_line_ppas -= rq_ppas;
kfree(pad_rq);
return ret;
-fail_free_bio:
+fail_free_rqd:
+ pblk_free_rqd(pblk, rqd, PBLK_WRITE_INT);
bio_put(bio);
-fail_free_meta:
- nvm_dev_dma_free(dev->parent, meta_list, dma_meta_list);
fail_free_pad:
kfree(pad_rq);
vfree(data);
return ret;
}
-/* When this function is called, it means that not all upper pages have been
- * written in a page that contains valid data. In order to recover this data, we
- * first find the write pointer on the device, then we pad all necessary
- * sectors, and finally attempt to read the valid data
- */
-static int pblk_recov_scan_all_oob(struct pblk *pblk, struct pblk_line *line,
- struct pblk_recov_alloc p)
+static int pblk_pad_distance(struct pblk *pblk, struct pblk_line *line)
{
struct nvm_tgt_dev *dev = pblk->dev;
struct nvm_geo *geo = &dev->geo;
- struct ppa_addr *ppa_list;
- struct pblk_sec_meta *meta_list;
- struct nvm_rq *rqd;
- struct bio *bio;
- void *data;
- dma_addr_t dma_ppa_list, dma_meta_list;
- u64 w_ptr = 0, r_ptr;
- int rq_ppas, rq_len;
- int i, j;
- int ret = 0;
- int rec_round;
- int left_ppas = pblk_calc_sec_in_line(pblk, line) - line->cur_sec;
-
- ppa_list = p.ppa_list;
- meta_list = p.meta_list;
- rqd = p.rqd;
- data = p.data;
- dma_ppa_list = p.dma_ppa_list;
- dma_meta_list = p.dma_meta_list;
-
- /* we could recover up until the line write pointer */
- r_ptr = line->cur_sec;
- rec_round = 0;
-
-next_rq:
- memset(rqd, 0, pblk_g_rq_size);
+ int distance = geo->mw_cunits * geo->all_luns * geo->ws_opt;
- rq_ppas = pblk_calc_secs(pblk, left_ppas, 0);
- if (!rq_ppas)
- rq_ppas = pblk->min_write_pgs;
- rq_len = rq_ppas * geo->csecs;
-
- bio = bio_map_kern(dev->q, data, rq_len, GFP_KERNEL);
- if (IS_ERR(bio))
- return PTR_ERR(bio);
-
- bio->bi_iter.bi_sector = 0; /* internal bio */
- bio_set_op_attrs(bio, REQ_OP_READ, 0);
+ return (distance > line->left_msecs) ? line->left_msecs : distance;
+}
- rqd->bio = bio;
- rqd->opcode = NVM_OP_PREAD;
- rqd->meta_list = meta_list;
- rqd->nr_ppas = rq_ppas;
- rqd->ppa_list = ppa_list;
- rqd->dma_ppa_list = dma_ppa_list;
- rqd->dma_meta_list = dma_meta_list;
+static int pblk_line_wp_is_unbalanced(struct pblk *pblk,
+ struct pblk_line *line)
+{
+ struct nvm_tgt_dev *dev = pblk->dev;
+ struct nvm_geo *geo = &dev->geo;
+ struct pblk_line_meta *lm = &pblk->lm;
+ struct pblk_lun *rlun;
+ struct nvm_chk_meta *chunk;
+ struct ppa_addr ppa;
+ u64 line_wp;
+ int pos, i;
- if (pblk_io_aligned(pblk, rq_ppas))
- rqd->flags = pblk_set_read_mode(pblk, PBLK_READ_SEQUENTIAL);
- else
- rqd->flags = pblk_set_read_mode(pblk, PBLK_READ_RANDOM);
+ rlun = &pblk->luns[0];
+ ppa = rlun->bppa;
+ pos = pblk_ppa_to_pos(geo, ppa);
+ chunk = &line->chks[pos];
- for (i = 0; i < rqd->nr_ppas; ) {
- struct ppa_addr ppa;
- int pos;
+ line_wp = chunk->wp;
- w_ptr = pblk_alloc_page(pblk, line, pblk->min_write_pgs);
- ppa = addr_to_gen_ppa(pblk, w_ptr, line->id);
+ for (i = 1; i < lm->blk_per_line; i++) {
+ rlun = &pblk->luns[i];
+ ppa = rlun->bppa;
pos = pblk_ppa_to_pos(geo, ppa);
+ chunk = &line->chks[pos];
- while (test_bit(pos, line->blk_bitmap)) {
- w_ptr += pblk->min_write_pgs;
- ppa = addr_to_gen_ppa(pblk, w_ptr, line->id);
- pos = pblk_ppa_to_pos(geo, ppa);
- }
-
- for (j = 0; j < pblk->min_write_pgs; j++, i++, w_ptr++)
- rqd->ppa_list[i] =
- addr_to_gen_ppa(pblk, w_ptr, line->id);
- }
-
- ret = pblk_submit_io_sync(pblk, rqd);
- if (ret) {
- pblk_err(pblk, "I/O submission failed: %d\n", ret);
- return ret;
- }
-
- atomic_dec(&pblk->inflight_io);
-
- /* This should not happen since the read failed during normal recovery,
- * but the media works funny sometimes...
- */
- if (!rec_round++ && !rqd->error) {
- rec_round = 0;
- for (i = 0; i < rqd->nr_ppas; i++, r_ptr++) {
- u64 lba = le64_to_cpu(meta_list[i].lba);
-
- if (lba == ADDR_EMPTY || lba > pblk->rl.nr_secs)
- continue;
-
- pblk_update_map(pblk, lba, rqd->ppa_list[i]);
- }
- }
-
- /* Reached the end of the written line */
- if (rqd->error == NVM_RSP_ERR_EMPTYPAGE) {
- int pad_secs, nr_error_bits, bit;
- int ret;
-
- bit = find_first_bit((void *)&rqd->ppa_status, rqd->nr_ppas);
- nr_error_bits = rqd->nr_ppas - bit;
-
- /* Roll back failed sectors */
- line->cur_sec -= nr_error_bits;
- line->left_msecs += nr_error_bits;
- bitmap_clear(line->map_bitmap, line->cur_sec, nr_error_bits);
-
- pad_secs = pblk_pad_distance(pblk);
- if (pad_secs > line->left_msecs)
- pad_secs = line->left_msecs;
-
- ret = pblk_recov_pad_oob(pblk, line, pad_secs);
- if (ret)
- pblk_err(pblk, "OOB padding failed (err:%d)\n", ret);
-
- ret = pblk_recov_read_oob(pblk, line, p, r_ptr);
- if (ret)
- pblk_err(pblk, "OOB read failed (err:%d)\n", ret);
-
- left_ppas = 0;
+ if (chunk->wp > line_wp)
+ return 1;
+ else if (chunk->wp < line_wp)
+ line_wp = chunk->wp;
}
- left_ppas -= rq_ppas;
- if (left_ppas > 0)
- goto next_rq;
-
- return ret;
+ return 0;
}
static int pblk_recov_scan_oob(struct pblk *pblk, struct pblk_line *line,
- struct pblk_recov_alloc p, int *done)
+ struct pblk_recov_alloc p)
{
struct nvm_tgt_dev *dev = pblk->dev;
struct nvm_geo *geo = &dev->geo;
struct bio *bio;
void *data;
dma_addr_t dma_ppa_list, dma_meta_list;
- u64 paddr;
+ __le64 *lba_list;
+ u64 paddr = 0;
+ bool padded = false;
int rq_ppas, rq_len;
int i, j;
- int ret = 0;
- int left_ppas = pblk_calc_sec_in_line(pblk, line);
+ int ret;
+ u64 left_ppas = pblk_sec_in_open_line(pblk, line);
+
+ if (pblk_line_wp_is_unbalanced(pblk, line))
+ pblk_warn(pblk, "recovering unbalanced line (%d)\n", line->id);
ppa_list = p.ppa_list;
meta_list = p.meta_list;
dma_ppa_list = p.dma_ppa_list;
dma_meta_list = p.dma_meta_list;
- *done = 1;
+ lba_list = emeta_to_lbas(pblk, line->emeta->buf);
next_rq:
memset(rqd, 0, pblk_g_rq_size);
rqd->dma_meta_list = dma_meta_list;
if (pblk_io_aligned(pblk, rq_ppas))
- rqd->flags = pblk_set_read_mode(pblk, PBLK_READ_SEQUENTIAL);
- else
- rqd->flags = pblk_set_read_mode(pblk, PBLK_READ_RANDOM);
+ rqd->is_seq = 1;
+retry_rq:
for (i = 0; i < rqd->nr_ppas; ) {
struct ppa_addr ppa;
int pos;
- paddr = pblk_alloc_page(pblk, line, pblk->min_write_pgs);
ppa = addr_to_gen_ppa(pblk, paddr, line->id);
pos = pblk_ppa_to_pos(geo, ppa);
pos = pblk_ppa_to_pos(geo, ppa);
}
- for (j = 0; j < pblk->min_write_pgs; j++, i++, paddr++)
+ for (j = 0; j < pblk->min_write_pgs; j++, i++)
rqd->ppa_list[i] =
- addr_to_gen_ppa(pblk, paddr, line->id);
+ addr_to_gen_ppa(pblk, paddr + j, line->id);
}
ret = pblk_submit_io_sync(pblk, rqd);
atomic_dec(&pblk->inflight_io);
- /* Reached the end of the written line */
+ /* If a read fails, do a best effort by padding the line and retrying */
if (rqd->error) {
- int nr_error_bits, bit;
+ int pad_distance, ret;
- bit = find_first_bit((void *)&rqd->ppa_status, rqd->nr_ppas);
- nr_error_bits = rqd->nr_ppas - bit;
-
- /* Roll back failed sectors */
- line->cur_sec -= nr_error_bits;
- line->left_msecs += nr_error_bits;
- bitmap_clear(line->map_bitmap, line->cur_sec, nr_error_bits);
+ if (padded) {
+ pblk_log_read_err(pblk, rqd);
+ return -EINTR;
+ }
- left_ppas = 0;
- rqd->nr_ppas = bit;
+ pad_distance = pblk_pad_distance(pblk, line);
+ ret = pblk_recov_pad_line(pblk, line, pad_distance);
+ if (ret)
+ return ret;
- if (rqd->error != NVM_RSP_ERR_EMPTYPAGE)
- *done = 0;
+ padded = true;
+ goto retry_rq;
}
for (i = 0; i < rqd->nr_ppas; i++) {
u64 lba = le64_to_cpu(meta_list[i].lba);
+ lba_list[paddr++] = cpu_to_le64(lba);
+
if (lba == ADDR_EMPTY || lba > pblk->rl.nr_secs)
continue;
+ line->nr_valid_lbas++;
pblk_update_map(pblk, lba, rqd->ppa_list[i]);
}
if (left_ppas > 0)
goto next_rq;
- return ret;
+#ifdef CONFIG_NVM_PBLK_DEBUG
+ WARN_ON(padded && !pblk_line_is_full(line));
+#endif
+
+ return 0;
}
/* Scan line for lbas on out of bound area */
struct pblk_recov_alloc p;
void *data;
dma_addr_t dma_ppa_list, dma_meta_list;
- int done, ret = 0;
+ int ret = 0;
meta_list = nvm_dev_dma_alloc(dev->parent, GFP_KERNEL, &dma_meta_list);
if (!meta_list)
goto free_meta_list;
}
- rqd = pblk_alloc_rqd(pblk, PBLK_READ);
+ rqd = mempool_alloc(&pblk->r_rq_pool, GFP_KERNEL);
+ memset(rqd, 0, pblk_g_rq_size);
p.ppa_list = ppa_list;
p.meta_list = meta_list;
p.dma_ppa_list = dma_ppa_list;
p.dma_meta_list = dma_meta_list;
- ret = pblk_recov_scan_oob(pblk, line, p, &done);
+ ret = pblk_recov_scan_oob(pblk, line, p);
if (ret) {
- pblk_err(pblk, "could not recover L2P from OOB\n");
+ pblk_err(pblk, "could not recover L2P form OOB\n");
goto out;
}
- if (!done) {
- ret = pblk_recov_scan_all_oob(pblk, line, p);
- if (ret) {
- pblk_err(pblk, "could not recover L2P from OOB\n");
- goto out;
- }
- }
-
if (pblk_line_is_full(line))
pblk_line_recov_close(pblk, line);
out:
+ mempool_free(rqd, &pblk->r_rq_pool);
kfree(data);
free_meta_list:
nvm_dev_dma_free(dev->parent, meta_list, dma_meta_list);
}
static int pblk_line_was_written(struct pblk_line *line,
- struct pblk *pblk)
+ struct pblk *pblk)
{
struct pblk_line_meta *lm = &pblk->lm;
return 1;
}
+static bool pblk_line_is_open(struct pblk *pblk, struct pblk_line *line)
+{
+ struct pblk_line_meta *lm = &pblk->lm;
+ int i;
+
+ for (i = 0; i < lm->blk_per_line; i++)
+ if (line->chks[i].state & NVM_CHK_ST_OPEN)
+ return true;
+
+ return false;
+}
+
struct pblk_line *pblk_recov_l2p(struct pblk *pblk)
{
struct pblk_line_meta *lm = &pblk->lm;
continue;
/* Lines that cannot be read are assumed as not written here */
- if (pblk_line_read_smeta(pblk, line))
+ if (pblk_line_smeta_read(pblk, line))
continue;
crc = pblk_calc_smeta_crc(pblk, smeta_buf);
line->emeta = emeta;
memset(line->emeta->buf, 0, lm->emeta_len[0]);
- if (pblk_line_read_emeta(pblk, line, line->emeta->buf)) {
+ if (pblk_line_is_open(pblk, line)) {
+ pblk_recov_l2p_from_oob(pblk, line);
+ goto next;
+ }
+
+ if (pblk_line_emeta_read(pblk, line, line->emeta->buf)) {
pblk_recov_l2p_from_oob(pblk, line);
goto next;
}
spin_lock(&line->lock);
line->state = PBLK_LINESTATE_CLOSED;
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
move_list = pblk_line_gc_list(pblk, line);
spin_unlock(&line->lock);
list_move_tail(&line->list, move_list);
spin_unlock(&l_mg->gc_lock);
- kfree(line->map_bitmap);
+ mempool_free(line->map_bitmap, l_mg->bitmap_pool);
line->map_bitmap = NULL;
line->smeta = NULL;
line->emeta = NULL;
} else {
- if (open_lines > 1)
- pblk_err(pblk, "failed to recover L2P\n");
+ spin_lock(&line->lock);
+ line->state = PBLK_LINESTATE_OPEN;
+ spin_unlock(&line->lock);
+
+ line->emeta->mem = 0;
+ atomic_set(&line->emeta->sync, 0);
+
+ trace_pblk_line_state(pblk_disk_name(pblk), line->id,
+ line->state);
- open_lines++;
- line->meta_line = meta_line;
data_line = line;
+ line->meta_line = meta_line;
+
+ open_lines++;
}
}
- spin_lock(&l_mg->free_lock);
if (!open_lines) {
+ spin_lock(&l_mg->free_lock);
WARN_ON_ONCE(!test_and_clear_bit(meta_line,
&l_mg->meta_bitmap));
+ spin_unlock(&l_mg->free_lock);
pblk_line_replace_data(pblk);
} else {
+ spin_lock(&l_mg->free_lock);
/* Allocate next line for preparation */
l_mg->data_next = pblk_line_get(pblk);
if (l_mg->data_next) {
l_mg->data_next->type = PBLK_LINETYPE_DATA;
is_next = 1;
}
+ spin_unlock(&l_mg->free_lock);
}
- spin_unlock(&l_mg->free_lock);
if (is_next)
pblk_line_erase(pblk, l_mg->data_next);
left_msecs = line->left_msecs;
spin_unlock(&l_mg->free_lock);
- ret = pblk_recov_pad_oob(pblk, line, left_msecs);
+ ret = pblk_recov_pad_line(pblk, line, left_msecs);
if (ret) {
pblk_err(pblk, "tear down padding failed (%d)\n", ret);
return ret;
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2016 CNEX Labs
* Initial release: Javier Gonzalez <javier@cnexlabs.com>
} else if (free_blocks < rl->high) {
int shift = rl->high_pw - rl->rb_windows_pw;
int user_windows = free_blocks >> shift;
- int user_max = user_windows << PBLK_MAX_REQ_ADDRS_PW;
+ int user_max = user_windows << ilog2(NVM_MAX_VLBA);
rl->rb_user_max = user_max;
rl->rb_gc_max = max - user_max;
rl->rsv_blocks = min_blocks;
/* This will always be a power-of-2 */
- rb_windows = budget / PBLK_MAX_REQ_ADDRS;
+ rb_windows = budget / NVM_MAX_VLBA;
rl->rb_windows_pw = get_count_order(rb_windows);
/* To start with, all buffer is available to user I/O writers */
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2016 CNEX Labs
* Initial release: Javier Gonzalez <javier@cnexlabs.com>
sec_in_line = l_mg->data_line->sec_in_line;
meta_weight = bitmap_weight(&l_mg->meta_bitmap,
PBLK_DATA_LINES);
- map_weight = bitmap_weight(l_mg->data_line->map_bitmap,
+
+ spin_lock(&l_mg->data_line->lock);
+ if (l_mg->data_line->map_bitmap)
+ map_weight = bitmap_weight(l_mg->data_line->map_bitmap,
lm->sec_per_line);
+ else
+ map_weight = 0;
+ spin_unlock(&l_mg->data_line->lock);
}
spin_unlock(&l_mg->free_lock);
{
int sz;
-
sz = snprintf(page, PAGE_SIZE,
"user:%lld gc:%lld pad:%lld WA:",
user, gc, pad);
u32 wa_frac;
wa_int = (user + gc + pad) * 100000;
- wa_int = div_u64(wa_int, user);
+ wa_int = div64_u64(wa_int, user);
wa_int = div_u64_rem(wa_int, 100000, &wa_frac);
sz += snprintf(page + sz, PAGE_SIZE - sz, "%llu.%05u\n",
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM pblk
+
+#if !defined(_TRACE_PBLK_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_PBLK_H
+
+#include <linux/tracepoint.h>
+
+struct ppa_addr;
+
+#define show_chunk_flags(state) __print_flags(state, "", \
+ { NVM_CHK_ST_FREE, "FREE", }, \
+ { NVM_CHK_ST_CLOSED, "CLOSED", }, \
+ { NVM_CHK_ST_OPEN, "OPEN", }, \
+ { NVM_CHK_ST_OFFLINE, "OFFLINE", })
+
+#define show_line_state(state) __print_symbolic(state, \
+ { PBLK_LINESTATE_NEW, "NEW", }, \
+ { PBLK_LINESTATE_FREE, "FREE", }, \
+ { PBLK_LINESTATE_OPEN, "OPEN", }, \
+ { PBLK_LINESTATE_CLOSED, "CLOSED", }, \
+ { PBLK_LINESTATE_GC, "GC", }, \
+ { PBLK_LINESTATE_BAD, "BAD", }, \
+ { PBLK_LINESTATE_CORRUPT, "CORRUPT" })
+
+
+#define show_pblk_state(state) __print_symbolic(state, \
+ { PBLK_STATE_RUNNING, "RUNNING", }, \
+ { PBLK_STATE_STOPPING, "STOPPING", }, \
+ { PBLK_STATE_RECOVERING, "RECOVERING", }, \
+ { PBLK_STATE_STOPPED, "STOPPED" })
+
+#define show_chunk_erase_state(state) __print_symbolic(state, \
+ { PBLK_CHUNK_RESET_START, "START", }, \
+ { PBLK_CHUNK_RESET_DONE, "OK", }, \
+ { PBLK_CHUNK_RESET_FAILED, "FAILED" })
+
+
+TRACE_EVENT(pblk_chunk_reset,
+
+ TP_PROTO(const char *name, struct ppa_addr *ppa, int state),
+
+ TP_ARGS(name, ppa, state),
+
+ TP_STRUCT__entry(
+ __string(name, name)
+ __field(u64, ppa)
+ __field(int, state);
+ ),
+
+ TP_fast_assign(
+ __assign_str(name, name);
+ __entry->ppa = ppa->ppa;
+ __entry->state = state;
+ ),
+
+ TP_printk("dev=%s grp=%llu pu=%llu chk=%llu state=%s", __get_str(name),
+ (u64)(((struct ppa_addr *)(&__entry->ppa))->m.grp),
+ (u64)(((struct ppa_addr *)(&__entry->ppa))->m.pu),
+ (u64)(((struct ppa_addr *)(&__entry->ppa))->m.chk),
+ show_chunk_erase_state((int)__entry->state))
+
+);
+
+TRACE_EVENT(pblk_chunk_state,
+
+ TP_PROTO(const char *name, struct ppa_addr *ppa, int state),
+
+ TP_ARGS(name, ppa, state),
+
+ TP_STRUCT__entry(
+ __string(name, name)
+ __field(u64, ppa)
+ __field(int, state);
+ ),
+
+ TP_fast_assign(
+ __assign_str(name, name);
+ __entry->ppa = ppa->ppa;
+ __entry->state = state;
+ ),
+
+ TP_printk("dev=%s grp=%llu pu=%llu chk=%llu state=%s", __get_str(name),
+ (u64)(((struct ppa_addr *)(&__entry->ppa))->m.grp),
+ (u64)(((struct ppa_addr *)(&__entry->ppa))->m.pu),
+ (u64)(((struct ppa_addr *)(&__entry->ppa))->m.chk),
+ show_chunk_flags((int)__entry->state))
+
+);
+
+TRACE_EVENT(pblk_line_state,
+
+ TP_PROTO(const char *name, int line, int state),
+
+ TP_ARGS(name, line, state),
+
+ TP_STRUCT__entry(
+ __string(name, name)
+ __field(int, line)
+ __field(int, state);
+ ),
+
+ TP_fast_assign(
+ __assign_str(name, name);
+ __entry->line = line;
+ __entry->state = state;
+ ),
+
+ TP_printk("dev=%s line=%d state=%s", __get_str(name),
+ (int)__entry->line,
+ show_line_state((int)__entry->state))
+
+);
+
+TRACE_EVENT(pblk_state,
+
+ TP_PROTO(const char *name, int state),
+
+ TP_ARGS(name, state),
+
+ TP_STRUCT__entry(
+ __string(name, name)
+ __field(int, state);
+ ),
+
+ TP_fast_assign(
+ __assign_str(name, name);
+ __entry->state = state;
+ ),
+
+ TP_printk("dev=%s state=%s", __get_str(name),
+ show_pblk_state((int)__entry->state))
+
+);
+
+#endif /* !defined(_TRACE_PBLK_H) || defined(TRACE_HEADER_MULTI_READ) */
+
+/* This part must be outside protection */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH ../../../drivers/lightnvm
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE pblk-trace
+#include <trace/define_trace.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2016 CNEX Labs
* Initial release: Javier Gonzalez <javier@cnexlabs.com>
*/
#include "pblk.h"
+#include "pblk-trace.h"
static unsigned long pblk_end_w_bio(struct pblk *pblk, struct nvm_rq *rqd,
struct pblk_c_ctx *c_ctx)
#ifdef CONFIG_NVM_PBLK_DEBUG
atomic_long_sub(c_ctx->nr_valid, &pblk->inflight_writes);
#endif
-
- pblk_up_rq(pblk, rqd->ppa_list, rqd->nr_ppas, c_ctx->lun_bitmap);
+ pblk_up_rq(pblk, c_ctx->lun_bitmap);
pos = pblk_rb_sync_init(&pblk->rwb, &flags);
if (pos == c_ctx->sentry) {
/* Map remaining sectors in chunk, starting from ppa */
static void pblk_map_remaining(struct pblk *pblk, struct ppa_addr *ppa)
{
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
struct pblk_line *line;
struct ppa_addr map_ppa = *ppa;
u64 paddr;
int done = 0;
- line = &pblk->lines[pblk_ppa_to_line(*ppa)];
+ line = pblk_ppa_to_line(pblk, *ppa);
spin_lock(&line->lock);
while (!done) {
if (!test_and_set_bit(paddr, line->invalid_bitmap))
le32_add_cpu(line->vsc, -1);
- if (geo->version == NVM_OCSSD_SPEC_12) {
- map_ppa.ppa++;
- if (map_ppa.g.pg == geo->num_pg)
- done = 1;
- } else {
- map_ppa.m.sec++;
- if (map_ppa.m.sec == geo->clba)
- done = 1;
- }
+ done = nvm_next_ppa_in_chk(pblk->dev, &map_ppa);
}
line->w_err_gc->has_write_err = 1;
struct pblk_w_ctx *w_ctx;
struct ppa_addr ppa_l2p;
int flags;
- unsigned int pos, i;
+ unsigned int i;
spin_lock(&pblk->trans_lock);
- pos = sentry;
for (i = 0; i < nr_entries; i++) {
- entry = &rb->entries[pos];
+ entry = &rb->entries[pblk_rb_ptr_wrap(rb, sentry, i)];
w_ctx = &entry->w_ctx;
/* Check if the lba has been overwritten */
/* Release flags on write context. Protect from writes */
smp_store_release(&w_ctx->flags, flags);
- /* Decrese the reference count to the line as we will
+ /* Decrease the reference count to the line as we will
* re-map these entries
*/
- line = &pblk->lines[pblk_ppa_to_line(w_ctx->ppa)];
+ line = pblk_ppa_to_line(pblk, w_ctx->ppa);
kref_put(&line->ref, pblk_line_put);
-
- pos = (pos + 1) & (rb->nr_entries - 1);
}
spin_unlock(&pblk->trans_lock);
}
struct pblk *pblk = recovery->pblk;
struct nvm_rq *rqd = recovery->rqd;
struct pblk_c_ctx *c_ctx = nvm_rq_to_pdu(rqd);
- struct ppa_addr *ppa_list;
+ struct ppa_addr *ppa_list = nvm_rq_to_ppa_list(rqd);
pblk_log_write_err(pblk, rqd);
- if (rqd->nr_ppas == 1)
- ppa_list = &rqd->ppa_addr;
- else
- ppa_list = rqd->ppa_list;
-
pblk_map_remaining(pblk, ppa_list);
pblk_queue_resubmit(pblk, c_ctx);
- pblk_up_rq(pblk, rqd->ppa_list, rqd->nr_ppas, c_ctx->lun_bitmap);
+ pblk_up_rq(pblk, c_ctx->lun_bitmap);
if (c_ctx->nr_padded)
pblk_bio_free_pages(pblk, rqd->bio, c_ctx->nr_valid,
c_ctx->nr_padded);
if (rqd->error) {
pblk_end_w_fail(pblk, rqd);
return;
- }
+ } else {
+ if (trace_pblk_chunk_state_enabled())
+ pblk_check_chunk_state_update(pblk, rqd);
#ifdef CONFIG_NVM_PBLK_DEBUG
- else
WARN_ONCE(rqd->bio->bi_status, "pblk: corrupted write error\n");
#endif
+ }
pblk_complete_write(pblk, rqd, c_ctx);
atomic_dec(&pblk->inflight_io);
struct pblk_g_ctx *m_ctx = nvm_rq_to_pdu(rqd);
struct pblk_line *line = m_ctx->private;
struct pblk_emeta *emeta = line->emeta;
+ struct ppa_addr *ppa_list = nvm_rq_to_ppa_list(rqd);
int sync;
- pblk_up_page(pblk, rqd->ppa_list, rqd->nr_ppas);
+ pblk_up_chunk(pblk, ppa_list[0]);
if (rqd->error) {
pblk_log_write_err(pblk, rqd);
pblk_err(pblk, "metadata I/O failed. Line %d\n", line->id);
line->w_err_gc->has_write_err = 1;
+ } else {
+ if (trace_pblk_chunk_state_enabled())
+ pblk_check_chunk_state_update(pblk, rqd);
}
sync = atomic_add_return(rqd->nr_ppas, &emeta->sync);
}
static int pblk_alloc_w_rq(struct pblk *pblk, struct nvm_rq *rqd,
- unsigned int nr_secs,
- nvm_end_io_fn(*end_io))
+ unsigned int nr_secs, nvm_end_io_fn(*end_io))
{
- struct nvm_tgt_dev *dev = pblk->dev;
-
/* Setup write request */
rqd->opcode = NVM_OP_PWRITE;
rqd->nr_ppas = nr_secs;
- rqd->flags = pblk_set_progr_mode(pblk, PBLK_WRITE);
+ rqd->is_seq = 1;
rqd->private = pblk;
rqd->end_io = end_io;
- rqd->meta_list = nvm_dev_dma_alloc(dev->parent, GFP_KERNEL,
- &rqd->dma_meta_list);
- if (!rqd->meta_list)
- return -ENOMEM;
-
- rqd->ppa_list = rqd->meta_list + pblk_dma_meta_size;
- rqd->dma_ppa_list = rqd->dma_meta_list + pblk_dma_meta_size;
-
- return 0;
+ return pblk_alloc_rqd_meta(pblk, rqd);
}
static int pblk_setup_w_rq(struct pblk *pblk, struct nvm_rq *rqd,
struct pblk_line_mgmt *l_mg = &pblk->l_mg;
struct pblk_line_meta *lm = &pblk->lm;
struct pblk_emeta *emeta = meta_line->emeta;
+ struct ppa_addr *ppa_list;
struct pblk_g_ctx *m_ctx;
struct bio *bio;
struct nvm_rq *rqd;
if (ret)
goto fail_free_bio;
+ ppa_list = nvm_rq_to_ppa_list(rqd);
for (i = 0; i < rqd->nr_ppas; ) {
spin_lock(&meta_line->lock);
paddr = __pblk_alloc_page(pblk, meta_line, rq_ppas);
spin_unlock(&meta_line->lock);
for (j = 0; j < rq_ppas; j++, i++, paddr++)
- rqd->ppa_list[i] = addr_to_gen_ppa(pblk, paddr, id);
+ ppa_list[i] = addr_to_gen_ppa(pblk, paddr, id);
}
+ spin_lock(&l_mg->close_lock);
emeta->mem += rq_len;
- if (emeta->mem >= lm->emeta_len[0]) {
- spin_lock(&l_mg->close_lock);
+ if (emeta->mem >= lm->emeta_len[0])
list_del(&meta_line->list);
- spin_unlock(&l_mg->close_lock);
- }
+ spin_unlock(&l_mg->close_lock);
- pblk_down_page(pblk, rqd->ppa_list, rqd->nr_ppas);
+ pblk_down_chunk(pblk, ppa_list[0]);
ret = pblk_submit_io(pblk, rqd);
if (ret) {
return NVM_IO_OK;
fail_rollback:
- pblk_up_page(pblk, rqd->ppa_list, rqd->nr_ppas);
+ pblk_up_chunk(pblk, ppa_list[0]);
spin_lock(&l_mg->close_lock);
pblk_dealloc_page(pblk, meta_line, rq_ppas);
list_add(&meta_line->list, &meta_line->list);
struct pblk_line *meta_line;
spin_lock(&l_mg->close_lock);
-retry:
if (list_empty(&l_mg->emeta_list)) {
spin_unlock(&l_mg->close_lock);
return NULL;
}
meta_line = list_first_entry(&l_mg->emeta_list, struct pblk_line, list);
- if (meta_line->emeta->mem >= lm->emeta_len[0])
- goto retry;
+ if (meta_line->emeta->mem >= lm->emeta_len[0]) {
+ spin_unlock(&l_mg->close_lock);
+ return NULL;
+ }
spin_unlock(&l_mg->close_lock);
if (!pblk_valid_meta_ppa(pblk, meta_line, data_rqd))
+/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2015 IT University of Copenhagen (rrpc.h)
* Copyright (C) 2016 CNEX Labs
#define PBLK_SECTOR (512)
#define PBLK_EXPOSED_PAGE_SIZE (4096)
-#define PBLK_MAX_REQ_ADDRS (64)
-#define PBLK_MAX_REQ_ADDRS_PW (6)
#define PBLK_NR_CLOSE_JOBS (4)
PBLK_BLK_ST_CLOSED = 0x2,
};
+enum {
+ PBLK_CHUNK_RESET_START,
+ PBLK_CHUNK_RESET_DONE,
+ PBLK_CHUNK_RESET_FAILED,
+};
+
struct pblk_sec_meta {
u64 reserved;
__le64 lba;
PBLK_RL_LOW = 4
};
-#define pblk_dma_meta_size (sizeof(struct pblk_sec_meta) * PBLK_MAX_REQ_ADDRS)
-#define pblk_dma_ppa_size (sizeof(u64) * PBLK_MAX_REQ_ADDRS)
+#define pblk_dma_meta_size (sizeof(struct pblk_sec_meta) * NVM_MAX_VLBA)
+#define pblk_dma_ppa_size (sizeof(u64) * NVM_MAX_VLBA)
/* write buffer completion context */
struct pblk_c_ctx {
* will be 4KB
*/
+ unsigned int back_thres; /* Threshold that shall be maintained by
+ * the backpointer in order to respect
+ * geo->mw_cunits on a per chunk basis
+ */
+
struct list_head pages; /* List of data pages */
spinlock_t w_lock; /* Write lock */
struct pblk_gc_rq {
struct pblk_line *line;
void *data;
- u64 paddr_list[PBLK_MAX_REQ_ADDRS];
- u64 lba_list[PBLK_MAX_REQ_ADDRS];
+ u64 paddr_list[NVM_MAX_VLBA];
+ u64 lba_list[NVM_MAX_VLBA];
int nr_secs;
int secs_to_gc;
struct list_head list;
struct pblk_emeta *eline_meta[PBLK_DATA_LINES];
unsigned long meta_bitmap;
+ /* Cache and mempool for map/invalid bitmaps */
+ struct kmem_cache *bitmap_cache;
+ mempool_t *bitmap_pool;
+
/* Helpers for fast bitmap calculations */
unsigned long *bb_template;
unsigned long *bb_aux;
/*
* pblk ring buffer operations
*/
-int pblk_rb_init(struct pblk_rb *rb, struct pblk_rb_entry *rb_entry_base,
- unsigned int power_size, unsigned int power_seg_sz);
-unsigned int pblk_rb_calculate_size(unsigned int nr_entries);
-void *pblk_rb_entries_ref(struct pblk_rb *rb);
+int pblk_rb_init(struct pblk_rb *rb, unsigned int size, unsigned int threshold,
+ unsigned int seg_sz);
int pblk_rb_may_write_user(struct pblk_rb *rb, struct bio *bio,
unsigned int nr_entries, unsigned int *pos);
int pblk_rb_may_write_gc(struct pblk_rb *rb, unsigned int nr_entries,
unsigned int pblk_rb_sync_init(struct pblk_rb *rb, unsigned long *flags);
unsigned int pblk_rb_sync_advance(struct pblk_rb *rb, unsigned int nr_entries);
-struct pblk_rb_entry *pblk_rb_sync_scan_entry(struct pblk_rb *rb,
- struct ppa_addr *ppa);
+unsigned int pblk_rb_ptr_wrap(struct pblk_rb *rb, unsigned int p,
+ unsigned int nr_entries);
void pblk_rb_sync_end(struct pblk_rb *rb, unsigned long *flags);
unsigned int pblk_rb_flush_point_count(struct pblk_rb *rb);
int pblk_rb_tear_down_check(struct pblk_rb *rb);
int pblk_rb_pos_oob(struct pblk_rb *rb, u64 pos);
-void pblk_rb_data_free(struct pblk_rb *rb);
+void pblk_rb_free(struct pblk_rb *rb);
ssize_t pblk_rb_sysfs(struct pblk_rb *rb, char *buf);
/*
*/
struct nvm_rq *pblk_alloc_rqd(struct pblk *pblk, int type);
void pblk_free_rqd(struct pblk *pblk, struct nvm_rq *rqd, int type);
+int pblk_alloc_rqd_meta(struct pblk *pblk, struct nvm_rq *rqd);
+void pblk_free_rqd_meta(struct pblk *pblk, struct nvm_rq *rqd);
void pblk_set_sec_per_write(struct pblk *pblk, int sec_per_write);
int pblk_setup_w_rec_rq(struct pblk *pblk, struct nvm_rq *rqd,
struct pblk_c_ctx *c_ctx);
void pblk_discard(struct pblk *pblk, struct bio *bio);
-struct nvm_chk_meta *pblk_chunk_get_info(struct pblk *pblk);
+struct nvm_chk_meta *pblk_get_chunk_meta(struct pblk *pblk);
struct nvm_chk_meta *pblk_chunk_get_off(struct pblk *pblk,
struct nvm_chk_meta *lp,
struct ppa_addr ppa);
void pblk_log_read_err(struct pblk *pblk, struct nvm_rq *rqd);
int pblk_submit_io(struct pblk *pblk, struct nvm_rq *rqd);
int pblk_submit_io_sync(struct pblk *pblk, struct nvm_rq *rqd);
+int pblk_submit_io_sync_sem(struct pblk *pblk, struct nvm_rq *rqd);
int pblk_submit_meta_io(struct pblk *pblk, struct pblk_line *meta_line);
+void pblk_check_chunk_state_update(struct pblk *pblk, struct nvm_rq *rqd);
struct bio *pblk_bio_map_addr(struct pblk *pblk, void *data,
unsigned int nr_secs, unsigned int len,
int alloc_type, gfp_t gfp_mask);
struct pblk_line *pblk_line_get(struct pblk *pblk);
struct pblk_line *pblk_line_get_first_data(struct pblk *pblk);
struct pblk_line *pblk_line_replace_data(struct pblk *pblk);
+void pblk_ppa_to_line_put(struct pblk *pblk, struct ppa_addr ppa);
+void pblk_rq_to_line_put(struct pblk *pblk, struct nvm_rq *rqd);
int pblk_line_recov_alloc(struct pblk *pblk, struct pblk_line *line);
void pblk_line_recov_close(struct pblk *pblk, struct pblk_line *line);
struct pblk_line *pblk_line_get_data(struct pblk *pblk);
void (*work)(struct work_struct *), gfp_t gfp_mask,
struct workqueue_struct *wq);
u64 pblk_line_smeta_start(struct pblk *pblk, struct pblk_line *line);
-int pblk_line_read_smeta(struct pblk *pblk, struct pblk_line *line);
-int pblk_line_read_emeta(struct pblk *pblk, struct pblk_line *line,
+int pblk_line_smeta_read(struct pblk *pblk, struct pblk_line *line);
+int pblk_line_emeta_read(struct pblk *pblk, struct pblk_line *line,
void *emeta_buf);
int pblk_blk_erase_async(struct pblk *pblk, struct ppa_addr erase_ppa);
void pblk_line_put(struct kref *ref);
u64 __pblk_alloc_page(struct pblk *pblk, struct pblk_line *line, int nr_secs);
int pblk_calc_secs(struct pblk *pblk, unsigned long secs_avail,
unsigned long secs_to_flush);
-void pblk_up_page(struct pblk *pblk, struct ppa_addr *ppa_list, int nr_ppas);
-void pblk_down_rq(struct pblk *pblk, struct ppa_addr *ppa_list, int nr_ppas,
+void pblk_down_rq(struct pblk *pblk, struct ppa_addr ppa,
unsigned long *lun_bitmap);
-void pblk_down_page(struct pblk *pblk, struct ppa_addr *ppa_list, int nr_ppas);
-void pblk_up_rq(struct pblk *pblk, struct ppa_addr *ppa_list, int nr_ppas,
- unsigned long *lun_bitmap);
+void pblk_down_chunk(struct pblk *pblk, struct ppa_addr ppa);
+void pblk_up_chunk(struct pblk *pblk, struct ppa_addr ppa);
+void pblk_up_rq(struct pblk *pblk, unsigned long *lun_bitmap);
int pblk_bio_add_pages(struct pblk *pblk, struct bio *bio, gfp_t flags,
int nr_pages);
void pblk_bio_free_pages(struct pblk *pblk, struct bio *bio, int off,
return le32_to_cpu(*line->vsc);
}
-static inline int pblk_pad_distance(struct pblk *pblk)
+static inline int pblk_ppa_to_line_id(struct ppa_addr p)
{
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
-
- return geo->mw_cunits * geo->all_luns * geo->ws_opt;
+ return p.a.blk;
}
-static inline int pblk_ppa_to_line(struct ppa_addr p)
+static inline struct pblk_line *pblk_ppa_to_line(struct pblk *pblk,
+ struct ppa_addr p)
{
- return p.a.blk;
+ return &pblk->lines[pblk_ppa_to_line_id(p)];
}
static inline int pblk_ppa_to_pos(struct nvm_geo *geo, struct ppa_addr p)
return ppa;
}
+static inline struct nvm_chk_meta *pblk_dev_ppa_to_chunk(struct pblk *pblk,
+ struct ppa_addr p)
+{
+ struct nvm_tgt_dev *dev = pblk->dev;
+ struct nvm_geo *geo = &dev->geo;
+ struct pblk_line *line = pblk_ppa_to_line(pblk, p);
+ int pos = pblk_ppa_to_pos(geo, p);
+
+ return &line->chks[pos];
+}
+
+static inline u64 pblk_dev_ppa_to_chunk_addr(struct pblk *pblk,
+ struct ppa_addr p)
+{
+ struct nvm_tgt_dev *dev = pblk->dev;
+
+ return dev_to_chunk_addr(dev->parent, &pblk->addrf, p);
+}
+
static inline u64 pblk_dev_ppa_to_line_addr(struct pblk *pblk,
struct ppa_addr p)
{
static inline struct ppa_addr pblk_ppa32_to_ppa64(struct pblk *pblk, u32 ppa32)
{
- struct ppa_addr ppa64;
-
- ppa64.ppa = 0;
-
- if (ppa32 == -1) {
- ppa64.ppa = ADDR_EMPTY;
- } else if (ppa32 & (1U << 31)) {
- ppa64.c.line = ppa32 & ((~0U) >> 1);
- ppa64.c.is_cached = 1;
- } else {
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
-
- if (geo->version == NVM_OCSSD_SPEC_12) {
- struct nvm_addrf_12 *ppaf =
- (struct nvm_addrf_12 *)&pblk->addrf;
-
- ppa64.g.ch = (ppa32 & ppaf->ch_mask) >>
- ppaf->ch_offset;
- ppa64.g.lun = (ppa32 & ppaf->lun_mask) >>
- ppaf->lun_offset;
- ppa64.g.blk = (ppa32 & ppaf->blk_mask) >>
- ppaf->blk_offset;
- ppa64.g.pg = (ppa32 & ppaf->pg_mask) >>
- ppaf->pg_offset;
- ppa64.g.pl = (ppa32 & ppaf->pln_mask) >>
- ppaf->pln_offset;
- ppa64.g.sec = (ppa32 & ppaf->sec_mask) >>
- ppaf->sec_offset;
- } else {
- struct nvm_addrf *lbaf = &pblk->addrf;
-
- ppa64.m.grp = (ppa32 & lbaf->ch_mask) >>
- lbaf->ch_offset;
- ppa64.m.pu = (ppa32 & lbaf->lun_mask) >>
- lbaf->lun_offset;
- ppa64.m.chk = (ppa32 & lbaf->chk_mask) >>
- lbaf->chk_offset;
- ppa64.m.sec = (ppa32 & lbaf->sec_mask) >>
- lbaf->sec_offset;
- }
- }
+ struct nvm_tgt_dev *dev = pblk->dev;
- return ppa64;
+ return nvm_ppa32_to_ppa64(dev->parent, &pblk->addrf, ppa32);
}
static inline u32 pblk_ppa64_to_ppa32(struct pblk *pblk, struct ppa_addr ppa64)
{
- u32 ppa32 = 0;
-
- if (ppa64.ppa == ADDR_EMPTY) {
- ppa32 = ~0U;
- } else if (ppa64.c.is_cached) {
- ppa32 |= ppa64.c.line;
- ppa32 |= 1U << 31;
- } else {
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
-
- if (geo->version == NVM_OCSSD_SPEC_12) {
- struct nvm_addrf_12 *ppaf =
- (struct nvm_addrf_12 *)&pblk->addrf;
-
- ppa32 |= ppa64.g.ch << ppaf->ch_offset;
- ppa32 |= ppa64.g.lun << ppaf->lun_offset;
- ppa32 |= ppa64.g.blk << ppaf->blk_offset;
- ppa32 |= ppa64.g.pg << ppaf->pg_offset;
- ppa32 |= ppa64.g.pl << ppaf->pln_offset;
- ppa32 |= ppa64.g.sec << ppaf->sec_offset;
- } else {
- struct nvm_addrf *lbaf = &pblk->addrf;
-
- ppa32 |= ppa64.m.grp << lbaf->ch_offset;
- ppa32 |= ppa64.m.pu << lbaf->lun_offset;
- ppa32 |= ppa64.m.chk << lbaf->chk_offset;
- ppa32 |= ppa64.m.sec << lbaf->sec_offset;
- }
- }
+ struct nvm_tgt_dev *dev = pblk->dev;
- return ppa32;
+ return nvm_ppa64_to_ppa32(dev->parent, &pblk->addrf, ppa64);
}
static inline struct ppa_addr pblk_trans_map_get(struct pblk *pblk,
return crc;
}
-static inline int pblk_set_progr_mode(struct pblk *pblk, int type)
-{
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
- int flags;
-
- if (geo->version == NVM_OCSSD_SPEC_20)
- return 0;
-
- flags = geo->pln_mode >> 1;
-
- if (type == PBLK_WRITE)
- flags |= NVM_IO_SCRAMBLE_ENABLE;
-
- return flags;
-}
-
-enum {
- PBLK_READ_RANDOM = 0,
- PBLK_READ_SEQUENTIAL = 1,
-};
-
-static inline int pblk_set_read_mode(struct pblk *pblk, int type)
-{
- struct nvm_tgt_dev *dev = pblk->dev;
- struct nvm_geo *geo = &dev->geo;
- int flags;
-
- if (geo->version == NVM_OCSSD_SPEC_20)
- return 0;
-
- flags = NVM_IO_SUSPEND | NVM_IO_SCRAMBLE_ENABLE;
- if (type == PBLK_READ_SEQUENTIAL)
- flags |= geo->pln_mode >> 1;
-
- return flags;
-}
-
static inline int pblk_io_aligned(struct pblk *pblk, int nr_secs)
{
return !(nr_secs % pblk->min_write_pgs);
static inline int pblk_check_io(struct pblk *pblk, struct nvm_rq *rqd)
{
struct nvm_tgt_dev *dev = pblk->dev;
- struct ppa_addr *ppa_list;
-
- ppa_list = (rqd->nr_ppas > 1) ? rqd->ppa_list : &rqd->ppa_addr;
+ struct ppa_addr *ppa_list = nvm_rq_to_ppa_list(rqd);
if (pblk_boundary_ppa_checks(dev, ppa_list, rqd->nr_ppas)) {
WARN_ON(1);
if (rqd->opcode == NVM_OP_PWRITE) {
struct pblk_line *line;
- struct ppa_addr ppa;
int i;
for (i = 0; i < rqd->nr_ppas; i++) {
- ppa = ppa_list[i];
- line = &pblk->lines[pblk_ppa_to_line(ppa)];
+ line = pblk_ppa_to_line(pblk, ppa_list[i]);
spin_lock(&line->lock);
if (line->state != PBLK_LINESTATE_OPEN) {
uuid_le_gen(&uuid);
memcpy(pblk->instance_uuid, uuid.b, 16);
}
+
+static inline char *pblk_disk_name(struct pblk *pblk)
+{
+ struct gendisk *disk = pblk->disk;
+
+ return disk->disk_name;
+}
#endif /* PBLK_H_ */
count = acpi_table_parse_entries_array(ACPI_SIG_PCCT,
sizeof(struct acpi_table_pcct), proc,
ACPI_PCCT_TYPE_RESERVED, MAX_PCC_SUBSPACES);
- if (count == 0 || count > MAX_PCC_SUBSPACES) {
- pr_warn("Invalid PCCT: %d PCC subspaces\n", count);
+ if (count <= 0 || count > MAX_PCC_SUBSPACES) {
+ if (count < 0)
+ pr_warn("Error parsing PCC subspaces from PCCT\n");
+ else
+ pr_warn("Invalid PCCT: %d PCC subspaces\n", count);
return -EINVAL;
}
int i;
lockdep_assert_held(&c->bucket_lock);
- BUG_ON(!n || n > c->caches_loaded || n > 8);
+ BUG_ON(!n || n > c->caches_loaded || n > MAX_CACHES_PER_SET);
bkey_init(k);
int bch_cache_allocator_start(struct cache *ca);
void bch_debug_exit(void);
-void bch_debug_init(struct kobject *kobj);
+void bch_debug_init(void);
void bch_request_exit(void);
int bch_request_init(void);
struct keybuf *buf = refill->buf;
int ret = MAP_CONTINUE;
- if (bkey_cmp(k, refill->end) >= 0) {
+ if (bkey_cmp(k, refill->end) > 0) {
ret = MAP_DONE;
goto out;
}
} while (0)
/**
- * closure_return - finish execution of a closure, with destructor
+ * closure_return_with_destructor - finish execution of a closure,
+ * with destructor
*
* Works like closure_return(), except @destructor will be called when all
* outstanding refs on @cl have been dropped; @destructor may be used to safely
debugfs_remove_recursive(bcache_debug);
}
-void __init bch_debug_init(struct kobject *kobj)
+void __init bch_debug_init(void)
{
/*
* it is unnecessary to check return value of
for (i = 0; i < KEY_PTRS(k); i++) {
stale = ptr_stale(b->c, k, i);
- btree_bug_on(stale > 96, b,
+ btree_bug_on(stale > BUCKET_GC_GEN_MAX, b,
"key too stale: %i, need_gc %u",
stale, b->c->need_gc);
* unless the read-ahead request is for metadata (eg, for gfs2).
*/
if (bio->bi_opf & (REQ_RAHEAD|REQ_BACKGROUND) &&
- !(bio->bi_opf & REQ_META))
+ !(bio->bi_opf & REQ_PRIO))
goto skip;
if (bio->bi_iter.bi_sector & (c->sb.block_size - 1) ||
bch_mark_cache_accounting(s->iop.c, s->d,
!s->cache_missed, s->iop.bypass);
- trace_bcache_read(s->orig_bio, !s->cache_miss, s->iop.bypass);
+ trace_bcache_read(s->orig_bio, !s->cache_missed, s->iop.bypass);
if (s->iop.status)
continue_at_nobarrier(cl, cached_dev_read_error, bcache_wq);
}
if (!(bio->bi_opf & REQ_RAHEAD) &&
- !(bio->bi_opf & REQ_META) &&
+ !(bio->bi_opf & REQ_PRIO) &&
s->iop.c->gc_stats.in_use < CUTOFF_CACHE_READA)
reada = min_t(sector_t, dc->readahead >> 9,
get_capacity(bio->bi_disk) - bio_end_sector(bio));
{
struct cached_dev *dc = container_of(d, struct cached_dev, disk);
+ if (dc->io_disable)
+ return -EIO;
+
return __blkdev_driver_ioctl(dc->bdev, mode, cmd, arg);
}
void bch_cached_dev_request_init(struct cached_dev *dc);
void bch_flash_dev_request_init(struct bcache_device *d);
-extern struct kmem_cache *bch_search_cache, *bch_passthrough_cache;
+extern struct kmem_cache *bch_search_cache;
#endif /* _BCACHE_REQUEST_H_ */
{
BKEY_PADDED(key) k;
struct closure cl;
+ struct cache *ca;
closure_init_stack(&cl);
lockdep_assert_held(&bch_register_lock);
uuid_io(c, REQ_OP_WRITE, 0, &k.key, &cl);
closure_sync(&cl);
+ /* Only one bucket used for uuid write */
+ ca = PTR_CACHE(c, &k.key, 0);
+ atomic_long_add(ca->sb.bucket_size, &ca->meta_sectors_written);
+
bkey_copy(&c->uuid_bucket, &k.key);
bkey_put(c, &k.key);
return 0;
unsigned int cmd, unsigned long arg)
{
struct bcache_device *d = b->bd_disk->private_data;
- struct cached_dev *dc = container_of(d, struct cached_dev, disk);
-
- if (dc->io_disable)
- return -EIO;
return d->ioctl(d, mode, cmd, arg);
}
bch_write_bdev_super(dc, &cl);
closure_sync(&cl);
+ calc_cached_dev_sectors(dc->disk.c);
bcache_device_detach(&dc->disk);
list_move(&dc->list, &uncached_devices);
}
if (BDEV_STATE(&dc->sb) == BDEV_STATE_DIRTY) {
- bch_sectors_dirty_init(&dc->disk);
atomic_set(&dc->has_dirty, 1);
bch_writeback_queue(dc);
}
+ bch_sectors_dirty_init(&dc->disk);
+
bch_cached_dev_run(dc);
bcache_device_link(&dc->disk, c, "bdev");
atomic_inc(&c->attached_dev_nr);
size_t free;
size_t btree_buckets;
struct bucket *b;
+ int ret = -ENOMEM;
+ const char *err = NULL;
__module_get(THIS_MODULE);
kobject_init(&ca->kobj, &bch_cache_ktype);
*/
btree_buckets = ca->sb.njournal_buckets ?: 8;
free = roundup_pow_of_two(ca->sb.nbuckets) >> 10;
+ if (!free) {
+ ret = -EPERM;
+ err = "ca->sb.nbuckets is too small";
+ goto err_free;
+ }
- if (!init_fifo(&ca->free[RESERVE_BTREE], btree_buckets, GFP_KERNEL) ||
- !init_fifo_exact(&ca->free[RESERVE_PRIO], prio_buckets(ca), GFP_KERNEL) ||
- !init_fifo(&ca->free[RESERVE_MOVINGGC], free, GFP_KERNEL) ||
- !init_fifo(&ca->free[RESERVE_NONE], free, GFP_KERNEL) ||
- !init_fifo(&ca->free_inc, free << 2, GFP_KERNEL) ||
- !init_heap(&ca->heap, free << 3, GFP_KERNEL) ||
- !(ca->buckets = vzalloc(array_size(sizeof(struct bucket),
- ca->sb.nbuckets))) ||
- !(ca->prio_buckets = kzalloc(array3_size(sizeof(uint64_t),
- prio_buckets(ca), 2),
- GFP_KERNEL)) ||
- !(ca->disk_buckets = alloc_bucket_pages(GFP_KERNEL, ca)))
- return -ENOMEM;
+ if (!init_fifo(&ca->free[RESERVE_BTREE], btree_buckets,
+ GFP_KERNEL)) {
+ err = "ca->free[RESERVE_BTREE] alloc failed";
+ goto err_btree_alloc;
+ }
+
+ if (!init_fifo_exact(&ca->free[RESERVE_PRIO], prio_buckets(ca),
+ GFP_KERNEL)) {
+ err = "ca->free[RESERVE_PRIO] alloc failed";
+ goto err_prio_alloc;
+ }
+
+ if (!init_fifo(&ca->free[RESERVE_MOVINGGC], free, GFP_KERNEL)) {
+ err = "ca->free[RESERVE_MOVINGGC] alloc failed";
+ goto err_movinggc_alloc;
+ }
+
+ if (!init_fifo(&ca->free[RESERVE_NONE], free, GFP_KERNEL)) {
+ err = "ca->free[RESERVE_NONE] alloc failed";
+ goto err_none_alloc;
+ }
+
+ if (!init_fifo(&ca->free_inc, free << 2, GFP_KERNEL)) {
+ err = "ca->free_inc alloc failed";
+ goto err_free_inc_alloc;
+ }
+
+ if (!init_heap(&ca->heap, free << 3, GFP_KERNEL)) {
+ err = "ca->heap alloc failed";
+ goto err_heap_alloc;
+ }
+
+ ca->buckets = vzalloc(array_size(sizeof(struct bucket),
+ ca->sb.nbuckets));
+ if (!ca->buckets) {
+ err = "ca->buckets alloc failed";
+ goto err_buckets_alloc;
+ }
+
+ ca->prio_buckets = kzalloc(array3_size(sizeof(uint64_t),
+ prio_buckets(ca), 2),
+ GFP_KERNEL);
+ if (!ca->prio_buckets) {
+ err = "ca->prio_buckets alloc failed";
+ goto err_prio_buckets_alloc;
+ }
+
+ ca->disk_buckets = alloc_bucket_pages(GFP_KERNEL, ca);
+ if (!ca->disk_buckets) {
+ err = "ca->disk_buckets alloc failed";
+ goto err_disk_buckets_alloc;
+ }
ca->prio_last_buckets = ca->prio_buckets + prio_buckets(ca);
for_each_bucket(b, ca)
atomic_set(&b->pin, 0);
-
return 0;
+
+err_disk_buckets_alloc:
+ kfree(ca->prio_buckets);
+err_prio_buckets_alloc:
+ vfree(ca->buckets);
+err_buckets_alloc:
+ free_heap(&ca->heap);
+err_heap_alloc:
+ free_fifo(&ca->free_inc);
+err_free_inc_alloc:
+ free_fifo(&ca->free[RESERVE_NONE]);
+err_none_alloc:
+ free_fifo(&ca->free[RESERVE_MOVINGGC]);
+err_movinggc_alloc:
+ free_fifo(&ca->free[RESERVE_PRIO]);
+err_prio_alloc:
+ free_fifo(&ca->free[RESERVE_BTREE]);
+err_btree_alloc:
+err_free:
+ module_put(THIS_MODULE);
+ if (err)
+ pr_notice("error %s: %s", ca->cache_dev_name, err);
+ return ret;
}
static int register_cache(struct cache_sb *sb, struct page *sb_page,
blkdev_put(bdev, FMODE_READ|FMODE_WRITE|FMODE_EXCL);
if (ret == -ENOMEM)
err = "cache_alloc(): -ENOMEM";
+ else if (ret == -EPERM)
+ err = "cache_alloc(): cache device is too small";
else
err = "cache_alloc(): unknown error";
goto err;
sysfs_create_files(bcache_kobj, files))
goto err;
- bch_debug_init(bcache_kobj);
+ bch_debug_init();
closure_debug_init();
return 0;
1, WRITEBACK_RATE_UPDATE_SECS_MAX);
d_strtoul(writeback_rate_i_term_inverse);
d_strtoul_nonzero(writeback_rate_p_term_inverse);
+ d_strtoul_nonzero(writeback_rate_minimum);
sysfs_strtoul_clamp(io_error_limit, dc->error_limit, 0, INT_MAX);
&sysfs_writeback_rate_update_seconds,
&sysfs_writeback_rate_i_term_inverse,
&sysfs_writeback_rate_p_term_inverse,
+ &sysfs_writeback_rate_minimum,
&sysfs_writeback_rate_debug,
&sysfs_errors,
&sysfs_io_error_limit,
int r;
migration_cache = KMEM_CACHE(dm_cache_migration, 0);
- if (!migration_cache) {
- dm_unregister_target(&cache_target);
+ if (!migration_cache)
return -ENOMEM;
- }
r = dm_register_target(&cache_target);
if (r) {
DMERR("cache target registration failed: %d", r);
+ kmem_cache_destroy(migration_cache);
return r;
}
static struct target_type flakey_target = {
.name = "flakey",
.version = {1, 5, 0},
+#ifdef CONFIG_BLK_DEV_ZONED
.features = DM_TARGET_ZONED_HM,
+#endif
.module = THIS_MODULE,
.ctr = flakey_ctr,
.dtr = flakey_dtr,
r = -ENOMEM;
goto bad;
}
- ic->recalc_tags = kvmalloc((RECALC_SECTORS >> ic->sb->log2_sectors_per_block) * ic->tag_size, GFP_KERNEL);
+ ic->recalc_tags = kvmalloc_array(RECALC_SECTORS >> ic->sb->log2_sectors_per_block,
+ ic->tag_size, GFP_KERNEL);
if (!ic->recalc_tags) {
ti->error = "Cannot allocate tags for recalculating";
r = -ENOMEM;
return DM_MAPIO_REMAPPED;
}
+#ifdef CONFIG_BLK_DEV_ZONED
static int linear_end_io(struct dm_target *ti, struct bio *bio,
blk_status_t *error)
{
return DM_ENDIO_DONE;
}
+#endif
static void linear_status(struct dm_target *ti, status_type_t type,
unsigned status_flags, char *result, unsigned maxlen)
static struct target_type linear_target = {
.name = "linear",
.version = {1, 4, 0},
+#ifdef CONFIG_BLK_DEV_ZONED
+ .end_io = linear_end_io,
.features = DM_TARGET_PASSES_INTEGRITY | DM_TARGET_ZONED_HM,
+#else
+ .features = DM_TARGET_PASSES_INTEGRITY,
+#endif
.module = THIS_MODULE,
.ctr = linear_ctr,
.dtr = linear_dtr,
.map = linear_map,
- .end_io = linear_end_io,
.status = linear_status,
.prepare_ioctl = linear_prepare_ioctl,
.iterate_devices = linear_iterate_devices,
EXPORT_SYMBOL_GPL(dm_accept_partial_bio);
/*
- * The zone descriptors obtained with a zone report indicate
- * zone positions within the target device. The zone descriptors
- * must be remapped to match their position within the dm device.
- * A target may call dm_remap_zone_report after completion of a
- * REQ_OP_ZONE_REPORT bio to remap the zone descriptors obtained
- * from the target device mapping to the dm device.
+ * The zone descriptors obtained with a zone report indicate zone positions
+ * within the target backing device, regardless of that device is a partition
+ * and regardless of the target mapping start sector on the device or partition.
+ * The zone descriptors start sector and write pointer position must be adjusted
+ * to match their relative position within the dm device.
+ * A target may call dm_remap_zone_report() after completion of a
+ * REQ_OP_ZONE_REPORT bio to remap the zone descriptors obtained from the
+ * backing device.
*/
void dm_remap_zone_report(struct dm_target *ti, struct bio *bio, sector_t start)
{
struct blk_zone *zone;
unsigned int nr_rep = 0;
unsigned int ofst;
+ sector_t part_offset;
struct bio_vec bvec;
struct bvec_iter iter;
void *addr;
if (bio->bi_status)
return;
+ /*
+ * bio sector was incremented by the request size on completion. Taking
+ * into account the original request sector, the target start offset on
+ * the backing device and the target mapping offset (ti->begin), the
+ * start sector of the backing device. The partition offset is always 0
+ * if the target uses a whole device.
+ */
+ part_offset = bio->bi_iter.bi_sector + ti->begin - (start + bio_end_sector(report_bio));
+
/*
* Remap the start sector of the reported zones. For sequential zones,
* also remap the write pointer position.
/* Set zones start sector */
while (hdr->nr_zones && ofst < bvec.bv_len) {
zone = addr + ofst;
+ zone->start -= part_offset;
if (zone->start >= start + ti->len) {
hdr->nr_zones = 0;
break;
else if (zone->cond == BLK_ZONE_COND_EMPTY)
zone->wp = zone->start;
else
- zone->wp = zone->wp + ti->begin - start;
+ zone->wp = zone->wp + ti->begin - start - part_offset;
}
ofst += sizeof(struct blk_zone);
hdr->nr_zones--;
!discard_bio)
continue;
bio_chain(discard_bio, bio);
- bio_clone_blkcg_association(discard_bio, bio);
+ bio_clone_blkg_association(discard_bio, bio);
if (mddev->gendisk)
trace_block_bio_remap(bdev_get_queue(rdev->bdev),
discard_bio, disk_devt(mddev->gendisk),
set_disk_ro(msb->disk, 1);
msb_start(card);
- device_add_disk(&card->dev, msb->disk);
+ device_add_disk(&card->dev, msb->disk, NULL);
dbg("Disk added");
return 0;
set_capacity(msb->disk, capacity);
dev_dbg(&card->dev, "capacity set %ld\n", capacity);
- device_add_disk(&card->dev, msb->disk);
+ device_add_disk(&card->dev, msb->disk, NULL);
msb->active = 1;
return 0;
.cache_type = REGCACHE_NONE,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.read_flag_mask = 1,
.write_flag_mask = 0,
config.reg_bits = 7;
config.pad_bits = 1;
config.val_bits = 8;
- config.use_single_rw = 1;
+ config.use_single_read = true;
+ config.use_single_write = true;
da9052->regmap = devm_regmap_init_spi(spi, &config);
if (IS_ERR(da9052->regmap)) {
.max_register = MC13XXX_NUMREGS,
.cache_type = REGCACHE_NONE,
- .use_single_rw = 1,
+ .use_single_read = true,
+ .use_single_write = true,
};
static int mc13xxx_spi_read(void *context, const void *reg, size_t reg_size,
//
// Copyright (C) 2018 ROHM Semiconductors
//
-// ROHM BD71837MWV PMIC driver
+// ROHM BD71837MWV and BD71847MWV PMIC driver
//
-// Datasheet available from
+// Datasheet for BD71837MWV available from
// https://www.rohm.com/datasheet/BD71837MWV/bd71837mwv-e
+#include <linux/gpio_keys.h>
#include <linux/i2c.h>
#include <linux/input.h>
#include <linux/interrupt.h>
#include <linux/mfd/rohm-bd718x7.h>
#include <linux/mfd/core.h>
#include <linux/module.h>
+#include <linux/of_device.h>
#include <linux/regmap.h>
-
-/*
- * gpio_keys.h requires definiton of bool. It is brought in
- * by above includes. Keep this as last until gpio_keys.h gets fixed.
- */
-#include <linux/gpio_keys.h>
-
-static const u8 supported_revisions[] = { 0xA2 /* BD71837 */ };
+#include <linux/types.h>
static struct gpio_keys_button button = {
.code = KEY_POWER,
.name = "bd718xx-pwrkey",
};
-static struct mfd_cell bd71837_mfd_cells[] = {
+static struct mfd_cell bd718xx_mfd_cells[] = {
{
.name = "gpio-keys",
.platform_data = &bd718xx_powerkey_data,
.pdata_size = sizeof(bd718xx_powerkey_data),
},
- { .name = "bd71837-clk", },
- { .name = "bd71837-pmic", },
+ { .name = "bd718xx-clk", },
+ { .name = "bd718xx-pmic", },
};
-static const struct regmap_irq bd71837_irqs[] = {
- REGMAP_IRQ_REG(BD71837_INT_SWRST, 0, BD71837_INT_SWRST_MASK),
- REGMAP_IRQ_REG(BD71837_INT_PWRBTN_S, 0, BD71837_INT_PWRBTN_S_MASK),
- REGMAP_IRQ_REG(BD71837_INT_PWRBTN_L, 0, BD71837_INT_PWRBTN_L_MASK),
- REGMAP_IRQ_REG(BD71837_INT_PWRBTN, 0, BD71837_INT_PWRBTN_MASK),
- REGMAP_IRQ_REG(BD71837_INT_WDOG, 0, BD71837_INT_WDOG_MASK),
- REGMAP_IRQ_REG(BD71837_INT_ON_REQ, 0, BD71837_INT_ON_REQ_MASK),
- REGMAP_IRQ_REG(BD71837_INT_STBY_REQ, 0, BD71837_INT_STBY_REQ_MASK),
+static const struct regmap_irq bd718xx_irqs[] = {
+ REGMAP_IRQ_REG(BD718XX_INT_SWRST, 0, BD718XX_INT_SWRST_MASK),
+ REGMAP_IRQ_REG(BD718XX_INT_PWRBTN_S, 0, BD718XX_INT_PWRBTN_S_MASK),
+ REGMAP_IRQ_REG(BD718XX_INT_PWRBTN_L, 0, BD718XX_INT_PWRBTN_L_MASK),
+ REGMAP_IRQ_REG(BD718XX_INT_PWRBTN, 0, BD718XX_INT_PWRBTN_MASK),
+ REGMAP_IRQ_REG(BD718XX_INT_WDOG, 0, BD718XX_INT_WDOG_MASK),
+ REGMAP_IRQ_REG(BD718XX_INT_ON_REQ, 0, BD718XX_INT_ON_REQ_MASK),
+ REGMAP_IRQ_REG(BD718XX_INT_STBY_REQ, 0, BD718XX_INT_STBY_REQ_MASK),
};
-static struct regmap_irq_chip bd71837_irq_chip = {
- .name = "bd71837-irq",
- .irqs = bd71837_irqs,
- .num_irqs = ARRAY_SIZE(bd71837_irqs),
+static struct regmap_irq_chip bd718xx_irq_chip = {
+ .name = "bd718xx-irq",
+ .irqs = bd718xx_irqs,
+ .num_irqs = ARRAY_SIZE(bd718xx_irqs),
.num_regs = 1,
.irq_reg_stride = 1,
- .status_base = BD71837_REG_IRQ,
- .mask_base = BD71837_REG_MIRQ,
- .ack_base = BD71837_REG_IRQ,
+ .status_base = BD718XX_REG_IRQ,
+ .mask_base = BD718XX_REG_MIRQ,
+ .ack_base = BD718XX_REG_IRQ,
.init_ack_masked = true,
.mask_invert = false,
};
static const struct regmap_range pmic_status_range = {
- .range_min = BD71837_REG_IRQ,
- .range_max = BD71837_REG_POW_STATE,
+ .range_min = BD718XX_REG_IRQ,
+ .range_max = BD718XX_REG_POW_STATE,
};
static const struct regmap_access_table volatile_regs = {
.n_yes_ranges = 1,
};
-static const struct regmap_config bd71837_regmap_config = {
+static const struct regmap_config bd718xx_regmap_config = {
.reg_bits = 8,
.val_bits = 8,
.volatile_table = &volatile_regs,
- .max_register = BD71837_MAX_REGISTER - 1,
+ .max_register = BD718XX_MAX_REGISTER - 1,
.cache_type = REGCACHE_RBTREE,
};
-static int bd71837_i2c_probe(struct i2c_client *i2c,
+static int bd718xx_i2c_probe(struct i2c_client *i2c,
const struct i2c_device_id *id)
{
- struct bd71837 *bd71837;
- int ret, i;
- unsigned int val;
-
- bd71837 = devm_kzalloc(&i2c->dev, sizeof(struct bd71837), GFP_KERNEL);
+ struct bd718xx *bd718xx;
+ int ret;
- if (!bd71837)
- return -ENOMEM;
-
- bd71837->chip_irq = i2c->irq;
-
- if (!bd71837->chip_irq) {
+ if (!i2c->irq) {
dev_err(&i2c->dev, "No IRQ configured\n");
return -EINVAL;
}
- bd71837->dev = &i2c->dev;
- dev_set_drvdata(&i2c->dev, bd71837);
+ bd718xx = devm_kzalloc(&i2c->dev, sizeof(struct bd718xx), GFP_KERNEL);
- bd71837->regmap = devm_regmap_init_i2c(i2c, &bd71837_regmap_config);
- if (IS_ERR(bd71837->regmap)) {
- dev_err(&i2c->dev, "regmap initialization failed\n");
- return PTR_ERR(bd71837->regmap);
- }
+ if (!bd718xx)
+ return -ENOMEM;
- ret = regmap_read(bd71837->regmap, BD71837_REG_REV, &val);
- if (ret) {
- dev_err(&i2c->dev, "Read BD71837_REG_DEVICE failed\n");
- return ret;
- }
- for (i = 0; i < ARRAY_SIZE(supported_revisions); i++)
- if (supported_revisions[i] == val)
- break;
+ bd718xx->chip_irq = i2c->irq;
+ bd718xx->chip_type = (unsigned int)(uintptr_t)
+ of_device_get_match_data(&i2c->dev);
+ bd718xx->dev = &i2c->dev;
+ dev_set_drvdata(&i2c->dev, bd718xx);
- if (i == ARRAY_SIZE(supported_revisions)) {
- dev_err(&i2c->dev, "Unsupported chip revision\n");
- return -ENODEV;
+ bd718xx->regmap = devm_regmap_init_i2c(i2c, &bd718xx_regmap_config);
+ if (IS_ERR(bd718xx->regmap)) {
+ dev_err(&i2c->dev, "regmap initialization failed\n");
+ return PTR_ERR(bd718xx->regmap);
}
- ret = devm_regmap_add_irq_chip(&i2c->dev, bd71837->regmap,
- bd71837->chip_irq, IRQF_ONESHOT, 0,
- &bd71837_irq_chip, &bd71837->irq_data);
+ ret = devm_regmap_add_irq_chip(&i2c->dev, bd718xx->regmap,
+ bd718xx->chip_irq, IRQF_ONESHOT, 0,
+ &bd718xx_irq_chip, &bd718xx->irq_data);
if (ret) {
dev_err(&i2c->dev, "Failed to add irq_chip\n");
return ret;
}
/* Configure short press to 10 milliseconds */
- ret = regmap_update_bits(bd71837->regmap,
- BD71837_REG_PWRONCONFIG0,
+ ret = regmap_update_bits(bd718xx->regmap,
+ BD718XX_REG_PWRONCONFIG0,
BD718XX_PWRBTN_PRESS_DURATION_MASK,
BD718XX_PWRBTN_SHORT_PRESS_10MS);
if (ret) {
}
/* Configure long press to 10 seconds */
- ret = regmap_update_bits(bd71837->regmap,
- BD71837_REG_PWRONCONFIG1,
+ ret = regmap_update_bits(bd718xx->regmap,
+ BD718XX_REG_PWRONCONFIG1,
BD718XX_PWRBTN_PRESS_DURATION_MASK,
BD718XX_PWRBTN_LONG_PRESS_10S);
return ret;
}
- ret = regmap_irq_get_virq(bd71837->irq_data, BD71837_INT_PWRBTN_S);
+ ret = regmap_irq_get_virq(bd718xx->irq_data, BD718XX_INT_PWRBTN_S);
if (ret < 0) {
dev_err(&i2c->dev, "Failed to get the IRQ\n");
button.irq = ret;
- ret = devm_mfd_add_devices(bd71837->dev, PLATFORM_DEVID_AUTO,
- bd71837_mfd_cells,
- ARRAY_SIZE(bd71837_mfd_cells), NULL, 0,
- regmap_irq_get_domain(bd71837->irq_data));
+ ret = devm_mfd_add_devices(bd718xx->dev, PLATFORM_DEVID_AUTO,
+ bd718xx_mfd_cells,
+ ARRAY_SIZE(bd718xx_mfd_cells), NULL, 0,
+ regmap_irq_get_domain(bd718xx->irq_data));
if (ret)
dev_err(&i2c->dev, "Failed to create subdevices\n");
return ret;
}
-static const struct of_device_id bd71837_of_match[] = {
- { .compatible = "rohm,bd71837", },
+static const struct of_device_id bd718xx_of_match[] = {
+ {
+ .compatible = "rohm,bd71837",
+ .data = (void *)BD718XX_TYPE_BD71837,
+ },
+ {
+ .compatible = "rohm,bd71847",
+ .data = (void *)BD718XX_TYPE_BD71847,
+ },
{ }
};
-MODULE_DEVICE_TABLE(of, bd71837_of_match);
+MODULE_DEVICE_TABLE(of, bd718xx_of_match);
-static struct i2c_driver bd71837_i2c_driver = {
+static struct i2c_driver bd718xx_i2c_driver = {
.driver = {
.name = "rohm-bd718x7",
- .of_match_table = bd71837_of_match,
+ .of_match_table = bd718xx_of_match,
},
- .probe = bd71837_i2c_probe,
+ .probe = bd718xx_i2c_probe,
};
-static int __init bd71837_i2c_init(void)
+static int __init bd718xx_i2c_init(void)
{
- return i2c_add_driver(&bd71837_i2c_driver);
+ return i2c_add_driver(&bd718xx_i2c_driver);
}
/* Initialise early so consumer devices can complete system boot */
-subsys_initcall(bd71837_i2c_init);
+subsys_initcall(bd718xx_i2c_init);
-static void __exit bd71837_i2c_exit(void)
+static void __exit bd718xx_i2c_exit(void)
{
- i2c_del_driver(&bd71837_i2c_driver);
+ i2c_del_driver(&bd718xx_i2c_driver);
}
-module_exit(bd71837_i2c_exit);
+module_exit(bd718xx_i2c_exit);
MODULE_AUTHOR("Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>");
-MODULE_DESCRIPTION("ROHM BD71837 Power Management IC driver");
+MODULE_DESCRIPTION("ROHM BD71837/BD71847 Power Management IC driver");
MODULE_LICENSE("GPL");
.writeable_reg = twl6040_writeable_reg,
.cache_type = REGCACHE_RBTREE,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
static const struct regmap_irq twl6040_irqs[] = {
CRASHTYPE(USERCOPY_STACK_FRAME_FROM),
CRASHTYPE(USERCOPY_STACK_BEYOND),
CRASHTYPE(USERCOPY_KERNEL),
+ CRASHTYPE(USERCOPY_KERNEL_DS),
};
void lkdtm_USERCOPY_STACK_FRAME_FROM(void);
void lkdtm_USERCOPY_STACK_BEYOND(void);
void lkdtm_USERCOPY_KERNEL(void);
+void lkdtm_USERCOPY_KERNEL_DS(void);
#endif
vm_munmap(user_addr, PAGE_SIZE);
}
+void lkdtm_USERCOPY_KERNEL_DS(void)
+{
+ char __user *user_ptr = (char __user *)ERR_PTR(-EINVAL);
+ mm_segment_t old_fs = get_fs();
+ char buf[10] = {0};
+
+ pr_info("attempting copy_to_user on unmapped kernel address\n");
+ set_fs(KERNEL_DS);
+ if (copy_to_user(user_ptr, buf, sizeof(buf)))
+ pr_info("copy_to_user un unmapped kernel address failed\n");
+ set_fs(old_fs);
+}
+
void __init lkdtm_usercopy_init(void)
{
/* Prepare cache that lacks SLAB_USERCOPY flag. */
config PWRSEQ_SD8787
tristate "HW reset support for SD8787 BT + Wifi module"
- depends on OF && (MWIFIEX || BT_MRVL_SDIO)
+ depends on OF && (MWIFIEX || BT_MRVL_SDIO || LIBERTAS_SDIO)
help
This selects hardware reset support for the SD8787 BT + Wifi
module. By default this option is set to n.
brq->data.blocks = card->host->max_blk_count;
if (brq->data.blocks > 1) {
+ /*
+ * Some SD cards in SPI mode return a CRC error or even lock up
+ * completely when trying to read the last block using a
+ * multiblock read command.
+ */
+ if (mmc_host_is_spi(card->host) && (rq_data_dir(req) == READ) &&
+ (blk_rq_pos(req) + blk_rq_sectors(req) ==
+ get_capacity(md->disk)))
+ brq->data.blocks--;
+
/*
* After a read error, we redo the request one sector
* at a time in order to accurately determine which
int ret;
struct mmc_card *card = md->queue.card;
- device_add_disk(md->parent, md->disk);
+ device_add_disk(md->parent, md->disk, NULL);
md->force_ro.show = force_ro_show;
md->force_ro.store = force_ro_store;
sysfs_attr_init(&md->force_ro.attr);
struct gpio_descs *reset_gpios = pwrseq->reset_gpios;
if (!IS_ERR(reset_gpios)) {
- int i, *values;
+ unsigned long *values;
int nvalues = reset_gpios->ndescs;
- values = kmalloc_array(nvalues, sizeof(int), GFP_KERNEL);
+ values = bitmap_alloc(nvalues, GFP_KERNEL);
if (!values)
return;
- for (i = 0; i < nvalues; i++)
- values[i] = value;
+ if (value)
+ bitmap_fill(values, nvalues);
+ else
+ bitmap_zero(values, nvalues);
+
+ gpiod_set_array_value_cansleep(nvalues, reset_gpios->desc,
+ reset_gpios->info, values);
- gpiod_set_array_value_cansleep(nvalues, reset_gpios->desc, values);
kfree(values);
}
}
if unsure, say N.
+config MMC_STM32_SDMMC
+ bool "STMicroelectronics STM32 SDMMC Controller"
+ depends on MMC_ARMMMCI
+ default y
+ help
+ This selects the STMicroelectronics STM32 SDMMC host controller.
+ If you have a STM32 sdmmc host with internal DMA say Y here.
+
+ If unsure, say N.
+
config MMC_PXA
tristate "Intel PXA25x/26x/27x Multimedia Card Interface support"
depends on ARCH_PXA
tristate "SDHCI support for the BCM2835 & iProc SD/MMC Controller"
depends on ARCH_BCM2835 || ARCH_BCM_IPROC || COMPILE_TEST
depends on MMC_SDHCI_PLTFM
+ depends on OF || ACPI
default ARCH_BCM_IPROC
select MMC_SDHCI_IO_ACCESSORS
help
To compile this driver as a module, choose M here: the
module will be called sdricoh_cs.
+config MMC_SDHCI_SPRD
+ tristate "Spreadtrum SDIO host Controller"
+ depends on ARCH_SPRD
+ depends on MMC_SDHCI_PLTFM
+ select MMC_SDHCI_IO_ACCESSORS
+ help
+ This selects the SDIO Host Controller in Spreadtrum
+ SoCs, this driver supports R11(IP version: R11P0).
+
+ If you have a controller with this interface, say Y or M here.
+
+ If unsure, say N.
+
config MMC_TMIO_CORE
tristate
config MMC_SDHI_INTERNAL_DMAC
tristate "DMA for SDHI SD/SDIO controllers using on-chip bus mastering"
- depends on ARM64 || COMPILE_TEST
+ depends on ARM64 || ARCH_R8A77470 || COMPILE_TEST
depends on MMC_SDHI
- default MMC_SDHI if ARM64
+ default MMC_SDHI if (ARM64 || ARCH_R8A77470)
help
This provides DMA support for SDHI SD/SDIO controllers
using on-chip bus mastering. This supports the controllers
found in arm64 based SoCs.
+config MMC_UNIPHIER
+ tristate "UniPhier SD/eMMC Host Controller support"
+ depends on ARCH_UNIPHIER || COMPILE_TEST
+ depends on OF
+ select MMC_TMIO_CORE
+ help
+ This provides support for the SD/eMMC controller found in
+ UniPhier SoCs. The eMMC variant of this controller is used
+ only for 32-bit SoCs.
+
config MMC_CB710
tristate "ENE CB710 MMC/SD Interface support"
depends on PCI
config MMC_JZ4740
tristate "Ingenic JZ47xx SD/Multimedia Card Interface support"
- depends on MACH_JZ4740 || MACH_JZ4780
+ depends on MIPS
help
This selects support for the SD/MMC controller on Ingenic
JZ4740, JZ4750, JZ4770 and JZ4780 SoCs.
obj-$(CONFIG_MMC_ARMMMCI) += armmmci.o
armmmci-y := mmci.o
armmmci-$(CONFIG_MMC_QCOM_DML) += mmci_qcom_dml.o
+armmmci-$(CONFIG_MMC_STM32_SDMMC) += mmci_stm32_sdmmc.o
obj-$(CONFIG_MMC_PXA) += pxamci.o
obj-$(CONFIG_MMC_MXC) += mxcmmc.o
obj-$(CONFIG_MMC_MXS) += mxs-mmc.o
obj-$(CONFIG_MMC_SDHI) += renesas_sdhi_core.o
obj-$(CONFIG_MMC_SDHI_SYS_DMAC) += renesas_sdhi_sys_dmac.o
obj-$(CONFIG_MMC_SDHI_INTERNAL_DMAC) += renesas_sdhi_internal_dmac.o
+obj-$(CONFIG_MMC_UNIPHIER) += uniphier-sd.o
obj-$(CONFIG_MMC_CB710) += cb710-mmc.o
obj-$(CONFIG_MMC_VIA_SDMMC) += via-sdmmc.o
octeon-mmc-objs := cavium.o cavium-octeon.o
obj-$(CONFIG_MMC_SDHCI_MICROCHIP_PIC32) += sdhci-pic32.o
obj-$(CONFIG_MMC_SDHCI_BRCMSTB) += sdhci-brcmstb.o
obj-$(CONFIG_MMC_SDHCI_OMAP) += sdhci-omap.o
+obj-$(CONFIG_MMC_SDHCI_SPRD) += sdhci-sprd.o
obj-$(CONFIG_MMC_CQHCI) += cqhci.o
ifeq ($(CONFIG_CB710_DEBUG),y)
if (timing == MMC_TIMING_MMC_HS400) {
dqs |= DATA_STROBE_EN;
strobe = DQS_CTRL_RD_DELAY(strobe, priv->dqs_delay);
+ } else if (timing == MMC_TIMING_UHS_SDR104) {
+ dqs &= 0xffffff00;
} else {
dqs &= ~DATA_STROBE_EN;
}
if (ios->bus_width == MMC_BUS_WIDTH_8)
wanted <<= 1;
break;
+ case MMC_TIMING_UHS_SDR104:
+ case MMC_TIMING_UHS_SDR50:
+ clksel = (priv->sdr_timing & 0xfff8ffff) |
+ (priv->ciu_div << 16);
+ break;
+ case MMC_TIMING_UHS_DDR50:
+ clksel = (priv->ddr_timing & 0xfff8ffff) |
+ (priv->ciu_div << 16);
+ break;
default:
clksel = priv->sdr_timing;
}
struct clk *drive_clk;
};
+static unsigned long dw_mci_hi3798cv200_caps[] = {
+ MMC_CAP_CMD23,
+ MMC_CAP_CMD23,
+ MMC_CAP_CMD23
+};
+
static void dw_mci_hi3798cv200_set_ios(struct dw_mci *host, struct mmc_ios *ios)
{
struct hi3798cv200_priv *priv = host->priv;
}
static const struct dw_mci_drv_data hi3798cv200_data = {
+ .caps = dw_mci_hi3798cv200_caps,
+ .num_caps = ARRAY_SIZE(dw_mci_hi3798cv200_caps),
.init = dw_mci_hi3798cv200_init,
.set_ios = dw_mci_hi3798cv200_set_ios,
.execute_tuning = dw_mci_hi3798cv200_execute_tuning,
enum jz4740_mmc_version {
JZ_MMC_JZ4740,
- JZ_MMC_JZ4750,
+ JZ_MMC_JZ4725B,
JZ_MMC_JZ4780,
};
static void jz4740_mmc_write_irq_mask(struct jz4740_mmc_host *host,
uint32_t val)
{
- if (host->version >= JZ_MMC_JZ4750)
+ if (host->version >= JZ_MMC_JZ4725B)
return writel(val, host->base + JZ_REG_MMC_IMASK);
else
return writew(val, host->base + JZ_REG_MMC_IMASK);
static const struct of_device_id jz4740_mmc_of_match[] = {
{ .compatible = "ingenic,jz4740-mmc", .data = (void *) JZ_MMC_JZ4740 },
+ { .compatible = "ingenic,jz4725b-mmc", .data = (void *)JZ_MMC_JZ4725B },
{ .compatible = "ingenic,jz4780-mmc", .data = (void *) JZ_MMC_JZ4780 },
{},
};
switch (ios->power_mode) {
case MMC_POWER_OFF:
vdd = 0;
- /* fall-through: */
+ /* fall through */
case MMC_POWER_UP:
if (!IS_ERR(mmc->supply.vmmc)) {
host->error = mmc_regulator_set_ocr(mmc,
#include <linux/amba/bus.h>
#include <linux/clk.h>
#include <linux/scatterlist.h>
-#include <linux/gpio.h>
-#include <linux/of_gpio.h>
+#include <linux/of.h>
#include <linux/regulator/consumer.h>
#include <linux/dmaengine.h>
#include <linux/dma-mapping.h>
#include <linux/pm_runtime.h>
#include <linux/types.h>
#include <linux/pinctrl/consumer.h>
+#include <linux/reset.h>
#include <asm/div64.h>
#include <asm/io.h>
#define DRIVER_NAME "mmci-pl18x"
+#ifdef CONFIG_DMA_ENGINE
+void mmci_variant_init(struct mmci_host *host);
+#else
+static inline void mmci_variant_init(struct mmci_host *host) {}
+#endif
+
+#ifdef CONFIG_MMC_STM32_SDMMC
+void sdmmc_variant_init(struct mmci_host *host);
+#else
+static inline void sdmmc_variant_init(struct mmci_host *host) {}
+#endif
+
static unsigned int fmax = 515633;
static struct variant_data variant_arm = {
.fifosize = 16 * 4,
.fifohalfsize = 8 * 4,
+ .cmdreg_cpsm_enable = MCI_CPSM_ENABLE,
+ .cmdreg_lrsp_crc = MCI_CPSM_RESPONSE | MCI_CPSM_LONGRSP,
+ .cmdreg_srsp_crc = MCI_CPSM_RESPONSE,
+ .cmdreg_srsp = MCI_CPSM_RESPONSE,
.datalength_bits = 16,
+ .datactrl_blocksz = 11,
+ .datactrl_dpsm_enable = MCI_DPSM_ENABLE,
.pwrreg_powerup = MCI_PWR_UP,
.f_max = 100000000,
.reversed_irq_handling = true,
.mmcimask1 = true,
+ .irq_pio_mask = MCI_IRQ_PIO_MASK,
.start_err = MCI_STARTBITERR,
.opendrain = MCI_ROD,
+ .init = mmci_variant_init,
};
static struct variant_data variant_arm_extended_fifo = {
.fifosize = 128 * 4,
.fifohalfsize = 64 * 4,
+ .cmdreg_cpsm_enable = MCI_CPSM_ENABLE,
+ .cmdreg_lrsp_crc = MCI_CPSM_RESPONSE | MCI_CPSM_LONGRSP,
+ .cmdreg_srsp_crc = MCI_CPSM_RESPONSE,
+ .cmdreg_srsp = MCI_CPSM_RESPONSE,
.datalength_bits = 16,
+ .datactrl_blocksz = 11,
+ .datactrl_dpsm_enable = MCI_DPSM_ENABLE,
.pwrreg_powerup = MCI_PWR_UP,
.f_max = 100000000,
.mmcimask1 = true,
+ .irq_pio_mask = MCI_IRQ_PIO_MASK,
.start_err = MCI_STARTBITERR,
.opendrain = MCI_ROD,
+ .init = mmci_variant_init,
};
static struct variant_data variant_arm_extended_fifo_hwfc = {
.fifosize = 128 * 4,
.fifohalfsize = 64 * 4,
.clkreg_enable = MCI_ARM_HWFCEN,
+ .cmdreg_cpsm_enable = MCI_CPSM_ENABLE,
+ .cmdreg_lrsp_crc = MCI_CPSM_RESPONSE | MCI_CPSM_LONGRSP,
+ .cmdreg_srsp_crc = MCI_CPSM_RESPONSE,
+ .cmdreg_srsp = MCI_CPSM_RESPONSE,
.datalength_bits = 16,
+ .datactrl_blocksz = 11,
+ .datactrl_dpsm_enable = MCI_DPSM_ENABLE,
.pwrreg_powerup = MCI_PWR_UP,
.f_max = 100000000,
.mmcimask1 = true,
+ .irq_pio_mask = MCI_IRQ_PIO_MASK,
.start_err = MCI_STARTBITERR,
.opendrain = MCI_ROD,
+ .init = mmci_variant_init,
};
static struct variant_data variant_u300 = {
.fifohalfsize = 8 * 4,
.clkreg_enable = MCI_ST_U300_HWFCEN,
.clkreg_8bit_bus_enable = MCI_ST_8BIT_BUS,
+ .cmdreg_cpsm_enable = MCI_CPSM_ENABLE,
+ .cmdreg_lrsp_crc = MCI_CPSM_RESPONSE | MCI_CPSM_LONGRSP,
+ .cmdreg_srsp_crc = MCI_CPSM_RESPONSE,
+ .cmdreg_srsp = MCI_CPSM_RESPONSE,
.datalength_bits = 16,
+ .datactrl_blocksz = 11,
+ .datactrl_dpsm_enable = MCI_DPSM_ENABLE,
.datactrl_mask_sdio = MCI_DPSM_ST_SDIOEN,
.st_sdio = true,
.pwrreg_powerup = MCI_PWR_ON,
.pwrreg_clkgate = true,
.pwrreg_nopower = true,
.mmcimask1 = true,
+ .irq_pio_mask = MCI_IRQ_PIO_MASK,
.start_err = MCI_STARTBITERR,
.opendrain = MCI_OD,
+ .init = mmci_variant_init,
};
static struct variant_data variant_nomadik = {
.fifohalfsize = 8 * 4,
.clkreg = MCI_CLK_ENABLE,
.clkreg_8bit_bus_enable = MCI_ST_8BIT_BUS,
+ .cmdreg_cpsm_enable = MCI_CPSM_ENABLE,
+ .cmdreg_lrsp_crc = MCI_CPSM_RESPONSE | MCI_CPSM_LONGRSP,
+ .cmdreg_srsp_crc = MCI_CPSM_RESPONSE,
+ .cmdreg_srsp = MCI_CPSM_RESPONSE,
.datalength_bits = 24,
+ .datactrl_blocksz = 11,
+ .datactrl_dpsm_enable = MCI_DPSM_ENABLE,
.datactrl_mask_sdio = MCI_DPSM_ST_SDIOEN,
.st_sdio = true,
.st_clkdiv = true,
.pwrreg_clkgate = true,
.pwrreg_nopower = true,
.mmcimask1 = true,
+ .irq_pio_mask = MCI_IRQ_PIO_MASK,
.start_err = MCI_STARTBITERR,
.opendrain = MCI_OD,
+ .init = mmci_variant_init,
};
static struct variant_data variant_ux500 = {
.clkreg_enable = MCI_ST_UX500_HWFCEN,
.clkreg_8bit_bus_enable = MCI_ST_8BIT_BUS,
.clkreg_neg_edge_enable = MCI_ST_UX500_NEG_EDGE,
+ .cmdreg_cpsm_enable = MCI_CPSM_ENABLE,
+ .cmdreg_lrsp_crc = MCI_CPSM_RESPONSE | MCI_CPSM_LONGRSP,
+ .cmdreg_srsp_crc = MCI_CPSM_RESPONSE,
+ .cmdreg_srsp = MCI_CPSM_RESPONSE,
.datalength_bits = 24,
+ .datactrl_blocksz = 11,
+ .datactrl_dpsm_enable = MCI_DPSM_ENABLE,
.datactrl_mask_sdio = MCI_DPSM_ST_SDIOEN,
.st_sdio = true,
.st_clkdiv = true,
.busy_detect_mask = MCI_ST_BUSYENDMASK,
.pwrreg_nopower = true,
.mmcimask1 = true,
+ .irq_pio_mask = MCI_IRQ_PIO_MASK,
.start_err = MCI_STARTBITERR,
.opendrain = MCI_OD,
+ .init = mmci_variant_init,
};
static struct variant_data variant_ux500v2 = {
.clkreg_enable = MCI_ST_UX500_HWFCEN,
.clkreg_8bit_bus_enable = MCI_ST_8BIT_BUS,
.clkreg_neg_edge_enable = MCI_ST_UX500_NEG_EDGE,
+ .cmdreg_cpsm_enable = MCI_CPSM_ENABLE,
+ .cmdreg_lrsp_crc = MCI_CPSM_RESPONSE | MCI_CPSM_LONGRSP,
+ .cmdreg_srsp_crc = MCI_CPSM_RESPONSE,
+ .cmdreg_srsp = MCI_CPSM_RESPONSE,
.datactrl_mask_ddrmode = MCI_DPSM_ST_DDRMODE,
.datalength_bits = 24,
+ .datactrl_blocksz = 11,
+ .datactrl_dpsm_enable = MCI_DPSM_ENABLE,
.datactrl_mask_sdio = MCI_DPSM_ST_SDIOEN,
.st_sdio = true,
.st_clkdiv = true,
.busy_detect_mask = MCI_ST_BUSYENDMASK,
.pwrreg_nopower = true,
.mmcimask1 = true,
+ .irq_pio_mask = MCI_IRQ_PIO_MASK,
.start_err = MCI_STARTBITERR,
.opendrain = MCI_OD,
+ .init = mmci_variant_init,
};
static struct variant_data variant_stm32 = {
.clkreg_enable = MCI_ST_UX500_HWFCEN,
.clkreg_8bit_bus_enable = MCI_ST_8BIT_BUS,
.clkreg_neg_edge_enable = MCI_ST_UX500_NEG_EDGE,
+ .cmdreg_cpsm_enable = MCI_CPSM_ENABLE,
+ .cmdreg_lrsp_crc = MCI_CPSM_RESPONSE | MCI_CPSM_LONGRSP,
+ .cmdreg_srsp_crc = MCI_CPSM_RESPONSE,
+ .cmdreg_srsp = MCI_CPSM_RESPONSE,
+ .irq_pio_mask = MCI_IRQ_PIO_MASK,
.datalength_bits = 24,
+ .datactrl_blocksz = 11,
+ .datactrl_dpsm_enable = MCI_DPSM_ENABLE,
.datactrl_mask_sdio = MCI_DPSM_ST_SDIOEN,
.st_sdio = true,
.st_clkdiv = true,
.f_max = 48000000,
.pwrreg_clkgate = true,
.pwrreg_nopower = true,
+ .init = mmci_variant_init,
+};
+
+static struct variant_data variant_stm32_sdmmc = {
+ .fifosize = 16 * 4,
+ .fifohalfsize = 8 * 4,
+ .f_max = 208000000,
+ .stm32_clkdiv = true,
+ .cmdreg_cpsm_enable = MCI_CPSM_STM32_ENABLE,
+ .cmdreg_lrsp_crc = MCI_CPSM_STM32_LRSP_CRC,
+ .cmdreg_srsp_crc = MCI_CPSM_STM32_SRSP_CRC,
+ .cmdreg_srsp = MCI_CPSM_STM32_SRSP,
+ .data_cmd_enable = MCI_CPSM_STM32_CMDTRANS,
+ .irq_pio_mask = MCI_IRQ_PIO_STM32_MASK,
+ .datactrl_first = true,
+ .datacnt_useless = true,
+ .datalength_bits = 25,
+ .datactrl_blocksz = 14,
+ .stm32_idmabsize_mask = GENMASK(12, 5),
+ .init = sdmmc_variant_init,
};
static struct variant_data variant_qcom = {
MCI_QCOM_CLK_SELECT_IN_FBCLK,
.clkreg_8bit_bus_enable = MCI_QCOM_CLK_WIDEBUS_8,
.datactrl_mask_ddrmode = MCI_QCOM_CLK_SELECT_IN_DDR_MODE,
+ .cmdreg_cpsm_enable = MCI_CPSM_ENABLE,
+ .cmdreg_lrsp_crc = MCI_CPSM_RESPONSE | MCI_CPSM_LONGRSP,
+ .cmdreg_srsp_crc = MCI_CPSM_RESPONSE,
+ .cmdreg_srsp = MCI_CPSM_RESPONSE,
.data_cmd_enable = MCI_CPSM_QCOM_DATCMD,
.blksz_datactrl4 = true,
.datalength_bits = 24,
+ .datactrl_blocksz = 11,
+ .datactrl_dpsm_enable = MCI_DPSM_ENABLE,
.pwrreg_powerup = MCI_PWR_UP,
.f_max = 208000000,
.explicit_mclk_control = true,
.qcom_fifo = true,
.qcom_dml = true,
.mmcimask1 = true,
+ .irq_pio_mask = MCI_IRQ_PIO_MASK,
.start_err = MCI_STARTBITERR,
.opendrain = MCI_ROD,
.init = qcom_variant_init,
return busy;
}
-/*
- * Validate mmc prerequisites
- */
-static int mmci_validate_data(struct mmci_host *host,
- struct mmc_data *data)
-{
- if (!data)
- return 0;
-
- if (!is_power_of_2(data->blksz)) {
- dev_err(mmc_dev(host->mmc),
- "unsupported block size (%d bytes)\n", data->blksz);
- return -EINVAL;
- }
-
- return 0;
-}
-
static void mmci_reg_delay(struct mmci_host *host)
{
/*
/*
* This must be called with host->lock held
*/
-static void mmci_write_clkreg(struct mmci_host *host, u32 clk)
+void mmci_write_clkreg(struct mmci_host *host, u32 clk)
{
if (host->clk_reg != clk) {
host->clk_reg = clk;
/*
* This must be called with host->lock held
*/
-static void mmci_write_pwrreg(struct mmci_host *host, u32 pwr)
+void mmci_write_pwrreg(struct mmci_host *host, u32 pwr)
{
if (host->pwr_reg != pwr) {
host->pwr_reg = pwr;
mmci_write_clkreg(host, clk);
}
+void mmci_dma_release(struct mmci_host *host)
+{
+ if (host->ops && host->ops->dma_release)
+ host->ops->dma_release(host);
+
+ host->use_dma = false;
+}
+
+void mmci_dma_setup(struct mmci_host *host)
+{
+ if (!host->ops || !host->ops->dma_setup)
+ return;
+
+ if (host->ops->dma_setup(host))
+ return;
+
+ /* initialize pre request cookie */
+ host->next_cookie = 1;
+
+ host->use_dma = true;
+}
+
+/*
+ * Validate mmc prerequisites
+ */
+static int mmci_validate_data(struct mmci_host *host,
+ struct mmc_data *data)
+{
+ if (!data)
+ return 0;
+
+ if (!is_power_of_2(data->blksz)) {
+ dev_err(mmc_dev(host->mmc),
+ "unsupported block size (%d bytes)\n", data->blksz);
+ return -EINVAL;
+ }
+
+ if (host->ops && host->ops->validate_data)
+ return host->ops->validate_data(host, data);
+
+ return 0;
+}
+
+int mmci_prep_data(struct mmci_host *host, struct mmc_data *data, bool next)
+{
+ int err;
+
+ if (!host->ops || !host->ops->prep_data)
+ return 0;
+
+ err = host->ops->prep_data(host, data, next);
+
+ if (next && !err)
+ data->host_cookie = ++host->next_cookie < 0 ?
+ 1 : host->next_cookie;
+
+ return err;
+}
+
+void mmci_unprep_data(struct mmci_host *host, struct mmc_data *data,
+ int err)
+{
+ if (host->ops && host->ops->unprep_data)
+ host->ops->unprep_data(host, data, err);
+
+ data->host_cookie = 0;
+}
+
+void mmci_get_next_data(struct mmci_host *host, struct mmc_data *data)
+{
+ WARN_ON(data->host_cookie && data->host_cookie != host->next_cookie);
+
+ if (host->ops && host->ops->get_next_data)
+ host->ops->get_next_data(host, data);
+}
+
+int mmci_dma_start(struct mmci_host *host, unsigned int datactrl)
+{
+ struct mmc_data *data = host->data;
+ int ret;
+
+ if (!host->use_dma)
+ return -EINVAL;
+
+ ret = mmci_prep_data(host, data, false);
+ if (ret)
+ return ret;
+
+ if (!host->ops || !host->ops->dma_start)
+ return -EINVAL;
+
+ /* Okay, go for it. */
+ dev_vdbg(mmc_dev(host->mmc),
+ "Submit MMCI DMA job, sglen %d blksz %04x blks %04x flags %08x\n",
+ data->sg_len, data->blksz, data->blocks, data->flags);
+
+ host->ops->dma_start(host, &datactrl);
+
+ /* Trigger the DMA transfer */
+ mmci_write_datactrlreg(host, datactrl);
+
+ /*
+ * Let the MMCI say when the data is ended and it's time
+ * to fire next DMA request. When that happens, MMCI will
+ * call mmci_data_end()
+ */
+ writel(readl(host->base + MMCIMASK0) | MCI_DATAENDMASK,
+ host->base + MMCIMASK0);
+ return 0;
+}
+
+void mmci_dma_finalize(struct mmci_host *host, struct mmc_data *data)
+{
+ if (!host->use_dma)
+ return;
+
+ if (host->ops && host->ops->dma_finalize)
+ host->ops->dma_finalize(host, data);
+}
+
+void mmci_dma_error(struct mmci_host *host)
+{
+ if (!host->use_dma)
+ return;
+
+ if (host->ops && host->ops->dma_error)
+ host->ops->dma_error(host);
+}
+
static void
mmci_request_end(struct mmci_host *host, struct mmc_request *mrq)
{
if (host->singleirq) {
unsigned int mask0 = readl(base + MMCIMASK0);
- mask0 &= ~MCI_IRQ1MASK;
+ mask0 &= ~variant->irq_pio_mask;
mask0 |= mask;
writel(mask0, base + MMCIMASK0);
* no custom DMA interfaces are supported.
*/
#ifdef CONFIG_DMA_ENGINE
-static void mmci_dma_setup(struct mmci_host *host)
+struct mmci_dmae_next {
+ struct dma_async_tx_descriptor *desc;
+ struct dma_chan *chan;
+};
+
+struct mmci_dmae_priv {
+ struct dma_chan *cur;
+ struct dma_chan *rx_channel;
+ struct dma_chan *tx_channel;
+ struct dma_async_tx_descriptor *desc_current;
+ struct mmci_dmae_next next_data;
+};
+
+int mmci_dmae_setup(struct mmci_host *host)
{
const char *rxname, *txname;
+ struct mmci_dmae_priv *dmae;
- host->dma_rx_channel = dma_request_slave_channel(mmc_dev(host->mmc), "rx");
- host->dma_tx_channel = dma_request_slave_channel(mmc_dev(host->mmc), "tx");
+ dmae = devm_kzalloc(mmc_dev(host->mmc), sizeof(*dmae), GFP_KERNEL);
+ if (!dmae)
+ return -ENOMEM;
- /* initialize pre request cookie */
- host->next_data.cookie = 1;
+ host->dma_priv = dmae;
+
+ dmae->rx_channel = dma_request_slave_channel(mmc_dev(host->mmc),
+ "rx");
+ dmae->tx_channel = dma_request_slave_channel(mmc_dev(host->mmc),
+ "tx");
/*
* If only an RX channel is specified, the driver will
* attempt to use it bidirectionally, however if it is
* is specified but cannot be located, DMA will be disabled.
*/
- if (host->dma_rx_channel && !host->dma_tx_channel)
- host->dma_tx_channel = host->dma_rx_channel;
+ if (dmae->rx_channel && !dmae->tx_channel)
+ dmae->tx_channel = dmae->rx_channel;
- if (host->dma_rx_channel)
- rxname = dma_chan_name(host->dma_rx_channel);
+ if (dmae->rx_channel)
+ rxname = dma_chan_name(dmae->rx_channel);
else
rxname = "none";
- if (host->dma_tx_channel)
- txname = dma_chan_name(host->dma_tx_channel);
+ if (dmae->tx_channel)
+ txname = dma_chan_name(dmae->tx_channel);
else
txname = "none";
* Limit the maximum segment size in any SG entry according to
* the parameters of the DMA engine device.
*/
- if (host->dma_tx_channel) {
- struct device *dev = host->dma_tx_channel->device->dev;
+ if (dmae->tx_channel) {
+ struct device *dev = dmae->tx_channel->device->dev;
unsigned int max_seg_size = dma_get_max_seg_size(dev);
if (max_seg_size < host->mmc->max_seg_size)
host->mmc->max_seg_size = max_seg_size;
}
- if (host->dma_rx_channel) {
- struct device *dev = host->dma_rx_channel->device->dev;
+ if (dmae->rx_channel) {
+ struct device *dev = dmae->rx_channel->device->dev;
unsigned int max_seg_size = dma_get_max_seg_size(dev);
if (max_seg_size < host->mmc->max_seg_size)
host->mmc->max_seg_size = max_seg_size;
}
- if (host->ops && host->ops->dma_setup)
- host->ops->dma_setup(host);
+ if (!dmae->tx_channel || !dmae->rx_channel) {
+ mmci_dmae_release(host);
+ return -EINVAL;
+ }
+
+ return 0;
}
/*
* This is used in or so inline it
* so it can be discarded.
*/
-static inline void mmci_dma_release(struct mmci_host *host)
+void mmci_dmae_release(struct mmci_host *host)
{
- if (host->dma_rx_channel)
- dma_release_channel(host->dma_rx_channel);
- if (host->dma_tx_channel)
- dma_release_channel(host->dma_tx_channel);
- host->dma_rx_channel = host->dma_tx_channel = NULL;
-}
+ struct mmci_dmae_priv *dmae = host->dma_priv;
-static void mmci_dma_data_error(struct mmci_host *host)
-{
- dev_err(mmc_dev(host->mmc), "error during DMA transfer!\n");
- dmaengine_terminate_all(host->dma_current);
- host->dma_in_progress = false;
- host->dma_current = NULL;
- host->dma_desc_current = NULL;
- host->data->host_cookie = 0;
+ if (dmae->rx_channel)
+ dma_release_channel(dmae->rx_channel);
+ if (dmae->tx_channel)
+ dma_release_channel(dmae->tx_channel);
+ dmae->rx_channel = dmae->tx_channel = NULL;
}
static void mmci_dma_unmap(struct mmci_host *host, struct mmc_data *data)
{
+ struct mmci_dmae_priv *dmae = host->dma_priv;
struct dma_chan *chan;
if (data->flags & MMC_DATA_READ)
- chan = host->dma_rx_channel;
+ chan = dmae->rx_channel;
else
- chan = host->dma_tx_channel;
+ chan = dmae->tx_channel;
dma_unmap_sg(chan->device->dev, data->sg, data->sg_len,
mmc_get_dma_dir(data));
}
-static void mmci_dma_finalize(struct mmci_host *host, struct mmc_data *data)
+void mmci_dmae_error(struct mmci_host *host)
{
+ struct mmci_dmae_priv *dmae = host->dma_priv;
+
+ if (!dma_inprogress(host))
+ return;
+
+ dev_err(mmc_dev(host->mmc), "error during DMA transfer!\n");
+ dmaengine_terminate_all(dmae->cur);
+ host->dma_in_progress = false;
+ dmae->cur = NULL;
+ dmae->desc_current = NULL;
+ host->data->host_cookie = 0;
+
+ mmci_dma_unmap(host, host->data);
+}
+
+void mmci_dmae_finalize(struct mmci_host *host, struct mmc_data *data)
+{
+ struct mmci_dmae_priv *dmae = host->dma_priv;
u32 status;
int i;
+ if (!dma_inprogress(host))
+ return;
+
/* Wait up to 1ms for the DMA to complete */
for (i = 0; ; i++) {
status = readl(host->base + MMCISTATUS);
* contiguous buffers. On TX, we'll get a FIFO underrun error.
*/
if (status & MCI_RXDATAAVLBLMASK) {
- mmci_dma_data_error(host);
+ mmci_dma_error(host);
if (!data->error)
data->error = -EIO;
- }
-
- if (!data->host_cookie)
+ } else if (!data->host_cookie) {
mmci_dma_unmap(host, data);
+ }
/*
* Use of DMA with scatter-gather is impossible.
}
host->dma_in_progress = false;
- host->dma_current = NULL;
- host->dma_desc_current = NULL;
+ dmae->cur = NULL;
+ dmae->desc_current = NULL;
}
/* prepares DMA channel and DMA descriptor, returns non-zero on failure */
-static int __mmci_dma_prep_data(struct mmci_host *host, struct mmc_data *data,
+static int _mmci_dmae_prep_data(struct mmci_host *host, struct mmc_data *data,
struct dma_chan **dma_chan,
struct dma_async_tx_descriptor **dma_desc)
{
+ struct mmci_dmae_priv *dmae = host->dma_priv;
struct variant_data *variant = host->variant;
struct dma_slave_config conf = {
.src_addr = host->phybase + MMCIFIFO,
if (data->flags & MMC_DATA_READ) {
conf.direction = DMA_DEV_TO_MEM;
- chan = host->dma_rx_channel;
+ chan = dmae->rx_channel;
} else {
conf.direction = DMA_MEM_TO_DEV;
- chan = host->dma_tx_channel;
+ chan = dmae->tx_channel;
}
/* If there's no DMA channel, fall back to PIO */
return -ENOMEM;
}
-static inline int mmci_dma_prep_data(struct mmci_host *host,
- struct mmc_data *data)
+int mmci_dmae_prep_data(struct mmci_host *host,
+ struct mmc_data *data,
+ bool next)
{
+ struct mmci_dmae_priv *dmae = host->dma_priv;
+ struct mmci_dmae_next *nd = &dmae->next_data;
+
+ if (!host->use_dma)
+ return -EINVAL;
+
+ if (next)
+ return _mmci_dmae_prep_data(host, data, &nd->chan, &nd->desc);
/* Check if next job is already prepared. */
- if (host->dma_current && host->dma_desc_current)
+ if (dmae->cur && dmae->desc_current)
return 0;
/* No job were prepared thus do it now. */
- return __mmci_dma_prep_data(host, data, &host->dma_current,
- &host->dma_desc_current);
-}
-
-static inline int mmci_dma_prep_next(struct mmci_host *host,
- struct mmc_data *data)
-{
- struct mmci_host_next *nd = &host->next_data;
- return __mmci_dma_prep_data(host, data, &nd->dma_chan, &nd->dma_desc);
+ return _mmci_dmae_prep_data(host, data, &dmae->cur,
+ &dmae->desc_current);
}
-static int mmci_dma_start_data(struct mmci_host *host, unsigned int datactrl)
+int mmci_dmae_start(struct mmci_host *host, unsigned int *datactrl)
{
- int ret;
+ struct mmci_dmae_priv *dmae = host->dma_priv;
struct mmc_data *data = host->data;
- ret = mmci_dma_prep_data(host, host->data);
- if (ret)
- return ret;
-
- /* Okay, go for it. */
- dev_vdbg(mmc_dev(host->mmc),
- "Submit MMCI DMA job, sglen %d blksz %04x blks %04x flags %08x\n",
- data->sg_len, data->blksz, data->blocks, data->flags);
host->dma_in_progress = true;
- dmaengine_submit(host->dma_desc_current);
- dma_async_issue_pending(host->dma_current);
+ dmaengine_submit(dmae->desc_current);
+ dma_async_issue_pending(dmae->cur);
if (host->variant->qcom_dml)
dml_start_xfer(host, data);
- datactrl |= MCI_DPSM_DMAENABLE;
+ *datactrl |= MCI_DPSM_DMAENABLE;
- /* Trigger the DMA transfer */
- mmci_write_datactrlreg(host, datactrl);
-
- /*
- * Let the MMCI say when the data is ended and it's time
- * to fire next DMA request. When that happens, MMCI will
- * call mmci_data_end()
- */
- writel(readl(host->base + MMCIMASK0) | MCI_DATAENDMASK,
- host->base + MMCIMASK0);
return 0;
}
-static void mmci_get_next_data(struct mmci_host *host, struct mmc_data *data)
+void mmci_dmae_get_next_data(struct mmci_host *host, struct mmc_data *data)
{
- struct mmci_host_next *next = &host->next_data;
-
- WARN_ON(data->host_cookie && data->host_cookie != next->cookie);
- WARN_ON(!data->host_cookie && (next->dma_desc || next->dma_chan));
-
- host->dma_desc_current = next->dma_desc;
- host->dma_current = next->dma_chan;
- next->dma_desc = NULL;
- next->dma_chan = NULL;
-}
+ struct mmci_dmae_priv *dmae = host->dma_priv;
+ struct mmci_dmae_next *next = &dmae->next_data;
-static void mmci_pre_request(struct mmc_host *mmc, struct mmc_request *mrq)
-{
- struct mmci_host *host = mmc_priv(mmc);
- struct mmc_data *data = mrq->data;
- struct mmci_host_next *nd = &host->next_data;
-
- if (!data)
+ if (!host->use_dma)
return;
- BUG_ON(data->host_cookie);
+ WARN_ON(!data->host_cookie && (next->desc || next->chan));
- if (mmci_validate_data(host, data))
- return;
-
- if (!mmci_dma_prep_next(host, data))
- data->host_cookie = ++nd->cookie < 0 ? 1 : nd->cookie;
+ dmae->desc_current = next->desc;
+ dmae->cur = next->chan;
+ next->desc = NULL;
+ next->chan = NULL;
}
-static void mmci_post_request(struct mmc_host *mmc, struct mmc_request *mrq,
- int err)
+void mmci_dmae_unprep_data(struct mmci_host *host,
+ struct mmc_data *data, int err)
+
{
- struct mmci_host *host = mmc_priv(mmc);
- struct mmc_data *data = mrq->data;
+ struct mmci_dmae_priv *dmae = host->dma_priv;
- if (!data || !data->host_cookie)
+ if (!host->use_dma)
return;
mmci_dma_unmap(host, data);
if (err) {
- struct mmci_host_next *next = &host->next_data;
+ struct mmci_dmae_next *next = &dmae->next_data;
struct dma_chan *chan;
if (data->flags & MMC_DATA_READ)
- chan = host->dma_rx_channel;
+ chan = dmae->rx_channel;
else
- chan = host->dma_tx_channel;
+ chan = dmae->tx_channel;
dmaengine_terminate_all(chan);
- if (host->dma_desc_current == next->dma_desc)
- host->dma_desc_current = NULL;
+ if (dmae->desc_current == next->desc)
+ dmae->desc_current = NULL;
- if (host->dma_current == next->dma_chan) {
+ if (dmae->cur == next->chan) {
host->dma_in_progress = false;
- host->dma_current = NULL;
+ dmae->cur = NULL;
}
- next->dma_desc = NULL;
- next->dma_chan = NULL;
- data->host_cookie = 0;
+ next->desc = NULL;
+ next->chan = NULL;
}
}
-#else
-/* Blank functions if the DMA engine is not available */
-static void mmci_get_next_data(struct mmci_host *host, struct mmc_data *data)
-{
-}
-static inline void mmci_dma_setup(struct mmci_host *host)
-{
-}
+static struct mmci_host_ops mmci_variant_ops = {
+ .prep_data = mmci_dmae_prep_data,
+ .unprep_data = mmci_dmae_unprep_data,
+ .get_next_data = mmci_dmae_get_next_data,
+ .dma_setup = mmci_dmae_setup,
+ .dma_release = mmci_dmae_release,
+ .dma_start = mmci_dmae_start,
+ .dma_finalize = mmci_dmae_finalize,
+ .dma_error = mmci_dmae_error,
+};
-static inline void mmci_dma_release(struct mmci_host *host)
+void mmci_variant_init(struct mmci_host *host)
{
+ host->ops = &mmci_variant_ops;
}
+#endif
-static inline void mmci_dma_unmap(struct mmci_host *host, struct mmc_data *data)
+static void mmci_pre_request(struct mmc_host *mmc, struct mmc_request *mrq)
{
-}
+ struct mmci_host *host = mmc_priv(mmc);
+ struct mmc_data *data = mrq->data;
-static inline void mmci_dma_finalize(struct mmci_host *host,
- struct mmc_data *data)
-{
-}
+ if (!data)
+ return;
-static inline void mmci_dma_data_error(struct mmci_host *host)
-{
+ WARN_ON(data->host_cookie);
+
+ if (mmci_validate_data(host, data))
+ return;
+
+ mmci_prep_data(host, data, true);
}
-static inline int mmci_dma_start_data(struct mmci_host *host, unsigned int datactrl)
+static void mmci_post_request(struct mmc_host *mmc, struct mmc_request *mrq,
+ int err)
{
- return -ENOSYS;
-}
+ struct mmci_host *host = mmc_priv(mmc);
+ struct mmc_data *data = mrq->data;
-#define mmci_pre_request NULL
-#define mmci_post_request NULL
+ if (!data || !data->host_cookie)
+ return;
-#endif
+ mmci_unprep_data(host, data, err);
+}
static void mmci_start_data(struct mmci_host *host, struct mmc_data *data)
{
BUG_ON(1 << blksz_bits != data->blksz);
if (variant->blksz_datactrl16)
- datactrl = MCI_DPSM_ENABLE | (data->blksz << 16);
+ datactrl = variant->datactrl_dpsm_enable | (data->blksz << 16);
else if (variant->blksz_datactrl4)
- datactrl = MCI_DPSM_ENABLE | (data->blksz << 4);
+ datactrl = variant->datactrl_dpsm_enable | (data->blksz << 4);
else
- datactrl = MCI_DPSM_ENABLE | blksz_bits << 4;
+ datactrl = variant->datactrl_dpsm_enable | blksz_bits << 4;
if (data->flags & MMC_DATA_READ)
datactrl |= MCI_DPSM_DIRECTION;
* Attempt to use DMA operation mode, if this
* should fail, fall back to PIO mode
*/
- if (!mmci_dma_start_data(host, datactrl))
+ if (!mmci_dma_start(host, datactrl))
return;
/* IRQ mode, map the SG list for CPU reading/writing */
dev_dbg(mmc_dev(host->mmc), "op %02x arg %08x flags %08x\n",
cmd->opcode, cmd->arg, cmd->flags);
- if (readl(base + MMCICOMMAND) & MCI_CPSM_ENABLE) {
+ if (readl(base + MMCICOMMAND) & host->variant->cmdreg_cpsm_enable) {
writel(0, base + MMCICOMMAND);
mmci_reg_delay(host);
}
- c |= cmd->opcode | MCI_CPSM_ENABLE;
+ c |= cmd->opcode | host->variant->cmdreg_cpsm_enable;
if (cmd->flags & MMC_RSP_PRESENT) {
if (cmd->flags & MMC_RSP_136)
- c |= MCI_CPSM_LONGRSP;
- c |= MCI_CPSM_RESPONSE;
+ c |= host->variant->cmdreg_lrsp_crc;
+ else if (cmd->flags & MMC_RSP_CRC)
+ c |= host->variant->cmdreg_srsp_crc;
+ else
+ c |= host->variant->cmdreg_srsp;
}
if (/*interrupt*/0)
c |= MCI_CPSM_INTERRUPT;
mmci_data_irq(struct mmci_host *host, struct mmc_data *data,
unsigned int status)
{
+ unsigned int status_err;
+
/* Make sure we have data to handle */
if (!data)
return;
/* First check for errors */
- if (status & (MCI_DATACRCFAIL | MCI_DATATIMEOUT |
- host->variant->start_err |
- MCI_TXUNDERRUN | MCI_RXOVERRUN)) {
+ status_err = status & (host->variant->start_err |
+ MCI_DATACRCFAIL | MCI_DATATIMEOUT |
+ MCI_TXUNDERRUN | MCI_RXOVERRUN);
+
+ if (status_err) {
u32 remain, success;
/* Terminate the DMA transfer */
- if (dma_inprogress(host)) {
- mmci_dma_data_error(host);
- mmci_dma_unmap(host, data);
- }
+ mmci_dma_error(host);
/*
* Calculate how far we are into the transfer. Note that
* can be as much as a FIFO-worth of data ahead. This
* matters for FIFO overruns only.
*/
- remain = readl(host->base + MMCIDATACNT);
- success = data->blksz * data->blocks - remain;
+ if (!host->variant->datacnt_useless) {
+ remain = readl(host->base + MMCIDATACNT);
+ success = data->blksz * data->blocks - remain;
+ } else {
+ success = 0;
+ }
dev_dbg(mmc_dev(host->mmc), "MCI ERROR IRQ, status 0x%08x at 0x%08x\n",
- status, success);
- if (status & MCI_DATACRCFAIL) {
+ status_err, success);
+ if (status_err & MCI_DATACRCFAIL) {
/* Last block was not successful */
success -= 1;
data->error = -EILSEQ;
- } else if (status & MCI_DATATIMEOUT) {
+ } else if (status_err & MCI_DATATIMEOUT) {
data->error = -ETIMEDOUT;
- } else if (status & MCI_STARTBITERR) {
+ } else if (status_err & MCI_STARTBITERR) {
data->error = -ECOMM;
- } else if (status & MCI_TXUNDERRUN) {
+ } else if (status_err & MCI_TXUNDERRUN) {
data->error = -EIO;
- } else if (status & MCI_RXOVERRUN) {
+ } else if (status_err & MCI_RXOVERRUN) {
if (success > host->variant->fifosize)
success -= host->variant->fifosize;
else
dev_err(mmc_dev(host->mmc), "stray MCI_DATABLOCKEND interrupt\n");
if (status & MCI_DATAEND || data->error) {
- if (dma_inprogress(host))
- mmci_dma_finalize(host, data);
+ mmci_dma_finalize(host, data);
+
mmci_stop_data(host);
if (!data->error)
if ((!sbc && !cmd->data) || cmd->error) {
if (host->data) {
/* Terminate the DMA transfer */
- if (dma_inprogress(host)) {
- mmci_dma_data_error(host);
- mmci_dma_unmap(host, host->data);
- }
+ mmci_dma_error(host);
+
mmci_stop_data(host);
}
mmci_request_end(host, host->mrq);
} else if (sbc) {
mmci_start_command(host, host->mrq->cmd, 0);
- } else if (!(cmd->data->flags & MMC_DATA_READ)) {
+ } else if (!host->variant->datactrl_first &&
+ !(cmd->data->flags & MMC_DATA_READ)) {
mmci_start_data(host, cmd->data);
}
}
if (status & host->mask1_reg)
mmci_pio_irq(irq, dev_id);
- status &= ~MCI_IRQ1MASK;
+ status &= ~host->variant->irq_pio_mask;
}
/*
if (mrq->data)
mmci_get_next_data(host, mrq->data);
- if (mrq->data && mrq->data->flags & MMC_DATA_READ)
+ if (mrq->data &&
+ (host->variant->datactrl_first || mrq->data->flags & MMC_DATA_READ))
mmci_start_data(host, mrq->data);
if (mrq->sbc)
spin_lock_irqsave(&host->lock, flags);
- mmci_set_clkreg(host, ios->clock);
- mmci_write_pwrreg(host, pwr);
+ if (host->ops && host->ops->set_clkreg)
+ host->ops->set_clkreg(host, ios->clock);
+ else
+ mmci_set_clkreg(host, ios->clock);
+
+ if (host->ops && host->ops->set_pwrreg)
+ host->ops->set_pwrreg(host, pwr);
+ else
+ mmci_write_pwrreg(host, pwr);
+
mmci_reg_delay(host);
spin_unlock_irqrestore(&host->lock, flags);
host->pwr_reg_add |= MCI_ST_CMDDIREN;
if (of_get_property(np, "st,sig-pin-fbclk", NULL))
host->pwr_reg_add |= MCI_ST_FBCLKEN;
+ if (of_get_property(np, "st,sig-dir", NULL))
+ host->pwr_reg_add |= MCI_STM32_DIRPOL;
+ if (of_get_property(np, "st,neg-edge", NULL))
+ host->clk_reg_add |= MCI_STM32_CLK_NEGEDGE;
+ if (of_get_property(np, "st,use-ckin", NULL))
+ host->clk_reg_add |= MCI_STM32_CLK_SELCKIN;
if (of_get_property(np, "mmc-cap-mmc-highspeed", NULL))
mmc->caps |= MMC_CAP_MMC_HIGHSPEED;
*/
if (variant->st_clkdiv)
mmc->f_min = DIV_ROUND_UP(host->mclk, 257);
+ else if (variant->stm32_clkdiv)
+ mmc->f_min = DIV_ROUND_UP(host->mclk, 2046);
else if (variant->explicit_mclk_control)
mmc->f_min = clk_round_rate(host->clk, 100000);
else
dev_dbg(mmc_dev(mmc), "clocking block at %u Hz\n", mmc->f_max);
+ host->rst = devm_reset_control_get_optional_exclusive(&dev->dev, NULL);
+ if (IS_ERR(host->rst)) {
+ ret = PTR_ERR(host->rst);
+ goto clk_disable;
+ }
+
/* Get regulators and the supported OCR mask */
ret = mmc_regulator_get_supply(mmc);
if (ret)
else if (plat->ocr_mask)
dev_warn(mmc_dev(mmc), "Platform OCR mask is ignored\n");
- /* DT takes precedence over platform data. */
- if (!np) {
- if (!plat->cd_invert)
- mmc->caps2 |= MMC_CAP2_CD_ACTIVE_HIGH;
- mmc->caps2 |= MMC_CAP2_RO_ACTIVE_HIGH;
- }
-
/* We support these capabilities. */
mmc->caps |= MMC_CAP_CMD23;
/*
* Block size can be up to 2048 bytes, but must be a power of two.
*/
- mmc->max_blk_size = 1 << 11;
+ mmc->max_blk_size = 1 << variant->datactrl_blocksz;
/*
* Limit the number of blocks transferred so that we don't overflow
* the maximum request size.
*/
- mmc->max_blk_count = mmc->max_req_size >> 11;
+ mmc->max_blk_count = mmc->max_req_size >> variant->datactrl_blocksz;
spin_lock_init(&host->lock);
* - not using DT but using a descriptor table, or
* - using a table of descriptors ALONGSIDE DT, or
* look up these descriptors named "cd" and "wp" right here, fail
- * silently of these do not exist and proceed to try platform data
+ * silently of these do not exist
*/
if (!np) {
ret = mmc_gpiod_request_cd(mmc, "cd", 0, false, 0, NULL);
- if (ret < 0) {
- if (ret == -EPROBE_DEFER)
- goto clk_disable;
- else if (gpio_is_valid(plat->gpio_cd)) {
- ret = mmc_gpio_request_cd(mmc, plat->gpio_cd, 0);
- if (ret)
- goto clk_disable;
- }
- }
+ if (ret == -EPROBE_DEFER)
+ goto clk_disable;
ret = mmc_gpiod_request_ro(mmc, "wp", 0, false, 0, NULL);
- if (ret < 0) {
- if (ret == -EPROBE_DEFER)
- goto clk_disable;
- else if (gpio_is_valid(plat->gpio_wp)) {
- ret = mmc_gpio_request_ro(mmc, plat->gpio_wp);
- if (ret)
- goto clk_disable;
- }
- }
+ if (ret == -EPROBE_DEFER)
+ goto clk_disable;
}
ret = devm_request_irq(&dev->dev, dev->irq[0], mmci_irq, IRQF_SHARED,
goto clk_disable;
}
- writel(MCI_IRQENABLE, host->base + MMCIMASK0);
+ writel(MCI_IRQENABLE | variant->start_err, host->base + MMCIMASK0);
amba_set_drvdata(dev, mmc);
writel(host->datactrl_reg, host->base + MMCIDATACTRL);
writel(host->pwr_reg, host->base + MMCIPOWER);
}
- writel(MCI_IRQENABLE, host->base + MMCIMASK0);
+ writel(MCI_IRQENABLE | host->variant->start_err,
+ host->base + MMCIMASK0);
mmci_reg_delay(host);
spin_unlock_irqrestore(&host->lock, flags);
.mask = 0x00ffffff,
.data = &variant_stm32,
},
+ {
+ .id = 0x10153180,
+ .mask = 0xf0ffffff,
+ .data = &variant_stm32_sdmmc,
+ },
/* Qualcomm variants */
{
.id = 0x00051180,
#define MCI_ST_DATA31DIREN (1 << 5)
#define MCI_ST_FBCLKEN (1 << 7)
#define MCI_ST_DATA74DIREN (1 << 8)
+/*
+ * The STM32 sdmmc does not have PWR_UP/OD/ROD
+ * and uses the power register for
+ */
+#define MCI_STM32_PWR_CYC 0x02
+#define MCI_STM32_VSWITCH BIT(2)
+#define MCI_STM32_VSWITCHEN BIT(3)
+#define MCI_STM32_DIRPOL BIT(4)
#define MMCICLOCK 0x004
#define MCI_CLK_ENABLE (1 << 8)
#define MCI_QCOM_CLK_SELECT_IN_FBCLK BIT(15)
#define MCI_QCOM_CLK_SELECT_IN_DDR_MODE (BIT(14) | BIT(15))
+/* Modified on STM32 sdmmc */
+#define MCI_STM32_CLK_CLKDIV_MSK GENMASK(9, 0)
+#define MCI_STM32_CLK_WIDEBUS_4 BIT(14)
+#define MCI_STM32_CLK_WIDEBUS_8 BIT(15)
+#define MCI_STM32_CLK_NEGEDGE BIT(16)
+#define MCI_STM32_CLK_HWFCEN BIT(17)
+#define MCI_STM32_CLK_DDR BIT(18)
+#define MCI_STM32_CLK_BUSSPEED BIT(19)
+#define MCI_STM32_CLK_SEL_MSK GENMASK(21, 20)
+#define MCI_STM32_CLK_SELCK (0 << 20)
+#define MCI_STM32_CLK_SELCKIN (1 << 20)
+#define MCI_STM32_CLK_SELFBCK (2 << 20)
+
#define MMCIARGUMENT 0x008
/* The command register controls the Command Path State Machine (CPSM) */
#define MCI_CPSM_QCOM_CCSDISABLE BIT(15)
#define MCI_CPSM_QCOM_AUTO_CMD19 BIT(16)
#define MCI_CPSM_QCOM_AUTO_CMD21 BIT(21)
+/* Command register in STM32 sdmmc versions */
+#define MCI_CPSM_STM32_CMDTRANS BIT(6)
+#define MCI_CPSM_STM32_CMDSTOP BIT(7)
+#define MCI_CPSM_STM32_WAITRESP_MASK GENMASK(9, 8)
+#define MCI_CPSM_STM32_NORSP (0 << 8)
+#define MCI_CPSM_STM32_SRSP_CRC (1 << 8)
+#define MCI_CPSM_STM32_SRSP (2 << 8)
+#define MCI_CPSM_STM32_LRSP_CRC (3 << 8)
+#define MCI_CPSM_STM32_ENABLE BIT(12)
#define MMCIRESPCMD 0x010
#define MMCIRESPONSE0 0x014
#define MCI_ST_SDIOIT (1 << 22)
#define MCI_ST_CEATAEND (1 << 23)
#define MCI_ST_CARDBUSY (1 << 24)
+/* Extended status bits for the STM32 variants */
+#define MCI_STM32_BUSYD0 BIT(20)
#define MMCICLEAR 0x038
#define MCI_CMDCRCFAILCLR (1 << 0)
#define MCI_ST_SDIOITMASK (1 << 22)
#define MCI_ST_CEATAENDMASK (1 << 23)
#define MCI_ST_BUSYENDMASK (1 << 24)
+/* Extended status bits for the STM32 variants */
+#define MCI_STM32_BUSYD0ENDMASK BIT(21)
#define MMCIMASK1 0x040
#define MMCIFIFOCNT 0x048
#define MMCIFIFO 0x080 /* to 0x0bc */
+/* STM32 sdmmc registers for IDMA (Internal DMA) */
+#define MMCI_STM32_IDMACTRLR 0x050
+#define MMCI_STM32_IDMAEN BIT(0)
+#define MMCI_STM32_IDMALLIEN BIT(1)
+
+#define MMCI_STM32_IDMABSIZER 0x054
+#define MMCI_STM32_IDMABNDT_SHIFT 5
+#define MMCI_STM32_IDMABNDT_MASK GENMASK(12, 5)
+
+#define MMCI_STM32_IDMABASE0R 0x058
+
+#define MMCI_STM32_IDMALAR 0x64
+#define MMCI_STM32_IDMALA_MASK GENMASK(13, 0)
+#define MMCI_STM32_ABR BIT(29)
+#define MMCI_STM32_ULS BIT(30)
+#define MMCI_STM32_ULA BIT(31)
+
+#define MMCI_STM32_IDMABAR 0x68
+
#define MCI_IRQENABLE \
- (MCI_CMDCRCFAILMASK|MCI_DATACRCFAILMASK|MCI_CMDTIMEOUTMASK| \
- MCI_DATATIMEOUTMASK|MCI_TXUNDERRUNMASK|MCI_RXOVERRUNMASK| \
- MCI_CMDRESPENDMASK|MCI_CMDSENTMASK|MCI_STARTBITERRMASK)
+ (MCI_CMDCRCFAILMASK | MCI_DATACRCFAILMASK | MCI_CMDTIMEOUTMASK | \
+ MCI_DATATIMEOUTMASK | MCI_TXUNDERRUNMASK | MCI_RXOVERRUNMASK | \
+ MCI_CMDRESPENDMASK | MCI_CMDSENTMASK)
/* These interrupts are directed to IRQ1 when two IRQ lines are available */
-#define MCI_IRQ1MASK \
+#define MCI_IRQ_PIO_MASK \
(MCI_RXFIFOHALFFULLMASK | MCI_RXDATAAVLBLMASK | \
MCI_TXFIFOHALFEMPTYMASK)
+#define MCI_IRQ_PIO_STM32_MASK \
+ (MCI_RXFIFOHALFFULLMASK | MCI_TXFIFOHALFEMPTYMASK)
+
#define NR_SG 128
#define MMCI_PINCTRL_STATE_OPENDRAIN "opendrain"
* @clkreg_enable: enable value for MMCICLOCK register
* @clkreg_8bit_bus_enable: enable value for 8 bit bus
* @clkreg_neg_edge_enable: enable value for inverted data/cmd output
+ * @cmdreg_cpsm_enable: enable value for CPSM
+ * @cmdreg_lrsp_crc: enable value for long response with crc
+ * @cmdreg_srsp_crc: enable value for short response with crc
+ * @cmdreg_srsp: enable value for short response without crc
* @datalength_bits: number of bits in the MMCIDATALENGTH register
* @fifosize: number of bytes that can be written when MMCI_TXFIFOEMPTY
* is asserted (likewise for RX)
* @data_cmd_enable: enable value for data commands.
* @st_sdio: enable ST specific SDIO logic
* @st_clkdiv: true if using a ST-specific clock divider algorithm
+ * @stm32_clkdiv: true if using a STM32-specific clock divider algorithm
* @datactrl_mask_ddrmode: ddr mode mask in datactrl register.
* @blksz_datactrl16: true if Block size is at b16..b30 position in datactrl register
* @blksz_datactrl4: true if Block size is at b4..b16 position in datactrl
* register
* @datactrl_mask_sdio: SDIO enable mask in datactrl register
+ * @datactrl_blksz: block size in power of two
+ * @datactrl_dpsm_enable: enable value for DPSM
+ * @datactrl_first: true if data must be setup before send command
+ * @datacnt_useless: true if you could not use datacnt register to read
+ * remaining data
* @pwrreg_powerup: power up value for MMCIPOWER register
* @f_max: maximum clk frequency supported by the controller.
* @signal_direction: input/out direction of bus signals can be indicated
* @qcom_dml: enables qcom specific dma glue for dma transfers.
* @reversed_irq_handling: handle data irq before cmd irq.
* @mmcimask1: true if variant have a MMCIMASK1 register.
+ * @irq_pio_mask: bitmask used to manage interrupt pio transfert in mmcimask
+ * register
* @start_err: bitmask identifying the STARTBITERR bit inside MMCISTATUS
* register.
* @opendrain: bitmask identifying the OPENDRAIN bit inside MMCIPOWER register
+ * @dma_lli: true if variant has dma link list feature.
+ * @stm32_idmabsize_mask: stm32 sdmmc idma buffer size.
*/
struct variant_data {
unsigned int clkreg;
unsigned int clkreg_enable;
unsigned int clkreg_8bit_bus_enable;
unsigned int clkreg_neg_edge_enable;
+ unsigned int cmdreg_cpsm_enable;
+ unsigned int cmdreg_lrsp_crc;
+ unsigned int cmdreg_srsp_crc;
+ unsigned int cmdreg_srsp;
unsigned int datalength_bits;
unsigned int fifosize;
unsigned int fifohalfsize;
unsigned int data_cmd_enable;
unsigned int datactrl_mask_ddrmode;
unsigned int datactrl_mask_sdio;
- bool st_sdio;
- bool st_clkdiv;
- bool blksz_datactrl16;
- bool blksz_datactrl4;
+ unsigned int datactrl_blocksz;
+ unsigned int datactrl_dpsm_enable;
+ u8 datactrl_first:1;
+ u8 datacnt_useless:1;
+ u8 st_sdio:1;
+ u8 st_clkdiv:1;
+ u8 stm32_clkdiv:1;
+ u8 blksz_datactrl16:1;
+ u8 blksz_datactrl4:1;
u32 pwrreg_powerup;
u32 f_max;
- bool signal_direction;
- bool pwrreg_clkgate;
- bool busy_detect;
+ u8 signal_direction:1;
+ u8 pwrreg_clkgate:1;
+ u8 busy_detect:1;
u32 busy_dpsm_flag;
u32 busy_detect_flag;
u32 busy_detect_mask;
- bool pwrreg_nopower;
- bool explicit_mclk_control;
- bool qcom_fifo;
- bool qcom_dml;
- bool reversed_irq_handling;
- bool mmcimask1;
+ u8 pwrreg_nopower:1;
+ u8 explicit_mclk_control:1;
+ u8 qcom_fifo:1;
+ u8 qcom_dml:1;
+ u8 reversed_irq_handling:1;
+ u8 mmcimask1:1;
+ unsigned int irq_pio_mask;
u32 start_err;
u32 opendrain;
+ u8 dma_lli:1;
+ u32 stm32_idmabsize_mask;
void (*init)(struct mmci_host *host);
};
/* mmci variant callbacks */
struct mmci_host_ops {
- void (*dma_setup)(struct mmci_host *host);
-};
-
-struct mmci_host_next {
- struct dma_async_tx_descriptor *dma_desc;
- struct dma_chan *dma_chan;
- s32 cookie;
+ int (*validate_data)(struct mmci_host *host, struct mmc_data *data);
+ int (*prep_data)(struct mmci_host *host, struct mmc_data *data,
+ bool next);
+ void (*unprep_data)(struct mmci_host *host, struct mmc_data *data,
+ int err);
+ void (*get_next_data)(struct mmci_host *host, struct mmc_data *data);
+ int (*dma_setup)(struct mmci_host *host);
+ void (*dma_release)(struct mmci_host *host);
+ int (*dma_start)(struct mmci_host *host, unsigned int *datactrl);
+ void (*dma_finalize)(struct mmci_host *host, struct mmc_data *data);
+ void (*dma_error)(struct mmci_host *host);
+ void (*set_clkreg)(struct mmci_host *host, unsigned int desired);
+ void (*set_pwrreg)(struct mmci_host *host, unsigned int pwr);
};
struct mmci_host {
struct mmc_data *data;
struct mmc_host *mmc;
struct clk *clk;
- bool singleirq;
+ u8 singleirq:1;
+
+ struct reset_control *rst;
spinlock_t lock;
u32 pwr_reg;
u32 pwr_reg_add;
u32 clk_reg;
+ u32 clk_reg_add;
u32 datactrl_reg;
u32 busy_status;
u32 mask1_reg;
- bool vqmmc_enabled;
+ u8 vqmmc_enabled:1;
struct mmci_platform_data *plat;
struct mmci_host_ops *ops;
struct variant_data *variant;
unsigned int size;
int (*get_rx_fifocnt)(struct mmci_host *h, u32 status, int remain);
-#ifdef CONFIG_DMA_ENGINE
- /* DMA stuff */
- struct dma_chan *dma_current;
- struct dma_chan *dma_rx_channel;
- struct dma_chan *dma_tx_channel;
- struct dma_async_tx_descriptor *dma_desc_current;
- struct mmci_host_next next_data;
- bool dma_in_progress;
+ u8 use_dma:1;
+ u8 dma_in_progress:1;
+ void *dma_priv;
-#define dma_inprogress(host) ((host)->dma_in_progress)
-#else
-#define dma_inprogress(host) (0)
-#endif
+ s32 next_cookie;
};
+#define dma_inprogress(host) ((host)->dma_in_progress)
+
+void mmci_write_clkreg(struct mmci_host *host, u32 clk);
+void mmci_write_pwrreg(struct mmci_host *host, u32 pwr);
+
+int mmci_dmae_prep_data(struct mmci_host *host, struct mmc_data *data,
+ bool next);
+void mmci_dmae_unprep_data(struct mmci_host *host, struct mmc_data *data,
+ int err);
+void mmci_dmae_get_next_data(struct mmci_host *host, struct mmc_data *data);
+int mmci_dmae_setup(struct mmci_host *host);
+void mmci_dmae_release(struct mmci_host *host);
+int mmci_dmae_start(struct mmci_host *host, unsigned int *datactrl);
+void mmci_dmae_finalize(struct mmci_host *host, struct mmc_data *data);
+void mmci_dmae_error(struct mmci_host *host);
}
/* Initialize the dml hardware connected to SD Card controller */
-static void qcom_dma_setup(struct mmci_host *host)
+static int qcom_dma_setup(struct mmci_host *host)
{
u32 config;
void __iomem *base;
int consumer_id, producer_id;
struct device_node *np = host->mmc->parent->of_node;
+ if (mmci_dmae_setup(host))
+ return -EINVAL;
+
consumer_id = of_get_dml_pipe_index(np, "tx");
producer_id = of_get_dml_pipe_index(np, "rx");
if (producer_id < 0 || consumer_id < 0) {
host->variant->qcom_dml = false;
- return;
+ mmci_dmae_release(host);
+ return -EINVAL;
}
base = host->base + DML_OFFSET;
/* Make sure dml initialization is finished */
mb();
+
+ return 0;
}
static struct mmci_host_ops qcom_variant_ops = {
+ .prep_data = mmci_dmae_prep_data,
+ .unprep_data = mmci_dmae_unprep_data,
+ .get_next_data = mmci_dmae_get_next_data,
.dma_setup = qcom_dma_setup,
+ .dma_release = mmci_dmae_release,
+ .dma_start = mmci_dmae_start,
+ .dma_finalize = mmci_dmae_finalize,
+ .dma_error = mmci_dmae_error,
};
void qcom_variant_init(struct mmci_host *host)
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) STMicroelectronics 2018 - All Rights Reserved
+ * Author: Ludovic.barre@st.com for STMicroelectronics.
+ */
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/mmc/host.h>
+#include <linux/mmc/card.h>
+#include <linux/reset.h>
+#include <linux/scatterlist.h>
+#include "mmci.h"
+
+#define SDMMC_LLI_BUF_LEN PAGE_SIZE
+#define SDMMC_IDMA_BURST BIT(MMCI_STM32_IDMABNDT_SHIFT)
+
+struct sdmmc_lli_desc {
+ u32 idmalar;
+ u32 idmabase;
+ u32 idmasize;
+};
+
+struct sdmmc_priv {
+ dma_addr_t sg_dma;
+ void *sg_cpu;
+};
+
+int sdmmc_idma_validate_data(struct mmci_host *host,
+ struct mmc_data *data)
+{
+ struct scatterlist *sg;
+ int i;
+
+ /*
+ * idma has constraints on idmabase & idmasize for each element
+ * excepted the last element which has no constraint on idmasize
+ */
+ for_each_sg(data->sg, sg, data->sg_len - 1, i) {
+ if (!IS_ALIGNED(sg_dma_address(data->sg), sizeof(u32)) ||
+ !IS_ALIGNED(sg_dma_len(data->sg), SDMMC_IDMA_BURST)) {
+ dev_err(mmc_dev(host->mmc),
+ "unaligned scatterlist: ofst:%x length:%d\n",
+ data->sg->offset, data->sg->length);
+ return -EINVAL;
+ }
+ }
+
+ if (!IS_ALIGNED(sg_dma_address(data->sg), sizeof(u32))) {
+ dev_err(mmc_dev(host->mmc),
+ "unaligned last scatterlist: ofst:%x length:%d\n",
+ data->sg->offset, data->sg->length);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int _sdmmc_idma_prep_data(struct mmci_host *host,
+ struct mmc_data *data)
+{
+ int n_elem;
+
+ n_elem = dma_map_sg(mmc_dev(host->mmc),
+ data->sg,
+ data->sg_len,
+ mmc_get_dma_dir(data));
+
+ if (!n_elem) {
+ dev_err(mmc_dev(host->mmc), "dma_map_sg failed\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int sdmmc_idma_prep_data(struct mmci_host *host,
+ struct mmc_data *data, bool next)
+{
+ /* Check if job is already prepared. */
+ if (!next && data->host_cookie == host->next_cookie)
+ return 0;
+
+ return _sdmmc_idma_prep_data(host, data);
+}
+
+static void sdmmc_idma_unprep_data(struct mmci_host *host,
+ struct mmc_data *data, int err)
+{
+ dma_unmap_sg(mmc_dev(host->mmc), data->sg, data->sg_len,
+ mmc_get_dma_dir(data));
+}
+
+static int sdmmc_idma_setup(struct mmci_host *host)
+{
+ struct sdmmc_priv *idma;
+
+ idma = devm_kzalloc(mmc_dev(host->mmc), sizeof(*idma), GFP_KERNEL);
+ if (!idma)
+ return -ENOMEM;
+
+ host->dma_priv = idma;
+
+ if (host->variant->dma_lli) {
+ idma->sg_cpu = dmam_alloc_coherent(mmc_dev(host->mmc),
+ SDMMC_LLI_BUF_LEN,
+ &idma->sg_dma, GFP_KERNEL);
+ if (!idma->sg_cpu) {
+ dev_err(mmc_dev(host->mmc),
+ "Failed to alloc IDMA descriptor\n");
+ return -ENOMEM;
+ }
+ host->mmc->max_segs = SDMMC_LLI_BUF_LEN /
+ sizeof(struct sdmmc_lli_desc);
+ host->mmc->max_seg_size = host->variant->stm32_idmabsize_mask;
+ } else {
+ host->mmc->max_segs = 1;
+ host->mmc->max_seg_size = host->mmc->max_req_size;
+ }
+
+ return 0;
+}
+
+static int sdmmc_idma_start(struct mmci_host *host, unsigned int *datactrl)
+
+{
+ struct sdmmc_priv *idma = host->dma_priv;
+ struct sdmmc_lli_desc *desc = (struct sdmmc_lli_desc *)idma->sg_cpu;
+ struct mmc_data *data = host->data;
+ struct scatterlist *sg;
+ int i;
+
+ if (!host->variant->dma_lli || data->sg_len == 1) {
+ writel_relaxed(sg_dma_address(data->sg),
+ host->base + MMCI_STM32_IDMABASE0R);
+ writel_relaxed(MMCI_STM32_IDMAEN,
+ host->base + MMCI_STM32_IDMACTRLR);
+ return 0;
+ }
+
+ for_each_sg(data->sg, sg, data->sg_len, i) {
+ desc[i].idmalar = (i + 1) * sizeof(struct sdmmc_lli_desc);
+ desc[i].idmalar |= MMCI_STM32_ULA | MMCI_STM32_ULS
+ | MMCI_STM32_ABR;
+ desc[i].idmabase = sg_dma_address(sg);
+ desc[i].idmasize = sg_dma_len(sg);
+ }
+
+ /* notice the end of link list */
+ desc[data->sg_len - 1].idmalar &= ~MMCI_STM32_ULA;
+
+ dma_wmb();
+ writel_relaxed(idma->sg_dma, host->base + MMCI_STM32_IDMABAR);
+ writel_relaxed(desc[0].idmalar, host->base + MMCI_STM32_IDMALAR);
+ writel_relaxed(desc[0].idmabase, host->base + MMCI_STM32_IDMABASE0R);
+ writel_relaxed(desc[0].idmasize, host->base + MMCI_STM32_IDMABSIZER);
+ writel_relaxed(MMCI_STM32_IDMAEN | MMCI_STM32_IDMALLIEN,
+ host->base + MMCI_STM32_IDMACTRLR);
+
+ return 0;
+}
+
+static void sdmmc_idma_finalize(struct mmci_host *host, struct mmc_data *data)
+{
+ writel_relaxed(0, host->base + MMCI_STM32_IDMACTRLR);
+}
+
+static void mmci_sdmmc_set_clkreg(struct mmci_host *host, unsigned int desired)
+{
+ unsigned int clk = 0, ddr = 0;
+
+ if (host->mmc->ios.timing == MMC_TIMING_MMC_DDR52 ||
+ host->mmc->ios.timing == MMC_TIMING_UHS_DDR50)
+ ddr = MCI_STM32_CLK_DDR;
+
+ /*
+ * cclk = mclk / (2 * clkdiv)
+ * clkdiv 0 => bypass
+ * in ddr mode bypass is not possible
+ */
+ if (desired) {
+ if (desired >= host->mclk && !ddr) {
+ host->cclk = host->mclk;
+ } else {
+ clk = DIV_ROUND_UP(host->mclk, 2 * desired);
+ if (clk > MCI_STM32_CLK_CLKDIV_MSK)
+ clk = MCI_STM32_CLK_CLKDIV_MSK;
+ host->cclk = host->mclk / (2 * clk);
+ }
+ } else {
+ /*
+ * while power-on phase the clock can't be define to 0,
+ * Only power-off and power-cyc deactivate the clock.
+ * if desired clock is 0, set max divider
+ */
+ clk = MCI_STM32_CLK_CLKDIV_MSK;
+ host->cclk = host->mclk / (2 * clk);
+ }
+
+ /* Set actual clock for debug */
+ if (host->mmc->ios.power_mode == MMC_POWER_ON)
+ host->mmc->actual_clock = host->cclk;
+ else
+ host->mmc->actual_clock = 0;
+
+ if (host->mmc->ios.bus_width == MMC_BUS_WIDTH_4)
+ clk |= MCI_STM32_CLK_WIDEBUS_4;
+ if (host->mmc->ios.bus_width == MMC_BUS_WIDTH_8)
+ clk |= MCI_STM32_CLK_WIDEBUS_8;
+
+ clk |= MCI_STM32_CLK_HWFCEN;
+ clk |= host->clk_reg_add;
+ clk |= ddr;
+
+ /*
+ * SDMMC_FBCK is selected when an external Delay Block is needed
+ * with SDR104.
+ */
+ if (host->mmc->ios.timing >= MMC_TIMING_UHS_SDR50) {
+ clk |= MCI_STM32_CLK_BUSSPEED;
+ if (host->mmc->ios.timing == MMC_TIMING_UHS_SDR104) {
+ clk &= ~MCI_STM32_CLK_SEL_MSK;
+ clk |= MCI_STM32_CLK_SELFBCK;
+ }
+ }
+
+ mmci_write_clkreg(host, clk);
+}
+
+static void mmci_sdmmc_set_pwrreg(struct mmci_host *host, unsigned int pwr)
+{
+ struct mmc_ios ios = host->mmc->ios;
+
+ pwr = host->pwr_reg_add;
+
+ if (ios.power_mode == MMC_POWER_OFF) {
+ /* Only a reset could power-off sdmmc */
+ reset_control_assert(host->rst);
+ udelay(2);
+ reset_control_deassert(host->rst);
+
+ /*
+ * Set the SDMMC in Power-cycle state.
+ * This will make that the SDMMC_D[7:0], SDMMC_CMD and SDMMC_CK
+ * are driven low, to prevent the Card from being supplied
+ * through the signal lines.
+ */
+ mmci_write_pwrreg(host, MCI_STM32_PWR_CYC | pwr);
+ } else if (ios.power_mode == MMC_POWER_ON) {
+ /*
+ * After power-off (reset): the irq mask defined in probe
+ * functionis lost
+ * ault irq mask (probe) must be activated
+ */
+ writel(MCI_IRQENABLE | host->variant->start_err,
+ host->base + MMCIMASK0);
+
+ /*
+ * After a power-cycle state, we must set the SDMMC in
+ * Power-off. The SDMMC_D[7:0], SDMMC_CMD and SDMMC_CK are
+ * driven high. Then we can set the SDMMC to Power-on state
+ */
+ mmci_write_pwrreg(host, MCI_PWR_OFF | pwr);
+ mdelay(1);
+ mmci_write_pwrreg(host, MCI_PWR_ON | pwr);
+ }
+}
+
+static struct mmci_host_ops sdmmc_variant_ops = {
+ .validate_data = sdmmc_idma_validate_data,
+ .prep_data = sdmmc_idma_prep_data,
+ .unprep_data = sdmmc_idma_unprep_data,
+ .dma_setup = sdmmc_idma_setup,
+ .dma_start = sdmmc_idma_start,
+ .dma_finalize = sdmmc_idma_finalize,
+ .set_clkreg = mmci_sdmmc_set_clkreg,
+ .set_pwrreg = mmci_sdmmc_set_pwrreg,
+};
+
+void sdmmc_variant_init(struct mmci_host *host)
+{
+ host->ops = &sdmmc_variant_ops;
+}
#define EMMC50_CFG3 0x220
#define SDC_FIFO_CFG 0x228
+/*--------------------------------------------------------------------------*/
+/* Top Pad Register Offset */
+/*--------------------------------------------------------------------------*/
+#define EMMC_TOP_CONTROL 0x00
+#define EMMC_TOP_CMD 0x04
+#define EMMC50_PAD_DS_TUNE 0x0c
+
/*--------------------------------------------------------------------------*/
/* Register Mask */
/*--------------------------------------------------------------------------*/
#define SDC_FIFO_CFG_WRVALIDSEL (0x1 << 24) /* RW */
#define SDC_FIFO_CFG_RDVALIDSEL (0x1 << 25) /* RW */
+/* EMMC_TOP_CONTROL mask */
+#define PAD_RXDLY_SEL (0x1 << 0) /* RW */
+#define DELAY_EN (0x1 << 1) /* RW */
+#define PAD_DAT_RD_RXDLY2 (0x1f << 2) /* RW */
+#define PAD_DAT_RD_RXDLY (0x1f << 7) /* RW */
+#define PAD_DAT_RD_RXDLY2_SEL (0x1 << 12) /* RW */
+#define PAD_DAT_RD_RXDLY_SEL (0x1 << 13) /* RW */
+#define DATA_K_VALUE_SEL (0x1 << 14) /* RW */
+#define SDC_RX_ENH_EN (0x1 << 15) /* TW */
+
+/* EMMC_TOP_CMD mask */
+#define PAD_CMD_RXDLY2 (0x1f << 0) /* RW */
+#define PAD_CMD_RXDLY (0x1f << 5) /* RW */
+#define PAD_CMD_RD_RXDLY2_SEL (0x1 << 10) /* RW */
+#define PAD_CMD_RD_RXDLY_SEL (0x1 << 11) /* RW */
+#define PAD_CMD_TX_DLY (0x1f << 12) /* RW */
+
#define REQ_CMD_EIO (0x1 << 0)
#define REQ_CMD_TMO (0x1 << 1)
#define REQ_DAT_ERR (0x1 << 2)
u32 emmc50_cfg0;
u32 emmc50_cfg3;
u32 sdc_fifo_cfg;
+ u32 emmc_top_control;
+ u32 emmc_top_cmd;
+ u32 emmc50_pad_ds_tune;
};
struct mtk_mmc_compatible {
u32 iocon;
u32 pad_tune;
u32 pad_cmd_tune;
+ u32 emmc_top_control;
+ u32 emmc_top_cmd;
};
struct msdc_delay_phase {
int error;
void __iomem *base; /* host base address */
+ void __iomem *top_base; /* host top register base address */
struct msdc_dma dma; /* dma channel */
u64 dma_mask;
struct clk *src_clk; /* msdc source clock */
struct clk *h_clk; /* msdc h_clk */
+ struct clk *bus_clk; /* bus clock which used to access register */
struct clk *src_clk_cg; /* msdc source clock control gate */
u32 mclk; /* mmc subsystem clock frequency */
u32 src_clk_freq; /* source clock frequency */
- u32 sclk; /* SD/MS bus clock frequency */
unsigned char timing;
bool vqmmc_enabled;
u32 latch_ck;
.support_64g = false,
};
+static const struct mtk_mmc_compatible mt8183_compat = {
+ .clk_div_bits = 12,
+ .hs400_tune = false,
+ .pad_tune_reg = MSDC_PAD_TUNE0,
+ .async_fifo = true,
+ .data_tune = true,
+ .busy_check = true,
+ .stop_clk_fix = true,
+ .enhance_rx = true,
+ .support_64g = true,
+};
+
static const struct mtk_mmc_compatible mt2701_compat = {
.clk_div_bits = 12,
.hs400_tune = false,
static const struct of_device_id msdc_of_ids[] = {
{ .compatible = "mediatek,mt8135-mmc", .data = &mt8135_compat},
{ .compatible = "mediatek,mt8173-mmc", .data = &mt8173_compat},
+ { .compatible = "mediatek,mt8183-mmc", .data = &mt8183_compat},
{ .compatible = "mediatek,mt2701-mmc", .data = &mt2701_compat},
{ .compatible = "mediatek,mt2712-mmc", .data = &mt2712_compat},
{ .compatible = "mediatek,mt7622-mmc", .data = &mt7622_compat},
host->timeout_ns = ns;
host->timeout_clks = clks;
- if (host->sclk == 0) {
+ if (host->mmc->actual_clock == 0) {
timeout = 0;
} else {
- clk_ns = 1000000000UL / host->sclk;
+ clk_ns = 1000000000UL / host->mmc->actual_clock;
timeout = (ns + clk_ns - 1) / clk_ns + clks;
/* in 1048576 sclk cycle unit */
timeout = (timeout + (0x1 << 20) - 1) >> 20;
{
clk_disable_unprepare(host->src_clk_cg);
clk_disable_unprepare(host->src_clk);
+ clk_disable_unprepare(host->bus_clk);
clk_disable_unprepare(host->h_clk);
}
static void msdc_ungate_clock(struct msdc_host *host)
{
clk_prepare_enable(host->h_clk);
+ clk_prepare_enable(host->bus_clk);
clk_prepare_enable(host->src_clk);
clk_prepare_enable(host->src_clk_cg);
while (!(readl(host->base + MSDC_CFG) & MSDC_CFG_CKSTB))
if (!hz) {
dev_dbg(host->dev, "set mclk to 0\n");
host->mclk = 0;
+ host->mmc->actual_clock = 0;
sdr_clr_bits(host->base + MSDC_CFG, MSDC_CFG_CKPDN);
return;
}
while (!(readl(host->base + MSDC_CFG) & MSDC_CFG_CKSTB))
cpu_relax();
sdr_set_bits(host->base + MSDC_CFG, MSDC_CFG_CKPDN);
- host->sclk = sclk;
+ host->mmc->actual_clock = sclk;
host->mclk = hz;
host->timing = timing;
/* need because clk changed. */
* mmc_select_hs400() will drop to 50Mhz and High speed mode,
* tune result of hs200/200Mhz is not suitable for 50Mhz
*/
- if (host->sclk <= 52000000) {
+ if (host->mmc->actual_clock <= 52000000) {
writel(host->def_tune_para.iocon, host->base + MSDC_IOCON);
- writel(host->def_tune_para.pad_tune, host->base + tune_reg);
+ if (host->top_base) {
+ writel(host->def_tune_para.emmc_top_control,
+ host->top_base + EMMC_TOP_CONTROL);
+ writel(host->def_tune_para.emmc_top_cmd,
+ host->top_base + EMMC_TOP_CMD);
+ } else {
+ writel(host->def_tune_para.pad_tune,
+ host->base + tune_reg);
+ }
} else {
writel(host->saved_tune_para.iocon, host->base + MSDC_IOCON);
- writel(host->saved_tune_para.pad_tune, host->base + tune_reg);
writel(host->saved_tune_para.pad_cmd_tune,
host->base + PAD_CMD_TUNE);
+ if (host->top_base) {
+ writel(host->saved_tune_para.emmc_top_control,
+ host->top_base + EMMC_TOP_CONTROL);
+ writel(host->saved_tune_para.emmc_top_cmd,
+ host->top_base + EMMC_TOP_CMD);
+ } else {
+ writel(host->saved_tune_para.pad_tune,
+ host->base + tune_reg);
+ }
}
if (timing == MMC_TIMING_MMC_HS400 &&
sdr_set_field(host->base + PAD_CMD_TUNE,
MSDC_PAD_TUNE_CMDRRDLY,
host->hs400_cmd_int_delay);
- dev_dbg(host->dev, "sclk: %d, timing: %d\n", host->sclk, timing);
+ dev_dbg(host->dev, "sclk: %d, timing: %d\n", host->mmc->actual_clock,
+ timing);
}
static inline u32 msdc_cmd_find_resp(struct msdc_host *host,
WARN_ON(host->cmd);
host->cmd = cmd;
+ mod_delayed_work(system_wq, &host->req_timeout, DAT_TIMEOUT);
if (!msdc_cmd_is_ready(host, mrq, cmd))
return;
cmd->error = 0;
rawcmd = msdc_cmd_prepare_raw_cmd(host, mrq, cmd);
- mod_delayed_work(system_wq, &host->req_timeout, DAT_TIMEOUT);
sdr_set_bits(host->base + MSDC_INTEN, cmd_ints_mask);
writel(cmd->arg, host->base + SDC_ARG);
val = readl(host->base + MSDC_INT);
writel(val, host->base + MSDC_INT);
- writel(0, host->base + tune_reg);
+ if (host->top_base) {
+ writel(0, host->top_base + EMMC_TOP_CONTROL);
+ writel(0, host->top_base + EMMC_TOP_CMD);
+ } else {
+ writel(0, host->base + tune_reg);
+ }
writel(0, host->base + MSDC_IOCON);
sdr_set_field(host->base + MSDC_IOCON, MSDC_IOCON_DDLSEL, 0);
writel(0x403c0046, host->base + MSDC_PATCH_BIT);
sdr_set_field(host->base + MSDC_PATCH_BIT2,
MSDC_PB2_RESPWAIT, 3);
if (host->dev_comp->enhance_rx) {
- sdr_set_bits(host->base + SDC_ADV_CFG0,
- SDC_RX_ENHANCE_EN);
+ if (host->top_base)
+ sdr_set_bits(host->top_base + EMMC_TOP_CONTROL,
+ SDC_RX_ENH_EN);
+ else
+ sdr_set_bits(host->base + SDC_ADV_CFG0,
+ SDC_RX_ENHANCE_EN);
} else {
sdr_set_field(host->base + MSDC_PATCH_BIT2,
MSDC_PB2_RESPSTSENSEL, 2);
sdr_set_bits(host->base + MSDC_PATCH_BIT2,
MSDC_PB2_SUPPORT_64G);
if (host->dev_comp->data_tune) {
- sdr_set_bits(host->base + tune_reg,
- MSDC_PAD_TUNE_RD_SEL | MSDC_PAD_TUNE_CMD_SEL);
+ if (host->top_base) {
+ sdr_set_bits(host->top_base + EMMC_TOP_CONTROL,
+ PAD_DAT_RD_RXDLY_SEL);
+ sdr_clr_bits(host->top_base + EMMC_TOP_CONTROL,
+ DATA_K_VALUE_SEL);
+ sdr_set_bits(host->top_base + EMMC_TOP_CMD,
+ PAD_CMD_RD_RXDLY_SEL);
+ } else {
+ sdr_set_bits(host->base + tune_reg,
+ MSDC_PAD_TUNE_RD_SEL |
+ MSDC_PAD_TUNE_CMD_SEL);
+ }
} else {
/* choose clock tune */
- sdr_set_bits(host->base + tune_reg, MSDC_PAD_TUNE_RXDLYSEL);
+ if (host->top_base)
+ sdr_set_bits(host->top_base + EMMC_TOP_CONTROL,
+ PAD_RXDLY_SEL);
+ else
+ sdr_set_bits(host->base + tune_reg,
+ MSDC_PAD_TUNE_RXDLYSEL);
}
/* Configure to enable SDIO mode.
sdr_set_field(host->base + SDC_CFG, SDC_CFG_DTOC, 3);
host->def_tune_para.iocon = readl(host->base + MSDC_IOCON);
- host->def_tune_para.pad_tune = readl(host->base + tune_reg);
host->saved_tune_para.iocon = readl(host->base + MSDC_IOCON);
- host->saved_tune_para.pad_tune = readl(host->base + tune_reg);
+ if (host->top_base) {
+ host->def_tune_para.emmc_top_control =
+ readl(host->top_base + EMMC_TOP_CONTROL);
+ host->def_tune_para.emmc_top_cmd =
+ readl(host->top_base + EMMC_TOP_CMD);
+ host->saved_tune_para.emmc_top_control =
+ readl(host->top_base + EMMC_TOP_CONTROL);
+ host->saved_tune_para.emmc_top_cmd =
+ readl(host->top_base + EMMC_TOP_CMD);
+ } else {
+ host->def_tune_para.pad_tune = readl(host->base + tune_reg);
+ host->saved_tune_para.pad_tune = readl(host->base + tune_reg);
+ }
dev_dbg(host->dev, "init hardware done!");
}
return delay_phase;
}
+static inline void msdc_set_cmd_delay(struct msdc_host *host, u32 value)
+{
+ u32 tune_reg = host->dev_comp->pad_tune_reg;
+
+ if (host->top_base)
+ sdr_set_field(host->top_base + EMMC_TOP_CMD, PAD_CMD_RXDLY,
+ value);
+ else
+ sdr_set_field(host->base + tune_reg, MSDC_PAD_TUNE_CMDRDLY,
+ value);
+}
+
+static inline void msdc_set_data_delay(struct msdc_host *host, u32 value)
+{
+ u32 tune_reg = host->dev_comp->pad_tune_reg;
+
+ if (host->top_base)
+ sdr_set_field(host->top_base + EMMC_TOP_CONTROL,
+ PAD_DAT_RD_RXDLY, value);
+ else
+ sdr_set_field(host->base + tune_reg, MSDC_PAD_TUNE_DATRRDLY,
+ value);
+}
+
static int msdc_tune_response(struct mmc_host *mmc, u32 opcode)
{
struct msdc_host *host = mmc_priv(mmc);
sdr_clr_bits(host->base + MSDC_IOCON, MSDC_IOCON_RSPL);
for (i = 0 ; i < PAD_DELAY_MAX; i++) {
- sdr_set_field(host->base + tune_reg,
- MSDC_PAD_TUNE_CMDRDLY, i);
+ msdc_set_cmd_delay(host, i);
/*
* Using the same parameters, it may sometimes pass the test,
* but sometimes it may fail. To make sure the parameters are
sdr_set_bits(host->base + MSDC_IOCON, MSDC_IOCON_RSPL);
for (i = 0; i < PAD_DELAY_MAX; i++) {
- sdr_set_field(host->base + tune_reg,
- MSDC_PAD_TUNE_CMDRDLY, i);
+ msdc_set_cmd_delay(host, i);
/*
* Using the same parameters, it may sometimes pass the test,
* but sometimes it may fail. To make sure the parameters are
final_maxlen = final_fall_delay.maxlen;
if (final_maxlen == final_rise_delay.maxlen) {
sdr_clr_bits(host->base + MSDC_IOCON, MSDC_IOCON_RSPL);
- sdr_set_field(host->base + tune_reg, MSDC_PAD_TUNE_CMDRDLY,
- final_rise_delay.final_phase);
final_delay = final_rise_delay.final_phase;
} else {
sdr_set_bits(host->base + MSDC_IOCON, MSDC_IOCON_RSPL);
- sdr_set_field(host->base + tune_reg, MSDC_PAD_TUNE_CMDRDLY,
- final_fall_delay.final_phase);
final_delay = final_fall_delay.final_phase;
}
+ msdc_set_cmd_delay(host, final_delay);
+
if (host->dev_comp->async_fifo || host->hs200_cmd_int_delay)
goto skip_internal;
u32 rise_delay = 0, fall_delay = 0;
struct msdc_delay_phase final_rise_delay, final_fall_delay = { 0,};
u8 final_delay, final_maxlen;
- u32 tune_reg = host->dev_comp->pad_tune_reg;
int i, ret;
sdr_set_field(host->base + MSDC_PATCH_BIT, MSDC_INT_DAT_LATCH_CK_SEL,
sdr_clr_bits(host->base + MSDC_IOCON, MSDC_IOCON_DSPL);
sdr_clr_bits(host->base + MSDC_IOCON, MSDC_IOCON_W_DSPL);
for (i = 0 ; i < PAD_DELAY_MAX; i++) {
- sdr_set_field(host->base + tune_reg,
- MSDC_PAD_TUNE_DATRRDLY, i);
+ msdc_set_data_delay(host, i);
ret = mmc_send_tuning(mmc, opcode, NULL);
if (!ret)
rise_delay |= (1 << i);
sdr_set_bits(host->base + MSDC_IOCON, MSDC_IOCON_DSPL);
sdr_set_bits(host->base + MSDC_IOCON, MSDC_IOCON_W_DSPL);
for (i = 0; i < PAD_DELAY_MAX; i++) {
- sdr_set_field(host->base + tune_reg,
- MSDC_PAD_TUNE_DATRRDLY, i);
+ msdc_set_data_delay(host, i);
ret = mmc_send_tuning(mmc, opcode, NULL);
if (!ret)
fall_delay |= (1 << i);
if (final_maxlen == final_rise_delay.maxlen) {
sdr_clr_bits(host->base + MSDC_IOCON, MSDC_IOCON_DSPL);
sdr_clr_bits(host->base + MSDC_IOCON, MSDC_IOCON_W_DSPL);
- sdr_set_field(host->base + tune_reg,
- MSDC_PAD_TUNE_DATRRDLY,
- final_rise_delay.final_phase);
final_delay = final_rise_delay.final_phase;
} else {
sdr_set_bits(host->base + MSDC_IOCON, MSDC_IOCON_DSPL);
sdr_set_bits(host->base + MSDC_IOCON, MSDC_IOCON_W_DSPL);
- sdr_set_field(host->base + tune_reg,
- MSDC_PAD_TUNE_DATRRDLY,
- final_fall_delay.final_phase);
final_delay = final_fall_delay.final_phase;
}
+ msdc_set_data_delay(host, final_delay);
dev_dbg(host->dev, "Final data pad delay: %x\n", final_delay);
return final_delay == 0xff ? -EIO : 0;
}
+/*
+ * MSDC IP which supports data tune + async fifo can do CMD/DAT tune
+ * together, which can save the tuning time.
+ */
+static int msdc_tune_together(struct mmc_host *mmc, u32 opcode)
+{
+ struct msdc_host *host = mmc_priv(mmc);
+ u32 rise_delay = 0, fall_delay = 0;
+ struct msdc_delay_phase final_rise_delay, final_fall_delay = { 0,};
+ u8 final_delay, final_maxlen;
+ int i, ret;
+
+ sdr_set_field(host->base + MSDC_PATCH_BIT, MSDC_INT_DAT_LATCH_CK_SEL,
+ host->latch_ck);
+
+ sdr_clr_bits(host->base + MSDC_IOCON, MSDC_IOCON_RSPL);
+ sdr_clr_bits(host->base + MSDC_IOCON,
+ MSDC_IOCON_DSPL | MSDC_IOCON_W_DSPL);
+ for (i = 0 ; i < PAD_DELAY_MAX; i++) {
+ msdc_set_cmd_delay(host, i);
+ msdc_set_data_delay(host, i);
+ ret = mmc_send_tuning(mmc, opcode, NULL);
+ if (!ret)
+ rise_delay |= (1 << i);
+ }
+ final_rise_delay = get_best_delay(host, rise_delay);
+ /* if rising edge has enough margin, then do not scan falling edge */
+ if (final_rise_delay.maxlen >= 12 ||
+ (final_rise_delay.start == 0 && final_rise_delay.maxlen >= 4))
+ goto skip_fall;
+
+ sdr_set_bits(host->base + MSDC_IOCON, MSDC_IOCON_RSPL);
+ sdr_set_bits(host->base + MSDC_IOCON,
+ MSDC_IOCON_DSPL | MSDC_IOCON_W_DSPL);
+ for (i = 0; i < PAD_DELAY_MAX; i++) {
+ msdc_set_cmd_delay(host, i);
+ msdc_set_data_delay(host, i);
+ ret = mmc_send_tuning(mmc, opcode, NULL);
+ if (!ret)
+ fall_delay |= (1 << i);
+ }
+ final_fall_delay = get_best_delay(host, fall_delay);
+
+skip_fall:
+ final_maxlen = max(final_rise_delay.maxlen, final_fall_delay.maxlen);
+ if (final_maxlen == final_rise_delay.maxlen) {
+ sdr_clr_bits(host->base + MSDC_IOCON, MSDC_IOCON_RSPL);
+ sdr_clr_bits(host->base + MSDC_IOCON,
+ MSDC_IOCON_DSPL | MSDC_IOCON_W_DSPL);
+ final_delay = final_rise_delay.final_phase;
+ } else {
+ sdr_set_bits(host->base + MSDC_IOCON, MSDC_IOCON_RSPL);
+ sdr_set_bits(host->base + MSDC_IOCON,
+ MSDC_IOCON_DSPL | MSDC_IOCON_W_DSPL);
+ final_delay = final_fall_delay.final_phase;
+ }
+
+ msdc_set_cmd_delay(host, final_delay);
+ msdc_set_data_delay(host, final_delay);
+
+ dev_dbg(host->dev, "Final pad delay: %x\n", final_delay);
+ return final_delay == 0xff ? -EIO : 0;
+}
+
static int msdc_execute_tuning(struct mmc_host *mmc, u32 opcode)
{
struct msdc_host *host = mmc_priv(mmc);
int ret;
u32 tune_reg = host->dev_comp->pad_tune_reg;
+ if (host->dev_comp->data_tune && host->dev_comp->async_fifo) {
+ ret = msdc_tune_together(mmc, opcode);
+ if (host->hs400_mode) {
+ sdr_clr_bits(host->base + MSDC_IOCON,
+ MSDC_IOCON_DSPL | MSDC_IOCON_W_DSPL);
+ msdc_set_data_delay(host, 0);
+ }
+ goto tune_done;
+ }
if (host->hs400_mode &&
host->dev_comp->hs400_tune)
ret = hs400_tune_response(mmc, opcode);
dev_err(host->dev, "Tune data fail!\n");
}
+tune_done:
host->saved_tune_para.iocon = readl(host->base + MSDC_IOCON);
host->saved_tune_para.pad_tune = readl(host->base + tune_reg);
host->saved_tune_para.pad_cmd_tune = readl(host->base + PAD_CMD_TUNE);
+ if (host->top_base) {
+ host->saved_tune_para.emmc_top_control = readl(host->top_base +
+ EMMC_TOP_CONTROL);
+ host->saved_tune_para.emmc_top_cmd = readl(host->top_base +
+ EMMC_TOP_CMD);
+ }
return ret;
}
struct msdc_host *host = mmc_priv(mmc);
host->hs400_mode = true;
- writel(host->hs400_ds_delay, host->base + PAD_DS_TUNE);
+ if (host->top_base)
+ writel(host->hs400_ds_delay,
+ host->top_base + EMMC50_PAD_DS_TUNE);
+ else
+ writel(host->hs400_ds_delay, host->base + PAD_DS_TUNE);
/* hs400 mode must set it to 0 */
sdr_clr_bits(host->base + MSDC_PATCH_BIT2, MSDC_PATCH_BIT2_CFGCRCSTS);
/* to improve read performance, set outstanding to 2 */
goto host_free;
}
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+ host->top_base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(host->top_base))
+ host->top_base = NULL;
+
ret = mmc_regulator_get_supply(mmc);
if (ret)
goto host_free;
goto host_free;
}
+ host->bus_clk = devm_clk_get(&pdev->dev, "bus_clk");
+ if (IS_ERR(host->bus_clk))
+ host->bus_clk = NULL;
/*source clock control gate is optional clock*/
host->src_clk_cg = devm_clk_get(&pdev->dev, "source_cg");
if (IS_ERR(host->src_clk_cg))
host->save_para.msdc_cfg = readl(host->base + MSDC_CFG);
host->save_para.iocon = readl(host->base + MSDC_IOCON);
host->save_para.sdc_cfg = readl(host->base + SDC_CFG);
- host->save_para.pad_tune = readl(host->base + tune_reg);
host->save_para.patch_bit0 = readl(host->base + MSDC_PATCH_BIT);
host->save_para.patch_bit1 = readl(host->base + MSDC_PATCH_BIT1);
host->save_para.patch_bit2 = readl(host->base + MSDC_PATCH_BIT2);
host->save_para.emmc50_cfg0 = readl(host->base + EMMC50_CFG0);
host->save_para.emmc50_cfg3 = readl(host->base + EMMC50_CFG3);
host->save_para.sdc_fifo_cfg = readl(host->base + SDC_FIFO_CFG);
+ if (host->top_base) {
+ host->save_para.emmc_top_control =
+ readl(host->top_base + EMMC_TOP_CONTROL);
+ host->save_para.emmc_top_cmd =
+ readl(host->top_base + EMMC_TOP_CMD);
+ host->save_para.emmc50_pad_ds_tune =
+ readl(host->top_base + EMMC50_PAD_DS_TUNE);
+ } else {
+ host->save_para.pad_tune = readl(host->base + tune_reg);
+ }
}
static void msdc_restore_reg(struct msdc_host *host)
writel(host->save_para.msdc_cfg, host->base + MSDC_CFG);
writel(host->save_para.iocon, host->base + MSDC_IOCON);
writel(host->save_para.sdc_cfg, host->base + SDC_CFG);
- writel(host->save_para.pad_tune, host->base + tune_reg);
writel(host->save_para.patch_bit0, host->base + MSDC_PATCH_BIT);
writel(host->save_para.patch_bit1, host->base + MSDC_PATCH_BIT1);
writel(host->save_para.patch_bit2, host->base + MSDC_PATCH_BIT2);
writel(host->save_para.emmc50_cfg0, host->base + EMMC50_CFG0);
writel(host->save_para.emmc50_cfg3, host->base + EMMC50_CFG3);
writel(host->save_para.sdc_fifo_cfg, host->base + SDC_FIFO_CFG);
+ if (host->top_base) {
+ writel(host->save_para.emmc_top_control,
+ host->top_base + EMMC_TOP_CONTROL);
+ writel(host->save_para.emmc_top_cmd,
+ host->top_base + EMMC_TOP_CMD);
+ writel(host->save_para.emmc50_pad_ds_tune,
+ host->top_base + EMMC50_PAD_DS_TUNE);
+ } else {
+ writel(host->save_para.pad_tune, host->base + tune_reg);
+ }
}
static int msdc_runtime_suspend(struct device *dev)
static irqreturn_t mxcmci_irq(int irq, void *devid)
{
struct mxcmci_host *host = devid;
- unsigned long flags;
bool sdio_irq;
u32 stat;
dev_dbg(mmc_dev(host->mmc), "%s: 0x%08x\n", __func__, stat);
- spin_lock_irqsave(&host->lock, flags);
+ spin_lock(&host->lock);
sdio_irq = (stat & STATUS_SDIO_INT_ACTIVE) && host->use_sdio;
- spin_unlock_irqrestore(&host->lock, flags);
+ spin_unlock(&host->lock);
if (mxcmci_use_dma(host) && (stat & (STATUS_WRITE_OP_DONE)))
mxcmci_writel(host, STATUS_WRITE_OP_DONE, MMC_REG_STATUS);
#include <linux/clk.h>
#include <linux/of.h>
#include <linux/of_irq.h>
-#include <linux/of_gpio.h>
#include <linux/of_device.h>
#include <linux/mmc/host.h>
#include <linux/mmc/core.h>
#include <linux/mmc/slot-gpio.h>
#include <linux/io.h>
#include <linux/irq.h>
-#include <linux/gpio.h>
#include <linux/regulator/consumer.h>
#include <linux/pinctrl/consumer.h>
#include <linux/pm_runtime.h>
struct dma_chan *rx_chan;
int response_busy;
int context_loss;
- int protect_card;
int reqs_blocked;
int req_in_progress;
unsigned long clk_rate;
#define HSMMC_SDIO_IRQ_ENABLED (1 << 1) /* SDIO irq enabled */
struct omap_hsmmc_next next_data;
struct omap_hsmmc_platform_data *pdata;
-
- /* return MMC cover switch state, can be NULL if not supported.
- *
- * possible return values:
- * 0 - closed
- * 1 - open
- */
- int (*get_cover_state)(struct device *dev);
-
- int (*card_detect)(struct device *dev);
};
struct omap_mmc_of_data {
static void omap_hsmmc_start_dma_transfer(struct omap_hsmmc_host *host);
-static int omap_hsmmc_card_detect(struct device *dev)
-{
- struct omap_hsmmc_host *host = dev_get_drvdata(dev);
-
- return mmc_gpio_get_cd(host->mmc);
-}
-
-static int omap_hsmmc_get_cover_state(struct device *dev)
-{
- struct omap_hsmmc_host *host = dev_get_drvdata(dev);
-
- return mmc_gpio_get_cd(host->mmc);
-}
-
static int omap_hsmmc_enable_supply(struct mmc_host *mmc)
{
int ret;
return 0;
}
-static irqreturn_t omap_hsmmc_cover_irq(int irq, void *dev_id);
-
-static int omap_hsmmc_gpio_init(struct mmc_host *mmc,
- struct omap_hsmmc_host *host,
- struct omap_hsmmc_platform_data *pdata)
-{
- int ret;
-
- if (gpio_is_valid(pdata->gpio_cod)) {
- ret = mmc_gpio_request_cd(mmc, pdata->gpio_cod, 0);
- if (ret)
- return ret;
-
- host->get_cover_state = omap_hsmmc_get_cover_state;
- mmc_gpio_set_cd_isr(mmc, omap_hsmmc_cover_irq);
- } else if (gpio_is_valid(pdata->gpio_cd)) {
- ret = mmc_gpio_request_cd(mmc, pdata->gpio_cd, 0);
- if (ret)
- return ret;
-
- host->card_detect = omap_hsmmc_card_detect;
- }
-
- if (gpio_is_valid(pdata->gpio_wp)) {
- ret = mmc_gpio_request_ro(mmc, pdata->gpio_wp);
- if (ret)
- return ret;
- }
-
- return 0;
-}
-
/*
* Start clock to the card
*/
int reg = 0;
unsigned long timeout;
- if (host->protect_card)
- return;
-
disable_irq(host->irq);
OMAP_HSMMC_WRITE(host->base, IE, INT_EN_MASK);
enable_irq(host->irq);
}
-static inline
-int omap_hsmmc_cover_is_closed(struct omap_hsmmc_host *host)
-{
- int r = 1;
-
- if (host->get_cover_state)
- r = host->get_cover_state(host->dev);
- return r;
-}
-
-static ssize_t
-omap_hsmmc_show_cover_switch(struct device *dev, struct device_attribute *attr,
- char *buf)
-{
- struct mmc_host *mmc = container_of(dev, struct mmc_host, class_dev);
- struct omap_hsmmc_host *host = mmc_priv(mmc);
-
- return sprintf(buf, "%s\n",
- omap_hsmmc_cover_is_closed(host) ? "closed" : "open");
-}
-
-static DEVICE_ATTR(cover_switch, S_IRUGO, omap_hsmmc_show_cover_switch, NULL);
-
static ssize_t
omap_hsmmc_show_slot_name(struct device *dev, struct device_attribute *attr,
char *buf)
return ret;
}
-/* Protect the card while the cover is open */
-static void omap_hsmmc_protect_card(struct omap_hsmmc_host *host)
-{
- if (!host->get_cover_state)
- return;
-
- host->reqs_blocked = 0;
- if (host->get_cover_state(host->dev)) {
- if (host->protect_card) {
- dev_info(host->dev, "%s: cover is closed, "
- "card is now accessible\n",
- mmc_hostname(host->mmc));
- host->protect_card = 0;
- }
- } else {
- if (!host->protect_card) {
- dev_info(host->dev, "%s: cover is open, "
- "card is now inaccessible\n",
- mmc_hostname(host->mmc));
- host->protect_card = 1;
- }
- }
-}
-
-/*
- * irq handler when (cell-phone) cover is mounted/removed
- */
-static irqreturn_t omap_hsmmc_cover_irq(int irq, void *dev_id)
-{
- struct omap_hsmmc_host *host = dev_id;
-
- sysfs_notify(&host->mmc->class_dev.kobj, NULL, "cover_switch");
-
- omap_hsmmc_protect_card(host);
- mmc_detect_change(host->mmc, (HZ * 200) / 1000);
- return IRQ_HANDLED;
-}
-
static void omap_hsmmc_dma_callback(void *param)
{
struct omap_hsmmc_host *host = param;
BUG_ON(host->req_in_progress);
BUG_ON(host->dma_ch != -1);
- if (host->protect_card) {
- if (host->reqs_blocked < 3) {
- /*
- * Ensure the controller is left in a consistent
- * state by resetting the command and data state
- * machines.
- */
- omap_hsmmc_reset_controller_fsm(host, SRD);
- omap_hsmmc_reset_controller_fsm(host, SRC);
- host->reqs_blocked += 1;
- }
- req->cmd->error = -EBADF;
- if (req->data)
- req->data->error = -EBADF;
- req->cmd->retries = 0;
- mmc_request_done(mmc, req);
- return;
- } else if (host->reqs_blocked)
+ if (host->reqs_blocked)
host->reqs_blocked = 0;
WARN_ON(host->mrq != NULL);
host->mrq = req;
omap_hsmmc_set_bus_mode(host);
}
-static int omap_hsmmc_get_cd(struct mmc_host *mmc)
-{
- struct omap_hsmmc_host *host = mmc_priv(mmc);
-
- if (!host->card_detect)
- return -ENOSYS;
- return host->card_detect(host->dev);
-}
-
static void omap_hsmmc_init_card(struct mmc_host *mmc, struct mmc_card *card)
{
struct omap_hsmmc_host *host = mmc_priv(mmc);
.pre_req = omap_hsmmc_pre_req,
.request = omap_hsmmc_request,
.set_ios = omap_hsmmc_set_ios,
- .get_cd = omap_hsmmc_get_cd,
+ .get_cd = mmc_gpio_get_cd,
.get_ro = mmc_gpio_get_ro,
.init_card = omap_hsmmc_init_card,
.enable_sdio_irq = omap_hsmmc_enable_sdio_irq,
if (of_find_property(np, "ti,dual-volt", NULL))
pdata->controller_flags |= OMAP_HSMMC_SUPPORTS_DUAL_VOLT;
- pdata->gpio_cd = -EINVAL;
- pdata->gpio_cod = -EINVAL;
- pdata->gpio_wp = -EINVAL;
-
if (of_find_property(np, "ti,non-removable", NULL)) {
pdata->nonremovable = true;
pdata->no_regulator_off_init = true;
host->pbias_enabled = 0;
host->vqmmc_enabled = 0;
- ret = omap_hsmmc_gpio_init(mmc, host, pdata);
- if (ret)
- goto err_gpio;
-
platform_set_drvdata(pdev, host);
if (pdev->dev.of_node)
if (!ret)
mmc->caps |= MMC_CAP_SDIO_IRQ;
- omap_hsmmc_protect_card(host);
-
mmc_add_host(mmc);
if (mmc_pdata(host)->name != NULL) {
if (ret < 0)
goto err_slot_name;
}
- if (host->get_cover_state) {
- ret = device_create_file(&mmc->class_dev,
- &dev_attr_cover_switch);
- if (ret < 0)
- goto err_slot_name;
- }
omap_hsmmc_debugfs(mmc);
pm_runtime_mark_last_busy(host->dev);
if (host->dbclk)
clk_disable_unprepare(host->dbclk);
err1:
-err_gpio:
mmc_free_host(mmc);
err:
return ret;
if (!(host->mmc->pm_flags & MMC_PM_KEEP_POWER))
omap_hsmmc_conf_bus_power(host);
- omap_hsmmc_protect_card(host);
pm_runtime_mark_last_busy(host->dev);
pm_runtime_put_autosuspend(host->dev);
return 0;
+/* SPDX-License-Identifier: GPL-2.0 */
/*
* Renesas Mobile SDHI
*
* Copyright (C) 2017 Horms Solutions Ltd., Simon Horman
* Copyright (C) 2017 Renesas Electronics Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
*/
#ifndef RENESAS_SDHI_H
+// SPDX-License-Identifier: GPL-2.0
/*
* Renesas SDHI
*
* Copyright (C) 2016-17 Horms Solutions, Simon Horman
* Copyright (C) 2009 Magnus Damm
*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
* Based on "Compaq ASIC3 support":
*
* Copyright 2001 Compaq Computer Corporation.
return ret == 0 ? best_freq : clk_get_rate(priv->clk);
}
+static void renesas_sdhi_set_clock(struct tmio_mmc_host *host,
+ unsigned int new_clock)
+{
+ u32 clk = 0, clock;
+
+ sd_ctrl_write16(host, CTL_SD_CARD_CLK_CTL, ~CLK_CTL_SCLKEN &
+ sd_ctrl_read16(host, CTL_SD_CARD_CLK_CTL));
+
+ if (new_clock == 0)
+ goto out;
+
+ /*
+ * Both HS400 and HS200/SD104 set 200MHz, but some devices need to
+ * set 400MHz to distinguish the CPG settings in HS400.
+ */
+ if (host->mmc->ios.timing == MMC_TIMING_MMC_HS400 &&
+ host->pdata->flags & TMIO_MMC_HAVE_4TAP_HS400 &&
+ new_clock == 200000000)
+ new_clock = 400000000;
+
+ clock = renesas_sdhi_clk_update(host, new_clock) / 512;
+
+ for (clk = 0x80000080; new_clock >= (clock << 1); clk >>= 1)
+ clock <<= 1;
+
+ /* 1/1 clock is option */
+ if ((host->pdata->flags & TMIO_MMC_CLK_ACTUAL) && ((clk >> 22) & 0x1)) {
+ if (!(host->mmc->ios.timing == MMC_TIMING_MMC_HS400))
+ clk |= 0xff;
+ else
+ clk &= ~0xff;
+ }
+
+ sd_ctrl_write16(host, CTL_SD_CARD_CLK_CTL, clk & CLK_CTL_DIV_MASK);
+ if (!(host->pdata->flags & TMIO_MMC_MIN_RCAR2))
+ usleep_range(10000, 11000);
+
+ sd_ctrl_write16(host, CTL_SD_CARD_CLK_CTL, CLK_CTL_SCLKEN |
+ sd_ctrl_read16(host, CTL_SD_CARD_CLK_CTL));
+
+out:
+ /* HW engineers overrode docs: no sleep needed on R-Car2+ */
+ if (!(host->pdata->flags & TMIO_MMC_MIN_RCAR2))
+ usleep_range(10000, 11000);
+}
+
static void renesas_sdhi_clk_disable(struct tmio_mmc_host *host)
{
struct renesas_sdhi *priv = host_to_priv(host);
static bool renesas_sdhi_check_scc_error(struct tmio_mmc_host *host)
{
struct renesas_sdhi *priv = host_to_priv(host);
+ bool use_4tap = host->pdata->flags & TMIO_MMC_HAVE_4TAP_HS400;
+
+ /*
+ * Skip checking SCC errors when running on 4 taps in HS400 mode as
+ * any retuning would still result in the same 4 taps being used.
+ */
+ if (!(host->mmc->ios.timing == MMC_TIMING_UHS_SDR104) &&
+ !(host->mmc->ios.timing == MMC_TIMING_MMC_HS200) &&
+ !(host->mmc->ios.timing == MMC_TIMING_MMC_HS400 && !use_4tap))
+ return false;
+
+ if (mmc_doing_retune(host->mmc))
+ return false;
/* Check SCC error */
if (sd_scc_read32(host, priv, SH_MOBILE_SDHI_SCC_RVSCNTL) &
host->write16_hook = renesas_sdhi_write16_hook;
host->clk_enable = renesas_sdhi_clk_enable;
- host->clk_update = renesas_sdhi_clk_update;
host->clk_disable = renesas_sdhi_clk_disable;
+ host->set_clock = renesas_sdhi_set_clock;
host->multi_io_quirk = renesas_sdhi_multi_io_quirk;
host->dma_ops = dma_ops;
+// SPDX-License-Identifier: GPL-2.0
/*
* DMA support for Internal DMAC with SDHI SD/SDIO controller
*
* Copyright (C) 2016-17 Renesas Electronics Corporation
* Copyright (C) 2016-17 Horms Solutions, Simon Horman
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
*/
#include <linux/bitops.h>
/* DM_CM_DTRAN_MODE */
#define DTRAN_MODE_CH_NUM_CH0 0 /* "downstream" = for write commands */
-#define DTRAN_MODE_CH_NUM_CH1 BIT(16) /* "uptream" = for read commands */
-#define DTRAN_MODE_BUS_WID_TH (BIT(5) | BIT(4))
+#define DTRAN_MODE_CH_NUM_CH1 BIT(16) /* "upstream" = for read commands */
+#define DTRAN_MODE_BUS_WIDTH (BIT(5) | BIT(4))
#define DTRAN_MODE_ADDR_MODE BIT(0) /* 1 = Increment address */
/* DM_CM_DTRAN_CTRL */
};
static const struct of_device_id renesas_sdhi_internal_dmac_of_match[] = {
+ { .compatible = "renesas,sdhi-mmc-r8a77470", .data = &of_rcar_gen3_compatible, },
{ .compatible = "renesas,sdhi-r8a7795", .data = &of_rcar_r8a7795_compatible, },
{ .compatible = "renesas,sdhi-r8a7796", .data = &of_rcar_r8a7795_compatible, },
{ .compatible = "renesas,rcar-gen3-sdhi", .data = &of_rcar_gen3_compatible, },
struct mmc_data *data)
{
struct scatterlist *sg = host->sg_ptr;
- u32 dtran_mode = DTRAN_MODE_BUS_WID_TH | DTRAN_MODE_ADDR_MODE;
+ u32 dtran_mode = DTRAN_MODE_BUS_WIDTH | DTRAN_MODE_ADDR_MODE;
if (!dma_map_sg(&host->pdev->dev, sg, host->sg_len,
mmc_get_dma_dir(data)))
renesas_sdhi_internal_dmac_dm_write(host, DM_DTRAN_ADDR,
sg_dma_address(sg));
+ host->dma_on = true;
+
return;
force_pio_with_unmap:
dma_unmap_sg(&host->pdev->dev, sg, host->sg_len, mmc_get_dma_dir(data));
force_pio:
- host->force_pio = true;
renesas_sdhi_internal_dmac_enable_dma(host, false);
}
* Whitelist of specific R-Car Gen3 SoC ES versions to use this DMAC
* implementation as others may use a different implementation.
*/
-static const struct soc_device_attribute gen3_soc_whitelist[] = {
+static const struct soc_device_attribute soc_whitelist[] = {
/* specific ones */
{ .soc_id = "r8a7795", .revision = "ES1.*",
.data = (void *)BIT(SDHI_INTERNAL_DMAC_ONE_RX_ONLY) },
{ .soc_id = "r8a7796", .revision = "ES1.0",
.data = (void *)BIT(SDHI_INTERNAL_DMAC_ONE_RX_ONLY) },
/* generic ones */
+ { .soc_id = "r8a774a1" },
+ { .soc_id = "r8a77470" },
{ .soc_id = "r8a7795" },
{ .soc_id = "r8a7796" },
{ .soc_id = "r8a77965" },
+ { .soc_id = "r8a77970" },
{ .soc_id = "r8a77980" },
{ .soc_id = "r8a77995" },
{ /* sentinel */ }
static int renesas_sdhi_internal_dmac_probe(struct platform_device *pdev)
{
- const struct soc_device_attribute *soc = soc_device_match(gen3_soc_whitelist);
+ const struct soc_device_attribute *soc = soc_device_match(soc_whitelist);
+ struct device *dev = &pdev->dev;
if (!soc)
return -ENODEV;
global_flags |= (unsigned long)soc->data;
+ dev->dma_parms = devm_kzalloc(dev, sizeof(*dev->dma_parms), GFP_KERNEL);
+ if (!dev->dma_parms)
+ return -ENOMEM;
+
+ /* value is max of SD_SECCNT. Confirmed by HW engineers */
+ dma_set_max_seg_size(dev, 0xffffffff);
+
return renesas_sdhi_probe(pdev, &renesas_sdhi_internal_dmac_dma_ops);
}
+// SPDX-License-Identifier: GPL-2.0
/*
* DMA support use of SYS DMAC with SDHI SD/SDIO controller
*
* Copyright (C) 2016-17 Sang Engineering, Wolfram Sang
* Copyright (C) 2017 Horms Solutions, Simon Horman
* Copyright (C) 2010-2011 Guennadi Liakhovetski
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
*/
#include <linux/device.h>
goto pio;
}
- if (sg->length < TMIO_MMC_MIN_DMA_LEN) {
- host->force_pio = true;
+ if (sg->length < TMIO_MMC_MIN_DMA_LEN)
return;
- }
/* The only sg element can be unaligned, use our bounce buffer then */
if (!aligned) {
desc = NULL;
ret = cookie;
}
+ host->dma_on = true;
}
pio:
if (!desc) {
goto pio;
}
- if (sg->length < TMIO_MMC_MIN_DMA_LEN) {
- host->force_pio = true;
+ if (sg->length < TMIO_MMC_MIN_DMA_LEN)
return;
- }
/* The only sg element can be unaligned, use our bounce buffer then */
if (!aligned) {
desc = NULL;
ret = cookie;
}
+ host->dma_on = true;
}
pio:
if (!desc) {
size_t priv_size;
int (*probe_slot)(struct platform_device *, const char *, const char *);
int (*remove_slot)(struct platform_device *);
+ int (*free_slot)(struct platform_device *pdev);
int (*setup_host)(struct platform_device *pdev);
};
static bool sdhci_acpi_byt(void)
{
static const struct x86_cpu_id byt[] = {
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT1 },
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT },
{}
};
.priv_size = sizeof(struct intel_host),
};
+#define VENDOR_SPECIFIC_PWRCTL_CLEAR_REG 0x1a8
+#define VENDOR_SPECIFIC_PWRCTL_CTL_REG 0x1ac
+static irqreturn_t sdhci_acpi_qcom_handler(int irq, void *ptr)
+{
+ struct sdhci_host *host = ptr;
+
+ sdhci_writel(host, 0x3, VENDOR_SPECIFIC_PWRCTL_CLEAR_REG);
+ sdhci_writel(host, 0x1, VENDOR_SPECIFIC_PWRCTL_CTL_REG);
+
+ return IRQ_HANDLED;
+}
+
+static int qcom_probe_slot(struct platform_device *pdev, const char *hid,
+ const char *uid)
+{
+ struct sdhci_acpi_host *c = platform_get_drvdata(pdev);
+ struct sdhci_host *host = c->host;
+ int *irq = sdhci_acpi_priv(c);
+
+ *irq = -EINVAL;
+
+ if (strcmp(hid, "QCOM8051"))
+ return 0;
+
+ *irq = platform_get_irq(pdev, 1);
+ if (*irq < 0)
+ return 0;
+
+ return request_threaded_irq(*irq, NULL, sdhci_acpi_qcom_handler,
+ IRQF_ONESHOT | IRQF_TRIGGER_HIGH,
+ "sdhci_qcom", host);
+}
+
+static int qcom_free_slot(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct sdhci_acpi_host *c = platform_get_drvdata(pdev);
+ struct sdhci_host *host = c->host;
+ struct acpi_device *adev;
+ int *irq = sdhci_acpi_priv(c);
+ const char *hid;
+
+ adev = ACPI_COMPANION(dev);
+ if (!adev)
+ return -ENODEV;
+
+ hid = acpi_device_hid(adev);
+ if (strcmp(hid, "QCOM8051"))
+ return 0;
+
+ if (*irq < 0)
+ return 0;
+
+ free_irq(*irq, host);
+ return 0;
+}
+
static const struct sdhci_acpi_slot sdhci_acpi_slot_qcom_sd_3v = {
.quirks = SDHCI_QUIRK_BROKEN_CARD_DETECTION,
.quirks2 = SDHCI_QUIRK2_NO_1_8_V,
.caps = MMC_CAP_NONREMOVABLE,
+ .priv_size = sizeof(int),
+ .probe_slot = qcom_probe_slot,
+ .free_slot = qcom_free_slot,
};
static const struct sdhci_acpi_slot sdhci_acpi_slot_qcom_sd = {
err_cleanup:
sdhci_cleanup_host(c->host);
err_free:
+ if (c->slot && c->slot->free_slot)
+ c->slot->free_slot(pdev);
+
sdhci_free_host(c->host);
return err;
}
dead = (sdhci_readl(c->host, SDHCI_INT_STATUS) == ~0);
sdhci_remove_host(c->host, dead);
+
+ if (c->slot && c->slot->free_slot)
+ c->slot->free_slot(pdev);
+
sdhci_free_host(c->host);
return 0;
/* Tuning Block Control Register */
#define ESDHC_TBCTL 0x120
#define ESDHC_TB_EN 0x00000004
+#define ESDHC_TBPTR 0x128
/* Control Register for DMA transfer */
#define ESDHC_DMA_SYSCTL 0x40c
* iProc SDHCI platform driver
*/
+#include <linux/acpi.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/mmc/host.h>
sdhci_iproc_writel(host, newval, reg & ~3);
}
+static unsigned int sdhci_iproc_get_max_clock(struct sdhci_host *host)
+{
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+
+ if (pltfm_host->clk)
+ return sdhci_pltfm_clk_get_max_clock(host);
+ else
+ return pltfm_host->clock;
+}
+
static const struct sdhci_ops sdhci_iproc_ops = {
.set_clock = sdhci_set_clock,
- .get_max_clock = sdhci_pltfm_clk_get_max_clock,
+ .get_max_clock = sdhci_iproc_get_max_clock,
.set_bus_width = sdhci_set_bus_width,
.reset = sdhci_reset,
.set_uhs_signaling = sdhci_set_uhs_signaling,
.write_w = sdhci_iproc_writew,
.write_b = sdhci_iproc_writeb,
.set_clock = sdhci_set_clock,
- .get_max_clock = sdhci_pltfm_clk_get_max_clock,
+ .get_max_clock = sdhci_iproc_get_max_clock,
.set_bus_width = sdhci_set_bus_width,
.reset = sdhci_reset,
.set_uhs_signaling = sdhci_set_uhs_signaling,
};
MODULE_DEVICE_TABLE(of, sdhci_iproc_of_match);
+static const struct acpi_device_id sdhci_iproc_acpi_ids[] = {
+ { .id = "BRCM5871", .driver_data = (kernel_ulong_t)&iproc_cygnus_data },
+ { .id = "BRCM5872", .driver_data = (kernel_ulong_t)&iproc_data },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(acpi, sdhci_iproc_acpi_ids);
+
static int sdhci_iproc_probe(struct platform_device *pdev)
{
- const struct of_device_id *match;
- const struct sdhci_iproc_data *iproc_data;
+ struct device *dev = &pdev->dev;
+ const struct sdhci_iproc_data *iproc_data = NULL;
struct sdhci_host *host;
struct sdhci_iproc_host *iproc_host;
struct sdhci_pltfm_host *pltfm_host;
int ret;
- match = of_match_device(sdhci_iproc_of_match, &pdev->dev);
- if (!match)
- return -EINVAL;
- iproc_data = match->data;
+ iproc_data = device_get_match_data(dev);
+ if (!iproc_data)
+ return -ENODEV;
host = sdhci_pltfm_init(pdev, iproc_data->pdata, sizeof(*iproc_host));
if (IS_ERR(host))
iproc_host->data = iproc_data;
mmc_of_parse(host->mmc);
- sdhci_get_of_property(pdev);
+ sdhci_get_property(pdev);
host->mmc->caps |= iproc_host->data->mmc_caps;
- pltfm_host->clk = devm_clk_get(&pdev->dev, NULL);
- if (IS_ERR(pltfm_host->clk)) {
- ret = PTR_ERR(pltfm_host->clk);
- goto err;
- }
- ret = clk_prepare_enable(pltfm_host->clk);
- if (ret) {
- dev_err(&pdev->dev, "failed to enable host clk\n");
- goto err;
+ if (dev->of_node) {
+ pltfm_host->clk = devm_clk_get(dev, NULL);
+ if (IS_ERR(pltfm_host->clk)) {
+ ret = PTR_ERR(pltfm_host->clk);
+ goto err;
+ }
+ ret = clk_prepare_enable(pltfm_host->clk);
+ if (ret) {
+ dev_err(dev, "failed to enable host clk\n");
+ goto err;
+ }
}
if (iproc_host->data->pdata->quirks & SDHCI_QUIRK_MISSING_CAPS) {
return 0;
err_clk:
- clk_disable_unprepare(pltfm_host->clk);
+ if (dev->of_node)
+ clk_disable_unprepare(pltfm_host->clk);
err:
sdhci_pltfm_free(pdev);
return ret;
.driver = {
.name = "sdhci-iproc",
.of_match_table = sdhci_iproc_of_match,
+ .acpi_match_table = ACPI_PTR(sdhci_iproc_acpi_ids),
.pm = &sdhci_pltfm_pmops,
},
.probe = sdhci_iproc_probe,
#define SDHCI_ARASAN_QUIRK_CLOCK_UNSTABLE BIT(1)
};
+struct sdhci_arasan_of_data {
+ const struct sdhci_arasan_soc_ctl_map *soc_ctl_map;
+ const struct sdhci_pltfm_data *pdata;
+};
+
static const struct sdhci_arasan_soc_ctl_map rk3399_soc_ctl_map = {
.baseclkfreq = { .reg = 0xf000, .width = 8, .shift = 8 },
.clockmultiplier = { .reg = 0xf02c, .width = 8, .shift = 0},
}
}
+static void sdhci_arasan_am654_set_clock(struct sdhci_host *host,
+ unsigned int clock)
+{
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct sdhci_arasan_data *sdhci_arasan = sdhci_pltfm_priv(pltfm_host);
+
+ if (sdhci_arasan->is_phy_on) {
+ phy_power_off(sdhci_arasan->phy);
+ sdhci_arasan->is_phy_on = false;
+ }
+
+ sdhci_set_clock(host, clock);
+
+ if (clock > PHY_CLK_TOO_SLOW_HZ) {
+ phy_power_on(sdhci_arasan->phy);
+ sdhci_arasan->is_phy_on = true;
+ }
+}
+
static void sdhci_arasan_hs400_enhanced_strobe(struct mmc_host *mmc,
struct mmc_ios *ios)
{
SDHCI_QUIRK2_STOP_WITH_TC,
};
+static struct sdhci_arasan_of_data sdhci_arasan_data = {
+ .pdata = &sdhci_arasan_pdata,
+};
+
+static const struct sdhci_ops sdhci_arasan_am654_ops = {
+ .set_clock = sdhci_arasan_am654_set_clock,
+ .get_max_clock = sdhci_pltfm_clk_get_max_clock,
+ .get_timeout_clock = sdhci_pltfm_clk_get_max_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_arasan_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
+};
+
+static const struct sdhci_pltfm_data sdhci_arasan_am654_pdata = {
+ .ops = &sdhci_arasan_am654_ops,
+ .quirks = SDHCI_QUIRK_CAP_CLOCK_BASE_BROKEN |
+ SDHCI_QUIRK_INVERTED_WRITE_PROTECT |
+ SDHCI_QUIRK_MULTIBLOCK_READ_ACMD12,
+ .quirks2 = SDHCI_QUIRK2_PRESET_VALUE_BROKEN |
+ SDHCI_QUIRK2_CLOCK_DIV_ZERO_BROKEN |
+ SDHCI_QUIRK2_CAPS_BIT63_FOR_HS400,
+};
+
+static const struct sdhci_arasan_of_data sdhci_arasan_am654_data = {
+ .pdata = &sdhci_arasan_am654_pdata,
+};
+
static u32 sdhci_arasan_cqhci_irq(struct sdhci_host *host, u32 intmask)
{
int cmd_error = 0;
SDHCI_QUIRK2_CLOCK_DIV_ZERO_BROKEN,
};
+static struct sdhci_arasan_of_data sdhci_arasan_rk3399_data = {
+ .soc_ctl_map = &rk3399_soc_ctl_map,
+ .pdata = &sdhci_arasan_cqe_pdata,
+};
+
#ifdef CONFIG_PM_SLEEP
/**
* sdhci_arasan_suspend - Suspend method for the driver
/* SoC-specific compatible strings w/ soc_ctl_map */
{
.compatible = "rockchip,rk3399-sdhci-5.1",
- .data = &rk3399_soc_ctl_map,
+ .data = &sdhci_arasan_rk3399_data,
+ },
+ {
+ .compatible = "ti,am654-sdhci-5.1",
+ .data = &sdhci_arasan_am654_data,
},
-
/* Generic compatible below here */
- { .compatible = "arasan,sdhci-8.9a" },
- { .compatible = "arasan,sdhci-5.1" },
- { .compatible = "arasan,sdhci-4.9a" },
-
+ {
+ .compatible = "arasan,sdhci-8.9a",
+ .data = &sdhci_arasan_data,
+ },
+ {
+ .compatible = "arasan,sdhci-5.1",
+ .data = &sdhci_arasan_data,
+ },
+ {
+ .compatible = "arasan,sdhci-4.9a",
+ .data = &sdhci_arasan_data,
+ },
{ /* sentinel */ }
};
MODULE_DEVICE_TABLE(of, sdhci_arasan_of_match);
struct sdhci_pltfm_host *pltfm_host;
struct sdhci_arasan_data *sdhci_arasan;
struct device_node *np = pdev->dev.of_node;
- const struct sdhci_pltfm_data *pdata;
-
- if (of_device_is_compatible(pdev->dev.of_node, "arasan,sdhci-5.1"))
- pdata = &sdhci_arasan_cqe_pdata;
- else
- pdata = &sdhci_arasan_pdata;
+ const struct sdhci_arasan_of_data *data;
- host = sdhci_pltfm_init(pdev, pdata, sizeof(*sdhci_arasan));
+ match = of_match_node(sdhci_arasan_of_match, pdev->dev.of_node);
+ data = match->data;
+ host = sdhci_pltfm_init(pdev, data->pdata, sizeof(*sdhci_arasan));
if (IS_ERR(host))
return PTR_ERR(host);
sdhci_arasan = sdhci_pltfm_priv(pltfm_host);
sdhci_arasan->host = host;
- match = of_match_node(sdhci_arasan_of_match, pdev->dev.of_node);
- sdhci_arasan->soc_ctl_map = match->data;
+ sdhci_arasan->soc_ctl_map = data->soc_ctl_map;
node = of_parse_phandle(pdev->dev.of_node, "arasan,soc-ctl-syscon", 0);
if (node) {
ret = mmc_of_parse(host->mmc);
if (ret) {
- dev_err(&pdev->dev, "parsing dt failed (%d)\n", ret);
+ if (ret != -EPROBE_DEFER)
+ dev_err(&pdev->dev, "parsing dt failed (%d)\n", ret);
goto unreg_clk;
}
*/
#include <linux/clk.h>
+#include <linux/dma-mapping.h>
+#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/of.h>
+#include <linux/sizes.h>
#include "sdhci-pltfm.h"
+#define BOUNDARY_OK(addr, len) \
+ ((addr | (SZ_128M - 1)) == ((addr + len - 1) | (SZ_128M - 1)))
+
struct dwcmshc_priv {
struct clk *bus_clk;
};
+/*
+ * If DMA addr spans 128MB boundary, we split the DMA transfer into two
+ * so that each DMA transfer doesn't exceed the boundary.
+ */
+static void dwcmshc_adma_write_desc(struct sdhci_host *host, void **desc,
+ dma_addr_t addr, int len, unsigned int cmd)
+{
+ int tmplen, offset;
+
+ if (likely(!len || BOUNDARY_OK(addr, len))) {
+ sdhci_adma_write_desc(host, desc, addr, len, cmd);
+ return;
+ }
+
+ offset = addr & (SZ_128M - 1);
+ tmplen = SZ_128M - offset;
+ sdhci_adma_write_desc(host, desc, addr, tmplen, cmd);
+
+ addr += tmplen;
+ len -= tmplen;
+ sdhci_adma_write_desc(host, desc, addr, len, cmd);
+}
+
static const struct sdhci_ops sdhci_dwcmshc_ops = {
.set_clock = sdhci_set_clock,
.set_bus_width = sdhci_set_bus_width,
.set_uhs_signaling = sdhci_set_uhs_signaling,
.get_max_clock = sdhci_pltfm_clk_get_max_clock,
.reset = sdhci_reset,
+ .adma_write_desc = dwcmshc_adma_write_desc,
};
static const struct sdhci_pltfm_data sdhci_dwcmshc_pdata = {
struct sdhci_host *host;
struct dwcmshc_priv *priv;
int err;
+ u32 extra;
host = sdhci_pltfm_init(pdev, &sdhci_dwcmshc_pdata,
sizeof(struct dwcmshc_priv));
if (IS_ERR(host))
return PTR_ERR(host);
+ /*
+ * extra adma table cnt for cross 128M boundary handling.
+ */
+ extra = DIV_ROUND_UP_ULL(dma_get_required_mask(&pdev->dev), SZ_128M);
+ if (extra > SDHCI_MAX_SEGS)
+ extra = SDHCI_MAX_SEGS;
+ host->adma_table_cnt += extra;
+
pltfm_host = sdhci_priv(host);
priv = sdhci_pltfm_priv(pltfm_host);
u8 vendor_ver;
u8 spec_ver;
bool quirk_incorrect_hostver;
+ bool quirk_fixup_tuning;
unsigned int peripheral_clock;
const struct esdhc_clk_fixup *clk_fixup;
+ u32 div_ratio;
};
/**
dev_dbg(mmc_dev(host->mmc), "desired SD clock: %d, actual: %d\n",
clock, host->max_clk / pre_div / div);
host->mmc->actual_clock = host->max_clk / pre_div / div;
+ esdhc->div_ratio = pre_div * div;
pre_div >>= 1;
div--;
}
}
+static struct soc_device_attribute soc_fixup_tuning[] = {
+ { .family = "QorIQ T1040", .revision = "1.0", },
+ { .family = "QorIQ T2080", .revision = "1.0", },
+ { .family = "QorIQ T1023", .revision = "1.0", },
+ { .family = "QorIQ LS1021A", .revision = "1.0", },
+ { .family = "QorIQ LS1080A", .revision = "1.0", },
+ { .family = "QorIQ LS2080A", .revision = "1.0", },
+ { .family = "QorIQ LS1012A", .revision = "1.0", },
+ { .family = "QorIQ LS1043A", .revision = "1.*", },
+ { .family = "QorIQ LS1046A", .revision = "1.0", },
+ { },
+};
+
static int esdhc_execute_tuning(struct mmc_host *mmc, u32 opcode)
{
struct sdhci_host *host = mmc_priv(mmc);
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct sdhci_esdhc *esdhc = sdhci_pltfm_priv(pltfm_host);
u32 val;
/* Use tuning block for tuning procedure */
sdhci_writel(host, val, ESDHC_TBCTL);
esdhc_clock_enable(host, true);
- return sdhci_execute_tuning(mmc, opcode);
+ sdhci_execute_tuning(mmc, opcode);
+ if (host->tuning_err == -EAGAIN && esdhc->quirk_fixup_tuning) {
+
+ /* program TBPTR[TB_WNDW_END_PTR] = 3*DIV_RATIO and
+ * program TBPTR[TB_WNDW_START_PTR] = 5*DIV_RATIO
+ */
+ val = sdhci_readl(host, ESDHC_TBPTR);
+ val = (val & ~((0x7f << 8) | 0x7f)) |
+ (3 * esdhc->div_ratio) | ((5 * esdhc->div_ratio) << 8);
+ sdhci_writel(host, val, ESDHC_TBPTR);
+
+ /* program the software tuning mode by setting
+ * TBCTL[TB_MODE]=2'h3
+ */
+ val = sdhci_readl(host, ESDHC_TBCTL);
+ val |= 0x3;
+ sdhci_writel(host, val, ESDHC_TBCTL);
+ sdhci_execute_tuning(mmc, opcode);
+ }
+ return 0;
}
#ifdef CONFIG_PM_SLEEP
pltfm_host = sdhci_priv(host);
esdhc = sdhci_pltfm_priv(pltfm_host);
+ if (soc_device_match(soc_fixup_tuning))
+ esdhc->quirk_fixup_tuning = true;
+ else
+ esdhc->quirk_fixup_tuning = false;
+
if (esdhc->vendor_ver == VENDOR_V_22)
host->quirks2 |= SDHCI_QUIRK2_HOST_NO_CMD23;
pci_write_config_byte(chip->pdev, O2_SD_LOCK_WP, scratch);
break;
case PCI_DEVICE_ID_O2_SEABIRD0:
+ if (chip->pdev->revision == 0x01)
+ chip->quirks |= SDHCI_QUIRK_DELAY_AFTER_POWER;
+ /* fall through */
case PCI_DEVICE_ID_O2_SEABIRD1:
/* UnLock WP */
ret = pci_read_config_byte(chip->pdev,
#include <linux/err.h>
#include <linux/module.h>
+#include <linux/property.h>
#include <linux/of.h>
#ifdef CONFIG_PPC
#include <asm/machdep.h>
.set_uhs_signaling = sdhci_set_uhs_signaling,
};
-#ifdef CONFIG_OF
-static bool sdhci_of_wp_inverted(struct device_node *np)
+static bool sdhci_wp_inverted(struct device *dev)
{
- if (of_get_property(np, "sdhci,wp-inverted", NULL) ||
- of_get_property(np, "wp-inverted", NULL))
+ if (device_property_present(dev, "sdhci,wp-inverted") ||
+ device_property_present(dev, "wp-inverted"))
return true;
/* Old device trees don't have the wp-inverted property. */
#endif /* CONFIG_PPC */
}
-void sdhci_get_of_property(struct platform_device *pdev)
+#ifdef CONFIG_OF
+static void sdhci_get_compatibility(struct platform_device *pdev)
{
+ struct sdhci_host *host = platform_get_drvdata(pdev);
struct device_node *np = pdev->dev.of_node;
+
+ if (!np)
+ return;
+
+ if (of_device_is_compatible(np, "fsl,p2020-rev1-esdhc"))
+ host->quirks |= SDHCI_QUIRK_BROKEN_DMA;
+
+ if (of_device_is_compatible(np, "fsl,p2020-esdhc") ||
+ of_device_is_compatible(np, "fsl,p1010-esdhc") ||
+ of_device_is_compatible(np, "fsl,t4240-esdhc") ||
+ of_device_is_compatible(np, "fsl,mpc8536-esdhc"))
+ host->quirks |= SDHCI_QUIRK_BROKEN_TIMEOUT_VAL;
+}
+#else
+void sdhci_get_compatibility(struct platform_device *pdev) {}
+#endif /* CONFIG_OF */
+
+void sdhci_get_property(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
struct sdhci_host *host = platform_get_drvdata(pdev);
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
u32 bus_width;
- if (of_get_property(np, "sdhci,auto-cmd12", NULL))
+ if (device_property_present(dev, "sdhci,auto-cmd12"))
host->quirks |= SDHCI_QUIRK_MULTIBLOCK_READ_ACMD12;
- if (of_get_property(np, "sdhci,1-bit-only", NULL) ||
- (of_property_read_u32(np, "bus-width", &bus_width) == 0 &&
+ if (device_property_present(dev, "sdhci,1-bit-only") ||
+ (device_property_read_u32(dev, "bus-width", &bus_width) == 0 &&
bus_width == 1))
host->quirks |= SDHCI_QUIRK_FORCE_1_BIT_DATA;
- if (sdhci_of_wp_inverted(np))
+ if (sdhci_wp_inverted(dev))
host->quirks |= SDHCI_QUIRK_INVERTED_WRITE_PROTECT;
- if (of_get_property(np, "broken-cd", NULL))
+ if (device_property_present(dev, "broken-cd"))
host->quirks |= SDHCI_QUIRK_BROKEN_CARD_DETECTION;
- if (of_get_property(np, "no-1-8-v", NULL))
+ if (device_property_present(dev, "no-1-8-v"))
host->quirks2 |= SDHCI_QUIRK2_NO_1_8_V;
- if (of_device_is_compatible(np, "fsl,p2020-rev1-esdhc"))
- host->quirks |= SDHCI_QUIRK_BROKEN_DMA;
-
- if (of_device_is_compatible(np, "fsl,p2020-esdhc") ||
- of_device_is_compatible(np, "fsl,p1010-esdhc") ||
- of_device_is_compatible(np, "fsl,t4240-esdhc") ||
- of_device_is_compatible(np, "fsl,mpc8536-esdhc"))
- host->quirks |= SDHCI_QUIRK_BROKEN_TIMEOUT_VAL;
+ sdhci_get_compatibility(pdev);
- of_property_read_u32(np, "clock-frequency", &pltfm_host->clock);
+ device_property_read_u32(dev, "clock-frequency", &pltfm_host->clock);
- if (of_find_property(np, "keep-power-in-suspend", NULL))
+ if (device_property_present(dev, "keep-power-in-suspend"))
host->mmc->pm_caps |= MMC_PM_KEEP_POWER;
- if (of_property_read_bool(np, "wakeup-source") ||
- of_property_read_bool(np, "enable-sdio-wakeup")) /* legacy */
+ if (device_property_read_bool(dev, "wakeup-source") ||
+ device_property_read_bool(dev, "enable-sdio-wakeup")) /* legacy */
host->mmc->pm_caps |= MMC_PM_WAKE_SDIO_IRQ;
}
-#else
-void sdhci_get_of_property(struct platform_device *pdev) {}
-#endif /* CONFIG_OF */
-EXPORT_SYMBOL_GPL(sdhci_get_of_property);
+EXPORT_SYMBOL_GPL(sdhci_get_property);
struct sdhci_host *sdhci_pltfm_init(struct platform_device *pdev,
const struct sdhci_pltfm_data *pdata,
if (IS_ERR(host))
return PTR_ERR(host);
- sdhci_get_of_property(pdev);
+ sdhci_get_property(pdev);
ret = sdhci_add_host(host);
if (ret)
}
#endif /* CONFIG_MMC_SDHCI_BIG_ENDIAN_32BIT_BYTE_SWAPPER */
-extern void sdhci_get_of_property(struct platform_device *pdev);
+void sdhci_get_property(struct platform_device *pdev);
+
+static inline void sdhci_get_of_property(struct platform_device *pdev)
+{
+ return sdhci_get_property(pdev);
+}
extern struct sdhci_host *sdhci_pltfm_init(struct platform_device *pdev,
const struct sdhci_pltfm_data *pdata,
#include <linux/platform_device.h>
#include <linux/clk.h>
#include <linux/io.h>
-#include <linux/gpio.h>
#include <linux/mmc/card.h>
#include <linux/mmc/host.h>
-#include <linux/mmc/slot-gpio.h>
#include <linux/platform_data/pxa_sdhci.h>
#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_device.h>
-#include <linux/of_gpio.h>
#include <linux/pm.h>
#include <linux/pm_runtime.h>
#include <linux/mbus.h>
host->mmc->caps2 |= pdata->host_caps2;
if (pdata->pm_caps)
host->mmc->pm_caps |= pdata->pm_caps;
-
- if (gpio_is_valid(pdata->ext_cd_gpio)) {
- ret = mmc_gpio_request_cd(host->mmc, pdata->ext_cd_gpio,
- 0);
- if (ret) {
- dev_err(mmc_dev(host->mmc),
- "failed to allocate card detect gpio\n");
- goto err_cd_req;
- }
- }
}
pm_runtime_get_noresume(&pdev->dev);
pm_runtime_disable(&pdev->dev);
pm_runtime_put_noidle(&pdev->dev);
err_of_parse:
-err_cd_req:
err_mbus_win:
clk_disable_unprepare(pxa->clk_io);
clk_disable_unprepare(pxa->clk_core);
#include <linux/mmc/host.h>
#include <linux/module.h>
#include <linux/of.h>
-#include <linux/of_gpio.h>
#include <linux/mmc/slot-gpio.h>
#include "sdhci-pltfm.h"
#define SDHCI_SIRF_8BITBUS BIT(3)
#define SIRF_TUNING_COUNT 16384
-struct sdhci_sirf_priv {
- int gpio_cd;
-};
-
static void sdhci_sirf_set_bus_width(struct sdhci_host *host, int width)
{
u8 ctrl;
{
struct sdhci_host *host;
struct sdhci_pltfm_host *pltfm_host;
- struct sdhci_sirf_priv *priv;
struct clk *clk;
- int gpio_cd;
int ret;
clk = devm_clk_get(&pdev->dev, NULL);
return PTR_ERR(clk);
}
- if (pdev->dev.of_node)
- gpio_cd = of_get_named_gpio(pdev->dev.of_node, "cd-gpios", 0);
- else
- gpio_cd = -EINVAL;
-
- host = sdhci_pltfm_init(pdev, &sdhci_sirf_pdata, sizeof(struct sdhci_sirf_priv));
+ host = sdhci_pltfm_init(pdev, &sdhci_sirf_pdata, 0);
if (IS_ERR(host))
return PTR_ERR(host);
pltfm_host = sdhci_priv(host);
pltfm_host->clk = clk;
- priv = sdhci_pltfm_priv(pltfm_host);
- priv->gpio_cd = gpio_cd;
sdhci_get_of_property(pdev);
* We must request the IRQ after sdhci_add_host(), as the tasklet only
* gets setup in sdhci_add_host() and we oops.
*/
- if (gpio_is_valid(priv->gpio_cd)) {
- ret = mmc_gpio_request_cd(host->mmc, priv->gpio_cd, 0);
- if (ret) {
- dev_err(&pdev->dev, "card detect irq request failed: %d\n",
- ret);
- goto err_request_cd;
- }
+ ret = mmc_gpiod_request_cd(host->mmc, "cd", 0, false, 0, NULL);
+ if (ret == -EPROBE_DEFER)
+ goto err_request_cd;
+ if (!ret)
mmc_gpiod_request_cd_irq(host->mmc);
- }
return 0;
#include <linux/clk.h>
#include <linux/delay.h>
-#include <linux/gpio.h>
#include <linux/highmem.h>
#include <linux/module.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/of.h>
-#include <linux/of_gpio.h>
#include <linux/platform_device.h>
#include <linux/pm.h>
#include <linux/slab.h>
struct spear_sdhci {
struct clk *clk;
- int card_int_gpio;
};
/* sdhci ops */
.set_uhs_signaling = sdhci_set_uhs_signaling,
};
-static void sdhci_probe_config_dt(struct device_node *np,
- struct spear_sdhci *host)
-{
- int cd_gpio;
-
- cd_gpio = of_get_named_gpio(np, "cd-gpios", 0);
- if (!gpio_is_valid(cd_gpio))
- cd_gpio = -1;
-
- host->card_int_gpio = cd_gpio;
-}
-
static int sdhci_probe(struct platform_device *pdev)
{
struct sdhci_host *host;
dev_dbg(&pdev->dev, "Error setting desired clk, clk=%lu\n",
clk_get_rate(sdhci->clk));
- sdhci_probe_config_dt(pdev->dev.of_node, sdhci);
/*
- * It is optional to use GPIOs for sdhci card detection. If
- * sdhci->card_int_gpio < 0, then use original sdhci lines otherwise
- * GPIO lines. We use the built-in GPIO support for this.
+ * It is optional to use GPIOs for sdhci card detection. If we
+ * find a descriptor using slot GPIO, we use it.
*/
- if (sdhci->card_int_gpio >= 0) {
- ret = mmc_gpio_request_cd(host->mmc, sdhci->card_int_gpio, 0);
- if (ret < 0) {
- dev_dbg(&pdev->dev,
- "failed to request card-detect gpio%d\n",
- sdhci->card_int_gpio);
- goto disable_clk;
- }
- }
+ ret = mmc_gpiod_request_cd(host->mmc, "cd", 0, false, 0, NULL);
+ if (ret == -EPROBE_DEFER)
+ goto disable_clk;
ret = sdhci_add_host(host);
if (ret)
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+//
+// Secure Digital Host Controller
+//
+// Copyright (C) 2018 Spreadtrum, Inc.
+// Author: Chunyan Zhang <chunyan.zhang@unisoc.com>
+
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/highmem.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/of_gpio.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
+
+#include "sdhci-pltfm.h"
+
+/* SDHCI_ARGUMENT2 register high 16bit */
+#define SDHCI_SPRD_ARG2_STUFF GENMASK(31, 16)
+
+#define SDHCI_SPRD_REG_32_DLL_DLY_OFFSET 0x208
+#define SDHCIBSPRD_IT_WR_DLY_INV BIT(5)
+#define SDHCI_SPRD_BIT_CMD_DLY_INV BIT(13)
+#define SDHCI_SPRD_BIT_POSRD_DLY_INV BIT(21)
+#define SDHCI_SPRD_BIT_NEGRD_DLY_INV BIT(29)
+
+#define SDHCI_SPRD_REG_32_BUSY_POSI 0x250
+#define SDHCI_SPRD_BIT_OUTR_CLK_AUTO_EN BIT(25)
+#define SDHCI_SPRD_BIT_INNR_CLK_AUTO_EN BIT(24)
+
+#define SDHCI_SPRD_REG_DEBOUNCE 0x28C
+#define SDHCI_SPRD_BIT_DLL_BAK BIT(0)
+#define SDHCI_SPRD_BIT_DLL_VAL BIT(1)
+
+#define SDHCI_SPRD_INT_SIGNAL_MASK 0x1B7F410B
+
+/* SDHCI_HOST_CONTROL2 */
+#define SDHCI_SPRD_CTRL_HS200 0x0005
+#define SDHCI_SPRD_CTRL_HS400 0x0006
+
+/*
+ * According to the standard specification, BIT(3) of SDHCI_SOFTWARE_RESET is
+ * reserved, and only used on Spreadtrum's design, the hardware cannot work
+ * if this bit is cleared.
+ * 1 : normal work
+ * 0 : hardware reset
+ */
+#define SDHCI_HW_RESET_CARD BIT(3)
+
+#define SDHCI_SPRD_MAX_CUR 0xFFFFFF
+#define SDHCI_SPRD_CLK_MAX_DIV 1023
+
+#define SDHCI_SPRD_CLK_DEF_RATE 26000000
+
+struct sdhci_sprd_host {
+ u32 version;
+ struct clk *clk_sdio;
+ struct clk *clk_enable;
+ u32 base_rate;
+ int flags; /* backup of host attribute */
+};
+
+#define TO_SPRD_HOST(host) sdhci_pltfm_priv(sdhci_priv(host))
+
+static void sdhci_sprd_init_config(struct sdhci_host *host)
+{
+ u16 val;
+
+ /* set dll backup mode */
+ val = sdhci_readl(host, SDHCI_SPRD_REG_DEBOUNCE);
+ val |= SDHCI_SPRD_BIT_DLL_BAK | SDHCI_SPRD_BIT_DLL_VAL;
+ sdhci_writel(host, val, SDHCI_SPRD_REG_DEBOUNCE);
+}
+
+static inline u32 sdhci_sprd_readl(struct sdhci_host *host, int reg)
+{
+ if (unlikely(reg == SDHCI_MAX_CURRENT))
+ return SDHCI_SPRD_MAX_CUR;
+
+ return readl_relaxed(host->ioaddr + reg);
+}
+
+static inline void sdhci_sprd_writel(struct sdhci_host *host, u32 val, int reg)
+{
+ /* SDHCI_MAX_CURRENT is reserved on Spreadtrum's platform */
+ if (unlikely(reg == SDHCI_MAX_CURRENT))
+ return;
+
+ if (unlikely(reg == SDHCI_SIGNAL_ENABLE || reg == SDHCI_INT_ENABLE))
+ val = val & SDHCI_SPRD_INT_SIGNAL_MASK;
+
+ writel_relaxed(val, host->ioaddr + reg);
+}
+
+static inline void sdhci_sprd_writew(struct sdhci_host *host, u16 val, int reg)
+{
+ /* SDHCI_BLOCK_COUNT is Read Only on Spreadtrum's platform */
+ if (unlikely(reg == SDHCI_BLOCK_COUNT))
+ return;
+
+ writew_relaxed(val, host->ioaddr + reg);
+}
+
+static inline void sdhci_sprd_writeb(struct sdhci_host *host, u8 val, int reg)
+{
+ /*
+ * Since BIT(3) of SDHCI_SOFTWARE_RESET is reserved according to the
+ * standard specification, sdhci_reset() write this register directly
+ * without checking other reserved bits, that will clear BIT(3) which
+ * is defined as hardware reset on Spreadtrum's platform and clearing
+ * it by mistake will lead the card not work. So here we need to work
+ * around it.
+ */
+ if (unlikely(reg == SDHCI_SOFTWARE_RESET)) {
+ if (readb_relaxed(host->ioaddr + reg) & SDHCI_HW_RESET_CARD)
+ val |= SDHCI_HW_RESET_CARD;
+ }
+
+ writeb_relaxed(val, host->ioaddr + reg);
+}
+
+static inline void sdhci_sprd_sd_clk_off(struct sdhci_host *host)
+{
+ u16 ctrl = sdhci_readw(host, SDHCI_CLOCK_CONTROL);
+
+ ctrl &= ~SDHCI_CLOCK_CARD_EN;
+ sdhci_writew(host, ctrl, SDHCI_CLOCK_CONTROL);
+}
+
+static inline void
+sdhci_sprd_set_dll_invert(struct sdhci_host *host, u32 mask, bool en)
+{
+ u32 dll_dly_offset;
+
+ dll_dly_offset = sdhci_readl(host, SDHCI_SPRD_REG_32_DLL_DLY_OFFSET);
+ if (en)
+ dll_dly_offset |= mask;
+ else
+ dll_dly_offset &= ~mask;
+ sdhci_writel(host, dll_dly_offset, SDHCI_SPRD_REG_32_DLL_DLY_OFFSET);
+}
+
+static inline u32 sdhci_sprd_calc_div(u32 base_clk, u32 clk)
+{
+ u32 div;
+
+ /* select 2x clock source */
+ if (base_clk <= clk * 2)
+ return 0;
+
+ div = (u32) (base_clk / (clk * 2));
+
+ if ((base_clk / div) > (clk * 2))
+ div++;
+
+ if (div > SDHCI_SPRD_CLK_MAX_DIV)
+ div = SDHCI_SPRD_CLK_MAX_DIV;
+
+ if (div % 2)
+ div = (div + 1) / 2;
+ else
+ div = div / 2;
+
+ return div;
+}
+
+static inline void _sdhci_sprd_set_clock(struct sdhci_host *host,
+ unsigned int clk)
+{
+ struct sdhci_sprd_host *sprd_host = TO_SPRD_HOST(host);
+ u32 div, val, mask;
+
+ div = sdhci_sprd_calc_div(sprd_host->base_rate, clk);
+
+ clk |= ((div & 0x300) >> 2) | ((div & 0xFF) << 8);
+ sdhci_enable_clk(host, clk);
+
+ /* enable auto gate sdhc_enable_auto_gate */
+ val = sdhci_readl(host, SDHCI_SPRD_REG_32_BUSY_POSI);
+ mask = SDHCI_SPRD_BIT_OUTR_CLK_AUTO_EN |
+ SDHCI_SPRD_BIT_INNR_CLK_AUTO_EN;
+ if (mask != (val & mask)) {
+ val |= mask;
+ sdhci_writel(host, val, SDHCI_SPRD_REG_32_BUSY_POSI);
+ }
+}
+
+static void sdhci_sprd_set_clock(struct sdhci_host *host, unsigned int clock)
+{
+ bool en = false;
+
+ if (clock == 0) {
+ sdhci_writew(host, 0, SDHCI_CLOCK_CONTROL);
+ } else if (clock != host->clock) {
+ sdhci_sprd_sd_clk_off(host);
+ _sdhci_sprd_set_clock(host, clock);
+
+ if (clock <= 400000)
+ en = true;
+ sdhci_sprd_set_dll_invert(host, SDHCI_SPRD_BIT_CMD_DLY_INV |
+ SDHCI_SPRD_BIT_POSRD_DLY_INV, en);
+ } else {
+ _sdhci_sprd_set_clock(host, clock);
+ }
+}
+
+static unsigned int sdhci_sprd_get_max_clock(struct sdhci_host *host)
+{
+ struct sdhci_sprd_host *sprd_host = TO_SPRD_HOST(host);
+
+ return clk_round_rate(sprd_host->clk_sdio, ULONG_MAX);
+}
+
+static unsigned int sdhci_sprd_get_min_clock(struct sdhci_host *host)
+{
+ return 400000;
+}
+
+static void sdhci_sprd_set_uhs_signaling(struct sdhci_host *host,
+ unsigned int timing)
+{
+ u16 ctrl_2;
+
+ if (timing == host->timing)
+ return;
+
+ ctrl_2 = sdhci_readw(host, SDHCI_HOST_CONTROL2);
+ /* Select Bus Speed Mode for host */
+ ctrl_2 &= ~SDHCI_CTRL_UHS_MASK;
+ switch (timing) {
+ case MMC_TIMING_UHS_SDR12:
+ ctrl_2 |= SDHCI_CTRL_UHS_SDR12;
+ break;
+ case MMC_TIMING_MMC_HS:
+ case MMC_TIMING_SD_HS:
+ case MMC_TIMING_UHS_SDR25:
+ ctrl_2 |= SDHCI_CTRL_UHS_SDR25;
+ break;
+ case MMC_TIMING_UHS_SDR50:
+ ctrl_2 |= SDHCI_CTRL_UHS_SDR50;
+ break;
+ case MMC_TIMING_UHS_SDR104:
+ ctrl_2 |= SDHCI_CTRL_UHS_SDR104;
+ break;
+ case MMC_TIMING_UHS_DDR50:
+ case MMC_TIMING_MMC_DDR52:
+ ctrl_2 |= SDHCI_CTRL_UHS_DDR50;
+ break;
+ case MMC_TIMING_MMC_HS200:
+ ctrl_2 |= SDHCI_SPRD_CTRL_HS200;
+ break;
+ case MMC_TIMING_MMC_HS400:
+ ctrl_2 |= SDHCI_SPRD_CTRL_HS400;
+ break;
+ default:
+ break;
+ }
+
+ sdhci_writew(host, ctrl_2, SDHCI_HOST_CONTROL2);
+}
+
+static void sdhci_sprd_hw_reset(struct sdhci_host *host)
+{
+ int val;
+
+ /*
+ * Note: don't use sdhci_writeb() API here since it is redirected to
+ * sdhci_sprd_writeb() in which we have a workaround for
+ * SDHCI_SOFTWARE_RESET which would make bit SDHCI_HW_RESET_CARD can
+ * not be cleared.
+ */
+ val = readb_relaxed(host->ioaddr + SDHCI_SOFTWARE_RESET);
+ val &= ~SDHCI_HW_RESET_CARD;
+ writeb_relaxed(val, host->ioaddr + SDHCI_SOFTWARE_RESET);
+ /* wait for 10 us */
+ usleep_range(10, 20);
+
+ val |= SDHCI_HW_RESET_CARD;
+ writeb_relaxed(val, host->ioaddr + SDHCI_SOFTWARE_RESET);
+ usleep_range(300, 500);
+}
+
+static struct sdhci_ops sdhci_sprd_ops = {
+ .read_l = sdhci_sprd_readl,
+ .write_l = sdhci_sprd_writel,
+ .write_b = sdhci_sprd_writeb,
+ .set_clock = sdhci_sprd_set_clock,
+ .get_max_clock = sdhci_sprd_get_max_clock,
+ .get_min_clock = sdhci_sprd_get_min_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_sprd_set_uhs_signaling,
+ .hw_reset = sdhci_sprd_hw_reset,
+};
+
+static void sdhci_sprd_request(struct mmc_host *mmc, struct mmc_request *mrq)
+{
+ struct sdhci_host *host = mmc_priv(mmc);
+ struct sdhci_sprd_host *sprd_host = TO_SPRD_HOST(host);
+
+ host->flags |= sprd_host->flags & SDHCI_AUTO_CMD23;
+
+ /*
+ * From version 4.10 onward, ARGUMENT2 register is also as 32-bit
+ * block count register which doesn't support stuff bits of
+ * CMD23 argument on Spreadtrum's sd host controller.
+ */
+ if (host->version >= SDHCI_SPEC_410 &&
+ mrq->sbc && (mrq->sbc->arg & SDHCI_SPRD_ARG2_STUFF) &&
+ (host->flags & SDHCI_AUTO_CMD23))
+ host->flags &= ~SDHCI_AUTO_CMD23;
+
+ sdhci_request(mmc, mrq);
+}
+
+static const struct sdhci_pltfm_data sdhci_sprd_pdata = {
+ .quirks = SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK,
+ .quirks2 = SDHCI_QUIRK2_BROKEN_HS200 |
+ SDHCI_QUIRK2_USE_32BIT_BLK_CNT,
+ .ops = &sdhci_sprd_ops,
+};
+
+static int sdhci_sprd_probe(struct platform_device *pdev)
+{
+ struct sdhci_host *host;
+ struct sdhci_sprd_host *sprd_host;
+ struct clk *clk;
+ int ret = 0;
+
+ host = sdhci_pltfm_init(pdev, &sdhci_sprd_pdata, sizeof(*sprd_host));
+ if (IS_ERR(host))
+ return PTR_ERR(host);
+
+ host->dma_mask = DMA_BIT_MASK(64);
+ pdev->dev.dma_mask = &host->dma_mask;
+ host->mmc_host_ops.request = sdhci_sprd_request;
+
+ host->mmc->caps = MMC_CAP_SD_HIGHSPEED | MMC_CAP_MMC_HIGHSPEED |
+ MMC_CAP_ERASE | MMC_CAP_CMD23;
+ ret = mmc_of_parse(host->mmc);
+ if (ret)
+ goto pltfm_free;
+
+ sprd_host = TO_SPRD_HOST(host);
+
+ clk = devm_clk_get(&pdev->dev, "sdio");
+ if (IS_ERR(clk)) {
+ ret = PTR_ERR(clk);
+ goto pltfm_free;
+ }
+ sprd_host->clk_sdio = clk;
+ sprd_host->base_rate = clk_get_rate(sprd_host->clk_sdio);
+ if (!sprd_host->base_rate)
+ sprd_host->base_rate = SDHCI_SPRD_CLK_DEF_RATE;
+
+ clk = devm_clk_get(&pdev->dev, "enable");
+ if (IS_ERR(clk)) {
+ ret = PTR_ERR(clk);
+ goto pltfm_free;
+ }
+ sprd_host->clk_enable = clk;
+
+ ret = clk_prepare_enable(sprd_host->clk_sdio);
+ if (ret)
+ goto pltfm_free;
+
+ clk_prepare_enable(sprd_host->clk_enable);
+ if (ret)
+ goto clk_disable;
+
+ sdhci_sprd_init_config(host);
+ host->version = sdhci_readw(host, SDHCI_HOST_VERSION);
+ sprd_host->version = ((host->version & SDHCI_VENDOR_VER_MASK) >>
+ SDHCI_VENDOR_VER_SHIFT);
+
+ pm_runtime_get_noresume(&pdev->dev);
+ pm_runtime_set_active(&pdev->dev);
+ pm_runtime_enable(&pdev->dev);
+ pm_runtime_set_autosuspend_delay(&pdev->dev, 50);
+ pm_runtime_use_autosuspend(&pdev->dev);
+ pm_suspend_ignore_children(&pdev->dev, 1);
+
+ sdhci_enable_v4_mode(host);
+
+ ret = sdhci_setup_host(host);
+ if (ret)
+ goto pm_runtime_disable;
+
+ sprd_host->flags = host->flags;
+
+ ret = __sdhci_add_host(host);
+ if (ret)
+ goto err_cleanup_host;
+
+ pm_runtime_mark_last_busy(&pdev->dev);
+ pm_runtime_put_autosuspend(&pdev->dev);
+
+ return 0;
+
+err_cleanup_host:
+ sdhci_cleanup_host(host);
+
+pm_runtime_disable:
+ pm_runtime_disable(&pdev->dev);
+ pm_runtime_set_suspended(&pdev->dev);
+
+ clk_disable_unprepare(sprd_host->clk_enable);
+
+clk_disable:
+ clk_disable_unprepare(sprd_host->clk_sdio);
+
+pltfm_free:
+ sdhci_pltfm_free(pdev);
+ return ret;
+}
+
+static int sdhci_sprd_remove(struct platform_device *pdev)
+{
+ struct sdhci_host *host = platform_get_drvdata(pdev);
+ struct sdhci_sprd_host *sprd_host = TO_SPRD_HOST(host);
+ struct mmc_host *mmc = host->mmc;
+
+ mmc_remove_host(mmc);
+ clk_disable_unprepare(sprd_host->clk_sdio);
+ clk_disable_unprepare(sprd_host->clk_enable);
+
+ mmc_free_host(mmc);
+
+ return 0;
+}
+
+static const struct of_device_id sdhci_sprd_of_match[] = {
+ { .compatible = "sprd,sdhci-r11", },
+ { }
+};
+MODULE_DEVICE_TABLE(of, sdhci_sprd_of_match);
+
+#ifdef CONFIG_PM
+static int sdhci_sprd_runtime_suspend(struct device *dev)
+{
+ struct sdhci_host *host = dev_get_drvdata(dev);
+ struct sdhci_sprd_host *sprd_host = TO_SPRD_HOST(host);
+
+ sdhci_runtime_suspend_host(host);
+
+ clk_disable_unprepare(sprd_host->clk_sdio);
+ clk_disable_unprepare(sprd_host->clk_enable);
+
+ return 0;
+}
+
+static int sdhci_sprd_runtime_resume(struct device *dev)
+{
+ struct sdhci_host *host = dev_get_drvdata(dev);
+ struct sdhci_sprd_host *sprd_host = TO_SPRD_HOST(host);
+ int ret;
+
+ ret = clk_prepare_enable(sprd_host->clk_enable);
+ if (ret)
+ return ret;
+
+ ret = clk_prepare_enable(sprd_host->clk_sdio);
+ if (ret) {
+ clk_disable_unprepare(sprd_host->clk_enable);
+ return ret;
+ }
+
+ sdhci_runtime_resume_host(host);
+
+ return 0;
+}
+#endif
+
+static const struct dev_pm_ops sdhci_sprd_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+ SET_RUNTIME_PM_OPS(sdhci_sprd_runtime_suspend,
+ sdhci_sprd_runtime_resume, NULL)
+};
+
+static struct platform_driver sdhci_sprd_driver = {
+ .probe = sdhci_sprd_probe,
+ .remove = sdhci_sprd_remove,
+ .driver = {
+ .name = "sdhci_sprd_r11",
+ .of_match_table = of_match_ptr(sdhci_sprd_of_match),
+ .pm = &sdhci_sprd_pm_ops,
+ },
+};
+module_platform_driver(sdhci_sprd_driver);
+
+MODULE_DESCRIPTION("Spreadtrum sdio host controller r11 driver");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:sdhci-sprd-r11");
#include <linux/err.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/iopoll.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/of.h>
#include <linux/of_device.h>
+#include <linux/pinctrl/consumer.h>
+#include <linux/regulator/consumer.h>
#include <linux/reset.h>
#include <linux/mmc/card.h>
#include <linux/mmc/host.h>
#include <linux/mmc/mmc.h>
#include <linux/mmc/slot-gpio.h>
#include <linux/gpio/consumer.h>
+#include <linux/ktime.h>
#include "sdhci-pltfm.h"
#define SDHCI_TEGRA_VENDOR_CLOCK_CTRL 0x100
#define SDHCI_CLOCK_CTRL_TAP_MASK 0x00ff0000
#define SDHCI_CLOCK_CTRL_TAP_SHIFT 16
+#define SDHCI_CLOCK_CTRL_TRIM_MASK 0x1f000000
+#define SDHCI_CLOCK_CTRL_TRIM_SHIFT 24
#define SDHCI_CLOCK_CTRL_SDR50_TUNING_OVERRIDE BIT(5)
#define SDHCI_CLOCK_CTRL_PADPIPE_CLKEN_OVERRIDE BIT(3)
#define SDHCI_CLOCK_CTRL_SPI_MODE_CLKEN_OVERRIDE BIT(2)
-#define SDHCI_TEGRA_VENDOR_MISC_CTRL 0x120
-#define SDHCI_MISC_CTRL_ENABLE_SDR104 0x8
-#define SDHCI_MISC_CTRL_ENABLE_SDR50 0x10
-#define SDHCI_MISC_CTRL_ENABLE_SDHCI_SPEC_300 0x20
-#define SDHCI_MISC_CTRL_ENABLE_DDR50 0x200
+#define SDHCI_TEGRA_VENDOR_SYS_SW_CTRL 0x104
+#define SDHCI_TEGRA_SYS_SW_CTRL_ENHANCED_STROBE BIT(31)
-#define SDHCI_TEGRA_AUTO_CAL_CONFIG 0x1e4
-#define SDHCI_AUTO_CAL_START BIT(31)
-#define SDHCI_AUTO_CAL_ENABLE BIT(29)
+#define SDHCI_TEGRA_VENDOR_CAP_OVERRIDES 0x10c
+#define SDHCI_TEGRA_CAP_OVERRIDES_DQS_TRIM_MASK 0x00003f00
+#define SDHCI_TEGRA_CAP_OVERRIDES_DQS_TRIM_SHIFT 8
-#define NVQUIRK_FORCE_SDHCI_SPEC_200 BIT(0)
-#define NVQUIRK_ENABLE_BLOCK_GAP_DET BIT(1)
-#define NVQUIRK_ENABLE_SDHCI_SPEC_300 BIT(2)
-#define NVQUIRK_ENABLE_SDR50 BIT(3)
-#define NVQUIRK_ENABLE_SDR104 BIT(4)
-#define NVQUIRK_ENABLE_DDR50 BIT(5)
-#define NVQUIRK_HAS_PADCALIB BIT(6)
+#define SDHCI_TEGRA_VENDOR_MISC_CTRL 0x120
+#define SDHCI_MISC_CTRL_ENABLE_SDR104 0x8
+#define SDHCI_MISC_CTRL_ENABLE_SDR50 0x10
+#define SDHCI_MISC_CTRL_ENABLE_SDHCI_SPEC_300 0x20
+#define SDHCI_MISC_CTRL_ENABLE_DDR50 0x200
+
+#define SDHCI_TEGRA_VENDOR_DLLCAL_CFG 0x1b0
+#define SDHCI_TEGRA_DLLCAL_CALIBRATE BIT(31)
+
+#define SDHCI_TEGRA_VENDOR_DLLCAL_STA 0x1bc
+#define SDHCI_TEGRA_DLLCAL_STA_ACTIVE BIT(31)
+
+#define SDHCI_VNDR_TUN_CTRL0_0 0x1c0
+#define SDHCI_VNDR_TUN_CTRL0_TUN_HW_TAP 0x20000
+
+#define SDHCI_TEGRA_AUTO_CAL_CONFIG 0x1e4
+#define SDHCI_AUTO_CAL_START BIT(31)
+#define SDHCI_AUTO_CAL_ENABLE BIT(29)
+#define SDHCI_AUTO_CAL_PDPU_OFFSET_MASK 0x0000ffff
+
+#define SDHCI_TEGRA_SDMEM_COMP_PADCTRL 0x1e0
+#define SDHCI_TEGRA_SDMEM_COMP_PADCTRL_VREF_SEL_MASK 0x0000000f
+#define SDHCI_TEGRA_SDMEM_COMP_PADCTRL_VREF_SEL_VAL 0x7
+#define SDHCI_TEGRA_SDMEM_COMP_PADCTRL_E_INPUT_E_PWRD BIT(31)
+
+#define SDHCI_TEGRA_AUTO_CAL_STATUS 0x1ec
+#define SDHCI_TEGRA_AUTO_CAL_ACTIVE BIT(31)
+
+#define NVQUIRK_FORCE_SDHCI_SPEC_200 BIT(0)
+#define NVQUIRK_ENABLE_BLOCK_GAP_DET BIT(1)
+#define NVQUIRK_ENABLE_SDHCI_SPEC_300 BIT(2)
+#define NVQUIRK_ENABLE_SDR50 BIT(3)
+#define NVQUIRK_ENABLE_SDR104 BIT(4)
+#define NVQUIRK_ENABLE_DDR50 BIT(5)
+#define NVQUIRK_HAS_PADCALIB BIT(6)
+#define NVQUIRK_NEEDS_PAD_CONTROL BIT(7)
+#define NVQUIRK_DIS_CARD_CLK_CONFIG_TAP BIT(8)
struct sdhci_tegra_soc_data {
const struct sdhci_pltfm_data *pdata;
u32 nvquirks;
};
+/* Magic pull up and pull down pad calibration offsets */
+struct sdhci_tegra_autocal_offsets {
+ u32 pull_up_3v3;
+ u32 pull_down_3v3;
+ u32 pull_up_3v3_timeout;
+ u32 pull_down_3v3_timeout;
+ u32 pull_up_1v8;
+ u32 pull_down_1v8;
+ u32 pull_up_1v8_timeout;
+ u32 pull_down_1v8_timeout;
+ u32 pull_up_sdr104;
+ u32 pull_down_sdr104;
+ u32 pull_up_hs400;
+ u32 pull_down_hs400;
+};
+
struct sdhci_tegra {
const struct sdhci_tegra_soc_data *soc_data;
struct gpio_desc *power_gpio;
bool ddr_signaling;
bool pad_calib_required;
+ bool pad_control_available;
struct reset_control *rst;
+ struct pinctrl *pinctrl_sdmmc;
+ struct pinctrl_state *pinctrl_state_3v3;
+ struct pinctrl_state *pinctrl_state_1v8;
+
+ struct sdhci_tegra_autocal_offsets autocal_offsets;
+ ktime_t last_calib;
+
+ u32 default_tap;
+ u32 default_trim;
+ u32 dqs_trim;
};
static u16 tegra_sdhci_readw(struct sdhci_host *host, int reg)
}
}
+static bool tegra_sdhci_configure_card_clk(struct sdhci_host *host, bool enable)
+{
+ bool status;
+ u32 reg;
+
+ reg = sdhci_readw(host, SDHCI_CLOCK_CONTROL);
+ status = !!(reg & SDHCI_CLOCK_CARD_EN);
+
+ if (status == enable)
+ return status;
+
+ if (enable)
+ reg |= SDHCI_CLOCK_CARD_EN;
+ else
+ reg &= ~SDHCI_CLOCK_CARD_EN;
+
+ sdhci_writew(host, reg, SDHCI_CLOCK_CONTROL);
+
+ return status;
+}
+
+static void tegra210_sdhci_writew(struct sdhci_host *host, u16 val, int reg)
+{
+ bool is_tuning_cmd = 0;
+ bool clk_enabled;
+ u8 cmd;
+
+ if (reg == SDHCI_COMMAND) {
+ cmd = SDHCI_GET_CMD(val);
+ is_tuning_cmd = cmd == MMC_SEND_TUNING_BLOCK ||
+ cmd == MMC_SEND_TUNING_BLOCK_HS200;
+ }
+
+ if (is_tuning_cmd)
+ clk_enabled = tegra_sdhci_configure_card_clk(host, 0);
+
+ writew(val, host->ioaddr + reg);
+
+ if (is_tuning_cmd) {
+ udelay(1);
+ tegra_sdhci_configure_card_clk(host, clk_enabled);
+ }
+}
+
static unsigned int tegra_sdhci_get_ro(struct sdhci_host *host)
{
return mmc_gpio_get_ro(host->mmc);
}
+static bool tegra_sdhci_is_pad_and_regulator_valid(struct sdhci_host *host)
+{
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct sdhci_tegra *tegra_host = sdhci_pltfm_priv(pltfm_host);
+ int has_1v8, has_3v3;
+
+ /*
+ * The SoCs which have NVQUIRK_NEEDS_PAD_CONTROL require software pad
+ * voltage configuration in order to perform voltage switching. This
+ * means that valid pinctrl info is required on SDHCI instances capable
+ * of performing voltage switching. Whether or not an SDHCI instance is
+ * capable of voltage switching is determined based on the regulator.
+ */
+
+ if (!(tegra_host->soc_data->nvquirks & NVQUIRK_NEEDS_PAD_CONTROL))
+ return true;
+
+ if (IS_ERR(host->mmc->supply.vqmmc))
+ return false;
+
+ has_1v8 = regulator_is_supported_voltage(host->mmc->supply.vqmmc,
+ 1700000, 1950000);
+
+ has_3v3 = regulator_is_supported_voltage(host->mmc->supply.vqmmc,
+ 2700000, 3600000);
+
+ if (has_1v8 == 1 && has_3v3 == 1)
+ return tegra_host->pad_control_available;
+
+ /* Fixed voltage, no pad control required. */
+ return true;
+}
+
+static void tegra_sdhci_set_tap(struct sdhci_host *host, unsigned int tap)
+{
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct sdhci_tegra *tegra_host = sdhci_pltfm_priv(pltfm_host);
+ const struct sdhci_tegra_soc_data *soc_data = tegra_host->soc_data;
+ bool card_clk_enabled = false;
+ u32 reg;
+
+ /*
+ * Touching the tap values is a bit tricky on some SoC generations.
+ * The quirk enables a workaround for a glitch that sometimes occurs if
+ * the tap values are changed.
+ */
+
+ if (soc_data->nvquirks & NVQUIRK_DIS_CARD_CLK_CONFIG_TAP)
+ card_clk_enabled = tegra_sdhci_configure_card_clk(host, false);
+
+ reg = sdhci_readl(host, SDHCI_TEGRA_VENDOR_CLOCK_CTRL);
+ reg &= ~SDHCI_CLOCK_CTRL_TAP_MASK;
+ reg |= tap << SDHCI_CLOCK_CTRL_TAP_SHIFT;
+ sdhci_writel(host, reg, SDHCI_TEGRA_VENDOR_CLOCK_CTRL);
+
+ if (soc_data->nvquirks & NVQUIRK_DIS_CARD_CLK_CONFIG_TAP &&
+ card_clk_enabled) {
+ udelay(1);
+ sdhci_reset(host, SDHCI_RESET_CMD | SDHCI_RESET_DATA);
+ tegra_sdhci_configure_card_clk(host, card_clk_enabled);
+ }
+}
+
+static void tegra_sdhci_hs400_enhanced_strobe(struct mmc_host *mmc,
+ struct mmc_ios *ios)
+{
+ struct sdhci_host *host = mmc_priv(mmc);
+ u32 val;
+
+ val = sdhci_readl(host, SDHCI_TEGRA_VENDOR_SYS_SW_CTRL);
+
+ if (ios->enhanced_strobe)
+ val |= SDHCI_TEGRA_SYS_SW_CTRL_ENHANCED_STROBE;
+ else
+ val &= ~SDHCI_TEGRA_SYS_SW_CTRL_ENHANCED_STROBE;
+
+ sdhci_writel(host, val, SDHCI_TEGRA_VENDOR_SYS_SW_CTRL);
+
+}
+
static void tegra_sdhci_reset(struct sdhci_host *host, u8 mask)
{
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
struct sdhci_tegra *tegra_host = sdhci_pltfm_priv(pltfm_host);
const struct sdhci_tegra_soc_data *soc_data = tegra_host->soc_data;
- u32 misc_ctrl, clk_ctrl;
+ u32 misc_ctrl, clk_ctrl, pad_ctrl;
sdhci_reset(host, mask);
if (!(mask & SDHCI_RESET_ALL))
return;
+ tegra_sdhci_set_tap(host, tegra_host->default_tap);
+
misc_ctrl = sdhci_readl(host, SDHCI_TEGRA_VENDOR_MISC_CTRL);
clk_ctrl = sdhci_readl(host, SDHCI_TEGRA_VENDOR_CLOCK_CTRL);
SDHCI_MISC_CTRL_ENABLE_DDR50 |
SDHCI_MISC_CTRL_ENABLE_SDR104);
- clk_ctrl &= ~SDHCI_CLOCK_CTRL_SPI_MODE_CLKEN_OVERRIDE;
+ clk_ctrl &= ~(SDHCI_CLOCK_CTRL_TRIM_MASK |
+ SDHCI_CLOCK_CTRL_SPI_MODE_CLKEN_OVERRIDE);
- /*
- * If the board does not define a regulator for the SDHCI
- * IO voltage, then don't advertise support for UHS modes
- * even if the device supports it because the IO voltage
- * cannot be configured.
- */
- if (!IS_ERR(host->mmc->supply.vqmmc)) {
+ if (tegra_sdhci_is_pad_and_regulator_valid(host)) {
/* Erratum: Enable SDHCI spec v3.00 support */
if (soc_data->nvquirks & NVQUIRK_ENABLE_SDHCI_SPEC_300)
misc_ctrl |= SDHCI_MISC_CTRL_ENABLE_SDHCI_SPEC_300;
clk_ctrl |= SDHCI_CLOCK_CTRL_SDR50_TUNING_OVERRIDE;
}
+ clk_ctrl |= tegra_host->default_trim << SDHCI_CLOCK_CTRL_TRIM_SHIFT;
+
sdhci_writel(host, misc_ctrl, SDHCI_TEGRA_VENDOR_MISC_CTRL);
sdhci_writel(host, clk_ctrl, SDHCI_TEGRA_VENDOR_CLOCK_CTRL);
- if (soc_data->nvquirks & NVQUIRK_HAS_PADCALIB)
+ if (soc_data->nvquirks & NVQUIRK_HAS_PADCALIB) {
+ pad_ctrl = sdhci_readl(host, SDHCI_TEGRA_SDMEM_COMP_PADCTRL);
+ pad_ctrl &= ~SDHCI_TEGRA_SDMEM_COMP_PADCTRL_VREF_SEL_MASK;
+ pad_ctrl |= SDHCI_TEGRA_SDMEM_COMP_PADCTRL_VREF_SEL_VAL;
+ sdhci_writel(host, pad_ctrl, SDHCI_TEGRA_SDMEM_COMP_PADCTRL);
+
tegra_host->pad_calib_required = true;
+ }
tegra_host->ddr_signaling = false;
}
-static void tegra_sdhci_pad_autocalib(struct sdhci_host *host)
+static void tegra_sdhci_configure_cal_pad(struct sdhci_host *host, bool enable)
{
u32 val;
- mdelay(1);
+ /*
+ * Enable or disable the additional I/O pad used by the drive strength
+ * calibration process.
+ */
+ val = sdhci_readl(host, SDHCI_TEGRA_SDMEM_COMP_PADCTRL);
+
+ if (enable)
+ val |= SDHCI_TEGRA_SDMEM_COMP_PADCTRL_E_INPUT_E_PWRD;
+ else
+ val &= ~SDHCI_TEGRA_SDMEM_COMP_PADCTRL_E_INPUT_E_PWRD;
+
+ sdhci_writel(host, val, SDHCI_TEGRA_SDMEM_COMP_PADCTRL);
+
+ if (enable)
+ usleep_range(1, 2);
+}
+
+static void tegra_sdhci_set_pad_autocal_offset(struct sdhci_host *host,
+ u16 pdpu)
+{
+ u32 reg;
+
+ reg = sdhci_readl(host, SDHCI_TEGRA_AUTO_CAL_CONFIG);
+ reg &= ~SDHCI_AUTO_CAL_PDPU_OFFSET_MASK;
+ reg |= pdpu;
+ sdhci_writel(host, reg, SDHCI_TEGRA_AUTO_CAL_CONFIG);
+}
+
+static void tegra_sdhci_pad_autocalib(struct sdhci_host *host)
+{
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct sdhci_tegra *tegra_host = sdhci_pltfm_priv(pltfm_host);
+ struct sdhci_tegra_autocal_offsets offsets =
+ tegra_host->autocal_offsets;
+ struct mmc_ios *ios = &host->mmc->ios;
+ bool card_clk_enabled;
+ u16 pdpu;
+ u32 reg;
+ int ret;
+
+ switch (ios->timing) {
+ case MMC_TIMING_UHS_SDR104:
+ pdpu = offsets.pull_down_sdr104 << 8 | offsets.pull_up_sdr104;
+ break;
+ case MMC_TIMING_MMC_HS400:
+ pdpu = offsets.pull_down_hs400 << 8 | offsets.pull_up_hs400;
+ break;
+ default:
+ if (ios->signal_voltage == MMC_SIGNAL_VOLTAGE_180)
+ pdpu = offsets.pull_down_1v8 << 8 | offsets.pull_up_1v8;
+ else
+ pdpu = offsets.pull_down_3v3 << 8 | offsets.pull_up_3v3;
+ }
+
+ tegra_sdhci_set_pad_autocal_offset(host, pdpu);
+
+ card_clk_enabled = tegra_sdhci_configure_card_clk(host, false);
+
+ tegra_sdhci_configure_cal_pad(host, true);
+
+ reg = sdhci_readl(host, SDHCI_TEGRA_AUTO_CAL_CONFIG);
+ reg |= SDHCI_AUTO_CAL_ENABLE | SDHCI_AUTO_CAL_START;
+ sdhci_writel(host, reg, SDHCI_TEGRA_AUTO_CAL_CONFIG);
+
+ usleep_range(1, 2);
+ /* 10 ms timeout */
+ ret = readl_poll_timeout(host->ioaddr + SDHCI_TEGRA_AUTO_CAL_STATUS,
+ reg, !(reg & SDHCI_TEGRA_AUTO_CAL_ACTIVE),
+ 1000, 10000);
+
+ tegra_sdhci_configure_cal_pad(host, false);
+
+ tegra_sdhci_configure_card_clk(host, card_clk_enabled);
+
+ if (ret) {
+ dev_err(mmc_dev(host->mmc), "Pad autocal timed out\n");
+
+ if (ios->signal_voltage == MMC_SIGNAL_VOLTAGE_180)
+ pdpu = offsets.pull_down_1v8_timeout << 8 |
+ offsets.pull_up_1v8_timeout;
+ else
+ pdpu = offsets.pull_down_3v3_timeout << 8 |
+ offsets.pull_up_3v3_timeout;
+
+ /* Disable automatic calibration and use fixed offsets */
+ reg = sdhci_readl(host, SDHCI_TEGRA_AUTO_CAL_CONFIG);
+ reg &= ~SDHCI_AUTO_CAL_ENABLE;
+ sdhci_writel(host, reg, SDHCI_TEGRA_AUTO_CAL_CONFIG);
- val = sdhci_readl(host, SDHCI_TEGRA_AUTO_CAL_CONFIG);
- val |= SDHCI_AUTO_CAL_ENABLE | SDHCI_AUTO_CAL_START;
- sdhci_writel(host,val, SDHCI_TEGRA_AUTO_CAL_CONFIG);
+ tegra_sdhci_set_pad_autocal_offset(host, pdpu);
+ }
+}
+
+static void tegra_sdhci_parse_pad_autocal_dt(struct sdhci_host *host)
+{
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct sdhci_tegra *tegra_host = sdhci_pltfm_priv(pltfm_host);
+ struct sdhci_tegra_autocal_offsets *autocal =
+ &tegra_host->autocal_offsets;
+ int err;
+
+ err = device_property_read_u32(host->mmc->parent,
+ "nvidia,pad-autocal-pull-up-offset-3v3",
+ &autocal->pull_up_3v3);
+ if (err)
+ autocal->pull_up_3v3 = 0;
+
+ err = device_property_read_u32(host->mmc->parent,
+ "nvidia,pad-autocal-pull-down-offset-3v3",
+ &autocal->pull_down_3v3);
+ if (err)
+ autocal->pull_down_3v3 = 0;
+
+ err = device_property_read_u32(host->mmc->parent,
+ "nvidia,pad-autocal-pull-up-offset-1v8",
+ &autocal->pull_up_1v8);
+ if (err)
+ autocal->pull_up_1v8 = 0;
+
+ err = device_property_read_u32(host->mmc->parent,
+ "nvidia,pad-autocal-pull-down-offset-1v8",
+ &autocal->pull_down_1v8);
+ if (err)
+ autocal->pull_down_1v8 = 0;
+
+ err = device_property_read_u32(host->mmc->parent,
+ "nvidia,pad-autocal-pull-up-offset-3v3-timeout",
+ &autocal->pull_up_3v3);
+ if (err)
+ autocal->pull_up_3v3_timeout = 0;
+
+ err = device_property_read_u32(host->mmc->parent,
+ "nvidia,pad-autocal-pull-down-offset-3v3-timeout",
+ &autocal->pull_down_3v3);
+ if (err)
+ autocal->pull_down_3v3_timeout = 0;
+
+ err = device_property_read_u32(host->mmc->parent,
+ "nvidia,pad-autocal-pull-up-offset-1v8-timeout",
+ &autocal->pull_up_1v8);
+ if (err)
+ autocal->pull_up_1v8_timeout = 0;
+
+ err = device_property_read_u32(host->mmc->parent,
+ "nvidia,pad-autocal-pull-down-offset-1v8-timeout",
+ &autocal->pull_down_1v8);
+ if (err)
+ autocal->pull_down_1v8_timeout = 0;
+
+ err = device_property_read_u32(host->mmc->parent,
+ "nvidia,pad-autocal-pull-up-offset-sdr104",
+ &autocal->pull_up_sdr104);
+ if (err)
+ autocal->pull_up_sdr104 = autocal->pull_up_1v8;
+
+ err = device_property_read_u32(host->mmc->parent,
+ "nvidia,pad-autocal-pull-down-offset-sdr104",
+ &autocal->pull_down_sdr104);
+ if (err)
+ autocal->pull_down_sdr104 = autocal->pull_down_1v8;
+
+ err = device_property_read_u32(host->mmc->parent,
+ "nvidia,pad-autocal-pull-up-offset-hs400",
+ &autocal->pull_up_hs400);
+ if (err)
+ autocal->pull_up_hs400 = autocal->pull_up_1v8;
+
+ err = device_property_read_u32(host->mmc->parent,
+ "nvidia,pad-autocal-pull-down-offset-hs400",
+ &autocal->pull_down_hs400);
+ if (err)
+ autocal->pull_down_hs400 = autocal->pull_down_1v8;
+}
+
+static void tegra_sdhci_request(struct mmc_host *mmc, struct mmc_request *mrq)
+{
+ struct sdhci_host *host = mmc_priv(mmc);
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct sdhci_tegra *tegra_host = sdhci_pltfm_priv(pltfm_host);
+ ktime_t since_calib = ktime_sub(ktime_get(), tegra_host->last_calib);
+
+ /* 100 ms calibration interval is specified in the TRM */
+ if (ktime_to_ms(since_calib) > 100) {
+ tegra_sdhci_pad_autocalib(host);
+ tegra_host->last_calib = ktime_get();
+ }
+
+ sdhci_request(mmc, mrq);
+}
+
+static void tegra_sdhci_parse_tap_and_trim(struct sdhci_host *host)
+{
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct sdhci_tegra *tegra_host = sdhci_pltfm_priv(pltfm_host);
+ int err;
+
+ err = device_property_read_u32(host->mmc->parent, "nvidia,default-tap",
+ &tegra_host->default_tap);
+ if (err)
+ tegra_host->default_tap = 0;
+
+ err = device_property_read_u32(host->mmc->parent, "nvidia,default-trim",
+ &tegra_host->default_trim);
+ if (err)
+ tegra_host->default_trim = 0;
+
+ err = device_property_read_u32(host->mmc->parent, "nvidia,dqs-trim",
+ &tegra_host->dqs_trim);
+ if (err)
+ tegra_host->dqs_trim = 0x11;
}
static void tegra_sdhci_set_clock(struct sdhci_host *host, unsigned int clock)
}
}
-static void tegra_sdhci_set_uhs_signaling(struct sdhci_host *host,
- unsigned timing)
+static unsigned int tegra_sdhci_get_max_clock(struct sdhci_host *host)
{
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
- struct sdhci_tegra *tegra_host = sdhci_pltfm_priv(pltfm_host);
- if (timing == MMC_TIMING_UHS_DDR50 ||
- timing == MMC_TIMING_MMC_DDR52)
- tegra_host->ddr_signaling = true;
-
- sdhci_set_uhs_signaling(host, timing);
+ return clk_round_rate(pltfm_host->clk, UINT_MAX);
}
-static unsigned int tegra_sdhci_get_max_clock(struct sdhci_host *host)
+static void tegra_sdhci_set_dqs_trim(struct sdhci_host *host, u8 trim)
{
- struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ u32 val;
- return clk_round_rate(pltfm_host->clk, UINT_MAX);
+ val = sdhci_readl(host, SDHCI_TEGRA_VENDOR_CAP_OVERRIDES);
+ val &= ~SDHCI_TEGRA_CAP_OVERRIDES_DQS_TRIM_MASK;
+ val |= trim << SDHCI_TEGRA_CAP_OVERRIDES_DQS_TRIM_SHIFT;
+ sdhci_writel(host, val, SDHCI_TEGRA_VENDOR_CAP_OVERRIDES);
}
-static void tegra_sdhci_set_tap(struct sdhci_host *host, unsigned int tap)
+static void tegra_sdhci_hs400_dll_cal(struct sdhci_host *host)
{
u32 reg;
+ int err;
+
+ reg = sdhci_readl(host, SDHCI_TEGRA_VENDOR_DLLCAL_CFG);
+ reg |= SDHCI_TEGRA_DLLCAL_CALIBRATE;
+ sdhci_writel(host, reg, SDHCI_TEGRA_VENDOR_DLLCAL_CFG);
+
+ /* 1 ms sleep, 5 ms timeout */
+ err = readl_poll_timeout(host->ioaddr + SDHCI_TEGRA_VENDOR_DLLCAL_STA,
+ reg, !(reg & SDHCI_TEGRA_DLLCAL_STA_ACTIVE),
+ 1000, 5000);
+ if (err)
+ dev_err(mmc_dev(host->mmc),
+ "HS400 delay line calibration timed out\n");
+}
- reg = sdhci_readl(host, SDHCI_TEGRA_VENDOR_CLOCK_CTRL);
- reg &= ~SDHCI_CLOCK_CTRL_TAP_MASK;
- reg |= tap << SDHCI_CLOCK_CTRL_TAP_SHIFT;
- sdhci_writel(host, reg, SDHCI_TEGRA_VENDOR_CLOCK_CTRL);
+static void tegra_sdhci_set_uhs_signaling(struct sdhci_host *host,
+ unsigned timing)
+{
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct sdhci_tegra *tegra_host = sdhci_pltfm_priv(pltfm_host);
+ bool set_default_tap = false;
+ bool set_dqs_trim = false;
+ bool do_hs400_dll_cal = false;
+
+ switch (timing) {
+ case MMC_TIMING_UHS_SDR50:
+ case MMC_TIMING_UHS_SDR104:
+ case MMC_TIMING_MMC_HS200:
+ /* Don't set default tap on tunable modes. */
+ break;
+ case MMC_TIMING_MMC_HS400:
+ set_dqs_trim = true;
+ do_hs400_dll_cal = true;
+ break;
+ case MMC_TIMING_MMC_DDR52:
+ case MMC_TIMING_UHS_DDR50:
+ tegra_host->ddr_signaling = true;
+ set_default_tap = true;
+ break;
+ default:
+ set_default_tap = true;
+ break;
+ }
+
+ sdhci_set_uhs_signaling(host, timing);
+
+ tegra_sdhci_pad_autocalib(host);
+
+ if (set_default_tap)
+ tegra_sdhci_set_tap(host, tegra_host->default_tap);
+
+ if (set_dqs_trim)
+ tegra_sdhci_set_dqs_trim(host, tegra_host->dqs_trim);
+
+ if (do_hs400_dll_cal)
+ tegra_sdhci_hs400_dll_cal(host);
}
static int tegra_sdhci_execute_tuning(struct sdhci_host *host, u32 opcode)
return mmc_send_tuning(host->mmc, opcode, NULL);
}
+static int tegra_sdhci_set_padctrl(struct sdhci_host *host, int voltage)
+{
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct sdhci_tegra *tegra_host = sdhci_pltfm_priv(pltfm_host);
+ int ret;
+
+ if (!tegra_host->pad_control_available)
+ return 0;
+
+ if (voltage == MMC_SIGNAL_VOLTAGE_180) {
+ ret = pinctrl_select_state(tegra_host->pinctrl_sdmmc,
+ tegra_host->pinctrl_state_1v8);
+ if (ret < 0)
+ dev_err(mmc_dev(host->mmc),
+ "setting 1.8V failed, ret: %d\n", ret);
+ } else {
+ ret = pinctrl_select_state(tegra_host->pinctrl_sdmmc,
+ tegra_host->pinctrl_state_3v3);
+ if (ret < 0)
+ dev_err(mmc_dev(host->mmc),
+ "setting 3.3V failed, ret: %d\n", ret);
+ }
+
+ return ret;
+}
+
+static int sdhci_tegra_start_signal_voltage_switch(struct mmc_host *mmc,
+ struct mmc_ios *ios)
+{
+ struct sdhci_host *host = mmc_priv(mmc);
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct sdhci_tegra *tegra_host = sdhci_pltfm_priv(pltfm_host);
+ int ret = 0;
+
+ if (ios->signal_voltage == MMC_SIGNAL_VOLTAGE_330) {
+ ret = tegra_sdhci_set_padctrl(host, ios->signal_voltage);
+ if (ret < 0)
+ return ret;
+ ret = sdhci_start_signal_voltage_switch(mmc, ios);
+ } else if (ios->signal_voltage == MMC_SIGNAL_VOLTAGE_180) {
+ ret = sdhci_start_signal_voltage_switch(mmc, ios);
+ if (ret < 0)
+ return ret;
+ ret = tegra_sdhci_set_padctrl(host, ios->signal_voltage);
+ }
+
+ if (tegra_host->pad_calib_required)
+ tegra_sdhci_pad_autocalib(host);
+
+ return ret;
+}
+
+static int tegra_sdhci_init_pinctrl_info(struct device *dev,
+ struct sdhci_tegra *tegra_host)
+{
+ tegra_host->pinctrl_sdmmc = devm_pinctrl_get(dev);
+ if (IS_ERR(tegra_host->pinctrl_sdmmc)) {
+ dev_dbg(dev, "No pinctrl info, err: %ld\n",
+ PTR_ERR(tegra_host->pinctrl_sdmmc));
+ return -1;
+ }
+
+ tegra_host->pinctrl_state_3v3 =
+ pinctrl_lookup_state(tegra_host->pinctrl_sdmmc, "sdmmc-3v3");
+ if (IS_ERR(tegra_host->pinctrl_state_3v3)) {
+ dev_warn(dev, "Missing 3.3V pad state, err: %ld\n",
+ PTR_ERR(tegra_host->pinctrl_state_3v3));
+ return -1;
+ }
+
+ tegra_host->pinctrl_state_1v8 =
+ pinctrl_lookup_state(tegra_host->pinctrl_sdmmc, "sdmmc-1v8");
+ if (IS_ERR(tegra_host->pinctrl_state_1v8)) {
+ dev_warn(dev, "Missing 1.8V pad state, err: %ld\n",
+ PTR_ERR(tegra_host->pinctrl_state_1v8));
+ return -1;
+ }
+
+ tegra_host->pad_control_available = true;
+
+ return 0;
+}
+
static void tegra_sdhci_voltage_switch(struct sdhci_host *host)
{
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
.pdata = &sdhci_tegra124_pdata,
};
+static const struct sdhci_ops tegra210_sdhci_ops = {
+ .get_ro = tegra_sdhci_get_ro,
+ .read_w = tegra_sdhci_readw,
+ .write_w = tegra210_sdhci_writew,
+ .write_l = tegra_sdhci_writel,
+ .set_clock = tegra_sdhci_set_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = tegra_sdhci_reset,
+ .set_uhs_signaling = tegra_sdhci_set_uhs_signaling,
+ .voltage_switch = tegra_sdhci_voltage_switch,
+ .get_max_clock = tegra_sdhci_get_max_clock,
+};
+
static const struct sdhci_pltfm_data sdhci_tegra210_pdata = {
.quirks = SDHCI_QUIRK_BROKEN_TIMEOUT_VAL |
SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK |
SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC |
SDHCI_QUIRK_CAP_CLOCK_BASE_BROKEN,
.quirks2 = SDHCI_QUIRK2_PRESET_VALUE_BROKEN,
- .ops = &tegra114_sdhci_ops,
+ .ops = &tegra210_sdhci_ops,
};
static const struct sdhci_tegra_soc_data soc_data_tegra210 = {
.pdata = &sdhci_tegra210_pdata,
+ .nvquirks = NVQUIRK_NEEDS_PAD_CONTROL |
+ NVQUIRK_HAS_PADCALIB |
+ NVQUIRK_DIS_CARD_CLK_CONFIG_TAP |
+ NVQUIRK_ENABLE_SDR50 |
+ NVQUIRK_ENABLE_SDR104,
+};
+
+static const struct sdhci_ops tegra186_sdhci_ops = {
+ .get_ro = tegra_sdhci_get_ro,
+ .read_w = tegra_sdhci_readw,
+ .write_l = tegra_sdhci_writel,
+ .set_clock = tegra_sdhci_set_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = tegra_sdhci_reset,
+ .set_uhs_signaling = tegra_sdhci_set_uhs_signaling,
+ .voltage_switch = tegra_sdhci_voltage_switch,
+ .get_max_clock = tegra_sdhci_get_max_clock,
};
static const struct sdhci_pltfm_data sdhci_tegra186_pdata = {
* But it is not supported as of now.
*/
SDHCI_QUIRK2_BROKEN_64_BIT_DMA,
- .ops = &tegra114_sdhci_ops,
+ .ops = &tegra186_sdhci_ops,
};
static const struct sdhci_tegra_soc_data soc_data_tegra186 = {
.pdata = &sdhci_tegra186_pdata,
+ .nvquirks = NVQUIRK_NEEDS_PAD_CONTROL |
+ NVQUIRK_HAS_PADCALIB |
+ NVQUIRK_DIS_CARD_CLK_CONFIG_TAP |
+ NVQUIRK_ENABLE_SDR50 |
+ NVQUIRK_ENABLE_SDR104,
};
static const struct of_device_id sdhci_tegra_dt_match[] = {
tegra_host = sdhci_pltfm_priv(pltfm_host);
tegra_host->ddr_signaling = false;
tegra_host->pad_calib_required = false;
+ tegra_host->pad_control_available = false;
tegra_host->soc_data = soc_data;
+ if (soc_data->nvquirks & NVQUIRK_NEEDS_PAD_CONTROL) {
+ rc = tegra_sdhci_init_pinctrl_info(&pdev->dev, tegra_host);
+ if (rc == 0)
+ host->mmc_host_ops.start_signal_voltage_switch =
+ sdhci_tegra_start_signal_voltage_switch;
+ }
+
+ /* Hook to periodically rerun pad calibration */
+ if (soc_data->nvquirks & NVQUIRK_HAS_PADCALIB)
+ host->mmc_host_ops.request = tegra_sdhci_request;
+
+ host->mmc_host_ops.hs400_enhanced_strobe =
+ tegra_sdhci_hs400_enhanced_strobe;
+
rc = mmc_of_parse(host->mmc);
if (rc)
goto err_parse_dt;
if (tegra_host->soc_data->nvquirks & NVQUIRK_ENABLE_DDR50)
host->mmc->caps |= MMC_CAP_1_8V_DDR;
+ tegra_sdhci_parse_pad_autocal_dt(host);
+
+ tegra_sdhci_parse_tap_and_trim(host);
+
tegra_host->power_gpio = devm_gpiod_get_optional(&pdev->dev, "power",
GPIOD_OUT_HIGH);
if (IS_ERR(tegra_host->power_gpio)) {
return 0;
if (of_address_to_resource(np, 1, &iomem)) {
- dev_err(mmc_dev(host->mmc), "Unable to find SoC PAD ctrl register address for %s\n",
- np->name);
+ dev_err(mmc_dev(host->mmc), "Unable to find SoC PAD ctrl register address for %pOFn\n",
+ np);
return -EINVAL;
}
* *
\*****************************************************************************/
+static void sdhci_do_enable_v4_mode(struct sdhci_host *host)
+{
+ u16 ctrl2;
+
+ ctrl2 = sdhci_readb(host, SDHCI_HOST_CONTROL2);
+ if (ctrl2 & SDHCI_CTRL_V4_MODE)
+ return;
+
+ ctrl2 |= SDHCI_CTRL_V4_MODE;
+ sdhci_writeb(host, ctrl2, SDHCI_HOST_CONTROL);
+}
+
+/*
+ * This can be called before sdhci_add_host() by Vendor's host controller
+ * driver to enable v4 mode if supported.
+ */
+void sdhci_enable_v4_mode(struct sdhci_host *host)
+{
+ host->v4_mode = true;
+ sdhci_do_enable_v4_mode(host);
+}
+EXPORT_SYMBOL_GPL(sdhci_enable_v4_mode);
+
static inline bool sdhci_data_line_cmd(struct mmc_command *cmd)
{
return cmd->data || cmd->flags & MMC_RSP_BUSY;
sdhci_writel(host, host->ier, SDHCI_SIGNAL_ENABLE);
}
+static void sdhci_config_dma(struct sdhci_host *host)
+{
+ u8 ctrl;
+ u16 ctrl2;
+
+ if (host->version < SDHCI_SPEC_200)
+ return;
+
+ ctrl = sdhci_readb(host, SDHCI_HOST_CONTROL);
+
+ /*
+ * Always adjust the DMA selection as some controllers
+ * (e.g. JMicron) can't do PIO properly when the selection
+ * is ADMA.
+ */
+ ctrl &= ~SDHCI_CTRL_DMA_MASK;
+ if (!(host->flags & SDHCI_REQ_USE_DMA))
+ goto out;
+
+ /* Note if DMA Select is zero then SDMA is selected */
+ if (host->flags & SDHCI_USE_ADMA)
+ ctrl |= SDHCI_CTRL_ADMA32;
+
+ if (host->flags & SDHCI_USE_64_BIT_DMA) {
+ /*
+ * If v4 mode, all supported DMA can be 64-bit addressing if
+ * controller supports 64-bit system address, otherwise only
+ * ADMA can support 64-bit addressing.
+ */
+ if (host->v4_mode) {
+ ctrl2 = sdhci_readw(host, SDHCI_HOST_CONTROL2);
+ ctrl2 |= SDHCI_CTRL_64BIT_ADDR;
+ sdhci_writew(host, ctrl2, SDHCI_HOST_CONTROL2);
+ } else if (host->flags & SDHCI_USE_ADMA) {
+ /*
+ * Don't need to undo SDHCI_CTRL_ADMA32 in order to
+ * set SDHCI_CTRL_ADMA64.
+ */
+ ctrl |= SDHCI_CTRL_ADMA64;
+ }
+ }
+
+out:
+ sdhci_writeb(host, ctrl, SDHCI_HOST_CONTROL);
+}
+
static void sdhci_init(struct sdhci_host *host, int soft)
{
struct mmc_host *mmc = host->mmc;
else
sdhci_do_reset(host, SDHCI_RESET_ALL);
+ if (host->v4_mode)
+ sdhci_do_enable_v4_mode(host);
+
sdhci_set_default_irqs(host);
host->cqe_on = false;
local_irq_restore(*flags);
}
-static void sdhci_adma_write_desc(struct sdhci_host *host, void *desc,
- dma_addr_t addr, int len, unsigned cmd)
+void sdhci_adma_write_desc(struct sdhci_host *host, void **desc,
+ dma_addr_t addr, int len, unsigned int cmd)
{
- struct sdhci_adma2_64_desc *dma_desc = desc;
+ struct sdhci_adma2_64_desc *dma_desc = *desc;
/* 32-bit and 64-bit descriptors have these members in same position */
dma_desc->cmd = cpu_to_le16(cmd);
if (host->flags & SDHCI_USE_64_BIT_DMA)
dma_desc->addr_hi = cpu_to_le32((u64)addr >> 32);
+
+ *desc += host->desc_sz;
+}
+EXPORT_SYMBOL_GPL(sdhci_adma_write_desc);
+
+static inline void __sdhci_adma_write_desc(struct sdhci_host *host,
+ void **desc, dma_addr_t addr,
+ int len, unsigned int cmd)
+{
+ if (host->ops->adma_write_desc)
+ host->ops->adma_write_desc(host, desc, addr, len, cmd);
+ else
+ sdhci_adma_write_desc(host, desc, addr, len, cmd);
}
static void sdhci_adma_mark_end(void *desc)
}
/* tran, valid */
- sdhci_adma_write_desc(host, desc, align_addr, offset,
- ADMA2_TRAN_VALID);
+ __sdhci_adma_write_desc(host, &desc, align_addr,
+ offset, ADMA2_TRAN_VALID);
BUG_ON(offset > 65536);
align += SDHCI_ADMA2_ALIGN;
align_addr += SDHCI_ADMA2_ALIGN;
- desc += host->desc_sz;
-
addr += offset;
len -= offset;
}
BUG_ON(len > 65536);
- if (len) {
- /* tran, valid */
- sdhci_adma_write_desc(host, desc, addr, len,
- ADMA2_TRAN_VALID);
- desc += host->desc_sz;
- }
+ /* tran, valid */
+ if (len)
+ __sdhci_adma_write_desc(host, &desc, addr, len,
+ ADMA2_TRAN_VALID);
/*
* If this triggers then we have a calculation bug
}
} else {
/* Add a terminating entry - nop, end, valid */
- sdhci_adma_write_desc(host, desc, 0, 0, ADMA2_NOP_END_VALID);
+ __sdhci_adma_write_desc(host, &desc, 0, 0, ADMA2_NOP_END_VALID);
}
}
}
}
-static u32 sdhci_sdma_address(struct sdhci_host *host)
+static dma_addr_t sdhci_sdma_address(struct sdhci_host *host)
{
if (host->bounce_buffer)
return host->bounce_addr;
return sg_dma_address(host->data->sg);
}
+static void sdhci_set_sdma_addr(struct sdhci_host *host, dma_addr_t addr)
+{
+ if (host->v4_mode) {
+ sdhci_writel(host, addr, SDHCI_ADMA_ADDRESS);
+ if (host->flags & SDHCI_USE_64_BIT_DMA)
+ sdhci_writel(host, (u64)addr >> 32, SDHCI_ADMA_ADDRESS_HI);
+ } else {
+ sdhci_writel(host, addr, SDHCI_DMA_ADDRESS);
+ }
+}
+
static unsigned int sdhci_target_timeout(struct sdhci_host *host,
struct mmc_command *cmd,
struct mmc_data *data)
static void sdhci_prepare_data(struct sdhci_host *host, struct mmc_command *cmd)
{
- u8 ctrl;
struct mmc_data *data = cmd->data;
host->data_timeout = 0;
SDHCI_ADMA_ADDRESS_HI);
} else {
WARN_ON(sg_cnt != 1);
- sdhci_writel(host, sdhci_sdma_address(host),
- SDHCI_DMA_ADDRESS);
+ sdhci_set_sdma_addr(host, sdhci_sdma_address(host));
}
}
- /*
- * Always adjust the DMA selection as some controllers
- * (e.g. JMicron) can't do PIO properly when the selection
- * is ADMA.
- */
- if (host->version >= SDHCI_SPEC_200) {
- ctrl = sdhci_readb(host, SDHCI_HOST_CONTROL);
- ctrl &= ~SDHCI_CTRL_DMA_MASK;
- if ((host->flags & SDHCI_REQ_USE_DMA) &&
- (host->flags & SDHCI_USE_ADMA)) {
- if (host->flags & SDHCI_USE_64_BIT_DMA)
- ctrl |= SDHCI_CTRL_ADMA64;
- else
- ctrl |= SDHCI_CTRL_ADMA32;
- } else {
- ctrl |= SDHCI_CTRL_SDMA;
- }
- sdhci_writeb(host, ctrl, SDHCI_HOST_CONTROL);
- }
+ sdhci_config_dma(host);
if (!(host->flags & SDHCI_REQ_USE_DMA)) {
int flags;
/* Set the DMA boundary value and block size */
sdhci_writew(host, SDHCI_MAKE_BLKSZ(host->sdma_boundary, data->blksz),
SDHCI_BLOCK_SIZE);
- sdhci_writew(host, data->blocks, SDHCI_BLOCK_COUNT);
+
+ /*
+ * For Version 4.10 onwards, if v4 mode is enabled, 32-bit Block Count
+ * can be supported, in that case 16-bit block count register must be 0.
+ */
+ if (host->version >= SDHCI_SPEC_410 && host->v4_mode &&
+ (host->quirks2 & SDHCI_QUIRK2_USE_32BIT_BLK_CNT)) {
+ if (sdhci_readw(host, SDHCI_BLOCK_COUNT))
+ sdhci_writew(host, 0, SDHCI_BLOCK_COUNT);
+ sdhci_writew(host, data->blocks, SDHCI_32BIT_BLK_CNT);
+ } else {
+ sdhci_writew(host, data->blocks, SDHCI_BLOCK_COUNT);
+ }
}
static inline bool sdhci_auto_cmd12(struct sdhci_host *host,
!mrq->cap_cmd_during_tfr;
}
+static inline void sdhci_auto_cmd_select(struct sdhci_host *host,
+ struct mmc_command *cmd,
+ u16 *mode)
+{
+ bool use_cmd12 = sdhci_auto_cmd12(host, cmd->mrq) &&
+ (cmd->opcode != SD_IO_RW_EXTENDED);
+ bool use_cmd23 = cmd->mrq->sbc && (host->flags & SDHCI_AUTO_CMD23);
+ u16 ctrl2;
+
+ /*
+ * In case of Version 4.10 or later, use of 'Auto CMD Auto
+ * Select' is recommended rather than use of 'Auto CMD12
+ * Enable' or 'Auto CMD23 Enable'.
+ */
+ if (host->version >= SDHCI_SPEC_410 && (use_cmd12 || use_cmd23)) {
+ *mode |= SDHCI_TRNS_AUTO_SEL;
+
+ ctrl2 = sdhci_readw(host, SDHCI_HOST_CONTROL2);
+ if (use_cmd23)
+ ctrl2 |= SDHCI_CMD23_ENABLE;
+ else
+ ctrl2 &= ~SDHCI_CMD23_ENABLE;
+ sdhci_writew(host, ctrl2, SDHCI_HOST_CONTROL2);
+
+ return;
+ }
+
+ /*
+ * If we are sending CMD23, CMD12 never gets sent
+ * on successful completion (so no Auto-CMD12).
+ */
+ if (use_cmd12)
+ *mode |= SDHCI_TRNS_AUTO_CMD12;
+ else if (use_cmd23)
+ *mode |= SDHCI_TRNS_AUTO_CMD23;
+}
+
static void sdhci_set_transfer_mode(struct sdhci_host *host,
struct mmc_command *cmd)
{
if (mmc_op_multi(cmd->opcode) || data->blocks > 1) {
mode = SDHCI_TRNS_BLK_CNT_EN | SDHCI_TRNS_MULTI;
- /*
- * If we are sending CMD23, CMD12 never gets sent
- * on successful completion (so no Auto-CMD12).
- */
- if (sdhci_auto_cmd12(host, cmd->mrq) &&
- (cmd->opcode != SD_IO_RW_EXTENDED))
- mode |= SDHCI_TRNS_AUTO_CMD12;
- else if (cmd->mrq->sbc && (host->flags & SDHCI_AUTO_CMD23)) {
- mode |= SDHCI_TRNS_AUTO_CMD23;
+ sdhci_auto_cmd_select(host, cmd, &mode);
+ if (cmd->mrq->sbc && (host->flags & SDHCI_AUTO_CMD23))
sdhci_writel(host, cmd->mrq->sbc->arg, SDHCI_ARGUMENT2);
- }
}
if (data->flags & MMC_DATA_READ)
* *
\*****************************************************************************/
-static void sdhci_request(struct mmc_host *mmc, struct mmc_request *mrq)
+void sdhci_request(struct mmc_host *mmc, struct mmc_request *mrq)
{
struct sdhci_host *host;
int present;
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
}
+EXPORT_SYMBOL_GPL(sdhci_request);
void sdhci_set_bus_width(struct sdhci_host *host, int width)
{
}
EXPORT_SYMBOL_GPL(sdhci_send_tuning);
-static void __sdhci_execute_tuning(struct sdhci_host *host, u32 opcode)
+static int __sdhci_execute_tuning(struct sdhci_host *host, u32 opcode)
{
int i;
pr_info("%s: Tuning timeout, falling back to fixed sampling clock\n",
mmc_hostname(host->mmc));
sdhci_abort_tuning(host, opcode);
- return;
+ return -ETIMEDOUT;
}
ctrl = sdhci_readw(host, SDHCI_HOST_CONTROL2);
if (!(ctrl & SDHCI_CTRL_EXEC_TUNING)) {
if (ctrl & SDHCI_CTRL_TUNED_CLK)
- return; /* Success! */
+ return 0; /* Success! */
break;
}
pr_info("%s: Tuning failed, falling back to fixed sampling clock\n",
mmc_hostname(host->mmc));
sdhci_reset_tuning(host);
+ return -EAGAIN;
}
int sdhci_execute_tuning(struct mmc_host *mmc, u32 opcode)
sdhci_start_tuning(host);
- __sdhci_execute_tuning(host, opcode);
+ host->tuning_err = __sdhci_execute_tuning(host, opcode);
sdhci_end_tuning(host);
out:
* some controllers are faulty, don't trust them.
*/
if (intmask & SDHCI_INT_DMA_END) {
- u32 dmastart, dmanow;
+ dma_addr_t dmastart, dmanow;
dmastart = sdhci_sdma_address(host);
dmanow = dmastart + host->data->bytes_xfered;
* Force update to the next DMA block boundary.
*/
dmanow = (dmanow &
- ~(SDHCI_DEFAULT_BOUNDARY_SIZE - 1)) +
+ ~((dma_addr_t)SDHCI_DEFAULT_BOUNDARY_SIZE - 1)) +
SDHCI_DEFAULT_BOUNDARY_SIZE;
host->data->bytes_xfered = dmanow - dmastart;
- DBG("DMA base 0x%08x, transferred 0x%06x bytes, next 0x%08x\n",
- dmastart, host->data->bytes_xfered, dmanow);
- sdhci_writel(host, dmanow, SDHCI_DMA_ADDRESS);
+ DBG("DMA base %pad, transferred 0x%06x bytes, next %pad\n",
+ &dmastart, host->data->bytes_xfered, &dmanow);
+ sdhci_set_sdma_addr(host, dmanow);
}
if (intmask & SDHCI_INT_DATA_END) {
host->sdma_boundary = SDHCI_DEFAULT_BOUNDARY_ARG;
+ /*
+ * The DMA table descriptor count is calculated as the maximum
+ * number of segments times 2, to allow for an alignment
+ * descriptor for each segment, plus 1 for a nop end descriptor.
+ */
+ host->adma_table_cnt = SDHCI_MAX_SEGS * 2 + 1;
+
return host;
}
sdhci_do_reset(host, SDHCI_RESET_ALL);
+ if (host->v4_mode)
+ sdhci_do_enable_v4_mode(host);
+
of_property_read_u64(mmc_dev(host->mmc)->of_node,
"sdhci-caps-mask", &dt_caps_mask);
of_property_read_u64(mmc_dev(host->mmc)->of_node,
return 0;
}
+static inline bool sdhci_can_64bit_dma(struct sdhci_host *host)
+{
+ /*
+ * According to SD Host Controller spec v4.10, bit[27] added from
+ * version 4.10 in Capabilities Register is used as 64-bit System
+ * Address support for V4 mode.
+ */
+ if (host->version >= SDHCI_SPEC_410 && host->v4_mode)
+ return host->caps & SDHCI_CAN_64BIT_V4;
+
+ return host->caps & SDHCI_CAN_64BIT;
+}
+
int sdhci_setup_host(struct sdhci_host *host)
{
struct mmc_host *mmc;
override_timeout_clk = host->timeout_clk;
- if (host->version > SDHCI_SPEC_300) {
+ if (host->version > SDHCI_SPEC_420) {
pr_err("%s: Unknown controller version (%d). You may experience problems.\n",
mmc_hostname(mmc), host->version);
}
* SDHCI_QUIRK2_BROKEN_64_BIT_DMA must be left to the drivers to
* implement.
*/
- if (host->caps & SDHCI_CAN_64BIT)
+ if (sdhci_can_64bit_dma(host))
host->flags |= SDHCI_USE_64_BIT_DMA;
if (host->flags & (SDHCI_USE_SDMA | SDHCI_USE_ADMA)) {
}
}
- /* SDMA does not support 64-bit DMA */
- if (host->flags & SDHCI_USE_64_BIT_DMA)
+ /* SDMA does not support 64-bit DMA if v4 mode not set */
+ if ((host->flags & SDHCI_USE_64_BIT_DMA) && !host->v4_mode)
host->flags &= ~SDHCI_USE_SDMA;
if (host->flags & SDHCI_USE_ADMA) {
dma_addr_t dma;
void *buf;
- /*
- * The DMA descriptor table size is calculated as the maximum
- * number of segments times 2, to allow for an alignment
- * descriptor for each segment, plus 1 for a nop end descriptor,
- * all multipled by the descriptor size.
- */
if (host->flags & SDHCI_USE_64_BIT_DMA) {
- host->adma_table_sz = (SDHCI_MAX_SEGS * 2 + 1) *
- SDHCI_ADMA2_64_DESC_SZ;
- host->desc_sz = SDHCI_ADMA2_64_DESC_SZ;
+ host->adma_table_sz = host->adma_table_cnt *
+ SDHCI_ADMA2_64_DESC_SZ(host);
+ host->desc_sz = SDHCI_ADMA2_64_DESC_SZ(host);
} else {
- host->adma_table_sz = (SDHCI_MAX_SEGS * 2 + 1) *
+ host->adma_table_sz = host->adma_table_cnt *
SDHCI_ADMA2_32_DESC_SZ;
host->desc_sz = SDHCI_ADMA2_32_DESC_SZ;
}
host->align_buffer_sz = SDHCI_MAX_SEGS * SDHCI_ADMA2_ALIGN;
- buf = dma_alloc_coherent(mmc_dev(mmc), host->align_buffer_sz +
+ /*
+ * Use zalloc to zero the reserved high 32-bits of 128-bit
+ * descriptors so that they never need to be written.
+ */
+ buf = dma_zalloc_coherent(mmc_dev(mmc), host->align_buffer_sz +
host->adma_table_sz, &dma, GFP_KERNEL);
if (!buf) {
pr_warn("%s: Unable to allocate ADMA buffers - falling back to standard DMA\n",
if (host->quirks & SDHCI_QUIRK_MULTIBLOCK_READ_ACMD12)
host->flags |= SDHCI_AUTO_CMD12;
- /* Auto-CMD23 stuff only works in ADMA or PIO. */
+ /*
+ * For v3 mode, Auto-CMD23 stuff only works in ADMA or PIO.
+ * For v4 mode, SDMA may use Auto-CMD23 as well.
+ */
if ((host->version >= SDHCI_SPEC_300) &&
((host->flags & SDHCI_USE_ADMA) ||
- !(host->flags & SDHCI_USE_SDMA)) &&
+ !(host->flags & SDHCI_USE_SDMA) || host->v4_mode) &&
!(host->quirks2 & SDHCI_QUIRK2_ACMD23_BROKEN)) {
host->flags |= SDHCI_AUTO_CMD23;
DBG("Auto-CMD23 available\n");
#define SDHCI_DMA_ADDRESS 0x00
#define SDHCI_ARGUMENT2 SDHCI_DMA_ADDRESS
+#define SDHCI_32BIT_BLK_CNT SDHCI_DMA_ADDRESS
#define SDHCI_BLOCK_SIZE 0x04
#define SDHCI_MAKE_BLKSZ(dma, blksz) (((dma & 0x7) << 12) | (blksz & 0xFFF))
#define SDHCI_TRNS_BLK_CNT_EN 0x02
#define SDHCI_TRNS_AUTO_CMD12 0x04
#define SDHCI_TRNS_AUTO_CMD23 0x08
+#define SDHCI_TRNS_AUTO_SEL 0x0C
#define SDHCI_TRNS_READ 0x10
#define SDHCI_TRNS_MULTI 0x20
#define SDHCI_CTRL_DRV_TYPE_D 0x0030
#define SDHCI_CTRL_EXEC_TUNING 0x0040
#define SDHCI_CTRL_TUNED_CLK 0x0080
+#define SDHCI_CMD23_ENABLE 0x0800
+#define SDHCI_CTRL_V4_MODE 0x1000
+#define SDHCI_CTRL_64BIT_ADDR 0x2000
#define SDHCI_CTRL_PRESET_VAL_ENABLE 0x8000
#define SDHCI_CAPABILITIES 0x40
#define SDHCI_CAN_VDD_330 0x01000000
#define SDHCI_CAN_VDD_300 0x02000000
#define SDHCI_CAN_VDD_180 0x04000000
+#define SDHCI_CAN_64BIT_V4 0x08000000
#define SDHCI_CAN_64BIT 0x10000000
#define SDHCI_SUPPORT_SDR50 0x00000001
#define SDHCI_SPEC_100 0
#define SDHCI_SPEC_200 1
#define SDHCI_SPEC_300 2
+#define SDHCI_SPEC_400 3
+#define SDHCI_SPEC_410 4
+#define SDHCI_SPEC_420 5
/*
* End of controller registers.
*/
#define SDHCI_ADMA2_DESC_ALIGN 8
-/* ADMA2 64-bit DMA descriptor size */
-#define SDHCI_ADMA2_64_DESC_SZ 12
+/*
+ * ADMA2 64-bit DMA descriptor size
+ * According to SD Host Controller spec v4.10, there are two kinds of
+ * descriptors for 64-bit addressing mode: 96-bit Descriptor and 128-bit
+ * Descriptor, if Host Version 4 Enable is set in the Host Control 2
+ * register, 128-bit Descriptor will be selected.
+ */
+#define SDHCI_ADMA2_64_DESC_SZ(host) ((host)->v4_mode ? 16 : 12)
/*
* ADMA2 64-bit descriptor. Note 12-byte descriptor can't always be 8-byte
* obtainable timeout.
*/
#define SDHCI_QUIRK2_DISABLE_HW_TIMEOUT (1<<17)
+/*
+ * 32-bit block count may not support eMMC where upper bits of CMD23 are used
+ * for other purposes. Consequently we support 16-bit block count by default.
+ * Otherwise, SDHCI_QUIRK2_USE_32BIT_BLK_CNT can be selected to use 32-bit
+ * block count.
+ */
+#define SDHCI_QUIRK2_USE_32BIT_BLK_CNT (1<<18)
int irq; /* Device IRQ */
void __iomem *ioaddr; /* Mapped address */
bool preset_enabled; /* Preset is enabled */
bool pending_reset; /* Cmd/data reset is pending */
bool irq_wake_enabled; /* IRQ wakeup is enabled */
+ bool v4_mode; /* Host Version 4 Enable */
struct mmc_request *mrqs_done[SDHCI_MAX_MRQS]; /* Requests done */
struct mmc_command *cmd; /* Current command */
unsigned int tuning_count; /* Timer count for re-tuning */
unsigned int tuning_mode; /* Re-tuning mode supported by host */
+ unsigned int tuning_err; /* Error code for re-tuning */
#define SDHCI_TUNING_MODE_1 0
#define SDHCI_TUNING_MODE_2 1
#define SDHCI_TUNING_MODE_3 2
/* Host SDMA buffer boundary. */
u32 sdma_boundary;
+ /* Host ADMA table count */
+ u32 adma_table_cnt;
+
u64 data_timeout;
unsigned long private[0] ____cacheline_aligned;
void (*adma_workaround)(struct sdhci_host *host, u32 intmask);
void (*card_event)(struct sdhci_host *host);
void (*voltage_switch)(struct sdhci_host *host);
+ void (*adma_write_desc)(struct sdhci_host *host, void **desc,
+ dma_addr_t addr, int len, unsigned int cmd);
};
#ifdef CONFIG_MMC_SDHCI_IO_ACCESSORS
unsigned short vdd);
void sdhci_set_power_noreg(struct sdhci_host *host, unsigned char mode,
unsigned short vdd);
+void sdhci_request(struct mmc_host *mmc, struct mmc_request *mrq);
void sdhci_set_bus_width(struct sdhci_host *host, int width);
void sdhci_reset(struct sdhci_host *host, u8 mask);
void sdhci_set_uhs_signaling(struct sdhci_host *host, unsigned timing);
int sdhci_start_signal_voltage_switch(struct mmc_host *mmc,
struct mmc_ios *ios);
void sdhci_enable_sdio_irq(struct mmc_host *mmc, int enable);
+void sdhci_adma_write_desc(struct sdhci_host *host, void **desc,
+ dma_addr_t addr, int len, unsigned int cmd);
#ifdef CONFIG_PM
int sdhci_suspend_host(struct sdhci_host *host);
int *data_error);
void sdhci_dumpregs(struct sdhci_host *host);
+void sdhci_enable_v4_mode(struct sdhci_host *host);
void sdhci_start_tuning(struct sdhci_host *host);
void sdhci_end_tuning(struct sdhci_host *host);
+// SPDX-License-Identifier: GPL-2.0
/*
* MMCIF eMMC driver.
*
* Copyright (C) 2010 Renesas Solutions Corp.
* Yusuke Goda <yusuke.goda.sx@renesas.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License.
*/
/*
module_platform_driver(sh_mmcif_driver);
MODULE_DESCRIPTION("SuperH on-chip MMC/eMMC interface driver");
-MODULE_LICENSE("GPL");
+MODULE_LICENSE("GPL v2");
MODULE_ALIAS("platform:" DRIVER_NAME);
MODULE_AUTHOR("Yusuke Goda <yusuke.goda.sx@renesas.com>");
/* Does DATA0 needs to be masked while the clock is updated */
bool mask_data0;
- /* hardware only supports new timing mode */
+ /*
+ * hardware only supports new timing mode, either due to lack of
+ * a mode switch in the clock controller, or the mmc controller
+ * is permanently configured in the new timing mode, without the
+ * NTSR mode switch.
+ */
bool needs_new_timings;
- /* hardware can switch between old and new timing modes */
- bool has_timings_switch;
+ /* clock hardware can switch between old and new timing modes */
+ bool ccu_has_timings_switch;
};
struct sunxi_mmc_host {
clock <<= 1;
}
- if (host->use_new_timings && host->cfg->has_timings_switch) {
+ if (host->use_new_timings && host->cfg->ccu_has_timings_switch) {
ret = sunxi_ccu_set_mmc_timing_mode(host->clk_mmc, true);
if (ret) {
dev_err(mmc_dev(mmc),
/* update card clock rate to account for internal divider */
rate /= div;
+ /*
+ * Configure the controller to use the new timing mode if needed.
+ * On controllers that only support the new timing mode, such as
+ * the eMMC controller on the A64, this register does not exist,
+ * and any writes to it are ignored.
+ */
if (host->use_new_timings) {
/* Don't touch the delay bits */
rval = mmc_readl(host, REG_SD_NTSR);
.idma_des_size_bits = 16,
.clk_delays = sunxi_mmc_clk_delays,
.can_calibrate = false,
- .has_timings_switch = true,
+ .ccu_has_timings_switch = true,
};
static const struct sunxi_mmc_cfg sun9i_a80_cfg = {
.idma_des_size_bits = 13,
.clk_delays = NULL,
.can_calibrate = true,
+ .needs_new_timings = true,
};
static const struct of_device_id sunxi_mmc_of_match[] = {
goto error_free_host;
}
- if (host->cfg->has_timings_switch) {
+ if (host->cfg->ccu_has_timings_switch) {
/*
* Supports both old and new timing modes.
* Try setting the clk to new timing mode.
rc |= TIFM_MMCSD_RSP_R0;
break;
case MMC_RSP_R1B:
- rc |= TIFM_MMCSD_RSP_BUSY; // deliberate fall-through
+ rc |= TIFM_MMCSD_RSP_BUSY;
+ /* fall-through */
case MMC_RSP_R1:
rc |= TIFM_MMCSD_RSP_R1;
break;
+// SPDX-License-Identifier: GPL-2.0
/*
* Driver for the MMC / SD / SDIO cell found in:
*
* Copyright (C) 2017 Horms Solutions, Simon Horman
* Copyright (C) 2007 Ian Molton
* Copyright (C) 2004 Ian Molton
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
*/
+#include <linux/delay.h>
#include <linux/device.h>
#include <linux/mfd/core.h>
#include <linux/mfd/tmio.h>
#include "tmio_mmc.h"
+/* Registers specific to this variant */
+#define CTL_SDIO_REGS 0x100
+#define CTL_CLK_AND_WAIT_CTL 0x138
+#define CTL_RESET_SDIO 0x1e0
+
+static void tmio_mmc_clk_start(struct tmio_mmc_host *host)
+{
+ sd_ctrl_write16(host, CTL_SD_CARD_CLK_CTL, CLK_CTL_SCLKEN |
+ sd_ctrl_read16(host, CTL_SD_CARD_CLK_CTL));
+
+ usleep_range(10000, 11000);
+ sd_ctrl_write16(host, CTL_CLK_AND_WAIT_CTL, 0x0100);
+ usleep_range(10000, 11000);
+}
+
+static void tmio_mmc_clk_stop(struct tmio_mmc_host *host)
+{
+ sd_ctrl_write16(host, CTL_CLK_AND_WAIT_CTL, 0x0000);
+ usleep_range(10000, 11000);
+
+ sd_ctrl_write16(host, CTL_SD_CARD_CLK_CTL, ~CLK_CTL_SCLKEN &
+ sd_ctrl_read16(host, CTL_SD_CARD_CLK_CTL));
+
+ usleep_range(10000, 11000);
+}
+
+static void tmio_mmc_set_clock(struct tmio_mmc_host *host,
+ unsigned int new_clock)
+{
+ unsigned int divisor;
+ u32 clk = 0;
+ int clk_sel;
+
+ if (new_clock == 0) {
+ tmio_mmc_clk_stop(host);
+ return;
+ }
+
+ divisor = host->pdata->hclk / new_clock;
+
+ /* bit7 set: 1/512, ... bit0 set: 1/4, all bits clear: 1/2 */
+ clk_sel = (divisor <= 1);
+ clk = clk_sel ? 0 : (roundup_pow_of_two(divisor) >> 2);
+
+ host->pdata->set_clk_div(host->pdev, clk_sel);
+
+ sd_ctrl_write16(host, CTL_SD_CARD_CLK_CTL, ~CLK_CTL_SCLKEN &
+ sd_ctrl_read16(host, CTL_SD_CARD_CLK_CTL));
+ sd_ctrl_write16(host, CTL_SD_CARD_CLK_CTL, clk & CLK_CTL_DIV_MASK);
+ usleep_range(10000, 11000);
+
+ tmio_mmc_clk_start(host);
+}
+
+static void tmio_mmc_reset(struct tmio_mmc_host *host)
+{
+ /* FIXME - should we set stop clock reg here */
+ sd_ctrl_write16(host, CTL_RESET_SD, 0x0000);
+ sd_ctrl_write16(host, CTL_RESET_SDIO, 0x0000);
+ usleep_range(10000, 11000);
+ sd_ctrl_write16(host, CTL_RESET_SD, 0x0001);
+ sd_ctrl_write16(host, CTL_RESET_SDIO, 0x0001);
+ usleep_range(10000, 11000);
+
+ if (host->pdata->flags & TMIO_MMC_SDIO_IRQ) {
+ sd_ctrl_write16(host, CTL_SDIO_IRQ_MASK, host->sdio_irq_mask);
+ sd_ctrl_write16(host, CTL_TRANSACTION_CTL, 0x0001);
+ }
+}
+
#ifdef CONFIG_PM_SLEEP
static int tmio_mmc_suspend(struct device *dev)
{
goto cell_disable;
}
- pdata->flags |= TMIO_MMC_HAVE_HIGH_REG;
-
host = tmio_mmc_host_alloc(pdev, pdata);
if (IS_ERR(host)) {
ret = PTR_ERR(host);
/* SD control register space size is 0x200, 0x400 for bus_shift=1 */
host->bus_shift = resource_size(res) >> 10;
+ host->set_clock = tmio_mmc_set_clock;
+ host->reset = tmio_mmc_reset;
host->mmc->f_max = pdata->hclk;
host->mmc->f_min = pdata->hclk / 512;
+/* SPDX-License-Identifier: GPL-2.0 */
/*
* Driver for the MMC / SD / SDIO cell found in:
*
* Copyright (C) 2016-17 Horms Solutions, Simon Horman
* Copyright (C) 2007 Ian Molton
* Copyright (C) 2004 Ian Molton
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
*/
#ifndef TMIO_MMC_H
#define CTL_RESET_SD 0xe0
#define CTL_VERSION 0xe2
#define CTL_SDIF_MODE 0xe6
-#define CTL_SDIO_REGS 0x100
-#define CTL_CLK_AND_WAIT_CTL 0x138
-#define CTL_RESET_SDIO 0x1e0
/* Definitions for values the CTL_STOP_INTERNAL_ACTION register can take */
#define TMIO_STOP_STP BIT(0)
/* Callbacks for clock / power control */
void (*set_pwr)(struct platform_device *host, int state);
- void (*set_clk_div)(struct platform_device *host, int state);
/* pio related stuff */
struct scatterlist *sg_ptr;
struct tmio_mmc_data *pdata;
/* DMA support */
- bool force_pio;
+ bool dma_on;
struct dma_chan *chan_rx;
struct dma_chan *chan_tx;
struct tasklet_struct dma_issue;
/* Mandatory callback */
int (*clk_enable)(struct tmio_mmc_host *host);
+ void (*set_clock)(struct tmio_mmc_host *host, unsigned int clock);
/* Optional callbacks */
- unsigned int (*clk_update)(struct tmio_mmc_host *host,
- unsigned int new_clock);
void (*clk_disable)(struct tmio_mmc_host *host);
int (*multi_io_quirk)(struct mmc_card *card,
unsigned int direction, int blk_size);
int (*write16_hook)(struct tmio_mmc_host *host, int addr);
+ void (*reset)(struct tmio_mmc_host *host);
void (*hw_reset)(struct tmio_mmc_host *host);
void (*prepare_tuning)(struct tmio_mmc_host *host, unsigned long tap);
bool (*check_scc_error)(struct tmio_mmc_host *host);
+// SPDX-License-Identifier: GPL-2.0
/*
* Driver for the MMC / SD / SDIO IP found in:
*
* Copyright (C) 2007 Ian Molton
* Copyright (C) 2004 Ian Molton
*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
* This driver draws mainly on scattered spec sheets, Reverse engineering
* of the toshiba e800 SD driver and some parts of the 2.4 ASIC3 driver (4 bit
* support). (Further 4 bit support from a later datasheet).
}
}
-static void tmio_mmc_clk_start(struct tmio_mmc_host *host)
-{
- sd_ctrl_write16(host, CTL_SD_CARD_CLK_CTL, CLK_CTL_SCLKEN |
- sd_ctrl_read16(host, CTL_SD_CARD_CLK_CTL));
-
- /* HW engineers overrode docs: no sleep needed on R-Car2+ */
- if (!(host->pdata->flags & TMIO_MMC_MIN_RCAR2))
- usleep_range(10000, 11000);
-
- if (host->pdata->flags & TMIO_MMC_HAVE_HIGH_REG) {
- sd_ctrl_write16(host, CTL_CLK_AND_WAIT_CTL, 0x0100);
- usleep_range(10000, 11000);
- }
-}
-
-static void tmio_mmc_clk_stop(struct tmio_mmc_host *host)
-{
- if (host->pdata->flags & TMIO_MMC_HAVE_HIGH_REG) {
- sd_ctrl_write16(host, CTL_CLK_AND_WAIT_CTL, 0x0000);
- usleep_range(10000, 11000);
- }
-
- sd_ctrl_write16(host, CTL_SD_CARD_CLK_CTL, ~CLK_CTL_SCLKEN &
- sd_ctrl_read16(host, CTL_SD_CARD_CLK_CTL));
-
- /* HW engineers overrode docs: no sleep needed on R-Car2+ */
- if (!(host->pdata->flags & TMIO_MMC_MIN_RCAR2))
- usleep_range(10000, 11000);
-}
-
-static void tmio_mmc_set_clock(struct tmio_mmc_host *host,
- unsigned int new_clock)
-{
- u32 clk = 0, clock;
-
- if (new_clock == 0) {
- tmio_mmc_clk_stop(host);
- return;
- }
- /*
- * Both HS400 and HS200/SD104 set 200MHz, but some devices need to
- * set 400MHz to distinguish the CPG settings in HS400.
- */
- if (host->mmc->ios.timing == MMC_TIMING_MMC_HS400 &&
- host->pdata->flags & TMIO_MMC_HAVE_4TAP_HS400 &&
- new_clock == 200000000)
- new_clock = 400000000;
-
- if (host->clk_update)
- clock = host->clk_update(host, new_clock) / 512;
- else
- clock = host->mmc->f_min;
-
- for (clk = 0x80000080; new_clock >= (clock << 1); clk >>= 1)
- clock <<= 1;
-
- /* 1/1 clock is option */
- if ((host->pdata->flags & TMIO_MMC_CLK_ACTUAL) &&
- ((clk >> 22) & 0x1)) {
- if (!(host->mmc->ios.timing == MMC_TIMING_MMC_HS400))
- clk |= 0xff;
- else
- clk &= ~0xff;
- }
-
- if (host->set_clk_div)
- host->set_clk_div(host->pdev, (clk >> 22) & 1);
-
- sd_ctrl_write16(host, CTL_SD_CARD_CLK_CTL, ~CLK_CTL_SCLKEN &
- sd_ctrl_read16(host, CTL_SD_CARD_CLK_CTL));
- sd_ctrl_write16(host, CTL_SD_CARD_CLK_CTL, clk & CLK_CTL_DIV_MASK);
- if (!(host->pdata->flags & TMIO_MMC_MIN_RCAR2))
- usleep_range(10000, 11000);
-
- tmio_mmc_clk_start(host);
-}
-
static void tmio_mmc_reset(struct tmio_mmc_host *host)
{
/* FIXME - should we set stop clock reg here */
sd_ctrl_write16(host, CTL_RESET_SD, 0x0000);
- if (host->pdata->flags & TMIO_MMC_HAVE_HIGH_REG)
- sd_ctrl_write16(host, CTL_RESET_SDIO, 0x0000);
usleep_range(10000, 11000);
sd_ctrl_write16(host, CTL_RESET_SD, 0x0001);
- if (host->pdata->flags & TMIO_MMC_HAVE_HIGH_REG)
- sd_ctrl_write16(host, CTL_RESET_SDIO, 0x0001);
usleep_range(10000, 11000);
if (host->pdata->flags & TMIO_MMC_SDIO_IRQ) {
sd_ctrl_write16(host, CTL_SDIO_IRQ_MASK, host->sdio_irq_mask);
sd_ctrl_write16(host, CTL_TRANSACTION_CTL, 0x0001);
}
-
}
static void tmio_mmc_reset_work(struct work_struct *work)
spin_unlock_irqrestore(&host->lock, flags);
- tmio_mmc_reset(host);
+ host->reset(host);
/* Ready for new calls */
host->mrq = NULL;
unsigned int count;
unsigned long flags;
- if ((host->chan_tx || host->chan_rx) && !host->force_pio) {
+ if (host->dma_on) {
pr_err("PIO IRQ in DMA mode!\n");
return;
} else if (!data) {
*/
if (data->flags & MMC_DATA_READ) {
- if (host->chan_rx && !host->force_pio)
+ if (host->dma_on)
tmio_mmc_check_bounce_buffer(host);
dev_dbg(&host->pdev->dev, "Complete Rx request %p\n",
host->mrq);
if (stat & TMIO_STAT_CRCFAIL || stat & TMIO_STAT_STOPBIT_ERR ||
stat & TMIO_STAT_TXUNDERRUN)
data->error = -EILSEQ;
- if (host->chan_tx && (data->flags & MMC_DATA_WRITE) && !host->force_pio) {
+ if (host->dma_on && (data->flags & MMC_DATA_WRITE)) {
u32 status = sd_ctrl_read16_and_16_as_32(host, CTL_STATUS);
bool done = false;
tmio_mmc_disable_mmc_irqs(host, TMIO_STAT_DATAEND);
tmio_mmc_dataend_dma(host);
}
- } else if (host->chan_rx && (data->flags & MMC_DATA_READ) && !host->force_pio) {
+ } else if (host->dma_on && (data->flags & MMC_DATA_READ)) {
tmio_mmc_disable_mmc_irqs(host, TMIO_STAT_DATAEND);
tmio_mmc_dataend_dma(host);
} else {
*/
if (host->data && (!cmd->error || cmd->error == -EILSEQ)) {
if (host->data->flags & MMC_DATA_READ) {
- if (host->force_pio || !host->chan_rx) {
+ if (!host->dma_on) {
tmio_mmc_enable_mmc_irqs(host, TMIO_MASK_READOP);
} else {
tmio_mmc_disable_mmc_irqs(host,
tasklet_schedule(&host->dma_issue);
}
} else {
- if (host->force_pio || !host->chan_tx) {
+ if (!host->dma_on) {
tmio_mmc_enable_mmc_irqs(host, TMIO_MASK_WRITEOP);
} else {
tmio_mmc_disable_mmc_irqs(host,
tmio_mmc_init_sg(host, data);
host->data = data;
- host->force_pio = false;
+ host->dma_on = false;
/* Set transfer length / blocksize */
sd_ctrl_write16(host, CTL_SD_XFER_LEN, data->blksz);
if (mrq->cmd->error || (mrq->data && mrq->data->error))
tmio_mmc_abort_dma(host);
- if (host->check_scc_error)
- host->check_scc_error(host);
+ if (host->check_scc_error && host->check_scc_error(host))
+ mrq->cmd->error = -EILSEQ;
/* If SET_BLOCK_COUNT, continue with main command */
if (host->mrq && !mrq->cmd->error) {
switch (ios->power_mode) {
case MMC_POWER_OFF:
tmio_mmc_power_off(host);
- tmio_mmc_clk_stop(host);
+ host->set_clock(host, 0);
break;
case MMC_POWER_UP:
tmio_mmc_power_on(host, ios->vdd);
- tmio_mmc_set_clock(host, ios->clock);
+ host->set_clock(host, ios->clock);
tmio_mmc_set_bus_width(host, ios->bus_width);
break;
case MMC_POWER_ON:
- tmio_mmc_set_clock(host, ios->clock);
+ host->set_clock(host, ios->clock);
tmio_mmc_set_bus_width(host, ios->bus_width);
break;
}
int ret;
/*
- * Check the sanity of mmc->f_min to prevent tmio_mmc_set_clock() from
+ * Check the sanity of mmc->f_min to prevent host->set_clock() from
* looping forever...
*/
if (mmc->f_min == 0)
_host->write16_hook = NULL;
_host->set_pwr = pdata->set_pwr;
- _host->set_clk_div = pdata->set_clk_div;
ret = tmio_mmc_init_ocr(_host);
if (ret < 0)
mmc->caps & MMC_CAP_NEEDS_POLL ||
!mmc_card_is_removable(mmc));
+ if (!_host->reset)
+ _host->reset = tmio_mmc_reset;
+
/*
* On Gen2+, eMMC with NONREMOVABLE currently fails because native
* hotplug gets disabled. It seems RuntimePM related yet we need further
if (pdata->flags & TMIO_MMC_SDIO_IRQ)
_host->sdio_irq_mask = TMIO_SDIO_MASK_ALL;
- tmio_mmc_clk_stop(_host);
- tmio_mmc_reset(_host);
+ _host->set_clock(_host, 0);
+ _host->reset(_host);
_host->sdcard_irq_mask = sd_ctrl_read16_and_16_as_32(_host, CTL_IRQ_MASK);
tmio_mmc_disable_mmc_irqs(_host, TMIO_MASK_ALL);
tmio_mmc_disable_mmc_irqs(host, TMIO_MASK_ALL);
if (host->clk_cache)
- tmio_mmc_clk_stop(host);
+ host->set_clock(host, 0);
tmio_mmc_clk_disable(host);
{
struct tmio_mmc_host *host = dev_get_drvdata(dev);
- tmio_mmc_reset(host);
+ host->reset(host);
tmio_mmc_clk_enable(host);
if (host->clk_cache)
- tmio_mmc_set_clock(host, host->clk_cache);
+ host->set_clock(host, host->clk_cache);
if (host->native_hotplug)
tmio_mmc_enable_mmc_irqs(host,
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (C) 2017-2018 Socionext Inc.
+// Author: Masahiro Yamada <yamada.masahiro@socionext.com>
+
+#include <linux/bitfield.h>
+#include <linux/bitops.h>
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/mfd/tmio.h>
+#include <linux/mmc/host.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/pinctrl/consumer.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+
+#include "tmio_mmc.h"
+
+#define UNIPHIER_SD_CLK_CTL_DIV1024 BIT(16)
+#define UNIPHIER_SD_CLK_CTL_DIV1 BIT(10)
+#define UNIPHIER_SD_CLKCTL_OFFEN BIT(9) // auto SDCLK stop
+#define UNIPHIER_SD_CC_EXT_MODE 0x1b0
+#define UNIPHIER_SD_CC_EXT_MODE_DMA BIT(1)
+#define UNIPHIER_SD_HOST_MODE 0x1c8
+#define UNIPHIER_SD_VOLT 0x1e4
+#define UNIPHIER_SD_VOLT_MASK GENMASK(1, 0)
+#define UNIPHIER_SD_VOLT_OFF 0
+#define UNIPHIER_SD_VOLT_330 1 // 3.3V signal
+#define UNIPHIER_SD_VOLT_180 2 // 1.8V signal
+#define UNIPHIER_SD_DMA_MODE 0x410
+#define UNIPHIER_SD_DMA_MODE_DIR_MASK GENMASK(17, 16)
+#define UNIPHIER_SD_DMA_MODE_DIR_TO_DEV 0
+#define UNIPHIER_SD_DMA_MODE_DIR_FROM_DEV 1
+#define UNIPHIER_SD_DMA_MODE_WIDTH_MASK GENMASK(5, 4)
+#define UNIPHIER_SD_DMA_MODE_WIDTH_8 0
+#define UNIPHIER_SD_DMA_MODE_WIDTH_16 1
+#define UNIPHIER_SD_DMA_MODE_WIDTH_32 2
+#define UNIPHIER_SD_DMA_MODE_WIDTH_64 3
+#define UNIPHIER_SD_DMA_MODE_ADDR_INC BIT(0) // 1: inc, 0: fixed
+#define UNIPHIER_SD_DMA_CTL 0x414
+#define UNIPHIER_SD_DMA_CTL_START BIT(0) // start DMA (auto cleared)
+#define UNIPHIER_SD_DMA_RST 0x418
+#define UNIPHIER_SD_DMA_RST_CH1 BIT(9)
+#define UNIPHIER_SD_DMA_RST_CH0 BIT(8)
+#define UNIPHIER_SD_DMA_ADDR_L 0x440
+#define UNIPHIER_SD_DMA_ADDR_H 0x444
+
+/*
+ * IP is extended to support various features: built-in DMA engine,
+ * 1/1024 divisor, etc.
+ */
+#define UNIPHIER_SD_CAP_EXTENDED_IP BIT(0)
+/* RX channel of the built-in DMA controller is broken (Pro5) */
+#define UNIPHIER_SD_CAP_BROKEN_DMA_RX BIT(1)
+
+struct uniphier_sd_priv {
+ struct tmio_mmc_data tmio_data;
+ struct pinctrl *pinctrl;
+ struct pinctrl_state *pinstate_default;
+ struct pinctrl_state *pinstate_uhs;
+ struct clk *clk;
+ struct reset_control *rst;
+ struct reset_control *rst_br;
+ struct reset_control *rst_hw;
+ struct dma_chan *chan;
+ enum dma_data_direction dma_dir;
+ unsigned long clk_rate;
+ unsigned long caps;
+};
+
+static void *uniphier_sd_priv(struct tmio_mmc_host *host)
+{
+ return container_of(host->pdata, struct uniphier_sd_priv, tmio_data);
+}
+
+static void uniphier_sd_dma_endisable(struct tmio_mmc_host *host, int enable)
+{
+ sd_ctrl_write16(host, CTL_DMA_ENABLE, enable ? DMA_ENABLE_DMASDRW : 0);
+}
+
+/* external DMA engine */
+static void uniphier_sd_external_dma_issue(unsigned long arg)
+{
+ struct tmio_mmc_host *host = (void *)arg;
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+
+ uniphier_sd_dma_endisable(host, 1);
+ dma_async_issue_pending(priv->chan);
+}
+
+static void uniphier_sd_external_dma_callback(void *param,
+ const struct dmaengine_result *result)
+{
+ struct tmio_mmc_host *host = param;
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+ unsigned long flags;
+
+ dma_unmap_sg(mmc_dev(host->mmc), host->sg_ptr, host->sg_len,
+ priv->dma_dir);
+
+ spin_lock_irqsave(&host->lock, flags);
+
+ if (result->result == DMA_TRANS_NOERROR) {
+ /*
+ * When the external DMA engine is enabled, strangely enough,
+ * the DATAEND flag can be asserted even if the DMA engine has
+ * not been kicked yet. Enable the TMIO_STAT_DATAEND irq only
+ * after we make sure the DMA engine finishes the transfer,
+ * hence, in this callback.
+ */
+ tmio_mmc_enable_mmc_irqs(host, TMIO_STAT_DATAEND);
+ } else {
+ host->data->error = -ETIMEDOUT;
+ tmio_mmc_do_data_irq(host);
+ }
+
+ spin_unlock_irqrestore(&host->lock, flags);
+}
+
+static void uniphier_sd_external_dma_start(struct tmio_mmc_host *host,
+ struct mmc_data *data)
+{
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+ enum dma_transfer_direction dma_tx_dir;
+ struct dma_async_tx_descriptor *desc;
+ dma_cookie_t cookie;
+ int sg_len;
+
+ if (!priv->chan)
+ goto force_pio;
+
+ if (data->flags & MMC_DATA_READ) {
+ priv->dma_dir = DMA_FROM_DEVICE;
+ dma_tx_dir = DMA_DEV_TO_MEM;
+ } else {
+ priv->dma_dir = DMA_TO_DEVICE;
+ dma_tx_dir = DMA_MEM_TO_DEV;
+ }
+
+ sg_len = dma_map_sg(mmc_dev(host->mmc), host->sg_ptr, host->sg_len,
+ priv->dma_dir);
+ if (sg_len == 0)
+ goto force_pio;
+
+ desc = dmaengine_prep_slave_sg(priv->chan, host->sg_ptr, sg_len,
+ dma_tx_dir, DMA_CTRL_ACK);
+ if (!desc)
+ goto unmap_sg;
+
+ desc->callback_result = uniphier_sd_external_dma_callback;
+ desc->callback_param = host;
+
+ cookie = dmaengine_submit(desc);
+ if (cookie < 0)
+ goto unmap_sg;
+
+ host->dma_on = true;
+
+ return;
+
+unmap_sg:
+ dma_unmap_sg(mmc_dev(host->mmc), host->sg_ptr, host->sg_len,
+ priv->dma_dir);
+force_pio:
+ uniphier_sd_dma_endisable(host, 0);
+}
+
+static void uniphier_sd_external_dma_enable(struct tmio_mmc_host *host,
+ bool enable)
+{
+}
+
+static void uniphier_sd_external_dma_request(struct tmio_mmc_host *host,
+ struct tmio_mmc_data *pdata)
+{
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+ struct dma_chan *chan;
+
+ chan = dma_request_chan(mmc_dev(host->mmc), "rx-tx");
+ if (IS_ERR(chan)) {
+ dev_warn(mmc_dev(host->mmc),
+ "failed to request DMA channel. falling back to PIO\n");
+ return; /* just use PIO even for -EPROBE_DEFER */
+ }
+
+ /* this driver uses a single channel for both RX an TX */
+ priv->chan = chan;
+ host->chan_rx = chan;
+ host->chan_tx = chan;
+
+ tasklet_init(&host->dma_issue, uniphier_sd_external_dma_issue,
+ (unsigned long)host);
+}
+
+static void uniphier_sd_external_dma_release(struct tmio_mmc_host *host)
+{
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+
+ if (priv->chan)
+ dma_release_channel(priv->chan);
+}
+
+static void uniphier_sd_external_dma_abort(struct tmio_mmc_host *host)
+{
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+
+ uniphier_sd_dma_endisable(host, 0);
+
+ if (priv->chan)
+ dmaengine_terminate_sync(priv->chan);
+}
+
+static void uniphier_sd_external_dma_dataend(struct tmio_mmc_host *host)
+{
+ uniphier_sd_dma_endisable(host, 0);
+
+ tmio_mmc_do_data_irq(host);
+}
+
+static const struct tmio_mmc_dma_ops uniphier_sd_external_dma_ops = {
+ .start = uniphier_sd_external_dma_start,
+ .enable = uniphier_sd_external_dma_enable,
+ .request = uniphier_sd_external_dma_request,
+ .release = uniphier_sd_external_dma_release,
+ .abort = uniphier_sd_external_dma_abort,
+ .dataend = uniphier_sd_external_dma_dataend,
+};
+
+static void uniphier_sd_internal_dma_issue(unsigned long arg)
+{
+ struct tmio_mmc_host *host = (void *)arg;
+ unsigned long flags;
+
+ spin_lock_irqsave(&host->lock, flags);
+ tmio_mmc_enable_mmc_irqs(host, TMIO_STAT_DATAEND);
+ spin_unlock_irqrestore(&host->lock, flags);
+
+ uniphier_sd_dma_endisable(host, 1);
+ writel(UNIPHIER_SD_DMA_CTL_START, host->ctl + UNIPHIER_SD_DMA_CTL);
+}
+
+static void uniphier_sd_internal_dma_start(struct tmio_mmc_host *host,
+ struct mmc_data *data)
+{
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+ struct scatterlist *sg = host->sg_ptr;
+ dma_addr_t dma_addr;
+ unsigned int dma_mode_dir;
+ u32 dma_mode;
+ int sg_len;
+
+ if ((data->flags & MMC_DATA_READ) && !host->chan_rx)
+ goto force_pio;
+
+ if (WARN_ON(host->sg_len != 1))
+ goto force_pio;
+
+ if (!IS_ALIGNED(sg->offset, 8))
+ goto force_pio;
+
+ if (data->flags & MMC_DATA_READ) {
+ priv->dma_dir = DMA_FROM_DEVICE;
+ dma_mode_dir = UNIPHIER_SD_DMA_MODE_DIR_FROM_DEV;
+ } else {
+ priv->dma_dir = DMA_TO_DEVICE;
+ dma_mode_dir = UNIPHIER_SD_DMA_MODE_DIR_TO_DEV;
+ }
+
+ sg_len = dma_map_sg(mmc_dev(host->mmc), sg, 1, priv->dma_dir);
+ if (sg_len == 0)
+ goto force_pio;
+
+ dma_mode = FIELD_PREP(UNIPHIER_SD_DMA_MODE_DIR_MASK, dma_mode_dir);
+ dma_mode |= FIELD_PREP(UNIPHIER_SD_DMA_MODE_WIDTH_MASK,
+ UNIPHIER_SD_DMA_MODE_WIDTH_64);
+ dma_mode |= UNIPHIER_SD_DMA_MODE_ADDR_INC;
+
+ writel(dma_mode, host->ctl + UNIPHIER_SD_DMA_MODE);
+
+ dma_addr = sg_dma_address(data->sg);
+ writel(lower_32_bits(dma_addr), host->ctl + UNIPHIER_SD_DMA_ADDR_L);
+ writel(upper_32_bits(dma_addr), host->ctl + UNIPHIER_SD_DMA_ADDR_H);
+
+ host->dma_on = true;
+
+ return;
+force_pio:
+ uniphier_sd_dma_endisable(host, 0);
+}
+
+static void uniphier_sd_internal_dma_enable(struct tmio_mmc_host *host,
+ bool enable)
+{
+}
+
+static void uniphier_sd_internal_dma_request(struct tmio_mmc_host *host,
+ struct tmio_mmc_data *pdata)
+{
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+
+ /*
+ * Due to a hardware bug, Pro5 cannot use DMA for RX.
+ * We can still use DMA for TX, but PIO for RX.
+ */
+ if (!(priv->caps & UNIPHIER_SD_CAP_BROKEN_DMA_RX))
+ host->chan_rx = (void *)0xdeadbeaf;
+
+ host->chan_tx = (void *)0xdeadbeaf;
+
+ tasklet_init(&host->dma_issue, uniphier_sd_internal_dma_issue,
+ (unsigned long)host);
+}
+
+static void uniphier_sd_internal_dma_release(struct tmio_mmc_host *host)
+{
+ /* Each value is set to zero to assume "disabling" each DMA */
+ host->chan_rx = NULL;
+ host->chan_tx = NULL;
+}
+
+static void uniphier_sd_internal_dma_abort(struct tmio_mmc_host *host)
+{
+ u32 tmp;
+
+ uniphier_sd_dma_endisable(host, 0);
+
+ tmp = readl(host->ctl + UNIPHIER_SD_DMA_RST);
+ tmp &= ~(UNIPHIER_SD_DMA_RST_CH1 | UNIPHIER_SD_DMA_RST_CH0);
+ writel(tmp, host->ctl + UNIPHIER_SD_DMA_RST);
+
+ tmp |= UNIPHIER_SD_DMA_RST_CH1 | UNIPHIER_SD_DMA_RST_CH0;
+ writel(tmp, host->ctl + UNIPHIER_SD_DMA_RST);
+}
+
+static void uniphier_sd_internal_dma_dataend(struct tmio_mmc_host *host)
+{
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+
+ uniphier_sd_dma_endisable(host, 0);
+ dma_unmap_sg(mmc_dev(host->mmc), host->sg_ptr, 1, priv->dma_dir);
+
+ tmio_mmc_do_data_irq(host);
+}
+
+static const struct tmio_mmc_dma_ops uniphier_sd_internal_dma_ops = {
+ .start = uniphier_sd_internal_dma_start,
+ .enable = uniphier_sd_internal_dma_enable,
+ .request = uniphier_sd_internal_dma_request,
+ .release = uniphier_sd_internal_dma_release,
+ .abort = uniphier_sd_internal_dma_abort,
+ .dataend = uniphier_sd_internal_dma_dataend,
+};
+
+static int uniphier_sd_clk_enable(struct tmio_mmc_host *host)
+{
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+ struct mmc_host *mmc = host->mmc;
+ int ret;
+
+ ret = clk_prepare_enable(priv->clk);
+ if (ret)
+ return ret;
+
+ ret = clk_set_rate(priv->clk, ULONG_MAX);
+ if (ret)
+ goto disable_clk;
+
+ priv->clk_rate = clk_get_rate(priv->clk);
+
+ /* If max-frequency property is set, use it. */
+ if (!mmc->f_max)
+ mmc->f_max = priv->clk_rate;
+
+ /*
+ * 1/512 is the finest divisor in the original IP. Newer versions
+ * also supports 1/1024 divisor. (UniPhier-specific extension)
+ */
+ if (priv->caps & UNIPHIER_SD_CAP_EXTENDED_IP)
+ mmc->f_min = priv->clk_rate / 1024;
+ else
+ mmc->f_min = priv->clk_rate / 512;
+
+ ret = reset_control_deassert(priv->rst);
+ if (ret)
+ goto disable_clk;
+
+ ret = reset_control_deassert(priv->rst_br);
+ if (ret)
+ goto assert_rst;
+
+ return 0;
+
+assert_rst:
+ reset_control_assert(priv->rst);
+disable_clk:
+ clk_disable_unprepare(priv->clk);
+
+ return ret;
+}
+
+static void uniphier_sd_clk_disable(struct tmio_mmc_host *host)
+{
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+
+ reset_control_assert(priv->rst_br);
+ reset_control_assert(priv->rst);
+ clk_disable_unprepare(priv->clk);
+}
+
+static void uniphier_sd_hw_reset(struct tmio_mmc_host *host)
+{
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+
+ reset_control_assert(priv->rst_hw);
+ /* For eMMC, minimum is 1us but give it 9us for good measure */
+ udelay(9);
+ reset_control_deassert(priv->rst_hw);
+ /* For eMMC, minimum is 200us but give it 300us for good measure */
+ usleep_range(300, 1000);
+}
+
+static void uniphier_sd_set_clock(struct tmio_mmc_host *host,
+ unsigned int clock)
+{
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+ unsigned long divisor;
+ u32 tmp;
+
+ tmp = readl(host->ctl + (CTL_SD_CARD_CLK_CTL << 1));
+
+ /* stop the clock before changing its rate to avoid a glitch signal */
+ tmp &= ~CLK_CTL_SCLKEN;
+ writel(tmp, host->ctl + (CTL_SD_CARD_CLK_CTL << 1));
+
+ if (clock == 0)
+ return;
+
+ tmp &= ~UNIPHIER_SD_CLK_CTL_DIV1024;
+ tmp &= ~UNIPHIER_SD_CLK_CTL_DIV1;
+ tmp &= ~CLK_CTL_DIV_MASK;
+
+ divisor = priv->clk_rate / clock;
+
+ /*
+ * In the original IP, bit[7:0] represents the divisor.
+ * bit7 set: 1/512, ... bit0 set:1/4, all bits clear: 1/2
+ *
+ * The IP does not define a way to achieve 1/1. For UniPhier variants,
+ * bit10 is used for 1/1. Newer versions of UniPhier variants use
+ * bit16 for 1/1024.
+ */
+ if (divisor <= 1)
+ tmp |= UNIPHIER_SD_CLK_CTL_DIV1;
+ else if (priv->caps & UNIPHIER_SD_CAP_EXTENDED_IP && divisor > 512)
+ tmp |= UNIPHIER_SD_CLK_CTL_DIV1024;
+ else
+ tmp |= roundup_pow_of_two(divisor) >> 2;
+
+ writel(tmp, host->ctl + (CTL_SD_CARD_CLK_CTL << 1));
+
+ tmp |= CLK_CTL_SCLKEN;
+ writel(tmp, host->ctl + (CTL_SD_CARD_CLK_CTL << 1));
+}
+
+static void uniphier_sd_host_init(struct tmio_mmc_host *host)
+{
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+ u32 val;
+
+ /*
+ * Connected to 32bit AXI.
+ * This register holds settings for SoC-specific internal bus
+ * connection. What is worse, the register spec was changed,
+ * breaking the backward compatibility. Write an appropriate
+ * value depending on a flag associated with a compatible string.
+ */
+ if (priv->caps & UNIPHIER_SD_CAP_EXTENDED_IP)
+ val = 0x00000101;
+ else
+ val = 0x00000000;
+
+ writel(val, host->ctl + UNIPHIER_SD_HOST_MODE);
+
+ val = 0;
+ /*
+ * If supported, the controller can automatically
+ * enable/disable the clock line to the card.
+ */
+ if (priv->caps & UNIPHIER_SD_CAP_EXTENDED_IP)
+ val |= UNIPHIER_SD_CLKCTL_OFFEN;
+
+ writel(val, host->ctl + (CTL_SD_CARD_CLK_CTL << 1));
+}
+
+static int uniphier_sd_start_signal_voltage_switch(struct mmc_host *mmc,
+ struct mmc_ios *ios)
+{
+ struct tmio_mmc_host *host = mmc_priv(mmc);
+ struct uniphier_sd_priv *priv = uniphier_sd_priv(host);
+ struct pinctrl_state *pinstate;
+ u32 val, tmp;
+
+ switch (ios->signal_voltage) {
+ case MMC_SIGNAL_VOLTAGE_330:
+ val = UNIPHIER_SD_VOLT_330;
+ pinstate = priv->pinstate_default;
+ break;
+ case MMC_SIGNAL_VOLTAGE_180:
+ val = UNIPHIER_SD_VOLT_180;
+ pinstate = priv->pinstate_uhs;
+ break;
+ default:
+ return -ENOTSUPP;
+ }
+
+ tmp = readl(host->ctl + UNIPHIER_SD_VOLT);
+ tmp &= ~UNIPHIER_SD_VOLT_MASK;
+ tmp |= FIELD_PREP(UNIPHIER_SD_VOLT_MASK, val);
+ writel(tmp, host->ctl + UNIPHIER_SD_VOLT);
+
+ pinctrl_select_state(priv->pinctrl, pinstate);
+
+ return 0;
+}
+
+static int uniphier_sd_uhs_init(struct tmio_mmc_host *host,
+ struct uniphier_sd_priv *priv)
+{
+ priv->pinctrl = devm_pinctrl_get(mmc_dev(host->mmc));
+ if (IS_ERR(priv->pinctrl))
+ return PTR_ERR(priv->pinctrl);
+
+ priv->pinstate_default = pinctrl_lookup_state(priv->pinctrl,
+ PINCTRL_STATE_DEFAULT);
+ if (IS_ERR(priv->pinstate_default))
+ return PTR_ERR(priv->pinstate_default);
+
+ priv->pinstate_uhs = pinctrl_lookup_state(priv->pinctrl, "uhs");
+ if (IS_ERR(priv->pinstate_uhs))
+ return PTR_ERR(priv->pinstate_uhs);
+
+ host->ops.start_signal_voltage_switch =
+ uniphier_sd_start_signal_voltage_switch;
+
+ return 0;
+}
+
+static int uniphier_sd_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct uniphier_sd_priv *priv;
+ struct tmio_mmc_data *tmio_data;
+ struct tmio_mmc_host *host;
+ int irq, ret;
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0) {
+ dev_err(dev, "failed to get IRQ number");
+ return irq;
+ }
+
+ priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ priv->caps = (unsigned long)of_device_get_match_data(dev);
+
+ priv->clk = devm_clk_get(dev, NULL);
+ if (IS_ERR(priv->clk)) {
+ dev_err(dev, "failed to get clock\n");
+ return PTR_ERR(priv->clk);
+ }
+
+ priv->rst = devm_reset_control_get_shared(dev, "host");
+ if (IS_ERR(priv->rst)) {
+ dev_err(dev, "failed to get host reset\n");
+ return PTR_ERR(priv->rst);
+ }
+
+ /* old version has one more reset */
+ if (!(priv->caps & UNIPHIER_SD_CAP_EXTENDED_IP)) {
+ priv->rst_br = devm_reset_control_get_shared(dev, "bridge");
+ if (IS_ERR(priv->rst_br)) {
+ dev_err(dev, "failed to get bridge reset\n");
+ return PTR_ERR(priv->rst_br);
+ }
+ }
+
+ tmio_data = &priv->tmio_data;
+ tmio_data->flags |= TMIO_MMC_32BIT_DATA_PORT;
+
+ host = tmio_mmc_host_alloc(pdev, tmio_data);
+ if (IS_ERR(host))
+ return PTR_ERR(host);
+
+ if (host->mmc->caps & MMC_CAP_HW_RESET) {
+ priv->rst_hw = devm_reset_control_get_exclusive(dev, "hw");
+ if (IS_ERR(priv->rst_hw)) {
+ dev_err(dev, "failed to get hw reset\n");
+ ret = PTR_ERR(priv->rst_hw);
+ goto free_host;
+ }
+ host->hw_reset = uniphier_sd_hw_reset;
+ }
+
+ if (host->mmc->caps & MMC_CAP_UHS) {
+ ret = uniphier_sd_uhs_init(host, priv);
+ if (ret) {
+ dev_warn(dev,
+ "failed to setup UHS (error %d). Disabling UHS.",
+ ret);
+ host->mmc->caps &= ~MMC_CAP_UHS;
+ }
+ }
+
+ ret = devm_request_irq(dev, irq, tmio_mmc_irq, IRQF_SHARED,
+ dev_name(dev), host);
+ if (ret)
+ goto free_host;
+
+ if (priv->caps & UNIPHIER_SD_CAP_EXTENDED_IP)
+ host->dma_ops = &uniphier_sd_internal_dma_ops;
+ else
+ host->dma_ops = &uniphier_sd_external_dma_ops;
+
+ host->bus_shift = 1;
+ host->clk_enable = uniphier_sd_clk_enable;
+ host->clk_disable = uniphier_sd_clk_disable;
+ host->set_clock = uniphier_sd_set_clock;
+
+ ret = uniphier_sd_clk_enable(host);
+ if (ret)
+ goto free_host;
+
+ uniphier_sd_host_init(host);
+
+ tmio_data->ocr_mask = MMC_VDD_32_33 | MMC_VDD_33_34;
+ if (host->mmc->caps & MMC_CAP_UHS)
+ tmio_data->ocr_mask |= MMC_VDD_165_195;
+
+ tmio_data->max_segs = 1;
+ tmio_data->max_blk_count = U16_MAX;
+
+ ret = tmio_mmc_host_probe(host);
+ if (ret)
+ goto free_host;
+
+ return 0;
+
+free_host:
+ tmio_mmc_host_free(host);
+
+ return ret;
+}
+
+static int uniphier_sd_remove(struct platform_device *pdev)
+{
+ struct tmio_mmc_host *host = platform_get_drvdata(pdev);
+
+ tmio_mmc_host_remove(host);
+ uniphier_sd_clk_disable(host);
+
+ return 0;
+}
+
+static const struct of_device_id uniphier_sd_match[] = {
+ {
+ .compatible = "socionext,uniphier-sd-v2.91",
+ },
+ {
+ .compatible = "socionext,uniphier-sd-v3.1",
+ .data = (void *)(UNIPHIER_SD_CAP_EXTENDED_IP |
+ UNIPHIER_SD_CAP_BROKEN_DMA_RX),
+ },
+ {
+ .compatible = "socionext,uniphier-sd-v3.1.1",
+ .data = (void *)UNIPHIER_SD_CAP_EXTENDED_IP,
+ },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, uniphier_sd_match);
+
+static struct platform_driver uniphier_sd_driver = {
+ .probe = uniphier_sd_probe,
+ .remove = uniphier_sd_remove,
+ .driver = {
+ .name = "uniphier-sd",
+ .of_match_table = uniphier_sd_match,
+ },
+};
+module_platform_driver(uniphier_sd_driver);
+
+MODULE_AUTHOR("Masahiro Yamada <yamada.masahiro@socionext.com>");
+MODULE_DESCRIPTION("UniPhier SD/eMMC host controller driver");
+MODULE_LICENSE("GPL v2");
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2013-2014 Renesas Electronics Europe Ltd.
* Author: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
*/
#include <linux/clk.h>
SPI_MEM_OP_ADDR(nor->addr_width, to, 1),
SPI_MEM_OP_NO_DUMMY,
SPI_MEM_OP_DATA_OUT(len, buf, 1));
- size_t remaining = len;
int ret;
/* get transfer protocols. */
if (nor->program_opcode == SPINOR_OP_AAI_WP && nor->sst_write_second)
op.addr.nbytes = 0;
- while (remaining) {
- op.data.nbytes = remaining < UINT_MAX ? remaining : UINT_MAX;
- ret = spi_mem_adjust_op_size(flash->spimem, &op);
- if (ret)
- return ret;
-
- ret = spi_mem_exec_op(flash->spimem, &op);
- if (ret)
- return ret;
+ ret = spi_mem_adjust_op_size(flash->spimem, &op);
+ if (ret)
+ return ret;
+ op.data.nbytes = len < op.data.nbytes ? len : op.data.nbytes;
- op.addr.val += op.data.nbytes;
- remaining -= op.data.nbytes;
- op.data.buf.out += op.data.nbytes;
- }
+ ret = spi_mem_exec_op(flash->spimem, &op);
+ if (ret)
+ return ret;
- return len;
+ return op.data.nbytes;
}
/*
*/
#include <linux/gpio.h>
+#include <linux/gpio/consumer.h>
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/types.h>
-#define pr_devinit(fmt, args...) \
- ({ static const char __fmt[] = fmt; printk(__fmt, ## args); })
+#define win_mask(x) ((BIT(x)) - 1)
#define DRIVER_NAME "gpio-addr-flash"
-#define PFX DRIVER_NAME ": "
/**
* struct async_state - keep GPIO flash state
* @mtd: MTD state for this mapping
* @map: MTD map state for this flash
- * @gpio_count: number of GPIOs used to address
- * @gpio_addrs: array of GPIOs to twiddle
+ * @gpios: Struct containing the array of GPIO descriptors
* @gpio_values: cached GPIO values
- * @win_size: dedicated memory size (if no GPIOs)
+ * @win_order: dedicated memory size (if no GPIOs)
*/
struct async_state {
struct mtd_info *mtd;
struct map_info map;
- size_t gpio_count;
- unsigned *gpio_addrs;
- int *gpio_values;
- unsigned long win_size;
+ struct gpio_descs *gpios;
+ unsigned int gpio_values;
+ unsigned int win_order;
};
#define gf_map_info_to_state(mi) ((struct async_state *)(mi)->map_priv_1)
*
* Rather than call the GPIO framework every time, cache the last-programmed
* value. This speeds up sequential accesses (which are by far the most common
- * type). We rely on the GPIO framework to treat non-zero value as high so
- * that we don't have to normalize the bits.
+ * type).
*/
static void gf_set_gpios(struct async_state *state, unsigned long ofs)
{
- size_t i = 0;
- int value;
- ofs /= state->win_size;
- do {
- value = ofs & (1 << i);
- if (state->gpio_values[i] != value) {
- gpio_set_value(state->gpio_addrs[i], value);
- state->gpio_values[i] = value;
- }
- } while (++i < state->gpio_count);
+ int i;
+
+ ofs >>= state->win_order;
+
+ if (ofs == state->gpio_values)
+ return;
+
+ for (i = 0; i < state->gpios->ndescs; i++) {
+ if ((ofs & BIT(i)) == (state->gpio_values & BIT(i)))
+ continue;
+
+ gpiod_set_value(state->gpios->desc[i], !!(ofs & BIT(i)));
+ }
+
+ state->gpio_values = ofs;
}
/**
gf_set_gpios(state, ofs);
- word = readw(map->virt + (ofs % state->win_size));
+ word = readw(map->virt + (ofs & win_mask(state->win_order)));
test.x[0] = word;
return test;
}
int this_len;
while (len) {
- if ((from % state->win_size) + len > state->win_size)
- this_len = state->win_size - (from % state->win_size);
- else
- this_len = len;
+ this_len = from & win_mask(state->win_order);
+ this_len = BIT(state->win_order) - this_len;
+ this_len = min_t(int, len, this_len);
gf_set_gpios(state, from);
- memcpy_fromio(to, map->virt + (from % state->win_size),
- this_len);
+ memcpy_fromio(to,
+ map->virt + (from & win_mask(state->win_order)),
+ this_len);
len -= this_len;
from += this_len;
to += this_len;
gf_set_gpios(state, ofs);
d = d1.x[0];
- writew(d, map->virt + (ofs % state->win_size));
+ writew(d, map->virt + (ofs & win_mask(state->win_order)));
}
/**
int this_len;
while (len) {
- if ((to % state->win_size) + len > state->win_size)
- this_len = state->win_size - (to % state->win_size);
- else
- this_len = len;
+ this_len = to & win_mask(state->win_order);
+ this_len = BIT(state->win_order) - this_len;
+ this_len = min_t(int, len, this_len);
gf_set_gpios(state, to);
- memcpy_toio(map->virt + (to % state->win_size), from, len);
+ memcpy_toio(map->virt + (to & win_mask(state->win_order)),
+ from, len);
len -= this_len;
to += this_len;
* The platform resource layout expected looks something like:
* struct mtd_partition partitions[] = { ... };
* struct physmap_flash_data flash_data = { ... };
- * unsigned flash_gpios[] = { GPIO_XX, GPIO_XX, ... };
+ * static struct gpiod_lookup_table addr_flash_gpios = {
+ * .dev_id = "gpio-addr-flash.0",
+ * .table = {
+ * GPIO_LOOKUP_IDX("gpio.0", 15, "addr", 0, GPIO_ACTIVE_HIGH),
+ * GPIO_LOOKUP_IDX("gpio.0", 16, "addr", 1, GPIO_ACTIVE_HIGH),
+ * );
+ * };
+ * gpiod_add_lookup_table(&addr_flash_gpios);
+ *
* struct resource flash_resource[] = {
* {
* .name = "cfi_probe",
* .start = 0x20000000,
* .end = 0x201fffff,
* .flags = IORESOURCE_MEM,
- * }, {
- * .start = (unsigned long)flash_gpios,
- * .end = ARRAY_SIZE(flash_gpios),
- * .flags = IORESOURCE_IRQ,
- * }
+ * },
* };
* struct platform_device flash_device = {
* .name = "gpio-addr-flash",
*/
static int gpio_flash_probe(struct platform_device *pdev)
{
- size_t i, arr_size;
struct physmap_flash_data *pdata;
struct resource *memory;
- struct resource *gpios;
struct async_state *state;
pdata = dev_get_platdata(&pdev->dev);
memory = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- gpios = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
- if (!memory || !gpios || !gpios->end)
+ if (!memory)
return -EINVAL;
- arr_size = sizeof(int) * gpios->end;
- state = kzalloc(sizeof(*state) + arr_size, GFP_KERNEL);
+ state = devm_kzalloc(&pdev->dev, sizeof(*state), GFP_KERNEL);
if (!state)
return -ENOMEM;
- /*
- * We cast start/end to known types in the boards file, so cast
- * away their pointer types here to the known types (gpios->xxx).
- */
- state->gpio_count = gpios->end;
- state->gpio_addrs = (void *)(unsigned long)gpios->start;
- state->gpio_values = (void *)(state + 1);
- state->win_size = resource_size(memory);
- memset(state->gpio_values, 0xff, arr_size);
+ state->gpios = devm_gpiod_get_array(&pdev->dev, "addr", GPIOD_OUT_LOW);
+ if (IS_ERR(state->gpios))
+ return PTR_ERR(state->gpios);
+
+ state->win_order = get_bitmask_order(resource_size(memory)) - 1;
state->map.name = DRIVER_NAME;
state->map.read = gf_read;
state->map.write = gf_write;
state->map.copy_to = gf_copy_to;
state->map.bankwidth = pdata->width;
- state->map.size = state->win_size * (1 << state->gpio_count);
- state->map.virt = ioremap_nocache(memory->start, state->map.size);
- if (!state->map.virt)
- return -ENOMEM;
+ state->map.size = BIT(state->win_order + state->gpios->ndescs);
+ state->map.virt = devm_ioremap_resource(&pdev->dev, memory);
+ if (IS_ERR(state->map.virt))
+ return PTR_ERR(state->map.virt);
state->map.phys = NO_XIP;
state->map.map_priv_1 = (unsigned long)state;
platform_set_drvdata(pdev, state);
- i = 0;
- do {
- if (gpio_request(state->gpio_addrs[i], DRIVER_NAME)) {
- pr_devinit(KERN_ERR PFX "failed to request gpio %d\n",
- state->gpio_addrs[i]);
- while (i--)
- gpio_free(state->gpio_addrs[i]);
- kfree(state);
- return -EBUSY;
- }
- gpio_direction_output(state->gpio_addrs[i], 0);
- } while (++i < state->gpio_count);
-
- pr_devinit(KERN_NOTICE PFX "probing %d-bit flash bus\n",
- state->map.bankwidth * 8);
+ dev_notice(&pdev->dev, "probing %d-bit flash bus\n",
+ state->map.bankwidth * 8);
state->mtd = do_map_probe(memory->name, &state->map);
- if (!state->mtd) {
- for (i = 0; i < state->gpio_count; ++i)
- gpio_free(state->gpio_addrs[i]);
- kfree(state);
+ if (!state->mtd)
return -ENXIO;
- }
state->mtd->dev.parent = &pdev->dev;
mtd_device_parse_register(state->mtd, part_probe_types, NULL,
static int gpio_flash_remove(struct platform_device *pdev)
{
struct async_state *state = platform_get_drvdata(pdev);
- size_t i = 0;
- do {
- gpio_free(state->gpio_addrs[i]);
- } while (++i < state->gpio_count);
+
mtd_device_unregister(state->mtd);
map_destroy(state->mtd);
- kfree(state);
return 0;
}
struct of_flash_list {
struct mtd_info *mtd;
struct map_info map;
- struct resource *res;
};
struct of_flash {
mtd_concat_destroy(info->cmtd);
}
- for (i = 0; i < info->list_size; i++) {
+ for (i = 0; i < info->list_size; i++)
if (info->list[i].mtd)
map_destroy(info->list[i].mtd);
- if (info->list[i].map.virt)
- iounmap(info->list[i].map.virt);
-
- if (info->list[i].res) {
- release_resource(info->list[i].res);
- kfree(info->list[i].res);
- }
- }
return 0;
}
err = -EBUSY;
res_size = resource_size(&res);
- info->list[i].res = request_mem_region(res.start, res_size,
- dev_name(&dev->dev));
- if (!info->list[i].res)
+ info->list[i].map.virt = devm_ioremap_resource(&dev->dev, &res);
+ if (IS_ERR(info->list[i].map.virt)) {
+ err = PTR_ERR(info->list[i].map.virt);
goto err_out;
+ }
err = -ENXIO;
width = of_get_property(dp, "bank-width", NULL);
if (err)
goto err_out;
- err = -ENOMEM;
- info->list[i].map.virt = ioremap(info->list[i].map.phys,
- info->list[i].map.size);
- if (!info->list[i].map.virt) {
- dev_err(&dev->dev, "Failed to ioremap() flash"
- " region\n");
- goto err_out;
- }
-
simple_map_init(&info->list[i].map);
/*
#define FLASH_PARALLEL_HIGH_PIN_CNT (1 << 20) /* else low pin cnt */
-static const struct of_device_id syscon_match[] = {
- { .compatible = "cortina,gemini-syscon" },
- { },
-};
-
int of_flash_probe_gemini(struct platform_device *pdev,
struct device_node *np,
struct map_info *map)
#include <linux/mtd/blktrans.h>
#include <linux/mtd/mtd.h>
#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/blkpg.h>
#include <linux/spinlock.h>
#include <linux/hdreg.h>
dev->disk->private_data = NULL;
blk_cleanup_queue(dev->rq);
+ blk_mq_free_tag_set(dev->tag_set);
+ kfree(dev->tag_set);
put_disk(dev->disk);
list_del(&dev->list);
kfree(dev);
}
EXPORT_SYMBOL_GPL(mtd_blktrans_cease_background);
-static void mtd_blktrans_work(struct work_struct *work)
+static struct request *mtd_next_request(struct mtd_blktrans_dev *dev)
+{
+ struct request *rq;
+
+ rq = list_first_entry_or_null(&dev->rq_list, struct request, queuelist);
+ if (rq) {
+ list_del_init(&rq->queuelist);
+ blk_mq_start_request(rq);
+ return rq;
+ }
+
+ return NULL;
+}
+
+static void mtd_blktrans_work(struct mtd_blktrans_dev *dev)
+ __releases(&dev->queue_lock)
+ __acquires(&dev->queue_lock)
{
- struct mtd_blktrans_dev *dev =
- container_of(work, struct mtd_blktrans_dev, work);
struct mtd_blktrans_ops *tr = dev->tr;
- struct request_queue *rq = dev->rq;
struct request *req = NULL;
int background_done = 0;
- spin_lock_irq(rq->queue_lock);
-
while (1) {
blk_status_t res;
dev->bg_stop = false;
- if (!req && !(req = blk_fetch_request(rq))) {
+ if (!req && !(req = mtd_next_request(dev))) {
if (tr->background && !background_done) {
- spin_unlock_irq(rq->queue_lock);
+ spin_unlock_irq(&dev->queue_lock);
mutex_lock(&dev->lock);
tr->background(dev);
mutex_unlock(&dev->lock);
- spin_lock_irq(rq->queue_lock);
+ spin_lock_irq(&dev->queue_lock);
/*
* Do background processing just once per idle
* period.
break;
}
- spin_unlock_irq(rq->queue_lock);
+ spin_unlock_irq(&dev->queue_lock);
mutex_lock(&dev->lock);
res = do_blktrans_request(dev->tr, dev, req);
mutex_unlock(&dev->lock);
- spin_lock_irq(rq->queue_lock);
-
- if (!__blk_end_request_cur(req, res))
+ if (!blk_update_request(req, res, blk_rq_cur_bytes(req))) {
+ __blk_mq_end_request(req, res);
req = NULL;
+ }
background_done = 0;
+ spin_lock_irq(&dev->queue_lock);
}
-
- spin_unlock_irq(rq->queue_lock);
}
-static void mtd_blktrans_request(struct request_queue *rq)
+static blk_status_t mtd_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
struct mtd_blktrans_dev *dev;
- struct request *req = NULL;
- dev = rq->queuedata;
+ dev = hctx->queue->queuedata;
+ if (!dev) {
+ blk_mq_start_request(bd->rq);
+ return BLK_STS_IOERR;
+ }
+
+ spin_lock_irq(&dev->queue_lock);
+ list_add_tail(&bd->rq->queuelist, &dev->rq_list);
+ mtd_blktrans_work(dev);
+ spin_unlock_irq(&dev->queue_lock);
- if (!dev)
- while ((req = blk_fetch_request(rq)) != NULL)
- __blk_end_request_all(req, BLK_STS_IOERR);
- else
- queue_work(dev->wq, &dev->work);
+ return BLK_STS_OK;
}
static int blktrans_open(struct block_device *bdev, fmode_t mode)
.getgeo = blktrans_getgeo,
};
+static const struct blk_mq_ops mtd_mq_ops = {
+ .queue_rq = mtd_queue_rq,
+};
+
int add_mtd_blktrans_dev(struct mtd_blktrans_dev *new)
{
struct mtd_blktrans_ops *tr = new->tr;
/* Create the request queue */
spin_lock_init(&new->queue_lock);
- new->rq = blk_init_queue(mtd_blktrans_request, &new->queue_lock);
+ INIT_LIST_HEAD(&new->rq_list);
- if (!new->rq)
+ new->tag_set = kzalloc(sizeof(*new->tag_set), GFP_KERNEL);
+ if (!new->tag_set)
goto error3;
+ new->rq = blk_mq_init_sq_queue(new->tag_set, &mtd_mq_ops, 2,
+ BLK_MQ_F_SHOULD_MERGE | BLK_MQ_F_BLOCKING);
+ if (IS_ERR(new->rq)) {
+ ret = PTR_ERR(new->rq);
+ new->rq = NULL;
+ goto error4;
+ }
+
if (tr->flush)
blk_queue_write_cache(new->rq, true, false);
gd->queue = new->rq;
- /* Create processing workqueue */
- new->wq = alloc_workqueue("%s%d", 0, 0,
- tr->name, new->mtd->index);
- if (!new->wq)
- goto error4;
- INIT_WORK(&new->work, mtd_blktrans_work);
-
if (new->readonly)
set_disk_ro(gd, 1);
- device_add_disk(&new->mtd->dev, gd);
+ device_add_disk(&new->mtd->dev, gd, NULL);
if (new->disk_attributes) {
ret = sysfs_create_group(&disk_to_dev(gd)->kobj,
}
return 0;
error4:
- blk_cleanup_queue(new->rq);
+ kfree(new->tag_set);
error3:
put_disk(new->disk);
error2:
/* Stop new requests to arrive */
del_gendisk(old->disk);
- /* Stop workqueue. This will perform any pending request. */
- destroy_workqueue(old->wq);
-
/* Kill current requests */
spin_lock_irqsave(&old->queue_lock, flags);
old->rq->queuedata = NULL;
- blk_start_queue(old->rq);
spin_unlock_irqrestore(&old->queue_lock, flags);
+ /* freeze+quiesce queue to ensure all requests are flushed */
+ blk_mq_freeze_queue(old->rq);
+ blk_mq_quiesce_queue(old->rq);
+ blk_mq_unquiesce_queue(old->rq);
+ blk_mq_unfreeze_queue(old->rq);
+
/* If the device is currently open, tell trans driver to close it,
then put mtd device, and don't touch it again */
mutex_lock(&old->lock);
load time (assuming you build diskonchip as a module) with the module
parameter "inftl_bbt_write=1".
-config MTD_NAND_DOCG4
- tristate "Support for DiskOnChip G4"
- depends on HAS_IOMEM
- select BCH
- select BITREVERSE
- help
- Support for diskonchip G4 nand flash, found in various smartphones and
- PDAs, among them the Palm Treo680, HTC Prophet and Wizard, Toshiba
- Portege G900, Asus P526, and O2 XDA Zinc.
-
- With this driver you will be able to use UBI and create a ubifs on the
- device, so you may wish to consider enabling UBI and UBIFS as well.
-
- These devices ship with the Mys/Sandisk SAFTL formatting, for which
- there is currently no mtd parser, so you may want to use command line
- partitioning to segregate write-protected blocks. On the Treo680, the
- first five erase blocks (256KiB each) are write-protected, followed
- by the block containing the saftl partition table. This is probably
- typical.
-
config MTD_NAND_SHARPSL
tristate "Support for NAND Flash on Sharp SL Series (C7xx + others)"
depends on ARCH_PXA || COMPILE_TEST
obj-$(CONFIG_MTD_NAND_TANGO) += tango_nand.o
obj-$(CONFIG_MTD_NAND_DAVINCI) += davinci_nand.o
obj-$(CONFIG_MTD_NAND_DISKONCHIP) += diskonchip.o
-obj-$(CONFIG_MTD_NAND_DOCG4) += docg4.o
obj-$(CONFIG_MTD_NAND_FSMC) += fsmc_nand.o
obj-$(CONFIG_MTD_NAND_SHARPSL) += sharpsl.o
obj-$(CONFIG_MTD_NAND_NANDSIM) += nandsim.o
obj-$(CONFIG_MTD_NAND_MTK) += mtk_ecc.o mtk_nand.o
obj-$(CONFIG_MTD_NAND_TEGRA) += tegra_nand.o
-nand-objs := nand_base.o nand_bbt.o nand_timings.o nand_ids.o
+nand-objs := nand_base.o nand_legacy.o nand_bbt.o nand_timings.o nand_ids.o
+nand-objs += nand_onfi.o
+nand-objs += nand_jedec.o
nand-objs += nand_amd.o
+nand-objs += nand_esmt.o
nand-objs += nand_hynix.o
nand-objs += nand_macronix.o
nand-objs += nand_micron.o
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/rawnand.h>
#include <linux/mtd/partitions.h>
-#include <linux/gpio.h>
#include <linux/platform_data/gpio-omap.h>
#include <asm/io.h>
#include <asm/sizes.h>
-#include <mach/board-ams-delta.h>
-
#include <mach/hardware.h>
/*
* MTD structure for E3 (Delta)
*/
-static struct mtd_info *ams_delta_mtd = NULL;
+
+struct ams_delta_nand {
+ struct nand_chip nand_chip;
+ struct gpio_desc *gpiod_rdy;
+ struct gpio_desc *gpiod_nce;
+ struct gpio_desc *gpiod_nre;
+ struct gpio_desc *gpiod_nwp;
+ struct gpio_desc *gpiod_nwe;
+ struct gpio_desc *gpiod_ale;
+ struct gpio_desc *gpiod_cle;
+ void __iomem *io_base;
+ bool data_in;
+};
/*
* Define partitions for flash devices
.size = 3 * SZ_256K },
};
-static void ams_delta_write_byte(struct mtd_info *mtd, u_char byte)
+static void ams_delta_io_write(struct ams_delta_nand *priv, u_char byte)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- void __iomem *io_base = (void __iomem *)nand_get_controller_data(this);
-
- writew(0, io_base + OMAP_MPUIO_IO_CNTL);
- writew(byte, this->IO_ADDR_W);
- gpio_set_value(AMS_DELTA_GPIO_PIN_NAND_NWE, 0);
+ writew(byte, priv->nand_chip.legacy.IO_ADDR_W);
+ gpiod_set_value(priv->gpiod_nwe, 0);
ndelay(40);
- gpio_set_value(AMS_DELTA_GPIO_PIN_NAND_NWE, 1);
+ gpiod_set_value(priv->gpiod_nwe, 1);
}
-static u_char ams_delta_read_byte(struct mtd_info *mtd)
+static u_char ams_delta_io_read(struct ams_delta_nand *priv)
{
u_char res;
- struct nand_chip *this = mtd_to_nand(mtd);
- void __iomem *io_base = (void __iomem *)nand_get_controller_data(this);
- gpio_set_value(AMS_DELTA_GPIO_PIN_NAND_NRE, 0);
+ gpiod_set_value(priv->gpiod_nre, 0);
ndelay(40);
- writew(~0, io_base + OMAP_MPUIO_IO_CNTL);
- res = readw(this->IO_ADDR_R);
- gpio_set_value(AMS_DELTA_GPIO_PIN_NAND_NRE, 1);
+ res = readw(priv->nand_chip.legacy.IO_ADDR_R);
+ gpiod_set_value(priv->gpiod_nre, 1);
return res;
}
-static void ams_delta_write_buf(struct mtd_info *mtd, const u_char *buf,
+static void ams_delta_dir_input(struct ams_delta_nand *priv, bool in)
+{
+ writew(in ? ~0 : 0, priv->io_base + OMAP_MPUIO_IO_CNTL);
+ priv->data_in = in;
+}
+
+static void ams_delta_write_buf(struct nand_chip *this, const u_char *buf,
int len)
{
+ struct ams_delta_nand *priv = nand_get_controller_data(this);
int i;
- for (i=0; i<len; i++)
- ams_delta_write_byte(mtd, buf[i]);
+ if (priv->data_in)
+ ams_delta_dir_input(priv, false);
+
+ for (i = 0; i < len; i++)
+ ams_delta_io_write(priv, buf[i]);
}
-static void ams_delta_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void ams_delta_read_buf(struct nand_chip *this, u_char *buf, int len)
{
+ struct ams_delta_nand *priv = nand_get_controller_data(this);
int i;
- for (i=0; i<len; i++)
- buf[i] = ams_delta_read_byte(mtd);
+ if (!priv->data_in)
+ ams_delta_dir_input(priv, true);
+
+ for (i = 0; i < len; i++)
+ buf[i] = ams_delta_io_read(priv);
+}
+
+static u_char ams_delta_read_byte(struct nand_chip *this)
+{
+ u_char res;
+
+ ams_delta_read_buf(this, &res, 1);
+
+ return res;
}
/*
* NAND_CLE: bit 1 -> bit 7
* NAND_ALE: bit 2 -> bit 6
*/
-static void ams_delta_hwcontrol(struct mtd_info *mtd, int cmd,
+static void ams_delta_hwcontrol(struct nand_chip *this, int cmd,
unsigned int ctrl)
{
+ struct ams_delta_nand *priv = nand_get_controller_data(this);
if (ctrl & NAND_CTRL_CHANGE) {
- gpio_set_value(AMS_DELTA_GPIO_PIN_NAND_NCE,
- (ctrl & NAND_NCE) == 0);
- gpio_set_value(AMS_DELTA_GPIO_PIN_NAND_CLE,
- (ctrl & NAND_CLE) != 0);
- gpio_set_value(AMS_DELTA_GPIO_PIN_NAND_ALE,
- (ctrl & NAND_ALE) != 0);
+ gpiod_set_value(priv->gpiod_nce, !(ctrl & NAND_NCE));
+ gpiod_set_value(priv->gpiod_cle, !!(ctrl & NAND_CLE));
+ gpiod_set_value(priv->gpiod_ale, !!(ctrl & NAND_ALE));
}
- if (cmd != NAND_CMD_NONE)
- ams_delta_write_byte(mtd, cmd);
+ if (cmd != NAND_CMD_NONE) {
+ u_char byte = cmd;
+
+ ams_delta_write_buf(this, &byte, 1);
+ }
}
-static int ams_delta_nand_ready(struct mtd_info *mtd)
+static int ams_delta_nand_ready(struct nand_chip *this)
{
- return gpio_get_value(AMS_DELTA_GPIO_PIN_NAND_RB);
+ struct ams_delta_nand *priv = nand_get_controller_data(this);
+
+ return gpiod_get_value(priv->gpiod_rdy);
}
-static const struct gpio _mandatory_gpio[] = {
- {
- .gpio = AMS_DELTA_GPIO_PIN_NAND_NCE,
- .flags = GPIOF_OUT_INIT_HIGH,
- .label = "nand_nce",
- },
- {
- .gpio = AMS_DELTA_GPIO_PIN_NAND_NRE,
- .flags = GPIOF_OUT_INIT_HIGH,
- .label = "nand_nre",
- },
- {
- .gpio = AMS_DELTA_GPIO_PIN_NAND_NWP,
- .flags = GPIOF_OUT_INIT_HIGH,
- .label = "nand_nwp",
- },
- {
- .gpio = AMS_DELTA_GPIO_PIN_NAND_NWE,
- .flags = GPIOF_OUT_INIT_HIGH,
- .label = "nand_nwe",
- },
- {
- .gpio = AMS_DELTA_GPIO_PIN_NAND_ALE,
- .flags = GPIOF_OUT_INIT_LOW,
- .label = "nand_ale",
- },
- {
- .gpio = AMS_DELTA_GPIO_PIN_NAND_CLE,
- .flags = GPIOF_OUT_INIT_LOW,
- .label = "nand_cle",
- },
-};
/*
* Main initialization routine
*/
static int ams_delta_init(struct platform_device *pdev)
{
+ struct ams_delta_nand *priv;
struct nand_chip *this;
+ struct mtd_info *mtd;
struct resource *res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
void __iomem *io_base;
int err = 0;
return -ENXIO;
/* Allocate memory for MTD device structure and private data */
- this = kzalloc(sizeof(struct nand_chip), GFP_KERNEL);
- if (!this) {
+ priv = devm_kzalloc(&pdev->dev, sizeof(struct ams_delta_nand),
+ GFP_KERNEL);
+ if (!priv) {
pr_warn("Unable to allocate E3 NAND MTD device structure.\n");
- err = -ENOMEM;
- goto out;
+ return -ENOMEM;
}
+ this = &priv->nand_chip;
- ams_delta_mtd = nand_to_mtd(this);
- ams_delta_mtd->owner = THIS_MODULE;
+ mtd = nand_to_mtd(this);
+ mtd->dev.parent = &pdev->dev;
/*
* Don't try to request the memory region from here,
goto out_free;
}
- nand_set_controller_data(this, (void *)io_base);
+ priv->io_base = io_base;
+ nand_set_controller_data(this, priv);
/* Set address of NAND IO lines */
- this->IO_ADDR_R = io_base + OMAP_MPUIO_INPUT_LATCH;
- this->IO_ADDR_W = io_base + OMAP_MPUIO_OUTPUT;
- this->read_byte = ams_delta_read_byte;
- this->write_buf = ams_delta_write_buf;
- this->read_buf = ams_delta_read_buf;
- this->cmd_ctrl = ams_delta_hwcontrol;
- if (gpio_request(AMS_DELTA_GPIO_PIN_NAND_RB, "nand_rdy") == 0) {
- this->dev_ready = ams_delta_nand_ready;
- } else {
- this->dev_ready = NULL;
- pr_notice("Couldn't request gpio for Delta NAND ready.\n");
+ this->legacy.IO_ADDR_R = io_base + OMAP_MPUIO_INPUT_LATCH;
+ this->legacy.IO_ADDR_W = io_base + OMAP_MPUIO_OUTPUT;
+ this->legacy.read_byte = ams_delta_read_byte;
+ this->legacy.write_buf = ams_delta_write_buf;
+ this->legacy.read_buf = ams_delta_read_buf;
+ this->legacy.cmd_ctrl = ams_delta_hwcontrol;
+
+ priv->gpiod_rdy = devm_gpiod_get_optional(&pdev->dev, "rdy", GPIOD_IN);
+ if (IS_ERR(priv->gpiod_rdy)) {
+ err = PTR_ERR(priv->gpiod_rdy);
+ dev_warn(&pdev->dev, "RDY GPIO request failed (%d)\n", err);
+ goto out_mtd;
}
+
+ if (priv->gpiod_rdy)
+ this->legacy.dev_ready = ams_delta_nand_ready;
+
/* 25 us command delay time */
- this->chip_delay = 30;
+ this->legacy.chip_delay = 30;
this->ecc.mode = NAND_ECC_SOFT;
this->ecc.algo = NAND_ECC_HAMMING;
- platform_set_drvdata(pdev, io_base);
+ platform_set_drvdata(pdev, priv);
/* Set chip enabled, but */
- err = gpio_request_array(_mandatory_gpio, ARRAY_SIZE(_mandatory_gpio));
- if (err)
- goto out_gpio;
+ priv->gpiod_nwp = devm_gpiod_get(&pdev->dev, "nwp", GPIOD_OUT_HIGH);
+ if (IS_ERR(priv->gpiod_nwp)) {
+ err = PTR_ERR(priv->gpiod_nwp);
+ dev_err(&pdev->dev, "NWP GPIO request failed (%d)\n", err);
+ goto out_mtd;
+ }
+
+ priv->gpiod_nce = devm_gpiod_get(&pdev->dev, "nce", GPIOD_OUT_HIGH);
+ if (IS_ERR(priv->gpiod_nce)) {
+ err = PTR_ERR(priv->gpiod_nce);
+ dev_err(&pdev->dev, "NCE GPIO request failed (%d)\n", err);
+ goto out_mtd;
+ }
+
+ priv->gpiod_nre = devm_gpiod_get(&pdev->dev, "nre", GPIOD_OUT_HIGH);
+ if (IS_ERR(priv->gpiod_nre)) {
+ err = PTR_ERR(priv->gpiod_nre);
+ dev_err(&pdev->dev, "NRE GPIO request failed (%d)\n", err);
+ goto out_mtd;
+ }
+
+ priv->gpiod_nwe = devm_gpiod_get(&pdev->dev, "nwe", GPIOD_OUT_HIGH);
+ if (IS_ERR(priv->gpiod_nwe)) {
+ err = PTR_ERR(priv->gpiod_nwe);
+ dev_err(&pdev->dev, "NWE GPIO request failed (%d)\n", err);
+ goto out_mtd;
+ }
+
+ priv->gpiod_ale = devm_gpiod_get(&pdev->dev, "ale", GPIOD_OUT_LOW);
+ if (IS_ERR(priv->gpiod_ale)) {
+ err = PTR_ERR(priv->gpiod_ale);
+ dev_err(&pdev->dev, "ALE GPIO request failed (%d)\n", err);
+ goto out_mtd;
+ }
+
+ priv->gpiod_cle = devm_gpiod_get(&pdev->dev, "cle", GPIOD_OUT_LOW);
+ if (IS_ERR(priv->gpiod_cle)) {
+ err = PTR_ERR(priv->gpiod_cle);
+ dev_err(&pdev->dev, "CLE GPIO request failed (%d)\n", err);
+ goto out_mtd;
+ }
+
+ /* Initialize data port direction to a known state */
+ ams_delta_dir_input(priv, true);
/* Scan to find existence of the device */
- err = nand_scan(ams_delta_mtd, 1);
+ err = nand_scan(this, 1);
if (err)
goto out_mtd;
/* Register the partitions */
- mtd_device_register(ams_delta_mtd, partition_info,
- ARRAY_SIZE(partition_info));
+ mtd_device_register(mtd, partition_info, ARRAY_SIZE(partition_info));
goto out;
out_mtd:
- gpio_free_array(_mandatory_gpio, ARRAY_SIZE(_mandatory_gpio));
-out_gpio:
- gpio_free(AMS_DELTA_GPIO_PIN_NAND_RB);
iounmap(io_base);
out_free:
- kfree(this);
out:
return err;
}
*/
static int ams_delta_cleanup(struct platform_device *pdev)
{
- void __iomem *io_base = platform_get_drvdata(pdev);
+ struct ams_delta_nand *priv = platform_get_drvdata(pdev);
+ struct mtd_info *mtd = nand_to_mtd(&priv->nand_chip);
+ void __iomem *io_base = priv->io_base;
/* Release resources, unregister device */
- nand_release(ams_delta_mtd);
+ nand_release(mtd_to_nand(mtd));
- gpio_free_array(_mandatory_gpio, ARRAY_SIZE(_mandatory_gpio));
- gpio_free(AMS_DELTA_GPIO_PIN_NAND_RB);
iounmap(io_base);
- /* Free the MTD device structure */
- kfree(mtd_to_nand(ams_delta_mtd));
-
return 0;
}
return -EIO;
}
-static u8 atmel_nand_read_byte(struct mtd_info *mtd)
+static u8 atmel_nand_read_byte(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct atmel_nand *nand = to_atmel_nand(chip);
return ioread8(nand->activecs->io.virt);
}
-static u16 atmel_nand_read_word(struct mtd_info *mtd)
+static void atmel_nand_write_byte(struct nand_chip *chip, u8 byte)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
- struct atmel_nand *nand = to_atmel_nand(chip);
-
- return ioread16(nand->activecs->io.virt);
-}
-
-static void atmel_nand_write_byte(struct mtd_info *mtd, u8 byte)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct atmel_nand *nand = to_atmel_nand(chip);
if (chip->options & NAND_BUSWIDTH_16)
iowrite8(byte, nand->activecs->io.virt);
}
-static void atmel_nand_read_buf(struct mtd_info *mtd, u8 *buf, int len)
+static void atmel_nand_read_buf(struct nand_chip *chip, u8 *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct atmel_nand *nand = to_atmel_nand(chip);
struct atmel_nand_controller *nc;
ioread8_rep(nand->activecs->io.virt, buf, len);
}
-static void atmel_nand_write_buf(struct mtd_info *mtd, const u8 *buf, int len)
+static void atmel_nand_write_buf(struct nand_chip *chip, const u8 *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct atmel_nand *nand = to_atmel_nand(chip);
struct atmel_nand_controller *nc;
iowrite8_rep(nand->activecs->io.virt, buf, len);
}
-static int atmel_nand_dev_ready(struct mtd_info *mtd)
+static int atmel_nand_dev_ready(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct atmel_nand *nand = to_atmel_nand(chip);
return gpiod_get_value(nand->activecs->rb.gpio);
}
-static void atmel_nand_select_chip(struct mtd_info *mtd, int cs)
+static void atmel_nand_select_chip(struct nand_chip *chip, int cs)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct atmel_nand *nand = to_atmel_nand(chip);
if (cs < 0 || cs >= nand->numcs) {
nand->activecs = NULL;
- chip->dev_ready = NULL;
+ chip->legacy.dev_ready = NULL;
return;
}
nand->activecs = &nand->cs[cs];
if (nand->activecs->rb.type == ATMEL_NAND_GPIO_RB)
- chip->dev_ready = atmel_nand_dev_ready;
+ chip->legacy.dev_ready = atmel_nand_dev_ready;
}
-static int atmel_hsmc_nand_dev_ready(struct mtd_info *mtd)
+static int atmel_hsmc_nand_dev_ready(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct atmel_nand *nand = to_atmel_nand(chip);
struct atmel_hsmc_nand_controller *nc;
u32 status;
return status & ATMEL_HSMC_NFC_SR_RBEDGE(nand->activecs->rb.id);
}
-static void atmel_hsmc_nand_select_chip(struct mtd_info *mtd, int cs)
+static void atmel_hsmc_nand_select_chip(struct nand_chip *chip, int cs)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct atmel_nand *nand = to_atmel_nand(chip);
struct atmel_hsmc_nand_controller *nc;
nc = to_hsmc_nand_controller(chip->controller);
- atmel_nand_select_chip(mtd, cs);
+ atmel_nand_select_chip(chip, cs);
if (!nand->activecs) {
regmap_write(nc->base.smc, ATMEL_HSMC_NFC_CTRL,
}
if (nand->activecs->rb.type == ATMEL_NAND_NATIVE_RB)
- chip->dev_ready = atmel_hsmc_nand_dev_ready;
+ chip->legacy.dev_ready = atmel_hsmc_nand_dev_ready;
regmap_update_bits(nc->base.smc, ATMEL_HSMC_NFC_CFG,
ATMEL_HSMC_NFC_CFG_PAGESIZE_MASK |
return ret;
}
-static void atmel_hsmc_nand_cmd_ctrl(struct mtd_info *mtd, int dat,
+static void atmel_hsmc_nand_cmd_ctrl(struct nand_chip *chip, int dat,
unsigned int ctrl)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct atmel_nand *nand = to_atmel_nand(chip);
struct atmel_hsmc_nand_controller *nc;
}
}
-static void atmel_nand_cmd_ctrl(struct mtd_info *mtd, int cmd,
+static void atmel_nand_cmd_ctrl(struct nand_chip *chip, int cmd,
unsigned int ctrl)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct atmel_nand *nand = to_atmel_nand(chip);
struct atmel_nand_controller *nc;
if (ret)
return ret;
- atmel_nand_write_buf(mtd, buf, mtd->writesize);
+ atmel_nand_write_buf(chip, buf, mtd->writesize);
ret = atmel_nand_pmecc_generate_eccbytes(chip, raw);
if (ret) {
atmel_nand_pmecc_disable(chip, raw);
- atmel_nand_write_buf(mtd, chip->oob_poi, mtd->oobsize);
+ atmel_nand_write_buf(chip, chip->oob_poi, mtd->oobsize);
return nand_prog_page_end_op(chip);
}
-static int atmel_nand_pmecc_write_page(struct mtd_info *mtd,
- struct nand_chip *chip, const u8 *buf,
+static int atmel_nand_pmecc_write_page(struct nand_chip *chip, const u8 *buf,
int oob_required, int page)
{
return atmel_nand_pmecc_write_pg(chip, buf, oob_required, page, false);
}
-static int atmel_nand_pmecc_write_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int atmel_nand_pmecc_write_page_raw(struct nand_chip *chip,
const u8 *buf, int oob_required,
int page)
{
if (ret)
return ret;
- atmel_nand_read_buf(mtd, buf, mtd->writesize);
- atmel_nand_read_buf(mtd, chip->oob_poi, mtd->oobsize);
+ atmel_nand_read_buf(chip, buf, mtd->writesize);
+ atmel_nand_read_buf(chip, chip->oob_poi, mtd->oobsize);
ret = atmel_nand_pmecc_correct_data(chip, buf, raw);
return ret;
}
-static int atmel_nand_pmecc_read_page(struct mtd_info *mtd,
- struct nand_chip *chip, u8 *buf,
+static int atmel_nand_pmecc_read_page(struct nand_chip *chip, u8 *buf,
int oob_required, int page)
{
return atmel_nand_pmecc_read_pg(chip, buf, oob_required, page, false);
}
-static int atmel_nand_pmecc_read_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip, u8 *buf,
+static int atmel_nand_pmecc_read_page_raw(struct nand_chip *chip, u8 *buf,
int oob_required, int page)
{
return atmel_nand_pmecc_read_pg(chip, buf, oob_required, page, true);
if (ret)
return ret;
- atmel_nand_write_buf(mtd, chip->oob_poi, mtd->oobsize);
+ atmel_nand_write_buf(chip, chip->oob_poi, mtd->oobsize);
nc->op.cmds[0] = NAND_CMD_PAGEPROG;
nc->op.ncmds = 1;
dev_err(nc->base.dev, "Failed to program NAND page (err = %d)\n",
ret);
- status = chip->waitfunc(mtd, chip);
+ status = chip->legacy.waitfunc(chip);
if (status & NAND_STATUS_FAIL)
return -EIO;
return ret;
}
-static int atmel_hsmc_nand_pmecc_write_page(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int atmel_hsmc_nand_pmecc_write_page(struct nand_chip *chip,
const u8 *buf, int oob_required,
int page)
{
false);
}
-static int atmel_hsmc_nand_pmecc_write_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int atmel_hsmc_nand_pmecc_write_page_raw(struct nand_chip *chip,
const u8 *buf,
int oob_required, int page)
{
return ret;
}
-static int atmel_hsmc_nand_pmecc_read_page(struct mtd_info *mtd,
- struct nand_chip *chip, u8 *buf,
+static int atmel_hsmc_nand_pmecc_read_page(struct nand_chip *chip, u8 *buf,
int oob_required, int page)
{
return atmel_hsmc_nand_pmecc_read_pg(chip, buf, oob_required, page,
false);
}
-static int atmel_hsmc_nand_pmecc_read_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int atmel_hsmc_nand_pmecc_read_page_raw(struct nand_chip *chip,
u8 *buf, int oob_required,
int page)
{
return 0;
}
-static int atmel_nand_setup_data_interface(struct mtd_info *mtd, int csline,
+static int atmel_nand_setup_data_interface(struct nand_chip *chip, int csline,
const struct nand_data_interface *conf)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct atmel_nand *nand = to_atmel_nand(chip);
struct atmel_nand_controller *nc;
mtd->dev.parent = nc->dev;
nand->base.controller = &nc->base;
- chip->cmd_ctrl = atmel_nand_cmd_ctrl;
- chip->read_byte = atmel_nand_read_byte;
- chip->read_word = atmel_nand_read_word;
- chip->write_byte = atmel_nand_write_byte;
- chip->read_buf = atmel_nand_read_buf;
- chip->write_buf = atmel_nand_write_buf;
+ chip->legacy.cmd_ctrl = atmel_nand_cmd_ctrl;
+ chip->legacy.read_byte = atmel_nand_read_byte;
+ chip->legacy.write_byte = atmel_nand_write_byte;
+ chip->legacy.read_buf = atmel_nand_read_buf;
+ chip->legacy.write_buf = atmel_nand_write_buf;
chip->select_chip = atmel_nand_select_chip;
if (nc->mck && nc->caps->ops->setup_data_interface)
chip->setup_data_interface = atmel_nand_setup_data_interface;
/* Some NANDs require a longer delay than the default one (20us). */
- chip->chip_delay = 40;
+ chip->legacy.chip_delay = 40;
/*
* Use a bounce buffer when the buffer passed by the MTD user is not
atmel_nand_init(nc, nand);
/* Overload some methods for the HSMC controller. */
- chip->cmd_ctrl = atmel_hsmc_nand_cmd_ctrl;
+ chip->legacy.cmd_ctrl = atmel_hsmc_nand_cmd_ctrl;
chip->select_chip = atmel_hsmc_nand_select_chip;
}
return ERR_PTR(-EINVAL);
}
- nand = devm_kzalloc(nc->dev,
- sizeof(*nand) + (numcs * sizeof(*nand->cs)),
- GFP_KERNEL);
+ nand = devm_kzalloc(nc->dev, struct_size(nand, cs, numcs), GFP_KERNEL);
if (!nand) {
dev_err(nc->dev, "Failed to allocate NAND object\n");
return ERR_PTR(-ENOMEM);
nc->caps->ops->nand_init(nc, nand);
- ret = nand_scan(mtd, nand->numcs);
+ ret = nand_scan(chip, nand->numcs);
if (ret) {
dev_err(nc->dev, "NAND scan failed: %d\n", ret);
return ret;
nand_np = dev->of_node;
nfc_np = of_find_compatible_node(dev->of_node, NULL,
"atmel,sama5d3-nfc");
+ if (!nfc_np) {
+ dev_err(dev, "Could not find device node for sama5d3-nfc\n");
+ return -ENODEV;
+ }
nc->clk = of_clk_get(nfc_np, 0);
if (IS_ERR(nc->clk)) {
int cs;
void __iomem *base;
- void (*write_byte)(struct mtd_info *, u_char);
+ void (*write_byte)(struct nand_chip *, u_char);
};
/**
* au_read_byte - read one byte from the chip
- * @mtd: MTD device structure
+ * @this: NAND chip object
*
* read function for 8bit buswidth
*/
-static u_char au_read_byte(struct mtd_info *mtd)
+static u_char au_read_byte(struct nand_chip *this)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- u_char ret = readb(this->IO_ADDR_R);
+ u_char ret = readb(this->legacy.IO_ADDR_R);
wmb(); /* drain writebuffer */
return ret;
}
/**
* au_write_byte - write one byte to the chip
- * @mtd: MTD device structure
+ * @this: NAND chip object
* @byte: pointer to data byte to write
*
* write function for 8it buswidth
*/
-static void au_write_byte(struct mtd_info *mtd, u_char byte)
+static void au_write_byte(struct nand_chip *this, u_char byte)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- writeb(byte, this->IO_ADDR_W);
+ writeb(byte, this->legacy.IO_ADDR_W);
wmb(); /* drain writebuffer */
}
/**
* au_read_byte16 - read one byte endianness aware from the chip
- * @mtd: MTD device structure
+ * @this: NAND chip object
*
* read function for 16bit buswidth with endianness conversion
*/
-static u_char au_read_byte16(struct mtd_info *mtd)
+static u_char au_read_byte16(struct nand_chip *this)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- u_char ret = (u_char) cpu_to_le16(readw(this->IO_ADDR_R));
+ u_char ret = (u_char) cpu_to_le16(readw(this->legacy.IO_ADDR_R));
wmb(); /* drain writebuffer */
return ret;
}
/**
* au_write_byte16 - write one byte endianness aware to the chip
- * @mtd: MTD device structure
+ * @this: NAND chip object
* @byte: pointer to data byte to write
*
* write function for 16bit buswidth with endianness conversion
*/
-static void au_write_byte16(struct mtd_info *mtd, u_char byte)
+static void au_write_byte16(struct nand_chip *this, u_char byte)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- writew(le16_to_cpu((u16) byte), this->IO_ADDR_W);
+ writew(le16_to_cpu((u16) byte), this->legacy.IO_ADDR_W);
wmb(); /* drain writebuffer */
}
-/**
- * au_read_word - read one word from the chip
- * @mtd: MTD device structure
- *
- * read function for 16bit buswidth without endianness conversion
- */
-static u16 au_read_word(struct mtd_info *mtd)
-{
- struct nand_chip *this = mtd_to_nand(mtd);
- u16 ret = readw(this->IO_ADDR_R);
- wmb(); /* drain writebuffer */
- return ret;
-}
-
/**
* au_write_buf - write buffer to chip
- * @mtd: MTD device structure
+ * @this: NAND chip object
* @buf: data buffer
* @len: number of bytes to write
*
* write function for 8bit buswidth
*/
-static void au_write_buf(struct mtd_info *mtd, const u_char *buf, int len)
+static void au_write_buf(struct nand_chip *this, const u_char *buf, int len)
{
int i;
- struct nand_chip *this = mtd_to_nand(mtd);
for (i = 0; i < len; i++) {
- writeb(buf[i], this->IO_ADDR_W);
+ writeb(buf[i], this->legacy.IO_ADDR_W);
wmb(); /* drain writebuffer */
}
}
/**
* au_read_buf - read chip data into buffer
- * @mtd: MTD device structure
+ * @this: NAND chip object
* @buf: buffer to store date
* @len: number of bytes to read
*
* read function for 8bit buswidth
*/
-static void au_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void au_read_buf(struct nand_chip *this, u_char *buf, int len)
{
int i;
- struct nand_chip *this = mtd_to_nand(mtd);
for (i = 0; i < len; i++) {
- buf[i] = readb(this->IO_ADDR_R);
+ buf[i] = readb(this->legacy.IO_ADDR_R);
wmb(); /* drain writebuffer */
}
}
/**
* au_write_buf16 - write buffer to chip
- * @mtd: MTD device structure
+ * @this: NAND chip object
* @buf: data buffer
* @len: number of bytes to write
*
* write function for 16bit buswidth
*/
-static void au_write_buf16(struct mtd_info *mtd, const u_char *buf, int len)
+static void au_write_buf16(struct nand_chip *this, const u_char *buf, int len)
{
int i;
- struct nand_chip *this = mtd_to_nand(mtd);
u16 *p = (u16 *) buf;
len >>= 1;
for (i = 0; i < len; i++) {
- writew(p[i], this->IO_ADDR_W);
+ writew(p[i], this->legacy.IO_ADDR_W);
wmb(); /* drain writebuffer */
}
len >>= 1;
for (i = 0; i < len; i++) {
- p[i] = readw(this->IO_ADDR_R);
+ p[i] = readw(this->legacy.IO_ADDR_R);
wmb(); /* drain writebuffer */
}
}
switch (cmd) {
case NAND_CTL_SETCLE:
- this->IO_ADDR_W = ctx->base + MEM_STNAND_CMD;
+ this->legacy.IO_ADDR_W = ctx->base + MEM_STNAND_CMD;
break;
case NAND_CTL_CLRCLE:
- this->IO_ADDR_W = ctx->base + MEM_STNAND_DATA;
+ this->legacy.IO_ADDR_W = ctx->base + MEM_STNAND_DATA;
break;
case NAND_CTL_SETALE:
- this->IO_ADDR_W = ctx->base + MEM_STNAND_ADDR;
+ this->legacy.IO_ADDR_W = ctx->base + MEM_STNAND_ADDR;
break;
case NAND_CTL_CLRALE:
- this->IO_ADDR_W = ctx->base + MEM_STNAND_DATA;
+ this->legacy.IO_ADDR_W = ctx->base + MEM_STNAND_DATA;
/* FIXME: Nobody knows why this is necessary,
* but it works only that way */
udelay(1);
break;
}
- this->IO_ADDR_R = this->IO_ADDR_W;
+ this->legacy.IO_ADDR_R = this->legacy.IO_ADDR_W;
wmb(); /* Drain the writebuffer */
}
-int au1550_device_ready(struct mtd_info *mtd)
+int au1550_device_ready(struct nand_chip *this)
{
return (alchemy_rdsmem(AU1000_MEM_STSTAT) & 0x1) ? 1 : 0;
}
* chip needs it to be asserted during chip not ready time but the NAND
* controller keeps it released.
*
- * @mtd: MTD device structure
+ * @this: NAND chip object
* @chip: chipnumber to select, -1 for deselect
*/
-static void au1550_select_chip(struct mtd_info *mtd, int chip)
+static void au1550_select_chip(struct nand_chip *this, int chip)
{
}
/**
* au1550_command - Send command to NAND device
- * @mtd: MTD device structure
+ * @this: NAND chip object
* @command: the command to be sent
* @column: the column address for this command, -1 if none
* @page_addr: the page address for this command, -1 if none
*/
-static void au1550_command(struct mtd_info *mtd, unsigned command, int column, int page_addr)
+static void au1550_command(struct nand_chip *this, unsigned command,
+ int column, int page_addr)
{
- struct nand_chip *this = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(this);
struct au1550nd_ctx *ctx = container_of(this, struct au1550nd_ctx,
chip);
int ce_override = 0, i;
column -= 256;
readcmd = NAND_CMD_READ1;
}
- ctx->write_byte(mtd, readcmd);
+ ctx->write_byte(this, readcmd);
}
- ctx->write_byte(mtd, command);
+ ctx->write_byte(this, command);
/* Set ALE and clear CLE to start address cycle */
au1550_hwcontrol(mtd, NAND_CTL_CLRCLE);
if (this->options & NAND_BUSWIDTH_16 &&
!nand_opcode_8bits(command))
column >>= 1;
- ctx->write_byte(mtd, column);
+ ctx->write_byte(this, column);
}
if (page_addr != -1) {
- ctx->write_byte(mtd, (u8)(page_addr & 0xff));
+ ctx->write_byte(this, (u8)(page_addr & 0xff));
if (command == NAND_CMD_READ0 ||
command == NAND_CMD_READ1 ||
au1550_hwcontrol(mtd, NAND_CTL_SETNCE);
}
- ctx->write_byte(mtd, (u8)(page_addr >> 8));
+ ctx->write_byte(this, (u8)(page_addr >> 8));
if (this->options & NAND_ROW_ADDR_3)
- ctx->write_byte(mtd,
+ ctx->write_byte(this,
((page_addr >> 16) & 0x0f));
}
/* Latch in address */
/* Apply a short delay always to ensure that we do wait tWB. */
ndelay(100);
/* Wait for a chip to become ready... */
- for (i = this->chip_delay; !this->dev_ready(mtd) && i > 0; --i)
+ for (i = this->legacy.chip_delay;
+ !this->legacy.dev_ready(this) && i > 0; --i)
udelay(1);
/* Release -CE and re-enable interrupts. */
/* Apply this short delay always to ensure that we do wait tWB. */
ndelay(100);
- while(!this->dev_ready(mtd));
+ while(!this->legacy.dev_ready(this));
}
static int find_nand_cs(unsigned long nand_base)
}
ctx->cs = cs;
- this->dev_ready = au1550_device_ready;
+ this->legacy.dev_ready = au1550_device_ready;
this->select_chip = au1550_select_chip;
- this->cmdfunc = au1550_command;
+ this->legacy.cmdfunc = au1550_command;
/* 30 us command delay time */
- this->chip_delay = 30;
+ this->legacy.chip_delay = 30;
this->ecc.mode = NAND_ECC_SOFT;
this->ecc.algo = NAND_ECC_HAMMING;
if (pd->devwidth)
this->options |= NAND_BUSWIDTH_16;
- this->read_byte = (pd->devwidth) ? au_read_byte16 : au_read_byte;
+ this->legacy.read_byte = (pd->devwidth) ? au_read_byte16 : au_read_byte;
ctx->write_byte = (pd->devwidth) ? au_write_byte16 : au_write_byte;
- this->read_word = au_read_word;
- this->write_buf = (pd->devwidth) ? au_write_buf16 : au_write_buf;
- this->read_buf = (pd->devwidth) ? au_read_buf16 : au_read_buf;
+ this->legacy.write_buf = (pd->devwidth) ? au_write_buf16 : au_write_buf;
+ this->legacy.read_buf = (pd->devwidth) ? au_read_buf16 : au_read_buf;
- ret = nand_scan(mtd, 1);
+ ret = nand_scan(this, 1);
if (ret) {
dev_err(&pdev->dev, "NAND scan failed with %d\n", ret);
goto out3;
struct au1550nd_ctx *ctx = platform_get_drvdata(pdev);
struct resource *r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- nand_release(nand_to_mtd(&ctx->chip));
+ nand_release(&ctx->chip);
iounmap(ctx->base);
release_mem_region(r->start, 0x1000);
kfree(ctx);
{
struct bcm47xxnflash *nflash = platform_get_drvdata(pdev);
- nand_release(nand_to_mtd(&nflash->nand_chip));
+ nand_release(&nflash->nand_chip);
return 0;
}
* NAND chip ops
**************************************************/
-static void bcm47xxnflash_ops_bcm4706_cmd_ctrl(struct mtd_info *mtd, int cmd,
- unsigned int ctrl)
+static void bcm47xxnflash_ops_bcm4706_cmd_ctrl(struct nand_chip *nand_chip,
+ int cmd, unsigned int ctrl)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
struct bcm47xxnflash *b47n = nand_get_controller_data(nand_chip);
u32 code = 0;
}
/* Default nand_select_chip calls cmd_ctrl, which is not used in BCM4706 */
-static void bcm47xxnflash_ops_bcm4706_select_chip(struct mtd_info *mtd,
- int chip)
+static void bcm47xxnflash_ops_bcm4706_select_chip(struct nand_chip *chip,
+ int cs)
{
return;
}
-static int bcm47xxnflash_ops_bcm4706_dev_ready(struct mtd_info *mtd)
+static int bcm47xxnflash_ops_bcm4706_dev_ready(struct nand_chip *nand_chip)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
struct bcm47xxnflash *b47n = nand_get_controller_data(nand_chip);
return !!(bcma_cc_read32(b47n->cc, BCMA_CC_NFLASH_CTL) & NCTL_READY);
* registers of ChipCommon core. Hacking cmd_ctrl to understand and convert
* standard commands would be much more complicated.
*/
-static void bcm47xxnflash_ops_bcm4706_cmdfunc(struct mtd_info *mtd,
+static void bcm47xxnflash_ops_bcm4706_cmdfunc(struct nand_chip *nand_chip,
unsigned command, int column,
int page_addr)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(nand_chip);
struct bcm47xxnflash *b47n = nand_get_controller_data(nand_chip);
struct bcma_drv_cc *cc = b47n->cc;
u32 ctlcode;
switch (command) {
case NAND_CMD_RESET:
- nand_chip->cmd_ctrl(mtd, command, NAND_CTRL_CLE);
+ nand_chip->legacy.cmd_ctrl(nand_chip, command, NAND_CTRL_CLE);
ndelay(100);
- nand_wait_ready(mtd);
+ nand_wait_ready(nand_chip);
break;
case NAND_CMD_READID:
ctlcode = NCTL_CSA | 0x01000000 | NCTL_CMD1W | NCTL_CMD0;
b47n->curr_command = command;
}
-static u8 bcm47xxnflash_ops_bcm4706_read_byte(struct mtd_info *mtd)
+static u8 bcm47xxnflash_ops_bcm4706_read_byte(struct nand_chip *nand_chip)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(nand_chip);
struct bcm47xxnflash *b47n = nand_get_controller_data(nand_chip);
struct bcma_drv_cc *cc = b47n->cc;
u32 tmp = 0;
return 0;
}
-static void bcm47xxnflash_ops_bcm4706_read_buf(struct mtd_info *mtd,
+static void bcm47xxnflash_ops_bcm4706_read_buf(struct nand_chip *nand_chip,
uint8_t *buf, int len)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
struct bcm47xxnflash *b47n = nand_get_controller_data(nand_chip);
switch (b47n->curr_command) {
case NAND_CMD_READ0:
case NAND_CMD_READOOB:
- bcm47xxnflash_ops_bcm4706_read(mtd, buf, len);
+ bcm47xxnflash_ops_bcm4706_read(nand_to_mtd(nand_chip), buf,
+ len);
return;
}
pr_err("Invalid command for buf read: 0x%X\n", b47n->curr_command);
}
-static void bcm47xxnflash_ops_bcm4706_write_buf(struct mtd_info *mtd,
+static void bcm47xxnflash_ops_bcm4706_write_buf(struct nand_chip *nand_chip,
const uint8_t *buf, int len)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
struct bcm47xxnflash *b47n = nand_get_controller_data(nand_chip);
switch (b47n->curr_command) {
case NAND_CMD_SEQIN:
- bcm47xxnflash_ops_bcm4706_write(mtd, buf, len);
+ bcm47xxnflash_ops_bcm4706_write(nand_to_mtd(nand_chip), buf,
+ len);
return;
}
u32 val;
b47n->nand_chip.select_chip = bcm47xxnflash_ops_bcm4706_select_chip;
- nand_chip->cmd_ctrl = bcm47xxnflash_ops_bcm4706_cmd_ctrl;
- nand_chip->dev_ready = bcm47xxnflash_ops_bcm4706_dev_ready;
- b47n->nand_chip.cmdfunc = bcm47xxnflash_ops_bcm4706_cmdfunc;
- b47n->nand_chip.read_byte = bcm47xxnflash_ops_bcm4706_read_byte;
- b47n->nand_chip.read_buf = bcm47xxnflash_ops_bcm4706_read_buf;
- b47n->nand_chip.write_buf = bcm47xxnflash_ops_bcm4706_write_buf;
- b47n->nand_chip.set_features = nand_get_set_features_notsupp;
- b47n->nand_chip.get_features = nand_get_set_features_notsupp;
-
- nand_chip->chip_delay = 50;
+ nand_chip->legacy.cmd_ctrl = bcm47xxnflash_ops_bcm4706_cmd_ctrl;
+ nand_chip->legacy.dev_ready = bcm47xxnflash_ops_bcm4706_dev_ready;
+ b47n->nand_chip.legacy.cmdfunc = bcm47xxnflash_ops_bcm4706_cmdfunc;
+ b47n->nand_chip.legacy.read_byte = bcm47xxnflash_ops_bcm4706_read_byte;
+ b47n->nand_chip.legacy.read_buf = bcm47xxnflash_ops_bcm4706_read_buf;
+ b47n->nand_chip.legacy.write_buf = bcm47xxnflash_ops_bcm4706_write_buf;
+ b47n->nand_chip.legacy.set_features = nand_get_set_features_notsupp;
+ b47n->nand_chip.legacy.get_features = nand_get_set_features_notsupp;
+
+ nand_chip->legacy.chip_delay = 50;
b47n->nand_chip.bbt_options = NAND_BBT_USE_FLASH;
b47n->nand_chip.ecc.mode = NAND_ECC_NONE; /* TODO: implement ECC */
(w4 << 24 | w3 << 18 | w2 << 12 | w1 << 6 | w0));
/* Scan NAND */
- err = nand_scan(nand_to_mtd(&b47n->nand_chip), 1);
+ err = nand_scan(&b47n->nand_chip, 1);
if (err) {
pr_err("Could not scan NAND flash: %d\n", err);
goto exit;
* NAND MTD API: read/program/erase
***********************************************************************/
-static void brcmnand_cmd_ctrl(struct mtd_info *mtd, int dat,
- unsigned int ctrl)
+static void brcmnand_cmd_ctrl(struct nand_chip *chip, int dat,
+ unsigned int ctrl)
{
/* intentionally left blank */
}
-static int brcmnand_waitfunc(struct mtd_info *mtd, struct nand_chip *this)
+static int brcmnand_waitfunc(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct brcmnand_host *host = nand_get_controller_data(chip);
struct brcmnand_controller *ctrl = host->ctrl;
unsigned long timeo = msecs_to_jiffies(100);
enum brcmnand_llop_type type, u32 data,
bool last_op)
{
- struct mtd_info *mtd = nand_to_mtd(&host->chip);
struct nand_chip *chip = &host->chip;
struct brcmnand_controller *ctrl = host->ctrl;
u32 tmp;
(void)brcmnand_read_reg(ctrl, BRCMNAND_LL_OP);
brcmnand_send_cmd(host, CMD_LOW_LEVEL_OP);
- return brcmnand_waitfunc(mtd, chip);
+ return brcmnand_waitfunc(chip);
}
-static void brcmnand_cmdfunc(struct mtd_info *mtd, unsigned command,
+static void brcmnand_cmdfunc(struct nand_chip *chip, unsigned command,
int column, int page_addr)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct brcmnand_host *host = nand_get_controller_data(chip);
struct brcmnand_controller *ctrl = host->ctrl;
u64 addr = (u64)page_addr << chip->page_shift;
(void)brcmnand_read_reg(ctrl, BRCMNAND_CMD_ADDRESS);
brcmnand_send_cmd(host, native_cmd);
- brcmnand_waitfunc(mtd, chip);
+ brcmnand_waitfunc(chip);
if (native_cmd == CMD_PARAMETER_READ ||
native_cmd == CMD_PARAMETER_CHANGE_COL) {
brcmnand_wp(mtd, 1);
}
-static uint8_t brcmnand_read_byte(struct mtd_info *mtd)
+static uint8_t brcmnand_read_byte(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct brcmnand_host *host = nand_get_controller_data(chip);
struct brcmnand_controller *ctrl = host->ctrl;
uint8_t ret = 0;
return ret;
}
-static void brcmnand_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void brcmnand_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
{
int i;
for (i = 0; i < len; i++, buf++)
- *buf = brcmnand_read_byte(mtd);
+ *buf = brcmnand_read_byte(chip);
}
-static void brcmnand_write_buf(struct mtd_info *mtd, const uint8_t *buf,
- int len)
+static void brcmnand_write_buf(struct nand_chip *chip, const uint8_t *buf,
+ int len)
{
int i;
- struct nand_chip *chip = mtd_to_nand(mtd);
struct brcmnand_host *host = nand_get_controller_data(chip);
switch (host->last_cmd) {
(void)brcmnand_read_reg(ctrl, BRCMNAND_CMD_ADDRESS);
/* SPARE_AREA_READ does not use ECC, so just use PAGE_READ */
brcmnand_send_cmd(host, CMD_PAGE_READ);
- brcmnand_waitfunc(mtd, chip);
+ brcmnand_waitfunc(chip);
if (likely(buf)) {
brcmnand_soc_data_bus_prepare(ctrl->soc, false);
sas = mtd->oobsize / chip->ecc.steps;
/* read without ecc for verification */
- ret = chip->ecc.read_page_raw(mtd, chip, buf, true, page);
+ ret = chip->ecc.read_page_raw(chip, buf, true, page);
if (ret)
return ret;
return 0;
}
-static int brcmnand_read_page(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int brcmnand_read_page(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct brcmnand_host *host = nand_get_controller_data(chip);
u8 *oob = oob_required ? (u8 *)chip->oob_poi : NULL;
mtd->writesize >> FC_SHIFT, (u32 *)buf, oob);
}
-static int brcmnand_read_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int brcmnand_read_page_raw(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
struct brcmnand_host *host = nand_get_controller_data(chip);
+ struct mtd_info *mtd = nand_to_mtd(chip);
u8 *oob = oob_required ? (u8 *)chip->oob_poi : NULL;
int ret;
return ret;
}
-static int brcmnand_read_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int brcmnand_read_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
return brcmnand_read(mtd, chip, (u64)page << chip->page_shift,
mtd->writesize >> FC_SHIFT,
NULL, (u8 *)chip->oob_poi);
}
-static int brcmnand_read_oob_raw(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int brcmnand_read_oob_raw(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct brcmnand_host *host = nand_get_controller_data(chip);
brcmnand_set_ecc_enabled(host, 0);
/* we cannot use SPARE_AREA_PROGRAM when PARTIAL_PAGE_EN=0 */
brcmnand_send_cmd(host, CMD_PROGRAM_PAGE);
- status = brcmnand_waitfunc(mtd, chip);
+ status = brcmnand_waitfunc(chip);
if (status & NAND_STATUS_FAIL) {
dev_info(ctrl->dev, "program failed at %llx\n",
return ret;
}
-static int brcmnand_write_page(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page)
+static int brcmnand_write_page(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct brcmnand_host *host = nand_get_controller_data(chip);
void *oob = oob_required ? chip->oob_poi : NULL;
return nand_prog_page_end_op(chip);
}
-static int brcmnand_write_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip, const uint8_t *buf,
+static int brcmnand_write_page_raw(struct nand_chip *chip, const uint8_t *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct brcmnand_host *host = nand_get_controller_data(chip);
void *oob = oob_required ? chip->oob_poi : NULL;
return nand_prog_page_end_op(chip);
}
-static int brcmnand_write_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int brcmnand_write_oob(struct nand_chip *chip, int page)
{
- return brcmnand_write(mtd, chip, (u64)page << chip->page_shift,
- NULL, chip->oob_poi);
+ return brcmnand_write(nand_to_mtd(chip), chip,
+ (u64)page << chip->page_shift, NULL,
+ chip->oob_poi);
}
-static int brcmnand_write_oob_raw(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int brcmnand_write_oob_raw(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct brcmnand_host *host = nand_get_controller_data(chip);
int ret;
mtd->owner = THIS_MODULE;
mtd->dev.parent = &pdev->dev;
- chip->IO_ADDR_R = (void __iomem *)0xdeadbeef;
- chip->IO_ADDR_W = (void __iomem *)0xdeadbeef;
-
- chip->cmd_ctrl = brcmnand_cmd_ctrl;
- chip->cmdfunc = brcmnand_cmdfunc;
- chip->waitfunc = brcmnand_waitfunc;
- chip->read_byte = brcmnand_read_byte;
- chip->read_buf = brcmnand_read_buf;
- chip->write_buf = brcmnand_write_buf;
+ chip->legacy.cmd_ctrl = brcmnand_cmd_ctrl;
+ chip->legacy.cmdfunc = brcmnand_cmdfunc;
+ chip->legacy.waitfunc = brcmnand_waitfunc;
+ chip->legacy.read_byte = brcmnand_read_byte;
+ chip->legacy.read_buf = brcmnand_read_buf;
+ chip->legacy.write_buf = brcmnand_write_buf;
chip->ecc.mode = NAND_ECC_HW;
chip->ecc.read_page = brcmnand_read_page;
nand_writereg(ctrl, cfg_offs,
nand_readreg(ctrl, cfg_offs) & ~CFG_BUS_WIDTH);
- ret = nand_scan(mtd, 1);
+ ret = nand_scan(chip, 1);
if (ret)
return ret;
struct brcmnand_host *host;
list_for_each_entry(host, &ctrl->host_list, node)
- nand_release(nand_to_mtd(&host->chip));
+ nand_release(&host->chip);
clk_disable_unprepare(ctrl->clk);
#define cafe_readl(cafe, addr) readl((cafe)->mmio + CAFE_##addr)
#define cafe_writel(cafe, datum, addr) writel(datum, (cafe)->mmio + CAFE_##addr)
-static int cafe_device_ready(struct mtd_info *mtd)
+static int cafe_device_ready(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct cafe_priv *cafe = nand_get_controller_data(chip);
int result = !!(cafe_readl(cafe, NAND_STATUS) & 0x40000000);
uint32_t irqs = cafe_readl(cafe, NAND_IRQ);
}
-static void cafe_write_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
+static void cafe_write_buf(struct nand_chip *chip, const uint8_t *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct cafe_priv *cafe = nand_get_controller_data(chip);
if (cafe->usedma)
len, cafe->datalen);
}
-static void cafe_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void cafe_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct cafe_priv *cafe = nand_get_controller_data(chip);
if (cafe->usedma)
cafe->datalen += len;
}
-static uint8_t cafe_read_byte(struct mtd_info *mtd)
+static uint8_t cafe_read_byte(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct cafe_priv *cafe = nand_get_controller_data(chip);
uint8_t d;
- cafe_read_buf(mtd, &d, 1);
+ cafe_read_buf(chip, &d, 1);
cafe_dev_dbg(&cafe->pdev->dev, "Read %02x\n", d);
return d;
}
-static void cafe_nand_cmdfunc(struct mtd_info *mtd, unsigned command,
+static void cafe_nand_cmdfunc(struct nand_chip *chip, unsigned command,
int column, int page_addr)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct cafe_priv *cafe = nand_get_controller_data(chip);
int adrbytes = 0;
uint32_t ctl1;
cafe_writel(cafe, cafe->ctl2, NAND_CTRL2);
return;
}
- nand_wait_ready(mtd);
+ nand_wait_ready(chip);
cafe_writel(cafe, cafe->ctl2, NAND_CTRL2);
}
-static void cafe_select_chip(struct mtd_info *mtd, int chipnr)
+static void cafe_select_chip(struct nand_chip *chip, int chipnr)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct cafe_priv *cafe = nand_get_controller_data(chip);
cafe_dev_dbg(&cafe->pdev->dev, "select_chip %d\n", chipnr);
return IRQ_HANDLED;
}
-static int cafe_nand_write_oob(struct mtd_info *mtd,
- struct nand_chip *chip, int page)
+static int cafe_nand_write_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
return nand_prog_page_op(chip, page, mtd->writesize, chip->oob_poi,
mtd->oobsize);
}
/* Don't use -- use nand_read_oob_std for now */
-static int cafe_nand_read_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int cafe_nand_read_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
return nand_read_oob_op(chip, page, 0, chip->oob_poi, mtd->oobsize);
}
/**
* The hw generator calculates the error syndrome automatically. Therefore
* we need a special oob layout and handling.
*/
-static int cafe_nand_read_page(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int cafe_nand_read_page(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct cafe_priv *cafe = nand_get_controller_data(chip);
unsigned int max_bitflips = 0;
cafe_readl(cafe, NAND_ECC_SYN01));
nand_read_page_op(chip, page, 0, buf, mtd->writesize);
- chip->read_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.read_buf(chip, chip->oob_poi, mtd->oobsize);
if (checkecc && cafe_readl(cafe, NAND_ECC_RESULT) & (1<<18)) {
unsigned short syn[8], pat[4];
};
-static int cafe_nand_write_page_lowlevel(struct mtd_info *mtd,
- struct nand_chip *chip,
- const uint8_t *buf, int oob_required,
- int page)
+static int cafe_nand_write_page_lowlevel(struct nand_chip *chip,
+ const uint8_t *buf, int oob_required,
+ int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct cafe_priv *cafe = nand_get_controller_data(chip);
nand_prog_page_begin_op(chip, page, 0, buf, mtd->writesize);
- chip->write_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.write_buf(chip, chip->oob_poi, mtd->oobsize);
/* Set up ECC autogeneration */
cafe->ctl2 |= (1<<30);
return nand_prog_page_end_op(chip);
}
-static int cafe_nand_block_bad(struct mtd_info *mtd, loff_t ofs)
+static int cafe_nand_block_bad(struct nand_chip *chip, loff_t ofs)
{
return 0;
}
goto out_ior;
}
- cafe->nand.cmdfunc = cafe_nand_cmdfunc;
- cafe->nand.dev_ready = cafe_device_ready;
- cafe->nand.read_byte = cafe_read_byte;
- cafe->nand.read_buf = cafe_read_buf;
- cafe->nand.write_buf = cafe_write_buf;
+ cafe->nand.legacy.cmdfunc = cafe_nand_cmdfunc;
+ cafe->nand.legacy.dev_ready = cafe_device_ready;
+ cafe->nand.legacy.read_byte = cafe_read_byte;
+ cafe->nand.legacy.read_buf = cafe_read_buf;
+ cafe->nand.legacy.write_buf = cafe_write_buf;
cafe->nand.select_chip = cafe_select_chip;
- cafe->nand.set_features = nand_get_set_features_notsupp;
- cafe->nand.get_features = nand_get_set_features_notsupp;
+ cafe->nand.legacy.set_features = nand_get_set_features_notsupp;
+ cafe->nand.legacy.get_features = nand_get_set_features_notsupp;
- cafe->nand.chip_delay = 0;
+ cafe->nand.legacy.chip_delay = 0;
/* Enable the following for a flash based bad block table */
cafe->nand.bbt_options = NAND_BBT_USE_FLASH;
if (skipbbt) {
cafe->nand.options |= NAND_SKIP_BBTSCAN;
- cafe->nand.block_bad = cafe_nand_block_bad;
+ cafe->nand.legacy.block_bad = cafe_nand_block_bad;
}
if (numtimings && numtimings != 3) {
/* Scan to find existence of the device */
cafe->nand.dummy_controller.ops = &cafe_nand_controller_ops;
- err = nand_scan(mtd, 2);
+ err = nand_scan(&cafe->nand, 2);
if (err)
goto out_irq;
/* Disable NAND IRQ in global IRQ mask register */
cafe_writel(cafe, ~1 & cafe_readl(cafe, GLOBAL_IRQ_MASK), GLOBAL_IRQ_MASK);
free_irq(pdev->irq, mtd);
- nand_release(mtd);
+ nand_release(chip);
free_rs(cafe->rs);
pci_iounmap(pdev, cafe->mmio);
dma_free_coherent(&cafe->pdev->dev, 2112, cafe->dmabuf, cafe->dmaaddr);
};
#define NUM_PARTITIONS (ARRAY_SIZE(partition_info))
-static u_char cmx270_read_byte(struct mtd_info *mtd)
+static u_char cmx270_read_byte(struct nand_chip *this)
{
- struct nand_chip *this = mtd_to_nand(mtd);
-
- return (readl(this->IO_ADDR_R) >> 16);
+ return (readl(this->legacy.IO_ADDR_R) >> 16);
}
-static void cmx270_write_buf(struct mtd_info *mtd, const u_char *buf, int len)
+static void cmx270_write_buf(struct nand_chip *this, const u_char *buf,
+ int len)
{
int i;
- struct nand_chip *this = mtd_to_nand(mtd);
for (i=0; i<len; i++)
- writel((*buf++ << 16), this->IO_ADDR_W);
+ writel((*buf++ << 16), this->legacy.IO_ADDR_W);
}
-static void cmx270_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void cmx270_read_buf(struct nand_chip *this, u_char *buf, int len)
{
int i;
- struct nand_chip *this = mtd_to_nand(mtd);
for (i=0; i<len; i++)
- *buf++ = readl(this->IO_ADDR_R) >> 16;
+ *buf++ = readl(this->legacy.IO_ADDR_R) >> 16;
}
static inline void nand_cs_on(void)
/*
* hardware specific access to control-lines
*/
-static void cmx270_hwcontrol(struct mtd_info *mtd, int dat,
+static void cmx270_hwcontrol(struct nand_chip *this, int dat,
unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- unsigned int nandaddr = (unsigned int)this->IO_ADDR_W;
+ unsigned int nandaddr = (unsigned int)this->legacy.IO_ADDR_W;
dsb();
}
dsb();
- this->IO_ADDR_W = (void __iomem*)nandaddr;
+ this->legacy.IO_ADDR_W = (void __iomem*)nandaddr;
if (dat != NAND_CMD_NONE)
- writel((dat << 16), this->IO_ADDR_W);
+ writel((dat << 16), this->legacy.IO_ADDR_W);
dsb();
}
/*
* read device ready pin
*/
-static int cmx270_device_ready(struct mtd_info *mtd)
+static int cmx270_device_ready(struct nand_chip *this)
{
dsb();
cmx270_nand_mtd->owner = THIS_MODULE;
/* insert callbacks */
- this->IO_ADDR_R = cmx270_nand_io;
- this->IO_ADDR_W = cmx270_nand_io;
- this->cmd_ctrl = cmx270_hwcontrol;
- this->dev_ready = cmx270_device_ready;
+ this->legacy.IO_ADDR_R = cmx270_nand_io;
+ this->legacy.IO_ADDR_W = cmx270_nand_io;
+ this->legacy.cmd_ctrl = cmx270_hwcontrol;
+ this->legacy.dev_ready = cmx270_device_ready;
/* 15 us command delay time */
- this->chip_delay = 20;
+ this->legacy.chip_delay = 20;
this->ecc.mode = NAND_ECC_SOFT;
this->ecc.algo = NAND_ECC_HAMMING;
/* read/write functions */
- this->read_byte = cmx270_read_byte;
- this->read_buf = cmx270_read_buf;
- this->write_buf = cmx270_write_buf;
+ this->legacy.read_byte = cmx270_read_byte;
+ this->legacy.read_buf = cmx270_read_buf;
+ this->legacy.write_buf = cmx270_write_buf;
/* Scan to find existence of the device */
- ret = nand_scan(cmx270_nand_mtd, 1);
+ ret = nand_scan(this, 1);
if (ret) {
pr_notice("No NAND device\n");
goto err_scan;
static void __exit cmx270_cleanup(void)
{
/* Release resources, unregister device */
- nand_release(cmx270_nand_mtd);
+ nand_release(mtd_to_nand(cmx270_nand_mtd));
gpio_free(GPIO_NAND_RB);
gpio_free(GPIO_NAND_CS);
#define CS_NAND_ECC_CLRECC (1<<1)
#define CS_NAND_ECC_ENECC (1<<0)
-static void cs553x_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void cs553x_read_buf(struct nand_chip *this, u_char *buf, int len)
{
- struct nand_chip *this = mtd_to_nand(mtd);
-
while (unlikely(len > 0x800)) {
- memcpy_fromio(buf, this->IO_ADDR_R, 0x800);
+ memcpy_fromio(buf, this->legacy.IO_ADDR_R, 0x800);
buf += 0x800;
len -= 0x800;
}
- memcpy_fromio(buf, this->IO_ADDR_R, len);
+ memcpy_fromio(buf, this->legacy.IO_ADDR_R, len);
}
-static void cs553x_write_buf(struct mtd_info *mtd, const u_char *buf, int len)
+static void cs553x_write_buf(struct nand_chip *this, const u_char *buf, int len)
{
- struct nand_chip *this = mtd_to_nand(mtd);
-
while (unlikely(len > 0x800)) {
- memcpy_toio(this->IO_ADDR_R, buf, 0x800);
+ memcpy_toio(this->legacy.IO_ADDR_R, buf, 0x800);
buf += 0x800;
len -= 0x800;
}
- memcpy_toio(this->IO_ADDR_R, buf, len);
+ memcpy_toio(this->legacy.IO_ADDR_R, buf, len);
}
-static unsigned char cs553x_read_byte(struct mtd_info *mtd)
+static unsigned char cs553x_read_byte(struct nand_chip *this)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- return readb(this->IO_ADDR_R);
+ return readb(this->legacy.IO_ADDR_R);
}
-static void cs553x_write_byte(struct mtd_info *mtd, u_char byte)
+static void cs553x_write_byte(struct nand_chip *this, u_char byte)
{
- struct nand_chip *this = mtd_to_nand(mtd);
int i = 100000;
- while (i && readb(this->IO_ADDR_R + MM_NAND_STS) & CS_NAND_CTLR_BUSY) {
+ while (i && readb(this->legacy.IO_ADDR_R + MM_NAND_STS) & CS_NAND_CTLR_BUSY) {
udelay(1);
i--;
}
- writeb(byte, this->IO_ADDR_W + 0x801);
+ writeb(byte, this->legacy.IO_ADDR_W + 0x801);
}
-static void cs553x_hwcontrol(struct mtd_info *mtd, int cmd,
+static void cs553x_hwcontrol(struct nand_chip *this, int cmd,
unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- void __iomem *mmio_base = this->IO_ADDR_R;
+ void __iomem *mmio_base = this->legacy.IO_ADDR_R;
if (ctrl & NAND_CTRL_CHANGE) {
unsigned char ctl = (ctrl & ~NAND_CTRL_CHANGE ) ^ 0x01;
writeb(ctl, mmio_base + MM_NAND_CTL);
}
if (cmd != NAND_CMD_NONE)
- cs553x_write_byte(mtd, cmd);
+ cs553x_write_byte(this, cmd);
}
-static int cs553x_device_ready(struct mtd_info *mtd)
+static int cs553x_device_ready(struct nand_chip *this)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- void __iomem *mmio_base = this->IO_ADDR_R;
+ void __iomem *mmio_base = this->legacy.IO_ADDR_R;
unsigned char foo = readb(mmio_base + MM_NAND_STS);
return (foo & CS_NAND_STS_FLASH_RDY) && !(foo & CS_NAND_CTLR_BUSY);
}
-static void cs_enable_hwecc(struct mtd_info *mtd, int mode)
+static void cs_enable_hwecc(struct nand_chip *this, int mode)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- void __iomem *mmio_base = this->IO_ADDR_R;
+ void __iomem *mmio_base = this->legacy.IO_ADDR_R;
writeb(0x07, mmio_base + MM_NAND_ECC_CTL);
}
-static int cs_calculate_ecc(struct mtd_info *mtd, const u_char *dat, u_char *ecc_code)
+static int cs_calculate_ecc(struct nand_chip *this, const u_char *dat,
+ u_char *ecc_code)
{
uint32_t ecc;
- struct nand_chip *this = mtd_to_nand(mtd);
- void __iomem *mmio_base = this->IO_ADDR_R;
+ void __iomem *mmio_base = this->legacy.IO_ADDR_R;
ecc = readl(mmio_base + MM_NAND_STS);
new_mtd->owner = THIS_MODULE;
/* map physical address */
- this->IO_ADDR_R = this->IO_ADDR_W = ioremap(adr, 4096);
- if (!this->IO_ADDR_R) {
+ this->legacy.IO_ADDR_R = this->legacy.IO_ADDR_W = ioremap(adr, 4096);
+ if (!this->legacy.IO_ADDR_R) {
pr_warn("ioremap cs553x NAND @0x%08lx failed\n", adr);
err = -EIO;
goto out_mtd;
}
- this->cmd_ctrl = cs553x_hwcontrol;
- this->dev_ready = cs553x_device_ready;
- this->read_byte = cs553x_read_byte;
- this->read_buf = cs553x_read_buf;
- this->write_buf = cs553x_write_buf;
+ this->legacy.cmd_ctrl = cs553x_hwcontrol;
+ this->legacy.dev_ready = cs553x_device_ready;
+ this->legacy.read_byte = cs553x_read_byte;
+ this->legacy.read_buf = cs553x_read_buf;
+ this->legacy.write_buf = cs553x_write_buf;
- this->chip_delay = 0;
+ this->legacy.chip_delay = 0;
this->ecc.mode = NAND_ECC_HW;
this->ecc.size = 256;
}
/* Scan to find existence of the device */
- err = nand_scan(new_mtd, 1);
+ err = nand_scan(this, 1);
if (err)
goto out_free;
out_free:
kfree(new_mtd->name);
out_ior:
- iounmap(this->IO_ADDR_R);
+ iounmap(this->legacy.IO_ADDR_R);
out_mtd:
kfree(this);
out:
continue;
this = mtd_to_nand(mtd);
- mmio_base = this->IO_ADDR_R;
+ mmio_base = this->legacy.IO_ADDR_R;
/* Release resources, unregister device */
- nand_release(mtd);
+ nand_release(this);
kfree(mtd->name);
cs553x_mtd[i] = NULL;
* Access to hardware control lines: ALE, CLE, secondary chipselect.
*/
-static void nand_davinci_hwcontrol(struct mtd_info *mtd, int cmd,
+static void nand_davinci_hwcontrol(struct nand_chip *nand, int cmd,
unsigned int ctrl)
{
- struct davinci_nand_info *info = to_davinci_nand(mtd);
+ struct davinci_nand_info *info = to_davinci_nand(nand_to_mtd(nand));
void __iomem *addr = info->current_cs;
- struct nand_chip *nand = mtd_to_nand(mtd);
/* Did the control lines change? */
if (ctrl & NAND_CTRL_CHANGE) {
else if ((ctrl & NAND_CTRL_ALE) == NAND_CTRL_ALE)
addr += info->mask_ale;
- nand->IO_ADDR_W = addr;
+ nand->legacy.IO_ADDR_W = addr;
}
if (cmd != NAND_CMD_NONE)
- iowrite8(cmd, nand->IO_ADDR_W);
+ iowrite8(cmd, nand->legacy.IO_ADDR_W);
}
-static void nand_davinci_select_chip(struct mtd_info *mtd, int chip)
+static void nand_davinci_select_chip(struct nand_chip *nand, int chip)
{
- struct davinci_nand_info *info = to_davinci_nand(mtd);
+ struct davinci_nand_info *info = to_davinci_nand(nand_to_mtd(nand));
info->current_cs = info->vaddr;
if (chip > 0)
info->current_cs += info->mask_chipsel;
- info->chip.IO_ADDR_W = info->current_cs;
- info->chip.IO_ADDR_R = info->chip.IO_ADDR_W;
+ info->chip.legacy.IO_ADDR_W = info->current_cs;
+ info->chip.legacy.IO_ADDR_R = info->chip.legacy.IO_ADDR_W;
}
/*----------------------------------------------------------------------*/
+ 4 * info->core_chipsel);
}
-static void nand_davinci_hwctl_1bit(struct mtd_info *mtd, int mode)
+static void nand_davinci_hwctl_1bit(struct nand_chip *chip, int mode)
{
struct davinci_nand_info *info;
uint32_t nandcfr;
unsigned long flags;
- info = to_davinci_nand(mtd);
+ info = to_davinci_nand(nand_to_mtd(chip));
/* Reset ECC hardware */
- nand_davinci_readecc_1bit(mtd);
+ nand_davinci_readecc_1bit(nand_to_mtd(chip));
spin_lock_irqsave(&davinci_nand_lock, flags);
/*
* Read hardware ECC value and pack into three bytes
*/
-static int nand_davinci_calculate_1bit(struct mtd_info *mtd,
- const u_char *dat, u_char *ecc_code)
+static int nand_davinci_calculate_1bit(struct nand_chip *chip,
+ const u_char *dat, u_char *ecc_code)
{
- unsigned int ecc_val = nand_davinci_readecc_1bit(mtd);
+ unsigned int ecc_val = nand_davinci_readecc_1bit(nand_to_mtd(chip));
unsigned int ecc24 = (ecc_val & 0x0fff) | ((ecc_val & 0x0fff0000) >> 4);
/* invert so that erased block ecc is correct */
return 0;
}
-static int nand_davinci_correct_1bit(struct mtd_info *mtd, u_char *dat,
+static int nand_davinci_correct_1bit(struct nand_chip *chip, u_char *dat,
u_char *read_ecc, u_char *calc_ecc)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
uint32_t eccNand = read_ecc[0] | (read_ecc[1] << 8) |
(read_ecc[2] << 16);
uint32_t eccCalc = calc_ecc[0] | (calc_ecc[1] << 8) |
* OOB without recomputing ECC.
*/
-static void nand_davinci_hwctl_4bit(struct mtd_info *mtd, int mode)
+static void nand_davinci_hwctl_4bit(struct nand_chip *chip, int mode)
{
- struct davinci_nand_info *info = to_davinci_nand(mtd);
+ struct davinci_nand_info *info = to_davinci_nand(nand_to_mtd(chip));
unsigned long flags;
u32 val;
}
/* Terminate read ECC; or return ECC (as bytes) of data written to NAND. */
-static int nand_davinci_calculate_4bit(struct mtd_info *mtd,
- const u_char *dat, u_char *ecc_code)
+static int nand_davinci_calculate_4bit(struct nand_chip *chip,
+ const u_char *dat, u_char *ecc_code)
{
- struct davinci_nand_info *info = to_davinci_nand(mtd);
+ struct davinci_nand_info *info = to_davinci_nand(nand_to_mtd(chip));
u32 raw_ecc[4], *p;
unsigned i;
/* Correct up to 4 bits in data we just read, using state left in the
* hardware plus the ecc_code computed when it was first written.
*/
-static int nand_davinci_correct_4bit(struct mtd_info *mtd,
- u_char *data, u_char *ecc_code, u_char *null)
+static int nand_davinci_correct_4bit(struct nand_chip *chip, u_char *data,
+ u_char *ecc_code, u_char *null)
{
int i;
- struct davinci_nand_info *info = to_davinci_nand(mtd);
+ struct davinci_nand_info *info = to_davinci_nand(nand_to_mtd(chip));
unsigned short ecc10[8];
unsigned short *ecc16;
u32 syndrome[4];
* the two LSBs for NAND access ... so we can issue 32-bit reads/writes
* and have that transparently morphed into multiple NAND operations.
*/
-static void nand_davinci_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void nand_davinci_read_buf(struct nand_chip *chip, uint8_t *buf,
+ int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
if ((0x03 & ((uintptr_t)buf)) == 0 && (0x03 & len) == 0)
- ioread32_rep(chip->IO_ADDR_R, buf, len >> 2);
+ ioread32_rep(chip->legacy.IO_ADDR_R, buf, len >> 2);
else if ((0x01 & ((uintptr_t)buf)) == 0 && (0x01 & len) == 0)
- ioread16_rep(chip->IO_ADDR_R, buf, len >> 1);
+ ioread16_rep(chip->legacy.IO_ADDR_R, buf, len >> 1);
else
- ioread8_rep(chip->IO_ADDR_R, buf, len);
+ ioread8_rep(chip->legacy.IO_ADDR_R, buf, len);
}
-static void nand_davinci_write_buf(struct mtd_info *mtd,
- const uint8_t *buf, int len)
+static void nand_davinci_write_buf(struct nand_chip *chip, const uint8_t *buf,
+ int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
if ((0x03 & ((uintptr_t)buf)) == 0 && (0x03 & len) == 0)
- iowrite32_rep(chip->IO_ADDR_R, buf, len >> 2);
+ iowrite32_rep(chip->legacy.IO_ADDR_R, buf, len >> 2);
else if ((0x01 & ((uintptr_t)buf)) == 0 && (0x01 & len) == 0)
- iowrite16_rep(chip->IO_ADDR_R, buf, len >> 1);
+ iowrite16_rep(chip->legacy.IO_ADDR_R, buf, len >> 1);
else
- iowrite8_rep(chip->IO_ADDR_R, buf, len);
+ iowrite8_rep(chip->legacy.IO_ADDR_R, buf, len);
}
/*
* Check hardware register for wait status. Returns 1 if device is ready,
* 0 if it is still busy.
*/
-static int nand_davinci_dev_ready(struct mtd_info *mtd)
+static int nand_davinci_dev_ready(struct nand_chip *chip)
{
- struct davinci_nand_info *info = to_davinci_nand(mtd);
+ struct davinci_nand_info *info = to_davinci_nand(nand_to_mtd(chip));
return davinci_nand_readl(info, NANDFSR_OFFSET) & BIT(0);
}
mtd->dev.parent = &pdev->dev;
nand_set_flash_node(&info->chip, pdev->dev.of_node);
- info->chip.IO_ADDR_R = vaddr;
- info->chip.IO_ADDR_W = vaddr;
- info->chip.chip_delay = 0;
+ info->chip.legacy.IO_ADDR_R = vaddr;
+ info->chip.legacy.IO_ADDR_W = vaddr;
+ info->chip.legacy.chip_delay = 0;
info->chip.select_chip = nand_davinci_select_chip;
/* options such as NAND_BBT_USE_FLASH */
info->mask_cle = pdata->mask_cle ? : MASK_CLE;
/* Set address of hardware control function */
- info->chip.cmd_ctrl = nand_davinci_hwcontrol;
- info->chip.dev_ready = nand_davinci_dev_ready;
+ info->chip.legacy.cmd_ctrl = nand_davinci_hwcontrol;
+ info->chip.legacy.dev_ready = nand_davinci_dev_ready;
/* Speed up buffer I/O */
- info->chip.read_buf = nand_davinci_read_buf;
- info->chip.write_buf = nand_davinci_write_buf;
+ info->chip.legacy.read_buf = nand_davinci_read_buf;
+ info->chip.legacy.write_buf = nand_davinci_write_buf;
/* Use board-specific ECC config */
info->chip.ecc.mode = pdata->ecc_mode;
/* Scan to find existence of the device(s) */
info->chip.dummy_controller.ops = &davinci_nand_controller_ops;
- ret = nand_scan(mtd, pdata->mask_chipsel ? 2 : 1);
+ ret = nand_scan(&info->chip, pdata->mask_chipsel ? 2 : 1);
if (ret < 0) {
dev_dbg(&pdev->dev, "no NAND chip(s) found\n");
return ret;
ecc4_busy = false;
spin_unlock_irq(&davinci_nand_lock);
- nand_release(nand_to_mtd(&info->chip));
+ nand_release(&info->chip);
return 0;
}
+// SPDX-License-Identifier: GPL-2.0
/*
* NAND Flash Controller Device Driver
* Copyright © 2009-2010, Intel Corporation and its suppliers.
*
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
+ * Copyright (c) 2017 Socionext Inc.
+ * Reworked by Masahiro Yamada <yamada.masahiro@socionext.com>
*/
#include <linux/bitfield.h>
#include "denali.h"
-MODULE_LICENSE("GPL");
-
#define DENALI_NAND_NAME "denali-nand"
+#define DENALI_DEFAULT_OOB_SKIP_BYTES 8
/* for Indexed Addressing */
#define DENALI_INDEXED_CTRL 0x00
return irq_status;
}
-static void denali_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void denali_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct denali_nand_info *denali = mtd_to_denali(mtd);
u32 addr = DENALI_MAP11_DATA | DENALI_BANK(denali);
int i;
buf[i] = denali->host_read(denali, addr);
}
-static void denali_write_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
+static void denali_write_buf(struct nand_chip *chip, const uint8_t *buf,
+ int len)
{
- struct denali_nand_info *denali = mtd_to_denali(mtd);
+ struct denali_nand_info *denali = mtd_to_denali(nand_to_mtd(chip));
u32 addr = DENALI_MAP11_DATA | DENALI_BANK(denali);
int i;
denali->host_write(denali, addr, buf[i]);
}
-static void denali_read_buf16(struct mtd_info *mtd, uint8_t *buf, int len)
+static void denali_read_buf16(struct nand_chip *chip, uint8_t *buf, int len)
{
- struct denali_nand_info *denali = mtd_to_denali(mtd);
+ struct denali_nand_info *denali = mtd_to_denali(nand_to_mtd(chip));
u32 addr = DENALI_MAP11_DATA | DENALI_BANK(denali);
uint16_t *buf16 = (uint16_t *)buf;
int i;
buf16[i] = denali->host_read(denali, addr);
}
-static void denali_write_buf16(struct mtd_info *mtd, const uint8_t *buf,
+static void denali_write_buf16(struct nand_chip *chip, const uint8_t *buf,
int len)
{
- struct denali_nand_info *denali = mtd_to_denali(mtd);
+ struct denali_nand_info *denali = mtd_to_denali(nand_to_mtd(chip));
u32 addr = DENALI_MAP11_DATA | DENALI_BANK(denali);
const uint16_t *buf16 = (const uint16_t *)buf;
int i;
denali->host_write(denali, addr, buf16[i]);
}
-static uint8_t denali_read_byte(struct mtd_info *mtd)
+static uint8_t denali_read_byte(struct nand_chip *chip)
{
uint8_t byte;
- denali_read_buf(mtd, &byte, 1);
+ denali_read_buf(chip, &byte, 1);
return byte;
}
-static void denali_write_byte(struct mtd_info *mtd, uint8_t byte)
-{
- denali_write_buf(mtd, &byte, 1);
-}
-
-static uint16_t denali_read_word(struct mtd_info *mtd)
+static void denali_write_byte(struct nand_chip *chip, uint8_t byte)
{
- uint16_t word;
-
- denali_read_buf16(mtd, (uint8_t *)&word, 2);
-
- return word;
+ denali_write_buf(chip, &byte, 1);
}
-static void denali_cmd_ctrl(struct mtd_info *mtd, int dat, unsigned int ctrl)
+static void denali_cmd_ctrl(struct nand_chip *chip, int dat, unsigned int ctrl)
{
- struct denali_nand_info *denali = mtd_to_denali(mtd);
+ struct denali_nand_info *denali = mtd_to_denali(nand_to_mtd(chip));
uint32_t type;
if (ctrl & NAND_CLE)
return;
/*
- * Some commands are followed by chip->dev_ready or chip->waitfunc.
+ * Some commands are followed by chip->legacy.dev_ready or
+ * chip->legacy.waitfunc.
* irq_status must be cleared here to catch the R/B# interrupt later.
*/
if (ctrl & NAND_CTRL_CHANGE)
denali->host_write(denali, DENALI_BANK(denali) | type, dat);
}
-static int denali_dev_ready(struct mtd_info *mtd)
+static int denali_dev_ready(struct nand_chip *chip)
{
- struct denali_nand_info *denali = mtd_to_denali(mtd);
+ struct denali_nand_info *denali = mtd_to_denali(nand_to_mtd(chip));
return !!(denali_check_irq(denali) & INTR__INT_ACT);
}
false);
}
-static int denali_read_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int denali_read_page_raw(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct denali_nand_info *denali = mtd_to_denali(mtd);
int writesize = mtd->writesize;
int oobsize = mtd->oobsize;
return 0;
}
-static int denali_read_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int denali_read_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
denali_oob_xfer(mtd, chip, page, 0);
return 0;
}
-static int denali_write_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int denali_write_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct denali_nand_info *denali = mtd_to_denali(mtd);
denali_reset_irq(denali);
return nand_prog_page_end_op(chip);
}
-static int denali_read_page(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int denali_read_page(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct denali_nand_info *denali = mtd_to_denali(mtd);
unsigned long uncor_ecc_flags = 0;
int stat = 0;
return stat;
if (uncor_ecc_flags) {
- ret = denali_read_oob(mtd, chip, page);
+ ret = denali_read_oob(chip, page);
if (ret)
return ret;
return stat;
}
-static int denali_write_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page)
+static int denali_write_page_raw(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct denali_nand_info *denali = mtd_to_denali(mtd);
int writesize = mtd->writesize;
int oobsize = mtd->oobsize;
return denali_data_xfer(denali, tmp_buf, size, page, 1, 1);
}
-static int denali_write_page(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page)
+static int denali_write_page(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct denali_nand_info *denali = mtd_to_denali(mtd);
return denali_data_xfer(denali, (void *)buf, mtd->writesize,
page, 0, 1);
}
-static void denali_select_chip(struct mtd_info *mtd, int chip)
+static void denali_select_chip(struct nand_chip *chip, int cs)
{
- struct denali_nand_info *denali = mtd_to_denali(mtd);
+ struct denali_nand_info *denali = mtd_to_denali(nand_to_mtd(chip));
- denali->active_bank = chip;
+ denali->active_bank = cs;
}
-static int denali_waitfunc(struct mtd_info *mtd, struct nand_chip *chip)
+static int denali_waitfunc(struct nand_chip *chip)
{
- struct denali_nand_info *denali = mtd_to_denali(mtd);
+ struct denali_nand_info *denali = mtd_to_denali(nand_to_mtd(chip));
uint32_t irq_status;
/* R/B# pin transitioned from low to high? */
return irq_status & INTR__INT_ACT ? 0 : NAND_STATUS_FAIL;
}
-static int denali_erase(struct mtd_info *mtd, int page)
+static int denali_erase(struct nand_chip *chip, int page)
{
- struct denali_nand_info *denali = mtd_to_denali(mtd);
+ struct denali_nand_info *denali = mtd_to_denali(nand_to_mtd(chip));
uint32_t irq_status;
denali_reset_irq(denali);
return irq_status & INTR__ERASE_COMP ? 0 : -EIO;
}
-static int denali_setup_data_interface(struct mtd_info *mtd, int chipnr,
+static int denali_setup_data_interface(struct nand_chip *chip, int chipnr,
const struct nand_data_interface *conf)
{
- struct denali_nand_info *denali = mtd_to_denali(mtd);
+ struct denali_nand_info *denali = mtd_to_denali(nand_to_mtd(chip));
const struct nand_sdr_timings *timings;
unsigned long t_x, mult_x;
int acc_clks, re_2_we, re_2_re, we_2_re, addr_2_data;
denali->revision = swab16(ioread32(denali->reg + REVISION));
/*
- * tell driver how many bit controller will skip before
- * writing ECC code in OOB, this register may be already
- * set by firmware. So we read this value out.
- * if this value is 0, just let it be.
+ * Set how many bytes should be skipped before writing data in OOB.
+ * If a non-zero value has already been set (by firmware or something),
+ * just use it. Otherwise, set the driver default.
*/
denali->oob_skip_bytes = ioread32(denali->reg + SPARE_AREA_SKIP_BYTES);
+ if (!denali->oob_skip_bytes) {
+ denali->oob_skip_bytes = DENALI_DEFAULT_OOB_SKIP_BYTES;
+ iowrite32(denali->oob_skip_bytes,
+ denali->reg + SPARE_AREA_SKIP_BYTES);
+ }
+
denali_detect_max_banks(denali);
iowrite32(0x0F, denali->reg + RB_PIN_ENABLED);
iowrite32(CHIP_EN_DONT_CARE__FLAG, denali->reg + CHIP_ENABLE_DONT_CARE);
mtd_set_ooblayout(mtd, &denali_ooblayout_ops);
if (chip->options & NAND_BUSWIDTH_16) {
- chip->read_buf = denali_read_buf16;
- chip->write_buf = denali_write_buf16;
+ chip->legacy.read_buf = denali_read_buf16;
+ chip->legacy.write_buf = denali_write_buf16;
} else {
- chip->read_buf = denali_read_buf;
- chip->write_buf = denali_write_buf;
+ chip->legacy.read_buf = denali_read_buf;
+ chip->legacy.write_buf = denali_write_buf;
}
chip->ecc.read_page = denali_read_page;
chip->ecc.read_page_raw = denali_read_page_raw;
chip->ecc.write_page_raw = denali_write_page_raw;
chip->ecc.read_oob = denali_read_oob;
chip->ecc.write_oob = denali_write_oob;
- chip->erase = denali_erase;
+ chip->legacy.erase = denali_erase;
ret = denali_multidev_fixup(denali);
if (ret)
mtd->name = "denali-nand";
chip->select_chip = denali_select_chip;
- chip->read_byte = denali_read_byte;
- chip->write_byte = denali_write_byte;
- chip->read_word = denali_read_word;
- chip->cmd_ctrl = denali_cmd_ctrl;
- chip->dev_ready = denali_dev_ready;
- chip->waitfunc = denali_waitfunc;
+ chip->legacy.read_byte = denali_read_byte;
+ chip->legacy.write_byte = denali_write_byte;
+ chip->legacy.cmd_ctrl = denali_cmd_ctrl;
+ chip->legacy.dev_ready = denali_dev_ready;
+ chip->legacy.waitfunc = denali_waitfunc;
if (features & FEATURES__INDEX_ADDR) {
denali->host_read = denali_indexed_read;
chip->setup_data_interface = denali_setup_data_interface;
chip->dummy_controller.ops = &denali_controller_ops;
- ret = nand_scan(mtd, denali->max_banks);
+ ret = nand_scan(chip, denali->max_banks);
if (ret)
goto disable_irq;
void denali_remove(struct denali_nand_info *denali)
{
- struct mtd_info *mtd = nand_to_mtd(&denali->nand);
-
- nand_release(mtd);
+ nand_release(&denali->nand);
denali_disable_irq(denali);
}
EXPORT_SYMBOL(denali_remove);
+
+MODULE_DESCRIPTION("Driver core for Denali NAND controller");
+MODULE_AUTHOR("Intel Corporation and its suppliers");
+MODULE_LICENSE("GPL v2");
+/* SPDX-License-Identifier: GPL-2.0 */
/*
* NAND Flash Controller Device Driver
* Copyright (c) 2009 - 2010, Intel Corporation and its suppliers.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
*/
#ifndef __DENALI_H__
+// SPDX-License-Identifier: GPL-2.0
/*
* NAND Flash Controller Device Driver for DT
*
* Copyright © 2011, Picochip.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
*/
#include <linux/clk.h>
};
module_platform_driver(denali_dt_driver);
-MODULE_LICENSE("GPL");
+MODULE_LICENSE("GPL v2");
MODULE_AUTHOR("Jamie Iles");
MODULE_DESCRIPTION("DT driver for Denali NAND controller");
+// SPDX-License-Identifier: GPL-2.0
/*
* NAND Flash Controller Device Driver
* Copyright © 2009-2010, Intel Corporation and its suppliers.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
*/
#include <linux/errno.h>
#define DoC_is_Millennium(doc) ((doc)->ChipID == DOC_ChipID_DocMil)
#define DoC_is_2000(doc) ((doc)->ChipID == DOC_ChipID_Doc2k)
-static void doc200x_hwcontrol(struct mtd_info *mtd, int cmd,
+static void doc200x_hwcontrol(struct nand_chip *this, int cmd,
unsigned int bitmask);
-static void doc200x_select_chip(struct mtd_info *mtd, int chip);
+static void doc200x_select_chip(struct nand_chip *this, int chip);
static int debug = 0;
module_param(debug, int, 0);
return ret;
}
-static void doc2000_write_byte(struct mtd_info *mtd, u_char datum)
+static void doc2000_write_byte(struct nand_chip *this, u_char datum)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
WriteDOC(datum, docptr, 2k_CDSN_IO);
}
-static u_char doc2000_read_byte(struct mtd_info *mtd)
+static u_char doc2000_read_byte(struct nand_chip *this)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
u_char ret;
return ret;
}
-static void doc2000_writebuf(struct mtd_info *mtd, const u_char *buf, int len)
+static void doc2000_writebuf(struct nand_chip *this, const u_char *buf,
+ int len)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
int i;
printk("\n");
}
-static void doc2000_readbuf(struct mtd_info *mtd, u_char *buf, int len)
+static void doc2000_readbuf(struct nand_chip *this, u_char *buf, int len)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
int i;
if (debug)
printk("readbuf of %d bytes: ", len);
- for (i = 0; i < len; i++) {
+ for (i = 0; i < len; i++)
buf[i] = ReadDOC(docptr, 2k_CDSN_IO + i);
- }
}
-static void doc2000_readbuf_dword(struct mtd_info *mtd, u_char *buf, int len)
+static void doc2000_readbuf_dword(struct nand_chip *this, u_char *buf, int len)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
int i;
struct doc_priv *doc = nand_get_controller_data(this);
uint16_t ret;
- doc200x_select_chip(mtd, nr);
- doc200x_hwcontrol(mtd, NAND_CMD_READID,
+ doc200x_select_chip(this, nr);
+ doc200x_hwcontrol(this, NAND_CMD_READID,
NAND_CTRL_CLE | NAND_CTRL_CHANGE);
- doc200x_hwcontrol(mtd, 0, NAND_CTRL_ALE | NAND_CTRL_CHANGE);
- doc200x_hwcontrol(mtd, NAND_CMD_NONE, NAND_NCE | NAND_CTRL_CHANGE);
+ doc200x_hwcontrol(this, 0, NAND_CTRL_ALE | NAND_CTRL_CHANGE);
+ doc200x_hwcontrol(this, NAND_CMD_NONE, NAND_NCE | NAND_CTRL_CHANGE);
/* We can't use dev_ready here, but at least we wait for the
* command to complete
*/
udelay(50);
- ret = this->read_byte(mtd) << 8;
- ret |= this->read_byte(mtd);
+ ret = this->legacy.read_byte(this) << 8;
+ ret |= this->legacy.read_byte(this);
if (doc->ChipID == DOC_ChipID_Doc2k && try_dword && !nr) {
/* First chip probe. See if we get same results by 32-bit access */
} ident;
void __iomem *docptr = doc->virtadr;
- doc200x_hwcontrol(mtd, NAND_CMD_READID,
+ doc200x_hwcontrol(this, NAND_CMD_READID,
NAND_CTRL_CLE | NAND_CTRL_CHANGE);
- doc200x_hwcontrol(mtd, 0, NAND_CTRL_ALE | NAND_CTRL_CHANGE);
- doc200x_hwcontrol(mtd, NAND_CMD_NONE,
+ doc200x_hwcontrol(this, 0, NAND_CTRL_ALE | NAND_CTRL_CHANGE);
+ doc200x_hwcontrol(this, NAND_CMD_NONE,
NAND_NCE | NAND_CTRL_CHANGE);
udelay(50);
ident.dword = readl(docptr + DoC_2k_CDSN_IO);
if (((ident.byte[0] << 8) | ident.byte[1]) == ret) {
pr_info("DiskOnChip 2000 responds to DWORD access\n");
- this->read_buf = &doc2000_readbuf_dword;
+ this->legacy.read_buf = &doc2000_readbuf_dword;
}
}
pr_debug("Detected %d chips per floor.\n", i);
}
-static int doc200x_wait(struct mtd_info *mtd, struct nand_chip *this)
+static int doc200x_wait(struct nand_chip *this)
{
struct doc_priv *doc = nand_get_controller_data(this);
DoC_WaitReady(doc);
nand_status_op(this, NULL);
DoC_WaitReady(doc);
- status = (int)this->read_byte(mtd);
+ status = (int)this->legacy.read_byte(this);
return status;
}
-static void doc2001_write_byte(struct mtd_info *mtd, u_char datum)
+static void doc2001_write_byte(struct nand_chip *this, u_char datum)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
WriteDOC(datum, docptr, WritePipeTerm);
}
-static u_char doc2001_read_byte(struct mtd_info *mtd)
+static u_char doc2001_read_byte(struct nand_chip *this)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
return ReadDOC(docptr, LastDataRead);
}
-static void doc2001_writebuf(struct mtd_info *mtd, const u_char *buf, int len)
+static void doc2001_writebuf(struct nand_chip *this, const u_char *buf, int len)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
int i;
WriteDOC(0x00, docptr, WritePipeTerm);
}
-static void doc2001_readbuf(struct mtd_info *mtd, u_char *buf, int len)
+static void doc2001_readbuf(struct nand_chip *this, u_char *buf, int len)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
int i;
buf[i] = ReadDOC(docptr, LastDataRead);
}
-static u_char doc2001plus_read_byte(struct mtd_info *mtd)
+static u_char doc2001plus_read_byte(struct nand_chip *this)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
u_char ret;
return ret;
}
-static void doc2001plus_writebuf(struct mtd_info *mtd, const u_char *buf, int len)
+static void doc2001plus_writebuf(struct nand_chip *this, const u_char *buf, int len)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
int i;
printk("\n");
}
-static void doc2001plus_readbuf(struct mtd_info *mtd, u_char *buf, int len)
+static void doc2001plus_readbuf(struct nand_chip *this, u_char *buf, int len)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
int i;
printk("\n");
}
-static void doc2001plus_select_chip(struct mtd_info *mtd, int chip)
+static void doc2001plus_select_chip(struct nand_chip *this, int chip)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
int floor = 0;
doc->curfloor = floor;
}
-static void doc200x_select_chip(struct mtd_info *mtd, int chip)
+static void doc200x_select_chip(struct nand_chip *this, int chip)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
int floor = 0;
chip -= (floor * doc->chips_per_floor);
/* 11.4.4 -- deassert CE before changing chip */
- doc200x_hwcontrol(mtd, NAND_CMD_NONE, 0 | NAND_CTRL_CHANGE);
+ doc200x_hwcontrol(this, NAND_CMD_NONE, 0 | NAND_CTRL_CHANGE);
WriteDOC(floor, docptr, FloorSelect);
WriteDOC(chip, docptr, CDSNDeviceSelect);
- doc200x_hwcontrol(mtd, NAND_CMD_NONE, NAND_NCE | NAND_CTRL_CHANGE);
+ doc200x_hwcontrol(this, NAND_CMD_NONE, NAND_NCE | NAND_CTRL_CHANGE);
doc->curchip = chip;
doc->curfloor = floor;
#define CDSN_CTRL_MSK (CDSN_CTRL_CE | CDSN_CTRL_CLE | CDSN_CTRL_ALE)
-static void doc200x_hwcontrol(struct mtd_info *mtd, int cmd,
+static void doc200x_hwcontrol(struct nand_chip *this, int cmd,
unsigned int ctrl)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
}
if (cmd != NAND_CMD_NONE) {
if (DoC_is_2000(doc))
- doc2000_write_byte(mtd, cmd);
+ doc2000_write_byte(this, cmd);
else
- doc2001_write_byte(mtd, cmd);
+ doc2001_write_byte(this, cmd);
}
}
-static void doc2001plus_command(struct mtd_info *mtd, unsigned command, int column, int page_addr)
+static void doc2001plus_command(struct nand_chip *this, unsigned command,
+ int column, int page_addr)
{
- struct nand_chip *this = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(this);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
return;
case NAND_CMD_RESET:
- if (this->dev_ready)
+ if (this->legacy.dev_ready)
break;
- udelay(this->chip_delay);
+ udelay(this->legacy.chip_delay);
WriteDOC(NAND_CMD_STATUS, docptr, Mplus_FlashCmd);
WriteDOC(0, docptr, Mplus_WritePipeTerm);
WriteDOC(0, docptr, Mplus_WritePipeTerm);
- while (!(this->read_byte(mtd) & 0x40)) ;
+ while (!(this->legacy.read_byte(this) & 0x40)) ;
return;
/* This applies to read commands */
* If we don't have access to the busy pin, we apply the given
* command delay
*/
- if (!this->dev_ready) {
- udelay(this->chip_delay);
+ if (!this->legacy.dev_ready) {
+ udelay(this->legacy.chip_delay);
return;
}
}
* any case on any machine. */
ndelay(100);
/* wait until command is processed */
- while (!this->dev_ready(mtd)) ;
+ while (!this->legacy.dev_ready(this)) ;
}
-static int doc200x_dev_ready(struct mtd_info *mtd)
+static int doc200x_dev_ready(struct nand_chip *this)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
}
}
-static int doc200x_block_bad(struct mtd_info *mtd, loff_t ofs)
+static int doc200x_block_bad(struct nand_chip *this, loff_t ofs)
{
/* This is our last resort if we couldn't find or create a BBT. Just
pretend all blocks are good. */
return 0;
}
-static void doc200x_enable_hwecc(struct mtd_info *mtd, int mode)
+static void doc200x_enable_hwecc(struct nand_chip *this, int mode)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
}
}
-static void doc2001plus_enable_hwecc(struct mtd_info *mtd, int mode)
+static void doc2001plus_enable_hwecc(struct nand_chip *this, int mode)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
}
/* This code is only called on write */
-static int doc200x_calculate_ecc(struct mtd_info *mtd, const u_char *dat, unsigned char *ecc_code)
+static int doc200x_calculate_ecc(struct nand_chip *this, const u_char *dat,
+ unsigned char *ecc_code)
{
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
int i;
return 0;
}
-static int doc200x_correct_data(struct mtd_info *mtd, u_char *dat,
+static int doc200x_correct_data(struct nand_chip *this, u_char *dat,
u_char *read_ecc, u_char *isnull)
{
int i, ret = 0;
- struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
void __iomem *docptr = doc->virtadr;
uint8_t calc_ecc[6];
struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
- this->read_byte = doc2000_read_byte;
- this->write_buf = doc2000_writebuf;
- this->read_buf = doc2000_readbuf;
+ this->legacy.read_byte = doc2000_read_byte;
+ this->legacy.write_buf = doc2000_writebuf;
+ this->legacy.read_buf = doc2000_readbuf;
doc->late_init = nftl_scan_bbt;
doc->CDSNControl = CDSN_CTRL_FLASH_IO | CDSN_CTRL_ECC_IO;
struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
- this->read_byte = doc2001_read_byte;
- this->write_buf = doc2001_writebuf;
- this->read_buf = doc2001_readbuf;
+ this->legacy.read_byte = doc2001_read_byte;
+ this->legacy.write_buf = doc2001_writebuf;
+ this->legacy.read_buf = doc2001_readbuf;
ReadDOC(doc->virtadr, ChipID);
ReadDOC(doc->virtadr, ChipID);
struct nand_chip *this = mtd_to_nand(mtd);
struct doc_priv *doc = nand_get_controller_data(this);
- this->read_byte = doc2001plus_read_byte;
- this->write_buf = doc2001plus_writebuf;
- this->read_buf = doc2001plus_readbuf;
+ this->legacy.read_byte = doc2001plus_read_byte;
+ this->legacy.write_buf = doc2001plus_writebuf;
+ this->legacy.read_buf = doc2001plus_readbuf;
doc->late_init = inftl_scan_bbt;
- this->cmd_ctrl = NULL;
+ this->legacy.cmd_ctrl = NULL;
this->select_chip = doc2001plus_select_chip;
- this->cmdfunc = doc2001plus_command;
+ this->legacy.cmdfunc = doc2001plus_command;
this->ecc.hwctl = doc2001plus_enable_hwecc;
doc->chips_per_floor = 1;
nand_set_controller_data(nand, doc);
nand->select_chip = doc200x_select_chip;
- nand->cmd_ctrl = doc200x_hwcontrol;
- nand->dev_ready = doc200x_dev_ready;
- nand->waitfunc = doc200x_wait;
- nand->block_bad = doc200x_block_bad;
+ nand->legacy.cmd_ctrl = doc200x_hwcontrol;
+ nand->legacy.dev_ready = doc200x_dev_ready;
+ nand->legacy.waitfunc = doc200x_wait;
+ nand->legacy.block_bad = doc200x_block_bad;
nand->ecc.hwctl = doc200x_enable_hwecc;
nand->ecc.calculate = doc200x_calculate_ecc;
nand->ecc.correct = doc200x_correct_data;
else
numchips = doc2001_init(mtd);
- if ((ret = nand_scan(mtd, numchips)) || (ret = doc->late_init(mtd))) {
+ if ((ret = nand_scan(nand, numchips)) || (ret = doc->late_init(mtd))) {
/* DBB note: i believe nand_release is necessary here, as
buffers may have been allocated in nand_base. Check with
Thomas. FIX ME! */
/* nand_release will call mtd_device_unregister, but we
haven't yet added it. This is handled without incident by
mtd_device_unregister, as far as I can tell. */
- nand_release(mtd);
+ nand_release(nand);
goto fail;
}
doc = nand_get_controller_data(nand);
nextmtd = doc->nextdoc;
- nand_release(mtd);
+ nand_release(nand);
iounmap(doc->virtadr);
release_mem_region(doc->physadr, DOC_IOREMAP_LEN);
free_rs(doc->rs_decoder);
+++ /dev/null
-/*
- * Copyright © 2012 Mike Dunn <mikedunn@newsguy.com>
- *
- * mtd nand driver for M-Systems DiskOnChip G4
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * Tested on the Palm Treo 680. The G4 is also present on Toshiba Portege, Asus
- * P526, some HTC smartphones (Wizard, Prophet, ...), O2 XDA Zinc, maybe others.
- * Should work on these as well. Let me know!
- *
- * TODO:
- *
- * Mechanism for management of password-protected areas
- *
- * Hamming ecc when reading oob only
- *
- * According to the M-Sys documentation, this device is also available in a
- * "dual-die" configuration having a 256MB capacity, but no mechanism for
- * detecting this variant is documented. Currently this driver assumes 128MB
- * capacity.
- *
- * Support for multiple cascaded devices ("floors"). Not sure which gadgets
- * contain multiple G4s in a cascaded configuration, if any.
- *
- */
-
-#include <linux/kernel.h>
-#include <linux/slab.h>
-#include <linux/init.h>
-#include <linux/string.h>
-#include <linux/sched.h>
-#include <linux/delay.h>
-#include <linux/module.h>
-#include <linux/export.h>
-#include <linux/platform_device.h>
-#include <linux/io.h>
-#include <linux/bitops.h>
-#include <linux/mtd/partitions.h>
-#include <linux/mtd/mtd.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/bch.h>
-#include <linux/bitrev.h>
-#include <linux/jiffies.h>
-
-/*
- * In "reliable mode" consecutive 2k pages are used in parallel (in some
- * fashion) to store the same data. The data can be read back from the
- * even-numbered pages in the normal manner; odd-numbered pages will appear to
- * contain junk. Systems that boot from the docg4 typically write the secondary
- * program loader (SPL) code in this mode. The SPL is loaded by the initial
- * program loader (IPL, stored in the docg4's 2k NOR-like region that is mapped
- * to the reset vector address). This module parameter enables you to use this
- * driver to write the SPL. When in this mode, no more than 2k of data can be
- * written at a time, because the addresses do not increment in the normal
- * manner, and the starting offset must be within an even-numbered 2k region;
- * i.e., invalid starting offsets are 0x800, 0xa00, 0xc00, 0xe00, 0x1800,
- * 0x1a00, ... Reliable mode is a special case and should not be used unless
- * you know what you're doing.
- */
-static bool reliable_mode;
-module_param(reliable_mode, bool, 0);
-MODULE_PARM_DESC(reliable_mode, "pages are programmed in reliable mode");
-
-/*
- * You'll want to ignore badblocks if you're reading a partition that contains
- * data written by the TrueFFS library (i.e., by PalmOS, Windows, etc), since
- * it does not use mtd nand's method for marking bad blocks (using oob area).
- * This will also skip the check of the "page written" flag.
- */
-static bool ignore_badblocks;
-module_param(ignore_badblocks, bool, 0);
-MODULE_PARM_DESC(ignore_badblocks, "no badblock checking performed");
-
-struct docg4_priv {
- struct mtd_info *mtd;
- struct device *dev;
- void __iomem *virtadr;
- int status;
- struct {
- unsigned int command;
- int column;
- int page;
- } last_command;
- uint8_t oob_buf[16];
- uint8_t ecc_buf[7];
- int oob_page;
- struct bch_control *bch;
-};
-
-/*
- * Defines prefixed with DOCG4 are unique to the diskonchip G4. All others are
- * shared with other diskonchip devices (P3, G3 at least).
- *
- * Functions with names prefixed with docg4_ are mtd / nand interface functions
- * (though they may also be called internally). All others are internal.
- */
-
-#define DOC_IOSPACE_DATA 0x0800
-
-/* register offsets */
-#define DOC_CHIPID 0x1000
-#define DOC_DEVICESELECT 0x100a
-#define DOC_ASICMODE 0x100c
-#define DOC_DATAEND 0x101e
-#define DOC_NOP 0x103e
-
-#define DOC_FLASHSEQUENCE 0x1032
-#define DOC_FLASHCOMMAND 0x1034
-#define DOC_FLASHADDRESS 0x1036
-#define DOC_FLASHCONTROL 0x1038
-#define DOC_ECCCONF0 0x1040
-#define DOC_ECCCONF1 0x1042
-#define DOC_HAMMINGPARITY 0x1046
-#define DOC_BCH_SYNDROM(idx) (0x1048 + idx)
-
-#define DOC_ASICMODECONFIRM 0x1072
-#define DOC_CHIPID_INV 0x1074
-#define DOC_POWERMODE 0x107c
-
-#define DOCG4_MYSTERY_REG 0x1050
-
-/* apparently used only to write oob bytes 6 and 7 */
-#define DOCG4_OOB_6_7 0x1052
-
-/* DOC_FLASHSEQUENCE register commands */
-#define DOC_SEQ_RESET 0x00
-#define DOCG4_SEQ_PAGE_READ 0x03
-#define DOCG4_SEQ_FLUSH 0x29
-#define DOCG4_SEQ_PAGEWRITE 0x16
-#define DOCG4_SEQ_PAGEPROG 0x1e
-#define DOCG4_SEQ_BLOCKERASE 0x24
-#define DOCG4_SEQ_SETMODE 0x45
-
-/* DOC_FLASHCOMMAND register commands */
-#define DOCG4_CMD_PAGE_READ 0x00
-#define DOC_CMD_ERASECYCLE2 0xd0
-#define DOCG4_CMD_FLUSH 0x70
-#define DOCG4_CMD_READ2 0x30
-#define DOC_CMD_PROG_BLOCK_ADDR 0x60
-#define DOCG4_CMD_PAGEWRITE 0x80
-#define DOC_CMD_PROG_CYCLE2 0x10
-#define DOCG4_CMD_FAST_MODE 0xa3 /* functionality guessed */
-#define DOC_CMD_RELIABLE_MODE 0x22
-#define DOC_CMD_RESET 0xff
-
-/* DOC_POWERMODE register bits */
-#define DOC_POWERDOWN_READY 0x80
-
-/* DOC_FLASHCONTROL register bits */
-#define DOC_CTRL_CE 0x10
-#define DOC_CTRL_UNKNOWN 0x40
-#define DOC_CTRL_FLASHREADY 0x01
-
-/* DOC_ECCCONF0 register bits */
-#define DOC_ECCCONF0_READ_MODE 0x8000
-#define DOC_ECCCONF0_UNKNOWN 0x2000
-#define DOC_ECCCONF0_ECC_ENABLE 0x1000
-#define DOC_ECCCONF0_DATA_BYTES_MASK 0x07ff
-
-/* DOC_ECCCONF1 register bits */
-#define DOC_ECCCONF1_BCH_SYNDROM_ERR 0x80
-#define DOC_ECCCONF1_ECC_ENABLE 0x07
-#define DOC_ECCCONF1_PAGE_IS_WRITTEN 0x20
-
-/* DOC_ASICMODE register bits */
-#define DOC_ASICMODE_RESET 0x00
-#define DOC_ASICMODE_NORMAL 0x01
-#define DOC_ASICMODE_POWERDOWN 0x02
-#define DOC_ASICMODE_MDWREN 0x04
-#define DOC_ASICMODE_BDETCT_RESET 0x08
-#define DOC_ASICMODE_RSTIN_RESET 0x10
-#define DOC_ASICMODE_RAM_WE 0x20
-
-/* good status values read after read/write/erase operations */
-#define DOCG4_PROGSTATUS_GOOD 0x51
-#define DOCG4_PROGSTATUS_GOOD_2 0xe0
-
-/*
- * On read operations (page and oob-only), the first byte read from I/O reg is a
- * status. On error, it reads 0x73; otherwise, it reads either 0x71 (first read
- * after reset only) or 0x51, so bit 1 is presumed to be an error indicator.
- */
-#define DOCG4_READ_ERROR 0x02 /* bit 1 indicates read error */
-
-/* anatomy of the device */
-#define DOCG4_CHIP_SIZE 0x8000000
-#define DOCG4_PAGE_SIZE 0x200
-#define DOCG4_PAGES_PER_BLOCK 0x200
-#define DOCG4_BLOCK_SIZE (DOCG4_PAGES_PER_BLOCK * DOCG4_PAGE_SIZE)
-#define DOCG4_NUMBLOCKS (DOCG4_CHIP_SIZE / DOCG4_BLOCK_SIZE)
-#define DOCG4_OOB_SIZE 0x10
-#define DOCG4_CHIP_SHIFT 27 /* log_2(DOCG4_CHIP_SIZE) */
-#define DOCG4_PAGE_SHIFT 9 /* log_2(DOCG4_PAGE_SIZE) */
-#define DOCG4_ERASE_SHIFT 18 /* log_2(DOCG4_BLOCK_SIZE) */
-
-/* all but the last byte is included in ecc calculation */
-#define DOCG4_BCH_SIZE (DOCG4_PAGE_SIZE + DOCG4_OOB_SIZE - 1)
-
-#define DOCG4_USERDATA_LEN 520 /* 512 byte page plus 8 oob avail to user */
-
-/* expected values from the ID registers */
-#define DOCG4_IDREG1_VALUE 0x0400
-#define DOCG4_IDREG2_VALUE 0xfbff
-
-/* primitive polynomial used to build the Galois field used by hw ecc gen */
-#define DOCG4_PRIMITIVE_POLY 0x4443
-
-#define DOCG4_M 14 /* Galois field is of order 2^14 */
-#define DOCG4_T 4 /* BCH alg corrects up to 4 bit errors */
-
-#define DOCG4_FACTORY_BBT_PAGE 16 /* page where read-only factory bbt lives */
-#define DOCG4_REDUNDANT_BBT_PAGE 24 /* page where redundant factory bbt lives */
-
-/*
- * Bytes 0, 1 are used as badblock marker.
- * Bytes 2 - 6 are available to the user.
- * Byte 7 is hamming ecc for first 7 oob bytes only.
- * Bytes 8 - 14 are hw-generated ecc covering entire page + oob bytes 0 - 14.
- * Byte 15 (the last) is used by the driver as a "page written" flag.
- */
-static int docg4_ooblayout_ecc(struct mtd_info *mtd, int section,
- struct mtd_oob_region *oobregion)
-{
- if (section)
- return -ERANGE;
-
- oobregion->offset = 7;
- oobregion->length = 9;
-
- return 0;
-}
-
-static int docg4_ooblayout_free(struct mtd_info *mtd, int section,
- struct mtd_oob_region *oobregion)
-{
- if (section)
- return -ERANGE;
-
- oobregion->offset = 2;
- oobregion->length = 5;
-
- return 0;
-}
-
-static const struct mtd_ooblayout_ops docg4_ooblayout_ops = {
- .ecc = docg4_ooblayout_ecc,
- .free = docg4_ooblayout_free,
-};
-
-/*
- * The device has a nop register which M-Sys claims is for the purpose of
- * inserting precise delays. But beware; at least some operations fail if the
- * nop writes are replaced with a generic delay!
- */
-static inline void write_nop(void __iomem *docptr)
-{
- writew(0, docptr + DOC_NOP);
-}
-
-static void docg4_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
-{
- int i;
- struct nand_chip *nand = mtd_to_nand(mtd);
- uint16_t *p = (uint16_t *) buf;
- len >>= 1;
-
- for (i = 0; i < len; i++)
- p[i] = readw(nand->IO_ADDR_R);
-}
-
-static void docg4_write_buf16(struct mtd_info *mtd, const uint8_t *buf, int len)
-{
- int i;
- struct nand_chip *nand = mtd_to_nand(mtd);
- uint16_t *p = (uint16_t *) buf;
- len >>= 1;
-
- for (i = 0; i < len; i++)
- writew(p[i], nand->IO_ADDR_W);
-}
-
-static int poll_status(struct docg4_priv *doc)
-{
- /*
- * Busy-wait for the FLASHREADY bit to be set in the FLASHCONTROL
- * register. Operations known to take a long time (e.g., block erase)
- * should sleep for a while before calling this.
- */
-
- uint16_t flash_status;
- unsigned long timeo;
- void __iomem *docptr = doc->virtadr;
-
- dev_dbg(doc->dev, "%s...\n", __func__);
-
- /* hardware quirk requires reading twice initially */
- flash_status = readw(docptr + DOC_FLASHCONTROL);
-
- timeo = jiffies + msecs_to_jiffies(200); /* generous timeout */
- do {
- cpu_relax();
- flash_status = readb(docptr + DOC_FLASHCONTROL);
- } while (!(flash_status & DOC_CTRL_FLASHREADY) &&
- time_before(jiffies, timeo));
-
- if (unlikely(!(flash_status & DOC_CTRL_FLASHREADY))) {
- dev_err(doc->dev, "%s: timed out!\n", __func__);
- return NAND_STATUS_FAIL;
- }
-
- return 0;
-}
-
-
-static int docg4_wait(struct mtd_info *mtd, struct nand_chip *nand)
-{
-
- struct docg4_priv *doc = nand_get_controller_data(nand);
- int status = NAND_STATUS_WP; /* inverse logic?? */
- dev_dbg(doc->dev, "%s...\n", __func__);
-
- /* report any previously unreported error */
- if (doc->status) {
- status |= doc->status;
- doc->status = 0;
- return status;
- }
-
- status |= poll_status(doc);
- return status;
-}
-
-static void docg4_select_chip(struct mtd_info *mtd, int chip)
-{
- /*
- * Select among multiple cascaded chips ("floors"). Multiple floors are
- * not yet supported, so the only valid non-negative value is 0.
- */
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
- void __iomem *docptr = doc->virtadr;
-
- dev_dbg(doc->dev, "%s: chip %d\n", __func__, chip);
-
- if (chip < 0)
- return; /* deselected */
-
- if (chip > 0)
- dev_warn(doc->dev, "multiple floors currently unsupported\n");
-
- writew(0, docptr + DOC_DEVICESELECT);
-}
-
-static void reset(struct mtd_info *mtd)
-{
- /* full device reset */
-
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
- void __iomem *docptr = doc->virtadr;
-
- writew(DOC_ASICMODE_RESET | DOC_ASICMODE_MDWREN,
- docptr + DOC_ASICMODE);
- writew(~(DOC_ASICMODE_RESET | DOC_ASICMODE_MDWREN),
- docptr + DOC_ASICMODECONFIRM);
- write_nop(docptr);
-
- writew(DOC_ASICMODE_NORMAL | DOC_ASICMODE_MDWREN,
- docptr + DOC_ASICMODE);
- writew(~(DOC_ASICMODE_NORMAL | DOC_ASICMODE_MDWREN),
- docptr + DOC_ASICMODECONFIRM);
-
- writew(DOC_ECCCONF1_ECC_ENABLE, docptr + DOC_ECCCONF1);
-
- poll_status(doc);
-}
-
-static void read_hw_ecc(void __iomem *docptr, uint8_t *ecc_buf)
-{
- /* read the 7 hw-generated ecc bytes */
-
- int i;
- for (i = 0; i < 7; i++) { /* hw quirk; read twice */
- ecc_buf[i] = readb(docptr + DOC_BCH_SYNDROM(i));
- ecc_buf[i] = readb(docptr + DOC_BCH_SYNDROM(i));
- }
-}
-
-static int correct_data(struct mtd_info *mtd, uint8_t *buf, int page)
-{
- /*
- * Called after a page read when hardware reports bitflips.
- * Up to four bitflips can be corrected.
- */
-
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
- void __iomem *docptr = doc->virtadr;
- int i, numerrs, errpos[4];
- const uint8_t blank_read_hwecc[8] = {
- 0xcf, 0x72, 0xfc, 0x1b, 0xa9, 0xc7, 0xb9, 0 };
-
- read_hw_ecc(docptr, doc->ecc_buf); /* read 7 hw-generated ecc bytes */
-
- /* check if read error is due to a blank page */
- if (!memcmp(doc->ecc_buf, blank_read_hwecc, 7))
- return 0; /* yes */
-
- /* skip additional check of "written flag" if ignore_badblocks */
- if (ignore_badblocks == false) {
-
- /*
- * If the hw ecc bytes are not those of a blank page, there's
- * still a chance that the page is blank, but was read with
- * errors. Check the "written flag" in last oob byte, which
- * is set to zero when a page is written. If more than half
- * the bits are set, assume a blank page. Unfortunately, the
- * bit flips(s) are not reported in stats.
- */
-
- if (nand->oob_poi[15]) {
- int bit, numsetbits = 0;
- unsigned long written_flag = nand->oob_poi[15];
- for_each_set_bit(bit, &written_flag, 8)
- numsetbits++;
- if (numsetbits > 4) { /* assume blank */
- dev_warn(doc->dev,
- "error(s) in blank page "
- "at offset %08x\n",
- page * DOCG4_PAGE_SIZE);
- return 0;
- }
- }
- }
-
- /*
- * The hardware ecc unit produces oob_ecc ^ calc_ecc. The kernel's bch
- * algorithm is used to decode this. However the hw operates on page
- * data in a bit order that is the reverse of that of the bch alg,
- * requiring that the bits be reversed on the result. Thanks to Ivan
- * Djelic for his analysis!
- */
- for (i = 0; i < 7; i++)
- doc->ecc_buf[i] = bitrev8(doc->ecc_buf[i]);
-
- numerrs = decode_bch(doc->bch, NULL, DOCG4_USERDATA_LEN, NULL,
- doc->ecc_buf, NULL, errpos);
-
- if (numerrs == -EBADMSG) {
- dev_warn(doc->dev, "uncorrectable errors at offset %08x\n",
- page * DOCG4_PAGE_SIZE);
- return -EBADMSG;
- }
-
- BUG_ON(numerrs < 0); /* -EINVAL, or anything other than -EBADMSG */
-
- /* undo last step in BCH alg (modulo mirroring not needed) */
- for (i = 0; i < numerrs; i++)
- errpos[i] = (errpos[i] & ~7)|(7-(errpos[i] & 7));
-
- /* fix the errors */
- for (i = 0; i < numerrs; i++) {
-
- /* ignore if error within oob ecc bytes */
- if (errpos[i] > DOCG4_USERDATA_LEN * 8)
- continue;
-
- /* if error within oob area preceeding ecc bytes... */
- if (errpos[i] > DOCG4_PAGE_SIZE * 8)
- change_bit(errpos[i] - DOCG4_PAGE_SIZE * 8,
- (unsigned long *)nand->oob_poi);
-
- else /* error in page data */
- change_bit(errpos[i], (unsigned long *)buf);
- }
-
- dev_notice(doc->dev, "%d error(s) corrected at offset %08x\n",
- numerrs, page * DOCG4_PAGE_SIZE);
-
- return numerrs;
-}
-
-static uint8_t docg4_read_byte(struct mtd_info *mtd)
-{
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
-
- dev_dbg(doc->dev, "%s\n", __func__);
-
- if (doc->last_command.command == NAND_CMD_STATUS) {
- int status;
-
- /*
- * Previous nand command was status request, so nand
- * infrastructure code expects to read the status here. If an
- * error occurred in a previous operation, report it.
- */
- doc->last_command.command = 0;
-
- if (doc->status) {
- status = doc->status;
- doc->status = 0;
- }
-
- /* why is NAND_STATUS_WP inverse logic?? */
- else
- status = NAND_STATUS_WP | NAND_STATUS_READY;
-
- return status;
- }
-
- dev_warn(doc->dev, "unexpected call to read_byte()\n");
-
- return 0;
-}
-
-static void write_addr(struct docg4_priv *doc, uint32_t docg4_addr)
-{
- /* write the four address bytes packed in docg4_addr to the device */
-
- void __iomem *docptr = doc->virtadr;
- writeb(docg4_addr & 0xff, docptr + DOC_FLASHADDRESS);
- docg4_addr >>= 8;
- writeb(docg4_addr & 0xff, docptr + DOC_FLASHADDRESS);
- docg4_addr >>= 8;
- writeb(docg4_addr & 0xff, docptr + DOC_FLASHADDRESS);
- docg4_addr >>= 8;
- writeb(docg4_addr & 0xff, docptr + DOC_FLASHADDRESS);
-}
-
-static int read_progstatus(struct docg4_priv *doc)
-{
- /*
- * This apparently checks the status of programming. Done after an
- * erasure, and after page data is written. On error, the status is
- * saved, to be later retrieved by the nand infrastructure code.
- */
- void __iomem *docptr = doc->virtadr;
-
- /* status is read from the I/O reg */
- uint16_t status1 = readw(docptr + DOC_IOSPACE_DATA);
- uint16_t status2 = readw(docptr + DOC_IOSPACE_DATA);
- uint16_t status3 = readw(docptr + DOCG4_MYSTERY_REG);
-
- dev_dbg(doc->dev, "docg4: %s: %02x %02x %02x\n",
- __func__, status1, status2, status3);
-
- if (status1 != DOCG4_PROGSTATUS_GOOD
- || status2 != DOCG4_PROGSTATUS_GOOD_2
- || status3 != DOCG4_PROGSTATUS_GOOD_2) {
- doc->status = NAND_STATUS_FAIL;
- dev_warn(doc->dev, "read_progstatus failed: "
- "%02x, %02x, %02x\n", status1, status2, status3);
- return -EIO;
- }
- return 0;
-}
-
-static int pageprog(struct mtd_info *mtd)
-{
- /*
- * Final step in writing a page. Writes the contents of its
- * internal buffer out to the flash array, or some such.
- */
-
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
- void __iomem *docptr = doc->virtadr;
- int retval = 0;
-
- dev_dbg(doc->dev, "docg4: %s\n", __func__);
-
- writew(DOCG4_SEQ_PAGEPROG, docptr + DOC_FLASHSEQUENCE);
- writew(DOC_CMD_PROG_CYCLE2, docptr + DOC_FLASHCOMMAND);
- write_nop(docptr);
- write_nop(docptr);
-
- /* Just busy-wait; usleep_range() slows things down noticeably. */
- poll_status(doc);
-
- writew(DOCG4_SEQ_FLUSH, docptr + DOC_FLASHSEQUENCE);
- writew(DOCG4_CMD_FLUSH, docptr + DOC_FLASHCOMMAND);
- writew(DOC_ECCCONF0_READ_MODE | 4, docptr + DOC_ECCCONF0);
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
-
- retval = read_progstatus(doc);
- writew(0, docptr + DOC_DATAEND);
- write_nop(docptr);
- poll_status(doc);
- write_nop(docptr);
-
- return retval;
-}
-
-static void sequence_reset(struct mtd_info *mtd)
-{
- /* common starting sequence for all operations */
-
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
- void __iomem *docptr = doc->virtadr;
-
- writew(DOC_CTRL_UNKNOWN | DOC_CTRL_CE, docptr + DOC_FLASHCONTROL);
- writew(DOC_SEQ_RESET, docptr + DOC_FLASHSEQUENCE);
- writew(DOC_CMD_RESET, docptr + DOC_FLASHCOMMAND);
- write_nop(docptr);
- write_nop(docptr);
- poll_status(doc);
- write_nop(docptr);
-}
-
-static void read_page_prologue(struct mtd_info *mtd, uint32_t docg4_addr)
-{
- /* first step in reading a page */
-
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
- void __iomem *docptr = doc->virtadr;
-
- dev_dbg(doc->dev,
- "docg4: %s: g4 page %08x\n", __func__, docg4_addr);
-
- sequence_reset(mtd);
-
- writew(DOCG4_SEQ_PAGE_READ, docptr + DOC_FLASHSEQUENCE);
- writew(DOCG4_CMD_PAGE_READ, docptr + DOC_FLASHCOMMAND);
- write_nop(docptr);
-
- write_addr(doc, docg4_addr);
-
- write_nop(docptr);
- writew(DOCG4_CMD_READ2, docptr + DOC_FLASHCOMMAND);
- write_nop(docptr);
- write_nop(docptr);
-
- poll_status(doc);
-}
-
-static void write_page_prologue(struct mtd_info *mtd, uint32_t docg4_addr)
-{
- /* first step in writing a page */
-
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
- void __iomem *docptr = doc->virtadr;
-
- dev_dbg(doc->dev,
- "docg4: %s: g4 addr: %x\n", __func__, docg4_addr);
- sequence_reset(mtd);
-
- if (unlikely(reliable_mode)) {
- writew(DOCG4_SEQ_SETMODE, docptr + DOC_FLASHSEQUENCE);
- writew(DOCG4_CMD_FAST_MODE, docptr + DOC_FLASHCOMMAND);
- writew(DOC_CMD_RELIABLE_MODE, docptr + DOC_FLASHCOMMAND);
- write_nop(docptr);
- }
-
- writew(DOCG4_SEQ_PAGEWRITE, docptr + DOC_FLASHSEQUENCE);
- writew(DOCG4_CMD_PAGEWRITE, docptr + DOC_FLASHCOMMAND);
- write_nop(docptr);
- write_addr(doc, docg4_addr);
- write_nop(docptr);
- write_nop(docptr);
- poll_status(doc);
-}
-
-static uint32_t mtd_to_docg4_address(int page, int column)
-{
- /*
- * Convert mtd address to format used by the device, 32 bit packed.
- *
- * Some notes on G4 addressing... The M-Sys documentation on this device
- * claims that pages are 2K in length, and indeed, the format of the
- * address used by the device reflects that. But within each page are
- * four 512 byte "sub-pages", each with its own oob data that is
- * read/written immediately after the 512 bytes of page data. This oob
- * data contains the ecc bytes for the preceeding 512 bytes.
- *
- * Rather than tell the mtd nand infrastructure that page size is 2k,
- * with four sub-pages each, we engage in a little subterfuge and tell
- * the infrastructure code that pages are 512 bytes in size. This is
- * done because during the course of reverse-engineering the device, I
- * never observed an instance where an entire 2K "page" was read or
- * written as a unit. Each "sub-page" is always addressed individually,
- * its data read/written, and ecc handled before the next "sub-page" is
- * addressed.
- *
- * This requires us to convert addresses passed by the mtd nand
- * infrastructure code to those used by the device.
- *
- * The address that is written to the device consists of four bytes: the
- * first two are the 2k page number, and the second is the index into
- * the page. The index is in terms of 16-bit half-words and includes
- * the preceeding oob data, so e.g., the index into the second
- * "sub-page" is 0x108, and the full device address of the start of mtd
- * page 0x201 is 0x00800108.
- */
- int g4_page = page / 4; /* device's 2K page */
- int g4_index = (page % 4) * 0x108 + column/2; /* offset into page */
- return (g4_page << 16) | g4_index; /* pack */
-}
-
-static void docg4_command(struct mtd_info *mtd, unsigned command, int column,
- int page_addr)
-{
- /* handle standard nand commands */
-
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
- uint32_t g4_addr = mtd_to_docg4_address(page_addr, column);
-
- dev_dbg(doc->dev, "%s %x, page_addr=%x, column=%x\n",
- __func__, command, page_addr, column);
-
- /*
- * Save the command and its arguments. This enables emulation of
- * standard flash devices, and also some optimizations.
- */
- doc->last_command.command = command;
- doc->last_command.column = column;
- doc->last_command.page = page_addr;
-
- switch (command) {
-
- case NAND_CMD_RESET:
- reset(mtd);
- break;
-
- case NAND_CMD_READ0:
- read_page_prologue(mtd, g4_addr);
- break;
-
- case NAND_CMD_STATUS:
- /* next call to read_byte() will expect a status */
- break;
-
- case NAND_CMD_SEQIN:
- if (unlikely(reliable_mode)) {
- uint16_t g4_page = g4_addr >> 16;
-
- /* writes to odd-numbered 2k pages are invalid */
- if (g4_page & 0x01)
- dev_warn(doc->dev,
- "invalid reliable mode address\n");
- }
-
- write_page_prologue(mtd, g4_addr);
-
- /* hack for deferred write of oob bytes */
- if (doc->oob_page == page_addr)
- memcpy(nand->oob_poi, doc->oob_buf, 16);
- break;
-
- case NAND_CMD_PAGEPROG:
- pageprog(mtd);
- break;
-
- /* we don't expect these, based on review of nand_base.c */
- case NAND_CMD_READOOB:
- case NAND_CMD_READID:
- case NAND_CMD_ERASE1:
- case NAND_CMD_ERASE2:
- dev_warn(doc->dev, "docg4_command: "
- "unexpected nand command 0x%x\n", command);
- break;
-
- }
-}
-
-static int read_page(struct mtd_info *mtd, struct nand_chip *nand,
- uint8_t *buf, int page, bool use_ecc)
-{
- struct docg4_priv *doc = nand_get_controller_data(nand);
- void __iomem *docptr = doc->virtadr;
- uint16_t status, edc_err, *buf16;
- int bits_corrected = 0;
-
- dev_dbg(doc->dev, "%s: page %08x\n", __func__, page);
-
- nand_read_page_op(nand, page, 0, NULL, 0);
-
- writew(DOC_ECCCONF0_READ_MODE |
- DOC_ECCCONF0_ECC_ENABLE |
- DOC_ECCCONF0_UNKNOWN |
- DOCG4_BCH_SIZE,
- docptr + DOC_ECCCONF0);
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
-
- /* the 1st byte from the I/O reg is a status; the rest is page data */
- status = readw(docptr + DOC_IOSPACE_DATA);
- if (status & DOCG4_READ_ERROR) {
- dev_err(doc->dev,
- "docg4_read_page: bad status: 0x%02x\n", status);
- writew(0, docptr + DOC_DATAEND);
- return -EIO;
- }
-
- dev_dbg(doc->dev, "%s: status = 0x%x\n", __func__, status);
-
- docg4_read_buf(mtd, buf, DOCG4_PAGE_SIZE); /* read the page data */
-
- /* this device always reads oob after page data */
- /* first 14 oob bytes read from I/O reg */
- docg4_read_buf(mtd, nand->oob_poi, 14);
-
- /* last 2 read from another reg */
- buf16 = (uint16_t *)(nand->oob_poi + 14);
- *buf16 = readw(docptr + DOCG4_MYSTERY_REG);
-
- write_nop(docptr);
-
- if (likely(use_ecc == true)) {
-
- /* read the register that tells us if bitflip(s) detected */
- edc_err = readw(docptr + DOC_ECCCONF1);
- edc_err = readw(docptr + DOC_ECCCONF1);
- dev_dbg(doc->dev, "%s: edc_err = 0x%02x\n", __func__, edc_err);
-
- /* If bitflips are reported, attempt to correct with ecc */
- if (edc_err & DOC_ECCCONF1_BCH_SYNDROM_ERR) {
- bits_corrected = correct_data(mtd, buf, page);
- if (bits_corrected == -EBADMSG)
- mtd->ecc_stats.failed++;
- else
- mtd->ecc_stats.corrected += bits_corrected;
- }
- }
-
- writew(0, docptr + DOC_DATAEND);
- if (bits_corrected == -EBADMSG) /* uncorrectable errors */
- return 0;
- return bits_corrected;
-}
-
-
-static int docg4_read_page_raw(struct mtd_info *mtd, struct nand_chip *nand,
- uint8_t *buf, int oob_required, int page)
-{
- return read_page(mtd, nand, buf, page, false);
-}
-
-static int docg4_read_page(struct mtd_info *mtd, struct nand_chip *nand,
- uint8_t *buf, int oob_required, int page)
-{
- return read_page(mtd, nand, buf, page, true);
-}
-
-static int docg4_read_oob(struct mtd_info *mtd, struct nand_chip *nand,
- int page)
-{
- struct docg4_priv *doc = nand_get_controller_data(nand);
- void __iomem *docptr = doc->virtadr;
- uint16_t status;
-
- dev_dbg(doc->dev, "%s: page %x\n", __func__, page);
-
- nand_read_page_op(nand, page, nand->ecc.size, NULL, 0);
-
- writew(DOC_ECCCONF0_READ_MODE | DOCG4_OOB_SIZE, docptr + DOC_ECCCONF0);
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
-
- /* the 1st byte from the I/O reg is a status; the rest is oob data */
- status = readw(docptr + DOC_IOSPACE_DATA);
- if (status & DOCG4_READ_ERROR) {
- dev_warn(doc->dev,
- "docg4_read_oob failed: status = 0x%02x\n", status);
- return -EIO;
- }
-
- dev_dbg(doc->dev, "%s: status = 0x%x\n", __func__, status);
-
- docg4_read_buf(mtd, nand->oob_poi, 16);
-
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
- writew(0, docptr + DOC_DATAEND);
- write_nop(docptr);
-
- return 0;
-}
-
-static int docg4_erase_block(struct mtd_info *mtd, int page)
-{
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
- void __iomem *docptr = doc->virtadr;
- uint16_t g4_page;
- int status;
-
- dev_dbg(doc->dev, "%s: page %04x\n", __func__, page);
-
- sequence_reset(mtd);
-
- writew(DOCG4_SEQ_BLOCKERASE, docptr + DOC_FLASHSEQUENCE);
- writew(DOC_CMD_PROG_BLOCK_ADDR, docptr + DOC_FLASHCOMMAND);
- write_nop(docptr);
-
- /* only 2 bytes of address are written to specify erase block */
- g4_page = (uint16_t)(page / 4); /* to g4's 2k page addressing */
- writeb(g4_page & 0xff, docptr + DOC_FLASHADDRESS);
- g4_page >>= 8;
- writeb(g4_page & 0xff, docptr + DOC_FLASHADDRESS);
- write_nop(docptr);
-
- /* start the erasure */
- writew(DOC_CMD_ERASECYCLE2, docptr + DOC_FLASHCOMMAND);
- write_nop(docptr);
- write_nop(docptr);
-
- usleep_range(500, 1000); /* erasure is long; take a snooze */
- poll_status(doc);
- writew(DOCG4_SEQ_FLUSH, docptr + DOC_FLASHSEQUENCE);
- writew(DOCG4_CMD_FLUSH, docptr + DOC_FLASHCOMMAND);
- writew(DOC_ECCCONF0_READ_MODE | 4, docptr + DOC_ECCCONF0);
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
- write_nop(docptr);
-
- read_progstatus(doc);
-
- writew(0, docptr + DOC_DATAEND);
- write_nop(docptr);
- poll_status(doc);
- write_nop(docptr);
-
- status = nand->waitfunc(mtd, nand);
- if (status < 0)
- return status;
-
- return status & NAND_STATUS_FAIL ? -EIO : 0;
-}
-
-static int write_page(struct mtd_info *mtd, struct nand_chip *nand,
- const uint8_t *buf, int page, bool use_ecc)
-{
- struct docg4_priv *doc = nand_get_controller_data(nand);
- void __iomem *docptr = doc->virtadr;
- uint8_t ecc_buf[8];
-
- dev_dbg(doc->dev, "%s...\n", __func__);
-
- nand_prog_page_begin_op(nand, page, 0, NULL, 0);
-
- writew(DOC_ECCCONF0_ECC_ENABLE |
- DOC_ECCCONF0_UNKNOWN |
- DOCG4_BCH_SIZE,
- docptr + DOC_ECCCONF0);
- write_nop(docptr);
-
- /* write the page data */
- docg4_write_buf16(mtd, buf, DOCG4_PAGE_SIZE);
-
- /* oob bytes 0 through 5 are written to I/O reg */
- docg4_write_buf16(mtd, nand->oob_poi, 6);
-
- /* oob byte 6 written to a separate reg */
- writew(nand->oob_poi[6], docptr + DOCG4_OOB_6_7);
-
- write_nop(docptr);
- write_nop(docptr);
-
- /* write hw-generated ecc bytes to oob */
- if (likely(use_ecc == true)) {
- /* oob byte 7 is hamming code */
- uint8_t hamming = readb(docptr + DOC_HAMMINGPARITY);
- hamming = readb(docptr + DOC_HAMMINGPARITY); /* 2nd read */
- writew(hamming, docptr + DOCG4_OOB_6_7);
- write_nop(docptr);
-
- /* read the 7 bch bytes from ecc regs */
- read_hw_ecc(docptr, ecc_buf);
- ecc_buf[7] = 0; /* clear the "page written" flag */
- }
-
- /* write user-supplied bytes to oob */
- else {
- writew(nand->oob_poi[7], docptr + DOCG4_OOB_6_7);
- write_nop(docptr);
- memcpy(ecc_buf, &nand->oob_poi[8], 8);
- }
-
- docg4_write_buf16(mtd, ecc_buf, 8);
- write_nop(docptr);
- write_nop(docptr);
- writew(0, docptr + DOC_DATAEND);
- write_nop(docptr);
-
- return nand_prog_page_end_op(nand);
-}
-
-static int docg4_write_page_raw(struct mtd_info *mtd, struct nand_chip *nand,
- const uint8_t *buf, int oob_required, int page)
-{
- return write_page(mtd, nand, buf, page, false);
-}
-
-static int docg4_write_page(struct mtd_info *mtd, struct nand_chip *nand,
- const uint8_t *buf, int oob_required, int page)
-{
- return write_page(mtd, nand, buf, page, true);
-}
-
-static int docg4_write_oob(struct mtd_info *mtd, struct nand_chip *nand,
- int page)
-{
- /*
- * Writing oob-only is not really supported, because MLC nand must write
- * oob bytes at the same time as page data. Nonetheless, we save the
- * oob buffer contents here, and then write it along with the page data
- * if the same page is subsequently written. This allows user space
- * utilities that write the oob data prior to the page data to work
- * (e.g., nandwrite). The disdvantage is that, if the intention was to
- * write oob only, the operation is quietly ignored. Also, oob can get
- * corrupted if two concurrent processes are running nandwrite.
- */
-
- /* note that bytes 7..14 are hw generated hamming/ecc and overwritten */
- struct docg4_priv *doc = nand_get_controller_data(nand);
- doc->oob_page = page;
- memcpy(doc->oob_buf, nand->oob_poi, 16);
- return 0;
-}
-
-static int __init read_factory_bbt(struct mtd_info *mtd)
-{
- /*
- * The device contains a read-only factory bad block table. Read it and
- * update the memory-based bbt accordingly.
- */
-
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
- uint32_t g4_addr = mtd_to_docg4_address(DOCG4_FACTORY_BBT_PAGE, 0);
- uint8_t *buf;
- int i, block;
- __u32 eccfailed_stats = mtd->ecc_stats.failed;
-
- buf = kzalloc(DOCG4_PAGE_SIZE, GFP_KERNEL);
- if (buf == NULL)
- return -ENOMEM;
-
- read_page_prologue(mtd, g4_addr);
- docg4_read_page(mtd, nand, buf, 0, DOCG4_FACTORY_BBT_PAGE);
-
- /*
- * If no memory-based bbt was created, exit. This will happen if module
- * parameter ignore_badblocks is set. Then why even call this function?
- * For an unknown reason, block erase always fails if it's the first
- * operation after device power-up. The above read ensures it never is.
- * Ugly, I know.
- */
- if (nand->bbt == NULL) /* no memory-based bbt */
- goto exit;
-
- if (mtd->ecc_stats.failed > eccfailed_stats) {
- /*
- * Whoops, an ecc failure ocurred reading the factory bbt.
- * It is stored redundantly, so we get another chance.
- */
- eccfailed_stats = mtd->ecc_stats.failed;
- docg4_read_page(mtd, nand, buf, 0, DOCG4_REDUNDANT_BBT_PAGE);
- if (mtd->ecc_stats.failed > eccfailed_stats) {
- dev_warn(doc->dev,
- "The factory bbt could not be read!\n");
- goto exit;
- }
- }
-
- /*
- * Parse factory bbt and update memory-based bbt. Factory bbt format is
- * simple: one bit per block, block numbers increase left to right (msb
- * to lsb). Bit clear means bad block.
- */
- for (i = block = 0; block < DOCG4_NUMBLOCKS; block += 8, i++) {
- int bitnum;
- unsigned long bits = ~buf[i];
- for_each_set_bit(bitnum, &bits, 8) {
- int badblock = block + 7 - bitnum;
- nand->bbt[badblock / 4] |=
- 0x03 << ((badblock % 4) * 2);
- mtd->ecc_stats.badblocks++;
- dev_notice(doc->dev, "factory-marked bad block: %d\n",
- badblock);
- }
- }
- exit:
- kfree(buf);
- return 0;
-}
-
-static int docg4_block_markbad(struct mtd_info *mtd, loff_t ofs)
-{
- /*
- * Mark a block as bad. Bad blocks are marked in the oob area of the
- * first page of the block. The default scan_bbt() in the nand
- * infrastructure code works fine for building the memory-based bbt
- * during initialization, as does the nand infrastructure function that
- * checks if a block is bad by reading the bbt. This function replaces
- * the nand default because writes to oob-only are not supported.
- */
-
- int ret, i;
- uint8_t *buf;
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
- struct nand_bbt_descr *bbtd = nand->badblock_pattern;
- int page = (int)(ofs >> nand->page_shift);
- uint32_t g4_addr = mtd_to_docg4_address(page, 0);
-
- dev_dbg(doc->dev, "%s: %08llx\n", __func__, ofs);
-
- if (unlikely(ofs & (DOCG4_BLOCK_SIZE - 1)))
- dev_warn(doc->dev, "%s: ofs %llx not start of block!\n",
- __func__, ofs);
-
- /* allocate blank buffer for page data */
- buf = kzalloc(DOCG4_PAGE_SIZE, GFP_KERNEL);
- if (buf == NULL)
- return -ENOMEM;
-
- /* write bit-wise negation of pattern to oob buffer */
- memset(nand->oob_poi, 0xff, mtd->oobsize);
- for (i = 0; i < bbtd->len; i++)
- nand->oob_poi[bbtd->offs + i] = ~bbtd->pattern[i];
-
- /* write first page of block */
- write_page_prologue(mtd, g4_addr);
- docg4_write_page(mtd, nand, buf, 1, page);
- ret = pageprog(mtd);
-
- kfree(buf);
-
- return ret;
-}
-
-static int docg4_block_neverbad(struct mtd_info *mtd, loff_t ofs)
-{
- /* only called when module_param ignore_badblocks is set */
- return 0;
-}
-
-static int docg4_suspend(struct platform_device *pdev, pm_message_t state)
-{
- /*
- * Put the device into "deep power-down" mode. Note that CE# must be
- * deasserted for this to take effect. The xscale, e.g., can be
- * configured to float this signal when the processor enters power-down,
- * and a suitable pull-up ensures its deassertion.
- */
-
- int i;
- uint8_t pwr_down;
- struct docg4_priv *doc = platform_get_drvdata(pdev);
- void __iomem *docptr = doc->virtadr;
-
- dev_dbg(doc->dev, "%s...\n", __func__);
-
- /* poll the register that tells us we're ready to go to sleep */
- for (i = 0; i < 10; i++) {
- pwr_down = readb(docptr + DOC_POWERMODE);
- if (pwr_down & DOC_POWERDOWN_READY)
- break;
- usleep_range(1000, 4000);
- }
-
- if (pwr_down & DOC_POWERDOWN_READY) {
- dev_err(doc->dev, "suspend failed; "
- "timeout polling DOC_POWERDOWN_READY\n");
- return -EIO;
- }
-
- writew(DOC_ASICMODE_POWERDOWN | DOC_ASICMODE_MDWREN,
- docptr + DOC_ASICMODE);
- writew(~(DOC_ASICMODE_POWERDOWN | DOC_ASICMODE_MDWREN),
- docptr + DOC_ASICMODECONFIRM);
-
- write_nop(docptr);
-
- return 0;
-}
-
-static int docg4_resume(struct platform_device *pdev)
-{
-
- /*
- * Exit power-down. Twelve consecutive reads of the address below
- * accomplishes this, assuming CE# has been asserted.
- */
-
- struct docg4_priv *doc = platform_get_drvdata(pdev);
- void __iomem *docptr = doc->virtadr;
- int i;
-
- dev_dbg(doc->dev, "%s...\n", __func__);
-
- for (i = 0; i < 12; i++)
- readb(docptr + 0x1fff);
-
- return 0;
-}
-
-static void init_mtd_structs(struct mtd_info *mtd)
-{
- /* initialize mtd and nand data structures */
-
- /*
- * Note that some of the following initializations are not usually
- * required within a nand driver because they are performed by the nand
- * infrastructure code as part of nand_scan(). In this case they need
- * to be initialized here because we skip call to nand_scan_ident() (the
- * first half of nand_scan()). The call to nand_scan_ident() could be
- * skipped because for this device the chip id is not read in the manner
- * of a standard nand device.
- */
-
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
-
- mtd->size = DOCG4_CHIP_SIZE;
- mtd->name = "Msys_Diskonchip_G4";
- mtd->writesize = DOCG4_PAGE_SIZE;
- mtd->erasesize = DOCG4_BLOCK_SIZE;
- mtd->oobsize = DOCG4_OOB_SIZE;
- mtd_set_ooblayout(mtd, &docg4_ooblayout_ops);
- nand->chipsize = DOCG4_CHIP_SIZE;
- nand->chip_shift = DOCG4_CHIP_SHIFT;
- nand->bbt_erase_shift = nand->phys_erase_shift = DOCG4_ERASE_SHIFT;
- nand->chip_delay = 20;
- nand->page_shift = DOCG4_PAGE_SHIFT;
- nand->pagemask = 0x3ffff;
- nand->badblockpos = NAND_LARGE_BADBLOCK_POS;
- nand->badblockbits = 8;
- nand->ecc.mode = NAND_ECC_HW_SYNDROME;
- nand->ecc.size = DOCG4_PAGE_SIZE;
- nand->ecc.prepad = 8;
- nand->ecc.bytes = 8;
- nand->ecc.strength = DOCG4_T;
- nand->options = NAND_BUSWIDTH_16 | NAND_NO_SUBPAGE_WRITE;
- nand->IO_ADDR_R = nand->IO_ADDR_W = doc->virtadr + DOC_IOSPACE_DATA;
- nand->controller = &nand->dummy_controller;
- nand_controller_init(nand->controller);
-
- /* methods */
- nand->cmdfunc = docg4_command;
- nand->waitfunc = docg4_wait;
- nand->select_chip = docg4_select_chip;
- nand->read_byte = docg4_read_byte;
- nand->block_markbad = docg4_block_markbad;
- nand->read_buf = docg4_read_buf;
- nand->write_buf = docg4_write_buf16;
- nand->erase = docg4_erase_block;
- nand->set_features = nand_get_set_features_notsupp;
- nand->get_features = nand_get_set_features_notsupp;
- nand->ecc.read_page = docg4_read_page;
- nand->ecc.write_page = docg4_write_page;
- nand->ecc.read_page_raw = docg4_read_page_raw;
- nand->ecc.write_page_raw = docg4_write_page_raw;
- nand->ecc.read_oob = docg4_read_oob;
- nand->ecc.write_oob = docg4_write_oob;
-
- /*
- * The way the nand infrastructure code is written, a memory-based bbt
- * is not created if NAND_SKIP_BBTSCAN is set. With no memory bbt,
- * nand->block_bad() is used. So when ignoring bad blocks, we skip the
- * scan and define a dummy block_bad() which always returns 0.
- */
- if (ignore_badblocks) {
- nand->options |= NAND_SKIP_BBTSCAN;
- nand->block_bad = docg4_block_neverbad;
- }
-
-}
-
-static int read_id_reg(struct mtd_info *mtd)
-{
- struct nand_chip *nand = mtd_to_nand(mtd);
- struct docg4_priv *doc = nand_get_controller_data(nand);
- void __iomem *docptr = doc->virtadr;
- uint16_t id1, id2;
-
- /* check for presence of g4 chip by reading id registers */
- id1 = readw(docptr + DOC_CHIPID);
- id1 = readw(docptr + DOCG4_MYSTERY_REG);
- id2 = readw(docptr + DOC_CHIPID_INV);
- id2 = readw(docptr + DOCG4_MYSTERY_REG);
-
- if (id1 == DOCG4_IDREG1_VALUE && id2 == DOCG4_IDREG2_VALUE) {
- dev_info(doc->dev,
- "NAND device: 128MiB Diskonchip G4 detected\n");
- return 0;
- }
-
- return -ENODEV;
-}
-
-static char const *part_probes[] = { "cmdlinepart", "saftlpart", NULL };
-
-static int docg4_attach_chip(struct nand_chip *chip)
-{
- struct mtd_info *mtd = nand_to_mtd(chip);
- struct docg4_priv *doc = (struct docg4_priv *)(chip + 1);
- int ret;
-
- init_mtd_structs(mtd);
-
- /* Initialize kernel BCH algorithm */
- doc->bch = init_bch(DOCG4_M, DOCG4_T, DOCG4_PRIMITIVE_POLY);
- if (!doc->bch)
- return -EINVAL;
-
- reset(mtd);
-
- ret = read_id_reg(mtd);
- if (ret)
- free_bch(doc->bch);
-
- return ret;
-}
-
-static void docg4_detach_chip(struct nand_chip *chip)
-{
- struct docg4_priv *doc = (struct docg4_priv *)(chip + 1);
-
- free_bch(doc->bch);
-}
-
-static const struct nand_controller_ops docg4_controller_ops = {
- .attach_chip = docg4_attach_chip,
- .detach_chip = docg4_detach_chip,
-};
-
-static int __init probe_docg4(struct platform_device *pdev)
-{
- struct mtd_info *mtd;
- struct nand_chip *nand;
- void __iomem *virtadr;
- struct docg4_priv *doc;
- int len, retval;
- struct resource *r;
- struct device *dev = &pdev->dev;
-
- r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (r == NULL) {
- dev_err(dev, "no io memory resource defined!\n");
- return -ENODEV;
- }
-
- virtadr = ioremap(r->start, resource_size(r));
- if (!virtadr) {
- dev_err(dev, "Diskonchip ioremap failed: %pR\n", r);
- return -EIO;
- }
-
- len = sizeof(struct nand_chip) + sizeof(struct docg4_priv);
- nand = kzalloc(len, GFP_KERNEL);
- if (nand == NULL) {
- retval = -ENOMEM;
- goto unmap;
- }
-
- mtd = nand_to_mtd(nand);
- doc = (struct docg4_priv *) (nand + 1);
- nand_set_controller_data(nand, doc);
- mtd->dev.parent = &pdev->dev;
- doc->virtadr = virtadr;
- doc->dev = dev;
- platform_set_drvdata(pdev, doc);
-
- /*
- * Running nand_scan() with maxchips == 0 will skip nand_scan_ident(),
- * which is a specific operation with this driver and done in the
- * ->attach_chip callback.
- */
- nand->dummy_controller.ops = &docg4_controller_ops;
- retval = nand_scan(mtd, 0);
- if (retval)
- goto free_nand;
-
- retval = read_factory_bbt(mtd);
- if (retval)
- goto cleanup_nand;
-
- retval = mtd_device_parse_register(mtd, part_probes, NULL, NULL, 0);
- if (retval)
- goto cleanup_nand;
-
- doc->mtd = mtd;
-
- return 0;
-
-cleanup_nand:
- nand_cleanup(nand);
-free_nand:
- kfree(nand);
-unmap:
- iounmap(virtadr);
-
- return retval;
-}
-
-static int __exit cleanup_docg4(struct platform_device *pdev)
-{
- struct docg4_priv *doc = platform_get_drvdata(pdev);
- nand_release(doc->mtd);
- kfree(mtd_to_nand(doc->mtd));
- iounmap(doc->virtadr);
- return 0;
-}
-
-static struct platform_driver docg4_driver = {
- .driver = {
- .name = "docg4",
- },
- .suspend = docg4_suspend,
- .resume = docg4_resume,
- .remove = __exit_p(cleanup_docg4),
-};
-
-module_platform_driver_probe(docg4_driver, probe_docg4);
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Mike Dunn");
-MODULE_DESCRIPTION("M-Systems DiskOnChip G4 device driver");
}
/* cmdfunc send commands to the FCM */
-static void fsl_elbc_cmdfunc(struct mtd_info *mtd, unsigned int command,
+static void fsl_elbc_cmdfunc(struct nand_chip *chip, unsigned int command,
int column, int page_addr)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct fsl_elbc_mtd *priv = nand_get_controller_data(chip);
struct fsl_lbc_ctrl *ctrl = priv->ctrl;
struct fsl_elbc_fcm_ctrl *elbc_fcm_ctrl = ctrl->nand;
}
}
-static void fsl_elbc_select_chip(struct mtd_info *mtd, int chip)
+static void fsl_elbc_select_chip(struct nand_chip *chip, int cs)
{
/* The hardware does not seem to support multiple
* chips per bank.
/*
* Write buf to the FCM Controller Data Buffer
*/
-static void fsl_elbc_write_buf(struct mtd_info *mtd, const u8 *buf, int len)
+static void fsl_elbc_write_buf(struct nand_chip *chip, const u8 *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct fsl_elbc_mtd *priv = nand_get_controller_data(chip);
struct fsl_elbc_fcm_ctrl *elbc_fcm_ctrl = priv->ctrl->nand;
unsigned int bufsize = mtd->writesize + mtd->oobsize;
* read a byte from either the FCM hardware buffer if it has any data left
* otherwise issue a command to read a single byte.
*/
-static u8 fsl_elbc_read_byte(struct mtd_info *mtd)
+static u8 fsl_elbc_read_byte(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct fsl_elbc_mtd *priv = nand_get_controller_data(chip);
struct fsl_elbc_fcm_ctrl *elbc_fcm_ctrl = priv->ctrl->nand;
/*
* Read from the FCM Controller Data Buffer
*/
-static void fsl_elbc_read_buf(struct mtd_info *mtd, u8 *buf, int len)
+static void fsl_elbc_read_buf(struct nand_chip *chip, u8 *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct fsl_elbc_mtd *priv = nand_get_controller_data(chip);
struct fsl_elbc_fcm_ctrl *elbc_fcm_ctrl = priv->ctrl->nand;
int avail;
/* This function is called after Program and Erase Operations to
* check for success or failure.
*/
-static int fsl_elbc_wait(struct mtd_info *mtd, struct nand_chip *chip)
+static int fsl_elbc_wait(struct nand_chip *chip)
{
struct fsl_elbc_mtd *priv = nand_get_controller_data(chip);
struct fsl_elbc_fcm_ctrl *elbc_fcm_ctrl = priv->ctrl->nand;
chip->chipsize);
dev_dbg(priv->dev, "fsl_elbc_init: nand->pagemask = %8x\n",
chip->pagemask);
- dev_dbg(priv->dev, "fsl_elbc_init: nand->chip_delay = %d\n",
- chip->chip_delay);
+ dev_dbg(priv->dev, "fsl_elbc_init: nand->legacy.chip_delay = %d\n",
+ chip->legacy.chip_delay);
dev_dbg(priv->dev, "fsl_elbc_init: nand->badblockpos = %d\n",
chip->badblockpos);
dev_dbg(priv->dev, "fsl_elbc_init: nand->chip_shift = %d\n",
.attach_chip = fsl_elbc_attach_chip,
};
-static int fsl_elbc_read_page(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int fsl_elbc_read_page(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct fsl_elbc_mtd *priv = nand_get_controller_data(chip);
struct fsl_lbc_ctrl *ctrl = priv->ctrl;
struct fsl_elbc_fcm_ctrl *elbc_fcm_ctrl = ctrl->nand;
nand_read_page_op(chip, page, 0, buf, mtd->writesize);
if (oob_required)
- fsl_elbc_read_buf(mtd, chip->oob_poi, mtd->oobsize);
+ fsl_elbc_read_buf(chip, chip->oob_poi, mtd->oobsize);
- if (fsl_elbc_wait(mtd, chip) & NAND_STATUS_FAIL)
+ if (fsl_elbc_wait(chip) & NAND_STATUS_FAIL)
mtd->ecc_stats.failed++;
return elbc_fcm_ctrl->max_bitflips;
/* ECC will be calculated automatically, and errors will be detected in
* waitfunc.
*/
-static int fsl_elbc_write_page(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page)
+static int fsl_elbc_write_page(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
nand_prog_page_begin_op(chip, page, 0, buf, mtd->writesize);
- fsl_elbc_write_buf(mtd, chip->oob_poi, mtd->oobsize);
+ fsl_elbc_write_buf(chip, chip->oob_poi, mtd->oobsize);
return nand_prog_page_end_op(chip);
}
/* ECC will be calculated automatically, and errors will be detected in
* waitfunc.
*/
-static int fsl_elbc_write_subpage(struct mtd_info *mtd, struct nand_chip *chip,
- uint32_t offset, uint32_t data_len,
- const uint8_t *buf, int oob_required, int page)
+static int fsl_elbc_write_subpage(struct nand_chip *chip, uint32_t offset,
+ uint32_t data_len, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
nand_prog_page_begin_op(chip, page, 0, NULL, 0);
- fsl_elbc_write_buf(mtd, buf, mtd->writesize);
- fsl_elbc_write_buf(mtd, chip->oob_poi, mtd->oobsize);
+ fsl_elbc_write_buf(chip, buf, mtd->writesize);
+ fsl_elbc_write_buf(chip, chip->oob_poi, mtd->oobsize);
return nand_prog_page_end_op(chip);
}
/* fill in nand_chip structure */
/* set up function call table */
- chip->read_byte = fsl_elbc_read_byte;
- chip->write_buf = fsl_elbc_write_buf;
- chip->read_buf = fsl_elbc_read_buf;
+ chip->legacy.read_byte = fsl_elbc_read_byte;
+ chip->legacy.write_buf = fsl_elbc_write_buf;
+ chip->legacy.read_buf = fsl_elbc_read_buf;
chip->select_chip = fsl_elbc_select_chip;
- chip->cmdfunc = fsl_elbc_cmdfunc;
- chip->waitfunc = fsl_elbc_wait;
- chip->set_features = nand_get_set_features_notsupp;
- chip->get_features = nand_get_set_features_notsupp;
+ chip->legacy.cmdfunc = fsl_elbc_cmdfunc;
+ chip->legacy.waitfunc = fsl_elbc_wait;
+ chip->legacy.set_features = nand_get_set_features_notsupp;
+ chip->legacy.get_features = nand_get_set_features_notsupp;
chip->bbt_td = &bbt_main_descr;
chip->bbt_md = &bbt_mirror_descr;
goto err;
priv->chip.controller->ops = &fsl_elbc_controller_ops;
- ret = nand_scan(mtd, 1);
+ ret = nand_scan(&priv->chip, 1);
if (ret)
goto err;
{
struct fsl_elbc_fcm_ctrl *elbc_fcm_ctrl = fsl_lbc_ctrl_dev->nand;
struct fsl_elbc_mtd *priv = dev_get_drvdata(&pdev->dev);
- struct mtd_info *mtd = nand_to_mtd(&priv->chip);
- nand_release(mtd);
+ nand_release(&priv->chip);
fsl_elbc_chip_remove(priv);
mutex_lock(&fsl_elbc_nand_mutex);
#include <linux/mtd/partitions.h>
#include <linux/mtd/nand_ecc.h>
#include <linux/fsl_ifc.h>
+#include <linux/iopoll.h>
#define ERR_BYTE 0xFF /* Value returned for read
bytes when read failed */
}
/* cmdfunc send commands to the IFC NAND Machine */
-static void fsl_ifc_cmdfunc(struct mtd_info *mtd, unsigned int command,
- int column, int page_addr) {
- struct nand_chip *chip = mtd_to_nand(mtd);
+static void fsl_ifc_cmdfunc(struct nand_chip *chip, unsigned int command,
+ int column, int page_addr) {
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct fsl_ifc_mtd *priv = nand_get_controller_data(chip);
struct fsl_ifc_ctrl *ctrl = priv->ctrl;
struct fsl_ifc_runtime __iomem *ifc = ctrl->rregs;
}
}
-static void fsl_ifc_select_chip(struct mtd_info *mtd, int chip)
+static void fsl_ifc_select_chip(struct nand_chip *chip, int cs)
{
/* The hardware does not seem to support multiple
* chips per bank.
/*
* Write buf to the IFC NAND Controller Data Buffer
*/
-static void fsl_ifc_write_buf(struct mtd_info *mtd, const u8 *buf, int len)
+static void fsl_ifc_write_buf(struct nand_chip *chip, const u8 *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct fsl_ifc_mtd *priv = nand_get_controller_data(chip);
unsigned int bufsize = mtd->writesize + mtd->oobsize;
* Read a byte from either the IFC hardware buffer
* read function for 8-bit buswidth
*/
-static uint8_t fsl_ifc_read_byte(struct mtd_info *mtd)
+static uint8_t fsl_ifc_read_byte(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct fsl_ifc_mtd *priv = nand_get_controller_data(chip);
unsigned int offset;
* Read two bytes from the IFC hardware buffer
* read function for 16-bit buswith
*/
-static uint8_t fsl_ifc_read_byte16(struct mtd_info *mtd)
+static uint8_t fsl_ifc_read_byte16(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct fsl_ifc_mtd *priv = nand_get_controller_data(chip);
uint16_t data;
/*
* Read from the IFC Controller Data Buffer
*/
-static void fsl_ifc_read_buf(struct mtd_info *mtd, u8 *buf, int len)
+static void fsl_ifc_read_buf(struct nand_chip *chip, u8 *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct fsl_ifc_mtd *priv = nand_get_controller_data(chip);
int avail;
* This function is called after Program and Erase Operations to
* check for success or failure.
*/
-static int fsl_ifc_wait(struct mtd_info *mtd, struct nand_chip *chip)
+static int fsl_ifc_wait(struct nand_chip *chip)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct fsl_ifc_mtd *priv = nand_get_controller_data(chip);
struct fsl_ifc_ctrl *ctrl = priv->ctrl;
struct fsl_ifc_runtime __iomem *ifc = ctrl->rregs;
return bitflips;
}
-static int fsl_ifc_read_page(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int fsl_ifc_read_page(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct fsl_ifc_mtd *priv = nand_get_controller_data(chip);
struct fsl_ifc_ctrl *ctrl = priv->ctrl;
struct fsl_ifc_nand_ctrl *nctrl = ifc_nand_ctrl;
nand_read_page_op(chip, page, 0, buf, mtd->writesize);
if (oob_required)
- fsl_ifc_read_buf(mtd, chip->oob_poi, mtd->oobsize);
+ fsl_ifc_read_buf(chip, chip->oob_poi, mtd->oobsize);
if (ctrl->nand_stat & IFC_NAND_EVTER_STAT_ECCER) {
if (!oob_required)
- fsl_ifc_read_buf(mtd, chip->oob_poi, mtd->oobsize);
+ fsl_ifc_read_buf(chip, chip->oob_poi, mtd->oobsize);
return check_erased_page(chip, buf);
}
/* ECC will be calculated automatically, and errors will be detected in
* waitfunc.
*/
-static int fsl_ifc_write_page(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page)
+static int fsl_ifc_write_page(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
nand_prog_page_begin_op(chip, page, 0, buf, mtd->writesize);
- fsl_ifc_write_buf(mtd, chip->oob_poi, mtd->oobsize);
+ fsl_ifc_write_buf(chip, chip->oob_poi, mtd->oobsize);
return nand_prog_page_end_op(chip);
}
chip->chipsize);
dev_dbg(priv->dev, "%s: nand->pagemask = %8x\n", __func__,
chip->pagemask);
- dev_dbg(priv->dev, "%s: nand->chip_delay = %d\n", __func__,
- chip->chip_delay);
+ dev_dbg(priv->dev, "%s: nand->legacy.chip_delay = %d\n", __func__,
+ chip->legacy.chip_delay);
dev_dbg(priv->dev, "%s: nand->badblockpos = %d\n", __func__,
chip->badblockpos);
dev_dbg(priv->dev, "%s: nand->chip_shift = %d\n", __func__,
.attach_chip = fsl_ifc_attach_chip,
};
-static void fsl_ifc_sram_init(struct fsl_ifc_mtd *priv)
+static int fsl_ifc_sram_init(struct fsl_ifc_mtd *priv)
{
struct fsl_ifc_ctrl *ctrl = priv->ctrl;
struct fsl_ifc_runtime __iomem *ifc_runtime = ctrl->rregs;
uint32_t csor = 0, csor_8k = 0, csor_ext = 0;
uint32_t cs = priv->bank;
+ if (ctrl->version < FSL_IFC_VERSION_1_1_0)
+ return 0;
+
+ if (ctrl->version > FSL_IFC_VERSION_1_1_0) {
+ u32 ncfgr, status;
+ int ret;
+
+ /* Trigger auto initialization */
+ ncfgr = ifc_in32(&ifc_runtime->ifc_nand.ncfgr);
+ ifc_out32(ncfgr | IFC_NAND_NCFGR_SRAM_INIT_EN, &ifc_runtime->ifc_nand.ncfgr);
+
+ /* Wait until done */
+ ret = readx_poll_timeout(ifc_in32, &ifc_runtime->ifc_nand.ncfgr,
+ status, !(status & IFC_NAND_NCFGR_SRAM_INIT_EN),
+ 10, IFC_TIMEOUT_MSECS * 1000);
+ if (ret)
+ dev_err(priv->dev, "Failed to initialize SRAM!\n");
+
+ return ret;
+ }
+
/* Save CSOR and CSOR_ext */
csor = ifc_in32(&ifc_global->csor_cs[cs].csor);
csor_ext = ifc_in32(&ifc_global->csor_cs[cs].csor_ext);
wait_event_timeout(ctrl->nand_wait, ctrl->nand_stat,
msecs_to_jiffies(IFC_TIMEOUT_MSECS));
- if (ctrl->nand_stat != IFC_NAND_EVTER_STAT_OPC)
+ if (ctrl->nand_stat != IFC_NAND_EVTER_STAT_OPC) {
pr_err("fsl-ifc: Failed to Initialise SRAM\n");
+ return -ETIMEDOUT;
+ }
/* Restore CSOR and CSOR_ext */
ifc_out32(csor, &ifc_global->csor_cs[cs].csor);
ifc_out32(csor_ext, &ifc_global->csor_cs[cs].csor_ext);
+
+ return 0;
}
static int fsl_ifc_chip_init(struct fsl_ifc_mtd *priv)
struct nand_chip *chip = &priv->chip;
struct mtd_info *mtd = nand_to_mtd(&priv->chip);
u32 csor;
+ int ret;
/* Fill in fsl_ifc_mtd structure */
mtd->dev.parent = priv->dev;
/* set up function call table */
if ((ifc_in32(&ifc_global->cspr_cs[priv->bank].cspr))
& CSPR_PORT_SIZE_16)
- chip->read_byte = fsl_ifc_read_byte16;
+ chip->legacy.read_byte = fsl_ifc_read_byte16;
else
- chip->read_byte = fsl_ifc_read_byte;
+ chip->legacy.read_byte = fsl_ifc_read_byte;
- chip->write_buf = fsl_ifc_write_buf;
- chip->read_buf = fsl_ifc_read_buf;
+ chip->legacy.write_buf = fsl_ifc_write_buf;
+ chip->legacy.read_buf = fsl_ifc_read_buf;
chip->select_chip = fsl_ifc_select_chip;
- chip->cmdfunc = fsl_ifc_cmdfunc;
- chip->waitfunc = fsl_ifc_wait;
- chip->set_features = nand_get_set_features_notsupp;
- chip->get_features = nand_get_set_features_notsupp;
+ chip->legacy.cmdfunc = fsl_ifc_cmdfunc;
+ chip->legacy.waitfunc = fsl_ifc_wait;
+ chip->legacy.set_features = nand_get_set_features_notsupp;
+ chip->legacy.get_features = nand_get_set_features_notsupp;
chip->bbt_td = &bbt_main_descr;
chip->bbt_md = &bbt_mirror_descr;
if (ifc_in32(&ifc_global->cspr_cs[priv->bank].cspr)
& CSPR_PORT_SIZE_16) {
- chip->read_byte = fsl_ifc_read_byte16;
+ chip->legacy.read_byte = fsl_ifc_read_byte16;
chip->options |= NAND_BUSWIDTH_16;
} else {
- chip->read_byte = fsl_ifc_read_byte;
+ chip->legacy.read_byte = fsl_ifc_read_byte;
}
chip->controller = &ifc_nand_ctrl->controller;
chip->ecc.algo = NAND_ECC_HAMMING;
}
- if (ctrl->version >= FSL_IFC_VERSION_1_1_0)
- fsl_ifc_sram_init(priv);
+ ret = fsl_ifc_sram_init(priv);
+ if (ret)
+ return ret;
/*
* As IFC version 2.0.0 has 16KB of internal SRAM as compared to older
goto err;
priv->chip.controller->ops = &fsl_ifc_controller_ops;
- ret = nand_scan(mtd, 1);
+ ret = nand_scan(&priv->chip, 1);
if (ret)
goto err;
static int fsl_ifc_nand_remove(struct platform_device *dev)
{
struct fsl_ifc_mtd *priv = dev_get_drvdata(&dev->dev);
- struct mtd_info *mtd = nand_to_mtd(&priv->chip);
- nand_release(mtd);
+ nand_release(&priv->chip);
fsl_ifc_chip_remove(priv);
mutex_lock(&fsl_ifc_nand_mutex);
chip);
}
-static int fun_chip_ready(struct mtd_info *mtd)
+static int fun_chip_ready(struct nand_chip *chip)
{
- struct fsl_upm_nand *fun = to_fsl_upm_nand(mtd);
+ struct fsl_upm_nand *fun = to_fsl_upm_nand(nand_to_mtd(chip));
if (gpio_get_value(fun->rnb_gpio[fun->mchip_number]))
return 1;
struct mtd_info *mtd = nand_to_mtd(&fun->chip);
int cnt = 1000000;
- while (--cnt && !fun_chip_ready(mtd))
+ while (--cnt && !fun_chip_ready(&fun->chip))
cpu_relax();
if (!cnt)
dev_err(fun->dev, "tired waiting for RNB\n");
}
}
-static void fun_cmd_ctrl(struct mtd_info *mtd, int cmd, unsigned int ctrl)
+static void fun_cmd_ctrl(struct nand_chip *chip, int cmd, unsigned int ctrl)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
- struct fsl_upm_nand *fun = to_fsl_upm_nand(mtd);
+ struct fsl_upm_nand *fun = to_fsl_upm_nand(nand_to_mtd(chip));
u32 mar;
if (!(ctrl & fun->last_ctrl)) {
mar = (cmd << (32 - fun->upm.width)) |
fun->mchip_offsets[fun->mchip_number];
- fsl_upm_run_pattern(&fun->upm, chip->IO_ADDR_R, mar);
+ fsl_upm_run_pattern(&fun->upm, chip->legacy.IO_ADDR_R, mar);
if (fun->wait_flags & FSL_UPM_WAIT_RUN_PATTERN)
fun_wait_rnb(fun);
}
-static void fun_select_chip(struct mtd_info *mtd, int mchip_nr)
+static void fun_select_chip(struct nand_chip *chip, int mchip_nr)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
- struct fsl_upm_nand *fun = to_fsl_upm_nand(mtd);
+ struct fsl_upm_nand *fun = to_fsl_upm_nand(nand_to_mtd(chip));
if (mchip_nr == -1) {
- chip->cmd_ctrl(mtd, NAND_CMD_NONE, 0 | NAND_CTRL_CHANGE);
+ chip->legacy.cmd_ctrl(chip, NAND_CMD_NONE, 0 | NAND_CTRL_CHANGE);
} else if (mchip_nr >= 0 && mchip_nr < NAND_MAX_CHIPS) {
fun->mchip_number = mchip_nr;
- chip->IO_ADDR_R = fun->io_base + fun->mchip_offsets[mchip_nr];
- chip->IO_ADDR_W = chip->IO_ADDR_R;
+ chip->legacy.IO_ADDR_R = fun->io_base + fun->mchip_offsets[mchip_nr];
+ chip->legacy.IO_ADDR_W = chip->legacy.IO_ADDR_R;
} else {
BUG();
}
}
-static uint8_t fun_read_byte(struct mtd_info *mtd)
+static uint8_t fun_read_byte(struct nand_chip *chip)
{
- struct fsl_upm_nand *fun = to_fsl_upm_nand(mtd);
+ struct fsl_upm_nand *fun = to_fsl_upm_nand(nand_to_mtd(chip));
- return in_8(fun->chip.IO_ADDR_R);
+ return in_8(fun->chip.legacy.IO_ADDR_R);
}
-static void fun_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void fun_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
{
- struct fsl_upm_nand *fun = to_fsl_upm_nand(mtd);
+ struct fsl_upm_nand *fun = to_fsl_upm_nand(nand_to_mtd(chip));
int i;
for (i = 0; i < len; i++)
- buf[i] = in_8(fun->chip.IO_ADDR_R);
+ buf[i] = in_8(fun->chip.legacy.IO_ADDR_R);
}
-static void fun_write_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
+static void fun_write_buf(struct nand_chip *chip, const uint8_t *buf, int len)
{
- struct fsl_upm_nand *fun = to_fsl_upm_nand(mtd);
+ struct fsl_upm_nand *fun = to_fsl_upm_nand(nand_to_mtd(chip));
int i;
for (i = 0; i < len; i++) {
- out_8(fun->chip.IO_ADDR_W, buf[i]);
+ out_8(fun->chip.legacy.IO_ADDR_W, buf[i]);
if (fun->wait_flags & FSL_UPM_WAIT_WRITE_BYTE)
fun_wait_rnb(fun);
}
int ret;
struct device_node *flash_np;
- fun->chip.IO_ADDR_R = fun->io_base;
- fun->chip.IO_ADDR_W = fun->io_base;
- fun->chip.cmd_ctrl = fun_cmd_ctrl;
- fun->chip.chip_delay = fun->chip_delay;
- fun->chip.read_byte = fun_read_byte;
- fun->chip.read_buf = fun_read_buf;
- fun->chip.write_buf = fun_write_buf;
+ fun->chip.legacy.IO_ADDR_R = fun->io_base;
+ fun->chip.legacy.IO_ADDR_W = fun->io_base;
+ fun->chip.legacy.cmd_ctrl = fun_cmd_ctrl;
+ fun->chip.legacy.chip_delay = fun->chip_delay;
+ fun->chip.legacy.read_byte = fun_read_byte;
+ fun->chip.legacy.read_buf = fun_read_buf;
+ fun->chip.legacy.write_buf = fun_write_buf;
fun->chip.ecc.mode = NAND_ECC_SOFT;
fun->chip.ecc.algo = NAND_ECC_HAMMING;
if (fun->mchip_count > 1)
fun->chip.select_chip = fun_select_chip;
if (fun->rnb_gpio[0] >= 0)
- fun->chip.dev_ready = fun_chip_ready;
+ fun->chip.legacy.dev_ready = fun_chip_ready;
mtd->dev.parent = fun->dev;
return -ENODEV;
nand_set_flash_node(&fun->chip, flash_np);
- mtd->name = kasprintf(GFP_KERNEL, "0x%llx.%s", (u64)io_res->start,
- flash_np->name);
+ mtd->name = kasprintf(GFP_KERNEL, "0x%llx.%pOFn", (u64)io_res->start,
+ flash_np);
if (!mtd->name) {
ret = -ENOMEM;
goto err;
}
- ret = nand_scan(mtd, fun->mchip_count);
+ ret = nand_scan(&fun->chip, fun->mchip_count);
if (ret)
goto err;
struct mtd_info *mtd = nand_to_mtd(&fun->chip);
int i;
- nand_release(mtd);
+ nand_release(&fun->chip);
kfree(mtd->name);
for (i = 0; i < fun->mchip_count; i++) {
return 0;
}
-static int fsmc_setup_data_interface(struct mtd_info *mtd, int csline,
+static int fsmc_setup_data_interface(struct nand_chip *nand, int csline,
const struct nand_data_interface *conf)
{
- struct nand_chip *nand = mtd_to_nand(mtd);
struct fsmc_nand_data *host = nand_get_controller_data(nand);
struct fsmc_nand_timings tims;
const struct nand_sdr_timings *sdrt;
/*
* fsmc_enable_hwecc - Enables Hardware ECC through FSMC registers
*/
-static void fsmc_enable_hwecc(struct mtd_info *mtd, int mode)
+static void fsmc_enable_hwecc(struct nand_chip *chip, int mode)
{
- struct fsmc_nand_data *host = mtd_to_fsmc(mtd);
+ struct fsmc_nand_data *host = mtd_to_fsmc(nand_to_mtd(chip));
writel_relaxed(readl(host->regs_va + FSMC_PC) & ~FSMC_ECCPLEN_256,
host->regs_va + FSMC_PC);
* FSMC. ECC is 13 bytes for 512 bytes of data (supports error correction up to
* max of 8-bits)
*/
-static int fsmc_read_hwecc_ecc4(struct mtd_info *mtd, const uint8_t *data,
+static int fsmc_read_hwecc_ecc4(struct nand_chip *chip, const uint8_t *data,
uint8_t *ecc)
{
- struct fsmc_nand_data *host = mtd_to_fsmc(mtd);
+ struct fsmc_nand_data *host = mtd_to_fsmc(nand_to_mtd(chip));
uint32_t ecc_tmp;
unsigned long deadline = jiffies + FSMC_BUSY_WAIT_TIMEOUT;
* FSMC. ECC is 3 bytes for 512 bytes of data (supports error correction up to
* max of 1-bit)
*/
-static int fsmc_read_hwecc_ecc1(struct mtd_info *mtd, const uint8_t *data,
+static int fsmc_read_hwecc_ecc1(struct nand_chip *chip, const uint8_t *data,
uint8_t *ecc)
{
- struct fsmc_nand_data *host = mtd_to_fsmc(mtd);
+ struct fsmc_nand_data *host = mtd_to_fsmc(nand_to_mtd(chip));
uint32_t ecc_tmp;
ecc_tmp = readl_relaxed(host->regs_va + ECC1);
}
/* fsmc_select_chip - assert or deassert nCE */
-static void fsmc_select_chip(struct mtd_info *mtd, int chipnr)
+static void fsmc_select_chip(struct nand_chip *chip, int chipnr)
{
- struct fsmc_nand_data *host = mtd_to_fsmc(mtd);
+ struct fsmc_nand_data *host = mtd_to_fsmc(nand_to_mtd(chip));
u32 pc;
/* Support only one CS */
/*
* fsmc_read_page_hwecc
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: buffer to store read data
* @oob_required: caller expects OOB data read to chip->oob_poi
* After this read, fsmc hardware generates and reports error data bits(up to a
* max of 8 bits)
*/
-static int fsmc_read_page_hwecc(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int fsmc_read_page_hwecc(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int i, j, s, stat, eccsize = chip->ecc.size;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
for (i = 0, s = 0; s < eccsteps; s++, i += eccbytes, p += eccsize) {
nand_read_page_op(chip, page, s * eccsize, NULL, 0);
- chip->ecc.hwctl(mtd, NAND_ECC_READ);
+ chip->ecc.hwctl(chip, NAND_ECC_READ);
nand_read_data_op(chip, p, eccsize, false);
for (j = 0; j < eccbytes;) {
}
memcpy(&ecc_code[i], oob, chip->ecc.bytes);
- chip->ecc.calculate(mtd, p, &ecc_calc[i]);
+ chip->ecc.calculate(chip, p, &ecc_calc[i]);
- stat = chip->ecc.correct(mtd, p, &ecc_code[i], &ecc_calc[i]);
+ stat = chip->ecc.correct(chip, p, &ecc_code[i], &ecc_calc[i]);
if (stat < 0) {
mtd->ecc_stats.failed++;
} else {
* calc_ecc is a 104 bit information containing maximum of 8 error
* offset informations of 13 bits each in 512 bytes of read data.
*/
-static int fsmc_bch8_correct_data(struct mtd_info *mtd, uint8_t *dat,
- uint8_t *read_ecc, uint8_t *calc_ecc)
+static int fsmc_bch8_correct_data(struct nand_chip *chip, uint8_t *dat,
+ uint8_t *read_ecc, uint8_t *calc_ecc)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
- struct fsmc_nand_data *host = mtd_to_fsmc(mtd);
+ struct fsmc_nand_data *host = mtd_to_fsmc(nand_to_mtd(chip));
uint32_t err_idx[8];
uint32_t num_err, i;
uint32_t ecc1, ecc2, ecc3, ecc4;
nand->ecc.correct = nand_correct_data;
nand->ecc.bytes = 3;
nand->ecc.strength = 1;
+ nand->ecc.options |= NAND_ECC_SOFT_HAMMING_SM_ORDER;
break;
case NAND_ECC_SOFT:
mtd->dev.parent = &pdev->dev;
nand->exec_op = fsmc_exec_op;
nand->select_chip = fsmc_select_chip;
- nand->chip_delay = 30;
/*
* Setup default ECC mode. nand_dt_init() called from nand_scan_ident()
* Scan to find existence of the device
*/
nand->dummy_controller.ops = &fsmc_nand_controller_ops;
- ret = nand_scan(mtd, 1);
+ ret = nand_scan(nand, 1);
if (ret)
goto release_dma_write_chan;
struct fsmc_nand_data *host = platform_get_drvdata(pdev);
if (host) {
- nand_release(nand_to_mtd(&host->nand));
+ nand_release(&host->nand);
if (host->mode == USE_DMA_ACCESS) {
dma_release_channel(host->write_dma_chan);
static inline void gpio_nand_dosync(struct gpiomtd *gpiomtd) {}
#endif
-static void gpio_nand_cmd_ctrl(struct mtd_info *mtd, int cmd, unsigned int ctrl)
+static void gpio_nand_cmd_ctrl(struct nand_chip *chip, int cmd,
+ unsigned int ctrl)
{
- struct gpiomtd *gpiomtd = gpio_nand_getpriv(mtd);
+ struct gpiomtd *gpiomtd = gpio_nand_getpriv(nand_to_mtd(chip));
gpio_nand_dosync(gpiomtd);
if (cmd == NAND_CMD_NONE)
return;
- writeb(cmd, gpiomtd->nand_chip.IO_ADDR_W);
+ writeb(cmd, gpiomtd->nand_chip.legacy.IO_ADDR_W);
gpio_nand_dosync(gpiomtd);
}
-static int gpio_nand_devready(struct mtd_info *mtd)
+static int gpio_nand_devready(struct nand_chip *chip)
{
- struct gpiomtd *gpiomtd = gpio_nand_getpriv(mtd);
+ struct gpiomtd *gpiomtd = gpio_nand_getpriv(nand_to_mtd(chip));
return gpiod_get_value(gpiomtd->rdy);
}
{
struct gpiomtd *gpiomtd = platform_get_drvdata(pdev);
- nand_release(nand_to_mtd(&gpiomtd->nand_chip));
+ nand_release(&gpiomtd->nand_chip);
/* Enable write protection and disable the chip */
if (gpiomtd->nwp && !IS_ERR(gpiomtd->nwp))
chip = &gpiomtd->nand_chip;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- chip->IO_ADDR_R = devm_ioremap_resource(dev, res);
- if (IS_ERR(chip->IO_ADDR_R))
- return PTR_ERR(chip->IO_ADDR_R);
+ chip->legacy.IO_ADDR_R = devm_ioremap_resource(dev, res);
+ if (IS_ERR(chip->legacy.IO_ADDR_R))
+ return PTR_ERR(chip->legacy.IO_ADDR_R);
res = gpio_nand_get_io_sync(pdev);
if (res) {
}
/* Using RDY pin */
if (gpiomtd->rdy)
- chip->dev_ready = gpio_nand_devready;
+ chip->legacy.dev_ready = gpio_nand_devready;
nand_set_flash_node(chip, pdev->dev.of_node);
- chip->IO_ADDR_W = chip->IO_ADDR_R;
+ chip->legacy.IO_ADDR_W = chip->legacy.IO_ADDR_R;
chip->ecc.mode = NAND_ECC_SOFT;
chip->ecc.algo = NAND_ECC_HAMMING;
chip->options = gpiomtd->plat.options;
- chip->chip_delay = gpiomtd->plat.chip_delay;
- chip->cmd_ctrl = gpio_nand_cmd_ctrl;
+ chip->legacy.chip_delay = gpiomtd->plat.chip_delay;
+ chip->legacy.cmd_ctrl = gpio_nand_cmd_ctrl;
mtd = nand_to_mtd(chip);
mtd->dev.parent = dev;
if (gpiomtd->nwp && !IS_ERR(gpiomtd->nwp))
gpiod_direction_output(gpiomtd->nwp, 1);
- ret = nand_scan(mtd, 1);
+ ret = nand_scan(chip, 1);
if (ret)
goto err_wp;
udelay(dll_wait_time_us);
}
-int gpmi_setup_data_interface(struct mtd_info *mtd, int chipnr,
+int gpmi_setup_data_interface(struct nand_chip *chip, int chipnr,
const struct nand_data_interface *conf)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct gpmi_nand_data *this = nand_get_controller_data(chip);
const struct nand_sdr_timings *sdr;
return -ENOMEM;
}
-static void gpmi_cmd_ctrl(struct mtd_info *mtd, int data, unsigned int ctrl)
+static void gpmi_cmd_ctrl(struct nand_chip *chip, int data, unsigned int ctrl)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct gpmi_nand_data *this = nand_get_controller_data(chip);
int ret;
this->command_length = 0;
}
-static int gpmi_dev_ready(struct mtd_info *mtd)
+static int gpmi_dev_ready(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct gpmi_nand_data *this = nand_get_controller_data(chip);
return gpmi_is_ready(this, this->current_chip);
}
-static void gpmi_select_chip(struct mtd_info *mtd, int chipnr)
+static void gpmi_select_chip(struct nand_chip *chip, int chipnr)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct gpmi_nand_data *this = nand_get_controller_data(chip);
int ret;
this->current_chip = chipnr;
}
-static void gpmi_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void gpmi_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct gpmi_nand_data *this = nand_get_controller_data(chip);
dev_dbg(this->dev, "len is %d\n", len);
gpmi_read_data(this, buf, len);
}
-static void gpmi_write_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
+static void gpmi_write_buf(struct nand_chip *chip, const uint8_t *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct gpmi_nand_data *this = nand_get_controller_data(chip);
dev_dbg(this->dev, "len is %d\n", len);
gpmi_send_data(this, buf, len);
}
-static uint8_t gpmi_read_byte(struct mtd_info *mtd)
+static uint8_t gpmi_read_byte(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct gpmi_nand_data *this = nand_get_controller_data(chip);
uint8_t *buf = this->data_buffer_dma;
- gpmi_read_buf(mtd, buf, 1);
+ gpmi_read_buf(chip, buf, 1);
return buf[0];
}
return max_bitflips;
}
-static int gpmi_ecc_read_page(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int gpmi_ecc_read_page(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
nand_read_page_op(chip, page, 0, NULL, 0);
}
/* Fake a virtual small page for the subpage read */
-static int gpmi_ecc_read_subpage(struct mtd_info *mtd, struct nand_chip *chip,
- uint32_t offs, uint32_t len, uint8_t *buf, int page)
+static int gpmi_ecc_read_subpage(struct nand_chip *chip, uint32_t offs,
+ uint32_t len, uint8_t *buf, int page)
{
struct gpmi_nand_data *this = nand_get_controller_data(chip);
void __iomem *bch_regs = this->resources.bch_regs;
dev_dbg(this->dev,
"page:%d, first:%d, last:%d, marker at:%d\n",
page, first, last, marker_pos);
- return gpmi_ecc_read_page(mtd, chip, buf, 0, page);
+ return gpmi_ecc_read_page(chip, buf, 0, page);
}
}
return max_bitflips;
}
-static int gpmi_ecc_write_page(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page)
+static int gpmi_ecc_write_page(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct gpmi_nand_data *this = nand_get_controller_data(chip);
struct bch_geometry *nfc_geo = &this->bch_geometry;
const void *payload_virt;
* ECC-based or raw view of the page is implicit in which function it calls
* (there is a similar pair of ECC-based/raw functions for writing).
*/
-static int gpmi_ecc_read_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int gpmi_ecc_read_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct gpmi_nand_data *this = nand_get_controller_data(chip);
dev_dbg(this->dev, "page number is %d\n", page);
/* Read out the conventional OOB. */
nand_read_page_op(chip, page, mtd->writesize, NULL, 0);
- chip->read_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.read_buf(chip, chip->oob_poi, mtd->oobsize);
/*
* Now, we want to make sure the block mark is correct. In the
if (GPMI_IS_MX23(this)) {
/* Read the block mark into the first byte of the OOB buffer. */
nand_read_page_op(chip, page, 0, NULL, 0);
- chip->oob_poi[0] = chip->read_byte(mtd);
+ chip->oob_poi[0] = chip->legacy.read_byte(chip);
}
return 0;
}
-static int
-gpmi_ecc_write_oob(struct mtd_info *mtd, struct nand_chip *chip, int page)
+static int gpmi_ecc_write_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct mtd_oob_region of = { };
/* Do we have available oob area? */
* See set_geometry_by_ecc_info inline comments to have a full description
* of the layout used by the GPMI controller.
*/
-static int gpmi_ecc_read_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip, uint8_t *buf,
+static int gpmi_ecc_read_page_raw(struct nand_chip *chip, uint8_t *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct gpmi_nand_data *this = nand_get_controller_data(chip);
struct bch_geometry *nfc_geo = &this->bch_geometry;
int eccsize = nfc_geo->ecc_chunk_size;
* See set_geometry_by_ecc_info inline comments to have a full description
* of the layout used by the GPMI controller.
*/
-static int gpmi_ecc_write_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip,
- const uint8_t *buf,
+static int gpmi_ecc_write_page_raw(struct nand_chip *chip, const uint8_t *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct gpmi_nand_data *this = nand_get_controller_data(chip);
struct bch_geometry *nfc_geo = &this->bch_geometry;
int eccsize = nfc_geo->ecc_chunk_size;
mtd->writesize + mtd->oobsize);
}
-static int gpmi_ecc_read_oob_raw(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int gpmi_ecc_read_oob_raw(struct nand_chip *chip, int page)
{
- return gpmi_ecc_read_page_raw(mtd, chip, NULL, 1, page);
+ return gpmi_ecc_read_page_raw(chip, NULL, 1, page);
}
-static int gpmi_ecc_write_oob_raw(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int gpmi_ecc_write_oob_raw(struct nand_chip *chip, int page)
{
- return gpmi_ecc_write_page_raw(mtd, chip, NULL, 1, page);
+ return gpmi_ecc_write_page_raw(chip, NULL, 1, page);
}
-static int gpmi_block_markbad(struct mtd_info *mtd, loff_t ofs)
+static int gpmi_block_markbad(struct nand_chip *chip, loff_t ofs)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct gpmi_nand_data *this = nand_get_controller_data(chip);
int ret = 0;
uint8_t *block_mark;
int column, page, chipnr;
chipnr = (int)(ofs >> chip->chip_shift);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
column = !GPMI_IS_MX23(this) ? mtd->writesize : 0;
ret = nand_prog_page_op(chip, page, column, block_mark, 1);
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
return ret;
}
struct boot_rom_geometry *rom_geo = &this->rom_geometry;
struct device *dev = this->dev;
struct nand_chip *chip = &this->nand;
- struct mtd_info *mtd = nand_to_mtd(chip);
unsigned int search_area_size_in_strides;
unsigned int stride;
unsigned int page;
search_area_size_in_strides = 1 << rom_geo->search_area_stride_exponent;
saved_chip_number = this->current_chip;
- chip->select_chip(mtd, 0);
+ chip->select_chip(chip, 0);
/*
* Loop through the first search area, looking for the NCB fingerprint.
* and starts in the 12th byte of the page.
*/
nand_read_page_op(chip, page, 12, NULL, 0);
- chip->read_buf(mtd, buffer, strlen(fingerprint));
+ chip->legacy.read_buf(chip, buffer, strlen(fingerprint));
/* Look for the fingerprint. */
if (!memcmp(buffer, fingerprint, strlen(fingerprint))) {
}
- chip->select_chip(mtd, saved_chip_number);
+ chip->select_chip(chip, saved_chip_number);
if (found_an_ncb_fingerprint)
dev_dbg(dev, "\tFound a fingerprint\n");
/* Select chip 0. */
saved_chip_number = this->current_chip;
- chip->select_chip(mtd, 0);
+ chip->select_chip(chip, 0);
/* Loop over blocks in the first search area, erasing them. */
dev_dbg(dev, "Erasing the search area...\n");
/* Write the first page of the current stride. */
dev_dbg(dev, "Writing an NCB fingerprint in page 0x%x\n", page);
- status = chip->ecc.write_page_raw(mtd, chip, buffer, 0, page);
+ status = chip->ecc.write_page_raw(chip, buffer, 0, page);
if (status)
dev_err(dev, "[%s] Write failed.\n", __func__);
}
/* Deselect chip 0. */
- chip->select_chip(mtd, saved_chip_number);
+ chip->select_chip(chip, saved_chip_number);
return 0;
}
byte = block << chip->phys_erase_shift;
/* Send the command to read the conventional block mark. */
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
nand_read_page_op(chip, page, mtd->writesize, NULL, 0);
- block_mark = chip->read_byte(mtd);
- chip->select_chip(mtd, -1);
+ block_mark = chip->legacy.read_byte(chip);
+ chip->select_chip(chip, -1);
/*
* Check if the block is marked bad. If so, we need to mark it
*/
if (block_mark != 0xff) {
dev_dbg(dev, "Transcribing mark in block %u\n", block);
- ret = chip->block_markbad(mtd, byte);
+ ret = chip->legacy.block_markbad(chip, byte);
if (ret)
dev_err(dev,
"Failed to mark block bad with ret %d\n",
nand_set_flash_node(chip, this->pdev->dev.of_node);
chip->select_chip = gpmi_select_chip;
chip->setup_data_interface = gpmi_setup_data_interface;
- chip->cmd_ctrl = gpmi_cmd_ctrl;
- chip->dev_ready = gpmi_dev_ready;
- chip->read_byte = gpmi_read_byte;
- chip->read_buf = gpmi_read_buf;
- chip->write_buf = gpmi_write_buf;
+ chip->legacy.cmd_ctrl = gpmi_cmd_ctrl;
+ chip->legacy.dev_ready = gpmi_dev_ready;
+ chip->legacy.read_byte = gpmi_read_byte;
+ chip->legacy.read_buf = gpmi_read_buf;
+ chip->legacy.write_buf = gpmi_write_buf;
chip->badblock_pattern = &gpmi_bbt_descr;
- chip->block_markbad = gpmi_block_markbad;
+ chip->legacy.block_markbad = gpmi_block_markbad;
chip->options |= NAND_NO_SUBPAGE_WRITE;
/* Set up swap_block_mark, must be set before the gpmi_set_geometry() */
goto err_out;
chip->dummy_controller.ops = &gpmi_nand_controller_ops;
- ret = nand_scan(mtd, GPMI_IS_MX6(this) ? 2 : 1);
+ ret = nand_scan(chip, GPMI_IS_MX6(this) ? 2 : 1);
if (ret)
goto err_out;
{
struct gpmi_nand_data *this = platform_get_drvdata(pdev);
- nand_release(nand_to_mtd(&this->nand));
+ nand_release(&this->nand);
gpmi_free_dma_buffer(this);
release_resources(this);
return 0;
int gpmi_send_command(struct gpmi_nand_data *);
int gpmi_enable_clk(struct gpmi_nand_data *this);
int gpmi_disable_clk(struct gpmi_nand_data *this);
-int gpmi_setup_data_interface(struct mtd_info *mtd, int chipnr,
+int gpmi_setup_data_interface(struct nand_chip *chip, int chipnr,
const struct nand_data_interface *conf);
void gpmi_nfc_apply_timings(struct gpmi_nand_data *this);
int gpmi_read_data(struct gpmi_nand_data *, void *buf, int len);
return 0;
}
-static void hisi_nfc_select_chip(struct mtd_info *mtd, int chipselect)
+static void hisi_nfc_select_chip(struct nand_chip *chip, int chipselect)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct hinfc_host *host = nand_get_controller_data(chip);
if (chipselect < 0)
host->chipselect = chipselect;
}
-static uint8_t hisi_nfc_read_byte(struct mtd_info *mtd)
+static uint8_t hisi_nfc_read_byte(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct hinfc_host *host = nand_get_controller_data(chip);
if (host->command == NAND_CMD_STATUS)
return *(uint8_t *)(host->buffer + host->offset - 1);
}
-static u16 hisi_nfc_read_word(struct mtd_info *mtd)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
- struct hinfc_host *host = nand_get_controller_data(chip);
-
- host->offset += 2;
- return *(u16 *)(host->buffer + host->offset - 2);
-}
-
static void
-hisi_nfc_write_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
+hisi_nfc_write_buf(struct nand_chip *chip, const uint8_t *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct hinfc_host *host = nand_get_controller_data(chip);
memcpy(host->buffer + host->offset, buf, len);
host->offset += len;
}
-static void hisi_nfc_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void hisi_nfc_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct hinfc_host *host = nand_get_controller_data(chip);
memcpy(buf, host->buffer + host->offset, len);
}
}
-static void hisi_nfc_cmdfunc(struct mtd_info *mtd, unsigned command, int column,
- int page_addr)
+static void hisi_nfc_cmdfunc(struct nand_chip *chip, unsigned command,
+ int column, int page_addr)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct hinfc_host *host = nand_get_controller_data(chip);
int is_cache_invalid = 1;
unsigned int flag = 0;
return IRQ_HANDLED;
}
-static int hisi_nand_read_page_hwecc(struct mtd_info *mtd,
- struct nand_chip *chip, uint8_t *buf, int oob_required, int page)
+static int hisi_nand_read_page_hwecc(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct hinfc_host *host = nand_get_controller_data(chip);
int max_bitflips = 0, stat = 0, stat_max = 0, status_ecc;
int stat_1, stat_2;
nand_read_page_op(chip, page, 0, buf, mtd->writesize);
- chip->read_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.read_buf(chip, chip->oob_poi, mtd->oobsize);
/* errors which can not be corrected by ECC */
if (host->irq_status & HINFC504_INTS_UE) {
return max_bitflips;
}
-static int hisi_nand_read_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int hisi_nand_read_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct hinfc_host *host = nand_get_controller_data(chip);
nand_read_oob_op(chip, page, 0, chip->oob_poi, mtd->oobsize);
return 0;
}
-static int hisi_nand_write_page_hwecc(struct mtd_info *mtd,
- struct nand_chip *chip, const uint8_t *buf, int oob_required,
- int page)
+static int hisi_nand_write_page_hwecc(struct nand_chip *chip,
+ const uint8_t *buf, int oob_required,
+ int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
nand_prog_page_begin_op(chip, page, 0, buf, mtd->writesize);
if (oob_required)
- chip->write_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.write_buf(chip, chip->oob_poi, mtd->oobsize);
return nand_prog_page_end_op(chip);
}
nand_set_controller_data(chip, host);
nand_set_flash_node(chip, np);
- chip->cmdfunc = hisi_nfc_cmdfunc;
+ chip->legacy.cmdfunc = hisi_nfc_cmdfunc;
chip->select_chip = hisi_nfc_select_chip;
- chip->read_byte = hisi_nfc_read_byte;
- chip->read_word = hisi_nfc_read_word;
- chip->write_buf = hisi_nfc_write_buf;
- chip->read_buf = hisi_nfc_read_buf;
- chip->chip_delay = HINFC504_CHIP_DELAY;
- chip->set_features = nand_get_set_features_notsupp;
- chip->get_features = nand_get_set_features_notsupp;
+ chip->legacy.read_byte = hisi_nfc_read_byte;
+ chip->legacy.write_buf = hisi_nfc_write_buf;
+ chip->legacy.read_buf = hisi_nfc_read_buf;
+ chip->legacy.chip_delay = HINFC504_CHIP_DELAY;
+ chip->legacy.set_features = nand_get_set_features_notsupp;
+ chip->legacy.get_features = nand_get_set_features_notsupp;
hisi_nfc_host_init(host);
}
chip->dummy_controller.ops = &hisi_nfc_controller_ops;
- ret = nand_scan(mtd, max_chips);
+ ret = nand_scan(chip, max_chips);
if (ret)
return ret;
static int hisi_nfc_remove(struct platform_device *pdev)
{
struct hinfc_host *host = platform_get_drvdata(pdev);
- struct mtd_info *mtd = nand_to_mtd(&host->chip);
- nand_release(mtd);
+ nand_release(&host->chip);
return 0;
}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 - Bootlin
+ *
+ * Author: Boris Brezillon <boris.brezillon@bootlin.com>
+ *
+ * Header containing internal definitions to be used only by core files.
+ * NAND controller drivers should not include this file.
+ */
+
+#ifndef __LINUX_RAWNAND_INTERNALS
+#define __LINUX_RAWNAND_INTERNALS
+
+#include <linux/mtd/rawnand.h>
+
+/*
+ * NAND Flash Manufacturer ID Codes
+ */
+#define NAND_MFR_AMD 0x01
+#define NAND_MFR_ATO 0x9b
+#define NAND_MFR_EON 0x92
+#define NAND_MFR_ESMT 0xc8
+#define NAND_MFR_FUJITSU 0x04
+#define NAND_MFR_HYNIX 0xad
+#define NAND_MFR_INTEL 0x89
+#define NAND_MFR_MACRONIX 0xc2
+#define NAND_MFR_MICRON 0x2c
+#define NAND_MFR_NATIONAL 0x8f
+#define NAND_MFR_RENESAS 0x07
+#define NAND_MFR_SAMSUNG 0xec
+#define NAND_MFR_SANDISK 0x45
+#define NAND_MFR_STMICRO 0x20
+#define NAND_MFR_TOSHIBA 0x98
+#define NAND_MFR_WINBOND 0xef
+
+/**
+ * struct nand_manufacturer_ops - NAND Manufacturer operations
+ * @detect: detect the NAND memory organization and capabilities
+ * @init: initialize all vendor specific fields (like the ->read_retry()
+ * implementation) if any.
+ * @cleanup: the ->init() function may have allocated resources, ->cleanup()
+ * is here to let vendor specific code release those resources.
+ * @fixup_onfi_param_page: apply vendor specific fixups to the ONFI parameter
+ * page. This is called after the checksum is verified.
+ */
+struct nand_manufacturer_ops {
+ void (*detect)(struct nand_chip *chip);
+ int (*init)(struct nand_chip *chip);
+ void (*cleanup)(struct nand_chip *chip);
+ void (*fixup_onfi_param_page)(struct nand_chip *chip,
+ struct nand_onfi_params *p);
+};
+
+/**
+ * struct nand_manufacturer - NAND Flash Manufacturer structure
+ * @name: Manufacturer name
+ * @id: manufacturer ID code of device.
+ * @ops: manufacturer operations
+ */
+struct nand_manufacturer {
+ int id;
+ char *name;
+ const struct nand_manufacturer_ops *ops;
+};
+
+
+extern struct nand_flash_dev nand_flash_ids[];
+
+extern const struct nand_manufacturer_ops amd_nand_manuf_ops;
+extern const struct nand_manufacturer_ops esmt_nand_manuf_ops;
+extern const struct nand_manufacturer_ops hynix_nand_manuf_ops;
+extern const struct nand_manufacturer_ops macronix_nand_manuf_ops;
+extern const struct nand_manufacturer_ops micron_nand_manuf_ops;
+extern const struct nand_manufacturer_ops samsung_nand_manuf_ops;
+extern const struct nand_manufacturer_ops toshiba_nand_manuf_ops;
+
+/* Core functions */
+const struct nand_manufacturer *nand_get_manufacturer(u8 id);
+int nand_markbad_bbm(struct nand_chip *chip, loff_t ofs);
+int nand_erase_nand(struct nand_chip *chip, struct erase_info *instr,
+ int allowbbt);
+int onfi_fill_data_interface(struct nand_chip *chip,
+ enum nand_data_interface_type type,
+ int timing_mode);
+int nand_get_features(struct nand_chip *chip, int addr, u8 *subfeature_param);
+int nand_set_features(struct nand_chip *chip, int addr, u8 *subfeature_param);
+int nand_read_page_raw_notsupp(struct nand_chip *chip, u8 *buf,
+ int oob_required, int page);
+int nand_write_page_raw_notsupp(struct nand_chip *chip, const u8 *buf,
+ int oob_required, int page);
+int nand_exit_status_op(struct nand_chip *chip);
+int nand_read_param_page_op(struct nand_chip *chip, u8 page, void *buf,
+ unsigned int len);
+void nand_decode_ext_id(struct nand_chip *chip);
+void panic_nand_wait(struct nand_chip *chip, unsigned long timeo);
+void sanitize_string(uint8_t *s, size_t len);
+
+/* BBT functions */
+int nand_markbad_bbt(struct nand_chip *chip, loff_t offs);
+int nand_isreserved_bbt(struct nand_chip *chip, loff_t offs);
+int nand_isbad_bbt(struct nand_chip *chip, loff_t offs, int allowbbt);
+
+/* Legacy */
+void nand_legacy_set_defaults(struct nand_chip *chip);
+void nand_legacy_adjust_cmdfunc(struct nand_chip *chip);
+int nand_legacy_check_hooks(struct nand_chip *chip);
+
+/* ONFI functions */
+u16 onfi_crc16(u16 crc, u8 const *p, size_t len);
+int nand_onfi_detect(struct nand_chip *chip);
+
+/* JEDEC functions */
+int nand_jedec_detect(struct nand_chip *chip);
+
+#endif /* __LINUX_RAWNAND_INTERNALS */
return container_of(mtd_to_nand(mtd), struct jz_nand, chip);
}
-static void jz_nand_select_chip(struct mtd_info *mtd, int chipnr)
+static void jz_nand_select_chip(struct nand_chip *chip, int chipnr)
{
- struct jz_nand *nand = mtd_to_jz_nand(mtd);
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct jz_nand *nand = mtd_to_jz_nand(nand_to_mtd(chip));
uint32_t ctrl;
int banknr;
banknr = -1;
} else {
banknr = nand->banks[chipnr] - 1;
- chip->IO_ADDR_R = nand->bank_base[banknr];
- chip->IO_ADDR_W = nand->bank_base[banknr];
+ chip->legacy.IO_ADDR_R = nand->bank_base[banknr];
+ chip->legacy.IO_ADDR_W = nand->bank_base[banknr];
}
writel(ctrl, nand->base + JZ_REG_NAND_CTRL);
nand->selected_bank = banknr;
}
-static void jz_nand_cmd_ctrl(struct mtd_info *mtd, int dat, unsigned int ctrl)
+static void jz_nand_cmd_ctrl(struct nand_chip *chip, int dat,
+ unsigned int ctrl)
{
- struct jz_nand *nand = mtd_to_jz_nand(mtd);
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct jz_nand *nand = mtd_to_jz_nand(nand_to_mtd(chip));
uint32_t reg;
void __iomem *bank_base = nand->bank_base[nand->selected_bank];
bank_base += JZ_NAND_MEM_ADDR_OFFSET;
else if (ctrl & NAND_CLE)
bank_base += JZ_NAND_MEM_CMD_OFFSET;
- chip->IO_ADDR_W = bank_base;
+ chip->legacy.IO_ADDR_W = bank_base;
reg = readl(nand->base + JZ_REG_NAND_CTRL);
if (ctrl & NAND_NCE)
writel(reg, nand->base + JZ_REG_NAND_CTRL);
}
if (dat != NAND_CMD_NONE)
- writeb(dat, chip->IO_ADDR_W);
+ writeb(dat, chip->legacy.IO_ADDR_W);
}
-static int jz_nand_dev_ready(struct mtd_info *mtd)
+static int jz_nand_dev_ready(struct nand_chip *chip)
{
- struct jz_nand *nand = mtd_to_jz_nand(mtd);
+ struct jz_nand *nand = mtd_to_jz_nand(nand_to_mtd(chip));
return gpiod_get_value_cansleep(nand->busy_gpio);
}
-static void jz_nand_hwctl(struct mtd_info *mtd, int mode)
+static void jz_nand_hwctl(struct nand_chip *chip, int mode)
{
- struct jz_nand *nand = mtd_to_jz_nand(mtd);
+ struct jz_nand *nand = mtd_to_jz_nand(nand_to_mtd(chip));
uint32_t reg;
writel(0, nand->base + JZ_REG_NAND_IRQ_STAT);
writel(reg, nand->base + JZ_REG_NAND_ECC_CTRL);
}
-static int jz_nand_calculate_ecc_rs(struct mtd_info *mtd, const uint8_t *dat,
- uint8_t *ecc_code)
+static int jz_nand_calculate_ecc_rs(struct nand_chip *chip, const uint8_t *dat,
+ uint8_t *ecc_code)
{
- struct jz_nand *nand = mtd_to_jz_nand(mtd);
+ struct jz_nand *nand = mtd_to_jz_nand(nand_to_mtd(chip));
uint32_t reg, status;
int i;
unsigned int timeout = 1000;
dat[index+1] = (data >> 8) & 0xff;
}
-static int jz_nand_correct_ecc_rs(struct mtd_info *mtd, uint8_t *dat,
- uint8_t *read_ecc, uint8_t *calc_ecc)
+static int jz_nand_correct_ecc_rs(struct nand_chip *chip, uint8_t *dat,
+ uint8_t *read_ecc, uint8_t *calc_ecc)
{
- struct jz_nand *nand = mtd_to_jz_nand(mtd);
+ struct jz_nand *nand = mtd_to_jz_nand(nand_to_mtd(chip));
int i, error_count, index;
uint32_t reg, status, error;
unsigned int timeout = 1000;
if (chipnr == 0) {
/* Detect first chip. */
- ret = nand_scan(mtd, 1);
+ ret = nand_scan(chip, 1);
if (ret)
goto notfound_id;
/* Retrieve the IDs from the first chip. */
- chip->select_chip(mtd, 0);
+ chip->select_chip(chip, 0);
nand_reset_op(chip);
nand_readid_op(chip, 0, id, sizeof(id));
*nand_maf_id = id[0];
*nand_dev_id = id[1];
} else {
/* Detect additional chip. */
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
nand_reset_op(chip);
nand_readid_op(chip, 0, id, sizeof(id));
if (*nand_maf_id != id[0] || *nand_dev_id != id[1]) {
chip->ecc.strength = 4;
chip->ecc.options = NAND_ECC_GENERIC_ERASED_CHECK;
- chip->chip_delay = 50;
- chip->cmd_ctrl = jz_nand_cmd_ctrl;
+ chip->legacy.chip_delay = 50;
+ chip->legacy.cmd_ctrl = jz_nand_cmd_ctrl;
chip->select_chip = jz_nand_select_chip;
chip->dummy_controller.ops = &jz_nand_controller_ops;
if (nand->busy_gpio)
- chip->dev_ready = jz_nand_dev_ready;
+ chip->legacy.dev_ready = jz_nand_dev_ready;
platform_set_drvdata(pdev, nand);
struct jz_nand *nand = platform_get_drvdata(pdev);
size_t i;
- nand_release(nand_to_mtd(&nand->chip));
+ nand_release(&nand->chip);
/* Deassert and disable all chips */
writel(0, nand->base + JZ_REG_NAND_CTRL);
return container_of(ctrl, struct jz4780_nand_controller, controller);
}
-static void jz4780_nand_select_chip(struct mtd_info *mtd, int chipnr)
+static void jz4780_nand_select_chip(struct nand_chip *chip, int chipnr)
{
- struct jz4780_nand_chip *nand = to_jz4780_nand_chip(mtd);
+ struct jz4780_nand_chip *nand = to_jz4780_nand_chip(nand_to_mtd(chip));
struct jz4780_nand_controller *nfc = to_jz4780_nand_controller(nand->chip.controller);
struct jz4780_nand_cs *cs;
nfc->selected = chipnr;
}
-static void jz4780_nand_cmd_ctrl(struct mtd_info *mtd, int cmd,
+static void jz4780_nand_cmd_ctrl(struct nand_chip *chip, int cmd,
unsigned int ctrl)
{
- struct jz4780_nand_chip *nand = to_jz4780_nand_chip(mtd);
+ struct jz4780_nand_chip *nand = to_jz4780_nand_chip(nand_to_mtd(chip));
struct jz4780_nand_controller *nfc = to_jz4780_nand_controller(nand->chip.controller);
struct jz4780_nand_cs *cs;
writeb(cmd, cs->base + OFFSET_CMD);
}
-static int jz4780_nand_dev_ready(struct mtd_info *mtd)
+static int jz4780_nand_dev_ready(struct nand_chip *chip)
{
- struct jz4780_nand_chip *nand = to_jz4780_nand_chip(mtd);
+ struct jz4780_nand_chip *nand = to_jz4780_nand_chip(nand_to_mtd(chip));
return !gpiod_get_value_cansleep(nand->busy_gpio);
}
-static void jz4780_nand_ecc_hwctl(struct mtd_info *mtd, int mode)
+static void jz4780_nand_ecc_hwctl(struct nand_chip *chip, int mode)
{
- struct jz4780_nand_chip *nand = to_jz4780_nand_chip(mtd);
+ struct jz4780_nand_chip *nand = to_jz4780_nand_chip(nand_to_mtd(chip));
nand->reading = (mode == NAND_ECC_READ);
}
-static int jz4780_nand_ecc_calculate(struct mtd_info *mtd, const u8 *dat,
+static int jz4780_nand_ecc_calculate(struct nand_chip *chip, const u8 *dat,
u8 *ecc_code)
{
- struct jz4780_nand_chip *nand = to_jz4780_nand_chip(mtd);
+ struct jz4780_nand_chip *nand = to_jz4780_nand_chip(nand_to_mtd(chip));
struct jz4780_nand_controller *nfc = to_jz4780_nand_controller(nand->chip.controller);
struct jz4780_bch_params params;
return jz4780_bch_calculate(nfc->bch, ¶ms, dat, ecc_code);
}
-static int jz4780_nand_ecc_correct(struct mtd_info *mtd, u8 *dat,
+static int jz4780_nand_ecc_correct(struct nand_chip *chip, u8 *dat,
u8 *read_ecc, u8 *calc_ecc)
{
- struct jz4780_nand_chip *nand = to_jz4780_nand_chip(mtd);
+ struct jz4780_nand_chip *nand = to_jz4780_nand_chip(nand_to_mtd(chip));
struct jz4780_nand_controller *nfc = to_jz4780_nand_controller(nand->chip.controller);
struct jz4780_bch_params params;
dev_err(dev, "failed to request busy GPIO: %d\n", ret);
return ret;
} else if (nand->busy_gpio) {
- nand->chip.dev_ready = jz4780_nand_dev_ready;
+ nand->chip.legacy.dev_ready = jz4780_nand_dev_ready;
}
nand->wp_gpio = devm_gpiod_get_optional(dev, "wp", GPIOD_OUT_LOW);
return -ENOMEM;
mtd->dev.parent = dev;
- chip->IO_ADDR_R = cs->base + OFFSET_DATA;
- chip->IO_ADDR_W = cs->base + OFFSET_DATA;
- chip->chip_delay = RB_DELAY_US;
+ chip->legacy.IO_ADDR_R = cs->base + OFFSET_DATA;
+ chip->legacy.IO_ADDR_W = cs->base + OFFSET_DATA;
+ chip->legacy.chip_delay = RB_DELAY_US;
chip->options = NAND_NO_SUBPAGE_WRITE;
chip->select_chip = jz4780_nand_select_chip;
- chip->cmd_ctrl = jz4780_nand_cmd_ctrl;
+ chip->legacy.cmd_ctrl = jz4780_nand_cmd_ctrl;
chip->ecc.mode = NAND_ECC_HW;
chip->controller = &nfc->controller;
nand_set_flash_node(chip, np);
chip->controller->ops = &jz4780_nand_controller_ops;
- ret = nand_scan(mtd, 1);
+ ret = nand_scan(chip, 1);
if (ret)
return ret;
ret = mtd_device_register(mtd, NULL, 0);
if (ret) {
- nand_release(mtd);
+ nand_release(chip);
return ret;
}
while (!list_empty(&nfc->chips)) {
chip = list_first_entry(&nfc->chips, struct jz4780_nand_chip, chip_list);
- nand_release(nand_to_mtd(&chip->chip));
+ nand_release(&chip->chip);
list_del(&chip->chip_list);
}
}
return -ENODEV;
}
- nfc = devm_kzalloc(dev, sizeof(*nfc) + (sizeof(nfc->cs[0]) * num_banks), GFP_KERNEL);
+ nfc = devm_kzalloc(dev, struct_size(nfc, cs, num_banks), GFP_KERNEL);
if (!nfc)
return -ENOMEM;
/*
* Hardware specific access to control lines
*/
-static void lpc32xx_nand_cmd_ctrl(struct mtd_info *mtd, int cmd,
+static void lpc32xx_nand_cmd_ctrl(struct nand_chip *nand_chip, int cmd,
unsigned int ctrl)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
struct lpc32xx_nand_host *host = nand_get_controller_data(nand_chip);
if (cmd != NAND_CMD_NONE) {
/*
* Read Device Ready (NAND device _and_ controller ready)
*/
-static int lpc32xx_nand_device_ready(struct mtd_info *mtd)
+static int lpc32xx_nand_device_ready(struct nand_chip *nand_chip)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
struct lpc32xx_nand_host *host = nand_get_controller_data(nand_chip);
if ((readb(MLC_ISR(host->io_base)) &
return IRQ_HANDLED;
}
-static int lpc32xx_waitfunc_nand(struct mtd_info *mtd, struct nand_chip *chip)
+static int lpc32xx_waitfunc_nand(struct nand_chip *chip)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
if (readb(MLC_ISR(host->io_base)) & MLCISR_NAND_READY)
return NAND_STATUS_READY;
}
-static int lpc32xx_waitfunc_controller(struct mtd_info *mtd,
- struct nand_chip *chip)
+static int lpc32xx_waitfunc_controller(struct nand_chip *chip)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
if (readb(MLC_ISR(host->io_base)) & MLCISR_CONTROLLER_READY)
return NAND_STATUS_READY;
}
-static int lpc32xx_waitfunc(struct mtd_info *mtd, struct nand_chip *chip)
+static int lpc32xx_waitfunc(struct nand_chip *chip)
{
- lpc32xx_waitfunc_nand(mtd, chip);
- lpc32xx_waitfunc_controller(mtd, chip);
+ lpc32xx_waitfunc_nand(chip);
+ lpc32xx_waitfunc_controller(chip);
return NAND_STATUS_READY;
}
return -ENXIO;
}
-static int lpc32xx_read_page(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int lpc32xx_read_page(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
int i, j;
uint8_t *oobbuf = chip->oob_poi;
writeb(0x00, MLC_ECC_AUTO_DEC_REG(host->io_base));
/* Wait for Controller Ready */
- lpc32xx_waitfunc_controller(mtd, chip);
+ lpc32xx_waitfunc_controller(chip);
/* Check ECC Error status */
mlc_isr = readl(MLC_ISR(host->io_base));
return 0;
}
-static int lpc32xx_write_page_lowlevel(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int lpc32xx_write_page_lowlevel(struct nand_chip *chip,
const uint8_t *buf, int oob_required,
int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
const uint8_t *oobbuf = chip->oob_poi;
uint8_t *dma_buf = (uint8_t *)buf;
writeb(0x00, MLC_ECC_AUTO_ENC_REG(host->io_base));
/* Wait for Controller Ready */
- lpc32xx_waitfunc_controller(mtd, chip);
+ lpc32xx_waitfunc_controller(chip);
}
return nand_prog_page_end_op(chip);
}
-static int lpc32xx_read_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int lpc32xx_read_oob(struct nand_chip *chip, int page)
{
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
/* Read whole page - necessary with MLC controller! */
- lpc32xx_read_page(mtd, chip, host->dummy_buf, 1, page);
+ lpc32xx_read_page(chip, host->dummy_buf, 1, page);
return 0;
}
-static int lpc32xx_write_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int lpc32xx_write_oob(struct nand_chip *chip, int page)
{
/* None, write_oob conflicts with the automatic LPC MLC ECC decoder! */
return 0;
}
/* Prepares MLC for transfers with H/W ECC enabled: always enabled anyway */
-static void lpc32xx_ecc_enable(struct mtd_info *mtd, int mode)
+static void lpc32xx_ecc_enable(struct nand_chip *chip, int mode)
{
/* Always enabled! */
}
if (res)
goto put_clk;
- nand_chip->cmd_ctrl = lpc32xx_nand_cmd_ctrl;
- nand_chip->dev_ready = lpc32xx_nand_device_ready;
- nand_chip->chip_delay = 25; /* us */
- nand_chip->IO_ADDR_R = MLC_DATA(host->io_base);
- nand_chip->IO_ADDR_W = MLC_DATA(host->io_base);
+ nand_chip->legacy.cmd_ctrl = lpc32xx_nand_cmd_ctrl;
+ nand_chip->legacy.dev_ready = lpc32xx_nand_device_ready;
+ nand_chip->legacy.chip_delay = 25; /* us */
+ nand_chip->legacy.IO_ADDR_R = MLC_DATA(host->io_base);
+ nand_chip->legacy.IO_ADDR_W = MLC_DATA(host->io_base);
/* Init NAND controller */
lpc32xx_nand_setup(host);
nand_chip->ecc.read_oob = lpc32xx_read_oob;
nand_chip->ecc.strength = 4;
nand_chip->ecc.bytes = 10;
- nand_chip->waitfunc = lpc32xx_waitfunc;
+ nand_chip->legacy.waitfunc = lpc32xx_waitfunc;
nand_chip->options = NAND_NO_SUBPAGE_WRITE;
nand_chip->bbt_options = NAND_BBT_USE_FLASH | NAND_BBT_NO_OOB;
* SMALL block or LARGE block.
*/
nand_chip->dummy_controller.ops = &lpc32xx_nand_controller_ops;
- res = nand_scan(mtd, 1);
+ res = nand_scan(nand_chip, 1);
if (res)
goto free_irq;
static int lpc32xx_nand_remove(struct platform_device *pdev)
{
struct lpc32xx_nand_host *host = platform_get_drvdata(pdev);
- struct mtd_info *mtd = nand_to_mtd(&host->nand_chip);
- nand_release(mtd);
+ nand_release(&host->nand_chip);
free_irq(host->irq, host);
if (use_dma)
dma_release_channel(host->dma_chan);
/*
* Hardware specific access to control lines
*/
-static void lpc32xx_nand_cmd_ctrl(struct mtd_info *mtd, int cmd,
- unsigned int ctrl)
+static void lpc32xx_nand_cmd_ctrl(struct nand_chip *chip, int cmd,
+ unsigned int ctrl)
{
uint32_t tmp;
- struct nand_chip *chip = mtd_to_nand(mtd);
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
/* Does CE state need to be changed? */
/*
* Read the Device Ready pin
*/
-static int lpc32xx_nand_device_ready(struct mtd_info *mtd)
+static int lpc32xx_nand_device_ready(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
int rdy = 0;
/*
* Prepares SLC for transfers with H/W ECC enabled
*/
-static void lpc32xx_nand_ecc_enable(struct mtd_info *mtd, int mode)
+static void lpc32xx_nand_ecc_enable(struct nand_chip *chip, int mode)
{
/* Hardware ECC is enabled automatically in hardware as needed */
}
/*
* Calculates the ECC for the data
*/
-static int lpc32xx_nand_ecc_calculate(struct mtd_info *mtd,
+static int lpc32xx_nand_ecc_calculate(struct nand_chip *chip,
const unsigned char *buf,
unsigned char *code)
{
/*
* Read a single byte from NAND device
*/
-static uint8_t lpc32xx_nand_read_byte(struct mtd_info *mtd)
+static uint8_t lpc32xx_nand_read_byte(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
return (uint8_t)readl(SLC_DATA(host->io_base));
/*
* Simple device read without ECC
*/
-static void lpc32xx_nand_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void lpc32xx_nand_read_buf(struct nand_chip *chip, u_char *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
/* Direct device read with no ECC */
/*
* Simple device write without ECC
*/
-static void lpc32xx_nand_write_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
+static void lpc32xx_nand_write_buf(struct nand_chip *chip, const uint8_t *buf,
+ int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
/* Direct device write with no ECC */
/*
* Read the OOB data from the device without ECC using FIFO method
*/
-static int lpc32xx_nand_read_oob_syndrome(struct mtd_info *mtd,
- struct nand_chip *chip, int page)
+static int lpc32xx_nand_read_oob_syndrome(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
return nand_read_oob_op(chip, page, 0, chip->oob_poi, mtd->oobsize);
}
/*
* Write the OOB data to the device without ECC using FIFO method
*/
-static int lpc32xx_nand_write_oob_syndrome(struct mtd_info *mtd,
- struct nand_chip *chip, int page)
+static int lpc32xx_nand_write_oob_syndrome(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
return nand_prog_page_op(chip, page, mtd->writesize, chip->oob_poi,
mtd->oobsize);
}
* Read the data and OOB data from the device, use ECC correction with the
* data, disable ECC for the OOB data
*/
-static int lpc32xx_nand_read_page_syndrome(struct mtd_info *mtd,
- struct nand_chip *chip, uint8_t *buf,
+static int lpc32xx_nand_read_page_syndrome(struct nand_chip *chip, uint8_t *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
struct mtd_oob_region oobregion = { };
int stat, i, status, error;
status = lpc32xx_xfer(mtd, buf, chip->ecc.steps, 1);
/* Get OOB data */
- chip->read_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.read_buf(chip, chip->oob_poi, mtd->oobsize);
/* Convert to stored ECC format */
lpc32xx_slc_ecc_copy(tmpecc, (uint32_t *) host->ecc_buf, chip->ecc.steps);
oobecc = chip->oob_poi + oobregion.offset;
for (i = 0; i < chip->ecc.steps; i++) {
- stat = chip->ecc.correct(mtd, buf, oobecc,
+ stat = chip->ecc.correct(chip, buf, oobecc,
&tmpecc[i * chip->ecc.bytes]);
if (stat < 0)
mtd->ecc_stats.failed++;
* Read the data and OOB data from the device, no ECC correction with the
* data or OOB data
*/
-static int lpc32xx_nand_read_page_raw_syndrome(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int lpc32xx_nand_read_page_raw_syndrome(struct nand_chip *chip,
uint8_t *buf, int oob_required,
int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
/* Issue read command */
nand_read_page_op(chip, page, 0, NULL, 0);
/* Raw reads can just use the FIFO interface */
- chip->read_buf(mtd, buf, chip->ecc.size * chip->ecc.steps);
- chip->read_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.read_buf(chip, buf, chip->ecc.size * chip->ecc.steps);
+ chip->legacy.read_buf(chip, chip->oob_poi, mtd->oobsize);
return 0;
}
* Write the data and OOB data to the device, use ECC with the data,
* disable ECC for the OOB data
*/
-static int lpc32xx_nand_write_page_syndrome(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int lpc32xx_nand_write_page_syndrome(struct nand_chip *chip,
const uint8_t *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
struct mtd_oob_region oobregion = { };
uint8_t *pb;
lpc32xx_slc_ecc_copy(pb, (uint32_t *)host->ecc_buf, chip->ecc.steps);
/* Write ECC data to device */
- chip->write_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.write_buf(chip, chip->oob_poi, mtd->oobsize);
return nand_prog_page_end_op(chip);
}
* Write the data and OOB data to the device, no ECC correction with the
* data or OOB data
*/
-static int lpc32xx_nand_write_page_raw_syndrome(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int lpc32xx_nand_write_page_raw_syndrome(struct nand_chip *chip,
const uint8_t *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
/* Raw writes can just use the FIFO interface */
nand_prog_page_begin_op(chip, page, 0, buf,
chip->ecc.size * chip->ecc.steps);
- chip->write_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.write_buf(chip, chip->oob_poi, mtd->oobsize);
return nand_prog_page_end_op(chip);
}
goto enable_wp;
/* Set NAND IO addresses and command/ready functions */
- chip->IO_ADDR_R = SLC_DATA(host->io_base);
- chip->IO_ADDR_W = SLC_DATA(host->io_base);
- chip->cmd_ctrl = lpc32xx_nand_cmd_ctrl;
- chip->dev_ready = lpc32xx_nand_device_ready;
- chip->chip_delay = 20; /* 20us command delay time */
+ chip->legacy.IO_ADDR_R = SLC_DATA(host->io_base);
+ chip->legacy.IO_ADDR_W = SLC_DATA(host->io_base);
+ chip->legacy.cmd_ctrl = lpc32xx_nand_cmd_ctrl;
+ chip->legacy.dev_ready = lpc32xx_nand_device_ready;
+ chip->legacy.chip_delay = 20; /* 20us command delay time */
/* Init NAND controller */
lpc32xx_nand_setup(host);
/* NAND callbacks for LPC32xx SLC hardware */
chip->ecc.mode = NAND_ECC_HW_SYNDROME;
- chip->read_byte = lpc32xx_nand_read_byte;
- chip->read_buf = lpc32xx_nand_read_buf;
- chip->write_buf = lpc32xx_nand_write_buf;
+ chip->legacy.read_byte = lpc32xx_nand_read_byte;
+ chip->legacy.read_buf = lpc32xx_nand_read_buf;
+ chip->legacy.write_buf = lpc32xx_nand_write_buf;
chip->ecc.read_page_raw = lpc32xx_nand_read_page_raw_syndrome;
chip->ecc.read_page = lpc32xx_nand_read_page_syndrome;
chip->ecc.write_page_raw = lpc32xx_nand_write_page_raw_syndrome;
/* Find NAND device */
chip->dummy_controller.ops = &lpc32xx_nand_controller_ops;
- res = nand_scan(mtd, 1);
+ res = nand_scan(chip, 1);
if (res)
goto release_dma;
{
uint32_t tmp;
struct lpc32xx_nand_host *host = platform_get_drvdata(pdev);
- struct mtd_info *mtd = nand_to_mtd(&host->nand_chip);
- nand_release(mtd);
+ nand_release(&host->nand_chip);
dma_release_channel(host->dma_chan);
/* Force CE high */
* Copyright (C) 2017 Marvell
* Author: Miquel RAYNAL <miquel.raynal@free-electrons.com>
*
+ *
+ * This NAND controller driver handles two versions of the hardware,
+ * one is called NFCv1 and is available on PXA SoCs and the other is
+ * called NFCv2 and is available on Armada SoCs.
+ *
+ * The main visible difference is that NFCv1 only has Hamming ECC
+ * capabilities, while NFCv2 also embeds a BCH ECC engine. Also, DMA
+ * is not used with NFCv2.
+ *
+ * The ECC layouts are depicted in details in Marvell AN-379, but here
+ * is a brief description.
+ *
+ * When using Hamming, the data is split in 512B chunks (either 1, 2
+ * or 4) and each chunk will have its own ECC "digest" of 6B at the
+ * beginning of the OOB area and eventually the remaining free OOB
+ * bytes (also called "spare" bytes in the driver). This engine
+ * corrects up to 1 bit per chunk and detects reliably an error if
+ * there are at most 2 bitflips. Here is the page layout used by the
+ * controller when Hamming is chosen:
+ *
+ * +-------------------------------------------------------------+
+ * | Data 1 | ... | Data N | ECC 1 | ... | ECCN | Free OOB bytes |
+ * +-------------------------------------------------------------+
+ *
+ * When using the BCH engine, there are N identical (data + free OOB +
+ * ECC) sections and potentially an extra one to deal with
+ * configurations where the chosen (data + free OOB + ECC) sizes do
+ * not align with the page (data + OOB) size. ECC bytes are always
+ * 30B per ECC chunk. Here is the page layout used by the controller
+ * when BCH is chosen:
+ *
+ * +-----------------------------------------
+ * | Data 1 | Free OOB bytes 1 | ECC 1 | ...
+ * +-----------------------------------------
+ *
+ * -------------------------------------------
+ * ... | Data N | Free OOB bytes N | ECC N |
+ * -------------------------------------------
+ *
+ * --------------------------------------------+
+ * Last Data | Last Free OOB bytes | Last ECC |
+ * --------------------------------------------+
+ *
+ * In both cases, the layout seen by the user is always: all data
+ * first, then all free OOB bytes and finally all ECC bytes. With BCH,
+ * ECC bytes are 30B long and are padded with 0xFF to align on 32
+ * bytes.
+ *
+ * The controller has certain limitations that are handled by the
+ * driver:
+ * - It can only read 2k at a time. To overcome this limitation, the
+ * driver issues data cycles on the bus, without issuing new
+ * CMD + ADDR cycles. The Marvell term is "naked" operations.
+ * - The ECC strength in BCH mode cannot be tuned. It is fixed 16
+ * bits. What can be tuned is the ECC block size as long as it
+ * stays between 512B and 2kiB. It's usually chosen based on the
+ * chip ECC requirements. For instance, using 2kiB ECC chunks
+ * provides 4b/512B correctability.
+ * - The controller will always treat data bytes, free OOB bytes
+ * and ECC bytes in that order, no matter what the real layout is
+ * (which is usually all data then all OOB bytes). The
+ * marvell_nfc_layouts array below contains the currently
+ * supported layouts.
+ * - Because of these weird layouts, the Bad Block Markers can be
+ * located in data section. In this case, the NAND_BBT_NO_OOB_BBM
+ * option must be set to prevent scanning/writing bad block
+ * markers.
*/
#include <linux/module.h>
MARVELL_LAYOUT( 512, 512, 1, 1, 1, 512, 8, 8, 0, 0, 0),
MARVELL_LAYOUT( 2048, 512, 1, 1, 1, 2048, 40, 24, 0, 0, 0),
MARVELL_LAYOUT( 2048, 512, 4, 1, 1, 2048, 32, 30, 0, 0, 0),
+ MARVELL_LAYOUT( 2048, 512, 8, 2, 1, 1024, 0, 30,1024,32, 30),
MARVELL_LAYOUT( 4096, 512, 4, 2, 2, 2048, 32, 30, 0, 0, 0),
MARVELL_LAYOUT( 4096, 512, 8, 5, 4, 1024, 0, 30, 0, 64, 30),
+ MARVELL_LAYOUT( 8192, 512, 4, 4, 4, 2048, 0, 30, 0, 0, 0),
+ MARVELL_LAYOUT( 8192, 512, 8, 9, 8, 1024, 0, 30, 0, 160, 30),
};
/**
return 0;
}
-static void marvell_nfc_select_chip(struct mtd_info *mtd, int die_nr)
+static void marvell_nfc_select_chip(struct nand_chip *chip, int die_nr)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct marvell_nand_chip *marvell_nand = to_marvell_nand(chip);
struct marvell_nfc *nfc = to_marvell_nfc(chip->controller);
u32 ndcr_generic;
marvell_nfc_disable_int(nfc, st & NDCR_ALL_INT);
- if (!(st & (NDSR_RDDREQ | NDSR_WRDREQ | NDSR_WRCMDREQ)))
+ if (st & (NDSR_RDY(0) | NDSR_RDY(1)))
complete(&nfc->complete);
return IRQ_HANDLED;
return ret;
}
-static int marvell_nfc_hw_ecc_hmg_read_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip, u8 *buf,
+static int marvell_nfc_hw_ecc_hmg_read_page_raw(struct nand_chip *chip, u8 *buf,
int oob_required, int page)
{
return marvell_nfc_hw_ecc_hmg_do_read_page(chip, buf, chip->oob_poi,
true, page);
}
-static int marvell_nfc_hw_ecc_hmg_read_page(struct mtd_info *mtd,
- struct nand_chip *chip,
- u8 *buf, int oob_required,
- int page)
+static int marvell_nfc_hw_ecc_hmg_read_page(struct nand_chip *chip, u8 *buf,
+ int oob_required, int page)
{
const struct marvell_hw_ecc_layout *lt = to_marvell_nand(chip)->layout;
unsigned int full_sz = lt->data_bytes + lt->spare_bytes + lt->ecc_bytes;
* it appears before the ECC bytes when reading), the ->read_oob_raw() function
* also stands for ->read_oob().
*/
-static int marvell_nfc_hw_ecc_hmg_read_oob_raw(struct mtd_info *mtd,
- struct nand_chip *chip, int page)
+static int marvell_nfc_hw_ecc_hmg_read_oob_raw(struct nand_chip *chip, int page)
{
/* Invalidate page cache */
chip->pagebuf = -1;
return ret;
}
-static int marvell_nfc_hw_ecc_hmg_write_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int marvell_nfc_hw_ecc_hmg_write_page_raw(struct nand_chip *chip,
const u8 *buf,
int oob_required, int page)
{
true, page);
}
-static int marvell_nfc_hw_ecc_hmg_write_page(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int marvell_nfc_hw_ecc_hmg_write_page(struct nand_chip *chip,
const u8 *buf,
int oob_required, int page)
{
* it appears before the ECC bytes when reading), the ->write_oob_raw() function
* also stands for ->write_oob().
*/
-static int marvell_nfc_hw_ecc_hmg_write_oob_raw(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int marvell_nfc_hw_ecc_hmg_write_oob_raw(struct nand_chip *chip,
int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
/* Invalidate page cache */
chip->pagebuf = -1;
}
/* BCH read helpers */
-static int marvell_nfc_hw_ecc_bch_read_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip, u8 *buf,
+static int marvell_nfc_hw_ecc_bch_read_page_raw(struct nand_chip *chip, u8 *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
const struct marvell_hw_ecc_layout *lt = to_marvell_nand(chip)->layout;
u8 *oob = chip->oob_poi;
int chunk_size = lt->data_bytes + lt->spare_bytes + lt->ecc_bytes;
}
}
-static int marvell_nfc_hw_ecc_bch_read_page(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int marvell_nfc_hw_ecc_bch_read_page(struct nand_chip *chip,
u8 *buf, int oob_required,
int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
const struct marvell_hw_ecc_layout *lt = to_marvell_nand(chip)->layout;
- int data_len = lt->data_bytes, spare_len = lt->spare_bytes, ecc_len;
- u8 *data = buf, *spare = chip->oob_poi, *ecc;
+ int data_len = lt->data_bytes, spare_len = lt->spare_bytes;
+ u8 *data = buf, *spare = chip->oob_poi;
int max_bitflips = 0;
u32 failure_mask = 0;
- int chunk, ecc_offset_in_page, ret;
+ int chunk, ret;
/*
* With BCH, OOB is not fully used (and thus not read entirely), not
* the controller in normal mode and must be re-read in raw mode. To
* avoid dropping the performances, we prefer not to include them. The
* user should re-read the page in raw mode if ECC bytes are required.
+ */
+
+ /*
+ * In case there is any subpage read error reported by ->correct(), we
+ * usually re-read only ECC bytes in raw mode and check if the whole
+ * page is empty. In this case, it is normal that the ECC check failed
+ * and we just ignore the error.
*
- * However, for any subpage read error reported by ->correct(), the ECC
- * bytes must be read in raw mode and the full subpage must be checked
- * to see if it is entirely empty of if there was an actual error.
+ * However, it has been empirically observed that for some layouts (e.g
+ * 2k page, 8b strength per 512B chunk), the controller tries to correct
+ * bits and may create itself bitflips in the erased area. To overcome
+ * this strange behavior, the whole page is re-read in raw mode, not
+ * only the ECC bytes.
*/
for (chunk = 0; chunk < lt->nchunks; chunk++) {
+ int data_off_in_page, spare_off_in_page, ecc_off_in_page;
+ int data_off, spare_off, ecc_off;
+ int data_len, spare_len, ecc_len;
+
/* No failure reported for this chunk, move to the next one */
if (!(failure_mask & BIT(chunk)))
continue;
- /* Derive ECC bytes positions (in page/buffer) and length */
- ecc = chip->oob_poi +
- (lt->full_chunk_cnt * lt->spare_bytes) +
- lt->last_spare_bytes +
- (chunk * ALIGN(lt->ecc_bytes, 32));
- ecc_offset_in_page =
- (chunk * (lt->data_bytes + lt->spare_bytes +
- lt->ecc_bytes)) +
- (chunk < lt->full_chunk_cnt ?
- lt->data_bytes + lt->spare_bytes :
- lt->last_data_bytes + lt->last_spare_bytes);
- ecc_len = chunk < lt->full_chunk_cnt ?
- lt->ecc_bytes : lt->last_ecc_bytes;
-
- /* Do the actual raw read of the ECC bytes */
- nand_change_read_column_op(chip, ecc_offset_in_page,
- ecc, ecc_len, false);
-
- /* Derive data/spare bytes positions (in buffer) and length */
- data = buf + (chunk * lt->data_bytes);
- data_len = chunk < lt->full_chunk_cnt ?
- lt->data_bytes : lt->last_data_bytes;
- spare = chip->oob_poi + (chunk * (lt->spare_bytes +
- lt->ecc_bytes));
- spare_len = chunk < lt->full_chunk_cnt ?
- lt->spare_bytes : lt->last_spare_bytes;
+ data_off_in_page = chunk * (lt->data_bytes + lt->spare_bytes +
+ lt->ecc_bytes);
+ spare_off_in_page = data_off_in_page +
+ (chunk < lt->full_chunk_cnt ? lt->data_bytes :
+ lt->last_data_bytes);
+ ecc_off_in_page = spare_off_in_page +
+ (chunk < lt->full_chunk_cnt ? lt->spare_bytes :
+ lt->last_spare_bytes);
+
+ data_off = chunk * lt->data_bytes;
+ spare_off = chunk * lt->spare_bytes;
+ ecc_off = (lt->full_chunk_cnt * lt->spare_bytes) +
+ lt->last_spare_bytes +
+ (chunk * (lt->ecc_bytes + 2));
+
+ data_len = chunk < lt->full_chunk_cnt ? lt->data_bytes :
+ lt->last_data_bytes;
+ spare_len = chunk < lt->full_chunk_cnt ? lt->spare_bytes :
+ lt->last_spare_bytes;
+ ecc_len = chunk < lt->full_chunk_cnt ? lt->ecc_bytes :
+ lt->last_ecc_bytes;
+
+ /*
+ * Only re-read the ECC bytes, unless we are using the 2k/8b
+ * layout which is buggy in the sense that the ECC engine will
+ * try to correct data bytes anyway, creating bitflips. In this
+ * case, re-read the entire page.
+ */
+ if (lt->writesize == 2048 && lt->strength == 8) {
+ nand_change_read_column_op(chip, data_off_in_page,
+ buf + data_off, data_len,
+ false);
+ nand_change_read_column_op(chip, spare_off_in_page,
+ chip->oob_poi + spare_off, spare_len,
+ false);
+ }
+
+ nand_change_read_column_op(chip, ecc_off_in_page,
+ chip->oob_poi + ecc_off, ecc_len,
+ false);
/* Check the entire chunk (data + spare + ecc) for emptyness */
- marvell_nfc_check_empty_chunk(chip, data, data_len, spare,
- spare_len, ecc, ecc_len,
+ marvell_nfc_check_empty_chunk(chip, buf + data_off, data_len,
+ chip->oob_poi + spare_off, spare_len,
+ chip->oob_poi + ecc_off, ecc_len,
&max_bitflips);
}
return max_bitflips;
}
-static int marvell_nfc_hw_ecc_bch_read_oob_raw(struct mtd_info *mtd,
- struct nand_chip *chip, int page)
+static int marvell_nfc_hw_ecc_bch_read_oob_raw(struct nand_chip *chip, int page)
{
/* Invalidate page cache */
chip->pagebuf = -1;
- return chip->ecc.read_page_raw(mtd, chip, chip->data_buf, true, page);
+ return chip->ecc.read_page_raw(chip, chip->data_buf, true, page);
}
-static int marvell_nfc_hw_ecc_bch_read_oob(struct mtd_info *mtd,
- struct nand_chip *chip, int page)
+static int marvell_nfc_hw_ecc_bch_read_oob(struct nand_chip *chip, int page)
{
/* Invalidate page cache */
chip->pagebuf = -1;
- return chip->ecc.read_page(mtd, chip, chip->data_buf, true, page);
+ return chip->ecc.read_page(chip, chip->data_buf, true, page);
}
/* BCH write helpers */
-static int marvell_nfc_hw_ecc_bch_write_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int marvell_nfc_hw_ecc_bch_write_page_raw(struct nand_chip *chip,
const u8 *buf,
int oob_required, int page)
{
return 0;
}
-static int marvell_nfc_hw_ecc_bch_write_page(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int marvell_nfc_hw_ecc_bch_write_page(struct nand_chip *chip,
const u8 *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
const struct marvell_hw_ecc_layout *lt = to_marvell_nand(chip)->layout;
const u8 *data = buf;
const u8 *spare = chip->oob_poi;
return 0;
}
-static int marvell_nfc_hw_ecc_bch_write_oob_raw(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int marvell_nfc_hw_ecc_bch_write_oob_raw(struct nand_chip *chip,
int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
/* Invalidate page cache */
chip->pagebuf = -1;
memset(chip->data_buf, 0xFF, mtd->writesize);
- return chip->ecc.write_page_raw(mtd, chip, chip->data_buf, true, page);
+ return chip->ecc.write_page_raw(chip, chip->data_buf, true, page);
}
-static int marvell_nfc_hw_ecc_bch_write_oob(struct mtd_info *mtd,
- struct nand_chip *chip, int page)
+static int marvell_nfc_hw_ecc_bch_write_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
/* Invalidate page cache */
chip->pagebuf = -1;
memset(chip->data_buf, 0xFF, mtd->writesize);
- return chip->ecc.write_page(mtd, chip, chip->data_buf, true, page);
+ return chip->ecc.write_page(chip, chip->data_buf, true, page);
}
/* NAND framework ->exec_op() hooks and related helpers */
return -ENOTSUPP;
}
+ /* Special care for the layout 2k/8-bit/512B */
+ if (l->writesize == 2048 && l->strength == 8) {
+ if (mtd->oobsize < 128) {
+ dev_err(nfc->dev, "Requested layout needs at least 128 OOB bytes\n");
+ return -ENOTSUPP;
+ } else {
+ chip->bbt_options |= NAND_BBT_NO_OOB_BBM;
+ }
+ }
+
mtd_set_ooblayout(mtd, &marvell_nand_ooblayout_ops);
ecc->steps = l->nchunks;
ecc->size = l->data_bytes;
.pattern = bbt_mirror_pattern
};
-static int marvell_nfc_setup_data_interface(struct mtd_info *mtd, int chipnr,
+static int marvell_nfc_setup_data_interface(struct nand_chip *chip, int chipnr,
const struct nand_data_interface
*conf)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct marvell_nand_chip *marvell_nand = to_marvell_nand(chip);
struct marvell_nfc *nfc = to_marvell_nfc(chip->controller);
unsigned int period_ns = 1000000000 / clk_get_rate(nfc->core_clk) * 2;
chip->options |= NAND_BUSWIDTH_AUTO;
- ret = nand_scan(mtd, marvell_nand->nsels);
+ ret = nand_scan(chip, marvell_nand->nsels);
if (ret) {
dev_err(dev, "could not scan the nand chip\n");
return ret;
ret = mtd_device_register(mtd, NULL, 0);
if (ret) {
dev_err(dev, "failed to register mtd device: %d\n", ret);
- nand_release(mtd);
+ nand_release(chip);
return ret;
}
struct marvell_nand_chip *entry, *temp;
list_for_each_entry_safe(entry, temp, &nfc->chips, node) {
- nand_release(nand_to_mtd(&entry->chip));
+ nand_release(&entry->chip);
list_del(&entry->node);
}
}
struct regmap *sysctrl_base =
syscon_regmap_lookup_by_phandle(np,
"marvell,system-controller");
- u32 reg;
if (IS_ERR(sysctrl_base))
return PTR_ERR(sysctrl_base);
- reg = GENCONF_SOC_DEVICE_MUX_NFC_EN |
- GENCONF_SOC_DEVICE_MUX_ECC_CLK_RST |
- GENCONF_SOC_DEVICE_MUX_ECC_CORE_RST |
- GENCONF_SOC_DEVICE_MUX_NFC_INT_EN;
- regmap_write(sysctrl_base, GENCONF_SOC_DEVICE_MUX, reg);
+ regmap_write(sysctrl_base, GENCONF_SOC_DEVICE_MUX,
+ GENCONF_SOC_DEVICE_MUX_NFC_EN |
+ GENCONF_SOC_DEVICE_MUX_ECC_CLK_RST |
+ GENCONF_SOC_DEVICE_MUX_ECC_CORE_RST |
+ GENCONF_SOC_DEVICE_MUX_NFC_INT_EN);
- regmap_read(sysctrl_base, GENCONF_CLK_GATING_CTRL, ®);
- reg |= GENCONF_CLK_GATING_CTRL_ND_GATE;
- regmap_write(sysctrl_base, GENCONF_CLK_GATING_CTRL, reg);
+ regmap_update_bits(sysctrl_base, GENCONF_CLK_GATING_CTRL,
+ GENCONF_CLK_GATING_CTRL_ND_GATE,
+ GENCONF_CLK_GATING_CTRL_ND_GATE);
- regmap_read(sysctrl_base, GENCONF_ND_CLK_CTRL, ®);
- reg |= GENCONF_ND_CLK_CTRL_EN;
- regmap_write(sysctrl_base, GENCONF_ND_CLK_CTRL, reg);
+ regmap_update_bits(sysctrl_base, GENCONF_ND_CLK_CTRL,
+ GENCONF_ND_CLK_CTRL_EN,
+ GENCONF_ND_CLK_CTRL_EN);
}
/* Configure the DMA if appropriate */
}
/* Control chip select signals */
-static void mpc5121_nfc_select_chip(struct mtd_info *mtd, int chip)
+static void mpc5121_nfc_select_chip(struct nand_chip *nand, int chip)
{
+ struct mtd_info *mtd = nand_to_mtd(nand);
+
if (chip < 0) {
nfc_clear(mtd, NFC_CONFIG1, NFC_CE);
return;
}
/* Control chips select signal on ADS5121 board */
-static void ads5121_select_chip(struct mtd_info *mtd, int chip)
+static void ads5121_select_chip(struct nand_chip *nand, int chip)
{
- struct nand_chip *nand = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(nand);
struct mpc5121_nfc_prv *prv = nand_get_controller_data(nand);
u8 v;
v |= 0x0F;
if (chip >= 0) {
- mpc5121_nfc_select_chip(mtd, 0);
+ mpc5121_nfc_select_chip(nand, 0);
v &= ~(1 << chip);
} else
- mpc5121_nfc_select_chip(mtd, -1);
+ mpc5121_nfc_select_chip(nand, -1);
out_8(prv->csreg, v);
}
/* Read NAND Ready/Busy signal */
-static int mpc5121_nfc_dev_ready(struct mtd_info *mtd)
+static int mpc5121_nfc_dev_ready(struct nand_chip *nand)
{
/*
* NFC handles ready/busy signal internally. Therefore, this function
}
/* Write command to NAND flash */
-static void mpc5121_nfc_command(struct mtd_info *mtd, unsigned command,
- int column, int page)
+static void mpc5121_nfc_command(struct nand_chip *chip, unsigned command,
+ int column, int page)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct mpc5121_nfc_prv *prv = nand_get_controller_data(chip);
prv->column = (column >= 0) ? column : 0;
break;
case NAND_CMD_SEQIN:
- mpc5121_nfc_command(mtd, NAND_CMD_READ0, column, page);
+ mpc5121_nfc_command(chip, NAND_CMD_READ0, column, page);
column = 0;
break;
}
/* Read data from NFC buffers */
-static void mpc5121_nfc_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void mpc5121_nfc_read_buf(struct nand_chip *chip, u_char *buf, int len)
{
- mpc5121_nfc_buf_copy(mtd, buf, len, 0);
+ mpc5121_nfc_buf_copy(nand_to_mtd(chip), buf, len, 0);
}
/* Write data to NFC buffers */
-static void mpc5121_nfc_write_buf(struct mtd_info *mtd,
- const u_char *buf, int len)
+static void mpc5121_nfc_write_buf(struct nand_chip *chip, const u_char *buf,
+ int len)
{
- mpc5121_nfc_buf_copy(mtd, (u_char *)buf, len, 1);
+ mpc5121_nfc_buf_copy(nand_to_mtd(chip), (u_char *)buf, len, 1);
}
/* Read byte from NFC buffers */
-static u8 mpc5121_nfc_read_byte(struct mtd_info *mtd)
+static u8 mpc5121_nfc_read_byte(struct nand_chip *chip)
{
u8 tmp;
- mpc5121_nfc_read_buf(mtd, &tmp, sizeof(tmp));
-
- return tmp;
-}
-
-/* Read word from NFC buffers */
-static u16 mpc5121_nfc_read_word(struct mtd_info *mtd)
-{
- u16 tmp;
-
- mpc5121_nfc_read_buf(mtd, (u_char *)&tmp, sizeof(tmp));
+ mpc5121_nfc_read_buf(chip, &tmp, sizeof(tmp));
return tmp;
}
}
mtd->name = "MPC5121 NAND";
- chip->dev_ready = mpc5121_nfc_dev_ready;
- chip->cmdfunc = mpc5121_nfc_command;
- chip->read_byte = mpc5121_nfc_read_byte;
- chip->read_word = mpc5121_nfc_read_word;
- chip->read_buf = mpc5121_nfc_read_buf;
- chip->write_buf = mpc5121_nfc_write_buf;
+ chip->legacy.dev_ready = mpc5121_nfc_dev_ready;
+ chip->legacy.cmdfunc = mpc5121_nfc_command;
+ chip->legacy.read_byte = mpc5121_nfc_read_byte;
+ chip->legacy.read_buf = mpc5121_nfc_read_buf;
+ chip->legacy.write_buf = mpc5121_nfc_write_buf;
chip->select_chip = mpc5121_nfc_select_chip;
- chip->set_features = nand_get_set_features_notsupp;
- chip->get_features = nand_get_set_features_notsupp;
+ chip->legacy.set_features = nand_get_set_features_notsupp;
+ chip->legacy.get_features = nand_get_set_features_notsupp;
chip->bbt_options = NAND_BBT_USE_FLASH;
chip->ecc.mode = NAND_ECC_SOFT;
chip->ecc.algo = NAND_ECC_HAMMING;
}
/* Detect NAND chips */
- retval = nand_scan(mtd, be32_to_cpup(chips_no));
+ retval = nand_scan(chip, be32_to_cpup(chips_no));
if (retval) {
dev_err(dev, "NAND Flash not found !\n");
goto error;
struct device *dev = &op->dev;
struct mtd_info *mtd = dev_get_drvdata(dev);
- nand_release(mtd);
+ nand_release(mtd_to_nand(mtd));
mpc5121_nfc_free(dev, mtd);
return 0;
return 0;
}
-static void mtk_nfc_select_chip(struct mtd_info *mtd, int chip)
+static void mtk_nfc_select_chip(struct nand_chip *nand, int chip)
{
- struct nand_chip *nand = mtd_to_nand(mtd);
struct mtk_nfc *nfc = nand_get_controller_data(nand);
struct mtk_nfc_nand_chip *mtk_nand = to_mtk_nand(nand);
if (chip < 0)
return;
- mtk_nfc_hw_runtime_config(mtd);
+ mtk_nfc_hw_runtime_config(nand_to_mtd(nand));
nfi_writel(nfc, mtk_nand->sels[chip], NFI_CSEL);
}
-static int mtk_nfc_dev_ready(struct mtd_info *mtd)
+static int mtk_nfc_dev_ready(struct nand_chip *nand)
{
- struct mtk_nfc *nfc = nand_get_controller_data(mtd_to_nand(mtd));
+ struct mtk_nfc *nfc = nand_get_controller_data(nand);
if (nfi_readl(nfc, NFI_STA) & STA_BUSY)
return 0;
return 1;
}
-static void mtk_nfc_cmd_ctrl(struct mtd_info *mtd, int dat, unsigned int ctrl)
+static void mtk_nfc_cmd_ctrl(struct nand_chip *chip, int dat,
+ unsigned int ctrl)
{
- struct mtk_nfc *nfc = nand_get_controller_data(mtd_to_nand(mtd));
+ struct mtk_nfc *nfc = nand_get_controller_data(chip);
if (ctrl & NAND_ALE) {
mtk_nfc_send_address(nfc, dat);
dev_err(nfc->dev, "data not ready\n");
}
-static inline u8 mtk_nfc_read_byte(struct mtd_info *mtd)
+static inline u8 mtk_nfc_read_byte(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct mtk_nfc *nfc = nand_get_controller_data(chip);
u32 reg;
return nfi_readb(nfc, NFI_DATAR);
}
-static void mtk_nfc_read_buf(struct mtd_info *mtd, u8 *buf, int len)
+static void mtk_nfc_read_buf(struct nand_chip *chip, u8 *buf, int len)
{
int i;
for (i = 0; i < len; i++)
- buf[i] = mtk_nfc_read_byte(mtd);
+ buf[i] = mtk_nfc_read_byte(chip);
}
-static void mtk_nfc_write_byte(struct mtd_info *mtd, u8 byte)
+static void mtk_nfc_write_byte(struct nand_chip *chip, u8 byte)
{
- struct mtk_nfc *nfc = nand_get_controller_data(mtd_to_nand(mtd));
+ struct mtk_nfc *nfc = nand_get_controller_data(chip);
u32 reg;
reg = nfi_readl(nfc, NFI_STA) & NFI_FSM_MASK;
nfi_writeb(nfc, byte, NFI_DATAW);
}
-static void mtk_nfc_write_buf(struct mtd_info *mtd, const u8 *buf, int len)
+static void mtk_nfc_write_buf(struct nand_chip *chip, const u8 *buf, int len)
{
int i;
for (i = 0; i < len; i++)
- mtk_nfc_write_byte(mtd, buf[i]);
+ mtk_nfc_write_byte(chip, buf[i]);
}
-static int mtk_nfc_setup_data_interface(struct mtd_info *mtd, int csline,
+static int mtk_nfc_setup_data_interface(struct nand_chip *chip, int csline,
const struct nand_data_interface *conf)
{
- struct mtk_nfc *nfc = nand_get_controller_data(mtd_to_nand(mtd));
+ struct mtk_nfc *nfc = nand_get_controller_data(chip);
const struct nand_sdr_timings *timings;
u32 rate, tpoecs, tprecs, tc2r, tw2r, twh, twst, trlt;
return nand_prog_page_end_op(chip);
}
-static int mtk_nfc_write_page_hwecc(struct mtd_info *mtd,
- struct nand_chip *chip, const u8 *buf,
+static int mtk_nfc_write_page_hwecc(struct nand_chip *chip, const u8 *buf,
int oob_on, int page)
{
- return mtk_nfc_write_page(mtd, chip, buf, page, 0);
+ return mtk_nfc_write_page(nand_to_mtd(chip), chip, buf, page, 0);
}
-static int mtk_nfc_write_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- const u8 *buf, int oob_on, int pg)
+static int mtk_nfc_write_page_raw(struct nand_chip *chip, const u8 *buf,
+ int oob_on, int pg)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct mtk_nfc *nfc = nand_get_controller_data(chip);
mtk_nfc_format_page(mtd, buf);
return mtk_nfc_write_page(mtd, chip, nfc->buffer, pg, 1);
}
-static int mtk_nfc_write_subpage_hwecc(struct mtd_info *mtd,
- struct nand_chip *chip, u32 offset,
+static int mtk_nfc_write_subpage_hwecc(struct nand_chip *chip, u32 offset,
u32 data_len, const u8 *buf,
int oob_on, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct mtk_nfc *nfc = nand_get_controller_data(chip);
int ret;
return mtk_nfc_write_page(mtd, chip, nfc->buffer, page, 1);
}
-static int mtk_nfc_write_oob_std(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int mtk_nfc_write_oob_std(struct nand_chip *chip, int page)
{
- return mtk_nfc_write_page_raw(mtd, chip, NULL, 1, page);
+ return mtk_nfc_write_page_raw(chip, NULL, 1, page);
}
static int mtk_nfc_update_ecc_stats(struct mtd_info *mtd, u8 *buf, u32 sectors)
return bitflips;
}
-static int mtk_nfc_read_subpage_hwecc(struct mtd_info *mtd,
- struct nand_chip *chip, u32 off,
+static int mtk_nfc_read_subpage_hwecc(struct nand_chip *chip, u32 off,
u32 len, u8 *p, int pg)
{
- return mtk_nfc_read_subpage(mtd, chip, off, len, p, pg, 0);
+ return mtk_nfc_read_subpage(nand_to_mtd(chip), chip, off, len, p, pg,
+ 0);
}
-static int mtk_nfc_read_page_hwecc(struct mtd_info *mtd,
- struct nand_chip *chip, u8 *p,
- int oob_on, int pg)
+static int mtk_nfc_read_page_hwecc(struct nand_chip *chip, u8 *p, int oob_on,
+ int pg)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
return mtk_nfc_read_subpage(mtd, chip, 0, mtd->writesize, p, pg, 0);
}
-static int mtk_nfc_read_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- u8 *buf, int oob_on, int page)
+static int mtk_nfc_read_page_raw(struct nand_chip *chip, u8 *buf, int oob_on,
+ int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct mtk_nfc_nand_chip *mtk_nand = to_mtk_nand(chip);
struct mtk_nfc *nfc = nand_get_controller_data(chip);
struct mtk_nfc_fdm *fdm = &mtk_nand->fdm;
return ret;
}
-static int mtk_nfc_read_oob_std(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int mtk_nfc_read_oob_std(struct nand_chip *chip, int page)
{
- return mtk_nfc_read_page_raw(mtd, chip, NULL, 1, page);
+ return mtk_nfc_read_page_raw(chip, NULL, 1, page);
}
static inline void mtk_nfc_hw_init(struct mtk_nfc *nfc)
nand_set_controller_data(nand, nfc);
nand->options |= NAND_USE_BOUNCE_BUFFER | NAND_SUBPAGE_READ;
- nand->dev_ready = mtk_nfc_dev_ready;
+ nand->legacy.dev_ready = mtk_nfc_dev_ready;
nand->select_chip = mtk_nfc_select_chip;
- nand->write_byte = mtk_nfc_write_byte;
- nand->write_buf = mtk_nfc_write_buf;
- nand->read_byte = mtk_nfc_read_byte;
- nand->read_buf = mtk_nfc_read_buf;
- nand->cmd_ctrl = mtk_nfc_cmd_ctrl;
+ nand->legacy.write_byte = mtk_nfc_write_byte;
+ nand->legacy.write_buf = mtk_nfc_write_buf;
+ nand->legacy.read_byte = mtk_nfc_read_byte;
+ nand->legacy.read_buf = mtk_nfc_read_buf;
+ nand->legacy.cmd_ctrl = mtk_nfc_cmd_ctrl;
nand->setup_data_interface = mtk_nfc_setup_data_interface;
/* set default mode in case dt entry is missing */
mtk_nfc_hw_init(nfc);
- ret = nand_scan(mtd, nsels);
+ ret = nand_scan(nand, nsels);
if (ret)
return ret;
ret = mtd_device_register(mtd, NULL, 0);
if (ret) {
dev_err(dev, "mtd parse partition error\n");
- nand_release(mtd);
+ nand_release(nand);
return ret;
}
while (!list_empty(&nfc->chips)) {
chip = list_first_entry(&nfc->chips, struct mtk_nfc_nand_chip,
node);
- nand_release(nand_to_mtd(&chip->nand));
+ nand_release(&chip->nand);
list_del(&chip->node);
}
void (*irq_control)(struct mxc_nand_host *, int);
u32 (*get_ecc_status)(struct mxc_nand_host *);
const struct mtd_ooblayout_ops *ooblayout;
- void (*select_chip)(struct mtd_info *mtd, int chip);
- int (*setup_data_interface)(struct mtd_info *mtd, int csline,
+ void (*select_chip)(struct nand_chip *chip, int cs);
+ int (*setup_data_interface)(struct nand_chip *chip, int csline,
const struct nand_data_interface *conf);
void (*enable_hwecc)(struct nand_chip *chip, bool enable);
}
/* This functions is used by upper layer to checks if device is ready */
-static int mxc_nand_dev_ready(struct mtd_info *mtd)
+static int mxc_nand_dev_ready(struct nand_chip *chip)
{
/*
* NFC handles R/B internally. Therefore, this function
return max_bitflips;
}
-static int mxc_nand_read_page(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int mxc_nand_read_page(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
struct mxc_nand_host *host = nand_get_controller_data(chip);
void *oob_buf;
return host->devtype_data->read_page(chip, buf, oob_buf, 1, page);
}
-static int mxc_nand_read_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int mxc_nand_read_page_raw(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
struct mxc_nand_host *host = nand_get_controller_data(chip);
void *oob_buf;
return host->devtype_data->read_page(chip, buf, oob_buf, 0, page);
}
-static int mxc_nand_read_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int mxc_nand_read_oob(struct nand_chip *chip, int page)
{
struct mxc_nand_host *host = nand_get_controller_data(chip);
return 0;
}
-static int mxc_nand_write_page_ecc(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required,
- int page)
+static int mxc_nand_write_page_ecc(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
return mxc_nand_write_page(chip, buf, true, page);
}
-static int mxc_nand_write_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page)
+static int mxc_nand_write_page_raw(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
return mxc_nand_write_page(chip, buf, false, page);
}
-static int mxc_nand_write_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int mxc_nand_write_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct mxc_nand_host *host = nand_get_controller_data(chip);
memset(host->data_buf, 0xff, mtd->writesize);
return mxc_nand_write_page(chip, host->data_buf, false, page);
}
-static u_char mxc_nand_read_byte(struct mtd_info *mtd)
+static u_char mxc_nand_read_byte(struct nand_chip *nand_chip)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
struct mxc_nand_host *host = nand_get_controller_data(nand_chip);
uint8_t ret;
return ret;
}
-static uint16_t mxc_nand_read_word(struct mtd_info *mtd)
-{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
- struct mxc_nand_host *host = nand_get_controller_data(nand_chip);
- uint16_t ret;
-
- ret = *(uint16_t *)(host->data_buf + host->buf_start);
- host->buf_start += 2;
-
- return ret;
-}
-
/* Write data of length len to buffer buf. The data to be
* written on NAND Flash is first copied to RAMbuffer. After the Data Input
* Operation by the NFC, the data is written to NAND Flash */
-static void mxc_nand_write_buf(struct mtd_info *mtd,
- const u_char *buf, int len)
+static void mxc_nand_write_buf(struct nand_chip *nand_chip, const u_char *buf,
+ int len)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(nand_chip);
struct mxc_nand_host *host = nand_get_controller_data(nand_chip);
u16 col = host->buf_start;
int n = mtd->oobsize + mtd->writesize - col;
* Flash first the data output cycle is initiated by the NFC, which copies
* the data to RAMbuffer. This data of length len is then copied to buffer buf.
*/
-static void mxc_nand_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void mxc_nand_read_buf(struct nand_chip *nand_chip, u_char *buf,
+ int len)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(nand_chip);
struct mxc_nand_host *host = nand_get_controller_data(nand_chip);
u16 col = host->buf_start;
int n = mtd->oobsize + mtd->writesize - col;
/* This function is used by upper layer for select and
* deselect of the NAND chip */
-static void mxc_nand_select_chip_v1_v3(struct mtd_info *mtd, int chip)
+static void mxc_nand_select_chip_v1_v3(struct nand_chip *nand_chip, int chip)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
struct mxc_nand_host *host = nand_get_controller_data(nand_chip);
if (chip == -1) {
}
}
-static void mxc_nand_select_chip_v2(struct mtd_info *mtd, int chip)
+static void mxc_nand_select_chip_v2(struct nand_chip *nand_chip, int chip)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
struct mxc_nand_host *host = nand_get_controller_data(nand_chip);
if (chip == -1) {
writew(0x4, NFC_V1_V2_WRPROT);
}
-static int mxc_nand_v2_setup_data_interface(struct mtd_info *mtd, int csline,
+static int mxc_nand_v2_setup_data_interface(struct nand_chip *chip, int csline,
const struct nand_data_interface *conf)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
- struct mxc_nand_host *host = nand_get_controller_data(nand_chip);
+ struct mxc_nand_host *host = nand_get_controller_data(chip);
int tRC_min_ns, tRC_ps, ret;
unsigned long rate, rate_round;
const struct nand_sdr_timings *timings;
/* Used by the upper layer to write command to NAND Flash for
* different operations to be carried out on NAND Flash */
-static void mxc_nand_command(struct mtd_info *mtd, unsigned command,
- int column, int page_addr)
+static void mxc_nand_command(struct nand_chip *nand_chip, unsigned command,
+ int column, int page_addr)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(nand_chip);
struct mxc_nand_host *host = nand_get_controller_data(nand_chip);
dev_dbg(host->dev, "mxc_nand_command (cmd = 0x%x, col = 0x%x, page = 0x%x)\n",
}
}
-static int mxc_nand_set_features(struct mtd_info *mtd, struct nand_chip *chip,
- int addr, u8 *subfeature_param)
+static int mxc_nand_set_features(struct nand_chip *chip, int addr,
+ u8 *subfeature_param)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
- struct mxc_nand_host *host = nand_get_controller_data(nand_chip);
+ struct mtd_info *mtd = nand_to_mtd(chip);
+ struct mxc_nand_host *host = nand_get_controller_data(chip);
int i;
host->buf_start = 0;
for (i = 0; i < ONFI_SUBFEATURE_PARAM_LEN; ++i)
- chip->write_byte(mtd, subfeature_param[i]);
+ chip->legacy.write_byte(chip, subfeature_param[i]);
memcpy32_toio(host->main_area0, host->data_buf, mtd->writesize);
host->devtype_data->send_cmd(host, NAND_CMD_SET_FEATURES, false);
return 0;
}
-static int mxc_nand_get_features(struct mtd_info *mtd, struct nand_chip *chip,
- int addr, u8 *subfeature_param)
+static int mxc_nand_get_features(struct nand_chip *chip, int addr,
+ u8 *subfeature_param)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
- struct mxc_nand_host *host = nand_get_controller_data(nand_chip);
+ struct mtd_info *mtd = nand_to_mtd(chip);
+ struct mxc_nand_host *host = nand_get_controller_data(chip);
int i;
host->devtype_data->send_cmd(host, NAND_CMD_GET_FEATURES, false);
host->buf_start = 0;
for (i = 0; i < ONFI_SUBFEATURE_PARAM_LEN; ++i)
- *subfeature_param++ = chip->read_byte(mtd);
+ *subfeature_param++ = chip->legacy.read_byte(chip);
return 0;
}
mtd->name = DRIVER_NAME;
/* 50 us command delay time */
- this->chip_delay = 5;
+ this->legacy.chip_delay = 5;
nand_set_controller_data(this, host);
nand_set_flash_node(this, pdev->dev.of_node),
- this->dev_ready = mxc_nand_dev_ready;
- this->cmdfunc = mxc_nand_command;
- this->read_byte = mxc_nand_read_byte;
- this->read_word = mxc_nand_read_word;
- this->write_buf = mxc_nand_write_buf;
- this->read_buf = mxc_nand_read_buf;
- this->set_features = mxc_nand_set_features;
- this->get_features = mxc_nand_get_features;
+ this->legacy.dev_ready = mxc_nand_dev_ready;
+ this->legacy.cmdfunc = mxc_nand_command;
+ this->legacy.read_byte = mxc_nand_read_byte;
+ this->legacy.write_buf = mxc_nand_write_buf;
+ this->legacy.read_buf = mxc_nand_read_buf;
+ this->legacy.set_features = mxc_nand_set_features;
+ this->legacy.get_features = mxc_nand_get_features;
host->clk = devm_clk_get(&pdev->dev, NULL);
if (IS_ERR(host->clk))
/* Scan the NAND device */
this->dummy_controller.ops = &mxcnd_controller_ops;
- err = nand_scan(mtd, is_imx25_nfc(host) ? 4 : 1);
+ err = nand_scan(this, is_imx25_nfc(host) ? 4 : 1);
if (err)
goto escan;
{
struct mxc_nand_host *host = platform_get_drvdata(pdev);
- nand_release(nand_to_mtd(&host->nand));
+ nand_release(&host->nand);
if (host->clk_act)
clk_disable_unprepare(host->clk);
* GNU General Public License for more details.
*/
-#include <linux/mtd/rawnand.h>
+#include "internals.h"
static void amd_nand_decode_id(struct nand_chip *chip)
{
#include <linux/sched.h>
#include <linux/slab.h>
#include <linux/mm.h>
-#include <linux/nmi.h>
#include <linux/types.h>
#include <linux/mtd/mtd.h>
-#include <linux/mtd/rawnand.h>
#include <linux/mtd/nand_ecc.h>
#include <linux/mtd/nand_bch.h>
#include <linux/interrupt.h>
#include <linux/mtd/partitions.h>
#include <linux/of.h>
+#include "internals.h"
+
static int nand_get_device(struct mtd_info *mtd, int new_state);
static int nand_do_write_oob(struct mtd_info *mtd, loff_t to,
spin_unlock(&chip->controller->lock);
}
-/**
- * nand_read_byte - [DEFAULT] read one byte from the chip
- * @mtd: MTD device structure
- *
- * Default read function for 8bit buswidth
- */
-static uint8_t nand_read_byte(struct mtd_info *mtd)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
- return readb(chip->IO_ADDR_R);
-}
-
-/**
- * nand_read_byte16 - [DEFAULT] read one byte endianness aware from the chip
- * @mtd: MTD device structure
- *
- * Default read function for 16bit buswidth with endianness conversion.
- *
- */
-static uint8_t nand_read_byte16(struct mtd_info *mtd)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
- return (uint8_t) cpu_to_le16(readw(chip->IO_ADDR_R));
-}
-
-/**
- * nand_read_word - [DEFAULT] read one word from the chip
- * @mtd: MTD device structure
- *
- * Default read function for 16bit buswidth without endianness conversion.
- */
-static u16 nand_read_word(struct mtd_info *mtd)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
- return readw(chip->IO_ADDR_R);
-}
-
-/**
- * nand_select_chip - [DEFAULT] control CE line
- * @mtd: MTD device structure
- * @chipnr: chipnumber to select, -1 for deselect
- *
- * Default select function for 1 chip devices.
- */
-static void nand_select_chip(struct mtd_info *mtd, int chipnr)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
- switch (chipnr) {
- case -1:
- chip->cmd_ctrl(mtd, NAND_CMD_NONE, 0 | NAND_CTRL_CHANGE);
- break;
- case 0:
- break;
-
- default:
- BUG();
- }
-}
-
-/**
- * nand_write_byte - [DEFAULT] write single byte to chip
- * @mtd: MTD device structure
- * @byte: value to write
- *
- * Default function to write a byte to I/O[7:0]
- */
-static void nand_write_byte(struct mtd_info *mtd, uint8_t byte)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
- chip->write_buf(mtd, &byte, 1);
-}
-
-/**
- * nand_write_byte16 - [DEFAULT] write single byte to a chip with width 16
- * @mtd: MTD device structure
- * @byte: value to write
- *
- * Default function to write a byte to I/O[7:0] on a 16-bit wide chip.
- */
-static void nand_write_byte16(struct mtd_info *mtd, uint8_t byte)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
- uint16_t word = byte;
-
- /*
- * It's not entirely clear what should happen to I/O[15:8] when writing
- * a byte. The ONFi spec (Revision 3.1; 2012-09-19, Section 2.16) reads:
- *
- * When the host supports a 16-bit bus width, only data is
- * transferred at the 16-bit width. All address and command line
- * transfers shall use only the lower 8-bits of the data bus. During
- * command transfers, the host may place any value on the upper
- * 8-bits of the data bus. During address transfers, the host shall
- * set the upper 8-bits of the data bus to 00h.
- *
- * One user of the write_byte callback is nand_set_features. The
- * four parameters are specified to be written to I/O[7:0], but this is
- * neither an address nor a command transfer. Let's assume a 0 on the
- * upper I/O lines is OK.
- */
- chip->write_buf(mtd, (uint8_t *)&word, 2);
-}
-
-/**
- * nand_write_buf - [DEFAULT] write buffer to chip
- * @mtd: MTD device structure
- * @buf: data buffer
- * @len: number of bytes to write
- *
- * Default write function for 8bit buswidth.
- */
-static void nand_write_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
- iowrite8_rep(chip->IO_ADDR_W, buf, len);
-}
-
-/**
- * nand_read_buf - [DEFAULT] read chip data into buffer
- * @mtd: MTD device structure
- * @buf: buffer to store date
- * @len: number of bytes to read
- *
- * Default read function for 8bit buswidth.
- */
-static void nand_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
- ioread8_rep(chip->IO_ADDR_R, buf, len);
-}
-
-/**
- * nand_write_buf16 - [DEFAULT] write buffer to chip
- * @mtd: MTD device structure
- * @buf: data buffer
- * @len: number of bytes to write
- *
- * Default write function for 16bit buswidth.
- */
-static void nand_write_buf16(struct mtd_info *mtd, const uint8_t *buf, int len)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
- u16 *p = (u16 *) buf;
-
- iowrite16_rep(chip->IO_ADDR_W, p, len >> 1);
-}
-
-/**
- * nand_read_buf16 - [DEFAULT] read chip data into buffer
- * @mtd: MTD device structure
- * @buf: buffer to store date
- * @len: number of bytes to read
- *
- * Default read function for 16bit buswidth.
- */
-static void nand_read_buf16(struct mtd_info *mtd, uint8_t *buf, int len)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
- u16 *p = (u16 *) buf;
-
- ioread16_rep(chip->IO_ADDR_R, p, len >> 1);
-}
-
/**
* nand_block_bad - [DEFAULT] Read bad block marker from the chip
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @ofs: offset from device start
*
* Check, if the block is bad.
*/
-static int nand_block_bad(struct mtd_info *mtd, loff_t ofs)
+static int nand_block_bad(struct nand_chip *chip, loff_t ofs)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int page, page_end, res;
- struct nand_chip *chip = mtd_to_nand(mtd);
u8 bad;
if (chip->bbt_options & NAND_BBT_SCANLASTPAGE)
page_end = page + (chip->bbt_options & NAND_BBT_SCAN2NDPAGE ? 2 : 1);
for (; page < page_end; page++) {
- res = chip->ecc.read_oob(mtd, chip, page);
+ res = chip->ecc.read_oob(chip, page);
if (res < 0)
return res;
/**
* nand_default_block_markbad - [DEFAULT] mark a block bad via bad block marker
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @ofs: offset from device start
*
* This is the default implementation, which can be overridden by a hardware
* specific driver. It provides the details for writing a bad block marker to a
* block.
*/
-static int nand_default_block_markbad(struct mtd_info *mtd, loff_t ofs)
+static int nand_default_block_markbad(struct nand_chip *chip, loff_t ofs)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct mtd_oob_ops ops;
uint8_t buf[2] = { 0, 0 };
int ret = 0, res, i = 0;
return ret;
}
+/**
+ * nand_markbad_bbm - mark a block by updating the BBM
+ * @chip: NAND chip object
+ * @ofs: offset of the block to mark bad
+ */
+int nand_markbad_bbm(struct nand_chip *chip, loff_t ofs)
+{
+ if (chip->legacy.block_markbad)
+ return chip->legacy.block_markbad(chip, ofs);
+
+ return nand_default_block_markbad(chip, ofs);
+}
+
+static int nand_isbad_bbm(struct nand_chip *chip, loff_t ofs)
+{
+ if (chip->legacy.block_bad)
+ return chip->legacy.block_bad(chip, ofs);
+
+ return nand_block_bad(chip, ofs);
+}
+
/**
* nand_block_markbad_lowlevel - mark a block bad
* @mtd: MTD device structure
*
* This function performs the generic NAND bad block marking steps (i.e., bad
* block table(s) and/or marker(s)). We only allow the hardware driver to
- * specify how to write bad block markers to OOB (chip->block_markbad).
+ * specify how to write bad block markers to OOB (chip->legacy.block_markbad).
*
* We try operations in the following order:
*
memset(&einfo, 0, sizeof(einfo));
einfo.addr = ofs;
einfo.len = 1ULL << chip->phys_erase_shift;
- nand_erase_nand(mtd, &einfo, 0);
+ nand_erase_nand(chip, &einfo, 0);
/* Write bad block marker to OOB */
nand_get_device(mtd, FL_WRITING);
- ret = chip->block_markbad(mtd, ofs);
+ ret = nand_markbad_bbm(chip, ofs);
nand_release_device(mtd);
}
/* Mark block bad in BBT */
if (chip->bbt) {
- res = nand_markbad_bbt(mtd, ofs);
+ res = nand_markbad_bbt(chip, ofs);
if (!ret)
ret = res;
}
if (!chip->bbt)
return 0;
/* Return info from the table */
- return nand_isreserved_bbt(mtd, ofs);
+ return nand_isreserved_bbt(chip, ofs);
}
/**
{
struct nand_chip *chip = mtd_to_nand(mtd);
- if (!chip->bbt)
- return chip->block_bad(mtd, ofs);
-
/* Return info from the table */
- return nand_isbad_bbt(mtd, ofs, allowbbt);
-}
-
-/**
- * panic_nand_wait_ready - [GENERIC] Wait for the ready pin after commands.
- * @mtd: MTD device structure
- * @timeo: Timeout
- *
- * Helper function for nand_wait_ready used when needing to wait in interrupt
- * context.
- */
-static void panic_nand_wait_ready(struct mtd_info *mtd, unsigned long timeo)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
- int i;
+ if (chip->bbt)
+ return nand_isbad_bbt(chip, ofs, allowbbt);
- /* Wait for the device to get ready */
- for (i = 0; i < timeo; i++) {
- if (chip->dev_ready(mtd))
- break;
- touch_softlockup_watchdog();
- mdelay(1);
- }
+ return nand_isbad_bbm(chip, ofs);
}
-/**
- * nand_wait_ready - [GENERIC] Wait for the ready pin after commands.
- * @mtd: MTD device structure
- *
- * Wait for the ready pin after a command, and warn if a timeout occurs.
- */
-void nand_wait_ready(struct mtd_info *mtd)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
- unsigned long timeo = 400;
-
- if (in_interrupt() || oops_in_progress)
- return panic_nand_wait_ready(mtd, timeo);
-
- /* Wait until command is processed or timeout occurs */
- timeo = jiffies + msecs_to_jiffies(timeo);
- do {
- if (chip->dev_ready(mtd))
- return;
- cond_resched();
- } while (time_before(jiffies, timeo));
-
- if (!chip->dev_ready(mtd))
- pr_warn_ratelimited("timeout while waiting for chip to become ready\n");
-}
-EXPORT_SYMBOL_GPL(nand_wait_ready);
-
-/**
- * nand_wait_status_ready - [GENERIC] Wait for the ready status after commands.
- * @mtd: MTD device structure
- * @timeo: Timeout in ms
- *
- * Wait for status ready (i.e. command done) or timeout.
- */
-static void nand_wait_status_ready(struct mtd_info *mtd, unsigned long timeo)
-{
- register struct nand_chip *chip = mtd_to_nand(mtd);
- int ret;
-
- timeo = jiffies + msecs_to_jiffies(timeo);
- do {
- u8 status;
-
- ret = nand_read_data_op(chip, &status, sizeof(status), true);
- if (ret)
- return;
-
- if (status & NAND_STATUS_READY)
- break;
- touch_softlockup_watchdog();
- } while (time_before(jiffies, timeo));
-};
-
/**
* nand_soft_waitrdy - Poll STATUS reg until RDY bit is set to 1
* @chip: NAND chip structure
};
EXPORT_SYMBOL_GPL(nand_soft_waitrdy);
-/**
- * nand_command - [DEFAULT] Send command to NAND device
- * @mtd: MTD device structure
- * @command: the command to be sent
- * @column: the column address for this command, -1 if none
- * @page_addr: the page address for this command, -1 if none
- *
- * Send command to NAND device. This function is used for small page devices
- * (512 Bytes per page).
- */
-static void nand_command(struct mtd_info *mtd, unsigned int command,
- int column, int page_addr)
-{
- register struct nand_chip *chip = mtd_to_nand(mtd);
- int ctrl = NAND_CTRL_CLE | NAND_CTRL_CHANGE;
-
- /* Write out the command to the device */
- if (command == NAND_CMD_SEQIN) {
- int readcmd;
-
- if (column >= mtd->writesize) {
- /* OOB area */
- column -= mtd->writesize;
- readcmd = NAND_CMD_READOOB;
- } else if (column < 256) {
- /* First 256 bytes --> READ0 */
- readcmd = NAND_CMD_READ0;
- } else {
- column -= 256;
- readcmd = NAND_CMD_READ1;
- }
- chip->cmd_ctrl(mtd, readcmd, ctrl);
- ctrl &= ~NAND_CTRL_CHANGE;
- }
- if (command != NAND_CMD_NONE)
- chip->cmd_ctrl(mtd, command, ctrl);
-
- /* Address cycle, when necessary */
- ctrl = NAND_CTRL_ALE | NAND_CTRL_CHANGE;
- /* Serially input address */
- if (column != -1) {
- /* Adjust columns for 16 bit buswidth */
- if (chip->options & NAND_BUSWIDTH_16 &&
- !nand_opcode_8bits(command))
- column >>= 1;
- chip->cmd_ctrl(mtd, column, ctrl);
- ctrl &= ~NAND_CTRL_CHANGE;
- }
- if (page_addr != -1) {
- chip->cmd_ctrl(mtd, page_addr, ctrl);
- ctrl &= ~NAND_CTRL_CHANGE;
- chip->cmd_ctrl(mtd, page_addr >> 8, ctrl);
- if (chip->options & NAND_ROW_ADDR_3)
- chip->cmd_ctrl(mtd, page_addr >> 16, ctrl);
- }
- chip->cmd_ctrl(mtd, NAND_CMD_NONE, NAND_NCE | NAND_CTRL_CHANGE);
-
- /*
- * Program and erase have their own busy handlers status and sequential
- * in needs no delay
- */
- switch (command) {
-
- case NAND_CMD_NONE:
- case NAND_CMD_PAGEPROG:
- case NAND_CMD_ERASE1:
- case NAND_CMD_ERASE2:
- case NAND_CMD_SEQIN:
- case NAND_CMD_STATUS:
- case NAND_CMD_READID:
- case NAND_CMD_SET_FEATURES:
- return;
-
- case NAND_CMD_RESET:
- if (chip->dev_ready)
- break;
- udelay(chip->chip_delay);
- chip->cmd_ctrl(mtd, NAND_CMD_STATUS,
- NAND_CTRL_CLE | NAND_CTRL_CHANGE);
- chip->cmd_ctrl(mtd,
- NAND_CMD_NONE, NAND_NCE | NAND_CTRL_CHANGE);
- /* EZ-NAND can take upto 250ms as per ONFi v4.0 */
- nand_wait_status_ready(mtd, 250);
- return;
-
- /* This applies to read commands */
- case NAND_CMD_READ0:
- /*
- * READ0 is sometimes used to exit GET STATUS mode. When this
- * is the case no address cycles are requested, and we can use
- * this information to detect that we should not wait for the
- * device to be ready.
- */
- if (column == -1 && page_addr == -1)
- return;
-
- default:
- /*
- * If we don't have access to the busy pin, we apply the given
- * command delay
- */
- if (!chip->dev_ready) {
- udelay(chip->chip_delay);
- return;
- }
- }
- /*
- * Apply this short delay always to ensure that we do wait tWB in
- * any case on any machine.
- */
- ndelay(100);
-
- nand_wait_ready(mtd);
-}
-
-static void nand_ccs_delay(struct nand_chip *chip)
-{
- /*
- * The controller already takes care of waiting for tCCS when the RNDIN
- * or RNDOUT command is sent, return directly.
- */
- if (!(chip->options & NAND_WAIT_TCCS))
- return;
-
- /*
- * Wait tCCS_min if it is correctly defined, otherwise wait 500ns
- * (which should be safe for all NANDs).
- */
- if (chip->setup_data_interface)
- ndelay(chip->data_interface.timings.sdr.tCCS_min / 1000);
- else
- ndelay(500);
-}
-
-/**
- * nand_command_lp - [DEFAULT] Send command to NAND large page device
- * @mtd: MTD device structure
- * @command: the command to be sent
- * @column: the column address for this command, -1 if none
- * @page_addr: the page address for this command, -1 if none
- *
- * Send command to NAND device. This is the version for the new large page
- * devices. We don't have the separate regions as we have in the small page
- * devices. We must emulate NAND_CMD_READOOB to keep the code compatible.
- */
-static void nand_command_lp(struct mtd_info *mtd, unsigned int command,
- int column, int page_addr)
-{
- register struct nand_chip *chip = mtd_to_nand(mtd);
-
- /* Emulate NAND_CMD_READOOB */
- if (command == NAND_CMD_READOOB) {
- column += mtd->writesize;
- command = NAND_CMD_READ0;
- }
-
- /* Command latch cycle */
- if (command != NAND_CMD_NONE)
- chip->cmd_ctrl(mtd, command,
- NAND_NCE | NAND_CLE | NAND_CTRL_CHANGE);
-
- if (column != -1 || page_addr != -1) {
- int ctrl = NAND_CTRL_CHANGE | NAND_NCE | NAND_ALE;
-
- /* Serially input address */
- if (column != -1) {
- /* Adjust columns for 16 bit buswidth */
- if (chip->options & NAND_BUSWIDTH_16 &&
- !nand_opcode_8bits(command))
- column >>= 1;
- chip->cmd_ctrl(mtd, column, ctrl);
- ctrl &= ~NAND_CTRL_CHANGE;
-
- /* Only output a single addr cycle for 8bits opcodes. */
- if (!nand_opcode_8bits(command))
- chip->cmd_ctrl(mtd, column >> 8, ctrl);
- }
- if (page_addr != -1) {
- chip->cmd_ctrl(mtd, page_addr, ctrl);
- chip->cmd_ctrl(mtd, page_addr >> 8,
- NAND_NCE | NAND_ALE);
- if (chip->options & NAND_ROW_ADDR_3)
- chip->cmd_ctrl(mtd, page_addr >> 16,
- NAND_NCE | NAND_ALE);
- }
- }
- chip->cmd_ctrl(mtd, NAND_CMD_NONE, NAND_NCE | NAND_CTRL_CHANGE);
-
- /*
- * Program and erase have their own busy handlers status, sequential
- * in and status need no delay.
- */
- switch (command) {
-
- case NAND_CMD_NONE:
- case NAND_CMD_CACHEDPROG:
- case NAND_CMD_PAGEPROG:
- case NAND_CMD_ERASE1:
- case NAND_CMD_ERASE2:
- case NAND_CMD_SEQIN:
- case NAND_CMD_STATUS:
- case NAND_CMD_READID:
- case NAND_CMD_SET_FEATURES:
- return;
-
- case NAND_CMD_RNDIN:
- nand_ccs_delay(chip);
- return;
-
- case NAND_CMD_RESET:
- if (chip->dev_ready)
- break;
- udelay(chip->chip_delay);
- chip->cmd_ctrl(mtd, NAND_CMD_STATUS,
- NAND_NCE | NAND_CLE | NAND_CTRL_CHANGE);
- chip->cmd_ctrl(mtd, NAND_CMD_NONE,
- NAND_NCE | NAND_CTRL_CHANGE);
- /* EZ-NAND can take upto 250ms as per ONFi v4.0 */
- nand_wait_status_ready(mtd, 250);
- return;
-
- case NAND_CMD_RNDOUT:
- /* No ready / busy check necessary */
- chip->cmd_ctrl(mtd, NAND_CMD_RNDOUTSTART,
- NAND_NCE | NAND_CLE | NAND_CTRL_CHANGE);
- chip->cmd_ctrl(mtd, NAND_CMD_NONE,
- NAND_NCE | NAND_CTRL_CHANGE);
-
- nand_ccs_delay(chip);
- return;
-
- case NAND_CMD_READ0:
- /*
- * READ0 is sometimes used to exit GET STATUS mode. When this
- * is the case no address cycles are requested, and we can use
- * this information to detect that READSTART should not be
- * issued.
- */
- if (column == -1 && page_addr == -1)
- return;
-
- chip->cmd_ctrl(mtd, NAND_CMD_READSTART,
- NAND_NCE | NAND_CLE | NAND_CTRL_CHANGE);
- chip->cmd_ctrl(mtd, NAND_CMD_NONE,
- NAND_NCE | NAND_CTRL_CHANGE);
-
- /* This applies to read commands */
- default:
- /*
- * If we don't have access to the busy pin, we apply the given
- * command delay.
- */
- if (!chip->dev_ready) {
- udelay(chip->chip_delay);
- return;
- }
- }
-
- /*
- * Apply this short delay always to ensure that we do wait tWB in
- * any case on any machine.
- */
- ndelay(100);
-
- nand_wait_ready(mtd);
-}
-
/**
* panic_nand_get_device - [GENERIC] Get chip for selected access
* @chip: the nand chip descriptor
* we are in interrupt context. May happen when in panic and trying to write
* an oops through mtdoops.
*/
-static void panic_nand_wait(struct mtd_info *mtd, struct nand_chip *chip,
- unsigned long timeo)
+void panic_nand_wait(struct nand_chip *chip, unsigned long timeo)
{
int i;
for (i = 0; i < timeo; i++) {
- if (chip->dev_ready) {
- if (chip->dev_ready(mtd))
+ if (chip->legacy.dev_ready) {
+ if (chip->legacy.dev_ready(chip))
break;
} else {
int ret;
}
}
-/**
- * nand_wait - [DEFAULT] wait until the command is done
- * @mtd: MTD device structure
- * @chip: NAND chip structure
- *
- * Wait for command done. This applies to erase and program only.
- */
-static int nand_wait(struct mtd_info *mtd, struct nand_chip *chip)
-{
-
- unsigned long timeo = 400;
- u8 status;
- int ret;
-
- /*
- * Apply this short delay always to ensure that we do wait tWB in any
- * case on any machine.
- */
- ndelay(100);
-
- ret = nand_status_op(chip, NULL);
- if (ret)
- return ret;
-
- if (in_interrupt() || oops_in_progress)
- panic_nand_wait(mtd, chip, timeo);
- else {
- timeo = jiffies + msecs_to_jiffies(timeo);
- do {
- if (chip->dev_ready) {
- if (chip->dev_ready(mtd))
- break;
- } else {
- ret = nand_read_data_op(chip, &status,
- sizeof(status), true);
- if (ret)
- return ret;
-
- if (status & NAND_STATUS_READY)
- break;
- }
- cond_resched();
- } while (time_before(jiffies, timeo));
- }
-
- ret = nand_read_data_op(chip, &status, sizeof(status), true);
- if (ret)
- return ret;
-
- /* This can happen if in case of timeout or buggy dev_ready */
- WARN_ON(!(status & NAND_STATUS_READY));
- return status;
-}
-
static bool nand_supports_get_features(struct nand_chip *chip, int addr)
{
return (chip->parameters.supports_set_get_features &&
test_bit(addr, chip->parameters.set_feature_list));
}
-/**
- * nand_get_features - wrapper to perform a GET_FEATURE
- * @chip: NAND chip info structure
- * @addr: feature address
- * @subfeature_param: the subfeature parameters, a four bytes array
- *
- * Returns 0 for success, a negative error otherwise. Returns -ENOTSUPP if the
- * operation cannot be handled.
- */
-int nand_get_features(struct nand_chip *chip, int addr,
- u8 *subfeature_param)
-{
- struct mtd_info *mtd = nand_to_mtd(chip);
-
- if (!nand_supports_get_features(chip, addr))
- return -ENOTSUPP;
-
- return chip->get_features(mtd, chip, addr, subfeature_param);
-}
-EXPORT_SYMBOL_GPL(nand_get_features);
-
-/**
- * nand_set_features - wrapper to perform a SET_FEATURE
- * @chip: NAND chip info structure
- * @addr: feature address
- * @subfeature_param: the subfeature parameters, a four bytes array
- *
- * Returns 0 for success, a negative error otherwise. Returns -ENOTSUPP if the
- * operation cannot be handled.
- */
-int nand_set_features(struct nand_chip *chip, int addr,
- u8 *subfeature_param)
-{
- struct mtd_info *mtd = nand_to_mtd(chip);
-
- if (!nand_supports_set_features(chip, addr))
- return -ENOTSUPP;
-
- return chip->set_features(mtd, chip, addr, subfeature_param);
-}
-EXPORT_SYMBOL_GPL(nand_set_features);
-
/**
* nand_reset_data_interface - Reset data interface and timings
* @chip: The NAND chip
*/
static int nand_reset_data_interface(struct nand_chip *chip, int chipnr)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
int ret;
if (!chip->setup_data_interface)
*/
onfi_fill_data_interface(chip, NAND_SDR_IFACE, 0);
- ret = chip->setup_data_interface(mtd, chipnr, &chip->data_interface);
+ ret = chip->setup_data_interface(chip, chipnr, &chip->data_interface);
if (ret)
pr_err("Failed to configure data interface to SDR timing mode 0\n");
*/
static int nand_setup_data_interface(struct nand_chip *chip, int chipnr)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
u8 tmode_param[ONFI_SUBFEATURE_PARAM_LEN] = {
chip->onfi_timing_mode_default,
};
/* Change the mode on the chip side (if supported by the NAND chip) */
if (nand_supports_set_features(chip, ONFI_FEATURE_ADDR_TIMING_MODE)) {
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
ret = nand_set_features(chip, ONFI_FEATURE_ADDR_TIMING_MODE,
tmode_param);
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
if (ret)
return ret;
}
/* Change the mode on the controller side */
- ret = chip->setup_data_interface(mtd, chipnr, &chip->data_interface);
+ ret = chip->setup_data_interface(chip, chipnr, &chip->data_interface);
if (ret)
return ret;
return 0;
memset(tmode_param, 0, ONFI_SUBFEATURE_PARAM_LEN);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
ret = nand_get_features(chip, ONFI_FEATURE_ADDR_TIMING_MODE,
tmode_param);
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
if (ret)
goto err_reset_chip;
* timing mode.
*/
nand_reset_data_interface(chip, chipnr);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
nand_reset_op(chip);
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
return ret;
}
*/
static int nand_init_data_interface(struct nand_chip *chip)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
int modes, mode, ret;
if (!chip->setup_data_interface)
* if the NAND does not support ONFI, fallback to the default ONFI
* timing mode.
*/
- modes = onfi_get_async_timing_mode(chip);
- if (modes == ONFI_TIMING_MODE_UNKNOWN) {
+ if (chip->parameters.onfi) {
+ modes = chip->parameters.onfi->async_timing_mode;
+ } else {
if (!chip->onfi_timing_mode_default)
return 0;
modes = GENMASK(chip->onfi_timing_mode_default, 0);
}
-
for (mode = fls(modes) - 1; mode >= 0; mode--) {
ret = onfi_fill_data_interface(chip, NAND_SDR_IFACE, mode);
if (ret)
* Pass NAND_DATA_IFACE_CHECK_ONLY to only check if the
* controller supports the requested timings.
*/
- ret = chip->setup_data_interface(mtd,
+ ret = chip->setup_data_interface(chip,
NAND_DATA_IFACE_CHECK_ONLY,
&chip->data_interface);
if (!ret) {
buf, len);
}
- chip->cmdfunc(mtd, NAND_CMD_READ0, offset_in_page, page);
+ chip->legacy.cmdfunc(chip, NAND_CMD_READ0, offset_in_page, page);
if (len)
- chip->read_buf(mtd, buf, len);
+ chip->legacy.read_buf(chip, buf, len);
return 0;
}
*
* Returns 0 on success, a negative error code otherwise.
*/
-static int nand_read_param_page_op(struct nand_chip *chip, u8 page, void *buf,
- unsigned int len)
+int nand_read_param_page_op(struct nand_chip *chip, u8 page, void *buf,
+ unsigned int len)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
unsigned int i;
u8 *p = buf;
return nand_exec_op(chip, &op);
}
- chip->cmdfunc(mtd, NAND_CMD_PARAM, page, -1);
+ chip->legacy.cmdfunc(chip, NAND_CMD_PARAM, page, -1);
for (i = 0; i < len; i++)
- p[i] = chip->read_byte(mtd);
+ p[i] = chip->legacy.read_byte(chip);
return 0;
}
return nand_exec_op(chip, &op);
}
- chip->cmdfunc(mtd, NAND_CMD_RNDOUT, offset_in_page, -1);
+ chip->legacy.cmdfunc(chip, NAND_CMD_RNDOUT, offset_in_page, -1);
if (len)
- chip->read_buf(mtd, buf, len);
+ chip->legacy.read_buf(chip, buf, len);
return 0;
}
mtd->writesize + offset_in_oob,
buf, len);
- chip->cmdfunc(mtd, NAND_CMD_READOOB, offset_in_oob, page);
+ chip->legacy.cmdfunc(chip, NAND_CMD_READOOB, offset_in_oob, page);
if (len)
- chip->read_buf(mtd, buf, len);
+ chip->legacy.read_buf(chip, buf, len);
return 0;
}
return nand_exec_prog_page_op(chip, page, offset_in_page, buf,
len, false);
- chip->cmdfunc(mtd, NAND_CMD_SEQIN, offset_in_page, page);
+ chip->legacy.cmdfunc(chip, NAND_CMD_SEQIN, offset_in_page, page);
if (buf)
- chip->write_buf(mtd, buf, len);
+ chip->legacy.write_buf(chip, buf, len);
return 0;
}
*/
int nand_prog_page_end_op(struct nand_chip *chip)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
int ret;
u8 status;
if (ret)
return ret;
} else {
- chip->cmdfunc(mtd, NAND_CMD_PAGEPROG, -1, -1);
- ret = chip->waitfunc(mtd, chip);
+ chip->legacy.cmdfunc(chip, NAND_CMD_PAGEPROG, -1, -1);
+ ret = chip->legacy.waitfunc(chip);
if (ret < 0)
return ret;
status = nand_exec_prog_page_op(chip, page, offset_in_page, buf,
len, true);
} else {
- chip->cmdfunc(mtd, NAND_CMD_SEQIN, offset_in_page, page);
- chip->write_buf(mtd, buf, len);
- chip->cmdfunc(mtd, NAND_CMD_PAGEPROG, -1, -1);
- status = chip->waitfunc(mtd, chip);
+ chip->legacy.cmdfunc(chip, NAND_CMD_SEQIN, offset_in_page,
+ page);
+ chip->legacy.write_buf(chip, buf, len);
+ chip->legacy.cmdfunc(chip, NAND_CMD_PAGEPROG, -1, -1);
+ status = chip->legacy.waitfunc(chip);
}
if (status & NAND_STATUS_FAIL)
return nand_exec_op(chip, &op);
}
- chip->cmdfunc(mtd, NAND_CMD_RNDIN, offset_in_page, -1);
+ chip->legacy.cmdfunc(chip, NAND_CMD_RNDIN, offset_in_page, -1);
if (len)
- chip->write_buf(mtd, buf, len);
+ chip->legacy.write_buf(chip, buf, len);
return 0;
}
int nand_readid_op(struct nand_chip *chip, u8 addr, void *buf,
unsigned int len)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
unsigned int i;
u8 *id = buf;
return nand_exec_op(chip, &op);
}
- chip->cmdfunc(mtd, NAND_CMD_READID, addr, -1);
+ chip->legacy.cmdfunc(chip, NAND_CMD_READID, addr, -1);
for (i = 0; i < len; i++)
- id[i] = chip->read_byte(mtd);
+ id[i] = chip->legacy.read_byte(chip);
return 0;
}
*/
int nand_status_op(struct nand_chip *chip, u8 *status)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
-
if (chip->exec_op) {
const struct nand_sdr_timings *sdr =
nand_get_sdr_timings(&chip->data_interface);
return nand_exec_op(chip, &op);
}
- chip->cmdfunc(mtd, NAND_CMD_STATUS, -1, -1);
+ chip->legacy.cmdfunc(chip, NAND_CMD_STATUS, -1, -1);
if (status)
- *status = chip->read_byte(mtd);
+ *status = chip->legacy.read_byte(chip);
return 0;
}
*/
int nand_exit_status_op(struct nand_chip *chip)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
-
if (chip->exec_op) {
struct nand_op_instr instrs[] = {
NAND_OP_CMD(NAND_CMD_READ0, 0),
return nand_exec_op(chip, &op);
}
- chip->cmdfunc(mtd, NAND_CMD_READ0, -1, -1);
+ chip->legacy.cmdfunc(chip, NAND_CMD_READ0, -1, -1);
return 0;
}
-EXPORT_SYMBOL_GPL(nand_exit_status_op);
/**
* nand_erase_op - Do an erase operation
*/
int nand_erase_op(struct nand_chip *chip, unsigned int eraseblock)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
unsigned int page = eraseblock <<
(chip->phys_erase_shift - chip->page_shift);
int ret;
if (ret)
return ret;
} else {
- chip->cmdfunc(mtd, NAND_CMD_ERASE1, -1, page);
- chip->cmdfunc(mtd, NAND_CMD_ERASE2, -1, -1);
+ chip->legacy.cmdfunc(chip, NAND_CMD_ERASE1, -1, page);
+ chip->legacy.cmdfunc(chip, NAND_CMD_ERASE2, -1, -1);
- ret = chip->waitfunc(mtd, chip);
+ ret = chip->legacy.waitfunc(chip);
if (ret < 0)
return ret;
static int nand_set_features_op(struct nand_chip *chip, u8 feature,
const void *data)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
const u8 *params = data;
int i, ret;
return nand_exec_op(chip, &op);
}
- chip->cmdfunc(mtd, NAND_CMD_SET_FEATURES, feature, -1);
+ chip->legacy.cmdfunc(chip, NAND_CMD_SET_FEATURES, feature, -1);
for (i = 0; i < ONFI_SUBFEATURE_PARAM_LEN; ++i)
- chip->write_byte(mtd, params[i]);
+ chip->legacy.write_byte(chip, params[i]);
- ret = chip->waitfunc(mtd, chip);
+ ret = chip->legacy.waitfunc(chip);
if (ret < 0)
return ret;
static int nand_get_features_op(struct nand_chip *chip, u8 feature,
void *data)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
u8 *params = data;
int i;
return nand_exec_op(chip, &op);
}
- chip->cmdfunc(mtd, NAND_CMD_GET_FEATURES, feature, -1);
+ chip->legacy.cmdfunc(chip, NAND_CMD_GET_FEATURES, feature, -1);
for (i = 0; i < ONFI_SUBFEATURE_PARAM_LEN; ++i)
- params[i] = chip->read_byte(mtd);
+ params[i] = chip->legacy.read_byte(chip);
+
+ return 0;
+}
+
+static int nand_wait_rdy_op(struct nand_chip *chip, unsigned int timeout_ms,
+ unsigned int delay_ns)
+{
+ if (chip->exec_op) {
+ struct nand_op_instr instrs[] = {
+ NAND_OP_WAIT_RDY(PSEC_TO_MSEC(timeout_ms),
+ PSEC_TO_NSEC(delay_ns)),
+ };
+ struct nand_operation op = NAND_OPERATION(instrs);
+
+ return nand_exec_op(chip, &op);
+ }
+
+ /* Apply delay or wait for ready/busy pin */
+ if (!chip->legacy.dev_ready)
+ udelay(chip->legacy.chip_delay);
+ else
+ nand_wait_ready(chip);
return 0;
}
*/
int nand_reset_op(struct nand_chip *chip)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
-
if (chip->exec_op) {
const struct nand_sdr_timings *sdr =
nand_get_sdr_timings(&chip->data_interface);
return nand_exec_op(chip, &op);
}
- chip->cmdfunc(mtd, NAND_CMD_RESET, -1, -1);
+ chip->legacy.cmdfunc(chip, NAND_CMD_RESET, -1, -1);
return 0;
}
int nand_read_data_op(struct nand_chip *chip, void *buf, unsigned int len,
bool force_8bit)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
-
if (!len || !buf)
return -EINVAL;
unsigned int i;
for (i = 0; i < len; i++)
- p[i] = chip->read_byte(mtd);
+ p[i] = chip->legacy.read_byte(chip);
} else {
- chip->read_buf(mtd, buf, len);
+ chip->legacy.read_buf(chip, buf, len);
}
return 0;
int nand_write_data_op(struct nand_chip *chip, const void *buf,
unsigned int len, bool force_8bit)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
-
if (!len || !buf)
return -EINVAL;
unsigned int i;
for (i = 0; i < len; i++)
- chip->write_byte(mtd, p[i]);
+ chip->legacy.write_byte(chip, p[i]);
} else {
- chip->write_buf(mtd, buf, len);
+ chip->legacy.write_buf(chip, buf, len);
}
return 0;
*/
int nand_reset(struct nand_chip *chip, int chipnr)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
struct nand_data_interface saved_data_intf = chip->data_interface;
int ret;
* The CS line has to be released before we can apply the new NAND
* interface settings, hence this weird ->select_chip() dance.
*/
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
ret = nand_reset_op(chip);
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
if (ret)
return ret;
}
EXPORT_SYMBOL_GPL(nand_reset);
+/**
+ * nand_get_features - wrapper to perform a GET_FEATURE
+ * @chip: NAND chip info structure
+ * @addr: feature address
+ * @subfeature_param: the subfeature parameters, a four bytes array
+ *
+ * Returns 0 for success, a negative error otherwise. Returns -ENOTSUPP if the
+ * operation cannot be handled.
+ */
+int nand_get_features(struct nand_chip *chip, int addr,
+ u8 *subfeature_param)
+{
+ if (!nand_supports_get_features(chip, addr))
+ return -ENOTSUPP;
+
+ if (chip->legacy.get_features)
+ return chip->legacy.get_features(chip, addr, subfeature_param);
+
+ return nand_get_features_op(chip, addr, subfeature_param);
+}
+
+/**
+ * nand_set_features - wrapper to perform a SET_FEATURE
+ * @chip: NAND chip info structure
+ * @addr: feature address
+ * @subfeature_param: the subfeature parameters, a four bytes array
+ *
+ * Returns 0 for success, a negative error otherwise. Returns -ENOTSUPP if the
+ * operation cannot be handled.
+ */
+int nand_set_features(struct nand_chip *chip, int addr,
+ u8 *subfeature_param)
+{
+ if (!nand_supports_set_features(chip, addr))
+ return -ENOTSUPP;
+
+ if (chip->legacy.set_features)
+ return chip->legacy.set_features(chip, addr, subfeature_param);
+
+ return nand_set_features_op(chip, addr, subfeature_param);
+}
+
/**
* nand_check_erased_buf - check if a buffer contains (almost) only 0xff data
* @buf: buffer to test
/**
* nand_read_page_raw_notsupp - dummy read raw page function
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: buffer to store read data
* @oob_required: caller requires OOB data read to chip->oob_poi
*
* Returns -ENOTSUPP unconditionally.
*/
-int nand_read_page_raw_notsupp(struct mtd_info *mtd, struct nand_chip *chip,
- u8 *buf, int oob_required, int page)
+int nand_read_page_raw_notsupp(struct nand_chip *chip, u8 *buf,
+ int oob_required, int page)
{
return -ENOTSUPP;
}
-EXPORT_SYMBOL(nand_read_page_raw_notsupp);
/**
* nand_read_page_raw - [INTERN] read raw page data without ecc
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: buffer to store read data
* @oob_required: caller requires OOB data read to chip->oob_poi
*
* Not for syndrome calculating ECC controllers, which use a special oob layout.
*/
-int nand_read_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+int nand_read_page_raw(struct nand_chip *chip, uint8_t *buf, int oob_required,
+ int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int ret;
ret = nand_read_page_op(chip, page, 0, buf, mtd->writesize);
/**
* nand_read_page_raw_syndrome - [INTERN] read raw page data without ecc
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: buffer to store read data
* @oob_required: caller requires OOB data read to chip->oob_poi
*
* We need a special oob layout and handling even when OOB isn't used.
*/
-static int nand_read_page_raw_syndrome(struct mtd_info *mtd,
- struct nand_chip *chip, uint8_t *buf,
+static int nand_read_page_raw_syndrome(struct nand_chip *chip, uint8_t *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int eccsize = chip->ecc.size;
int eccbytes = chip->ecc.bytes;
uint8_t *oob = chip->oob_poi;
/**
* nand_read_page_swecc - [REPLACEABLE] software ECC based page read function
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: buffer to store read data
* @oob_required: caller requires OOB data read to chip->oob_poi
* @page: page number to read
*/
-static int nand_read_page_swecc(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int nand_read_page_swecc(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int i, eccsize = chip->ecc.size, ret;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
uint8_t *ecc_code = chip->ecc.code_buf;
unsigned int max_bitflips = 0;
- chip->ecc.read_page_raw(mtd, chip, buf, 1, page);
+ chip->ecc.read_page_raw(chip, buf, 1, page);
for (i = 0; eccsteps; eccsteps--, i += eccbytes, p += eccsize)
- chip->ecc.calculate(mtd, p, &ecc_calc[i]);
+ chip->ecc.calculate(chip, p, &ecc_calc[i]);
ret = mtd_ooblayout_get_eccbytes(mtd, ecc_code, chip->oob_poi, 0,
chip->ecc.total);
for (i = 0 ; eccsteps; eccsteps--, i += eccbytes, p += eccsize) {
int stat;
- stat = chip->ecc.correct(mtd, p, &ecc_code[i], &ecc_calc[i]);
+ stat = chip->ecc.correct(chip, p, &ecc_code[i], &ecc_calc[i]);
if (stat < 0) {
mtd->ecc_stats.failed++;
} else {
/**
* nand_read_subpage - [REPLACEABLE] ECC based sub-page read function
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @data_offs: offset of requested data within the page
* @readlen: data length
* @bufpoi: buffer to store read data
* @page: page number to read
*/
-static int nand_read_subpage(struct mtd_info *mtd, struct nand_chip *chip,
- uint32_t data_offs, uint32_t readlen, uint8_t *bufpoi,
- int page)
+static int nand_read_subpage(struct nand_chip *chip, uint32_t data_offs,
+ uint32_t readlen, uint8_t *bufpoi, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int start_step, end_step, num_steps, ret;
uint8_t *p;
int data_col_addr, i, gaps = 0;
/* Calculate ECC */
for (i = 0; i < eccfrag_len ; i += chip->ecc.bytes, p += chip->ecc.size)
- chip->ecc.calculate(mtd, p, &chip->ecc.calc_buf[i]);
+ chip->ecc.calculate(chip, p, &chip->ecc.calc_buf[i]);
/*
* The performance is faster if we position offsets according to
for (i = 0; i < eccfrag_len ; i += chip->ecc.bytes, p += chip->ecc.size) {
int stat;
- stat = chip->ecc.correct(mtd, p, &chip->ecc.code_buf[i],
+ stat = chip->ecc.correct(chip, p, &chip->ecc.code_buf[i],
&chip->ecc.calc_buf[i]);
if (stat == -EBADMSG &&
(chip->ecc.options & NAND_ECC_GENERIC_ERASED_CHECK)) {
/**
* nand_read_page_hwecc - [REPLACEABLE] hardware ECC based page read function
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: buffer to store read data
* @oob_required: caller requires OOB data read to chip->oob_poi
*
* Not for syndrome calculating ECC controllers which need a special oob layout.
*/
-static int nand_read_page_hwecc(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int nand_read_page_hwecc(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int i, eccsize = chip->ecc.size, ret;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
return ret;
for (i = 0; eccsteps; eccsteps--, i += eccbytes, p += eccsize) {
- chip->ecc.hwctl(mtd, NAND_ECC_READ);
+ chip->ecc.hwctl(chip, NAND_ECC_READ);
ret = nand_read_data_op(chip, p, eccsize, false);
if (ret)
return ret;
- chip->ecc.calculate(mtd, p, &ecc_calc[i]);
+ chip->ecc.calculate(chip, p, &ecc_calc[i]);
}
ret = nand_read_data_op(chip, chip->oob_poi, mtd->oobsize, false);
for (i = 0 ; eccsteps; eccsteps--, i += eccbytes, p += eccsize) {
int stat;
- stat = chip->ecc.correct(mtd, p, &ecc_code[i], &ecc_calc[i]);
+ stat = chip->ecc.correct(chip, p, &ecc_code[i], &ecc_calc[i]);
if (stat == -EBADMSG &&
(chip->ecc.options & NAND_ECC_GENERIC_ERASED_CHECK)) {
/* check for empty pages with bitflips */
/**
* nand_read_page_hwecc_oob_first - [REPLACEABLE] hw ecc, read oob first
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: buffer to store read data
* @oob_required: caller requires OOB data read to chip->oob_poi
* multiple ECC steps, follows the "infix ECC" scheme and reads/writes ECC from
* the data area, by overwriting the NAND manufacturer bad block markings.
*/
-static int nand_read_page_hwecc_oob_first(struct mtd_info *mtd,
- struct nand_chip *chip, uint8_t *buf, int oob_required, int page)
+static int nand_read_page_hwecc_oob_first(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int i, eccsize = chip->ecc.size, ret;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
for (i = 0; eccsteps; eccsteps--, i += eccbytes, p += eccsize) {
int stat;
- chip->ecc.hwctl(mtd, NAND_ECC_READ);
+ chip->ecc.hwctl(chip, NAND_ECC_READ);
ret = nand_read_data_op(chip, p, eccsize, false);
if (ret)
return ret;
- chip->ecc.calculate(mtd, p, &ecc_calc[i]);
+ chip->ecc.calculate(chip, p, &ecc_calc[i]);
- stat = chip->ecc.correct(mtd, p, &ecc_code[i], NULL);
+ stat = chip->ecc.correct(chip, p, &ecc_code[i], NULL);
if (stat == -EBADMSG &&
(chip->ecc.options & NAND_ECC_GENERIC_ERASED_CHECK)) {
/* check for empty pages with bitflips */
/**
* nand_read_page_syndrome - [REPLACEABLE] hardware ECC syndrome based page read
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: buffer to store read data
* @oob_required: caller requires OOB data read to chip->oob_poi
* The hw generator calculates the error syndrome automatically. Therefore we
* need a special oob layout and handling.
*/
-static int nand_read_page_syndrome(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int nand_read_page_syndrome(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int ret, i, eccsize = chip->ecc.size;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
for (i = 0; eccsteps; eccsteps--, i += eccbytes, p += eccsize) {
int stat;
- chip->ecc.hwctl(mtd, NAND_ECC_READ);
+ chip->ecc.hwctl(chip, NAND_ECC_READ);
ret = nand_read_data_op(chip, p, eccsize, false);
if (ret)
oob += chip->ecc.prepad;
}
- chip->ecc.hwctl(mtd, NAND_ECC_READSYN);
+ chip->ecc.hwctl(chip, NAND_ECC_READSYN);
ret = nand_read_data_op(chip, oob, eccbytes, false);
if (ret)
return ret;
- stat = chip->ecc.correct(mtd, p, oob, NULL);
+ stat = chip->ecc.correct(chip, p, oob, NULL);
oob += eccbytes;
/**
* nand_setup_read_retry - [INTERN] Set the READ RETRY mode
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @retry_mode: the retry mode to use
*
* Some vendors supply a special command to shift the Vt threshold, to be used
* when there are too many bitflips in a page (i.e., ECC error). After setting
* a new threshold, the host should retry reading the page.
*/
-static int nand_setup_read_retry(struct mtd_info *mtd, int retry_mode)
+static int nand_setup_read_retry(struct nand_chip *chip, int retry_mode)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
pr_debug("setting READ RETRY mode %d\n", retry_mode);
if (retry_mode >= chip->read_retries)
if (!chip->setup_read_retry)
return -EOPNOTSUPP;
- return chip->setup_read_retry(mtd, retry_mode);
+ return chip->setup_read_retry(chip, retry_mode);
+}
+
+static void nand_wait_readrdy(struct nand_chip *chip)
+{
+ const struct nand_sdr_timings *sdr;
+
+ if (!(chip->options & NAND_NEED_READRDY))
+ return;
+
+ sdr = nand_get_sdr_timings(&chip->data_interface);
+ WARN_ON(nand_wait_rdy_op(chip, PSEC_TO_MSEC(sdr->tR_max), 0));
}
/**
bool ecc_fail = false;
chipnr = (int)(from >> chip->chip_shift);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
realpage = (int)(from >> chip->page_shift);
page = realpage & chip->pagemask;
* the read methods return max bitflips per ecc step.
*/
if (unlikely(ops->mode == MTD_OPS_RAW))
- ret = chip->ecc.read_page_raw(mtd, chip, bufpoi,
+ ret = chip->ecc.read_page_raw(chip, bufpoi,
oob_required,
page);
else if (!aligned && NAND_HAS_SUBPAGE_READ(chip) &&
!oob)
- ret = chip->ecc.read_subpage(mtd, chip,
- col, bytes, bufpoi,
- page);
+ ret = chip->ecc.read_subpage(chip, col, bytes,
+ bufpoi, page);
else
- ret = chip->ecc.read_page(mtd, chip, bufpoi,
+ ret = chip->ecc.read_page(chip, bufpoi,
oob_required, page);
if (ret < 0) {
if (use_bufpoi)
}
}
- if (chip->options & NAND_NEED_READRDY) {
- /* Apply delay or wait for ready/busy pin */
- if (!chip->dev_ready)
- udelay(chip->chip_delay);
- else
- nand_wait_ready(mtd);
- }
+ nand_wait_readrdy(chip);
if (mtd->ecc_stats.failed - ecc_failures) {
if (retry_mode + 1 < chip->read_retries) {
retry_mode++;
- ret = nand_setup_read_retry(mtd,
+ ret = nand_setup_read_retry(chip,
retry_mode);
if (ret < 0)
break;
/* Reset to retry mode 0 */
if (retry_mode) {
- ret = nand_setup_read_retry(mtd, 0);
+ ret = nand_setup_read_retry(chip, 0);
if (ret < 0)
break;
retry_mode = 0;
/* Check, if we cross a chip boundary */
if (!page) {
chipnr++;
- chip->select_chip(mtd, -1);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, -1);
+ chip->select_chip(chip, chipnr);
}
}
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
ops->retlen = ops->len - (size_t) readlen;
if (oob)
/**
* nand_read_oob_std - [REPLACEABLE] the most common OOB data read function
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @page: page number to read
*/
-int nand_read_oob_std(struct mtd_info *mtd, struct nand_chip *chip, int page)
+int nand_read_oob_std(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
return nand_read_oob_op(chip, page, 0, chip->oob_poi, mtd->oobsize);
}
EXPORT_SYMBOL(nand_read_oob_std);
/**
* nand_read_oob_syndrome - [REPLACEABLE] OOB data read function for HW ECC
* with syndromes
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @page: page number to read
*/
-int nand_read_oob_syndrome(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int nand_read_oob_syndrome(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int length = mtd->oobsize;
int chunk = chip->ecc.bytes + chip->ecc.prepad + chip->ecc.postpad;
int eccsize = chip->ecc.size;
return 0;
}
-EXPORT_SYMBOL(nand_read_oob_syndrome);
/**
* nand_write_oob_std - [REPLACEABLE] the most common OOB data write function
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @page: page number to write
*/
-int nand_write_oob_std(struct mtd_info *mtd, struct nand_chip *chip, int page)
+int nand_write_oob_std(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
return nand_prog_page_op(chip, page, mtd->writesize, chip->oob_poi,
mtd->oobsize);
}
/**
* nand_write_oob_syndrome - [REPLACEABLE] OOB data write function for HW ECC
* with syndrome - only for large page flash
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @page: page number to write
*/
-int nand_write_oob_syndrome(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int nand_write_oob_syndrome(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int chunk = chip->ecc.bytes + chip->ecc.prepad + chip->ecc.postpad;
int eccsize = chip->ecc.size, length = mtd->oobsize;
int ret, i, len, pos, sndcmd = 0, steps = chip->ecc.steps;
return nand_prog_page_end_op(chip);
}
-EXPORT_SYMBOL(nand_write_oob_syndrome);
/**
* nand_do_read_oob - [INTERN] NAND read out-of-band
len = mtd_oobavail(mtd, ops);
chipnr = (int)(from >> chip->chip_shift);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
/* Shift to get page */
realpage = (int)(from >> chip->page_shift);
while (1) {
if (ops->mode == MTD_OPS_RAW)
- ret = chip->ecc.read_oob_raw(mtd, chip, page);
+ ret = chip->ecc.read_oob_raw(chip, page);
else
- ret = chip->ecc.read_oob(mtd, chip, page);
+ ret = chip->ecc.read_oob(chip, page);
if (ret < 0)
break;
len = min(len, readlen);
buf = nand_transfer_oob(mtd, buf, ops, len);
- if (chip->options & NAND_NEED_READRDY) {
- /* Apply delay or wait for ready/busy pin */
- if (!chip->dev_ready)
- udelay(chip->chip_delay);
- else
- nand_wait_ready(mtd);
- }
+ nand_wait_readrdy(chip);
max_bitflips = max_t(unsigned int, max_bitflips, ret);
/* Check, if we cross a chip boundary */
if (!page) {
chipnr++;
- chip->select_chip(mtd, -1);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, -1);
+ chip->select_chip(chip, chipnr);
}
}
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
ops->oobretlen = ops->ooblen - readlen;
/**
* nand_write_page_raw_notsupp - dummy raw page write function
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: data buffer
* @oob_required: must write chip->oob_poi to OOB
*
* Returns -ENOTSUPP unconditionally.
*/
-int nand_write_page_raw_notsupp(struct mtd_info *mtd, struct nand_chip *chip,
- const u8 *buf, int oob_required, int page)
+int nand_write_page_raw_notsupp(struct nand_chip *chip, const u8 *buf,
+ int oob_required, int page)
{
return -ENOTSUPP;
}
-EXPORT_SYMBOL(nand_write_page_raw_notsupp);
/**
* nand_write_page_raw - [INTERN] raw page write function
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: data buffer
* @oob_required: must write chip->oob_poi to OOB
*
* Not for syndrome calculating ECC controllers, which use a special oob layout.
*/
-int nand_write_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page)
+int nand_write_page_raw(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int ret;
ret = nand_prog_page_begin_op(chip, page, 0, buf, mtd->writesize);
/**
* nand_write_page_raw_syndrome - [INTERN] raw page write function
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: data buffer
* @oob_required: must write chip->oob_poi to OOB
*
* We need a special oob layout and handling even when ECC isn't checked.
*/
-static int nand_write_page_raw_syndrome(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int nand_write_page_raw_syndrome(struct nand_chip *chip,
const uint8_t *buf, int oob_required,
int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int eccsize = chip->ecc.size;
int eccbytes = chip->ecc.bytes;
uint8_t *oob = chip->oob_poi;
}
/**
* nand_write_page_swecc - [REPLACEABLE] software ECC based page write function
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: data buffer
* @oob_required: must write chip->oob_poi to OOB
* @page: page number to write
*/
-static int nand_write_page_swecc(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required,
- int page)
+static int nand_write_page_swecc(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int i, eccsize = chip->ecc.size, ret;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
/* Software ECC calculation */
for (i = 0; eccsteps; eccsteps--, i += eccbytes, p += eccsize)
- chip->ecc.calculate(mtd, p, &ecc_calc[i]);
+ chip->ecc.calculate(chip, p, &ecc_calc[i]);
ret = mtd_ooblayout_set_eccbytes(mtd, ecc_calc, chip->oob_poi, 0,
chip->ecc.total);
if (ret)
return ret;
- return chip->ecc.write_page_raw(mtd, chip, buf, 1, page);
+ return chip->ecc.write_page_raw(chip, buf, 1, page);
}
/**
* nand_write_page_hwecc - [REPLACEABLE] hardware ECC based page write function
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: data buffer
* @oob_required: must write chip->oob_poi to OOB
* @page: page number to write
*/
-static int nand_write_page_hwecc(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required,
- int page)
+static int nand_write_page_hwecc(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int i, eccsize = chip->ecc.size, ret;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
return ret;
for (i = 0; eccsteps; eccsteps--, i += eccbytes, p += eccsize) {
- chip->ecc.hwctl(mtd, NAND_ECC_WRITE);
+ chip->ecc.hwctl(chip, NAND_ECC_WRITE);
ret = nand_write_data_op(chip, p, eccsize, false);
if (ret)
return ret;
- chip->ecc.calculate(mtd, p, &ecc_calc[i]);
+ chip->ecc.calculate(chip, p, &ecc_calc[i]);
}
ret = mtd_ooblayout_set_eccbytes(mtd, ecc_calc, chip->oob_poi, 0,
/**
* nand_write_subpage_hwecc - [REPLACEABLE] hardware ECC based subpage write
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @offset: column address of subpage within the page
* @data_len: data length
* @oob_required: must write chip->oob_poi to OOB
* @page: page number to write
*/
-static int nand_write_subpage_hwecc(struct mtd_info *mtd,
- struct nand_chip *chip, uint32_t offset,
- uint32_t data_len, const uint8_t *buf,
- int oob_required, int page)
+static int nand_write_subpage_hwecc(struct nand_chip *chip, uint32_t offset,
+ uint32_t data_len, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
uint8_t *oob_buf = chip->oob_poi;
uint8_t *ecc_calc = chip->ecc.calc_buf;
int ecc_size = chip->ecc.size;
for (step = 0; step < ecc_steps; step++) {
/* configure controller for WRITE access */
- chip->ecc.hwctl(mtd, NAND_ECC_WRITE);
+ chip->ecc.hwctl(chip, NAND_ECC_WRITE);
/* write data (untouched subpages already masked by 0xFF) */
ret = nand_write_data_op(chip, buf, ecc_size, false);
if ((step < start_step) || (step > end_step))
memset(ecc_calc, 0xff, ecc_bytes);
else
- chip->ecc.calculate(mtd, buf, ecc_calc);
+ chip->ecc.calculate(chip, buf, ecc_calc);
/* mask OOB of un-touched subpages by padding 0xFF */
/* if oob_required, preserve OOB metadata of written subpage */
/**
* nand_write_page_syndrome - [REPLACEABLE] hardware ECC syndrome based page write
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: data buffer
* @oob_required: must write chip->oob_poi to OOB
* The hw generator calculates the error syndrome automatically. Therefore we
* need a special oob layout and handling.
*/
-static int nand_write_page_syndrome(struct mtd_info *mtd,
- struct nand_chip *chip,
- const uint8_t *buf, int oob_required,
- int page)
+static int nand_write_page_syndrome(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int i, eccsize = chip->ecc.size;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
return ret;
for (i = 0; eccsteps; eccsteps--, i += eccbytes, p += eccsize) {
- chip->ecc.hwctl(mtd, NAND_ECC_WRITE);
+ chip->ecc.hwctl(chip, NAND_ECC_WRITE);
ret = nand_write_data_op(chip, p, eccsize, false);
if (ret)
oob += chip->ecc.prepad;
}
- chip->ecc.calculate(mtd, p, oob);
+ chip->ecc.calculate(chip, p, oob);
ret = nand_write_data_op(chip, oob, eccbytes, false);
if (ret)
subpage = 0;
if (unlikely(raw))
- status = chip->ecc.write_page_raw(mtd, chip, buf,
- oob_required, page);
+ status = chip->ecc.write_page_raw(chip, buf, oob_required,
+ page);
else if (subpage)
- status = chip->ecc.write_subpage(mtd, chip, offset, data_len,
- buf, oob_required, page);
+ status = chip->ecc.write_subpage(chip, offset, data_len, buf,
+ oob_required, page);
else
- status = chip->ecc.write_page(mtd, chip, buf, oob_required,
- page);
+ status = chip->ecc.write_page(chip, buf, oob_required, page);
if (status < 0)
return status;
column = to & (mtd->writesize - 1);
chipnr = (int)(to >> chip->chip_shift);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
/* Check, if it is write protected */
if (nand_check_wp(mtd)) {
/* Check, if we cross a chip boundary */
if (!page) {
chipnr++;
- chip->select_chip(mtd, -1);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, -1);
+ chip->select_chip(chip, chipnr);
}
}
ops->oobretlen = ops->ooblen;
err_out:
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
return ret;
}
/* Grab the device */
panic_nand_get_device(chip, mtd, FL_WRITING);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
/* Wait for the device to get ready */
- panic_nand_wait(mtd, chip, 400);
+ panic_nand_wait(chip, 400);
memset(&ops, 0, sizeof(ops));
ops.len = len;
*/
nand_reset(chip, chipnr);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
/* Shift to get page */
page = (int)(to >> chip->page_shift);
/* Check, if it is write protected */
if (nand_check_wp(mtd)) {
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
return -EROFS;
}
nand_fill_oob(mtd, ops->oobbuf, ops->ooblen, ops);
if (ops->mode == MTD_OPS_RAW)
- status = chip->ecc.write_oob_raw(mtd, chip, page & chip->pagemask);
+ status = chip->ecc.write_oob_raw(chip, page & chip->pagemask);
else
- status = chip->ecc.write_oob(mtd, chip, page & chip->pagemask);
+ status = chip->ecc.write_oob(chip, page & chip->pagemask);
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
if (status)
return status;
/**
* single_erase - [GENERIC] NAND standard block erase command function
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @page: the page address of the block which will be erased
*
* Standard erase command for NAND chips. Returns NAND status.
*/
-static int single_erase(struct mtd_info *mtd, int page)
+static int single_erase(struct nand_chip *chip, int page)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
unsigned int eraseblock;
/* Send commands to erase a block */
*/
static int nand_erase(struct mtd_info *mtd, struct erase_info *instr)
{
- return nand_erase_nand(mtd, instr, 0);
+ return nand_erase_nand(mtd_to_nand(mtd), instr, 0);
}
/**
* nand_erase_nand - [INTERN] erase block(s)
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @instr: erase instruction
* @allowbbt: allow erasing the bbt area
*
* Erase one ore more blocks.
*/
-int nand_erase_nand(struct mtd_info *mtd, struct erase_info *instr,
+int nand_erase_nand(struct nand_chip *chip, struct erase_info *instr,
int allowbbt)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int page, status, pages_per_block, ret, chipnr;
- struct nand_chip *chip = mtd_to_nand(mtd);
loff_t len;
pr_debug("%s: start = 0x%012llx, len = %llu\n",
pages_per_block = 1 << (chip->phys_erase_shift - chip->page_shift);
/* Select the NAND device */
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
/* Check, if it is write protected */
if (nand_check_wp(mtd)) {
(page + pages_per_block))
chip->pagebuf = -1;
- status = chip->erase(mtd, page & chip->pagemask);
+ if (chip->legacy.erase)
+ status = chip->legacy.erase(chip,
+ page & chip->pagemask);
+ else
+ status = single_erase(chip, page & chip->pagemask);
/* See if block erase succeeded */
if (status) {
/* Check, if we cross a chip boundary */
if (len && !(page & chip->pagemask)) {
chipnr++;
- chip->select_chip(mtd, -1);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, -1);
+ chip->select_chip(chip, chipnr);
}
}
erase_exit:
/* Deselect and wake up anyone waiting on the device */
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
nand_release_device(mtd);
/* Return more or less happy */
/* Select the NAND device */
nand_get_device(mtd, FL_READING);
- chip->select_chip(mtd, chipnr);
+ chip->select_chip(chip, chipnr);
ret = nand_block_checkbad(mtd, offs, 0);
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
nand_release_device(mtd);
return ret;
return chip->max_bb_per_die * (part_end_die - part_start_die + 1);
}
-/**
- * nand_default_set_features- [REPLACEABLE] set NAND chip features
- * @mtd: MTD device structure
- * @chip: nand chip info structure
- * @addr: feature address.
- * @subfeature_param: the subfeature parameters, a four bytes array.
- */
-static int nand_default_set_features(struct mtd_info *mtd,
- struct nand_chip *chip, int addr,
- uint8_t *subfeature_param)
-{
- return nand_set_features_op(chip, addr, subfeature_param);
-}
-
-/**
- * nand_default_get_features- [REPLACEABLE] get NAND chip features
- * @mtd: MTD device structure
- * @chip: nand chip info structure
- * @addr: feature address.
- * @subfeature_param: the subfeature parameters, a four bytes array.
- */
-static int nand_default_get_features(struct mtd_info *mtd,
- struct nand_chip *chip, int addr,
- uint8_t *subfeature_param)
-{
- return nand_get_features_op(chip, addr, subfeature_param);
-}
-
-/**
- * nand_get_set_features_notsupp - set/get features stub returning -ENOTSUPP
- * @mtd: MTD device structure
- * @chip: nand chip info structure
- * @addr: feature address.
- * @subfeature_param: the subfeature parameters, a four bytes array.
- *
- * Should be used by NAND controller drivers that do not support the SET/GET
- * FEATURES operations.
- */
-int nand_get_set_features_notsupp(struct mtd_info *mtd, struct nand_chip *chip,
- int addr, u8 *subfeature_param)
-{
- return -ENOTSUPP;
-}
-EXPORT_SYMBOL(nand_get_set_features_notsupp);
-
/**
* nand_suspend - [MTD Interface] Suspend the NAND flash
* @mtd: MTD device structure
/* Set default functions */
static void nand_set_defaults(struct nand_chip *chip)
{
- unsigned int busw = chip->options & NAND_BUSWIDTH_16;
-
- /* check for proper chip_delay setup, set 20us if not */
- if (!chip->chip_delay)
- chip->chip_delay = 20;
-
- /* check, if a user supplied command function given */
- if (!chip->cmdfunc && !chip->exec_op)
- chip->cmdfunc = nand_command;
-
- /* check, if a user supplied wait function given */
- if (chip->waitfunc == NULL)
- chip->waitfunc = nand_wait;
-
- if (!chip->select_chip)
- chip->select_chip = nand_select_chip;
-
- /* set for ONFI nand */
- if (!chip->set_features)
- chip->set_features = nand_default_set_features;
- if (!chip->get_features)
- chip->get_features = nand_default_get_features;
-
- /* If called twice, pointers that depend on busw may need to be reset */
- if (!chip->read_byte || chip->read_byte == nand_read_byte)
- chip->read_byte = busw ? nand_read_byte16 : nand_read_byte;
- if (!chip->read_word)
- chip->read_word = nand_read_word;
- if (!chip->block_bad)
- chip->block_bad = nand_block_bad;
- if (!chip->block_markbad)
- chip->block_markbad = nand_default_block_markbad;
- if (!chip->write_buf || chip->write_buf == nand_write_buf)
- chip->write_buf = busw ? nand_write_buf16 : nand_write_buf;
- if (!chip->write_byte || chip->write_byte == nand_write_byte)
- chip->write_byte = busw ? nand_write_byte16 : nand_write_byte;
- if (!chip->read_buf || chip->read_buf == nand_read_buf)
- chip->read_buf = busw ? nand_read_buf16 : nand_read_buf;
+ nand_legacy_set_defaults(chip);
if (!chip->controller) {
chip->controller = &chip->dummy_controller;
}
/* Sanitize ONFI strings so we can safely print them */
-static void sanitize_string(uint8_t *s, size_t len)
+void sanitize_string(uint8_t *s, size_t len)
{
ssize_t i;
strim(s);
}
-static u16 onfi_crc16(u16 crc, u8 const *p, size_t len)
-{
- int i;
- while (len--) {
- crc ^= *p++ << 8;
- for (i = 0; i < 8; i++)
- crc = (crc << 1) ^ ((crc & 0x8000) ? 0x8005 : 0);
- }
-
- return crc;
-}
-
-/* Parse the Extended Parameter Page. */
-static int nand_flash_detect_ext_param_page(struct nand_chip *chip,
- struct nand_onfi_params *p)
-{
- struct onfi_ext_param_page *ep;
- struct onfi_ext_section *s;
- struct onfi_ext_ecc_info *ecc;
- uint8_t *cursor;
- int ret;
- int len;
- int i;
-
- len = le16_to_cpu(p->ext_param_page_length) * 16;
- ep = kmalloc(len, GFP_KERNEL);
- if (!ep)
- return -ENOMEM;
-
- /* Send our own NAND_CMD_PARAM. */
- ret = nand_read_param_page_op(chip, 0, NULL, 0);
- if (ret)
- goto ext_out;
-
- /* Use the Change Read Column command to skip the ONFI param pages. */
- ret = nand_change_read_column_op(chip,
- sizeof(*p) * p->num_of_param_pages,
- ep, len, true);
- if (ret)
- goto ext_out;
-
- ret = -EINVAL;
- if ((onfi_crc16(ONFI_CRC_BASE, ((uint8_t *)ep) + 2, len - 2)
- != le16_to_cpu(ep->crc))) {
- pr_debug("fail in the CRC.\n");
- goto ext_out;
- }
-
- /*
- * Check the signature.
- * Do not strictly follow the ONFI spec, maybe changed in future.
- */
- if (strncmp(ep->sig, "EPPS", 4)) {
- pr_debug("The signature is invalid.\n");
- goto ext_out;
- }
-
- /* find the ECC section. */
- cursor = (uint8_t *)(ep + 1);
- for (i = 0; i < ONFI_EXT_SECTION_MAX; i++) {
- s = ep->sections + i;
- if (s->type == ONFI_SECTION_TYPE_2)
- break;
- cursor += s->length * 16;
- }
- if (i == ONFI_EXT_SECTION_MAX) {
- pr_debug("We can not find the ECC section.\n");
- goto ext_out;
- }
-
- /* get the info we want. */
- ecc = (struct onfi_ext_ecc_info *)cursor;
-
- if (!ecc->codeword_size) {
- pr_debug("Invalid codeword size\n");
- goto ext_out;
- }
-
- chip->ecc_strength_ds = ecc->ecc_bits;
- chip->ecc_step_ds = 1 << ecc->codeword_size;
- ret = 0;
-
-ext_out:
- kfree(ep);
- return ret;
-}
-
-/*
- * Recover data with bit-wise majority
- */
-static void nand_bit_wise_majority(const void **srcbufs,
- unsigned int nsrcbufs,
- void *dstbuf,
- unsigned int bufsize)
-{
- int i, j, k;
-
- for (i = 0; i < bufsize; i++) {
- u8 val = 0;
-
- for (j = 0; j < 8; j++) {
- unsigned int cnt = 0;
-
- for (k = 0; k < nsrcbufs; k++) {
- const u8 *srcbuf = srcbufs[k];
-
- if (srcbuf[i] & BIT(j))
- cnt++;
- }
-
- if (cnt > nsrcbufs / 2)
- val |= BIT(j);
- }
-
- ((u8 *)dstbuf)[i] = val;
- }
-}
-
-/*
- * Check if the NAND chip is ONFI compliant, returns 1 if it is, 0 otherwise.
- */
-static int nand_flash_detect_onfi(struct nand_chip *chip)
-{
- struct mtd_info *mtd = nand_to_mtd(chip);
- struct nand_onfi_params *p;
- struct onfi_params *onfi;
- int onfi_version = 0;
- char id[4];
- int i, ret, val;
-
- /* Try ONFI for unknown chip or LP */
- ret = nand_readid_op(chip, 0x20, id, sizeof(id));
- if (ret || strncmp(id, "ONFI", 4))
- return 0;
-
- /* ONFI chip: allocate a buffer to hold its parameter page */
- p = kzalloc((sizeof(*p) * 3), GFP_KERNEL);
- if (!p)
- return -ENOMEM;
-
- ret = nand_read_param_page_op(chip, 0, NULL, 0);
- if (ret) {
- ret = 0;
- goto free_onfi_param_page;
- }
-
- for (i = 0; i < 3; i++) {
- ret = nand_read_data_op(chip, &p[i], sizeof(*p), true);
- if (ret) {
- ret = 0;
- goto free_onfi_param_page;
- }
-
- if (onfi_crc16(ONFI_CRC_BASE, (u8 *)&p[i], 254) ==
- le16_to_cpu(p->crc)) {
- if (i)
- memcpy(p, &p[i], sizeof(*p));
- break;
- }
- }
-
- if (i == 3) {
- const void *srcbufs[3] = {p, p + 1, p + 2};
-
- pr_warn("Could not find a valid ONFI parameter page, trying bit-wise majority to recover it\n");
- nand_bit_wise_majority(srcbufs, ARRAY_SIZE(srcbufs), p,
- sizeof(*p));
-
- if (onfi_crc16(ONFI_CRC_BASE, (u8 *)p, 254) !=
- le16_to_cpu(p->crc)) {
- pr_err("ONFI parameter recovery failed, aborting\n");
- goto free_onfi_param_page;
- }
- }
-
- if (chip->manufacturer.desc && chip->manufacturer.desc->ops &&
- chip->manufacturer.desc->ops->fixup_onfi_param_page)
- chip->manufacturer.desc->ops->fixup_onfi_param_page(chip, p);
-
- /* Check version */
- val = le16_to_cpu(p->revision);
- if (val & ONFI_VERSION_2_3)
- onfi_version = 23;
- else if (val & ONFI_VERSION_2_2)
- onfi_version = 22;
- else if (val & ONFI_VERSION_2_1)
- onfi_version = 21;
- else if (val & ONFI_VERSION_2_0)
- onfi_version = 20;
- else if (val & ONFI_VERSION_1_0)
- onfi_version = 10;
-
- if (!onfi_version) {
- pr_info("unsupported ONFI version: %d\n", val);
- goto free_onfi_param_page;
- }
-
- sanitize_string(p->manufacturer, sizeof(p->manufacturer));
- sanitize_string(p->model, sizeof(p->model));
- chip->parameters.model = kstrdup(p->model, GFP_KERNEL);
- if (!chip->parameters.model) {
- ret = -ENOMEM;
- goto free_onfi_param_page;
- }
-
- mtd->writesize = le32_to_cpu(p->byte_per_page);
-
- /*
- * pages_per_block and blocks_per_lun may not be a power-of-2 size
- * (don't ask me who thought of this...). MTD assumes that these
- * dimensions will be power-of-2, so just truncate the remaining area.
- */
- mtd->erasesize = 1 << (fls(le32_to_cpu(p->pages_per_block)) - 1);
- mtd->erasesize *= mtd->writesize;
-
- mtd->oobsize = le16_to_cpu(p->spare_bytes_per_page);
-
- /* See erasesize comment */
- chip->chipsize = 1 << (fls(le32_to_cpu(p->blocks_per_lun)) - 1);
- chip->chipsize *= (uint64_t)mtd->erasesize * p->lun_count;
- chip->bits_per_cell = p->bits_per_cell;
-
- chip->max_bb_per_die = le16_to_cpu(p->bb_per_lun);
- chip->blocks_per_die = le32_to_cpu(p->blocks_per_lun);
-
- if (le16_to_cpu(p->features) & ONFI_FEATURE_16_BIT_BUS)
- chip->options |= NAND_BUSWIDTH_16;
-
- if (p->ecc_bits != 0xff) {
- chip->ecc_strength_ds = p->ecc_bits;
- chip->ecc_step_ds = 512;
- } else if (onfi_version >= 21 &&
- (le16_to_cpu(p->features) & ONFI_FEATURE_EXT_PARAM_PAGE)) {
-
- /*
- * The nand_flash_detect_ext_param_page() uses the
- * Change Read Column command which maybe not supported
- * by the chip->cmdfunc. So try to update the chip->cmdfunc
- * now. We do not replace user supplied command function.
- */
- if (mtd->writesize > 512 && chip->cmdfunc == nand_command)
- chip->cmdfunc = nand_command_lp;
-
- /* The Extended Parameter Page is supported since ONFI 2.1. */
- if (nand_flash_detect_ext_param_page(chip, p))
- pr_warn("Failed to detect ONFI extended param page\n");
- } else {
- pr_warn("Could not retrieve ONFI ECC requirements\n");
- }
-
- /* Save some parameters from the parameter page for future use */
- if (le16_to_cpu(p->opt_cmd) & ONFI_OPT_CMD_SET_GET_FEATURES) {
- chip->parameters.supports_set_get_features = true;
- bitmap_set(chip->parameters.get_feature_list,
- ONFI_FEATURE_ADDR_TIMING_MODE, 1);
- bitmap_set(chip->parameters.set_feature_list,
- ONFI_FEATURE_ADDR_TIMING_MODE, 1);
- }
-
- onfi = kzalloc(sizeof(*onfi), GFP_KERNEL);
- if (!onfi) {
- ret = -ENOMEM;
- goto free_model;
- }
-
- onfi->version = onfi_version;
- onfi->tPROG = le16_to_cpu(p->t_prog);
- onfi->tBERS = le16_to_cpu(p->t_bers);
- onfi->tR = le16_to_cpu(p->t_r);
- onfi->tCCS = le16_to_cpu(p->t_ccs);
- onfi->async_timing_mode = le16_to_cpu(p->async_timing_mode);
- onfi->vendor_revision = le16_to_cpu(p->vendor_revision);
- memcpy(onfi->vendor, p->vendor, sizeof(p->vendor));
- chip->parameters.onfi = onfi;
-
- /* Identification done, free the full ONFI parameter page and exit */
- kfree(p);
-
- return 1;
-
-free_model:
- kfree(chip->parameters.model);
-free_onfi_param_page:
- kfree(p);
-
- return ret;
-}
-
-/*
- * Check if the NAND chip is JEDEC compliant, returns 1 if it is, 0 otherwise.
- */
-static int nand_flash_detect_jedec(struct nand_chip *chip)
-{
- struct mtd_info *mtd = nand_to_mtd(chip);
- struct nand_jedec_params *p;
- struct jedec_ecc_info *ecc;
- int jedec_version = 0;
- char id[5];
- int i, val, ret;
-
- /* Try JEDEC for unknown chip or LP */
- ret = nand_readid_op(chip, 0x40, id, sizeof(id));
- if (ret || strncmp(id, "JEDEC", sizeof(id)))
- return 0;
-
- /* JEDEC chip: allocate a buffer to hold its parameter page */
- p = kzalloc(sizeof(*p), GFP_KERNEL);
- if (!p)
- return -ENOMEM;
-
- ret = nand_read_param_page_op(chip, 0x40, NULL, 0);
- if (ret) {
- ret = 0;
- goto free_jedec_param_page;
- }
-
- for (i = 0; i < 3; i++) {
- ret = nand_read_data_op(chip, p, sizeof(*p), true);
- if (ret) {
- ret = 0;
- goto free_jedec_param_page;
- }
-
- if (onfi_crc16(ONFI_CRC_BASE, (uint8_t *)p, 510) ==
- le16_to_cpu(p->crc))
- break;
- }
-
- if (i == 3) {
- pr_err("Could not find valid JEDEC parameter page; aborting\n");
- goto free_jedec_param_page;
- }
-
- /* Check version */
- val = le16_to_cpu(p->revision);
- if (val & (1 << 2))
- jedec_version = 10;
- else if (val & (1 << 1))
- jedec_version = 1; /* vendor specific version */
-
- if (!jedec_version) {
- pr_info("unsupported JEDEC version: %d\n", val);
- goto free_jedec_param_page;
- }
-
- sanitize_string(p->manufacturer, sizeof(p->manufacturer));
- sanitize_string(p->model, sizeof(p->model));
- chip->parameters.model = kstrdup(p->model, GFP_KERNEL);
- if (!chip->parameters.model) {
- ret = -ENOMEM;
- goto free_jedec_param_page;
- }
-
- mtd->writesize = le32_to_cpu(p->byte_per_page);
-
- /* Please reference to the comment for nand_flash_detect_onfi. */
- mtd->erasesize = 1 << (fls(le32_to_cpu(p->pages_per_block)) - 1);
- mtd->erasesize *= mtd->writesize;
-
- mtd->oobsize = le16_to_cpu(p->spare_bytes_per_page);
-
- /* Please reference to the comment for nand_flash_detect_onfi. */
- chip->chipsize = 1 << (fls(le32_to_cpu(p->blocks_per_lun)) - 1);
- chip->chipsize *= (uint64_t)mtd->erasesize * p->lun_count;
- chip->bits_per_cell = p->bits_per_cell;
-
- if (le16_to_cpu(p->features) & JEDEC_FEATURE_16_BIT_BUS)
- chip->options |= NAND_BUSWIDTH_16;
-
- /* ECC info */
- ecc = &p->ecc_info[0];
-
- if (ecc->codeword_size >= 9) {
- chip->ecc_strength_ds = ecc->ecc_bits;
- chip->ecc_step_ds = 1 << ecc->codeword_size;
- } else {
- pr_warn("Invalid codeword size\n");
- }
-
-free_jedec_param_page:
- kfree(p);
- return ret;
-}
-
/*
* nand_id_has_period - Check if an ID string has a given wraparound period
* @id_data: the ID string
chip->manufacturer.desc->ops->cleanup(chip);
}
+static const char *
+nand_manufacturer_name(const struct nand_manufacturer *manufacturer)
+{
+ return manufacturer ? manufacturer->name : "Unknown";
+}
+
/*
* Get the flash and manufacturer id and lookup if the type is supported.
*/
return ret;
/* Select the device */
- chip->select_chip(mtd, 0);
+ chip->select_chip(chip, 0);
/* Send the command for reading device ID */
ret = nand_readid_op(chip, 0, id_data, 2);
if (!type->name || !type->pagesize) {
/* Check if the chip is ONFI compliant */
- ret = nand_flash_detect_onfi(chip);
+ ret = nand_onfi_detect(chip);
if (ret < 0)
return ret;
else if (ret)
goto ident_done;
/* Check if the chip is JEDEC compliant */
- ret = nand_flash_detect_jedec(chip);
+ ret = nand_jedec_detect(chip);
if (ret < 0)
return ret;
else if (ret)
chip->options |= NAND_ROW_ADDR_3;
chip->badblockbits = 8;
- chip->erase = single_erase;
- /* Do not replace user supplied command function! */
- if (mtd->writesize > 512 && chip->cmdfunc == nand_command)
- chip->cmdfunc = nand_command_lp;
+ nand_legacy_adjust_cmdfunc(chip);
pr_info("device found, Manufacturer ID: 0x%02x, Chip ID: 0x%02x\n",
maf_id, dev_id);
/**
* nand_scan_ident - Scan for the NAND device
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @maxchips: number of chips to scan for
* @table: alternative NAND ID table
*
* prevented dynamic allocations during this phase which was unconvenient and
* as been banned for the benefit of the ->init_ecc()/cleanup_ecc() hooks.
*/
-static int nand_scan_ident(struct mtd_info *mtd, int maxchips,
+static int nand_scan_ident(struct nand_chip *chip, unsigned int maxchips,
struct nand_flash_dev *table)
{
- int i, nand_maf_id, nand_dev_id;
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
+ int nand_maf_id, nand_dev_id;
+ unsigned int i;
int ret;
/* Enforce the right timings for reset/detection */
if (!mtd->name && mtd->dev.parent)
mtd->name = dev_name(mtd->dev.parent);
- /*
- * ->cmdfunc() is legacy and will only be used if ->exec_op() is not
- * populated.
- */
- if (!chip->exec_op) {
- /*
- * Default functions assigned for ->cmdfunc() and
- * ->select_chip() both expect ->cmd_ctrl() to be populated.
- */
- if ((!chip->cmdfunc || !chip->select_chip) && !chip->cmd_ctrl) {
- pr_err("->cmd_ctrl() should be provided\n");
- return -EINVAL;
- }
+ if (chip->exec_op && !chip->select_chip) {
+ pr_err("->select_chip() is mandatory when implementing ->exec_op()\n");
+ return -EINVAL;
}
+ ret = nand_legacy_check_hooks(chip);
+ if (ret)
+ return ret;
+
/* Set the default functions */
nand_set_defaults(chip);
if (ret) {
if (!(chip->options & NAND_SCAN_SILENT_NODEV))
pr_warn("No NAND device found\n");
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
return ret;
}
nand_maf_id = chip->id.data[0];
nand_dev_id = chip->id.data[1];
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
/* Check for a chip array */
for (i = 1; i < maxchips; i++) {
/* See comment in nand_get_flash_type for reset */
nand_reset(chip, i);
- chip->select_chip(mtd, i);
+ chip->select_chip(chip, i);
/* Send the command for reading device ID */
nand_readid_op(chip, 0, id, sizeof(id));
/* Read manufacturer and device IDs */
if (nand_maf_id != id[0] || nand_dev_id != id[1]) {
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
break;
}
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
}
if (i > 1)
pr_info("%d chips detected\n", i);
ecc->size = 256;
ecc->bytes = 3;
ecc->strength = 1;
+
+ if (IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC))
+ ecc->options |= NAND_ECC_SOFT_HAMMING_SM_ORDER;
+
return 0;
case NAND_ECC_BCH:
if (!mtd_nand_has_bch()) {
/**
* nand_scan_tail - Scan for the NAND device
- * @mtd: MTD device structure
+ * @chip: NAND chip object
*
* This is the second phase of the normal nand_scan() function. It fills out
* all the uninitialized function pointers with the defaults and scans for a
* bad block table if appropriate.
*/
-static int nand_scan_tail(struct mtd_info *mtd)
+static int nand_scan_tail(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
int ret, i;
* to explictly select the relevant die when interacting with the NAND
* chip.
*/
- chip->select_chip(mtd, 0);
+ chip->select_chip(chip, 0);
ret = nand_manufacturer_init(chip);
- chip->select_chip(mtd, -1);
+ chip->select_chip(chip, -1);
if (ret)
goto err_free_buf;
/**
* nand_scan_with_ids - [NAND Interface] Scan for the NAND device
- * @mtd: MTD device structure
- * @maxchips: number of chips to scan for. @nand_scan_ident() will not be run if
- * this parameter is zero (useful for specific drivers that must
- * handle this part of the process themselves, e.g docg4).
+ * @chip: NAND chip object
+ * @maxchips: number of chips to scan for.
* @ids: optional flash IDs table
*
* This fills out all the uninitialized function pointers with the defaults.
* The flash ID is read and the mtd/chip structures are filled with the
* appropriate values.
*/
-int nand_scan_with_ids(struct mtd_info *mtd, int maxchips,
+int nand_scan_with_ids(struct nand_chip *chip, unsigned int maxchips,
struct nand_flash_dev *ids)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
int ret;
- if (maxchips) {
- ret = nand_scan_ident(mtd, maxchips, ids);
- if (ret)
- return ret;
- }
+ if (!maxchips)
+ return -EINVAL;
+
+ ret = nand_scan_ident(chip, maxchips, ids);
+ if (ret)
+ return ret;
ret = nand_attach(chip);
if (ret)
goto cleanup_ident;
- ret = nand_scan_tail(mtd);
+ ret = nand_scan_tail(chip);
if (ret)
goto detach_chip;
/**
* nand_release - [NAND Interface] Unregister the MTD device and free resources
* held by the NAND device
- * @mtd: MTD device structure
+ * @chip: NAND chip object
*/
-void nand_release(struct mtd_info *mtd)
+void nand_release(struct nand_chip *chip)
{
- mtd_device_unregister(mtd);
- nand_cleanup(mtd_to_nand(mtd));
+ mtd_device_unregister(nand_to_mtd(chip));
+ nand_cleanup(chip);
}
EXPORT_SYMBOL_GPL(nand_release);
#include <linux/types.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/bbm.h>
-#include <linux/mtd/rawnand.h>
#include <linux/bitops.h>
#include <linux/delay.h>
#include <linux/vmalloc.h>
#include <linux/export.h>
#include <linux/string.h>
+#include "internals.h"
+
#define BBT_BLOCK_GOOD 0x00
#define BBT_BLOCK_WORN 0x01
#define BBT_BLOCK_RESERVED 0x02
struct nand_bbt_descr *td,
int chip, int block)
{
- struct mtd_info *mtd = nand_to_mtd(this);
loff_t to;
int res;
bbt_mark_entry(this, block, BBT_BLOCK_WORN);
to = (loff_t)block << this->bbt_erase_shift;
- res = this->block_markbad(mtd, to);
+ res = nand_markbad_bbm(this, to);
if (res)
pr_warn("nand_bbt: error %d while marking block %d bad\n",
res, block);
memset(&einfo, 0, sizeof(einfo));
einfo.addr = to;
einfo.len = 1 << this->bbt_erase_shift;
- res = nand_erase_nand(mtd, &einfo, 1);
+ res = nand_erase_nand(this, &einfo, 1);
if (res < 0) {
pr_warn("nand_bbt: error while erasing BBT block %d\n",
res);
/**
* nand_isreserved_bbt - [NAND Interface] Check if a block is reserved
- * @mtd: MTD device structure
+ * @this: NAND chip object
* @offs: offset in the device
*/
-int nand_isreserved_bbt(struct mtd_info *mtd, loff_t offs)
+int nand_isreserved_bbt(struct nand_chip *this, loff_t offs)
{
- struct nand_chip *this = mtd_to_nand(mtd);
int block;
block = (int)(offs >> this->bbt_erase_shift);
/**
* nand_isbad_bbt - [NAND Interface] Check if a block is bad
- * @mtd: MTD device structure
+ * @this: NAND chip object
* @offs: offset in the device
* @allowbbt: allow access to bad block table region
*/
-int nand_isbad_bbt(struct mtd_info *mtd, loff_t offs, int allowbbt)
+int nand_isbad_bbt(struct nand_chip *this, loff_t offs, int allowbbt)
{
- struct nand_chip *this = mtd_to_nand(mtd);
int block, res;
block = (int)(offs >> this->bbt_erase_shift);
/**
* nand_markbad_bbt - [NAND Interface] Mark a block bad in the BBT
- * @mtd: MTD device structure
+ * @this: NAND chip object
* @offs: offset of the bad block
*/
-int nand_markbad_bbt(struct mtd_info *mtd, loff_t offs)
+int nand_markbad_bbt(struct nand_chip *this, loff_t offs)
{
- struct nand_chip *this = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(this);
int block, ret = 0;
block = (int)(offs >> this->bbt_erase_shift);
/**
* nand_bch_calculate_ecc - [NAND Interface] Calculate ECC for data block
- * @mtd: MTD block structure
+ * @chip: NAND chip object
* @buf: input buffer with raw data
* @code: output buffer with ECC
*/
-int nand_bch_calculate_ecc(struct mtd_info *mtd, const unsigned char *buf,
+int nand_bch_calculate_ecc(struct nand_chip *chip, const unsigned char *buf,
unsigned char *code)
{
- const struct nand_chip *chip = mtd_to_nand(mtd);
struct nand_bch_control *nbc = chip->ecc.priv;
unsigned int i;
/**
* nand_bch_correct_data - [NAND Interface] Detect and correct bit error(s)
- * @mtd: MTD block structure
+ * @chip: NAND chip object
* @buf: raw data read from the chip
* @read_ecc: ECC from the chip
* @calc_ecc: the ECC calculated from raw data
*
* Detect and correct bit errors for a data byte block
*/
-int nand_bch_correct_data(struct mtd_info *mtd, unsigned char *buf,
+int nand_bch_correct_data(struct nand_chip *chip, unsigned char *buf,
unsigned char *read_ecc, unsigned char *calc_ecc)
{
- const struct nand_chip *chip = mtd_to_nand(mtd);
struct nand_bch_control *nbc = chip->ecc.priv;
unsigned int *errloc = nbc->errloc;
int i, count;
* @buf: input buffer with raw data
* @eccsize: data bytes per ECC step (256 or 512)
* @code: output buffer with ECC
+ * @sm_order: Smart Media byte ordering
*/
void __nand_calculate_ecc(const unsigned char *buf, unsigned int eccsize,
- unsigned char *code)
+ unsigned char *code, bool sm_order)
{
int i;
const uint32_t *bp = (uint32_t *)buf;
* possible, but benchmarks showed that on the system this is developed
* the code below is the fastest
*/
-#ifdef CONFIG_MTD_NAND_ECC_SMC
- code[0] =
- (invparity[rp7] << 7) |
- (invparity[rp6] << 6) |
- (invparity[rp5] << 5) |
- (invparity[rp4] << 4) |
- (invparity[rp3] << 3) |
- (invparity[rp2] << 2) |
- (invparity[rp1] << 1) |
- (invparity[rp0]);
- code[1] =
- (invparity[rp15] << 7) |
- (invparity[rp14] << 6) |
- (invparity[rp13] << 5) |
- (invparity[rp12] << 4) |
- (invparity[rp11] << 3) |
- (invparity[rp10] << 2) |
- (invparity[rp9] << 1) |
- (invparity[rp8]);
-#else
- code[1] =
- (invparity[rp7] << 7) |
- (invparity[rp6] << 6) |
- (invparity[rp5] << 5) |
- (invparity[rp4] << 4) |
- (invparity[rp3] << 3) |
- (invparity[rp2] << 2) |
- (invparity[rp1] << 1) |
- (invparity[rp0]);
- code[0] =
- (invparity[rp15] << 7) |
- (invparity[rp14] << 6) |
- (invparity[rp13] << 5) |
- (invparity[rp12] << 4) |
- (invparity[rp11] << 3) |
- (invparity[rp10] << 2) |
- (invparity[rp9] << 1) |
- (invparity[rp8]);
-#endif
+ if (sm_order) {
+ code[0] = (invparity[rp7] << 7) | (invparity[rp6] << 6) |
+ (invparity[rp5] << 5) | (invparity[rp4] << 4) |
+ (invparity[rp3] << 3) | (invparity[rp2] << 2) |
+ (invparity[rp1] << 1) | (invparity[rp0]);
+ code[1] = (invparity[rp15] << 7) | (invparity[rp14] << 6) |
+ (invparity[rp13] << 5) | (invparity[rp12] << 4) |
+ (invparity[rp11] << 3) | (invparity[rp10] << 2) |
+ (invparity[rp9] << 1) | (invparity[rp8]);
+ } else {
+ code[1] = (invparity[rp7] << 7) | (invparity[rp6] << 6) |
+ (invparity[rp5] << 5) | (invparity[rp4] << 4) |
+ (invparity[rp3] << 3) | (invparity[rp2] << 2) |
+ (invparity[rp1] << 1) | (invparity[rp0]);
+ code[0] = (invparity[rp15] << 7) | (invparity[rp14] << 6) |
+ (invparity[rp13] << 5) | (invparity[rp12] << 4) |
+ (invparity[rp11] << 3) | (invparity[rp10] << 2) |
+ (invparity[rp9] << 1) | (invparity[rp8]);
+ }
+
if (eccsize_mult == 1)
code[2] =
(invparity[par & 0xf0] << 7) |
/**
* nand_calculate_ecc - [NAND Interface] Calculate 3-byte ECC for 256/512-byte
* block
- * @mtd: MTD block structure
+ * @chip: NAND chip object
* @buf: input buffer with raw data
* @code: output buffer with ECC
*/
-int nand_calculate_ecc(struct mtd_info *mtd, const unsigned char *buf,
+int nand_calculate_ecc(struct nand_chip *chip, const unsigned char *buf,
unsigned char *code)
{
- __nand_calculate_ecc(buf,
- mtd_to_nand(mtd)->ecc.size, code);
+ bool sm_order = chip->ecc.options & NAND_ECC_SOFT_HAMMING_SM_ORDER;
+
+ __nand_calculate_ecc(buf, chip->ecc.size, code, sm_order);
return 0;
}
* @read_ecc: ECC from the chip
* @calc_ecc: the ECC calculated from raw data
* @eccsize: data bytes per ECC step (256 or 512)
+ * @sm_order: Smart Media byte order
*
* Detect and correct a 1 bit error for eccsize byte block
*/
int __nand_correct_data(unsigned char *buf,
unsigned char *read_ecc, unsigned char *calc_ecc,
- unsigned int eccsize)
+ unsigned int eccsize, bool sm_order)
{
unsigned char b0, b1, b2, bit_addr;
unsigned int byte_addr;
* we might need the xor result more than once,
* so keep them in a local var
*/
-#ifdef CONFIG_MTD_NAND_ECC_SMC
- b0 = read_ecc[0] ^ calc_ecc[0];
- b1 = read_ecc[1] ^ calc_ecc[1];
-#else
- b0 = read_ecc[1] ^ calc_ecc[1];
- b1 = read_ecc[0] ^ calc_ecc[0];
-#endif
+ if (sm_order) {
+ b0 = read_ecc[0] ^ calc_ecc[0];
+ b1 = read_ecc[1] ^ calc_ecc[1];
+ } else {
+ b0 = read_ecc[1] ^ calc_ecc[1];
+ b1 = read_ecc[0] ^ calc_ecc[0];
+ }
+
b2 = read_ecc[2] ^ calc_ecc[2];
/* check if there are any bitfaults */
/**
* nand_correct_data - [NAND Interface] Detect and correct bit error(s)
- * @mtd: MTD block structure
+ * @chip: NAND chip object
* @buf: raw data read from the chip
* @read_ecc: ECC from the chip
* @calc_ecc: the ECC calculated from raw data
*
* Detect and correct a 1 bit error for 256/512 byte block
*/
-int nand_correct_data(struct mtd_info *mtd, unsigned char *buf,
+int nand_correct_data(struct nand_chip *chip, unsigned char *buf,
unsigned char *read_ecc, unsigned char *calc_ecc)
{
- return __nand_correct_data(buf, read_ecc, calc_ecc,
- mtd_to_nand(mtd)->ecc.size);
+ bool sm_order = chip->ecc.options & NAND_ECC_SOFT_HAMMING_SM_ORDER;
+
+ return __nand_correct_data(buf, read_ecc, calc_ecc, chip->ecc.size,
+ sm_order);
}
EXPORT_SYMBOL(nand_correct_data);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 Toradex AG
+ *
+ * Author: Marcel Ziswiler <marcel.ziswiler@toradex.com>
+ */
+
+#include <linux/mtd/rawnand.h>
+#include "internals.h"
+
+static void esmt_nand_decode_id(struct nand_chip *chip)
+{
+ nand_decode_ext_id(chip);
+
+ /* Extract ECC requirements from 5th id byte. */
+ if (chip->id.len >= 5 && nand_is_slc(chip)) {
+ chip->ecc_step_ds = 512;
+ switch (chip->id.data[4] & 0x3) {
+ case 0x0:
+ chip->ecc_strength_ds = 4;
+ break;
+ case 0x1:
+ chip->ecc_strength_ds = 2;
+ break;
+ case 0x2:
+ chip->ecc_strength_ds = 1;
+ break;
+ default:
+ WARN(1, "Could not get ECC info");
+ chip->ecc_step_ds = 0;
+ break;
+ }
+ }
+}
+
+static int esmt_nand_init(struct nand_chip *chip)
+{
+ if (nand_is_slc(chip))
+ chip->bbt_options |= NAND_BBT_SCAN2NDPAGE;
+
+ return 0;
+}
+
+const struct nand_manufacturer_ops esmt_nand_manuf_ops = {
+ .detect = esmt_nand_decode_id,
+ .init = esmt_nand_init,
+};
* GNU General Public License for more details.
*/
-#include <linux/mtd/rawnand.h>
#include <linux/sizes.h>
#include <linux/slab.h>
+#include "internals.h"
+
#define NAND_HYNIX_CMD_SET_PARAMS 0x36
#define NAND_HYNIX_CMD_APPLY_PARAMS 0x16
static int hynix_nand_cmd_op(struct nand_chip *chip, u8 cmd)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
-
if (chip->exec_op) {
struct nand_op_instr instrs[] = {
NAND_OP_CMD(cmd, 0),
return nand_exec_op(chip, &op);
}
- chip->cmdfunc(mtd, cmd, -1, -1);
+ chip->legacy.cmdfunc(chip, cmd, -1, -1);
return 0;
}
static int hynix_nand_reg_write_op(struct nand_chip *chip, u8 addr, u8 val)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
u16 column = ((u16)addr << 8) | addr;
if (chip->exec_op) {
return nand_exec_op(chip, &op);
}
- chip->cmdfunc(mtd, NAND_CMD_NONE, column, -1);
- chip->write_byte(mtd, val);
+ chip->legacy.cmdfunc(chip, NAND_CMD_NONE, column, -1);
+ chip->legacy.write_byte(chip, val);
return 0;
}
-static int hynix_nand_setup_read_retry(struct mtd_info *mtd, int retry_mode)
+static int hynix_nand_setup_read_retry(struct nand_chip *chip, int retry_mode)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct hynix_nand *hynix = nand_get_manufacturer_data(chip);
const u8 *values;
int i, ret;
* published by the Free Software Foundation.
*
*/
-#include <linux/mtd/rawnand.h>
+
#include <linux/sizes.h>
+#include "internals.h"
+
#define LP_OPTIONS 0
#define LP_OPTIONS16 (LP_OPTIONS | NAND_BUSWIDTH_16)
/* Manufacturer IDs */
static const struct nand_manufacturer nand_manufacturers[] = {
- {NAND_MFR_TOSHIBA, "Toshiba", &toshiba_nand_manuf_ops},
- {NAND_MFR_ESMT, "ESMT"},
- {NAND_MFR_SAMSUNG, "Samsung", &samsung_nand_manuf_ops},
+ {NAND_MFR_AMD, "AMD/Spansion", &amd_nand_manuf_ops},
+ {NAND_MFR_ATO, "ATO"},
+ {NAND_MFR_EON, "Eon"},
+ {NAND_MFR_ESMT, "ESMT", &esmt_nand_manuf_ops},
{NAND_MFR_FUJITSU, "Fujitsu"},
- {NAND_MFR_NATIONAL, "National"},
- {NAND_MFR_RENESAS, "Renesas"},
- {NAND_MFR_STMICRO, "ST Micro"},
{NAND_MFR_HYNIX, "Hynix", &hynix_nand_manuf_ops},
- {NAND_MFR_MICRON, "Micron", µn_nand_manuf_ops},
- {NAND_MFR_AMD, "AMD/Spansion", &amd_nand_manuf_ops},
+ {NAND_MFR_INTEL, "Intel"},
{NAND_MFR_MACRONIX, "Macronix", ¯onix_nand_manuf_ops},
- {NAND_MFR_EON, "Eon"},
+ {NAND_MFR_MICRON, "Micron", µn_nand_manuf_ops},
+ {NAND_MFR_NATIONAL, "National"},
+ {NAND_MFR_RENESAS, "Renesas"},
+ {NAND_MFR_SAMSUNG, "Samsung", &samsung_nand_manuf_ops},
{NAND_MFR_SANDISK, "SanDisk"},
- {NAND_MFR_INTEL, "Intel"},
- {NAND_MFR_ATO, "ATO"},
+ {NAND_MFR_STMICRO, "ST Micro"},
+ {NAND_MFR_TOSHIBA, "Toshiba", &toshiba_nand_manuf_ops},
{NAND_MFR_WINBOND, "Winbond"},
};
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2000 Steven J. Hill (sjhill@realitydiluted.com)
+ * 2002-2006 Thomas Gleixner (tglx@linutronix.de)
+ *
+ * Credits:
+ * David Woodhouse for adding multichip support
+ *
+ * Aleph One Ltd. and Toby Churchill Ltd. for supporting the
+ * rework for 2K page size chips
+ *
+ * This file contains all ONFI helpers.
+ */
+
+#include <linux/slab.h>
+
+#include "internals.h"
+
+/*
+ * Check if the NAND chip is JEDEC compliant, returns 1 if it is, 0 otherwise.
+ */
+int nand_jedec_detect(struct nand_chip *chip)
+{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+ struct nand_jedec_params *p;
+ struct jedec_ecc_info *ecc;
+ int jedec_version = 0;
+ char id[5];
+ int i, val, ret;
+
+ /* Try JEDEC for unknown chip or LP */
+ ret = nand_readid_op(chip, 0x40, id, sizeof(id));
+ if (ret || strncmp(id, "JEDEC", sizeof(id)))
+ return 0;
+
+ /* JEDEC chip: allocate a buffer to hold its parameter page */
+ p = kzalloc(sizeof(*p), GFP_KERNEL);
+ if (!p)
+ return -ENOMEM;
+
+ ret = nand_read_param_page_op(chip, 0x40, NULL, 0);
+ if (ret) {
+ ret = 0;
+ goto free_jedec_param_page;
+ }
+
+ for (i = 0; i < 3; i++) {
+ ret = nand_read_data_op(chip, p, sizeof(*p), true);
+ if (ret) {
+ ret = 0;
+ goto free_jedec_param_page;
+ }
+
+ if (onfi_crc16(ONFI_CRC_BASE, (uint8_t *)p, 510) ==
+ le16_to_cpu(p->crc))
+ break;
+ }
+
+ if (i == 3) {
+ pr_err("Could not find valid JEDEC parameter page; aborting\n");
+ goto free_jedec_param_page;
+ }
+
+ /* Check version */
+ val = le16_to_cpu(p->revision);
+ if (val & (1 << 2))
+ jedec_version = 10;
+ else if (val & (1 << 1))
+ jedec_version = 1; /* vendor specific version */
+
+ if (!jedec_version) {
+ pr_info("unsupported JEDEC version: %d\n", val);
+ goto free_jedec_param_page;
+ }
+
+ sanitize_string(p->manufacturer, sizeof(p->manufacturer));
+ sanitize_string(p->model, sizeof(p->model));
+ chip->parameters.model = kstrdup(p->model, GFP_KERNEL);
+ if (!chip->parameters.model) {
+ ret = -ENOMEM;
+ goto free_jedec_param_page;
+ }
+
+ mtd->writesize = le32_to_cpu(p->byte_per_page);
+
+ /* Please reference to the comment for nand_flash_detect_onfi. */
+ mtd->erasesize = 1 << (fls(le32_to_cpu(p->pages_per_block)) - 1);
+ mtd->erasesize *= mtd->writesize;
+
+ mtd->oobsize = le16_to_cpu(p->spare_bytes_per_page);
+
+ /* Please reference to the comment for nand_flash_detect_onfi. */
+ chip->chipsize = 1 << (fls(le32_to_cpu(p->blocks_per_lun)) - 1);
+ chip->chipsize *= (uint64_t)mtd->erasesize * p->lun_count;
+ chip->bits_per_cell = p->bits_per_cell;
+
+ if (le16_to_cpu(p->features) & JEDEC_FEATURE_16_BIT_BUS)
+ chip->options |= NAND_BUSWIDTH_16;
+
+ /* ECC info */
+ ecc = &p->ecc_info[0];
+
+ if (ecc->codeword_size >= 9) {
+ chip->ecc_strength_ds = ecc->ecc_bits;
+ chip->ecc_step_ds = 1 << ecc->codeword_size;
+ } else {
+ pr_warn("Invalid codeword size\n");
+ }
+
+free_jedec_param_page:
+ kfree(p);
+ return ret;
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2000 Steven J. Hill (sjhill@realitydiluted.com)
+ * 2002-2006 Thomas Gleixner (tglx@linutronix.de)
+ *
+ * Credits:
+ * David Woodhouse for adding multichip support
+ *
+ * Aleph One Ltd. and Toby Churchill Ltd. for supporting the
+ * rework for 2K page size chips
+ *
+ * This file contains all legacy helpers/code that should be removed
+ * at some point.
+ */
+
+#include <linux/delay.h>
+#include <linux/io.h>
+#include <linux/nmi.h>
+
+#include "internals.h"
+
+/**
+ * nand_read_byte - [DEFAULT] read one byte from the chip
+ * @chip: NAND chip object
+ *
+ * Default read function for 8bit buswidth
+ */
+static uint8_t nand_read_byte(struct nand_chip *chip)
+{
+ return readb(chip->legacy.IO_ADDR_R);
+}
+
+/**
+ * nand_read_byte16 - [DEFAULT] read one byte endianness aware from the chip
+ * @chip: NAND chip object
+ *
+ * Default read function for 16bit buswidth with endianness conversion.
+ *
+ */
+static uint8_t nand_read_byte16(struct nand_chip *chip)
+{
+ return (uint8_t) cpu_to_le16(readw(chip->legacy.IO_ADDR_R));
+}
+
+/**
+ * nand_select_chip - [DEFAULT] control CE line
+ * @chip: NAND chip object
+ * @chipnr: chipnumber to select, -1 for deselect
+ *
+ * Default select function for 1 chip devices.
+ */
+static void nand_select_chip(struct nand_chip *chip, int chipnr)
+{
+ switch (chipnr) {
+ case -1:
+ chip->legacy.cmd_ctrl(chip, NAND_CMD_NONE,
+ 0 | NAND_CTRL_CHANGE);
+ break;
+ case 0:
+ break;
+
+ default:
+ BUG();
+ }
+}
+
+/**
+ * nand_write_byte - [DEFAULT] write single byte to chip
+ * @chip: NAND chip object
+ * @byte: value to write
+ *
+ * Default function to write a byte to I/O[7:0]
+ */
+static void nand_write_byte(struct nand_chip *chip, uint8_t byte)
+{
+ chip->legacy.write_buf(chip, &byte, 1);
+}
+
+/**
+ * nand_write_byte16 - [DEFAULT] write single byte to a chip with width 16
+ * @chip: NAND chip object
+ * @byte: value to write
+ *
+ * Default function to write a byte to I/O[7:0] on a 16-bit wide chip.
+ */
+static void nand_write_byte16(struct nand_chip *chip, uint8_t byte)
+{
+ uint16_t word = byte;
+
+ /*
+ * It's not entirely clear what should happen to I/O[15:8] when writing
+ * a byte. The ONFi spec (Revision 3.1; 2012-09-19, Section 2.16) reads:
+ *
+ * When the host supports a 16-bit bus width, only data is
+ * transferred at the 16-bit width. All address and command line
+ * transfers shall use only the lower 8-bits of the data bus. During
+ * command transfers, the host may place any value on the upper
+ * 8-bits of the data bus. During address transfers, the host shall
+ * set the upper 8-bits of the data bus to 00h.
+ *
+ * One user of the write_byte callback is nand_set_features. The
+ * four parameters are specified to be written to I/O[7:0], but this is
+ * neither an address nor a command transfer. Let's assume a 0 on the
+ * upper I/O lines is OK.
+ */
+ chip->legacy.write_buf(chip, (uint8_t *)&word, 2);
+}
+
+/**
+ * nand_write_buf - [DEFAULT] write buffer to chip
+ * @chip: NAND chip object
+ * @buf: data buffer
+ * @len: number of bytes to write
+ *
+ * Default write function for 8bit buswidth.
+ */
+static void nand_write_buf(struct nand_chip *chip, const uint8_t *buf, int len)
+{
+ iowrite8_rep(chip->legacy.IO_ADDR_W, buf, len);
+}
+
+/**
+ * nand_read_buf - [DEFAULT] read chip data into buffer
+ * @chip: NAND chip object
+ * @buf: buffer to store date
+ * @len: number of bytes to read
+ *
+ * Default read function for 8bit buswidth.
+ */
+static void nand_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
+{
+ ioread8_rep(chip->legacy.IO_ADDR_R, buf, len);
+}
+
+/**
+ * nand_write_buf16 - [DEFAULT] write buffer to chip
+ * @chip: NAND chip object
+ * @buf: data buffer
+ * @len: number of bytes to write
+ *
+ * Default write function for 16bit buswidth.
+ */
+static void nand_write_buf16(struct nand_chip *chip, const uint8_t *buf,
+ int len)
+{
+ u16 *p = (u16 *) buf;
+
+ iowrite16_rep(chip->legacy.IO_ADDR_W, p, len >> 1);
+}
+
+/**
+ * nand_read_buf16 - [DEFAULT] read chip data into buffer
+ * @chip: NAND chip object
+ * @buf: buffer to store date
+ * @len: number of bytes to read
+ *
+ * Default read function for 16bit buswidth.
+ */
+static void nand_read_buf16(struct nand_chip *chip, uint8_t *buf, int len)
+{
+ u16 *p = (u16 *) buf;
+
+ ioread16_rep(chip->legacy.IO_ADDR_R, p, len >> 1);
+}
+
+/**
+ * panic_nand_wait_ready - [GENERIC] Wait for the ready pin after commands.
+ * @mtd: MTD device structure
+ * @timeo: Timeout
+ *
+ * Helper function for nand_wait_ready used when needing to wait in interrupt
+ * context.
+ */
+static void panic_nand_wait_ready(struct mtd_info *mtd, unsigned long timeo)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ int i;
+
+ /* Wait for the device to get ready */
+ for (i = 0; i < timeo; i++) {
+ if (chip->legacy.dev_ready(chip))
+ break;
+ touch_softlockup_watchdog();
+ mdelay(1);
+ }
+}
+
+/**
+ * nand_wait_ready - [GENERIC] Wait for the ready pin after commands.
+ * @chip: NAND chip object
+ *
+ * Wait for the ready pin after a command, and warn if a timeout occurs.
+ */
+void nand_wait_ready(struct nand_chip *chip)
+{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+ unsigned long timeo = 400;
+
+ if (in_interrupt() || oops_in_progress)
+ return panic_nand_wait_ready(mtd, timeo);
+
+ /* Wait until command is processed or timeout occurs */
+ timeo = jiffies + msecs_to_jiffies(timeo);
+ do {
+ if (chip->legacy.dev_ready(chip))
+ return;
+ cond_resched();
+ } while (time_before(jiffies, timeo));
+
+ if (!chip->legacy.dev_ready(chip))
+ pr_warn_ratelimited("timeout while waiting for chip to become ready\n");
+}
+EXPORT_SYMBOL_GPL(nand_wait_ready);
+
+/**
+ * nand_wait_status_ready - [GENERIC] Wait for the ready status after commands.
+ * @mtd: MTD device structure
+ * @timeo: Timeout in ms
+ *
+ * Wait for status ready (i.e. command done) or timeout.
+ */
+static void nand_wait_status_ready(struct mtd_info *mtd, unsigned long timeo)
+{
+ register struct nand_chip *chip = mtd_to_nand(mtd);
+ int ret;
+
+ timeo = jiffies + msecs_to_jiffies(timeo);
+ do {
+ u8 status;
+
+ ret = nand_read_data_op(chip, &status, sizeof(status), true);
+ if (ret)
+ return;
+
+ if (status & NAND_STATUS_READY)
+ break;
+ touch_softlockup_watchdog();
+ } while (time_before(jiffies, timeo));
+};
+
+/**
+ * nand_command - [DEFAULT] Send command to NAND device
+ * @chip: NAND chip object
+ * @command: the command to be sent
+ * @column: the column address for this command, -1 if none
+ * @page_addr: the page address for this command, -1 if none
+ *
+ * Send command to NAND device. This function is used for small page devices
+ * (512 Bytes per page).
+ */
+static void nand_command(struct nand_chip *chip, unsigned int command,
+ int column, int page_addr)
+{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+ int ctrl = NAND_CTRL_CLE | NAND_CTRL_CHANGE;
+
+ /* Write out the command to the device */
+ if (command == NAND_CMD_SEQIN) {
+ int readcmd;
+
+ if (column >= mtd->writesize) {
+ /* OOB area */
+ column -= mtd->writesize;
+ readcmd = NAND_CMD_READOOB;
+ } else if (column < 256) {
+ /* First 256 bytes --> READ0 */
+ readcmd = NAND_CMD_READ0;
+ } else {
+ column -= 256;
+ readcmd = NAND_CMD_READ1;
+ }
+ chip->legacy.cmd_ctrl(chip, readcmd, ctrl);
+ ctrl &= ~NAND_CTRL_CHANGE;
+ }
+ if (command != NAND_CMD_NONE)
+ chip->legacy.cmd_ctrl(chip, command, ctrl);
+
+ /* Address cycle, when necessary */
+ ctrl = NAND_CTRL_ALE | NAND_CTRL_CHANGE;
+ /* Serially input address */
+ if (column != -1) {
+ /* Adjust columns for 16 bit buswidth */
+ if (chip->options & NAND_BUSWIDTH_16 &&
+ !nand_opcode_8bits(command))
+ column >>= 1;
+ chip->legacy.cmd_ctrl(chip, column, ctrl);
+ ctrl &= ~NAND_CTRL_CHANGE;
+ }
+ if (page_addr != -1) {
+ chip->legacy.cmd_ctrl(chip, page_addr, ctrl);
+ ctrl &= ~NAND_CTRL_CHANGE;
+ chip->legacy.cmd_ctrl(chip, page_addr >> 8, ctrl);
+ if (chip->options & NAND_ROW_ADDR_3)
+ chip->legacy.cmd_ctrl(chip, page_addr >> 16, ctrl);
+ }
+ chip->legacy.cmd_ctrl(chip, NAND_CMD_NONE,
+ NAND_NCE | NAND_CTRL_CHANGE);
+
+ /*
+ * Program and erase have their own busy handlers status and sequential
+ * in needs no delay
+ */
+ switch (command) {
+
+ case NAND_CMD_NONE:
+ case NAND_CMD_PAGEPROG:
+ case NAND_CMD_ERASE1:
+ case NAND_CMD_ERASE2:
+ case NAND_CMD_SEQIN:
+ case NAND_CMD_STATUS:
+ case NAND_CMD_READID:
+ case NAND_CMD_SET_FEATURES:
+ return;
+
+ case NAND_CMD_RESET:
+ if (chip->legacy.dev_ready)
+ break;
+ udelay(chip->legacy.chip_delay);
+ chip->legacy.cmd_ctrl(chip, NAND_CMD_STATUS,
+ NAND_CTRL_CLE | NAND_CTRL_CHANGE);
+ chip->legacy.cmd_ctrl(chip, NAND_CMD_NONE,
+ NAND_NCE | NAND_CTRL_CHANGE);
+ /* EZ-NAND can take upto 250ms as per ONFi v4.0 */
+ nand_wait_status_ready(mtd, 250);
+ return;
+
+ /* This applies to read commands */
+ case NAND_CMD_READ0:
+ /*
+ * READ0 is sometimes used to exit GET STATUS mode. When this
+ * is the case no address cycles are requested, and we can use
+ * this information to detect that we should not wait for the
+ * device to be ready.
+ */
+ if (column == -1 && page_addr == -1)
+ return;
+
+ default:
+ /*
+ * If we don't have access to the busy pin, we apply the given
+ * command delay
+ */
+ if (!chip->legacy.dev_ready) {
+ udelay(chip->legacy.chip_delay);
+ return;
+ }
+ }
+ /*
+ * Apply this short delay always to ensure that we do wait tWB in
+ * any case on any machine.
+ */
+ ndelay(100);
+
+ nand_wait_ready(chip);
+}
+
+static void nand_ccs_delay(struct nand_chip *chip)
+{
+ /*
+ * The controller already takes care of waiting for tCCS when the RNDIN
+ * or RNDOUT command is sent, return directly.
+ */
+ if (!(chip->options & NAND_WAIT_TCCS))
+ return;
+
+ /*
+ * Wait tCCS_min if it is correctly defined, otherwise wait 500ns
+ * (which should be safe for all NANDs).
+ */
+ if (chip->setup_data_interface)
+ ndelay(chip->data_interface.timings.sdr.tCCS_min / 1000);
+ else
+ ndelay(500);
+}
+
+/**
+ * nand_command_lp - [DEFAULT] Send command to NAND large page device
+ * @chip: NAND chip object
+ * @command: the command to be sent
+ * @column: the column address for this command, -1 if none
+ * @page_addr: the page address for this command, -1 if none
+ *
+ * Send command to NAND device. This is the version for the new large page
+ * devices. We don't have the separate regions as we have in the small page
+ * devices. We must emulate NAND_CMD_READOOB to keep the code compatible.
+ */
+static void nand_command_lp(struct nand_chip *chip, unsigned int command,
+ int column, int page_addr)
+{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
+ /* Emulate NAND_CMD_READOOB */
+ if (command == NAND_CMD_READOOB) {
+ column += mtd->writesize;
+ command = NAND_CMD_READ0;
+ }
+
+ /* Command latch cycle */
+ if (command != NAND_CMD_NONE)
+ chip->legacy.cmd_ctrl(chip, command,
+ NAND_NCE | NAND_CLE | NAND_CTRL_CHANGE);
+
+ if (column != -1 || page_addr != -1) {
+ int ctrl = NAND_CTRL_CHANGE | NAND_NCE | NAND_ALE;
+
+ /* Serially input address */
+ if (column != -1) {
+ /* Adjust columns for 16 bit buswidth */
+ if (chip->options & NAND_BUSWIDTH_16 &&
+ !nand_opcode_8bits(command))
+ column >>= 1;
+ chip->legacy.cmd_ctrl(chip, column, ctrl);
+ ctrl &= ~NAND_CTRL_CHANGE;
+
+ /* Only output a single addr cycle for 8bits opcodes. */
+ if (!nand_opcode_8bits(command))
+ chip->legacy.cmd_ctrl(chip, column >> 8, ctrl);
+ }
+ if (page_addr != -1) {
+ chip->legacy.cmd_ctrl(chip, page_addr, ctrl);
+ chip->legacy.cmd_ctrl(chip, page_addr >> 8,
+ NAND_NCE | NAND_ALE);
+ if (chip->options & NAND_ROW_ADDR_3)
+ chip->legacy.cmd_ctrl(chip, page_addr >> 16,
+ NAND_NCE | NAND_ALE);
+ }
+ }
+ chip->legacy.cmd_ctrl(chip, NAND_CMD_NONE,
+ NAND_NCE | NAND_CTRL_CHANGE);
+
+ /*
+ * Program and erase have their own busy handlers status, sequential
+ * in and status need no delay.
+ */
+ switch (command) {
+
+ case NAND_CMD_NONE:
+ case NAND_CMD_CACHEDPROG:
+ case NAND_CMD_PAGEPROG:
+ case NAND_CMD_ERASE1:
+ case NAND_CMD_ERASE2:
+ case NAND_CMD_SEQIN:
+ case NAND_CMD_STATUS:
+ case NAND_CMD_READID:
+ case NAND_CMD_SET_FEATURES:
+ return;
+
+ case NAND_CMD_RNDIN:
+ nand_ccs_delay(chip);
+ return;
+
+ case NAND_CMD_RESET:
+ if (chip->legacy.dev_ready)
+ break;
+ udelay(chip->legacy.chip_delay);
+ chip->legacy.cmd_ctrl(chip, NAND_CMD_STATUS,
+ NAND_NCE | NAND_CLE | NAND_CTRL_CHANGE);
+ chip->legacy.cmd_ctrl(chip, NAND_CMD_NONE,
+ NAND_NCE | NAND_CTRL_CHANGE);
+ /* EZ-NAND can take upto 250ms as per ONFi v4.0 */
+ nand_wait_status_ready(mtd, 250);
+ return;
+
+ case NAND_CMD_RNDOUT:
+ /* No ready / busy check necessary */
+ chip->legacy.cmd_ctrl(chip, NAND_CMD_RNDOUTSTART,
+ NAND_NCE | NAND_CLE | NAND_CTRL_CHANGE);
+ chip->legacy.cmd_ctrl(chip, NAND_CMD_NONE,
+ NAND_NCE | NAND_CTRL_CHANGE);
+
+ nand_ccs_delay(chip);
+ return;
+
+ case NAND_CMD_READ0:
+ /*
+ * READ0 is sometimes used to exit GET STATUS mode. When this
+ * is the case no address cycles are requested, and we can use
+ * this information to detect that READSTART should not be
+ * issued.
+ */
+ if (column == -1 && page_addr == -1)
+ return;
+
+ chip->legacy.cmd_ctrl(chip, NAND_CMD_READSTART,
+ NAND_NCE | NAND_CLE | NAND_CTRL_CHANGE);
+ chip->legacy.cmd_ctrl(chip, NAND_CMD_NONE,
+ NAND_NCE | NAND_CTRL_CHANGE);
+
+ /* This applies to read commands */
+ default:
+ /*
+ * If we don't have access to the busy pin, we apply the given
+ * command delay.
+ */
+ if (!chip->legacy.dev_ready) {
+ udelay(chip->legacy.chip_delay);
+ return;
+ }
+ }
+
+ /*
+ * Apply this short delay always to ensure that we do wait tWB in
+ * any case on any machine.
+ */
+ ndelay(100);
+
+ nand_wait_ready(chip);
+}
+
+/**
+ * nand_get_set_features_notsupp - set/get features stub returning -ENOTSUPP
+ * @chip: nand chip info structure
+ * @addr: feature address.
+ * @subfeature_param: the subfeature parameters, a four bytes array.
+ *
+ * Should be used by NAND controller drivers that do not support the SET/GET
+ * FEATURES operations.
+ */
+int nand_get_set_features_notsupp(struct nand_chip *chip, int addr,
+ u8 *subfeature_param)
+{
+ return -ENOTSUPP;
+}
+EXPORT_SYMBOL(nand_get_set_features_notsupp);
+
+/**
+ * nand_wait - [DEFAULT] wait until the command is done
+ * @mtd: MTD device structure
+ * @chip: NAND chip structure
+ *
+ * Wait for command done. This applies to erase and program only.
+ */
+static int nand_wait(struct nand_chip *chip)
+{
+
+ unsigned long timeo = 400;
+ u8 status;
+ int ret;
+
+ /*
+ * Apply this short delay always to ensure that we do wait tWB in any
+ * case on any machine.
+ */
+ ndelay(100);
+
+ ret = nand_status_op(chip, NULL);
+ if (ret)
+ return ret;
+
+ if (in_interrupt() || oops_in_progress)
+ panic_nand_wait(chip, timeo);
+ else {
+ timeo = jiffies + msecs_to_jiffies(timeo);
+ do {
+ if (chip->legacy.dev_ready) {
+ if (chip->legacy.dev_ready(chip))
+ break;
+ } else {
+ ret = nand_read_data_op(chip, &status,
+ sizeof(status), true);
+ if (ret)
+ return ret;
+
+ if (status & NAND_STATUS_READY)
+ break;
+ }
+ cond_resched();
+ } while (time_before(jiffies, timeo));
+ }
+
+ ret = nand_read_data_op(chip, &status, sizeof(status), true);
+ if (ret)
+ return ret;
+
+ /* This can happen if in case of timeout or buggy dev_ready */
+ WARN_ON(!(status & NAND_STATUS_READY));
+ return status;
+}
+
+void nand_legacy_set_defaults(struct nand_chip *chip)
+{
+ unsigned int busw = chip->options & NAND_BUSWIDTH_16;
+
+ if (chip->exec_op)
+ return;
+
+ /* check for proper chip_delay setup, set 20us if not */
+ if (!chip->legacy.chip_delay)
+ chip->legacy.chip_delay = 20;
+
+ /* check, if a user supplied command function given */
+ if (!chip->legacy.cmdfunc && !chip->exec_op)
+ chip->legacy.cmdfunc = nand_command;
+
+ /* check, if a user supplied wait function given */
+ if (chip->legacy.waitfunc == NULL)
+ chip->legacy.waitfunc = nand_wait;
+
+ if (!chip->select_chip)
+ chip->select_chip = nand_select_chip;
+
+ /* If called twice, pointers that depend on busw may need to be reset */
+ if (!chip->legacy.read_byte || chip->legacy.read_byte == nand_read_byte)
+ chip->legacy.read_byte = busw ? nand_read_byte16 : nand_read_byte;
+ if (!chip->legacy.write_buf || chip->legacy.write_buf == nand_write_buf)
+ chip->legacy.write_buf = busw ? nand_write_buf16 : nand_write_buf;
+ if (!chip->legacy.write_byte || chip->legacy.write_byte == nand_write_byte)
+ chip->legacy.write_byte = busw ? nand_write_byte16 : nand_write_byte;
+ if (!chip->legacy.read_buf || chip->legacy.read_buf == nand_read_buf)
+ chip->legacy.read_buf = busw ? nand_read_buf16 : nand_read_buf;
+}
+
+void nand_legacy_adjust_cmdfunc(struct nand_chip *chip)
+{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
+ /* Do not replace user supplied command function! */
+ if (mtd->writesize > 512 && chip->legacy.cmdfunc == nand_command)
+ chip->legacy.cmdfunc = nand_command_lp;
+}
+
+int nand_legacy_check_hooks(struct nand_chip *chip)
+{
+ /*
+ * ->legacy.cmdfunc() is legacy and will only be used if ->exec_op() is
+ * not populated.
+ */
+ if (chip->exec_op)
+ return 0;
+
+ /*
+ * Default functions assigned for ->legacy.cmdfunc() and
+ * ->select_chip() both expect ->legacy.cmd_ctrl() to be populated.
+ */
+ if ((!chip->legacy.cmdfunc || !chip->select_chip) &&
+ !chip->legacy.cmd_ctrl) {
+ pr_err("->legacy.cmd_ctrl() should be provided\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
* GNU General Public License for more details.
*/
-#include <linux/mtd/rawnand.h>
+#include "internals.h"
/*
* Macronix AC series does not support using SET/GET_FEATURES to change
* GNU General Public License for more details.
*/
-#include <linux/mtd/rawnand.h>
#include <linux/slab.h>
+#include "internals.h"
+
/*
* Special Micron status bit 3 indicates that the block has been
* corrected by on-die ECC and should be rewritten.
struct micron_on_die_ecc ecc;
};
-static int micron_nand_setup_read_retry(struct mtd_info *mtd, int retry_mode)
+static int micron_nand_setup_read_retry(struct nand_chip *chip, int retry_mode)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
u8 feature[ONFI_SUBFEATURE_PARAM_LEN] = {retry_mode};
return nand_set_features(chip, ONFI_FEATURE_ADDR_READ_RETRY, feature);
}
static int
-micron_nand_read_page_on_die_ecc(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required,
- int page)
+micron_nand_read_page_on_die_ecc(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
u8 status;
int ret, max_bitflips = 0;
}
static int
-micron_nand_write_page_on_die_ecc(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required,
- int page)
+micron_nand_write_page_on_die_ecc(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
int ret;
if (ret)
return ret;
- ret = nand_write_page_raw(mtd, chip, buf, oob_required, page);
+ ret = nand_write_page_raw(chip, buf, oob_required, page);
micron_nand_on_die_ecc_setup(chip, false);
return ret;
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2000 Steven J. Hill (sjhill@realitydiluted.com)
+ * 2002-2006 Thomas Gleixner (tglx@linutronix.de)
+ *
+ * Credits:
+ * David Woodhouse for adding multichip support
+ *
+ * Aleph One Ltd. and Toby Churchill Ltd. for supporting the
+ * rework for 2K page size chips
+ *
+ * This file contains all ONFI helpers.
+ */
+
+#include <linux/slab.h>
+
+#include "internals.h"
+
+u16 onfi_crc16(u16 crc, u8 const *p, size_t len)
+{
+ int i;
+ while (len--) {
+ crc ^= *p++ << 8;
+ for (i = 0; i < 8; i++)
+ crc = (crc << 1) ^ ((crc & 0x8000) ? 0x8005 : 0);
+ }
+
+ return crc;
+}
+
+/* Parse the Extended Parameter Page. */
+static int nand_flash_detect_ext_param_page(struct nand_chip *chip,
+ struct nand_onfi_params *p)
+{
+ struct onfi_ext_param_page *ep;
+ struct onfi_ext_section *s;
+ struct onfi_ext_ecc_info *ecc;
+ uint8_t *cursor;
+ int ret;
+ int len;
+ int i;
+
+ len = le16_to_cpu(p->ext_param_page_length) * 16;
+ ep = kmalloc(len, GFP_KERNEL);
+ if (!ep)
+ return -ENOMEM;
+
+ /* Send our own NAND_CMD_PARAM. */
+ ret = nand_read_param_page_op(chip, 0, NULL, 0);
+ if (ret)
+ goto ext_out;
+
+ /* Use the Change Read Column command to skip the ONFI param pages. */
+ ret = nand_change_read_column_op(chip,
+ sizeof(*p) * p->num_of_param_pages,
+ ep, len, true);
+ if (ret)
+ goto ext_out;
+
+ ret = -EINVAL;
+ if ((onfi_crc16(ONFI_CRC_BASE, ((uint8_t *)ep) + 2, len - 2)
+ != le16_to_cpu(ep->crc))) {
+ pr_debug("fail in the CRC.\n");
+ goto ext_out;
+ }
+
+ /*
+ * Check the signature.
+ * Do not strictly follow the ONFI spec, maybe changed in future.
+ */
+ if (strncmp(ep->sig, "EPPS", 4)) {
+ pr_debug("The signature is invalid.\n");
+ goto ext_out;
+ }
+
+ /* find the ECC section. */
+ cursor = (uint8_t *)(ep + 1);
+ for (i = 0; i < ONFI_EXT_SECTION_MAX; i++) {
+ s = ep->sections + i;
+ if (s->type == ONFI_SECTION_TYPE_2)
+ break;
+ cursor += s->length * 16;
+ }
+ if (i == ONFI_EXT_SECTION_MAX) {
+ pr_debug("We can not find the ECC section.\n");
+ goto ext_out;
+ }
+
+ /* get the info we want. */
+ ecc = (struct onfi_ext_ecc_info *)cursor;
+
+ if (!ecc->codeword_size) {
+ pr_debug("Invalid codeword size\n");
+ goto ext_out;
+ }
+
+ chip->ecc_strength_ds = ecc->ecc_bits;
+ chip->ecc_step_ds = 1 << ecc->codeword_size;
+ ret = 0;
+
+ext_out:
+ kfree(ep);
+ return ret;
+}
+
+/*
+ * Recover data with bit-wise majority
+ */
+static void nand_bit_wise_majority(const void **srcbufs,
+ unsigned int nsrcbufs,
+ void *dstbuf,
+ unsigned int bufsize)
+{
+ int i, j, k;
+
+ for (i = 0; i < bufsize; i++) {
+ u8 val = 0;
+
+ for (j = 0; j < 8; j++) {
+ unsigned int cnt = 0;
+
+ for (k = 0; k < nsrcbufs; k++) {
+ const u8 *srcbuf = srcbufs[k];
+
+ if (srcbuf[i] & BIT(j))
+ cnt++;
+ }
+
+ if (cnt > nsrcbufs / 2)
+ val |= BIT(j);
+ }
+
+ ((u8 *)dstbuf)[i] = val;
+ }
+}
+
+/*
+ * Check if the NAND chip is ONFI compliant, returns 1 if it is, 0 otherwise.
+ */
+int nand_onfi_detect(struct nand_chip *chip)
+{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+ struct nand_onfi_params *p;
+ struct onfi_params *onfi;
+ int onfi_version = 0;
+ char id[4];
+ int i, ret, val;
+
+ /* Try ONFI for unknown chip or LP */
+ ret = nand_readid_op(chip, 0x20, id, sizeof(id));
+ if (ret || strncmp(id, "ONFI", 4))
+ return 0;
+
+ /* ONFI chip: allocate a buffer to hold its parameter page */
+ p = kzalloc((sizeof(*p) * 3), GFP_KERNEL);
+ if (!p)
+ return -ENOMEM;
+
+ ret = nand_read_param_page_op(chip, 0, NULL, 0);
+ if (ret) {
+ ret = 0;
+ goto free_onfi_param_page;
+ }
+
+ for (i = 0; i < 3; i++) {
+ ret = nand_read_data_op(chip, &p[i], sizeof(*p), true);
+ if (ret) {
+ ret = 0;
+ goto free_onfi_param_page;
+ }
+
+ if (onfi_crc16(ONFI_CRC_BASE, (u8 *)&p[i], 254) ==
+ le16_to_cpu(p->crc)) {
+ if (i)
+ memcpy(p, &p[i], sizeof(*p));
+ break;
+ }
+ }
+
+ if (i == 3) {
+ const void *srcbufs[3] = {p, p + 1, p + 2};
+
+ pr_warn("Could not find a valid ONFI parameter page, trying bit-wise majority to recover it\n");
+ nand_bit_wise_majority(srcbufs, ARRAY_SIZE(srcbufs), p,
+ sizeof(*p));
+
+ if (onfi_crc16(ONFI_CRC_BASE, (u8 *)p, 254) !=
+ le16_to_cpu(p->crc)) {
+ pr_err("ONFI parameter recovery failed, aborting\n");
+ goto free_onfi_param_page;
+ }
+ }
+
+ if (chip->manufacturer.desc && chip->manufacturer.desc->ops &&
+ chip->manufacturer.desc->ops->fixup_onfi_param_page)
+ chip->manufacturer.desc->ops->fixup_onfi_param_page(chip, p);
+
+ /* Check version */
+ val = le16_to_cpu(p->revision);
+ if (val & ONFI_VERSION_2_3)
+ onfi_version = 23;
+ else if (val & ONFI_VERSION_2_2)
+ onfi_version = 22;
+ else if (val & ONFI_VERSION_2_1)
+ onfi_version = 21;
+ else if (val & ONFI_VERSION_2_0)
+ onfi_version = 20;
+ else if (val & ONFI_VERSION_1_0)
+ onfi_version = 10;
+
+ if (!onfi_version) {
+ pr_info("unsupported ONFI version: %d\n", val);
+ goto free_onfi_param_page;
+ }
+
+ sanitize_string(p->manufacturer, sizeof(p->manufacturer));
+ sanitize_string(p->model, sizeof(p->model));
+ chip->parameters.model = kstrdup(p->model, GFP_KERNEL);
+ if (!chip->parameters.model) {
+ ret = -ENOMEM;
+ goto free_onfi_param_page;
+ }
+
+ mtd->writesize = le32_to_cpu(p->byte_per_page);
+
+ /*
+ * pages_per_block and blocks_per_lun may not be a power-of-2 size
+ * (don't ask me who thought of this...). MTD assumes that these
+ * dimensions will be power-of-2, so just truncate the remaining area.
+ */
+ mtd->erasesize = 1 << (fls(le32_to_cpu(p->pages_per_block)) - 1);
+ mtd->erasesize *= mtd->writesize;
+
+ mtd->oobsize = le16_to_cpu(p->spare_bytes_per_page);
+
+ /* See erasesize comment */
+ chip->chipsize = 1 << (fls(le32_to_cpu(p->blocks_per_lun)) - 1);
+ chip->chipsize *= (uint64_t)mtd->erasesize * p->lun_count;
+ chip->bits_per_cell = p->bits_per_cell;
+
+ chip->max_bb_per_die = le16_to_cpu(p->bb_per_lun);
+ chip->blocks_per_die = le32_to_cpu(p->blocks_per_lun);
+
+ if (le16_to_cpu(p->features) & ONFI_FEATURE_16_BIT_BUS)
+ chip->options |= NAND_BUSWIDTH_16;
+
+ if (p->ecc_bits != 0xff) {
+ chip->ecc_strength_ds = p->ecc_bits;
+ chip->ecc_step_ds = 512;
+ } else if (onfi_version >= 21 &&
+ (le16_to_cpu(p->features) & ONFI_FEATURE_EXT_PARAM_PAGE)) {
+
+ /*
+ * The nand_flash_detect_ext_param_page() uses the
+ * Change Read Column command which maybe not supported
+ * by the chip->legacy.cmdfunc. So try to update the
+ * chip->legacy.cmdfunc now. We do not replace user supplied
+ * command function.
+ */
+ nand_legacy_adjust_cmdfunc(chip);
+
+ /* The Extended Parameter Page is supported since ONFI 2.1. */
+ if (nand_flash_detect_ext_param_page(chip, p))
+ pr_warn("Failed to detect ONFI extended param page\n");
+ } else {
+ pr_warn("Could not retrieve ONFI ECC requirements\n");
+ }
+
+ /* Save some parameters from the parameter page for future use */
+ if (le16_to_cpu(p->opt_cmd) & ONFI_OPT_CMD_SET_GET_FEATURES) {
+ chip->parameters.supports_set_get_features = true;
+ bitmap_set(chip->parameters.get_feature_list,
+ ONFI_FEATURE_ADDR_TIMING_MODE, 1);
+ bitmap_set(chip->parameters.set_feature_list,
+ ONFI_FEATURE_ADDR_TIMING_MODE, 1);
+ }
+
+ onfi = kzalloc(sizeof(*onfi), GFP_KERNEL);
+ if (!onfi) {
+ ret = -ENOMEM;
+ goto free_model;
+ }
+
+ onfi->version = onfi_version;
+ onfi->tPROG = le16_to_cpu(p->t_prog);
+ onfi->tBERS = le16_to_cpu(p->t_bers);
+ onfi->tR = le16_to_cpu(p->t_r);
+ onfi->tCCS = le16_to_cpu(p->t_ccs);
+ onfi->async_timing_mode = le16_to_cpu(p->async_timing_mode);
+ onfi->vendor_revision = le16_to_cpu(p->vendor_revision);
+ memcpy(onfi->vendor, p->vendor, sizeof(p->vendor));
+ chip->parameters.onfi = onfi;
+
+ /* Identification done, free the full ONFI parameter page and exit */
+ kfree(p);
+
+ return 1;
+
+free_model:
+ kfree(chip->parameters.model);
+free_onfi_param_page:
+ kfree(p);
+
+ return ret;
+}
* GNU General Public License for more details.
*/
-#include <linux/mtd/rawnand.h>
+#include "internals.h"
static void samsung_nand_decode_id(struct nand_chip *chip)
{
#include <linux/kernel.h>
#include <linux/err.h>
#include <linux/export.h>
-#include <linux/mtd/rawnand.h>
+
+#include "internals.h"
#define ONFI_DYN_TIMING_MAX U16_MAX
},
};
-/**
- * onfi_async_timing_mode_to_sdr_timings - [NAND Interface] Retrieve NAND
- * timings according to the given ONFI timing mode
- * @mode: ONFI timing mode
- */
-const struct nand_sdr_timings *onfi_async_timing_mode_to_sdr_timings(int mode)
-{
- if (mode < 0 || mode >= ARRAY_SIZE(onfi_sdr_timings))
- return ERR_PTR(-EINVAL);
-
- return &onfi_sdr_timings[mode].timings.sdr;
-}
-EXPORT_SYMBOL(onfi_async_timing_mode_to_sdr_timings);
-
/**
* onfi_fill_data_interface - [NAND Interface] Initialize a data interface from
* given ONFI mode
return 0;
}
-EXPORT_SYMBOL(onfi_fill_data_interface);
* GNU General Public License for more details.
*/
-#include <linux/mtd/rawnand.h>
+#include "internals.h"
+
+/* Bit for detecting BENAND */
+#define TOSHIBA_NAND_ID4_IS_BENAND BIT(7)
+
+/* Recommended to rewrite for BENAND */
+#define TOSHIBA_NAND_STATUS_REWRITE_RECOMMENDED BIT(3)
+
+static int toshiba_nand_benand_eccstatus(struct nand_chip *chip)
+{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+ int ret;
+ unsigned int max_bitflips = 0;
+ u8 status;
+
+ /* Check Status */
+ ret = nand_status_op(chip, &status);
+ if (ret)
+ return ret;
+
+ if (status & NAND_STATUS_FAIL) {
+ /* uncorrected */
+ mtd->ecc_stats.failed++;
+ } else if (status & TOSHIBA_NAND_STATUS_REWRITE_RECOMMENDED) {
+ /* corrected */
+ max_bitflips = mtd->bitflip_threshold;
+ mtd->ecc_stats.corrected += max_bitflips;
+ }
+
+ return max_bitflips;
+}
+
+static int
+toshiba_nand_read_page_benand(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
+{
+ int ret;
+
+ ret = nand_read_page_raw(chip, buf, oob_required, page);
+ if (ret)
+ return ret;
+
+ return toshiba_nand_benand_eccstatus(chip);
+}
+
+static int
+toshiba_nand_read_subpage_benand(struct nand_chip *chip, uint32_t data_offs,
+ uint32_t readlen, uint8_t *bufpoi, int page)
+{
+ int ret;
+
+ ret = nand_read_page_op(chip, page, data_offs,
+ bufpoi + data_offs, readlen);
+ if (ret)
+ return ret;
+
+ return toshiba_nand_benand_eccstatus(chip);
+}
+
+static void toshiba_nand_benand_init(struct nand_chip *chip)
+{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
+ /*
+ * On BENAND, the entire OOB region can be used by the MTD user.
+ * The calculated ECC bytes are stored into other isolated
+ * area which is not accessible to users.
+ * This is why chip->ecc.bytes = 0.
+ */
+ chip->ecc.bytes = 0;
+ chip->ecc.size = 512;
+ chip->ecc.strength = 8;
+ chip->ecc.read_page = toshiba_nand_read_page_benand;
+ chip->ecc.read_subpage = toshiba_nand_read_subpage_benand;
+ chip->ecc.write_page = nand_write_page_raw;
+ chip->ecc.read_page_raw = nand_read_page_raw_notsupp;
+ chip->ecc.write_page_raw = nand_write_page_raw_notsupp;
+
+ chip->options |= NAND_SUBPAGE_READ;
+
+ mtd_set_ooblayout(mtd, &nand_ooblayout_lp_ops);
+}
static void toshiba_nand_decode_id(struct nand_chip *chip)
{
if (nand_is_slc(chip))
chip->bbt_options |= NAND_BBT_SCAN2NDPAGE;
+ /* Check that chip is BENAND and ECC mode is on-die */
+ if (nand_is_slc(chip) && chip->ecc.mode == NAND_ECC_ON_DIE &&
+ chip->id.data[4] & TOSHIBA_NAND_ID4_IS_BENAND)
+ toshiba_nand_benand_init(chip);
+
return 0;
}
}
/* Force mtd to not do delays */
- chip->chip_delay = 0;
+ chip->legacy.chip_delay = 0;
/* Initialize the NAND flash parameters */
ns->busw = chip->options & NAND_BUSWIDTH_16 ? 16 : 8;
}
}
-static u_char ns_nand_read_byte(struct mtd_info *mtd)
+static u_char ns_nand_read_byte(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct nandsim *ns = nand_get_controller_data(chip);
u_char outb = 0x00;
return outb;
}
-static void ns_nand_write_byte(struct mtd_info *mtd, u_char byte)
+static void ns_nand_write_byte(struct nand_chip *chip, u_char byte)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct nandsim *ns = nand_get_controller_data(chip);
/* Sanity and correctness checks */
return;
}
-static void ns_hwcontrol(struct mtd_info *mtd, int cmd, unsigned int bitmask)
+static void ns_hwcontrol(struct nand_chip *chip, int cmd, unsigned int bitmask)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct nandsim *ns = nand_get_controller_data(chip);
ns->lines.cle = bitmask & NAND_CLE ? 1 : 0;
ns->lines.ce = bitmask & NAND_NCE ? 1 : 0;
if (cmd != NAND_CMD_NONE)
- ns_nand_write_byte(mtd, cmd);
+ ns_nand_write_byte(chip, cmd);
}
-static int ns_device_ready(struct mtd_info *mtd)
+static int ns_device_ready(struct nand_chip *chip)
{
NS_DBG("device_ready\n");
return 1;
}
-static uint16_t ns_nand_read_word(struct mtd_info *mtd)
+static void ns_nand_write_buf(struct nand_chip *chip, const u_char *buf,
+ int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
- NS_DBG("read_word\n");
-
- return chip->read_byte(mtd) | (chip->read_byte(mtd) << 8);
-}
-
-static void ns_nand_write_buf(struct mtd_info *mtd, const u_char *buf, int len)
-{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct nandsim *ns = nand_get_controller_data(chip);
/* Check that chip is expecting data input */
}
}
-static void ns_nand_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void ns_nand_read_buf(struct nand_chip *chip, u_char *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct nandsim *ns = nand_get_controller_data(chip);
/* Sanity and correctness checks */
int i;
for (i = 0; i < len; i++)
- buf[i] = mtd_to_nand(mtd)->read_byte(mtd);
+ buf[i] = chip->legacy.read_byte(chip);
return;
}
/*
* Register simulator's callbacks.
*/
- chip->cmd_ctrl = ns_hwcontrol;
- chip->read_byte = ns_nand_read_byte;
- chip->dev_ready = ns_device_ready;
- chip->write_buf = ns_nand_write_buf;
- chip->read_buf = ns_nand_read_buf;
- chip->read_word = ns_nand_read_word;
+ chip->legacy.cmd_ctrl = ns_hwcontrol;
+ chip->legacy.read_byte = ns_nand_read_byte;
+ chip->legacy.dev_ready = ns_device_ready;
+ chip->legacy.write_buf = ns_nand_write_buf;
+ chip->legacy.read_buf = ns_nand_read_buf;
chip->ecc.mode = NAND_ECC_SOFT;
chip->ecc.algo = NAND_ECC_HAMMING;
/* The NAND_SKIP_BBTSCAN option is necessary for 'overridesize' */
goto error;
chip->dummy_controller.ops = &ns_controller_ops;
- retval = nand_scan(nsmtd, 1);
+ retval = nand_scan(chip, 1);
if (retval) {
NS_ERR("Could not scan NAND Simulator device\n");
goto error;
err_exit:
free_nandsim(nand);
- nand_release(nsmtd);
+ nand_release(chip);
for (i = 0;i < ARRAY_SIZE(nand->partitions); ++i)
kfree(nand->partitions[i].name);
error:
int i;
free_nandsim(ns); /* Free nandsim private resources */
- nand_release(nsmtd); /* Unregister driver */
+ nand_release(chip); /* Unregister driver */
for (i = 0;i < ARRAY_SIZE(ns->partitions); ++i)
kfree(ns->partitions[i].name);
kfree(mtd_to_nand(nsmtd)); /* Free other structures */
static struct ndfc_controller ndfc_ctrl[NDFC_MAX_CS];
-static void ndfc_select_chip(struct mtd_info *mtd, int chip)
+static void ndfc_select_chip(struct nand_chip *nchip, int chip)
{
uint32_t ccr;
- struct nand_chip *nchip = mtd_to_nand(mtd);
struct ndfc_controller *ndfc = nand_get_controller_data(nchip);
ccr = in_be32(ndfc->ndfcbase + NDFC_CCR);
out_be32(ndfc->ndfcbase + NDFC_CCR, ccr);
}
-static void ndfc_hwcontrol(struct mtd_info *mtd, int cmd, unsigned int ctrl)
+static void ndfc_hwcontrol(struct nand_chip *chip, int cmd, unsigned int ctrl)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct ndfc_controller *ndfc = nand_get_controller_data(chip);
if (cmd == NAND_CMD_NONE)
writel(cmd & 0xFF, ndfc->ndfcbase + NDFC_ALE);
}
-static int ndfc_ready(struct mtd_info *mtd)
+static int ndfc_ready(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct ndfc_controller *ndfc = nand_get_controller_data(chip);
return in_be32(ndfc->ndfcbase + NDFC_STAT) & NDFC_STAT_IS_READY;
}
-static void ndfc_enable_hwecc(struct mtd_info *mtd, int mode)
+static void ndfc_enable_hwecc(struct nand_chip *chip, int mode)
{
uint32_t ccr;
- struct nand_chip *chip = mtd_to_nand(mtd);
struct ndfc_controller *ndfc = nand_get_controller_data(chip);
ccr = in_be32(ndfc->ndfcbase + NDFC_CCR);
wmb();
}
-static int ndfc_calculate_ecc(struct mtd_info *mtd,
+static int ndfc_calculate_ecc(struct nand_chip *chip,
const u_char *dat, u_char *ecc_code)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct ndfc_controller *ndfc = nand_get_controller_data(chip);
uint32_t ecc;
uint8_t *p = (uint8_t *)&ecc;
* functions. No further checking, as nand_base will always read/write
* page aligned.
*/
-static void ndfc_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void ndfc_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct ndfc_controller *ndfc = nand_get_controller_data(chip);
uint32_t *p = (uint32_t *) buf;
*p++ = in_be32(ndfc->ndfcbase + NDFC_DATA);
}
-static void ndfc_write_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
+static void ndfc_write_buf(struct nand_chip *chip, const uint8_t *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct ndfc_controller *ndfc = nand_get_controller_data(chip);
uint32_t *p = (uint32_t *) buf;
struct mtd_info *mtd = nand_to_mtd(chip);
int ret;
- chip->IO_ADDR_R = ndfc->ndfcbase + NDFC_DATA;
- chip->IO_ADDR_W = ndfc->ndfcbase + NDFC_DATA;
- chip->cmd_ctrl = ndfc_hwcontrol;
- chip->dev_ready = ndfc_ready;
+ chip->legacy.IO_ADDR_R = ndfc->ndfcbase + NDFC_DATA;
+ chip->legacy.IO_ADDR_W = ndfc->ndfcbase + NDFC_DATA;
+ chip->legacy.cmd_ctrl = ndfc_hwcontrol;
+ chip->legacy.dev_ready = ndfc_ready;
chip->select_chip = ndfc_select_chip;
- chip->chip_delay = 50;
+ chip->legacy.chip_delay = 50;
chip->controller = &ndfc->ndfc_control;
- chip->read_buf = ndfc_read_buf;
- chip->write_buf = ndfc_write_buf;
+ chip->legacy.read_buf = ndfc_read_buf;
+ chip->legacy.write_buf = ndfc_write_buf;
chip->ecc.correct = nand_correct_data;
chip->ecc.hwctl = ndfc_enable_hwecc;
chip->ecc.calculate = ndfc_calculate_ecc;
return -ENODEV;
nand_set_flash_node(chip, flash_np);
- mtd->name = kasprintf(GFP_KERNEL, "%s.%s", dev_name(&ndfc->ofdev->dev),
- flash_np->name);
+ mtd->name = kasprintf(GFP_KERNEL, "%s.%pOFn", dev_name(&ndfc->ofdev->dev),
+ flash_np);
if (!mtd->name) {
ret = -ENOMEM;
goto err;
}
- ret = nand_scan(mtd, 1);
+ ret = nand_scan(chip, 1);
if (ret)
goto err;
struct ndfc_controller *ndfc = dev_get_drvdata(&ofdev->dev);
struct mtd_info *mtd = nand_to_mtd(&ndfc->chip);
- nand_release(mtd);
+ nand_release(&ndfc->chip);
kfree(mtd->name);
return 0;
}
};
-static unsigned char nuc900_nand_read_byte(struct mtd_info *mtd)
+static unsigned char nuc900_nand_read_byte(struct nand_chip *chip)
{
unsigned char ret;
- struct nuc900_nand *nand = mtd_to_nuc900(mtd);
+ struct nuc900_nand *nand = mtd_to_nuc900(nand_to_mtd(chip));
ret = (unsigned char)read_data_reg(nand);
return ret;
}
-static void nuc900_nand_read_buf(struct mtd_info *mtd,
+static void nuc900_nand_read_buf(struct nand_chip *chip,
unsigned char *buf, int len)
{
int i;
- struct nuc900_nand *nand = mtd_to_nuc900(mtd);
+ struct nuc900_nand *nand = mtd_to_nuc900(nand_to_mtd(chip));
for (i = 0; i < len; i++)
buf[i] = (unsigned char)read_data_reg(nand);
}
-static void nuc900_nand_write_buf(struct mtd_info *mtd,
+static void nuc900_nand_write_buf(struct nand_chip *chip,
const unsigned char *buf, int len)
{
int i;
- struct nuc900_nand *nand = mtd_to_nuc900(mtd);
+ struct nuc900_nand *nand = mtd_to_nuc900(nand_to_mtd(chip));
for (i = 0; i < len; i++)
write_data_reg(nand, buf[i]);
return val;
}
-static int nuc900_nand_devready(struct mtd_info *mtd)
+static int nuc900_nand_devready(struct nand_chip *chip)
{
- struct nuc900_nand *nand = mtd_to_nuc900(mtd);
+ struct nuc900_nand *nand = mtd_to_nuc900(nand_to_mtd(chip));
int ready;
ready = (nuc900_check_rb(nand)) ? 1 : 0;
return ready;
}
-static void nuc900_nand_command_lp(struct mtd_info *mtd, unsigned int command,
+static void nuc900_nand_command_lp(struct nand_chip *chip,
+ unsigned int command,
int column, int page_addr)
{
- register struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct nuc900_nand *nand = mtd_to_nuc900(mtd);
if (command == NAND_CMD_READOOB) {
return;
case NAND_CMD_RESET:
- if (chip->dev_ready)
+ if (chip->legacy.dev_ready)
break;
- udelay(chip->chip_delay);
+ udelay(chip->legacy.chip_delay);
write_cmd_reg(nand, NAND_CMD_STATUS);
write_cmd_reg(nand, command);
write_cmd_reg(nand, NAND_CMD_READSTART);
default:
- if (!chip->dev_ready) {
- udelay(chip->chip_delay);
+ if (!chip->legacy.dev_ready) {
+ udelay(chip->legacy.chip_delay);
return;
}
}
* any case on any machine. */
ndelay(100);
- while (!chip->dev_ready(mtd))
+ while (!chip->legacy.dev_ready(chip))
;
}
return -ENOENT;
clk_enable(nuc900_nand->clk);
- chip->cmdfunc = nuc900_nand_command_lp;
- chip->dev_ready = nuc900_nand_devready;
- chip->read_byte = nuc900_nand_read_byte;
- chip->write_buf = nuc900_nand_write_buf;
- chip->read_buf = nuc900_nand_read_buf;
- chip->chip_delay = 50;
+ chip->legacy.cmdfunc = nuc900_nand_command_lp;
+ chip->legacy.dev_ready = nuc900_nand_devready;
+ chip->legacy.read_byte = nuc900_nand_read_byte;
+ chip->legacy.write_buf = nuc900_nand_write_buf;
+ chip->legacy.read_buf = nuc900_nand_read_buf;
+ chip->legacy.chip_delay = 50;
chip->options = 0;
chip->ecc.mode = NAND_ECC_SOFT;
chip->ecc.algo = NAND_ECC_HAMMING;
nuc900_nand_enable(nuc900_nand);
- if (nand_scan(mtd, 1))
+ if (nand_scan(chip, 1))
return -ENXIO;
mtd_device_register(mtd, partitions, ARRAY_SIZE(partitions));
{
struct nuc900_nand *nuc900_nand = platform_get_drvdata(pdev);
- nand_release(nand_to_mtd(&nuc900_nand->chip));
+ nand_release(&nuc900_nand->chip);
clk_disable(nuc900_nand->clk);
return 0;
/**
* omap_hwcontrol - hardware specific access to control-lines
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @cmd: command to device
* @ctrl:
* NAND_NCE: bit 0 -> don't care
*
* NOTE: boards may use different bits for these!!
*/
-static void omap_hwcontrol(struct mtd_info *mtd, int cmd, unsigned int ctrl)
+static void omap_hwcontrol(struct nand_chip *chip, int cmd, unsigned int ctrl)
{
- struct omap_nand_info *info = mtd_to_omap(mtd);
+ struct omap_nand_info *info = mtd_to_omap(nand_to_mtd(chip));
if (cmd != NAND_CMD_NONE) {
if (ctrl & NAND_CLE)
{
struct nand_chip *nand = mtd_to_nand(mtd);
- ioread8_rep(nand->IO_ADDR_R, buf, len);
+ ioread8_rep(nand->legacy.IO_ADDR_R, buf, len);
}
/**
bool status;
while (len--) {
- iowrite8(*p++, info->nand.IO_ADDR_W);
+ iowrite8(*p++, info->nand.legacy.IO_ADDR_W);
/* wait until buffer is available for write */
do {
status = info->ops->nand_writebuffer_empty();
{
struct nand_chip *nand = mtd_to_nand(mtd);
- ioread16_rep(nand->IO_ADDR_R, buf, len / 2);
+ ioread16_rep(nand->legacy.IO_ADDR_R, buf, len / 2);
}
/**
len >>= 1;
while (len--) {
- iowrite16(*p++, info->nand.IO_ADDR_W);
+ iowrite16(*p++, info->nand.legacy.IO_ADDR_W);
/* wait until buffer is available for write */
do {
status = info->ops->nand_writebuffer_empty();
/**
* omap_read_buf_pref - read data from NAND controller into buffer
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @buf: buffer to store date
* @len: number of bytes to read
*/
-static void omap_read_buf_pref(struct mtd_info *mtd, u_char *buf, int len)
+static void omap_read_buf_pref(struct nand_chip *chip, u_char *buf, int len)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct omap_nand_info *info = mtd_to_omap(mtd);
uint32_t r_count = 0;
int ret = 0;
r_count = readl(info->reg.gpmc_prefetch_status);
r_count = PREFETCH_STATUS_FIFO_CNT(r_count);
r_count = r_count >> 2;
- ioread32_rep(info->nand.IO_ADDR_R, p, r_count);
+ ioread32_rep(info->nand.legacy.IO_ADDR_R, p, r_count);
p += r_count;
len -= r_count << 2;
} while (len);
/**
* omap_write_buf_pref - write buffer to NAND controller
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @buf: data buffer
* @len: number of bytes to write
*/
-static void omap_write_buf_pref(struct mtd_info *mtd,
- const u_char *buf, int len)
+static void omap_write_buf_pref(struct nand_chip *chip, const u_char *buf,
+ int len)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct omap_nand_info *info = mtd_to_omap(mtd);
uint32_t w_count = 0;
int i = 0, ret = 0;
/* take care of subpage writes */
if (len % 2 != 0) {
- writeb(*buf, info->nand.IO_ADDR_W);
+ writeb(*buf, info->nand.legacy.IO_ADDR_W);
p = (u16 *)(buf + 1);
len--;
}
w_count = PREFETCH_STATUS_FIFO_CNT(w_count);
w_count = w_count >> 1;
for (i = 0; (i < w_count) && len; i++, len -= 2)
- iowrite16(*p++, info->nand.IO_ADDR_W);
+ iowrite16(*p++, info->nand.legacy.IO_ADDR_W);
}
/* wait for data to flushed-out before reset the prefetch */
tim = 0;
/**
* omap_read_buf_dma_pref - read data from NAND controller into buffer
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @buf: buffer to store date
* @len: number of bytes to read
*/
-static void omap_read_buf_dma_pref(struct mtd_info *mtd, u_char *buf, int len)
+static void omap_read_buf_dma_pref(struct nand_chip *chip, u_char *buf,
+ int len)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
if (len <= mtd->oobsize)
- omap_read_buf_pref(mtd, buf, len);
+ omap_read_buf_pref(chip, buf, len);
else
/* start transfer in DMA mode */
omap_nand_dma_transfer(mtd, buf, len, 0x0);
/**
* omap_write_buf_dma_pref - write buffer to NAND controller
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @buf: data buffer
* @len: number of bytes to write
*/
-static void omap_write_buf_dma_pref(struct mtd_info *mtd,
- const u_char *buf, int len)
+static void omap_write_buf_dma_pref(struct nand_chip *chip, const u_char *buf,
+ int len)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
if (len <= mtd->oobsize)
- omap_write_buf_pref(mtd, buf, len);
+ omap_write_buf_pref(chip, buf, len);
else
/* start transfer in DMA mode */
- omap_nand_dma_transfer(mtd, (u_char *) buf, len, 0x1);
+ omap_nand_dma_transfer(mtd, (u_char *)buf, len, 0x1);
}
/*
bytes = info->buf_len;
else if (!info->buf_len)
bytes = 0;
- iowrite32_rep(info->nand.IO_ADDR_W,
- (u32 *)info->buf, bytes >> 2);
+ iowrite32_rep(info->nand.legacy.IO_ADDR_W, (u32 *)info->buf,
+ bytes >> 2);
info->buf = info->buf + bytes;
info->buf_len -= bytes;
} else {
- ioread32_rep(info->nand.IO_ADDR_R,
- (u32 *)info->buf, bytes >> 2);
+ ioread32_rep(info->nand.legacy.IO_ADDR_R, (u32 *)info->buf,
+ bytes >> 2);
info->buf = info->buf + bytes;
if (this_irq == info->gpmc_irq_count)
/*
* omap_read_buf_irq_pref - read data from NAND controller into buffer
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @buf: buffer to store date
* @len: number of bytes to read
*/
-static void omap_read_buf_irq_pref(struct mtd_info *mtd, u_char *buf, int len)
+static void omap_read_buf_irq_pref(struct nand_chip *chip, u_char *buf,
+ int len)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct omap_nand_info *info = mtd_to_omap(mtd);
int ret = 0;
if (len <= mtd->oobsize) {
- omap_read_buf_pref(mtd, buf, len);
+ omap_read_buf_pref(chip, buf, len);
return;
}
/*
* omap_write_buf_irq_pref - write buffer to NAND controller
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @buf: data buffer
* @len: number of bytes to write
*/
-static void omap_write_buf_irq_pref(struct mtd_info *mtd,
- const u_char *buf, int len)
+static void omap_write_buf_irq_pref(struct nand_chip *chip, const u_char *buf,
+ int len)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct omap_nand_info *info = mtd_to_omap(mtd);
int ret = 0;
unsigned long tim, limit;
u32 val;
if (len <= mtd->oobsize) {
- omap_write_buf_pref(mtd, buf, len);
+ omap_write_buf_pref(chip, buf, len);
return;
}
/**
* omap_correct_data - Compares the ECC read with HW generated ECC
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @dat: page data
* @read_ecc: ecc read from nand flash
* @calc_ecc: ecc read from HW ECC registers
* corrected errors is returned. If uncorrectable errors exist, %-1 is
* returned.
*/
-static int omap_correct_data(struct mtd_info *mtd, u_char *dat,
- u_char *read_ecc, u_char *calc_ecc)
+static int omap_correct_data(struct nand_chip *chip, u_char *dat,
+ u_char *read_ecc, u_char *calc_ecc)
{
- struct omap_nand_info *info = mtd_to_omap(mtd);
+ struct omap_nand_info *info = mtd_to_omap(nand_to_mtd(chip));
int blockCnt = 0, i = 0, ret = 0;
int stat = 0;
/**
* omap_calcuate_ecc - Generate non-inverted ECC bytes.
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @dat: The pointer to data on which ecc is computed
* @ecc_code: The ecc_code buffer
*
* an erased page will produce an ECC mismatch between generated and read
* ECC bytes that has to be dealt with separately.
*/
-static int omap_calculate_ecc(struct mtd_info *mtd, const u_char *dat,
- u_char *ecc_code)
+static int omap_calculate_ecc(struct nand_chip *chip, const u_char *dat,
+ u_char *ecc_code)
{
- struct omap_nand_info *info = mtd_to_omap(mtd);
+ struct omap_nand_info *info = mtd_to_omap(nand_to_mtd(chip));
u32 val;
val = readl(info->reg.gpmc_ecc_config);
* @mtd: MTD device structure
* @mode: Read/Write mode
*/
-static void omap_enable_hwecc(struct mtd_info *mtd, int mode)
+static void omap_enable_hwecc(struct nand_chip *chip, int mode)
{
- struct omap_nand_info *info = mtd_to_omap(mtd);
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct omap_nand_info *info = mtd_to_omap(nand_to_mtd(chip));
unsigned int dev_width = (chip->options & NAND_BUSWIDTH_16) ? 1 : 0;
u32 val;
/**
* omap_wait - wait until the command is done
- * @mtd: MTD device structure
- * @chip: NAND Chip structure
+ * @this: NAND Chip structure
*
* Wait function is called during Program and erase operations and
* the way it is called from MTD layer, we should wait till the NAND
* Erase can take up to 400ms and program up to 20ms according to
* general NAND and SmartMedia specs
*/
-static int omap_wait(struct mtd_info *mtd, struct nand_chip *chip)
+static int omap_wait(struct nand_chip *this)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- struct omap_nand_info *info = mtd_to_omap(mtd);
+ struct omap_nand_info *info = mtd_to_omap(nand_to_mtd(this));
unsigned long timeo = jiffies;
int status, state = this->state;
*
* Returns true if ready and false if busy.
*/
-static int omap_dev_ready(struct mtd_info *mtd)
+static int omap_dev_ready(struct nand_chip *chip)
{
- struct omap_nand_info *info = mtd_to_omap(mtd);
+ struct omap_nand_info *info = mtd_to_omap(nand_to_mtd(chip));
return gpiod_get_value(info->ready_gpiod);
}
* eccsize0 = 0 (no additional protected byte in spare area)
* eccsize1 = 32 (skip 32 nibbles = 16 bytes per sector in spare area)
*/
-static void __maybe_unused omap_enable_hwecc_bch(struct mtd_info *mtd, int mode)
+static void __maybe_unused omap_enable_hwecc_bch(struct nand_chip *chip,
+ int mode)
{
unsigned int bch_type;
unsigned int dev_width, nsectors;
- struct omap_nand_info *info = mtd_to_omap(mtd);
+ struct omap_nand_info *info = mtd_to_omap(nand_to_mtd(chip));
enum omap_ecc ecc_opt = info->ecc_opt;
- struct nand_chip *chip = mtd_to_nand(mtd);
u32 val, wr_mode;
unsigned int ecc_size1, ecc_size0;
/**
* omap_calculate_ecc_bch_sw - ECC generator for sector for SW based correction
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @dat: The pointer to data on which ecc is computed
* @ecc_code: The ecc_code buffer
*
* when SW based correction is required as ECC is required for one sector
* at a time.
*/
-static int omap_calculate_ecc_bch_sw(struct mtd_info *mtd,
+static int omap_calculate_ecc_bch_sw(struct nand_chip *chip,
const u_char *dat, u_char *ecc_calc)
{
- return _omap_calculate_ecc_bch(mtd, dat, ecc_calc, 0);
+ return _omap_calculate_ecc_bch(nand_to_mtd(chip), dat, ecc_calc, 0);
}
/**
/**
* omap_elm_correct_data - corrects page data area in case error reported
- * @mtd: MTD device structure
+ * @chip: NAND chip object
* @data: page data
* @read_ecc: ecc read from nand flash
* @calc_ecc: ecc read from HW ECC registers
* In case of non-zero ecc vector, first filter out erased-pages, and
* then process data via ELM to detect bit-flips.
*/
-static int omap_elm_correct_data(struct mtd_info *mtd, u_char *data,
- u_char *read_ecc, u_char *calc_ecc)
+static int omap_elm_correct_data(struct nand_chip *chip, u_char *data,
+ u_char *read_ecc, u_char *calc_ecc)
{
- struct omap_nand_info *info = mtd_to_omap(mtd);
+ struct omap_nand_info *info = mtd_to_omap(nand_to_mtd(chip));
struct nand_ecc_ctrl *ecc = &info->nand.ecc;
int eccsteps = info->nand.ecc.steps;
int i , j, stat = 0;
/**
* omap_write_page_bch - BCH ecc based write page function for entire page
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: data buffer
* @oob_required: must write chip->oob_poi to OOB
*
* Custom write page method evolved to support multi sector writing in one shot
*/
-static int omap_write_page_bch(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page)
+static int omap_write_page_bch(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int ret;
uint8_t *ecc_calc = chip->ecc.calc_buf;
nand_prog_page_begin_op(chip, page, 0, NULL, 0);
/* Enable GPMC ecc engine */
- chip->ecc.hwctl(mtd, NAND_ECC_WRITE);
+ chip->ecc.hwctl(chip, NAND_ECC_WRITE);
/* Write data */
- chip->write_buf(mtd, buf, mtd->writesize);
+ chip->legacy.write_buf(chip, buf, mtd->writesize);
/* Update ecc vector from GPMC result registers */
omap_calculate_ecc_bch_multi(mtd, buf, &ecc_calc[0]);
return ret;
/* Write ecc vector to OOB area */
- chip->write_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.write_buf(chip, chip->oob_poi, mtd->oobsize);
return nand_prog_page_end_op(chip);
}
/**
* omap_write_subpage_bch - BCH hardware ECC based subpage write
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @offset: column address of subpage within the page
* @data_len: data length
*
* OMAP optimized subpage write method.
*/
-static int omap_write_subpage_bch(struct mtd_info *mtd,
- struct nand_chip *chip, u32 offset,
+static int omap_write_subpage_bch(struct nand_chip *chip, u32 offset,
u32 data_len, const u8 *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
u8 *ecc_calc = chip->ecc.calc_buf;
int ecc_size = chip->ecc.size;
int ecc_bytes = chip->ecc.bytes;
nand_prog_page_begin_op(chip, page, 0, NULL, 0);
/* Enable GPMC ECC engine */
- chip->ecc.hwctl(mtd, NAND_ECC_WRITE);
+ chip->ecc.hwctl(chip, NAND_ECC_WRITE);
/* Write data */
- chip->write_buf(mtd, buf, mtd->writesize);
+ chip->legacy.write_buf(chip, buf, mtd->writesize);
for (step = 0; step < ecc_steps; step++) {
/* mask ECC of un-touched subpages by padding 0xFF */
return ret;
/* write OOB buffer to NAND device */
- chip->write_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.write_buf(chip, chip->oob_poi, mtd->oobsize);
return nand_prog_page_end_op(chip);
}
/**
* omap_read_page_bch - BCH ecc based page read function for entire page
- * @mtd: mtd info structure
* @chip: nand chip info structure
* @buf: buffer to store read data
* @oob_required: caller requires OOB data read to chip->oob_poi
* ecc engine enabled. ecc vector updated after read of OOB data.
* For non error pages ecc vector reported as zero.
*/
-static int omap_read_page_bch(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int omap_read_page_bch(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
uint8_t *ecc_calc = chip->ecc.calc_buf;
uint8_t *ecc_code = chip->ecc.code_buf;
int stat, ret;
nand_read_page_op(chip, page, 0, NULL, 0);
/* Enable GPMC ecc engine */
- chip->ecc.hwctl(mtd, NAND_ECC_READ);
+ chip->ecc.hwctl(chip, NAND_ECC_READ);
/* Read data */
- chip->read_buf(mtd, buf, mtd->writesize);
+ chip->legacy.read_buf(chip, buf, mtd->writesize);
/* Read oob bytes */
nand_change_read_column_op(chip,
if (ret)
return ret;
- stat = chip->ecc.correct(mtd, buf, ecc_code, ecc_calc);
+ stat = chip->ecc.correct(chip, buf, ecc_code, ecc_calc);
if (stat < 0) {
mtd->ecc_stats.failed++;
/* Re-populate low-level callbacks based on xfer modes */
switch (info->xfer_type) {
case NAND_OMAP_PREFETCH_POLLED:
- chip->read_buf = omap_read_buf_pref;
- chip->write_buf = omap_write_buf_pref;
+ chip->legacy.read_buf = omap_read_buf_pref;
+ chip->legacy.write_buf = omap_write_buf_pref;
break;
case NAND_OMAP_POLLED:
err);
return err;
}
- chip->read_buf = omap_read_buf_dma_pref;
- chip->write_buf = omap_write_buf_dma_pref;
+ chip->legacy.read_buf = omap_read_buf_dma_pref;
+ chip->legacy.write_buf = omap_write_buf_dma_pref;
}
break;
return err;
}
- chip->read_buf = omap_read_buf_irq_pref;
- chip->write_buf = omap_write_buf_irq_pref;
+ chip->legacy.read_buf = omap_read_buf_irq_pref;
+ chip->legacy.write_buf = omap_write_buf_irq_pref;
break;
}
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- nand_chip->IO_ADDR_R = devm_ioremap_resource(&pdev->dev, res);
- if (IS_ERR(nand_chip->IO_ADDR_R))
- return PTR_ERR(nand_chip->IO_ADDR_R);
+ nand_chip->legacy.IO_ADDR_R = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(nand_chip->legacy.IO_ADDR_R))
+ return PTR_ERR(nand_chip->legacy.IO_ADDR_R);
info->phys_base = res->start;
nand_chip->controller = &omap_gpmc_controller;
- nand_chip->IO_ADDR_W = nand_chip->IO_ADDR_R;
- nand_chip->cmd_ctrl = omap_hwcontrol;
+ nand_chip->legacy.IO_ADDR_W = nand_chip->legacy.IO_ADDR_R;
+ nand_chip->legacy.cmd_ctrl = omap_hwcontrol;
info->ready_gpiod = devm_gpiod_get_optional(&pdev->dev, "rb",
GPIOD_IN);
* device and read status register until you get a failure or success
*/
if (info->ready_gpiod) {
- nand_chip->dev_ready = omap_dev_ready;
- nand_chip->chip_delay = 0;
+ nand_chip->legacy.dev_ready = omap_dev_ready;
+ nand_chip->legacy.chip_delay = 0;
} else {
- nand_chip->waitfunc = omap_wait;
- nand_chip->chip_delay = 50;
+ nand_chip->legacy.waitfunc = omap_wait;
+ nand_chip->legacy.chip_delay = 50;
}
if (info->flash_bbt)
/* scan NAND device connected to chip controller */
nand_chip->options |= info->devsize & NAND_BUSWIDTH_16;
- err = nand_scan(mtd, 1);
+ err = nand_scan(nand_chip, 1);
if (err)
goto return_error;
}
if (info->dma)
dma_release_channel(info->dma);
- nand_release(mtd);
+ nand_release(nand_chip);
return 0;
}
struct clk *clk;
};
-static void orion_nand_cmd_ctrl(struct mtd_info *mtd, int cmd, unsigned int ctrl)
+static void orion_nand_cmd_ctrl(struct nand_chip *nc, int cmd,
+ unsigned int ctrl)
{
- struct nand_chip *nc = mtd_to_nand(mtd);
struct orion_nand_data *board = nand_get_controller_data(nc);
u32 offs;
if (nc->options & NAND_BUSWIDTH_16)
offs <<= 1;
- writeb(cmd, nc->IO_ADDR_W + offs);
+ writeb(cmd, nc->legacy.IO_ADDR_W + offs);
}
-static void orion_nand_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void orion_nand_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
- void __iomem *io_base = chip->IO_ADDR_R;
+ void __iomem *io_base = chip->legacy.IO_ADDR_R;
#if defined(__LINUX_ARM_ARCH__) && __LINUX_ARM_ARCH__ >= 5
uint64_t *buf64;
#endif
nand_set_controller_data(nc, board);
nand_set_flash_node(nc, pdev->dev.of_node);
- nc->IO_ADDR_R = nc->IO_ADDR_W = io_base;
- nc->cmd_ctrl = orion_nand_cmd_ctrl;
- nc->read_buf = orion_nand_read_buf;
+ nc->legacy.IO_ADDR_R = nc->legacy.IO_ADDR_W = io_base;
+ nc->legacy.cmd_ctrl = orion_nand_cmd_ctrl;
+ nc->legacy.read_buf = orion_nand_read_buf;
nc->ecc.mode = NAND_ECC_SOFT;
nc->ecc.algo = NAND_ECC_HAMMING;
if (board->chip_delay)
- nc->chip_delay = board->chip_delay;
+ nc->legacy.chip_delay = board->chip_delay;
WARN(board->width > 16,
"%d bit bus width out of range",
return ret;
}
- ret = nand_scan(mtd, 1);
+ ret = nand_scan(nc, 1);
if (ret)
goto no_dev;
mtd->name = "orion_nand";
ret = mtd_device_register(mtd, board->parts, board->nr_parts);
if (ret) {
- nand_release(mtd);
+ nand_release(nc);
goto no_dev;
}
{
struct orion_nand_info *info = platform_get_drvdata(pdev);
struct nand_chip *chip = &info->chip;
- struct mtd_info *mtd = nand_to_mtd(chip);
- nand_release(mtd);
+ nand_release(chip);
clk_disable_unprepare(info->clk);
struct nand_chip *chips[OXNAS_NAND_MAX_CHIPS];
};
-static uint8_t oxnas_nand_read_byte(struct mtd_info *mtd)
+static uint8_t oxnas_nand_read_byte(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct oxnas_nand_ctrl *oxnas = nand_get_controller_data(chip);
return readb(oxnas->io_base);
}
-static void oxnas_nand_read_buf(struct mtd_info *mtd, u8 *buf, int len)
+static void oxnas_nand_read_buf(struct nand_chip *chip, u8 *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct oxnas_nand_ctrl *oxnas = nand_get_controller_data(chip);
ioread8_rep(oxnas->io_base, buf, len);
}
-static void oxnas_nand_write_buf(struct mtd_info *mtd, const u8 *buf, int len)
+static void oxnas_nand_write_buf(struct nand_chip *chip, const u8 *buf,
+ int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct oxnas_nand_ctrl *oxnas = nand_get_controller_data(chip);
iowrite8_rep(oxnas->io_base, buf, len);
}
/* Single CS command control */
-static void oxnas_nand_cmd_ctrl(struct mtd_info *mtd, int cmd,
+static void oxnas_nand_cmd_ctrl(struct nand_chip *chip, int cmd,
unsigned int ctrl)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct oxnas_nand_ctrl *oxnas = nand_get_controller_data(chip);
if (ctrl & NAND_CLE)
mtd->dev.parent = &pdev->dev;
mtd->priv = chip;
- chip->cmd_ctrl = oxnas_nand_cmd_ctrl;
- chip->read_buf = oxnas_nand_read_buf;
- chip->read_byte = oxnas_nand_read_byte;
- chip->write_buf = oxnas_nand_write_buf;
- chip->chip_delay = 30;
+ chip->legacy.cmd_ctrl = oxnas_nand_cmd_ctrl;
+ chip->legacy.read_buf = oxnas_nand_read_buf;
+ chip->legacy.read_byte = oxnas_nand_read_byte;
+ chip->legacy.write_buf = oxnas_nand_write_buf;
+ chip->legacy.chip_delay = 30;
/* Scan to find existence of the device */
- err = nand_scan(mtd, 1);
+ err = nand_scan(chip, 1);
if (err)
goto err_clk_unprepare;
err = mtd_device_register(mtd, NULL, 0);
if (err) {
- nand_release(mtd);
+ nand_release(chip);
goto err_clk_unprepare;
}
struct oxnas_nand_ctrl *oxnas = platform_get_drvdata(pdev);
if (oxnas->chips[0])
- nand_release(nand_to_mtd(oxnas->chips[0]));
+ nand_release(oxnas->chips[0]);
clk_disable_unprepare(oxnas->clk);
static struct mtd_info *pasemi_nand_mtd;
static const char driver_name[] = "pasemi-nand";
-static void pasemi_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void pasemi_read_buf(struct nand_chip *chip, u_char *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
while (len > 0x800) {
- memcpy_fromio(buf, chip->IO_ADDR_R, 0x800);
+ memcpy_fromio(buf, chip->legacy.IO_ADDR_R, 0x800);
buf += 0x800;
len -= 0x800;
}
- memcpy_fromio(buf, chip->IO_ADDR_R, len);
+ memcpy_fromio(buf, chip->legacy.IO_ADDR_R, len);
}
-static void pasemi_write_buf(struct mtd_info *mtd, const u_char *buf, int len)
+static void pasemi_write_buf(struct nand_chip *chip, const u_char *buf,
+ int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
while (len > 0x800) {
- memcpy_toio(chip->IO_ADDR_R, buf, 0x800);
+ memcpy_toio(chip->legacy.IO_ADDR_R, buf, 0x800);
buf += 0x800;
len -= 0x800;
}
- memcpy_toio(chip->IO_ADDR_R, buf, len);
+ memcpy_toio(chip->legacy.IO_ADDR_R, buf, len);
}
-static void pasemi_hwcontrol(struct mtd_info *mtd, int cmd,
+static void pasemi_hwcontrol(struct nand_chip *chip, int cmd,
unsigned int ctrl)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
-
if (cmd == NAND_CMD_NONE)
return;
if (ctrl & NAND_CLE)
- out_8(chip->IO_ADDR_W + (1 << CLE_PIN_CTL), cmd);
+ out_8(chip->legacy.IO_ADDR_W + (1 << CLE_PIN_CTL), cmd);
else
- out_8(chip->IO_ADDR_W + (1 << ALE_PIN_CTL), cmd);
+ out_8(chip->legacy.IO_ADDR_W + (1 << ALE_PIN_CTL), cmd);
/* Push out posted writes */
eieio();
inl(lpcctl);
}
-int pasemi_device_ready(struct mtd_info *mtd)
+int pasemi_device_ready(struct nand_chip *chip)
{
return !!(inl(lpcctl) & LBICTRL_LPCCTL_NR);
}
/* Link the private data with the MTD structure */
pasemi_nand_mtd->dev.parent = dev;
- chip->IO_ADDR_R = of_iomap(np, 0);
- chip->IO_ADDR_W = chip->IO_ADDR_R;
+ chip->legacy.IO_ADDR_R = of_iomap(np, 0);
+ chip->legacy.IO_ADDR_W = chip->legacy.IO_ADDR_R;
- if (!chip->IO_ADDR_R) {
+ if (!chip->legacy.IO_ADDR_R) {
err = -EIO;
goto out_mtd;
}
goto out_ior;
}
- chip->cmd_ctrl = pasemi_hwcontrol;
- chip->dev_ready = pasemi_device_ready;
- chip->read_buf = pasemi_read_buf;
- chip->write_buf = pasemi_write_buf;
- chip->chip_delay = 0;
+ chip->legacy.cmd_ctrl = pasemi_hwcontrol;
+ chip->legacy.dev_ready = pasemi_device_ready;
+ chip->legacy.read_buf = pasemi_read_buf;
+ chip->legacy.write_buf = pasemi_write_buf;
+ chip->legacy.chip_delay = 0;
chip->ecc.mode = NAND_ECC_SOFT;
chip->ecc.algo = NAND_ECC_HAMMING;
chip->bbt_options = NAND_BBT_USE_FLASH;
/* Scan to find existence of the device */
- err = nand_scan(pasemi_nand_mtd, 1);
+ err = nand_scan(chip, 1);
if (err)
goto out_lpc;
out_lpc:
release_region(lpcctl, 4);
out_ior:
- iounmap(chip->IO_ADDR_R);
+ iounmap(chip->legacy.IO_ADDR_R);
out_mtd:
kfree(chip);
out:
chip = mtd_to_nand(pasemi_nand_mtd);
/* Release resources, unregister device */
- nand_release(pasemi_nand_mtd);
+ nand_release(chip);
release_region(lpcctl, 4);
- iounmap(chip->IO_ADDR_R);
+ iounmap(chip->legacy.IO_ADDR_R);
/* Free the MTD device structure */
kfree(chip);
#include <linux/platform_device.h>
#include <linux/slab.h>
#include <linux/mtd/mtd.h>
-#include <linux/mtd/rawnand.h>
-#include <linux/mtd/partitions.h>
+#include <linux/mtd/platnand.h>
struct plat_nand_data {
struct nand_chip chip;
mtd = nand_to_mtd(&data->chip);
mtd->dev.parent = &pdev->dev;
- data->chip.IO_ADDR_R = data->io_base;
- data->chip.IO_ADDR_W = data->io_base;
- data->chip.cmd_ctrl = pdata->ctrl.cmd_ctrl;
- data->chip.dev_ready = pdata->ctrl.dev_ready;
+ data->chip.legacy.IO_ADDR_R = data->io_base;
+ data->chip.legacy.IO_ADDR_W = data->io_base;
+ data->chip.legacy.cmd_ctrl = pdata->ctrl.cmd_ctrl;
+ data->chip.legacy.dev_ready = pdata->ctrl.dev_ready;
data->chip.select_chip = pdata->ctrl.select_chip;
- data->chip.write_buf = pdata->ctrl.write_buf;
- data->chip.read_buf = pdata->ctrl.read_buf;
- data->chip.chip_delay = pdata->chip.chip_delay;
+ data->chip.legacy.write_buf = pdata->ctrl.write_buf;
+ data->chip.legacy.read_buf = pdata->ctrl.read_buf;
+ data->chip.legacy.chip_delay = pdata->chip.chip_delay;
data->chip.options |= pdata->chip.options;
data->chip.bbt_options |= pdata->chip.bbt_options;
}
/* Scan to find existence of the device */
- err = nand_scan(mtd, pdata->chip.nr_chips);
+ err = nand_scan(&data->chip, pdata->chip.nr_chips);
if (err)
goto out;
if (!err)
return err;
- nand_release(mtd);
+ nand_release(&data->chip);
out:
if (pdata->ctrl.remove)
pdata->ctrl.remove(pdev);
struct plat_nand_data *data = platform_get_drvdata(pdev);
struct platform_nand_data *pdata = dev_get_platdata(&pdev->dev);
- nand_release(nand_to_mtd(&data->chip));
+ nand_release(&data->chip);
if (pdata->ctrl.remove)
pdata->ctrl.remove(pdev);
#include <linux/of_device.h>
#include <linux/delay.h>
#include <linux/dma/qcom_bam_dma.h>
-#include <linux/dma-direct.h> /* XXX: drivers shall never use this directly! */
/* NANDc reg offsets */
#define NAND_FLASH_CMD 0x00
* @data_buffer: our local DMA buffer for page read/writes,
* used when we can't use the buffer provided
* by upper layers directly
- * @buf_size/count/start: markers for chip->read_buf/write_buf functions
+ * @buf_size/count/start: markers for chip->legacy.read_buf/write_buf
+ * functions
* @reg_read_buf: local buffer for reading back registers via DMA
* @reg_read_dma: contains dma address for register read buffer
* @reg_read_pos: marker for data read in reg_read_buf
}
/*
- * the following functions are used within chip->cmdfunc() to perform different
- * NAND_CMD_* commands
+ * the following functions are used within chip->legacy.cmdfunc() to
+ * perform different NAND_CMD_* commands
*/
/* sets up descriptors for NAND_CMD_PARAM */
}
/*
- * Implements chip->cmdfunc. It's only used for a limited set of commands.
- * The rest of the commands wouldn't be called by upper layers. For example,
- * NAND_CMD_READOOB would never be called because we have our own versions
- * of read_oob ops for nand_ecc_ctrl.
+ * Implements chip->legacy.cmdfunc. It's only used for a limited set of
+ * commands. The rest of the commands wouldn't be called by upper layers.
+ * For example, NAND_CMD_READOOB would never be called because we have our own
+ * versions of read_oob ops for nand_ecc_ctrl.
*/
-static void qcom_nandc_command(struct mtd_info *mtd, unsigned int command,
+static void qcom_nandc_command(struct nand_chip *chip, unsigned int command,
int column, int page_addr)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct qcom_nand_host *host = to_qcom_nand_host(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
}
/* implements ecc->read_page() */
-static int qcom_nandc_read_page(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int qcom_nandc_read_page(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
struct qcom_nand_host *host = to_qcom_nand_host(chip);
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
}
/* implements ecc->read_page_raw() */
-static int qcom_nandc_read_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip, uint8_t *buf,
+static int qcom_nandc_read_page_raw(struct nand_chip *chip, uint8_t *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct qcom_nand_host *host = to_qcom_nand_host(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
int cw, ret;
}
/* implements ecc->read_oob() */
-static int qcom_nandc_read_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int qcom_nandc_read_oob(struct nand_chip *chip, int page)
{
struct qcom_nand_host *host = to_qcom_nand_host(chip);
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
}
/* implements ecc->write_page() */
-static int qcom_nandc_write_page(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page)
+static int qcom_nandc_write_page(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
struct qcom_nand_host *host = to_qcom_nand_host(chip);
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
}
/* implements ecc->write_page_raw() */
-static int qcom_nandc_write_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip, const uint8_t *buf,
- int oob_required, int page)
+static int qcom_nandc_write_page_raw(struct nand_chip *chip,
+ const uint8_t *buf, int oob_required,
+ int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct qcom_nand_host *host = to_qcom_nand_host(chip);
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
* since ECC is calculated for the combined codeword. So update the OOB from
* chip->oob_poi, and pad the data area with OxFF before writing.
*/
-static int qcom_nandc_write_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int qcom_nandc_write_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct qcom_nand_host *host = to_qcom_nand_host(chip);
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
return nand_prog_page_end_op(chip);
}
-static int qcom_nandc_block_bad(struct mtd_info *mtd, loff_t ofs)
+static int qcom_nandc_block_bad(struct nand_chip *chip, loff_t ofs)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct qcom_nand_host *host = to_qcom_nand_host(chip);
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
return bad;
}
-static int qcom_nandc_block_markbad(struct mtd_info *mtd, loff_t ofs)
+static int qcom_nandc_block_markbad(struct nand_chip *chip, loff_t ofs)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct qcom_nand_host *host = to_qcom_nand_host(chip);
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
}
/*
- * the three functions below implement chip->read_byte(), chip->read_buf()
- * and chip->write_buf() respectively. these aren't used for
- * reading/writing page data, they are used for smaller data like reading
- * id, status etc
+ * the three functions below implement chip->legacy.read_byte(),
+ * chip->legacy.read_buf() and chip->legacy.write_buf() respectively. these
+ * aren't used for reading/writing page data, they are used for smaller data
+ * like reading id, status etc
*/
-static uint8_t qcom_nandc_read_byte(struct mtd_info *mtd)
+static uint8_t qcom_nandc_read_byte(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct qcom_nand_host *host = to_qcom_nand_host(chip);
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
u8 *buf = nandc->data_buffer;
return ret;
}
-static void qcom_nandc_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void qcom_nandc_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
int real_len = min_t(size_t, len, nandc->buf_count - nandc->buf_start);
nandc->buf_start += real_len;
}
-static void qcom_nandc_write_buf(struct mtd_info *mtd, const uint8_t *buf,
+static void qcom_nandc_write_buf(struct nand_chip *chip, const uint8_t *buf,
int len)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
int real_len = min_t(size_t, len, nandc->buf_count - nandc->buf_start);
}
/* we support only one external chip for now */
-static void qcom_nandc_select_chip(struct mtd_info *mtd, int chipnr)
+static void qcom_nandc_select_chip(struct nand_chip *chip, int chipnr)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
if (chipnr <= 0)
mtd->owner = THIS_MODULE;
mtd->dev.parent = dev;
- chip->cmdfunc = qcom_nandc_command;
+ chip->legacy.cmdfunc = qcom_nandc_command;
chip->select_chip = qcom_nandc_select_chip;
- chip->read_byte = qcom_nandc_read_byte;
- chip->read_buf = qcom_nandc_read_buf;
- chip->write_buf = qcom_nandc_write_buf;
- chip->set_features = nand_get_set_features_notsupp;
- chip->get_features = nand_get_set_features_notsupp;
+ chip->legacy.read_byte = qcom_nandc_read_byte;
+ chip->legacy.read_buf = qcom_nandc_read_buf;
+ chip->legacy.write_buf = qcom_nandc_write_buf;
+ chip->legacy.set_features = nand_get_set_features_notsupp;
+ chip->legacy.get_features = nand_get_set_features_notsupp;
/*
* the bad block marker is readable only when we read the last codeword
* and block_markbad helpers until we permanently switch to using
* MTD_OPS_RAW for all drivers (with the help of badblockbits)
*/
- chip->block_bad = qcom_nandc_block_bad;
- chip->block_markbad = qcom_nandc_block_markbad;
+ chip->legacy.block_bad = qcom_nandc_block_bad;
+ chip->legacy.block_markbad = qcom_nandc_block_markbad;
chip->controller = &nandc->controller;
chip->options |= NAND_NO_SUBPAGE_WRITE | NAND_USE_BOUNCE_BUFFER |
/* set up initial status value */
host->status = NAND_STATUS_READY | NAND_STATUS_WP;
- ret = nand_scan(mtd, 1);
+ ret = nand_scan(chip, 1);
if (ret)
return ret;
struct qcom_nand_host *host;
list_for_each_entry(host, &nandc->host_list, node)
- nand_release(nand_to_mtd(&host->chip));
+ nand_release(&host->chip);
qcom_nandc_unalloc(nandc);
/*
* Program data lines of the nand chip to send data to it
*/
-static void r852_write_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
+static void r852_write_buf(struct nand_chip *chip, const uint8_t *buf, int len)
{
- struct r852_device *dev = r852_get_dev(mtd);
+ struct r852_device *dev = r852_get_dev(nand_to_mtd(chip));
uint32_t reg;
/* Don't allow any access to hardware if we suspect card removal */
/*
* Read data lines of the nand chip to retrieve data
*/
-static void r852_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void r852_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
{
- struct r852_device *dev = r852_get_dev(mtd);
+ struct r852_device *dev = r852_get_dev(nand_to_mtd(chip));
uint32_t reg;
if (dev->card_unstable) {
/*
* Read one byte from nand chip
*/
-static uint8_t r852_read_byte(struct mtd_info *mtd)
+static uint8_t r852_read_byte(struct nand_chip *chip)
{
- struct r852_device *dev = r852_get_dev(mtd);
+ struct r852_device *dev = r852_get_dev(nand_to_mtd(chip));
/* Same problem as in r852_read_buf.... */
if (dev->card_unstable)
/*
* Control several chip lines & send commands
*/
-static void r852_cmdctl(struct mtd_info *mtd, int dat, unsigned int ctrl)
+static void r852_cmdctl(struct nand_chip *chip, int dat, unsigned int ctrl)
{
- struct r852_device *dev = r852_get_dev(mtd);
+ struct r852_device *dev = r852_get_dev(nand_to_mtd(chip));
if (dev->card_unstable)
return;
* Wait till card is ready.
* based on nand_wait, but returns errors on DMA error
*/
-static int r852_wait(struct mtd_info *mtd, struct nand_chip *chip)
+static int r852_wait(struct nand_chip *chip)
{
struct r852_device *dev = nand_get_controller_data(chip);
msecs_to_jiffies(400) : msecs_to_jiffies(20));
while (time_before(jiffies, timeout))
- if (chip->dev_ready(mtd))
+ if (chip->legacy.dev_ready(chip))
break;
nand_status_op(chip, &status);
* Check if card is ready
*/
-static int r852_ready(struct mtd_info *mtd)
+static int r852_ready(struct nand_chip *chip)
{
- struct r852_device *dev = r852_get_dev(mtd);
+ struct r852_device *dev = r852_get_dev(nand_to_mtd(chip));
return !(r852_read_reg(dev, R852_CARD_STA) & R852_CARD_STA_BUSY);
}
* Set ECC engine mode
*/
-static void r852_ecc_hwctl(struct mtd_info *mtd, int mode)
+static void r852_ecc_hwctl(struct nand_chip *chip, int mode)
{
- struct r852_device *dev = r852_get_dev(mtd);
+ struct r852_device *dev = r852_get_dev(nand_to_mtd(chip));
if (dev->card_unstable)
return;
* Calculate ECC, only used for writes
*/
-static int r852_ecc_calculate(struct mtd_info *mtd, const uint8_t *dat,
- uint8_t *ecc_code)
+static int r852_ecc_calculate(struct nand_chip *chip, const uint8_t *dat,
+ uint8_t *ecc_code)
{
- struct r852_device *dev = r852_get_dev(mtd);
+ struct r852_device *dev = r852_get_dev(nand_to_mtd(chip));
struct sm_oob *oob = (struct sm_oob *)ecc_code;
uint32_t ecc1, ecc2;
* Correct the data using ECC, hw did almost everything for us
*/
-static int r852_ecc_correct(struct mtd_info *mtd, uint8_t *dat,
- uint8_t *read_ecc, uint8_t *calc_ecc)
+static int r852_ecc_correct(struct nand_chip *chip, uint8_t *dat,
+ uint8_t *read_ecc, uint8_t *calc_ecc)
{
uint32_t ecc_reg;
uint8_t ecc_status, err_byte;
int i, error = 0;
- struct r852_device *dev = r852_get_dev(mtd);
+ struct r852_device *dev = r852_get_dev(nand_to_mtd(chip));
if (dev->card_unstable)
return 0;
* This is copy of nand_read_oob_std
* nand_read_oob_syndrome assumes we can send column address - we can't
*/
-static int r852_read_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int r852_read_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
return nand_read_oob_op(chip, page, 0, chip->oob_poi, mtd->oobsize);
}
{
struct mtd_info *mtd = nand_to_mtd(dev->chip);
- WARN_ON(dev->card_registred);
+ WARN_ON(dev->card_registered);
mtd->dev.parent = &dev->pci_dev->dev;
goto error3;
}
- dev->card_registred = 1;
+ dev->card_registered = 1;
return 0;
error3:
- nand_release(mtd);
+ nand_release(dev->chip);
error1:
/* Force card redetect */
dev->card_detected = 0;
{
struct mtd_info *mtd = nand_to_mtd(dev->chip);
- if (!dev->card_registred)
+ if (!dev->card_registered)
return;
device_remove_file(&mtd->dev, &dev_attr_media_type);
- nand_release(mtd);
+ nand_release(dev->chip);
r852_engine_disable(dev);
- dev->card_registred = 0;
+ dev->card_registered = 0;
}
/* Card state updater */
dev->card_unstable = 0;
/* False alarm */
- if (dev->card_detected == dev->card_registred)
+ if (dev->card_detected == dev->card_registered)
goto exit;
/* Read media properties */
goto error4;
/* commands */
- chip->cmd_ctrl = r852_cmdctl;
- chip->waitfunc = r852_wait;
- chip->dev_ready = r852_ready;
+ chip->legacy.cmd_ctrl = r852_cmdctl;
+ chip->legacy.waitfunc = r852_wait;
+ chip->legacy.dev_ready = r852_ready;
/* I/O */
- chip->read_byte = r852_read_byte;
- chip->read_buf = r852_read_buf;
- chip->write_buf = r852_write_buf;
+ chip->legacy.read_byte = r852_read_byte;
+ chip->legacy.read_buf = r852_read_buf;
+ chip->legacy.write_buf = r852_write_buf;
/* ecc */
chip->ecc.mode = NAND_ECC_HW_SYNDROME;
static int r852_resume(struct device *device)
{
struct r852_device *dev = pci_get_drvdata(to_pci_dev(device));
- struct mtd_info *mtd = nand_to_mtd(dev->chip);
r852_disable_irqs(dev);
r852_card_update_present(dev);
/* If card status changed, just do the work */
- if (dev->card_detected != dev->card_registred) {
+ if (dev->card_detected != dev->card_registered) {
dbg("card was %s during low power state",
dev->card_detected ? "added" : "removed");
}
/* Otherwise, initialize the card */
- if (dev->card_registred) {
+ if (dev->card_registered) {
r852_engine_enable(dev);
- dev->chip->select_chip(mtd, 0);
+ dev->chip->select_chip(dev->chip, 0);
nand_reset_op(dev->chip);
- dev->chip->select_chip(mtd, -1);
+ dev->chip->select_chip(dev->chip, -1);
}
/* Program card detection IRQ */
/* card status area */
struct delayed_work card_detect_work;
struct workqueue_struct *card_workqueue;
- int card_registred; /* card registered with mtd */
+ int card_registered; /* card registered with mtd */
int card_detected; /* card detected in slot */
int card_unstable; /* whenever the card is inserted,
is not known yet */
/**
* s3c2410_nand_select_chip - select the given nand chip
- * @mtd: The MTD instance for this chip.
+ * @this: NAND chip object.
* @chip: The chip number.
*
* This is called by the MTD layer to either select a given chip for the
* platform specific selection code is called to route nFCE to the specific
* chip.
*/
-static void s3c2410_nand_select_chip(struct mtd_info *mtd, int chip)
+static void s3c2410_nand_select_chip(struct nand_chip *this, int chip)
{
struct s3c2410_nand_info *info;
struct s3c2410_nand_mtd *nmtd;
- struct nand_chip *this = mtd_to_nand(mtd);
unsigned long cur;
nmtd = nand_get_controller_data(this);
* Issue command and address cycles to the chip
*/
-static void s3c2410_nand_hwcontrol(struct mtd_info *mtd, int cmd,
+static void s3c2410_nand_hwcontrol(struct nand_chip *chip, int cmd,
unsigned int ctrl)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
if (cmd == NAND_CMD_NONE)
/* command and control functions */
-static void s3c2440_nand_hwcontrol(struct mtd_info *mtd, int cmd,
+static void s3c2440_nand_hwcontrol(struct nand_chip *chip, int cmd,
unsigned int ctrl)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
if (cmd == NAND_CMD_NONE)
* returns 0 if the nand is busy, 1 if it is ready
*/
-static int s3c2410_nand_devready(struct mtd_info *mtd)
+static int s3c2410_nand_devready(struct nand_chip *chip)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
return readb(info->regs + S3C2410_NFSTAT) & S3C2410_NFSTAT_BUSY;
}
-static int s3c2440_nand_devready(struct mtd_info *mtd)
+static int s3c2440_nand_devready(struct nand_chip *chip)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
return readb(info->regs + S3C2440_NFSTAT) & S3C2440_NFSTAT_READY;
}
-static int s3c2412_nand_devready(struct mtd_info *mtd)
+static int s3c2412_nand_devready(struct nand_chip *chip)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
return readb(info->regs + S3C2412_NFSTAT) & S3C2412_NFSTAT_READY;
}
/* ECC handling functions */
-static int s3c2410_nand_correct_data(struct mtd_info *mtd, u_char *dat,
+static int s3c2410_nand_correct_data(struct nand_chip *chip, u_char *dat,
u_char *read_ecc, u_char *calc_ecc)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
unsigned int diff0, diff1, diff2;
unsigned int bit, byte;
* generator block to ECC the data as it passes through]
*/
-static void s3c2410_nand_enable_hwecc(struct mtd_info *mtd, int mode)
+static void s3c2410_nand_enable_hwecc(struct nand_chip *chip, int mode)
{
- struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
+ struct s3c2410_nand_info *info;
unsigned long ctrl;
+ info = s3c2410_nand_mtd_toinfo(nand_to_mtd(chip));
ctrl = readl(info->regs + S3C2410_NFCONF);
ctrl |= S3C2410_NFCONF_INITECC;
writel(ctrl, info->regs + S3C2410_NFCONF);
}
-static void s3c2412_nand_enable_hwecc(struct mtd_info *mtd, int mode)
+static void s3c2412_nand_enable_hwecc(struct nand_chip *chip, int mode)
{
- struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
+ struct s3c2410_nand_info *info;
unsigned long ctrl;
+ info = s3c2410_nand_mtd_toinfo(nand_to_mtd(chip));
ctrl = readl(info->regs + S3C2440_NFCONT);
writel(ctrl | S3C2412_NFCONT_INIT_MAIN_ECC,
info->regs + S3C2440_NFCONT);
}
-static void s3c2440_nand_enable_hwecc(struct mtd_info *mtd, int mode)
+static void s3c2440_nand_enable_hwecc(struct nand_chip *chip, int mode)
{
- struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
+ struct s3c2410_nand_info *info;
unsigned long ctrl;
+ info = s3c2410_nand_mtd_toinfo(nand_to_mtd(chip));
ctrl = readl(info->regs + S3C2440_NFCONT);
writel(ctrl | S3C2440_NFCONT_INITECC, info->regs + S3C2440_NFCONT);
}
-static int s3c2410_nand_calculate_ecc(struct mtd_info *mtd, const u_char *dat,
- u_char *ecc_code)
+static int s3c2410_nand_calculate_ecc(struct nand_chip *chip,
+ const u_char *dat, u_char *ecc_code)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
ecc_code[0] = readb(info->regs + S3C2410_NFECC + 0);
return 0;
}
-static int s3c2412_nand_calculate_ecc(struct mtd_info *mtd, const u_char *dat,
- u_char *ecc_code)
+static int s3c2412_nand_calculate_ecc(struct nand_chip *chip,
+ const u_char *dat, u_char *ecc_code)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
unsigned long ecc = readl(info->regs + S3C2412_NFMECC0);
return 0;
}
-static int s3c2440_nand_calculate_ecc(struct mtd_info *mtd, const u_char *dat,
- u_char *ecc_code)
+static int s3c2440_nand_calculate_ecc(struct nand_chip *chip,
+ const u_char *dat, u_char *ecc_code)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
unsigned long ecc = readl(info->regs + S3C2440_NFMECC0);
* use read/write block to move the data buffers to/from the controller
*/
-static void s3c2410_nand_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void s3c2410_nand_read_buf(struct nand_chip *this, u_char *buf, int len)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- readsb(this->IO_ADDR_R, buf, len);
+ readsb(this->legacy.IO_ADDR_R, buf, len);
}
-static void s3c2440_nand_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void s3c2440_nand_read_buf(struct nand_chip *this, u_char *buf, int len)
{
+ struct mtd_info *mtd = nand_to_mtd(this);
struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
readsl(info->regs + S3C2440_NFDATA, buf, len >> 2);
}
}
-static void s3c2410_nand_write_buf(struct mtd_info *mtd, const u_char *buf,
+static void s3c2410_nand_write_buf(struct nand_chip *this, const u_char *buf,
int len)
{
- struct nand_chip *this = mtd_to_nand(mtd);
- writesb(this->IO_ADDR_W, buf, len);
+ writesb(this->legacy.IO_ADDR_W, buf, len);
}
-static void s3c2440_nand_write_buf(struct mtd_info *mtd, const u_char *buf,
+static void s3c2440_nand_write_buf(struct nand_chip *this, const u_char *buf,
int len)
{
+ struct mtd_info *mtd = nand_to_mtd(this);
struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
writesl(info->regs + S3C2440_NFDATA, buf, len >> 2);
for (mtdno = 0; mtdno < info->mtd_count; mtdno++, ptr++) {
pr_debug("releasing mtd %d (%p)\n", mtdno, ptr);
- nand_release(nand_to_mtd(&ptr->chip));
+ nand_release(&ptr->chip);
}
}
return -ENODEV;
}
-static int s3c2410_nand_setup_data_interface(struct mtd_info *mtd, int csline,
+static int s3c2410_nand_setup_data_interface(struct nand_chip *chip, int csline,
const struct nand_data_interface *conf)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct s3c2410_nand_info *info = s3c2410_nand_mtd_toinfo(mtd);
struct s3c2410_platform_nand *pdata = info->platform;
const struct nand_sdr_timings *timings;
nand_set_flash_node(chip, set->of_node);
- chip->write_buf = s3c2410_nand_write_buf;
- chip->read_buf = s3c2410_nand_read_buf;
+ chip->legacy.write_buf = s3c2410_nand_write_buf;
+ chip->legacy.read_buf = s3c2410_nand_read_buf;
chip->select_chip = s3c2410_nand_select_chip;
- chip->chip_delay = 50;
+ chip->legacy.chip_delay = 50;
nand_set_controller_data(chip, nmtd);
chip->options = set->options;
chip->controller = &info->controller;
switch (info->cpu_type) {
case TYPE_S3C2410:
- chip->IO_ADDR_W = regs + S3C2410_NFDATA;
+ chip->legacy.IO_ADDR_W = regs + S3C2410_NFDATA;
info->sel_reg = regs + S3C2410_NFCONF;
info->sel_bit = S3C2410_NFCONF_nFCE;
- chip->cmd_ctrl = s3c2410_nand_hwcontrol;
- chip->dev_ready = s3c2410_nand_devready;
+ chip->legacy.cmd_ctrl = s3c2410_nand_hwcontrol;
+ chip->legacy.dev_ready = s3c2410_nand_devready;
break;
case TYPE_S3C2440:
- chip->IO_ADDR_W = regs + S3C2440_NFDATA;
+ chip->legacy.IO_ADDR_W = regs + S3C2440_NFDATA;
info->sel_reg = regs + S3C2440_NFCONT;
info->sel_bit = S3C2440_NFCONT_nFCE;
- chip->cmd_ctrl = s3c2440_nand_hwcontrol;
- chip->dev_ready = s3c2440_nand_devready;
- chip->read_buf = s3c2440_nand_read_buf;
- chip->write_buf = s3c2440_nand_write_buf;
+ chip->legacy.cmd_ctrl = s3c2440_nand_hwcontrol;
+ chip->legacy.dev_ready = s3c2440_nand_devready;
+ chip->legacy.read_buf = s3c2440_nand_read_buf;
+ chip->legacy.write_buf = s3c2440_nand_write_buf;
break;
case TYPE_S3C2412:
- chip->IO_ADDR_W = regs + S3C2440_NFDATA;
+ chip->legacy.IO_ADDR_W = regs + S3C2440_NFDATA;
info->sel_reg = regs + S3C2440_NFCONT;
info->sel_bit = S3C2412_NFCONT_nFCE0;
- chip->cmd_ctrl = s3c2440_nand_hwcontrol;
- chip->dev_ready = s3c2412_nand_devready;
+ chip->legacy.cmd_ctrl = s3c2440_nand_hwcontrol;
+ chip->legacy.dev_ready = s3c2412_nand_devready;
if (readl(regs + S3C2410_NFCONF) & S3C2412_NFCONF_NANDBOOT)
dev_info(info->device, "System booted from NAND\n");
break;
}
- chip->IO_ADDR_R = chip->IO_ADDR_W;
+ chip->legacy.IO_ADDR_R = chip->legacy.IO_ADDR_W;
nmtd->info = info;
nmtd->set = set;
mtd->dev.parent = &pdev->dev;
s3c2410_nand_init_chip(info, nmtd, sets);
- err = nand_scan(mtd, sets ? sets->nr_chips : 1);
+ err = nand_scan(&nmtd->chip, sets ? sets->nr_chips : 1);
if (err)
goto exit_error;
/* initiate DMA transfer */
if (flctl->chan_fifo0_rx && rlen >= 32 &&
- flctl_dma_fifo0_transfer(flctl, buf, rlen, DMA_DEV_TO_MEM) > 0)
+ flctl_dma_fifo0_transfer(flctl, buf, rlen, DMA_FROM_DEVICE) > 0)
goto convert; /* DMA success */
/* do polling transfer */
/* initiate DMA transfer */
if (flctl->chan_fifo0_tx && rlen >= 32 &&
- flctl_dma_fifo0_transfer(flctl, buf, rlen, DMA_MEM_TO_DEV) > 0)
+ flctl_dma_fifo0_transfer(flctl, buf, rlen, DMA_TO_DEVICE) > 0)
return; /* DMA success */
/* do polling transfer */
writel(flcmcdr_val, FLCMCDR(flctl));
}
-static int flctl_read_page_hwecc(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int flctl_read_page_hwecc(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
nand_read_page_op(chip, page, 0, buf, mtd->writesize);
if (oob_required)
- chip->read_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.read_buf(chip, chip->oob_poi, mtd->oobsize);
return 0;
}
-static int flctl_write_page_hwecc(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required,
- int page)
+static int flctl_write_page_hwecc(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
nand_prog_page_begin_op(chip, page, 0, buf, mtd->writesize);
- chip->write_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.write_buf(chip, chip->oob_poi, mtd->oobsize);
return nand_prog_page_end_op(chip);
}
}
}
-static void flctl_cmdfunc(struct mtd_info *mtd, unsigned int command,
+static void flctl_cmdfunc(struct nand_chip *chip, unsigned int command,
int column, int page_addr)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct sh_flctl *flctl = mtd_to_flctl(mtd);
uint32_t read_cmd = 0;
return;
}
-static void flctl_select_chip(struct mtd_info *mtd, int chipnr)
+static void flctl_select_chip(struct nand_chip *chip, int chipnr)
{
- struct sh_flctl *flctl = mtd_to_flctl(mtd);
+ struct sh_flctl *flctl = mtd_to_flctl(nand_to_mtd(chip));
int ret;
switch (chipnr) {
}
}
-static void flctl_write_buf(struct mtd_info *mtd, const uint8_t *buf, int len)
+static void flctl_write_buf(struct nand_chip *chip, const uint8_t *buf, int len)
{
- struct sh_flctl *flctl = mtd_to_flctl(mtd);
+ struct sh_flctl *flctl = mtd_to_flctl(nand_to_mtd(chip));
memcpy(&flctl->done_buff[flctl->index], buf, len);
flctl->index += len;
}
-static uint8_t flctl_read_byte(struct mtd_info *mtd)
+static uint8_t flctl_read_byte(struct nand_chip *chip)
{
- struct sh_flctl *flctl = mtd_to_flctl(mtd);
+ struct sh_flctl *flctl = mtd_to_flctl(nand_to_mtd(chip));
uint8_t data;
data = flctl->done_buff[flctl->index];
return data;
}
-static uint16_t flctl_read_word(struct mtd_info *mtd)
+static void flctl_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
{
- struct sh_flctl *flctl = mtd_to_flctl(mtd);
- uint16_t *buf = (uint16_t *)&flctl->done_buff[flctl->index];
-
- flctl->index += 2;
- return *buf;
-}
-
-static void flctl_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
-{
- struct sh_flctl *flctl = mtd_to_flctl(mtd);
+ struct sh_flctl *flctl = mtd_to_flctl(nand_to_mtd(chip));
memcpy(buf, &flctl->done_buff[flctl->index], len);
flctl->index += len;
/* Set address of hardware control function */
/* 20 us command delay time */
- nand->chip_delay = 20;
+ nand->legacy.chip_delay = 20;
- nand->read_byte = flctl_read_byte;
- nand->read_word = flctl_read_word;
- nand->write_buf = flctl_write_buf;
- nand->read_buf = flctl_read_buf;
+ nand->legacy.read_byte = flctl_read_byte;
+ nand->legacy.write_buf = flctl_write_buf;
+ nand->legacy.read_buf = flctl_read_buf;
nand->select_chip = flctl_select_chip;
- nand->cmdfunc = flctl_cmdfunc;
- nand->set_features = nand_get_set_features_notsupp;
- nand->get_features = nand_get_set_features_notsupp;
+ nand->legacy.cmdfunc = flctl_cmdfunc;
+ nand->legacy.set_features = nand_get_set_features_notsupp;
+ nand->legacy.get_features = nand_get_set_features_notsupp;
if (pdata->flcmncr_val & SEL_16BIT)
nand->options |= NAND_BUSWIDTH_16;
flctl_setup_dma(flctl);
nand->dummy_controller.ops = &flctl_nand_controller_ops;
- ret = nand_scan(flctl_mtd, 1);
+ ret = nand_scan(nand, 1);
if (ret)
goto err_chip;
struct sh_flctl *flctl = platform_get_drvdata(pdev);
flctl_release_dma(flctl);
- nand_release(nand_to_mtd(&flctl->chip));
+ nand_release(&flctl->chip);
pm_runtime_disable(&pdev->dev);
return 0;
* NAND_ALE: bit 2 -> bit 2
*
*/
-static void sharpsl_nand_hwcontrol(struct mtd_info *mtd, int cmd,
+static void sharpsl_nand_hwcontrol(struct nand_chip *chip, int cmd,
unsigned int ctrl)
{
- struct sharpsl_nand *sharpsl = mtd_to_sharpsl(mtd);
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct sharpsl_nand *sharpsl = mtd_to_sharpsl(nand_to_mtd(chip));
if (ctrl & NAND_CTRL_CHANGE) {
unsigned char bits = ctrl & 0x07;
}
if (cmd != NAND_CMD_NONE)
- writeb(cmd, chip->IO_ADDR_W);
+ writeb(cmd, chip->legacy.IO_ADDR_W);
}
-static int sharpsl_nand_dev_ready(struct mtd_info *mtd)
+static int sharpsl_nand_dev_ready(struct nand_chip *chip)
{
- struct sharpsl_nand *sharpsl = mtd_to_sharpsl(mtd);
+ struct sharpsl_nand *sharpsl = mtd_to_sharpsl(nand_to_mtd(chip));
return !((readb(sharpsl->io + FLASHCTL) & FLRYBY) == 0);
}
-static void sharpsl_nand_enable_hwecc(struct mtd_info *mtd, int mode)
+static void sharpsl_nand_enable_hwecc(struct nand_chip *chip, int mode)
{
- struct sharpsl_nand *sharpsl = mtd_to_sharpsl(mtd);
+ struct sharpsl_nand *sharpsl = mtd_to_sharpsl(nand_to_mtd(chip));
writeb(0, sharpsl->io + ECCCLRR);
}
-static int sharpsl_nand_calculate_ecc(struct mtd_info *mtd, const u_char * dat, u_char * ecc_code)
+static int sharpsl_nand_calculate_ecc(struct nand_chip *chip,
+ const u_char * dat, u_char * ecc_code)
{
- struct sharpsl_nand *sharpsl = mtd_to_sharpsl(mtd);
+ struct sharpsl_nand *sharpsl = mtd_to_sharpsl(nand_to_mtd(chip));
ecc_code[0] = ~readb(sharpsl->io + ECCLPUB);
ecc_code[1] = ~readb(sharpsl->io + ECCLPLB);
ecc_code[2] = (~readb(sharpsl->io + ECCCP) << 2) | 0x03;
writeb(readb(sharpsl->io + FLASHCTL) | FLWP, sharpsl->io + FLASHCTL);
/* Set address of NAND IO lines */
- this->IO_ADDR_R = sharpsl->io + FLASHIO;
- this->IO_ADDR_W = sharpsl->io + FLASHIO;
+ this->legacy.IO_ADDR_R = sharpsl->io + FLASHIO;
+ this->legacy.IO_ADDR_W = sharpsl->io + FLASHIO;
/* Set address of hardware control function */
- this->cmd_ctrl = sharpsl_nand_hwcontrol;
- this->dev_ready = sharpsl_nand_dev_ready;
+ this->legacy.cmd_ctrl = sharpsl_nand_hwcontrol;
+ this->legacy.dev_ready = sharpsl_nand_dev_ready;
/* 15 us command delay time */
- this->chip_delay = 15;
+ this->legacy.chip_delay = 15;
/* set eccmode using hardware ECC */
this->ecc.mode = NAND_ECC_HW;
this->ecc.size = 256;
this->ecc.correct = nand_correct_data;
/* Scan to find existence of the device */
- err = nand_scan(mtd, 1);
+ err = nand_scan(this, 1);
if (err)
goto err_scan;
return 0;
err_add:
- nand_release(mtd);
+ nand_release(this);
err_scan:
iounmap(sharpsl->io);
struct sharpsl_nand *sharpsl = platform_get_drvdata(pdev);
/* Release resources, unregister device */
- nand_release(nand_to_mtd(&sharpsl->chip));
+ nand_release(&sharpsl->chip);
iounmap(sharpsl->io);
.free = oob_sm_small_ooblayout_free,
};
-static int sm_block_markbad(struct mtd_info *mtd, loff_t ofs)
+static int sm_block_markbad(struct nand_chip *chip, loff_t ofs)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct mtd_oob_ops ops;
struct sm_oob oob;
int ret;
/* Bad block marker position */
chip->badblockpos = 0x05;
chip->badblockbits = 7;
- chip->block_markbad = sm_block_markbad;
+ chip->legacy.block_markbad = sm_block_markbad;
/* ECC layout */
if (mtd->writesize == SM_SECTOR_SIZE)
/* Scan for card properties */
chip->dummy_controller.ops = &sm_controller_ops;
flash_ids = smartmedia ? nand_smartmedia_flash_ids : nand_xd_flash_ids;
- ret = nand_scan_with_ids(mtd, 1, flash_ids);
+ ret = nand_scan_with_ids(chip, 1, flash_ids);
if (ret)
return ret;
/**
* socrates_nand_write_buf - write buffer to chip
- * @mtd: MTD device structure
+ * @this: NAND chip object
* @buf: data buffer
* @len: number of bytes to write
*/
-static void socrates_nand_write_buf(struct mtd_info *mtd,
- const uint8_t *buf, int len)
+static void socrates_nand_write_buf(struct nand_chip *this, const uint8_t *buf,
+ int len)
{
int i;
- struct nand_chip *this = mtd_to_nand(mtd);
struct socrates_nand_host *host = nand_get_controller_data(this);
for (i = 0; i < len; i++) {
/**
* socrates_nand_read_buf - read chip data into buffer
- * @mtd: MTD device structure
+ * @this: NAND chip object
* @buf: buffer to store date
* @len: number of bytes to read
*/
-static void socrates_nand_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void socrates_nand_read_buf(struct nand_chip *this, uint8_t *buf,
+ int len)
{
int i;
- struct nand_chip *this = mtd_to_nand(mtd);
struct socrates_nand_host *host = nand_get_controller_data(this);
uint32_t val;
* socrates_nand_read_byte - read one byte from the chip
* @mtd: MTD device structure
*/
-static uint8_t socrates_nand_read_byte(struct mtd_info *mtd)
+static uint8_t socrates_nand_read_byte(struct nand_chip *this)
{
uint8_t byte;
- socrates_nand_read_buf(mtd, &byte, sizeof(byte));
+ socrates_nand_read_buf(this, &byte, sizeof(byte));
return byte;
}
-/**
- * socrates_nand_read_word - read one word from the chip
- * @mtd: MTD device structure
- */
-static uint16_t socrates_nand_read_word(struct mtd_info *mtd)
-{
- uint16_t word;
- socrates_nand_read_buf(mtd, (uint8_t *)&word, sizeof(word));
- return word;
-}
-
/*
* Hardware specific access to control-lines
*/
-static void socrates_nand_cmd_ctrl(struct mtd_info *mtd, int cmd,
- unsigned int ctrl)
+static void socrates_nand_cmd_ctrl(struct nand_chip *nand_chip, int cmd,
+ unsigned int ctrl)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
struct socrates_nand_host *host = nand_get_controller_data(nand_chip);
uint32_t val;
/*
* Read the Device Ready pin.
*/
-static int socrates_nand_device_ready(struct mtd_info *mtd)
+static int socrates_nand_device_ready(struct nand_chip *nand_chip)
{
- struct nand_chip *nand_chip = mtd_to_nand(mtd);
struct socrates_nand_host *host = nand_get_controller_data(nand_chip);
if (in_be32(host->io_base) & FPGA_NAND_BUSY)
mtd->name = "socrates_nand";
mtd->dev.parent = &ofdev->dev;
- /*should never be accessed directly */
- nand_chip->IO_ADDR_R = (void *)0xdeadbeef;
- nand_chip->IO_ADDR_W = (void *)0xdeadbeef;
-
- nand_chip->cmd_ctrl = socrates_nand_cmd_ctrl;
- nand_chip->read_byte = socrates_nand_read_byte;
- nand_chip->read_word = socrates_nand_read_word;
- nand_chip->write_buf = socrates_nand_write_buf;
- nand_chip->read_buf = socrates_nand_read_buf;
- nand_chip->dev_ready = socrates_nand_device_ready;
+ nand_chip->legacy.cmd_ctrl = socrates_nand_cmd_ctrl;
+ nand_chip->legacy.read_byte = socrates_nand_read_byte;
+ nand_chip->legacy.write_buf = socrates_nand_write_buf;
+ nand_chip->legacy.read_buf = socrates_nand_read_buf;
+ nand_chip->legacy.dev_ready = socrates_nand_device_ready;
nand_chip->ecc.mode = NAND_ECC_SOFT; /* enable ECC */
nand_chip->ecc.algo = NAND_ECC_HAMMING;
/* TODO: I have no idea what real delay is. */
- nand_chip->chip_delay = 20; /* 20us command delay time */
+ nand_chip->legacy.chip_delay = 20; /* 20us command delay time */
dev_set_drvdata(&ofdev->dev, host);
- res = nand_scan(mtd, 1);
+ res = nand_scan(nand_chip, 1);
if (res)
goto out;
if (!res)
return res;
- nand_release(mtd);
+ nand_release(nand_chip);
out:
iounmap(host->io_base);
static int socrates_nand_remove(struct platform_device *ofdev)
{
struct socrates_nand_host *host = dev_get_drvdata(&ofdev->dev);
- struct mtd_info *mtd = nand_to_mtd(&host->nand_chip);
- nand_release(mtd);
+ nand_release(&host->nand_chip);
iounmap(host->io_base);
nfc->regs + NFC_REG_CTL);
}
-static int sunxi_nfc_dev_ready(struct mtd_info *mtd)
+static int sunxi_nfc_dev_ready(struct nand_chip *nand)
{
- struct nand_chip *nand = mtd_to_nand(mtd);
struct sunxi_nand_chip *sunxi_nand = to_sunxi_nand(nand);
struct sunxi_nfc *nfc = to_sunxi_nfc(sunxi_nand->nand.controller);
u32 mask;
return !!(readl(nfc->regs + NFC_REG_ST) & mask);
}
-static void sunxi_nfc_select_chip(struct mtd_info *mtd, int chip)
+static void sunxi_nfc_select_chip(struct nand_chip *nand, int chip)
{
- struct nand_chip *nand = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(nand);
struct sunxi_nand_chip *sunxi_nand = to_sunxi_nand(nand);
struct sunxi_nfc *nfc = to_sunxi_nfc(sunxi_nand->nand.controller);
struct sunxi_nand_chip_sel *sel;
ctl |= NFC_CE_SEL(sel->cs) | NFC_EN |
NFC_PAGE_SHIFT(nand->page_shift);
if (sel->rb < 0) {
- nand->dev_ready = NULL;
+ nand->legacy.dev_ready = NULL;
} else {
- nand->dev_ready = sunxi_nfc_dev_ready;
+ nand->legacy.dev_ready = sunxi_nfc_dev_ready;
ctl |= NFC_RB_SEL(sel->rb);
}
sunxi_nand->selected = chip;
}
-static void sunxi_nfc_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void sunxi_nfc_read_buf(struct nand_chip *nand, uint8_t *buf, int len)
{
- struct nand_chip *nand = mtd_to_nand(mtd);
struct sunxi_nand_chip *sunxi_nand = to_sunxi_nand(nand);
struct sunxi_nfc *nfc = to_sunxi_nfc(sunxi_nand->nand.controller);
int ret;
}
}
-static void sunxi_nfc_write_buf(struct mtd_info *mtd, const uint8_t *buf,
+static void sunxi_nfc_write_buf(struct nand_chip *nand, const uint8_t *buf,
int len)
{
- struct nand_chip *nand = mtd_to_nand(mtd);
struct sunxi_nand_chip *sunxi_nand = to_sunxi_nand(nand);
struct sunxi_nfc *nfc = to_sunxi_nfc(sunxi_nand->nand.controller);
int ret;
}
}
-static uint8_t sunxi_nfc_read_byte(struct mtd_info *mtd)
+static uint8_t sunxi_nfc_read_byte(struct nand_chip *nand)
{
uint8_t ret = 0;
- sunxi_nfc_read_buf(mtd, &ret, 1);
+ sunxi_nfc_read_buf(nand, &ret, 1);
return ret;
}
-static void sunxi_nfc_cmd_ctrl(struct mtd_info *mtd, int dat,
+static void sunxi_nfc_cmd_ctrl(struct nand_chip *nand, int dat,
unsigned int ctrl)
{
- struct nand_chip *nand = mtd_to_nand(mtd);
struct sunxi_nand_chip *sunxi_nand = to_sunxi_nand(nand);
struct sunxi_nfc *nfc = to_sunxi_nfc(sunxi_nand->nand.controller);
int ret;
{
sunxi_nfc_randomizer_config(mtd, page, ecc);
sunxi_nfc_randomizer_enable(mtd);
- sunxi_nfc_write_buf(mtd, buf, len);
+ sunxi_nfc_write_buf(mtd_to_nand(mtd), buf, len);
sunxi_nfc_randomizer_disable(mtd);
}
{
sunxi_nfc_randomizer_config(mtd, page, ecc);
sunxi_nfc_randomizer_enable(mtd);
- sunxi_nfc_read_buf(mtd, buf, len);
+ sunxi_nfc_read_buf(mtd_to_nand(mtd), buf, len);
sunxi_nfc_randomizer_disable(mtd);
}
false);
if (!randomize)
- sunxi_nfc_read_buf(mtd, oob + offset, len);
+ sunxi_nfc_read_buf(nand, oob + offset, len);
else
sunxi_nfc_randomizer_read_buf(mtd, oob + offset, len,
false, page);
*cur_off = mtd->oobsize + mtd->writesize;
}
-static int sunxi_nfc_hw_ecc_read_page(struct mtd_info *mtd,
- struct nand_chip *chip, uint8_t *buf,
+static int sunxi_nfc_hw_ecc_read_page(struct nand_chip *chip, uint8_t *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
unsigned int max_bitflips = 0;
int ret, i, cur_off = 0;
return max_bitflips;
}
-static int sunxi_nfc_hw_ecc_read_page_dma(struct mtd_info *mtd,
- struct nand_chip *chip, u8 *buf,
+static int sunxi_nfc_hw_ecc_read_page_dma(struct nand_chip *chip, u8 *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int ret;
nand_read_page_op(chip, page, 0, NULL, 0);
return ret;
/* Fallback to PIO mode */
- return sunxi_nfc_hw_ecc_read_page(mtd, chip, buf, oob_required, page);
+ return sunxi_nfc_hw_ecc_read_page(chip, buf, oob_required, page);
}
-static int sunxi_nfc_hw_ecc_read_subpage(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int sunxi_nfc_hw_ecc_read_subpage(struct nand_chip *chip,
u32 data_offs, u32 readlen,
u8 *bufpoi, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
int ret, i, cur_off = 0;
unsigned int max_bitflips = 0;
return max_bitflips;
}
-static int sunxi_nfc_hw_ecc_read_subpage_dma(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int sunxi_nfc_hw_ecc_read_subpage_dma(struct nand_chip *chip,
u32 data_offs, u32 readlen,
u8 *buf, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int nchunks = DIV_ROUND_UP(data_offs + readlen, chip->ecc.size);
int ret;
return ret;
/* Fallback to PIO mode */
- return sunxi_nfc_hw_ecc_read_subpage(mtd, chip, data_offs, readlen,
+ return sunxi_nfc_hw_ecc_read_subpage(chip, data_offs, readlen,
buf, page);
}
-static int sunxi_nfc_hw_ecc_write_page(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int sunxi_nfc_hw_ecc_write_page(struct nand_chip *chip,
const uint8_t *buf, int oob_required,
int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
int ret, i, cur_off = 0;
return nand_prog_page_end_op(chip);
}
-static int sunxi_nfc_hw_ecc_write_subpage(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int sunxi_nfc_hw_ecc_write_subpage(struct nand_chip *chip,
u32 data_offs, u32 data_len,
const u8 *buf, int oob_required,
int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
int ret, i, cur_off = 0;
return nand_prog_page_end_op(chip);
}
-static int sunxi_nfc_hw_ecc_write_page_dma(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int sunxi_nfc_hw_ecc_write_page_dma(struct nand_chip *chip,
const u8 *buf,
int oob_required,
int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct nand_chip *nand = mtd_to_nand(mtd);
struct sunxi_nfc *nfc = to_sunxi_nfc(nand->controller);
struct nand_ecc_ctrl *ecc = &nand->ecc;
return nand_prog_page_end_op(chip);
pio_fallback:
- return sunxi_nfc_hw_ecc_write_page(mtd, chip, buf, oob_required, page);
+ return sunxi_nfc_hw_ecc_write_page(chip, buf, oob_required, page);
}
-static int sunxi_nfc_hw_ecc_read_oob(struct mtd_info *mtd,
- struct nand_chip *chip,
- int page)
+static int sunxi_nfc_hw_ecc_read_oob(struct nand_chip *chip, int page)
{
chip->pagebuf = -1;
- return chip->ecc.read_page(mtd, chip, chip->data_buf, 1, page);
+ return chip->ecc.read_page(chip, chip->data_buf, 1, page);
}
-static int sunxi_nfc_hw_ecc_write_oob(struct mtd_info *mtd,
- struct nand_chip *chip,
- int page)
+static int sunxi_nfc_hw_ecc_write_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
int ret;
chip->pagebuf = -1;
memset(chip->data_buf, 0xff, mtd->writesize);
- ret = chip->ecc.write_page(mtd, chip, chip->data_buf, 1, page);
+ ret = chip->ecc.write_page(chip, chip->data_buf, 1, page);
if (ret)
return ret;
#define sunxi_nand_lookup_timing(l, p, c) \
_sunxi_nand_lookup_timing(l, ARRAY_SIZE(l), p, c)
-static int sunxi_nfc_setup_data_interface(struct mtd_info *mtd, int csline,
+static int sunxi_nfc_setup_data_interface(struct nand_chip *nand, int csline,
const struct nand_data_interface *conf)
{
- struct nand_chip *nand = mtd_to_nand(mtd);
struct sunxi_nand_chip *chip = to_sunxi_nand(nand);
struct sunxi_nfc *nfc = to_sunxi_nfc(chip->nand.controller);
const struct nand_sdr_timings *timings;
nand = &chip->nand;
/* Default tR value specified in the ONFI spec (chapter 4.15.1) */
- nand->chip_delay = 200;
+ nand->legacy.chip_delay = 200;
nand->controller = &nfc->controller;
nand->controller->ops = &sunxi_nand_controller_ops;
nand->ecc.mode = NAND_ECC_HW;
nand_set_flash_node(nand, np);
nand->select_chip = sunxi_nfc_select_chip;
- nand->cmd_ctrl = sunxi_nfc_cmd_ctrl;
- nand->read_buf = sunxi_nfc_read_buf;
- nand->write_buf = sunxi_nfc_write_buf;
- nand->read_byte = sunxi_nfc_read_byte;
+ nand->legacy.cmd_ctrl = sunxi_nfc_cmd_ctrl;
+ nand->legacy.read_buf = sunxi_nfc_read_buf;
+ nand->legacy.write_buf = sunxi_nfc_write_buf;
+ nand->legacy.read_byte = sunxi_nfc_read_byte;
nand->setup_data_interface = sunxi_nfc_setup_data_interface;
mtd = nand_to_mtd(nand);
mtd->dev.parent = dev;
- ret = nand_scan(mtd, nsels);
+ ret = nand_scan(nand, nsels);
if (ret)
return ret;
ret = mtd_device_register(mtd, NULL, 0);
if (ret) {
dev_err(dev, "failed to register mtd device: %d\n", ret);
- nand_release(mtd);
+ nand_release(nand);
return ret;
}
while (!list_empty(&nfc->chips)) {
chip = list_first_entry(&nfc->chips, struct sunxi_nand_chip,
node);
- nand_release(nand_to_mtd(&chip->nand));
+ nand_release(&chip->nand);
sunxi_nand_ecc_cleanup(&chip->nand.ecc);
list_del(&chip->node);
}
#define TIMING(t0, t1, t2, t3) ((t0) << 24 | (t1) << 16 | (t2) << 8 | (t3))
-static void tango_cmd_ctrl(struct mtd_info *mtd, int dat, unsigned int ctrl)
+static void tango_cmd_ctrl(struct nand_chip *chip, int dat, unsigned int ctrl)
{
- struct tango_chip *tchip = to_tango_chip(mtd_to_nand(mtd));
+ struct tango_chip *tchip = to_tango_chip(chip);
if (ctrl & NAND_CLE)
writeb_relaxed(dat, tchip->base + PBUS_CMD);
writeb_relaxed(dat, tchip->base + PBUS_ADDR);
}
-static int tango_dev_ready(struct mtd_info *mtd)
+static int tango_dev_ready(struct nand_chip *chip)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct tango_nfc *nfc = to_tango_nfc(chip->controller);
return readl_relaxed(nfc->pbus_base + PBUS_CS_CTRL) & PBUS_IORDY;
}
-static u8 tango_read_byte(struct mtd_info *mtd)
+static u8 tango_read_byte(struct nand_chip *chip)
{
- struct tango_chip *tchip = to_tango_chip(mtd_to_nand(mtd));
+ struct tango_chip *tchip = to_tango_chip(chip);
return readb_relaxed(tchip->base + PBUS_DATA);
}
-static void tango_read_buf(struct mtd_info *mtd, u8 *buf, int len)
+static void tango_read_buf(struct nand_chip *chip, u8 *buf, int len)
{
- struct tango_chip *tchip = to_tango_chip(mtd_to_nand(mtd));
+ struct tango_chip *tchip = to_tango_chip(chip);
ioread8_rep(tchip->base + PBUS_DATA, buf, len);
}
-static void tango_write_buf(struct mtd_info *mtd, const u8 *buf, int len)
+static void tango_write_buf(struct nand_chip *chip, const u8 *buf, int len)
{
- struct tango_chip *tchip = to_tango_chip(mtd_to_nand(mtd));
+ struct tango_chip *tchip = to_tango_chip(chip);
iowrite8_rep(tchip->base + PBUS_DATA, buf, len);
}
-static void tango_select_chip(struct mtd_info *mtd, int idx)
+static void tango_select_chip(struct nand_chip *chip, int idx)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct tango_nfc *nfc = to_tango_nfc(chip->controller);
struct tango_chip *tchip = to_tango_chip(chip);
return err;
}
-static int tango_read_page(struct mtd_info *mtd, struct nand_chip *chip,
- u8 *buf, int oob_required, int page)
+static int tango_read_page(struct nand_chip *chip, u8 *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct tango_nfc *nfc = to_tango_nfc(chip->controller);
int err, res, len = mtd->writesize;
if (oob_required)
- chip->ecc.read_oob(mtd, chip, page);
+ chip->ecc.read_oob(chip, page);
err = do_dma(nfc, DMA_FROM_DEVICE, NFC_READ, buf, len, page);
if (err)
res = decode_error_report(chip);
if (res < 0) {
- chip->ecc.read_oob_raw(mtd, chip, page);
+ chip->ecc.read_oob_raw(chip, page);
res = check_erased_page(chip, buf);
}
return res;
}
-static int tango_write_page(struct mtd_info *mtd, struct nand_chip *chip,
- const u8 *buf, int oob_required, int page)
+static int tango_write_page(struct nand_chip *chip, const u8 *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct tango_nfc *nfc = to_tango_nfc(chip->controller);
int err, status, len = mtd->writesize;
if (err)
return err;
- status = chip->waitfunc(mtd, chip);
+ status = chip->legacy.waitfunc(chip);
if (status & NAND_STATUS_FAIL)
return -EIO;
static void aux_read(struct nand_chip *chip, u8 **buf, int len, int *pos)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
-
*pos += len;
if (!*buf) {
/* skip over "len" bytes */
nand_change_read_column_op(chip, *pos, NULL, 0, false);
} else {
- tango_read_buf(mtd, *buf, len);
+ tango_read_buf(chip, *buf, len);
*buf += len;
}
}
static void aux_write(struct nand_chip *chip, const u8 **buf, int len, int *pos)
{
- struct mtd_info *mtd = nand_to_mtd(chip);
-
*pos += len;
if (!*buf) {
/* skip over "len" bytes */
nand_change_write_column_op(chip, *pos, NULL, 0, false);
} else {
- tango_write_buf(mtd, *buf, len);
+ tango_write_buf(chip, *buf, len);
*buf += len;
}
}
aux_write(chip, &oob, ecc_size, &pos);
}
-static int tango_read_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- u8 *buf, int oob_required, int page)
+static int tango_read_page_raw(struct nand_chip *chip, u8 *buf,
+ int oob_required, int page)
{
nand_read_page_op(chip, page, 0, NULL, 0);
raw_read(chip, buf, chip->oob_poi);
return 0;
}
-static int tango_write_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- const u8 *buf, int oob_required, int page)
+static int tango_write_page_raw(struct nand_chip *chip, const u8 *buf,
+ int oob_required, int page)
{
nand_prog_page_begin_op(chip, page, 0, NULL, 0);
raw_write(chip, buf, chip->oob_poi);
return nand_prog_page_end_op(chip);
}
-static int tango_read_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int tango_read_oob(struct nand_chip *chip, int page)
{
nand_read_page_op(chip, page, 0, NULL, 0);
raw_read(chip, NULL, chip->oob_poi);
return 0;
}
-static int tango_write_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int tango_write_oob(struct nand_chip *chip, int page)
{
nand_prog_page_begin_op(chip, page, 0, NULL, 0);
raw_write(chip, NULL, chip->oob_poi);
return DIV_ROUND_UP_ULL((u64)kHz * ps, NSEC_PER_SEC);
}
-static int tango_set_timings(struct mtd_info *mtd, int csline,
+static int tango_set_timings(struct nand_chip *chip, int csline,
const struct nand_data_interface *conf)
{
const struct nand_sdr_timings *sdr = nand_get_sdr_timings(conf);
- struct nand_chip *chip = mtd_to_nand(mtd);
struct tango_nfc *nfc = to_tango_nfc(chip->controller);
struct tango_chip *tchip = to_tango_chip(chip);
u32 Trdy, Textw, Twc, Twpw, Tacc, Thold, Trpw, Textr;
ecc = &chip->ecc;
mtd = nand_to_mtd(chip);
- chip->read_byte = tango_read_byte;
- chip->write_buf = tango_write_buf;
- chip->read_buf = tango_read_buf;
+ chip->legacy.read_byte = tango_read_byte;
+ chip->legacy.write_buf = tango_write_buf;
+ chip->legacy.read_buf = tango_read_buf;
chip->select_chip = tango_select_chip;
- chip->cmd_ctrl = tango_cmd_ctrl;
- chip->dev_ready = tango_dev_ready;
+ chip->legacy.cmd_ctrl = tango_cmd_ctrl;
+ chip->legacy.dev_ready = tango_dev_ready;
chip->setup_data_interface = tango_set_timings;
chip->options = NAND_USE_BOUNCE_BUFFER |
NAND_NO_SUBPAGE_WRITE |
mtd_set_ooblayout(mtd, &tango_nand_ooblayout_ops);
mtd->dev.parent = dev;
- err = nand_scan(mtd, 1);
+ err = nand_scan(chip, 1);
if (err)
return err;
for (cs = 0; cs < MAX_CS; ++cs) {
if (nfc->chips[cs])
- nand_release(nand_to_mtd(&nfc->chips[cs]->nand_chip));
+ nand_release(&nfc->chips[cs]->nand_chip);
}
return 0;
check_only);
}
-static void tegra_nand_select_chip(struct mtd_info *mtd, int die_nr)
+static void tegra_nand_select_chip(struct nand_chip *chip, int die_nr)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct tegra_nand_chip *nand = to_tegra_chip(chip);
struct tegra_nand_controller *ctrl = to_tegra_ctrl(chip->controller);
return ret;
}
-static int tegra_nand_read_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip, u8 *buf,
+static int tegra_nand_read_page_raw(struct nand_chip *chip, u8 *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
void *oob_buf = oob_required ? chip->oob_poi : NULL;
return tegra_nand_page_xfer(mtd, chip, buf, oob_buf,
mtd->oobsize, page, true);
}
-static int tegra_nand_write_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip, const u8 *buf,
+static int tegra_nand_write_page_raw(struct nand_chip *chip, const u8 *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
void *oob_buf = oob_required ? chip->oob_poi : NULL;
return tegra_nand_page_xfer(mtd, chip, (void *)buf, oob_buf,
mtd->oobsize, page, false);
}
-static int tegra_nand_read_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int tegra_nand_read_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
return tegra_nand_page_xfer(mtd, chip, NULL, chip->oob_poi,
mtd->oobsize, page, true);
}
-static int tegra_nand_write_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int tegra_nand_write_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
return tegra_nand_page_xfer(mtd, chip, NULL, chip->oob_poi,
mtd->oobsize, page, false);
}
-static int tegra_nand_read_page_hwecc(struct mtd_info *mtd,
- struct nand_chip *chip, u8 *buf,
+static int tegra_nand_read_page_hwecc(struct nand_chip *chip, u8 *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct tegra_nand_controller *ctrl = to_tegra_ctrl(chip->controller);
struct tegra_nand_chip *nand = to_tegra_chip(chip);
void *oob_buf = oob_required ? chip->oob_poi : NULL;
* erased or if error correction just failed for all sub-
* pages.
*/
- ret = tegra_nand_read_oob(mtd, chip, page);
+ ret = tegra_nand_read_oob(chip, page);
if (ret < 0)
return ret;
}
}
-static int tegra_nand_write_page_hwecc(struct mtd_info *mtd,
- struct nand_chip *chip, const u8 *buf,
+static int tegra_nand_write_page_hwecc(struct nand_chip *chip, const u8 *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct tegra_nand_controller *ctrl = to_tegra_ctrl(chip->controller);
void *oob_buf = oob_required ? chip->oob_poi : NULL;
int ret;
writel_relaxed(reg, ctrl->regs + TIMING_2);
}
-static int tegra_nand_setup_data_interface(struct mtd_info *mtd, int csline,
+static int tegra_nand_setup_data_interface(struct nand_chip *chip, int csline,
const struct nand_data_interface *conf)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct tegra_nand_controller *ctrl = to_tegra_ctrl(chip->controller);
const struct nand_sdr_timings *timings;
chip->select_chip = tegra_nand_select_chip;
chip->setup_data_interface = tegra_nand_setup_data_interface;
- ret = nand_scan(mtd, 1);
+ ret = nand_scan(chip, 1);
if (ret)
return ret;
/*--------------------------------------------------------------------------*/
-static void tmio_nand_hwcontrol(struct mtd_info *mtd, int cmd,
- unsigned int ctrl)
+static void tmio_nand_hwcontrol(struct nand_chip *chip, int cmd,
+ unsigned int ctrl)
{
- struct tmio_nand *tmio = mtd_to_tmio(mtd);
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct tmio_nand *tmio = mtd_to_tmio(nand_to_mtd(chip));
if (ctrl & NAND_CTRL_CHANGE) {
u8 mode;
}
if (cmd != NAND_CMD_NONE)
- tmio_iowrite8(cmd, chip->IO_ADDR_W);
+ tmio_iowrite8(cmd, chip->legacy.IO_ADDR_W);
}
-static int tmio_nand_dev_ready(struct mtd_info *mtd)
+static int tmio_nand_dev_ready(struct nand_chip *chip)
{
- struct tmio_nand *tmio = mtd_to_tmio(mtd);
+ struct tmio_nand *tmio = mtd_to_tmio(nand_to_mtd(chip));
return !(tmio_ioread8(tmio->fcr + FCR_STATUS) & FCR_STATUS_BUSY);
}
*erase and write, we enable it to wake us up. The irq handler
*disables the interrupt.
*/
-static int
-tmio_nand_wait(struct mtd_info *mtd, struct nand_chip *nand_chip)
+static int tmio_nand_wait(struct nand_chip *nand_chip)
{
- struct tmio_nand *tmio = mtd_to_tmio(mtd);
+ struct tmio_nand *tmio = mtd_to_tmio(nand_to_mtd(nand_chip));
long timeout;
u8 status;
tmio_iowrite8(0x81, tmio->fcr + FCR_IMR);
timeout = wait_event_timeout(nand_chip->controller->wq,
- tmio_nand_dev_ready(mtd),
+ tmio_nand_dev_ready(nand_chip),
msecs_to_jiffies(nand_chip->state == FL_ERASING ? 400 : 20));
- if (unlikely(!tmio_nand_dev_ready(mtd))) {
+ if (unlikely(!tmio_nand_dev_ready(nand_chip))) {
tmio_iowrite8(0x00, tmio->fcr + FCR_IMR);
dev_warn(&tmio->dev->dev, "still busy with %s after %d ms\n",
nand_chip->state == FL_ERASING ? "erase" : "program",
*To prevent stale data from being read, tmio_nand_hwcontrol() clears
*tmio->read_good.
*/
-static u_char tmio_nand_read_byte(struct mtd_info *mtd)
+static u_char tmio_nand_read_byte(struct nand_chip *chip)
{
- struct tmio_nand *tmio = mtd_to_tmio(mtd);
+ struct tmio_nand *tmio = mtd_to_tmio(nand_to_mtd(chip));
unsigned int data;
if (tmio->read_good--)
*buffer functions.
*/
static void
-tmio_nand_write_buf(struct mtd_info *mtd, const u_char *buf, int len)
+tmio_nand_write_buf(struct nand_chip *chip, const u_char *buf, int len)
{
- struct tmio_nand *tmio = mtd_to_tmio(mtd);
+ struct tmio_nand *tmio = mtd_to_tmio(nand_to_mtd(chip));
tmio_iowrite16_rep(tmio->fcr + FCR_DATA, buf, len >> 1);
}
-static void tmio_nand_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void tmio_nand_read_buf(struct nand_chip *chip, u_char *buf, int len)
{
- struct tmio_nand *tmio = mtd_to_tmio(mtd);
+ struct tmio_nand *tmio = mtd_to_tmio(nand_to_mtd(chip));
tmio_ioread16_rep(tmio->fcr + FCR_DATA, buf, len >> 1);
}
-static void tmio_nand_enable_hwecc(struct mtd_info *mtd, int mode)
+static void tmio_nand_enable_hwecc(struct nand_chip *chip, int mode)
{
- struct tmio_nand *tmio = mtd_to_tmio(mtd);
+ struct tmio_nand *tmio = mtd_to_tmio(nand_to_mtd(chip));
tmio_iowrite8(FCR_MODE_HWECC_RESET, tmio->fcr + FCR_MODE);
tmio_ioread8(tmio->fcr + FCR_DATA); /* dummy read */
tmio_iowrite8(FCR_MODE_HWECC_CALC, tmio->fcr + FCR_MODE);
}
-static int tmio_nand_calculate_ecc(struct mtd_info *mtd, const u_char *dat,
- u_char *ecc_code)
+static int tmio_nand_calculate_ecc(struct nand_chip *chip, const u_char *dat,
+ u_char *ecc_code)
{
- struct tmio_nand *tmio = mtd_to_tmio(mtd);
+ struct tmio_nand *tmio = mtd_to_tmio(nand_to_mtd(chip));
unsigned int ecc;
tmio_iowrite8(FCR_MODE_HWECC_RESULT, tmio->fcr + FCR_MODE);
return 0;
}
-static int tmio_nand_correct_data(struct mtd_info *mtd, unsigned char *buf,
- unsigned char *read_ecc, unsigned char *calc_ecc)
+static int tmio_nand_correct_data(struct nand_chip *chip, unsigned char *buf,
+ unsigned char *read_ecc,
+ unsigned char *calc_ecc)
{
int r0, r1;
/* assume ecc.size = 512 and ecc.bytes = 6 */
- r0 = __nand_correct_data(buf, read_ecc, calc_ecc, 256);
+ r0 = __nand_correct_data(buf, read_ecc, calc_ecc, 256, false);
if (r0 < 0)
return r0;
- r1 = __nand_correct_data(buf + 256, read_ecc + 3, calc_ecc + 3, 256);
+ r1 = __nand_correct_data(buf + 256, read_ecc + 3, calc_ecc + 3, 256,
+ false);
if (r1 < 0)
return r1;
return r0 + r1;
return retval;
/* Set address of NAND IO lines */
- nand_chip->IO_ADDR_R = tmio->fcr;
- nand_chip->IO_ADDR_W = tmio->fcr;
+ nand_chip->legacy.IO_ADDR_R = tmio->fcr;
+ nand_chip->legacy.IO_ADDR_W = tmio->fcr;
/* Set address of hardware control function */
- nand_chip->cmd_ctrl = tmio_nand_hwcontrol;
- nand_chip->dev_ready = tmio_nand_dev_ready;
- nand_chip->read_byte = tmio_nand_read_byte;
- nand_chip->write_buf = tmio_nand_write_buf;
- nand_chip->read_buf = tmio_nand_read_buf;
+ nand_chip->legacy.cmd_ctrl = tmio_nand_hwcontrol;
+ nand_chip->legacy.dev_ready = tmio_nand_dev_ready;
+ nand_chip->legacy.read_byte = tmio_nand_read_byte;
+ nand_chip->legacy.write_buf = tmio_nand_write_buf;
+ nand_chip->legacy.read_buf = tmio_nand_read_buf;
/* set eccmode using hardware ECC */
nand_chip->ecc.mode = NAND_ECC_HW;
nand_chip->badblock_pattern = data->badblock_pattern;
/* 15 us command delay time */
- nand_chip->chip_delay = 15;
+ nand_chip->legacy.chip_delay = 15;
retval = devm_request_irq(&dev->dev, irq, &tmio_irq, 0,
dev_name(&dev->dev), tmio);
}
tmio->irq = irq;
- nand_chip->waitfunc = tmio_nand_wait;
+ nand_chip->legacy.waitfunc = tmio_nand_wait;
/* Scan to find existence of the device */
- retval = nand_scan(mtd, 1);
+ retval = nand_scan(nand_chip, 1);
if (retval)
goto err_irq;
if (!retval)
return retval;
- nand_release(mtd);
+ nand_release(nand_chip);
err_irq:
tmio_hw_stop(dev, tmio);
{
struct tmio_nand *tmio = platform_get_drvdata(dev);
- nand_release(nand_to_mtd(&tmio->chip));
+ nand_release(&tmio->chip);
tmio_hw_stop(dev, tmio);
return 0;
}
__raw_writel(val, ndregaddr(dev, reg));
}
-static uint8_t txx9ndfmc_read_byte(struct mtd_info *mtd)
+static uint8_t txx9ndfmc_read_byte(struct nand_chip *chip)
{
- struct platform_device *dev = mtd_to_platdev(mtd);
+ struct platform_device *dev = mtd_to_platdev(nand_to_mtd(chip));
return txx9ndfmc_read(dev, TXX9_NDFDTR);
}
-static void txx9ndfmc_write_buf(struct mtd_info *mtd, const uint8_t *buf,
+static void txx9ndfmc_write_buf(struct nand_chip *chip, const uint8_t *buf,
int len)
{
- struct platform_device *dev = mtd_to_platdev(mtd);
+ struct platform_device *dev = mtd_to_platdev(nand_to_mtd(chip));
void __iomem *ndfdtr = ndregaddr(dev, TXX9_NDFDTR);
u32 mcr = txx9ndfmc_read(dev, TXX9_NDFMCR);
txx9ndfmc_write(dev, mcr, TXX9_NDFMCR);
}
-static void txx9ndfmc_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
+static void txx9ndfmc_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
{
- struct platform_device *dev = mtd_to_platdev(mtd);
+ struct platform_device *dev = mtd_to_platdev(nand_to_mtd(chip));
void __iomem *ndfdtr = ndregaddr(dev, TXX9_NDFDTR);
while (len--)
*buf++ = __raw_readl(ndfdtr);
}
-static void txx9ndfmc_cmd_ctrl(struct mtd_info *mtd, int cmd,
+static void txx9ndfmc_cmd_ctrl(struct nand_chip *chip, int cmd,
unsigned int ctrl)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct txx9ndfmc_priv *txx9_priv = nand_get_controller_data(chip);
struct platform_device *dev = txx9_priv->dev;
struct txx9ndfmc_platform_data *plat = dev_get_platdata(&dev->dev);
mmiowb();
}
-static int txx9ndfmc_dev_ready(struct mtd_info *mtd)
+static int txx9ndfmc_dev_ready(struct nand_chip *chip)
{
- struct platform_device *dev = mtd_to_platdev(mtd);
+ struct platform_device *dev = mtd_to_platdev(nand_to_mtd(chip));
return !(txx9ndfmc_read(dev, TXX9_NDFSR) & TXX9_NDFSR_BUSY);
}
-static int txx9ndfmc_calculate_ecc(struct mtd_info *mtd, const uint8_t *dat,
+static int txx9ndfmc_calculate_ecc(struct nand_chip *chip, const uint8_t *dat,
uint8_t *ecc_code)
{
- struct platform_device *dev = mtd_to_platdev(mtd);
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct platform_device *dev = mtd_to_platdev(nand_to_mtd(chip));
int eccbytes;
u32 mcr = txx9ndfmc_read(dev, TXX9_NDFMCR);
return 0;
}
-static int txx9ndfmc_correct_data(struct mtd_info *mtd, unsigned char *buf,
- unsigned char *read_ecc, unsigned char *calc_ecc)
+static int txx9ndfmc_correct_data(struct nand_chip *chip, unsigned char *buf,
+ unsigned char *read_ecc,
+ unsigned char *calc_ecc)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
int eccsize;
int corrected = 0;
int stat;
for (eccsize = chip->ecc.size; eccsize > 0; eccsize -= 256) {
- stat = __nand_correct_data(buf, read_ecc, calc_ecc, 256);
+ stat = __nand_correct_data(buf, read_ecc, calc_ecc, 256,
+ false);
if (stat < 0)
return stat;
corrected += stat;
return corrected;
}
-static void txx9ndfmc_enable_hwecc(struct mtd_info *mtd, int mode)
+static void txx9ndfmc_enable_hwecc(struct nand_chip *chip, int mode)
{
- struct platform_device *dev = mtd_to_platdev(mtd);
+ struct platform_device *dev = mtd_to_platdev(nand_to_mtd(chip));
u32 mcr = txx9ndfmc_read(dev, TXX9_NDFMCR);
mcr &= ~TXX9_NDFMCR_ECC_ALL;
mtd = nand_to_mtd(chip);
mtd->dev.parent = &dev->dev;
- chip->read_byte = txx9ndfmc_read_byte;
- chip->read_buf = txx9ndfmc_read_buf;
- chip->write_buf = txx9ndfmc_write_buf;
- chip->cmd_ctrl = txx9ndfmc_cmd_ctrl;
- chip->dev_ready = txx9ndfmc_dev_ready;
+ chip->legacy.read_byte = txx9ndfmc_read_byte;
+ chip->legacy.read_buf = txx9ndfmc_read_buf;
+ chip->legacy.write_buf = txx9ndfmc_write_buf;
+ chip->legacy.cmd_ctrl = txx9ndfmc_cmd_ctrl;
+ chip->legacy.dev_ready = txx9ndfmc_dev_ready;
chip->ecc.calculate = txx9ndfmc_calculate_ecc;
chip->ecc.correct = txx9ndfmc_correct_data;
chip->ecc.hwctl = txx9ndfmc_enable_hwecc;
chip->ecc.mode = NAND_ECC_HW;
chip->ecc.strength = 1;
- chip->chip_delay = 100;
+ chip->legacy.chip_delay = 100;
chip->controller = &drvdata->controller;
nand_set_controller_data(chip, txx9_priv);
if (plat->wide_mask & (1 << i))
chip->options |= NAND_BUSWIDTH_16;
- if (nand_scan(mtd, 1)) {
+ if (nand_scan(chip, 1)) {
kfree(txx9_priv->mtdname);
kfree(txx9_priv);
continue;
chip = mtd_to_nand(mtd);
txx9_priv = nand_get_controller_data(chip);
- nand_release(mtd);
+ nand_release(chip);
kfree(txx9_priv->mtdname);
kfree(txx9_priv);
}
/*
* This function supports Vybrid only (MPC5125 would have full RB and four CS)
*/
-static void vf610_nfc_select_chip(struct mtd_info *mtd, int chip)
+static void vf610_nfc_select_chip(struct nand_chip *chip, int cs)
{
- struct vf610_nfc *nfc = mtd_to_nfc(mtd);
+ struct vf610_nfc *nfc = mtd_to_nfc(nand_to_mtd(chip));
u32 tmp = vf610_nfc_read(nfc, NFC_ROW_ADDR);
/* Vybrid only (MPC5125 would have full RB and four CS) */
tmp &= ~(ROW_ADDR_CHIP_SEL_RB_MASK | ROW_ADDR_CHIP_SEL_MASK);
- if (chip >= 0) {
+ if (cs >= 0) {
tmp |= 1 << ROW_ADDR_CHIP_SEL_RB_SHIFT;
- tmp |= BIT(chip) << ROW_ADDR_CHIP_SEL_SHIFT;
+ tmp |= BIT(cs) << ROW_ADDR_CHIP_SEL_SHIFT;
}
vf610_nfc_write(nfc, NFC_ROW_ADDR, tmp);
}
}
-static int vf610_nfc_read_page(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page)
+static int vf610_nfc_read_page(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct vf610_nfc *nfc = mtd_to_nfc(mtd);
int trfr_sz = mtd->writesize + mtd->oobsize;
u32 row = 0, cmd1 = 0, cmd2 = 0, code = 0;
}
}
-static int vf610_nfc_write_page(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page)
+static int vf610_nfc_write_page(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct vf610_nfc *nfc = mtd_to_nfc(mtd);
int trfr_sz = mtd->writesize + mtd->oobsize;
u32 row = 0, cmd1 = 0, cmd2 = 0, code = 0;
return 0;
}
-static int vf610_nfc_read_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip, u8 *buf,
+static int vf610_nfc_read_page_raw(struct nand_chip *chip, u8 *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct vf610_nfc *nfc = mtd_to_nfc(mtd);
int ret;
nfc->data_access = true;
- ret = nand_read_page_raw(mtd, chip, buf, oob_required, page);
+ ret = nand_read_page_raw(chip, buf, oob_required, page);
nfc->data_access = false;
return ret;
}
-static int vf610_nfc_write_page_raw(struct mtd_info *mtd,
- struct nand_chip *chip, const u8 *buf,
+static int vf610_nfc_write_page_raw(struct nand_chip *chip, const u8 *buf,
int oob_required, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct vf610_nfc *nfc = mtd_to_nfc(mtd);
int ret;
return nand_prog_page_end_op(chip);
}
-static int vf610_nfc_read_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int vf610_nfc_read_oob(struct nand_chip *chip, int page)
{
- struct vf610_nfc *nfc = mtd_to_nfc(mtd);
+ struct vf610_nfc *nfc = mtd_to_nfc(nand_to_mtd(chip));
int ret;
nfc->data_access = true;
- ret = nand_read_oob_std(mtd, chip, page);
+ ret = nand_read_oob_std(chip, page);
nfc->data_access = false;
return ret;
}
-static int vf610_nfc_write_oob(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+static int vf610_nfc_write_oob(struct nand_chip *chip, int page)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct vf610_nfc *nfc = mtd_to_nfc(mtd);
int ret;
/* Scan the NAND chip */
chip->dummy_controller.ops = &vf610_nfc_controller_ops;
- err = nand_scan(mtd, 1);
+ err = nand_scan(chip, 1);
if (err)
goto err_disable_clk;
struct mtd_info *mtd = platform_get_drvdata(pdev);
struct vf610_nfc *nfc = mtd_to_nfc(mtd);
- nand_release(mtd);
+ nand_release(mtd_to_nand(mtd));
clk_disable_unprepare(nfc->clk);
return 0;
}
writeb(value, data->nandaddr + op);
}
-static void xway_select_chip(struct mtd_info *mtd, int select)
+static void xway_select_chip(struct nand_chip *chip, int select)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
struct xway_nand_data *data = nand_get_controller_data(chip);
switch (select) {
}
}
-static void xway_cmd_ctrl(struct mtd_info *mtd, int cmd, unsigned int ctrl)
+static void xway_cmd_ctrl(struct nand_chip *chip, int cmd, unsigned int ctrl)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
+
if (cmd == NAND_CMD_NONE)
return;
;
}
-static int xway_dev_ready(struct mtd_info *mtd)
+static int xway_dev_ready(struct nand_chip *chip)
{
return ltq_ebu_r32(EBU_NAND_WAIT) & NAND_WAIT_RD;
}
-static unsigned char xway_read_byte(struct mtd_info *mtd)
+static unsigned char xway_read_byte(struct nand_chip *chip)
{
- return xway_readb(mtd, NAND_READ_DATA);
+ return xway_readb(nand_to_mtd(chip), NAND_READ_DATA);
}
-static void xway_read_buf(struct mtd_info *mtd, u_char *buf, int len)
+static void xway_read_buf(struct nand_chip *chip, u_char *buf, int len)
{
int i;
for (i = 0; i < len; i++)
- buf[i] = xway_readb(mtd, NAND_WRITE_DATA);
+ buf[i] = xway_readb(nand_to_mtd(chip), NAND_WRITE_DATA);
}
-static void xway_write_buf(struct mtd_info *mtd, const u_char *buf, int len)
+static void xway_write_buf(struct nand_chip *chip, const u_char *buf, int len)
{
int i;
for (i = 0; i < len; i++)
- xway_writeb(mtd, NAND_WRITE_DATA, buf[i]);
+ xway_writeb(nand_to_mtd(chip), NAND_WRITE_DATA, buf[i]);
}
/*
mtd = nand_to_mtd(&data->chip);
mtd->dev.parent = &pdev->dev;
- data->chip.cmd_ctrl = xway_cmd_ctrl;
- data->chip.dev_ready = xway_dev_ready;
+ data->chip.legacy.cmd_ctrl = xway_cmd_ctrl;
+ data->chip.legacy.dev_ready = xway_dev_ready;
data->chip.select_chip = xway_select_chip;
- data->chip.write_buf = xway_write_buf;
- data->chip.read_buf = xway_read_buf;
- data->chip.read_byte = xway_read_byte;
- data->chip.chip_delay = 30;
+ data->chip.legacy.write_buf = xway_write_buf;
+ data->chip.legacy.read_buf = xway_read_buf;
+ data->chip.legacy.read_byte = xway_read_byte;
+ data->chip.legacy.chip_delay = 30;
data->chip.ecc.mode = NAND_ECC_SOFT;
data->chip.ecc.algo = NAND_ECC_HAMMING;
| cs_flag, EBU_NAND_CON);
/* Scan to find existence of the device */
- err = nand_scan(mtd, 1);
+ err = nand_scan(&data->chip, 1);
if (err)
return err;
err = mtd_device_register(mtd, NULL, 0);
if (err)
- nand_release(mtd);
+ nand_release(&data->chip);
return err;
}
{
struct xway_nand_data *data = platform_get_drvdata(pdev);
- nand_release(nand_to_mtd(&data->chip));
+ nand_release(&data->chip);
return 0;
}
{
uint8_t ecc[3];
- __nand_calculate_ecc(buffer, SM_SMALL_PAGE, ecc);
- if (__nand_correct_data(buffer, ecc, oob->ecc1, SM_SMALL_PAGE) < 0)
+ __nand_calculate_ecc(buffer, SM_SMALL_PAGE, ecc,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC));
+ if (__nand_correct_data(buffer, ecc, oob->ecc1, SM_SMALL_PAGE,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC)) < 0)
return -EIO;
buffer += SM_SMALL_PAGE;
- __nand_calculate_ecc(buffer, SM_SMALL_PAGE, ecc);
- if (__nand_correct_data(buffer, ecc, oob->ecc2, SM_SMALL_PAGE) < 0)
+ __nand_calculate_ecc(buffer, SM_SMALL_PAGE, ecc,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC));
+ if (__nand_correct_data(buffer, ecc, oob->ecc2, SM_SMALL_PAGE,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC)) < 0)
return -EIO;
return 0;
}
}
if (ftl->smallpagenand) {
- __nand_calculate_ecc(buf + boffset,
- SM_SMALL_PAGE, oob.ecc1);
+ __nand_calculate_ecc(buf + boffset, SM_SMALL_PAGE,
+ oob.ecc1,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC));
__nand_calculate_ecc(buf + boffset + SM_SMALL_PAGE,
- SM_SMALL_PAGE, oob.ecc2);
+ SM_SMALL_PAGE, oob.ecc2,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC));
}
if (!sm_write_sector(ftl, zone, block, boffset,
buf + boffset, &oob))
return 0;
}
- dma_dst = dma_map_single(nor->dev, buf, len, DMA_DEV_TO_MEM);
+ dma_dst = dma_map_single(nor->dev, buf, len, DMA_FROM_DEVICE);
if (dma_mapping_error(nor->dev, dma_dst)) {
dev_err(nor->dev, "dma mapping failed\n");
return -ENOMEM;
}
err_unmap:
- dma_unmap_single(nor->dev, dma_dst, len, DMA_DEV_TO_MEM);
+ dma_unmap_single(nor->dev, dma_dst, len, DMA_FROM_DEVICE);
return 0;
}
{
switch (cmd) {
case SPINOR_OP_READ_1_1_4:
+ case SPINOR_OP_READ_1_1_4_4B:
return SEQID_READ;
case SPINOR_OP_WREN:
return SEQID_WREN;
/* trigger the LUT now */
seqid = fsl_qspi_get_seqid(q, cmd);
+ if (seqid < 0)
+ return seqid;
+
qspi_writel(q, (seqid << QUADSPI_IPCR_SEQID_SHIFT) | len,
base + QUADSPI_IPCR);
* causes the controller to clear the buffer, and use the sequence pointed
* by the QUADSPI_BFGENCR[SEQID] to initiate a read from the flash.
*/
-static void fsl_qspi_init_ahb_read(struct fsl_qspi *q)
+static int fsl_qspi_init_ahb_read(struct fsl_qspi *q)
{
void __iomem *base = q->iobase;
int seqid;
/* Set the default lut sequence for AHB Read. */
seqid = fsl_qspi_get_seqid(q, q->nor[0].read_opcode);
+ if (seqid < 0)
+ return seqid;
+
qspi_writel(q, seqid << QUADSPI_BFGENCR_SEQID_SHIFT,
q->iobase + QUADSPI_BFGENCR);
+
+ return 0;
}
/* This function was used to prepare and enable QSPI clock */
fsl_qspi_init_lut(q);
/* Init for AHB read */
- fsl_qspi_init_ahb_read(q);
-
- return 0;
+ return fsl_qspi_init_ahb_read(q);
}
static const struct of_device_id fsl_qspi_dt_ids[] = {
static const struct pci_device_id intel_spi_pci_ids[] = {
{ PCI_VDEVICE(INTEL, 0x18e0), (unsigned long)&bxt_info },
{ PCI_VDEVICE(INTEL, 0x19e0), (unsigned long)&bxt_info },
+ { PCI_VDEVICE(INTEL, 0x34a4), (unsigned long)&bxt_info },
{ PCI_VDEVICE(INTEL, 0xa1a4), (unsigned long)&bxt_info },
{ PCI_VDEVICE(INTEL, 0xa224), (unsigned long)&bxt_info },
{ },
#include <linux/math64.h>
#include <linux/sizes.h>
#include <linux/slab.h>
+#include <linux/sort.h>
#include <linux/mtd/mtd.h>
#include <linux/of_platform.h>
nor->read_opcode = spi_nor_convert_3to4_read(nor->read_opcode);
nor->program_opcode = spi_nor_convert_3to4_program(nor->program_opcode);
nor->erase_opcode = spi_nor_convert_3to4_erase(nor->erase_opcode);
+
+ if (!spi_nor_has_uniform_erase(nor)) {
+ struct spi_nor_erase_map *map = &nor->erase_map;
+ struct spi_nor_erase_type *erase;
+ int i;
+
+ for (i = 0; i < SNOR_ERASE_TYPE_MAX; i++) {
+ erase = &map->erase_type[i];
+ erase->opcode =
+ spi_nor_convert_3to4_erase(erase->opcode);
+ }
+ }
}
/* Enable/disable 4-byte addressing mode. */
return nor->write_reg(nor, nor->erase_opcode, buf, nor->addr_width);
}
+/**
+ * spi_nor_div_by_erase_size() - calculate remainder and update new dividend
+ * @erase: pointer to a structure that describes a SPI NOR erase type
+ * @dividend: dividend value
+ * @remainder: pointer to u32 remainder (will be updated)
+ *
+ * Return: the result of the division
+ */
+static u64 spi_nor_div_by_erase_size(const struct spi_nor_erase_type *erase,
+ u64 dividend, u32 *remainder)
+{
+ /* JEDEC JESD216B Standard imposes erase sizes to be power of 2. */
+ *remainder = (u32)dividend & erase->size_mask;
+ return dividend >> erase->size_shift;
+}
+
+/**
+ * spi_nor_find_best_erase_type() - find the best erase type for the given
+ * offset in the serial flash memory and the
+ * number of bytes to erase. The region in
+ * which the address fits is expected to be
+ * provided.
+ * @map: the erase map of the SPI NOR
+ * @region: pointer to a structure that describes a SPI NOR erase region
+ * @addr: offset in the serial flash memory
+ * @len: number of bytes to erase
+ *
+ * Return: a pointer to the best fitted erase type, NULL otherwise.
+ */
+static const struct spi_nor_erase_type *
+spi_nor_find_best_erase_type(const struct spi_nor_erase_map *map,
+ const struct spi_nor_erase_region *region,
+ u64 addr, u32 len)
+{
+ const struct spi_nor_erase_type *erase;
+ u32 rem;
+ int i;
+ u8 erase_mask = region->offset & SNOR_ERASE_TYPE_MASK;
+
+ /*
+ * Erase types are ordered by size, with the biggest erase type at
+ * index 0.
+ */
+ for (i = SNOR_ERASE_TYPE_MAX - 1; i >= 0; i--) {
+ /* Does the erase region support the tested erase type? */
+ if (!(erase_mask & BIT(i)))
+ continue;
+
+ erase = &map->erase_type[i];
+
+ /* Don't erase more than what the user has asked for. */
+ if (erase->size > len)
+ continue;
+
+ /* Alignment is not mandatory for overlaid regions */
+ if (region->offset & SNOR_OVERLAID_REGION)
+ return erase;
+
+ spi_nor_div_by_erase_size(erase, addr, &rem);
+ if (rem)
+ continue;
+ else
+ return erase;
+ }
+
+ return NULL;
+}
+
+/**
+ * spi_nor_region_next() - get the next spi nor region
+ * @region: pointer to a structure that describes a SPI NOR erase region
+ *
+ * Return: the next spi nor region or NULL if last region.
+ */
+static struct spi_nor_erase_region *
+spi_nor_region_next(struct spi_nor_erase_region *region)
+{
+ if (spi_nor_region_is_last(region))
+ return NULL;
+ region++;
+ return region;
+}
+
+/**
+ * spi_nor_find_erase_region() - find the region of the serial flash memory in
+ * which the offset fits
+ * @map: the erase map of the SPI NOR
+ * @addr: offset in the serial flash memory
+ *
+ * Return: a pointer to the spi_nor_erase_region struct, ERR_PTR(-errno)
+ * otherwise.
+ */
+static struct spi_nor_erase_region *
+spi_nor_find_erase_region(const struct spi_nor_erase_map *map, u64 addr)
+{
+ struct spi_nor_erase_region *region = map->regions;
+ u64 region_start = region->offset & ~SNOR_ERASE_FLAGS_MASK;
+ u64 region_end = region_start + region->size;
+
+ while (addr < region_start || addr >= region_end) {
+ region = spi_nor_region_next(region);
+ if (!region)
+ return ERR_PTR(-EINVAL);
+
+ region_start = region->offset & ~SNOR_ERASE_FLAGS_MASK;
+ region_end = region_start + region->size;
+ }
+
+ return region;
+}
+
+/**
+ * spi_nor_init_erase_cmd() - initialize an erase command
+ * @region: pointer to a structure that describes a SPI NOR erase region
+ * @erase: pointer to a structure that describes a SPI NOR erase type
+ *
+ * Return: the pointer to the allocated erase command, ERR_PTR(-errno)
+ * otherwise.
+ */
+static struct spi_nor_erase_command *
+spi_nor_init_erase_cmd(const struct spi_nor_erase_region *region,
+ const struct spi_nor_erase_type *erase)
+{
+ struct spi_nor_erase_command *cmd;
+
+ cmd = kmalloc(sizeof(*cmd), GFP_KERNEL);
+ if (!cmd)
+ return ERR_PTR(-ENOMEM);
+
+ INIT_LIST_HEAD(&cmd->list);
+ cmd->opcode = erase->opcode;
+ cmd->count = 1;
+
+ if (region->offset & SNOR_OVERLAID_REGION)
+ cmd->size = region->size;
+ else
+ cmd->size = erase->size;
+
+ return cmd;
+}
+
+/**
+ * spi_nor_destroy_erase_cmd_list() - destroy erase command list
+ * @erase_list: list of erase commands
+ */
+static void spi_nor_destroy_erase_cmd_list(struct list_head *erase_list)
+{
+ struct spi_nor_erase_command *cmd, *next;
+
+ list_for_each_entry_safe(cmd, next, erase_list, list) {
+ list_del(&cmd->list);
+ kfree(cmd);
+ }
+}
+
+/**
+ * spi_nor_init_erase_cmd_list() - initialize erase command list
+ * @nor: pointer to a 'struct spi_nor'
+ * @erase_list: list of erase commands to be executed once we validate that the
+ * erase can be performed
+ * @addr: offset in the serial flash memory
+ * @len: number of bytes to erase
+ *
+ * Builds the list of best fitted erase commands and verifies if the erase can
+ * be performed.
+ *
+ * Return: 0 on success, -errno otherwise.
+ */
+static int spi_nor_init_erase_cmd_list(struct spi_nor *nor,
+ struct list_head *erase_list,
+ u64 addr, u32 len)
+{
+ const struct spi_nor_erase_map *map = &nor->erase_map;
+ const struct spi_nor_erase_type *erase, *prev_erase = NULL;
+ struct spi_nor_erase_region *region;
+ struct spi_nor_erase_command *cmd = NULL;
+ u64 region_end;
+ int ret = -EINVAL;
+
+ region = spi_nor_find_erase_region(map, addr);
+ if (IS_ERR(region))
+ return PTR_ERR(region);
+
+ region_end = spi_nor_region_end(region);
+
+ while (len) {
+ erase = spi_nor_find_best_erase_type(map, region, addr, len);
+ if (!erase)
+ goto destroy_erase_cmd_list;
+
+ if (prev_erase != erase ||
+ region->offset & SNOR_OVERLAID_REGION) {
+ cmd = spi_nor_init_erase_cmd(region, erase);
+ if (IS_ERR(cmd)) {
+ ret = PTR_ERR(cmd);
+ goto destroy_erase_cmd_list;
+ }
+
+ list_add_tail(&cmd->list, erase_list);
+ } else {
+ cmd->count++;
+ }
+
+ addr += cmd->size;
+ len -= cmd->size;
+
+ if (len && addr >= region_end) {
+ region = spi_nor_region_next(region);
+ if (!region)
+ goto destroy_erase_cmd_list;
+ region_end = spi_nor_region_end(region);
+ }
+
+ prev_erase = erase;
+ }
+
+ return 0;
+
+destroy_erase_cmd_list:
+ spi_nor_destroy_erase_cmd_list(erase_list);
+ return ret;
+}
+
+/**
+ * spi_nor_erase_multi_sectors() - perform a non-uniform erase
+ * @nor: pointer to a 'struct spi_nor'
+ * @addr: offset in the serial flash memory
+ * @len: number of bytes to erase
+ *
+ * Build a list of best fitted erase commands and execute it once we validate
+ * that the erase can be performed.
+ *
+ * Return: 0 on success, -errno otherwise.
+ */
+static int spi_nor_erase_multi_sectors(struct spi_nor *nor, u64 addr, u32 len)
+{
+ LIST_HEAD(erase_list);
+ struct spi_nor_erase_command *cmd, *next;
+ int ret;
+
+ ret = spi_nor_init_erase_cmd_list(nor, &erase_list, addr, len);
+ if (ret)
+ return ret;
+
+ list_for_each_entry_safe(cmd, next, &erase_list, list) {
+ nor->erase_opcode = cmd->opcode;
+ while (cmd->count) {
+ write_enable(nor);
+
+ ret = spi_nor_erase_sector(nor, addr);
+ if (ret)
+ goto destroy_erase_cmd_list;
+
+ addr += cmd->size;
+ cmd->count--;
+
+ ret = spi_nor_wait_till_ready(nor);
+ if (ret)
+ goto destroy_erase_cmd_list;
+ }
+ list_del(&cmd->list);
+ kfree(cmd);
+ }
+
+ return 0;
+
+destroy_erase_cmd_list:
+ spi_nor_destroy_erase_cmd_list(&erase_list);
+ return ret;
+}
+
/*
* Erase an address range on the nor chip. The address range may extend
* one or more erase sectors. Return an error is there is a problem erasing.
dev_dbg(nor->dev, "at 0x%llx, len %lld\n", (long long)instr->addr,
(long long)instr->len);
- div_u64_rem(instr->len, mtd->erasesize, &rem);
- if (rem)
- return -EINVAL;
+ if (spi_nor_has_uniform_erase(nor)) {
+ div_u64_rem(instr->len, mtd->erasesize, &rem);
+ if (rem)
+ return -EINVAL;
+ }
addr = instr->addr;
len = instr->len;
*/
/* "sector"-at-a-time erase */
- } else {
+ } else if (spi_nor_has_uniform_erase(nor)) {
while (len) {
write_enable(nor);
if (ret)
goto erase_err;
}
+
+ /* erase multiple sectors */
+ } else {
+ ret = spi_nor_erase_multi_sectors(nor, addr, len);
+ if (ret)
+ goto erase_err;
}
write_disable(nor);
goto write_err;
*retlen += written;
i += written;
- if (written != page_remain) {
- dev_err(nor->dev,
- "While writing %zu bytes written %zd bytes\n",
- page_remain, written);
- ret = -EIO;
- goto write_err;
- }
}
write_err:
* Serial Flash Discoverable Parameters (SFDP) parsing.
*/
+/**
+ * spi_nor_read_raw() - raw read of serial flash memory. read_opcode,
+ * addr_width and read_dummy members of the struct spi_nor
+ * should be previously
+ * set.
+ * @nor: pointer to a 'struct spi_nor'
+ * @addr: offset in the serial flash memory
+ * @len: number of bytes to read
+ * @buf: buffer where the data is copied into
+ *
+ * Return: 0 on success, -errno otherwise.
+ */
+static int spi_nor_read_raw(struct spi_nor *nor, u32 addr, size_t len, u8 *buf)
+{
+ int ret;
+
+ while (len) {
+ ret = nor->read(nor, addr, len, buf);
+ if (!ret || ret > len)
+ return -EIO;
+ if (ret < 0)
+ return ret;
+
+ buf += ret;
+ addr += ret;
+ len -= ret;
+ }
+ return 0;
+}
+
/**
* spi_nor_read_sfdp() - read Serial Flash Discoverable Parameters.
* @nor: pointer to a 'struct spi_nor'
nor->addr_width = 3;
nor->read_dummy = 8;
- while (len) {
- ret = nor->read(nor, addr, len, (u8 *)buf);
- if (!ret || ret > len) {
- ret = -EIO;
- goto read_err;
- }
- if (ret < 0)
- goto read_err;
-
- buf += ret;
- addr += ret;
- len -= ret;
- }
- ret = 0;
+ ret = spi_nor_read_raw(nor, addr, len, buf);
-read_err:
nor->read_opcode = read_opcode;
nor->addr_width = addr_width;
nor->read_dummy = read_dummy;
static int spi_nor_hwcaps_read2cmd(u32 hwcaps);
+/**
+ * spi_nor_set_erase_type() - set a SPI NOR erase type
+ * @erase: pointer to a structure that describes a SPI NOR erase type
+ * @size: the size of the sector/block erased by the erase type
+ * @opcode: the SPI command op code to erase the sector/block
+ */
+static void spi_nor_set_erase_type(struct spi_nor_erase_type *erase,
+ u32 size, u8 opcode)
+{
+ erase->size = size;
+ erase->opcode = opcode;
+ /* JEDEC JESD216B Standard imposes erase sizes to be power of 2. */
+ erase->size_shift = ffs(erase->size) - 1;
+ erase->size_mask = (1 << erase->size_shift) - 1;
+}
+
+/**
+ * spi_nor_set_erase_settings_from_bfpt() - set erase type settings from BFPT
+ * @erase: pointer to a structure that describes a SPI NOR erase type
+ * @size: the size of the sector/block erased by the erase type
+ * @opcode: the SPI command op code to erase the sector/block
+ * @i: erase type index as sorted in the Basic Flash Parameter Table
+ *
+ * The supported Erase Types will be sorted at init in ascending order, with
+ * the smallest Erase Type size being the first member in the erase_type array
+ * of the spi_nor_erase_map structure. Save the Erase Type index as sorted in
+ * the Basic Flash Parameter Table since it will be used later on to
+ * synchronize with the supported Erase Types defined in SFDP optional tables.
+ */
+static void
+spi_nor_set_erase_settings_from_bfpt(struct spi_nor_erase_type *erase,
+ u32 size, u8 opcode, u8 i)
+{
+ erase->idx = i;
+ spi_nor_set_erase_type(erase, size, opcode);
+}
+
+/**
+ * spi_nor_map_cmp_erase_type() - compare the map's erase types by size
+ * @l: member in the left half of the map's erase_type array
+ * @r: member in the right half of the map's erase_type array
+ *
+ * Comparison function used in the sort() call to sort in ascending order the
+ * map's erase types, the smallest erase type size being the first member in the
+ * sorted erase_type array.
+ *
+ * Return: the result of @l->size - @r->size
+ */
+static int spi_nor_map_cmp_erase_type(const void *l, const void *r)
+{
+ const struct spi_nor_erase_type *left = l, *right = r;
+
+ return left->size - right->size;
+}
+
+/**
+ * spi_nor_regions_sort_erase_types() - sort erase types in each region
+ * @map: the erase map of the SPI NOR
+ *
+ * Function assumes that the erase types defined in the erase map are already
+ * sorted in ascending order, with the smallest erase type size being the first
+ * member in the erase_type array. It replicates the sort done for the map's
+ * erase types. Each region's erase bitmask will indicate which erase types are
+ * supported from the sorted erase types defined in the erase map.
+ * Sort the all region's erase type at init in order to speed up the process of
+ * finding the best erase command at runtime.
+ */
+static void spi_nor_regions_sort_erase_types(struct spi_nor_erase_map *map)
+{
+ struct spi_nor_erase_region *region = map->regions;
+ struct spi_nor_erase_type *erase_type = map->erase_type;
+ int i;
+ u8 region_erase_mask, sorted_erase_mask;
+
+ while (region) {
+ region_erase_mask = region->offset & SNOR_ERASE_TYPE_MASK;
+
+ /* Replicate the sort done for the map's erase types. */
+ sorted_erase_mask = 0;
+ for (i = 0; i < SNOR_ERASE_TYPE_MAX; i++)
+ if (erase_type[i].size &&
+ region_erase_mask & BIT(erase_type[i].idx))
+ sorted_erase_mask |= BIT(i);
+
+ /* Overwrite erase mask. */
+ region->offset = (region->offset & ~SNOR_ERASE_TYPE_MASK) |
+ sorted_erase_mask;
+
+ region = spi_nor_region_next(region);
+ }
+}
+
+/**
+ * spi_nor_init_uniform_erase_map() - Initialize uniform erase map
+ * @map: the erase map of the SPI NOR
+ * @erase_mask: bitmask encoding erase types that can erase the entire
+ * flash memory
+ * @flash_size: the spi nor flash memory size
+ */
+static void spi_nor_init_uniform_erase_map(struct spi_nor_erase_map *map,
+ u8 erase_mask, u64 flash_size)
+{
+ /* Offset 0 with erase_mask and SNOR_LAST_REGION bit set */
+ map->uniform_region.offset = (erase_mask & SNOR_ERASE_TYPE_MASK) |
+ SNOR_LAST_REGION;
+ map->uniform_region.size = flash_size;
+ map->regions = &map->uniform_region;
+ map->uniform_erase_type = erase_mask;
+}
+
/**
* spi_nor_parse_bfpt() - read and parse the Basic Flash Parameter Table.
* @nor: pointer to a 'struct spi_nor'
const struct sfdp_parameter_header *bfpt_header,
struct spi_nor_flash_parameter *params)
{
- struct mtd_info *mtd = &nor->mtd;
+ struct spi_nor_erase_map *map = &nor->erase_map;
+ struct spi_nor_erase_type *erase_type = map->erase_type;
struct sfdp_bfpt bfpt;
size_t len;
int i, cmd, err;
u32 addr;
u16 half;
+ u8 erase_mask;
/* JESD216 Basic Flash Parameter Table length is at least 9 DWORDs. */
if (bfpt_header->length < BFPT_DWORD_MAX_JESD216)
spi_nor_set_read_settings_from_bfpt(read, half, rd->proto);
}
- /* Sector Erase settings. */
+ /*
+ * Sector Erase settings. Reinitialize the uniform erase map using the
+ * Erase Types defined in the bfpt table.
+ */
+ erase_mask = 0;
+ memset(&nor->erase_map, 0, sizeof(nor->erase_map));
for (i = 0; i < ARRAY_SIZE(sfdp_bfpt_erases); i++) {
const struct sfdp_bfpt_erase *er = &sfdp_bfpt_erases[i];
u32 erasesize;
erasesize = 1U << erasesize;
opcode = (half >> 8) & 0xff;
-#ifdef CONFIG_MTD_SPI_NOR_USE_4K_SECTORS
- if (erasesize == SZ_4K) {
- nor->erase_opcode = opcode;
- mtd->erasesize = erasesize;
- break;
- }
-#endif
- if (!mtd->erasesize || mtd->erasesize < erasesize) {
- nor->erase_opcode = opcode;
- mtd->erasesize = erasesize;
- }
+ erase_mask |= BIT(i);
+ spi_nor_set_erase_settings_from_bfpt(&erase_type[i], erasesize,
+ opcode, i);
}
+ spi_nor_init_uniform_erase_map(map, erase_mask, params->size);
+ /*
+ * Sort all the map's Erase Types in ascending order with the smallest
+ * erase size being the first member in the erase_type array.
+ */
+ sort(erase_type, SNOR_ERASE_TYPE_MAX, sizeof(erase_type[0]),
+ spi_nor_map_cmp_erase_type, NULL);
+ /*
+ * Sort the erase types in the uniform region in order to update the
+ * uniform_erase_type bitmask. The bitmask will be used later on when
+ * selecting the uniform erase.
+ */
+ spi_nor_regions_sort_erase_types(map);
+ map->uniform_erase_type = map->uniform_region.offset &
+ SNOR_ERASE_TYPE_MASK;
/* Stop here if not JESD216 rev A or later. */
if (bfpt_header->length < BFPT_DWORD_MAX)
return 0;
}
+#define SMPT_CMD_ADDRESS_LEN_MASK GENMASK(23, 22)
+#define SMPT_CMD_ADDRESS_LEN_0 (0x0UL << 22)
+#define SMPT_CMD_ADDRESS_LEN_3 (0x1UL << 22)
+#define SMPT_CMD_ADDRESS_LEN_4 (0x2UL << 22)
+#define SMPT_CMD_ADDRESS_LEN_USE_CURRENT (0x3UL << 22)
+
+#define SMPT_CMD_READ_DUMMY_MASK GENMASK(19, 16)
+#define SMPT_CMD_READ_DUMMY_SHIFT 16
+#define SMPT_CMD_READ_DUMMY(_cmd) \
+ (((_cmd) & SMPT_CMD_READ_DUMMY_MASK) >> SMPT_CMD_READ_DUMMY_SHIFT)
+#define SMPT_CMD_READ_DUMMY_IS_VARIABLE 0xfUL
+
+#define SMPT_CMD_READ_DATA_MASK GENMASK(31, 24)
+#define SMPT_CMD_READ_DATA_SHIFT 24
+#define SMPT_CMD_READ_DATA(_cmd) \
+ (((_cmd) & SMPT_CMD_READ_DATA_MASK) >> SMPT_CMD_READ_DATA_SHIFT)
+
+#define SMPT_CMD_OPCODE_MASK GENMASK(15, 8)
+#define SMPT_CMD_OPCODE_SHIFT 8
+#define SMPT_CMD_OPCODE(_cmd) \
+ (((_cmd) & SMPT_CMD_OPCODE_MASK) >> SMPT_CMD_OPCODE_SHIFT)
+
+#define SMPT_MAP_REGION_COUNT_MASK GENMASK(23, 16)
+#define SMPT_MAP_REGION_COUNT_SHIFT 16
+#define SMPT_MAP_REGION_COUNT(_header) \
+ ((((_header) & SMPT_MAP_REGION_COUNT_MASK) >> \
+ SMPT_MAP_REGION_COUNT_SHIFT) + 1)
+
+#define SMPT_MAP_ID_MASK GENMASK(15, 8)
+#define SMPT_MAP_ID_SHIFT 8
+#define SMPT_MAP_ID(_header) \
+ (((_header) & SMPT_MAP_ID_MASK) >> SMPT_MAP_ID_SHIFT)
+
+#define SMPT_MAP_REGION_SIZE_MASK GENMASK(31, 8)
+#define SMPT_MAP_REGION_SIZE_SHIFT 8
+#define SMPT_MAP_REGION_SIZE(_region) \
+ (((((_region) & SMPT_MAP_REGION_SIZE_MASK) >> \
+ SMPT_MAP_REGION_SIZE_SHIFT) + 1) * 256)
+
+#define SMPT_MAP_REGION_ERASE_TYPE_MASK GENMASK(3, 0)
+#define SMPT_MAP_REGION_ERASE_TYPE(_region) \
+ ((_region) & SMPT_MAP_REGION_ERASE_TYPE_MASK)
+
+#define SMPT_DESC_TYPE_MAP BIT(1)
+#define SMPT_DESC_END BIT(0)
+
+/**
+ * spi_nor_smpt_addr_width() - return the address width used in the
+ * configuration detection command.
+ * @nor: pointer to a 'struct spi_nor'
+ * @settings: configuration detection command descriptor, dword1
+ */
+static u8 spi_nor_smpt_addr_width(const struct spi_nor *nor, const u32 settings)
+{
+ switch (settings & SMPT_CMD_ADDRESS_LEN_MASK) {
+ case SMPT_CMD_ADDRESS_LEN_0:
+ return 0;
+ case SMPT_CMD_ADDRESS_LEN_3:
+ return 3;
+ case SMPT_CMD_ADDRESS_LEN_4:
+ return 4;
+ case SMPT_CMD_ADDRESS_LEN_USE_CURRENT:
+ /* fall through */
+ default:
+ return nor->addr_width;
+ }
+}
+
+/**
+ * spi_nor_smpt_read_dummy() - return the configuration detection command read
+ * latency, in clock cycles.
+ * @nor: pointer to a 'struct spi_nor'
+ * @settings: configuration detection command descriptor, dword1
+ *
+ * Return: the number of dummy cycles for an SMPT read
+ */
+static u8 spi_nor_smpt_read_dummy(const struct spi_nor *nor, const u32 settings)
+{
+ u8 read_dummy = SMPT_CMD_READ_DUMMY(settings);
+
+ if (read_dummy == SMPT_CMD_READ_DUMMY_IS_VARIABLE)
+ return nor->read_dummy;
+ return read_dummy;
+}
+
+/**
+ * spi_nor_get_map_in_use() - get the configuration map in use
+ * @nor: pointer to a 'struct spi_nor'
+ * @smpt: pointer to the sector map parameter table
+ */
+static const u32 *spi_nor_get_map_in_use(struct spi_nor *nor, const u32 *smpt)
+{
+ const u32 *ret = NULL;
+ u32 i, addr;
+ int err;
+ u8 addr_width, read_opcode, read_dummy;
+ u8 read_data_mask, data_byte, map_id;
+
+ addr_width = nor->addr_width;
+ read_dummy = nor->read_dummy;
+ read_opcode = nor->read_opcode;
+
+ map_id = 0;
+ i = 0;
+ /* Determine if there are any optional Detection Command Descriptors */
+ while (!(smpt[i] & SMPT_DESC_TYPE_MAP)) {
+ read_data_mask = SMPT_CMD_READ_DATA(smpt[i]);
+ nor->addr_width = spi_nor_smpt_addr_width(nor, smpt[i]);
+ nor->read_dummy = spi_nor_smpt_read_dummy(nor, smpt[i]);
+ nor->read_opcode = SMPT_CMD_OPCODE(smpt[i]);
+ addr = smpt[i + 1];
+
+ err = spi_nor_read_raw(nor, addr, 1, &data_byte);
+ if (err)
+ goto out;
+
+ /*
+ * Build an index value that is used to select the Sector Map
+ * Configuration that is currently in use.
+ */
+ map_id = map_id << 1 | !!(data_byte & read_data_mask);
+ i = i + 2;
+ }
+
+ /* Find the matching configuration map */
+ while (SMPT_MAP_ID(smpt[i]) != map_id) {
+ if (smpt[i] & SMPT_DESC_END)
+ goto out;
+ /* increment the table index to the next map */
+ i += SMPT_MAP_REGION_COUNT(smpt[i]) + 1;
+ }
+
+ ret = smpt + i;
+ /* fall through */
+out:
+ nor->addr_width = addr_width;
+ nor->read_dummy = read_dummy;
+ nor->read_opcode = read_opcode;
+ return ret;
+}
+
+/**
+ * spi_nor_region_check_overlay() - set overlay bit when the region is overlaid
+ * @region: pointer to a structure that describes a SPI NOR erase region
+ * @erase: pointer to a structure that describes a SPI NOR erase type
+ * @erase_type: erase type bitmask
+ */
+static void
+spi_nor_region_check_overlay(struct spi_nor_erase_region *region,
+ const struct spi_nor_erase_type *erase,
+ const u8 erase_type)
+{
+ int i;
+
+ for (i = 0; i < SNOR_ERASE_TYPE_MAX; i++) {
+ if (!(erase_type & BIT(i)))
+ continue;
+ if (region->size & erase[i].size_mask) {
+ spi_nor_region_mark_overlay(region);
+ return;
+ }
+ }
+}
+
+/**
+ * spi_nor_init_non_uniform_erase_map() - initialize the non-uniform erase map
+ * @nor: pointer to a 'struct spi_nor'
+ * @smpt: pointer to the sector map parameter table
+ *
+ * Return: 0 on success, -errno otherwise.
+ */
+static int spi_nor_init_non_uniform_erase_map(struct spi_nor *nor,
+ const u32 *smpt)
+{
+ struct spi_nor_erase_map *map = &nor->erase_map;
+ const struct spi_nor_erase_type *erase = map->erase_type;
+ struct spi_nor_erase_region *region;
+ u64 offset;
+ u32 region_count;
+ int i, j;
+ u8 erase_type;
+
+ region_count = SMPT_MAP_REGION_COUNT(*smpt);
+ /*
+ * The regions will be freed when the driver detaches from the
+ * device.
+ */
+ region = devm_kcalloc(nor->dev, region_count, sizeof(*region),
+ GFP_KERNEL);
+ if (!region)
+ return -ENOMEM;
+ map->regions = region;
+
+ map->uniform_erase_type = 0xff;
+ offset = 0;
+ /* Populate regions. */
+ for (i = 0; i < region_count; i++) {
+ j = i + 1; /* index for the region dword */
+ region[i].size = SMPT_MAP_REGION_SIZE(smpt[j]);
+ erase_type = SMPT_MAP_REGION_ERASE_TYPE(smpt[j]);
+ region[i].offset = offset | erase_type;
+
+ spi_nor_region_check_overlay(®ion[i], erase, erase_type);
+
+ /*
+ * Save the erase types that are supported in all regions and
+ * can erase the entire flash memory.
+ */
+ map->uniform_erase_type &= erase_type;
+
+ offset = (region[i].offset & ~SNOR_ERASE_FLAGS_MASK) +
+ region[i].size;
+ }
+
+ spi_nor_region_mark_end(®ion[i - 1]);
+
+ return 0;
+}
+
+/**
+ * spi_nor_parse_smpt() - parse Sector Map Parameter Table
+ * @nor: pointer to a 'struct spi_nor'
+ * @smpt_header: sector map parameter table header
+ *
+ * This table is optional, but when available, we parse it to identify the
+ * location and size of sectors within the main data array of the flash memory
+ * device and to identify which Erase Types are supported by each sector.
+ *
+ * Return: 0 on success, -errno otherwise.
+ */
+static int spi_nor_parse_smpt(struct spi_nor *nor,
+ const struct sfdp_parameter_header *smpt_header)
+{
+ const u32 *sector_map;
+ u32 *smpt;
+ size_t len;
+ u32 addr;
+ int i, ret;
+
+ /* Read the Sector Map Parameter Table. */
+ len = smpt_header->length * sizeof(*smpt);
+ smpt = kzalloc(len, GFP_KERNEL);
+ if (!smpt)
+ return -ENOMEM;
+
+ addr = SFDP_PARAM_HEADER_PTP(smpt_header);
+ ret = spi_nor_read_sfdp(nor, addr, len, smpt);
+ if (ret)
+ goto out;
+
+ /* Fix endianness of the SMPT DWORDs. */
+ for (i = 0; i < smpt_header->length; i++)
+ smpt[i] = le32_to_cpu(smpt[i]);
+
+ sector_map = spi_nor_get_map_in_use(nor, smpt);
+ if (!sector_map) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ret = spi_nor_init_non_uniform_erase_map(nor, sector_map);
+ if (ret)
+ goto out;
+
+ spi_nor_regions_sort_erase_types(&nor->erase_map);
+ /* fall through */
+out:
+ kfree(smpt);
+ return ret;
+}
+
/**
* spi_nor_parse_sfdp() - parse the Serial Flash Discoverable Parameters.
* @nor: pointer to a 'struct spi_nor'
switch (SFDP_PARAM_HEADER_ID(param_header)) {
case SFDP_SECTOR_MAP_ID:
- dev_info(dev, "non-uniform erase sector maps are not supported yet.\n");
+ err = spi_nor_parse_smpt(nor, param_header);
break;
default:
const struct flash_info *info,
struct spi_nor_flash_parameter *params)
{
+ struct spi_nor_erase_map *map = &nor->erase_map;
+ u8 i, erase_mask;
+
/* Set legacy flash parameters as default. */
memset(params, 0, sizeof(*params));
spi_nor_set_pp_settings(¶ms->page_programs[SNOR_CMD_PP],
SPINOR_OP_PP, SNOR_PROTO_1_1_1);
+ /*
+ * Sector Erase settings. Sort Erase Types in ascending order, with the
+ * smallest erase size starting at BIT(0).
+ */
+ erase_mask = 0;
+ i = 0;
+ if (info->flags & SECT_4K_PMC) {
+ erase_mask |= BIT(i);
+ spi_nor_set_erase_type(&map->erase_type[i], 4096u,
+ SPINOR_OP_BE_4K_PMC);
+ i++;
+ } else if (info->flags & SECT_4K) {
+ erase_mask |= BIT(i);
+ spi_nor_set_erase_type(&map->erase_type[i], 4096u,
+ SPINOR_OP_BE_4K);
+ i++;
+ }
+ erase_mask |= BIT(i);
+ spi_nor_set_erase_type(&map->erase_type[i], info->sector_size,
+ SPINOR_OP_SE);
+ spi_nor_init_uniform_erase_map(map, erase_mask, params->size);
+
/* Select the procedure to set the Quad Enable bit. */
if (params->hwcaps.mask & (SNOR_HWCAPS_READ_QUAD |
SNOR_HWCAPS_PP_QUAD)) {
params->quad_enable = info->quad_enable;
}
- /* Override the parameters with data read from SFDP tables. */
- nor->addr_width = 0;
- nor->mtd.erasesize = 0;
if ((info->flags & (SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ)) &&
!(info->flags & SPI_NOR_SKIP_SFDP)) {
struct spi_nor_flash_parameter sfdp_params;
+ struct spi_nor_erase_map prev_map;
memcpy(&sfdp_params, params, sizeof(sfdp_params));
- if (spi_nor_parse_sfdp(nor, &sfdp_params)) {
- nor->addr_width = 0;
- nor->mtd.erasesize = 0;
- } else {
+ memcpy(&prev_map, &nor->erase_map, sizeof(prev_map));
+
+ if (spi_nor_parse_sfdp(nor, &sfdp_params))
+ /* restore previous erase map */
+ memcpy(&nor->erase_map, &prev_map,
+ sizeof(nor->erase_map));
+ else
memcpy(params, &sfdp_params, sizeof(*params));
- }
}
return 0;
return 0;
}
-static int spi_nor_select_erase(struct spi_nor *nor,
- const struct flash_info *info)
+/**
+ * spi_nor_select_uniform_erase() - select optimum uniform erase type
+ * @map: the erase map of the SPI NOR
+ * @wanted_size: the erase type size to search for. Contains the value of
+ * info->sector_size or of the "small sector" size in case
+ * CONFIG_MTD_SPI_NOR_USE_4K_SECTORS is defined.
+ *
+ * Once the optimum uniform sector erase command is found, disable all the
+ * other.
+ *
+ * Return: pointer to erase type on success, NULL otherwise.
+ */
+static const struct spi_nor_erase_type *
+spi_nor_select_uniform_erase(struct spi_nor_erase_map *map,
+ const u32 wanted_size)
{
- struct mtd_info *mtd = &nor->mtd;
+ const struct spi_nor_erase_type *tested_erase, *erase = NULL;
+ int i;
+ u8 uniform_erase_type = map->uniform_erase_type;
- /* Do nothing if already configured from SFDP. */
- if (mtd->erasesize)
- return 0;
+ for (i = SNOR_ERASE_TYPE_MAX - 1; i >= 0; i--) {
+ if (!(uniform_erase_type & BIT(i)))
+ continue;
+
+ tested_erase = &map->erase_type[i];
+
+ /*
+ * If the current erase size is the one, stop here:
+ * we have found the right uniform Sector Erase command.
+ */
+ if (tested_erase->size == wanted_size) {
+ erase = tested_erase;
+ break;
+ }
+
+ /*
+ * Otherwise, the current erase size is still a valid canditate.
+ * Select the biggest valid candidate.
+ */
+ if (!erase && tested_erase->size)
+ erase = tested_erase;
+ /* keep iterating to find the wanted_size */
+ }
+
+ if (!erase)
+ return NULL;
+ /* Disable all other Sector Erase commands. */
+ map->uniform_erase_type &= ~SNOR_ERASE_TYPE_MASK;
+ map->uniform_erase_type |= BIT(erase - map->erase_type);
+ return erase;
+}
+
+static int spi_nor_select_erase(struct spi_nor *nor, u32 wanted_size)
+{
+ struct spi_nor_erase_map *map = &nor->erase_map;
+ const struct spi_nor_erase_type *erase = NULL;
+ struct mtd_info *mtd = &nor->mtd;
+ int i;
+
+ /*
+ * The previous implementation handling Sector Erase commands assumed
+ * that the SPI flash memory has an uniform layout then used only one
+ * of the supported erase sizes for all Sector Erase commands.
+ * So to be backward compatible, the new implementation also tries to
+ * manage the SPI flash memory as uniform with a single erase sector
+ * size, when possible.
+ */
#ifdef CONFIG_MTD_SPI_NOR_USE_4K_SECTORS
/* prefer "small sector" erase if possible */
- if (info->flags & SECT_4K) {
- nor->erase_opcode = SPINOR_OP_BE_4K;
- mtd->erasesize = 4096;
- } else if (info->flags & SECT_4K_PMC) {
- nor->erase_opcode = SPINOR_OP_BE_4K_PMC;
- mtd->erasesize = 4096;
- } else
+ wanted_size = 4096u;
#endif
- {
- nor->erase_opcode = SPINOR_OP_SE;
- mtd->erasesize = info->sector_size;
+
+ if (spi_nor_has_uniform_erase(nor)) {
+ erase = spi_nor_select_uniform_erase(map, wanted_size);
+ if (!erase)
+ return -EINVAL;
+ nor->erase_opcode = erase->opcode;
+ mtd->erasesize = erase->size;
+ return 0;
}
+
+ /*
+ * For non-uniform SPI flash memory, set mtd->erasesize to the
+ * maximum erase sector size. No need to set nor->erase_opcode.
+ */
+ for (i = SNOR_ERASE_TYPE_MAX - 1; i >= 0; i--) {
+ if (map->erase_type[i].size) {
+ erase = &map->erase_type[i];
+ break;
+ }
+ }
+
+ if (!erase)
+ return -EINVAL;
+
+ mtd->erasesize = erase->size;
return 0;
}
}
/* Select the Sector Erase command. */
- err = spi_nor_select_erase(nor, info);
+ err = spi_nor_select_erase(nor, info->sector_size);
if (err) {
dev_err(nor->dev,
"can't select erase settings supported by both the SPI controller and memory.\n");
unsigned char calc_ecc[3];
int ret;
- __nand_calculate_ecc(error_data, size, calc_ecc);
- ret = __nand_correct_data(error_data, error_ecc, calc_ecc, size);
+ __nand_calculate_ecc(error_data, size, calc_ecc,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC));
+ ret = __nand_correct_data(error_data, error_ecc, calc_ecc, size,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC));
if (ret == 0 && !memcmp(correct_data, error_data, size))
return 0;
unsigned char calc_ecc[3];
int ret;
- __nand_calculate_ecc(error_data, size, calc_ecc);
- ret = __nand_correct_data(error_data, error_ecc, calc_ecc, size);
+ __nand_calculate_ecc(error_data, size, calc_ecc,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC));
+ ret = __nand_correct_data(error_data, error_ecc, calc_ecc, size,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC));
if (ret == 1 && !memcmp(correct_data, error_data, size))
return 0;
unsigned char calc_ecc[3];
int ret;
- __nand_calculate_ecc(error_data, size, calc_ecc);
- ret = __nand_correct_data(error_data, error_ecc, calc_ecc, size);
+ __nand_calculate_ecc(error_data, size, calc_ecc,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC));
+ ret = __nand_correct_data(error_data, error_ecc, calc_ecc, size,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC));
return (ret == -EBADMSG) ? 0 : -EINVAL;
}
}
prandom_bytes(correct_data, size);
- __nand_calculate_ecc(correct_data, size, correct_ecc);
+ __nand_calculate_ecc(correct_data, size, correct_ecc,
+ IS_ENABLED(CONFIG_MTD_NAND_ECC_SMC));
for (i = 0; i < ARRAY_SIZE(nand_ecc_test); i++) {
nand_ecc_test[i].prepare(error_data, error_ecc,
MODULE_AUTHOR("Mircea Caprioru <mircea.caprioru@analog.com>");
MODULE_DESCRIPTION("Analog Devices ADGS1408 MUX driver");
-MODULE_LICENSE("GPL v2");
+MODULE_LICENSE("GPL");
struct mux_gpio {
struct gpio_descs *gpios;
- int *val;
};
static int mux_gpio_set(struct mux_control *mux, int state)
{
struct mux_gpio *mux_gpio = mux_chip_priv(mux->chip);
- int i;
+ DECLARE_BITMAP(values, BITS_PER_TYPE(state));
- for (i = 0; i < mux_gpio->gpios->ndescs; i++)
- mux_gpio->val[i] = (state >> i) & 1;
+ values[0] = state;
gpiod_set_array_value_cansleep(mux_gpio->gpios->ndescs,
mux_gpio->gpios->desc,
- mux_gpio->val);
+ mux_gpio->gpios->info, values);
return 0;
}
if (pins < 0)
return pins;
- mux_chip = devm_mux_chip_alloc(dev, 1, sizeof(*mux_gpio) +
- pins * sizeof(*mux_gpio->val));
+ mux_chip = devm_mux_chip_alloc(dev, 1, sizeof(*mux_gpio));
if (IS_ERR(mux_chip))
return PTR_ERR(mux_chip);
mux_gpio = mux_chip_priv(mux_chip);
- mux_gpio->val = (int *)(mux_gpio + 1);
mux_chip->ops = &mux_gpio_ops;
mux_gpio->gpios = devm_gpiod_get_array(dev, "mux", GPIOD_OUT_LOW);
static int bcm_sf2_sw_resume(struct dsa_switch *ds)
{
struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds);
- unsigned int port;
int ret;
ret = bcm_sf2_sw_rst(priv);
if (priv->hw_params.num_gphy == 1)
bcm_sf2_gphy_enable_set(ds, true);
- for (port = 0; port < DSA_MAX_PORTS; port++) {
- if (dsa_is_user_port(ds, port))
- bcm_sf2_port_setup(ds, port, NULL);
- else if (dsa_is_cpu_port(ds, port))
- bcm_sf2_imp_setup(ds, port);
- }
-
- bcm_sf2_enable_acb(ds);
+ ds->ops->setup(ds);
return 0;
}
{
struct bcm_sf2_priv *priv = platform_get_drvdata(pdev);
- /* Disable all ports and interrupts */
priv->wol_ports_mask = 0;
- bcm_sf2_sw_suspend(priv->dev->ds);
dsa_unregister_switch(priv->dev->ds);
+ /* Disable all ports and interrupts */
+ bcm_sf2_sw_suspend(priv->dev->ds);
bcm_sf2_mdio_unregister(priv);
return 0;
(cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_L4_PROTO_IDX_MASK) >>
ENA_ETH_IO_RX_CDESC_BASE_L4_PROTO_IDX_SHIFT;
ena_rx_ctx->l3_csum_err =
- (cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM_ERR_MASK) >>
- ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM_ERR_SHIFT;
+ !!((cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM_ERR_MASK) >>
+ ENA_ETH_IO_RX_CDESC_BASE_L3_CSUM_ERR_SHIFT);
ena_rx_ctx->l4_csum_err =
- (cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_ERR_MASK) >>
- ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_ERR_SHIFT;
+ !!((cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_ERR_MASK) >>
+ ENA_ETH_IO_RX_CDESC_BASE_L4_CSUM_ERR_SHIFT);
ena_rx_ctx->hash = cdesc->hash;
ena_rx_ctx->frag =
(cdesc->status & ENA_ETH_IO_RX_CDESC_BASE_IPV4_FRAG_MASK) >>
if (rc)
return rc;
- ena_init_napi(adapter);
-
ena_change_mtu(adapter->netdev, adapter->netdev->mtu);
ena_refill_all_rx_bufs(adapter);
ena_setup_io_intr(adapter);
+ /* napi poll functions should be initialized before running
+ * request_irq(), to handle a rare condition where there is a pending
+ * interrupt, causing the ISR to fire immediately while the poll
+ * function wasn't set yet, causing a null dereference
+ */
+ ena_init_napi(adapter);
+
rc = ena_request_io_irq(adapter);
if (rc)
goto err_req_irq;
ena_free_mgmnt_irq(adapter);
ena_disable_msix(adapter);
err_device_destroy:
+ ena_com_abort_admin_commands(ena_dev);
+ ena_com_wait_for_abort_completion(ena_dev);
ena_com_admin_destroy(ena_dev);
+ ena_com_mmio_reg_read_request_destroy(ena_dev);
+ ena_com_dev_reset(ena_dev, ENA_REGS_RESET_DRIVER_INVALID_STATE);
err:
clear_bit(ENA_FLAG_DEVICE_RUNNING, &adapter->flags);
clear_bit(ENA_FLAG_ONGOING_RESET, &adapter->flags);
static void ena_release_bars(struct ena_com_dev *ena_dev, struct pci_dev *pdev)
{
- int release_bars;
-
- if (ena_dev->mem_bar)
- devm_iounmap(&pdev->dev, ena_dev->mem_bar);
-
- if (ena_dev->reg_bar)
- devm_iounmap(&pdev->dev, ena_dev->reg_bar);
+ int release_bars = pci_select_bars(pdev, IORESOURCE_MEM) & ENA_BAR_MASK;
- release_bars = pci_select_bars(pdev, IORESOURCE_MEM) & ENA_BAR_MASK;
pci_release_selected_regions(pdev, release_bars);
}
phydev->advertising = phydev->supported;
/* The internal PHY has its link interrupts routed to the
- * Ethernet MAC ISRs
+ * Ethernet MAC ISRs. On GENETv5 there is a hardware issue
+ * that prevents the signaling of link UP interrupts when
+ * the link operates at 10Mbps, so fallback to polling for
+ * those versions of GENET.
*/
- if (priv->internal_phy)
+ if (priv->internal_phy && !GENET_IS_V5(priv))
dev->phydev->irq = PHY_IGNORE_INTERRUPT;
return 0;
* initialisation.
*/
#define FEC_QUIRK_MIB_CLEAR (1 << 15)
+/* Only i.MX25/i.MX27/i.MX28 controller supports FRBR,FRSR registers,
+ * those FIFO receive registers are resolved in other platforms.
+ */
+#define FEC_QUIRK_HAS_FRREG (1 << 16)
struct bufdesc_prop {
int qid;
.driver_data = 0,
}, {
.name = "imx25-fec",
- .driver_data = FEC_QUIRK_USE_GASKET | FEC_QUIRK_MIB_CLEAR,
+ .driver_data = FEC_QUIRK_USE_GASKET | FEC_QUIRK_MIB_CLEAR |
+ FEC_QUIRK_HAS_FRREG,
}, {
.name = "imx27-fec",
- .driver_data = FEC_QUIRK_MIB_CLEAR,
+ .driver_data = FEC_QUIRK_MIB_CLEAR | FEC_QUIRK_HAS_FRREG,
}, {
.name = "imx28-fec",
.driver_data = FEC_QUIRK_ENET_MAC | FEC_QUIRK_SWAP_FRAME |
- FEC_QUIRK_SINGLE_MDIO | FEC_QUIRK_HAS_RACC,
+ FEC_QUIRK_SINGLE_MDIO | FEC_QUIRK_HAS_RACC |
+ FEC_QUIRK_HAS_FRREG,
}, {
.name = "imx6q-fec",
.driver_data = FEC_QUIRK_ENET_MAC | FEC_QUIRK_HAS_GBIT |
memset(buf, 0, regs->len);
for (i = 0; i < ARRAY_SIZE(fec_enet_register_offset); i++) {
- off = fec_enet_register_offset[i] / 4;
+ off = fec_enet_register_offset[i];
+
+ if ((off == FEC_R_BOUND || off == FEC_R_FSTART) &&
+ !(fep->quirks & FEC_QUIRK_HAS_FRREG))
+ continue;
+
+ off >>= 2;
buf[off] = readl(&theregs[off]);
}
}
NULL, NULL, NULL),
};
-static void mlx4_devlink_set_init_value(struct devlink *devlink, u32 param_id,
- union devlink_param_value init_val)
-{
- struct mlx4_priv *priv = devlink_priv(devlink);
- struct mlx4_dev *dev = &priv->dev;
- int err;
-
- err = devlink_param_driverinit_value_set(devlink, param_id, init_val);
- if (err)
- mlx4_warn(dev,
- "devlink set parameter %u value failed (err = %d)",
- param_id, err);
-}
-
static void mlx4_devlink_set_params_init_values(struct devlink *devlink)
{
union devlink_param_value value;
value.vbool = !!mlx4_internal_err_reset;
- mlx4_devlink_set_init_value(devlink,
- DEVLINK_PARAM_GENERIC_ID_INT_ERR_RESET,
- value);
+ devlink_param_driverinit_value_set(devlink,
+ DEVLINK_PARAM_GENERIC_ID_INT_ERR_RESET,
+ value);
value.vu32 = 1UL << log_num_mac;
- mlx4_devlink_set_init_value(devlink,
- DEVLINK_PARAM_GENERIC_ID_MAX_MACS, value);
+ devlink_param_driverinit_value_set(devlink,
+ DEVLINK_PARAM_GENERIC_ID_MAX_MACS,
+ value);
value.vbool = enable_64b_cqe_eqe;
- mlx4_devlink_set_init_value(devlink,
- MLX4_DEVLINK_PARAM_ID_ENABLE_64B_CQE_EQE,
- value);
+ devlink_param_driverinit_value_set(devlink,
+ MLX4_DEVLINK_PARAM_ID_ENABLE_64B_CQE_EQE,
+ value);
value.vbool = enable_4k_uar;
- mlx4_devlink_set_init_value(devlink,
- MLX4_DEVLINK_PARAM_ID_ENABLE_4K_UAR,
- value);
+ devlink_param_driverinit_value_set(devlink,
+ MLX4_DEVLINK_PARAM_ID_ENABLE_4K_UAR,
+ value);
value.vbool = false;
- mlx4_devlink_set_init_value(devlink,
- DEVLINK_PARAM_GENERIC_ID_REGION_SNAPSHOT,
- value);
+ devlink_param_driverinit_value_set(devlink,
+ DEVLINK_PARAM_GENERIC_ID_REGION_SNAPSHOT,
+ value);
}
static inline void mlx4_set_num_reserved_uars(struct mlx4_dev *dev,
static inline void mlx5e_fill_icosq_frag_edge(struct mlx5e_icosq *sq,
struct mlx5_wq_cyc *wq,
- u16 pi, u16 frag_pi)
+ u16 pi, u16 nnops)
{
struct mlx5e_sq_wqe_info *edge_wi, *wi = &sq->db.ico_wqe[pi];
- u8 nnops = mlx5_wq_cyc_get_frag_size(wq) - frag_pi;
edge_wi = wi + nnops;
struct mlx5_wq_cyc *wq = &sq->wq;
struct mlx5e_umr_wqe *umr_wqe;
u16 xlt_offset = ix << (MLX5E_LOG_ALIGNED_MPWQE_PPW - 1);
- u16 pi, frag_pi;
+ u16 pi, contig_wqebbs_room;
int err;
int i;
pi = mlx5_wq_cyc_ctr2ix(wq, sq->pc);
- frag_pi = mlx5_wq_cyc_ctr2fragix(wq, sq->pc);
-
- if (unlikely(frag_pi + MLX5E_UMR_WQEBBS > mlx5_wq_cyc_get_frag_size(wq))) {
- mlx5e_fill_icosq_frag_edge(sq, wq, pi, frag_pi);
+ contig_wqebbs_room = mlx5_wq_cyc_get_contig_wqebbs(wq, pi);
+ if (unlikely(contig_wqebbs_room < MLX5E_UMR_WQEBBS)) {
+ mlx5e_fill_icosq_frag_edge(sq, wq, pi, contig_wqebbs_room);
pi = mlx5_wq_cyc_ctr2ix(wq, sq->pc);
}
static inline void mlx5e_fill_sq_frag_edge(struct mlx5e_txqsq *sq,
struct mlx5_wq_cyc *wq,
- u16 pi, u16 frag_pi)
+ u16 pi, u16 nnops)
{
struct mlx5e_tx_wqe_info *edge_wi, *wi = &sq->db.wqe_info[pi];
- u8 nnops = mlx5_wq_cyc_get_frag_size(wq) - frag_pi;
edge_wi = wi + nnops;
struct mlx5e_tx_wqe_info *wi;
struct mlx5e_sq_stats *stats = sq->stats;
+ u16 headlen, ihs, contig_wqebbs_room;
u16 ds_cnt, ds_cnt_inl = 0;
- u16 headlen, ihs, frag_pi;
u8 num_wqebbs, opcode;
u32 num_bytes;
int num_dma;
}
num_wqebbs = DIV_ROUND_UP(ds_cnt, MLX5_SEND_WQEBB_NUM_DS);
- frag_pi = mlx5_wq_cyc_ctr2fragix(wq, sq->pc);
- if (unlikely(frag_pi + num_wqebbs > mlx5_wq_cyc_get_frag_size(wq))) {
- mlx5e_fill_sq_frag_edge(sq, wq, pi, frag_pi);
+ contig_wqebbs_room = mlx5_wq_cyc_get_contig_wqebbs(wq, pi);
+ if (unlikely(contig_wqebbs_room < num_wqebbs)) {
+ mlx5e_fill_sq_frag_edge(sq, wq, pi, contig_wqebbs_room);
mlx5e_sq_fetch_wqe(sq, &wqe, &pi);
}
struct mlx5e_tx_wqe_info *wi;
struct mlx5e_sq_stats *stats = sq->stats;
- u16 headlen, ihs, pi, frag_pi;
+ u16 headlen, ihs, pi, contig_wqebbs_room;
u16 ds_cnt, ds_cnt_inl = 0;
u8 num_wqebbs, opcode;
u32 num_bytes;
}
num_wqebbs = DIV_ROUND_UP(ds_cnt, MLX5_SEND_WQEBB_NUM_DS);
- frag_pi = mlx5_wq_cyc_ctr2fragix(wq, sq->pc);
- if (unlikely(frag_pi + num_wqebbs > mlx5_wq_cyc_get_frag_size(wq))) {
+ pi = mlx5_wq_cyc_ctr2ix(wq, sq->pc);
+ contig_wqebbs_room = mlx5_wq_cyc_get_contig_wqebbs(wq, pi);
+ if (unlikely(contig_wqebbs_room < num_wqebbs)) {
+ mlx5e_fill_sq_frag_edge(sq, wq, pi, contig_wqebbs_room);
pi = mlx5_wq_cyc_ctr2ix(wq, sq->pc);
- mlx5e_fill_sq_frag_edge(sq, wq, pi, frag_pi);
}
- mlx5i_sq_fetch_wqe(sq, &wqe, &pi);
+ mlx5i_sq_fetch_wqe(sq, &wqe, pi);
/* fill wqe */
wi = &sq->db.wqe_info[pi];
case MLX5_PFAULT_SUBTYPE_WQE:
/* WQE based event */
pfault->type =
- be32_to_cpu(pf_eqe->wqe.pftype_wq) >> 24;
+ (be32_to_cpu(pf_eqe->wqe.pftype_wq) >> 24) & 0x7;
pfault->token =
be32_to_cpu(pf_eqe->wqe.token);
pfault->wqe.wq_num =
return ERR_PTR(res);
}
- /* Context will be freed by wait func after completion */
+ /* Context should be freed by the caller after completion. */
return context;
}
cmd.cmd = htonl(MLX5_FPGA_IPSEC_CMD_OP_SET_CAP);
cmd.flags = htonl(flags);
context = mlx5_fpga_ipsec_cmd_exec(mdev, &cmd, sizeof(cmd));
- if (IS_ERR(context)) {
- err = PTR_ERR(context);
- goto out;
- }
+ if (IS_ERR(context))
+ return PTR_ERR(context);
err = mlx5_fpga_ipsec_cmd_wait(context);
if (err)
}
out:
+ kfree(context);
return err;
}
static inline void mlx5i_sq_fetch_wqe(struct mlx5e_txqsq *sq,
struct mlx5i_tx_wqe **wqe,
- u16 *pi)
+ u16 pi)
{
struct mlx5_wq_cyc *wq = &sq->wq;
- *pi = mlx5_wq_cyc_ctr2ix(wq, sq->pc);
- *wqe = mlx5_wq_cyc_get_wqe(wq, *pi);
+ *wqe = mlx5_wq_cyc_get_wqe(wq, pi);
memset(*wqe, 0, sizeof(**wqe));
}
return (u32)wq->fbc.sz_m1 + 1;
}
-u16 mlx5_wq_cyc_get_frag_size(struct mlx5_wq_cyc *wq)
-{
- return wq->fbc.frag_sz_m1 + 1;
-}
-
u32 mlx5_cqwq_get_size(struct mlx5_cqwq *wq)
{
return wq->fbc.sz_m1 + 1;
void *wqc, struct mlx5_wq_cyc *wq,
struct mlx5_wq_ctrl *wq_ctrl);
u32 mlx5_wq_cyc_get_size(struct mlx5_wq_cyc *wq);
-u16 mlx5_wq_cyc_get_frag_size(struct mlx5_wq_cyc *wq);
int mlx5_wq_qp_create(struct mlx5_core_dev *mdev, struct mlx5_wq_param *param,
void *qpc, struct mlx5_wq_qp *wq,
return ctr & wq->fbc.sz_m1;
}
-static inline u16 mlx5_wq_cyc_ctr2fragix(struct mlx5_wq_cyc *wq, u16 ctr)
-{
- return ctr & wq->fbc.frag_sz_m1;
-}
-
static inline u16 mlx5_wq_cyc_get_head(struct mlx5_wq_cyc *wq)
{
return mlx5_wq_cyc_ctr2ix(wq, wq->wqe_ctr);
return mlx5_frag_buf_get_wqe(&wq->fbc, ix);
}
+static inline u16 mlx5_wq_cyc_get_contig_wqebbs(struct mlx5_wq_cyc *wq, u16 ix)
+{
+ return mlx5_frag_buf_get_idx_last_contig_stride(&wq->fbc, ix) - ix + 1;
+}
+
static inline int mlx5_wq_cyc_cc_bigger(u16 cc1, u16 cc2)
{
int equal = (cc1 == cc2);
err_driver_init:
mlxsw_thermal_fini(mlxsw_core->thermal);
err_thermal_init:
+ mlxsw_hwmon_fini(mlxsw_core->hwmon);
err_hwmon_init:
if (!reload)
devlink_unregister(devlink);
if (mlxsw_core->driver->fini)
mlxsw_core->driver->fini(mlxsw_core);
mlxsw_thermal_fini(mlxsw_core->thermal);
+ mlxsw_hwmon_fini(mlxsw_core->hwmon);
if (!reload)
devlink_unregister(devlink);
mlxsw_emad_fini(mlxsw_core);
return 0;
}
+static inline void mlxsw_hwmon_fini(struct mlxsw_hwmon *mlxsw_hwmon)
+{
+}
+
#endif
struct mlxsw_thermal;
struct device *hwmon_dev;
int err;
- mlxsw_hwmon = devm_kzalloc(mlxsw_bus_info->dev, sizeof(*mlxsw_hwmon),
- GFP_KERNEL);
+ mlxsw_hwmon = kzalloc(sizeof(*mlxsw_hwmon), GFP_KERNEL);
if (!mlxsw_hwmon)
return -ENOMEM;
mlxsw_hwmon->core = mlxsw_core;
mlxsw_hwmon->groups[0] = &mlxsw_hwmon->group;
mlxsw_hwmon->group.attrs = mlxsw_hwmon->attrs;
- hwmon_dev = devm_hwmon_device_register_with_groups(mlxsw_bus_info->dev,
- "mlxsw",
- mlxsw_hwmon,
- mlxsw_hwmon->groups);
+ hwmon_dev = hwmon_device_register_with_groups(mlxsw_bus_info->dev,
+ "mlxsw", mlxsw_hwmon,
+ mlxsw_hwmon->groups);
if (IS_ERR(hwmon_dev)) {
err = PTR_ERR(hwmon_dev);
goto err_hwmon_register;
err_hwmon_register:
err_fans_init:
err_temp_init:
+ kfree(mlxsw_hwmon);
return err;
}
+
+void mlxsw_hwmon_fini(struct mlxsw_hwmon *mlxsw_hwmon)
+{
+ hwmon_device_unregister(mlxsw_hwmon->hwmon_dev);
+ kfree(mlxsw_hwmon);
+}
{
unsigned int val, timeout = 10;
- /* Wait for the issued mac table command to be completed, or timeout.
- * When the command read from ANA_TABLES_MACACCESS is
- * MACACCESS_CMD_IDLE, the issued command completed successfully.
+ /* Wait for the issued vlan table command to be completed, or timeout.
+ * When the command read from ANA_TABLES_VLANACCESS is
+ * VLANACCESS_CMD_IDLE, the issued command completed successfully.
*/
do {
val = ocelot_read(ocelot, ANA_TABLES_VLANACCESS);
switch (off) {
case offsetof(struct iphdr, daddr):
- set_ip_addr->ipv4_dst_mask = mask;
- set_ip_addr->ipv4_dst = exact;
+ set_ip_addr->ipv4_dst_mask |= mask;
+ set_ip_addr->ipv4_dst &= ~mask;
+ set_ip_addr->ipv4_dst |= exact & mask;
break;
case offsetof(struct iphdr, saddr):
- set_ip_addr->ipv4_src_mask = mask;
- set_ip_addr->ipv4_src = exact;
+ set_ip_addr->ipv4_src_mask |= mask;
+ set_ip_addr->ipv4_src &= ~mask;
+ set_ip_addr->ipv4_src |= exact & mask;
break;
default:
return -EOPNOTSUPP;
}
static void
-nfp_fl_set_ip6_helper(int opcode_tag, int idx, __be32 exact, __be32 mask,
+nfp_fl_set_ip6_helper(int opcode_tag, u8 word, __be32 exact, __be32 mask,
struct nfp_fl_set_ipv6_addr *ip6)
{
- ip6->ipv6[idx % 4].mask = mask;
- ip6->ipv6[idx % 4].exact = exact;
+ ip6->ipv6[word].mask |= mask;
+ ip6->ipv6[word].exact &= ~mask;
+ ip6->ipv6[word].exact |= exact & mask;
ip6->reserved = cpu_to_be16(0);
ip6->head.jump_id = opcode_tag;
struct nfp_fl_set_ipv6_addr *ip_src)
{
__be32 exact, mask;
+ u8 word;
/* We are expecting tcf_pedit to return a big endian value */
mask = (__force __be32)~tcf_pedit_mask(action, idx);
if (exact & ~mask)
return -EOPNOTSUPP;
- if (off < offsetof(struct ipv6hdr, saddr))
+ if (off < offsetof(struct ipv6hdr, saddr)) {
return -EOPNOTSUPP;
- else if (off < offsetof(struct ipv6hdr, daddr))
- nfp_fl_set_ip6_helper(NFP_FL_ACTION_OPCODE_SET_IPV6_SRC, idx,
+ } else if (off < offsetof(struct ipv6hdr, daddr)) {
+ word = (off - offsetof(struct ipv6hdr, saddr)) / sizeof(exact);
+ nfp_fl_set_ip6_helper(NFP_FL_ACTION_OPCODE_SET_IPV6_SRC, word,
exact, mask, ip_src);
- else if (off < offsetof(struct ipv6hdr, daddr) +
- sizeof(struct in6_addr))
- nfp_fl_set_ip6_helper(NFP_FL_ACTION_OPCODE_SET_IPV6_DST, idx,
+ } else if (off < offsetof(struct ipv6hdr, daddr) +
+ sizeof(struct in6_addr)) {
+ word = (off - offsetof(struct ipv6hdr, daddr)) / sizeof(exact);
+ nfp_fl_set_ip6_helper(NFP_FL_ACTION_OPCODE_SET_IPV6_DST, word,
exact, mask, ip_dst);
- else
+ } else {
return -EOPNOTSUPP;
+ }
return 0;
}
struct nfp_fl_set_eth set_eth;
enum pedit_header_type htype;
int idx, nkeys, err;
- size_t act_size;
+ size_t act_size = 0;
u32 offset, cmd;
u8 ip_proto = 0;
act_size = sizeof(set_eth);
memcpy(nfp_action, &set_eth, act_size);
*a_len += act_size;
- } else if (set_ip_addr.head.len_lw) {
+ }
+ if (set_ip_addr.head.len_lw) {
+ nfp_action += act_size;
act_size = sizeof(set_ip_addr);
memcpy(nfp_action, &set_ip_addr, act_size);
*a_len += act_size;
/* Hardware will automatically fix IPv4 and TCP/UDP checksum. */
*csum_updated |= TCA_CSUM_UPDATE_FLAG_IPV4HDR |
nfp_fl_csum_l4_to_flag(ip_proto);
- } else if (set_ip6_dst.head.len_lw && set_ip6_src.head.len_lw) {
+ }
+ if (set_ip6_dst.head.len_lw && set_ip6_src.head.len_lw) {
/* TC compiles set src and dst IPv6 address as a single action,
* the hardware requires this to be 2 separate actions.
*/
+ nfp_action += act_size;
act_size = sizeof(set_ip6_src);
memcpy(nfp_action, &set_ip6_src, act_size);
*a_len += act_size;
/* Hardware will automatically fix TCP/UDP checksum. */
*csum_updated |= nfp_fl_csum_l4_to_flag(ip_proto);
} else if (set_ip6_dst.head.len_lw) {
+ nfp_action += act_size;
act_size = sizeof(set_ip6_dst);
memcpy(nfp_action, &set_ip6_dst, act_size);
*a_len += act_size;
/* Hardware will automatically fix TCP/UDP checksum. */
*csum_updated |= nfp_fl_csum_l4_to_flag(ip_proto);
} else if (set_ip6_src.head.len_lw) {
+ nfp_action += act_size;
act_size = sizeof(set_ip6_src);
memcpy(nfp_action, &set_ip6_src, act_size);
*a_len += act_size;
/* Hardware will automatically fix TCP/UDP checksum. */
*csum_updated |= nfp_fl_csum_l4_to_flag(ip_proto);
- } else if (set_tport.head.len_lw) {
+ }
+ if (set_tport.head.len_lw) {
+ nfp_action += act_size;
act_size = sizeof(set_tport);
memcpy(nfp_action, &set_tport, act_size);
*a_len += act_size;
attn_master_to_str(GET_FIELD(tmp, QED_GRC_ATTENTION_MASTER)),
GET_FIELD(tmp2, QED_GRC_ATTENTION_PF),
(GET_FIELD(tmp2, QED_GRC_ATTENTION_PRIV) ==
- QED_GRC_ATTENTION_PRIV_VF) ? "VF" : "(Ireelevant)",
+ QED_GRC_ATTENTION_PRIV_VF) ? "VF" : "(Irrelevant)",
GET_FIELD(tmp2, QED_GRC_ATTENTION_VF));
out:
qdev->eeprom_cmd_data = AUBURN_EEPROM_CS_1;
ql_write_nvram_reg(qdev, spir, ISP_NVRAM_MASK | qdev->eeprom_cmd_data);
- ql_write_nvram_reg(qdev, spir,
- ((ISP_NVRAM_MASK << 16) | qdev->eeprom_cmd_data));
}
/*
RTL_W32(tp, RxConfig, RX_FIFO_THRESH | RX_DMA_BURST);
break;
case RTL_GIGA_MAC_VER_18 ... RTL_GIGA_MAC_VER_24:
- case RTL_GIGA_MAC_VER_34:
- case RTL_GIGA_MAC_VER_35:
+ case RTL_GIGA_MAC_VER_34 ... RTL_GIGA_MAC_VER_36:
+ case RTL_GIGA_MAC_VER_38:
RTL_W32(tp, RxConfig, RX128_INT_EN | RX_MULTI_EN | RX_DMA_BURST);
break;
case RTL_GIGA_MAC_VER_40 ... RTL_GIGA_MAC_VER_51:
struct rtl8169_private *tp = container_of(napi, struct rtl8169_private, napi);
struct net_device *dev = tp->dev;
u16 enable_mask = RTL_EVENT_NAPI | tp->event_slow;
- int work_done= 0;
+ int work_done;
u16 status;
status = rtl_get_events(tp);
rtl_ack_events(tp, status & ~tp->event_slow);
- if (status & RTL_EVENT_NAPI_RX)
- work_done = rtl_rx(dev, tp, (u32) budget);
+ work_done = rtl_rx(dev, tp, (u32) budget);
- if (status & RTL_EVENT_NAPI_TX)
- rtl_tx(dev, tp);
+ rtl_tx(dev, tp);
if (status & tp->event_slow) {
enable_mask &= ~tp->event_slow;
{
unsigned int flags;
- switch (tp->mac_version) {
- case RTL_GIGA_MAC_VER_01 ... RTL_GIGA_MAC_VER_06:
+ if (tp->mac_version <= RTL_GIGA_MAC_VER_06) {
RTL_W8(tp, Cfg9346, Cfg9346_Unlock);
RTL_W8(tp, Config2, RTL_R8(tp, Config2) & ~MSIEnable);
RTL_W8(tp, Cfg9346, Cfg9346_Lock);
flags = PCI_IRQ_LEGACY;
- break;
- case RTL_GIGA_MAC_VER_39 ... RTL_GIGA_MAC_VER_40:
- /* This version was reported to have issues with resume
- * from suspend when using MSI-X
- */
- flags = PCI_IRQ_LEGACY | PCI_IRQ_MSI;
- break;
- default:
+ } else {
flags = PCI_IRQ_ALL_TYPES;
}
u16 idx = dring->tail;
struct netsec_de *de = dring->vaddr + (DESC_SZ * idx);
- if (de->attr & (1U << NETSEC_RX_PKT_OWN_FIELD))
+ if (de->attr & (1U << NETSEC_RX_PKT_OWN_FIELD)) {
+ /* reading the register clears the irq */
+ netsec_read(priv, NETSEC_REG_NRM_RX_PKTCNT);
break;
+ }
/* This barrier is needed to keep us from reading
* any other fields out of the netsec_de until we have
if (IS_ERR(rt))
return PTR_ERR(rt);
- if (skb_dst(skb)) {
- int mtu = dst_mtu(&rt->dst) - GENEVE_IPV4_HLEN -
- info->options_len;
-
- skb_dst_update_pmtu(skb, mtu);
- }
+ skb_tunnel_check_pmtu(skb, &rt->dst,
+ GENEVE_IPV4_HLEN + info->options_len);
sport = udp_flow_src_port(geneve->net, skb, 1, USHRT_MAX, true);
if (geneve->collect_md) {
if (IS_ERR(dst))
return PTR_ERR(dst);
- if (skb_dst(skb)) {
- int mtu = dst_mtu(dst) - GENEVE_IPV6_HLEN - info->options_len;
-
- skb_dst_update_pmtu(skb, mtu);
- }
+ skb_tunnel_check_pmtu(skb, dst, GENEVE_IPV6_HLEN + info->options_len);
sport = udp_flow_src_port(geneve->net, skb, 1, USHRT_MAX, true);
if (geneve->collect_md) {
struct mdio_mux_gpio_state {
struct gpio_descs *gpios;
void *mux_handle;
- int values[];
};
static int mdio_mux_gpio_switch_fn(int current_child, int desired_child,
void *data)
{
struct mdio_mux_gpio_state *s = data;
- unsigned int n;
+ DECLARE_BITMAP(values, BITS_PER_TYPE(desired_child));
if (current_child == desired_child)
return 0;
- for (n = 0; n < s->gpios->ndescs; n++)
- s->values[n] = (desired_child >> n) & 1;
+ values[0] = desired_child;
gpiod_set_array_value_cansleep(s->gpios->ndescs, s->gpios->desc,
- s->values);
+ s->gpios->info, values);
return 0;
}
if (IS_ERR(gpios))
return PTR_ERR(gpios);
- s = devm_kzalloc(&pdev->dev, struct_size(s, values, gpios->ndescs),
- GFP_KERNEL);
+ s = devm_kzalloc(&pdev->dev, sizeof(*s), GFP_KERNEL);
if (!s) {
gpiod_put_array(gpios);
return -ENOMEM;
/* Give this long for the PHY to reset. */
#define T_PHY_RESET_MS 50
-static DEFINE_MUTEX(sfp_mutex);
-
struct sff_data {
unsigned int gpios;
bool (*module_supported)(const struct sfp_eeprom_id *id);
{QMI_FIXED_INTF(0x0b3c, 0xc00b, 4)}, /* Olivetti Olicard 500 */
{QMI_FIXED_INTF(0x1e2d, 0x0060, 4)}, /* Cinterion PLxx */
{QMI_FIXED_INTF(0x1e2d, 0x0053, 4)}, /* Cinterion PHxx,PXxx */
+ {QMI_FIXED_INTF(0x1e2d, 0x0063, 10)}, /* Cinterion ALASxx (1 RmNet) */
{QMI_FIXED_INTF(0x1e2d, 0x0082, 4)}, /* Cinterion PHxx,PXxx (2 RmNet) */
{QMI_FIXED_INTF(0x1e2d, 0x0082, 5)}, /* Cinterion PHxx,PXxx (2 RmNet) */
{QMI_FIXED_INTF(0x1e2d, 0x0083, 4)}, /* Cinterion PHxx,PXxx (1 RmNet + USB Audio)*/
/* Make sure no work handler is accessing the device */
flush_work(&vi->config_work);
+ netif_tx_lock_bh(vi->dev);
netif_device_detach(vi->dev);
- netif_tx_disable(vi->dev);
+ netif_tx_unlock_bh(vi->dev);
cancel_delayed_work_sync(&vi->refill);
if (netif_running(vi->dev)) {
}
}
+ netif_tx_lock_bh(vi->dev);
netif_device_attach(vi->dev);
+ netif_tx_unlock_bh(vi->dev);
return err;
}
}
ndst = &rt->dst;
- if (skb_dst(skb)) {
- int mtu = dst_mtu(ndst) - VXLAN_HEADROOM;
-
- skb_dst_update_pmtu(skb, mtu);
- }
+ skb_tunnel_check_pmtu(skb, ndst, VXLAN_HEADROOM);
tos = ip_tunnel_ecn_encap(tos, old_iph, skb);
ttl = ttl ? : ip4_dst_hoplimit(&rt->dst);
goto out_unlock;
}
- if (skb_dst(skb)) {
- int mtu = dst_mtu(ndst) - VXLAN6_HEADROOM;
-
- skb_dst_update_pmtu(skb, mtu);
- }
+ skb_tunnel_check_pmtu(skb, ndst, VXLAN6_HEADROOM);
tos = ip_tunnel_ecn_encap(tos, old_iph, skb);
ttl = ttl ? : ip6_dst_hoplimit(ndst);
if (priv->wol_criteria == EHS_REMOVE_WAKEUP) {
dev_info(dev, "Suspend without wake params -- powering down card\n");
if (priv->fw_ready) {
+ ret = lbs_suspend(priv);
+ if (ret)
+ return ret;
+
priv->power_up_on_resume = true;
if_sdio_power_off(card);
}
if (!buf->urb)
return -ENOMEM;
- buf->urb->sg = devm_kzalloc(dev->dev, nsgs * sizeof(*buf->urb->sg),
+ buf->urb->sg = devm_kcalloc(dev->dev, nsgs, sizeof(*buf->urb->sg),
gfp);
if (!buf->urb->sg)
return -ENOMEM;
int i, err, nsgs;
spin_lock_init(&q->lock);
- q->entry = devm_kzalloc(dev->dev,
- MT_NUM_RX_ENTRIES * sizeof(*q->entry),
+ q->entry = devm_kcalloc(dev->dev,
+ MT_NUM_RX_ENTRIES, sizeof(*q->entry),
GFP_KERNEL);
if (!q->entry)
return -ENOMEM;
INIT_LIST_HEAD(&q->swq);
q->hw_idx = q2hwq(i);
- q->entry = devm_kzalloc(dev->dev,
- MT_NUM_TX_ENTRIES * sizeof(*q->entry),
+ q->entry = devm_kcalloc(dev->dev,
+ MT_NUM_TX_ENTRIES, sizeof(*q->entry),
GFP_KERNEL);
if (!q->entry)
return -ENOMEM;
}
set_capacity(disk, available_disk_size >> SECTOR_SHIFT);
- device_add_disk(dev, disk);
+ device_add_disk(dev, disk, NULL);
revalidate_disk(disk);
return 0;
}
}
}
set_capacity(btt->btt_disk, btt->nlba * btt->sector_size >> 9);
- device_add_disk(&btt->nd_btt->dev, btt->btt_disk);
+ device_add_disk(&btt->nd_btt->dev, btt->btt_disk, NULL);
btt->nd_btt->size = btt->nlba * (u64)btt->sector_size;
revalidate_disk(btt->btt_disk);
gendev = disk_to_dev(disk);
gendev->groups = pmem_attribute_groups;
- device_add_disk(dev, disk);
+ device_add_disk(dev, disk, NULL);
if (devm_add_action_or_reset(dev, pmem_release_disk, pmem))
return -ENOMEM;
uuid_copy(&ids->uuid, data + pos + sizeof(*cur));
break;
default:
- /* Skip unnkown types */
+ /* Skip unknown types */
len = cur->nidl;
break;
}
return nvme_submit_user_cmd(ns->queue, &c,
(void __user *)(uintptr_t)io.addr, length,
- metadata, meta_len, io.slba, NULL, 0);
+ metadata, meta_len, lower_32_bits(io.slba), NULL, 0);
}
static u32 nvme_known_admin_effects(u8 opcode)
nqnlen = strnlen(id->subnqn, NVMF_NQN_SIZE);
if (nqnlen > 0 && nqnlen < NVMF_NQN_SIZE) {
- strncpy(subsys->subnqn, id->subnqn, NVMF_NQN_SIZE);
+ strlcpy(subsys->subnqn, id->subnqn, NVMF_NQN_SIZE);
return;
}
return a->mode;
}
-const struct attribute_group nvme_ns_id_attr_group = {
+static const struct attribute_group nvme_ns_id_attr_group = {
.attrs = nvme_ns_id_attrs,
.is_visible = nvme_ns_id_attrs_are_visible,
};
+const struct attribute_group *nvme_ns_id_attr_groups[] = {
+ &nvme_ns_id_attr_group,
+#ifdef CONFIG_NVM
+ &nvme_nvm_attr_group,
+#endif
+ NULL,
+};
+
#define nvme_show_str_function(field) \
static ssize_t field##_show(struct device *dev, \
struct device_attribute *attr, char *buf) \
unsigned nsid, struct nvme_id_ns *id)
{
struct nvme_ns_head *head;
+ size_t size = sizeof(*head);
int ret = -ENOMEM;
- head = kzalloc(sizeof(*head), GFP_KERNEL);
+#ifdef CONFIG_NVME_MULTIPATH
+ size += num_possible_nodes() * sizeof(struct nvme_ns *);
+#endif
+
+ head = kzalloc(size, GFP_KERNEL);
if (!head)
goto out;
ret = ida_simple_get(&ctrl->subsys->ns_ida, 1, 0, GFP_KERNEL);
nvme_get_ctrl(ctrl);
- device_add_disk(ctrl->device, ns->disk);
- if (sysfs_create_group(&disk_to_dev(ns->disk)->kobj,
- &nvme_ns_id_attr_group))
- pr_warn("%s: failed to create sysfs group for identification\n",
- ns->disk->disk_name);
- if (ns->ndev && nvme_nvm_register_sysfs(ns))
- pr_warn("%s: failed to register lightnvm sysfs group for identification\n",
- ns->disk->disk_name);
+ device_add_disk(ctrl->device, ns->disk, nvme_ns_id_attr_groups);
nvme_mpath_add_disk(ns, id);
nvme_fault_inject_init(ns);
nvme_fault_inject_fini(ns);
if (ns->disk && ns->disk->flags & GENHD_FL_UP) {
- sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
- &nvme_ns_id_attr_group);
- if (ns->ndev)
- nvme_nvm_unregister_sysfs(ns);
del_gendisk(ns->disk);
blk_cleanup_queue(ns->queue);
if (blk_get_integrity(ns->disk))
}
mutex_lock(&ns->ctrl->subsys->lock);
- nvme_mpath_clear_current_path(ns);
list_del_rcu(&ns->siblings);
+ nvme_mpath_clear_current_path(ns);
mutex_unlock(&ns->ctrl->subsys->lock);
down_write(&ns->ctrl->namespaces_rwsem);
static void nvme_handle_aen_notice(struct nvme_ctrl *ctrl, u32 result)
{
- switch ((result & 0xff00) >> 8) {
+ u32 aer_notice_type = (result & 0xff00) >> 8;
+
+ switch (aer_notice_type) {
case NVME_AER_NOTICE_NS_CHANGED:
+ trace_nvme_async_event(ctrl, aer_notice_type);
set_bit(NVME_AER_NOTICE_NS_CHANGED, &ctrl->events);
nvme_queue_scan(ctrl);
break;
case NVME_AER_NOTICE_FW_ACT_STARTING:
+ trace_nvme_async_event(ctrl, aer_notice_type);
queue_work(nvme_wq, &ctrl->fw_act_work);
break;
#ifdef CONFIG_NVME_MULTIPATH
case NVME_AER_NOTICE_ANA:
+ trace_nvme_async_event(ctrl, aer_notice_type);
if (!ctrl->ana_log_buf)
break;
queue_work(nvme_wq, &ctrl->ana_work);
volatile union nvme_result *res)
{
u32 result = le32_to_cpu(res->u32);
+ u32 aer_type = result & 0x07;
if (le16_to_cpu(status) >> 1 != NVME_SC_SUCCESS)
return;
- switch (result & 0x7) {
+ switch (aer_type) {
case NVME_AER_NOTICE:
nvme_handle_aen_notice(ctrl, result);
break;
case NVME_AER_SMART:
case NVME_AER_CSS:
case NVME_AER_VS:
+ trace_nvme_async_event(ctrl, aer_type);
ctrl->aen_result = result;
break;
default:
ctrl->state != NVME_CTRL_DEAD &&
!blk_noretry_request(rq) && !(rq->cmd_flags & REQ_NVME_MPATH))
return BLK_STS_RESOURCE;
- nvme_req(rq)->status = NVME_SC_ABORT_REQ;
- return BLK_STS_IOERR;
+
+ nvme_req(rq)->status = NVME_SC_HOST_PATH_ERROR;
+ blk_mq_start_request(rq);
+ nvme_complete_rq(rq);
+ return BLK_STS_OK;
}
EXPORT_SYMBOL_GPL(nvmf_fail_nonready_command);
return 0;
}
+bool nvmf_ip_options_match(struct nvme_ctrl *ctrl,
+ struct nvmf_ctrl_options *opts)
+{
+ if (!nvmf_ctlr_matches_baseopts(ctrl, opts) ||
+ strcmp(opts->traddr, ctrl->opts->traddr) ||
+ strcmp(opts->trsvcid, ctrl->opts->trsvcid))
+ return false;
+
+ /*
+ * Checking the local address is rough. In most cases, none is specified
+ * and the host port is selected by the stack.
+ *
+ * Assume no match if:
+ * - local address is specified and address is not the same
+ * - local address is not specified but remote is, or vice versa
+ * (admin using specific host_traddr when it matters).
+ */
+ if ((opts->mask & NVMF_OPT_HOST_TRADDR) &&
+ (ctrl->opts->mask & NVMF_OPT_HOST_TRADDR)) {
+ if (strcmp(opts->host_traddr, ctrl->opts->host_traddr))
+ return false;
+ } else if ((opts->mask & NVMF_OPT_HOST_TRADDR) ||
+ (ctrl->opts->mask & NVMF_OPT_HOST_TRADDR)) {
+ return false;
+ }
+
+ return true;
+}
+EXPORT_SYMBOL_GPL(nvmf_ip_options_match);
+
static int nvmf_check_allowed_opts(struct nvmf_ctrl_options *opts,
unsigned int allowed_opts)
{
struct request *rq);
bool __nvmf_check_ready(struct nvme_ctrl *ctrl, struct request *rq,
bool queue_live);
+bool nvmf_ip_options_match(struct nvme_ctrl *ctrl,
+ struct nvmf_ctrl_options *opts);
static inline bool nvmf_check_ready(struct nvme_ctrl *ctrl, struct request *rq,
bool queue_live)
#include <uapi/scsi/fc/fc_fs.h>
#include <uapi/scsi/fc/fc_els.h>
#include <linux/delay.h>
+#include <linux/overflow.h>
#include "nvme.h"
#include "fabrics.h"
struct nvme_fc_ersp_iu rsp_iu;
};
+struct nvme_fcp_op_w_sgl {
+ struct nvme_fc_fcp_op op;
+ struct scatterlist sgl[SG_CHUNK_SIZE];
+ uint8_t priv[0];
+};
+
struct nvme_fc_lport {
struct nvme_fc_local_port localport;
struct list_head endp_list; /* for lport->endp_list */
struct list_head ctrl_list;
struct list_head ls_req_list;
+ struct list_head disc_list;
struct device *dev; /* physical device for dma */
struct nvme_fc_lport *lport;
spinlock_t lock;
* These items are short-term. They will eventually be moved into
* a generic FC class. See comments in module init.
*/
-static struct class *fc_class;
static struct device *fc_udev_device;
* @template: LLDD entrypoints and operational parameters for the port
* @dev: physical hardware device node port corresponds to. Will be
* used for DMA mappings
- * @lport_p: pointer to a local port pointer. Upon success, the routine
+ * @portptr: pointer to a local port pointer. Upon success, the routine
* will allocate a nvme_fc_local_port structure and place its
* address in the local port pointer. Upon failure, local port
* pointer will be set to 0.
* nvme_fc_unregister_localport - transport entry point called by an
* LLDD to deregister/remove a previously
* registered a NVME host FC port.
- * @localport: pointer to the (registered) local port that is to be
- * deregistered.
+ * @portptr: pointer to the (registered) local port that is to be deregistered.
*
* Returns:
* a completion status. Must be 0 upon success; a negative errno
list_del(&rport->endp_list);
spin_unlock_irqrestore(&nvme_fc_lock, flags);
+ WARN_ON(!list_empty(&rport->disc_list));
ida_simple_remove(&lport->endp_cnt, rport->remoteport.port_num);
kfree(rport);
* @localport: pointer to the (registered) local port that the remote
* subsystem port is connected to.
* @pinfo: pointer to information about the port to be registered
- * @rport_p: pointer to a remote port pointer. Upon success, the routine
+ * @portptr: pointer to a remote port pointer. Upon success, the routine
* will allocate a nvme_fc_remote_port structure and place its
* address in the remote port pointer. Upon failure, remote port
* pointer will be set to 0.
INIT_LIST_HEAD(&newrec->endp_list);
INIT_LIST_HEAD(&newrec->ctrl_list);
INIT_LIST_HEAD(&newrec->ls_req_list);
+ INIT_LIST_HEAD(&newrec->disc_list);
kref_init(&newrec->ref);
atomic_set(&newrec->act_ctrl_cnt, 0);
spin_lock_init(&newrec->lock);
* nvme_fc_unregister_remoteport - transport entry point called by an
* LLDD to deregister/remove a previously
* registered a NVME subsystem FC port.
- * @remoteport: pointer to the (registered) remote port that is to be
- * deregistered.
+ * @portptr: pointer to the (registered) remote port that is to be
+ * deregistered.
*
* Returns:
* a completion status. Must be 0 upon success; a negative errno
__nvme_fc_finish_ls_req(lsop);
- /* fc-nvme iniator doesn't care about success or failure of cmd */
+ /* fc-nvme initiator doesn't care about success or failure of cmd */
kfree(lsop);
}
struct nvme_fc_queue *queue, struct nvme_fc_fcp_op *op,
struct request *rq, u32 rqno)
{
+ struct nvme_fcp_op_w_sgl *op_w_sgl =
+ container_of(op, typeof(*op_w_sgl), op);
struct nvme_fc_cmd_iu *cmdiu = &op->cmd_iu;
int ret = 0;
op->fcp_req.rspaddr = &op->rsp_iu;
op->fcp_req.rsplen = sizeof(op->rsp_iu);
op->fcp_req.done = nvme_fc_fcpio_done;
- op->fcp_req.first_sgl = (struct scatterlist *)&op[1];
op->fcp_req.private = &op->fcp_req.first_sgl[SG_CHUNK_SIZE];
op->ctrl = ctrl;
op->queue = queue;
unsigned int hctx_idx, unsigned int numa_node)
{
struct nvme_fc_ctrl *ctrl = set->driver_data;
- struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(rq);
+ struct nvme_fcp_op_w_sgl *op = blk_mq_rq_to_pdu(rq);
int queue_idx = (set == &ctrl->tag_set) ? hctx_idx + 1 : 0;
struct nvme_fc_queue *queue = &ctrl->queues[queue_idx];
+ int res;
nvme_req(rq)->ctrl = &ctrl->ctrl;
- return __nvme_fc_init_request(ctrl, queue, op, rq, queue->rqcnt++);
+ res = __nvme_fc_init_request(ctrl, queue, &op->op, rq, queue->rqcnt++);
+ if (res)
+ return res;
+ op->op.fcp_req.first_sgl = &op->sgl[0];
+ return res;
}
static int
}
aen_op->flags = FCOP_FLAGS_AEN;
- aen_op->fcp_req.first_sgl = NULL; /* no sg list */
aen_op->fcp_req.private = private;
memset(sqe, 0, sizeof(*sqe));
ctrl->tag_set.reserved_tags = 1; /* fabric connect */
ctrl->tag_set.numa_node = NUMA_NO_NODE;
ctrl->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
- ctrl->tag_set.cmd_size = sizeof(struct nvme_fc_fcp_op) +
- (SG_CHUNK_SIZE *
- sizeof(struct scatterlist)) +
- ctrl->lport->ops->fcprqst_priv_sz;
+ ctrl->tag_set.cmd_size =
+ struct_size((struct nvme_fcp_op_w_sgl *)NULL, priv,
+ ctrl->lport->ops->fcprqst_priv_sz);
ctrl->tag_set.driver_data = ctrl;
ctrl->tag_set.nr_hw_queues = ctrl->ctrl.queue_count - 1;
ctrl->tag_set.timeout = NVME_IO_TIMEOUT;
ctrl->admin_tag_set.queue_depth = NVME_AQ_MQ_TAG_DEPTH;
ctrl->admin_tag_set.reserved_tags = 2; /* fabric connect + Keep-Alive */
ctrl->admin_tag_set.numa_node = NUMA_NO_NODE;
- ctrl->admin_tag_set.cmd_size = sizeof(struct nvme_fc_fcp_op) +
- (SG_CHUNK_SIZE *
- sizeof(struct scatterlist)) +
- ctrl->lport->ops->fcprqst_priv_sz;
+ ctrl->admin_tag_set.cmd_size =
+ struct_size((struct nvme_fcp_op_w_sgl *)NULL, priv,
+ ctrl->lport->ops->fcprqst_priv_sz);
ctrl->admin_tag_set.driver_data = ctrl;
ctrl->admin_tag_set.nr_hw_queues = 1;
ctrl->admin_tag_set.timeout = ADMIN_TIMEOUT;
substring_t wwn = { name, &name[sizeof(name)-1] };
int nnoffset, pnoffset;
- /* validate it string one of the 2 allowed formats */
+ /* validate if string is one of the 2 allowed formats */
if (strnlen(buf, blen) == NVME_FC_TRADDR_MAXLENGTH &&
!strncmp(buf, "nn-0x", NVME_FC_TRADDR_OXNNLEN) &&
!strncmp(&buf[NVME_FC_TRADDR_MAX_PN_OFFSET],
.create_ctrl = nvme_fc_create_ctrl,
};
+/* Arbitrary successive failures max. With lots of subsystems could be high */
+#define DISCOVERY_MAX_FAIL 20
+
+static ssize_t nvme_fc_nvme_discovery_store(struct device *dev,
+ struct device_attribute *attr, const char *buf, size_t count)
+{
+ unsigned long flags;
+ LIST_HEAD(local_disc_list);
+ struct nvme_fc_lport *lport;
+ struct nvme_fc_rport *rport;
+ int failcnt = 0;
+
+ spin_lock_irqsave(&nvme_fc_lock, flags);
+restart:
+ list_for_each_entry(lport, &nvme_fc_lport_list, port_list) {
+ list_for_each_entry(rport, &lport->endp_list, endp_list) {
+ if (!nvme_fc_lport_get(lport))
+ continue;
+ if (!nvme_fc_rport_get(rport)) {
+ /*
+ * This is a temporary condition. Upon restart
+ * this rport will be gone from the list.
+ *
+ * Revert the lport put and retry. Anything
+ * added to the list already will be skipped (as
+ * they are no longer list_empty). Loops should
+ * resume at rports that were not yet seen.
+ */
+ nvme_fc_lport_put(lport);
+
+ if (failcnt++ < DISCOVERY_MAX_FAIL)
+ goto restart;
+
+ pr_err("nvme_discovery: too many reference "
+ "failures\n");
+ goto process_local_list;
+ }
+ if (list_empty(&rport->disc_list))
+ list_add_tail(&rport->disc_list,
+ &local_disc_list);
+ }
+ }
+
+process_local_list:
+ while (!list_empty(&local_disc_list)) {
+ rport = list_first_entry(&local_disc_list,
+ struct nvme_fc_rport, disc_list);
+ list_del_init(&rport->disc_list);
+ spin_unlock_irqrestore(&nvme_fc_lock, flags);
+
+ lport = rport->lport;
+ /* signal discovery. Won't hurt if it repeats */
+ nvme_fc_signal_discovery_scan(lport, rport);
+ nvme_fc_rport_put(rport);
+ nvme_fc_lport_put(lport);
+
+ spin_lock_irqsave(&nvme_fc_lock, flags);
+ }
+ spin_unlock_irqrestore(&nvme_fc_lock, flags);
+
+ return count;
+}
+static DEVICE_ATTR(nvme_discovery, 0200, NULL, nvme_fc_nvme_discovery_store);
+
+static struct attribute *nvme_fc_attrs[] = {
+ &dev_attr_nvme_discovery.attr,
+ NULL
+};
+
+static struct attribute_group nvme_fc_attr_group = {
+ .attrs = nvme_fc_attrs,
+};
+
+static const struct attribute_group *nvme_fc_attr_groups[] = {
+ &nvme_fc_attr_group,
+ NULL
+};
+
+static struct class fc_class = {
+ .name = "fc",
+ .dev_groups = nvme_fc_attr_groups,
+ .owner = THIS_MODULE,
+};
+
static int __init nvme_fc_init_module(void)
{
int ret;
* put in place, this code will move to a more generic
* location for the class.
*/
- fc_class = class_create(THIS_MODULE, "fc");
- if (IS_ERR(fc_class)) {
+ ret = class_register(&fc_class);
+ if (ret) {
pr_err("couldn't register class fc\n");
- return PTR_ERR(fc_class);
+ return ret;
}
/*
* Create a device for the FC-centric udev events
*/
- fc_udev_device = device_create(fc_class, NULL, MKDEV(0, 0), NULL,
+ fc_udev_device = device_create(&fc_class, NULL, MKDEV(0, 0), NULL,
"fc_udev_device");
if (IS_ERR(fc_udev_device)) {
pr_err("couldn't create fc_udev device!\n");
return 0;
out_destroy_device:
- device_destroy(fc_class, MKDEV(0, 0));
+ device_destroy(&fc_class, MKDEV(0, 0));
out_destroy_class:
- class_destroy(fc_class);
+ class_unregister(&fc_class);
return ret;
}
ida_destroy(&nvme_fc_local_port_cnt);
ida_destroy(&nvme_fc_ctrl_cnt);
- device_destroy(fc_class, MKDEV(0, 0));
- class_destroy(fc_class);
+ device_destroy(&fc_class, MKDEV(0, 0));
+ class_unregister(&fc_class);
}
module_init(nvme_fc_init_module);
* Expect the lba in device format
*/
static int nvme_nvm_get_chk_meta(struct nvm_dev *ndev,
- struct nvm_chk_meta *meta,
- sector_t slba, int nchks)
+ sector_t slba, int nchks,
+ struct nvm_chk_meta *meta)
{
struct nvm_geo *geo = &ndev->geo;
struct nvme_ns *ns = ndev->q->queuedata;
struct nvme_ctrl *ctrl = ns->ctrl;
- struct nvme_nvm_chk_meta *dev_meta = (struct nvme_nvm_chk_meta *)meta;
+ struct nvme_nvm_chk_meta *dev_meta, *dev_meta_off;
struct ppa_addr ppa;
size_t left = nchks * sizeof(struct nvme_nvm_chk_meta);
size_t log_pos, offset, len;
*/
max_len = min_t(unsigned int, ctrl->max_hw_sectors << 9, 256 * 1024);
+ dev_meta = kmalloc(max_len, GFP_KERNEL);
+ if (!dev_meta)
+ return -ENOMEM;
+
/* Normalize lba address space to obtain log offset */
ppa.ppa = slba;
ppa = dev_to_generic_addr(ndev, ppa);
while (left) {
len = min_t(unsigned int, left, max_len);
+ memset(dev_meta, 0, max_len);
+ dev_meta_off = dev_meta;
+
ret = nvme_get_log(ctrl, ns->head->ns_id,
NVME_NVM_LOG_REPORT_CHUNK, 0, dev_meta, len,
offset);
}
for (i = 0; i < len; i += sizeof(struct nvme_nvm_chk_meta)) {
- meta->state = dev_meta->state;
- meta->type = dev_meta->type;
- meta->wi = dev_meta->wi;
- meta->slba = le64_to_cpu(dev_meta->slba);
- meta->cnlb = le64_to_cpu(dev_meta->cnlb);
- meta->wp = le64_to_cpu(dev_meta->wp);
+ meta->state = dev_meta_off->state;
+ meta->type = dev_meta_off->type;
+ meta->wi = dev_meta_off->wi;
+ meta->slba = le64_to_cpu(dev_meta_off->slba);
+ meta->cnlb = le64_to_cpu(dev_meta_off->cnlb);
+ meta->wp = le64_to_cpu(dev_meta_off->wp);
meta++;
- dev_meta++;
+ dev_meta_off++;
}
offset += len;
left -= len;
}
+ kfree(dev_meta);
+
return ret;
}
struct nvm_dev *ndev = ns->ndev;
struct nvm_geo *geo = &ndev->geo;
+ if (geo->version == NVM_OCSSD_SPEC_12)
+ return;
+
geo->csecs = 1 << ns->lba_shift;
geo->sos = ns->ms;
}
static NVM_DEV_ATTR_12_RO(media_capabilities);
static NVM_DEV_ATTR_12_RO(max_phys_secs);
-static struct attribute *nvm_dev_attrs_12[] = {
+/* 2.0 values */
+static NVM_DEV_ATTR_20_RO(groups);
+static NVM_DEV_ATTR_20_RO(punits);
+static NVM_DEV_ATTR_20_RO(chunks);
+static NVM_DEV_ATTR_20_RO(clba);
+static NVM_DEV_ATTR_20_RO(ws_min);
+static NVM_DEV_ATTR_20_RO(ws_opt);
+static NVM_DEV_ATTR_20_RO(maxoc);
+static NVM_DEV_ATTR_20_RO(maxocpu);
+static NVM_DEV_ATTR_20_RO(mw_cunits);
+static NVM_DEV_ATTR_20_RO(write_typ);
+static NVM_DEV_ATTR_20_RO(write_max);
+static NVM_DEV_ATTR_20_RO(reset_typ);
+static NVM_DEV_ATTR_20_RO(reset_max);
+
+static struct attribute *nvm_dev_attrs[] = {
+ /* version agnostic attrs */
&dev_attr_version.attr,
&dev_attr_capabilities.attr,
+ &dev_attr_read_typ.attr,
+ &dev_attr_read_max.attr,
+ /* 1.2 attrs */
&dev_attr_vendor_opcode.attr,
&dev_attr_device_mode.attr,
&dev_attr_media_manager.attr,
&dev_attr_page_size.attr,
&dev_attr_hw_sector_size.attr,
&dev_attr_oob_sector_size.attr,
- &dev_attr_read_typ.attr,
- &dev_attr_read_max.attr,
&dev_attr_prog_typ.attr,
&dev_attr_prog_max.attr,
&dev_attr_erase_typ.attr,
&dev_attr_media_capabilities.attr,
&dev_attr_max_phys_secs.attr,
- NULL,
-};
-
-static const struct attribute_group nvm_dev_attr_group_12 = {
- .name = "lightnvm",
- .attrs = nvm_dev_attrs_12,
-};
-
-/* 2.0 values */
-static NVM_DEV_ATTR_20_RO(groups);
-static NVM_DEV_ATTR_20_RO(punits);
-static NVM_DEV_ATTR_20_RO(chunks);
-static NVM_DEV_ATTR_20_RO(clba);
-static NVM_DEV_ATTR_20_RO(ws_min);
-static NVM_DEV_ATTR_20_RO(ws_opt);
-static NVM_DEV_ATTR_20_RO(maxoc);
-static NVM_DEV_ATTR_20_RO(maxocpu);
-static NVM_DEV_ATTR_20_RO(mw_cunits);
-static NVM_DEV_ATTR_20_RO(write_typ);
-static NVM_DEV_ATTR_20_RO(write_max);
-static NVM_DEV_ATTR_20_RO(reset_typ);
-static NVM_DEV_ATTR_20_RO(reset_max);
-
-static struct attribute *nvm_dev_attrs_20[] = {
- &dev_attr_version.attr,
- &dev_attr_capabilities.attr,
-
+ /* 2.0 attrs */
&dev_attr_groups.attr,
&dev_attr_punits.attr,
&dev_attr_chunks.attr,
&dev_attr_maxocpu.attr,
&dev_attr_mw_cunits.attr,
- &dev_attr_read_typ.attr,
- &dev_attr_read_max.attr,
&dev_attr_write_typ.attr,
&dev_attr_write_max.attr,
&dev_attr_reset_typ.attr,
NULL,
};
-static const struct attribute_group nvm_dev_attr_group_20 = {
- .name = "lightnvm",
- .attrs = nvm_dev_attrs_20,
-};
-
-int nvme_nvm_register_sysfs(struct nvme_ns *ns)
+static umode_t nvm_dev_attrs_visible(struct kobject *kobj,
+ struct attribute *attr, int index)
{
+ struct device *dev = container_of(kobj, struct device, kobj);
+ struct gendisk *disk = dev_to_disk(dev);
+ struct nvme_ns *ns = disk->private_data;
struct nvm_dev *ndev = ns->ndev;
- struct nvm_geo *geo = &ndev->geo;
+ struct device_attribute *dev_attr =
+ container_of(attr, typeof(*dev_attr), attr);
if (!ndev)
- return -EINVAL;
-
- switch (geo->major_ver_id) {
- case 1:
- return sysfs_create_group(&disk_to_dev(ns->disk)->kobj,
- &nvm_dev_attr_group_12);
- case 2:
- return sysfs_create_group(&disk_to_dev(ns->disk)->kobj,
- &nvm_dev_attr_group_20);
- }
+ return 0;
- return -EINVAL;
-}
+ if (dev_attr->show == nvm_dev_attr_show)
+ return attr->mode;
-void nvme_nvm_unregister_sysfs(struct nvme_ns *ns)
-{
- struct nvm_dev *ndev = ns->ndev;
- struct nvm_geo *geo = &ndev->geo;
-
- switch (geo->major_ver_id) {
+ switch (ndev->geo.major_ver_id) {
case 1:
- sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
- &nvm_dev_attr_group_12);
+ if (dev_attr->show == nvm_dev_attr_show_12)
+ return attr->mode;
break;
case 2:
- sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
- &nvm_dev_attr_group_20);
+ if (dev_attr->show == nvm_dev_attr_show_20)
+ return attr->mode;
break;
}
+
+ return 0;
}
+
+const struct attribute_group nvme_nvm_attr_group = {
+ .name = "lightnvm",
+ .attrs = nvm_dev_attrs,
+ .is_visible = nvm_dev_attrs_visible,
+};
queue_work(nvme_wq, &ns->ctrl->ana_work);
}
break;
+ case NVME_SC_HOST_PATH_ERROR:
+ /*
+ * Temporary transport disruption in talking to the controller.
+ * Try to send on a new path.
+ */
+ nvme_mpath_clear_current_path(ns);
+ break;
default:
/*
* Reset the controller for any non-ANA error as we don't know
[NVME_ANA_CHANGE] = "change",
};
-static struct nvme_ns *__nvme_find_path(struct nvme_ns_head *head)
+void nvme_mpath_clear_current_path(struct nvme_ns *ns)
+{
+ struct nvme_ns_head *head = ns->head;
+ int node;
+
+ if (!head)
+ return;
+
+ for_each_node(node) {
+ if (ns == rcu_access_pointer(head->current_path[node]))
+ rcu_assign_pointer(head->current_path[node], NULL);
+ }
+}
+
+static struct nvme_ns *__nvme_find_path(struct nvme_ns_head *head, int node)
{
- struct nvme_ns *ns, *fallback = NULL;
+ int found_distance = INT_MAX, fallback_distance = INT_MAX, distance;
+ struct nvme_ns *found = NULL, *fallback = NULL, *ns;
list_for_each_entry_rcu(ns, &head->list, siblings) {
if (ns->ctrl->state != NVME_CTRL_LIVE ||
test_bit(NVME_NS_ANA_PENDING, &ns->flags))
continue;
+
+ distance = node_distance(node, dev_to_node(ns->ctrl->dev));
+
switch (ns->ana_state) {
case NVME_ANA_OPTIMIZED:
- rcu_assign_pointer(head->current_path, ns);
- return ns;
+ if (distance < found_distance) {
+ found_distance = distance;
+ found = ns;
+ }
+ break;
case NVME_ANA_NONOPTIMIZED:
- fallback = ns;
+ if (distance < fallback_distance) {
+ fallback_distance = distance;
+ fallback = ns;
+ }
break;
default:
break;
}
}
- if (fallback)
- rcu_assign_pointer(head->current_path, fallback);
- return fallback;
+ if (!found)
+ found = fallback;
+ if (found)
+ rcu_assign_pointer(head->current_path[node], found);
+ return found;
}
static inline bool nvme_path_is_optimized(struct nvme_ns *ns)
inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head)
{
- struct nvme_ns *ns = srcu_dereference(head->current_path, &head->srcu);
+ int node = numa_node_id();
+ struct nvme_ns *ns;
+ ns = srcu_dereference(head->current_path[node], &head->srcu);
if (unlikely(!ns || !nvme_path_is_optimized(ns)))
- ns = __nvme_find_path(head);
+ ns = __nvme_find_path(head, node);
return ns;
}
int srcu_idx;
srcu_idx = srcu_read_lock(&head->srcu);
- ns = srcu_dereference(head->current_path, &head->srcu);
+ ns = srcu_dereference(head->current_path[numa_node_id()], &head->srcu);
if (likely(ns && nvme_path_is_optimized(ns)))
found = ns->queue->poll_fn(q, qc);
srcu_read_unlock(&head->srcu, srcu_idx);
if (!head->disk)
return;
- if (!(head->disk->flags & GENHD_FL_UP)) {
- device_add_disk(&head->subsys->dev, head->disk);
- if (sysfs_create_group(&disk_to_dev(head->disk)->kobj,
- &nvme_ns_id_attr_group))
- dev_warn(&head->subsys->dev,
- "failed to create id group.\n");
+ if (!(head->disk->flags & GENHD_FL_UP))
+ device_add_disk(&head->subsys->dev, head->disk,
+ nvme_ns_id_attr_groups);
+
+ if (nvme_path_is_optimized(ns)) {
+ int node, srcu_idx;
+
+ srcu_idx = srcu_read_lock(&head->srcu);
+ for_each_node(node)
+ __nvme_find_path(head, node);
+ srcu_read_unlock(&head->srcu, srcu_idx);
}
kblockd_schedule_work(&ns->head->requeue_work);
{
if (!head->disk)
return;
- if (head->disk->flags & GENHD_FL_UP) {
- sysfs_remove_group(&disk_to_dev(head->disk)->kobj,
- &nvme_ns_id_attr_group);
+ if (head->disk->flags & GENHD_FL_UP)
del_gendisk(head->disk);
- }
blk_set_queue_dying(head->disk->queue);
/* make sure all pending bios are cleaned up */
kblockd_schedule_work(&head->requeue_work);
* only ever has a single entry for private namespaces.
*/
struct nvme_ns_head {
-#ifdef CONFIG_NVME_MULTIPATH
- struct gendisk *disk;
- struct nvme_ns __rcu *current_path;
- struct bio_list requeue_list;
- spinlock_t requeue_lock;
- struct work_struct requeue_work;
- struct mutex lock;
-#endif
struct list_head list;
struct srcu_struct srcu;
struct nvme_subsystem *subsys;
struct list_head entry;
struct kref ref;
int instance;
+#ifdef CONFIG_NVME_MULTIPATH
+ struct gendisk *disk;
+ struct bio_list requeue_list;
+ spinlock_t requeue_lock;
+ struct work_struct requeue_work;
+ struct mutex lock;
+ struct nvme_ns __rcu *current_path[];
+#endif
};
#ifdef CONFIG_FAULT_INJECTION_DEBUG_FS
int nvme_get_log(struct nvme_ctrl *ctrl, u32 nsid, u8 log_page, u8 lsp,
void *log, size_t size, u64 offset);
-extern const struct attribute_group nvme_ns_id_attr_group;
+extern const struct attribute_group *nvme_ns_id_attr_groups[];
extern const struct block_device_operations nvme_ns_head_ops;
#ifdef CONFIG_NVME_MULTIPATH
int nvme_mpath_init(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id);
void nvme_mpath_uninit(struct nvme_ctrl *ctrl);
void nvme_mpath_stop(struct nvme_ctrl *ctrl);
-
-static inline void nvme_mpath_clear_current_path(struct nvme_ns *ns)
-{
- struct nvme_ns_head *head = ns->head;
-
- if (head && ns == rcu_access_pointer(head->current_path))
- rcu_assign_pointer(head->current_path, NULL);
-}
+void nvme_mpath_clear_current_path(struct nvme_ns *ns);
struct nvme_ns *nvme_find_path(struct nvme_ns_head *head);
static inline void nvme_mpath_check_last_path(struct nvme_ns *ns)
void nvme_nvm_update_nvm_info(struct nvme_ns *ns);
int nvme_nvm_register(struct nvme_ns *ns, char *disk_name, int node);
void nvme_nvm_unregister(struct nvme_ns *ns);
-int nvme_nvm_register_sysfs(struct nvme_ns *ns);
-void nvme_nvm_unregister_sysfs(struct nvme_ns *ns);
+extern const struct attribute_group nvme_nvm_attr_group;
int nvme_nvm_ioctl(struct nvme_ns *ns, unsigned int cmd, unsigned long arg);
#else
static inline void nvme_nvm_update_nvm_info(struct nvme_ns *ns) {};
}
static inline void nvme_nvm_unregister(struct nvme_ns *ns) {};
-static inline int nvme_nvm_register_sysfs(struct nvme_ns *ns)
-{
- return 0;
-}
-static inline void nvme_nvm_unregister_sysfs(struct nvme_ns *ns) {};
static inline int nvme_nvm_ioctl(struct nvme_ns *ns, unsigned int cmd,
unsigned long arg)
{
if (!dma_map_sg(dev->dev, &iod->meta_sg, 1, dma_dir))
goto out_unmap;
- }
- if (blk_integrity_rq(req))
cmnd->rw.metadata = cpu_to_le64(sg_dma_address(&iod->meta_sg));
+ }
+
return BLK_STS_OK;
out_unmap:
/**
* nvme_suspend_queue - put queue into suspended state
- * @nvmeq - queue to suspend
+ * @nvmeq: queue to suspend
*/
static int nvme_suspend_queue(struct nvme_queue *nvmeq)
{
struct nvme_dev *dev = pci_get_drvdata(pdev);
nvme_change_ctrl_state(&dev->ctrl, NVME_CTRL_DELETING);
-
- cancel_work_sync(&dev->ctrl.reset_work);
pci_set_drvdata(pdev, NULL);
if (!pci_device_is_present(pdev)) {
nvme_change_ctrl_state(&dev->ctrl, NVME_CTRL_DEAD);
nvme_dev_disable(dev, true);
+ nvme_dev_remove_admin(dev);
}
flush_work(&dev->ctrl.reset_work);
static int nvme_rdma_wait_for_cm(struct nvme_rdma_queue *queue)
{
- wait_for_completion_interruptible_timeout(&queue->cm_done,
+ int ret;
+
+ ret = wait_for_completion_interruptible_timeout(&queue->cm_done,
msecs_to_jiffies(NVME_RDMA_CONNECT_TIMEOUT_MS) + 1);
+ if (ret < 0)
+ return ret;
+ if (ret == 0)
+ return -ETIMEDOUT;
+ WARN_ON_ONCE(queue->cm_error > 0);
return queue->cm_error;
}
.stop_ctrl = nvme_rdma_stop_ctrl,
};
-static inline bool
-__nvme_rdma_options_match(struct nvme_rdma_ctrl *ctrl,
- struct nvmf_ctrl_options *opts)
-{
- char *stdport = __stringify(NVME_RDMA_IP_PORT);
-
-
- if (!nvmf_ctlr_matches_baseopts(&ctrl->ctrl, opts) ||
- strcmp(opts->traddr, ctrl->ctrl.opts->traddr))
- return false;
-
- if (opts->mask & NVMF_OPT_TRSVCID &&
- ctrl->ctrl.opts->mask & NVMF_OPT_TRSVCID) {
- if (strcmp(opts->trsvcid, ctrl->ctrl.opts->trsvcid))
- return false;
- } else if (opts->mask & NVMF_OPT_TRSVCID) {
- if (strcmp(opts->trsvcid, stdport))
- return false;
- } else if (ctrl->ctrl.opts->mask & NVMF_OPT_TRSVCID) {
- if (strcmp(stdport, ctrl->ctrl.opts->trsvcid))
- return false;
- }
- /* else, it's a match as both have stdport. Fall to next checks */
-
- /*
- * checking the local address is rough. In most cases, one
- * is not specified and the host port is selected by the stack.
- *
- * Assume no match if:
- * local address is specified and address is not the same
- * local address is not specified but remote is, or vice versa
- * (admin using specific host_traddr when it matters).
- */
- if (opts->mask & NVMF_OPT_HOST_TRADDR &&
- ctrl->ctrl.opts->mask & NVMF_OPT_HOST_TRADDR) {
- if (strcmp(opts->host_traddr, ctrl->ctrl.opts->host_traddr))
- return false;
- } else if (opts->mask & NVMF_OPT_HOST_TRADDR ||
- ctrl->ctrl.opts->mask & NVMF_OPT_HOST_TRADDR)
- return false;
- /*
- * if neither controller had an host port specified, assume it's
- * a match as everything else matched.
- */
-
- return true;
-}
-
/*
* Fails a connection request if it matches an existing controller
* (association) with the same tuple:
mutex_lock(&nvme_rdma_ctrl_mutex);
list_for_each_entry(ctrl, &nvme_rdma_ctrl_list, list) {
- found = __nvme_rdma_options_match(ctrl, opts);
+ found = nvmf_ip_options_match(&ctrl->ctrl, opts);
if (found)
break;
}
struct nvme_rdma_ctrl *ctrl;
int ret;
bool changed;
- char *port;
ctrl = kzalloc(sizeof(*ctrl), GFP_KERNEL);
if (!ctrl)
ctrl->ctrl.opts = opts;
INIT_LIST_HEAD(&ctrl->list);
- if (opts->mask & NVMF_OPT_TRSVCID)
- port = opts->trsvcid;
- else
- port = __stringify(NVME_RDMA_IP_PORT);
+ if (!(opts->mask & NVMF_OPT_TRSVCID)) {
+ opts->trsvcid =
+ kstrdup(__stringify(NVME_RDMA_IP_PORT), GFP_KERNEL);
+ if (!opts->trsvcid) {
+ ret = -ENOMEM;
+ goto out_free_ctrl;
+ }
+ opts->mask |= NVMF_OPT_TRSVCID;
+ }
ret = inet_pton_with_scope(&init_net, AF_UNSPEC,
- opts->traddr, port, &ctrl->addr);
+ opts->traddr, opts->trsvcid, &ctrl->addr);
if (ret) {
- pr_err("malformed address passed: %s:%s\n", opts->traddr, port);
+ pr_err("malformed address passed: %s:%s\n",
+ opts->traddr, opts->trsvcid);
goto out_free_ctrl;
}
);
+#define aer_name(aer) { aer, #aer }
+
+TRACE_EVENT(nvme_async_event,
+ TP_PROTO(struct nvme_ctrl *ctrl, u32 result),
+ TP_ARGS(ctrl, result),
+ TP_STRUCT__entry(
+ __field(int, ctrl_id)
+ __field(u32, result)
+ ),
+ TP_fast_assign(
+ __entry->ctrl_id = ctrl->instance;
+ __entry->result = result;
+ ),
+ TP_printk("nvme%d: NVME_AEN=%#08x [%s]",
+ __entry->ctrl_id, __entry->result,
+ __print_symbolic(__entry->result,
+ aer_name(NVME_AER_NOTICE_NS_CHANGED),
+ aer_name(NVME_AER_NOTICE_ANA),
+ aer_name(NVME_AER_NOTICE_FW_ACT_STARTING),
+ aer_name(NVME_AER_ERROR),
+ aer_name(NVME_AER_SMART),
+ aer_name(NVME_AER_CSS),
+ aer_name(NVME_AER_VS))
+ )
+);
+
+#undef aer_name
+
#endif /* _TRACE_NVME_H */
#undef TRACE_INCLUDE_PATH
ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->get_log_page.nsid);
if (!ns) {
- pr_err("nvmet : Could not find namespace id : %d\n",
+ pr_err("Could not find namespace id : %d\n",
le32_to_cpu(req->cmd->get_log_page.nsid));
return NVME_SC_INVALID_NS;
}
if (req->port->inline_data_size)
id->sgls |= cpu_to_le32(1 << 20);
- strcpy(id->subnqn, ctrl->subsys->subsysnqn);
+ strlcpy(id->subnqn, ctrl->subsys->subsysnqn, sizeof(id->subnqn));
/* Max command capsule size is sqe + single page of in-capsule data */
id->ioccsz = cpu_to_le32((sizeof(struct nvme_command) +
if (!port)
return NULL;
- if (!strncmp(NVME_DISC_SUBSYS_NAME, subsysnqn,
- NVMF_NQN_SIZE)) {
+ if (!strcmp(NVME_DISC_SUBSYS_NAME, subsysnqn)) {
if (!kref_get_unless_zero(&nvmet_disc_subsys->ref))
return NULL;
return nvmet_disc_subsys;
if (req->port->inline_data_size)
id->sgls |= cpu_to_le32(1 << 20);
- strcpy(id->subnqn, ctrl->subsys->subsysnqn);
+ strlcpy(id->subnqn, ctrl->subsys->subsysnqn, sizeof(id->subnqn));
status = nvmet_copy_to_sgl(req, 0, id, sizeof(*id));
return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
}
default:
- pr_err("unsupported cmd %d\n", cmd->common.opcode);
+ pr_err("unhandled cmd %d\n", cmd->common.opcode);
return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
}
- pr_err("unhandled cmd %d\n", cmd->common.opcode);
- return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
}
int __init nvmet_init_discovery(void)
struct list_head ls_busylist;
struct list_head assoc_list;
struct ida assoc_cnt;
- struct nvmet_port *port;
+ struct nvmet_fc_port_entry *pe;
struct kref ref;
u32 max_sg_cnt;
};
+struct nvmet_fc_port_entry {
+ struct nvmet_fc_tgtport *tgtport;
+ struct nvmet_port *port;
+ u64 node_name;
+ u64 port_name;
+ struct list_head pe_list;
+};
+
struct nvmet_fc_defer_fcp_req {
struct list_head req_list;
struct nvmefc_tgt_fcp_req *fcp_req;
atomic_t zrspcnt;
atomic_t rsn;
spinlock_t qlock;
- struct nvmet_port *port;
struct nvmet_cq nvme_cq;
struct nvmet_sq nvme_sq;
struct nvmet_fc_tgt_assoc *assoc;
static LIST_HEAD(nvmet_fc_target_list);
static DEFINE_IDA(nvmet_fc_tgtport_cnt);
+static LIST_HEAD(nvmet_fc_portentry_list);
static void nvmet_fc_handle_ls_rqst_work(struct work_struct *work);
queue->qid = qid;
queue->sqsize = sqsize;
queue->assoc = assoc;
- queue->port = assoc->tgtport->port;
queue->cpu = nvmet_fc_queue_to_cpu(assoc->tgtport, qid);
INIT_LIST_HEAD(&queue->fod_list);
INIT_LIST_HEAD(&queue->avail_defer_list);
return ret;
}
+static void
+nvmet_fc_portentry_bind(struct nvmet_fc_tgtport *tgtport,
+ struct nvmet_fc_port_entry *pe,
+ struct nvmet_port *port)
+{
+ lockdep_assert_held(&nvmet_fc_tgtlock);
+
+ pe->tgtport = tgtport;
+ tgtport->pe = pe;
+
+ pe->port = port;
+ port->priv = pe;
+
+ pe->node_name = tgtport->fc_target_port.node_name;
+ pe->port_name = tgtport->fc_target_port.port_name;
+ INIT_LIST_HEAD(&pe->pe_list);
+
+ list_add_tail(&pe->pe_list, &nvmet_fc_portentry_list);
+}
+
+static void
+nvmet_fc_portentry_unbind(struct nvmet_fc_port_entry *pe)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&nvmet_fc_tgtlock, flags);
+ if (pe->tgtport)
+ pe->tgtport->pe = NULL;
+ list_del(&pe->pe_list);
+ spin_unlock_irqrestore(&nvmet_fc_tgtlock, flags);
+}
+
+/*
+ * called when a targetport deregisters. Breaks the relationship
+ * with the nvmet port, but leaves the port_entry in place so that
+ * re-registration can resume operation.
+ */
+static void
+nvmet_fc_portentry_unbind_tgt(struct nvmet_fc_tgtport *tgtport)
+{
+ struct nvmet_fc_port_entry *pe;
+ unsigned long flags;
+
+ spin_lock_irqsave(&nvmet_fc_tgtlock, flags);
+ pe = tgtport->pe;
+ if (pe)
+ pe->tgtport = NULL;
+ tgtport->pe = NULL;
+ spin_unlock_irqrestore(&nvmet_fc_tgtlock, flags);
+}
+
+/*
+ * called when a new targetport is registered. Looks in the
+ * existing nvmet port_entries to see if the nvmet layer is
+ * configured for the targetport's wwn's. (the targetport existed,
+ * nvmet configured, the lldd unregistered the tgtport, and is now
+ * reregistering the same targetport). If so, set the nvmet port
+ * port entry on the targetport.
+ */
+static void
+nvmet_fc_portentry_rebind_tgt(struct nvmet_fc_tgtport *tgtport)
+{
+ struct nvmet_fc_port_entry *pe;
+ unsigned long flags;
+
+ spin_lock_irqsave(&nvmet_fc_tgtlock, flags);
+ list_for_each_entry(pe, &nvmet_fc_portentry_list, pe_list) {
+ if (tgtport->fc_target_port.node_name == pe->node_name &&
+ tgtport->fc_target_port.port_name == pe->port_name) {
+ WARN_ON(pe->tgtport);
+ tgtport->pe = pe;
+ pe->tgtport = tgtport;
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&nvmet_fc_tgtlock, flags);
+}
/**
* nvme_fc_register_targetport - transport entry point called by an
goto out_free_newrec;
}
+ nvmet_fc_portentry_rebind_tgt(newrec);
+
spin_lock_irqsave(&nvmet_fc_tgtlock, flags);
list_add_tail(&newrec->tgt_list, &nvmet_fc_target_list);
spin_unlock_irqrestore(&nvmet_fc_tgtlock, flags);
* nvme_fc_unregister_targetport - transport entry point called by an
* LLDD to deregister/remove a previously
* registered a local NVME subsystem FC port.
- * @tgtport: pointer to the (registered) target port that is to be
- * deregistered.
+ * @target_port: pointer to the (registered) target port that is to be
+ * deregistered.
*
* Returns:
* a completion status. Must be 0 upon success; a negative errno
{
struct nvmet_fc_tgtport *tgtport = targetport_to_tgtport(target_port);
+ nvmet_fc_portentry_unbind_tgt(tgtport);
+
/* terminate any outstanding associations */
__nvmet_fc_free_assocs(tgtport);
*
* If this routine returns error, the LLDD should abort the exchange.
*
- * @tgtport: pointer to the (registered) target port the LS was
+ * @target_port: pointer to the (registered) target port the LS was
* received on.
* @lsreq: pointer to a lsreq request structure to be used to reference
* the exchange corresponding to the LS.
/*
- * Actual processing routine for received FC-NVME LS Requests from the LLD
+ * Actual processing routine for received FC-NVME I/O Requests from the LLD
*/
static void
nvmet_fc_handle_fcp_rqst(struct nvmet_fc_tgtport *tgtport,
u32 xfrlen = be32_to_cpu(cmdiu->data_len);
int ret;
+ /*
+ * if there is no nvmet mapping to the targetport there
+ * shouldn't be requests. just terminate them.
+ */
+ if (!tgtport->pe)
+ goto transport_error;
+
/*
* Fused commands are currently not supported in the linux
* implementation.
fod->req.cmd = &fod->cmdiubuf.sqe;
fod->req.rsp = &fod->rspiubuf.cqe;
- fod->req.port = fod->queue->port;
+ fod->req.port = tgtport->pe->port;
/* clear any response payload */
memset(&fod->rspiubuf, 0, sizeof(fod->rspiubuf));
substring_t wwn = { name, &name[sizeof(name)-1] };
int nnoffset, pnoffset;
- /* validate it string one of the 2 allowed formats */
+ /* validate if string is one of the 2 allowed formats */
if (strnlen(buf, blen) == NVME_FC_TRADDR_MAXLENGTH &&
!strncmp(buf, "nn-0x", NVME_FC_TRADDR_OXNNLEN) &&
!strncmp(&buf[NVME_FC_TRADDR_MAX_PN_OFFSET],
nvmet_fc_add_port(struct nvmet_port *port)
{
struct nvmet_fc_tgtport *tgtport;
+ struct nvmet_fc_port_entry *pe;
struct nvmet_fc_traddr traddr = { 0L, 0L };
unsigned long flags;
int ret;
if (ret)
return ret;
+ pe = kzalloc(sizeof(*pe), GFP_KERNEL);
+ if (!pe)
+ return -ENOMEM;
+
ret = -ENXIO;
spin_lock_irqsave(&nvmet_fc_tgtlock, flags);
list_for_each_entry(tgtport, &nvmet_fc_target_list, tgt_list) {
if ((tgtport->fc_target_port.node_name == traddr.nn) &&
(tgtport->fc_target_port.port_name == traddr.pn)) {
- tgtport->port = port;
- ret = 0;
+ /* a FC port can only be 1 nvmet port id */
+ if (!tgtport->pe) {
+ nvmet_fc_portentry_bind(tgtport, pe, port);
+ ret = 0;
+ } else
+ ret = -EALREADY;
break;
}
}
spin_unlock_irqrestore(&nvmet_fc_tgtlock, flags);
+
+ if (ret)
+ kfree(pe);
+
return ret;
}
static void
nvmet_fc_remove_port(struct nvmet_port *port)
{
- /* nothing to do */
+ struct nvmet_fc_port_entry *pe = port->priv;
+
+ nvmet_fc_portentry_unbind(pe);
+
+ kfree(pe);
}
static const struct nvmet_fabrics_ops nvmet_fc_tgt_fcp_ops = {
break;
/* Fall-Thru to RSP handling */
+ /* FALLTHRU */
case NVMET_FCOP_RSP:
if (fcpreq) {
static void nvmet_bdev_execute_rw(struct nvmet_req *req)
{
int sg_cnt = req->sg_cnt;
- struct bio *bio = &req->b.inline_bio;
+ struct bio *bio;
struct scatterlist *sg;
sector_t sector;
blk_qc_t cookie;
sector = le64_to_cpu(req->cmd->rw.slba);
sector <<= (req->ns->blksize_shift - 9);
- bio_init(bio, req->inline_bvec, ARRAY_SIZE(req->inline_bvec));
+ if (req->data_len <= NVMET_MAX_INLINE_DATA_LEN) {
+ bio = &req->b.inline_bio;
+ bio_init(bio, req->inline_bvec, ARRAY_SIZE(req->inline_bvec));
+ } else {
+ bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
+ }
bio_set_dev(bio, req->ns->bdev);
bio->bi_iter.bi_sector = sector;
bio->bi_private = req;
break;
offset = le64_to_cpu(range.slba) << req->ns->blksize_shift;
- len = le32_to_cpu(range.nlb) << req->ns->blksize_shift;
+ len = le32_to_cpu(range.nlb);
+ len <<= req->ns->blksize_shift;
if (offset + len > req->ns->size) {
ret = NVME_SC_LBA_RANGE | NVME_SC_DNR;
break;
};
#define NVMET_MAX_INLINE_BIOVEC 8
+#define NVMET_MAX_INLINE_DATA_LEN NVMET_MAX_INLINE_BIOVEC * PAGE_SIZE
struct nvmet_req {
struct nvme_command *cmd;
int inline_page_count;
};
+static struct workqueue_struct *nvmet_rdma_delete_wq;
static bool nvmet_rdma_use_srq;
module_param_named(use_srq, nvmet_rdma_use_srq, bool, 0444);
MODULE_PARM_DESC(use_srq, "Use shared receive queue.");
if (queue->host_qid == 0) {
/* Let inflight controller teardown complete */
- flush_scheduled_work();
+ flush_workqueue(nvmet_rdma_delete_wq);
}
ret = nvmet_rdma_cm_accept(cm_id, queue, &event->param.conn);
if (ret) {
- schedule_work(&queue->release_work);
+ queue_work(nvmet_rdma_delete_wq, &queue->release_work);
/* Destroying rdma_cm id is not needed here */
return 0;
}
if (disconnect) {
rdma_disconnect(queue->cm_id);
- schedule_work(&queue->release_work);
+ queue_work(nvmet_rdma_delete_wq, &queue->release_work);
}
}
mutex_unlock(&nvmet_rdma_queue_mutex);
pr_err("failed to connect queue %d\n", queue->idx);
- schedule_work(&queue->release_work);
+ queue_work(nvmet_rdma_delete_wq, &queue->release_work);
}
/**
if (ret)
goto err_ib_client;
+ nvmet_rdma_delete_wq = alloc_workqueue("nvmet-rdma-delete-wq",
+ WQ_UNBOUND | WQ_MEM_RECLAIM | WQ_SYSFS, 0);
+ if (!nvmet_rdma_delete_wq) {
+ ret = -ENOMEM;
+ goto err_unreg_transport;
+ }
+
return 0;
+err_unreg_transport:
+ nvmet_unregister_transport(&nvmet_rdma_ops);
err_ib_client:
ib_unregister_client(&nvmet_rdma_ib_client);
return ret;
static void __exit nvmet_rdma_exit(void)
{
+ destroy_workqueue(nvmet_rdma_delete_wq);
nvmet_unregister_transport(&nvmet_rdma_ops);
ib_unregister_client(&nvmet_rdma_ib_client);
WARN_ON_ONCE(!list_empty(&nvmet_rdma_queue_list));
}
EXPORT_SYMBOL_GPL(of_dma_configure);
-/**
- * of_dma_deconfigure - Clean up DMA configuration
- * @dev: Device for which to clean up DMA configuration
- *
- * Clean up all configuration performed by of_dma_configure_ops() and free all
- * resources that have been allocated.
- */
-void of_dma_deconfigure(struct device *dev)
-{
- arch_teardown_dma_ops(dev);
-}
-
int of_device_register(struct platform_device *pdev)
{
device_initialize(&pdev->dev);
struct of_phandle_args args;
int i, rc;
+ if (of_irq_workarounds & OF_IMAP_OLDWORLD_MAC)
+ return;
+
np = of_find_node_by_path("/testcase-data/interrupts/interrupts0");
if (!np) {
pr_err("missing testcase data\n");
struct of_phandle_args args;
int i, rc;
+ if (of_irq_workarounds & OF_IMAP_OLDWORLD_MAC)
+ return;
+
np = of_find_node_by_path("/testcase-data/interrupts/interrupts-extended0");
if (!np) {
pr_err("missing testcase data\n");
pdev = of_find_device_by_node(np);
unittest(pdev, "device 1 creation failed\n");
- irq = platform_get_irq(pdev, 0);
- unittest(irq == -EPROBE_DEFER, "device deferred probe failed - %d\n", irq);
+ if (!(of_irq_workarounds & OF_IMAP_OLDWORLD_MAC)) {
+ irq = platform_get_irq(pdev, 0);
+ unittest(irq == -EPROBE_DEFER,
+ "device deferred probe failed - %d\n", irq);
- /* Test that a parsing failure does not return -EPROBE_DEFER */
- np = of_find_node_by_path("/testcase-data/testcase-device2");
- pdev = of_find_device_by_node(np);
- unittest(pdev, "device 2 creation failed\n");
- irq = platform_get_irq(pdev, 0);
- unittest(irq < 0 && irq != -EPROBE_DEFER, "device parsing error failed - %d\n", irq);
+ /* Test that a parsing failure does not return -EPROBE_DEFER */
+ np = of_find_node_by_path("/testcase-data/testcase-device2");
+ pdev = of_find_device_by_node(np);
+ unittest(pdev, "device 2 creation failed\n");
+ irq = platform_get_irq(pdev, 0);
+ unittest(irq < 0 && irq != -EPROBE_DEFER,
+ "device parsing error failed - %d\n", irq);
+ }
np = of_find_node_by_path("/testcase-data/platform-tests");
unittest(np, "No testcase data in device tree\n");
static struct opp_table *_find_opp_table_unlocked(struct device *dev)
{
struct opp_table *opp_table;
+ bool found;
list_for_each_entry(opp_table, &opp_tables, node) {
- if (_find_opp_dev(dev, opp_table)) {
+ mutex_lock(&opp_table->lock);
+ found = !!_find_opp_dev(dev, opp_table);
+ mutex_unlock(&opp_table->lock);
+
+ if (found) {
_get_opp_table_kref(opp_table);
return opp_table;
count = PTR_ERR(opp_table);
dev_dbg(dev, "%s: OPP table not found (%d)\n",
__func__, count);
- return 0;
+ return count;
}
count = _get_opp_count(opp_table);
kfree(opp_dev);
}
-struct opp_device *_add_opp_dev(const struct device *dev,
- struct opp_table *opp_table)
+static struct opp_device *_add_opp_dev_unlocked(const struct device *dev,
+ struct opp_table *opp_table)
{
struct opp_device *opp_dev;
int ret;
/* Initialize opp-dev */
opp_dev->dev = dev;
+
list_add(&opp_dev->node, &opp_table->dev_list);
/* Create debugfs entries for the opp_table */
return opp_dev;
}
-static struct opp_table *_allocate_opp_table(struct device *dev)
+struct opp_device *_add_opp_dev(const struct device *dev,
+ struct opp_table *opp_table)
+{
+ struct opp_device *opp_dev;
+
+ mutex_lock(&opp_table->lock);
+ opp_dev = _add_opp_dev_unlocked(dev, opp_table);
+ mutex_unlock(&opp_table->lock);
+
+ return opp_dev;
+}
+
+static struct opp_table *_allocate_opp_table(struct device *dev, int index)
{
struct opp_table *opp_table;
struct opp_device *opp_dev;
if (!opp_table)
return NULL;
+ mutex_init(&opp_table->lock);
INIT_LIST_HEAD(&opp_table->dev_list);
opp_dev = _add_opp_dev(dev, opp_table);
return NULL;
}
- _of_init_opp_table(opp_table, dev);
+ _of_init_opp_table(opp_table, dev, index);
/* Find clk for the device */
opp_table->clk = clk_get(dev, NULL);
BLOCKING_INIT_NOTIFIER_HEAD(&opp_table->head);
INIT_LIST_HEAD(&opp_table->opp_list);
- mutex_init(&opp_table->lock);
kref_init(&opp_table->kref);
/* Secure the device table modification */
kref_get(&opp_table->kref);
}
-struct opp_table *dev_pm_opp_get_opp_table(struct device *dev)
+static struct opp_table *_opp_get_opp_table(struct device *dev, int index)
{
struct opp_table *opp_table;
if (!IS_ERR(opp_table))
goto unlock;
- opp_table = _allocate_opp_table(dev);
+ opp_table = _managed_opp(dev, index);
+ if (opp_table) {
+ if (!_add_opp_dev_unlocked(dev, opp_table)) {
+ dev_pm_opp_put_opp_table(opp_table);
+ opp_table = NULL;
+ }
+ goto unlock;
+ }
+
+ opp_table = _allocate_opp_table(dev, index);
unlock:
mutex_unlock(&opp_table_lock);
return opp_table;
}
+
+struct opp_table *dev_pm_opp_get_opp_table(struct device *dev)
+{
+ return _opp_get_opp_table(dev, 0);
+}
EXPORT_SYMBOL_GPL(dev_pm_opp_get_opp_table);
+struct opp_table *dev_pm_opp_get_opp_table_indexed(struct device *dev,
+ int index)
+{
+ return _opp_get_opp_table(dev, index);
+}
+
static void _opp_table_kref_release(struct kref *kref)
{
struct opp_table *opp_table = container_of(kref, struct opp_table, kref);
- struct opp_device *opp_dev;
+ struct opp_device *opp_dev, *temp;
/* Release clk */
if (!IS_ERR(opp_table->clk))
clk_put(opp_table->clk);
- opp_dev = list_first_entry(&opp_table->dev_list, struct opp_device,
- node);
+ WARN_ON(!list_empty(&opp_table->opp_list));
- _remove_opp_dev(opp_dev, opp_table);
+ list_for_each_entry_safe(opp_dev, temp, &opp_table->dev_list, node) {
+ /*
+ * The OPP table is getting removed, drop the performance state
+ * constraints.
+ */
+ if (opp_table->genpd_performance_state)
+ dev_pm_genpd_set_performance_state((struct device *)(opp_dev->dev), 0);
- /* dev_list must be empty now */
- WARN_ON(!list_empty(&opp_table->dev_list));
+ _remove_opp_dev(opp_dev, opp_table);
+ }
mutex_destroy(&opp_table->lock);
list_del(&opp_table->node);
mutex_unlock(&opp_table_lock);
}
+void _opp_remove_all_static(struct opp_table *opp_table)
+{
+ struct dev_pm_opp *opp, *tmp;
+
+ list_for_each_entry_safe(opp, tmp, &opp_table->opp_list, node) {
+ if (!opp->dynamic)
+ dev_pm_opp_put(opp);
+ }
+
+ opp_table->parsed_static_opps = false;
+}
+
+static void _opp_table_list_kref_release(struct kref *kref)
+{
+ struct opp_table *opp_table = container_of(kref, struct opp_table,
+ list_kref);
+
+ _opp_remove_all_static(opp_table);
+ mutex_unlock(&opp_table_lock);
+}
+
+void _put_opp_list_kref(struct opp_table *opp_table)
+{
+ kref_put_mutex(&opp_table->list_kref, _opp_table_list_kref_release,
+ &opp_table_lock);
+}
+
void dev_pm_opp_put_opp_table(struct opp_table *opp_table)
{
kref_put_mutex(&opp_table->kref, _opp_table_kref_release,
kfree(opp);
mutex_unlock(&opp_table->lock);
- dev_pm_opp_put_opp_table(opp_table);
}
void dev_pm_opp_get(struct dev_pm_opp *opp)
if (found) {
dev_pm_opp_put(opp);
+
+ /* Drop the reference taken by dev_pm_opp_add() */
+ dev_pm_opp_put_opp_table(opp_table);
} else {
dev_warn(dev, "%s: Couldn't find OPP with freq: %lu\n",
__func__, freq);
}
+ /* Drop the reference taken by _find_opp_table() */
dev_pm_opp_put_opp_table(opp_table);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_remove);
new_opp->opp_table = opp_table;
kref_init(&new_opp->kref);
- /* Get a reference to the OPP table */
- _get_opp_table_kref(opp_table);
-
ret = opp_debug_create_one(new_opp, opp_table);
if (ret)
dev_err(dev, "%s: Failed to register opp to debugfs (%d)\n",
return -ENOMEM;
ret = _opp_add_v1(opp_table, dev, freq, u_volt, true);
+ if (ret)
+ dev_pm_opp_put_opp_table(opp_table);
- dev_pm_opp_put_opp_table(opp_table);
return ret;
}
EXPORT_SYMBOL_GPL(dev_pm_opp_add);
}
EXPORT_SYMBOL(dev_pm_opp_unregister_notifier);
-/*
- * Free OPPs either created using static entries present in DT or even the
- * dynamically added entries based on remove_all param.
- */
-void _dev_pm_opp_remove_table(struct opp_table *opp_table, struct device *dev,
- bool remove_all)
-{
- struct dev_pm_opp *opp, *tmp;
-
- /* Find if opp_table manages a single device */
- if (list_is_singular(&opp_table->dev_list)) {
- /* Free static OPPs */
- list_for_each_entry_safe(opp, tmp, &opp_table->opp_list, node) {
- if (remove_all || !opp->dynamic)
- dev_pm_opp_put(opp);
- }
-
- /*
- * The OPP table is getting removed, drop the performance state
- * constraints.
- */
- if (opp_table->genpd_performance_state)
- dev_pm_genpd_set_performance_state(dev, 0);
- } else {
- _remove_opp_dev(_find_opp_dev(dev, opp_table), opp_table);
- }
-}
-
-void _dev_pm_opp_find_and_remove_table(struct device *dev, bool remove_all)
+void _dev_pm_opp_find_and_remove_table(struct device *dev)
{
struct opp_table *opp_table;
return;
}
- _dev_pm_opp_remove_table(opp_table, dev, remove_all);
+ _put_opp_list_kref(opp_table);
+
+ /* Drop reference taken by _find_opp_table() */
+ dev_pm_opp_put_opp_table(opp_table);
+ /* Drop reference taken while the OPP table was added */
dev_pm_opp_put_opp_table(opp_table);
}
*/
void dev_pm_opp_remove_table(struct device *dev)
{
- _dev_pm_opp_find_and_remove_table(dev, true);
+ _dev_pm_opp_find_and_remove_table(dev);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_remove_table);
EXPORT_SYMBOL_GPL(dev_pm_opp_free_cpufreq_table);
#endif /* CONFIG_CPU_FREQ */
-void _dev_pm_opp_cpumask_remove_table(const struct cpumask *cpumask, bool of)
+void _dev_pm_opp_cpumask_remove_table(const struct cpumask *cpumask,
+ int last_cpu)
{
struct device *cpu_dev;
int cpu;
WARN_ON(cpumask_empty(cpumask));
for_each_cpu(cpu, cpumask) {
+ if (cpu == last_cpu)
+ break;
+
cpu_dev = get_cpu_device(cpu);
if (!cpu_dev) {
pr_err("%s: failed to get cpu%d device\n", __func__,
continue;
}
- if (of)
- dev_pm_opp_of_remove_table(cpu_dev);
- else
- dev_pm_opp_remove_table(cpu_dev);
+ _dev_pm_opp_find_and_remove_table(cpu_dev);
}
}
*/
void dev_pm_opp_cpumask_remove_table(const struct cpumask *cpumask)
{
- _dev_pm_opp_cpumask_remove_table(cpumask, false);
+ _dev_pm_opp_cpumask_remove_table(cpumask, -1);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_cpumask_remove_table);
cpumask_clear(cpumask);
if (opp_table->shared_opp == OPP_TABLE_ACCESS_SHARED) {
+ mutex_lock(&opp_table->lock);
list_for_each_entry(opp_dev, &opp_table->dev_list, node)
cpumask_set_cpu(opp_dev->dev->id, cpumask);
+ mutex_unlock(&opp_table->lock);
} else {
cpumask_set_cpu(cpu_dev->id, cpumask);
}
#include "opp.h"
-static struct opp_table *_managed_opp(const struct device_node *np)
+/*
+ * Returns opp descriptor node for a device node, caller must
+ * do of_node_put().
+ */
+static struct device_node *_opp_of_get_opp_desc_node(struct device_node *np,
+ int index)
+{
+ /* "operating-points-v2" can be an array for power domain providers */
+ return of_parse_phandle(np, "operating-points-v2", index);
+}
+
+/* Returns opp descriptor node for a device, caller must do of_node_put() */
+struct device_node *dev_pm_opp_of_get_opp_desc_node(struct device *dev)
+{
+ return _opp_of_get_opp_desc_node(dev->of_node, 0);
+}
+EXPORT_SYMBOL_GPL(dev_pm_opp_of_get_opp_desc_node);
+
+struct opp_table *_managed_opp(struct device *dev, int index)
{
struct opp_table *opp_table, *managed_table = NULL;
+ struct device_node *np;
- mutex_lock(&opp_table_lock);
+ np = _opp_of_get_opp_desc_node(dev->of_node, index);
+ if (!np)
+ return NULL;
list_for_each_entry(opp_table, &opp_tables, node) {
if (opp_table->np == np) {
}
}
- mutex_unlock(&opp_table_lock);
+ of_node_put(np);
return managed_table;
}
-void _of_init_opp_table(struct opp_table *opp_table, struct device *dev)
+void _of_init_opp_table(struct opp_table *opp_table, struct device *dev,
+ int index)
{
- struct device_node *np;
+ struct device_node *np, *opp_np;
+ u32 val;
/*
* Only required for backward compatibility with v1 bindings, but isn't
* harmful for other cases. And so we do it unconditionally.
*/
np = of_node_get(dev->of_node);
- if (np) {
- u32 val;
-
- if (!of_property_read_u32(np, "clock-latency", &val))
- opp_table->clock_latency_ns_max = val;
- of_property_read_u32(np, "voltage-tolerance",
- &opp_table->voltage_tolerance_v1);
- of_node_put(np);
- }
+ if (!np)
+ return;
+
+ if (!of_property_read_u32(np, "clock-latency", &val))
+ opp_table->clock_latency_ns_max = val;
+ of_property_read_u32(np, "voltage-tolerance",
+ &opp_table->voltage_tolerance_v1);
+
+ /* Get OPP table node */
+ opp_np = _opp_of_get_opp_desc_node(np, index);
+ of_node_put(np);
+
+ if (!opp_np)
+ return;
+
+ if (of_property_read_bool(opp_np, "opp-shared"))
+ opp_table->shared_opp = OPP_TABLE_ACCESS_SHARED;
+ else
+ opp_table->shared_opp = OPP_TABLE_ACCESS_EXCLUSIVE;
+
+ opp_table->np = opp_np;
+
+ of_node_put(opp_np);
}
static bool _opp_is_supported(struct device *dev, struct opp_table *opp_table,
*/
void dev_pm_opp_of_remove_table(struct device *dev)
{
- _dev_pm_opp_find_and_remove_table(dev, false);
+ _dev_pm_opp_find_and_remove_table(dev);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_of_remove_table);
-/* Returns opp descriptor node for a device node, caller must
- * do of_node_put() */
-static struct device_node *_opp_of_get_opp_desc_node(struct device_node *np,
- int index)
-{
- /* "operating-points-v2" can be an array for power domain providers */
- return of_parse_phandle(np, "operating-points-v2", index);
-}
-
-/* Returns opp descriptor node for a device, caller must do of_node_put() */
-struct device_node *dev_pm_opp_of_get_opp_desc_node(struct device *dev)
-{
- return _opp_of_get_opp_desc_node(dev->of_node, 0);
-}
-EXPORT_SYMBOL_GPL(dev_pm_opp_of_get_opp_desc_node);
-
/**
* _opp_add_static_v2() - Allocate static OPPs (As per 'v2' DT bindings)
* @opp_table: OPP table
* removed by dev_pm_opp_remove.
*
* Return:
- * 0 On success OR
+ * Valid OPP pointer:
+ * On success
+ * NULL:
* Duplicate OPPs (both freq and volt are same) and opp->available
- * -EEXIST Freq are same and volt are different OR
+ * OR if the OPP is not supported by hardware.
+ * ERR_PTR(-EEXIST):
+ * Freq are same and volt are different OR
* Duplicate OPPs (both freq and volt are same) and !opp->available
- * -ENOMEM Memory allocation failure
- * -EINVAL Failed parsing the OPP node
+ * ERR_PTR(-ENOMEM):
+ * Memory allocation failure
+ * ERR_PTR(-EINVAL):
+ * Failed parsing the OPP node
*/
-static int _opp_add_static_v2(struct opp_table *opp_table, struct device *dev,
- struct device_node *np)
+static struct dev_pm_opp *_opp_add_static_v2(struct opp_table *opp_table,
+ struct device *dev, struct device_node *np)
{
struct dev_pm_opp *new_opp;
u64 rate = 0;
new_opp = _opp_allocate(opp_table);
if (!new_opp)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
ret = of_property_read_u64(np, "opp-hz", &rate);
if (ret < 0) {
* frequency/voltage list.
*/
blocking_notifier_call_chain(&opp_table->head, OPP_EVENT_ADD, new_opp);
- return 0;
+ return new_opp;
free_opp:
_opp_free(new_opp);
- return ret;
+ return ERR_PTR(ret);
}
/* Initializes OPP tables based on new bindings */
-static int _of_add_opp_table_v2(struct device *dev, struct device_node *opp_np)
+static int _of_add_opp_table_v2(struct device *dev, struct opp_table *opp_table)
{
struct device_node *np;
- struct opp_table *opp_table;
- int ret = 0, count = 0, pstate_count = 0;
+ int ret, count = 0, pstate_count = 0;
struct dev_pm_opp *opp;
- opp_table = _managed_opp(opp_np);
- if (opp_table) {
- /* OPPs are already managed */
- if (!_add_opp_dev(dev, opp_table))
- ret = -ENOMEM;
- goto put_opp_table;
+ /* OPP table is already initialized for the device */
+ if (opp_table->parsed_static_opps) {
+ kref_get(&opp_table->list_kref);
+ return 0;
}
- opp_table = dev_pm_opp_get_opp_table(dev);
- if (!opp_table)
- return -ENOMEM;
+ kref_init(&opp_table->list_kref);
/* We have opp-table node now, iterate over it and add OPPs */
- for_each_available_child_of_node(opp_np, np) {
- count++;
-
- ret = _opp_add_static_v2(opp_table, dev, np);
- if (ret) {
+ for_each_available_child_of_node(opp_table->np, np) {
+ opp = _opp_add_static_v2(opp_table, dev, np);
+ if (IS_ERR(opp)) {
+ ret = PTR_ERR(opp);
dev_err(dev, "%s: Failed to add OPP, %d\n", __func__,
ret);
- _dev_pm_opp_remove_table(opp_table, dev, false);
of_node_put(np);
- goto put_opp_table;
+ goto put_list_kref;
+ } else if (opp) {
+ count++;
}
}
/* There should be one of more OPP defined */
if (WARN_ON(!count)) {
ret = -ENOENT;
- goto put_opp_table;
+ goto put_list_kref;
}
list_for_each_entry(opp, &opp_table->opp_list, node)
dev_err(dev, "Not all nodes have performance state set (%d: %d)\n",
count, pstate_count);
ret = -ENOENT;
- goto put_opp_table;
+ goto put_list_kref;
}
if (pstate_count)
opp_table->genpd_performance_state = true;
- opp_table->np = opp_np;
- if (of_property_read_bool(opp_np, "opp-shared"))
- opp_table->shared_opp = OPP_TABLE_ACCESS_SHARED;
- else
- opp_table->shared_opp = OPP_TABLE_ACCESS_EXCLUSIVE;
+ opp_table->parsed_static_opps = true;
-put_opp_table:
- dev_pm_opp_put_opp_table(opp_table);
+ return 0;
+
+put_list_kref:
+ _put_opp_list_kref(opp_table);
return ret;
}
/* Initializes OPP tables based on old-deprecated bindings */
-static int _of_add_opp_table_v1(struct device *dev)
+static int _of_add_opp_table_v1(struct device *dev, struct opp_table *opp_table)
{
- struct opp_table *opp_table;
const struct property *prop;
const __be32 *val;
int nr, ret = 0;
return -EINVAL;
}
- opp_table = dev_pm_opp_get_opp_table(dev);
- if (!opp_table)
- return -ENOMEM;
+ kref_init(&opp_table->list_kref);
val = prop->value;
while (nr) {
if (ret) {
dev_err(dev, "%s: Failed to add OPP %ld (%d)\n",
__func__, freq, ret);
- _dev_pm_opp_remove_table(opp_table, dev, false);
- break;
+ _put_opp_list_kref(opp_table);
+ return ret;
}
nr -= 2;
}
- dev_pm_opp_put_opp_table(opp_table);
return ret;
}
*/
int dev_pm_opp_of_add_table(struct device *dev)
{
- struct device_node *opp_np;
+ struct opp_table *opp_table;
int ret;
+ opp_table = dev_pm_opp_get_opp_table_indexed(dev, 0);
+ if (!opp_table)
+ return -ENOMEM;
+
/*
- * OPPs have two version of bindings now. The older one is deprecated,
- * try for the new binding first.
+ * OPPs have two version of bindings now. Also try the old (v1)
+ * bindings for backward compatibility with older dtbs.
*/
- opp_np = dev_pm_opp_of_get_opp_desc_node(dev);
- if (!opp_np) {
- /*
- * Try old-deprecated bindings for backward compatibility with
- * older dtbs.
- */
- return _of_add_opp_table_v1(dev);
- }
+ if (opp_table->np)
+ ret = _of_add_opp_table_v2(dev, opp_table);
+ else
+ ret = _of_add_opp_table_v1(dev, opp_table);
- ret = _of_add_opp_table_v2(dev, opp_np);
- of_node_put(opp_np);
+ if (ret)
+ dev_pm_opp_put_opp_table(opp_table);
return ret;
}
*/
int dev_pm_opp_of_add_table_indexed(struct device *dev, int index)
{
- struct device_node *opp_np;
+ struct opp_table *opp_table;
int ret, count;
-again:
- opp_np = _opp_of_get_opp_desc_node(dev->of_node, index);
- if (!opp_np) {
+ if (index) {
/*
* If only one phandle is present, then the same OPP table
* applies for all index requests.
*/
count = of_count_phandle_with_args(dev->of_node,
"operating-points-v2", NULL);
- if (count == 1 && index) {
- index = 0;
- goto again;
- }
+ if (count != 1)
+ return -ENODEV;
- return -ENODEV;
+ index = 0;
}
- ret = _of_add_opp_table_v2(dev, opp_np);
- of_node_put(opp_np);
+ opp_table = dev_pm_opp_get_opp_table_indexed(dev, index);
+ if (!opp_table)
+ return -ENOMEM;
+
+ ret = _of_add_opp_table_v2(dev, opp_table);
+ if (ret)
+ dev_pm_opp_put_opp_table(opp_table);
return ret;
}
*/
void dev_pm_opp_of_cpumask_remove_table(const struct cpumask *cpumask)
{
- _dev_pm_opp_cpumask_remove_table(cpumask, true);
+ _dev_pm_opp_cpumask_remove_table(cpumask, -1);
}
EXPORT_SYMBOL_GPL(dev_pm_opp_of_cpumask_remove_table);
int dev_pm_opp_of_cpumask_add_table(const struct cpumask *cpumask)
{
struct device *cpu_dev;
- int cpu, ret = 0;
+ int cpu, ret;
- WARN_ON(cpumask_empty(cpumask));
+ if (WARN_ON(cpumask_empty(cpumask)))
+ return -ENODEV;
for_each_cpu(cpu, cpumask) {
cpu_dev = get_cpu_device(cpu);
if (!cpu_dev) {
pr_err("%s: failed to get cpu%d device\n", __func__,
cpu);
- continue;
+ ret = -ENODEV;
+ goto remove_table;
}
ret = dev_pm_opp_of_add_table(cpu_dev);
pr_debug("%s: couldn't find opp table for cpu:%d, %d\n",
__func__, cpu, ret);
- /* Free all other OPPs */
- dev_pm_opp_of_cpumask_remove_table(cpumask);
- break;
+ goto remove_table;
}
}
+ return 0;
+
+remove_table:
+ /* Free all other OPPs */
+ _dev_pm_opp_cpumask_remove_table(cpumask, cpu);
+
return ret;
}
EXPORT_SYMBOL_GPL(dev_pm_opp_of_cpumask_add_table);
* @dev_list: list of devices that share these OPPs
* @opp_list: table of opps
* @kref: for reference count of the table.
- * @lock: mutex protecting the opp_list.
+ * @list_kref: for reference count of the OPP list.
+ * @lock: mutex protecting the opp_list and dev_list.
* @np: struct device_node pointer for opp's DT node.
* @clock_latency_ns_max: Max clock latency in nanoseconds.
+ * @parsed_static_opps: True if OPPs are initialized from DT.
* @shared_opp: OPP is shared between multiple devices.
* @suspend_opp: Pointer to OPP to be used during device suspend.
* @supported_hw: Array of version number to support.
struct list_head dev_list;
struct list_head opp_list;
struct kref kref;
+ struct kref list_kref;
struct mutex lock;
struct device_node *np;
/* For backward compatibility with v1 bindings */
unsigned int voltage_tolerance_v1;
+ bool parsed_static_opps;
enum opp_table_access shared_opp;
struct dev_pm_opp *suspend_opp;
/* Routines internal to opp core */
void dev_pm_opp_get(struct dev_pm_opp *opp);
+void _opp_remove_all_static(struct opp_table *opp_table);
void _get_opp_table_kref(struct opp_table *opp_table);
int _get_opp_count(struct opp_table *opp_table);
struct opp_table *_find_opp_table(struct device *dev);
struct opp_device *_add_opp_dev(const struct device *dev, struct opp_table *opp_table);
-void _dev_pm_opp_remove_table(struct opp_table *opp_table, struct device *dev, bool remove_all);
-void _dev_pm_opp_find_and_remove_table(struct device *dev, bool remove_all);
+void _dev_pm_opp_find_and_remove_table(struct device *dev);
struct dev_pm_opp *_opp_allocate(struct opp_table *opp_table);
void _opp_free(struct dev_pm_opp *opp);
int _opp_add(struct device *dev, struct dev_pm_opp *new_opp, struct opp_table *opp_table, bool rate_not_available);
int _opp_add_v1(struct opp_table *opp_table, struct device *dev, unsigned long freq, long u_volt, bool dynamic);
-void _dev_pm_opp_cpumask_remove_table(const struct cpumask *cpumask, bool of);
+void _dev_pm_opp_cpumask_remove_table(const struct cpumask *cpumask, int last_cpu);
struct opp_table *_add_opp_table(struct device *dev);
+void _put_opp_list_kref(struct opp_table *opp_table);
#ifdef CONFIG_OF
-void _of_init_opp_table(struct opp_table *opp_table, struct device *dev);
+void _of_init_opp_table(struct opp_table *opp_table, struct device *dev, int index);
+struct opp_table *_managed_opp(struct device *dev, int index);
#else
-static inline void _of_init_opp_table(struct opp_table *opp_table, struct device *dev) {}
+static inline void _of_init_opp_table(struct opp_table *opp_table, struct device *dev, int index) {}
+static inline struct opp_table *_managed_opp(struct device *dev, int index) { return NULL; }
#endif
#ifdef CONFIG_DEBUG_FS
return 0;
}
- phy = devm_kzalloc(dev, sizeof(*phy) * phy_count, GFP_KERNEL);
+ phy = devm_kcalloc(dev, phy_count, sizeof(*phy), GFP_KERNEL);
if (!phy)
return -ENOMEM;
- link = devm_kzalloc(dev, sizeof(*link) * phy_count, GFP_KERNEL);
+ link = devm_kcalloc(dev, phy_count, sizeof(*link), GFP_KERNEL);
if (!link)
return -ENOMEM;
return vmd_dma_ops(dev)->dma_supported(to_vmd_dev(dev), mask);
}
-#ifdef ARCH_HAS_DMA_GET_REQUIRED_MASK
static u64 vmd_get_required_mask(struct device *dev)
{
return vmd_dma_ops(dev)->get_required_mask(to_vmd_dev(dev));
}
-#endif
static void vmd_teardown_dma_ops(struct vmd_dev *vmd)
{
ASSIGN_VMD_DMA_OPS(source, dest, sync_sg_for_device);
ASSIGN_VMD_DMA_OPS(source, dest, mapping_error);
ASSIGN_VMD_DMA_OPS(source, dest, dma_supported);
-#ifdef ARCH_HAS_DMA_GET_REQUIRED_MASK
ASSIGN_VMD_DMA_OPS(source, dest, get_required_mask);
-#endif
add_dma_domain(domain);
}
#undef ASSIGN_VMD_DMA_OPS
* arch/x86/platform/intel-mid/pwr.c.
*/
static const struct x86_cpu_id lpss_cpu_ids[] = {
- ICPU(INTEL_FAM6_ATOM_PENWELL),
- ICPU(INTEL_FAM6_ATOM_MERRIFIELD),
+ ICPU(INTEL_FAM6_ATOM_SALTWELL_MID),
+ ICPU(INTEL_FAM6_ATOM_SILVERMONT_MID),
{}
};
io_on.stop = s->io[i].res->end;
s->ops->set_io_map(s, &io_off);
- mdelay(40);
+ msleep(40);
s->ops->set_io_map(s, &io_on);
}
unlock:
!(flags & CONF_ENABLE_PULSE_IRQ))
option |= COR_LEVEL_REQ;
pcmcia_write_cis_mem(s, 1, (base + CISREG_COR)>>1, 1, &option);
- mdelay(40);
+ msleep(40);
}
if (p_dev->config_regs & PRESENT_STATUS)
pcmcia_write_cis_mem(s, 1, (base + CISREG_CCSR)>>1, 1, &status);
#define RL5C4XX_MISC_CONTROL 0x2F /* 8 bit */
#define RL5C4XX_ZV_ENABLE 0x08
+/* Misc Control 3 Register */
+#define RL5C4XX_MISC3 0x00A2 /* 16 bit */
+#define RL5C47X_MISC3_CB_CLKRUN_DIS BIT(1)
+
#ifdef __YENTA_H
#define rl_misc(socket) ((socket)->private[0])
}
}
+static void ricoh_set_clkrun(struct yenta_socket *socket, bool quiet)
+{
+ u16 misc3;
+
+ /*
+ * RL5C475II likely has this setting, too, however no datasheet
+ * is publicly available for this chip
+ */
+ if (socket->dev->device != PCI_DEVICE_ID_RICOH_RL5C476 &&
+ socket->dev->device != PCI_DEVICE_ID_RICOH_RL5C478)
+ return;
+
+ if (socket->dev->revision < 0x80)
+ return;
+
+ misc3 = config_readw(socket, RL5C4XX_MISC3);
+ if (misc3 & RL5C47X_MISC3_CB_CLKRUN_DIS) {
+ if (!quiet)
+ dev_dbg(&socket->dev->dev,
+ "CLKRUN feature already disabled\n");
+ } else if (disable_clkrun) {
+ if (!quiet)
+ dev_info(&socket->dev->dev,
+ "Disabling CLKRUN feature\n");
+ misc3 |= RL5C47X_MISC3_CB_CLKRUN_DIS;
+ config_writew(socket, RL5C4XX_MISC3, misc3);
+ }
+}
+
static void ricoh_save_state(struct yenta_socket *socket)
{
rl_misc(socket) = config_readw(socket, RL5C4XX_MISC);
config_writew(socket, RL5C4XX_16BIT_IO_0, rl_io(socket));
config_writew(socket, RL5C4XX_16BIT_MEM_0, rl_mem(socket));
config_writew(socket, RL5C4XX_CONFIG, rl_config(socket));
+ ricoh_set_clkrun(socket, true);
}
config_writew(socket, RL5C4XX_CONFIG, config);
ricoh_set_zv(socket);
+ ricoh_set_clkrun(socket, false);
return 0;
}
if (ret == 0) {
struct gpio_desc *descs[2];
- int values[2], n = 0;
+ DECLARE_BITMAP(values, 2);
+ int n = 0;
if (skt->gpio_reset) {
descs[n] = skt->gpio_reset;
- values[n++] = !!(state->flags & SS_RESET);
+ __assign_bit(n++, values, state->flags & SS_RESET);
}
if (skt->gpio_bus_enable) {
descs[n] = skt->gpio_bus_enable;
- values[n++] = !!(state->flags & SS_OUTPUT_ENA);
+ __assign_bit(n++, values, state->flags & SS_OUTPUT_ENA);
}
if (n)
- gpiod_set_array_value_cansleep(n, descs, values);
+ gpiod_set_array_value_cansleep(n, descs, NULL, values);
/*
* This really needs a better solution. The IRQ
static bool disable_clkrun;
module_param(disable_clkrun, bool, 0444);
-MODULE_PARM_DESC(disable_clkrun, "If PC card doesn't function properly, please try this option");
+MODULE_PARM_DESC(disable_clkrun,
+ "If PC card doesn't function properly, please try this option (TI and Ricoh bridges only)");
static bool isa_probe = 1;
module_param(isa_probe, bool, 0444);
{
struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
unsigned int cpu = smp_processor_id();
- return cpumask_test_cpu(cpu, &armpmu->supported_cpus);
+ int ret;
+
+ ret = cpumask_test_cpu(cpu, &armpmu->supported_cpus);
+ if (ret && armpmu->filter_match)
+ return armpmu->filter_match(event);
+
+ return ret;
}
static ssize_t armpmu_cpumask_show(struct device *dev,
dn = of_parse_phandle(node, "interrupt-affinity", i);
if (!dn) {
- pr_warn("failed to parse interrupt-affinity[%d] for %s\n",
- i, node->name);
+ pr_warn("failed to parse interrupt-affinity[%d] for %pOFn\n",
+ i, node);
return -EINVAL;
}
cpu = of_cpu_node_to_id(dn);
if (cpu < 0) {
- pr_warn("failed to find logical CPU for %s\n", dn->name);
+ pr_warn("failed to find logical CPU for %pOFn\n", dn);
cpu = nr_cpu_ids;
}
*/
static void phy_mdm6600_cmd(struct phy_mdm6600 *ddata, int val)
{
- int values[PHY_MDM6600_NR_CMD_LINES];
- int i;
+ DECLARE_BITMAP(values, PHY_MDM6600_NR_CMD_LINES);
- val &= (1 << PHY_MDM6600_NR_CMD_LINES) - 1;
- for (i = 0; i < PHY_MDM6600_NR_CMD_LINES; i++)
- values[i] = (val & BIT(i)) >> i;
+ values[0] = val;
gpiod_set_array_value_cansleep(PHY_MDM6600_NR_CMD_LINES,
- ddata->cmd_gpios->desc, values);
+ ddata->cmd_gpios->desc,
+ ddata->cmd_gpios->info, values);
}
/**
{
struct phy_mdm6600 *ddata;
struct device *dev;
- int values[PHY_MDM6600_NR_STATUS_LINES];
+ DECLARE_BITMAP(values, PHY_MDM6600_NR_STATUS_LINES);
int error, i, val = 0;
ddata = container_of(work, struct phy_mdm6600, status_work.work);
error = gpiod_get_array_value_cansleep(PHY_MDM6600_NR_STATUS_LINES,
ddata->status_gpios->desc,
+ ddata->status_gpios->info,
values);
if (error)
return;
for (i = 0; i < PHY_MDM6600_NR_STATUS_LINES; i++) {
- val |= values[i] << i;
+ val |= test_bit(i, values) << i;
dev_dbg(ddata->dev, "XXX %s: i: %i values[i]: %i val: %i\n",
- __func__, i, values[i], val);
+ __func__, i, test_bit(i, values), val);
}
- ddata->status = val;
+ ddata->status = values[0];
dev_info(dev, "modem status: %i %s\n",
ddata->status,
help
This selects pinctrl driver for Renesas RZ/A1 platforms.
+config PINCTRL_RZN1
+ bool "Renesas RZ/N1 pinctrl driver"
+ depends on OF
+ depends on ARCH_RZN1 || COMPILE_TEST
+ select GENERIC_PINCTRL_GROUPS
+ select GENERIC_PINMUX_FUNCTIONS
+ select GENERIC_PINCONF
+ help
+ This selects pinctrl driver for Renesas RZ/N1 devices.
+
config PINCTRL_SINGLE
tristate "One-register-per-pin type device tree based pinctrl driver"
depends on OF
config PINCTRL_INGENIC
bool "Pinctrl driver for the Ingenic JZ47xx SoCs"
- default y
+ default MACH_INGENIC
depends on OF
- depends on MACH_INGENIC || COMPILE_TEST
+ depends on MIPS || COMPILE_TEST
select GENERIC_PINCONF
select GENERIC_PINCTRL_GROUPS
select GENERIC_PINMUX_FUNCTIONS
+ select GPIOLIB
+ select GPIOLIB_IRQCHIP
select REGMAP_MMIO
config PINCTRL_RK805
source "drivers/pinctrl/intel/Kconfig"
source "drivers/pinctrl/mvebu/Kconfig"
source "drivers/pinctrl/nomadik/Kconfig"
+source "drivers/pinctrl/nuvoton/Kconfig"
source "drivers/pinctrl/pxa/Kconfig"
source "drivers/pinctrl/qcom/Kconfig"
source "drivers/pinctrl/samsung/Kconfig"
obj-$(CONFIG_PINCTRL_PISTACHIO) += pinctrl-pistachio.o
obj-$(CONFIG_PINCTRL_ROCKCHIP) += pinctrl-rockchip.o
obj-$(CONFIG_PINCTRL_RZA1) += pinctrl-rza1.o
+obj-$(CONFIG_PINCTRL_RZN1) += pinctrl-rzn1.o
obj-$(CONFIG_PINCTRL_SINGLE) += pinctrl-single.o
obj-$(CONFIG_PINCTRL_SIRF) += sirf/
obj-$(CONFIG_PINCTRL_SX150X) += pinctrl-sx150x.o
obj-$(CONFIG_X86) += intel/
obj-y += mvebu/
obj-y += nomadik/
+obj-$(CONFIG_ARCH_NPCM7XX) += nuvoton/
obj-$(CONFIG_PINCTRL_PXA) += pxa/
obj-$(CONFIG_ARCH_QCOM) += qcom/
obj-$(CONFIG_PINCTRL_SAMSUNG) += samsung/
pmap = find_pinconf_map(param, MAP_TYPE_ARG, arg);
- if (unlikely(WARN_ON(!pmap)))
+ if (WARN_ON(!pmap))
return -EINVAL;
val = pmap->val << pconf->bit;
configuration, with the exception that certain individual pins
can be overridden to GPIO function
+config PINCTRL_NS
+ bool "Broadcom Northstar pins driver"
+ depends on OF && (ARCH_BCM_5301X || COMPILE_TEST)
+ select PINMUX
+ select GENERIC_PINCONF
+ default ARCH_BCM_5301X
+ help
+ Say yes here to enable the Broadcom NS SoC pins driver.
+
+ The Broadcom Northstar pins driver supports muxing multi-purpose pins
+ that can be used for various functions (e.g. SPI, I2C, UART) as well
+ as GPIOs.
+
config PINCTRL_NSP_GPIO
bool "Broadcom NSP GPIO (with PINCONF) driver"
depends on OF_GPIO && (ARCH_BCM_NSP || COMPILE_TEST)
obj-$(CONFIG_PINCTRL_BCM2835) += pinctrl-bcm2835.o
obj-$(CONFIG_PINCTRL_IPROC_GPIO) += pinctrl-iproc-gpio.o
obj-$(CONFIG_PINCTRL_CYGNUS_MUX) += pinctrl-cygnus-mux.o
+obj-$(CONFIG_PINCTRL_NS) += pinctrl-ns.o
obj-$(CONFIG_PINCTRL_NSP_GPIO) += pinctrl-nsp-gpio.o
obj-$(CONFIG_PINCTRL_NS2_MUX) += pinctrl-ns2-mux.o
obj-$(CONFIG_PINCTRL_NSP_MUX) += pinctrl-nsp-mux.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 Rafał Miłecki <rafal@milecki.pl>
+ */
+
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/pinctrl/pinconf-generic.h>
+#include <linux/pinctrl/pinctrl.h>
+#include <linux/pinctrl/pinmux.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+
+#define FLAG_BCM4708 BIT(1)
+#define FLAG_BCM4709 BIT(2)
+#define FLAG_BCM53012 BIT(3)
+
+struct ns_pinctrl {
+ struct device *dev;
+ unsigned int chipset_flag;
+ struct pinctrl_dev *pctldev;
+ void __iomem *base;
+
+ struct pinctrl_desc pctldesc;
+ struct ns_pinctrl_group *groups;
+ unsigned int num_groups;
+ struct ns_pinctrl_function *functions;
+ unsigned int num_functions;
+};
+
+/*
+ * Pins
+ */
+
+static const struct pinctrl_pin_desc ns_pinctrl_pins[] = {
+ { 0, "spi_clk", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 1, "spi_ss", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 2, "spi_mosi", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 3, "spi_miso", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 4, "i2c_scl", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 5, "i2c_sda", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 6, "mdc", (void *)(FLAG_BCM4709 | FLAG_BCM53012) },
+ { 7, "mdio", (void *)(FLAG_BCM4709 | FLAG_BCM53012) },
+ { 8, "pwm0", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 9, "pwm1", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 10, "pwm2", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 11, "pwm3", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 12, "uart1_rx", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 13, "uart1_tx", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 14, "uart1_cts", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 15, "uart1_rts", (void *)(FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012) },
+ { 16, "uart2_rx", (void *)(FLAG_BCM4709 | FLAG_BCM53012) },
+ { 17, "uart2_tx", (void *)(FLAG_BCM4709 | FLAG_BCM53012) },
+/* TODO { ??, "xtal_out", (void *)(FLAG_BCM4709) }, */
+ { 22, "sdio_pwr", (void *)(FLAG_BCM4709 | FLAG_BCM53012) },
+ { 23, "sdio_en_1p8v", (void *)(FLAG_BCM4709 | FLAG_BCM53012) },
+};
+
+/*
+ * Groups
+ */
+
+struct ns_pinctrl_group {
+ const char *name;
+ const unsigned int *pins;
+ const unsigned int num_pins;
+ unsigned int chipsets;
+};
+
+static const unsigned int spi_pins[] = { 0, 1, 2, 3 };
+static const unsigned int i2c_pins[] = { 4, 5 };
+static const unsigned int mdio_pins[] = { 6, 7 };
+static const unsigned int pwm0_pins[] = { 8 };
+static const unsigned int pwm1_pins[] = { 9 };
+static const unsigned int pwm2_pins[] = { 10 };
+static const unsigned int pwm3_pins[] = { 11 };
+static const unsigned int uart1_pins[] = { 12, 13, 14, 15 };
+static const unsigned int uart2_pins[] = { 16, 17 };
+static const unsigned int sdio_pwr_pins[] = { 22 };
+static const unsigned int sdio_1p8v_pins[] = { 23 };
+
+#define NS_GROUP(_name, _pins, _chipsets) \
+{ \
+ .name = _name, \
+ .pins = _pins, \
+ .num_pins = ARRAY_SIZE(_pins), \
+ .chipsets = _chipsets, \
+}
+
+static const struct ns_pinctrl_group ns_pinctrl_groups[] = {
+ NS_GROUP("spi_grp", spi_pins, FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012),
+ NS_GROUP("i2c_grp", i2c_pins, FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012),
+ NS_GROUP("mdio_grp", mdio_pins, FLAG_BCM4709 | FLAG_BCM53012),
+ NS_GROUP("pwm0_grp", pwm0_pins, FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012),
+ NS_GROUP("pwm1_grp", pwm1_pins, FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012),
+ NS_GROUP("pwm2_grp", pwm2_pins, FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012),
+ NS_GROUP("pwm3_grp", pwm3_pins, FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012),
+ NS_GROUP("uart1_grp", uart1_pins, FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012),
+ NS_GROUP("uart2_grp", uart2_pins, FLAG_BCM4709 | FLAG_BCM53012),
+ NS_GROUP("sdio_pwr_grp", sdio_pwr_pins, FLAG_BCM4709 | FLAG_BCM53012),
+ NS_GROUP("sdio_1p8v_grp", sdio_1p8v_pins, FLAG_BCM4709 | FLAG_BCM53012),
+};
+
+/*
+ * Functions
+ */
+
+struct ns_pinctrl_function {
+ const char *name;
+ const char * const *groups;
+ const unsigned int num_groups;
+ unsigned int chipsets;
+};
+
+static const char * const spi_groups[] = { "spi_grp" };
+static const char * const i2c_groups[] = { "i2c_grp" };
+static const char * const mdio_groups[] = { "mdio_grp" };
+static const char * const pwm_groups[] = { "pwm0_grp", "pwm1_grp", "pwm2_grp",
+ "pwm3_grp" };
+static const char * const uart1_groups[] = { "uart1_grp" };
+static const char * const uart2_groups[] = { "uart2_grp" };
+static const char * const sdio_groups[] = { "sdio_pwr_grp", "sdio_1p8v_grp" };
+
+#define NS_FUNCTION(_name, _groups, _chipsets) \
+{ \
+ .name = _name, \
+ .groups = _groups, \
+ .num_groups = ARRAY_SIZE(_groups), \
+ .chipsets = _chipsets, \
+}
+
+static const struct ns_pinctrl_function ns_pinctrl_functions[] = {
+ NS_FUNCTION("spi", spi_groups, FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012),
+ NS_FUNCTION("i2c", i2c_groups, FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012),
+ NS_FUNCTION("mdio", mdio_groups, FLAG_BCM4709 | FLAG_BCM53012),
+ NS_FUNCTION("pwm", pwm_groups, FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012),
+ NS_FUNCTION("uart1", uart1_groups, FLAG_BCM4708 | FLAG_BCM4709 | FLAG_BCM53012),
+ NS_FUNCTION("uart2", uart2_groups, FLAG_BCM4709 | FLAG_BCM53012),
+ NS_FUNCTION("sdio", sdio_groups, FLAG_BCM4709 | FLAG_BCM53012),
+};
+
+/*
+ * Groups code
+ */
+
+static int ns_pinctrl_get_groups_count(struct pinctrl_dev *pctrl_dev)
+{
+ struct ns_pinctrl *ns_pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+
+ return ns_pinctrl->num_groups;
+}
+
+static const char *ns_pinctrl_get_group_name(struct pinctrl_dev *pctrl_dev,
+ unsigned int selector)
+{
+ struct ns_pinctrl *ns_pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+
+ return ns_pinctrl->groups[selector].name;
+}
+
+static int ns_pinctrl_get_group_pins(struct pinctrl_dev *pctrl_dev,
+ unsigned int selector,
+ const unsigned int **pins,
+ unsigned int *num_pins)
+{
+ struct ns_pinctrl *ns_pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+
+ *pins = ns_pinctrl->groups[selector].pins;
+ *num_pins = ns_pinctrl->groups[selector].num_pins;
+
+ return 0;
+}
+
+static const struct pinctrl_ops ns_pinctrl_ops = {
+ .get_groups_count = ns_pinctrl_get_groups_count,
+ .get_group_name = ns_pinctrl_get_group_name,
+ .get_group_pins = ns_pinctrl_get_group_pins,
+ .dt_node_to_map = pinconf_generic_dt_node_to_map_group,
+ .dt_free_map = pinconf_generic_dt_free_map,
+};
+
+/*
+ * Functions code
+ */
+
+static int ns_pinctrl_get_functions_count(struct pinctrl_dev *pctrl_dev)
+{
+ struct ns_pinctrl *ns_pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+
+ return ns_pinctrl->num_functions;
+}
+
+static const char *ns_pinctrl_get_function_name(struct pinctrl_dev *pctrl_dev,
+ unsigned int selector)
+{
+ struct ns_pinctrl *ns_pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+
+ return ns_pinctrl->functions[selector].name;
+}
+
+static int ns_pinctrl_get_function_groups(struct pinctrl_dev *pctrl_dev,
+ unsigned int selector,
+ const char * const **groups,
+ unsigned * const num_groups)
+{
+ struct ns_pinctrl *ns_pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+
+ *groups = ns_pinctrl->functions[selector].groups;
+ *num_groups = ns_pinctrl->functions[selector].num_groups;
+
+ return 0;
+}
+
+static int ns_pinctrl_set_mux(struct pinctrl_dev *pctrl_dev,
+ unsigned int func_select,
+ unsigned int grp_select)
+{
+ struct ns_pinctrl *ns_pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+ u32 unset = 0;
+ u32 tmp;
+ int i;
+
+ for (i = 0; i < ns_pinctrl->groups[grp_select].num_pins; i++) {
+ int pin_number = ns_pinctrl->groups[grp_select].pins[i];
+
+ unset |= BIT(pin_number);
+ }
+
+ tmp = readl(ns_pinctrl->base);
+ tmp &= ~unset;
+ writel(tmp, ns_pinctrl->base);
+
+ return 0;
+}
+
+static const struct pinmux_ops ns_pinctrl_pmxops = {
+ .get_functions_count = ns_pinctrl_get_functions_count,
+ .get_function_name = ns_pinctrl_get_function_name,
+ .get_function_groups = ns_pinctrl_get_function_groups,
+ .set_mux = ns_pinctrl_set_mux,
+};
+
+/*
+ * Controller code
+ */
+
+static struct pinctrl_desc ns_pinctrl_desc = {
+ .name = "pinctrl-ns",
+ .pctlops = &ns_pinctrl_ops,
+ .pmxops = &ns_pinctrl_pmxops,
+};
+
+static const struct of_device_id ns_pinctrl_of_match_table[] = {
+ { .compatible = "brcm,bcm4708-pinmux", .data = (void *)FLAG_BCM4708, },
+ { .compatible = "brcm,bcm4709-pinmux", .data = (void *)FLAG_BCM4709, },
+ { .compatible = "brcm,bcm53012-pinmux", .data = (void *)FLAG_BCM53012, },
+ { }
+};
+
+static int ns_pinctrl_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ const struct of_device_id *of_id;
+ struct ns_pinctrl *ns_pinctrl;
+ struct pinctrl_desc *pctldesc;
+ struct pinctrl_pin_desc *pin;
+ struct ns_pinctrl_group *group;
+ struct ns_pinctrl_function *function;
+ struct resource *res;
+ int i;
+
+ ns_pinctrl = devm_kzalloc(dev, sizeof(*ns_pinctrl), GFP_KERNEL);
+ if (!ns_pinctrl)
+ return -ENOMEM;
+ pctldesc = &ns_pinctrl->pctldesc;
+ platform_set_drvdata(pdev, ns_pinctrl);
+
+ /* Set basic properties */
+
+ ns_pinctrl->dev = dev;
+
+ of_id = of_match_device(ns_pinctrl_of_match_table, dev);
+ if (!of_id)
+ return -EINVAL;
+ ns_pinctrl->chipset_flag = (uintptr_t)of_id->data;
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
+ "cru_gpio_control");
+ ns_pinctrl->base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(ns_pinctrl->base)) {
+ dev_err(dev, "Failed to map pinctrl regs\n");
+ return PTR_ERR(ns_pinctrl->base);
+ }
+
+ memcpy(pctldesc, &ns_pinctrl_desc, sizeof(*pctldesc));
+
+ /* Set pinctrl properties */
+
+ pctldesc->pins = devm_kcalloc(dev, ARRAY_SIZE(ns_pinctrl_pins),
+ sizeof(struct pinctrl_pin_desc),
+ GFP_KERNEL);
+ if (!pctldesc->pins)
+ return -ENOMEM;
+ for (i = 0, pin = (struct pinctrl_pin_desc *)&pctldesc->pins[0];
+ i < ARRAY_SIZE(ns_pinctrl_pins); i++) {
+ const struct pinctrl_pin_desc *src = &ns_pinctrl_pins[i];
+ unsigned int chipsets = (uintptr_t)src->drv_data;
+
+ if (chipsets & ns_pinctrl->chipset_flag) {
+ memcpy(pin++, src, sizeof(*src));
+ pctldesc->npins++;
+ }
+ }
+
+ ns_pinctrl->groups = devm_kcalloc(dev, ARRAY_SIZE(ns_pinctrl_groups),
+ sizeof(struct ns_pinctrl_group),
+ GFP_KERNEL);
+ if (!ns_pinctrl->groups)
+ return -ENOMEM;
+ for (i = 0, group = &ns_pinctrl->groups[0];
+ i < ARRAY_SIZE(ns_pinctrl_groups); i++) {
+ const struct ns_pinctrl_group *src = &ns_pinctrl_groups[i];
+
+ if (src->chipsets & ns_pinctrl->chipset_flag) {
+ memcpy(group++, src, sizeof(*src));
+ ns_pinctrl->num_groups++;
+ }
+ }
+
+ ns_pinctrl->functions = devm_kcalloc(dev,
+ ARRAY_SIZE(ns_pinctrl_functions),
+ sizeof(struct ns_pinctrl_function),
+ GFP_KERNEL);
+ if (!ns_pinctrl->functions)
+ return -ENOMEM;
+ for (i = 0, function = &ns_pinctrl->functions[0];
+ i < ARRAY_SIZE(ns_pinctrl_functions); i++) {
+ const struct ns_pinctrl_function *src = &ns_pinctrl_functions[i];
+
+ if (src->chipsets & ns_pinctrl->chipset_flag) {
+ memcpy(function++, src, sizeof(*src));
+ ns_pinctrl->num_functions++;
+ }
+ }
+
+ /* Register */
+
+ ns_pinctrl->pctldev = devm_pinctrl_register(dev, pctldesc, ns_pinctrl);
+ if (IS_ERR(ns_pinctrl->pctldev)) {
+ dev_err(dev, "Failed to register pinctrl\n");
+ return PTR_ERR(ns_pinctrl->pctldev);
+ }
+
+ return 0;
+}
+
+static struct platform_driver ns_pinctrl_driver = {
+ .probe = ns_pinctrl_probe,
+ .driver = {
+ .name = "ns-pinmux",
+ .of_match_table = ns_pinctrl_of_match_table,
+ },
+};
+
+module_platform_driver(ns_pinctrl_driver);
+
+MODULE_AUTHOR("Rafał Miłecki");
+MODULE_LICENSE("GPL v2");
+MODULE_DEVICE_TABLE(of, ns_pinctrl_of_match_table);
ret = of_property_read_string(node, "function", &function_name);
if (ret) {
dev_err(pctrl->dev,
- "missing function property in node %s\n",
- node->name);
+ "missing function property in node %pOFn\n", node);
return -EINVAL;
}
ngroups = of_property_count_strings(node, "groups");
if (ngroups < 0) {
dev_err(pctrl->dev,
- "missing groups property in node %s\n",
- node->name);
+ "missing groups property in node %pOFn\n", node);
return -EINVAL;
}
seq_printf(s, " DRV=%umA", madera_pin_unmake_drv_str(priv, conf[1]));
if (conf[0] & MADERA_GP1_IP_CFG_MASK)
- seq_puts(s, "SCHMITT");
+ seq_puts(s, " SCHMITT");
}
unsigned int n_chip_groups = priv->chip->n_pin_groups;
const char *func_name = madera_mux_funcs[selector].name;
unsigned int reg;
- int i, ret;
+ int i, ret = 0;
dev_dbg(priv->dev, "%s selecting %u (%s) for group %u (%s)\n",
__func__, selector, func_name, group,
result = 1;
break;
default:
- break;
+ return -ENOTSUPP;
}
*config = pinconf_to_config_packed(param, result);
conf[1] &= ~MADERA_GP1_DIR;
break;
default:
- break;
+ return -ENOTSUPP;
}
++configs;
}
static const struct pinconf_ops madera_pin_conf_ops = {
+ .is_generic = true,
.pin_config_get = madera_pin_conf_get,
.pin_config_set = madera_pin_conf_set,
.pin_config_group_set = madera_pin_conf_group_set,
-
};
static struct pinctrl_desc madera_pin_desc = {
while (selector < ngroups) {
const char *gname = ops->get_group_name(pctldev, selector);
- if (!strcmp(function, gname))
+ if (gname && !strcmp(function, gname))
return selector;
selector++;
while (group_selector < ngroups) {
const char *gname = pctlops->get_group_name(pctldev,
group_selector);
- if (!strcmp(gname, pin_group)) {
+ if (gname && !strcmp(gname, pin_group)) {
dev_dbg(pctldev->dev,
"found group selector %u for %s\n",
group_selector,
*/
grp = imx_pinctrl_find_group_by_name(pctldev, np->name);
if (!grp) {
- dev_err(ipctl->dev, "unable to find group for node %s\n",
- np->name);
+ dev_err(ipctl->dev, "unable to find group for node %pOFn\n", np);
return -EINVAL;
}
int i;
u32 config;
- dev_dbg(ipctl->dev, "group(%d): %s\n", index, np->name);
+ dev_dbg(ipctl->dev, "group(%d): %pOFn\n", index, np);
if (info->flags & SHARE_MUX_CONF_REG)
pin_size = FSL_PIN_SHARE_SIZE;
struct group_desc *grp;
u32 i = 0;
- dev_dbg(pctl->dev, "parse function(%d): %s\n", index, np->name);
+ dev_dbg(pctl->dev, "parse function(%d): %pOFn\n", index, np);
func = pinmux_generic_get_function(pctl, index);
if (!func)
*/
grp = imx1_pinctrl_find_group_by_name(info, np->name);
if (!grp) {
- dev_err(info->dev, "unable to find group for node %s\n",
- np->name);
+ dev_err(info->dev, "unable to find group for node %pOFn\n",
+ np);
return -EINVAL;
}
const __be32 *list;
int i;
- dev_dbg(info->dev, "group(%d): %s\n", index, np->name);
+ dev_dbg(info->dev, "group(%d): %pOFn\n", index, np);
/* Initialise group */
grp->name = np->name;
list = of_get_property(np, "fsl,pins", &size);
/* we do not check return since it's safe node passed down */
if (!size || size % 12) {
- dev_notice(info->dev, "Not a valid fsl,pins property (%s)\n",
- np->name);
+ dev_notice(info->dev, "Not a valid fsl,pins property (%pOFn)\n",
+ np);
return -EINVAL;
}
static u32 grp_index;
u32 i = 0;
- dev_dbg(info->dev, "parse function(%d): %s\n", index, np->name);
+ dev_dbg(info->dev, "parse function(%d): %pOFn\n", index, np);
func = &info->functions[index];
iounmap(d->base);
return ret;
}
-EXPORT_SYMBOL_GPL(mxs_pinctrl_probe);
* Author: Mathias Nyman <mathias.nyman@linux.intel.com>
*/
-#include <linux/kernel.h>
-#include <linux/init.h>
-#include <linux/types.h>
+#include <linux/acpi.h>
#include <linux/bitops.h>
-#include <linux/interrupt.h>
-#include <linux/gpio.h>
#include <linux/gpio/driver.h>
-#include <linux/acpi.h>
-#include <linux/platform_device.h>
-#include <linux/seq_file.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
+#include <linux/property.h>
+#include <linux/seq_file.h>
+
#include <linux/pinctrl/pinctrl.h>
#include <linux/pinctrl/pinmux.h>
#include <linux/pinctrl/pinconf.h>
PINCTRL_PIN(27, "GPIO_NCORE27"),
};
-static unsigned const byt_ncore_pins_map[BYT_NGPIO_NCORE] = {
+static const unsigned int byt_ncore_pins_map[BYT_NGPIO_NCORE] = {
19, 18, 17, 20, 21, 22, 24, 25, 23, 16,
14, 15, 12, 26, 27, 1, 4, 8, 11, 0,
3, 6, 10, 13, 2, 5, 9, 7,
return 0;
}
-static u32 byt_get_gpio_mux(struct byt_gpio *vg, unsigned offset)
+static u32 byt_get_gpio_mux(struct byt_gpio *vg, unsigned int offset)
{
/* SCORE pin 92-93 */
if (!strcmp(vg->soc_data->uid, BYT_SCORE_ACPI_UID) &&
.owner = THIS_MODULE,
};
-static int byt_gpio_get(struct gpio_chip *chip, unsigned offset)
+static int byt_gpio_get(struct gpio_chip *chip, unsigned int offset)
{
struct byt_gpio *vg = gpiochip_get_data(chip);
void __iomem *reg = byt_gpio_reg(vg, offset, BYT_VAL_REG);
return !!(val & BYT_LEVEL);
}
-static void byt_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
+static void byt_gpio_set(struct gpio_chip *chip, unsigned int offset, int value)
{
struct byt_gpio *vg = gpiochip_get_data(chip);
void __iomem *reg = byt_gpio_reg(vg, offset, BYT_VAL_REG);
raw_spin_unlock_irqrestore(&vg->lock, flags);
if (!(value & BYT_OUTPUT_EN))
- return GPIOF_DIR_OUT;
+ return 0;
if (!(value & BYT_INPUT_EN))
- return GPIOF_DIR_IN;
+ return 1;
return -EINVAL;
}
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
struct byt_gpio *vg = gpiochip_get_data(gc);
- unsigned offset = irqd_to_hwirq(d);
+ unsigned int offset = irqd_to_hwirq(d);
void __iomem *reg;
reg = byt_gpio_reg(vg, offset, BYT_INT_STAT_REG);
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
struct byt_gpio *vg = gpiochip_get_data(gc);
- unsigned offset = irqd_to_hwirq(d);
+ unsigned int offset = irqd_to_hwirq(d);
unsigned long flags;
void __iomem *reg;
u32 value;
{ "INT33FC", (kernel_ulong_t)byt_soc_data },
{ }
};
-MODULE_DEVICE_TABLE(acpi, byt_gpio_acpi_match);
static int byt_pinctrl_probe(struct platform_device *pdev)
{
const struct byt_pinctrl_soc_data *soc_data = NULL;
const struct byt_pinctrl_soc_data **soc_table;
- const struct acpi_device_id *acpi_id;
struct acpi_device *acpi_dev;
struct byt_gpio *vg;
int i, ret;
if (!acpi_dev)
return -ENODEV;
- acpi_id = acpi_match_device(byt_gpio_acpi_match, &pdev->dev);
- if (!acpi_id)
- return -ENODEV;
-
- soc_table = (const struct byt_pinctrl_soc_data **)acpi_id->driver_data;
+ soc_table = (const struct byt_pinctrl_soc_data **)device_get_match_data(&pdev->dev);
for (i = 0; soc_table[i]; i++) {
if (!strcmp(acpi_dev->pnp.unique_id, soc_table[i]->uid)) {
* Author: Mika Westerberg <mika.westerberg@linux.intel.com>
*/
-#include <linux/acpi.h>
+#include <linux/mod_devicetable.h>
#include <linux/module.h>
#include <linux/platform_device.h>
-#include <linux/pm.h>
+
#include <linux/pinctrl/pinctrl.h>
#include "pinctrl-intel.h"
PINCTRL_PIN(82, "TDO"),
};
-static const unsigned bxt_north_pwm0_pins[] = { 34 };
-static const unsigned bxt_north_pwm1_pins[] = { 35 };
-static const unsigned bxt_north_pwm2_pins[] = { 36 };
-static const unsigned bxt_north_pwm3_pins[] = { 37 };
-static const unsigned bxt_north_uart0_pins[] = { 38, 39, 40, 41 };
-static const unsigned bxt_north_uart1_pins[] = { 42, 43, 44, 45 };
-static const unsigned bxt_north_uart2_pins[] = { 46, 47, 48, 49 };
-static const unsigned bxt_north_uart0b_pins[] = { 50, 51, 52, 53 };
-static const unsigned bxt_north_uart1b_pins[] = { 54, 55, 56, 57 };
-static const unsigned bxt_north_uart2b_pins[] = { 58, 59, 60, 61 };
-static const unsigned bxt_north_uart3_pins[] = { 58, 59, 60, 61 };
+static const unsigned int bxt_north_pwm0_pins[] = { 34 };
+static const unsigned int bxt_north_pwm1_pins[] = { 35 };
+static const unsigned int bxt_north_pwm2_pins[] = { 36 };
+static const unsigned int bxt_north_pwm3_pins[] = { 37 };
+static const unsigned int bxt_north_uart0_pins[] = { 38, 39, 40, 41 };
+static const unsigned int bxt_north_uart1_pins[] = { 42, 43, 44, 45 };
+static const unsigned int bxt_north_uart2_pins[] = { 46, 47, 48, 49 };
+static const unsigned int bxt_north_uart0b_pins[] = { 50, 51, 52, 53 };
+static const unsigned int bxt_north_uart1b_pins[] = { 54, 55, 56, 57 };
+static const unsigned int bxt_north_uart2b_pins[] = { 58, 59, 60, 61 };
+static const unsigned int bxt_north_uart3_pins[] = { 58, 59, 60, 61 };
static const struct intel_pingroup bxt_north_groups[] = {
PIN_GROUP("pwm0_grp", bxt_north_pwm0_pins, 1),
PINCTRL_PIN(71, "GP_SSP_2_TXD"),
};
-static const unsigned bxt_northwest_ssp0_pins[] = { 53, 54, 55, 56, 57, 58 };
-static const unsigned bxt_northwest_ssp1_pins[] = {
+static const unsigned int bxt_northwest_ssp0_pins[] = { 53, 54, 55, 56, 57, 58 };
+static const unsigned int bxt_northwest_ssp1_pins[] = {
59, 60, 61, 62, 63, 64, 65
};
-static const unsigned bxt_northwest_ssp2_pins[] = { 66, 67, 68, 69, 70, 71 };
-static const unsigned bxt_northwest_uart3_pins[] = { 67, 68, 69, 70 };
+static const unsigned int bxt_northwest_ssp2_pins[] = { 66, 67, 68, 69, 70, 71 };
+static const unsigned int bxt_northwest_uart3_pins[] = { 67, 68, 69, 70 };
static const struct intel_pingroup bxt_northwest_groups[] = {
PIN_GROUP("ssp0_grp", bxt_northwest_ssp0_pins, 1),
PINCTRL_PIN(41, "OSC_CLK_OUT_3"),
};
-static const unsigned bxt_west_i2c0_pins[] = { 0, 1 };
-static const unsigned bxt_west_i2c1_pins[] = { 2, 3 };
-static const unsigned bxt_west_i2c2_pins[] = { 4, 5 };
-static const unsigned bxt_west_i2c3_pins[] = { 6, 7 };
-static const unsigned bxt_west_i2c4_pins[] = { 8, 9 };
-static const unsigned bxt_west_i2c5_pins[] = { 10, 11 };
-static const unsigned bxt_west_i2c6_pins[] = { 12, 13 };
-static const unsigned bxt_west_i2c7_pins[] = { 14, 15 };
-static const unsigned bxt_west_i2c5b_pins[] = { 16, 17 };
-static const unsigned bxt_west_i2c6b_pins[] = { 18, 19 };
-static const unsigned bxt_west_i2c7b_pins[] = { 20, 21 };
+static const unsigned int bxt_west_i2c0_pins[] = { 0, 1 };
+static const unsigned int bxt_west_i2c1_pins[] = { 2, 3 };
+static const unsigned int bxt_west_i2c2_pins[] = { 4, 5 };
+static const unsigned int bxt_west_i2c3_pins[] = { 6, 7 };
+static const unsigned int bxt_west_i2c4_pins[] = { 8, 9 };
+static const unsigned int bxt_west_i2c5_pins[] = { 10, 11 };
+static const unsigned int bxt_west_i2c6_pins[] = { 12, 13 };
+static const unsigned int bxt_west_i2c7_pins[] = { 14, 15 };
+static const unsigned int bxt_west_i2c5b_pins[] = { 16, 17 };
+static const unsigned int bxt_west_i2c6b_pins[] = { 18, 19 };
+static const unsigned int bxt_west_i2c7b_pins[] = { 20, 21 };
static const struct intel_pingroup bxt_west_groups[] = {
PIN_GROUP("i2c0_grp", bxt_west_i2c0_pins, 1),
PINCTRL_PIN(30, "SDCARD_LVL_WP"),
};
-static const unsigned bxt_southwest_emmc0_pins[] = {
+static const unsigned int bxt_southwest_emmc0_pins[] = {
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 26,
};
-static const unsigned bxt_southwest_sdio_pins[] = {
+static const unsigned int bxt_southwest_sdio_pins[] = {
10, 11, 12, 13, 14, 15, 27,
};
-static const unsigned bxt_southwest_sdcard_pins[] = {
+static const unsigned int bxt_southwest_sdcard_pins[] = {
16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 28, 29, 30,
};
PINCTRL_PIN(77, "SVID0_CLK"),
};
-static const unsigned apl_north_pwm0_pins[] = { 34 };
-static const unsigned apl_north_pwm1_pins[] = { 35 };
-static const unsigned apl_north_pwm2_pins[] = { 36 };
-static const unsigned apl_north_pwm3_pins[] = { 37 };
-static const unsigned apl_north_uart0_pins[] = { 38, 39, 40, 41 };
-static const unsigned apl_north_uart1_pins[] = { 42, 43, 44, 45 };
-static const unsigned apl_north_uart2_pins[] = { 46, 47, 48, 49 };
+static const unsigned int apl_north_pwm0_pins[] = { 34 };
+static const unsigned int apl_north_pwm1_pins[] = { 35 };
+static const unsigned int apl_north_pwm2_pins[] = { 36 };
+static const unsigned int apl_north_pwm3_pins[] = { 37 };
+static const unsigned int apl_north_uart0_pins[] = { 38, 39, 40, 41 };
+static const unsigned int apl_north_uart1_pins[] = { 42, 43, 44, 45 };
+static const unsigned int apl_north_uart2_pins[] = { 46, 47, 48, 49 };
static const struct intel_pingroup apl_north_groups[] = {
PIN_GROUP("pwm0_grp", apl_north_pwm0_pins, 1),
PINCTRL_PIN(76, "GP_SSP_2_TXD"),
};
-static const unsigned apl_northwest_ssp0_pins[] = { 61, 62, 63, 64, 65 };
-static const unsigned apl_northwest_ssp1_pins[] = { 66, 67, 68, 69, 70 };
-static const unsigned apl_northwest_ssp2_pins[] = { 71, 72, 73, 74, 75, 76 };
-static const unsigned apl_northwest_uart3_pins[] = { 67, 68, 69, 70 };
+static const unsigned int apl_northwest_ssp0_pins[] = { 61, 62, 63, 64, 65 };
+static const unsigned int apl_northwest_ssp1_pins[] = { 66, 67, 68, 69, 70 };
+static const unsigned int apl_northwest_ssp2_pins[] = { 71, 72, 73, 74, 75, 76 };
+static const unsigned int apl_northwest_uart3_pins[] = { 67, 68, 69, 70 };
static const struct intel_pingroup apl_northwest_groups[] = {
PIN_GROUP("ssp0_grp", apl_northwest_ssp0_pins, 1),
PINCTRL_PIN(46, "SUSPWRDNACK"),
};
-static const unsigned apl_west_i2c0_pins[] = { 0, 1 };
-static const unsigned apl_west_i2c1_pins[] = { 2, 3 };
-static const unsigned apl_west_i2c2_pins[] = { 4, 5 };
-static const unsigned apl_west_i2c3_pins[] = { 6, 7 };
-static const unsigned apl_west_i2c4_pins[] = { 8, 9 };
-static const unsigned apl_west_i2c5_pins[] = { 10, 11 };
-static const unsigned apl_west_i2c6_pins[] = { 12, 13 };
-static const unsigned apl_west_i2c7_pins[] = { 14, 15 };
-static const unsigned apl_west_uart2_pins[] = { 20, 21, 22, 34 };
+static const unsigned int apl_west_i2c0_pins[] = { 0, 1 };
+static const unsigned int apl_west_i2c1_pins[] = { 2, 3 };
+static const unsigned int apl_west_i2c2_pins[] = { 4, 5 };
+static const unsigned int apl_west_i2c3_pins[] = { 6, 7 };
+static const unsigned int apl_west_i2c4_pins[] = { 8, 9 };
+static const unsigned int apl_west_i2c5_pins[] = { 10, 11 };
+static const unsigned int apl_west_i2c6_pins[] = { 12, 13 };
+static const unsigned int apl_west_i2c7_pins[] = { 14, 15 };
+static const unsigned int apl_west_uart2_pins[] = { 20, 21, 22, 34 };
static const struct intel_pingroup apl_west_groups[] = {
PIN_GROUP("i2c0_grp", apl_west_i2c0_pins, 1),
PINCTRL_PIN(42, "LPC_FRAMEB"),
};
-static const unsigned apl_southwest_emmc0_pins[] = {
+static const unsigned int apl_southwest_emmc0_pins[] = {
4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 29,
};
-static const unsigned apl_southwest_sdio_pins[] = {
+static const unsigned int apl_southwest_sdio_pins[] = {
14, 15, 16, 17, 18, 19, 30,
};
-static const unsigned apl_southwest_sdcard_pins[] = {
+static const unsigned int apl_southwest_sdcard_pins[] = {
20, 21, 22, 23, 24, 25, 26, 27, 28,
};
-static const unsigned apl_southwest_i2c7_pins[] = { 32, 33 };
+static const unsigned int apl_southwest_i2c7_pins[] = { 32, 33 };
static const struct intel_pingroup apl_southwest_groups[] = {
PIN_GROUP("emmc0_grp", apl_southwest_emmc0_pins, 1),
static int bxt_pinctrl_probe(struct platform_device *pdev)
{
- const struct intel_pinctrl_soc_data *soc_data = NULL;
- const struct intel_pinctrl_soc_data **soc_table;
- struct acpi_device *adev;
- int i;
-
- adev = ACPI_COMPANION(&pdev->dev);
- if (adev) {
- const struct acpi_device_id *id;
-
- id = acpi_match_device(bxt_pinctrl_acpi_match, &pdev->dev);
- if (!id)
- return -ENODEV;
-
- soc_table = (const struct intel_pinctrl_soc_data **)
- id->driver_data;
-
- for (i = 0; soc_table[i]; i++) {
- if (!strcmp(adev->pnp.unique_id, soc_table[i]->uid)) {
- soc_data = soc_table[i];
- break;
- }
- }
- } else {
- const struct platform_device_id *pid;
-
- pid = platform_get_device_id(pdev);
- if (!pid)
- return -ENODEV;
-
- soc_table = (const struct intel_pinctrl_soc_data **)
- pid->driver_data;
- soc_data = soc_table[pdev->id];
- }
-
- if (!soc_data)
- return -ENODEV;
-
- return intel_pinctrl_probe(pdev, soc_data);
+ return intel_pinctrl_probe_by_uid(pdev);
}
-static const struct dev_pm_ops bxt_pinctrl_pm_ops = {
- SET_LATE_SYSTEM_SLEEP_PM_OPS(intel_pinctrl_suspend,
- intel_pinctrl_resume)
-};
+static INTEL_PINCTRL_PM_OPS(bxt_pinctrl_pm_ops);
static struct platform_driver bxt_pinctrl_driver = {
.probe = bxt_pinctrl_probe,
* Mika Westerberg <mika.westerberg@linux.intel.com>
*/
-#include <linux/acpi.h>
+#include <linux/mod_devicetable.h>
#include <linux/module.h>
#include <linux/platform_device.h>
-#include <linux/pm.h>
+
#include <linux/pinctrl/pinctrl.h>
#include "pinctrl-intel.h"
static int cnl_pinctrl_probe(struct platform_device *pdev)
{
- const struct intel_pinctrl_soc_data *soc_data;
- const struct acpi_device_id *id;
-
- id = acpi_match_device(cnl_pinctrl_acpi_match, &pdev->dev);
- if (!id || !id->driver_data)
- return -ENODEV;
-
- soc_data = (const struct intel_pinctrl_soc_data *)id->driver_data;
- return intel_pinctrl_probe(pdev, soc_data);
+ return intel_pinctrl_probe_by_hid(pdev);
}
-static const struct dev_pm_ops cnl_pinctrl_pm_ops = {
- SET_LATE_SYSTEM_SLEEP_PM_OPS(intel_pinctrl_suspend,
- intel_pinctrl_resume)
-};
+static INTEL_PINCTRL_PM_OPS(cnl_pinctrl_pm_ops);
static struct platform_driver cnl_pinctrl_driver = {
.probe = cnl_pinctrl_probe,
#include <linux/acpi.h>
#include <linux/module.h>
#include <linux/platform_device.h>
-#include <linux/pm.h>
+
#include <linux/pinctrl/pinctrl.h>
#include "pinctrl-intel.h"
return intel_pinctrl_probe(pdev, &cdf_soc_data);
}
-static const struct dev_pm_ops cdf_pinctrl_pm_ops = {
- SET_LATE_SYSTEM_SLEEP_PM_OPS(intel_pinctrl_suspend,
- intel_pinctrl_resume)
-};
+static INTEL_PINCTRL_PM_OPS(cdf_pinctrl_pm_ops);
static const struct acpi_device_id cdf_pinctrl_acpi_match[] = {
{ "INTC3001" },
* Alan Cox <alan@linux.intel.com>
*/
+#include <linux/acpi.h>
#include <linux/dmi.h>
+#include <linux/gpio/driver.h>
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/init.h>
+#include <linux/platform_device.h>
#include <linux/types.h>
-#include <linux/gpio.h>
-#include <linux/gpio/driver.h>
-#include <linux/acpi.h>
+
#include <linux/pinctrl/pinctrl.h>
#include <linux/pinctrl/pinmux.h>
#include <linux/pinctrl/pinconf.h>
#include <linux/pinctrl/pinconf-generic.h>
-#include <linux/platform_device.h>
+
+#include "pinctrl-intel.h"
#define CHV_INTSTAT 0x300
#define CHV_INTMASK 0x380
* @invert_oe: Invert OE for this pin
*/
struct chv_alternate_function {
- unsigned pin;
+ unsigned int pin;
u8 mode;
bool invert_oe;
};
*/
struct chv_pingroup {
const char *name;
- const unsigned *pins;
+ const unsigned int *pins;
size_t npins;
struct chv_alternate_function altfunc;
const struct chv_alternate_function *overrides;
size_t noverrides;
};
-/**
- * struct chv_function - A CHV pinmux function
- * @name: Name of the function
- * @groups: An array of groups for this function
- * @ngroups: Number of groups in @groups
- */
-struct chv_function {
- const char *name;
- const char * const *groups;
- size_t ngroups;
-};
-
/**
* struct chv_gpio_pinrange - A range of pins that can be used as GPIOs
* @base: Start pin number
* @npins: Number of pins in this range
*/
struct chv_gpio_pinrange {
- unsigned base;
- unsigned npins;
+ unsigned int base;
+ unsigned int npins;
};
/**
* @gpio_ranges: An array of GPIO ranges in this community
* @ngpio_ranges: Number of GPIO ranges
* @nirqs: Total number of IRQs this community can generate
+ * @acpi_space_id: An address space ID for ACPI OpRegion handler
*/
struct chv_community {
const char *uid;
size_t npins;
const struct chv_pingroup *groups;
size_t ngroups;
- const struct chv_function *functions;
+ const struct intel_function *functions;
size_t nfunctions;
const struct chv_gpio_pinrange *gpio_ranges;
size_t ngpio_ranges;
* @intr_lines: Stores mapping between 16 HW interrupt wires and GPIO
* offset (in GPIO number space)
* @community: Community this pinctrl instance represents
+ * @saved_intmask: Interrupt mask saved for system sleep
+ * @saved_pin_context: Pointer to a context of the pins saved for system sleep
*
* The first group in @groups is expected to contain all pins that can be
* used as GPIOs.
.invert_oe = (i), \
}
-#define PIN_GROUP(n, p, m, i) \
+#define PIN_GROUP_WITH_ALT(n, p, m, i) \
{ \
.name = (n), \
.pins = (p), \
.noverrides = ARRAY_SIZE((o)), \
}
-#define FUNCTION(n, g) \
- { \
- .name = (n), \
- .groups = (g), \
- .ngroups = ARRAY_SIZE((g)), \
- }
-
#define GPIO_PINRANGE(start, end) \
{ \
.base = (start), \
PINCTRL_PIN(97, "GP_SSP_2_TXD"),
};
-static const unsigned southwest_fspi_pins[] = { 0, 1, 2, 3, 4, 5, 6, 7 };
static const unsigned southwest_uart0_pins[] = { 16, 20 };
static const unsigned southwest_uart1_pins[] = { 15, 16, 18, 20 };
static const unsigned southwest_uart2_pins[] = { 17, 19, 21, 22 };
static const unsigned southwest_i2c5_pins[] = { 45, 48 };
static const unsigned southwest_i2c6_pins[] = { 47, 51 };
static const unsigned southwest_i2c_nfc_pins[] = { 49, 52 };
-static const unsigned southwest_smbus_pins[] = { 79, 81, 82 };
static const unsigned southwest_spi3_pins[] = { 76, 79, 80, 81, 82 };
/* LPE I2S TXD pins need to have invert_oe set */
};
static const struct chv_pingroup southwest_groups[] = {
- PIN_GROUP("uart0_grp", southwest_uart0_pins, 2, false),
- PIN_GROUP("uart1_grp", southwest_uart1_pins, 1, false),
- PIN_GROUP("uart2_grp", southwest_uart2_pins, 1, false),
- PIN_GROUP("hda_grp", southwest_hda_pins, 2, false),
- PIN_GROUP("i2c0_grp", southwest_i2c0_pins, 1, true),
- PIN_GROUP("i2c1_grp", southwest_i2c1_pins, 1, true),
- PIN_GROUP("i2c2_grp", southwest_i2c2_pins, 1, true),
- PIN_GROUP("i2c3_grp", southwest_i2c3_pins, 1, true),
- PIN_GROUP("i2c4_grp", southwest_i2c4_pins, 1, true),
- PIN_GROUP("i2c5_grp", southwest_i2c5_pins, 1, true),
- PIN_GROUP("i2c6_grp", southwest_i2c6_pins, 1, true),
- PIN_GROUP("i2c_nfc_grp", southwest_i2c_nfc_pins, 2, true),
+ PIN_GROUP_WITH_ALT("uart0_grp", southwest_uart0_pins, 2, false),
+ PIN_GROUP_WITH_ALT("uart1_grp", southwest_uart1_pins, 1, false),
+ PIN_GROUP_WITH_ALT("uart2_grp", southwest_uart2_pins, 1, false),
+ PIN_GROUP_WITH_ALT("hda_grp", southwest_hda_pins, 2, false),
+ PIN_GROUP_WITH_ALT("i2c0_grp", southwest_i2c0_pins, 1, true),
+ PIN_GROUP_WITH_ALT("i2c1_grp", southwest_i2c1_pins, 1, true),
+ PIN_GROUP_WITH_ALT("i2c2_grp", southwest_i2c2_pins, 1, true),
+ PIN_GROUP_WITH_ALT("i2c3_grp", southwest_i2c3_pins, 1, true),
+ PIN_GROUP_WITH_ALT("i2c4_grp", southwest_i2c4_pins, 1, true),
+ PIN_GROUP_WITH_ALT("i2c5_grp", southwest_i2c5_pins, 1, true),
+ PIN_GROUP_WITH_ALT("i2c6_grp", southwest_i2c6_pins, 1, true),
+ PIN_GROUP_WITH_ALT("i2c_nfc_grp", southwest_i2c_nfc_pins, 2, true),
PIN_GROUP_WITH_OVERRIDE("lpe_grp", southwest_lpe_pins, 1, false,
southwest_lpe_altfuncs),
* Only do pinmuxing for certain LPSS devices for now. Rest of the pins are
* enabled only as GPIOs.
*/
-static const struct chv_function southwest_functions[] = {
+static const struct intel_function southwest_functions[] = {
FUNCTION("uart0", southwest_uart0_groups),
FUNCTION("uart1", southwest_uart1_groups),
FUNCTION("uart2", southwest_uart2_groups),
static const unsigned southeast_spi2_pins[] = { 2, 3, 4, 6, 7 };
static const struct chv_pingroup southeast_groups[] = {
- PIN_GROUP("pwm0_grp", southeast_pwm0_pins, 1, false),
- PIN_GROUP("pwm1_grp", southeast_pwm1_pins, 1, false),
- PIN_GROUP("sdmmc1_grp", southeast_sdmmc1_pins, 1, false),
- PIN_GROUP("sdmmc2_grp", southeast_sdmmc2_pins, 1, false),
- PIN_GROUP("sdmmc3_grp", southeast_sdmmc3_pins, 1, false),
- PIN_GROUP("spi1_grp", southeast_spi1_pins, 1, false),
- PIN_GROUP("spi2_grp", southeast_spi2_pins, 4, false),
+ PIN_GROUP_WITH_ALT("pwm0_grp", southeast_pwm0_pins, 1, false),
+ PIN_GROUP_WITH_ALT("pwm1_grp", southeast_pwm1_pins, 1, false),
+ PIN_GROUP_WITH_ALT("sdmmc1_grp", southeast_sdmmc1_pins, 1, false),
+ PIN_GROUP_WITH_ALT("sdmmc2_grp", southeast_sdmmc2_pins, 1, false),
+ PIN_GROUP_WITH_ALT("sdmmc3_grp", southeast_sdmmc3_pins, 1, false),
+ PIN_GROUP_WITH_ALT("spi1_grp", southeast_spi1_pins, 1, false),
+ PIN_GROUP_WITH_ALT("spi2_grp", southeast_spi2_pins, 4, false),
};
static const char * const southeast_pwm0_groups[] = { "pwm0_grp" };
static const char * const southeast_spi1_groups[] = { "spi1_grp" };
static const char * const southeast_spi2_groups[] = { "spi2_grp" };
-static const struct chv_function southeast_functions[] = {
+static const struct intel_function southeast_functions[] = {
FUNCTION("pwm0", southeast_pwm0_groups),
FUNCTION("pwm1", southeast_pwm1_groups),
FUNCTION("sdmmc1", southeast_sdmmc1_groups),
*/
static DEFINE_RAW_SPINLOCK(chv_lock);
-static void __iomem *chv_padreg(struct chv_pinctrl *pctrl, unsigned offset,
- unsigned reg)
+static void __iomem *chv_padreg(struct chv_pinctrl *pctrl, unsigned int offset,
+ unsigned int reg)
{
- unsigned family_no = offset / MAX_FAMILY_PAD_GPIO_NO;
- unsigned pad_no = offset % MAX_FAMILY_PAD_GPIO_NO;
+ unsigned int family_no = offset / MAX_FAMILY_PAD_GPIO_NO;
+ unsigned int pad_no = offset % MAX_FAMILY_PAD_GPIO_NO;
offset = FAMILY_PAD_REGS_OFF + FAMILY_PAD_REGS_SIZE * family_no +
GPIO_REGS_SIZE * pad_no;
}
/* When Pad Cfg is locked, driver can only change GPIOTXState or GPIORXState */
-static bool chv_pad_locked(struct chv_pinctrl *pctrl, unsigned offset)
+static bool chv_pad_locked(struct chv_pinctrl *pctrl, unsigned int offset)
{
void __iomem *reg;
}
static const char *chv_get_group_name(struct pinctrl_dev *pctldev,
- unsigned group)
+ unsigned int group)
{
struct chv_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
return pctrl->community->groups[group].name;
}
-static int chv_get_group_pins(struct pinctrl_dev *pctldev, unsigned group,
- const unsigned **pins, unsigned *npins)
+static int chv_get_group_pins(struct pinctrl_dev *pctldev, unsigned int group,
+ const unsigned int **pins, unsigned int *npins)
{
struct chv_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
}
static void chv_pin_dbg_show(struct pinctrl_dev *pctldev, struct seq_file *s,
- unsigned offset)
+ unsigned int offset)
{
struct chv_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
unsigned long flags;
}
static const char *chv_get_function_name(struct pinctrl_dev *pctldev,
- unsigned function)
+ unsigned int function)
{
struct chv_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
}
static int chv_get_function_groups(struct pinctrl_dev *pctldev,
- unsigned function,
+ unsigned int function,
const char * const **groups,
- unsigned * const ngroups)
+ unsigned int * const ngroups)
{
struct chv_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
return 0;
}
-static int chv_pinmux_set_mux(struct pinctrl_dev *pctldev, unsigned function,
- unsigned group)
+static int chv_pinmux_set_mux(struct pinctrl_dev *pctldev,
+ unsigned int function, unsigned int group)
{
struct chv_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
const struct chv_pingroup *grp;
static int chv_gpio_request_enable(struct pinctrl_dev *pctldev,
struct pinctrl_gpio_range *range,
- unsigned offset)
+ unsigned int offset)
{
struct chv_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
unsigned long flags;
static void chv_gpio_disable_free(struct pinctrl_dev *pctldev,
struct pinctrl_gpio_range *range,
- unsigned offset)
+ unsigned int offset)
{
struct chv_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
unsigned long flags;
static int chv_gpio_set_direction(struct pinctrl_dev *pctldev,
struct pinctrl_gpio_range *range,
- unsigned offset, bool input)
+ unsigned int offset, bool input)
{
struct chv_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
void __iomem *reg = chv_padreg(pctrl, offset, CHV_PADCTRL0);
.gpio_set_direction = chv_gpio_set_direction,
};
-static int chv_config_get(struct pinctrl_dev *pctldev, unsigned pin,
+static int chv_config_get(struct pinctrl_dev *pctldev, unsigned int pin,
unsigned long *config)
{
struct chv_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
return 0;
}
-static int chv_config_set_pull(struct chv_pinctrl *pctrl, unsigned pin,
+static int chv_config_set_pull(struct chv_pinctrl *pctrl, unsigned int pin,
enum pin_config_param param, u32 arg)
{
void __iomem *reg = chv_padreg(pctrl, pin, CHV_PADCTRL0);
return 0;
}
-static int chv_config_set(struct pinctrl_dev *pctldev, unsigned pin,
- unsigned long *configs, unsigned nconfigs)
+static int chv_config_set(struct pinctrl_dev *pctldev, unsigned int pin,
+ unsigned long *configs, unsigned int nconfigs)
{
struct chv_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
enum pin_config_param param;
.owner = THIS_MODULE,
};
-static int chv_gpio_get(struct gpio_chip *chip, unsigned offset)
+static int chv_gpio_get(struct gpio_chip *chip, unsigned int offset)
{
struct chv_pinctrl *pctrl = gpiochip_get_data(chip);
unsigned long flags;
return !!(ctrl0 & CHV_PADCTRL0_GPIORXSTATE);
}
-static void chv_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
+static void chv_gpio_set(struct gpio_chip *chip, unsigned int offset, int value)
{
struct chv_pinctrl *pctrl = gpiochip_get_data(chip);
unsigned long flags;
raw_spin_unlock_irqrestore(&chv_lock, flags);
}
-static int chv_gpio_get_direction(struct gpio_chip *chip, unsigned offset)
+static int chv_gpio_get_direction(struct gpio_chip *chip, unsigned int offset)
{
struct chv_pinctrl *pctrl = gpiochip_get_data(chip);
u32 ctrl0, direction;
return direction != CHV_PADCTRL0_GPIOCFG_GPO;
}
-static int chv_gpio_direction_input(struct gpio_chip *chip, unsigned offset)
+static int chv_gpio_direction_input(struct gpio_chip *chip, unsigned int offset)
{
return pinctrl_gpio_direction_input(chip->base + offset);
}
-static int chv_gpio_direction_output(struct gpio_chip *chip, unsigned offset,
+static int chv_gpio_direction_output(struct gpio_chip *chip, unsigned int offset,
int value)
{
chv_gpio_set(chip, offset, value);
if (irqd_get_trigger_type(d) == IRQ_TYPE_NONE) {
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
struct chv_pinctrl *pctrl = gpiochip_get_data(gc);
- unsigned pin = irqd_to_hwirq(d);
+ unsigned int pin = irqd_to_hwirq(d);
irq_flow_handler_t handler;
unsigned long flags;
u32 intsel, value;
return 0;
}
-static int chv_gpio_irq_type(struct irq_data *d, unsigned type)
+static int chv_gpio_irq_type(struct irq_data *d, unsigned int type)
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
struct chv_pinctrl *pctrl = gpiochip_get_data(gc);
- unsigned pin = irqd_to_hwirq(d);
+ unsigned int pin = irqd_to_hwirq(d);
unsigned long flags;
u32 value;
#include <linux/acpi.h>
#include <linux/module.h>
#include <linux/platform_device.h>
-#include <linux/pm.h>
+
#include <linux/pinctrl/pinctrl.h>
#include "pinctrl-intel.h"
return intel_pinctrl_probe(pdev, &dnv_soc_data);
}
-static const struct dev_pm_ops dnv_pinctrl_pm_ops = {
- SET_LATE_SYSTEM_SLEEP_PM_OPS(intel_pinctrl_suspend,
- intel_pinctrl_resume)
-};
+static INTEL_PINCTRL_PM_OPS(dnv_pinctrl_pm_ops);
static const struct acpi_device_id dnv_pinctrl_acpi_match[] = {
{ "INTC3000" },
* Author: Mika Westerberg <mika.westerberg@linux.intel.com>
*/
-#include <linux/acpi.h>
+#include <linux/mod_devicetable.h>
#include <linux/module.h>
#include <linux/platform_device.h>
-#include <linux/pm.h>
+
#include <linux/pinctrl/pinctrl.h>
#include "pinctrl-intel.h"
#define GLK_PAD_OWN 0x020
-#define GLK_HOSTSW_OWN 0x0b0
#define GLK_PADCFGLOCK 0x080
+#define GLK_HOSTSW_OWN 0x0b0
#define GLK_GPI_IE 0x110
#define GLK_COMMUNITY(s, e) \
PINCTRL_PIN(23, "GPIO_23"),
PINCTRL_PIN(24, "GPIO_24"),
PINCTRL_PIN(25, "GPIO_25"),
- PINCTRL_PIN(26, "GPIO_26"),
- PINCTRL_PIN(27, "GPIO_27"),
- PINCTRL_PIN(28, "GPIO_28"),
- PINCTRL_PIN(29, "GPIO_29"),
- PINCTRL_PIN(30, "GPIO_30"),
- PINCTRL_PIN(31, "GPIO_31"),
- PINCTRL_PIN(32, "GPIO_32"),
- PINCTRL_PIN(33, "GPIO_33"),
- PINCTRL_PIN(34, "GPIO_34"),
- PINCTRL_PIN(35, "GPIO_35"),
+ PINCTRL_PIN(26, "ISH_GPIO_0"),
+ PINCTRL_PIN(27, "ISH_GPIO_1"),
+ PINCTRL_PIN(28, "ISH_GPIO_2"),
+ PINCTRL_PIN(29, "ISH_GPIO_3"),
+ PINCTRL_PIN(30, "ISH_GPIO_4"),
+ PINCTRL_PIN(31, "ISH_GPIO_5"),
+ PINCTRL_PIN(32, "ISH_GPIO_6"),
+ PINCTRL_PIN(33, "ISH_GPIO_7"),
+ PINCTRL_PIN(34, "ISH_GPIO_8"),
+ PINCTRL_PIN(35, "ISH_GPIO_9"),
PINCTRL_PIN(36, "GPIO_36"),
PINCTRL_PIN(37, "GPIO_37"),
PINCTRL_PIN(38, "GPIO_38"),
PINCTRL_PIN(5, "LPSS_SPI_0_FS1"),
PINCTRL_PIN(6, "LPSS_SPI_0_RXD"),
PINCTRL_PIN(7, "LPSS_SPI_0_TXD"),
- PINCTRL_PIN(8, "LPSS_SPI_1_CLK"),
- PINCTRL_PIN(9, "LPSS_SPI_1_FS0"),
- PINCTRL_PIN(10, "LPSS_SPI_1_FS1"),
- PINCTRL_PIN(11, "LPSS_SPI_1_FS2"),
- PINCTRL_PIN(12, "LPSS_SPI_1_RXD"),
- PINCTRL_PIN(13, "LPSS_SPI_1_TXD"),
+ PINCTRL_PIN(8, "LPSS_SPI_2_CLK"),
+ PINCTRL_PIN(9, "LPSS_SPI_2_FS0"),
+ PINCTRL_PIN(10, "LPSS_SPI_2_FS1"),
+ PINCTRL_PIN(11, "LPSS_SPI_2_FS2"),
+ PINCTRL_PIN(12, "LPSS_SPI_2_RXD"),
+ PINCTRL_PIN(13, "LPSS_SPI_2_TXD"),
PINCTRL_PIN(14, "FST_SPI_CS0_B"),
PINCTRL_PIN(15, "FST_SPI_CS1_B"),
PINCTRL_PIN(16, "FST_SPI_MOSI_IO0"),
PINCTRL_PIN(25, "PMU_SLP_S3_B"),
PINCTRL_PIN(26, "PMU_SLP_S4_B"),
PINCTRL_PIN(27, "SUSPWRDNACK"),
- PINCTRL_PIN(28, "EMMC_PWR_EN_B"),
- PINCTRL_PIN(29, "PMU_AC_PRESENT"),
+ PINCTRL_PIN(28, "EMMC_DNX_PWR_EN_B"),
+ PINCTRL_PIN(29, "GPIO_105"),
PINCTRL_PIN(30, "PMU_BATLOW_B"),
PINCTRL_PIN(31, "PMU_RESETBUTTON_B"),
PINCTRL_PIN(32, "PMU_SUSCLK"),
};
static const struct acpi_device_id glk_pinctrl_acpi_match[] = {
- { "INT3453" },
+ { "INT3453", (kernel_ulong_t)glk_pinctrl_soc_data },
{ }
};
MODULE_DEVICE_TABLE(acpi, glk_pinctrl_acpi_match);
-static int glk_pinctrl_probe(struct platform_device *pdev)
-{
- const struct intel_pinctrl_soc_data *soc_data = NULL;
- struct acpi_device *adev;
- int i;
-
- adev = ACPI_COMPANION(&pdev->dev);
- if (!adev)
- return -ENODEV;
-
- for (i = 0; glk_pinctrl_soc_data[i]; i++) {
- if (!strcmp(adev->pnp.unique_id,
- glk_pinctrl_soc_data[i]->uid)) {
- soc_data = glk_pinctrl_soc_data[i];
- break;
- }
- }
-
- if (!soc_data)
- return -ENODEV;
-
- return intel_pinctrl_probe(pdev, soc_data);
-}
-
-static const struct dev_pm_ops glk_pinctrl_pm_ops = {
- SET_LATE_SYSTEM_SLEEP_PM_OPS(intel_pinctrl_suspend,
- intel_pinctrl_resume)
-};
+static INTEL_PINCTRL_PM_OPS(glk_pinctrl_pm_ops);
static struct platform_driver glk_pinctrl_driver = {
- .probe = glk_pinctrl_probe,
+ .probe = intel_pinctrl_probe_by_uid,
.driver = {
.name = "geminilake-pinctrl",
.acpi_match_table = glk_pinctrl_acpi_match,
#include <linux/acpi.h>
#include <linux/module.h>
#include <linux/platform_device.h>
-#include <linux/pm.h>
+
#include <linux/pinctrl/pinctrl.h>
#include "pinctrl-intel.h"
return intel_pinctrl_probe(pdev, &icllp_soc_data);
}
-static const struct dev_pm_ops icl_pinctrl_pm_ops = {
- SET_LATE_SYSTEM_SLEEP_PM_OPS(intel_pinctrl_suspend,
- intel_pinctrl_resume)
-};
+static INTEL_PINCTRL_PM_OPS(icl_pinctrl_pm_ops);
static const struct acpi_device_id icl_pinctrl_acpi_match[] = {
{ "INT3455" },
* Mika Westerberg <mika.westerberg@linux.intel.com>
*/
+#include <linux/acpi.h>
#include <linux/module.h>
#include <linux/interrupt.h>
#include <linux/gpio/driver.h>
#include <linux/log2.h>
#include <linux/platform_device.h>
+#include <linux/property.h>
+
#include <linux/pinctrl/pinctrl.h>
#include <linux/pinctrl/pinmux.h>
#include <linux/pinctrl/pinconf.h>
#define padgroup_offset(g, p) ((p) - (g)->base)
static struct intel_community *intel_get_community(struct intel_pinctrl *pctrl,
- unsigned pin)
+ unsigned int pin)
{
struct intel_community *community;
int i;
static const struct intel_padgroup *
intel_community_get_padgroup(const struct intel_community *community,
- unsigned pin)
+ unsigned int pin)
{
int i;
return NULL;
}
-static void __iomem *intel_get_padcfg(struct intel_pinctrl *pctrl, unsigned pin,
- unsigned reg)
+static void __iomem *intel_get_padcfg(struct intel_pinctrl *pctrl,
+ unsigned int pin, unsigned int reg)
{
const struct intel_community *community;
- unsigned padno;
+ unsigned int padno;
size_t nregs;
community = intel_get_community(pctrl, pin);
return community->pad_regs + reg + padno * nregs * 4;
}
-static bool intel_pad_owned_by_host(struct intel_pinctrl *pctrl, unsigned pin)
+static bool intel_pad_owned_by_host(struct intel_pinctrl *pctrl, unsigned int pin)
{
const struct intel_community *community;
const struct intel_padgroup *padgrp;
- unsigned gpp, offset, gpp_offset;
+ unsigned int gpp, offset, gpp_offset;
void __iomem *padown;
community = intel_get_community(pctrl, pin);
return !(readl(padown) & PADOWN_MASK(gpp_offset));
}
-static bool intel_pad_acpi_mode(struct intel_pinctrl *pctrl, unsigned pin)
+static bool intel_pad_acpi_mode(struct intel_pinctrl *pctrl, unsigned int pin)
{
const struct intel_community *community;
const struct intel_padgroup *padgrp;
- unsigned offset, gpp_offset;
+ unsigned int offset, gpp_offset;
void __iomem *hostown;
community = intel_get_community(pctrl, pin);
return !(readl(hostown) & BIT(gpp_offset));
}
-static bool intel_pad_locked(struct intel_pinctrl *pctrl, unsigned pin)
+static bool intel_pad_locked(struct intel_pinctrl *pctrl, unsigned int pin)
{
struct intel_community *community;
const struct intel_padgroup *padgrp;
- unsigned offset, gpp_offset;
+ unsigned int offset, gpp_offset;
u32 value;
community = intel_get_community(pctrl, pin);
return false;
}
-static bool intel_pad_usable(struct intel_pinctrl *pctrl, unsigned pin)
+static bool intel_pad_usable(struct intel_pinctrl *pctrl, unsigned int pin)
{
return intel_pad_owned_by_host(pctrl, pin) &&
!intel_pad_locked(pctrl, pin);
}
static const char *intel_get_group_name(struct pinctrl_dev *pctldev,
- unsigned group)
+ unsigned int group)
{
struct intel_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
return pctrl->soc->groups[group].name;
}
-static int intel_get_group_pins(struct pinctrl_dev *pctldev, unsigned group,
- const unsigned **pins, unsigned *npins)
+static int intel_get_group_pins(struct pinctrl_dev *pctldev, unsigned int group,
+ const unsigned int **pins, unsigned int *npins)
{
struct intel_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
}
static void intel_pin_dbg_show(struct pinctrl_dev *pctldev, struct seq_file *s,
- unsigned pin)
+ unsigned int pin)
{
struct intel_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
void __iomem *padcfg;
}
static const char *intel_get_function_name(struct pinctrl_dev *pctldev,
- unsigned function)
+ unsigned int function)
{
struct intel_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
}
static int intel_get_function_groups(struct pinctrl_dev *pctldev,
- unsigned function,
+ unsigned int function,
const char * const **groups,
- unsigned * const ngroups)
+ unsigned int * const ngroups)
{
struct intel_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
return 0;
}
-static int intel_pinmux_set_mux(struct pinctrl_dev *pctldev, unsigned function,
- unsigned group)
+static int intel_pinmux_set_mux(struct pinctrl_dev *pctldev,
+ unsigned int function, unsigned int group)
{
struct intel_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
const struct intel_pingroup *grp = &pctrl->soc->groups[group];
static int intel_gpio_request_enable(struct pinctrl_dev *pctldev,
struct pinctrl_gpio_range *range,
- unsigned pin)
+ unsigned int pin)
{
struct intel_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
void __iomem *padcfg0;
static int intel_gpio_set_direction(struct pinctrl_dev *pctldev,
struct pinctrl_gpio_range *range,
- unsigned pin, bool input)
+ unsigned int pin, bool input)
{
struct intel_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
void __iomem *padcfg0;
.gpio_set_direction = intel_gpio_set_direction,
};
-static int intel_config_get(struct pinctrl_dev *pctldev, unsigned pin,
+static int intel_config_get(struct pinctrl_dev *pctldev, unsigned int pin,
unsigned long *config)
{
struct intel_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
return 0;
}
-static int intel_config_set_pull(struct intel_pinctrl *pctrl, unsigned pin,
+static int intel_config_set_pull(struct intel_pinctrl *pctrl, unsigned int pin,
unsigned long config)
{
- unsigned param = pinconf_to_config_param(config);
- unsigned arg = pinconf_to_config_argument(config);
+ unsigned int param = pinconf_to_config_param(config);
+ unsigned int arg = pinconf_to_config_argument(config);
const struct intel_community *community;
void __iomem *padcfg1;
unsigned long flags;
return ret;
}
-static int intel_config_set_debounce(struct intel_pinctrl *pctrl, unsigned pin,
- unsigned debounce)
+static int intel_config_set_debounce(struct intel_pinctrl *pctrl,
+ unsigned int pin, unsigned int debounce)
{
void __iomem *padcfg0, *padcfg2;
unsigned long flags;
return ret;
}
-static int intel_config_set(struct pinctrl_dev *pctldev, unsigned pin,
- unsigned long *configs, unsigned nconfigs)
+static int intel_config_set(struct pinctrl_dev *pctldev, unsigned int pin,
+ unsigned long *configs, unsigned int nconfigs)
{
struct intel_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
int i, ret;
* intel_gpio_to_pin() - Translate from GPIO offset to pin number
* @pctrl: Pinctrl structure
* @offset: GPIO offset from gpiolib
- * @commmunity: Community is filled here if not %NULL
+ * @community: Community is filled here if not %NULL
* @padgrp: Pad group is filled here if not %NULL
*
* When coming through gpiolib irqchip, the GPIO offset is not
* automatically translated to pinctrl pin number. This function can be
* used to find out the corresponding pinctrl pin.
*/
-static int intel_gpio_to_pin(struct intel_pinctrl *pctrl, unsigned offset,
+static int intel_gpio_to_pin(struct intel_pinctrl *pctrl, unsigned int offset,
const struct intel_community **community,
const struct intel_padgroup **padgrp)
{
return -EINVAL;
}
-static int intel_gpio_get(struct gpio_chip *chip, unsigned offset)
+static int intel_gpio_get(struct gpio_chip *chip, unsigned int offset)
{
struct intel_pinctrl *pctrl = gpiochip_get_data(chip);
void __iomem *reg;
return !!(padcfg0 & PADCFG0_GPIORXSTATE);
}
-static void intel_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
+static void intel_gpio_set(struct gpio_chip *chip, unsigned int offset,
+ int value)
{
struct intel_pinctrl *pctrl = gpiochip_get_data(chip);
unsigned long flags;
return !!(padcfg0 & PADCFG0_GPIOTXDIS);
}
-static int intel_gpio_direction_input(struct gpio_chip *chip, unsigned offset)
+static int intel_gpio_direction_input(struct gpio_chip *chip, unsigned int offset)
{
return pinctrl_gpio_direction_input(chip->base + offset);
}
-static int intel_gpio_direction_output(struct gpio_chip *chip, unsigned offset,
+static int intel_gpio_direction_output(struct gpio_chip *chip, unsigned int offset,
int value)
{
intel_gpio_set(chip, offset, value);
pin = intel_gpio_to_pin(pctrl, irqd_to_hwirq(d), &community, &padgrp);
if (pin >= 0) {
- unsigned gpp, gpp_offset, is_offset;
+ unsigned int gpp, gpp_offset, is_offset;
gpp = padgrp->reg_num;
gpp_offset = padgroup_offset(padgrp, pin);
pin = intel_gpio_to_pin(pctrl, irqd_to_hwirq(d), &community, &padgrp);
if (pin >= 0) {
- unsigned gpp, gpp_offset, is_offset;
+ unsigned int gpp, gpp_offset, is_offset;
unsigned long flags;
u32 value;
pin = intel_gpio_to_pin(pctrl, irqd_to_hwirq(d), &community, &padgrp);
if (pin >= 0) {
- unsigned gpp, gpp_offset;
+ unsigned int gpp, gpp_offset;
unsigned long flags;
void __iomem *reg;
u32 value;
intel_gpio_irq_mask_unmask(d, false);
}
-static int intel_gpio_irq_type(struct irq_data *d, unsigned type)
+static int intel_gpio_irq_type(struct irq_data *d, unsigned int type)
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
struct intel_pinctrl *pctrl = gpiochip_get_data(gc);
- unsigned pin = intel_gpio_to_pin(pctrl, irqd_to_hwirq(d), NULL, NULL);
+ unsigned int pin = intel_gpio_to_pin(pctrl, irqd_to_hwirq(d), NULL, NULL);
unsigned long flags;
void __iomem *reg;
u32 value;
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
struct intel_pinctrl *pctrl = gpiochip_get_data(gc);
- unsigned pin = intel_gpio_to_pin(pctrl, irqd_to_hwirq(d), NULL, NULL);
+ unsigned int pin = intel_gpio_to_pin(pctrl, irqd_to_hwirq(d), NULL, NULL);
if (on)
enable_irq_wake(pctrl->irq);
static unsigned intel_gpio_ngpio(const struct intel_pinctrl *pctrl)
{
const struct intel_community *community;
- unsigned ngpio = 0;
+ unsigned int ngpio = 0;
int i, j;
for (i = 0; i < pctrl->ncommunities; i++) {
struct intel_community *community)
{
struct intel_padgroup *gpps;
- unsigned npins = community->npins;
- unsigned padown_num = 0;
+ unsigned int npins = community->npins;
+ unsigned int padown_num = 0;
size_t ngpps, i;
if (community->gpps)
if (community->gpps) {
gpps[i] = community->gpps[i];
} else {
- unsigned gpp_size = community->gpp_size;
+ unsigned int gpp_size = community->gpp_size;
gpps[i].reg_num = i;
gpps[i].base = community->pin_base + i * gpp_size;
}
EXPORT_SYMBOL_GPL(intel_pinctrl_probe);
+int intel_pinctrl_probe_by_hid(struct platform_device *pdev)
+{
+ const struct intel_pinctrl_soc_data *data;
+
+ data = device_get_match_data(&pdev->dev);
+ return intel_pinctrl_probe(pdev, data);
+}
+EXPORT_SYMBOL_GPL(intel_pinctrl_probe_by_hid);
+
+int intel_pinctrl_probe_by_uid(struct platform_device *pdev)
+{
+ const struct intel_pinctrl_soc_data *data = NULL;
+ const struct intel_pinctrl_soc_data **table;
+ struct acpi_device *adev;
+ unsigned int i;
+
+ adev = ACPI_COMPANION(&pdev->dev);
+ if (adev) {
+ const void *match = device_get_match_data(&pdev->dev);
+
+ table = (const struct intel_pinctrl_soc_data **)match;
+ for (i = 0; table[i]; i++) {
+ if (!strcmp(adev->pnp.unique_id, table[i]->uid)) {
+ data = table[i];
+ break;
+ }
+ }
+ } else {
+ const struct platform_device_id *id;
+
+ id = platform_get_device_id(pdev);
+ if (!id)
+ return -ENODEV;
+
+ table = (const struct intel_pinctrl_soc_data **)id->driver_data;
+ data = table[pdev->id];
+ }
+ if (!data)
+ return -ENODEV;
+
+ return intel_pinctrl_probe(pdev, data);
+}
+EXPORT_SYMBOL_GPL(intel_pinctrl_probe_by_uid);
+
#ifdef CONFIG_PM_SLEEP
-static bool intel_pinctrl_should_save(struct intel_pinctrl *pctrl, unsigned pin)
+static bool intel_pinctrl_should_save(struct intel_pinctrl *pctrl, unsigned int pin)
{
const struct pin_desc *pd = pin_desc_get(pctrl->pctldev, pin);
for (i = 0; i < pctrl->ncommunities; i++) {
struct intel_community *community = &pctrl->communities[i];
void __iomem *base;
- unsigned gpp;
+ unsigned int gpp;
base = community->regs + community->ie_offset;
for (gpp = 0; gpp < community->ngpps; gpp++)
for (i = 0; i < pctrl->ncommunities; i++) {
const struct intel_community *community;
void __iomem *base;
- unsigned gpp;
+ unsigned int gpp;
community = &pctrl->communities[i];
base = community->regs;
for (i = 0; i < pctrl->ncommunities; i++) {
struct intel_community *community = &pctrl->communities[i];
void __iomem *base;
- unsigned gpp;
+ unsigned int gpp;
base = community->regs + community->ie_offset;
for (gpp = 0; gpp < community->ngpps; gpp++) {
#ifndef PINCTRL_INTEL_H
#define PINCTRL_INTEL_H
+#include <linux/pm.h>
+
struct pinctrl_pin_desc;
struct platform_device;
struct device;
*/
struct intel_pingroup {
const char *name;
- const unsigned *pins;
+ const unsigned int *pins;
size_t npins;
unsigned short mode;
- const unsigned *modes;
+ const unsigned int *modes;
};
/**
* to specify them.
*/
struct intel_padgroup {
- unsigned reg_num;
- unsigned base;
- unsigned size;
+ unsigned int reg_num;
+ unsigned int base;
+ unsigned int size;
int gpio_base;
- unsigned padown_num;
+ unsigned int padown_num;
};
/**
* pass custom @gpps and @ngpps instead.
*/
struct intel_community {
- unsigned barno;
- unsigned padown_offset;
- unsigned padcfglock_offset;
- unsigned hostown_offset;
- unsigned is_offset;
- unsigned ie_offset;
- unsigned pin_base;
- unsigned gpp_size;
- unsigned gpp_num_padown_regs;
+ unsigned int barno;
+ unsigned int padown_offset;
+ unsigned int padcfglock_offset;
+ unsigned int hostown_offset;
+ unsigned int is_offset;
+ unsigned int ie_offset;
+ unsigned int pin_base;
+ unsigned int gpp_size;
+ unsigned int gpp_num_padown_regs;
size_t npins;
- unsigned features;
+ unsigned int features;
const struct intel_padgroup *gpps;
size_t ngpps;
/* Reserved for the core driver */
int intel_pinctrl_probe(struct platform_device *pdev,
const struct intel_pinctrl_soc_data *soc_data);
+int intel_pinctrl_probe_by_hid(struct platform_device *pdev);
+int intel_pinctrl_probe_by_uid(struct platform_device *pdev);
+
#ifdef CONFIG_PM_SLEEP
int intel_pinctrl_suspend(struct device *dev);
int intel_pinctrl_resume(struct device *dev);
#endif
+#define INTEL_PINCTRL_PM_OPS(_name) \
+const struct dev_pm_ops _name = { \
+ SET_LATE_SYSTEM_SLEEP_PM_OPS(intel_pinctrl_suspend, intel_pinctrl_resume) \
+}
+
#endif /* PINCTRL_INTEL_H */
#include <linux/acpi.h>
#include <linux/module.h>
#include <linux/platform_device.h>
-#include <linux/pm.h>
+
#include <linux/pinctrl/pinctrl.h>
#include "pinctrl-intel.h"
return intel_pinctrl_probe(pdev, &lbg_soc_data);
}
-static const struct dev_pm_ops lbg_pinctrl_pm_ops = {
- SET_LATE_SYSTEM_SLEEP_PM_OPS(intel_pinctrl_suspend,
- intel_pinctrl_resume)
-};
+static INTEL_PINCTRL_PM_OPS(lbg_pinctrl_pm_ops);
static const struct acpi_device_id lbg_pinctrl_acpi_match[] = {
{ "INT3536" },
return family->regs + BUFCFG_OFFSET + bufno * 4;
}
+static int mrfld_read_bufcfg(struct mrfld_pinctrl *mp, unsigned int pin, u32 *value)
+{
+ void __iomem *bufcfg;
+
+ if (!mrfld_buf_available(mp, pin))
+ return -EBUSY;
+
+ bufcfg = mrfld_get_bufcfg(mp, pin);
+ *value = readl(bufcfg);
+
+ return 0;
+}
+
+static void mrfld_update_bufcfg(struct mrfld_pinctrl *mp, unsigned int pin,
+ u32 bits, u32 mask)
+{
+ void __iomem *bufcfg;
+ u32 value;
+
+ bufcfg = mrfld_get_bufcfg(mp, pin);
+ value = readl(bufcfg);
+
+ value &= ~mask;
+ value |= bits & mask;
+
+ writel(value, bufcfg);
+}
+
static int mrfld_get_groups_count(struct pinctrl_dev *pctldev)
{
struct mrfld_pinctrl *mp = pinctrl_dev_get_drvdata(pctldev);
unsigned int pin)
{
struct mrfld_pinctrl *mp = pinctrl_dev_get_drvdata(pctldev);
- void __iomem *bufcfg;
u32 value, mode;
+ int ret;
- if (!mrfld_buf_available(mp, pin)) {
+ ret = mrfld_read_bufcfg(mp, pin, &value);
+ if (ret) {
seq_puts(s, "not available");
return;
}
- bufcfg = mrfld_get_bufcfg(mp, pin);
- value = readl(bufcfg);
-
mode = (value & BUFCFG_PINMODE_MASK) >> BUFCFG_PINMODE_SHIFT;
if (!mode)
seq_puts(s, "GPIO ");
return 0;
}
-static void mrfld_update_bufcfg(struct mrfld_pinctrl *mp, unsigned int pin,
- u32 bits, u32 mask)
-{
- void __iomem *bufcfg;
- u32 value;
-
- bufcfg = mrfld_get_bufcfg(mp, pin);
- value = readl(bufcfg);
-
- value &= ~mask;
- value |= bits & mask;
-
- writel(value, bufcfg);
-}
-
static int mrfld_pinmux_set_mux(struct pinctrl_dev *pctldev,
unsigned int function,
unsigned int group)
enum pin_config_param param = pinconf_to_config_param(*config);
u32 value, term;
u16 arg = 0;
+ int ret;
- if (!mrfld_buf_available(mp, pin))
+ ret = mrfld_read_bufcfg(mp, pin, &value);
+ if (ret)
return -ENOTSUPP;
- value = readl(mrfld_get_bufcfg(mp, pin));
term = (value & BUFCFG_PUPD_VAL_MASK) >> BUFCFG_PUPD_VAL_SHIFT;
switch (param) {
* Mika Westerberg <mika.westerberg@linux.intel.com>
*/
-#include <linux/acpi.h>
+#include <linux/mod_devicetable.h>
#include <linux/module.h>
#include <linux/platform_device.h>
-#include <linux/pm.h>
+
#include <linux/pinctrl/pinctrl.h>
#include "pinctrl-intel.h"
static int spt_pinctrl_probe(struct platform_device *pdev)
{
- const struct intel_pinctrl_soc_data *soc_data;
- const struct acpi_device_id *id;
-
- id = acpi_match_device(spt_pinctrl_acpi_match, &pdev->dev);
- if (!id || !id->driver_data)
- return -ENODEV;
-
- soc_data = (const struct intel_pinctrl_soc_data *)id->driver_data;
- return intel_pinctrl_probe(pdev, soc_data);
+ return intel_pinctrl_probe_by_hid(pdev);
}
-static const struct dev_pm_ops spt_pinctrl_pm_ops = {
- SET_LATE_SYSTEM_SLEEP_PM_OPS(intel_pinctrl_suspend,
- intel_pinctrl_resume)
-};
+static INTEL_PINCTRL_PM_OPS(spt_pinctrl_pm_ops);
static struct platform_driver spt_pinctrl_driver = {
.probe = spt_pinctrl_probe,
config EINT_MTK
bool "MediaTek External Interrupt Support"
- depends on PINCTRL_MTK || PINCTRL_MT7622 || COMPILE_TEST
+ depends on PINCTRL_MTK || PINCTRL_MTK_MOORE || COMPILE_TEST
+ select GPIOLIB
select IRQ_DOMAIN
config PINCTRL_MTK
select EINT_MTK
select OF_GPIO
+config PINCTRL_MTK_MOORE
+ bool "MediaTek Moore Core that implements generic binding"
+ depends on OF
+ select GENERIC_PINCONF
+ select GENERIC_PINCTRL_GROUPS
+ select GENERIC_PINMUX_FUNCTIONS
+ select GPIOLIB
+ select OF_GPIO
+
+config PINCTRL_MTK_PARIS
+ bool "MediaTek Paris Core that implements vendor binding"
+ depends on OF
+ select PINMUX
+ select GENERIC_PINCONF
+ select GPIOLIB
+ select EINT_MTK
+ select OF_GPIO
+
# For ARMv7 SoCs
config PINCTRL_MT2701
bool "Mediatek MT2701 pin control"
default MACH_MT2701
select PINCTRL_MTK
+config PINCTRL_MT7623
+ bool "Mediatek MT7623 pin control with generic binding"
+ depends on MACH_MT7623 || COMPILE_TEST
+ depends on PINCTRL_MTK_MOORE
+ default y
+
config PINCTRL_MT8135
bool "Mediatek MT8135 pin control"
depends on MACH_MT8135 || COMPILE_TEST
default ARM64 && ARCH_MEDIATEK
select PINCTRL_MTK
+config PINCTRL_MT6765
+ bool "Mediatek MT6765 pin control"
+ depends on OF
+ depends on ARM64 || COMPILE_TEST
+ default ARM64 && ARCH_MEDIATEK
+ select PINCTRL_MTK_PARIS
+
config PINCTRL_MT7622
bool "MediaTek MT7622 pin control"
- depends on OF
depends on ARM64 || COMPILE_TEST
- select GENERIC_PINCONF
- select GENERIC_PINCTRL_GROUPS
- select GENERIC_PINMUX_FUNCTIONS
- select GPIOLIB
- select OF_GPIO
+ depends on PINCTRL_MTK_MOORE
+ default y
config PINCTRL_MT8173
bool "Mediatek MT8173 pin control"
default ARM64 && ARCH_MEDIATEK
select PINCTRL_MTK
+config PINCTRL_MT8183
+ bool "Mediatek MT8183 pin control"
+ depends on OF
+ depends on ARM64 || COMPILE_TEST
+ default ARM64 && ARCH_MEDIATEK
+ select PINCTRL_MTK_PARIS
+
# For PMIC
config PINCTRL_MT6397
bool "Mediatek MT6397 pin control"
# Core
obj-$(CONFIG_EINT_MTK) += mtk-eint.o
obj-$(CONFIG_PINCTRL_MTK) += pinctrl-mtk-common.o
+obj-$(CONFIG_PINCTRL_MTK_MOORE) += pinctrl-moore.o pinctrl-mtk-common-v2.o
+obj-$(CONFIG_PINCTRL_MTK_PARIS) += pinctrl-paris.o pinctrl-mtk-common-v2.o
# SoC Drivers
obj-$(CONFIG_PINCTRL_MT2701) += pinctrl-mt2701.o
obj-$(CONFIG_PINCTRL_MT2712) += pinctrl-mt2712.o
obj-$(CONFIG_PINCTRL_MT8135) += pinctrl-mt8135.o
obj-$(CONFIG_PINCTRL_MT8127) += pinctrl-mt8127.o
+obj-$(CONFIG_PINCTRL_MT6765) += pinctrl-mt6765.o
obj-$(CONFIG_PINCTRL_MT7622) += pinctrl-mt7622.o
+obj-$(CONFIG_PINCTRL_MT7623) += pinctrl-mt7623.o
obj-$(CONFIG_PINCTRL_MT8173) += pinctrl-mt8173.o
+obj-$(CONFIG_PINCTRL_MT8183) += pinctrl-mt8183.o
obj-$(CONFIG_PINCTRL_MT6397) += pinctrl-mt6397.o
#include <linux/delay.h>
#include <linux/err.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/io.h>
#include <linux/irqchip/chained_irq.h>
#include <linux/irqdomain.h>
return -EOPNOTSUPP;
}
-int mtk_eint_set_debounce(struct mtk_eint *eint, unsigned long eint_n,
+static inline int mtk_eint_set_debounce(struct mtk_eint *eint, unsigned long eint_n,
unsigned int debounce)
{
return -EOPNOTSUPP;
}
-int mtk_eint_find_irq(struct mtk_eint *eint, unsigned long eint_n)
+static inline int mtk_eint_find_irq(struct mtk_eint *eint, unsigned long eint_n)
{
return -EOPNOTSUPP;
}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * MediaTek Pinctrl Moore Driver, which implement the generic dt-binding
+ * pinctrl-bindings.txt for MediaTek SoC.
+ *
+ * Copyright (C) 2017-2018 MediaTek Inc.
+ * Author: Sean Wang <sean.wang@mediatek.com>
+ *
+ */
+
+#include <linux/gpio/driver.h>
+#include "pinctrl-moore.h"
+
+#define PINCTRL_PINCTRL_DEV KBUILD_MODNAME
+
+/* Custom pinconf parameters */
+#define MTK_PIN_CONFIG_TDSEL (PIN_CONFIG_END + 1)
+#define MTK_PIN_CONFIG_RDSEL (PIN_CONFIG_END + 2)
+#define MTK_PIN_CONFIG_PU_ADV (PIN_CONFIG_END + 3)
+#define MTK_PIN_CONFIG_PD_ADV (PIN_CONFIG_END + 4)
+
+static const struct pinconf_generic_params mtk_custom_bindings[] = {
+ {"mediatek,tdsel", MTK_PIN_CONFIG_TDSEL, 0},
+ {"mediatek,rdsel", MTK_PIN_CONFIG_RDSEL, 0},
+ {"mediatek,pull-up-adv", MTK_PIN_CONFIG_PU_ADV, 1},
+ {"mediatek,pull-down-adv", MTK_PIN_CONFIG_PD_ADV, 1},
+};
+
+#ifdef CONFIG_DEBUG_FS
+static const struct pin_config_item mtk_conf_items[] = {
+ PCONFDUMP(MTK_PIN_CONFIG_TDSEL, "tdsel", NULL, true),
+ PCONFDUMP(MTK_PIN_CONFIG_RDSEL, "rdsel", NULL, true),
+ PCONFDUMP(MTK_PIN_CONFIG_PU_ADV, "pu-adv", NULL, true),
+ PCONFDUMP(MTK_PIN_CONFIG_PD_ADV, "pd-adv", NULL, true),
+};
+#endif
+
+static int mtk_pinmux_set_mux(struct pinctrl_dev *pctldev,
+ unsigned int selector, unsigned int group)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+ struct function_desc *func;
+ struct group_desc *grp;
+ int i;
+
+ func = pinmux_generic_get_function(pctldev, selector);
+ if (!func)
+ return -EINVAL;
+
+ grp = pinctrl_generic_get_group(pctldev, group);
+ if (!grp)
+ return -EINVAL;
+
+ dev_dbg(pctldev->dev, "enable function %s group %s\n",
+ func->name, grp->name);
+
+ for (i = 0; i < grp->num_pins; i++) {
+ const struct mtk_pin_desc *desc;
+ int *pin_modes = grp->data;
+ int pin = grp->pins[i];
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[pin];
+
+ mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_MODE,
+ pin_modes[i]);
+ }
+
+ return 0;
+}
+
+static int mtk_pinmux_gpio_request_enable(struct pinctrl_dev *pctldev,
+ struct pinctrl_gpio_range *range,
+ unsigned int pin)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+ const struct mtk_pin_desc *desc;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[pin];
+
+ return mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_MODE,
+ hw->soc->gpio_m);
+}
+
+static int mtk_pinmux_gpio_set_direction(struct pinctrl_dev *pctldev,
+ struct pinctrl_gpio_range *range,
+ unsigned int pin, bool input)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+ const struct mtk_pin_desc *desc;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[pin];
+
+ /* hardware would take 0 as input direction */
+ return mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DIR, !input);
+}
+
+static int mtk_pinconf_get(struct pinctrl_dev *pctldev,
+ unsigned int pin, unsigned long *config)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+ u32 param = pinconf_to_config_param(*config);
+ int val, val2, err, reg, ret = 1;
+ const struct mtk_pin_desc *desc;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[pin];
+
+ switch (param) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ if (hw->soc->bias_disable_get) {
+ err = hw->soc->bias_disable_get(hw, desc, &ret);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case PIN_CONFIG_BIAS_PULL_UP:
+ if (hw->soc->bias_get) {
+ err = hw->soc->bias_get(hw, desc, 1, &ret);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ if (hw->soc->bias_get) {
+ err = hw->soc->bias_get(hw, desc, 0, &ret);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case PIN_CONFIG_SLEW_RATE:
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_SR, &val);
+ if (err)
+ return err;
+
+ if (!val)
+ return -EINVAL;
+
+ break;
+ case PIN_CONFIG_INPUT_ENABLE:
+ case PIN_CONFIG_OUTPUT_ENABLE:
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_DIR, &val);
+ if (err)
+ return err;
+
+ /* HW takes input mode as zero; output mode as non-zero */
+ if ((val && param == PIN_CONFIG_INPUT_ENABLE) ||
+ (!val && param == PIN_CONFIG_OUTPUT_ENABLE))
+ return -EINVAL;
+
+ break;
+ case PIN_CONFIG_INPUT_SCHMITT_ENABLE:
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_DIR, &val);
+ if (err)
+ return err;
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_SMT, &val2);
+ if (err)
+ return err;
+
+ if (val || !val2)
+ return -EINVAL;
+
+ break;
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ if (hw->soc->drive_get) {
+ err = hw->soc->drive_get(hw, desc, &ret);
+ if (err)
+ return err;
+ } else {
+ err = -ENOTSUPP;
+ }
+ break;
+ case MTK_PIN_CONFIG_TDSEL:
+ case MTK_PIN_CONFIG_RDSEL:
+ reg = (param == MTK_PIN_CONFIG_TDSEL) ?
+ PINCTRL_PIN_REG_TDSEL : PINCTRL_PIN_REG_RDSEL;
+
+ err = mtk_hw_get_value(hw, desc, reg, &val);
+ if (err)
+ return err;
+
+ ret = val;
+
+ break;
+ case MTK_PIN_CONFIG_PU_ADV:
+ case MTK_PIN_CONFIG_PD_ADV:
+ if (hw->soc->adv_pull_get) {
+ bool pullup;
+
+ pullup = param == MTK_PIN_CONFIG_PU_ADV;
+ err = hw->soc->adv_pull_get(hw, desc, pullup, &ret);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ default:
+ return -ENOTSUPP;
+ }
+
+ *config = pinconf_to_config_packed(param, ret);
+
+ return 0;
+}
+
+static int mtk_pinconf_set(struct pinctrl_dev *pctldev, unsigned int pin,
+ unsigned long *configs, unsigned int num_configs)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+ const struct mtk_pin_desc *desc;
+ u32 reg, param, arg;
+ int cfg, err = 0;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[pin];
+
+ for (cfg = 0; cfg < num_configs; cfg++) {
+ param = pinconf_to_config_param(configs[cfg]);
+ arg = pinconf_to_config_argument(configs[cfg]);
+
+ switch (param) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ if (hw->soc->bias_disable_set) {
+ err = hw->soc->bias_disable_set(hw, desc);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case PIN_CONFIG_BIAS_PULL_UP:
+ if (hw->soc->bias_set) {
+ err = hw->soc->bias_set(hw, desc, 1);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ if (hw->soc->bias_set) {
+ err = hw->soc->bias_set(hw, desc, 0);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case PIN_CONFIG_OUTPUT_ENABLE:
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_SMT,
+ MTK_DISABLE);
+ if (err)
+ goto err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DIR,
+ MTK_OUTPUT);
+ if (err)
+ goto err;
+ break;
+ case PIN_CONFIG_INPUT_ENABLE:
+
+ if (hw->soc->ies_present) {
+ mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_IES,
+ MTK_ENABLE);
+ }
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DIR,
+ MTK_INPUT);
+ if (err)
+ goto err;
+ break;
+ case PIN_CONFIG_SLEW_RATE:
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_SR,
+ arg);
+ if (err)
+ goto err;
+
+ break;
+ case PIN_CONFIG_OUTPUT:
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DIR,
+ MTK_OUTPUT);
+ if (err)
+ goto err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DO,
+ arg);
+ if (err)
+ goto err;
+ break;
+ case PIN_CONFIG_INPUT_SCHMITT_ENABLE:
+ /* arg = 1: Input mode & SMT enable ;
+ * arg = 0: Output mode & SMT disable
+ */
+ arg = arg ? 2 : 1;
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DIR,
+ arg & 1);
+ if (err)
+ goto err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_SMT,
+ !!(arg & 2));
+ if (err)
+ goto err;
+ break;
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ if (hw->soc->drive_set) {
+ err = hw->soc->drive_set(hw, desc, arg);
+ if (err)
+ return err;
+ } else {
+ err = -ENOTSUPP;
+ }
+ break;
+ case MTK_PIN_CONFIG_TDSEL:
+ case MTK_PIN_CONFIG_RDSEL:
+ reg = (param == MTK_PIN_CONFIG_TDSEL) ?
+ PINCTRL_PIN_REG_TDSEL : PINCTRL_PIN_REG_RDSEL;
+
+ err = mtk_hw_set_value(hw, desc, reg, arg);
+ if (err)
+ goto err;
+ break;
+ case MTK_PIN_CONFIG_PU_ADV:
+ case MTK_PIN_CONFIG_PD_ADV:
+ if (hw->soc->adv_pull_set) {
+ bool pullup;
+
+ pullup = param == MTK_PIN_CONFIG_PU_ADV;
+ err = hw->soc->adv_pull_set(hw, desc, pullup,
+ arg);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ default:
+ err = -ENOTSUPP;
+ }
+ }
+err:
+ return err;
+}
+
+static int mtk_pinconf_group_get(struct pinctrl_dev *pctldev,
+ unsigned int group, unsigned long *config)
+{
+ const unsigned int *pins;
+ unsigned int i, npins, old = 0;
+ int ret;
+
+ ret = pinctrl_generic_get_group_pins(pctldev, group, &pins, &npins);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < npins; i++) {
+ if (mtk_pinconf_get(pctldev, pins[i], config))
+ return -ENOTSUPP;
+
+ /* configs do not match between two pins */
+ if (i && old != *config)
+ return -ENOTSUPP;
+
+ old = *config;
+ }
+
+ return 0;
+}
+
+static int mtk_pinconf_group_set(struct pinctrl_dev *pctldev,
+ unsigned int group, unsigned long *configs,
+ unsigned int num_configs)
+{
+ const unsigned int *pins;
+ unsigned int i, npins;
+ int ret;
+
+ ret = pinctrl_generic_get_group_pins(pctldev, group, &pins, &npins);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < npins; i++) {
+ ret = mtk_pinconf_set(pctldev, pins[i], configs, num_configs);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static const struct pinctrl_ops mtk_pctlops = {
+ .get_groups_count = pinctrl_generic_get_group_count,
+ .get_group_name = pinctrl_generic_get_group_name,
+ .get_group_pins = pinctrl_generic_get_group_pins,
+ .dt_node_to_map = pinconf_generic_dt_node_to_map_all,
+ .dt_free_map = pinconf_generic_dt_free_map,
+};
+
+static const struct pinmux_ops mtk_pmxops = {
+ .get_functions_count = pinmux_generic_get_function_count,
+ .get_function_name = pinmux_generic_get_function_name,
+ .get_function_groups = pinmux_generic_get_function_groups,
+ .set_mux = mtk_pinmux_set_mux,
+ .gpio_request_enable = mtk_pinmux_gpio_request_enable,
+ .gpio_set_direction = mtk_pinmux_gpio_set_direction,
+ .strict = true,
+};
+
+static const struct pinconf_ops mtk_confops = {
+ .is_generic = true,
+ .pin_config_get = mtk_pinconf_get,
+ .pin_config_set = mtk_pinconf_set,
+ .pin_config_group_get = mtk_pinconf_group_get,
+ .pin_config_group_set = mtk_pinconf_group_set,
+ .pin_config_config_dbg_show = pinconf_generic_dump_config,
+};
+
+static struct pinctrl_desc mtk_desc = {
+ .name = PINCTRL_PINCTRL_DEV,
+ .pctlops = &mtk_pctlops,
+ .pmxops = &mtk_pmxops,
+ .confops = &mtk_confops,
+ .owner = THIS_MODULE,
+};
+
+static int mtk_gpio_get(struct gpio_chip *chip, unsigned int gpio)
+{
+ struct mtk_pinctrl *hw = gpiochip_get_data(chip);
+ const struct mtk_pin_desc *desc;
+ int value, err;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[gpio];
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_DI, &value);
+ if (err)
+ return err;
+
+ return !!value;
+}
+
+static void mtk_gpio_set(struct gpio_chip *chip, unsigned int gpio, int value)
+{
+ struct mtk_pinctrl *hw = gpiochip_get_data(chip);
+ const struct mtk_pin_desc *desc;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[gpio];
+
+ mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DO, !!value);
+}
+
+static int mtk_gpio_direction_input(struct gpio_chip *chip, unsigned int gpio)
+{
+ return pinctrl_gpio_direction_input(chip->base + gpio);
+}
+
+static int mtk_gpio_direction_output(struct gpio_chip *chip, unsigned int gpio,
+ int value)
+{
+ mtk_gpio_set(chip, gpio, value);
+
+ return pinctrl_gpio_direction_output(chip->base + gpio);
+}
+
+static int mtk_gpio_to_irq(struct gpio_chip *chip, unsigned int offset)
+{
+ struct mtk_pinctrl *hw = gpiochip_get_data(chip);
+ const struct mtk_pin_desc *desc;
+
+ if (!hw->eint)
+ return -ENOTSUPP;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[offset];
+
+ if (desc->eint.eint_n == (u16)EINT_NA)
+ return -ENOTSUPP;
+
+ return mtk_eint_find_irq(hw->eint, desc->eint.eint_n);
+}
+
+static int mtk_gpio_set_config(struct gpio_chip *chip, unsigned int offset,
+ unsigned long config)
+{
+ struct mtk_pinctrl *hw = gpiochip_get_data(chip);
+ const struct mtk_pin_desc *desc;
+ u32 debounce;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[offset];
+
+ if (!hw->eint ||
+ pinconf_to_config_param(config) != PIN_CONFIG_INPUT_DEBOUNCE ||
+ desc->eint.eint_n == (u16)EINT_NA)
+ return -ENOTSUPP;
+
+ debounce = pinconf_to_config_argument(config);
+
+ return mtk_eint_set_debounce(hw->eint, desc->eint.eint_n, debounce);
+}
+
+static int mtk_build_gpiochip(struct mtk_pinctrl *hw, struct device_node *np)
+{
+ struct gpio_chip *chip = &hw->chip;
+ int ret;
+
+ chip->label = PINCTRL_PINCTRL_DEV;
+ chip->parent = hw->dev;
+ chip->request = gpiochip_generic_request;
+ chip->free = gpiochip_generic_free;
+ chip->direction_input = mtk_gpio_direction_input;
+ chip->direction_output = mtk_gpio_direction_output;
+ chip->get = mtk_gpio_get;
+ chip->set = mtk_gpio_set;
+ chip->to_irq = mtk_gpio_to_irq,
+ chip->set_config = mtk_gpio_set_config,
+ chip->base = -1;
+ chip->ngpio = hw->soc->npins;
+ chip->of_node = np;
+ chip->of_gpio_n_cells = 2;
+
+ ret = gpiochip_add_data(chip, hw);
+ if (ret < 0)
+ return ret;
+
+ /* Just for backward compatible for these old pinctrl nodes without
+ * "gpio-ranges" property. Otherwise, called directly from a
+ * DeviceTree-supported pinctrl driver is DEPRECATED.
+ * Please see Section 2.1 of
+ * Documentation/devicetree/bindings/gpio/gpio.txt on how to
+ * bind pinctrl and gpio drivers via the "gpio-ranges" property.
+ */
+ if (!of_find_property(np, "gpio-ranges", NULL)) {
+ ret = gpiochip_add_pin_range(chip, dev_name(hw->dev), 0, 0,
+ chip->ngpio);
+ if (ret < 0) {
+ gpiochip_remove(chip);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static int mtk_build_groups(struct mtk_pinctrl *hw)
+{
+ int err, i;
+
+ for (i = 0; i < hw->soc->ngrps; i++) {
+ const struct group_desc *group = hw->soc->grps + i;
+
+ err = pinctrl_generic_add_group(hw->pctrl, group->name,
+ group->pins, group->num_pins,
+ group->data);
+ if (err < 0) {
+ dev_err(hw->dev, "Failed to register group %s\n",
+ group->name);
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+static int mtk_build_functions(struct mtk_pinctrl *hw)
+{
+ int i, err;
+
+ for (i = 0; i < hw->soc->nfuncs ; i++) {
+ const struct function_desc *func = hw->soc->funcs + i;
+
+ err = pinmux_generic_add_function(hw->pctrl, func->name,
+ func->group_names,
+ func->num_group_names,
+ func->data);
+ if (err < 0) {
+ dev_err(hw->dev, "Failed to register function %s\n",
+ func->name);
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+int mtk_moore_pinctrl_probe(struct platform_device *pdev,
+ const struct mtk_pin_soc *soc)
+{
+ struct pinctrl_pin_desc *pins;
+ struct resource *res;
+ struct mtk_pinctrl *hw;
+ int err, i;
+
+ hw = devm_kzalloc(&pdev->dev, sizeof(*hw), GFP_KERNEL);
+ if (!hw)
+ return -ENOMEM;
+
+ hw->soc = soc;
+ hw->dev = &pdev->dev;
+
+ if (!hw->soc->nbase_names) {
+ dev_err(&pdev->dev,
+ "SoC should be assigned at least one register base\n");
+ return -EINVAL;
+ }
+
+ hw->base = devm_kmalloc_array(&pdev->dev, hw->soc->nbase_names,
+ sizeof(*hw->base), GFP_KERNEL);
+ if (!hw->base)
+ return -ENOMEM;
+
+ for (i = 0; i < hw->soc->nbase_names; i++) {
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
+ hw->soc->base_names[i]);
+ if (!res) {
+ dev_err(&pdev->dev, "missing IO resource\n");
+ return -ENXIO;
+ }
+
+ hw->base[i] = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(hw->base[i]))
+ return PTR_ERR(hw->base[i]);
+ }
+
+ hw->nbase = hw->soc->nbase_names;
+
+ /* Copy from internal struct mtk_pin_desc to register to the core */
+ pins = devm_kmalloc_array(&pdev->dev, hw->soc->npins, sizeof(*pins),
+ GFP_KERNEL);
+ if (!pins)
+ return -ENOMEM;
+
+ for (i = 0; i < hw->soc->npins; i++) {
+ pins[i].number = hw->soc->pins[i].number;
+ pins[i].name = hw->soc->pins[i].name;
+ }
+
+ /* Setup pins descriptions per SoC types */
+ mtk_desc.pins = (const struct pinctrl_pin_desc *)pins;
+ mtk_desc.npins = hw->soc->npins;
+ mtk_desc.num_custom_params = ARRAY_SIZE(mtk_custom_bindings);
+ mtk_desc.custom_params = mtk_custom_bindings;
+#ifdef CONFIG_DEBUG_FS
+ mtk_desc.custom_conf_items = mtk_conf_items;
+#endif
+
+ err = devm_pinctrl_register_and_init(&pdev->dev, &mtk_desc, hw,
+ &hw->pctrl);
+ if (err)
+ return err;
+
+ /* Setup groups descriptions per SoC types */
+ err = mtk_build_groups(hw);
+ if (err) {
+ dev_err(&pdev->dev, "Failed to build groups\n");
+ return err;
+ }
+
+ /* Setup functions descriptions per SoC types */
+ err = mtk_build_functions(hw);
+ if (err) {
+ dev_err(&pdev->dev, "Failed to build functions\n");
+ return err;
+ }
+
+ /* For able to make pinctrl_claim_hogs, we must not enable pinctrl
+ * until all groups and functions are being added one.
+ */
+ err = pinctrl_enable(hw->pctrl);
+ if (err)
+ return err;
+
+ err = mtk_build_eint(hw, pdev);
+ if (err)
+ dev_warn(&pdev->dev,
+ "Failed to add EINT, but pinctrl still can work\n");
+
+ /* Build gpiochip should be after pinctrl_enable is done */
+ err = mtk_build_gpiochip(hw, pdev->dev.of_node);
+ if (err) {
+ dev_err(&pdev->dev, "Failed to add gpio_chip\n");
+ return err;
+ }
+
+ platform_set_drvdata(pdev, hw);
+
+ return 0;
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2017-2018 MediaTek Inc.
+ *
+ * Author: Sean Wang <sean.wang@mediatek.com>
+ *
+ */
+#ifndef __PINCTRL_MOORE_H
+#define __PINCTRL_MOORE_H
+
+#include <linux/io.h>
+#include <linux/init.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <linux/pinctrl/pinctrl.h>
+#include <linux/pinctrl/pinmux.h>
+#include <linux/pinctrl/pinconf.h>
+#include <linux/pinctrl/pinconf-generic.h>
+
+#include "../core.h"
+#include "../pinconf.h"
+#include "../pinmux.h"
+#include "mtk-eint.h"
+#include "pinctrl-mtk-common-v2.h"
+
+#define MTK_RANGE(_a) { .range = (_a), .nranges = ARRAY_SIZE(_a), }
+
+#define MTK_PIN(_number, _name, _eint_m, _eint_n, _drv_n) { \
+ .number = _number, \
+ .name = _name, \
+ .eint = { \
+ .eint_m = _eint_m, \
+ .eint_n = _eint_n, \
+ }, \
+ .drv_n = _drv_n, \
+ .funcs = NULL, \
+ }
+
+#define PINCTRL_PIN_GROUP(name, id) \
+ { \
+ name, \
+ id##_pins, \
+ ARRAY_SIZE(id##_pins), \
+ id##_funcs, \
+ }
+
+int mtk_moore_pinctrl_probe(struct platform_device *pdev,
+ const struct mtk_pin_soc *soc);
+
+#endif /* __PINCTRL_MOORE_H */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 MediaTek Inc.
+ *
+ * Author: ZH Chen <zh.chen@mediatek.com>
+ *
+ */
+
+#include "pinctrl-mtk-mt6765.h"
+#include "pinctrl-paris.h"
+
+/* MT6765 have multiple bases to program pin configuration listed as the below:
+ * iocfg[0]:0x10005000, iocfg[1]:0x10002C00, iocfg[2]:0x10002800,
+ * iocfg[3]:0x10002A00, iocfg[4]:0x10002000, iocfg[5]:0x10002200,
+ * iocfg[6]:0x10002500, iocfg[7]:0x10002600.
+ * _i_base could be used to indicate what base the pin should be mapped into.
+ */
+
+#define PIN_FIELD_BASE(_s_pin, _e_pin, _i_base, _s_addr, _x_addrs, _s_bit, _x_bits) \
+ PIN_FIELD_CALC(_s_pin, _e_pin, _i_base, _s_addr, _x_addrs, _s_bit, \
+ _x_bits, 32, 0)
+
+#define PINS_FIELD_BASE(_s_pin, _e_pin, _i_base, _s_addr, _x_addrs, _s_bit, _x_bits) \
+ PIN_FIELD_CALC(_s_pin, _e_pin, _i_base, _s_addr, _x_addrs, _s_bit, \
+ _x_bits, 32, 1)
+
+static const struct mtk_pin_field_calc mt6765_pin_mode_range[] = {
+ PIN_FIELD(0, 202, 0x300, 0x10, 0, 4),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_dir_range[] = {
+ PIN_FIELD(0, 202, 0x0, 0x10, 0, 1),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_di_range[] = {
+ PIN_FIELD(0, 202, 0x200, 0x10, 0, 1),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_do_range[] = {
+ PIN_FIELD(0, 202, 0x100, 0x10, 0, 1),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_smt_range[] = {
+ PINS_FIELD_BASE(0, 3, 2, 0x00b0, 0x10, 4, 1),
+ PINS_FIELD_BASE(4, 7, 2, 0x00b0, 0x10, 5, 1),
+ PIN_FIELD_BASE(8, 8, 3, 0x0080, 0x10, 3, 1),
+ PINS_FIELD_BASE(9, 11, 2, 0x00b0, 0x10, 6, 1),
+ PIN_FIELD_BASE(12, 12, 5, 0x0060, 0x10, 9, 1),
+ PINS_FIELD_BASE(13, 16, 6, 0x00b0, 0x10, 10, 1),
+ PINS_FIELD_BASE(17, 20, 6, 0x00b0, 0x10, 8, 1),
+ PINS_FIELD_BASE(21, 24, 6, 0x00b0, 0x10, 9, 1),
+ PINS_FIELD_BASE(25, 28, 6, 0x00b0, 0x10, 7, 1),
+ PIN_FIELD_BASE(29, 29, 6, 0x00b0, 0x10, 0, 1),
+ PIN_FIELD_BASE(30, 30, 6, 0x00b0, 0x10, 1, 1),
+ PINS_FIELD_BASE(31, 34, 6, 0x00b0, 0x10, 2, 1),
+ PINS_FIELD_BASE(35, 36, 6, 0x00b0, 0x10, 5, 1),
+ PIN_FIELD_BASE(37, 37, 6, 0x00b0, 0x10, 6, 1),
+ PIN_FIELD_BASE(38, 38, 6, 0x00b0, 0x10, 4, 1),
+ PINS_FIELD_BASE(39, 40, 6, 0x00b0, 0x10, 3, 1),
+ PINS_FIELD_BASE(41, 42, 7, 0x00c0, 0x10, 6, 1),
+ PIN_FIELD_BASE(43, 43, 7, 0x00c0, 0x10, 3, 1),
+ PIN_FIELD_BASE(44, 44, 7, 0x00c0, 0x10, 4, 1),
+ PIN_FIELD_BASE(45, 45, 7, 0x00c0, 0x10, 8, 1),
+ PINS_FIELD_BASE(46, 47, 7, 0x00c0, 0x10, 7, 1),
+ PIN_FIELD_BASE(48, 48, 7, 0x00c0, 0x10, 15, 1),
+ PIN_FIELD_BASE(49, 49, 7, 0x00c0, 0x10, 17, 1),
+ PIN_FIELD_BASE(50, 50, 7, 0x00c0, 0x10, 14, 1),
+ PIN_FIELD_BASE(51, 51, 7, 0x00c0, 0x10, 16, 1),
+ PINS_FIELD_BASE(52, 57, 7, 0x00c0, 0x10, 0, 1),
+ PINS_FIELD_BASE(58, 60, 7, 0x00c0, 0x10, 12, 1),
+ PINS_FIELD_BASE(61, 62, 3, 0x0080, 0x10, 5, 1),
+ PINS_FIELD_BASE(63, 64, 3, 0x0080, 0x10, 4, 1),
+ PINS_FIELD_BASE(65, 66, 3, 0x0080, 0x10, 7, 1),
+ PINS_FIELD_BASE(67, 68, 3, 0x0080, 0x10, 6, 1),
+ PINS_FIELD_BASE(69, 73, 3, 0x0080, 0x10, 1, 1),
+ PINS_FIELD_BASE(74, 78, 3, 0x0080, 0x10, 2, 1),
+ PINS_FIELD_BASE(79, 80, 3, 0x0080, 0x10, 0, 1),
+ PIN_FIELD_BASE(81, 81, 3, 0x0080, 0x10, 12, 1),
+ PIN_FIELD_BASE(82, 82, 3, 0x0080, 0x10, 11, 1),
+ PIN_FIELD_BASE(83, 83, 3, 0x0080, 0x10, 9, 1),
+ PIN_FIELD_BASE(84, 84, 3, 0x0080, 0x10, 10, 1),
+ PIN_FIELD_BASE(85, 85, 7, 0x00c0, 0x10, 12, 1),
+ PIN_FIELD_BASE(86, 86, 7, 0x00c0, 0x10, 13, 1),
+ PIN_FIELD_BASE(87, 87, 7, 0x00c0, 0x10, 2, 1),
+ PIN_FIELD_BASE(88, 88, 7, 0x00c0, 0x10, 1, 1),
+ PIN_FIELD_BASE(89, 89, 2, 0x00b0, 0x10, 13, 1),
+ PIN_FIELD_BASE(90, 90, 3, 0x0080, 0x10, 8, 1),
+ PINS_FIELD_BASE(91, 92, 2, 0x00b0, 0x10, 8, 1),
+ PINS_FIELD_BASE(93, 94, 2, 0x00b0, 0x10, 7, 1),
+ PINS_FIELD_BASE(95, 96, 2, 0x00b0, 0x10, 14, 1),
+ PINS_FIELD_BASE(97, 98, 2, 0x00b0, 0x10, 2, 1),
+ PIN_FIELD_BASE(99, 99, 2, 0x00b0, 0x10, 0, 1),
+ PIN_FIELD_BASE(100, 100, 2, 0x00b0, 0x10, 1, 1),
+ PINS_FIELD_BASE(101, 102, 2, 0x00b0, 0x10, 3, 1),
+ PIN_FIELD_BASE(103, 103, 2, 0x00b0, 0x10, 9, 1),
+ PIN_FIELD_BASE(104, 104, 2, 0x00b0, 0x10, 11, 1),
+ PIN_FIELD_BASE(105, 105, 2, 0x00b0, 0x10, 10, 1),
+ PIN_FIELD_BASE(106, 106, 2, 0x00b0, 0x10, 12, 1),
+ PIN_FIELD_BASE(107, 107, 1, 0x0080, 0x10, 4, 1),
+ PIN_FIELD_BASE(108, 108, 1, 0x0080, 0x10, 3, 1),
+ PIN_FIELD_BASE(109, 109, 1, 0x0080, 0x10, 5, 1),
+ PIN_FIELD_BASE(110, 110, 1, 0x0080, 0x10, 0, 1),
+ PIN_FIELD_BASE(111, 111, 1, 0x0080, 0x10, 1, 1),
+ PIN_FIELD_BASE(112, 112, 1, 0x0080, 0x10, 2, 1),
+ PIN_FIELD_BASE(113, 113, 1, 0x0080, 0x10, 9, 1),
+ PIN_FIELD_BASE(114, 114, 1, 0x0080, 0x10, 10, 1),
+ PIN_FIELD_BASE(115, 115, 1, 0x0080, 0x10, 6, 1),
+ PIN_FIELD_BASE(116, 116, 1, 0x0080, 0x10, 7, 1),
+ PIN_FIELD_BASE(117, 117, 1, 0x0080, 0x10, 12, 1),
+ PIN_FIELD_BASE(118, 118, 1, 0x0080, 0x10, 13, 1),
+ PIN_FIELD_BASE(119, 119, 1, 0x0080, 0x10, 14, 1),
+ PIN_FIELD_BASE(120, 120, 1, 0x0080, 0x10, 11, 1),
+ PIN_FIELD_BASE(121, 121, 1, 0x0080, 0x10, 8, 1),
+ PIN_FIELD_BASE(122, 122, 4, 0x0080, 0x10, 2, 1),
+ PIN_FIELD_BASE(123, 123, 4, 0x0080, 0x10, 3, 1),
+ PIN_FIELD_BASE(124, 124, 4, 0x0080, 0x10, 1, 1),
+ PIN_FIELD_BASE(125, 125, 4, 0x0080, 0x10, 5, 1),
+ PIN_FIELD_BASE(126, 126, 4, 0x0080, 0x10, 7, 1),
+ PIN_FIELD_BASE(127, 127, 4, 0x0080, 0x10, 9, 1),
+ PIN_FIELD_BASE(128, 128, 4, 0x0080, 0x10, 4, 1),
+ PIN_FIELD_BASE(129, 129, 4, 0x0080, 0x10, 8, 1),
+ PIN_FIELD_BASE(130, 130, 4, 0x0080, 0x10, 10, 1),
+ PIN_FIELD_BASE(131, 131, 4, 0x0080, 0x10, 11, 1),
+ PIN_FIELD_BASE(132, 132, 4, 0x0080, 0x10, 6, 1),
+ PIN_FIELD_BASE(133, 133, 4, 0x0080, 0x10, 12, 1),
+ PIN_FIELD_BASE(134, 134, 5, 0x0060, 0x10, 11, 1),
+ PIN_FIELD_BASE(135, 135, 5, 0x0060, 0x10, 13, 1),
+ PIN_FIELD_BASE(136, 136, 5, 0x0060, 0x10, 1, 1),
+ PIN_FIELD_BASE(137, 137, 5, 0x0060, 0x10, 7, 1),
+ PIN_FIELD_BASE(138, 138, 5, 0x0060, 0x10, 4, 1),
+ PIN_FIELD_BASE(139, 139, 5, 0x0060, 0x10, 5, 1),
+ PIN_FIELD_BASE(140, 140, 5, 0x0060, 0x10, 0, 1),
+ PIN_FIELD_BASE(141, 141, 5, 0x0060, 0x10, 6, 1),
+ PIN_FIELD_BASE(142, 142, 5, 0x0060, 0x10, 2, 1),
+ PIN_FIELD_BASE(143, 143, 5, 0x0060, 0x10, 3, 1),
+ PINS_FIELD_BASE(144, 147, 5, 0x0060, 0x10, 10, 1),
+ PINS_FIELD_BASE(148, 149, 5, 0x0060, 0x10, 12, 1),
+ PINS_FIELD_BASE(150, 151, 7, 0x00c0, 0x10, 9, 1),
+ PINS_FIELD_BASE(152, 153, 7, 0x00c0, 0x10, 10, 1),
+ PIN_FIELD_BASE(154, 154, 7, 0x00c0, 0x10, 11, 1),
+ PINS_FIELD_BASE(155, 158, 3, 0x0080, 0x10, 13, 1),
+ PIN_FIELD_BASE(159, 159, 7, 0x00c0, 0x10, 11, 1),
+ PIN_FIELD_BASE(160, 160, 5, 0x0060, 0x10, 8, 1),
+ PIN_FIELD_BASE(161, 161, 1, 0x0080, 0x10, 15, 1),
+ PIN_FIELD_BASE(162, 162, 1, 0x0080, 0x10, 16, 1),
+ PINS_FIELD_BASE(163, 170, 4, 0x0080, 0x10, 0, 1),
+ PINS_FIELD_BASE(171, 179, 7, 0x00c0, 0x10, 5, 1),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_pd_range[] = {
+ PIN_FIELD_BASE(0, 0, 2, 0x0040, 0x10, 6, 1),
+ PIN_FIELD_BASE(1, 1, 2, 0x0040, 0x10, 7, 1),
+ PIN_FIELD_BASE(2, 2, 2, 0x0040, 0x10, 10, 1),
+ PIN_FIELD_BASE(3, 3, 2, 0x0040, 0x10, 11, 1),
+ PIN_FIELD_BASE(4, 4, 2, 0x0040, 0x10, 12, 1),
+ PIN_FIELD_BASE(5, 5, 2, 0x0040, 0x10, 13, 1),
+ PIN_FIELD_BASE(6, 6, 2, 0x0040, 0x10, 14, 1),
+ PIN_FIELD_BASE(7, 7, 2, 0x0040, 0x10, 15, 1),
+ PIN_FIELD_BASE(8, 8, 3, 0x0040, 0x10, 12, 1),
+ PIN_FIELD_BASE(9, 9, 2, 0x0040, 0x10, 16, 1),
+ PIN_FIELD_BASE(10, 10, 2, 0x0040, 0x10, 8, 1),
+ PIN_FIELD_BASE(11, 11, 2, 0x0040, 0x10, 9, 1),
+ PIN_FIELD_BASE(12, 12, 5, 0x0030, 0x10, 9, 1),
+ PIN_FIELD_BASE(13, 13, 6, 0x0040, 0x10, 14, 1),
+ PIN_FIELD_BASE(14, 14, 6, 0x0040, 0x10, 13, 1),
+ PIN_FIELD_BASE(15, 15, 6, 0x0040, 0x10, 15, 1),
+ PIN_FIELD_BASE(16, 16, 6, 0x0040, 0x10, 12, 1),
+ PIN_FIELD_BASE(17, 17, 6, 0x0040, 0x10, 7, 1),
+ PIN_FIELD_BASE(18, 18, 6, 0x0040, 0x10, 4, 1),
+ PIN_FIELD_BASE(19, 19, 6, 0x0040, 0x10, 6, 1),
+ PIN_FIELD_BASE(20, 20, 6, 0x0040, 0x10, 5, 1),
+ PIN_FIELD_BASE(21, 21, 6, 0x0040, 0x10, 10, 1),
+ PIN_FIELD_BASE(22, 22, 6, 0x0040, 0x10, 9, 1),
+ PIN_FIELD_BASE(23, 23, 6, 0x0040, 0x10, 11, 1),
+ PIN_FIELD_BASE(24, 24, 6, 0x0040, 0x10, 8, 1),
+ PIN_FIELD_BASE(25, 25, 6, 0x0040, 0x10, 2, 1),
+ PIN_FIELD_BASE(26, 26, 6, 0x0040, 0x10, 1, 1),
+ PIN_FIELD_BASE(27, 27, 6, 0x0040, 0x10, 3, 1),
+ PINS_FIELD_BASE(28, 40, 6, 0x0040, 0x10, 0, 1),
+ PIN_FIELD_BASE(41, 41, 7, 0x0060, 0x10, 19, 1),
+ PIN_FIELD_BASE(42, 42, 7, 0x0060, 0x10, 9, 1),
+ PIN_FIELD_BASE(43, 43, 7, 0x0060, 0x10, 8, 1),
+ PIN_FIELD_BASE(44, 44, 7, 0x0060, 0x10, 10, 1),
+ PIN_FIELD_BASE(45, 45, 7, 0x0060, 0x10, 22, 1),
+ PIN_FIELD_BASE(46, 46, 7, 0x0060, 0x10, 21, 1),
+ PIN_FIELD_BASE(47, 47, 7, 0x0060, 0x10, 20, 1),
+ PIN_FIELD_BASE(48, 48, 7, 0x0070, 0x10, 3, 1),
+ PIN_FIELD_BASE(49, 49, 7, 0x0070, 0x10, 5, 1),
+ PIN_FIELD_BASE(50, 50, 7, 0x0070, 0x10, 2, 1),
+ PIN_FIELD_BASE(51, 51, 7, 0x0070, 0x10, 4, 1),
+ PIN_FIELD_BASE(52, 52, 7, 0x0060, 0x10, 1, 1),
+ PIN_FIELD_BASE(53, 53, 7, 0x0060, 0x10, 0, 1),
+ PIN_FIELD_BASE(54, 54, 7, 0x0060, 0x10, 5, 1),
+ PIN_FIELD_BASE(55, 55, 7, 0x0060, 0x10, 3, 1),
+ PIN_FIELD_BASE(56, 56, 7, 0x0060, 0x10, 4, 1),
+ PIN_FIELD_BASE(57, 57, 7, 0x0060, 0x10, 2, 1),
+ PIN_FIELD_BASE(58, 58, 7, 0x0070, 0x10, 0, 1),
+ PIN_FIELD_BASE(59, 59, 7, 0x0060, 0x10, 31, 1),
+ PIN_FIELD_BASE(60, 60, 7, 0x0060, 0x10, 30, 1),
+ PIN_FIELD_BASE(61, 61, 3, 0x0040, 0x10, 18, 1),
+ PIN_FIELD_BASE(62, 62, 3, 0x0040, 0x10, 14, 1),
+ PIN_FIELD_BASE(63, 63, 3, 0x0040, 0x10, 17, 1),
+ PIN_FIELD_BASE(64, 64, 3, 0x0040, 0x10, 13, 1),
+ PIN_FIELD_BASE(65, 65, 3, 0x0040, 0x10, 20, 1),
+ PIN_FIELD_BASE(66, 66, 3, 0x0040, 0x10, 16, 1),
+ PIN_FIELD_BASE(67, 67, 3, 0x0040, 0x10, 19, 1),
+ PIN_FIELD_BASE(68, 68, 3, 0x0040, 0x10, 15, 1),
+ PIN_FIELD_BASE(69, 69, 3, 0x0040, 0x10, 8, 1),
+ PIN_FIELD_BASE(70, 70, 3, 0x0040, 0x10, 7, 1),
+ PIN_FIELD_BASE(71, 71, 3, 0x0040, 0x10, 6, 1),
+ PIN_FIELD_BASE(72, 72, 3, 0x0040, 0x10, 5, 1),
+ PIN_FIELD_BASE(73, 73, 3, 0x0040, 0x10, 4, 1),
+ PIN_FIELD_BASE(74, 74, 3, 0x0040, 0x10, 3, 1),
+ PIN_FIELD_BASE(75, 75, 3, 0x0040, 0x10, 2, 1),
+ PIN_FIELD_BASE(76, 76, 3, 0x0040, 0x10, 1, 1),
+ PIN_FIELD_BASE(77, 77, 3, 0x0040, 0x10, 0, 1),
+ PIN_FIELD_BASE(78, 78, 3, 0x0040, 0x10, 9, 1),
+ PIN_FIELD_BASE(79, 79, 3, 0x0040, 0x10, 11, 1),
+ PIN_FIELD_BASE(80, 80, 3, 0x0040, 0x10, 10, 1),
+ PIN_FIELD_BASE(81, 81, 3, 0x0040, 0x10, 25, 1),
+ PIN_FIELD_BASE(82, 82, 3, 0x0040, 0x10, 24, 1),
+ PIN_FIELD_BASE(83, 83, 3, 0x0040, 0x10, 22, 1),
+ PIN_FIELD_BASE(84, 84, 3, 0x0040, 0x10, 23, 1),
+ PIN_FIELD_BASE(85, 85, 7, 0x0070, 0x10, 1, 1),
+ PIN_FIELD_BASE(86, 86, 7, 0x0060, 0x10, 29, 1),
+ PIN_FIELD_BASE(87, 87, 7, 0x0060, 0x10, 7, 1),
+ PIN_FIELD_BASE(88, 88, 7, 0x0060, 0x10, 6, 1),
+ PIN_FIELD_BASE(89, 89, 2, 0x0040, 0x10, 21, 1),
+ PINS_FIELD_BASE(90, 94, 3, 0x0040, 0x10, 21, 1),
+ PIN_FIELD_BASE(95, 95, 2, 0x0040, 0x10, 22, 1),
+ PIN_FIELD_BASE(96, 96, 2, 0x0040, 0x10, 23, 1),
+ PIN_FIELD_BASE(97, 97, 2, 0x0040, 0x10, 2, 1),
+ PIN_FIELD_BASE(98, 98, 2, 0x0040, 0x10, 3, 1),
+ PIN_FIELD_BASE(99, 99, 2, 0x0040, 0x10, 0, 1),
+ PIN_FIELD_BASE(100, 100, 2, 0x0040, 0x10, 1, 1),
+ PIN_FIELD_BASE(101, 101, 2, 0x0040, 0x10, 4, 1),
+ PIN_FIELD_BASE(102, 102, 2, 0x0040, 0x10, 5, 1),
+ PIN_FIELD_BASE(103, 103, 2, 0x0040, 0x10, 17, 1),
+ PIN_FIELD_BASE(104, 104, 2, 0x0040, 0x10, 19, 1),
+ PIN_FIELD_BASE(105, 105, 2, 0x0040, 0x10, 18, 1),
+ PIN_FIELD_BASE(106, 106, 2, 0x0040, 0x10, 20, 1),
+ PIN_FIELD_BASE(107, 107, 1, 0x0040, 0x10, 4, 1),
+ PIN_FIELD_BASE(108, 108, 1, 0x0040, 0x10, 3, 1),
+ PIN_FIELD_BASE(109, 109, 1, 0x0040, 0x10, 5, 1),
+ PIN_FIELD_BASE(110, 110, 1, 0x0040, 0x10, 0, 1),
+ PIN_FIELD_BASE(111, 111, 1, 0x0040, 0x10, 1, 1),
+ PIN_FIELD_BASE(112, 112, 1, 0x0040, 0x10, 2, 1),
+ PIN_FIELD_BASE(113, 113, 1, 0x0040, 0x10, 9, 1),
+ PIN_FIELD_BASE(114, 114, 1, 0x0040, 0x10, 10, 1),
+ PIN_FIELD_BASE(115, 115, 1, 0x0040, 0x10, 6, 1),
+ PIN_FIELD_BASE(116, 116, 1, 0x0040, 0x10, 7, 1),
+ PIN_FIELD_BASE(117, 117, 1, 0x0040, 0x10, 12, 1),
+ PIN_FIELD_BASE(118, 118, 1, 0x0040, 0x10, 13, 1),
+ PIN_FIELD_BASE(119, 119, 1, 0x0040, 0x10, 14, 1),
+ PIN_FIELD_BASE(120, 120, 1, 0x0040, 0x10, 11, 1),
+ PINS_FIELD_BASE(121, 133, 1, 0x0040, 0x10, 8, 1),
+ PIN_FIELD_BASE(134, 134, 5, 0x0030, 0x10, 14, 1),
+ PIN_FIELD_BASE(135, 135, 5, 0x0030, 0x10, 19, 1),
+ PIN_FIELD_BASE(136, 136, 5, 0x0030, 0x10, 1, 1),
+ PIN_FIELD_BASE(137, 137, 5, 0x0030, 0x10, 7, 1),
+ PIN_FIELD_BASE(138, 138, 5, 0x0030, 0x10, 4, 1),
+ PIN_FIELD_BASE(139, 139, 5, 0x0030, 0x10, 5, 1),
+ PIN_FIELD_BASE(140, 140, 5, 0x0030, 0x10, 0, 1),
+ PIN_FIELD_BASE(141, 141, 5, 0x0030, 0x10, 6, 1),
+ PIN_FIELD_BASE(142, 142, 5, 0x0030, 0x10, 2, 1),
+ PIN_FIELD_BASE(143, 143, 5, 0x0030, 0x10, 3, 1),
+ PIN_FIELD_BASE(144, 144, 5, 0x0030, 0x10, 12, 1),
+ PIN_FIELD_BASE(145, 145, 5, 0x0030, 0x10, 11, 1),
+ PIN_FIELD_BASE(146, 146, 5, 0x0030, 0x10, 13, 1),
+ PIN_FIELD_BASE(147, 147, 5, 0x0030, 0x10, 10, 1),
+ PIN_FIELD_BASE(148, 148, 5, 0x0030, 0x10, 15, 1),
+ PIN_FIELD_BASE(149, 149, 5, 0x0030, 0x10, 16, 1),
+ PIN_FIELD_BASE(150, 150, 7, 0x0060, 0x10, 23, 1),
+ PIN_FIELD_BASE(151, 151, 7, 0x0060, 0x10, 24, 1),
+ PIN_FIELD_BASE(152, 152, 7, 0x0060, 0x10, 25, 1),
+ PIN_FIELD_BASE(153, 153, 7, 0x0060, 0x10, 26, 1),
+ PIN_FIELD_BASE(154, 154, 7, 0x0060, 0x10, 28, 1),
+ PIN_FIELD_BASE(155, 155, 3, 0x0040, 0x10, 28, 1),
+ PIN_FIELD_BASE(156, 156, 3, 0x0040, 0x10, 27, 1),
+ PIN_FIELD_BASE(157, 157, 3, 0x0040, 0x10, 29, 1),
+ PIN_FIELD_BASE(158, 158, 3, 0x0040, 0x10, 26, 1),
+ PIN_FIELD_BASE(159, 159, 7, 0x0060, 0x10, 27, 1),
+ PIN_FIELD_BASE(160, 160, 5, 0x0030, 0x10, 8, 1),
+ PIN_FIELD_BASE(161, 161, 1, 0x0040, 0x10, 15, 1),
+ PIN_FIELD_BASE(162, 162, 1, 0x0040, 0x10, 16, 1),
+ PIN_FIELD_BASE(163, 163, 4, 0x0020, 0x10, 0, 1),
+ PIN_FIELD_BASE(164, 164, 4, 0x0020, 0x10, 1, 1),
+ PIN_FIELD_BASE(165, 165, 4, 0x0020, 0x10, 2, 1),
+ PIN_FIELD_BASE(166, 166, 4, 0x0020, 0x10, 3, 1),
+ PIN_FIELD_BASE(167, 167, 4, 0x0020, 0x10, 4, 1),
+ PIN_FIELD_BASE(168, 168, 4, 0x0020, 0x10, 5, 1),
+ PIN_FIELD_BASE(169, 169, 4, 0x0020, 0x10, 6, 1),
+ PIN_FIELD_BASE(170, 170, 4, 0x0020, 0x10, 7, 1),
+ PIN_FIELD_BASE(171, 171, 7, 0x0060, 0x10, 17, 1),
+ PIN_FIELD_BASE(172, 172, 7, 0x0060, 0x10, 18, 1),
+ PIN_FIELD_BASE(173, 173, 7, 0x0060, 0x10, 11, 1),
+ PIN_FIELD_BASE(174, 174, 7, 0x0060, 0x10, 12, 1),
+ PIN_FIELD_BASE(175, 175, 7, 0x0060, 0x10, 13, 1),
+ PIN_FIELD_BASE(176, 176, 7, 0x0060, 0x10, 14, 1),
+ PIN_FIELD_BASE(177, 177, 7, 0x0060, 0x10, 15, 1),
+ PINS_FIELD_BASE(178, 179, 7, 0x0060, 0x10, 16, 1),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_pu_range[] = {
+ PIN_FIELD_BASE(0, 0, 2, 0x0060, 0x10, 6, 1),
+ PIN_FIELD_BASE(1, 1, 2, 0x0060, 0x10, 7, 1),
+ PIN_FIELD_BASE(2, 2, 2, 0x0060, 0x10, 10, 1),
+ PIN_FIELD_BASE(3, 3, 2, 0x0060, 0x10, 11, 1),
+ PIN_FIELD_BASE(4, 4, 2, 0x0060, 0x10, 12, 1),
+ PIN_FIELD_BASE(5, 5, 2, 0x0060, 0x10, 13, 1),
+ PIN_FIELD_BASE(6, 6, 2, 0x0060, 0x10, 14, 1),
+ PIN_FIELD_BASE(7, 7, 2, 0x0060, 0x10, 15, 1),
+ PIN_FIELD_BASE(8, 8, 3, 0x0050, 0x10, 12, 1),
+ PIN_FIELD_BASE(9, 9, 2, 0x0060, 0x10, 16, 1),
+ PIN_FIELD_BASE(10, 10, 2, 0x0060, 0x10, 8, 1),
+ PIN_FIELD_BASE(11, 11, 2, 0x0060, 0x10, 9, 1),
+ PIN_FIELD_BASE(12, 12, 5, 0x0040, 0x10, 9, 1),
+ PIN_FIELD_BASE(13, 13, 6, 0x0060, 0x10, 14, 1),
+ PIN_FIELD_BASE(14, 14, 6, 0x0060, 0x10, 13, 1),
+ PIN_FIELD_BASE(15, 15, 6, 0x0060, 0x10, 15, 1),
+ PIN_FIELD_BASE(16, 16, 6, 0x0060, 0x10, 12, 1),
+ PIN_FIELD_BASE(17, 17, 6, 0x0060, 0x10, 7, 1),
+ PIN_FIELD_BASE(18, 18, 6, 0x0060, 0x10, 4, 1),
+ PIN_FIELD_BASE(19, 19, 6, 0x0060, 0x10, 6, 1),
+ PIN_FIELD_BASE(20, 20, 6, 0x0060, 0x10, 5, 1),
+ PIN_FIELD_BASE(21, 21, 6, 0x0060, 0x10, 10, 1),
+ PIN_FIELD_BASE(22, 22, 6, 0x0060, 0x10, 9, 1),
+ PIN_FIELD_BASE(23, 23, 6, 0x0060, 0x10, 11, 1),
+ PIN_FIELD_BASE(24, 24, 6, 0x0060, 0x10, 8, 1),
+ PIN_FIELD_BASE(25, 25, 6, 0x0060, 0x10, 2, 1),
+ PIN_FIELD_BASE(26, 26, 6, 0x0060, 0x10, 1, 1),
+ PIN_FIELD_BASE(27, 27, 6, 0x0060, 0x10, 3, 1),
+ PINS_FIELD_BASE(28, 40, 6, 0x0060, 0x10, 0, 1),
+ PIN_FIELD_BASE(41, 41, 7, 0x0080, 0x10, 19, 1),
+ PIN_FIELD_BASE(42, 42, 7, 0x0080, 0x10, 9, 1),
+ PIN_FIELD_BASE(43, 43, 7, 0x0080, 0x10, 8, 1),
+ PIN_FIELD_BASE(44, 44, 7, 0x0080, 0x10, 10, 1),
+ PIN_FIELD_BASE(45, 45, 7, 0x0080, 0x10, 22, 1),
+ PIN_FIELD_BASE(46, 46, 7, 0x0080, 0x10, 21, 1),
+ PIN_FIELD_BASE(47, 47, 7, 0x0080, 0x10, 20, 1),
+ PIN_FIELD_BASE(48, 48, 7, 0x0090, 0x10, 3, 1),
+ PIN_FIELD_BASE(49, 49, 7, 0x0090, 0x10, 5, 1),
+ PIN_FIELD_BASE(50, 50, 7, 0x0090, 0x10, 2, 1),
+ PIN_FIELD_BASE(51, 51, 7, 0x0090, 0x10, 4, 1),
+ PIN_FIELD_BASE(52, 52, 7, 0x0080, 0x10, 1, 1),
+ PIN_FIELD_BASE(53, 53, 7, 0x0080, 0x10, 0, 1),
+ PIN_FIELD_BASE(54, 54, 7, 0x0080, 0x10, 5, 1),
+ PIN_FIELD_BASE(55, 55, 7, 0x0080, 0x10, 3, 1),
+ PIN_FIELD_BASE(56, 56, 7, 0x0080, 0x10, 4, 1),
+ PIN_FIELD_BASE(57, 57, 7, 0x0080, 0x10, 2, 1),
+ PIN_FIELD_BASE(58, 58, 7, 0x0090, 0x10, 0, 1),
+ PIN_FIELD_BASE(59, 59, 7, 0x0080, 0x10, 31, 1),
+ PIN_FIELD_BASE(60, 60, 7, 0x0080, 0x10, 30, 1),
+ PIN_FIELD_BASE(61, 61, 3, 0x0050, 0x10, 18, 1),
+ PIN_FIELD_BASE(62, 62, 3, 0x0050, 0x10, 14, 1),
+ PIN_FIELD_BASE(63, 63, 3, 0x0050, 0x10, 17, 1),
+ PIN_FIELD_BASE(64, 64, 3, 0x0050, 0x10, 13, 1),
+ PIN_FIELD_BASE(65, 65, 3, 0x0050, 0x10, 20, 1),
+ PIN_FIELD_BASE(66, 66, 3, 0x0050, 0x10, 16, 1),
+ PIN_FIELD_BASE(67, 67, 3, 0x0050, 0x10, 19, 1),
+ PIN_FIELD_BASE(68, 68, 3, 0x0050, 0x10, 15, 1),
+ PIN_FIELD_BASE(69, 69, 3, 0x0050, 0x10, 8, 1),
+ PIN_FIELD_BASE(70, 70, 3, 0x0050, 0x10, 7, 1),
+ PIN_FIELD_BASE(71, 71, 3, 0x0050, 0x10, 6, 1),
+ PIN_FIELD_BASE(72, 72, 3, 0x0050, 0x10, 5, 1),
+ PIN_FIELD_BASE(73, 73, 3, 0x0050, 0x10, 4, 1),
+ PIN_FIELD_BASE(74, 74, 3, 0x0050, 0x10, 3, 1),
+ PIN_FIELD_BASE(75, 75, 3, 0x0050, 0x10, 2, 1),
+ PIN_FIELD_BASE(76, 76, 3, 0x0050, 0x10, 1, 1),
+ PIN_FIELD_BASE(77, 77, 3, 0x0050, 0x10, 0, 1),
+ PIN_FIELD_BASE(78, 78, 3, 0x0050, 0x10, 9, 1),
+ PIN_FIELD_BASE(79, 79, 3, 0x0050, 0x10, 11, 1),
+ PIN_FIELD_BASE(80, 80, 3, 0x0050, 0x10, 10, 1),
+ PIN_FIELD_BASE(81, 81, 3, 0x0050, 0x10, 25, 1),
+ PIN_FIELD_BASE(82, 82, 3, 0x0050, 0x10, 24, 1),
+ PIN_FIELD_BASE(83, 83, 3, 0x0050, 0x10, 22, 1),
+ PIN_FIELD_BASE(84, 84, 3, 0x0050, 0x10, 23, 1),
+ PIN_FIELD_BASE(85, 85, 7, 0x0090, 0x10, 1, 1),
+ PIN_FIELD_BASE(86, 86, 7, 0x0080, 0x10, 29, 1),
+ PIN_FIELD_BASE(87, 87, 7, 0x0080, 0x10, 7, 1),
+ PIN_FIELD_BASE(88, 88, 7, 0x0080, 0x10, 6, 1),
+ PIN_FIELD_BASE(89, 89, 2, 0x0060, 0x10, 21, 1),
+ PINS_FIELD_BASE(90, 94, 3, 0x0050, 0x10, 21, 1),
+ PIN_FIELD_BASE(95, 95, 2, 0x0060, 0x10, 22, 1),
+ PIN_FIELD_BASE(96, 96, 2, 0x0060, 0x10, 23, 1),
+ PIN_FIELD_BASE(97, 97, 2, 0x0060, 0x10, 2, 1),
+ PIN_FIELD_BASE(98, 98, 2, 0x0060, 0x10, 3, 1),
+ PIN_FIELD_BASE(99, 99, 2, 0x0060, 0x10, 0, 1),
+ PIN_FIELD_BASE(100, 100, 2, 0x0060, 0x10, 1, 1),
+ PIN_FIELD_BASE(101, 101, 2, 0x0060, 0x10, 4, 1),
+ PIN_FIELD_BASE(102, 102, 2, 0x0060, 0x10, 5, 1),
+ PIN_FIELD_BASE(103, 103, 2, 0x0060, 0x10, 17, 1),
+ PIN_FIELD_BASE(104, 104, 2, 0x0060, 0x10, 19, 1),
+ PIN_FIELD_BASE(105, 105, 2, 0x0060, 0x10, 18, 1),
+ PIN_FIELD_BASE(106, 106, 2, 0x0060, 0x10, 20, 1),
+ PIN_FIELD_BASE(107, 107, 1, 0x0050, 0x10, 4, 1),
+ PIN_FIELD_BASE(108, 108, 1, 0x0050, 0x10, 3, 1),
+ PIN_FIELD_BASE(109, 109, 1, 0x0050, 0x10, 5, 1),
+ PIN_FIELD_BASE(110, 110, 1, 0x0050, 0x10, 0, 1),
+ PIN_FIELD_BASE(111, 111, 1, 0x0050, 0x10, 1, 1),
+ PIN_FIELD_BASE(112, 112, 1, 0x0050, 0x10, 2, 1),
+ PIN_FIELD_BASE(113, 113, 1, 0x0050, 0x10, 9, 1),
+ PIN_FIELD_BASE(114, 114, 1, 0x0050, 0x10, 10, 1),
+ PIN_FIELD_BASE(115, 115, 1, 0x0050, 0x10, 6, 1),
+ PIN_FIELD_BASE(116, 116, 1, 0x0050, 0x10, 7, 1),
+ PIN_FIELD_BASE(117, 117, 1, 0x0050, 0x10, 12, 1),
+ PIN_FIELD_BASE(118, 118, 1, 0x0050, 0x10, 13, 1),
+ PIN_FIELD_BASE(119, 119, 1, 0x0050, 0x10, 14, 1),
+ PIN_FIELD_BASE(120, 120, 1, 0x0050, 0x10, 11, 1),
+ PINS_FIELD_BASE(121, 133, 1, 0x0050, 0x10, 8, 1),
+ PIN_FIELD_BASE(134, 134, 5, 0x0040, 0x10, 14, 1),
+ PIN_FIELD_BASE(135, 135, 5, 0x0040, 0x10, 19, 1),
+ PIN_FIELD_BASE(136, 136, 5, 0x0040, 0x10, 1, 1),
+ PIN_FIELD_BASE(137, 137, 5, 0x0040, 0x10, 7, 1),
+ PIN_FIELD_BASE(138, 138, 5, 0x0040, 0x10, 4, 1),
+ PIN_FIELD_BASE(139, 139, 5, 0x0040, 0x10, 5, 1),
+ PIN_FIELD_BASE(140, 140, 5, 0x0040, 0x10, 0, 1),
+ PIN_FIELD_BASE(141, 141, 5, 0x0040, 0x10, 6, 1),
+ PIN_FIELD_BASE(142, 142, 5, 0x0040, 0x10, 2, 1),
+ PIN_FIELD_BASE(143, 143, 5, 0x0040, 0x10, 3, 1),
+ PIN_FIELD_BASE(144, 144, 5, 0x0040, 0x10, 12, 1),
+ PIN_FIELD_BASE(145, 145, 5, 0x0040, 0x10, 11, 1),
+ PIN_FIELD_BASE(146, 146, 5, 0x0040, 0x10, 13, 1),
+ PIN_FIELD_BASE(147, 147, 5, 0x0040, 0x10, 10, 1),
+ PIN_FIELD_BASE(148, 148, 5, 0x0040, 0x10, 15, 1),
+ PIN_FIELD_BASE(149, 149, 5, 0x0040, 0x10, 16, 1),
+ PIN_FIELD_BASE(150, 150, 7, 0x0080, 0x10, 23, 1),
+ PIN_FIELD_BASE(151, 151, 7, 0x0080, 0x10, 24, 1),
+ PIN_FIELD_BASE(152, 152, 7, 0x0080, 0x10, 25, 1),
+ PIN_FIELD_BASE(153, 153, 7, 0x0080, 0x10, 26, 1),
+ PIN_FIELD_BASE(154, 154, 7, 0x0080, 0x10, 28, 1),
+ PIN_FIELD_BASE(155, 155, 3, 0x0050, 0x10, 28, 1),
+ PIN_FIELD_BASE(156, 156, 3, 0x0050, 0x10, 27, 1),
+ PIN_FIELD_BASE(157, 157, 3, 0x0050, 0x10, 29, 1),
+ PIN_FIELD_BASE(158, 158, 3, 0x0050, 0x10, 26, 1),
+ PIN_FIELD_BASE(159, 159, 7, 0x0080, 0x10, 27, 1),
+ PIN_FIELD_BASE(160, 160, 5, 0x0040, 0x10, 8, 1),
+ PIN_FIELD_BASE(161, 161, 1, 0x0050, 0x10, 15, 1),
+ PIN_FIELD_BASE(162, 162, 1, 0x0050, 0x10, 16, 1),
+ PIN_FIELD_BASE(163, 163, 4, 0x0040, 0x10, 0, 1),
+ PIN_FIELD_BASE(164, 164, 4, 0x0040, 0x10, 1, 1),
+ PIN_FIELD_BASE(165, 165, 4, 0x0040, 0x10, 2, 1),
+ PIN_FIELD_BASE(166, 166, 4, 0x0040, 0x10, 3, 1),
+ PIN_FIELD_BASE(167, 167, 4, 0x0040, 0x10, 4, 1),
+ PIN_FIELD_BASE(168, 168, 4, 0x0040, 0x10, 5, 1),
+ PIN_FIELD_BASE(169, 169, 4, 0x0040, 0x10, 6, 1),
+ PIN_FIELD_BASE(170, 170, 4, 0x0040, 0x10, 7, 1),
+ PIN_FIELD_BASE(171, 171, 7, 0x0080, 0x10, 17, 1),
+ PIN_FIELD_BASE(172, 172, 7, 0x0080, 0x10, 18, 1),
+ PIN_FIELD_BASE(173, 173, 7, 0x0080, 0x10, 11, 1),
+ PIN_FIELD_BASE(174, 174, 7, 0x0080, 0x10, 12, 1),
+ PIN_FIELD_BASE(175, 175, 7, 0x0080, 0x10, 13, 1),
+ PIN_FIELD_BASE(176, 176, 7, 0x0080, 0x10, 14, 1),
+ PIN_FIELD_BASE(177, 177, 7, 0x0080, 0x10, 15, 1),
+ PINS_FIELD_BASE(178, 179, 7, 0x0080, 0x10, 16, 1),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_tdsel_range[] = {
+ PINS_FIELD_BASE(0, 3, 2, 0x00c0, 0x10, 16, 4),
+ PINS_FIELD_BASE(4, 7, 2, 0x00c0, 0x10, 20, 4),
+ PIN_FIELD_BASE(8, 8, 3, 0x0090, 0x10, 12, 4),
+ PINS_FIELD_BASE(9, 11, 2, 0x00c0, 0x10, 24, 4),
+ PIN_FIELD_BASE(12, 12, 5, 0x0080, 0x10, 4, 4),
+ PINS_FIELD_BASE(13, 16, 6, 0x00e0, 0x10, 8, 4),
+ PINS_FIELD_BASE(17, 20, 6, 0x00e0, 0x10, 0, 4),
+ PINS_FIELD_BASE(21, 24, 6, 0x00e0, 0x10, 4, 4),
+ PINS_FIELD_BASE(25, 28, 6, 0x00d0, 0x10, 28, 4),
+ PIN_FIELD_BASE(29, 29, 6, 0x00d0, 0x10, 0, 4),
+ PIN_FIELD_BASE(30, 30, 6, 0x00d0, 0x10, 4, 4),
+ PINS_FIELD_BASE(31, 34, 6, 0x00d0, 0x10, 8, 4),
+ PINS_FIELD_BASE(35, 36, 6, 0x00d0, 0x10, 20, 4),
+ PIN_FIELD_BASE(37, 37, 6, 0x00d0, 0x10, 24, 4),
+ PIN_FIELD_BASE(38, 38, 6, 0x00d0, 0x10, 16, 4),
+ PINS_FIELD_BASE(39, 40, 6, 0x00d0, 0x10, 12, 4),
+ PINS_FIELD_BASE(41, 42, 7, 0x00d0, 0x10, 24, 4),
+ PIN_FIELD_BASE(43, 43, 7, 0x00d0, 0x10, 12, 4),
+ PIN_FIELD_BASE(44, 44, 7, 0x00d0, 0x10, 16, 4),
+ PIN_FIELD_BASE(45, 45, 7, 0x00e0, 0x10, 0, 4),
+ PINS_FIELD_BASE(46, 47, 7, 0x00d0, 0x10, 28, 4),
+ PINS_FIELD_BASE(48, 49, 7, 0x00e0, 0x10, 28, 4),
+ PINS_FIELD_BASE(50, 51, 7, 0x00e0, 0x10, 24, 4),
+ PINS_FIELD_BASE(52, 57, 7, 0x00d0, 0x10, 0, 4),
+ PINS_FIELD_BASE(58, 60, 7, 0x00e0, 0x10, 16, 4),
+ PINS_FIELD_BASE(61, 62, 3, 0x0090, 0x10, 20, 4),
+ PINS_FIELD_BASE(63, 64, 3, 0x0090, 0x10, 16, 4),
+ PINS_FIELD_BASE(65, 66, 3, 0x0090, 0x10, 28, 4),
+ PINS_FIELD_BASE(67, 68, 3, 0x0090, 0x10, 24, 4),
+ PINS_FIELD_BASE(69, 73, 3, 0x0090, 0x10, 4, 4),
+ PINS_FIELD_BASE(74, 78, 3, 0x0090, 0x10, 8, 4),
+ PINS_FIELD_BASE(79, 80, 3, 0x0090, 0x10, 0, 4),
+ PIN_FIELD_BASE(81, 81, 3, 0x00a0, 0x10, 8, 4),
+ PINS_FIELD_BASE(82, 83, 3, 0x00a0, 0x10, 4, 4),
+ PIN_FIELD_BASE(84, 84, 3, 0x00a0, 0x10, 8, 4),
+ PIN_FIELD_BASE(85, 85, 7, 0x00e0, 0x10, 16, 4),
+ PIN_FIELD_BASE(86, 86, 7, 0x00e0, 0x10, 20, 4),
+ PIN_FIELD_BASE(87, 87, 7, 0x00d0, 0x10, 8, 4),
+ PIN_FIELD_BASE(88, 88, 7, 0x00d0, 0x10, 4, 4),
+ PIN_FIELD_BASE(89, 89, 2, 0x00d0, 0x10, 12, 4),
+ PIN_FIELD_BASE(90, 90, 3, 0x00a0, 0x10, 0, 4),
+ PINS_FIELD_BASE(91, 92, 2, 0x00d0, 0x10, 0, 4),
+ PINS_FIELD_BASE(93, 94, 2, 0x00c0, 0x10, 28, 4),
+ PINS_FIELD_BASE(95, 96, 2, 0x00d0, 0x10, 16, 4),
+ PINS_FIELD_BASE(97, 98, 2, 0x00c0, 0x10, 8, 4),
+ PIN_FIELD_BASE(99, 99, 2, 0x00c0, 0x10, 0, 4),
+ PIN_FIELD_BASE(100, 100, 2, 0x00c0, 0x10, 4, 4),
+ PINS_FIELD_BASE(101, 102, 2, 0x00c0, 0x10, 12, 4),
+ PINS_FIELD_BASE(103, 104, 2, 0x00d0, 0x10, 4, 4),
+ PINS_FIELD_BASE(105, 106, 2, 0x00d0, 0x10, 8, 4),
+ PIN_FIELD_BASE(107, 107, 1, 0x0090, 0x10, 16, 4),
+ PIN_FIELD_BASE(108, 108, 1, 0x0090, 0x10, 12, 4),
+ PIN_FIELD_BASE(109, 109, 1, 0x0090, 0x10, 20, 4),
+ PIN_FIELD_BASE(110, 110, 1, 0x0090, 0x10, 0, 4),
+ PIN_FIELD_BASE(111, 111, 1, 0x0090, 0x10, 4, 4),
+ PIN_FIELD_BASE(112, 112, 1, 0x0090, 0x10, 8, 4),
+ PIN_FIELD_BASE(113, 113, 1, 0x00a0, 0x10, 4, 4),
+ PIN_FIELD_BASE(114, 114, 1, 0x00a0, 0x10, 8, 4),
+ PIN_FIELD_BASE(115, 115, 1, 0x0090, 0x10, 24, 4),
+ PIN_FIELD_BASE(116, 116, 1, 0x0090, 0x10, 28, 4),
+ PIN_FIELD_BASE(117, 117, 1, 0x00a0, 0x10, 16, 4),
+ PIN_FIELD_BASE(118, 118, 1, 0x00a0, 0x10, 20, 4),
+ PIN_FIELD_BASE(119, 119, 1, 0x00a0, 0x10, 24, 4),
+ PIN_FIELD_BASE(120, 120, 1, 0x00a0, 0x10, 12, 4),
+ PIN_FIELD_BASE(121, 121, 1, 0x00a0, 0x10, 0, 4),
+ PIN_FIELD_BASE(122, 122, 4, 0x0090, 0x10, 8, 4),
+ PIN_FIELD_BASE(123, 123, 4, 0x0090, 0x10, 12, 4),
+ PIN_FIELD_BASE(124, 124, 4, 0x0090, 0x10, 4, 4),
+ PINS_FIELD_BASE(125, 130, 4, 0x0090, 0x10, 12, 4),
+ PIN_FIELD_BASE(131, 131, 4, 0x0090, 0x10, 16, 4),
+ PIN_FIELD_BASE(132, 132, 4, 0x0090, 0x10, 12, 4),
+ PIN_FIELD_BASE(133, 133, 4, 0x0090, 0x10, 20, 4),
+ PIN_FIELD_BASE(134, 134, 5, 0x0080, 0x10, 12, 4),
+ PIN_FIELD_BASE(135, 135, 5, 0x0080, 0x10, 20, 4),
+ PIN_FIELD_BASE(136, 136, 5, 0x0070, 0x10, 4, 4),
+ PIN_FIELD_BASE(137, 137, 5, 0x0070, 0x10, 28, 4),
+ PIN_FIELD_BASE(138, 138, 5, 0x0070, 0x10, 16, 4),
+ PIN_FIELD_BASE(139, 139, 5, 0x0070, 0x10, 20, 4),
+ PIN_FIELD_BASE(140, 140, 5, 0x0070, 0x10, 0, 4),
+ PIN_FIELD_BASE(141, 141, 5, 0x0070, 0x10, 24, 4),
+ PIN_FIELD_BASE(142, 142, 5, 0x0070, 0x10, 8, 4),
+ PIN_FIELD_BASE(143, 143, 5, 0x0070, 0x10, 12, 4),
+ PINS_FIELD_BASE(144, 147, 5, 0x0080, 0x10, 8, 4),
+ PINS_FIELD_BASE(148, 149, 5, 0x0080, 0x10, 16, 4),
+ PINS_FIELD_BASE(150, 151, 7, 0x00e0, 0x10, 4, 4),
+ PINS_FIELD_BASE(152, 153, 7, 0x00e0, 0x10, 8, 4),
+ PIN_FIELD_BASE(154, 154, 7, 0x00e0, 0x10, 12, 4),
+ PINS_FIELD_BASE(155, 158, 3, 0x00a0, 0x10, 12, 4),
+ PIN_FIELD_BASE(159, 159, 7, 0x00e0, 0x10, 12, 4),
+ PIN_FIELD_BASE(160, 160, 5, 0x0080, 0x10, 0, 4),
+ PINS_FIELD_BASE(161, 162, 1, 0x00a0, 0x10, 28, 4),
+ PINS_FIELD_BASE(163, 170, 4, 0x0090, 0x10, 0, 4),
+ PINS_FIELD_BASE(171, 179, 7, 0x00d0, 0x10, 20, 4),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_rdsel_range[] = {
+ PINS_FIELD_BASE(0, 3, 2, 0x0090, 0x10, 8, 2),
+ PINS_FIELD_BASE(4, 7, 2, 0x0090, 0x10, 10, 2),
+ PIN_FIELD_BASE(8, 8, 3, 0x0060, 0x10, 6, 2),
+ PINS_FIELD_BASE(9, 11, 2, 0x0090, 0x10, 12, 2),
+ PIN_FIELD_BASE(12, 12, 5, 0x0050, 0x10, 18, 2),
+ PINS_FIELD_BASE(13, 16, 6, 0x00a0, 0x10, 18, 2),
+ PINS_FIELD_BASE(17, 20, 6, 0x00a0, 0x10, 14, 2),
+ PINS_FIELD_BASE(21, 24, 6, 0x00a0, 0x10, 16, 2),
+ PINS_FIELD_BASE(25, 28, 6, 0x00a0, 0x10, 12, 2),
+ PIN_FIELD_BASE(29, 29, 6, 0x0090, 0x10, 0, 6),
+ PIN_FIELD_BASE(30, 30, 6, 0x0090, 0x10, 6, 6),
+ PINS_FIELD_BASE(31, 34, 6, 0x0090, 0x10, 12, 6),
+ PINS_FIELD_BASE(35, 36, 6, 0x00a0, 0x10, 0, 6),
+ PIN_FIELD_BASE(37, 37, 6, 0x00a0, 0x10, 6, 6),
+ PIN_FIELD_BASE(38, 38, 6, 0x0090, 0x10, 24, 6),
+ PINS_FIELD_BASE(39, 40, 6, 0x0090, 0x10, 18, 6),
+ PINS_FIELD_BASE(41, 42, 7, 0x00a0, 0x10, 12, 2),
+ PIN_FIELD_BASE(43, 43, 7, 0x00a0, 0x10, 6, 2),
+ PIN_FIELD_BASE(44, 44, 7, 0x00a0, 0x10, 8, 2),
+ PIN_FIELD_BASE(45, 45, 7, 0x00a0, 0x10, 16, 2),
+ PINS_FIELD_BASE(46, 47, 7, 0x00a0, 0x10, 14, 2),
+ PINS_FIELD_BASE(48, 49, 7, 0x00a0, 0x10, 30, 2),
+ PINS_FIELD_BASE(50, 51, 7, 0x00a0, 0x10, 28, 2),
+ PINS_FIELD_BASE(52, 57, 7, 0x00a0, 0x10, 0, 2),
+ PINS_FIELD_BASE(58, 60, 7, 0x00a0, 0x10, 24, 2),
+ PINS_FIELD_BASE(61, 62, 3, 0x0060, 0x10, 10, 2),
+ PINS_FIELD_BASE(63, 64, 3, 0x0060, 0x10, 8, 2),
+ PINS_FIELD_BASE(65, 66, 3, 0x0060, 0x10, 14, 2),
+ PINS_FIELD_BASE(67, 68, 3, 0x0060, 0x10, 12, 2),
+ PINS_FIELD_BASE(69, 73, 3, 0x0060, 0x10, 2, 2),
+ PINS_FIELD_BASE(74, 78, 3, 0x0060, 0x10, 4, 2),
+ PINS_FIELD_BASE(79, 80, 3, 0x0060, 0x10, 0, 2),
+ PIN_FIELD_BASE(81, 81, 3, 0x0060, 0x10, 20, 2),
+ PINS_FIELD_BASE(82, 83, 3, 0x0060, 0x10, 18, 2),
+ PIN_FIELD_BASE(84, 84, 3, 0x0060, 0x10, 20, 2),
+ PIN_FIELD_BASE(85, 85, 7, 0x00a0, 0x10, 24, 2),
+ PIN_FIELD_BASE(86, 86, 7, 0x00a0, 0x10, 26, 2),
+ PIN_FIELD_BASE(87, 87, 7, 0x00a0, 0x10, 4, 2),
+ PIN_FIELD_BASE(88, 88, 7, 0x00a0, 0x10, 2, 2),
+ PIN_FIELD_BASE(89, 89, 2, 0x0090, 0x10, 22, 2),
+ PIN_FIELD_BASE(90, 90, 3, 0x0060, 0x10, 16, 2),
+ PINS_FIELD_BASE(91, 92, 2, 0x0090, 0x10, 16, 2),
+ PINS_FIELD_BASE(93, 94, 2, 0x0090, 0x10, 14, 2),
+ PINS_FIELD_BASE(95, 96, 2, 0x0090, 0x10, 24, 2),
+ PINS_FIELD_BASE(97, 98, 2, 0x0090, 0x10, 4, 2),
+ PIN_FIELD_BASE(99, 99, 2, 0x0090, 0x10, 0, 2),
+ PIN_FIELD_BASE(100, 100, 2, 0x0090, 0x10, 2, 2),
+ PINS_FIELD_BASE(101, 102, 2, 0x0090, 0x10, 6, 2),
+ PINS_FIELD_BASE(103, 104, 2, 0x0090, 0x10, 18, 2),
+ PINS_FIELD_BASE(105, 106, 2, 0x0090, 0x10, 20, 2),
+ PIN_FIELD_BASE(107, 107, 1, 0x0060, 0x10, 8, 2),
+ PIN_FIELD_BASE(108, 108, 1, 0x0060, 0x10, 6, 2),
+ PIN_FIELD_BASE(109, 109, 1, 0x0060, 0x10, 10, 2),
+ PIN_FIELD_BASE(110, 110, 1, 0x0060, 0x10, 0, 2),
+ PIN_FIELD_BASE(111, 111, 1, 0x0060, 0x10, 2, 2),
+ PIN_FIELD_BASE(112, 112, 1, 0x0060, 0x10, 4, 2),
+ PIN_FIELD_BASE(113, 113, 1, 0x0060, 0x10, 18, 2),
+ PIN_FIELD_BASE(114, 114, 1, 0x0060, 0x10, 20, 2),
+ PIN_FIELD_BASE(115, 115, 1, 0x0060, 0x10, 12, 2),
+ PIN_FIELD_BASE(116, 116, 1, 0x0060, 0x10, 14, 2),
+ PIN_FIELD_BASE(117, 117, 1, 0x0060, 0x10, 24, 2),
+ PIN_FIELD_BASE(118, 118, 1, 0x0060, 0x10, 26, 2),
+ PIN_FIELD_BASE(119, 119, 1, 0x0060, 0x10, 28, 2),
+ PIN_FIELD_BASE(120, 120, 1, 0x0060, 0x10, 22, 2),
+ PIN_FIELD_BASE(121, 121, 1, 0x0060, 0x10, 16, 2),
+ PIN_FIELD_BASE(122, 122, 4, 0x0070, 0x10, 8, 6),
+ PIN_FIELD_BASE(123, 123, 4, 0x0070, 0x10, 14, 6),
+ PIN_FIELD_BASE(124, 124, 4, 0x0070, 0x10, 2, 6),
+ PINS_FIELD_BASE(125, 130, 4, 0x0070, 0x10, 14, 6),
+ PIN_FIELD_BASE(131, 131, 4, 0x0070, 0x10, 20, 6),
+ PIN_FIELD_BASE(132, 132, 4, 0x0070, 0x10, 14, 6),
+ PIN_FIELD_BASE(133, 133, 4, 0x0070, 0x10, 26, 6),
+ PIN_FIELD_BASE(134, 134, 5, 0x0050, 0x10, 22, 2),
+ PIN_FIELD_BASE(135, 135, 5, 0x0050, 0x10, 30, 2),
+ PIN_FIELD_BASE(136, 136, 5, 0x0050, 0x10, 2, 2),
+ PIN_FIELD_BASE(137, 137, 5, 0x0050, 0x10, 14, 2),
+ PIN_FIELD_BASE(138, 138, 5, 0x0050, 0x10, 8, 2),
+ PIN_FIELD_BASE(139, 139, 5, 0x0050, 0x10, 10, 2),
+ PIN_FIELD_BASE(140, 140, 5, 0x0050, 0x10, 0, 2),
+ PIN_FIELD_BASE(141, 141, 5, 0x0050, 0x10, 12, 2),
+ PIN_FIELD_BASE(142, 142, 5, 0x0050, 0x10, 4, 2),
+ PIN_FIELD_BASE(143, 143, 5, 0x0050, 0x10, 6, 2),
+ PINS_FIELD_BASE(144, 147, 5, 0x0050, 0x10, 20, 2),
+ PINS_FIELD_BASE(148, 149, 5, 0x0050, 0x10, 24, 2),
+ PINS_FIELD_BASE(150, 151, 7, 0x00a0, 0x10, 18, 2),
+ PINS_FIELD_BASE(152, 153, 7, 0x00a0, 0x10, 20, 2),
+ PIN_FIELD_BASE(154, 154, 7, 0x00a0, 0x10, 22, 2),
+ PINS_FIELD_BASE(155, 158, 3, 0x0060, 0x10, 22, 2),
+ PIN_FIELD_BASE(159, 159, 7, 0x00a0, 0x10, 22, 2),
+ PIN_FIELD_BASE(160, 160, 5, 0x0050, 0x10, 16, 2),
+ PINS_FIELD_BASE(161, 162, 1, 0x0060, 0x10, 30, 2),
+ PINS_FIELD_BASE(163, 170, 4, 0x0070, 0x10, 0, 2),
+ PINS_FIELD_BASE(171, 179, 7, 0x00a0, 0x10, 10, 2),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_drv_range[] = {
+ PINS_FIELD_BASE(0, 2, 2, 0x0000, 0x10, 12, 3),
+ PIN_FIELD_BASE(3, 3, 2, 0x0000, 0x10, 15, 3),
+ PINS_FIELD_BASE(4, 6, 2, 0x0000, 0x10, 18, 3),
+ PIN_FIELD_BASE(7, 7, 2, 0x0000, 0x10, 21, 3),
+ PIN_FIELD_BASE(8, 8, 3, 0x0000, 0x10, 9, 3),
+ PINS_FIELD_BASE(9, 11, 2, 0x0000, 0x10, 24, 3),
+ PIN_FIELD_BASE(12, 12, 5, 0x0000, 0x10, 27, 3),
+ PINS_FIELD_BASE(13, 15, 6, 0x0010, 0x10, 3, 3),
+ PIN_FIELD_BASE(16, 16, 6, 0x0010, 0x10, 6, 3),
+ PIN_FIELD_BASE(17, 17, 6, 0x0000, 0x10, 23, 3),
+ PIN_FIELD_BASE(18, 18, 6, 0x0000, 0x10, 26, 3),
+ PINS_FIELD_BASE(19, 20, 6, 0x0000, 0x10, 23, 3),
+ PINS_FIELD_BASE(21, 23, 6, 0x0000, 0x10, 29, 3),
+ PIN_FIELD_BASE(24, 24, 6, 0x0010, 0x10, 0, 3),
+ PINS_FIELD_BASE(25, 27, 6, 0x0000, 0x10, 17, 3),
+ PIN_FIELD_BASE(28, 28, 6, 0x0000, 0x10, 20, 3),
+ PIN_FIELD_BASE(29, 29, 6, 0x0000, 0x10, 0, 3),
+ PIN_FIELD_BASE(30, 30, 6, 0x0000, 0x10, 3, 3),
+ PINS_FIELD_BASE(31, 34, 6, 0x0000, 0x10, 6, 3),
+ PINS_FIELD_BASE(35, 36, 6, 0x0000, 0x10, 13, 2),
+ PIN_FIELD_BASE(37, 37, 6, 0x0000, 0x10, 15, 2),
+ PIN_FIELD_BASE(38, 38, 6, 0x0000, 0x10, 11, 2),
+ PINS_FIELD_BASE(39, 40, 6, 0x0000, 0x10, 9, 2),
+ PINS_FIELD_BASE(41, 42, 7, 0x0000, 0x10, 21, 3),
+ PIN_FIELD_BASE(43, 43, 7, 0x0000, 0x10, 9, 3),
+ PIN_FIELD_BASE(44, 44, 7, 0x0000, 0x10, 12, 3),
+ PIN_FIELD_BASE(45, 45, 7, 0x0000, 0x10, 27, 3),
+ PINS_FIELD_BASE(46, 47, 7, 0x0000, 0x10, 24, 3),
+ PINS_FIELD_BASE(48, 49, 7, 0x0010, 0x10, 18, 3),
+ PINS_FIELD_BASE(50, 51, 7, 0x0010, 0x10, 15, 3),
+ PINS_FIELD_BASE(52, 57, 7, 0x0000, 0x10, 0, 3),
+ PINS_FIELD_BASE(58, 60, 7, 0x0010, 0x10, 9, 3),
+ PINS_FIELD_BASE(61, 62, 3, 0x0000, 0x10, 15, 3),
+ PINS_FIELD_BASE(63, 64, 3, 0x0000, 0x10, 12, 3),
+ PINS_FIELD_BASE(65, 66, 3, 0x0000, 0x10, 21, 3),
+ PINS_FIELD_BASE(67, 68, 3, 0x0000, 0x10, 18, 3),
+ PINS_FIELD_BASE(69, 73, 3, 0x0000, 0x10, 3, 3),
+ PINS_FIELD_BASE(74, 78, 3, 0x0000, 0x10, 6, 3),
+ PINS_FIELD_BASE(79, 80, 3, 0x0000, 0x10, 0, 3),
+ PIN_FIELD_BASE(81, 81, 3, 0x0010, 0x10, 0, 3),
+ PINS_FIELD_BASE(82, 83, 3, 0x0000, 0x10, 27, 3),
+ PIN_FIELD_BASE(84, 84, 3, 0x0010, 0x10, 0, 3),
+ PIN_FIELD_BASE(85, 85, 7, 0x0010, 0x10, 9, 3),
+ PIN_FIELD_BASE(86, 86, 7, 0x0010, 0x10, 12, 3),
+ PIN_FIELD_BASE(87, 87, 7, 0x0000, 0x10, 6, 3),
+ PIN_FIELD_BASE(88, 88, 7, 0x0000, 0x10, 3, 3),
+ PIN_FIELD_BASE(89, 89, 2, 0x0010, 0x10, 15, 3),
+ PIN_FIELD_BASE(90, 90, 3, 0x0000, 0x10, 24, 3),
+ PIN_FIELD_BASE(91, 91, 2, 0x0010, 0x10, 6, 3),
+ PIN_FIELD_BASE(92, 92, 2, 0x0010, 0x10, 3, 3),
+ PIN_FIELD_BASE(93, 93, 2, 0x0000, 0x10, 27, 3),
+ PIN_FIELD_BASE(94, 94, 2, 0x0010, 0x10, 0, 3),
+ PINS_FIELD_BASE(95, 96, 2, 0x0010, 0x10, 18, 3),
+ PINS_FIELD_BASE(97, 98, 2, 0x0000, 0x10, 6, 3),
+ PIN_FIELD_BASE(99, 99, 2, 0x0000, 0x10, 0, 3),
+ PIN_FIELD_BASE(100, 100, 2, 0x0000, 0x10, 3, 3),
+ PINS_FIELD_BASE(101, 102, 2, 0x0000, 0x10, 9, 3),
+ PINS_FIELD_BASE(103, 104, 2, 0x0010, 0x10, 9, 3),
+ PINS_FIELD_BASE(105, 106, 2, 0x0010, 0x10, 12, 3),
+ PIN_FIELD_BASE(107, 107, 1, 0x0000, 0x10, 12, 3),
+ PIN_FIELD_BASE(108, 108, 1, 0x0000, 0x10, 9, 3),
+ PIN_FIELD_BASE(109, 109, 1, 0x0000, 0x10, 15, 3),
+ PIN_FIELD_BASE(110, 110, 1, 0x0000, 0x10, 0, 3),
+ PIN_FIELD_BASE(111, 111, 1, 0x0000, 0x10, 3, 3),
+ PIN_FIELD_BASE(112, 112, 1, 0x0000, 0x10, 6, 3),
+ PIN_FIELD_BASE(113, 113, 1, 0x0000, 0x10, 27, 3),
+ PIN_FIELD_BASE(114, 114, 1, 0x0010, 0x10, 0, 3),
+ PIN_FIELD_BASE(115, 115, 1, 0x0000, 0x10, 18, 3),
+ PIN_FIELD_BASE(116, 116, 1, 0x0000, 0x10, 21, 3),
+ PIN_FIELD_BASE(117, 117, 1, 0x0010, 0x10, 6, 3),
+ PIN_FIELD_BASE(118, 118, 1, 0x0010, 0x10, 9, 3),
+ PIN_FIELD_BASE(119, 119, 1, 0x0010, 0x10, 12, 3),
+ PIN_FIELD_BASE(120, 120, 1, 0x0010, 0x10, 3, 3),
+ PIN_FIELD_BASE(121, 121, 1, 0x0000, 0x10, 24, 3),
+ PIN_FIELD_BASE(122, 122, 4, 0x0000, 0x10, 9, 3),
+ PIN_FIELD_BASE(123, 123, 4, 0x0000, 0x10, 12, 3),
+ PIN_FIELD_BASE(124, 124, 4, 0x0000, 0x10, 6, 3),
+ PINS_FIELD_BASE(125, 130, 4, 0x0000, 0x10, 12, 3),
+ PIN_FIELD_BASE(131, 131, 4, 0x0000, 0x10, 15, 3),
+ PIN_FIELD_BASE(132, 132, 4, 0x0000, 0x10, 12, 3),
+ PIN_FIELD_BASE(133, 133, 4, 0x0000, 0x10, 18, 3),
+ PIN_FIELD_BASE(134, 134, 5, 0x0010, 0x10, 6, 3),
+ PIN_FIELD_BASE(135, 135, 5, 0x0010, 0x10, 12, 3),
+ PIN_FIELD_BASE(136, 136, 5, 0x0000, 0x10, 3, 3),
+ PIN_FIELD_BASE(137, 137, 5, 0x0000, 0x10, 21, 3),
+ PIN_FIELD_BASE(138, 138, 5, 0x0000, 0x10, 12, 3),
+ PIN_FIELD_BASE(139, 139, 5, 0x0000, 0x10, 15, 3),
+ PIN_FIELD_BASE(140, 140, 5, 0x0000, 0x10, 0, 3),
+ PIN_FIELD_BASE(141, 141, 5, 0x0000, 0x10, 18, 3),
+ PIN_FIELD_BASE(142, 142, 5, 0x0000, 0x10, 6, 3),
+ PIN_FIELD_BASE(143, 143, 5, 0x0000, 0x10, 9, 3),
+ PINS_FIELD_BASE(144, 146, 5, 0x0010, 0x10, 0, 3),
+ PIN_FIELD_BASE(147, 147, 5, 0x0010, 0x10, 3, 3),
+ PINS_FIELD_BASE(148, 149, 5, 0x0010, 0x10, 9, 3),
+ PINS_FIELD_BASE(150, 151, 7, 0x0010, 0x10, 0, 3),
+ PINS_FIELD_BASE(152, 153, 7, 0x0010, 0x10, 3, 3),
+ PIN_FIELD_BASE(154, 154, 7, 0x0010, 0x10, 6, 3),
+ PINS_FIELD_BASE(155, 157, 3, 0x0010, 0x10, 3, 3),
+ PIN_FIELD_BASE(158, 158, 3, 0x0010, 0x10, 6, 3),
+ PIN_FIELD_BASE(159, 159, 7, 0x0010, 0x10, 6, 3),
+ PIN_FIELD_BASE(160, 160, 5, 0x0000, 0x10, 24, 3),
+ PINS_FIELD_BASE(161, 162, 1, 0x0010, 0x10, 15, 3),
+ PINS_FIELD_BASE(163, 166, 4, 0x0000, 0x10, 0, 3),
+ PINS_FIELD_BASE(167, 170, 4, 0x0000, 0x10, 3, 3),
+ PINS_FIELD_BASE(171, 174, 7, 0x0000, 0x10, 18, 3),
+ PINS_FIELD_BASE(175, 179, 7, 0x0000, 0x10, 15, 3),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_pupd_range[] = {
+ PINS_FIELD_BASE(0, 28, 0, 0x0050, 0x10, 18, 1),
+ PIN_FIELD_BASE(29, 29, 6, 0x0050, 0x10, 0, 1),
+ PIN_FIELD_BASE(30, 30, 6, 0x0050, 0x10, 1, 1),
+ PIN_FIELD_BASE(31, 31, 6, 0x0050, 0x10, 5, 1),
+ PIN_FIELD_BASE(32, 32, 6, 0x0050, 0x10, 2, 1),
+ PIN_FIELD_BASE(33, 33, 6, 0x0050, 0x10, 4, 1),
+ PIN_FIELD_BASE(34, 34, 6, 0x0050, 0x10, 3, 1),
+ PIN_FIELD_BASE(35, 35, 6, 0x0050, 0x10, 10, 1),
+ PIN_FIELD_BASE(36, 36, 6, 0x0050, 0x10, 11, 1),
+ PIN_FIELD_BASE(37, 37, 6, 0x0050, 0x10, 9, 1),
+ PIN_FIELD_BASE(38, 38, 6, 0x0050, 0x10, 6, 1),
+ PIN_FIELD_BASE(39, 39, 6, 0x0050, 0x10, 8, 1),
+ PINS_FIELD_BASE(40, 90, 6, 0x0050, 0x10, 7, 1),
+ PIN_FIELD_BASE(91, 91, 2, 0x0050, 0x10, 3, 1),
+ PIN_FIELD_BASE(92, 92, 2, 0x0050, 0x10, 2, 1),
+ PIN_FIELD_BASE(93, 93, 2, 0x0050, 0x10, 0, 1),
+ PINS_FIELD_BASE(94, 121, 2, 0x0050, 0x10, 1, 1),
+ PIN_FIELD_BASE(122, 122, 4, 0x0030, 0x10, 1, 1),
+ PIN_FIELD_BASE(123, 123, 4, 0x0030, 0x10, 2, 1),
+ PIN_FIELD_BASE(124, 124, 4, 0x0030, 0x10, 0, 1),
+ PIN_FIELD_BASE(125, 125, 4, 0x0030, 0x10, 4, 1),
+ PIN_FIELD_BASE(126, 126, 4, 0x0030, 0x10, 6, 1),
+ PIN_FIELD_BASE(127, 127, 4, 0x0030, 0x10, 8, 1),
+ PIN_FIELD_BASE(128, 128, 4, 0x0030, 0x10, 3, 1),
+ PIN_FIELD_BASE(129, 129, 4, 0x0030, 0x10, 7, 1),
+ PIN_FIELD_BASE(130, 130, 4, 0x0030, 0x10, 9, 1),
+ PIN_FIELD_BASE(131, 131, 4, 0x0030, 0x10, 10, 1),
+ PIN_FIELD_BASE(132, 132, 4, 0x0030, 0x10, 5, 1),
+ PINS_FIELD_BASE(133, 179, 4, 0x0030, 0x10, 11, 1),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_r0_range[] = {
+ PINS_FIELD_BASE(0, 28, 4, 0x0030, 0x10, 11, 1),
+ PIN_FIELD_BASE(29, 29, 6, 0x0070, 0x10, 0, 1),
+ PIN_FIELD_BASE(30, 30, 6, 0x0070, 0x10, 1, 1),
+ PIN_FIELD_BASE(31, 31, 6, 0x0070, 0x10, 5, 1),
+ PIN_FIELD_BASE(32, 32, 6, 0x0070, 0x10, 2, 1),
+ PIN_FIELD_BASE(33, 33, 6, 0x0070, 0x10, 4, 1),
+ PIN_FIELD_BASE(34, 34, 6, 0x0070, 0x10, 3, 1),
+ PIN_FIELD_BASE(35, 35, 6, 0x0070, 0x10, 10, 1),
+ PIN_FIELD_BASE(36, 36, 6, 0x0070, 0x10, 11, 1),
+ PIN_FIELD_BASE(37, 37, 6, 0x0070, 0x10, 9, 1),
+ PIN_FIELD_BASE(38, 38, 6, 0x0070, 0x10, 6, 1),
+ PIN_FIELD_BASE(39, 39, 6, 0x0070, 0x10, 8, 1),
+ PINS_FIELD_BASE(40, 90, 6, 0x0070, 0x10, 7, 1),
+ PIN_FIELD_BASE(91, 91, 2, 0x0070, 0x10, 3, 1),
+ PIN_FIELD_BASE(92, 92, 2, 0x0070, 0x10, 2, 1),
+ PIN_FIELD_BASE(93, 93, 2, 0x0070, 0x10, 0, 1),
+ PINS_FIELD_BASE(94, 121, 2, 0x0070, 0x10, 1, 1),
+ PIN_FIELD_BASE(122, 122, 4, 0x0050, 0x10, 1, 1),
+ PIN_FIELD_BASE(123, 123, 4, 0x0050, 0x10, 2, 1),
+ PIN_FIELD_BASE(124, 124, 4, 0x0050, 0x10, 0, 1),
+ PIN_FIELD_BASE(125, 125, 4, 0x0050, 0x10, 4, 1),
+ PIN_FIELD_BASE(126, 126, 4, 0x0050, 0x10, 6, 1),
+ PIN_FIELD_BASE(127, 127, 4, 0x0050, 0x10, 8, 1),
+ PIN_FIELD_BASE(128, 128, 4, 0x0050, 0x10, 3, 1),
+ PIN_FIELD_BASE(129, 129, 4, 0x0050, 0x10, 7, 1),
+ PIN_FIELD_BASE(130, 130, 4, 0x0050, 0x10, 9, 1),
+ PIN_FIELD_BASE(131, 131, 4, 0x0050, 0x10, 10, 1),
+ PIN_FIELD_BASE(132, 132, 4, 0x0050, 0x10, 5, 1),
+ PINS_FIELD_BASE(133, 179, 4, 0x0050, 0x10, 11, 1),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_r1_range[] = {
+ PINS_FIELD_BASE(0, 28, 4, 0x0050, 0x10, 11, 1),
+ PIN_FIELD_BASE(29, 29, 6, 0x0080, 0x10, 0, 1),
+ PIN_FIELD_BASE(30, 30, 6, 0x0080, 0x10, 1, 1),
+ PIN_FIELD_BASE(31, 31, 6, 0x0080, 0x10, 5, 1),
+ PIN_FIELD_BASE(32, 32, 6, 0x0080, 0x10, 2, 1),
+ PIN_FIELD_BASE(33, 33, 6, 0x0080, 0x10, 4, 1),
+ PIN_FIELD_BASE(34, 34, 6, 0x0080, 0x10, 3, 1),
+ PIN_FIELD_BASE(35, 35, 6, 0x0080, 0x10, 10, 1),
+ PIN_FIELD_BASE(36, 36, 6, 0x0080, 0x10, 11, 1),
+ PIN_FIELD_BASE(37, 37, 6, 0x0080, 0x10, 9, 1),
+ PIN_FIELD_BASE(38, 38, 6, 0x0080, 0x10, 6, 1),
+ PIN_FIELD_BASE(39, 39, 6, 0x0080, 0x10, 8, 1),
+ PINS_FIELD_BASE(40, 90, 6, 0x0080, 0x10, 7, 1),
+ PIN_FIELD_BASE(91, 91, 2, 0x0080, 0x10, 3, 1),
+ PIN_FIELD_BASE(92, 92, 2, 0x0080, 0x10, 2, 1),
+ PIN_FIELD_BASE(93, 93, 2, 0x0080, 0x10, 0, 1),
+ PINS_FIELD_BASE(94, 121, 2, 0x0080, 0x10, 1, 1),
+ PIN_FIELD_BASE(122, 122, 4, 0x0060, 0x10, 1, 1),
+ PIN_FIELD_BASE(123, 123, 4, 0x0060, 0x10, 2, 1),
+ PIN_FIELD_BASE(124, 124, 4, 0x0060, 0x10, 0, 1),
+ PIN_FIELD_BASE(125, 125, 4, 0x0060, 0x10, 4, 1),
+ PIN_FIELD_BASE(126, 126, 4, 0x0060, 0x10, 6, 1),
+ PIN_FIELD_BASE(127, 127, 4, 0x0060, 0x10, 8, 1),
+ PIN_FIELD_BASE(128, 128, 4, 0x0060, 0x10, 3, 1),
+ PIN_FIELD_BASE(129, 129, 4, 0x0060, 0x10, 7, 1),
+ PIN_FIELD_BASE(130, 130, 4, 0x0060, 0x10, 9, 1),
+ PIN_FIELD_BASE(131, 131, 4, 0x0060, 0x10, 10, 1),
+ PIN_FIELD_BASE(132, 132, 4, 0x0060, 0x10, 5, 1),
+ PINS_FIELD_BASE(133, 179, 4, 0x0060, 0x10, 11, 1),
+};
+
+static const struct mtk_pin_field_calc mt6765_pin_ies_range[] = {
+ PIN_FIELD_BASE(0, 0, 2, 0x0030, 0x10, 6, 1),
+ PIN_FIELD_BASE(1, 1, 2, 0x0030, 0x10, 7, 1),
+ PIN_FIELD_BASE(2, 2, 2, 0x0030, 0x10, 10, 1),
+ PIN_FIELD_BASE(3, 3, 2, 0x0030, 0x10, 11, 1),
+ PIN_FIELD_BASE(4, 4, 2, 0x0030, 0x10, 12, 1),
+ PIN_FIELD_BASE(5, 5, 2, 0x0030, 0x10, 13, 1),
+ PIN_FIELD_BASE(6, 6, 2, 0x0030, 0x10, 14, 1),
+ PIN_FIELD_BASE(7, 7, 2, 0x0030, 0x10, 15, 1),
+ PIN_FIELD_BASE(8, 8, 3, 0x0030, 0x10, 12, 1),
+ PIN_FIELD_BASE(9, 9, 2, 0x0030, 0x10, 16, 1),
+ PIN_FIELD_BASE(10, 10, 2, 0x0030, 0x10, 8, 1),
+ PIN_FIELD_BASE(11, 11, 2, 0x0030, 0x10, 9, 1),
+ PIN_FIELD_BASE(12, 12, 5, 0x0020, 0x10, 9, 1),
+ PIN_FIELD_BASE(13, 13, 6, 0x0020, 0x10, 26, 1),
+ PIN_FIELD_BASE(14, 14, 6, 0x0020, 0x10, 25, 1),
+ PIN_FIELD_BASE(15, 15, 6, 0x0020, 0x10, 27, 1),
+ PIN_FIELD_BASE(16, 16, 6, 0x0020, 0x10, 24, 1),
+ PIN_FIELD_BASE(17, 17, 6, 0x0020, 0x10, 19, 1),
+ PIN_FIELD_BASE(18, 18, 6, 0x0020, 0x10, 16, 1),
+ PIN_FIELD_BASE(19, 19, 6, 0x0020, 0x10, 18, 1),
+ PIN_FIELD_BASE(20, 20, 6, 0x0020, 0x10, 17, 1),
+ PIN_FIELD_BASE(21, 21, 6, 0x0020, 0x10, 22, 1),
+ PIN_FIELD_BASE(22, 22, 6, 0x0020, 0x10, 21, 1),
+ PIN_FIELD_BASE(23, 23, 6, 0x0020, 0x10, 23, 1),
+ PIN_FIELD_BASE(24, 24, 6, 0x0020, 0x10, 20, 1),
+ PIN_FIELD_BASE(25, 25, 6, 0x0020, 0x10, 14, 1),
+ PIN_FIELD_BASE(26, 26, 6, 0x0020, 0x10, 13, 1),
+ PIN_FIELD_BASE(27, 27, 6, 0x0020, 0x10, 15, 1),
+ PIN_FIELD_BASE(28, 28, 6, 0x0020, 0x10, 12, 1),
+ PIN_FIELD_BASE(29, 29, 6, 0x0020, 0x10, 0, 1),
+ PIN_FIELD_BASE(30, 30, 6, 0x0020, 0x10, 1, 1),
+ PIN_FIELD_BASE(31, 31, 6, 0x0020, 0x10, 5, 1),
+ PIN_FIELD_BASE(32, 32, 6, 0x0020, 0x10, 2, 1),
+ PIN_FIELD_BASE(33, 33, 6, 0x0020, 0x10, 4, 1),
+ PIN_FIELD_BASE(34, 34, 6, 0x0020, 0x10, 3, 1),
+ PIN_FIELD_BASE(35, 35, 6, 0x0020, 0x10, 10, 1),
+ PIN_FIELD_BASE(36, 36, 6, 0x0020, 0x10, 11, 1),
+ PIN_FIELD_BASE(37, 37, 6, 0x0020, 0x10, 9, 1),
+ PIN_FIELD_BASE(38, 38, 6, 0x0020, 0x10, 6, 1),
+ PIN_FIELD_BASE(39, 39, 6, 0x0020, 0x10, 8, 1),
+ PIN_FIELD_BASE(40, 40, 6, 0x0020, 0x10, 7, 1),
+ PIN_FIELD_BASE(41, 41, 7, 0x0040, 0x10, 19, 1),
+ PIN_FIELD_BASE(42, 42, 7, 0x0040, 0x10, 9, 1),
+ PIN_FIELD_BASE(43, 43, 7, 0x0040, 0x10, 8, 1),
+ PIN_FIELD_BASE(44, 44, 7, 0x0040, 0x10, 10, 1),
+ PIN_FIELD_BASE(45, 45, 7, 0x0040, 0x10, 22, 1),
+ PIN_FIELD_BASE(46, 46, 7, 0x0040, 0x10, 21, 1),
+ PIN_FIELD_BASE(47, 47, 7, 0x0040, 0x10, 20, 1),
+ PIN_FIELD_BASE(48, 48, 7, 0x0050, 0x10, 3, 1),
+ PIN_FIELD_BASE(49, 49, 7, 0x0050, 0x10, 5, 1),
+ PIN_FIELD_BASE(50, 50, 7, 0x0050, 0x10, 2, 1),
+ PIN_FIELD_BASE(51, 51, 7, 0x0050, 0x10, 4, 1),
+ PIN_FIELD_BASE(52, 52, 7, 0x0040, 0x10, 1, 1),
+ PIN_FIELD_BASE(53, 53, 7, 0x0040, 0x10, 0, 1),
+ PIN_FIELD_BASE(54, 54, 7, 0x0040, 0x10, 5, 1),
+ PIN_FIELD_BASE(55, 55, 7, 0x0040, 0x10, 3, 1),
+ PIN_FIELD_BASE(56, 56, 7, 0x0040, 0x10, 4, 1),
+ PIN_FIELD_BASE(57, 57, 7, 0x0040, 0x10, 2, 1),
+ PIN_FIELD_BASE(58, 58, 7, 0x0050, 0x10, 0, 1),
+ PIN_FIELD_BASE(59, 59, 7, 0x0040, 0x10, 31, 1),
+ PIN_FIELD_BASE(60, 60, 7, 0x0040, 0x10, 30, 1),
+ PIN_FIELD_BASE(61, 61, 3, 0x0030, 0x10, 18, 1),
+ PIN_FIELD_BASE(62, 62, 3, 0x0030, 0x10, 14, 1),
+ PIN_FIELD_BASE(63, 63, 3, 0x0030, 0x10, 17, 1),
+ PIN_FIELD_BASE(64, 64, 3, 0x0030, 0x10, 13, 1),
+ PIN_FIELD_BASE(65, 65, 3, 0x0030, 0x10, 20, 1),
+ PIN_FIELD_BASE(66, 66, 3, 0x0030, 0x10, 16, 1),
+ PIN_FIELD_BASE(67, 67, 3, 0x0030, 0x10, 19, 1),
+ PIN_FIELD_BASE(68, 68, 3, 0x0030, 0x10, 15, 1),
+ PIN_FIELD_BASE(69, 69, 3, 0x0030, 0x10, 8, 1),
+ PIN_FIELD_BASE(70, 70, 3, 0x0030, 0x10, 7, 1),
+ PIN_FIELD_BASE(71, 71, 3, 0x0030, 0x10, 6, 1),
+ PIN_FIELD_BASE(72, 72, 3, 0x0030, 0x10, 5, 1),
+ PIN_FIELD_BASE(73, 73, 3, 0x0030, 0x10, 4, 1),
+ PIN_FIELD_BASE(74, 74, 3, 0x0030, 0x10, 3, 1),
+ PIN_FIELD_BASE(75, 75, 3, 0x0030, 0x10, 2, 1),
+ PIN_FIELD_BASE(76, 76, 3, 0x0030, 0x10, 1, 1),
+ PIN_FIELD_BASE(77, 77, 3, 0x0030, 0x10, 0, 1),
+ PIN_FIELD_BASE(78, 78, 3, 0x0030, 0x10, 9, 1),
+ PIN_FIELD_BASE(79, 79, 3, 0x0030, 0x10, 11, 1),
+ PIN_FIELD_BASE(80, 80, 3, 0x0030, 0x10, 10, 1),
+ PIN_FIELD_BASE(81, 81, 3, 0x0030, 0x10, 25, 1),
+ PIN_FIELD_BASE(82, 82, 3, 0x0030, 0x10, 24, 1),
+ PIN_FIELD_BASE(83, 83, 3, 0x0030, 0x10, 22, 1),
+ PIN_FIELD_BASE(84, 84, 3, 0x0030, 0x10, 23, 1),
+ PIN_FIELD_BASE(85, 85, 7, 0x0050, 0x10, 1, 1),
+ PIN_FIELD_BASE(86, 86, 7, 0x0040, 0x10, 29, 1),
+ PIN_FIELD_BASE(87, 87, 7, 0x0040, 0x10, 7, 1),
+ PIN_FIELD_BASE(88, 88, 7, 0x0040, 0x10, 6, 1),
+ PIN_FIELD_BASE(89, 89, 2, 0x0030, 0x10, 25, 1),
+ PIN_FIELD_BASE(90, 90, 3, 0x0030, 0x10, 21, 1),
+ PIN_FIELD_BASE(91, 91, 2, 0x0030, 0x10, 20, 1),
+ PIN_FIELD_BASE(92, 92, 2, 0x0030, 0x10, 19, 1),
+ PIN_FIELD_BASE(93, 93, 2, 0x0030, 0x10, 17, 1),
+ PIN_FIELD_BASE(94, 94, 2, 0x0030, 0x10, 18, 1),
+ PIN_FIELD_BASE(95, 95, 2, 0x0030, 0x10, 26, 1),
+ PIN_FIELD_BASE(96, 96, 2, 0x0030, 0x10, 27, 1),
+ PIN_FIELD_BASE(97, 97, 2, 0x0030, 0x10, 2, 1),
+ PIN_FIELD_BASE(98, 98, 2, 0x0030, 0x10, 3, 1),
+ PIN_FIELD_BASE(99, 99, 2, 0x0030, 0x10, 0, 1),
+ PIN_FIELD_BASE(100, 100, 2, 0x0030, 0x10, 1, 1),
+ PIN_FIELD_BASE(101, 101, 2, 0x0030, 0x10, 4, 1),
+ PIN_FIELD_BASE(102, 102, 2, 0x0030, 0x10, 5, 1),
+ PIN_FIELD_BASE(103, 103, 2, 0x0030, 0x10, 21, 1),
+ PIN_FIELD_BASE(104, 104, 2, 0x0030, 0x10, 23, 1),
+ PIN_FIELD_BASE(105, 105, 2, 0x0030, 0x10, 22, 1),
+ PIN_FIELD_BASE(106, 106, 2, 0x0030, 0x10, 24, 1),
+ PIN_FIELD_BASE(107, 107, 1, 0x0030, 0x10, 4, 1),
+ PIN_FIELD_BASE(108, 108, 1, 0x0030, 0x10, 3, 1),
+ PIN_FIELD_BASE(109, 109, 1, 0x0030, 0x10, 5, 1),
+ PIN_FIELD_BASE(110, 110, 1, 0x0030, 0x10, 0, 1),
+ PIN_FIELD_BASE(111, 111, 1, 0x0030, 0x10, 1, 1),
+ PIN_FIELD_BASE(112, 112, 1, 0x0030, 0x10, 2, 1),
+ PIN_FIELD_BASE(113, 113, 1, 0x0030, 0x10, 9, 1),
+ PIN_FIELD_BASE(114, 114, 1, 0x0030, 0x10, 10, 1),
+ PIN_FIELD_BASE(115, 115, 1, 0x0030, 0x10, 6, 1),
+ PIN_FIELD_BASE(116, 116, 1, 0x0030, 0x10, 7, 1),
+ PIN_FIELD_BASE(117, 117, 1, 0x0030, 0x10, 12, 1),
+ PIN_FIELD_BASE(118, 118, 1, 0x0030, 0x10, 13, 1),
+ PIN_FIELD_BASE(119, 119, 1, 0x0030, 0x10, 14, 1),
+ PIN_FIELD_BASE(120, 120, 1, 0x0030, 0x10, 11, 1),
+ PIN_FIELD_BASE(121, 121, 1, 0x0030, 0x10, 8, 1),
+ PIN_FIELD_BASE(122, 122, 4, 0x0010, 0x10, 9, 1),
+ PIN_FIELD_BASE(123, 123, 4, 0x0010, 0x10, 10, 1),
+ PIN_FIELD_BASE(124, 124, 4, 0x0010, 0x10, 8, 1),
+ PIN_FIELD_BASE(125, 125, 4, 0x0010, 0x10, 12, 1),
+ PIN_FIELD_BASE(126, 126, 4, 0x0010, 0x10, 14, 1),
+ PIN_FIELD_BASE(127, 127, 4, 0x0010, 0x10, 16, 1),
+ PIN_FIELD_BASE(128, 128, 4, 0x0010, 0x10, 11, 1),
+ PIN_FIELD_BASE(129, 129, 4, 0x0010, 0x10, 15, 1),
+ PIN_FIELD_BASE(130, 130, 4, 0x0010, 0x10, 17, 1),
+ PIN_FIELD_BASE(131, 131, 4, 0x0010, 0x10, 18, 1),
+ PIN_FIELD_BASE(132, 132, 4, 0x0010, 0x10, 13, 1),
+ PIN_FIELD_BASE(133, 133, 4, 0x0010, 0x10, 19, 1),
+ PIN_FIELD_BASE(134, 134, 5, 0x0020, 0x10, 14, 1),
+ PIN_FIELD_BASE(135, 135, 5, 0x0020, 0x10, 17, 1),
+ PIN_FIELD_BASE(136, 136, 5, 0x0020, 0x10, 1, 1),
+ PIN_FIELD_BASE(137, 137, 5, 0x0020, 0x10, 7, 1),
+ PIN_FIELD_BASE(138, 138, 5, 0x0020, 0x10, 4, 1),
+ PIN_FIELD_BASE(139, 139, 5, 0x0020, 0x10, 5, 1),
+ PIN_FIELD_BASE(140, 140, 5, 0x0020, 0x10, 0, 1),
+ PIN_FIELD_BASE(141, 141, 5, 0x0020, 0x10, 6, 1),
+ PIN_FIELD_BASE(142, 142, 5, 0x0020, 0x10, 2, 1),
+ PIN_FIELD_BASE(143, 143, 5, 0x0020, 0x10, 3, 1),
+ PIN_FIELD_BASE(144, 144, 5, 0x0020, 0x10, 12, 1),
+ PIN_FIELD_BASE(145, 145, 5, 0x0020, 0x10, 11, 1),
+ PIN_FIELD_BASE(146, 146, 5, 0x0020, 0x10, 13, 1),
+ PIN_FIELD_BASE(147, 147, 5, 0x0020, 0x10, 10, 1),
+ PIN_FIELD_BASE(148, 148, 5, 0x0020, 0x10, 15, 1),
+ PIN_FIELD_BASE(149, 149, 5, 0x0020, 0x10, 16, 1),
+ PIN_FIELD_BASE(150, 150, 7, 0x0040, 0x10, 23, 1),
+ PIN_FIELD_BASE(151, 151, 7, 0x0040, 0x10, 24, 1),
+ PIN_FIELD_BASE(152, 152, 7, 0x0040, 0x10, 25, 1),
+ PIN_FIELD_BASE(153, 153, 7, 0x0040, 0x10, 26, 1),
+ PIN_FIELD_BASE(154, 154, 7, 0x0040, 0x10, 28, 1),
+ PIN_FIELD_BASE(155, 155, 3, 0x0030, 0x10, 28, 1),
+ PIN_FIELD_BASE(156, 156, 3, 0x0030, 0x10, 27, 1),
+ PIN_FIELD_BASE(157, 157, 3, 0x0030, 0x10, 29, 1),
+ PIN_FIELD_BASE(158, 158, 3, 0x0030, 0x10, 26, 1),
+ PIN_FIELD_BASE(159, 159, 7, 0x0040, 0x10, 27, 1),
+ PIN_FIELD_BASE(160, 160, 5, 0x0020, 0x10, 8, 1),
+ PIN_FIELD_BASE(161, 161, 1, 0x0030, 0x10, 15, 1),
+ PIN_FIELD_BASE(162, 162, 1, 0x0030, 0x10, 16, 1),
+ PIN_FIELD_BASE(163, 163, 4, 0x0010, 0x10, 0, 1),
+ PIN_FIELD_BASE(164, 164, 4, 0x0010, 0x10, 1, 1),
+ PIN_FIELD_BASE(165, 165, 4, 0x0010, 0x10, 2, 1),
+ PIN_FIELD_BASE(166, 166, 4, 0x0010, 0x10, 3, 1),
+ PIN_FIELD_BASE(167, 167, 4, 0x0010, 0x10, 4, 1),
+ PIN_FIELD_BASE(168, 168, 4, 0x0010, 0x10, 5, 1),
+ PIN_FIELD_BASE(169, 169, 4, 0x0010, 0x10, 6, 1),
+ PIN_FIELD_BASE(170, 170, 4, 0x0010, 0x10, 7, 1),
+ PIN_FIELD_BASE(171, 171, 7, 0x0040, 0x10, 17, 1),
+ PIN_FIELD_BASE(172, 172, 7, 0x0040, 0x10, 18, 1),
+ PIN_FIELD_BASE(173, 173, 7, 0x0040, 0x10, 11, 1),
+ PIN_FIELD_BASE(174, 174, 7, 0x0040, 0x10, 12, 1),
+ PIN_FIELD_BASE(175, 175, 7, 0x0040, 0x10, 13, 1),
+ PIN_FIELD_BASE(176, 176, 7, 0x0040, 0x10, 14, 1),
+ PIN_FIELD_BASE(177, 177, 7, 0x0040, 0x10, 15, 1),
+ PINS_FIELD_BASE(178, 179, 7, 0x0040, 0x10, 16, 1),
+};
+
+static const struct mtk_pin_reg_calc mt6765_reg_cals[PINCTRL_PIN_REG_MAX] = {
+ [PINCTRL_PIN_REG_MODE] = MTK_RANGE(mt6765_pin_mode_range),
+ [PINCTRL_PIN_REG_DIR] = MTK_RANGE(mt6765_pin_dir_range),
+ [PINCTRL_PIN_REG_DI] = MTK_RANGE(mt6765_pin_di_range),
+ [PINCTRL_PIN_REG_DO] = MTK_RANGE(mt6765_pin_do_range),
+ [PINCTRL_PIN_REG_SMT] = MTK_RANGE(mt6765_pin_smt_range),
+ [PINCTRL_PIN_REG_PD] = MTK_RANGE(mt6765_pin_pd_range),
+ [PINCTRL_PIN_REG_PU] = MTK_RANGE(mt6765_pin_pu_range),
+ [PINCTRL_PIN_REG_TDSEL] = MTK_RANGE(mt6765_pin_tdsel_range),
+ [PINCTRL_PIN_REG_RDSEL] = MTK_RANGE(mt6765_pin_rdsel_range),
+ [PINCTRL_PIN_REG_DRV] = MTK_RANGE(mt6765_pin_drv_range),
+ [PINCTRL_PIN_REG_PUPD] = MTK_RANGE(mt6765_pin_pupd_range),
+ [PINCTRL_PIN_REG_R0] = MTK_RANGE(mt6765_pin_r0_range),
+ [PINCTRL_PIN_REG_R1] = MTK_RANGE(mt6765_pin_r1_range),
+ [PINCTRL_PIN_REG_IES] = MTK_RANGE(mt6765_pin_ies_range),
+};
+
+static const char * const mt6765_pinctrl_register_base_names[] = {
+ "iocfg0", "iocfg1", "iocfg2", "iocfg3", "iocfg4", "iocfg5",
+ "iocfg6", "iocfg7",
+};
+
+static const struct mtk_eint_hw mt6765_eint_hw = {
+ .port_mask = 7,
+ .ports = 6,
+ .ap_num = 160,
+ .db_cnt = 13,
+};
+
+static const struct mtk_pin_soc mt6765_data = {
+ .reg_cal = mt6765_reg_cals,
+ .pins = mtk_pins_mt6765,
+ .npins = ARRAY_SIZE(mtk_pins_mt6765),
+ .ngrps = ARRAY_SIZE(mtk_pins_mt6765),
+ .eint_hw = &mt6765_eint_hw,
+ .gpio_m = 0,
+ .ies_present = true,
+ .base_names = mt6765_pinctrl_register_base_names,
+ .nbase_names = ARRAY_SIZE(mt6765_pinctrl_register_base_names),
+ .bias_disable_set = mtk_pinconf_bias_disable_set,
+ .bias_disable_get = mtk_pinconf_bias_disable_get,
+ .bias_set = mtk_pinconf_bias_set,
+ .bias_get = mtk_pinconf_bias_get,
+ .drive_set = mtk_pinconf_drive_set_rev1,
+ .drive_get = mtk_pinconf_drive_get_rev1,
+ .adv_pull_get = mtk_pinconf_adv_pull_get,
+ .adv_pull_set = mtk_pinconf_adv_pull_set,
+};
+
+static const struct of_device_id mt6765_pinctrl_of_match[] = {
+ { .compatible = "mediatek,mt6765-pinctrl", },
+ { }
+};
+
+static int mt6765_pinctrl_probe(struct platform_device *pdev)
+{
+ return mtk_paris_pinctrl_probe(pdev, &mt6765_data);
+}
+
+static struct platform_driver mt6765_pinctrl_driver = {
+ .driver = {
+ .name = "mt6765-pinctrl",
+ .of_match_table = mt6765_pinctrl_of_match,
+ },
+ .probe = mt6765_pinctrl_probe,
+};
+
+static int __init mt6765_pinctrl_init(void)
+{
+ return platform_driver_register(&mt6765_pinctrl_driver);
+}
+arch_initcall(mt6765_pinctrl_init);
+// SPDX-License-Identifier: GPL-2.0
/*
- * MediaTek MT7622 Pinctrl Driver
+ * Copyright (C) 2017-2018 MediaTek Inc.
*
- * Copyright (C) 2017 Sean Wang <sean.wang@mediatek.com>
+ * Author: Sean Wang <sean.wang@mediatek.com>
*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- */
-
-#include <linux/gpio.h>
-#include <linux/gpio/driver.h>
-#include <linux/io.h>
-#include <linux/init.h>
-#include <linux/mfd/syscon.h>
-#include <linux/of.h>
-#include <linux/of_irq.h>
-#include <linux/of_platform.h>
-#include <linux/platform_device.h>
-#include <linux/pinctrl/pinctrl.h>
-#include <linux/pinctrl/pinmux.h>
-#include <linux/pinctrl/pinconf.h>
-#include <linux/pinctrl/pinconf-generic.h>
-#include <linux/regmap.h>
-
-#include "../core.h"
-#include "../pinconf.h"
-#include "../pinmux.h"
-#include "mtk-eint.h"
-
-#define PINCTRL_PINCTRL_DEV KBUILD_MODNAME
-#define MTK_RANGE(_a) { .range = (_a), .nranges = ARRAY_SIZE(_a), }
-#define PINCTRL_PIN_GROUP(name, id) \
- { \
- name, \
- id##_pins, \
- ARRAY_SIZE(id##_pins), \
- id##_funcs, \
- }
-
-#define MTK_GPIO_MODE 1
-#define MTK_INPUT 0
-#define MTK_OUTPUT 1
-#define MTK_DISABLE 0
-#define MTK_ENABLE 1
-
-/* Custom pinconf parameters */
-#define MTK_PIN_CONFIG_TDSEL (PIN_CONFIG_END + 1)
-#define MTK_PIN_CONFIG_RDSEL (PIN_CONFIG_END + 2)
-
-/* List these attributes which could be modified for the pin */
-enum {
- PINCTRL_PIN_REG_MODE,
- PINCTRL_PIN_REG_DIR,
- PINCTRL_PIN_REG_DI,
- PINCTRL_PIN_REG_DO,
- PINCTRL_PIN_REG_SR,
- PINCTRL_PIN_REG_SMT,
- PINCTRL_PIN_REG_PD,
- PINCTRL_PIN_REG_PU,
- PINCTRL_PIN_REG_E4,
- PINCTRL_PIN_REG_E8,
- PINCTRL_PIN_REG_TDSEL,
- PINCTRL_PIN_REG_RDSEL,
- PINCTRL_PIN_REG_MAX,
-};
-
-/* struct mtk_pin_field - the structure that holds the information of the field
- * used to describe the attribute for the pin
- * @offset: the register offset relative to the base address
- * @mask: the mask used to filter out the field from the register
- * @bitpos: the start bit relative to the register
- * @next: the indication that the field would be extended to the
- next register
*/
-struct mtk_pin_field {
- u32 offset;
- u32 mask;
- u8 bitpos;
- u8 next;
-};
-/* struct mtk_pin_field_calc - the structure that holds the range providing
- * the guide used to look up the relevant field
- * @s_pin: the start pin within the range
- * @e_pin: the end pin within the range
- * @s_addr: the start address for the range
- * @x_addrs: the address distance between two consecutive registers
- * within the range
- * @s_bit: the start bit for the first register within the range
- * @x_bits: the bit distance between two consecutive pins within
- * the range
- */
-struct mtk_pin_field_calc {
- u16 s_pin;
- u16 e_pin;
- u32 s_addr;
- u8 x_addrs;
- u8 s_bit;
- u8 x_bits;
-};
+#include "pinctrl-moore.h"
-/* struct mtk_pin_reg_calc - the structure that holds all ranges used to
- * determine which register the pin would make use of
- * for certain pin attribute.
- * @range: the start address for the range
- * @nranges: the number of items in the range
- */
-struct mtk_pin_reg_calc {
- const struct mtk_pin_field_calc *range;
- unsigned int nranges;
-};
-
-/* struct mtk_pin_soc - the structure that holds SoC-specific data */
-struct mtk_pin_soc {
- const struct mtk_pin_reg_calc *reg_cal;
- const struct pinctrl_pin_desc *pins;
- unsigned int npins;
- const struct group_desc *grps;
- unsigned int ngrps;
- const struct function_desc *funcs;
- unsigned int nfuncs;
- const struct mtk_eint_regs *eint_regs;
- const struct mtk_eint_hw *eint_hw;
-};
-
-struct mtk_pinctrl {
- struct pinctrl_dev *pctrl;
- void __iomem *base;
- struct device *dev;
- struct gpio_chip chip;
- const struct mtk_pin_soc *soc;
- struct mtk_eint *eint;
-};
+#define MT7622_PIN(_number, _name) \
+ MTK_PIN(_number, _name, 1, _number, DRV_GRP0)
static const struct mtk_pin_field_calc mt7622_pin_mode_range[] = {
- {0, 0, 0x320, 0x10, 16, 4},
- {1, 4, 0x3a0, 0x10, 16, 4},
- {5, 5, 0x320, 0x10, 0, 4},
- {6, 6, 0x300, 0x10, 4, 4},
- {7, 7, 0x300, 0x10, 4, 4},
- {8, 9, 0x350, 0x10, 20, 4},
- {10, 10, 0x300, 0x10, 8, 4},
- {11, 11, 0x300, 0x10, 8, 4},
- {12, 12, 0x300, 0x10, 8, 4},
- {13, 13, 0x300, 0x10, 8, 4},
- {14, 15, 0x320, 0x10, 4, 4},
- {16, 17, 0x320, 0x10, 20, 4},
- {18, 21, 0x310, 0x10, 16, 4},
- {22, 22, 0x380, 0x10, 16, 4},
- {23, 23, 0x300, 0x10, 24, 4},
- {24, 24, 0x300, 0x10, 24, 4},
- {25, 25, 0x300, 0x10, 12, 4},
- {25, 25, 0x300, 0x10, 12, 4},
- {26, 26, 0x300, 0x10, 12, 4},
- {27, 27, 0x300, 0x10, 12, 4},
- {28, 28, 0x300, 0x10, 12, 4},
- {29, 29, 0x300, 0x10, 12, 4},
- {30, 30, 0x300, 0x10, 12, 4},
- {31, 31, 0x300, 0x10, 12, 4},
- {32, 32, 0x300, 0x10, 12, 4},
- {33, 33, 0x300, 0x10, 12, 4},
- {34, 34, 0x300, 0x10, 12, 4},
- {35, 35, 0x300, 0x10, 12, 4},
- {36, 36, 0x300, 0x10, 12, 4},
- {37, 37, 0x300, 0x10, 20, 4},
- {38, 38, 0x300, 0x10, 20, 4},
- {39, 39, 0x300, 0x10, 20, 4},
- {40, 40, 0x300, 0x10, 20, 4},
- {41, 41, 0x300, 0x10, 20, 4},
- {42, 42, 0x300, 0x10, 20, 4},
- {43, 43, 0x300, 0x10, 20, 4},
- {44, 44, 0x300, 0x10, 20, 4},
- {45, 46, 0x300, 0x10, 20, 4},
- {47, 47, 0x300, 0x10, 20, 4},
- {48, 48, 0x300, 0x10, 20, 4},
- {49, 49, 0x300, 0x10, 20, 4},
- {50, 50, 0x300, 0x10, 20, 4},
- {51, 70, 0x330, 0x10, 4, 4},
- {71, 71, 0x300, 0x10, 16, 4},
- {72, 72, 0x300, 0x10, 16, 4},
- {73, 76, 0x310, 0x10, 0, 4},
- {77, 77, 0x320, 0x10, 28, 4},
- {78, 78, 0x320, 0x10, 12, 4},
- {79, 82, 0x3a0, 0x10, 0, 4},
- {83, 83, 0x350, 0x10, 28, 4},
- {84, 84, 0x330, 0x10, 0, 4},
- {85, 90, 0x360, 0x10, 4, 4},
- {91, 94, 0x390, 0x10, 16, 4},
- {95, 97, 0x380, 0x10, 20, 4},
- {98, 101, 0x390, 0x10, 0, 4},
- {102, 102, 0x360, 0x10, 0, 4},
+ PIN_FIELD(0, 0, 0x320, 0x10, 16, 4),
+ PIN_FIELD(1, 4, 0x3a0, 0x10, 16, 4),
+ PIN_FIELD(5, 5, 0x320, 0x10, 0, 4),
+ PINS_FIELD(6, 7, 0x300, 0x10, 4, 4),
+ PIN_FIELD(8, 9, 0x350, 0x10, 20, 4),
+ PINS_FIELD(10, 13, 0x300, 0x10, 8, 4),
+ PIN_FIELD(14, 15, 0x320, 0x10, 4, 4),
+ PIN_FIELD(16, 17, 0x320, 0x10, 20, 4),
+ PIN_FIELD(18, 21, 0x310, 0x10, 16, 4),
+ PIN_FIELD(22, 22, 0x380, 0x10, 16, 4),
+ PINS_FIELD(23, 24, 0x300, 0x10, 24, 4),
+ PINS_FIELD(25, 36, 0x300, 0x10, 12, 4),
+ PINS_FIELD(37, 50, 0x300, 0x10, 20, 4),
+ PIN_FIELD(51, 70, 0x330, 0x10, 4, 4),
+ PINS_FIELD(71, 72, 0x300, 0x10, 16, 4),
+ PIN_FIELD(73, 76, 0x310, 0x10, 0, 4),
+ PIN_FIELD(77, 77, 0x320, 0x10, 28, 4),
+ PIN_FIELD(78, 78, 0x320, 0x10, 12, 4),
+ PIN_FIELD(79, 82, 0x3a0, 0x10, 0, 4),
+ PIN_FIELD(83, 83, 0x350, 0x10, 28, 4),
+ PIN_FIELD(84, 84, 0x330, 0x10, 0, 4),
+ PIN_FIELD(85, 90, 0x360, 0x10, 4, 4),
+ PIN_FIELD(91, 94, 0x390, 0x10, 16, 4),
+ PIN_FIELD(95, 97, 0x380, 0x10, 20, 4),
+ PIN_FIELD(98, 101, 0x390, 0x10, 0, 4),
+ PIN_FIELD(102, 102, 0x360, 0x10, 0, 4),
};
static const struct mtk_pin_field_calc mt7622_pin_dir_range[] = {
- {0, 102, 0x0, 0x10, 0, 1},
+ PIN_FIELD(0, 102, 0x0, 0x10, 0, 1),
};
static const struct mtk_pin_field_calc mt7622_pin_di_range[] = {
- {0, 102, 0x200, 0x10, 0, 1},
+ PIN_FIELD(0, 102, 0x200, 0x10, 0, 1),
};
static const struct mtk_pin_field_calc mt7622_pin_do_range[] = {
- {0, 102, 0x100, 0x10, 0, 1},
+ PIN_FIELD(0, 102, 0x100, 0x10, 0, 1),
};
static const struct mtk_pin_field_calc mt7622_pin_sr_range[] = {
- {0, 31, 0x910, 0x10, 0, 1},
- {32, 50, 0xa10, 0x10, 0, 1},
- {51, 70, 0x810, 0x10, 0, 1},
- {71, 72, 0xb10, 0x10, 0, 1},
- {73, 86, 0xb10, 0x10, 4, 1},
- {87, 90, 0xc10, 0x10, 0, 1},
- {91, 102, 0xb10, 0x10, 18, 1},
+ PIN_FIELD(0, 31, 0x910, 0x10, 0, 1),
+ PIN_FIELD(32, 50, 0xa10, 0x10, 0, 1),
+ PIN_FIELD(51, 70, 0x810, 0x10, 0, 1),
+ PIN_FIELD(71, 72, 0xb10, 0x10, 0, 1),
+ PIN_FIELD(73, 86, 0xb10, 0x10, 4, 1),
+ PIN_FIELD(87, 90, 0xc10, 0x10, 0, 1),
+ PIN_FIELD(91, 102, 0xb10, 0x10, 18, 1),
};
static const struct mtk_pin_field_calc mt7622_pin_smt_range[] = {
- {0, 31, 0x920, 0x10, 0, 1},
- {32, 50, 0xa20, 0x10, 0, 1},
- {51, 70, 0x820, 0x10, 0, 1},
- {71, 72, 0xb20, 0x10, 0, 1},
- {73, 86, 0xb20, 0x10, 4, 1},
- {87, 90, 0xc20, 0x10, 0, 1},
- {91, 102, 0xb20, 0x10, 18, 1},
+ PIN_FIELD(0, 31, 0x920, 0x10, 0, 1),
+ PIN_FIELD(32, 50, 0xa20, 0x10, 0, 1),
+ PIN_FIELD(51, 70, 0x820, 0x10, 0, 1),
+ PIN_FIELD(71, 72, 0xb20, 0x10, 0, 1),
+ PIN_FIELD(73, 86, 0xb20, 0x10, 4, 1),
+ PIN_FIELD(87, 90, 0xc20, 0x10, 0, 1),
+ PIN_FIELD(91, 102, 0xb20, 0x10, 18, 1),
};
static const struct mtk_pin_field_calc mt7622_pin_pu_range[] = {
- {0, 31, 0x930, 0x10, 0, 1},
- {32, 50, 0xa30, 0x10, 0, 1},
- {51, 70, 0x830, 0x10, 0, 1},
- {71, 72, 0xb30, 0x10, 0, 1},
- {73, 86, 0xb30, 0x10, 4, 1},
- {87, 90, 0xc30, 0x10, 0, 1},
- {91, 102, 0xb30, 0x10, 18, 1},
+ PIN_FIELD(0, 31, 0x930, 0x10, 0, 1),
+ PIN_FIELD(32, 50, 0xa30, 0x10, 0, 1),
+ PIN_FIELD(51, 70, 0x830, 0x10, 0, 1),
+ PIN_FIELD(71, 72, 0xb30, 0x10, 0, 1),
+ PIN_FIELD(73, 86, 0xb30, 0x10, 4, 1),
+ PIN_FIELD(87, 90, 0xc30, 0x10, 0, 1),
+ PIN_FIELD(91, 102, 0xb30, 0x10, 18, 1),
};
static const struct mtk_pin_field_calc mt7622_pin_pd_range[] = {
- {0, 31, 0x940, 0x10, 0, 1},
- {32, 50, 0xa40, 0x10, 0, 1},
- {51, 70, 0x840, 0x10, 0, 1},
- {71, 72, 0xb40, 0x10, 0, 1},
- {73, 86, 0xb40, 0x10, 4, 1},
- {87, 90, 0xc40, 0x10, 0, 1},
- {91, 102, 0xb40, 0x10, 18, 1},
+ PIN_FIELD(0, 31, 0x940, 0x10, 0, 1),
+ PIN_FIELD(32, 50, 0xa40, 0x10, 0, 1),
+ PIN_FIELD(51, 70, 0x840, 0x10, 0, 1),
+ PIN_FIELD(71, 72, 0xb40, 0x10, 0, 1),
+ PIN_FIELD(73, 86, 0xb40, 0x10, 4, 1),
+ PIN_FIELD(87, 90, 0xc40, 0x10, 0, 1),
+ PIN_FIELD(91, 102, 0xb40, 0x10, 18, 1),
};
static const struct mtk_pin_field_calc mt7622_pin_e4_range[] = {
- {0, 31, 0x960, 0x10, 0, 1},
- {32, 50, 0xa60, 0x10, 0, 1},
- {51, 70, 0x860, 0x10, 0, 1},
- {71, 72, 0xb60, 0x10, 0, 1},
- {73, 86, 0xb60, 0x10, 4, 1},
- {87, 90, 0xc60, 0x10, 0, 1},
- {91, 102, 0xb60, 0x10, 18, 1},
+ PIN_FIELD(0, 31, 0x960, 0x10, 0, 1),
+ PIN_FIELD(32, 50, 0xa60, 0x10, 0, 1),
+ PIN_FIELD(51, 70, 0x860, 0x10, 0, 1),
+ PIN_FIELD(71, 72, 0xb60, 0x10, 0, 1),
+ PIN_FIELD(73, 86, 0xb60, 0x10, 4, 1),
+ PIN_FIELD(87, 90, 0xc60, 0x10, 0, 1),
+ PIN_FIELD(91, 102, 0xb60, 0x10, 18, 1),
};
static const struct mtk_pin_field_calc mt7622_pin_e8_range[] = {
- {0, 31, 0x970, 0x10, 0, 1},
- {32, 50, 0xa70, 0x10, 0, 1},
- {51, 70, 0x870, 0x10, 0, 1},
- {71, 72, 0xb70, 0x10, 0, 1},
- {73, 86, 0xb70, 0x10, 4, 1},
- {87, 90, 0xc70, 0x10, 0, 1},
- {91, 102, 0xb70, 0x10, 18, 1},
+ PIN_FIELD(0, 31, 0x970, 0x10, 0, 1),
+ PIN_FIELD(32, 50, 0xa70, 0x10, 0, 1),
+ PIN_FIELD(51, 70, 0x870, 0x10, 0, 1),
+ PIN_FIELD(71, 72, 0xb70, 0x10, 0, 1),
+ PIN_FIELD(73, 86, 0xb70, 0x10, 4, 1),
+ PIN_FIELD(87, 90, 0xc70, 0x10, 0, 1),
+ PIN_FIELD(91, 102, 0xb70, 0x10, 18, 1),
};
static const struct mtk_pin_field_calc mt7622_pin_tdsel_range[] = {
- {0, 31, 0x980, 0x4, 0, 4},
- {32, 50, 0xa80, 0x4, 0, 4},
- {51, 70, 0x880, 0x4, 0, 4},
- {71, 72, 0xb80, 0x4, 0, 4},
- {73, 86, 0xb80, 0x4, 16, 4},
- {87, 90, 0xc80, 0x4, 0, 4},
- {91, 102, 0xb88, 0x4, 8, 4},
+ PIN_FIELD(0, 31, 0x980, 0x4, 0, 4),
+ PIN_FIELD(32, 50, 0xa80, 0x4, 0, 4),
+ PIN_FIELD(51, 70, 0x880, 0x4, 0, 4),
+ PIN_FIELD(71, 72, 0xb80, 0x4, 0, 4),
+ PIN_FIELD(73, 86, 0xb80, 0x4, 16, 4),
+ PIN_FIELD(87, 90, 0xc80, 0x4, 0, 4),
+ PIN_FIELD(91, 102, 0xb88, 0x4, 8, 4),
};
static const struct mtk_pin_field_calc mt7622_pin_rdsel_range[] = {
- {0, 31, 0x990, 0x4, 0, 6},
- {32, 50, 0xa90, 0x4, 0, 6},
- {51, 58, 0x890, 0x4, 0, 6},
- {59, 60, 0x894, 0x4, 28, 6},
- {61, 62, 0x894, 0x4, 16, 6},
- {63, 66, 0x898, 0x4, 8, 6},
- {67, 68, 0x89c, 0x4, 12, 6},
- {69, 70, 0x89c, 0x4, 0, 6},
- {71, 72, 0xb90, 0x4, 0, 6},
- {73, 86, 0xb90, 0x4, 24, 6},
- {87, 90, 0xc90, 0x4, 0, 6},
- {91, 102, 0xb9c, 0x4, 12, 6},
+ PIN_FIELD(0, 31, 0x990, 0x4, 0, 6),
+ PIN_FIELD(32, 50, 0xa90, 0x4, 0, 6),
+ PIN_FIELD(51, 58, 0x890, 0x4, 0, 6),
+ PIN_FIELD(59, 60, 0x894, 0x4, 28, 6),
+ PIN_FIELD(61, 62, 0x894, 0x4, 16, 6),
+ PIN_FIELD(63, 66, 0x898, 0x4, 8, 6),
+ PIN_FIELD(67, 68, 0x89c, 0x4, 12, 6),
+ PIN_FIELD(69, 70, 0x89c, 0x4, 0, 6),
+ PIN_FIELD(71, 72, 0xb90, 0x4, 0, 6),
+ PIN_FIELD(73, 86, 0xb90, 0x4, 24, 6),
+ PIN_FIELD(87, 90, 0xc90, 0x4, 0, 6),
+ PIN_FIELD(91, 102, 0xb9c, 0x4, 12, 6),
};
static const struct mtk_pin_reg_calc mt7622_reg_cals[PINCTRL_PIN_REG_MAX] = {
[PINCTRL_PIN_REG_RDSEL] = MTK_RANGE(mt7622_pin_rdsel_range),
};
-static const struct pinctrl_pin_desc mt7622_pins[] = {
- PINCTRL_PIN(0, "GPIO_A"),
- PINCTRL_PIN(1, "I2S1_IN"),
- PINCTRL_PIN(2, "I2S1_OUT"),
- PINCTRL_PIN(3, "I2S_BCLK"),
- PINCTRL_PIN(4, "I2S_WS"),
- PINCTRL_PIN(5, "I2S_MCLK"),
- PINCTRL_PIN(6, "TXD0"),
- PINCTRL_PIN(7, "RXD0"),
- PINCTRL_PIN(8, "SPI_WP"),
- PINCTRL_PIN(9, "SPI_HOLD"),
- PINCTRL_PIN(10, "SPI_CLK"),
- PINCTRL_PIN(11, "SPI_MOSI"),
- PINCTRL_PIN(12, "SPI_MISO"),
- PINCTRL_PIN(13, "SPI_CS"),
- PINCTRL_PIN(14, "I2C_SDA"),
- PINCTRL_PIN(15, "I2C_SCL"),
- PINCTRL_PIN(16, "I2S2_IN"),
- PINCTRL_PIN(17, "I2S3_IN"),
- PINCTRL_PIN(18, "I2S4_IN"),
- PINCTRL_PIN(19, "I2S2_OUT"),
- PINCTRL_PIN(20, "I2S3_OUT"),
- PINCTRL_PIN(21, "I2S4_OUT"),
- PINCTRL_PIN(22, "GPIO_B"),
- PINCTRL_PIN(23, "MDC"),
- PINCTRL_PIN(24, "MDIO"),
- PINCTRL_PIN(25, "G2_TXD0"),
- PINCTRL_PIN(26, "G2_TXD1"),
- PINCTRL_PIN(27, "G2_TXD2"),
- PINCTRL_PIN(28, "G2_TXD3"),
- PINCTRL_PIN(29, "G2_TXEN"),
- PINCTRL_PIN(30, "G2_TXC"),
- PINCTRL_PIN(31, "G2_RXD0"),
- PINCTRL_PIN(32, "G2_RXD1"),
- PINCTRL_PIN(33, "G2_RXD2"),
- PINCTRL_PIN(34, "G2_RXD3"),
- PINCTRL_PIN(35, "G2_RXDV"),
- PINCTRL_PIN(36, "G2_RXC"),
- PINCTRL_PIN(37, "NCEB"),
- PINCTRL_PIN(38, "NWEB"),
- PINCTRL_PIN(39, "NREB"),
- PINCTRL_PIN(40, "NDL4"),
- PINCTRL_PIN(41, "NDL5"),
- PINCTRL_PIN(42, "NDL6"),
- PINCTRL_PIN(43, "NDL7"),
- PINCTRL_PIN(44, "NRB"),
- PINCTRL_PIN(45, "NCLE"),
- PINCTRL_PIN(46, "NALE"),
- PINCTRL_PIN(47, "NDL0"),
- PINCTRL_PIN(48, "NDL1"),
- PINCTRL_PIN(49, "NDL2"),
- PINCTRL_PIN(50, "NDL3"),
- PINCTRL_PIN(51, "MDI_TP_P0"),
- PINCTRL_PIN(52, "MDI_TN_P0"),
- PINCTRL_PIN(53, "MDI_RP_P0"),
- PINCTRL_PIN(54, "MDI_RN_P0"),
- PINCTRL_PIN(55, "MDI_TP_P1"),
- PINCTRL_PIN(56, "MDI_TN_P1"),
- PINCTRL_PIN(57, "MDI_RP_P1"),
- PINCTRL_PIN(58, "MDI_RN_P1"),
- PINCTRL_PIN(59, "MDI_RP_P2"),
- PINCTRL_PIN(60, "MDI_RN_P2"),
- PINCTRL_PIN(61, "MDI_TP_P2"),
- PINCTRL_PIN(62, "MDI_TN_P2"),
- PINCTRL_PIN(63, "MDI_TP_P3"),
- PINCTRL_PIN(64, "MDI_TN_P3"),
- PINCTRL_PIN(65, "MDI_RP_P3"),
- PINCTRL_PIN(66, "MDI_RN_P3"),
- PINCTRL_PIN(67, "MDI_RP_P4"),
- PINCTRL_PIN(68, "MDI_RN_P4"),
- PINCTRL_PIN(69, "MDI_TP_P4"),
- PINCTRL_PIN(70, "MDI_TN_P4"),
- PINCTRL_PIN(71, "PMIC_SCL"),
- PINCTRL_PIN(72, "PMIC_SDA"),
- PINCTRL_PIN(73, "SPIC1_CLK"),
- PINCTRL_PIN(74, "SPIC1_MOSI"),
- PINCTRL_PIN(75, "SPIC1_MISO"),
- PINCTRL_PIN(76, "SPIC1_CS"),
- PINCTRL_PIN(77, "GPIO_D"),
- PINCTRL_PIN(78, "WATCHDOG"),
- PINCTRL_PIN(79, "RTS3_N"),
- PINCTRL_PIN(80, "CTS3_N"),
- PINCTRL_PIN(81, "TXD3"),
- PINCTRL_PIN(82, "RXD3"),
- PINCTRL_PIN(83, "PERST0_N"),
- PINCTRL_PIN(84, "PERST1_N"),
- PINCTRL_PIN(85, "WLED_N"),
- PINCTRL_PIN(86, "EPHY_LED0_N"),
- PINCTRL_PIN(87, "AUXIN0"),
- PINCTRL_PIN(88, "AUXIN1"),
- PINCTRL_PIN(89, "AUXIN2"),
- PINCTRL_PIN(90, "AUXIN3"),
- PINCTRL_PIN(91, "TXD4"),
- PINCTRL_PIN(92, "RXD4"),
- PINCTRL_PIN(93, "RTS4_N"),
- PINCTRL_PIN(94, "CTS4_N"),
- PINCTRL_PIN(95, "PWM1"),
- PINCTRL_PIN(96, "PWM2"),
- PINCTRL_PIN(97, "PWM3"),
- PINCTRL_PIN(98, "PWM4"),
- PINCTRL_PIN(99, "PWM5"),
- PINCTRL_PIN(100, "PWM6"),
- PINCTRL_PIN(101, "PWM7"),
- PINCTRL_PIN(102, "GPIO_E"),
+static const struct mtk_pin_desc mt7622_pins[] = {
+ MT7622_PIN(0, "GPIO_A"),
+ MT7622_PIN(1, "I2S1_IN"),
+ MT7622_PIN(2, "I2S1_OUT"),
+ MT7622_PIN(3, "I2S_BCLK"),
+ MT7622_PIN(4, "I2S_WS"),
+ MT7622_PIN(5, "I2S_MCLK"),
+ MT7622_PIN(6, "TXD0"),
+ MT7622_PIN(7, "RXD0"),
+ MT7622_PIN(8, "SPI_WP"),
+ MT7622_PIN(9, "SPI_HOLD"),
+ MT7622_PIN(10, "SPI_CLK"),
+ MT7622_PIN(11, "SPI_MOSI"),
+ MT7622_PIN(12, "SPI_MISO"),
+ MT7622_PIN(13, "SPI_CS"),
+ MT7622_PIN(14, "I2C_SDA"),
+ MT7622_PIN(15, "I2C_SCL"),
+ MT7622_PIN(16, "I2S2_IN"),
+ MT7622_PIN(17, "I2S3_IN"),
+ MT7622_PIN(18, "I2S4_IN"),
+ MT7622_PIN(19, "I2S2_OUT"),
+ MT7622_PIN(20, "I2S3_OUT"),
+ MT7622_PIN(21, "I2S4_OUT"),
+ MT7622_PIN(22, "GPIO_B"),
+ MT7622_PIN(23, "MDC"),
+ MT7622_PIN(24, "MDIO"),
+ MT7622_PIN(25, "G2_TXD0"),
+ MT7622_PIN(26, "G2_TXD1"),
+ MT7622_PIN(27, "G2_TXD2"),
+ MT7622_PIN(28, "G2_TXD3"),
+ MT7622_PIN(29, "G2_TXEN"),
+ MT7622_PIN(30, "G2_TXC"),
+ MT7622_PIN(31, "G2_RXD0"),
+ MT7622_PIN(32, "G2_RXD1"),
+ MT7622_PIN(33, "G2_RXD2"),
+ MT7622_PIN(34, "G2_RXD3"),
+ MT7622_PIN(35, "G2_RXDV"),
+ MT7622_PIN(36, "G2_RXC"),
+ MT7622_PIN(37, "NCEB"),
+ MT7622_PIN(38, "NWEB"),
+ MT7622_PIN(39, "NREB"),
+ MT7622_PIN(40, "NDL4"),
+ MT7622_PIN(41, "NDL5"),
+ MT7622_PIN(42, "NDL6"),
+ MT7622_PIN(43, "NDL7"),
+ MT7622_PIN(44, "NRB"),
+ MT7622_PIN(45, "NCLE"),
+ MT7622_PIN(46, "NALE"),
+ MT7622_PIN(47, "NDL0"),
+ MT7622_PIN(48, "NDL1"),
+ MT7622_PIN(49, "NDL2"),
+ MT7622_PIN(50, "NDL3"),
+ MT7622_PIN(51, "MDI_TP_P0"),
+ MT7622_PIN(52, "MDI_TN_P0"),
+ MT7622_PIN(53, "MDI_RP_P0"),
+ MT7622_PIN(54, "MDI_RN_P0"),
+ MT7622_PIN(55, "MDI_TP_P1"),
+ MT7622_PIN(56, "MDI_TN_P1"),
+ MT7622_PIN(57, "MDI_RP_P1"),
+ MT7622_PIN(58, "MDI_RN_P1"),
+ MT7622_PIN(59, "MDI_RP_P2"),
+ MT7622_PIN(60, "MDI_RN_P2"),
+ MT7622_PIN(61, "MDI_TP_P2"),
+ MT7622_PIN(62, "MDI_TN_P2"),
+ MT7622_PIN(63, "MDI_TP_P3"),
+ MT7622_PIN(64, "MDI_TN_P3"),
+ MT7622_PIN(65, "MDI_RP_P3"),
+ MT7622_PIN(66, "MDI_RN_P3"),
+ MT7622_PIN(67, "MDI_RP_P4"),
+ MT7622_PIN(68, "MDI_RN_P4"),
+ MT7622_PIN(69, "MDI_TP_P4"),
+ MT7622_PIN(70, "MDI_TN_P4"),
+ MT7622_PIN(71, "PMIC_SCL"),
+ MT7622_PIN(72, "PMIC_SDA"),
+ MT7622_PIN(73, "SPIC1_CLK"),
+ MT7622_PIN(74, "SPIC1_MOSI"),
+ MT7622_PIN(75, "SPIC1_MISO"),
+ MT7622_PIN(76, "SPIC1_CS"),
+ MT7622_PIN(77, "GPIO_D"),
+ MT7622_PIN(78, "WATCHDOG"),
+ MT7622_PIN(79, "RTS3_N"),
+ MT7622_PIN(80, "CTS3_N"),
+ MT7622_PIN(81, "TXD3"),
+ MT7622_PIN(82, "RXD3"),
+ MT7622_PIN(83, "PERST0_N"),
+ MT7622_PIN(84, "PERST1_N"),
+ MT7622_PIN(85, "WLED_N"),
+ MT7622_PIN(86, "EPHY_LED0_N"),
+ MT7622_PIN(87, "AUXIN0"),
+ MT7622_PIN(88, "AUXIN1"),
+ MT7622_PIN(89, "AUXIN2"),
+ MT7622_PIN(90, "AUXIN3"),
+ MT7622_PIN(91, "TXD4"),
+ MT7622_PIN(92, "RXD4"),
+ MT7622_PIN(93, "RTS4_N"),
+ MT7622_PIN(94, "CTS4_N"),
+ MT7622_PIN(95, "PWM1"),
+ MT7622_PIN(96, "PWM2"),
+ MT7622_PIN(97, "PWM3"),
+ MT7622_PIN(98, "PWM4"),
+ MT7622_PIN(99, "PWM5"),
+ MT7622_PIN(100, "PWM6"),
+ MT7622_PIN(101, "PWM7"),
+ MT7622_PIN(102, "GPIO_E"),
};
/* List all groups consisting of these pins dedicated to the enablement of
{"watchdog", mt7622_wdt_groups, ARRAY_SIZE(mt7622_wdt_groups)},
};
-static const struct pinconf_generic_params mtk_custom_bindings[] = {
- {"mediatek,tdsel", MTK_PIN_CONFIG_TDSEL, 0},
- {"mediatek,rdsel", MTK_PIN_CONFIG_RDSEL, 0},
-};
-
-#ifdef CONFIG_DEBUG_FS
-static const struct pin_config_item mtk_conf_items[] = {
- PCONFDUMP(MTK_PIN_CONFIG_TDSEL, "tdsel", NULL, true),
- PCONFDUMP(MTK_PIN_CONFIG_RDSEL, "rdsel", NULL, true),
-};
-#endif
-
static const struct mtk_eint_hw mt7622_eint_hw = {
.port_mask = 7,
.ports = 7,
.funcs = mt7622_functions,
.nfuncs = ARRAY_SIZE(mt7622_functions),
.eint_hw = &mt7622_eint_hw,
-};
-
-static void mtk_w32(struct mtk_pinctrl *pctl, u32 reg, u32 val)
-{
- writel_relaxed(val, pctl->base + reg);
-}
-
-static u32 mtk_r32(struct mtk_pinctrl *pctl, u32 reg)
-{
- return readl_relaxed(pctl->base + reg);
-}
-
-static void mtk_rmw(struct mtk_pinctrl *pctl, u32 reg, u32 mask, u32 set)
-{
- u32 val;
-
- val = mtk_r32(pctl, reg);
- val &= ~mask;
- val |= set;
- mtk_w32(pctl, reg, val);
-}
-
-static int mtk_hw_pin_field_lookup(struct mtk_pinctrl *hw, int pin,
- const struct mtk_pin_reg_calc *rc,
- struct mtk_pin_field *pfd)
-{
- const struct mtk_pin_field_calc *c, *e;
- u32 bits;
-
- c = rc->range;
- e = c + rc->nranges;
-
- while (c < e) {
- if (pin >= c->s_pin && pin <= c->e_pin)
- break;
- c++;
- }
-
- if (c >= e) {
- dev_err(hw->dev, "Out of range for pin = %d\n", pin);
- return -EINVAL;
- }
-
- /* Caculated bits as the overall offset the pin is located at */
- bits = c->s_bit + (pin - c->s_pin) * (c->x_bits);
-
- /* Fill pfd from bits and 32-bit register applied is assumed */
- pfd->offset = c->s_addr + c->x_addrs * (bits / 32);
- pfd->bitpos = bits % 32;
- pfd->mask = (1 << c->x_bits) - 1;
-
- /* pfd->next is used for indicating that bit wrapping-around happens
- * which requires the manipulation for bit 0 starting in the next
- * register to form the complete field read/write.
- */
- pfd->next = pfd->bitpos + c->x_bits - 1 > 31 ? c->x_addrs : 0;
-
- return 0;
-}
-
-static int mtk_hw_pin_field_get(struct mtk_pinctrl *hw, int pin,
- int field, struct mtk_pin_field *pfd)
-{
- const struct mtk_pin_reg_calc *rc;
-
- if (field < 0 || field >= PINCTRL_PIN_REG_MAX) {
- dev_err(hw->dev, "Invalid Field %d\n", field);
- return -EINVAL;
- }
-
- if (hw->soc->reg_cal && hw->soc->reg_cal[field].range) {
- rc = &hw->soc->reg_cal[field];
- } else {
- dev_err(hw->dev, "Undefined range for field %d\n", field);
- return -EINVAL;
- }
-
- return mtk_hw_pin_field_lookup(hw, pin, rc, pfd);
-}
-
-static void mtk_hw_bits_part(struct mtk_pin_field *pf, int *h, int *l)
-{
- *l = 32 - pf->bitpos;
- *h = get_count_order(pf->mask) - *l;
-}
-
-static void mtk_hw_write_cross_field(struct mtk_pinctrl *hw,
- struct mtk_pin_field *pf, int value)
-{
- int nbits_l, nbits_h;
-
- mtk_hw_bits_part(pf, &nbits_h, &nbits_l);
-
- mtk_rmw(hw, pf->offset, pf->mask << pf->bitpos,
- (value & pf->mask) << pf->bitpos);
-
- mtk_rmw(hw, pf->offset + pf->next, BIT(nbits_h) - 1,
- (value & pf->mask) >> nbits_l);
-}
-
-static void mtk_hw_read_cross_field(struct mtk_pinctrl *hw,
- struct mtk_pin_field *pf, int *value)
-{
- int nbits_l, nbits_h, h, l;
-
- mtk_hw_bits_part(pf, &nbits_h, &nbits_l);
-
- l = (mtk_r32(hw, pf->offset) >> pf->bitpos) & (BIT(nbits_l) - 1);
- h = (mtk_r32(hw, pf->offset + pf->next)) & (BIT(nbits_h) - 1);
-
- *value = (h << nbits_l) | l;
-}
-
-static int mtk_hw_set_value(struct mtk_pinctrl *hw, int pin, int field,
- int value)
-{
- struct mtk_pin_field pf;
- int err;
-
- err = mtk_hw_pin_field_get(hw, pin, field, &pf);
- if (err)
- return err;
-
- if (!pf.next)
- mtk_rmw(hw, pf.offset, pf.mask << pf.bitpos,
- (value & pf.mask) << pf.bitpos);
- else
- mtk_hw_write_cross_field(hw, &pf, value);
-
- return 0;
-}
-
-static int mtk_hw_get_value(struct mtk_pinctrl *hw, int pin, int field,
- int *value)
-{
- struct mtk_pin_field pf;
- int err;
-
- err = mtk_hw_pin_field_get(hw, pin, field, &pf);
- if (err)
- return err;
-
- if (!pf.next)
- *value = (mtk_r32(hw, pf.offset) >> pf.bitpos) & pf.mask;
- else
- mtk_hw_read_cross_field(hw, &pf, value);
-
- return 0;
-}
-
-static int mtk_pinmux_set_mux(struct pinctrl_dev *pctldev,
- unsigned int selector, unsigned int group)
-{
- struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
- struct function_desc *func;
- struct group_desc *grp;
- int i;
-
- func = pinmux_generic_get_function(pctldev, selector);
- if (!func)
- return -EINVAL;
-
- grp = pinctrl_generic_get_group(pctldev, group);
- if (!grp)
- return -EINVAL;
-
- dev_dbg(pctldev->dev, "enable function %s group %s\n",
- func->name, grp->name);
-
- for (i = 0; i < grp->num_pins; i++) {
- int *pin_modes = grp->data;
-
- mtk_hw_set_value(hw, grp->pins[i], PINCTRL_PIN_REG_MODE,
- pin_modes[i]);
- }
-
- return 0;
-}
-
-static int mtk_pinmux_gpio_request_enable(struct pinctrl_dev *pctldev,
- struct pinctrl_gpio_range *range,
- unsigned int pin)
-{
- struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
-
- return mtk_hw_set_value(hw, pin, PINCTRL_PIN_REG_MODE, MTK_GPIO_MODE);
-}
-
-static int mtk_pinmux_gpio_set_direction(struct pinctrl_dev *pctldev,
- struct pinctrl_gpio_range *range,
- unsigned int pin, bool input)
-{
- struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
-
- /* hardware would take 0 as input direction */
- return mtk_hw_set_value(hw, pin, PINCTRL_PIN_REG_DIR, !input);
-}
-
-static int mtk_pinconf_get(struct pinctrl_dev *pctldev,
- unsigned int pin, unsigned long *config)
-{
- struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
- u32 param = pinconf_to_config_param(*config);
- int val, val2, err, reg, ret = 1;
-
- switch (param) {
- case PIN_CONFIG_BIAS_DISABLE:
- err = mtk_hw_get_value(hw, pin, PINCTRL_PIN_REG_PU, &val);
- if (err)
- return err;
-
- err = mtk_hw_get_value(hw, pin, PINCTRL_PIN_REG_PD, &val2);
- if (err)
- return err;
-
- if (val || val2)
- return -EINVAL;
-
- break;
- case PIN_CONFIG_BIAS_PULL_UP:
- case PIN_CONFIG_BIAS_PULL_DOWN:
- case PIN_CONFIG_SLEW_RATE:
- reg = (param == PIN_CONFIG_BIAS_PULL_UP) ?
- PINCTRL_PIN_REG_PU :
- (param == PIN_CONFIG_BIAS_PULL_DOWN) ?
- PINCTRL_PIN_REG_PD : PINCTRL_PIN_REG_SR;
-
- err = mtk_hw_get_value(hw, pin, reg, &val);
- if (err)
- return err;
-
- if (!val)
- return -EINVAL;
-
- break;
- case PIN_CONFIG_INPUT_ENABLE:
- case PIN_CONFIG_OUTPUT_ENABLE:
- err = mtk_hw_get_value(hw, pin, PINCTRL_PIN_REG_DIR, &val);
- if (err)
- return err;
-
- /* HW takes input mode as zero; output mode as non-zero */
- if ((val && param == PIN_CONFIG_INPUT_ENABLE) ||
- (!val && param == PIN_CONFIG_OUTPUT_ENABLE))
- return -EINVAL;
-
- break;
- case PIN_CONFIG_INPUT_SCHMITT_ENABLE:
- err = mtk_hw_get_value(hw, pin, PINCTRL_PIN_REG_DIR, &val);
- if (err)
- return err;
-
- err = mtk_hw_get_value(hw, pin, PINCTRL_PIN_REG_SMT, &val2);
- if (err)
- return err;
-
- if (val || !val2)
- return -EINVAL;
-
- break;
- case PIN_CONFIG_DRIVE_STRENGTH:
- err = mtk_hw_get_value(hw, pin, PINCTRL_PIN_REG_E4, &val);
- if (err)
- return err;
-
- err = mtk_hw_get_value(hw, pin, PINCTRL_PIN_REG_E8, &val2);
- if (err)
- return err;
-
- /* 4mA when (e8, e4) = (0, 0); 8mA when (e8, e4) = (0, 1)
- * 12mA when (e8, e4) = (1, 0); 16mA when (e8, e4) = (1, 1)
- */
- ret = ((val2 << 1) + val + 1) * 4;
-
- break;
- case MTK_PIN_CONFIG_TDSEL:
- case MTK_PIN_CONFIG_RDSEL:
- reg = (param == MTK_PIN_CONFIG_TDSEL) ?
- PINCTRL_PIN_REG_TDSEL : PINCTRL_PIN_REG_RDSEL;
-
- err = mtk_hw_get_value(hw, pin, reg, &val);
- if (err)
- return err;
-
- ret = val;
-
- break;
- default:
- return -ENOTSUPP;
- }
-
- *config = pinconf_to_config_packed(param, ret);
-
- return 0;
-}
-
-static int mtk_pinconf_set(struct pinctrl_dev *pctldev, unsigned int pin,
- unsigned long *configs, unsigned int num_configs)
-{
- struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
- u32 reg, param, arg;
- int cfg, err = 0;
-
- for (cfg = 0; cfg < num_configs; cfg++) {
- param = pinconf_to_config_param(configs[cfg]);
- arg = pinconf_to_config_argument(configs[cfg]);
-
- switch (param) {
- case PIN_CONFIG_BIAS_DISABLE:
- case PIN_CONFIG_BIAS_PULL_UP:
- case PIN_CONFIG_BIAS_PULL_DOWN:
- arg = (param == PIN_CONFIG_BIAS_DISABLE) ? 0 :
- (param == PIN_CONFIG_BIAS_PULL_UP) ? 1 : 2;
-
- err = mtk_hw_set_value(hw, pin, PINCTRL_PIN_REG_PU,
- arg & 1);
- if (err)
- goto err;
-
- err = mtk_hw_set_value(hw, pin, PINCTRL_PIN_REG_PD,
- !!(arg & 2));
- if (err)
- goto err;
- break;
- case PIN_CONFIG_OUTPUT_ENABLE:
- err = mtk_hw_set_value(hw, pin, PINCTRL_PIN_REG_SMT,
- MTK_DISABLE);
- if (err)
- goto err;
- /* else: fall through */
- case PIN_CONFIG_INPUT_ENABLE:
- case PIN_CONFIG_SLEW_RATE:
- reg = (param == PIN_CONFIG_SLEW_RATE) ?
- PINCTRL_PIN_REG_SR : PINCTRL_PIN_REG_DIR;
-
- arg = (param == PIN_CONFIG_INPUT_ENABLE) ? 0 :
- (param == PIN_CONFIG_OUTPUT_ENABLE) ? 1 : arg;
- err = mtk_hw_set_value(hw, pin, reg, arg);
- if (err)
- goto err;
-
- break;
- case PIN_CONFIG_OUTPUT:
- err = mtk_hw_set_value(hw, pin, PINCTRL_PIN_REG_DIR,
- MTK_OUTPUT);
- if (err)
- goto err;
-
- err = mtk_hw_set_value(hw, pin, PINCTRL_PIN_REG_DO,
- arg);
- if (err)
- goto err;
- break;
- case PIN_CONFIG_INPUT_SCHMITT_ENABLE:
- /* arg = 1: Input mode & SMT enable ;
- * arg = 0: Output mode & SMT disable
- */
- arg = arg ? 2 : 1;
- err = mtk_hw_set_value(hw, pin, PINCTRL_PIN_REG_DIR,
- arg & 1);
- if (err)
- goto err;
-
- err = mtk_hw_set_value(hw, pin, PINCTRL_PIN_REG_SMT,
- !!(arg & 2));
- if (err)
- goto err;
- break;
- case PIN_CONFIG_DRIVE_STRENGTH:
- /* 4mA when (e8, e4) = (0, 0);
- * 8mA when (e8, e4) = (0, 1);
- * 12mA when (e8, e4) = (1, 0);
- * 16mA when (e8, e4) = (1, 1)
- */
- if (!(arg % 4) && (arg >= 4 && arg <= 16)) {
- arg = arg / 4 - 1;
- err = mtk_hw_set_value(hw, pin,
- PINCTRL_PIN_REG_E4,
- arg & 0x1);
- if (err)
- goto err;
-
- err = mtk_hw_set_value(hw, pin,
- PINCTRL_PIN_REG_E8,
- (arg & 0x2) >> 1);
- if (err)
- goto err;
- } else {
- err = -ENOTSUPP;
- }
- break;
- case MTK_PIN_CONFIG_TDSEL:
- case MTK_PIN_CONFIG_RDSEL:
- reg = (param == MTK_PIN_CONFIG_TDSEL) ?
- PINCTRL_PIN_REG_TDSEL : PINCTRL_PIN_REG_RDSEL;
-
- err = mtk_hw_set_value(hw, pin, reg, arg);
- if (err)
- goto err;
- break;
- default:
- err = -ENOTSUPP;
- }
- }
-err:
- return err;
-}
-
-static int mtk_pinconf_group_get(struct pinctrl_dev *pctldev,
- unsigned int group, unsigned long *config)
-{
- const unsigned int *pins;
- unsigned int i, npins, old = 0;
- int ret;
-
- ret = pinctrl_generic_get_group_pins(pctldev, group, &pins, &npins);
- if (ret)
- return ret;
-
- for (i = 0; i < npins; i++) {
- if (mtk_pinconf_get(pctldev, pins[i], config))
- return -ENOTSUPP;
-
- /* configs do not match between two pins */
- if (i && old != *config)
- return -ENOTSUPP;
-
- old = *config;
- }
-
- return 0;
-}
-
-static int mtk_pinconf_group_set(struct pinctrl_dev *pctldev,
- unsigned int group, unsigned long *configs,
- unsigned int num_configs)
-{
- const unsigned int *pins;
- unsigned int i, npins;
- int ret;
-
- ret = pinctrl_generic_get_group_pins(pctldev, group, &pins, &npins);
- if (ret)
- return ret;
-
- for (i = 0; i < npins; i++) {
- ret = mtk_pinconf_set(pctldev, pins[i], configs, num_configs);
- if (ret)
- return ret;
- }
-
- return 0;
-}
-
-static const struct pinctrl_ops mtk_pctlops = {
- .get_groups_count = pinctrl_generic_get_group_count,
- .get_group_name = pinctrl_generic_get_group_name,
- .get_group_pins = pinctrl_generic_get_group_pins,
- .dt_node_to_map = pinconf_generic_dt_node_to_map_all,
- .dt_free_map = pinconf_generic_dt_free_map,
-};
-
-static const struct pinmux_ops mtk_pmxops = {
- .get_functions_count = pinmux_generic_get_function_count,
- .get_function_name = pinmux_generic_get_function_name,
- .get_function_groups = pinmux_generic_get_function_groups,
- .set_mux = mtk_pinmux_set_mux,
- .gpio_request_enable = mtk_pinmux_gpio_request_enable,
- .gpio_set_direction = mtk_pinmux_gpio_set_direction,
- .strict = true,
-};
-
-static const struct pinconf_ops mtk_confops = {
- .is_generic = true,
- .pin_config_get = mtk_pinconf_get,
- .pin_config_set = mtk_pinconf_set,
- .pin_config_group_get = mtk_pinconf_group_get,
- .pin_config_group_set = mtk_pinconf_group_set,
- .pin_config_config_dbg_show = pinconf_generic_dump_config,
-};
-
-static struct pinctrl_desc mtk_desc = {
- .name = PINCTRL_PINCTRL_DEV,
- .pctlops = &mtk_pctlops,
- .pmxops = &mtk_pmxops,
- .confops = &mtk_confops,
- .owner = THIS_MODULE,
-};
-
-static int mtk_gpio_get(struct gpio_chip *chip, unsigned int gpio)
-{
- struct mtk_pinctrl *hw = gpiochip_get_data(chip);
- int value, err;
-
- err = mtk_hw_get_value(hw, gpio, PINCTRL_PIN_REG_DI, &value);
- if (err)
- return err;
-
- return !!value;
-}
-
-static void mtk_gpio_set(struct gpio_chip *chip, unsigned int gpio, int value)
-{
- struct mtk_pinctrl *hw = gpiochip_get_data(chip);
-
- mtk_hw_set_value(hw, gpio, PINCTRL_PIN_REG_DO, !!value);
-}
-
-static int mtk_gpio_direction_input(struct gpio_chip *chip, unsigned int gpio)
-{
- return pinctrl_gpio_direction_input(chip->base + gpio);
-}
-
-static int mtk_gpio_direction_output(struct gpio_chip *chip, unsigned int gpio,
- int value)
-{
- mtk_gpio_set(chip, gpio, value);
-
- return pinctrl_gpio_direction_output(chip->base + gpio);
-}
-
-static int mtk_gpio_to_irq(struct gpio_chip *chip, unsigned int offset)
-{
- struct mtk_pinctrl *hw = gpiochip_get_data(chip);
- unsigned long eint_n;
-
- if (!hw->eint)
- return -ENOTSUPP;
-
- eint_n = offset;
-
- return mtk_eint_find_irq(hw->eint, eint_n);
-}
-
-static int mtk_gpio_set_config(struct gpio_chip *chip, unsigned int offset,
- unsigned long config)
-{
- struct mtk_pinctrl *hw = gpiochip_get_data(chip);
- unsigned long eint_n;
- u32 debounce;
-
- if (!hw->eint ||
- pinconf_to_config_param(config) != PIN_CONFIG_INPUT_DEBOUNCE)
- return -ENOTSUPP;
-
- debounce = pinconf_to_config_argument(config);
- eint_n = offset;
-
- return mtk_eint_set_debounce(hw->eint, eint_n, debounce);
-}
-
-static int mtk_build_gpiochip(struct mtk_pinctrl *hw, struct device_node *np)
-{
- struct gpio_chip *chip = &hw->chip;
- int ret;
-
- chip->label = PINCTRL_PINCTRL_DEV;
- chip->parent = hw->dev;
- chip->request = gpiochip_generic_request;
- chip->free = gpiochip_generic_free;
- chip->direction_input = mtk_gpio_direction_input;
- chip->direction_output = mtk_gpio_direction_output;
- chip->get = mtk_gpio_get;
- chip->set = mtk_gpio_set;
- chip->to_irq = mtk_gpio_to_irq,
- chip->set_config = mtk_gpio_set_config,
- chip->base = -1;
- chip->ngpio = hw->soc->npins;
- chip->of_node = np;
- chip->of_gpio_n_cells = 2;
-
- ret = gpiochip_add_data(chip, hw);
- if (ret < 0)
- return ret;
-
- /* Just for backward compatible for these old pinctrl nodes without
- * "gpio-ranges" property. Otherwise, called directly from a
- * DeviceTree-supported pinctrl driver is DEPRECATED.
- * Please see Section 2.1 of
- * Documentation/devicetree/bindings/gpio/gpio.txt on how to
- * bind pinctrl and gpio drivers via the "gpio-ranges" property.
- */
- if (!of_find_property(np, "gpio-ranges", NULL)) {
- ret = gpiochip_add_pin_range(chip, dev_name(hw->dev), 0, 0,
- chip->ngpio);
- if (ret < 0) {
- gpiochip_remove(chip);
- return ret;
- }
- }
-
- return 0;
-}
-
-static int mtk_build_groups(struct mtk_pinctrl *hw)
-{
- int err, i;
-
- for (i = 0; i < hw->soc->ngrps; i++) {
- const struct group_desc *group = hw->soc->grps + i;
-
- err = pinctrl_generic_add_group(hw->pctrl, group->name,
- group->pins, group->num_pins,
- group->data);
- if (err < 0) {
- dev_err(hw->dev, "Failed to register group %s\n",
- group->name);
- return err;
- }
- }
-
- return 0;
-}
-
-static int mtk_build_functions(struct mtk_pinctrl *hw)
-{
- int i, err;
-
- for (i = 0; i < hw->soc->nfuncs ; i++) {
- const struct function_desc *func = hw->soc->funcs + i;
-
- err = pinmux_generic_add_function(hw->pctrl, func->name,
- func->group_names,
- func->num_group_names,
- func->data);
- if (err < 0) {
- dev_err(hw->dev, "Failed to register function %s\n",
- func->name);
- return err;
- }
- }
-
- return 0;
-}
-
-static int mtk_xt_get_gpio_n(void *data, unsigned long eint_n,
- unsigned int *gpio_n,
- struct gpio_chip **gpio_chip)
-{
- struct mtk_pinctrl *hw = (struct mtk_pinctrl *)data;
-
- *gpio_chip = &hw->chip;
- *gpio_n = eint_n;
-
- return 0;
-}
-
-static int mtk_xt_get_gpio_state(void *data, unsigned long eint_n)
-{
- struct mtk_pinctrl *hw = (struct mtk_pinctrl *)data;
- struct gpio_chip *gpio_chip;
- unsigned int gpio_n;
- int err;
-
- err = mtk_xt_get_gpio_n(hw, eint_n, &gpio_n, &gpio_chip);
- if (err)
- return err;
-
- return mtk_gpio_get(gpio_chip, gpio_n);
-}
-
-static int mtk_xt_set_gpio_as_eint(void *data, unsigned long eint_n)
-{
- struct mtk_pinctrl *hw = (struct mtk_pinctrl *)data;
- struct gpio_chip *gpio_chip;
- unsigned int gpio_n;
- int err;
-
- err = mtk_xt_get_gpio_n(hw, eint_n, &gpio_n, &gpio_chip);
- if (err)
- return err;
-
- err = mtk_hw_set_value(hw, gpio_n, PINCTRL_PIN_REG_MODE,
- MTK_GPIO_MODE);
- if (err)
- return err;
-
- err = mtk_hw_set_value(hw, gpio_n, PINCTRL_PIN_REG_DIR, MTK_INPUT);
- if (err)
- return err;
-
- err = mtk_hw_set_value(hw, gpio_n, PINCTRL_PIN_REG_SMT, MTK_ENABLE);
- if (err)
- return err;
-
- return 0;
-}
-
-static const struct mtk_eint_xt mtk_eint_xt = {
- .get_gpio_n = mtk_xt_get_gpio_n,
- .get_gpio_state = mtk_xt_get_gpio_state,
- .set_gpio_as_eint = mtk_xt_set_gpio_as_eint,
-};
-
-static int
-mtk_build_eint(struct mtk_pinctrl *hw, struct platform_device *pdev)
-{
- struct device_node *np = pdev->dev.of_node;
- struct resource *res;
-
- if (!IS_ENABLED(CONFIG_EINT_MTK))
- return 0;
-
- if (!of_property_read_bool(np, "interrupt-controller"))
- return -ENODEV;
-
- hw->eint = devm_kzalloc(hw->dev, sizeof(*hw->eint), GFP_KERNEL);
- if (!hw->eint)
- return -ENOMEM;
-
- res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "eint");
- if (!res) {
- dev_err(&pdev->dev, "Unable to get eint resource\n");
- return -ENODEV;
- }
-
- hw->eint->base = devm_ioremap_resource(&pdev->dev, res);
- if (IS_ERR(hw->eint->base))
- return PTR_ERR(hw->eint->base);
-
- hw->eint->irq = irq_of_parse_and_map(np, 0);
- if (!hw->eint->irq)
- return -EINVAL;
-
- hw->eint->dev = &pdev->dev;
- hw->eint->hw = hw->soc->eint_hw;
- hw->eint->pctl = hw;
- hw->eint->gpio_xlate = &mtk_eint_xt;
-
- return mtk_eint_do_init(hw->eint);
-}
-
-static const struct of_device_id mtk_pinctrl_of_match[] = {
- { .compatible = "mediatek,mt7622-pinctrl", .data = &mt7622_data},
+ .gpio_m = 1,
+ .ies_present = false,
+ .base_names = mtk_default_register_base_names,
+ .nbase_names = ARRAY_SIZE(mtk_default_register_base_names),
+ .bias_disable_set = mtk_pinconf_bias_disable_set,
+ .bias_disable_get = mtk_pinconf_bias_disable_get,
+ .bias_set = mtk_pinconf_bias_set,
+ .bias_get = mtk_pinconf_bias_get,
+ .drive_set = mtk_pinconf_drive_set,
+ .drive_get = mtk_pinconf_drive_get,
+};
+
+static const struct of_device_id mt7622_pinctrl_of_match[] = {
+ { .compatible = "mediatek,mt7622-pinctrl", },
{ }
};
-static int mtk_pinctrl_probe(struct platform_device *pdev)
+static int mt7622_pinctrl_probe(struct platform_device *pdev)
{
- struct resource *res;
- struct mtk_pinctrl *hw;
- const struct of_device_id *of_id =
- of_match_device(mtk_pinctrl_of_match, &pdev->dev);
- int err;
-
- hw = devm_kzalloc(&pdev->dev, sizeof(*hw), GFP_KERNEL);
- if (!hw)
- return -ENOMEM;
-
- hw->soc = of_id->data;
-
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (!res) {
- dev_err(&pdev->dev, "missing IO resource\n");
- return -ENXIO;
- }
-
- hw->dev = &pdev->dev;
- hw->base = devm_ioremap_resource(&pdev->dev, res);
- if (IS_ERR(hw->base))
- return PTR_ERR(hw->base);
-
- /* Setup pins descriptions per SoC types */
- mtk_desc.pins = hw->soc->pins;
- mtk_desc.npins = hw->soc->npins;
- mtk_desc.num_custom_params = ARRAY_SIZE(mtk_custom_bindings);
- mtk_desc.custom_params = mtk_custom_bindings;
-#ifdef CONFIG_DEBUG_FS
- mtk_desc.custom_conf_items = mtk_conf_items;
-#endif
-
- err = devm_pinctrl_register_and_init(&pdev->dev, &mtk_desc, hw,
- &hw->pctrl);
- if (err)
- return err;
-
- /* Setup groups descriptions per SoC types */
- err = mtk_build_groups(hw);
- if (err) {
- dev_err(&pdev->dev, "Failed to build groups\n");
- return err;
- }
-
- /* Setup functions descriptions per SoC types */
- err = mtk_build_functions(hw);
- if (err) {
- dev_err(&pdev->dev, "Failed to build functions\n");
- return err;
- }
-
- /* For able to make pinctrl_claim_hogs, we must not enable pinctrl
- * until all groups and functions are being added one.
- */
- err = pinctrl_enable(hw->pctrl);
- if (err)
- return err;
-
- err = mtk_build_eint(hw, pdev);
- if (err)
- dev_warn(&pdev->dev,
- "Failed to add EINT, but pinctrl still can work\n");
-
- /* Build gpiochip should be after pinctrl_enable is done */
- err = mtk_build_gpiochip(hw, pdev->dev.of_node);
- if (err) {
- dev_err(&pdev->dev, "Failed to add gpio_chip\n");
- return err;
- }
-
- platform_set_drvdata(pdev, hw);
-
- return 0;
+ return mtk_moore_pinctrl_probe(pdev, &mt7622_data);
}
-static struct platform_driver mtk_pinctrl_driver = {
+static struct platform_driver mt7622_pinctrl_driver = {
.driver = {
- .name = "mtk-pinctrl",
- .of_match_table = mtk_pinctrl_of_match,
+ .name = "mt7622-pinctrl",
+ .of_match_table = mt7622_pinctrl_of_match,
},
- .probe = mtk_pinctrl_probe,
+ .probe = mt7622_pinctrl_probe,
};
-static int __init mtk_pinctrl_init(void)
+static int __init mt7622_pinctrl_init(void)
{
- return platform_driver_register(&mtk_pinctrl_driver);
+ return platform_driver_register(&mt7622_pinctrl_driver);
}
-arch_initcall(mtk_pinctrl_init);
+arch_initcall(mt7622_pinctrl_init);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * The MT7623 driver based on Linux generic pinctrl binding.
+ *
+ * Copyright (C) 2015 - 2018 MediaTek Inc.
+ * Author: Biao Huang <biao.huang@mediatek.com>
+ * Ryder Lee <ryder.lee@mediatek.com>
+ * Sean Wang <sean.wang@mediatek.com>
+ */
+
+#include "pinctrl-moore.h"
+
+#define PIN_BOND_REG0 0xb10
+#define PIN_BOND_REG1 0xf20
+#define PIN_BOND_REG2 0xef0
+#define BOND_PCIE_CLR (0x77 << 3)
+#define BOND_I2S_CLR 0x3
+#define BOND_MSDC0E_CLR 0x1
+
+#define PIN_FIELD15(_s_pin, _e_pin, _s_addr, _x_addrs, _s_bit, _x_bits) \
+ PIN_FIELD_CALC(_s_pin, _e_pin, 0, _s_addr, _x_addrs, _s_bit, \
+ _x_bits, 15, false)
+
+#define PIN_FIELD16(_s_pin, _e_pin, _s_addr, _x_addrs, _s_bit, _x_bits) \
+ PIN_FIELD_CALC(_s_pin, _e_pin, 0, _s_addr, _x_addrs, _s_bit, \
+ _x_bits, 16, 0)
+
+#define PINS_FIELD16(_s_pin, _e_pin, _s_addr, _x_addrs, _s_bit, _x_bits) \
+ PIN_FIELD_CALC(_s_pin, _e_pin, 0, _s_addr, _x_addrs, _s_bit, \
+ _x_bits, 16, 1)
+
+#define MT7623_PIN(_number, _name, _eint_n, _drv_grp) \
+ MTK_PIN(_number, _name, 0, _eint_n, _drv_grp)
+
+static const struct mtk_pin_field_calc mt7623_pin_mode_range[] = {
+ PIN_FIELD15(0, 278, 0x760, 0x10, 0, 3),
+};
+
+static const struct mtk_pin_field_calc mt7623_pin_dir_range[] = {
+ PIN_FIELD16(0, 175, 0x0, 0x10, 0, 1),
+ PIN_FIELD16(176, 278, 0xc0, 0x10, 0, 1),
+};
+
+static const struct mtk_pin_field_calc mt7623_pin_di_range[] = {
+ PIN_FIELD16(0, 278, 0x630, 0x10, 0, 1),
+};
+
+static const struct mtk_pin_field_calc mt7623_pin_do_range[] = {
+ PIN_FIELD16(0, 278, 0x500, 0x10, 0, 1),
+};
+
+static const struct mtk_pin_field_calc mt7623_pin_ies_range[] = {
+ PINS_FIELD16(0, 6, 0xb20, 0x10, 0, 1),
+ PINS_FIELD16(7, 9, 0xb20, 0x10, 1, 1),
+ PINS_FIELD16(10, 13, 0xb30, 0x10, 3, 1),
+ PINS_FIELD16(14, 15, 0xb30, 0x10, 13, 1),
+ PINS_FIELD16(16, 17, 0xb40, 0x10, 7, 1),
+ PINS_FIELD16(18, 29, 0xb40, 0x10, 13, 1),
+ PINS_FIELD16(30, 32, 0xb40, 0x10, 7, 1),
+ PINS_FIELD16(33, 37, 0xb40, 0x10, 13, 1),
+ PIN_FIELD16(38, 38, 0xb20, 0x10, 13, 1),
+ PINS_FIELD16(39, 42, 0xb40, 0x10, 13, 1),
+ PINS_FIELD16(43, 45, 0xb20, 0x10, 10, 1),
+ PINS_FIELD16(47, 48, 0xb20, 0x10, 11, 1),
+ PIN_FIELD16(49, 49, 0xb20, 0x10, 12, 1),
+ PINS_FIELD16(50, 52, 0xb20, 0x10, 13, 1),
+ PINS_FIELD16(53, 56, 0xb20, 0x10, 14, 1),
+ PINS_FIELD16(57, 58, 0xb20, 0x10, 15, 1),
+ PIN_FIELD16(59, 59, 0xb30, 0x10, 10, 1),
+ PINS_FIELD16(60, 62, 0xb30, 0x10, 0, 1),
+ PINS_FIELD16(63, 65, 0xb30, 0x10, 1, 1),
+ PINS_FIELD16(66, 71, 0xb30, 0x10, 2, 1),
+ PINS_FIELD16(72, 74, 0xb20, 0x10, 12, 1),
+ PINS_FIELD16(75, 76, 0xb30, 0x10, 3, 1),
+ PINS_FIELD16(77, 78, 0xb30, 0x10, 4, 1),
+ PINS_FIELD16(79, 82, 0xb30, 0x10, 5, 1),
+ PINS_FIELD16(83, 84, 0xb30, 0x10, 2, 1),
+ PIN_FIELD16(85, 85, 0xda0, 0x10, 4, 1),
+ PIN_FIELD16(86, 86, 0xd90, 0x10, 4, 1),
+ PINS_FIELD16(87, 90, 0xdb0, 0x10, 4, 1),
+ PINS_FIELD16(101, 104, 0xb30, 0x10, 6, 1),
+ PIN_FIELD16(105, 105, 0xd40, 0x10, 4, 1),
+ PIN_FIELD16(106, 106, 0xd30, 0x10, 4, 1),
+ PINS_FIELD16(107, 110, 0xd50, 0x10, 4, 1),
+ PINS_FIELD16(111, 115, 0xce0, 0x10, 4, 1),
+ PIN_FIELD16(116, 116, 0xcd0, 0x10, 4, 1),
+ PIN_FIELD16(117, 117, 0xcc0, 0x10, 4, 1),
+ PINS_FIELD16(118, 121, 0xce0, 0x10, 4, 1),
+ PINS_FIELD16(122, 125, 0xb30, 0x10, 7, 1),
+ PIN_FIELD16(126, 126, 0xb20, 0x10, 12, 1),
+ PINS_FIELD16(127, 142, 0xb30, 0x10, 9, 1),
+ PINS_FIELD16(143, 160, 0xb30, 0x10, 10, 1),
+ PINS_FIELD16(161, 168, 0xb30, 0x10, 12, 1),
+ PINS_FIELD16(169, 183, 0xb30, 0x10, 10, 1),
+ PINS_FIELD16(184, 186, 0xb30, 0x10, 9, 1),
+ PIN_FIELD16(187, 187, 0xb30, 0x10, 14, 1),
+ PIN_FIELD16(188, 188, 0xb20, 0x10, 13, 1),
+ PINS_FIELD16(189, 193, 0xb30, 0x10, 15, 1),
+ PINS_FIELD16(194, 198, 0xb40, 0x10, 0, 1),
+ PIN_FIELD16(199, 199, 0xb20, 0x10, 1, 1),
+ PINS_FIELD16(200, 202, 0xb40, 0x10, 1, 1),
+ PINS_FIELD16(203, 207, 0xb40, 0x10, 2, 1),
+ PINS_FIELD16(208, 209, 0xb40, 0x10, 3, 1),
+ PIN_FIELD16(210, 210, 0xb40, 0x10, 4, 1),
+ PINS_FIELD16(211, 235, 0xb40, 0x10, 5, 1),
+ PINS_FIELD16(236, 241, 0xb40, 0x10, 6, 1),
+ PINS_FIELD16(242, 243, 0xb40, 0x10, 7, 1),
+ PINS_FIELD16(244, 247, 0xb40, 0x10, 8, 1),
+ PIN_FIELD16(248, 248, 0xb40, 0x10, 9, 1),
+ PINS_FIELD16(249, 257, 0xfc0, 0x10, 4, 1),
+ PIN_FIELD16(258, 258, 0xcb0, 0x10, 4, 1),
+ PIN_FIELD16(259, 259, 0xc90, 0x10, 4, 1),
+ PIN_FIELD16(260, 260, 0x3a0, 0x10, 4, 1),
+ PIN_FIELD16(261, 261, 0xd50, 0x10, 4, 1),
+ PINS_FIELD16(262, 277, 0xb40, 0x10, 12, 1),
+ PIN_FIELD16(278, 278, 0xb40, 0x10, 13, 1),
+};
+
+static const struct mtk_pin_field_calc mt7623_pin_smt_range[] = {
+ PINS_FIELD16(0, 6, 0xb50, 0x10, 0, 1),
+ PINS_FIELD16(7, 9, 0xb50, 0x10, 1, 1),
+ PINS_FIELD16(10, 13, 0xb60, 0x10, 3, 1),
+ PINS_FIELD16(14, 15, 0xb60, 0x10, 13, 1),
+ PINS_FIELD16(16, 17, 0xb70, 0x10, 7, 1),
+ PINS_FIELD16(18, 29, 0xb70, 0x10, 13, 1),
+ PINS_FIELD16(30, 32, 0xb70, 0x10, 7, 1),
+ PINS_FIELD16(33, 37, 0xb70, 0x10, 13, 1),
+ PIN_FIELD16(38, 38, 0xb50, 0x10, 13, 1),
+ PINS_FIELD16(39, 42, 0xb70, 0x10, 13, 1),
+ PINS_FIELD16(43, 45, 0xb50, 0x10, 10, 1),
+ PINS_FIELD16(47, 48, 0xb50, 0x10, 11, 1),
+ PIN_FIELD16(49, 49, 0xb50, 0x10, 12, 1),
+ PINS_FIELD16(50, 52, 0xb50, 0x10, 13, 1),
+ PINS_FIELD16(53, 56, 0xb50, 0x10, 14, 1),
+ PINS_FIELD16(57, 58, 0xb50, 0x10, 15, 1),
+ PIN_FIELD16(59, 59, 0xb60, 0x10, 10, 1),
+ PINS_FIELD16(60, 62, 0xb60, 0x10, 0, 1),
+ PINS_FIELD16(63, 65, 0xb60, 0x10, 1, 1),
+ PINS_FIELD16(66, 71, 0xb60, 0x10, 2, 1),
+ PINS_FIELD16(72, 74, 0xb50, 0x10, 12, 1),
+ PINS_FIELD16(75, 76, 0xb60, 0x10, 3, 1),
+ PINS_FIELD16(77, 78, 0xb60, 0x10, 4, 1),
+ PINS_FIELD16(79, 82, 0xb60, 0x10, 5, 1),
+ PINS_FIELD16(83, 84, 0xb60, 0x10, 2, 1),
+ PIN_FIELD16(85, 85, 0xda0, 0x10, 11, 1),
+ PIN_FIELD16(86, 86, 0xd90, 0x10, 11, 1),
+ PIN_FIELD16(87, 87, 0xdc0, 0x10, 3, 1),
+ PIN_FIELD16(88, 88, 0xdc0, 0x10, 7, 1),
+ PIN_FIELD16(89, 89, 0xdc0, 0x10, 11, 1),
+ PIN_FIELD16(90, 90, 0xdc0, 0x10, 15, 1),
+ PINS_FIELD16(101, 104, 0xb60, 0x10, 6, 1),
+ PIN_FIELD16(105, 105, 0xd40, 0x10, 11, 1),
+ PIN_FIELD16(106, 106, 0xd30, 0x10, 11, 1),
+ PIN_FIELD16(107, 107, 0xd60, 0x10, 3, 1),
+ PIN_FIELD16(108, 108, 0xd60, 0x10, 7, 1),
+ PIN_FIELD16(109, 109, 0xd60, 0x10, 11, 1),
+ PIN_FIELD16(110, 110, 0xd60, 0x10, 15, 1),
+ PIN_FIELD16(111, 111, 0xd00, 0x10, 15, 1),
+ PIN_FIELD16(112, 112, 0xd00, 0x10, 11, 1),
+ PIN_FIELD16(113, 113, 0xd00, 0x10, 7, 1),
+ PIN_FIELD16(114, 114, 0xd00, 0x10, 3, 1),
+ PIN_FIELD16(115, 115, 0xd10, 0x10, 3, 1),
+ PIN_FIELD16(116, 116, 0xcd0, 0x10, 11, 1),
+ PIN_FIELD16(117, 117, 0xcc0, 0x10, 11, 1),
+ PIN_FIELD16(118, 118, 0xcf0, 0x10, 15, 1),
+ PIN_FIELD16(119, 119, 0xcf0, 0x10, 7, 1),
+ PIN_FIELD16(120, 120, 0xcf0, 0x10, 3, 1),
+ PIN_FIELD16(121, 121, 0xcf0, 0x10, 7, 1),
+ PINS_FIELD16(122, 125, 0xb60, 0x10, 7, 1),
+ PIN_FIELD16(126, 126, 0xb50, 0x10, 12, 1),
+ PINS_FIELD16(127, 142, 0xb60, 0x10, 9, 1),
+ PINS_FIELD16(143, 160, 0xb60, 0x10, 10, 1),
+ PINS_FIELD16(161, 168, 0xb60, 0x10, 12, 1),
+ PINS_FIELD16(169, 183, 0xb60, 0x10, 10, 1),
+ PINS_FIELD16(184, 186, 0xb60, 0x10, 9, 1),
+ PIN_FIELD16(187, 187, 0xb60, 0x10, 14, 1),
+ PIN_FIELD16(188, 188, 0xb50, 0x10, 13, 1),
+ PINS_FIELD16(189, 193, 0xb60, 0x10, 15, 1),
+ PINS_FIELD16(194, 198, 0xb70, 0x10, 0, 1),
+ PIN_FIELD16(199, 199, 0xb50, 0x10, 1, 1),
+ PINS_FIELD16(200, 202, 0xb70, 0x10, 1, 1),
+ PINS_FIELD16(203, 207, 0xb70, 0x10, 2, 1),
+ PINS_FIELD16(208, 209, 0xb70, 0x10, 3, 1),
+ PIN_FIELD16(210, 210, 0xb70, 0x10, 4, 1),
+ PINS_FIELD16(211, 235, 0xb70, 0x10, 5, 1),
+ PINS_FIELD16(236, 241, 0xb70, 0x10, 6, 1),
+ PINS_FIELD16(242, 243, 0xb70, 0x10, 7, 1),
+ PINS_FIELD16(244, 247, 0xb70, 0x10, 8, 1),
+ PIN_FIELD16(248, 248, 0xb70, 0x10, 9, 10),
+ PIN_FIELD16(249, 249, 0x140, 0x10, 3, 1),
+ PIN_FIELD16(250, 250, 0x130, 0x10, 15, 1),
+ PIN_FIELD16(251, 251, 0x130, 0x10, 11, 1),
+ PIN_FIELD16(252, 252, 0x130, 0x10, 7, 1),
+ PIN_FIELD16(253, 253, 0x130, 0x10, 3, 1),
+ PIN_FIELD16(254, 254, 0xf40, 0x10, 15, 1),
+ PIN_FIELD16(255, 255, 0xf40, 0x10, 11, 1),
+ PIN_FIELD16(256, 256, 0xf40, 0x10, 7, 1),
+ PIN_FIELD16(257, 257, 0xf40, 0x10, 3, 1),
+ PIN_FIELD16(258, 258, 0xcb0, 0x10, 11, 1),
+ PIN_FIELD16(259, 259, 0xc90, 0x10, 11, 1),
+ PIN_FIELD16(260, 260, 0x3a0, 0x10, 11, 1),
+ PIN_FIELD16(261, 261, 0x0b0, 0x10, 3, 1),
+ PINS_FIELD16(262, 277, 0xb70, 0x10, 12, 1),
+ PIN_FIELD16(278, 278, 0xb70, 0x10, 13, 1),
+};
+
+static const struct mtk_pin_field_calc mt7623_pin_pullen_range[] = {
+ PIN_FIELD16(0, 278, 0x150, 0x10, 0, 1),
+};
+
+static const struct mtk_pin_field_calc mt7623_pin_pullsel_range[] = {
+ PIN_FIELD16(0, 278, 0x280, 0x10, 0, 1),
+};
+
+static const struct mtk_pin_field_calc mt7623_pin_drv_range[] = {
+ PINS_FIELD16(0, 6, 0xf50, 0x10, 0, 4),
+ PINS_FIELD16(7, 9, 0xf50, 0x10, 4, 4),
+ PINS_FIELD16(10, 13, 0xf50, 0x10, 4, 4),
+ PINS_FIELD16(14, 15, 0xf50, 0x10, 12, 4),
+ PINS_FIELD16(16, 17, 0xf60, 0x10, 0, 4),
+ PINS_FIELD16(18, 21, 0xf60, 0x10, 0, 4),
+ PINS_FIELD16(22, 26, 0xf60, 0x10, 8, 4),
+ PINS_FIELD16(27, 29, 0xf60, 0x10, 12, 4),
+ PINS_FIELD16(30, 32, 0xf60, 0x10, 0, 4),
+ PINS_FIELD16(33, 37, 0xf70, 0x10, 0, 4),
+ PIN_FIELD16(38, 38, 0xf70, 0x10, 4, 4),
+ PINS_FIELD16(39, 42, 0xf70, 0x10, 8, 4),
+ PINS_FIELD16(43, 45, 0xf70, 0x10, 12, 4),
+ PINS_FIELD16(47, 48, 0xf80, 0x10, 0, 4),
+ PIN_FIELD16(49, 49, 0xf80, 0x10, 4, 4),
+ PINS_FIELD16(50, 52, 0xf70, 0x10, 4, 4),
+ PINS_FIELD16(53, 56, 0xf80, 0x10, 12, 4),
+ PINS_FIELD16(60, 62, 0xf90, 0x10, 8, 4),
+ PINS_FIELD16(63, 65, 0xf90, 0x10, 12, 4),
+ PINS_FIELD16(66, 71, 0xfa0, 0x10, 0, 4),
+ PINS_FIELD16(72, 74, 0xf80, 0x10, 4, 4),
+ PIN_FIELD16(85, 85, 0xda0, 0x10, 0, 4),
+ PIN_FIELD16(86, 86, 0xd90, 0x10, 0, 4),
+ PINS_FIELD16(87, 90, 0xdb0, 0x10, 0, 4),
+ PIN_FIELD16(105, 105, 0xd40, 0x10, 0, 4),
+ PIN_FIELD16(106, 106, 0xd30, 0x10, 0, 4),
+ PINS_FIELD16(107, 110, 0xd50, 0x10, 0, 4),
+ PINS_FIELD16(111, 115, 0xce0, 0x10, 0, 4),
+ PIN_FIELD16(116, 116, 0xcd0, 0x10, 0, 4),
+ PIN_FIELD16(117, 117, 0xcc0, 0x10, 0, 4),
+ PINS_FIELD16(118, 121, 0xce0, 0x10, 0, 4),
+ PIN_FIELD16(126, 126, 0xf80, 0x10, 4, 4),
+ PIN_FIELD16(188, 188, 0xf70, 0x10, 4, 4),
+ PINS_FIELD16(189, 193, 0xfe0, 0x10, 8, 4),
+ PINS_FIELD16(194, 198, 0xfe0, 0x10, 12, 4),
+ PIN_FIELD16(199, 199, 0xf50, 0x10, 4, 4),
+ PINS_FIELD16(200, 202, 0xfd0, 0x10, 0, 4),
+ PINS_FIELD16(203, 207, 0xfd0, 0x10, 4, 4),
+ PINS_FIELD16(208, 209, 0xfd0, 0x10, 8, 4),
+ PIN_FIELD16(210, 210, 0xfd0, 0x10, 12, 4),
+ PINS_FIELD16(211, 235, 0xff0, 0x10, 0, 4),
+ PINS_FIELD16(236, 241, 0xff0, 0x10, 4, 4),
+ PINS_FIELD16(242, 243, 0xff0, 0x10, 8, 4),
+ PIN_FIELD16(248, 248, 0xf00, 0x10, 0, 4),
+ PINS_FIELD16(249, 256, 0xfc0, 0x10, 0, 4),
+ PIN_FIELD16(257, 257, 0xce0, 0x10, 0, 4),
+ PIN_FIELD16(258, 258, 0xcb0, 0x10, 0, 4),
+ PIN_FIELD16(259, 259, 0xc90, 0x10, 0, 4),
+ PIN_FIELD16(260, 260, 0x3a0, 0x10, 0, 4),
+ PIN_FIELD16(261, 261, 0xd50, 0x10, 0, 4),
+ PINS_FIELD16(262, 277, 0xf00, 0x10, 8, 4),
+ PIN_FIELD16(278, 278, 0xf70, 0x10, 8, 4),
+};
+
+static const struct mtk_pin_field_calc mt7623_pin_tdsel_range[] = {
+ PINS_FIELD16(262, 276, 0x4c0, 0x10, 0, 4),
+};
+
+static const struct mtk_pin_field_calc mt7623_pin_pupd_range[] = {
+ /* MSDC0 */
+ PIN_FIELD16(111, 111, 0xd00, 0x10, 12, 1),
+ PIN_FIELD16(112, 112, 0xd00, 0x10, 8, 1),
+ PIN_FIELD16(113, 113, 0xd00, 0x10, 4, 1),
+ PIN_FIELD16(114, 114, 0xd00, 0x10, 0, 1),
+ PIN_FIELD16(115, 115, 0xd10, 0x10, 0, 1),
+ PIN_FIELD16(116, 116, 0xcd0, 0x10, 8, 1),
+ PIN_FIELD16(117, 117, 0xcc0, 0x10, 8, 1),
+ PIN_FIELD16(118, 118, 0xcf0, 0x10, 12, 1),
+ PIN_FIELD16(119, 119, 0xcf0, 0x10, 8, 1),
+ PIN_FIELD16(120, 120, 0xcf0, 0x10, 4, 1),
+ PIN_FIELD16(121, 121, 0xcf0, 0x10, 0, 1),
+ /* MSDC1 */
+ PIN_FIELD16(105, 105, 0xd40, 0x10, 8, 1),
+ PIN_FIELD16(106, 106, 0xd30, 0x10, 8, 1),
+ PIN_FIELD16(107, 107, 0xd60, 0x10, 0, 1),
+ PIN_FIELD16(108, 108, 0xd60, 0x10, 10, 1),
+ PIN_FIELD16(109, 109, 0xd60, 0x10, 4, 1),
+ PIN_FIELD16(110, 110, 0xc60, 0x10, 12, 1),
+ /* MSDC1 */
+ PIN_FIELD16(85, 85, 0xda0, 0x10, 8, 1),
+ PIN_FIELD16(86, 86, 0xd90, 0x10, 8, 1),
+ PIN_FIELD16(87, 87, 0xdc0, 0x10, 0, 1),
+ PIN_FIELD16(88, 88, 0xdc0, 0x10, 10, 1),
+ PIN_FIELD16(89, 89, 0xdc0, 0x10, 4, 1),
+ PIN_FIELD16(90, 90, 0xdc0, 0x10, 12, 1),
+ /* MSDC0E */
+ PIN_FIELD16(249, 249, 0x140, 0x10, 0, 1),
+ PIN_FIELD16(250, 250, 0x130, 0x10, 12, 1),
+ PIN_FIELD16(251, 251, 0x130, 0x10, 8, 1),
+ PIN_FIELD16(252, 252, 0x130, 0x10, 4, 1),
+ PIN_FIELD16(253, 253, 0x130, 0x10, 0, 1),
+ PIN_FIELD16(254, 254, 0xf40, 0x10, 12, 1),
+ PIN_FIELD16(255, 255, 0xf40, 0x10, 8, 1),
+ PIN_FIELD16(256, 256, 0xf40, 0x10, 4, 1),
+ PIN_FIELD16(257, 257, 0xf40, 0x10, 0, 1),
+ PIN_FIELD16(258, 258, 0xcb0, 0x10, 8, 1),
+ PIN_FIELD16(259, 259, 0xc90, 0x10, 8, 1),
+ PIN_FIELD16(261, 261, 0x140, 0x10, 8, 1),
+};
+
+static const struct mtk_pin_field_calc mt7623_pin_r1_range[] = {
+ /* MSDC0 */
+ PIN_FIELD16(111, 111, 0xd00, 0x10, 13, 1),
+ PIN_FIELD16(112, 112, 0xd00, 0x10, 9, 1),
+ PIN_FIELD16(113, 113, 0xd00, 0x10, 5, 1),
+ PIN_FIELD16(114, 114, 0xd00, 0x10, 1, 1),
+ PIN_FIELD16(115, 115, 0xd10, 0x10, 1, 1),
+ PIN_FIELD16(116, 116, 0xcd0, 0x10, 9, 1),
+ PIN_FIELD16(117, 117, 0xcc0, 0x10, 9, 1),
+ PIN_FIELD16(118, 118, 0xcf0, 0x10, 13, 1),
+ PIN_FIELD16(119, 119, 0xcf0, 0x10, 9, 1),
+ PIN_FIELD16(120, 120, 0xcf0, 0x10, 5, 1),
+ PIN_FIELD16(121, 121, 0xcf0, 0x10, 1, 1),
+ /* MSDC1 */
+ PIN_FIELD16(105, 105, 0xd40, 0x10, 9, 1),
+ PIN_FIELD16(106, 106, 0xd30, 0x10, 9, 1),
+ PIN_FIELD16(107, 107, 0xd60, 0x10, 1, 1),
+ PIN_FIELD16(108, 108, 0xd60, 0x10, 9, 1),
+ PIN_FIELD16(109, 109, 0xd60, 0x10, 5, 1),
+ PIN_FIELD16(110, 110, 0xc60, 0x10, 13, 1),
+ /* MSDC2 */
+ PIN_FIELD16(85, 85, 0xda0, 0x10, 9, 1),
+ PIN_FIELD16(86, 86, 0xd90, 0x10, 9, 1),
+ PIN_FIELD16(87, 87, 0xdc0, 0x10, 1, 1),
+ PIN_FIELD16(88, 88, 0xdc0, 0x10, 9, 1),
+ PIN_FIELD16(89, 89, 0xdc0, 0x10, 5, 1),
+ PIN_FIELD16(90, 90, 0xdc0, 0x10, 13, 1),
+ /* MSDC0E */
+ PIN_FIELD16(249, 249, 0x140, 0x10, 1, 1),
+ PIN_FIELD16(250, 250, 0x130, 0x10, 13, 1),
+ PIN_FIELD16(251, 251, 0x130, 0x10, 9, 1),
+ PIN_FIELD16(252, 252, 0x130, 0x10, 5, 1),
+ PIN_FIELD16(253, 253, 0x130, 0x10, 1, 1),
+ PIN_FIELD16(254, 254, 0xf40, 0x10, 13, 1),
+ PIN_FIELD16(255, 255, 0xf40, 0x10, 9, 1),
+ PIN_FIELD16(256, 256, 0xf40, 0x10, 5, 1),
+ PIN_FIELD16(257, 257, 0xf40, 0x10, 1, 1),
+ PIN_FIELD16(258, 258, 0xcb0, 0x10, 9, 1),
+ PIN_FIELD16(259, 259, 0xc90, 0x10, 9, 1),
+ PIN_FIELD16(261, 261, 0x140, 0x10, 9, 1),
+};
+
+static const struct mtk_pin_field_calc mt7623_pin_r0_range[] = {
+ /* MSDC0 */
+ PIN_FIELD16(111, 111, 0xd00, 0x10, 14, 1),
+ PIN_FIELD16(112, 112, 0xd00, 0x10, 10, 1),
+ PIN_FIELD16(113, 113, 0xd00, 0x10, 6, 1),
+ PIN_FIELD16(114, 114, 0xd00, 0x10, 2, 1),
+ PIN_FIELD16(115, 115, 0xd10, 0x10, 2, 1),
+ PIN_FIELD16(116, 116, 0xcd0, 0x10, 10, 1),
+ PIN_FIELD16(117, 117, 0xcc0, 0x10, 10, 1),
+ PIN_FIELD16(118, 118, 0xcf0, 0x10, 14, 1),
+ PIN_FIELD16(119, 119, 0xcf0, 0x10, 10, 1),
+ PIN_FIELD16(120, 120, 0xcf0, 0x10, 6, 1),
+ PIN_FIELD16(121, 121, 0xcf0, 0x10, 2, 1),
+ /* MSDC1 */
+ PIN_FIELD16(105, 105, 0xd40, 0x10, 10, 1),
+ PIN_FIELD16(106, 106, 0xd30, 0x10, 10, 1),
+ PIN_FIELD16(107, 107, 0xd60, 0x10, 2, 1),
+ PIN_FIELD16(108, 108, 0xd60, 0x10, 8, 1),
+ PIN_FIELD16(109, 109, 0xd60, 0x10, 6, 1),
+ PIN_FIELD16(110, 110, 0xc60, 0x10, 14, 1),
+ /* MSDC2 */
+ PIN_FIELD16(85, 85, 0xda0, 0x10, 10, 1),
+ PIN_FIELD16(86, 86, 0xd90, 0x10, 10, 1),
+ PIN_FIELD16(87, 87, 0xdc0, 0x10, 2, 1),
+ PIN_FIELD16(88, 88, 0xdc0, 0x10, 8, 1),
+ PIN_FIELD16(89, 89, 0xdc0, 0x10, 6, 1),
+ PIN_FIELD16(90, 90, 0xdc0, 0x10, 14, 1),
+ /* MSDC0E */
+ PIN_FIELD16(249, 249, 0x140, 0x10, 2, 1),
+ PIN_FIELD16(250, 250, 0x130, 0x10, 14, 1),
+ PIN_FIELD16(251, 251, 0x130, 0x10, 10, 1),
+ PIN_FIELD16(252, 252, 0x130, 0x10, 6, 1),
+ PIN_FIELD16(253, 253, 0x130, 0x10, 2, 1),
+ PIN_FIELD16(254, 254, 0xf40, 0x10, 14, 1),
+ PIN_FIELD16(255, 255, 0xf40, 0x10, 10, 1),
+ PIN_FIELD16(256, 256, 0xf40, 0x10, 6, 1),
+ PIN_FIELD16(257, 257, 0xf40, 0x10, 5, 1),
+ PIN_FIELD16(258, 258, 0xcb0, 0x10, 10, 1),
+ PIN_FIELD16(259, 259, 0xc90, 0x10, 10, 1),
+ PIN_FIELD16(261, 261, 0x140, 0x10, 10, 1),
+};
+
+static const struct mtk_pin_reg_calc mt7623_reg_cals[] = {
+ [PINCTRL_PIN_REG_MODE] = MTK_RANGE(mt7623_pin_mode_range),
+ [PINCTRL_PIN_REG_DIR] = MTK_RANGE(mt7623_pin_dir_range),
+ [PINCTRL_PIN_REG_DI] = MTK_RANGE(mt7623_pin_di_range),
+ [PINCTRL_PIN_REG_DO] = MTK_RANGE(mt7623_pin_do_range),
+ [PINCTRL_PIN_REG_SMT] = MTK_RANGE(mt7623_pin_smt_range),
+ [PINCTRL_PIN_REG_PULLSEL] = MTK_RANGE(mt7623_pin_pullsel_range),
+ [PINCTRL_PIN_REG_PULLEN] = MTK_RANGE(mt7623_pin_pullen_range),
+ [PINCTRL_PIN_REG_DRV] = MTK_RANGE(mt7623_pin_drv_range),
+ [PINCTRL_PIN_REG_TDSEL] = MTK_RANGE(mt7623_pin_tdsel_range),
+ [PINCTRL_PIN_REG_IES] = MTK_RANGE(mt7623_pin_ies_range),
+ [PINCTRL_PIN_REG_PUPD] = MTK_RANGE(mt7623_pin_pupd_range),
+ [PINCTRL_PIN_REG_R0] = MTK_RANGE(mt7623_pin_r0_range),
+ [PINCTRL_PIN_REG_R1] = MTK_RANGE(mt7623_pin_r1_range),
+};
+
+static const struct mtk_pin_desc mt7623_pins[] = {
+ MT7623_PIN(0, "PWRAP_SPI0_MI", 148, DRV_GRP3),
+ MT7623_PIN(1, "PWRAP_SPI0_MO", 149, DRV_GRP3),
+ MT7623_PIN(2, "PWRAP_INT", 150, DRV_GRP3),
+ MT7623_PIN(3, "PWRAP_SPI0_CK", 151, DRV_GRP3),
+ MT7623_PIN(4, "PWRAP_SPI0_CSN", 152, DRV_GRP3),
+ MT7623_PIN(5, "PWRAP_SPI0_CK2", 153, DRV_GRP3),
+ MT7623_PIN(6, "PWRAP_SPI0_CSN2", 154, DRV_GRP3),
+ MT7623_PIN(7, "SPI1_CSN", 155, DRV_GRP3),
+ MT7623_PIN(8, "SPI1_MI", 156, DRV_GRP3),
+ MT7623_PIN(9, "SPI1_MO", 157, DRV_GRP3),
+ MT7623_PIN(10, "RTC32K_CK", 158, DRV_GRP3),
+ MT7623_PIN(11, "WATCHDOG", 159, DRV_GRP3),
+ MT7623_PIN(12, "SRCLKENA", 160, DRV_GRP3),
+ MT7623_PIN(13, "SRCLKENAI", 161, DRV_GRP3),
+ MT7623_PIN(14, "URXD2", 162, DRV_GRP1),
+ MT7623_PIN(15, "UTXD2", 163, DRV_GRP1),
+ MT7623_PIN(16, "I2S5_DATA_IN", 164, DRV_GRP1),
+ MT7623_PIN(17, "I2S5_BCK", 165, DRV_GRP1),
+ MT7623_PIN(18, "PCM_CLK", 166, DRV_GRP1),
+ MT7623_PIN(19, "PCM_SYNC", 167, DRV_GRP1),
+ MT7623_PIN(20, "PCM_RX", EINT_NA, DRV_GRP1),
+ MT7623_PIN(21, "PCM_TX", EINT_NA, DRV_GRP1),
+ MT7623_PIN(22, "EINT0", 0, DRV_GRP1),
+ MT7623_PIN(23, "EINT1", 1, DRV_GRP1),
+ MT7623_PIN(24, "EINT2", 2, DRV_GRP1),
+ MT7623_PIN(25, "EINT3", 3, DRV_GRP1),
+ MT7623_PIN(26, "EINT4", 4, DRV_GRP1),
+ MT7623_PIN(27, "EINT5", 5, DRV_GRP1),
+ MT7623_PIN(28, "EINT6", 6, DRV_GRP1),
+ MT7623_PIN(29, "EINT7", 7, DRV_GRP1),
+ MT7623_PIN(30, "I2S5_LRCK", 12, DRV_GRP1),
+ MT7623_PIN(31, "I2S5_MCLK", 13, DRV_GRP1),
+ MT7623_PIN(32, "I2S5_DATA", 14, DRV_GRP1),
+ MT7623_PIN(33, "I2S1_DATA", 15, DRV_GRP1),
+ MT7623_PIN(34, "I2S1_DATA_IN", 16, DRV_GRP1),
+ MT7623_PIN(35, "I2S1_BCK", 17, DRV_GRP1),
+ MT7623_PIN(36, "I2S1_LRCK", 18, DRV_GRP1),
+ MT7623_PIN(37, "I2S1_MCLK", 19, DRV_GRP1),
+ MT7623_PIN(38, "I2S2_DATA", 20, DRV_GRP1),
+ MT7623_PIN(39, "JTMS", 21, DRV_GRP3),
+ MT7623_PIN(40, "JTCK", 22, DRV_GRP3),
+ MT7623_PIN(41, "JTDI", 23, DRV_GRP3),
+ MT7623_PIN(42, "JTDO", 24, DRV_GRP3),
+ MT7623_PIN(43, "NCLE", 25, DRV_GRP1),
+ MT7623_PIN(44, "NCEB1", 26, DRV_GRP1),
+ MT7623_PIN(45, "NCEB0", 27, DRV_GRP1),
+ MT7623_PIN(46, "IR", 28, DRV_FIXED),
+ MT7623_PIN(47, "NREB", 29, DRV_GRP1),
+ MT7623_PIN(48, "NRNB", 30, DRV_GRP1),
+ MT7623_PIN(49, "I2S0_DATA", 31, DRV_GRP1),
+ MT7623_PIN(50, "I2S2_BCK", 32, DRV_GRP1),
+ MT7623_PIN(51, "I2S2_DATA_IN", 33, DRV_GRP1),
+ MT7623_PIN(52, "I2S2_LRCK", 34, DRV_GRP1),
+ MT7623_PIN(53, "SPI0_CSN", 35, DRV_GRP1),
+ MT7623_PIN(54, "SPI0_CK", 36, DRV_GRP1),
+ MT7623_PIN(55, "SPI0_MI", 37, DRV_GRP1),
+ MT7623_PIN(56, "SPI0_MO", 38, DRV_GRP1),
+ MT7623_PIN(57, "SDA1", 39, DRV_FIXED),
+ MT7623_PIN(58, "SCL1", 40, DRV_FIXED),
+ MT7623_PIN(59, "RAMBUF_I_CLK", EINT_NA, DRV_FIXED),
+ MT7623_PIN(60, "WB_RSTB", 41, DRV_GRP3),
+ MT7623_PIN(61, "F2W_DATA", 42, DRV_GRP3),
+ MT7623_PIN(62, "F2W_CLK", 43, DRV_GRP3),
+ MT7623_PIN(63, "WB_SCLK", 44, DRV_GRP3),
+ MT7623_PIN(64, "WB_SDATA", 45, DRV_GRP3),
+ MT7623_PIN(65, "WB_SEN", 46, DRV_GRP3),
+ MT7623_PIN(66, "WB_CRTL0", 47, DRV_GRP3),
+ MT7623_PIN(67, "WB_CRTL1", 48, DRV_GRP3),
+ MT7623_PIN(68, "WB_CRTL2", 49, DRV_GRP3),
+ MT7623_PIN(69, "WB_CRTL3", 50, DRV_GRP3),
+ MT7623_PIN(70, "WB_CRTL4", 51, DRV_GRP3),
+ MT7623_PIN(71, "WB_CRTL5", 52, DRV_GRP3),
+ MT7623_PIN(72, "I2S0_DATA_IN", 53, DRV_GRP1),
+ MT7623_PIN(73, "I2S0_LRCK", 54, DRV_GRP1),
+ MT7623_PIN(74, "I2S0_BCK", 55, DRV_GRP1),
+ MT7623_PIN(75, "SDA0", 56, DRV_FIXED),
+ MT7623_PIN(76, "SCL0", 57, DRV_FIXED),
+ MT7623_PIN(77, "SDA2", 58, DRV_FIXED),
+ MT7623_PIN(78, "SCL2", 59, DRV_FIXED),
+ MT7623_PIN(79, "URXD0", 60, DRV_FIXED),
+ MT7623_PIN(80, "UTXD0", 61, DRV_FIXED),
+ MT7623_PIN(81, "URXD1", 62, DRV_FIXED),
+ MT7623_PIN(82, "UTXD1", 63, DRV_FIXED),
+ MT7623_PIN(83, "LCM_RST", 64, DRV_FIXED),
+ MT7623_PIN(84, "DSI_TE", 65, DRV_FIXED),
+ MT7623_PIN(85, "MSDC2_CMD", 66, DRV_GRP4),
+ MT7623_PIN(86, "MSDC2_CLK", 67, DRV_GRP4),
+ MT7623_PIN(87, "MSDC2_DAT0", 68, DRV_GRP4),
+ MT7623_PIN(88, "MSDC2_DAT1", 69, DRV_GRP4),
+ MT7623_PIN(89, "MSDC2_DAT2", 70, DRV_GRP4),
+ MT7623_PIN(90, "MSDC2_DAT3", 71, DRV_GRP4),
+ MT7623_PIN(91, "TDN3", EINT_NA, DRV_FIXED),
+ MT7623_PIN(92, "TDP3", EINT_NA, DRV_FIXED),
+ MT7623_PIN(93, "TDN2", EINT_NA, DRV_FIXED),
+ MT7623_PIN(94, "TDP2", EINT_NA, DRV_FIXED),
+ MT7623_PIN(95, "TCN", EINT_NA, DRV_FIXED),
+ MT7623_PIN(96, "TCP", EINT_NA, DRV_FIXED),
+ MT7623_PIN(97, "TDN1", EINT_NA, DRV_FIXED),
+ MT7623_PIN(98, "TDP1", EINT_NA, DRV_FIXED),
+ MT7623_PIN(99, "TDN0", EINT_NA, DRV_FIXED),
+ MT7623_PIN(100, "TDP0", EINT_NA, DRV_FIXED),
+ MT7623_PIN(101, "SPI2_CSN", 74, DRV_FIXED),
+ MT7623_PIN(102, "SPI2_MI", 75, DRV_FIXED),
+ MT7623_PIN(103, "SPI2_MO", 76, DRV_FIXED),
+ MT7623_PIN(104, "SPI2_CLK", 77, DRV_FIXED),
+ MT7623_PIN(105, "MSDC1_CMD", 78, DRV_GRP4),
+ MT7623_PIN(106, "MSDC1_CLK", 79, DRV_GRP4),
+ MT7623_PIN(107, "MSDC1_DAT0", 80, DRV_GRP4),
+ MT7623_PIN(108, "MSDC1_DAT1", 81, DRV_GRP4),
+ MT7623_PIN(109, "MSDC1_DAT2", 82, DRV_GRP4),
+ MT7623_PIN(110, "MSDC1_DAT3", 83, DRV_GRP4),
+ MT7623_PIN(111, "MSDC0_DAT7", 84, DRV_GRP4),
+ MT7623_PIN(112, "MSDC0_DAT6", 85, DRV_GRP4),
+ MT7623_PIN(113, "MSDC0_DAT5", 86, DRV_GRP4),
+ MT7623_PIN(114, "MSDC0_DAT4", 87, DRV_GRP4),
+ MT7623_PIN(115, "MSDC0_RSTB", 88, DRV_GRP4),
+ MT7623_PIN(116, "MSDC0_CMD", 89, DRV_GRP4),
+ MT7623_PIN(117, "MSDC0_CLK", 90, DRV_GRP4),
+ MT7623_PIN(118, "MSDC0_DAT3", 91, DRV_GRP4),
+ MT7623_PIN(119, "MSDC0_DAT2", 92, DRV_GRP4),
+ MT7623_PIN(120, "MSDC0_DAT1", 93, DRV_GRP4),
+ MT7623_PIN(121, "MSDC0_DAT0", 94, DRV_GRP4),
+ MT7623_PIN(122, "CEC", 95, DRV_FIXED),
+ MT7623_PIN(123, "HTPLG", 96, DRV_FIXED),
+ MT7623_PIN(124, "HDMISCK", 97, DRV_FIXED),
+ MT7623_PIN(125, "HDMISD", 98, DRV_FIXED),
+ MT7623_PIN(126, "I2S0_MCLK", 99, DRV_GRP1),
+ MT7623_PIN(127, "RAMBUF_IDATA0", EINT_NA, DRV_FIXED),
+ MT7623_PIN(128, "RAMBUF_IDATA1", EINT_NA, DRV_FIXED),
+ MT7623_PIN(129, "RAMBUF_IDATA2", EINT_NA, DRV_FIXED),
+ MT7623_PIN(130, "RAMBUF_IDATA3", EINT_NA, DRV_FIXED),
+ MT7623_PIN(131, "RAMBUF_IDATA4", EINT_NA, DRV_FIXED),
+ MT7623_PIN(132, "RAMBUF_IDATA5", EINT_NA, DRV_FIXED),
+ MT7623_PIN(133, "RAMBUF_IDATA6", EINT_NA, DRV_FIXED),
+ MT7623_PIN(134, "RAMBUF_IDATA7", EINT_NA, DRV_FIXED),
+ MT7623_PIN(135, "RAMBUF_IDATA8", EINT_NA, DRV_FIXED),
+ MT7623_PIN(136, "RAMBUF_IDATA9", EINT_NA, DRV_FIXED),
+ MT7623_PIN(137, "RAMBUF_IDATA10", EINT_NA, DRV_FIXED),
+ MT7623_PIN(138, "RAMBUF_IDATA11", EINT_NA, DRV_FIXED),
+ MT7623_PIN(139, "RAMBUF_IDATA12", EINT_NA, DRV_FIXED),
+ MT7623_PIN(140, "RAMBUF_IDATA13", EINT_NA, DRV_FIXED),
+ MT7623_PIN(141, "RAMBUF_IDATA14", EINT_NA, DRV_FIXED),
+ MT7623_PIN(142, "RAMBUF_IDATA15", EINT_NA, DRV_FIXED),
+ MT7623_PIN(143, "RAMBUF_ODATA0", EINT_NA, DRV_FIXED),
+ MT7623_PIN(144, "RAMBUF_ODATA1", EINT_NA, DRV_FIXED),
+ MT7623_PIN(145, "RAMBUF_ODATA2", EINT_NA, DRV_FIXED),
+ MT7623_PIN(146, "RAMBUF_ODATA3", EINT_NA, DRV_FIXED),
+ MT7623_PIN(147, "RAMBUF_ODATA4", EINT_NA, DRV_FIXED),
+ MT7623_PIN(148, "RAMBUF_ODATA5", EINT_NA, DRV_FIXED),
+ MT7623_PIN(149, "RAMBUF_ODATA6", EINT_NA, DRV_FIXED),
+ MT7623_PIN(150, "RAMBUF_ODATA7", EINT_NA, DRV_FIXED),
+ MT7623_PIN(151, "RAMBUF_ODATA8", EINT_NA, DRV_FIXED),
+ MT7623_PIN(152, "RAMBUF_ODATA9", EINT_NA, DRV_FIXED),
+ MT7623_PIN(153, "RAMBUF_ODATA10", EINT_NA, DRV_FIXED),
+ MT7623_PIN(154, "RAMBUF_ODATA11", EINT_NA, DRV_FIXED),
+ MT7623_PIN(155, "RAMBUF_ODATA12", EINT_NA, DRV_FIXED),
+ MT7623_PIN(156, "RAMBUF_ODATA13", EINT_NA, DRV_FIXED),
+ MT7623_PIN(157, "RAMBUF_ODATA14", EINT_NA, DRV_FIXED),
+ MT7623_PIN(158, "RAMBUF_ODATA15", EINT_NA, DRV_FIXED),
+ MT7623_PIN(159, "RAMBUF_BE0", EINT_NA, DRV_FIXED),
+ MT7623_PIN(160, "RAMBUF_BE1", EINT_NA, DRV_FIXED),
+ MT7623_PIN(161, "AP2PT_INT", EINT_NA, DRV_FIXED),
+ MT7623_PIN(162, "AP2PT_INT_CLR", EINT_NA, DRV_FIXED),
+ MT7623_PIN(163, "PT2AP_INT", EINT_NA, DRV_FIXED),
+ MT7623_PIN(164, "PT2AP_INT_CLR", EINT_NA, DRV_FIXED),
+ MT7623_PIN(165, "AP2UP_INT", EINT_NA, DRV_FIXED),
+ MT7623_PIN(166, "AP2UP_INT_CLR", EINT_NA, DRV_FIXED),
+ MT7623_PIN(167, "UP2AP_INT", EINT_NA, DRV_FIXED),
+ MT7623_PIN(168, "UP2AP_INT_CLR", EINT_NA, DRV_FIXED),
+ MT7623_PIN(169, "RAMBUF_ADDR0", EINT_NA, DRV_FIXED),
+ MT7623_PIN(170, "RAMBUF_ADDR1", EINT_NA, DRV_FIXED),
+ MT7623_PIN(171, "RAMBUF_ADDR2", EINT_NA, DRV_FIXED),
+ MT7623_PIN(172, "RAMBUF_ADDR3", EINT_NA, DRV_FIXED),
+ MT7623_PIN(173, "RAMBUF_ADDR4", EINT_NA, DRV_FIXED),
+ MT7623_PIN(174, "RAMBUF_ADDR5", EINT_NA, DRV_FIXED),
+ MT7623_PIN(175, "RAMBUF_ADDR6", EINT_NA, DRV_FIXED),
+ MT7623_PIN(176, "RAMBUF_ADDR7", EINT_NA, DRV_FIXED),
+ MT7623_PIN(177, "RAMBUF_ADDR8", EINT_NA, DRV_FIXED),
+ MT7623_PIN(178, "RAMBUF_ADDR9", EINT_NA, DRV_FIXED),
+ MT7623_PIN(179, "RAMBUF_ADDR10", EINT_NA, DRV_FIXED),
+ MT7623_PIN(180, "RAMBUF_RW", EINT_NA, DRV_FIXED),
+ MT7623_PIN(181, "RAMBUF_LAST", EINT_NA, DRV_FIXED),
+ MT7623_PIN(182, "RAMBUF_HP", EINT_NA, DRV_FIXED),
+ MT7623_PIN(183, "RAMBUF_REQ", EINT_NA, DRV_FIXED),
+ MT7623_PIN(184, "RAMBUF_ALE", EINT_NA, DRV_FIXED),
+ MT7623_PIN(185, "RAMBUF_DLE", EINT_NA, DRV_FIXED),
+ MT7623_PIN(186, "RAMBUF_WDLE", EINT_NA, DRV_FIXED),
+ MT7623_PIN(187, "RAMBUF_O_CLK", EINT_NA, DRV_FIXED),
+ MT7623_PIN(188, "I2S2_MCLK", 100, DRV_GRP1),
+ MT7623_PIN(189, "I2S3_DATA", 101, DRV_GRP1),
+ MT7623_PIN(190, "I2S3_DATA_IN", 102, DRV_GRP1),
+ MT7623_PIN(191, "I2S3_BCK", 103, DRV_GRP1),
+ MT7623_PIN(192, "I2S3_LRCK", 104, DRV_GRP1),
+ MT7623_PIN(193, "I2S3_MCLK", 105, DRV_GRP1),
+ MT7623_PIN(194, "I2S4_DATA", 106, DRV_GRP1),
+ MT7623_PIN(195, "I2S4_DATA_IN", 107, DRV_GRP1),
+ MT7623_PIN(196, "I2S4_BCK", 108, DRV_GRP1),
+ MT7623_PIN(197, "I2S4_LRCK", 109, DRV_GRP1),
+ MT7623_PIN(198, "I2S4_MCLK", 110, DRV_GRP1),
+ MT7623_PIN(199, "SPI1_CLK", 111, DRV_GRP3),
+ MT7623_PIN(200, "SPDIF_OUT", 112, DRV_GRP1),
+ MT7623_PIN(201, "SPDIF_IN0", 113, DRV_GRP1),
+ MT7623_PIN(202, "SPDIF_IN1", 114, DRV_GRP1),
+ MT7623_PIN(203, "PWM0", 115, DRV_GRP1),
+ MT7623_PIN(204, "PWM1", 116, DRV_GRP1),
+ MT7623_PIN(205, "PWM2", 117, DRV_GRP1),
+ MT7623_PIN(206, "PWM3", 118, DRV_GRP1),
+ MT7623_PIN(207, "PWM4", 119, DRV_GRP1),
+ MT7623_PIN(208, "AUD_EXT_CK1", 120, DRV_GRP1),
+ MT7623_PIN(209, "AUD_EXT_CK2", 121, DRV_GRP1),
+ MT7623_PIN(210, "AUD_CLOCK", EINT_NA, DRV_GRP3),
+ MT7623_PIN(211, "DVP_RESET", EINT_NA, DRV_GRP3),
+ MT7623_PIN(212, "DVP_CLOCK", EINT_NA, DRV_GRP3),
+ MT7623_PIN(213, "DVP_CS", EINT_NA, DRV_GRP3),
+ MT7623_PIN(214, "DVP_CK", EINT_NA, DRV_GRP3),
+ MT7623_PIN(215, "DVP_DI", EINT_NA, DRV_GRP3),
+ MT7623_PIN(216, "DVP_DO", EINT_NA, DRV_GRP3),
+ MT7623_PIN(217, "AP_CS", EINT_NA, DRV_GRP3),
+ MT7623_PIN(218, "AP_CK", EINT_NA, DRV_GRP3),
+ MT7623_PIN(219, "AP_DI", EINT_NA, DRV_GRP3),
+ MT7623_PIN(220, "AP_DO", EINT_NA, DRV_GRP3),
+ MT7623_PIN(221, "DVD_BCLK", EINT_NA, DRV_GRP3),
+ MT7623_PIN(222, "T8032_CLK", EINT_NA, DRV_GRP3),
+ MT7623_PIN(223, "AP_BCLK", EINT_NA, DRV_GRP3),
+ MT7623_PIN(224, "HOST_CS", EINT_NA, DRV_GRP3),
+ MT7623_PIN(225, "HOST_CK", EINT_NA, DRV_GRP3),
+ MT7623_PIN(226, "HOST_DO0", EINT_NA, DRV_GRP3),
+ MT7623_PIN(227, "HOST_DO1", EINT_NA, DRV_GRP3),
+ MT7623_PIN(228, "SLV_CS", EINT_NA, DRV_GRP3),
+ MT7623_PIN(229, "SLV_CK", EINT_NA, DRV_GRP3),
+ MT7623_PIN(230, "SLV_DI0", EINT_NA, DRV_GRP3),
+ MT7623_PIN(231, "SLV_DI1", EINT_NA, DRV_GRP3),
+ MT7623_PIN(232, "AP2DSP_INT", EINT_NA, DRV_GRP3),
+ MT7623_PIN(233, "AP2DSP_INT_CLR", EINT_NA, DRV_GRP3),
+ MT7623_PIN(234, "DSP2AP_INT", EINT_NA, DRV_GRP3),
+ MT7623_PIN(235, "DSP2AP_INT_CLR", EINT_NA, DRV_GRP3),
+ MT7623_PIN(236, "EXT_SDIO3", 122, DRV_GRP1),
+ MT7623_PIN(237, "EXT_SDIO2", 123, DRV_GRP1),
+ MT7623_PIN(238, "EXT_SDIO1", 124, DRV_GRP1),
+ MT7623_PIN(239, "EXT_SDIO0", 125, DRV_GRP1),
+ MT7623_PIN(240, "EXT_XCS", 126, DRV_GRP1),
+ MT7623_PIN(241, "EXT_SCK", 127, DRV_GRP1),
+ MT7623_PIN(242, "URTS2", 128, DRV_GRP1),
+ MT7623_PIN(243, "UCTS2", 129, DRV_GRP1),
+ MT7623_PIN(244, "HDMI_SDA_RX", 130, DRV_FIXED),
+ MT7623_PIN(245, "HDMI_SCL_RX", 131, DRV_FIXED),
+ MT7623_PIN(246, "MHL_SENCE", 132, DRV_FIXED),
+ MT7623_PIN(247, "HDMI_HPD_CBUS_RX", 69, DRV_FIXED),
+ MT7623_PIN(248, "HDMI_TESTOUTP_RX", 133, DRV_GRP1),
+ MT7623_PIN(249, "MSDC0E_RSTB", 134, DRV_GRP4),
+ MT7623_PIN(250, "MSDC0E_DAT7", 135, DRV_GRP4),
+ MT7623_PIN(251, "MSDC0E_DAT6", 136, DRV_GRP4),
+ MT7623_PIN(252, "MSDC0E_DAT5", 137, DRV_GRP4),
+ MT7623_PIN(253, "MSDC0E_DAT4", 138, DRV_GRP4),
+ MT7623_PIN(254, "MSDC0E_DAT3", 139, DRV_GRP4),
+ MT7623_PIN(255, "MSDC0E_DAT2", 140, DRV_GRP4),
+ MT7623_PIN(256, "MSDC0E_DAT1", 141, DRV_GRP4),
+ MT7623_PIN(257, "MSDC0E_DAT0", 142, DRV_GRP4),
+ MT7623_PIN(258, "MSDC0E_CMD", 143, DRV_GRP4),
+ MT7623_PIN(259, "MSDC0E_CLK", 144, DRV_GRP4),
+ MT7623_PIN(260, "MSDC0E_DSL", 145, DRV_GRP4),
+ MT7623_PIN(261, "MSDC1_INS", 146, DRV_GRP4),
+ MT7623_PIN(262, "G2_TXEN", 8, DRV_GRP1),
+ MT7623_PIN(263, "G2_TXD3", 9, DRV_GRP1),
+ MT7623_PIN(264, "G2_TXD2", 10, DRV_GRP1),
+ MT7623_PIN(265, "G2_TXD1", 11, DRV_GRP1),
+ MT7623_PIN(266, "G2_TXD0", EINT_NA, DRV_GRP1),
+ MT7623_PIN(267, "G2_TXC", EINT_NA, DRV_GRP1),
+ MT7623_PIN(268, "G2_RXC", EINT_NA, DRV_GRP1),
+ MT7623_PIN(269, "G2_RXD0", EINT_NA, DRV_GRP1),
+ MT7623_PIN(270, "G2_RXD1", EINT_NA, DRV_GRP1),
+ MT7623_PIN(271, "G2_RXD2", EINT_NA, DRV_GRP1),
+ MT7623_PIN(272, "G2_RXD3", EINT_NA, DRV_GRP1),
+ MT7623_PIN(273, "ESW_INT", 168, DRV_GRP1),
+ MT7623_PIN(274, "G2_RXDV", EINT_NA, DRV_GRP1),
+ MT7623_PIN(275, "MDC", EINT_NA, DRV_GRP1),
+ MT7623_PIN(276, "MDIO", EINT_NA, DRV_GRP1),
+ MT7623_PIN(277, "ESW_RST", EINT_NA, DRV_GRP1),
+ MT7623_PIN(278, "JTAG_RESET", 147, DRV_GRP3),
+ MT7623_PIN(279, "USB3_RES_BOND", EINT_NA, DRV_GRP1),
+};
+
+/* List all groups consisting of these pins dedicated to the enablement of
+ * certain hardware block and the corresponding mode for all of the pins.
+ * The hardware probably has multiple combinations of these pinouts.
+ */
+
+/* AUDIO EXT CLK */
+static int mt7623_aud_ext_clk0_pins[] = { 208, };
+static int mt7623_aud_ext_clk0_funcs[] = { 1, };
+static int mt7623_aud_ext_clk1_pins[] = { 209, };
+static int mt7623_aud_ext_clk1_funcs[] = { 1, };
+
+/* DISP PWM */
+static int mt7623_disp_pwm_0_pins[] = { 72, };
+static int mt7623_disp_pwm_0_funcs[] = { 5, };
+static int mt7623_disp_pwm_1_pins[] = { 203, };
+static int mt7623_disp_pwm_1_funcs[] = { 2, };
+static int mt7623_disp_pwm_2_pins[] = { 208, };
+static int mt7623_disp_pwm_2_funcs[] = { 5, };
+
+/* ESW */
+static int mt7623_esw_int_pins[] = { 273, };
+static int mt7623_esw_int_funcs[] = { 1, };
+static int mt7623_esw_rst_pins[] = { 277, };
+static int mt7623_esw_rst_funcs[] = { 1, };
+
+/* EPHY */
+static int mt7623_ephy_pins[] = { 262, 263, 264, 265, 266, 267, 268,
+ 269, 270, 271, 272, 274, };
+static int mt7623_ephy_funcs[] = { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, };
+
+/* EXT_SDIO */
+static int mt7623_ext_sdio_pins[] = { 236, 237, 238, 239, 240, 241, };
+static int mt7623_ext_sdio_funcs[] = { 1, 1, 1, 1, 1, 1, };
+
+/* HDMI RX */
+static int mt7623_hdmi_rx_pins[] = { 247, 248, };
+static int mt7623_hdmi_rx_funcs[] = { 1, 1 };
+static int mt7623_hdmi_rx_i2c_pins[] = { 244, 245, };
+static int mt7623_hdmi_rx_i2c_funcs[] = { 1, 1 };
+
+/* HDMI TX */
+static int mt7623_hdmi_cec_pins[] = { 122, };
+static int mt7623_hdmi_cec_funcs[] = { 1, };
+static int mt7623_hdmi_htplg_pins[] = { 123, };
+static int mt7623_hdmi_htplg_funcs[] = { 1, };
+static int mt7623_hdmi_i2c_pins[] = { 124, 125, };
+static int mt7623_hdmi_i2c_funcs[] = { 1, 1 };
+
+/* I2C */
+static int mt7623_i2c0_pins[] = { 75, 76, };
+static int mt7623_i2c0_funcs[] = { 1, 1, };
+static int mt7623_i2c1_0_pins[] = { 57, 58, };
+static int mt7623_i2c1_0_funcs[] = { 1, 1, };
+static int mt7623_i2c1_1_pins[] = { 242, 243, };
+static int mt7623_i2c1_1_funcs[] = { 4, 4, };
+static int mt7623_i2c1_2_pins[] = { 85, 86, };
+static int mt7623_i2c1_2_funcs[] = { 3, 3, };
+static int mt7623_i2c1_3_pins[] = { 105, 106, };
+static int mt7623_i2c1_3_funcs[] = { 3, 3, };
+static int mt7623_i2c1_4_pins[] = { 124, 125, };
+static int mt7623_i2c1_4_funcs[] = { 4, 4, };
+static int mt7623_i2c2_0_pins[] = { 77, 78, };
+static int mt7623_i2c2_0_funcs[] = { 1, 1, };
+static int mt7623_i2c2_1_pins[] = { 89, 90, };
+static int mt7623_i2c2_1_funcs[] = { 3, 3, };
+static int mt7623_i2c2_2_pins[] = { 109, 110, };
+static int mt7623_i2c2_2_funcs[] = { 3, 3, };
+static int mt7623_i2c2_3_pins[] = { 122, 123, };
+static int mt7623_i2c2_3_funcs[] = { 4, 4, };
+
+/* I2S */
+static int mt7623_i2s0_pins[] = { 49, 72, 73, 74, 126, };
+static int mt7623_i2s0_funcs[] = { 1, 1, 1, 1, 1, };
+static int mt7623_i2s1_pins[] = { 33, 34, 35, 36, 37, };
+static int mt7623_i2s1_funcs[] = { 1, 1, 1, 1, 1, };
+static int mt7623_i2s2_bclk_lrclk_mclk_pins[] = { 50, 52, 188, };
+static int mt7623_i2s2_bclk_lrclk_mclk_funcs[] = { 1, 1, 1, };
+static int mt7623_i2s2_data_in_pins[] = { 51, };
+static int mt7623_i2s2_data_in_funcs[] = { 1, };
+static int mt7623_i2s2_data_0_pins[] = { 203, };
+static int mt7623_i2s2_data_0_funcs[] = { 9, };
+static int mt7623_i2s2_data_1_pins[] = { 38, };
+static int mt7623_i2s2_data_1_funcs[] = { 4, };
+static int mt7623_i2s3_bclk_lrclk_mclk_pins[] = { 191, 192, 193, };
+static int mt7623_i2s3_bclk_lrclk_mclk_funcs[] = { 1, 1, 1, };
+static int mt7623_i2s3_data_in_pins[] = { 190, };
+static int mt7623_i2s3_data_in_funcs[] = { 1, };
+static int mt7623_i2s3_data_0_pins[] = { 204, };
+static int mt7623_i2s3_data_0_funcs[] = { 9, };
+static int mt7623_i2s3_data_1_pins[] = { 2, };
+static int mt7623_i2s3_data_1_funcs[] = { 0, };
+static int mt7623_i2s4_pins[] = { 194, 195, 196, 197, 198, };
+static int mt7623_i2s4_funcs[] = { 1, 1, 1, 1, 1, };
+static int mt7623_i2s5_pins[] = { 16, 17, 30, 31, 32, };
+static int mt7623_i2s5_funcs[] = { 1, 1, 1, 1, 1, };
+
+/* IR */
+static int mt7623_ir_pins[] = { 46, };
+static int mt7623_ir_funcs[] = { 1, };
+
+/* LCD */
+static int mt7623_mipi_tx_pins[] = { 91, 92, 93, 94, 95, 96, 97, 98,
+ 99, 100, };
+static int mt7623_mipi_tx_funcs[] = { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, };
+static int mt7623_dsi_te_pins[] = { 84, };
+static int mt7623_dsi_te_funcs[] = { 1, };
+static int mt7623_lcm_rst_pins[] = { 83, };
+static int mt7623_lcm_rst_funcs[] = { 1, };
+
+/* MDC/MDIO */
+static int mt7623_mdc_mdio_pins[] = { 275, 276, };
+static int mt7623_mdc_mdio_funcs[] = { 1, 1, };
+
+/* MSDC */
+static int mt7623_msdc0_pins[] = { 111, 112, 113, 114, 115, 116, 117, 118,
+ 119, 120, 121, };
+static int mt7623_msdc0_funcs[] = { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, };
+static int mt7623_msdc1_pins[] = { 105, 106, 107, 108, 109, 110, };
+static int mt7623_msdc1_funcs[] = { 1, 1, 1, 1, 1, 1, };
+static int mt7623_msdc1_ins_pins[] = { 261, };
+static int mt7623_msdc1_ins_funcs[] = { 1, };
+static int mt7623_msdc1_wp_0_pins[] = { 29, };
+static int mt7623_msdc1_wp_0_funcs[] = { 1, };
+static int mt7623_msdc1_wp_1_pins[] = { 55, };
+static int mt7623_msdc1_wp_1_funcs[] = { 3, };
+static int mt7623_msdc1_wp_2_pins[] = { 209, };
+static int mt7623_msdc1_wp_2_funcs[] = { 2, };
+static int mt7623_msdc2_pins[] = { 85, 86, 87, 88, 89, 90, };
+static int mt7623_msdc2_funcs[] = { 1, 1, 1, 1, 1, 1, };
+static int mt7623_msdc3_pins[] = { 249, 250, 251, 252, 253, 254, 255, 256,
+ 257, 258, 259, 260, };
+static int mt7623_msdc3_funcs[] = { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, };
+
+/* NAND */
+static int mt7623_nandc_pins[] = { 43, 47, 48, 111, 112, 113, 114, 115,
+ 116, 117, 118, 119, 120, 121, };
+static int mt7623_nandc_funcs[] = { 1, 1, 1, 4, 4, 4, 4, 4, 4, 4, 4, 4,
+ 4, 4, };
+static int mt7623_nandc_ceb0_pins[] = { 45, };
+static int mt7623_nandc_ceb0_funcs[] = { 1, };
+static int mt7623_nandc_ceb1_pins[] = { 44, };
+static int mt7623_nandc_ceb1_funcs[] = { 1, };
+
+/* RTC */
+static int mt7623_rtc_pins[] = { 10, };
+static int mt7623_rtc_funcs[] = { 1, };
+
+/* OTG */
+static int mt7623_otg_iddig0_0_pins[] = { 29, };
+static int mt7623_otg_iddig0_0_funcs[] = { 1, };
+static int mt7623_otg_iddig0_1_pins[] = { 44, };
+static int mt7623_otg_iddig0_1_funcs[] = { 2, };
+static int mt7623_otg_iddig0_2_pins[] = { 236, };
+static int mt7623_otg_iddig0_2_funcs[] = { 2, };
+static int mt7623_otg_iddig1_0_pins[] = { 27, };
+static int mt7623_otg_iddig1_0_funcs[] = { 2, };
+static int mt7623_otg_iddig1_1_pins[] = { 47, };
+static int mt7623_otg_iddig1_1_funcs[] = { 2, };
+static int mt7623_otg_iddig1_2_pins[] = { 238, };
+static int mt7623_otg_iddig1_2_funcs[] = { 2, };
+static int mt7623_otg_drv_vbus0_0_pins[] = { 28, };
+static int mt7623_otg_drv_vbus0_0_funcs[] = { 1, };
+static int mt7623_otg_drv_vbus0_1_pins[] = { 45, };
+static int mt7623_otg_drv_vbus0_1_funcs[] = { 2, };
+static int mt7623_otg_drv_vbus0_2_pins[] = { 237, };
+static int mt7623_otg_drv_vbus0_2_funcs[] = { 2, };
+static int mt7623_otg_drv_vbus1_0_pins[] = { 26, };
+static int mt7623_otg_drv_vbus1_0_funcs[] = { 2, };
+static int mt7623_otg_drv_vbus1_1_pins[] = { 48, };
+static int mt7623_otg_drv_vbus1_1_funcs[] = { 2, };
+static int mt7623_otg_drv_vbus1_2_pins[] = { 239, };
+static int mt7623_otg_drv_vbus1_2_funcs[] = { 2, };
+
+/* PCIE */
+static int mt7623_pcie0_0_perst_pins[] = { 208, };
+static int mt7623_pcie0_0_perst_funcs[] = { 3, };
+static int mt7623_pcie0_1_perst_pins[] = { 22, };
+static int mt7623_pcie0_1_perst_funcs[] = { 2, };
+static int mt7623_pcie1_0_perst_pins[] = { 209, };
+static int mt7623_pcie1_0_perst_funcs[] = { 3, };
+static int mt7623_pcie1_1_perst_pins[] = { 23, };
+static int mt7623_pcie1_1_perst_funcs[] = { 2, };
+static int mt7623_pcie2_0_perst_pins[] = { 24, };
+static int mt7623_pcie2_0_perst_funcs[] = { 2, };
+static int mt7623_pcie2_1_perst_pins[] = { 29, };
+static int mt7623_pcie2_1_perst_funcs[] = { 6, };
+static int mt7623_pcie0_0_wake_pins[] = { 28, };
+static int mt7623_pcie0_0_wake_funcs[] = { 6, };
+static int mt7623_pcie0_1_wake_pins[] = { 251, };
+static int mt7623_pcie0_1_wake_funcs[] = { 6, };
+static int mt7623_pcie1_0_wake_pins[] = { 27, };
+static int mt7623_pcie1_0_wake_funcs[] = { 6, };
+static int mt7623_pcie1_1_wake_pins[] = { 253, };
+static int mt7623_pcie1_1_wake_funcs[] = { 6, };
+static int mt7623_pcie2_0_wake_pins[] = { 26, };
+static int mt7623_pcie2_0_wake_funcs[] = { 6, };
+static int mt7623_pcie2_1_wake_pins[] = { 255, };
+static int mt7623_pcie2_1_wake_funcs[] = { 6, };
+static int mt7623_pcie0_clkreq_pins[] = { 250, };
+static int mt7623_pcie0_clkreq_funcs[] = { 6, };
+static int mt7623_pcie1_clkreq_pins[] = { 252, };
+static int mt7623_pcie1_clkreq_funcs[] = { 6, };
+static int mt7623_pcie2_clkreq_pins[] = { 254, };
+static int mt7623_pcie2_clkreq_funcs[] = { 6, };
+
+/* the pcie_*_rev are only used for MT7623 */
+static int mt7623_pcie0_0_rev_perst_pins[] = { 208, };
+static int mt7623_pcie0_0_rev_perst_funcs[] = { 11, };
+static int mt7623_pcie0_1_rev_perst_pins[] = { 22, };
+static int mt7623_pcie0_1_rev_perst_funcs[] = { 10, };
+static int mt7623_pcie1_0_rev_perst_pins[] = { 209, };
+static int mt7623_pcie1_0_rev_perst_funcs[] = { 11, };
+static int mt7623_pcie1_1_rev_perst_pins[] = { 23, };
+static int mt7623_pcie1_1_rev_perst_funcs[] = { 10, };
+static int mt7623_pcie2_0_rev_perst_pins[] = { 24, };
+static int mt7623_pcie2_0_rev_perst_funcs[] = { 11, };
+static int mt7623_pcie2_1_rev_perst_pins[] = { 29, };
+static int mt7623_pcie2_1_rev_perst_funcs[] = { 14, };
+
+/* PCM */
+static int mt7623_pcm_clk_0_pins[] = { 18, };
+static int mt7623_pcm_clk_0_funcs[] = { 1, };
+static int mt7623_pcm_clk_1_pins[] = { 17, };
+static int mt7623_pcm_clk_1_funcs[] = { 3, };
+static int mt7623_pcm_clk_2_pins[] = { 35, };
+static int mt7623_pcm_clk_2_funcs[] = { 3, };
+static int mt7623_pcm_clk_3_pins[] = { 50, };
+static int mt7623_pcm_clk_3_funcs[] = { 3, };
+static int mt7623_pcm_clk_4_pins[] = { 74, };
+static int mt7623_pcm_clk_4_funcs[] = { 3, };
+static int mt7623_pcm_clk_5_pins[] = { 191, };
+static int mt7623_pcm_clk_5_funcs[] = { 3, };
+static int mt7623_pcm_clk_6_pins[] = { 196, };
+static int mt7623_pcm_clk_6_funcs[] = { 3, };
+static int mt7623_pcm_sync_0_pins[] = { 19, };
+static int mt7623_pcm_sync_0_funcs[] = { 1, };
+static int mt7623_pcm_sync_1_pins[] = { 30, };
+static int mt7623_pcm_sync_1_funcs[] = { 3, };
+static int mt7623_pcm_sync_2_pins[] = { 36, };
+static int mt7623_pcm_sync_2_funcs[] = { 3, };
+static int mt7623_pcm_sync_3_pins[] = { 52, };
+static int mt7623_pcm_sync_3_funcs[] = { 31, };
+static int mt7623_pcm_sync_4_pins[] = { 73, };
+static int mt7623_pcm_sync_4_funcs[] = { 3, };
+static int mt7623_pcm_sync_5_pins[] = { 192, };
+static int mt7623_pcm_sync_5_funcs[] = { 3, };
+static int mt7623_pcm_sync_6_pins[] = { 197, };
+static int mt7623_pcm_sync_6_funcs[] = { 3, };
+static int mt7623_pcm_rx_0_pins[] = { 20, };
+static int mt7623_pcm_rx_0_funcs[] = { 1, };
+static int mt7623_pcm_rx_1_pins[] = { 16, };
+static int mt7623_pcm_rx_1_funcs[] = { 3, };
+static int mt7623_pcm_rx_2_pins[] = { 34, };
+static int mt7623_pcm_rx_2_funcs[] = { 3, };
+static int mt7623_pcm_rx_3_pins[] = { 51, };
+static int mt7623_pcm_rx_3_funcs[] = { 3, };
+static int mt7623_pcm_rx_4_pins[] = { 72, };
+static int mt7623_pcm_rx_4_funcs[] = { 3, };
+static int mt7623_pcm_rx_5_pins[] = { 190, };
+static int mt7623_pcm_rx_5_funcs[] = { 3, };
+static int mt7623_pcm_rx_6_pins[] = { 195, };
+static int mt7623_pcm_rx_6_funcs[] = { 3, };
+static int mt7623_pcm_tx_0_pins[] = { 21, };
+static int mt7623_pcm_tx_0_funcs[] = { 1, };
+static int mt7623_pcm_tx_1_pins[] = { 32, };
+static int mt7623_pcm_tx_1_funcs[] = { 3, };
+static int mt7623_pcm_tx_2_pins[] = { 33, };
+static int mt7623_pcm_tx_2_funcs[] = { 3, };
+static int mt7623_pcm_tx_3_pins[] = { 38, };
+static int mt7623_pcm_tx_3_funcs[] = { 3, };
+static int mt7623_pcm_tx_4_pins[] = { 49, };
+static int mt7623_pcm_tx_4_funcs[] = { 3, };
+static int mt7623_pcm_tx_5_pins[] = { 189, };
+static int mt7623_pcm_tx_5_funcs[] = { 3, };
+static int mt7623_pcm_tx_6_pins[] = { 194, };
+static int mt7623_pcm_tx_6_funcs[] = { 3, };
+
+/* PWM */
+static int mt7623_pwm_ch1_0_pins[] = { 203, };
+static int mt7623_pwm_ch1_0_funcs[] = { 1, };
+static int mt7623_pwm_ch1_1_pins[] = { 208, };
+static int mt7623_pwm_ch1_1_funcs[] = { 2, };
+static int mt7623_pwm_ch1_2_pins[] = { 72, };
+static int mt7623_pwm_ch1_2_funcs[] = { 4, };
+static int mt7623_pwm_ch1_3_pins[] = { 88, };
+static int mt7623_pwm_ch1_3_funcs[] = { 3, };
+static int mt7623_pwm_ch1_4_pins[] = { 108, };
+static int mt7623_pwm_ch1_4_funcs[] = { 3, };
+static int mt7623_pwm_ch2_0_pins[] = { 204, };
+static int mt7623_pwm_ch2_0_funcs[] = { 1, };
+static int mt7623_pwm_ch2_1_pins[] = { 53, };
+static int mt7623_pwm_ch2_1_funcs[] = { 5, };
+static int mt7623_pwm_ch2_2_pins[] = { 88, };
+static int mt7623_pwm_ch2_2_funcs[] = { 6, };
+static int mt7623_pwm_ch2_3_pins[] = { 108, };
+static int mt7623_pwm_ch2_3_funcs[] = { 6, };
+static int mt7623_pwm_ch2_4_pins[] = { 209, };
+static int mt7623_pwm_ch2_4_funcs[] = { 5, };
+static int mt7623_pwm_ch3_0_pins[] = { 205, };
+static int mt7623_pwm_ch3_0_funcs[] = { 1, };
+static int mt7623_pwm_ch3_1_pins[] = { 55, };
+static int mt7623_pwm_ch3_1_funcs[] = { 5, };
+static int mt7623_pwm_ch3_2_pins[] = { 89, };
+static int mt7623_pwm_ch3_2_funcs[] = { 6, };
+static int mt7623_pwm_ch3_3_pins[] = { 109, };
+static int mt7623_pwm_ch3_3_funcs[] = { 6, };
+static int mt7623_pwm_ch4_0_pins[] = { 206, };
+static int mt7623_pwm_ch4_0_funcs[] = { 1, };
+static int mt7623_pwm_ch4_1_pins[] = { 90, };
+static int mt7623_pwm_ch4_1_funcs[] = { 6, };
+static int mt7623_pwm_ch4_2_pins[] = { 110, };
+static int mt7623_pwm_ch4_2_funcs[] = { 6, };
+static int mt7623_pwm_ch4_3_pins[] = { 124, };
+static int mt7623_pwm_ch4_3_funcs[] = { 5, };
+static int mt7623_pwm_ch5_0_pins[] = { 207, };
+static int mt7623_pwm_ch5_0_funcs[] = { 1, };
+static int mt7623_pwm_ch5_1_pins[] = { 125, };
+static int mt7623_pwm_ch5_1_funcs[] = { 5, };
+
+/* PWRAP */
+static int mt7623_pwrap_pins[] = { 0, 1, 2, 3, 4, 5, 6, };
+static int mt7623_pwrap_funcs[] = { 1, 1, 1, 1, 1, 1, 1, };
+
+/* SPDIF */
+static int mt7623_spdif_in0_0_pins[] = { 56, };
+static int mt7623_spdif_in0_0_funcs[] = { 3, };
+static int mt7623_spdif_in0_1_pins[] = { 201, };
+static int mt7623_spdif_in0_1_funcs[] = { 1, };
+static int mt7623_spdif_in1_0_pins[] = { 54, };
+static int mt7623_spdif_in1_0_funcs[] = { 3, };
+static int mt7623_spdif_in1_1_pins[] = { 202, };
+static int mt7623_spdif_in1_1_funcs[] = { 1, };
+static int mt7623_spdif_out_pins[] = { 202, };
+static int mt7623_spdif_out_funcs[] = { 1, };
+
+/* SPI */
+static int mt7623_spi0_pins[] = { 53, 54, 55, 56, };
+static int mt7623_spi0_funcs[] = { 1, 1, 1, 1, };
+static int mt7623_spi1_pins[] = { 7, 199, 8, 9, };
+static int mt7623_spi1_funcs[] = { 1, 1, 1, 1, };
+static int mt7623_spi2_pins[] = { 101, 104, 102, 103, };
+static int mt7623_spi2_funcs[] = { 1, 1, 1, 1, };
+
+/* UART */
+static int mt7623_uart0_0_txd_rxd_pins[] = { 79, 80, };
+static int mt7623_uart0_0_txd_rxd_funcs[] = { 1, 1, };
+static int mt7623_uart0_1_txd_rxd_pins[] = { 87, 88, };
+static int mt7623_uart0_1_txd_rxd_funcs[] = { 5, 5, };
+static int mt7623_uart0_2_txd_rxd_pins[] = { 107, 108, };
+static int mt7623_uart0_2_txd_rxd_funcs[] = { 5, 5, };
+static int mt7623_uart0_3_txd_rxd_pins[] = { 123, 122, };
+static int mt7623_uart0_3_txd_rxd_funcs[] = { 5, 5, };
+static int mt7623_uart0_rts_cts_pins[] = { 22, 23, };
+static int mt7623_uart0_rts_cts_funcs[] = { 1, 1, };
+static int mt7623_uart1_0_txd_rxd_pins[] = { 81, 82, };
+static int mt7623_uart1_0_txd_rxd_funcs[] = { 1, 1, };
+static int mt7623_uart1_1_txd_rxd_pins[] = { 89, 90, };
+static int mt7623_uart1_1_txd_rxd_funcs[] = { 5, 5, };
+static int mt7623_uart1_2_txd_rxd_pins[] = { 109, 110, };
+static int mt7623_uart1_2_txd_rxd_funcs[] = { 5, 5, };
+static int mt7623_uart1_rts_cts_pins[] = { 24, 25, };
+static int mt7623_uart1_rts_cts_funcs[] = { 1, 1, };
+static int mt7623_uart2_0_txd_rxd_pins[] = { 14, 15, };
+static int mt7623_uart2_0_txd_rxd_funcs[] = { 1, 1, };
+static int mt7623_uart2_1_txd_rxd_pins[] = { 200, 201, };
+static int mt7623_uart2_1_txd_rxd_funcs[] = { 6, 6, };
+static int mt7623_uart2_rts_cts_pins[] = { 242, 243, };
+static int mt7623_uart2_rts_cts_funcs[] = { 1, 1, };
+static int mt7623_uart3_txd_rxd_pins[] = { 242, 243, };
+static int mt7623_uart3_txd_rxd_funcs[] = { 2, 2, };
+static int mt7623_uart3_rts_cts_pins[] = { 26, 27, };
+static int mt7623_uart3_rts_cts_funcs[] = { 1, 1, };
+
+/* Watchdog */
+static int mt7623_watchdog_0_pins[] = { 11, };
+static int mt7623_watchdog_0_funcs[] = { 1, };
+static int mt7623_watchdog_1_pins[] = { 121, };
+static int mt7623_watchdog_1_funcs[] = { 5, };
+
+static const struct group_desc mt7623_groups[] = {
+ PINCTRL_PIN_GROUP("aud_ext_clk0", mt7623_aud_ext_clk0),
+ PINCTRL_PIN_GROUP("aud_ext_clk1", mt7623_aud_ext_clk1),
+ PINCTRL_PIN_GROUP("dsi_te", mt7623_dsi_te),
+ PINCTRL_PIN_GROUP("disp_pwm_0", mt7623_disp_pwm_0),
+ PINCTRL_PIN_GROUP("disp_pwm_1", mt7623_disp_pwm_1),
+ PINCTRL_PIN_GROUP("disp_pwm_2", mt7623_disp_pwm_2),
+ PINCTRL_PIN_GROUP("ephy", mt7623_ephy),
+ PINCTRL_PIN_GROUP("esw_int", mt7623_esw_int),
+ PINCTRL_PIN_GROUP("esw_rst", mt7623_esw_rst),
+ PINCTRL_PIN_GROUP("ext_sdio", mt7623_ext_sdio),
+ PINCTRL_PIN_GROUP("hdmi_cec", mt7623_hdmi_cec),
+ PINCTRL_PIN_GROUP("hdmi_htplg", mt7623_hdmi_htplg),
+ PINCTRL_PIN_GROUP("hdmi_i2c", mt7623_hdmi_i2c),
+ PINCTRL_PIN_GROUP("hdmi_rx", mt7623_hdmi_rx),
+ PINCTRL_PIN_GROUP("hdmi_rx_i2c", mt7623_hdmi_rx_i2c),
+ PINCTRL_PIN_GROUP("i2c0", mt7623_i2c0),
+ PINCTRL_PIN_GROUP("i2c1_0", mt7623_i2c1_0),
+ PINCTRL_PIN_GROUP("i2c1_1", mt7623_i2c1_1),
+ PINCTRL_PIN_GROUP("i2c1_2", mt7623_i2c1_2),
+ PINCTRL_PIN_GROUP("i2c1_3", mt7623_i2c1_3),
+ PINCTRL_PIN_GROUP("i2c1_4", mt7623_i2c1_4),
+ PINCTRL_PIN_GROUP("i2c2_0", mt7623_i2c2_0),
+ PINCTRL_PIN_GROUP("i2c2_1", mt7623_i2c2_1),
+ PINCTRL_PIN_GROUP("i2c2_2", mt7623_i2c2_2),
+ PINCTRL_PIN_GROUP("i2c2_3", mt7623_i2c2_3),
+ PINCTRL_PIN_GROUP("i2s0", mt7623_i2s0),
+ PINCTRL_PIN_GROUP("i2s1", mt7623_i2s1),
+ PINCTRL_PIN_GROUP("i2s4", mt7623_i2s4),
+ PINCTRL_PIN_GROUP("i2s5", mt7623_i2s5),
+ PINCTRL_PIN_GROUP("i2s2_bclk_lrclk_mclk", mt7623_i2s2_bclk_lrclk_mclk),
+ PINCTRL_PIN_GROUP("i2s3_bclk_lrclk_mclk", mt7623_i2s3_bclk_lrclk_mclk),
+ PINCTRL_PIN_GROUP("i2s2_data_in", mt7623_i2s2_data_in),
+ PINCTRL_PIN_GROUP("i2s3_data_in", mt7623_i2s3_data_in),
+ PINCTRL_PIN_GROUP("i2s2_data_0", mt7623_i2s2_data_0),
+ PINCTRL_PIN_GROUP("i2s2_data_1", mt7623_i2s2_data_1),
+ PINCTRL_PIN_GROUP("i2s3_data_0", mt7623_i2s3_data_0),
+ PINCTRL_PIN_GROUP("i2s3_data_1", mt7623_i2s3_data_1),
+ PINCTRL_PIN_GROUP("ir", mt7623_ir),
+ PINCTRL_PIN_GROUP("lcm_rst", mt7623_lcm_rst),
+ PINCTRL_PIN_GROUP("mdc_mdio", mt7623_mdc_mdio),
+ PINCTRL_PIN_GROUP("mipi_tx", mt7623_mipi_tx),
+ PINCTRL_PIN_GROUP("msdc0", mt7623_msdc0),
+ PINCTRL_PIN_GROUP("msdc1", mt7623_msdc1),
+ PINCTRL_PIN_GROUP("msdc1_ins", mt7623_msdc1_ins),
+ PINCTRL_PIN_GROUP("msdc1_wp_0", mt7623_msdc1_wp_0),
+ PINCTRL_PIN_GROUP("msdc1_wp_1", mt7623_msdc1_wp_1),
+ PINCTRL_PIN_GROUP("msdc1_wp_2", mt7623_msdc1_wp_2),
+ PINCTRL_PIN_GROUP("msdc2", mt7623_msdc2),
+ PINCTRL_PIN_GROUP("msdc3", mt7623_msdc3),
+ PINCTRL_PIN_GROUP("nandc", mt7623_nandc),
+ PINCTRL_PIN_GROUP("nandc_ceb0", mt7623_nandc_ceb0),
+ PINCTRL_PIN_GROUP("nandc_ceb1", mt7623_nandc_ceb1),
+ PINCTRL_PIN_GROUP("otg_iddig0_0", mt7623_otg_iddig0_0),
+ PINCTRL_PIN_GROUP("otg_iddig0_1", mt7623_otg_iddig0_1),
+ PINCTRL_PIN_GROUP("otg_iddig0_2", mt7623_otg_iddig0_2),
+ PINCTRL_PIN_GROUP("otg_iddig1_0", mt7623_otg_iddig1_0),
+ PINCTRL_PIN_GROUP("otg_iddig1_1", mt7623_otg_iddig1_1),
+ PINCTRL_PIN_GROUP("otg_iddig1_2", mt7623_otg_iddig1_2),
+ PINCTRL_PIN_GROUP("otg_drv_vbus0_0", mt7623_otg_drv_vbus0_0),
+ PINCTRL_PIN_GROUP("otg_drv_vbus0_1", mt7623_otg_drv_vbus0_1),
+ PINCTRL_PIN_GROUP("otg_drv_vbus0_2", mt7623_otg_drv_vbus0_2),
+ PINCTRL_PIN_GROUP("otg_drv_vbus1_0", mt7623_otg_drv_vbus1_0),
+ PINCTRL_PIN_GROUP("otg_drv_vbus1_1", mt7623_otg_drv_vbus1_1),
+ PINCTRL_PIN_GROUP("otg_drv_vbus1_2", mt7623_otg_drv_vbus1_2),
+ PINCTRL_PIN_GROUP("pcie0_0_perst", mt7623_pcie0_0_perst),
+ PINCTRL_PIN_GROUP("pcie0_1_perst", mt7623_pcie0_1_perst),
+ PINCTRL_PIN_GROUP("pcie1_0_perst", mt7623_pcie1_0_perst),
+ PINCTRL_PIN_GROUP("pcie1_1_perst", mt7623_pcie1_1_perst),
+ PINCTRL_PIN_GROUP("pcie1_1_perst", mt7623_pcie1_1_perst),
+ PINCTRL_PIN_GROUP("pcie0_0_rev_perst", mt7623_pcie0_0_rev_perst),
+ PINCTRL_PIN_GROUP("pcie0_1_rev_perst", mt7623_pcie0_1_rev_perst),
+ PINCTRL_PIN_GROUP("pcie1_0_rev_perst", mt7623_pcie1_0_rev_perst),
+ PINCTRL_PIN_GROUP("pcie1_1_rev_perst", mt7623_pcie1_1_rev_perst),
+ PINCTRL_PIN_GROUP("pcie2_0_rev_perst", mt7623_pcie2_0_rev_perst),
+ PINCTRL_PIN_GROUP("pcie2_1_rev_perst", mt7623_pcie2_1_rev_perst),
+ PINCTRL_PIN_GROUP("pcie2_0_perst", mt7623_pcie2_0_perst),
+ PINCTRL_PIN_GROUP("pcie2_1_perst", mt7623_pcie2_1_perst),
+ PINCTRL_PIN_GROUP("pcie0_0_wake", mt7623_pcie0_0_wake),
+ PINCTRL_PIN_GROUP("pcie0_1_wake", mt7623_pcie0_1_wake),
+ PINCTRL_PIN_GROUP("pcie1_0_wake", mt7623_pcie1_0_wake),
+ PINCTRL_PIN_GROUP("pcie1_1_wake", mt7623_pcie1_1_wake),
+ PINCTRL_PIN_GROUP("pcie2_0_wake", mt7623_pcie2_0_wake),
+ PINCTRL_PIN_GROUP("pcie2_1_wake", mt7623_pcie2_1_wake),
+ PINCTRL_PIN_GROUP("pcie0_clkreq", mt7623_pcie0_clkreq),
+ PINCTRL_PIN_GROUP("pcie1_clkreq", mt7623_pcie1_clkreq),
+ PINCTRL_PIN_GROUP("pcie2_clkreq", mt7623_pcie2_clkreq),
+ PINCTRL_PIN_GROUP("pcm_clk_0", mt7623_pcm_clk_0),
+ PINCTRL_PIN_GROUP("pcm_clk_1", mt7623_pcm_clk_1),
+ PINCTRL_PIN_GROUP("pcm_clk_2", mt7623_pcm_clk_2),
+ PINCTRL_PIN_GROUP("pcm_clk_3", mt7623_pcm_clk_3),
+ PINCTRL_PIN_GROUP("pcm_clk_4", mt7623_pcm_clk_4),
+ PINCTRL_PIN_GROUP("pcm_clk_5", mt7623_pcm_clk_5),
+ PINCTRL_PIN_GROUP("pcm_clk_6", mt7623_pcm_clk_6),
+ PINCTRL_PIN_GROUP("pcm_sync_0", mt7623_pcm_sync_0),
+ PINCTRL_PIN_GROUP("pcm_sync_1", mt7623_pcm_sync_1),
+ PINCTRL_PIN_GROUP("pcm_sync_2", mt7623_pcm_sync_2),
+ PINCTRL_PIN_GROUP("pcm_sync_3", mt7623_pcm_sync_3),
+ PINCTRL_PIN_GROUP("pcm_sync_4", mt7623_pcm_sync_4),
+ PINCTRL_PIN_GROUP("pcm_sync_5", mt7623_pcm_sync_5),
+ PINCTRL_PIN_GROUP("pcm_sync_6", mt7623_pcm_sync_6),
+ PINCTRL_PIN_GROUP("pcm_rx_0", mt7623_pcm_rx_0),
+ PINCTRL_PIN_GROUP("pcm_rx_1", mt7623_pcm_rx_1),
+ PINCTRL_PIN_GROUP("pcm_rx_2", mt7623_pcm_rx_2),
+ PINCTRL_PIN_GROUP("pcm_rx_3", mt7623_pcm_rx_3),
+ PINCTRL_PIN_GROUP("pcm_rx_4", mt7623_pcm_rx_4),
+ PINCTRL_PIN_GROUP("pcm_rx_5", mt7623_pcm_rx_5),
+ PINCTRL_PIN_GROUP("pcm_rx_6", mt7623_pcm_rx_6),
+ PINCTRL_PIN_GROUP("pcm_tx_0", mt7623_pcm_tx_0),
+ PINCTRL_PIN_GROUP("pcm_tx_1", mt7623_pcm_tx_1),
+ PINCTRL_PIN_GROUP("pcm_tx_2", mt7623_pcm_tx_2),
+ PINCTRL_PIN_GROUP("pcm_tx_3", mt7623_pcm_tx_3),
+ PINCTRL_PIN_GROUP("pcm_tx_4", mt7623_pcm_tx_4),
+ PINCTRL_PIN_GROUP("pcm_tx_5", mt7623_pcm_tx_5),
+ PINCTRL_PIN_GROUP("pcm_tx_6", mt7623_pcm_tx_6),
+ PINCTRL_PIN_GROUP("pwm_ch1_0", mt7623_pwm_ch1_0),
+ PINCTRL_PIN_GROUP("pwm_ch1_1", mt7623_pwm_ch1_1),
+ PINCTRL_PIN_GROUP("pwm_ch1_2", mt7623_pwm_ch1_2),
+ PINCTRL_PIN_GROUP("pwm_ch1_3", mt7623_pwm_ch1_3),
+ PINCTRL_PIN_GROUP("pwm_ch1_4", mt7623_pwm_ch1_4),
+ PINCTRL_PIN_GROUP("pwm_ch2_0", mt7623_pwm_ch2_0),
+ PINCTRL_PIN_GROUP("pwm_ch2_1", mt7623_pwm_ch2_1),
+ PINCTRL_PIN_GROUP("pwm_ch2_2", mt7623_pwm_ch2_2),
+ PINCTRL_PIN_GROUP("pwm_ch2_3", mt7623_pwm_ch2_3),
+ PINCTRL_PIN_GROUP("pwm_ch2_4", mt7623_pwm_ch2_4),
+ PINCTRL_PIN_GROUP("pwm_ch3_0", mt7623_pwm_ch3_0),
+ PINCTRL_PIN_GROUP("pwm_ch3_1", mt7623_pwm_ch3_1),
+ PINCTRL_PIN_GROUP("pwm_ch3_2", mt7623_pwm_ch3_2),
+ PINCTRL_PIN_GROUP("pwm_ch3_3", mt7623_pwm_ch3_3),
+ PINCTRL_PIN_GROUP("pwm_ch4_0", mt7623_pwm_ch4_0),
+ PINCTRL_PIN_GROUP("pwm_ch4_1", mt7623_pwm_ch4_1),
+ PINCTRL_PIN_GROUP("pwm_ch4_2", mt7623_pwm_ch4_2),
+ PINCTRL_PIN_GROUP("pwm_ch4_3", mt7623_pwm_ch4_3),
+ PINCTRL_PIN_GROUP("pwm_ch5_0", mt7623_pwm_ch5_0),
+ PINCTRL_PIN_GROUP("pwm_ch5_1", mt7623_pwm_ch5_1),
+ PINCTRL_PIN_GROUP("pwrap", mt7623_pwrap),
+ PINCTRL_PIN_GROUP("rtc", mt7623_rtc),
+ PINCTRL_PIN_GROUP("spdif_in0_0", mt7623_spdif_in0_0),
+ PINCTRL_PIN_GROUP("spdif_in0_1", mt7623_spdif_in0_1),
+ PINCTRL_PIN_GROUP("spdif_in1_0", mt7623_spdif_in1_0),
+ PINCTRL_PIN_GROUP("spdif_in1_1", mt7623_spdif_in1_1),
+ PINCTRL_PIN_GROUP("spdif_out", mt7623_spdif_out),
+ PINCTRL_PIN_GROUP("spi0", mt7623_spi0),
+ PINCTRL_PIN_GROUP("spi1", mt7623_spi1),
+ PINCTRL_PIN_GROUP("spi2", mt7623_spi2),
+ PINCTRL_PIN_GROUP("uart0_0_txd_rxd", mt7623_uart0_0_txd_rxd),
+ PINCTRL_PIN_GROUP("uart0_1_txd_rxd", mt7623_uart0_1_txd_rxd),
+ PINCTRL_PIN_GROUP("uart0_2_txd_rxd", mt7623_uart0_2_txd_rxd),
+ PINCTRL_PIN_GROUP("uart0_3_txd_rxd", mt7623_uart0_3_txd_rxd),
+ PINCTRL_PIN_GROUP("uart1_0_txd_rxd", mt7623_uart1_0_txd_rxd),
+ PINCTRL_PIN_GROUP("uart1_1_txd_rxd", mt7623_uart1_1_txd_rxd),
+ PINCTRL_PIN_GROUP("uart1_2_txd_rxd", mt7623_uart1_2_txd_rxd),
+ PINCTRL_PIN_GROUP("uart2_0_txd_rxd", mt7623_uart2_0_txd_rxd),
+ PINCTRL_PIN_GROUP("uart2_1_txd_rxd", mt7623_uart2_1_txd_rxd),
+ PINCTRL_PIN_GROUP("uart3_txd_rxd", mt7623_uart3_txd_rxd),
+ PINCTRL_PIN_GROUP("uart0_rts_cts", mt7623_uart0_rts_cts),
+ PINCTRL_PIN_GROUP("uart1_rts_cts", mt7623_uart1_rts_cts),
+ PINCTRL_PIN_GROUP("uart2_rts_cts", mt7623_uart2_rts_cts),
+ PINCTRL_PIN_GROUP("uart3_rts_cts", mt7623_uart3_rts_cts),
+ PINCTRL_PIN_GROUP("watchdog_0", mt7623_watchdog_0),
+ PINCTRL_PIN_GROUP("watchdog_1", mt7623_watchdog_1),
+};
+
+/* Joint those groups owning the same capability in user point of view which
+ * allows that people tend to use through the device tree.
+ */
+static const char *mt7623_aud_clk_groups[] = { "aud_ext_clk0",
+ "aud_ext_clk1", };
+static const char *mt7623_disp_pwm_groups[] = { "disp_pwm_0", "disp_pwm_1",
+ "disp_pwm_2", };
+static const char *mt7623_ethernet_groups[] = { "esw_int", "esw_rst",
+ "ephy", "mdc_mdio", };
+static const char *mt7623_ext_sdio_groups[] = { "ext_sdio", };
+static const char *mt7623_hdmi_groups[] = { "hdmi_cec", "hdmi_htplg",
+ "hdmi_i2c", "hdmi_rx",
+ "hdmi_rx_i2c", };
+static const char *mt7623_i2c_groups[] = { "i2c0", "i2c1_0", "i2c1_1",
+ "i2c1_2", "i2c1_3", "i2c1_4",
+ "i2c2_0", "i2c2_1", "i2c2_2",
+ "i2c2_3", };
+static const char *mt7623_i2s_groups[] = { "i2s0", "i2s1",
+ "i2s2_bclk_lrclk_mclk",
+ "i2s3_bclk_lrclk_mclk",
+ "i2s4", "i2s5",
+ "i2s2_data_in", "i2s3_data_in",
+ "i2s2_data_0", "i2s2_data_1",
+ "i2s3_data_0", "i2s3_data_1", };
+static const char *mt7623_ir_groups[] = { "ir", };
+static const char *mt7623_lcd_groups[] = { "dsi_te", "lcm_rst", "mipi_tx", };
+static const char *mt7623_msdc_groups[] = { "msdc0", "msdc1", "msdc1_ins",
+ "msdc1_wp_0", "msdc1_wp_1",
+ "msdc1_wp_2", "msdc2",
+ "msdc3", };
+static const char *mt7623_nandc_groups[] = { "nandc", "nandc_ceb0",
+ "nandc_ceb1", };
+static const char *mt7623_otg_groups[] = { "otg_iddig0_0", "otg_iddig0_1",
+ "otg_iddig0_2", "otg_iddig1_0",
+ "otg_iddig1_1", "otg_iddig1_2",
+ "otg_drv_vbus0_0",
+ "otg_drv_vbus0_1",
+ "otg_drv_vbus0_2",
+ "otg_drv_vbus1_0",
+ "otg_drv_vbus1_1",
+ "otg_drv_vbus1_2", };
+static const char *mt7623_pcie_groups[] = { "pcie0_0_perst", "pcie0_1_perst",
+ "pcie1_0_perst", "pcie1_1_perst",
+ "pcie2_0_perst", "pcie2_1_perst",
+ "pcie0_0_rev_perst",
+ "pcie0_1_rev_perst",
+ "pcie1_0_rev_perst",
+ "pcie1_1_rev_perst",
+ "pcie2_0_rev_perst",
+ "pcie2_1_rev_perst",
+ "pcie0_0_wake", "pcie0_1_wake",
+ "pcie2_0_wake", "pcie2_1_wake",
+ "pcie0_clkreq", "pcie1_clkreq",
+ "pcie2_clkreq", };
+static const char *mt7623_pcm_groups[] = { "pcm_clk_0", "pcm_clk_1",
+ "pcm_clk_2", "pcm_clk_3",
+ "pcm_clk_4", "pcm_clk_5",
+ "pcm_clk_6", "pcm_sync_0",
+ "pcm_sync_1", "pcm_sync_2",
+ "pcm_sync_3", "pcm_sync_4",
+ "pcm_sync_5", "pcm_sync_6",
+ "pcm_rx_0", "pcm_rx_1",
+ "pcm_rx_2", "pcm_rx_3",
+ "pcm_rx_4", "pcm_rx_5",
+ "pcm_rx_6", "pcm_tx_0",
+ "pcm_tx_1", "pcm_tx_2",
+ "pcm_tx_3", "pcm_tx_4",
+ "pcm_tx_5", "pcm_tx_6", };
+static const char *mt7623_pwm_groups[] = { "pwm_ch1_0", "pwm_ch1_1",
+ "pwm_ch1_2", "pwm_ch2_0",
+ "pwm_ch2_1", "pwm_ch2_2",
+ "pwm_ch3_0", "pwm_ch3_1",
+ "pwm_ch3_2", "pwm_ch4_0",
+ "pwm_ch4_1", "pwm_ch4_2",
+ "pwm_ch4_3", "pwm_ch5_0",
+ "pwm_ch5_1", "pwm_ch5_2",
+ "pwm_ch6_0", "pwm_ch6_1",
+ "pwm_ch6_2", "pwm_ch6_3",
+ "pwm_ch7_0", "pwm_ch7_1",
+ "pwm_ch7_2", };
+static const char *mt7623_pwrap_groups[] = { "pwrap", };
+static const char *mt7623_rtc_groups[] = { "rtc", };
+static const char *mt7623_spi_groups[] = { "spi0", "spi2", "spi2", };
+static const char *mt7623_spdif_groups[] = { "spdif_in0_0", "spdif_in0_1",
+ "spdif_in1_0", "spdif_in1_1",
+ "spdif_out", };
+static const char *mt7623_uart_groups[] = { "uart0_0_txd_rxd",
+ "uart0_1_txd_rxd",
+ "uart0_2_txd_rxd",
+ "uart0_3_txd_rxd",
+ "uart1_0_txd_rxd",
+ "uart1_1_txd_rxd",
+ "uart1_2_txd_rxd",
+ "uart2_0_txd_rxd",
+ "uart2_1_txd_rxd",
+ "uart3_txd_rxd",
+ "uart0_rts_cts",
+ "uart1_rts_cts",
+ "uart2_rts_cts",
+ "uart3_rts_cts", };
+static const char *mt7623_wdt_groups[] = { "watchdog_0", "watchdog_1", };
+
+static const struct function_desc mt7623_functions[] = {
+ {"audck", mt7623_aud_clk_groups, ARRAY_SIZE(mt7623_aud_clk_groups)},
+ {"disp", mt7623_disp_pwm_groups, ARRAY_SIZE(mt7623_disp_pwm_groups)},
+ {"eth", mt7623_ethernet_groups, ARRAY_SIZE(mt7623_ethernet_groups)},
+ {"sdio", mt7623_ext_sdio_groups, ARRAY_SIZE(mt7623_ext_sdio_groups)},
+ {"hdmi", mt7623_hdmi_groups, ARRAY_SIZE(mt7623_hdmi_groups)},
+ {"i2c", mt7623_i2c_groups, ARRAY_SIZE(mt7623_i2c_groups)},
+ {"i2s", mt7623_i2s_groups, ARRAY_SIZE(mt7623_i2s_groups)},
+ {"ir", mt7623_ir_groups, ARRAY_SIZE(mt7623_ir_groups)},
+ {"lcd", mt7623_lcd_groups, ARRAY_SIZE(mt7623_lcd_groups)},
+ {"msdc", mt7623_msdc_groups, ARRAY_SIZE(mt7623_msdc_groups)},
+ {"nand", mt7623_nandc_groups, ARRAY_SIZE(mt7623_nandc_groups)},
+ {"otg", mt7623_otg_groups, ARRAY_SIZE(mt7623_otg_groups)},
+ {"pcie", mt7623_pcie_groups, ARRAY_SIZE(mt7623_pcie_groups)},
+ {"pcm", mt7623_pcm_groups, ARRAY_SIZE(mt7623_pcm_groups)},
+ {"pwm", mt7623_pwm_groups, ARRAY_SIZE(mt7623_pwm_groups)},
+ {"pwrap", mt7623_pwrap_groups, ARRAY_SIZE(mt7623_pwrap_groups)},
+ {"rtc", mt7623_rtc_groups, ARRAY_SIZE(mt7623_rtc_groups)},
+ {"spi", mt7623_spi_groups, ARRAY_SIZE(mt7623_spi_groups)},
+ {"spdif", mt7623_spdif_groups, ARRAY_SIZE(mt7623_spdif_groups)},
+ {"uart", mt7623_uart_groups, ARRAY_SIZE(mt7623_uart_groups)},
+ {"watchdog", mt7623_wdt_groups, ARRAY_SIZE(mt7623_wdt_groups)},
+};
+
+static const struct mtk_eint_hw mt7623_eint_hw = {
+ .port_mask = 6,
+ .ports = 6,
+ .ap_num = 169,
+ .db_cnt = 20,
+};
+
+static struct mtk_pin_soc mt7623_data = {
+ .reg_cal = mt7623_reg_cals,
+ .pins = mt7623_pins,
+ .npins = ARRAY_SIZE(mt7623_pins),
+ .grps = mt7623_groups,
+ .ngrps = ARRAY_SIZE(mt7623_groups),
+ .funcs = mt7623_functions,
+ .nfuncs = ARRAY_SIZE(mt7623_functions),
+ .eint_hw = &mt7623_eint_hw,
+ .gpio_m = 0,
+ .ies_present = true,
+ .base_names = mtk_default_register_base_names,
+ .nbase_names = ARRAY_SIZE(mtk_default_register_base_names),
+ .bias_disable_set = mtk_pinconf_bias_disable_set_rev1,
+ .bias_disable_get = mtk_pinconf_bias_disable_get_rev1,
+ .bias_set = mtk_pinconf_bias_set_rev1,
+ .bias_get = mtk_pinconf_bias_get_rev1,
+ .drive_set = mtk_pinconf_drive_set_rev1,
+ .drive_get = mtk_pinconf_drive_get_rev1,
+ .adv_pull_get = mtk_pinconf_adv_pull_get,
+ .adv_pull_set = mtk_pinconf_adv_pull_set,
+};
+
+/*
+ * There are some specific pins have mux functions greater than 8,
+ * and if we want to switch thees high modes we need to disable
+ * bonding constraints firstly.
+ */
+static void mt7623_bonding_disable(struct platform_device *pdev)
+{
+ struct mtk_pinctrl *hw = platform_get_drvdata(pdev);
+
+ mtk_rmw(hw, 0, PIN_BOND_REG0, BOND_PCIE_CLR, BOND_PCIE_CLR);
+ mtk_rmw(hw, 0, PIN_BOND_REG1, BOND_I2S_CLR, BOND_I2S_CLR);
+ mtk_rmw(hw, 0, PIN_BOND_REG2, BOND_MSDC0E_CLR, BOND_MSDC0E_CLR);
+}
+
+static const struct of_device_id mt7623_pctrl_match[] = {
+ { .compatible = "mediatek,mt7623-moore-pinctrl", },
+ {}
+};
+
+static int mt7623_pinctrl_probe(struct platform_device *pdev)
+{
+ int err;
+
+ err = mtk_moore_pinctrl_probe(pdev, &mt7623_data);
+ if (err)
+ return err;
+
+ mt7623_bonding_disable(pdev);
+
+ return 0;
+}
+
+static struct platform_driver mtk_pinctrl_driver = {
+ .probe = mt7623_pinctrl_probe,
+ .driver = {
+ .name = "mt7623-moore-pinctrl",
+ .of_match_table = mt7623_pctrl_match,
+ },
+};
+
+static int __init mtk_pinctrl_init(void)
+{
+ return platform_driver_register(&mtk_pinctrl_driver);
+}
+arch_initcall(mtk_pinctrl_init);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 MediaTek Inc.
+ *
+ * Author: Zhiyong Tao <zhiyong.tao@mediatek.com>
+ *
+ */
+
+#include "pinctrl-mtk-mt8183.h"
+#include "pinctrl-paris.h"
+
+/* MT8183 have multiple bases to program pin configuration listed as the below:
+ * iocfg[0]:0x10005000, iocfg[1]:0x11F20000, iocfg[2]:0x11E80000,
+ * iocfg[3]:0x11E70000, iocfg[4]:0x11E90000, iocfg[5]:0x11D30000,
+ * iocfg[6]:0x11D20000, iocfg[7]:0x11C50000, iocfg[8]:0x11F30000.
+ * _i_based could be used to indicate what base the pin should be mapped into.
+ */
+
+#define PIN_FIELD_BASE(_s_pin, _e_pin, _i_base, _s_addr, _x_addrs, _s_bit, _x_bits) \
+ PIN_FIELD_CALC(_s_pin, _e_pin, _i_base, _s_addr, _x_addrs, _s_bit, \
+ _x_bits, 32, 0)
+
+#define PINS_FIELD_BASE(_s_pin, _e_pin, _i_base, _s_addr, _x_addrs, _s_bit, _x_bits) \
+ PIN_FIELD_CALC(_s_pin, _e_pin, _i_base, _s_addr, _x_addrs, _s_bit, \
+ _x_bits, 32, 1)
+
+static const struct mtk_pin_field_calc mt8183_pin_mode_range[] = {
+ PIN_FIELD(0, 192, 0x300, 0x10, 0, 4),
+};
+
+static const struct mtk_pin_field_calc mt8183_pin_dir_range[] = {
+ PIN_FIELD(0, 192, 0x0, 0x10, 0, 1),
+};
+
+static const struct mtk_pin_field_calc mt8183_pin_di_range[] = {
+ PIN_FIELD(0, 192, 0x200, 0x10, 0, 1),
+};
+
+static const struct mtk_pin_field_calc mt8183_pin_do_range[] = {
+ PIN_FIELD(0, 192, 0x100, 0x10, 0, 1),
+};
+
+static const struct mtk_pin_field_calc mt8183_pin_ies_range[] = {
+ PINS_FIELD_BASE(0, 3, 6, 0x000, 0x10, 3, 1),
+ PINS_FIELD_BASE(4, 7, 6, 0x000, 0x10, 5, 1),
+ PIN_FIELD_BASE(8, 8, 6, 0x000, 0x10, 0, 1),
+ PINS_FIELD_BASE(9, 10, 6, 0x000, 0x10, 12, 1),
+ PIN_FIELD_BASE(11, 11, 1, 0x000, 0x10, 3, 1),
+ PIN_FIELD_BASE(12, 12, 1, 0x000, 0x10, 7, 1),
+ PINS_FIELD_BASE(13, 16, 2, 0x000, 0x10, 2, 1),
+ PINS_FIELD_BASE(17, 20, 2, 0x000, 0x10, 3, 1),
+ PINS_FIELD_BASE(21, 24, 2, 0x000, 0x10, 4, 1),
+ PINS_FIELD_BASE(25, 28, 2, 0x000, 0x10, 5, 1),
+ PIN_FIELD_BASE(29, 29, 2, 0x000, 0x10, 6, 1),
+ PIN_FIELD_BASE(30, 30, 2, 0x000, 0x10, 7, 1),
+ PINS_FIELD_BASE(31, 31, 2, 0x000, 0x10, 8, 1),
+ PINS_FIELD_BASE(32, 34, 2, 0x000, 0x10, 7, 1),
+ PINS_FIELD_BASE(35, 37, 3, 0x000, 0x10, 0, 1),
+ PINS_FIELD_BASE(38, 40, 3, 0x000, 0x10, 1, 1),
+ PINS_FIELD_BASE(41, 42, 3, 0x000, 0x10, 2, 1),
+ PINS_FIELD_BASE(43, 45, 3, 0x000, 0x10, 3, 1),
+ PINS_FIELD_BASE(46, 47, 3, 0x000, 0x10, 4, 1),
+ PINS_FIELD_BASE(48, 49, 3, 0x000, 0x10, 5, 1),
+ PINS_FIELD_BASE(50, 51, 4, 0x000, 0x10, 0, 1),
+ PINS_FIELD_BASE(52, 57, 4, 0x000, 0x10, 1, 1),
+ PINS_FIELD_BASE(58, 60, 4, 0x000, 0x10, 2, 1),
+ PINS_FIELD_BASE(61, 64, 5, 0x000, 0x10, 0, 1),
+ PINS_FIELD_BASE(65, 66, 5, 0x000, 0x10, 1, 1),
+ PINS_FIELD_BASE(67, 68, 5, 0x000, 0x10, 2, 1),
+ PINS_FIELD_BASE(69, 71, 5, 0x000, 0x10, 3, 1),
+ PINS_FIELD_BASE(72, 76, 5, 0x000, 0x10, 4, 1),
+ PINS_FIELD_BASE(77, 80, 5, 0x000, 0x10, 5, 1),
+ PIN_FIELD_BASE(81, 81, 5, 0x000, 0x10, 6, 1),
+ PINS_FIELD_BASE(82, 83, 5, 0x000, 0x10, 7, 1),
+ PIN_FIELD_BASE(84, 84, 5, 0x000, 0x10, 6, 1),
+ PINS_FIELD_BASE(85, 88, 5, 0x000, 0x10, 8, 1),
+ PIN_FIELD_BASE(89, 89, 6, 0x000, 0x10, 11, 1),
+ PIN_FIELD_BASE(90, 90, 6, 0x000, 0x10, 1, 1),
+ PINS_FIELD_BASE(91, 94, 6, 0x000, 0x10, 2, 1),
+ PINS_FIELD_BASE(95, 96, 6, 0x000, 0x10, 6, 1),
+ PINS_FIELD_BASE(97, 98, 6, 0x000, 0x10, 7, 1),
+ PIN_FIELD_BASE(99, 99, 6, 0x000, 0x10, 8, 1),
+ PIN_FIELD_BASE(100, 100, 6, 0x000, 0x10, 9, 1),
+ PINS_FIELD_BASE(101, 102, 6, 0x000, 0x10, 10, 1),
+ PINS_FIELD_BASE(103, 104, 6, 0x000, 0x10, 13, 1),
+ PINS_FIELD_BASE(105, 106, 6, 0x000, 0x10, 14, 1),
+ PIN_FIELD_BASE(107, 107, 7, 0x000, 0x10, 0, 1),
+ PIN_FIELD_BASE(108, 108, 7, 0x000, 0x10, 1, 1),
+ PIN_FIELD_BASE(109, 109, 7, 0x000, 0x10, 2, 1),
+ PIN_FIELD_BASE(110, 110, 7, 0x000, 0x10, 0, 1),
+ PIN_FIELD_BASE(111, 111, 7, 0x000, 0x10, 3, 1),
+ PIN_FIELD_BASE(112, 112, 7, 0x000, 0x10, 2, 1),
+ PIN_FIELD_BASE(113, 113, 7, 0x000, 0x10, 4, 1),
+ PIN_FIELD_BASE(114, 114, 7, 0x000, 0x10, 5, 1),
+ PIN_FIELD_BASE(115, 115, 7, 0x000, 0x10, 6, 1),
+ PIN_FIELD_BASE(116, 116, 7, 0x000, 0x10, 7, 1),
+ PIN_FIELD_BASE(117, 117, 7, 0x000, 0x10, 8, 1),
+ PIN_FIELD_BASE(118, 118, 7, 0x000, 0x10, 9, 1),
+ PIN_FIELD_BASE(119, 119, 7, 0x000, 0x10, 10, 1),
+ PIN_FIELD_BASE(120, 120, 7, 0x000, 0x10, 11, 1),
+ PIN_FIELD_BASE(121, 121, 7, 0x000, 0x10, 12, 1),
+ PIN_FIELD_BASE(122, 122, 8, 0x000, 0x10, 0, 1),
+ PIN_FIELD_BASE(123, 123, 8, 0x000, 0x10, 1, 1),
+ PIN_FIELD_BASE(124, 124, 8, 0x000, 0x10, 2, 1),
+ PINS_FIELD_BASE(125, 130, 8, 0x000, 0x10, 1, 1),
+ PIN_FIELD_BASE(131, 131, 8, 0x000, 0x10, 3, 1),
+ PIN_FIELD_BASE(132, 132, 8, 0x000, 0x10, 1, 1),
+ PIN_FIELD_BASE(133, 133, 8, 0x000, 0x10, 4, 1),
+ PIN_FIELD_BASE(134, 134, 1, 0x000, 0x10, 0, 1),
+ PIN_FIELD_BASE(135, 135, 1, 0x000, 0x10, 1, 1),
+ PINS_FIELD_BASE(136, 143, 1, 0x000, 0x10, 2, 1),
+ PINS_FIELD_BASE(144, 147, 1, 0x000, 0x10, 4, 1),
+ PIN_FIELD_BASE(148, 148, 1, 0x000, 0x10, 5, 1),
+ PIN_FIELD_BASE(149, 149, 1, 0x000, 0x10, 6, 1),
+ PINS_FIELD_BASE(150, 153, 1, 0x000, 0x10, 8, 1),
+ PIN_FIELD_BASE(154, 154, 1, 0x000, 0x10, 9, 1),
+ PINS_FIELD_BASE(155, 157, 1, 0x000, 0x10, 10, 1),
+ PINS_FIELD_BASE(158, 160, 1, 0x000, 0x10, 8, 1),
+ PINS_FIELD_BASE(161, 164, 2, 0x000, 0x10, 0, 1),
+ PINS_FIELD_BASE(165, 166, 2, 0x000, 0x10, 1, 1),
+ PINS_FIELD_BASE(167, 168, 4, 0x000, 0x10, 2, 1),
+ PIN_FIELD_BASE(169, 169, 4, 0x000, 0x10, 3, 1),
+ PINS_FIELD_BASE(170, 174, 4, 0x000, 0x10, 4, 1),
+ PINS_FIELD_BASE(175, 176, 4, 0x000, 0x10, 3, 1),
+ PINS_FIELD_BASE(177, 179, 6, 0x000, 0x10, 4, 1),
+};
+
+static const struct mtk_pin_field_calc mt8183_pin_smt_range[] = {
+ PINS_FIELD_BASE(0, 3, 6, 0x010, 0x10, 3, 1),
+ PINS_FIELD_BASE(4, 7, 6, 0x010, 0x10, 5, 1),
+ PIN_FIELD_BASE(8, 8, 6, 0x010, 0x10, 0, 1),
+ PINS_FIELD_BASE(9, 10, 6, 0x010, 0x10, 12, 1),
+ PIN_FIELD_BASE(11, 11, 1, 0x010, 0x10, 3, 1),
+ PIN_FIELD_BASE(12, 12, 1, 0x010, 0x10, 7, 1),
+ PINS_FIELD_BASE(13, 16, 2, 0x010, 0x10, 2, 1),
+ PINS_FIELD_BASE(17, 20, 2, 0x010, 0x10, 3, 1),
+ PINS_FIELD_BASE(21, 24, 2, 0x010, 0x10, 4, 1),
+ PINS_FIELD_BASE(25, 28, 2, 0x010, 0x10, 5, 1),
+ PIN_FIELD_BASE(29, 29, 2, 0x010, 0x10, 6, 1),
+ PIN_FIELD_BASE(30, 30, 2, 0x010, 0x10, 7, 1),
+ PINS_FIELD_BASE(31, 31, 2, 0x010, 0x10, 8, 1),
+ PINS_FIELD_BASE(32, 34, 2, 0x010, 0x10, 7, 1),
+ PINS_FIELD_BASE(35, 37, 3, 0x010, 0x10, 0, 1),
+ PINS_FIELD_BASE(38, 40, 3, 0x010, 0x10, 1, 1),
+ PINS_FIELD_BASE(41, 42, 3, 0x010, 0x10, 2, 1),
+ PINS_FIELD_BASE(43, 45, 3, 0x010, 0x10, 3, 1),
+ PINS_FIELD_BASE(46, 47, 3, 0x010, 0x10, 4, 1),
+ PINS_FIELD_BASE(48, 49, 3, 0x010, 0x10, 5, 1),
+ PINS_FIELD_BASE(50, 51, 4, 0x010, 0x10, 0, 1),
+ PINS_FIELD_BASE(52, 57, 4, 0x010, 0x10, 1, 1),
+ PINS_FIELD_BASE(58, 60, 4, 0x010, 0x10, 2, 1),
+ PINS_FIELD_BASE(61, 64, 5, 0x010, 0x10, 0, 1),
+ PINS_FIELD_BASE(65, 66, 5, 0x010, 0x10, 1, 1),
+ PINS_FIELD_BASE(67, 68, 5, 0x010, 0x10, 2, 1),
+ PINS_FIELD_BASE(69, 71, 5, 0x010, 0x10, 3, 1),
+ PINS_FIELD_BASE(72, 76, 5, 0x010, 0x10, 4, 1),
+ PINS_FIELD_BASE(77, 80, 5, 0x010, 0x10, 5, 1),
+ PIN_FIELD_BASE(81, 81, 5, 0x010, 0x10, 6, 1),
+ PINS_FIELD_BASE(82, 83, 5, 0x010, 0x10, 7, 1),
+ PIN_FIELD_BASE(84, 84, 5, 0x010, 0x10, 6, 1),
+ PINS_FIELD_BASE(85, 88, 5, 0x010, 0x10, 8, 1),
+ PIN_FIELD_BASE(89, 89, 6, 0x010, 0x10, 11, 1),
+ PIN_FIELD_BASE(90, 90, 6, 0x010, 0x10, 1, 1),
+ PINS_FIELD_BASE(91, 94, 6, 0x010, 0x10, 2, 1),
+ PINS_FIELD_BASE(95, 96, 6, 0x010, 0x10, 6, 1),
+ PINS_FIELD_BASE(97, 98, 6, 0x010, 0x10, 7, 1),
+ PIN_FIELD_BASE(99, 99, 6, 0x010, 0x10, 8, 1),
+ PIN_FIELD_BASE(100, 100, 6, 0x010, 0x10, 9, 1),
+ PINS_FIELD_BASE(101, 102, 6, 0x010, 0x10, 10, 1),
+ PINS_FIELD_BASE(103, 104, 6, 0x010, 0x10, 13, 1),
+ PINS_FIELD_BASE(105, 106, 6, 0x010, 0x10, 14, 1),
+ PIN_FIELD_BASE(107, 107, 7, 0x010, 0x10, 0, 1),
+ PIN_FIELD_BASE(108, 108, 7, 0x010, 0x10, 1, 1),
+ PIN_FIELD_BASE(109, 109, 7, 0x010, 0x10, 2, 1),
+ PIN_FIELD_BASE(110, 110, 7, 0x010, 0x10, 0, 1),
+ PIN_FIELD_BASE(111, 111, 7, 0x010, 0x10, 3, 1),
+ PIN_FIELD_BASE(112, 112, 7, 0x010, 0x10, 2, 1),
+ PIN_FIELD_BASE(113, 113, 7, 0x010, 0x10, 4, 1),
+ PIN_FIELD_BASE(114, 114, 7, 0x010, 0x10, 5, 1),
+ PIN_FIELD_BASE(115, 115, 7, 0x010, 0x10, 6, 1),
+ PIN_FIELD_BASE(116, 116, 7, 0x010, 0x10, 7, 1),
+ PIN_FIELD_BASE(117, 117, 7, 0x010, 0x10, 8, 1),
+ PIN_FIELD_BASE(118, 118, 7, 0x010, 0x10, 9, 1),
+ PIN_FIELD_BASE(119, 119, 7, 0x010, 0x10, 10, 1),
+ PIN_FIELD_BASE(120, 120, 7, 0x010, 0x10, 11, 1),
+ PIN_FIELD_BASE(121, 121, 7, 0x010, 0x10, 12, 1),
+ PIN_FIELD_BASE(122, 122, 8, 0x010, 0x10, 0, 1),
+ PIN_FIELD_BASE(123, 123, 8, 0x010, 0x10, 1, 1),
+ PIN_FIELD_BASE(124, 124, 8, 0x010, 0x10, 2, 1),
+ PINS_FIELD_BASE(125, 130, 8, 0x010, 0x10, 1, 1),
+ PIN_FIELD_BASE(131, 131, 8, 0x010, 0x10, 3, 1),
+ PIN_FIELD_BASE(132, 132, 8, 0x010, 0x10, 1, 1),
+ PIN_FIELD_BASE(133, 133, 8, 0x010, 0x10, 4, 1),
+ PIN_FIELD_BASE(134, 134, 1, 0x010, 0x10, 0, 1),
+ PIN_FIELD_BASE(135, 135, 1, 0x010, 0x10, 1, 1),
+ PINS_FIELD_BASE(136, 143, 1, 0x010, 0x10, 2, 1),
+ PINS_FIELD_BASE(144, 147, 1, 0x010, 0x10, 4, 1),
+ PIN_FIELD_BASE(148, 148, 1, 0x010, 0x10, 5, 1),
+ PIN_FIELD_BASE(149, 149, 1, 0x010, 0x10, 6, 1),
+ PINS_FIELD_BASE(150, 153, 1, 0x010, 0x10, 8, 1),
+ PIN_FIELD_BASE(154, 154, 1, 0x010, 0x10, 9, 1),
+ PINS_FIELD_BASE(155, 157, 1, 0x010, 0x10, 10, 1),
+ PINS_FIELD_BASE(158, 160, 1, 0x010, 0x10, 8, 1),
+ PINS_FIELD_BASE(161, 164, 2, 0x010, 0x10, 0, 1),
+ PINS_FIELD_BASE(165, 166, 2, 0x010, 0x10, 1, 1),
+ PINS_FIELD_BASE(167, 168, 4, 0x010, 0x10, 2, 1),
+ PIN_FIELD_BASE(169, 169, 4, 0x010, 0x10, 3, 1),
+ PINS_FIELD_BASE(170, 174, 4, 0x010, 0x10, 4, 1),
+ PINS_FIELD_BASE(175, 176, 4, 0x010, 0x10, 3, 1),
+ PINS_FIELD_BASE(177, 179, 6, 0x010, 0x10, 4, 1),
+};
+
+static const struct mtk_pin_field_calc mt8183_pin_pullen_range[] = {
+ PIN_FIELD_BASE(0, 3, 6, 0x060, 0x10, 6, 1),
+ PIN_FIELD_BASE(4, 7, 6, 0x060, 0x10, 11, 1),
+ PIN_FIELD_BASE(8, 8, 6, 0x060, 0x10, 0, 1),
+ PIN_FIELD_BASE(9, 10, 6, 0x060, 0x10, 26, 1),
+ PIN_FIELD_BASE(11, 11, 1, 0x060, 0x10, 10, 1),
+ PIN_FIELD_BASE(12, 12, 1, 0x060, 0x10, 17, 1),
+ PIN_FIELD_BASE(13, 28, 2, 0x060, 0x10, 6, 1),
+ PIN_FIELD_BASE(43, 49, 3, 0x060, 0x10, 8, 1),
+ PIN_FIELD_BASE(50, 60, 4, 0x060, 0x10, 0, 1),
+ PIN_FIELD_BASE(61, 88, 5, 0x060, 0x10, 0, 1),
+ PIN_FIELD_BASE(89, 89, 6, 0x060, 0x10, 24, 1),
+ PIN_FIELD_BASE(90, 90, 6, 0x060, 0x10, 1, 1),
+ PIN_FIELD_BASE(95, 95, 6, 0x060, 0x10, 15, 1),
+ PIN_FIELD_BASE(96, 102, 6, 0x060, 0x10, 17, 1),
+ PIN_FIELD_BASE(103, 106, 6, 0x060, 0x10, 28, 1),
+ PIN_FIELD_BASE(107, 121, 7, 0x060, 0x10, 0, 1),
+ PIN_FIELD_BASE(134, 143, 1, 0x060, 0x10, 0, 1),
+ PIN_FIELD_BASE(144, 149, 1, 0x060, 0x10, 11, 1),
+ PIN_FIELD_BASE(150, 160, 1, 0x060, 0x10, 18, 1),
+ PIN_FIELD_BASE(161, 166, 2, 0x060, 0x10, 0, 1),
+ PIN_FIELD_BASE(167, 176, 4, 0x060, 0x10, 11, 1),
+ PIN_FIELD_BASE(177, 177, 6, 0x060, 0x10, 10, 1),
+ PIN_FIELD_BASE(178, 178, 6, 0x060, 0x10, 16, 1),
+ PIN_FIELD_BASE(179, 179, 6, 0x060, 0x10, 25, 1),
+};
+
+static const struct mtk_pin_field_calc mt8183_pin_pullsel_range[] = {
+ PIN_FIELD_BASE(0, 3, 6, 0x080, 0x10, 6, 1),
+ PIN_FIELD_BASE(4, 7, 6, 0x080, 0x10, 11, 1),
+ PIN_FIELD_BASE(8, 8, 6, 0x080, 0x10, 0, 1),
+ PIN_FIELD_BASE(9, 10, 6, 0x080, 0x10, 26, 1),
+ PIN_FIELD_BASE(11, 11, 1, 0x080, 0x10, 10, 1),
+ PIN_FIELD_BASE(12, 12, 1, 0x080, 0x10, 17, 1),
+ PIN_FIELD_BASE(13, 28, 2, 0x080, 0x10, 6, 1),
+ PIN_FIELD_BASE(43, 49, 3, 0x080, 0x10, 8, 1),
+ PIN_FIELD_BASE(50, 60, 4, 0x080, 0x10, 0, 1),
+ PIN_FIELD_BASE(61, 88, 5, 0x080, 0x10, 0, 1),
+ PIN_FIELD_BASE(89, 89, 6, 0x080, 0x10, 24, 1),
+ PIN_FIELD_BASE(90, 90, 6, 0x080, 0x10, 1, 1),
+ PIN_FIELD_BASE(95, 95, 6, 0x080, 0x10, 15, 1),
+ PIN_FIELD_BASE(96, 102, 6, 0x080, 0x10, 17, 1),
+ PIN_FIELD_BASE(103, 106, 6, 0x080, 0x10, 28, 1),
+ PIN_FIELD_BASE(107, 121, 7, 0x080, 0x10, 0, 1),
+ PIN_FIELD_BASE(134, 143, 1, 0x080, 0x10, 0, 1),
+ PIN_FIELD_BASE(144, 149, 1, 0x080, 0x10, 11, 1),
+ PIN_FIELD_BASE(150, 160, 1, 0x080, 0x10, 18, 1),
+ PIN_FIELD_BASE(161, 166, 2, 0x080, 0x10, 0, 1),
+ PIN_FIELD_BASE(167, 176, 4, 0x080, 0x10, 11, 1),
+ PIN_FIELD_BASE(177, 177, 6, 0x080, 0x10, 10, 1),
+ PIN_FIELD_BASE(178, 178, 6, 0x080, 0x10, 16, 1),
+ PIN_FIELD_BASE(179, 179, 6, 0x080, 0x10, 25, 1),
+};
+
+static const struct mtk_pin_field_calc mt8183_pin_drv_range[] = {
+ PINS_FIELD_BASE(0, 3, 6, 0x0A0, 0x10, 12, 3),
+ PINS_FIELD_BASE(4, 7, 6, 0x0A0, 0x10, 20, 3),
+ PIN_FIELD_BASE(8, 8, 6, 0x0A0, 0x10, 0, 3),
+ PINS_FIELD_BASE(9, 10, 6, 0x0B0, 0x10, 16, 3),
+ PIN_FIELD_BASE(11, 11, 1, 0x0A0, 0x10, 12, 3),
+ PIN_FIELD_BASE(12, 12, 1, 0x0A0, 0x10, 28, 3),
+ PINS_FIELD_BASE(13, 16, 2, 0x0A0, 0x10, 8, 3),
+ PINS_FIELD_BASE(17, 20, 2, 0x0A0, 0x10, 12, 3),
+ PINS_FIELD_BASE(21, 24, 2, 0x0A0, 0x10, 16, 3),
+ PINS_FIELD_BASE(25, 28, 2, 0x0A0, 0x10, 20, 3),
+ PIN_FIELD_BASE(29, 29, 2, 0x0A0, 0x10, 24, 3),
+ PIN_FIELD_BASE(30, 30, 2, 0x0A0, 0x10, 28, 3),
+ PINS_FIELD_BASE(31, 31, 2, 0x0B0, 0x10, 0, 3),
+ PINS_FIELD_BASE(32, 34, 2, 0x0A0, 0x10, 28, 3),
+ PINS_FIELD_BASE(35, 37, 3, 0x0A0, 0x10, 0, 3),
+ PINS_FIELD_BASE(38, 40, 3, 0x0A0, 0x10, 4, 3),
+ PINS_FIELD_BASE(41, 42, 3, 0x0A0, 0x10, 8, 3),
+ PINS_FIELD_BASE(43, 45, 3, 0x0A0, 0x10, 12, 3),
+ PINS_FIELD_BASE(46, 47, 3, 0x0A0, 0x10, 16, 3),
+ PINS_FIELD_BASE(48, 49, 3, 0x0A0, 0x10, 20, 3),
+ PINS_FIELD_BASE(50, 51, 4, 0x0A0, 0x10, 0, 3),
+ PINS_FIELD_BASE(52, 57, 4, 0x0A0, 0x10, 4, 3),
+ PINS_FIELD_BASE(58, 60, 4, 0x0A0, 0x10, 8, 3),
+ PINS_FIELD_BASE(61, 64, 5, 0x0A0, 0x10, 0, 3),
+ PINS_FIELD_BASE(65, 66, 5, 0x0A0, 0x10, 4, 3),
+ PINS_FIELD_BASE(67, 68, 5, 0x0A0, 0x10, 8, 3),
+ PINS_FIELD_BASE(69, 71, 5, 0x0A0, 0x10, 12, 3),
+ PINS_FIELD_BASE(72, 76, 5, 0x0A0, 0x10, 16, 3),
+ PINS_FIELD_BASE(77, 80, 5, 0x0A0, 0x10, 20, 3),
+ PIN_FIELD_BASE(81, 81, 5, 0x0A0, 0x10, 24, 3),
+ PINS_FIELD_BASE(82, 83, 5, 0x0A0, 0x10, 28, 3),
+ PIN_FIELD_BASE(84, 84, 5, 0x0A0, 0x10, 24, 3),
+ PINS_FIELD_BASE(85, 88, 5, 0x0B0, 0x10, 0, 3),
+ PIN_FIELD_BASE(89, 89, 6, 0x0B0, 0x10, 12, 3),
+ PIN_FIELD_BASE(90, 90, 6, 0x0A0, 0x10, 4, 3),
+ PINS_FIELD_BASE(91, 94, 6, 0x0A0, 0x10, 8, 3),
+ PINS_FIELD_BASE(95, 96, 6, 0x0A0, 0x10, 24, 3),
+ PINS_FIELD_BASE(97, 98, 6, 0x0A0, 0x10, 28, 3),
+ PIN_FIELD_BASE(99, 99, 6, 0x0B0, 0x10, 0, 3),
+ PIN_FIELD_BASE(100, 100, 6, 0x0B0, 0x10, 4, 3),
+ PINS_FIELD_BASE(101, 102, 6, 0x0B0, 0x10, 8, 3),
+ PINS_FIELD_BASE(103, 104, 6, 0x0B0, 0x10, 20, 3),
+ PINS_FIELD_BASE(105, 106, 6, 0x0B0, 0x10, 24, 3),
+ PIN_FIELD_BASE(107, 107, 7, 0x0A0, 0x10, 0, 3),
+ PIN_FIELD_BASE(108, 108, 7, 0x0A0, 0x10, 4, 3),
+ PIN_FIELD_BASE(109, 109, 7, 0x0A0, 0x10, 8, 3),
+ PIN_FIELD_BASE(110, 110, 7, 0x0A0, 0x10, 0, 3),
+ PIN_FIELD_BASE(111, 111, 7, 0x0A0, 0x10, 4, 3),
+ PIN_FIELD_BASE(112, 112, 7, 0x0A0, 0x10, 8, 3),
+ PIN_FIELD_BASE(113, 113, 7, 0x0A0, 0x10, 16, 3),
+ PIN_FIELD_BASE(114, 114, 7, 0x0A0, 0x10, 20, 3),
+ PIN_FIELD_BASE(115, 115, 7, 0x0A0, 0x10, 24, 3),
+ PIN_FIELD_BASE(116, 116, 7, 0x0A0, 0x10, 28, 3),
+ PIN_FIELD_BASE(117, 117, 7, 0x0B0, 0x10, 0, 3),
+ PIN_FIELD_BASE(118, 118, 7, 0x0B0, 0x10, 4, 3),
+ PIN_FIELD_BASE(119, 119, 7, 0x0B0, 0x10, 8, 3),
+ PIN_FIELD_BASE(120, 120, 7, 0x0B0, 0x10, 12, 3),
+ PIN_FIELD_BASE(121, 121, 7, 0x0B0, 0x10, 16, 3),
+ PIN_FIELD_BASE(122, 122, 8, 0x0A0, 0x10, 0, 3),
+ PIN_FIELD_BASE(123, 123, 8, 0x0A0, 0x10, 4, 3),
+ PIN_FIELD_BASE(124, 124, 8, 0x0A0, 0x10, 8, 3),
+ PINS_FIELD_BASE(125, 130, 8, 0x0A0, 0x10, 4, 3),
+ PIN_FIELD_BASE(131, 131, 8, 0x0A0, 0x10, 12, 3),
+ PIN_FIELD_BASE(132, 132, 8, 0x0A0, 0x10, 4, 3),
+ PIN_FIELD_BASE(133, 133, 8, 0x0A0, 0x10, 16, 3),
+ PIN_FIELD_BASE(134, 134, 1, 0x0A0, 0x10, 0, 3),
+ PIN_FIELD_BASE(135, 135, 1, 0x0A0, 0x10, 4, 3),
+ PINS_FIELD_BASE(136, 143, 1, 0x0A0, 0x10, 8, 3),
+ PINS_FIELD_BASE(144, 147, 1, 0x0A0, 0x10, 16, 3),
+ PIN_FIELD_BASE(148, 148, 1, 0x0A0, 0x10, 20, 3),
+ PIN_FIELD_BASE(149, 149, 1, 0x0A0, 0x10, 24, 3),
+ PINS_FIELD_BASE(150, 153, 1, 0x0B0, 0x10, 0, 3),
+ PIN_FIELD_BASE(154, 154, 1, 0x0B0, 0x10, 4, 3),
+ PINS_FIELD_BASE(155, 157, 1, 0x0B0, 0x10, 8, 3),
+ PINS_FIELD_BASE(158, 160, 1, 0x0B0, 0x10, 0, 3),
+ PINS_FIELD_BASE(161, 164, 2, 0x0A0, 0x10, 0, 3),
+ PINS_FIELD_BASE(165, 166, 2, 0x0A0, 0x10, 4, 3),
+ PINS_FIELD_BASE(167, 168, 4, 0x0A0, 0x10, 8, 3),
+ PIN_FIELD_BASE(169, 169, 4, 0x0A0, 0x10, 12, 3),
+ PINS_FIELD_BASE(170, 174, 4, 0x0A0, 0x10, 16, 3),
+ PINS_FIELD_BASE(175, 176, 4, 0x0A0, 0x10, 12, 3),
+ PINS_FIELD_BASE(177, 179, 6, 0x0A0, 0x10, 16, 3),
+};
+
+static const struct mtk_pin_field_calc mt8183_pin_pupd_range[] = {
+ PIN_FIELD_BASE(29, 29, 2, 0x0C0, 0x10, 2, 1),
+ PIN_FIELD_BASE(30, 30, 2, 0x0C0, 0x10, 6, 1),
+ PIN_FIELD_BASE(31, 31, 2, 0x0C0, 0x10, 10, 1),
+ PIN_FIELD_BASE(32, 32, 2, 0x0C0, 0x10, 14, 1),
+ PIN_FIELD_BASE(33, 33, 2, 0x0C0, 0x10, 18, 1),
+ PIN_FIELD_BASE(34, 34, 2, 0x0C0, 0x10, 22, 1),
+ PIN_FIELD_BASE(35, 35, 3, 0x0C0, 0x10, 2, 1),
+ PIN_FIELD_BASE(36, 36, 3, 0x0C0, 0x10, 6, 1),
+ PIN_FIELD_BASE(37, 37, 3, 0x0C0, 0x10, 10, 1),
+ PIN_FIELD_BASE(38, 38, 3, 0x0C0, 0x10, 14, 1),
+ PIN_FIELD_BASE(39, 39, 3, 0x0C0, 0x10, 18, 1),
+ PIN_FIELD_BASE(40, 40, 3, 0x0C0, 0x10, 22, 1),
+ PIN_FIELD_BASE(41, 41, 3, 0x0C0, 0x10, 26, 1),
+ PIN_FIELD_BASE(42, 42, 3, 0x0C0, 0x10, 30, 1),
+ PIN_FIELD_BASE(91, 91, 6, 0x0C0, 0x10, 2, 1),
+ PIN_FIELD_BASE(92, 92, 6, 0x0C0, 0x10, 6, 1),
+ PIN_FIELD_BASE(93, 93, 6, 0x0C0, 0x10, 10, 1),
+ PIN_FIELD_BASE(94, 94, 6, 0x0C0, 0x10, 14, 1),
+ PIN_FIELD_BASE(122, 122, 8, 0x0C0, 0x10, 2, 1),
+ PIN_FIELD_BASE(123, 123, 8, 0x0C0, 0x10, 6, 1),
+ PIN_FIELD_BASE(124, 124, 8, 0x0C0, 0x10, 10, 1),
+ PIN_FIELD_BASE(125, 125, 8, 0x0C0, 0x10, 14, 1),
+ PIN_FIELD_BASE(126, 126, 8, 0x0C0, 0x10, 18, 1),
+ PIN_FIELD_BASE(127, 127, 8, 0x0C0, 0x10, 22, 1),
+ PIN_FIELD_BASE(128, 128, 8, 0x0C0, 0x10, 26, 1),
+ PIN_FIELD_BASE(129, 129, 8, 0x0C0, 0x10, 30, 1),
+ PIN_FIELD_BASE(130, 130, 8, 0x0D0, 0x10, 2, 1),
+ PIN_FIELD_BASE(131, 131, 8, 0x0D0, 0x10, 6, 1),
+ PIN_FIELD_BASE(132, 132, 8, 0x0D0, 0x10, 10, 1),
+ PIN_FIELD_BASE(133, 133, 8, 0x0D0, 0x10, 14, 1),
+};
+
+static const struct mtk_pin_field_calc mt8183_pin_r0_range[] = {
+ PIN_FIELD_BASE(29, 29, 2, 0x0C0, 0x10, 0, 1),
+ PIN_FIELD_BASE(30, 30, 2, 0x0C0, 0x10, 4, 1),
+ PIN_FIELD_BASE(31, 31, 2, 0x0C0, 0x10, 8, 1),
+ PIN_FIELD_BASE(32, 32, 2, 0x0C0, 0x10, 12, 1),
+ PIN_FIELD_BASE(33, 33, 2, 0x0C0, 0x10, 16, 1),
+ PIN_FIELD_BASE(34, 34, 2, 0x0C0, 0x10, 20, 1),
+ PIN_FIELD_BASE(35, 35, 3, 0x0C0, 0x10, 0, 1),
+ PIN_FIELD_BASE(36, 36, 3, 0x0C0, 0x10, 4, 1),
+ PIN_FIELD_BASE(37, 37, 3, 0x0C0, 0x10, 8, 1),
+ PIN_FIELD_BASE(38, 38, 3, 0x0C0, 0x10, 12, 1),
+ PIN_FIELD_BASE(39, 39, 3, 0x0C0, 0x10, 16, 1),
+ PIN_FIELD_BASE(40, 40, 3, 0x0C0, 0x10, 20, 1),
+ PIN_FIELD_BASE(41, 41, 3, 0x0C0, 0x10, 24, 1),
+ PIN_FIELD_BASE(42, 42, 3, 0x0C0, 0x10, 28, 1),
+ PIN_FIELD_BASE(48, 48, 3, 0x0F0, 0x10, 18, 1),
+ PIN_FIELD_BASE(49, 49, 3, 0x0F0, 0x10, 13, 1),
+ PIN_FIELD_BASE(50, 50, 4, 0x0F0, 0x10, 10, 1),
+ PIN_FIELD_BASE(51, 51, 4, 0x0F0, 0x10, 5, 1),
+ PIN_FIELD_BASE(81, 81, 5, 0x0F0, 0x10, 7, 1),
+ PIN_FIELD_BASE(82, 82, 5, 0x0F0, 0x10, 5, 1),
+ PIN_FIELD_BASE(83, 83, 5, 0x0F0, 0x10, 15, 1),
+ PIN_FIELD_BASE(84, 84, 5, 0x0F0, 0x10, 17, 1),
+ PIN_FIELD_BASE(91, 91, 6, 0x0C0, 0x10, 0, 1),
+ PIN_FIELD_BASE(92, 92, 6, 0x0C0, 0x10, 4, 1),
+ PIN_FIELD_BASE(93, 93, 6, 0x0C0, 0x10, 8, 1),
+ PIN_FIELD_BASE(94, 94, 6, 0x0C0, 0x10, 12, 1),
+ PIN_FIELD_BASE(103, 103, 6, 0x0F0, 0x10, 20, 1),
+ PIN_FIELD_BASE(104, 104, 6, 0x0F0, 0x10, 10, 1),
+ PIN_FIELD_BASE(105, 105, 6, 0x0F0, 0x10, 22, 1),
+ PIN_FIELD_BASE(106, 106, 6, 0x0F0, 0x10, 12, 1),
+ PIN_FIELD_BASE(122, 122, 8, 0x0C0, 0x10, 0, 1),
+ PIN_FIELD_BASE(123, 123, 8, 0x0C0, 0x10, 4, 1),
+ PIN_FIELD_BASE(124, 124, 8, 0x0C0, 0x10, 8, 1),
+ PIN_FIELD_BASE(125, 125, 8, 0x0C0, 0x10, 12, 1),
+ PIN_FIELD_BASE(126, 126, 8, 0x0C0, 0x10, 16, 1),
+ PIN_FIELD_BASE(127, 127, 8, 0x0C0, 0x10, 20, 1),
+ PIN_FIELD_BASE(128, 128, 8, 0x0C0, 0x10, 24, 1),
+ PIN_FIELD_BASE(129, 129, 8, 0x0C0, 0x10, 28, 1),
+ PIN_FIELD_BASE(130, 130, 8, 0x0D0, 0x10, 0, 1),
+ PIN_FIELD_BASE(131, 131, 8, 0x0D0, 0x10, 4, 1),
+ PIN_FIELD_BASE(132, 132, 8, 0x0D0, 0x10, 8, 1),
+ PIN_FIELD_BASE(133, 133, 8, 0x0D0, 0x10, 12, 1),
+};
+
+static const struct mtk_pin_field_calc mt8183_pin_r1_range[] = {
+ PIN_FIELD_BASE(29, 29, 2, 0x0C0, 0x10, 1, 1),
+ PIN_FIELD_BASE(30, 30, 2, 0x0C0, 0x10, 5, 1),
+ PIN_FIELD_BASE(31, 31, 2, 0x0C0, 0x10, 9, 1),
+ PIN_FIELD_BASE(32, 32, 2, 0x0C0, 0x10, 13, 1),
+ PIN_FIELD_BASE(33, 33, 2, 0x0C0, 0x10, 17, 1),
+ PIN_FIELD_BASE(34, 34, 2, 0x0C0, 0x10, 21, 1),
+ PIN_FIELD_BASE(35, 35, 3, 0x0C0, 0x10, 1, 1),
+ PIN_FIELD_BASE(36, 36, 3, 0x0C0, 0x10, 5, 1),
+ PIN_FIELD_BASE(37, 37, 3, 0x0C0, 0x10, 9, 1),
+ PIN_FIELD_BASE(38, 38, 3, 0x0C0, 0x10, 13, 1),
+ PIN_FIELD_BASE(39, 39, 3, 0x0C0, 0x10, 17, 1),
+ PIN_FIELD_BASE(40, 40, 3, 0x0C0, 0x10, 21, 1),
+ PIN_FIELD_BASE(41, 41, 3, 0x0C0, 0x10, 25, 1),
+ PIN_FIELD_BASE(42, 42, 3, 0x0C0, 0x10, 29, 1),
+ PIN_FIELD_BASE(48, 48, 3, 0x0F0, 0x10, 19, 1),
+ PIN_FIELD_BASE(49, 49, 3, 0x0F0, 0x10, 14, 1),
+ PIN_FIELD_BASE(50, 50, 4, 0x0F0, 0x10, 11, 1),
+ PIN_FIELD_BASE(51, 51, 4, 0x0F0, 0x10, 6, 1),
+ PIN_FIELD_BASE(81, 81, 5, 0x0F0, 0x10, 8, 1),
+ PIN_FIELD_BASE(82, 82, 5, 0x0F0, 0x10, 6, 1),
+ PIN_FIELD_BASE(83, 83, 5, 0x0F0, 0x10, 16, 1),
+ PIN_FIELD_BASE(84, 84, 5, 0x0F0, 0x10, 18, 1),
+ PIN_FIELD_BASE(91, 91, 6, 0x0C0, 0x10, 1, 1),
+ PIN_FIELD_BASE(92, 92, 6, 0x0C0, 0x10, 5, 1),
+ PIN_FIELD_BASE(93, 93, 6, 0x0C0, 0x10, 9, 1),
+ PIN_FIELD_BASE(94, 94, 6, 0x0C0, 0x10, 13, 1),
+ PIN_FIELD_BASE(103, 103, 6, 0x0F0, 0x10, 21, 1),
+ PIN_FIELD_BASE(104, 104, 6, 0x0F0, 0x10, 11, 1),
+ PIN_FIELD_BASE(105, 105, 6, 0x0F0, 0x10, 23, 1),
+ PIN_FIELD_BASE(106, 106, 6, 0x0F0, 0x10, 13, 1),
+ PIN_FIELD_BASE(122, 122, 8, 0x0C0, 0x10, 1, 1),
+ PIN_FIELD_BASE(123, 123, 8, 0x0C0, 0x10, 5, 1),
+ PIN_FIELD_BASE(124, 124, 8, 0x0C0, 0x10, 9, 1),
+ PIN_FIELD_BASE(125, 125, 8, 0x0C0, 0x10, 13, 1),
+ PIN_FIELD_BASE(126, 126, 8, 0x0C0, 0x10, 17, 1),
+ PIN_FIELD_BASE(127, 127, 8, 0x0C0, 0x10, 21, 1),
+ PIN_FIELD_BASE(128, 128, 8, 0x0C0, 0x10, 25, 1),
+ PIN_FIELD_BASE(129, 129, 8, 0x0C0, 0x10, 29, 1),
+ PIN_FIELD_BASE(130, 130, 8, 0x0D0, 0x10, 1, 1),
+ PIN_FIELD_BASE(131, 131, 8, 0x0D0, 0x10, 5, 1),
+ PIN_FIELD_BASE(132, 132, 8, 0x0D0, 0x10, 9, 1),
+ PIN_FIELD_BASE(133, 133, 8, 0x0D0, 0x10, 13, 1),
+};
+
+static const struct mtk_pin_reg_calc mt8183_reg_cals[PINCTRL_PIN_REG_MAX] = {
+ [PINCTRL_PIN_REG_MODE] = MTK_RANGE(mt8183_pin_mode_range),
+ [PINCTRL_PIN_REG_DIR] = MTK_RANGE(mt8183_pin_dir_range),
+ [PINCTRL_PIN_REG_DI] = MTK_RANGE(mt8183_pin_di_range),
+ [PINCTRL_PIN_REG_DO] = MTK_RANGE(mt8183_pin_do_range),
+ [PINCTRL_PIN_REG_SMT] = MTK_RANGE(mt8183_pin_smt_range),
+ [PINCTRL_PIN_REG_IES] = MTK_RANGE(mt8183_pin_ies_range),
+ [PINCTRL_PIN_REG_PULLEN] = MTK_RANGE(mt8183_pin_pullen_range),
+ [PINCTRL_PIN_REG_PULLSEL] = MTK_RANGE(mt8183_pin_pullsel_range),
+ [PINCTRL_PIN_REG_DRV] = MTK_RANGE(mt8183_pin_drv_range),
+ [PINCTRL_PIN_REG_PUPD] = MTK_RANGE(mt8183_pin_pupd_range),
+ [PINCTRL_PIN_REG_R0] = MTK_RANGE(mt8183_pin_r0_range),
+ [PINCTRL_PIN_REG_R1] = MTK_RANGE(mt8183_pin_r1_range),
+};
+
+static const char * const mt8183_pinctrl_register_base_names[] = {
+ "iocfg0", "iocfg1", "iocfg2", "iocfg3", "iocfg4", "iocfg5",
+ "iocfg6", "iocfg7", "iocfg8",
+};
+
+static const struct mtk_eint_hw mt8183_eint_hw = {
+ .port_mask = 7,
+ .ports = 6,
+ .ap_num = 212,
+ .db_cnt = 13,
+};
+
+static const struct mtk_pin_soc mt8183_data = {
+ .reg_cal = mt8183_reg_cals,
+ .pins = mtk_pins_mt8183,
+ .npins = ARRAY_SIZE(mtk_pins_mt8183),
+ .ngrps = ARRAY_SIZE(mtk_pins_mt8183),
+ .eint_hw = &mt8183_eint_hw,
+ .gpio_m = 0,
+ .ies_present = true,
+ .base_names = mt8183_pinctrl_register_base_names,
+ .nbase_names = ARRAY_SIZE(mt8183_pinctrl_register_base_names),
+ .bias_disable_set = mtk_pinconf_bias_disable_set_rev1,
+ .bias_disable_get = mtk_pinconf_bias_disable_get_rev1,
+ .bias_set = mtk_pinconf_bias_set_rev1,
+ .bias_get = mtk_pinconf_bias_get_rev1,
+ .drive_set = mtk_pinconf_drive_set_rev1,
+ .drive_get = mtk_pinconf_drive_get_rev1,
+ .adv_pull_get = mtk_pinconf_adv_pull_get,
+ .adv_pull_set = mtk_pinconf_adv_pull_set,
+};
+
+static const struct of_device_id mt8183_pinctrl_of_match[] = {
+ { .compatible = "mediatek,mt8183-pinctrl", },
+ { }
+};
+
+static int mt8183_pinctrl_probe(struct platform_device *pdev)
+{
+ return mtk_paris_pinctrl_probe(pdev, &mt8183_data);
+}
+
+static struct platform_driver mt8183_pinctrl_driver = {
+ .driver = {
+ .name = "mt8183-pinctrl",
+ .of_match_table = mt8183_pinctrl_of_match,
+ },
+ .probe = mt8183_pinctrl_probe,
+};
+
+static int __init mt8183_pinctrl_init(void)
+{
+ return platform_driver_register(&mt8183_pinctrl_driver);
+}
+arch_initcall(mt8183_pinctrl_init);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 MediaTek Inc.
+ *
+ * Author: Sean Wang <sean.wang@mediatek.com>
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/err.h>
+#include <linux/gpio/driver.h>
+#include <linux/platform_device.h>
+#include <linux/io.h>
+#include <linux/of_irq.h>
+
+#include "mtk-eint.h"
+#include "pinctrl-mtk-common-v2.h"
+
+/**
+ * struct mtk_drive_desc - the structure that holds the information
+ * of the driving current
+ * @min: the minimum current of this group
+ * @max: the maximum current of this group
+ * @step: the step current of this group
+ * @scal: the weight factor
+ *
+ * formula: output = ((input) / step - 1) * scal
+ */
+struct mtk_drive_desc {
+ u8 min;
+ u8 max;
+ u8 step;
+ u8 scal;
+};
+
+/* The groups of drive strength */
+static const struct mtk_drive_desc mtk_drive[] = {
+ [DRV_GRP0] = { 4, 16, 4, 1 },
+ [DRV_GRP1] = { 4, 16, 4, 2 },
+ [DRV_GRP2] = { 2, 8, 2, 1 },
+ [DRV_GRP3] = { 2, 8, 2, 2 },
+ [DRV_GRP4] = { 2, 16, 2, 1 },
+};
+
+static void mtk_w32(struct mtk_pinctrl *pctl, u8 i, u32 reg, u32 val)
+{
+ writel_relaxed(val, pctl->base[i] + reg);
+}
+
+static u32 mtk_r32(struct mtk_pinctrl *pctl, u8 i, u32 reg)
+{
+ return readl_relaxed(pctl->base[i] + reg);
+}
+
+void mtk_rmw(struct mtk_pinctrl *pctl, u8 i, u32 reg, u32 mask, u32 set)
+{
+ u32 val;
+
+ val = mtk_r32(pctl, i, reg);
+ val &= ~mask;
+ val |= set;
+ mtk_w32(pctl, i, reg, val);
+}
+
+static int mtk_hw_pin_field_lookup(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc,
+ int field, struct mtk_pin_field *pfd)
+{
+ const struct mtk_pin_field_calc *c, *e;
+ const struct mtk_pin_reg_calc *rc;
+ u32 bits;
+
+ if (hw->soc->reg_cal && hw->soc->reg_cal[field].range) {
+ rc = &hw->soc->reg_cal[field];
+ } else {
+ dev_dbg(hw->dev,
+ "Not support field %d for pin %d (%s)\n",
+ field, desc->number, desc->name);
+ return -ENOTSUPP;
+ }
+
+ c = rc->range;
+ e = c + rc->nranges;
+
+ while (c < e) {
+ if (desc->number >= c->s_pin && desc->number <= c->e_pin)
+ break;
+ c++;
+ }
+
+ if (c >= e) {
+ dev_dbg(hw->dev, "Not support field %d for pin = %d (%s)\n",
+ field, desc->number, desc->name);
+ return -ENOTSUPP;
+ }
+
+ if (c->i_base > hw->nbase - 1) {
+ dev_err(hw->dev,
+ "Invalid base for field %d for pin = %d (%s)\n",
+ field, desc->number, desc->name);
+ return -EINVAL;
+ }
+
+ /* Calculated bits as the overall offset the pin is located at,
+ * if c->fixed is held, that determines the all the pins in the
+ * range use the same field with the s_pin.
+ */
+ bits = c->fixed ? c->s_bit : c->s_bit +
+ (desc->number - c->s_pin) * (c->x_bits);
+
+ /* Fill pfd from bits. For example 32-bit register applied is assumed
+ * when c->sz_reg is equal to 32.
+ */
+ pfd->index = c->i_base;
+ pfd->offset = c->s_addr + c->x_addrs * (bits / c->sz_reg);
+ pfd->bitpos = bits % c->sz_reg;
+ pfd->mask = (1 << c->x_bits) - 1;
+
+ /* pfd->next is used for indicating that bit wrapping-around happens
+ * which requires the manipulation for bit 0 starting in the next
+ * register to form the complete field read/write.
+ */
+ pfd->next = pfd->bitpos + c->x_bits > c->sz_reg ? c->x_addrs : 0;
+
+ return 0;
+}
+
+static int mtk_hw_pin_field_get(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc,
+ int field, struct mtk_pin_field *pfd)
+{
+ if (field < 0 || field >= PINCTRL_PIN_REG_MAX) {
+ dev_err(hw->dev, "Invalid Field %d\n", field);
+ return -EINVAL;
+ }
+
+ return mtk_hw_pin_field_lookup(hw, desc, field, pfd);
+}
+
+static void mtk_hw_bits_part(struct mtk_pin_field *pf, int *h, int *l)
+{
+ *l = 32 - pf->bitpos;
+ *h = get_count_order(pf->mask) - *l;
+}
+
+static void mtk_hw_write_cross_field(struct mtk_pinctrl *hw,
+ struct mtk_pin_field *pf, int value)
+{
+ int nbits_l, nbits_h;
+
+ mtk_hw_bits_part(pf, &nbits_h, &nbits_l);
+
+ mtk_rmw(hw, pf->index, pf->offset, pf->mask << pf->bitpos,
+ (value & pf->mask) << pf->bitpos);
+
+ mtk_rmw(hw, pf->index, pf->offset + pf->next, BIT(nbits_h) - 1,
+ (value & pf->mask) >> nbits_l);
+}
+
+static void mtk_hw_read_cross_field(struct mtk_pinctrl *hw,
+ struct mtk_pin_field *pf, int *value)
+{
+ int nbits_l, nbits_h, h, l;
+
+ mtk_hw_bits_part(pf, &nbits_h, &nbits_l);
+
+ l = (mtk_r32(hw, pf->index, pf->offset)
+ >> pf->bitpos) & (BIT(nbits_l) - 1);
+ h = (mtk_r32(hw, pf->index, pf->offset + pf->next))
+ & (BIT(nbits_h) - 1);
+
+ *value = (h << nbits_l) | l;
+}
+
+int mtk_hw_set_value(struct mtk_pinctrl *hw, const struct mtk_pin_desc *desc,
+ int field, int value)
+{
+ struct mtk_pin_field pf;
+ int err;
+
+ err = mtk_hw_pin_field_get(hw, desc, field, &pf);
+ if (err)
+ return err;
+
+ if (!pf.next)
+ mtk_rmw(hw, pf.index, pf.offset, pf.mask << pf.bitpos,
+ (value & pf.mask) << pf.bitpos);
+ else
+ mtk_hw_write_cross_field(hw, &pf, value);
+
+ return 0;
+}
+
+int mtk_hw_get_value(struct mtk_pinctrl *hw, const struct mtk_pin_desc *desc,
+ int field, int *value)
+{
+ struct mtk_pin_field pf;
+ int err;
+
+ err = mtk_hw_pin_field_get(hw, desc, field, &pf);
+ if (err)
+ return err;
+
+ if (!pf.next)
+ *value = (mtk_r32(hw, pf.index, pf.offset)
+ >> pf.bitpos) & pf.mask;
+ else
+ mtk_hw_read_cross_field(hw, &pf, value);
+
+ return 0;
+}
+
+static int mtk_xt_find_eint_num(struct mtk_pinctrl *hw, unsigned long eint_n)
+{
+ const struct mtk_pin_desc *desc;
+ int i = 0;
+
+ desc = (const struct mtk_pin_desc *)hw->soc->pins;
+
+ while (i < hw->soc->npins) {
+ if (desc[i].eint.eint_n == eint_n)
+ return desc[i].number;
+ i++;
+ }
+
+ return EINT_NA;
+}
+
+static int mtk_xt_get_gpio_n(void *data, unsigned long eint_n,
+ unsigned int *gpio_n,
+ struct gpio_chip **gpio_chip)
+{
+ struct mtk_pinctrl *hw = (struct mtk_pinctrl *)data;
+ const struct mtk_pin_desc *desc;
+
+ desc = (const struct mtk_pin_desc *)hw->soc->pins;
+ *gpio_chip = &hw->chip;
+
+ /* Be greedy to guess first gpio_n is equal to eint_n */
+ if (desc[eint_n].eint.eint_n == eint_n)
+ *gpio_n = eint_n;
+ else
+ *gpio_n = mtk_xt_find_eint_num(hw, eint_n);
+
+ return *gpio_n == EINT_NA ? -EINVAL : 0;
+}
+
+static int mtk_xt_get_gpio_state(void *data, unsigned long eint_n)
+{
+ struct mtk_pinctrl *hw = (struct mtk_pinctrl *)data;
+ const struct mtk_pin_desc *desc;
+ struct gpio_chip *gpio_chip;
+ unsigned int gpio_n;
+ int value, err;
+
+ err = mtk_xt_get_gpio_n(hw, eint_n, &gpio_n, &gpio_chip);
+ if (err)
+ return err;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[gpio_n];
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_DI, &value);
+ if (err)
+ return err;
+
+ return !!value;
+}
+
+static int mtk_xt_set_gpio_as_eint(void *data, unsigned long eint_n)
+{
+ struct mtk_pinctrl *hw = (struct mtk_pinctrl *)data;
+ const struct mtk_pin_desc *desc;
+ struct gpio_chip *gpio_chip;
+ unsigned int gpio_n;
+ int err;
+
+ err = mtk_xt_get_gpio_n(hw, eint_n, &gpio_n, &gpio_chip);
+ if (err)
+ return err;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[gpio_n];
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_MODE,
+ desc->eint.eint_m);
+ if (err)
+ return err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DIR, MTK_INPUT);
+ if (err)
+ return err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_SMT, MTK_ENABLE);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static const struct mtk_eint_xt mtk_eint_xt = {
+ .get_gpio_n = mtk_xt_get_gpio_n,
+ .get_gpio_state = mtk_xt_get_gpio_state,
+ .set_gpio_as_eint = mtk_xt_set_gpio_as_eint,
+};
+
+int mtk_build_eint(struct mtk_pinctrl *hw, struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct resource *res;
+
+ if (!IS_ENABLED(CONFIG_EINT_MTK))
+ return 0;
+
+ if (!of_property_read_bool(np, "interrupt-controller"))
+ return -ENODEV;
+
+ hw->eint = devm_kzalloc(hw->dev, sizeof(*hw->eint), GFP_KERNEL);
+ if (!hw->eint)
+ return -ENOMEM;
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "eint");
+ if (!res) {
+ dev_err(&pdev->dev, "Unable to get eint resource\n");
+ return -ENODEV;
+ }
+
+ hw->eint->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(hw->eint->base))
+ return PTR_ERR(hw->eint->base);
+
+ hw->eint->irq = irq_of_parse_and_map(np, 0);
+ if (!hw->eint->irq)
+ return -EINVAL;
+
+ if (!hw->soc->eint_hw)
+ return -ENODEV;
+
+ hw->eint->dev = &pdev->dev;
+ hw->eint->hw = hw->soc->eint_hw;
+ hw->eint->pctl = hw;
+ hw->eint->gpio_xlate = &mtk_eint_xt;
+
+ return mtk_eint_do_init(hw->eint);
+}
+
+/* Revision 0 */
+int mtk_pinconf_bias_disable_set(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc)
+{
+ int err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_PU,
+ MTK_DISABLE);
+ if (err)
+ return err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_PD,
+ MTK_DISABLE);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+int mtk_pinconf_bias_disable_get(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, int *res)
+{
+ int v, v2;
+ int err;
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_PU, &v);
+ if (err)
+ return err;
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_PD, &v2);
+ if (err)
+ return err;
+
+ if (v == MTK_ENABLE || v2 == MTK_ENABLE)
+ return -EINVAL;
+
+ *res = 1;
+
+ return 0;
+}
+
+int mtk_pinconf_bias_set(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup)
+{
+ int err, arg;
+
+ arg = pullup ? 1 : 2;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_PU, arg & 1);
+ if (err)
+ return err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_PD,
+ !!(arg & 2));
+ if (err)
+ return err;
+
+ return 0;
+}
+
+int mtk_pinconf_bias_get(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup, int *res)
+{
+ int reg, err, v;
+
+ reg = pullup ? PINCTRL_PIN_REG_PU : PINCTRL_PIN_REG_PD;
+
+ err = mtk_hw_get_value(hw, desc, reg, &v);
+ if (err)
+ return err;
+
+ if (!v)
+ return -EINVAL;
+
+ *res = 1;
+
+ return 0;
+}
+
+/* Revision 1 */
+int mtk_pinconf_bias_disable_set_rev1(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc)
+{
+ int err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_PULLEN,
+ MTK_DISABLE);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+int mtk_pinconf_bias_disable_get_rev1(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, int *res)
+{
+ int v, err;
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_PULLEN, &v);
+ if (err)
+ return err;
+
+ if (v == MTK_ENABLE)
+ return -EINVAL;
+
+ *res = 1;
+
+ return 0;
+}
+
+int mtk_pinconf_bias_set_rev1(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup)
+{
+ int err, arg;
+
+ arg = pullup ? MTK_PULLUP : MTK_PULLDOWN;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_PULLEN,
+ MTK_ENABLE);
+ if (err)
+ return err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_PULLSEL, arg);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+int mtk_pinconf_bias_get_rev1(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup,
+ int *res)
+{
+ int err, v;
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_PULLEN, &v);
+ if (err)
+ return err;
+
+ if (v == MTK_DISABLE)
+ return -EINVAL;
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_PULLSEL, &v);
+ if (err)
+ return err;
+
+ if (pullup ^ (v == MTK_PULLUP))
+ return -EINVAL;
+
+ *res = 1;
+
+ return 0;
+}
+
+/* Revision 0 */
+int mtk_pinconf_drive_set(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, u32 arg)
+{
+ const struct mtk_drive_desc *tb;
+ int err = -ENOTSUPP;
+
+ tb = &mtk_drive[desc->drv_n];
+ /* 4mA when (e8, e4) = (0, 0)
+ * 8mA when (e8, e4) = (0, 1)
+ * 12mA when (e8, e4) = (1, 0)
+ * 16mA when (e8, e4) = (1, 1)
+ */
+ if ((arg >= tb->min && arg <= tb->max) && !(arg % tb->step)) {
+ arg = (arg / tb->step - 1) * tb->scal;
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_E4,
+ arg & 0x1);
+ if (err)
+ return err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_E8,
+ (arg & 0x2) >> 1);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+int mtk_pinconf_drive_get(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, int *val)
+{
+ const struct mtk_drive_desc *tb;
+ int err, val1, val2;
+
+ tb = &mtk_drive[desc->drv_n];
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_E4, &val1);
+ if (err)
+ return err;
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_E8, &val2);
+ if (err)
+ return err;
+
+ /* 4mA when (e8, e4) = (0, 0); 8mA when (e8, e4) = (0, 1)
+ * 12mA when (e8, e4) = (1, 0); 16mA when (e8, e4) = (1, 1)
+ */
+ *val = (((val2 << 1) + val1) / tb->scal + 1) * tb->step;
+
+ return 0;
+}
+
+/* Revision 1 */
+int mtk_pinconf_drive_set_rev1(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, u32 arg)
+{
+ const struct mtk_drive_desc *tb;
+ int err = -ENOTSUPP;
+
+ tb = &mtk_drive[desc->drv_n];
+
+ if ((arg >= tb->min && arg <= tb->max) && !(arg % tb->step)) {
+ arg = (arg / tb->step - 1) * tb->scal;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DRV,
+ arg);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+int mtk_pinconf_drive_get_rev1(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, int *val)
+{
+ const struct mtk_drive_desc *tb;
+ int err, val1;
+
+ tb = &mtk_drive[desc->drv_n];
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_DRV, &val1);
+ if (err)
+ return err;
+
+ *val = ((val1 & 0x7) / tb->scal + 1) * tb->step;
+
+ return 0;
+}
+
+int mtk_pinconf_adv_pull_set(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup,
+ u32 arg)
+{
+ int err;
+
+ /* 10K off & 50K (75K) off, when (R0, R1) = (0, 0);
+ * 10K off & 50K (75K) on, when (R0, R1) = (0, 1);
+ * 10K on & 50K (75K) off, when (R0, R1) = (1, 0);
+ * 10K on & 50K (75K) on, when (R0, R1) = (1, 1)
+ */
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_R0, arg & 1);
+ if (err)
+ return 0;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_R1,
+ !!(arg & 2));
+ if (err)
+ return 0;
+
+ arg = pullup ? 0 : 1;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_PUPD, arg);
+
+ /* If PUPD register is not supported for that pin, let's fallback to
+ * general bias control.
+ */
+ if (err == -ENOTSUPP) {
+ if (hw->soc->bias_set) {
+ err = hw->soc->bias_set(hw, desc, pullup);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ }
+
+ return err;
+}
+
+int mtk_pinconf_adv_pull_get(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup,
+ u32 *val)
+{
+ u32 t, t2;
+ int err;
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_PUPD, &t);
+
+ /* If PUPD register is not supported for that pin, let's fallback to
+ * general bias control.
+ */
+ if (err == -ENOTSUPP) {
+ if (hw->soc->bias_get) {
+ err = hw->soc->bias_get(hw, desc, pullup, val);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ } else {
+ /* t == 0 supposes PULLUP for the customized PULL setup */
+ if (err)
+ return err;
+
+ if (pullup ^ !t)
+ return -EINVAL;
+ }
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_R0, &t);
+ if (err)
+ return err;
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_R1, &t2);
+ if (err)
+ return err;
+
+ *val = (t | t2 << 1) & 0x7;
+
+ return 0;
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 MediaTek Inc.
+ *
+ * Author: Sean Wang <sean.wang@mediatek.com>
+ *
+ */
+
+#ifndef __PINCTRL_MTK_COMMON_V2_H
+#define __PINCTRL_MTK_COMMON_V2_H
+
+#include <linux/gpio/driver.h>
+
+#define MTK_INPUT 0
+#define MTK_OUTPUT 1
+#define MTK_DISABLE 0
+#define MTK_ENABLE 1
+#define MTK_PULLDOWN 0
+#define MTK_PULLUP 1
+
+#define EINT_NA U16_MAX
+#define NO_EINT_SUPPORT EINT_NA
+
+#define PIN_FIELD_CALC(_s_pin, _e_pin, _i_base, _s_addr, _x_addrs, \
+ _s_bit, _x_bits, _sz_reg, _fixed) { \
+ .s_pin = _s_pin, \
+ .e_pin = _e_pin, \
+ .i_base = _i_base, \
+ .s_addr = _s_addr, \
+ .x_addrs = _x_addrs, \
+ .s_bit = _s_bit, \
+ .x_bits = _x_bits, \
+ .sz_reg = _sz_reg, \
+ .fixed = _fixed, \
+ }
+
+#define PIN_FIELD(_s_pin, _e_pin, _s_addr, _x_addrs, _s_bit, _x_bits) \
+ PIN_FIELD_CALC(_s_pin, _e_pin, 0, _s_addr, _x_addrs, _s_bit, \
+ _x_bits, 32, 0)
+
+#define PINS_FIELD(_s_pin, _e_pin, _s_addr, _x_addrs, _s_bit, _x_bits) \
+ PIN_FIELD_CALC(_s_pin, _e_pin, 0, _s_addr, _x_addrs, _s_bit, \
+ _x_bits, 32, 1)
+
+/* List these attributes which could be modified for the pin */
+enum {
+ PINCTRL_PIN_REG_MODE,
+ PINCTRL_PIN_REG_DIR,
+ PINCTRL_PIN_REG_DI,
+ PINCTRL_PIN_REG_DO,
+ PINCTRL_PIN_REG_SR,
+ PINCTRL_PIN_REG_SMT,
+ PINCTRL_PIN_REG_PD,
+ PINCTRL_PIN_REG_PU,
+ PINCTRL_PIN_REG_E4,
+ PINCTRL_PIN_REG_E8,
+ PINCTRL_PIN_REG_TDSEL,
+ PINCTRL_PIN_REG_RDSEL,
+ PINCTRL_PIN_REG_DRV,
+ PINCTRL_PIN_REG_PUPD,
+ PINCTRL_PIN_REG_R0,
+ PINCTRL_PIN_REG_R1,
+ PINCTRL_PIN_REG_IES,
+ PINCTRL_PIN_REG_PULLEN,
+ PINCTRL_PIN_REG_PULLSEL,
+ PINCTRL_PIN_REG_MAX,
+};
+
+/* Group the pins by the driving current */
+enum {
+ DRV_FIXED,
+ DRV_GRP0,
+ DRV_GRP1,
+ DRV_GRP2,
+ DRV_GRP3,
+ DRV_GRP4,
+ DRV_GRP_MAX,
+};
+
+static const char * const mtk_default_register_base_names[] = {
+ "base",
+};
+
+/* struct mtk_pin_field - the structure that holds the information of the field
+ * used to describe the attribute for the pin
+ * @base: the index pointing to the entry in base address list
+ * @offset: the register offset relative to the base address
+ * @mask: the mask used to filter out the field from the register
+ * @bitpos: the start bit relative to the register
+ * @next: the indication that the field would be extended to the
+ next register
+ */
+struct mtk_pin_field {
+ u8 index;
+ u32 offset;
+ u32 mask;
+ u8 bitpos;
+ u8 next;
+};
+
+/* struct mtk_pin_field_calc - the structure that holds the range providing
+ * the guide used to look up the relevant field
+ * @s_pin: the start pin within the range
+ * @e_pin: the end pin within the range
+ * @i_base: the index pointing to the entry in base address list
+ * @s_addr: the start address for the range
+ * @x_addrs: the address distance between two consecutive registers
+ * within the range
+ * @s_bit: the start bit for the first register within the range
+ * @x_bits: the bit distance between two consecutive pins within
+ * the range
+ * @sz_reg: the size of bits in a register
+ * @fixed: the consecutive pins share the same bits with the 1st
+ * pin
+ */
+struct mtk_pin_field_calc {
+ u16 s_pin;
+ u16 e_pin;
+ u8 i_base;
+ u32 s_addr;
+ u8 x_addrs;
+ u8 s_bit;
+ u8 x_bits;
+ u8 sz_reg;
+ u8 fixed;
+};
+
+/* struct mtk_pin_reg_calc - the structure that holds all ranges used to
+ * determine which register the pin would make use of
+ * for certain pin attribute.
+ * @range: the start address for the range
+ * @nranges: the number of items in the range
+ */
+struct mtk_pin_reg_calc {
+ const struct mtk_pin_field_calc *range;
+ unsigned int nranges;
+};
+
+/**
+ * struct mtk_func_desc - the structure that providing information
+ * all the funcs for this pin
+ * @name: the name of function
+ * @muxval: the mux to the function
+ */
+struct mtk_func_desc {
+ const char *name;
+ u8 muxval;
+};
+
+/**
+ * struct mtk_eint_desc - the structure that providing information
+ * for eint data per pin
+ * @eint_m: the eint mux for this pin
+ * @eitn_n: the eint number for this pin
+ */
+struct mtk_eint_desc {
+ u16 eint_m;
+ u16 eint_n;
+};
+
+/**
+ * struct mtk_pin_desc - the structure that providing information
+ * for each pin of chips
+ * @number: unique pin number from the global pin number space
+ * @name: name for this pin
+ * @eint: the eint data for this pin
+ * @drv_n: the index with the driving group
+ * @funcs: all available functions for this pins (only used in
+ * those drivers compatible to pinctrl-mtk-common.c-like
+ * ones)
+ */
+struct mtk_pin_desc {
+ unsigned int number;
+ const char *name;
+ struct mtk_eint_desc eint;
+ u8 drv_n;
+ struct mtk_func_desc *funcs;
+};
+
+struct mtk_pinctrl_group {
+ const char *name;
+ unsigned long config;
+ unsigned pin;
+};
+
+struct mtk_pinctrl;
+
+/* struct mtk_pin_soc - the structure that holds SoC-specific data */
+struct mtk_pin_soc {
+ const struct mtk_pin_reg_calc *reg_cal;
+ const struct mtk_pin_desc *pins;
+ unsigned int npins;
+ const struct group_desc *grps;
+ unsigned int ngrps;
+ const struct function_desc *funcs;
+ unsigned int nfuncs;
+ const struct mtk_eint_regs *eint_regs;
+ const struct mtk_eint_hw *eint_hw;
+
+ /* Specific parameters per SoC */
+ u8 gpio_m;
+ bool ies_present;
+ const char * const *base_names;
+ unsigned int nbase_names;
+
+ /* Specific pinconfig operations */
+ int (*bias_disable_set)(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc);
+ int (*bias_disable_get)(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, int *res);
+ int (*bias_set)(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup);
+ int (*bias_get)(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup, int *res);
+
+ int (*drive_set)(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, u32 arg);
+ int (*drive_get)(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, int *val);
+
+ int (*adv_pull_set)(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup,
+ u32 arg);
+ int (*adv_pull_get)(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup,
+ u32 *val);
+
+ /* Specific driver data */
+ void *driver_data;
+};
+
+struct mtk_pinctrl {
+ struct pinctrl_dev *pctrl;
+ void __iomem **base;
+ u8 nbase;
+ struct device *dev;
+ struct gpio_chip chip;
+ const struct mtk_pin_soc *soc;
+ struct mtk_eint *eint;
+ struct mtk_pinctrl_group *groups;
+ const char **grp_names;
+};
+
+void mtk_rmw(struct mtk_pinctrl *pctl, u8 i, u32 reg, u32 mask, u32 set);
+
+int mtk_hw_set_value(struct mtk_pinctrl *hw, const struct mtk_pin_desc *desc,
+ int field, int value);
+int mtk_hw_get_value(struct mtk_pinctrl *hw, const struct mtk_pin_desc *desc,
+ int field, int *value);
+
+int mtk_build_eint(struct mtk_pinctrl *hw, struct platform_device *pdev);
+
+int mtk_pinconf_bias_disable_set(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc);
+int mtk_pinconf_bias_disable_get(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, int *res);
+int mtk_pinconf_bias_set(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup);
+int mtk_pinconf_bias_get(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup,
+ int *res);
+
+int mtk_pinconf_bias_disable_set_rev1(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc);
+int mtk_pinconf_bias_disable_get_rev1(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc,
+ int *res);
+int mtk_pinconf_bias_set_rev1(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup);
+int mtk_pinconf_bias_get_rev1(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup,
+ int *res);
+
+int mtk_pinconf_drive_set(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, u32 arg);
+int mtk_pinconf_drive_get(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, int *val);
+
+int mtk_pinconf_drive_set_rev1(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, u32 arg);
+int mtk_pinconf_drive_get_rev1(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, int *val);
+
+int mtk_pinconf_adv_pull_set(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup,
+ u32 arg);
+int mtk_pinconf_adv_pull_get(struct mtk_pinctrl *hw,
+ const struct mtk_pin_desc *desc, bool pullup,
+ u32 *val);
+
+#endif /* __PINCTRL_MTK_COMMON_V2_H */
pins = of_find_property(node, "pinmux", NULL);
if (!pins) {
- dev_err(pctl->dev, "missing pins property in node %s .\n",
- node->name);
+ dev_err(pctl->dev, "missing pins property in node %pOFn .\n",
+ node);
return -EINVAL;
}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 MediaTek Inc.
+ *
+ * Author: ZH Chen <zh.chen@mediatek.com>
+ *
+ */
+
+#ifndef __PINCTRL_MTK_MT6765_H
+#define __PINCTRL_MTK_MT6765_H
+
+#include "pinctrl-paris.h"
+
+static struct mtk_pin_desc mtk_pins_mt6765[] = {
+ MTK_PIN(
+ 0, "GPIO0",
+ MTK_EINT_FUNCTION(0, 0),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO0"),
+ MTK_FUNCTION(1, "UTXD1"),
+ MTK_FUNCTION(2, "CLKM0"),
+ MTK_FUNCTION(3, "MD_INT0"),
+ MTK_FUNCTION(4, "I2S0_MCK"),
+ MTK_FUNCTION(5, "MD_UTXD1"),
+ MTK_FUNCTION(6, "TP_GPIO0_AO"),
+ MTK_FUNCTION(7, "DBG_MON_B9")
+ ),
+ MTK_PIN(
+ 1, "GPIO1",
+ MTK_EINT_FUNCTION(0, 1),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO1"),
+ MTK_FUNCTION(1, "URXD1"),
+ MTK_FUNCTION(2, "CLKM1"),
+ MTK_FUNCTION(4, "I2S0_BCK"),
+ MTK_FUNCTION(5, "MD_URXD1"),
+ MTK_FUNCTION(6, "TP_GPIO1_AO"),
+ MTK_FUNCTION(7, "DBG_MON_B10")
+ ),
+ MTK_PIN(
+ 2, "GPIO2",
+ MTK_EINT_FUNCTION(0, 2),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO2"),
+ MTK_FUNCTION(1, "UCTS0"),
+ MTK_FUNCTION(2, "CLKM2"),
+ MTK_FUNCTION(3, "UTXD1"),
+ MTK_FUNCTION(4, "I2S0_LRCK"),
+ MTK_FUNCTION(5, "ANT_SEL6"),
+ MTK_FUNCTION(6, "TP_GPIO2_AO"),
+ MTK_FUNCTION(7, "DBG_MON_B11")
+ ),
+ MTK_PIN(
+ 3, "GPIO3",
+ MTK_EINT_FUNCTION(0, 3),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO3"),
+ MTK_FUNCTION(1, "URTS0"),
+ MTK_FUNCTION(2, "CLKM3"),
+ MTK_FUNCTION(3, "URXD1"),
+ MTK_FUNCTION(4, "I2S0_DI"),
+ MTK_FUNCTION(5, "ANT_SEL7"),
+ MTK_FUNCTION(6, "TP_GPIO3_AO"),
+ MTK_FUNCTION(7, "DBG_MON_B12")
+ ),
+ MTK_PIN(
+ 4, "GPIO4",
+ MTK_EINT_FUNCTION(0, 4),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO4"),
+ MTK_FUNCTION(1, "SPI1_B_MI"),
+ MTK_FUNCTION(2, "SCP_SPI1_MI"),
+ MTK_FUNCTION(3, "UCTS0"),
+ MTK_FUNCTION(4, "I2S3_MCK"),
+ MTK_FUNCTION(5, "SSPM_URXD_AO"),
+ MTK_FUNCTION(6, "TP_GPIO4_AO")
+ ),
+ MTK_PIN(
+ 5, "GPIO5",
+ MTK_EINT_FUNCTION(0, 5),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO5"),
+ MTK_FUNCTION(1, "SPI1_B_CSB"),
+ MTK_FUNCTION(2, "SCP_SPI1_CS"),
+ MTK_FUNCTION(3, "URTS0"),
+ MTK_FUNCTION(4, "I2S3_BCK"),
+ MTK_FUNCTION(5, "SSPM_UTXD_AO"),
+ MTK_FUNCTION(6, "TP_GPIO5_AO")
+ ),
+ MTK_PIN(
+ 6, "GPIO6",
+ MTK_EINT_FUNCTION(0, 6),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO6"),
+ MTK_FUNCTION(1, "SPI1_B_MO"),
+ MTK_FUNCTION(2, "SCP_SPI1_MO"),
+ MTK_FUNCTION(3, "PWM0"),
+ MTK_FUNCTION(4, "I2S3_LRCK"),
+ MTK_FUNCTION(5, "MD_UTXD0"),
+ MTK_FUNCTION(6, "TP_GPIO6_AO")
+ ),
+ MTK_PIN(
+ 7, "GPIO7",
+ MTK_EINT_FUNCTION(0, 7),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO7"),
+ MTK_FUNCTION(1, "SPI1_B_CLK"),
+ MTK_FUNCTION(2, "SCP_SPI1_CK"),
+ MTK_FUNCTION(3, "PWM1"),
+ MTK_FUNCTION(4, "I2S3_DO"),
+ MTK_FUNCTION(5, "MD_URXD0"),
+ MTK_FUNCTION(6, "TP_GPIO7_AO")
+ ),
+ MTK_PIN(
+ 8, "GPIO8",
+ MTK_EINT_FUNCTION(0, 8),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO8"),
+ MTK_FUNCTION(1, "UTXD1"),
+ MTK_FUNCTION(2, "SRCLKENAI0"),
+ MTK_FUNCTION(3, "MD_INT1_C2K_UIM0_HOT_PLUG"),
+ MTK_FUNCTION(4, "ANT_SEL3"),
+ MTK_FUNCTION(5, "MFG_JTAG_TRSTN"),
+ MTK_FUNCTION(6, "I2S2_MCK"),
+ MTK_FUNCTION(7, "JTRSTN_SEL1")
+ ),
+ MTK_PIN(
+ 9, "GPIO9",
+ MTK_EINT_FUNCTION(0, 9),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO9"),
+ MTK_FUNCTION(1, "MD_INT0"),
+ MTK_FUNCTION(2, "CMMCLK2"),
+ MTK_FUNCTION(3, "CONN_MCU_TRST_B"),
+ MTK_FUNCTION(4, "IDDIG"),
+ MTK_FUNCTION(5, "SDA_6306"),
+ MTK_FUNCTION(6, "MCUPM_JTAG_TRSTN"),
+ MTK_FUNCTION(7, "DBG_MON_B22")
+ ),
+ MTK_PIN(
+ 10, "GPIO10",
+ MTK_EINT_FUNCTION(0, 10),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO10"),
+ MTK_FUNCTION(1, "MD_INT1_C2K_UIM0_HOT_PLUG"),
+ MTK_FUNCTION(3, "CONN_MCU_DBGI_N"),
+ MTK_FUNCTION(4, "SRCLKENAI1"),
+ MTK_FUNCTION(5, "EXT_FRAME_SYNC"),
+ MTK_FUNCTION(6, "CMVREF1"),
+ MTK_FUNCTION(7, "DBG_MON_B23")
+ ),
+ MTK_PIN(
+ 11, "GPIO11",
+ MTK_EINT_FUNCTION(0, 11),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO11"),
+ MTK_FUNCTION(1, "MD_INT2_C2K_UIM1_HOT_PLUG"),
+ MTK_FUNCTION(2, "CLKM3"),
+ MTK_FUNCTION(3, "ANT_SEL6"),
+ MTK_FUNCTION(4, "SRCLKENAI0"),
+ MTK_FUNCTION(5, "EXT_FRAME_SYNC"),
+ MTK_FUNCTION(6, "UCTS1"),
+ MTK_FUNCTION(7, "DBG_MON_B24")
+ ),
+ MTK_PIN(
+ 12, "GPIO12",
+ MTK_EINT_FUNCTION(0, 12),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO12"),
+ MTK_FUNCTION(1, "PWM0"),
+ MTK_FUNCTION(2, "SRCLKENAI1"),
+ MTK_FUNCTION(3, "EXT_FRAME_SYNC"),
+ MTK_FUNCTION(4, "MD_INT0"),
+ MTK_FUNCTION(5, "DVFSRC_EXT_REQ"),
+ MTK_FUNCTION(6, "URTS1")
+ ),
+ MTK_PIN(
+ 13, "GPIO13",
+ MTK_EINT_FUNCTION(0, 13),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO13"),
+ MTK_FUNCTION(1, "ANT_SEL0"),
+ MTK_FUNCTION(2, "SPI4_MI"),
+ MTK_FUNCTION(3, "SCP_SPI0_MI"),
+ MTK_FUNCTION(4, "MD_URXD0"),
+ MTK_FUNCTION(5, "CLKM0"),
+ MTK_FUNCTION(6, "I2S0_MCK"),
+ MTK_FUNCTION(7, "DBG_MON_A0")
+ ),
+ MTK_PIN(
+ 14, "GPIO14",
+ MTK_EINT_FUNCTION(0, 14),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO14"),
+ MTK_FUNCTION(1, "ANT_SEL1"),
+ MTK_FUNCTION(2, "SPI4_CSB"),
+ MTK_FUNCTION(3, "SCP_SPI0_CS"),
+ MTK_FUNCTION(4, "MD_UTXD0"),
+ MTK_FUNCTION(5, "CLKM1"),
+ MTK_FUNCTION(6, "I2S0_BCK"),
+ MTK_FUNCTION(7, "DBG_MON_A1")
+ ),
+ MTK_PIN(
+ 15, "GPIO15",
+ MTK_EINT_FUNCTION(0, 15),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO15"),
+ MTK_FUNCTION(1, "ANT_SEL2"),
+ MTK_FUNCTION(2, "SPI4_MO"),
+ MTK_FUNCTION(3, "SCP_SPI0_MO"),
+ MTK_FUNCTION(4, "MD_URXD1"),
+ MTK_FUNCTION(5, "CLKM2"),
+ MTK_FUNCTION(6, "I2S0_LRCK"),
+ MTK_FUNCTION(7, "DBG_MON_A2")
+ ),
+ MTK_PIN(
+ 16, "GPIO16",
+ MTK_EINT_FUNCTION(0, 16),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO16"),
+ MTK_FUNCTION(1, "ANT_SEL3"),
+ MTK_FUNCTION(2, "SPI4_CLK"),
+ MTK_FUNCTION(3, "SCP_SPI0_CK"),
+ MTK_FUNCTION(4, "MD_UTXD1"),
+ MTK_FUNCTION(5, "CLKM3"),
+ MTK_FUNCTION(6, "I2S3_MCK"),
+ MTK_FUNCTION(7, "DBG_MON_A3")
+ ),
+ MTK_PIN(
+ 17, "GPIO17",
+ MTK_EINT_FUNCTION(0, 17),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO17"),
+ MTK_FUNCTION(1, "ANT_SEL4"),
+ MTK_FUNCTION(2, "SPI2_MO"),
+ MTK_FUNCTION(3, "SCP_SPI0_MO"),
+ MTK_FUNCTION(4, "PWM1"),
+ MTK_FUNCTION(5, "IDDIG"),
+ MTK_FUNCTION(6, "I2S0_DI"),
+ MTK_FUNCTION(7, "DBG_MON_A4")
+ ),
+ MTK_PIN(
+ 18, "GPIO18",
+ MTK_EINT_FUNCTION(0, 18),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO18"),
+ MTK_FUNCTION(1, "ANT_SEL5"),
+ MTK_FUNCTION(2, "SPI2_CLK"),
+ MTK_FUNCTION(3, "SCP_SPI0_CK"),
+ MTK_FUNCTION(4, "MD_INT0"),
+ MTK_FUNCTION(5, "USB_DRVVBUS"),
+ MTK_FUNCTION(6, "I2S3_BCK"),
+ MTK_FUNCTION(7, "DBG_MON_A5")
+ ),
+ MTK_PIN(
+ 19, "GPIO19",
+ MTK_EINT_FUNCTION(0, 19),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO19"),
+ MTK_FUNCTION(1, "ANT_SEL6"),
+ MTK_FUNCTION(2, "SPI2_MI"),
+ MTK_FUNCTION(3, "SCP_SPI0_MI"),
+ MTK_FUNCTION(4, "MD_INT2_C2K_UIM1_HOT_PLUG"),
+ MTK_FUNCTION(6, "I2S3_LRCK"),
+ MTK_FUNCTION(7, "DBG_MON_A6")
+ ),
+ MTK_PIN(
+ 20, "GPIO20",
+ MTK_EINT_FUNCTION(0, 20),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO20"),
+ MTK_FUNCTION(1, "ANT_SEL7"),
+ MTK_FUNCTION(2, "SPI2_CSB"),
+ MTK_FUNCTION(3, "SCP_SPI0_CS"),
+ MTK_FUNCTION(4, "MD_INT1_C2K_UIM0_HOT_PLUG"),
+ MTK_FUNCTION(5, "CMMCLK3"),
+ MTK_FUNCTION(6, "I2S3_DO"),
+ MTK_FUNCTION(7, "DBG_MON_A7")
+ ),
+ MTK_PIN(
+ 21, "GPIO21",
+ MTK_EINT_FUNCTION(0, 21),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO21"),
+ MTK_FUNCTION(1, "SPI3_MI"),
+ MTK_FUNCTION(2, "SRCLKENAI1"),
+ MTK_FUNCTION(3, "DAP_MD32_SWD"),
+ MTK_FUNCTION(4, "CMVREF0"),
+ MTK_FUNCTION(5, "SCP_SPI0_MI"),
+ MTK_FUNCTION(6, "I2S2_MCK"),
+ MTK_FUNCTION(7, "DBG_MON_A8")
+ ),
+ MTK_PIN(
+ 22, "GPIO22",
+ MTK_EINT_FUNCTION(0, 22),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO22"),
+ MTK_FUNCTION(1, "SPI3_CSB"),
+ MTK_FUNCTION(2, "SRCLKENAI0"),
+ MTK_FUNCTION(3, "DAP_MD32_SWCK"),
+ MTK_FUNCTION(4, "CMVREF1"),
+ MTK_FUNCTION(5, "SCP_SPI0_CS"),
+ MTK_FUNCTION(6, "I2S2_BCK"),
+ MTK_FUNCTION(7, "DBG_MON_A9")
+ ),
+ MTK_PIN(
+ 23, "GPIO23",
+ MTK_EINT_FUNCTION(0, 23),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO23"),
+ MTK_FUNCTION(1, "SPI3_MO"),
+ MTK_FUNCTION(2, "PWM0"),
+ MTK_FUNCTION(3, "KPROW7"),
+ MTK_FUNCTION(4, "ANT_SEL3"),
+ MTK_FUNCTION(5, "SCP_SPI0_MO"),
+ MTK_FUNCTION(6, "I2S2_LRCK"),
+ MTK_FUNCTION(7, "DBG_MON_A10")
+ ),
+ MTK_PIN(
+ 24, "GPIO24",
+ MTK_EINT_FUNCTION(0, 24),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO24"),
+ MTK_FUNCTION(1, "SPI3_CLK"),
+ MTK_FUNCTION(2, "UDI_TCK"),
+ MTK_FUNCTION(3, "IO_JTAG_TCK"),
+ MTK_FUNCTION(4, "SSPM_JTAG_TCK"),
+ MTK_FUNCTION(5, "SCP_SPI0_CK"),
+ MTK_FUNCTION(6, "I2S2_DI"),
+ MTK_FUNCTION(7, "DBG_MON_A11")
+ ),
+ MTK_PIN(
+ 25, "GPIO25",
+ MTK_EINT_FUNCTION(0, 25),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO25"),
+ MTK_FUNCTION(1, "SPI1_A_MI"),
+ MTK_FUNCTION(2, "UDI_TMS"),
+ MTK_FUNCTION(3, "IO_JTAG_TMS"),
+ MTK_FUNCTION(4, "SSPM_JTAG_TMS"),
+ MTK_FUNCTION(5, "KPROW3"),
+ MTK_FUNCTION(6, "I2S1_MCK"),
+ MTK_FUNCTION(7, "DBG_MON_A12")
+ ),
+ MTK_PIN(
+ 26, "GPIO26",
+ MTK_EINT_FUNCTION(0, 26),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO26"),
+ MTK_FUNCTION(1, "SPI1_A_CSB"),
+ MTK_FUNCTION(2, "UDI_TDI"),
+ MTK_FUNCTION(3, "IO_JTAG_TDI"),
+ MTK_FUNCTION(4, "SSPM_JTAG_TDI"),
+ MTK_FUNCTION(5, "KPROW4"),
+ MTK_FUNCTION(6, "I2S1_BCK"),
+ MTK_FUNCTION(7, "DBG_MON_A13")
+ ),
+ MTK_PIN(
+ 27, "GPIO27",
+ MTK_EINT_FUNCTION(0, 27),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO27"),
+ MTK_FUNCTION(1, "SPI1_A_MO"),
+ MTK_FUNCTION(2, "UDI_TDO"),
+ MTK_FUNCTION(3, "IO_JTAG_TDO"),
+ MTK_FUNCTION(4, "SSPM_JTAG_TDO"),
+ MTK_FUNCTION(5, "KPROW5"),
+ MTK_FUNCTION(6, "I2S1_LRCK"),
+ MTK_FUNCTION(7, "DBG_MON_A14")
+ ),
+ MTK_PIN(
+ 28, "GPIO28",
+ MTK_EINT_FUNCTION(0, 28),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO28"),
+ MTK_FUNCTION(1, "SPI1_A_CLK"),
+ MTK_FUNCTION(2, "UDI_NTRST"),
+ MTK_FUNCTION(3, "IO_JTAG_TRSTN"),
+ MTK_FUNCTION(4, "SSPM_JTAG_TRSTN"),
+ MTK_FUNCTION(5, "KPROW6"),
+ MTK_FUNCTION(6, "I2S1_DO"),
+ MTK_FUNCTION(7, "DBG_MON_A15")
+ ),
+ MTK_PIN(
+ 29, "GPIO29",
+ MTK_EINT_FUNCTION(0, 29),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO29"),
+ MTK_FUNCTION(1, "MSDC1_CLK"),
+ MTK_FUNCTION(2, "IO_JTAG_TCK"),
+ MTK_FUNCTION(3, "UDI_TCK"),
+ MTK_FUNCTION(4, "CONN_DSP_JCK"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TCK"),
+ MTK_FUNCTION(6, "CONN_MCU_AICE_TCKC"),
+ MTK_FUNCTION(7, "DAP_MD32_SWCK")
+ ),
+ MTK_PIN(
+ 30, "GPIO30",
+ MTK_EINT_FUNCTION(0, 30),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO30"),
+ MTK_FUNCTION(1, "MSDC1_CMD"),
+ MTK_FUNCTION(2, "IO_JTAG_TMS"),
+ MTK_FUNCTION(3, "UDI_TMS"),
+ MTK_FUNCTION(4, "CONN_DSP_JMS"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TMS"),
+ MTK_FUNCTION(6, "CONN_MCU_AICE_TMSC"),
+ MTK_FUNCTION(7, "DAP_MD32_SWD")
+ ),
+ MTK_PIN(
+ 31, "GPIO31",
+ MTK_EINT_FUNCTION(0, 31),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO31"),
+ MTK_FUNCTION(1, "MSDC1_DAT3")
+ ),
+ MTK_PIN(
+ 32, "GPIO32",
+ MTK_EINT_FUNCTION(0, 32),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO32"),
+ MTK_FUNCTION(1, "MSDC1_DAT0"),
+ MTK_FUNCTION(2, "IO_JTAG_TDI"),
+ MTK_FUNCTION(3, "UDI_TDI"),
+ MTK_FUNCTION(4, "CONN_DSP_JDI"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TDI")
+ ),
+ MTK_PIN(
+ 33, "GPIO33",
+ MTK_EINT_FUNCTION(0, 33),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO33"),
+ MTK_FUNCTION(1, "MSDC1_DAT2"),
+ MTK_FUNCTION(2, "IO_JTAG_TRSTN"),
+ MTK_FUNCTION(3, "UDI_NTRST"),
+ MTK_FUNCTION(4, "CONN_DSP_JINTP"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TRSTN")
+ ),
+ MTK_PIN(
+ 34, "GPIO34",
+ MTK_EINT_FUNCTION(0, 34),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO34"),
+ MTK_FUNCTION(1, "MSDC1_DAT1"),
+ MTK_FUNCTION(2, "IO_JTAG_TDO"),
+ MTK_FUNCTION(3, "UDI_TDO"),
+ MTK_FUNCTION(4, "CONN_DSP_JDO"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TDO")
+ ),
+ MTK_PIN(
+ 35, "GPIO35",
+ MTK_EINT_FUNCTION(0, 35),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO35"),
+ MTK_FUNCTION(1, "MD1_SIM2_SIO"),
+ MTK_FUNCTION(2, "CCU_JTAG_TDO"),
+ MTK_FUNCTION(3, "MD1_SIM1_SIO"),
+ MTK_FUNCTION(5, "SCP_JTAG_TDO"),
+ MTK_FUNCTION(6, "CONN_DSP_JDO"),
+ MTK_FUNCTION(7, "DBG_MON_A16")
+ ),
+ MTK_PIN(
+ 36, "GPIO36",
+ MTK_EINT_FUNCTION(0, 36),
+ DRV_GRP0,
+ MTK_FUNCTION(0, "GPIO36"),
+ MTK_FUNCTION(1, "MD1_SIM2_SRST"),
+ MTK_FUNCTION(2, "CCU_JTAG_TMS"),
+ MTK_FUNCTION(3, "MD1_SIM1_SRST"),
+ MTK_FUNCTION(4, "CONN_MCU_AICE_TMSC"),
+ MTK_FUNCTION(5, "SCP_JTAG_TMS"),
+ MTK_FUNCTION(6, "CONN_DSP_JMS"),
+ MTK_FUNCTION(7, "DBG_MON_A17")
+ ),
+ MTK_PIN(
+ 37, "GPIO37",
+ MTK_EINT_FUNCTION(0, 37),
+ DRV_GRP0,
+ MTK_FUNCTION(0, "GPIO37"),
+ MTK_FUNCTION(1, "MD1_SIM2_SCLK"),
+ MTK_FUNCTION(2, "CCU_JTAG_TDI"),
+ MTK_FUNCTION(3, "MD1_SIM1_SCLK"),
+ MTK_FUNCTION(5, "SCP_JTAG_TDI"),
+ MTK_FUNCTION(6, "CONN_DSP_JDI"),
+ MTK_FUNCTION(7, "DBG_MON_A18")
+ ),
+ MTK_PIN(
+ 38, "GPIO38",
+ MTK_EINT_FUNCTION(0, 38),
+ DRV_GRP0,
+ MTK_FUNCTION(0, "GPIO38"),
+ MTK_FUNCTION(1, "MD1_SIM1_SCLK"),
+ MTK_FUNCTION(3, "MD1_SIM2_SCLK"),
+ MTK_FUNCTION(7, "DBG_MON_A19")
+ ),
+ MTK_PIN(
+ 39, "GPIO39",
+ MTK_EINT_FUNCTION(0, 39),
+ DRV_GRP0,
+ MTK_FUNCTION(0, "GPIO39"),
+ MTK_FUNCTION(1, "MD1_SIM1_SRST"),
+ MTK_FUNCTION(2, "CCU_JTAG_TCK"),
+ MTK_FUNCTION(3, "MD1_SIM2_SRST"),
+ MTK_FUNCTION(4, "CONN_MCU_AICE_TCKC"),
+ MTK_FUNCTION(5, "SCP_JTAG_TCK"),
+ MTK_FUNCTION(6, "CONN_DSP_JCK"),
+ MTK_FUNCTION(7, "DBG_MON_A20")
+ ),
+ MTK_PIN(
+ 40, "GPIO40",
+ MTK_EINT_FUNCTION(0, 40),
+ DRV_GRP0,
+ MTK_FUNCTION(0, "GPIO40"),
+ MTK_FUNCTION(1, "MD1_SIM1_SIO"),
+ MTK_FUNCTION(2, "CCU_JTAG_TRST"),
+ MTK_FUNCTION(3, "MD1_SIM2_SIO"),
+ MTK_FUNCTION(5, "SCP_JTAG_TRSTN"),
+ MTK_FUNCTION(6, "CONN_DSP_JINTP"),
+ MTK_FUNCTION(7, "DBG_MON_A21")
+ ),
+ MTK_PIN(
+ 41, "GPIO41",
+ MTK_EINT_FUNCTION(0, 41),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO41"),
+ MTK_FUNCTION(1, "IDDIG"),
+ MTK_FUNCTION(2, "URXD1"),
+ MTK_FUNCTION(3, "UCTS0"),
+ MTK_FUNCTION(4, "KPCOL2"),
+ MTK_FUNCTION(5, "SSPM_UTXD_AO"),
+ MTK_FUNCTION(6, "MD_INT0"),
+ MTK_FUNCTION(7, "DBG_MON_A22")
+ ),
+ MTK_PIN(
+ 42, "GPIO42",
+ MTK_EINT_FUNCTION(0, 42),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO42"),
+ MTK_FUNCTION(1, "USB_DRVVBUS"),
+ MTK_FUNCTION(2, "UTXD1"),
+ MTK_FUNCTION(3, "URTS0"),
+ MTK_FUNCTION(4, "KPROW2"),
+ MTK_FUNCTION(5, "SSPM_URXD_AO"),
+ MTK_FUNCTION(6, "MD_INT1_C2K_UIM0_HOT_PLUG"),
+ MTK_FUNCTION(7, "DBG_MON_A23")
+ ),
+ MTK_PIN(
+ 43, "GPIO43",
+ MTK_EINT_FUNCTION(0, 43),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO43"),
+ MTK_FUNCTION(1, "DISP_PWM"),
+ MTK_FUNCTION(7, "DBG_MON_A24")
+ ),
+ MTK_PIN(
+ 44, "GPIO44",
+ MTK_EINT_FUNCTION(0, 44),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO44"),
+ MTK_FUNCTION(1, "DSI_TE"),
+ MTK_FUNCTION(7, "DBG_MON_A25")
+ ),
+ MTK_PIN(
+ 45, "GPIO45",
+ MTK_EINT_FUNCTION(0, 45),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO45"),
+ MTK_FUNCTION(1, "LCM_RST"),
+ MTK_FUNCTION(7, "DBG_MON_A26")
+ ),
+ MTK_PIN(
+ 46, "GPIO46",
+ MTK_EINT_FUNCTION(0, 46),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO46"),
+ MTK_FUNCTION(1, "MD_INT2_C2K_UIM1_HOT_PLUG"),
+ MTK_FUNCTION(2, "UCTS0"),
+ MTK_FUNCTION(3, "UCTS1"),
+ MTK_FUNCTION(4, "IDDIG"),
+ MTK_FUNCTION(5, "SCL_6306"),
+ MTK_FUNCTION(6, "TP_UCTS1_AO"),
+ MTK_FUNCTION(7, "DBG_MON_A27")
+ ),
+ MTK_PIN(
+ 47, "GPIO47",
+ MTK_EINT_FUNCTION(0, 47),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO47"),
+ MTK_FUNCTION(1, "MD_INT1_C2K_UIM0_HOT_PLUG"),
+ MTK_FUNCTION(2, "URTS0"),
+ MTK_FUNCTION(3, "URTS1"),
+ MTK_FUNCTION(4, "USB_DRVVBUS"),
+ MTK_FUNCTION(5, "SDA_6306"),
+ MTK_FUNCTION(6, "TP_URTS1_AO"),
+ MTK_FUNCTION(7, "DBG_MON_A28")
+ ),
+ MTK_PIN(
+ 48, "GPIO48",
+ MTK_EINT_FUNCTION(0, 48),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO48"),
+ MTK_FUNCTION(1, "SCL5"),
+ MTK_FUNCTION(7, "DBG_MON_A29")
+ ),
+ MTK_PIN(
+ 49, "GPIO49",
+ MTK_EINT_FUNCTION(0, 49),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO49"),
+ MTK_FUNCTION(1, "SDA5"),
+ MTK_FUNCTION(7, "DBG_MON_A30")
+ ),
+ MTK_PIN(
+ 50, "GPIO50",
+ MTK_EINT_FUNCTION(0, 50),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO50"),
+ MTK_FUNCTION(1, "SCL3"),
+ MTK_FUNCTION(2, "URXD1"),
+ MTK_FUNCTION(3, "MD_URXD1"),
+ MTK_FUNCTION(4, "SSPM_URXD_AO"),
+ MTK_FUNCTION(5, "IDDIG"),
+ MTK_FUNCTION(6, "TP_URXD1_AO"),
+ MTK_FUNCTION(7, "DBG_MON_A31")
+ ),
+ MTK_PIN(
+ 51, "GPIO51",
+ MTK_EINT_FUNCTION(0, 51),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO51"),
+ MTK_FUNCTION(1, "SDA3"),
+ MTK_FUNCTION(2, "UTXD1"),
+ MTK_FUNCTION(3, "MD_UTXD1"),
+ MTK_FUNCTION(4, "SSPM_UTXD_AO"),
+ MTK_FUNCTION(5, "USB_DRVVBUS"),
+ MTK_FUNCTION(6, "TP_UTXD1_AO"),
+ MTK_FUNCTION(7, "DBG_MON_A32")
+ ),
+ MTK_PIN(
+ 52, "GPIO52",
+ MTK_EINT_FUNCTION(0, 52),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO52"),
+ MTK_FUNCTION(1, "BPI_BUS15")
+ ),
+ MTK_PIN(
+ 53, "GPIO53",
+ MTK_EINT_FUNCTION(0, 53),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO53"),
+ MTK_FUNCTION(1, "BPI_BUS13")
+ ),
+ MTK_PIN(
+ 54, "GPIO54",
+ MTK_EINT_FUNCTION(0, 54),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO54"),
+ MTK_FUNCTION(1, "BPI_BUS12")
+ ),
+ MTK_PIN(
+ 55, "GPIO55",
+ MTK_EINT_FUNCTION(0, 55),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO55"),
+ MTK_FUNCTION(1, "BPI_BUS8")
+ ),
+ MTK_PIN(
+ 56, "GPIO56",
+ MTK_EINT_FUNCTION(0, 56),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO56"),
+ MTK_FUNCTION(1, "BPI_BUS9"),
+ MTK_FUNCTION(2, "SCL_6306")
+ ),
+ MTK_PIN(
+ 57, "GPIO57",
+ MTK_EINT_FUNCTION(0, 57),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO57"),
+ MTK_FUNCTION(1, "BPI_BUS10"),
+ MTK_FUNCTION(2, "SDA_6306")
+ ),
+ MTK_PIN(
+ 58, "GPIO58",
+ MTK_EINT_FUNCTION(0, 58),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO58"),
+ MTK_FUNCTION(1, "RFIC0_BSI_D2")
+ ),
+ MTK_PIN(
+ 59, "GPIO59",
+ MTK_EINT_FUNCTION(0, 59),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO59"),
+ MTK_FUNCTION(1, "RFIC0_BSI_D1")
+ ),
+ MTK_PIN(
+ 60, "GPIO60",
+ MTK_EINT_FUNCTION(0, 60),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO60"),
+ MTK_FUNCTION(1, "RFIC0_BSI_D0")
+ ),
+ MTK_PIN(
+ 61, "GPIO61",
+ MTK_EINT_FUNCTION(0, 61),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO61"),
+ MTK_FUNCTION(1, "MIPI1_SDATA")
+ ),
+ MTK_PIN(
+ 62, "GPIO62",
+ MTK_EINT_FUNCTION(0, 62),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO62"),
+ MTK_FUNCTION(1, "MIPI1_SCLK")
+ ),
+ MTK_PIN(
+ 63, "GPIO63",
+ MTK_EINT_FUNCTION(0, 63),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO63"),
+ MTK_FUNCTION(1, "MIPI0_SDATA")
+ ),
+ MTK_PIN(
+ 64, "GPIO64",
+ MTK_EINT_FUNCTION(0, 64),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO64"),
+ MTK_FUNCTION(1, "MIPI0_SCLK")
+ ),
+ MTK_PIN(
+ 65, "GPIO65",
+ MTK_EINT_FUNCTION(0, 65),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO65"),
+ MTK_FUNCTION(1, "MIPI3_SDATA"),
+ MTK_FUNCTION(2, "BPI_BUS16")
+ ),
+ MTK_PIN(
+ 66, "GPIO66",
+ MTK_EINT_FUNCTION(0, 66),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO66"),
+ MTK_FUNCTION(1, "MIPI3_SCLK"),
+ MTK_FUNCTION(2, "BPI_BUS17")
+ ),
+ MTK_PIN(
+ 67, "GPIO67",
+ MTK_EINT_FUNCTION(0, 67),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO67"),
+ MTK_FUNCTION(1, "MIPI2_SDATA")
+ ),
+ MTK_PIN(
+ 68, "GPIO68",
+ MTK_EINT_FUNCTION(0, 68),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO68"),
+ MTK_FUNCTION(1, "MIPI2_SCLK")
+ ),
+ MTK_PIN(
+ 69, "GPIO69",
+ MTK_EINT_FUNCTION(0, 69),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO69"),
+ MTK_FUNCTION(1, "BPI_BUS7")
+ ),
+ MTK_PIN(
+ 70, "GPIO70",
+ MTK_EINT_FUNCTION(0, 70),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO70"),
+ MTK_FUNCTION(1, "BPI_BUS6")
+ ),
+ MTK_PIN(
+ 71, "GPIO71",
+ MTK_EINT_FUNCTION(0, 71),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO71"),
+ MTK_FUNCTION(1, "BPI_BUS5")
+ ),
+ MTK_PIN(
+ 72, "GPIO72",
+ MTK_EINT_FUNCTION(0, 72),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO72"),
+ MTK_FUNCTION(1, "BPI_BUS4")
+ ),
+ MTK_PIN(
+ 73, "GPIO73",
+ MTK_EINT_FUNCTION(0, 73),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO73"),
+ MTK_FUNCTION(1, "BPI_BUS3")
+ ),
+ MTK_PIN(
+ 74, "GPIO74",
+ MTK_EINT_FUNCTION(0, 74),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO74"),
+ MTK_FUNCTION(1, "BPI_BUS2")
+ ),
+ MTK_PIN(
+ 75, "GPIO75",
+ MTK_EINT_FUNCTION(0, 75),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO75"),
+ MTK_FUNCTION(1, "BPI_BUS1")
+ ),
+ MTK_PIN(
+ 76, "GPIO76",
+ MTK_EINT_FUNCTION(0, 76),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO76"),
+ MTK_FUNCTION(1, "BPI_BUS0")
+ ),
+ MTK_PIN(
+ 77, "GPIO77",
+ MTK_EINT_FUNCTION(0, 77),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO77"),
+ MTK_FUNCTION(1, "BPI_BUS14")
+ ),
+ MTK_PIN(
+ 78, "GPIO78",
+ MTK_EINT_FUNCTION(0, 78),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO78"),
+ MTK_FUNCTION(1, "BPI_BUS11")
+ ),
+ MTK_PIN(
+ 79, "GPIO79",
+ MTK_EINT_FUNCTION(0, 79),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO79"),
+ MTK_FUNCTION(1, "BPI_PA_VM1"),
+ MTK_FUNCTION(2, "MIPI4_SDATA")
+ ),
+ MTK_PIN(
+ 80, "GPIO80",
+ MTK_EINT_FUNCTION(0, 80),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO80"),
+ MTK_FUNCTION(1, "BPI_PA_VM0"),
+ MTK_FUNCTION(2, "MIPI4_SCLK")
+ ),
+ MTK_PIN(
+ 81, "GPIO81",
+ MTK_EINT_FUNCTION(0, 81),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO81"),
+ MTK_FUNCTION(1, "SDA1"),
+ MTK_FUNCTION(7, "DBG_MON_B0")
+ ),
+ MTK_PIN(
+ 82, "GPIO82",
+ MTK_EINT_FUNCTION(0, 82),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO82"),
+ MTK_FUNCTION(1, "SDA0"),
+ MTK_FUNCTION(7, "DBG_MON_B1")
+ ),
+ MTK_PIN(
+ 83, "GPIO83",
+ MTK_EINT_FUNCTION(0, 83),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO83"),
+ MTK_FUNCTION(1, "SCL0"),
+ MTK_FUNCTION(7, "DBG_MON_B2")
+ ),
+ MTK_PIN(
+ 84, "GPIO84",
+ MTK_EINT_FUNCTION(0, 84),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO84"),
+ MTK_FUNCTION(1, "SCL1"),
+ MTK_FUNCTION(7, "DBG_MON_B3")
+ ),
+ MTK_PIN(
+ 85, "GPIO85",
+ MTK_EINT_FUNCTION(0, 85),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO85"),
+ MTK_FUNCTION(1, "RFIC0_BSI_EN")
+ ),
+ MTK_PIN(
+ 86, "GPIO86",
+ MTK_EINT_FUNCTION(0, 86),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO86"),
+ MTK_FUNCTION(1, "RFIC0_BSI_CK")
+ ),
+ MTK_PIN(
+ 87, "GPIO87",
+ MTK_EINT_FUNCTION(0, 87),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO87"),
+ MTK_FUNCTION(2, "MD_INT1_C2K_UIM0_HOT_PLUG"),
+ MTK_FUNCTION(3, "CMVREF0"),
+ MTK_FUNCTION(4, "MD_URXD0"),
+ MTK_FUNCTION(5, "AGPS_SYNC"),
+ MTK_FUNCTION(6, "EXT_FRAME_SYNC")
+ ),
+ MTK_PIN(
+ 88, "GPIO88",
+ MTK_EINT_FUNCTION(0, 88),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO88"),
+ MTK_FUNCTION(1, "CMMCLK3"),
+ MTK_FUNCTION(2, "MD_INT2_C2K_UIM1_HOT_PLUG"),
+ MTK_FUNCTION(3, "CMVREF1"),
+ MTK_FUNCTION(4, "MD_UTXD0"),
+ MTK_FUNCTION(5, "AGPS_SYNC"),
+ MTK_FUNCTION(6, "DVFSRC_EXT_REQ")
+ ),
+ MTK_PIN(
+ 89, "GPIO89",
+ MTK_EINT_FUNCTION(0, 89),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO89"),
+ MTK_FUNCTION(1, "SRCLKENAI0"),
+ MTK_FUNCTION(2, "PWM2"),
+ MTK_FUNCTION(3, "MD_INT0"),
+ MTK_FUNCTION(4, "USB_DRVVBUS"),
+ MTK_FUNCTION(5, "SCL_6306"),
+ MTK_FUNCTION(6, "TP_GPIO4_AO"),
+ MTK_FUNCTION(7, "DBG_MON_B21")
+ ),
+ MTK_PIN(
+ 90, "GPIO90",
+ MTK_EINT_FUNCTION(0, 90),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO90"),
+ MTK_FUNCTION(1, "URXD1"),
+ MTK_FUNCTION(2, "PWM0"),
+ MTK_FUNCTION(3, "MD_INT2_C2K_UIM1_HOT_PLUG"),
+ MTK_FUNCTION(4, "ANT_SEL4"),
+ MTK_FUNCTION(5, "USB_DRVVBUS"),
+ MTK_FUNCTION(6, "I2S2_BCK"),
+ MTK_FUNCTION(7, "DBG_MON_B4")
+ ),
+ MTK_PIN(
+ 91, "GPIO91",
+ MTK_EINT_FUNCTION(0, 91),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO91"),
+ MTK_FUNCTION(1, "KPROW1"),
+ MTK_FUNCTION(2, "PWM2"),
+ MTK_FUNCTION(3, "MD_INT0"),
+ MTK_FUNCTION(4, "ANT_SEL5"),
+ MTK_FUNCTION(5, "IDDIG"),
+ MTK_FUNCTION(6, "I2S2_LRCK"),
+ MTK_FUNCTION(7, "DBG_MON_B5")
+ ),
+ MTK_PIN(
+ 92, "GPIO92",
+ MTK_EINT_FUNCTION(0, 92),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO92"),
+ MTK_FUNCTION(1, "KPROW0"),
+ MTK_FUNCTION(5, "DVFSRC_EXT_REQ"),
+ MTK_FUNCTION(6, "I2S2_DI"),
+ MTK_FUNCTION(7, "DBG_MON_B6")
+ ),
+ MTK_PIN(
+ 93, "GPIO93",
+ MTK_EINT_FUNCTION(0, 93),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO93"),
+ MTK_FUNCTION(1, "KPCOL0"),
+ MTK_FUNCTION(7, "DBG_MON_B7")
+ ),
+ MTK_PIN(
+ 94, "GPIO94",
+ MTK_EINT_FUNCTION(0, 94),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO94"),
+ MTK_FUNCTION(1, "KPCOL1"),
+ MTK_FUNCTION(5, "CMFLASH"),
+ MTK_FUNCTION(6, "CMVREF0"),
+ MTK_FUNCTION(7, "DBG_MON_B8")
+ ),
+ MTK_PIN(
+ 95, "GPIO95",
+ MTK_EINT_FUNCTION(0, 95),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO95"),
+ MTK_FUNCTION(1, "URXD0"),
+ MTK_FUNCTION(2, "UTXD0"),
+ MTK_FUNCTION(3, "MD_URXD0"),
+ MTK_FUNCTION(4, "PTA_RXD"),
+ MTK_FUNCTION(5, "SSPM_URXD_AO"),
+ MTK_FUNCTION(6, "WIFI_RXD")
+ ),
+ MTK_PIN(
+ 96, "GPIO96",
+ MTK_EINT_FUNCTION(0, 96),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO96"),
+ MTK_FUNCTION(1, "UTXD0"),
+ MTK_FUNCTION(2, "URXD0"),
+ MTK_FUNCTION(3, "MD_UTXD0"),
+ MTK_FUNCTION(4, "PTA_TXD"),
+ MTK_FUNCTION(5, "SSPM_UTXD_AO"),
+ MTK_FUNCTION(6, "WIFI_TXD")
+ ),
+ MTK_PIN(
+ 97, "GPIO97",
+ MTK_EINT_FUNCTION(0, 97),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO97"),
+ MTK_FUNCTION(1, "UCTS0"),
+ MTK_FUNCTION(2, "I2S1_MCK"),
+ MTK_FUNCTION(3, "CONN_MCU_TDO"),
+ MTK_FUNCTION(4, "SPI5_MI"),
+ MTK_FUNCTION(5, "SCL_6306"),
+ MTK_FUNCTION(6, "MCUPM_JTAG_TDO"),
+ MTK_FUNCTION(7, "DBG_MON_B15")
+ ),
+ MTK_PIN(
+ 98, "GPIO98",
+ MTK_EINT_FUNCTION(0, 98),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO98"),
+ MTK_FUNCTION(1, "URTS0"),
+ MTK_FUNCTION(2, "I2S1_BCK"),
+ MTK_FUNCTION(3, "CONN_MCU_TMS"),
+ MTK_FUNCTION(4, "SPI5_CSB"),
+ MTK_FUNCTION(6, "MCUPM_JTAG_TMS"),
+ MTK_FUNCTION(7, "DBG_MON_B16")
+ ),
+ MTK_PIN(
+ 99, "GPIO99",
+ MTK_EINT_FUNCTION(0, 99),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO99"),
+ MTK_FUNCTION(1, "CMMCLK0"),
+ MTK_FUNCTION(4, "AUXIF_CLK"),
+ MTK_FUNCTION(5, "PTA_RXD"),
+ MTK_FUNCTION(6, "CONN_UART0_RXD"),
+ MTK_FUNCTION(7, "DBG_MON_B17")
+ ),
+
+ MTK_PIN(
+ 100, "GPIO100",
+ MTK_EINT_FUNCTION(0, 100),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO100"),
+ MTK_FUNCTION(1, "CMMCLK1"),
+ MTK_FUNCTION(4, "AUXIF_ST"),
+ MTK_FUNCTION(5, "PTA_TXD"),
+ MTK_FUNCTION(6, "CONN_UART0_TXD"),
+ MTK_FUNCTION(7, "DBG_MON_B18")
+ ),
+ MTK_PIN(
+ 101, "GPIO101",
+ MTK_EINT_FUNCTION(0, 101),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO101"),
+ MTK_FUNCTION(1, "CMFLASH"),
+ MTK_FUNCTION(2, "I2S1_LRCK"),
+ MTK_FUNCTION(3, "CONN_MCU_TCK"),
+ MTK_FUNCTION(4, "SPI5_MO"),
+ MTK_FUNCTION(6, "MCUPM_JTAG_TCK"),
+ MTK_FUNCTION(7, "DBG_MON_B19")
+ ),
+ MTK_PIN(
+ 102, "GPIO102",
+ MTK_EINT_FUNCTION(0, 102),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO102"),
+ MTK_FUNCTION(1, "CMVREF0"),
+ MTK_FUNCTION(2, "I2S1_DO"),
+ MTK_FUNCTION(3, "CONN_MCU_TDI"),
+ MTK_FUNCTION(4, "SPI5_CLK"),
+ MTK_FUNCTION(5, "AGPS_SYNC"),
+ MTK_FUNCTION(6, "MCUPM_JTAG_TDI"),
+ MTK_FUNCTION(7, "DBG_MON_B20")
+ ),
+ MTK_PIN(
+ 103, "GPIO103",
+ MTK_EINT_FUNCTION(0, 103),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO103"),
+ MTK_FUNCTION(1, "SCL2"),
+ MTK_FUNCTION(2, "TP_UTXD1_AO"),
+ MTK_FUNCTION(3, "MD_UTXD0"),
+ MTK_FUNCTION(4, "MD_UTXD1"),
+ MTK_FUNCTION(5, "TP_URTS2_AO"),
+ MTK_FUNCTION(6, "WIFI_TXD"),
+ MTK_FUNCTION(7, "DBG_MON_B25")
+ ),
+ MTK_PIN(
+ 104, "GPIO104",
+ MTK_EINT_FUNCTION(0, 104),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO104"),
+ MTK_FUNCTION(1, "SDA2"),
+ MTK_FUNCTION(2, "TP_URXD1_AO"),
+ MTK_FUNCTION(3, "MD_URXD0"),
+ MTK_FUNCTION(4, "MD_URXD1"),
+ MTK_FUNCTION(5, "TP_UCTS2_AO"),
+ MTK_FUNCTION(6, "WIFI_RXD"),
+ MTK_FUNCTION(7, "DBG_MON_B26")
+ ),
+ MTK_PIN(
+ 105, "GPIO105",
+ MTK_EINT_FUNCTION(0, 105),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO105"),
+ MTK_FUNCTION(1, "SCL4"),
+ MTK_FUNCTION(3, "MD_UTXD1"),
+ MTK_FUNCTION(4, "MD_UTXD0"),
+ MTK_FUNCTION(5, "TP_UTXD2_AO"),
+ MTK_FUNCTION(6, "PTA_TXD"),
+ MTK_FUNCTION(7, "DBG_MON_B27")
+ ),
+ MTK_PIN(
+ 106, "GPIO106",
+ MTK_EINT_FUNCTION(0, 106),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO106"),
+ MTK_FUNCTION(1, "SDA4"),
+ MTK_FUNCTION(3, "MD_URXD1"),
+ MTK_FUNCTION(4, "MD_URXD0"),
+ MTK_FUNCTION(5, "TP_URXD2_AO"),
+ MTK_FUNCTION(6, "PTA_RXD"),
+ MTK_FUNCTION(7, "DBG_MON_B28")
+ ),
+ MTK_PIN(
+ 107, "GPIO107",
+ MTK_EINT_FUNCTION(0, 107),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO107"),
+ MTK_FUNCTION(1, "UTXD1"),
+ MTK_FUNCTION(2, "MD_UTXD0"),
+ MTK_FUNCTION(3, "SDA_6306"),
+ MTK_FUNCTION(4, "KPCOL3"),
+ MTK_FUNCTION(5, "CMVREF0"),
+ MTK_FUNCTION(6, "URTS0"),
+ MTK_FUNCTION(7, "DBG_MON_B29")
+ ),
+ MTK_PIN(
+ 108, "GPIO108",
+ MTK_EINT_FUNCTION(0, 108),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO108"),
+ MTK_FUNCTION(1, "CMMCLK2"),
+ MTK_FUNCTION(2, "MD_INT0"),
+ MTK_FUNCTION(3, "CONN_MCU_DBGACK_N"),
+ MTK_FUNCTION(4, "KPCOL4"),
+ MTK_FUNCTION(6, "I2S3_MCK"),
+ MTK_FUNCTION(7, "DBG_MON_B30")
+ ),
+ MTK_PIN(
+ 109, "GPIO109",
+ MTK_EINT_FUNCTION(0, 109),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO109"),
+ MTK_FUNCTION(1, "URXD1"),
+ MTK_FUNCTION(2, "MD_URXD0"),
+ MTK_FUNCTION(3, "ANT_SEL7"),
+ MTK_FUNCTION(4, "KPCOL5"),
+ MTK_FUNCTION(5, "CMVREF1"),
+ MTK_FUNCTION(6, "UCTS0"),
+ MTK_FUNCTION(7, "DBG_MON_B31")
+ ),
+ MTK_PIN(
+ 110, "GPIO110",
+ MTK_EINT_FUNCTION(0, 110),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO110"),
+ MTK_FUNCTION(1, "ANT_SEL0"),
+ MTK_FUNCTION(2, "CLKM0"),
+ MTK_FUNCTION(3, "PWM3"),
+ MTK_FUNCTION(4, "MD_INT0"),
+ MTK_FUNCTION(5, "IDDIG"),
+ MTK_FUNCTION(6, "I2S3_BCK"),
+ MTK_FUNCTION(7, "DBG_MON_B13")
+ ),
+ MTK_PIN(
+ 111, "GPIO111",
+ MTK_EINT_FUNCTION(0, 111),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO111"),
+ MTK_FUNCTION(1, "ANT_SEL1"),
+ MTK_FUNCTION(2, "CLKM1"),
+ MTK_FUNCTION(3, "PWM4"),
+ MTK_FUNCTION(4, "PTA_RXD"),
+ MTK_FUNCTION(5, "CMVREF0"),
+ MTK_FUNCTION(6, "I2S3_LRCK"),
+ MTK_FUNCTION(7, "DBG_MON_B14")
+ ),
+ MTK_PIN(
+ 112, "GPIO112",
+ MTK_EINT_FUNCTION(0, 112),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO112"),
+ MTK_FUNCTION(1, "ANT_SEL2"),
+ MTK_FUNCTION(2, "CLKM2"),
+ MTK_FUNCTION(3, "PWM5"),
+ MTK_FUNCTION(4, "PTA_TXD"),
+ MTK_FUNCTION(5, "CMVREF1"),
+ MTK_FUNCTION(6, "I2S3_DO")
+ ),
+ MTK_PIN(
+ 113, "GPIO113",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO113"),
+ MTK_FUNCTION(1, "CONN_TOP_CLK")
+ ),
+ MTK_PIN(
+ 114, "GPIO114",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO114"),
+ MTK_FUNCTION(1, "CONN_TOP_DATA")
+ ),
+ MTK_PIN(
+ 115, "GPIO115",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO115"),
+ MTK_FUNCTION(1, "CONN_BT_CLK")
+ ),
+ MTK_PIN(
+ 116, "GPIO116",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO116"),
+ MTK_FUNCTION(1, "CONN_BT_DATA")
+ ),
+ MTK_PIN(
+ 117, "GPIO117",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO117"),
+ MTK_FUNCTION(1, "CONN_WF_CTRL0")
+ ),
+ MTK_PIN(
+ 118, "GPIO118",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO118"),
+ MTK_FUNCTION(1, "CONN_WF_CTRL1")
+ ),
+ MTK_PIN(
+ 119, "GPIO119",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO119"),
+ MTK_FUNCTION(1, "CONN_WF_CTRL2")
+ ),
+ MTK_PIN(
+ 120, "GPIO120",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO120"),
+ MTK_FUNCTION(1, "CONN_WB_PTA")
+ ),
+ MTK_PIN(
+ 121, "GPIO121",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO121"),
+ MTK_FUNCTION(1, "CONN_HRST_B")
+ ),
+ MTK_PIN(
+ 122, "GPIO122",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO122"),
+ MTK_FUNCTION(1, "MSDC0_CMD"),
+ MTK_FUNCTION(2, "MSDC0_CMD")
+ ),
+ MTK_PIN(
+ 123, "GPIO123",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO123"),
+ MTK_FUNCTION(1, "MSDC0_DAT0"),
+ MTK_FUNCTION(2, "MSDC0_DAT4")
+ ),
+ MTK_PIN(
+ 124, "GPIO124",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO124"),
+ MTK_FUNCTION(1, "MSDC0_CLK"),
+ MTK_FUNCTION(2, "MSDC0_CLK")
+ ),
+ MTK_PIN(
+ 125, "GPIO125",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO125"),
+ MTK_FUNCTION(1, "MSDC0_DAT2"),
+ MTK_FUNCTION(2, "MSDC0_DAT5")
+ ),
+ MTK_PIN(
+ 126, "GPIO126",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO126"),
+ MTK_FUNCTION(1, "MSDC0_DAT4"),
+ MTK_FUNCTION(2, "MSDC0_DAT2")
+ ),
+ MTK_PIN(
+ 127, "GPIO127",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO127"),
+ MTK_FUNCTION(1, "MSDC0_DAT6"),
+ MTK_FUNCTION(2, "MSDC0_DAT1")
+ ),
+ MTK_PIN(
+ 128, "GPIO128",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO128"),
+ MTK_FUNCTION(1, "MSDC0_DAT1"),
+ MTK_FUNCTION(2, "MSDC0_DAT6")
+ ),
+ MTK_PIN(
+ 129, "GPIO129",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO129"),
+ MTK_FUNCTION(1, "MSDC0_DAT5"),
+ MTK_FUNCTION(2, "MSDC0_DAT0")
+ ),
+ MTK_PIN(
+ 130, "GPIO130",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO130"),
+ MTK_FUNCTION(1, "MSDC0_DAT7"),
+ MTK_FUNCTION(2, "MSDC0_DAT7")
+ ),
+ MTK_PIN(
+ 131, "GPIO131",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO131"),
+ MTK_FUNCTION(1, "MSDC0_DSL"),
+ MTK_FUNCTION(2, "MSDC0_DSL")
+ ),
+ MTK_PIN(
+ 132, "GPIO132",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO132"),
+ MTK_FUNCTION(1, "MSDC0_DAT3"),
+ MTK_FUNCTION(2, "MSDC0_DAT3")
+ ),
+ MTK_PIN(
+ 133, "GPIO133",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO133"),
+ MTK_FUNCTION(1, "MSDC0_RSTB"),
+ MTK_FUNCTION(2, "MSDC0_RSTB")
+ ),
+ MTK_PIN(
+ 134, "GPIO134",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO134"),
+ MTK_FUNCTION(1, "RTC32K_CK")
+ ),
+ MTK_PIN(
+ 135, "GPIO135",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO135"),
+ MTK_FUNCTION(1, "WATCHDOG")
+ ),
+ MTK_PIN(
+ 136, "GPIO136",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO136"),
+ MTK_FUNCTION(1, "AUD_CLK_MOSI"),
+ MTK_FUNCTION(2, "AUD_CLK_MISO"),
+ MTK_FUNCTION(3, "I2S1_MCK")
+ ),
+ MTK_PIN(
+ 137, "GPIO137",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO137"),
+ MTK_FUNCTION(1, "AUD_SYNC_MOSI"),
+ MTK_FUNCTION(2, "AUD_SYNC_MISO"),
+ MTK_FUNCTION(3, "I2S1_BCK")
+ ),
+ MTK_PIN(
+ 138, "GPIO138",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO138"),
+ MTK_FUNCTION(1, "AUD_DAT_MOSI0"),
+ MTK_FUNCTION(2, "AUD_DAT_MISO0"),
+ MTK_FUNCTION(3, "I2S1_LRCK")
+ ),
+ MTK_PIN(
+ 139, "GPIO139",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO139"),
+ MTK_FUNCTION(1, "AUD_DAT_MOSI1"),
+ MTK_FUNCTION(2, "AUD_DAT_MISO1"),
+ MTK_FUNCTION(3, "I2S1_DO")
+ ),
+ MTK_PIN(
+ 140, "GPIO140",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO140"),
+ MTK_FUNCTION(1, "AUD_CLK_MISO"),
+ MTK_FUNCTION(2, "AUD_CLK_MOSI"),
+ MTK_FUNCTION(3, "I2S2_MCK")
+ ),
+ MTK_PIN(
+ 141, "GPIO141",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO141"),
+ MTK_FUNCTION(1, "AUD_SYNC_MISO"),
+ MTK_FUNCTION(2, "AUD_SYNC_MOSI"),
+ MTK_FUNCTION(3, "I2S2_BCK")
+ ),
+ MTK_PIN(
+ 142, "GPIO142",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO142"),
+ MTK_FUNCTION(1, "AUD_DAT_MISO0"),
+ MTK_FUNCTION(2, "AUD_DAT_MOSI0"),
+ MTK_FUNCTION(3, "I2S2_LRCK")
+ ),
+ MTK_PIN(
+ 143, "GPIO143",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO143"),
+ MTK_FUNCTION(1, "AUD_DAT_MISO1"),
+ MTK_FUNCTION(2, "AUD_DAT_MOSI1"),
+ MTK_FUNCTION(3, "I2S2_DI")
+ ),
+ MTK_PIN(
+ 144, "GPIO144",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO144"),
+ MTK_FUNCTION(1, "PWRAP_SPI0_MI"),
+ MTK_FUNCTION(2, "PWRAP_SPI0_MO")
+ ),
+ MTK_PIN(
+ 145, "GPIO145",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO145"),
+ MTK_FUNCTION(1, "PWRAP_SPI0_CSN")
+ ),
+ MTK_PIN(
+ 146, "GPIO146",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO146"),
+ MTK_FUNCTION(1, "PWRAP_SPI0_MO"),
+ MTK_FUNCTION(2, "PWRAP_SPI0_MI")
+ ),
+ MTK_PIN(
+ 147, "GPIO147",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO147"),
+ MTK_FUNCTION(1, "PWRAP_SPI0_CK")
+ ),
+ MTK_PIN(
+ 148, "GPIO148",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO148"),
+ MTK_FUNCTION(1, "SRCLKENA0")
+ ),
+ MTK_PIN(
+ 149, "GPIO149",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO149"),
+ MTK_FUNCTION(1, "SRCLKENA1")
+ ),
+ MTK_PIN(
+ 150, "GPIO150",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO150"),
+ MTK_FUNCTION(1, "PWM0"),
+ MTK_FUNCTION(2, "CMFLASH"),
+ MTK_FUNCTION(3, "ANT_SEL3"),
+ MTK_FUNCTION(5, "MD_URXD0"),
+ MTK_FUNCTION(6, "TP_URXD2_AO")
+ ),
+ MTK_PIN(
+ 151, "GPIO151",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO151"),
+ MTK_FUNCTION(1, "PWM1"),
+ MTK_FUNCTION(2, "CMVREF0"),
+ MTK_FUNCTION(3, "ANT_SEL4"),
+ MTK_FUNCTION(5, "MD_UTXD0"),
+ MTK_FUNCTION(6, "TP_UTXD2_AO")
+ ),
+ MTK_PIN(
+ 152, "GPIO152",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO152"),
+ MTK_FUNCTION(1, "PWM2"),
+ MTK_FUNCTION(2, "CMVREF1"),
+ MTK_FUNCTION(3, "ANT_SEL5"),
+ MTK_FUNCTION(5, "MD_URXD1"),
+ MTK_FUNCTION(6, "TP_UCTS1_AO")
+ ),
+ MTK_PIN(
+ 153, "GPIO153",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO153"),
+ MTK_FUNCTION(1, "PWM3"),
+ MTK_FUNCTION(2, "CLKM0"),
+ MTK_FUNCTION(3, "ANT_SEL6"),
+ MTK_FUNCTION(5, "MD_UTXD1"),
+ MTK_FUNCTION(6, "TP_URTS1_AO")
+ ),
+ MTK_PIN(
+ 154, "GPIO154",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO154"),
+ MTK_FUNCTION(1, "PWM5"),
+ MTK_FUNCTION(2, "CLKM2"),
+ MTK_FUNCTION(3, "USB_DRVVBUS"),
+ MTK_FUNCTION(5, "PTA_TXD"),
+ MTK_FUNCTION(6, "CONN_UART0_TXD")
+ ),
+ MTK_PIN(
+ 155, "GPIO155",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO155"),
+ MTK_FUNCTION(1, "SPI0_MI"),
+ MTK_FUNCTION(2, "IDDIG"),
+ MTK_FUNCTION(3, "AGPS_SYNC"),
+ MTK_FUNCTION(4, "TP_GPIO0_AO"),
+ MTK_FUNCTION(5, "MFG_JTAG_TDO"),
+ MTK_FUNCTION(6, "DFD_TDO"),
+ MTK_FUNCTION(7, "JTDO_SEL1")
+ ),
+ MTK_PIN(
+ 156, "GPIO156",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO156"),
+ MTK_FUNCTION(1, "SPI0_CSB"),
+ MTK_FUNCTION(2, "USB_DRVVBUS"),
+ MTK_FUNCTION(3, "DVFSRC_EXT_REQ"),
+ MTK_FUNCTION(4, "TP_GPIO1_AO"),
+ MTK_FUNCTION(5, "MFG_JTAG_TMS"),
+ MTK_FUNCTION(6, "DFD_TMS"),
+ MTK_FUNCTION(7, "JTMS_SEL1")
+ ),
+ MTK_PIN(
+ 157, "GPIO157",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO157"),
+ MTK_FUNCTION(1, "SPI0_MO"),
+ MTK_FUNCTION(2, "MD_INT1_C2K_UIM0_HOT_PLUG"),
+ MTK_FUNCTION(3, "CLKM0"),
+ MTK_FUNCTION(4, "TP_GPIO2_AO"),
+ MTK_FUNCTION(5, "MFG_JTAG_TDI"),
+ MTK_FUNCTION(6, "DFD_TDI"),
+ MTK_FUNCTION(7, "JTDI_SEL1")
+ ),
+ MTK_PIN(
+ 158, "GPIO158",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO158"),
+ MTK_FUNCTION(1, "SPI0_CLK"),
+ MTK_FUNCTION(2, "MD_INT2_C2K_UIM1_HOT_PLUG"),
+ MTK_FUNCTION(3, "EXT_FRAME_SYNC"),
+ MTK_FUNCTION(4, "TP_GPIO3_AO"),
+ MTK_FUNCTION(5, "MFG_JTAG_TCK"),
+ MTK_FUNCTION(6, "DFD_TCK_XI"),
+ MTK_FUNCTION(7, "JTCK_SEL1")
+ ),
+ MTK_PIN(
+ 159, "GPIO159",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO159"),
+ MTK_FUNCTION(1, "PWM4"),
+ MTK_FUNCTION(2, "CLKM1"),
+ MTK_FUNCTION(3, "ANT_SEL7"),
+ MTK_FUNCTION(5, "PTA_RXD"),
+ MTK_FUNCTION(6, "CONN_UART0_RXD")
+ ),
+ MTK_PIN(
+ 160, "GPIO160",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO160"),
+ MTK_FUNCTION(1, "CLKM0"),
+ MTK_FUNCTION(2, "PWM2"),
+ MTK_FUNCTION(3, "EXT_FRAME_SYNC"),
+ MTK_FUNCTION(4, "TP_GPIO5_AO"),
+ MTK_FUNCTION(5, "AGPS_SYNC"),
+ MTK_FUNCTION(6, "DVFSRC_EXT_REQ")
+ ),
+ MTK_PIN(
+ 161, "GPIO161",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO161"),
+ MTK_FUNCTION(1, "SCL6"),
+ MTK_FUNCTION(2, "SCL_6306"),
+ MTK_FUNCTION(3, "TP_GPIO6_AO"),
+ MTK_FUNCTION(4, "KPCOL6"),
+ MTK_FUNCTION(5, "PTA_RXD"),
+ MTK_FUNCTION(6, "CONN_UART0_RXD")
+ ),
+ MTK_PIN(
+ 162, "GPIO162",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO162"),
+ MTK_FUNCTION(1, "SDA6"),
+ MTK_FUNCTION(2, "SDA_6306"),
+ MTK_FUNCTION(3, "TP_GPIO7_AO"),
+ MTK_FUNCTION(4, "KPCOL7"),
+ MTK_FUNCTION(5, "PTA_TXD"),
+ MTK_FUNCTION(6, "CONN_UART0_TXD")
+ ),
+ MTK_PIN(
+ 163, "GPIO163",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO163")
+ ),
+ MTK_PIN(
+ 164, "GPIO164",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO164")
+ ),
+ MTK_PIN(
+ 165, "GPIO165",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO165")
+ ),
+ MTK_PIN(
+ 166, "GPIO166",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO166")
+ ),
+ MTK_PIN(
+ 167, "GPIO167",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO167")
+ ),
+ MTK_PIN(
+ 168, "GPIO168",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO168")
+ ),
+ MTK_PIN(
+ 169, "GPIO169",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO169")
+ ),
+ MTK_PIN(
+ 170, "GPIO170",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO170")
+ ),
+ MTK_PIN(
+ 171, "GPIO171",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO171")
+ ),
+ MTK_PIN(
+ 172, "GPIO172",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO172")
+ ),
+ MTK_PIN(
+ 173, "GPIO173",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO173")
+ ),
+ MTK_PIN(
+ 174, "GPIO174",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO174")
+ ),
+ MTK_PIN(
+ 175, "GPIO175",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO175")
+ ),
+ MTK_PIN(
+ 176, "GPIO176",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO176")
+ ),
+ MTK_PIN(
+ 177, "GPIO177",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO177")
+ ),
+ MTK_PIN(
+ 178, "GPIO178",
+ MTK_EINT_FUNCTION(0, NO_EINT_SUPPORT),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO178")
+ ),
+ MTK_PIN(
+ 179, "GPIO179",
+ MTK_EINT_FUNCTION(0, 151),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO179")
+ ),
+};
+
+#endif /* __PINCTRL_MTK_MT6765_H */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 MediaTek Inc.
+ *
+ * Author: Zhiyong Tao <zhiyong.tao@mediatek.com>
+ *
+ */
+
+#ifndef __PINCTRL_MTK_MT8183_H
+#define __PINCTRL_MTK_MT8183_H
+
+#include "pinctrl-paris.h"
+
+static struct mtk_pin_desc mtk_pins_mt8183[] = {
+ MTK_PIN(
+ 0, "GPIO0",
+ MTK_EINT_FUNCTION(0, 0),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO0"),
+ MTK_FUNCTION(1, "MRG_SYNC"),
+ MTK_FUNCTION(2, "PCM0_SYNC"),
+ MTK_FUNCTION(3, "TP_GPIO0_AO"),
+ MTK_FUNCTION(4, "SRCLKENAI0"),
+ MTK_FUNCTION(5, "SCP_SPI2_CS"),
+ MTK_FUNCTION(6, "I2S3_MCK"),
+ MTK_FUNCTION(7, "SPI2_CSB")
+ ),
+ MTK_PIN(
+ 1, "GPIO1",
+ MTK_EINT_FUNCTION(0, 1),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO1"),
+ MTK_FUNCTION(1, "MRG_CLK"),
+ MTK_FUNCTION(2, "PCM0_CLK"),
+ MTK_FUNCTION(3, "TP_GPIO1_AO"),
+ MTK_FUNCTION(4, "CLKM3"),
+ MTK_FUNCTION(5, "SCP_SPI2_MO"),
+ MTK_FUNCTION(6, "I2S3_BCK"),
+ MTK_FUNCTION(7, "SPI2_MO")
+ ),
+ MTK_PIN(
+ 2, "GPIO2",
+ MTK_EINT_FUNCTION(0, 2),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO2"),
+ MTK_FUNCTION(1, "MRG_DO"),
+ MTK_FUNCTION(2, "PCM0_DO"),
+ MTK_FUNCTION(3, "TP_GPIO2_AO"),
+ MTK_FUNCTION(4, "SCL6"),
+ MTK_FUNCTION(5, "SCP_SPI2_CK"),
+ MTK_FUNCTION(6, "I2S3_LRCK"),
+ MTK_FUNCTION(7, "SPI2_CLK")
+ ),
+ MTK_PIN(
+ 3, "GPIO3",
+ MTK_EINT_FUNCTION(0, 3),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO3"),
+ MTK_FUNCTION(1, "MRG_DI"),
+ MTK_FUNCTION(2, "PCM0_DI"),
+ MTK_FUNCTION(3, "TP_GPIO3_AO"),
+ MTK_FUNCTION(4, "SDA6"),
+ MTK_FUNCTION(5, "TDM_MCK"),
+ MTK_FUNCTION(6, "I2S3_DO"),
+ MTK_FUNCTION(7, "SCP_VREQ_VAO")
+ ),
+ MTK_PIN(
+ 4, "GPIO4",
+ MTK_EINT_FUNCTION(0, 4),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO4"),
+ MTK_FUNCTION(1, "PWM_B"),
+ MTK_FUNCTION(2, "I2S0_MCK"),
+ MTK_FUNCTION(3, "SSPM_UTXD_AO"),
+ MTK_FUNCTION(4, "MD_URXD1"),
+ MTK_FUNCTION(5, "TDM_BCK"),
+ MTK_FUNCTION(6, "TP_GPIO4_AO"),
+ MTK_FUNCTION(7, "DAP_MD32_SWD")
+ ),
+ MTK_PIN(
+ 5, "GPIO5",
+ MTK_EINT_FUNCTION(0, 5),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO5"),
+ MTK_FUNCTION(1, "PWM_C"),
+ MTK_FUNCTION(2, "I2S0_BCK"),
+ MTK_FUNCTION(3, "SSPM_URXD_AO"),
+ MTK_FUNCTION(4, "MD_UTXD1"),
+ MTK_FUNCTION(5, "TDM_LRCK"),
+ MTK_FUNCTION(6, "TP_GPIO5_AO"),
+ MTK_FUNCTION(7, "DAP_MD32_SWCK")
+ ),
+ MTK_PIN(
+ 6, "GPIO6",
+ MTK_EINT_FUNCTION(0, 6),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO6"),
+ MTK_FUNCTION(1, "PWM_A"),
+ MTK_FUNCTION(2, "I2S0_LRCK"),
+ MTK_FUNCTION(3, "IDDIG"),
+ MTK_FUNCTION(4, "MD_URXD0"),
+ MTK_FUNCTION(5, "TDM_DATA0"),
+ MTK_FUNCTION(6, "TP_GPIO6_AO"),
+ MTK_FUNCTION(7, "CMFLASH")
+ ),
+ MTK_PIN(
+ 7, "GPIO7",
+ MTK_EINT_FUNCTION(0, 7),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO7"),
+ MTK_FUNCTION(1, "SPI1_B_MI"),
+ MTK_FUNCTION(2, "I2S0_DI"),
+ MTK_FUNCTION(3, "USB_DRVVBUS"),
+ MTK_FUNCTION(4, "MD_UTXD0"),
+ MTK_FUNCTION(5, "TDM_DATA1"),
+ MTK_FUNCTION(6, "TP_GPIO7_AO"),
+ MTK_FUNCTION(7, "DVFSRC_EXT_REQ")
+ ),
+ MTK_PIN(
+ 8, "GPIO8",
+ MTK_EINT_FUNCTION(0, 8),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO8"),
+ MTK_FUNCTION(1, "SPI1_B_CSB"),
+ MTK_FUNCTION(2, "ANT_SEL3"),
+ MTK_FUNCTION(3, "SCL7"),
+ MTK_FUNCTION(4, "CONN_MCU_TRST_B"),
+ MTK_FUNCTION(5, "TDM_DATA2"),
+ MTK_FUNCTION(6, "MD_INT0"),
+ MTK_FUNCTION(7, "JTRSTN_SEL1")
+ ),
+ MTK_PIN(
+ 9, "GPIO9",
+ MTK_EINT_FUNCTION(0, 9),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO9"),
+ MTK_FUNCTION(1, "SPI1_B_MO"),
+ MTK_FUNCTION(2, "ANT_SEL4"),
+ MTK_FUNCTION(3, "CMMCLK2"),
+ MTK_FUNCTION(4, "CONN_MCU_DBGACK_N"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TRSTN"),
+ MTK_FUNCTION(6, "IO_JTAG_TRSTN"),
+ MTK_FUNCTION(7, "DBG_MON_B10")
+ ),
+ MTK_PIN(
+ 10, "GPIO10",
+ MTK_EINT_FUNCTION(0, 10),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO10"),
+ MTK_FUNCTION(1, "SPI1_B_CLK"),
+ MTK_FUNCTION(2, "ANT_SEL5"),
+ MTK_FUNCTION(3, "CMMCLK3"),
+ MTK_FUNCTION(4, "CONN_MCU_DBGI_N"),
+ MTK_FUNCTION(5, "TDM_DATA3"),
+ MTK_FUNCTION(6, "EXT_FRAME_SYNC"),
+ MTK_FUNCTION(7, "DBG_MON_B11")
+ ),
+ MTK_PIN(
+ 11, "GPIO11",
+ MTK_EINT_FUNCTION(0, 11),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO11"),
+ MTK_FUNCTION(1, "TP_URXD1_AO"),
+ MTK_FUNCTION(2, "IDDIG"),
+ MTK_FUNCTION(3, "SCL6"),
+ MTK_FUNCTION(4, "UCTS1"),
+ MTK_FUNCTION(5, "UCTS0"),
+ MTK_FUNCTION(6, "SRCLKENAI1"),
+ MTK_FUNCTION(7, "I2S5_MCK")
+ ),
+ MTK_PIN(
+ 12, "GPIO12",
+ MTK_EINT_FUNCTION(0, 12),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO12"),
+ MTK_FUNCTION(1, "TP_UTXD1_AO"),
+ MTK_FUNCTION(2, "USB_DRVVBUS"),
+ MTK_FUNCTION(3, "SDA6"),
+ MTK_FUNCTION(4, "URTS1"),
+ MTK_FUNCTION(5, "URTS0"),
+ MTK_FUNCTION(6, "I2S2_DI2"),
+ MTK_FUNCTION(7, "I2S5_BCK")
+ ),
+ MTK_PIN(
+ 13, "GPIO13",
+ MTK_EINT_FUNCTION(0, 13),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO13"),
+ MTK_FUNCTION(1, "DBPI_D0"),
+ MTK_FUNCTION(2, "SPI5_MI"),
+ MTK_FUNCTION(3, "PCM0_SYNC"),
+ MTK_FUNCTION(4, "MD_URXD0"),
+ MTK_FUNCTION(5, "ANT_SEL3"),
+ MTK_FUNCTION(6, "I2S0_MCK"),
+ MTK_FUNCTION(7, "DBG_MON_B15")
+ ),
+ MTK_PIN(
+ 14, "GPIO14",
+ MTK_EINT_FUNCTION(0, 14),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO14"),
+ MTK_FUNCTION(1, "DBPI_D1"),
+ MTK_FUNCTION(2, "SPI5_CSB"),
+ MTK_FUNCTION(3, "PCM0_CLK"),
+ MTK_FUNCTION(4, "MD_UTXD0"),
+ MTK_FUNCTION(5, "ANT_SEL4"),
+ MTK_FUNCTION(6, "I2S0_BCK"),
+ MTK_FUNCTION(7, "DBG_MON_B16")
+ ),
+ MTK_PIN(
+ 15, "GPIO15",
+ MTK_EINT_FUNCTION(0, 15),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO15"),
+ MTK_FUNCTION(1, "DBPI_D2"),
+ MTK_FUNCTION(2, "SPI5_MO"),
+ MTK_FUNCTION(3, "PCM0_DO"),
+ MTK_FUNCTION(4, "MD_URXD1"),
+ MTK_FUNCTION(5, "ANT_SEL5"),
+ MTK_FUNCTION(6, "I2S0_LRCK"),
+ MTK_FUNCTION(7, "DBG_MON_B17")
+ ),
+ MTK_PIN(
+ 16, "GPIO16",
+ MTK_EINT_FUNCTION(0, 16),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO16"),
+ MTK_FUNCTION(1, "DBPI_D3"),
+ MTK_FUNCTION(2, "SPI5_CLK"),
+ MTK_FUNCTION(3, "PCM0_DI"),
+ MTK_FUNCTION(4, "MD_UTXD1"),
+ MTK_FUNCTION(5, "ANT_SEL6"),
+ MTK_FUNCTION(6, "I2S0_DI"),
+ MTK_FUNCTION(7, "DBG_MON_B23")
+ ),
+ MTK_PIN(
+ 17, "GPIO17",
+ MTK_EINT_FUNCTION(0, 17),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO17"),
+ MTK_FUNCTION(1, "DBPI_D4"),
+ MTK_FUNCTION(2, "SPI4_MI"),
+ MTK_FUNCTION(3, "CONN_MCU_TRST_B"),
+ MTK_FUNCTION(4, "MD_INT0"),
+ MTK_FUNCTION(5, "ANT_SEL7"),
+ MTK_FUNCTION(6, "I2S3_MCK"),
+ MTK_FUNCTION(7, "DBG_MON_A1")
+ ),
+ MTK_PIN(
+ 18, "GPIO18",
+ MTK_EINT_FUNCTION(0, 18),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO18"),
+ MTK_FUNCTION(1, "DBPI_D5"),
+ MTK_FUNCTION(2, "SPI4_CSB"),
+ MTK_FUNCTION(3, "CONN_MCU_DBGI_N"),
+ MTK_FUNCTION(4, "MD_INT0"),
+ MTK_FUNCTION(5, "SCP_VREQ_VAO"),
+ MTK_FUNCTION(6, "I2S3_BCK"),
+ MTK_FUNCTION(7, "DBG_MON_A2")
+ ),
+ MTK_PIN(
+ 19, "GPIO19",
+ MTK_EINT_FUNCTION(0, 19),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO19"),
+ MTK_FUNCTION(1, "DBPI_D6"),
+ MTK_FUNCTION(2, "SPI4_MO"),
+ MTK_FUNCTION(3, "CONN_MCU_TDO"),
+ MTK_FUNCTION(4, "MD_INT2_C2K_UIM1_HOT_PLUG"),
+ MTK_FUNCTION(5, "URXD1"),
+ MTK_FUNCTION(6, "I2S3_LRCK"),
+ MTK_FUNCTION(7, "DBG_MON_A3")
+ ),
+ MTK_PIN(
+ 20, "GPIO20",
+ MTK_EINT_FUNCTION(0, 20),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO20"),
+ MTK_FUNCTION(1, "DBPI_D7"),
+ MTK_FUNCTION(2, "SPI4_CLK"),
+ MTK_FUNCTION(3, "CONN_MCU_DBGACK_N"),
+ MTK_FUNCTION(4, "MD_INT1_C2K_UIM0_HOT_PLUG"),
+ MTK_FUNCTION(5, "UTXD1"),
+ MTK_FUNCTION(6, "I2S3_DO"),
+ MTK_FUNCTION(7, "DBG_MON_A19")
+ ),
+ MTK_PIN(
+ 21, "GPIO21",
+ MTK_EINT_FUNCTION(0, 21),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO21"),
+ MTK_FUNCTION(1, "DBPI_D8"),
+ MTK_FUNCTION(2, "SPI3_MI"),
+ MTK_FUNCTION(3, "CONN_MCU_TMS"),
+ MTK_FUNCTION(4, "DAP_MD32_SWD"),
+ MTK_FUNCTION(5, "CONN_MCU_AICE_TMSC"),
+ MTK_FUNCTION(6, "I2S2_MCK"),
+ MTK_FUNCTION(7, "DBG_MON_B5")
+ ),
+ MTK_PIN(
+ 22, "GPIO22",
+ MTK_EINT_FUNCTION(0, 22),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO22"),
+ MTK_FUNCTION(1, "DBPI_D9"),
+ MTK_FUNCTION(2, "SPI3_CSB"),
+ MTK_FUNCTION(3, "CONN_MCU_TCK"),
+ MTK_FUNCTION(4, "DAP_MD32_SWCK"),
+ MTK_FUNCTION(5, "CONN_MCU_AICE_TCKC"),
+ MTK_FUNCTION(6, "I2S2_BCK"),
+ MTK_FUNCTION(7, "DBG_MON_B6")
+ ),
+ MTK_PIN(
+ 23, "GPIO23",
+ MTK_EINT_FUNCTION(0, 23),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO23"),
+ MTK_FUNCTION(1, "DBPI_D10"),
+ MTK_FUNCTION(2, "SPI3_MO"),
+ MTK_FUNCTION(3, "CONN_MCU_TDI"),
+ MTK_FUNCTION(4, "UCTS1"),
+ MTK_FUNCTION(5, "EXT_FRAME_SYNC"),
+ MTK_FUNCTION(6, "I2S2_LRCK"),
+ MTK_FUNCTION(7, "DBG_MON_B7")
+ ),
+ MTK_PIN(
+ 24, "GPIO24",
+ MTK_EINT_FUNCTION(0, 24),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO24"),
+ MTK_FUNCTION(1, "DBPI_D11"),
+ MTK_FUNCTION(2, "SPI3_CLK"),
+ MTK_FUNCTION(3, "SRCLKENAI0"),
+ MTK_FUNCTION(4, "URTS1"),
+ MTK_FUNCTION(5, "IO_JTAG_TCK"),
+ MTK_FUNCTION(6, "I2S2_DI"),
+ MTK_FUNCTION(7, "DBG_MON_B31")
+ ),
+ MTK_PIN(
+ 25, "GPIO25",
+ MTK_EINT_FUNCTION(0, 25),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO25"),
+ MTK_FUNCTION(1, "DBPI_HSYNC"),
+ MTK_FUNCTION(2, "ANT_SEL0"),
+ MTK_FUNCTION(3, "SCL6"),
+ MTK_FUNCTION(4, "KPCOL2"),
+ MTK_FUNCTION(5, "IO_JTAG_TMS"),
+ MTK_FUNCTION(6, "I2S1_MCK"),
+ MTK_FUNCTION(7, "DBG_MON_B0")
+ ),
+ MTK_PIN(
+ 26, "GPIO26",
+ MTK_EINT_FUNCTION(0, 26),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO26"),
+ MTK_FUNCTION(1, "DBPI_VSYNC"),
+ MTK_FUNCTION(2, "ANT_SEL1"),
+ MTK_FUNCTION(3, "SDA6"),
+ MTK_FUNCTION(4, "KPROW2"),
+ MTK_FUNCTION(5, "IO_JTAG_TDI"),
+ MTK_FUNCTION(6, "I2S1_BCK"),
+ MTK_FUNCTION(7, "DBG_MON_B1")
+ ),
+ MTK_PIN(
+ 27, "GPIO27",
+ MTK_EINT_FUNCTION(0, 27),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO27"),
+ MTK_FUNCTION(1, "DBPI_DE"),
+ MTK_FUNCTION(2, "ANT_SEL2"),
+ MTK_FUNCTION(3, "SCL7"),
+ MTK_FUNCTION(4, "DMIC_CLK"),
+ MTK_FUNCTION(5, "IO_JTAG_TDO"),
+ MTK_FUNCTION(6, "I2S1_LRCK"),
+ MTK_FUNCTION(7, "DBG_MON_B9")
+ ),
+ MTK_PIN(
+ 28, "GPIO28",
+ MTK_EINT_FUNCTION(0, 28),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO28"),
+ MTK_FUNCTION(1, "DBPI_CK"),
+ MTK_FUNCTION(2, "DVFSRC_EXT_REQ"),
+ MTK_FUNCTION(3, "SDA7"),
+ MTK_FUNCTION(4, "DMIC_DAT"),
+ MTK_FUNCTION(5, "IO_JTAG_TRSTN"),
+ MTK_FUNCTION(6, "I2S1_DO"),
+ MTK_FUNCTION(7, "DBG_MON_B32")
+ ),
+ MTK_PIN(
+ 29, "GPIO29",
+ MTK_EINT_FUNCTION(0, 29),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO29"),
+ MTK_FUNCTION(1, "MSDC1_CLK"),
+ MTK_FUNCTION(2, "IO_JTAG_TCK"),
+ MTK_FUNCTION(3, "UDI_TCK"),
+ MTK_FUNCTION(4, "CONN_DSP_JCK"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TCK"),
+ MTK_FUNCTION(6, "PCM1_CLK"),
+ MTK_FUNCTION(7, "DBG_MON_A6")
+ ),
+ MTK_PIN(
+ 30, "GPIO30",
+ MTK_EINT_FUNCTION(0, 30),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO30"),
+ MTK_FUNCTION(1, "MSDC1_DAT3"),
+ MTK_FUNCTION(2, "DAP_MD32_SWD"),
+ MTK_FUNCTION(3, "CONN_MCU_AICE_TMSC"),
+ MTK_FUNCTION(4, "CONN_DSP_JINTP"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TRSTN"),
+ MTK_FUNCTION(6, "PCM1_DI"),
+ MTK_FUNCTION(7, "DBG_MON_A7")
+ ),
+ MTK_PIN(
+ 31, "GPIO31",
+ MTK_EINT_FUNCTION(0, 31),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO31"),
+ MTK_FUNCTION(1, "MSDC1_CMD"),
+ MTK_FUNCTION(2, "IO_JTAG_TMS"),
+ MTK_FUNCTION(3, "UDI_TMS"),
+ MTK_FUNCTION(4, "CONN_DSP_JMS"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TMS"),
+ MTK_FUNCTION(6, "PCM1_SYNC"),
+ MTK_FUNCTION(7, "DBG_MON_A8")
+ ),
+ MTK_PIN(
+ 32, "GPIO32",
+ MTK_EINT_FUNCTION(0, 32),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO32"),
+ MTK_FUNCTION(1, "MSDC1_DAT0"),
+ MTK_FUNCTION(2, "IO_JTAG_TDI"),
+ MTK_FUNCTION(3, "UDI_TDI"),
+ MTK_FUNCTION(4, "CONN_DSP_JDI"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TDI"),
+ MTK_FUNCTION(6, "PCM1_DO0"),
+ MTK_FUNCTION(7, "DBG_MON_A9")
+ ),
+ MTK_PIN(
+ 33, "GPIO33",
+ MTK_EINT_FUNCTION(0, 33),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO33"),
+ MTK_FUNCTION(1, "MSDC1_DAT2"),
+ MTK_FUNCTION(2, "IO_JTAG_TRSTN"),
+ MTK_FUNCTION(3, "UDI_NTRST"),
+ MTK_FUNCTION(4, "DAP_MD32_SWCK"),
+ MTK_FUNCTION(5, "CONN_MCU_AICE_TCKC"),
+ MTK_FUNCTION(6, "PCM1_DO2"),
+ MTK_FUNCTION(7, "DBG_MON_A10")
+ ),
+ MTK_PIN(
+ 34, "GPIO34",
+ MTK_EINT_FUNCTION(0, 34),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO34"),
+ MTK_FUNCTION(1, "MSDC1_DAT1"),
+ MTK_FUNCTION(2, "IO_JTAG_TDO"),
+ MTK_FUNCTION(3, "UDI_TDO"),
+ MTK_FUNCTION(4, "CONN_DSP_JDO"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TDO"),
+ MTK_FUNCTION(6, "PCM1_DO1"),
+ MTK_FUNCTION(7, "DBG_MON_A11")
+ ),
+ MTK_PIN(
+ 35, "GPIO35",
+ MTK_EINT_FUNCTION(0, 35),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO35"),
+ MTK_FUNCTION(1, "MD1_SIM2_SIO"),
+ MTK_FUNCTION(2, "CCU_JTAG_TDO"),
+ MTK_FUNCTION(3, "MD1_SIM1_SIO"),
+ MTK_FUNCTION(5, "SCP_JTAG_TDO"),
+ MTK_FUNCTION(6, "CONN_DSP_JMS"),
+ MTK_FUNCTION(7, "DBG_MON_A28")
+ ),
+ MTK_PIN(
+ 36, "GPIO36",
+ MTK_EINT_FUNCTION(0, 36),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO36"),
+ MTK_FUNCTION(1, "MD1_SIM2_SRST"),
+ MTK_FUNCTION(2, "CCU_JTAG_TMS"),
+ MTK_FUNCTION(3, "MD1_SIM1_SRST"),
+ MTK_FUNCTION(4, "CONN_MCU_AICE_TMSC"),
+ MTK_FUNCTION(5, "SCP_JTAG_TMS"),
+ MTK_FUNCTION(6, "CONN_DSP_JINTP"),
+ MTK_FUNCTION(7, "DBG_MON_A29")
+ ),
+ MTK_PIN(
+ 37, "GPIO37",
+ MTK_EINT_FUNCTION(0, 37),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO37"),
+ MTK_FUNCTION(1, "MD1_SIM2_SCLK"),
+ MTK_FUNCTION(2, "CCU_JTAG_TDI"),
+ MTK_FUNCTION(3, "MD1_SIM1_SCLK"),
+ MTK_FUNCTION(5, "SCP_JTAG_TDI"),
+ MTK_FUNCTION(6, "CONN_DSP_JDO"),
+ MTK_FUNCTION(7, "DBG_MON_A30")
+ ),
+ MTK_PIN(
+ 38, "GPIO38",
+ MTK_EINT_FUNCTION(0, 38),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO38"),
+ MTK_FUNCTION(1, "MD1_SIM1_SCLK"),
+ MTK_FUNCTION(3, "MD1_SIM2_SCLK"),
+ MTK_FUNCTION(4, "CONN_MCU_AICE_TCKC"),
+ MTK_FUNCTION(7, "DBG_MON_A20")
+ ),
+ MTK_PIN(
+ 39, "GPIO39",
+ MTK_EINT_FUNCTION(0, 39),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO39"),
+ MTK_FUNCTION(1, "MD1_SIM1_SRST"),
+ MTK_FUNCTION(2, "CCU_JTAG_TCK"),
+ MTK_FUNCTION(3, "MD1_SIM2_SRST"),
+ MTK_FUNCTION(5, "SCP_JTAG_TCK"),
+ MTK_FUNCTION(6, "CONN_DSP_JCK"),
+ MTK_FUNCTION(7, "DBG_MON_A31")
+ ),
+ MTK_PIN(
+ 40, "GPIO40",
+ MTK_EINT_FUNCTION(0, 40),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO40"),
+ MTK_FUNCTION(1, "MD1_SIM1_SIO"),
+ MTK_FUNCTION(2, "CCU_JTAG_TRST"),
+ MTK_FUNCTION(3, "MD1_SIM2_SIO"),
+ MTK_FUNCTION(5, "SCP_JTAG_TRSTN"),
+ MTK_FUNCTION(6, "CONN_DSP_JDI"),
+ MTK_FUNCTION(7, "DBG_MON_A32")
+ ),
+ MTK_PIN(
+ 41, "GPIO41",
+ MTK_EINT_FUNCTION(0, 41),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO41"),
+ MTK_FUNCTION(1, "IDDIG"),
+ MTK_FUNCTION(2, "URXD1"),
+ MTK_FUNCTION(3, "UCTS0"),
+ MTK_FUNCTION(4, "SSPM_UTXD_AO"),
+ MTK_FUNCTION(5, "EXT_FRAME_SYNC"),
+ MTK_FUNCTION(6, "DMIC_CLK")
+ ),
+ MTK_PIN(
+ 42, "GPIO42",
+ MTK_EINT_FUNCTION(0, 42),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO42"),
+ MTK_FUNCTION(1, "USB_DRVVBUS"),
+ MTK_FUNCTION(2, "UTXD1"),
+ MTK_FUNCTION(3, "URTS0"),
+ MTK_FUNCTION(4, "SSPM_URXD_AO"),
+ MTK_FUNCTION(5, "EXT_FRAME_SYNC"),
+ MTK_FUNCTION(6, "DMIC_DAT")
+ ),
+ MTK_PIN(
+ 43, "GPIO43",
+ MTK_EINT_FUNCTION(0, 43),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO43"),
+ MTK_FUNCTION(1, "DISP_PWM")
+ ),
+ MTK_PIN(
+ 44, "GPIO44",
+ MTK_EINT_FUNCTION(0, 44),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO44"),
+ MTK_FUNCTION(1, "DSI_TE")
+ ),
+ MTK_PIN(
+ 45, "GPIO45",
+ MTK_EINT_FUNCTION(0, 45),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO45"),
+ MTK_FUNCTION(1, "LCM_RST")
+ ),
+ MTK_PIN(
+ 46, "GPIO46",
+ MTK_EINT_FUNCTION(0, 46),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO46"),
+ MTK_FUNCTION(1, "MD_INT2_C2K_UIM1_HOT_PLUG"),
+ MTK_FUNCTION(2, "URXD1"),
+ MTK_FUNCTION(3, "UCTS1"),
+ MTK_FUNCTION(4, "CCU_UTXD_AO"),
+ MTK_FUNCTION(5, "TP_UCTS1_AO"),
+ MTK_FUNCTION(6, "IDDIG"),
+ MTK_FUNCTION(7, "I2S5_LRCK")
+ ),
+ MTK_PIN(
+ 47, "GPIO47",
+ MTK_EINT_FUNCTION(0, 47),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO47"),
+ MTK_FUNCTION(1, "MD_INT1_C2K_UIM0_HOT_PLUG"),
+ MTK_FUNCTION(2, "UTXD1"),
+ MTK_FUNCTION(3, "URTS1"),
+ MTK_FUNCTION(4, "CCU_URXD_AO"),
+ MTK_FUNCTION(5, "TP_URTS1_AO"),
+ MTK_FUNCTION(6, "USB_DRVVBUS"),
+ MTK_FUNCTION(7, "I2S5_DO")
+ ),
+ MTK_PIN(
+ 48, "GPIO48",
+ MTK_EINT_FUNCTION(0, 48),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO48"),
+ MTK_FUNCTION(1, "SCL5")
+ ),
+ MTK_PIN(
+ 49, "GPIO49",
+ MTK_EINT_FUNCTION(0, 49),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO49"),
+ MTK_FUNCTION(1, "SDA5")
+ ),
+ MTK_PIN(
+ 50, "GPIO50",
+ MTK_EINT_FUNCTION(0, 50),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO50"),
+ MTK_FUNCTION(1, "SCL3")
+ ),
+ MTK_PIN(
+ 51, "GPIO51",
+ MTK_EINT_FUNCTION(0, 51),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO51"),
+ MTK_FUNCTION(1, "SDA3")
+ ),
+ MTK_PIN(
+ 52, "GPIO52",
+ MTK_EINT_FUNCTION(0, 52),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO52"),
+ MTK_FUNCTION(1, "BPI_ANT2")
+ ),
+ MTK_PIN(
+ 53, "GPIO53",
+ MTK_EINT_FUNCTION(0, 53),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO53"),
+ MTK_FUNCTION(1, "BPI_ANT0")
+ ),
+ MTK_PIN(
+ 54, "GPIO54",
+ MTK_EINT_FUNCTION(0, 54),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO54"),
+ MTK_FUNCTION(1, "BPI_OLAT1")
+ ),
+ MTK_PIN(
+ 55, "GPIO55",
+ MTK_EINT_FUNCTION(0, 55),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO55"),
+ MTK_FUNCTION(1, "BPI_BUS8")
+ ),
+ MTK_PIN(
+ 56, "GPIO56",
+ MTK_EINT_FUNCTION(0, 56),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO56"),
+ MTK_FUNCTION(1, "BPI_BUS9"),
+ MTK_FUNCTION(2, "SCL_6306")
+ ),
+ MTK_PIN(
+ 57, "GPIO57",
+ MTK_EINT_FUNCTION(0, 57),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO57"),
+ MTK_FUNCTION(1, "BPI_BUS10"),
+ MTK_FUNCTION(2, "SDA_6306")
+ ),
+ MTK_PIN(
+ 58, "GPIO58",
+ MTK_EINT_FUNCTION(0, 58),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO58"),
+ MTK_FUNCTION(1, "RFIC0_BSI_D2"),
+ MTK_FUNCTION(2, "SPM_BSI_D2"),
+ MTK_FUNCTION(3, "PWM_B")
+ ),
+ MTK_PIN(
+ 59, "GPIO59",
+ MTK_EINT_FUNCTION(0, 59),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO59"),
+ MTK_FUNCTION(1, "RFIC0_BSI_D1"),
+ MTK_FUNCTION(2, "SPM_BSI_D1")
+ ),
+ MTK_PIN(
+ 60, "GPIO60",
+ MTK_EINT_FUNCTION(0, 60),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO60"),
+ MTK_FUNCTION(1, "RFIC0_BSI_D0"),
+ MTK_FUNCTION(2, "SPM_BSI_D0")
+ ),
+ MTK_PIN(
+ 61, "GPIO61",
+ MTK_EINT_FUNCTION(0, 61),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO61"),
+ MTK_FUNCTION(1, "MIPI1_SDATA")
+ ),
+ MTK_PIN(
+ 62, "GPIO62",
+ MTK_EINT_FUNCTION(0, 62),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO62"),
+ MTK_FUNCTION(1, "MIPI1_SCLK")
+ ),
+ MTK_PIN(
+ 63, "GPIO63",
+ MTK_EINT_FUNCTION(0, 63),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO63"),
+ MTK_FUNCTION(1, "MIPI0_SDATA")
+ ),
+ MTK_PIN(
+ 64, "GPIO64",
+ MTK_EINT_FUNCTION(0, 64),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO64"),
+ MTK_FUNCTION(1, "MIPI0_SCLK")
+ ),
+ MTK_PIN(
+ 65, "GPIO65",
+ MTK_EINT_FUNCTION(0, 65),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO65"),
+ MTK_FUNCTION(1, "MIPI3_SDATA"),
+ MTK_FUNCTION(2, "BPI_OLAT2")
+ ),
+ MTK_PIN(
+ 66, "GPIO66",
+ MTK_EINT_FUNCTION(0, 66),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO66"),
+ MTK_FUNCTION(1, "MIPI3_SCLK"),
+ MTK_FUNCTION(2, "BPI_OLAT3")
+ ),
+ MTK_PIN(
+ 67, "GPIO67",
+ MTK_EINT_FUNCTION(0, 67),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO67"),
+ MTK_FUNCTION(1, "MIPI2_SDATA")
+ ),
+ MTK_PIN(
+ 68, "GPIO68",
+ MTK_EINT_FUNCTION(0, 68),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO68"),
+ MTK_FUNCTION(1, "MIPI2_SCLK")
+ ),
+ MTK_PIN(
+ 69, "GPIO69",
+ MTK_EINT_FUNCTION(0, 69),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO69"),
+ MTK_FUNCTION(1, "BPI_BUS7")
+ ),
+ MTK_PIN(
+ 70, "GPIO70",
+ MTK_EINT_FUNCTION(0, 70),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO70"),
+ MTK_FUNCTION(1, "BPI_BUS6")
+ ),
+ MTK_PIN(
+ 71, "GPIO71",
+ MTK_EINT_FUNCTION(0, 71),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO71"),
+ MTK_FUNCTION(1, "BPI_BUS5")
+ ),
+ MTK_PIN(
+ 72, "GPIO72",
+ MTK_EINT_FUNCTION(0, 72),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO72"),
+ MTK_FUNCTION(1, "BPI_BUS4")
+ ),
+ MTK_PIN(
+ 73, "GPIO73",
+ MTK_EINT_FUNCTION(0, 73),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO73"),
+ MTK_FUNCTION(1, "BPI_BUS3")
+ ),
+ MTK_PIN(
+ 74, "GPIO74",
+ MTK_EINT_FUNCTION(0, 74),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO74"),
+ MTK_FUNCTION(1, "BPI_BUS2")
+ ),
+ MTK_PIN(
+ 75, "GPIO75",
+ MTK_EINT_FUNCTION(0, 75),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO75"),
+ MTK_FUNCTION(1, "BPI_BUS1")
+ ),
+ MTK_PIN(
+ 76, "GPIO76",
+ MTK_EINT_FUNCTION(0, 76),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO76"),
+ MTK_FUNCTION(1, "BPI_BUS0")
+ ),
+ MTK_PIN(
+ 77, "GPIO77",
+ MTK_EINT_FUNCTION(0, 77),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO77"),
+ MTK_FUNCTION(1, "BPI_ANT1")
+ ),
+ MTK_PIN(
+ 78, "GPIO78",
+ MTK_EINT_FUNCTION(0, 78),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO78"),
+ MTK_FUNCTION(1, "BPI_OLAT0")
+ ),
+ MTK_PIN(
+ 79, "GPIO79",
+ MTK_EINT_FUNCTION(0, 79),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO79"),
+ MTK_FUNCTION(1, "BPI_PA_VM1"),
+ MTK_FUNCTION(2, "MIPI4_SDATA")
+ ),
+ MTK_PIN(
+ 80, "GPIO80",
+ MTK_EINT_FUNCTION(0, 80),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO80"),
+ MTK_FUNCTION(1, "BPI_PA_VM0"),
+ MTK_FUNCTION(2, "MIPI4_SCLK")
+ ),
+ MTK_PIN(
+ 81, "GPIO81",
+ MTK_EINT_FUNCTION(0, 81),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO81"),
+ MTK_FUNCTION(1, "SDA1")
+ ),
+ MTK_PIN(
+ 82, "GPIO82",
+ MTK_EINT_FUNCTION(0, 82),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO82"),
+ MTK_FUNCTION(1, "SDA0")
+ ),
+ MTK_PIN(
+ 83, "GPIO83",
+ MTK_EINT_FUNCTION(0, 83),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO83"),
+ MTK_FUNCTION(1, "SCL0")
+ ),
+ MTK_PIN(
+ 84, "GPIO84",
+ MTK_EINT_FUNCTION(0, 84),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO84"),
+ MTK_FUNCTION(1, "SCL1")
+ ),
+ MTK_PIN(
+ 85, "GPIO85",
+ MTK_EINT_FUNCTION(0, 85),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO85"),
+ MTK_FUNCTION(1, "SPI0_MI"),
+ MTK_FUNCTION(2, "SCP_SPI0_MI"),
+ MTK_FUNCTION(3, "CLKM3"),
+ MTK_FUNCTION(4, "I2S1_BCK"),
+ MTK_FUNCTION(5, "MFG_DFD_JTAG_TDO"),
+ MTK_FUNCTION(6, "DFD_TDO"),
+ MTK_FUNCTION(7, "JTDO_SEL1")
+ ),
+ MTK_PIN(
+ 86, "GPIO86",
+ MTK_EINT_FUNCTION(0, 86),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO86"),
+ MTK_FUNCTION(1, "SPI0_CSB"),
+ MTK_FUNCTION(2, "SCP_SPI0_CS"),
+ MTK_FUNCTION(3, "CLKM0"),
+ MTK_FUNCTION(4, "I2S1_LRCK"),
+ MTK_FUNCTION(5, "MFG_DFD_JTAG_TMS"),
+ MTK_FUNCTION(6, "DFD_TMS"),
+ MTK_FUNCTION(7, "JTMS_SEL1")
+ ),
+ MTK_PIN(
+ 87, "GPIO87",
+ MTK_EINT_FUNCTION(0, 87),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO87"),
+ MTK_FUNCTION(1, "SPI0_MO"),
+ MTK_FUNCTION(2, "SCP_SPI0_MO"),
+ MTK_FUNCTION(3, "SDA1"),
+ MTK_FUNCTION(4, "I2S1_DO"),
+ MTK_FUNCTION(5, "MFG_DFD_JTAG_TDI"),
+ MTK_FUNCTION(6, "DFD_TDI"),
+ MTK_FUNCTION(7, "JTDI_SEL1")
+ ),
+ MTK_PIN(
+ 88, "GPIO88",
+ MTK_EINT_FUNCTION(0, 88),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO88"),
+ MTK_FUNCTION(1, "SPI0_CLK"),
+ MTK_FUNCTION(2, "SCP_SPI0_CK"),
+ MTK_FUNCTION(3, "SCL1"),
+ MTK_FUNCTION(4, "I2S1_MCK"),
+ MTK_FUNCTION(5, "MFG_DFD_JTAG_TCK"),
+ MTK_FUNCTION(6, "DFD_TCK_XI"),
+ MTK_FUNCTION(7, "JTCK_SEL1")
+ ),
+ MTK_PIN(
+ 89, "GPIO89",
+ MTK_EINT_FUNCTION(0, 89),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO89"),
+ MTK_FUNCTION(1, "SRCLKENAI0"),
+ MTK_FUNCTION(2, "PWM_C"),
+ MTK_FUNCTION(3, "I2S5_BCK"),
+ MTK_FUNCTION(4, "ANT_SEL6"),
+ MTK_FUNCTION(5, "SDA8"),
+ MTK_FUNCTION(6, "CMVREF0"),
+ MTK_FUNCTION(7, "DBG_MON_A21")
+ ),
+ MTK_PIN(
+ 90, "GPIO90",
+ MTK_EINT_FUNCTION(0, 90),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO90"),
+ MTK_FUNCTION(1, "PWM_A"),
+ MTK_FUNCTION(2, "CMMCLK2"),
+ MTK_FUNCTION(3, "I2S5_LRCK"),
+ MTK_FUNCTION(4, "SCP_VREQ_VAO"),
+ MTK_FUNCTION(5, "SCL8"),
+ MTK_FUNCTION(6, "PTA_RXD"),
+ MTK_FUNCTION(7, "DBG_MON_A22")
+ ),
+ MTK_PIN(
+ 91, "GPIO91",
+ MTK_EINT_FUNCTION(0, 91),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO91"),
+ MTK_FUNCTION(1, "KPROW1"),
+ MTK_FUNCTION(2, "PWM_B"),
+ MTK_FUNCTION(3, "I2S5_DO"),
+ MTK_FUNCTION(4, "ANT_SEL7"),
+ MTK_FUNCTION(5, "CMMCLK3"),
+ MTK_FUNCTION(6, "PTA_TXD")
+ ),
+ MTK_PIN(
+ 92, "GPIO92",
+ MTK_EINT_FUNCTION(0, 92),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO92"),
+ MTK_FUNCTION(1, "KPROW0")
+ ),
+ MTK_PIN(
+ 93, "GPIO93",
+ MTK_EINT_FUNCTION(0, 93),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO93"),
+ MTK_FUNCTION(1, "KPCOL0"),
+ MTK_FUNCTION(7, "DBG_MON_B27")
+ ),
+ MTK_PIN(
+ 94, "GPIO94",
+ MTK_EINT_FUNCTION(0, 94),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO94"),
+ MTK_FUNCTION(1, "KPCOL1"),
+ MTK_FUNCTION(2, "I2S2_DI2"),
+ MTK_FUNCTION(3, "I2S5_MCK"),
+ MTK_FUNCTION(4, "CMMCLK2"),
+ MTK_FUNCTION(5, "SCP_SPI2_MI"),
+ MTK_FUNCTION(6, "SRCLKENAI1"),
+ MTK_FUNCTION(7, "SPI2_MI")
+ ),
+ MTK_PIN(
+ 95, "GPIO95",
+ MTK_EINT_FUNCTION(0, 95),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO95"),
+ MTK_FUNCTION(1, "URXD0"),
+ MTK_FUNCTION(2, "UTXD0"),
+ MTK_FUNCTION(3, "MD_URXD0"),
+ MTK_FUNCTION(4, "MD_URXD1"),
+ MTK_FUNCTION(5, "SSPM_URXD_AO"),
+ MTK_FUNCTION(6, "CCU_URXD_AO")
+ ),
+ MTK_PIN(
+ 96, "GPIO96",
+ MTK_EINT_FUNCTION(0, 96),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO96"),
+ MTK_FUNCTION(1, "UTXD0"),
+ MTK_FUNCTION(2, "URXD0"),
+ MTK_FUNCTION(3, "MD_UTXD0"),
+ MTK_FUNCTION(4, "MD_UTXD1"),
+ MTK_FUNCTION(5, "SSPM_UTXD_AO"),
+ MTK_FUNCTION(6, "CCU_UTXD_AO"),
+ MTK_FUNCTION(7, "DBG_MON_B2")
+ ),
+ MTK_PIN(
+ 97, "GPIO97",
+ MTK_EINT_FUNCTION(0, 97),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO97"),
+ MTK_FUNCTION(1, "UCTS0"),
+ MTK_FUNCTION(2, "I2S2_MCK"),
+ MTK_FUNCTION(3, "IDDIG"),
+ MTK_FUNCTION(4, "CONN_MCU_TDO"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TDO"),
+ MTK_FUNCTION(6, "IO_JTAG_TDO"),
+ MTK_FUNCTION(7, "DBG_MON_B3")
+ ),
+ MTK_PIN(
+ 98, "GPIO98",
+ MTK_EINT_FUNCTION(0, 98),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO98"),
+ MTK_FUNCTION(1, "URTS0"),
+ MTK_FUNCTION(2, "I2S2_BCK"),
+ MTK_FUNCTION(3, "USB_DRVVBUS"),
+ MTK_FUNCTION(4, "CONN_MCU_TMS"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TMS"),
+ MTK_FUNCTION(6, "IO_JTAG_TMS"),
+ MTK_FUNCTION(7, "DBG_MON_B4")
+ ),
+ MTK_PIN(
+ 99, "GPIO99",
+ MTK_EINT_FUNCTION(0, 99),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO99"),
+ MTK_FUNCTION(1, "CMMCLK0"),
+ MTK_FUNCTION(4, "CONN_MCU_AICE_TMSC"),
+ MTK_FUNCTION(7, "DBG_MON_B28")
+ ),
+ MTK_PIN(
+ 100, "GPIO100",
+ MTK_EINT_FUNCTION(0, 100),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO100"),
+ MTK_FUNCTION(1, "CMMCLK1"),
+ MTK_FUNCTION(2, "PWM_C"),
+ MTK_FUNCTION(3, "MD_INT1_C2K_UIM0_HOT_PLUG"),
+ MTK_FUNCTION(4, "CONN_MCU_AICE_TCKC"),
+ MTK_FUNCTION(7, "DBG_MON_B29")
+ ),
+ MTK_PIN(
+ 101, "GPIO101",
+ MTK_EINT_FUNCTION(0, 101),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO101"),
+ MTK_FUNCTION(1, "CLKM2"),
+ MTK_FUNCTION(2, "I2S2_LRCK"),
+ MTK_FUNCTION(3, "CMVREF1"),
+ MTK_FUNCTION(4, "CONN_MCU_TCK"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TCK"),
+ MTK_FUNCTION(6, "IO_JTAG_TCK")
+ ),
+ MTK_PIN(
+ 102, "GPIO102",
+ MTK_EINT_FUNCTION(0, 102),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO102"),
+ MTK_FUNCTION(1, "CLKM1"),
+ MTK_FUNCTION(2, "I2S2_DI"),
+ MTK_FUNCTION(3, "DVFSRC_EXT_REQ"),
+ MTK_FUNCTION(4, "CONN_MCU_TDI"),
+ MTK_FUNCTION(5, "SSPM_JTAG_TDI"),
+ MTK_FUNCTION(6, "IO_JTAG_TDI"),
+ MTK_FUNCTION(7, "DBG_MON_B8")
+ ),
+ MTK_PIN(
+ 103, "GPIO103",
+ MTK_EINT_FUNCTION(0, 103),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO103"),
+ MTK_FUNCTION(1, "SCL2")
+ ),
+ MTK_PIN(
+ 104, "GPIO104",
+ MTK_EINT_FUNCTION(0, 104),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO104"),
+ MTK_FUNCTION(1, "SDA2")
+ ),
+ MTK_PIN(
+ 105, "GPIO105",
+ MTK_EINT_FUNCTION(0, 105),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO105"),
+ MTK_FUNCTION(1, "SCL4")
+ ),
+ MTK_PIN(
+ 106, "GPIO106",
+ MTK_EINT_FUNCTION(0, 106),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO106"),
+ MTK_FUNCTION(1, "SDA4")
+ ),
+ MTK_PIN(
+ 107, "GPIO107",
+ MTK_EINT_FUNCTION(0, 107),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO107"),
+ MTK_FUNCTION(1, "DMIC_CLK"),
+ MTK_FUNCTION(2, "ANT_SEL0"),
+ MTK_FUNCTION(3, "CLKM0"),
+ MTK_FUNCTION(4, "SDA7"),
+ MTK_FUNCTION(5, "EXT_FRAME_SYNC"),
+ MTK_FUNCTION(6, "PWM_A"),
+ MTK_FUNCTION(7, "DBG_MON_B12")
+ ),
+ MTK_PIN(
+ 108, "GPIO108",
+ MTK_EINT_FUNCTION(0, 108),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO108"),
+ MTK_FUNCTION(1, "CMMCLK2"),
+ MTK_FUNCTION(2, "ANT_SEL1"),
+ MTK_FUNCTION(3, "CLKM1"),
+ MTK_FUNCTION(4, "SCL8"),
+ MTK_FUNCTION(5, "DAP_MD32_SWD"),
+ MTK_FUNCTION(6, "PWM_B"),
+ MTK_FUNCTION(7, "DBG_MON_B13")
+ ),
+ MTK_PIN(
+ 109, "GPIO109",
+ MTK_EINT_FUNCTION(0, 109),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO109"),
+ MTK_FUNCTION(1, "DMIC_DAT"),
+ MTK_FUNCTION(2, "ANT_SEL2"),
+ MTK_FUNCTION(3, "CLKM2"),
+ MTK_FUNCTION(4, "SDA8"),
+ MTK_FUNCTION(5, "DAP_MD32_SWCK"),
+ MTK_FUNCTION(6, "PWM_C"),
+ MTK_FUNCTION(7, "DBG_MON_B14")
+ ),
+ MTK_PIN(
+ 110, "GPIO110",
+ MTK_EINT_FUNCTION(0, 110),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO110"),
+ MTK_FUNCTION(1, "SCL7"),
+ MTK_FUNCTION(2, "ANT_SEL0"),
+ MTK_FUNCTION(3, "TP_URXD1_AO"),
+ MTK_FUNCTION(4, "USB_DRVVBUS"),
+ MTK_FUNCTION(5, "SRCLKENAI1"),
+ MTK_FUNCTION(6, "KPCOL2"),
+ MTK_FUNCTION(7, "URXD1")
+ ),
+ MTK_PIN(
+ 111, "GPIO111",
+ MTK_EINT_FUNCTION(0, 111),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO111"),
+ MTK_FUNCTION(1, "CMMCLK3"),
+ MTK_FUNCTION(2, "ANT_SEL1"),
+ MTK_FUNCTION(3, "SRCLKENAI0"),
+ MTK_FUNCTION(4, "SCP_VREQ_VAO"),
+ MTK_FUNCTION(5, "MD_INT2_C2K_UIM1_HOT_PLUG"),
+ MTK_FUNCTION(7, "DVFSRC_EXT_REQ")
+ ),
+ MTK_PIN(
+ 112, "GPIO112",
+ MTK_EINT_FUNCTION(0, 112),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO112"),
+ MTK_FUNCTION(1, "SDA7"),
+ MTK_FUNCTION(2, "ANT_SEL2"),
+ MTK_FUNCTION(3, "TP_UTXD1_AO"),
+ MTK_FUNCTION(4, "IDDIG"),
+ MTK_FUNCTION(5, "AGPS_SYNC"),
+ MTK_FUNCTION(6, "KPROW2"),
+ MTK_FUNCTION(7, "UTXD1")
+ ),
+ MTK_PIN(
+ 113, "GPIO113",
+ MTK_EINT_FUNCTION(0, 113),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO113"),
+ MTK_FUNCTION(1, "CONN_TOP_CLK"),
+ MTK_FUNCTION(3, "SCL6"),
+ MTK_FUNCTION(4, "AUXIF_CLK0"),
+ MTK_FUNCTION(6, "TP_UCTS1_AO")
+ ),
+ MTK_PIN(
+ 114, "GPIO114",
+ MTK_EINT_FUNCTION(0, 114),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO114"),
+ MTK_FUNCTION(1, "CONN_TOP_DATA"),
+ MTK_FUNCTION(3, "SDA6"),
+ MTK_FUNCTION(4, "AUXIF_ST0"),
+ MTK_FUNCTION(6, "TP_URTS1_AO")
+ ),
+ MTK_PIN(
+ 115, "GPIO115",
+ MTK_EINT_FUNCTION(0, 115),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO115"),
+ MTK_FUNCTION(1, "CONN_BT_CLK"),
+ MTK_FUNCTION(2, "UTXD1"),
+ MTK_FUNCTION(3, "PTA_TXD"),
+ MTK_FUNCTION(4, "AUXIF_CLK1"),
+ MTK_FUNCTION(5, "DAP_MD32_SWD"),
+ MTK_FUNCTION(6, "TP_UTXD1_AO")
+ ),
+ MTK_PIN(
+ 116, "GPIO116",
+ MTK_EINT_FUNCTION(0, 116),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO116"),
+ MTK_FUNCTION(1, "CONN_BT_DATA"),
+ MTK_FUNCTION(2, "IPU_JTAG_TRST"),
+ MTK_FUNCTION(4, "AUXIF_ST1"),
+ MTK_FUNCTION(5, "DAP_MD32_SWCK"),
+ MTK_FUNCTION(6, "TP_URXD2_AO"),
+ MTK_FUNCTION(7, "DBG_MON_A0")
+ ),
+ MTK_PIN(
+ 117, "GPIO117",
+ MTK_EINT_FUNCTION(0, 117),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO117"),
+ MTK_FUNCTION(1, "CONN_WF_HB0"),
+ MTK_FUNCTION(2, "IPU_JTAG_TDO"),
+ MTK_FUNCTION(6, "TP_UTXD2_AO"),
+ MTK_FUNCTION(7, "DBG_MON_A4")
+ ),
+ MTK_PIN(
+ 118, "GPIO118",
+ MTK_EINT_FUNCTION(0, 118),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO118"),
+ MTK_FUNCTION(1, "CONN_WF_HB1"),
+ MTK_FUNCTION(2, "IPU_JTAG_TDI"),
+ MTK_FUNCTION(5, "SSPM_URXD_AO"),
+ MTK_FUNCTION(6, "TP_UCTS2_AO"),
+ MTK_FUNCTION(7, "DBG_MON_A5")
+ ),
+ MTK_PIN(
+ 119, "GPIO119",
+ MTK_EINT_FUNCTION(0, 119),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO119"),
+ MTK_FUNCTION(1, "CONN_WF_HB2"),
+ MTK_FUNCTION(2, "IPU_JTAG_TCK"),
+ MTK_FUNCTION(5, "SSPM_UTXD_AO"),
+ MTK_FUNCTION(6, "TP_URTS2_AO")
+ ),
+ MTK_PIN(
+ 120, "GPIO120",
+ MTK_EINT_FUNCTION(0, 120),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO120"),
+ MTK_FUNCTION(1, "CONN_WB_PTA"),
+ MTK_FUNCTION(2, "IPU_JTAG_TMS"),
+ MTK_FUNCTION(5, "CCU_URXD_AO")
+ ),
+ MTK_PIN(
+ 121, "GPIO121",
+ MTK_EINT_FUNCTION(0, 121),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO121"),
+ MTK_FUNCTION(1, "CONN_HRST_B"),
+ MTK_FUNCTION(2, "URXD1"),
+ MTK_FUNCTION(3, "PTA_RXD"),
+ MTK_FUNCTION(5, "CCU_UTXD_AO"),
+ MTK_FUNCTION(6, "TP_URXD1_AO")
+ ),
+ MTK_PIN(
+ 122, "GPIO122",
+ MTK_EINT_FUNCTION(0, 122),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO122"),
+ MTK_FUNCTION(1, "MSDC0_CMD"),
+ MTK_FUNCTION(2, "SSPM_URXD2_AO"),
+ MTK_FUNCTION(3, "ANT_SEL1"),
+ MTK_FUNCTION(7, "DBG_MON_A12")
+ ),
+ MTK_PIN(
+ 123, "GPIO123",
+ MTK_EINT_FUNCTION(0, 123),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO123"),
+ MTK_FUNCTION(1, "MSDC0_DAT0"),
+ MTK_FUNCTION(3, "ANT_SEL0"),
+ MTK_FUNCTION(7, "DBG_MON_A13")
+ ),
+ MTK_PIN(
+ 124, "GPIO124",
+ MTK_EINT_FUNCTION(0, 124),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO124"),
+ MTK_FUNCTION(1, "MSDC0_CLK"),
+ MTK_FUNCTION(7, "DBG_MON_A14")
+ ),
+ MTK_PIN(
+ 125, "GPIO125",
+ MTK_EINT_FUNCTION(0, 125),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO125"),
+ MTK_FUNCTION(1, "MSDC0_DAT2"),
+ MTK_FUNCTION(3, "MRG_CLK"),
+ MTK_FUNCTION(7, "DBG_MON_A15")
+ ),
+ MTK_PIN(
+ 126, "GPIO126",
+ MTK_EINT_FUNCTION(0, 126),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO126"),
+ MTK_FUNCTION(1, "MSDC0_DAT4"),
+ MTK_FUNCTION(3, "ANT_SEL5"),
+ MTK_FUNCTION(6, "UFS_MPHY_SCL"),
+ MTK_FUNCTION(7, "DBG_MON_A16")
+ ),
+ MTK_PIN(
+ 127, "GPIO127",
+ MTK_EINT_FUNCTION(0, 127),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO127"),
+ MTK_FUNCTION(1, "MSDC0_DAT6"),
+ MTK_FUNCTION(3, "ANT_SEL4"),
+ MTK_FUNCTION(6, "UFS_MPHY_SDA"),
+ MTK_FUNCTION(7, "DBG_MON_A17")
+ ),
+ MTK_PIN(
+ 128, "GPIO128",
+ MTK_EINT_FUNCTION(0, 128),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO128"),
+ MTK_FUNCTION(1, "MSDC0_DAT1"),
+ MTK_FUNCTION(3, "ANT_SEL2"),
+ MTK_FUNCTION(6, "UFS_UNIPRO_SDA"),
+ MTK_FUNCTION(7, "DBG_MON_A18")
+ ),
+ MTK_PIN(
+ 129, "GPIO129",
+ MTK_EINT_FUNCTION(0, 129),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO129"),
+ MTK_FUNCTION(1, "MSDC0_DAT5"),
+ MTK_FUNCTION(3, "ANT_SEL3"),
+ MTK_FUNCTION(6, "UFS_UNIPRO_SCL"),
+ MTK_FUNCTION(7, "DBG_MON_A23")
+ ),
+ MTK_PIN(
+ 130, "GPIO130",
+ MTK_EINT_FUNCTION(0, 130),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO130"),
+ MTK_FUNCTION(1, "MSDC0_DAT7"),
+ MTK_FUNCTION(3, "MRG_DO"),
+ MTK_FUNCTION(7, "DBG_MON_A24")
+ ),
+ MTK_PIN(
+ 131, "GPIO131",
+ MTK_EINT_FUNCTION(0, 131),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO131"),
+ MTK_FUNCTION(1, "MSDC0_DSL"),
+ MTK_FUNCTION(3, "MRG_SYNC"),
+ MTK_FUNCTION(7, "DBG_MON_A25")
+ ),
+ MTK_PIN(
+ 132, "GPIO132",
+ MTK_EINT_FUNCTION(0, 132),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO132"),
+ MTK_FUNCTION(1, "MSDC0_DAT3"),
+ MTK_FUNCTION(3, "MRG_DI"),
+ MTK_FUNCTION(7, "DBG_MON_A26")
+ ),
+ MTK_PIN(
+ 133, "GPIO133",
+ MTK_EINT_FUNCTION(0, 133),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO133"),
+ MTK_FUNCTION(1, "MSDC0_RSTB"),
+ MTK_FUNCTION(3, "AGPS_SYNC"),
+ MTK_FUNCTION(7, "DBG_MON_A27")
+ ),
+ MTK_PIN(
+ 134, "GPIO134",
+ MTK_EINT_FUNCTION(0, 134),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO134"),
+ MTK_FUNCTION(1, "RTC32K_CK")
+ ),
+ MTK_PIN(
+ 135, "GPIO135",
+ MTK_EINT_FUNCTION(0, 135),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO135"),
+ MTK_FUNCTION(1, "WATCHDOG")
+ ),
+ MTK_PIN(
+ 136, "GPIO136",
+ MTK_EINT_FUNCTION(0, 136),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO136"),
+ MTK_FUNCTION(1, "AUD_CLK_MOSI"),
+ MTK_FUNCTION(2, "AUD_CLK_MISO"),
+ MTK_FUNCTION(3, "I2S1_MCK"),
+ MTK_FUNCTION(6, "UFS_UNIPRO_SCL")
+ ),
+ MTK_PIN(
+ 137, "GPIO137",
+ MTK_EINT_FUNCTION(0, 137),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO137"),
+ MTK_FUNCTION(1, "AUD_SYNC_MOSI"),
+ MTK_FUNCTION(2, "AUD_SYNC_MISO"),
+ MTK_FUNCTION(3, "I2S1_BCK")
+ ),
+ MTK_PIN(
+ 138, "GPIO138",
+ MTK_EINT_FUNCTION(0, 138),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO138"),
+ MTK_FUNCTION(1, "AUD_DAT_MOSI0"),
+ MTK_FUNCTION(2, "AUD_DAT_MISO0"),
+ MTK_FUNCTION(3, "I2S1_LRCK"),
+ MTK_FUNCTION(7, "DBG_MON_B24")
+ ),
+ MTK_PIN(
+ 139, "GPIO139",
+ MTK_EINT_FUNCTION(0, 139),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO139"),
+ MTK_FUNCTION(1, "AUD_DAT_MOSI1"),
+ MTK_FUNCTION(2, "AUD_DAT_MISO1"),
+ MTK_FUNCTION(3, "I2S1_DO"),
+ MTK_FUNCTION(6, "UFS_MPHY_SDA")
+ ),
+ MTK_PIN(
+ 140, "GPIO140",
+ MTK_EINT_FUNCTION(0, 140),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO140"),
+ MTK_FUNCTION(1, "AUD_CLK_MISO"),
+ MTK_FUNCTION(2, "AUD_CLK_MOSI"),
+ MTK_FUNCTION(3, "I2S0_MCK"),
+ MTK_FUNCTION(6, "UFS_UNIPRO_SDA")
+ ),
+ MTK_PIN(
+ 141, "GPIO141",
+ MTK_EINT_FUNCTION(0, 141),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO141"),
+ MTK_FUNCTION(1, "AUD_SYNC_MISO"),
+ MTK_FUNCTION(2, "AUD_SYNC_MOSI"),
+ MTK_FUNCTION(3, "I2S0_BCK")
+ ),
+ MTK_PIN(
+ 142, "GPIO142",
+ MTK_EINT_FUNCTION(0, 142),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO142"),
+ MTK_FUNCTION(1, "AUD_DAT_MISO0"),
+ MTK_FUNCTION(2, "AUD_DAT_MOSI0"),
+ MTK_FUNCTION(3, "I2S0_LRCK"),
+ MTK_FUNCTION(4, "VOW_DAT_MISO"),
+ MTK_FUNCTION(7, "DBG_MON_B25")
+ ),
+ MTK_PIN(
+ 143, "GPIO143",
+ MTK_EINT_FUNCTION(0, 143),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO143"),
+ MTK_FUNCTION(1, "AUD_DAT_MISO1"),
+ MTK_FUNCTION(2, "AUD_DAT_MOSI1"),
+ MTK_FUNCTION(3, "I2S0_DI"),
+ MTK_FUNCTION(4, "VOW_CLK_MISO"),
+ MTK_FUNCTION(6, "UFS_MPHY_SCL"),
+ MTK_FUNCTION(7, "DBG_MON_B26")
+ ),
+ MTK_PIN(
+ 144, "GPIO144",
+ MTK_EINT_FUNCTION(0, 144),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO144"),
+ MTK_FUNCTION(1, "PWRAP_SPI0_MI"),
+ MTK_FUNCTION(2, "PWRAP_SPI0_MO")
+ ),
+ MTK_PIN(
+ 145, "GPIO145",
+ MTK_EINT_FUNCTION(0, 145),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO145"),
+ MTK_FUNCTION(1, "PWRAP_SPI0_CSN")
+ ),
+ MTK_PIN(
+ 146, "GPIO146",
+ MTK_EINT_FUNCTION(0, 146),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO146"),
+ MTK_FUNCTION(1, "PWRAP_SPI0_MO"),
+ MTK_FUNCTION(2, "PWRAP_SPI0_MI")
+ ),
+ MTK_PIN(
+ 147, "GPIO147",
+ MTK_EINT_FUNCTION(0, 147),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO147"),
+ MTK_FUNCTION(1, "PWRAP_SPI0_CK")
+ ),
+ MTK_PIN(
+ 148, "GPIO148",
+ MTK_EINT_FUNCTION(0, 148),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO148"),
+ MTK_FUNCTION(1, "SRCLKENA0")
+ ),
+ MTK_PIN(
+ 149, "GPIO149",
+ MTK_EINT_FUNCTION(0, 149),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO149"),
+ MTK_FUNCTION(1, "SRCLKENA1")
+ ),
+ MTK_PIN(
+ 150, "GPIO150",
+ MTK_EINT_FUNCTION(0, 150),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO150"),
+ MTK_FUNCTION(1, "PWM_A"),
+ MTK_FUNCTION(2, "CMFLASH"),
+ MTK_FUNCTION(3, "CLKM0"),
+ MTK_FUNCTION(7, "DBG_MON_B30")
+ ),
+ MTK_PIN(
+ 151, "GPIO151",
+ MTK_EINT_FUNCTION(0, 151),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO151"),
+ MTK_FUNCTION(1, "PWM_B"),
+ MTK_FUNCTION(2, "CMVREF0"),
+ MTK_FUNCTION(3, "CLKM1"),
+ MTK_FUNCTION(7, "DBG_MON_B20")
+ ),
+ MTK_PIN(
+ 152, "GPIO152",
+ MTK_EINT_FUNCTION(0, 152),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO152"),
+ MTK_FUNCTION(1, "PWM_C"),
+ MTK_FUNCTION(2, "CMFLASH"),
+ MTK_FUNCTION(3, "CLKM2"),
+ MTK_FUNCTION(7, "DBG_MON_B21")
+ ),
+ MTK_PIN(
+ 153, "GPIO153",
+ MTK_EINT_FUNCTION(0, 153),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO153"),
+ MTK_FUNCTION(1, "PWM_A"),
+ MTK_FUNCTION(2, "CMVREF0"),
+ MTK_FUNCTION(3, "CLKM3"),
+ MTK_FUNCTION(7, "DBG_MON_B22")
+ ),
+ MTK_PIN(
+ 154, "GPIO154",
+ MTK_EINT_FUNCTION(0, 154),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO154"),
+ MTK_FUNCTION(1, "SCP_VREQ_VAO"),
+ MTK_FUNCTION(2, "DVFSRC_EXT_REQ"),
+ MTK_FUNCTION(7, "DBG_MON_B18")
+ ),
+ MTK_PIN(
+ 155, "GPIO155",
+ MTK_EINT_FUNCTION(0, 155),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO155"),
+ MTK_FUNCTION(1, "ANT_SEL0"),
+ MTK_FUNCTION(2, "DVFSRC_EXT_REQ"),
+ MTK_FUNCTION(3, "CMVREF1"),
+ MTK_FUNCTION(7, "SCP_JTAG_TDI")
+ ),
+ MTK_PIN(
+ 156, "GPIO156",
+ MTK_EINT_FUNCTION(0, 156),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO156"),
+ MTK_FUNCTION(1, "ANT_SEL1"),
+ MTK_FUNCTION(2, "SRCLKENAI0"),
+ MTK_FUNCTION(3, "SCL6"),
+ MTK_FUNCTION(4, "KPCOL2"),
+ MTK_FUNCTION(5, "IDDIG"),
+ MTK_FUNCTION(7, "SCP_JTAG_TCK")
+ ),
+ MTK_PIN(
+ 157, "GPIO157",
+ MTK_EINT_FUNCTION(0, 157),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO157"),
+ MTK_FUNCTION(1, "ANT_SEL2"),
+ MTK_FUNCTION(2, "SRCLKENAI1"),
+ MTK_FUNCTION(3, "SDA6"),
+ MTK_FUNCTION(4, "KPROW2"),
+ MTK_FUNCTION(5, "USB_DRVVBUS"),
+ MTK_FUNCTION(7, "SCP_JTAG_TRSTN")
+ ),
+ MTK_PIN(
+ 158, "GPIO158",
+ MTK_EINT_FUNCTION(0, 158),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO158"),
+ MTK_FUNCTION(1, "ANT_SEL3")
+ ),
+ MTK_PIN(
+ 159, "GPIO159",
+ MTK_EINT_FUNCTION(0, 159),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO159"),
+ MTK_FUNCTION(1, "ANT_SEL4")
+ ),
+ MTK_PIN(
+ 160, "GPIO160",
+ MTK_EINT_FUNCTION(0, 160),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO160"),
+ MTK_FUNCTION(1, "ANT_SEL5")
+ ),
+ MTK_PIN(
+ 161, "GPIO161",
+ MTK_EINT_FUNCTION(0, 161),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO161"),
+ MTK_FUNCTION(1, "SPI1_A_MI"),
+ MTK_FUNCTION(2, "SCP_SPI1_MI"),
+ MTK_FUNCTION(3, "IDDIG"),
+ MTK_FUNCTION(4, "ANT_SEL6"),
+ MTK_FUNCTION(5, "KPCOL2"),
+ MTK_FUNCTION(6, "PTA_RXD"),
+ MTK_FUNCTION(7, "DBG_MON_B19")
+ ),
+ MTK_PIN(
+ 162, "GPIO162",
+ MTK_EINT_FUNCTION(0, 162),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO162"),
+ MTK_FUNCTION(1, "SPI1_A_CSB"),
+ MTK_FUNCTION(2, "SCP_SPI1_CS"),
+ MTK_FUNCTION(3, "USB_DRVVBUS"),
+ MTK_FUNCTION(4, "ANT_SEL5"),
+ MTK_FUNCTION(5, "KPROW2"),
+ MTK_FUNCTION(6, "PTA_TXD")
+ ),
+ MTK_PIN(
+ 163, "GPIO163",
+ MTK_EINT_FUNCTION(0, 163),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO163"),
+ MTK_FUNCTION(1, "SPI1_A_MO"),
+ MTK_FUNCTION(2, "SCP_SPI1_MO"),
+ MTK_FUNCTION(3, "SDA1"),
+ MTK_FUNCTION(4, "ANT_SEL4"),
+ MTK_FUNCTION(5, "CMMCLK2"),
+ MTK_FUNCTION(6, "DMIC_CLK")
+ ),
+ MTK_PIN(
+ 164, "GPIO164",
+ MTK_EINT_FUNCTION(0, 164),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO164"),
+ MTK_FUNCTION(1, "SPI1_A_CLK"),
+ MTK_FUNCTION(2, "SCP_SPI1_CK"),
+ MTK_FUNCTION(3, "SCL1"),
+ MTK_FUNCTION(4, "ANT_SEL3"),
+ MTK_FUNCTION(5, "CMMCLK3"),
+ MTK_FUNCTION(6, "DMIC_DAT")
+ ),
+ MTK_PIN(
+ 165, "GPIO165",
+ MTK_EINT_FUNCTION(0, 165),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO165"),
+ MTK_FUNCTION(1, "PWM_B"),
+ MTK_FUNCTION(2, "CMMCLK2"),
+ MTK_FUNCTION(3, "SCP_VREQ_VAO"),
+ MTK_FUNCTION(6, "TDM_MCK_2ND"),
+ MTK_FUNCTION(7, "SCP_JTAG_TDO")
+ ),
+ MTK_PIN(
+ 166, "GPIO166",
+ MTK_EINT_FUNCTION(0, 166),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO166"),
+ MTK_FUNCTION(1, "ANT_SEL6")
+ ),
+ MTK_PIN(
+ 167, "GPIO167",
+ MTK_EINT_FUNCTION(0, 167),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO167"),
+ MTK_FUNCTION(1, "RFIC0_BSI_EN"),
+ MTK_FUNCTION(2, "SPM_BSI_EN")
+ ),
+ MTK_PIN(
+ 168, "GPIO168",
+ MTK_EINT_FUNCTION(0, 168),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO168"),
+ MTK_FUNCTION(1, "RFIC0_BSI_CK"),
+ MTK_FUNCTION(2, "SPM_BSI_CK")
+ ),
+ MTK_PIN(
+ 169, "GPIO169",
+ MTK_EINT_FUNCTION(0, 169),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO169"),
+ MTK_FUNCTION(1, "PWM_C"),
+ MTK_FUNCTION(2, "CMMCLK3"),
+ MTK_FUNCTION(3, "CMVREF1"),
+ MTK_FUNCTION(4, "ANT_SEL7"),
+ MTK_FUNCTION(5, "AGPS_SYNC"),
+ MTK_FUNCTION(6, "TDM_BCK_2ND"),
+ MTK_FUNCTION(7, "SCP_JTAG_TMS")
+ ),
+ MTK_PIN(
+ 170, "GPIO170",
+ MTK_EINT_FUNCTION(0, 170),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO170"),
+ MTK_FUNCTION(1, "I2S1_BCK"),
+ MTK_FUNCTION(2, "I2S3_BCK"),
+ MTK_FUNCTION(3, "SCL7"),
+ MTK_FUNCTION(4, "I2S5_BCK"),
+ MTK_FUNCTION(5, "EXT_FRAME_SYNC"),
+ MTK_FUNCTION(6, "TDM_LRCK_2ND"),
+ MTK_FUNCTION(7, "ANT_SEL3")
+ ),
+ MTK_PIN(
+ 171, "GPIO171",
+ MTK_EINT_FUNCTION(0, 184),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO171"),
+ MTK_FUNCTION(1, "I2S1_LRCK"),
+ MTK_FUNCTION(2, "I2S3_LRCK"),
+ MTK_FUNCTION(3, "SDA7"),
+ MTK_FUNCTION(4, "I2S5_LRCK"),
+ MTK_FUNCTION(5, "URXD1"),
+ MTK_FUNCTION(6, "TDM_DATA0_2ND"),
+ MTK_FUNCTION(7, "ANT_SEL4")
+ ),
+ MTK_PIN(
+ 172, "GPIO172",
+ MTK_EINT_FUNCTION(0, 185),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO172"),
+ MTK_FUNCTION(1, "I2S1_DO"),
+ MTK_FUNCTION(2, "I2S3_DO"),
+ MTK_FUNCTION(3, "SCL8"),
+ MTK_FUNCTION(4, "I2S5_DO"),
+ MTK_FUNCTION(5, "UTXD1"),
+ MTK_FUNCTION(6, "TDM_DATA1_2ND"),
+ MTK_FUNCTION(7, "ANT_SEL5")
+ ),
+ MTK_PIN(
+ 173, "GPIO173",
+ MTK_EINT_FUNCTION(0, 186),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO173"),
+ MTK_FUNCTION(1, "I2S1_MCK"),
+ MTK_FUNCTION(2, "I2S3_MCK"),
+ MTK_FUNCTION(3, "SDA8"),
+ MTK_FUNCTION(4, "I2S5_MCK"),
+ MTK_FUNCTION(5, "UCTS0"),
+ MTK_FUNCTION(6, "TDM_DATA2_2ND"),
+ MTK_FUNCTION(7, "ANT_SEL6")
+ ),
+ MTK_PIN(
+ 174, "GPIO174",
+ MTK_EINT_FUNCTION(0, 187),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO174"),
+ MTK_FUNCTION(1, "I2S2_DI"),
+ MTK_FUNCTION(2, "I2S0_DI"),
+ MTK_FUNCTION(3, "DVFSRC_EXT_REQ"),
+ MTK_FUNCTION(4, "I2S2_DI2"),
+ MTK_FUNCTION(5, "URTS0"),
+ MTK_FUNCTION(6, "TDM_DATA3_2ND"),
+ MTK_FUNCTION(7, "ANT_SEL7")
+ ),
+ MTK_PIN(
+ 175, "GPIO175",
+ MTK_EINT_FUNCTION(0, 188),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO175"),
+ MTK_FUNCTION(1, "ANT_SEL7")
+ ),
+ MTK_PIN(
+ 176, "GPIO176",
+ MTK_EINT_FUNCTION(0, 189),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO176")
+ ),
+ MTK_PIN(
+ 177, "GPIO177",
+ MTK_EINT_FUNCTION(0, 190),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO177")
+ ),
+ MTK_PIN(
+ 178, "GPIO178",
+ MTK_EINT_FUNCTION(0, 191),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO178")
+ ),
+ MTK_PIN(
+ 179, "GPIO179",
+ MTK_EINT_FUNCTION(0, 192),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO179")
+ ),
+ MTK_PIN(
+ 180, "GPIO180",
+ MTK_EINT_FUNCTION(0, 171),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO180")
+ ),
+ MTK_PIN(
+ 181, "GPIO181",
+ MTK_EINT_FUNCTION(0, 172),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO181")
+ ),
+ MTK_PIN(
+ 182, "GPIO182",
+ MTK_EINT_FUNCTION(0, 173),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO182")
+ ),
+ MTK_PIN(
+ 183, "GPIO183",
+ MTK_EINT_FUNCTION(0, 174),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO183")
+ ),
+ MTK_PIN(
+ 184, "GPIO184",
+ MTK_EINT_FUNCTION(0, 175),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO184")
+ ),
+ MTK_PIN(
+ 185, "GPIO185",
+ MTK_EINT_FUNCTION(0, 177),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO185")
+ ),
+ MTK_PIN(
+ 186, "GPIO186",
+ MTK_EINT_FUNCTION(0, 178),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO186")
+ ),
+ MTK_PIN(
+ 187, "GPIO187",
+ MTK_EINT_FUNCTION(0, 179),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO187")
+ ),
+ MTK_PIN(
+ 188, "GPIO188",
+ MTK_EINT_FUNCTION(0, 180),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO188")
+ ),
+ MTK_PIN(
+ 189, "GPIO189",
+ MTK_EINT_FUNCTION(0, 181),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO189")
+ ),
+ MTK_PIN(
+ 190, "GPIO190",
+ MTK_EINT_FUNCTION(0, 182),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO190")
+ ),
+ MTK_PIN(
+ 191, "GPIO191",
+ MTK_EINT_FUNCTION(0, 183),
+ DRV_GRP4,
+ MTK_FUNCTION(0, "GPIO191")
+ ),
+};
+
+#endif /* __PINCTRL_MTK_MT8183_H */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * MediaTek Pinctrl Paris Driver, which implement the vendor per-pin
+ * bindings for MediaTek SoC.
+ *
+ * Copyright (C) 2018 MediaTek Inc.
+ * Author: Sean Wang <sean.wang@mediatek.com>
+ * Zhiyong Tao <zhiyong.tao@mediatek.com>
+ * Hongzhou.Yang <hongzhou.yang@mediatek.com>
+ */
+
+#include <linux/gpio/driver.h>
+#include <dt-bindings/pinctrl/mt65xx.h>
+#include "pinctrl-paris.h"
+
+#define PINCTRL_PINCTRL_DEV KBUILD_MODNAME
+
+/* Custom pinconf parameters */
+#define MTK_PIN_CONFIG_TDSEL (PIN_CONFIG_END + 1)
+#define MTK_PIN_CONFIG_RDSEL (PIN_CONFIG_END + 2)
+#define MTK_PIN_CONFIG_PU_ADV (PIN_CONFIG_END + 3)
+#define MTK_PIN_CONFIG_PD_ADV (PIN_CONFIG_END + 4)
+
+static const struct pinconf_generic_params mtk_custom_bindings[] = {
+ {"mediatek,tdsel", MTK_PIN_CONFIG_TDSEL, 0},
+ {"mediatek,rdsel", MTK_PIN_CONFIG_RDSEL, 0},
+ {"mediatek,pull-up-adv", MTK_PIN_CONFIG_PU_ADV, 1},
+ {"mediatek,pull-down-adv", MTK_PIN_CONFIG_PD_ADV, 1},
+};
+
+#ifdef CONFIG_DEBUG_FS
+static const struct pin_config_item mtk_conf_items[] = {
+ PCONFDUMP(MTK_PIN_CONFIG_TDSEL, "tdsel", NULL, true),
+ PCONFDUMP(MTK_PIN_CONFIG_RDSEL, "rdsel", NULL, true),
+ PCONFDUMP(MTK_PIN_CONFIG_PU_ADV, "pu-adv", NULL, true),
+ PCONFDUMP(MTK_PIN_CONFIG_PD_ADV, "pd-adv", NULL, true),
+};
+#endif
+
+static const char * const mtk_gpio_functions[] = {
+ "func0", "func1", "func2", "func3",
+ "func4", "func5", "func6", "func7",
+ "func8", "func9", "func10", "func11",
+ "func12", "func13", "func14", "func15",
+};
+
+static int mtk_pinmux_gpio_request_enable(struct pinctrl_dev *pctldev,
+ struct pinctrl_gpio_range *range,
+ unsigned int pin)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+ const struct mtk_pin_desc *desc;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[pin];
+
+ return mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_MODE,
+ hw->soc->gpio_m);
+}
+
+static int mtk_pinmux_gpio_set_direction(struct pinctrl_dev *pctldev,
+ struct pinctrl_gpio_range *range,
+ unsigned int pin, bool input)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+ const struct mtk_pin_desc *desc;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[pin];
+
+ /* hardware would take 0 as input direction */
+ return mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DIR, !input);
+}
+
+static int mtk_pinconf_get(struct pinctrl_dev *pctldev,
+ unsigned int pin, unsigned long *config)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+ u32 param = pinconf_to_config_param(*config);
+ int val, val2, err, reg, ret = 1;
+ const struct mtk_pin_desc *desc;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[pin];
+
+ switch (param) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ if (hw->soc->bias_disable_get) {
+ err = hw->soc->bias_disable_get(hw, desc, &ret);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case PIN_CONFIG_BIAS_PULL_UP:
+ if (hw->soc->bias_get) {
+ err = hw->soc->bias_get(hw, desc, 1, &ret);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ if (hw->soc->bias_get) {
+ err = hw->soc->bias_get(hw, desc, 0, &ret);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case PIN_CONFIG_SLEW_RATE:
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_SR, &val);
+ if (err)
+ return err;
+
+ if (!val)
+ return -EINVAL;
+
+ break;
+ case PIN_CONFIG_INPUT_ENABLE:
+ case PIN_CONFIG_OUTPUT_ENABLE:
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_DIR, &val);
+ if (err)
+ return err;
+
+ /* HW takes input mode as zero; output mode as non-zero */
+ if ((val && param == PIN_CONFIG_INPUT_ENABLE) ||
+ (!val && param == PIN_CONFIG_OUTPUT_ENABLE))
+ return -EINVAL;
+
+ break;
+ case PIN_CONFIG_INPUT_SCHMITT_ENABLE:
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_DIR, &val);
+ if (err)
+ return err;
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_SMT, &val2);
+ if (err)
+ return err;
+
+ if (val || !val2)
+ return -EINVAL;
+
+ break;
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ if (hw->soc->drive_get) {
+ err = hw->soc->drive_get(hw, desc, &ret);
+ if (err)
+ return err;
+ } else {
+ err = -ENOTSUPP;
+ }
+ break;
+ case MTK_PIN_CONFIG_TDSEL:
+ case MTK_PIN_CONFIG_RDSEL:
+ reg = (param == MTK_PIN_CONFIG_TDSEL) ?
+ PINCTRL_PIN_REG_TDSEL : PINCTRL_PIN_REG_RDSEL;
+
+ err = mtk_hw_get_value(hw, desc, reg, &val);
+ if (err)
+ return err;
+
+ ret = val;
+
+ break;
+ case MTK_PIN_CONFIG_PU_ADV:
+ case MTK_PIN_CONFIG_PD_ADV:
+ if (hw->soc->adv_pull_get) {
+ bool pullup;
+
+ pullup = param == MTK_PIN_CONFIG_PU_ADV;
+ err = hw->soc->adv_pull_get(hw, desc, pullup, &ret);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ default:
+ return -ENOTSUPP;
+ }
+
+ *config = pinconf_to_config_packed(param, ret);
+
+ return 0;
+}
+
+static int mtk_pinconf_set(struct pinctrl_dev *pctldev, unsigned int pin,
+ enum pin_config_param param,
+ enum pin_config_param arg)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+ const struct mtk_pin_desc *desc;
+ int err = 0;
+ u32 reg;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[pin];
+
+ switch ((u32)param) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ if (hw->soc->bias_disable_set) {
+ err = hw->soc->bias_disable_set(hw, desc);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case PIN_CONFIG_BIAS_PULL_UP:
+ if (hw->soc->bias_set) {
+ err = hw->soc->bias_set(hw, desc, 1);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ if (hw->soc->bias_set) {
+ err = hw->soc->bias_set(hw, desc, 0);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case PIN_CONFIG_OUTPUT_ENABLE:
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_SMT,
+ MTK_DISABLE);
+ if (err)
+ goto err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DIR,
+ MTK_OUTPUT);
+ if (err)
+ goto err;
+ break;
+ case PIN_CONFIG_INPUT_ENABLE:
+ if (hw->soc->ies_present) {
+ mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_IES,
+ MTK_ENABLE);
+ }
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DIR,
+ MTK_INPUT);
+ if (err)
+ goto err;
+ break;
+ case PIN_CONFIG_SLEW_RATE:
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_SR,
+ arg);
+ if (err)
+ goto err;
+
+ break;
+ case PIN_CONFIG_OUTPUT:
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DIR,
+ MTK_OUTPUT);
+ if (err)
+ goto err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DO,
+ arg);
+ if (err)
+ goto err;
+ break;
+ case PIN_CONFIG_INPUT_SCHMITT_ENABLE:
+ /* arg = 1: Input mode & SMT enable ;
+ * arg = 0: Output mode & SMT disable
+ */
+ arg = arg ? 2 : 1;
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DIR,
+ arg & 1);
+ if (err)
+ goto err;
+
+ err = mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_SMT,
+ !!(arg & 2));
+ if (err)
+ goto err;
+ break;
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ if (hw->soc->drive_set) {
+ err = hw->soc->drive_set(hw, desc, arg);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ case MTK_PIN_CONFIG_TDSEL:
+ case MTK_PIN_CONFIG_RDSEL:
+ reg = (param == MTK_PIN_CONFIG_TDSEL) ?
+ PINCTRL_PIN_REG_TDSEL : PINCTRL_PIN_REG_RDSEL;
+
+ err = mtk_hw_set_value(hw, desc, reg, arg);
+ if (err)
+ goto err;
+ break;
+ case MTK_PIN_CONFIG_PU_ADV:
+ case MTK_PIN_CONFIG_PD_ADV:
+ if (hw->soc->adv_pull_set) {
+ bool pullup;
+
+ pullup = param == MTK_PIN_CONFIG_PU_ADV;
+ err = hw->soc->adv_pull_set(hw, desc, pullup,
+ arg);
+ if (err)
+ return err;
+ } else {
+ return -ENOTSUPP;
+ }
+ break;
+ default:
+ err = -ENOTSUPP;
+ }
+
+err:
+ return err;
+}
+
+static struct mtk_pinctrl_group *
+mtk_pctrl_find_group_by_pin(struct mtk_pinctrl *hw, u32 pin)
+{
+ int i;
+
+ for (i = 0; i < hw->soc->ngrps; i++) {
+ struct mtk_pinctrl_group *grp = hw->groups + i;
+
+ if (grp->pin == pin)
+ return grp;
+ }
+
+ return NULL;
+}
+
+static const struct mtk_func_desc *
+mtk_pctrl_find_function_by_pin(struct mtk_pinctrl *hw, u32 pin_num, u32 fnum)
+{
+ const struct mtk_pin_desc *pin = hw->soc->pins + pin_num;
+ const struct mtk_func_desc *func = pin->funcs;
+
+ while (func && func->name) {
+ if (func->muxval == fnum)
+ return func;
+ func++;
+ }
+
+ return NULL;
+}
+
+static bool mtk_pctrl_is_function_valid(struct mtk_pinctrl *hw, u32 pin_num,
+ u32 fnum)
+{
+ int i;
+
+ for (i = 0; i < hw->soc->npins; i++) {
+ const struct mtk_pin_desc *pin = hw->soc->pins + i;
+
+ if (pin->number == pin_num) {
+ const struct mtk_func_desc *func = pin->funcs;
+
+ while (func && func->name) {
+ if (func->muxval == fnum)
+ return true;
+ func++;
+ }
+
+ break;
+ }
+ }
+
+ return false;
+}
+
+static int mtk_pctrl_dt_node_to_map_func(struct mtk_pinctrl *pctl,
+ u32 pin, u32 fnum,
+ struct mtk_pinctrl_group *grp,
+ struct pinctrl_map **map,
+ unsigned *reserved_maps,
+ unsigned *num_maps)
+{
+ bool ret;
+
+ if (*num_maps == *reserved_maps)
+ return -ENOSPC;
+
+ (*map)[*num_maps].type = PIN_MAP_TYPE_MUX_GROUP;
+ (*map)[*num_maps].data.mux.group = grp->name;
+
+ ret = mtk_pctrl_is_function_valid(pctl, pin, fnum);
+ if (!ret) {
+ dev_err(pctl->dev, "invalid function %d on pin %d .\n",
+ fnum, pin);
+ return -EINVAL;
+ }
+
+ (*map)[*num_maps].data.mux.function = mtk_gpio_functions[fnum];
+ (*num_maps)++;
+
+ return 0;
+}
+
+static int mtk_pctrl_dt_subnode_to_map(struct pinctrl_dev *pctldev,
+ struct device_node *node,
+ struct pinctrl_map **map,
+ unsigned *reserved_maps,
+ unsigned *num_maps)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+ int num_pins, num_funcs, maps_per_pin, i, err;
+ struct mtk_pinctrl_group *grp;
+ unsigned int num_configs;
+ bool has_config = false;
+ unsigned long *configs;
+ u32 pinfunc, pin, func;
+ struct property *pins;
+ unsigned reserve = 0;
+
+ pins = of_find_property(node, "pinmux", NULL);
+ if (!pins) {
+ dev_err(hw->dev, "missing pins property in node %s .\n",
+ node->name);
+ return -EINVAL;
+ }
+
+ err = pinconf_generic_parse_dt_config(node, pctldev, &configs,
+ &num_configs);
+ if (err)
+ return err;
+
+ if (num_configs)
+ has_config = true;
+
+ num_pins = pins->length / sizeof(u32);
+ num_funcs = num_pins;
+ maps_per_pin = 0;
+ if (num_funcs)
+ maps_per_pin++;
+ if (has_config && num_pins >= 1)
+ maps_per_pin++;
+
+ if (!num_pins || !maps_per_pin) {
+ err = -EINVAL;
+ goto exit;
+ }
+
+ reserve = num_pins * maps_per_pin;
+
+ err = pinctrl_utils_reserve_map(pctldev, map, reserved_maps, num_maps,
+ reserve);
+ if (err < 0)
+ goto exit;
+
+ for (i = 0; i < num_pins; i++) {
+ err = of_property_read_u32_index(node, "pinmux", i, &pinfunc);
+ if (err)
+ goto exit;
+
+ pin = MTK_GET_PIN_NO(pinfunc);
+ func = MTK_GET_PIN_FUNC(pinfunc);
+
+ if (pin >= hw->soc->npins ||
+ func >= ARRAY_SIZE(mtk_gpio_functions)) {
+ dev_err(hw->dev, "invalid pins value.\n");
+ err = -EINVAL;
+ goto exit;
+ }
+
+ grp = mtk_pctrl_find_group_by_pin(hw, pin);
+ if (!grp) {
+ dev_err(hw->dev, "unable to match pin %d to group\n",
+ pin);
+ err = -EINVAL;
+ goto exit;
+ }
+
+ err = mtk_pctrl_dt_node_to_map_func(hw, pin, func, grp, map,
+ reserved_maps, num_maps);
+ if (err < 0)
+ goto exit;
+
+ if (has_config) {
+ err = pinctrl_utils_add_map_configs(pctldev, map,
+ reserved_maps,
+ num_maps,
+ grp->name,
+ configs,
+ num_configs,
+ PIN_MAP_TYPE_CONFIGS_GROUP);
+ if (err < 0)
+ goto exit;
+ }
+ }
+
+ err = 0;
+
+exit:
+ kfree(configs);
+ return err;
+}
+
+static int mtk_pctrl_dt_node_to_map(struct pinctrl_dev *pctldev,
+ struct device_node *np_config,
+ struct pinctrl_map **map,
+ unsigned *num_maps)
+{
+ struct device_node *np;
+ unsigned reserved_maps;
+ int ret;
+
+ *map = NULL;
+ *num_maps = 0;
+ reserved_maps = 0;
+
+ for_each_child_of_node(np_config, np) {
+ ret = mtk_pctrl_dt_subnode_to_map(pctldev, np, map,
+ &reserved_maps,
+ num_maps);
+ if (ret < 0) {
+ pinctrl_utils_free_map(pctldev, *map, *num_maps);
+ of_node_put(np);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static int mtk_pctrl_get_groups_count(struct pinctrl_dev *pctldev)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+
+ return hw->soc->ngrps;
+}
+
+static const char *mtk_pctrl_get_group_name(struct pinctrl_dev *pctldev,
+ unsigned group)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+
+ return hw->groups[group].name;
+}
+
+static int mtk_pctrl_get_group_pins(struct pinctrl_dev *pctldev,
+ unsigned group, const unsigned **pins,
+ unsigned *num_pins)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+
+ *pins = (unsigned *)&hw->groups[group].pin;
+ *num_pins = 1;
+
+ return 0;
+}
+
+static const struct pinctrl_ops mtk_pctlops = {
+ .dt_node_to_map = mtk_pctrl_dt_node_to_map,
+ .dt_free_map = pinctrl_utils_free_map,
+ .get_groups_count = mtk_pctrl_get_groups_count,
+ .get_group_name = mtk_pctrl_get_group_name,
+ .get_group_pins = mtk_pctrl_get_group_pins,
+};
+
+static int mtk_pmx_get_funcs_cnt(struct pinctrl_dev *pctldev)
+{
+ return ARRAY_SIZE(mtk_gpio_functions);
+}
+
+static const char *mtk_pmx_get_func_name(struct pinctrl_dev *pctldev,
+ unsigned selector)
+{
+ return mtk_gpio_functions[selector];
+}
+
+static int mtk_pmx_get_func_groups(struct pinctrl_dev *pctldev,
+ unsigned function,
+ const char * const **groups,
+ unsigned * const num_groups)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+
+ *groups = hw->grp_names;
+ *num_groups = hw->soc->ngrps;
+
+ return 0;
+}
+
+static int mtk_pmx_set_mux(struct pinctrl_dev *pctldev,
+ unsigned function,
+ unsigned group)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+ struct mtk_pinctrl_group *grp = hw->groups + group;
+ const struct mtk_func_desc *desc_func;
+ const struct mtk_pin_desc *desc;
+ bool ret;
+
+ ret = mtk_pctrl_is_function_valid(hw, grp->pin, function);
+ if (!ret) {
+ dev_err(hw->dev, "invalid function %d on group %d .\n",
+ function, group);
+ return -EINVAL;
+ }
+
+ desc_func = mtk_pctrl_find_function_by_pin(hw, grp->pin, function);
+ if (!desc_func)
+ return -EINVAL;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[grp->pin];
+ mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_MODE, desc_func->muxval);
+
+ return 0;
+}
+
+static const struct pinmux_ops mtk_pmxops = {
+ .get_functions_count = mtk_pmx_get_funcs_cnt,
+ .get_function_name = mtk_pmx_get_func_name,
+ .get_function_groups = mtk_pmx_get_func_groups,
+ .set_mux = mtk_pmx_set_mux,
+ .gpio_set_direction = mtk_pinmux_gpio_set_direction,
+ .gpio_request_enable = mtk_pinmux_gpio_request_enable,
+};
+
+static int mtk_pconf_group_get(struct pinctrl_dev *pctldev, unsigned group,
+ unsigned long *config)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+
+ *config = hw->groups[group].config;
+
+ return 0;
+}
+
+static int mtk_pconf_group_set(struct pinctrl_dev *pctldev, unsigned group,
+ unsigned long *configs, unsigned num_configs)
+{
+ struct mtk_pinctrl *hw = pinctrl_dev_get_drvdata(pctldev);
+ struct mtk_pinctrl_group *grp = &hw->groups[group];
+ int i, ret;
+
+ for (i = 0; i < num_configs; i++) {
+ ret = mtk_pinconf_set(pctldev, grp->pin,
+ pinconf_to_config_param(configs[i]),
+ pinconf_to_config_argument(configs[i]));
+ if (ret < 0)
+ return ret;
+
+ grp->config = configs[i];
+ }
+
+ return 0;
+}
+
+static const struct pinconf_ops mtk_confops = {
+ .pin_config_get = mtk_pinconf_get,
+ .pin_config_group_get = mtk_pconf_group_get,
+ .pin_config_group_set = mtk_pconf_group_set,
+};
+
+static struct pinctrl_desc mtk_desc = {
+ .name = PINCTRL_PINCTRL_DEV,
+ .pctlops = &mtk_pctlops,
+ .pmxops = &mtk_pmxops,
+ .confops = &mtk_confops,
+ .owner = THIS_MODULE,
+};
+
+static int mtk_gpio_get_direction(struct gpio_chip *chip, unsigned int gpio)
+{
+ struct mtk_pinctrl *hw = gpiochip_get_data(chip);
+ const struct mtk_pin_desc *desc;
+ int value, err;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[gpio];
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_DIR, &value);
+ if (err)
+ return err;
+
+ return !value;
+}
+
+static int mtk_gpio_get(struct gpio_chip *chip, unsigned int gpio)
+{
+ struct mtk_pinctrl *hw = gpiochip_get_data(chip);
+ const struct mtk_pin_desc *desc;
+ int value, err;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[gpio];
+
+ err = mtk_hw_get_value(hw, desc, PINCTRL_PIN_REG_DI, &value);
+ if (err)
+ return err;
+
+ return !!value;
+}
+
+static void mtk_gpio_set(struct gpio_chip *chip, unsigned int gpio, int value)
+{
+ struct mtk_pinctrl *hw = gpiochip_get_data(chip);
+ const struct mtk_pin_desc *desc;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[gpio];
+
+ mtk_hw_set_value(hw, desc, PINCTRL_PIN_REG_DO, !!value);
+}
+
+static int mtk_gpio_direction_input(struct gpio_chip *chip, unsigned int gpio)
+{
+ return pinctrl_gpio_direction_input(chip->base + gpio);
+}
+
+static int mtk_gpio_direction_output(struct gpio_chip *chip, unsigned int gpio,
+ int value)
+{
+ mtk_gpio_set(chip, gpio, value);
+
+ return pinctrl_gpio_direction_output(chip->base + gpio);
+}
+
+static int mtk_gpio_to_irq(struct gpio_chip *chip, unsigned int offset)
+{
+ struct mtk_pinctrl *hw = gpiochip_get_data(chip);
+ const struct mtk_pin_desc *desc;
+
+ if (!hw->eint)
+ return -ENOTSUPP;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[offset];
+
+ if (desc->eint.eint_n == EINT_NA)
+ return -ENOTSUPP;
+
+ return mtk_eint_find_irq(hw->eint, desc->eint.eint_n);
+}
+
+static int mtk_gpio_set_config(struct gpio_chip *chip, unsigned int offset,
+ unsigned long config)
+{
+ struct mtk_pinctrl *hw = gpiochip_get_data(chip);
+ const struct mtk_pin_desc *desc;
+ u32 debounce;
+
+ desc = (const struct mtk_pin_desc *)&hw->soc->pins[offset];
+
+ if (!hw->eint ||
+ pinconf_to_config_param(config) != PIN_CONFIG_INPUT_DEBOUNCE ||
+ desc->eint.eint_n == EINT_NA)
+ return -ENOTSUPP;
+
+ debounce = pinconf_to_config_argument(config);
+
+ return mtk_eint_set_debounce(hw->eint, desc->eint.eint_n, debounce);
+}
+
+static int mtk_build_gpiochip(struct mtk_pinctrl *hw, struct device_node *np)
+{
+ struct gpio_chip *chip = &hw->chip;
+ int ret;
+
+ chip->label = PINCTRL_PINCTRL_DEV;
+ chip->parent = hw->dev;
+ chip->request = gpiochip_generic_request;
+ chip->free = gpiochip_generic_free;
+ chip->get_direction = mtk_gpio_get_direction;
+ chip->direction_input = mtk_gpio_direction_input;
+ chip->direction_output = mtk_gpio_direction_output;
+ chip->get = mtk_gpio_get;
+ chip->set = mtk_gpio_set;
+ chip->to_irq = mtk_gpio_to_irq,
+ chip->set_config = mtk_gpio_set_config,
+ chip->base = -1;
+ chip->ngpio = hw->soc->npins;
+ chip->of_node = np;
+ chip->of_gpio_n_cells = 2;
+
+ ret = gpiochip_add_data(chip, hw);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static int mtk_pctrl_build_state(struct platform_device *pdev)
+{
+ struct mtk_pinctrl *hw = platform_get_drvdata(pdev);
+ int i;
+
+ /* Allocate groups */
+ hw->groups = devm_kmalloc_array(&pdev->dev, hw->soc->ngrps,
+ sizeof(*hw->groups), GFP_KERNEL);
+ if (!hw->groups)
+ return -ENOMEM;
+
+ /* We assume that one pin is one group, use pin name as group name. */
+ hw->grp_names = devm_kmalloc_array(&pdev->dev, hw->soc->ngrps,
+ sizeof(*hw->grp_names), GFP_KERNEL);
+ if (!hw->grp_names)
+ return -ENOMEM;
+
+ for (i = 0; i < hw->soc->npins; i++) {
+ const struct mtk_pin_desc *pin = hw->soc->pins + i;
+ struct mtk_pinctrl_group *group = hw->groups + i;
+
+ group->name = pin->name;
+ group->pin = pin->number;
+
+ hw->grp_names[i] = pin->name;
+ }
+
+ return 0;
+}
+
+int mtk_paris_pinctrl_probe(struct platform_device *pdev,
+ const struct mtk_pin_soc *soc)
+{
+ struct pinctrl_pin_desc *pins;
+ struct mtk_pinctrl *hw;
+ struct resource *res;
+ int err, i;
+
+ hw = devm_kzalloc(&pdev->dev, sizeof(*hw), GFP_KERNEL);
+ if (!hw)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, hw);
+ hw->soc = soc;
+ hw->dev = &pdev->dev;
+
+ if (!hw->soc->nbase_names) {
+ dev_err(&pdev->dev,
+ "SoC should be assigned at least one register base\n");
+ return -EINVAL;
+ }
+
+ hw->base = devm_kmalloc_array(&pdev->dev, hw->soc->nbase_names,
+ sizeof(*hw->base), GFP_KERNEL);
+ if (!hw->base)
+ return -ENOMEM;
+
+ for (i = 0; i < hw->soc->nbase_names; i++) {
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
+ hw->soc->base_names[i]);
+ if (!res) {
+ dev_err(&pdev->dev, "missing IO resource\n");
+ return -ENXIO;
+ }
+
+ hw->base[i] = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(hw->base[i]))
+ return PTR_ERR(hw->base[i]);
+ }
+
+ hw->nbase = hw->soc->nbase_names;
+
+ err = mtk_pctrl_build_state(pdev);
+ if (err) {
+ dev_err(&pdev->dev, "build state failed: %d\n", err);
+ return -EINVAL;
+ }
+
+ /* Copy from internal struct mtk_pin_desc to register to the core */
+ pins = devm_kmalloc_array(&pdev->dev, hw->soc->npins, sizeof(*pins),
+ GFP_KERNEL);
+ if (!pins)
+ return -ENOMEM;
+
+ for (i = 0; i < hw->soc->npins; i++) {
+ pins[i].number = hw->soc->pins[i].number;
+ pins[i].name = hw->soc->pins[i].name;
+ }
+
+ /* Setup pins descriptions per SoC types */
+ mtk_desc.pins = (const struct pinctrl_pin_desc *)pins;
+ mtk_desc.npins = hw->soc->npins;
+ mtk_desc.num_custom_params = ARRAY_SIZE(mtk_custom_bindings);
+ mtk_desc.custom_params = mtk_custom_bindings;
+#ifdef CONFIG_DEBUG_FS
+ mtk_desc.custom_conf_items = mtk_conf_items;
+#endif
+
+ err = devm_pinctrl_register_and_init(&pdev->dev, &mtk_desc, hw,
+ &hw->pctrl);
+ if (err)
+ return err;
+
+ err = pinctrl_enable(hw->pctrl);
+ if (err)
+ return err;
+
+ err = mtk_build_eint(hw, pdev);
+ if (err)
+ dev_warn(&pdev->dev,
+ "Failed to add EINT, but pinctrl still can work\n");
+
+ /* Build gpiochip should be after pinctrl_enable is done */
+ err = mtk_build_gpiochip(hw, pdev->dev.of_node);
+ if (err) {
+ dev_err(&pdev->dev, "Failed to add gpio_chip\n");
+ return err;
+ }
+
+ platform_set_drvdata(pdev, hw);
+
+ return 0;
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 MediaTek Inc.
+ *
+ * Author: Sean Wang <sean.wang@mediatek.com>
+ * Zhiyong Tao <zhiyong.tao@mediatek.com>
+ * Hongzhou.Yang <hongzhou.yang@mediatek.com>
+ */
+#ifndef __PINCTRL_PARIS_H
+#define __PINCTRL_PARIS_H
+
+#include <linux/io.h>
+#include <linux/init.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <linux/pinctrl/pinctrl.h>
+#include <linux/pinctrl/pinmux.h>
+#include <linux/pinctrl/pinconf.h>
+#include <linux/pinctrl/pinconf-generic.h>
+
+#include "../core.h"
+#include "../pinconf.h"
+#include "../pinctrl-utils.h"
+#include "../pinmux.h"
+#include "mtk-eint.h"
+#include "pinctrl-mtk-common-v2.h"
+
+#define MTK_RANGE(_a) { .range = (_a), .nranges = ARRAY_SIZE(_a), }
+
+#define MTK_EINT_FUNCTION(_eintmux, _eintnum) \
+ { \
+ .eint_m = _eintmux, \
+ .eint_n = _eintnum, \
+ }
+
+#define MTK_FUNCTION(_val, _name) \
+ { \
+ .muxval = _val, \
+ .name = _name, \
+ }
+
+#define MTK_PIN(_number, _name, _eint, _drv_n, ...) { \
+ .number = _number, \
+ .name = _name, \
+ .eint = _eint, \
+ .drv_n = _drv_n, \
+ .funcs = (struct mtk_func_desc[]){ \
+ __VA_ARGS__, { } }, \
+ }
+
+#define PINCTRL_PIN_GROUP(name, id) \
+ { \
+ name, \
+ id##_pins, \
+ ARRAY_SIZE(id##_pins), \
+ id##_funcs, \
+ }
+
+int mtk_paris_pinctrl_probe(struct platform_device *pdev,
+ const struct mtk_pin_soc *soc);
+
+#endif /* __PINCTRL_PARIS_H */
config PINCTRL_MESON_AXG_PMX
bool
+config PINCTRL_MESON_G12A
+ bool "Meson g12a Soc pinctrl driver"
+ depends on ARM64
+ select PINCTRL_MESON_AXG_PMX
+ default y
+
endif
obj-$(CONFIG_PINCTRL_MESON_GXL) += pinctrl-meson-gxl.o
obj-$(CONFIG_PINCTRL_MESON_AXG_PMX) += pinctrl-meson-axg-pmx.o
obj-$(CONFIG_PINCTRL_MESON_AXG) += pinctrl-meson-axg.o
+obj-$(CONFIG_PINCTRL_MESON_G12A) += pinctrl-meson-g12a.o
--- /dev/null
+// SPDX-License-Identifier: (GPL-2.0+ or MIT)
+/*
+ * Pin controller and GPIO driver for Amlogic Meson G12A SoC.
+ *
+ * Copyright (c) 2018 Amlogic, Inc. All rights reserved.
+ * Author: Xingyu Chen <xingyu.chen@amlogic.com>
+ * Author: Yixun Lan <yixun.lan@amlogic.com>
+ */
+
+#include <dt-bindings/gpio/meson-g12a-gpio.h>
+#include "pinctrl-meson.h"
+#include "pinctrl-meson-axg-pmx.h"
+
+static const struct pinctrl_pin_desc meson_g12a_periphs_pins[] = {
+ MESON_PIN(GPIOZ_0),
+ MESON_PIN(GPIOZ_1),
+ MESON_PIN(GPIOZ_2),
+ MESON_PIN(GPIOZ_3),
+ MESON_PIN(GPIOZ_4),
+ MESON_PIN(GPIOZ_5),
+ MESON_PIN(GPIOZ_6),
+ MESON_PIN(GPIOZ_7),
+ MESON_PIN(GPIOZ_8),
+ MESON_PIN(GPIOZ_9),
+ MESON_PIN(GPIOZ_10),
+ MESON_PIN(GPIOZ_11),
+ MESON_PIN(GPIOZ_12),
+ MESON_PIN(GPIOZ_13),
+ MESON_PIN(GPIOZ_14),
+ MESON_PIN(GPIOZ_15),
+ MESON_PIN(GPIOH_0),
+ MESON_PIN(GPIOH_1),
+ MESON_PIN(GPIOH_2),
+ MESON_PIN(GPIOH_3),
+ MESON_PIN(GPIOH_4),
+ MESON_PIN(GPIOH_5),
+ MESON_PIN(GPIOH_6),
+ MESON_PIN(GPIOH_7),
+ MESON_PIN(GPIOH_8),
+ MESON_PIN(BOOT_0),
+ MESON_PIN(BOOT_1),
+ MESON_PIN(BOOT_2),
+ MESON_PIN(BOOT_3),
+ MESON_PIN(BOOT_4),
+ MESON_PIN(BOOT_5),
+ MESON_PIN(BOOT_6),
+ MESON_PIN(BOOT_7),
+ MESON_PIN(BOOT_8),
+ MESON_PIN(BOOT_9),
+ MESON_PIN(BOOT_10),
+ MESON_PIN(BOOT_11),
+ MESON_PIN(BOOT_12),
+ MESON_PIN(BOOT_13),
+ MESON_PIN(BOOT_14),
+ MESON_PIN(BOOT_15),
+ MESON_PIN(GPIOC_0),
+ MESON_PIN(GPIOC_1),
+ MESON_PIN(GPIOC_2),
+ MESON_PIN(GPIOC_3),
+ MESON_PIN(GPIOC_4),
+ MESON_PIN(GPIOC_5),
+ MESON_PIN(GPIOC_6),
+ MESON_PIN(GPIOC_7),
+ MESON_PIN(GPIOA_0),
+ MESON_PIN(GPIOA_1),
+ MESON_PIN(GPIOA_2),
+ MESON_PIN(GPIOA_3),
+ MESON_PIN(GPIOA_4),
+ MESON_PIN(GPIOA_5),
+ MESON_PIN(GPIOA_6),
+ MESON_PIN(GPIOA_7),
+ MESON_PIN(GPIOA_8),
+ MESON_PIN(GPIOA_9),
+ MESON_PIN(GPIOA_10),
+ MESON_PIN(GPIOA_11),
+ MESON_PIN(GPIOA_12),
+ MESON_PIN(GPIOA_13),
+ MESON_PIN(GPIOA_14),
+ MESON_PIN(GPIOA_15),
+ MESON_PIN(GPIOX_0),
+ MESON_PIN(GPIOX_1),
+ MESON_PIN(GPIOX_2),
+ MESON_PIN(GPIOX_3),
+ MESON_PIN(GPIOX_4),
+ MESON_PIN(GPIOX_5),
+ MESON_PIN(GPIOX_6),
+ MESON_PIN(GPIOX_7),
+ MESON_PIN(GPIOX_8),
+ MESON_PIN(GPIOX_9),
+ MESON_PIN(GPIOX_10),
+ MESON_PIN(GPIOX_11),
+ MESON_PIN(GPIOX_12),
+ MESON_PIN(GPIOX_13),
+ MESON_PIN(GPIOX_14),
+ MESON_PIN(GPIOX_15),
+ MESON_PIN(GPIOX_16),
+ MESON_PIN(GPIOX_17),
+ MESON_PIN(GPIOX_18),
+ MESON_PIN(GPIOX_19),
+};
+
+static const struct pinctrl_pin_desc meson_g12a_aobus_pins[] = {
+ MESON_PIN(GPIOAO_0),
+ MESON_PIN(GPIOAO_1),
+ MESON_PIN(GPIOAO_2),
+ MESON_PIN(GPIOAO_3),
+ MESON_PIN(GPIOAO_4),
+ MESON_PIN(GPIOAO_5),
+ MESON_PIN(GPIOAO_6),
+ MESON_PIN(GPIOAO_7),
+ MESON_PIN(GPIOAO_8),
+ MESON_PIN(GPIOAO_9),
+ MESON_PIN(GPIOAO_10),
+ MESON_PIN(GPIOAO_11),
+ MESON_PIN(GPIOE_0),
+ MESON_PIN(GPIOE_1),
+ MESON_PIN(GPIOE_2),
+};
+
+/* emmc */
+static const unsigned int emmc_nand_d0_pins[] = { BOOT_0 };
+static const unsigned int emmc_nand_d1_pins[] = { BOOT_1 };
+static const unsigned int emmc_nand_d2_pins[] = { BOOT_2 };
+static const unsigned int emmc_nand_d3_pins[] = { BOOT_3 };
+static const unsigned int emmc_nand_d4_pins[] = { BOOT_4 };
+static const unsigned int emmc_nand_d5_pins[] = { BOOT_5 };
+static const unsigned int emmc_nand_d6_pins[] = { BOOT_6 };
+static const unsigned int emmc_nand_d7_pins[] = { BOOT_7 };
+static const unsigned int emmc_clk_pins[] = { BOOT_8 };
+static const unsigned int emmc_cmd_pins[] = { BOOT_10 };
+static const unsigned int emmc_nand_ds_pins[] = { BOOT_13 };
+
+/* nand */
+static const unsigned int nand_wen_clk_pins[] = { BOOT_8 };
+static const unsigned int nand_ale_pins[] = { BOOT_9 };
+static const unsigned int nand_cle_pins[] = { BOOT_10 };
+static const unsigned int nand_ce0_pins[] = { BOOT_11 };
+static const unsigned int nand_ren_wr_pins[] = { BOOT_12 };
+static const unsigned int nand_rb0_pins[] = { BOOT_14 };
+static const unsigned int nand_ce1_pins[] = { BOOT_15 };
+
+/* nor */
+static const unsigned int nor_hold_pins[] = { BOOT_3 };
+static const unsigned int nor_d_pins[] = { BOOT_4 };
+static const unsigned int nor_q_pins[] = { BOOT_5 };
+static const unsigned int nor_c_pins[] = { BOOT_6 };
+static const unsigned int nor_wp_pins[] = { BOOT_7 };
+static const unsigned int nor_cs_pins[] = { BOOT_14 };
+
+/* sdio */
+static const unsigned int sdio_d0_pins[] = { GPIOX_0 };
+static const unsigned int sdio_d1_pins[] = { GPIOX_1 };
+static const unsigned int sdio_d2_pins[] = { GPIOX_2 };
+static const unsigned int sdio_d3_pins[] = { GPIOX_3 };
+static const unsigned int sdio_clk_pins[] = { GPIOX_4 };
+static const unsigned int sdio_cmd_pins[] = { GPIOX_5 };
+
+/* sdcard */
+static const unsigned int sdcard_d0_c_pins[] = { GPIOC_0 };
+static const unsigned int sdcard_d1_c_pins[] = { GPIOC_1 };
+static const unsigned int sdcard_d2_c_pins[] = { GPIOC_2 };
+static const unsigned int sdcard_d3_c_pins[] = { GPIOC_3 };
+static const unsigned int sdcard_clk_c_pins[] = { GPIOC_4 };
+static const unsigned int sdcard_cmd_c_pins[] = { GPIOC_5 };
+
+static const unsigned int sdcard_d0_z_pins[] = { GPIOZ_2 };
+static const unsigned int sdcard_d1_z_pins[] = { GPIOZ_3 };
+static const unsigned int sdcard_d2_z_pins[] = { GPIOZ_4 };
+static const unsigned int sdcard_d3_z_pins[] = { GPIOZ_5 };
+static const unsigned int sdcard_clk_z_pins[] = { GPIOZ_6 };
+static const unsigned int sdcard_cmd_z_pins[] = { GPIOZ_7 };
+
+/* spi0 */
+static const unsigned int spi0_mosi_c_pins[] = { GPIOC_0 };
+static const unsigned int spi0_miso_c_pins[] = { GPIOC_1 };
+static const unsigned int spi0_ss0_c_pins[] = { GPIOC_2 };
+static const unsigned int spi0_clk_c_pins[] = { GPIOC_3 };
+
+static const unsigned int spi0_mosi_x_pins[] = { GPIOX_8 };
+static const unsigned int spi0_miso_x_pins[] = { GPIOX_9 };
+static const unsigned int spi0_ss0_x_pins[] = { GPIOX_10 };
+static const unsigned int spi0_clk_x_pins[] = { GPIOX_11 };
+
+/* spi1 */
+static const unsigned int spi1_mosi_pins[] = { GPIOH_4 };
+static const unsigned int spi1_miso_pins[] = { GPIOH_5 };
+static const unsigned int spi1_ss0_pins[] = { GPIOH_6 };
+static const unsigned int spi1_clk_pins[] = { GPIOH_7 };
+
+/* i2c0 */
+static const unsigned int i2c0_sda_c_pins[] = { GPIOC_5 };
+static const unsigned int i2c0_sck_c_pins[] = { GPIOC_6 };
+static const unsigned int i2c0_sda_z0_pins[] = { GPIOZ_0 };
+static const unsigned int i2c0_sck_z1_pins[] = { GPIOZ_1 };
+static const unsigned int i2c0_sda_z7_pins[] = { GPIOZ_7 };
+static const unsigned int i2c0_sck_z8_pins[] = { GPIOZ_8 };
+
+/* i2c1 */
+static const unsigned int i2c1_sda_x_pins[] = { GPIOX_10 };
+static const unsigned int i2c1_sck_x_pins[] = { GPIOX_11 };
+static const unsigned int i2c1_sda_h2_pins[] = { GPIOH_2 };
+static const unsigned int i2c1_sck_h3_pins[] = { GPIOH_3 };
+static const unsigned int i2c1_sda_h6_pins[] = { GPIOH_6 };
+static const unsigned int i2c1_sck_h7_pins[] = { GPIOH_7 };
+
+/* i2c2 */
+static const unsigned int i2c2_sda_x_pins[] = { GPIOX_17 };
+static const unsigned int i2c2_sck_x_pins[] = { GPIOX_18 };
+static const unsigned int i2c2_sda_z_pins[] = { GPIOZ_14 };
+static const unsigned int i2c2_sck_z_pins[] = { GPIOZ_15 };
+
+/* i2c3 */
+static const unsigned int i2c3_sda_h_pins[] = { GPIOH_0 };
+static const unsigned int i2c3_sck_h_pins[] = { GPIOH_1 };
+static const unsigned int i2c3_sda_a_pins[] = { GPIOA_14 };
+static const unsigned int i2c3_sck_a_pins[] = { GPIOA_15 };
+
+/* uart_a */
+static const unsigned int uart_a_tx_pins[] = { GPIOX_12 };
+static const unsigned int uart_a_rx_pins[] = { GPIOX_13 };
+static const unsigned int uart_a_cts_pins[] = { GPIOX_14 };
+static const unsigned int uart_a_rts_pins[] = { GPIOX_15 };
+
+/* uart_b */
+static const unsigned int uart_b_tx_pins[] = { GPIOX_6 };
+static const unsigned int uart_b_rx_pins[] = { GPIOX_7 };
+
+/* uart_c */
+static const unsigned int uart_c_rts_pins[] = { GPIOH_4 };
+static const unsigned int uart_c_cts_pins[] = { GPIOH_5 };
+static const unsigned int uart_c_rx_pins[] = { GPIOH_6 };
+static const unsigned int uart_c_tx_pins[] = { GPIOH_7 };
+
+/* uart_ao_a_c */
+static const unsigned int uart_ao_a_rx_c_pins[] = { GPIOC_2 };
+static const unsigned int uart_ao_a_tx_c_pins[] = { GPIOC_3 };
+
+/* iso7816 */
+static const unsigned int iso7816_clk_c_pins[] = { GPIOC_5 };
+static const unsigned int iso7816_data_c_pins[] = { GPIOC_6 };
+static const unsigned int iso7816_clk_x_pins[] = { GPIOX_8 };
+static const unsigned int iso7816_data_x_pins[] = { GPIOX_9 };
+static const unsigned int iso7816_clk_h_pins[] = { GPIOH_6 };
+static const unsigned int iso7816_data_h_pins[] = { GPIOH_7 };
+static const unsigned int iso7816_clk_z_pins[] = { GPIOZ_0 };
+static const unsigned int iso7816_data_z_pins[] = { GPIOZ_1 };
+
+/* eth */
+static const unsigned int eth_mdio_pins[] = { GPIOZ_0 };
+static const unsigned int eth_mdc_pins[] = { GPIOZ_1 };
+static const unsigned int eth_rgmii_rx_clk_pins[] = { GPIOZ_2 };
+static const unsigned int eth_rx_dv_pins[] = { GPIOZ_3 };
+static const unsigned int eth_rxd0_pins[] = { GPIOZ_4 };
+static const unsigned int eth_rxd1_pins[] = { GPIOZ_5 };
+static const unsigned int eth_rxd2_rgmii_pins[] = { GPIOZ_6 };
+static const unsigned int eth_rxd3_rgmii_pins[] = { GPIOZ_7 };
+static const unsigned int eth_rgmii_tx_clk_pins[] = { GPIOZ_8 };
+static const unsigned int eth_txen_pins[] = { GPIOZ_9 };
+static const unsigned int eth_txd0_pins[] = { GPIOZ_10 };
+static const unsigned int eth_txd1_pins[] = { GPIOZ_11 };
+static const unsigned int eth_txd2_rgmii_pins[] = { GPIOZ_12 };
+static const unsigned int eth_txd3_rgmii_pins[] = { GPIOZ_13 };
+static const unsigned int eth_link_led_pins[] = { GPIOZ_14 };
+static const unsigned int eth_act_led_pins[] = { GPIOZ_15 };
+
+/* pwm_a */
+static const unsigned int pwm_a_pins[] = { GPIOX_6 };
+
+/* pwm_b */
+static const unsigned int pwm_b_x7_pins[] = { GPIOX_7 };
+static const unsigned int pwm_b_x19_pins[] = { GPIOX_19 };
+
+/* pwm_c */
+static const unsigned int pwm_c_c_pins[] = { GPIOC_4 };
+static const unsigned int pwm_c_x5_pins[] = { GPIOX_5 };
+static const unsigned int pwm_c_x8_pins[] = { GPIOX_8 };
+
+/* pwm_d */
+static const unsigned int pwm_d_x3_pins[] = { GPIOX_3 };
+static const unsigned int pwm_d_x6_pins[] = { GPIOX_6 };
+
+/* pwm_e */
+static const unsigned int pwm_e_pins[] = { GPIOX_16 };
+
+/* pwm_f */
+static const unsigned int pwm_f_x_pins[] = { GPIOX_7 };
+static const unsigned int pwm_f_h_pins[] = { GPIOH_5 };
+
+/* cec_ao */
+static const unsigned int cec_ao_a_h_pins[] = { GPIOH_3 };
+static const unsigned int cec_ao_b_h_pins[] = { GPIOH_3 };
+
+/* jtag_b */
+static const unsigned int jtag_b_tdo_pins[] = { GPIOC_0 };
+static const unsigned int jtag_b_tdi_pins[] = { GPIOC_1 };
+static const unsigned int jtag_b_clk_pins[] = { GPIOC_4 };
+static const unsigned int jtag_b_tms_pins[] = { GPIOC_5 };
+
+/* bt565_a */
+static const unsigned int bt565_a_vs_pins[] = { GPIOZ_0 };
+static const unsigned int bt565_a_hs_pins[] = { GPIOZ_1 };
+static const unsigned int bt565_a_clk_pins[] = { GPIOZ_3 };
+static const unsigned int bt565_a_din0_pins[] = { GPIOZ_4 };
+static const unsigned int bt565_a_din1_pins[] = { GPIOZ_5 };
+static const unsigned int bt565_a_din2_pins[] = { GPIOZ_6 };
+static const unsigned int bt565_a_din3_pins[] = { GPIOZ_7 };
+static const unsigned int bt565_a_din4_pins[] = { GPIOZ_8 };
+static const unsigned int bt565_a_din5_pins[] = { GPIOZ_9 };
+static const unsigned int bt565_a_din6_pins[] = { GPIOZ_10 };
+static const unsigned int bt565_a_din7_pins[] = { GPIOZ_11 };
+
+/* tsin_a */
+static const unsigned int tsin_a_valid_pins[] = { GPIOX_2 };
+static const unsigned int tsin_a_sop_pins[] = { GPIOX_1 };
+static const unsigned int tsin_a_din0_pins[] = { GPIOX_0 };
+static const unsigned int tsin_a_clk_pins[] = { GPIOX_3 };
+
+/* tsin_b */
+static const unsigned int tsin_b_valid_x_pins[] = { GPIOX_9 };
+static const unsigned int tsin_b_sop_x_pins[] = { GPIOX_8 };
+static const unsigned int tsin_b_din0_x_pins[] = { GPIOX_10 };
+static const unsigned int tsin_b_clk_x_pins[] = { GPIOX_11 };
+
+static const unsigned int tsin_b_valid_z_pins[] = { GPIOZ_2 };
+static const unsigned int tsin_b_sop_z_pins[] = { GPIOZ_3 };
+static const unsigned int tsin_b_din0_z_pins[] = { GPIOZ_4 };
+static const unsigned int tsin_b_clk_z_pins[] = { GPIOZ_5 };
+
+static const unsigned int tsin_b_fail_pins[] = { GPIOZ_6 };
+static const unsigned int tsin_b_din1_pins[] = { GPIOZ_7 };
+static const unsigned int tsin_b_din2_pins[] = { GPIOZ_8 };
+static const unsigned int tsin_b_din3_pins[] = { GPIOZ_9 };
+static const unsigned int tsin_b_din4_pins[] = { GPIOZ_10 };
+static const unsigned int tsin_b_din5_pins[] = { GPIOZ_11 };
+static const unsigned int tsin_b_din6_pins[] = { GPIOZ_12 };
+static const unsigned int tsin_b_din7_pins[] = { GPIOZ_13 };
+
+/* hdmitx */
+static const unsigned int hdmitx_sda_pins[] = { GPIOH_0 };
+static const unsigned int hdmitx_sck_pins[] = { GPIOH_1 };
+static const unsigned int hdmitx_hpd_in_pins[] = { GPIOH_2 };
+
+/* pdm */
+static const unsigned int pdm_din0_c_pins[] = { GPIOC_0 };
+static const unsigned int pdm_din1_c_pins[] = { GPIOC_1 };
+static const unsigned int pdm_din2_c_pins[] = { GPIOC_2 };
+static const unsigned int pdm_din3_c_pins[] = { GPIOC_3 };
+static const unsigned int pdm_dclk_c_pins[] = { GPIOC_4 };
+
+static const unsigned int pdm_din0_x_pins[] = { GPIOX_0 };
+static const unsigned int pdm_din1_x_pins[] = { GPIOX_1 };
+static const unsigned int pdm_din2_x_pins[] = { GPIOX_2 };
+static const unsigned int pdm_din3_x_pins[] = { GPIOX_3 };
+static const unsigned int pdm_dclk_x_pins[] = { GPIOX_4 };
+
+static const unsigned int pdm_din0_z_pins[] = { GPIOZ_2 };
+static const unsigned int pdm_din1_z_pins[] = { GPIOZ_3 };
+static const unsigned int pdm_din2_z_pins[] = { GPIOZ_4 };
+static const unsigned int pdm_din3_z_pins[] = { GPIOZ_5 };
+static const unsigned int pdm_dclk_z_pins[] = { GPIOZ_6 };
+
+static const unsigned int pdm_din0_a_pins[] = { GPIOA_8 };
+static const unsigned int pdm_din1_a_pins[] = { GPIOA_9 };
+static const unsigned int pdm_din2_a_pins[] = { GPIOA_6 };
+static const unsigned int pdm_din3_a_pins[] = { GPIOA_5 };
+static const unsigned int pdm_dclk_a_pins[] = { GPIOA_7 };
+
+/* spdif_in */
+static const unsigned int spdif_in_h_pins[] = { GPIOH_5 };
+static const unsigned int spdif_in_a10_pins[] = { GPIOA_10 };
+static const unsigned int spdif_in_a12_pins[] = { GPIOA_12 };
+
+/* spdif_out */
+static const unsigned int spdif_out_h_pins[] = { GPIOH_4 };
+static const unsigned int spdif_out_a11_pins[] = { GPIOA_11 };
+static const unsigned int spdif_out_a13_pins[] = { GPIOA_13 };
+
+/* mclk0 */
+static const unsigned int mclk0_a_pins[] = { GPIOA_0 };
+
+/* mclk1 */
+static const unsigned int mclk1_x_pins[] = { GPIOX_5 };
+static const unsigned int mclk1_z_pins[] = { GPIOZ_8 };
+static const unsigned int mclk1_a_pins[] = { GPIOA_11 };
+
+/* tdm */
+static const unsigned int tdm_a_slv_sclk_pins[] = { GPIOX_11 };
+static const unsigned int tdm_a_slv_fs_pins[] = { GPIOX_10 };
+static const unsigned int tdm_a_sclk_pins[] = { GPIOX_11 };
+static const unsigned int tdm_a_fs_pins[] = { GPIOX_10 };
+static const unsigned int tdm_a_din0_pins[] = { GPIOX_9 };
+static const unsigned int tdm_a_din1_pins[] = { GPIOX_8 };
+static const unsigned int tdm_a_dout0_pins[] = { GPIOX_9 };
+static const unsigned int tdm_a_dout1_pins[] = { GPIOX_8 };
+
+static const unsigned int tdm_b_slv_sclk_pins[] = { GPIOA_1 };
+static const unsigned int tdm_b_slv_fs_pins[] = { GPIOA_2 };
+static const unsigned int tdm_b_sclk_pins[] = { GPIOA_1 };
+static const unsigned int tdm_b_fs_pins[] = { GPIOA_2 };
+static const unsigned int tdm_b_din0_pins[] = { GPIOA_3 };
+static const unsigned int tdm_b_din1_pins[] = { GPIOA_4 };
+static const unsigned int tdm_b_din2_pins[] = { GPIOA_5 };
+static const unsigned int tdm_b_din3_a_pins[] = { GPIOA_6 };
+static const unsigned int tdm_b_din3_h_pins[] = { GPIOH_5 };
+static const unsigned int tdm_b_dout0_pins[] = { GPIOA_3 };
+static const unsigned int tdm_b_dout1_pins[] = { GPIOA_4 };
+static const unsigned int tdm_b_dout2_pins[] = { GPIOA_5 };
+static const unsigned int tdm_b_dout3_a_pins[] = { GPIOA_6 };
+static const unsigned int tdm_b_dout3_h_pins[] = { GPIOH_5 };
+
+static const unsigned int tdm_c_slv_sclk_a_pins[] = { GPIOA_12 };
+static const unsigned int tdm_c_slv_fs_a_pins[] = { GPIOA_13 };
+static const unsigned int tdm_c_slv_sclk_z_pins[] = { GPIOZ_7 };
+static const unsigned int tdm_c_slv_fs_z_pins[] = { GPIOZ_6 };
+static const unsigned int tdm_c_sclk_a_pins[] = { GPIOA_12 };
+static const unsigned int tdm_c_fs_a_pins[] = { GPIOA_13 };
+static const unsigned int tdm_c_sclk_z_pins[] = { GPIOZ_7 };
+static const unsigned int tdm_c_fs_z_pins[] = { GPIOZ_6 };
+static const unsigned int tdm_c_din0_a_pins[] = { GPIOA_10 };
+static const unsigned int tdm_c_din1_a_pins[] = { GPIOA_9 };
+static const unsigned int tdm_c_din2_a_pins[] = { GPIOA_8 };
+static const unsigned int tdm_c_din3_a_pins[] = { GPIOA_7 };
+static const unsigned int tdm_c_din0_z_pins[] = { GPIOZ_2 };
+static const unsigned int tdm_c_din1_z_pins[] = { GPIOZ_3 };
+static const unsigned int tdm_c_din2_z_pins[] = { GPIOZ_4 };
+static const unsigned int tdm_c_din3_z_pins[] = { GPIOZ_5 };
+static const unsigned int tdm_c_dout0_a_pins[] = { GPIOA_10 };
+static const unsigned int tdm_c_dout1_a_pins[] = { GPIOA_9 };
+static const unsigned int tdm_c_dout2_a_pins[] = { GPIOA_8 };
+static const unsigned int tdm_c_dout3_a_pins[] = { GPIOA_7 };
+static const unsigned int tdm_c_dout0_z_pins[] = { GPIOZ_2 };
+static const unsigned int tdm_c_dout1_z_pins[] = { GPIOZ_3 };
+static const unsigned int tdm_c_dout2_z_pins[] = { GPIOZ_4 };
+static const unsigned int tdm_c_dout3_z_pins[] = { GPIOZ_5 };
+
+static struct meson_pmx_group meson_g12a_periphs_groups[] = {
+ GPIO_GROUP(GPIOZ_0),
+ GPIO_GROUP(GPIOZ_1),
+ GPIO_GROUP(GPIOZ_2),
+ GPIO_GROUP(GPIOZ_3),
+ GPIO_GROUP(GPIOZ_4),
+ GPIO_GROUP(GPIOZ_5),
+ GPIO_GROUP(GPIOZ_6),
+ GPIO_GROUP(GPIOZ_7),
+ GPIO_GROUP(GPIOZ_8),
+ GPIO_GROUP(GPIOZ_9),
+ GPIO_GROUP(GPIOZ_10),
+ GPIO_GROUP(GPIOZ_11),
+ GPIO_GROUP(GPIOZ_12),
+ GPIO_GROUP(GPIOZ_13),
+ GPIO_GROUP(GPIOZ_14),
+ GPIO_GROUP(GPIOZ_15),
+ GPIO_GROUP(GPIOH_0),
+ GPIO_GROUP(GPIOH_1),
+ GPIO_GROUP(GPIOH_2),
+ GPIO_GROUP(GPIOH_3),
+ GPIO_GROUP(GPIOH_4),
+ GPIO_GROUP(GPIOH_5),
+ GPIO_GROUP(GPIOH_6),
+ GPIO_GROUP(GPIOH_7),
+ GPIO_GROUP(GPIOH_8),
+ GPIO_GROUP(BOOT_0),
+ GPIO_GROUP(BOOT_1),
+ GPIO_GROUP(BOOT_2),
+ GPIO_GROUP(BOOT_3),
+ GPIO_GROUP(BOOT_4),
+ GPIO_GROUP(BOOT_5),
+ GPIO_GROUP(BOOT_6),
+ GPIO_GROUP(BOOT_7),
+ GPIO_GROUP(BOOT_8),
+ GPIO_GROUP(BOOT_9),
+ GPIO_GROUP(BOOT_10),
+ GPIO_GROUP(BOOT_11),
+ GPIO_GROUP(BOOT_12),
+ GPIO_GROUP(BOOT_13),
+ GPIO_GROUP(BOOT_14),
+ GPIO_GROUP(BOOT_15),
+ GPIO_GROUP(GPIOC_0),
+ GPIO_GROUP(GPIOC_1),
+ GPIO_GROUP(GPIOC_2),
+ GPIO_GROUP(GPIOC_3),
+ GPIO_GROUP(GPIOC_4),
+ GPIO_GROUP(GPIOC_5),
+ GPIO_GROUP(GPIOC_6),
+ GPIO_GROUP(GPIOC_7),
+ GPIO_GROUP(GPIOA_0),
+ GPIO_GROUP(GPIOA_1),
+ GPIO_GROUP(GPIOA_2),
+ GPIO_GROUP(GPIOA_3),
+ GPIO_GROUP(GPIOA_4),
+ GPIO_GROUP(GPIOA_5),
+ GPIO_GROUP(GPIOA_6),
+ GPIO_GROUP(GPIOA_7),
+ GPIO_GROUP(GPIOA_8),
+ GPIO_GROUP(GPIOA_9),
+ GPIO_GROUP(GPIOA_10),
+ GPIO_GROUP(GPIOA_11),
+ GPIO_GROUP(GPIOA_12),
+ GPIO_GROUP(GPIOA_13),
+ GPIO_GROUP(GPIOA_14),
+ GPIO_GROUP(GPIOA_15),
+ GPIO_GROUP(GPIOX_0),
+ GPIO_GROUP(GPIOX_1),
+ GPIO_GROUP(GPIOX_2),
+ GPIO_GROUP(GPIOX_3),
+ GPIO_GROUP(GPIOX_4),
+ GPIO_GROUP(GPIOX_5),
+ GPIO_GROUP(GPIOX_6),
+ GPIO_GROUP(GPIOX_7),
+ GPIO_GROUP(GPIOX_8),
+ GPIO_GROUP(GPIOX_9),
+ GPIO_GROUP(GPIOX_10),
+ GPIO_GROUP(GPIOX_11),
+ GPIO_GROUP(GPIOX_12),
+ GPIO_GROUP(GPIOX_13),
+ GPIO_GROUP(GPIOX_14),
+ GPIO_GROUP(GPIOX_15),
+ GPIO_GROUP(GPIOX_16),
+ GPIO_GROUP(GPIOX_17),
+ GPIO_GROUP(GPIOX_18),
+ GPIO_GROUP(GPIOX_19),
+
+ /* bank BOOT */
+ GROUP(emmc_nand_d0, 1),
+ GROUP(emmc_nand_d1, 1),
+ GROUP(emmc_nand_d2, 1),
+ GROUP(emmc_nand_d3, 1),
+ GROUP(emmc_nand_d4, 1),
+ GROUP(emmc_nand_d5, 1),
+ GROUP(emmc_nand_d6, 1),
+ GROUP(emmc_nand_d7, 1),
+ GROUP(emmc_clk, 1),
+ GROUP(emmc_cmd, 1),
+ GROUP(emmc_nand_ds, 1),
+ GROUP(nand_ce0, 2),
+ GROUP(nand_ale, 2),
+ GROUP(nand_cle, 2),
+ GROUP(nand_wen_clk, 2),
+ GROUP(nand_ren_wr, 2),
+ GROUP(nand_rb0, 2),
+ GROUP(nand_ce1, 2),
+ GROUP(nor_hold, 3),
+ GROUP(nor_d, 3),
+ GROUP(nor_q, 3),
+ GROUP(nor_c, 3),
+ GROUP(nor_wp, 3),
+ GROUP(nor_cs, 3),
+
+ /* bank GPIOZ */
+ GROUP(sdcard_d0_z, 5),
+ GROUP(sdcard_d1_z, 5),
+ GROUP(sdcard_d2_z, 5),
+ GROUP(sdcard_d3_z, 5),
+ GROUP(sdcard_clk_z, 5),
+ GROUP(sdcard_cmd_z, 5),
+ GROUP(i2c0_sda_z0, 4),
+ GROUP(i2c0_sck_z1, 4),
+ GROUP(i2c0_sda_z7, 7),
+ GROUP(i2c0_sck_z8, 7),
+ GROUP(i2c2_sda_z, 3),
+ GROUP(i2c2_sck_z, 3),
+ GROUP(iso7816_clk_z, 3),
+ GROUP(iso7816_data_z, 3),
+ GROUP(eth_mdio, 1),
+ GROUP(eth_mdc, 1),
+ GROUP(eth_rgmii_rx_clk, 1),
+ GROUP(eth_rx_dv, 1),
+ GROUP(eth_rxd0, 1),
+ GROUP(eth_rxd1, 1),
+ GROUP(eth_rxd2_rgmii, 1),
+ GROUP(eth_rxd3_rgmii, 1),
+ GROUP(eth_rgmii_tx_clk, 1),
+ GROUP(eth_txen, 1),
+ GROUP(eth_txd0, 1),
+ GROUP(eth_txd1, 1),
+ GROUP(eth_txd2_rgmii, 1),
+ GROUP(eth_txd3_rgmii, 1),
+ GROUP(eth_link_led, 1),
+ GROUP(eth_act_led, 1),
+ GROUP(bt565_a_vs, 2),
+ GROUP(bt565_a_hs, 2),
+ GROUP(bt565_a_clk, 2),
+ GROUP(bt565_a_din0, 2),
+ GROUP(bt565_a_din1, 2),
+ GROUP(bt565_a_din2, 2),
+ GROUP(bt565_a_din3, 2),
+ GROUP(bt565_a_din4, 2),
+ GROUP(bt565_a_din5, 2),
+ GROUP(bt565_a_din6, 2),
+ GROUP(bt565_a_din7, 2),
+ GROUP(tsin_b_valid_z, 3),
+ GROUP(tsin_b_sop_z, 3),
+ GROUP(tsin_b_din0_z, 3),
+ GROUP(tsin_b_clk_z, 3),
+ GROUP(tsin_b_fail, 3),
+ GROUP(tsin_b_din1, 3),
+ GROUP(tsin_b_din2, 3),
+ GROUP(tsin_b_din3, 3),
+ GROUP(tsin_b_din4, 3),
+ GROUP(tsin_b_din5, 3),
+ GROUP(tsin_b_din6, 3),
+ GROUP(tsin_b_din7, 3),
+ GROUP(pdm_din0_z, 7),
+ GROUP(pdm_din1_z, 7),
+ GROUP(pdm_din2_z, 7),
+ GROUP(pdm_din3_z, 7),
+ GROUP(pdm_dclk_z, 7),
+ GROUP(tdm_c_slv_sclk_z, 6),
+ GROUP(tdm_c_slv_fs_z, 6),
+ GROUP(tdm_c_din0_z, 6),
+ GROUP(tdm_c_din1_z, 6),
+ GROUP(tdm_c_din2_z, 6),
+ GROUP(tdm_c_din3_z, 6),
+ GROUP(tdm_c_sclk_z, 4),
+ GROUP(tdm_c_fs_z, 4),
+ GROUP(tdm_c_dout0_z, 4),
+ GROUP(tdm_c_dout1_z, 4),
+ GROUP(tdm_c_dout2_z, 4),
+ GROUP(tdm_c_dout3_z, 4),
+ GROUP(mclk1_z, 4),
+
+ /* bank GPIOX */
+ GROUP(sdio_d0, 1),
+ GROUP(sdio_d1, 1),
+ GROUP(sdio_d2, 1),
+ GROUP(sdio_d3, 1),
+ GROUP(sdio_clk, 1),
+ GROUP(sdio_cmd, 1),
+ GROUP(spi0_mosi_x, 4),
+ GROUP(spi0_miso_x, 4),
+ GROUP(spi0_ss0_x, 4),
+ GROUP(spi0_clk_x, 4),
+ GROUP(i2c1_sda_x, 5),
+ GROUP(i2c1_sck_x, 5),
+ GROUP(i2c2_sda_x, 1),
+ GROUP(i2c2_sck_x, 1),
+ GROUP(uart_a_tx, 1),
+ GROUP(uart_a_rx, 1),
+ GROUP(uart_a_cts, 1),
+ GROUP(uart_a_rts, 1),
+ GROUP(uart_b_tx, 2),
+ GROUP(uart_b_rx, 2),
+ GROUP(iso7816_clk_x, 6),
+ GROUP(iso7816_data_x, 6),
+ GROUP(pwm_a, 1),
+ GROUP(pwm_b_x7, 4),
+ GROUP(pwm_b_x19, 1),
+ GROUP(pwm_c_x5, 4),
+ GROUP(pwm_c_x8, 5),
+ GROUP(pwm_d_x3, 4),
+ GROUP(pwm_d_x6, 4),
+ GROUP(pwm_e, 1),
+ GROUP(pwm_f_x, 1),
+ GROUP(tsin_a_valid, 3),
+ GROUP(tsin_a_sop, 3),
+ GROUP(tsin_a_din0, 3),
+ GROUP(tsin_a_clk, 3),
+ GROUP(tsin_b_valid_x, 3),
+ GROUP(tsin_b_sop_x, 3),
+ GROUP(tsin_b_din0_x, 3),
+ GROUP(tsin_b_clk_x, 3),
+ GROUP(pdm_din0_x, 2),
+ GROUP(pdm_din1_x, 2),
+ GROUP(pdm_din2_x, 2),
+ GROUP(pdm_din3_x, 2),
+ GROUP(pdm_dclk_x, 2),
+ GROUP(tdm_a_slv_sclk, 2),
+ GROUP(tdm_a_slv_fs, 2),
+ GROUP(tdm_a_din0, 2),
+ GROUP(tdm_a_din1, 2),
+ GROUP(tdm_a_sclk, 1),
+ GROUP(tdm_a_fs, 1),
+ GROUP(tdm_a_dout0, 1),
+ GROUP(tdm_a_dout1, 1),
+ GROUP(mclk1_x, 2),
+
+ /* bank GPIOC */
+ GROUP(sdcard_d0_c, 1),
+ GROUP(sdcard_d1_c, 1),
+ GROUP(sdcard_d2_c, 1),
+ GROUP(sdcard_d3_c, 1),
+ GROUP(sdcard_clk_c, 1),
+ GROUP(sdcard_cmd_c, 1),
+ GROUP(spi0_mosi_c, 5),
+ GROUP(spi0_miso_c, 5),
+ GROUP(spi0_ss0_c, 5),
+ GROUP(spi0_clk_c, 5),
+ GROUP(i2c0_sda_c, 3),
+ GROUP(i2c0_sck_c, 3),
+ GROUP(uart_ao_a_rx_c, 2),
+ GROUP(uart_ao_a_tx_c, 2),
+ GROUP(iso7816_clk_c, 5),
+ GROUP(iso7816_data_c, 5),
+ GROUP(pwm_c_c, 5),
+ GROUP(jtag_b_tdo, 2),
+ GROUP(jtag_b_tdi, 2),
+ GROUP(jtag_b_clk, 2),
+ GROUP(jtag_b_tms, 2),
+ GROUP(pdm_din0_c, 4),
+ GROUP(pdm_din1_c, 4),
+ GROUP(pdm_din2_c, 4),
+ GROUP(pdm_din3_c, 4),
+ GROUP(pdm_dclk_c, 4),
+
+ /* bank GPIOH */
+ GROUP(spi1_mosi, 3),
+ GROUP(spi1_miso, 3),
+ GROUP(spi1_ss0, 3),
+ GROUP(spi1_clk, 3),
+ GROUP(i2c1_sda_h2, 2),
+ GROUP(i2c1_sck_h3, 2),
+ GROUP(i2c1_sda_h6, 4),
+ GROUP(i2c1_sck_h7, 4),
+ GROUP(i2c3_sda_h, 2),
+ GROUP(i2c3_sck_h, 2),
+ GROUP(uart_c_tx, 2),
+ GROUP(uart_c_rx, 2),
+ GROUP(uart_c_cts, 2),
+ GROUP(uart_c_rts, 2),
+ GROUP(iso7816_clk_h, 1),
+ GROUP(iso7816_data_h, 1),
+ GROUP(pwm_f_h, 4),
+ GROUP(cec_ao_a_h, 4),
+ GROUP(cec_ao_b_h, 5),
+ GROUP(hdmitx_sda, 1),
+ GROUP(hdmitx_sck, 1),
+ GROUP(hdmitx_hpd_in, 1),
+ GROUP(spdif_out_h, 1),
+ GROUP(spdif_in_h, 1),
+ GROUP(tdm_b_din3_h, 6),
+ GROUP(tdm_b_dout3_h, 5),
+
+ /* bank GPIOA */
+ GROUP(i2c3_sda_a, 2),
+ GROUP(i2c3_sck_a, 2),
+ GROUP(pdm_din0_a, 1),
+ GROUP(pdm_din1_a, 1),
+ GROUP(pdm_din2_a, 1),
+ GROUP(pdm_din3_a, 1),
+ GROUP(pdm_dclk_a, 1),
+ GROUP(spdif_in_a10, 1),
+ GROUP(spdif_in_a12, 1),
+ GROUP(spdif_out_a11, 1),
+ GROUP(spdif_out_a13, 1),
+ GROUP(tdm_b_slv_sclk, 2),
+ GROUP(tdm_b_slv_fs, 2),
+ GROUP(tdm_b_din0, 2),
+ GROUP(tdm_b_din1, 2),
+ GROUP(tdm_b_din2, 2),
+ GROUP(tdm_b_din3_a, 2),
+ GROUP(tdm_b_sclk, 1),
+ GROUP(tdm_b_fs, 1),
+ GROUP(tdm_b_dout0, 1),
+ GROUP(tdm_b_dout1, 1),
+ GROUP(tdm_b_dout2, 3),
+ GROUP(tdm_b_dout3_a, 3),
+ GROUP(tdm_c_slv_sclk_a, 3),
+ GROUP(tdm_c_slv_fs_a, 3),
+ GROUP(tdm_c_din0_a, 3),
+ GROUP(tdm_c_din1_a, 3),
+ GROUP(tdm_c_din2_a, 3),
+ GROUP(tdm_c_din3_a, 3),
+ GROUP(tdm_c_sclk_a, 2),
+ GROUP(tdm_c_fs_a, 2),
+ GROUP(tdm_c_dout0_a, 2),
+ GROUP(tdm_c_dout1_a, 2),
+ GROUP(tdm_c_dout2_a, 2),
+ GROUP(tdm_c_dout3_a, 2),
+ GROUP(mclk0_a, 1),
+ GROUP(mclk1_a, 2),
+};
+
+/* uart_ao_a */
+static const unsigned int uart_ao_a_tx_pins[] = { GPIOAO_0 };
+static const unsigned int uart_ao_a_rx_pins[] = { GPIOAO_1 };
+static const unsigned int uart_ao_a_cts_pins[] = { GPIOE_0 };
+static const unsigned int uart_ao_a_rts_pins[] = { GPIOE_1 };
+
+/* uart_ao_b */
+static const unsigned int uart_ao_b_tx_2_pins[] = { GPIOAO_2 };
+static const unsigned int uart_ao_b_rx_3_pins[] = { GPIOAO_3 };
+static const unsigned int uart_ao_b_tx_8_pins[] = { GPIOAO_8 };
+static const unsigned int uart_ao_b_rx_9_pins[] = { GPIOAO_9 };
+static const unsigned int uart_ao_b_cts_pins[] = { GPIOE_0 };
+static const unsigned int uart_ao_b_rts_pins[] = { GPIOE_1 };
+
+/* i2c_ao */
+static const unsigned int i2c_ao_sck_pins[] = { GPIOAO_2 };
+static const unsigned int i2c_ao_sda_pins[] = { GPIOAO_3 };
+
+static const unsigned int i2c_ao_sck_e_pins[] = { GPIOE_0 };
+static const unsigned int i2c_ao_sda_e_pins[] = { GPIOE_1 };
+
+/* i2c_ao_slave */
+static const unsigned int i2c_ao_slave_sck_pins[] = { GPIOAO_2 };
+static const unsigned int i2c_ao_slave_sda_pins[] = { GPIOAO_3 };
+
+/* ir_in */
+static const unsigned int remote_ao_input_pins[] = { GPIOAO_5 };
+
+/* ir_out */
+static const unsigned int remote_ao_out_pins[] = { GPIOAO_4 };
+
+/* pwm_ao_a */
+static const unsigned int pwm_ao_a_pins[] = { GPIOAO_11 };
+static const unsigned int pwm_ao_a_hiz_pins[] = { GPIOAO_11 };
+
+/* pwm_ao_b */
+static const unsigned int pwm_ao_b_pins[] = { GPIOE_0 };
+
+/* pwm_ao_c */
+static const unsigned int pwm_ao_c_4_pins[] = { GPIOAO_4 };
+static const unsigned int pwm_ao_c_hiz_pins[] = { GPIOAO_4 };
+static const unsigned int pwm_ao_c_6_pins[] = { GPIOAO_6 };
+
+/* pwm_ao_d */
+static const unsigned int pwm_ao_d_5_pins[] = { GPIOAO_5 };
+static const unsigned int pwm_ao_d_10_pins[] = { GPIOAO_10 };
+static const unsigned int pwm_ao_d_e_pins[] = { GPIOE_1 };
+
+/* jtag_a */
+static const unsigned int jtag_a_tdi_pins[] = { GPIOAO_8 };
+static const unsigned int jtag_a_tdo_pins[] = { GPIOAO_9 };
+static const unsigned int jtag_a_clk_pins[] = { GPIOAO_6 };
+static const unsigned int jtag_a_tms_pins[] = { GPIOAO_7 };
+
+/* cec_ao */
+static const unsigned int cec_ao_a_pins[] = { GPIOAO_10 };
+static const unsigned int cec_ao_b_pins[] = { GPIOAO_10 };
+
+/* tsin_ao_a */
+static const unsigned int tsin_ao_asop_pins[] = { GPIOAO_6 };
+static const unsigned int tsin_ao_adin0_pins[] = { GPIOAO_7 };
+static const unsigned int tsin_ao_aclk_pins[] = { GPIOAO_8 };
+static const unsigned int tsin_ao_a_valid_pins[] = { GPIOAO_9 };
+
+/* spdif_ao_out */
+static const unsigned int spdif_ao_out_pins[] = { GPIOAO_10 };
+
+/* tdm_ao_b */
+static const unsigned int tdm_ao_b_slv_fs_pins[] = { GPIOAO_7 };
+static const unsigned int tdm_ao_b_slv_sclk_pins[] = { GPIOAO_8 };
+static const unsigned int tdm_ao_b_fs_pins[] = { GPIOAO_7 };
+static const unsigned int tdm_ao_b_sclk_pins[] = { GPIOAO_8 };
+static const unsigned int tdm_ao_b_din0_pins[] = { GPIOAO_4 };
+static const unsigned int tdm_ao_b_din1_pins[] = { GPIOAO_10 };
+static const unsigned int tdm_ao_b_din2_pins[] = { GPIOAO_6 };
+static const unsigned int tdm_ao_b_dout0_pins[] = { GPIOAO_4 };
+static const unsigned int tdm_ao_b_dout1_pins[] = { GPIOAO_10 };
+static const unsigned int tdm_ao_b_dout2_pins[] = { GPIOAO_6 };
+
+/* mclk0_ao */
+static const unsigned int mclk0_ao_pins[] = { GPIOAO_9 };
+
+static struct meson_pmx_group meson_g12a_aobus_groups[] = {
+ GPIO_GROUP(GPIOAO_0),
+ GPIO_GROUP(GPIOAO_1),
+ GPIO_GROUP(GPIOAO_2),
+ GPIO_GROUP(GPIOAO_3),
+ GPIO_GROUP(GPIOAO_4),
+ GPIO_GROUP(GPIOAO_5),
+ GPIO_GROUP(GPIOAO_6),
+ GPIO_GROUP(GPIOAO_7),
+ GPIO_GROUP(GPIOAO_8),
+ GPIO_GROUP(GPIOAO_9),
+ GPIO_GROUP(GPIOAO_10),
+ GPIO_GROUP(GPIOAO_11),
+ GPIO_GROUP(GPIOE_0),
+ GPIO_GROUP(GPIOE_1),
+ GPIO_GROUP(GPIOE_2),
+
+ /* bank AO */
+ GROUP(uart_ao_a_tx, 1),
+ GROUP(uart_ao_a_rx, 1),
+ GROUP(uart_ao_a_cts, 1),
+ GROUP(uart_ao_a_rts, 1),
+ GROUP(uart_ao_b_tx_2, 2),
+ GROUP(uart_ao_b_rx_3, 2),
+ GROUP(uart_ao_b_tx_8, 3),
+ GROUP(uart_ao_b_rx_9, 3),
+ GROUP(uart_ao_b_cts, 2),
+ GROUP(uart_ao_b_rts, 2),
+ GROUP(i2c_ao_sck, 1),
+ GROUP(i2c_ao_sda, 1),
+ GROUP(i2c_ao_sck_e, 4),
+ GROUP(i2c_ao_sda_e, 4),
+ GROUP(i2c_ao_slave_sck, 3),
+ GROUP(i2c_ao_slave_sda, 3),
+ GROUP(remote_ao_input, 1),
+ GROUP(remote_ao_out, 1),
+ GROUP(pwm_ao_a, 3),
+ GROUP(pwm_ao_a_hiz, 2),
+ GROUP(pwm_ao_b, 3),
+ GROUP(pwm_ao_c_4, 3),
+ GROUP(pwm_ao_c_hiz, 4),
+ GROUP(pwm_ao_c_6, 3),
+ GROUP(pwm_ao_d_5, 3),
+ GROUP(pwm_ao_d_10, 3),
+ GROUP(pwm_ao_d_e, 3),
+ GROUP(jtag_a_tdi, 1),
+ GROUP(jtag_a_tdo, 1),
+ GROUP(jtag_a_clk, 1),
+ GROUP(jtag_a_tms, 1),
+ GROUP(cec_ao_a, 1),
+ GROUP(cec_ao_b, 2),
+ GROUP(tsin_ao_asop, 4),
+ GROUP(tsin_ao_adin0, 4),
+ GROUP(tsin_ao_aclk, 4),
+ GROUP(tsin_ao_a_valid, 4),
+ GROUP(spdif_ao_out, 4),
+ GROUP(tdm_ao_b_dout0, 5),
+ GROUP(tdm_ao_b_dout1, 5),
+ GROUP(tdm_ao_b_dout2, 5),
+ GROUP(tdm_ao_b_fs, 5),
+ GROUP(tdm_ao_b_sclk, 5),
+ GROUP(tdm_ao_b_din0, 6),
+ GROUP(tdm_ao_b_din1, 6),
+ GROUP(tdm_ao_b_din2, 6),
+ GROUP(tdm_ao_b_slv_fs, 6),
+ GROUP(tdm_ao_b_slv_sclk, 6),
+ GROUP(mclk0_ao, 5),
+};
+
+static const char * const gpio_periphs_groups[] = {
+ "GPIOZ_0", "GPIOZ_1", "GPIOZ_2", "GPIOZ_3", "GPIOZ_4",
+ "GPIOZ_5", "GPIOZ_6", "GPIOZ_7", "GPIOZ_8", "GPIOZ_9",
+ "GPIOZ_10", "GPIOZ_11", "GPIOZ_12", "GPIOZ_13", "GPIOZ_14",
+ "GPIOZ_15",
+
+ "GPIOH_0", "GPIOH_1", "GPIOH_2", "GPIOH_3", "GPIOH_4",
+ "GPIOH_5", "GPIOH_6", "GPIOH_7", "GPIOH_8",
+
+ "BOOT_0", "BOOT_1", "BOOT_2", "BOOT_3", "BOOT_4",
+ "BOOT_5", "BOOT_6", "BOOT_7", "BOOT_8", "BOOT_9",
+ "BOOT_10", "BOOT_11", "BOOT_12", "BOOT_13", "BOOT_14",
+ "BOOT_15",
+
+ "GPIOC_0", "GPIOC_1", "GPIOC_2", "GPIOC_3", "GPIOC_4",
+ "GPIOC_5", "GPIOC_6", "GPIOC_7",
+
+ "GPIOA_0", "GPIOA_1", "GPIOA_2", "GPIOA_3", "GPIOA_4",
+ "GPIOA_5", "GPIOA_6", "GPIOA_7", "GPIOA_8", "GPIOA_9",
+ "GPIOA_10", "GPIOA_11", "GPIOA_12", "GPIOA_13", "GPIOA_14",
+ "GPIOA_15",
+
+ "GPIOX_0", "GPIOX_1", "GPIOX_2", "GPIOX_3", "GPIOX_4",
+ "GPIOX_5", "GPIOX_6", "GPIOX_7", "GPIOX_8", "GPIOX_9",
+ "GPIOX_10", "GPIOX_11", "GPIOX_12", "GPIOX_13", "GPIOX_14",
+ "GPIOX_15", "GPIOX_16", "GPIOX_17", "GPIOX_18", "GPIOX_19",
+};
+
+static const char * const emmc_groups[] = {
+ "emmc_nand_d0", "emmc_nand_d1", "emmc_nand_d2",
+ "emmc_nand_d3", "emmc_nand_d4", "emmc_nand_d5",
+ "emmc_nand_d6", "emmc_nand_d7",
+ "emmc_clk", "emmc_cmd", "emmc_nand_ds",
+};
+
+static const char * const nand_groups[] = {
+ "emmc_nand_d0", "emmc_nand_d1", "emmc_nand_d2",
+ "emmc_nand_d3", "emmc_nand_d4", "emmc_nand_d5",
+ "emmc_nand_d6", "emmc_nand_d7",
+ "nand_ce0", "nand_ale", "nand_cle",
+ "nand_wen_clk", "nand_ren_wr", "nand_rb0",
+ "emmc_nand_ds", "nand_ce1",
+};
+
+static const char * const nor_groups[] = {
+ "nor_d", "nor_q", "nor_c", "nor_cs",
+ "nor_hold", "nor_wp",
+};
+
+static const char * const sdio_groups[] = {
+ "sdio_d0", "sdio_d1", "sdio_d2", "sdio_d3",
+ "sdio_cmd", "sdio_clk", "sdio_dummy",
+};
+
+static const char * const sdcard_groups[] = {
+ "sdcard_d0_c", "sdcard_d1_c", "sdcard_d2_c", "sdcard_d3_c",
+ "sdcard_clk_c", "sdcard_cmd_c",
+ "sdcard_d0_z", "sdcard_d1_z", "sdcard_d2_z", "sdcard_d3_z",
+ "sdcard_clk_z", "sdcard_cmd_z",
+};
+
+static const char * const spi0_groups[] = {
+ "spi0_mosi_c", "spi0_miso_c", "spi0_ss0_c", "spi0_clk_c",
+ "spi0_mosi_x", "spi0_miso_x", "spi0_ss0_x", "spi0_clk_x",
+};
+
+static const char * const spi1_groups[] = {
+ "spi1_mosi", "spi1_miso", "spi1_ss0", "spi1_clk",
+};
+
+static const char * const i2c0_groups[] = {
+ "i2c0_sda_c", "i2c0_sck_c",
+ "i2c0_sda_z0", "i2c0_sck_z1",
+ "i2c0_sda_z7", "i2c0_sck_z8",
+};
+
+static const char * const i2c1_groups[] = {
+ "i2c1_sda_x", "i2c1_sck_x",
+ "i2c1_sda_h2", "i2c1_sck_h3",
+ "i2c1_sda_h6", "i2c1_sck_h7",
+};
+
+static const char * const i2c2_groups[] = {
+ "i2c2_sda_x", "i2c2_sck_x",
+ "i2c2_sda_z", "i2c2_sck_z",
+};
+
+static const char * const i2c3_groups[] = {
+ "i2c3_sda_h", "i2c3_sck_h",
+ "i2c3_sda_a", "i2c3_sck_a",
+};
+
+static const char * const uart_a_groups[] = {
+ "uart_a_tx", "uart_a_rx", "uart_a_cts", "uart_a_rts",
+};
+
+static const char * const uart_b_groups[] = {
+ "uart_b_tx", "uart_b_rx",
+};
+
+static const char * const uart_c_groups[] = {
+ "uart_c_tx", "uart_c_rx", "uart_c_cts", "uart_c_rts",
+};
+
+static const char * const uart_ao_a_c_groups[] = {
+ "uart_ao_a_rx_c", "uart_ao_a_tx_c",
+};
+
+static const char * const iso7816_groups[] = {
+ "iso7816_clk_c", "iso7816_data_c",
+ "iso7816_clk_x", "iso7816_data_x",
+ "iso7816_clk_h", "iso7816_data_h",
+ "iso7816_clk_z", "iso7816_data_z",
+};
+
+static const char * const eth_groups[] = {
+ "eth_rxd2_rgmii", "eth_rxd3_rgmii", "eth_rgmii_tx_clk",
+ "eth_txd2_rgmii", "eth_txd3_rgmii", "eth_rgmii_rx_clk",
+ "eth_txd0", "eth_txd1", "eth_txen", "eth_mdc",
+ "eth_rxd0", "eth_rxd1", "eth_rx_dv", "eth_mdio",
+ "eth_link_led", "eth_act_led",
+};
+
+static const char * const pwm_a_groups[] = {
+ "pwm_a",
+};
+
+static const char * const pwm_b_groups[] = {
+ "pwm_b_x7", "pwm_b_x19",
+};
+
+static const char * const pwm_c_groups[] = {
+ "pwm_c_c", "pwm_c_x5", "pwm_c_x8",
+};
+
+static const char * const pwm_d_groups[] = {
+ "pwm_d_x3", "pwm_d_x6",
+};
+
+static const char * const pwm_e_groups[] = {
+ "pwm_e",
+};
+
+static const char * const pwm_f_groups[] = {
+ "pwm_f_x", "pwm_f_h",
+};
+
+static const char * const cec_ao_a_h_groups[] = {
+ "cec_ao_a_h",
+};
+
+static const char * const cec_ao_b_h_groups[] = {
+ "cec_ao_b_h",
+};
+
+static const char * const jtag_b_groups[] = {
+ "jtag_b_tdi", "jtag_b_tdo", "jtag_b_clk", "jtag_b_tms",
+};
+
+static const char * const bt565_a_groups[] = {
+ "bt565_a_vs", "bt565_a_hs", "bt565_a_clk",
+ "bt565_a_din0", "bt565_a_din1", "bt565_a_din2",
+ "bt565_a_din3", "bt565_a_din4", "bt565_a_din5",
+ "bt565_a_din6", "bt565_a_din7",
+};
+
+static const char * const tsin_a_groups[] = {
+ "tsin_a_valid", "tsin_a_sop", "tsin_a_din0",
+ "tsin_a_clk",
+};
+
+static const char * const tsin_b_groups[] = {
+ "tsin_b_valid_x", "tsin_b_sop_x", "tsin_b_din0_x", "tsin_b_clk_x",
+ "tsin_b_valid_z", "tsin_b_sop_z", "tsin_b_din0_z", "tsin_b_clk_z",
+ "tsin_b_fail", "tsin_b_din1", "tsin_b_din2", "tsin_b_din3",
+ "tsin_b_din4", "tsin_b_din5", "tsin_b_din6", "tsin_b_din7",
+};
+
+static const char * const hdmitx_groups[] = {
+ "hdmitx_sda", "hdmitx_sck", "hdmitx_hpd_in",
+};
+
+static const char * const pdm_groups[] = {
+ "pdm_din0_c", "pdm_din1_c", "pdm_din2_c", "pdm_din3_c",
+ "pdm_dclk_c",
+ "pdm_din0_x", "pdm_din1_x", "pdm_din2_x", "pdm_din3_x",
+ "pdm_dclk_x",
+ "pdm_din0_z", "pdm_din1_z", "pdm_din2_z", "pdm_din3_z",
+ "pdm_dclk_z",
+ "pdm_din0_a", "pdm_din1_a", "pdm_din2_a", "pdm_din3_a",
+ "pdm_dclk_a",
+};
+
+static const char * const spdif_in_groups[] = {
+ "spdif_in_h", "spdif_in_a10", "spdif_in_a12",
+};
+
+static const char * const spdif_out_groups[] = {
+ "spdif_out_h", "spdif_out_a11", "spdif_out_a13",
+};
+
+static const char * const mclk0_groups[] = {
+ "mclk0_a",
+};
+
+static const char * const mclk1_groups[] = {
+ "mclk1_x", "mclk1_z", "mclk1_a",
+};
+
+static const char * const tdm_a_groups[] = {
+ "tdm_a_slv_sclk", "tdm_a_slv_fs", "tdm_a_sclk", "tdm_a_fs",
+ "tdm_a_din0", "tdm_a_din1", "tdm_a_dout0", "tdm_a_dout1",
+};
+
+static const char * const tdm_b_groups[] = {
+ "tdm_b_slv_sclk", "tdm_b_slv_fs", "tdm_b_sclk", "tdm_b_fs",
+ "tdm_b_din0", "tdm_b_din1", "tdm_b_din2",
+ "tdm_b_din3_a", "tdm_b_din3_h",
+ "tdm_b_dout0", "tdm_b_dout1", "tdm_b_dout2",
+ "tdm_b_dout3_a", "tdm_b_dout3_h",
+};
+
+static const char * const tdm_c_groups[] = {
+ "tdm_c_slv_sclk_a", "tdm_c_slv_fs_a",
+ "tdm_c_slv_sclk_z", "tdm_c_slv_fs_z",
+ "tdm_c_sclk_a", "tdm_c_fs_a",
+ "tdm_c_sclk_z", "tdm_c_fs_z",
+ "tdm_c_din0_a", "tdm_c_din1_a",
+ "tdm_c_din2_a", "tdm_c_din3_a",
+ "tdm_c_din0_z", "tdm_c_din1_z",
+ "tdm_c_din2_z", "tdm_c_din3_z",
+ "tdm_c_dout0_a", "tdm_c_dout1_a",
+ "tdm_c_dout2_a", "tdm_c_dout3_a",
+ "tdm_c_dout0_z", "tdm_c_dout1_z",
+ "tdm_c_dout2_z", "tdm_c_dout3_z",
+};
+
+static const char * const gpio_aobus_groups[] = {
+ "GPIOAO_0", "GPIOAO_1", "GPIOAO_2", "GPIOAO_3", "GPIOAO_4",
+ "GPIOAO_5", "GPIOAO_6", "GPIOAO_7", "GPIOAO_8", "GPIOAO_9",
+ "GPIOAO_10", "GPIOAO_11", "GPIOE_0", "GPIOE_1", "GPIOE_2",
+};
+
+static const char * const uart_ao_a_groups[] = {
+ "uart_ao_a_tx", "uart_ao_a_rx",
+ "uart_ao_a_cts", "uart_ao_a_rts",
+};
+
+static const char * const uart_ao_b_groups[] = {
+ "uart_ao_b_tx_2", "uart_ao_b_rx_3",
+ "uart_ao_b_tx_8", "uart_ao_b_rx_9",
+ "uart_ao_b_cts", "uart_ao_b_rts",
+};
+
+static const char * const i2c_ao_groups[] = {
+ "i2c_ao_sck", "i2c_ao_sda",
+ "i2c_ao_sck_e", "i2c_ao_sda_e",
+};
+
+static const char * const i2c_ao_slave_groups[] = {
+ "i2c_ao_slave_sck", "i2c_ao_slave_sda",
+};
+
+static const char * const remote_ao_input_groups[] = {
+ "remote_ao_input",
+};
+
+static const char * const remote_ao_out_groups[] = {
+ "remote_ao_out",
+};
+
+static const char * const pwm_ao_a_groups[] = {
+ "pwm_ao_a", "pwm_ao_a_hiz",
+};
+
+static const char * const pwm_ao_b_groups[] = {
+ "pwm_ao_b",
+};
+
+static const char * const pwm_ao_c_groups[] = {
+ "pwm_ao_c_4", "pwm_ao_c_hiz",
+ "pwm_ao_c_6",
+};
+
+static const char * const pwm_ao_d_groups[] = {
+ "pwm_ao_d_5", "pwm_ao_d_10", "pwm_ao_d_e",
+};
+
+static const char * const jtag_a_groups[] = {
+ "jtag_a_tdi", "jtag_a_tdo", "jtag_a_clk", "jtag_a_tms",
+};
+
+static const char * const cec_ao_a_groups[] = {
+ "cec_ao_a",
+};
+
+static const char * const cec_ao_b_groups[] = {
+ "cec_ao_b",
+};
+
+static const char * const tsin_ao_a_groups[] = {
+ "tsin_ao_asop", "tsin_ao_adin0", "tsin_ao_aclk", "tsin_ao_a_valid",
+};
+
+static const char * const spdif_ao_out_groups[] = {
+ "spdif_ao_out",
+};
+
+static const char * const tdm_ao_b_groups[] = {
+ "tdm_ao_b_dout0", "tdm_ao_b_dout1", "tdm_ao_b_dout2",
+ "tdm_ao_b_fs", "tdm_ao_b_sclk",
+ "tdm_ao_b_din0", "tdm_ao_b_din1", "tdm_ao_b_din2",
+ "tdm_ao_b_slv_fs", "tdm_ao_b_slv_sclk",
+};
+
+static const char * const mclk0_ao_groups[] = {
+ "mclk0_ao",
+};
+
+static struct meson_pmx_func meson_g12a_periphs_functions[] = {
+ FUNCTION(gpio_periphs),
+ FUNCTION(emmc),
+ FUNCTION(nor),
+ FUNCTION(spi0),
+ FUNCTION(spi1),
+ FUNCTION(sdio),
+ FUNCTION(nand),
+ FUNCTION(sdcard),
+ FUNCTION(i2c0),
+ FUNCTION(i2c1),
+ FUNCTION(i2c2),
+ FUNCTION(i2c3),
+ FUNCTION(uart_a),
+ FUNCTION(uart_b),
+ FUNCTION(uart_c),
+ FUNCTION(uart_ao_a_c),
+ FUNCTION(iso7816),
+ FUNCTION(eth),
+ FUNCTION(pwm_a),
+ FUNCTION(pwm_b),
+ FUNCTION(pwm_c),
+ FUNCTION(pwm_d),
+ FUNCTION(pwm_e),
+ FUNCTION(pwm_f),
+ FUNCTION(cec_ao_a_h),
+ FUNCTION(cec_ao_b_h),
+ FUNCTION(jtag_b),
+ FUNCTION(bt565_a),
+ FUNCTION(tsin_a),
+ FUNCTION(tsin_b),
+ FUNCTION(hdmitx),
+ FUNCTION(pdm),
+ FUNCTION(spdif_out),
+ FUNCTION(spdif_in),
+ FUNCTION(mclk0),
+ FUNCTION(mclk1),
+ FUNCTION(tdm_a),
+ FUNCTION(tdm_b),
+ FUNCTION(tdm_c),
+};
+
+static struct meson_pmx_func meson_g12a_aobus_functions[] = {
+ FUNCTION(gpio_aobus),
+ FUNCTION(uart_ao_a),
+ FUNCTION(uart_ao_b),
+ FUNCTION(i2c_ao),
+ FUNCTION(i2c_ao_slave),
+ FUNCTION(remote_ao_input),
+ FUNCTION(remote_ao_out),
+ FUNCTION(pwm_ao_a),
+ FUNCTION(pwm_ao_b),
+ FUNCTION(pwm_ao_c),
+ FUNCTION(pwm_ao_d),
+ FUNCTION(jtag_a),
+ FUNCTION(cec_ao_a),
+ FUNCTION(cec_ao_b),
+ FUNCTION(tsin_ao_a),
+ FUNCTION(spdif_ao_out),
+ FUNCTION(tdm_ao_b),
+ FUNCTION(mclk0_ao),
+};
+
+static struct meson_bank meson_g12a_periphs_banks[] = {
+ /* name first last irq pullen pull dir out in */
+ BANK("Z", GPIOZ_0, GPIOZ_15, 12, 27,
+ 4, 0, 4, 0, 12, 0, 13, 0, 14, 0),
+ BANK("H", GPIOH_0, GPIOH_8, 28, 36,
+ 3, 0, 3, 0, 9, 0, 10, 0, 11, 0),
+ BANK("BOOT", BOOT_0, BOOT_15, 37, 52,
+ 0, 0, 0, 0, 0, 0, 1, 0, 2, 0),
+ BANK("C", GPIOC_0, GPIOC_7, 53, 60,
+ 1, 0, 1, 0, 3, 0, 4, 0, 5, 0),
+ BANK("A", GPIOA_0, GPIOA_15, 61, 76,
+ 5, 0, 5, 0, 16, 0, 17, 0, 18, 0),
+ BANK("X", GPIOX_0, GPIOX_19, 77, 96,
+ 2, 0, 2, 0, 6, 0, 7, 0, 8, 0),
+};
+
+static struct meson_bank meson_g12a_aobus_banks[] = {
+ /* name first last irq pullen pull dir out in */
+ BANK("AO", GPIOAO_0, GPIOAO_11, 0, 11,
+ 3, 0, 2, 0, 0, 0, 4, 0, 1, 0),
+ /* GPIOE actually located in the AO bank */
+ BANK("E", GPIOE_0, GPIOE_2, 97, 99,
+ 3, 16, 2, 16, 0, 16, 4, 16, 1, 16),
+};
+
+static struct meson_pmx_bank meson_g12a_periphs_pmx_banks[] = {
+ /* name first lask reg offset */
+ BANK_PMX("Z", GPIOZ_0, GPIOZ_15, 0x6, 0),
+ BANK_PMX("H", GPIOH_0, GPIOH_8, 0xb, 0),
+ BANK_PMX("BOOT", BOOT_0, BOOT_15, 0x0, 0),
+ BANK_PMX("C", GPIOC_0, GPIOC_7, 0x9, 0),
+ BANK_PMX("A", GPIOA_0, GPIOA_15, 0xd, 0),
+ BANK_PMX("X", GPIOX_0, GPIOX_19, 0x3, 0),
+};
+
+static struct meson_axg_pmx_data meson_g12a_periphs_pmx_banks_data = {
+ .pmx_banks = meson_g12a_periphs_pmx_banks,
+ .num_pmx_banks = ARRAY_SIZE(meson_g12a_periphs_pmx_banks),
+};
+
+static struct meson_pmx_bank meson_g12a_aobus_pmx_banks[] = {
+ BANK_PMX("AO", GPIOAO_0, GPIOAO_11, 0x0, 0),
+ BANK_PMX("E", GPIOE_0, GPIOE_2, 0x1, 16),
+};
+
+static struct meson_axg_pmx_data meson_g12a_aobus_pmx_banks_data = {
+ .pmx_banks = meson_g12a_aobus_pmx_banks,
+ .num_pmx_banks = ARRAY_SIZE(meson_g12a_aobus_pmx_banks),
+};
+
+static struct meson_pinctrl_data meson_g12a_periphs_pinctrl_data = {
+ .name = "periphs-banks",
+ .pins = meson_g12a_periphs_pins,
+ .groups = meson_g12a_periphs_groups,
+ .funcs = meson_g12a_periphs_functions,
+ .banks = meson_g12a_periphs_banks,
+ .num_pins = ARRAY_SIZE(meson_g12a_periphs_pins),
+ .num_groups = ARRAY_SIZE(meson_g12a_periphs_groups),
+ .num_funcs = ARRAY_SIZE(meson_g12a_periphs_functions),
+ .num_banks = ARRAY_SIZE(meson_g12a_periphs_banks),
+ .pmx_ops = &meson_axg_pmx_ops,
+ .pmx_data = &meson_g12a_periphs_pmx_banks_data,
+};
+
+static struct meson_pinctrl_data meson_g12a_aobus_pinctrl_data = {
+ .name = "aobus-banks",
+ .pins = meson_g12a_aobus_pins,
+ .groups = meson_g12a_aobus_groups,
+ .funcs = meson_g12a_aobus_functions,
+ .banks = meson_g12a_aobus_banks,
+ .num_pins = ARRAY_SIZE(meson_g12a_aobus_pins),
+ .num_groups = ARRAY_SIZE(meson_g12a_aobus_groups),
+ .num_funcs = ARRAY_SIZE(meson_g12a_aobus_functions),
+ .num_banks = ARRAY_SIZE(meson_g12a_aobus_banks),
+ .pmx_ops = &meson_axg_pmx_ops,
+ .pmx_data = &meson_g12a_aobus_pmx_banks_data,
+};
+
+static const struct of_device_id meson_g12a_pinctrl_dt_match[] = {
+ {
+ .compatible = "amlogic,meson-g12a-periphs-pinctrl",
+ .data = &meson_g12a_periphs_pinctrl_data,
+ },
+ {
+ .compatible = "amlogic,meson-g12a-aobus-pinctrl",
+ .data = &meson_g12a_aobus_pinctrl_data,
+ },
+ { },
+};
+
+static struct platform_driver meson_g12a_pinctrl_driver = {
+ .probe = meson_pinctrl_probe,
+ .driver = {
+ .name = "meson-g12a-pinctrl",
+ .of_match_table = meson_g12a_pinctrl_dt_match,
+ },
+};
+
+builtin_platform_driver(meson_g12a_pinctrl_driver);
*/
#include <linux/device.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/init.h>
#include <linux/io.h>
#include <linux/of.h>
meson_regmap_config.max_register = resource_size(&res) - 4;
meson_regmap_config.name = devm_kasprintf(pc->dev, GFP_KERNEL,
- "%s-%s", node->name,
+ "%pOFn-%s", node,
name);
if (!meson_regmap_config.name)
return ERR_PTR(-ENOMEM);
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/pinctrl/pinctrl.h>
#include <linux/platform_device.h>
#include <linux/regmap.h>
#include <linux/of_address.h>
#include <linux/of_platform.h>
#include <linux/err.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/pinctrl/machine.h>
#include <linux/pinctrl/pinconf.h>
#include <linux/pinctrl/pinctrl.h>
ret = of_property_read_string(np, "marvell,function", &function);
if (ret) {
dev_err(pctl->dev,
- "missing marvell,function in node %s\n", np->name);
+ "missing marvell,function in node %pOFn\n", np);
return 0;
}
nmaps = of_property_count_strings(np, "marvell,pins");
if (nmaps < 0) {
dev_err(pctl->dev,
- "missing marvell,pins in node %s\n", np->name);
+ "missing marvell,pins in node %pOFn\n", np);
return 0;
}
*/
#include <linux/kernel.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/pinctrl/pinctrl.h>
#include <linux/mfd/abx500/ab8500.h>
#include "pinctrl-abx500.h"
*/
#include <linux/kernel.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/pinctrl/pinctrl.h>
#include <linux/mfd/abx500/ab8500.h>
#include "pinctrl-abx500.h"
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/irq.h>
#include <linux/irqdomain.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/clk.h>
#include <linux/err.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/spinlock.h>
#include <linux/interrupt.h>
#include <linux/slab.h>
#define GPIO_BLOCK_SHIFT 5
#define NMK_GPIO_PER_CHIP (1 << GPIO_BLOCK_SHIFT)
-#define NMK_MAX_BANKS DIV_ROUND_UP(ARCH_NR_GPIOS, NMK_GPIO_PER_CHIP)
+#define NMK_MAX_BANKS DIV_ROUND_UP(512, NMK_GPIO_PER_CHIP)
/* Register in the logic block */
#define NMK_GPIO_DAT 0x00
data_out ? "hi" : "lo",
(mode < 0) ? "unknown" : modes[mode]);
} else {
- int irq = gpio_to_irq(gpio);
+ int irq = chip->to_irq(chip, offset);
struct irq_desc *desc = irq_to_desc(irq);
int pullidx = 0;
int val;
gpio_pdev = of_find_device_by_node(np);
if (!gpio_pdev) {
- pr_err("populate \"%s\": device not found\n", np->name);
+ pr_err("populate \"%pOFn\": device not found\n", np);
return ERR_PTR(-ENODEV);
}
if (of_property_read_u32(np, "gpio-bank", &id)) {
gpio_np = of_parse_phandle(np, "nomadik-gpio-chips", i);
if (gpio_np) {
dev_info(&pdev->dev,
- "populate NMK GPIO %d \"%s\"\n",
- i, gpio_np->name);
+ "populate NMK GPIO %d \"%pOFn\"\n",
+ i, gpio_np);
nmk_chip = nmk_gpio_populate_chip(gpio_np, pdev);
if (IS_ERR(nmk_chip))
dev_err(&pdev->dev,
--- /dev/null
+config PINCTRL_NPCM7XX
+ bool "Pinctrl and GPIO driver for Nuvoton NPCM7XX"
+ depends on (ARCH_NPCM7XX || COMPILE_TEST) && OF
+ select PINMUX
+ select PINCONF
+ select GENERIC_PINCONF
+ select GPIOLIB
+ select GPIO_GENERIC
+ select GPIOLIB_IRQCHIP
+ help
+ Say Y here to enable pin controller and GPIO support
+ for Nuvoton NPCM750/730/715/705 SoCs.
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+# Nuvoton pinctrl support
+
+obj-$(CONFIG_PINCTRL_NPCM7XX) += pinctrl-npcm7xx.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2016-2018 Nuvoton Technology corporation.
+// Copyright (c) 2016, Dell Inc
+
+#include <linux/device.h>
+#include <linux/gpio/driver.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/mfd/syscon.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/pinctrl/machine.h>
+#include <linux/pinctrl/pinconf.h>
+#include <linux/pinctrl/pinconf-generic.h>
+#include <linux/pinctrl/pinctrl.h>
+#include <linux/pinctrl/pinmux.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+
+/* GCR registers */
+#define NPCM7XX_GCR_PDID 0x00
+#define NPCM7XX_GCR_MFSEL1 0x0C
+#define NPCM7XX_GCR_MFSEL2 0x10
+#define NPCM7XX_GCR_MFSEL3 0x64
+#define NPCM7XX_GCR_MFSEL4 0xb0
+#define NPCM7XX_GCR_CPCTL 0xD0
+#define NPCM7XX_GCR_CP2BST 0xD4
+#define NPCM7XX_GCR_B2CPNT 0xD8
+#define NPCM7XX_GCR_I2CSEGSEL 0xE0
+#define NPCM7XX_GCR_I2CSEGCTL 0xE4
+#define NPCM7XX_GCR_SRCNT 0x68
+#define NPCM7XX_GCR_FLOCKR1 0x74
+#define NPCM7XX_GCR_DSCNT 0x78
+
+#define SRCNT_ESPI BIT(3)
+
+/* GPIO registers */
+#define NPCM7XX_GP_N_TLOCK1 0x00
+#define NPCM7XX_GP_N_DIN 0x04 /* Data IN */
+#define NPCM7XX_GP_N_POL 0x08 /* Polarity */
+#define NPCM7XX_GP_N_DOUT 0x0c /* Data OUT */
+#define NPCM7XX_GP_N_OE 0x10 /* Output Enable */
+#define NPCM7XX_GP_N_OTYP 0x14
+#define NPCM7XX_GP_N_MP 0x18
+#define NPCM7XX_GP_N_PU 0x1c /* Pull-up */
+#define NPCM7XX_GP_N_PD 0x20 /* Pull-down */
+#define NPCM7XX_GP_N_DBNC 0x24 /* Debounce */
+#define NPCM7XX_GP_N_EVTYP 0x28 /* Event Type */
+#define NPCM7XX_GP_N_EVBE 0x2c /* Event Both Edge */
+#define NPCM7XX_GP_N_OBL0 0x30
+#define NPCM7XX_GP_N_OBL1 0x34
+#define NPCM7XX_GP_N_OBL2 0x38
+#define NPCM7XX_GP_N_OBL3 0x3c
+#define NPCM7XX_GP_N_EVEN 0x40 /* Event Enable */
+#define NPCM7XX_GP_N_EVENS 0x44 /* Event Set (enable) */
+#define NPCM7XX_GP_N_EVENC 0x48 /* Event Clear (disable) */
+#define NPCM7XX_GP_N_EVST 0x4c /* Event Status */
+#define NPCM7XX_GP_N_SPLCK 0x50
+#define NPCM7XX_GP_N_MPLCK 0x54
+#define NPCM7XX_GP_N_IEM 0x58 /* Input Enable */
+#define NPCM7XX_GP_N_OSRC 0x5c
+#define NPCM7XX_GP_N_ODSC 0x60
+#define NPCM7XX_GP_N_DOS 0x68 /* Data OUT Set */
+#define NPCM7XX_GP_N_DOC 0x6c /* Data OUT Clear */
+#define NPCM7XX_GP_N_OES 0x70 /* Output Enable Set */
+#define NPCM7XX_GP_N_OEC 0x74 /* Output Enable Clear */
+#define NPCM7XX_GP_N_TLOCK2 0x7c
+
+#define NPCM7XX_GPIO_PER_BANK 32
+#define NPCM7XX_GPIO_BANK_NUM 8
+#define NPCM7XX_GCR_NONE 0
+
+/* Structure for register banks */
+struct npcm7xx_gpio {
+ void __iomem *base;
+ struct gpio_chip gc;
+ int irqbase;
+ int irq;
+ void *priv;
+ struct irq_chip irq_chip;
+ u32 pinctrl_id;
+ int (*direction_input)(struct gpio_chip *chip, unsigned offset);
+ int (*direction_output)(struct gpio_chip *chip, unsigned offset,
+ int value);
+ int (*request)(struct gpio_chip *chip, unsigned offset);
+ void (*free)(struct gpio_chip *chip, unsigned offset);
+};
+
+struct npcm7xx_pinctrl {
+ struct pinctrl_dev *pctldev;
+ struct device *dev;
+ struct npcm7xx_gpio gpio_bank[NPCM7XX_GPIO_BANK_NUM];
+ struct irq_domain *domain;
+ struct regmap *gcr_regmap;
+ void __iomem *regs;
+ u32 bank_num;
+};
+
+/* GPIO handling in the pinctrl driver */
+static void npcm_gpio_set(struct gpio_chip *gc, void __iomem *reg,
+ unsigned int pinmask)
+{
+ unsigned long flags;
+ unsigned long val;
+
+ spin_lock_irqsave(&gc->bgpio_lock, flags);
+
+ val = ioread32(reg) | pinmask;
+ iowrite32(val, reg);
+
+ spin_unlock_irqrestore(&gc->bgpio_lock, flags);
+}
+
+static void npcm_gpio_clr(struct gpio_chip *gc, void __iomem *reg,
+ unsigned int pinmask)
+{
+ unsigned long flags;
+ unsigned long val;
+
+ spin_lock_irqsave(&gc->bgpio_lock, flags);
+
+ val = ioread32(reg) & ~pinmask;
+ iowrite32(val, reg);
+
+ spin_unlock_irqrestore(&gc->bgpio_lock, flags);
+}
+
+static void npcmgpio_dbg_show(struct seq_file *s, struct gpio_chip *chip)
+{
+ struct npcm7xx_gpio *bank = gpiochip_get_data(chip);
+
+ seq_printf(s, "-- module %d [gpio%d - %d]\n",
+ bank->gc.base / bank->gc.ngpio,
+ bank->gc.base,
+ bank->gc.base + bank->gc.ngpio);
+ seq_printf(s, "DIN :%.8x DOUT:%.8x IE :%.8x OE :%.8x\n",
+ ioread32(bank->base + NPCM7XX_GP_N_DIN),
+ ioread32(bank->base + NPCM7XX_GP_N_DOUT),
+ ioread32(bank->base + NPCM7XX_GP_N_IEM),
+ ioread32(bank->base + NPCM7XX_GP_N_OE));
+ seq_printf(s, "PU :%.8x PD :%.8x DB :%.8x POL :%.8x\n",
+ ioread32(bank->base + NPCM7XX_GP_N_PU),
+ ioread32(bank->base + NPCM7XX_GP_N_PD),
+ ioread32(bank->base + NPCM7XX_GP_N_DBNC),
+ ioread32(bank->base + NPCM7XX_GP_N_POL));
+ seq_printf(s, "ETYP:%.8x EVBE:%.8x EVEN:%.8x EVST:%.8x\n",
+ ioread32(bank->base + NPCM7XX_GP_N_EVTYP),
+ ioread32(bank->base + NPCM7XX_GP_N_EVBE),
+ ioread32(bank->base + NPCM7XX_GP_N_EVEN),
+ ioread32(bank->base + NPCM7XX_GP_N_EVST));
+ seq_printf(s, "OTYP:%.8x OSRC:%.8x ODSC:%.8x\n",
+ ioread32(bank->base + NPCM7XX_GP_N_OTYP),
+ ioread32(bank->base + NPCM7XX_GP_N_OSRC),
+ ioread32(bank->base + NPCM7XX_GP_N_ODSC));
+ seq_printf(s, "OBL0:%.8x OBL1:%.8x OBL2:%.8x OBL3:%.8x\n",
+ ioread32(bank->base + NPCM7XX_GP_N_OBL0),
+ ioread32(bank->base + NPCM7XX_GP_N_OBL1),
+ ioread32(bank->base + NPCM7XX_GP_N_OBL2),
+ ioread32(bank->base + NPCM7XX_GP_N_OBL3));
+ seq_printf(s, "SLCK:%.8x MLCK:%.8x\n",
+ ioread32(bank->base + NPCM7XX_GP_N_SPLCK),
+ ioread32(bank->base + NPCM7XX_GP_N_MPLCK));
+}
+
+static int npcmgpio_direction_input(struct gpio_chip *chip, unsigned int offset)
+{
+ struct npcm7xx_gpio *bank = gpiochip_get_data(chip);
+ int ret;
+
+ ret = pinctrl_gpio_direction_input(offset + chip->base);
+ if (ret)
+ return ret;
+
+ return bank->direction_input(chip, offset);
+}
+
+/* Set GPIO to Output with initial value */
+static int npcmgpio_direction_output(struct gpio_chip *chip,
+ unsigned int offset, int value)
+{
+ struct npcm7xx_gpio *bank = gpiochip_get_data(chip);
+ int ret;
+
+ dev_dbg(chip->parent, "gpio_direction_output: offset%d = %x\n", offset,
+ value);
+
+ ret = pinctrl_gpio_direction_output(offset + chip->base);
+ if (ret)
+ return ret;
+
+ return bank->direction_output(chip, offset, value);
+}
+
+static int npcmgpio_gpio_request(struct gpio_chip *chip, unsigned int offset)
+{
+ struct npcm7xx_gpio *bank = gpiochip_get_data(chip);
+ int ret;
+
+ dev_dbg(chip->parent, "gpio_request: offset%d\n", offset);
+ ret = pinctrl_gpio_request(offset + chip->base);
+ if (ret)
+ return ret;
+
+ return bank->request(chip, offset);
+}
+
+static void npcmgpio_gpio_free(struct gpio_chip *chip, unsigned int offset)
+{
+ dev_dbg(chip->parent, "gpio_free: offset%d\n", offset);
+ pinctrl_gpio_free(offset + chip->base);
+}
+
+static void npcmgpio_irq_handler(struct irq_desc *desc)
+{
+ struct gpio_chip *gc;
+ struct irq_chip *chip;
+ struct npcm7xx_gpio *bank;
+ u32 sts, en, bit;
+
+ gc = irq_desc_get_handler_data(desc);
+ bank = gpiochip_get_data(gc);
+ chip = irq_desc_get_chip(desc);
+
+ chained_irq_enter(chip, desc);
+ sts = ioread32(bank->base + NPCM7XX_GP_N_EVST);
+ en = ioread32(bank->base + NPCM7XX_GP_N_EVEN);
+ dev_dbg(chip->parent_device, "==> got irq sts %.8x %.8x\n", sts,
+ en);
+
+ sts &= en;
+ for_each_set_bit(bit, (const void *)&sts, NPCM7XX_GPIO_PER_BANK)
+ generic_handle_irq(irq_linear_revmap(gc->irq.domain, bit));
+ chained_irq_exit(chip, desc);
+}
+
+static int npcmgpio_set_irq_type(struct irq_data *d, unsigned int type)
+{
+ struct npcm7xx_gpio *bank =
+ gpiochip_get_data(irq_data_get_irq_chip_data(d));
+ unsigned int gpio = BIT(d->hwirq);
+
+ dev_dbg(d->chip->parent_device, "setirqtype: %u.%u = %u\n", gpio,
+ d->irq, type);
+ switch (type) {
+ case IRQ_TYPE_EDGE_RISING:
+ dev_dbg(d->chip->parent_device, "edge.rising\n");
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_EVBE, gpio);
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_POL, gpio);
+ break;
+ case IRQ_TYPE_EDGE_FALLING:
+ dev_dbg(d->chip->parent_device, "edge.falling\n");
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_EVBE, gpio);
+ npcm_gpio_set(&bank->gc, bank->base + NPCM7XX_GP_N_POL, gpio);
+ break;
+ case IRQ_TYPE_EDGE_BOTH:
+ dev_dbg(d->chip->parent_device, "edge.both\n");
+ npcm_gpio_set(&bank->gc, bank->base + NPCM7XX_GP_N_EVBE, gpio);
+ break;
+ case IRQ_TYPE_LEVEL_LOW:
+ dev_dbg(d->chip->parent_device, "level.low\n");
+ npcm_gpio_set(&bank->gc, bank->base + NPCM7XX_GP_N_POL, gpio);
+ break;
+ case IRQ_TYPE_LEVEL_HIGH:
+ dev_dbg(d->chip->parent_device, "level.high\n");
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_POL, gpio);
+ break;
+ default:
+ dev_dbg(d->chip->parent_device, "invalid irq type\n");
+ return -EINVAL;
+ }
+
+ if (type & (IRQ_TYPE_LEVEL_HIGH | IRQ_TYPE_LEVEL_LOW)) {
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_EVTYP, gpio);
+ irq_set_handler_locked(d, handle_level_irq);
+ } else if (type & (IRQ_TYPE_EDGE_BOTH | IRQ_TYPE_EDGE_RISING
+ | IRQ_TYPE_EDGE_FALLING)) {
+ npcm_gpio_set(&bank->gc, bank->base + NPCM7XX_GP_N_EVTYP, gpio);
+ irq_set_handler_locked(d, handle_edge_irq);
+ }
+
+ return 0;
+}
+
+static void npcmgpio_irq_ack(struct irq_data *d)
+{
+ struct npcm7xx_gpio *bank =
+ gpiochip_get_data(irq_data_get_irq_chip_data(d));
+ unsigned int gpio = d->hwirq;
+
+ dev_dbg(d->chip->parent_device, "irq_ack: %u.%u\n", gpio, d->irq);
+ iowrite32(BIT(gpio), bank->base + NPCM7XX_GP_N_EVST);
+}
+
+/* Disable GPIO interrupt */
+static void npcmgpio_irq_mask(struct irq_data *d)
+{
+ struct npcm7xx_gpio *bank =
+ gpiochip_get_data(irq_data_get_irq_chip_data(d));
+ unsigned int gpio = d->hwirq;
+
+ /* Clear events */
+ dev_dbg(d->chip->parent_device, "irq_mask: %u.%u\n", gpio, d->irq);
+ iowrite32(BIT(gpio), bank->base + NPCM7XX_GP_N_EVENC);
+}
+
+/* Enable GPIO interrupt */
+static void npcmgpio_irq_unmask(struct irq_data *d)
+{
+ struct npcm7xx_gpio *bank =
+ gpiochip_get_data(irq_data_get_irq_chip_data(d));
+ unsigned int gpio = d->hwirq;
+
+ /* Enable events */
+ dev_dbg(d->chip->parent_device, "irq_unmask: %u.%u\n", gpio, d->irq);
+ iowrite32(BIT(gpio), bank->base + NPCM7XX_GP_N_EVENS);
+}
+
+static unsigned int npcmgpio_irq_startup(struct irq_data *d)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ unsigned int gpio = d->hwirq;
+
+ /* active-high, input, clear interrupt, enable interrupt */
+ dev_dbg(d->chip->parent_device, "startup: %u.%u\n", gpio, d->irq);
+ npcmgpio_direction_input(gc, gpio);
+ npcmgpio_irq_ack(d);
+ npcmgpio_irq_unmask(d);
+
+ return 0;
+}
+
+static struct irq_chip npcmgpio_irqchip = {
+ .name = "NPCM7XX-GPIO-IRQ",
+ .irq_ack = npcmgpio_irq_ack,
+ .irq_unmask = npcmgpio_irq_unmask,
+ .irq_mask = npcmgpio_irq_mask,
+ .irq_set_type = npcmgpio_set_irq_type,
+ .irq_startup = npcmgpio_irq_startup,
+};
+
+/* pinmux handing in the pinctrl driver*/
+static const int smb0_pins[] = { 115, 114 };
+static const int smb0b_pins[] = { 195, 194 };
+static const int smb0c_pins[] = { 202, 196 };
+static const int smb0d_pins[] = { 198, 199 };
+static const int smb0den_pins[] = { 197 };
+
+static const int smb1_pins[] = { 117, 116 };
+static const int smb1b_pins[] = { 126, 127 };
+static const int smb1c_pins[] = { 124, 125 };
+static const int smb1d_pins[] = { 4, 5 };
+
+static const int smb2_pins[] = { 119, 118 };
+static const int smb2b_pins[] = { 122, 123 };
+static const int smb2c_pins[] = { 120, 121 };
+static const int smb2d_pins[] = { 6, 7 };
+
+static const int smb3_pins[] = { 30, 31 };
+static const int smb3b_pins[] = { 39, 40 };
+static const int smb3c_pins[] = { 37, 38 };
+static const int smb3d_pins[] = { 59, 60 };
+
+static const int smb4_pins[] = { 28, 29 };
+static const int smb4b_pins[] = { 18, 19 };
+static const int smb4c_pins[] = { 20, 21 };
+static const int smb4d_pins[] = { 22, 23 };
+static const int smb4den_pins[] = { 17 };
+
+static const int smb5_pins[] = { 26, 27 };
+static const int smb5b_pins[] = { 13, 12 };
+static const int smb5c_pins[] = { 15, 14 };
+static const int smb5d_pins[] = { 94, 93 };
+static const int ga20kbc_pins[] = { 94, 93 };
+
+static const int smb6_pins[] = { 172, 171 };
+static const int smb7_pins[] = { 174, 173 };
+static const int smb8_pins[] = { 129, 128 };
+static const int smb9_pins[] = { 131, 130 };
+static const int smb10_pins[] = { 133, 132 };
+static const int smb11_pins[] = { 135, 134 };
+static const int smb12_pins[] = { 221, 220 };
+static const int smb13_pins[] = { 223, 222 };
+static const int smb14_pins[] = { 22, 23 };
+static const int smb15_pins[] = { 20, 21 };
+
+static const int fanin0_pins[] = { 64 };
+static const int fanin1_pins[] = { 65 };
+static const int fanin2_pins[] = { 66 };
+static const int fanin3_pins[] = { 67 };
+static const int fanin4_pins[] = { 68 };
+static const int fanin5_pins[] = { 69 };
+static const int fanin6_pins[] = { 70 };
+static const int fanin7_pins[] = { 71 };
+static const int fanin8_pins[] = { 72 };
+static const int fanin9_pins[] = { 73 };
+static const int fanin10_pins[] = { 74 };
+static const int fanin11_pins[] = { 75 };
+static const int fanin12_pins[] = { 76 };
+static const int fanin13_pins[] = { 77 };
+static const int fanin14_pins[] = { 78 };
+static const int fanin15_pins[] = { 79 };
+static const int faninx_pins[] = { 175, 176, 177, 203 };
+
+static const int pwm0_pins[] = { 80 };
+static const int pwm1_pins[] = { 81 };
+static const int pwm2_pins[] = { 82 };
+static const int pwm3_pins[] = { 83 };
+static const int pwm4_pins[] = { 144 };
+static const int pwm5_pins[] = { 145 };
+static const int pwm6_pins[] = { 146 };
+static const int pwm7_pins[] = { 147 };
+
+static const int uart1_pins[] = { 43, 44, 45, 46, 47, 61, 62, 63 };
+static const int uart2_pins[] = { 48, 49, 50, 51, 52, 53, 54, 55 };
+
+/* RGMII 1 pin group */
+static const int rg1_pins[] = { 96, 97, 98, 99, 100, 101, 102, 103, 104, 105,
+ 106, 107 };
+/* RGMII 1 MD interface pin group */
+static const int rg1mdio_pins[] = { 108, 109 };
+
+/* RGMII 2 pin group */
+static const int rg2_pins[] = { 110, 111, 112, 113, 208, 209, 210, 211, 212,
+ 213, 214, 215 };
+/* RGMII 2 MD interface pin group */
+static const int rg2mdio_pins[] = { 216, 217 };
+
+static const int ddr_pins[] = { 110, 111, 112, 113, 208, 209, 210, 211, 212,
+ 213, 214, 215, 216, 217 };
+/* Serial I/O Expander 1 */
+static const int iox1_pins[] = { 0, 1, 2, 3 };
+/* Serial I/O Expander 2 */
+static const int iox2_pins[] = { 4, 5, 6, 7 };
+/* Host Serial I/O Expander 2 */
+static const int ioxh_pins[] = { 10, 11, 24, 25 };
+
+static const int mmc_pins[] = { 152, 154, 156, 157, 158, 159 };
+static const int mmcwp_pins[] = { 153 };
+static const int mmccd_pins[] = { 155 };
+static const int mmcrst_pins[] = { 155 };
+static const int mmc8_pins[] = { 148, 149, 150, 151 };
+
+/* RMII 1 pin groups */
+static const int r1_pins[] = { 178, 179, 180, 181, 182, 193, 201 };
+static const int r1err_pins[] = { 56 };
+static const int r1md_pins[] = { 57, 58 };
+
+/* RMII 2 pin groups */
+static const int r2_pins[] = { 84, 85, 86, 87, 88, 89, 200 };
+static const int r2err_pins[] = { 90 };
+static const int r2md_pins[] = { 91, 92 };
+
+static const int sd1_pins[] = { 136, 137, 138, 139, 140, 141, 142, 143 };
+static const int sd1pwr_pins[] = { 143 };
+
+static const int wdog1_pins[] = { 218 };
+static const int wdog2_pins[] = { 219 };
+
+/* BMC serial port 0 */
+static const int bmcuart0a_pins[] = { 41, 42 };
+static const int bmcuart0b_pins[] = { 48, 49 };
+
+static const int bmcuart1_pins[] = { 43, 44, 62, 63 };
+
+/* System Control Interrupt and Power Management Event pin group */
+static const int scipme_pins[] = { 169 };
+/* System Management Interrupt pin group */
+static const int sci_pins[] = { 170 };
+/* Serial Interrupt Line pin group */
+static const int serirq_pins[] = { 162 };
+
+static const int clkout_pins[] = { 160 };
+static const int clkreq_pins[] = { 231 };
+
+static const int jtag2_pins[] = { 43, 44, 45, 46, 47 };
+/* Graphics SPI Clock pin group */
+static const int gspi_pins[] = { 12, 13, 14, 15 };
+
+static const int spix_pins[] = { 224, 225, 226, 227, 229, 230 };
+static const int spixcs1_pins[] = { 228 };
+
+static const int pspi1_pins[] = { 175, 176, 177 };
+static const int pspi2_pins[] = { 17, 18, 19 };
+
+static const int spi0cs1_pins[] = { 32 };
+
+static const int spi3_pins[] = { 183, 184, 185, 186 };
+static const int spi3cs1_pins[] = { 187 };
+static const int spi3quad_pins[] = { 188, 189 };
+static const int spi3cs2_pins[] = { 188 };
+static const int spi3cs3_pins[] = { 189 };
+
+static const int ddc_pins[] = { 204, 205, 206, 207 };
+
+static const int lpc_pins[] = { 95, 161, 163, 164, 165, 166, 167 };
+static const int lpcclk_pins[] = { 168 };
+static const int espi_pins[] = { 95, 161, 163, 164, 165, 166, 167, 168 };
+
+static const int lkgpo0_pins[] = { 16 };
+static const int lkgpo1_pins[] = { 8 };
+static const int lkgpo2_pins[] = { 9 };
+
+static const int nprd_smi_pins[] = { 190 };
+
+/*
+ * pin: name, number
+ * group: name, npins, pins
+ * function: name, ngroups, groups
+ */
+struct npcm7xx_group {
+ const char *name;
+ const unsigned int *pins;
+ int npins;
+};
+
+#define NPCM7XX_GRPS \
+ NPCM7XX_GRP(smb0), \
+ NPCM7XX_GRP(smb0b), \
+ NPCM7XX_GRP(smb0c), \
+ NPCM7XX_GRP(smb0d), \
+ NPCM7XX_GRP(smb0den), \
+ NPCM7XX_GRP(smb1), \
+ NPCM7XX_GRP(smb1b), \
+ NPCM7XX_GRP(smb1c), \
+ NPCM7XX_GRP(smb1d), \
+ NPCM7XX_GRP(smb2), \
+ NPCM7XX_GRP(smb2b), \
+ NPCM7XX_GRP(smb2c), \
+ NPCM7XX_GRP(smb2d), \
+ NPCM7XX_GRP(smb3), \
+ NPCM7XX_GRP(smb3b), \
+ NPCM7XX_GRP(smb3c), \
+ NPCM7XX_GRP(smb3d), \
+ NPCM7XX_GRP(smb4), \
+ NPCM7XX_GRP(smb4b), \
+ NPCM7XX_GRP(smb4c), \
+ NPCM7XX_GRP(smb4d), \
+ NPCM7XX_GRP(smb4den), \
+ NPCM7XX_GRP(smb5), \
+ NPCM7XX_GRP(smb5b), \
+ NPCM7XX_GRP(smb5c), \
+ NPCM7XX_GRP(smb5d), \
+ NPCM7XX_GRP(ga20kbc), \
+ NPCM7XX_GRP(smb6), \
+ NPCM7XX_GRP(smb7), \
+ NPCM7XX_GRP(smb8), \
+ NPCM7XX_GRP(smb9), \
+ NPCM7XX_GRP(smb10), \
+ NPCM7XX_GRP(smb11), \
+ NPCM7XX_GRP(smb12), \
+ NPCM7XX_GRP(smb13), \
+ NPCM7XX_GRP(smb14), \
+ NPCM7XX_GRP(smb15), \
+ NPCM7XX_GRP(fanin0), \
+ NPCM7XX_GRP(fanin1), \
+ NPCM7XX_GRP(fanin2), \
+ NPCM7XX_GRP(fanin3), \
+ NPCM7XX_GRP(fanin4), \
+ NPCM7XX_GRP(fanin5), \
+ NPCM7XX_GRP(fanin6), \
+ NPCM7XX_GRP(fanin7), \
+ NPCM7XX_GRP(fanin8), \
+ NPCM7XX_GRP(fanin9), \
+ NPCM7XX_GRP(fanin10), \
+ NPCM7XX_GRP(fanin11), \
+ NPCM7XX_GRP(fanin12), \
+ NPCM7XX_GRP(fanin13), \
+ NPCM7XX_GRP(fanin14), \
+ NPCM7XX_GRP(fanin15), \
+ NPCM7XX_GRP(faninx), \
+ NPCM7XX_GRP(pwm0), \
+ NPCM7XX_GRP(pwm1), \
+ NPCM7XX_GRP(pwm2), \
+ NPCM7XX_GRP(pwm3), \
+ NPCM7XX_GRP(pwm4), \
+ NPCM7XX_GRP(pwm5), \
+ NPCM7XX_GRP(pwm6), \
+ NPCM7XX_GRP(pwm7), \
+ NPCM7XX_GRP(rg1), \
+ NPCM7XX_GRP(rg1mdio), \
+ NPCM7XX_GRP(rg2), \
+ NPCM7XX_GRP(rg2mdio), \
+ NPCM7XX_GRP(ddr), \
+ NPCM7XX_GRP(uart1), \
+ NPCM7XX_GRP(uart2), \
+ NPCM7XX_GRP(bmcuart0a), \
+ NPCM7XX_GRP(bmcuart0b), \
+ NPCM7XX_GRP(bmcuart1), \
+ NPCM7XX_GRP(iox1), \
+ NPCM7XX_GRP(iox2), \
+ NPCM7XX_GRP(ioxh), \
+ NPCM7XX_GRP(gspi), \
+ NPCM7XX_GRP(mmc), \
+ NPCM7XX_GRP(mmcwp), \
+ NPCM7XX_GRP(mmccd), \
+ NPCM7XX_GRP(mmcrst), \
+ NPCM7XX_GRP(mmc8), \
+ NPCM7XX_GRP(r1), \
+ NPCM7XX_GRP(r1err), \
+ NPCM7XX_GRP(r1md), \
+ NPCM7XX_GRP(r2), \
+ NPCM7XX_GRP(r2err), \
+ NPCM7XX_GRP(r2md), \
+ NPCM7XX_GRP(sd1), \
+ NPCM7XX_GRP(sd1pwr), \
+ NPCM7XX_GRP(wdog1), \
+ NPCM7XX_GRP(wdog2), \
+ NPCM7XX_GRP(scipme), \
+ NPCM7XX_GRP(sci), \
+ NPCM7XX_GRP(serirq), \
+ NPCM7XX_GRP(jtag2), \
+ NPCM7XX_GRP(spix), \
+ NPCM7XX_GRP(spixcs1), \
+ NPCM7XX_GRP(pspi1), \
+ NPCM7XX_GRP(pspi2), \
+ NPCM7XX_GRP(ddc), \
+ NPCM7XX_GRP(clkreq), \
+ NPCM7XX_GRP(clkout), \
+ NPCM7XX_GRP(spi3), \
+ NPCM7XX_GRP(spi3cs1), \
+ NPCM7XX_GRP(spi3quad), \
+ NPCM7XX_GRP(spi3cs2), \
+ NPCM7XX_GRP(spi3cs3), \
+ NPCM7XX_GRP(spi0cs1), \
+ NPCM7XX_GRP(lpc), \
+ NPCM7XX_GRP(lpcclk), \
+ NPCM7XX_GRP(espi), \
+ NPCM7XX_GRP(lkgpo0), \
+ NPCM7XX_GRP(lkgpo1), \
+ NPCM7XX_GRP(lkgpo2), \
+ NPCM7XX_GRP(nprd_smi), \
+ \
+
+enum {
+#define NPCM7XX_GRP(x) fn_ ## x
+ NPCM7XX_GRPS
+ /* add placeholder for none/gpio */
+ NPCM7XX_GRP(none),
+ NPCM7XX_GRP(gpio),
+#undef NPCM7XX_GRP
+};
+
+static struct npcm7xx_group npcm7xx_groups[] = {
+#define NPCM7XX_GRP(x) { .name = #x, .pins = x ## _pins, \
+ .npins = ARRAY_SIZE(x ## _pins) }
+ NPCM7XX_GRPS
+#undef NPCM7XX_GRP
+};
+
+#define NPCM7XX_SFUNC(a) NPCM7XX_FUNC(a, #a)
+#define NPCM7XX_FUNC(a, b...) static const char *a ## _grp[] = { b }
+#define NPCM7XX_MKFUNC(nm) { .name = #nm, .ngroups = ARRAY_SIZE(nm ## _grp), \
+ .groups = nm ## _grp }
+struct npcm7xx_func {
+ const char *name;
+ const unsigned int ngroups;
+ const char *const *groups;
+};
+
+NPCM7XX_SFUNC(smb0);
+NPCM7XX_SFUNC(smb0b);
+NPCM7XX_SFUNC(smb0c);
+NPCM7XX_SFUNC(smb0d);
+NPCM7XX_SFUNC(smb0den);
+NPCM7XX_SFUNC(smb1);
+NPCM7XX_SFUNC(smb1b);
+NPCM7XX_SFUNC(smb1c);
+NPCM7XX_SFUNC(smb1d);
+NPCM7XX_SFUNC(smb2);
+NPCM7XX_SFUNC(smb2b);
+NPCM7XX_SFUNC(smb2c);
+NPCM7XX_SFUNC(smb2d);
+NPCM7XX_SFUNC(smb3);
+NPCM7XX_SFUNC(smb3b);
+NPCM7XX_SFUNC(smb3c);
+NPCM7XX_SFUNC(smb3d);
+NPCM7XX_SFUNC(smb4);
+NPCM7XX_SFUNC(smb4b);
+NPCM7XX_SFUNC(smb4c);
+NPCM7XX_SFUNC(smb4d);
+NPCM7XX_SFUNC(smb4den);
+NPCM7XX_SFUNC(smb5);
+NPCM7XX_SFUNC(smb5b);
+NPCM7XX_SFUNC(smb5c);
+NPCM7XX_SFUNC(smb5d);
+NPCM7XX_SFUNC(ga20kbc);
+NPCM7XX_SFUNC(smb6);
+NPCM7XX_SFUNC(smb7);
+NPCM7XX_SFUNC(smb8);
+NPCM7XX_SFUNC(smb9);
+NPCM7XX_SFUNC(smb10);
+NPCM7XX_SFUNC(smb11);
+NPCM7XX_SFUNC(smb12);
+NPCM7XX_SFUNC(smb13);
+NPCM7XX_SFUNC(smb14);
+NPCM7XX_SFUNC(smb15);
+NPCM7XX_SFUNC(fanin0);
+NPCM7XX_SFUNC(fanin1);
+NPCM7XX_SFUNC(fanin2);
+NPCM7XX_SFUNC(fanin3);
+NPCM7XX_SFUNC(fanin4);
+NPCM7XX_SFUNC(fanin5);
+NPCM7XX_SFUNC(fanin6);
+NPCM7XX_SFUNC(fanin7);
+NPCM7XX_SFUNC(fanin8);
+NPCM7XX_SFUNC(fanin9);
+NPCM7XX_SFUNC(fanin10);
+NPCM7XX_SFUNC(fanin11);
+NPCM7XX_SFUNC(fanin12);
+NPCM7XX_SFUNC(fanin13);
+NPCM7XX_SFUNC(fanin14);
+NPCM7XX_SFUNC(fanin15);
+NPCM7XX_SFUNC(faninx);
+NPCM7XX_SFUNC(pwm0);
+NPCM7XX_SFUNC(pwm1);
+NPCM7XX_SFUNC(pwm2);
+NPCM7XX_SFUNC(pwm3);
+NPCM7XX_SFUNC(pwm4);
+NPCM7XX_SFUNC(pwm5);
+NPCM7XX_SFUNC(pwm6);
+NPCM7XX_SFUNC(pwm7);
+NPCM7XX_SFUNC(rg1);
+NPCM7XX_SFUNC(rg1mdio);
+NPCM7XX_SFUNC(rg2);
+NPCM7XX_SFUNC(rg2mdio);
+NPCM7XX_SFUNC(ddr);
+NPCM7XX_SFUNC(uart1);
+NPCM7XX_SFUNC(uart2);
+NPCM7XX_SFUNC(bmcuart0a);
+NPCM7XX_SFUNC(bmcuart0b);
+NPCM7XX_SFUNC(bmcuart1);
+NPCM7XX_SFUNC(iox1);
+NPCM7XX_SFUNC(iox2);
+NPCM7XX_SFUNC(ioxh);
+NPCM7XX_SFUNC(gspi);
+NPCM7XX_SFUNC(mmc);
+NPCM7XX_SFUNC(mmcwp);
+NPCM7XX_SFUNC(mmccd);
+NPCM7XX_SFUNC(mmcrst);
+NPCM7XX_SFUNC(mmc8);
+NPCM7XX_SFUNC(r1);
+NPCM7XX_SFUNC(r1err);
+NPCM7XX_SFUNC(r1md);
+NPCM7XX_SFUNC(r2);
+NPCM7XX_SFUNC(r2err);
+NPCM7XX_SFUNC(r2md);
+NPCM7XX_SFUNC(sd1);
+NPCM7XX_SFUNC(sd1pwr);
+NPCM7XX_SFUNC(wdog1);
+NPCM7XX_SFUNC(wdog2);
+NPCM7XX_SFUNC(scipme);
+NPCM7XX_SFUNC(sci);
+NPCM7XX_SFUNC(serirq);
+NPCM7XX_SFUNC(jtag2);
+NPCM7XX_SFUNC(spix);
+NPCM7XX_SFUNC(spixcs1);
+NPCM7XX_SFUNC(pspi1);
+NPCM7XX_SFUNC(pspi2);
+NPCM7XX_SFUNC(ddc);
+NPCM7XX_SFUNC(clkreq);
+NPCM7XX_SFUNC(clkout);
+NPCM7XX_SFUNC(spi3);
+NPCM7XX_SFUNC(spi3cs1);
+NPCM7XX_SFUNC(spi3quad);
+NPCM7XX_SFUNC(spi3cs2);
+NPCM7XX_SFUNC(spi3cs3);
+NPCM7XX_SFUNC(spi0cs1);
+NPCM7XX_SFUNC(lpc);
+NPCM7XX_SFUNC(lpcclk);
+NPCM7XX_SFUNC(espi);
+NPCM7XX_SFUNC(lkgpo0);
+NPCM7XX_SFUNC(lkgpo1);
+NPCM7XX_SFUNC(lkgpo2);
+NPCM7XX_SFUNC(nprd_smi);
+
+/* Function names */
+static struct npcm7xx_func npcm7xx_funcs[] = {
+ NPCM7XX_MKFUNC(smb0),
+ NPCM7XX_MKFUNC(smb0b),
+ NPCM7XX_MKFUNC(smb0c),
+ NPCM7XX_MKFUNC(smb0d),
+ NPCM7XX_MKFUNC(smb0den),
+ NPCM7XX_MKFUNC(smb1),
+ NPCM7XX_MKFUNC(smb1b),
+ NPCM7XX_MKFUNC(smb1c),
+ NPCM7XX_MKFUNC(smb1d),
+ NPCM7XX_MKFUNC(smb2),
+ NPCM7XX_MKFUNC(smb2b),
+ NPCM7XX_MKFUNC(smb2c),
+ NPCM7XX_MKFUNC(smb2d),
+ NPCM7XX_MKFUNC(smb3),
+ NPCM7XX_MKFUNC(smb3b),
+ NPCM7XX_MKFUNC(smb3c),
+ NPCM7XX_MKFUNC(smb3d),
+ NPCM7XX_MKFUNC(smb4),
+ NPCM7XX_MKFUNC(smb4b),
+ NPCM7XX_MKFUNC(smb4c),
+ NPCM7XX_MKFUNC(smb4d),
+ NPCM7XX_MKFUNC(smb4den),
+ NPCM7XX_MKFUNC(smb5),
+ NPCM7XX_MKFUNC(smb5b),
+ NPCM7XX_MKFUNC(smb5c),
+ NPCM7XX_MKFUNC(smb5d),
+ NPCM7XX_MKFUNC(ga20kbc),
+ NPCM7XX_MKFUNC(smb6),
+ NPCM7XX_MKFUNC(smb7),
+ NPCM7XX_MKFUNC(smb8),
+ NPCM7XX_MKFUNC(smb9),
+ NPCM7XX_MKFUNC(smb10),
+ NPCM7XX_MKFUNC(smb11),
+ NPCM7XX_MKFUNC(smb12),
+ NPCM7XX_MKFUNC(smb13),
+ NPCM7XX_MKFUNC(smb14),
+ NPCM7XX_MKFUNC(smb15),
+ NPCM7XX_MKFUNC(fanin0),
+ NPCM7XX_MKFUNC(fanin1),
+ NPCM7XX_MKFUNC(fanin2),
+ NPCM7XX_MKFUNC(fanin3),
+ NPCM7XX_MKFUNC(fanin4),
+ NPCM7XX_MKFUNC(fanin5),
+ NPCM7XX_MKFUNC(fanin6),
+ NPCM7XX_MKFUNC(fanin7),
+ NPCM7XX_MKFUNC(fanin8),
+ NPCM7XX_MKFUNC(fanin9),
+ NPCM7XX_MKFUNC(fanin10),
+ NPCM7XX_MKFUNC(fanin11),
+ NPCM7XX_MKFUNC(fanin12),
+ NPCM7XX_MKFUNC(fanin13),
+ NPCM7XX_MKFUNC(fanin14),
+ NPCM7XX_MKFUNC(fanin15),
+ NPCM7XX_MKFUNC(faninx),
+ NPCM7XX_MKFUNC(pwm0),
+ NPCM7XX_MKFUNC(pwm1),
+ NPCM7XX_MKFUNC(pwm2),
+ NPCM7XX_MKFUNC(pwm3),
+ NPCM7XX_MKFUNC(pwm4),
+ NPCM7XX_MKFUNC(pwm5),
+ NPCM7XX_MKFUNC(pwm6),
+ NPCM7XX_MKFUNC(pwm7),
+ NPCM7XX_MKFUNC(rg1),
+ NPCM7XX_MKFUNC(rg1mdio),
+ NPCM7XX_MKFUNC(rg2),
+ NPCM7XX_MKFUNC(rg2mdio),
+ NPCM7XX_MKFUNC(ddr),
+ NPCM7XX_MKFUNC(uart1),
+ NPCM7XX_MKFUNC(uart2),
+ NPCM7XX_MKFUNC(bmcuart0a),
+ NPCM7XX_MKFUNC(bmcuart0b),
+ NPCM7XX_MKFUNC(bmcuart1),
+ NPCM7XX_MKFUNC(iox1),
+ NPCM7XX_MKFUNC(iox2),
+ NPCM7XX_MKFUNC(ioxh),
+ NPCM7XX_MKFUNC(gspi),
+ NPCM7XX_MKFUNC(mmc),
+ NPCM7XX_MKFUNC(mmcwp),
+ NPCM7XX_MKFUNC(mmccd),
+ NPCM7XX_MKFUNC(mmcrst),
+ NPCM7XX_MKFUNC(mmc8),
+ NPCM7XX_MKFUNC(r1),
+ NPCM7XX_MKFUNC(r1err),
+ NPCM7XX_MKFUNC(r1md),
+ NPCM7XX_MKFUNC(r2),
+ NPCM7XX_MKFUNC(r2err),
+ NPCM7XX_MKFUNC(r2md),
+ NPCM7XX_MKFUNC(sd1),
+ NPCM7XX_MKFUNC(sd1pwr),
+ NPCM7XX_MKFUNC(wdog1),
+ NPCM7XX_MKFUNC(wdog2),
+ NPCM7XX_MKFUNC(scipme),
+ NPCM7XX_MKFUNC(sci),
+ NPCM7XX_MKFUNC(serirq),
+ NPCM7XX_MKFUNC(jtag2),
+ NPCM7XX_MKFUNC(spix),
+ NPCM7XX_MKFUNC(spixcs1),
+ NPCM7XX_MKFUNC(pspi1),
+ NPCM7XX_MKFUNC(pspi2),
+ NPCM7XX_MKFUNC(ddc),
+ NPCM7XX_MKFUNC(clkreq),
+ NPCM7XX_MKFUNC(clkout),
+ NPCM7XX_MKFUNC(spi3),
+ NPCM7XX_MKFUNC(spi3cs1),
+ NPCM7XX_MKFUNC(spi3quad),
+ NPCM7XX_MKFUNC(spi3cs2),
+ NPCM7XX_MKFUNC(spi3cs3),
+ NPCM7XX_MKFUNC(spi0cs1),
+ NPCM7XX_MKFUNC(lpc),
+ NPCM7XX_MKFUNC(lpcclk),
+ NPCM7XX_MKFUNC(espi),
+ NPCM7XX_MKFUNC(lkgpo0),
+ NPCM7XX_MKFUNC(lkgpo1),
+ NPCM7XX_MKFUNC(lkgpo2),
+ NPCM7XX_MKFUNC(nprd_smi),
+};
+
+#define NPCM7XX_PINCFG(a, b, c, d, e, f, g, h, i, j, k) \
+ [a] { .fn0 = fn_ ## b, .reg0 = NPCM7XX_GCR_ ## c, .bit0 = d, \
+ .fn1 = fn_ ## e, .reg1 = NPCM7XX_GCR_ ## f, .bit1 = g, \
+ .fn2 = fn_ ## h, .reg2 = NPCM7XX_GCR_ ## i, .bit2 = j, \
+ .flag = k }
+
+/* Drive strength controlled by NPCM7XX_GP_N_ODSC */
+#define DRIVE_STRENGTH_LO_SHIFT 8
+#define DRIVE_STRENGTH_HI_SHIFT 12
+#define DRIVE_STRENGTH_MASK 0x0000FF00
+
+#define DS(lo, hi) (((lo) << DRIVE_STRENGTH_LO_SHIFT) | \
+ ((hi) << DRIVE_STRENGTH_HI_SHIFT))
+#define DSLO(x) (((x) >> DRIVE_STRENGTH_LO_SHIFT) & 0xF)
+#define DSHI(x) (((x) >> DRIVE_STRENGTH_HI_SHIFT) & 0xF)
+
+#define GPI 0x1 /* Not GPO */
+#define GPO 0x2 /* Not GPI */
+#define SLEW 0x4 /* Has Slew Control, NPCM7XX_GP_N_OSRC */
+#define SLEWLPC 0x8 /* Has Slew Control, SRCNT.3 */
+
+struct npcm7xx_pincfg {
+ int flag;
+ int fn0, reg0, bit0;
+ int fn1, reg1, bit1;
+ int fn2, reg2, bit2;
+};
+
+static const struct npcm7xx_pincfg pincfg[] = {
+ /* PIN FUNCTION 1 FUNCTION 2 FUNCTION 3 FLAGS */
+ NPCM7XX_PINCFG(0, iox1, MFSEL1, 30, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(1, iox1, MFSEL1, 30, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(2, iox1, MFSEL1, 30, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(3, iox1, MFSEL1, 30, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(4, iox2, MFSEL3, 14, smb1d, I2CSEGSEL, 7, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(5, iox2, MFSEL3, 14, smb1d, I2CSEGSEL, 7, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(6, iox2, MFSEL3, 14, smb2d, I2CSEGSEL, 10, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(7, iox2, MFSEL3, 14, smb2d, I2CSEGSEL, 10, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(8, lkgpo1, FLOCKR1, 4, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(9, lkgpo2, FLOCKR1, 8, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(10, ioxh, MFSEL3, 18, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(11, ioxh, MFSEL3, 18, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(12, gspi, MFSEL1, 24, smb5b, I2CSEGSEL, 19, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(13, gspi, MFSEL1, 24, smb5b, I2CSEGSEL, 19, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(14, gspi, MFSEL1, 24, smb5c, I2CSEGSEL, 20, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(15, gspi, MFSEL1, 24, smb5c, I2CSEGSEL, 20, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(16, lkgpo0, FLOCKR1, 0, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(17, pspi2, MFSEL3, 13, smb4den, I2CSEGSEL, 23, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(18, pspi2, MFSEL3, 13, smb4b, I2CSEGSEL, 14, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(19, pspi2, MFSEL3, 13, smb4b, I2CSEGSEL, 14, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(20, smb4c, I2CSEGSEL, 15, smb15, MFSEL3, 8, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(21, smb4c, I2CSEGSEL, 15, smb15, MFSEL3, 8, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(22, smb4d, I2CSEGSEL, 16, smb14, MFSEL3, 7, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(23, smb4d, I2CSEGSEL, 16, smb14, MFSEL3, 7, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(24, ioxh, MFSEL3, 18, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(25, ioxh, MFSEL3, 18, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(26, smb5, MFSEL1, 2, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(27, smb5, MFSEL1, 2, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(28, smb4, MFSEL1, 1, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(29, smb4, MFSEL1, 1, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(30, smb3, MFSEL1, 0, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(31, smb3, MFSEL1, 0, none, NONE, 0, none, NONE, 0, 0),
+
+ NPCM7XX_PINCFG(32, spi0cs1, MFSEL1, 3, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(33, none, NONE, 0, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(34, none, NONE, 0, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(37, smb3c, I2CSEGSEL, 12, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(38, smb3c, I2CSEGSEL, 12, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(39, smb3b, I2CSEGSEL, 11, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(40, smb3b, I2CSEGSEL, 11, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(41, bmcuart0a, MFSEL1, 9, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(42, bmcuart0a, MFSEL1, 9, none, NONE, 0, none, NONE, 0, DS(2, 4) | GPO),
+ NPCM7XX_PINCFG(43, uart1, MFSEL1, 10, jtag2, MFSEL4, 0, bmcuart1, MFSEL3, 24, 0),
+ NPCM7XX_PINCFG(44, uart1, MFSEL1, 10, jtag2, MFSEL4, 0, bmcuart1, MFSEL3, 24, 0),
+ NPCM7XX_PINCFG(45, uart1, MFSEL1, 10, jtag2, MFSEL4, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(46, uart1, MFSEL1, 10, jtag2, MFSEL4, 0, none, NONE, 0, DS(2, 8)),
+ NPCM7XX_PINCFG(47, uart1, MFSEL1, 10, jtag2, MFSEL4, 0, none, NONE, 0, DS(2, 8)),
+ NPCM7XX_PINCFG(48, uart2, MFSEL1, 11, bmcuart0b, MFSEL4, 1, none, NONE, 0, GPO),
+ NPCM7XX_PINCFG(49, uart2, MFSEL1, 11, bmcuart0b, MFSEL4, 1, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(50, uart2, MFSEL1, 11, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(51, uart2, MFSEL1, 11, none, NONE, 0, none, NONE, 0, GPO),
+ NPCM7XX_PINCFG(52, uart2, MFSEL1, 11, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(53, uart2, MFSEL1, 11, none, NONE, 0, none, NONE, 0, GPO),
+ NPCM7XX_PINCFG(54, uart2, MFSEL1, 11, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(55, uart2, MFSEL1, 11, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(56, r1err, MFSEL1, 12, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(57, r1md, MFSEL1, 13, none, NONE, 0, none, NONE, 0, DS(2, 4)),
+ NPCM7XX_PINCFG(58, r1md, MFSEL1, 13, none, NONE, 0, none, NONE, 0, DS(2, 4)),
+ NPCM7XX_PINCFG(59, smb3d, I2CSEGSEL, 13, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(60, smb3d, I2CSEGSEL, 13, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(61, uart1, MFSEL1, 10, none, NONE, 0, none, NONE, 0, GPO),
+ NPCM7XX_PINCFG(62, uart1, MFSEL1, 10, bmcuart1, MFSEL3, 24, none, NONE, 0, GPO),
+ NPCM7XX_PINCFG(63, uart1, MFSEL1, 10, bmcuart1, MFSEL3, 24, none, NONE, 0, GPO),
+
+ NPCM7XX_PINCFG(64, fanin0, MFSEL2, 0, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(65, fanin1, MFSEL2, 1, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(66, fanin2, MFSEL2, 2, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(67, fanin3, MFSEL2, 3, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(68, fanin4, MFSEL2, 4, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(69, fanin5, MFSEL2, 5, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(70, fanin6, MFSEL2, 6, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(71, fanin7, MFSEL2, 7, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(72, fanin8, MFSEL2, 8, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(73, fanin9, MFSEL2, 9, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(74, fanin10, MFSEL2, 10, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(75, fanin11, MFSEL2, 11, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(76, fanin12, MFSEL2, 12, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(77, fanin13, MFSEL2, 13, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(78, fanin14, MFSEL2, 14, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(79, fanin15, MFSEL2, 15, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(80, pwm0, MFSEL2, 16, none, NONE, 0, none, NONE, 0, DS(4, 8)),
+ NPCM7XX_PINCFG(81, pwm1, MFSEL2, 17, none, NONE, 0, none, NONE, 0, DS(4, 8)),
+ NPCM7XX_PINCFG(82, pwm2, MFSEL2, 18, none, NONE, 0, none, NONE, 0, DS(4, 8)),
+ NPCM7XX_PINCFG(83, pwm3, MFSEL2, 19, none, NONE, 0, none, NONE, 0, DS(4, 8)),
+ NPCM7XX_PINCFG(84, r2, MFSEL1, 14, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(85, r2, MFSEL1, 14, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(86, r2, MFSEL1, 14, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(87, r2, MFSEL1, 14, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(88, r2, MFSEL1, 14, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(89, r2, MFSEL1, 14, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(90, r2err, MFSEL1, 15, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(91, r2md, MFSEL1, 16, none, NONE, 0, none, NONE, 0, DS(2, 4)),
+ NPCM7XX_PINCFG(92, r2md, MFSEL1, 16, none, NONE, 0, none, NONE, 0, DS(2, 4)),
+ NPCM7XX_PINCFG(93, ga20kbc, MFSEL1, 17, smb5d, I2CSEGSEL, 21, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(94, ga20kbc, MFSEL1, 17, smb5d, I2CSEGSEL, 21, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(95, lpc, NONE, 0, espi, MFSEL4, 8, gpio, MFSEL1, 26, 0),
+
+ NPCM7XX_PINCFG(96, rg1, MFSEL4, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(97, rg1, MFSEL4, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(98, rg1, MFSEL4, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(99, rg1, MFSEL4, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(100, rg1, MFSEL4, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(101, rg1, MFSEL4, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(102, rg1, MFSEL4, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(103, rg1, MFSEL4, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(104, rg1, MFSEL4, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(105, rg1, MFSEL4, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(106, rg1, MFSEL4, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(107, rg1, MFSEL4, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(108, rg1mdio, MFSEL4, 21, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(109, rg1mdio, MFSEL4, 21, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(110, rg2, MFSEL4, 24, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(111, rg2, MFSEL4, 24, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(112, rg2, MFSEL4, 24, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(113, rg2, MFSEL4, 24, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(114, smb0, MFSEL1, 6, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(115, smb0, MFSEL1, 6, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(116, smb1, MFSEL1, 7, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(117, smb1, MFSEL1, 7, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(118, smb2, MFSEL1, 8, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(119, smb2, MFSEL1, 8, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(120, smb2c, I2CSEGSEL, 9, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(121, smb2c, I2CSEGSEL, 9, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(122, smb2b, I2CSEGSEL, 8, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(123, smb2b, I2CSEGSEL, 8, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(124, smb1c, I2CSEGSEL, 6, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(125, smb1c, I2CSEGSEL, 6, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(126, smb1b, I2CSEGSEL, 5, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(127, smb1b, I2CSEGSEL, 5, none, NONE, 0, none, NONE, 0, SLEW),
+
+ NPCM7XX_PINCFG(128, smb8, MFSEL4, 11, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(129, smb8, MFSEL4, 11, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(130, smb9, MFSEL4, 12, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(131, smb9, MFSEL4, 12, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(132, smb10, MFSEL4, 13, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(133, smb10, MFSEL4, 13, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(134, smb11, MFSEL4, 14, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(135, smb11, MFSEL4, 14, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(136, sd1, MFSEL3, 12, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(137, sd1, MFSEL3, 12, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(138, sd1, MFSEL3, 12, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(139, sd1, MFSEL3, 12, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(140, sd1, MFSEL3, 12, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(141, sd1, MFSEL3, 12, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(142, sd1, MFSEL3, 12, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(143, sd1, MFSEL3, 12, sd1pwr, MFSEL4, 5, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(144, pwm4, MFSEL2, 20, none, NONE, 0, none, NONE, 0, DS(4, 8)),
+ NPCM7XX_PINCFG(145, pwm5, MFSEL2, 21, none, NONE, 0, none, NONE, 0, DS(4, 8)),
+ NPCM7XX_PINCFG(146, pwm6, MFSEL2, 22, none, NONE, 0, none, NONE, 0, DS(4, 8)),
+ NPCM7XX_PINCFG(147, pwm7, MFSEL2, 23, none, NONE, 0, none, NONE, 0, DS(4, 8)),
+ NPCM7XX_PINCFG(148, mmc8, MFSEL3, 11, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(149, mmc8, MFSEL3, 11, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(150, mmc8, MFSEL3, 11, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(151, mmc8, MFSEL3, 11, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(152, mmc, MFSEL3, 10, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(153, mmcwp, FLOCKR1, 24, none, NONE, 0, none, NONE, 0, 0), /* Z1/A1 */
+ NPCM7XX_PINCFG(154, mmc, MFSEL3, 10, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(155, mmccd, MFSEL3, 25, mmcrst, MFSEL4, 6, none, NONE, 0, 0), /* Z1/A1 */
+ NPCM7XX_PINCFG(156, mmc, MFSEL3, 10, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(157, mmc, MFSEL3, 10, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(158, mmc, MFSEL3, 10, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(159, mmc, MFSEL3, 10, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+
+ NPCM7XX_PINCFG(160, clkout, MFSEL1, 21, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(161, lpc, NONE, 0, espi, MFSEL4, 8, gpio, MFSEL1, 26, DS(8, 12)),
+ NPCM7XX_PINCFG(162, serirq, NONE, 0, gpio, MFSEL1, 31, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(163, lpc, NONE, 0, espi, MFSEL4, 8, gpio, MFSEL1, 26, 0),
+ NPCM7XX_PINCFG(164, lpc, NONE, 0, espi, MFSEL4, 8, gpio, MFSEL1, 26, SLEWLPC),
+ NPCM7XX_PINCFG(165, lpc, NONE, 0, espi, MFSEL4, 8, gpio, MFSEL1, 26, SLEWLPC),
+ NPCM7XX_PINCFG(166, lpc, NONE, 0, espi, MFSEL4, 8, gpio, MFSEL1, 26, SLEWLPC),
+ NPCM7XX_PINCFG(167, lpc, NONE, 0, espi, MFSEL4, 8, gpio, MFSEL1, 26, SLEWLPC),
+ NPCM7XX_PINCFG(168, lpcclk, NONE, 0, espi, MFSEL4, 8, gpio, MFSEL3, 16, 0),
+ NPCM7XX_PINCFG(169, scipme, MFSEL3, 0, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(170, sci, MFSEL1, 22, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(171, smb6, MFSEL3, 1, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(172, smb6, MFSEL3, 1, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(173, smb7, MFSEL3, 2, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(174, smb7, MFSEL3, 2, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(175, pspi1, MFSEL3, 4, faninx, MFSEL3, 3, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(176, pspi1, MFSEL3, 4, faninx, MFSEL3, 3, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(177, pspi1, MFSEL3, 4, faninx, MFSEL3, 3, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(178, r1, MFSEL3, 9, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(179, r1, MFSEL3, 9, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(180, r1, MFSEL3, 9, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(181, r1, MFSEL3, 9, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(182, r1, MFSEL3, 9, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(183, spi3, MFSEL4, 16, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(184, spi3, MFSEL4, 16, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW | GPO),
+ NPCM7XX_PINCFG(185, spi3, MFSEL4, 16, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW | GPO),
+ NPCM7XX_PINCFG(186, spi3, MFSEL4, 16, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(187, spi3cs1, MFSEL4, 17, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(188, spi3quad, MFSEL4, 20, spi3cs2, MFSEL4, 18, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(189, spi3quad, MFSEL4, 20, spi3cs3, MFSEL4, 19, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(190, gpio, FLOCKR1, 20, nprd_smi, NONE, 0, none, NONE, 0, DS(2, 4)),
+ NPCM7XX_PINCFG(191, none, NONE, 0, none, NONE, 0, none, NONE, 0, DS(8, 12)), /* XX */
+
+ NPCM7XX_PINCFG(192, none, NONE, 0, none, NONE, 0, none, NONE, 0, DS(8, 12)), /* XX */
+ NPCM7XX_PINCFG(193, r1, MFSEL3, 9, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(194, smb0b, I2CSEGSEL, 0, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(195, smb0b, I2CSEGSEL, 0, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(196, smb0c, I2CSEGSEL, 1, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(197, smb0den, I2CSEGSEL, 22, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(198, smb0d, I2CSEGSEL, 2, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(199, smb0d, I2CSEGSEL, 2, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(200, r2, MFSEL1, 14, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(201, r1, MFSEL3, 9, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(202, smb0c, I2CSEGSEL, 1, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(203, faninx, MFSEL3, 3, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(204, ddc, NONE, 0, gpio, MFSEL3, 22, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(205, ddc, NONE, 0, gpio, MFSEL3, 22, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(206, ddc, NONE, 0, gpio, MFSEL3, 22, none, NONE, 0, DS(4, 8)),
+ NPCM7XX_PINCFG(207, ddc, NONE, 0, gpio, MFSEL3, 22, none, NONE, 0, DS(4, 8)),
+ NPCM7XX_PINCFG(208, rg2, MFSEL4, 24, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(209, rg2, MFSEL4, 24, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(210, rg2, MFSEL4, 24, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(211, rg2, MFSEL4, 24, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(212, rg2, MFSEL4, 24, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(213, rg2, MFSEL4, 24, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(214, rg2, MFSEL4, 24, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(215, rg2, MFSEL4, 24, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(216, rg2mdio, MFSEL4, 23, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(217, rg2mdio, MFSEL4, 23, ddr, MFSEL3, 26, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(218, wdog1, MFSEL3, 19, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(219, wdog2, MFSEL3, 20, none, NONE, 0, none, NONE, 0, DS(4, 8)),
+ NPCM7XX_PINCFG(220, smb12, MFSEL3, 5, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(221, smb12, MFSEL3, 5, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(222, smb13, MFSEL3, 6, none, NONE, 0, none, NONE, 0, 0),
+ NPCM7XX_PINCFG(223, smb13, MFSEL3, 6, none, NONE, 0, none, NONE, 0, 0),
+
+ NPCM7XX_PINCFG(224, spix, MFSEL4, 27, none, NONE, 0, none, NONE, 0, SLEW),
+ NPCM7XX_PINCFG(225, spix, MFSEL4, 27, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW | GPO),
+ NPCM7XX_PINCFG(226, spix, MFSEL4, 27, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW | GPO),
+ NPCM7XX_PINCFG(227, spix, MFSEL4, 27, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(228, spixcs1, MFSEL4, 28, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(229, spix, MFSEL4, 27, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(230, spix, MFSEL4, 27, none, NONE, 0, none, NONE, 0, DS(8, 12) | SLEW),
+ NPCM7XX_PINCFG(231, clkreq, MFSEL4, 9, none, NONE, 0, none, NONE, 0, DS(8, 12)),
+ NPCM7XX_PINCFG(253, none, NONE, 0, none, NONE, 0, none, NONE, 0, GPI), /* SDHC1 power */
+ NPCM7XX_PINCFG(254, none, NONE, 0, none, NONE, 0, none, NONE, 0, GPI), /* SDHC2 power */
+ NPCM7XX_PINCFG(255, none, NONE, 0, none, NONE, 0, none, NONE, 0, GPI), /* DACOSEL */
+};
+
+/* number, name, drv_data */
+static const struct pinctrl_pin_desc npcm7xx_pins[] = {
+ PINCTRL_PIN(0, "GPIO0/IOX1DI"),
+ PINCTRL_PIN(1, "GPIO1/IOX1LD"),
+ PINCTRL_PIN(2, "GPIO2/IOX1CK"),
+ PINCTRL_PIN(3, "GPIO3/IOX1D0"),
+ PINCTRL_PIN(4, "GPIO4/IOX2DI/SMB1DSDA"),
+ PINCTRL_PIN(5, "GPIO5/IOX2LD/SMB1DSCL"),
+ PINCTRL_PIN(6, "GPIO6/IOX2CK/SMB2DSDA"),
+ PINCTRL_PIN(7, "GPIO7/IOX2D0/SMB2DSCL"),
+ PINCTRL_PIN(8, "GPIO8/LKGPO1"),
+ PINCTRL_PIN(9, "GPIO9/LKGPO2"),
+ PINCTRL_PIN(10, "GPIO10/IOXHLD"),
+ PINCTRL_PIN(11, "GPIO11/IOXHCK"),
+ PINCTRL_PIN(12, "GPIO12/GSPICK/SMB5BSCL"),
+ PINCTRL_PIN(13, "GPIO13/GSPIDO/SMB5BSDA"),
+ PINCTRL_PIN(14, "GPIO14/GSPIDI/SMB5CSCL"),
+ PINCTRL_PIN(15, "GPIO15/GSPICS/SMB5CSDA"),
+ PINCTRL_PIN(16, "GPIO16/LKGPO0"),
+ PINCTRL_PIN(17, "GPIO17/PSPI2DI/SMB4DEN"),
+ PINCTRL_PIN(18, "GPIO18/PSPI2D0/SMB4BSDA"),
+ PINCTRL_PIN(19, "GPIO19/PSPI2CK/SMB4BSCL"),
+ PINCTRL_PIN(20, "GPIO20/SMB4CSDA/SMB15SDA"),
+ PINCTRL_PIN(21, "GPIO21/SMB4CSCL/SMB15SCL"),
+ PINCTRL_PIN(22, "GPIO22/SMB4DSDA/SMB14SDA"),
+ PINCTRL_PIN(23, "GPIO23/SMB4DSCL/SMB14SCL"),
+ PINCTRL_PIN(24, "GPIO24/IOXHDO"),
+ PINCTRL_PIN(25, "GPIO25/IOXHDI"),
+ PINCTRL_PIN(26, "GPIO26/SMB5SDA"),
+ PINCTRL_PIN(27, "GPIO27/SMB5SCL"),
+ PINCTRL_PIN(28, "GPIO28/SMB4SDA"),
+ PINCTRL_PIN(29, "GPIO29/SMB4SCL"),
+ PINCTRL_PIN(30, "GPIO30/SMB3SDA"),
+ PINCTRL_PIN(31, "GPIO31/SMB3SCL"),
+
+ PINCTRL_PIN(32, "GPIO32/nSPI0CS1"),
+ PINCTRL_PIN(33, "SPI0D2"),
+ PINCTRL_PIN(34, "SPI0D3"),
+ PINCTRL_PIN(37, "GPIO37/SMB3CSDA"),
+ PINCTRL_PIN(38, "GPIO38/SMB3CSCL"),
+ PINCTRL_PIN(39, "GPIO39/SMB3BSDA"),
+ PINCTRL_PIN(40, "GPIO40/SMB3BSCL"),
+ PINCTRL_PIN(41, "GPIO41/BSPRXD"),
+ PINCTRL_PIN(42, "GPO42/BSPTXD/STRAP11"),
+ PINCTRL_PIN(43, "GPIO43/RXD1/JTMS2/BU1RXD"),
+ PINCTRL_PIN(44, "GPIO44/nCTS1/JTDI2/BU1CTS"),
+ PINCTRL_PIN(45, "GPIO45/nDCD1/JTDO2"),
+ PINCTRL_PIN(46, "GPIO46/nDSR1/JTCK2"),
+ PINCTRL_PIN(47, "GPIO47/nRI1/JCP_RDY2"),
+ PINCTRL_PIN(48, "GPIO48/TXD2/BSPTXD"),
+ PINCTRL_PIN(49, "GPIO49/RXD2/BSPRXD"),
+ PINCTRL_PIN(50, "GPIO50/nCTS2"),
+ PINCTRL_PIN(51, "GPO51/nRTS2/STRAP2"),
+ PINCTRL_PIN(52, "GPIO52/nDCD2"),
+ PINCTRL_PIN(53, "GPO53/nDTR2_BOUT2/STRAP1"),
+ PINCTRL_PIN(54, "GPIO54/nDSR2"),
+ PINCTRL_PIN(55, "GPIO55/nRI2"),
+ PINCTRL_PIN(56, "GPIO56/R1RXERR"),
+ PINCTRL_PIN(57, "GPIO57/R1MDC"),
+ PINCTRL_PIN(58, "GPIO58/R1MDIO"),
+ PINCTRL_PIN(59, "GPIO59/SMB3DSDA"),
+ PINCTRL_PIN(60, "GPIO60/SMB3DSCL"),
+ PINCTRL_PIN(61, "GPO61/nDTR1_BOUT1/STRAP6"),
+ PINCTRL_PIN(62, "GPO62/nRTST1/STRAP5"),
+ PINCTRL_PIN(63, "GPO63/TXD1/STRAP4"),
+
+ PINCTRL_PIN(64, "GPIO64/FANIN0"),
+ PINCTRL_PIN(65, "GPIO65/FANIN1"),
+ PINCTRL_PIN(66, "GPIO66/FANIN2"),
+ PINCTRL_PIN(67, "GPIO67/FANIN3"),
+ PINCTRL_PIN(68, "GPIO68/FANIN4"),
+ PINCTRL_PIN(69, "GPIO69/FANIN5"),
+ PINCTRL_PIN(70, "GPIO70/FANIN6"),
+ PINCTRL_PIN(71, "GPIO71/FANIN7"),
+ PINCTRL_PIN(72, "GPIO72/FANIN8"),
+ PINCTRL_PIN(73, "GPIO73/FANIN9"),
+ PINCTRL_PIN(74, "GPIO74/FANIN10"),
+ PINCTRL_PIN(75, "GPIO75/FANIN11"),
+ PINCTRL_PIN(76, "GPIO76/FANIN12"),
+ PINCTRL_PIN(77, "GPIO77/FANIN13"),
+ PINCTRL_PIN(78, "GPIO78/FANIN14"),
+ PINCTRL_PIN(79, "GPIO79/FANIN15"),
+ PINCTRL_PIN(80, "GPIO80/PWM0"),
+ PINCTRL_PIN(81, "GPIO81/PWM1"),
+ PINCTRL_PIN(82, "GPIO82/PWM2"),
+ PINCTRL_PIN(83, "GPIO83/PWM3"),
+ PINCTRL_PIN(84, "GPIO84/R2TXD0"),
+ PINCTRL_PIN(85, "GPIO85/R2TXD1"),
+ PINCTRL_PIN(86, "GPIO86/R2TXEN"),
+ PINCTRL_PIN(87, "GPIO87/R2RXD0"),
+ PINCTRL_PIN(88, "GPIO88/R2RXD1"),
+ PINCTRL_PIN(89, "GPIO89/R2CRSDV"),
+ PINCTRL_PIN(90, "GPIO90/R2RXERR"),
+ PINCTRL_PIN(91, "GPIO91/R2MDC"),
+ PINCTRL_PIN(92, "GPIO92/R2MDIO"),
+ PINCTRL_PIN(93, "GPIO93/GA20/SMB5DSCL"),
+ PINCTRL_PIN(94, "GPIO94/nKBRST/SMB5DSDA"),
+ PINCTRL_PIN(95, "GPIO95/nLRESET/nESPIRST"),
+
+ PINCTRL_PIN(96, "GPIO96/RG1TXD0"),
+ PINCTRL_PIN(97, "GPIO97/RG1TXD1"),
+ PINCTRL_PIN(98, "GPIO98/RG1TXD2"),
+ PINCTRL_PIN(99, "GPIO99/RG1TXD3"),
+ PINCTRL_PIN(100, "GPIO100/RG1TXC"),
+ PINCTRL_PIN(101, "GPIO101/RG1TXCTL"),
+ PINCTRL_PIN(102, "GPIO102/RG1RXD0"),
+ PINCTRL_PIN(103, "GPIO103/RG1RXD1"),
+ PINCTRL_PIN(104, "GPIO104/RG1RXD2"),
+ PINCTRL_PIN(105, "GPIO105/RG1RXD3"),
+ PINCTRL_PIN(106, "GPIO106/RG1RXC"),
+ PINCTRL_PIN(107, "GPIO107/RG1RXCTL"),
+ PINCTRL_PIN(108, "GPIO108/RG1MDC"),
+ PINCTRL_PIN(109, "GPIO109/RG1MDIO"),
+ PINCTRL_PIN(110, "GPIO110/RG2TXD0/DDRV0"),
+ PINCTRL_PIN(111, "GPIO111/RG2TXD1/DDRV1"),
+ PINCTRL_PIN(112, "GPIO112/RG2TXD2/DDRV2"),
+ PINCTRL_PIN(113, "GPIO113/RG2TXD3/DDRV3"),
+ PINCTRL_PIN(114, "GPIO114/SMB0SCL"),
+ PINCTRL_PIN(115, "GPIO115/SMB0SDA"),
+ PINCTRL_PIN(116, "GPIO116/SMB1SCL"),
+ PINCTRL_PIN(117, "GPIO117/SMB1SDA"),
+ PINCTRL_PIN(118, "GPIO118/SMB2SCL"),
+ PINCTRL_PIN(119, "GPIO119/SMB2SDA"),
+ PINCTRL_PIN(120, "GPIO120/SMB2CSDA"),
+ PINCTRL_PIN(121, "GPIO121/SMB2CSCL"),
+ PINCTRL_PIN(122, "GPIO122/SMB2BSDA"),
+ PINCTRL_PIN(123, "GPIO123/SMB2BSCL"),
+ PINCTRL_PIN(124, "GPIO124/SMB1CSDA"),
+ PINCTRL_PIN(125, "GPIO125/SMB1CSCL"),
+ PINCTRL_PIN(126, "GPIO126/SMB1BSDA"),
+ PINCTRL_PIN(127, "GPIO127/SMB1BSCL"),
+
+ PINCTRL_PIN(128, "GPIO128/SMB8SCL"),
+ PINCTRL_PIN(129, "GPIO129/SMB8SDA"),
+ PINCTRL_PIN(130, "GPIO130/SMB9SCL"),
+ PINCTRL_PIN(131, "GPIO131/SMB9SDA"),
+ PINCTRL_PIN(132, "GPIO132/SMB10SCL"),
+ PINCTRL_PIN(133, "GPIO133/SMB10SDA"),
+ PINCTRL_PIN(134, "GPIO134/SMB11SCL"),
+ PINCTRL_PIN(135, "GPIO135/SMB11SDA"),
+ PINCTRL_PIN(136, "GPIO136/SD1DT0"),
+ PINCTRL_PIN(137, "GPIO137/SD1DT1"),
+ PINCTRL_PIN(138, "GPIO138/SD1DT2"),
+ PINCTRL_PIN(139, "GPIO139/SD1DT3"),
+ PINCTRL_PIN(140, "GPIO140/SD1CLK"),
+ PINCTRL_PIN(141, "GPIO141/SD1WP"),
+ PINCTRL_PIN(142, "GPIO142/SD1CMD"),
+ PINCTRL_PIN(143, "GPIO143/SD1CD/SD1PWR"),
+ PINCTRL_PIN(144, "GPIO144/PWM4"),
+ PINCTRL_PIN(145, "GPIO145/PWM5"),
+ PINCTRL_PIN(146, "GPIO146/PWM6"),
+ PINCTRL_PIN(147, "GPIO147/PWM7"),
+ PINCTRL_PIN(148, "GPIO148/MMCDT4"),
+ PINCTRL_PIN(149, "GPIO149/MMCDT5"),
+ PINCTRL_PIN(150, "GPIO150/MMCDT6"),
+ PINCTRL_PIN(151, "GPIO151/MMCDT7"),
+ PINCTRL_PIN(152, "GPIO152/MMCCLK"),
+ PINCTRL_PIN(153, "GPIO153/MMCWP"),
+ PINCTRL_PIN(154, "GPIO154/MMCCMD"),
+ PINCTRL_PIN(155, "GPIO155/nMMCCD/nMMCRST"),
+ PINCTRL_PIN(156, "GPIO156/MMCDT0"),
+ PINCTRL_PIN(157, "GPIO157/MMCDT1"),
+ PINCTRL_PIN(158, "GPIO158/MMCDT2"),
+ PINCTRL_PIN(159, "GPIO159/MMCDT3"),
+
+ PINCTRL_PIN(160, "GPIO160/CLKOUT/RNGOSCOUT"),
+ PINCTRL_PIN(161, "GPIO161/nLFRAME/nESPICS"),
+ PINCTRL_PIN(162, "GPIO162/SERIRQ"),
+ PINCTRL_PIN(163, "GPIO163/LCLK/ESPICLK"),
+ PINCTRL_PIN(164, "GPIO164/LAD0/ESPI_IO0"/*dscnt6*/),
+ PINCTRL_PIN(165, "GPIO165/LAD1/ESPI_IO1"/*dscnt6*/),
+ PINCTRL_PIN(166, "GPIO166/LAD2/ESPI_IO2"/*dscnt6*/),
+ PINCTRL_PIN(167, "GPIO167/LAD3/ESPI_IO3"/*dscnt6*/),
+ PINCTRL_PIN(168, "GPIO168/nCLKRUN/nESPIALERT"),
+ PINCTRL_PIN(169, "GPIO169/nSCIPME"),
+ PINCTRL_PIN(170, "GPIO170/nSMI"),
+ PINCTRL_PIN(171, "GPIO171/SMB6SCL"),
+ PINCTRL_PIN(172, "GPIO172/SMB6SDA"),
+ PINCTRL_PIN(173, "GPIO173/SMB7SCL"),
+ PINCTRL_PIN(174, "GPIO174/SMB7SDA"),
+ PINCTRL_PIN(175, "GPIO175/PSPI1CK/FANIN19"),
+ PINCTRL_PIN(176, "GPIO176/PSPI1DO/FANIN18"),
+ PINCTRL_PIN(177, "GPIO177/PSPI1DI/FANIN17"),
+ PINCTRL_PIN(178, "GPIO178/R1TXD0"),
+ PINCTRL_PIN(179, "GPIO179/R1TXD1"),
+ PINCTRL_PIN(180, "GPIO180/R1TXEN"),
+ PINCTRL_PIN(181, "GPIO181/R1RXD0"),
+ PINCTRL_PIN(182, "GPIO182/R1RXD1"),
+ PINCTRL_PIN(183, "GPIO183/SPI3CK"),
+ PINCTRL_PIN(184, "GPO184/SPI3D0/STRAP9"),
+ PINCTRL_PIN(185, "GPO185/SPI3D1/STRAP10"),
+ PINCTRL_PIN(186, "GPIO186/nSPI3CS0"),
+ PINCTRL_PIN(187, "GPIO187/nSPI3CS1"),
+ PINCTRL_PIN(188, "GPIO188/SPI3D2/nSPI3CS2"),
+ PINCTRL_PIN(189, "GPIO189/SPI3D3/nSPI3CS3"),
+ PINCTRL_PIN(190, "GPIO190/nPRD_SMI"),
+ PINCTRL_PIN(191, "GPIO191"),
+
+ PINCTRL_PIN(192, "GPIO192"),
+ PINCTRL_PIN(193, "GPIO193/R1CRSDV"),
+ PINCTRL_PIN(194, "GPIO194/SMB0BSCL"),
+ PINCTRL_PIN(195, "GPIO195/SMB0BSDA"),
+ PINCTRL_PIN(196, "GPIO196/SMB0CSCL"),
+ PINCTRL_PIN(197, "GPIO197/SMB0DEN"),
+ PINCTRL_PIN(198, "GPIO198/SMB0DSDA"),
+ PINCTRL_PIN(199, "GPIO199/SMB0DSCL"),
+ PINCTRL_PIN(200, "GPIO200/R2CK"),
+ PINCTRL_PIN(201, "GPIO201/R1CK"),
+ PINCTRL_PIN(202, "GPIO202/SMB0CSDA"),
+ PINCTRL_PIN(203, "GPIO203/FANIN16"),
+ PINCTRL_PIN(204, "GPIO204/DDC2SCL"),
+ PINCTRL_PIN(205, "GPIO205/DDC2SDA"),
+ PINCTRL_PIN(206, "GPIO206/HSYNC2"),
+ PINCTRL_PIN(207, "GPIO207/VSYNC2"),
+ PINCTRL_PIN(208, "GPIO208/RG2TXC/DVCK"),
+ PINCTRL_PIN(209, "GPIO209/RG2TXCTL/DDRV4"),
+ PINCTRL_PIN(210, "GPIO210/RG2RXD0/DDRV5"),
+ PINCTRL_PIN(211, "GPIO211/RG2RXD1/DDRV6"),
+ PINCTRL_PIN(212, "GPIO212/RG2RXD2/DDRV7"),
+ PINCTRL_PIN(213, "GPIO213/RG2RXD3/DDRV8"),
+ PINCTRL_PIN(214, "GPIO214/RG2RXC/DDRV9"),
+ PINCTRL_PIN(215, "GPIO215/RG2RXCTL/DDRV10"),
+ PINCTRL_PIN(216, "GPIO216/RG2MDC/DDRV11"),
+ PINCTRL_PIN(217, "GPIO217/RG2MDIO/DVHSYNC"),
+ PINCTRL_PIN(218, "GPIO218/nWDO1"),
+ PINCTRL_PIN(219, "GPIO219/nWDO2"),
+ PINCTRL_PIN(220, "GPIO220/SMB12SCL"),
+ PINCTRL_PIN(221, "GPIO221/SMB12SDA"),
+ PINCTRL_PIN(222, "GPIO222/SMB13SCL"),
+ PINCTRL_PIN(223, "GPIO223/SMB13SDA"),
+
+ PINCTRL_PIN(224, "GPIO224/SPIXCK"),
+ PINCTRL_PIN(225, "GPO225/SPIXD0/STRAP12"),
+ PINCTRL_PIN(226, "GPO226/SPIXD1/STRAP13"),
+ PINCTRL_PIN(227, "GPIO227/nSPIXCS0"),
+ PINCTRL_PIN(228, "GPIO228/nSPIXCS1"),
+ PINCTRL_PIN(229, "GPO229/SPIXD2/STRAP3"),
+ PINCTRL_PIN(230, "GPIO230/SPIXD3"),
+ PINCTRL_PIN(231, "GPIO231/nCLKREQ"),
+ PINCTRL_PIN(255, "GPI255/DACOSEL"),
+};
+
+/* Enable mode in pin group */
+static void npcm7xx_setfunc(struct regmap *gcr_regmap, const unsigned int *pin,
+ int pin_number, int mode)
+{
+ const struct npcm7xx_pincfg *cfg;
+ int i;
+
+ for (i = 0 ; i < pin_number ; i++) {
+ cfg = &pincfg[pin[i]];
+ if (mode == fn_gpio || cfg->fn0 == mode || cfg->fn1 == mode || cfg->fn2 == mode) {
+ if (cfg->reg0)
+ regmap_update_bits(gcr_regmap, cfg->reg0,
+ BIT(cfg->bit0),
+ !!(cfg->fn0 == mode) ?
+ BIT(cfg->bit0) : 0);
+ if (cfg->reg1)
+ regmap_update_bits(gcr_regmap, cfg->reg1,
+ BIT(cfg->bit1),
+ !!(cfg->fn1 == mode) ?
+ BIT(cfg->bit1) : 0);
+ if (cfg->reg2)
+ regmap_update_bits(gcr_regmap, cfg->reg2,
+ BIT(cfg->bit2),
+ !!(cfg->fn2 == mode) ?
+ BIT(cfg->bit2) : 0);
+ }
+ }
+}
+
+/* Get slew rate of pin (high/low) */
+static int npcm7xx_get_slew_rate(struct npcm7xx_gpio *bank,
+ struct regmap *gcr_regmap, unsigned int pin)
+{
+ u32 val;
+ int gpio = (pin % bank->gc.ngpio);
+ unsigned long pinmask = BIT(gpio);
+
+ if (pincfg[pin].flag & SLEW)
+ return ioread32(bank->base + NPCM7XX_GP_N_OSRC)
+ & pinmask;
+ /* LPC Slew rate in SRCNT register */
+ if (pincfg[pin].flag & SLEWLPC) {
+ regmap_read(gcr_regmap, NPCM7XX_GCR_SRCNT, &val);
+ return !!(val & SRCNT_ESPI);
+ }
+
+ return -EINVAL;
+}
+
+/* Set slew rate of pin (high/low) */
+static int npcm7xx_set_slew_rate(struct npcm7xx_gpio *bank,
+ struct regmap *gcr_regmap, unsigned int pin,
+ int arg)
+{
+ int gpio = BIT(pin % bank->gc.ngpio);
+
+ if (pincfg[pin].flag & SLEW) {
+ switch (arg) {
+ case 0:
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_OSRC,
+ gpio);
+ return 0;
+ case 1:
+ npcm_gpio_set(&bank->gc, bank->base + NPCM7XX_GP_N_OSRC,
+ gpio);
+ return 0;
+ default:
+ return -EINVAL;
+ }
+ }
+ /* LPC Slew rate in SRCNT register */
+ if (pincfg[pin].flag & SLEWLPC) {
+ switch (arg) {
+ case 0:
+ regmap_update_bits(gcr_regmap, NPCM7XX_GCR_SRCNT,
+ SRCNT_ESPI, 0);
+ return 0;
+ case 1:
+ regmap_update_bits(gcr_regmap, NPCM7XX_GCR_SRCNT,
+ SRCNT_ESPI, SRCNT_ESPI);
+ return 0;
+ default:
+ return -EINVAL;
+ }
+ }
+
+ return -EINVAL;
+}
+
+/* Get drive strength for a pin, if supported */
+static int npcm7xx_get_drive_strength(struct pinctrl_dev *pctldev,
+ unsigned int pin)
+{
+ struct npcm7xx_pinctrl *npcm = pinctrl_dev_get_drvdata(pctldev);
+ struct npcm7xx_gpio *bank =
+ &npcm->gpio_bank[pin / NPCM7XX_GPIO_PER_BANK];
+ int gpio = (pin % bank->gc.ngpio);
+ unsigned long pinmask = BIT(gpio);
+ u32 ds = 0;
+ int flg, val;
+
+ flg = pincfg[pin].flag;
+ if (flg & DRIVE_STRENGTH_MASK) {
+ /* Get standard reading */
+ val = ioread32(bank->base + NPCM7XX_GP_N_ODSC)
+ & pinmask;
+ ds = val ? DSHI(flg) : DSLO(flg);
+ dev_dbg(bank->gc.parent,
+ "pin %d strength %d = %d\n", pin, val, ds);
+ return ds;
+ }
+
+ return -EINVAL;
+}
+
+/* Set drive strength for a pin, if supported */
+static int npcm7xx_set_drive_strength(struct npcm7xx_pinctrl *npcm,
+ unsigned int pin, int nval)
+{
+ int v;
+ struct npcm7xx_gpio *bank =
+ &npcm->gpio_bank[pin / NPCM7XX_GPIO_PER_BANK];
+ int gpio = BIT(pin % bank->gc.ngpio);
+
+ v = (pincfg[pin].flag & DRIVE_STRENGTH_MASK);
+ if (!nval || !v)
+ return -ENOTSUPP;
+ if (DSLO(v) == nval) {
+ dev_dbg(bank->gc.parent,
+ "setting pin %d to low strength [%d]\n", pin, nval);
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_ODSC, gpio);
+ return 0;
+ } else if (DSHI(v) == nval) {
+ dev_dbg(bank->gc.parent,
+ "setting pin %d to high strength [%d]\n", pin, nval);
+ npcm_gpio_set(&bank->gc, bank->base + NPCM7XX_GP_N_ODSC, gpio);
+ return 0;
+ }
+
+ return -ENOTSUPP;
+}
+
+/* pinctrl_ops */
+static void npcm7xx_pin_dbg_show(struct pinctrl_dev *pctldev,
+ struct seq_file *s, unsigned int offset)
+{
+ seq_printf(s, "pinctrl_ops.dbg: %d", offset);
+}
+
+static int npcm7xx_get_groups_count(struct pinctrl_dev *pctldev)
+{
+ struct npcm7xx_pinctrl *npcm = pinctrl_dev_get_drvdata(pctldev);
+
+ dev_dbg(npcm->dev, "group size: %d\n", ARRAY_SIZE(npcm7xx_groups));
+ return ARRAY_SIZE(npcm7xx_groups);
+}
+
+static const char *npcm7xx_get_group_name(struct pinctrl_dev *pctldev,
+ unsigned int selector)
+{
+ return npcm7xx_groups[selector].name;
+}
+
+static int npcm7xx_get_group_pins(struct pinctrl_dev *pctldev,
+ unsigned int selector,
+ const unsigned int **pins,
+ unsigned int *npins)
+{
+ *npins = npcm7xx_groups[selector].npins;
+ *pins = npcm7xx_groups[selector].pins;
+
+ return 0;
+}
+
+static int npcm7xx_dt_node_to_map(struct pinctrl_dev *pctldev,
+ struct device_node *np_config,
+ struct pinctrl_map **map,
+ u32 *num_maps)
+{
+ struct npcm7xx_pinctrl *npcm = pinctrl_dev_get_drvdata(pctldev);
+
+ dev_dbg(npcm->dev, "dt_node_to_map: %s\n", np_config->name);
+ return pinconf_generic_dt_node_to_map(pctldev, np_config,
+ map, num_maps,
+ PIN_MAP_TYPE_INVALID);
+}
+
+static void npcm7xx_dt_free_map(struct pinctrl_dev *pctldev,
+ struct pinctrl_map *map, u32 num_maps)
+{
+ kfree(map);
+}
+
+static struct pinctrl_ops npcm7xx_pinctrl_ops = {
+ .get_groups_count = npcm7xx_get_groups_count,
+ .get_group_name = npcm7xx_get_group_name,
+ .get_group_pins = npcm7xx_get_group_pins,
+ .pin_dbg_show = npcm7xx_pin_dbg_show,
+ .dt_node_to_map = npcm7xx_dt_node_to_map,
+ .dt_free_map = npcm7xx_dt_free_map,
+};
+
+/* pinmux_ops */
+static int npcm7xx_get_functions_count(struct pinctrl_dev *pctldev)
+{
+ return ARRAY_SIZE(npcm7xx_funcs);
+}
+
+static const char *npcm7xx_get_function_name(struct pinctrl_dev *pctldev,
+ unsigned int function)
+{
+ return npcm7xx_funcs[function].name;
+}
+
+static int npcm7xx_get_function_groups(struct pinctrl_dev *pctldev,
+ unsigned int function,
+ const char * const **groups,
+ unsigned int * const ngroups)
+{
+ *ngroups = npcm7xx_funcs[function].ngroups;
+ *groups = npcm7xx_funcs[function].groups;
+
+ return 0;
+}
+
+static int npcm7xx_pinmux_set_mux(struct pinctrl_dev *pctldev,
+ unsigned int function,
+ unsigned int group)
+{
+ struct npcm7xx_pinctrl *npcm = pinctrl_dev_get_drvdata(pctldev);
+
+ dev_dbg(npcm->dev, "set_mux: %d, %d[%s]\n", function, group,
+ npcm7xx_groups[group].name);
+
+ npcm7xx_setfunc(npcm->gcr_regmap, npcm7xx_groups[group].pins,
+ npcm7xx_groups[group].npins, group);
+
+ return 0;
+}
+
+static int npcm7xx_gpio_request_enable(struct pinctrl_dev *pctldev,
+ struct pinctrl_gpio_range *range,
+ unsigned int offset)
+{
+ struct npcm7xx_pinctrl *npcm = pinctrl_dev_get_drvdata(pctldev);
+
+ if (!range) {
+ dev_err(npcm->dev, "invalid range\n");
+ return -EINVAL;
+ }
+ if (!range->gc) {
+ dev_err(npcm->dev, "invalid gpiochip\n");
+ return -EINVAL;
+ }
+
+ npcm7xx_setfunc(npcm->gcr_regmap, &offset, 1, fn_gpio);
+
+ return 0;
+}
+
+/* Release GPIO back to pinctrl mode */
+static void npcm7xx_gpio_request_free(struct pinctrl_dev *pctldev,
+ struct pinctrl_gpio_range *range,
+ unsigned int offset)
+{
+ struct npcm7xx_pinctrl *npcm = pinctrl_dev_get_drvdata(pctldev);
+ int virq;
+
+ virq = irq_find_mapping(npcm->domain, offset);
+ if (virq)
+ irq_dispose_mapping(virq);
+}
+
+/* Set GPIO direction */
+static int npcm_gpio_set_direction(struct pinctrl_dev *pctldev,
+ struct pinctrl_gpio_range *range,
+ unsigned int offset, bool input)
+{
+ struct npcm7xx_pinctrl *npcm = pinctrl_dev_get_drvdata(pctldev);
+ struct npcm7xx_gpio *bank =
+ &npcm->gpio_bank[offset / NPCM7XX_GPIO_PER_BANK];
+ int gpio = BIT(offset % bank->gc.ngpio);
+
+ dev_dbg(bank->gc.parent, "GPIO Set Direction: %d = %d\n", offset,
+ input);
+ if (input)
+ iowrite32(gpio, bank->base + NPCM7XX_GP_N_OEC);
+ else
+ iowrite32(gpio, bank->base + NPCM7XX_GP_N_OES);
+
+ return 0;
+}
+
+static struct pinmux_ops npcm7xx_pinmux_ops = {
+ .get_functions_count = npcm7xx_get_functions_count,
+ .get_function_name = npcm7xx_get_function_name,
+ .get_function_groups = npcm7xx_get_function_groups,
+ .set_mux = npcm7xx_pinmux_set_mux,
+ .gpio_request_enable = npcm7xx_gpio_request_enable,
+ .gpio_disable_free = npcm7xx_gpio_request_free,
+ .gpio_set_direction = npcm_gpio_set_direction,
+};
+
+/* pinconf_ops */
+static int npcm7xx_config_get(struct pinctrl_dev *pctldev, unsigned int pin,
+ unsigned long *config)
+{
+ enum pin_config_param param = pinconf_to_config_param(*config);
+ struct npcm7xx_pinctrl *npcm = pinctrl_dev_get_drvdata(pctldev);
+ struct npcm7xx_gpio *bank =
+ &npcm->gpio_bank[pin / NPCM7XX_GPIO_PER_BANK];
+ int gpio = (pin % bank->gc.ngpio);
+ unsigned long pinmask = BIT(gpio);
+ u32 ie, oe, pu, pd;
+ int rc = 0;
+
+ switch (param) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ case PIN_CONFIG_BIAS_PULL_UP:
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ pu = ioread32(bank->base + NPCM7XX_GP_N_PU) & pinmask;
+ pd = ioread32(bank->base + NPCM7XX_GP_N_PD) & pinmask;
+ if (param == PIN_CONFIG_BIAS_DISABLE)
+ rc = (!pu && !pd);
+ else if (param == PIN_CONFIG_BIAS_PULL_UP)
+ rc = (pu && !pd);
+ else if (param == PIN_CONFIG_BIAS_PULL_DOWN)
+ rc = (!pu && pd);
+ break;
+ case PIN_CONFIG_OUTPUT:
+ case PIN_CONFIG_INPUT_ENABLE:
+ ie = ioread32(bank->base + NPCM7XX_GP_N_IEM) & pinmask;
+ oe = ioread32(bank->base + NPCM7XX_GP_N_OE) & pinmask;
+ if (param == PIN_CONFIG_INPUT_ENABLE)
+ rc = (ie && !oe);
+ else if (param == PIN_CONFIG_OUTPUT)
+ rc = (!ie && oe);
+ break;
+ case PIN_CONFIG_DRIVE_PUSH_PULL:
+ rc = !(ioread32(bank->base + NPCM7XX_GP_N_OTYP) & pinmask);
+ break;
+ case PIN_CONFIG_DRIVE_OPEN_DRAIN:
+ rc = ioread32(bank->base + NPCM7XX_GP_N_OTYP) & pinmask;
+ break;
+ case PIN_CONFIG_INPUT_DEBOUNCE:
+ rc = ioread32(bank->base + NPCM7XX_GP_N_DBNC) & pinmask;
+ break;
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ rc = npcm7xx_get_drive_strength(pctldev, pin);
+ if (rc)
+ *config = pinconf_to_config_packed(param, rc);
+ break;
+ case PIN_CONFIG_SLEW_RATE:
+ rc = npcm7xx_get_slew_rate(bank, npcm->gcr_regmap, pin);
+ if (rc >= 0)
+ *config = pinconf_to_config_packed(param, rc);
+ break;
+ default:
+ return -ENOTSUPP;
+ }
+
+ if (!rc)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int npcm7xx_config_set_one(struct npcm7xx_pinctrl *npcm,
+ unsigned int pin, unsigned long config)
+{
+ enum pin_config_param param = pinconf_to_config_param(config);
+ u16 arg = pinconf_to_config_argument(config);
+ struct npcm7xx_gpio *bank =
+ &npcm->gpio_bank[pin / NPCM7XX_GPIO_PER_BANK];
+ int gpio = BIT(pin % bank->gc.ngpio);
+
+ dev_dbg(bank->gc.parent, "param=%d %d[GPIO]\n", param, pin);
+ switch (param) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_PU, gpio);
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_PD, gpio);
+ break;
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_PU, gpio);
+ npcm_gpio_set(&bank->gc, bank->base + NPCM7XX_GP_N_PD, gpio);
+ break;
+ case PIN_CONFIG_BIAS_PULL_UP:
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_PD, gpio);
+ npcm_gpio_set(&bank->gc, bank->base + NPCM7XX_GP_N_PU, gpio);
+ break;
+ case PIN_CONFIG_INPUT_ENABLE:
+ if (arg) {
+ iowrite32(gpio, bank->base + NPCM7XX_GP_N_OEC);
+ npcm_gpio_set(&bank->gc, bank->base + NPCM7XX_GP_N_IEM,
+ gpio);
+ } else
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_IEM,
+ gpio);
+ break;
+ case PIN_CONFIG_OUTPUT:
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_IEM, gpio);
+ iowrite32(gpio, arg ? bank->base + NPCM7XX_GP_N_DOS :
+ bank->base + NPCM7XX_GP_N_DOC);
+ iowrite32(gpio, bank->base + NPCM7XX_GP_N_OES);
+ break;
+ case PIN_CONFIG_DRIVE_PUSH_PULL:
+ npcm_gpio_clr(&bank->gc, bank->base + NPCM7XX_GP_N_OTYP, gpio);
+ break;
+ case PIN_CONFIG_DRIVE_OPEN_DRAIN:
+ npcm_gpio_set(&bank->gc, bank->base + NPCM7XX_GP_N_OTYP, gpio);
+ break;
+ case PIN_CONFIG_INPUT_DEBOUNCE:
+ npcm_gpio_set(&bank->gc, bank->base + NPCM7XX_GP_N_DBNC, gpio);
+ break;
+ case PIN_CONFIG_SLEW_RATE:
+ return npcm7xx_set_slew_rate(bank, npcm->gcr_regmap, pin, arg);
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ return npcm7xx_set_drive_strength(npcm, pin, arg);
+ default:
+ return -ENOTSUPP;
+ }
+
+ return 0;
+}
+
+/* Set multiple configuration settings for a pin */
+static int npcm7xx_config_set(struct pinctrl_dev *pctldev, unsigned int pin,
+ unsigned long *configs, unsigned int num_configs)
+{
+ struct npcm7xx_pinctrl *npcm = pinctrl_dev_get_drvdata(pctldev);
+ int rc;
+
+ while (num_configs--) {
+ rc = npcm7xx_config_set_one(npcm, pin, *configs++);
+ if (rc)
+ return rc;
+ }
+
+ return 0;
+}
+
+static struct pinconf_ops npcm7xx_pinconf_ops = {
+ .is_generic = true,
+ .pin_config_get = npcm7xx_config_get,
+ .pin_config_set = npcm7xx_config_set,
+};
+
+/* pinctrl_desc */
+static struct pinctrl_desc npcm7xx_pinctrl_desc = {
+ .name = "npcm7xx-pinctrl",
+ .pins = npcm7xx_pins,
+ .npins = ARRAY_SIZE(npcm7xx_pins),
+ .pctlops = &npcm7xx_pinctrl_ops,
+ .pmxops = &npcm7xx_pinmux_ops,
+ .confops = &npcm7xx_pinconf_ops,
+ .owner = THIS_MODULE,
+};
+
+static int npcm7xx_gpio_of(struct npcm7xx_pinctrl *pctrl)
+{
+ int ret = -ENXIO;
+ struct resource res;
+ int id = 0, irq;
+ struct device_node *np;
+ struct of_phandle_args pinspec;
+
+ for_each_available_child_of_node(pctrl->dev->of_node, np)
+ if (of_find_property(np, "gpio-controller", NULL)) {
+ ret = of_address_to_resource(np, 0, &res);
+ if (ret < 0) {
+ dev_err(pctrl->dev,
+ "Resource fail for GPIO bank %u\n", id);
+ return ret;
+ }
+
+ pctrl->gpio_bank[id].base =
+ ioremap(res.start, resource_size(&res));
+
+ irq = irq_of_parse_and_map(np, 0);
+ if (irq < 0) {
+ dev_err(pctrl->dev,
+ "No IRQ for GPIO bank %u\n", id);
+ ret = irq;
+ return ret;
+ }
+
+ ret = bgpio_init(&pctrl->gpio_bank[id].gc,
+ pctrl->dev, 4,
+ pctrl->gpio_bank[id].base +
+ NPCM7XX_GP_N_DIN,
+ pctrl->gpio_bank[id].base +
+ NPCM7XX_GP_N_DOUT,
+ NULL,
+ NULL,
+ pctrl->gpio_bank[id].base +
+ NPCM7XX_GP_N_IEM,
+ BGPIOF_READ_OUTPUT_REG_SET);
+ if (ret) {
+ dev_err(pctrl->dev, "bgpio_init() failed\n");
+ return ret;
+ }
+
+ ret = of_parse_phandle_with_fixed_args(np,
+ "gpio-ranges", 3,
+ 0, &pinspec);
+ if (ret < 0) {
+ dev_err(pctrl->dev,
+ "gpio-ranges fail for GPIO bank %u\n",
+ id);
+ return ret;
+ }
+
+ pctrl->gpio_bank[id].irq = irq;
+ pctrl->gpio_bank[id].irq_chip = npcmgpio_irqchip;
+ pctrl->gpio_bank[id].gc.parent = pctrl->dev;
+ pctrl->gpio_bank[id].irqbase =
+ id * NPCM7XX_GPIO_PER_BANK;
+ pctrl->gpio_bank[id].pinctrl_id = pinspec.args[0];
+ pctrl->gpio_bank[id].gc.base = pinspec.args[1];
+ pctrl->gpio_bank[id].gc.ngpio = pinspec.args[2];
+ pctrl->gpio_bank[id].gc.owner = THIS_MODULE;
+ pctrl->gpio_bank[id].gc.label =
+ devm_kasprintf(pctrl->dev, GFP_KERNEL, "%pOF",
+ np);
+ pctrl->gpio_bank[id].gc.dbg_show = npcmgpio_dbg_show;
+ pctrl->gpio_bank[id].direction_input =
+ pctrl->gpio_bank[id].gc.direction_input;
+ pctrl->gpio_bank[id].gc.direction_input =
+ npcmgpio_direction_input;
+ pctrl->gpio_bank[id].direction_output =
+ pctrl->gpio_bank[id].gc.direction_output;
+ pctrl->gpio_bank[id].gc.direction_output =
+ npcmgpio_direction_output;
+ pctrl->gpio_bank[id].request =
+ pctrl->gpio_bank[id].gc.request;
+ pctrl->gpio_bank[id].gc.request = npcmgpio_gpio_request;
+ pctrl->gpio_bank[id].gc.free = npcmgpio_gpio_free;
+ pctrl->gpio_bank[id].gc.of_node = np;
+ id++;
+ }
+
+ pctrl->bank_num = id;
+ return ret;
+}
+
+static int npcm7xx_gpio_register(struct npcm7xx_pinctrl *pctrl)
+{
+ int ret, id;
+
+ for (id = 0 ; id < pctrl->bank_num ; id++) {
+ ret = devm_gpiochip_add_data(pctrl->dev,
+ &pctrl->gpio_bank[id].gc,
+ &pctrl->gpio_bank[id]);
+ if (ret) {
+ dev_err(pctrl->dev, "Failed to add GPIO chip %u\n", id);
+ goto err_register;
+ }
+
+ ret = gpiochip_add_pin_range(&pctrl->gpio_bank[id].gc,
+ dev_name(pctrl->dev),
+ pctrl->gpio_bank[id].pinctrl_id,
+ pctrl->gpio_bank[id].gc.base,
+ pctrl->gpio_bank[id].gc.ngpio);
+ if (ret < 0) {
+ dev_err(pctrl->dev, "Failed to add GPIO bank %u\n", id);
+ gpiochip_remove(&pctrl->gpio_bank[id].gc);
+ goto err_register;
+ }
+
+ ret = gpiochip_irqchip_add(&pctrl->gpio_bank[id].gc,
+ &pctrl->gpio_bank[id].irq_chip,
+ 0, handle_level_irq,
+ IRQ_TYPE_NONE);
+ if (ret < 0) {
+ dev_err(pctrl->dev,
+ "Failed to add IRQ chip %u\n", id);
+ gpiochip_remove(&pctrl->gpio_bank[id].gc);
+ goto err_register;
+ }
+
+ gpiochip_set_chained_irqchip(&pctrl->gpio_bank[id].gc,
+ &pctrl->gpio_bank[id].irq_chip,
+ pctrl->gpio_bank[id].irq,
+ npcmgpio_irq_handler);
+ }
+
+ return 0;
+
+err_register:
+ for (; id > 0; id--)
+ gpiochip_remove(&pctrl->gpio_bank[id - 1].gc);
+
+ return ret;
+}
+
+static int npcm7xx_pinctrl_probe(struct platform_device *pdev)
+{
+ struct npcm7xx_pinctrl *pctrl;
+ int ret;
+
+ pctrl = devm_kzalloc(&pdev->dev, sizeof(*pctrl), GFP_KERNEL);
+ if (!pctrl)
+ return -ENOMEM;
+
+ pctrl->dev = &pdev->dev;
+ dev_set_drvdata(&pdev->dev, pctrl);
+
+ pctrl->gcr_regmap =
+ syscon_regmap_lookup_by_compatible("nuvoton,npcm750-gcr");
+ if (IS_ERR(pctrl->gcr_regmap)) {
+ dev_err(pctrl->dev, "didn't find nuvoton,npcm750-gcr\n");
+ return PTR_ERR(pctrl->gcr_regmap);
+ }
+
+ ret = npcm7xx_gpio_of(pctrl);
+ if (ret < 0) {
+ dev_err(pctrl->dev, "Failed to gpio dt-binding %u\n", ret);
+ return ret;
+ }
+
+ pctrl->pctldev = devm_pinctrl_register(&pdev->dev,
+ &npcm7xx_pinctrl_desc, pctrl);
+ if (IS_ERR(pctrl->pctldev)) {
+ dev_err(&pdev->dev, "Failed to register pinctrl device\n");
+ return PTR_ERR(pctrl->pctldev);
+ }
+
+ ret = npcm7xx_gpio_register(pctrl);
+ if (ret < 0) {
+ dev_err(pctrl->dev, "Failed to register gpio %u\n", ret);
+ return ret;
+ }
+
+ pr_info("NPCM7xx Pinctrl driver probed\n");
+ return 0;
+}
+
+static const struct of_device_id npcm7xx_pinctrl_match[] = {
+ { .compatible = "nuvoton,npcm750-pinctrl" },
+ { },
+};
+MODULE_DEVICE_TABLE(of, npcm7xx_pinctrl_match);
+
+static struct platform_driver npcm7xx_pinctrl_driver = {
+ .probe = npcm7xx_pinctrl_probe,
+ .driver = {
+ .name = "npcm7xx-pinctrl",
+ .of_match_table = npcm7xx_pinctrl_match,
+ .suppress_bind_attrs = true,
+ },
+};
+
+static int __init npcm7xx_pinctrl_register(void)
+{
+ return platform_driver_register(&npcm7xx_pinctrl_driver);
+}
+arch_initcall(npcm7xx_pinctrl_register);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("jordan_hargrave@dell.com");
+MODULE_AUTHOR("tomer.maimon@nuvoton.com");
+MODULE_DESCRIPTION("Nuvoton NPCM7XX Pinctrl and GPIO driver");
#include <linux/errno.h>
#include <linux/log2.h>
#include <linux/io.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/mutex.h>
*/
#include <linux/delay.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/mfd/as3722.h>
#include <dt-bindings/pinctrl/at91.h>
#include <linux/clk.h>
#include <linux/gpio/driver.h>
-/* FIXME: needed for gpio_to_irq(), get rid of this */
-#include <linux/gpio.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/init.h>
.irq_set_wake = atmel_gpio_irq_set_wake,
};
+static int atmel_gpio_to_irq(struct gpio_chip *chip, unsigned offset)
+{
+ struct atmel_pioctrl *atmel_pioctrl = gpiochip_get_data(chip);
+
+ return irq_find_mapping(atmel_pioctrl->irq_domain, offset);
+}
+
static void atmel_gpio_irq_handler(struct irq_desc *desc)
{
unsigned int irq = irq_desc_get_irq(desc);
break;
for_each_set_bit(n, &isr, BITS_PER_LONG)
- generic_handle_irq(gpio_to_irq(bank *
- ATMEL_PIO_NPINS_PER_BANK + n));
+ generic_handle_irq(atmel_gpio_to_irq(
+ atmel_pioctrl->gpio_chip,
+ bank * ATMEL_PIO_NPINS_PER_BANK + n));
}
chained_irq_exit(chip, desc);
BIT(pin->line));
}
-static int atmel_gpio_to_irq(struct gpio_chip *chip, unsigned offset)
-{
- struct atmel_pioctrl *atmel_pioctrl = gpiochip_get_data(chip);
-
- return irq_find_mapping(atmel_pioctrl->irq_domain, offset);
-}
-
static struct gpio_chip atmel_gpio_chip = {
.direction_input = atmel_gpio_direction_input,
.get = atmel_gpio_get,
unsigned num_pins, num_configs, reserve;
unsigned long *configs;
struct property *pins;
- bool has_config;
u32 pinfunc;
int ret, i;
return ret;
}
- if (num_configs)
- has_config = true;
-
num_pins = pins->length / sizeof(u32);
if (!num_pins) {
dev_err(pctldev->dev, "no pins found in node %pOF\n", np);
* map for each pin.
*/
reserve = 1;
- if (has_config && num_pins >= 1)
+ if (num_configs)
reserve++;
reserve *= num_pins;
ret = pinctrl_utils_reserve_map(pctldev, map, reserved_maps, num_maps,
pinctrl_utils_add_map_mux(pctldev, map, reserved_maps, num_maps,
group, func);
- if (has_config) {
+ if (num_configs) {
ret = pinctrl_utils_add_map_configs(pctldev, map,
reserved_maps, num_maps, group,
configs, num_configs,
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/io.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/pinctrl/machine.h>
#include <linux/pinctrl/pinconf.h>
#include <linux/pinctrl/pinctrl.h>
*/
grp = at91_pinctrl_find_group_by_name(info, np->name);
if (!grp) {
- dev_err(info->dev, "unable to find group for node %s\n",
- np->name);
+ dev_err(info->dev, "unable to find group for node %pOFn\n",
+ np);
return -EINVAL;
}
const __be32 *list;
int i, j;
- dev_dbg(info->dev, "group(%d): %s\n", index, np->name);
+ dev_dbg(info->dev, "group(%d): %pOFn\n", index, np);
/* Initialise group */
grp->name = np->name;
static u32 grp_index;
u32 i = 0;
- dev_dbg(info->dev, "parse function(%d): %s\n", index, np->name);
+ dev_dbg(info->dev, "parse function(%d): %pOFn\n", index, np);
func = &info->functions[index];
return 0;
case IRQ_TYPE_NONE:
default:
- pr_warn("AT91: No type for irq %d\n", gpio_to_irq(d->irq));
+ pr_warn("AT91: No type for GPIO irq offset %d\n", d->irq);
return -EINVAL;
}
#define gpio_irq_set_wake NULL
#endif /* CONFIG_PM */
-static struct irq_chip gpio_irqchip = {
- .name = "GPIO",
- .irq_ack = gpio_irq_ack,
- .irq_disable = gpio_irq_mask,
- .irq_mask = gpio_irq_mask,
- .irq_unmask = gpio_irq_unmask,
- /* .irq_set_type is set dynamically */
- .irq_set_wake = gpio_irq_set_wake,
-};
-
static void gpio_irq_handler(struct irq_desc *desc)
{
struct irq_chip *chip = irq_desc_get_chip(desc);
struct gpio_chip *gpiochip_prev = NULL;
struct at91_gpio_chip *prev = NULL;
struct irq_data *d = irq_get_irq_data(at91_gpio->pioc_virq);
+ struct irq_chip *gpio_irqchip;
int ret, i;
+ gpio_irqchip = devm_kzalloc(&pdev->dev, sizeof(*gpio_irqchip), GFP_KERNEL);
+ if (!gpio_irqchip)
+ return -ENOMEM;
+
at91_gpio->pioc_hwirq = irqd_to_hwirq(d);
- /* Setup proper .irq_set_type function */
- gpio_irqchip.irq_set_type = at91_gpio->ops->irq_type;
+ gpio_irqchip->name = "GPIO";
+ gpio_irqchip->irq_ack = gpio_irq_ack;
+ gpio_irqchip->irq_disable = gpio_irq_mask;
+ gpio_irqchip->irq_mask = gpio_irq_mask;
+ gpio_irqchip->irq_unmask = gpio_irq_unmask;
+ gpio_irqchip->irq_set_wake = gpio_irq_set_wake,
+ gpio_irqchip->irq_set_type = at91_gpio->ops->irq_type;
/* Disable irqs of this PIO controller */
writel_relaxed(~0, at91_gpio->regbase + PIO_IDR);
* interrupt.
*/
ret = gpiochip_irqchip_add(&at91_gpio->chip,
- &gpio_irqchip,
+ gpio_irqchip,
0,
handle_edge_irq,
IRQ_TYPE_NONE);
if (!gpiochip_prev) {
/* Then register the chain on the parent IRQ */
gpiochip_set_chained_irqchip(&at91_gpio->chip,
- &gpio_irqchip,
+ gpio_irqchip,
at91_gpio->pioc_virq,
gpio_irq_handler);
return 0;
#include <linux/clk.h>
#include <linux/err.h>
#include <linux/platform_device.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/slab.h>
#include <linux/pinctrl/consumer.h>
#include <linux/pinctrl/pinconf-generic.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/io.h>
-#include <linux/gpio.h>
#include <linux/gpio/driver.h>
#include <linux/spinlock.h>
#include <linux/pinctrl/machine.h>
* Copyright (C) 2012 John Crispin <john@phrozen.org>
*/
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/interrupt.h>
#include <linux/slab.h>
#include <linux/export.h>
319, /* TVC_DATA[1] */
301, /* TVC_DATA[2] */
283, /* TVC_DATA[3] */
- 265, /* TVC_CLK */
320, /* TVC_DATA[4] */
302, /* TVC_DATA[5] */
284, /* TVC_DATA[6] */
266, /* TVC_DATA[7] */
};
+static const unsigned int tvc_clk_3512_pins[] = {
+ 265, /* TVC_CLK */
+};
+
/* NAND flash pins */
static const unsigned int nflash_3512_pins[] = {
199, 200, 201, 202, 216, 217, 218, 219, 220, 234, 235, 236, 237, 252,
/* Serial flash pins CE0, CE1, DI, DO, CK */
static const unsigned int sflash_3512_pins[] = { 230, 231, 232, 233, 211 };
-/* The GPIO0A (0) pin overlap with TVC and extended parallel flash */
+/* The GPIO0A (0) pin overlap with TVC CLK and extended parallel flash */
static const unsigned int gpio0a_3512_pins[] = { 265 };
/* The GPIO0B (1-4) pins overlap with TVC and ICE */
.num_pins = ARRAY_SIZE(tvc_3512_pins),
/* Conflict with character LCD and ICE */
.mask = LCD_PADS_ENABLE,
- .value = TVC_PADS_ENABLE | TVC_CLK_PAD_ENABLE,
+ .value = TVC_PADS_ENABLE,
+ },
+ {
+ .name = "tvcclkgrp",
+ .pins = tvc_clk_3512_pins,
+ .num_pins = ARRAY_SIZE(tvc_clk_3512_pins),
+ .value = TVC_CLK_PAD_ENABLE,
},
/*
* The construction is done such that it is possible to use a serial
.name = "gpio0agrp",
.pins = gpio0a_3512_pins,
.num_pins = ARRAY_SIZE(gpio0a_3512_pins),
- /* Conflict with TVC */
- .mask = TVC_PADS_ENABLE,
+ /* Conflict with TVC CLK */
+ .mask = TVC_CLK_PAD_ENABLE,
},
{
.name = "gpio0bgrp",
311, /* TVC_DATA[1] */
394, /* TVC_DATA[2] */
374, /* TVC_DATA[3] */
- 333, /* TVC_CLK */
354, /* TVC_DATA[4] */
395, /* TVC_DATA[5] */
312, /* TVC_DATA[6] */
334, /* TVC_DATA[7] */
};
+static const unsigned int tvc_clk_3516_pins[] = {
+ 333, /* TVC_CLK */
+};
+
/* NAND flash pins */
static const unsigned int nflash_3516_pins[] = {
243, 260, 261, 224, 280, 262, 281, 264, 300, 263, 282, 301, 320, 283,
static const unsigned int sflash_3516_pins[] = { 296, 338, 295, 359, 339 };
/* The GPIO0A (0-4) pins overlap with TVC and extended parallel flash */
-static const unsigned int gpio0a_3516_pins[] = { 333, 354, 395, 312, 334 };
+static const unsigned int gpio0a_3516_pins[] = { 354, 395, 312, 334 };
/* The GPIO0B (5-7) pins overlap with ICE */
static const unsigned int gpio0b_3516_pins[] = { 375, 396, 376 };
/* The GPIO0K (30,31) pins overlap with NAND flash */
static const unsigned int gpio0k_3516_pins[] = { 275, 298 };
+/* The GPIO0L (0) pins overlap with TVC_CLK */
+static const unsigned int gpio0l_3516_pins[] = { 333 };
+
/* The GPIO1A (0-4) pins that overlap with IDE and parallel flash */
static const unsigned int gpio1a_3516_pins[] = { 221, 200, 222, 201, 220 };
.num_pins = ARRAY_SIZE(tvc_3516_pins),
/* Conflict with character LCD */
.mask = LCD_PADS_ENABLE,
- .value = TVC_PADS_ENABLE | TVC_CLK_PAD_ENABLE,
+ .value = TVC_PADS_ENABLE,
+ },
+ {
+ .name = "tvcclkgrp",
+ .pins = tvc_clk_3516_pins,
+ .num_pins = ARRAY_SIZE(tvc_clk_3516_pins),
+ .value = TVC_CLK_PAD_ENABLE,
},
/*
* The construction is done such that it is possible to use a serial
/* Conflict with parallel and NAND flash */
.value = PFLASH_PADS_DISABLE | NAND_PADS_DISABLE,
},
+ {
+ .name = "gpio0lgrp",
+ .pins = gpio0l_3516_pins,
+ .num_pins = ARRAY_SIZE(gpio0l_3516_pins),
+ /* Conflict with TVE CLK */
+ .mask = TVC_CLK_PAD_ENABLE,
+ },
{
.name = "gpio1agrp",
.pins = gpio1a_3516_pins,
return -ENODEV;
}
- dev_info(pmx->dev,
- "ACTIVATE function \"%s\" with group \"%s\"\n",
- func->name, grp->name);
+ dev_dbg(pmx->dev,
+ "ACTIVATE function \"%s\" with group \"%s\"\n",
+ func->name, grp->name);
regmap_read(pmx->map, GLOBAL_MISC_CTRL, &before);
- regmap_update_bits(pmx->map, GLOBAL_MISC_CTRL, grp->mask,
+ regmap_update_bits(pmx->map, GLOBAL_MISC_CTRL,
+ grp->mask | grp->value,
grp->value);
regmap_read(pmx->map, GLOBAL_MISC_CTRL, &after);
"GLOBAL MISC CTRL before: %08x, after %08x, expected %08x\n",
before, after, expected);
} else {
- dev_info(pmx->dev,
- "padgroup %s %s\n",
- gemini_padgroups[i],
- enabled ? "enabled" : "disabled");
+ dev_dbg(pmx->dev,
+ "padgroup %s %s\n",
+ gemini_padgroups[i],
+ enabled ? "enabled" : "disabled");
}
}
"GLOBAL MISC CTRL before: %08x, after %08x, expected %08x\n",
before, after, expected);
} else {
- dev_info(pmx->dev,
- "padgroup %s %s\n",
- gemini_padgroups[i],
- enabled ? "enabled" : "disabled");
+ dev_dbg(pmx->dev,
+ "padgroup %s %s\n",
+ gemini_padgroups[i],
+ enabled ? "enabled" : "disabled");
}
}
regmap_update_bits(pmx->map, GLOBAL_IODRIVE,
grp->driving_mask,
val);
- dev_info(pmx->dev,
- "set group %s to %d mA drive strength mask %08x val %08x\n",
- grp->name, arg, grp->driving_mask, val);
+ dev_dbg(pmx->dev,
+ "set group %s to %d mA drive strength mask %08x val %08x\n",
+ grp->name, arg, grp->driving_mask, val);
break;
default:
dev_err(pmx->dev, "invalid config param %04x\n", param);
/* Print initial state */
tmp = val;
for_each_set_bit(i, &tmp, PADS_MAXBIT) {
- dev_info(dev, "pad group %s %s\n", gemini_padgroups[i],
- (val & BIT(i)) ? "enabled" : "disabled");
+ dev_dbg(dev, "pad group %s %s\n", gemini_padgroups[i],
+ (val & BIT(i)) ? "enabled" : "disabled");
}
/* Check if flash pin is set */
*/
#include <linux/compiler.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/of_device.h>
+#include <linux/of_irq.h>
#include <linux/of_platform.h>
#include <linux/pinctrl/pinctrl.h>
#include <linux/pinctrl/pinmux.h>
#include "pinconf.h"
#include "pinmux.h"
+#define GPIO_PIN 0x00
+#define GPIO_MSK 0x20
+
#define JZ4740_GPIO_DATA 0x10
#define JZ4740_GPIO_PULL_DIS 0x30
#define JZ4740_GPIO_FUNC 0x40
#define JZ4740_GPIO_FLAG 0x80
#define JZ4770_GPIO_INT 0x10
-#define JZ4770_GPIO_MSK 0x20
#define JZ4770_GPIO_PAT1 0x30
#define JZ4770_GPIO_PAT0 0x40
#define JZ4770_GPIO_FLAG 0x50
enum jz_version {
ID_JZ4740,
+ ID_JZ4725B,
ID_JZ4770,
ID_JZ4780,
};
const struct ingenic_chip_info *info;
};
+struct ingenic_gpio_chip {
+ struct ingenic_pinctrl *jzpc;
+ struct gpio_chip gc;
+ struct irq_chip irq_chip;
+ unsigned int irq, reg_base;
+};
+
static const u32 jz4740_pull_ups[4] = {
0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff,
};
.pull_downs = jz4740_pull_downs,
};
+static int jz4725b_mmc0_1bit_pins[] = { 0x48, 0x49, 0x5c, };
+static int jz4725b_mmc0_4bit_pins[] = { 0x5d, 0x5b, 0x56, };
+static int jz4725b_mmc1_1bit_pins[] = { 0x7a, 0x7b, 0x7c, };
+static int jz4725b_mmc1_4bit_pins[] = { 0x7d, 0x7e, 0x7f, };
+static int jz4725b_uart_data_pins[] = { 0x4c, 0x4d, };
+static int jz4725b_nand_cs1_pins[] = { 0x55, };
+static int jz4725b_nand_cs2_pins[] = { 0x56, };
+static int jz4725b_nand_cs3_pins[] = { 0x57, };
+static int jz4725b_nand_cs4_pins[] = { 0x58, };
+static int jz4725b_nand_cle_ale_pins[] = { 0x48, 0x49 };
+static int jz4725b_nand_fre_fwe_pins[] = { 0x5c, 0x5d };
+static int jz4725b_pwm_pwm0_pins[] = { 0x4a, };
+static int jz4725b_pwm_pwm1_pins[] = { 0x4b, };
+static int jz4725b_pwm_pwm2_pins[] = { 0x4c, };
+static int jz4725b_pwm_pwm3_pins[] = { 0x4d, };
+static int jz4725b_pwm_pwm4_pins[] = { 0x4e, };
+static int jz4725b_pwm_pwm5_pins[] = { 0x4f, };
+
+static int jz4725b_mmc0_1bit_funcs[] = { 1, 1, 1, };
+static int jz4725b_mmc0_4bit_funcs[] = { 1, 0, 1, };
+static int jz4725b_mmc1_1bit_funcs[] = { 0, 0, 0, };
+static int jz4725b_mmc1_4bit_funcs[] = { 0, 0, 0, };
+static int jz4725b_uart_data_funcs[] = { 1, 1, };
+static int jz4725b_nand_cs1_funcs[] = { 0, };
+static int jz4725b_nand_cs2_funcs[] = { 0, };
+static int jz4725b_nand_cs3_funcs[] = { 0, };
+static int jz4725b_nand_cs4_funcs[] = { 0, };
+static int jz4725b_nand_cle_ale_funcs[] = { 0, 0, };
+static int jz4725b_nand_fre_fwe_funcs[] = { 0, 0, };
+static int jz4725b_pwm_pwm0_funcs[] = { 0, };
+static int jz4725b_pwm_pwm1_funcs[] = { 0, };
+static int jz4725b_pwm_pwm2_funcs[] = { 0, };
+static int jz4725b_pwm_pwm3_funcs[] = { 0, };
+static int jz4725b_pwm_pwm4_funcs[] = { 0, };
+static int jz4725b_pwm_pwm5_funcs[] = { 0, };
+
+static const struct group_desc jz4725b_groups[] = {
+ INGENIC_PIN_GROUP("mmc0-1bit", jz4725b_mmc0_1bit),
+ INGENIC_PIN_GROUP("mmc0-4bit", jz4725b_mmc0_4bit),
+ INGENIC_PIN_GROUP("mmc1-1bit", jz4725b_mmc1_1bit),
+ INGENIC_PIN_GROUP("mmc1-4bit", jz4725b_mmc1_4bit),
+ INGENIC_PIN_GROUP("uart-data", jz4725b_uart_data),
+ INGENIC_PIN_GROUP("nand-cs1", jz4725b_nand_cs1),
+ INGENIC_PIN_GROUP("nand-cs2", jz4725b_nand_cs2),
+ INGENIC_PIN_GROUP("nand-cs3", jz4725b_nand_cs3),
+ INGENIC_PIN_GROUP("nand-cs4", jz4725b_nand_cs4),
+ INGENIC_PIN_GROUP("nand-cle-ale", jz4725b_nand_cle_ale),
+ INGENIC_PIN_GROUP("nand-fre-fwe", jz4725b_nand_fre_fwe),
+ INGENIC_PIN_GROUP("pwm0", jz4725b_pwm_pwm0),
+ INGENIC_PIN_GROUP("pwm1", jz4725b_pwm_pwm1),
+ INGENIC_PIN_GROUP("pwm2", jz4725b_pwm_pwm2),
+ INGENIC_PIN_GROUP("pwm3", jz4725b_pwm_pwm3),
+ INGENIC_PIN_GROUP("pwm4", jz4725b_pwm_pwm4),
+ INGENIC_PIN_GROUP("pwm5", jz4725b_pwm_pwm5),
+};
+
+static const char *jz4725b_mmc0_groups[] = { "mmc0-1bit", "mmc0-4bit", };
+static const char *jz4725b_mmc1_groups[] = { "mmc1-1bit", "mmc1-4bit", };
+static const char *jz4725b_uart_groups[] = { "uart-data", };
+static const char *jz4725b_nand_groups[] = {
+ "nand-cs1", "nand-cs2", "nand-cs3", "nand-cs4",
+ "nand-cle-ale", "nand-fre-fwe",
+};
+static const char *jz4725b_pwm0_groups[] = { "pwm0", };
+static const char *jz4725b_pwm1_groups[] = { "pwm1", };
+static const char *jz4725b_pwm2_groups[] = { "pwm2", };
+static const char *jz4725b_pwm3_groups[] = { "pwm3", };
+static const char *jz4725b_pwm4_groups[] = { "pwm4", };
+static const char *jz4725b_pwm5_groups[] = { "pwm5", };
+
+static const struct function_desc jz4725b_functions[] = {
+ { "mmc0", jz4725b_mmc0_groups, ARRAY_SIZE(jz4725b_mmc0_groups), },
+ { "mmc1", jz4725b_mmc1_groups, ARRAY_SIZE(jz4725b_mmc1_groups), },
+ { "uart", jz4725b_uart_groups, ARRAY_SIZE(jz4725b_uart_groups), },
+ { "nand", jz4725b_nand_groups, ARRAY_SIZE(jz4725b_nand_groups), },
+ { "pwm0", jz4725b_pwm0_groups, ARRAY_SIZE(jz4725b_pwm0_groups), },
+ { "pwm1", jz4725b_pwm1_groups, ARRAY_SIZE(jz4725b_pwm1_groups), },
+ { "pwm2", jz4725b_pwm2_groups, ARRAY_SIZE(jz4725b_pwm2_groups), },
+ { "pwm3", jz4725b_pwm3_groups, ARRAY_SIZE(jz4725b_pwm3_groups), },
+ { "pwm4", jz4725b_pwm4_groups, ARRAY_SIZE(jz4725b_pwm4_groups), },
+ { "pwm5", jz4725b_pwm5_groups, ARRAY_SIZE(jz4725b_pwm5_groups), },
+};
+
+static const struct ingenic_chip_info jz4725b_chip_info = {
+ .num_chips = 4,
+ .groups = jz4725b_groups,
+ .num_groups = ARRAY_SIZE(jz4725b_groups),
+ .functions = jz4725b_functions,
+ .num_functions = ARRAY_SIZE(jz4725b_functions),
+ .pull_ups = jz4740_pull_ups,
+ .pull_downs = jz4740_pull_downs,
+};
+
static const u32 jz4770_pull_ups[6] = {
0x3fffffff, 0xfff0030c, 0xffffffff, 0xffff4fff, 0xfffffb7c, 0xffa7f00f,
};
.pull_downs = jz4770_pull_downs,
};
+static u32 gpio_ingenic_read_reg(struct ingenic_gpio_chip *jzgc, u8 reg)
+{
+ unsigned int val;
+
+ regmap_read(jzgc->jzpc->map, jzgc->reg_base + reg, &val);
+
+ return (u32) val;
+}
+
+static void gpio_ingenic_set_bit(struct ingenic_gpio_chip *jzgc,
+ u8 reg, u8 offset, bool set)
+{
+ if (set)
+ reg = REG_SET(reg);
+ else
+ reg = REG_CLEAR(reg);
+
+ regmap_write(jzgc->jzpc->map, jzgc->reg_base + reg, BIT(offset));
+}
+
+static inline bool ingenic_gpio_get_value(struct ingenic_gpio_chip *jzgc,
+ u8 offset)
+{
+ unsigned int val = gpio_ingenic_read_reg(jzgc, GPIO_PIN);
+
+ return !!(val & BIT(offset));
+}
+
+static void ingenic_gpio_set_value(struct ingenic_gpio_chip *jzgc,
+ u8 offset, int value)
+{
+ if (jzgc->jzpc->version >= ID_JZ4770)
+ gpio_ingenic_set_bit(jzgc, JZ4770_GPIO_PAT0, offset, !!value);
+ else
+ gpio_ingenic_set_bit(jzgc, JZ4740_GPIO_DATA, offset, !!value);
+}
+
+static void irq_set_type(struct ingenic_gpio_chip *jzgc,
+ u8 offset, unsigned int type)
+{
+ u8 reg1, reg2;
+
+ if (jzgc->jzpc->version >= ID_JZ4770) {
+ reg1 = JZ4770_GPIO_PAT1;
+ reg2 = JZ4770_GPIO_PAT0;
+ } else {
+ reg1 = JZ4740_GPIO_TRIG;
+ reg2 = JZ4740_GPIO_DIR;
+ }
+
+ switch (type) {
+ case IRQ_TYPE_EDGE_RISING:
+ gpio_ingenic_set_bit(jzgc, reg2, offset, true);
+ gpio_ingenic_set_bit(jzgc, reg1, offset, true);
+ break;
+ case IRQ_TYPE_EDGE_FALLING:
+ gpio_ingenic_set_bit(jzgc, reg2, offset, false);
+ gpio_ingenic_set_bit(jzgc, reg1, offset, true);
+ break;
+ case IRQ_TYPE_LEVEL_HIGH:
+ gpio_ingenic_set_bit(jzgc, reg2, offset, true);
+ gpio_ingenic_set_bit(jzgc, reg1, offset, false);
+ break;
+ case IRQ_TYPE_LEVEL_LOW:
+ default:
+ gpio_ingenic_set_bit(jzgc, reg2, offset, false);
+ gpio_ingenic_set_bit(jzgc, reg1, offset, false);
+ break;
+ }
+}
+
+static void ingenic_gpio_irq_mask(struct irq_data *irqd)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
+ struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
+
+ gpio_ingenic_set_bit(jzgc, GPIO_MSK, irqd->hwirq, true);
+}
+
+static void ingenic_gpio_irq_unmask(struct irq_data *irqd)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
+ struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
+
+ gpio_ingenic_set_bit(jzgc, GPIO_MSK, irqd->hwirq, false);
+}
+
+static void ingenic_gpio_irq_enable(struct irq_data *irqd)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
+ struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
+ int irq = irqd->hwirq;
+
+ if (jzgc->jzpc->version >= ID_JZ4770)
+ gpio_ingenic_set_bit(jzgc, JZ4770_GPIO_INT, irq, true);
+ else
+ gpio_ingenic_set_bit(jzgc, JZ4740_GPIO_SELECT, irq, true);
+
+ ingenic_gpio_irq_unmask(irqd);
+}
+
+static void ingenic_gpio_irq_disable(struct irq_data *irqd)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
+ struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
+ int irq = irqd->hwirq;
+
+ ingenic_gpio_irq_mask(irqd);
+
+ if (jzgc->jzpc->version >= ID_JZ4770)
+ gpio_ingenic_set_bit(jzgc, JZ4770_GPIO_INT, irq, false);
+ else
+ gpio_ingenic_set_bit(jzgc, JZ4740_GPIO_SELECT, irq, false);
+}
+
+static void ingenic_gpio_irq_ack(struct irq_data *irqd)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
+ struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
+ int irq = irqd->hwirq;
+ bool high;
+
+ if (irqd_get_trigger_type(irqd) == IRQ_TYPE_EDGE_BOTH) {
+ /*
+ * Switch to an interrupt for the opposite edge to the one that
+ * triggered the interrupt being ACKed.
+ */
+ high = ingenic_gpio_get_value(jzgc, irq);
+ if (high)
+ irq_set_type(jzgc, irq, IRQ_TYPE_EDGE_FALLING);
+ else
+ irq_set_type(jzgc, irq, IRQ_TYPE_EDGE_RISING);
+ }
+
+ if (jzgc->jzpc->version >= ID_JZ4770)
+ gpio_ingenic_set_bit(jzgc, JZ4770_GPIO_FLAG, irq, false);
+ else
+ gpio_ingenic_set_bit(jzgc, JZ4740_GPIO_DATA, irq, true);
+}
+
+static int ingenic_gpio_irq_set_type(struct irq_data *irqd, unsigned int type)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
+ struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
+
+ switch (type) {
+ case IRQ_TYPE_EDGE_BOTH:
+ case IRQ_TYPE_EDGE_RISING:
+ case IRQ_TYPE_EDGE_FALLING:
+ irq_set_handler_locked(irqd, handle_edge_irq);
+ break;
+ case IRQ_TYPE_LEVEL_HIGH:
+ case IRQ_TYPE_LEVEL_LOW:
+ irq_set_handler_locked(irqd, handle_level_irq);
+ break;
+ default:
+ irq_set_handler_locked(irqd, handle_bad_irq);
+ }
+
+ if (type == IRQ_TYPE_EDGE_BOTH) {
+ /*
+ * The hardware does not support interrupts on both edges. The
+ * best we can do is to set up a single-edge interrupt and then
+ * switch to the opposing edge when ACKing the interrupt.
+ */
+ bool high = ingenic_gpio_get_value(jzgc, irqd->hwirq);
+
+ type = high ? IRQ_TYPE_EDGE_FALLING : IRQ_TYPE_EDGE_RISING;
+ }
+
+ irq_set_type(jzgc, irqd->hwirq, type);
+ return 0;
+}
+
+static int ingenic_gpio_irq_set_wake(struct irq_data *irqd, unsigned int on)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(irqd);
+ struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
+
+ return irq_set_irq_wake(jzgc->irq, on);
+}
+
+static void ingenic_gpio_irq_handler(struct irq_desc *desc)
+{
+ struct gpio_chip *gc = irq_desc_get_handler_data(desc);
+ struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
+ struct irq_chip *irq_chip = irq_data_get_irq_chip(&desc->irq_data);
+ unsigned long flag, i;
+
+ chained_irq_enter(irq_chip, desc);
+
+ if (jzgc->jzpc->version >= ID_JZ4770)
+ flag = gpio_ingenic_read_reg(jzgc, JZ4770_GPIO_FLAG);
+ else
+ flag = gpio_ingenic_read_reg(jzgc, JZ4740_GPIO_FLAG);
+
+ for_each_set_bit(i, &flag, 32)
+ generic_handle_irq(irq_linear_revmap(gc->irq.domain, i));
+ chained_irq_exit(irq_chip, desc);
+}
+
+static void ingenic_gpio_set(struct gpio_chip *gc,
+ unsigned int offset, int value)
+{
+ struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
+
+ ingenic_gpio_set_value(jzgc, offset, value);
+}
+
+static int ingenic_gpio_get(struct gpio_chip *gc, unsigned int offset)
+{
+ struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
+
+ return (int) ingenic_gpio_get_value(jzgc, offset);
+}
+
+static int ingenic_gpio_direction_input(struct gpio_chip *gc,
+ unsigned int offset)
+{
+ return pinctrl_gpio_direction_input(gc->base + offset);
+}
+
+static int ingenic_gpio_direction_output(struct gpio_chip *gc,
+ unsigned int offset, int value)
+{
+ ingenic_gpio_set(gc, offset, value);
+ return pinctrl_gpio_direction_output(gc->base + offset);
+}
+
static inline void ingenic_config_pin(struct ingenic_pinctrl *jzpc,
unsigned int pin, u8 reg, bool set)
{
return val & BIT(idx);
}
+static int ingenic_gpio_get_direction(struct gpio_chip *gc, unsigned int offset)
+{
+ struct ingenic_gpio_chip *jzgc = gpiochip_get_data(gc);
+ struct ingenic_pinctrl *jzpc = jzgc->jzpc;
+ unsigned int pin = gc->base + offset;
+
+ if (jzpc->version >= ID_JZ4770)
+ return ingenic_get_pin_config(jzpc, pin, JZ4770_GPIO_PAT1);
+
+ if (ingenic_get_pin_config(jzpc, pin, JZ4740_GPIO_SELECT))
+ return true;
+
+ return !ingenic_get_pin_config(jzpc, pin, JZ4740_GPIO_DIR);
+}
+
static const struct pinctrl_ops ingenic_pctlops = {
.get_groups_count = pinctrl_generic_get_group_count,
.get_group_name = pinctrl_generic_get_group_name,
if (jzpc->version >= ID_JZ4770) {
ingenic_config_pin(jzpc, pin, JZ4770_GPIO_INT, false);
- ingenic_config_pin(jzpc, pin, JZ4770_GPIO_MSK, false);
+ ingenic_config_pin(jzpc, pin, GPIO_MSK, false);
ingenic_config_pin(jzpc, pin, JZ4770_GPIO_PAT1, func & 0x2);
ingenic_config_pin(jzpc, pin, JZ4770_GPIO_PAT0, func & 0x1);
} else {
if (jzpc->version >= ID_JZ4770) {
ingenic_config_pin(jzpc, pin, JZ4770_GPIO_INT, false);
- ingenic_config_pin(jzpc, pin, JZ4770_GPIO_MSK, true);
+ ingenic_config_pin(jzpc, pin, GPIO_MSK, true);
ingenic_config_pin(jzpc, pin, JZ4770_GPIO_PAT1, input);
} else {
ingenic_config_pin(jzpc, pin, JZ4740_GPIO_SELECT, false);
static const struct of_device_id ingenic_pinctrl_of_match[] = {
{ .compatible = "ingenic,jz4740-pinctrl", .data = (void *) ID_JZ4740 },
+ { .compatible = "ingenic,jz4725b-pinctrl", .data = (void *)ID_JZ4725B },
{ .compatible = "ingenic,jz4770-pinctrl", .data = (void *) ID_JZ4770 },
{ .compatible = "ingenic,jz4780-pinctrl", .data = (void *) ID_JZ4780 },
{},
};
-static int ingenic_pinctrl_probe(struct platform_device *pdev)
+static const struct of_device_id ingenic_gpio_of_match[] __initconst = {
+ { .compatible = "ingenic,jz4740-gpio", },
+ { .compatible = "ingenic,jz4770-gpio", },
+ { .compatible = "ingenic,jz4780-gpio", },
+ {},
+};
+
+static int __init ingenic_gpio_probe(struct ingenic_pinctrl *jzpc,
+ struct device_node *node)
+{
+ struct ingenic_gpio_chip *jzgc;
+ struct device *dev = jzpc->dev;
+ unsigned int bank;
+ int err;
+
+ err = of_property_read_u32(node, "reg", &bank);
+ if (err) {
+ dev_err(dev, "Cannot read \"reg\" property: %i\n", err);
+ return err;
+ }
+
+ jzgc = devm_kzalloc(dev, sizeof(*jzgc), GFP_KERNEL);
+ if (!jzgc)
+ return -ENOMEM;
+
+ jzgc->jzpc = jzpc;
+ jzgc->reg_base = bank * 0x100;
+
+ jzgc->gc.label = devm_kasprintf(dev, GFP_KERNEL, "GPIO%c", 'A' + bank);
+ if (!jzgc->gc.label)
+ return -ENOMEM;
+
+ /* DO NOT EXPAND THIS: FOR BACKWARD GPIO NUMBERSPACE COMPATIBIBILITY
+ * ONLY: WORK TO TRANSITION CONSUMERS TO USE THE GPIO DESCRIPTOR API IN
+ * <linux/gpio/consumer.h> INSTEAD.
+ */
+ jzgc->gc.base = bank * 32;
+
+ jzgc->gc.ngpio = 32;
+ jzgc->gc.parent = dev;
+ jzgc->gc.of_node = node;
+ jzgc->gc.owner = THIS_MODULE;
+
+ jzgc->gc.set = ingenic_gpio_set;
+ jzgc->gc.get = ingenic_gpio_get;
+ jzgc->gc.direction_input = ingenic_gpio_direction_input;
+ jzgc->gc.direction_output = ingenic_gpio_direction_output;
+ jzgc->gc.get_direction = ingenic_gpio_get_direction;
+
+ if (of_property_read_bool(node, "gpio-ranges")) {
+ jzgc->gc.request = gpiochip_generic_request;
+ jzgc->gc.free = gpiochip_generic_free;
+ }
+
+ err = devm_gpiochip_add_data(dev, &jzgc->gc, jzgc);
+ if (err)
+ return err;
+
+ jzgc->irq = irq_of_parse_and_map(node, 0);
+ if (!jzgc->irq)
+ return -EINVAL;
+
+ jzgc->irq_chip.name = jzgc->gc.label;
+ jzgc->irq_chip.irq_enable = ingenic_gpio_irq_enable;
+ jzgc->irq_chip.irq_disable = ingenic_gpio_irq_disable;
+ jzgc->irq_chip.irq_unmask = ingenic_gpio_irq_unmask;
+ jzgc->irq_chip.irq_mask = ingenic_gpio_irq_mask;
+ jzgc->irq_chip.irq_ack = ingenic_gpio_irq_ack;
+ jzgc->irq_chip.irq_set_type = ingenic_gpio_irq_set_type;
+ jzgc->irq_chip.irq_set_wake = ingenic_gpio_irq_set_wake;
+ jzgc->irq_chip.flags = IRQCHIP_MASK_ON_SUSPEND;
+
+ err = gpiochip_irqchip_add(&jzgc->gc, &jzgc->irq_chip, 0,
+ handle_level_irq, IRQ_TYPE_NONE);
+ if (err)
+ return err;
+
+ gpiochip_set_chained_irqchip(&jzgc->gc, &jzgc->irq_chip,
+ jzgc->irq, ingenic_gpio_irq_handler);
+ return 0;
+}
+
+static int __init ingenic_pinctrl_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct ingenic_pinctrl *jzpc;
const struct of_device_id *of_id = of_match_device(
ingenic_pinctrl_of_match, dev);
const struct ingenic_chip_info *chip_info;
+ struct device_node *node;
unsigned int i;
int err;
if (jzpc->version >= ID_JZ4770)
chip_info = &jz4770_chip_info;
+ else if (jzpc->version >= ID_JZ4725B)
+ chip_info = &jz4725b_chip_info;
else
chip_info = &jz4740_chip_info;
jzpc->info = chip_info;
dev_set_drvdata(dev, jzpc->map);
- if (dev->of_node) {
- err = of_platform_populate(dev->of_node, NULL, NULL, dev);
- if (err) {
- dev_err(dev, "Failed to probe GPIO devices\n");
- return err;
+ for_each_child_of_node(dev->of_node, node) {
+ if (of_match_node(ingenic_gpio_of_match, node)) {
+ err = ingenic_gpio_probe(jzpc, node);
+ if (err)
+ return err;
}
}
static const struct platform_device_id ingenic_pinctrl_ids[] = {
{ "jz4740-pinctrl", ID_JZ4740 },
+ { "jz4725b-pinctrl", ID_JZ4725B },
{ "jz4770-pinctrl", ID_JZ4770 },
{ "jz4780-pinctrl", ID_JZ4780 },
{},
.driver = {
.name = "pinctrl-ingenic",
.of_match_table = of_match_ptr(ingenic_pinctrl_of_match),
- .suppress_bind_attrs = true,
},
- .probe = ingenic_pinctrl_probe,
.id_table = ingenic_pinctrl_ids,
};
static int __init ingenic_pinctrl_drv_register(void)
{
- return platform_driver_register(&ingenic_pinctrl_driver);
+ return platform_driver_probe(&ingenic_pinctrl_driver,
+ ingenic_pinctrl_probe);
}
-postcore_initcall(ingenic_pinctrl_drv_register);
+subsys_initcall(ingenic_pinctrl_drv_register);
int ret, i;
if (!pins && !groups) {
- dev_err(pctldev->dev, "%s defines neither pins nor groups\n",
- np->name);
+ dev_err(pctldev->dev, "%pOFn defines neither pins nor groups\n",
+ np);
return;
}
if (pins && groups) {
- dev_err(pctldev->dev, "%s defines both pins and groups\n",
- np->name);
+ dev_err(pctldev->dev, "%pOFn defines both pins and groups\n",
+ np);
return;
}
*arg = (reg & LPC18XX_SCU_PIN_EHD_MASK) >> LPC18XX_SCU_PIN_EHD_POS;
switch (*arg) {
case 3: *arg += 5;
+ /* fall through */
case 2: *arg += 5;
+ /* fall through */
case 1: *arg += 3;
+ /* fall through */
case 0: *arg += 4;
}
break;
switch (param_val) {
case 20: param_val -= 5;
+ /* fall through */
case 14: param_val -= 5;
+ /* fall through */
case 8: param_val -= 3;
+ /* fall through */
case 4: param_val -= 4;
break;
default:
#include <linux/device.h>
#include <linux/mutex.h>
#include <linux/module.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/i2c.h>
#include <linux/spi/spi.h>
#include <linux/spi/mcp23s08.h>
return err;
}
+ return 0;
+}
+
+static int mcp23s08_irqchip_setup(struct mcp23s08 *mcp)
+{
+ struct gpio_chip *chip = &mcp->chip;
+ int err;
+
err = gpiochip_irqchip_add_nested(chip,
&mcp23s08_irq_chip,
0,
}
if (mcp->irq && mcp->irq_controller) {
- ret = mcp23s08_irq_setup(mcp);
+ ret = mcp23s08_irqchip_setup(mcp);
if (ret)
goto fail;
}
goto fail;
}
+ if (mcp->irq)
+ ret = mcp23s08_irq_setup(mcp);
+
fail:
if (ret < 0)
dev_dbg(dev, "can't setup chip %d, --> %d\n", addr, ret);
* version 2, as published by the Free Software Foundation.
*/
-#include <linux/gpio.h>
#include <linux/gpio/driver.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/platform_device.h>
#include <linux/io.h>
#include <linux/bitops.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
#include <linux/pinctrl/machine.h>
*/
grp = pinctrl_name_to_group(info, np->name);
if (!grp) {
- dev_err(info->dev, "unable to find group for node %s\n",
- np->name);
+ dev_err(info->dev, "unable to find group for node %pOFn\n",
+ np);
return -EINVAL;
}
int i, j;
int ret;
- dev_dbg(info->dev, "group(%d): %s\n", index, np->name);
+ dev_dbg(info->dev, "group(%d): %pOFn\n", index, np);
/* Initialise group */
grp->name = np->name;
static u32 grp_index;
u32 i = 0;
- dev_dbg(info->dev, "parse function(%d): %s\n", index, np->name);
+ dev_dbg(info->dev, "parse function(%d): %pOFn\n", index, np);
func = &info->functions[index];
+// SPDX-License-Identifier: GPL-2.0
/*
* Combined GPIO and pin controller support for Renesas RZ/A1 (r7s72100) SoC
*
* Copyright (C) 2017 Jacopo Mondi
- *
- * This file is licensed under the terms of the GNU General Public
- * License version 2. This program is licensed "as is" without any
- * warranty of any kind, whether express or implied.
*/
/*
&npin_configs);
if (ret) {
dev_err(rza1_pctl->dev,
- "Unable to parse pin configuration options for %s\n",
- np->name);
+ "Unable to parse pin configuration options for %pOFn\n",
+ np);
return ret;
}
*chip = rza1_gpiochip_template;
chip->base = -1;
- chip->label = devm_kasprintf(rza1_pctl->dev, GFP_KERNEL, "%s",
- np->name);
+ chip->label = devm_kasprintf(rza1_pctl->dev, GFP_KERNEL, "%pOFn",
+ np);
chip->ngpio = of_args.args[2];
chip->of_node = np;
chip->parent = rza1_pctl->dev;
ret = rza1_parse_gpiochip(rza1_pctl, child, &gpio_chips[i],
&gpio_ranges[i]);
if (ret)
- goto gpiochip_remove;
+ return ret;
++i;
}
dev_info(rza1_pctl->dev, "Registered %u gpio controllers\n", i);
return 0;
-
-gpiochip_remove:
- for (; i > 0; i--)
- devm_gpiochip_remove(rza1_pctl->dev, &gpio_chips[i - 1]);
-
- return ret;
}
/**
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2014-2018 Renesas Electronics Europe Limited
+ *
+ * Phil Edworthy <phil.edworthy@renesas.com>
+ * Based on a driver originally written by Michel Pollet at Renesas.
+ */
+
+#include <dt-bindings/pinctrl/rzn1-pinctrl.h>
+#include <linux/clk.h>
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/pinctrl/pinconf-generic.h>
+#include <linux/pinctrl/pinctrl.h>
+#include <linux/pinctrl/pinmux.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include "core.h"
+#include "pinconf.h"
+#include "pinctrl-utils.h"
+
+/* Field positions and masks in the pinmux registers */
+#define RZN1_L1_PIN_DRIVE_STRENGTH 10
+#define RZN1_L1_PIN_DRIVE_STRENGTH_4MA 0
+#define RZN1_L1_PIN_DRIVE_STRENGTH_6MA 1
+#define RZN1_L1_PIN_DRIVE_STRENGTH_8MA 2
+#define RZN1_L1_PIN_DRIVE_STRENGTH_12MA 3
+#define RZN1_L1_PIN_PULL 8
+#define RZN1_L1_PIN_PULL_NONE 0
+#define RZN1_L1_PIN_PULL_UP 1
+#define RZN1_L1_PIN_PULL_DOWN 3
+#define RZN1_L1_FUNCTION 0
+#define RZN1_L1_FUNC_MASK 0xf
+#define RZN1_L1_FUNCTION_L2 0xf
+
+/*
+ * The hardware manual describes two levels of multiplexing, but it's more
+ * logical to think of the hardware as three levels, with level 3 consisting of
+ * the multiplexing for Ethernet MDIO signals.
+ *
+ * Level 1 functions go from 0 to 9, with level 1 function '15' (0xf) specifying
+ * that level 2 functions are used instead. Level 2 has a lot more options,
+ * going from 0 to 61. Level 3 allows selection of MDIO functions which can be
+ * floating, or one of seven internal peripherals. Unfortunately, there are two
+ * level 2 functions that can select MDIO, and two MDIO channels so we have four
+ * sets of level 3 functions.
+ *
+ * For this driver, we've compounded the numbers together, so:
+ * 0 to 9 is level 1
+ * 10 to 71 is 10 + level 2 number
+ * 72 to 79 is 72 + MDIO0 source for level 2 MDIO function.
+ * 80 to 87 is 80 + MDIO0 source for level 2 MDIO_E1 function.
+ * 88 to 95 is 88 + MDIO1 source for level 2 MDIO function.
+ * 96 to 103 is 96 + MDIO1 source for level 2 MDIO_E1 function.
+ * Examples:
+ * Function 28 corresponds UART0
+ * Function 73 corresponds to MDIO0 to GMAC0
+ *
+ * There are 170 configurable pins (called PL_GPIO in the datasheet).
+ */
+
+/*
+ * Structure detailing the HW registers on the RZ/N1 devices.
+ * Both the Level 1 mux registers and Level 2 mux registers have the same
+ * structure. The only difference is that Level 2 has additional MDIO registers
+ * at the end.
+ */
+struct rzn1_pinctrl_regs {
+ u32 conf[170];
+ u32 pad0[86];
+ u32 status_protect; /* 0x400 */
+ /* MDIO mux registers, level2 only */
+ u32 l2_mdio[2];
+};
+
+/**
+ * struct rzn1_pmx_func - describes rzn1 pinmux functions
+ * @name: the name of this specific function
+ * @groups: corresponding pin groups
+ * @num_groups: the number of groups
+ */
+struct rzn1_pmx_func {
+ const char *name;
+ const char **groups;
+ unsigned int num_groups;
+};
+
+/**
+ * struct rzn1_pin_group - describes an rzn1 pin group
+ * @name: the name of this specific pin group
+ * @func: the name of the function selected by this group
+ * @npins: the number of pins in this group array, i.e. the number of
+ * elements in .pins so we can iterate over that array
+ * @pins: array of pins. Needed due to pinctrl_ops.get_group_pins()
+ * @pin_ids: array of pin_ids, i.e. the value used to select the mux
+ */
+struct rzn1_pin_group {
+ const char *name;
+ const char *func;
+ unsigned int npins;
+ unsigned int *pins;
+ u8 *pin_ids;
+};
+
+struct rzn1_pinctrl {
+ struct device *dev;
+ struct clk *clk;
+ struct pinctrl_dev *pctl;
+ struct rzn1_pinctrl_regs __iomem *lev1;
+ struct rzn1_pinctrl_regs __iomem *lev2;
+ u32 lev1_protect_phys;
+ u32 lev2_protect_phys;
+ u32 mdio_func[2];
+
+ struct rzn1_pin_group *groups;
+ unsigned int ngroups;
+
+ struct rzn1_pmx_func *functions;
+ unsigned int nfunctions;
+};
+
+#define RZN1_PINS_PROP "pinmux"
+
+#define RZN1_PIN(pin) PINCTRL_PIN(pin, "pl_gpio"#pin)
+
+static const struct pinctrl_pin_desc rzn1_pins[] = {
+ RZN1_PIN(0), RZN1_PIN(1), RZN1_PIN(2), RZN1_PIN(3), RZN1_PIN(4),
+ RZN1_PIN(5), RZN1_PIN(6), RZN1_PIN(7), RZN1_PIN(8), RZN1_PIN(9),
+ RZN1_PIN(10), RZN1_PIN(11), RZN1_PIN(12), RZN1_PIN(13), RZN1_PIN(14),
+ RZN1_PIN(15), RZN1_PIN(16), RZN1_PIN(17), RZN1_PIN(18), RZN1_PIN(19),
+ RZN1_PIN(20), RZN1_PIN(21), RZN1_PIN(22), RZN1_PIN(23), RZN1_PIN(24),
+ RZN1_PIN(25), RZN1_PIN(26), RZN1_PIN(27), RZN1_PIN(28), RZN1_PIN(29),
+ RZN1_PIN(30), RZN1_PIN(31), RZN1_PIN(32), RZN1_PIN(33), RZN1_PIN(34),
+ RZN1_PIN(35), RZN1_PIN(36), RZN1_PIN(37), RZN1_PIN(38), RZN1_PIN(39),
+ RZN1_PIN(40), RZN1_PIN(41), RZN1_PIN(42), RZN1_PIN(43), RZN1_PIN(44),
+ RZN1_PIN(45), RZN1_PIN(46), RZN1_PIN(47), RZN1_PIN(48), RZN1_PIN(49),
+ RZN1_PIN(50), RZN1_PIN(51), RZN1_PIN(52), RZN1_PIN(53), RZN1_PIN(54),
+ RZN1_PIN(55), RZN1_PIN(56), RZN1_PIN(57), RZN1_PIN(58), RZN1_PIN(59),
+ RZN1_PIN(60), RZN1_PIN(61), RZN1_PIN(62), RZN1_PIN(63), RZN1_PIN(64),
+ RZN1_PIN(65), RZN1_PIN(66), RZN1_PIN(67), RZN1_PIN(68), RZN1_PIN(69),
+ RZN1_PIN(70), RZN1_PIN(71), RZN1_PIN(72), RZN1_PIN(73), RZN1_PIN(74),
+ RZN1_PIN(75), RZN1_PIN(76), RZN1_PIN(77), RZN1_PIN(78), RZN1_PIN(79),
+ RZN1_PIN(80), RZN1_PIN(81), RZN1_PIN(82), RZN1_PIN(83), RZN1_PIN(84),
+ RZN1_PIN(85), RZN1_PIN(86), RZN1_PIN(87), RZN1_PIN(88), RZN1_PIN(89),
+ RZN1_PIN(90), RZN1_PIN(91), RZN1_PIN(92), RZN1_PIN(93), RZN1_PIN(94),
+ RZN1_PIN(95), RZN1_PIN(96), RZN1_PIN(97), RZN1_PIN(98), RZN1_PIN(99),
+ RZN1_PIN(100), RZN1_PIN(101), RZN1_PIN(102), RZN1_PIN(103),
+ RZN1_PIN(104), RZN1_PIN(105), RZN1_PIN(106), RZN1_PIN(107),
+ RZN1_PIN(108), RZN1_PIN(109), RZN1_PIN(110), RZN1_PIN(111),
+ RZN1_PIN(112), RZN1_PIN(113), RZN1_PIN(114), RZN1_PIN(115),
+ RZN1_PIN(116), RZN1_PIN(117), RZN1_PIN(118), RZN1_PIN(119),
+ RZN1_PIN(120), RZN1_PIN(121), RZN1_PIN(122), RZN1_PIN(123),
+ RZN1_PIN(124), RZN1_PIN(125), RZN1_PIN(126), RZN1_PIN(127),
+ RZN1_PIN(128), RZN1_PIN(129), RZN1_PIN(130), RZN1_PIN(131),
+ RZN1_PIN(132), RZN1_PIN(133), RZN1_PIN(134), RZN1_PIN(135),
+ RZN1_PIN(136), RZN1_PIN(137), RZN1_PIN(138), RZN1_PIN(139),
+ RZN1_PIN(140), RZN1_PIN(141), RZN1_PIN(142), RZN1_PIN(143),
+ RZN1_PIN(144), RZN1_PIN(145), RZN1_PIN(146), RZN1_PIN(147),
+ RZN1_PIN(148), RZN1_PIN(149), RZN1_PIN(150), RZN1_PIN(151),
+ RZN1_PIN(152), RZN1_PIN(153), RZN1_PIN(154), RZN1_PIN(155),
+ RZN1_PIN(156), RZN1_PIN(157), RZN1_PIN(158), RZN1_PIN(159),
+ RZN1_PIN(160), RZN1_PIN(161), RZN1_PIN(162), RZN1_PIN(163),
+ RZN1_PIN(164), RZN1_PIN(165), RZN1_PIN(166), RZN1_PIN(167),
+ RZN1_PIN(168), RZN1_PIN(169),
+};
+
+enum {
+ LOCK_LEVEL1 = 0x1,
+ LOCK_LEVEL2 = 0x2,
+ LOCK_ALL = LOCK_LEVEL1 | LOCK_LEVEL2,
+};
+
+static void rzn1_hw_set_lock(struct rzn1_pinctrl *ipctl, u8 lock, u8 value)
+{
+ /*
+ * The pinmux configuration is locked by writing the physical address of
+ * the status_protect register to itself. It is unlocked by writing the
+ * address | 1.
+ */
+ if (lock & LOCK_LEVEL1) {
+ u32 val = ipctl->lev1_protect_phys | !(value & LOCK_LEVEL1);
+
+ writel(val, &ipctl->lev1->status_protect);
+ }
+
+ if (lock & LOCK_LEVEL2) {
+ u32 val = ipctl->lev2_protect_phys | !(value & LOCK_LEVEL2);
+
+ writel(val, &ipctl->lev2->status_protect);
+ }
+}
+
+static void rzn1_pinctrl_mdio_select(struct rzn1_pinctrl *ipctl, int mdio,
+ u32 func)
+{
+ if (ipctl->mdio_func[mdio] >= 0 && ipctl->mdio_func[mdio] != func)
+ dev_warn(ipctl->dev, "conflicting setting for mdio%d!\n", mdio);
+ ipctl->mdio_func[mdio] = func;
+
+ dev_dbg(ipctl->dev, "setting mdio%d to %u\n", mdio, func);
+
+ writel(func, &ipctl->lev2->l2_mdio[mdio]);
+}
+
+/*
+ * Using a composite pin description, set the hardware pinmux registers
+ * with the corresponding values.
+ * Make sure to unlock write protection and reset it afterward.
+ *
+ * NOTE: There is no protection for potential concurrency, it is assumed these
+ * calls are serialized already.
+ */
+static int rzn1_set_hw_pin_func(struct rzn1_pinctrl *ipctl, unsigned int pin,
+ u32 pin_config, u8 use_locks)
+{
+ u32 l1_cache;
+ u32 l2_cache;
+ u32 l1;
+ u32 l2;
+
+ /* Level 3 MDIO multiplexing */
+ if (pin_config >= RZN1_FUNC_MDIO0_HIGHZ &&
+ pin_config <= RZN1_FUNC_MDIO1_E1_SWITCH) {
+ int mdio_channel;
+ u32 mdio_func;
+
+ if (pin_config <= RZN1_FUNC_MDIO1_HIGHZ)
+ mdio_channel = 0;
+ else
+ mdio_channel = 1;
+
+ /* Get MDIO func, and convert the func to the level 2 number */
+ if (pin_config <= RZN1_FUNC_MDIO0_SWITCH) {
+ mdio_func = pin_config - RZN1_FUNC_MDIO0_HIGHZ;
+ pin_config = RZN1_FUNC_ETH_MDIO;
+ } else if (pin_config <= RZN1_FUNC_MDIO0_E1_SWITCH) {
+ mdio_func = pin_config - RZN1_FUNC_MDIO0_E1_HIGHZ;
+ pin_config = RZN1_FUNC_ETH_MDIO_E1;
+ } else if (pin_config <= RZN1_FUNC_MDIO1_SWITCH) {
+ mdio_func = pin_config - RZN1_FUNC_MDIO1_HIGHZ;
+ pin_config = RZN1_FUNC_ETH_MDIO;
+ } else {
+ mdio_func = pin_config - RZN1_FUNC_MDIO1_E1_HIGHZ;
+ pin_config = RZN1_FUNC_ETH_MDIO_E1;
+ }
+ rzn1_pinctrl_mdio_select(ipctl, mdio_channel, mdio_func);
+ }
+
+ /* Note here, we do not allow anything past the MDIO Mux values */
+ if (pin >= ARRAY_SIZE(ipctl->lev1->conf) ||
+ pin_config >= RZN1_FUNC_MDIO0_HIGHZ)
+ return -EINVAL;
+
+ l1 = readl(&ipctl->lev1->conf[pin]);
+ l1_cache = l1;
+ l2 = readl(&ipctl->lev2->conf[pin]);
+ l2_cache = l2;
+
+ dev_dbg(ipctl->dev, "setting func for pin %u to %u\n", pin, pin_config);
+
+ l1 &= ~(RZN1_L1_FUNC_MASK << RZN1_L1_FUNCTION);
+
+ if (pin_config < RZN1_FUNC_L2_OFFSET) {
+ l1 |= (pin_config << RZN1_L1_FUNCTION);
+ } else {
+ l1 |= (RZN1_L1_FUNCTION_L2 << RZN1_L1_FUNCTION);
+
+ l2 = pin_config - RZN1_FUNC_L2_OFFSET;
+ }
+
+ /* If either configuration changes, we update both anyway */
+ if (l1 != l1_cache || l2 != l2_cache) {
+ writel(l1, &ipctl->lev1->conf[pin]);
+ writel(l2, &ipctl->lev2->conf[pin]);
+ }
+
+ return 0;
+}
+
+static const struct rzn1_pin_group *rzn1_pinctrl_find_group_by_name(
+ const struct rzn1_pinctrl *ipctl, const char *name)
+{
+ unsigned int i;
+
+ for (i = 0; i < ipctl->ngroups; i++) {
+ if (!strcmp(ipctl->groups[i].name, name))
+ return &ipctl->groups[i];
+ }
+
+ return NULL;
+}
+
+static int rzn1_get_groups_count(struct pinctrl_dev *pctldev)
+{
+ struct rzn1_pinctrl *ipctl = pinctrl_dev_get_drvdata(pctldev);
+
+ return ipctl->ngroups;
+}
+
+static const char *rzn1_get_group_name(struct pinctrl_dev *pctldev,
+ unsigned int selector)
+{
+ struct rzn1_pinctrl *ipctl = pinctrl_dev_get_drvdata(pctldev);
+
+ return ipctl->groups[selector].name;
+}
+
+static int rzn1_get_group_pins(struct pinctrl_dev *pctldev,
+ unsigned int selector, const unsigned int **pins,
+ unsigned int *npins)
+{
+ struct rzn1_pinctrl *ipctl = pinctrl_dev_get_drvdata(pctldev);
+
+ if (selector >= ipctl->ngroups)
+ return -EINVAL;
+
+ *pins = ipctl->groups[selector].pins;
+ *npins = ipctl->groups[selector].npins;
+
+ return 0;
+}
+
+/*
+ * This function is called for each pinctl 'Function' node.
+ * Sub-nodes can be used to describe multiple 'Groups' for the 'Function'
+ * If there aren't any sub-nodes, the 'Group' is essentially the 'Function'.
+ * Each 'Group' uses pinmux = <...> to detail the pins and data used to select
+ * the functionality. Each 'Group' has optional pin configurations that apply
+ * to all pins in the 'Group'.
+ */
+static int rzn1_dt_node_to_map_one(struct pinctrl_dev *pctldev,
+ struct device_node *np,
+ struct pinctrl_map **map,
+ unsigned int *num_maps)
+{
+ struct rzn1_pinctrl *ipctl = pinctrl_dev_get_drvdata(pctldev);
+ const struct rzn1_pin_group *grp;
+ unsigned long *configs = NULL;
+ unsigned int reserved_maps = *num_maps;
+ unsigned int num_configs = 0;
+ unsigned int reserve = 1;
+ int ret;
+
+ dev_dbg(ipctl->dev, "processing node %pOF\n", np);
+
+ grp = rzn1_pinctrl_find_group_by_name(ipctl, np->name);
+ if (!grp) {
+ dev_err(ipctl->dev, "unable to find group for node %pOF\n", np);
+
+ return -EINVAL;
+ }
+
+ /* Get the group's pin configuration */
+ ret = pinconf_generic_parse_dt_config(np, pctldev, &configs,
+ &num_configs);
+ if (ret < 0) {
+ dev_err(ipctl->dev, "%pOF: could not parse property\n", np);
+
+ return ret;
+ }
+
+ if (num_configs)
+ reserve++;
+
+ /* Increase the number of maps to cover this group */
+ ret = pinctrl_utils_reserve_map(pctldev, map, &reserved_maps, num_maps,
+ reserve);
+ if (ret < 0)
+ goto out;
+
+ /* Associate the group with the function */
+ ret = pinctrl_utils_add_map_mux(pctldev, map, &reserved_maps, num_maps,
+ grp->name, grp->func);
+ if (ret < 0)
+ goto out;
+
+ if (num_configs) {
+ /* Associate the group's pin configuration with the group */
+ ret = pinctrl_utils_add_map_configs(pctldev, map,
+ &reserved_maps, num_maps, grp->name,
+ configs, num_configs,
+ PIN_MAP_TYPE_CONFIGS_GROUP);
+ if (ret < 0)
+ goto out;
+ }
+
+ dev_dbg(pctldev->dev, "maps: function %s group %s (%d pins)\n",
+ grp->func, grp->name, grp->npins);
+
+out:
+ kfree(configs);
+
+ return ret;
+}
+
+static int rzn1_dt_node_to_map(struct pinctrl_dev *pctldev,
+ struct device_node *np,
+ struct pinctrl_map **map,
+ unsigned int *num_maps)
+{
+ struct device_node *child;
+ int ret;
+
+ *map = NULL;
+ *num_maps = 0;
+
+ ret = rzn1_dt_node_to_map_one(pctldev, np, map, num_maps);
+ if (ret < 0)
+ return ret;
+
+ for_each_child_of_node(np, child) {
+ ret = rzn1_dt_node_to_map_one(pctldev, child, map, num_maps);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static const struct pinctrl_ops rzn1_pctrl_ops = {
+ .get_groups_count = rzn1_get_groups_count,
+ .get_group_name = rzn1_get_group_name,
+ .get_group_pins = rzn1_get_group_pins,
+ .dt_node_to_map = rzn1_dt_node_to_map,
+ .dt_free_map = pinctrl_utils_free_map,
+};
+
+static int rzn1_pmx_get_funcs_count(struct pinctrl_dev *pctldev)
+{
+ struct rzn1_pinctrl *ipctl = pinctrl_dev_get_drvdata(pctldev);
+
+ return ipctl->nfunctions;
+}
+
+static const char *rzn1_pmx_get_func_name(struct pinctrl_dev *pctldev,
+ unsigned int selector)
+{
+ struct rzn1_pinctrl *ipctl = pinctrl_dev_get_drvdata(pctldev);
+
+ return ipctl->functions[selector].name;
+}
+
+static int rzn1_pmx_get_groups(struct pinctrl_dev *pctldev,
+ unsigned int selector,
+ const char * const **groups,
+ unsigned int * const num_groups)
+{
+ struct rzn1_pinctrl *ipctl = pinctrl_dev_get_drvdata(pctldev);
+
+ *groups = ipctl->functions[selector].groups;
+ *num_groups = ipctl->functions[selector].num_groups;
+
+ return 0;
+}
+
+static int rzn1_set_mux(struct pinctrl_dev *pctldev, unsigned int selector,
+ unsigned int group)
+{
+ struct rzn1_pinctrl *ipctl = pinctrl_dev_get_drvdata(pctldev);
+ struct rzn1_pin_group *grp = &ipctl->groups[group];
+ unsigned int i, grp_pins = grp->npins;
+
+ dev_dbg(ipctl->dev, "set mux %s(%d) group %s(%d)\n",
+ ipctl->functions[selector].name, selector, grp->name, group);
+
+ rzn1_hw_set_lock(ipctl, LOCK_ALL, LOCK_ALL);
+ for (i = 0; i < grp_pins; i++)
+ rzn1_set_hw_pin_func(ipctl, grp->pins[i], grp->pin_ids[i], 0);
+ rzn1_hw_set_lock(ipctl, LOCK_ALL, 0);
+
+ return 0;
+}
+
+static const struct pinmux_ops rzn1_pmx_ops = {
+ .get_functions_count = rzn1_pmx_get_funcs_count,
+ .get_function_name = rzn1_pmx_get_func_name,
+ .get_function_groups = rzn1_pmx_get_groups,
+ .set_mux = rzn1_set_mux,
+};
+
+static int rzn1_pinconf_get(struct pinctrl_dev *pctldev, unsigned int pin,
+ unsigned long *config)
+{
+ struct rzn1_pinctrl *ipctl = pinctrl_dev_get_drvdata(pctldev);
+ enum pin_config_param param = pinconf_to_config_param(*config);
+ const u32 reg_drive[4] = { 4, 6, 8, 12 };
+ u32 pull, drive, l1mux;
+ u32 l1, l2, arg = 0;
+
+ if (pin >= ARRAY_SIZE(ipctl->lev1->conf))
+ return -EINVAL;
+
+ l1 = readl(&ipctl->lev1->conf[pin]);
+
+ l1mux = l1 & RZN1_L1_FUNC_MASK;
+ pull = (l1 >> RZN1_L1_PIN_PULL) & 0x3;
+ drive = (l1 >> RZN1_L1_PIN_DRIVE_STRENGTH) & 0x3;
+
+ switch (param) {
+ case PIN_CONFIG_BIAS_PULL_UP:
+ if (pull != RZN1_L1_PIN_PULL_UP)
+ return -EINVAL;
+ break;
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ if (pull != RZN1_L1_PIN_PULL_DOWN)
+ return -EINVAL;
+ break;
+ case PIN_CONFIG_BIAS_DISABLE:
+ if (pull != RZN1_L1_PIN_PULL_NONE)
+ return -EINVAL;
+ break;
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ arg = reg_drive[drive];
+ break;
+ case PIN_CONFIG_BIAS_HIGH_IMPEDANCE:
+ l2 = readl(&ipctl->lev2->conf[pin]);
+ if (l1mux == RZN1_L1_FUNCTION_L2) {
+ if (l2 != 0)
+ return -EINVAL;
+ } else if (l1mux != RZN1_FUNC_HIGHZ) {
+ return -EINVAL;
+ }
+ break;
+ default:
+ return -ENOTSUPP;
+ }
+
+ *config = pinconf_to_config_packed(param, arg);
+
+ return 0;
+}
+
+static int rzn1_pinconf_set(struct pinctrl_dev *pctldev, unsigned int pin,
+ unsigned long *configs, unsigned int num_configs)
+{
+ struct rzn1_pinctrl *ipctl = pinctrl_dev_get_drvdata(pctldev);
+ enum pin_config_param param;
+ unsigned int i;
+ u32 l1, l1_cache;
+ u32 drv;
+ u32 arg;
+
+ if (pin >= ARRAY_SIZE(ipctl->lev1->conf))
+ return -EINVAL;
+
+ l1 = readl(&ipctl->lev1->conf[pin]);
+ l1_cache = l1;
+
+ for (i = 0; i < num_configs; i++) {
+ param = pinconf_to_config_param(configs[i]);
+ arg = pinconf_to_config_argument(configs[i]);
+
+ switch (param) {
+ case PIN_CONFIG_BIAS_PULL_UP:
+ dev_dbg(ipctl->dev, "set pin %d pull up\n", pin);
+ l1 &= ~(0x3 << RZN1_L1_PIN_PULL);
+ l1 |= (RZN1_L1_PIN_PULL_UP << RZN1_L1_PIN_PULL);
+ break;
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ dev_dbg(ipctl->dev, "set pin %d pull down\n", pin);
+ l1 &= ~(0x3 << RZN1_L1_PIN_PULL);
+ l1 |= (RZN1_L1_PIN_PULL_DOWN << RZN1_L1_PIN_PULL);
+ break;
+ case PIN_CONFIG_BIAS_DISABLE:
+ dev_dbg(ipctl->dev, "set pin %d bias off\n", pin);
+ l1 &= ~(0x3 << RZN1_L1_PIN_PULL);
+ l1 |= (RZN1_L1_PIN_PULL_NONE << RZN1_L1_PIN_PULL);
+ break;
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ dev_dbg(ipctl->dev, "set pin %d drv %umA\n", pin, arg);
+ switch (arg) {
+ case 4:
+ drv = RZN1_L1_PIN_DRIVE_STRENGTH_4MA;
+ break;
+ case 6:
+ drv = RZN1_L1_PIN_DRIVE_STRENGTH_6MA;
+ break;
+ case 8:
+ drv = RZN1_L1_PIN_DRIVE_STRENGTH_8MA;
+ break;
+ case 12:
+ drv = RZN1_L1_PIN_DRIVE_STRENGTH_12MA;
+ break;
+ default:
+ dev_err(ipctl->dev,
+ "Drive strength %umA not supported\n",
+ arg);
+
+ return -EINVAL;
+ }
+
+ l1 &= ~(0x3 << RZN1_L1_PIN_DRIVE_STRENGTH);
+ l1 |= (drv << RZN1_L1_PIN_DRIVE_STRENGTH);
+ break;
+
+ case PIN_CONFIG_BIAS_HIGH_IMPEDANCE:
+ dev_dbg(ipctl->dev, "set pin %d High-Z\n", pin);
+ l1 &= ~RZN1_L1_FUNC_MASK;
+ l1 |= RZN1_FUNC_HIGHZ;
+ break;
+ default:
+ return -ENOTSUPP;
+ }
+ }
+
+ if (l1 != l1_cache) {
+ rzn1_hw_set_lock(ipctl, LOCK_LEVEL1, LOCK_LEVEL1);
+ writel(l1, &ipctl->lev1->conf[pin]);
+ rzn1_hw_set_lock(ipctl, LOCK_LEVEL1, 0);
+ }
+
+ return 0;
+}
+
+static int rzn1_pinconf_group_get(struct pinctrl_dev *pctldev,
+ unsigned int selector,
+ unsigned long *config)
+{
+ struct rzn1_pinctrl *ipctl = pinctrl_dev_get_drvdata(pctldev);
+ struct rzn1_pin_group *grp = &ipctl->groups[selector];
+ unsigned long old = 0;
+ unsigned int i;
+
+ dev_dbg(ipctl->dev, "group get %s selector:%u\n", grp->name, selector);
+
+ for (i = 0; i < grp->npins; i++) {
+ if (rzn1_pinconf_get(pctldev, grp->pins[i], config))
+ return -ENOTSUPP;
+
+ /* configs do not match between two pins */
+ if (i && (old != *config))
+ return -ENOTSUPP;
+
+ old = *config;
+ }
+
+ return 0;
+}
+
+static int rzn1_pinconf_group_set(struct pinctrl_dev *pctldev,
+ unsigned int selector,
+ unsigned long *configs,
+ unsigned int num_configs)
+{
+ struct rzn1_pinctrl *ipctl = pinctrl_dev_get_drvdata(pctldev);
+ struct rzn1_pin_group *grp = &ipctl->groups[selector];
+ unsigned int i;
+ int ret;
+
+ dev_dbg(ipctl->dev, "group set %s selector:%u configs:%p/%d\n",
+ grp->name, selector, configs, num_configs);
+
+ for (i = 0; i < grp->npins; i++) {
+ unsigned int pin = grp->pins[i];
+
+ ret = rzn1_pinconf_set(pctldev, pin, configs, num_configs);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static const struct pinconf_ops rzn1_pinconf_ops = {
+ .is_generic = true,
+ .pin_config_get = rzn1_pinconf_get,
+ .pin_config_set = rzn1_pinconf_set,
+ .pin_config_group_get = rzn1_pinconf_group_get,
+ .pin_config_group_set = rzn1_pinconf_group_set,
+ .pin_config_config_dbg_show = pinconf_generic_dump_config,
+};
+
+static struct pinctrl_desc rzn1_pinctrl_desc = {
+ .pctlops = &rzn1_pctrl_ops,
+ .pmxops = &rzn1_pmx_ops,
+ .confops = &rzn1_pinconf_ops,
+ .owner = THIS_MODULE,
+};
+
+static int rzn1_pinctrl_parse_groups(struct device_node *np,
+ struct rzn1_pin_group *grp,
+ struct rzn1_pinctrl *ipctl)
+{
+ const __be32 *list;
+ unsigned int i;
+ int size;
+
+ dev_dbg(ipctl->dev, "%s: %s\n", __func__, np->name);
+
+ /* Initialise group */
+ grp->name = np->name;
+
+ /*
+ * The binding format is
+ * pinmux = <PIN_FUNC_ID CONFIG ...>,
+ * do sanity check and calculate pins number
+ */
+ list = of_get_property(np, RZN1_PINS_PROP, &size);
+ if (!list) {
+ dev_err(ipctl->dev,
+ "no " RZN1_PINS_PROP " property in node %pOF\n", np);
+
+ return -EINVAL;
+ }
+
+ if (!size) {
+ dev_err(ipctl->dev, "Invalid " RZN1_PINS_PROP " in node %pOF\n",
+ np);
+
+ return -EINVAL;
+ }
+
+ grp->npins = size / sizeof(list[0]);
+ grp->pin_ids = devm_kmalloc_array(ipctl->dev,
+ grp->npins, sizeof(grp->pin_ids[0]),
+ GFP_KERNEL);
+ grp->pins = devm_kmalloc_array(ipctl->dev,
+ grp->npins, sizeof(grp->pins[0]),
+ GFP_KERNEL);
+ if (!grp->pin_ids || !grp->pins)
+ return -ENOMEM;
+
+ for (i = 0; i < grp->npins; i++) {
+ u32 pin_id = be32_to_cpu(*list++);
+
+ grp->pins[i] = pin_id & 0xff;
+ grp->pin_ids[i] = (pin_id >> 8) & 0x7f;
+ }
+
+ return grp->npins;
+}
+
+static int rzn1_pinctrl_count_function_groups(struct device_node *np)
+{
+ struct device_node *child;
+ int count = 0;
+
+ if (of_property_count_u32_elems(np, RZN1_PINS_PROP) > 0)
+ count++;
+
+ for_each_child_of_node(np, child) {
+ if (of_property_count_u32_elems(child, RZN1_PINS_PROP) > 0)
+ count++;
+ }
+
+ return count;
+}
+
+static int rzn1_pinctrl_parse_functions(struct device_node *np,
+ struct rzn1_pinctrl *ipctl,
+ unsigned int index)
+{
+ struct rzn1_pmx_func *func;
+ struct rzn1_pin_group *grp;
+ struct device_node *child;
+ unsigned int i = 0;
+ int ret;
+
+ func = &ipctl->functions[index];
+
+ /* Initialise function */
+ func->name = np->name;
+ func->num_groups = rzn1_pinctrl_count_function_groups(np);
+ if (func->num_groups == 0) {
+ dev_err(ipctl->dev, "no groups defined in %pOF\n", np);
+ return -EINVAL;
+ }
+ dev_dbg(ipctl->dev, "function %s has %d groups\n",
+ np->name, func->num_groups);
+
+ func->groups = devm_kmalloc_array(ipctl->dev,
+ func->num_groups, sizeof(char *),
+ GFP_KERNEL);
+ if (!func->groups)
+ return -ENOMEM;
+
+ if (of_property_count_u32_elems(np, RZN1_PINS_PROP) > 0) {
+ func->groups[i] = np->name;
+ grp = &ipctl->groups[ipctl->ngroups];
+ grp->func = func->name;
+ ret = rzn1_pinctrl_parse_groups(np, grp, ipctl);
+ if (ret < 0)
+ return ret;
+ i++;
+ ipctl->ngroups++;
+ }
+
+ for_each_child_of_node(np, child) {
+ func->groups[i] = child->name;
+ grp = &ipctl->groups[ipctl->ngroups];
+ grp->func = func->name;
+ ret = rzn1_pinctrl_parse_groups(child, grp, ipctl);
+ if (ret < 0)
+ return ret;
+ i++;
+ ipctl->ngroups++;
+ }
+
+ dev_dbg(ipctl->dev, "function %s parsed %u/%u groups\n",
+ np->name, i, func->num_groups);
+
+ return 0;
+}
+
+static int rzn1_pinctrl_probe_dt(struct platform_device *pdev,
+ struct rzn1_pinctrl *ipctl)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct device_node *child;
+ unsigned int maxgroups = 0;
+ unsigned int nfuncs = 0;
+ unsigned int i = 0;
+ int ret;
+
+ nfuncs = of_get_child_count(np);
+ if (nfuncs <= 0)
+ return 0;
+
+ ipctl->nfunctions = nfuncs;
+ ipctl->functions = devm_kmalloc_array(&pdev->dev, nfuncs,
+ sizeof(*ipctl->functions),
+ GFP_KERNEL);
+ if (!ipctl->functions)
+ return -ENOMEM;
+
+ ipctl->ngroups = 0;
+ for_each_child_of_node(np, child)
+ maxgroups += rzn1_pinctrl_count_function_groups(child);
+
+ ipctl->groups = devm_kmalloc_array(&pdev->dev,
+ maxgroups,
+ sizeof(*ipctl->groups),
+ GFP_KERNEL);
+ if (!ipctl->groups)
+ return -ENOMEM;
+
+ for_each_child_of_node(np, child) {
+ ret = rzn1_pinctrl_parse_functions(child, ipctl, i++);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int rzn1_pinctrl_probe(struct platform_device *pdev)
+{
+ struct rzn1_pinctrl *ipctl;
+ struct resource *res;
+ int ret;
+
+ /* Create state holders etc for this driver */
+ ipctl = devm_kzalloc(&pdev->dev, sizeof(*ipctl), GFP_KERNEL);
+ if (!ipctl)
+ return -ENOMEM;
+
+ ipctl->mdio_func[0] = -1;
+ ipctl->mdio_func[1] = -1;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ ipctl->lev1_protect_phys = (u32)res->start + 0x400;
+ ipctl->lev1 = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(ipctl->lev1))
+ return PTR_ERR(ipctl->lev1);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+ ipctl->lev2_protect_phys = (u32)res->start + 0x400;
+ ipctl->lev2 = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(ipctl->lev2))
+ return PTR_ERR(ipctl->lev2);
+
+ ipctl->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(ipctl->clk))
+ return PTR_ERR(ipctl->clk);
+ ret = clk_prepare_enable(ipctl->clk);
+ if (ret)
+ return ret;
+
+ ipctl->dev = &pdev->dev;
+ rzn1_pinctrl_desc.name = dev_name(&pdev->dev);
+ rzn1_pinctrl_desc.pins = rzn1_pins;
+ rzn1_pinctrl_desc.npins = ARRAY_SIZE(rzn1_pins);
+
+ ret = rzn1_pinctrl_probe_dt(pdev, ipctl);
+ if (ret) {
+ dev_err(&pdev->dev, "fail to probe dt properties\n");
+ goto err_clk;
+ }
+
+ platform_set_drvdata(pdev, ipctl);
+
+ ret = devm_pinctrl_register_and_init(&pdev->dev, &rzn1_pinctrl_desc,
+ ipctl, &ipctl->pctl);
+ if (ret) {
+ dev_err(&pdev->dev, "could not register rzn1 pinctrl driver\n");
+ goto err_clk;
+ }
+
+ ret = pinctrl_enable(ipctl->pctl);
+ if (ret)
+ goto err_clk;
+
+ dev_info(&pdev->dev, "probed\n");
+
+ return 0;
+
+err_clk:
+ clk_disable_unprepare(ipctl->clk);
+
+ return ret;
+}
+
+static int rzn1_pinctrl_remove(struct platform_device *pdev)
+{
+ struct rzn1_pinctrl *ipctl = platform_get_drvdata(pdev);
+
+ clk_disable_unprepare(ipctl->clk);
+
+ return 0;
+}
+
+static const struct of_device_id rzn1_pinctrl_match[] = {
+ { .compatible = "renesas,rzn1-pinctrl", },
+ {}
+};
+MODULE_DEVICE_TABLE(of, rzn1_pinctrl_match);
+
+static struct platform_driver rzn1_pinctrl_driver = {
+ .probe = rzn1_pinctrl_probe,
+ .remove = rzn1_pinctrl_remove,
+ .driver = {
+ .name = "rzn1-pinctrl",
+ .of_match_table = rzn1_pinctrl_match,
+ },
+};
+
+static int __init _pinctrl_drv_register(void)
+{
+ return platform_driver_register(&rzn1_pinctrl_driver);
+}
+subsys_initcall(_pinctrl_drv_register);
+
+MODULE_AUTHOR("Phil Edworthy <phil.edworthy@renesas.com>");
+MODULE_DESCRIPTION("Renesas RZ/N1 pinctrl driver");
+MODULE_LICENSE("GPL v2");
vals[found].reg = pcs->base + offset;
vals[found].val = pinctrl_spec.args[1];
- dev_dbg(pcs->dev, "%s index: 0x%x value: 0x%x\n",
- pinctrl_spec.np->name, offset, pinctrl_spec.args[1]);
+ dev_dbg(pcs->dev, "%pOFn index: 0x%x value: 0x%x\n",
+ pinctrl_spec.np, offset, pinctrl_spec.args[1]);
pin = pcs_get_pin_by_offset(pcs, offset);
if (pin < 0) {
dev_err(pcs->dev,
- "could not add functions for %s %ux\n",
- np->name, offset);
+ "could not add functions for %pOFn %ux\n",
+ np, offset);
break;
}
pins[found++] = pin;
val = pinctrl_spec.args[1];
mask = pinctrl_spec.args[2];
- dev_dbg(pcs->dev, "%s index: 0x%x value: 0x%x mask: 0x%x\n",
- pinctrl_spec.np->name, offset, val, mask);
+ dev_dbg(pcs->dev, "%pOFn index: 0x%x value: 0x%x mask: 0x%x\n",
+ pinctrl_spec.np, offset, val, mask);
/* Parse pins in each row from LSB */
while (mask) {
if ((mask & mask_pos) == 0) {
dev_err(pcs->dev,
- "Invalid mask for %s at 0x%x\n",
- np->name, offset);
+ "Invalid mask for %pOFn at 0x%x\n",
+ np, offset);
break;
}
if (submask != mask_pos) {
dev_warn(pcs->dev,
- "Invalid submask 0x%x for %s at 0x%x\n",
- submask, np->name, offset);
+ "Invalid submask 0x%x for %pOFn at 0x%x\n",
+ submask, np, offset);
continue;
}
pin = pcs_get_pin_by_offset(pcs, offset);
if (pin < 0) {
dev_err(pcs->dev,
- "could not add functions for %s %ux\n",
- np->name, offset);
+ "could not add functions for %pOFn %ux\n",
+ np, offset);
break;
}
pins[found++] = pin + pin_num_from_lsb;
ret = pcs_parse_bits_in_pinctrl_entry(pcs, np_config, map,
num_maps, pgnames);
if (ret < 0) {
- dev_err(pcs->dev, "no pins entries for %s\n",
- np_config->name);
+ dev_err(pcs->dev, "no pins entries for %pOFn\n",
+ np_config);
goto free_pgnames;
}
} else {
ret = pcs_parse_one_pinctrl_entry(pcs, np_config, map,
num_maps, pgnames);
if (ret < 0) {
- dev_err(pcs->dev, "no pins entries for %s\n",
- np_config->name);
+ dev_err(pcs->dev, "no pins entries for %pOFn\n",
+ np_config);
goto free_pgnames;
}
}
grp = st_pctl_find_group_by_name(info, np->name);
if (!grp) {
- dev_err(info->dev, "unable to find group for node %s\n",
- np->name);
+ dev_err(info->dev, "unable to find group for node %pOFn\n",
+ np);
return -EINVAL;
}
if (pp->length / sizeof(__be32) >= OF_GPIO_ARGS_MIN) {
npins++;
} else {
- pr_warn("Invalid st,pins in %s node\n", np->name);
+ pr_warn("Invalid st,pins in %pOFn node\n", np);
return -EINVAL;
}
}
This is the pinctrl, pinmux, pinconf and gpiolib driver for the
Qualcomm TLMM block found in the Qualcomm MSM8998 platform.
+config PINCTRL_QCS404
+ tristate "Qualcomm QCS404 pin controller driver"
+ depends on GPIOLIB && OF
+ select PINCTRL_MSM
+ help
+ This is the pinctrl, pinmux, pinconf and gpiolib driver for the
+ TLMM block found in the Qualcomm QCS404 platform.
+
config PINCTRL_QDF2XXX
tristate "Qualcomm Technologies QDF2xxx pin controller driver"
depends on GPIOLIB && ACPI
which are using SSBI for communication with SoC. Example PMIC's
devices are pm8058 and pm8921.
+config PINCTRL_SDM660
+ tristate "Qualcomm Technologies Inc SDM660 pin controller driver"
+ depends on GPIOLIB && OF
+ select PINCTRL_MSM
+ help
+ This is the pinctrl, pinmux, pinconf and gpiolib driver for the
+ Qualcomm Technologies Inc TLMM block found on the Qualcomm
+ Technologies Inc SDM660 platform.
+
config PINCTRL_SDM845
tristate "Qualcomm Technologies Inc SDM845 pin controller driver"
depends on GPIOLIB && OF
obj-$(CONFIG_PINCTRL_MSM8994) += pinctrl-msm8994.o
obj-$(CONFIG_PINCTRL_MSM8996) += pinctrl-msm8996.o
obj-$(CONFIG_PINCTRL_MSM8998) += pinctrl-msm8998.o
+obj-$(CONFIG_PINCTRL_QCS404) += pinctrl-qcs404.o
obj-$(CONFIG_PINCTRL_QDF2XXX) += pinctrl-qdf2xxx.o
obj-$(CONFIG_PINCTRL_MDM9615) += pinctrl-mdm9615.o
obj-$(CONFIG_PINCTRL_QCOM_SPMI_PMIC) += pinctrl-spmi-gpio.o
obj-$(CONFIG_PINCTRL_QCOM_SPMI_PMIC) += pinctrl-spmi-mpp.o
obj-$(CONFIG_PINCTRL_QCOM_SSBI_PMIC) += pinctrl-ssbi-gpio.o
obj-$(CONFIG_PINCTRL_QCOM_SSBI_PMIC) += pinctrl-ssbi-mpp.o
+obj-$(CONFIG_PINCTRL_SDM660) += pinctrl-sdm660.o
obj-$(CONFIG_PINCTRL_SDM845) += pinctrl-sdm845.o
#include <linux/pinctrl/pinconf.h>
#include <linux/pinctrl/pinconf-generic.h>
#include <linux/slab.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/reboot.h>
#include "../pinctrl-utils.h"
#define MAX_NR_GPIO 300
+#define MAX_NR_TILES 4
#define PS_HOLD_OFFSET 0x820
/**
* @dual_edge_irqs: Bitmap of irqs that need sw emulated dual edge
* detection.
* @soc; Reference to soc_data of platform specific data.
- * @regs: Base address for the TLMM register map.
+ * @regs: Base addresses for the TLMM tiles.
*/
struct msm_pinctrl {
struct device *dev;
DECLARE_BITMAP(enabled_irqs, MAX_NR_GPIO);
const struct msm_pinctrl_soc_data *soc;
- void __iomem *regs;
+ void __iomem *regs[MAX_NR_TILES];
};
+#define MSM_ACCESSOR(name) \
+static u32 msm_readl_##name(struct msm_pinctrl *pctrl, \
+ const struct msm_pingroup *g) \
+{ \
+ return readl(pctrl->regs[g->tile] + g->name##_reg); \
+} \
+static void msm_writel_##name(u32 val, struct msm_pinctrl *pctrl, \
+ const struct msm_pingroup *g) \
+{ \
+ writel(val, pctrl->regs[g->tile] + g->name##_reg); \
+}
+
+MSM_ACCESSOR(ctl)
+MSM_ACCESSOR(io)
+MSM_ACCESSOR(intr_cfg)
+MSM_ACCESSOR(intr_status)
+MSM_ACCESSOR(intr_target)
+
static int msm_get_groups_count(struct pinctrl_dev *pctldev)
{
struct msm_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
raw_spin_lock_irqsave(&pctrl->lock, flags);
- val = readl(pctrl->regs + g->ctl_reg);
+ val = msm_readl_ctl(pctrl, g);
val &= ~mask;
val |= i << g->mux_bit;
- writel(val, pctrl->regs + g->ctl_reg);
+ msm_writel_ctl(val, pctrl, g);
raw_spin_unlock_irqrestore(&pctrl->lock, flags);
return 0;
}
+static int msm_pinmux_request_gpio(struct pinctrl_dev *pctldev,
+ struct pinctrl_gpio_range *range,
+ unsigned offset)
+{
+ struct msm_pinctrl *pctrl = pinctrl_dev_get_drvdata(pctldev);
+ const struct msm_pingroup *g = &pctrl->soc->groups[offset];
+
+ /* No funcs? Probably ACPI so can't do anything here */
+ if (!g->nfuncs)
+ return 0;
+
+ /* For now assume function 0 is GPIO because it always is */
+ return msm_pinmux_set_mux(pctldev, g->funcs[0], offset);
+}
+
static const struct pinmux_ops msm_pinmux_ops = {
.request = msm_pinmux_request,
.get_functions_count = msm_get_functions_count,
.get_function_name = msm_get_function_name,
.get_function_groups = msm_get_function_groups,
+ .gpio_request_enable = msm_pinmux_request_gpio,
.set_mux = msm_pinmux_set_mux,
};
if (ret < 0)
return ret;
- val = readl(pctrl->regs + g->ctl_reg);
+ val = msm_readl_ctl(pctrl, g);
arg = (val >> bit) & mask;
/* Convert register value to pinconf value */
if (!arg)
return -EINVAL;
- val = readl(pctrl->regs + g->io_reg);
+ val = msm_readl_io(pctrl, g);
arg = !!(val & BIT(g->in_bit));
break;
case PIN_CONFIG_INPUT_ENABLE:
case PIN_CONFIG_OUTPUT:
/* set output value */
raw_spin_lock_irqsave(&pctrl->lock, flags);
- val = readl(pctrl->regs + g->io_reg);
+ val = msm_readl_io(pctrl, g);
if (arg)
val |= BIT(g->out_bit);
else
val &= ~BIT(g->out_bit);
- writel(val, pctrl->regs + g->io_reg);
+ msm_writel_io(val, pctrl, g);
raw_spin_unlock_irqrestore(&pctrl->lock, flags);
/* enable output */
}
raw_spin_lock_irqsave(&pctrl->lock, flags);
- val = readl(pctrl->regs + g->ctl_reg);
+ val = msm_readl_ctl(pctrl, g);
val &= ~(mask << bit);
val |= arg << bit;
- writel(val, pctrl->regs + g->ctl_reg);
+ msm_writel_ctl(val, pctrl, g);
raw_spin_unlock_irqrestore(&pctrl->lock, flags);
}
raw_spin_lock_irqsave(&pctrl->lock, flags);
- val = readl(pctrl->regs + g->ctl_reg);
+ val = msm_readl_ctl(pctrl, g);
val &= ~BIT(g->oe_bit);
- writel(val, pctrl->regs + g->ctl_reg);
+ msm_writel_ctl(val, pctrl, g);
raw_spin_unlock_irqrestore(&pctrl->lock, flags);
raw_spin_lock_irqsave(&pctrl->lock, flags);
- val = readl(pctrl->regs + g->io_reg);
+ val = msm_readl_io(pctrl, g);
if (value)
val |= BIT(g->out_bit);
else
val &= ~BIT(g->out_bit);
- writel(val, pctrl->regs + g->io_reg);
+ msm_writel_io(val, pctrl, g);
- val = readl(pctrl->regs + g->ctl_reg);
+ val = msm_readl_ctl(pctrl, g);
val |= BIT(g->oe_bit);
- writel(val, pctrl->regs + g->ctl_reg);
+ msm_writel_ctl(val, pctrl, g);
raw_spin_unlock_irqrestore(&pctrl->lock, flags);
g = &pctrl->soc->groups[offset];
- val = readl(pctrl->regs + g->ctl_reg);
+ val = msm_readl_ctl(pctrl, g);
/* 0 = output, 1 = input */
return val & BIT(g->oe_bit) ? 0 : 1;
g = &pctrl->soc->groups[offset];
- val = readl(pctrl->regs + g->io_reg);
+ val = msm_readl_io(pctrl, g);
return !!(val & BIT(g->in_bit));
}
raw_spin_lock_irqsave(&pctrl->lock, flags);
- val = readl(pctrl->regs + g->io_reg);
+ val = msm_readl_io(pctrl, g);
if (value)
val |= BIT(g->out_bit);
else
val &= ~BIT(g->out_bit);
- writel(val, pctrl->regs + g->io_reg);
+ msm_writel_io(val, pctrl, g);
raw_spin_unlock_irqrestore(&pctrl->lock, flags);
}
return;
g = &pctrl->soc->groups[offset];
- ctl_reg = readl(pctrl->regs + g->ctl_reg);
- io_reg = readl(pctrl->regs + g->io_reg);
+ ctl_reg = msm_readl_ctl(pctrl, g);
+ io_reg = msm_readl_io(pctrl, g);
is_out = !!(ctl_reg & BIT(g->oe_bit));
func = (ctl_reg >> g->mux_bit) & 7;
#define msm_gpio_dbg_show NULL
#endif
+static int msm_gpio_init_valid_mask(struct gpio_chip *chip)
+{
+ struct msm_pinctrl *pctrl = gpiochip_get_data(chip);
+ int ret;
+ unsigned int len, i;
+ unsigned int max_gpios = pctrl->soc->ngpios;
+ u16 *tmp;
+
+ /* The number of GPIOs in the ACPI tables */
+ len = ret = device_property_read_u16_array(pctrl->dev, "gpios", NULL,
+ 0);
+ if (ret < 0)
+ return 0;
+
+ if (ret > max_gpios)
+ return -EINVAL;
+
+ tmp = kmalloc_array(len, sizeof(*tmp), GFP_KERNEL);
+ if (!tmp)
+ return -ENOMEM;
+
+ ret = device_property_read_u16_array(pctrl->dev, "gpios", tmp, len);
+ if (ret < 0) {
+ dev_err(pctrl->dev, "could not read list of GPIOs\n");
+ goto out;
+ }
+
+ bitmap_zero(chip->valid_mask, max_gpios);
+ for (i = 0; i < len; i++)
+ set_bit(tmp[i], chip->valid_mask);
+
+out:
+ kfree(tmp);
+ return ret;
+}
+
static const struct gpio_chip msm_gpio_template = {
.direction_input = msm_gpio_direction_input,
.direction_output = msm_gpio_direction_output,
.request = gpiochip_generic_request,
.free = gpiochip_generic_free,
.dbg_show = msm_gpio_dbg_show,
+ .init_valid_mask = msm_gpio_init_valid_mask,
};
/* For dual-edge interrupts in software, since some hardware has no
unsigned pol;
do {
- val = readl(pctrl->regs + g->io_reg) & BIT(g->in_bit);
+ val = msm_readl_io(pctrl, g) & BIT(g->in_bit);
- pol = readl(pctrl->regs + g->intr_cfg_reg);
+ pol = msm_readl_intr_cfg(pctrl, g);
pol ^= BIT(g->intr_polarity_bit);
- writel(pol, pctrl->regs + g->intr_cfg_reg);
+ msm_writel_intr_cfg(val, pctrl, g);
- val2 = readl(pctrl->regs + g->io_reg) & BIT(g->in_bit);
- intstat = readl(pctrl->regs + g->intr_status_reg);
+ val2 = msm_readl_io(pctrl, g) & BIT(g->in_bit);
+ intstat = msm_readl_intr_status(pctrl, g);
if (intstat || (val == val2))
return;
} while (loop_limit-- > 0);
raw_spin_lock_irqsave(&pctrl->lock, flags);
- val = readl(pctrl->regs + g->intr_cfg_reg);
+ val = msm_readl_intr_cfg(pctrl, g);
/*
* There are two bits that control interrupt forwarding to the CPU. The
* RAW_STATUS_EN bit causes the level or edge sensed on the line to be
val &= ~BIT(g->intr_raw_status_bit);
val &= ~BIT(g->intr_enable_bit);
- writel(val, pctrl->regs + g->intr_cfg_reg);
+ msm_writel_intr_cfg(val, pctrl, g);
clear_bit(d->hwirq, pctrl->enabled_irqs);
raw_spin_lock_irqsave(&pctrl->lock, flags);
- val = readl(pctrl->regs + g->intr_cfg_reg);
+ val = msm_readl_intr_cfg(pctrl, g);
val |= BIT(g->intr_raw_status_bit);
val |= BIT(g->intr_enable_bit);
- writel(val, pctrl->regs + g->intr_cfg_reg);
+ msm_writel_intr_cfg(val, pctrl, g);
set_bit(d->hwirq, pctrl->enabled_irqs);
raw_spin_lock_irqsave(&pctrl->lock, flags);
- val = readl(pctrl->regs + g->intr_status_reg);
+ val = msm_readl_intr_status(pctrl, g);
if (g->intr_ack_high)
val |= BIT(g->intr_status_bit);
else
val &= ~BIT(g->intr_status_bit);
- writel(val, pctrl->regs + g->intr_status_reg);
+ msm_writel_intr_status(val, pctrl, g);
if (test_bit(d->hwirq, pctrl->dual_edge_irqs))
msm_gpio_update_dual_edge_pos(pctrl, g, d);
clear_bit(d->hwirq, pctrl->dual_edge_irqs);
/* Route interrupts to application cpu */
- val = readl(pctrl->regs + g->intr_target_reg);
+ val = msm_readl_intr_target(pctrl, g);
val &= ~(7 << g->intr_target_bit);
val |= g->intr_target_kpss_val << g->intr_target_bit;
- writel(val, pctrl->regs + g->intr_target_reg);
+ msm_writel_intr_target(val, pctrl, g);
/* Update configuration for gpio.
* RAW_STATUS_EN is left on for all gpio irqs. Due to the
* internal circuitry of TLMM, toggling the RAW_STATUS
* could cause the INTR_STATUS to be set for EDGE interrupts.
*/
- val = readl(pctrl->regs + g->intr_cfg_reg);
+ val = msm_readl_intr_cfg(pctrl, g);
val |= BIT(g->intr_raw_status_bit);
if (g->intr_detection_width == 2) {
val &= ~(3 << g->intr_detection_bit);
} else {
BUG();
}
- writel(val, pctrl->regs + g->intr_cfg_reg);
+ msm_writel_intr_cfg(val, pctrl, g);
if (test_bit(d->hwirq, pctrl->dual_edge_irqs))
msm_gpio_update_dual_edge_pos(pctrl, g, d);
return 0;
}
+static int msm_gpio_irq_reqres(struct irq_data *d)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct msm_pinctrl *pctrl = gpiochip_get_data(gc);
+ int ret;
+
+ if (!try_module_get(gc->owner))
+ return -ENODEV;
+
+ ret = msm_pinmux_request_gpio(pctrl->pctrl, NULL, d->hwirq);
+ if (ret)
+ goto out;
+ msm_gpio_direction_input(gc, d->hwirq);
+
+ if (gpiochip_lock_as_irq(gc, d->hwirq)) {
+ dev_err(gc->parent,
+ "unable to lock HW IRQ %lu for IRQ\n",
+ d->hwirq);
+ ret = -EINVAL;
+ goto out;
+ }
+ return 0;
+out:
+ module_put(gc->owner);
+ return ret;
+}
+
+static void msm_gpio_irq_relres(struct irq_data *d)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+
+ gpiochip_unlock_as_irq(gc, d->hwirq);
+ module_put(gc->owner);
+}
+
static void msm_gpio_irq_handler(struct irq_desc *desc)
{
struct gpio_chip *gc = irq_desc_get_handler_data(desc);
*/
for_each_set_bit(i, pctrl->enabled_irqs, pctrl->chip.ngpio) {
g = &pctrl->soc->groups[i];
- val = readl(pctrl->regs + g->intr_status_reg);
+ val = msm_readl_intr_status(pctrl, g);
if (val & BIT(g->intr_status_bit)) {
irq_pin = irq_find_mapping(gc->irq.domain, i);
generic_handle_irq(irq_pin);
chained_irq_exit(chip, desc);
}
-static int msm_gpio_init_valid_mask(struct gpio_chip *chip,
- struct msm_pinctrl *pctrl)
-{
- int ret;
- unsigned int len, i;
- unsigned int max_gpios = pctrl->soc->ngpios;
- u16 *tmp;
-
- /* The number of GPIOs in the ACPI tables */
- len = ret = device_property_read_u16_array(pctrl->dev, "gpios", NULL, 0);
- if (ret < 0)
- return 0;
-
- if (ret > max_gpios)
- return -EINVAL;
-
- tmp = kmalloc_array(len, sizeof(*tmp), GFP_KERNEL);
- if (!tmp)
- return -ENOMEM;
-
- ret = device_property_read_u16_array(pctrl->dev, "gpios", tmp, len);
- if (ret < 0) {
- dev_err(pctrl->dev, "could not read list of GPIOs\n");
- goto out;
- }
-
- bitmap_zero(chip->valid_mask, max_gpios);
- for (i = 0; i < len; i++)
- set_bit(tmp[i], chip->valid_mask);
-
-out:
- kfree(tmp);
- return ret;
-}
-
static bool msm_gpio_needs_valid_mask(struct msm_pinctrl *pctrl)
{
return device_property_read_u16_array(pctrl->dev, "gpios", NULL, 0) > 0;
pctrl->irq_chip.irq_ack = msm_gpio_irq_ack;
pctrl->irq_chip.irq_set_type = msm_gpio_irq_set_type;
pctrl->irq_chip.irq_set_wake = msm_gpio_irq_set_wake;
+ pctrl->irq_chip.irq_request_resources = msm_gpio_irq_reqres;
+ pctrl->irq_chip.irq_release_resources = msm_gpio_irq_relres;
ret = gpiochip_add_data(&pctrl->chip, pctrl);
if (ret) {
return ret;
}
- ret = msm_gpio_init_valid_mask(chip, pctrl);
- if (ret) {
- dev_err(pctrl->dev, "Failed to setup irq valid bits\n");
- gpiochip_remove(&pctrl->chip);
- return ret;
- }
-
/*
* For DeviceTree-supported systems, the gpio core checks the
* pinctrl's device node for the "gpio-ranges" property.
{
struct msm_pinctrl *pctrl = container_of(nb, struct msm_pinctrl, restart_nb);
- writel(0, pctrl->regs + PS_HOLD_OFFSET);
+ writel(0, pctrl->regs[0] + PS_HOLD_OFFSET);
mdelay(1000);
return NOTIFY_DONE;
}
struct msm_pinctrl *pctrl;
struct resource *res;
int ret;
+ int i;
pctrl = devm_kzalloc(&pdev->dev, sizeof(*pctrl), GFP_KERNEL);
if (!pctrl)
raw_spin_lock_init(&pctrl->lock);
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- pctrl->regs = devm_ioremap_resource(&pdev->dev, res);
- if (IS_ERR(pctrl->regs))
- return PTR_ERR(pctrl->regs);
+ if (soc_data->tiles) {
+ for (i = 0; i < soc_data->ntiles; i++) {
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
+ soc_data->tiles[i]);
+ pctrl->regs[i] = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(pctrl->regs[i]))
+ return PTR_ERR(pctrl->regs[i]);
+ }
+ } else {
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ pctrl->regs[0] = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(pctrl->regs[0]))
+ return PTR_ERR(pctrl->regs[0]);
+ }
msm_pinctrl_setup_pm_reset(pctrl);
u32 intr_status_reg;
u32 intr_target_reg;
+ unsigned int tile:2;
+
unsigned mux_bit:5;
unsigned pull_bit:5;
unsigned ngroups;
unsigned ngpios;
bool pull_no_keeper;
+ const char *const *tiles;
+ unsigned int ntiles;
};
int msm_pinctrl_probe(struct platform_device *pdev,
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018, The Linux Foundation. All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/pinctrl/pinctrl.h>
+
+#include "pinctrl-msm.h"
+
+static const char * const qcs404_tiles[] = {
+ "north",
+ "south",
+ "east"
+};
+
+enum {
+ NORTH,
+ SOUTH,
+ EAST
+};
+
+#define FUNCTION(fname) \
+ [msm_mux_##fname] = { \
+ .name = #fname, \
+ .groups = fname##_groups, \
+ .ngroups = ARRAY_SIZE(fname##_groups), \
+ }
+
+#define PINGROUP(id, _tile, f1, f2, f3, f4, f5, f6, f7, f8, f9) \
+ { \
+ .name = "gpio" #id, \
+ .pins = gpio##id##_pins, \
+ .npins = (unsigned int)ARRAY_SIZE(gpio##id##_pins), \
+ .funcs = (int[]){ \
+ msm_mux_gpio, /* gpio mode */ \
+ msm_mux_##f1, \
+ msm_mux_##f2, \
+ msm_mux_##f3, \
+ msm_mux_##f4, \
+ msm_mux_##f5, \
+ msm_mux_##f6, \
+ msm_mux_##f7, \
+ msm_mux_##f8, \
+ msm_mux_##f9 \
+ }, \
+ .nfuncs = 10, \
+ .ctl_reg = 0x1000 * id, \
+ .io_reg = 0x1000 * id + 0x4, \
+ .intr_cfg_reg = 0x1000 * id + 0x8, \
+ .intr_status_reg = 0x1000 * id + 0xc, \
+ .intr_target_reg = 0x1000 * id + 0x8, \
+ .tile = _tile, \
+ .mux_bit = 2, \
+ .pull_bit = 0, \
+ .drv_bit = 6, \
+ .oe_bit = 9, \
+ .in_bit = 0, \
+ .out_bit = 1, \
+ .intr_enable_bit = 0, \
+ .intr_status_bit = 0, \
+ .intr_target_bit = 5, \
+ .intr_target_kpss_val = 4, \
+ .intr_raw_status_bit = 4, \
+ .intr_polarity_bit = 1, \
+ .intr_detection_bit = 2, \
+ .intr_detection_width = 2, \
+ }
+
+#define SDC_QDSD_PINGROUP(pg_name, ctl, pull, drv) \
+ { \
+ .name = #pg_name, \
+ .pins = pg_name##_pins, \
+ .npins = (unsigned int)ARRAY_SIZE(pg_name##_pins), \
+ .ctl_reg = ctl, \
+ .io_reg = 0, \
+ .intr_cfg_reg = 0, \
+ .intr_status_reg = 0, \
+ .intr_target_reg = 0, \
+ .tile = NORTH, \
+ .mux_bit = -1, \
+ .pull_bit = pull, \
+ .drv_bit = drv, \
+ .oe_bit = -1, \
+ .in_bit = -1, \
+ .out_bit = -1, \
+ .intr_enable_bit = -1, \
+ .intr_status_bit = -1, \
+ .intr_target_bit = -1, \
+ .intr_raw_status_bit = -1, \
+ .intr_polarity_bit = -1, \
+ .intr_detection_bit = -1, \
+ .intr_detection_width = -1, \
+ }
+
+#define UFS_RESET(pg_name, offset) \
+ { \
+ .name = #pg_name, \
+ .pins = pg_name##_pins, \
+ .npins = (unsigned int)ARRAY_SIZE(pg_name##_pins), \
+ .ctl_reg = offset, \
+ .io_reg = offset + 0x4, \
+ .intr_cfg_reg = 0, \
+ .intr_status_reg = 0, \
+ .intr_target_reg = 0, \
+ .tile = NORTH, \
+ .mux_bit = -1, \
+ .pull_bit = 3, \
+ .drv_bit = 0, \
+ .oe_bit = -1, \
+ .in_bit = -1, \
+ .out_bit = 0, \
+ .intr_enable_bit = -1, \
+ .intr_status_bit = -1, \
+ .intr_target_bit = -1, \
+ .intr_raw_status_bit = -1, \
+ .intr_polarity_bit = -1, \
+ .intr_detection_bit = -1, \
+ .intr_detection_width = -1, \
+ }
+static const struct pinctrl_pin_desc qcs404_pins[] = {
+ PINCTRL_PIN(0, "GPIO_0"),
+ PINCTRL_PIN(1, "GPIO_1"),
+ PINCTRL_PIN(2, "GPIO_2"),
+ PINCTRL_PIN(3, "GPIO_3"),
+ PINCTRL_PIN(4, "GPIO_4"),
+ PINCTRL_PIN(5, "GPIO_5"),
+ PINCTRL_PIN(6, "GPIO_6"),
+ PINCTRL_PIN(7, "GPIO_7"),
+ PINCTRL_PIN(8, "GPIO_8"),
+ PINCTRL_PIN(9, "GPIO_9"),
+ PINCTRL_PIN(10, "GPIO_10"),
+ PINCTRL_PIN(11, "GPIO_11"),
+ PINCTRL_PIN(12, "GPIO_12"),
+ PINCTRL_PIN(13, "GPIO_13"),
+ PINCTRL_PIN(14, "GPIO_14"),
+ PINCTRL_PIN(15, "GPIO_15"),
+ PINCTRL_PIN(16, "GPIO_16"),
+ PINCTRL_PIN(17, "GPIO_17"),
+ PINCTRL_PIN(18, "GPIO_18"),
+ PINCTRL_PIN(19, "GPIO_19"),
+ PINCTRL_PIN(20, "GPIO_20"),
+ PINCTRL_PIN(21, "GPIO_21"),
+ PINCTRL_PIN(22, "GPIO_22"),
+ PINCTRL_PIN(23, "GPIO_23"),
+ PINCTRL_PIN(24, "GPIO_24"),
+ PINCTRL_PIN(25, "GPIO_25"),
+ PINCTRL_PIN(26, "GPIO_26"),
+ PINCTRL_PIN(27, "GPIO_27"),
+ PINCTRL_PIN(28, "GPIO_28"),
+ PINCTRL_PIN(29, "GPIO_29"),
+ PINCTRL_PIN(30, "GPIO_30"),
+ PINCTRL_PIN(31, "GPIO_31"),
+ PINCTRL_PIN(32, "GPIO_32"),
+ PINCTRL_PIN(33, "GPIO_33"),
+ PINCTRL_PIN(34, "GPIO_34"),
+ PINCTRL_PIN(35, "GPIO_35"),
+ PINCTRL_PIN(36, "GPIO_36"),
+ PINCTRL_PIN(37, "GPIO_37"),
+ PINCTRL_PIN(38, "GPIO_38"),
+ PINCTRL_PIN(39, "GPIO_39"),
+ PINCTRL_PIN(40, "GPIO_40"),
+ PINCTRL_PIN(41, "GPIO_41"),
+ PINCTRL_PIN(42, "GPIO_42"),
+ PINCTRL_PIN(43, "GPIO_43"),
+ PINCTRL_PIN(44, "GPIO_44"),
+ PINCTRL_PIN(45, "GPIO_45"),
+ PINCTRL_PIN(46, "GPIO_46"),
+ PINCTRL_PIN(47, "GPIO_47"),
+ PINCTRL_PIN(48, "GPIO_48"),
+ PINCTRL_PIN(49, "GPIO_49"),
+ PINCTRL_PIN(50, "GPIO_50"),
+ PINCTRL_PIN(51, "GPIO_51"),
+ PINCTRL_PIN(52, "GPIO_52"),
+ PINCTRL_PIN(53, "GPIO_53"),
+ PINCTRL_PIN(54, "GPIO_54"),
+ PINCTRL_PIN(55, "GPIO_55"),
+ PINCTRL_PIN(56, "GPIO_56"),
+ PINCTRL_PIN(57, "GPIO_57"),
+ PINCTRL_PIN(58, "GPIO_58"),
+ PINCTRL_PIN(59, "GPIO_59"),
+ PINCTRL_PIN(60, "GPIO_60"),
+ PINCTRL_PIN(61, "GPIO_61"),
+ PINCTRL_PIN(62, "GPIO_62"),
+ PINCTRL_PIN(63, "GPIO_63"),
+ PINCTRL_PIN(64, "GPIO_64"),
+ PINCTRL_PIN(65, "GPIO_65"),
+ PINCTRL_PIN(66, "GPIO_66"),
+ PINCTRL_PIN(67, "GPIO_67"),
+ PINCTRL_PIN(68, "GPIO_68"),
+ PINCTRL_PIN(69, "GPIO_69"),
+ PINCTRL_PIN(70, "GPIO_70"),
+ PINCTRL_PIN(71, "GPIO_71"),
+ PINCTRL_PIN(72, "GPIO_72"),
+ PINCTRL_PIN(73, "GPIO_73"),
+ PINCTRL_PIN(74, "GPIO_74"),
+ PINCTRL_PIN(75, "GPIO_75"),
+ PINCTRL_PIN(76, "GPIO_76"),
+ PINCTRL_PIN(77, "GPIO_77"),
+ PINCTRL_PIN(78, "GPIO_78"),
+ PINCTRL_PIN(79, "GPIO_79"),
+ PINCTRL_PIN(80, "GPIO_80"),
+ PINCTRL_PIN(81, "GPIO_81"),
+ PINCTRL_PIN(82, "GPIO_82"),
+ PINCTRL_PIN(83, "GPIO_83"),
+ PINCTRL_PIN(84, "GPIO_84"),
+ PINCTRL_PIN(85, "GPIO_85"),
+ PINCTRL_PIN(86, "GPIO_86"),
+ PINCTRL_PIN(87, "GPIO_87"),
+ PINCTRL_PIN(88, "GPIO_88"),
+ PINCTRL_PIN(89, "GPIO_89"),
+ PINCTRL_PIN(90, "GPIO_90"),
+ PINCTRL_PIN(91, "GPIO_91"),
+ PINCTRL_PIN(92, "GPIO_92"),
+ PINCTRL_PIN(93, "GPIO_93"),
+ PINCTRL_PIN(94, "GPIO_94"),
+ PINCTRL_PIN(95, "GPIO_95"),
+ PINCTRL_PIN(96, "GPIO_96"),
+ PINCTRL_PIN(97, "GPIO_97"),
+ PINCTRL_PIN(98, "GPIO_98"),
+ PINCTRL_PIN(99, "GPIO_99"),
+ PINCTRL_PIN(100, "GPIO_100"),
+ PINCTRL_PIN(101, "GPIO_101"),
+ PINCTRL_PIN(102, "GPIO_102"),
+ PINCTRL_PIN(103, "GPIO_103"),
+ PINCTRL_PIN(104, "GPIO_104"),
+ PINCTRL_PIN(105, "GPIO_105"),
+ PINCTRL_PIN(106, "GPIO_106"),
+ PINCTRL_PIN(107, "GPIO_107"),
+ PINCTRL_PIN(108, "GPIO_108"),
+ PINCTRL_PIN(109, "GPIO_109"),
+ PINCTRL_PIN(110, "GPIO_110"),
+ PINCTRL_PIN(111, "GPIO_111"),
+ PINCTRL_PIN(112, "GPIO_112"),
+ PINCTRL_PIN(113, "GPIO_113"),
+ PINCTRL_PIN(114, "GPIO_114"),
+ PINCTRL_PIN(115, "GPIO_115"),
+ PINCTRL_PIN(116, "GPIO_116"),
+ PINCTRL_PIN(117, "GPIO_117"),
+ PINCTRL_PIN(118, "GPIO_118"),
+ PINCTRL_PIN(119, "GPIO_119"),
+ PINCTRL_PIN(120, "SDC1_RCLK"),
+ PINCTRL_PIN(121, "SDC1_CLK"),
+ PINCTRL_PIN(122, "SDC1_CMD"),
+ PINCTRL_PIN(123, "SDC1_DATA"),
+ PINCTRL_PIN(124, "SDC2_CLK"),
+ PINCTRL_PIN(125, "SDC2_CMD"),
+ PINCTRL_PIN(126, "SDC2_DATA"),
+};
+
+#define DECLARE_MSM_GPIO_PINS(pin) \
+ static const unsigned int gpio##pin##_pins[] = { pin }
+DECLARE_MSM_GPIO_PINS(0);
+DECLARE_MSM_GPIO_PINS(1);
+DECLARE_MSM_GPIO_PINS(2);
+DECLARE_MSM_GPIO_PINS(3);
+DECLARE_MSM_GPIO_PINS(4);
+DECLARE_MSM_GPIO_PINS(5);
+DECLARE_MSM_GPIO_PINS(6);
+DECLARE_MSM_GPIO_PINS(7);
+DECLARE_MSM_GPIO_PINS(8);
+DECLARE_MSM_GPIO_PINS(9);
+DECLARE_MSM_GPIO_PINS(10);
+DECLARE_MSM_GPIO_PINS(11);
+DECLARE_MSM_GPIO_PINS(12);
+DECLARE_MSM_GPIO_PINS(13);
+DECLARE_MSM_GPIO_PINS(14);
+DECLARE_MSM_GPIO_PINS(15);
+DECLARE_MSM_GPIO_PINS(16);
+DECLARE_MSM_GPIO_PINS(17);
+DECLARE_MSM_GPIO_PINS(18);
+DECLARE_MSM_GPIO_PINS(19);
+DECLARE_MSM_GPIO_PINS(20);
+DECLARE_MSM_GPIO_PINS(21);
+DECLARE_MSM_GPIO_PINS(22);
+DECLARE_MSM_GPIO_PINS(23);
+DECLARE_MSM_GPIO_PINS(24);
+DECLARE_MSM_GPIO_PINS(25);
+DECLARE_MSM_GPIO_PINS(26);
+DECLARE_MSM_GPIO_PINS(27);
+DECLARE_MSM_GPIO_PINS(28);
+DECLARE_MSM_GPIO_PINS(29);
+DECLARE_MSM_GPIO_PINS(30);
+DECLARE_MSM_GPIO_PINS(31);
+DECLARE_MSM_GPIO_PINS(32);
+DECLARE_MSM_GPIO_PINS(33);
+DECLARE_MSM_GPIO_PINS(34);
+DECLARE_MSM_GPIO_PINS(35);
+DECLARE_MSM_GPIO_PINS(36);
+DECLARE_MSM_GPIO_PINS(37);
+DECLARE_MSM_GPIO_PINS(38);
+DECLARE_MSM_GPIO_PINS(39);
+DECLARE_MSM_GPIO_PINS(40);
+DECLARE_MSM_GPIO_PINS(41);
+DECLARE_MSM_GPIO_PINS(42);
+DECLARE_MSM_GPIO_PINS(43);
+DECLARE_MSM_GPIO_PINS(44);
+DECLARE_MSM_GPIO_PINS(45);
+DECLARE_MSM_GPIO_PINS(46);
+DECLARE_MSM_GPIO_PINS(47);
+DECLARE_MSM_GPIO_PINS(48);
+DECLARE_MSM_GPIO_PINS(49);
+DECLARE_MSM_GPIO_PINS(50);
+DECLARE_MSM_GPIO_PINS(51);
+DECLARE_MSM_GPIO_PINS(52);
+DECLARE_MSM_GPIO_PINS(53);
+DECLARE_MSM_GPIO_PINS(54);
+DECLARE_MSM_GPIO_PINS(55);
+DECLARE_MSM_GPIO_PINS(56);
+DECLARE_MSM_GPIO_PINS(57);
+DECLARE_MSM_GPIO_PINS(58);
+DECLARE_MSM_GPIO_PINS(59);
+DECLARE_MSM_GPIO_PINS(60);
+DECLARE_MSM_GPIO_PINS(61);
+DECLARE_MSM_GPIO_PINS(62);
+DECLARE_MSM_GPIO_PINS(63);
+DECLARE_MSM_GPIO_PINS(64);
+DECLARE_MSM_GPIO_PINS(65);
+DECLARE_MSM_GPIO_PINS(66);
+DECLARE_MSM_GPIO_PINS(67);
+DECLARE_MSM_GPIO_PINS(68);
+DECLARE_MSM_GPIO_PINS(69);
+DECLARE_MSM_GPIO_PINS(70);
+DECLARE_MSM_GPIO_PINS(71);
+DECLARE_MSM_GPIO_PINS(72);
+DECLARE_MSM_GPIO_PINS(73);
+DECLARE_MSM_GPIO_PINS(74);
+DECLARE_MSM_GPIO_PINS(75);
+DECLARE_MSM_GPIO_PINS(76);
+DECLARE_MSM_GPIO_PINS(77);
+DECLARE_MSM_GPIO_PINS(78);
+DECLARE_MSM_GPIO_PINS(79);
+DECLARE_MSM_GPIO_PINS(80);
+DECLARE_MSM_GPIO_PINS(81);
+DECLARE_MSM_GPIO_PINS(82);
+DECLARE_MSM_GPIO_PINS(83);
+DECLARE_MSM_GPIO_PINS(84);
+DECLARE_MSM_GPIO_PINS(85);
+DECLARE_MSM_GPIO_PINS(86);
+DECLARE_MSM_GPIO_PINS(87);
+DECLARE_MSM_GPIO_PINS(88);
+DECLARE_MSM_GPIO_PINS(89);
+DECLARE_MSM_GPIO_PINS(90);
+DECLARE_MSM_GPIO_PINS(91);
+DECLARE_MSM_GPIO_PINS(92);
+DECLARE_MSM_GPIO_PINS(93);
+DECLARE_MSM_GPIO_PINS(94);
+DECLARE_MSM_GPIO_PINS(95);
+DECLARE_MSM_GPIO_PINS(96);
+DECLARE_MSM_GPIO_PINS(97);
+DECLARE_MSM_GPIO_PINS(98);
+DECLARE_MSM_GPIO_PINS(99);
+DECLARE_MSM_GPIO_PINS(100);
+DECLARE_MSM_GPIO_PINS(101);
+DECLARE_MSM_GPIO_PINS(102);
+DECLARE_MSM_GPIO_PINS(103);
+DECLARE_MSM_GPIO_PINS(104);
+DECLARE_MSM_GPIO_PINS(105);
+DECLARE_MSM_GPIO_PINS(106);
+DECLARE_MSM_GPIO_PINS(107);
+DECLARE_MSM_GPIO_PINS(108);
+DECLARE_MSM_GPIO_PINS(109);
+DECLARE_MSM_GPIO_PINS(110);
+DECLARE_MSM_GPIO_PINS(111);
+DECLARE_MSM_GPIO_PINS(112);
+DECLARE_MSM_GPIO_PINS(113);
+DECLARE_MSM_GPIO_PINS(114);
+DECLARE_MSM_GPIO_PINS(115);
+DECLARE_MSM_GPIO_PINS(116);
+DECLARE_MSM_GPIO_PINS(117);
+DECLARE_MSM_GPIO_PINS(118);
+DECLARE_MSM_GPIO_PINS(119);
+
+static const unsigned int sdc1_rclk_pins[] = { 120 };
+static const unsigned int sdc1_clk_pins[] = { 121 };
+static const unsigned int sdc1_cmd_pins[] = { 122 };
+static const unsigned int sdc1_data_pins[] = { 123 };
+static const unsigned int sdc2_clk_pins[] = { 124 };
+static const unsigned int sdc2_cmd_pins[] = { 125 };
+static const unsigned int sdc2_data_pins[] = { 126 };
+
+enum qcs404_functions {
+ msm_mux_gpio,
+ msm_mux_hdmi_tx,
+ msm_mux_hdmi_ddc,
+ msm_mux_blsp_uart_tx_a2,
+ msm_mux_blsp_spi2,
+ msm_mux_m_voc,
+ msm_mux_qdss_cti_trig_in_a0,
+ msm_mux_blsp_uart_rx_a2,
+ msm_mux_qdss_tracectl_a,
+ msm_mux_blsp_uart2,
+ msm_mux_aud_cdc,
+ msm_mux_blsp_i2c_sda_a2,
+ msm_mux_qdss_tracedata_a,
+ msm_mux_blsp_i2c_scl_a2,
+ msm_mux_qdss_tracectl_b,
+ msm_mux_qdss_cti_trig_in_b0,
+ msm_mux_blsp_uart1,
+ msm_mux_blsp_spi_mosi_a1,
+ msm_mux_blsp_spi_miso_a1,
+ msm_mux_qdss_tracedata_b,
+ msm_mux_blsp_i2c1,
+ msm_mux_blsp_spi_cs_n_a1,
+ msm_mux_gcc_plltest,
+ msm_mux_blsp_spi_clk_a1,
+ msm_mux_rgb_data0,
+ msm_mux_blsp_uart5,
+ msm_mux_blsp_spi5,
+ msm_mux_adsp_ext,
+ msm_mux_rgb_data1,
+ msm_mux_prng_rosc,
+ msm_mux_rgb_data2,
+ msm_mux_blsp_i2c5,
+ msm_mux_gcc_gp1_clk_b,
+ msm_mux_rgb_data3,
+ msm_mux_gcc_gp2_clk_b,
+ msm_mux_blsp_spi0,
+ msm_mux_blsp_uart0,
+ msm_mux_gcc_gp3_clk_b,
+ msm_mux_blsp_i2c0,
+ msm_mux_qdss_traceclk_b,
+ msm_mux_pcie_clk,
+ msm_mux_nfc_irq,
+ msm_mux_blsp_spi4,
+ msm_mux_nfc_dwl,
+ msm_mux_audio_ts,
+ msm_mux_rgb_data4,
+ msm_mux_spi_lcd,
+ msm_mux_blsp_uart_tx_b2,
+ msm_mux_gcc_gp3_clk_a,
+ msm_mux_rgb_data5,
+ msm_mux_blsp_uart_rx_b2,
+ msm_mux_blsp_i2c_sda_b2,
+ msm_mux_blsp_i2c_scl_b2,
+ msm_mux_pwm_led11,
+ msm_mux_i2s_3_data0_a,
+ msm_mux_ebi2_lcd,
+ msm_mux_i2s_3_data1_a,
+ msm_mux_i2s_3_data2_a,
+ msm_mux_atest_char,
+ msm_mux_pwm_led3,
+ msm_mux_i2s_3_data3_a,
+ msm_mux_pwm_led4,
+ msm_mux_i2s_4,
+ msm_mux_ebi2_a,
+ msm_mux_dsd_clk_b,
+ msm_mux_pwm_led5,
+ msm_mux_pwm_led6,
+ msm_mux_pwm_led7,
+ msm_mux_pwm_led8,
+ msm_mux_pwm_led24,
+ msm_mux_spkr_dac0,
+ msm_mux_blsp_i2c4,
+ msm_mux_pwm_led9,
+ msm_mux_pwm_led10,
+ msm_mux_spdifrx_opt,
+ msm_mux_pwm_led12,
+ msm_mux_pwm_led13,
+ msm_mux_pwm_led14,
+ msm_mux_wlan1_adc1,
+ msm_mux_rgb_data_b0,
+ msm_mux_pwm_led15,
+ msm_mux_blsp_spi_mosi_b1,
+ msm_mux_wlan1_adc0,
+ msm_mux_rgb_data_b1,
+ msm_mux_pwm_led16,
+ msm_mux_blsp_spi_miso_b1,
+ msm_mux_qdss_cti_trig_out_b0,
+ msm_mux_wlan2_adc1,
+ msm_mux_rgb_data_b2,
+ msm_mux_pwm_led17,
+ msm_mux_blsp_spi_cs_n_b1,
+ msm_mux_wlan2_adc0,
+ msm_mux_rgb_data_b3,
+ msm_mux_pwm_led18,
+ msm_mux_blsp_spi_clk_b1,
+ msm_mux_rgb_data_b4,
+ msm_mux_pwm_led19,
+ msm_mux_ext_mclk1_b,
+ msm_mux_qdss_traceclk_a,
+ msm_mux_rgb_data_b5,
+ msm_mux_pwm_led20,
+ msm_mux_atest_char3,
+ msm_mux_i2s_3_sck_b,
+ msm_mux_ldo_update,
+ msm_mux_bimc_dte0,
+ msm_mux_rgb_hsync,
+ msm_mux_pwm_led21,
+ msm_mux_i2s_3_ws_b,
+ msm_mux_dbg_out,
+ msm_mux_rgb_vsync,
+ msm_mux_i2s_3_data0_b,
+ msm_mux_ldo_en,
+ msm_mux_hdmi_dtest,
+ msm_mux_rgb_de,
+ msm_mux_i2s_3_data1_b,
+ msm_mux_hdmi_lbk9,
+ msm_mux_rgb_clk,
+ msm_mux_atest_char1,
+ msm_mux_i2s_3_data2_b,
+ msm_mux_ebi_cdc,
+ msm_mux_hdmi_lbk8,
+ msm_mux_rgb_mdp,
+ msm_mux_atest_char0,
+ msm_mux_i2s_3_data3_b,
+ msm_mux_hdmi_lbk7,
+ msm_mux_rgb_data_b6,
+ msm_mux_rgb_data_b7,
+ msm_mux_hdmi_lbk6,
+ msm_mux_rgmii_int,
+ msm_mux_cri_trng1,
+ msm_mux_rgmii_wol,
+ msm_mux_cri_trng0,
+ msm_mux_gcc_tlmm,
+ msm_mux_rgmii_ck,
+ msm_mux_rgmii_tx,
+ msm_mux_hdmi_lbk5,
+ msm_mux_hdmi_pixel,
+ msm_mux_hdmi_rcv,
+ msm_mux_hdmi_lbk4,
+ msm_mux_rgmii_ctl,
+ msm_mux_ext_lpass,
+ msm_mux_rgmii_rx,
+ msm_mux_cri_trng,
+ msm_mux_hdmi_lbk3,
+ msm_mux_hdmi_lbk2,
+ msm_mux_qdss_cti_trig_out_b1,
+ msm_mux_rgmii_mdio,
+ msm_mux_hdmi_lbk1,
+ msm_mux_rgmii_mdc,
+ msm_mux_hdmi_lbk0,
+ msm_mux_ir_in,
+ msm_mux_wsa_en,
+ msm_mux_rgb_data6,
+ msm_mux_rgb_data7,
+ msm_mux_atest_char2,
+ msm_mux_ebi_ch0,
+ msm_mux_blsp_uart3,
+ msm_mux_blsp_spi3,
+ msm_mux_sd_write,
+ msm_mux_blsp_i2c3,
+ msm_mux_gcc_gp1_clk_a,
+ msm_mux_qdss_cti_trig_in_b1,
+ msm_mux_gcc_gp2_clk_a,
+ msm_mux_ext_mclk0,
+ msm_mux_mclk_in1,
+ msm_mux_i2s_1,
+ msm_mux_dsd_clk_a,
+ msm_mux_qdss_cti_trig_in_a1,
+ msm_mux_rgmi_dll1,
+ msm_mux_pwm_led22,
+ msm_mux_pwm_led23,
+ msm_mux_qdss_cti_trig_out_a0,
+ msm_mux_rgmi_dll2,
+ msm_mux_pwm_led1,
+ msm_mux_qdss_cti_trig_out_a1,
+ msm_mux_pwm_led2,
+ msm_mux_i2s_2,
+ msm_mux_pll_bist,
+ msm_mux_ext_mclk1_a,
+ msm_mux_mclk_in2,
+ msm_mux_bimc_dte1,
+ msm_mux_i2s_3_sck_a,
+ msm_mux_i2s_3_ws_a,
+ msm_mux__,
+};
+
+static const char * const gpio_groups[] = {
+ "gpio0", "gpio1", "gpio2", "gpio3", "gpio4", "gpio5", "gpio6", "gpio7",
+ "gpio8", "gpio9", "gpio10", "gpio11", "gpio12", "gpio13", "gpio14",
+ "gpio15", "gpio16", "gpio17", "gpio18", "gpio19", "gpio20", "gpio21",
+ "gpio21", "gpio21", "gpio22", "gpio22", "gpio23", "gpio23", "gpio24",
+ "gpio25", "gpio26", "gpio27", "gpio28", "gpio29", "gpio30", "gpio31",
+ "gpio32", "gpio33", "gpio34", "gpio35", "gpio36", "gpio36", "gpio36",
+ "gpio36", "gpio37", "gpio37", "gpio37", "gpio38", "gpio38", "gpio38",
+ "gpio39", "gpio39", "gpio40", "gpio40", "gpio41", "gpio41", "gpio41",
+ "gpio42", "gpio43", "gpio44", "gpio45", "gpio46", "gpio47", "gpio48",
+ "gpio49", "gpio50", "gpio51", "gpio52", "gpio53", "gpio54", "gpio55",
+ "gpio56", "gpio57", "gpio58", "gpio59", "gpio59", "gpio60", "gpio61",
+ "gpio62", "gpio63", "gpio64", "gpio65", "gpio66", "gpio67", "gpio68",
+ "gpio69", "gpio70", "gpio71", "gpio72", "gpio73", "gpio74", "gpio75",
+ "gpio76", "gpio77", "gpio77", "gpio78", "gpio78", "gpio78", "gpio79",
+ "gpio79", "gpio79", "gpio80", "gpio81", "gpio81", "gpio82", "gpio83",
+ "gpio84", "gpio85", "gpio86", "gpio87", "gpio88", "gpio89", "gpio90",
+ "gpio91", "gpio92", "gpio93", "gpio94", "gpio95", "gpio96", "gpio97",
+ "gpio98", "gpio99", "gpio100", "gpio101", "gpio102", "gpio103",
+ "gpio104", "gpio105", "gpio106", "gpio107", "gpio108", "gpio108",
+ "gpio108", "gpio109", "gpio109", "gpio110", "gpio111", "gpio112",
+ "gpio113", "gpio114", "gpio115", "gpio116", "gpio117", "gpio118",
+ "gpio119",
+};
+
+static const char * const hdmi_tx_groups[] = {
+ "gpio14",
+};
+
+static const char * const hdmi_ddc_groups[] = {
+ "gpio15", "gpio16",
+};
+
+static const char * const blsp_uart_tx_a2_groups[] = {
+ "gpio17",
+};
+
+static const char * const blsp_spi2_groups[] = {
+ "gpio17", "gpio18", "gpio19", "gpio20",
+};
+
+static const char * const m_voc_groups[] = {
+ "gpio17", "gpio21",
+};
+
+static const char * const qdss_cti_trig_in_a0_groups[] = {
+ "gpio17",
+};
+
+static const char * const blsp_uart_rx_a2_groups[] = {
+ "gpio18",
+};
+
+static const char * const qdss_tracectl_a_groups[] = {
+ "gpio18",
+};
+
+static const char * const blsp_uart2_groups[] = {
+ "gpio19", "gpio20",
+};
+
+static const char * const aud_cdc_groups[] = {
+ "gpio19", "gpio20",
+};
+
+static const char * const blsp_i2c_sda_a2_groups[] = {
+ "gpio19",
+};
+
+static const char * const qdss_tracedata_a_groups[] = {
+ "gpio19", "gpio24", "gpio25", "gpio26", "gpio27", "gpio28", "gpio30",
+ "gpio31", "gpio32", "gpio36", "gpio38", "gpio39", "gpio42", "gpio43",
+ "gpio82", "gpio83",
+};
+
+static const char * const blsp_i2c_scl_a2_groups[] = {
+ "gpio20",
+};
+
+static const char * const qdss_tracectl_b_groups[] = {
+ "gpio20",
+};
+
+static const char * const qdss_cti_trig_in_b0_groups[] = {
+ "gpio21",
+};
+
+static const char * const blsp_uart1_groups[] = {
+ "gpio22", "gpio23", "gpio24", "gpio25",
+};
+
+static const char * const blsp_spi_mosi_a1_groups[] = {
+ "gpio22",
+};
+
+static const char * const blsp_spi_miso_a1_groups[] = {
+ "gpio23",
+};
+
+static const char * const qdss_tracedata_b_groups[] = {
+ "gpio23", "gpio35", "gpio40", "gpio41", "gpio44", "gpio45", "gpio46",
+ "gpio47", "gpio49", "gpio50", "gpio55", "gpio61", "gpio62", "gpio85",
+ "gpio89", "gpio93",
+};
+
+static const char * const blsp_i2c1_groups[] = {
+ "gpio24", "gpio25",
+};
+
+static const char * const blsp_spi_cs_n_a1_groups[] = {
+ "gpio24",
+};
+
+static const char * const gcc_plltest_groups[] = {
+ "gpio24", "gpio25",
+};
+
+static const char * const blsp_spi_clk_a1_groups[] = {
+ "gpio25",
+};
+
+static const char * const rgb_data0_groups[] = {
+ "gpio26", "gpio41",
+};
+
+static const char * const blsp_uart5_groups[] = {
+ "gpio26", "gpio27", "gpio28", "gpio29",
+};
+
+static const char * const blsp_spi5_groups[] = {
+ "gpio26", "gpio27", "gpio28", "gpio29", "gpio44", "gpio45", "gpio46",
+};
+
+static const char * const adsp_ext_groups[] = {
+ "gpio26",
+};
+
+static const char * const rgb_data1_groups[] = {
+ "gpio27", "gpio42",
+};
+
+static const char * const prng_rosc_groups[] = {
+ "gpio27",
+};
+
+static const char * const rgb_data2_groups[] = {
+ "gpio28", "gpio43",
+};
+
+static const char * const blsp_i2c5_groups[] = {
+ "gpio28", "gpio29",
+};
+
+static const char * const gcc_gp1_clk_b_groups[] = {
+ "gpio28",
+};
+
+static const char * const rgb_data3_groups[] = {
+ "gpio29", "gpio44",
+};
+
+static const char * const gcc_gp2_clk_b_groups[] = {
+ "gpio29",
+};
+
+static const char * const blsp_spi0_groups[] = {
+ "gpio30", "gpio31", "gpio32", "gpio33",
+};
+
+static const char * const blsp_uart0_groups[] = {
+ "gpio30", "gpio31", "gpio32", "gpio33",
+};
+
+static const char * const gcc_gp3_clk_b_groups[] = {
+ "gpio30",
+};
+
+static const char * const blsp_i2c0_groups[] = {
+ "gpio32", "gpio33",
+};
+
+static const char * const qdss_traceclk_b_groups[] = {
+ "gpio34",
+};
+
+static const char * const pcie_clk_groups[] = {
+ "gpio35",
+};
+
+static const char * const nfc_irq_groups[] = {
+ "gpio37",
+};
+
+static const char * const blsp_spi4_groups[] = {
+ "gpio37", "gpio38", "gpio117", "gpio118",
+};
+
+static const char * const nfc_dwl_groups[] = {
+ "gpio38",
+};
+
+static const char * const audio_ts_groups[] = {
+ "gpio38",
+};
+
+static const char * const rgb_data4_groups[] = {
+ "gpio39", "gpio45",
+};
+
+static const char * const spi_lcd_groups[] = {
+ "gpio39", "gpio40",
+};
+
+static const char * const blsp_uart_tx_b2_groups[] = {
+ "gpio39",
+};
+
+static const char * const gcc_gp3_clk_a_groups[] = {
+ "gpio39",
+};
+
+static const char * const rgb_data5_groups[] = {
+ "gpio40", "gpio46",
+};
+
+static const char * const blsp_uart_rx_b2_groups[] = {
+ "gpio40",
+};
+
+static const char * const blsp_i2c_sda_b2_groups[] = {
+ "gpio41",
+};
+
+static const char * const blsp_i2c_scl_b2_groups[] = {
+ "gpio42",
+};
+
+static const char * const pwm_led11_groups[] = {
+ "gpio43",
+};
+
+static const char * const i2s_3_data0_a_groups[] = {
+ "gpio106",
+};
+
+static const char * const ebi2_lcd_groups[] = {
+ "gpio106", "gpio107", "gpio108", "gpio109",
+};
+
+static const char * const i2s_3_data1_a_groups[] = {
+ "gpio107",
+};
+
+static const char * const i2s_3_data2_a_groups[] = {
+ "gpio108",
+};
+
+static const char * const atest_char_groups[] = {
+ "gpio108",
+};
+
+static const char * const pwm_led3_groups[] = {
+ "gpio108",
+};
+
+static const char * const i2s_3_data3_a_groups[] = {
+ "gpio109",
+};
+
+static const char * const pwm_led4_groups[] = {
+ "gpio109",
+};
+
+static const char * const i2s_4_groups[] = {
+ "gpio110", "gpio111", "gpio111", "gpio112", "gpio112", "gpio113",
+ "gpio113", "gpio114", "gpio114", "gpio115", "gpio115", "gpio116",
+};
+
+static const char * const ebi2_a_groups[] = {
+ "gpio110",
+};
+
+static const char * const dsd_clk_b_groups[] = {
+ "gpio110",
+};
+
+static const char * const pwm_led5_groups[] = {
+ "gpio110",
+};
+
+static const char * const pwm_led6_groups[] = {
+ "gpio111",
+};
+
+static const char * const pwm_led7_groups[] = {
+ "gpio112",
+};
+
+static const char * const pwm_led8_groups[] = {
+ "gpio113",
+};
+
+static const char * const pwm_led24_groups[] = {
+ "gpio114",
+};
+
+static const char * const spkr_dac0_groups[] = {
+ "gpio116",
+};
+
+static const char * const blsp_i2c4_groups[] = {
+ "gpio117", "gpio118",
+};
+
+static const char * const pwm_led9_groups[] = {
+ "gpio117",
+};
+
+static const char * const pwm_led10_groups[] = {
+ "gpio118",
+};
+
+static const char * const spdifrx_opt_groups[] = {
+ "gpio119",
+};
+
+static const char * const pwm_led12_groups[] = {
+ "gpio44",
+};
+
+static const char * const pwm_led13_groups[] = {
+ "gpio45",
+};
+
+static const char * const pwm_led14_groups[] = {
+ "gpio46",
+};
+
+static const char * const wlan1_adc1_groups[] = {
+ "gpio46",
+};
+
+static const char * const rgb_data_b0_groups[] = {
+ "gpio47",
+};
+
+static const char * const pwm_led15_groups[] = {
+ "gpio47",
+};
+
+static const char * const blsp_spi_mosi_b1_groups[] = {
+ "gpio47",
+};
+
+static const char * const wlan1_adc0_groups[] = {
+ "gpio47",
+};
+
+static const char * const rgb_data_b1_groups[] = {
+ "gpio48",
+};
+
+static const char * const pwm_led16_groups[] = {
+ "gpio48",
+};
+
+static const char * const blsp_spi_miso_b1_groups[] = {
+ "gpio48",
+};
+
+static const char * const qdss_cti_trig_out_b0_groups[] = {
+ "gpio48",
+};
+
+static const char * const wlan2_adc1_groups[] = {
+ "gpio48",
+};
+
+static const char * const rgb_data_b2_groups[] = {
+ "gpio49",
+};
+
+static const char * const pwm_led17_groups[] = {
+ "gpio49",
+};
+
+static const char * const blsp_spi_cs_n_b1_groups[] = {
+ "gpio49",
+};
+
+static const char * const wlan2_adc0_groups[] = {
+ "gpio49",
+};
+
+static const char * const rgb_data_b3_groups[] = {
+ "gpio50",
+};
+
+static const char * const pwm_led18_groups[] = {
+ "gpio50",
+};
+
+static const char * const blsp_spi_clk_b1_groups[] = {
+ "gpio50",
+};
+
+static const char * const rgb_data_b4_groups[] = {
+ "gpio51",
+};
+
+static const char * const pwm_led19_groups[] = {
+ "gpio51",
+};
+
+static const char * const ext_mclk1_b_groups[] = {
+ "gpio51",
+};
+
+static const char * const qdss_traceclk_a_groups[] = {
+ "gpio51",
+};
+
+static const char * const rgb_data_b5_groups[] = {
+ "gpio52",
+};
+
+static const char * const pwm_led20_groups[] = {
+ "gpio52",
+};
+
+static const char * const atest_char3_groups[] = {
+ "gpio52",
+};
+
+static const char * const i2s_3_sck_b_groups[] = {
+ "gpio52",
+};
+
+static const char * const ldo_update_groups[] = {
+ "gpio52",
+};
+
+static const char * const bimc_dte0_groups[] = {
+ "gpio52", "gpio54",
+};
+
+static const char * const rgb_hsync_groups[] = {
+ "gpio53",
+};
+
+static const char * const pwm_led21_groups[] = {
+ "gpio53",
+};
+
+static const char * const i2s_3_ws_b_groups[] = {
+ "gpio53",
+};
+
+static const char * const dbg_out_groups[] = {
+ "gpio53",
+};
+
+static const char * const rgb_vsync_groups[] = {
+ "gpio54",
+};
+
+static const char * const i2s_3_data0_b_groups[] = {
+ "gpio54",
+};
+
+static const char * const ldo_en_groups[] = {
+ "gpio54",
+};
+
+static const char * const hdmi_dtest_groups[] = {
+ "gpio54",
+};
+
+static const char * const rgb_de_groups[] = {
+ "gpio55",
+};
+
+static const char * const i2s_3_data1_b_groups[] = {
+ "gpio55",
+};
+
+static const char * const hdmi_lbk9_groups[] = {
+ "gpio55",
+};
+
+static const char * const rgb_clk_groups[] = {
+ "gpio56",
+};
+
+static const char * const atest_char1_groups[] = {
+ "gpio56",
+};
+
+static const char * const i2s_3_data2_b_groups[] = {
+ "gpio56",
+};
+
+static const char * const ebi_cdc_groups[] = {
+ "gpio56", "gpio58", "gpio106", "gpio107", "gpio108", "gpio111",
+};
+
+static const char * const hdmi_lbk8_groups[] = {
+ "gpio56",
+};
+
+static const char * const rgb_mdp_groups[] = {
+ "gpio57",
+};
+
+static const char * const atest_char0_groups[] = {
+ "gpio57",
+};
+
+static const char * const i2s_3_data3_b_groups[] = {
+ "gpio57",
+};
+
+static const char * const hdmi_lbk7_groups[] = {
+ "gpio57",
+};
+
+static const char * const rgb_data_b6_groups[] = {
+ "gpio58",
+};
+
+static const char * const rgb_data_b7_groups[] = {
+ "gpio59",
+};
+
+static const char * const hdmi_lbk6_groups[] = {
+ "gpio59",
+};
+
+static const char * const rgmii_int_groups[] = {
+ "gpio61",
+};
+
+static const char * const cri_trng1_groups[] = {
+ "gpio61",
+};
+
+static const char * const rgmii_wol_groups[] = {
+ "gpio62",
+};
+
+static const char * const cri_trng0_groups[] = {
+ "gpio62",
+};
+
+static const char * const gcc_tlmm_groups[] = {
+ "gpio62",
+};
+
+static const char * const rgmii_ck_groups[] = {
+ "gpio63", "gpio69",
+};
+
+static const char * const rgmii_tx_groups[] = {
+ "gpio64", "gpio65", "gpio66", "gpio67",
+};
+
+static const char * const hdmi_lbk5_groups[] = {
+ "gpio64",
+};
+
+static const char * const hdmi_pixel_groups[] = {
+ "gpio65",
+};
+
+static const char * const hdmi_rcv_groups[] = {
+ "gpio66",
+};
+
+static const char * const hdmi_lbk4_groups[] = {
+ "gpio67",
+};
+
+static const char * const rgmii_ctl_groups[] = {
+ "gpio68", "gpio74",
+};
+
+static const char * const ext_lpass_groups[] = {
+ "gpio69",
+};
+
+static const char * const rgmii_rx_groups[] = {
+ "gpio70", "gpio71", "gpio72", "gpio73",
+};
+
+static const char * const cri_trng_groups[] = {
+ "gpio70",
+};
+
+static const char * const hdmi_lbk3_groups[] = {
+ "gpio71",
+};
+
+static const char * const hdmi_lbk2_groups[] = {
+ "gpio72",
+};
+
+static const char * const qdss_cti_trig_out_b1_groups[] = {
+ "gpio73",
+};
+
+static const char * const rgmii_mdio_groups[] = {
+ "gpio75",
+};
+
+static const char * const hdmi_lbk1_groups[] = {
+ "gpio75",
+};
+
+static const char * const rgmii_mdc_groups[] = {
+ "gpio76",
+};
+
+static const char * const hdmi_lbk0_groups[] = {
+ "gpio76",
+};
+
+static const char * const ir_in_groups[] = {
+ "gpio77",
+};
+
+static const char * const wsa_en_groups[] = {
+ "gpio77",
+};
+
+static const char * const rgb_data6_groups[] = {
+ "gpio78", "gpio80",
+};
+
+static const char * const rgb_data7_groups[] = {
+ "gpio79", "gpio81",
+};
+
+static const char * const atest_char2_groups[] = {
+ "gpio80",
+};
+
+static const char * const ebi_ch0_groups[] = {
+ "gpio81",
+};
+
+static const char * const blsp_uart3_groups[] = {
+ "gpio82", "gpio83", "gpio84", "gpio85",
+};
+
+static const char * const blsp_spi3_groups[] = {
+ "gpio82", "gpio83", "gpio84", "gpio85",
+};
+
+static const char * const sd_write_groups[] = {
+ "gpio82",
+};
+
+static const char * const blsp_i2c3_groups[] = {
+ "gpio84", "gpio85",
+};
+
+static const char * const gcc_gp1_clk_a_groups[] = {
+ "gpio84",
+};
+
+static const char * const qdss_cti_trig_in_b1_groups[] = {
+ "gpio84",
+};
+
+static const char * const gcc_gp2_clk_a_groups[] = {
+ "gpio85",
+};
+
+static const char * const ext_mclk0_groups[] = {
+ "gpio86",
+};
+
+static const char * const mclk_in1_groups[] = {
+ "gpio86",
+};
+
+static const char * const i2s_1_groups[] = {
+ "gpio87", "gpio88", "gpio88", "gpio89", "gpio89", "gpio90", "gpio90",
+ "gpio91", "gpio91", "gpio92", "gpio92", "gpio93", "gpio93", "gpio94",
+ "gpio94", "gpio95", "gpio95", "gpio96",
+};
+
+static const char * const dsd_clk_a_groups[] = {
+ "gpio87",
+};
+
+static const char * const qdss_cti_trig_in_a1_groups[] = {
+ "gpio92",
+};
+
+static const char * const rgmi_dll1_groups[] = {
+ "gpio92",
+};
+
+static const char * const pwm_led22_groups[] = {
+ "gpio93",
+};
+
+static const char * const pwm_led23_groups[] = {
+ "gpio94",
+};
+
+static const char * const qdss_cti_trig_out_a0_groups[] = {
+ "gpio94",
+};
+
+static const char * const rgmi_dll2_groups[] = {
+ "gpio94",
+};
+
+static const char * const pwm_led1_groups[] = {
+ "gpio95",
+};
+
+static const char * const qdss_cti_trig_out_a1_groups[] = {
+ "gpio95",
+};
+
+static const char * const pwm_led2_groups[] = {
+ "gpio96",
+};
+
+static const char * const i2s_2_groups[] = {
+ "gpio97", "gpio98", "gpio99", "gpio100", "gpio101", "gpio102",
+};
+
+static const char * const pll_bist_groups[] = {
+ "gpio100",
+};
+
+static const char * const ext_mclk1_a_groups[] = {
+ "gpio103",
+};
+
+static const char * const mclk_in2_groups[] = {
+ "gpio103",
+};
+
+static const char * const bimc_dte1_groups[] = {
+ "gpio103", "gpio109",
+};
+
+static const char * const i2s_3_sck_a_groups[] = {
+ "gpio104",
+};
+
+static const char * const i2s_3_ws_a_groups[] = {
+ "gpio105",
+};
+
+static const struct msm_function qcs404_functions[] = {
+ FUNCTION(gpio),
+ FUNCTION(hdmi_tx),
+ FUNCTION(hdmi_ddc),
+ FUNCTION(blsp_uart_tx_a2),
+ FUNCTION(blsp_spi2),
+ FUNCTION(m_voc),
+ FUNCTION(qdss_cti_trig_in_a0),
+ FUNCTION(blsp_uart_rx_a2),
+ FUNCTION(qdss_tracectl_a),
+ FUNCTION(blsp_uart2),
+ FUNCTION(aud_cdc),
+ FUNCTION(blsp_i2c_sda_a2),
+ FUNCTION(qdss_tracedata_a),
+ FUNCTION(blsp_i2c_scl_a2),
+ FUNCTION(qdss_tracectl_b),
+ FUNCTION(qdss_cti_trig_in_b0),
+ FUNCTION(blsp_uart1),
+ FUNCTION(blsp_spi_mosi_a1),
+ FUNCTION(blsp_spi_miso_a1),
+ FUNCTION(qdss_tracedata_b),
+ FUNCTION(blsp_i2c1),
+ FUNCTION(blsp_spi_cs_n_a1),
+ FUNCTION(gcc_plltest),
+ FUNCTION(blsp_spi_clk_a1),
+ FUNCTION(rgb_data0),
+ FUNCTION(blsp_uart5),
+ FUNCTION(blsp_spi5),
+ FUNCTION(adsp_ext),
+ FUNCTION(rgb_data1),
+ FUNCTION(prng_rosc),
+ FUNCTION(rgb_data2),
+ FUNCTION(blsp_i2c5),
+ FUNCTION(gcc_gp1_clk_b),
+ FUNCTION(rgb_data3),
+ FUNCTION(gcc_gp2_clk_b),
+ FUNCTION(blsp_spi0),
+ FUNCTION(blsp_uart0),
+ FUNCTION(gcc_gp3_clk_b),
+ FUNCTION(blsp_i2c0),
+ FUNCTION(qdss_traceclk_b),
+ FUNCTION(pcie_clk),
+ FUNCTION(nfc_irq),
+ FUNCTION(blsp_spi4),
+ FUNCTION(nfc_dwl),
+ FUNCTION(audio_ts),
+ FUNCTION(rgb_data4),
+ FUNCTION(spi_lcd),
+ FUNCTION(blsp_uart_tx_b2),
+ FUNCTION(gcc_gp3_clk_a),
+ FUNCTION(rgb_data5),
+ FUNCTION(blsp_uart_rx_b2),
+ FUNCTION(blsp_i2c_sda_b2),
+ FUNCTION(blsp_i2c_scl_b2),
+ FUNCTION(pwm_led11),
+ FUNCTION(i2s_3_data0_a),
+ FUNCTION(ebi2_lcd),
+ FUNCTION(i2s_3_data1_a),
+ FUNCTION(i2s_3_data2_a),
+ FUNCTION(atest_char),
+ FUNCTION(pwm_led3),
+ FUNCTION(i2s_3_data3_a),
+ FUNCTION(pwm_led4),
+ FUNCTION(i2s_4),
+ FUNCTION(ebi2_a),
+ FUNCTION(dsd_clk_b),
+ FUNCTION(pwm_led5),
+ FUNCTION(pwm_led6),
+ FUNCTION(pwm_led7),
+ FUNCTION(pwm_led8),
+ FUNCTION(pwm_led24),
+ FUNCTION(spkr_dac0),
+ FUNCTION(blsp_i2c4),
+ FUNCTION(pwm_led9),
+ FUNCTION(pwm_led10),
+ FUNCTION(spdifrx_opt),
+ FUNCTION(pwm_led12),
+ FUNCTION(pwm_led13),
+ FUNCTION(pwm_led14),
+ FUNCTION(wlan1_adc1),
+ FUNCTION(rgb_data_b0),
+ FUNCTION(pwm_led15),
+ FUNCTION(blsp_spi_mosi_b1),
+ FUNCTION(wlan1_adc0),
+ FUNCTION(rgb_data_b1),
+ FUNCTION(pwm_led16),
+ FUNCTION(blsp_spi_miso_b1),
+ FUNCTION(qdss_cti_trig_out_b0),
+ FUNCTION(wlan2_adc1),
+ FUNCTION(rgb_data_b2),
+ FUNCTION(pwm_led17),
+ FUNCTION(blsp_spi_cs_n_b1),
+ FUNCTION(wlan2_adc0),
+ FUNCTION(rgb_data_b3),
+ FUNCTION(pwm_led18),
+ FUNCTION(blsp_spi_clk_b1),
+ FUNCTION(rgb_data_b4),
+ FUNCTION(pwm_led19),
+ FUNCTION(ext_mclk1_b),
+ FUNCTION(qdss_traceclk_a),
+ FUNCTION(rgb_data_b5),
+ FUNCTION(pwm_led20),
+ FUNCTION(atest_char3),
+ FUNCTION(i2s_3_sck_b),
+ FUNCTION(ldo_update),
+ FUNCTION(bimc_dte0),
+ FUNCTION(rgb_hsync),
+ FUNCTION(pwm_led21),
+ FUNCTION(i2s_3_ws_b),
+ FUNCTION(dbg_out),
+ FUNCTION(rgb_vsync),
+ FUNCTION(i2s_3_data0_b),
+ FUNCTION(ldo_en),
+ FUNCTION(hdmi_dtest),
+ FUNCTION(rgb_de),
+ FUNCTION(i2s_3_data1_b),
+ FUNCTION(hdmi_lbk9),
+ FUNCTION(rgb_clk),
+ FUNCTION(atest_char1),
+ FUNCTION(i2s_3_data2_b),
+ FUNCTION(ebi_cdc),
+ FUNCTION(hdmi_lbk8),
+ FUNCTION(rgb_mdp),
+ FUNCTION(atest_char0),
+ FUNCTION(i2s_3_data3_b),
+ FUNCTION(hdmi_lbk7),
+ FUNCTION(rgb_data_b6),
+ FUNCTION(rgb_data_b7),
+ FUNCTION(hdmi_lbk6),
+ FUNCTION(rgmii_int),
+ FUNCTION(cri_trng1),
+ FUNCTION(rgmii_wol),
+ FUNCTION(cri_trng0),
+ FUNCTION(gcc_tlmm),
+ FUNCTION(rgmii_ck),
+ FUNCTION(rgmii_tx),
+ FUNCTION(hdmi_lbk5),
+ FUNCTION(hdmi_pixel),
+ FUNCTION(hdmi_rcv),
+ FUNCTION(hdmi_lbk4),
+ FUNCTION(rgmii_ctl),
+ FUNCTION(ext_lpass),
+ FUNCTION(rgmii_rx),
+ FUNCTION(cri_trng),
+ FUNCTION(hdmi_lbk3),
+ FUNCTION(hdmi_lbk2),
+ FUNCTION(qdss_cti_trig_out_b1),
+ FUNCTION(rgmii_mdio),
+ FUNCTION(hdmi_lbk1),
+ FUNCTION(rgmii_mdc),
+ FUNCTION(hdmi_lbk0),
+ FUNCTION(ir_in),
+ FUNCTION(wsa_en),
+ FUNCTION(rgb_data6),
+ FUNCTION(rgb_data7),
+ FUNCTION(atest_char2),
+ FUNCTION(ebi_ch0),
+ FUNCTION(blsp_uart3),
+ FUNCTION(blsp_spi3),
+ FUNCTION(sd_write),
+ FUNCTION(blsp_i2c3),
+ FUNCTION(gcc_gp1_clk_a),
+ FUNCTION(qdss_cti_trig_in_b1),
+ FUNCTION(gcc_gp2_clk_a),
+ FUNCTION(ext_mclk0),
+ FUNCTION(mclk_in1),
+ FUNCTION(i2s_1),
+ FUNCTION(dsd_clk_a),
+ FUNCTION(qdss_cti_trig_in_a1),
+ FUNCTION(rgmi_dll1),
+ FUNCTION(pwm_led22),
+ FUNCTION(pwm_led23),
+ FUNCTION(qdss_cti_trig_out_a0),
+ FUNCTION(rgmi_dll2),
+ FUNCTION(pwm_led1),
+ FUNCTION(qdss_cti_trig_out_a1),
+ FUNCTION(pwm_led2),
+ FUNCTION(i2s_2),
+ FUNCTION(pll_bist),
+ FUNCTION(ext_mclk1_a),
+ FUNCTION(mclk_in2),
+ FUNCTION(bimc_dte1),
+ FUNCTION(i2s_3_sck_a),
+ FUNCTION(i2s_3_ws_a),
+};
+
+/* Every pin is maintained as a single group, and missing or non-existing pin
+ * would be maintained as dummy group to synchronize pin group index with
+ * pin descriptor registered with pinctrl core.
+ * Clients would not be able to request these dummy pin groups.
+ */
+static const struct msm_pingroup qcs404_groups[] = {
+ [0] = PINGROUP(0, SOUTH, _, _, _, _, _, _, _, _, _),
+ [1] = PINGROUP(1, SOUTH, _, _, _, _, _, _, _, _, _),
+ [2] = PINGROUP(2, SOUTH, _, _, _, _, _, _, _, _, _),
+ [3] = PINGROUP(3, SOUTH, _, _, _, _, _, _, _, _, _),
+ [4] = PINGROUP(4, SOUTH, _, _, _, _, _, _, _, _, _),
+ [5] = PINGROUP(5, SOUTH, _, _, _, _, _, _, _, _, _),
+ [6] = PINGROUP(6, SOUTH, _, _, _, _, _, _, _, _, _),
+ [7] = PINGROUP(7, SOUTH, _, _, _, _, _, _, _, _, _),
+ [8] = PINGROUP(8, SOUTH, _, _, _, _, _, _, _, _, _),
+ [9] = PINGROUP(9, SOUTH, _, _, _, _, _, _, _, _, _),
+ [10] = PINGROUP(10, SOUTH, _, _, _, _, _, _, _, _, _),
+ [11] = PINGROUP(11, SOUTH, _, _, _, _, _, _, _, _, _),
+ [12] = PINGROUP(12, SOUTH, _, _, _, _, _, _, _, _, _),
+ [13] = PINGROUP(13, SOUTH, _, _, _, _, _, _, _, _, _),
+ [14] = PINGROUP(14, SOUTH, hdmi_tx, _, _, _, _, _, _, _, _),
+ [15] = PINGROUP(15, SOUTH, hdmi_ddc, _, _, _, _, _, _, _, _),
+ [16] = PINGROUP(16, SOUTH, hdmi_ddc, _, _, _, _, _, _, _, _),
+ [17] = PINGROUP(17, NORTH, blsp_uart_tx_a2, blsp_spi2, m_voc, _, _, _, _, _, _),
+ [18] = PINGROUP(18, NORTH, blsp_uart_rx_a2, blsp_spi2, _, _, _, _, _, qdss_tracectl_a, _),
+ [19] = PINGROUP(19, NORTH, blsp_uart2, aud_cdc, blsp_i2c_sda_a2, blsp_spi2, _, qdss_tracedata_a, _, _, _),
+ [20] = PINGROUP(20, NORTH, blsp_uart2, aud_cdc, blsp_i2c_scl_a2, blsp_spi2, _, _, _, _, _),
+ [21] = PINGROUP(21, SOUTH, m_voc, _, _, _, _, _, _, _, qdss_cti_trig_in_b0),
+ [22] = PINGROUP(22, NORTH, blsp_uart1, blsp_spi_mosi_a1, _, _, _, _, _, _, _),
+ [23] = PINGROUP(23, NORTH, blsp_uart1, blsp_spi_miso_a1, _, _, _, _, _, qdss_tracedata_b, _),
+ [24] = PINGROUP(24, NORTH, blsp_uart1, blsp_i2c1, blsp_spi_cs_n_a1, gcc_plltest, _, _, _, _, _),
+ [25] = PINGROUP(25, NORTH, blsp_uart1, blsp_i2c1, blsp_spi_clk_a1, gcc_plltest, _, _, _, _, _),
+ [26] = PINGROUP(26, EAST, rgb_data0, blsp_uart5, blsp_spi5, adsp_ext, _, _, _, _, _),
+ [27] = PINGROUP(27, EAST, rgb_data1, blsp_uart5, blsp_spi5, prng_rosc, _, _, _, _, _),
+ [28] = PINGROUP(28, EAST, rgb_data2, blsp_uart5, blsp_i2c5, blsp_spi5, gcc_gp1_clk_b, _, _, _, _),
+ [29] = PINGROUP(29, EAST, rgb_data3, blsp_uart5, blsp_i2c5, blsp_spi5, gcc_gp2_clk_b, _, _, _, _),
+ [30] = PINGROUP(30, NORTH, blsp_spi0, blsp_uart0, gcc_gp3_clk_b, _, _, _, _, _, _),
+ [31] = PINGROUP(31, NORTH, blsp_spi0, blsp_uart0, _, _, _, _, _, _, _),
+ [32] = PINGROUP(32, NORTH, blsp_spi0, blsp_uart0, blsp_i2c0, _, _, _, _, _, _),
+ [33] = PINGROUP(33, NORTH, blsp_spi0, blsp_uart0, blsp_i2c0, _, _, _, _, _, _),
+ [34] = PINGROUP(34, SOUTH, _, qdss_traceclk_b, _, _, _, _, _, _, _),
+ [35] = PINGROUP(35, SOUTH, pcie_clk, _, qdss_tracedata_b, _, _, _, _, _, _),
+ [36] = PINGROUP(36, NORTH, _, _, _, _, _, _, qdss_tracedata_a, _, _),
+ [37] = PINGROUP(37, NORTH, nfc_irq, blsp_spi4, _, _, _, _, _, _, _),
+ [38] = PINGROUP(38, NORTH, nfc_dwl, blsp_spi4, audio_ts, _, _, _, _, _, _),
+ [39] = PINGROUP(39, EAST, rgb_data4, spi_lcd, blsp_uart_tx_b2, gcc_gp3_clk_a, qdss_tracedata_a, _, _, _, _),
+ [40] = PINGROUP(40, EAST, rgb_data5, spi_lcd, blsp_uart_rx_b2, _, qdss_tracedata_b, _, _, _, _),
+ [41] = PINGROUP(41, EAST, rgb_data0, blsp_i2c_sda_b2, _, qdss_tracedata_b, _, _, _, _, _),
+ [42] = PINGROUP(42, EAST, rgb_data1, blsp_i2c_scl_b2, _, _, _, _, _, qdss_tracedata_a, _),
+ [43] = PINGROUP(43, EAST, rgb_data2, pwm_led11, _, _, _, _, _, _, _),
+ [44] = PINGROUP(44, EAST, rgb_data3, pwm_led12, blsp_spi5, _, _, _, _, _, _),
+ [45] = PINGROUP(45, EAST, rgb_data4, pwm_led13, blsp_spi5, qdss_tracedata_b, _, _, _, _, _),
+ [46] = PINGROUP(46, EAST, rgb_data5, pwm_led14, blsp_spi5, qdss_tracedata_b, _, wlan1_adc1, _, _, _),
+ [47] = PINGROUP(47, EAST, rgb_data_b0, pwm_led15, blsp_spi_mosi_b1, qdss_tracedata_b, _, wlan1_adc0, _, _, _),
+ [48] = PINGROUP(48, EAST, rgb_data_b1, pwm_led16, blsp_spi_miso_b1, _, qdss_cti_trig_out_b0, _, wlan2_adc1, _, _),
+ [49] = PINGROUP(49, EAST, rgb_data_b2, pwm_led17, blsp_spi_cs_n_b1, _, qdss_tracedata_b, _, wlan2_adc0, _, _),
+ [50] = PINGROUP(50, EAST, rgb_data_b3, pwm_led18, blsp_spi_clk_b1, qdss_tracedata_b, _, _, _, _, _),
+ [51] = PINGROUP(51, EAST, rgb_data_b4, pwm_led19, ext_mclk1_b, qdss_traceclk_a, _, _, _, _, _),
+ [52] = PINGROUP(52, EAST, rgb_data_b5, pwm_led20, atest_char3, i2s_3_sck_b, ldo_update, bimc_dte0, _, _, _),
+ [53] = PINGROUP(53, EAST, rgb_hsync, pwm_led21, i2s_3_ws_b, dbg_out, _, _, _, _, _),
+ [54] = PINGROUP(54, EAST, rgb_vsync, i2s_3_data0_b, ldo_en, bimc_dte0, _, hdmi_dtest, _, _, _),
+ [55] = PINGROUP(55, EAST, rgb_de, i2s_3_data1_b, _, qdss_tracedata_b, _, hdmi_lbk9, _, _, _),
+ [56] = PINGROUP(56, EAST, rgb_clk, atest_char1, i2s_3_data2_b, ebi_cdc, _, hdmi_lbk8, _, _, _),
+ [57] = PINGROUP(57, EAST, rgb_mdp, atest_char0, i2s_3_data3_b, _, hdmi_lbk7, _, _, _, _),
+ [58] = PINGROUP(58, EAST, rgb_data_b6, _, ebi_cdc, _, _, _, _, _, _),
+ [59] = PINGROUP(59, EAST, rgb_data_b7, _, hdmi_lbk6, _, _, _, _, _, _),
+ [60] = PINGROUP(60, NORTH, _, _, _, _, _, _, _, _, _),
+ [61] = PINGROUP(61, NORTH, rgmii_int, cri_trng1, qdss_tracedata_b, _, _, _, _, _, _),
+ [62] = PINGROUP(62, NORTH, rgmii_wol, cri_trng0, qdss_tracedata_b, gcc_tlmm, _, _, _, _, _),
+ [63] = PINGROUP(63, NORTH, rgmii_ck, _, _, _, _, _, _, _, _),
+ [64] = PINGROUP(64, NORTH, rgmii_tx, _, hdmi_lbk5, _, _, _, _, _, _),
+ [65] = PINGROUP(65, NORTH, rgmii_tx, _, hdmi_pixel, _, _, _, _, _, _),
+ [66] = PINGROUP(66, NORTH, rgmii_tx, _, hdmi_rcv, _, _, _, _, _, _),
+ [67] = PINGROUP(67, NORTH, rgmii_tx, _, hdmi_lbk4, _, _, _, _, _, _),
+ [68] = PINGROUP(68, NORTH, rgmii_ctl, _, _, _, _, _, _, _, _),
+ [69] = PINGROUP(69, NORTH, rgmii_ck, ext_lpass, _, _, _, _, _, _, _),
+ [70] = PINGROUP(70, NORTH, rgmii_rx, cri_trng, _, _, _, _, _, _, _),
+ [71] = PINGROUP(71, NORTH, rgmii_rx, _, hdmi_lbk3, _, _, _, _, _, _),
+ [72] = PINGROUP(72, NORTH, rgmii_rx, _, hdmi_lbk2, _, _, _, _, _, _),
+ [73] = PINGROUP(73, NORTH, rgmii_rx, _, _, _, _, qdss_cti_trig_out_b1, _, _, _),
+ [74] = PINGROUP(74, NORTH, rgmii_ctl, _, _, _, _, _, _, _, _),
+ [75] = PINGROUP(75, NORTH, rgmii_mdio, _, hdmi_lbk1, _, _, _, _, _, _),
+ [76] = PINGROUP(76, NORTH, rgmii_mdc, _, _, _, _, _, hdmi_lbk0, _, _),
+ [77] = PINGROUP(77, NORTH, ir_in, wsa_en, _, _, _, _, _, _, _),
+ [78] = PINGROUP(78, EAST, rgb_data6, _, _, _, _, _, _, _, _),
+ [79] = PINGROUP(79, EAST, rgb_data7, _, _, _, _, _, _, _, _),
+ [80] = PINGROUP(80, EAST, rgb_data6, atest_char2, _, _, _, _, _, _, _),
+ [81] = PINGROUP(81, EAST, rgb_data7, ebi_ch0, _, _, _, _, _, _, _),
+ [82] = PINGROUP(82, NORTH, blsp_uart3, blsp_spi3, sd_write, _, _, _, _, _, qdss_tracedata_a),
+ [83] = PINGROUP(83, NORTH, blsp_uart3, blsp_spi3, _, _, _, _, qdss_tracedata_a, _, _),
+ [84] = PINGROUP(84, NORTH, blsp_uart3, blsp_i2c3, blsp_spi3, gcc_gp1_clk_a, qdss_cti_trig_in_b1, _, _, _, _),
+ [85] = PINGROUP(85, NORTH, blsp_uart3, blsp_i2c3, blsp_spi3, gcc_gp2_clk_a, qdss_tracedata_b, _, _, _, _),
+ [86] = PINGROUP(86, EAST, ext_mclk0, mclk_in1, _, _, _, _, _, _, _),
+ [87] = PINGROUP(87, EAST, i2s_1, dsd_clk_a, _, _, _, _, _, _, _),
+ [88] = PINGROUP(88, EAST, i2s_1, i2s_1, _, _, _, _, _, _, _),
+ [89] = PINGROUP(89, EAST, i2s_1, i2s_1, _, _, _, _, _, _, qdss_tracedata_b),
+ [90] = PINGROUP(90, EAST, i2s_1, i2s_1, _, _, _, _, _, _, _),
+ [91] = PINGROUP(91, EAST, i2s_1, i2s_1, _, _, _, _, _, _, _),
+ [92] = PINGROUP(92, EAST, i2s_1, i2s_1, _, _, _, _, _, qdss_cti_trig_in_a1, _),
+ [93] = PINGROUP(93, EAST, i2s_1, pwm_led22, i2s_1, _, _, _, _, _, qdss_tracedata_b),
+ [94] = PINGROUP(94, EAST, i2s_1, pwm_led23, i2s_1, _, qdss_cti_trig_out_a0, _, rgmi_dll2, _, _),
+ [95] = PINGROUP(95, EAST, i2s_1, pwm_led1, i2s_1, _, qdss_cti_trig_out_a1, _, _, _, _),
+ [96] = PINGROUP(96, EAST, i2s_1, pwm_led2, _, _, _, _, _, _, _),
+ [97] = PINGROUP(97, EAST, i2s_2, _, _, _, _, _, _, _, _),
+ [98] = PINGROUP(98, EAST, i2s_2, _, _, _, _, _, _, _, _),
+ [99] = PINGROUP(99, EAST, i2s_2, _, _, _, _, _, _, _, _),
+ [100] = PINGROUP(100, EAST, i2s_2, pll_bist, _, _, _, _, _, _, _),
+ [101] = PINGROUP(101, EAST, i2s_2, _, _, _, _, _, _, _, _),
+ [102] = PINGROUP(102, EAST, i2s_2, _, _, _, _, _, _, _, _),
+ [103] = PINGROUP(103, EAST, ext_mclk1_a, mclk_in2, bimc_dte1, _, _, _, _, _, _),
+ [104] = PINGROUP(104, EAST, i2s_3_sck_a, _, _, _, _, _, _, _, _),
+ [105] = PINGROUP(105, EAST, i2s_3_ws_a, _, _, _, _, _, _, _, _),
+ [106] = PINGROUP(106, EAST, i2s_3_data0_a, ebi2_lcd, _, _, ebi_cdc, _, _, _, _),
+ [107] = PINGROUP(107, EAST, i2s_3_data1_a, ebi2_lcd, _, _, ebi_cdc, _, _, _, _),
+ [108] = PINGROUP(108, EAST, i2s_3_data2_a, ebi2_lcd, atest_char, pwm_led3, ebi_cdc, _, _, _, _),
+ [109] = PINGROUP(109, EAST, i2s_3_data3_a, ebi2_lcd, pwm_led4, bimc_dte1, _, _, _, _, _),
+ [110] = PINGROUP(110, EAST, i2s_4, ebi2_a, dsd_clk_b, pwm_led5, _, _, _, _, _),
+ [111] = PINGROUP(111, EAST, i2s_4, i2s_4, pwm_led6, ebi_cdc, _, _, _, _, _),
+ [112] = PINGROUP(112, EAST, i2s_4, i2s_4, pwm_led7, _, _, _, _, _, _),
+ [113] = PINGROUP(113, EAST, i2s_4, i2s_4, pwm_led8, _, _, _, _, _, _),
+ [114] = PINGROUP(114, EAST, i2s_4, i2s_4, pwm_led24, _, _, _, _, _, _),
+ [115] = PINGROUP(115, EAST, i2s_4, i2s_4, _, _, _, _, _, _, _),
+ [116] = PINGROUP(116, EAST, i2s_4, spkr_dac0, _, _, _, _, _, _, _),
+ [117] = PINGROUP(117, NORTH, blsp_i2c4, blsp_spi4, pwm_led9, _, _, _, _, _, _),
+ [118] = PINGROUP(118, NORTH, blsp_i2c4, blsp_spi4, pwm_led10, _, _, _, _, _, _),
+ [119] = PINGROUP(119, EAST, spdifrx_opt, _, _, _, _, _, _, _, _),
+ [120] = SDC_QDSD_PINGROUP(sdc1_rclk, 0xc2000, 15, 0),
+ [121] = SDC_QDSD_PINGROUP(sdc1_clk, 0xc2000, 13, 6),
+ [122] = SDC_QDSD_PINGROUP(sdc1_cmd, 0xc2000, 11, 3),
+ [123] = SDC_QDSD_PINGROUP(sdc1_data, 0xc2000, 9, 0),
+ [124] = SDC_QDSD_PINGROUP(sdc2_clk, 0xc3000, 14, 6),
+ [125] = SDC_QDSD_PINGROUP(sdc2_cmd, 0xc3000, 11, 3),
+ [126] = SDC_QDSD_PINGROUP(sdc2_data, 0xc3000, 9, 0),
+};
+
+static const struct msm_pinctrl_soc_data qcs404_pinctrl = {
+ .pins = qcs404_pins,
+ .npins = ARRAY_SIZE(qcs404_pins),
+ .functions = qcs404_functions,
+ .nfunctions = ARRAY_SIZE(qcs404_functions),
+ .groups = qcs404_groups,
+ .ngroups = ARRAY_SIZE(qcs404_groups),
+ .ngpios = 120,
+ .tiles = qcs404_tiles,
+ .ntiles = ARRAY_SIZE(qcs404_tiles),
+};
+
+static int qcs404_pinctrl_probe(struct platform_device *pdev)
+{
+ return msm_pinctrl_probe(pdev, &qcs404_pinctrl);
+}
+
+static const struct of_device_id qcs404_pinctrl_of_match[] = {
+ { .compatible = "qcom,qcs404-pinctrl", },
+ { },
+};
+
+static struct platform_driver qcs404_pinctrl_driver = {
+ .driver = {
+ .name = "qcs404-pinctrl",
+ .of_match_table = qcs404_pinctrl_of_match,
+ },
+ .probe = qcs404_pinctrl_probe,
+ .remove = msm_pinctrl_remove,
+};
+
+static int __init qcs404_pinctrl_init(void)
+{
+ return platform_driver_register(&qcs404_pinctrl_driver);
+}
+arch_initcall(qcs404_pinctrl_init);
+
+static void __exit qcs404_pinctrl_exit(void)
+{
+ platform_driver_unregister(&qcs404_pinctrl_driver);
+}
+module_exit(qcs404_pinctrl_exit);
+
+MODULE_DESCRIPTION("Qualcomm QCS404 pinctrl driver");
+MODULE_LICENSE("GPL v2");
+MODULE_DEVICE_TABLE(of, qcs404_pinctrl_of_match);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2016, The Linux Foundation. All rights reserved.
+ * Copyright (c) 2018, Craig Tatlor.
+ */
+
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/pinctrl/pinctrl.h>
+
+#include "pinctrl-msm.h"
+
+static const char * const sdm660_tiles[] = {
+ "north",
+ "center",
+ "south"
+};
+
+enum {
+ NORTH,
+ CENTER,
+ SOUTH
+};
+
+#define REG_SIZE 0x1000
+
+#define FUNCTION(fname) \
+ [msm_mux_##fname] = { \
+ .name = #fname, \
+ .groups = fname##_groups, \
+ .ngroups = ARRAY_SIZE(fname##_groups), \
+ }
+
+
+#define PINGROUP(id, base, f1, f2, f3, f4, f5, f6, f7, f8, f9) \
+ { \
+ .name = "gpio" #id, \
+ .pins = gpio##id##_pins, \
+ .npins = (unsigned)ARRAY_SIZE(gpio##id##_pins), \
+ .funcs = (int[]){ \
+ msm_mux_gpio, /* gpio mode */ \
+ msm_mux_##f1, \
+ msm_mux_##f2, \
+ msm_mux_##f3, \
+ msm_mux_##f4, \
+ msm_mux_##f5, \
+ msm_mux_##f6, \
+ msm_mux_##f7, \
+ msm_mux_##f8, \
+ msm_mux_##f9 \
+ }, \
+ .nfuncs = 10, \
+ .ctl_reg = base + REG_SIZE * id, \
+ .io_reg = base + 0x4 + REG_SIZE * id, \
+ .intr_cfg_reg = base + 0x8 + REG_SIZE * id, \
+ .intr_status_reg = base + 0xc + REG_SIZE * id, \
+ .intr_target_reg = base + 0x8 + REG_SIZE * id, \
+ .mux_bit = 2, \
+ .pull_bit = 0, \
+ .drv_bit = 6, \
+ .oe_bit = 9, \
+ .in_bit = 0, \
+ .out_bit = 1, \
+ .intr_enable_bit = 0, \
+ .intr_status_bit = 0, \
+ .intr_target_bit = 5, \
+ .intr_target_kpss_val = 3, \
+ .intr_raw_status_bit = 4, \
+ .intr_polarity_bit = 1, \
+ .intr_detection_bit = 2, \
+ .intr_detection_width = 2, \
+ }
+
+#define SDC_QDSD_PINGROUP(pg_name, ctl, pull, drv) \
+ { \
+ .name = #pg_name, \
+ .pins = pg_name##_pins, \
+ .npins = (unsigned)ARRAY_SIZE(pg_name##_pins), \
+ .ctl_reg = ctl, \
+ .io_reg = 0, \
+ .intr_cfg_reg = 0, \
+ .intr_status_reg = 0, \
+ .intr_target_reg = 0, \
+ .mux_bit = -1, \
+ .pull_bit = pull, \
+ .drv_bit = drv, \
+ .oe_bit = -1, \
+ .in_bit = -1, \
+ .out_bit = -1, \
+ .intr_enable_bit = -1, \
+ .intr_status_bit = -1, \
+ .intr_target_bit = -1, \
+ .intr_raw_status_bit = -1, \
+ .intr_polarity_bit = -1, \
+ .intr_detection_bit = -1, \
+ .intr_detection_width = -1, \
+ }
+
+static const struct pinctrl_pin_desc sdm660_pins[] = {
+ PINCTRL_PIN(0, "GPIO_0"),
+ PINCTRL_PIN(1, "GPIO_1"),
+ PINCTRL_PIN(2, "GPIO_2"),
+ PINCTRL_PIN(3, "GPIO_3"),
+ PINCTRL_PIN(4, "GPIO_4"),
+ PINCTRL_PIN(5, "GPIO_5"),
+ PINCTRL_PIN(6, "GPIO_6"),
+ PINCTRL_PIN(7, "GPIO_7"),
+ PINCTRL_PIN(8, "GPIO_8"),
+ PINCTRL_PIN(9, "GPIO_9"),
+ PINCTRL_PIN(10, "GPIO_10"),
+ PINCTRL_PIN(11, "GPIO_11"),
+ PINCTRL_PIN(12, "GPIO_12"),
+ PINCTRL_PIN(13, "GPIO_13"),
+ PINCTRL_PIN(14, "GPIO_14"),
+ PINCTRL_PIN(15, "GPIO_15"),
+ PINCTRL_PIN(16, "GPIO_16"),
+ PINCTRL_PIN(17, "GPIO_17"),
+ PINCTRL_PIN(18, "GPIO_18"),
+ PINCTRL_PIN(19, "GPIO_19"),
+ PINCTRL_PIN(20, "GPIO_20"),
+ PINCTRL_PIN(21, "GPIO_21"),
+ PINCTRL_PIN(22, "GPIO_22"),
+ PINCTRL_PIN(23, "GPIO_23"),
+ PINCTRL_PIN(24, "GPIO_24"),
+ PINCTRL_PIN(25, "GPIO_25"),
+ PINCTRL_PIN(26, "GPIO_26"),
+ PINCTRL_PIN(27, "GPIO_27"),
+ PINCTRL_PIN(28, "GPIO_28"),
+ PINCTRL_PIN(29, "GPIO_29"),
+ PINCTRL_PIN(30, "GPIO_30"),
+ PINCTRL_PIN(31, "GPIO_31"),
+ PINCTRL_PIN(32, "GPIO_32"),
+ PINCTRL_PIN(33, "GPIO_33"),
+ PINCTRL_PIN(34, "GPIO_34"),
+ PINCTRL_PIN(35, "GPIO_35"),
+ PINCTRL_PIN(36, "GPIO_36"),
+ PINCTRL_PIN(37, "GPIO_37"),
+ PINCTRL_PIN(38, "GPIO_38"),
+ PINCTRL_PIN(39, "GPIO_39"),
+ PINCTRL_PIN(40, "GPIO_40"),
+ PINCTRL_PIN(41, "GPIO_41"),
+ PINCTRL_PIN(42, "GPIO_42"),
+ PINCTRL_PIN(43, "GPIO_43"),
+ PINCTRL_PIN(44, "GPIO_44"),
+ PINCTRL_PIN(45, "GPIO_45"),
+ PINCTRL_PIN(46, "GPIO_46"),
+ PINCTRL_PIN(47, "GPIO_47"),
+ PINCTRL_PIN(48, "GPIO_48"),
+ PINCTRL_PIN(49, "GPIO_49"),
+ PINCTRL_PIN(50, "GPIO_50"),
+ PINCTRL_PIN(51, "GPIO_51"),
+ PINCTRL_PIN(52, "GPIO_52"),
+ PINCTRL_PIN(53, "GPIO_53"),
+ PINCTRL_PIN(54, "GPIO_54"),
+ PINCTRL_PIN(55, "GPIO_55"),
+ PINCTRL_PIN(56, "GPIO_56"),
+ PINCTRL_PIN(57, "GPIO_57"),
+ PINCTRL_PIN(58, "GPIO_58"),
+ PINCTRL_PIN(59, "GPIO_59"),
+ PINCTRL_PIN(60, "GPIO_60"),
+ PINCTRL_PIN(61, "GPIO_61"),
+ PINCTRL_PIN(62, "GPIO_62"),
+ PINCTRL_PIN(63, "GPIO_63"),
+ PINCTRL_PIN(64, "GPIO_64"),
+ PINCTRL_PIN(65, "GPIO_65"),
+ PINCTRL_PIN(66, "GPIO_66"),
+ PINCTRL_PIN(67, "GPIO_67"),
+ PINCTRL_PIN(68, "GPIO_68"),
+ PINCTRL_PIN(69, "GPIO_69"),
+ PINCTRL_PIN(70, "GPIO_70"),
+ PINCTRL_PIN(71, "GPIO_71"),
+ PINCTRL_PIN(72, "GPIO_72"),
+ PINCTRL_PIN(73, "GPIO_73"),
+ PINCTRL_PIN(74, "GPIO_74"),
+ PINCTRL_PIN(75, "GPIO_75"),
+ PINCTRL_PIN(76, "GPIO_76"),
+ PINCTRL_PIN(77, "GPIO_77"),
+ PINCTRL_PIN(78, "GPIO_78"),
+ PINCTRL_PIN(79, "GPIO_79"),
+ PINCTRL_PIN(80, "GPIO_80"),
+ PINCTRL_PIN(81, "GPIO_81"),
+ PINCTRL_PIN(82, "GPIO_82"),
+ PINCTRL_PIN(83, "GPIO_83"),
+ PINCTRL_PIN(84, "GPIO_84"),
+ PINCTRL_PIN(85, "GPIO_85"),
+ PINCTRL_PIN(86, "GPIO_86"),
+ PINCTRL_PIN(87, "GPIO_87"),
+ PINCTRL_PIN(88, "GPIO_88"),
+ PINCTRL_PIN(89, "GPIO_89"),
+ PINCTRL_PIN(90, "GPIO_90"),
+ PINCTRL_PIN(91, "GPIO_91"),
+ PINCTRL_PIN(92, "GPIO_92"),
+ PINCTRL_PIN(93, "GPIO_93"),
+ PINCTRL_PIN(94, "GPIO_94"),
+ PINCTRL_PIN(95, "GPIO_95"),
+ PINCTRL_PIN(96, "GPIO_96"),
+ PINCTRL_PIN(97, "GPIO_97"),
+ PINCTRL_PIN(98, "GPIO_98"),
+ PINCTRL_PIN(99, "GPIO_99"),
+ PINCTRL_PIN(100, "GPIO_100"),
+ PINCTRL_PIN(101, "GPIO_101"),
+ PINCTRL_PIN(102, "GPIO_102"),
+ PINCTRL_PIN(103, "GPIO_103"),
+ PINCTRL_PIN(104, "GPIO_104"),
+ PINCTRL_PIN(105, "GPIO_105"),
+ PINCTRL_PIN(106, "GPIO_106"),
+ PINCTRL_PIN(107, "GPIO_107"),
+ PINCTRL_PIN(108, "GPIO_108"),
+ PINCTRL_PIN(109, "GPIO_109"),
+ PINCTRL_PIN(110, "GPIO_110"),
+ PINCTRL_PIN(111, "GPIO_111"),
+ PINCTRL_PIN(112, "GPIO_112"),
+ PINCTRL_PIN(113, "GPIO_113"),
+ PINCTRL_PIN(114, "SDC1_CLK"),
+ PINCTRL_PIN(115, "SDC1_CMD"),
+ PINCTRL_PIN(116, "SDC1_DATA"),
+ PINCTRL_PIN(117, "SDC2_CLK"),
+ PINCTRL_PIN(118, "SDC2_CMD"),
+ PINCTRL_PIN(119, "SDC2_DATA"),
+ PINCTRL_PIN(120, "SDC1_RCLK"),
+};
+
+#define DECLARE_MSM_GPIO_PINS(pin) \
+ static const unsigned int gpio##pin##_pins[] = { pin }
+DECLARE_MSM_GPIO_PINS(0);
+DECLARE_MSM_GPIO_PINS(1);
+DECLARE_MSM_GPIO_PINS(2);
+DECLARE_MSM_GPIO_PINS(3);
+DECLARE_MSM_GPIO_PINS(4);
+DECLARE_MSM_GPIO_PINS(5);
+DECLARE_MSM_GPIO_PINS(6);
+DECLARE_MSM_GPIO_PINS(7);
+DECLARE_MSM_GPIO_PINS(8);
+DECLARE_MSM_GPIO_PINS(9);
+DECLARE_MSM_GPIO_PINS(10);
+DECLARE_MSM_GPIO_PINS(11);
+DECLARE_MSM_GPIO_PINS(12);
+DECLARE_MSM_GPIO_PINS(13);
+DECLARE_MSM_GPIO_PINS(14);
+DECLARE_MSM_GPIO_PINS(15);
+DECLARE_MSM_GPIO_PINS(16);
+DECLARE_MSM_GPIO_PINS(17);
+DECLARE_MSM_GPIO_PINS(18);
+DECLARE_MSM_GPIO_PINS(19);
+DECLARE_MSM_GPIO_PINS(20);
+DECLARE_MSM_GPIO_PINS(21);
+DECLARE_MSM_GPIO_PINS(22);
+DECLARE_MSM_GPIO_PINS(23);
+DECLARE_MSM_GPIO_PINS(24);
+DECLARE_MSM_GPIO_PINS(25);
+DECLARE_MSM_GPIO_PINS(26);
+DECLARE_MSM_GPIO_PINS(27);
+DECLARE_MSM_GPIO_PINS(28);
+DECLARE_MSM_GPIO_PINS(29);
+DECLARE_MSM_GPIO_PINS(30);
+DECLARE_MSM_GPIO_PINS(31);
+DECLARE_MSM_GPIO_PINS(32);
+DECLARE_MSM_GPIO_PINS(33);
+DECLARE_MSM_GPIO_PINS(34);
+DECLARE_MSM_GPIO_PINS(35);
+DECLARE_MSM_GPIO_PINS(36);
+DECLARE_MSM_GPIO_PINS(37);
+DECLARE_MSM_GPIO_PINS(38);
+DECLARE_MSM_GPIO_PINS(39);
+DECLARE_MSM_GPIO_PINS(40);
+DECLARE_MSM_GPIO_PINS(41);
+DECLARE_MSM_GPIO_PINS(42);
+DECLARE_MSM_GPIO_PINS(43);
+DECLARE_MSM_GPIO_PINS(44);
+DECLARE_MSM_GPIO_PINS(45);
+DECLARE_MSM_GPIO_PINS(46);
+DECLARE_MSM_GPIO_PINS(47);
+DECLARE_MSM_GPIO_PINS(48);
+DECLARE_MSM_GPIO_PINS(49);
+DECLARE_MSM_GPIO_PINS(50);
+DECLARE_MSM_GPIO_PINS(51);
+DECLARE_MSM_GPIO_PINS(52);
+DECLARE_MSM_GPIO_PINS(53);
+DECLARE_MSM_GPIO_PINS(54);
+DECLARE_MSM_GPIO_PINS(55);
+DECLARE_MSM_GPIO_PINS(56);
+DECLARE_MSM_GPIO_PINS(57);
+DECLARE_MSM_GPIO_PINS(58);
+DECLARE_MSM_GPIO_PINS(59);
+DECLARE_MSM_GPIO_PINS(60);
+DECLARE_MSM_GPIO_PINS(61);
+DECLARE_MSM_GPIO_PINS(62);
+DECLARE_MSM_GPIO_PINS(63);
+DECLARE_MSM_GPIO_PINS(64);
+DECLARE_MSM_GPIO_PINS(65);
+DECLARE_MSM_GPIO_PINS(66);
+DECLARE_MSM_GPIO_PINS(67);
+DECLARE_MSM_GPIO_PINS(68);
+DECLARE_MSM_GPIO_PINS(69);
+DECLARE_MSM_GPIO_PINS(70);
+DECLARE_MSM_GPIO_PINS(71);
+DECLARE_MSM_GPIO_PINS(72);
+DECLARE_MSM_GPIO_PINS(73);
+DECLARE_MSM_GPIO_PINS(74);
+DECLARE_MSM_GPIO_PINS(75);
+DECLARE_MSM_GPIO_PINS(76);
+DECLARE_MSM_GPIO_PINS(77);
+DECLARE_MSM_GPIO_PINS(78);
+DECLARE_MSM_GPIO_PINS(79);
+DECLARE_MSM_GPIO_PINS(80);
+DECLARE_MSM_GPIO_PINS(81);
+DECLARE_MSM_GPIO_PINS(82);
+DECLARE_MSM_GPIO_PINS(83);
+DECLARE_MSM_GPIO_PINS(84);
+DECLARE_MSM_GPIO_PINS(85);
+DECLARE_MSM_GPIO_PINS(86);
+DECLARE_MSM_GPIO_PINS(87);
+DECLARE_MSM_GPIO_PINS(88);
+DECLARE_MSM_GPIO_PINS(89);
+DECLARE_MSM_GPIO_PINS(90);
+DECLARE_MSM_GPIO_PINS(91);
+DECLARE_MSM_GPIO_PINS(92);
+DECLARE_MSM_GPIO_PINS(93);
+DECLARE_MSM_GPIO_PINS(94);
+DECLARE_MSM_GPIO_PINS(95);
+DECLARE_MSM_GPIO_PINS(96);
+DECLARE_MSM_GPIO_PINS(97);
+DECLARE_MSM_GPIO_PINS(98);
+DECLARE_MSM_GPIO_PINS(99);
+DECLARE_MSM_GPIO_PINS(100);
+DECLARE_MSM_GPIO_PINS(101);
+DECLARE_MSM_GPIO_PINS(102);
+DECLARE_MSM_GPIO_PINS(103);
+DECLARE_MSM_GPIO_PINS(104);
+DECLARE_MSM_GPIO_PINS(105);
+DECLARE_MSM_GPIO_PINS(106);
+DECLARE_MSM_GPIO_PINS(107);
+DECLARE_MSM_GPIO_PINS(108);
+DECLARE_MSM_GPIO_PINS(109);
+DECLARE_MSM_GPIO_PINS(110);
+DECLARE_MSM_GPIO_PINS(111);
+DECLARE_MSM_GPIO_PINS(112);
+DECLARE_MSM_GPIO_PINS(113);
+
+static const unsigned int sdc1_clk_pins[] = { 114 };
+static const unsigned int sdc1_cmd_pins[] = { 115 };
+static const unsigned int sdc1_data_pins[] = { 116 };
+static const unsigned int sdc1_rclk_pins[] = { 120 };
+static const unsigned int sdc2_clk_pins[] = { 117 };
+static const unsigned int sdc2_cmd_pins[] = { 118 };
+static const unsigned int sdc2_data_pins[] = { 119 };
+
+enum sdm660_functions {
+ msm_mux_adsp_ext,
+ msm_mux_agera_pll,
+ msm_mux_atest_char,
+ msm_mux_atest_char0,
+ msm_mux_atest_char1,
+ msm_mux_atest_char2,
+ msm_mux_atest_char3,
+ msm_mux_atest_gpsadc0,
+ msm_mux_atest_gpsadc1,
+ msm_mux_atest_tsens,
+ msm_mux_atest_tsens2,
+ msm_mux_atest_usb1,
+ msm_mux_atest_usb10,
+ msm_mux_atest_usb11,
+ msm_mux_atest_usb12,
+ msm_mux_atest_usb13,
+ msm_mux_atest_usb2,
+ msm_mux_atest_usb20,
+ msm_mux_atest_usb21,
+ msm_mux_atest_usb22,
+ msm_mux_atest_usb23,
+ msm_mux_audio_ref,
+ msm_mux_bimc_dte0,
+ msm_mux_bimc_dte1,
+ msm_mux_blsp_i2c1,
+ msm_mux_blsp_i2c2,
+ msm_mux_blsp_i2c3,
+ msm_mux_blsp_i2c4,
+ msm_mux_blsp_i2c5,
+ msm_mux_blsp_i2c6,
+ msm_mux_blsp_i2c7,
+ msm_mux_blsp_i2c8_a,
+ msm_mux_blsp_i2c8_b,
+ msm_mux_blsp_spi1,
+ msm_mux_blsp_spi2,
+ msm_mux_blsp_spi3,
+ msm_mux_blsp_spi3_cs1,
+ msm_mux_blsp_spi3_cs2,
+ msm_mux_blsp_spi4,
+ msm_mux_blsp_spi5,
+ msm_mux_blsp_spi6,
+ msm_mux_blsp_spi7,
+ msm_mux_blsp_spi8_a,
+ msm_mux_blsp_spi8_b,
+ msm_mux_blsp_spi8_cs1,
+ msm_mux_blsp_spi8_cs2,
+ msm_mux_blsp_uart1,
+ msm_mux_blsp_uart2,
+ msm_mux_blsp_uart5,
+ msm_mux_blsp_uart6_a,
+ msm_mux_blsp_uart6_b,
+ msm_mux_blsp_uim1,
+ msm_mux_blsp_uim2,
+ msm_mux_blsp_uim5,
+ msm_mux_blsp_uim6,
+ msm_mux_cam_mclk,
+ msm_mux_cci_async,
+ msm_mux_cci_i2c,
+ msm_mux_cri_trng,
+ msm_mux_cri_trng0,
+ msm_mux_cri_trng1,
+ msm_mux_dbg_out,
+ msm_mux_ddr_bist,
+ msm_mux_gcc_gp1,
+ msm_mux_gcc_gp2,
+ msm_mux_gcc_gp3,
+ msm_mux_gpio,
+ msm_mux_gps_tx_a,
+ msm_mux_gps_tx_b,
+ msm_mux_gps_tx_c,
+ msm_mux_isense_dbg,
+ msm_mux_jitter_bist,
+ msm_mux_ldo_en,
+ msm_mux_ldo_update,
+ msm_mux_m_voc,
+ msm_mux_mdp_vsync,
+ msm_mux_mdss_vsync0,
+ msm_mux_mdss_vsync1,
+ msm_mux_mdss_vsync2,
+ msm_mux_mdss_vsync3,
+ msm_mux_mss_lte,
+ msm_mux_nav_pps_a,
+ msm_mux_nav_pps_b,
+ msm_mux_nav_pps_c,
+ msm_mux_pa_indicator,
+ msm_mux_phase_flag0,
+ msm_mux_phase_flag1,
+ msm_mux_phase_flag2,
+ msm_mux_phase_flag3,
+ msm_mux_phase_flag4,
+ msm_mux_phase_flag5,
+ msm_mux_phase_flag6,
+ msm_mux_phase_flag7,
+ msm_mux_phase_flag8,
+ msm_mux_phase_flag9,
+ msm_mux_phase_flag10,
+ msm_mux_phase_flag11,
+ msm_mux_phase_flag12,
+ msm_mux_phase_flag13,
+ msm_mux_phase_flag14,
+ msm_mux_phase_flag15,
+ msm_mux_phase_flag16,
+ msm_mux_phase_flag17,
+ msm_mux_phase_flag18,
+ msm_mux_phase_flag19,
+ msm_mux_phase_flag20,
+ msm_mux_phase_flag21,
+ msm_mux_phase_flag22,
+ msm_mux_phase_flag23,
+ msm_mux_phase_flag24,
+ msm_mux_phase_flag25,
+ msm_mux_phase_flag26,
+ msm_mux_phase_flag27,
+ msm_mux_phase_flag28,
+ msm_mux_phase_flag29,
+ msm_mux_phase_flag30,
+ msm_mux_phase_flag31,
+ msm_mux_pll_bypassnl,
+ msm_mux_pll_reset,
+ msm_mux_pri_mi2s,
+ msm_mux_pri_mi2s_ws,
+ msm_mux_prng_rosc,
+ msm_mux_pwr_crypto,
+ msm_mux_pwr_modem,
+ msm_mux_pwr_nav,
+ msm_mux_qdss_cti0_a,
+ msm_mux_qdss_cti0_b,
+ msm_mux_qdss_cti1_a,
+ msm_mux_qdss_cti1_b,
+ msm_mux_qdss_gpio,
+ msm_mux_qdss_gpio0,
+ msm_mux_qdss_gpio1,
+ msm_mux_qdss_gpio10,
+ msm_mux_qdss_gpio11,
+ msm_mux_qdss_gpio12,
+ msm_mux_qdss_gpio13,
+ msm_mux_qdss_gpio14,
+ msm_mux_qdss_gpio15,
+ msm_mux_qdss_gpio2,
+ msm_mux_qdss_gpio3,
+ msm_mux_qdss_gpio4,
+ msm_mux_qdss_gpio5,
+ msm_mux_qdss_gpio6,
+ msm_mux_qdss_gpio7,
+ msm_mux_qdss_gpio8,
+ msm_mux_qdss_gpio9,
+ msm_mux_qlink_enable,
+ msm_mux_qlink_request,
+ msm_mux_qspi_clk,
+ msm_mux_qspi_cs,
+ msm_mux_qspi_data0,
+ msm_mux_qspi_data1,
+ msm_mux_qspi_data2,
+ msm_mux_qspi_data3,
+ msm_mux_qspi_resetn,
+ msm_mux_sec_mi2s,
+ msm_mux_sndwire_clk,
+ msm_mux_sndwire_data,
+ msm_mux_sp_cmu,
+ msm_mux_ssc_irq,
+ msm_mux_tgu_ch0,
+ msm_mux_tgu_ch1,
+ msm_mux_tsense_pwm1,
+ msm_mux_tsense_pwm2,
+ msm_mux_uim1_clk,
+ msm_mux_uim1_data,
+ msm_mux_uim1_present,
+ msm_mux_uim1_reset,
+ msm_mux_uim2_clk,
+ msm_mux_uim2_data,
+ msm_mux_uim2_present,
+ msm_mux_uim2_reset,
+ msm_mux_uim_batt,
+ msm_mux_vfr_1,
+ msm_mux_vsense_clkout,
+ msm_mux_vsense_data0,
+ msm_mux_vsense_data1,
+ msm_mux_vsense_mode,
+ msm_mux_wlan1_adc0,
+ msm_mux_wlan1_adc1,
+ msm_mux_wlan2_adc0,
+ msm_mux_wlan2_adc1,
+ msm_mux__,
+};
+
+static const char * const gpio_groups[] = {
+ "gpio0", "gpio1", "gpio2", "gpio3", "gpio4", "gpio5", "gpio6", "gpio7",
+ "gpio8", "gpio9", "gpio10", "gpio11", "gpio12", "gpio13", "gpio14",
+ "gpio15", "gpio16", "gpio17", "gpio18", "gpio19", "gpio20", "gpio21",
+ "gpio22", "gpio23", "gpio24", "gpio25", "gpio26", "gpio27", "gpio28",
+ "gpio29", "gpio30", "gpio31", "gpio32", "gpio33", "gpio34", "gpio35",
+ "gpio36", "gpio37", "gpio38", "gpio39", "gpio40", "gpio41", "gpio42",
+ "gpio43", "gpio44", "gpio45", "gpio46", "gpio47", "gpio48", "gpio49",
+ "gpio50", "gpio51", "gpio52", "gpio53", "gpio54", "gpio55", "gpio56",
+ "gpio57", "gpio58", "gpio59", "gpio60", "gpio61", "gpio62", "gpio63",
+ "gpio64", "gpio65", "gpio66", "gpio67", "gpio68", "gpio69", "gpio70",
+ "gpio71", "gpio72", "gpio73", "gpio74", "gpio75", "gpio76", "gpio77",
+ "gpio78", "gpio79", "gpio80", "gpio81", "gpio82", "gpio83", "gpio84",
+ "gpio85", "gpio86", "gpio87", "gpio88", "gpio89", "gpio90", "gpio91",
+ "gpio92", "gpio93", "gpio94", "gpio95", "gpio96", "gpio97", "gpio98",
+ "gpio99", "gpio100", "gpio101", "gpio102", "gpio103", "gpio104",
+ "gpio105", "gpio106", "gpio107", "gpio108", "gpio109", "gpio110",
+ "gpio111", "gpio112", "gpio113",
+};
+
+static const char * const adsp_ext_groups[] = {
+ "gpio65",
+};
+static const char * const agera_pll_groups[] = {
+ "gpio34", "gpio36",
+};
+static const char * const atest_char0_groups[] = {
+ "gpio62",
+};
+static const char * const atest_char1_groups[] = {
+ "gpio61",
+};
+static const char * const atest_char2_groups[] = {
+ "gpio60",
+};
+static const char * const atest_char3_groups[] = {
+ "gpio59",
+};
+static const char * const atest_char_groups[] = {
+ "gpio58",
+};
+static const char * const atest_gpsadc0_groups[] = {
+ "gpio1",
+};
+static const char * const atest_gpsadc1_groups[] = {
+ "gpio0",
+};
+static const char * const atest_tsens2_groups[] = {
+ "gpio3",
+};
+static const char * const atest_tsens_groups[] = {
+ "gpio36",
+};
+static const char * const atest_usb10_groups[] = {
+ "gpio11",
+};
+static const char * const atest_usb11_groups[] = {
+ "gpio10",
+};
+static const char * const atest_usb12_groups[] = {
+ "gpio9",
+};
+static const char * const atest_usb13_groups[] = {
+ "gpio8",
+};
+static const char * const atest_usb1_groups[] = {
+ "gpio3",
+};
+static const char * const atest_usb20_groups[] = {
+ "gpio56",
+};
+static const char * const atest_usb21_groups[] = {
+ "gpio36",
+};
+static const char * const atest_usb22_groups[] = {
+ "gpio57",
+};
+static const char * const atest_usb23_groups[] = {
+ "gpio37",
+};
+static const char * const atest_usb2_groups[] = {
+ "gpio35",
+};
+static const char * const audio_ref_groups[] = {
+ "gpio62",
+};
+static const char * const bimc_dte0_groups[] = {
+ "gpio9", "gpio11",
+};
+static const char * const bimc_dte1_groups[] = {
+ "gpio8", "gpio10",
+};
+static const char * const blsp_i2c1_groups[] = {
+ "gpio2", "gpio3",
+};
+static const char * const blsp_i2c2_groups[] = {
+ "gpio6", "gpio7",
+};
+static const char * const blsp_i2c3_groups[] = {
+ "gpio10", "gpio11",
+};
+static const char * const blsp_i2c4_groups[] = {
+ "gpio14", "gpio15",
+};
+static const char * const blsp_i2c5_groups[] = {
+ "gpio18", "gpio19",
+};
+static const char * const blsp_i2c6_groups[] = {
+ "gpio22", "gpio23",
+};
+static const char * const blsp_i2c7_groups[] = {
+ "gpio26", "gpio27",
+};
+static const char * const blsp_i2c8_a_groups[] = {
+ "gpio30", "gpio31",
+};
+static const char * const blsp_i2c8_b_groups[] = {
+ "gpio44", "gpio52",
+};
+static const char * const blsp_spi1_groups[] = {
+ "gpio0", "gpio1", "gpio2", "gpio3", "gpio46",
+};
+static const char * const blsp_spi2_groups[] = {
+ "gpio4", "gpio5", "gpio6", "gpio7",
+};
+static const char * const blsp_spi3_cs1_groups[] = {
+ "gpio30",
+};
+static const char * const blsp_spi3_cs2_groups[] = {
+ "gpio65",
+};
+static const char * const blsp_spi3_groups[] = {
+ "gpio8", "gpio9", "gpio10", "gpio11",
+};
+static const char * const blsp_spi4_groups[] = {
+ "gpio12", "gpio13", "gpio14", "gpio15",
+};
+static const char * const blsp_spi5_groups[] = {
+ "gpio16", "gpio17", "gpio18", "gpio19",
+};
+static const char * const blsp_spi6_groups[] = {
+ "gpio49", "gpio52", "gpio22", "gpio23",
+};
+static const char * const blsp_spi7_groups[] = {
+ "gpio24", "gpio25", "gpio26", "gpio27",
+};
+static const char * const blsp_spi8_a_groups[] = {
+ "gpio28", "gpio29", "gpio30", "gpio31",
+};
+static const char * const blsp_spi8_b_groups[] = {
+ "gpio40", "gpio41", "gpio44", "gpio52",
+};
+static const char * const blsp_spi8_cs1_groups[] = {
+ "gpio64",
+};
+static const char * const blsp_spi8_cs2_groups[] = {
+ "gpio76",
+};
+static const char * const blsp_uart1_groups[] = {
+ "gpio0", "gpio1", "gpio2", "gpio3",
+};
+static const char * const blsp_uart2_groups[] = {
+ "gpio4", "gpio5", "gpio6", "gpio7",
+};
+static const char * const blsp_uart5_groups[] = {
+ "gpio16", "gpio17", "gpio18", "gpio19",
+};
+static const char * const blsp_uart6_a_groups[] = {
+ "gpio24", "gpio25", "gpio26", "gpio27",
+};
+static const char * const blsp_uart6_b_groups[] = {
+ "gpio28", "gpio29", "gpio30", "gpio31",
+};
+static const char * const blsp_uim1_groups[] = {
+ "gpio0", "gpio1",
+};
+static const char * const blsp_uim2_groups[] = {
+ "gpio4", "gpio5",
+};
+static const char * const blsp_uim5_groups[] = {
+ "gpio16", "gpio17",
+};
+static const char * const blsp_uim6_groups[] = {
+ "gpio20", "gpio21",
+};
+static const char * const cam_mclk_groups[] = {
+ "gpio32", "gpio33", "gpio34", "gpio35",
+};
+static const char * const cci_async_groups[] = {
+ "gpio45",
+};
+static const char * const cci_i2c_groups[] = {
+ "gpio36", "gpio37", "gpio38", "gpio39",
+};
+static const char * const cri_trng0_groups[] = {
+ "gpio60",
+};
+static const char * const cri_trng1_groups[] = {
+ "gpio61",
+};
+static const char * const cri_trng_groups[] = {
+ "gpio62",
+};
+static const char * const dbg_out_groups[] = {
+ "gpio11",
+};
+static const char * const ddr_bist_groups[] = {
+ "gpio3", "gpio8", "gpio9", "gpio10",
+};
+static const char * const gcc_gp1_groups[] = {
+ "gpio57", "gpio78",
+};
+static const char * const gcc_gp2_groups[] = {
+ "gpio58", "gpio81",
+};
+static const char * const gcc_gp3_groups[] = {
+ "gpio59", "gpio82",
+};
+static const char * const gps_tx_a_groups[] = {
+ "gpio65",
+};
+static const char * const gps_tx_b_groups[] = {
+ "gpio98",
+};
+static const char * const gps_tx_c_groups[] = {
+ "gpio80",
+};
+static const char * const isense_dbg_groups[] = {
+ "gpio68",
+};
+static const char * const jitter_bist_groups[] = {
+ "gpio35",
+};
+static const char * const ldo_en_groups[] = {
+ "gpio97",
+};
+static const char * const ldo_update_groups[] = {
+ "gpio98",
+};
+static const char * const m_voc_groups[] = {
+ "gpio28",
+};
+static const char * const mdp_vsync_groups[] = {
+ "gpio59", "gpio74",
+};
+static const char * const mdss_vsync0_groups[] = {
+ "gpio42",
+};
+static const char * const mdss_vsync1_groups[] = {
+ "gpio42",
+};
+static const char * const mdss_vsync2_groups[] = {
+ "gpio42",
+};
+static const char * const mdss_vsync3_groups[] = {
+ "gpio42",
+};
+static const char * const mss_lte_groups[] = {
+ "gpio81", "gpio82",
+};
+static const char * const nav_pps_a_groups[] = {
+ "gpio65",
+};
+static const char * const nav_pps_b_groups[] = {
+ "gpio98",
+};
+static const char * const nav_pps_c_groups[] = {
+ "gpio80",
+};
+static const char * const pa_indicator_groups[] = {
+ "gpio92",
+};
+static const char * const phase_flag0_groups[] = {
+ "gpio68",
+};
+static const char * const phase_flag1_groups[] = {
+ "gpio48",
+};
+static const char * const phase_flag2_groups[] = {
+ "gpio49",
+};
+static const char * const phase_flag3_groups[] = {
+ "gpio4",
+};
+static const char * const phase_flag4_groups[] = {
+ "gpio57",
+};
+static const char * const phase_flag5_groups[] = {
+ "gpio17",
+};
+static const char * const phase_flag6_groups[] = {
+ "gpio53",
+};
+static const char * const phase_flag7_groups[] = {
+ "gpio69",
+};
+static const char * const phase_flag8_groups[] = {
+ "gpio70",
+};
+static const char * const phase_flag9_groups[] = {
+ "gpio50",
+};
+static const char * const phase_flag10_groups[] = {
+ "gpio56",
+};
+static const char * const phase_flag11_groups[] = {
+ "gpio21",
+};
+static const char * const phase_flag12_groups[] = {
+ "gpio22",
+};
+static const char * const phase_flag13_groups[] = {
+ "gpio23",
+};
+static const char * const phase_flag14_groups[] = {
+ "gpio5",
+};
+static const char * const phase_flag15_groups[] = {
+ "gpio51",
+};
+static const char * const phase_flag16_groups[] = {
+ "gpio52",
+};
+static const char * const phase_flag17_groups[] = {
+ "gpio24",
+};
+static const char * const phase_flag18_groups[] = {
+ "gpio25",
+};
+static const char * const phase_flag19_groups[] = {
+ "gpio26",
+};
+static const char * const phase_flag20_groups[] = {
+ "gpio27",
+};
+static const char * const phase_flag21_groups[] = {
+ "gpio28",
+};
+static const char * const phase_flag22_groups[] = {
+ "gpio29",
+};
+static const char * const phase_flag23_groups[] = {
+ "gpio30",
+};
+static const char * const phase_flag24_groups[] = {
+ "gpio31",
+};
+static const char * const phase_flag25_groups[] = {
+ "gpio55",
+};
+static const char * const phase_flag26_groups[] = {
+ "gpio12",
+};
+static const char * const phase_flag27_groups[] = {
+ "gpio13",
+};
+static const char * const phase_flag28_groups[] = {
+ "gpio14",
+};
+static const char * const phase_flag29_groups[] = {
+ "gpio54",
+};
+static const char * const phase_flag30_groups[] = {
+ "gpio47",
+};
+static const char * const phase_flag31_groups[] = {
+ "gpio6",
+};
+static const char * const pll_bypassnl_groups[] = {
+ "gpio36",
+};
+static const char * const pll_reset_groups[] = {
+ "gpio37",
+};
+static const char * const pri_mi2s_groups[] = {
+ "gpio12", "gpio14", "gpio15", "gpio61",
+};
+static const char * const pri_mi2s_ws_groups[] = {
+ "gpio13",
+};
+static const char * const prng_rosc_groups[] = {
+ "gpio102",
+};
+static const char * const pwr_crypto_groups[] = {
+ "gpio33",
+};
+static const char * const pwr_modem_groups[] = {
+ "gpio31",
+};
+static const char * const pwr_nav_groups[] = {
+ "gpio32",
+};
+static const char * const qdss_cti0_a_groups[] = {
+ "gpio49", "gpio50",
+};
+static const char * const qdss_cti0_b_groups[] = {
+ "gpio13", "gpio21",
+};
+static const char * const qdss_cti1_a_groups[] = {
+ "gpio53", "gpio55",
+};
+static const char * const qdss_cti1_b_groups[] = {
+ "gpio12", "gpio66",
+};
+static const char * const qdss_gpio0_groups[] = {
+ "gpio32", "gpio67",
+};
+static const char * const qdss_gpio10_groups[] = {
+ "gpio43", "gpio77",
+};
+static const char * const qdss_gpio11_groups[] = {
+ "gpio44", "gpio79",
+};
+static const char * const qdss_gpio12_groups[] = {
+ "gpio45", "gpio80",
+};
+static const char * const qdss_gpio13_groups[] = {
+ "gpio46", "gpio78",
+};
+static const char * const qdss_gpio14_groups[] = {
+ "gpio47", "gpio72",
+};
+static const char * const qdss_gpio15_groups[] = {
+ "gpio48", "gpio73",
+};
+static const char * const qdss_gpio1_groups[] = {
+ "gpio33", "gpio63",
+};
+static const char * const qdss_gpio2_groups[] = {
+ "gpio34", "gpio64",
+};
+static const char * const qdss_gpio3_groups[] = {
+ "gpio35", "gpio56",
+};
+static const char * const qdss_gpio4_groups[] = {
+ "gpio0", "gpio36",
+};
+static const char * const qdss_gpio5_groups[] = {
+ "gpio1", "gpio37",
+};
+static const char * const qdss_gpio6_groups[] = {
+ "gpio38", "gpio70",
+};
+static const char * const qdss_gpio7_groups[] = {
+ "gpio39", "gpio71",
+};
+static const char * const qdss_gpio8_groups[] = {
+ "gpio51", "gpio75",
+};
+static const char * const qdss_gpio9_groups[] = {
+ "gpio42", "gpio76",
+};
+static const char * const qdss_gpio_groups[] = {
+ "gpio31", "gpio52", "gpio68", "gpio69",
+};
+static const char * const qlink_enable_groups[] = {
+ "gpio100",
+};
+static const char * const qlink_request_groups[] = {
+ "gpio99",
+};
+static const char * const qspi_clk_groups[] = {
+ "gpio47",
+};
+static const char * const qspi_cs_groups[] = {
+ "gpio43", "gpio50",
+};
+static const char * const qspi_data0_groups[] = {
+ "gpio33",
+};
+static const char * const qspi_data1_groups[] = {
+ "gpio34",
+};
+static const char * const qspi_data2_groups[] = {
+ "gpio35",
+};
+static const char * const qspi_data3_groups[] = {
+ "gpio51",
+};
+static const char * const qspi_resetn_groups[] = {
+ "gpio48",
+};
+static const char * const sec_mi2s_groups[] = {
+ "gpio24", "gpio25", "gpio26", "gpio27", "gpio62",
+};
+static const char * const sndwire_clk_groups[] = {
+ "gpio24",
+};
+static const char * const sndwire_data_groups[] = {
+ "gpio25",
+};
+static const char * const sp_cmu_groups[] = {
+ "gpio64",
+};
+static const char * const ssc_irq_groups[] = {
+ "gpio67", "gpio68", "gpio69", "gpio70", "gpio71", "gpio72", "gpio74",
+ "gpio75", "gpio76",
+};
+static const char * const tgu_ch0_groups[] = {
+ "gpio0",
+};
+static const char * const tgu_ch1_groups[] = {
+ "gpio1",
+};
+static const char * const tsense_pwm1_groups[] = {
+ "gpio71",
+};
+static const char * const tsense_pwm2_groups[] = {
+ "gpio71",
+};
+static const char * const uim1_clk_groups[] = {
+ "gpio88",
+};
+static const char * const uim1_data_groups[] = {
+ "gpio87",
+};
+static const char * const uim1_present_groups[] = {
+ "gpio90",
+};
+static const char * const uim1_reset_groups[] = {
+ "gpio89",
+};
+static const char * const uim2_clk_groups[] = {
+ "gpio84",
+};
+static const char * const uim2_data_groups[] = {
+ "gpio83",
+};
+static const char * const uim2_present_groups[] = {
+ "gpio86",
+};
+static const char * const uim2_reset_groups[] = {
+ "gpio85",
+};
+static const char * const uim_batt_groups[] = {
+ "gpio91",
+};
+static const char * const vfr_1_groups[] = {
+ "gpio27",
+};
+static const char * const vsense_clkout_groups[] = {
+ "gpio24",
+};
+static const char * const vsense_data0_groups[] = {
+ "gpio21",
+};
+static const char * const vsense_data1_groups[] = {
+ "gpio22",
+};
+static const char * const vsense_mode_groups[] = {
+ "gpio23",
+};
+static const char * const wlan1_adc0_groups[] = {
+ "gpio9",
+};
+static const char * const wlan1_adc1_groups[] = {
+ "gpio8",
+};
+static const char * const wlan2_adc0_groups[] = {
+ "gpio11",
+};
+static const char * const wlan2_adc1_groups[] = {
+ "gpio10",
+};
+
+static const struct msm_function sdm660_functions[] = {
+ FUNCTION(adsp_ext),
+ FUNCTION(agera_pll),
+ FUNCTION(atest_char),
+ FUNCTION(atest_char0),
+ FUNCTION(atest_char1),
+ FUNCTION(atest_char2),
+ FUNCTION(atest_char3),
+ FUNCTION(atest_gpsadc0),
+ FUNCTION(atest_gpsadc1),
+ FUNCTION(atest_tsens),
+ FUNCTION(atest_tsens2),
+ FUNCTION(atest_usb1),
+ FUNCTION(atest_usb10),
+ FUNCTION(atest_usb11),
+ FUNCTION(atest_usb12),
+ FUNCTION(atest_usb13),
+ FUNCTION(atest_usb2),
+ FUNCTION(atest_usb20),
+ FUNCTION(atest_usb21),
+ FUNCTION(atest_usb22),
+ FUNCTION(atest_usb23),
+ FUNCTION(audio_ref),
+ FUNCTION(bimc_dte0),
+ FUNCTION(bimc_dte1),
+ FUNCTION(blsp_i2c1),
+ FUNCTION(blsp_i2c2),
+ FUNCTION(blsp_i2c3),
+ FUNCTION(blsp_i2c4),
+ FUNCTION(blsp_i2c5),
+ FUNCTION(blsp_i2c6),
+ FUNCTION(blsp_i2c7),
+ FUNCTION(blsp_i2c8_a),
+ FUNCTION(blsp_i2c8_b),
+ FUNCTION(blsp_spi1),
+ FUNCTION(blsp_spi2),
+ FUNCTION(blsp_spi3),
+ FUNCTION(blsp_spi3_cs1),
+ FUNCTION(blsp_spi3_cs2),
+ FUNCTION(blsp_spi4),
+ FUNCTION(blsp_spi5),
+ FUNCTION(blsp_spi6),
+ FUNCTION(blsp_spi7),
+ FUNCTION(blsp_spi8_a),
+ FUNCTION(blsp_spi8_b),
+ FUNCTION(blsp_spi8_cs1),
+ FUNCTION(blsp_spi8_cs2),
+ FUNCTION(blsp_uart1),
+ FUNCTION(blsp_uart2),
+ FUNCTION(blsp_uart5),
+ FUNCTION(blsp_uart6_a),
+ FUNCTION(blsp_uart6_b),
+ FUNCTION(blsp_uim1),
+ FUNCTION(blsp_uim2),
+ FUNCTION(blsp_uim5),
+ FUNCTION(blsp_uim6),
+ FUNCTION(cam_mclk),
+ FUNCTION(cci_async),
+ FUNCTION(cci_i2c),
+ FUNCTION(cri_trng),
+ FUNCTION(cri_trng0),
+ FUNCTION(cri_trng1),
+ FUNCTION(dbg_out),
+ FUNCTION(ddr_bist),
+ FUNCTION(gcc_gp1),
+ FUNCTION(gcc_gp2),
+ FUNCTION(gcc_gp3),
+ FUNCTION(gpio),
+ FUNCTION(gps_tx_a),
+ FUNCTION(gps_tx_b),
+ FUNCTION(gps_tx_c),
+ FUNCTION(isense_dbg),
+ FUNCTION(jitter_bist),
+ FUNCTION(ldo_en),
+ FUNCTION(ldo_update),
+ FUNCTION(m_voc),
+ FUNCTION(mdp_vsync),
+ FUNCTION(mdss_vsync0),
+ FUNCTION(mdss_vsync1),
+ FUNCTION(mdss_vsync2),
+ FUNCTION(mdss_vsync3),
+ FUNCTION(mss_lte),
+ FUNCTION(nav_pps_a),
+ FUNCTION(nav_pps_b),
+ FUNCTION(nav_pps_c),
+ FUNCTION(pa_indicator),
+ FUNCTION(phase_flag0),
+ FUNCTION(phase_flag1),
+ FUNCTION(phase_flag2),
+ FUNCTION(phase_flag3),
+ FUNCTION(phase_flag4),
+ FUNCTION(phase_flag5),
+ FUNCTION(phase_flag6),
+ FUNCTION(phase_flag7),
+ FUNCTION(phase_flag8),
+ FUNCTION(phase_flag9),
+ FUNCTION(phase_flag10),
+ FUNCTION(phase_flag11),
+ FUNCTION(phase_flag12),
+ FUNCTION(phase_flag13),
+ FUNCTION(phase_flag14),
+ FUNCTION(phase_flag15),
+ FUNCTION(phase_flag16),
+ FUNCTION(phase_flag17),
+ FUNCTION(phase_flag18),
+ FUNCTION(phase_flag19),
+ FUNCTION(phase_flag20),
+ FUNCTION(phase_flag21),
+ FUNCTION(phase_flag22),
+ FUNCTION(phase_flag23),
+ FUNCTION(phase_flag24),
+ FUNCTION(phase_flag25),
+ FUNCTION(phase_flag26),
+ FUNCTION(phase_flag27),
+ FUNCTION(phase_flag28),
+ FUNCTION(phase_flag29),
+ FUNCTION(phase_flag30),
+ FUNCTION(phase_flag31),
+ FUNCTION(pll_bypassnl),
+ FUNCTION(pll_reset),
+ FUNCTION(pri_mi2s),
+ FUNCTION(pri_mi2s_ws),
+ FUNCTION(prng_rosc),
+ FUNCTION(pwr_crypto),
+ FUNCTION(pwr_modem),
+ FUNCTION(pwr_nav),
+ FUNCTION(qdss_cti0_a),
+ FUNCTION(qdss_cti0_b),
+ FUNCTION(qdss_cti1_a),
+ FUNCTION(qdss_cti1_b),
+ FUNCTION(qdss_gpio),
+ FUNCTION(qdss_gpio0),
+ FUNCTION(qdss_gpio1),
+ FUNCTION(qdss_gpio10),
+ FUNCTION(qdss_gpio11),
+ FUNCTION(qdss_gpio12),
+ FUNCTION(qdss_gpio13),
+ FUNCTION(qdss_gpio14),
+ FUNCTION(qdss_gpio15),
+ FUNCTION(qdss_gpio2),
+ FUNCTION(qdss_gpio3),
+ FUNCTION(qdss_gpio4),
+ FUNCTION(qdss_gpio5),
+ FUNCTION(qdss_gpio6),
+ FUNCTION(qdss_gpio7),
+ FUNCTION(qdss_gpio8),
+ FUNCTION(qdss_gpio9),
+ FUNCTION(qlink_enable),
+ FUNCTION(qlink_request),
+ FUNCTION(qspi_clk),
+ FUNCTION(qspi_cs),
+ FUNCTION(qspi_data0),
+ FUNCTION(qspi_data1),
+ FUNCTION(qspi_data2),
+ FUNCTION(qspi_data3),
+ FUNCTION(qspi_resetn),
+ FUNCTION(sec_mi2s),
+ FUNCTION(sndwire_clk),
+ FUNCTION(sndwire_data),
+ FUNCTION(sp_cmu),
+ FUNCTION(ssc_irq),
+ FUNCTION(tgu_ch0),
+ FUNCTION(tgu_ch1),
+ FUNCTION(tsense_pwm1),
+ FUNCTION(tsense_pwm2),
+ FUNCTION(uim1_clk),
+ FUNCTION(uim1_data),
+ FUNCTION(uim1_present),
+ FUNCTION(uim1_reset),
+ FUNCTION(uim2_clk),
+ FUNCTION(uim2_data),
+ FUNCTION(uim2_present),
+ FUNCTION(uim2_reset),
+ FUNCTION(uim_batt),
+ FUNCTION(vfr_1),
+ FUNCTION(vsense_clkout),
+ FUNCTION(vsense_data0),
+ FUNCTION(vsense_data1),
+ FUNCTION(vsense_mode),
+ FUNCTION(wlan1_adc0),
+ FUNCTION(wlan1_adc1),
+ FUNCTION(wlan2_adc0),
+ FUNCTION(wlan2_adc1),
+};
+
+static const struct msm_pingroup sdm660_groups[] = {
+ PINGROUP(0, SOUTH, blsp_spi1, blsp_uart1, blsp_uim1, tgu_ch0, _, _, qdss_gpio4, atest_gpsadc1, _),
+ PINGROUP(1, SOUTH, blsp_spi1, blsp_uart1, blsp_uim1, tgu_ch1, _, _, qdss_gpio5, atest_gpsadc0, _),
+ PINGROUP(2, SOUTH, blsp_spi1, blsp_uart1, blsp_i2c1, _, _, _, _, _, _),
+ PINGROUP(3, SOUTH, blsp_spi1, blsp_uart1, blsp_i2c1, ddr_bist, _, _, atest_tsens2, atest_usb1, _),
+ PINGROUP(4, NORTH, blsp_spi2, blsp_uim2, blsp_uart2, phase_flag3, _, _, _, _, _),
+ PINGROUP(5, SOUTH, blsp_spi2, blsp_uim2, blsp_uart2, phase_flag14, _, _, _, _, _),
+ PINGROUP(6, SOUTH, blsp_spi2, blsp_i2c2, blsp_uart2, phase_flag31, _, _, _, _, _),
+ PINGROUP(7, SOUTH, blsp_spi2, blsp_i2c2, blsp_uart2, _, _, _, _, _, _),
+ PINGROUP(8, NORTH, blsp_spi3, ddr_bist, _, _, _, wlan1_adc1, atest_usb13, bimc_dte1, _),
+ PINGROUP(9, NORTH, blsp_spi3, ddr_bist, _, _, _, wlan1_adc0, atest_usb12, bimc_dte0, _),
+ PINGROUP(10, NORTH, blsp_spi3, blsp_i2c3, ddr_bist, _, _, wlan2_adc1, atest_usb11, bimc_dte1, _),
+ PINGROUP(11, NORTH, blsp_spi3, blsp_i2c3, _, dbg_out, wlan2_adc0, atest_usb10, bimc_dte0, _, _),
+ PINGROUP(12, NORTH, blsp_spi4, pri_mi2s, _, phase_flag26, qdss_cti1_b, _, _, _, _),
+ PINGROUP(13, NORTH, blsp_spi4, _, pri_mi2s_ws, _, _, phase_flag27, qdss_cti0_b, _, _),
+ PINGROUP(14, NORTH, blsp_spi4, blsp_i2c4, pri_mi2s, _, phase_flag28, _, _, _, _),
+ PINGROUP(15, NORTH, blsp_spi4, blsp_i2c4, pri_mi2s, _, _, _, _, _, _),
+ PINGROUP(16, CENTER, blsp_uart5, blsp_spi5, blsp_uim5, _, _, _, _, _, _),
+ PINGROUP(17, CENTER, blsp_uart5, blsp_spi5, blsp_uim5, _, phase_flag5, _, _, _, _),
+ PINGROUP(18, CENTER, blsp_uart5, blsp_spi5, blsp_i2c5, _, _, _, _, _, _),
+ PINGROUP(19, CENTER, blsp_uart5, blsp_spi5, blsp_i2c5, _, _, _, _, _, _),
+ PINGROUP(20, SOUTH, _, _, blsp_uim6, _, _, _, _, _, _),
+ PINGROUP(21, SOUTH, _, _, blsp_uim6, _, phase_flag11, qdss_cti0_b, vsense_data0, _, _),
+ PINGROUP(22, CENTER, blsp_spi6, _, blsp_i2c6, _, phase_flag12, vsense_data1, _, _, _),
+ PINGROUP(23, CENTER, blsp_spi6, _, blsp_i2c6, _, phase_flag13, vsense_mode, _, _, _),
+ PINGROUP(24, NORTH, blsp_spi7, blsp_uart6_a, sec_mi2s, sndwire_clk, _, _, phase_flag17, vsense_clkout, _),
+ PINGROUP(25, NORTH, blsp_spi7, blsp_uart6_a, sec_mi2s, sndwire_data, _, _, phase_flag18, _, _),
+ PINGROUP(26, NORTH, blsp_spi7, blsp_uart6_a, blsp_i2c7, sec_mi2s, _, phase_flag19, _, _, _),
+ PINGROUP(27, NORTH, blsp_spi7, blsp_uart6_a, blsp_i2c7, vfr_1, sec_mi2s, _, phase_flag20, _, _),
+ PINGROUP(28, CENTER, blsp_spi8_a, blsp_uart6_b, m_voc, _, phase_flag21, _, _, _, _),
+ PINGROUP(29, CENTER, blsp_spi8_a, blsp_uart6_b, _, _, phase_flag22, _, _, _, _),
+ PINGROUP(30, CENTER, blsp_spi8_a, blsp_uart6_b, blsp_i2c8_a, blsp_spi3_cs1, _, phase_flag23, _, _, _),
+ PINGROUP(31, CENTER, blsp_spi8_a, blsp_uart6_b, blsp_i2c8_a, pwr_modem, _, phase_flag24, qdss_gpio, _, _),
+ PINGROUP(32, SOUTH, cam_mclk, pwr_nav, _, _, qdss_gpio0, _, _, _, _),
+ PINGROUP(33, SOUTH, cam_mclk, qspi_data0, pwr_crypto, _, _, qdss_gpio1, _, _, _),
+ PINGROUP(34, SOUTH, cam_mclk, qspi_data1, agera_pll, _, _, qdss_gpio2, _, _, _),
+ PINGROUP(35, SOUTH, cam_mclk, qspi_data2, jitter_bist, _, _, qdss_gpio3, _, atest_usb2, _),
+ PINGROUP(36, SOUTH, cci_i2c, pll_bypassnl, agera_pll, _, _, qdss_gpio4, atest_tsens, atest_usb21, _),
+ PINGROUP(37, SOUTH, cci_i2c, pll_reset, _, _, qdss_gpio5, atest_usb23, _, _, _),
+ PINGROUP(38, SOUTH, cci_i2c, _, _, qdss_gpio6, _, _, _, _, _),
+ PINGROUP(39, SOUTH, cci_i2c, _, _, qdss_gpio7, _, _, _, _, _),
+ PINGROUP(40, SOUTH, _, _, blsp_spi8_b, _, _, _, _, _, _),
+ PINGROUP(41, SOUTH, _, _, blsp_spi8_b, _, _, _, _, _, _),
+ PINGROUP(42, SOUTH, mdss_vsync0, mdss_vsync1, mdss_vsync2, mdss_vsync3, _, _, qdss_gpio9, _, _),
+ PINGROUP(43, SOUTH, _, _, qspi_cs, _, _, qdss_gpio10, _, _, _),
+ PINGROUP(44, SOUTH, _, _, blsp_spi8_b, blsp_i2c8_b, _, _, qdss_gpio11, _, _),
+ PINGROUP(45, SOUTH, cci_async, _, _, qdss_gpio12, _, _, _, _, _),
+ PINGROUP(46, SOUTH, blsp_spi1, _, _, qdss_gpio13, _, _, _, _, _),
+ PINGROUP(47, SOUTH, qspi_clk, _, phase_flag30, qdss_gpio14, _, _, _, _, _),
+ PINGROUP(48, SOUTH, _, phase_flag1, qdss_gpio15, _, _, _, _, _, _),
+ PINGROUP(49, SOUTH, blsp_spi6, phase_flag2, qdss_cti0_a, _, _, _, _, _, _),
+ PINGROUP(50, SOUTH, qspi_cs, _, phase_flag9, qdss_cti0_a, _, _, _, _, _),
+ PINGROUP(51, SOUTH, qspi_data3, _, phase_flag15, qdss_gpio8, _, _, _, _, _),
+ PINGROUP(52, SOUTH, _, blsp_spi8_b, blsp_i2c8_b, blsp_spi6, phase_flag16, qdss_gpio, _, _, _),
+ PINGROUP(53, NORTH, _, phase_flag6, qdss_cti1_a, _, _, _, _, _, _),
+ PINGROUP(54, NORTH, _, _, phase_flag29, _, _, _, _, _, _),
+ PINGROUP(55, SOUTH, _, phase_flag25, qdss_cti1_a, _, _, _, _, _, _),
+ PINGROUP(56, SOUTH, _, phase_flag10, qdss_gpio3, _, atest_usb20, _, _, _, _),
+ PINGROUP(57, SOUTH, gcc_gp1, _, phase_flag4, atest_usb22, _, _, _, _, _),
+ PINGROUP(58, SOUTH, _, gcc_gp2, _, _, atest_char, _, _, _, _),
+ PINGROUP(59, NORTH, mdp_vsync, gcc_gp3, _, _, atest_char3, _, _, _, _),
+ PINGROUP(60, NORTH, cri_trng0, _, _, atest_char2, _, _, _, _, _),
+ PINGROUP(61, NORTH, pri_mi2s, cri_trng1, _, _, atest_char1, _, _, _, _),
+ PINGROUP(62, NORTH, sec_mi2s, audio_ref, _, cri_trng, _, _, atest_char0, _, _),
+ PINGROUP(63, NORTH, _, _, _, qdss_gpio1, _, _, _, _, _),
+ PINGROUP(64, SOUTH, blsp_spi8_cs1, sp_cmu, _, _, qdss_gpio2, _, _, _, _),
+ PINGROUP(65, SOUTH, _, nav_pps_a, nav_pps_a, gps_tx_a, blsp_spi3_cs2, adsp_ext, _, _, _),
+ PINGROUP(66, NORTH, _, _, qdss_cti1_b, _, _, _, _, _, _),
+ PINGROUP(67, NORTH, _, _, qdss_gpio0, _, _, _, _, _, _),
+ PINGROUP(68, NORTH, isense_dbg, _, phase_flag0, qdss_gpio, _, _, _, _, _),
+ PINGROUP(69, NORTH, _, phase_flag7, qdss_gpio, _, _, _, _, _, _),
+ PINGROUP(70, NORTH, _, phase_flag8, qdss_gpio6, _, _, _, _, _, _),
+ PINGROUP(71, NORTH, _, _, qdss_gpio7, tsense_pwm1, tsense_pwm2, _, _, _, _),
+ PINGROUP(72, NORTH, _, qdss_gpio14, _, _, _, _, _, _, _),
+ PINGROUP(73, NORTH, _, _, qdss_gpio15, _, _, _, _, _, _),
+ PINGROUP(74, NORTH, mdp_vsync, _, _, _, _, _, _, _, _),
+ PINGROUP(75, NORTH, _, _, qdss_gpio8, _, _, _, _, _, _),
+ PINGROUP(76, NORTH, blsp_spi8_cs2, _, _, _, qdss_gpio9, _, _, _, _),
+ PINGROUP(77, NORTH, _, _, qdss_gpio10, _, _, _, _, _, _),
+ PINGROUP(78, NORTH, gcc_gp1, _, qdss_gpio13, _, _, _, _, _, _),
+ PINGROUP(79, SOUTH, _, _, qdss_gpio11, _, _, _, _, _, _),
+ PINGROUP(80, SOUTH, nav_pps_b, nav_pps_b, gps_tx_c, _, _, qdss_gpio12, _, _, _),
+ PINGROUP(81, CENTER, mss_lte, gcc_gp2, _, _, _, _, _, _, _),
+ PINGROUP(82, CENTER, mss_lte, gcc_gp3, _, _, _, _, _, _, _),
+ PINGROUP(83, SOUTH, uim2_data, _, _, _, _, _, _, _, _),
+ PINGROUP(84, SOUTH, uim2_clk, _, _, _, _, _, _, _, _),
+ PINGROUP(85, SOUTH, uim2_reset, _, _, _, _, _, _, _, _),
+ PINGROUP(86, SOUTH, uim2_present, _, _, _, _, _, _, _, _),
+ PINGROUP(87, SOUTH, uim1_data, _, _, _, _, _, _, _, _),
+ PINGROUP(88, SOUTH, uim1_clk, _, _, _, _, _, _, _, _),
+ PINGROUP(89, SOUTH, uim1_reset, _, _, _, _, _, _, _, _),
+ PINGROUP(90, SOUTH, uim1_present, _, _, _, _, _, _, _, _),
+ PINGROUP(91, SOUTH, uim_batt, _, _, _, _, _, _, _, _),
+ PINGROUP(92, SOUTH, _, _, pa_indicator, _, _, _, _, _, _),
+ PINGROUP(93, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(94, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(95, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(96, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(97, SOUTH, _, ldo_en, _, _, _, _, _, _, _),
+ PINGROUP(98, SOUTH, _, nav_pps_c, nav_pps_c, gps_tx_b, ldo_update, _, _, _, _),
+ PINGROUP(99, SOUTH, qlink_request, _, _, _, _, _, _, _, _),
+ PINGROUP(100, SOUTH, qlink_enable, _, _, _, _, _, _, _, _),
+ PINGROUP(101, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(102, SOUTH, _, prng_rosc, _, _, _, _, _, _, _),
+ PINGROUP(103, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(104, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(105, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(106, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(107, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(108, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(109, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(110, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(111, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(112, SOUTH, _, _, _, _, _, _, _, _, _),
+ PINGROUP(113, SOUTH, _, _, _, _, _, _, _, _, _),
+ SDC_QDSD_PINGROUP(sdc1_clk, 0x99a000, 13, 6),
+ SDC_QDSD_PINGROUP(sdc1_cmd, 0x99a000, 11, 3),
+ SDC_QDSD_PINGROUP(sdc1_data, 0x99a000, 9, 0),
+ SDC_QDSD_PINGROUP(sdc2_clk, 0x99b000, 14, 6),
+ SDC_QDSD_PINGROUP(sdc2_cmd, 0x99b000, 11, 3),
+ SDC_QDSD_PINGROUP(sdc2_data, 0x99b000, 9, 0),
+ SDC_QDSD_PINGROUP(sdc1_rclk, 0x99a000, 15, 0),
+};
+
+static const struct msm_pinctrl_soc_data sdm660_pinctrl = {
+ .pins = sdm660_pins,
+ .npins = ARRAY_SIZE(sdm660_pins),
+ .functions = sdm660_functions,
+ .nfunctions = ARRAY_SIZE(sdm660_functions),
+ .groups = sdm660_groups,
+ .ngroups = ARRAY_SIZE(sdm660_groups),
+ .ngpios = 114,
+ .tiles = sdm660_tiles,
+ .ntiles = ARRAY_SIZE(sdm660_tiles),
+};
+
+static int sdm660_pinctrl_probe(struct platform_device *pdev)
+{
+ return msm_pinctrl_probe(pdev, &sdm660_pinctrl);
+}
+
+static const struct of_device_id sdm660_pinctrl_of_match[] = {
+ { .compatible = "qcom,sdm660-pinctrl", },
+ { .compatible = "qcom,sdm630-pinctrl", },
+ { },
+};
+
+static struct platform_driver sdm660_pinctrl_driver = {
+ .driver = {
+ .name = "sdm660-pinctrl",
+ .of_match_table = sdm660_pinctrl_of_match,
+ },
+ .probe = sdm660_pinctrl_probe,
+ .remove = msm_pinctrl_remove,
+};
+
+static int __init sdm660_pinctrl_init(void)
+{
+ return platform_driver_register(&sdm660_pinctrl_driver);
+}
+arch_initcall(sdm660_pinctrl_init);
+
+static void __exit sdm660_pinctrl_exit(void)
+{
+ platform_driver_unregister(&sdm660_pinctrl_driver);
+}
+module_exit(sdm660_pinctrl_exit);
+
+MODULE_DESCRIPTION("QTI sdm660 pinctrl driver");
+MODULE_LICENSE("GPL v2");
+MODULE_DEVICE_TABLE(of, sdm660_pinctrl_of_match);
* GNU General Public License for more details.
*/
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_irq.h>
* GNU General Public License for more details.
*/
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_irq.h>
pad->function = function;
ret = pmic_mpp_write_mode_ctl(state, pad);
+ if (ret < 0)
+ return ret;
val = pad->is_enabled << PMIC_MPP_REG_MASTER_EN_SHIFT;
switch (param) {
case PIN_CONFIG_BIAS_DISABLE:
- arg = pad->pullup == PMIC_MPP_PULL_UP_OPEN;
+ if (pad->pullup != PMIC_MPP_PULL_UP_OPEN)
+ return -EINVAL;
+ arg = 1;
break;
case PIN_CONFIG_BIAS_PULL_UP:
switch (pad->pullup) {
- case PMIC_MPP_PULL_UP_OPEN:
- arg = 0;
- break;
case PMIC_MPP_PULL_UP_0P6KOHM:
arg = 600;
break;
}
break;
case PIN_CONFIG_BIAS_HIGH_IMPEDANCE:
- arg = !pad->is_enabled;
+ if (pad->is_enabled)
+ return -EINVAL;
+ arg = 1;
break;
case PIN_CONFIG_POWER_SOURCE:
arg = pad->power_source;
break;
case PIN_CONFIG_INPUT_ENABLE:
- arg = pad->input_enabled;
+ if (!pad->input_enabled)
+ return -EINVAL;
+ arg = 1;
break;
case PIN_CONFIG_OUTPUT:
arg = pad->out_value;
arg = pad->amux_input;
break;
case PMIC_MPP_CONF_PAIRED:
- arg = pad->paired;
+ if (!pad->paired)
+ return -EINVAL;
+ arg = 1;
break;
case PIN_CONFIG_DRIVE_STRENGTH:
arg = pad->drive_strength;
pad->dtest = arg;
break;
case PIN_CONFIG_DRIVE_STRENGTH:
- arg = pad->drive_strength;
+ pad->drive_strength = arg;
break;
case PMIC_MPP_CONF_AMUX_ROUTE:
if (arg >= PMIC_MPP_AMUX_ROUTE_ABUS4)
if (ret < 0)
return ret;
+ ret = pmic_mpp_write(state, pad, PMIC_MPP_REG_SINK_CTL, pad->drive_strength);
+ if (ret < 0)
+ return ret;
+
val = pad->is_enabled << PMIC_MPP_REG_MASTER_EN_SHIFT;
return pmic_mpp_write(state, pad, PMIC_MPP_REG_EN_CTL, val);
#include <linux/pinctrl/pinconf-generic.h>
#include <linux/slab.h>
#include <linux/regmap.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/interrupt.h>
#include <linux/of_device.h>
#include <linux/of_irq.h>
switch (param) {
case PIN_CONFIG_BIAS_DISABLE:
- arg = pin->bias == PM8XXX_GPIO_BIAS_NP;
+ if (pin->bias != PM8XXX_GPIO_BIAS_NP)
+ return -EINVAL;
+ arg = 1;
break;
case PIN_CONFIG_BIAS_PULL_DOWN:
- arg = pin->bias == PM8XXX_GPIO_BIAS_PD;
+ if (pin->bias != PM8XXX_GPIO_BIAS_PD)
+ return -EINVAL;
+ arg = 1;
break;
case PIN_CONFIG_BIAS_PULL_UP:
- arg = pin->bias <= PM8XXX_GPIO_BIAS_PU_1P5_30;
+ if (pin->bias > PM8XXX_GPIO_BIAS_PU_1P5_30)
+ return -EINVAL;
+ arg = 1;
break;
case PM8XXX_QCOM_PULL_UP_STRENGTH:
arg = pin->pull_up_strength;
break;
case PIN_CONFIG_BIAS_HIGH_IMPEDANCE:
- arg = pin->disable;
+ if (!pin->disable)
+ return -EINVAL;
+ arg = 1;
break;
case PIN_CONFIG_INPUT_ENABLE:
- arg = pin->mode == PM8XXX_GPIO_MODE_INPUT;
+ if (pin->mode != PM8XXX_GPIO_MODE_INPUT)
+ return -EINVAL;
+ arg = 1;
break;
case PIN_CONFIG_OUTPUT:
if (pin->mode & PM8XXX_GPIO_MODE_OUTPUT)
arg = pin->output_strength;
break;
case PIN_CONFIG_DRIVE_PUSH_PULL:
- arg = !pin->open_drain;
+ if (pin->open_drain)
+ return -EINVAL;
+ arg = 1;
break;
case PIN_CONFIG_DRIVE_OPEN_DRAIN:
- arg = pin->open_drain;
+ if (!pin->open_drain)
+ return -EINVAL;
+ arg = 1;
break;
default:
return -EINVAL;
#include <linux/pinctrl/pinconf-generic.h>
#include <linux/slab.h>
#include <linux/regmap.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/interrupt.h>
#include <linux/of_device.h>
#include <linux/of_irq.h>
#include <linux/io.h>
#include <linux/slab.h>
#include <linux/err.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/irqdomain.h>
#include <linux/of_device.h>
#include <linux/spinlock.h>
#include <linux/pinctrl/consumer.h>
#include <linux/pinctrl/machine.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
/**
* enum pincfg_type - possible pin configuration types supported.
+# SPDX-License-Identifier: GPL-2.0
#
# Renesas SH and SH Mobile PINCTRL drivers
#
depends on ARCH_R8A7743
select PINCTRL_SH_PFC
+config PINCTRL_PFC_R8A7744
+ def_bool y
+ depends on ARCH_R8A7744
+ select PINCTRL_SH_PFC
+
config PINCTRL_PFC_R8A7745
def_bool y
depends on ARCH_R8A7745
depends on ARCH_R8A77470
select PINCTRL_SH_PFC
+config PINCTRL_PFC_R8A774A1
+ def_bool y
+ depends on ARCH_R8A774A1
+ select PINCTRL_SH_PFC
+
+config PINCTRL_PFC_R8A774C0
+ def_bool y
+ depends on ARCH_R8A774C0
+ select PINCTRL_SH_PFC
+
config PINCTRL_PFC_R8A7778
def_bool y
depends on ARCH_R8A7778
obj-$(CONFIG_PINCTRL_PFC_R8A73A4) += pfc-r8a73a4.o
obj-$(CONFIG_PINCTRL_PFC_R8A7740) += pfc-r8a7740.o
obj-$(CONFIG_PINCTRL_PFC_R8A7743) += pfc-r8a7791.o
+obj-$(CONFIG_PINCTRL_PFC_R8A7744) += pfc-r8a7791.o
obj-$(CONFIG_PINCTRL_PFC_R8A7745) += pfc-r8a7794.o
obj-$(CONFIG_PINCTRL_PFC_R8A77470) += pfc-r8a77470.o
+obj-$(CONFIG_PINCTRL_PFC_R8A774A1) += pfc-r8a7796.o
+obj-$(CONFIG_PINCTRL_PFC_R8A774C0) += pfc-r8a77990.o
obj-$(CONFIG_PINCTRL_PFC_R8A7778) += pfc-r8a7778.o
obj-$(CONFIG_PINCTRL_PFC_R8A7779) += pfc-r8a7779.o
obj-$(CONFIG_PINCTRL_PFC_R8A7790) += pfc-r8a7790.o
+// SPDX-License-Identifier: GPL-2.0
/*
* Pin Control and GPIO driver for SuperH Pin Function Controller.
*
*
* Copyright (C) 2008 Magnus Damm
* Copyright (C) 2009 - 2012 Paul Mundt
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#define DRV_NAME "sh-pfc"
.data = &r8a7743_pinmux_info,
},
#endif
+#ifdef CONFIG_PINCTRL_PFC_R8A7744
+ {
+ .compatible = "renesas,pfc-r8a7744",
+ .data = &r8a7744_pinmux_info,
+ },
+#endif
#ifdef CONFIG_PINCTRL_PFC_R8A7745
{
.compatible = "renesas,pfc-r8a7745",
.data = &r8a77470_pinmux_info,
},
#endif
+#ifdef CONFIG_PINCTRL_PFC_R8A774A1
+ {
+ .compatible = "renesas,pfc-r8a774a1",
+ .data = &r8a774a1_pinmux_info,
+ },
+#endif
+#ifdef CONFIG_PINCTRL_PFC_R8A774C0
+ {
+ .compatible = "renesas,pfc-r8a774c0",
+ .data = &r8a774c0_pinmux_info,
+ },
+#endif
#ifdef CONFIG_PINCTRL_PFC_R8A7778
{
.compatible = "renesas,pfc-r8a7778",
-/*
+/* SPDX-License-Identifier: GPL-2.0
+ *
* SuperH Pin Function Controller support.
*
* Copyright (C) 2012 Renesas Solutions Corp.
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#ifndef __SH_PFC_CORE_H__
#define __SH_PFC_CORE_H__
+// SPDX-License-Identifier: GPL-2.0
/*
* SuperH Pin Function Controller GPIO driver.
*
* Copyright (C) 2008 Magnus Damm
* Copyright (C) 2009 - 2012 Paul Mundt
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/device.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* Pin Function Controller Support
*
* Copyright (C) 2015 Niklas Söderlund
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/init.h>
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2012-2013 Renesas Solutions Corp.
* Copyright (C) 2013 Magnus Damm
* Copyright (C) 2012 Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; version 2 of the
- * License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
#include <linux/io.h>
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* R8A7740 processor support
*
* Copyright (C) 2011 Renesas Solutions Corp.
* Copyright (C) 2011 Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; version 2 of the
- * License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
#include <linux/io.h>
#include <linux/kernel.h>
PINMUX_GPIO_GP_ALL(),
};
+/* - AVB -------------------------------------------------------------------- */
+static const unsigned int avb_col_pins[] = {
+ RCAR_GP_PIN(5, 18),
+};
+static const unsigned int avb_col_mux[] = {
+ AVB_COL_MARK,
+};
+static const unsigned int avb_crs_pins[] = {
+ RCAR_GP_PIN(5, 17),
+};
+static const unsigned int avb_crs_mux[] = {
+ AVB_CRS_MARK,
+};
+static const unsigned int avb_link_pins[] = {
+ RCAR_GP_PIN(5, 14),
+};
+static const unsigned int avb_link_mux[] = {
+ AVB_LINK_MARK,
+};
+static const unsigned int avb_magic_pins[] = {
+ RCAR_GP_PIN(5, 15),
+};
+static const unsigned int avb_magic_mux[] = {
+ AVB_MAGIC_MARK,
+};
+static const unsigned int avb_phy_int_pins[] = {
+ RCAR_GP_PIN(5, 16),
+};
+static const unsigned int avb_phy_int_mux[] = {
+ AVB_PHY_INT_MARK,
+};
+static const unsigned int avb_mdio_pins[] = {
+ RCAR_GP_PIN(5, 12), RCAR_GP_PIN(5, 13),
+};
+static const unsigned int avb_mdio_mux[] = {
+ AVB_MDC_MARK, AVB_MDIO_MARK,
+};
+static const unsigned int avb_mii_tx_rx_pins[] = {
+ RCAR_GP_PIN(3, 12), RCAR_GP_PIN(3, 14), RCAR_GP_PIN(3, 15),
+ RCAR_GP_PIN(3, 16), RCAR_GP_PIN(3, 27), RCAR_GP_PIN(3, 13),
+
+ RCAR_GP_PIN(3, 0), RCAR_GP_PIN(3, 2), RCAR_GP_PIN(3, 3),
+ RCAR_GP_PIN(3, 4), RCAR_GP_PIN(3, 5), RCAR_GP_PIN(3, 1),
+ RCAR_GP_PIN(3, 10),
+};
+static const unsigned int avb_mii_tx_rx_mux[] = {
+ AVB_TX_CLK_MARK, AVB_TXD0_MARK, AVB_TXD1_MARK, AVB_TXD2_MARK,
+ AVB_TXD3_MARK, AVB_TX_EN_MARK,
+
+ AVB_RX_CLK_MARK, AVB_RXD0_MARK, AVB_RXD1_MARK, AVB_RXD2_MARK,
+ AVB_RXD3_MARK, AVB_RX_DV_MARK, AVB_RX_ER_MARK,
+};
+static const unsigned int avb_mii_tx_er_pins[] = {
+ RCAR_GP_PIN(5, 23),
+};
+static const unsigned int avb_mii_tx_er_mux[] = {
+ AVB_TX_ER_MARK,
+};
+static const unsigned int avb_gmii_tx_rx_pins[] = {
+ RCAR_GP_PIN(4, 1), RCAR_GP_PIN(3, 11), RCAR_GP_PIN(3, 12),
+ RCAR_GP_PIN(3, 14), RCAR_GP_PIN(3, 15), RCAR_GP_PIN(3, 16),
+ RCAR_GP_PIN(3, 27), RCAR_GP_PIN(3, 28), RCAR_GP_PIN(3, 29),
+ RCAR_GP_PIN(4, 0), RCAR_GP_PIN(5, 22), RCAR_GP_PIN(3, 13),
+ RCAR_GP_PIN(5, 23),
+
+ RCAR_GP_PIN(3, 0), RCAR_GP_PIN(3, 2), RCAR_GP_PIN(3, 3),
+ RCAR_GP_PIN(3, 4), RCAR_GP_PIN(3, 5), RCAR_GP_PIN(3, 6),
+ RCAR_GP_PIN(3, 7), RCAR_GP_PIN(3, 8), RCAR_GP_PIN(3, 9),
+ RCAR_GP_PIN(3, 1), RCAR_GP_PIN(3, 10),
+};
+static const unsigned int avb_gmii_tx_rx_mux[] = {
+ AVB_GTX_CLK_MARK, AVB_GTXREFCLK_MARK, AVB_TX_CLK_MARK, AVB_TXD0_MARK,
+ AVB_TXD1_MARK, AVB_TXD2_MARK, AVB_TXD3_MARK, AVB_TXD4_MARK,
+ AVB_TXD5_MARK, AVB_TXD6_MARK, AVB_TXD7_MARK, AVB_TX_EN_MARK,
+ AVB_TX_ER_MARK,
+
+ AVB_RX_CLK_MARK, AVB_RXD0_MARK, AVB_RXD1_MARK, AVB_RXD2_MARK,
+ AVB_RXD3_MARK, AVB_RXD4_MARK, AVB_RXD5_MARK, AVB_RXD6_MARK,
+ AVB_RXD7_MARK, AVB_RX_DV_MARK, AVB_RX_ER_MARK,
+};
+static const unsigned int avb_avtp_match_a_pins[] = {
+ RCAR_GP_PIN(1, 15),
+};
+static const unsigned int avb_avtp_match_a_mux[] = {
+ AVB_AVTP_MATCH_A_MARK,
+};
+static const unsigned int avb_avtp_capture_a_pins[] = {
+ RCAR_GP_PIN(1, 14),
+};
+static const unsigned int avb_avtp_capture_a_mux[] = {
+ AVB_AVTP_CAPTURE_A_MARK,
+};
+static const unsigned int avb_avtp_match_b_pins[] = {
+ RCAR_GP_PIN(5, 20),
+};
+static const unsigned int avb_avtp_match_b_mux[] = {
+ AVB_AVTP_MATCH_B_MARK,
+};
+static const unsigned int avb_avtp_capture_b_pins[] = {
+ RCAR_GP_PIN(5, 19),
+};
+static const unsigned int avb_avtp_capture_b_mux[] = {
+ AVB_AVTP_CAPTURE_B_MARK,
+};
+/* - DU --------------------------------------------------------------------- */
+static const unsigned int du0_rgb666_pins[] = {
+ /* R[7:2], G[7:2], B[7:2] */
+ RCAR_GP_PIN(2, 7), RCAR_GP_PIN(2, 6), RCAR_GP_PIN(2, 5),
+ RCAR_GP_PIN(2, 4), RCAR_GP_PIN(2, 3), RCAR_GP_PIN(2, 2),
+ RCAR_GP_PIN(2, 15), RCAR_GP_PIN(2, 14), RCAR_GP_PIN(2, 13),
+ RCAR_GP_PIN(2, 12), RCAR_GP_PIN(2, 11), RCAR_GP_PIN(2, 10),
+ RCAR_GP_PIN(2, 23), RCAR_GP_PIN(2, 22), RCAR_GP_PIN(2, 21),
+ RCAR_GP_PIN(2, 20), RCAR_GP_PIN(2, 19), RCAR_GP_PIN(2, 18),
+};
+static const unsigned int du0_rgb666_mux[] = {
+ DU0_DR7_MARK, DU0_DR6_MARK, DU0_DR5_MARK, DU0_DR4_MARK,
+ DU0_DR3_MARK, DU0_DR2_MARK,
+ DU0_DG7_MARK, DU0_DG6_MARK, DU0_DG5_MARK, DU0_DG4_MARK,
+ DU0_DG3_MARK, DU0_DG2_MARK,
+ DU0_DB7_MARK, DU0_DB6_MARK, DU0_DB5_MARK, DU0_DB4_MARK,
+ DU0_DB3_MARK, DU0_DB2_MARK,
+};
+static const unsigned int du0_rgb888_pins[] = {
+ /* R[7:0], G[7:0], B[7:0] */
+ RCAR_GP_PIN(2, 7), RCAR_GP_PIN(2, 6), RCAR_GP_PIN(2, 5),
+ RCAR_GP_PIN(2, 4), RCAR_GP_PIN(2, 3), RCAR_GP_PIN(2, 2),
+ RCAR_GP_PIN(2, 1), RCAR_GP_PIN(2, 0),
+ RCAR_GP_PIN(2, 15), RCAR_GP_PIN(2, 14), RCAR_GP_PIN(2, 13),
+ RCAR_GP_PIN(2, 12), RCAR_GP_PIN(2, 11), RCAR_GP_PIN(2, 10),
+ RCAR_GP_PIN(2, 9), RCAR_GP_PIN(2, 8),
+ RCAR_GP_PIN(2, 23), RCAR_GP_PIN(2, 22), RCAR_GP_PIN(2, 21),
+ RCAR_GP_PIN(2, 20), RCAR_GP_PIN(2, 19), RCAR_GP_PIN(2, 18),
+ RCAR_GP_PIN(2, 17), RCAR_GP_PIN(2, 16),
+};
+static const unsigned int du0_rgb888_mux[] = {
+ DU0_DR7_MARK, DU0_DR6_MARK, DU0_DR5_MARK, DU0_DR4_MARK,
+ DU0_DR3_MARK, DU0_DR2_MARK, DU0_DR1_MARK, DU0_DR0_MARK,
+ DU0_DG7_MARK, DU0_DG6_MARK, DU0_DG5_MARK, DU0_DG4_MARK,
+ DU0_DG3_MARK, DU0_DG2_MARK, DU0_DG1_MARK, DU0_DG0_MARK,
+ DU0_DB7_MARK, DU0_DB6_MARK, DU0_DB5_MARK, DU0_DB4_MARK,
+ DU0_DB3_MARK, DU0_DB2_MARK, DU0_DB1_MARK, DU0_DB0_MARK,
+};
+static const unsigned int du0_clk0_out_pins[] = {
+ /* DOTCLKOUT0 */
+ RCAR_GP_PIN(2, 25),
+};
+static const unsigned int du0_clk0_out_mux[] = {
+ DU0_DOTCLKOUT0_MARK
+};
+static const unsigned int du0_clk1_out_pins[] = {
+ /* DOTCLKOUT1 */
+ RCAR_GP_PIN(2, 26),
+};
+static const unsigned int du0_clk1_out_mux[] = {
+ DU0_DOTCLKOUT1_MARK
+};
+static const unsigned int du0_clk_in_pins[] = {
+ /* CLKIN */
+ RCAR_GP_PIN(2, 24),
+};
+static const unsigned int du0_clk_in_mux[] = {
+ DU0_DOTCLKIN_MARK
+};
+static const unsigned int du0_sync_pins[] = {
+ /* EXVSYNC/VSYNC, EXHSYNC/HSYNC */
+ RCAR_GP_PIN(2, 28), RCAR_GP_PIN(2, 27),
+};
+static const unsigned int du0_sync_mux[] = {
+ DU0_EXVSYNC_DU0_VSYNC_MARK, DU0_EXHSYNC_DU0_HSYNC_MARK
+};
+static const unsigned int du0_oddf_pins[] = {
+ /* EXODDF/ODDF/DISP/CDE */
+ RCAR_GP_PIN(2, 29),
+};
+static const unsigned int du0_oddf_mux[] = {
+ DU0_EXODDF_DU0_ODDF_DISP_CDE_MARK,
+};
+static const unsigned int du0_cde_pins[] = {
+ /* CDE */
+ RCAR_GP_PIN(2, 31),
+};
+static const unsigned int du0_cde_mux[] = {
+ DU0_CDE_MARK,
+};
+static const unsigned int du0_disp_pins[] = {
+ /* DISP */
+ RCAR_GP_PIN(2, 30),
+};
+static const unsigned int du0_disp_mux[] = {
+ DU0_DISP_MARK
+};
+/* - I2C4 ------------------------------------------------------------------- */
+static const unsigned int i2c4_a_pins[] = {
+ /* SCL, SDA */
+ RCAR_GP_PIN(4, 10), RCAR_GP_PIN(4, 11),
+};
+static const unsigned int i2c4_a_mux[] = {
+ SCL4_A_MARK, SDA4_A_MARK,
+};
+static const unsigned int i2c4_b_pins[] = {
+ /* SCL, SDA */
+ RCAR_GP_PIN(5, 30), RCAR_GP_PIN(5, 31),
+};
+static const unsigned int i2c4_b_mux[] = {
+ SCL4_B_MARK, SDA4_B_MARK,
+};
+static const unsigned int i2c4_c_pins[] = {
+ /* SCL, SDA */
+ RCAR_GP_PIN(5, 4), RCAR_GP_PIN(5, 5),
+};
+static const unsigned int i2c4_c_mux[] = {
+ SCL4_C_MARK, SDA4_C_MARK,
+};
+static const unsigned int i2c4_d_pins[] = {
+ /* SCL, SDA */
+ RCAR_GP_PIN(2, 16), RCAR_GP_PIN(2, 17),
+};
+static const unsigned int i2c4_d_mux[] = {
+ SCL4_D_MARK, SDA4_D_MARK,
+};
+static const unsigned int i2c4_e_pins[] = {
+ /* SCL, SDA */
+ RCAR_GP_PIN(5, 7), RCAR_GP_PIN(5, 6),
+};
+static const unsigned int i2c4_e_mux[] = {
+ SCL4_E_MARK, SDA4_E_MARK,
+};
/* - MMC -------------------------------------------------------------------- */
static const unsigned int mmc_data1_pins[] = {
/* D0 */
static const unsigned int mmc_ctrl_mux[] = {
MMC0_CLK_SDHI1_CLK_MARK, MMC0_CMD_SDHI1_CMD_MARK,
};
+/* - QSPI ------------------------------------------------------------------- */
+static const unsigned int qspi0_ctrl_pins[] = {
+ /* SPCLK, SSL */
+ RCAR_GP_PIN(1, 16), RCAR_GP_PIN(1, 21),
+};
+static const unsigned int qspi0_ctrl_mux[] = {
+ QSPI0_SPCLK_MARK, QSPI0_SSL_MARK,
+};
+static const unsigned int qspi0_data2_pins[] = {
+ /* MOSI_IO0, MISO_IO1 */
+ RCAR_GP_PIN(1, 17), RCAR_GP_PIN(1, 18),
+};
+static const unsigned int qspi0_data2_mux[] = {
+ QSPI0_MOSI_QSPI0_IO0_MARK, QSPI0_MISO_QSPI0_IO1_MARK,
+};
+static const unsigned int qspi0_data4_pins[] = {
+ /* MOSI_IO0, MISO_IO1, IO2, IO3 */
+ RCAR_GP_PIN(1, 17), RCAR_GP_PIN(1, 18), RCAR_GP_PIN(1, 19),
+ RCAR_GP_PIN(1, 20),
+};
+static const unsigned int qspi0_data4_mux[] = {
+ QSPI0_MOSI_QSPI0_IO0_MARK, QSPI0_MISO_QSPI0_IO1_MARK,
+ QSPI0_IO2_MARK, QSPI0_IO3_MARK,
+};
/* - SCIF0 ------------------------------------------------------------------ */
static const unsigned int scif0_data_a_pins[] = {
/* RX, TX */
static const unsigned int scif_clk_b_mux[] = {
SCIF_CLK_B_MARK,
};
+/* - SDHI2 ------------------------------------------------------------------ */
+static const unsigned int sdhi2_data1_pins[] = {
+ /* D0 */
+ RCAR_GP_PIN(4, 16),
+};
+static const unsigned int sdhi2_data1_mux[] = {
+ SD2_DAT0_MARK,
+};
+static const unsigned int sdhi2_data4_pins[] = {
+ /* D[0:3] */
+ RCAR_GP_PIN(4, 16), RCAR_GP_PIN(4, 17),
+ RCAR_GP_PIN(4, 18), RCAR_GP_PIN(4, 19),
+};
+static const unsigned int sdhi2_data4_mux[] = {
+ SD2_DAT0_MARK, SD2_DAT1_MARK, SD2_DAT2_MARK, SD2_DAT3_MARK,
+};
+static const unsigned int sdhi2_ctrl_pins[] = {
+ /* CLK, CMD */
+ RCAR_GP_PIN(4, 14), RCAR_GP_PIN(4, 15),
+};
+static const unsigned int sdhi2_ctrl_mux[] = {
+ SD2_CLK_MARK, SD2_CMD_MARK,
+};
+static const unsigned int sdhi2_cd_pins[] = {
+ /* CD */
+ RCAR_GP_PIN(4, 20),
+};
+static const unsigned int sdhi2_cd_mux[] = {
+ SD2_CD_MARK,
+};
+static const unsigned int sdhi2_wp_pins[] = {
+ /* WP */
+ RCAR_GP_PIN(4, 21),
+};
+static const unsigned int sdhi2_wp_mux[] = {
+ SD2_WP_MARK,
+};
+/* - USB0 ------------------------------------------------------------------- */
+static const unsigned int usb0_pins[] = {
+ RCAR_GP_PIN(0, 0), /* PWEN */
+ RCAR_GP_PIN(0, 1), /* OVC */
+};
+static const unsigned int usb0_mux[] = {
+ USB0_PWEN_MARK,
+ USB0_OVC_MARK,
+};
+/* - USB1 ------------------------------------------------------------------- */
+static const unsigned int usb1_pins[] = {
+ RCAR_GP_PIN(0, 2), /* PWEN */
+ RCAR_GP_PIN(0, 3), /* OVC */
+};
+static const unsigned int usb1_mux[] = {
+ USB1_PWEN_MARK,
+ USB1_OVC_MARK,
+};
static const struct sh_pfc_pin_group pinmux_groups[] = {
+ SH_PFC_PIN_GROUP(avb_col),
+ SH_PFC_PIN_GROUP(avb_crs),
+ SH_PFC_PIN_GROUP(avb_link),
+ SH_PFC_PIN_GROUP(avb_magic),
+ SH_PFC_PIN_GROUP(avb_phy_int),
+ SH_PFC_PIN_GROUP(avb_mdio),
+ SH_PFC_PIN_GROUP(avb_mii_tx_rx),
+ SH_PFC_PIN_GROUP(avb_mii_tx_er),
+ SH_PFC_PIN_GROUP(avb_gmii_tx_rx),
+ SH_PFC_PIN_GROUP(avb_avtp_match_a),
+ SH_PFC_PIN_GROUP(avb_avtp_capture_a),
+ SH_PFC_PIN_GROUP(avb_avtp_match_b),
+ SH_PFC_PIN_GROUP(avb_avtp_capture_b),
+ SH_PFC_PIN_GROUP(du0_rgb666),
+ SH_PFC_PIN_GROUP(du0_rgb888),
+ SH_PFC_PIN_GROUP(du0_clk0_out),
+ SH_PFC_PIN_GROUP(du0_clk1_out),
+ SH_PFC_PIN_GROUP(du0_clk_in),
+ SH_PFC_PIN_GROUP(du0_sync),
+ SH_PFC_PIN_GROUP(du0_oddf),
+ SH_PFC_PIN_GROUP(du0_cde),
+ SH_PFC_PIN_GROUP(du0_disp),
+ SH_PFC_PIN_GROUP(i2c4_a),
+ SH_PFC_PIN_GROUP(i2c4_b),
+ SH_PFC_PIN_GROUP(i2c4_c),
+ SH_PFC_PIN_GROUP(i2c4_d),
+ SH_PFC_PIN_GROUP(i2c4_e),
SH_PFC_PIN_GROUP(mmc_data1),
SH_PFC_PIN_GROUP(mmc_data4),
SH_PFC_PIN_GROUP(mmc_data8),
SH_PFC_PIN_GROUP(mmc_ctrl),
+ SH_PFC_PIN_GROUP(qspi0_ctrl),
+ SH_PFC_PIN_GROUP(qspi0_data2),
+ SH_PFC_PIN_GROUP(qspi0_data4),
SH_PFC_PIN_GROUP(scif0_data_a),
SH_PFC_PIN_GROUP(scif0_data_b),
SH_PFC_PIN_GROUP(scif0_data_c),
SH_PFC_PIN_GROUP(scif5_data_f),
SH_PFC_PIN_GROUP(scif_clk_a),
SH_PFC_PIN_GROUP(scif_clk_b),
+ SH_PFC_PIN_GROUP(sdhi2_data1),
+ SH_PFC_PIN_GROUP(sdhi2_data4),
+ SH_PFC_PIN_GROUP(sdhi2_ctrl),
+ SH_PFC_PIN_GROUP(sdhi2_cd),
+ SH_PFC_PIN_GROUP(sdhi2_wp),
+ SH_PFC_PIN_GROUP(usb0),
+ SH_PFC_PIN_GROUP(usb1),
+};
+
+static const char * const avb_groups[] = {
+ "avb_col",
+ "avb_crs",
+ "avb_link",
+ "avb_magic",
+ "avb_phy_int",
+ "avb_mdio",
+ "avb_mii_tx_rx",
+ "avb_mii_tx_er",
+ "avb_gmii_tx_rx",
+ "avb_avtp_match_a",
+ "avb_avtp_capture_a",
+ "avb_avtp_match_b",
+ "avb_avtp_capture_b",
+};
+
+static const char * const du0_groups[] = {
+ "du0_rgb666",
+ "du0_rgb888",
+ "du0_clk0_out",
+ "du0_clk1_out",
+ "du0_clk_in",
+ "du0_sync",
+ "du0_oddf",
+ "du0_cde",
+ "du0_disp",
+};
+
+static const char * const i2c4_groups[] = {
+ "i2c4_a",
+ "i2c4_b",
+ "i2c4_c",
+ "i2c4_d",
+ "i2c4_e",
};
static const char * const mmc_groups[] = {
"mmc_ctrl",
};
+static const char * const qspi0_groups[] = {
+ "qspi0_ctrl",
+ "qspi0_data2",
+ "qspi0_data4",
+};
+
static const char * const scif0_groups[] = {
"scif0_data_a",
"scif0_data_b",
"scif_clk_b",
};
+static const char * const sdhi2_groups[] = {
+ "sdhi2_data1",
+ "sdhi2_data4",
+ "sdhi2_ctrl",
+ "sdhi2_cd",
+ "sdhi2_wp",
+};
+
+static const char * const usb0_groups[] = {
+ "usb0",
+};
+
+static const char * const usb1_groups[] = {
+ "usb1",
+};
+
static const struct sh_pfc_function pinmux_functions[] = {
+ SH_PFC_FUNCTION(avb),
+ SH_PFC_FUNCTION(du0),
+ SH_PFC_FUNCTION(i2c4),
SH_PFC_FUNCTION(mmc),
+ SH_PFC_FUNCTION(qspi0),
SH_PFC_FUNCTION(scif0),
SH_PFC_FUNCTION(scif1),
SH_PFC_FUNCTION(scif2),
SH_PFC_FUNCTION(scif4),
SH_PFC_FUNCTION(scif5),
SH_PFC_FUNCTION(scif_clk),
+ SH_PFC_FUNCTION(sdhi2),
+ SH_PFC_FUNCTION(usb0),
+ SH_PFC_FUNCTION(usb1),
};
static const struct pinmux_cfg_reg pinmux_config_regs[] = {
+// SPDX-License-Identifier: GPL-2.0
/*
* r8a7778 processor support - PFC hardware block
*
* based on
* Copyright (C) 2011 Renesas Solutions Corp.
* Copyright (C) 2011 Magnus Damm
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
#include <linux/io.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* r8a7779 processor support - PFC hardware block
*
* Copyright (C) 2011, 2013 Renesas Solutions Corp.
* Copyright (C) 2011 Magnus Damm
* Copyright (C) 2013 Cogent Embedded, Inc.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* R8A7790 processor support
*
* Copyright (C) 2013 Magnus Damm
* Copyright (C) 2012 Renesas Solutions Corp.
* Copyright (C) 2012 Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; version 2 of the
- * License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
#include <linux/io.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* r8a7791/r8a7743 processor support - PFC hardware block.
*
* Copyright (C) 2013 Renesas Electronics Corporation
* Copyright (C) 2014-2017 Cogent Embedded, Inc.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2
- * as published by the Free Software Foundation.
*/
#include <linux/kernel.h>
static const struct {
struct sh_pfc_pin_group common[346];
- struct sh_pfc_pin_group r8a779x[9];
+ struct sh_pfc_pin_group automotive[9];
} pinmux_groups = {
.common = {
SH_PFC_PIN_GROUP(audio_clk_a),
SH_PFC_PIN_GROUP(vin2_clkenb),
SH_PFC_PIN_GROUP(vin2_clk),
},
- .r8a779x = {
+ .automotive = {
SH_PFC_PIN_GROUP(adi_common),
SH_PFC_PIN_GROUP(adi_chsel0),
SH_PFC_PIN_GROUP(adi_chsel1),
static const struct {
struct sh_pfc_function common[58];
- struct sh_pfc_function r8a779x[2];
+ struct sh_pfc_function automotive[2];
} pinmux_functions = {
.common = {
SH_PFC_FUNCTION(audio_clk),
SH_PFC_FUNCTION(vin1),
SH_PFC_FUNCTION(vin2),
},
- .r8a779x = {
+ .automotive = {
SH_PFC_FUNCTION(adi),
SH_PFC_FUNCTION(mlb),
}
};
#endif
+#ifdef CONFIG_PINCTRL_PFC_R8A7744
+const struct sh_pfc_soc_info r8a7744_pinmux_info = {
+ .name = "r8a77440_pfc",
+ .ops = &r8a7791_pinmux_ops,
+ .unlock_reg = 0xe6060000, /* PMMR */
+
+ .function = { PINMUX_FUNCTION_BEGIN, PINMUX_FUNCTION_END },
+
+ .pins = pinmux_pins,
+ .nr_pins = ARRAY_SIZE(pinmux_pins),
+ .groups = pinmux_groups.common,
+ .nr_groups = ARRAY_SIZE(pinmux_groups.common),
+ .functions = pinmux_functions.common,
+ .nr_functions = ARRAY_SIZE(pinmux_functions.common),
+
+ .cfg_regs = pinmux_config_regs,
+
+ .pinmux_data = pinmux_data,
+ .pinmux_data_size = ARRAY_SIZE(pinmux_data),
+};
+#endif
+
#ifdef CONFIG_PINCTRL_PFC_R8A7791
const struct sh_pfc_soc_info r8a7791_pinmux_info = {
.name = "r8a77910_pfc",
.nr_pins = ARRAY_SIZE(pinmux_pins),
.groups = pinmux_groups.common,
.nr_groups = ARRAY_SIZE(pinmux_groups.common) +
- ARRAY_SIZE(pinmux_groups.r8a779x),
+ ARRAY_SIZE(pinmux_groups.automotive),
.functions = pinmux_functions.common,
.nr_functions = ARRAY_SIZE(pinmux_functions.common) +
- ARRAY_SIZE(pinmux_functions.r8a779x),
+ ARRAY_SIZE(pinmux_functions.automotive),
.cfg_regs = pinmux_config_regs,
.nr_pins = ARRAY_SIZE(pinmux_pins),
.groups = pinmux_groups.common,
.nr_groups = ARRAY_SIZE(pinmux_groups.common) +
- ARRAY_SIZE(pinmux_groups.r8a779x),
+ ARRAY_SIZE(pinmux_groups.automotive),
.functions = pinmux_functions.common,
.nr_functions = ARRAY_SIZE(pinmux_functions.common) +
- ARRAY_SIZE(pinmux_functions.r8a779x),
+ ARRAY_SIZE(pinmux_functions.automotive),
.cfg_regs = pinmux_config_regs,
+// SPDX-License-Identifier: GPL-2.0
/*
* r8a7792 processor support - PFC hardware block.
*
* Copyright (C) 2013-2014 Renesas Electronics Corporation
* Copyright (C) 2016 Cogent Embedded, Inc., <source@cogentembedded.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2
- * as published by the Free Software Foundation.
*/
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* r8a7794/r8a7745 processor support - PFC hardware block.
*
* Copyright (C) 2014-2015 Renesas Electronics Corporation
* Copyright (C) 2015 Renesas Solutions Corp.
* Copyright (C) 2015-2017 Cogent Embedded, Inc. <source@cogentembedded.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2
- * as published by the Free Software Foundation.
*/
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* R8A7795 ES1.x processor support - PFC hardware block.
*
* Copyright (C) 2015-2017 Renesas Electronics Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
*/
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* R8A7795 ES2.0+ processor support - PFC hardware block.
*
* Copyright (C) 2015-2017 Renesas Electronics Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
*/
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* R8A7796 processor support - PFC hardware block.
*
* R-Car Gen3 processor support - PFC hardware block.
*
* Copyright (C) 2015 Renesas Electronics Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
*/
#include <linux/kernel.h>
VI5_CLK_MARK,
};
-static const struct sh_pfc_pin_group pinmux_groups[] = {
- SH_PFC_PIN_GROUP(audio_clk_a_a),
- SH_PFC_PIN_GROUP(audio_clk_a_b),
- SH_PFC_PIN_GROUP(audio_clk_a_c),
- SH_PFC_PIN_GROUP(audio_clk_b_a),
- SH_PFC_PIN_GROUP(audio_clk_b_b),
- SH_PFC_PIN_GROUP(audio_clk_c_a),
- SH_PFC_PIN_GROUP(audio_clk_c_b),
- SH_PFC_PIN_GROUP(audio_clkout_a),
- SH_PFC_PIN_GROUP(audio_clkout_b),
- SH_PFC_PIN_GROUP(audio_clkout_c),
- SH_PFC_PIN_GROUP(audio_clkout_d),
- SH_PFC_PIN_GROUP(audio_clkout1_a),
- SH_PFC_PIN_GROUP(audio_clkout1_b),
- SH_PFC_PIN_GROUP(audio_clkout2_a),
- SH_PFC_PIN_GROUP(audio_clkout2_b),
- SH_PFC_PIN_GROUP(audio_clkout3_a),
- SH_PFC_PIN_GROUP(audio_clkout3_b),
- SH_PFC_PIN_GROUP(avb_link),
- SH_PFC_PIN_GROUP(avb_magic),
- SH_PFC_PIN_GROUP(avb_phy_int),
- SH_PFC_PIN_GROUP_ALIAS(avb_mdc, avb_mdio), /* Deprecated */
- SH_PFC_PIN_GROUP(avb_mdio),
- SH_PFC_PIN_GROUP(avb_mii),
- SH_PFC_PIN_GROUP(avb_avtp_pps),
- SH_PFC_PIN_GROUP(avb_avtp_match_a),
- SH_PFC_PIN_GROUP(avb_avtp_capture_a),
- SH_PFC_PIN_GROUP(avb_avtp_match_b),
- SH_PFC_PIN_GROUP(avb_avtp_capture_b),
- SH_PFC_PIN_GROUP(can0_data_a),
- SH_PFC_PIN_GROUP(can0_data_b),
- SH_PFC_PIN_GROUP(can1_data),
- SH_PFC_PIN_GROUP(can_clk),
- SH_PFC_PIN_GROUP(canfd0_data_a),
- SH_PFC_PIN_GROUP(canfd0_data_b),
- SH_PFC_PIN_GROUP(canfd1_data),
- SH_PFC_PIN_GROUP(drif0_ctrl_a),
- SH_PFC_PIN_GROUP(drif0_data0_a),
- SH_PFC_PIN_GROUP(drif0_data1_a),
- SH_PFC_PIN_GROUP(drif0_ctrl_b),
- SH_PFC_PIN_GROUP(drif0_data0_b),
- SH_PFC_PIN_GROUP(drif0_data1_b),
- SH_PFC_PIN_GROUP(drif0_ctrl_c),
- SH_PFC_PIN_GROUP(drif0_data0_c),
- SH_PFC_PIN_GROUP(drif0_data1_c),
- SH_PFC_PIN_GROUP(drif1_ctrl_a),
- SH_PFC_PIN_GROUP(drif1_data0_a),
- SH_PFC_PIN_GROUP(drif1_data1_a),
- SH_PFC_PIN_GROUP(drif1_ctrl_b),
- SH_PFC_PIN_GROUP(drif1_data0_b),
- SH_PFC_PIN_GROUP(drif1_data1_b),
- SH_PFC_PIN_GROUP(drif1_ctrl_c),
- SH_PFC_PIN_GROUP(drif1_data0_c),
- SH_PFC_PIN_GROUP(drif1_data1_c),
- SH_PFC_PIN_GROUP(drif2_ctrl_a),
- SH_PFC_PIN_GROUP(drif2_data0_a),
- SH_PFC_PIN_GROUP(drif2_data1_a),
- SH_PFC_PIN_GROUP(drif2_ctrl_b),
- SH_PFC_PIN_GROUP(drif2_data0_b),
- SH_PFC_PIN_GROUP(drif2_data1_b),
- SH_PFC_PIN_GROUP(drif3_ctrl_a),
- SH_PFC_PIN_GROUP(drif3_data0_a),
- SH_PFC_PIN_GROUP(drif3_data1_a),
- SH_PFC_PIN_GROUP(drif3_ctrl_b),
- SH_PFC_PIN_GROUP(drif3_data0_b),
- SH_PFC_PIN_GROUP(drif3_data1_b),
- SH_PFC_PIN_GROUP(du_rgb666),
- SH_PFC_PIN_GROUP(du_rgb888),
- SH_PFC_PIN_GROUP(du_clk_out_0),
- SH_PFC_PIN_GROUP(du_clk_out_1),
- SH_PFC_PIN_GROUP(du_sync),
- SH_PFC_PIN_GROUP(du_oddf),
- SH_PFC_PIN_GROUP(du_cde),
- SH_PFC_PIN_GROUP(du_disp),
- SH_PFC_PIN_GROUP(hdmi0_cec),
- SH_PFC_PIN_GROUP(hscif0_data),
- SH_PFC_PIN_GROUP(hscif0_clk),
- SH_PFC_PIN_GROUP(hscif0_ctrl),
- SH_PFC_PIN_GROUP(hscif1_data_a),
- SH_PFC_PIN_GROUP(hscif1_clk_a),
- SH_PFC_PIN_GROUP(hscif1_ctrl_a),
- SH_PFC_PIN_GROUP(hscif1_data_b),
- SH_PFC_PIN_GROUP(hscif1_clk_b),
- SH_PFC_PIN_GROUP(hscif1_ctrl_b),
- SH_PFC_PIN_GROUP(hscif2_data_a),
- SH_PFC_PIN_GROUP(hscif2_clk_a),
- SH_PFC_PIN_GROUP(hscif2_ctrl_a),
- SH_PFC_PIN_GROUP(hscif2_data_b),
- SH_PFC_PIN_GROUP(hscif2_clk_b),
- SH_PFC_PIN_GROUP(hscif2_ctrl_b),
- SH_PFC_PIN_GROUP(hscif2_data_c),
- SH_PFC_PIN_GROUP(hscif2_clk_c),
- SH_PFC_PIN_GROUP(hscif2_ctrl_c),
- SH_PFC_PIN_GROUP(hscif3_data_a),
- SH_PFC_PIN_GROUP(hscif3_clk),
- SH_PFC_PIN_GROUP(hscif3_ctrl),
- SH_PFC_PIN_GROUP(hscif3_data_b),
- SH_PFC_PIN_GROUP(hscif3_data_c),
- SH_PFC_PIN_GROUP(hscif3_data_d),
- SH_PFC_PIN_GROUP(hscif4_data_a),
- SH_PFC_PIN_GROUP(hscif4_clk),
- SH_PFC_PIN_GROUP(hscif4_ctrl),
- SH_PFC_PIN_GROUP(hscif4_data_b),
- SH_PFC_PIN_GROUP(i2c1_a),
- SH_PFC_PIN_GROUP(i2c1_b),
- SH_PFC_PIN_GROUP(i2c2_a),
- SH_PFC_PIN_GROUP(i2c2_b),
- SH_PFC_PIN_GROUP(i2c6_a),
- SH_PFC_PIN_GROUP(i2c6_b),
- SH_PFC_PIN_GROUP(i2c6_c),
- SH_PFC_PIN_GROUP(intc_ex_irq0),
- SH_PFC_PIN_GROUP(intc_ex_irq1),
- SH_PFC_PIN_GROUP(intc_ex_irq2),
- SH_PFC_PIN_GROUP(intc_ex_irq3),
- SH_PFC_PIN_GROUP(intc_ex_irq4),
- SH_PFC_PIN_GROUP(intc_ex_irq5),
- SH_PFC_PIN_GROUP(msiof0_clk),
- SH_PFC_PIN_GROUP(msiof0_sync),
- SH_PFC_PIN_GROUP(msiof0_ss1),
- SH_PFC_PIN_GROUP(msiof0_ss2),
- SH_PFC_PIN_GROUP(msiof0_txd),
- SH_PFC_PIN_GROUP(msiof0_rxd),
- SH_PFC_PIN_GROUP(msiof1_clk_a),
- SH_PFC_PIN_GROUP(msiof1_sync_a),
- SH_PFC_PIN_GROUP(msiof1_ss1_a),
- SH_PFC_PIN_GROUP(msiof1_ss2_a),
- SH_PFC_PIN_GROUP(msiof1_txd_a),
- SH_PFC_PIN_GROUP(msiof1_rxd_a),
- SH_PFC_PIN_GROUP(msiof1_clk_b),
- SH_PFC_PIN_GROUP(msiof1_sync_b),
- SH_PFC_PIN_GROUP(msiof1_ss1_b),
- SH_PFC_PIN_GROUP(msiof1_ss2_b),
- SH_PFC_PIN_GROUP(msiof1_txd_b),
- SH_PFC_PIN_GROUP(msiof1_rxd_b),
- SH_PFC_PIN_GROUP(msiof1_clk_c),
- SH_PFC_PIN_GROUP(msiof1_sync_c),
- SH_PFC_PIN_GROUP(msiof1_ss1_c),
- SH_PFC_PIN_GROUP(msiof1_ss2_c),
- SH_PFC_PIN_GROUP(msiof1_txd_c),
- SH_PFC_PIN_GROUP(msiof1_rxd_c),
- SH_PFC_PIN_GROUP(msiof1_clk_d),
- SH_PFC_PIN_GROUP(msiof1_sync_d),
- SH_PFC_PIN_GROUP(msiof1_ss1_d),
- SH_PFC_PIN_GROUP(msiof1_ss2_d),
- SH_PFC_PIN_GROUP(msiof1_txd_d),
- SH_PFC_PIN_GROUP(msiof1_rxd_d),
- SH_PFC_PIN_GROUP(msiof1_clk_e),
- SH_PFC_PIN_GROUP(msiof1_sync_e),
- SH_PFC_PIN_GROUP(msiof1_ss1_e),
- SH_PFC_PIN_GROUP(msiof1_ss2_e),
- SH_PFC_PIN_GROUP(msiof1_txd_e),
- SH_PFC_PIN_GROUP(msiof1_rxd_e),
- SH_PFC_PIN_GROUP(msiof1_clk_f),
- SH_PFC_PIN_GROUP(msiof1_sync_f),
- SH_PFC_PIN_GROUP(msiof1_ss1_f),
- SH_PFC_PIN_GROUP(msiof1_ss2_f),
- SH_PFC_PIN_GROUP(msiof1_txd_f),
- SH_PFC_PIN_GROUP(msiof1_rxd_f),
- SH_PFC_PIN_GROUP(msiof1_clk_g),
- SH_PFC_PIN_GROUP(msiof1_sync_g),
- SH_PFC_PIN_GROUP(msiof1_ss1_g),
- SH_PFC_PIN_GROUP(msiof1_ss2_g),
- SH_PFC_PIN_GROUP(msiof1_txd_g),
- SH_PFC_PIN_GROUP(msiof1_rxd_g),
- SH_PFC_PIN_GROUP(msiof2_clk_a),
- SH_PFC_PIN_GROUP(msiof2_sync_a),
- SH_PFC_PIN_GROUP(msiof2_ss1_a),
- SH_PFC_PIN_GROUP(msiof2_ss2_a),
- SH_PFC_PIN_GROUP(msiof2_txd_a),
- SH_PFC_PIN_GROUP(msiof2_rxd_a),
- SH_PFC_PIN_GROUP(msiof2_clk_b),
- SH_PFC_PIN_GROUP(msiof2_sync_b),
- SH_PFC_PIN_GROUP(msiof2_ss1_b),
- SH_PFC_PIN_GROUP(msiof2_ss2_b),
- SH_PFC_PIN_GROUP(msiof2_txd_b),
- SH_PFC_PIN_GROUP(msiof2_rxd_b),
- SH_PFC_PIN_GROUP(msiof2_clk_c),
- SH_PFC_PIN_GROUP(msiof2_sync_c),
- SH_PFC_PIN_GROUP(msiof2_ss1_c),
- SH_PFC_PIN_GROUP(msiof2_ss2_c),
- SH_PFC_PIN_GROUP(msiof2_txd_c),
- SH_PFC_PIN_GROUP(msiof2_rxd_c),
- SH_PFC_PIN_GROUP(msiof2_clk_d),
- SH_PFC_PIN_GROUP(msiof2_sync_d),
- SH_PFC_PIN_GROUP(msiof2_ss1_d),
- SH_PFC_PIN_GROUP(msiof2_ss2_d),
- SH_PFC_PIN_GROUP(msiof2_txd_d),
- SH_PFC_PIN_GROUP(msiof2_rxd_d),
- SH_PFC_PIN_GROUP(msiof3_clk_a),
- SH_PFC_PIN_GROUP(msiof3_sync_a),
- SH_PFC_PIN_GROUP(msiof3_ss1_a),
- SH_PFC_PIN_GROUP(msiof3_ss2_a),
- SH_PFC_PIN_GROUP(msiof3_txd_a),
- SH_PFC_PIN_GROUP(msiof3_rxd_a),
- SH_PFC_PIN_GROUP(msiof3_clk_b),
- SH_PFC_PIN_GROUP(msiof3_sync_b),
- SH_PFC_PIN_GROUP(msiof3_ss1_b),
- SH_PFC_PIN_GROUP(msiof3_ss2_b),
- SH_PFC_PIN_GROUP(msiof3_txd_b),
- SH_PFC_PIN_GROUP(msiof3_rxd_b),
- SH_PFC_PIN_GROUP(msiof3_clk_c),
- SH_PFC_PIN_GROUP(msiof3_sync_c),
- SH_PFC_PIN_GROUP(msiof3_txd_c),
- SH_PFC_PIN_GROUP(msiof3_rxd_c),
- SH_PFC_PIN_GROUP(msiof3_clk_d),
- SH_PFC_PIN_GROUP(msiof3_sync_d),
- SH_PFC_PIN_GROUP(msiof3_ss1_d),
- SH_PFC_PIN_GROUP(msiof3_txd_d),
- SH_PFC_PIN_GROUP(msiof3_rxd_d),
- SH_PFC_PIN_GROUP(msiof3_clk_e),
- SH_PFC_PIN_GROUP(msiof3_sync_e),
- SH_PFC_PIN_GROUP(msiof3_ss1_e),
- SH_PFC_PIN_GROUP(msiof3_ss2_e),
- SH_PFC_PIN_GROUP(msiof3_txd_e),
- SH_PFC_PIN_GROUP(msiof3_rxd_e),
- SH_PFC_PIN_GROUP(pwm0),
- SH_PFC_PIN_GROUP(pwm1_a),
- SH_PFC_PIN_GROUP(pwm1_b),
- SH_PFC_PIN_GROUP(pwm2_a),
- SH_PFC_PIN_GROUP(pwm2_b),
- SH_PFC_PIN_GROUP(pwm3_a),
- SH_PFC_PIN_GROUP(pwm3_b),
- SH_PFC_PIN_GROUP(pwm4_a),
- SH_PFC_PIN_GROUP(pwm4_b),
- SH_PFC_PIN_GROUP(pwm5_a),
- SH_PFC_PIN_GROUP(pwm5_b),
- SH_PFC_PIN_GROUP(pwm6_a),
- SH_PFC_PIN_GROUP(pwm6_b),
- SH_PFC_PIN_GROUP(scif0_data),
- SH_PFC_PIN_GROUP(scif0_clk),
- SH_PFC_PIN_GROUP(scif0_ctrl),
- SH_PFC_PIN_GROUP(scif1_data_a),
- SH_PFC_PIN_GROUP(scif1_clk),
- SH_PFC_PIN_GROUP(scif1_ctrl),
- SH_PFC_PIN_GROUP(scif1_data_b),
- SH_PFC_PIN_GROUP(scif2_data_a),
- SH_PFC_PIN_GROUP(scif2_clk),
- SH_PFC_PIN_GROUP(scif2_data_b),
- SH_PFC_PIN_GROUP(scif3_data_a),
- SH_PFC_PIN_GROUP(scif3_clk),
- SH_PFC_PIN_GROUP(scif3_ctrl),
- SH_PFC_PIN_GROUP(scif3_data_b),
- SH_PFC_PIN_GROUP(scif4_data_a),
- SH_PFC_PIN_GROUP(scif4_clk_a),
- SH_PFC_PIN_GROUP(scif4_ctrl_a),
- SH_PFC_PIN_GROUP(scif4_data_b),
- SH_PFC_PIN_GROUP(scif4_clk_b),
- SH_PFC_PIN_GROUP(scif4_ctrl_b),
- SH_PFC_PIN_GROUP(scif4_data_c),
- SH_PFC_PIN_GROUP(scif4_clk_c),
- SH_PFC_PIN_GROUP(scif4_ctrl_c),
- SH_PFC_PIN_GROUP(scif5_data_a),
- SH_PFC_PIN_GROUP(scif5_clk_a),
- SH_PFC_PIN_GROUP(scif5_data_b),
- SH_PFC_PIN_GROUP(scif5_clk_b),
- SH_PFC_PIN_GROUP(scif_clk_a),
- SH_PFC_PIN_GROUP(scif_clk_b),
- SH_PFC_PIN_GROUP(sdhi0_data1),
- SH_PFC_PIN_GROUP(sdhi0_data4),
- SH_PFC_PIN_GROUP(sdhi0_ctrl),
- SH_PFC_PIN_GROUP(sdhi0_cd),
- SH_PFC_PIN_GROUP(sdhi0_wp),
- SH_PFC_PIN_GROUP(sdhi1_data1),
- SH_PFC_PIN_GROUP(sdhi1_data4),
- SH_PFC_PIN_GROUP(sdhi1_ctrl),
- SH_PFC_PIN_GROUP(sdhi1_cd),
- SH_PFC_PIN_GROUP(sdhi1_wp),
- SH_PFC_PIN_GROUP(sdhi2_data1),
- SH_PFC_PIN_GROUP(sdhi2_data4),
- SH_PFC_PIN_GROUP(sdhi2_data8),
- SH_PFC_PIN_GROUP(sdhi2_ctrl),
- SH_PFC_PIN_GROUP(sdhi2_cd_a),
- SH_PFC_PIN_GROUP(sdhi2_wp_a),
- SH_PFC_PIN_GROUP(sdhi2_cd_b),
- SH_PFC_PIN_GROUP(sdhi2_wp_b),
- SH_PFC_PIN_GROUP(sdhi2_ds),
- SH_PFC_PIN_GROUP(sdhi3_data1),
- SH_PFC_PIN_GROUP(sdhi3_data4),
- SH_PFC_PIN_GROUP(sdhi3_data8),
- SH_PFC_PIN_GROUP(sdhi3_ctrl),
- SH_PFC_PIN_GROUP(sdhi3_cd),
- SH_PFC_PIN_GROUP(sdhi3_wp),
- SH_PFC_PIN_GROUP(sdhi3_ds),
- SH_PFC_PIN_GROUP(ssi0_data),
- SH_PFC_PIN_GROUP(ssi01239_ctrl),
- SH_PFC_PIN_GROUP(ssi1_data_a),
- SH_PFC_PIN_GROUP(ssi1_data_b),
- SH_PFC_PIN_GROUP(ssi1_ctrl_a),
- SH_PFC_PIN_GROUP(ssi1_ctrl_b),
- SH_PFC_PIN_GROUP(ssi2_data_a),
- SH_PFC_PIN_GROUP(ssi2_data_b),
- SH_PFC_PIN_GROUP(ssi2_ctrl_a),
- SH_PFC_PIN_GROUP(ssi2_ctrl_b),
- SH_PFC_PIN_GROUP(ssi3_data),
- SH_PFC_PIN_GROUP(ssi349_ctrl),
- SH_PFC_PIN_GROUP(ssi4_data),
- SH_PFC_PIN_GROUP(ssi4_ctrl),
- SH_PFC_PIN_GROUP(ssi5_data),
- SH_PFC_PIN_GROUP(ssi5_ctrl),
- SH_PFC_PIN_GROUP(ssi6_data),
- SH_PFC_PIN_GROUP(ssi6_ctrl),
- SH_PFC_PIN_GROUP(ssi7_data),
- SH_PFC_PIN_GROUP(ssi78_ctrl),
- SH_PFC_PIN_GROUP(ssi8_data),
- SH_PFC_PIN_GROUP(ssi9_data_a),
- SH_PFC_PIN_GROUP(ssi9_data_b),
- SH_PFC_PIN_GROUP(ssi9_ctrl_a),
- SH_PFC_PIN_GROUP(ssi9_ctrl_b),
- SH_PFC_PIN_GROUP(tmu_tclk1_a),
- SH_PFC_PIN_GROUP(tmu_tclk1_b),
- SH_PFC_PIN_GROUP(tmu_tclk2_a),
- SH_PFC_PIN_GROUP(tmu_tclk2_b),
- SH_PFC_PIN_GROUP(usb0),
- SH_PFC_PIN_GROUP(usb1),
- SH_PFC_PIN_GROUP(usb30),
- VIN_DATA_PIN_GROUP(vin4_data_a, 8),
- VIN_DATA_PIN_GROUP(vin4_data_a, 10),
- VIN_DATA_PIN_GROUP(vin4_data_a, 12),
- VIN_DATA_PIN_GROUP(vin4_data_a, 16),
- SH_PFC_PIN_GROUP(vin4_data18_a),
- VIN_DATA_PIN_GROUP(vin4_data_a, 20),
- VIN_DATA_PIN_GROUP(vin4_data_a, 24),
- VIN_DATA_PIN_GROUP(vin4_data_b, 8),
- VIN_DATA_PIN_GROUP(vin4_data_b, 10),
- VIN_DATA_PIN_GROUP(vin4_data_b, 12),
- VIN_DATA_PIN_GROUP(vin4_data_b, 16),
- SH_PFC_PIN_GROUP(vin4_data18_b),
- VIN_DATA_PIN_GROUP(vin4_data_b, 20),
- VIN_DATA_PIN_GROUP(vin4_data_b, 24),
- SH_PFC_PIN_GROUP(vin4_sync),
- SH_PFC_PIN_GROUP(vin4_field),
- SH_PFC_PIN_GROUP(vin4_clkenb),
- SH_PFC_PIN_GROUP(vin4_clk),
- SH_PFC_PIN_GROUP(vin5_data8),
- SH_PFC_PIN_GROUP(vin5_data10),
- SH_PFC_PIN_GROUP(vin5_data12),
- SH_PFC_PIN_GROUP(vin5_data16),
- SH_PFC_PIN_GROUP(vin5_sync),
- SH_PFC_PIN_GROUP(vin5_field),
- SH_PFC_PIN_GROUP(vin5_clkenb),
- SH_PFC_PIN_GROUP(vin5_clk),
+static const struct {
+ struct sh_pfc_pin_group common[307];
+ struct sh_pfc_pin_group automotive[33];
+} pinmux_groups = {
+ .common = {
+ SH_PFC_PIN_GROUP(audio_clk_a_a),
+ SH_PFC_PIN_GROUP(audio_clk_a_b),
+ SH_PFC_PIN_GROUP(audio_clk_a_c),
+ SH_PFC_PIN_GROUP(audio_clk_b_a),
+ SH_PFC_PIN_GROUP(audio_clk_b_b),
+ SH_PFC_PIN_GROUP(audio_clk_c_a),
+ SH_PFC_PIN_GROUP(audio_clk_c_b),
+ SH_PFC_PIN_GROUP(audio_clkout_a),
+ SH_PFC_PIN_GROUP(audio_clkout_b),
+ SH_PFC_PIN_GROUP(audio_clkout_c),
+ SH_PFC_PIN_GROUP(audio_clkout_d),
+ SH_PFC_PIN_GROUP(audio_clkout1_a),
+ SH_PFC_PIN_GROUP(audio_clkout1_b),
+ SH_PFC_PIN_GROUP(audio_clkout2_a),
+ SH_PFC_PIN_GROUP(audio_clkout2_b),
+ SH_PFC_PIN_GROUP(audio_clkout3_a),
+ SH_PFC_PIN_GROUP(audio_clkout3_b),
+ SH_PFC_PIN_GROUP(avb_link),
+ SH_PFC_PIN_GROUP(avb_magic),
+ SH_PFC_PIN_GROUP(avb_phy_int),
+ SH_PFC_PIN_GROUP_ALIAS(avb_mdc, avb_mdio), /* Deprecated */
+ SH_PFC_PIN_GROUP(avb_mdio),
+ SH_PFC_PIN_GROUP(avb_mii),
+ SH_PFC_PIN_GROUP(avb_avtp_pps),
+ SH_PFC_PIN_GROUP(avb_avtp_match_a),
+ SH_PFC_PIN_GROUP(avb_avtp_capture_a),
+ SH_PFC_PIN_GROUP(avb_avtp_match_b),
+ SH_PFC_PIN_GROUP(avb_avtp_capture_b),
+ SH_PFC_PIN_GROUP(can0_data_a),
+ SH_PFC_PIN_GROUP(can0_data_b),
+ SH_PFC_PIN_GROUP(can1_data),
+ SH_PFC_PIN_GROUP(can_clk),
+ SH_PFC_PIN_GROUP(du_rgb666),
+ SH_PFC_PIN_GROUP(du_rgb888),
+ SH_PFC_PIN_GROUP(du_clk_out_0),
+ SH_PFC_PIN_GROUP(du_clk_out_1),
+ SH_PFC_PIN_GROUP(du_sync),
+ SH_PFC_PIN_GROUP(du_oddf),
+ SH_PFC_PIN_GROUP(du_cde),
+ SH_PFC_PIN_GROUP(du_disp),
+ SH_PFC_PIN_GROUP(hdmi0_cec),
+ SH_PFC_PIN_GROUP(hscif0_data),
+ SH_PFC_PIN_GROUP(hscif0_clk),
+ SH_PFC_PIN_GROUP(hscif0_ctrl),
+ SH_PFC_PIN_GROUP(hscif1_data_a),
+ SH_PFC_PIN_GROUP(hscif1_clk_a),
+ SH_PFC_PIN_GROUP(hscif1_ctrl_a),
+ SH_PFC_PIN_GROUP(hscif1_data_b),
+ SH_PFC_PIN_GROUP(hscif1_clk_b),
+ SH_PFC_PIN_GROUP(hscif1_ctrl_b),
+ SH_PFC_PIN_GROUP(hscif2_data_a),
+ SH_PFC_PIN_GROUP(hscif2_clk_a),
+ SH_PFC_PIN_GROUP(hscif2_ctrl_a),
+ SH_PFC_PIN_GROUP(hscif2_data_b),
+ SH_PFC_PIN_GROUP(hscif2_clk_b),
+ SH_PFC_PIN_GROUP(hscif2_ctrl_b),
+ SH_PFC_PIN_GROUP(hscif2_data_c),
+ SH_PFC_PIN_GROUP(hscif2_clk_c),
+ SH_PFC_PIN_GROUP(hscif2_ctrl_c),
+ SH_PFC_PIN_GROUP(hscif3_data_a),
+ SH_PFC_PIN_GROUP(hscif3_clk),
+ SH_PFC_PIN_GROUP(hscif3_ctrl),
+ SH_PFC_PIN_GROUP(hscif3_data_b),
+ SH_PFC_PIN_GROUP(hscif3_data_c),
+ SH_PFC_PIN_GROUP(hscif3_data_d),
+ SH_PFC_PIN_GROUP(hscif4_data_a),
+ SH_PFC_PIN_GROUP(hscif4_clk),
+ SH_PFC_PIN_GROUP(hscif4_ctrl),
+ SH_PFC_PIN_GROUP(hscif4_data_b),
+ SH_PFC_PIN_GROUP(i2c1_a),
+ SH_PFC_PIN_GROUP(i2c1_b),
+ SH_PFC_PIN_GROUP(i2c2_a),
+ SH_PFC_PIN_GROUP(i2c2_b),
+ SH_PFC_PIN_GROUP(i2c6_a),
+ SH_PFC_PIN_GROUP(i2c6_b),
+ SH_PFC_PIN_GROUP(i2c6_c),
+ SH_PFC_PIN_GROUP(intc_ex_irq0),
+ SH_PFC_PIN_GROUP(intc_ex_irq1),
+ SH_PFC_PIN_GROUP(intc_ex_irq2),
+ SH_PFC_PIN_GROUP(intc_ex_irq3),
+ SH_PFC_PIN_GROUP(intc_ex_irq4),
+ SH_PFC_PIN_GROUP(intc_ex_irq5),
+ SH_PFC_PIN_GROUP(msiof0_clk),
+ SH_PFC_PIN_GROUP(msiof0_sync),
+ SH_PFC_PIN_GROUP(msiof0_ss1),
+ SH_PFC_PIN_GROUP(msiof0_ss2),
+ SH_PFC_PIN_GROUP(msiof0_txd),
+ SH_PFC_PIN_GROUP(msiof0_rxd),
+ SH_PFC_PIN_GROUP(msiof1_clk_a),
+ SH_PFC_PIN_GROUP(msiof1_sync_a),
+ SH_PFC_PIN_GROUP(msiof1_ss1_a),
+ SH_PFC_PIN_GROUP(msiof1_ss2_a),
+ SH_PFC_PIN_GROUP(msiof1_txd_a),
+ SH_PFC_PIN_GROUP(msiof1_rxd_a),
+ SH_PFC_PIN_GROUP(msiof1_clk_b),
+ SH_PFC_PIN_GROUP(msiof1_sync_b),
+ SH_PFC_PIN_GROUP(msiof1_ss1_b),
+ SH_PFC_PIN_GROUP(msiof1_ss2_b),
+ SH_PFC_PIN_GROUP(msiof1_txd_b),
+ SH_PFC_PIN_GROUP(msiof1_rxd_b),
+ SH_PFC_PIN_GROUP(msiof1_clk_c),
+ SH_PFC_PIN_GROUP(msiof1_sync_c),
+ SH_PFC_PIN_GROUP(msiof1_ss1_c),
+ SH_PFC_PIN_GROUP(msiof1_ss2_c),
+ SH_PFC_PIN_GROUP(msiof1_txd_c),
+ SH_PFC_PIN_GROUP(msiof1_rxd_c),
+ SH_PFC_PIN_GROUP(msiof1_clk_d),
+ SH_PFC_PIN_GROUP(msiof1_sync_d),
+ SH_PFC_PIN_GROUP(msiof1_ss1_d),
+ SH_PFC_PIN_GROUP(msiof1_ss2_d),
+ SH_PFC_PIN_GROUP(msiof1_txd_d),
+ SH_PFC_PIN_GROUP(msiof1_rxd_d),
+ SH_PFC_PIN_GROUP(msiof1_clk_e),
+ SH_PFC_PIN_GROUP(msiof1_sync_e),
+ SH_PFC_PIN_GROUP(msiof1_ss1_e),
+ SH_PFC_PIN_GROUP(msiof1_ss2_e),
+ SH_PFC_PIN_GROUP(msiof1_txd_e),
+ SH_PFC_PIN_GROUP(msiof1_rxd_e),
+ SH_PFC_PIN_GROUP(msiof1_clk_f),
+ SH_PFC_PIN_GROUP(msiof1_sync_f),
+ SH_PFC_PIN_GROUP(msiof1_ss1_f),
+ SH_PFC_PIN_GROUP(msiof1_ss2_f),
+ SH_PFC_PIN_GROUP(msiof1_txd_f),
+ SH_PFC_PIN_GROUP(msiof1_rxd_f),
+ SH_PFC_PIN_GROUP(msiof1_clk_g),
+ SH_PFC_PIN_GROUP(msiof1_sync_g),
+ SH_PFC_PIN_GROUP(msiof1_ss1_g),
+ SH_PFC_PIN_GROUP(msiof1_ss2_g),
+ SH_PFC_PIN_GROUP(msiof1_txd_g),
+ SH_PFC_PIN_GROUP(msiof1_rxd_g),
+ SH_PFC_PIN_GROUP(msiof2_clk_a),
+ SH_PFC_PIN_GROUP(msiof2_sync_a),
+ SH_PFC_PIN_GROUP(msiof2_ss1_a),
+ SH_PFC_PIN_GROUP(msiof2_ss2_a),
+ SH_PFC_PIN_GROUP(msiof2_txd_a),
+ SH_PFC_PIN_GROUP(msiof2_rxd_a),
+ SH_PFC_PIN_GROUP(msiof2_clk_b),
+ SH_PFC_PIN_GROUP(msiof2_sync_b),
+ SH_PFC_PIN_GROUP(msiof2_ss1_b),
+ SH_PFC_PIN_GROUP(msiof2_ss2_b),
+ SH_PFC_PIN_GROUP(msiof2_txd_b),
+ SH_PFC_PIN_GROUP(msiof2_rxd_b),
+ SH_PFC_PIN_GROUP(msiof2_clk_c),
+ SH_PFC_PIN_GROUP(msiof2_sync_c),
+ SH_PFC_PIN_GROUP(msiof2_ss1_c),
+ SH_PFC_PIN_GROUP(msiof2_ss2_c),
+ SH_PFC_PIN_GROUP(msiof2_txd_c),
+ SH_PFC_PIN_GROUP(msiof2_rxd_c),
+ SH_PFC_PIN_GROUP(msiof2_clk_d),
+ SH_PFC_PIN_GROUP(msiof2_sync_d),
+ SH_PFC_PIN_GROUP(msiof2_ss1_d),
+ SH_PFC_PIN_GROUP(msiof2_ss2_d),
+ SH_PFC_PIN_GROUP(msiof2_txd_d),
+ SH_PFC_PIN_GROUP(msiof2_rxd_d),
+ SH_PFC_PIN_GROUP(msiof3_clk_a),
+ SH_PFC_PIN_GROUP(msiof3_sync_a),
+ SH_PFC_PIN_GROUP(msiof3_ss1_a),
+ SH_PFC_PIN_GROUP(msiof3_ss2_a),
+ SH_PFC_PIN_GROUP(msiof3_txd_a),
+ SH_PFC_PIN_GROUP(msiof3_rxd_a),
+ SH_PFC_PIN_GROUP(msiof3_clk_b),
+ SH_PFC_PIN_GROUP(msiof3_sync_b),
+ SH_PFC_PIN_GROUP(msiof3_ss1_b),
+ SH_PFC_PIN_GROUP(msiof3_ss2_b),
+ SH_PFC_PIN_GROUP(msiof3_txd_b),
+ SH_PFC_PIN_GROUP(msiof3_rxd_b),
+ SH_PFC_PIN_GROUP(msiof3_clk_c),
+ SH_PFC_PIN_GROUP(msiof3_sync_c),
+ SH_PFC_PIN_GROUP(msiof3_txd_c),
+ SH_PFC_PIN_GROUP(msiof3_rxd_c),
+ SH_PFC_PIN_GROUP(msiof3_clk_d),
+ SH_PFC_PIN_GROUP(msiof3_sync_d),
+ SH_PFC_PIN_GROUP(msiof3_ss1_d),
+ SH_PFC_PIN_GROUP(msiof3_txd_d),
+ SH_PFC_PIN_GROUP(msiof3_rxd_d),
+ SH_PFC_PIN_GROUP(msiof3_clk_e),
+ SH_PFC_PIN_GROUP(msiof3_sync_e),
+ SH_PFC_PIN_GROUP(msiof3_ss1_e),
+ SH_PFC_PIN_GROUP(msiof3_ss2_e),
+ SH_PFC_PIN_GROUP(msiof3_txd_e),
+ SH_PFC_PIN_GROUP(msiof3_rxd_e),
+ SH_PFC_PIN_GROUP(pwm0),
+ SH_PFC_PIN_GROUP(pwm1_a),
+ SH_PFC_PIN_GROUP(pwm1_b),
+ SH_PFC_PIN_GROUP(pwm2_a),
+ SH_PFC_PIN_GROUP(pwm2_b),
+ SH_PFC_PIN_GROUP(pwm3_a),
+ SH_PFC_PIN_GROUP(pwm3_b),
+ SH_PFC_PIN_GROUP(pwm4_a),
+ SH_PFC_PIN_GROUP(pwm4_b),
+ SH_PFC_PIN_GROUP(pwm5_a),
+ SH_PFC_PIN_GROUP(pwm5_b),
+ SH_PFC_PIN_GROUP(pwm6_a),
+ SH_PFC_PIN_GROUP(pwm6_b),
+ SH_PFC_PIN_GROUP(scif0_data),
+ SH_PFC_PIN_GROUP(scif0_clk),
+ SH_PFC_PIN_GROUP(scif0_ctrl),
+ SH_PFC_PIN_GROUP(scif1_data_a),
+ SH_PFC_PIN_GROUP(scif1_clk),
+ SH_PFC_PIN_GROUP(scif1_ctrl),
+ SH_PFC_PIN_GROUP(scif1_data_b),
+ SH_PFC_PIN_GROUP(scif2_data_a),
+ SH_PFC_PIN_GROUP(scif2_clk),
+ SH_PFC_PIN_GROUP(scif2_data_b),
+ SH_PFC_PIN_GROUP(scif3_data_a),
+ SH_PFC_PIN_GROUP(scif3_clk),
+ SH_PFC_PIN_GROUP(scif3_ctrl),
+ SH_PFC_PIN_GROUP(scif3_data_b),
+ SH_PFC_PIN_GROUP(scif4_data_a),
+ SH_PFC_PIN_GROUP(scif4_clk_a),
+ SH_PFC_PIN_GROUP(scif4_ctrl_a),
+ SH_PFC_PIN_GROUP(scif4_data_b),
+ SH_PFC_PIN_GROUP(scif4_clk_b),
+ SH_PFC_PIN_GROUP(scif4_ctrl_b),
+ SH_PFC_PIN_GROUP(scif4_data_c),
+ SH_PFC_PIN_GROUP(scif4_clk_c),
+ SH_PFC_PIN_GROUP(scif4_ctrl_c),
+ SH_PFC_PIN_GROUP(scif5_data_a),
+ SH_PFC_PIN_GROUP(scif5_clk_a),
+ SH_PFC_PIN_GROUP(scif5_data_b),
+ SH_PFC_PIN_GROUP(scif5_clk_b),
+ SH_PFC_PIN_GROUP(scif_clk_a),
+ SH_PFC_PIN_GROUP(scif_clk_b),
+ SH_PFC_PIN_GROUP(sdhi0_data1),
+ SH_PFC_PIN_GROUP(sdhi0_data4),
+ SH_PFC_PIN_GROUP(sdhi0_ctrl),
+ SH_PFC_PIN_GROUP(sdhi0_cd),
+ SH_PFC_PIN_GROUP(sdhi0_wp),
+ SH_PFC_PIN_GROUP(sdhi1_data1),
+ SH_PFC_PIN_GROUP(sdhi1_data4),
+ SH_PFC_PIN_GROUP(sdhi1_ctrl),
+ SH_PFC_PIN_GROUP(sdhi1_cd),
+ SH_PFC_PIN_GROUP(sdhi1_wp),
+ SH_PFC_PIN_GROUP(sdhi2_data1),
+ SH_PFC_PIN_GROUP(sdhi2_data4),
+ SH_PFC_PIN_GROUP(sdhi2_data8),
+ SH_PFC_PIN_GROUP(sdhi2_ctrl),
+ SH_PFC_PIN_GROUP(sdhi2_cd_a),
+ SH_PFC_PIN_GROUP(sdhi2_wp_a),
+ SH_PFC_PIN_GROUP(sdhi2_cd_b),
+ SH_PFC_PIN_GROUP(sdhi2_wp_b),
+ SH_PFC_PIN_GROUP(sdhi2_ds),
+ SH_PFC_PIN_GROUP(sdhi3_data1),
+ SH_PFC_PIN_GROUP(sdhi3_data4),
+ SH_PFC_PIN_GROUP(sdhi3_data8),
+ SH_PFC_PIN_GROUP(sdhi3_ctrl),
+ SH_PFC_PIN_GROUP(sdhi3_cd),
+ SH_PFC_PIN_GROUP(sdhi3_wp),
+ SH_PFC_PIN_GROUP(sdhi3_ds),
+ SH_PFC_PIN_GROUP(ssi0_data),
+ SH_PFC_PIN_GROUP(ssi01239_ctrl),
+ SH_PFC_PIN_GROUP(ssi1_data_a),
+ SH_PFC_PIN_GROUP(ssi1_data_b),
+ SH_PFC_PIN_GROUP(ssi1_ctrl_a),
+ SH_PFC_PIN_GROUP(ssi1_ctrl_b),
+ SH_PFC_PIN_GROUP(ssi2_data_a),
+ SH_PFC_PIN_GROUP(ssi2_data_b),
+ SH_PFC_PIN_GROUP(ssi2_ctrl_a),
+ SH_PFC_PIN_GROUP(ssi2_ctrl_b),
+ SH_PFC_PIN_GROUP(ssi3_data),
+ SH_PFC_PIN_GROUP(ssi349_ctrl),
+ SH_PFC_PIN_GROUP(ssi4_data),
+ SH_PFC_PIN_GROUP(ssi4_ctrl),
+ SH_PFC_PIN_GROUP(ssi5_data),
+ SH_PFC_PIN_GROUP(ssi5_ctrl),
+ SH_PFC_PIN_GROUP(ssi6_data),
+ SH_PFC_PIN_GROUP(ssi6_ctrl),
+ SH_PFC_PIN_GROUP(ssi7_data),
+ SH_PFC_PIN_GROUP(ssi78_ctrl),
+ SH_PFC_PIN_GROUP(ssi8_data),
+ SH_PFC_PIN_GROUP(ssi9_data_a),
+ SH_PFC_PIN_GROUP(ssi9_data_b),
+ SH_PFC_PIN_GROUP(ssi9_ctrl_a),
+ SH_PFC_PIN_GROUP(ssi9_ctrl_b),
+ SH_PFC_PIN_GROUP(tmu_tclk1_a),
+ SH_PFC_PIN_GROUP(tmu_tclk1_b),
+ SH_PFC_PIN_GROUP(tmu_tclk2_a),
+ SH_PFC_PIN_GROUP(tmu_tclk2_b),
+ SH_PFC_PIN_GROUP(usb0),
+ SH_PFC_PIN_GROUP(usb1),
+ SH_PFC_PIN_GROUP(usb30),
+ VIN_DATA_PIN_GROUP(vin4_data_a, 8),
+ VIN_DATA_PIN_GROUP(vin4_data_a, 10),
+ VIN_DATA_PIN_GROUP(vin4_data_a, 12),
+ VIN_DATA_PIN_GROUP(vin4_data_a, 16),
+ SH_PFC_PIN_GROUP(vin4_data18_a),
+ VIN_DATA_PIN_GROUP(vin4_data_a, 20),
+ VIN_DATA_PIN_GROUP(vin4_data_a, 24),
+ VIN_DATA_PIN_GROUP(vin4_data_b, 8),
+ VIN_DATA_PIN_GROUP(vin4_data_b, 10),
+ VIN_DATA_PIN_GROUP(vin4_data_b, 12),
+ VIN_DATA_PIN_GROUP(vin4_data_b, 16),
+ SH_PFC_PIN_GROUP(vin4_data18_b),
+ VIN_DATA_PIN_GROUP(vin4_data_b, 20),
+ VIN_DATA_PIN_GROUP(vin4_data_b, 24),
+ SH_PFC_PIN_GROUP(vin4_sync),
+ SH_PFC_PIN_GROUP(vin4_field),
+ SH_PFC_PIN_GROUP(vin4_clkenb),
+ SH_PFC_PIN_GROUP(vin4_clk),
+ SH_PFC_PIN_GROUP(vin5_data8),
+ SH_PFC_PIN_GROUP(vin5_data10),
+ SH_PFC_PIN_GROUP(vin5_data12),
+ SH_PFC_PIN_GROUP(vin5_data16),
+ SH_PFC_PIN_GROUP(vin5_sync),
+ SH_PFC_PIN_GROUP(vin5_field),
+ SH_PFC_PIN_GROUP(vin5_clkenb),
+ SH_PFC_PIN_GROUP(vin5_clk),
+ },
+ .automotive = {
+ SH_PFC_PIN_GROUP(canfd0_data_a),
+ SH_PFC_PIN_GROUP(canfd0_data_b),
+ SH_PFC_PIN_GROUP(canfd1_data),
+ SH_PFC_PIN_GROUP(drif0_ctrl_a),
+ SH_PFC_PIN_GROUP(drif0_data0_a),
+ SH_PFC_PIN_GROUP(drif0_data1_a),
+ SH_PFC_PIN_GROUP(drif0_ctrl_b),
+ SH_PFC_PIN_GROUP(drif0_data0_b),
+ SH_PFC_PIN_GROUP(drif0_data1_b),
+ SH_PFC_PIN_GROUP(drif0_ctrl_c),
+ SH_PFC_PIN_GROUP(drif0_data0_c),
+ SH_PFC_PIN_GROUP(drif0_data1_c),
+ SH_PFC_PIN_GROUP(drif1_ctrl_a),
+ SH_PFC_PIN_GROUP(drif1_data0_a),
+ SH_PFC_PIN_GROUP(drif1_data1_a),
+ SH_PFC_PIN_GROUP(drif1_ctrl_b),
+ SH_PFC_PIN_GROUP(drif1_data0_b),
+ SH_PFC_PIN_GROUP(drif1_data1_b),
+ SH_PFC_PIN_GROUP(drif1_ctrl_c),
+ SH_PFC_PIN_GROUP(drif1_data0_c),
+ SH_PFC_PIN_GROUP(drif1_data1_c),
+ SH_PFC_PIN_GROUP(drif2_ctrl_a),
+ SH_PFC_PIN_GROUP(drif2_data0_a),
+ SH_PFC_PIN_GROUP(drif2_data1_a),
+ SH_PFC_PIN_GROUP(drif2_ctrl_b),
+ SH_PFC_PIN_GROUP(drif2_data0_b),
+ SH_PFC_PIN_GROUP(drif2_data1_b),
+ SH_PFC_PIN_GROUP(drif3_ctrl_a),
+ SH_PFC_PIN_GROUP(drif3_data0_a),
+ SH_PFC_PIN_GROUP(drif3_data1_a),
+ SH_PFC_PIN_GROUP(drif3_ctrl_b),
+ SH_PFC_PIN_GROUP(drif3_data0_b),
+ SH_PFC_PIN_GROUP(drif3_data1_b),
+ }
};
static const char * const audio_clk_groups[] = {
"vin5_clk",
};
-static const struct sh_pfc_function pinmux_functions[] = {
- SH_PFC_FUNCTION(audio_clk),
- SH_PFC_FUNCTION(avb),
- SH_PFC_FUNCTION(can0),
- SH_PFC_FUNCTION(can1),
- SH_PFC_FUNCTION(can_clk),
- SH_PFC_FUNCTION(canfd0),
- SH_PFC_FUNCTION(canfd1),
- SH_PFC_FUNCTION(drif0),
- SH_PFC_FUNCTION(drif1),
- SH_PFC_FUNCTION(drif2),
- SH_PFC_FUNCTION(drif3),
- SH_PFC_FUNCTION(du),
- SH_PFC_FUNCTION(hdmi0),
- SH_PFC_FUNCTION(hscif0),
- SH_PFC_FUNCTION(hscif1),
- SH_PFC_FUNCTION(hscif2),
- SH_PFC_FUNCTION(hscif3),
- SH_PFC_FUNCTION(hscif4),
- SH_PFC_FUNCTION(i2c1),
- SH_PFC_FUNCTION(i2c2),
- SH_PFC_FUNCTION(i2c6),
- SH_PFC_FUNCTION(intc_ex),
- SH_PFC_FUNCTION(msiof0),
- SH_PFC_FUNCTION(msiof1),
- SH_PFC_FUNCTION(msiof2),
- SH_PFC_FUNCTION(msiof3),
- SH_PFC_FUNCTION(pwm0),
- SH_PFC_FUNCTION(pwm1),
- SH_PFC_FUNCTION(pwm2),
- SH_PFC_FUNCTION(pwm3),
- SH_PFC_FUNCTION(pwm4),
- SH_PFC_FUNCTION(pwm5),
- SH_PFC_FUNCTION(pwm6),
- SH_PFC_FUNCTION(scif0),
- SH_PFC_FUNCTION(scif1),
- SH_PFC_FUNCTION(scif2),
- SH_PFC_FUNCTION(scif3),
- SH_PFC_FUNCTION(scif4),
- SH_PFC_FUNCTION(scif5),
- SH_PFC_FUNCTION(scif_clk),
- SH_PFC_FUNCTION(sdhi0),
- SH_PFC_FUNCTION(sdhi1),
- SH_PFC_FUNCTION(sdhi2),
- SH_PFC_FUNCTION(sdhi3),
- SH_PFC_FUNCTION(ssi),
- SH_PFC_FUNCTION(tmu),
- SH_PFC_FUNCTION(usb0),
- SH_PFC_FUNCTION(usb1),
- SH_PFC_FUNCTION(usb30),
- SH_PFC_FUNCTION(vin4),
- SH_PFC_FUNCTION(vin5),
+static const struct {
+ struct sh_pfc_function common[45];
+ struct sh_pfc_function automotive[6];
+} pinmux_functions = {
+ .common = {
+ SH_PFC_FUNCTION(audio_clk),
+ SH_PFC_FUNCTION(avb),
+ SH_PFC_FUNCTION(can0),
+ SH_PFC_FUNCTION(can1),
+ SH_PFC_FUNCTION(can_clk),
+ SH_PFC_FUNCTION(du),
+ SH_PFC_FUNCTION(hdmi0),
+ SH_PFC_FUNCTION(hscif0),
+ SH_PFC_FUNCTION(hscif1),
+ SH_PFC_FUNCTION(hscif2),
+ SH_PFC_FUNCTION(hscif3),
+ SH_PFC_FUNCTION(hscif4),
+ SH_PFC_FUNCTION(i2c1),
+ SH_PFC_FUNCTION(i2c2),
+ SH_PFC_FUNCTION(i2c6),
+ SH_PFC_FUNCTION(intc_ex),
+ SH_PFC_FUNCTION(msiof0),
+ SH_PFC_FUNCTION(msiof1),
+ SH_PFC_FUNCTION(msiof2),
+ SH_PFC_FUNCTION(msiof3),
+ SH_PFC_FUNCTION(pwm0),
+ SH_PFC_FUNCTION(pwm1),
+ SH_PFC_FUNCTION(pwm2),
+ SH_PFC_FUNCTION(pwm3),
+ SH_PFC_FUNCTION(pwm4),
+ SH_PFC_FUNCTION(pwm5),
+ SH_PFC_FUNCTION(pwm6),
+ SH_PFC_FUNCTION(scif0),
+ SH_PFC_FUNCTION(scif1),
+ SH_PFC_FUNCTION(scif2),
+ SH_PFC_FUNCTION(scif3),
+ SH_PFC_FUNCTION(scif4),
+ SH_PFC_FUNCTION(scif5),
+ SH_PFC_FUNCTION(scif_clk),
+ SH_PFC_FUNCTION(sdhi0),
+ SH_PFC_FUNCTION(sdhi1),
+ SH_PFC_FUNCTION(sdhi2),
+ SH_PFC_FUNCTION(sdhi3),
+ SH_PFC_FUNCTION(ssi),
+ SH_PFC_FUNCTION(tmu),
+ SH_PFC_FUNCTION(usb0),
+ SH_PFC_FUNCTION(usb1),
+ SH_PFC_FUNCTION(usb30),
+ SH_PFC_FUNCTION(vin4),
+ SH_PFC_FUNCTION(vin5),
+ },
+ .automotive = {
+ SH_PFC_FUNCTION(canfd0),
+ SH_PFC_FUNCTION(canfd1),
+ SH_PFC_FUNCTION(drif0),
+ SH_PFC_FUNCTION(drif1),
+ SH_PFC_FUNCTION(drif2),
+ SH_PFC_FUNCTION(drif3),
+ }
};
static const struct pinmux_cfg_reg pinmux_config_regs[] = {
.set_bias = r8a7796_pinmux_set_bias,
};
+#ifdef CONFIG_PINCTRL_PFC_R8A774A1
+const struct sh_pfc_soc_info r8a774a1_pinmux_info = {
+ .name = "r8a774a1_pfc",
+ .ops = &r8a7796_pinmux_ops,
+ .unlock_reg = 0xe6060000, /* PMMR */
+
+ .function = { PINMUX_FUNCTION_BEGIN, PINMUX_FUNCTION_END },
+
+ .pins = pinmux_pins,
+ .nr_pins = ARRAY_SIZE(pinmux_pins),
+ .groups = pinmux_groups.common,
+ .nr_groups = ARRAY_SIZE(pinmux_groups.common),
+ .functions = pinmux_functions.common,
+ .nr_functions = ARRAY_SIZE(pinmux_functions.common),
+
+ .cfg_regs = pinmux_config_regs,
+ .drive_regs = pinmux_drive_regs,
+ .bias_regs = pinmux_bias_regs,
+ .ioctrl_regs = pinmux_ioctrl_regs,
+
+ .pinmux_data = pinmux_data,
+ .pinmux_data_size = ARRAY_SIZE(pinmux_data),
+};
+#endif
+
+#ifdef CONFIG_PINCTRL_PFC_R8A7796
const struct sh_pfc_soc_info r8a7796_pinmux_info = {
.name = "r8a77960_pfc",
.ops = &r8a7796_pinmux_ops,
.pins = pinmux_pins,
.nr_pins = ARRAY_SIZE(pinmux_pins),
- .groups = pinmux_groups,
- .nr_groups = ARRAY_SIZE(pinmux_groups),
- .functions = pinmux_functions,
- .nr_functions = ARRAY_SIZE(pinmux_functions),
+ .groups = pinmux_groups.common,
+ .nr_groups = ARRAY_SIZE(pinmux_groups.common) +
+ ARRAY_SIZE(pinmux_groups.automotive),
+ .functions = pinmux_functions.common,
+ .nr_functions = ARRAY_SIZE(pinmux_functions.common) +
+ ARRAY_SIZE(pinmux_functions.automotive),
.cfg_regs = pinmux_config_regs,
.drive_regs = pinmux_drive_regs,
.pinmux_data = pinmux_data,
.pinmux_data_size = ARRAY_SIZE(pinmux_data),
};
+#endif
SH_PFC_PIN_NAMED_CFG(ROW_GROUP_A('T'), 30, ASEBRK, CFG_FLAGS),
};
+/* - AUDIO CLOCK ------------------------------------------------------------ */
+static const unsigned int audio_clk_a_a_pins[] = {
+ /* CLK A */
+ RCAR_GP_PIN(6, 22),
+};
+static const unsigned int audio_clk_a_a_mux[] = {
+ AUDIO_CLKA_A_MARK,
+};
+static const unsigned int audio_clk_a_b_pins[] = {
+ /* CLK A */
+ RCAR_GP_PIN(5, 4),
+};
+static const unsigned int audio_clk_a_b_mux[] = {
+ AUDIO_CLKA_B_MARK,
+};
+static const unsigned int audio_clk_a_c_pins[] = {
+ /* CLK A */
+ RCAR_GP_PIN(5, 19),
+};
+static const unsigned int audio_clk_a_c_mux[] = {
+ AUDIO_CLKA_C_MARK,
+};
+static const unsigned int audio_clk_b_a_pins[] = {
+ /* CLK B */
+ RCAR_GP_PIN(5, 12),
+};
+static const unsigned int audio_clk_b_a_mux[] = {
+ AUDIO_CLKB_A_MARK,
+};
+static const unsigned int audio_clk_b_b_pins[] = {
+ /* CLK B */
+ RCAR_GP_PIN(6, 23),
+};
+static const unsigned int audio_clk_b_b_mux[] = {
+ AUDIO_CLKB_B_MARK,
+};
+static const unsigned int audio_clk_c_a_pins[] = {
+ /* CLK C */
+ RCAR_GP_PIN(5, 21),
+};
+static const unsigned int audio_clk_c_a_mux[] = {
+ AUDIO_CLKC_A_MARK,
+};
+static const unsigned int audio_clk_c_b_pins[] = {
+ /* CLK C */
+ RCAR_GP_PIN(5, 0),
+};
+static const unsigned int audio_clk_c_b_mux[] = {
+ AUDIO_CLKC_B_MARK,
+};
+static const unsigned int audio_clkout_a_pins[] = {
+ /* CLKOUT */
+ RCAR_GP_PIN(5, 18),
+};
+static const unsigned int audio_clkout_a_mux[] = {
+ AUDIO_CLKOUT_A_MARK,
+};
+static const unsigned int audio_clkout_b_pins[] = {
+ /* CLKOUT */
+ RCAR_GP_PIN(6, 28),
+};
+static const unsigned int audio_clkout_b_mux[] = {
+ AUDIO_CLKOUT_B_MARK,
+};
+static const unsigned int audio_clkout_c_pins[] = {
+ /* CLKOUT */
+ RCAR_GP_PIN(5, 3),
+};
+static const unsigned int audio_clkout_c_mux[] = {
+ AUDIO_CLKOUT_C_MARK,
+};
+static const unsigned int audio_clkout_d_pins[] = {
+ /* CLKOUT */
+ RCAR_GP_PIN(5, 21),
+};
+static const unsigned int audio_clkout_d_mux[] = {
+ AUDIO_CLKOUT_D_MARK,
+};
+static const unsigned int audio_clkout1_a_pins[] = {
+ /* CLKOUT1 */
+ RCAR_GP_PIN(5, 15),
+};
+static const unsigned int audio_clkout1_a_mux[] = {
+ AUDIO_CLKOUT1_A_MARK,
+};
+static const unsigned int audio_clkout1_b_pins[] = {
+ /* CLKOUT1 */
+ RCAR_GP_PIN(6, 29),
+};
+static const unsigned int audio_clkout1_b_mux[] = {
+ AUDIO_CLKOUT1_B_MARK,
+};
+static const unsigned int audio_clkout2_a_pins[] = {
+ /* CLKOUT2 */
+ RCAR_GP_PIN(5, 16),
+};
+static const unsigned int audio_clkout2_a_mux[] = {
+ AUDIO_CLKOUT2_A_MARK,
+};
+static const unsigned int audio_clkout2_b_pins[] = {
+ /* CLKOUT2 */
+ RCAR_GP_PIN(6, 30),
+};
+static const unsigned int audio_clkout2_b_mux[] = {
+ AUDIO_CLKOUT2_B_MARK,
+};
+
+static const unsigned int audio_clkout3_a_pins[] = {
+ /* CLKOUT3 */
+ RCAR_GP_PIN(5, 19),
+};
+static const unsigned int audio_clkout3_a_mux[] = {
+ AUDIO_CLKOUT3_A_MARK,
+};
+static const unsigned int audio_clkout3_b_pins[] = {
+ /* CLKOUT3 */
+ RCAR_GP_PIN(6, 31),
+};
+static const unsigned int audio_clkout3_b_mux[] = {
+ AUDIO_CLKOUT3_B_MARK,
+};
+
/* - EtherAVB --------------------------------------------------------------- */
static const unsigned int avb_link_pins[] = {
/* AVB_LINK */
PWM6_B_MARK,
};
+/* - SATA --------------------------------------------------------------------*/
+static const unsigned int sata0_devslp_a_pins[] = {
+ /* DEVSLP */
+ RCAR_GP_PIN(6, 16),
+};
+
+static const unsigned int sata0_devslp_a_mux[] = {
+ SATA_DEVSLP_A_MARK,
+};
+
+static const unsigned int sata0_devslp_b_pins[] = {
+ /* DEVSLP */
+ RCAR_GP_PIN(4, 6),
+};
+
+static const unsigned int sata0_devslp_b_mux[] = {
+ SATA_DEVSLP_B_MARK,
+};
+
/* - SCIF0 ------------------------------------------------------------------ */
static const unsigned int scif0_data_pins[] = {
/* RX, TX */
SD3_DS_MARK,
};
+/* - SSI -------------------------------------------------------------------- */
+static const unsigned int ssi0_data_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(6, 2),
+};
+static const unsigned int ssi0_data_mux[] = {
+ SSI_SDATA0_MARK,
+};
+static const unsigned int ssi01239_ctrl_pins[] = {
+ /* SCK, WS */
+ RCAR_GP_PIN(6, 0), RCAR_GP_PIN(6, 1),
+};
+static const unsigned int ssi01239_ctrl_mux[] = {
+ SSI_SCK01239_MARK, SSI_WS01239_MARK,
+};
+static const unsigned int ssi1_data_a_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(6, 3),
+};
+static const unsigned int ssi1_data_a_mux[] = {
+ SSI_SDATA1_A_MARK,
+};
+static const unsigned int ssi1_data_b_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(5, 12),
+};
+static const unsigned int ssi1_data_b_mux[] = {
+ SSI_SDATA1_B_MARK,
+};
+static const unsigned int ssi1_ctrl_a_pins[] = {
+ /* SCK, WS */
+ RCAR_GP_PIN(6, 26), RCAR_GP_PIN(6, 27),
+};
+static const unsigned int ssi1_ctrl_a_mux[] = {
+ SSI_SCK1_A_MARK, SSI_WS1_A_MARK,
+};
+static const unsigned int ssi1_ctrl_b_pins[] = {
+ /* SCK, WS */
+ RCAR_GP_PIN(6, 4), RCAR_GP_PIN(6, 21),
+};
+static const unsigned int ssi1_ctrl_b_mux[] = {
+ SSI_SCK1_B_MARK, SSI_WS1_B_MARK,
+};
+static const unsigned int ssi2_data_a_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(6, 4),
+};
+static const unsigned int ssi2_data_a_mux[] = {
+ SSI_SDATA2_A_MARK,
+};
+static const unsigned int ssi2_data_b_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(5, 13),
+};
+static const unsigned int ssi2_data_b_mux[] = {
+ SSI_SDATA2_B_MARK,
+};
+static const unsigned int ssi2_ctrl_a_pins[] = {
+ /* SCK, WS */
+ RCAR_GP_PIN(5, 19), RCAR_GP_PIN(5, 21),
+};
+static const unsigned int ssi2_ctrl_a_mux[] = {
+ SSI_SCK2_A_MARK, SSI_WS2_A_MARK,
+};
+static const unsigned int ssi2_ctrl_b_pins[] = {
+ /* SCK, WS */
+ RCAR_GP_PIN(6, 28), RCAR_GP_PIN(6, 29),
+};
+static const unsigned int ssi2_ctrl_b_mux[] = {
+ SSI_SCK2_B_MARK, SSI_WS2_B_MARK,
+};
+static const unsigned int ssi3_data_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(6, 7),
+};
+static const unsigned int ssi3_data_mux[] = {
+ SSI_SDATA3_MARK,
+};
+static const unsigned int ssi349_ctrl_pins[] = {
+ /* SCK, WS */
+ RCAR_GP_PIN(6, 5), RCAR_GP_PIN(6, 6),
+};
+static const unsigned int ssi349_ctrl_mux[] = {
+ SSI_SCK349_MARK, SSI_WS349_MARK,
+};
+static const unsigned int ssi4_data_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(6, 10),
+};
+static const unsigned int ssi4_data_mux[] = {
+ SSI_SDATA4_MARK,
+};
+static const unsigned int ssi4_ctrl_pins[] = {
+ /* SCK, WS */
+ RCAR_GP_PIN(6, 8), RCAR_GP_PIN(6, 9),
+};
+static const unsigned int ssi4_ctrl_mux[] = {
+ SSI_SCK4_MARK, SSI_WS4_MARK,
+};
+static const unsigned int ssi5_data_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(6, 13),
+};
+static const unsigned int ssi5_data_mux[] = {
+ SSI_SDATA5_MARK,
+};
+static const unsigned int ssi5_ctrl_pins[] = {
+ /* SCK, WS */
+ RCAR_GP_PIN(6, 11), RCAR_GP_PIN(6, 12),
+};
+static const unsigned int ssi5_ctrl_mux[] = {
+ SSI_SCK5_MARK, SSI_WS5_MARK,
+};
+static const unsigned int ssi6_data_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(6, 16),
+};
+static const unsigned int ssi6_data_mux[] = {
+ SSI_SDATA6_MARK,
+};
+static const unsigned int ssi6_ctrl_pins[] = {
+ /* SCK, WS */
+ RCAR_GP_PIN(6, 14), RCAR_GP_PIN(6, 15),
+};
+static const unsigned int ssi6_ctrl_mux[] = {
+ SSI_SCK6_MARK, SSI_WS6_MARK,
+};
+static const unsigned int ssi7_data_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(6, 19),
+};
+static const unsigned int ssi7_data_mux[] = {
+ SSI_SDATA7_MARK,
+};
+static const unsigned int ssi78_ctrl_pins[] = {
+ /* SCK, WS */
+ RCAR_GP_PIN(6, 17), RCAR_GP_PIN(6, 18),
+};
+static const unsigned int ssi78_ctrl_mux[] = {
+ SSI_SCK78_MARK, SSI_WS78_MARK,
+};
+static const unsigned int ssi8_data_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(6, 20),
+};
+static const unsigned int ssi8_data_mux[] = {
+ SSI_SDATA8_MARK,
+};
+static const unsigned int ssi9_data_a_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(6, 21),
+};
+static const unsigned int ssi9_data_a_mux[] = {
+ SSI_SDATA9_A_MARK,
+};
+static const unsigned int ssi9_data_b_pins[] = {
+ /* SDATA */
+ RCAR_GP_PIN(5, 14),
+};
+static const unsigned int ssi9_data_b_mux[] = {
+ SSI_SDATA9_B_MARK,
+};
+static const unsigned int ssi9_ctrl_a_pins[] = {
+ /* SCK, WS */
+ RCAR_GP_PIN(5, 15), RCAR_GP_PIN(5, 16),
+};
+static const unsigned int ssi9_ctrl_a_mux[] = {
+ SSI_SCK9_A_MARK, SSI_WS9_A_MARK,
+};
+static const unsigned int ssi9_ctrl_b_pins[] = {
+ /* SCK, WS */
+ RCAR_GP_PIN(6, 30), RCAR_GP_PIN(6, 31),
+};
+static const unsigned int ssi9_ctrl_b_mux[] = {
+ SSI_SCK9_B_MARK, SSI_WS9_B_MARK,
+};
+
+
/* - USB0 ------------------------------------------------------------------- */
static const unsigned int usb0_pins[] = {
/* PWEN, OVC */
};
static const struct sh_pfc_pin_group pinmux_groups[] = {
+ SH_PFC_PIN_GROUP(audio_clk_a_a),
+ SH_PFC_PIN_GROUP(audio_clk_a_b),
+ SH_PFC_PIN_GROUP(audio_clk_a_c),
+ SH_PFC_PIN_GROUP(audio_clk_b_a),
+ SH_PFC_PIN_GROUP(audio_clk_b_b),
+ SH_PFC_PIN_GROUP(audio_clk_c_a),
+ SH_PFC_PIN_GROUP(audio_clk_c_b),
+ SH_PFC_PIN_GROUP(audio_clkout_a),
+ SH_PFC_PIN_GROUP(audio_clkout_b),
+ SH_PFC_PIN_GROUP(audio_clkout_c),
+ SH_PFC_PIN_GROUP(audio_clkout_d),
+ SH_PFC_PIN_GROUP(audio_clkout1_a),
+ SH_PFC_PIN_GROUP(audio_clkout1_b),
+ SH_PFC_PIN_GROUP(audio_clkout2_a),
+ SH_PFC_PIN_GROUP(audio_clkout2_b),
+ SH_PFC_PIN_GROUP(audio_clkout3_a),
+ SH_PFC_PIN_GROUP(audio_clkout3_b),
SH_PFC_PIN_GROUP(avb_link),
SH_PFC_PIN_GROUP(avb_magic),
SH_PFC_PIN_GROUP(avb_phy_int),
SH_PFC_PIN_GROUP(pwm5_b),
SH_PFC_PIN_GROUP(pwm6_a),
SH_PFC_PIN_GROUP(pwm6_b),
+ SH_PFC_PIN_GROUP(sata0_devslp_a),
+ SH_PFC_PIN_GROUP(sata0_devslp_b),
SH_PFC_PIN_GROUP(scif0_data),
SH_PFC_PIN_GROUP(scif0_clk),
SH_PFC_PIN_GROUP(scif0_ctrl),
SH_PFC_PIN_GROUP(sdhi3_cd),
SH_PFC_PIN_GROUP(sdhi3_wp),
SH_PFC_PIN_GROUP(sdhi3_ds),
+ SH_PFC_PIN_GROUP(ssi0_data),
+ SH_PFC_PIN_GROUP(ssi01239_ctrl),
+ SH_PFC_PIN_GROUP(ssi1_data_a),
+ SH_PFC_PIN_GROUP(ssi1_data_b),
+ SH_PFC_PIN_GROUP(ssi1_ctrl_a),
+ SH_PFC_PIN_GROUP(ssi1_ctrl_b),
+ SH_PFC_PIN_GROUP(ssi2_data_a),
+ SH_PFC_PIN_GROUP(ssi2_data_b),
+ SH_PFC_PIN_GROUP(ssi2_ctrl_a),
+ SH_PFC_PIN_GROUP(ssi2_ctrl_b),
+ SH_PFC_PIN_GROUP(ssi3_data),
+ SH_PFC_PIN_GROUP(ssi349_ctrl),
+ SH_PFC_PIN_GROUP(ssi4_data),
+ SH_PFC_PIN_GROUP(ssi4_ctrl),
+ SH_PFC_PIN_GROUP(ssi5_data),
+ SH_PFC_PIN_GROUP(ssi5_ctrl),
+ SH_PFC_PIN_GROUP(ssi6_data),
+ SH_PFC_PIN_GROUP(ssi6_ctrl),
+ SH_PFC_PIN_GROUP(ssi7_data),
+ SH_PFC_PIN_GROUP(ssi78_ctrl),
+ SH_PFC_PIN_GROUP(ssi8_data),
+ SH_PFC_PIN_GROUP(ssi9_data_a),
+ SH_PFC_PIN_GROUP(ssi9_data_b),
+ SH_PFC_PIN_GROUP(ssi9_ctrl_a),
+ SH_PFC_PIN_GROUP(ssi9_ctrl_b),
SH_PFC_PIN_GROUP(usb0),
SH_PFC_PIN_GROUP(usb1),
SH_PFC_PIN_GROUP(usb30),
};
+static const char * const audio_clk_groups[] = {
+ "audio_clk_a_a",
+ "audio_clk_a_b",
+ "audio_clk_a_c",
+ "audio_clk_b_a",
+ "audio_clk_b_b",
+ "audio_clk_c_a",
+ "audio_clk_c_b",
+ "audio_clkout_a",
+ "audio_clkout_b",
+ "audio_clkout_c",
+ "audio_clkout_d",
+ "audio_clkout1_a",
+ "audio_clkout1_b",
+ "audio_clkout2_a",
+ "audio_clkout2_b",
+ "audio_clkout3_a",
+ "audio_clkout3_b",
+};
+
static const char * const avb_groups[] = {
"avb_link",
"avb_magic",
"pwm6_b",
};
+static const char * const sata0_groups[] = {
+ "sata0_devslp_a",
+ "sata0_devslp_b",
+};
+
static const char * const scif0_groups[] = {
"scif0_data",
"scif0_clk",
"sdhi3_ds",
};
+static const char * const ssi_groups[] = {
+ "ssi0_data",
+ "ssi01239_ctrl",
+ "ssi1_data_a",
+ "ssi1_data_b",
+ "ssi1_ctrl_a",
+ "ssi1_ctrl_b",
+ "ssi2_data_a",
+ "ssi2_data_b",
+ "ssi2_ctrl_a",
+ "ssi2_ctrl_b",
+ "ssi3_data",
+ "ssi349_ctrl",
+ "ssi4_data",
+ "ssi4_ctrl",
+ "ssi5_data",
+ "ssi5_ctrl",
+ "ssi6_data",
+ "ssi6_ctrl",
+ "ssi7_data",
+ "ssi78_ctrl",
+ "ssi8_data",
+ "ssi9_data_a",
+ "ssi9_data_b",
+ "ssi9_ctrl_a",
+ "ssi9_ctrl_b",
+};
+
static const char * const usb0_groups[] = {
"usb0",
};
};
static const struct sh_pfc_function pinmux_functions[] = {
+ SH_PFC_FUNCTION(audio_clk),
SH_PFC_FUNCTION(avb),
SH_PFC_FUNCTION(du),
SH_PFC_FUNCTION(hscif0),
SH_PFC_FUNCTION(pwm4),
SH_PFC_FUNCTION(pwm5),
SH_PFC_FUNCTION(pwm6),
+ SH_PFC_FUNCTION(sata0),
SH_PFC_FUNCTION(scif0),
SH_PFC_FUNCTION(scif1),
SH_PFC_FUNCTION(scif2),
SH_PFC_FUNCTION(sdhi1),
SH_PFC_FUNCTION(sdhi2),
SH_PFC_FUNCTION(sdhi3),
+ SH_PFC_FUNCTION(ssi),
SH_PFC_FUNCTION(usb0),
SH_PFC_FUNCTION(usb1),
SH_PFC_FUNCTION(usb30),
+// SPDX-License-Identifier: GPL-2.0
/*
* R8A77970 processor support - PFC hardware block.
*
* R-Car Gen3 processor support - PFC hardware block.
*
* Copyright (C) 2015 Renesas Electronics Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
*/
#include <linux/io.h>
AVB_AVTP_CAPTURE_A_MARK,
};
+/* - DU --------------------------------------------------------------------- */
+static const unsigned int du_rgb666_pins[] = {
+ /* R[7:2], G[7:2], B[7:2] */
+ RCAR_GP_PIN(0, 8), RCAR_GP_PIN(0, 6), RCAR_GP_PIN(0, 5),
+ RCAR_GP_PIN(0, 3), RCAR_GP_PIN(0, 2), RCAR_GP_PIN(0, 0),
+ RCAR_GP_PIN(1, 9), RCAR_GP_PIN(1, 12), RCAR_GP_PIN(1, 10),
+ RCAR_GP_PIN(1, 4), RCAR_GP_PIN(0, 15), RCAR_GP_PIN(0, 11),
+ RCAR_GP_PIN(0, 1), RCAR_GP_PIN(1, 17), RCAR_GP_PIN(1, 16),
+ RCAR_GP_PIN(1, 15), RCAR_GP_PIN(1, 14), RCAR_GP_PIN(1, 13),
+};
+static const unsigned int du_rgb666_mux[] = {
+ DU_DR7_MARK, DU_DR6_MARK, DU_DR5_MARK, DU_DR4_MARK,
+ DU_DR3_MARK, DU_DR2_MARK,
+ DU_DG7_MARK, DU_DG6_MARK, DU_DG5_MARK, DU_DG4_MARK,
+ DU_DG3_MARK, DU_DG2_MARK,
+ DU_DB7_MARK, DU_DB6_MARK, DU_DB5_MARK, DU_DB4_MARK,
+ DU_DB3_MARK, DU_DB2_MARK,
+};
+static const unsigned int du_rgb888_pins[] = {
+ /* R[7:0], G[7:0], B[7:0] */
+ RCAR_GP_PIN(0, 8), RCAR_GP_PIN(0, 6), RCAR_GP_PIN(0, 5),
+ RCAR_GP_PIN(0, 3), RCAR_GP_PIN(0, 2), RCAR_GP_PIN(0, 0),
+ RCAR_GP_PIN(1, 22), RCAR_GP_PIN(1, 21),
+ RCAR_GP_PIN(1, 9), RCAR_GP_PIN(1, 12), RCAR_GP_PIN(1, 10),
+ RCAR_GP_PIN(1, 4), RCAR_GP_PIN(0, 15), RCAR_GP_PIN(0, 11),
+ RCAR_GP_PIN(0, 10), RCAR_GP_PIN(0, 9),
+ RCAR_GP_PIN(0, 1), RCAR_GP_PIN(1, 17), RCAR_GP_PIN(1, 16),
+ RCAR_GP_PIN(1, 15), RCAR_GP_PIN(1, 14), RCAR_GP_PIN(1, 13),
+ RCAR_GP_PIN(1, 19), RCAR_GP_PIN(1, 18),
+};
+static const unsigned int du_rgb888_mux[] = {
+ DU_DR7_MARK, DU_DR6_MARK, DU_DR5_MARK, DU_DR4_MARK,
+ DU_DR3_MARK, DU_DR2_MARK, DU_DR1_MARK, DU_DR0_MARK,
+ DU_DG7_MARK, DU_DG6_MARK, DU_DG5_MARK, DU_DG4_MARK,
+ DU_DG3_MARK, DU_DG2_MARK, DU_DG1_MARK, DU_DG0_MARK,
+ DU_DB7_MARK, DU_DB6_MARK, DU_DB5_MARK, DU_DB4_MARK,
+ DU_DB3_MARK, DU_DB2_MARK, DU_DB1_MARK, DU_DB0_MARK,
+};
+static const unsigned int du_clk_in_0_pins[] = {
+ /* CLKIN0 */
+ RCAR_GP_PIN(0, 16),
+};
+static const unsigned int du_clk_in_0_mux[] = {
+ DU_DOTCLKIN0_MARK
+};
+static const unsigned int du_clk_in_1_pins[] = {
+ /* CLKIN1 */
+ RCAR_GP_PIN(1, 1),
+};
+static const unsigned int du_clk_in_1_mux[] = {
+ DU_DOTCLKIN1_MARK
+};
+static const unsigned int du_clk_out_0_pins[] = {
+ /* CLKOUT */
+ RCAR_GP_PIN(1, 3),
+};
+static const unsigned int du_clk_out_0_mux[] = {
+ DU_DOTCLKOUT0_MARK
+};
+static const unsigned int du_sync_pins[] = {
+ /* VSYNC, HSYNC */
+ RCAR_GP_PIN(1, 11), RCAR_GP_PIN(1, 8),
+};
+static const unsigned int du_sync_mux[] = {
+ DU_VSYNC_MARK, DU_HSYNC_MARK
+};
+static const unsigned int du_disp_cde_pins[] = {
+ /* DISP_CDE */
+ RCAR_GP_PIN(1, 1),
+};
+static const unsigned int du_disp_cde_mux[] = {
+ DU_DISP_CDE_MARK,
+};
+static const unsigned int du_cde_pins[] = {
+ /* CDE */
+ RCAR_GP_PIN(1, 0),
+};
+static const unsigned int du_cde_mux[] = {
+ DU_CDE_MARK,
+};
+static const unsigned int du_disp_pins[] = {
+ /* DISP */
+ RCAR_GP_PIN(1, 2),
+};
+static const unsigned int du_disp_mux[] = {
+ DU_DISP_MARK,
+};
+
/* - I2C -------------------------------------------------------------------- */
static const unsigned int i2c1_a_pins[] = {
/* SCL, SDA */
SCL7_B_MARK, SDA7_B_MARK,
};
+/* - INTC-EX ---------------------------------------------------------------- */
+static const unsigned int intc_ex_irq0_pins[] = {
+ /* IRQ0 */
+ RCAR_GP_PIN(1, 0),
+};
+static const unsigned int intc_ex_irq0_mux[] = {
+ IRQ0_MARK,
+};
+static const unsigned int intc_ex_irq1_pins[] = {
+ /* IRQ1 */
+ RCAR_GP_PIN(1, 1),
+};
+static const unsigned int intc_ex_irq1_mux[] = {
+ IRQ1_MARK,
+};
+static const unsigned int intc_ex_irq2_pins[] = {
+ /* IRQ2 */
+ RCAR_GP_PIN(1, 2),
+};
+static const unsigned int intc_ex_irq2_mux[] = {
+ IRQ2_MARK,
+};
+static const unsigned int intc_ex_irq3_pins[] = {
+ /* IRQ3 */
+ RCAR_GP_PIN(1, 9),
+};
+static const unsigned int intc_ex_irq3_mux[] = {
+ IRQ3_MARK,
+};
+static const unsigned int intc_ex_irq4_pins[] = {
+ /* IRQ4 */
+ RCAR_GP_PIN(1, 10),
+};
+static const unsigned int intc_ex_irq4_mux[] = {
+ IRQ4_MARK,
+};
+static const unsigned int intc_ex_irq5_pins[] = {
+ /* IRQ5 */
+ RCAR_GP_PIN(0, 7),
+};
+static const unsigned int intc_ex_irq5_mux[] = {
+ IRQ5_MARK,
+};
+
+/* - MSIOF0 ----------------------------------------------------------------- */
+static const unsigned int msiof0_clk_pins[] = {
+ /* SCK */
+ RCAR_GP_PIN(5, 10),
+};
+
+static const unsigned int msiof0_clk_mux[] = {
+ MSIOF0_SCK_MARK,
+};
+
+static const unsigned int msiof0_sync_pins[] = {
+ /* SYNC */
+ RCAR_GP_PIN(5, 13),
+};
+
+static const unsigned int msiof0_sync_mux[] = {
+ MSIOF0_SYNC_MARK,
+};
+
+static const unsigned int msiof0_ss1_pins[] = {
+ /* SS1 */
+ RCAR_GP_PIN(5, 14),
+};
+
+static const unsigned int msiof0_ss1_mux[] = {
+ MSIOF0_SS1_MARK,
+};
+
+static const unsigned int msiof0_ss2_pins[] = {
+ /* SS2 */
+ RCAR_GP_PIN(5, 15),
+};
+
+static const unsigned int msiof0_ss2_mux[] = {
+ MSIOF0_SS2_MARK,
+};
+
+static const unsigned int msiof0_txd_pins[] = {
+ /* TXD */
+ RCAR_GP_PIN(5, 12),
+};
+
+static const unsigned int msiof0_txd_mux[] = {
+ MSIOF0_TXD_MARK,
+};
+
+static const unsigned int msiof0_rxd_pins[] = {
+ /* RXD */
+ RCAR_GP_PIN(5, 11),
+};
+
+static const unsigned int msiof0_rxd_mux[] = {
+ MSIOF0_RXD_MARK,
+};
+
+/* - MSIOF1 ----------------------------------------------------------------- */
+static const unsigned int msiof1_clk_pins[] = {
+ /* SCK */
+ RCAR_GP_PIN(1, 19),
+};
+
+static const unsigned int msiof1_clk_mux[] = {
+ MSIOF1_SCK_MARK,
+};
+
+static const unsigned int msiof1_sync_pins[] = {
+ /* SYNC */
+ RCAR_GP_PIN(1, 16),
+};
+
+static const unsigned int msiof1_sync_mux[] = {
+ MSIOF1_SYNC_MARK,
+};
+
+static const unsigned int msiof1_ss1_pins[] = {
+ /* SS1 */
+ RCAR_GP_PIN(1, 14),
+};
+
+static const unsigned int msiof1_ss1_mux[] = {
+ MSIOF1_SS1_MARK,
+};
+
+static const unsigned int msiof1_ss2_pins[] = {
+ /* SS2 */
+ RCAR_GP_PIN(1, 15),
+};
+
+static const unsigned int msiof1_ss2_mux[] = {
+ MSIOF1_SS2_MARK,
+};
+
+static const unsigned int msiof1_txd_pins[] = {
+ /* TXD */
+ RCAR_GP_PIN(1, 18),
+};
+
+static const unsigned int msiof1_txd_mux[] = {
+ MSIOF1_TXD_MARK,
+};
+
+static const unsigned int msiof1_rxd_pins[] = {
+ /* RXD */
+ RCAR_GP_PIN(1, 17),
+};
+
+static const unsigned int msiof1_rxd_mux[] = {
+ MSIOF1_RXD_MARK,
+};
+
+/* - MSIOF2 ----------------------------------------------------------------- */
+static const unsigned int msiof2_clk_a_pins[] = {
+ /* SCK */
+ RCAR_GP_PIN(0, 8),
+};
+
+static const unsigned int msiof2_clk_a_mux[] = {
+ MSIOF2_SCK_A_MARK,
+};
+
+static const unsigned int msiof2_sync_a_pins[] = {
+ /* SYNC */
+ RCAR_GP_PIN(0, 9),
+};
+
+static const unsigned int msiof2_sync_a_mux[] = {
+ MSIOF2_SYNC_A_MARK,
+};
+
+static const unsigned int msiof2_ss1_a_pins[] = {
+ /* SS1 */
+ RCAR_GP_PIN(0, 15),
+};
+
+static const unsigned int msiof2_ss1_a_mux[] = {
+ MSIOF2_SS1_A_MARK,
+};
+
+static const unsigned int msiof2_ss2_a_pins[] = {
+ /* SS2 */
+ RCAR_GP_PIN(0, 14),
+};
+
+static const unsigned int msiof2_ss2_a_mux[] = {
+ MSIOF2_SS2_A_MARK,
+};
+
+static const unsigned int msiof2_txd_a_pins[] = {
+ /* TXD */
+ RCAR_GP_PIN(0, 11),
+};
+
+static const unsigned int msiof2_txd_a_mux[] = {
+ MSIOF2_TXD_A_MARK,
+};
+
+static const unsigned int msiof2_rxd_a_pins[] = {
+ /* RXD */
+ RCAR_GP_PIN(0, 10),
+};
+
+static const unsigned int msiof2_rxd_a_mux[] = {
+ MSIOF2_RXD_A_MARK,
+};
+
+static const unsigned int msiof2_clk_b_pins[] = {
+ /* SCK */
+ RCAR_GP_PIN(1, 13),
+};
+
+static const unsigned int msiof2_clk_b_mux[] = {
+ MSIOF2_SCK_B_MARK,
+};
+
+static const unsigned int msiof2_sync_b_pins[] = {
+ /* SYNC */
+ RCAR_GP_PIN(1, 10),
+};
+
+static const unsigned int msiof2_sync_b_mux[] = {
+ MSIOF2_SYNC_B_MARK,
+};
+
+static const unsigned int msiof2_ss1_b_pins[] = {
+ /* SS1 */
+ RCAR_GP_PIN(1, 16),
+};
+
+static const unsigned int msiof2_ss1_b_mux[] = {
+ MSIOF2_SS1_B_MARK,
+};
+
+static const unsigned int msiof2_ss2_b_pins[] = {
+ /* SS2 */
+ RCAR_GP_PIN(1, 12),
+};
+
+static const unsigned int msiof2_ss2_b_mux[] = {
+ MSIOF2_SS2_B_MARK,
+};
+
+static const unsigned int msiof2_txd_b_pins[] = {
+ /* TXD */
+ RCAR_GP_PIN(1, 15),
+};
+
+static const unsigned int msiof2_txd_b_mux[] = {
+ MSIOF2_TXD_B_MARK,
+};
+
+static const unsigned int msiof2_rxd_b_pins[] = {
+ /* RXD */
+ RCAR_GP_PIN(1, 14),
+};
+
+static const unsigned int msiof2_rxd_b_mux[] = {
+ MSIOF2_RXD_B_MARK,
+};
+
+/* - MSIOF3 ----------------------------------------------------------------- */
+static const unsigned int msiof3_clk_a_pins[] = {
+ /* SCK */
+ RCAR_GP_PIN(0, 0),
+};
+
+static const unsigned int msiof3_clk_a_mux[] = {
+ MSIOF3_SCK_A_MARK,
+};
+
+static const unsigned int msiof3_sync_a_pins[] = {
+ /* SYNC */
+ RCAR_GP_PIN(0, 1),
+};
+
+static const unsigned int msiof3_sync_a_mux[] = {
+ MSIOF3_SYNC_A_MARK,
+};
+
+static const unsigned int msiof3_ss1_a_pins[] = {
+ /* SS1 */
+ RCAR_GP_PIN(0, 15),
+};
+
+static const unsigned int msiof3_ss1_a_mux[] = {
+ MSIOF3_SS1_A_MARK,
+};
+
+static const unsigned int msiof3_ss2_a_pins[] = {
+ /* SS2 */
+ RCAR_GP_PIN(0, 4),
+};
+
+static const unsigned int msiof3_ss2_a_mux[] = {
+ MSIOF3_SS2_A_MARK,
+};
+
+static const unsigned int msiof3_txd_a_pins[] = {
+ /* TXD */
+ RCAR_GP_PIN(0, 3),
+};
+
+static const unsigned int msiof3_txd_a_mux[] = {
+ MSIOF3_TXD_A_MARK,
+};
+
+static const unsigned int msiof3_rxd_a_pins[] = {
+ /* RXD */
+ RCAR_GP_PIN(0, 2),
+};
+
+static const unsigned int msiof3_rxd_a_mux[] = {
+ MSIOF3_RXD_A_MARK,
+};
+
+static const unsigned int msiof3_clk_b_pins[] = {
+ /* SCK */
+ RCAR_GP_PIN(1, 5),
+};
+
+static const unsigned int msiof3_clk_b_mux[] = {
+ MSIOF3_SCK_B_MARK,
+};
+
+static const unsigned int msiof3_sync_b_pins[] = {
+ /* SYNC */
+ RCAR_GP_PIN(1, 4),
+};
+
+static const unsigned int msiof3_sync_b_mux[] = {
+ MSIOF3_SYNC_B_MARK,
+};
+
+static const unsigned int msiof3_ss1_b_pins[] = {
+ /* SS1 */
+ RCAR_GP_PIN(1, 0),
+};
+
+static const unsigned int msiof3_ss1_b_mux[] = {
+ MSIOF3_SS1_B_MARK,
+};
+
+static const unsigned int msiof3_txd_b_pins[] = {
+ /* TXD */
+ RCAR_GP_PIN(1, 7),
+};
+
+static const unsigned int msiof3_txd_b_mux[] = {
+ MSIOF3_TXD_B_MARK,
+};
+
+static const unsigned int msiof3_rxd_b_pins[] = {
+ /* RXD */
+ RCAR_GP_PIN(1, 6),
+};
+
+static const unsigned int msiof3_rxd_b_mux[] = {
+ MSIOF3_RXD_B_MARK,
+};
+
+/* - PWM0 --------------------------------------------------------------------*/
+static const unsigned int pwm0_a_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(2, 22),
+};
+
+static const unsigned int pwm0_a_mux[] = {
+ PWM0_A_MARK,
+};
+
+static const unsigned int pwm0_b_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(6, 3),
+};
+
+static const unsigned int pwm0_b_mux[] = {
+ PWM0_B_MARK,
+};
+
+/* - PWM1 --------------------------------------------------------------------*/
+static const unsigned int pwm1_a_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(2, 23),
+};
+
+static const unsigned int pwm1_a_mux[] = {
+ PWM1_A_MARK,
+};
+
+static const unsigned int pwm1_b_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(6, 4),
+};
+
+static const unsigned int pwm1_b_mux[] = {
+ PWM1_B_MARK,
+};
+
+/* - PWM2 --------------------------------------------------------------------*/
+static const unsigned int pwm2_a_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(1, 0),
+};
+
+static const unsigned int pwm2_a_mux[] = {
+ PWM2_A_MARK,
+};
+
+static const unsigned int pwm2_b_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(1, 4),
+};
+
+static const unsigned int pwm2_b_mux[] = {
+ PWM2_B_MARK,
+};
+
+static const unsigned int pwm2_c_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(6, 5),
+};
+
+static const unsigned int pwm2_c_mux[] = {
+ PWM2_C_MARK,
+};
+
+/* - PWM3 --------------------------------------------------------------------*/
+static const unsigned int pwm3_a_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(1, 1),
+};
+
+static const unsigned int pwm3_a_mux[] = {
+ PWM3_A_MARK,
+};
+
+static const unsigned int pwm3_b_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(1, 5),
+};
+
+static const unsigned int pwm3_b_mux[] = {
+ PWM3_B_MARK,
+};
+
+static const unsigned int pwm3_c_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(6, 6),
+};
+
+static const unsigned int pwm3_c_mux[] = {
+ PWM3_C_MARK,
+};
+
+/* - PWM4 --------------------------------------------------------------------*/
+static const unsigned int pwm4_a_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(1, 3),
+};
+
+static const unsigned int pwm4_a_mux[] = {
+ PWM4_A_MARK,
+};
+
+static const unsigned int pwm4_b_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(6, 7),
+};
+
+static const unsigned int pwm4_b_mux[] = {
+ PWM4_B_MARK,
+};
+
+/* - PWM5 --------------------------------------------------------------------*/
+static const unsigned int pwm5_a_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(2, 24),
+};
+
+static const unsigned int pwm5_a_mux[] = {
+ PWM5_A_MARK,
+};
+
+static const unsigned int pwm5_b_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(6, 10),
+};
+
+static const unsigned int pwm5_b_mux[] = {
+ PWM5_B_MARK,
+};
+
+/* - PWM6 --------------------------------------------------------------------*/
+static const unsigned int pwm6_a_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(2, 25),
+};
+
+static const unsigned int pwm6_a_mux[] = {
+ PWM6_A_MARK,
+};
+
+static const unsigned int pwm6_b_pins[] = {
+ /* PWM */
+ RCAR_GP_PIN(6, 11),
+};
+
+static const unsigned int pwm6_b_mux[] = {
+ PWM6_B_MARK,
+};
+
/* - SCIF0 ------------------------------------------------------------------ */
static const unsigned int scif0_data_a_pins[] = {
/* RX, TX */
USB3HS0_ID_MARK,
};
-static const struct sh_pfc_pin_group pinmux_groups[] = {
- SH_PFC_PIN_GROUP(avb_link),
- SH_PFC_PIN_GROUP(avb_magic),
- SH_PFC_PIN_GROUP(avb_phy_int),
- SH_PFC_PIN_GROUP(avb_mii),
- SH_PFC_PIN_GROUP(avb_avtp_pps),
- SH_PFC_PIN_GROUP(avb_avtp_match_a),
- SH_PFC_PIN_GROUP(avb_avtp_capture_a),
- SH_PFC_PIN_GROUP(i2c1_a),
- SH_PFC_PIN_GROUP(i2c1_b),
- SH_PFC_PIN_GROUP(i2c1_c),
- SH_PFC_PIN_GROUP(i2c1_d),
- SH_PFC_PIN_GROUP(i2c2_a),
- SH_PFC_PIN_GROUP(i2c2_b),
- SH_PFC_PIN_GROUP(i2c2_c),
- SH_PFC_PIN_GROUP(i2c2_d),
- SH_PFC_PIN_GROUP(i2c2_e),
- SH_PFC_PIN_GROUP(i2c4),
- SH_PFC_PIN_GROUP(i2c5),
- SH_PFC_PIN_GROUP(i2c6_a),
- SH_PFC_PIN_GROUP(i2c6_b),
- SH_PFC_PIN_GROUP(i2c7_a),
- SH_PFC_PIN_GROUP(i2c7_b),
- SH_PFC_PIN_GROUP(scif0_data_a),
- SH_PFC_PIN_GROUP(scif0_clk_a),
- SH_PFC_PIN_GROUP(scif0_ctrl_a),
- SH_PFC_PIN_GROUP(scif0_data_b),
- SH_PFC_PIN_GROUP(scif0_clk_b),
- SH_PFC_PIN_GROUP(scif1_data),
- SH_PFC_PIN_GROUP(scif1_clk),
- SH_PFC_PIN_GROUP(scif1_ctrl),
- SH_PFC_PIN_GROUP(scif2_data_a),
- SH_PFC_PIN_GROUP(scif2_clk_a),
- SH_PFC_PIN_GROUP(scif2_data_b),
- SH_PFC_PIN_GROUP(scif3_data_a),
- SH_PFC_PIN_GROUP(scif3_clk_a),
- SH_PFC_PIN_GROUP(scif3_ctrl_a),
- SH_PFC_PIN_GROUP(scif3_data_b),
- SH_PFC_PIN_GROUP(scif3_data_c),
- SH_PFC_PIN_GROUP(scif3_clk_c),
- SH_PFC_PIN_GROUP(scif4_data_a),
- SH_PFC_PIN_GROUP(scif4_clk_a),
- SH_PFC_PIN_GROUP(scif4_ctrl_a),
- SH_PFC_PIN_GROUP(scif4_data_b),
- SH_PFC_PIN_GROUP(scif4_clk_b),
- SH_PFC_PIN_GROUP(scif4_data_c),
- SH_PFC_PIN_GROUP(scif4_ctrl_c),
- SH_PFC_PIN_GROUP(scif5_data_a),
- SH_PFC_PIN_GROUP(scif5_clk_a),
- SH_PFC_PIN_GROUP(scif5_data_b),
- SH_PFC_PIN_GROUP(scif5_data_c),
- SH_PFC_PIN_GROUP(scif_clk_a),
- SH_PFC_PIN_GROUP(scif_clk_b),
- SH_PFC_PIN_GROUP(usb0_a),
- SH_PFC_PIN_GROUP(usb0_b),
- SH_PFC_PIN_GROUP(usb0_id),
- SH_PFC_PIN_GROUP(usb30),
- SH_PFC_PIN_GROUP(usb30_id),
+static const struct {
+ struct sh_pfc_pin_group common[123];
+ struct sh_pfc_pin_group automotive[0];
+} pinmux_groups = {
+ .common = {
+ SH_PFC_PIN_GROUP(avb_link),
+ SH_PFC_PIN_GROUP(avb_magic),
+ SH_PFC_PIN_GROUP(avb_phy_int),
+ SH_PFC_PIN_GROUP(avb_mii),
+ SH_PFC_PIN_GROUP(avb_avtp_pps),
+ SH_PFC_PIN_GROUP(avb_avtp_match_a),
+ SH_PFC_PIN_GROUP(avb_avtp_capture_a),
+ SH_PFC_PIN_GROUP(du_rgb666),
+ SH_PFC_PIN_GROUP(du_rgb888),
+ SH_PFC_PIN_GROUP(du_clk_in_0),
+ SH_PFC_PIN_GROUP(du_clk_in_1),
+ SH_PFC_PIN_GROUP(du_clk_out_0),
+ SH_PFC_PIN_GROUP(du_sync),
+ SH_PFC_PIN_GROUP(du_disp_cde),
+ SH_PFC_PIN_GROUP(du_cde),
+ SH_PFC_PIN_GROUP(du_disp),
+ SH_PFC_PIN_GROUP(i2c1_a),
+ SH_PFC_PIN_GROUP(i2c1_b),
+ SH_PFC_PIN_GROUP(i2c1_c),
+ SH_PFC_PIN_GROUP(i2c1_d),
+ SH_PFC_PIN_GROUP(i2c2_a),
+ SH_PFC_PIN_GROUP(i2c2_b),
+ SH_PFC_PIN_GROUP(i2c2_c),
+ SH_PFC_PIN_GROUP(i2c2_d),
+ SH_PFC_PIN_GROUP(i2c2_e),
+ SH_PFC_PIN_GROUP(i2c4),
+ SH_PFC_PIN_GROUP(i2c5),
+ SH_PFC_PIN_GROUP(i2c6_a),
+ SH_PFC_PIN_GROUP(i2c6_b),
+ SH_PFC_PIN_GROUP(i2c7_a),
+ SH_PFC_PIN_GROUP(i2c7_b),
+ SH_PFC_PIN_GROUP(intc_ex_irq0),
+ SH_PFC_PIN_GROUP(intc_ex_irq1),
+ SH_PFC_PIN_GROUP(intc_ex_irq2),
+ SH_PFC_PIN_GROUP(intc_ex_irq3),
+ SH_PFC_PIN_GROUP(intc_ex_irq4),
+ SH_PFC_PIN_GROUP(intc_ex_irq5),
+ SH_PFC_PIN_GROUP(msiof0_clk),
+ SH_PFC_PIN_GROUP(msiof0_sync),
+ SH_PFC_PIN_GROUP(msiof0_ss1),
+ SH_PFC_PIN_GROUP(msiof0_ss2),
+ SH_PFC_PIN_GROUP(msiof0_txd),
+ SH_PFC_PIN_GROUP(msiof0_rxd),
+ SH_PFC_PIN_GROUP(msiof1_clk),
+ SH_PFC_PIN_GROUP(msiof1_sync),
+ SH_PFC_PIN_GROUP(msiof1_ss1),
+ SH_PFC_PIN_GROUP(msiof1_ss2),
+ SH_PFC_PIN_GROUP(msiof1_txd),
+ SH_PFC_PIN_GROUP(msiof1_rxd),
+ SH_PFC_PIN_GROUP(msiof2_clk_a),
+ SH_PFC_PIN_GROUP(msiof2_sync_a),
+ SH_PFC_PIN_GROUP(msiof2_ss1_a),
+ SH_PFC_PIN_GROUP(msiof2_ss2_a),
+ SH_PFC_PIN_GROUP(msiof2_txd_a),
+ SH_PFC_PIN_GROUP(msiof2_rxd_a),
+ SH_PFC_PIN_GROUP(msiof2_clk_b),
+ SH_PFC_PIN_GROUP(msiof2_sync_b),
+ SH_PFC_PIN_GROUP(msiof2_ss1_b),
+ SH_PFC_PIN_GROUP(msiof2_ss2_b),
+ SH_PFC_PIN_GROUP(msiof2_txd_b),
+ SH_PFC_PIN_GROUP(msiof2_rxd_b),
+ SH_PFC_PIN_GROUP(msiof3_clk_a),
+ SH_PFC_PIN_GROUP(msiof3_sync_a),
+ SH_PFC_PIN_GROUP(msiof3_ss1_a),
+ SH_PFC_PIN_GROUP(msiof3_ss2_a),
+ SH_PFC_PIN_GROUP(msiof3_txd_a),
+ SH_PFC_PIN_GROUP(msiof3_rxd_a),
+ SH_PFC_PIN_GROUP(msiof3_clk_b),
+ SH_PFC_PIN_GROUP(msiof3_sync_b),
+ SH_PFC_PIN_GROUP(msiof3_ss1_b),
+ SH_PFC_PIN_GROUP(msiof3_txd_b),
+ SH_PFC_PIN_GROUP(msiof3_rxd_b),
+ SH_PFC_PIN_GROUP(pwm0_a),
+ SH_PFC_PIN_GROUP(pwm0_b),
+ SH_PFC_PIN_GROUP(pwm1_a),
+ SH_PFC_PIN_GROUP(pwm1_b),
+ SH_PFC_PIN_GROUP(pwm2_a),
+ SH_PFC_PIN_GROUP(pwm2_b),
+ SH_PFC_PIN_GROUP(pwm2_c),
+ SH_PFC_PIN_GROUP(pwm3_a),
+ SH_PFC_PIN_GROUP(pwm3_b),
+ SH_PFC_PIN_GROUP(pwm3_c),
+ SH_PFC_PIN_GROUP(pwm4_a),
+ SH_PFC_PIN_GROUP(pwm4_b),
+ SH_PFC_PIN_GROUP(pwm5_a),
+ SH_PFC_PIN_GROUP(pwm5_b),
+ SH_PFC_PIN_GROUP(pwm6_a),
+ SH_PFC_PIN_GROUP(pwm6_b),
+ SH_PFC_PIN_GROUP(scif0_data_a),
+ SH_PFC_PIN_GROUP(scif0_clk_a),
+ SH_PFC_PIN_GROUP(scif0_ctrl_a),
+ SH_PFC_PIN_GROUP(scif0_data_b),
+ SH_PFC_PIN_GROUP(scif0_clk_b),
+ SH_PFC_PIN_GROUP(scif1_data),
+ SH_PFC_PIN_GROUP(scif1_clk),
+ SH_PFC_PIN_GROUP(scif1_ctrl),
+ SH_PFC_PIN_GROUP(scif2_data_a),
+ SH_PFC_PIN_GROUP(scif2_clk_a),
+ SH_PFC_PIN_GROUP(scif2_data_b),
+ SH_PFC_PIN_GROUP(scif3_data_a),
+ SH_PFC_PIN_GROUP(scif3_clk_a),
+ SH_PFC_PIN_GROUP(scif3_ctrl_a),
+ SH_PFC_PIN_GROUP(scif3_data_b),
+ SH_PFC_PIN_GROUP(scif3_data_c),
+ SH_PFC_PIN_GROUP(scif3_clk_c),
+ SH_PFC_PIN_GROUP(scif4_data_a),
+ SH_PFC_PIN_GROUP(scif4_clk_a),
+ SH_PFC_PIN_GROUP(scif4_ctrl_a),
+ SH_PFC_PIN_GROUP(scif4_data_b),
+ SH_PFC_PIN_GROUP(scif4_clk_b),
+ SH_PFC_PIN_GROUP(scif4_data_c),
+ SH_PFC_PIN_GROUP(scif4_ctrl_c),
+ SH_PFC_PIN_GROUP(scif5_data_a),
+ SH_PFC_PIN_GROUP(scif5_clk_a),
+ SH_PFC_PIN_GROUP(scif5_data_b),
+ SH_PFC_PIN_GROUP(scif5_data_c),
+ SH_PFC_PIN_GROUP(scif_clk_a),
+ SH_PFC_PIN_GROUP(scif_clk_b),
+ SH_PFC_PIN_GROUP(usb0_a),
+ SH_PFC_PIN_GROUP(usb0_b),
+ SH_PFC_PIN_GROUP(usb0_id),
+ SH_PFC_PIN_GROUP(usb30),
+ SH_PFC_PIN_GROUP(usb30_id),
+ }
};
static const char * const avb_groups[] = {
"avb_avtp_capture_a",
};
+static const char * const du_groups[] = {
+ "du_rgb666",
+ "du_rgb888",
+ "du_clk_in_0",
+ "du_clk_in_1",
+ "du_clk_out_0",
+ "du_sync",
+ "du_disp_cde",
+ "du_cde",
+ "du_disp",
+};
+
static const char * const i2c1_groups[] = {
"i2c1_a",
"i2c1_b",
"i2c7_b",
};
+static const char * const intc_ex_groups[] = {
+ "intc_ex_irq0",
+ "intc_ex_irq1",
+ "intc_ex_irq2",
+ "intc_ex_irq3",
+ "intc_ex_irq4",
+ "intc_ex_irq5",
+};
+
+static const char * const msiof0_groups[] = {
+ "msiof0_clk",
+ "msiof0_sync",
+ "msiof0_ss1",
+ "msiof0_ss2",
+ "msiof0_txd",
+ "msiof0_rxd",
+};
+
+static const char * const msiof1_groups[] = {
+ "msiof1_clk",
+ "msiof1_sync",
+ "msiof1_ss1",
+ "msiof1_ss2",
+ "msiof1_txd",
+ "msiof1_rxd",
+};
+
+static const char * const msiof2_groups[] = {
+ "msiof2_clk_a",
+ "msiof2_sync_a",
+ "msiof2_ss1_a",
+ "msiof2_ss2_a",
+ "msiof2_txd_a",
+ "msiof2_rxd_a",
+ "msiof2_clk_b",
+ "msiof2_sync_b",
+ "msiof2_ss1_b",
+ "msiof2_ss2_b",
+ "msiof2_txd_b",
+ "msiof2_rxd_b",
+};
+
+static const char * const msiof3_groups[] = {
+ "msiof3_clk_a",
+ "msiof3_sync_a",
+ "msiof3_ss1_a",
+ "msiof3_ss2_a",
+ "msiof3_txd_a",
+ "msiof3_rxd_a",
+ "msiof3_clk_b",
+ "msiof3_sync_b",
+ "msiof3_ss1_b",
+ "msiof3_txd_b",
+ "msiof3_rxd_b",
+};
+
+static const char * const pwm0_groups[] = {
+ "pwm0_a",
+ "pwm0_b",
+};
+
+static const char * const pwm1_groups[] = {
+ "pwm1_a",
+ "pwm1_b",
+};
+
+static const char * const pwm2_groups[] = {
+ "pwm2_a",
+ "pwm2_b",
+ "pwm2_c",
+};
+
+static const char * const pwm3_groups[] = {
+ "pwm3_a",
+ "pwm3_b",
+ "pwm3_c",
+};
+
+static const char * const pwm4_groups[] = {
+ "pwm4_a",
+ "pwm4_b",
+};
+
+static const char * const pwm5_groups[] = {
+ "pwm5_a",
+ "pwm5_b",
+};
+
+static const char * const pwm6_groups[] = {
+ "pwm6_a",
+ "pwm6_b",
+};
+
static const char * const scif0_groups[] = {
"scif0_data_a",
"scif0_clk_a",
"usb30_id",
};
-static const struct sh_pfc_function pinmux_functions[] = {
- SH_PFC_FUNCTION(avb),
- SH_PFC_FUNCTION(i2c1),
- SH_PFC_FUNCTION(i2c2),
- SH_PFC_FUNCTION(i2c4),
- SH_PFC_FUNCTION(i2c5),
- SH_PFC_FUNCTION(i2c6),
- SH_PFC_FUNCTION(i2c7),
- SH_PFC_FUNCTION(scif0),
- SH_PFC_FUNCTION(scif1),
- SH_PFC_FUNCTION(scif2),
- SH_PFC_FUNCTION(scif3),
- SH_PFC_FUNCTION(scif4),
- SH_PFC_FUNCTION(scif5),
- SH_PFC_FUNCTION(scif_clk),
- SH_PFC_FUNCTION(usb0),
- SH_PFC_FUNCTION(usb30),
+static const struct {
+ struct sh_pfc_function common[29];
+ struct sh_pfc_function automotive[0];
+} pinmux_functions = {
+ .common = {
+ SH_PFC_FUNCTION(avb),
+ SH_PFC_FUNCTION(du),
+ SH_PFC_FUNCTION(i2c1),
+ SH_PFC_FUNCTION(i2c2),
+ SH_PFC_FUNCTION(i2c4),
+ SH_PFC_FUNCTION(i2c5),
+ SH_PFC_FUNCTION(i2c6),
+ SH_PFC_FUNCTION(i2c7),
+ SH_PFC_FUNCTION(intc_ex),
+ SH_PFC_FUNCTION(msiof0),
+ SH_PFC_FUNCTION(msiof1),
+ SH_PFC_FUNCTION(msiof2),
+ SH_PFC_FUNCTION(msiof3),
+ SH_PFC_FUNCTION(pwm0),
+ SH_PFC_FUNCTION(pwm1),
+ SH_PFC_FUNCTION(pwm2),
+ SH_PFC_FUNCTION(pwm3),
+ SH_PFC_FUNCTION(pwm4),
+ SH_PFC_FUNCTION(pwm5),
+ SH_PFC_FUNCTION(pwm6),
+ SH_PFC_FUNCTION(scif0),
+ SH_PFC_FUNCTION(scif1),
+ SH_PFC_FUNCTION(scif2),
+ SH_PFC_FUNCTION(scif3),
+ SH_PFC_FUNCTION(scif4),
+ SH_PFC_FUNCTION(scif5),
+ SH_PFC_FUNCTION(scif_clk),
+ SH_PFC_FUNCTION(usb0),
+ SH_PFC_FUNCTION(usb30),
+ }
};
static const struct pinmux_cfg_reg pinmux_config_regs[] = {
.set_bias = r8a77990_pinmux_set_bias,
};
+#ifdef CONFIG_PINCTRL_PFC_R8A774C0
+const struct sh_pfc_soc_info r8a774c0_pinmux_info = {
+ .name = "r8a774c0_pfc",
+ .ops = &r8a77990_pinmux_ops,
+ .unlock_reg = 0xe6060000, /* PMMR */
+
+ .function = { PINMUX_FUNCTION_BEGIN, PINMUX_FUNCTION_END },
+
+ .pins = pinmux_pins,
+ .nr_pins = ARRAY_SIZE(pinmux_pins),
+ .groups = pinmux_groups.common,
+ .nr_groups = ARRAY_SIZE(pinmux_groups.common),
+ .functions = pinmux_functions.common,
+ .nr_functions = ARRAY_SIZE(pinmux_functions.common),
+
+ .cfg_regs = pinmux_config_regs,
+ .bias_regs = pinmux_bias_regs,
+
+ .pinmux_data = pinmux_data,
+ .pinmux_data_size = ARRAY_SIZE(pinmux_data),
+};
+#endif
+
+#ifdef CONFIG_PINCTRL_PFC_R8A77990
const struct sh_pfc_soc_info r8a77990_pinmux_info = {
.name = "r8a77990_pfc",
.ops = &r8a77990_pinmux_ops,
.pins = pinmux_pins,
.nr_pins = ARRAY_SIZE(pinmux_pins),
- .groups = pinmux_groups,
- .nr_groups = ARRAY_SIZE(pinmux_groups),
- .functions = pinmux_functions,
- .nr_functions = ARRAY_SIZE(pinmux_functions),
+ .groups = pinmux_groups.common,
+ .nr_groups = ARRAY_SIZE(pinmux_groups.common) +
+ ARRAY_SIZE(pinmux_groups.automotive),
+ .functions = pinmux_functions.common,
+ .nr_functions = ARRAY_SIZE(pinmux_functions.common) +
+ ARRAY_SIZE(pinmux_functions.automotive),
.cfg_regs = pinmux_config_regs,
.bias_regs = pinmux_bias_regs,
.pinmux_data = pinmux_data,
.pinmux_data_size = ARRAY_SIZE(pinmux_data),
};
+#endif
+// SPDX-License-Identifier: GPL-2.0
/*
* R8A77995 processor support - PFC hardware block.
*
* R-Car Gen3 processor support - PFC hardware block.
*
* Copyright (C) 2015 Renesas Electronics Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
*/
#include <linux/kernel.h>
PINMUX_SINGLE(QSPI0_SPCLK),
PINMUX_SINGLE(SCL0),
PINMUX_SINGLE(SDA0),
+ PINMUX_SINGLE(MSIOF0_RXD),
+ PINMUX_SINGLE(MSIOF0_TXD),
+ PINMUX_SINGLE(MSIOF0_SYNC),
+ PINMUX_SINGLE(MSIOF0_SCK),
/* IPSR0 */
PINMUX_IPSR_MSEL(IP0_3_0, IRQ0_A, SEL_IRQ_0_0),
MMC_CLK_MARK, MMC_CMD_MARK,
};
+/* - MSIOF0 ----------------------------------------------------------------- */
+static const unsigned int msiof0_clk_pins[] = {
+ /* SCK */
+ RCAR_GP_PIN(4, 12),
+};
+
+static const unsigned int msiof0_clk_mux[] = {
+ MSIOF0_SCK_MARK,
+};
+
+static const unsigned int msiof0_sync_pins[] = {
+ /* SYNC */
+ RCAR_GP_PIN(4, 13),
+};
+
+static const unsigned int msiof0_sync_mux[] = {
+ MSIOF0_SYNC_MARK,
+};
+
+static const unsigned int msiof0_ss1_pins[] = {
+ /* SS1 */
+ RCAR_GP_PIN(4, 20),
+};
+
+static const unsigned int msiof0_ss1_mux[] = {
+ MSIOF0_SS1_MARK,
+};
+
+static const unsigned int msiof0_ss2_pins[] = {
+ /* SS2 */
+ RCAR_GP_PIN(4, 21),
+};
+
+static const unsigned int msiof0_ss2_mux[] = {
+ MSIOF0_SS2_MARK,
+};
+
+static const unsigned int msiof0_txd_pins[] = {
+ /* TXD */
+ RCAR_GP_PIN(4, 14),
+};
+
+static const unsigned int msiof0_txd_mux[] = {
+ MSIOF0_TXD_MARK,
+};
+
+static const unsigned int msiof0_rxd_pins[] = {
+ /* RXD */
+ RCAR_GP_PIN(4, 15),
+};
+
+static const unsigned int msiof0_rxd_mux[] = {
+ MSIOF0_RXD_MARK,
+};
+
+/* - MSIOF1 ----------------------------------------------------------------- */
+static const unsigned int msiof1_clk_pins[] = {
+ /* SCK */
+ RCAR_GP_PIN(4, 16),
+};
+
+static const unsigned int msiof1_clk_mux[] = {
+ MSIOF1_SCK_MARK,
+};
+
+static const unsigned int msiof1_sync_pins[] = {
+ /* SYNC */
+ RCAR_GP_PIN(4, 19),
+};
+
+static const unsigned int msiof1_sync_mux[] = {
+ MSIOF1_SYNC_MARK,
+};
+
+static const unsigned int msiof1_ss1_pins[] = {
+ /* SS1 */
+ RCAR_GP_PIN(4, 25),
+};
+
+static const unsigned int msiof1_ss1_mux[] = {
+ MSIOF1_SS1_MARK,
+};
+
+static const unsigned int msiof1_ss2_pins[] = {
+ /* SS2 */
+ RCAR_GP_PIN(4, 22),
+};
+
+static const unsigned int msiof1_ss2_mux[] = {
+ MSIOF1_SS2_MARK,
+};
+
+static const unsigned int msiof1_txd_pins[] = {
+ /* TXD */
+ RCAR_GP_PIN(4, 17),
+};
+
+static const unsigned int msiof1_txd_mux[] = {
+ MSIOF1_TXD_MARK,
+};
+
+static const unsigned int msiof1_rxd_pins[] = {
+ /* RXD */
+ RCAR_GP_PIN(4, 18),
+};
+
+static const unsigned int msiof1_rxd_mux[] = {
+ MSIOF1_RXD_MARK,
+};
+
+/* - MSIOF2 ----------------------------------------------------------------- */
+static const unsigned int msiof2_clk_pins[] = {
+ /* SCK */
+ RCAR_GP_PIN(0, 3),
+};
+
+static const unsigned int msiof2_clk_mux[] = {
+ MSIOF2_SCK_MARK,
+};
+
+static const unsigned int msiof2_sync_a_pins[] = {
+ /* SYNC */
+ RCAR_GP_PIN(0, 6),
+};
+
+static const unsigned int msiof2_sync_a_mux[] = {
+ MSIOF2_SYNC_A_MARK,
+};
+
+static const unsigned int msiof2_sync_b_pins[] = {
+ /* SYNC */
+ RCAR_GP_PIN(0, 2),
+};
+
+static const unsigned int msiof2_sync_b_mux[] = {
+ MSIOF2_SYNC_B_MARK,
+};
+
+static const unsigned int msiof2_ss1_pins[] = {
+ /* SS1 */
+ RCAR_GP_PIN(0, 7),
+};
+
+static const unsigned int msiof2_ss1_mux[] = {
+ MSIOF2_SS1_MARK,
+};
+
+static const unsigned int msiof2_ss2_pins[] = {
+ /* SS2 */
+ RCAR_GP_PIN(0, 8),
+};
+
+static const unsigned int msiof2_ss2_mux[] = {
+ MSIOF2_SS2_MARK,
+};
+
+static const unsigned int msiof2_txd_pins[] = {
+ /* TXD */
+ RCAR_GP_PIN(0, 4),
+};
+
+static const unsigned int msiof2_txd_mux[] = {
+ MSIOF2_TXD_MARK,
+};
+
+static const unsigned int msiof2_rxd_pins[] = {
+ /* RXD */
+ RCAR_GP_PIN(0, 5),
+};
+
+static const unsigned int msiof2_rxd_mux[] = {
+ MSIOF2_RXD_MARK,
+};
+
+/* - MSIOF3 ----------------------------------------------------------------- */
+static const unsigned int msiof3_clk_a_pins[] = {
+ /* SCK */
+ RCAR_GP_PIN(2, 24),
+};
+
+static const unsigned int msiof3_clk_a_mux[] = {
+ MSIOF3_SCK_A_MARK,
+};
+
+static const unsigned int msiof3_sync_a_pins[] = {
+ /* SYNC */
+ RCAR_GP_PIN(2, 21),
+};
+
+static const unsigned int msiof3_sync_a_mux[] = {
+ MSIOF3_SYNC_A_MARK,
+};
+
+static const unsigned int msiof3_ss1_a_pins[] = {
+ /* SS1 */
+ RCAR_GP_PIN(2, 14),
+};
+
+static const unsigned int msiof3_ss1_a_mux[] = {
+ MSIOF3_SS1_A_MARK,
+};
+
+static const unsigned int msiof3_ss2_a_pins[] = {
+ /* SS2 */
+ RCAR_GP_PIN(2, 10),
+};
+
+static const unsigned int msiof3_ss2_a_mux[] = {
+ MSIOF3_SS2_A_MARK,
+};
+
+static const unsigned int msiof3_txd_a_pins[] = {
+ /* TXD */
+ RCAR_GP_PIN(2, 22),
+};
+
+static const unsigned int msiof3_txd_a_mux[] = {
+ MSIOF3_TXD_A_MARK,
+};
+
+static const unsigned int msiof3_rxd_a_pins[] = {
+ /* RXD */
+ RCAR_GP_PIN(2, 23),
+};
+
+static const unsigned int msiof3_rxd_a_mux[] = {
+ MSIOF3_RXD_A_MARK,
+};
+
+static const unsigned int msiof3_clk_b_pins[] = {
+ /* SCK */
+ RCAR_GP_PIN(1, 8),
+};
+
+static const unsigned int msiof3_clk_b_mux[] = {
+ MSIOF3_SCK_B_MARK,
+};
+
+static const unsigned int msiof3_sync_b_pins[] = {
+ /* SYNC */
+ RCAR_GP_PIN(1, 9),
+};
+
+static const unsigned int msiof3_sync_b_mux[] = {
+ MSIOF3_SYNC_B_MARK,
+};
+
+static const unsigned int msiof3_ss1_b_pins[] = {
+ /* SS1 */
+ RCAR_GP_PIN(1, 6),
+};
+
+static const unsigned int msiof3_ss1_b_mux[] = {
+ MSIOF3_SS1_B_MARK,
+};
+
+static const unsigned int msiof3_ss2_b_pins[] = {
+ /* SS2 */
+ RCAR_GP_PIN(1, 7),
+};
+
+static const unsigned int msiof3_ss2_b_mux[] = {
+ MSIOF3_SS2_B_MARK,
+};
+
+static const unsigned int msiof3_txd_b_pins[] = {
+ /* TXD */
+ RCAR_GP_PIN(1, 0),
+};
+
+static const unsigned int msiof3_txd_b_mux[] = {
+ MSIOF3_TXD_B_MARK,
+};
+
+static const unsigned int msiof3_rxd_b_pins[] = {
+ /* RXD */
+ RCAR_GP_PIN(1, 1),
+};
+
+static const unsigned int msiof3_rxd_b_mux[] = {
+ MSIOF3_RXD_B_MARK,
+};
+
/* - PWM0 ------------------------------------------------------------------ */
static const unsigned int pwm0_a_pins[] = {
/* PWM */
SH_PFC_PIN_GROUP(mmc_data4),
SH_PFC_PIN_GROUP(mmc_data8),
SH_PFC_PIN_GROUP(mmc_ctrl),
+ SH_PFC_PIN_GROUP(msiof0_clk),
+ SH_PFC_PIN_GROUP(msiof0_sync),
+ SH_PFC_PIN_GROUP(msiof0_ss1),
+ SH_PFC_PIN_GROUP(msiof0_ss2),
+ SH_PFC_PIN_GROUP(msiof0_txd),
+ SH_PFC_PIN_GROUP(msiof0_rxd),
+ SH_PFC_PIN_GROUP(msiof1_clk),
+ SH_PFC_PIN_GROUP(msiof1_sync),
+ SH_PFC_PIN_GROUP(msiof1_ss1),
+ SH_PFC_PIN_GROUP(msiof1_ss2),
+ SH_PFC_PIN_GROUP(msiof1_txd),
+ SH_PFC_PIN_GROUP(msiof1_rxd),
+ SH_PFC_PIN_GROUP(msiof2_clk),
+ SH_PFC_PIN_GROUP(msiof2_sync_a),
+ SH_PFC_PIN_GROUP(msiof2_sync_b),
+ SH_PFC_PIN_GROUP(msiof2_ss1),
+ SH_PFC_PIN_GROUP(msiof2_ss2),
+ SH_PFC_PIN_GROUP(msiof2_txd),
+ SH_PFC_PIN_GROUP(msiof2_rxd),
+ SH_PFC_PIN_GROUP(msiof3_clk_a),
+ SH_PFC_PIN_GROUP(msiof3_sync_a),
+ SH_PFC_PIN_GROUP(msiof3_ss1_a),
+ SH_PFC_PIN_GROUP(msiof3_ss2_a),
+ SH_PFC_PIN_GROUP(msiof3_txd_a),
+ SH_PFC_PIN_GROUP(msiof3_rxd_a),
+ SH_PFC_PIN_GROUP(msiof3_clk_b),
+ SH_PFC_PIN_GROUP(msiof3_sync_b),
+ SH_PFC_PIN_GROUP(msiof3_ss1_b),
+ SH_PFC_PIN_GROUP(msiof3_ss2_b),
+ SH_PFC_PIN_GROUP(msiof3_txd_b),
+ SH_PFC_PIN_GROUP(msiof3_rxd_b),
SH_PFC_PIN_GROUP(pwm0_a),
SH_PFC_PIN_GROUP(pwm0_b),
SH_PFC_PIN_GROUP(pwm0_c),
"vin4_clk",
};
+static const char * const msiof0_groups[] = {
+ "msiof0_clk",
+ "msiof0_sync",
+ "msiof0_ss1",
+ "msiof0_ss2",
+ "msiof0_txd",
+ "msiof0_rxd",
+};
+
+static const char * const msiof1_groups[] = {
+ "msiof1_clk",
+ "msiof1_sync",
+ "msiof1_ss1",
+ "msiof1_ss2",
+ "msiof1_txd",
+ "msiof1_rxd",
+};
+
+static const char * const msiof2_groups[] = {
+ "msiof2_clk",
+ "msiof2_sync_a",
+ "msiof2_sync_b",
+ "msiof2_ss1",
+ "msiof2_ss2",
+ "msiof2_txd",
+ "msiof2_rxd",
+};
+
+static const char * const msiof3_groups[] = {
+ "msiof3_clk_a",
+ "msiof3_sync_a",
+ "msiof3_ss1_a",
+ "msiof3_ss2_a",
+ "msiof3_txd_a",
+ "msiof3_rxd_a",
+ "msiof3_clk_b",
+ "msiof3_sync_b",
+ "msiof3_ss1_b",
+ "msiof3_ss2_b",
+ "msiof3_txd_b",
+ "msiof3_rxd_b",
+};
+
static const struct sh_pfc_function pinmux_functions[] = {
SH_PFC_FUNCTION(audio_clk),
SH_PFC_FUNCTION(avb0),
SH_PFC_FUNCTION(i2c2),
SH_PFC_FUNCTION(i2c3),
SH_PFC_FUNCTION(mmc),
+ SH_PFC_FUNCTION(msiof0),
+ SH_PFC_FUNCTION(msiof1),
+ SH_PFC_FUNCTION(msiof2),
+ SH_PFC_FUNCTION(msiof3),
SH_PFC_FUNCTION(pwm0),
SH_PFC_FUNCTION(pwm1),
SH_PFC_FUNCTION(pwm2),
+// SPDX-License-Identifier: GPL-2.0
/*
* SH7203 Pinmux
*
* Copyright (C) 2008 Magnus Damm
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* SH7264 Pinmux
*
* Copyright (C) 2012 Renesas Electronics Europe Ltd
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* SH7269 Pinmux
*
* Copyright (C) 2012 Renesas Electronics Europe Ltd
* Copyright (C) 2012 Phil Edworthy
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* sh73a0 processor support - PFC hardware block
*
* Copyright (C) 2010 Renesas Solutions Corp.
* Copyright (C) 2010 NISHIMOTO Hiroki
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; version 2 of the
- * License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
#include <linux/io.h>
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* SH7720 Pinmux
*
* Copyright (C) 2008 Magnus Damm
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* SH7723 Pinmux
*
* Copyright (C) 2008 Magnus Damm
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/init.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* SH7724 Pinmux
*
*
* Based on SH7723 Pinmux
* Copyright (C) 2008 Magnus Damm
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/init.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* SH7734 processor support - PFC hardware block
*
* Copyright (C) 2012 Renesas Solutions Corp.
* Copyright (C) 2012 Nobuhiro Iwamatsu <nobuhiro.iwamatsu.yj@renesas.com>
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/init.h>
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* SH7757 (B0 step) Pinmux
*
*
* Based on SH7723 Pinmux
* Copyright (C) 2008 Magnus Damm
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/init.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* SH7785 Pinmux
*
* Copyright (C) 2008 Magnus Damm
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/init.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* SH7786 Pinmux
*
* Based on SH7785 pinmux
*
* Copyright (C) 2008 Magnus Damm
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/init.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* SH-X3 prototype CPU pinmux
*
* Copyright (C) 2010 Paul Mundt
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#include <linux/init.h>
#include <linux/kernel.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* SuperH Pin Function Controller pinmux support.
*
* Copyright (C) 2012 Paul Mundt
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#define DRV_NAME "sh-pfc"
-/*
+/* SPDX-License-Identifier: GPL-2.0
+ *
* SuperH Pin Function Controller Support
*
* Copyright (c) 2008 Magnus Damm
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
*/
#ifndef __SH_PFC_H
extern const struct sh_pfc_soc_info r8a73a4_pinmux_info;
extern const struct sh_pfc_soc_info r8a7740_pinmux_info;
extern const struct sh_pfc_soc_info r8a7743_pinmux_info;
+extern const struct sh_pfc_soc_info r8a7744_pinmux_info;
extern const struct sh_pfc_soc_info r8a7745_pinmux_info;
extern const struct sh_pfc_soc_info r8a77470_pinmux_info;
+extern const struct sh_pfc_soc_info r8a774a1_pinmux_info;
+extern const struct sh_pfc_soc_info r8a774c0_pinmux_info;
extern const struct sh_pfc_soc_info r8a7778_pinmux_info;
extern const struct sh_pfc_soc_info r8a7779_pinmux_info;
extern const struct sh_pfc_soc_info r8a7790_pinmux_info;
#include <linux/of_device.h>
#include <linux/of_platform.h>
#include <linux/of_irq.h>
-#include <linux/of_gpio.h>
#include <linux/pinctrl/machine.h>
#include <linux/pinctrl/pinconf.h>
#include <linux/pinctrl/pinctrl.h>
#include <linux/pinctrl/pinmux.h>
#include <linux/pinctrl/consumer.h>
#include <linux/pinctrl/pinconf-generic.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
/* Definition of Pad&Mux Properties */
#define N 0
{
struct atlas7_pmx *pmx = dev_get_drvdata(dev);
struct atlas7_pad_status *status;
- struct atlas7_pad_config *conf;
int idx;
- u32 bank;
for (idx = 0; idx < pmx->pctl_desc.npins; idx++) {
/* Get this Pad's descriptor from PINCTRL */
- conf = &pmx->pctl_data->confs[idx];
- bank = atlas7_pin_to_bank(idx);
status = &pmx->sleep_data[idx];
/* Restore Function selector */
ret = gpiochip_add_data(chip, a7gc);
if (ret) {
dev_err(&pdev->dev,
- "%s: error in probe function with status %d\n",
- np->name, ret);
+ "%pOF: error in probe function with status %d\n",
+ np, ret);
goto failed;
}
#include <linux/of_device.h>
#include <linux/of_platform.h>
#include <linux/bitops.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/of_gpio.h>
#include "pinctrl-sirf.h"
#ifndef __PINMUX_SPEAR_H__
#define __PINMUX_SPEAR_H__
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/io.h>
#include <linux/pinctrl/pinctrl.h>
#include <linux/types.h>
return ret;
}
+ ret = sprd_pinctrl_parse_dt(sprd_pctl);
+ if (ret) {
+ dev_err(&pdev->dev, "fail to parse dt properties\n");
+ return ret;
+ }
+
pin_desc = devm_kcalloc(&pdev->dev,
pinctrl_info->npins,
sizeof(struct pinctrl_pin_desc),
return PTR_ERR(sprd_pctl->pctl);
}
- ret = sprd_pinctrl_parse_dt(sprd_pctl);
- if (ret) {
- dev_err(&pdev->dev, "fail to parse dt properties\n");
- pinctrl_unregister(sprd_pctl->pctl);
- return ret;
- }
-
return 0;
}
pins = of_find_property(node, "pinmux", NULL);
if (!pins) {
- dev_err(pctl->dev, "missing pins property in node %s .\n",
- node->name);
+ dev_err(pctl->dev, "missing pins property in node %pOFn .\n",
+ node);
return -EINVAL;
}
function = sunxi_pctrl_parse_function_prop(node);
if (!function) {
- dev_err(pctl->dev, "missing function property in node %s\n",
- node->name);
+ dev_err(pctl->dev, "missing function property in node %pOFn\n",
+ node);
return -EINVAL;
}
pin_prop = sunxi_pctrl_find_pins_prop(node, &npins);
if (!pin_prop) {
- dev_err(pctl->dev, "missing pins property in node %s\n",
- node->name);
+ dev_err(pctl->dev, "missing pins property in node %pOFn\n",
+ node);
return -EINVAL;
}
static int sunxi_pinctrl_build_state(struct platform_device *pdev)
{
struct sunxi_pinctrl *pctl = platform_get_drvdata(pdev);
+ void *ptr;
int i;
/*
* We suppose that we won't have any more functions than pins,
* we'll reallocate that later anyway
*/
- pctl->functions = devm_kcalloc(&pdev->dev,
- pctl->ngroups,
- sizeof(*pctl->functions),
- GFP_KERNEL);
+ pctl->functions = kcalloc(pctl->ngroups,
+ sizeof(*pctl->functions),
+ GFP_KERNEL);
if (!pctl->functions)
return -ENOMEM;
}
/* And now allocated and fill the array for real */
- pctl->functions = krealloc(pctl->functions,
- pctl->nfunctions * sizeof(*pctl->functions),
- GFP_KERNEL);
- if (!pctl->functions) {
+ ptr = krealloc(pctl->functions,
+ pctl->nfunctions * sizeof(*pctl->functions),
+ GFP_KERNEL);
+ if (!ptr) {
kfree(pctl->functions);
+ pctl->functions = NULL;
return -ENOMEM;
}
+ pctl->functions = ptr;
for (i = 0; i < pctl->desc->npins; i++) {
const struct sunxi_desc_pin *pin = pctl->desc->pins + i;
func_item = sunxi_pinctrl_find_function_by_name(pctl,
func->name);
- if (!func_item)
+ if (!func_item) {
+ kfree(pctl->functions);
return -EINVAL;
+ }
if (!func_item->groups) {
func_item->groups =
func_item->ngroups,
sizeof(*func_item->groups),
GFP_KERNEL);
- if (!func_item->groups)
+ if (!func_item->groups) {
+ kfree(pctl->functions);
return -ENOMEM;
+ }
}
func_grp = func_item->groups;
return 0;
}
-EXPORT_SYMBOL_GPL(tegra_pinctrl_probe);
pin = ti_iodelay_offset_to_pin(iod, cfg[pin_index].offset);
if (pin < 0) {
- dev_err(iod->dev, "could not add functions for %s %ux\n",
- np->name, cfg[pin_index].offset);
+ dev_err(iod->dev, "could not add functions for %pOFn %ux\n",
+ np, cfg[pin_index].offset);
return -ENODEV;
}
pins[pin_index] = pin;
pd = &iod->pa[pin];
pd->drv_data = &cfg[pin_index];
- dev_dbg(iod->dev, "%s offset=%x a_delay = %d g_delay = %d\n",
- np->name, cfg[pin_index].offset, cfg[pin_index].a_delay,
+ dev_dbg(iod->dev, "%pOFn offset=%x a_delay = %d g_delay = %d\n",
+ np, cfg[pin_index].offset, cfg[pin_index].a_delay,
cfg[pin_index].g_delay);
return 0;
static const int nand_cs1_muxvals[] = {1, 1};
static const unsigned sd_pins[] = {150, 151, 152, 153, 154, 155, 156, 157, 158};
static const int sd_muxvals[] = {0, 0, 0, 0, 0, 0, 0, 0, 0};
-static const unsigned sd1_pins[] = {319, 320, 321, 322, 323, 324, 325, 326,
- 327};
-static const int sd1_muxvals[] = {0, 0, 0, 0, 0, 0, 0, 0, 0};
+static const unsigned int sd1_pins[] = {319, 320, 321, 322, 323, 324, 325, 326};
+static const int sd1_muxvals[] = {0, 0, 0, 0, 0, 0, 0, 0};
static const unsigned spi0_pins[] = {199, 200, 201, 202};
static const int spi0_muxvals[] = {11, 11, 11, 11};
static const unsigned spi1_pins[] = {195, 196, 197, 198, 235, 238, 239};
#ifndef __PINCTRL_UNIPHIER_H__
#define __PINCTRL_UNIPHIER_H__
-#include <linux/bitops.h>
+#include <linux/bits.h>
#include <linux/build_bug.h>
#include <linux/kernel.h>
#include <linux/types.h>
u32 val;
val = readl_relaxed(data->base + reg_dir);
- if (val & BIT(bit))
- return GPIOF_DIR_OUT;
- else
- return GPIOF_DIR_IN;
+ /* Return 0 == output, 1 == input */
+ return !(val & BIT(bit));
}
static int wmt_gpio_get_value(struct gpio_chip *chip, unsigned offset)
* more details.
*/
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
/* VT8500 has no enable register in the extgpio bank. */
#define NO_REG 0xFFFF
ret = cros_ec_cmd_xfer(ec_dev, msg);
if (ret > 0) {
ec_dev->event_size = ret - 1;
- memcpy(&ec_dev->event_data, msg->data, ec_dev->event_size);
+ memcpy(&ec_dev->event_data, msg->data, ret);
}
return ret;
#include <linux/i2c.h>
#include <linux/interrupt.h>
#include <linux/module.h>
+#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
#include <linux/slab.h>
{ }
};
-static int cht_int33fe_probe(struct i2c_client *client)
+static int cht_int33fe_probe(struct platform_device *pdev)
{
- struct device *dev = &client->dev;
+ struct device *dev = &pdev->dev;
struct i2c_board_info board_info;
struct cht_int33fe_data *data;
struct i2c_client *max17047;
if (!data->pi3usb30532)
goto out_unregister_fusb302;
- i2c_set_clientdata(client, data);
+ platform_set_drvdata(pdev, data);
return 0;
return -EPROBE_DEFER; /* Wait for the i2c-adapter to load */
}
-static int cht_int33fe_remove(struct i2c_client *i2c)
+static int cht_int33fe_remove(struct platform_device *pdev)
{
- struct cht_int33fe_data *data = i2c_get_clientdata(i2c);
+ struct cht_int33fe_data *data = platform_get_drvdata(pdev);
i2c_unregister_device(data->pi3usb30532);
i2c_unregister_device(data->fusb302);
return 0;
}
-static const struct i2c_device_id cht_int33fe_i2c_id[] = {
- { }
-};
-MODULE_DEVICE_TABLE(i2c, cht_int33fe_i2c_id);
-
static const struct acpi_device_id cht_int33fe_acpi_ids[] = {
{ "INT33FE", },
{ }
};
MODULE_DEVICE_TABLE(acpi, cht_int33fe_acpi_ids);
-static struct i2c_driver cht_int33fe_driver = {
+static struct platform_driver cht_int33fe_driver = {
.driver = {
.name = "Intel Cherry Trail ACPI INT33FE driver",
.acpi_match_table = ACPI_PTR(cht_int33fe_acpi_ids),
},
- .probe_new = cht_int33fe_probe,
+ .probe = cht_int33fe_probe,
.remove = cht_int33fe_remove,
- .id_table = cht_int33fe_i2c_id,
- .disable_i2c_core_irq_mapping = true,
};
-module_i2c_driver(cht_int33fe_driver);
+module_platform_driver(cht_int33fe_driver);
MODULE_DESCRIPTION("Intel Cherry Trail ACPI INT33FE pseudo device driver");
MODULE_AUTHOR("Hans de Goede <hdegoede@redhat.com>");
/*
* Limit ourselves to Cherry Trail for now, until testing shows we
* need to handle the INT0002 device on Baytrail too.
- * ICPU(INTEL_FAM6_ATOM_SILVERMONT1), * Valleyview, Bay Trail *
+ * ICPU(INTEL_FAM6_ATOM_SILVERMONT), * Valleyview, Bay Trail *
*/
ICPU(INTEL_FAM6_ATOM_AIRMONT), /* Braswell, Cherry Trail */
{}
{ X86_VENDOR_INTEL, 6, model, X86_FEATURE_ANY, (kernel_ulong_t)&ddata }
static const struct x86_cpu_id mid_pb_cpu_ids[] = {
- ICPU(INTEL_FAM6_ATOM_PENWELL, mfld_ddata),
- ICPU(INTEL_FAM6_ATOM_MERRIFIELD, mrfld_ddata),
+ ICPU(INTEL_FAM6_ATOM_SALTWELL_MID, mfld_ddata),
+ ICPU(INTEL_FAM6_ATOM_SILVERMONT_MID, mrfld_ddata),
{}
};
static const struct x86_cpu_id telemetry_debugfs_cpu_ids[] = {
TELEM_DEBUGFS_CPU(INTEL_FAM6_ATOM_GOLDMONT, telem_apl_debugfs_conf),
- TELEM_DEBUGFS_CPU(INTEL_FAM6_ATOM_GEMINI_LAKE, telem_apl_debugfs_conf),
+ TELEM_DEBUGFS_CPU(INTEL_FAM6_ATOM_GOLDMONT_PLUS, telem_apl_debugfs_conf),
{}
};
static const struct x86_cpu_id telemetry_cpu_ids[] = {
TELEM_CPU(INTEL_FAM6_ATOM_GOLDMONT, telem_apl_config),
- TELEM_CPU(INTEL_FAM6_ATOM_GEMINI_LAKE, telem_glk_config),
+ TELEM_CPU(INTEL_FAM6_ATOM_GOLDMONT_PLUS, telem_glk_config),
{}
};
.compute_time_window = rapl_compute_time_window_atom,
};
-#define RAPL_CPU(_model, _ops) { \
- .vendor = X86_VENDOR_INTEL, \
- .family = 6, \
- .model = _model, \
- .driver_data = (kernel_ulong_t)&_ops, \
- }
-
static const struct x86_cpu_id rapl_ids[] __initconst = {
- RAPL_CPU(INTEL_FAM6_SANDYBRIDGE, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_SANDYBRIDGE_X, rapl_defaults_core),
-
- RAPL_CPU(INTEL_FAM6_IVYBRIDGE, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_IVYBRIDGE_X, rapl_defaults_core),
-
- RAPL_CPU(INTEL_FAM6_HASWELL_CORE, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_HASWELL_ULT, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_HASWELL_GT3E, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_HASWELL_X, rapl_defaults_hsw_server),
-
- RAPL_CPU(INTEL_FAM6_BROADWELL_CORE, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_BROADWELL_GT3E, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_BROADWELL_XEON_D, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_BROADWELL_X, rapl_defaults_hsw_server),
-
- RAPL_CPU(INTEL_FAM6_SKYLAKE_DESKTOP, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_SKYLAKE_MOBILE, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_SKYLAKE_X, rapl_defaults_hsw_server),
- RAPL_CPU(INTEL_FAM6_KABYLAKE_MOBILE, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_KABYLAKE_DESKTOP, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_CANNONLAKE_MOBILE, rapl_defaults_core),
-
- RAPL_CPU(INTEL_FAM6_ATOM_SILVERMONT1, rapl_defaults_byt),
- RAPL_CPU(INTEL_FAM6_ATOM_AIRMONT, rapl_defaults_cht),
- RAPL_CPU(INTEL_FAM6_ATOM_MERRIFIELD, rapl_defaults_tng),
- RAPL_CPU(INTEL_FAM6_ATOM_MOOREFIELD, rapl_defaults_ann),
- RAPL_CPU(INTEL_FAM6_ATOM_GOLDMONT, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_ATOM_GEMINI_LAKE, rapl_defaults_core),
- RAPL_CPU(INTEL_FAM6_ATOM_DENVERTON, rapl_defaults_core),
-
- RAPL_CPU(INTEL_FAM6_XEON_PHI_KNL, rapl_defaults_hsw_server),
- RAPL_CPU(INTEL_FAM6_XEON_PHI_KNM, rapl_defaults_hsw_server),
+ INTEL_CPU_FAM6(SANDYBRIDGE, rapl_defaults_core),
+ INTEL_CPU_FAM6(SANDYBRIDGE_X, rapl_defaults_core),
+
+ INTEL_CPU_FAM6(IVYBRIDGE, rapl_defaults_core),
+ INTEL_CPU_FAM6(IVYBRIDGE_X, rapl_defaults_core),
+
+ INTEL_CPU_FAM6(HASWELL_CORE, rapl_defaults_core),
+ INTEL_CPU_FAM6(HASWELL_ULT, rapl_defaults_core),
+ INTEL_CPU_FAM6(HASWELL_GT3E, rapl_defaults_core),
+ INTEL_CPU_FAM6(HASWELL_X, rapl_defaults_hsw_server),
+
+ INTEL_CPU_FAM6(BROADWELL_CORE, rapl_defaults_core),
+ INTEL_CPU_FAM6(BROADWELL_GT3E, rapl_defaults_core),
+ INTEL_CPU_FAM6(BROADWELL_XEON_D, rapl_defaults_core),
+ INTEL_CPU_FAM6(BROADWELL_X, rapl_defaults_hsw_server),
+
+ INTEL_CPU_FAM6(SKYLAKE_DESKTOP, rapl_defaults_core),
+ INTEL_CPU_FAM6(SKYLAKE_MOBILE, rapl_defaults_core),
+ INTEL_CPU_FAM6(SKYLAKE_X, rapl_defaults_hsw_server),
+ INTEL_CPU_FAM6(KABYLAKE_MOBILE, rapl_defaults_core),
+ INTEL_CPU_FAM6(KABYLAKE_DESKTOP, rapl_defaults_core),
+ INTEL_CPU_FAM6(CANNONLAKE_MOBILE, rapl_defaults_core),
+
+ INTEL_CPU_FAM6(ATOM_SILVERMONT, rapl_defaults_byt),
+ INTEL_CPU_FAM6(ATOM_AIRMONT, rapl_defaults_cht),
+ INTEL_CPU_FAM6(ATOM_SILVERMONT_MID, rapl_defaults_tng),
+ INTEL_CPU_FAM6(ATOM_AIRMONT_MID, rapl_defaults_ann),
+ INTEL_CPU_FAM6(ATOM_GOLDMONT, rapl_defaults_core),
+ INTEL_CPU_FAM6(ATOM_GOLDMONT_PLUS, rapl_defaults_core),
+ INTEL_CPU_FAM6(ATOM_GOLDMONT_X, rapl_defaults_core),
+
+ INTEL_CPU_FAM6(XEON_PHI_KNL, rapl_defaults_hsw_server),
+ INTEL_CPU_FAM6(XEON_PHI_KNM, rapl_defaults_hsw_server),
{}
};
MODULE_DEVICE_TABLE(x86cpu, rapl_ids);
#include <linux/slab.h>
#include <linux/timekeeping.h>
+#include <linux/nospec.h>
+
#include "ptp_private.h"
static int ptp_disable_pinfunc(struct ptp_clock_info *ops,
err = -EINVAL;
break;
}
+ pin_index = array_index_nospec(pin_index, ops->n_pins);
if (mutex_lock_interruptible(&ptp->pincfg_mux))
return -ERESTARTSYS;
pd = ops->pin_config[pin_index];
err = -EINVAL;
break;
}
+ pin_index = array_index_nospec(pin_index, ops->n_pins);
if (mutex_lock_interruptible(&ptp->pincfg_mux))
return -ERESTARTSYS;
err = ptp_set_pinfunc(ptp, pin_index, pd.func, pd.chan);
and LDO regulators.
This driver can also be built as a module. If so, the module
- will be called bd71837-regulator.
+ will be called bd718x7-regulator.
config REGULATOR_BD9571MWV
tristate "ROHM BD9571MWV Regulators"
One boost output voltage is configurable and always on.
Other LDOs are used for the display module.
+config REGULATOR_LOCHNAGAR
+ tristate "Cirrus Logic Lochnagar regulator driver"
+ depends on MFD_LOCHNAGAR
+ help
+ This enables regulator support on the Cirrus Logic Lochnagar audio
+ development board.
+
config REGULATOR_LP3971
tristate "National Semiconductors LP3971 PMIC regulator driver"
depends on I2C
This driver can also be built as a module. If so, the module
will be called stm32-vrefbuf.
+config REGULATOR_STPMIC1
+ tristate "STMicroelectronics STPMIC1 PMIC Regulators"
+ depends on MFD_STPMIC1
+ help
+ This driver supports STMicroelectronics STPMIC1 PMIC voltage
+ regulators and switches. The STPMIC1 regulators supply power to
+ an application processor as well as to external system
+ peripherals such as DDR, Flash memories and system devices.
+
+ To compile this driver as a module, choose M here: the
+ module will be called stpmic1_regulator.
+
config REGULATOR_TI_ABB
tristate "TI Adaptive Body Bias on-chip LDO"
depends on ARCH_OMAP
obj-$(CONFIG_REGULATOR_AS3722) += as3722-regulator.o
obj-$(CONFIG_REGULATOR_AXP20X) += axp20x-regulator.o
obj-$(CONFIG_REGULATOR_BCM590XX) += bcm590xx-regulator.o
-obj-$(CONFIG_REGULATOR_BD718XX) += bd71837-regulator.o
+obj-$(CONFIG_REGULATOR_BD718XX) += bd718x7-regulator.o
obj-$(CONFIG_REGULATOR_BD9571MWV) += bd9571mwv-regulator.o
obj-$(CONFIG_REGULATOR_DA903X) += da903x.o
obj-$(CONFIG_REGULATOR_DA9052) += da9052-regulator.o
obj-$(CONFIG_REGULATOR_ISL6271A) += isl6271a-regulator.o
obj-$(CONFIG_REGULATOR_ISL9305) += isl9305.o
obj-$(CONFIG_REGULATOR_LM363X) += lm363x-regulator.o
+obj-$(CONFIG_REGULATOR_LOCHNAGAR) += lochnagar-regulator.o
obj-$(CONFIG_REGULATOR_LP3971) += lp3971.o
obj-$(CONFIG_REGULATOR_LP3972) += lp3972.o
obj-$(CONFIG_REGULATOR_LP872X) += lp872x.o
obj-$(CONFIG_REGULATOR_SC2731) += sc2731-regulator.o
obj-$(CONFIG_REGULATOR_SKY81452) += sky81452-regulator.o
obj-$(CONFIG_REGULATOR_STM32_VREFBUF) += stm32-vrefbuf.o
+obj-$(CONFIG_REGULATOR_STPMIC1) += stpmic1_regulator.o
obj-$(CONFIG_REGULATOR_STW481X_VMMC) += stw481x-vmmc.o
obj-$(CONFIG_REGULATOR_SY8106A) += sy8106a-regulator.o
obj-$(CONFIG_REGULATOR_TI_ABB) += ti-abb-regulator.o
* so clean up would happen at the wrong time
*/
config.ena_gpiod = gpiod_get_optional(parent_dev, "wlf,ldoena",
- GPIOD_OUT_LOW);
+ GPIOD_OUT_LOW | GPIOD_FLAGS_BIT_NONEXCLUSIVE);
if (IS_ERR(config.ena_gpiod))
return PTR_ERR(config.ena_gpiod);
/*
* AXP803/AXP813 DCDC work frequency setting has the same
* range and step as AXP22X, but at a different register.
- * Fall through to the check below.
* (See include/linux/mfd/axp20x.h)
*/
reg = AXP803_DCDC_FREQ_CTRL;
+ /* Fall through to the check below.*/
case AXP806_ID:
/*
* AXP806 also have DCDC work frequency setting register at a
*/
if (axp20x->variant == AXP806_ID)
reg = AXP806_DCDC_FREQ_CTRL;
+ /* Fall through */
case AXP221_ID:
case AXP223_ID:
case AXP809_ID:
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-// Copyright (C) 2018 ROHM Semiconductors
-// bd71837-regulator.c ROHM BD71837MWV regulator driver
-
-#include <linux/delay.h>
-#include <linux/err.h>
-#include <linux/gpio.h>
-#include <linux/interrupt.h>
-#include <linux/kernel.h>
-#include <linux/mfd/rohm-bd718x7.h>
-#include <linux/module.h>
-#include <linux/platform_device.h>
-#include <linux/regulator/driver.h>
-#include <linux/regulator/machine.h>
-#include <linux/regulator/of_regulator.h>
-#include <linux/slab.h>
-
-struct bd71837_pmic {
- struct regulator_desc descs[BD71837_REGULATOR_CNT];
- struct bd71837 *mfd;
- struct platform_device *pdev;
- struct regulator_dev *rdev[BD71837_REGULATOR_CNT];
-};
-
-/*
- * BUCK1/2/3/4
- * BUCK1RAMPRATE[1:0] BUCK1 DVS ramp rate setting
- * 00: 10.00mV/usec 10mV 1uS
- * 01: 5.00mV/usec 10mV 2uS
- * 10: 2.50mV/usec 10mV 4uS
- * 11: 1.25mV/usec 10mV 8uS
- */
-static int bd71837_buck1234_set_ramp_delay(struct regulator_dev *rdev,
- int ramp_delay)
-{
- struct bd71837_pmic *pmic = rdev_get_drvdata(rdev);
- struct bd71837 *mfd = pmic->mfd;
- int id = rdev->desc->id;
- unsigned int ramp_value = BUCK_RAMPRATE_10P00MV;
-
- dev_dbg(&pmic->pdev->dev, "Buck[%d] Set Ramp = %d\n", id + 1,
- ramp_delay);
- switch (ramp_delay) {
- case 1 ... 1250:
- ramp_value = BUCK_RAMPRATE_1P25MV;
- break;
- case 1251 ... 2500:
- ramp_value = BUCK_RAMPRATE_2P50MV;
- break;
- case 2501 ... 5000:
- ramp_value = BUCK_RAMPRATE_5P00MV;
- break;
- case 5001 ... 10000:
- ramp_value = BUCK_RAMPRATE_10P00MV;
- break;
- default:
- ramp_value = BUCK_RAMPRATE_10P00MV;
- dev_err(&pmic->pdev->dev,
- "%s: ramp_delay: %d not supported, setting 10000mV//us\n",
- rdev->desc->name, ramp_delay);
- }
-
- return regmap_update_bits(mfd->regmap, BD71837_REG_BUCK1_CTRL + id,
- BUCK_RAMPRATE_MASK, ramp_value << 6);
-}
-
-/* Bucks 1 to 4 support DVS. PWM mode is used when voltage is changed.
- * Bucks 5 to 8 and LDOs can use PFM and must be disabled when voltage
- * is changed. Hence we return -EBUSY for these if voltage is changed
- * when BUCK/LDO is enabled.
- */
-static int bd71837_set_voltage_sel_restricted(struct regulator_dev *rdev,
- unsigned int sel)
-{
- if (regulator_is_enabled_regmap(rdev))
- return -EBUSY;
-
- return regulator_set_voltage_sel_regmap(rdev, sel);
-}
-
-static struct regulator_ops bd71837_ldo_regulator_ops = {
- .enable = regulator_enable_regmap,
- .disable = regulator_disable_regmap,
- .is_enabled = regulator_is_enabled_regmap,
- .list_voltage = regulator_list_voltage_linear_range,
- .set_voltage_sel = bd71837_set_voltage_sel_restricted,
- .get_voltage_sel = regulator_get_voltage_sel_regmap,
-};
-
-static struct regulator_ops bd71837_ldo_regulator_nolinear_ops = {
- .enable = regulator_enable_regmap,
- .disable = regulator_disable_regmap,
- .is_enabled = regulator_is_enabled_regmap,
- .list_voltage = regulator_list_voltage_table,
- .set_voltage_sel = bd71837_set_voltage_sel_restricted,
- .get_voltage_sel = regulator_get_voltage_sel_regmap,
-};
-
-static struct regulator_ops bd71837_buck_regulator_ops = {
- .enable = regulator_enable_regmap,
- .disable = regulator_disable_regmap,
- .is_enabled = regulator_is_enabled_regmap,
- .list_voltage = regulator_list_voltage_linear_range,
- .set_voltage_sel = bd71837_set_voltage_sel_restricted,
- .get_voltage_sel = regulator_get_voltage_sel_regmap,
- .set_voltage_time_sel = regulator_set_voltage_time_sel,
-};
-
-static struct regulator_ops bd71837_buck_regulator_nolinear_ops = {
- .enable = regulator_enable_regmap,
- .disable = regulator_disable_regmap,
- .is_enabled = regulator_is_enabled_regmap,
- .list_voltage = regulator_list_voltage_table,
- .set_voltage_sel = bd71837_set_voltage_sel_restricted,
- .get_voltage_sel = regulator_get_voltage_sel_regmap,
- .set_voltage_time_sel = regulator_set_voltage_time_sel,
-};
-
-static struct regulator_ops bd71837_buck1234_regulator_ops = {
- .enable = regulator_enable_regmap,
- .disable = regulator_disable_regmap,
- .is_enabled = regulator_is_enabled_regmap,
- .list_voltage = regulator_list_voltage_linear_range,
- .set_voltage_sel = regulator_set_voltage_sel_regmap,
- .get_voltage_sel = regulator_get_voltage_sel_regmap,
- .set_voltage_time_sel = regulator_set_voltage_time_sel,
- .set_ramp_delay = bd71837_buck1234_set_ramp_delay,
-};
-
-/*
- * BUCK1/2/3/4
- * 0.70 to 1.30V (10mV step)
- */
-static const struct regulator_linear_range bd71837_buck1234_voltage_ranges[] = {
- REGULATOR_LINEAR_RANGE(700000, 0x00, 0x3C, 10000),
- REGULATOR_LINEAR_RANGE(1300000, 0x3D, 0x3F, 0),
-};
-
-/*
- * BUCK5
- * 0.9V to 1.35V ()
- */
-static const struct regulator_linear_range bd71837_buck5_voltage_ranges[] = {
- REGULATOR_LINEAR_RANGE(700000, 0x00, 0x03, 100000),
- REGULATOR_LINEAR_RANGE(1050000, 0x04, 0x05, 50000),
- REGULATOR_LINEAR_RANGE(1200000, 0x06, 0x07, 150000),
-};
-
-/*
- * BUCK6
- * 3.0V to 3.3V (step 100mV)
- */
-static const struct regulator_linear_range bd71837_buck6_voltage_ranges[] = {
- REGULATOR_LINEAR_RANGE(3000000, 0x00, 0x03, 100000),
-};
-
-/*
- * BUCK7
- * 000 = 1.605V
- * 001 = 1.695V
- * 010 = 1.755V
- * 011 = 1.8V (Initial)
- * 100 = 1.845V
- * 101 = 1.905V
- * 110 = 1.95V
- * 111 = 1.995V
- */
-static const unsigned int buck_7_volts[] = {
- 1605000, 1695000, 1755000, 1800000, 1845000, 1905000, 1950000, 1995000
-};
-
-/*
- * BUCK8
- * 0.8V to 1.40V (step 10mV)
- */
-static const struct regulator_linear_range bd71837_buck8_voltage_ranges[] = {
- REGULATOR_LINEAR_RANGE(800000, 0x00, 0x3C, 10000),
- REGULATOR_LINEAR_RANGE(1400000, 0x3D, 0x3F, 0),
-};
-
-/*
- * LDO1
- * 3.0 to 3.3V (100mV step)
- */
-static const struct regulator_linear_range bd71837_ldo1_voltage_ranges[] = {
- REGULATOR_LINEAR_RANGE(3000000, 0x00, 0x03, 100000),
-};
-
-/*
- * LDO2
- * 0.8 or 0.9V
- */
-static const unsigned int ldo_2_volts[] = {
- 900000, 800000
-};
-
-/*
- * LDO3
- * 1.8 to 3.3V (100mV step)
- */
-static const struct regulator_linear_range bd71837_ldo3_voltage_ranges[] = {
- REGULATOR_LINEAR_RANGE(1800000, 0x00, 0x0F, 100000),
-};
-
-/*
- * LDO4
- * 0.9 to 1.8V (100mV step)
- */
-static const struct regulator_linear_range bd71837_ldo4_voltage_ranges[] = {
- REGULATOR_LINEAR_RANGE(900000, 0x00, 0x09, 100000),
- REGULATOR_LINEAR_RANGE(1800000, 0x0A, 0x0F, 0),
-};
-
-/*
- * LDO5
- * 1.8 to 3.3V (100mV step)
- */
-static const struct regulator_linear_range bd71837_ldo5_voltage_ranges[] = {
- REGULATOR_LINEAR_RANGE(1800000, 0x00, 0x0F, 100000),
-};
-
-/*
- * LDO6
- * 0.9 to 1.8V (100mV step)
- */
-static const struct regulator_linear_range bd71837_ldo6_voltage_ranges[] = {
- REGULATOR_LINEAR_RANGE(900000, 0x00, 0x09, 100000),
- REGULATOR_LINEAR_RANGE(1800000, 0x0A, 0x0F, 0),
-};
-
-/*
- * LDO7
- * 1.8 to 3.3V (100mV step)
- */
-static const struct regulator_linear_range bd71837_ldo7_voltage_ranges[] = {
- REGULATOR_LINEAR_RANGE(1800000, 0x00, 0x0F, 100000),
-};
-
-static const struct regulator_desc bd71837_regulators[] = {
- {
- .name = "buck1",
- .of_match = of_match_ptr("BUCK1"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_BUCK1,
- .ops = &bd71837_buck1234_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_BUCK1_VOLTAGE_NUM,
- .linear_ranges = bd71837_buck1234_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_buck1234_voltage_ranges),
- .vsel_reg = BD71837_REG_BUCK1_VOLT_RUN,
- .vsel_mask = BUCK1_RUN_MASK,
- .enable_reg = BD71837_REG_BUCK1_CTRL,
- .enable_mask = BD71837_BUCK_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "buck2",
- .of_match = of_match_ptr("BUCK2"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_BUCK2,
- .ops = &bd71837_buck1234_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_BUCK2_VOLTAGE_NUM,
- .linear_ranges = bd71837_buck1234_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_buck1234_voltage_ranges),
- .vsel_reg = BD71837_REG_BUCK2_VOLT_RUN,
- .vsel_mask = BUCK2_RUN_MASK,
- .enable_reg = BD71837_REG_BUCK2_CTRL,
- .enable_mask = BD71837_BUCK_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "buck3",
- .of_match = of_match_ptr("BUCK3"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_BUCK3,
- .ops = &bd71837_buck1234_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_BUCK3_VOLTAGE_NUM,
- .linear_ranges = bd71837_buck1234_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_buck1234_voltage_ranges),
- .vsel_reg = BD71837_REG_BUCK3_VOLT_RUN,
- .vsel_mask = BUCK3_RUN_MASK,
- .enable_reg = BD71837_REG_BUCK3_CTRL,
- .enable_mask = BD71837_BUCK_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "buck4",
- .of_match = of_match_ptr("BUCK4"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_BUCK4,
- .ops = &bd71837_buck1234_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_BUCK4_VOLTAGE_NUM,
- .linear_ranges = bd71837_buck1234_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_buck1234_voltage_ranges),
- .vsel_reg = BD71837_REG_BUCK4_VOLT_RUN,
- .vsel_mask = BUCK4_RUN_MASK,
- .enable_reg = BD71837_REG_BUCK4_CTRL,
- .enable_mask = BD71837_BUCK_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "buck5",
- .of_match = of_match_ptr("BUCK5"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_BUCK5,
- .ops = &bd71837_buck_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_BUCK5_VOLTAGE_NUM,
- .linear_ranges = bd71837_buck5_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_buck5_voltage_ranges),
- .vsel_reg = BD71837_REG_BUCK5_VOLT,
- .vsel_mask = BUCK5_MASK,
- .enable_reg = BD71837_REG_BUCK5_CTRL,
- .enable_mask = BD71837_BUCK_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "buck6",
- .of_match = of_match_ptr("BUCK6"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_BUCK6,
- .ops = &bd71837_buck_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_BUCK6_VOLTAGE_NUM,
- .linear_ranges = bd71837_buck6_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_buck6_voltage_ranges),
- .vsel_reg = BD71837_REG_BUCK6_VOLT,
- .vsel_mask = BUCK6_MASK,
- .enable_reg = BD71837_REG_BUCK6_CTRL,
- .enable_mask = BD71837_BUCK_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "buck7",
- .of_match = of_match_ptr("BUCK7"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_BUCK7,
- .ops = &bd71837_buck_regulator_nolinear_ops,
- .type = REGULATOR_VOLTAGE,
- .volt_table = &buck_7_volts[0],
- .n_voltages = ARRAY_SIZE(buck_7_volts),
- .vsel_reg = BD71837_REG_BUCK7_VOLT,
- .vsel_mask = BUCK7_MASK,
- .enable_reg = BD71837_REG_BUCK7_CTRL,
- .enable_mask = BD71837_BUCK_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "buck8",
- .of_match = of_match_ptr("BUCK8"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_BUCK8,
- .ops = &bd71837_buck_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_BUCK8_VOLTAGE_NUM,
- .linear_ranges = bd71837_buck8_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_buck8_voltage_ranges),
- .vsel_reg = BD71837_REG_BUCK8_VOLT,
- .vsel_mask = BUCK8_MASK,
- .enable_reg = BD71837_REG_BUCK8_CTRL,
- .enable_mask = BD71837_BUCK_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "ldo1",
- .of_match = of_match_ptr("LDO1"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_LDO1,
- .ops = &bd71837_ldo_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_LDO1_VOLTAGE_NUM,
- .linear_ranges = bd71837_ldo1_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_ldo1_voltage_ranges),
- .vsel_reg = BD71837_REG_LDO1_VOLT,
- .vsel_mask = LDO1_MASK,
- .enable_reg = BD71837_REG_LDO1_VOLT,
- .enable_mask = BD71837_LDO_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "ldo2",
- .of_match = of_match_ptr("LDO2"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_LDO2,
- .ops = &bd71837_ldo_regulator_nolinear_ops,
- .type = REGULATOR_VOLTAGE,
- .volt_table = &ldo_2_volts[0],
- .vsel_reg = BD71837_REG_LDO2_VOLT,
- .vsel_mask = LDO2_MASK,
- .n_voltages = ARRAY_SIZE(ldo_2_volts),
- .n_voltages = BD71837_LDO2_VOLTAGE_NUM,
- .enable_reg = BD71837_REG_LDO2_VOLT,
- .enable_mask = BD71837_LDO_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "ldo3",
- .of_match = of_match_ptr("LDO3"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_LDO3,
- .ops = &bd71837_ldo_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_LDO3_VOLTAGE_NUM,
- .linear_ranges = bd71837_ldo3_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_ldo3_voltage_ranges),
- .vsel_reg = BD71837_REG_LDO3_VOLT,
- .vsel_mask = LDO3_MASK,
- .enable_reg = BD71837_REG_LDO3_VOLT,
- .enable_mask = BD71837_LDO_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "ldo4",
- .of_match = of_match_ptr("LDO4"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_LDO4,
- .ops = &bd71837_ldo_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_LDO4_VOLTAGE_NUM,
- .linear_ranges = bd71837_ldo4_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_ldo4_voltage_ranges),
- .vsel_reg = BD71837_REG_LDO4_VOLT,
- .vsel_mask = LDO4_MASK,
- .enable_reg = BD71837_REG_LDO4_VOLT,
- .enable_mask = BD71837_LDO_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "ldo5",
- .of_match = of_match_ptr("LDO5"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_LDO5,
- .ops = &bd71837_ldo_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_LDO5_VOLTAGE_NUM,
- .linear_ranges = bd71837_ldo5_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_ldo5_voltage_ranges),
- /* LDO5 is supplied by buck6 */
- .supply_name = "buck6",
- .vsel_reg = BD71837_REG_LDO5_VOLT,
- .vsel_mask = LDO5_MASK,
- .enable_reg = BD71837_REG_LDO5_VOLT,
- .enable_mask = BD71837_LDO_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "ldo6",
- .of_match = of_match_ptr("LDO6"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_LDO6,
- .ops = &bd71837_ldo_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_LDO6_VOLTAGE_NUM,
- .linear_ranges = bd71837_ldo6_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_ldo6_voltage_ranges),
- /* LDO6 is supplied by buck7 */
- .supply_name = "buck7",
- .vsel_reg = BD71837_REG_LDO6_VOLT,
- .vsel_mask = LDO6_MASK,
- .enable_reg = BD71837_REG_LDO6_VOLT,
- .enable_mask = BD71837_LDO_EN,
- .owner = THIS_MODULE,
- },
- {
- .name = "ldo7",
- .of_match = of_match_ptr("LDO7"),
- .regulators_node = of_match_ptr("regulators"),
- .id = BD71837_LDO7,
- .ops = &bd71837_ldo_regulator_ops,
- .type = REGULATOR_VOLTAGE,
- .n_voltages = BD71837_LDO7_VOLTAGE_NUM,
- .linear_ranges = bd71837_ldo7_voltage_ranges,
- .n_linear_ranges = ARRAY_SIZE(bd71837_ldo7_voltage_ranges),
- .vsel_reg = BD71837_REG_LDO7_VOLT,
- .vsel_mask = LDO7_MASK,
- .enable_reg = BD71837_REG_LDO7_VOLT,
- .enable_mask = BD71837_LDO_EN,
- .owner = THIS_MODULE,
- },
-};
-
-struct reg_init {
- unsigned int reg;
- unsigned int mask;
-};
-
-static int bd71837_probe(struct platform_device *pdev)
-{
- struct bd71837_pmic *pmic;
- struct regulator_config config = { 0 };
- struct reg_init pmic_regulator_inits[] = {
- {
- .reg = BD71837_REG_BUCK1_CTRL,
- .mask = BD71837_BUCK_SEL,
- }, {
- .reg = BD71837_REG_BUCK2_CTRL,
- .mask = BD71837_BUCK_SEL,
- }, {
- .reg = BD71837_REG_BUCK3_CTRL,
- .mask = BD71837_BUCK_SEL,
- }, {
- .reg = BD71837_REG_BUCK4_CTRL,
- .mask = BD71837_BUCK_SEL,
- }, {
- .reg = BD71837_REG_BUCK5_CTRL,
- .mask = BD71837_BUCK_SEL,
- }, {
- .reg = BD71837_REG_BUCK6_CTRL,
- .mask = BD71837_BUCK_SEL,
- }, {
- .reg = BD71837_REG_BUCK7_CTRL,
- .mask = BD71837_BUCK_SEL,
- }, {
- .reg = BD71837_REG_BUCK8_CTRL,
- .mask = BD71837_BUCK_SEL,
- }, {
- .reg = BD71837_REG_LDO1_VOLT,
- .mask = BD71837_LDO_SEL,
- }, {
- .reg = BD71837_REG_LDO2_VOLT,
- .mask = BD71837_LDO_SEL,
- }, {
- .reg = BD71837_REG_LDO3_VOLT,
- .mask = BD71837_LDO_SEL,
- }, {
- .reg = BD71837_REG_LDO4_VOLT,
- .mask = BD71837_LDO_SEL,
- }, {
- .reg = BD71837_REG_LDO5_VOLT,
- .mask = BD71837_LDO_SEL,
- }, {
- .reg = BD71837_REG_LDO6_VOLT,
- .mask = BD71837_LDO_SEL,
- }, {
- .reg = BD71837_REG_LDO7_VOLT,
- .mask = BD71837_LDO_SEL,
- }
- };
-
- int i, err;
-
- pmic = devm_kzalloc(&pdev->dev, sizeof(*pmic), GFP_KERNEL);
- if (!pmic)
- return -ENOMEM;
-
- memcpy(pmic->descs, bd71837_regulators, sizeof(pmic->descs));
-
- pmic->pdev = pdev;
- pmic->mfd = dev_get_drvdata(pdev->dev.parent);
-
- if (!pmic->mfd) {
- dev_err(&pdev->dev, "No MFD driver data\n");
- err = -EINVAL;
- goto err;
- }
- platform_set_drvdata(pdev, pmic);
-
- /* Register LOCK release */
- err = regmap_update_bits(pmic->mfd->regmap, BD71837_REG_REGLOCK,
- (REGLOCK_PWRSEQ | REGLOCK_VREG), 0);
- if (err) {
- dev_err(&pmic->pdev->dev, "Failed to unlock PMIC (%d)\n", err);
- goto err;
- } else {
- dev_dbg(&pmic->pdev->dev, "Unlocked lock register 0x%x\n",
- BD71837_REG_REGLOCK);
- }
-
- /*
- * There is a HW quirk in BD71837. The shutdown sequence timings for
- * bucks/LDOs which are controlled via register interface are changed.
- * At PMIC poweroff the voltage for BUCK6/7 is cut immediately at the
- * beginning of shut-down sequence. As bucks 6 and 7 are parent
- * supplies for LDO5 and LDO6 - this causes LDO5/6 voltage
- * monitoring to errorneously detect under voltage and force PMIC to
- * emergency state instead of poweroff. In order to avoid this we
- * disable voltage monitoring for LDO5 and LDO6
- */
- err = regmap_update_bits(pmic->mfd->regmap, BD718XX_REG_MVRFLTMASK2,
- BD718XX_LDO5_VRMON80 | BD718XX_LDO6_VRMON80,
- BD718XX_LDO5_VRMON80 | BD718XX_LDO6_VRMON80);
- if (err) {
- dev_err(&pmic->pdev->dev,
- "Failed to disable voltage monitoring\n");
- goto err;
- }
-
- for (i = 0; i < ARRAY_SIZE(pmic_regulator_inits); i++) {
-
- struct regulator_desc *desc;
- struct regulator_dev *rdev;
-
- desc = &pmic->descs[i];
-
- config.dev = pdev->dev.parent;
- config.driver_data = pmic;
- config.regmap = pmic->mfd->regmap;
-
- rdev = devm_regulator_register(&pdev->dev, desc, &config);
- if (IS_ERR(rdev)) {
- dev_err(pmic->mfd->dev,
- "failed to register %s regulator\n",
- desc->name);
- err = PTR_ERR(rdev);
- goto err;
- }
- /* Regulator register gets the regulator constraints and
- * applies them (set_machine_constraints). This should have
- * turned the control register(s) to correct values and we
- * can now switch the control from PMIC state machine to the
- * register interface
- */
- err = regmap_update_bits(pmic->mfd->regmap,
- pmic_regulator_inits[i].reg,
- pmic_regulator_inits[i].mask,
- 0xFFFFFFFF);
- if (err) {
- dev_err(&pmic->pdev->dev,
- "Failed to write BUCK/LDO SEL bit for (%s)\n",
- desc->name);
- goto err;
- }
-
- pmic->rdev[i] = rdev;
- }
-
-err:
- return err;
-}
-
-static struct platform_driver bd71837_regulator = {
- .driver = {
- .name = "bd71837-pmic",
- },
- .probe = bd71837_probe,
-};
-
-module_platform_driver(bd71837_regulator);
-
-MODULE_AUTHOR("Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>");
-MODULE_DESCRIPTION("BD71837 voltage regulator driver");
-MODULE_LICENSE("GPL");
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 ROHM Semiconductors
+// bd71837-regulator.c ROHM BD71837MWV/BD71847MWV regulator driver
+
+#include <linux/delay.h>
+#include <linux/err.h>
+#include <linux/gpio.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/mfd/rohm-bd718x7.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/regulator/driver.h>
+#include <linux/regulator/machine.h>
+#include <linux/regulator/of_regulator.h>
+#include <linux/slab.h>
+
+/*
+ * BUCK1/2/3/4
+ * BUCK1RAMPRATE[1:0] BUCK1 DVS ramp rate setting
+ * 00: 10.00mV/usec 10mV 1uS
+ * 01: 5.00mV/usec 10mV 2uS
+ * 10: 2.50mV/usec 10mV 4uS
+ * 11: 1.25mV/usec 10mV 8uS
+ */
+static int bd718xx_buck1234_set_ramp_delay(struct regulator_dev *rdev,
+ int ramp_delay)
+{
+ int id = rdev->desc->id;
+ unsigned int ramp_value = BUCK_RAMPRATE_10P00MV;
+
+ dev_dbg(&rdev->dev, "Buck[%d] Set Ramp = %d\n", id + 1,
+ ramp_delay);
+ switch (ramp_delay) {
+ case 1 ... 1250:
+ ramp_value = BUCK_RAMPRATE_1P25MV;
+ break;
+ case 1251 ... 2500:
+ ramp_value = BUCK_RAMPRATE_2P50MV;
+ break;
+ case 2501 ... 5000:
+ ramp_value = BUCK_RAMPRATE_5P00MV;
+ break;
+ case 5001 ... 10000:
+ ramp_value = BUCK_RAMPRATE_10P00MV;
+ break;
+ default:
+ ramp_value = BUCK_RAMPRATE_10P00MV;
+ dev_err(&rdev->dev,
+ "%s: ramp_delay: %d not supported, setting 10000mV//us\n",
+ rdev->desc->name, ramp_delay);
+ }
+
+ return regmap_update_bits(rdev->regmap, BD718XX_REG_BUCK1_CTRL + id,
+ BUCK_RAMPRATE_MASK, ramp_value << 6);
+}
+
+/* Bucks 1 to 4 support DVS. PWM mode is used when voltage is changed.
+ * Bucks 5 to 8 and LDOs can use PFM and must be disabled when voltage
+ * is changed. Hence we return -EBUSY for these if voltage is changed
+ * when BUCK/LDO is enabled.
+ */
+static int bd718xx_set_voltage_sel_restricted(struct regulator_dev *rdev,
+ unsigned int sel)
+{
+ if (regulator_is_enabled_regmap(rdev))
+ return -EBUSY;
+
+ return regulator_set_voltage_sel_regmap(rdev, sel);
+}
+
+static int bd718xx_set_voltage_sel_pickable_restricted(
+ struct regulator_dev *rdev, unsigned int sel)
+{
+ if (regulator_is_enabled_regmap(rdev))
+ return -EBUSY;
+
+ return regulator_set_voltage_sel_pickable_regmap(rdev, sel);
+}
+
+static struct regulator_ops bd718xx_pickable_range_ldo_ops = {
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+ .list_voltage = regulator_list_voltage_pickable_linear_range,
+ .set_voltage_sel = bd718xx_set_voltage_sel_pickable_restricted,
+ .get_voltage_sel = regulator_get_voltage_sel_pickable_regmap,
+};
+
+static struct regulator_ops bd718xx_pickable_range_buck_ops = {
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+ .list_voltage = regulator_list_voltage_pickable_linear_range,
+ .set_voltage_sel = bd718xx_set_voltage_sel_pickable_restricted,
+ .get_voltage_sel = regulator_get_voltage_sel_pickable_regmap,
+ .set_voltage_time_sel = regulator_set_voltage_time_sel,
+};
+
+static struct regulator_ops bd718xx_ldo_regulator_ops = {
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+ .list_voltage = regulator_list_voltage_linear_range,
+ .set_voltage_sel = bd718xx_set_voltage_sel_restricted,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+};
+
+static struct regulator_ops bd718xx_ldo_regulator_nolinear_ops = {
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+ .list_voltage = regulator_list_voltage_table,
+ .set_voltage_sel = bd718xx_set_voltage_sel_restricted,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+};
+
+static struct regulator_ops bd718xx_buck_regulator_ops = {
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+ .list_voltage = regulator_list_voltage_linear_range,
+ .set_voltage_sel = bd718xx_set_voltage_sel_restricted,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .set_voltage_time_sel = regulator_set_voltage_time_sel,
+};
+
+static struct regulator_ops bd718xx_buck_regulator_nolinear_ops = {
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+ .list_voltage = regulator_list_voltage_table,
+ .set_voltage_sel = bd718xx_set_voltage_sel_restricted,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .set_voltage_time_sel = regulator_set_voltage_time_sel,
+};
+
+static struct regulator_ops bd718xx_dvs_buck_regulator_ops = {
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+ .list_voltage = regulator_list_voltage_linear_range,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .set_voltage_time_sel = regulator_set_voltage_time_sel,
+ .set_ramp_delay = bd718xx_buck1234_set_ramp_delay,
+};
+
+/*
+ * BD71837 BUCK1/2/3/4
+ * BD71847 BUCK1/2
+ * 0.70 to 1.30V (10mV step)
+ */
+static const struct regulator_linear_range bd718xx_dvs_buck_volts[] = {
+ REGULATOR_LINEAR_RANGE(700000, 0x00, 0x3C, 10000),
+ REGULATOR_LINEAR_RANGE(1300000, 0x3D, 0x3F, 0),
+};
+
+/*
+ * BD71837 BUCK5
+ * 0.7V to 1.35V (range 0)
+ * and
+ * 0.675 to 1.325 (range 1)
+ */
+static const struct regulator_linear_range bd71837_buck5_volts[] = {
+ /* Ranges when VOLT_SEL bit is 0 */
+ REGULATOR_LINEAR_RANGE(700000, 0x00, 0x03, 100000),
+ REGULATOR_LINEAR_RANGE(1050000, 0x04, 0x05, 50000),
+ REGULATOR_LINEAR_RANGE(1200000, 0x06, 0x07, 150000),
+ /* Ranges when VOLT_SEL bit is 1 */
+ REGULATOR_LINEAR_RANGE(675000, 0x0, 0x3, 100000),
+ REGULATOR_LINEAR_RANGE(1025000, 0x4, 0x5, 50000),
+ REGULATOR_LINEAR_RANGE(1175000, 0x6, 0x7, 150000),
+};
+
+/*
+ * Range selector for first 3 linear ranges is 0x0
+ * and 0x1 for last 3 ranges.
+ */
+static const unsigned int bd71837_buck5_volt_range_sel[] = {
+ 0x0, 0x0, 0x0, 0x80, 0x80, 0x80
+};
+
+/*
+ * BD71847 BUCK3
+ */
+static const struct regulator_linear_range bd71847_buck3_volts[] = {
+ /* Ranges when VOLT_SEL bits are 00 */
+ REGULATOR_LINEAR_RANGE(700000, 0x00, 0x03, 100000),
+ REGULATOR_LINEAR_RANGE(1050000, 0x04, 0x05, 50000),
+ REGULATOR_LINEAR_RANGE(1200000, 0x06, 0x07, 150000),
+ /* Ranges when VOLT_SEL bits are 01 */
+ REGULATOR_LINEAR_RANGE(550000, 0x0, 0x7, 50000),
+ /* Ranges when VOLT_SEL bits are 11 */
+ REGULATOR_LINEAR_RANGE(675000, 0x0, 0x3, 100000),
+ REGULATOR_LINEAR_RANGE(1025000, 0x4, 0x5, 50000),
+ REGULATOR_LINEAR_RANGE(1175000, 0x6, 0x7, 150000),
+};
+
+static const unsigned int bd71847_buck3_volt_range_sel[] = {
+ 0x0, 0x0, 0x0, 0x40, 0x80, 0x80, 0x80
+};
+
+static const struct regulator_linear_range bd71847_buck4_volts[] = {
+ REGULATOR_LINEAR_RANGE(3000000, 0x00, 0x03, 100000),
+ REGULATOR_LINEAR_RANGE(2600000, 0x00, 0x03, 100000),
+};
+
+static const unsigned int bd71847_buck4_volt_range_sel[] = { 0x0, 0x40 };
+
+/*
+ * BUCK6
+ * 3.0V to 3.3V (step 100mV)
+ */
+static const struct regulator_linear_range bd71837_buck6_volts[] = {
+ REGULATOR_LINEAR_RANGE(3000000, 0x00, 0x03, 100000),
+};
+
+/*
+ * BD71837 BUCK7
+ * BD71847 BUCK5
+ * 000 = 1.605V
+ * 001 = 1.695V
+ * 010 = 1.755V
+ * 011 = 1.8V (Initial)
+ * 100 = 1.845V
+ * 101 = 1.905V
+ * 110 = 1.95V
+ * 111 = 1.995V
+ */
+static const unsigned int bd718xx_3rd_nodvs_buck_volts[] = {
+ 1605000, 1695000, 1755000, 1800000, 1845000, 1905000, 1950000, 1995000
+};
+
+/*
+ * BUCK8
+ * 0.8V to 1.40V (step 10mV)
+ */
+static const struct regulator_linear_range bd718xx_4th_nodvs_buck_volts[] = {
+ REGULATOR_LINEAR_RANGE(800000, 0x00, 0x3C, 10000),
+};
+
+/*
+ * LDO1
+ * 3.0 to 3.3V (100mV step)
+ */
+static const struct regulator_linear_range bd718xx_ldo1_volts[] = {
+ REGULATOR_LINEAR_RANGE(3000000, 0x00, 0x03, 100000),
+ REGULATOR_LINEAR_RANGE(1600000, 0x00, 0x03, 100000),
+};
+
+static const unsigned int bd718xx_ldo1_volt_range_sel[] = { 0x0, 0x20 };
+
+/*
+ * LDO2
+ * 0.8 or 0.9V
+ */
+static const unsigned int ldo_2_volts[] = {
+ 900000, 800000
+};
+
+/*
+ * LDO3
+ * 1.8 to 3.3V (100mV step)
+ */
+static const struct regulator_linear_range bd718xx_ldo3_volts[] = {
+ REGULATOR_LINEAR_RANGE(1800000, 0x00, 0x0F, 100000),
+};
+
+/*
+ * LDO4
+ * 0.9 to 1.8V (100mV step)
+ */
+static const struct regulator_linear_range bd718xx_ldo4_volts[] = {
+ REGULATOR_LINEAR_RANGE(900000, 0x00, 0x09, 100000),
+};
+
+/*
+ * LDO5 for BD71837
+ * 1.8 to 3.3V (100mV step)
+ */
+static const struct regulator_linear_range bd71837_ldo5_volts[] = {
+ REGULATOR_LINEAR_RANGE(1800000, 0x00, 0x0F, 100000),
+};
+
+/*
+ * LDO5 for BD71837
+ * 1.8 to 3.3V (100mV step)
+ */
+static const struct regulator_linear_range bd71847_ldo5_volts[] = {
+ REGULATOR_LINEAR_RANGE(1800000, 0x00, 0x0F, 100000),
+ REGULATOR_LINEAR_RANGE(800000, 0x00, 0x0F, 100000),
+};
+
+static const unsigned int bd71847_ldo5_volt_range_sel[] = { 0x0, 0x20 };
+
+/*
+ * LDO6
+ * 0.9 to 1.8V (100mV step)
+ */
+static const struct regulator_linear_range bd718xx_ldo6_volts[] = {
+ REGULATOR_LINEAR_RANGE(900000, 0x00, 0x09, 100000),
+};
+
+/*
+ * LDO7
+ * 1.8 to 3.3V (100mV step)
+ */
+static const struct regulator_linear_range bd71837_ldo7_volts[] = {
+ REGULATOR_LINEAR_RANGE(1800000, 0x00, 0x0F, 100000),
+};
+
+struct reg_init {
+ unsigned int reg;
+ unsigned int mask;
+ unsigned int val;
+};
+struct bd718xx_regulator_data {
+ struct regulator_desc desc;
+ const struct reg_init init;
+ const struct reg_init *additional_inits;
+ int additional_init_amnt;
+};
+
+/*
+ * There is a HW quirk in BD71837. The shutdown sequence timings for
+ * bucks/LDOs which are controlled via register interface are changed.
+ * At PMIC poweroff the voltage for BUCK6/7 is cut immediately at the
+ * beginning of shut-down sequence. As bucks 6 and 7 are parent
+ * supplies for LDO5 and LDO6 - this causes LDO5/6 voltage
+ * monitoring to errorneously detect under voltage and force PMIC to
+ * emergency state instead of poweroff. In order to avoid this we
+ * disable voltage monitoring for LDO5 and LDO6
+ */
+static const struct reg_init bd71837_ldo5_inits[] = {
+ {
+ .reg = BD718XX_REG_MVRFLTMASK2,
+ .mask = BD718XX_LDO5_VRMON80,
+ .val = BD718XX_LDO5_VRMON80,
+ },
+};
+
+static const struct reg_init bd71837_ldo6_inits[] = {
+ {
+ .reg = BD718XX_REG_MVRFLTMASK2,
+ .mask = BD718XX_LDO6_VRMON80,
+ .val = BD718XX_LDO6_VRMON80,
+ },
+};
+
+static const struct bd718xx_regulator_data bd71847_regulators[] = {
+ {
+ .desc = {
+ .name = "buck1",
+ .of_match = of_match_ptr("BUCK1"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK1,
+ .ops = &bd718xx_dvs_buck_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_DVS_BUCK_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_dvs_buck_volts,
+ .n_linear_ranges =
+ ARRAY_SIZE(bd718xx_dvs_buck_volts),
+ .vsel_reg = BD718XX_REG_BUCK1_VOLT_RUN,
+ .vsel_mask = DVS_BUCK_RUN_MASK,
+ .enable_reg = BD718XX_REG_BUCK1_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_BUCK1_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "buck2",
+ .of_match = of_match_ptr("BUCK2"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK2,
+ .ops = &bd718xx_dvs_buck_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_DVS_BUCK_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_dvs_buck_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_dvs_buck_volts),
+ .vsel_reg = BD718XX_REG_BUCK2_VOLT_RUN,
+ .vsel_mask = DVS_BUCK_RUN_MASK,
+ .enable_reg = BD718XX_REG_BUCK2_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_BUCK2_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "buck3",
+ .of_match = of_match_ptr("BUCK3"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK3,
+ .ops = &bd718xx_pickable_range_buck_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD71847_BUCK3_VOLTAGE_NUM,
+ .linear_ranges = bd71847_buck3_volts,
+ .n_linear_ranges =
+ ARRAY_SIZE(bd71847_buck3_volts),
+ .vsel_reg = BD718XX_REG_1ST_NODVS_BUCK_VOLT,
+ .vsel_mask = BD718XX_1ST_NODVS_BUCK_MASK,
+ .vsel_range_reg = BD718XX_REG_1ST_NODVS_BUCK_VOLT,
+ .vsel_range_mask = BD71847_BUCK3_RANGE_MASK,
+ .linear_range_selectors = bd71847_buck3_volt_range_sel,
+ .enable_reg = BD718XX_REG_1ST_NODVS_BUCK_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_1ST_NODVS_BUCK_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "buck4",
+ .of_match = of_match_ptr("BUCK4"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK4,
+ .ops = &bd718xx_pickable_range_buck_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD71847_BUCK4_VOLTAGE_NUM,
+ .linear_ranges = bd71847_buck4_volts,
+ .n_linear_ranges =
+ ARRAY_SIZE(bd71847_buck4_volts),
+ .enable_reg = BD718XX_REG_2ND_NODVS_BUCK_CTRL,
+ .vsel_reg = BD718XX_REG_2ND_NODVS_BUCK_VOLT,
+ .vsel_mask = BD71847_BUCK4_MASK,
+ .vsel_range_reg = BD718XX_REG_2ND_NODVS_BUCK_VOLT,
+ .vsel_range_mask = BD71847_BUCK4_RANGE_MASK,
+ .linear_range_selectors = bd71847_buck4_volt_range_sel,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_2ND_NODVS_BUCK_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "buck5",
+ .of_match = of_match_ptr("BUCK5"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK5,
+ .ops = &bd718xx_buck_regulator_nolinear_ops,
+ .type = REGULATOR_VOLTAGE,
+ .volt_table = &bd718xx_3rd_nodvs_buck_volts[0],
+ .n_voltages = ARRAY_SIZE(bd718xx_3rd_nodvs_buck_volts),
+ .vsel_reg = BD718XX_REG_3RD_NODVS_BUCK_VOLT,
+ .vsel_mask = BD718XX_3RD_NODVS_BUCK_MASK,
+ .enable_reg = BD718XX_REG_3RD_NODVS_BUCK_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_3RD_NODVS_BUCK_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "buck6",
+ .of_match = of_match_ptr("BUCK6"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK6,
+ .ops = &bd718xx_buck_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_4TH_NODVS_BUCK_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_4th_nodvs_buck_volts,
+ .n_linear_ranges =
+ ARRAY_SIZE(bd718xx_4th_nodvs_buck_volts),
+ .vsel_reg = BD718XX_REG_4TH_NODVS_BUCK_VOLT,
+ .vsel_mask = BD718XX_4TH_NODVS_BUCK_MASK,
+ .enable_reg = BD718XX_REG_4TH_NODVS_BUCK_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_4TH_NODVS_BUCK_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "ldo1",
+ .of_match = of_match_ptr("LDO1"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO1,
+ .ops = &bd718xx_pickable_range_ldo_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_LDO1_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_ldo1_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_ldo1_volts),
+ .vsel_reg = BD718XX_REG_LDO1_VOLT,
+ .vsel_mask = BD718XX_LDO1_MASK,
+ .vsel_range_reg = BD718XX_REG_LDO1_VOLT,
+ .vsel_range_mask = BD718XX_LDO1_RANGE_MASK,
+ .linear_range_selectors = bd718xx_ldo1_volt_range_sel,
+ .enable_reg = BD718XX_REG_LDO1_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_LDO1_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "ldo2",
+ .of_match = of_match_ptr("LDO2"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO2,
+ .ops = &bd718xx_ldo_regulator_nolinear_ops,
+ .type = REGULATOR_VOLTAGE,
+ .volt_table = &ldo_2_volts[0],
+ .vsel_reg = BD718XX_REG_LDO2_VOLT,
+ .vsel_mask = BD718XX_LDO2_MASK,
+ .n_voltages = ARRAY_SIZE(ldo_2_volts),
+ .enable_reg = BD718XX_REG_LDO2_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_LDO2_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "ldo3",
+ .of_match = of_match_ptr("LDO3"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO3,
+ .ops = &bd718xx_ldo_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_LDO3_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_ldo3_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_ldo3_volts),
+ .vsel_reg = BD718XX_REG_LDO3_VOLT,
+ .vsel_mask = BD718XX_LDO3_MASK,
+ .enable_reg = BD718XX_REG_LDO3_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_LDO3_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "ldo4",
+ .of_match = of_match_ptr("LDO4"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO4,
+ .ops = &bd718xx_ldo_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_LDO4_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_ldo4_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_ldo4_volts),
+ .vsel_reg = BD718XX_REG_LDO4_VOLT,
+ .vsel_mask = BD718XX_LDO4_MASK,
+ .enable_reg = BD718XX_REG_LDO4_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_LDO4_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "ldo5",
+ .of_match = of_match_ptr("LDO5"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO5,
+ .ops = &bd718xx_pickable_range_ldo_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD71847_LDO5_VOLTAGE_NUM,
+ .linear_ranges = bd71847_ldo5_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd71847_ldo5_volts),
+ .vsel_reg = BD718XX_REG_LDO5_VOLT,
+ .vsel_mask = BD71847_LDO5_MASK,
+ .vsel_range_reg = BD718XX_REG_LDO5_VOLT,
+ .vsel_range_mask = BD71847_LDO5_RANGE_MASK,
+ .linear_range_selectors = bd71847_ldo5_volt_range_sel,
+ .enable_reg = BD718XX_REG_LDO5_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_LDO5_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "ldo6",
+ .of_match = of_match_ptr("LDO6"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO6,
+ .ops = &bd718xx_ldo_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_LDO6_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_ldo6_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_ldo6_volts),
+ /* LDO6 is supplied by buck5 */
+ .supply_name = "buck5",
+ .vsel_reg = BD718XX_REG_LDO6_VOLT,
+ .vsel_mask = BD718XX_LDO6_MASK,
+ .enable_reg = BD718XX_REG_LDO6_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_LDO6_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ },
+};
+
+static const struct bd718xx_regulator_data bd71837_regulators[] = {
+ {
+ .desc = {
+ .name = "buck1",
+ .of_match = of_match_ptr("BUCK1"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK1,
+ .ops = &bd718xx_dvs_buck_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_DVS_BUCK_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_dvs_buck_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_dvs_buck_volts),
+ .vsel_reg = BD718XX_REG_BUCK1_VOLT_RUN,
+ .vsel_mask = DVS_BUCK_RUN_MASK,
+ .enable_reg = BD718XX_REG_BUCK1_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_BUCK1_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "buck2",
+ .of_match = of_match_ptr("BUCK2"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK2,
+ .ops = &bd718xx_dvs_buck_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_DVS_BUCK_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_dvs_buck_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_dvs_buck_volts),
+ .vsel_reg = BD718XX_REG_BUCK2_VOLT_RUN,
+ .vsel_mask = DVS_BUCK_RUN_MASK,
+ .enable_reg = BD718XX_REG_BUCK2_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_BUCK2_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "buck3",
+ .of_match = of_match_ptr("BUCK3"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK3,
+ .ops = &bd718xx_dvs_buck_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_DVS_BUCK_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_dvs_buck_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_dvs_buck_volts),
+ .vsel_reg = BD71837_REG_BUCK3_VOLT_RUN,
+ .vsel_mask = DVS_BUCK_RUN_MASK,
+ .enable_reg = BD71837_REG_BUCK3_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD71837_REG_BUCK3_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "buck4",
+ .of_match = of_match_ptr("BUCK4"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK4,
+ .ops = &bd718xx_dvs_buck_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_DVS_BUCK_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_dvs_buck_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_dvs_buck_volts),
+ .vsel_reg = BD71837_REG_BUCK4_VOLT_RUN,
+ .vsel_mask = DVS_BUCK_RUN_MASK,
+ .enable_reg = BD71837_REG_BUCK4_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD71837_REG_BUCK4_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "buck5",
+ .of_match = of_match_ptr("BUCK5"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK5,
+ .ops = &bd718xx_pickable_range_buck_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD71837_BUCK5_VOLTAGE_NUM,
+ .linear_ranges = bd71837_buck5_volts,
+ .n_linear_ranges =
+ ARRAY_SIZE(bd71837_buck5_volts),
+ .vsel_reg = BD718XX_REG_1ST_NODVS_BUCK_VOLT,
+ .vsel_mask = BD71837_BUCK5_MASK,
+ .vsel_range_reg = BD718XX_REG_1ST_NODVS_BUCK_VOLT,
+ .vsel_range_mask = BD71837_BUCK5_RANGE_MASK,
+ .linear_range_selectors = bd71837_buck5_volt_range_sel,
+ .enable_reg = BD718XX_REG_1ST_NODVS_BUCK_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_1ST_NODVS_BUCK_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "buck6",
+ .of_match = of_match_ptr("BUCK6"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK6,
+ .ops = &bd718xx_buck_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD71837_BUCK6_VOLTAGE_NUM,
+ .linear_ranges = bd71837_buck6_volts,
+ .n_linear_ranges =
+ ARRAY_SIZE(bd71837_buck6_volts),
+ .vsel_reg = BD718XX_REG_2ND_NODVS_BUCK_VOLT,
+ .vsel_mask = BD71837_BUCK6_MASK,
+ .enable_reg = BD718XX_REG_2ND_NODVS_BUCK_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_2ND_NODVS_BUCK_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "buck7",
+ .of_match = of_match_ptr("BUCK7"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK7,
+ .ops = &bd718xx_buck_regulator_nolinear_ops,
+ .type = REGULATOR_VOLTAGE,
+ .volt_table = &bd718xx_3rd_nodvs_buck_volts[0],
+ .n_voltages = ARRAY_SIZE(bd718xx_3rd_nodvs_buck_volts),
+ .vsel_reg = BD718XX_REG_3RD_NODVS_BUCK_VOLT,
+ .vsel_mask = BD718XX_3RD_NODVS_BUCK_MASK,
+ .enable_reg = BD718XX_REG_3RD_NODVS_BUCK_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_3RD_NODVS_BUCK_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "buck8",
+ .of_match = of_match_ptr("BUCK8"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_BUCK8,
+ .ops = &bd718xx_buck_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_4TH_NODVS_BUCK_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_4th_nodvs_buck_volts,
+ .n_linear_ranges =
+ ARRAY_SIZE(bd718xx_4th_nodvs_buck_volts),
+ .vsel_reg = BD718XX_REG_4TH_NODVS_BUCK_VOLT,
+ .vsel_mask = BD718XX_4TH_NODVS_BUCK_MASK,
+ .enable_reg = BD718XX_REG_4TH_NODVS_BUCK_CTRL,
+ .enable_mask = BD718XX_BUCK_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_4TH_NODVS_BUCK_CTRL,
+ .mask = BD718XX_BUCK_SEL,
+ .val = BD718XX_BUCK_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "ldo1",
+ .of_match = of_match_ptr("LDO1"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO1,
+ .ops = &bd718xx_pickable_range_ldo_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_LDO1_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_ldo1_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_ldo1_volts),
+ .vsel_reg = BD718XX_REG_LDO1_VOLT,
+ .vsel_mask = BD718XX_LDO1_MASK,
+ .vsel_range_reg = BD718XX_REG_LDO1_VOLT,
+ .vsel_range_mask = BD718XX_LDO1_RANGE_MASK,
+ .linear_range_selectors = bd718xx_ldo1_volt_range_sel,
+ .enable_reg = BD718XX_REG_LDO1_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_LDO1_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "ldo2",
+ .of_match = of_match_ptr("LDO2"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO2,
+ .ops = &bd718xx_ldo_regulator_nolinear_ops,
+ .type = REGULATOR_VOLTAGE,
+ .volt_table = &ldo_2_volts[0],
+ .vsel_reg = BD718XX_REG_LDO2_VOLT,
+ .vsel_mask = BD718XX_LDO2_MASK,
+ .n_voltages = ARRAY_SIZE(ldo_2_volts),
+ .enable_reg = BD718XX_REG_LDO2_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_LDO2_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "ldo3",
+ .of_match = of_match_ptr("LDO3"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO3,
+ .ops = &bd718xx_ldo_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_LDO3_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_ldo3_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_ldo3_volts),
+ .vsel_reg = BD718XX_REG_LDO3_VOLT,
+ .vsel_mask = BD718XX_LDO3_MASK,
+ .enable_reg = BD718XX_REG_LDO3_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_LDO3_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "ldo4",
+ .of_match = of_match_ptr("LDO4"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO4,
+ .ops = &bd718xx_ldo_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_LDO4_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_ldo4_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_ldo4_volts),
+ .vsel_reg = BD718XX_REG_LDO4_VOLT,
+ .vsel_mask = BD718XX_LDO4_MASK,
+ .enable_reg = BD718XX_REG_LDO4_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_LDO4_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ },
+ {
+ .desc = {
+ .name = "ldo5",
+ .of_match = of_match_ptr("LDO5"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO5,
+ .ops = &bd718xx_ldo_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD71837_LDO5_VOLTAGE_NUM,
+ .linear_ranges = bd71837_ldo5_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd71837_ldo5_volts),
+ /* LDO5 is supplied by buck6 */
+ .supply_name = "buck6",
+ .vsel_reg = BD718XX_REG_LDO5_VOLT,
+ .vsel_mask = BD71837_LDO5_MASK,
+ .enable_reg = BD718XX_REG_LDO5_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_LDO5_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ .additional_inits = bd71837_ldo5_inits,
+ .additional_init_amnt = ARRAY_SIZE(bd71837_ldo5_inits),
+ },
+ {
+ .desc = {
+ .name = "ldo6",
+ .of_match = of_match_ptr("LDO6"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO6,
+ .ops = &bd718xx_ldo_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD718XX_LDO6_VOLTAGE_NUM,
+ .linear_ranges = bd718xx_ldo6_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd718xx_ldo6_volts),
+ /* LDO6 is supplied by buck7 */
+ .supply_name = "buck7",
+ .vsel_reg = BD718XX_REG_LDO6_VOLT,
+ .vsel_mask = BD718XX_LDO6_MASK,
+ .enable_reg = BD718XX_REG_LDO6_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD718XX_REG_LDO6_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ .additional_inits = bd71837_ldo6_inits,
+ .additional_init_amnt = ARRAY_SIZE(bd71837_ldo6_inits),
+ },
+ {
+ .desc = {
+ .name = "ldo7",
+ .of_match = of_match_ptr("LDO7"),
+ .regulators_node = of_match_ptr("regulators"),
+ .id = BD718XX_LDO7,
+ .ops = &bd718xx_ldo_regulator_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = BD71837_LDO7_VOLTAGE_NUM,
+ .linear_ranges = bd71837_ldo7_volts,
+ .n_linear_ranges = ARRAY_SIZE(bd71837_ldo7_volts),
+ .vsel_reg = BD71837_REG_LDO7_VOLT,
+ .vsel_mask = BD71837_LDO7_MASK,
+ .enable_reg = BD71837_REG_LDO7_VOLT,
+ .enable_mask = BD718XX_LDO_EN,
+ .owner = THIS_MODULE,
+ },
+ .init = {
+ .reg = BD71837_REG_LDO7_VOLT,
+ .mask = BD718XX_LDO_SEL,
+ .val = BD718XX_LDO_SEL,
+ },
+ },
+};
+
+struct bd718xx_pmic_inits {
+ const struct bd718xx_regulator_data (*r_datas)[];
+ unsigned int r_amount;
+};
+
+static int bd718xx_probe(struct platform_device *pdev)
+{
+ struct bd718xx *mfd;
+ struct regulator_config config = { 0 };
+ struct bd718xx_pmic_inits pmic_regulators[] = {
+ [BD718XX_TYPE_BD71837] = {
+ .r_datas = &bd71837_regulators,
+ .r_amount = ARRAY_SIZE(bd71837_regulators),
+ },
+ [BD718XX_TYPE_BD71847] = {
+ .r_datas = &bd71847_regulators,
+ .r_amount = ARRAY_SIZE(bd71847_regulators),
+ },
+ };
+
+ int i, j, err;
+
+ mfd = dev_get_drvdata(pdev->dev.parent);
+ if (!mfd) {
+ dev_err(&pdev->dev, "No MFD driver data\n");
+ err = -EINVAL;
+ goto err;
+ }
+
+ if (mfd->chip_type >= BD718XX_TYPE_AMOUNT ||
+ !pmic_regulators[mfd->chip_type].r_datas) {
+ dev_err(&pdev->dev, "Unsupported chip type\n");
+ err = -EINVAL;
+ goto err;
+ }
+
+ /* Register LOCK release */
+ err = regmap_update_bits(mfd->regmap, BD718XX_REG_REGLOCK,
+ (REGLOCK_PWRSEQ | REGLOCK_VREG), 0);
+ if (err) {
+ dev_err(&pdev->dev, "Failed to unlock PMIC (%d)\n", err);
+ goto err;
+ } else {
+ dev_dbg(&pdev->dev, "Unlocked lock register 0x%x\n",
+ BD718XX_REG_REGLOCK);
+ }
+
+ for (i = 0; i < pmic_regulators[mfd->chip_type].r_amount; i++) {
+
+ const struct regulator_desc *desc;
+ struct regulator_dev *rdev;
+ const struct bd718xx_regulator_data *r;
+
+ r = &(*pmic_regulators[mfd->chip_type].r_datas)[i];
+ desc = &r->desc;
+
+ config.dev = pdev->dev.parent;
+ config.regmap = mfd->regmap;
+
+ rdev = devm_regulator_register(&pdev->dev, desc, &config);
+ if (IS_ERR(rdev)) {
+ dev_err(&pdev->dev,
+ "failed to register %s regulator\n",
+ desc->name);
+ err = PTR_ERR(rdev);
+ goto err;
+ }
+ /* Regulator register gets the regulator constraints and
+ * applies them (set_machine_constraints). This should have
+ * turned the control register(s) to correct values and we
+ * can now switch the control from PMIC state machine to the
+ * register interface
+ */
+ err = regmap_update_bits(mfd->regmap, r->init.reg,
+ r->init.mask, r->init.val);
+ if (err) {
+ dev_err(&pdev->dev,
+ "Failed to write BUCK/LDO SEL bit for (%s)\n",
+ desc->name);
+ goto err;
+ }
+ for (j = 0; j < r->additional_init_amnt; j++) {
+ err = regmap_update_bits(mfd->regmap,
+ r->additional_inits[j].reg,
+ r->additional_inits[j].mask,
+ r->additional_inits[j].val);
+ if (err) {
+ dev_err(&pdev->dev,
+ "Buck (%s) initialization failed\n",
+ desc->name);
+ goto err;
+ }
+ }
+ }
+
+err:
+ return err;
+}
+
+static struct platform_driver bd718xx_regulator = {
+ .driver = {
+ .name = "bd718xx-pmic",
+ },
+ .probe = bd718xx_probe,
+};
+
+module_platform_driver(bd718xx_regulator);
+
+MODULE_AUTHOR("Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>");
+MODULE_DESCRIPTION("BD71837/BD71847 voltage regulator driver");
+MODULE_LICENSE("GPL");
}
static DEVICE_ATTR_RO(name);
-static ssize_t regulator_print_opmode(char *buf, int mode)
+static const char *regulator_opmode_to_str(int mode)
{
switch (mode) {
case REGULATOR_MODE_FAST:
- return sprintf(buf, "fast\n");
+ return "fast";
case REGULATOR_MODE_NORMAL:
- return sprintf(buf, "normal\n");
+ return "normal";
case REGULATOR_MODE_IDLE:
- return sprintf(buf, "idle\n");
+ return "idle";
case REGULATOR_MODE_STANDBY:
- return sprintf(buf, "standby\n");
+ return "standby";
}
- return sprintf(buf, "unknown\n");
+ return "unknown";
+}
+
+static ssize_t regulator_print_opmode(char *buf, int mode)
+{
+ return sprintf(buf, "%s\n", regulator_opmode_to_str(mode));
}
static ssize_t regulator_opmode_show(struct device *dev,
if (desc->ops->list_voltage == regulator_list_voltage_linear_range)
return regulator_map_voltage_linear_range(rdev, min_uV, max_uV);
+ if (desc->ops->list_voltage ==
+ regulator_list_voltage_pickable_linear_range)
+ return regulator_map_voltage_pickable_linear_range(rdev,
+ min_uV, max_uV);
+
return regulator_map_voltage_iterate(rdev, min_uV, max_uV);
}
}
EXPORT_SYMBOL_GPL(regulator_set_current_limit);
+static int _regulator_get_current_limit_unlocked(struct regulator_dev *rdev)
+{
+ /* sanity check */
+ if (!rdev->desc->ops->get_current_limit)
+ return -EINVAL;
+
+ return rdev->desc->ops->get_current_limit(rdev);
+}
+
static int _regulator_get_current_limit(struct regulator_dev *rdev)
{
int ret;
regulator_lock(rdev);
-
- /* sanity check */
- if (!rdev->desc->ops->get_current_limit) {
- ret = -EINVAL;
- goto out;
- }
-
- ret = rdev->desc->ops->get_current_limit(rdev);
-out:
+ ret = _regulator_get_current_limit_unlocked(rdev);
regulator_unlock(rdev);
+
return ret;
}
}
EXPORT_SYMBOL_GPL(regulator_set_mode);
+static unsigned int _regulator_get_mode_unlocked(struct regulator_dev *rdev)
+{
+ /* sanity check */
+ if (!rdev->desc->ops->get_mode)
+ return -EINVAL;
+
+ return rdev->desc->ops->get_mode(rdev);
+}
+
static unsigned int _regulator_get_mode(struct regulator_dev *rdev)
{
int ret;
regulator_lock(rdev);
-
- /* sanity check */
- if (!rdev->desc->ops->get_mode) {
- ret = -EINVAL;
- goto out;
- }
-
- ret = rdev->desc->ops->get_mode(rdev);
-out:
+ ret = _regulator_get_mode_unlocked(rdev);
regulator_unlock(rdev);
+
return ret;
}
EXPORT_SYMBOL_GPL(regulator_unregister);
#ifdef CONFIG_SUSPEND
-static int _regulator_suspend(struct device *dev, void *data)
-{
- struct regulator_dev *rdev = dev_to_rdev(dev);
- suspend_state_t *state = data;
- int ret;
-
- regulator_lock(rdev);
- ret = suspend_set_state(rdev, *state);
- regulator_unlock(rdev);
-
- return ret;
-}
-
/**
* regulator_suspend - prepare regulators for system wide suspend
- * @state: system suspend state
+ * @dev: ``&struct device`` pointer that is passed to _regulator_suspend()
*
* Configure each regulator with it's suspend operating parameters for state.
*/
static int regulator_suspend(struct device *dev)
{
+ struct regulator_dev *rdev = dev_to_rdev(dev);
suspend_state_t state = pm_suspend_target_state;
+ int ret;
+
+ regulator_lock(rdev);
+ ret = suspend_set_state(rdev, state);
+ regulator_unlock(rdev);
- return class_for_each_device(®ulator_class, NULL, &state,
- _regulator_suspend);
+ return ret;
}
-static int _regulator_resume(struct device *dev, void *data)
+static int regulator_resume(struct device *dev)
{
- int ret = 0;
+ suspend_state_t state = pm_suspend_target_state;
struct regulator_dev *rdev = dev_to_rdev(dev);
- suspend_state_t *state = data;
struct regulator_state *rstate;
+ int ret = 0;
- rstate = regulator_get_suspend_state(rdev, *state);
+ rstate = regulator_get_suspend_state(rdev, state);
if (rstate == NULL)
return 0;
return ret;
}
-
-static int regulator_resume(struct device *dev)
-{
- suspend_state_t state = pm_suspend_target_state;
-
- return class_for_each_device(®ulator_class, NULL, &state,
- _regulator_resume);
-}
-
#else /* !CONFIG_SUSPEND */
#define regulator_suspend NULL
struct regulation_constraints *c;
struct regulator *consumer;
struct summary_data summary_data;
+ unsigned int opmode;
if (!rdev)
return;
- seq_printf(s, "%*s%-*s %3d %4d %6d ",
+ regulator_lock_nested(rdev, level);
+
+ opmode = _regulator_get_mode_unlocked(rdev);
+ seq_printf(s, "%*s%-*s %3d %4d %6d %7s ",
level * 3 + 1, "",
30 - level * 3, rdev_get_name(rdev),
- rdev->use_count, rdev->open_count, rdev->bypass_count);
+ rdev->use_count, rdev->open_count, rdev->bypass_count,
+ regulator_opmode_to_str(opmode));
seq_printf(s, "%5dmV ", _regulator_get_voltage(rdev) / 1000);
- seq_printf(s, "%5dmA ", _regulator_get_current_limit(rdev) / 1000);
+ seq_printf(s, "%5dmA ",
+ _regulator_get_current_limit_unlocked(rdev) / 1000);
c = rdev->constraints;
if (c) {
switch (rdev->desc->type) {
case REGULATOR_VOLTAGE:
- seq_printf(s, "%37dmV %5dmV",
+ seq_printf(s, "%37dmA %5dmV %5dmV",
+ consumer->uA_load / 1000,
consumer->voltage[PM_SUSPEND_ON].min_uV / 1000,
consumer->voltage[PM_SUSPEND_ON].max_uV / 1000);
break;
class_for_each_device(®ulator_class, NULL, &summary_data,
regulator_summary_show_children);
+
+ regulator_unlock(rdev);
}
static int regulator_summary_show_roots(struct device *dev, void *data)
static int regulator_summary_show(struct seq_file *s, void *data)
{
- seq_puts(s, " regulator use open bypass voltage current min max\n");
- seq_puts(s, "-------------------------------------------------------------------------------\n");
+ seq_puts(s, " regulator use open bypass opmode voltage current min max\n");
+ seq_puts(s, "---------------------------------------------------------------------------------------\n");
class_for_each_device(®ulator_class, NULL, s,
regulator_summary_show_roots);
config.dev = &pdev->dev;
config.driver_data = regulator;
config.regmap = da9052->regmap;
- if (pdata && pdata->regulators) {
+ if (pdata) {
config.init_data = pdata->regulators[cell->id];
} else {
#ifdef CONFIG_OF
config.driver_data = regulator;
config.regmap = da9055->regmap;
- if (pdata && pdata->regulators) {
+ if (pdata) {
config.init_data = pdata->regulators[pdev->id];
} else {
ret = da9055_regulator_dt_init(pdev, regulator, &config,
pdata->init_data[n] = da9211_matches[i].init_data;
pdata->reg_node[n] = da9211_matches[i].of_node;
pdata->gpiod_ren[n] = devm_gpiod_get_from_of_node(dev,
- da9211_matches[i].of_node,
- "enable",
- 0,
- GPIOD_OUT_HIGH,
- "da9211-enable");
+ da9211_matches[i].of_node,
+ "enable",
+ 0,
+ GPIOD_OUT_HIGH | GPIOD_FLAGS_BIT_NONEXCLUSIVE,
+ "da9211-enable");
n++;
}
}
data->cfg.microvolts = uv;
- data->cfg.gpio = -EINVAL;
data->cfg.enabled_at_boot = 1;
data->cfg.init_data = &data->init_data;
#include <linux/platform_device.h>
#include <linux/regulator/driver.h>
#include <linux/regulator/fixed.h>
-#include <linux/gpio.h>
+#include <linux/gpio/consumer.h>
#include <linux/slab.h>
#include <linux/of.h>
-#include <linux/of_gpio.h>
#include <linux/regulator/of_regulator.h>
#include <linux/regulator/machine.h>
if (init_data->constraints.boot_on)
config->enabled_at_boot = true;
- config->gpio = of_get_named_gpio(np, "gpio", 0);
- if ((config->gpio < 0) && (config->gpio != -ENOENT))
- return ERR_PTR(config->gpio);
-
of_property_read_u32(np, "startup-delay-us", &config->startup_delay);
- config->enable_high = of_property_read_bool(np, "enable-active-high");
- config->gpio_is_open_drain = of_property_read_bool(np,
- "gpio-open-drain");
+ /*
+ * FIXME: we pulled active low/high and open drain handling into
+ * gpiolib so it will be handled there. Delete this in the second
+ * step when we also remove the custom inversion handling for all
+ * legacy boardfiles.
+ */
+ config->enable_high = 1;
+ config->gpio_is_open_drain = 0;
if (of_find_property(np, "vin-supply", NULL))
config->input_supply = "vin";
struct fixed_voltage_config *config;
struct fixed_voltage_data *drvdata;
struct regulator_config cfg = { };
+ enum gpiod_flags gflags;
int ret;
drvdata = devm_kzalloc(&pdev->dev, sizeof(struct fixed_voltage_data),
drvdata->desc.fixed_uV = config->microvolts;
- if (gpio_is_valid(config->gpio)) {
- cfg.ena_gpio = config->gpio;
- if (pdev->dev.of_node)
- cfg.ena_gpio_initialized = true;
- }
cfg.ena_gpio_invert = !config->enable_high;
if (config->enabled_at_boot) {
if (config->enable_high)
- cfg.ena_gpio_flags |= GPIOF_OUT_INIT_HIGH;
+ gflags = GPIOD_OUT_HIGH;
else
- cfg.ena_gpio_flags |= GPIOF_OUT_INIT_LOW;
+ gflags = GPIOD_OUT_LOW;
} else {
if (config->enable_high)
- cfg.ena_gpio_flags |= GPIOF_OUT_INIT_LOW;
+ gflags = GPIOD_OUT_LOW;
else
- cfg.ena_gpio_flags |= GPIOF_OUT_INIT_HIGH;
+ gflags = GPIOD_OUT_HIGH;
}
- if (config->gpio_is_open_drain)
- cfg.ena_gpio_flags |= GPIOF_OPEN_DRAIN;
+ if (config->gpio_is_open_drain) {
+ if (gflags == GPIOD_OUT_HIGH)
+ gflags = GPIOD_OUT_HIGH_OPEN_DRAIN;
+ else
+ gflags = GPIOD_OUT_LOW_OPEN_DRAIN;
+ }
+
+ /*
+ * Some fixed regulators share the enable line between two
+ * regulators which makes it necessary to get a handle on the
+ * same descriptor for two different consumers. This will get
+ * the GPIO descriptor, but only the first call will initialize
+ * it so any flags such as inversion or open drain will only
+ * be set up by the first caller and assumed identical on the
+ * next caller.
+ *
+ * FIXME: find a better way to deal with this.
+ */
+ gflags |= GPIOD_FLAGS_BIT_NONEXCLUSIVE;
+
+ cfg.ena_gpiod = devm_gpiod_get_optional(&pdev->dev, NULL, gflags);
+ if (IS_ERR(cfg.ena_gpiod))
+ return PTR_ERR(cfg.ena_gpiod);
cfg.dev = &pdev->dev;
cfg.init_data = config->init_data;
}
EXPORT_SYMBOL_GPL(regulator_disable_regmap);
+static int regulator_range_selector_to_index(struct regulator_dev *rdev,
+ unsigned int rval)
+{
+ int i;
+
+ if (!rdev->desc->linear_range_selectors)
+ return -EINVAL;
+
+ rval &= rdev->desc->vsel_range_mask;
+
+ for (i = 0; i < rdev->desc->n_linear_ranges; i++) {
+ if (rdev->desc->linear_range_selectors[i] == rval)
+ return i;
+ }
+ return -EINVAL;
+}
+
+/**
+ * regulator_get_voltage_sel_pickable_regmap - pickable range get_voltage_sel
+ *
+ * @rdev: regulator to operate on
+ *
+ * Regulators that use regmap for their register I/O and use pickable
+ * ranges can set the vsel_reg, vsel_mask, vsel_range_reg and vsel_range_mask
+ * fields in their descriptor and then use this as their get_voltage_vsel
+ * operation, saving some code.
+ */
+int regulator_get_voltage_sel_pickable_regmap(struct regulator_dev *rdev)
+{
+ unsigned int r_val;
+ int range;
+ unsigned int val;
+ int ret, i;
+ unsigned int voltages_in_range = 0;
+
+ if (!rdev->desc->linear_ranges)
+ return -EINVAL;
+
+ ret = regmap_read(rdev->regmap, rdev->desc->vsel_reg, &val);
+ if (ret != 0)
+ return ret;
+
+ ret = regmap_read(rdev->regmap, rdev->desc->vsel_range_reg, &r_val);
+ if (ret != 0)
+ return ret;
+
+ val &= rdev->desc->vsel_mask;
+ val >>= ffs(rdev->desc->vsel_mask) - 1;
+
+ range = regulator_range_selector_to_index(rdev, r_val);
+ if (range < 0)
+ return -EINVAL;
+
+ for (i = 0; i < range; i++)
+ voltages_in_range += (rdev->desc->linear_ranges[i].max_sel -
+ rdev->desc->linear_ranges[i].min_sel) + 1;
+
+ return val + voltages_in_range;
+}
+EXPORT_SYMBOL_GPL(regulator_get_voltage_sel_pickable_regmap);
+
+/**
+ * regulator_set_voltage_sel_pickable_regmap - pickable range set_voltage_sel
+ *
+ * @rdev: regulator to operate on
+ * @sel: Selector to set
+ *
+ * Regulators that use regmap for their register I/O and use pickable
+ * ranges can set the vsel_reg, vsel_mask, vsel_range_reg and vsel_range_mask
+ * fields in their descriptor and then use this as their set_voltage_vsel
+ * operation, saving some code.
+ */
+int regulator_set_voltage_sel_pickable_regmap(struct regulator_dev *rdev,
+ unsigned int sel)
+{
+ unsigned int range;
+ int ret, i;
+ unsigned int voltages_in_range = 0;
+
+ for (i = 0; i < rdev->desc->n_linear_ranges; i++) {
+ voltages_in_range = (rdev->desc->linear_ranges[i].max_sel -
+ rdev->desc->linear_ranges[i].min_sel) + 1;
+ if (sel < voltages_in_range)
+ break;
+ sel -= voltages_in_range;
+ }
+
+ if (i == rdev->desc->n_linear_ranges)
+ return -EINVAL;
+
+ sel <<= ffs(rdev->desc->vsel_mask) - 1;
+ sel += rdev->desc->linear_ranges[i].min_sel;
+
+ range = rdev->desc->linear_range_selectors[i];
+
+ if (rdev->desc->vsel_reg == rdev->desc->vsel_range_reg) {
+ ret = regmap_update_bits(rdev->regmap,
+ rdev->desc->vsel_reg,
+ rdev->desc->vsel_range_mask |
+ rdev->desc->vsel_mask, sel | range);
+ } else {
+ ret = regmap_update_bits(rdev->regmap,
+ rdev->desc->vsel_range_reg,
+ rdev->desc->vsel_range_mask, range);
+ if (ret)
+ return ret;
+
+ ret = regmap_update_bits(rdev->regmap, rdev->desc->vsel_reg,
+ rdev->desc->vsel_mask, sel);
+ }
+
+ if (ret)
+ return ret;
+
+ if (rdev->desc->apply_bit)
+ ret = regmap_update_bits(rdev->regmap, rdev->desc->apply_reg,
+ rdev->desc->apply_bit,
+ rdev->desc->apply_bit);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(regulator_set_voltage_sel_pickable_regmap);
+
/**
* regulator_get_voltage_sel_regmap - standard get_voltage_sel for regmap users
*
ret += range->min_sel;
- break;
+ /*
+ * Map back into a voltage to verify we're still in bounds.
+ * If we are not, then continue checking rest of the ranges.
+ */
+ voltage = rdev->desc->ops->list_voltage(rdev, ret);
+ if (voltage >= min_uV && voltage <= max_uV)
+ break;
}
if (i == rdev->desc->n_linear_ranges)
return -EINVAL;
- /* Map back into a voltage to verify we're still in bounds */
- voltage = rdev->desc->ops->list_voltage(rdev, ret);
- if (voltage < min_uV || voltage > max_uV)
+ return ret;
+}
+EXPORT_SYMBOL_GPL(regulator_map_voltage_linear_range);
+
+/**
+ * regulator_map_voltage_pickable_linear_range - map_voltage, pickable ranges
+ *
+ * @rdev: Regulator to operate on
+ * @min_uV: Lower bound for voltage
+ * @max_uV: Upper bound for voltage
+ *
+ * Drivers providing pickable linear_ranges in their descriptor can use
+ * this as their map_voltage() callback.
+ */
+int regulator_map_voltage_pickable_linear_range(struct regulator_dev *rdev,
+ int min_uV, int max_uV)
+{
+ const struct regulator_linear_range *range;
+ int ret = -EINVAL;
+ int voltage, i;
+ unsigned int selector = 0;
+
+ if (!rdev->desc->n_linear_ranges) {
+ BUG_ON(!rdev->desc->n_linear_ranges);
+ return -EINVAL;
+ }
+
+ for (i = 0; i < rdev->desc->n_linear_ranges; i++) {
+ int linear_max_uV;
+
+ range = &rdev->desc->linear_ranges[i];
+ linear_max_uV = range->min_uV +
+ (range->max_sel - range->min_sel) * range->uV_step;
+
+ if (!(min_uV <= linear_max_uV && max_uV >= range->min_uV)) {
+ selector += (range->max_sel - range->min_sel + 1);
+ continue;
+ }
+
+ if (min_uV <= range->min_uV)
+ min_uV = range->min_uV;
+
+ /* range->uV_step == 0 means fixed voltage range */
+ if (range->uV_step == 0) {
+ ret = 0;
+ } else {
+ ret = DIV_ROUND_UP(min_uV - range->min_uV,
+ range->uV_step);
+ if (ret < 0)
+ return ret;
+ }
+
+ ret += selector;
+
+ voltage = rdev->desc->ops->list_voltage(rdev, ret);
+
+ /*
+ * Map back into a voltage to verify we're still in bounds.
+ * We may have overlapping voltage ranges. Hence we don't
+ * exit but retry until we have checked all ranges.
+ */
+ if (voltage < min_uV || voltage > max_uV)
+ selector += (range->max_sel - range->min_sel + 1);
+ else
+ break;
+ }
+
+ if (i == rdev->desc->n_linear_ranges)
return -EINVAL;
return ret;
}
-EXPORT_SYMBOL_GPL(regulator_map_voltage_linear_range);
+EXPORT_SYMBOL_GPL(regulator_map_voltage_pickable_linear_range);
/**
* regulator_list_voltage_linear - List voltages with simple calculation
}
EXPORT_SYMBOL_GPL(regulator_list_voltage_linear);
+/**
+ * regulator_list_voltage_pickable_linear_range - pickable range list voltages
+ *
+ * @rdev: Regulator device
+ * @selector: Selector to convert into a voltage
+ *
+ * list_voltage() operation, intended to be used by drivers utilizing pickable
+ * ranges helpers.
+ */
+int regulator_list_voltage_pickable_linear_range(struct regulator_dev *rdev,
+ unsigned int selector)
+{
+ const struct regulator_linear_range *range;
+ int i;
+ unsigned int all_sels = 0;
+
+ if (!rdev->desc->n_linear_ranges) {
+ BUG_ON(!rdev->desc->n_linear_ranges);
+ return -EINVAL;
+ }
+
+ for (i = 0; i < rdev->desc->n_linear_ranges; i++) {
+ unsigned int sels_in_range;
+
+ range = &rdev->desc->linear_ranges[i];
+
+ sels_in_range = range->max_sel - range->min_sel;
+
+ if (all_sels + sels_in_range >= selector) {
+ selector -= all_sels;
+ return range->min_uV + (range->uV_step * selector);
+ }
+
+ all_sels += (sels_in_range + 1);
+ }
+
+ return -EINVAL;
+}
+EXPORT_SYMBOL_GPL(regulator_list_voltage_pickable_linear_range);
+
/**
* regulator_list_voltage_linear_range - List voltages for linear ranges
*
.enable_mask = ISL9305_DCD1_EN,
.supply_name = "VINDCD1",
.ops = &isl9305_ops,
+ .owner = THIS_MODULE,
},
[ISL9305_DCD2] = {
.name = "DCD2",
.enable_mask = ISL9305_DCD2_EN,
.supply_name = "VINDCD2",
.ops = &isl9305_ops,
+ .owner = THIS_MODULE,
},
[ISL9305_LDO1] = {
.name = "LDO1",
.enable_mask = ISL9305_LDO1_EN,
.supply_name = "VINLDO1",
.ops = &isl9305_ops,
+ .owner = THIS_MODULE,
},
[ISL9305_LDO2] = {
.name = "LDO2",
.enable_mask = ISL9305_LDO2_EN,
.supply_name = "VINLDO2",
.ops = &isl9305_ops,
+ .owner = THIS_MODULE,
},
};
*/
switch (id) {
case LM3632_LDO_POS:
- return devm_gpiod_get_index_optional(dev, "enable", 0, GPIOD_OUT_LOW);
+ return devm_gpiod_get_index_optional(dev, "enable", 0,
+ GPIOD_OUT_LOW | GPIOD_FLAGS_BIT_NONEXCLUSIVE);
case LM3632_LDO_NEG:
- return devm_gpiod_get_index_optional(dev, "enable", 1, GPIOD_OUT_LOW);
+ return devm_gpiod_get_index_optional(dev, "enable", 1,
+ GPIOD_OUT_LOW | GPIOD_FLAGS_BIT_NONEXCLUSIVE);
default:
return NULL;
}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+//
+// Lochnagar regulator driver
+//
+// Copyright (c) 2017-2018 Cirrus Logic, Inc. and
+// Cirrus Logic International Semiconductor Ltd.
+//
+// Author: Charles Keepax <ckeepax@opensource.cirrus.com>
+
+#include <linux/bitops.h>
+#include <linux/device.h>
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <linux/regulator/driver.h>
+#include <linux/regulator/machine.h>
+#include <linux/regulator/of_regulator.h>
+
+#include <linux/mfd/lochnagar.h>
+
+static const struct regulator_ops lochnagar_micvdd_ops = {
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+
+ .list_voltage = regulator_list_voltage_linear_range,
+ .map_voltage = regulator_map_voltage_linear_range,
+
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+};
+
+static const struct regulator_linear_range lochnagar_micvdd_ranges[] = {
+ REGULATOR_LINEAR_RANGE(1000000, 0, 0xC, 50000),
+ REGULATOR_LINEAR_RANGE(1700000, 0xD, 0x1F, 100000),
+};
+
+static int lochnagar_micbias_enable(struct regulator_dev *rdev)
+{
+ struct lochnagar *lochnagar = rdev_get_drvdata(rdev);
+ int ret;
+
+ mutex_lock(&lochnagar->analogue_config_lock);
+
+ ret = regulator_enable_regmap(rdev);
+ if (ret < 0)
+ goto err;
+
+ ret = lochnagar_update_config(lochnagar);
+
+err:
+ mutex_unlock(&lochnagar->analogue_config_lock);
+
+ return ret;
+}
+
+static int lochnagar_micbias_disable(struct regulator_dev *rdev)
+{
+ struct lochnagar *lochnagar = rdev_get_drvdata(rdev);
+ int ret;
+
+ mutex_lock(&lochnagar->analogue_config_lock);
+
+ ret = regulator_disable_regmap(rdev);
+ if (ret < 0)
+ goto err;
+
+ ret = lochnagar_update_config(lochnagar);
+
+err:
+ mutex_unlock(&lochnagar->analogue_config_lock);
+
+ return ret;
+}
+
+static const struct regulator_ops lochnagar_micbias_ops = {
+ .enable = lochnagar_micbias_enable,
+ .disable = lochnagar_micbias_disable,
+ .is_enabled = regulator_is_enabled_regmap,
+};
+
+static const struct regulator_ops lochnagar_vddcore_ops = {
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+
+ .list_voltage = regulator_list_voltage_linear_range,
+ .map_voltage = regulator_map_voltage_linear_range,
+
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+};
+
+static const struct regulator_linear_range lochnagar_vddcore_ranges[] = {
+ REGULATOR_LINEAR_RANGE(600000, 0x8, 0x41, 12500),
+};
+
+enum lochnagar_regulators {
+ LOCHNAGAR_MICVDD,
+ LOCHNAGAR_MIC1VDD,
+ LOCHNAGAR_MIC2VDD,
+ LOCHNAGAR_VDDCORE,
+};
+
+static int lochnagar_micbias_of_parse(struct device_node *np,
+ const struct regulator_desc *desc,
+ struct regulator_config *config)
+{
+ struct lochnagar *lochnagar = config->driver_data;
+ int shift = (desc->id - LOCHNAGAR_MIC1VDD) *
+ LOCHNAGAR2_P2_MICBIAS_SRC_SHIFT;
+ int mask = LOCHNAGAR2_P1_MICBIAS_SRC_MASK << shift;
+ unsigned int val;
+ int ret;
+
+ ret = of_property_read_u32(np, "cirrus,micbias-input", &val);
+ if (ret >= 0) {
+ mutex_lock(&lochnagar->analogue_config_lock);
+ ret = regmap_update_bits(lochnagar->regmap,
+ LOCHNAGAR2_ANALOGUE_PATH_CTRL2,
+ mask, val << shift);
+ mutex_unlock(&lochnagar->analogue_config_lock);
+ if (ret < 0) {
+ dev_err(lochnagar->dev,
+ "Failed to update micbias source: %d\n", ret);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static const struct regulator_desc lochnagar_regulators[] = {
+ [LOCHNAGAR_MICVDD] = {
+ .name = "MICVDD",
+ .supply_name = "SYSVDD",
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = 32,
+ .ops = &lochnagar_micvdd_ops,
+
+ .id = LOCHNAGAR_MICVDD,
+ .of_match = of_match_ptr("MICVDD"),
+
+ .enable_reg = LOCHNAGAR2_MICVDD_CTRL1,
+ .enable_mask = LOCHNAGAR2_MICVDD_REG_ENA_MASK,
+ .vsel_reg = LOCHNAGAR2_MICVDD_CTRL2,
+ .vsel_mask = LOCHNAGAR2_MICVDD_VSEL_MASK,
+
+ .linear_ranges = lochnagar_micvdd_ranges,
+ .n_linear_ranges = ARRAY_SIZE(lochnagar_micvdd_ranges),
+
+ .enable_time = 3000,
+ .ramp_delay = 1000,
+
+ .owner = THIS_MODULE,
+ },
+ [LOCHNAGAR_MIC1VDD] = {
+ .name = "MIC1VDD",
+ .supply_name = "MICBIAS1",
+ .type = REGULATOR_VOLTAGE,
+ .ops = &lochnagar_micbias_ops,
+
+ .id = LOCHNAGAR_MIC1VDD,
+ .of_match = of_match_ptr("MIC1VDD"),
+ .of_parse_cb = lochnagar_micbias_of_parse,
+
+ .enable_reg = LOCHNAGAR2_ANALOGUE_PATH_CTRL2,
+ .enable_mask = LOCHNAGAR2_P1_INPUT_BIAS_ENA_MASK,
+
+ .owner = THIS_MODULE,
+ },
+ [LOCHNAGAR_MIC2VDD] = {
+ .name = "MIC2VDD",
+ .supply_name = "MICBIAS2",
+ .type = REGULATOR_VOLTAGE,
+ .ops = &lochnagar_micbias_ops,
+
+ .id = LOCHNAGAR_MIC2VDD,
+ .of_match = of_match_ptr("MIC2VDD"),
+ .of_parse_cb = lochnagar_micbias_of_parse,
+
+ .enable_reg = LOCHNAGAR2_ANALOGUE_PATH_CTRL2,
+ .enable_mask = LOCHNAGAR2_P2_INPUT_BIAS_ENA_MASK,
+
+ .owner = THIS_MODULE,
+ },
+ [LOCHNAGAR_VDDCORE] = {
+ .name = "VDDCORE",
+ .supply_name = "SYSVDD",
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = 57,
+ .ops = &lochnagar_vddcore_ops,
+
+ .id = LOCHNAGAR_VDDCORE,
+ .of_match = of_match_ptr("VDDCORE"),
+
+ .enable_reg = LOCHNAGAR2_VDDCORE_CDC_CTRL1,
+ .enable_mask = LOCHNAGAR2_VDDCORE_CDC_REG_ENA_MASK,
+ .vsel_reg = LOCHNAGAR2_VDDCORE_CDC_CTRL2,
+ .vsel_mask = LOCHNAGAR2_VDDCORE_CDC_VSEL_MASK,
+
+ .linear_ranges = lochnagar_vddcore_ranges,
+ .n_linear_ranges = ARRAY_SIZE(lochnagar_vddcore_ranges),
+
+ .enable_time = 3000,
+ .ramp_delay = 1000,
+
+ .owner = THIS_MODULE,
+ },
+};
+
+static int lochnagar_regulator_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct lochnagar *lochnagar = dev_get_drvdata(dev->parent);
+ struct regulator_config config = { };
+ struct regulator_dev *rdev;
+ int ret, i;
+
+ config.dev = lochnagar->dev;
+ config.regmap = lochnagar->regmap;
+ config.driver_data = lochnagar;
+
+ for (i = 0; i < ARRAY_SIZE(lochnagar_regulators); i++) {
+ const struct regulator_desc *desc = &lochnagar_regulators[i];
+
+ rdev = devm_regulator_register(dev, desc, &config);
+ if (IS_ERR(rdev)) {
+ ret = PTR_ERR(rdev);
+ dev_err(dev, "Failed to register %s regulator: %d\n",
+ desc->name, ret);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static struct platform_driver lochnagar_regulator_driver = {
+ .driver = {
+ .name = "lochnagar-regulator",
+ },
+
+ .probe = lochnagar_regulator_probe,
+};
+module_platform_driver(lochnagar_regulator_driver);
+
+MODULE_AUTHOR("Charles Keepax <ckeepax@opensource.cirrus.com>");
+MODULE_DESCRIPTION("Regulator driver for Cirrus Logic Lochnagar Board");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:lochnagar-regulator");
/* FIXME: check default mode for GPIO here: high or low? */
ldo->ena_gpiod = devm_gpiod_get_index_optional(&pdev->dev,
- "enable",
- enable_id,
- GPIOD_OUT_HIGH);
+ "enable",
+ enable_id,
+ GPIOD_OUT_HIGH |
+ GPIOD_FLAGS_BIT_NONEXCLUSIVE);
if (IS_ERR(ldo->ena_gpiod))
return PTR_ERR(ldo->ena_gpiod);
.max_register = LTC3589_L2DTV2,
.reg_defaults = ltc3589_reg_defaults,
.num_reg_defaults = ARRAY_SIZE(ltc3589_reg_defaults),
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.cache_type = REGCACHE_RBTREE,
};
.readable_reg = ltc3676_readable_reg,
.volatile_reg = ltc3676_volatile_reg,
.max_register = LTC3676_CLIRQ,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.cache_type = REGCACHE_RBTREE,
};
gflags = GPIOD_OUT_HIGH;
else
gflags = GPIOD_OUT_LOW;
+ gflags |= GPIOD_FLAGS_BIT_NONEXCLUSIVE;
gpiod = devm_gpiod_get_optional(&client->dev,
"max8952,en",
gflags);
gflags = GPIOD_OUT_HIGH;
else
gflags = GPIOD_OUT_LOW;
+ gflags |= GPIOD_FLAGS_BIT_NONEXCLUSIVE;
gpiod = devm_gpiod_get_optional(&client->dev,
"maxim,enable",
gflags);
break;
if (i == ARRAY_SIZE(regulators)) {
- dev_warn(&pdev->dev, "don't know how to configure regulator %s\n",
- reg_np->name);
+ dev_warn(&pdev->dev, "don't know how to configure regulator %pOFn\n",
+ reg_np);
continue;
}
if (!found)
dev_warn(&pdev->dev,
- "Unknown regulator: %s\n", child->name);
+ "Unknown regulator: %pOFn\n", child);
}
of_node_put(parent);
if (!ret)
constraints->settling_time_up = pval;
if (constraints->settling_time_up && constraints->settling_time) {
- pr_warn("%s: ambiguous configuration for settling time, ignoring 'regulator-settling-time-up-us'\n",
- np->name);
+ pr_warn("%pOFn: ambiguous configuration for settling time, ignoring 'regulator-settling-time-up-us'\n",
+ np);
constraints->settling_time_up = 0;
}
if (!ret)
constraints->settling_time_down = pval;
if (constraints->settling_time_down && constraints->settling_time) {
- pr_warn("%s: ambiguous configuration for settling time, ignoring 'regulator-settling-time-down-us'\n",
- np->name);
+ pr_warn("%pOFn: ambiguous configuration for settling time, ignoring 'regulator-settling-time-down-us'\n",
+ np);
constraints->settling_time_down = 0;
}
if (desc && desc->of_map_mode) {
mode = desc->of_map_mode(pval);
if (mode == REGULATOR_MODE_INVALID)
- pr_err("%s: invalid mode %u\n", np->name, pval);
+ pr_err("%pOFn: invalid mode %u\n", np, pval);
else
constraints->initial_mode = mode;
} else {
- pr_warn("%s: mapping for mode %d not defined\n",
- np->name, pval);
+ pr_warn("%pOFn: mapping for mode %d not defined\n",
+ np, pval);
}
}
ret = of_property_read_u32_index(np,
"regulator-allowed-modes", i, &pval);
if (ret) {
- pr_err("%s: couldn't read allowed modes index %d, ret=%d\n",
- np->name, i, ret);
+ pr_err("%pOFn: couldn't read allowed modes index %d, ret=%d\n",
+ np, i, ret);
break;
}
mode = desc->of_map_mode(pval);
if (mode == REGULATOR_MODE_INVALID)
- pr_err("%s: invalid regulator-allowed-modes element %u\n",
- np->name, pval);
+ pr_err("%pOFn: invalid regulator-allowed-modes element %u\n",
+ np, pval);
else
constraints->valid_modes_mask |= mode;
}
constraints->valid_ops_mask
|= REGULATOR_CHANGE_MODE;
} else {
- pr_warn("%s: mode mapping not defined\n", np->name);
+ pr_warn("%pOFn: mode mapping not defined\n", np);
}
}
if (desc && desc->of_map_mode) {
mode = desc->of_map_mode(pval);
if (mode == REGULATOR_MODE_INVALID)
- pr_err("%s: invalid mode %u\n",
- np->name, pval);
+ pr_err("%pOFn: invalid mode %u\n",
+ np, pval);
else
suspend_state->mode = mode;
} else {
- pr_warn("%s: mapping for mode %d not defined\n",
- np->name, pval);
+ pr_warn("%pOFn: mapping for mode %d not defined\n",
+ np, pval);
}
}
match->desc);
if (!match->init_data) {
dev_err(dev,
- "failed to parse DT for regulator %s\n",
- child->name);
+ "failed to parse DT for regulator %pOFn\n",
+ child);
of_node_put(child);
return -EINVAL;
}
init_data = of_get_regulator_init_data(dev, child, desc);
if (!init_data) {
dev_err(dev,
- "failed to parse DT for regulator %s\n",
- child->name);
+ "failed to parse DT for regulator %pOFn\n",
+ child);
break;
}
if (desc->of_parse_cb) {
if (desc->of_parse_cb(child, desc, config)) {
dev_err(dev,
- "driver callback failed to parse DT for regulator %s\n",
- child->name);
+ "driver callback failed to parse DT for regulator %pOFn\n",
+ child);
init_data = NULL;
break;
}
#define PFUZE100_COINVOL 0x1a
#define PFUZE100_SW1ABVOL 0x20
+#define PFUZE100_SW1ABMODE 0x23
#define PFUZE100_SW1CVOL 0x2e
+#define PFUZE100_SW1CMODE 0x31
#define PFUZE100_SW2VOL 0x35
+#define PFUZE100_SW2MODE 0x38
#define PFUZE100_SW3AVOL 0x3c
+#define PFUZE100_SW3AMODE 0x3f
#define PFUZE100_SW3BVOL 0x43
+#define PFUZE100_SW3BMODE 0x46
#define PFUZE100_SW4VOL 0x4a
+#define PFUZE100_SW4MODE 0x4d
#define PFUZE100_SWBSTCON1 0x66
#define PFUZE100_VREFDDRCON 0x6a
#define PFUZE100_VSNVSVOL 0x6b
#define PFUZE100_VGEN5VOL 0x70
#define PFUZE100_VGEN6VOL 0x71
+#define PFUZE100_SWxMODE_MASK 0xf
+#define PFUZE100_SWxMODE_APS_APS 0x8
+#define PFUZE100_SWxMODE_APS_OFF 0x4
+
+#define PFUZE100_VGENxLPWR BIT(6)
+#define PFUZE100_VGENxSTBY BIT(5)
+
enum chips { PFUZE100, PFUZE200, PFUZE3000 = 3, PFUZE3001 = 0x31, };
struct pfuze_regulator {
}
#endif
+static struct pfuze_chip *syspm_pfuze_chip;
+
+static void pfuze_power_off_prepare(void)
+{
+ dev_info(syspm_pfuze_chip->dev, "Configure standby mode for power off");
+
+ /* Switch from default mode: APS/APS to APS/Off */
+ regmap_update_bits(syspm_pfuze_chip->regmap, PFUZE100_SW1ABMODE,
+ PFUZE100_SWxMODE_MASK, PFUZE100_SWxMODE_APS_OFF);
+ regmap_update_bits(syspm_pfuze_chip->regmap, PFUZE100_SW1CMODE,
+ PFUZE100_SWxMODE_MASK, PFUZE100_SWxMODE_APS_OFF);
+ regmap_update_bits(syspm_pfuze_chip->regmap, PFUZE100_SW2MODE,
+ PFUZE100_SWxMODE_MASK, PFUZE100_SWxMODE_APS_OFF);
+ regmap_update_bits(syspm_pfuze_chip->regmap, PFUZE100_SW3AMODE,
+ PFUZE100_SWxMODE_MASK, PFUZE100_SWxMODE_APS_OFF);
+ regmap_update_bits(syspm_pfuze_chip->regmap, PFUZE100_SW3BMODE,
+ PFUZE100_SWxMODE_MASK, PFUZE100_SWxMODE_APS_OFF);
+ regmap_update_bits(syspm_pfuze_chip->regmap, PFUZE100_SW4MODE,
+ PFUZE100_SWxMODE_MASK, PFUZE100_SWxMODE_APS_OFF);
+
+ regmap_update_bits(syspm_pfuze_chip->regmap, PFUZE100_VGEN1VOL,
+ PFUZE100_VGENxLPWR | PFUZE100_VGENxSTBY,
+ PFUZE100_VGENxSTBY);
+ regmap_update_bits(syspm_pfuze_chip->regmap, PFUZE100_VGEN2VOL,
+ PFUZE100_VGENxLPWR | PFUZE100_VGENxSTBY,
+ PFUZE100_VGENxSTBY);
+ regmap_update_bits(syspm_pfuze_chip->regmap, PFUZE100_VGEN3VOL,
+ PFUZE100_VGENxLPWR | PFUZE100_VGENxSTBY,
+ PFUZE100_VGENxSTBY);
+ regmap_update_bits(syspm_pfuze_chip->regmap, PFUZE100_VGEN4VOL,
+ PFUZE100_VGENxLPWR | PFUZE100_VGENxSTBY,
+ PFUZE100_VGENxSTBY);
+ regmap_update_bits(syspm_pfuze_chip->regmap, PFUZE100_VGEN5VOL,
+ PFUZE100_VGENxLPWR | PFUZE100_VGENxSTBY,
+ PFUZE100_VGENxSTBY);
+ regmap_update_bits(syspm_pfuze_chip->regmap, PFUZE100_VGEN6VOL,
+ PFUZE100_VGENxLPWR | PFUZE100_VGENxSTBY,
+ PFUZE100_VGENxSTBY);
+}
+
+static int pfuze_power_off_prepare_init(struct pfuze_chip *pfuze_chip)
+{
+ if (pfuze_chip->chip_id != PFUZE100) {
+ dev_warn(pfuze_chip->dev, "Requested pm_power_off_prepare handler for not supported chip\n");
+ return -ENODEV;
+ }
+
+ if (pm_power_off_prepare) {
+ dev_warn(pfuze_chip->dev, "pm_power_off_prepare is already registered.\n");
+ return -EBUSY;
+ }
+
+ if (syspm_pfuze_chip) {
+ dev_warn(pfuze_chip->dev, "syspm_pfuze_chip is already set.\n");
+ return -EBUSY;
+ }
+
+ syspm_pfuze_chip = pfuze_chip;
+ pm_power_off_prepare = pfuze_power_off_prepare;
+
+ return 0;
+}
+
static int pfuze_identify(struct pfuze_chip *pfuze_chip)
{
unsigned int value;
}
}
+ if (of_property_read_bool(client->dev.of_node,
+ "fsl,pmic-stby-poweroff"))
+ return pfuze_power_off_prepare_init(pfuze_chip);
+
+ return 0;
+}
+
+static int pfuze100_regulator_remove(struct i2c_client *client)
+{
+ if (syspm_pfuze_chip) {
+ syspm_pfuze_chip = NULL;
+ pm_power_off_prepare = NULL;
+ }
+
return 0;
}
.of_match_table = pfuze_dt_ids,
},
.probe = pfuze100_regulator_probe,
+ .remove = pfuze100_regulator_remove,
};
module_i2c_driver(pfuze_driver);
break;
if (!rpmh_data->name) {
- dev_err(dev, "Unknown regulator %s\n", node->name);
+ dev_err(dev, "Unknown regulator %pOFn\n", node);
return -EINVAL;
}
vreg->addr = cmd_db_read_addr(rpmh_resource_name);
if (!vreg->addr) {
- dev_err(dev, "%s: could not find RPMh address for resource %s\n",
- node->name, rpmh_resource_name);
+ dev_err(dev, "%pOFn: could not find RPMh address for resource %s\n",
+ node, rpmh_resource_name);
return -ENODEV;
}
rdev = devm_regulator_register(dev, &vreg->rdesc, ®_config);
if (IS_ERR(rdev)) {
ret = PTR_ERR(rdev);
- dev_err(dev, "%s: devm_regulator_register() failed, ret=%d\n",
- node->name, ret);
+ dev_err(dev, "%pOFn: devm_regulator_register() failed, ret=%d\n",
+ node, ret);
return ret;
}
- dev_dbg(dev, "%s regulator registered for RPMh resource %s @ 0x%05X\n",
- node->name, rpmh_resource_name, vreg->addr);
+ dev_dbg(dev, "%pOFn regulator registered for RPMh resource %s @ 0x%05X\n",
+ node, rpmh_resource_name, vreg->addr);
return 0;
}
break;
default:
mode = REGULATOR_MODE_INVALID;
+ break;
}
return mode;
break;
default:
mode = REGULATOR_MODE_INVALID;
+ break;
}
return mode;
break;
default:
mode = REGULATOR_MODE_INVALID;
+ break;
}
return mode;
.ops = &rpm_bob_ops,
};
+static const struct regulator_desc pms405_hfsmps3 = {
+ .linear_ranges = (struct regulator_linear_range[]) {
+ REGULATOR_LINEAR_RANGE(320000, 0, 215, 8000),
+ },
+ .n_linear_ranges = 1,
+ .n_voltages = 216,
+ .ops = &rpm_smps_ldo_ops,
+};
+
+static const struct regulator_desc pms405_nldo300 = {
+ .linear_ranges = (struct regulator_linear_range[]) {
+ REGULATOR_LINEAR_RANGE(312000, 0, 127, 8000),
+ },
+ .n_linear_ranges = 1,
+ .n_voltages = 128,
+ .ops = &rpm_smps_ldo_ops,
+};
+
+static const struct regulator_desc pms405_nldo1200 = {
+ .linear_ranges = (struct regulator_linear_range[]) {
+ REGULATOR_LINEAR_RANGE(312000, 0, 127, 8000),
+ },
+ .n_linear_ranges = 1,
+ .n_voltages = 128,
+ .ops = &rpm_smps_ldo_ops,
+};
+
+static const struct regulator_desc pms405_pldo50 = {
+ .linear_ranges = (struct regulator_linear_range[]) {
+ REGULATOR_LINEAR_RANGE(1664000, 0, 128, 16000),
+ },
+ .n_linear_ranges = 1,
+ .n_voltages = 129,
+ .ops = &rpm_smps_ldo_ops,
+};
+
+static const struct regulator_desc pms405_pldo150 = {
+ .linear_ranges = (struct regulator_linear_range[]) {
+ REGULATOR_LINEAR_RANGE(1664000, 0, 128, 16000),
+ },
+ .n_linear_ranges = 1,
+ .n_voltages = 129,
+ .ops = &rpm_smps_ldo_ops,
+};
+
+static const struct regulator_desc pms405_pldo600 = {
+ .linear_ranges = (struct regulator_linear_range[]) {
+ REGULATOR_LINEAR_RANGE(1256000, 0, 98, 8000),
+ },
+ .n_linear_ranges = 1,
+ .n_voltages = 99,
+ .ops = &rpm_smps_ldo_ops,
+};
+
struct rpm_regulator_data {
const char *name;
u32 type;
{}
};
+static const struct rpm_regulator_data rpm_pms405_regulators[] = {
+ { "s1", QCOM_SMD_RPM_SMPA, 1, &pms405_hfsmps3, "vdd_s1" },
+ { "s2", QCOM_SMD_RPM_SMPA, 2, &pms405_hfsmps3, "vdd_s2" },
+ { "s3", QCOM_SMD_RPM_SMPA, 3, &pms405_hfsmps3, "vdd_s3" },
+ { "s4", QCOM_SMD_RPM_SMPA, 4, &pms405_hfsmps3, "vdd_s4" },
+ { "s5", QCOM_SMD_RPM_SMPA, 5, &pms405_hfsmps3, "vdd_s5" },
+ { "l1", QCOM_SMD_RPM_LDOA, 1, &pms405_nldo1200, "vdd_l1_l2" },
+ { "l2", QCOM_SMD_RPM_LDOA, 2, &pms405_nldo1200, "vdd_l1_l2" },
+ { "l3", QCOM_SMD_RPM_LDOA, 3, &pms405_nldo1200, "vdd_l3_l8" },
+ { "l4", QCOM_SMD_RPM_LDOA, 4, &pms405_nldo300, "vdd_l4" },
+ { "l5", QCOM_SMD_RPM_LDOA, 5, &pms405_pldo600, "vdd_l5_l6" },
+ { "l6", QCOM_SMD_RPM_LDOA, 6, &pms405_pldo600, "vdd_l5_l6" },
+ { "l7", QCOM_SMD_RPM_LDOA, 7, &pms405_pldo150, "vdd_l7" },
+ { "l8", QCOM_SMD_RPM_LDOA, 8, &pms405_nldo1200, "vdd_l3_l8" },
+ { "l9", QCOM_SMD_RPM_LDOA, 9, &pms405_nldo1200, "vdd_l9" },
+ { "l10", QCOM_SMD_RPM_LDOA, 10, &pms405_pldo50, "vdd_l10_l11_l12_l13" },
+ { "l11", QCOM_SMD_RPM_LDOA, 11, &pms405_pldo150, "vdd_l10_l11_l12_l13" },
+ { "l12", QCOM_SMD_RPM_LDOA, 12, &pms405_pldo150, "vdd_l10_l11_l12_l13" },
+ { "l13", QCOM_SMD_RPM_LDOA, 13, &pms405_pldo150, "vdd_l10_l11_l12_l13" },
+ {}
+};
+
static const struct of_device_id rpm_of_match[] = {
{ .compatible = "qcom,rpm-pm8841-regulators", .data = &rpm_pm8841_regulators },
{ .compatible = "qcom,rpm-pm8916-regulators", .data = &rpm_pm8916_regulators },
{ .compatible = "qcom,rpm-pm8998-regulators", .data = &rpm_pm8998_regulators },
{ .compatible = "qcom,rpm-pma8084-regulators", .data = &rpm_pma8084_regulators },
{ .compatible = "qcom,rpm-pmi8998-regulators", .data = &rpm_pmi8998_regulators },
+ { .compatible = "qcom,rpm-pms405-regulators", .data = &rpm_pms405_regulators },
{}
};
MODULE_DEVICE_TABLE(of, rpm_of_match);
}
if (mode != S5M8767_ENCTRL_USE_GPIO) {
dev_warn(s5m8767->dev,
- "ext-control for %s: mismatched op_mode (%x), ignoring\n",
- rdata->reg_node->name, mode);
+ "ext-control for %pOFn: mismatched op_mode (%x), ignoring\n",
+ rdata->reg_node, mode);
return;
}
if (!rdata->ext_control_gpiod) {
dev_warn(s5m8767->dev,
- "ext-control for %s: GPIO not valid, ignoring\n",
- rdata->reg_node->name);
+ "ext-control for %pOFn: GPIO not valid, ignoring\n",
+ rdata->reg_node);
return;
}
if (i == ARRAY_SIZE(regulators)) {
dev_warn(iodev->dev,
- "don't know how to configure regulator %s\n",
- reg_np->name);
+ "don't know how to configure regulator %pOFn\n",
+ reg_np);
continue;
}
- rdata->ext_control_gpiod = devm_gpiod_get_from_of_node(&pdev->dev,
- reg_np,
- "s5m8767,pmic-ext-control-gpios",
- 0,
- GPIOD_OUT_HIGH,
- "s5m8767");
+ rdata->ext_control_gpiod = devm_gpiod_get_from_of_node(
+ &pdev->dev,
+ reg_np,
+ "s5m8767,pmic-ext-control-gpios",
+ 0,
+ GPIOD_OUT_HIGH | GPIOD_FLAGS_BIT_NONEXCLUSIVE,
+ "s5m8767");
if (IS_ERR(rdata->ext_control_gpiod))
return PTR_ERR(rdata->ext_control_gpiod);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) STMicroelectronics 2018
+// Author: Pascal Paillet <p.paillet@st.com> for STMicroelectronics.
+
+#include <linux/interrupt.h>
+#include <linux/mfd/stpmic1.h>
+#include <linux/module.h>
+#include <linux/of_irq.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <linux/regulator/driver.h>
+#include <linux/regulator/machine.h>
+#include <linux/regulator/of_regulator.h>
+
+/**
+ * stpmic1 regulator description
+ * @desc: regulator framework description
+ * @mask_reset_reg: mask reset register address
+ * @mask_reset_mask: mask rank and mask reset register mask
+ * @icc_reg: icc register address
+ * @icc_mask: icc register mask
+ */
+struct stpmic1_regulator_cfg {
+ struct regulator_desc desc;
+ u8 mask_reset_reg;
+ u8 mask_reset_mask;
+ u8 icc_reg;
+ u8 icc_mask;
+};
+
+/**
+ * stpmic1 regulator data: this structure is used as driver data
+ * @regul_id: regulator id
+ * @reg_node: DT node of regulator (unused on non-DT platforms)
+ * @cfg: stpmic specific regulator description
+ * @mask_reset: mask_reset bit value
+ * @irq_curlim: current limit interrupt number
+ * @regmap: point to parent regmap structure
+ */
+struct stpmic1_regulator {
+ unsigned int regul_id;
+ struct device_node *reg_node;
+ struct stpmic1_regulator_cfg *cfg;
+ u8 mask_reset;
+ int irq_curlim;
+ struct regmap *regmap;
+};
+
+static int stpmic1_set_mode(struct regulator_dev *rdev, unsigned int mode);
+static unsigned int stpmic1_get_mode(struct regulator_dev *rdev);
+static int stpmic1_set_icc(struct regulator_dev *rdev);
+static int stpmic1_regulator_parse_dt(void *driver_data);
+static unsigned int stpmic1_map_mode(unsigned int mode);
+
+enum {
+ STPMIC1_BUCK1 = 0,
+ STPMIC1_BUCK2 = 1,
+ STPMIC1_BUCK3 = 2,
+ STPMIC1_BUCK4 = 3,
+ STPMIC1_LDO1 = 4,
+ STPMIC1_LDO2 = 5,
+ STPMIC1_LDO3 = 6,
+ STPMIC1_LDO4 = 7,
+ STPMIC1_LDO5 = 8,
+ STPMIC1_LDO6 = 9,
+ STPMIC1_VREF_DDR = 10,
+ STPMIC1_BOOST = 11,
+ STPMIC1_VBUS_OTG = 12,
+ STPMIC1_SW_OUT = 13,
+};
+
+/* Enable time worst case is 5000mV/(2250uV/uS) */
+#define PMIC_ENABLE_TIME_US 2200
+
+#define STPMIC1_BUCK_MODE_NORMAL 0
+#define STPMIC1_BUCK_MODE_LP BUCK_HPLP_ENABLE_MASK
+
+struct regulator_linear_range buck1_ranges[] = {
+ REGULATOR_LINEAR_RANGE(600000, 0, 30, 25000),
+ REGULATOR_LINEAR_RANGE(1350000, 31, 63, 0),
+};
+
+struct regulator_linear_range buck2_ranges[] = {
+ REGULATOR_LINEAR_RANGE(1000000, 0, 17, 0),
+ REGULATOR_LINEAR_RANGE(1050000, 18, 19, 0),
+ REGULATOR_LINEAR_RANGE(1100000, 20, 21, 0),
+ REGULATOR_LINEAR_RANGE(1150000, 22, 23, 0),
+ REGULATOR_LINEAR_RANGE(1200000, 24, 25, 0),
+ REGULATOR_LINEAR_RANGE(1250000, 26, 27, 0),
+ REGULATOR_LINEAR_RANGE(1300000, 28, 29, 0),
+ REGULATOR_LINEAR_RANGE(1350000, 30, 31, 0),
+ REGULATOR_LINEAR_RANGE(1400000, 32, 33, 0),
+ REGULATOR_LINEAR_RANGE(1450000, 34, 35, 0),
+ REGULATOR_LINEAR_RANGE(1500000, 36, 63, 0),
+};
+
+struct regulator_linear_range buck3_ranges[] = {
+ REGULATOR_LINEAR_RANGE(1000000, 0, 19, 0),
+ REGULATOR_LINEAR_RANGE(1100000, 20, 23, 0),
+ REGULATOR_LINEAR_RANGE(1200000, 24, 27, 0),
+ REGULATOR_LINEAR_RANGE(1300000, 28, 31, 0),
+ REGULATOR_LINEAR_RANGE(1400000, 32, 35, 0),
+ REGULATOR_LINEAR_RANGE(1500000, 36, 55, 100000),
+ REGULATOR_LINEAR_RANGE(3400000, 56, 63, 0),
+
+};
+
+struct regulator_linear_range buck4_ranges[] = {
+ REGULATOR_LINEAR_RANGE(600000, 0, 27, 25000),
+ REGULATOR_LINEAR_RANGE(1300000, 28, 29, 0),
+ REGULATOR_LINEAR_RANGE(1350000, 30, 31, 0),
+ REGULATOR_LINEAR_RANGE(1400000, 32, 33, 0),
+ REGULATOR_LINEAR_RANGE(1450000, 34, 35, 0),
+ REGULATOR_LINEAR_RANGE(1500000, 36, 60, 100000),
+ REGULATOR_LINEAR_RANGE(3900000, 61, 63, 0),
+
+};
+
+struct regulator_linear_range ldo1_ranges[] = {
+ REGULATOR_LINEAR_RANGE(1700000, 0, 7, 0),
+ REGULATOR_LINEAR_RANGE(1700000, 8, 24, 100000),
+ REGULATOR_LINEAR_RANGE(3300000, 25, 31, 0),
+
+};
+
+struct regulator_linear_range ldo2_ranges[] = {
+ REGULATOR_LINEAR_RANGE(1700000, 0, 7, 0),
+ REGULATOR_LINEAR_RANGE(1700000, 8, 24, 100000),
+ REGULATOR_LINEAR_RANGE(3300000, 25, 30, 0),
+
+};
+
+struct regulator_linear_range ldo3_ranges[] = {
+ REGULATOR_LINEAR_RANGE(1700000, 0, 7, 0),
+ REGULATOR_LINEAR_RANGE(1700000, 8, 24, 100000),
+ REGULATOR_LINEAR_RANGE(3300000, 25, 30, 0),
+ /* with index 31 LDO3 is in DDR mode */
+ REGULATOR_LINEAR_RANGE(500000, 31, 31, 0),
+};
+
+struct regulator_linear_range ldo5_ranges[] = {
+ REGULATOR_LINEAR_RANGE(1700000, 0, 7, 0),
+ REGULATOR_LINEAR_RANGE(1700000, 8, 30, 100000),
+ REGULATOR_LINEAR_RANGE(3900000, 31, 31, 0),
+};
+
+struct regulator_linear_range ldo6_ranges[] = {
+ REGULATOR_LINEAR_RANGE(900000, 0, 24, 100000),
+ REGULATOR_LINEAR_RANGE(3300000, 25, 31, 0),
+};
+
+static struct regulator_ops stpmic1_ldo_ops = {
+ .list_voltage = regulator_list_voltage_linear_range,
+ .map_voltage = regulator_map_voltage_linear_range,
+ .is_enabled = regulator_is_enabled_regmap,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+ .set_pull_down = regulator_set_pull_down_regmap,
+ .set_over_current_protection = stpmic1_set_icc,
+};
+
+static struct regulator_ops stpmic1_ldo3_ops = {
+ .list_voltage = regulator_list_voltage_linear_range,
+ .map_voltage = regulator_map_voltage_iterate,
+ .is_enabled = regulator_is_enabled_regmap,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+ .set_pull_down = regulator_set_pull_down_regmap,
+ .get_bypass = regulator_get_bypass_regmap,
+ .set_bypass = regulator_set_bypass_regmap,
+ .set_over_current_protection = stpmic1_set_icc,
+};
+
+static struct regulator_ops stpmic1_ldo4_fixed_regul_ops = {
+ .is_enabled = regulator_is_enabled_regmap,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .set_pull_down = regulator_set_pull_down_regmap,
+ .set_over_current_protection = stpmic1_set_icc,
+};
+
+static struct regulator_ops stpmic1_buck_ops = {
+ .list_voltage = regulator_list_voltage_linear_range,
+ .map_voltage = regulator_map_voltage_linear_range,
+ .is_enabled = regulator_is_enabled_regmap,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+ .set_pull_down = regulator_set_pull_down_regmap,
+ .set_mode = stpmic1_set_mode,
+ .get_mode = stpmic1_get_mode,
+ .set_over_current_protection = stpmic1_set_icc,
+};
+
+static struct regulator_ops stpmic1_vref_ddr_ops = {
+ .is_enabled = regulator_is_enabled_regmap,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .set_pull_down = regulator_set_pull_down_regmap,
+};
+
+static struct regulator_ops stpmic1_switch_regul_ops = {
+ .is_enabled = regulator_is_enabled_regmap,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .set_over_current_protection = stpmic1_set_icc,
+};
+
+#define REG_LDO(ids, base) { \
+ .name = #ids, \
+ .id = STPMIC1_##ids, \
+ .n_voltages = 32, \
+ .ops = &stpmic1_ldo_ops, \
+ .linear_ranges = base ## _ranges, \
+ .n_linear_ranges = ARRAY_SIZE(base ## _ranges), \
+ .type = REGULATOR_VOLTAGE, \
+ .owner = THIS_MODULE, \
+ .vsel_reg = ids##_ACTIVE_CR, \
+ .vsel_mask = LDO_VOLTAGE_MASK, \
+ .enable_reg = ids##_ACTIVE_CR, \
+ .enable_mask = LDO_ENABLE_MASK, \
+ .enable_val = 1, \
+ .disable_val = 0, \
+ .enable_time = PMIC_ENABLE_TIME_US, \
+ .pull_down_reg = ids##_PULL_DOWN_REG, \
+ .pull_down_mask = ids##_PULL_DOWN_MASK, \
+ .supply_name = #base, \
+}
+
+#define REG_LDO3(ids, base) { \
+ .name = #ids, \
+ .id = STPMIC1_##ids, \
+ .n_voltages = 32, \
+ .ops = &stpmic1_ldo3_ops, \
+ .linear_ranges = ldo3_ranges, \
+ .n_linear_ranges = ARRAY_SIZE(ldo3_ranges), \
+ .type = REGULATOR_VOLTAGE, \
+ .owner = THIS_MODULE, \
+ .vsel_reg = LDO3_ACTIVE_CR, \
+ .vsel_mask = LDO_VOLTAGE_MASK, \
+ .enable_reg = LDO3_ACTIVE_CR, \
+ .enable_mask = LDO_ENABLE_MASK, \
+ .enable_val = 1, \
+ .disable_val = 0, \
+ .enable_time = PMIC_ENABLE_TIME_US, \
+ .bypass_reg = LDO3_ACTIVE_CR, \
+ .bypass_mask = LDO_BYPASS_MASK, \
+ .bypass_val_on = LDO_BYPASS_MASK, \
+ .bypass_val_off = 0, \
+ .pull_down_reg = ids##_PULL_DOWN_REG, \
+ .pull_down_mask = ids##_PULL_DOWN_MASK, \
+ .supply_name = #base, \
+}
+
+#define REG_LDO4(ids, base) { \
+ .name = #ids, \
+ .id = STPMIC1_##ids, \
+ .n_voltages = 1, \
+ .ops = &stpmic1_ldo4_fixed_regul_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .owner = THIS_MODULE, \
+ .min_uV = 3300000, \
+ .fixed_uV = 3300000, \
+ .enable_reg = LDO4_ACTIVE_CR, \
+ .enable_mask = LDO_ENABLE_MASK, \
+ .enable_val = 1, \
+ .disable_val = 0, \
+ .enable_time = PMIC_ENABLE_TIME_US, \
+ .pull_down_reg = ids##_PULL_DOWN_REG, \
+ .pull_down_mask = ids##_PULL_DOWN_MASK, \
+ .supply_name = #base, \
+}
+
+#define REG_BUCK(ids, base) { \
+ .name = #ids, \
+ .id = STPMIC1_##ids, \
+ .ops = &stpmic1_buck_ops, \
+ .n_voltages = 64, \
+ .linear_ranges = base ## _ranges, \
+ .n_linear_ranges = ARRAY_SIZE(base ## _ranges), \
+ .type = REGULATOR_VOLTAGE, \
+ .owner = THIS_MODULE, \
+ .vsel_reg = ids##_ACTIVE_CR, \
+ .vsel_mask = BUCK_VOLTAGE_MASK, \
+ .enable_reg = ids##_ACTIVE_CR, \
+ .enable_mask = BUCK_ENABLE_MASK, \
+ .enable_val = 1, \
+ .disable_val = 0, \
+ .enable_time = PMIC_ENABLE_TIME_US, \
+ .of_map_mode = stpmic1_map_mode, \
+ .pull_down_reg = ids##_PULL_DOWN_REG, \
+ .pull_down_mask = ids##_PULL_DOWN_MASK, \
+ .supply_name = #base, \
+}
+
+#define REG_VREF_DDR(ids, base) { \
+ .name = #ids, \
+ .id = STPMIC1_##ids, \
+ .n_voltages = 1, \
+ .ops = &stpmic1_vref_ddr_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .owner = THIS_MODULE, \
+ .min_uV = 500000, \
+ .fixed_uV = 500000, \
+ .enable_reg = VREF_DDR_ACTIVE_CR, \
+ .enable_mask = BUCK_ENABLE_MASK, \
+ .enable_val = 1, \
+ .disable_val = 0, \
+ .enable_time = PMIC_ENABLE_TIME_US, \
+ .pull_down_reg = ids##_PULL_DOWN_REG, \
+ .pull_down_mask = ids##_PULL_DOWN_MASK, \
+ .supply_name = #base, \
+}
+
+#define REG_SWITCH(ids, base, reg, mask, val) { \
+ .name = #ids, \
+ .id = STPMIC1_##ids, \
+ .n_voltages = 1, \
+ .ops = &stpmic1_switch_regul_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .owner = THIS_MODULE, \
+ .min_uV = 0, \
+ .fixed_uV = 5000000, \
+ .enable_reg = (reg), \
+ .enable_mask = (mask), \
+ .enable_val = (val), \
+ .disable_val = 0, \
+ .enable_time = PMIC_ENABLE_TIME_US, \
+ .supply_name = #base, \
+}
+
+struct stpmic1_regulator_cfg stpmic1_regulator_cfgs[] = {
+ [STPMIC1_BUCK1] = {
+ .desc = REG_BUCK(BUCK1, buck1),
+ .icc_reg = BUCKS_ICCTO_CR,
+ .icc_mask = BIT(0),
+ .mask_reset_reg = BUCKS_MASK_RESET_CR,
+ .mask_reset_mask = BIT(0),
+ },
+ [STPMIC1_BUCK2] = {
+ .desc = REG_BUCK(BUCK2, buck2),
+ .icc_reg = BUCKS_ICCTO_CR,
+ .icc_mask = BIT(1),
+ .mask_reset_reg = BUCKS_MASK_RESET_CR,
+ .mask_reset_mask = BIT(1),
+ },
+ [STPMIC1_BUCK3] = {
+ .desc = REG_BUCK(BUCK3, buck3),
+ .icc_reg = BUCKS_ICCTO_CR,
+ .icc_mask = BIT(2),
+ .mask_reset_reg = BUCKS_MASK_RESET_CR,
+ .mask_reset_mask = BIT(2),
+ },
+ [STPMIC1_BUCK4] = {
+ .desc = REG_BUCK(BUCK4, buck4),
+ .icc_reg = BUCKS_ICCTO_CR,
+ .icc_mask = BIT(3),
+ .mask_reset_reg = BUCKS_MASK_RESET_CR,
+ .mask_reset_mask = BIT(3),
+ },
+ [STPMIC1_LDO1] = {
+ .desc = REG_LDO(LDO1, ldo1),
+ .icc_reg = LDOS_ICCTO_CR,
+ .icc_mask = BIT(0),
+ .mask_reset_reg = LDOS_MASK_RESET_CR,
+ .mask_reset_mask = BIT(0),
+ },
+ [STPMIC1_LDO2] = {
+ .desc = REG_LDO(LDO2, ldo2),
+ .icc_reg = LDOS_ICCTO_CR,
+ .icc_mask = BIT(1),
+ .mask_reset_reg = LDOS_MASK_RESET_CR,
+ .mask_reset_mask = BIT(1),
+ },
+ [STPMIC1_LDO3] = {
+ .desc = REG_LDO3(LDO3, ldo3),
+ .icc_reg = LDOS_ICCTO_CR,
+ .icc_mask = BIT(2),
+ .mask_reset_reg = LDOS_MASK_RESET_CR,
+ .mask_reset_mask = BIT(2),
+ },
+ [STPMIC1_LDO4] = {
+ .desc = REG_LDO4(LDO4, ldo4),
+ .icc_reg = LDOS_ICCTO_CR,
+ .icc_mask = BIT(3),
+ .mask_reset_reg = LDOS_MASK_RESET_CR,
+ .mask_reset_mask = BIT(3),
+ },
+ [STPMIC1_LDO5] = {
+ .desc = REG_LDO(LDO5, ldo5),
+ .icc_reg = LDOS_ICCTO_CR,
+ .icc_mask = BIT(4),
+ .mask_reset_reg = LDOS_MASK_RESET_CR,
+ .mask_reset_mask = BIT(4),
+ },
+ [STPMIC1_LDO6] = {
+ .desc = REG_LDO(LDO6, ldo6),
+ .icc_reg = LDOS_ICCTO_CR,
+ .icc_mask = BIT(5),
+ .mask_reset_reg = LDOS_MASK_RESET_CR,
+ .mask_reset_mask = BIT(5),
+ },
+ [STPMIC1_VREF_DDR] = {
+ .desc = REG_VREF_DDR(VREF_DDR, vref_ddr),
+ .mask_reset_reg = LDOS_MASK_RESET_CR,
+ .mask_reset_mask = BIT(6),
+ },
+ [STPMIC1_BOOST] = {
+ .desc = REG_SWITCH(BOOST, boost, BST_SW_CR,
+ BOOST_ENABLED,
+ BOOST_ENABLED),
+ .icc_reg = BUCKS_ICCTO_CR,
+ .icc_mask = BIT(6),
+ },
+ [STPMIC1_VBUS_OTG] = {
+ .desc = REG_SWITCH(VBUS_OTG, pwr_sw1, BST_SW_CR,
+ USBSW_OTG_SWITCH_ENABLED,
+ USBSW_OTG_SWITCH_ENABLED),
+ .icc_reg = BUCKS_ICCTO_CR,
+ .icc_mask = BIT(4),
+ },
+ [STPMIC1_SW_OUT] = {
+ .desc = REG_SWITCH(SW_OUT, pwr_sw2, BST_SW_CR,
+ SWIN_SWOUT_ENABLED,
+ SWIN_SWOUT_ENABLED),
+ .icc_reg = BUCKS_ICCTO_CR,
+ .icc_mask = BIT(5),
+ },
+};
+
+static unsigned int stpmic1_map_mode(unsigned int mode)
+{
+ switch (mode) {
+ case STPMIC1_BUCK_MODE_NORMAL:
+ return REGULATOR_MODE_NORMAL;
+ case STPMIC1_BUCK_MODE_LP:
+ return REGULATOR_MODE_STANDBY;
+ default:
+ return REGULATOR_MODE_INVALID;
+ }
+}
+
+static unsigned int stpmic1_get_mode(struct regulator_dev *rdev)
+{
+ int value;
+
+ regmap_read(rdev->regmap, rdev->desc->enable_reg, &value);
+
+ if (value & STPMIC1_BUCK_MODE_LP)
+ return REGULATOR_MODE_STANDBY;
+
+ return REGULATOR_MODE_NORMAL;
+}
+
+static int stpmic1_set_mode(struct regulator_dev *rdev, unsigned int mode)
+{
+ int value;
+
+ switch (mode) {
+ case REGULATOR_MODE_NORMAL:
+ value = STPMIC1_BUCK_MODE_NORMAL;
+ break;
+ case REGULATOR_MODE_STANDBY:
+ value = STPMIC1_BUCK_MODE_LP;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return regmap_update_bits(rdev->regmap, rdev->desc->enable_reg,
+ STPMIC1_BUCK_MODE_LP, value);
+}
+
+static int stpmic1_set_icc(struct regulator_dev *rdev)
+{
+ struct stpmic1_regulator *regul = rdev_get_drvdata(rdev);
+
+ /* enable switch off in case of over current */
+ return regmap_update_bits(regul->regmap, regul->cfg->icc_reg,
+ regul->cfg->icc_mask, regul->cfg->icc_mask);
+}
+
+static irqreturn_t stpmic1_curlim_irq_handler(int irq, void *data)
+{
+ struct regulator_dev *rdev = (struct regulator_dev *)data;
+
+ mutex_lock(&rdev->mutex);
+
+ /* Send an overcurrent notification */
+ regulator_notifier_call_chain(rdev,
+ REGULATOR_EVENT_OVER_CURRENT,
+ NULL);
+
+ mutex_unlock(&rdev->mutex);
+
+ return IRQ_HANDLED;
+}
+
+static int stpmic1_regulator_init(struct platform_device *pdev,
+ struct regulator_dev *rdev)
+{
+ struct stpmic1_regulator *regul = rdev_get_drvdata(rdev);
+ int ret = 0;
+
+ /* set mask reset */
+ if (regul->mask_reset && regul->cfg->mask_reset_reg != 0) {
+ ret = regmap_update_bits(regul->regmap,
+ regul->cfg->mask_reset_reg,
+ regul->cfg->mask_reset_mask,
+ regul->cfg->mask_reset_mask);
+ if (ret) {
+ dev_err(&pdev->dev, "set mask reset failed\n");
+ return ret;
+ }
+ }
+
+ /* setup an irq handler for over-current detection */
+ if (regul->irq_curlim > 0) {
+ ret = devm_request_threaded_irq(&pdev->dev,
+ regul->irq_curlim, NULL,
+ stpmic1_curlim_irq_handler,
+ IRQF_ONESHOT | IRQF_SHARED,
+ pdev->name, rdev);
+ if (ret) {
+ dev_err(&pdev->dev, "Request IRQ failed\n");
+ return ret;
+ }
+ }
+ return 0;
+}
+
+#define MATCH(_name, _id) \
+ [STPMIC1_##_id] = { \
+ .name = #_name, \
+ .desc = &stpmic1_regulator_cfgs[STPMIC1_##_id].desc, \
+ }
+
+static struct of_regulator_match stpmic1_regulators_matches[] = {
+ MATCH(buck1, BUCK1),
+ MATCH(buck2, BUCK2),
+ MATCH(buck3, BUCK3),
+ MATCH(buck4, BUCK4),
+ MATCH(ldo1, LDO1),
+ MATCH(ldo2, LDO2),
+ MATCH(ldo3, LDO3),
+ MATCH(ldo4, LDO4),
+ MATCH(ldo5, LDO5),
+ MATCH(ldo6, LDO6),
+ MATCH(vref_ddr, VREF_DDR),
+ MATCH(boost, BOOST),
+ MATCH(pwr_sw1, VBUS_OTG),
+ MATCH(pwr_sw2, SW_OUT),
+};
+
+static int stpmic1_regulator_parse_dt(void *driver_data)
+{
+ struct stpmic1_regulator *regul =
+ (struct stpmic1_regulator *)driver_data;
+
+ if (!regul)
+ return -EINVAL;
+
+ if (of_get_property(regul->reg_node, "st,mask-reset", NULL))
+ regul->mask_reset = 1;
+
+ regul->irq_curlim = of_irq_get(regul->reg_node, 0);
+
+ return 0;
+}
+
+static struct
+regulator_dev *stpmic1_regulator_register(struct platform_device *pdev, int id,
+ struct regulator_init_data *init_data,
+ struct stpmic1_regulator *regul)
+{
+ struct stpmic1 *pmic_dev = dev_get_drvdata(pdev->dev.parent);
+ struct regulator_dev *rdev;
+ struct regulator_config config = {};
+
+ config.dev = &pdev->dev;
+ config.init_data = init_data;
+ config.of_node = stpmic1_regulators_matches[id].of_node;
+ config.regmap = pmic_dev->regmap;
+ config.driver_data = regul;
+
+ regul->regul_id = id;
+ regul->reg_node = config.of_node;
+ regul->cfg = &stpmic1_regulator_cfgs[id];
+ regul->regmap = pmic_dev->regmap;
+
+ rdev = devm_regulator_register(&pdev->dev, ®ul->cfg->desc, &config);
+ if (IS_ERR(rdev)) {
+ dev_err(&pdev->dev, "failed to register %s regulator\n",
+ regul->cfg->desc.name);
+ }
+
+ return rdev;
+}
+
+static int stpmic1_regulator_probe(struct platform_device *pdev)
+{
+ struct regulator_dev *rdev;
+ struct stpmic1_regulator *regul;
+ struct regulator_init_data *init_data;
+ struct device_node *np;
+ int i, ret;
+
+ np = pdev->dev.of_node;
+
+ ret = of_regulator_match(&pdev->dev, np,
+ stpmic1_regulators_matches,
+ ARRAY_SIZE(stpmic1_regulators_matches));
+ if (ret < 0) {
+ dev_err(&pdev->dev,
+ "Error in PMIC regulator device tree node");
+ return ret;
+ }
+
+ regul = devm_kzalloc(&pdev->dev, ARRAY_SIZE(stpmic1_regulator_cfgs) *
+ sizeof(struct stpmic1_regulator),
+ GFP_KERNEL);
+ if (!regul)
+ return -ENOMEM;
+
+ for (i = 0; i < ARRAY_SIZE(stpmic1_regulator_cfgs); i++) {
+ /* Parse DT & find regulators to register */
+ init_data = stpmic1_regulators_matches[i].init_data;
+ if (init_data)
+ init_data->regulator_init = &stpmic1_regulator_parse_dt;
+
+ rdev = stpmic1_regulator_register(pdev, i, init_data, regul);
+ if (IS_ERR(rdev))
+ return PTR_ERR(rdev);
+
+ ret = stpmic1_regulator_init(pdev, rdev);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "failed to initialize regulator %d\n", ret);
+ return ret;
+ }
+
+ regul++;
+ }
+
+ dev_dbg(&pdev->dev, "stpmic1_regulator driver probed\n");
+
+ return 0;
+}
+
+static const struct of_device_id of_pmic_regulator_match[] = {
+ { .compatible = "st,stpmic1-regulators" },
+ { },
+};
+
+MODULE_DEVICE_TABLE(of, of_pmic_regulator_match);
+
+static struct platform_driver stpmic1_regulator_driver = {
+ .driver = {
+ .name = "stpmic1-regulator",
+ .of_match_table = of_match_ptr(of_pmic_regulator_match),
+ },
+ .probe = stpmic1_regulator_probe,
+};
+
+module_platform_driver(stpmic1_regulator_driver);
+
+MODULE_DESCRIPTION("STPMIC1 PMIC voltage regulator driver");
+MODULE_AUTHOR("Pascal Paillet <p.paillet@st.com>");
+MODULE_LICENSE("GPL v2");
gflags = GPIOD_OUT_HIGH;
else
gflags = GPIOD_OUT_LOW;
+ gflags |= GPIOD_FLAGS_BIT_NONEXCLUSIVE;
rpdata->gpiod = devm_gpiod_get_from_of_node(&pdev->dev,
tps65090_matches[idx].of_node,
dasd_proc_exit();
#endif
dasd_eer_exit();
- if (dasd_page_cache != NULL) {
- kmem_cache_destroy(dasd_page_cache);
- dasd_page_cache = NULL;
- }
+ kmem_cache_destroy(dasd_page_cache);
+ dasd_page_cache = NULL;
dasd_gendisk_exit();
dasd_devmap_exit();
if (dasd_debug_area != NULL) {
gdp->queue = block->request_queue;
block->gdp = gdp;
set_capacity(block->gdp, 0);
- device_add_disk(&base->cdev->dev, block->gdp);
+ device_add_disk(&base->cdev->dev, block->gdp, NULL);
return 0;
}
}
get_device(&dev_info->dev);
- device_add_disk(&dev_info->dev, dev_info->gd);
+ device_add_disk(&dev_info->dev, dev_info->gd, NULL);
switch (dev_info->segment_type) {
case SEG_TYPE_SR:
/* 512 byte sectors */
set_capacity(bdev->gendisk, scmdev->size >> 9);
- device_add_disk(&scmdev->dev, bdev->gendisk);
+ device_add_disk(&scmdev->dev, bdev->gendisk, NULL);
return 0;
out_queue:
GCOV_PROFILE_sclp_early_core.o := n
KCOV_INSTRUMENT_sclp_early_core.o := n
UBSAN_SANITIZE_sclp_early_core.o := n
+KASAN_SANITIZE_sclp_early_core.o := n
CFLAGS_sclp_early_core.o += -D__NO_FORTIFY
static int monwrite_diag(struct monwrite_hdr *myhdr, char *buffer, int fcn)
{
- struct appldata_product_id id;
+ struct appldata_parameter_list *parm_list;
+ struct appldata_product_id *id;
int rc;
- memcpy(id.prod_nr, "LNXAPPL", 7);
- id.prod_fn = myhdr->applid;
- id.record_nr = myhdr->record_num;
- id.version_nr = myhdr->version;
- id.release_nr = myhdr->release;
- id.mod_lvl = myhdr->mod_level;
- rc = appldata_asm(&id, fcn, (void *) buffer, myhdr->datalen);
+ id = kmalloc(sizeof(*id), GFP_KERNEL);
+ parm_list = kmalloc(sizeof(*parm_list), GFP_KERNEL);
+ rc = -ENOMEM;
+ if (!id || !parm_list)
+ goto out;
+ memcpy(id->prod_nr, "LNXAPPL", 7);
+ id->prod_fn = myhdr->applid;
+ id->record_nr = myhdr->record_num;
+ id->version_nr = myhdr->version;
+ id->release_nr = myhdr->release;
+ id->mod_lvl = myhdr->mod_level;
+ rc = appldata_asm(parm_list, id, fcn,
+ (void *) buffer, myhdr->datalen);
if (rc <= 0)
- return rc;
+ goto out;
pr_err("Writing monitor data failed with rc=%i\n", rc);
- if (rc == 5)
- return -EPERM;
- return -EINVAL;
+ rc = (rc == 5) ? -EPERM : -EINVAL;
+out:
+ kfree(id);
+ kfree(parm_list);
+ return rc;
}
static struct mon_buf *monwrite_find_hdr(struct mon_private *monpriv,
typedef unsigned int sclp_cmdw_t;
#define SCLP_CMDW_READ_CPU_INFO 0x00010001
+#define SCLP_CMDW_READ_SCP_INFO 0x00020001
+#define SCLP_CMDW_READ_STORAGE_INFO 0x00040001
+#define SCLP_CMDW_READ_SCP_INFO_FORCED 0x00120001
#define SCLP_CMDW_READ_EVENT_DATA 0x00770005
#define SCLP_CMDW_WRITE_EVENT_DATA 0x00760005
#define SCLP_CMDW_WRITE_EVENT_MASK 0x00780005
u8 reserved[4096 - 16];
} __attribute__((packed, aligned(PAGE_SIZE)));
+struct read_info_sccb {
+ struct sccb_header header; /* 0-7 */
+ u16 rnmax; /* 8-9 */
+ u8 rnsize; /* 10 */
+ u8 _pad_11[16 - 11]; /* 11-15 */
+ u16 ncpurl; /* 16-17 */
+ u16 cpuoff; /* 18-19 */
+ u8 _pad_20[24 - 20]; /* 20-23 */
+ u8 loadparm[8]; /* 24-31 */
+ u8 _pad_32[42 - 32]; /* 32-41 */
+ u8 fac42; /* 42 */
+ u8 fac43; /* 43 */
+ u8 _pad_44[48 - 44]; /* 44-47 */
+ u64 facilities; /* 48-55 */
+ u8 _pad_56[66 - 56]; /* 56-65 */
+ u8 fac66; /* 66 */
+ u8 _pad_67[76 - 67]; /* 67-83 */
+ u32 ibc; /* 76-79 */
+ u8 _pad80[84 - 80]; /* 80-83 */
+ u8 fac84; /* 84 */
+ u8 fac85; /* 85 */
+ u8 _pad_86[91 - 86]; /* 86-90 */
+ u8 fac91; /* 91 */
+ u8 _pad_92[98 - 92]; /* 92-97 */
+ u8 fac98; /* 98 */
+ u8 hamaxpow; /* 99 */
+ u32 rnsize2; /* 100-103 */
+ u64 rnmax2; /* 104-111 */
+ u32 hsa_size; /* 112-115 */
+ u8 fac116; /* 116 */
+ u8 fac117; /* 117 */
+ u8 fac118; /* 118 */
+ u8 fac119; /* 119 */
+ u16 hcpua; /* 120-121 */
+ u8 _pad_122[124 - 122]; /* 122-123 */
+ u32 hmfai; /* 124-127 */
+ u8 _pad_128[4096 - 128]; /* 128-4095 */
+} __packed __aligned(PAGE_SIZE);
+
+struct read_storage_sccb {
+ struct sccb_header header;
+ u16 max_id;
+ u16 assigned;
+ u16 standby;
+ u16 :16;
+ u32 entries[0];
+} __packed;
+
static inline void sclp_fill_core_info(struct sclp_core_info *info,
struct read_cpu_info_sccb *sccb)
{
int sclp_early_set_event_mask(struct init_sccb *sccb,
sccb_mask_t receive_mask,
sccb_mask_t send_mask);
+int sclp_early_get_info(struct read_info_sccb *info);
/* useful inlines */
return -EPERM;
}
-struct read_storage_sccb {
- struct sccb_header header;
- u16 max_id;
- u16 assigned;
- u16 standby;
- u16 :16;
- u32 entries[0];
-} __packed;
-
static const struct dev_pm_ops sclp_mem_pm_ops = {
.freeze = sclp_mem_freeze,
};
for (id = 0; id <= sclp_max_storage_id; id++) {
memset(sccb, 0, PAGE_SIZE);
sccb->header.length = PAGE_SIZE;
- rc = sclp_sync_request(0x00040001 | id << 8, sccb);
+ rc = sclp_sync_request(SCLP_CMDW_READ_STORAGE_INFO | id << 8, sccb);
if (rc)
goto out;
switch (sccb->header.response_code) {
#include "sclp_sdias.h"
#include "sclp.h"
-#define SCLP_CMDW_READ_SCP_INFO 0x00020001
-#define SCLP_CMDW_READ_SCP_INFO_FORCED 0x00120001
-
-struct read_info_sccb {
- struct sccb_header header; /* 0-7 */
- u16 rnmax; /* 8-9 */
- u8 rnsize; /* 10 */
- u8 _pad_11[16 - 11]; /* 11-15 */
- u16 ncpurl; /* 16-17 */
- u16 cpuoff; /* 18-19 */
- u8 _pad_20[24 - 20]; /* 20-23 */
- u8 loadparm[8]; /* 24-31 */
- u8 _pad_32[42 - 32]; /* 32-41 */
- u8 fac42; /* 42 */
- u8 fac43; /* 43 */
- u8 _pad_44[48 - 44]; /* 44-47 */
- u64 facilities; /* 48-55 */
- u8 _pad_56[66 - 56]; /* 56-65 */
- u8 fac66; /* 66 */
- u8 _pad_67[76 - 67]; /* 67-83 */
- u32 ibc; /* 76-79 */
- u8 _pad80[84 - 80]; /* 80-83 */
- u8 fac84; /* 84 */
- u8 fac85; /* 85 */
- u8 _pad_86[91 - 86]; /* 86-90 */
- u8 fac91; /* 91 */
- u8 _pad_92[98 - 92]; /* 92-97 */
- u8 fac98; /* 98 */
- u8 hamaxpow; /* 99 */
- u32 rnsize2; /* 100-103 */
- u64 rnmax2; /* 104-111 */
- u8 _pad_112[116 - 112]; /* 112-115 */
- u8 fac116; /* 116 */
- u8 fac117; /* 117 */
- u8 fac118; /* 118 */
- u8 fac119; /* 119 */
- u16 hcpua; /* 120-121 */
- u8 _pad_122[124 - 122]; /* 122-123 */
- u32 hmfai; /* 124-127 */
- u8 _pad_128[4096 - 128]; /* 128-4095 */
-} __packed __aligned(PAGE_SIZE);
-
static struct sclp_ipl_info sclp_ipl_info;
struct sclp_info sclp;
EXPORT_SYMBOL(sclp);
-static int __init sclp_early_read_info(struct read_info_sccb *sccb)
-{
- int i;
- sclp_cmdw_t commands[] = {SCLP_CMDW_READ_SCP_INFO_FORCED,
- SCLP_CMDW_READ_SCP_INFO};
-
- for (i = 0; i < ARRAY_SIZE(commands); i++) {
- memset(sccb, 0, sizeof(*sccb));
- sccb->header.length = sizeof(*sccb);
- sccb->header.function_code = 0x80;
- sccb->header.control_mask[2] = 0x80;
- if (sclp_early_cmd(commands[i], sccb))
- break;
- if (sccb->header.response_code == 0x10)
- return 0;
- if (sccb->header.response_code != 0x1f0)
- break;
- }
- return -EIO;
-}
-
static void __init sclp_early_facilities_detect(struct read_info_sccb *sccb)
{
struct sclp_core_entry *cpue;
u16 boot_cpu_address, cpu;
- if (sclp_early_read_info(sccb))
+ if (sclp_early_get_info(sccb))
return;
sclp.facilities = sccb->facilities;
sclp_ipl_info.has_dump = 1;
memcpy(&sclp_ipl_info.loadparm, &sccb->loadparm, LOADPARM_LEN);
+ if (sccb->hsa_size)
+ sclp.hsa_size = (sccb->hsa_size - 1) * PAGE_SIZE;
sclp.mtid = (sccb->fac42 & 0x80) ? (sccb->fac42 & 31) : 0;
sclp.mtid_cp = (sccb->fac42 & 0x80) ? (sccb->fac43 & 31) : 0;
sclp.mtid_prev = (sccb->fac42 & 0x80) ? (sccb->fac66 & 31) : 0;
return 0;
}
-static long __init sclp_early_hsa_size_init(struct sdias_sccb *sccb)
-{
- memset(sccb, 0, sizeof(*sccb));
- sccb->hdr.length = sizeof(*sccb);
- sccb->evbuf.hdr.length = sizeof(struct sdias_evbuf);
- sccb->evbuf.hdr.type = EVTYP_SDIAS;
- sccb->evbuf.event_qual = SDIAS_EQ_SIZE;
- sccb->evbuf.data_id = SDIAS_DI_FCP_DUMP;
- sccb->evbuf.event_id = 4712;
- sccb->evbuf.dbs = 1;
- if (sclp_early_cmd(SCLP_CMDW_WRITE_EVENT_DATA, sccb))
- return -EIO;
- if (sccb->hdr.response_code != 0x20)
- return -EIO;
- if (sccb->evbuf.blk_cnt == 0)
- return 0;
- return (sccb->evbuf.blk_cnt - 1) * PAGE_SIZE;
-}
-
-static long __init sclp_early_hsa_copy_wait(struct sdias_sccb *sccb)
-{
- memset(sccb, 0, PAGE_SIZE);
- sccb->hdr.length = PAGE_SIZE;
- if (sclp_early_cmd(SCLP_CMDW_READ_EVENT_DATA, sccb))
- return -EIO;
- if ((sccb->hdr.response_code != 0x20) && (sccb->hdr.response_code != 0x220))
- return -EIO;
- if (sccb->evbuf.blk_cnt == 0)
- return 0;
- return (sccb->evbuf.blk_cnt - 1) * PAGE_SIZE;
-}
-
-static void __init sclp_early_hsa_size_detect(void *sccb)
-{
- unsigned long flags;
- long size = -EIO;
-
- raw_local_irq_save(flags);
- if (sclp_early_set_event_mask(sccb, EVTYP_SDIAS_MASK, EVTYP_SDIAS_MASK))
- goto out;
- size = sclp_early_hsa_size_init(sccb);
- /* First check for synchronous response (LPAR) */
- if (size)
- goto out_mask;
- if (!(S390_lowcore.ext_params & 1))
- sclp_early_wait_irq();
- size = sclp_early_hsa_copy_wait(sccb);
-out_mask:
- sclp_early_set_event_mask(sccb, 0, 0);
-out:
- raw_local_irq_restore(flags);
- if (size > 0)
- sclp.hsa_size = size;
-}
-
static void __init sclp_early_console_detect(struct init_sccb *sccb)
{
if (sccb->header.response_code != 0x20)
sclp_early_facilities_detect(sccb);
sclp_early_init_core_info(sccb);
- sclp_early_hsa_size_detect(sccb);
/*
* Turn off SCLP event notifications. Also save remote masks in the
#include <asm/lowcore.h>
#include <asm/ebcdic.h>
#include <asm/irq.h>
+#include <asm/sections.h>
+#include <asm/mem_detect.h>
#include "sclp.h"
#include "sclp_rw.h"
+static struct read_info_sccb __bootdata(sclp_info_sccb);
+static int __bootdata(sclp_info_sccb_valid);
char sclp_early_sccb[PAGE_SIZE] __aligned(PAGE_SIZE) __section(.data);
int sclp_init_state __section(.data) = sclp_init_state_uninitialized;
/*
* Output one or more lines of text on the SCLP console (VT220 and /
* or line-mode).
*/
-void __sclp_early_printk(const char *str, unsigned int len)
+void __sclp_early_printk(const char *str, unsigned int len, unsigned int force)
{
int have_linemode, have_vt220;
- if (sclp_init_state != sclp_init_state_uninitialized)
+ if (!force && sclp_init_state != sclp_init_state_uninitialized)
return;
if (sclp_early_setup(0, &have_linemode, &have_vt220) != 0)
return;
void sclp_early_printk(const char *str)
{
- __sclp_early_printk(str, strlen(str));
+ __sclp_early_printk(str, strlen(str), 0);
+}
+
+void sclp_early_printk_force(const char *str)
+{
+ __sclp_early_printk(str, strlen(str), 1);
+}
+
+int __init sclp_early_read_info(void)
+{
+ int i;
+ struct read_info_sccb *sccb = &sclp_info_sccb;
+ sclp_cmdw_t commands[] = {SCLP_CMDW_READ_SCP_INFO_FORCED,
+ SCLP_CMDW_READ_SCP_INFO};
+
+ for (i = 0; i < ARRAY_SIZE(commands); i++) {
+ memset(sccb, 0, sizeof(*sccb));
+ sccb->header.length = sizeof(*sccb);
+ sccb->header.function_code = 0x80;
+ sccb->header.control_mask[2] = 0x80;
+ if (sclp_early_cmd(commands[i], sccb))
+ break;
+ if (sccb->header.response_code == 0x10) {
+ sclp_info_sccb_valid = 1;
+ return 0;
+ }
+ if (sccb->header.response_code != 0x1f0)
+ break;
+ }
+ return -EIO;
+}
+
+int __init sclp_early_get_info(struct read_info_sccb *info)
+{
+ if (!sclp_info_sccb_valid)
+ return -EIO;
+
+ *info = sclp_info_sccb;
+ return 0;
+}
+
+int __init sclp_early_get_memsize(unsigned long *mem)
+{
+ unsigned long rnmax;
+ unsigned long rnsize;
+ struct read_info_sccb *sccb = &sclp_info_sccb;
+
+ if (!sclp_info_sccb_valid)
+ return -EIO;
+
+ rnmax = sccb->rnmax ? sccb->rnmax : sccb->rnmax2;
+ rnsize = sccb->rnsize ? sccb->rnsize : sccb->rnsize2;
+ rnsize <<= 20;
+ *mem = rnsize * rnmax;
+ return 0;
+}
+
+int __init sclp_early_get_hsa_size(unsigned long *hsa_size)
+{
+ if (!sclp_info_sccb_valid)
+ return -EIO;
+
+ *hsa_size = 0;
+ if (sclp_info_sccb.hsa_size)
+ *hsa_size = (sclp_info_sccb.hsa_size - 1) * PAGE_SIZE;
+ return 0;
+}
+
+#define SCLP_STORAGE_INFO_FACILITY 0x0000400000000000UL
+
+void __weak __init add_mem_detect_block(u64 start, u64 end) {}
+int __init sclp_early_read_storage_info(void)
+{
+ struct read_storage_sccb *sccb = (struct read_storage_sccb *)&sclp_early_sccb;
+ int rc, id, max_id = 0;
+ unsigned long rn, rzm;
+ sclp_cmdw_t command;
+ u16 sn;
+
+ if (!sclp_info_sccb_valid)
+ return -EIO;
+
+ if (!(sclp_info_sccb.facilities & SCLP_STORAGE_INFO_FACILITY))
+ return -EOPNOTSUPP;
+
+ rzm = sclp_info_sccb.rnsize ?: sclp_info_sccb.rnsize2;
+ rzm <<= 20;
+
+ for (id = 0; id <= max_id; id++) {
+ memset(sclp_early_sccb, 0, sizeof(sclp_early_sccb));
+ sccb->header.length = sizeof(sclp_early_sccb);
+ command = SCLP_CMDW_READ_STORAGE_INFO | (id << 8);
+ rc = sclp_early_cmd(command, sccb);
+ if (rc)
+ goto fail;
+
+ max_id = sccb->max_id;
+ switch (sccb->header.response_code) {
+ case 0x0010:
+ for (sn = 0; sn < sccb->assigned; sn++) {
+ if (!sccb->entries[sn])
+ continue;
+ rn = sccb->entries[sn] >> 16;
+ add_mem_detect_block((rn - 1) * rzm, rn * rzm);
+ }
+ break;
+ case 0x0310:
+ case 0x0410:
+ break;
+ default:
+ goto fail;
+ }
+ }
+
+ return 0;
+fail:
+ mem_detect.count = 0;
+ return -EIO;
}
#define SCLP_ATYPE_PCI 2
+#define SCLP_ERRNOTIFY_AQ_RESET 0
#define SCLP_ERRNOTIFY_AQ_REPAIR 1
#define SCLP_ERRNOTIFY_AQ_INFO_LOG 2
if (report->version != 1)
return -EINVAL;
- if (report->action != SCLP_ERRNOTIFY_AQ_REPAIR &&
- report->action != SCLP_ERRNOTIFY_AQ_INFO_LOG)
+ switch (report->action) {
+ case SCLP_ERRNOTIFY_AQ_RESET:
+ case SCLP_ERRNOTIFY_AQ_REPAIR:
+ case SCLP_ERRNOTIFY_AQ_INFO_LOG:
+ break;
+ default:
return -EINVAL;
+ }
if (report->length > (PAGE_SIZE - sizeof(struct err_notify_sccb)))
return -EINVAL;
snprintf(exception, BUFSIZE, "Data degraded");
break;
case 0x03:
- snprintf(exception, BUFSIZE, "Data degraded in partion %i",
+ snprintf(exception, BUFSIZE, "Data degraded in partition %i",
sense->fmt.f70.mp);
break;
case 0x04:
}
};
-#define MAXMINOR (sizeof(sys_ser)/sizeof(struct vmlogrdr_priv_t))
+#define MAXMINOR ARRAY_SIZE(sys_ser)
static char FENCE[] = {"EOR"};
static int vmlogrdr_major = 0;
}
EXPORT_SYMBOL(ccwgroup_driver_unregister);
+static int __ccwgroupdev_check_busid(struct device *dev, void *id)
+{
+ char *bus_id = id;
+
+ return (strcmp(bus_id, dev_name(dev)) == 0);
+}
+
+/**
+ * get_ccwgroupdev_by_busid() - obtain device from a bus id
+ * @gdrv: driver the device is owned by
+ * @bus_id: bus id of the device to be searched
+ *
+ * This function searches all devices owned by @gdrv for a device with a bus
+ * id matching @bus_id.
+ * Returns:
+ * If a match is found, its reference count of the found device is increased
+ * and it is returned; else %NULL is returned.
+ */
+struct ccwgroup_device *get_ccwgroupdev_by_busid(struct ccwgroup_driver *gdrv,
+ char *bus_id)
+{
+ struct device *dev;
+
+ dev = driver_find_device(&gdrv->driver, NULL, bus_id,
+ __ccwgroupdev_check_busid);
+
+ return dev ? to_ccwgroupdev(dev) : NULL;
+}
+EXPORT_SYMBOL_GPL(get_ccwgroupdev_by_busid);
+
/**
* ccwgroup_probe_ccwdev() - probe function for slave devices
* @cdev: ccw device to be probed
return 0;
}
-static inline int contains_aobs(struct qdio_q *q)
-{
- return !q->is_input_q && q->u.out.use_cq;
-}
-
static inline void qdio_handle_aobs(struct qdio_q *q, int start, int count)
{
unsigned char state = 0;
int j, b = start;
- if (!contains_aobs(q))
- return;
-
for (j = 0; j < count; ++j) {
get_buf_state(q, b, &state, 0);
if (state == SLSB_P_OUTPUT_PENDING) {
q->u.out.sbal_state[b].flags |=
QDIO_OUTBUF_STATE_FLAG_PENDING;
q->u.out.aobs[b] = NULL;
- } else if (state == SLSB_P_OUTPUT_EMPTY) {
- q->u.out.sbal_state[b].aob = NULL;
}
b = next_buf(b);
}
q->aobs[bufnr] = aob;
}
if (q->aobs[bufnr]) {
- q->sbal_state[bufnr].aob = q->aobs[bufnr];
q->aobs[bufnr]->user1 = (u64) q->sbal_state[bufnr].user;
phys_aob = virt_to_phys(q->aobs[bufnr]);
WARN_ON_ONCE(phys_aob & 0xFF);
qperf_inc(q, outbound_handler);
DBF_DEV_EVENT(DBF_INFO, q->irq_ptr, "koh: s:%02x c:%02x",
start, count);
+ if (q->u.out.use_cq)
+ qdio_handle_aobs(q, start, count);
}
- qdio_handle_aobs(q, start, count);
-
q->handler(q->irq_ptr->cdev, q->qdio_error, q->nr, start, count,
q->irq_ptr->int_parm);
{
return kmem_cache_zalloc(qdio_aob_cache, GFP_ATOMIC);
}
-EXPORT_SYMBOL_GPL(qdio_allocate_aob);
void qdio_release_aob(struct qaob *aob)
{
for (i = 0; i < pat->pat_nr; i++, pa++)
for (j = 0; j < pa->pa_nr; j++)
- if (pa->pa_iova_pfn[i] == iova_pfn)
+ if (pa->pa_iova_pfn[j] == iova_pfn)
return true;
return false;
#include "vfio_ccw_private.h"
struct workqueue_struct *vfio_ccw_work_q;
+struct kmem_cache *vfio_ccw_io_region;
/*
* Helpers
cp_update_scsw(&private->cp, &irb->scsw);
cp_free(&private->cp);
}
- memcpy(private->io_region.irb_area, irb, sizeof(*irb));
+ memcpy(private->io_region->irb_area, irb, sizeof(*irb));
if (private->io_trigger)
eventfd_signal(private->io_trigger, 1);
private = kzalloc(sizeof(*private), GFP_KERNEL | GFP_DMA);
if (!private)
return -ENOMEM;
+
+ private->io_region = kmem_cache_zalloc(vfio_ccw_io_region,
+ GFP_KERNEL | GFP_DMA);
+ if (!private->io_region) {
+ kfree(private);
+ return -ENOMEM;
+ }
+
private->sch = sch;
dev_set_drvdata(&sch->dev, private);
cio_disable_subchannel(sch);
out_free:
dev_set_drvdata(&sch->dev, NULL);
+ kmem_cache_free(vfio_ccw_io_region, private->io_region);
kfree(private);
return ret;
}
dev_set_drvdata(&sch->dev, NULL);
+ kmem_cache_free(vfio_ccw_io_region, private->io_region);
kfree(private);
return 0;
if (!vfio_ccw_work_q)
return -ENOMEM;
+ vfio_ccw_io_region = kmem_cache_create_usercopy("vfio_ccw_io_region",
+ sizeof(struct ccw_io_region), 0,
+ SLAB_ACCOUNT, 0,
+ sizeof(struct ccw_io_region), NULL);
+ if (!vfio_ccw_io_region) {
+ destroy_workqueue(vfio_ccw_work_q);
+ return -ENOMEM;
+ }
+
isc_register(VFIO_CCW_ISC);
ret = css_driver_register(&vfio_ccw_sch_driver);
if (ret) {
isc_unregister(VFIO_CCW_ISC);
+ kmem_cache_destroy(vfio_ccw_io_region);
destroy_workqueue(vfio_ccw_work_q);
}
{
css_driver_unregister(&vfio_ccw_sch_driver);
isc_unregister(VFIO_CCW_ISC);
+ kmem_cache_destroy(vfio_ccw_io_region);
destroy_workqueue(vfio_ccw_work_q);
}
module_init(vfio_ccw_sch_init);
enum vfio_ccw_event event)
{
pr_err("vfio-ccw: FSM: I/O request from state:%d\n", private->state);
- private->io_region.ret_code = -EIO;
+ private->io_region->ret_code = -EIO;
}
static void fsm_io_busy(struct vfio_ccw_private *private,
enum vfio_ccw_event event)
{
- private->io_region.ret_code = -EBUSY;
+ private->io_region->ret_code = -EBUSY;
}
static void fsm_disabled_irq(struct vfio_ccw_private *private,
{
union orb *orb;
union scsw *scsw = &private->scsw;
- struct ccw_io_region *io_region = &private->io_region;
+ struct ccw_io_region *io_region = private->io_region;
struct mdev_device *mdev = private->mdev;
char *errstr = "request";
return -EINVAL;
private = dev_get_drvdata(mdev_parent_dev(mdev));
- region = &private->io_region;
+ region = private->io_region;
if (copy_to_user(buf, (void *)region + *ppos, count))
return -EFAULT;
if (private->state != VFIO_CCW_STATE_IDLE)
return -EACCES;
- region = &private->io_region;
+ region = private->io_region;
if (copy_from_user((void *)region + *ppos, buf, count))
return -EFAULT;
atomic_t avail;
struct mdev_device *mdev;
struct notifier_block nb;
- struct ccw_io_region io_region;
+ struct ccw_io_region *io_region;
struct channel_program cp;
struct irb irb;
zcrypt-objs += zcrypt_msgtype6.o zcrypt_msgtype50.o
obj-$(CONFIG_ZCRYPT) += zcrypt.o
# adapter drivers depend on ap.o and zcrypt.o
-obj-$(CONFIG_ZCRYPT) += zcrypt_pcixcc.o zcrypt_cex2a.o zcrypt_cex4.o
+obj-$(CONFIG_ZCRYPT) += zcrypt_cex2c.o zcrypt_cex2a.o zcrypt_cex4.o
# pkey kernel module
pkey-objs := pkey_api.o
DEFINE_SPINLOCK(ap_list_lock);
LIST_HEAD(ap_card_list);
-/* Default permissions (card and domain masking) */
-static struct ap_perms {
- DECLARE_BITMAP(apm, AP_DEVICES);
- DECLARE_BITMAP(aqm, AP_DOMAINS);
-} ap_perms;
-static DEFINE_MUTEX(ap_perms_mutex);
+/* Default permissions (ioctl, card and domain masking) */
+struct ap_perms ap_perms;
+EXPORT_SYMBOL(ap_perms);
+DEFINE_MUTEX(ap_perms_mutex);
+EXPORT_SYMBOL(ap_perms_mutex);
static struct ap_config_info *ap_configuration;
static bool initialised;
return 0;
}
-/*
- * process_mask_arg() - parse a bitmap string and clear/set the
- * bits in the bitmap accordingly. The string may be given as
- * absolute value, a hex string like 0x1F2E3D4C5B6A" simple over-
- * writing the current content of the bitmap. Or as relative string
- * like "+1-16,-32,-0x40,+128" where only single bits or ranges of
- * bits are cleared or set. Distinction is done based on the very
- * first character which may be '+' or '-' for the relative string
- * and othewise assume to be an absolute value string. If parsing fails
- * a negative errno value is returned. All arguments and bitmaps are
- * big endian order.
- */
-static int process_mask_arg(const char *str,
- unsigned long *bitmap, int bits,
- struct mutex *lock)
+int ap_parse_mask_str(const char *str,
+ unsigned long *bitmap, int bits,
+ struct mutex *lock)
{
unsigned long *newmap, size;
int rc;
kfree(newmap);
return rc;
}
+EXPORT_SYMBOL(ap_parse_mask_str);
/*
* AP bus attributes.
static BUS_ATTR_RO(ap_usage_domain_mask);
+static ssize_t ap_adapter_mask_show(struct bus_type *bus, char *buf)
+{
+ if (!ap_configuration) /* QCI not supported */
+ return snprintf(buf, PAGE_SIZE, "not supported\n");
+
+ return snprintf(buf, PAGE_SIZE,
+ "0x%08x%08x%08x%08x%08x%08x%08x%08x\n",
+ ap_configuration->apm[0], ap_configuration->apm[1],
+ ap_configuration->apm[2], ap_configuration->apm[3],
+ ap_configuration->apm[4], ap_configuration->apm[5],
+ ap_configuration->apm[6], ap_configuration->apm[7]);
+}
+
+static BUS_ATTR_RO(ap_adapter_mask);
+
static ssize_t ap_interrupts_show(struct bus_type *bus, char *buf)
{
return snprintf(buf, PAGE_SIZE, "%d\n",
{
int rc;
- rc = process_mask_arg(buf, ap_perms.apm, AP_DEVICES, &ap_perms_mutex);
+ rc = ap_parse_mask_str(buf, ap_perms.apm, AP_DEVICES, &ap_perms_mutex);
if (rc)
return rc;
{
int rc;
- rc = process_mask_arg(buf, ap_perms.aqm, AP_DOMAINS, &ap_perms_mutex);
+ rc = ap_parse_mask_str(buf, ap_perms.aqm, AP_DOMAINS, &ap_perms_mutex);
if (rc)
return rc;
&bus_attr_ap_domain,
&bus_attr_ap_control_domain_mask,
&bus_attr_ap_usage_domain_mask,
+ &bus_attr_ap_adapter_mask,
&bus_attr_config_time,
&bus_attr_poll_thread,
&bus_attr_ap_interrupts,
};
/**
- * ap_select_domain(): Select an AP domain.
- *
- * Pick one of the 16 AP domains.
+ * ap_select_domain(): Select an AP domain if possible and we haven't
+ * already done so before.
*/
-static int ap_select_domain(void)
+static void ap_select_domain(void)
{
int count, max_count, best_domain;
struct ap_queue_status status;
if (ap_domain_index >= 0) {
/* Domain has already been selected. */
spin_unlock_bh(&ap_domain_lock);
- return 0;
+ return;
}
best_domain = -1;
max_count = 0;
if (best_domain >= 0) {
ap_domain_index = best_domain;
AP_DBF(DBF_DEBUG, "new ap_domain_index=%d\n", ap_domain_index);
- spin_unlock_bh(&ap_domain_lock);
- return 0;
}
spin_unlock_bh(&ap_domain_lock);
- return -ENODEV;
}
/*
AP_DBF(DBF_DEBUG, "%s running\n", __func__);
ap_query_configuration(ap_configuration);
- if (ap_select_domain() != 0)
- goto out;
+ ap_select_domain();
for (id = 0; id < AP_DEVICES; id++) {
/* check if device is registered */
}
} /* end device loop */
- if (defdomdevs < 1)
+ if (ap_domain_index >= 0 && defdomdevs < 1)
AP_DBF(DBF_INFO,
"no queue device with default domain %d available\n",
ap_domain_index);
-out:
mod_timer(&ap_config_timer, jiffies + ap_config_time * HZ);
}
static void __init ap_perms_init(void)
{
/* all resources useable if no kernel parameter string given */
+ memset(&ap_perms.ioctlm, 0xFF, sizeof(ap_perms.ioctlm));
memset(&ap_perms.apm, 0xFF, sizeof(ap_perms.apm));
memset(&ap_perms.aqm, 0xFF, sizeof(ap_perms.aqm));
/* apm kernel parameter string */
if (apm_str) {
memset(&ap_perms.apm, 0, sizeof(ap_perms.apm));
- process_mask_arg(apm_str, ap_perms.apm, AP_DEVICES,
- &ap_perms_mutex);
+ ap_parse_mask_str(apm_str, ap_perms.apm, AP_DEVICES,
+ &ap_perms_mutex);
}
/* aqm kernel parameter string */
if (aqm_str) {
memset(&ap_perms.aqm, 0, sizeof(ap_perms.aqm));
- process_mask_arg(aqm_str, ap_perms.aqm, AP_DOMAINS,
- &ap_perms_mutex);
+ ap_parse_mask_str(aqm_str, ap_perms.aqm, AP_DOMAINS,
+ &ap_perms_mutex);
}
}
return -ENODEV;
}
- /* set up the AP permissions (ap and aq masks) */
+ /* set up the AP permissions (ioctls, ap and aq masks) */
ap_perms_init();
/* Get AP configuration data if available */
#define AP_DEVICES 256 /* Number of AP devices. */
#define AP_DOMAINS 256 /* Number of AP domains. */
+#define AP_IOCTLS 256 /* Number of ioctls. */
#define AP_RESET_TIMEOUT (HZ*0.7) /* Time in ticks for reset timeouts. */
#define AP_CONFIG_TIME 30 /* Time in seconds between AP bus rescans. */
#define AP_POLL_TIME 1 /* Time in ticks between receive polls. */
struct ap_card *ap_card_create(int id, int queue_depth, int raw_device_type,
int comp_device_type, unsigned int functions);
+struct ap_perms {
+ unsigned long ioctlm[BITS_TO_LONGS(AP_IOCTLS)];
+ unsigned long apm[BITS_TO_LONGS(AP_DEVICES)];
+ unsigned long aqm[BITS_TO_LONGS(AP_DOMAINS)];
+};
+extern struct ap_perms ap_perms;
+extern struct mutex ap_perms_mutex;
+
/*
* check APQN for owned/reserved by ap bus and default driver(s).
* Checks if this APQN is or will be in use by the ap bus
int ap_apqn_in_matrix_owned_by_def_drv(unsigned long *apm,
unsigned long *aqm);
+/*
+ * ap_parse_mask_str() - helper function to parse a bitmap string
+ * and clear/set the bits in the bitmap accordingly. The string may be
+ * given as absolute value, a hex string like 0x1F2E3D4C5B6A" simple
+ * overwriting the current content of the bitmap. Or as relative string
+ * like "+1-16,-32,-0x40,+128" where only single bits or ranges of
+ * bits are cleared or set. Distinction is done based on the very
+ * first character which may be '+' or '-' for the relative string
+ * and othewise assume to be an absolute value string. If parsing fails
+ * a negative errno value is returned. All arguments and bitmaps are
+ * big endian order.
+ */
+int ap_parse_mask_str(const char *str,
+ unsigned long *bitmap, int bits,
+ struct mutex *lock);
+
#endif /* _AP_BUS_H_ */
#include <linux/slab.h>
#include <linux/kallsyms.h>
#include <linux/debugfs.h>
+#include <linux/random.h>
+#include <linux/cpufeature.h>
#include <asm/zcrypt.h>
#include <asm/cpacf.h>
#include <asm/pkey.h>
+#include <crypto/aes.h>
#include "zcrypt_api.h"
/* Size of vardata block used for some of the cca requests/replies */
#define VARDATASIZE 4096
+/* mask of available pckmo subfunctions, fetched once at module init */
+static cpacf_mask_t pckmo_functions;
+
/*
* debug feature data and functions
*/
debug_unregister(debug_info);
}
+/* Key token types */
+#define TOKTYPE_NON_CCA 0x00 /* Non-CCA key token */
+#define TOKTYPE_CCA_INTERNAL 0x01 /* CCA internal key token */
+
+/* For TOKTYPE_NON_CCA: */
+#define TOKVER_PROTECTED_KEY 0x01 /* Protected key token */
+
+/* For TOKTYPE_CCA_INTERNAL: */
+#define TOKVER_CCA_AES 0x04 /* CCA AES key token */
+
+/* header part of a key token */
+struct keytoken_header {
+ u8 type; /* one of the TOKTYPE values */
+ u8 res0[3];
+ u8 version; /* one of the TOKVER values */
+ u8 res1[3];
+} __packed;
+
/* inside view of a secure key token (only type 0x01 version 0x04) */
struct secaeskeytoken {
u8 type; /* 0x01 for internal key token */
u8 tvv[4]; /* token validation value */
} __packed;
+/* inside view of a protected key token (only type 0x00 version 0x01) */
+struct protaeskeytoken {
+ u8 type; /* 0x00 for PAES specific key tokens */
+ u8 res0[3];
+ u8 version; /* should be 0x01 for protected AES key token */
+ u8 res1[3];
+ u32 keytype; /* key type, one of the PKEY_KEYTYPE values */
+ u32 len; /* bytes actually stored in protkey[] */
+ u8 protkey[MAXPROTKEYSIZE]; /* the protected key blob */
+} __packed;
+
/*
* Simple check if the token is a valid CCA secure AES key
* token. If keybitsize is given, the bitsize of the key is
{
struct secaeskeytoken *t = (struct secaeskeytoken *) token;
- if (t->type != 0x01) {
+ if (t->type != TOKTYPE_CCA_INTERNAL) {
DEBUG_ERR(
- "%s secure token check failed, type mismatch 0x%02x != 0x01\n",
- __func__, (int) t->type);
+ "%s secure token check failed, type mismatch 0x%02x != 0x%02x\n",
+ __func__, (int) t->type, TOKTYPE_CCA_INTERNAL);
return -EINVAL;
}
- if (t->version != 0x04) {
+ if (t->version != TOKVER_CCA_AES) {
DEBUG_ERR(
- "%s secure token check failed, version mismatch 0x%02x != 0x04\n",
- __func__, (int) t->version);
+ "%s secure token check failed, version mismatch 0x%02x != 0x%02x\n",
+ __func__, (int) t->version, TOKVER_CCA_AES);
return -EINVAL;
}
if (keybitsize > 0 && t->bitsize != keybitsize) {
return -EINVAL;
}
+ /*
+ * Check if the needed pckmo subfunction is available.
+ * These subfunctions can be enabled/disabled by customers
+ * in the LPAR profile or may even change on the fly.
+ */
+ if (!cpacf_test_func(&pckmo_functions, fc)) {
+ DEBUG_ERR("%s pckmo functions not available\n", __func__);
+ return -EOPNOTSUPP;
+ }
+
/* prepare param block */
memset(paramblock, 0, sizeof(paramblock));
memcpy(paramblock, clrkey->clrkey, keysize);
}
EXPORT_SYMBOL(pkey_verifykey);
+/*
+ * Generate a random protected key
+ */
+int pkey_genprotkey(__u32 keytype, struct pkey_protkey *protkey)
+{
+ struct pkey_clrkey clrkey;
+ int keysize;
+ int rc;
+
+ switch (keytype) {
+ case PKEY_KEYTYPE_AES_128:
+ keysize = 16;
+ break;
+ case PKEY_KEYTYPE_AES_192:
+ keysize = 24;
+ break;
+ case PKEY_KEYTYPE_AES_256:
+ keysize = 32;
+ break;
+ default:
+ DEBUG_ERR("%s unknown/unsupported keytype %d\n", __func__,
+ keytype);
+ return -EINVAL;
+ }
+
+ /* generate a dummy random clear key */
+ get_random_bytes(clrkey.clrkey, keysize);
+
+ /* convert it to a dummy protected key */
+ rc = pkey_clr2protkey(keytype, &clrkey, protkey);
+ if (rc)
+ return rc;
+
+ /* replace the key part of the protected key with random bytes */
+ get_random_bytes(protkey->protkey, keysize);
+
+ return 0;
+}
+EXPORT_SYMBOL(pkey_genprotkey);
+
+/*
+ * Verify if a protected key is still valid
+ */
+int pkey_verifyprotkey(const struct pkey_protkey *protkey)
+{
+ unsigned long fc;
+ struct {
+ u8 iv[AES_BLOCK_SIZE];
+ u8 key[MAXPROTKEYSIZE];
+ } param;
+ u8 null_msg[AES_BLOCK_SIZE];
+ u8 dest_buf[AES_BLOCK_SIZE];
+ unsigned int k;
+
+ switch (protkey->type) {
+ case PKEY_KEYTYPE_AES_128:
+ fc = CPACF_KMC_PAES_128;
+ break;
+ case PKEY_KEYTYPE_AES_192:
+ fc = CPACF_KMC_PAES_192;
+ break;
+ case PKEY_KEYTYPE_AES_256:
+ fc = CPACF_KMC_PAES_256;
+ break;
+ default:
+ DEBUG_ERR("%s unknown/unsupported keytype %d\n", __func__,
+ protkey->type);
+ return -EINVAL;
+ }
+
+ memset(null_msg, 0, sizeof(null_msg));
+
+ memset(param.iv, 0, sizeof(param.iv));
+ memcpy(param.key, protkey->protkey, sizeof(param.key));
+
+ k = cpacf_kmc(fc | CPACF_ENCRYPT, ¶m, null_msg, dest_buf,
+ sizeof(null_msg));
+ if (k != sizeof(null_msg)) {
+ DEBUG_ERR("%s protected key is not valid\n", __func__);
+ return -EKEYREJECTED;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL(pkey_verifyprotkey);
+
+/*
+ * Transform a non-CCA key token into a protected key
+ */
+static int pkey_nonccatok2pkey(const __u8 *key, __u32 keylen,
+ struct pkey_protkey *protkey)
+{
+ struct keytoken_header *hdr = (struct keytoken_header *)key;
+ struct protaeskeytoken *t;
+
+ switch (hdr->version) {
+ case TOKVER_PROTECTED_KEY:
+ if (keylen != sizeof(struct protaeskeytoken))
+ return -EINVAL;
+
+ t = (struct protaeskeytoken *)key;
+ protkey->len = t->len;
+ protkey->type = t->keytype;
+ memcpy(protkey->protkey, t->protkey,
+ sizeof(protkey->protkey));
+
+ return pkey_verifyprotkey(protkey);
+ default:
+ DEBUG_ERR("%s unknown/unsupported non-CCA token version %d\n",
+ __func__, hdr->version);
+ return -EINVAL;
+ }
+}
+
+/*
+ * Transform a CCA internal key token into a protected key
+ */
+static int pkey_ccainttok2pkey(const __u8 *key, __u32 keylen,
+ struct pkey_protkey *protkey)
+{
+ struct keytoken_header *hdr = (struct keytoken_header *)key;
+
+ switch (hdr->version) {
+ case TOKVER_CCA_AES:
+ if (keylen != sizeof(struct secaeskeytoken))
+ return -EINVAL;
+
+ return pkey_skey2pkey((struct pkey_seckey *)key,
+ protkey);
+ default:
+ DEBUG_ERR("%s unknown/unsupported CCA internal token version %d\n",
+ __func__, hdr->version);
+ return -EINVAL;
+ }
+}
+
+/*
+ * Transform a key blob (of any type) into a protected key
+ */
+int pkey_keyblob2pkey(const __u8 *key, __u32 keylen,
+ struct pkey_protkey *protkey)
+{
+ struct keytoken_header *hdr = (struct keytoken_header *)key;
+
+ if (keylen < sizeof(struct keytoken_header))
+ return -EINVAL;
+
+ switch (hdr->type) {
+ case TOKTYPE_NON_CCA:
+ return pkey_nonccatok2pkey(key, keylen, protkey);
+ case TOKTYPE_CCA_INTERNAL:
+ return pkey_ccainttok2pkey(key, keylen, protkey);
+ default:
+ DEBUG_ERR("%s unknown/unsupported blob type %d\n", __func__,
+ hdr->type);
+ return -EINVAL;
+ }
+}
+EXPORT_SYMBOL(pkey_keyblob2pkey);
+
/*
* File io functions
*/
return -EFAULT;
break;
}
+ case PKEY_GENPROTK: {
+ struct pkey_genprotk __user *ugp = (void __user *) arg;
+ struct pkey_genprotk kgp;
+
+ if (copy_from_user(&kgp, ugp, sizeof(kgp)))
+ return -EFAULT;
+ rc = pkey_genprotkey(kgp.keytype, &kgp.protkey);
+ DEBUG_DBG("%s pkey_genprotkey()=%d\n", __func__, rc);
+ if (rc)
+ break;
+ if (copy_to_user(ugp, &kgp, sizeof(kgp)))
+ return -EFAULT;
+ break;
+ }
+ case PKEY_VERIFYPROTK: {
+ struct pkey_verifyprotk __user *uvp = (void __user *) arg;
+ struct pkey_verifyprotk kvp;
+
+ if (copy_from_user(&kvp, uvp, sizeof(kvp)))
+ return -EFAULT;
+ rc = pkey_verifyprotkey(&kvp.protkey);
+ DEBUG_DBG("%s pkey_verifyprotkey()=%d\n", __func__, rc);
+ break;
+ }
+ case PKEY_KBLOB2PROTK: {
+ struct pkey_kblob2pkey __user *utp = (void __user *) arg;
+ struct pkey_kblob2pkey ktp;
+ __u8 __user *ukey;
+ __u8 *kkey;
+
+ if (copy_from_user(&ktp, utp, sizeof(ktp)))
+ return -EFAULT;
+ if (ktp.keylen < MINKEYBLOBSIZE ||
+ ktp.keylen > MAXKEYBLOBSIZE)
+ return -EINVAL;
+ ukey = ktp.key;
+ kkey = kmalloc(ktp.keylen, GFP_KERNEL);
+ if (kkey == NULL)
+ return -ENOMEM;
+ if (copy_from_user(kkey, ukey, ktp.keylen)) {
+ kfree(kkey);
+ return -EFAULT;
+ }
+ rc = pkey_keyblob2pkey(kkey, ktp.keylen, &ktp.protkey);
+ DEBUG_DBG("%s pkey_keyblob2pkey()=%d\n", __func__, rc);
+ kfree(kkey);
+ if (rc)
+ break;
+ if (copy_to_user(utp, &ktp, sizeof(ktp)))
+ return -EFAULT;
+ break;
+ }
default:
/* unknown/unsupported ioctl cmd */
return -ENOTTY;
/*
* Sysfs and file io operations
*/
+
+/*
+ * Sysfs attribute read function for all protected key binary attributes.
+ * The implementation can not deal with partial reads, because a new random
+ * protected key blob is generated with each read. In case of partial reads
+ * (i.e. off != 0 or count < key blob size) -EINVAL is returned.
+ */
+static ssize_t pkey_protkey_aes_attr_read(u32 keytype, bool is_xts, char *buf,
+ loff_t off, size_t count)
+{
+ struct protaeskeytoken protkeytoken;
+ struct pkey_protkey protkey;
+ int rc;
+
+ if (off != 0 || count < sizeof(protkeytoken))
+ return -EINVAL;
+ if (is_xts)
+ if (count < 2 * sizeof(protkeytoken))
+ return -EINVAL;
+
+ memset(&protkeytoken, 0, sizeof(protkeytoken));
+ protkeytoken.type = TOKTYPE_NON_CCA;
+ protkeytoken.version = TOKVER_PROTECTED_KEY;
+ protkeytoken.keytype = keytype;
+
+ rc = pkey_genprotkey(protkeytoken.keytype, &protkey);
+ if (rc)
+ return rc;
+
+ protkeytoken.len = protkey.len;
+ memcpy(&protkeytoken.protkey, &protkey.protkey, protkey.len);
+
+ memcpy(buf, &protkeytoken, sizeof(protkeytoken));
+
+ if (is_xts) {
+ rc = pkey_genprotkey(protkeytoken.keytype, &protkey);
+ if (rc)
+ return rc;
+
+ protkeytoken.len = protkey.len;
+ memcpy(&protkeytoken.protkey, &protkey.protkey, protkey.len);
+
+ memcpy(buf + sizeof(protkeytoken), &protkeytoken,
+ sizeof(protkeytoken));
+
+ return 2 * sizeof(protkeytoken);
+ }
+
+ return sizeof(protkeytoken);
+}
+
+static ssize_t protkey_aes_128_read(struct file *filp,
+ struct kobject *kobj,
+ struct bin_attribute *attr,
+ char *buf, loff_t off,
+ size_t count)
+{
+ return pkey_protkey_aes_attr_read(PKEY_KEYTYPE_AES_128, false, buf,
+ off, count);
+}
+
+static ssize_t protkey_aes_192_read(struct file *filp,
+ struct kobject *kobj,
+ struct bin_attribute *attr,
+ char *buf, loff_t off,
+ size_t count)
+{
+ return pkey_protkey_aes_attr_read(PKEY_KEYTYPE_AES_192, false, buf,
+ off, count);
+}
+
+static ssize_t protkey_aes_256_read(struct file *filp,
+ struct kobject *kobj,
+ struct bin_attribute *attr,
+ char *buf, loff_t off,
+ size_t count)
+{
+ return pkey_protkey_aes_attr_read(PKEY_KEYTYPE_AES_256, false, buf,
+ off, count);
+}
+
+static ssize_t protkey_aes_128_xts_read(struct file *filp,
+ struct kobject *kobj,
+ struct bin_attribute *attr,
+ char *buf, loff_t off,
+ size_t count)
+{
+ return pkey_protkey_aes_attr_read(PKEY_KEYTYPE_AES_128, true, buf,
+ off, count);
+}
+
+static ssize_t protkey_aes_256_xts_read(struct file *filp,
+ struct kobject *kobj,
+ struct bin_attribute *attr,
+ char *buf, loff_t off,
+ size_t count)
+{
+ return pkey_protkey_aes_attr_read(PKEY_KEYTYPE_AES_256, true, buf,
+ off, count);
+}
+
+static BIN_ATTR_RO(protkey_aes_128, sizeof(struct protaeskeytoken));
+static BIN_ATTR_RO(protkey_aes_192, sizeof(struct protaeskeytoken));
+static BIN_ATTR_RO(protkey_aes_256, sizeof(struct protaeskeytoken));
+static BIN_ATTR_RO(protkey_aes_128_xts, 2 * sizeof(struct protaeskeytoken));
+static BIN_ATTR_RO(protkey_aes_256_xts, 2 * sizeof(struct protaeskeytoken));
+
+static struct bin_attribute *protkey_attrs[] = {
+ &bin_attr_protkey_aes_128,
+ &bin_attr_protkey_aes_192,
+ &bin_attr_protkey_aes_256,
+ &bin_attr_protkey_aes_128_xts,
+ &bin_attr_protkey_aes_256_xts,
+ NULL
+};
+
+static struct attribute_group protkey_attr_group = {
+ .name = "protkey",
+ .bin_attrs = protkey_attrs,
+};
+
+/*
+ * Sysfs attribute read function for all secure key ccadata binary attributes.
+ * The implementation can not deal with partial reads, because a new random
+ * protected key blob is generated with each read. In case of partial reads
+ * (i.e. off != 0 or count < key blob size) -EINVAL is returned.
+ */
+static ssize_t pkey_ccadata_aes_attr_read(u32 keytype, bool is_xts, char *buf,
+ loff_t off, size_t count)
+{
+ int rc;
+
+ if (off != 0 || count < sizeof(struct secaeskeytoken))
+ return -EINVAL;
+ if (is_xts)
+ if (count < 2 * sizeof(struct secaeskeytoken))
+ return -EINVAL;
+
+ rc = pkey_genseckey(-1, -1, keytype, (struct pkey_seckey *)buf);
+ if (rc)
+ return rc;
+
+ if (is_xts) {
+ buf += sizeof(struct pkey_seckey);
+ rc = pkey_genseckey(-1, -1, keytype, (struct pkey_seckey *)buf);
+ if (rc)
+ return rc;
+
+ return 2 * sizeof(struct secaeskeytoken);
+ }
+
+ return sizeof(struct secaeskeytoken);
+}
+
+static ssize_t ccadata_aes_128_read(struct file *filp,
+ struct kobject *kobj,
+ struct bin_attribute *attr,
+ char *buf, loff_t off,
+ size_t count)
+{
+ return pkey_ccadata_aes_attr_read(PKEY_KEYTYPE_AES_128, false, buf,
+ off, count);
+}
+
+static ssize_t ccadata_aes_192_read(struct file *filp,
+ struct kobject *kobj,
+ struct bin_attribute *attr,
+ char *buf, loff_t off,
+ size_t count)
+{
+ return pkey_ccadata_aes_attr_read(PKEY_KEYTYPE_AES_192, false, buf,
+ off, count);
+}
+
+static ssize_t ccadata_aes_256_read(struct file *filp,
+ struct kobject *kobj,
+ struct bin_attribute *attr,
+ char *buf, loff_t off,
+ size_t count)
+{
+ return pkey_ccadata_aes_attr_read(PKEY_KEYTYPE_AES_256, false, buf,
+ off, count);
+}
+
+static ssize_t ccadata_aes_128_xts_read(struct file *filp,
+ struct kobject *kobj,
+ struct bin_attribute *attr,
+ char *buf, loff_t off,
+ size_t count)
+{
+ return pkey_ccadata_aes_attr_read(PKEY_KEYTYPE_AES_128, true, buf,
+ off, count);
+}
+
+static ssize_t ccadata_aes_256_xts_read(struct file *filp,
+ struct kobject *kobj,
+ struct bin_attribute *attr,
+ char *buf, loff_t off,
+ size_t count)
+{
+ return pkey_ccadata_aes_attr_read(PKEY_KEYTYPE_AES_256, true, buf,
+ off, count);
+}
+
+static BIN_ATTR_RO(ccadata_aes_128, sizeof(struct secaeskeytoken));
+static BIN_ATTR_RO(ccadata_aes_192, sizeof(struct secaeskeytoken));
+static BIN_ATTR_RO(ccadata_aes_256, sizeof(struct secaeskeytoken));
+static BIN_ATTR_RO(ccadata_aes_128_xts, 2 * sizeof(struct secaeskeytoken));
+static BIN_ATTR_RO(ccadata_aes_256_xts, 2 * sizeof(struct secaeskeytoken));
+
+static struct bin_attribute *ccadata_attrs[] = {
+ &bin_attr_ccadata_aes_128,
+ &bin_attr_ccadata_aes_192,
+ &bin_attr_ccadata_aes_256,
+ &bin_attr_ccadata_aes_128_xts,
+ &bin_attr_ccadata_aes_256_xts,
+ NULL
+};
+
+static struct attribute_group ccadata_attr_group = {
+ .name = "ccadata",
+ .bin_attrs = ccadata_attrs,
+};
+
+static const struct attribute_group *pkey_attr_groups[] = {
+ &protkey_attr_group,
+ &ccadata_attr_group,
+ NULL,
+};
+
static const struct file_operations pkey_fops = {
.owner = THIS_MODULE,
.open = nonseekable_open,
.minor = MISC_DYNAMIC_MINOR,
.mode = 0666,
.fops = &pkey_fops,
+ .groups = pkey_attr_groups,
};
/*
*/
static int __init pkey_init(void)
{
- cpacf_mask_t pckmo_functions;
+ cpacf_mask_t kmc_functions;
- /* check for pckmo instructions available */
+ /*
+ * The pckmo instruction should be available - even if we don't
+ * actually invoke it. This instruction comes with MSA 3 which
+ * is also the minimum level for the kmc instructions which
+ * are able to work with protected keys.
+ */
if (!cpacf_query(CPACF_PCKMO, &pckmo_functions))
return -EOPNOTSUPP;
- if (!cpacf_test_func(&pckmo_functions, CPACF_PCKMO_ENC_AES_128_KEY) ||
- !cpacf_test_func(&pckmo_functions, CPACF_PCKMO_ENC_AES_192_KEY) ||
- !cpacf_test_func(&pckmo_functions, CPACF_PCKMO_ENC_AES_256_KEY))
+
+ /* check for kmc instructions available */
+ if (!cpacf_query(CPACF_KMC, &kmc_functions))
+ return -EOPNOTSUPP;
+ if (!cpacf_test_func(&kmc_functions, CPACF_KMC_PAES_128) ||
+ !cpacf_test_func(&kmc_functions, CPACF_KMC_PAES_192) ||
+ !cpacf_test_func(&kmc_functions, CPACF_KMC_PAES_256))
return -EOPNOTSUPP;
pkey_debug_init();
pkey_debug_exit();
}
-module_init(pkey_init);
+module_cpu_feature_match(MSA, pkey_init);
module_exit(pkey_exit);
// SPDX-License-Identifier: GPL-2.0+
/*
- * zcrypt 2.1.0
- *
- * Copyright IBM Corp. 2001, 2012
+ * Copyright IBM Corp. 2001, 2018
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
* Cornelia Huck <cornelia.huck@de.ibm.com>
* Major cleanup & driver split: Martin Schwidefsky <schwidefsky@de.ibm.com>
* Ralph Wuerthner <rwuerthn@de.ibm.com>
* MSGTYPE restruct: Holger Dengler <hd@linux.vnet.ibm.com>
+ * Multiple device nodes: Harald Freudenberger <freude@linux.ibm.com>
*/
#include <linux/module.h>
#include <linux/uaccess.h>
#include <linux/hw_random.h>
#include <linux/debugfs.h>
+#include <linux/cdev.h>
+#include <linux/ctype.h>
#include <asm/debug.h>
#define CREATE_TRACE_POINTS
}
EXPORT_SYMBOL(zcrypt_msgtype);
+/*
+ * Multi device nodes extension functions.
+ */
+
+#ifdef CONFIG_ZCRYPT_MULTIDEVNODES
+
+struct zcdn_device;
+
+static struct class *zcrypt_class;
+static dev_t zcrypt_devt;
+static struct cdev zcrypt_cdev;
+
+struct zcdn_device {
+ struct device device;
+ struct ap_perms perms;
+};
+
+#define to_zcdn_dev(x) container_of((x), struct zcdn_device, device)
+
+#define ZCDN_MAX_NAME 32
+
+static int zcdn_create(const char *name);
+static int zcdn_destroy(const char *name);
+
+/* helper function, matches the name for find_zcdndev_by_name() */
+static int __match_zcdn_name(struct device *dev, const void *data)
+{
+ return strcmp(dev_name(dev), (const char *)data) == 0;
+}
+
+/* helper function, matches the devt value for find_zcdndev_by_devt() */
+static int __match_zcdn_devt(struct device *dev, const void *data)
+{
+ return dev->devt == *((dev_t *) data);
+}
+
+/*
+ * Find zcdn device by name.
+ * Returns reference to the zcdn device which needs to be released
+ * with put_device() after use.
+ */
+static inline struct zcdn_device *find_zcdndev_by_name(const char *name)
+{
+ struct device *dev =
+ class_find_device(zcrypt_class, NULL,
+ (void *) name,
+ __match_zcdn_name);
+
+ return dev ? to_zcdn_dev(dev) : NULL;
+}
+
+/*
+ * Find zcdn device by devt value.
+ * Returns reference to the zcdn device which needs to be released
+ * with put_device() after use.
+ */
+static inline struct zcdn_device *find_zcdndev_by_devt(dev_t devt)
+{
+ struct device *dev =
+ class_find_device(zcrypt_class, NULL,
+ (void *) &devt,
+ __match_zcdn_devt);
+
+ return dev ? to_zcdn_dev(dev) : NULL;
+}
+
+static ssize_t ioctlmask_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ int i, rc;
+ struct zcdn_device *zcdndev = to_zcdn_dev(dev);
+
+ if (mutex_lock_interruptible(&ap_perms_mutex))
+ return -ERESTARTSYS;
+
+ buf[0] = '0';
+ buf[1] = 'x';
+ for (i = 0; i < sizeof(zcdndev->perms.ioctlm) / sizeof(long); i++)
+ snprintf(buf + 2 + 2 * i * sizeof(long),
+ PAGE_SIZE - 2 - 2 * i * sizeof(long),
+ "%016lx", zcdndev->perms.ioctlm[i]);
+ buf[2 + 2 * i * sizeof(long)] = '\n';
+ buf[2 + 2 * i * sizeof(long) + 1] = '\0';
+ rc = 2 + 2 * i * sizeof(long) + 1;
+
+ mutex_unlock(&ap_perms_mutex);
+
+ return rc;
+}
+
+static ssize_t ioctlmask_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ int rc;
+ struct zcdn_device *zcdndev = to_zcdn_dev(dev);
+
+ rc = ap_parse_mask_str(buf, zcdndev->perms.ioctlm,
+ AP_IOCTLS, &ap_perms_mutex);
+ if (rc)
+ return rc;
+
+ return count;
+}
+
+static DEVICE_ATTR_RW(ioctlmask);
+
+static ssize_t apmask_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ int i, rc;
+ struct zcdn_device *zcdndev = to_zcdn_dev(dev);
+
+ if (mutex_lock_interruptible(&ap_perms_mutex))
+ return -ERESTARTSYS;
+
+ buf[0] = '0';
+ buf[1] = 'x';
+ for (i = 0; i < sizeof(zcdndev->perms.apm) / sizeof(long); i++)
+ snprintf(buf + 2 + 2 * i * sizeof(long),
+ PAGE_SIZE - 2 - 2 * i * sizeof(long),
+ "%016lx", zcdndev->perms.apm[i]);
+ buf[2 + 2 * i * sizeof(long)] = '\n';
+ buf[2 + 2 * i * sizeof(long) + 1] = '\0';
+ rc = 2 + 2 * i * sizeof(long) + 1;
+
+ mutex_unlock(&ap_perms_mutex);
+
+ return rc;
+}
+
+static ssize_t apmask_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ int rc;
+ struct zcdn_device *zcdndev = to_zcdn_dev(dev);
+
+ rc = ap_parse_mask_str(buf, zcdndev->perms.apm,
+ AP_DEVICES, &ap_perms_mutex);
+ if (rc)
+ return rc;
+
+ return count;
+}
+
+static DEVICE_ATTR_RW(apmask);
+
+static ssize_t aqmask_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ int i, rc;
+ struct zcdn_device *zcdndev = to_zcdn_dev(dev);
+
+ if (mutex_lock_interruptible(&ap_perms_mutex))
+ return -ERESTARTSYS;
+
+ buf[0] = '0';
+ buf[1] = 'x';
+ for (i = 0; i < sizeof(zcdndev->perms.aqm) / sizeof(long); i++)
+ snprintf(buf + 2 + 2 * i * sizeof(long),
+ PAGE_SIZE - 2 - 2 * i * sizeof(long),
+ "%016lx", zcdndev->perms.aqm[i]);
+ buf[2 + 2 * i * sizeof(long)] = '\n';
+ buf[2 + 2 * i * sizeof(long) + 1] = '\0';
+ rc = 2 + 2 * i * sizeof(long) + 1;
+
+ mutex_unlock(&ap_perms_mutex);
+
+ return rc;
+}
+
+static ssize_t aqmask_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ int rc;
+ struct zcdn_device *zcdndev = to_zcdn_dev(dev);
+
+ rc = ap_parse_mask_str(buf, zcdndev->perms.aqm,
+ AP_DOMAINS, &ap_perms_mutex);
+ if (rc)
+ return rc;
+
+ return count;
+}
+
+static DEVICE_ATTR_RW(aqmask);
+
+static struct attribute *zcdn_dev_attrs[] = {
+ &dev_attr_ioctlmask.attr,
+ &dev_attr_apmask.attr,
+ &dev_attr_aqmask.attr,
+ NULL
+};
+
+static struct attribute_group zcdn_dev_attr_group = {
+ .attrs = zcdn_dev_attrs
+};
+
+static const struct attribute_group *zcdn_dev_attr_groups[] = {
+ &zcdn_dev_attr_group,
+ NULL
+};
+
+static ssize_t zcdn_create_store(struct class *class,
+ struct class_attribute *attr,
+ const char *buf, size_t count)
+{
+ int rc;
+ char name[ZCDN_MAX_NAME];
+
+ strncpy(name, skip_spaces(buf), sizeof(name));
+ name[sizeof(name) - 1] = '\0';
+
+ rc = zcdn_create(strim(name));
+
+ return rc ? rc : count;
+}
+
+static const struct class_attribute class_attr_zcdn_create =
+ __ATTR(create, 0600, NULL, zcdn_create_store);
+
+static ssize_t zcdn_destroy_store(struct class *class,
+ struct class_attribute *attr,
+ const char *buf, size_t count)
+{
+ int rc;
+ char name[ZCDN_MAX_NAME];
+
+ strncpy(name, skip_spaces(buf), sizeof(name));
+ name[sizeof(name) - 1] = '\0';
+
+ rc = zcdn_destroy(strim(name));
+
+ return rc ? rc : count;
+}
+
+static const struct class_attribute class_attr_zcdn_destroy =
+ __ATTR(destroy, 0600, NULL, zcdn_destroy_store);
+
+static void zcdn_device_release(struct device *dev)
+{
+ struct zcdn_device *zcdndev = to_zcdn_dev(dev);
+
+ ZCRYPT_DBF(DBF_INFO, "releasing zcdn device %d:%d\n",
+ MAJOR(dev->devt), MINOR(dev->devt));
+
+ kfree(zcdndev);
+}
+
+static int zcdn_create(const char *name)
+{
+ dev_t devt;
+ int i, rc = 0;
+ char nodename[ZCDN_MAX_NAME];
+ struct zcdn_device *zcdndev;
+
+ if (mutex_lock_interruptible(&ap_perms_mutex))
+ return -ERESTARTSYS;
+
+ /* check if device node with this name already exists */
+ if (name[0]) {
+ zcdndev = find_zcdndev_by_name(name);
+ if (zcdndev) {
+ put_device(&zcdndev->device);
+ rc = -EEXIST;
+ goto unlockout;
+ }
+ }
+
+ /* find an unused minor number */
+ for (i = 0; i < ZCRYPT_MAX_MINOR_NODES; i++) {
+ devt = MKDEV(MAJOR(zcrypt_devt), MINOR(zcrypt_devt) + i);
+ zcdndev = find_zcdndev_by_devt(devt);
+ if (zcdndev)
+ put_device(&zcdndev->device);
+ else
+ break;
+ }
+ if (i == ZCRYPT_MAX_MINOR_NODES) {
+ rc = -ENOSPC;
+ goto unlockout;
+ }
+
+ /* alloc and prepare a new zcdn device */
+ zcdndev = kzalloc(sizeof(*zcdndev), GFP_KERNEL);
+ if (!zcdndev) {
+ rc = -ENOMEM;
+ goto unlockout;
+ }
+ zcdndev->device.release = zcdn_device_release;
+ zcdndev->device.class = zcrypt_class;
+ zcdndev->device.devt = devt;
+ zcdndev->device.groups = zcdn_dev_attr_groups;
+ if (name[0])
+ strncpy(nodename, name, sizeof(nodename));
+ else
+ snprintf(nodename, sizeof(nodename),
+ ZCRYPT_NAME "_%d", (int) MINOR(devt));
+ nodename[sizeof(nodename)-1] = '\0';
+ if (dev_set_name(&zcdndev->device, nodename)) {
+ rc = -EINVAL;
+ goto unlockout;
+ }
+ rc = device_register(&zcdndev->device);
+ if (rc) {
+ put_device(&zcdndev->device);
+ goto unlockout;
+ }
+
+ ZCRYPT_DBF(DBF_INFO, "created zcdn device %d:%d\n",
+ MAJOR(devt), MINOR(devt));
+
+unlockout:
+ mutex_unlock(&ap_perms_mutex);
+ return rc;
+}
+
+static int zcdn_destroy(const char *name)
+{
+ int rc = 0;
+ struct zcdn_device *zcdndev;
+
+ if (mutex_lock_interruptible(&ap_perms_mutex))
+ return -ERESTARTSYS;
+
+ /* try to find this zcdn device */
+ zcdndev = find_zcdndev_by_name(name);
+ if (!zcdndev) {
+ rc = -ENOENT;
+ goto unlockout;
+ }
+
+ /*
+ * The zcdn device is not hard destroyed. It is subject to
+ * reference counting and thus just needs to be unregistered.
+ */
+ put_device(&zcdndev->device);
+ device_unregister(&zcdndev->device);
+
+unlockout:
+ mutex_unlock(&ap_perms_mutex);
+ return rc;
+}
+
+static void zcdn_destroy_all(void)
+{
+ int i;
+ dev_t devt;
+ struct zcdn_device *zcdndev;
+
+ mutex_lock(&ap_perms_mutex);
+ for (i = 0; i < ZCRYPT_MAX_MINOR_NODES; i++) {
+ devt = MKDEV(MAJOR(zcrypt_devt), MINOR(zcrypt_devt) + i);
+ zcdndev = find_zcdndev_by_devt(devt);
+ if (zcdndev) {
+ put_device(&zcdndev->device);
+ device_unregister(&zcdndev->device);
+ }
+ }
+ mutex_unlock(&ap_perms_mutex);
+}
+
+#endif
+
/**
* zcrypt_read (): Not supported beyond zcrypt 1.3.1.
*
*/
static int zcrypt_open(struct inode *inode, struct file *filp)
{
+ struct ap_perms *perms = &ap_perms;
+
+#ifdef CONFIG_ZCRYPT_MULTIDEVNODES
+ if (filp->f_inode->i_cdev == &zcrypt_cdev) {
+ struct zcdn_device *zcdndev;
+
+ if (mutex_lock_interruptible(&ap_perms_mutex))
+ return -ERESTARTSYS;
+ zcdndev = find_zcdndev_by_devt(filp->f_inode->i_rdev);
+ /* find returns a reference, no get_device() needed */
+ mutex_unlock(&ap_perms_mutex);
+ if (zcdndev)
+ perms = &zcdndev->perms;
+ }
+#endif
+ filp->private_data = (void *) perms;
+
atomic_inc(&zcrypt_open_count);
return nonseekable_open(inode, filp);
}
*/
static int zcrypt_release(struct inode *inode, struct file *filp)
{
+#ifdef CONFIG_ZCRYPT_MULTIDEVNODES
+ if (filp->f_inode->i_cdev == &zcrypt_cdev) {
+ struct zcdn_device *zcdndev;
+
+ if (mutex_lock_interruptible(&ap_perms_mutex))
+ return -ERESTARTSYS;
+ zcdndev = find_zcdndev_by_devt(filp->f_inode->i_rdev);
+ mutex_unlock(&ap_perms_mutex);
+ if (zcdndev) {
+ /* 2 puts here: one for find, one for open */
+ put_device(&zcdndev->device);
+ put_device(&zcdndev->device);
+ }
+ }
+#endif
+
atomic_dec(&zcrypt_open_count);
return 0;
}
+static inline int zcrypt_check_ioctl(struct ap_perms *perms,
+ unsigned int cmd)
+{
+ int rc = -EPERM;
+ int ioctlnr = (cmd & _IOC_NRMASK) >> _IOC_NRSHIFT;
+
+ if (ioctlnr > 0 && ioctlnr < AP_IOCTLS) {
+ if (test_bit_inv(ioctlnr, perms->ioctlm))
+ rc = 0;
+ }
+
+ if (rc)
+ ZCRYPT_DBF(DBF_WARN,
+ "ioctl check failed: ioctlnr=0x%04x rc=%d\n",
+ ioctlnr, rc);
+
+ return rc;
+}
+
+static inline bool zcrypt_check_card(struct ap_perms *perms, int card)
+{
+ return test_bit_inv(card, perms->apm) ? true : false;
+}
+
+static inline bool zcrypt_check_queue(struct ap_perms *perms, int queue)
+{
+ return test_bit_inv(queue, perms->aqm) ? true : false;
+}
+
static inline struct zcrypt_queue *zcrypt_pick_queue(struct zcrypt_card *zc,
struct zcrypt_queue *zq,
unsigned int weight)
/*
* zcrypt ioctls.
*/
-static long zcrypt_rsa_modexpo(struct ica_rsa_modexpo *mex)
+static long zcrypt_rsa_modexpo(struct ap_perms *perms,
+ struct ica_rsa_modexpo *mex)
{
struct zcrypt_card *zc, *pref_zc;
struct zcrypt_queue *zq, *pref_zq;
if (zc->min_mod_size > mex->inputdatalength ||
zc->max_mod_size < mex->inputdatalength)
continue;
+ /* check if device node has admission for this card */
+ if (!zcrypt_check_card(perms, zc->card->id))
+ continue;
/* get weight index of the card device */
weight = zc->speed_rating[func_code];
if (zcrypt_card_compare(zc, pref_zc, weight, pref_weight))
/* check if device is online and eligible */
if (!zq->online || !zq->ops->rsa_modexpo)
continue;
+ /* check if device node has admission for this queue */
+ if (!zcrypt_check_queue(perms,
+ AP_QID_QUEUE(zq->queue->qid)))
+ continue;
if (zcrypt_queue_compare(zq, pref_zq,
weight, pref_weight))
continue;
return rc;
}
-static long zcrypt_rsa_crt(struct ica_rsa_modexpo_crt *crt)
+static long zcrypt_rsa_crt(struct ap_perms *perms,
+ struct ica_rsa_modexpo_crt *crt)
{
struct zcrypt_card *zc, *pref_zc;
struct zcrypt_queue *zq, *pref_zq;
if (zc->min_mod_size > crt->inputdatalength ||
zc->max_mod_size < crt->inputdatalength)
continue;
+ /* check if device node has admission for this card */
+ if (!zcrypt_check_card(perms, zc->card->id))
+ continue;
/* get weight index of the card device */
weight = zc->speed_rating[func_code];
if (zcrypt_card_compare(zc, pref_zc, weight, pref_weight))
/* check if device is online and eligible */
if (!zq->online || !zq->ops->rsa_modexpo_crt)
continue;
+ /* check if device node has admission for this queue */
+ if (!zcrypt_check_queue(perms,
+ AP_QID_QUEUE(zq->queue->qid)))
+ continue;
if (zcrypt_queue_compare(zq, pref_zq,
weight, pref_weight))
continue;
return rc;
}
-long zcrypt_send_cprb(struct ica_xcRB *xcRB)
+static long _zcrypt_send_cprb(struct ap_perms *perms,
+ struct ica_xcRB *xcRB)
{
struct zcrypt_card *zc, *pref_zc;
struct zcrypt_queue *zq, *pref_zq;
trace_s390_zcrypt_req(xcRB, TB_ZSECSENDCPRB);
+ xcRB->status = 0;
ap_init_message(&ap_msg);
rc = get_cprb_fc(xcRB, &ap_msg, &func_code, &domain);
if (rc)
if (xcRB->user_defined != AUTOSELECT &&
xcRB->user_defined != zc->card->id)
continue;
+ /* check if device node has admission for this card */
+ if (!zcrypt_check_card(perms, zc->card->id))
+ continue;
/* get weight index of the card device */
weight = speed_idx_cca(func_code) * zc->speed_rating[SECKEY];
if (zcrypt_card_compare(zc, pref_zc, weight, pref_weight))
((*domain != (unsigned short) AUTOSELECT) &&
(*domain != AP_QID_QUEUE(zq->queue->qid))))
continue;
+ /* check if device node has admission for this queue */
+ if (!zcrypt_check_queue(perms,
+ AP_QID_QUEUE(zq->queue->qid)))
+ continue;
if (zcrypt_queue_compare(zq, pref_zq,
weight, pref_weight))
continue;
AP_QID_CARD(qid), AP_QID_QUEUE(qid));
return rc;
}
+
+long zcrypt_send_cprb(struct ica_xcRB *xcRB)
+{
+ return _zcrypt_send_cprb(&ap_perms, xcRB);
+}
EXPORT_SYMBOL(zcrypt_send_cprb);
static bool is_desired_ep11_card(unsigned int dev_id,
return false;
}
-static long zcrypt_send_ep11_cprb(struct ep11_urb *xcrb)
+static long zcrypt_send_ep11_cprb(struct ap_perms *perms,
+ struct ep11_urb *xcrb)
{
struct zcrypt_card *zc, *pref_zc;
struct zcrypt_queue *zq, *pref_zq;
if (targets &&
!is_desired_ep11_card(zc->card->id, target_num, targets))
continue;
+ /* check if device node has admission for this card */
+ if (!zcrypt_check_card(perms, zc->card->id))
+ continue;
/* get weight index of the card device */
weight = speed_idx_ep11(func_code) * zc->speed_rating[SECKEY];
if (zcrypt_card_compare(zc, pref_zc, weight, pref_weight))
!is_desired_ep11_queue(zq->queue->qid,
target_num, targets)))
continue;
+ /* check if device node has admission for this queue */
+ if (!zcrypt_check_queue(perms,
+ AP_QID_QUEUE(zq->queue->qid)))
+ continue;
if (zcrypt_queue_compare(zq, pref_zq,
weight, pref_weight))
continue;
static long zcrypt_unlocked_ioctl(struct file *filp, unsigned int cmd,
unsigned long arg)
{
- int rc = 0;
+ int rc;
+ struct ap_perms *perms =
+ (struct ap_perms *) filp->private_data;
+
+ rc = zcrypt_check_ioctl(perms, cmd);
+ if (rc)
+ return rc;
switch (cmd) {
case ICARSAMODEXPO: {
if (copy_from_user(&mex, umex, sizeof(mex)))
return -EFAULT;
do {
- rc = zcrypt_rsa_modexpo(&mex);
+ rc = zcrypt_rsa_modexpo(perms, &mex);
} while (rc == -EAGAIN);
/* on failure: retry once again after a requested rescan */
if ((rc == -ENODEV) && (zcrypt_process_rescan()))
do {
- rc = zcrypt_rsa_modexpo(&mex);
+ rc = zcrypt_rsa_modexpo(perms, &mex);
} while (rc == -EAGAIN);
if (rc) {
ZCRYPT_DBF(DBF_DEBUG, "ioctl ICARSAMODEXPO rc=%d\n", rc);
if (copy_from_user(&crt, ucrt, sizeof(crt)))
return -EFAULT;
do {
- rc = zcrypt_rsa_crt(&crt);
+ rc = zcrypt_rsa_crt(perms, &crt);
} while (rc == -EAGAIN);
/* on failure: retry once again after a requested rescan */
if ((rc == -ENODEV) && (zcrypt_process_rescan()))
do {
- rc = zcrypt_rsa_crt(&crt);
+ rc = zcrypt_rsa_crt(perms, &crt);
} while (rc == -EAGAIN);
if (rc) {
ZCRYPT_DBF(DBF_DEBUG, "ioctl ICARSACRT rc=%d\n", rc);
if (copy_from_user(&xcRB, uxcRB, sizeof(xcRB)))
return -EFAULT;
do {
- rc = zcrypt_send_cprb(&xcRB);
+ rc = _zcrypt_send_cprb(perms, &xcRB);
} while (rc == -EAGAIN);
/* on failure: retry once again after a requested rescan */
if ((rc == -ENODEV) && (zcrypt_process_rescan()))
do {
- rc = zcrypt_send_cprb(&xcRB);
+ rc = _zcrypt_send_cprb(perms, &xcRB);
} while (rc == -EAGAIN);
if (rc)
- ZCRYPT_DBF(DBF_DEBUG, "ioctl ZSENDCPRB rc=%d\n", rc);
+ ZCRYPT_DBF(DBF_DEBUG, "ioctl ZSENDCPRB rc=%d status=0x%x\n",
+ rc, xcRB.status);
if (copy_to_user(uxcRB, &xcRB, sizeof(xcRB)))
return -EFAULT;
return rc;
if (copy_from_user(&xcrb, uxcrb, sizeof(xcrb)))
return -EFAULT;
do {
- rc = zcrypt_send_ep11_cprb(&xcrb);
+ rc = zcrypt_send_ep11_cprb(perms, &xcrb);
} while (rc == -EAGAIN);
/* on failure: retry once again after a requested rescan */
if ((rc == -ENODEV) && (zcrypt_process_rescan()))
do {
- rc = zcrypt_send_ep11_cprb(&xcrb);
+ rc = zcrypt_send_ep11_cprb(perms, &xcrb);
} while (rc == -EAGAIN);
if (rc)
ZCRYPT_DBF(DBF_DEBUG, "ioctl ZSENDEP11CPRB rc=%d\n", rc);
compat_uptr_t n_modulus;
};
-static long trans_modexpo32(struct file *filp, unsigned int cmd,
- unsigned long arg)
+static long trans_modexpo32(struct ap_perms *perms, struct file *filp,
+ unsigned int cmd, unsigned long arg)
{
struct compat_ica_rsa_modexpo __user *umex32 = compat_ptr(arg);
struct compat_ica_rsa_modexpo mex32;
mex64.b_key = compat_ptr(mex32.b_key);
mex64.n_modulus = compat_ptr(mex32.n_modulus);
do {
- rc = zcrypt_rsa_modexpo(&mex64);
+ rc = zcrypt_rsa_modexpo(perms, &mex64);
} while (rc == -EAGAIN);
/* on failure: retry once again after a requested rescan */
if ((rc == -ENODEV) && (zcrypt_process_rescan()))
do {
- rc = zcrypt_rsa_modexpo(&mex64);
+ rc = zcrypt_rsa_modexpo(perms, &mex64);
} while (rc == -EAGAIN);
if (rc)
return rc;
compat_uptr_t u_mult_inv;
};
-static long trans_modexpo_crt32(struct file *filp, unsigned int cmd,
- unsigned long arg)
+static long trans_modexpo_crt32(struct ap_perms *perms, struct file *filp,
+ unsigned int cmd, unsigned long arg)
{
struct compat_ica_rsa_modexpo_crt __user *ucrt32 = compat_ptr(arg);
struct compat_ica_rsa_modexpo_crt crt32;
crt64.nq_prime = compat_ptr(crt32.nq_prime);
crt64.u_mult_inv = compat_ptr(crt32.u_mult_inv);
do {
- rc = zcrypt_rsa_crt(&crt64);
+ rc = zcrypt_rsa_crt(perms, &crt64);
} while (rc == -EAGAIN);
/* on failure: retry once again after a requested rescan */
if ((rc == -ENODEV) && (zcrypt_process_rescan()))
do {
- rc = zcrypt_rsa_crt(&crt64);
+ rc = zcrypt_rsa_crt(perms, &crt64);
} while (rc == -EAGAIN);
if (rc)
return rc;
unsigned int status;
} __packed;
-static long trans_xcRB32(struct file *filp, unsigned int cmd,
- unsigned long arg)
+static long trans_xcRB32(struct ap_perms *perms, struct file *filp,
+ unsigned int cmd, unsigned long arg)
{
struct compat_ica_xcRB __user *uxcRB32 = compat_ptr(arg);
struct compat_ica_xcRB xcRB32;
xcRB64.priority_window = xcRB32.priority_window;
xcRB64.status = xcRB32.status;
do {
- rc = zcrypt_send_cprb(&xcRB64);
+ rc = _zcrypt_send_cprb(perms, &xcRB64);
} while (rc == -EAGAIN);
/* on failure: retry once again after a requested rescan */
if ((rc == -ENODEV) && (zcrypt_process_rescan()))
do {
- rc = zcrypt_send_cprb(&xcRB64);
+ rc = _zcrypt_send_cprb(perms, &xcRB64);
} while (rc == -EAGAIN);
xcRB32.reply_control_blk_length = xcRB64.reply_control_blk_length;
xcRB32.reply_data_length = xcRB64.reply_data_length;
static long zcrypt_compat_ioctl(struct file *filp, unsigned int cmd,
unsigned long arg)
{
+ int rc;
+ struct ap_perms *perms =
+ (struct ap_perms *) filp->private_data;
+
+ rc = zcrypt_check_ioctl(perms, cmd);
+ if (rc)
+ return rc;
+
if (cmd == ICARSAMODEXPO)
- return trans_modexpo32(filp, cmd, arg);
+ return trans_modexpo32(perms, filp, cmd, arg);
if (cmd == ICARSACRT)
- return trans_modexpo_crt32(filp, cmd, arg);
+ return trans_modexpo_crt32(perms, filp, cmd, arg);
if (cmd == ZSECSENDCPRB)
- return trans_xcRB32(filp, cmd, arg);
+ return trans_xcRB32(perms, filp, cmd, arg);
return zcrypt_unlocked_ioctl(filp, cmd, arg);
}
#endif
debug_unregister(zcrypt_dbf_info);
}
+#ifdef CONFIG_ZCRYPT_MULTIDEVNODES
+
+static int __init zcdn_init(void)
+{
+ int rc;
+
+ /* create a new class 'zcrypt' */
+ zcrypt_class = class_create(THIS_MODULE, ZCRYPT_NAME);
+ if (IS_ERR(zcrypt_class)) {
+ rc = PTR_ERR(zcrypt_class);
+ goto out_class_create_failed;
+ }
+ zcrypt_class->dev_release = zcdn_device_release;
+
+ /* alloc device minor range */
+ rc = alloc_chrdev_region(&zcrypt_devt,
+ 0, ZCRYPT_MAX_MINOR_NODES,
+ ZCRYPT_NAME);
+ if (rc)
+ goto out_alloc_chrdev_failed;
+
+ cdev_init(&zcrypt_cdev, &zcrypt_fops);
+ zcrypt_cdev.owner = THIS_MODULE;
+ rc = cdev_add(&zcrypt_cdev, zcrypt_devt, ZCRYPT_MAX_MINOR_NODES);
+ if (rc)
+ goto out_cdev_add_failed;
+
+ /* need some class specific sysfs attributes */
+ rc = class_create_file(zcrypt_class, &class_attr_zcdn_create);
+ if (rc)
+ goto out_class_create_file_1_failed;
+ rc = class_create_file(zcrypt_class, &class_attr_zcdn_destroy);
+ if (rc)
+ goto out_class_create_file_2_failed;
+
+ return 0;
+
+out_class_create_file_2_failed:
+ class_remove_file(zcrypt_class, &class_attr_zcdn_create);
+out_class_create_file_1_failed:
+ cdev_del(&zcrypt_cdev);
+out_cdev_add_failed:
+ unregister_chrdev_region(zcrypt_devt, ZCRYPT_MAX_MINOR_NODES);
+out_alloc_chrdev_failed:
+ class_destroy(zcrypt_class);
+out_class_create_failed:
+ return rc;
+}
+
+static void zcdn_exit(void)
+{
+ class_remove_file(zcrypt_class, &class_attr_zcdn_create);
+ class_remove_file(zcrypt_class, &class_attr_zcdn_destroy);
+ zcdn_destroy_all();
+ cdev_del(&zcrypt_cdev);
+ unregister_chrdev_region(zcrypt_devt, ZCRYPT_MAX_MINOR_NODES);
+ class_destroy(zcrypt_class);
+}
+
+#endif
+
/**
* zcrypt_api_init(): Module initialization.
*
if (rc)
goto out;
+#ifdef CONFIG_ZCRYPT_MULTIDEVNODES
+ rc = zcdn_init();
+ if (rc)
+ goto out;
+#endif
+
/* Register the request sprayer. */
rc = misc_register(&zcrypt_misc_device);
if (rc < 0)
- goto out;
+ goto out_misc_register_failed;
zcrypt_msgtype6_init();
zcrypt_msgtype50_init();
+
return 0;
+out_misc_register_failed:
+#ifdef CONFIG_ZCRYPT_MULTIDEVNODES
+ zcdn_exit();
+#endif
+ zcrypt_debug_exit();
out:
return rc;
}
*/
void __exit zcrypt_api_exit(void)
{
+#ifdef CONFIG_ZCRYPT_MULTIDEVNODES
+ zcdn_exit();
+#endif
misc_deregister(&zcrypt_misc_device);
zcrypt_msgtype6_exit();
zcrypt_msgtype50_exit();
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * zcrypt 2.1.0
- *
- * Copyright IBM Corp. 2001, 2012
+ * Copyright IBM Corp. 2001, 2018
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
* Cornelia Huck <cornelia.huck@de.ibm.com>
#include "ap_bus.h"
/**
- * device type for an actual device is either PCICA, PCICC, PCIXCC_MCL2,
- * PCIXCC_MCL3, CEX2C, or CEX2A
- *
- * NOTE: PCIXCC_MCL3 refers to a PCIXCC with May 2004 version of Licensed
- * Internal Code (LIC) (EC J12220 level 29).
- * PCIXCC_MCL2 refers to any LIC before this level.
+ * Supported device types
*/
-#define ZCRYPT_PCICA 1
-#define ZCRYPT_PCICC 2
-#define ZCRYPT_PCIXCC_MCL2 3
-#define ZCRYPT_PCIXCC_MCL3 4
#define ZCRYPT_CEX2C 5
#define ZCRYPT_CEX2A 6
#define ZCRYPT_CEX3C 7
// SPDX-License-Identifier: GPL-2.0+
/*
- * zcrypt 2.1.0
- *
* Copyright IBM Corp. 2001, 2012
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * zcrypt 2.1.0
- *
* Copyright IBM Corp. 2001, 2006
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
// SPDX-License-Identifier: GPL-2.0+
/*
- * zcrypt 2.1.0
- *
* Copyright IBM Corp. 2001, 2012
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
#define CEX3A_CLEANUP_TIME CEX2A_CLEANUP_TIME
MODULE_AUTHOR("IBM Corporation");
-MODULE_DESCRIPTION("CEX2A Cryptographic Coprocessor device driver, " \
- "Copyright IBM Corp. 2001, 2012");
+MODULE_DESCRIPTION("CEX2A/CEX3A Cryptographic Coprocessor device driver, " \
+ "Copyright IBM Corp. 2001, 2018");
MODULE_LICENSE("GPL");
static struct ap_device_id zcrypt_cex2a_card_ids[] = {
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * zcrypt 2.1.0
- *
* Copyright IBM Corp. 2001, 2006
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
#define _ZCRYPT_CEX2A_H_
/**
- * The type 50 message family is associated with a CEX2A card.
+ * The type 50 message family is associated with CEXxA cards.
*
* The four members of the family are described below.
*
} __packed;
/**
- * The type 80 response family is associated with a CEX2A card.
+ * The type 80 response family is associated with a CEXxA cards.
*
* Note that all unsigned char arrays are right-justified and left-padded
* with zeroes.
// SPDX-License-Identifier: GPL-2.0+
/*
- * zcrypt 2.1.0
- *
- * Copyright IBM Corp. 2001, 2012
+ * Copyright IBM Corp. 2001, 2018
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
*
#include "zcrypt_api.h"
#include "zcrypt_error.h"
#include "zcrypt_msgtype6.h"
-#include "zcrypt_pcixcc.h"
+#include "zcrypt_cex2c.h"
#include "zcrypt_cca_key.h"
-#define PCIXCC_MIN_MOD_SIZE 16 /* 128 bits */
-#define PCIXCC_MIN_MOD_SIZE_OLD 64 /* 512 bits */
-#define PCIXCC_MAX_MOD_SIZE 256 /* 2048 bits */
-#define CEX3C_MIN_MOD_SIZE PCIXCC_MIN_MOD_SIZE
+#define CEX2C_MIN_MOD_SIZE 16 /* 128 bits */
+#define CEX2C_MAX_MOD_SIZE 256 /* 2048 bits */
+#define CEX3C_MIN_MOD_SIZE 16 /* 128 bits */
#define CEX3C_MAX_MOD_SIZE 512 /* 4096 bits */
-
-#define PCIXCC_MAX_ICA_MESSAGE_SIZE 0x77c /* max size type6 v2 crt message */
-#define PCIXCC_MAX_ICA_RESPONSE_SIZE 0x77c /* max size type86 v2 reply */
-
-#define PCIXCC_MAX_XCRB_MESSAGE_SIZE (12*1024)
-
-#define PCIXCC_CLEANUP_TIME (15*HZ)
-
-#define CEIL4(x) ((((x)+3)/4)*4)
-
-struct response_type {
- struct completion work;
- int type;
-};
-#define PCIXCC_RESPONSE_TYPE_ICA 0
-#define PCIXCC_RESPONSE_TYPE_XCRB 1
+#define CEX2C_MAX_XCRB_MESSAGE_SIZE (12*1024)
+#define CEX2C_CLEANUP_TIME (15*HZ)
MODULE_AUTHOR("IBM Corporation");
-MODULE_DESCRIPTION("PCIXCC Cryptographic Coprocessor device driver, " \
- "Copyright IBM Corp. 2001, 2012");
+MODULE_DESCRIPTION("CEX2C/CEX3C Cryptographic Coprocessor device driver, " \
+ "Copyright IBM Corp. 2001, 2018");
MODULE_LICENSE("GPL");
-static struct ap_device_id zcrypt_pcixcc_card_ids[] = {
- { .dev_type = AP_DEVICE_TYPE_PCIXCC,
- .match_flags = AP_DEVICE_ID_MATCH_CARD_TYPE },
+static struct ap_device_id zcrypt_cex2c_card_ids[] = {
{ .dev_type = AP_DEVICE_TYPE_CEX2C,
.match_flags = AP_DEVICE_ID_MATCH_CARD_TYPE },
{ .dev_type = AP_DEVICE_TYPE_CEX3C,
{ /* end of list */ },
};
-MODULE_DEVICE_TABLE(ap, zcrypt_pcixcc_card_ids);
+MODULE_DEVICE_TABLE(ap, zcrypt_cex2c_card_ids);
-static struct ap_device_id zcrypt_pcixcc_queue_ids[] = {
- { .dev_type = AP_DEVICE_TYPE_PCIXCC,
- .match_flags = AP_DEVICE_ID_MATCH_QUEUE_TYPE },
+static struct ap_device_id zcrypt_cex2c_queue_ids[] = {
{ .dev_type = AP_DEVICE_TYPE_CEX2C,
.match_flags = AP_DEVICE_ID_MATCH_QUEUE_TYPE },
{ .dev_type = AP_DEVICE_TYPE_CEX3C,
{ /* end of list */ },
};
-MODULE_DEVICE_TABLE(ap, zcrypt_pcixcc_queue_ids);
+MODULE_DEVICE_TABLE(ap, zcrypt_cex2c_queue_ids);
/**
- * Large random number detection function. Its sends a message to a pcixcc
+ * Large random number detection function. Its sends a message to a CEX2C/CEX3C
* card to find out if large random numbers are supported.
* @ap_dev: pointer to the AP device.
*
* Returns 1 if large random numbers are supported, 0 if not and < 0 on error.
*/
-static int zcrypt_pcixcc_rng_supported(struct ap_queue *aq)
+static int zcrypt_cex2c_rng_supported(struct ap_queue *aq)
{
struct ap_message ap_msg;
unsigned long long psmid;
}
/**
- * Probe function for PCIXCC/CEX2C card devices. It always accepts the
- * AP device since the bus_match already checked the hardware type. The
- * PCIXCC cards come in two flavours: micro code level 2 and micro code
- * level 3. This is checked by sending a test message to the device.
+ * Probe function for CEX2C/CEX3C card devices. It always accepts the
+ * AP device since the bus_match already checked the hardware type.
* @ap_dev: pointer to the AP card device.
*/
-static int zcrypt_pcixcc_card_probe(struct ap_device *ap_dev)
+static int zcrypt_cex2c_card_probe(struct ap_device *ap_dev)
{
/*
* Normalized speed ratings per crypto adapter
zc->type_string = "CEX2C";
memcpy(zc->speed_rating, CEX2C_SPEED_IDX,
sizeof(CEX2C_SPEED_IDX));
- zc->min_mod_size = PCIXCC_MIN_MOD_SIZE;
- zc->max_mod_size = PCIXCC_MAX_MOD_SIZE;
- zc->max_exp_bit_length = PCIXCC_MAX_MOD_SIZE;
+ zc->min_mod_size = CEX2C_MIN_MOD_SIZE;
+ zc->max_mod_size = CEX2C_MAX_MOD_SIZE;
+ zc->max_exp_bit_length = CEX2C_MAX_MOD_SIZE;
break;
case AP_DEVICE_TYPE_CEX3C:
zc->user_space_type = ZCRYPT_CEX3C;
}
/**
- * This is called to remove the PCIXCC/CEX2C card driver information
+ * This is called to remove the CEX2C/CEX3C card driver information
* if an AP card device is removed.
*/
-static void zcrypt_pcixcc_card_remove(struct ap_device *ap_dev)
+static void zcrypt_cex2c_card_remove(struct ap_device *ap_dev)
{
struct zcrypt_card *zc = to_ap_card(&ap_dev->device)->private;
zcrypt_card_unregister(zc);
}
-static struct ap_driver zcrypt_pcixcc_card_driver = {
- .probe = zcrypt_pcixcc_card_probe,
- .remove = zcrypt_pcixcc_card_remove,
- .ids = zcrypt_pcixcc_card_ids,
+static struct ap_driver zcrypt_cex2c_card_driver = {
+ .probe = zcrypt_cex2c_card_probe,
+ .remove = zcrypt_cex2c_card_remove,
+ .ids = zcrypt_cex2c_card_ids,
.flags = AP_DRIVER_FLAG_DEFAULT,
};
/**
- * Probe function for PCIXCC/CEX2C queue devices. It always accepts the
- * AP device since the bus_match already checked the hardware type. The
- * PCIXCC cards come in two flavours: micro code level 2 and micro code
- * level 3. This is checked by sending a test message to the device.
+ * Probe function for CEX2C/CEX3C queue devices. It always accepts the
+ * AP device since the bus_match already checked the hardware type.
* @ap_dev: pointer to the AP card device.
*/
-static int zcrypt_pcixcc_queue_probe(struct ap_device *ap_dev)
+static int zcrypt_cex2c_queue_probe(struct ap_device *ap_dev)
{
struct ap_queue *aq = to_ap_queue(&ap_dev->device);
struct zcrypt_queue *zq;
int rc;
- zq = zcrypt_queue_alloc(PCIXCC_MAX_XCRB_MESSAGE_SIZE);
+ zq = zcrypt_queue_alloc(CEX2C_MAX_XCRB_MESSAGE_SIZE);
if (!zq)
return -ENOMEM;
zq->queue = aq;
zq->online = 1;
atomic_set(&zq->load, 0);
- rc = zcrypt_pcixcc_rng_supported(aq);
+ rc = zcrypt_cex2c_rng_supported(aq);
if (rc < 0) {
zcrypt_queue_free(zq);
return rc;
zq->ops = zcrypt_msgtype(MSGTYPE06_NAME,
MSGTYPE06_VARIANT_NORNG);
ap_queue_init_reply(aq, &zq->reply);
- aq->request_timeout = PCIXCC_CLEANUP_TIME,
+ aq->request_timeout = CEX2C_CLEANUP_TIME;
aq->private = zq;
rc = zcrypt_queue_register(zq);
if (rc) {
}
/**
- * This is called to remove the PCIXCC/CEX2C queue driver information
+ * This is called to remove the CEX2C/CEX3C queue driver information
* if an AP queue device is removed.
*/
-static void zcrypt_pcixcc_queue_remove(struct ap_device *ap_dev)
+static void zcrypt_cex2c_queue_remove(struct ap_device *ap_dev)
{
struct ap_queue *aq = to_ap_queue(&ap_dev->device);
struct zcrypt_queue *zq = aq->private;
zcrypt_queue_unregister(zq);
}
-static struct ap_driver zcrypt_pcixcc_queue_driver = {
- .probe = zcrypt_pcixcc_queue_probe,
- .remove = zcrypt_pcixcc_queue_remove,
+static struct ap_driver zcrypt_cex2c_queue_driver = {
+ .probe = zcrypt_cex2c_queue_probe,
+ .remove = zcrypt_cex2c_queue_remove,
.suspend = ap_queue_suspend,
.resume = ap_queue_resume,
- .ids = zcrypt_pcixcc_queue_ids,
+ .ids = zcrypt_cex2c_queue_ids,
.flags = AP_DRIVER_FLAG_DEFAULT,
};
-int __init zcrypt_pcixcc_init(void)
+int __init zcrypt_cex2c_init(void)
{
int rc;
- rc = ap_driver_register(&zcrypt_pcixcc_card_driver,
- THIS_MODULE, "pcixcccard");
+ rc = ap_driver_register(&zcrypt_cex2c_card_driver,
+ THIS_MODULE, "cex2card");
if (rc)
return rc;
- rc = ap_driver_register(&zcrypt_pcixcc_queue_driver,
- THIS_MODULE, "pcixccqueue");
+ rc = ap_driver_register(&zcrypt_cex2c_queue_driver,
+ THIS_MODULE, "cex2cqueue");
if (rc)
- ap_driver_unregister(&zcrypt_pcixcc_card_driver);
+ ap_driver_unregister(&zcrypt_cex2c_card_driver);
return rc;
}
-void zcrypt_pcixcc_exit(void)
+void zcrypt_cex2c_exit(void)
{
- ap_driver_unregister(&zcrypt_pcixcc_queue_driver);
- ap_driver_unregister(&zcrypt_pcixcc_card_driver);
+ ap_driver_unregister(&zcrypt_cex2c_queue_driver);
+ ap_driver_unregister(&zcrypt_cex2c_card_driver);
}
-module_init(zcrypt_pcixcc_init);
-module_exit(zcrypt_pcixcc_exit);
+module_init(zcrypt_cex2c_init);
+module_exit(zcrypt_cex2c_exit);
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * zcrypt 2.1.0
- *
- * Copyright IBM Corp. 2001, 2012
+ * Copyright IBM Corp. 2001, 2018
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
*
* MSGTYPE restruct: Holger Dengler <hd@linux.vnet.ibm.com>
*/
-#ifndef _ZCRYPT_PCIXCC_H_
-#define _ZCRYPT_PCIXCC_H_
+#ifndef _ZCRYPT_CEX2C_H_
+#define _ZCRYPT_CEX2C_H_
-int zcrypt_pcixcc_init(void);
-void zcrypt_pcixcc_exit(void);
+int zcrypt_cex2c_init(void);
+void zcrypt_cex2c_exit(void);
-#endif /* _ZCRYPT_PCIXCC_H_ */
+#endif /* _ZCRYPT_CEX2C_H_ */
#define CEX4_CLEANUP_TIME (900*HZ)
MODULE_AUTHOR("IBM Corporation");
-MODULE_DESCRIPTION("CEX4 Cryptographic Card device driver, " \
- "Copyright IBM Corp. 2012");
+MODULE_DESCRIPTION("CEX4/CEX5/CEX6 Cryptographic Card device driver, " \
+ "Copyright IBM Corp. 2018");
MODULE_LICENSE("GPL");
static struct ap_device_id zcrypt_cex4_card_ids[] = {
MODULE_DEVICE_TABLE(ap, zcrypt_cex4_queue_ids);
/**
- * Probe function for CEX4 card device. It always accepts the AP device
- * since the bus_match already checked the hardware type.
+ * Probe function for CEX4/CEX5/CEX6 card device. It always
+ * accepts the AP device since the bus_match already checked
+ * the hardware type.
* @ap_dev: pointer to the AP device.
*/
static int zcrypt_cex4_card_probe(struct ap_device *ap_dev)
}
/**
- * This is called to remove the CEX4 card driver information
+ * This is called to remove the CEX4/CEX5/CEX6 card driver information
* if an AP card device is removed.
*/
static void zcrypt_cex4_card_remove(struct ap_device *ap_dev)
};
/**
- * Probe function for CEX4 queue device. It always accepts the AP device
- * since the bus_match already checked the hardware type.
+ * Probe function for CEX4/CEX5/CEX6 queue device. It always
+ * accepts the AP device since the bus_match already checked
+ * the hardware type.
* @ap_dev: pointer to the AP device.
*/
static int zcrypt_cex4_queue_probe(struct ap_device *ap_dev)
}
/**
- * This is called to remove the CEX4 queue driver information
- * if an AP queue device is removed.
+ * This is called to remove the CEX4/CEX5/CEX6 queue driver
+ * information if an AP queue device is removed.
*/
static void zcrypt_cex4_queue_remove(struct ap_device *ap_dev)
{
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * zcrypt 2.1.0
- *
* Copyright IBM Corp. 2001, 2006
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
#include <linux/atomic.h>
#include "zcrypt_debug.h"
#include "zcrypt_api.h"
+#include "zcrypt_msgtype6.h"
/**
* Reply Messages
card, queue, ehdr->reply_code);
return -EAGAIN;
case REP82_ERROR_TRANSPORT_FAIL:
+ /* Card or infrastructure failure, disable card */
+ atomic_set(&zcrypt_rescan_req, 1);
+ zq->online = 0;
+ pr_err("Cryptographic device %02x.%04x failed and was set offline\n",
+ card, queue);
+ /* For type 86 response show the apfs value (failure reason) */
+ if (ehdr->type == TYPE86_RSP_CODE) {
+ struct {
+ struct type86_hdr hdr;
+ struct type86_fmt2_ext fmt2;
+ } __packed * head = reply->message;
+ unsigned int apfs = *((u32 *)head->fmt2.apfs);
+
+ ZCRYPT_DBF(DBF_ERR,
+ "device=%02x.%04x reply=0x%02x apfs=0x%x => online=0 rc=EAGAIN\n",
+ card, queue, apfs, ehdr->reply_code);
+ } else
+ ZCRYPT_DBF(DBF_ERR,
+ "device=%02x.%04x reply=0x%02x => online=0 rc=EAGAIN\n",
+ card, queue, ehdr->reply_code);
+ return -EAGAIN;
case REP82_ERROR_MACHINE_FAILURE:
// REP88_ERROR_MODULE_FAILURE // '10' CEX2A
/* If a card fails disable it and repeat the request. */
// SPDX-License-Identifier: GPL-2.0+
/*
- * zcrypt 2.1.0
- *
* Copyright IBM Corp. 2001, 2012
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
#include "zcrypt_error.h"
#include "zcrypt_msgtype50.h"
-/* 4096 bits */
+/* >= CEX3A: 4096 bits */
#define CEX3A_MAX_MOD_SIZE 512
-/* max outputdatalength + type80_hdr */
+/* CEX2A: max outputdatalength + type80_hdr */
#define CEX2A_MAX_RESPONSE_SIZE 0x110
-/* 512 bit modulus, (max outputdatalength) + type80_hdr */
+/* >= CEX3A: 512 bit modulus, (max outputdatalength) + type80_hdr */
#define CEX3A_MAX_RESPONSE_SIZE 0x210
MODULE_AUTHOR("IBM Corporation");
MODULE_LICENSE("GPL");
/**
- * The type 50 message family is associated with a CEX2A card.
+ * The type 50 message family is associated with a CEXxA cards.
*
* The four members of the family are described below.
*
} __packed;
/**
- * The type 80 response family is associated with a CEX2A card.
+ * The type 80 response family is associated with a CEXxA cards.
*
* Note that all unsigned char arrays are right-justified and left-padded
* with zeroes.
/*
* CEX2A and CEX3A w/o FW update can handle requests up to
* 256 byte modulus (2k keys).
- * CEX3A with FW update and CEX4A cards are able to handle
+ * CEX3A with FW update and newer CEXxA cards are able to handle
* 512 byte modulus (4k keys).
*/
if (mod_len <= 128) { /* up to 1024 bit key size */
unsigned char *data;
if (t80h->len < sizeof(*t80h) + outputdatalength) {
- /* The result is too short, the CEX2A card may not do that.. */
+ /* The result is too short, the CEXxA card may not do that.. */
zq->online = 0;
pr_err("Cryptographic device %02x.%04x failed and was set offline\n",
AP_QID_CARD(zq->queue->qid),
static atomic_t zcrypt_step = ATOMIC_INIT(0);
/**
- * The request distributor calls this function if it picked the CEX2A
+ * The request distributor calls this function if it picked the CEXxA
* device to handle a modexpo request.
* @zq: pointer to zcrypt_queue structure that identifies the
- * CEX2A device to the request distributor
+ * CEXxA device to the request distributor
* @mex: pointer to the modexpo request buffer
*/
static long zcrypt_cex2a_modexpo(struct zcrypt_queue *zq,
}
/**
- * The request distributor calls this function if it picked the CEX2A
+ * The request distributor calls this function if it picked the CEXxA
* device to handle a modexpo_crt request.
* @zq: pointer to zcrypt_queue structure that identifies the
- * CEX2A device to the request distributor
+ * CEXxA device to the request distributor
* @crt: pointer to the modexpoc_crt request buffer
*/
static long zcrypt_cex2a_modexpo_crt(struct zcrypt_queue *zq,
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * zcrypt 2.1.0
- *
* Copyright IBM Corp. 2001, 2012
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
// SPDX-License-Identifier: GPL-2.0+
/*
- * zcrypt 2.1.0
- *
* Copyright IBM Corp. 2001, 2012
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
#include "zcrypt_msgtype6.h"
#include "zcrypt_cca_key.h"
-#define PCIXCC_MIN_MOD_SIZE_OLD 64 /* 512 bits */
-#define PCIXCC_MAX_ICA_RESPONSE_SIZE 0x77c /* max size type86 v2 reply */
+#define CEXXC_MAX_ICA_RESPONSE_SIZE 0x77c /* max size type86 v2 reply */
#define CEIL4(x) ((((x)+3)/4)*4)
struct completion work;
int type;
};
-#define PCIXCC_RESPONSE_TYPE_ICA 0
-#define PCIXCC_RESPONSE_TYPE_XCRB 1
-#define PCIXCC_RESPONSE_TYPE_EP11 2
+#define CEXXC_RESPONSE_TYPE_ICA 0
+#define CEXXC_RESPONSE_TYPE_XCRB 1
+#define CEXXC_RESPONSE_TYPE_EP11 2
MODULE_AUTHOR("IBM Corporation");
MODULE_DESCRIPTION("Cryptographic Coprocessor (message type 6), " \
} __packed;
/**
- * The following is used to initialize the CPRBX passed to the PCIXCC/CEX2C
+ * The following is used to initialize the CPRBX passed to the CEXxC/CEXxP
* card in a type6 message. The 3 fields that must be filled in at execution
* time are req_parml, rpl_parml and usage_domain.
* Everything about this interface is ascii/big-endian, since the
/* message header, cprbx and f&r */
msg->hdr = static_type6_hdrX;
msg->hdr.ToCardLen1 = size - sizeof(msg->hdr);
- msg->hdr.FromCardLen1 = PCIXCC_MAX_ICA_RESPONSE_SIZE - sizeof(msg->hdr);
+ msg->hdr.FromCardLen1 = CEXXC_MAX_ICA_RESPONSE_SIZE - sizeof(msg->hdr);
msg->cprbx = static_cprbx;
msg->cprbx.domain = AP_QID_QUEUE(zq->queue->qid);
/* message header, cprbx and f&r */
msg->hdr = static_type6_hdrX;
msg->hdr.ToCardLen1 = size - sizeof(msg->hdr);
- msg->hdr.FromCardLen1 = PCIXCC_MAX_ICA_RESPONSE_SIZE - sizeof(msg->hdr);
+ msg->hdr.FromCardLen1 = CEXXC_MAX_ICA_RESPONSE_SIZE - sizeof(msg->hdr);
msg->cprbx = static_cprbx;
msg->cprbx.domain = AP_QID_QUEUE(zq->queue->qid);
(int) service_rc, (int) service_rs);
return -EINVAL;
}
- if (service_rc == 8 && service_rs == 783) {
- zq->zcard->min_mod_size =
- PCIXCC_MIN_MOD_SIZE_OLD;
- ZCRYPT_DBF(DBF_DEBUG,
- "device=%02x.%04x rc/rs=%d/%d => rc=EAGAIN\n",
- AP_QID_CARD(zq->queue->qid),
- AP_QID_QUEUE(zq->queue->qid),
- (int) service_rc, (int) service_rs);
- return -EAGAIN;
- }
zq->online = 0;
pr_err("Cryptographic device %02x.%04x failed and was set offline\n",
AP_QID_CARD(zq->queue->qid),
if (pad_len > 0) {
if (pad_len < 10)
return -EINVAL;
- /* 'restore' padding left in the PCICC/PCIXCC card. */
+ /* 'restore' padding left in the CEXXC card. */
if (copy_to_user(outputdata, static_pad, pad_len - 1))
return -EFAULT;
if (put_user(0, outputdata + pad_len - 1))
if (t86r->hdr.type == TYPE86_RSP_CODE &&
t86r->cprbx.cprb_ver_id == 0x02) {
switch (resp_type->type) {
- case PCIXCC_RESPONSE_TYPE_ICA:
+ case CEXXC_RESPONSE_TYPE_ICA:
length = sizeof(struct type86x_reply)
+ t86r->length - 2;
- length = min(PCIXCC_MAX_ICA_RESPONSE_SIZE, length);
+ length = min(CEXXC_MAX_ICA_RESPONSE_SIZE, length);
memcpy(msg->message, reply->message, length);
break;
- case PCIXCC_RESPONSE_TYPE_XCRB:
+ case CEXXC_RESPONSE_TYPE_XCRB:
length = t86r->fmt2.offset2 + t86r->fmt2.count2;
length = min(MSGTYPE06_MAX_MSG_SIZE, length);
memcpy(msg->message, reply->message, length);
if (t86r->hdr.type == TYPE86_RSP_CODE &&
t86r->cprbx.cprb_ver_id == 0x04) {
switch (resp_type->type) {
- case PCIXCC_RESPONSE_TYPE_EP11:
+ case CEXXC_RESPONSE_TYPE_EP11:
length = t86r->fmt2.offset1 + t86r->fmt2.count1;
length = min(MSGTYPE06_MAX_MSG_SIZE, length);
memcpy(msg->message, reply->message, length);
static atomic_t zcrypt_step = ATOMIC_INIT(0);
/**
- * The request distributor calls this function if it picked the PCIXCC/CEX2C
+ * The request distributor calls this function if it picked the CEXxC
* device to handle a modexpo request.
* @zq: pointer to zcrypt_queue structure that identifies the
- * PCIXCC/CEX2C device to the request distributor
+ * CEXxC device to the request distributor
* @mex: pointer to the modexpo request buffer
*/
static long zcrypt_msgtype6_modexpo(struct zcrypt_queue *zq,
{
struct ap_message ap_msg;
struct response_type resp_type = {
- .type = PCIXCC_RESPONSE_TYPE_ICA,
+ .type = CEXXC_RESPONSE_TYPE_ICA,
};
int rc;
}
/**
- * The request distributor calls this function if it picked the PCIXCC/CEX2C
+ * The request distributor calls this function if it picked the CEXxC
* device to handle a modexpo_crt request.
* @zq: pointer to zcrypt_queue structure that identifies the
- * PCIXCC/CEX2C device to the request distributor
+ * CEXxC device to the request distributor
* @crt: pointer to the modexpoc_crt request buffer
*/
static long zcrypt_msgtype6_modexpo_crt(struct zcrypt_queue *zq,
{
struct ap_message ap_msg;
struct response_type resp_type = {
- .type = PCIXCC_RESPONSE_TYPE_ICA,
+ .type = CEXXC_RESPONSE_TYPE_ICA,
};
int rc;
unsigned int *func_code, unsigned short **dom)
{
struct response_type resp_type = {
- .type = PCIXCC_RESPONSE_TYPE_XCRB,
+ .type = CEXXC_RESPONSE_TYPE_XCRB,
};
ap_msg->message = kmalloc(MSGTYPE06_MAX_MSG_SIZE, GFP_KERNEL);
ap_msg->receive = zcrypt_msgtype6_receive;
ap_msg->psmid = (((unsigned long long) current->pid) << 32) +
atomic_inc_return(&zcrypt_step);
- ap_msg->private = kmalloc(sizeof(resp_type), GFP_KERNEL);
+ ap_msg->private = kmemdup(&resp_type, sizeof(resp_type), GFP_KERNEL);
if (!ap_msg->private)
return -ENOMEM;
- memcpy(ap_msg->private, &resp_type, sizeof(resp_type));
return XCRB_msg_to_type6CPRB_msgX(ap_msg, xcRB, func_code, dom);
}
/**
- * The request distributor calls this function if it picked the PCIXCC/CEX2C
+ * The request distributor calls this function if it picked the CEXxC
* device to handle a send_cprb request.
* @zq: pointer to zcrypt_queue structure that identifies the
- * PCIXCC/CEX2C device to the request distributor
+ * CEXxC device to the request distributor
* @xcRB: pointer to the send_cprb request buffer
*/
static long zcrypt_msgtype6_send_cprb(struct zcrypt_queue *zq,
unsigned int *func_code)
{
struct response_type resp_type = {
- .type = PCIXCC_RESPONSE_TYPE_EP11,
+ .type = CEXXC_RESPONSE_TYPE_EP11,
};
ap_msg->message = kmalloc(MSGTYPE06_MAX_MSG_SIZE, GFP_KERNEL);
ap_msg->receive = zcrypt_msgtype6_receive_ep11;
ap_msg->psmid = (((unsigned long long) current->pid) << 32) +
atomic_inc_return(&zcrypt_step);
- ap_msg->private = kmalloc(sizeof(resp_type), GFP_KERNEL);
+ ap_msg->private = kmemdup(&resp_type, sizeof(resp_type), GFP_KERNEL);
if (!ap_msg->private)
return -ENOMEM;
- memcpy(ap_msg->private, &resp_type, sizeof(resp_type));
return xcrb_msg_to_type6_ep11cprb_msgx(ap_msg, xcrb, func_code);
}
unsigned int *domain)
{
struct response_type resp_type = {
- .type = PCIXCC_RESPONSE_TYPE_XCRB,
+ .type = CEXXC_RESPONSE_TYPE_XCRB,
};
ap_msg->message = kmalloc(MSGTYPE06_MAX_MSG_SIZE, GFP_KERNEL);
ap_msg->receive = zcrypt_msgtype6_receive;
ap_msg->psmid = (((unsigned long long) current->pid) << 32) +
atomic_inc_return(&zcrypt_step);
- ap_msg->private = kmalloc(sizeof(resp_type), GFP_KERNEL);
+ ap_msg->private = kmemdup(&resp_type, sizeof(resp_type), GFP_KERNEL);
if (!ap_msg->private)
return -ENOMEM;
- memcpy(ap_msg->private, &resp_type, sizeof(resp_type));
rng_type6CPRB_msgX(ap_msg, ZCRYPT_RNG_BUFFER_SIZE, domain);
}
/**
- * The request distributor calls this function if it picked the PCIXCC/CEX2C
+ * The request distributor calls this function if it picked the CEXxC
* device to generate random data.
* @zq: pointer to zcrypt_queue structure that identifies the
- * PCIXCC/CEX2C device to the request distributor
+ * CEXxC device to the request distributor
* @buffer: pointer to a memory page to return random data
*/
static long zcrypt_msgtype6_rng(struct zcrypt_queue *zq,
}
/**
- * The crypto operations for a PCIXCC/CEX2C card.
+ * The crypto operations for a CEXxC card.
*/
static struct zcrypt_ops zcrypt_msgtype6_norng_ops = {
.owner = THIS_MODULE,
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * zcrypt 2.1.0
- *
* Copyright IBM Corp. 2001, 2012
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
#define MSGTYPE06_MAX_MSG_SIZE (12*1024)
/**
- * The type 6 message family is associated with PCICC or PCIXCC cards.
+ * The type 6 message family is associated with CEXxC/CEXxP cards.
*
* It contains a message header followed by a CPRB, both of which
* are described below.
unsigned int offset2; /* 0x00000000 */
unsigned int offset3; /* 0x00000000 */
unsigned int offset4; /* 0x00000000 */
- unsigned char agent_id[16]; /* PCICC: */
- /* 0x0100 */
- /* 0x4343412d4150504c202020 */
- /* 0x010101 */
- /* PCIXCC: */
- /* 0x4341000000000000 */
- /* 0x0000000000000000 */
+ unsigned char agent_id[16]; /* 0x4341000000000000 */
+ /* 0x0000000000000000 */
unsigned char rqid[2]; /* rqid. internal to 603 */
unsigned char reserved5[2]; /* 0x0000 */
unsigned char function_code[2]; /* for PKD, 0x5044 (ascii 'PD') */
} __packed;
/**
- * The type 86 message family is associated with PCICC and PCIXCC cards.
+ * The type 86 message family is associated with CEXxC/CEXxP cards.
*
* It contains a message header followed by a CPRB. The CPRB is
* the same as the request CPRB, which is described above.
// SPDX-License-Identifier: GPL-2.0+
/*
- * zcrypt 2.1.0
- *
* Copyright IBM Corp. 2001, 2012
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
static int __init openprom_init(void)
{
- struct device_node *dp;
int err;
err = misc_register(&openprom_dev);
if (err)
return err;
- dp = of_find_node_by_path("/");
- dp = dp->child;
- while (dp) {
- if (!strcmp(dp->name, "options"))
- break;
- dp = dp->sibling;
- }
- options_node = dp;
-
+ options_node = of_get_child_by_name(of_find_node_by_path("/"), "options");
if (!options_node) {
misc_deregister(&openprom_dev);
return -EIO;
alloc_error:
kfree(ctx->ccb_buf);
done:
- if (ctx != NULL)
- kfree(ctx);
+ kfree(ctx);
return -ENOMEM;
}
.id_table = aha152x_ids,
.resume = aha152x_resume,
};
-
-static int __init init_aha152x_cs(void)
-{
- return pcmcia_register_driver(&aha152x_cs_driver);
-}
-
-static void __exit exit_aha152x_cs(void)
-{
- pcmcia_unregister_driver(&aha152x_cs_driver);
-}
-
-module_init(init_aha152x_cs);
-module_exit(exit_aha152x_cs);
+module_pcmcia_driver(aha152x_cs_driver);
.suspend = nsp_cs_suspend,
.resume = nsp_cs_resume,
};
-
-static int __init nsp_cs_init(void)
-{
- return pcmcia_register_driver(&nsp_driver);
-}
-
-static void __exit nsp_cs_exit(void)
-{
- pcmcia_unregister_driver(&nsp_driver);
-}
-
-
-module_init(nsp_cs_init)
-module_exit(nsp_cs_exit)
+module_pcmcia_driver(nsp_driver);
/* end */
/* Interrupt handler */
//static irqreturn_t nspintr(int irq, void *dev_id);
-/* Module entry point*/
-static int __init nsp_cs_init(void);
-static void __exit nsp_cs_exit(void);
-
/* Debug */
#ifdef NSP_DEBUG
static void show_command (struct scsi_cmnd *SCpnt);
static int qlogic_resume(struct pcmcia_device *link)
{
scsi_info_t *info = link->priv;
+ int ret;
+
+ ret = pcmcia_enable_device(link);
+ if (ret)
+ return ret;
- pcmcia_enable_device(link);
if ((info->manf_id == MANFID_MACNICA) ||
(info->manf_id == MANFID_PIONEER) ||
(info->manf_id == 0x0098)) {
.resume = qlogic_resume,
};
-static int __init init_qlogic_cs(void)
-{
- return pcmcia_register_driver(&qlogic_cs_driver);
-}
-
-static void __exit exit_qlogic_cs(void)
-{
- pcmcia_unregister_driver(&qlogic_cs_driver);
-}
-
MODULE_AUTHOR("Tom Zerucha, Michael Griffith");
MODULE_DESCRIPTION("Driver for the PCMCIA Qlogic FAS SCSI controllers");
MODULE_LICENSE("GPL");
-module_init(init_qlogic_cs);
-module_exit(exit_qlogic_cs);
+module_pcmcia_driver(qlogic_cs_driver);
.id_table = sym53c500_ids,
.resume = sym53c500_resume,
};
-
-static int __init
-init_sym53c500_cs(void)
-{
- return pcmcia_register_driver(&sym53c500_cs_driver);
-}
-
-static void __exit
-exit_sym53c500_cs(void)
-{
- pcmcia_unregister_driver(&sym53c500_cs_driver);
-}
-
-module_init(init_sym53c500_cs);
-module_exit(exit_sym53c500_cs);
+module_pcmcia_driver(sym53c500_cs_driver);
*/
WARN_ON_ONCE(sdev->quiesced_by && sdev->quiesced_by != current);
- blk_set_preempt_only(q);
+ if (sdev->quiesced_by == current)
+ return 0;
+
+ blk_set_pm_only(q);
blk_mq_freeze_queue(q);
/*
- * Ensure that the effect of blk_set_preempt_only() will be visible
+ * Ensure that the effect of blk_set_pm_only() will be visible
* for percpu_ref_tryget() callers that occur after the queue
* unfreeze even if the queue was already frozen before this function
* was called. See also https://lwn.net/Articles/573497/.
if (err == 0)
sdev->quiesced_by = current;
else
- blk_clear_preempt_only(q);
+ blk_clear_pm_only(q);
mutex_unlock(&sdev->state_mutex);
return err;
mutex_lock(&sdev->state_mutex);
WARN_ON_ONCE(!sdev->quiesced_by);
sdev->quiesced_by = NULL;
- blk_clear_preempt_only(sdev->request_queue);
+ blk_clear_pm_only(sdev->request_queue);
if (sdev->sdev_state == SDEV_QUIESCE)
scsi_device_set_state(sdev, SDEV_RUNNING);
mutex_unlock(&sdev->state_mutex);
#include <linux/pm_runtime.h>
#include <linux/export.h>
#include <linux/async.h>
+#include <linux/blk-pm.h>
#include <scsi/scsi.h>
#include <scsi/scsi_device.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/blkpg.h>
+#include <linux/blk-pm.h>
#include <linux/delay.h>
#include <linux/mutex.h>
#include <linux/string_helpers.h>
}
blk_pm_runtime_init(sdp->request_queue, dev);
- device_add_disk(dev, gd);
+ device_add_disk(dev, gd, NULL);
if (sdkp->capacity)
sd_dif_config_host(sdkp);
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/blkdev.h>
+#include <linux/blk-pm.h>
#include <linux/mutex.h>
#include <linux/slab.h>
#include <linux/pm_runtime.h>
dev_set_drvdata(dev, cd);
disk->flags |= GENHD_FL_REMOVABLE;
- device_add_disk(&sdev->sdev_gendev, disk);
+ device_add_disk(&sdev->sdev_gendev, disk, NULL);
sdev_printk(KERN_DEBUG, sdev,
"Attached scsi CD-ROM %s\n", cd->cdi.name);
*/
static dma_addr_t fbpr_a;
static size_t fbpr_sz;
+static int __bman_probed;
static int bman_fbpr(struct reserved_mem *rmem)
{
return IRQ_HANDLED;
}
+int bman_is_probed(void)
+{
+ return __bman_probed;
+}
+EXPORT_SYMBOL_GPL(bman_is_probed);
+
static int fsl_bman_probe(struct platform_device *pdev)
{
int ret, err_irq;
u16 id, bm_pool_cnt;
u8 major, minor;
+ __bman_probed = -1;
+
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (!res) {
dev_err(dev, "Can't get %pOF property 'IORESOURCE_MEM'\n",
return ret;
}
+ __bman_probed = 1;
+
return 0;
};
static u32 __iomem *qm_ccsr_start;
/* A SDQCR mask comprising all the available/visible pool channels */
static u32 qm_pools_sdqcr;
+static int __qman_probed;
static inline u32 qm_ccsr_in(u32 offset)
{
return 0;
}
+int qman_is_probed(void)
+{
+ return __qman_probed;
+}
+EXPORT_SYMBOL_GPL(qman_is_probed);
+
static int fsl_qman_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
u16 id;
u8 major, minor;
+ __qman_probed = -1;
+
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (!res) {
dev_err(dev, "Can't get %pOF property 'IORESOURCE_MEM'\n",
if (ret)
return ret;
+ __qman_probed = 1;
+
return 0;
}
int irq, cpu, err;
u32 val;
+ err = qman_is_probed();
+ if (!err)
+ return -EPROBE_DEFER;
+ if (err < 0) {
+ dev_err(&pdev->dev, "failing probe due to qman probe error\n");
+ return -ENODEV;
+ }
+
pcfg = devm_kmalloc(dev, sizeof(*pcfg), GFP_KERNEL);
if (!pcfg)
return -ENOMEM;
*/
int geni_se_clk_tbl_get(struct geni_se *se, unsigned long **tbl)
{
- unsigned long freq = 0;
+ long freq = 0;
int i;
if (se->clk_perf_tbl) {
for (i = 0; i < MAX_CLK_PERF_LEVEL; i++) {
freq = clk_round_rate(se->clk, freq + 1);
- if (!freq || freq == se->clk_perf_tbl[i - 1])
+ if (freq <= 0 || freq == se->clk_perf_tbl[i - 1])
break;
se->clk_perf_tbl[i] = freq;
}
* @se: Pointer to the concerned serial engine.
* @req_freq: Requested clock frequency.
* @index: Index of the resultant frequency in the table.
- * @res_freq: Resultant frequency which matches or is closer to the
- * requested frequency.
+ * @res_freq: Resultant frequency of the source clock.
* @exact: Flag to indicate exact multiple requirement of the requested
* frequency.
*
- * This function is called by the protocol drivers to determine the matching
- * or exact multiple of the requested frequency, as provided by the serial
- * engine clock in order to meet the performance requirements. If there is
- * no matching or exact multiple of the requested frequency found, then it
- * selects the closest floor frequency, if exact flag is not set.
+ * This function is called by the protocol drivers to determine the best match
+ * of the requested frequency as provided by the serial engine clock in order
+ * to meet the performance requirements.
+ *
+ * If we return success:
+ * - if @exact is true then @res_freq / <an_integer> == @req_freq
+ * - if @exact is false then @res_freq / <an_integer> <= @req_freq
*
* Return: 0 on success, standard Linux error codes on failure.
*/
unsigned long *tbl;
int num_clk_levels;
int i;
+ unsigned long best_delta;
+ unsigned long new_delta;
+ unsigned int divider;
num_clk_levels = geni_se_clk_tbl_get(se, &tbl);
if (num_clk_levels < 0)
if (num_clk_levels == 0)
return -EINVAL;
- *res_freq = 0;
+ best_delta = ULONG_MAX;
for (i = 0; i < num_clk_levels; i++) {
- if (!(tbl[i] % req_freq)) {
+ divider = DIV_ROUND_UP(tbl[i], req_freq);
+ new_delta = req_freq - tbl[i] / divider;
+ if (new_delta < best_delta) {
+ /* We have a new best! */
*index = i;
*res_freq = tbl[i];
- return 0;
- }
- if (!(*res_freq) || ((tbl[i] > *res_freq) &&
- (tbl[i] < req_freq))) {
- *index = i;
- *res_freq = tbl[i];
+ /* If the new best is exact then we're done */
+ if (new_delta == 0)
+ return 0;
+
+ /* Record how close we got */
+ best_delta = new_delta;
}
}
config SPI_BCM63XX_HSSPI
tristate "Broadcom BCM63XX HS SPI controller driver"
- depends on BCM63XX || COMPILE_TEST
+ depends on BCM63XX || ARCH_BCM_63XX || COMPILE_TEST
help
This enables support for the High Speed SPI controller present on
newer Broadcom BCM63XX SoCs.
help
SPI driver for Renesas RSPI and QSPI blocks.
+config SPI_QCOM_QSPI
+ tristate "QTI QSPI controller"
+ depends on ARCH_QCOM
+ help
+ QSPI(Quad SPI) driver for Qualcomm QSPI controller.
+
config SPI_QUP
tristate "Qualcomm SPI controller with QUP interface"
depends on ARCH_QCOM || (ARM && COMPILE_TEST)
This driver can also be built as a module. If so, the module
will be called spi_qup.
+config SPI_QCOM_GENI
+ tristate "Qualcomm GENI based SPI controller"
+ depends on QCOM_GENI_SE
+ help
+ This driver supports GENI serial engine based SPI controller in
+ master mode on the Qualcomm Technologies Inc.'s SoCs. If you say
+ yes to this option, support will be included for the built-in SPI
+ interface on the Qualcomm Technologies Inc.'s SoCs.
+
+ This driver can also be built as a module. If so, the module
+ will be called spi-geni-qcom.
+
config SPI_S3C24XX
tristate "Samsung S3C24XX series SPI"
depends on ARCH_S3C24XX
help
SPI driver for CSR SiRFprimaII SoCs
+config SPI_SLAVE_MT27XX
+ tristate "MediaTek SPI slave device"
+ depends on ARCH_MEDIATEK || COMPILE_TEST
+ depends on SPI_SLAVE
+ help
+ This selects the MediaTek(R) SPI slave device driver.
+ If you want to use MediaTek(R) SPI slave interface,
+ say Y or M here.If you are not sure, say N.
+ SPI slave drivers for Mediatek MT27XX series ARM SoCs.
+
+config SPI_SPRD
+ tristate "Spreadtrum SPI controller"
+ depends on ARCH_SPRD || COMPILE_TEST
+ help
+ SPI driver for Spreadtrum SoCs.
+
config SPI_SPRD_ADI
tristate "Spreadtrum ADI controller"
depends on ARCH_SPRD || COMPILE_TEST
is not available, the driver automatically falls back to
PIO mode.
+config SPI_STM32_QSPI
+ tristate "STMicroelectronics STM32 QUAD SPI controller"
+ depends on ARCH_STM32 || COMPILE_TEST
+ depends on OF
+ help
+ This enables support for the Quad SPI controller in master mode.
+ This driver does not support generic SPI. The implementation only
+ supports spi-mem interface.
+
config SPI_ST_SSC4
tristate "STMicroelectronics SPI SSC-based driver"
depends on ARCH_STI || COMPILE_TEST
spi-pxa2xx-platform-objs := spi-pxa2xx.o spi-pxa2xx-dma.o
obj-$(CONFIG_SPI_PXA2XX) += spi-pxa2xx-platform.o
obj-$(CONFIG_SPI_PXA2XX_PCI) += spi-pxa2xx-pci.o
+obj-$(CONFIG_SPI_QCOM_GENI) += spi-geni-qcom.o
+obj-$(CONFIG_SPI_QCOM_QSPI) += spi-qcom-qspi.o
obj-$(CONFIG_SPI_QUP) += spi-qup.o
obj-$(CONFIG_SPI_ROCKCHIP) += spi-rockchip.o
obj-$(CONFIG_SPI_RB4XX) += spi-rb4xx.o
obj-$(CONFIG_SPI_SH_MSIOF) += spi-sh-msiof.o
obj-$(CONFIG_SPI_SH_SCI) += spi-sh-sci.o
obj-$(CONFIG_SPI_SIRF) += spi-sirf.o
+obj-$(CONFIG_SPI_SLAVE_MT27XX) += spi-slave-mt27xx.o
+obj-$(CONFIG_SPI_SPRD) += spi-sprd.o
obj-$(CONFIG_SPI_SPRD_ADI) += spi-sprd-adi.o
obj-$(CONFIG_SPI_STM32) += spi-stm32.o
+obj-$(CONFIG_SPI_STM32_QSPI) += spi-stm32-qspi.o
obj-$(CONFIG_SPI_ST_SSC4) += spi-st-ssc4.o
obj-$(CONFIG_SPI_SUN4I) += spi-sun4i.o
obj-$(CONFIG_SPI_SUN6I) += spi-sun6i.o
/* Stop the queue running */
ret = spi_master_suspend(master);
- if (ret) {
- dev_warn(dev, "cannot suspend master\n");
+ if (ret)
return ret;
- }
if (!pm_runtime_suspended(dev))
atmel_spi_runtime_suspend(dev);
}
/* Start the queue running */
- ret = spi_master_resume(master);
- if (ret)
- dev_err(dev, "problem starting queue (%d)\n", ret);
-
- return ret;
+ return spi_master_resume(master);
}
#endif
#define BSPI_BPP_MODE_SELECT_MASK BIT(8)
#define BSPI_BPP_ADDR_SELECT_MASK BIT(16)
-#define BSPI_READ_LENGTH 512
+#define BSPI_READ_LENGTH 256
/* MSPI register offsets */
#define MSPI_SPCR0_LSB 0x000
int bpc = 0, bpp = 0;
u8 command = op->cmd.opcode;
int width = op->cmd.buswidth ? op->cmd.buswidth : SPI_NBITS_SINGLE;
- int addrlen = op->addr.nbytes * 8;
+ int addrlen = op->addr.nbytes;
int flex_mode = 1;
dev_dbg(&qspi->pdev->dev, "set flex mode w %x addrlen %x hp %d\n",
struct platform_device *pdev;
struct clk *clk;
+ struct clk *pll_clk;
void __iomem *regs;
u8 __iomem *fifo;
struct resource *res_mem;
void __iomem *regs;
struct device *dev = &pdev->dev;
- struct clk *clk;
+ struct clk *clk, *pll_clk = NULL;
int irq, ret;
u32 reg, rate, num_cs = HSSPI_SPI_MAX_CS;
rate = clk_get_rate(clk);
if (!rate) {
- struct clk *pll_clk = devm_clk_get(dev, "pll");
+ pll_clk = devm_clk_get(dev, "pll");
if (IS_ERR(pll_clk)) {
ret = PTR_ERR(pll_clk);
clk_disable_unprepare(pll_clk);
if (!rate) {
ret = -EINVAL;
- goto out_disable_clk;
+ goto out_disable_pll_clk;
}
}
master = spi_alloc_master(&pdev->dev, sizeof(*bs));
if (!master) {
ret = -ENOMEM;
- goto out_disable_clk;
+ goto out_disable_pll_clk;
}
bs = spi_master_get_devdata(master);
bs->pdev = pdev;
bs->clk = clk;
+ bs->pll_clk = pll_clk;
bs->regs = regs;
bs->speed_hz = rate;
bs->fifo = (u8 __iomem *)(bs->regs + HSSPI_FIFO_REG(0));
out_put_master:
spi_master_put(master);
+out_disable_pll_clk:
+ clk_disable_unprepare(pll_clk);
out_disable_clk:
clk_disable_unprepare(clk);
return ret;
/* reset the hardware and block queue progress */
__raw_writel(0, bs->regs + HSSPI_INT_MASK_REG);
+ clk_disable_unprepare(bs->pll_clk);
clk_disable_unprepare(bs->clk);
return 0;
struct bcm63xx_hsspi *bs = spi_master_get_devdata(master);
spi_master_suspend(master);
+ clk_disable_unprepare(bs->pll_clk);
clk_disable_unprepare(bs->clk);
return 0;
if (ret)
return ret;
+ if (bs->pll_clk) {
+ ret = clk_prepare_enable(bs->pll_clk);
+ if (ret)
+ return ret;
+ }
+
spi_master_resume(master);
return 0;
static void davinci_spi_chipselect(struct spi_device *spi, int value)
{
struct davinci_spi *dspi;
- struct davinci_spi_platform_data *pdata;
struct davinci_spi_config *spicfg = spi->controller_data;
u8 chip_sel = spi->chip_select;
u16 spidat1 = CS_DEFAULT;
dspi = spi_master_get_devdata(spi->master);
- pdata = &dspi->pdata;
/* program delay transfers if tx_delay is non zero */
if (spicfg && spicfg->wdelay)
!(spi->mode & SPI_CS_HIGH));
} else {
if (value == BITBANG_CS_ACTIVE) {
- spidat1 |= SPIDAT1_CSHOLD_MASK;
+ if (!(spi->mode & SPI_CS_WORD))
+ spidat1 |= SPIDAT1_CSHOLD_MASK;
spidat1 &= ~(0x1 << chip_sel);
}
}
{
int retval = 0;
struct davinci_spi *dspi;
- struct davinci_spi_platform_data *pdata;
struct spi_master *master = spi->master;
struct device_node *np = spi->dev.of_node;
bool internal_cs = true;
dspi = spi_master_get_devdata(spi->master);
- pdata = &dspi->pdata;
if (!(spi->mode & SPI_NO_CS)) {
if (np && (master->cs_gpios != NULL) && (spi->cs_gpio >= 0)) {
retval = gpio_direction_output(
spi->cs_gpio, !(spi->mode & SPI_CS_HIGH));
internal_cs = false;
- } else if (pdata->chip_sel &&
- spi->chip_select < pdata->num_chipselect &&
- pdata->chip_sel[spi->chip_select] != SPI_INTERN_CS) {
- spi->cs_gpio = pdata->chip_sel[spi->chip_select];
- retval = gpio_direction_output(
- spi->cs_gpio, !(spi->mode & SPI_CS_HIGH));
- internal_cs = false;
}
if (retval) {
return retval;
}
- if (internal_cs)
+ if (internal_cs) {
set_io_bits(dspi->base + SPIPC0, 1 << spi->chip_select);
+ }
}
if (spi->mode & SPI_READY)
dspi->prescaler_limit = pdata->prescaler_limit;
dspi->version = pdata->version;
- dspi->bitbang.flags = SPI_NO_CS | SPI_LSB_FIRST | SPI_LOOP;
+ dspi->bitbang.flags = SPI_NO_CS | SPI_LSB_FIRST | SPI_LOOP | SPI_CS_WORD;
if (dspi->version == SPI_VERSION_2)
dspi->bitbang.flags |= SPI_READY;
};
#define MSCC_CPU_SYSTEM_CTRL_GENERAL_CTRL 0x24
-#define OCELOT_IF_SI_OWNER_MASK GENMASK(5, 4)
#define OCELOT_IF_SI_OWNER_OFFSET 4
+#define JAGUAR2_IF_SI_OWNER_OFFSET 6
+#define MSCC_IF_SI_OWNER_MASK GENMASK(1, 0)
#define MSCC_IF_SI_OWNER_SISL 0
#define MSCC_IF_SI_OWNER_SIBM 1
#define MSCC_IF_SI_OWNER_SIMC 2
}
static int dw_spi_mscc_init(struct platform_device *pdev,
- struct dw_spi_mmio *dwsmmio)
+ struct dw_spi_mmio *dwsmmio,
+ const char *cpu_syscon, u32 if_si_owner_offset)
{
struct dw_spi_mscc *dwsmscc;
struct resource *res;
return PTR_ERR(dwsmscc->spi_mst);
}
- dwsmscc->syscon = syscon_regmap_lookup_by_compatible("mscc,ocelot-cpu-syscon");
+ dwsmscc->syscon = syscon_regmap_lookup_by_compatible(cpu_syscon);
if (IS_ERR(dwsmscc->syscon))
return PTR_ERR(dwsmscc->syscon);
/* Select the owner of the SI interface */
regmap_update_bits(dwsmscc->syscon, MSCC_CPU_SYSTEM_CTRL_GENERAL_CTRL,
- OCELOT_IF_SI_OWNER_MASK,
- MSCC_IF_SI_OWNER_SIMC << OCELOT_IF_SI_OWNER_OFFSET);
+ MSCC_IF_SI_OWNER_MASK << if_si_owner_offset,
+ MSCC_IF_SI_OWNER_SIMC << if_si_owner_offset);
dwsmmio->dws.set_cs = dw_spi_mscc_set_cs;
dwsmmio->priv = dwsmscc;
return 0;
}
+static int dw_spi_mscc_ocelot_init(struct platform_device *pdev,
+ struct dw_spi_mmio *dwsmmio)
+{
+ return dw_spi_mscc_init(pdev, dwsmmio, "mscc,ocelot-cpu-syscon",
+ OCELOT_IF_SI_OWNER_OFFSET);
+}
+
+static int dw_spi_mscc_jaguar2_init(struct platform_device *pdev,
+ struct dw_spi_mmio *dwsmmio)
+{
+ return dw_spi_mscc_init(pdev, dwsmmio, "mscc,jaguar2-cpu-syscon",
+ JAGUAR2_IF_SI_OWNER_OFFSET);
+}
+
+static int dw_spi_alpine_init(struct platform_device *pdev,
+ struct dw_spi_mmio *dwsmmio)
+{
+ dwsmmio->dws.cs_override = 1;
+
+ return 0;
+}
+
static int dw_spi_mmio_probe(struct platform_device *pdev)
{
int (*init_func)(struct platform_device *pdev,
static const struct of_device_id dw_spi_mmio_of_match[] = {
{ .compatible = "snps,dw-apb-ssi", },
- { .compatible = "mscc,ocelot-spi", .data = dw_spi_mscc_init},
+ { .compatible = "mscc,ocelot-spi", .data = dw_spi_mscc_ocelot_init},
+ { .compatible = "mscc,jaguar2-spi", .data = dw_spi_mscc_jaguar2_init},
+ { .compatible = "amazon,alpine-dw-apb-ssi", .data = dw_spi_alpine_init},
{ /* end of table */}
};
MODULE_DEVICE_TABLE(of, dw_spi_mmio_of_match);
if (!enable)
dw_writel(dws, DW_SPI_SER, BIT(spi->chip_select));
+ else if (dws->cs_override)
+ dw_writel(dws, DW_SPI_SER, 0);
}
EXPORT_SYMBOL_GPL(dw_spi_set_cs);
dws->current_freq = transfer->speed_hz;
spi_set_clk(dws, chip->clk_div);
}
- if (transfer->bits_per_word == 8) {
- dws->n_bytes = 1;
- dws->dma_width = 1;
- } else if (transfer->bits_per_word == 16) {
- dws->n_bytes = 2;
- dws->dma_width = 2;
- } else {
- return -EINVAL;
- }
+
+ dws->n_bytes = DIV_ROUND_UP(transfer->bits_per_word, BITS_PER_BYTE);
+ dws->dma_width = DIV_ROUND_UP(transfer->bits_per_word, BITS_PER_BYTE);
+
/* Default SPI mode is SCPOL = 0, SCPH = 0 */
cr0 = (transfer->bits_per_word - 1)
| (chip->type << SPI_FRF_OFFSET)
dws->fifo_len = (fifo == 1) ? 0 : fifo;
dev_dbg(dev, "Detected FIFO size: %u bytes\n", dws->fifo_len);
}
+
+ /* enable HW fixup for explicit CS deselect for Amazon's alpine chip */
+ if (dws->cs_override)
+ dw_writel(dws, DW_SPI_CS_OVERRIDE, 0xF);
}
int dw_spi_add_host(struct device *dev, struct dw_spi *dws)
}
master->mode_bits = SPI_CPOL | SPI_CPHA | SPI_LOOP;
- master->bits_per_word_mask = SPI_BPW_MASK(8) | SPI_BPW_MASK(16);
+ master->bits_per_word_mask = SPI_BPW_RANGE_MASK(4, 16);
master->bus_num = dws->bus_num;
master->num_chipselect = dws->num_cs;
master->setup = dw_spi_setup;
int dw_spi_resume_host(struct dw_spi *dws)
{
- int ret;
-
spi_hw_init(&dws->master->dev, dws);
- ret = spi_controller_resume(dws->master);
- if (ret)
- dev_err(&dws->master->dev, "fail to start queue (%d)\n", ret);
- return ret;
+ return spi_controller_resume(dws->master);
}
EXPORT_SYMBOL_GPL(dw_spi_resume_host);
#define DW_SPI_IDR 0x58
#define DW_SPI_VERSION 0x5c
#define DW_SPI_DR 0x60
+#define DW_SPI_CS_OVERRIDE 0xf4
/* Bit fields in CTRLR0 */
#define SPI_DFS_OFFSET 0
u32 fifo_len; /* depth of the FIFO buffer */
u32 max_freq; /* max bus freq supported */
+ int cs_override;
u32 reg_io_width; /* DR I/O width in bytes */
u16 bus_num;
u16 num_cs; /* supported slave numbers */
return -EINPROGRESS;
}
+static enum dma_transfer_direction
+ep93xx_dma_data_to_trans_dir(enum dma_data_direction dir)
+{
+ switch (dir) {
+ case DMA_TO_DEVICE:
+ return DMA_MEM_TO_DEV;
+ case DMA_FROM_DEVICE:
+ return DMA_DEV_TO_MEM;
+ default:
+ return DMA_TRANS_NONE;
+ }
+}
+
/**
* ep93xx_spi_dma_prepare() - prepares a DMA transfer
* @master: SPI master
*/
static struct dma_async_tx_descriptor *
ep93xx_spi_dma_prepare(struct spi_master *master,
- enum dma_transfer_direction dir)
+ enum dma_data_direction dir)
{
struct ep93xx_spi *espi = spi_master_get_devdata(master);
struct spi_transfer *xfer = master->cur_msg->state;
buswidth = DMA_SLAVE_BUSWIDTH_1_BYTE;
memset(&conf, 0, sizeof(conf));
- conf.direction = dir;
+ conf.direction = ep93xx_dma_data_to_trans_dir(dir);
- if (dir == DMA_DEV_TO_MEM) {
+ if (dir == DMA_FROM_DEVICE) {
chan = espi->dma_rx;
buf = xfer->rx_buf;
sgt = &espi->rx_sgt;
if (!nents)
return ERR_PTR(-ENOMEM);
- txd = dmaengine_prep_slave_sg(chan, sgt->sgl, nents, dir, DMA_CTRL_ACK);
+ txd = dmaengine_prep_slave_sg(chan, sgt->sgl, nents, conf.direction,
+ DMA_CTRL_ACK);
if (!txd) {
dma_unmap_sg(chan->device->dev, sgt->sgl, sgt->nents, dir);
return ERR_PTR(-ENOMEM);
* unmapped.
*/
static void ep93xx_spi_dma_finish(struct spi_master *master,
- enum dma_transfer_direction dir)
+ enum dma_data_direction dir)
{
struct ep93xx_spi *espi = spi_master_get_devdata(master);
struct dma_chan *chan;
struct sg_table *sgt;
- if (dir == DMA_DEV_TO_MEM) {
+ if (dir == DMA_FROM_DEVICE) {
chan = espi->dma_rx;
sgt = &espi->rx_sgt;
} else {
{
struct spi_master *master = callback_param;
- ep93xx_spi_dma_finish(master, DMA_MEM_TO_DEV);
- ep93xx_spi_dma_finish(master, DMA_DEV_TO_MEM);
+ ep93xx_spi_dma_finish(master, DMA_TO_DEVICE);
+ ep93xx_spi_dma_finish(master, DMA_FROM_DEVICE);
spi_finalize_current_transfer(master);
}
struct ep93xx_spi *espi = spi_master_get_devdata(master);
struct dma_async_tx_descriptor *rxd, *txd;
- rxd = ep93xx_spi_dma_prepare(master, DMA_DEV_TO_MEM);
+ rxd = ep93xx_spi_dma_prepare(master, DMA_FROM_DEVICE);
if (IS_ERR(rxd)) {
dev_err(&master->dev, "DMA RX failed: %ld\n", PTR_ERR(rxd));
return PTR_ERR(rxd);
}
- txd = ep93xx_spi_dma_prepare(master, DMA_MEM_TO_DEV);
+ txd = ep93xx_spi_dma_prepare(master, DMA_TO_DEVICE);
if (IS_ERR(txd)) {
- ep93xx_spi_dma_finish(master, DMA_DEV_TO_MEM);
+ ep93xx_spi_dma_finish(master, DMA_FROM_DEVICE);
dev_err(&master->dev, "DMA TX failed: %ld\n", PTR_ERR(txd));
return PTR_ERR(txd);
}
int ret;
ret = spi_master_suspend(master);
- if (ret) {
- dev_warn(dev, "cannot suspend master\n");
+ if (ret)
return ret;
- }
return pm_runtime_force_suspend(dev);
}
fsl_lpspi_set_watermark(fsl_lpspi);
- temp = CFGR1_PCSCFG | CFGR1_MASTER | CFGR1_NOSTALL;
+ temp = CFGR1_PCSCFG | CFGR1_MASTER;
if (fsl_lpspi->config.mode & SPI_CS_HIGH)
temp |= CFGR1_PCSPOL;
writel(temp, fsl_lpspi->base + IMX7ULP_CFGR1);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2017-2018, The Linux foundation. All rights reserved.
+
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/log2.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/qcom-geni-se.h>
+#include <linux/spi/spi.h>
+#include <linux/spinlock.h>
+
+/* SPI SE specific registers and respective register fields */
+#define SE_SPI_CPHA 0x224
+#define CPHA BIT(0)
+
+#define SE_SPI_LOOPBACK 0x22c
+#define LOOPBACK_ENABLE 0x1
+#define NORMAL_MODE 0x0
+#define LOOPBACK_MSK GENMASK(1, 0)
+
+#define SE_SPI_CPOL 0x230
+#define CPOL BIT(2)
+
+#define SE_SPI_DEMUX_OUTPUT_INV 0x24c
+#define CS_DEMUX_OUTPUT_INV_MSK GENMASK(3, 0)
+
+#define SE_SPI_DEMUX_SEL 0x250
+#define CS_DEMUX_OUTPUT_SEL GENMASK(3, 0)
+
+#define SE_SPI_TRANS_CFG 0x25c
+#define CS_TOGGLE BIT(0)
+
+#define SE_SPI_WORD_LEN 0x268
+#define WORD_LEN_MSK GENMASK(9, 0)
+#define MIN_WORD_LEN 4
+
+#define SE_SPI_TX_TRANS_LEN 0x26c
+#define SE_SPI_RX_TRANS_LEN 0x270
+#define TRANS_LEN_MSK GENMASK(23, 0)
+
+#define SE_SPI_PRE_POST_CMD_DLY 0x274
+
+#define SE_SPI_DELAY_COUNTERS 0x278
+#define SPI_INTER_WORDS_DELAY_MSK GENMASK(9, 0)
+#define SPI_CS_CLK_DELAY_MSK GENMASK(19, 10)
+#define SPI_CS_CLK_DELAY_SHFT 10
+
+/* M_CMD OP codes for SPI */
+#define SPI_TX_ONLY 1
+#define SPI_RX_ONLY 2
+#define SPI_FULL_DUPLEX 3
+#define SPI_TX_RX 7
+#define SPI_CS_ASSERT 8
+#define SPI_CS_DEASSERT 9
+#define SPI_SCK_ONLY 10
+/* M_CMD params for SPI */
+#define SPI_PRE_CMD_DELAY BIT(0)
+#define TIMESTAMP_BEFORE BIT(1)
+#define FRAGMENTATION BIT(2)
+#define TIMESTAMP_AFTER BIT(3)
+#define POST_CMD_DELAY BIT(4)
+
+/* SPI M_COMMAND OPCODE */
+enum spi_mcmd_code {
+ CMD_NONE,
+ CMD_XFER,
+ CMD_CS,
+ CMD_CANCEL,
+};
+
+
+struct spi_geni_master {
+ struct geni_se se;
+ struct device *dev;
+ u32 tx_fifo_depth;
+ u32 fifo_width_bits;
+ u32 tx_wm;
+ unsigned long cur_speed_hz;
+ unsigned int cur_bits_per_word;
+ unsigned int tx_rem_bytes;
+ unsigned int rx_rem_bytes;
+ const struct spi_transfer *cur_xfer;
+ struct completion xfer_done;
+ unsigned int oversampling;
+ spinlock_t lock;
+ unsigned int cur_mcmd;
+ int irq;
+};
+
+static void handle_fifo_timeout(struct spi_master *spi,
+ struct spi_message *msg);
+
+static int get_spi_clk_cfg(unsigned int speed_hz,
+ struct spi_geni_master *mas,
+ unsigned int *clk_idx,
+ unsigned int *clk_div)
+{
+ unsigned long sclk_freq;
+ unsigned int actual_hz;
+ struct geni_se *se = &mas->se;
+ int ret;
+
+ ret = geni_se_clk_freq_match(&mas->se,
+ speed_hz * mas->oversampling,
+ clk_idx, &sclk_freq, false);
+ if (ret) {
+ dev_err(mas->dev, "Failed(%d) to find src clk for %dHz\n",
+ ret, speed_hz);
+ return ret;
+ }
+
+ *clk_div = DIV_ROUND_UP(sclk_freq, mas->oversampling * speed_hz);
+ actual_hz = sclk_freq / (mas->oversampling * *clk_div);
+
+ dev_dbg(mas->dev, "req %u=>%u sclk %lu, idx %d, div %d\n", speed_hz,
+ actual_hz, sclk_freq, *clk_idx, *clk_div);
+ ret = clk_set_rate(se->clk, sclk_freq);
+ if (ret)
+ dev_err(mas->dev, "clk_set_rate failed %d\n", ret);
+ return ret;
+}
+
+static void spi_geni_set_cs(struct spi_device *slv, bool set_flag)
+{
+ struct spi_geni_master *mas = spi_master_get_devdata(slv->master);
+ struct spi_master *spi = dev_get_drvdata(mas->dev);
+ struct geni_se *se = &mas->se;
+ unsigned long timeout;
+
+ reinit_completion(&mas->xfer_done);
+ pm_runtime_get_sync(mas->dev);
+ if (!(slv->mode & SPI_CS_HIGH))
+ set_flag = !set_flag;
+
+ mas->cur_mcmd = CMD_CS;
+ if (set_flag)
+ geni_se_setup_m_cmd(se, SPI_CS_ASSERT, 0);
+ else
+ geni_se_setup_m_cmd(se, SPI_CS_DEASSERT, 0);
+
+ timeout = wait_for_completion_timeout(&mas->xfer_done, HZ);
+ if (!timeout)
+ handle_fifo_timeout(spi, NULL);
+
+ pm_runtime_put(mas->dev);
+}
+
+static void spi_setup_word_len(struct spi_geni_master *mas, u16 mode,
+ unsigned int bits_per_word)
+{
+ unsigned int pack_words;
+ bool msb_first = (mode & SPI_LSB_FIRST) ? false : true;
+ struct geni_se *se = &mas->se;
+ u32 word_len;
+
+ word_len = readl(se->base + SE_SPI_WORD_LEN);
+
+ /*
+ * If bits_per_word isn't a byte aligned value, set the packing to be
+ * 1 SPI word per FIFO word.
+ */
+ if (!(mas->fifo_width_bits % bits_per_word))
+ pack_words = mas->fifo_width_bits / bits_per_word;
+ else
+ pack_words = 1;
+ word_len &= ~WORD_LEN_MSK;
+ word_len |= ((bits_per_word - MIN_WORD_LEN) & WORD_LEN_MSK);
+ geni_se_config_packing(&mas->se, bits_per_word, pack_words, msb_first,
+ true, true);
+ writel(word_len, se->base + SE_SPI_WORD_LEN);
+}
+
+static int setup_fifo_params(struct spi_device *spi_slv,
+ struct spi_master *spi)
+{
+ struct spi_geni_master *mas = spi_master_get_devdata(spi);
+ struct geni_se *se = &mas->se;
+ u32 loopback_cfg, cpol, cpha, demux_output_inv;
+ u32 demux_sel, clk_sel, m_clk_cfg, idx, div;
+ int ret;
+
+ loopback_cfg = readl(se->base + SE_SPI_LOOPBACK);
+ cpol = readl(se->base + SE_SPI_CPOL);
+ cpha = readl(se->base + SE_SPI_CPHA);
+ demux_output_inv = 0;
+ loopback_cfg &= ~LOOPBACK_MSK;
+ cpol &= ~CPOL;
+ cpha &= ~CPHA;
+
+ if (spi_slv->mode & SPI_LOOP)
+ loopback_cfg |= LOOPBACK_ENABLE;
+
+ if (spi_slv->mode & SPI_CPOL)
+ cpol |= CPOL;
+
+ if (spi_slv->mode & SPI_CPHA)
+ cpha |= CPHA;
+
+ if (spi_slv->mode & SPI_CS_HIGH)
+ demux_output_inv = BIT(spi_slv->chip_select);
+
+ demux_sel = spi_slv->chip_select;
+ mas->cur_speed_hz = spi_slv->max_speed_hz;
+ mas->cur_bits_per_word = spi_slv->bits_per_word;
+
+ ret = get_spi_clk_cfg(mas->cur_speed_hz, mas, &idx, &div);
+ if (ret) {
+ dev_err(mas->dev, "Err setting clks ret(%d) for %ld\n",
+ ret, mas->cur_speed_hz);
+ return ret;
+ }
+
+ clk_sel = idx & CLK_SEL_MSK;
+ m_clk_cfg = (div << CLK_DIV_SHFT) | SER_CLK_EN;
+ spi_setup_word_len(mas, spi_slv->mode, spi_slv->bits_per_word);
+ writel(loopback_cfg, se->base + SE_SPI_LOOPBACK);
+ writel(demux_sel, se->base + SE_SPI_DEMUX_SEL);
+ writel(cpha, se->base + SE_SPI_CPHA);
+ writel(cpol, se->base + SE_SPI_CPOL);
+ writel(demux_output_inv, se->base + SE_SPI_DEMUX_OUTPUT_INV);
+ writel(clk_sel, se->base + SE_GENI_CLK_SEL);
+ writel(m_clk_cfg, se->base + GENI_SER_M_CLK_CFG);
+ return 0;
+}
+
+static int spi_geni_prepare_message(struct spi_master *spi,
+ struct spi_message *spi_msg)
+{
+ int ret;
+ struct spi_geni_master *mas = spi_master_get_devdata(spi);
+ struct geni_se *se = &mas->se;
+
+ geni_se_select_mode(se, GENI_SE_FIFO);
+ reinit_completion(&mas->xfer_done);
+ ret = setup_fifo_params(spi_msg->spi, spi);
+ if (ret)
+ dev_err(mas->dev, "Couldn't select mode %d\n", ret);
+ return ret;
+}
+
+static int spi_geni_init(struct spi_geni_master *mas)
+{
+ struct geni_se *se = &mas->se;
+ unsigned int proto, major, minor, ver;
+
+ pm_runtime_get_sync(mas->dev);
+
+ proto = geni_se_read_proto(se);
+ if (proto != GENI_SE_SPI) {
+ dev_err(mas->dev, "Invalid proto %d\n", proto);
+ pm_runtime_put(mas->dev);
+ return -ENXIO;
+ }
+ mas->tx_fifo_depth = geni_se_get_tx_fifo_depth(se);
+
+ /* Width of Tx and Rx FIFO is same */
+ mas->fifo_width_bits = geni_se_get_tx_fifo_width(se);
+
+ /*
+ * Hardware programming guide suggests to configure
+ * RX FIFO RFR level to fifo_depth-2.
+ */
+ geni_se_init(se, 0x0, mas->tx_fifo_depth - 2);
+ /* Transmit an entire FIFO worth of data per IRQ */
+ mas->tx_wm = 1;
+ ver = geni_se_get_qup_hw_version(se);
+ major = GENI_SE_VERSION_MAJOR(ver);
+ minor = GENI_SE_VERSION_MINOR(ver);
+
+ if (major == 1 && minor == 0)
+ mas->oversampling = 2;
+ else
+ mas->oversampling = 1;
+
+ pm_runtime_put(mas->dev);
+ return 0;
+}
+
+static void setup_fifo_xfer(struct spi_transfer *xfer,
+ struct spi_geni_master *mas,
+ u16 mode, struct spi_master *spi)
+{
+ u32 m_cmd = 0;
+ u32 spi_tx_cfg, len;
+ struct geni_se *se = &mas->se;
+
+ spi_tx_cfg = readl(se->base + SE_SPI_TRANS_CFG);
+ if (xfer->bits_per_word != mas->cur_bits_per_word) {
+ spi_setup_word_len(mas, mode, xfer->bits_per_word);
+ mas->cur_bits_per_word = xfer->bits_per_word;
+ }
+
+ /* Speed and bits per word can be overridden per transfer */
+ if (xfer->speed_hz != mas->cur_speed_hz) {
+ int ret;
+ u32 clk_sel, m_clk_cfg;
+ unsigned int idx, div;
+
+ ret = get_spi_clk_cfg(xfer->speed_hz, mas, &idx, &div);
+ if (ret) {
+ dev_err(mas->dev, "Err setting clks:%d\n", ret);
+ return;
+ }
+ /*
+ * SPI core clock gets configured with the requested frequency
+ * or the frequency closer to the requested frequency.
+ * For that reason requested frequency is stored in the
+ * cur_speed_hz and referred in the consecutive transfer instead
+ * of calling clk_get_rate() API.
+ */
+ mas->cur_speed_hz = xfer->speed_hz;
+ clk_sel = idx & CLK_SEL_MSK;
+ m_clk_cfg = (div << CLK_DIV_SHFT) | SER_CLK_EN;
+ writel(clk_sel, se->base + SE_GENI_CLK_SEL);
+ writel(m_clk_cfg, se->base + GENI_SER_M_CLK_CFG);
+ }
+
+ mas->tx_rem_bytes = 0;
+ mas->rx_rem_bytes = 0;
+ if (xfer->tx_buf && xfer->rx_buf)
+ m_cmd = SPI_FULL_DUPLEX;
+ else if (xfer->tx_buf)
+ m_cmd = SPI_TX_ONLY;
+ else if (xfer->rx_buf)
+ m_cmd = SPI_RX_ONLY;
+
+ spi_tx_cfg &= ~CS_TOGGLE;
+
+ if (!(mas->cur_bits_per_word % MIN_WORD_LEN))
+ len = xfer->len * BITS_PER_BYTE / mas->cur_bits_per_word;
+ else
+ len = xfer->len / (mas->cur_bits_per_word / BITS_PER_BYTE + 1);
+ len &= TRANS_LEN_MSK;
+
+ mas->cur_xfer = xfer;
+ if (m_cmd & SPI_TX_ONLY) {
+ mas->tx_rem_bytes = xfer->len;
+ writel(len, se->base + SE_SPI_TX_TRANS_LEN);
+ }
+
+ if (m_cmd & SPI_RX_ONLY) {
+ writel(len, se->base + SE_SPI_RX_TRANS_LEN);
+ mas->rx_rem_bytes = xfer->len;
+ }
+ writel(spi_tx_cfg, se->base + SE_SPI_TRANS_CFG);
+ mas->cur_mcmd = CMD_XFER;
+ geni_se_setup_m_cmd(se, m_cmd, FRAGMENTATION);
+
+ /*
+ * TX_WATERMARK_REG should be set after SPI configuration and
+ * setting up GENI SE engine, as driver starts data transfer
+ * for the watermark interrupt.
+ */
+ if (m_cmd & SPI_TX_ONLY)
+ writel(mas->tx_wm, se->base + SE_GENI_TX_WATERMARK_REG);
+}
+
+static void handle_fifo_timeout(struct spi_master *spi,
+ struct spi_message *msg)
+{
+ struct spi_geni_master *mas = spi_master_get_devdata(spi);
+ unsigned long time_left, flags;
+ struct geni_se *se = &mas->se;
+
+ spin_lock_irqsave(&mas->lock, flags);
+ reinit_completion(&mas->xfer_done);
+ mas->cur_mcmd = CMD_CANCEL;
+ geni_se_cancel_m_cmd(se);
+ writel(0, se->base + SE_GENI_TX_WATERMARK_REG);
+ spin_unlock_irqrestore(&mas->lock, flags);
+ time_left = wait_for_completion_timeout(&mas->xfer_done, HZ);
+ if (time_left)
+ return;
+
+ spin_lock_irqsave(&mas->lock, flags);
+ reinit_completion(&mas->xfer_done);
+ geni_se_abort_m_cmd(se);
+ spin_unlock_irqrestore(&mas->lock, flags);
+ time_left = wait_for_completion_timeout(&mas->xfer_done, HZ);
+ if (!time_left)
+ dev_err(mas->dev, "Failed to cancel/abort m_cmd\n");
+}
+
+static int spi_geni_transfer_one(struct spi_master *spi,
+ struct spi_device *slv,
+ struct spi_transfer *xfer)
+{
+ struct spi_geni_master *mas = spi_master_get_devdata(spi);
+
+ /* Terminate and return success for 0 byte length transfer */
+ if (!xfer->len)
+ return 0;
+
+ setup_fifo_xfer(xfer, mas, slv->mode, spi);
+ return 1;
+}
+
+static unsigned int geni_byte_per_fifo_word(struct spi_geni_master *mas)
+{
+ /*
+ * Calculate how many bytes we'll put in each FIFO word. If the
+ * transfer words don't pack cleanly into a FIFO word we'll just put
+ * one transfer word in each FIFO word. If they do pack we'll pack 'em.
+ */
+ if (mas->fifo_width_bits % mas->cur_bits_per_word)
+ return roundup_pow_of_two(DIV_ROUND_UP(mas->cur_bits_per_word,
+ BITS_PER_BYTE));
+
+ return mas->fifo_width_bits / BITS_PER_BYTE;
+}
+
+static void geni_spi_handle_tx(struct spi_geni_master *mas)
+{
+ struct geni_se *se = &mas->se;
+ unsigned int max_bytes;
+ const u8 *tx_buf;
+ unsigned int bytes_per_fifo_word = geni_byte_per_fifo_word(mas);
+ unsigned int i = 0;
+
+ max_bytes = (mas->tx_fifo_depth - mas->tx_wm) * bytes_per_fifo_word;
+ if (mas->tx_rem_bytes < max_bytes)
+ max_bytes = mas->tx_rem_bytes;
+
+ tx_buf = mas->cur_xfer->tx_buf + mas->cur_xfer->len - mas->tx_rem_bytes;
+ while (i < max_bytes) {
+ unsigned int j;
+ unsigned int bytes_to_write;
+ u32 fifo_word = 0;
+ u8 *fifo_byte = (u8 *)&fifo_word;
+
+ bytes_to_write = min(bytes_per_fifo_word, max_bytes - i);
+ for (j = 0; j < bytes_to_write; j++)
+ fifo_byte[j] = tx_buf[i++];
+ iowrite32_rep(se->base + SE_GENI_TX_FIFOn, &fifo_word, 1);
+ }
+ mas->tx_rem_bytes -= max_bytes;
+ if (!mas->tx_rem_bytes)
+ writel(0, se->base + SE_GENI_TX_WATERMARK_REG);
+}
+
+static void geni_spi_handle_rx(struct spi_geni_master *mas)
+{
+ struct geni_se *se = &mas->se;
+ u32 rx_fifo_status;
+ unsigned int rx_bytes;
+ unsigned int rx_last_byte_valid;
+ u8 *rx_buf;
+ unsigned int bytes_per_fifo_word = geni_byte_per_fifo_word(mas);
+ unsigned int i = 0;
+
+ rx_fifo_status = readl(se->base + SE_GENI_RX_FIFO_STATUS);
+ rx_bytes = (rx_fifo_status & RX_FIFO_WC_MSK) * bytes_per_fifo_word;
+ if (rx_fifo_status & RX_LAST) {
+ rx_last_byte_valid = rx_fifo_status & RX_LAST_BYTE_VALID_MSK;
+ rx_last_byte_valid >>= RX_LAST_BYTE_VALID_SHFT;
+ if (rx_last_byte_valid && rx_last_byte_valid < 4)
+ rx_bytes -= bytes_per_fifo_word - rx_last_byte_valid;
+ }
+ if (mas->rx_rem_bytes < rx_bytes)
+ rx_bytes = mas->rx_rem_bytes;
+
+ rx_buf = mas->cur_xfer->rx_buf + mas->cur_xfer->len - mas->rx_rem_bytes;
+ while (i < rx_bytes) {
+ u32 fifo_word = 0;
+ u8 *fifo_byte = (u8 *)&fifo_word;
+ unsigned int bytes_to_read;
+ unsigned int j;
+
+ bytes_to_read = min(bytes_per_fifo_word, rx_bytes - i);
+ ioread32_rep(se->base + SE_GENI_RX_FIFOn, &fifo_word, 1);
+ for (j = 0; j < bytes_to_read; j++)
+ rx_buf[i++] = fifo_byte[j];
+ }
+ mas->rx_rem_bytes -= rx_bytes;
+}
+
+static irqreturn_t geni_spi_isr(int irq, void *data)
+{
+ struct spi_master *spi = data;
+ struct spi_geni_master *mas = spi_master_get_devdata(spi);
+ struct geni_se *se = &mas->se;
+ u32 m_irq;
+ unsigned long flags;
+ irqreturn_t ret = IRQ_HANDLED;
+
+ if (mas->cur_mcmd == CMD_NONE)
+ return IRQ_NONE;
+
+ spin_lock_irqsave(&mas->lock, flags);
+ m_irq = readl(se->base + SE_GENI_M_IRQ_STATUS);
+
+ if ((m_irq & M_RX_FIFO_WATERMARK_EN) || (m_irq & M_RX_FIFO_LAST_EN))
+ geni_spi_handle_rx(mas);
+
+ if (m_irq & M_TX_FIFO_WATERMARK_EN)
+ geni_spi_handle_tx(mas);
+
+ if (m_irq & M_CMD_DONE_EN) {
+ if (mas->cur_mcmd == CMD_XFER)
+ spi_finalize_current_transfer(spi);
+ else if (mas->cur_mcmd == CMD_CS)
+ complete(&mas->xfer_done);
+ mas->cur_mcmd = CMD_NONE;
+ /*
+ * If this happens, then a CMD_DONE came before all the Tx
+ * buffer bytes were sent out. This is unusual, log this
+ * condition and disable the WM interrupt to prevent the
+ * system from stalling due an interrupt storm.
+ * If this happens when all Rx bytes haven't been received, log
+ * the condition.
+ * The only known time this can happen is if bits_per_word != 8
+ * and some registers that expect xfer lengths in num spi_words
+ * weren't written correctly.
+ */
+ if (mas->tx_rem_bytes) {
+ writel(0, se->base + SE_GENI_TX_WATERMARK_REG);
+ dev_err(mas->dev, "Premature done. tx_rem = %d bpw%d\n",
+ mas->tx_rem_bytes, mas->cur_bits_per_word);
+ }
+ if (mas->rx_rem_bytes)
+ dev_err(mas->dev, "Premature done. rx_rem = %d bpw%d\n",
+ mas->rx_rem_bytes, mas->cur_bits_per_word);
+ }
+
+ if ((m_irq & M_CMD_CANCEL_EN) || (m_irq & M_CMD_ABORT_EN)) {
+ mas->cur_mcmd = CMD_NONE;
+ complete(&mas->xfer_done);
+ }
+
+ writel(m_irq, se->base + SE_GENI_M_IRQ_CLEAR);
+ spin_unlock_irqrestore(&mas->lock, flags);
+ return ret;
+}
+
+static int spi_geni_probe(struct platform_device *pdev)
+{
+ int ret;
+ struct spi_master *spi;
+ struct spi_geni_master *mas;
+ struct resource *res;
+ struct geni_se *se;
+
+ spi = spi_alloc_master(&pdev->dev, sizeof(*mas));
+ if (!spi)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, spi);
+ mas = spi_master_get_devdata(spi);
+ mas->dev = &pdev->dev;
+ mas->se.dev = &pdev->dev;
+ mas->se.wrapper = dev_get_drvdata(pdev->dev.parent);
+ se = &mas->se;
+
+ spi->bus_num = -1;
+ spi->dev.of_node = pdev->dev.of_node;
+ mas->se.clk = devm_clk_get(&pdev->dev, "se");
+ if (IS_ERR(mas->se.clk)) {
+ ret = PTR_ERR(mas->se.clk);
+ dev_err(&pdev->dev, "Err getting SE Core clk %d\n", ret);
+ goto spi_geni_probe_err;
+ }
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ se->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(se->base)) {
+ ret = PTR_ERR(se->base);
+ goto spi_geni_probe_err;
+ }
+
+ spi->mode_bits = SPI_CPOL | SPI_CPHA | SPI_LOOP | SPI_CS_HIGH;
+ spi->bits_per_word_mask = SPI_BPW_RANGE_MASK(4, 32);
+ spi->num_chipselect = 4;
+ spi->max_speed_hz = 50000000;
+ spi->prepare_message = spi_geni_prepare_message;
+ spi->transfer_one = spi_geni_transfer_one;
+ spi->auto_runtime_pm = true;
+ spi->handle_err = handle_fifo_timeout;
+ spi->set_cs = spi_geni_set_cs;
+
+ init_completion(&mas->xfer_done);
+ spin_lock_init(&mas->lock);
+ pm_runtime_enable(&pdev->dev);
+
+ ret = spi_geni_init(mas);
+ if (ret)
+ goto spi_geni_probe_runtime_disable;
+
+ mas->irq = platform_get_irq(pdev, 0);
+ if (mas->irq < 0) {
+ ret = mas->irq;
+ dev_err(&pdev->dev, "Err getting IRQ %d\n", ret);
+ goto spi_geni_probe_runtime_disable;
+ }
+
+ ret = request_irq(mas->irq, geni_spi_isr,
+ IRQF_TRIGGER_HIGH, "spi_geni", spi);
+ if (ret)
+ goto spi_geni_probe_runtime_disable;
+
+ ret = spi_register_master(spi);
+ if (ret)
+ goto spi_geni_probe_free_irq;
+
+ return 0;
+spi_geni_probe_free_irq:
+ free_irq(mas->irq, spi);
+spi_geni_probe_runtime_disable:
+ pm_runtime_disable(&pdev->dev);
+spi_geni_probe_err:
+ spi_master_put(spi);
+ return ret;
+}
+
+static int spi_geni_remove(struct platform_device *pdev)
+{
+ struct spi_master *spi = platform_get_drvdata(pdev);
+ struct spi_geni_master *mas = spi_master_get_devdata(spi);
+
+ /* Unregister _before_ disabling pm_runtime() so we stop transfers */
+ spi_unregister_master(spi);
+
+ free_irq(mas->irq, spi);
+ pm_runtime_disable(&pdev->dev);
+ return 0;
+}
+
+static int __maybe_unused spi_geni_runtime_suspend(struct device *dev)
+{
+ struct spi_master *spi = dev_get_drvdata(dev);
+ struct spi_geni_master *mas = spi_master_get_devdata(spi);
+
+ return geni_se_resources_off(&mas->se);
+}
+
+static int __maybe_unused spi_geni_runtime_resume(struct device *dev)
+{
+ struct spi_master *spi = dev_get_drvdata(dev);
+ struct spi_geni_master *mas = spi_master_get_devdata(spi);
+
+ return geni_se_resources_on(&mas->se);
+}
+
+static int __maybe_unused spi_geni_suspend(struct device *dev)
+{
+ struct spi_master *spi = dev_get_drvdata(dev);
+ int ret;
+
+ ret = spi_master_suspend(spi);
+ if (ret)
+ return ret;
+
+ ret = pm_runtime_force_suspend(dev);
+ if (ret)
+ spi_master_resume(spi);
+
+ return ret;
+}
+
+static int __maybe_unused spi_geni_resume(struct device *dev)
+{
+ struct spi_master *spi = dev_get_drvdata(dev);
+ int ret;
+
+ ret = pm_runtime_force_resume(dev);
+ if (ret)
+ return ret;
+
+ ret = spi_master_resume(spi);
+ if (ret)
+ pm_runtime_force_suspend(dev);
+
+ return ret;
+}
+
+static const struct dev_pm_ops spi_geni_pm_ops = {
+ SET_RUNTIME_PM_OPS(spi_geni_runtime_suspend,
+ spi_geni_runtime_resume, NULL)
+ SET_SYSTEM_SLEEP_PM_OPS(spi_geni_suspend, spi_geni_resume)
+};
+
+static const struct of_device_id spi_geni_dt_match[] = {
+ { .compatible = "qcom,geni-spi" },
+ {}
+};
+MODULE_DEVICE_TABLE(of, spi_geni_dt_match);
+
+static struct platform_driver spi_geni_driver = {
+ .probe = spi_geni_probe,
+ .remove = spi_geni_remove,
+ .driver = {
+ .name = "geni_spi",
+ .pm = &spi_geni_pm_ops,
+ .of_match_table = spi_geni_dt_match,
+ },
+};
+module_platform_driver(spi_geni_driver);
+
+MODULE_DESCRIPTION("SPI driver for GENI based QUP cores");
+MODULE_LICENSE("GPL v2");
spi_gpio->miso = devm_gpiod_get_optional(dev, "miso", GPIOD_IN);
if (IS_ERR(spi_gpio->miso))
return PTR_ERR(spi_gpio->miso);
- if (!spi_gpio->miso)
- /* HW configuration without MISO pin */
- *mflags |= SPI_MASTER_NO_RX;
+ /*
+ * No setting SPI_MASTER_NO_RX here - if there is only a MOSI
+ * pin connected the host can still do RX by changing the
+ * direction of the line.
+ */
spi_gpio->sck = devm_gpiod_get(dev, "sck", GPIOD_OUT_LOW);
if (IS_ERR(spi_gpio->sck))
spi_gpio->bitbang.chipselect = spi_gpio_chipselect;
spi_gpio->bitbang.set_line_direction = spi_gpio_set_direction;
- if ((master_flags & (SPI_MASTER_NO_TX | SPI_MASTER_NO_RX)) == 0) {
+ if ((master_flags & SPI_MASTER_NO_TX) == 0) {
spi_gpio->bitbang.txrx_word[SPI_MODE_0] = spi_gpio_txrx_word_mode0;
spi_gpio->bitbang.txrx_word[SPI_MODE_1] = spi_gpio_txrx_word_mode1;
spi_gpio->bitbang.txrx_word[SPI_MODE_2] = spi_gpio_txrx_word_mode2;
static int spi_gpio_remove(struct platform_device *pdev)
{
struct spi_gpio *spi_gpio;
- struct spi_gpio_platform_data *pdata;
spi_gpio = platform_get_drvdata(pdev);
- pdata = dev_get_platdata(&pdev->dev);
/* stop() unregisters child devices too */
spi_bitbang_stop(&spi_gpio->bitbang);
void (*trigger)(struct spi_imx_data *);
int (*rx_available)(struct spi_imx_data *);
void (*reset)(struct spi_imx_data *);
+ void (*setup_wml)(struct spi_imx_data *);
void (*disable)(struct spi_imx_data *);
bool has_dmamode;
bool has_slavemode;
struct spi_transfer *transfer)
{
struct spi_imx_data *spi_imx = spi_master_get_devdata(master);
- unsigned int bytes_per_word, i;
if (!master->dma_rx)
return false;
if (spi_imx->slave_mode)
return false;
- bytes_per_word = spi_imx_bytes_per_word(transfer->bits_per_word);
-
- for (i = spi_imx->devtype_data->fifo_size / 2; i > 0; i--) {
- if (!(transfer->len % (i * bytes_per_word)))
- break;
- }
+ if (transfer->len < spi_imx->devtype_data->fifo_size)
+ return false;
- spi_imx->wml = i;
spi_imx->dynamic_burst = 0;
return true;
else /* SCLK is _very_ slow */
usleep_range(delay, delay + 10);
+ return 0;
+}
+
+static void mx51_setup_wml(struct spi_imx_data *spi_imx)
+{
/*
* Configure the DMA register: setup the watermark
* and enable DMA request.
*/
- writel(MX51_ECSPI_DMA_RX_WML(spi_imx->wml) |
+ writel(MX51_ECSPI_DMA_RX_WML(spi_imx->wml - 1) |
MX51_ECSPI_DMA_TX_WML(spi_imx->wml) |
MX51_ECSPI_DMA_RXT_WML(spi_imx->wml) |
MX51_ECSPI_DMA_TEDEN | MX51_ECSPI_DMA_RXDEN |
MX51_ECSPI_DMA_RXTDEN, spi_imx->base + MX51_ECSPI_DMA);
-
- return 0;
}
static int mx51_ecspi_rx_available(struct spi_imx_data *spi_imx)
.trigger = mx51_ecspi_trigger,
.rx_available = mx51_ecspi_rx_available,
.reset = mx51_ecspi_reset,
+ .setup_wml = mx51_setup_wml,
.fifo_size = 64,
.has_dmamode = true,
.dynamic_burst = true,
struct spi_transfer *t)
{
struct spi_imx_data *spi_imx = spi_master_get_devdata(spi->master);
- int ret;
if (!t)
return 0;
else
spi_imx->usedma = 0;
- if (spi_imx->usedma) {
- ret = spi_imx_dma_configure(spi->master);
- if (ret)
- return ret;
- }
-
if (is_imx53_ecspi(spi_imx) && spi_imx->slave_mode) {
spi_imx->rx = mx53_ecspi_rx_slave;
spi_imx->tx = mx53_ecspi_tx_slave;
unsigned long timeout;
struct spi_master *master = spi_imx->bitbang.master;
struct sg_table *tx = &transfer->tx_sg, *rx = &transfer->rx_sg;
+ struct scatterlist *last_sg = sg_last(rx->sgl, rx->nents);
+ unsigned int bytes_per_word, i;
+ int ret;
+
+ /* Get the right burst length from the last sg to ensure no tail data */
+ bytes_per_word = spi_imx_bytes_per_word(transfer->bits_per_word);
+ for (i = spi_imx->devtype_data->fifo_size / 2; i > 0; i--) {
+ if (!(sg_dma_len(last_sg) % (i * bytes_per_word)))
+ break;
+ }
+ /* Use 1 as wml in case no available burst length got */
+ if (i == 0)
+ i = 1;
+
+ spi_imx->wml = i;
+
+ ret = spi_imx_dma_configure(master);
+ if (ret)
+ return ret;
+
+ if (!spi_imx->devtype_data->setup_wml) {
+ dev_err(spi_imx->dev, "No setup_wml()?\n");
+ return -EINVAL;
+ }
+ spi_imx->devtype_data->setup_wml(spi_imx);
/*
* The TX DMA setup starts the transfer, so make sure RX is configured
#include "internals.h"
+#define SPI_MEM_MAX_BUSWIDTH 4
+
/**
* spi_controller_dma_map_mem_op_data() - DMA-map the buffer attached to a
* memory operation
}
EXPORT_SYMBOL_GPL(spi_mem_default_supports_op);
+static bool spi_mem_buswidth_is_valid(u8 buswidth)
+{
+ if (hweight8(buswidth) > 1 || buswidth > SPI_MEM_MAX_BUSWIDTH)
+ return false;
+
+ return true;
+}
+
+static int spi_mem_check_op(const struct spi_mem_op *op)
+{
+ if (!op->cmd.buswidth)
+ return -EINVAL;
+
+ if ((op->addr.nbytes && !op->addr.buswidth) ||
+ (op->dummy.nbytes && !op->dummy.buswidth) ||
+ (op->data.nbytes && !op->data.buswidth))
+ return -EINVAL;
+
+ if (!spi_mem_buswidth_is_valid(op->cmd.buswidth) ||
+ !spi_mem_buswidth_is_valid(op->addr.buswidth) ||
+ !spi_mem_buswidth_is_valid(op->dummy.buswidth) ||
+ !spi_mem_buswidth_is_valid(op->data.buswidth))
+ return -EINVAL;
+
+ return 0;
+}
+
+static bool spi_mem_internal_supports_op(struct spi_mem *mem,
+ const struct spi_mem_op *op)
+{
+ struct spi_controller *ctlr = mem->spi->controller;
+
+ if (ctlr->mem_ops && ctlr->mem_ops->supports_op)
+ return ctlr->mem_ops->supports_op(mem, op);
+
+ return spi_mem_default_supports_op(mem, op);
+}
+
/**
* spi_mem_supports_op() - Check if a memory device and the controller it is
* connected to support a specific memory operation
*/
bool spi_mem_supports_op(struct spi_mem *mem, const struct spi_mem_op *op)
{
- struct spi_controller *ctlr = mem->spi->controller;
-
- if (ctlr->mem_ops && ctlr->mem_ops->supports_op)
- return ctlr->mem_ops->supports_op(mem, op);
+ if (spi_mem_check_op(op))
+ return false;
- return spi_mem_default_supports_op(mem, op);
+ return spi_mem_internal_supports_op(mem, op);
}
EXPORT_SYMBOL_GPL(spi_mem_supports_op);
u8 *tmpbuf;
int ret;
- if (!spi_mem_supports_op(mem, op))
+ ret = spi_mem_check_op(op);
+ if (ret)
+ return ret;
+
+ if (!spi_mem_internal_supports_op(mem, op))
return -ENOTSUPP;
if (ctlr->mem_ops) {
int spi_mem_adjust_op_size(struct spi_mem *mem, struct spi_mem_op *op)
{
struct spi_controller *ctlr = mem->spi->controller;
+ size_t len;
+
+ len = sizeof(op->cmd.opcode) + op->addr.nbytes + op->dummy.nbytes;
if (ctlr->mem_ops && ctlr->mem_ops->adjust_op_size)
return ctlr->mem_ops->adjust_op_size(mem, op);
+ if (!ctlr->mem_ops || !ctlr->mem_ops->exec_op) {
+ if (len > spi_max_transfer_size(mem->spi))
+ return -EINVAL;
+
+ op->data.nbytes = min3((size_t)op->data.nbytes,
+ spi_max_transfer_size(mem->spi),
+ spi_max_message_size(mem->spi) -
+ len);
+ if (!op->data.nbytes)
+ return -EINVAL;
+ }
+
return 0;
}
EXPORT_SYMBOL_GPL(spi_mem_adjust_op_size);
struct clk *parent_clk, *sel_clk, *spi_clk;
struct spi_transfer *cur_transfer;
u32 xfer_len;
+ u32 num_xfered;
struct scatterlist *tx_sgl, *rx_sgl;
u32 tx_sgl_len, rx_sgl_len;
const struct mtk_spi_compatible *dev_comp;
mdata->cur_transfer = xfer;
mdata->xfer_len = min(MTK_SPI_MAX_FIFO_SIZE, xfer->len);
+ mdata->num_xfered = 0;
mtk_spi_prepare_transfer(master, xfer);
mtk_spi_setup_packet(master);
mdata->tx_sgl_len = 0;
mdata->rx_sgl_len = 0;
mdata->cur_transfer = xfer;
+ mdata->num_xfered = 0;
mtk_spi_prepare_transfer(master, xfer);
static irqreturn_t mtk_spi_interrupt(int irq, void *dev_id)
{
- u32 cmd, reg_val, cnt, remainder;
+ u32 cmd, reg_val, cnt, remainder, len;
struct spi_master *master = dev_id;
struct mtk_spi *mdata = spi_master_get_devdata(master);
struct spi_transfer *trans = mdata->cur_transfer;
if (trans->rx_buf) {
cnt = mdata->xfer_len / 4;
ioread32_rep(mdata->base + SPI_RX_DATA_REG,
- trans->rx_buf, cnt);
+ trans->rx_buf + mdata->num_xfered, cnt);
remainder = mdata->xfer_len % 4;
if (remainder > 0) {
reg_val = readl(mdata->base + SPI_RX_DATA_REG);
- memcpy(trans->rx_buf + (cnt * 4),
- ®_val, remainder);
+ memcpy(trans->rx_buf +
+ mdata->num_xfered +
+ (cnt * 4),
+ ®_val,
+ remainder);
}
}
- trans->len -= mdata->xfer_len;
- if (!trans->len) {
+ mdata->num_xfered += mdata->xfer_len;
+ if (mdata->num_xfered == trans->len) {
spi_finalize_current_transfer(master);
return IRQ_HANDLED;
}
- if (trans->tx_buf)
- trans->tx_buf += mdata->xfer_len;
- if (trans->rx_buf)
- trans->rx_buf += mdata->xfer_len;
-
- mdata->xfer_len = min(MTK_SPI_MAX_FIFO_SIZE, trans->len);
+ len = trans->len - mdata->num_xfered;
+ mdata->xfer_len = min(MTK_SPI_MAX_FIFO_SIZE, len);
mtk_spi_setup_packet(master);
- cnt = trans->len / 4;
- iowrite32_rep(mdata->base + SPI_TX_DATA_REG, trans->tx_buf, cnt);
+ cnt = len / 4;
+ iowrite32_rep(mdata->base + SPI_TX_DATA_REG,
+ trans->tx_buf + mdata->num_xfered, cnt);
- remainder = trans->len % 4;
+ remainder = len % 4;
if (remainder > 0) {
reg_val = 0;
- memcpy(®_val, trans->tx_buf + (cnt * 4), remainder);
+ memcpy(®_val,
+ trans->tx_buf + (cnt * 4) + mdata->num_xfered,
+ remainder);
writel(reg_val, mdata->base + SPI_TX_DATA_REG);
}
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/gcd.h>
+#include <linux/iopoll.h>
#include <linux/spi/spi.h>
#include <linux/gpio.h>
};
struct omap2_mcspi {
+ struct completion txdone;
struct spi_master *master;
/* Virtual base address of the controller */
void __iomem *base;
struct device *dev;
struct omap2_mcspi_regs ctx;
int fifo_depth;
+ bool slave_aborted;
unsigned int pin_dir:1;
};
}
}
-static void omap2_mcspi_set_master_mode(struct spi_master *master)
+static void omap2_mcspi_set_mode(struct spi_master *master)
{
struct omap2_mcspi *mcspi = spi_master_get_devdata(master);
struct omap2_mcspi_regs *ctx = &mcspi->ctx;
u32 l;
/*
- * Setup when switching from (reset default) slave mode
- * to single-channel master mode
+ * Choose master or slave mode
*/
l = mcspi_read_reg(master, OMAP2_MCSPI_MODULCTRL);
- l &= ~(OMAP2_MCSPI_MODULCTRL_STEST | OMAP2_MCSPI_MODULCTRL_MS);
- l |= OMAP2_MCSPI_MODULCTRL_SINGLE;
+ l &= ~(OMAP2_MCSPI_MODULCTRL_STEST);
+ if (spi_controller_is_slave(master)) {
+ l |= (OMAP2_MCSPI_MODULCTRL_MS);
+ } else {
+ l &= ~(OMAP2_MCSPI_MODULCTRL_MS);
+ l |= OMAP2_MCSPI_MODULCTRL_SINGLE;
+ }
mcspi_write_reg(master, OMAP2_MCSPI_MODULCTRL, l);
ctx->modulctrl = l;
struct omap2_mcspi_cs *cs = spi->controller_state;
struct omap2_mcspi *mcspi;
unsigned int wcnt;
- int max_fifo_depth, fifo_depth, bytes_per_word;
+ int max_fifo_depth, bytes_per_word;
u32 chconf, xferlevel;
mcspi = spi_master_get_devdata(master);
else
max_fifo_depth = OMAP2_MCSPI_MAX_FIFODEPTH;
- fifo_depth = gcd(t->len, max_fifo_depth);
- if (fifo_depth < 2 || fifo_depth % bytes_per_word != 0)
- goto disable_fifo;
-
wcnt = t->len / bytes_per_word;
if (wcnt > OMAP2_MCSPI_MAX_FIFOWCNT)
goto disable_fifo;
xferlevel = wcnt << 16;
if (t->rx_buf != NULL) {
chconf |= OMAP2_MCSPI_CHCONF_FFER;
- xferlevel |= (fifo_depth - 1) << 8;
+ xferlevel |= (bytes_per_word - 1) << 8;
}
+
if (t->tx_buf != NULL) {
chconf |= OMAP2_MCSPI_CHCONF_FFET;
- xferlevel |= fifo_depth - 1;
+ xferlevel |= bytes_per_word - 1;
}
mcspi_write_reg(master, OMAP2_MCSPI_XFERLEVEL, xferlevel);
mcspi_write_chconf0(spi, chconf);
- mcspi->fifo_depth = fifo_depth;
+ mcspi->fifo_depth = max_fifo_depth;
return;
}
static int mcspi_wait_for_reg_bit(void __iomem *reg, unsigned long bit)
{
- unsigned long timeout;
-
- timeout = jiffies + msecs_to_jiffies(1000);
- while (!(readl_relaxed(reg) & bit)) {
- if (time_after(jiffies, timeout)) {
- if (!(readl_relaxed(reg) & bit))
- return -ETIMEDOUT;
- else
- return 0;
- }
- cpu_relax();
+ u32 val;
+
+ return readl_poll_timeout(reg, val, val & bit, 1, MSEC_PER_SEC);
+}
+
+static int mcspi_wait_for_completion(struct omap2_mcspi *mcspi,
+ struct completion *x)
+{
+ if (spi_controller_is_slave(mcspi->master)) {
+ if (wait_for_completion_interruptible(x) ||
+ mcspi->slave_aborted)
+ return -EINTR;
+ } else {
+ wait_for_completion(x);
}
+
return 0;
}
dma_async_issue_pending(mcspi_dma->dma_rx);
omap2_mcspi_set_dma_req(spi, 1, 1);
- wait_for_completion(&mcspi_dma->dma_rx_completion);
+ ret = mcspi_wait_for_completion(mcspi, &mcspi_dma->dma_rx_completion);
+ if (ret || mcspi->slave_aborted) {
+ dmaengine_terminate_sync(mcspi_dma->dma_rx);
+ omap2_mcspi_set_dma_req(spi, 1, 0);
+ return 0;
+ }
for (x = 0; x < nb_sizes; x++)
kfree(sg_out[x]);
struct dma_slave_config cfg;
enum dma_slave_buswidth width;
unsigned es;
- u32 burst;
void __iomem *chstat_reg;
void __iomem *irqstat_reg;
int wait_res;
}
count = xfer->len;
- burst = 1;
-
- if (mcspi->fifo_depth > 0) {
- if (count > mcspi->fifo_depth)
- burst = mcspi->fifo_depth / es;
- else
- burst = count / es;
- }
memset(&cfg, 0, sizeof(cfg));
cfg.src_addr = cs->phys + OMAP2_MCSPI_RX0;
cfg.dst_addr = cs->phys + OMAP2_MCSPI_TX0;
cfg.src_addr_width = width;
cfg.dst_addr_width = width;
- cfg.src_maxburst = burst;
- cfg.dst_maxburst = burst;
+ cfg.src_maxburst = es;
+ cfg.dst_maxburst = es;
rx = xfer->rx_buf;
tx = xfer->tx_buf;
- if (tx != NULL)
+ mcspi->slave_aborted = false;
+ reinit_completion(&mcspi_dma->dma_tx_completion);
+ reinit_completion(&mcspi_dma->dma_rx_completion);
+ reinit_completion(&mcspi->txdone);
+ if (tx) {
+ /* Enable EOW IRQ to know end of tx in slave mode */
+ if (spi_controller_is_slave(spi->master))
+ mcspi_write_reg(spi->master,
+ OMAP2_MCSPI_IRQENABLE,
+ OMAP2_MCSPI_IRQSTATUS_EOW);
omap2_mcspi_tx_dma(spi, xfer, cfg);
+ }
if (rx != NULL)
count = omap2_mcspi_rx_dma(spi, xfer, cfg, es);
if (tx != NULL) {
- wait_for_completion(&mcspi_dma->dma_tx_completion);
+ int ret;
+
+ ret = mcspi_wait_for_completion(mcspi, &mcspi_dma->dma_tx_completion);
+ if (ret || mcspi->slave_aborted) {
+ dmaengine_terminate_sync(mcspi_dma->dma_tx);
+ omap2_mcspi_set_dma_req(spi, 0, 0);
+ return 0;
+ }
+
+ if (spi_controller_is_slave(mcspi->master)) {
+ ret = mcspi_wait_for_completion(mcspi, &mcspi->txdone);
+ if (ret || mcspi->slave_aborted)
+ return 0;
+ }
if (mcspi->fifo_depth > 0) {
irqstat_reg = mcspi->base + OMAP2_MCSPI_IRQSTATUS;
gpio_free(spi->cs_gpio);
}
+static irqreturn_t omap2_mcspi_irq_handler(int irq, void *data)
+{
+ struct omap2_mcspi *mcspi = data;
+ u32 irqstat;
+
+ irqstat = mcspi_read_reg(mcspi->master, OMAP2_MCSPI_IRQSTATUS);
+ if (!irqstat)
+ return IRQ_NONE;
+
+ /* Disable IRQ and wakeup slave xfer task */
+ mcspi_write_reg(mcspi->master, OMAP2_MCSPI_IRQENABLE, 0);
+ if (irqstat & OMAP2_MCSPI_IRQSTATUS_EOW)
+ complete(&mcspi->txdone);
+
+ return IRQ_HANDLED;
+}
+
+static int omap2_mcspi_slave_abort(struct spi_master *master)
+{
+ struct omap2_mcspi *mcspi = spi_master_get_devdata(master);
+ struct omap2_mcspi_dma *mcspi_dma = mcspi->dma_channels;
+
+ mcspi->slave_aborted = true;
+ complete(&mcspi_dma->dma_rx_completion);
+ complete(&mcspi_dma->dma_tx_completion);
+ complete(&mcspi->txdone);
+
+ return 0;
+}
+
static int omap2_mcspi_transfer_one(struct spi_master *master,
struct spi_device *spi,
struct spi_transfer *t)
struct spi_device *spi,
struct spi_transfer *xfer)
{
+ struct omap2_mcspi *mcspi = spi_master_get_devdata(spi->master);
+ struct omap2_mcspi_dma *mcspi_dma =
+ &mcspi->dma_channels[spi->chip_select];
+
+ if (!mcspi_dma->dma_rx || !mcspi_dma->dma_tx)
+ return false;
+
+ if (spi_controller_is_slave(master))
+ return true;
+
return (xfer->len >= DMA_MIN_BYTES);
}
-static int omap2_mcspi_master_setup(struct omap2_mcspi *mcspi)
+static int omap2_mcspi_controller_setup(struct omap2_mcspi *mcspi)
{
struct spi_master *master = mcspi->master;
struct omap2_mcspi_regs *ctx = &mcspi->ctx;
OMAP2_MCSPI_WAKEUPENABLE_WKEN);
ctx->wakeupenable = OMAP2_MCSPI_WAKEUPENABLE_WKEN;
- omap2_mcspi_set_master_mode(master);
+ omap2_mcspi_set_mode(master);
pm_runtime_mark_last_busy(mcspi->dev);
pm_runtime_put_autosuspend(mcspi->dev);
return 0;
struct device_node *node = pdev->dev.of_node;
const struct of_device_id *match;
- master = spi_alloc_master(&pdev->dev, sizeof *mcspi);
- if (master == NULL) {
- dev_dbg(&pdev->dev, "master allocation failed\n");
+ if (of_property_read_bool(node, "spi-slave"))
+ master = spi_alloc_slave(&pdev->dev, sizeof(*mcspi));
+ else
+ master = spi_alloc_master(&pdev->dev, sizeof(*mcspi));
+ if (!master)
return -ENOMEM;
- }
/* the spi->mode bits understood by this driver: */
master->mode_bits = SPI_CPOL | SPI_CPHA | SPI_CS_HIGH;
master->transfer_one = omap2_mcspi_transfer_one;
master->set_cs = omap2_mcspi_set_cs;
master->cleanup = omap2_mcspi_cleanup;
+ master->slave_abort = omap2_mcspi_slave_abort;
master->dev.of_node = node;
master->max_speed_hz = OMAP2_MCSPI_MAX_FREQ;
master->min_speed_hz = OMAP2_MCSPI_MAX_FREQ >> 15;
sprintf(mcspi->dma_channels[i].dma_tx_ch_name, "tx%d", i);
}
+ status = platform_get_irq(pdev, 0);
+ if (status == -EPROBE_DEFER)
+ goto free_master;
+ if (status < 0) {
+ dev_err(&pdev->dev, "no irq resource found\n");
+ goto free_master;
+ }
+ init_completion(&mcspi->txdone);
+ status = devm_request_irq(&pdev->dev, status,
+ omap2_mcspi_irq_handler, 0, pdev->name,
+ mcspi);
+ if (status) {
+ dev_err(&pdev->dev, "Cannot request IRQ");
+ goto free_master;
+ }
+
pm_runtime_use_autosuspend(&pdev->dev);
pm_runtime_set_autosuspend_delay(&pdev->dev, SPI_AUTOSUSPEND_TIMEOUT);
pm_runtime_enable(&pdev->dev);
- status = omap2_mcspi_master_setup(mcspi);
+ status = omap2_mcspi_controller_setup(mcspi);
if (status < 0)
goto disable_pm;
- status = devm_spi_register_master(&pdev->dev, master);
+ status = devm_spi_register_controller(&pdev->dev, master);
if (status < 0)
goto disable_pm;
int word_len;
struct orion_spi *orion_spi;
int cs = spi->chip_select;
+ void __iomem *vaddr;
word_len = spi->bits_per_word;
count = xfer->len;
* Use SPI direct write mode if base address is available. Otherwise
* fall back to PIO mode for this transfer.
*/
- if ((orion_spi->child[cs].direct_access.vaddr) && (xfer->tx_buf) &&
- (word_len == 8)) {
+ vaddr = orion_spi->child[cs].direct_access.vaddr;
+
+ if (vaddr && xfer->tx_buf && word_len == 8) {
unsigned int cnt = count / 4;
unsigned int rem = count % 4;
* Send the TX-data to the SPI device via the direct
* mapped address window
*/
- iowrite32_rep(orion_spi->child[cs].direct_access.vaddr,
- xfer->tx_buf, cnt);
+ iowrite32_rep(vaddr, xfer->tx_buf, cnt);
if (rem) {
u32 *buf = (u32 *)xfer->tx_buf;
- iowrite8_rep(orion_spi->child[cs].direct_access.vaddr,
- &buf[cnt], rem);
+ iowrite8_rep(vaddr, &buf[cnt], rem);
}
return count;
}
for_each_available_child_of_node(pdev->dev.of_node, np) {
+ struct orion_direct_acc *dir_acc;
u32 cs;
int cs_gpio;
* This needs to get extended for the direct SPI-NOR / SPI-NAND
* support, once this gets implemented.
*/
- spi->child[cs].direct_access.vaddr = devm_ioremap(&pdev->dev,
- r->start,
- PAGE_SIZE);
- if (!spi->child[cs].direct_access.vaddr) {
+ dir_acc = &spi->child[cs].direct_access;
+ dir_acc->vaddr = devm_ioremap(&pdev->dev, r->start, PAGE_SIZE);
+ if (!dir_acc->vaddr) {
status = -ENOMEM;
goto out_rel_axi_clk;
}
- spi->child[cs].direct_access.size = PAGE_SIZE;
+ dir_acc->size = PAGE_SIZE;
dev_info(&pdev->dev, "CS%d configured for direct access\n", cs);
}
/* allocate coherent DMAable memory for hardware buffer descriptors. */
sqi->bd = dma_zalloc_coherent(&sqi->master->dev,
sizeof(*bd) * PESQI_BD_COUNT,
- &sqi->bd_dma, GFP_DMA32);
+ &sqi->bd_dma, GFP_KERNEL);
if (!sqi->bd) {
dev_err(&sqi->master->dev, "failed allocating dma buffer\n");
return -ENOMEM;
master->max_speed_hz = clk_get_rate(sqi->base_clk);
master->dma_alignment = 32;
master->max_dma_len = PESQI_BD_BUF_LEN_MAX;
- master->dev.of_node = of_node_get(pdev->dev.of_node);
+ master->dev.of_node = pdev->dev.of_node;
master->mode_bits = SPI_MODE_3 | SPI_MODE_0 | SPI_TX_DUAL |
SPI_RX_DUAL | SPI_TX_QUAD | SPI_RX_QUAD;
master->flags = SPI_MASTER_HALF_DUPLEX;
desc_rx = dmaengine_prep_slave_sg(master->dma_rx,
xfer->rx_sg.sgl,
xfer->rx_sg.nents,
- DMA_FROM_DEVICE,
+ DMA_DEV_TO_MEM,
DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
if (!desc_rx) {
ret = -EINVAL;
desc_tx = dmaengine_prep_slave_sg(master->dma_tx,
xfer->tx_sg.sgl,
xfer->tx_sg.nents,
- DMA_TO_DEVICE,
+ DMA_MEM_TO_DEV,
DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
if (!desc_tx) {
ret = -EINVAL;
if (ret)
goto err_master;
- master->dev.of_node = of_node_get(pdev->dev.of_node);
+ master->dev.of_node = pdev->dev.of_node;
master->mode_bits = SPI_MODE_3 | SPI_MODE_0 | SPI_CS_HIGH;
master->num_chipselect = 1; /* single chip-select */
master->max_speed_hz = clk_get_rate(pic32s->clk);
struct spi_message *message = NULL;
struct spi_transfer *transfer = NULL;
struct spi_transfer *previous = NULL;
- struct chip_data *chip;
unsigned long time, timeout;
- chip = pl022->cur_chip;
message = pl022->cur_msg;
while (message->state != STATE_DONE) {
int ret;
ret = spi_master_suspend(pl022->master);
- if (ret) {
- dev_warn(dev, "cannot suspend master\n");
+ if (ret)
return ret;
- }
ret = pm_runtime_force_suspend(dev);
if (ret) {
/* Start the queue running */
ret = spi_master_resume(pl022->master);
- if (ret)
- dev_err(dev, "problem starting queue (%d)\n", ret);
- else
+ if (!ret)
dev_dbg(dev, "resumed\n");
return ret;
#include <linux/clk.h>
#include <linux/pm_runtime.h>
#include <linux/acpi.h>
+#include <linux/of_device.h>
#include "spi-pxa2xx.h"
bytes_left = drv_data->rx_end - drv_data->rx;
switch (drv_data->n_bytes) {
case 4:
- bytes_left >>= 1;
+ bytes_left >>= 2;
+ break;
case 2:
bytes_left >>= 1;
+ break;
}
rx_thre = pxa2xx_spi_get_rx_default_thre(drv_data);
kfree(chip);
}
-#ifdef CONFIG_PCI
-#ifdef CONFIG_ACPI
-
static const struct acpi_device_id pxa2xx_spi_acpi_match[] = {
{ "INT33C0", LPSS_LPT_SSP },
{ "INT33C1", LPSS_LPT_SSP },
};
MODULE_DEVICE_TABLE(acpi, pxa2xx_spi_acpi_match);
-static int pxa2xx_spi_get_port_id(struct acpi_device *adev)
-{
- unsigned int devid;
- int port_id = -1;
-
- if (adev && adev->pnp.unique_id &&
- !kstrtouint(adev->pnp.unique_id, 0, &devid))
- port_id = devid;
- return port_id;
-}
-#else /* !CONFIG_ACPI */
-static int pxa2xx_spi_get_port_id(struct acpi_device *adev)
-{
- return -1;
-}
-#endif
-
/*
* PCI IDs of compound devices that integrate both host controller and private
* integrated DMA engine. Please note these are not used in module
{ },
};
+static const struct of_device_id pxa2xx_spi_of_match[] = {
+ { .compatible = "marvell,mmp2-ssp", .data = (void *)MMP2_SSP },
+ {},
+};
+MODULE_DEVICE_TABLE(of, pxa2xx_spi_of_match);
+
+#ifdef CONFIG_ACPI
+
+static int pxa2xx_spi_get_port_id(struct acpi_device *adev)
+{
+ unsigned int devid;
+ int port_id = -1;
+
+ if (adev && adev->pnp.unique_id &&
+ !kstrtouint(adev->pnp.unique_id, 0, &devid))
+ port_id = devid;
+ return port_id;
+}
+
+#else /* !CONFIG_ACPI */
+
+static int pxa2xx_spi_get_port_id(struct acpi_device *adev)
+{
+ return -1;
+}
+
+#endif /* CONFIG_ACPI */
+
+
+#ifdef CONFIG_PCI
+
static bool pxa2xx_spi_idma_filter(struct dma_chan *chan, void *param)
{
struct device *dev = param;
return true;
}
+#endif /* CONFIG_PCI */
+
static struct pxa2xx_spi_master *
pxa2xx_spi_init_pdata(struct platform_device *pdev)
{
struct resource *res;
const struct acpi_device_id *adev_id = NULL;
const struct pci_device_id *pcidev_id = NULL;
- int type;
+ const struct of_device_id *of_id = NULL;
+ enum pxa_ssp_type type;
adev = ACPI_COMPANION(&pdev->dev);
- if (dev_is_pci(pdev->dev.parent))
+ if (pdev->dev.of_node)
+ of_id = of_match_device(pdev->dev.driver->of_match_table,
+ &pdev->dev);
+ else if (dev_is_pci(pdev->dev.parent))
pcidev_id = pci_match_id(pxa2xx_spi_pci_compound_match,
to_pci_dev(pdev->dev.parent));
else if (adev)
return NULL;
if (adev_id)
- type = (int)adev_id->driver_data;
+ type = (enum pxa_ssp_type)adev_id->driver_data;
else if (pcidev_id)
- type = (int)pcidev_id->driver_data;
+ type = (enum pxa_ssp_type)pcidev_id->driver_data;
+ else if (of_id)
+ type = (enum pxa_ssp_type)of_id->data;
else
return NULL;
if (IS_ERR(ssp->mmio_base))
return NULL;
+#ifdef CONFIG_PCI
if (pcidev_id) {
pdata->tx_param = pdev->dev.parent;
pdata->rx_param = pdev->dev.parent;
pdata->dma_filter = pxa2xx_spi_idma_filter;
}
+#endif
ssp->clk = devm_clk_get(&pdev->dev, NULL);
ssp->irq = platform_get_irq(pdev, 0);
return pdata;
}
-#else /* !CONFIG_PCI */
-static inline struct pxa2xx_spi_master *
-pxa2xx_spi_init_pdata(struct platform_device *pdev)
-{
- return NULL;
-}
-#endif
-
static int pxa2xx_spi_fw_translate_cs(struct spi_controller *master,
unsigned int cs)
{
return 0;
}
-static void pxa2xx_spi_shutdown(struct platform_device *pdev)
-{
- int status = 0;
-
- if ((status = pxa2xx_spi_remove(pdev)) != 0)
- dev_err(&pdev->dev, "shutdown failed with %d\n", status);
-}
-
#ifdef CONFIG_PM_SLEEP
static int pxa2xx_spi_suspend(struct device *dev)
{
lpss_ssp_setup(drv_data);
/* Start the queue running */
- status = spi_controller_resume(drv_data->master);
- if (status != 0) {
- dev_err(dev, "problem starting queue (%d)\n", status);
- return status;
- }
-
- return 0;
+ return spi_controller_resume(drv_data->master);
}
#endif
.name = "pxa2xx-spi",
.pm = &pxa2xx_spi_pm_ops,
.acpi_match_table = ACPI_PTR(pxa2xx_spi_acpi_match),
+ .of_match_table = of_match_ptr(pxa2xx_spi_of_match),
},
.probe = pxa2xx_spi_probe,
.remove = pxa2xx_spi_remove,
- .shutdown = pxa2xx_spi_shutdown,
};
static int __init pxa2xx_spi_init(void)
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2017-2018, The Linux foundation. All rights reserved.
+
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/pm_runtime.h>
+#include <linux/spi/spi.h>
+#include <linux/spi/spi-mem.h>
+
+
+#define QSPI_NUM_CS 2
+#define QSPI_BYTES_PER_WORD 4
+
+#define MSTR_CONFIG 0x0000
+#define FULL_CYCLE_MODE BIT(3)
+#define FB_CLK_EN BIT(4)
+#define PIN_HOLDN BIT(6)
+#define PIN_WPN BIT(7)
+#define DMA_ENABLE BIT(8)
+#define BIG_ENDIAN_MODE BIT(9)
+#define SPI_MODE_MSK 0xc00
+#define SPI_MODE_SHFT 10
+#define CHIP_SELECT_NUM BIT(12)
+#define SBL_EN BIT(13)
+#define LPA_BASE_MSK 0x3c000
+#define LPA_BASE_SHFT 14
+#define TX_DATA_DELAY_MSK 0xc0000
+#define TX_DATA_DELAY_SHFT 18
+#define TX_CLK_DELAY_MSK 0x300000
+#define TX_CLK_DELAY_SHFT 20
+#define TX_CS_N_DELAY_MSK 0xc00000
+#define TX_CS_N_DELAY_SHFT 22
+#define TX_DATA_OE_DELAY_MSK 0x3000000
+#define TX_DATA_OE_DELAY_SHFT 24
+
+#define AHB_MASTER_CFG 0x0004
+#define HMEM_TYPE_START_MID_TRANS_MSK 0x7
+#define HMEM_TYPE_START_MID_TRANS_SHFT 0
+#define HMEM_TYPE_LAST_TRANS_MSK 0x38
+#define HMEM_TYPE_LAST_TRANS_SHFT 3
+#define USE_HMEMTYPE_LAST_ON_DESC_OR_CHAIN_MSK 0xc0
+#define USE_HMEMTYPE_LAST_ON_DESC_OR_CHAIN_SHFT 6
+#define HMEMTYPE_READ_TRANS_MSK 0x700
+#define HMEMTYPE_READ_TRANS_SHFT 8
+#define HSHARED BIT(11)
+#define HINNERSHARED BIT(12)
+
+#define MSTR_INT_EN 0x000C
+#define MSTR_INT_STATUS 0x0010
+#define RESP_FIFO_UNDERRUN BIT(0)
+#define RESP_FIFO_NOT_EMPTY BIT(1)
+#define RESP_FIFO_RDY BIT(2)
+#define HRESP_FROM_NOC_ERR BIT(3)
+#define WR_FIFO_EMPTY BIT(9)
+#define WR_FIFO_FULL BIT(10)
+#define WR_FIFO_OVERRUN BIT(11)
+#define TRANSACTION_DONE BIT(16)
+#define QSPI_ERR_IRQS (RESP_FIFO_UNDERRUN | HRESP_FROM_NOC_ERR | \
+ WR_FIFO_OVERRUN)
+#define QSPI_ALL_IRQS (QSPI_ERR_IRQS | RESP_FIFO_RDY | \
+ WR_FIFO_EMPTY | WR_FIFO_FULL | \
+ TRANSACTION_DONE)
+
+#define PIO_XFER_CTRL 0x0014
+#define REQUEST_COUNT_MSK 0xffff
+
+#define PIO_XFER_CFG 0x0018
+#define TRANSFER_DIRECTION BIT(0)
+#define MULTI_IO_MODE_MSK 0xe
+#define MULTI_IO_MODE_SHFT 1
+#define TRANSFER_FRAGMENT BIT(8)
+#define SDR_1BIT 1
+#define SDR_2BIT 2
+#define SDR_4BIT 3
+#define DDR_1BIT 5
+#define DDR_2BIT 6
+#define DDR_4BIT 7
+#define DMA_DESC_SINGLE_SPI 1
+#define DMA_DESC_DUAL_SPI 2
+#define DMA_DESC_QUAD_SPI 3
+
+#define PIO_XFER_STATUS 0x001c
+#define WR_FIFO_BYTES_MSK 0xffff0000
+#define WR_FIFO_BYTES_SHFT 16
+
+#define PIO_DATAOUT_1B 0x0020
+#define PIO_DATAOUT_4B 0x0024
+
+#define RD_FIFO_STATUS 0x002c
+#define FIFO_EMPTY BIT(11)
+#define WR_CNTS_MSK 0x7f0
+#define WR_CNTS_SHFT 4
+#define RDY_64BYTE BIT(3)
+#define RDY_32BYTE BIT(2)
+#define RDY_16BYTE BIT(1)
+#define FIFO_RDY BIT(0)
+
+#define RD_FIFO_CFG 0x0028
+#define CONTINUOUS_MODE BIT(0)
+
+#define RD_FIFO_RESET 0x0030
+#define RESET_FIFO BIT(0)
+
+#define CUR_MEM_ADDR 0x0048
+#define HW_VERSION 0x004c
+#define RD_FIFO 0x0050
+#define SAMPLING_CLK_CFG 0x0090
+#define SAMPLING_CLK_STATUS 0x0094
+
+
+enum qspi_dir {
+ QSPI_READ,
+ QSPI_WRITE,
+};
+
+struct qspi_xfer {
+ union {
+ const void *tx_buf;
+ void *rx_buf;
+ };
+ unsigned int rem_bytes;
+ unsigned int buswidth;
+ enum qspi_dir dir;
+ bool is_last;
+};
+
+enum qspi_clocks {
+ QSPI_CLK_CORE,
+ QSPI_CLK_IFACE,
+ QSPI_NUM_CLKS
+};
+
+struct qcom_qspi {
+ void __iomem *base;
+ struct device *dev;
+ struct clk_bulk_data clks[QSPI_NUM_CLKS];
+ struct qspi_xfer xfer;
+ /* Lock to protect data accessed by IRQs */
+ spinlock_t lock;
+};
+
+static u32 qspi_buswidth_to_iomode(struct qcom_qspi *ctrl,
+ unsigned int buswidth)
+{
+ switch (buswidth) {
+ case 1:
+ return SDR_1BIT << MULTI_IO_MODE_SHFT;
+ case 2:
+ return SDR_2BIT << MULTI_IO_MODE_SHFT;
+ case 4:
+ return SDR_4BIT << MULTI_IO_MODE_SHFT;
+ default:
+ dev_warn_once(ctrl->dev,
+ "Unexpected bus width: %u\n", buswidth);
+ return SDR_1BIT << MULTI_IO_MODE_SHFT;
+ }
+}
+
+static void qcom_qspi_pio_xfer_cfg(struct qcom_qspi *ctrl)
+{
+ u32 pio_xfer_cfg;
+ const struct qspi_xfer *xfer;
+
+ xfer = &ctrl->xfer;
+ pio_xfer_cfg = readl(ctrl->base + PIO_XFER_CFG);
+ pio_xfer_cfg &= ~TRANSFER_DIRECTION;
+ pio_xfer_cfg |= xfer->dir;
+ if (xfer->is_last)
+ pio_xfer_cfg &= ~TRANSFER_FRAGMENT;
+ else
+ pio_xfer_cfg |= TRANSFER_FRAGMENT;
+ pio_xfer_cfg &= ~MULTI_IO_MODE_MSK;
+ pio_xfer_cfg |= qspi_buswidth_to_iomode(ctrl, xfer->buswidth);
+
+ writel(pio_xfer_cfg, ctrl->base + PIO_XFER_CFG);
+}
+
+static void qcom_qspi_pio_xfer_ctrl(struct qcom_qspi *ctrl)
+{
+ u32 pio_xfer_ctrl;
+
+ pio_xfer_ctrl = readl(ctrl->base + PIO_XFER_CTRL);
+ pio_xfer_ctrl &= ~REQUEST_COUNT_MSK;
+ pio_xfer_ctrl |= ctrl->xfer.rem_bytes;
+ writel(pio_xfer_ctrl, ctrl->base + PIO_XFER_CTRL);
+}
+
+static void qcom_qspi_pio_xfer(struct qcom_qspi *ctrl)
+{
+ u32 ints;
+
+ qcom_qspi_pio_xfer_cfg(ctrl);
+
+ /* Ack any previous interrupts that might be hanging around */
+ writel(QSPI_ALL_IRQS, ctrl->base + MSTR_INT_STATUS);
+
+ /* Setup new interrupts */
+ if (ctrl->xfer.dir == QSPI_WRITE)
+ ints = QSPI_ERR_IRQS | WR_FIFO_EMPTY;
+ else
+ ints = QSPI_ERR_IRQS | RESP_FIFO_RDY;
+ writel(ints, ctrl->base + MSTR_INT_EN);
+
+ /* Kick off the transfer */
+ qcom_qspi_pio_xfer_ctrl(ctrl);
+}
+
+static void qcom_qspi_handle_err(struct spi_master *master,
+ struct spi_message *msg)
+{
+ struct qcom_qspi *ctrl = spi_master_get_devdata(master);
+ unsigned long flags;
+
+ spin_lock_irqsave(&ctrl->lock, flags);
+ writel(0, ctrl->base + MSTR_INT_EN);
+ ctrl->xfer.rem_bytes = 0;
+ spin_unlock_irqrestore(&ctrl->lock, flags);
+}
+
+static int qcom_qspi_transfer_one(struct spi_master *master,
+ struct spi_device *slv,
+ struct spi_transfer *xfer)
+{
+ struct qcom_qspi *ctrl = spi_master_get_devdata(master);
+ int ret;
+ unsigned long speed_hz;
+ unsigned long flags;
+
+ speed_hz = slv->max_speed_hz;
+ if (xfer->speed_hz)
+ speed_hz = xfer->speed_hz;
+
+ /* In regular operation (SBL_EN=1) core must be 4x transfer clock */
+ ret = clk_set_rate(ctrl->clks[QSPI_CLK_CORE].clk, speed_hz * 4);
+ if (ret) {
+ dev_err(ctrl->dev, "Failed to set core clk %d\n", ret);
+ return ret;
+ }
+
+ spin_lock_irqsave(&ctrl->lock, flags);
+
+ /* We are half duplex, so either rx or tx will be set */
+ if (xfer->rx_buf) {
+ ctrl->xfer.dir = QSPI_READ;
+ ctrl->xfer.buswidth = xfer->rx_nbits;
+ ctrl->xfer.rx_buf = xfer->rx_buf;
+ } else {
+ ctrl->xfer.dir = QSPI_WRITE;
+ ctrl->xfer.buswidth = xfer->tx_nbits;
+ ctrl->xfer.tx_buf = xfer->tx_buf;
+ }
+ ctrl->xfer.is_last = list_is_last(&xfer->transfer_list,
+ &master->cur_msg->transfers);
+ ctrl->xfer.rem_bytes = xfer->len;
+ qcom_qspi_pio_xfer(ctrl);
+
+ spin_unlock_irqrestore(&ctrl->lock, flags);
+
+ /* We'll call spi_finalize_current_transfer() when done */
+ return 1;
+}
+
+static int qcom_qspi_prepare_message(struct spi_master *master,
+ struct spi_message *message)
+{
+ u32 mstr_cfg;
+ struct qcom_qspi *ctrl;
+ int tx_data_oe_delay = 1;
+ int tx_data_delay = 1;
+ unsigned long flags;
+
+ ctrl = spi_master_get_devdata(master);
+ spin_lock_irqsave(&ctrl->lock, flags);
+
+ mstr_cfg = readl(ctrl->base + MSTR_CONFIG);
+ mstr_cfg &= ~CHIP_SELECT_NUM;
+ if (message->spi->chip_select)
+ mstr_cfg |= CHIP_SELECT_NUM;
+
+ mstr_cfg |= FB_CLK_EN | PIN_WPN | PIN_HOLDN | SBL_EN | FULL_CYCLE_MODE;
+ mstr_cfg &= ~(SPI_MODE_MSK | TX_DATA_OE_DELAY_MSK | TX_DATA_DELAY_MSK);
+ mstr_cfg |= message->spi->mode << SPI_MODE_SHFT;
+ mstr_cfg |= tx_data_oe_delay << TX_DATA_OE_DELAY_SHFT;
+ mstr_cfg |= tx_data_delay << TX_DATA_DELAY_SHFT;
+ mstr_cfg &= ~DMA_ENABLE;
+
+ writel(mstr_cfg, ctrl->base + MSTR_CONFIG);
+ spin_unlock_irqrestore(&ctrl->lock, flags);
+
+ return 0;
+}
+
+static irqreturn_t pio_read(struct qcom_qspi *ctrl)
+{
+ u32 rd_fifo_status;
+ u32 rd_fifo;
+ unsigned int wr_cnts;
+ unsigned int bytes_to_read;
+ unsigned int words_to_read;
+ u32 *word_buf;
+ u8 *byte_buf;
+ int i;
+
+ rd_fifo_status = readl(ctrl->base + RD_FIFO_STATUS);
+
+ if (!(rd_fifo_status & FIFO_RDY)) {
+ dev_dbg(ctrl->dev, "Spurious IRQ %#x\n", rd_fifo_status);
+ return IRQ_NONE;
+ }
+
+ wr_cnts = (rd_fifo_status & WR_CNTS_MSK) >> WR_CNTS_SHFT;
+ wr_cnts = min(wr_cnts, ctrl->xfer.rem_bytes);
+
+ words_to_read = wr_cnts / QSPI_BYTES_PER_WORD;
+ bytes_to_read = wr_cnts % QSPI_BYTES_PER_WORD;
+
+ if (words_to_read) {
+ word_buf = ctrl->xfer.rx_buf;
+ ctrl->xfer.rem_bytes -= words_to_read * QSPI_BYTES_PER_WORD;
+ ioread32_rep(ctrl->base + RD_FIFO, word_buf, words_to_read);
+ ctrl->xfer.rx_buf = word_buf + words_to_read;
+ }
+
+ if (bytes_to_read) {
+ byte_buf = ctrl->xfer.rx_buf;
+ rd_fifo = readl(ctrl->base + RD_FIFO);
+ ctrl->xfer.rem_bytes -= bytes_to_read;
+ for (i = 0; i < bytes_to_read; i++)
+ *byte_buf++ = rd_fifo >> (i * BITS_PER_BYTE);
+ ctrl->xfer.rx_buf = byte_buf;
+ }
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t pio_write(struct qcom_qspi *ctrl)
+{
+ const void *xfer_buf = ctrl->xfer.tx_buf;
+ const int *word_buf;
+ const char *byte_buf;
+ unsigned int wr_fifo_bytes;
+ unsigned int wr_fifo_words;
+ unsigned int wr_size;
+ unsigned int rem_words;
+
+ wr_fifo_bytes = readl(ctrl->base + PIO_XFER_STATUS);
+ wr_fifo_bytes >>= WR_FIFO_BYTES_SHFT;
+
+ if (ctrl->xfer.rem_bytes < QSPI_BYTES_PER_WORD) {
+ /* Process the last 1-3 bytes */
+ wr_size = min(wr_fifo_bytes, ctrl->xfer.rem_bytes);
+ ctrl->xfer.rem_bytes -= wr_size;
+
+ byte_buf = xfer_buf;
+ while (wr_size--)
+ writel(*byte_buf++,
+ ctrl->base + PIO_DATAOUT_1B);
+ ctrl->xfer.tx_buf = byte_buf;
+ } else {
+ /*
+ * Process all the whole words; to keep things simple we'll
+ * just wait for the next interrupt to handle the last 1-3
+ * bytes if we don't have an even number of words.
+ */
+ rem_words = ctrl->xfer.rem_bytes / QSPI_BYTES_PER_WORD;
+ wr_fifo_words = wr_fifo_bytes / QSPI_BYTES_PER_WORD;
+
+ wr_size = min(rem_words, wr_fifo_words);
+ ctrl->xfer.rem_bytes -= wr_size * QSPI_BYTES_PER_WORD;
+
+ word_buf = xfer_buf;
+ iowrite32_rep(ctrl->base + PIO_DATAOUT_4B, word_buf, wr_size);
+ ctrl->xfer.tx_buf = word_buf + wr_size;
+
+ }
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t qcom_qspi_irq(int irq, void *dev_id)
+{
+ u32 int_status;
+ struct qcom_qspi *ctrl = dev_id;
+ irqreturn_t ret = IRQ_NONE;
+ unsigned long flags;
+
+ spin_lock_irqsave(&ctrl->lock, flags);
+
+ int_status = readl(ctrl->base + MSTR_INT_STATUS);
+ writel(int_status, ctrl->base + MSTR_INT_STATUS);
+
+ if (ctrl->xfer.dir == QSPI_WRITE) {
+ if (int_status & WR_FIFO_EMPTY)
+ ret = pio_write(ctrl);
+ } else {
+ if (int_status & RESP_FIFO_RDY)
+ ret = pio_read(ctrl);
+ }
+
+ if (int_status & QSPI_ERR_IRQS) {
+ if (int_status & RESP_FIFO_UNDERRUN)
+ dev_err(ctrl->dev, "IRQ error: FIFO underrun\n");
+ if (int_status & WR_FIFO_OVERRUN)
+ dev_err(ctrl->dev, "IRQ error: FIFO overrun\n");
+ if (int_status & HRESP_FROM_NOC_ERR)
+ dev_err(ctrl->dev, "IRQ error: NOC response error\n");
+ ret = IRQ_HANDLED;
+ }
+
+ if (!ctrl->xfer.rem_bytes) {
+ writel(0, ctrl->base + MSTR_INT_EN);
+ spi_finalize_current_transfer(dev_get_drvdata(ctrl->dev));
+ }
+
+ spin_unlock_irqrestore(&ctrl->lock, flags);
+ return ret;
+}
+
+static int qcom_qspi_probe(struct platform_device *pdev)
+{
+ int ret;
+ struct device *dev;
+ struct resource *res;
+ struct spi_master *master;
+ struct qcom_qspi *ctrl;
+
+ dev = &pdev->dev;
+
+ master = spi_alloc_master(dev, sizeof(*ctrl));
+ if (!master)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, master);
+
+ ctrl = spi_master_get_devdata(master);
+
+ spin_lock_init(&ctrl->lock);
+ ctrl->dev = dev;
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ ctrl->base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(ctrl->base)) {
+ ret = PTR_ERR(ctrl->base);
+ goto exit_probe_master_put;
+ }
+
+ ctrl->clks[QSPI_CLK_CORE].id = "core";
+ ctrl->clks[QSPI_CLK_IFACE].id = "iface";
+ ret = devm_clk_bulk_get(dev, QSPI_NUM_CLKS, ctrl->clks);
+ if (ret)
+ goto exit_probe_master_put;
+
+ ret = platform_get_irq(pdev, 0);
+ if (ret < 0) {
+ dev_err(dev, "Failed to get irq %d\n", ret);
+ goto exit_probe_master_put;
+ }
+ ret = devm_request_irq(dev, ret, qcom_qspi_irq,
+ IRQF_TRIGGER_HIGH, dev_name(dev), ctrl);
+ if (ret) {
+ dev_err(dev, "Failed to request irq %d\n", ret);
+ goto exit_probe_master_put;
+ }
+
+ master->max_speed_hz = 300000000;
+ master->num_chipselect = QSPI_NUM_CS;
+ master->bus_num = -1;
+ master->dev.of_node = pdev->dev.of_node;
+ master->mode_bits = SPI_MODE_0 |
+ SPI_TX_DUAL | SPI_RX_DUAL |
+ SPI_TX_QUAD | SPI_RX_QUAD;
+ master->flags = SPI_MASTER_HALF_DUPLEX;
+ master->prepare_message = qcom_qspi_prepare_message;
+ master->transfer_one = qcom_qspi_transfer_one;
+ master->handle_err = qcom_qspi_handle_err;
+ master->auto_runtime_pm = true;
+
+ pm_runtime_enable(dev);
+
+ ret = spi_register_master(master);
+ if (!ret)
+ return 0;
+
+ pm_runtime_disable(dev);
+
+exit_probe_master_put:
+ spi_master_put(master);
+
+ return ret;
+}
+
+static int qcom_qspi_remove(struct platform_device *pdev)
+{
+ struct spi_master *master = platform_get_drvdata(pdev);
+
+ /* Unregister _before_ disabling pm_runtime() so we stop transfers */
+ spi_unregister_master(master);
+
+ pm_runtime_disable(&pdev->dev);
+
+ return 0;
+}
+
+static int __maybe_unused qcom_qspi_runtime_suspend(struct device *dev)
+{
+ struct spi_master *master = dev_get_drvdata(dev);
+ struct qcom_qspi *ctrl = spi_master_get_devdata(master);
+
+ clk_bulk_disable_unprepare(QSPI_NUM_CLKS, ctrl->clks);
+
+ return 0;
+}
+
+static int __maybe_unused qcom_qspi_runtime_resume(struct device *dev)
+{
+ struct spi_master *master = dev_get_drvdata(dev);
+ struct qcom_qspi *ctrl = spi_master_get_devdata(master);
+
+ return clk_bulk_prepare_enable(QSPI_NUM_CLKS, ctrl->clks);
+}
+
+static int __maybe_unused qcom_qspi_suspend(struct device *dev)
+{
+ struct spi_master *master = dev_get_drvdata(dev);
+ int ret;
+
+ ret = spi_master_suspend(master);
+ if (ret)
+ return ret;
+
+ ret = pm_runtime_force_suspend(dev);
+ if (ret)
+ spi_master_resume(master);
+
+ return ret;
+}
+
+static int __maybe_unused qcom_qspi_resume(struct device *dev)
+{
+ struct spi_master *master = dev_get_drvdata(dev);
+ int ret;
+
+ ret = pm_runtime_force_resume(dev);
+ if (ret)
+ return ret;
+
+ ret = spi_master_resume(master);
+ if (ret)
+ pm_runtime_force_suspend(dev);
+
+ return ret;
+}
+
+static const struct dev_pm_ops qcom_qspi_dev_pm_ops = {
+ SET_RUNTIME_PM_OPS(qcom_qspi_runtime_suspend,
+ qcom_qspi_runtime_resume, NULL)
+ SET_SYSTEM_SLEEP_PM_OPS(qcom_qspi_suspend, qcom_qspi_resume)
+};
+
+static const struct of_device_id qcom_qspi_dt_match[] = {
+ { .compatible = "qcom,qspi-v1", },
+ { }
+};
+MODULE_DEVICE_TABLE(of, qcom_qspi_dt_match);
+
+static struct platform_driver qcom_qspi_driver = {
+ .driver = {
+ .name = "qcom_qspi",
+ .pm = &qcom_qspi_dev_pm_ops,
+ .of_match_table = qcom_qspi_dt_match,
+ },
+ .probe = qcom_qspi_probe,
+ .remove = qcom_qspi_remove,
+};
+module_platform_driver(qcom_qspi_driver);
+
+MODULE_DESCRIPTION("SPI driver for QSPI cores");
+MODULE_LICENSE("GPL v2");
master->bus_num = 0;
master->num_chipselect = 3;
master->mode_bits = SPI_TX_DUAL;
- master->bits_per_word_mask = BIT(7);
+ master->bits_per_word_mask = SPI_BPW_MASK(8);
master->flags = SPI_MASTER_MUST_TX;
master->transfer_one = rb4xx_transfer_one;
master->set_cs = rb4xx_set_cs;
struct rockchip_spi_dma_data {
struct dma_chan *ch;
- enum dma_transfer_direction direction;
dma_addr_t addr;
};
bool cs_asserted[ROCKCHIP_SPI_MAX_CS_NUM];
- u32 use_dma;
+ bool use_dma;
struct sg_table tx_sg;
struct sg_table rx_sg;
struct rockchip_spi_dma_data dma_rx;
struct rockchip_spi_dma_data dma_tx;
- struct dma_slave_caps dma_caps;
};
static inline void spi_enable_chip(struct rockchip_spi *rs, int enable)
{
int remain = 0;
+ spi_enable_chip(rs, 1);
+
do {
if (rs->tx) {
remain = rs->tx_end - rs->tx;
struct dma_slave_config rxconf, txconf;
struct dma_async_tx_descriptor *rxdesc, *txdesc;
+ memset(&rxconf, 0, sizeof(rxconf));
+ memset(&txconf, 0, sizeof(txconf));
+
spin_lock_irqsave(&rs->lock, flags);
rs->state &= ~RXBUSY;
rs->state &= ~TXBUSY;
rxdesc = NULL;
if (rs->rx) {
- rxconf.direction = rs->dma_rx.direction;
+ rxconf.direction = DMA_DEV_TO_MEM;
rxconf.src_addr = rs->dma_rx.addr;
rxconf.src_addr_width = rs->n_bytes;
- if (rs->dma_caps.max_burst > 4)
- rxconf.src_maxburst = 4;
- else
- rxconf.src_maxburst = 1;
+ rxconf.src_maxburst = 1;
dmaengine_slave_config(rs->dma_rx.ch, &rxconf);
rxdesc = dmaengine_prep_slave_sg(
rs->dma_rx.ch,
rs->rx_sg.sgl, rs->rx_sg.nents,
- rs->dma_rx.direction, DMA_PREP_INTERRUPT);
+ DMA_DEV_TO_MEM, DMA_PREP_INTERRUPT);
if (!rxdesc)
return -EINVAL;
txdesc = NULL;
if (rs->tx) {
- txconf.direction = rs->dma_tx.direction;
+ txconf.direction = DMA_MEM_TO_DEV;
txconf.dst_addr = rs->dma_tx.addr;
txconf.dst_addr_width = rs->n_bytes;
- if (rs->dma_caps.max_burst > 4)
- txconf.dst_maxburst = 4;
- else
- txconf.dst_maxburst = 1;
+ txconf.dst_maxburst = rs->fifo_len / 2;
dmaengine_slave_config(rs->dma_tx.ch, &txconf);
txdesc = dmaengine_prep_slave_sg(
rs->dma_tx.ch,
rs->tx_sg.sgl, rs->tx_sg.nents,
- rs->dma_tx.direction, DMA_PREP_INTERRUPT);
+ DMA_MEM_TO_DEV, DMA_PREP_INTERRUPT);
if (!txdesc) {
if (rxdesc)
dmaengine_terminate_sync(rs->dma_rx.ch);
dma_async_issue_pending(rs->dma_rx.ch);
}
+ spi_enable_chip(rs, 1);
+
if (txdesc) {
spin_lock_irqsave(&rs->lock, flags);
rs->state |= TXBUSY;
dma_async_issue_pending(rs->dma_tx.ch);
}
- return 0;
+ /* 1 means the transfer is in progress */
+ return 1;
}
static void rockchip_spi_config(struct rockchip_spi *rs)
writel_relaxed(rs->fifo_len / 2 - 1, rs->regs + ROCKCHIP_SPI_TXFTLR);
writel_relaxed(rs->fifo_len / 2 - 1, rs->regs + ROCKCHIP_SPI_RXFTLR);
- writel_relaxed(0, rs->regs + ROCKCHIP_SPI_DMATDLR);
+ writel_relaxed(rs->fifo_len / 2 - 1, rs->regs + ROCKCHIP_SPI_DMATDLR);
writel_relaxed(0, rs->regs + ROCKCHIP_SPI_DMARDLR);
writel_relaxed(dmacr, rs->regs + ROCKCHIP_SPI_DMACR);
struct spi_device *spi,
struct spi_transfer *xfer)
{
- int ret = 0;
struct rockchip_spi *rs = spi_master_get_devdata(master);
WARN_ON(readl_relaxed(rs->regs + ROCKCHIP_SPI_SSIENR) &&
/* we need prepare dma before spi was enabled */
if (master->can_dma && master->can_dma(master, spi, xfer))
- rs->use_dma = 1;
+ rs->use_dma = true;
else
- rs->use_dma = 0;
+ rs->use_dma = false;
rockchip_spi_config(rs);
- if (rs->use_dma) {
- if (rs->tmode == CR0_XFM_RO) {
- /* rx: dma must be prepared first */
- ret = rockchip_spi_prepare_dma(rs);
- spi_enable_chip(rs, 1);
- } else {
- /* tx or tr: spi must be enabled first */
- spi_enable_chip(rs, 1);
- ret = rockchip_spi_prepare_dma(rs);
- }
- /* successful DMA prepare means the transfer is in progress */
- ret = ret ? ret : 1;
- } else {
- spi_enable_chip(rs, 1);
- ret = rockchip_spi_pio_transfer(rs);
- }
+ if (rs->use_dma)
+ return rockchip_spi_prepare_dma(rs);
- return ret;
+ return rockchip_spi_pio_transfer(rs);
}
static bool rockchip_spi_can_dma(struct spi_master *master,
}
if (rs->dma_tx.ch && rs->dma_rx.ch) {
- dma_get_slave_caps(rs->dma_rx.ch, &(rs->dma_caps));
rs->dma_tx.addr = (dma_addr_t)(mem->start + ROCKCHIP_SPI_TXDR);
rs->dma_rx.addr = (dma_addr_t)(mem->start + ROCKCHIP_SPI_RXDR);
- rs->dma_tx.direction = DMA_MEM_TO_DEV;
- rs->dma_rx.direction = DMA_DEV_TO_MEM;
master->can_dma = rockchip_spi_can_dma;
master->dma_tx = rs->dma_tx.ch;
+// SPDX-License-Identifier: GPL-2.0
/*
* SH RSPI driver
*
*
* Based on spi-sh.c:
* Copyright (C) 2011 Renesas Solutions Corp.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
#include <linux/module.h>
+// SPDX-License-Identifier: GPL-2.0
/*
* SuperH HSPI bus driver
*
* Based on pxa2xx_spi.c:
* Copyright (C) 2011 Renesas Solutions Corp.
* Copyright (C) 2005 Stephen Street / StreetFire Sound Labs
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
#include <linux/clk.h>
module_platform_driver(hspi_driver);
MODULE_DESCRIPTION("SuperH HSPI bus driver");
-MODULE_LICENSE("GPL");
+MODULE_LICENSE("GPL v2");
MODULE_AUTHOR("Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>");
MODULE_ALIAS("platform:sh-hspi");
+// SPDX-License-Identifier: GPL-2.0
/*
* SuperH MSIOF SPI Master Interface
*
* Copyright (c) 2009 Magnus Damm
* Copyright (C) 2014 Renesas Electronics Corporation
* Copyright (C) 2014-2017 Glider bvba
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
*/
#include <linux/bitmap.h>
i = platform_get_irq(pdev, 0);
if (i < 0) {
- dev_err(&pdev->dev, "cannot get platform IRQ\n");
- ret = -ENOENT;
+ dev_err(&pdev->dev, "cannot get IRQ\n");
+ ret = i;
goto err1;
}
+// SPDX-License-Identifier: GPL-2.0
/*
* SH SPI bus driver
*
*
* Based on pxa2xx_spi.c:
* Copyright (C) 2005 Stephen Street / StreetFire Sound Labs
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
#include <linux/module.h>
module_platform_driver(spi_sh_driver);
MODULE_DESCRIPTION("SH SPI bus driver");
-MODULE_LICENSE("GPL");
+MODULE_LICENSE("GPL v2");
MODULE_AUTHOR("Yoshihiro Shimoda");
MODULE_ALIAS("platform:sh_spi");
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright (c) 2018 MediaTek Inc.
+
+#include <linux/clk.h>
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/spi/spi.h>
+
+#define SPIS_IRQ_EN_REG 0x0
+#define SPIS_IRQ_CLR_REG 0x4
+#define SPIS_IRQ_ST_REG 0x8
+#define SPIS_IRQ_MASK_REG 0xc
+#define SPIS_CFG_REG 0x10
+#define SPIS_RX_DATA_REG 0x14
+#define SPIS_TX_DATA_REG 0x18
+#define SPIS_RX_DST_REG 0x1c
+#define SPIS_TX_SRC_REG 0x20
+#define SPIS_DMA_CFG_REG 0x30
+#define SPIS_SOFT_RST_REG 0x40
+
+/* SPIS_IRQ_EN_REG */
+#define DMA_DONE_EN BIT(7)
+#define DATA_DONE_EN BIT(2)
+#define RSTA_DONE_EN BIT(1)
+#define CMD_INVALID_EN BIT(0)
+
+/* SPIS_IRQ_ST_REG */
+#define DMA_DONE_ST BIT(7)
+#define DATA_DONE_ST BIT(2)
+#define RSTA_DONE_ST BIT(1)
+#define CMD_INVALID_ST BIT(0)
+
+/* SPIS_IRQ_MASK_REG */
+#define DMA_DONE_MASK BIT(7)
+#define DATA_DONE_MASK BIT(2)
+#define RSTA_DONE_MASK BIT(1)
+#define CMD_INVALID_MASK BIT(0)
+
+/* SPIS_CFG_REG */
+#define SPIS_TX_ENDIAN BIT(7)
+#define SPIS_RX_ENDIAN BIT(6)
+#define SPIS_TXMSBF BIT(5)
+#define SPIS_RXMSBF BIT(4)
+#define SPIS_CPHA BIT(3)
+#define SPIS_CPOL BIT(2)
+#define SPIS_TX_EN BIT(1)
+#define SPIS_RX_EN BIT(0)
+
+/* SPIS_DMA_CFG_REG */
+#define TX_DMA_TRIG_EN BIT(31)
+#define TX_DMA_EN BIT(30)
+#define RX_DMA_EN BIT(29)
+#define TX_DMA_LEN 0xfffff
+
+/* SPIS_SOFT_RST_REG */
+#define SPIS_DMA_ADDR_EN BIT(1)
+#define SPIS_SOFT_RST BIT(0)
+
+#define MTK_SPI_SLAVE_MAX_FIFO_SIZE 512U
+
+struct mtk_spi_slave {
+ struct device *dev;
+ void __iomem *base;
+ struct clk *spi_clk;
+ struct completion xfer_done;
+ struct spi_transfer *cur_transfer;
+ bool slave_aborted;
+};
+
+static const struct of_device_id mtk_spi_slave_of_match[] = {
+ { .compatible = "mediatek,mt2712-spi-slave", },
+ {}
+};
+MODULE_DEVICE_TABLE(of, mtk_spi_slave_of_match);
+
+static void mtk_spi_slave_disable_dma(struct mtk_spi_slave *mdata)
+{
+ u32 reg_val;
+
+ reg_val = readl(mdata->base + SPIS_DMA_CFG_REG);
+ reg_val &= ~RX_DMA_EN;
+ reg_val &= ~TX_DMA_EN;
+ writel(reg_val, mdata->base + SPIS_DMA_CFG_REG);
+}
+
+static void mtk_spi_slave_disable_xfer(struct mtk_spi_slave *mdata)
+{
+ u32 reg_val;
+
+ reg_val = readl(mdata->base + SPIS_CFG_REG);
+ reg_val &= ~SPIS_TX_EN;
+ reg_val &= ~SPIS_RX_EN;
+ writel(reg_val, mdata->base + SPIS_CFG_REG);
+}
+
+static int mtk_spi_slave_wait_for_completion(struct mtk_spi_slave *mdata)
+{
+ if (wait_for_completion_interruptible(&mdata->xfer_done) ||
+ mdata->slave_aborted) {
+ dev_err(mdata->dev, "interrupted\n");
+ return -EINTR;
+ }
+
+ return 0;
+}
+
+static int mtk_spi_slave_prepare_message(struct spi_controller *ctlr,
+ struct spi_message *msg)
+{
+ struct mtk_spi_slave *mdata = spi_controller_get_devdata(ctlr);
+ struct spi_device *spi = msg->spi;
+ bool cpha, cpol;
+ u32 reg_val;
+
+ cpha = spi->mode & SPI_CPHA ? 1 : 0;
+ cpol = spi->mode & SPI_CPOL ? 1 : 0;
+
+ reg_val = readl(mdata->base + SPIS_CFG_REG);
+ if (cpha)
+ reg_val |= SPIS_CPHA;
+ else
+ reg_val &= ~SPIS_CPHA;
+ if (cpol)
+ reg_val |= SPIS_CPOL;
+ else
+ reg_val &= ~SPIS_CPOL;
+
+ if (spi->mode & SPI_LSB_FIRST)
+ reg_val &= ~(SPIS_TXMSBF | SPIS_RXMSBF);
+ else
+ reg_val |= SPIS_TXMSBF | SPIS_RXMSBF;
+
+ reg_val &= ~SPIS_TX_ENDIAN;
+ reg_val &= ~SPIS_RX_ENDIAN;
+ writel(reg_val, mdata->base + SPIS_CFG_REG);
+
+ return 0;
+}
+
+static int mtk_spi_slave_fifo_transfer(struct spi_controller *ctlr,
+ struct spi_device *spi,
+ struct spi_transfer *xfer)
+{
+ struct mtk_spi_slave *mdata = spi_controller_get_devdata(ctlr);
+ int reg_val, cnt, remainder, ret;
+
+ writel(SPIS_SOFT_RST, mdata->base + SPIS_SOFT_RST_REG);
+
+ reg_val = readl(mdata->base + SPIS_CFG_REG);
+ if (xfer->rx_buf)
+ reg_val |= SPIS_RX_EN;
+ if (xfer->tx_buf)
+ reg_val |= SPIS_TX_EN;
+ writel(reg_val, mdata->base + SPIS_CFG_REG);
+
+ cnt = xfer->len / 4;
+ if (xfer->tx_buf)
+ iowrite32_rep(mdata->base + SPIS_TX_DATA_REG,
+ xfer->tx_buf, cnt);
+
+ remainder = xfer->len % 4;
+ if (xfer->tx_buf && remainder > 0) {
+ reg_val = 0;
+ memcpy(®_val, xfer->tx_buf + cnt * 4, remainder);
+ writel(reg_val, mdata->base + SPIS_TX_DATA_REG);
+ }
+
+ ret = mtk_spi_slave_wait_for_completion(mdata);
+ if (ret) {
+ mtk_spi_slave_disable_xfer(mdata);
+ writel(SPIS_SOFT_RST, mdata->base + SPIS_SOFT_RST_REG);
+ }
+
+ return ret;
+}
+
+static int mtk_spi_slave_dma_transfer(struct spi_controller *ctlr,
+ struct spi_device *spi,
+ struct spi_transfer *xfer)
+{
+ struct mtk_spi_slave *mdata = spi_controller_get_devdata(ctlr);
+ struct device *dev = mdata->dev;
+ int reg_val, ret;
+
+ writel(SPIS_SOFT_RST, mdata->base + SPIS_SOFT_RST_REG);
+
+ if (xfer->tx_buf) {
+ /* tx_buf is a const void* where we need a void * for
+ * the dma mapping
+ */
+ void *nonconst_tx = (void *)xfer->tx_buf;
+
+ xfer->tx_dma = dma_map_single(dev, nonconst_tx,
+ xfer->len, DMA_TO_DEVICE);
+ if (dma_mapping_error(dev, xfer->tx_dma)) {
+ ret = -ENOMEM;
+ goto disable_transfer;
+ }
+ }
+
+ if (xfer->rx_buf) {
+ xfer->rx_dma = dma_map_single(dev, xfer->rx_buf,
+ xfer->len, DMA_FROM_DEVICE);
+ if (dma_mapping_error(dev, xfer->rx_dma)) {
+ ret = -ENOMEM;
+ goto unmap_txdma;
+ }
+ }
+
+ writel(xfer->tx_dma, mdata->base + SPIS_TX_SRC_REG);
+ writel(xfer->rx_dma, mdata->base + SPIS_RX_DST_REG);
+
+ writel(SPIS_DMA_ADDR_EN, mdata->base + SPIS_SOFT_RST_REG);
+
+ /* enable config reg tx rx_enable */
+ reg_val = readl(mdata->base + SPIS_CFG_REG);
+ if (xfer->tx_buf)
+ reg_val |= SPIS_TX_EN;
+ if (xfer->rx_buf)
+ reg_val |= SPIS_RX_EN;
+ writel(reg_val, mdata->base + SPIS_CFG_REG);
+
+ /* config dma */
+ reg_val = 0;
+ reg_val |= (xfer->len - 1) & TX_DMA_LEN;
+ writel(reg_val, mdata->base + SPIS_DMA_CFG_REG);
+
+ reg_val = readl(mdata->base + SPIS_DMA_CFG_REG);
+ if (xfer->tx_buf)
+ reg_val |= TX_DMA_EN;
+ if (xfer->rx_buf)
+ reg_val |= RX_DMA_EN;
+ reg_val |= TX_DMA_TRIG_EN;
+ writel(reg_val, mdata->base + SPIS_DMA_CFG_REG);
+
+ ret = mtk_spi_slave_wait_for_completion(mdata);
+ if (ret)
+ goto unmap_rxdma;
+
+ return 0;
+
+unmap_rxdma:
+ if (xfer->rx_buf)
+ dma_unmap_single(dev, xfer->rx_dma,
+ xfer->len, DMA_FROM_DEVICE);
+
+unmap_txdma:
+ if (xfer->tx_buf)
+ dma_unmap_single(dev, xfer->tx_dma,
+ xfer->len, DMA_TO_DEVICE);
+
+disable_transfer:
+ mtk_spi_slave_disable_dma(mdata);
+ mtk_spi_slave_disable_xfer(mdata);
+ writel(SPIS_SOFT_RST, mdata->base + SPIS_SOFT_RST_REG);
+
+ return ret;
+}
+
+static int mtk_spi_slave_transfer_one(struct spi_controller *ctlr,
+ struct spi_device *spi,
+ struct spi_transfer *xfer)
+{
+ struct mtk_spi_slave *mdata = spi_controller_get_devdata(ctlr);
+
+ reinit_completion(&mdata->xfer_done);
+ mdata->slave_aborted = false;
+ mdata->cur_transfer = xfer;
+
+ if (xfer->len > MTK_SPI_SLAVE_MAX_FIFO_SIZE)
+ return mtk_spi_slave_dma_transfer(ctlr, spi, xfer);
+ else
+ return mtk_spi_slave_fifo_transfer(ctlr, spi, xfer);
+}
+
+static int mtk_spi_slave_setup(struct spi_device *spi)
+{
+ struct mtk_spi_slave *mdata = spi_controller_get_devdata(spi->master);
+ u32 reg_val;
+
+ reg_val = DMA_DONE_EN | DATA_DONE_EN |
+ RSTA_DONE_EN | CMD_INVALID_EN;
+ writel(reg_val, mdata->base + SPIS_IRQ_EN_REG);
+
+ reg_val = DMA_DONE_MASK | DATA_DONE_MASK |
+ RSTA_DONE_MASK | CMD_INVALID_MASK;
+ writel(reg_val, mdata->base + SPIS_IRQ_MASK_REG);
+
+ mtk_spi_slave_disable_dma(mdata);
+ mtk_spi_slave_disable_xfer(mdata);
+
+ return 0;
+}
+
+static int mtk_slave_abort(struct spi_controller *ctlr)
+{
+ struct mtk_spi_slave *mdata = spi_controller_get_devdata(ctlr);
+
+ mdata->slave_aborted = true;
+ complete(&mdata->xfer_done);
+
+ return 0;
+}
+
+static irqreturn_t mtk_spi_slave_interrupt(int irq, void *dev_id)
+{
+ struct spi_controller *ctlr = dev_id;
+ struct mtk_spi_slave *mdata = spi_controller_get_devdata(ctlr);
+ struct spi_transfer *trans = mdata->cur_transfer;
+ u32 int_status, reg_val, cnt, remainder;
+
+ int_status = readl(mdata->base + SPIS_IRQ_ST_REG);
+ writel(int_status, mdata->base + SPIS_IRQ_CLR_REG);
+
+ if (!trans)
+ return IRQ_NONE;
+
+ if ((int_status & DMA_DONE_ST) &&
+ ((int_status & DATA_DONE_ST) ||
+ (int_status & RSTA_DONE_ST))) {
+ writel(SPIS_SOFT_RST, mdata->base + SPIS_SOFT_RST_REG);
+
+ if (trans->tx_buf)
+ dma_unmap_single(mdata->dev, trans->tx_dma,
+ trans->len, DMA_TO_DEVICE);
+ if (trans->rx_buf)
+ dma_unmap_single(mdata->dev, trans->rx_dma,
+ trans->len, DMA_FROM_DEVICE);
+
+ mtk_spi_slave_disable_dma(mdata);
+ mtk_spi_slave_disable_xfer(mdata);
+ }
+
+ if ((!(int_status & DMA_DONE_ST)) &&
+ ((int_status & DATA_DONE_ST) ||
+ (int_status & RSTA_DONE_ST))) {
+ cnt = trans->len / 4;
+ if (trans->rx_buf)
+ ioread32_rep(mdata->base + SPIS_RX_DATA_REG,
+ trans->rx_buf, cnt);
+ remainder = trans->len % 4;
+ if (trans->rx_buf && remainder > 0) {
+ reg_val = readl(mdata->base + SPIS_RX_DATA_REG);
+ memcpy(trans->rx_buf + (cnt * 4),
+ ®_val, remainder);
+ }
+
+ mtk_spi_slave_disable_xfer(mdata);
+ }
+
+ if (int_status & CMD_INVALID_ST) {
+ dev_warn(&ctlr->dev, "cmd invalid\n");
+ return IRQ_NONE;
+ }
+
+ mdata->cur_transfer = NULL;
+ complete(&mdata->xfer_done);
+
+ return IRQ_HANDLED;
+}
+
+static int mtk_spi_slave_probe(struct platform_device *pdev)
+{
+ struct spi_controller *ctlr;
+ struct mtk_spi_slave *mdata;
+ struct resource *res;
+ int irq, ret;
+
+ ctlr = spi_alloc_slave(&pdev->dev, sizeof(*mdata));
+ if (!ctlr) {
+ dev_err(&pdev->dev, "failed to alloc spi slave\n");
+ return -ENOMEM;
+ }
+
+ ctlr->auto_runtime_pm = true;
+ ctlr->dev.of_node = pdev->dev.of_node;
+ ctlr->mode_bits = SPI_CPOL | SPI_CPHA;
+ ctlr->mode_bits |= SPI_LSB_FIRST;
+
+ ctlr->prepare_message = mtk_spi_slave_prepare_message;
+ ctlr->transfer_one = mtk_spi_slave_transfer_one;
+ ctlr->setup = mtk_spi_slave_setup;
+ ctlr->slave_abort = mtk_slave_abort;
+
+ mdata = spi_controller_get_devdata(ctlr);
+
+ platform_set_drvdata(pdev, ctlr);
+
+ init_completion(&mdata->xfer_done);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res) {
+ ret = -ENODEV;
+ dev_err(&pdev->dev, "failed to determine base address\n");
+ goto err_put_ctlr;
+ }
+
+ mdata->dev = &pdev->dev;
+
+ mdata->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(mdata->base)) {
+ ret = PTR_ERR(mdata->base);
+ goto err_put_ctlr;
+ }
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0) {
+ dev_err(&pdev->dev, "failed to get irq (%d)\n", irq);
+ ret = irq;
+ goto err_put_ctlr;
+ }
+
+ ret = devm_request_irq(&pdev->dev, irq, mtk_spi_slave_interrupt,
+ IRQF_TRIGGER_NONE, dev_name(&pdev->dev), ctlr);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to register irq (%d)\n", ret);
+ goto err_put_ctlr;
+ }
+
+ mdata->spi_clk = devm_clk_get(&pdev->dev, "spi");
+ if (IS_ERR(mdata->spi_clk)) {
+ ret = PTR_ERR(mdata->spi_clk);
+ dev_err(&pdev->dev, "failed to get spi-clk: %d\n", ret);
+ goto err_put_ctlr;
+ }
+
+ ret = clk_prepare_enable(mdata->spi_clk);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "failed to enable spi_clk (%d)\n", ret);
+ goto err_put_ctlr;
+ }
+
+ pm_runtime_enable(&pdev->dev);
+
+ ret = devm_spi_register_controller(&pdev->dev, ctlr);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "failed to register slave controller(%d)\n", ret);
+ clk_disable_unprepare(mdata->spi_clk);
+ goto err_disable_runtime_pm;
+ }
+
+ clk_disable_unprepare(mdata->spi_clk);
+
+ return 0;
+
+err_disable_runtime_pm:
+ pm_runtime_disable(&pdev->dev);
+err_put_ctlr:
+ spi_controller_put(ctlr);
+
+ return ret;
+}
+
+static int mtk_spi_slave_remove(struct platform_device *pdev)
+{
+ pm_runtime_disable(&pdev->dev);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int mtk_spi_slave_suspend(struct device *dev)
+{
+ struct spi_controller *ctlr = dev_get_drvdata(dev);
+ struct mtk_spi_slave *mdata = spi_controller_get_devdata(ctlr);
+ int ret;
+
+ ret = spi_controller_suspend(ctlr);
+ if (ret)
+ return ret;
+
+ if (!pm_runtime_suspended(dev))
+ clk_disable_unprepare(mdata->spi_clk);
+
+ return ret;
+}
+
+static int mtk_spi_slave_resume(struct device *dev)
+{
+ struct spi_controller *ctlr = dev_get_drvdata(dev);
+ struct mtk_spi_slave *mdata = spi_controller_get_devdata(ctlr);
+ int ret;
+
+ if (!pm_runtime_suspended(dev)) {
+ ret = clk_prepare_enable(mdata->spi_clk);
+ if (ret < 0) {
+ dev_err(dev, "failed to enable spi_clk (%d)\n", ret);
+ return ret;
+ }
+ }
+
+ ret = spi_controller_resume(ctlr);
+ if (ret < 0)
+ clk_disable_unprepare(mdata->spi_clk);
+
+ return ret;
+}
+#endif /* CONFIG_PM_SLEEP */
+
+#ifdef CONFIG_PM
+static int mtk_spi_slave_runtime_suspend(struct device *dev)
+{
+ struct spi_controller *ctlr = dev_get_drvdata(dev);
+ struct mtk_spi_slave *mdata = spi_controller_get_devdata(ctlr);
+
+ clk_disable_unprepare(mdata->spi_clk);
+
+ return 0;
+}
+
+static int mtk_spi_slave_runtime_resume(struct device *dev)
+{
+ struct spi_controller *ctlr = dev_get_drvdata(dev);
+ struct mtk_spi_slave *mdata = spi_controller_get_devdata(ctlr);
+ int ret;
+
+ ret = clk_prepare_enable(mdata->spi_clk);
+ if (ret < 0) {
+ dev_err(dev, "failed to enable spi_clk (%d)\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+#endif /* CONFIG_PM */
+
+static const struct dev_pm_ops mtk_spi_slave_pm = {
+ SET_SYSTEM_SLEEP_PM_OPS(mtk_spi_slave_suspend, mtk_spi_slave_resume)
+ SET_RUNTIME_PM_OPS(mtk_spi_slave_runtime_suspend,
+ mtk_spi_slave_runtime_resume, NULL)
+};
+
+static struct platform_driver mtk_spi_slave_driver = {
+ .driver = {
+ .name = "mtk-spi-slave",
+ .pm = &mtk_spi_slave_pm,
+ .of_match_table = mtk_spi_slave_of_match,
+ },
+ .probe = mtk_spi_slave_probe,
+ .remove = mtk_spi_slave_remove,
+};
+
+module_platform_driver(mtk_spi_slave_driver);
+
+MODULE_DESCRIPTION("MTK SPI Slave Controller driver");
+MODULE_AUTHOR("Leilk Liu <leilk.liu@mediatek.com>");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:mtk-spi-slave");
case CMD_REBOOT:
dev_info(&priv->spi->dev, "Rebooting system...\n");
kernel_restart(NULL);
+ break;
case CMD_POWEROFF:
dev_info(&priv->spi->dev, "Powering off system...\n");
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Spreadtrum Communications Inc.
+
+#include <linux/clk.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/spi/spi.h>
+
+#define SPRD_SPI_TXD 0x0
+#define SPRD_SPI_CLKD 0x4
+#define SPRD_SPI_CTL0 0x8
+#define SPRD_SPI_CTL1 0xc
+#define SPRD_SPI_CTL2 0x10
+#define SPRD_SPI_CTL3 0x14
+#define SPRD_SPI_CTL4 0x18
+#define SPRD_SPI_CTL5 0x1c
+#define SPRD_SPI_INT_EN 0x20
+#define SPRD_SPI_INT_CLR 0x24
+#define SPRD_SPI_INT_RAW_STS 0x28
+#define SPRD_SPI_INT_MASK_STS 0x2c
+#define SPRD_SPI_STS1 0x30
+#define SPRD_SPI_STS2 0x34
+#define SPRD_SPI_DSP_WAIT 0x38
+#define SPRD_SPI_STS3 0x3c
+#define SPRD_SPI_CTL6 0x40
+#define SPRD_SPI_STS4 0x44
+#define SPRD_SPI_FIFO_RST 0x48
+#define SPRD_SPI_CTL7 0x4c
+#define SPRD_SPI_STS5 0x50
+#define SPRD_SPI_CTL8 0x54
+#define SPRD_SPI_CTL9 0x58
+#define SPRD_SPI_CTL10 0x5c
+#define SPRD_SPI_CTL11 0x60
+#define SPRD_SPI_CTL12 0x64
+#define SPRD_SPI_STS6 0x68
+#define SPRD_SPI_STS7 0x6c
+#define SPRD_SPI_STS8 0x70
+#define SPRD_SPI_STS9 0x74
+
+/* Bits & mask definition for register CTL0 */
+#define SPRD_SPI_SCK_REV BIT(13)
+#define SPRD_SPI_NG_TX BIT(1)
+#define SPRD_SPI_NG_RX BIT(0)
+#define SPRD_SPI_CHNL_LEN_MASK GENMASK(4, 0)
+#define SPRD_SPI_CSN_MASK GENMASK(11, 8)
+#define SPRD_SPI_CS0_VALID BIT(8)
+
+/* Bits & mask definition for register SPI_INT_EN */
+#define SPRD_SPI_TX_END_INT_EN BIT(8)
+#define SPRD_SPI_RX_END_INT_EN BIT(9)
+
+/* Bits & mask definition for register SPI_INT_RAW_STS */
+#define SPRD_SPI_TX_END_RAW BIT(8)
+#define SPRD_SPI_RX_END_RAW BIT(9)
+
+/* Bits & mask definition for register SPI_INT_CLR */
+#define SPRD_SPI_TX_END_CLR BIT(8)
+#define SPRD_SPI_RX_END_CLR BIT(9)
+
+/* Bits & mask definition for register INT_MASK_STS */
+#define SPRD_SPI_MASK_RX_END BIT(9)
+#define SPRD_SPI_MASK_TX_END BIT(8)
+
+/* Bits & mask definition for register STS2 */
+#define SPRD_SPI_TX_BUSY BIT(8)
+
+/* Bits & mask definition for register CTL1 */
+#define SPRD_SPI_RX_MODE BIT(12)
+#define SPRD_SPI_TX_MODE BIT(13)
+#define SPRD_SPI_RTX_MD_MASK GENMASK(13, 12)
+
+/* Bits & mask definition for register CTL2 */
+#define SPRD_SPI_DMA_EN BIT(6)
+
+/* Bits & mask definition for register CTL4 */
+#define SPRD_SPI_START_RX BIT(9)
+#define SPRD_SPI_ONLY_RECV_MASK GENMASK(8, 0)
+
+/* Bits & mask definition for register SPI_INT_CLR */
+#define SPRD_SPI_RX_END_INT_CLR BIT(9)
+#define SPRD_SPI_TX_END_INT_CLR BIT(8)
+
+/* Bits & mask definition for register SPI_INT_RAW */
+#define SPRD_SPI_RX_END_IRQ BIT(9)
+#define SPRD_SPI_TX_END_IRQ BIT(8)
+
+/* Bits & mask definition for register CTL12 */
+#define SPRD_SPI_SW_RX_REQ BIT(0)
+#define SPRD_SPI_SW_TX_REQ BIT(1)
+
+/* Bits & mask definition for register CTL7 */
+#define SPRD_SPI_DATA_LINE2_EN BIT(15)
+#define SPRD_SPI_MODE_MASK GENMASK(5, 3)
+#define SPRD_SPI_MODE_OFFSET 3
+#define SPRD_SPI_3WIRE_MODE 4
+#define SPRD_SPI_4WIRE_MODE 0
+
+/* Bits & mask definition for register CTL8 */
+#define SPRD_SPI_TX_MAX_LEN_MASK GENMASK(19, 0)
+#define SPRD_SPI_TX_LEN_H_MASK GENMASK(3, 0)
+#define SPRD_SPI_TX_LEN_H_OFFSET 16
+
+/* Bits & mask definition for register CTL9 */
+#define SPRD_SPI_TX_LEN_L_MASK GENMASK(15, 0)
+
+/* Bits & mask definition for register CTL10 */
+#define SPRD_SPI_RX_MAX_LEN_MASK GENMASK(19, 0)
+#define SPRD_SPI_RX_LEN_H_MASK GENMASK(3, 0)
+#define SPRD_SPI_RX_LEN_H_OFFSET 16
+
+/* Bits & mask definition for register CTL11 */
+#define SPRD_SPI_RX_LEN_L_MASK GENMASK(15, 0)
+
+/* Default & maximum word delay cycles */
+#define SPRD_SPI_MIN_DELAY_CYCLE 14
+#define SPRD_SPI_MAX_DELAY_CYCLE 130
+
+#define SPRD_SPI_FIFO_SIZE 32
+#define SPRD_SPI_CHIP_CS_NUM 0x4
+#define SPRD_SPI_CHNL_LEN 2
+#define SPRD_SPI_DEFAULT_SOURCE 26000000
+#define SPRD_SPI_MAX_SPEED_HZ 48000000
+#define SPRD_SPI_AUTOSUSPEND_DELAY 100
+
+struct sprd_spi {
+ void __iomem *base;
+ struct device *dev;
+ struct clk *clk;
+ u32 src_clk;
+ u32 hw_mode;
+ u32 trans_len;
+ u32 trans_mode;
+ u32 word_delay;
+ u32 hw_speed_hz;
+ u32 len;
+ int status;
+ const void *tx_buf;
+ void *rx_buf;
+ int (*read_bufs)(struct sprd_spi *ss, u32 len);
+ int (*write_bufs)(struct sprd_spi *ss, u32 len);
+};
+
+static u32 sprd_spi_transfer_max_timeout(struct sprd_spi *ss,
+ struct spi_transfer *t)
+{
+ /*
+ * The time spent on transmission of the full FIFO data is the maximum
+ * SPI transmission time.
+ */
+ u32 size = t->bits_per_word * SPRD_SPI_FIFO_SIZE;
+ u32 bit_time_us = DIV_ROUND_UP(USEC_PER_SEC, ss->hw_speed_hz);
+ u32 total_time_us = size * bit_time_us;
+ /*
+ * There is an interval between data and the data in our SPI hardware,
+ * so the total transmission time need add the interval time.
+ */
+ u32 interval_cycle = SPRD_SPI_FIFO_SIZE * ss->word_delay;
+ u32 interval_time_us = DIV_ROUND_UP(interval_cycle * USEC_PER_SEC,
+ ss->src_clk);
+
+ return total_time_us + interval_time_us;
+}
+
+static int sprd_spi_wait_for_tx_end(struct sprd_spi *ss, struct spi_transfer *t)
+{
+ u32 val, us;
+ int ret;
+
+ us = sprd_spi_transfer_max_timeout(ss, t);
+ ret = readl_relaxed_poll_timeout(ss->base + SPRD_SPI_INT_RAW_STS, val,
+ val & SPRD_SPI_TX_END_IRQ, 0, us);
+ if (ret) {
+ dev_err(ss->dev, "SPI error, spi send timeout!\n");
+ return ret;
+ }
+
+ ret = readl_relaxed_poll_timeout(ss->base + SPRD_SPI_STS2, val,
+ !(val & SPRD_SPI_TX_BUSY), 0, us);
+ if (ret) {
+ dev_err(ss->dev, "SPI error, spi busy timeout!\n");
+ return ret;
+ }
+
+ writel_relaxed(SPRD_SPI_TX_END_INT_CLR, ss->base + SPRD_SPI_INT_CLR);
+
+ return 0;
+}
+
+static int sprd_spi_wait_for_rx_end(struct sprd_spi *ss, struct spi_transfer *t)
+{
+ u32 val, us;
+ int ret;
+
+ us = sprd_spi_transfer_max_timeout(ss, t);
+ ret = readl_relaxed_poll_timeout(ss->base + SPRD_SPI_INT_RAW_STS, val,
+ val & SPRD_SPI_RX_END_IRQ, 0, us);
+ if (ret) {
+ dev_err(ss->dev, "SPI error, spi rx timeout!\n");
+ return ret;
+ }
+
+ writel_relaxed(SPRD_SPI_RX_END_INT_CLR, ss->base + SPRD_SPI_INT_CLR);
+
+ return 0;
+}
+
+static void sprd_spi_tx_req(struct sprd_spi *ss)
+{
+ writel_relaxed(SPRD_SPI_SW_TX_REQ, ss->base + SPRD_SPI_CTL12);
+}
+
+static void sprd_spi_rx_req(struct sprd_spi *ss)
+{
+ writel_relaxed(SPRD_SPI_SW_RX_REQ, ss->base + SPRD_SPI_CTL12);
+}
+
+static void sprd_spi_enter_idle(struct sprd_spi *ss)
+{
+ u32 val = readl_relaxed(ss->base + SPRD_SPI_CTL1);
+
+ val &= ~SPRD_SPI_RTX_MD_MASK;
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL1);
+}
+
+static void sprd_spi_set_transfer_bits(struct sprd_spi *ss, u32 bits)
+{
+ u32 val = readl_relaxed(ss->base + SPRD_SPI_CTL0);
+
+ /* Set the valid bits for every transaction */
+ val &= ~(SPRD_SPI_CHNL_LEN_MASK << SPRD_SPI_CHNL_LEN);
+ val |= bits << SPRD_SPI_CHNL_LEN;
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL0);
+}
+
+static void sprd_spi_set_tx_length(struct sprd_spi *ss, u32 length)
+{
+ u32 val = readl_relaxed(ss->base + SPRD_SPI_CTL8);
+
+ length &= SPRD_SPI_TX_MAX_LEN_MASK;
+ val &= ~SPRD_SPI_TX_LEN_H_MASK;
+ val |= length >> SPRD_SPI_TX_LEN_H_OFFSET;
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL8);
+
+ val = length & SPRD_SPI_TX_LEN_L_MASK;
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL9);
+}
+
+static void sprd_spi_set_rx_length(struct sprd_spi *ss, u32 length)
+{
+ u32 val = readl_relaxed(ss->base + SPRD_SPI_CTL10);
+
+ length &= SPRD_SPI_RX_MAX_LEN_MASK;
+ val &= ~SPRD_SPI_RX_LEN_H_MASK;
+ val |= length >> SPRD_SPI_RX_LEN_H_OFFSET;
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL10);
+
+ val = length & SPRD_SPI_RX_LEN_L_MASK;
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL11);
+}
+
+static void sprd_spi_chipselect(struct spi_device *sdev, bool cs)
+{
+ struct spi_controller *sctlr = sdev->controller;
+ struct sprd_spi *ss = spi_controller_get_devdata(sctlr);
+ u32 val;
+
+ val = readl_relaxed(ss->base + SPRD_SPI_CTL0);
+ /* The SPI controller will pull down CS pin if cs is 0 */
+ if (!cs) {
+ val &= ~SPRD_SPI_CS0_VALID;
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL0);
+ } else {
+ val |= SPRD_SPI_CSN_MASK;
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL0);
+ }
+}
+
+static int sprd_spi_write_only_receive(struct sprd_spi *ss, u32 len)
+{
+ u32 val;
+
+ /* Clear the start receive bit and reset receive data number */
+ val = readl_relaxed(ss->base + SPRD_SPI_CTL4);
+ val &= ~(SPRD_SPI_START_RX | SPRD_SPI_ONLY_RECV_MASK);
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL4);
+
+ /* Set the receive data length */
+ val = readl_relaxed(ss->base + SPRD_SPI_CTL4);
+ val |= len & SPRD_SPI_ONLY_RECV_MASK;
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL4);
+
+ /* Trigger to receive data */
+ val = readl_relaxed(ss->base + SPRD_SPI_CTL4);
+ val |= SPRD_SPI_START_RX;
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL4);
+
+ return len;
+}
+
+static int sprd_spi_write_bufs_u8(struct sprd_spi *ss, u32 len)
+{
+ u8 *tx_p = (u8 *)ss->tx_buf;
+ int i;
+
+ for (i = 0; i < len; i++)
+ writeb_relaxed(tx_p[i], ss->base + SPRD_SPI_TXD);
+
+ ss->tx_buf += i;
+ return i;
+}
+
+static int sprd_spi_write_bufs_u16(struct sprd_spi *ss, u32 len)
+{
+ u16 *tx_p = (u16 *)ss->tx_buf;
+ int i;
+
+ for (i = 0; i < len; i++)
+ writew_relaxed(tx_p[i], ss->base + SPRD_SPI_TXD);
+
+ ss->tx_buf += i << 1;
+ return i << 1;
+}
+
+static int sprd_spi_write_bufs_u32(struct sprd_spi *ss, u32 len)
+{
+ u32 *tx_p = (u32 *)ss->tx_buf;
+ int i;
+
+ for (i = 0; i < len; i++)
+ writel_relaxed(tx_p[i], ss->base + SPRD_SPI_TXD);
+
+ ss->tx_buf += i << 2;
+ return i << 2;
+}
+
+static int sprd_spi_read_bufs_u8(struct sprd_spi *ss, u32 len)
+{
+ u8 *rx_p = (u8 *)ss->rx_buf;
+ int i;
+
+ for (i = 0; i < len; i++)
+ rx_p[i] = readb_relaxed(ss->base + SPRD_SPI_TXD);
+
+ ss->rx_buf += i;
+ return i;
+}
+
+static int sprd_spi_read_bufs_u16(struct sprd_spi *ss, u32 len)
+{
+ u16 *rx_p = (u16 *)ss->rx_buf;
+ int i;
+
+ for (i = 0; i < len; i++)
+ rx_p[i] = readw_relaxed(ss->base + SPRD_SPI_TXD);
+
+ ss->rx_buf += i << 1;
+ return i << 1;
+}
+
+static int sprd_spi_read_bufs_u32(struct sprd_spi *ss, u32 len)
+{
+ u32 *rx_p = (u32 *)ss->rx_buf;
+ int i;
+
+ for (i = 0; i < len; i++)
+ rx_p[i] = readl_relaxed(ss->base + SPRD_SPI_TXD);
+
+ ss->rx_buf += i << 2;
+ return i << 2;
+}
+
+static int sprd_spi_txrx_bufs(struct spi_device *sdev, struct spi_transfer *t)
+{
+ struct sprd_spi *ss = spi_controller_get_devdata(sdev->controller);
+ u32 trans_len = ss->trans_len, len;
+ int ret, write_size = 0;
+
+ while (trans_len) {
+ len = trans_len > SPRD_SPI_FIFO_SIZE ? SPRD_SPI_FIFO_SIZE :
+ trans_len;
+ if (ss->trans_mode & SPRD_SPI_TX_MODE) {
+ sprd_spi_set_tx_length(ss, len);
+ write_size += ss->write_bufs(ss, len);
+
+ /*
+ * For our 3 wires mode or dual TX line mode, we need
+ * to request the controller to transfer.
+ */
+ if (ss->hw_mode & SPI_3WIRE || ss->hw_mode & SPI_TX_DUAL)
+ sprd_spi_tx_req(ss);
+
+ ret = sprd_spi_wait_for_tx_end(ss, t);
+ } else {
+ sprd_spi_set_rx_length(ss, len);
+
+ /*
+ * For our 3 wires mode or dual TX line mode, we need
+ * to request the controller to read.
+ */
+ if (ss->hw_mode & SPI_3WIRE || ss->hw_mode & SPI_TX_DUAL)
+ sprd_spi_rx_req(ss);
+ else
+ write_size += ss->write_bufs(ss, len);
+
+ ret = sprd_spi_wait_for_rx_end(ss, t);
+ }
+
+ if (ret)
+ goto complete;
+
+ if (ss->trans_mode & SPRD_SPI_RX_MODE)
+ ss->read_bufs(ss, len);
+
+ trans_len -= len;
+ }
+
+ ret = write_size;
+
+complete:
+ sprd_spi_enter_idle(ss);
+
+ return ret;
+}
+
+static void sprd_spi_set_speed(struct sprd_spi *ss, u32 speed_hz)
+{
+ /*
+ * From SPI datasheet, the prescale calculation formula:
+ * prescale = SPI source clock / (2 * SPI_freq) - 1;
+ */
+ u32 clk_div = DIV_ROUND_UP(ss->src_clk, speed_hz << 1) - 1;
+
+ /* Save the real hardware speed */
+ ss->hw_speed_hz = (ss->src_clk >> 1) / (clk_div + 1);
+ writel_relaxed(clk_div, ss->base + SPRD_SPI_CLKD);
+}
+
+static void sprd_spi_init_hw(struct sprd_spi *ss, struct spi_transfer *t)
+{
+ u16 word_delay, interval;
+ u32 val;
+
+ val = readl_relaxed(ss->base + SPRD_SPI_CTL7);
+ val &= ~(SPRD_SPI_SCK_REV | SPRD_SPI_NG_TX | SPRD_SPI_NG_RX);
+ /* Set default chip selection, clock phase and clock polarity */
+ val |= ss->hw_mode & SPI_CPHA ? SPRD_SPI_NG_RX : SPRD_SPI_NG_TX;
+ val |= ss->hw_mode & SPI_CPOL ? SPRD_SPI_SCK_REV : 0;
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL0);
+
+ /*
+ * Set the intervals of two SPI frames, and the inteval calculation
+ * formula as below per datasheet:
+ * interval time (source clock cycles) = interval * 4 + 10.
+ */
+ word_delay = clamp_t(u16, t->word_delay, SPRD_SPI_MIN_DELAY_CYCLE,
+ SPRD_SPI_MAX_DELAY_CYCLE);
+ interval = DIV_ROUND_UP(word_delay - 10, 4);
+ ss->word_delay = interval * 4 + 10;
+ writel_relaxed(interval, ss->base + SPRD_SPI_CTL5);
+
+ /* Reset SPI fifo */
+ writel_relaxed(1, ss->base + SPRD_SPI_FIFO_RST);
+ writel_relaxed(0, ss->base + SPRD_SPI_FIFO_RST);
+
+ /* Set SPI work mode */
+ val = readl_relaxed(ss->base + SPRD_SPI_CTL7);
+ val &= ~SPRD_SPI_MODE_MASK;
+
+ if (ss->hw_mode & SPI_3WIRE)
+ val |= SPRD_SPI_3WIRE_MODE << SPRD_SPI_MODE_OFFSET;
+ else
+ val |= SPRD_SPI_4WIRE_MODE << SPRD_SPI_MODE_OFFSET;
+
+ if (ss->hw_mode & SPI_TX_DUAL)
+ val |= SPRD_SPI_DATA_LINE2_EN;
+ else
+ val &= ~SPRD_SPI_DATA_LINE2_EN;
+
+ writel_relaxed(val, ss->base + SPRD_SPI_CTL7);
+}
+
+static int sprd_spi_setup_transfer(struct spi_device *sdev,
+ struct spi_transfer *t)
+{
+ struct sprd_spi *ss = spi_controller_get_devdata(sdev->controller);
+ u8 bits_per_word = t->bits_per_word;
+ u32 val, mode = 0;
+
+ ss->len = t->len;
+ ss->tx_buf = t->tx_buf;
+ ss->rx_buf = t->rx_buf;
+
+ ss->hw_mode = sdev->mode;
+ sprd_spi_init_hw(ss, t);
+
+ /* Set tansfer speed and valid bits */
+ sprd_spi_set_speed(ss, t->speed_hz);
+ sprd_spi_set_transfer_bits(ss, bits_per_word);
+
+ if (bits_per_word > 16)
+ bits_per_word = round_up(bits_per_word, 16);
+ else
+ bits_per_word = round_up(bits_per_word, 8);
+
+ switch (bits_per_word) {
+ case 8:
+ ss->trans_len = t->len;
+ ss->read_bufs = sprd_spi_read_bufs_u8;
+ ss->write_bufs = sprd_spi_write_bufs_u8;
+ break;
+ case 16:
+ ss->trans_len = t->len >> 1;
+ ss->read_bufs = sprd_spi_read_bufs_u16;
+ ss->write_bufs = sprd_spi_write_bufs_u16;
+ break;
+ case 32:
+ ss->trans_len = t->len >> 2;
+ ss->read_bufs = sprd_spi_read_bufs_u32;
+ ss->write_bufs = sprd_spi_write_bufs_u32;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ /* Set transfer read or write mode */
+ val = readl_relaxed(ss->base + SPRD_SPI_CTL1);
+ val &= ~SPRD_SPI_RTX_MD_MASK;
+ if (t->tx_buf)
+ mode |= SPRD_SPI_TX_MODE;
+ if (t->rx_buf)
+ mode |= SPRD_SPI_RX_MODE;
+
+ writel_relaxed(val | mode, ss->base + SPRD_SPI_CTL1);
+
+ ss->trans_mode = mode;
+
+ /*
+ * If in only receive mode, we need to trigger the SPI controller to
+ * receive data automatically.
+ */
+ if (ss->trans_mode == SPRD_SPI_RX_MODE)
+ ss->write_bufs = sprd_spi_write_only_receive;
+
+ return 0;
+}
+
+static int sprd_spi_transfer_one(struct spi_controller *sctlr,
+ struct spi_device *sdev,
+ struct spi_transfer *t)
+{
+ int ret;
+
+ ret = sprd_spi_setup_transfer(sdev, t);
+ if (ret)
+ goto setup_err;
+
+ ret = sprd_spi_txrx_bufs(sdev, t);
+ if (ret == t->len)
+ ret = 0;
+ else if (ret >= 0)
+ ret = -EREMOTEIO;
+
+setup_err:
+ spi_finalize_current_transfer(sctlr);
+
+ return ret;
+}
+
+static int sprd_spi_clk_init(struct platform_device *pdev, struct sprd_spi *ss)
+{
+ struct clk *clk_spi, *clk_parent;
+
+ clk_spi = devm_clk_get(&pdev->dev, "spi");
+ if (IS_ERR(clk_spi)) {
+ dev_warn(&pdev->dev, "can't get the spi clock\n");
+ clk_spi = NULL;
+ }
+
+ clk_parent = devm_clk_get(&pdev->dev, "source");
+ if (IS_ERR(clk_parent)) {
+ dev_warn(&pdev->dev, "can't get the source clock\n");
+ clk_parent = NULL;
+ }
+
+ ss->clk = devm_clk_get(&pdev->dev, "enable");
+ if (IS_ERR(ss->clk)) {
+ dev_err(&pdev->dev, "can't get the enable clock\n");
+ return PTR_ERR(ss->clk);
+ }
+
+ if (!clk_set_parent(clk_spi, clk_parent))
+ ss->src_clk = clk_get_rate(clk_spi);
+ else
+ ss->src_clk = SPRD_SPI_DEFAULT_SOURCE;
+
+ return 0;
+}
+
+static int sprd_spi_probe(struct platform_device *pdev)
+{
+ struct spi_controller *sctlr;
+ struct resource *res;
+ struct sprd_spi *ss;
+ int ret;
+
+ pdev->id = of_alias_get_id(pdev->dev.of_node, "spi");
+ sctlr = spi_alloc_master(&pdev->dev, sizeof(*ss));
+ if (!sctlr)
+ return -ENOMEM;
+
+ ss = spi_controller_get_devdata(sctlr);
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ ss->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(ss->base)) {
+ ret = PTR_ERR(ss->base);
+ goto free_controller;
+ }
+
+ ss->dev = &pdev->dev;
+ sctlr->dev.of_node = pdev->dev.of_node;
+ sctlr->mode_bits = SPI_CPOL | SPI_CPHA | SPI_3WIRE | SPI_TX_DUAL;
+ sctlr->bus_num = pdev->id;
+ sctlr->set_cs = sprd_spi_chipselect;
+ sctlr->transfer_one = sprd_spi_transfer_one;
+ sctlr->auto_runtime_pm = true;
+ sctlr->max_speed_hz = min_t(u32, ss->src_clk >> 1,
+ SPRD_SPI_MAX_SPEED_HZ);
+
+ platform_set_drvdata(pdev, sctlr);
+ ret = sprd_spi_clk_init(pdev, ss);
+ if (ret)
+ goto free_controller;
+
+ ret = clk_prepare_enable(ss->clk);
+ if (ret)
+ goto free_controller;
+
+ ret = pm_runtime_set_active(&pdev->dev);
+ if (ret < 0)
+ goto disable_clk;
+
+ pm_runtime_set_autosuspend_delay(&pdev->dev,
+ SPRD_SPI_AUTOSUSPEND_DELAY);
+ pm_runtime_use_autosuspend(&pdev->dev);
+ pm_runtime_enable(&pdev->dev);
+ ret = pm_runtime_get_sync(&pdev->dev);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "failed to resume SPI controller\n");
+ goto err_rpm_put;
+ }
+
+ ret = devm_spi_register_controller(&pdev->dev, sctlr);
+ if (ret)
+ goto err_rpm_put;
+
+ pm_runtime_mark_last_busy(&pdev->dev);
+ pm_runtime_put_autosuspend(&pdev->dev);
+
+ return 0;
+
+err_rpm_put:
+ pm_runtime_put_noidle(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
+disable_clk:
+ clk_disable_unprepare(ss->clk);
+free_controller:
+ spi_controller_put(sctlr);
+
+ return ret;
+}
+
+static int sprd_spi_remove(struct platform_device *pdev)
+{
+ struct spi_controller *sctlr = platform_get_drvdata(pdev);
+ struct sprd_spi *ss = spi_controller_get_devdata(sctlr);
+ int ret;
+
+ ret = pm_runtime_get_sync(ss->dev);
+ if (ret < 0) {
+ dev_err(ss->dev, "failed to resume SPI controller\n");
+ return ret;
+ }
+
+ clk_disable_unprepare(ss->clk);
+ pm_runtime_put_noidle(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
+
+ return 0;
+}
+
+static int __maybe_unused sprd_spi_runtime_suspend(struct device *dev)
+{
+ struct spi_controller *sctlr = dev_get_drvdata(dev);
+ struct sprd_spi *ss = spi_controller_get_devdata(sctlr);
+
+ clk_disable_unprepare(ss->clk);
+
+ return 0;
+}
+
+static int __maybe_unused sprd_spi_runtime_resume(struct device *dev)
+{
+ struct spi_controller *sctlr = dev_get_drvdata(dev);
+ struct sprd_spi *ss = spi_controller_get_devdata(sctlr);
+ int ret;
+
+ ret = clk_prepare_enable(ss->clk);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static const struct dev_pm_ops sprd_spi_pm_ops = {
+ SET_RUNTIME_PM_OPS(sprd_spi_runtime_suspend,
+ sprd_spi_runtime_resume, NULL)
+};
+
+static const struct of_device_id sprd_spi_of_match[] = {
+ { .compatible = "sprd,sc9860-spi", },
+ { /* sentinel */ }
+};
+
+static struct platform_driver sprd_spi_driver = {
+ .driver = {
+ .name = "sprd-spi",
+ .of_match_table = sprd_spi_of_match,
+ .pm = &sprd_spi_pm_ops,
+ },
+ .probe = sprd_spi_probe,
+ .remove = sprd_spi_remove,
+};
+
+module_platform_driver(sprd_spi_driver);
+
+MODULE_DESCRIPTION("Spreadtrum SPI Controller driver");
+MODULE_AUTHOR("Lanqing Liu <lanqing.liu@spreadtrum.com>");
+MODULE_LICENSE("GPL v2");
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) STMicroelectronics 2018 - All Rights Reserved
+ * Author: Ludovic Barre <ludovic.barre@st.com> for STMicroelectronics.
+ */
+#include <linux/bitfield.h>
+#include <linux/clk.h>
+#include <linux/errno.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+#include <linux/sizes.h>
+#include <linux/spi/spi-mem.h>
+
+#define QSPI_CR 0x00
+#define CR_EN BIT(0)
+#define CR_ABORT BIT(1)
+#define CR_DMAEN BIT(2)
+#define CR_TCEN BIT(3)
+#define CR_SSHIFT BIT(4)
+#define CR_DFM BIT(6)
+#define CR_FSEL BIT(7)
+#define CR_FTHRES_MASK GENMASK(12, 8)
+#define CR_TEIE BIT(16)
+#define CR_TCIE BIT(17)
+#define CR_FTIE BIT(18)
+#define CR_SMIE BIT(19)
+#define CR_TOIE BIT(20)
+#define CR_PRESC_MASK GENMASK(31, 24)
+
+#define QSPI_DCR 0x04
+#define DCR_FSIZE_MASK GENMASK(20, 16)
+
+#define QSPI_SR 0x08
+#define SR_TEF BIT(0)
+#define SR_TCF BIT(1)
+#define SR_FTF BIT(2)
+#define SR_SMF BIT(3)
+#define SR_TOF BIT(4)
+#define SR_BUSY BIT(5)
+#define SR_FLEVEL_MASK GENMASK(13, 8)
+
+#define QSPI_FCR 0x0c
+#define FCR_CTEF BIT(0)
+#define FCR_CTCF BIT(1)
+
+#define QSPI_DLR 0x10
+
+#define QSPI_CCR 0x14
+#define CCR_INST_MASK GENMASK(7, 0)
+#define CCR_IMODE_MASK GENMASK(9, 8)
+#define CCR_ADMODE_MASK GENMASK(11, 10)
+#define CCR_ADSIZE_MASK GENMASK(13, 12)
+#define CCR_DCYC_MASK GENMASK(22, 18)
+#define CCR_DMODE_MASK GENMASK(25, 24)
+#define CCR_FMODE_MASK GENMASK(27, 26)
+#define CCR_FMODE_INDW (0U << 26)
+#define CCR_FMODE_INDR (1U << 26)
+#define CCR_FMODE_APM (2U << 26)
+#define CCR_FMODE_MM (3U << 26)
+#define CCR_BUSWIDTH_0 0x0
+#define CCR_BUSWIDTH_1 0x1
+#define CCR_BUSWIDTH_2 0x2
+#define CCR_BUSWIDTH_4 0x3
+
+#define QSPI_AR 0x18
+#define QSPI_ABR 0x1c
+#define QSPI_DR 0x20
+#define QSPI_PSMKR 0x24
+#define QSPI_PSMAR 0x28
+#define QSPI_PIR 0x2c
+#define QSPI_LPTR 0x30
+#define LPTR_DFT_TIMEOUT 0x10
+
+#define STM32_QSPI_MAX_MMAP_SZ SZ_256M
+#define STM32_QSPI_MAX_NORCHIP 2
+
+#define STM32_FIFO_TIMEOUT_US 30000
+#define STM32_BUSY_TIMEOUT_US 100000
+#define STM32_ABT_TIMEOUT_US 100000
+
+struct stm32_qspi_flash {
+ struct stm32_qspi *qspi;
+ u32 cs;
+ u32 presc;
+};
+
+struct stm32_qspi {
+ struct device *dev;
+ void __iomem *io_base;
+ void __iomem *mm_base;
+ resource_size_t mm_size;
+ struct clk *clk;
+ u32 clk_rate;
+ struct stm32_qspi_flash flash[STM32_QSPI_MAX_NORCHIP];
+ struct completion data_completion;
+ u32 fmode;
+
+ /*
+ * to protect device configuration, could be different between
+ * 2 flash access (bk1, bk2)
+ */
+ struct mutex lock;
+};
+
+static irqreturn_t stm32_qspi_irq(int irq, void *dev_id)
+{
+ struct stm32_qspi *qspi = (struct stm32_qspi *)dev_id;
+ u32 cr, sr;
+
+ sr = readl_relaxed(qspi->io_base + QSPI_SR);
+
+ if (sr & (SR_TEF | SR_TCF)) {
+ /* disable irq */
+ cr = readl_relaxed(qspi->io_base + QSPI_CR);
+ cr &= ~CR_TCIE & ~CR_TEIE;
+ writel_relaxed(cr, qspi->io_base + QSPI_CR);
+ complete(&qspi->data_completion);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static void stm32_qspi_read_fifo(u8 *val, void __iomem *addr)
+{
+ *val = readb_relaxed(addr);
+}
+
+static void stm32_qspi_write_fifo(u8 *val, void __iomem *addr)
+{
+ writeb_relaxed(*val, addr);
+}
+
+static int stm32_qspi_tx_poll(struct stm32_qspi *qspi,
+ const struct spi_mem_op *op)
+{
+ void (*tx_fifo)(u8 *val, void __iomem *addr);
+ u32 len = op->data.nbytes, sr;
+ u8 *buf;
+ int ret;
+
+ if (op->data.dir == SPI_MEM_DATA_IN) {
+ tx_fifo = stm32_qspi_read_fifo;
+ buf = op->data.buf.in;
+
+ } else {
+ tx_fifo = stm32_qspi_write_fifo;
+ buf = (u8 *)op->data.buf.out;
+ }
+
+ while (len--) {
+ ret = readl_relaxed_poll_timeout_atomic(qspi->io_base + QSPI_SR,
+ sr, (sr & SR_FTF), 1,
+ STM32_FIFO_TIMEOUT_US);
+ if (ret) {
+ dev_err(qspi->dev, "fifo timeout (len:%d stat:%#x)\n",
+ len, sr);
+ return ret;
+ }
+ tx_fifo(buf++, qspi->io_base + QSPI_DR);
+ }
+
+ return 0;
+}
+
+static int stm32_qspi_tx_mm(struct stm32_qspi *qspi,
+ const struct spi_mem_op *op)
+{
+ memcpy_fromio(op->data.buf.in, qspi->mm_base + op->addr.val,
+ op->data.nbytes);
+ return 0;
+}
+
+static int stm32_qspi_tx(struct stm32_qspi *qspi, const struct spi_mem_op *op)
+{
+ if (!op->data.nbytes)
+ return 0;
+
+ if (qspi->fmode == CCR_FMODE_MM)
+ return stm32_qspi_tx_mm(qspi, op);
+
+ return stm32_qspi_tx_poll(qspi, op);
+}
+
+static int stm32_qspi_wait_nobusy(struct stm32_qspi *qspi)
+{
+ u32 sr;
+
+ return readl_relaxed_poll_timeout_atomic(qspi->io_base + QSPI_SR, sr,
+ !(sr & SR_BUSY), 1,
+ STM32_BUSY_TIMEOUT_US);
+}
+
+static int stm32_qspi_wait_cmd(struct stm32_qspi *qspi,
+ const struct spi_mem_op *op)
+{
+ u32 cr, sr;
+ int err = 0;
+
+ if (!op->data.nbytes)
+ return stm32_qspi_wait_nobusy(qspi);
+
+ if (readl_relaxed(qspi->io_base + QSPI_SR) & SR_TCF)
+ goto out;
+
+ reinit_completion(&qspi->data_completion);
+ cr = readl_relaxed(qspi->io_base + QSPI_CR);
+ writel_relaxed(cr | CR_TCIE | CR_TEIE, qspi->io_base + QSPI_CR);
+
+ if (!wait_for_completion_interruptible_timeout(&qspi->data_completion,
+ msecs_to_jiffies(1000))) {
+ err = -ETIMEDOUT;
+ } else {
+ sr = readl_relaxed(qspi->io_base + QSPI_SR);
+ if (sr & SR_TEF)
+ err = -EIO;
+ }
+
+out:
+ /* clear flags */
+ writel_relaxed(FCR_CTCF | FCR_CTEF, qspi->io_base + QSPI_FCR);
+
+ return err;
+}
+
+static int stm32_qspi_get_mode(struct stm32_qspi *qspi, u8 buswidth)
+{
+ if (buswidth == 4)
+ return CCR_BUSWIDTH_4;
+
+ return buswidth;
+}
+
+static int stm32_qspi_send(struct spi_mem *mem, const struct spi_mem_op *op)
+{
+ struct stm32_qspi *qspi = spi_controller_get_devdata(mem->spi->master);
+ struct stm32_qspi_flash *flash = &qspi->flash[mem->spi->chip_select];
+ u32 ccr, cr, addr_max;
+ int timeout, err = 0;
+
+ dev_dbg(qspi->dev, "cmd:%#x mode:%d.%d.%d.%d addr:%#llx len:%#x\n",
+ op->cmd.opcode, op->cmd.buswidth, op->addr.buswidth,
+ op->dummy.buswidth, op->data.buswidth,
+ op->addr.val, op->data.nbytes);
+
+ err = stm32_qspi_wait_nobusy(qspi);
+ if (err)
+ goto abort;
+
+ addr_max = op->addr.val + op->data.nbytes + 1;
+
+ if (op->data.dir == SPI_MEM_DATA_IN) {
+ if (addr_max < qspi->mm_size &&
+ op->addr.buswidth)
+ qspi->fmode = CCR_FMODE_MM;
+ else
+ qspi->fmode = CCR_FMODE_INDR;
+ } else {
+ qspi->fmode = CCR_FMODE_INDW;
+ }
+
+ cr = readl_relaxed(qspi->io_base + QSPI_CR);
+ cr &= ~CR_PRESC_MASK & ~CR_FSEL;
+ cr |= FIELD_PREP(CR_PRESC_MASK, flash->presc);
+ cr |= FIELD_PREP(CR_FSEL, flash->cs);
+ writel_relaxed(cr, qspi->io_base + QSPI_CR);
+
+ if (op->data.nbytes)
+ writel_relaxed(op->data.nbytes - 1,
+ qspi->io_base + QSPI_DLR);
+ else
+ qspi->fmode = CCR_FMODE_INDW;
+
+ ccr = qspi->fmode;
+ ccr |= FIELD_PREP(CCR_INST_MASK, op->cmd.opcode);
+ ccr |= FIELD_PREP(CCR_IMODE_MASK,
+ stm32_qspi_get_mode(qspi, op->cmd.buswidth));
+
+ if (op->addr.nbytes) {
+ ccr |= FIELD_PREP(CCR_ADMODE_MASK,
+ stm32_qspi_get_mode(qspi, op->addr.buswidth));
+ ccr |= FIELD_PREP(CCR_ADSIZE_MASK, op->addr.nbytes - 1);
+ }
+
+ if (op->dummy.buswidth && op->dummy.nbytes)
+ ccr |= FIELD_PREP(CCR_DCYC_MASK,
+ op->dummy.nbytes * 8 / op->dummy.buswidth);
+
+ if (op->data.nbytes) {
+ ccr |= FIELD_PREP(CCR_DMODE_MASK,
+ stm32_qspi_get_mode(qspi, op->data.buswidth));
+ }
+
+ writel_relaxed(ccr, qspi->io_base + QSPI_CCR);
+
+ if (op->addr.nbytes && qspi->fmode != CCR_FMODE_MM)
+ writel_relaxed(op->addr.val, qspi->io_base + QSPI_AR);
+
+ err = stm32_qspi_tx(qspi, op);
+
+ /*
+ * Abort in:
+ * -error case
+ * -read memory map: prefetching must be stopped if we read the last
+ * byte of device (device size - fifo size). like device size is not
+ * knows, the prefetching is always stop.
+ */
+ if (err || qspi->fmode == CCR_FMODE_MM)
+ goto abort;
+
+ /* wait end of tx in indirect mode */
+ err = stm32_qspi_wait_cmd(qspi, op);
+ if (err)
+ goto abort;
+
+ return 0;
+
+abort:
+ cr = readl_relaxed(qspi->io_base + QSPI_CR) | CR_ABORT;
+ writel_relaxed(cr, qspi->io_base + QSPI_CR);
+
+ /* wait clear of abort bit by hw */
+ timeout = readl_relaxed_poll_timeout_atomic(qspi->io_base + QSPI_CR,
+ cr, !(cr & CR_ABORT), 1,
+ STM32_ABT_TIMEOUT_US);
+
+ writel_relaxed(FCR_CTCF, qspi->io_base + QSPI_FCR);
+
+ if (err || timeout)
+ dev_err(qspi->dev, "%s err:%d abort timeout:%d\n",
+ __func__, err, timeout);
+
+ return err;
+}
+
+static int stm32_qspi_exec_op(struct spi_mem *mem, const struct spi_mem_op *op)
+{
+ struct stm32_qspi *qspi = spi_controller_get_devdata(mem->spi->master);
+ int ret;
+
+ mutex_lock(&qspi->lock);
+ ret = stm32_qspi_send(mem, op);
+ mutex_unlock(&qspi->lock);
+
+ return ret;
+}
+
+static int stm32_qspi_setup(struct spi_device *spi)
+{
+ struct spi_controller *ctrl = spi->master;
+ struct stm32_qspi *qspi = spi_controller_get_devdata(ctrl);
+ struct stm32_qspi_flash *flash;
+ u32 cr, presc;
+
+ if (ctrl->busy)
+ return -EBUSY;
+
+ if (!spi->max_speed_hz)
+ return -EINVAL;
+
+ presc = DIV_ROUND_UP(qspi->clk_rate, spi->max_speed_hz) - 1;
+
+ flash = &qspi->flash[spi->chip_select];
+ flash->qspi = qspi;
+ flash->cs = spi->chip_select;
+ flash->presc = presc;
+
+ mutex_lock(&qspi->lock);
+ writel_relaxed(LPTR_DFT_TIMEOUT, qspi->io_base + QSPI_LPTR);
+ cr = FIELD_PREP(CR_FTHRES_MASK, 3) | CR_TCEN | CR_SSHIFT | CR_EN;
+ writel_relaxed(cr, qspi->io_base + QSPI_CR);
+
+ /* set dcr fsize to max address */
+ writel_relaxed(DCR_FSIZE_MASK, qspi->io_base + QSPI_DCR);
+ mutex_unlock(&qspi->lock);
+
+ return 0;
+}
+
+/*
+ * no special host constraint, so use default spi_mem_default_supports_op
+ * to check supported mode.
+ */
+static const struct spi_controller_mem_ops stm32_qspi_mem_ops = {
+ .exec_op = stm32_qspi_exec_op,
+};
+
+static void stm32_qspi_release(struct stm32_qspi *qspi)
+{
+ /* disable qspi */
+ writel_relaxed(0, qspi->io_base + QSPI_CR);
+ mutex_destroy(&qspi->lock);
+ clk_disable_unprepare(qspi->clk);
+}
+
+static int stm32_qspi_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct spi_controller *ctrl;
+ struct reset_control *rstc;
+ struct stm32_qspi *qspi;
+ struct resource *res;
+ int ret, irq;
+
+ ctrl = spi_alloc_master(dev, sizeof(*qspi));
+ if (!ctrl)
+ return -ENOMEM;
+
+ qspi = spi_controller_get_devdata(ctrl);
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "qspi");
+ qspi->io_base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(qspi->io_base))
+ return PTR_ERR(qspi->io_base);
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "qspi_mm");
+ qspi->mm_base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(qspi->mm_base))
+ return PTR_ERR(qspi->mm_base);
+
+ qspi->mm_size = resource_size(res);
+ if (qspi->mm_size > STM32_QSPI_MAX_MMAP_SZ)
+ return -EINVAL;
+
+ irq = platform_get_irq(pdev, 0);
+ ret = devm_request_irq(dev, irq, stm32_qspi_irq, 0,
+ dev_name(dev), qspi);
+ if (ret) {
+ dev_err(dev, "failed to request irq\n");
+ return ret;
+ }
+
+ init_completion(&qspi->data_completion);
+
+ qspi->clk = devm_clk_get(dev, NULL);
+ if (IS_ERR(qspi->clk))
+ return PTR_ERR(qspi->clk);
+
+ qspi->clk_rate = clk_get_rate(qspi->clk);
+ if (!qspi->clk_rate)
+ return -EINVAL;
+
+ ret = clk_prepare_enable(qspi->clk);
+ if (ret) {
+ dev_err(dev, "can not enable the clock\n");
+ return ret;
+ }
+
+ rstc = devm_reset_control_get_exclusive(dev, NULL);
+ if (!IS_ERR(rstc)) {
+ reset_control_assert(rstc);
+ udelay(2);
+ reset_control_deassert(rstc);
+ }
+
+ qspi->dev = dev;
+ platform_set_drvdata(pdev, qspi);
+ mutex_init(&qspi->lock);
+
+ ctrl->mode_bits = SPI_RX_DUAL | SPI_RX_QUAD
+ | SPI_TX_DUAL | SPI_TX_QUAD;
+ ctrl->setup = stm32_qspi_setup;
+ ctrl->bus_num = -1;
+ ctrl->mem_ops = &stm32_qspi_mem_ops;
+ ctrl->num_chipselect = STM32_QSPI_MAX_NORCHIP;
+ ctrl->dev.of_node = dev->of_node;
+
+ ret = devm_spi_register_master(dev, ctrl);
+ if (ret)
+ goto err_spi_register;
+
+ return 0;
+
+err_spi_register:
+ stm32_qspi_release(qspi);
+
+ return ret;
+}
+
+static int stm32_qspi_remove(struct platform_device *pdev)
+{
+ struct stm32_qspi *qspi = platform_get_drvdata(pdev);
+
+ stm32_qspi_release(qspi);
+ return 0;
+}
+
+static const struct of_device_id stm32_qspi_match[] = {
+ {.compatible = "st,stm32f469-qspi"},
+ {}
+};
+MODULE_DEVICE_TABLE(of, stm32_qspi_match);
+
+static struct platform_driver stm32_qspi_driver = {
+ .probe = stm32_qspi_probe,
+ .remove = stm32_qspi_remove,
+ .driver = {
+ .name = "stm32-qspi",
+ .of_match_table = stm32_qspi_match,
+ },
+};
+module_platform_driver(stm32_qspi_driver);
+
+MODULE_AUTHOR("Ludovic Barre <ludovic.barre@st.com>");
+MODULE_DESCRIPTION("STMicroelectronics STM32 quad spi driver");
+MODULE_LICENSE("GPL v2");
+// SPDX-License-Identifier: GPL-2.0-or-later
/*
* SPI init/core code
*
* Copyright (C) 2005 David Brownell
* Copyright (C) 2008 Secret Lab Technologies Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
#include <linux/kernel.h>
spi->controller->cleanup(spi);
spi_controller_put(spi->controller);
+ kfree(spi->driver_override);
kfree(spi);
}
}
static DEVICE_ATTR_RO(modalias);
+static ssize_t driver_override_store(struct device *dev,
+ struct device_attribute *a,
+ const char *buf, size_t count)
+{
+ struct spi_device *spi = to_spi_device(dev);
+ const char *end = memchr(buf, '\n', count);
+ const size_t len = end ? end - buf : count;
+ const char *driver_override, *old;
+
+ /* We need to keep extra room for a newline when displaying value */
+ if (len >= (PAGE_SIZE - 1))
+ return -EINVAL;
+
+ driver_override = kstrndup(buf, len, GFP_KERNEL);
+ if (!driver_override)
+ return -ENOMEM;
+
+ device_lock(dev);
+ old = spi->driver_override;
+ if (len) {
+ spi->driver_override = driver_override;
+ } else {
+ /* Emptry string, disable driver override */
+ spi->driver_override = NULL;
+ kfree(driver_override);
+ }
+ device_unlock(dev);
+ kfree(old);
+
+ return count;
+}
+
+static ssize_t driver_override_show(struct device *dev,
+ struct device_attribute *a, char *buf)
+{
+ const struct spi_device *spi = to_spi_device(dev);
+ ssize_t len;
+
+ device_lock(dev);
+ len = snprintf(buf, PAGE_SIZE, "%s\n", spi->driver_override ? : "");
+ device_unlock(dev);
+ return len;
+}
+static DEVICE_ATTR_RW(driver_override);
+
#define SPI_STATISTICS_ATTRS(field, file) \
static ssize_t spi_controller_##field##_show(struct device *dev, \
struct device_attribute *attr, \
static struct attribute *spi_dev_attrs[] = {
&dev_attr_modalias.attr,
+ &dev_attr_driver_override.attr,
NULL,
};
const struct spi_device *spi = to_spi_device(dev);
const struct spi_driver *sdrv = to_spi_driver(drv);
+ /* Check override first, and if set, only use the named driver */
+ if (spi->driver_override)
+ return strcmp(spi->driver_override, drv->name) == 0;
+
/* Attempt an OF style match */
if (of_driver_match_device(dev, drv))
return 1;
enable = !enable;
if (gpio_is_valid(spi->cs_gpio)) {
- gpio_set_value(spi->cs_gpio, !enable);
+ /* Honour the SPI_NO_CS flag */
+ if (!(spi->mode & SPI_NO_CS))
+ gpio_set_value(spi->cs_gpio, !enable);
/* Some SPI masters need both GPIO CS & slave_select */
if ((spi->controller->flags & SPI_MASTER_GPIO_SS) &&
spi->controller->set_cs)
return -EINVAL;
/* help drivers fail *cleanly* when they need options
* that aren't supported with their current controller
+ * SPI_CS_WORD has a fallback software implementation,
+ * so it is ignored here.
*/
- bad_bits = spi->mode & ~spi->controller->mode_bits;
+ bad_bits = spi->mode & ~(spi->controller->mode_bits | SPI_CS_WORD);
ugly_bits = bad_bits &
(SPI_TX_DUAL | SPI_TX_QUAD | SPI_RX_DUAL | SPI_RX_QUAD);
if (ugly_bits) {
if (list_empty(&message->transfers))
return -EINVAL;
+ /* If an SPI controller does not support toggling the CS line on each
+ * transfer (indicated by the SPI_CS_WORD flag) or we are using a GPIO
+ * for the CS line, we can emulate the CS-per-word hardware function by
+ * splitting transfers into one-word transfers and ensuring that
+ * cs_change is set for each transfer.
+ */
+ if ((spi->mode & SPI_CS_WORD) && (!(ctlr->mode_bits & SPI_CS_WORD) ||
+ gpio_is_valid(spi->cs_gpio))) {
+ size_t maxsize;
+ int ret;
+
+ maxsize = (spi->bits_per_word + 7) / 8;
+
+ /* spi_split_transfers_maxsize() requires message->spi */
+ message->spi = spi;
+
+ ret = spi_split_transfers_maxsize(ctlr, message, maxsize,
+ GFP_KERNEL);
+ if (ret)
+ return ret;
+
+ list_for_each_entry(xfer, &message->transfers, transfer_list) {
+ /* don't change cs_change on the last entry in the list */
+ if (list_is_last(&xfer->transfer_list, &message->transfers))
+ break;
+ xfer->cs_change = 1;
+ }
+ }
+
/* Half-duplex links include original MicroWire, and ones with
* only one data pin like SPI_3WIRE (switches direction) or where
* either MOSI or MISO is missing. They can also be caused by
/*-------------------------------------------------------------------------*/
-#if IS_ENABLED(CONFIG_OF_DYNAMIC)
+#if IS_ENABLED(CONFIG_OF)
static int __spi_of_device_match(struct device *dev, void *data)
{
return dev->of_node == data;
}
/* must call put_device() when done with returned spi_device device */
-static struct spi_device *of_find_spi_device_by_node(struct device_node *node)
+struct spi_device *of_find_spi_device_by_node(struct device_node *node)
{
struct device *dev = bus_find_device(&spi_bus_type, NULL, node,
__spi_of_device_match);
return dev ? to_spi_device(dev) : NULL;
}
+EXPORT_SYMBOL_GPL(of_find_spi_device_by_node);
+#endif /* IS_ENABLED(CONFIG_OF) */
+#if IS_ENABLED(CONFIG_OF_DYNAMIC)
static int __spi_of_controller_match(struct device *dev, const void *data)
{
return dev->of_node == data;
{ .compatible = "lineartechnology,ltc2488" },
{ .compatible = "ge,achc" },
{ .compatible = "semtech,sx1301" },
+ { .compatible = "lwn,bk4" },
{},
};
MODULE_DEVICE_TABLE(of, spidev_dt_ids);
* compatible string, it is a Linux implementation thing
* rather than a description of the hardware.
*/
- if (spi->dev.of_node && !of_match_device(spidev_dt_ids, &spi->dev)) {
- dev_err(&spi->dev, "buggy DT: spidev listed directly in DT\n");
- WARN_ON(spi->dev.of_node &&
- !of_match_device(spidev_dt_ids, &spi->dev));
- }
+ WARN(spi->dev.of_node &&
+ of_device_is_compatible(spi->dev.of_node, "spidev"),
+ "%pOF: buggy DT: spidev listed directly in DT\n", spi->dev.of_node);
spidev_probe_acpi(spi);
long mask)
{
struct ad7606_state *st = iio_priv(indio_dev);
- int values[3];
+ DECLARE_BITMAP(values, 3);
int ret, i;
switch (mask) {
if (ret < 0)
return ret;
- values[0] = (ret >> 0) & 1;
- values[1] = (ret >> 1) & 1;
- values[2] = (ret >> 2) & 1;
+ values[0] = ret;
mutex_lock(&st->lock);
- gpiod_set_array_value(ARRAY_SIZE(values), st->gpio_os->desc,
+ gpiod_set_array_value(3, st->gpio_os->desc, st->gpio_os->info,
values);
st->oversampling = val;
mutex_unlock(&st->lock);
}
#ifdef CONFIG_MTD_SPINAND_ONDIEECC
-static int spinand_write_page_hwecc(struct mtd_info *mtd,
- struct nand_chip *chip,
+static int spinand_write_page_hwecc(struct nand_chip *chip,
const u8 *buf, int oob_required,
int page)
{
return nand_prog_page_op(chip, page, 0, p, eccsize * eccsteps);
}
-static int spinand_read_page_hwecc(struct mtd_info *mtd, struct nand_chip *chip,
- u8 *buf, int oob_required, int page)
+static int spinand_read_page_hwecc(struct nand_chip *chip, u8 *buf,
+ int oob_required, int page)
{
int retval;
u8 status;
u8 *p = buf;
int eccsize = chip->ecc.size;
int eccsteps = chip->ecc.steps;
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct spinand_info *info = nand_get_controller_data(chip);
enable_read_hw_ecc = 1;
nand_read_page_op(chip, page, 0, p, eccsize * eccsteps);
if (oob_required)
- chip->read_buf(mtd, chip->oob_poi, mtd->oobsize);
+ chip->legacy.read_buf(chip, chip->oob_poi, mtd->oobsize);
while (1) {
retval = spinand_read_status(info->spi, &status);
}
#endif
-static void spinand_select_chip(struct mtd_info *mtd, int dev)
+static void spinand_select_chip(struct nand_chip *chip, int dev)
{
}
-static u8 spinand_read_byte(struct mtd_info *mtd)
+static u8 spinand_read_byte(struct nand_chip *chip)
{
- struct spinand_state *state = mtd_to_state(mtd);
+ struct spinand_state *state = mtd_to_state(nand_to_mtd(chip));
u8 data;
data = state->buf[state->buf_ptr];
return data;
}
-static int spinand_wait(struct mtd_info *mtd, struct nand_chip *chip)
+static int spinand_wait(struct nand_chip *chip)
{
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct spinand_info *info = nand_get_controller_data(chip);
unsigned long timeo = jiffies;
return 0;
}
-static void spinand_write_buf(struct mtd_info *mtd, const u8 *buf, int len)
+static void spinand_write_buf(struct nand_chip *chip, const u8 *buf, int len)
{
- struct spinand_state *state = mtd_to_state(mtd);
+ struct spinand_state *state = mtd_to_state(nand_to_mtd(chip));
memcpy(state->buf + state->buf_ptr, buf, len);
state->buf_ptr += len;
}
-static void spinand_read_buf(struct mtd_info *mtd, u8 *buf, int len)
+static void spinand_read_buf(struct nand_chip *chip, u8 *buf, int len)
{
- struct spinand_state *state = mtd_to_state(mtd);
+ struct spinand_state *state = mtd_to_state(nand_to_mtd(chip));
memcpy(buf, state->buf + state->buf_ptr, len);
state->buf_ptr += len;
dev_err(&spi_nand->dev, "wait timedout!\n");
}
-static void spinand_cmdfunc(struct mtd_info *mtd, unsigned int command,
+static void spinand_cmdfunc(struct nand_chip *chip, unsigned int command,
int column, int page)
{
- struct nand_chip *chip = mtd_to_nand(mtd);
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct spinand_info *info = nand_get_controller_data(chip);
struct spinand_state *state = info->priv;
nand_set_flash_node(chip, spi_nand->dev.of_node);
nand_set_controller_data(chip, info);
- chip->read_buf = spinand_read_buf;
- chip->write_buf = spinand_write_buf;
- chip->read_byte = spinand_read_byte;
- chip->cmdfunc = spinand_cmdfunc;
- chip->waitfunc = spinand_wait;
+ chip->legacy.read_buf = spinand_read_buf;
+ chip->legacy.write_buf = spinand_write_buf;
+ chip->legacy.read_byte = spinand_read_byte;
+ chip->legacy.cmdfunc = spinand_cmdfunc;
+ chip->legacy.waitfunc = spinand_wait;
chip->options |= NAND_CACHEPRG;
chip->select_chip = spinand_select_chip;
- chip->set_features = nand_get_set_features_notsupp;
- chip->get_features = nand_get_set_features_notsupp;
+ chip->legacy.set_features = nand_get_set_features_notsupp;
+ chip->legacy.get_features = nand_get_set_features_notsupp;
mtd = nand_to_mtd(chip);
mtd_set_ooblayout(mtd, &spinand_oob_64_ops);
#endif
- if (nand_scan(mtd, 1))
+ if (nand_scan(chip, 1))
return -ENXIO;
return mtd_device_register(mtd, NULL, 0);
/*
* The unmap_zeroes_data set means that the underlying device supports
- * REQ_DISCARD and has the discard_zeroes_data bit set. This satisfies
- * the SBC requirements for LBPRZ, meaning that a subsequent read
- * will return zeroes after an UNMAP or WRITE SAME (16) to an LBA
+ * REQ_OP_DISCARD and has the discard_zeroes_data bit set. This
+ * satisfies the SBC requirements for LBPRZ, meaning that a subsequent
+ * read will return zeroes after an UNMAP or WRITE SAME (16) to an LBA
* See sbc4r36 6.6.4.
*/
if (((dev->dev_attrib.emulate_tpu != 0) ||
}
static const struct x86_cpu_id soc_thermal_ids[] = {
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT1, 0,
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT, 0,
BYT_SOC_DTS_APIC_IRQ},
{}
};
geni_se_init(&port->se, port->rx_wm, port->rx_rfr);
geni_se_select_mode(&port->se, port->xfer_mode);
if (!uart_console(uport)) {
- port->rx_fifo = devm_kzalloc(uport->dev,
- port->rx_fifo_depth * sizeof(u32), GFP_KERNEL);
+ port->rx_fifo = devm_kcalloc(uport->dev,
+ port->rx_fifo_depth, sizeof(u32), GFP_KERNEL);
if (!port->rx_fifo)
return -ENOMEM;
}
{
enum mctrl_gpio_idx i;
struct gpio_desc *desc_array[UART_GPIO_MAX];
- int value_array[UART_GPIO_MAX];
+ DECLARE_BITMAP(values, UART_GPIO_MAX);
unsigned int count = 0;
if (gpios == NULL)
for (i = 0; i < UART_GPIO_MAX; i++)
if (gpios->gpio[i] && mctrl_gpios_desc[i].dir_out) {
desc_array[count] = gpios->gpio[i];
- value_array[count] = !!(mctrl & mctrl_gpios_desc[i].mctrl);
+ __assign_bit(count, values,
+ mctrl & mctrl_gpios_desc[i].mctrl);
count++;
}
- gpiod_set_array_value(count, desc_array, value_array);
+ gpiod_set_array_value(count, desc_array, NULL, values);
}
EXPORT_SYMBOL_GPL(mctrl_gpio_set);
if (difference & ACM_CTRL_DSR)
acm->iocount.dsr++;
- if (difference & ACM_CTRL_BRK)
- acm->iocount.brk++;
- if (difference & ACM_CTRL_RI)
- acm->iocount.rng++;
if (difference & ACM_CTRL_DCD)
acm->iocount.dcd++;
- if (difference & ACM_CTRL_FRAMING)
+ if (newctrl & ACM_CTRL_BRK)
+ acm->iocount.brk++;
+ if (newctrl & ACM_CTRL_RI)
+ acm->iocount.rng++;
+ if (newctrl & ACM_CTRL_FRAMING)
acm->iocount.frame++;
- if (difference & ACM_CTRL_PARITY)
+ if (newctrl & ACM_CTRL_PARITY)
acm->iocount.parity++;
- if (difference & ACM_CTRL_OVERRUN)
+ if (newctrl & ACM_CTRL_OVERRUN)
acm->iocount.overrun++;
spin_unlock_irqrestore(&acm->read_lock, flags);
case -ENOENT:
case -ESHUTDOWN:
/* this urb is terminated, clean up */
- acm->nb_index = 0;
dev_dbg(&acm->control->dev,
"%s - urb shutting down with status: %d\n",
__func__, status);
struct acm *acm = usb_get_intfdata(intf);
clear_bit(EVENT_RX_STALL, &acm->flags);
+ acm->nb_index = 0; /* pending control transfers are lost */
return 0;
}
u = 0;
switch (uurb->type) {
case USBDEVFS_URB_TYPE_CONTROL:
- if (is_in)
- allow_short = true;
if (!usb_endpoint_xfer_control(&ep->desc))
return -EINVAL;
/* min 8 byte setup packet */
is_in = 0;
uurb->endpoint &= ~USB_DIR_IN;
}
+ if (is_in)
+ allow_short = true;
snoop(&ps->dev->dev, "control urb: bRequestType=%02x "
"bRequest=%02x wValue=%04x "
"wIndex=%04x wLength=%04x\n",
#include <linux/usb/gadget.h>
#include <linux/usb/composite.h>
+#include <linux/nospec.h>
+
#include "configfs.h"
fsg_opts = to_fsg_opts(&group->cg_item);
if (num >= FSG_MAX_LUNS)
return ERR_PTR(-ERANGE);
+ num = array_index_nospec(num, FSG_MAX_LUNS);
mutex_lock(&fsg_opts->lock);
if (fsg_opts->refcnt || fsg_opts->common->luns[num]) {
xhci->quirks |= XHCI_PME_STUCK_QUIRK;
}
if (pdev->vendor == PCI_VENDOR_ID_INTEL &&
- pdev->device == PCI_DEVICE_ID_INTEL_CHERRYVIEW_XHCI) {
+ pdev->device == PCI_DEVICE_ID_INTEL_CHERRYVIEW_XHCI)
xhci->quirks |= XHCI_SSIC_PORT_UNUSED;
+ if (pdev->vendor == PCI_VENDOR_ID_INTEL &&
+ (pdev->device == PCI_DEVICE_ID_INTEL_CHERRYVIEW_XHCI ||
+ pdev->device == PCI_DEVICE_ID_INTEL_APL_XHCI))
xhci->quirks |= XHCI_INTEL_USB_ROLE_SW;
- }
if (pdev->vendor == PCI_VENDOR_ID_INTEL &&
(pdev->device == PCI_DEVICE_ID_INTEL_CHERRYVIEW_XHCI ||
pdev->device == PCI_DEVICE_ID_INTEL_SUNRISEPOINT_LP_XHCI ||
{
struct intel_xhci_usb_data *data = platform_get_drvdata(pdev);
+ pm_runtime_disable(&pdev->dev);
+
usb_role_switch_unregister(data->role_sw);
return 0;
}
struct vhci_hcd *vhci_hcd;
struct vhci *vhci;
int retval = 0;
- int rhport;
+ int rhport = -1;
unsigned long flags;
+ bool invalid_rhport = false;
u32 prev_port_status[VHCI_HC_PORTS];
usbip_dbg_vhci_rh("typeReq %x wValue %x wIndex %x\n", typeReq, wValue,
wIndex);
- if (wIndex > VHCI_HC_PORTS)
- pr_err("invalid port number %d\n", wIndex);
- rhport = wIndex - 1;
+ /*
+ * wIndex can be 0 for some request types (typeReq). rhport is
+ * in valid range when wIndex >= 1 and < VHCI_HC_PORTS.
+ *
+ * Reference port_status[] only with valid rhport when
+ * invalid_rhport is false.
+ */
+ if (wIndex < 1 || wIndex > VHCI_HC_PORTS) {
+ invalid_rhport = true;
+ if (wIndex > VHCI_HC_PORTS)
+ pr_err("invalid port number %d\n", wIndex);
+ } else
+ rhport = wIndex - 1;
vhci_hcd = hcd_to_vhci_hcd(hcd);
vhci = vhci_hcd->vhci;
/* store old status and compare now and old later */
if (usbip_dbg_flag_vhci_rh) {
- memcpy(prev_port_status, vhci_hcd->port_status,
- sizeof(prev_port_status));
+ if (!invalid_rhport)
+ memcpy(prev_port_status, vhci_hcd->port_status,
+ sizeof(prev_port_status));
}
switch (typeReq) {
usbip_dbg_vhci_rh(" ClearHubFeature\n");
break;
case ClearPortFeature:
- if (rhport < 0)
+ if (invalid_rhport) {
+ pr_err("invalid port number %d\n", wIndex);
goto error;
+ }
switch (wValue) {
case USB_PORT_FEAT_SUSPEND:
if (hcd->speed == HCD_USB3) {
break;
case GetPortStatus:
usbip_dbg_vhci_rh(" GetPortStatus port %x\n", wIndex);
- if (wIndex < 1) {
+ if (invalid_rhport) {
pr_err("invalid port number %d\n", wIndex);
retval = -EPIPE;
+ goto error;
}
/* we do not care about resume. */
goto error;
}
- if (rhport < 0)
+ if (invalid_rhport) {
+ pr_err("invalid port number %d\n", wIndex);
goto error;
+ }
vhci_hcd->port_status[rhport] |= USB_PORT_STAT_SUSPEND;
break;
case USB_PORT_FEAT_POWER:
usbip_dbg_vhci_rh(
" SetPortFeature: USB_PORT_FEAT_POWER\n");
- if (rhport < 0)
+ if (invalid_rhport) {
+ pr_err("invalid port number %d\n", wIndex);
goto error;
+ }
if (hcd->speed == HCD_USB3)
vhci_hcd->port_status[rhport] |= USB_SS_PORT_STAT_POWER;
else
case USB_PORT_FEAT_BH_PORT_RESET:
usbip_dbg_vhci_rh(
" SetPortFeature: USB_PORT_FEAT_BH_PORT_RESET\n");
- if (rhport < 0)
+ if (invalid_rhport) {
+ pr_err("invalid port number %d\n", wIndex);
goto error;
+ }
/* Applicable only for USB3.0 hub */
if (hcd->speed != HCD_USB3) {
pr_err("USB_PORT_FEAT_BH_PORT_RESET req not "
case USB_PORT_FEAT_RESET:
usbip_dbg_vhci_rh(
" SetPortFeature: USB_PORT_FEAT_RESET\n");
- if (rhport < 0)
+ if (invalid_rhport) {
+ pr_err("invalid port number %d\n", wIndex);
goto error;
+ }
/* if it's already enabled, disable */
if (hcd->speed == HCD_USB3) {
vhci_hcd->port_status[rhport] = 0;
default:
usbip_dbg_vhci_rh(" SetPortFeature: default %d\n",
wValue);
- if (rhport < 0)
+ if (invalid_rhport) {
+ pr_err("invalid port number %d\n", wIndex);
goto error;
+ }
if (hcd->speed == HCD_USB3) {
if ((vhci_hcd->port_status[rhport] &
USB_SS_PORT_STAT_POWER) != 0) {
if (usbip_dbg_flag_vhci_rh) {
pr_debug("port %d\n", rhport);
/* Only dump valid port status */
- if (rhport >= 0) {
+ if (!invalid_rhport) {
dump_port_status_diff(prev_port_status[rhport],
vhci_hcd->port_status[rhport],
hcd->speed == HCD_USB3);
spin_unlock_irqrestore(&vhci->lock, flags);
- if ((vhci_hcd->port_status[rhport] & PORT_C_MASK) != 0)
+ if (!invalid_rhport &&
+ (vhci_hcd->port_status[rhport] & PORT_C_MASK) != 0) {
usb_hcd_poll_rh_status(hcd);
+ }
return retval;
}
extern void aty_set_pll_ct(const struct fb_info *info, const union aty_pll *pll);
extern u8 aty_ld_pll_ct(int offset, const struct atyfb_par *par);
+extern const u8 aty_postdividers[8];
+
/*
* Hardware cursor support
extern void aty_reset_engine(const struct atyfb_par *par);
extern void aty_init_engine(struct atyfb_par *par, struct fb_info *info);
-extern u8 aty_ld_pll_ct(int offset, const struct atyfb_par *par);
void atyfb_copyarea(struct fb_info *info, const struct fb_copyarea *area);
void atyfb_fillrect(struct fb_info *info, const struct fb_fillrect *rect);
/*
* PLL Reference Divider M:
*/
- M = pll_regs[2];
+ M = pll_regs[PLL_REF_DIV];
/*
* PLL Feedback Divider N (Dependent on CLOCK_CNTL):
*/
- N = pll_regs[7 + (clock_cntl & 3)];
+ N = pll_regs[VCLK0_FB_DIV + (clock_cntl & 3)];
/*
* PLL Post Divider P (Dependent on CLOCK_CNTL):
*/
- P = 1 << (pll_regs[6] >> ((clock_cntl & 3) << 1));
+ P = aty_postdividers[((pll_regs[VCLK_POST_DIV] >> ((clock_cntl & 3) << 1)) & 3) |
+ ((pll_regs[PLL_EXT_CNTL] >> (2 + (clock_cntl & 3))) & 4)];
/*
* PLL Divider Q:
*/
#define Maximum_DSP_PRECISION 7
-static u8 postdividers[] = {1,2,4,8,3};
+const u8 aty_postdividers[8] = {1,2,4,8,3,5,6,12};
static int aty_dsp_gt(const struct fb_info *info, u32 bpp, struct pll_ct *pll)
{
pll->vclk_post_div += (q < 64*8);
pll->vclk_post_div += (q < 32*8);
}
- pll->vclk_post_div_real = postdividers[pll->vclk_post_div];
+ pll->vclk_post_div_real = aty_postdividers[pll->vclk_post_div];
// pll->vclk_post_div <<= 6;
pll->vclk_fb_div = q * pll->vclk_post_div_real / 8;
pllvclk = (1000000 * 2 * pll->vclk_fb_div) /
u8 mclk_fb_div, pll_ext_cntl;
pll->ct.pll_ref_div = aty_ld_pll_ct(PLL_REF_DIV, par);
pll_ext_cntl = aty_ld_pll_ct(PLL_EXT_CNTL, par);
- pll->ct.xclk_post_div_real = postdividers[pll_ext_cntl & 0x07];
+ pll->ct.xclk_post_div_real = aty_postdividers[pll_ext_cntl & 0x07];
mclk_fb_div = aty_ld_pll_ct(MCLK_FB_DIV, par);
if (pll_ext_cntl & PLL_MFB_TIMES_4_2B)
mclk_fb_div <<= 1;
xpost_div += (q < 64*8);
xpost_div += (q < 32*8);
}
- pll->ct.xclk_post_div_real = postdividers[xpost_div];
+ pll->ct.xclk_post_div_real = aty_postdividers[xpost_div];
pll->ct.mclk_fb_div = q * pll->ct.xclk_post_div_real / 8;
#ifdef CONFIG_PPC
mpost_div += (q < 64*8);
mpost_div += (q < 32*8);
}
- sclk_post_div_real = postdividers[mpost_div];
+ sclk_post_div_real = aty_postdividers[mpost_div];
pll->ct.sclk_fb_div = q * sclk_post_div_real / 8;
pll->ct.spll_cntl2 = mpost_div << 4;
#ifdef DEBUG
// SPDX-License-Identifier: GPL-2.0
#include <linux/bio.h>
-#include <linux/io.h>
#include <linux/export.h>
+#include <xen/xen.h>
#include <xen/page.h>
bool xen_biovec_phys_mergeable(const struct bio_vec *vec1,
return false;
#endif
}
-EXPORT_SYMBOL(xen_biovec_phys_mergeable);
*/
flags &= ~(__GFP_DMA | __GFP_HIGHMEM);
+ /* Convert the size to actually allocated. */
+ size = 1UL << (order + XEN_PAGE_SHIFT);
+
/* On ARM this function returns an ioremap'ped virtual address for
* which virt_to_phys doesn't return the corresponding physical
* address. In fact on ARM virt_to_phys only works for kernel direct
* physical address */
phys = xen_bus_to_phys(dev_addr);
+ /* Convert the size to actually allocated. */
+ size = 1UL << (order + XEN_PAGE_SHIFT);
+
if (((dev_addr + size - 1 <= dma_mask)) ||
range_straddles_page_boundary(phys, size))
xen_destroy_contiguous_region(phys, order);
return xen_get_dma_ops(dev)->mmap(dev, vma, cpu_addr,
dma_addr, size, attrs);
#endif
- return dma_common_mmap(dev, vma, cpu_addr, dma_addr, size);
+ return dma_common_mmap(dev, vma, cpu_addr, dma_addr, size, attrs);
}
/*
handle, size, attrs);
}
#endif
- return dma_common_get_sgtable(dev, sgt, cpu_addr, handle, size);
+ return dma_common_get_sgtable(dev, sgt, cpu_addr, handle, size, attrs);
}
static int xen_swiotlb_mapping_error(struct device *dev, dma_addr_t dma_addr)
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/acpi.h>
+#include <xen/xen.h>
#include <xen/interface/version.h>
#include <xen/xen-ops.h>
#include <asm/xen/hypercall.h>
*/
static int afs_activate_cell(struct afs_net *net, struct afs_cell *cell)
{
+ struct hlist_node **p;
+ struct afs_cell *pcell;
int ret;
if (!cell->anonymous_key) {
return ret;
mutex_lock(&net->proc_cells_lock);
- list_add_tail(&cell->proc_link, &net->proc_cells);
+ for (p = &net->proc_cells.first; *p; p = &(*p)->next) {
+ pcell = hlist_entry(*p, struct afs_cell, proc_link);
+ if (strcmp(cell->name, pcell->name) < 0)
+ break;
+ }
+
+ cell->proc_link.pprev = p;
+ cell->proc_link.next = *p;
+ rcu_assign_pointer(*p, &cell->proc_link.next);
+ if (cell->proc_link.next)
+ cell->proc_link.next->pprev = &cell->proc_link.next;
+
afs_dynroot_mkdir(net, cell);
mutex_unlock(&net->proc_cells_lock);
return 0;
afs_proc_cell_remove(cell);
mutex_lock(&net->proc_cells_lock);
- list_del_init(&cell->proc_link);
+ hlist_del_rcu(&cell->proc_link);
afs_dynroot_rmdir(net, cell);
mutex_unlock(&net->proc_cells_lock);
return -ERESTARTSYS;
net->dynroot_sb = sb;
- list_for_each_entry(cell, &net->proc_cells, proc_link) {
+ hlist_for_each_entry(cell, &net->proc_cells, proc_link) {
ret = afs_dynroot_mkdir(net, cell);
if (ret < 0)
goto error;
seqlock_t cells_lock;
struct mutex proc_cells_lock;
- struct list_head proc_cells;
+ struct hlist_head proc_cells;
/* Known servers. Theoretically each fileserver can only be in one
* cell, but in practice, people create aliases and subsets and there's
struct afs_net *net;
struct key *anonymous_key; /* anonymous user key for this cell */
struct work_struct manager; /* Manager for init/deinit/dns */
- struct list_head proc_link; /* /proc cell list link */
+ struct hlist_node proc_link; /* /proc cell list link */
#ifdef CONFIG_AFS_FSCACHE
struct fscache_cookie *cache; /* caching cookie */
#endif
timer_setup(&net->cells_timer, afs_cells_timer, 0);
mutex_init(&net->proc_cells_lock);
- INIT_LIST_HEAD(&net->proc_cells);
+ INIT_HLIST_HEAD(&net->proc_cells);
seqlock_init(&net->fs_lock);
net->fs_servers = RB_ROOT;
static int afs_proc_cells_show(struct seq_file *m, void *v)
{
struct afs_cell *cell = list_entry(v, struct afs_cell, proc_link);
- struct afs_net *net = afs_seq2net(m);
- if (v == &net->proc_cells) {
+ if (v == SEQ_START_TOKEN) {
/* display header on line 1 */
seq_puts(m, "USE NAME\n");
return 0;
__acquires(rcu)
{
rcu_read_lock();
- return seq_list_start_head(&afs_seq2net(m)->proc_cells, *_pos);
+ return seq_hlist_start_head_rcu(&afs_seq2net(m)->proc_cells, *_pos);
}
static void *afs_proc_cells_next(struct seq_file *m, void *v, loff_t *pos)
{
- return seq_list_next(v, &afs_seq2net(m)->proc_cells, pos);
+ return seq_hlist_next_rcu(v, &afs_seq2net(m)->proc_cells, pos);
}
static void afs_proc_cells_stop(struct seq_file *m, void *v)
}
if (call->state == AFS_CALL_COMPLETE) {
- call->reply[0] = NULL;
-
/* We have two refs to release - one from the alloc and one
* queued with the work item - and we can't just deallocate the
* call because the work item may be queued again.
*/
bio = bio_alloc(GFP_NOIO, 1);
- if (wbc) {
- wbc_init_bio(wbc, bio);
- wbc_account_io(wbc, bh->b_page, bh->b_size);
- }
-
bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9);
bio_set_dev(bio, bh->b_bdev);
bio->bi_write_hint = write_hint;
op_flags |= REQ_PRIO;
bio_set_op_attrs(bio, op, op_flags);
+ if (wbc) {
+ wbc_init_bio(wbc, bio);
+ wbc_account_io(wbc, bh->b_page, bh->b_size);
+ }
+
submit_bio(bio);
return 0;
}
trap = lock_rename(cache->graveyard, dir);
/* do some checks before getting the grave dentry */
- if (rep->d_parent != dir) {
+ if (rep->d_parent != dir || IS_DEADDIR(d_inode(rep))) {
/* the entry was probably culled when we dropped the parent dir
* lock */
unlock_rename(cache->graveyard, dir);
while (index < end && pagevec_lookup_entries(&pvec, mapping, index,
min(end - index, (pgoff_t)PAGEVEC_SIZE),
indices)) {
+ pgoff_t nr_pages = 1;
+
for (i = 0; i < pagevec_count(&pvec); i++) {
struct page *pvec_ent = pvec.pages[i];
void *entry;
xa_lock_irq(&mapping->i_pages);
entry = get_unlocked_mapping_entry(mapping, index, NULL);
- if (entry)
+ if (entry) {
page = dax_busy_page(entry);
+ /*
+ * Account for multi-order entries at
+ * the end of the pagevec.
+ */
+ if (i + 1 >= pagevec_count(&pvec))
+ nr_pages = 1UL << dax_radix_order(entry);
+ }
put_unlocked_mapping_entry(mapping, index, entry);
xa_unlock_irq(&mapping->i_pages);
if (page)
*/
pagevec_remove_exceptionals(&pvec);
pagevec_release(&pvec);
- index++;
+ index += nr_pages;
if (page)
break;
bio = bio_alloc(GFP_NOIO, BIO_MAX_PAGES);
if (!bio)
return -ENOMEM;
- wbc_init_bio(io->io_wbc, bio);
bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9);
bio_set_dev(bio, bh->b_bdev);
bio->bi_end_io = ext4_end_bio;
bio->bi_private = ext4_get_io_end(io->io_end);
io->io_bio = bio;
io->io_next_block = bh->b_blocknr;
+ wbc_init_bio(io->io_wbc, bio);
return 0;
}
if (ops->ent_get(&fatent) == FAT_ENT_FREE)
free++;
} while (fat_ent_next(sbi, &fatent));
+ cond_resched();
}
sbi->free_clusters = free;
sbi->free_clus_valid = 1;
}
/*
- * initialise an cookie jar slab element prior to any use
- */
-void fscache_cookie_init_once(void *_cookie)
-{
- struct fscache_cookie *cookie = _cookie;
-
- memset(cookie, 0, sizeof(*cookie));
- spin_lock_init(&cookie->lock);
- spin_lock_init(&cookie->stores_lock);
- INIT_HLIST_HEAD(&cookie->backing_objects);
-}
-
-/*
- * Set the index key in a cookie. The cookie struct has space for a 12-byte
+ * Set the index key in a cookie. The cookie struct has space for a 16-byte
* key plus length and hash, but if that's not big enough, it's instead a
* pointer to a buffer containing 3 bytes of hash, 1 byte of length and then
* the key data.
{
unsigned long long h;
u32 *buf;
+ int bufs;
int i;
- cookie->key_len = index_key_len;
+ bufs = DIV_ROUND_UP(index_key_len, sizeof(*buf));
if (index_key_len > sizeof(cookie->inline_key)) {
- buf = kzalloc(index_key_len, GFP_KERNEL);
+ buf = kcalloc(bufs, sizeof(*buf), GFP_KERNEL);
if (!buf)
return -ENOMEM;
cookie->key = buf;
} else {
buf = (u32 *)cookie->inline_key;
- buf[0] = 0;
- buf[1] = 0;
- buf[2] = 0;
}
memcpy(buf, index_key, index_key_len);
*/
h = (unsigned long)cookie->parent;
h += index_key_len + cookie->type;
- for (i = 0; i < (index_key_len + sizeof(u32) - 1) / sizeof(u32); i++)
+
+ for (i = 0; i < bufs; i++)
h += buf[i];
cookie->key_hash = h ^ (h >> 32);
struct fscache_cookie *cookie;
/* allocate and initialise a cookie */
- cookie = kmem_cache_alloc(fscache_cookie_jar, GFP_KERNEL);
+ cookie = kmem_cache_zalloc(fscache_cookie_jar, GFP_KERNEL);
if (!cookie)
return NULL;
cookie->netfs_data = netfs_data;
cookie->flags = (1 << FSCACHE_COOKIE_NO_DATA_YET);
cookie->type = def->type;
+ spin_lock_init(&cookie->lock);
+ spin_lock_init(&cookie->stores_lock);
+ INIT_HLIST_HEAD(&cookie->backing_objects);
/* radix tree insertion won't use the preallocation pool unless it's
* told it may not wait */
extern struct kmem_cache *fscache_cookie_jar;
extern void fscache_free_cookie(struct fscache_cookie *);
-extern void fscache_cookie_init_once(void *);
extern struct fscache_cookie *fscache_alloc_cookie(struct fscache_cookie *,
const struct fscache_cookie_def *,
const void *, size_t,
fscache_cookie_jar = kmem_cache_create("fscache_cookie_jar",
sizeof(struct fscache_cookie),
- 0,
- 0,
- fscache_cookie_init_once);
+ 0, 0, NULL);
if (!fscache_cookie_jar) {
pr_notice("Failed to allocate a cookie jar\n");
ret = -ENOMEM;
}
}
release_metapath(&mp);
- if (gfs2_is_jdata(ip))
+ if (!gfs2_is_stuffed(ip) && gfs2_is_jdata(ip))
iomap->page_done = gfs2_iomap_journaled_page_done;
return 0;
sb->s_fs_info = c;
ret = jffs2_parse_options(c, data);
- if (ret) {
- kfree(c);
+ if (ret)
return -EINVAL;
- }
/* Initialize JFFS2 superblock locks, the further initialization will
* be done later */
if (!access_ok(VERIFY_READ, from, n))
return n;
+ current->kernel_uaccess_faults_ok++;
while (n) {
if (__get_user(c, f)) {
memset(t, 0, n);
f++;
n--;
}
+ current->kernel_uaccess_faults_ok--;
return n;
}
};
/* Lockdep class keys */
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
static struct lock_class_key lockdep_keys[OCFS2_NUM_LOCK_TYPES];
+#endif
static int ocfs2_check_meta_downconvert(struct ocfs2_lock_res *lockres,
int new_level);
static void __exit ubifs_exit(void)
{
- WARN_ON(list_empty(&ubifs_infos));
- WARN_ON(atomic_long_read(&ubifs_clean_zn_cnt) == 0);
+ WARN_ON(!list_empty(&ubifs_infos));
+ WARN_ON(atomic_long_read(&ubifs_clean_zn_cnt) != 0);
dbg_debugfs_exit();
ubifs_compressors_exit();
return 0;
}
+/* Unlock both inodes after they've been prepped for a range clone. */
+STATIC void
+xfs_reflink_remap_unlock(
+ struct file *file_in,
+ struct file *file_out)
+{
+ struct inode *inode_in = file_inode(file_in);
+ struct xfs_inode *src = XFS_I(inode_in);
+ struct inode *inode_out = file_inode(file_out);
+ struct xfs_inode *dest = XFS_I(inode_out);
+ bool same_inode = (inode_in == inode_out);
+
+ xfs_iunlock(dest, XFS_MMAPLOCK_EXCL);
+ if (!same_inode)
+ xfs_iunlock(src, XFS_MMAPLOCK_SHARED);
+ inode_unlock(inode_out);
+ if (!same_inode)
+ inode_unlock_shared(inode_in);
+}
+
/*
- * Link a range of blocks from one file to another.
+ * If we're reflinking to a point past the destination file's EOF, we must
+ * zero any speculative post-EOF preallocations that sit between the old EOF
+ * and the destination file offset.
*/
-int
-xfs_reflink_remap_range(
+static int
+xfs_reflink_zero_posteof(
+ struct xfs_inode *ip,
+ loff_t pos)
+{
+ loff_t isize = i_size_read(VFS_I(ip));
+
+ if (pos <= isize)
+ return 0;
+
+ trace_xfs_zero_eof(ip, isize, pos - isize);
+ return iomap_zero_range(VFS_I(ip), isize, pos - isize, NULL,
+ &xfs_iomap_ops);
+}
+
+/*
+ * Prepare two files for range cloning. Upon a successful return both inodes
+ * will have the iolock and mmaplock held, the page cache of the out file will
+ * be truncated, and any leases on the out file will have been broken. This
+ * function borrows heavily from xfs_file_aio_write_checks.
+ *
+ * The VFS allows partial EOF blocks to "match" for dedupe even though it hasn't
+ * checked that the bytes beyond EOF physically match. Hence we cannot use the
+ * EOF block in the source dedupe range because it's not a complete block match,
+ * hence can introduce a corruption into the file that has it's block replaced.
+ *
+ * In similar fashion, the VFS file cloning also allows partial EOF blocks to be
+ * "block aligned" for the purposes of cloning entire files. However, if the
+ * source file range includes the EOF block and it lands within the existing EOF
+ * of the destination file, then we can expose stale data from beyond the source
+ * file EOF in the destination file.
+ *
+ * XFS doesn't support partial block sharing, so in both cases we have check
+ * these cases ourselves. For dedupe, we can simply round the length to dedupe
+ * down to the previous whole block and ignore the partial EOF block. While this
+ * means we can't dedupe the last block of a file, this is an acceptible
+ * tradeoff for simplicity on implementation.
+ *
+ * For cloning, we want to share the partial EOF block if it is also the new EOF
+ * block of the destination file. If the partial EOF block lies inside the
+ * existing destination EOF, then we have to abort the clone to avoid exposing
+ * stale data in the destination file. Hence we reject these clone attempts with
+ * -EINVAL in this case.
+ */
+STATIC int
+xfs_reflink_remap_prep(
struct file *file_in,
loff_t pos_in,
struct file *file_out,
loff_t pos_out,
- u64 len,
+ u64 *len,
bool is_dedupe)
{
struct inode *inode_in = file_inode(file_in);
struct xfs_inode *src = XFS_I(inode_in);
struct inode *inode_out = file_inode(file_out);
struct xfs_inode *dest = XFS_I(inode_out);
- struct xfs_mount *mp = src->i_mount;
bool same_inode = (inode_in == inode_out);
- xfs_fileoff_t sfsbno, dfsbno;
- xfs_filblks_t fsblen;
- xfs_extlen_t cowextsize;
+ u64 blkmask = i_blocksize(inode_in) - 1;
ssize_t ret;
- if (!xfs_sb_version_hasreflink(&mp->m_sb))
- return -EOPNOTSUPP;
-
- if (XFS_FORCED_SHUTDOWN(mp))
- return -EIO;
-
/* Lock both files against IO */
ret = xfs_iolock_two_inodes_and_break_layout(inode_in, inode_out);
if (ret)
goto out_unlock;
ret = vfs_clone_file_prep_inodes(inode_in, pos_in, inode_out, pos_out,
- &len, is_dedupe);
+ len, is_dedupe);
if (ret <= 0)
goto out_unlock;
+ /*
+ * If the dedupe data matches, chop off the partial EOF block
+ * from the source file so we don't try to dedupe the partial
+ * EOF block.
+ */
+ if (is_dedupe) {
+ *len &= ~blkmask;
+ } else if (*len & blkmask) {
+ /*
+ * The user is attempting to share a partial EOF block,
+ * if it's inside the destination EOF then reject it.
+ */
+ if (pos_out + *len < i_size_read(inode_out)) {
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+ }
+
/* Attach dquots to dest inode before changing block map */
ret = xfs_qm_dqattach(dest);
if (ret)
goto out_unlock;
- trace_xfs_reflink_remap_range(src, pos_in, len, dest, pos_out);
-
/*
- * Clear out post-eof preallocations because we don't have page cache
- * backing the delayed allocations and they'll never get freed on
- * their own.
+ * Zero existing post-eof speculative preallocations in the destination
+ * file.
*/
- if (xfs_can_free_eofblocks(dest, true)) {
- ret = xfs_free_eofblocks(dest);
- if (ret)
- goto out_unlock;
- }
+ ret = xfs_reflink_zero_posteof(dest, pos_out);
+ if (ret)
+ goto out_unlock;
/* Set flags and remap blocks. */
ret = xfs_reflink_set_inode_flag(src, dest);
if (ret)
goto out_unlock;
+ /* Zap any page cache for the destination file's range. */
+ truncate_inode_pages_range(&inode_out->i_data, pos_out,
+ PAGE_ALIGN(pos_out + *len) - 1);
+
+ /* If we're altering the file contents... */
+ if (!is_dedupe) {
+ /*
+ * ...update the timestamps (which will grab the ilock again
+ * from xfs_fs_dirty_inode, so we have to call it before we
+ * take the ilock).
+ */
+ if (!(file_out->f_mode & FMODE_NOCMTIME)) {
+ ret = file_update_time(file_out);
+ if (ret)
+ goto out_unlock;
+ }
+
+ /*
+ * ...clear the security bits if the process is not being run
+ * by root. This keeps people from modifying setuid and setgid
+ * binaries.
+ */
+ ret = file_remove_privs(file_out);
+ if (ret)
+ goto out_unlock;
+ }
+
+ return 1;
+out_unlock:
+ xfs_reflink_remap_unlock(file_in, file_out);
+ return ret;
+}
+
+/*
+ * Link a range of blocks from one file to another.
+ */
+int
+xfs_reflink_remap_range(
+ struct file *file_in,
+ loff_t pos_in,
+ struct file *file_out,
+ loff_t pos_out,
+ u64 len,
+ bool is_dedupe)
+{
+ struct inode *inode_in = file_inode(file_in);
+ struct xfs_inode *src = XFS_I(inode_in);
+ struct inode *inode_out = file_inode(file_out);
+ struct xfs_inode *dest = XFS_I(inode_out);
+ struct xfs_mount *mp = src->i_mount;
+ xfs_fileoff_t sfsbno, dfsbno;
+ xfs_filblks_t fsblen;
+ xfs_extlen_t cowextsize;
+ ssize_t ret;
+
+ if (!xfs_sb_version_hasreflink(&mp->m_sb))
+ return -EOPNOTSUPP;
+
+ if (XFS_FORCED_SHUTDOWN(mp))
+ return -EIO;
+
+ /* Prepare and then clone file data. */
+ ret = xfs_reflink_remap_prep(file_in, pos_in, file_out, pos_out,
+ &len, is_dedupe);
+ if (ret <= 0)
+ return ret;
+
+ trace_xfs_reflink_remap_range(src, pos_in, len, dest, pos_out);
+
dfsbno = XFS_B_TO_FSBT(mp, pos_out);
sfsbno = XFS_B_TO_FSBT(mp, pos_in);
fsblen = XFS_B_TO_FSB(mp, len);
if (ret)
goto out_unlock;
- /* Zap any page cache for the destination file's range. */
- truncate_inode_pages_range(&inode_out->i_data, pos_out,
- PAGE_ALIGN(pos_out + len) - 1);
-
/*
* Carry the cowextsize hint from src to dest if we're sharing the
* entire source file to the entire destination file, the source file
is_dedupe);
out_unlock:
- xfs_iunlock(dest, XFS_MMAPLOCK_EXCL);
- if (!same_inode)
- xfs_iunlock(src, XFS_MMAPLOCK_SHARED);
- inode_unlock(inode_out);
- if (!same_inode)
- inode_unlock_shared(inode_in);
+ xfs_reflink_remap_unlock(file_in, file_out);
if (ret)
trace_xfs_reflink_remap_range_error(dest, ret, _RET_IP_);
return ret;
#define ACPI_RSDP_CHECKSUM_LENGTH 20
#define ACPI_RSDP_XCHECKSUM_LENGTH 36
-/* SMBus, GSBus and IPMI bidirectional buffer size */
+/*
+ * SMBus, GSBus and IPMI buffer sizes. All have a 2-byte header,
+ * containing both Status and Length.
+ */
+#define ACPI_SERIAL_HEADER_SIZE 2 /* Common for below. Status and Length fields */
+
+#define ACPI_SMBUS_DATA_SIZE 32
+#define ACPI_SMBUS_BUFFER_SIZE ACPI_SERIAL_HEADER_SIZE + ACPI_SMBUS_DATA_SIZE
+
+#define ACPI_IPMI_DATA_SIZE 64
+#define ACPI_IPMI_BUFFER_SIZE ACPI_SERIAL_HEADER_SIZE + ACPI_IPMI_DATA_SIZE
-#define ACPI_SMBUS_BUFFER_SIZE 34
-#define ACPI_GSBUS_BUFFER_SIZE 34
-#define ACPI_IPMI_BUFFER_SIZE 66
+#define ACPI_MAX_GSBUS_DATA_SIZE 255
+#define ACPI_MAX_GSBUS_BUFFER_SIZE ACPI_SERIAL_HEADER_SIZE + ACPI_MAX_GSBUS_DATA_SIZE
/* _sx_d and _sx_w control methods */
#define AE_AML_LOOP_TIMEOUT EXCEP_AML (0x0021)
#define AE_AML_UNINITIALIZED_NODE EXCEP_AML (0x0022)
#define AE_AML_TARGET_TYPE EXCEP_AML (0x0023)
+#define AE_AML_PROTOCOL EXCEP_AML (0x0024)
+#define AE_AML_BUFFER_LENGTH EXCEP_AML (0x0025)
-#define AE_CODE_AML_MAX 0x0023
+#define AE_CODE_AML_MAX 0x0025
/*
* Internal exceptions used for control
EXCEP_TXT("AE_AML_UNINITIALIZED_NODE",
"A namespace node is uninitialized or unresolved"),
EXCEP_TXT("AE_AML_TARGET_TYPE",
- "A target operand of an incorrect type was encountered")
+ "A target operand of an incorrect type was encountered"),
+ EXCEP_TXT("AE_AML_PROTOCOL", "Violation of a fixed ACPI protocol"),
+ EXCEP_TXT("AE_AML_BUFFER_LENGTH",
+ "The length of the buffer is invalid/incorrect")
};
static const struct acpi_exception_info acpi_gbl_exception_names_ctrl[] = {
int acpi_dma_get_range(struct device *dev, u64 *dma_addr, u64 *offset,
u64 *size);
int acpi_dma_configure(struct device *dev, enum dev_dma_attr attr);
-void acpi_dma_deconfigure(struct device *dev);
struct acpi_device *acpi_find_child_device(struct acpi_device *parent,
u64 address, bool check_children);
/* Current ACPICA subsystem version in YYYYMMDD format */
-#define ACPI_CA_VERSION 0x20180810
+#define ACPI_CA_VERSION 0x20181003
#include <acpi/acconfig.h>
#include <acpi/actypes.h>
*/
ACPI_INIT_GLOBAL(u8, acpi_gbl_do_not_use_xsdt, FALSE);
-/*
- * Optionally support group module level code.
- * NOTE, this is essentially obsolete and will be removed soon
- * (01/2018).
- */
-ACPI_INIT_GLOBAL(u8, acpi_gbl_group_module_level_code, FALSE);
-
/*
* Optionally support module level code by parsing an entire table as
* a method as it is loaded. Default is TRUE.
* today.
*/
struct cppc_perf_caps {
+ u32 guaranteed_perf;
u32 highest_perf;
u32 nominal_perf;
u32 lowest_perf;
#ifndef __ASSEMBLY__
#include <linux/kernel.h>
-#ifdef CONFIG_BUG
-
-#ifdef CONFIG_GENERIC_BUG
struct bug_entry {
+#ifdef CONFIG_GENERIC_BUG
#ifndef CONFIG_GENERIC_BUG_RELATIVE_POINTERS
unsigned long bug_addr;
#else
unsigned short line;
#endif
unsigned short flags;
-};
#endif /* CONFIG_GENERIC_BUG */
+};
+
+#ifdef CONFIG_BUG
/*
* Don't use BUG() or BUG_ON() unless there's really no way out; one
static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
- /*
- * Use the non-coherent ops if available. If an architecture wants a
- * more fine-grained selection of operations it will have to implement
- * get_arch_dma_ops itself or use the per-device dma_ops.
- */
-#ifdef CONFIG_DMA_NONCOHERENT_OPS
- return &dma_noncoherent_ops;
-#else
return &dma_direct_ops;
-#endif
}
#endif /* _ASM_GENERIC_DMA_MAPPING_H */
if (unlikely(cnts))
return 0;
- return likely(atomic_cmpxchg_acquire(&lock->cnts,
- cnts, cnts | _QW_LOCKED) == cnts);
+ return likely(atomic_try_cmpxchg_acquire(&lock->cnts, &cnts,
+ _QW_LOCKED));
}
/**
* queued_read_lock - acquire read lock of a queue rwlock
*/
static inline void queued_write_lock(struct qrwlock *lock)
{
+ u32 cnts = 0;
/* Optimize for the unfair lock case where the fair flag is 0. */
- if (atomic_cmpxchg_acquire(&lock->cnts, 0, _QW_LOCKED) == 0)
+ if (likely(atomic_try_cmpxchg_acquire(&lock->cnts, &cnts, _QW_LOCKED)))
return;
queued_write_lock_slowpath(lock);
*/
static __always_inline int queued_spin_trylock(struct qspinlock *lock)
{
- if (!atomic_read(&lock->val) &&
- (atomic_cmpxchg_acquire(&lock->val, 0, _Q_LOCKED_VAL) == 0))
- return 1;
- return 0;
+ u32 val = atomic_read(&lock->val);
+
+ if (unlikely(val))
+ return 0;
+
+ return likely(atomic_try_cmpxchg_acquire(&lock->val, &val, _Q_LOCKED_VAL));
}
extern void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
*/
static __always_inline void queued_spin_lock(struct qspinlock *lock)
{
- u32 val;
+ u32 val = 0;
- val = atomic_cmpxchg_acquire(&lock->val, 0, _Q_LOCKED_VAL);
- if (likely(val == 0))
+ if (likely(atomic_try_cmpxchg_acquire(&lock->val, &val, _Q_LOCKED_VAL)))
return;
+
queued_spin_lock_slowpath(lock, val);
}
#include <asm/pgalloc.h>
#include <asm/tlbflush.h>
+#ifdef CONFIG_MMU
+
#ifdef CONFIG_HAVE_RCU_TABLE_FREE
/*
* Semi RCU freeing of the page directories.
#endif
unsigned long start;
unsigned long end;
- /* we are in the middle of an operation to clear
- * a full mm and can make some optimizations */
- unsigned int fullmm : 1,
- /* we have performed an operation which
- * requires a complete flush of the tlb */
- need_flush_all : 1;
+ /*
+ * we are in the middle of an operation to clear
+ * a full mm and can make some optimizations
+ */
+ unsigned int fullmm : 1;
+
+ /*
+ * we have performed an operation which
+ * requires a complete flush of the tlb
+ */
+ unsigned int need_flush_all : 1;
+
+ /*
+ * we have removed page directories
+ */
+ unsigned int freed_tables : 1;
+
+ /*
+ * at which levels have we cleared entries?
+ */
+ unsigned int cleared_ptes : 1;
+ unsigned int cleared_pmds : 1;
+ unsigned int cleared_puds : 1;
+ unsigned int cleared_p4ds : 1;
struct mmu_gather_batch *active;
struct mmu_gather_batch local;
void tlb_flush_mmu(struct mmu_gather *tlb);
void arch_tlb_finish_mmu(struct mmu_gather *tlb,
unsigned long start, unsigned long end, bool force);
+void tlb_flush_mmu_free(struct mmu_gather *tlb);
extern bool __tlb_remove_page_size(struct mmu_gather *tlb, struct page *page,
int page_size);
tlb->start = TASK_SIZE;
tlb->end = 0;
}
+ tlb->freed_tables = 0;
+ tlb->cleared_ptes = 0;
+ tlb->cleared_pmds = 0;
+ tlb->cleared_puds = 0;
+ tlb->cleared_p4ds = 0;
}
static inline void tlb_flush_mmu_tlbonly(struct mmu_gather *tlb)
}
#endif
+static inline unsigned long tlb_get_unmap_shift(struct mmu_gather *tlb)
+{
+ if (tlb->cleared_ptes)
+ return PAGE_SHIFT;
+ if (tlb->cleared_pmds)
+ return PMD_SHIFT;
+ if (tlb->cleared_puds)
+ return PUD_SHIFT;
+ if (tlb->cleared_p4ds)
+ return P4D_SHIFT;
+
+ return PAGE_SHIFT;
+}
+
+static inline unsigned long tlb_get_unmap_size(struct mmu_gather *tlb)
+{
+ return 1UL << tlb_get_unmap_shift(tlb);
+}
+
/*
* In the case of tlb vma handling, we can optimise these away in the
* case where we're doing a full MM flush. When we're doing a munmap,
#define tlb_remove_tlb_entry(tlb, ptep, address) \
do { \
__tlb_adjust_range(tlb, address, PAGE_SIZE); \
+ tlb->cleared_ptes = 1; \
__tlb_remove_tlb_entry(tlb, ptep, address); \
} while (0)
-#define tlb_remove_huge_tlb_entry(h, tlb, ptep, address) \
- do { \
- __tlb_adjust_range(tlb, address, huge_page_size(h)); \
- __tlb_remove_tlb_entry(tlb, ptep, address); \
+#define tlb_remove_huge_tlb_entry(h, tlb, ptep, address) \
+ do { \
+ unsigned long _sz = huge_page_size(h); \
+ __tlb_adjust_range(tlb, address, _sz); \
+ if (_sz == PMD_SIZE) \
+ tlb->cleared_pmds = 1; \
+ else if (_sz == PUD_SIZE) \
+ tlb->cleared_puds = 1; \
+ __tlb_remove_tlb_entry(tlb, ptep, address); \
} while (0)
/**
#define tlb_remove_pmd_tlb_entry(tlb, pmdp, address) \
do { \
__tlb_adjust_range(tlb, address, HPAGE_PMD_SIZE); \
+ tlb->cleared_pmds = 1; \
__tlb_remove_pmd_tlb_entry(tlb, pmdp, address); \
} while (0)
#define tlb_remove_pud_tlb_entry(tlb, pudp, address) \
do { \
__tlb_adjust_range(tlb, address, HPAGE_PUD_SIZE); \
+ tlb->cleared_puds = 1; \
__tlb_remove_pud_tlb_entry(tlb, pudp, address); \
} while (0)
#define pte_free_tlb(tlb, ptep, address) \
do { \
__tlb_adjust_range(tlb, address, PAGE_SIZE); \
+ tlb->freed_tables = 1; \
+ tlb->cleared_pmds = 1; \
__pte_free_tlb(tlb, ptep, address); \
} while (0)
#endif
#ifndef pmd_free_tlb
#define pmd_free_tlb(tlb, pmdp, address) \
do { \
- __tlb_adjust_range(tlb, address, PAGE_SIZE); \
+ __tlb_adjust_range(tlb, address, PAGE_SIZE); \
+ tlb->freed_tables = 1; \
+ tlb->cleared_puds = 1; \
__pmd_free_tlb(tlb, pmdp, address); \
} while (0)
#endif
#define pud_free_tlb(tlb, pudp, address) \
do { \
__tlb_adjust_range(tlb, address, PAGE_SIZE); \
+ tlb->freed_tables = 1; \
+ tlb->cleared_p4ds = 1; \
__pud_free_tlb(tlb, pudp, address); \
} while (0)
#endif
#ifndef p4d_free_tlb
#define p4d_free_tlb(tlb, pudp, address) \
do { \
- __tlb_adjust_range(tlb, address, PAGE_SIZE); \
+ __tlb_adjust_range(tlb, address, PAGE_SIZE); \
+ tlb->freed_tables = 1; \
__p4d_free_tlb(tlb, pudp, address); \
} while (0)
#endif
#endif
+#endif /* CONFIG_MMU */
+
#define tlb_migrate_finish(mm) do {} while (0)
#endif /* _ASM_GENERIC__TLB_H */
*/
#ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
#define TEXT_MAIN .text .text.[0-9a-zA-Z_]*
-#define DATA_MAIN .data .data.[0-9a-zA-Z_]*
+#define DATA_MAIN .data .data.[0-9a-zA-Z_]* .data..LPBX*
#define SDATA_MAIN .sdata .sdata.[0-9a-zA-Z_]*
#define RODATA_MAIN .rodata .rodata.[0-9a-zA-Z_]*
#define BSS_MAIN .bss .bss.[0-9a-zA-Z_]*
STRUCT_ALIGN(); \
*(__tracepoints) \
/* implement dynamic printk debug */ \
- . = ALIGN(8); \
- __start___jump_table = .; \
- KEEP(*(__jump_table)) \
- __stop___jump_table = .; \
. = ALIGN(8); \
__start___verbose = .; \
KEEP(*(__verbose)) \
. = __start_init_task + THREAD_SIZE; \
__end_init_task = .;
+#define JUMP_TABLE_DATA \
+ . = ALIGN(8); \
+ __start___jump_table = .; \
+ KEEP(*(__jump_table)) \
+ __stop___jump_table = .;
+
/*
* Allow architectures to handle ro_after_init data on their
* own by defining an empty RO_AFTER_INIT_DATA.
#define RO_AFTER_INIT_DATA \
__start_ro_after_init = .; \
*(.data..ro_after_init) \
+ JUMP_TABLE_DATA \
__end_ro_after_init = .;
#endif
#define EXIT_DATA \
*(.exit.data .exit.data.*) \
- *(.fini_array) \
- *(.dtors) \
+ *(.fini_array .fini_array.*) \
+ *(.dtors .dtors.*) \
MEM_DISCARD(exit.data*) \
MEM_DISCARD(exit.rodata*)
struct __drm_crtcs_state {
struct drm_crtc *ptr;
struct drm_crtc_state *state, *old_state, *new_state;
+
+ /**
+ * @commit:
+ *
+ * A reference to the CRTC commit object that is kept for use by
+ * drm_atomic_helper_wait_for_flip_done() after
+ * drm_atomic_helper_commit_hw_done() is called. This ensures that a
+ * concurrent commit won't free a commit object that is still in use.
+ */
+ struct drm_crtc_commit *commit;
+
s32 __user *out_fence_ptr;
u64 last_vblank_count;
};
#define DRM_EDID_HDMI_DC_Y444 (1 << 3)
/* YCBCR 420 deep color modes */
-#define DRM_EDID_YCBCR420_DC_48 (1 << 6)
-#define DRM_EDID_YCBCR420_DC_36 (1 << 5)
-#define DRM_EDID_YCBCR420_DC_30 (1 << 4)
+#define DRM_EDID_YCBCR420_DC_48 (1 << 2)
+#define DRM_EDID_YCBCR420_DC_36 (1 << 1)
+#define DRM_EDID_YCBCR420_DC_30 (1 << 0)
#define DRM_EDID_YCBCR420_DC_MASK (DRM_EDID_YCBCR420_DC_48 | \
DRM_EDID_YCBCR420_DC_36 | \
DRM_EDID_YCBCR420_DC_30)
--- /dev/null
+/* SPDX-License-Identifier: (GPL-2.0+ or MIT) */
+/*
+ * Copyright (c) 2018 Amlogic, Inc. All rights reserved.
+ * Author: Xingyu Chen <xingyu.chen@amlogic.com>
+ */
+
+#ifndef _DT_BINDINGS_MESON_G12A_GPIO_H
+#define _DT_BINDINGS_MESON_G12A_GPIO_H
+
+/* First GPIO chip */
+#define GPIOAO_0 0
+#define GPIOAO_1 1
+#define GPIOAO_2 2
+#define GPIOAO_3 3
+#define GPIOAO_4 4
+#define GPIOAO_5 5
+#define GPIOAO_6 6
+#define GPIOAO_7 7
+#define GPIOAO_8 8
+#define GPIOAO_9 9
+#define GPIOAO_10 10
+#define GPIOAO_11 11
+#define GPIOE_0 12
+#define GPIOE_1 13
+#define GPIOE_2 14
+
+/* Second GPIO chip */
+#define GPIOZ_0 0
+#define GPIOZ_1 1
+#define GPIOZ_2 2
+#define GPIOZ_3 3
+#define GPIOZ_4 4
+#define GPIOZ_5 5
+#define GPIOZ_6 6
+#define GPIOZ_7 7
+#define GPIOZ_8 8
+#define GPIOZ_9 9
+#define GPIOZ_10 10
+#define GPIOZ_11 11
+#define GPIOZ_12 12
+#define GPIOZ_13 13
+#define GPIOZ_14 14
+#define GPIOZ_15 15
+#define GPIOH_0 16
+#define GPIOH_1 17
+#define GPIOH_2 18
+#define GPIOH_3 19
+#define GPIOH_4 20
+#define GPIOH_5 21
+#define GPIOH_6 22
+#define GPIOH_7 23
+#define GPIOH_8 24
+#define BOOT_0 25
+#define BOOT_1 26
+#define BOOT_2 27
+#define BOOT_3 28
+#define BOOT_4 29
+#define BOOT_5 30
+#define BOOT_6 31
+#define BOOT_7 32
+#define BOOT_8 33
+#define BOOT_9 34
+#define BOOT_10 35
+#define BOOT_11 36
+#define BOOT_12 37
+#define BOOT_13 38
+#define BOOT_14 39
+#define BOOT_15 40
+#define GPIOC_0 41
+#define GPIOC_1 42
+#define GPIOC_2 43
+#define GPIOC_3 44
+#define GPIOC_4 45
+#define GPIOC_5 46
+#define GPIOC_6 47
+#define GPIOC_7 48
+#define GPIOA_0 49
+#define GPIOA_1 50
+#define GPIOA_2 51
+#define GPIOA_3 52
+#define GPIOA_4 53
+#define GPIOA_5 54
+#define GPIOA_6 55
+#define GPIOA_7 56
+#define GPIOA_8 57
+#define GPIOA_9 58
+#define GPIOA_10 59
+#define GPIOA_11 60
+#define GPIOA_12 61
+#define GPIOA_13 62
+#define GPIOA_14 63
+#define GPIOA_15 64
+#define GPIOX_0 65
+#define GPIOX_1 66
+#define GPIOX_2 67
+#define GPIOX_3 68
+#define GPIOX_4 69
+#define GPIOX_5 70
+#define GPIOX_6 71
+#define GPIOX_7 72
+#define GPIOX_8 73
+#define GPIOX_9 74
+#define GPIOX_10 75
+#define GPIOX_11 76
+#define GPIOX_12 77
+#define GPIOX_13 78
+#define GPIOX_14 79
+#define GPIOX_15 80
+#define GPIOX_16 81
+#define GPIOX_17 82
+#define GPIOX_18 83
+#define GPIOX_19 84
+
+#endif /* _DT_BINDINGS_MESON_G12A_GPIO_H */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Defines macros and constants for Renesas RZ/N1 pin controller pin
+ * muxing functions.
+ */
+#ifndef __DT_BINDINGS_RZN1_PINCTRL_H
+#define __DT_BINDINGS_RZN1_PINCTRL_H
+
+#define RZN1_PINMUX(_gpio, _func) \
+ (((_func) << 8) | (_gpio))
+
+/*
+ * Given the different levels of muxing on the SoC, it was decided to
+ * 'linearize' them into one numerical space. So mux level 1, 2 and the MDIO
+ * muxes are all represented by one single value.
+ *
+ * You can derive the hardware value pretty easily too, as
+ * 0...9 are Level 1
+ * 10...71 are Level 2. The Level 2 mux will be set to this
+ * value - RZN1_FUNC_L2_OFFSET, and the Level 1 mux will be
+ * set accordingly.
+ * 72...103 are for the 2 MDIO muxes.
+ */
+#define RZN1_FUNC_HIGHZ 0
+#define RZN1_FUNC_0L 1
+#define RZN1_FUNC_CLK_ETH_MII_RGMII_RMII 2
+#define RZN1_FUNC_CLK_ETH_NAND 3
+#define RZN1_FUNC_QSPI 4
+#define RZN1_FUNC_SDIO 5
+#define RZN1_FUNC_LCD 6
+#define RZN1_FUNC_LCD_E 7
+#define RZN1_FUNC_MSEBIM 8
+#define RZN1_FUNC_MSEBIS 9
+#define RZN1_FUNC_L2_OFFSET 10 /* I'm Special */
+
+#define RZN1_FUNC_HIGHZ1 (RZN1_FUNC_L2_OFFSET + 0)
+#define RZN1_FUNC_ETHERCAT (RZN1_FUNC_L2_OFFSET + 1)
+#define RZN1_FUNC_SERCOS3 (RZN1_FUNC_L2_OFFSET + 2)
+#define RZN1_FUNC_SDIO_E (RZN1_FUNC_L2_OFFSET + 3)
+#define RZN1_FUNC_ETH_MDIO (RZN1_FUNC_L2_OFFSET + 4)
+#define RZN1_FUNC_ETH_MDIO_E1 (RZN1_FUNC_L2_OFFSET + 5)
+#define RZN1_FUNC_USB (RZN1_FUNC_L2_OFFSET + 6)
+#define RZN1_FUNC_MSEBIM_E (RZN1_FUNC_L2_OFFSET + 7)
+#define RZN1_FUNC_MSEBIS_E (RZN1_FUNC_L2_OFFSET + 8)
+#define RZN1_FUNC_RSV (RZN1_FUNC_L2_OFFSET + 9)
+#define RZN1_FUNC_RSV_E (RZN1_FUNC_L2_OFFSET + 10)
+#define RZN1_FUNC_RSV_E1 (RZN1_FUNC_L2_OFFSET + 11)
+#define RZN1_FUNC_UART0_I (RZN1_FUNC_L2_OFFSET + 12)
+#define RZN1_FUNC_UART0_I_E (RZN1_FUNC_L2_OFFSET + 13)
+#define RZN1_FUNC_UART1_I (RZN1_FUNC_L2_OFFSET + 14)
+#define RZN1_FUNC_UART1_I_E (RZN1_FUNC_L2_OFFSET + 15)
+#define RZN1_FUNC_UART2_I (RZN1_FUNC_L2_OFFSET + 16)
+#define RZN1_FUNC_UART2_I_E (RZN1_FUNC_L2_OFFSET + 17)
+#define RZN1_FUNC_UART0 (RZN1_FUNC_L2_OFFSET + 18)
+#define RZN1_FUNC_UART0_E (RZN1_FUNC_L2_OFFSET + 19)
+#define RZN1_FUNC_UART1 (RZN1_FUNC_L2_OFFSET + 20)
+#define RZN1_FUNC_UART1_E (RZN1_FUNC_L2_OFFSET + 21)
+#define RZN1_FUNC_UART2 (RZN1_FUNC_L2_OFFSET + 22)
+#define RZN1_FUNC_UART2_E (RZN1_FUNC_L2_OFFSET + 23)
+#define RZN1_FUNC_UART3 (RZN1_FUNC_L2_OFFSET + 24)
+#define RZN1_FUNC_UART3_E (RZN1_FUNC_L2_OFFSET + 25)
+#define RZN1_FUNC_UART4 (RZN1_FUNC_L2_OFFSET + 26)
+#define RZN1_FUNC_UART4_E (RZN1_FUNC_L2_OFFSET + 27)
+#define RZN1_FUNC_UART5 (RZN1_FUNC_L2_OFFSET + 28)
+#define RZN1_FUNC_UART5_E (RZN1_FUNC_L2_OFFSET + 29)
+#define RZN1_FUNC_UART6 (RZN1_FUNC_L2_OFFSET + 30)
+#define RZN1_FUNC_UART6_E (RZN1_FUNC_L2_OFFSET + 31)
+#define RZN1_FUNC_UART7 (RZN1_FUNC_L2_OFFSET + 32)
+#define RZN1_FUNC_UART7_E (RZN1_FUNC_L2_OFFSET + 33)
+#define RZN1_FUNC_SPI0_M (RZN1_FUNC_L2_OFFSET + 34)
+#define RZN1_FUNC_SPI0_M_E (RZN1_FUNC_L2_OFFSET + 35)
+#define RZN1_FUNC_SPI1_M (RZN1_FUNC_L2_OFFSET + 36)
+#define RZN1_FUNC_SPI1_M_E (RZN1_FUNC_L2_OFFSET + 37)
+#define RZN1_FUNC_SPI2_M (RZN1_FUNC_L2_OFFSET + 38)
+#define RZN1_FUNC_SPI2_M_E (RZN1_FUNC_L2_OFFSET + 39)
+#define RZN1_FUNC_SPI3_M (RZN1_FUNC_L2_OFFSET + 40)
+#define RZN1_FUNC_SPI3_M_E (RZN1_FUNC_L2_OFFSET + 41)
+#define RZN1_FUNC_SPI4_S (RZN1_FUNC_L2_OFFSET + 42)
+#define RZN1_FUNC_SPI4_S_E (RZN1_FUNC_L2_OFFSET + 43)
+#define RZN1_FUNC_SPI5_S (RZN1_FUNC_L2_OFFSET + 44)
+#define RZN1_FUNC_SPI5_S_E (RZN1_FUNC_L2_OFFSET + 45)
+#define RZN1_FUNC_SGPIO0_M (RZN1_FUNC_L2_OFFSET + 46)
+#define RZN1_FUNC_SGPIO1_M (RZN1_FUNC_L2_OFFSET + 47)
+#define RZN1_FUNC_GPIO (RZN1_FUNC_L2_OFFSET + 48)
+#define RZN1_FUNC_CAN (RZN1_FUNC_L2_OFFSET + 49)
+#define RZN1_FUNC_I2C (RZN1_FUNC_L2_OFFSET + 50)
+#define RZN1_FUNC_SAFE (RZN1_FUNC_L2_OFFSET + 51)
+#define RZN1_FUNC_PTO_PWM (RZN1_FUNC_L2_OFFSET + 52)
+#define RZN1_FUNC_PTO_PWM1 (RZN1_FUNC_L2_OFFSET + 53)
+#define RZN1_FUNC_PTO_PWM2 (RZN1_FUNC_L2_OFFSET + 54)
+#define RZN1_FUNC_PTO_PWM3 (RZN1_FUNC_L2_OFFSET + 55)
+#define RZN1_FUNC_PTO_PWM4 (RZN1_FUNC_L2_OFFSET + 56)
+#define RZN1_FUNC_DELTA_SIGMA (RZN1_FUNC_L2_OFFSET + 57)
+#define RZN1_FUNC_SGPIO2_M (RZN1_FUNC_L2_OFFSET + 58)
+#define RZN1_FUNC_SGPIO3_M (RZN1_FUNC_L2_OFFSET + 59)
+#define RZN1_FUNC_SGPIO4_S (RZN1_FUNC_L2_OFFSET + 60)
+#define RZN1_FUNC_MAC_MTIP_SWITCH (RZN1_FUNC_L2_OFFSET + 61)
+
+#define RZN1_FUNC_MDIO_OFFSET (RZN1_FUNC_L2_OFFSET + 62)
+
+/* These are MDIO0 peripherals for the RZN1_FUNC_ETH_MDIO function */
+#define RZN1_FUNC_MDIO0_HIGHZ (RZN1_FUNC_MDIO_OFFSET + 0)
+#define RZN1_FUNC_MDIO0_GMAC0 (RZN1_FUNC_MDIO_OFFSET + 1)
+#define RZN1_FUNC_MDIO0_GMAC1 (RZN1_FUNC_MDIO_OFFSET + 2)
+#define RZN1_FUNC_MDIO0_ECAT (RZN1_FUNC_MDIO_OFFSET + 3)
+#define RZN1_FUNC_MDIO0_S3_MDIO0 (RZN1_FUNC_MDIO_OFFSET + 4)
+#define RZN1_FUNC_MDIO0_S3_MDIO1 (RZN1_FUNC_MDIO_OFFSET + 5)
+#define RZN1_FUNC_MDIO0_HWRTOS (RZN1_FUNC_MDIO_OFFSET + 6)
+#define RZN1_FUNC_MDIO0_SWITCH (RZN1_FUNC_MDIO_OFFSET + 7)
+/* These are MDIO0 peripherals for the RZN1_FUNC_ETH_MDIO_E1 function */
+#define RZN1_FUNC_MDIO0_E1_HIGHZ (RZN1_FUNC_MDIO_OFFSET + 8)
+#define RZN1_FUNC_MDIO0_E1_GMAC0 (RZN1_FUNC_MDIO_OFFSET + 9)
+#define RZN1_FUNC_MDIO0_E1_GMAC1 (RZN1_FUNC_MDIO_OFFSET + 10)
+#define RZN1_FUNC_MDIO0_E1_ECAT (RZN1_FUNC_MDIO_OFFSET + 11)
+#define RZN1_FUNC_MDIO0_E1_S3_MDIO0 (RZN1_FUNC_MDIO_OFFSET + 12)
+#define RZN1_FUNC_MDIO0_E1_S3_MDIO1 (RZN1_FUNC_MDIO_OFFSET + 13)
+#define RZN1_FUNC_MDIO0_E1_HWRTOS (RZN1_FUNC_MDIO_OFFSET + 14)
+#define RZN1_FUNC_MDIO0_E1_SWITCH (RZN1_FUNC_MDIO_OFFSET + 15)
+
+/* These are MDIO1 peripherals for the RZN1_FUNC_ETH_MDIO function */
+#define RZN1_FUNC_MDIO1_HIGHZ (RZN1_FUNC_MDIO_OFFSET + 16)
+#define RZN1_FUNC_MDIO1_GMAC0 (RZN1_FUNC_MDIO_OFFSET + 17)
+#define RZN1_FUNC_MDIO1_GMAC1 (RZN1_FUNC_MDIO_OFFSET + 18)
+#define RZN1_FUNC_MDIO1_ECAT (RZN1_FUNC_MDIO_OFFSET + 19)
+#define RZN1_FUNC_MDIO1_S3_MDIO0 (RZN1_FUNC_MDIO_OFFSET + 20)
+#define RZN1_FUNC_MDIO1_S3_MDIO1 (RZN1_FUNC_MDIO_OFFSET + 21)
+#define RZN1_FUNC_MDIO1_HWRTOS (RZN1_FUNC_MDIO_OFFSET + 22)
+#define RZN1_FUNC_MDIO1_SWITCH (RZN1_FUNC_MDIO_OFFSET + 23)
+/* These are MDIO1 peripherals for the RZN1_FUNC_ETH_MDIO_E1 function */
+#define RZN1_FUNC_MDIO1_E1_HIGHZ (RZN1_FUNC_MDIO_OFFSET + 24)
+#define RZN1_FUNC_MDIO1_E1_GMAC0 (RZN1_FUNC_MDIO_OFFSET + 25)
+#define RZN1_FUNC_MDIO1_E1_GMAC1 (RZN1_FUNC_MDIO_OFFSET + 26)
+#define RZN1_FUNC_MDIO1_E1_ECAT (RZN1_FUNC_MDIO_OFFSET + 27)
+#define RZN1_FUNC_MDIO1_E1_S3_MDIO0 (RZN1_FUNC_MDIO_OFFSET + 28)
+#define RZN1_FUNC_MDIO1_E1_S3_MDIO1 (RZN1_FUNC_MDIO_OFFSET + 29)
+#define RZN1_FUNC_MDIO1_E1_HWRTOS (RZN1_FUNC_MDIO_OFFSET + 30)
+#define RZN1_FUNC_MDIO1_E1_SWITCH (RZN1_FUNC_MDIO_OFFSET + 31)
+
+#define RZN1_FUNC_MAX (RZN1_FUNC_MDIO_OFFSET + 32)
+
+#endif /* __DT_BINDINGS_RZN1_PINCTRL_H */
return 0;
}
-static inline void acpi_dma_deconfigure(struct device *dev) { }
-
#define ACPI_PTR(_ptr) (NULL)
static inline void acpi_device_set_enumerated(struct acpi_device *adev)
* mask into a value to be binary (or set some other custom bits
* in MMCIPWR) or:ed and written into the MMCIPWR register of the
* block. May also control external power based on the power_mode.
- * @status: if no GPIO read function was given to the block in
- * gpio_wp (below) this function will be called to determine
- * whether a card is present in the MMC slot or not
- * @gpio_wp: read this GPIO pin to see if the card is write protected
- * @gpio_cd: read this GPIO pin to detect card insertion
- * @cd_invert: true if the gpio_cd pin value is active low
+ * @status: if no GPIO line was given to the block in this function will
+ * be called to determine whether a card is present in the MMC slot or not
*/
struct mmci_platform_data {
unsigned int ocr_mask;
int (*ios_handler)(struct device *, struct mmc_ios *);
unsigned int (*status)(struct device *);
- int gpio_wp;
- int gpio_cd;
- bool cd_invert;
};
#endif
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _AMIFD_H
-#define _AMIFD_H
-
-/* Definitions for the Amiga floppy driver */
-
-#include <linux/fd.h>
-
-#define FD_MAX_UNITS 4 /* Max. Number of drives */
-#define FLOPPY_MAX_SECTORS 22 /* Max. Number of sectors per track */
-
-#ifndef ASSEMBLER
-
-struct fd_data_type {
- char *name; /* description of data type */
- int sects; /* sectors per track */
-#ifdef __STDC__
- int (*read_fkt)(int);
- void (*write_fkt)(int);
-#else
- int (*read_fkt)(); /* read whole track */
- void (*write_fkt)(); /* write whole track */
-#endif
-};
-
-/*
-** Floppy type descriptions
-*/
-
-struct fd_drive_type {
- unsigned long code; /* code returned from drive */
- char *name; /* description of drive */
- unsigned int tracks; /* number of tracks */
- unsigned int heads; /* number of heads */
- unsigned int read_size; /* raw read size for one track */
- unsigned int write_size; /* raw write size for one track */
- unsigned int sect_mult; /* sectors and gap multiplier (HD = 2) */
- unsigned int precomp1; /* start track for precomp 1 */
- unsigned int precomp2; /* start track for precomp 2 */
- unsigned int step_delay; /* time (in ms) for delay after step */
- unsigned int settle_time; /* time to settle after dir change */
- unsigned int side_time; /* time needed to change sides */
-};
-
-struct amiga_floppy_struct {
- struct fd_drive_type *type; /* type of floppy for this unit */
- struct fd_data_type *dtype; /* type of floppy for this unit */
- int track; /* current track (-1 == unknown) */
- unsigned char *trackbuf; /* current track (kmaloc()'d */
-
- int blocks; /* total # blocks on disk */
-
- int changed; /* true when not known */
- int disk; /* disk in drive (-1 == unknown) */
- int motor; /* true when motor is at speed */
- int busy; /* true when drive is active */
- int dirty; /* true when trackbuf is not on disk */
- int status; /* current error code for unit */
- struct gendisk *gendisk;
-};
-#endif
-
-#endif
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _LINUX_AMIFDREG_H
-#define _LINUX_AMIFDREG_H
-
-/*
-** CIAAPRA bits (read only)
-*/
-
-#define DSKRDY (0x1<<5) /* disk ready when low */
-#define DSKTRACK0 (0x1<<4) /* head at track zero when low */
-#define DSKPROT (0x1<<3) /* disk protected when low */
-#define DSKCHANGE (0x1<<2) /* low when disk removed */
-
-/*
-** CIAAPRB bits (read/write)
-*/
-
-#define DSKMOTOR (0x1<<7) /* motor on when low */
-#define DSKSEL3 (0x1<<6) /* select drive 3 when low */
-#define DSKSEL2 (0x1<<5) /* select drive 2 when low */
-#define DSKSEL1 (0x1<<4) /* select drive 1 when low */
-#define DSKSEL0 (0x1<<3) /* select drive 0 when low */
-#define DSKSIDE (0x1<<2) /* side selection: 0 = upper, 1 = lower */
-#define DSKDIREC (0x1<<1) /* step direction: 0=in, 1=out (to trk 0) */
-#define DSKSTEP (0x1) /* pulse low to step head 1 track */
-
-/*
-** DSKBYTR bits (read only)
-*/
-
-#define DSKBYT (1<<15) /* register contains valid byte when set */
-#define DMAON (1<<14) /* disk DMA enabled */
-#define DISKWRITE (1<<13) /* disk write bit in DSKLEN enabled */
-#define WORDEQUAL (1<<12) /* DSKSYNC register match when true */
-/* bits 7-0 are data */
-
-/*
-** ADKCON/ADKCONR bits
-*/
-
-#ifndef SETCLR
-#define ADK_SETCLR (1<<15) /* control bit */
-#endif
-#define ADK_PRECOMP1 (1<<14) /* precompensation selection */
-#define ADK_PRECOMP0 (1<<13) /* 00=none, 01=140ns, 10=280ns, 11=500ns */
-#define ADK_MFMPREC (1<<12) /* 0=GCR precomp., 1=MFM precomp. */
-#define ADK_WORDSYNC (1<<10) /* enable DSKSYNC auto DMA */
-#define ADK_MSBSYNC (1<<9) /* when 1, enable sync on MSbit (for GCR) */
-#define ADK_FAST (1<<8) /* bit cell: 0=2us (GCR), 1=1us (MFM) */
-
-/*
-** DSKLEN bits
-*/
-
-#define DSKLEN_DMAEN (1<<15)
-#define DSKLEN_WRITE (1<<14)
-
-/*
-** INTENA/INTREQ bits
-*/
-
-#define DSKINDEX (0x1<<4) /* DSKINDEX bit */
-
-/*
-** Misc
-*/
-
-#define MFM_SYNC 0x4489 /* standard MFM sync value */
-
-/* Values for FD_COMMAND */
-#define FD_RECALIBRATE 0x07 /* move to track 0 */
-#define FD_SEEK 0x0F /* seek track */
-#define FD_READ 0xE6 /* read with MT, MFM, SKip deleted */
-#define FD_WRITE 0xC5 /* write with MT, MFM */
-#define FD_SENSEI 0x08 /* Sense Interrupt Status */
-#define FD_SPECIFY 0x03 /* specify HUT etc */
-#define FD_FORMAT 0x4D /* format one track */
-#define FD_VERSION 0x10 /* get version code */
-#define FD_CONFIGURE 0x13 /* configure FIFO operation */
-#define FD_PERPENDICULAR 0x12 /* perpendicular r/w mode */
-
-#endif /* _LINUX_AMIFDREG_H */
#include <linux/percpu.h>
void topology_normalize_cpu_scale(void);
+int topology_update_cpu_topology(void);
struct device_node;
bool topology_parse_cpu_capacity(struct device_node *cpu_node, int cpu);
#include <linux/highmem.h>
#include <linux/mempool.h>
#include <linux/ioprio.h>
-#include <linux/bug.h>
#ifdef CONFIG_BLOCK
-
-#include <asm/io.h>
-
/* struct bio, bio_vec and BIO_* flags are defined in blk_types.h */
#include <linux/blk_types.h>
return bio->bi_vcnt >= bio->bi_max_vecs;
}
-/*
- * will die
- */
-#define bvec_to_phys(bv) (page_to_phys((bv)->bv_page) + (unsigned long) (bv)->bv_offset)
-
-/*
- * merge helpers etc
- */
-
-/* Default implementation of BIOVEC_PHYS_MERGEABLE */
-#define __BIOVEC_PHYS_MERGEABLE(vec1, vec2) \
- ((bvec_to_phys((vec1)) + (vec1)->bv_len) == bvec_to_phys((vec2)))
-
-/*
- * allow arch override, for eg virtualized architectures (put in asm/io.h)
- */
-#ifndef BIOVEC_PHYS_MERGEABLE
-#define BIOVEC_PHYS_MERGEABLE(vec1, vec2) \
- __BIOVEC_PHYS_MERGEABLE(vec1, vec2)
-#endif
-
-#define __BIO_SEG_BOUNDARY(addr1, addr2, mask) \
- (((addr1) | (mask)) == (((addr2) - 1) | (mask)))
-#define BIOVEC_SEG_BOUNDARY(q, b1, b2) \
- __BIO_SEG_BOUNDARY(bvec_to_phys((b1)), bvec_to_phys((b2)) + (b2)->bv_len, queue_segment_boundary((q)))
-
/*
* drivers should _never_ use the all version - the bio may have been split
* before it got to the driver and the driver won't own all of it
{
iter->bi_sector += bytes >> 9;
- if (bio_no_advance_iter(bio)) {
+ if (bio_no_advance_iter(bio))
iter->bi_size -= bytes;
- iter->bi_done += bytes;
- } else {
+ else
bvec_iter_advance(bio->bi_io_vec, iter, bytes);
/* TODO: It is reasonable to complete bio with error here. */
- }
-}
-
-static inline bool bio_rewind_iter(struct bio *bio, struct bvec_iter *iter,
- unsigned int bytes)
-{
- iter->bi_sector -= bytes >> 9;
-
- if (bio_no_advance_iter(bio)) {
- iter->bi_size += bytes;
- iter->bi_done -= bytes;
- return true;
- }
-
- return bvec_iter_rewind(bio->bi_io_vec, iter, bytes);
}
#define __bio_for_each_segment(bvl, bio, iter, start) \
unsigned short bip_max_vcnt; /* integrity bio_vec slots */
unsigned short bip_flags; /* control flags */
+ struct bvec_iter bio_iter; /* for rewinding parent bio */
+
struct work_struct bip_work; /* I/O completion */
struct bio_vec *bip_vec;
disk_devt((bio)->bi_disk)
#if defined(CONFIG_MEMCG) && defined(CONFIG_BLK_CGROUP)
-int bio_associate_blkcg_from_page(struct bio *bio, struct page *page);
+int bio_associate_blkg_from_page(struct bio *bio, struct page *page);
#else
-static inline int bio_associate_blkcg_from_page(struct bio *bio,
- struct page *page) { return 0; }
+static inline int bio_associate_blkg_from_page(struct bio *bio,
+ struct page *page) { return 0; }
#endif
#ifdef CONFIG_BLK_CGROUP
-int bio_associate_blkcg(struct bio *bio, struct cgroup_subsys_state *blkcg_css);
int bio_associate_blkg(struct bio *bio, struct blkcg_gq *blkg);
+int bio_associate_blkg_from_css(struct bio *bio,
+ struct cgroup_subsys_state *css);
+int bio_associate_create_blkg(struct request_queue *q, struct bio *bio);
+int bio_reassociate_blkg(struct request_queue *q, struct bio *bio);
void bio_disassociate_task(struct bio *bio);
-void bio_clone_blkcg_association(struct bio *dst, struct bio *src);
+void bio_clone_blkg_association(struct bio *dst, struct bio *src);
#else /* CONFIG_BLK_CGROUP */
-static inline int bio_associate_blkcg(struct bio *bio,
- struct cgroup_subsys_state *blkcg_css) { return 0; }
+static inline int bio_associate_blkg_from_css(struct bio *bio,
+ struct cgroup_subsys_state *css)
+{ return 0; }
+static inline int bio_associate_create_blkg(struct request_queue *q,
+ struct bio *bio) { return 0; }
+static inline int bio_reassociate_blkg(struct request_queue *q, struct bio *bio)
+{ return 0; }
static inline void bio_disassociate_task(struct bio *bio) { }
-static inline void bio_clone_blkcg_association(struct bio *dst,
- struct bio *src) { }
+static inline void bio_clone_blkg_association(struct bio *dst,
+ struct bio *src) { }
#endif /* CONFIG_BLK_CGROUP */
#ifdef CONFIG_HIGHMEM
struct request_list rl;
/* reference count */
- atomic_t refcnt;
+ struct percpu_ref refcnt;
/* is this blkg online? protected by both blkcg and q locks */
bool online;
struct blkcg_gq *blkg_lookup_slowpath(struct blkcg *blkcg,
struct request_queue *q, bool update_hint);
+struct blkcg_gq *__blkg_lookup_create(struct blkcg *blkcg,
+ struct request_queue *q);
struct blkcg_gq *blkg_lookup_create(struct blkcg *blkcg,
struct request_queue *q);
int blkcg_init_queue(struct request_queue *q);
char *input, struct blkg_conf_ctx *ctx);
void blkg_conf_finish(struct blkg_conf_ctx *ctx);
+/**
+ * blkcg_css - find the current css
+ *
+ * Find the css associated with either the kthread or the current task.
+ * This may return a dying css, so it is up to the caller to use tryget logic
+ * to confirm it is alive and well.
+ */
+static inline struct cgroup_subsys_state *blkcg_css(void)
+{
+ struct cgroup_subsys_state *css;
+
+ css = kthread_blkcg();
+ if (css)
+ return css;
+ return task_css(current, io_cgrp_id);
+}
static inline struct blkcg *css_to_blkcg(struct cgroup_subsys_state *css)
{
return css ? container_of(css, struct blkcg, css) : NULL;
}
-static inline struct blkcg *bio_blkcg(struct bio *bio)
+/**
+ * __bio_blkcg - internal version of bio_blkcg for bfq and cfq
+ *
+ * DO NOT USE.
+ * There is a flaw using this version of the function. In particular, this was
+ * used in a broken paradigm where association was called on the given css. It
+ * is possible though that the returned css from task_css() is in the process
+ * of dying due to migration of the current task. So it is improper to assume
+ * *_get() is going to succeed. Both BFQ and CFQ rely on this logic and will
+ * take additional work to handle more gracefully.
+ */
+static inline struct blkcg *__bio_blkcg(struct bio *bio)
{
- struct cgroup_subsys_state *css;
+ if (bio && bio->bi_blkg)
+ return bio->bi_blkg->blkcg;
+ return css_to_blkcg(blkcg_css());
+}
- if (bio && bio->bi_css)
- return css_to_blkcg(bio->bi_css);
- css = kthread_blkcg();
- if (css)
- return css_to_blkcg(css);
- return css_to_blkcg(task_css(current, io_cgrp_id));
+/**
+ * bio_blkcg - grab the blkcg associated with a bio
+ * @bio: target bio
+ *
+ * This returns the blkcg associated with a bio, NULL if not associated.
+ * Callers are expected to either handle NULL or know association has been
+ * done prior to calling this.
+ */
+static inline struct blkcg *bio_blkcg(struct bio *bio)
+{
+ if (bio && bio->bi_blkg)
+ return bio->bi_blkg->blkcg;
+ return NULL;
}
static inline bool blk_cgroup_congested(void)
*/
static inline void blkg_get(struct blkcg_gq *blkg)
{
- WARN_ON_ONCE(atomic_read(&blkg->refcnt) <= 0);
- atomic_inc(&blkg->refcnt);
+ percpu_ref_get(&blkg->refcnt);
}
/**
- * blkg_try_get - try and get a blkg reference
+ * blkg_tryget - try and get a blkg reference
* @blkg: blkg to get
*
* This is for use when doing an RCU lookup of the blkg. We may be in the midst
* of freeing this blkg, so we can only use it if the refcnt is not zero.
*/
-static inline struct blkcg_gq *blkg_try_get(struct blkcg_gq *blkg)
+static inline bool blkg_tryget(struct blkcg_gq *blkg)
{
- if (atomic_inc_not_zero(&blkg->refcnt))
- return blkg;
- return NULL;
+ return percpu_ref_tryget(&blkg->refcnt);
}
+/**
+ * blkg_tryget_closest - try and get a blkg ref on the closet blkg
+ * @blkg: blkg to get
+ *
+ * This walks up the blkg tree to find the closest non-dying blkg and returns
+ * the blkg that it did association with as it may not be the passed in blkg.
+ */
+static inline struct blkcg_gq *blkg_tryget_closest(struct blkcg_gq *blkg)
+{
+ while (!percpu_ref_tryget(&blkg->refcnt))
+ blkg = blkg->parent;
-void __blkg_release_rcu(struct rcu_head *rcu);
+ return blkg;
+}
/**
* blkg_put - put a blkg reference
*/
static inline void blkg_put(struct blkcg_gq *blkg)
{
- WARN_ON_ONCE(atomic_read(&blkg->refcnt) <= 0);
- if (atomic_dec_and_test(&blkg->refcnt))
- call_rcu(&blkg->rcu_head, __blkg_release_rcu);
+ percpu_ref_put(&blkg->refcnt);
}
/**
rcu_read_lock();
- blkcg = bio_blkcg(bio);
+ if (bio && bio->bi_blkg) {
+ blkcg = bio->bi_blkg->blkcg;
+ if (blkcg == &blkcg_root)
+ goto rl_use_root;
+
+ blkg_get(bio->bi_blkg);
+ rcu_read_unlock();
+ return &bio->bi_blkg->rl;
+ }
- /* bypass blkg lookup and use @q->root_rl directly for root */
+ blkcg = css_to_blkcg(blkcg_css());
if (blkcg == &blkcg_root)
- goto root_rl;
+ goto rl_use_root;
- /*
- * Try to use blkg->rl. blkg lookup may fail under memory pressure
- * or if either the blkcg or queue is going away. Fall back to
- * root_rl in such cases.
- */
blkg = blkg_lookup(blkcg, q);
if (unlikely(!blkg))
- goto root_rl;
+ blkg = __blkg_lookup_create(blkcg, q);
+
+ if (blkg->blkcg == &blkcg_root || !blkg_tryget(blkg))
+ goto rl_use_root;
- blkg_get(blkg);
rcu_read_unlock();
return &blkg->rl;
-root_rl:
+
+ /*
+ * Each blkg has its own request_list, however, the root blkcg
+ * uses the request_queue's root_rl. This is to avoid most
+ * overhead for the root blkcg.
+ */
+rl_use_root:
rcu_read_unlock();
return &q->root_rl;
}
struct bio *bio) { return false; }
#endif
+
+static inline void blkcg_bio_issue_init(struct bio *bio)
+{
+ bio_issue_init(&bio->bi_issue, bio_sectors(bio));
+}
+
static inline bool blkcg_bio_issue_check(struct request_queue *q,
struct bio *bio)
{
- struct blkcg *blkcg;
struct blkcg_gq *blkg;
bool throtl = false;
rcu_read_lock();
- blkcg = bio_blkcg(bio);
-
- /* associate blkcg if bio hasn't attached one */
- bio_associate_blkcg(bio, &blkcg->css);
- blkg = blkg_lookup(blkcg, q);
- if (unlikely(!blkg)) {
- spin_lock_irq(q->queue_lock);
- blkg = blkg_lookup_create(blkcg, q);
- if (IS_ERR(blkg))
- blkg = NULL;
- spin_unlock_irq(q->queue_lock);
- }
+ bio_associate_create_blkg(q, bio);
+ blkg = bio->bi_blkg;
throtl = blk_throtl_bio(q, blkg, bio);
if (!throtl) {
- blkg = blkg ?: q->root_blkg;
/*
* If the bio is flagged with BIO_QUEUE_ENTERED it means this
* is a split bio and we would have already accounted for the
blkg_rwstat_add(&blkg->stat_ios, bio->bi_opf, 1);
}
+ blkcg_bio_issue_init(bio);
+
rcu_read_unlock();
return !throtl;
}
static inline void blkcg_deactivate_policy(struct request_queue *q,
const struct blkcg_policy *pol) { }
+static inline struct blkcg *__bio_blkcg(struct bio *bio) { return NULL; }
static inline struct blkcg *bio_blkcg(struct bio *bio) { return NULL; }
static inline struct blkg_policy_data *blkg_to_pd(struct blkcg_gq *blkg,
static inline void blk_rq_set_rl(struct request *rq, struct request_list *rl) { }
static inline struct request_list *blk_rq_rl(struct request *rq) { return &rq->q->root_rl; }
+static inline void blkcg_bio_issue_init(struct bio *bio) { }
static inline bool blkcg_bio_issue_check(struct request_queue *q,
struct bio *bio) { return true; }
struct request_queue *blk_mq_init_queue(struct blk_mq_tag_set *);
struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set,
struct request_queue *q);
+struct request_queue *blk_mq_init_sq_queue(struct blk_mq_tag_set *set,
+ const struct blk_mq_ops *ops,
+ unsigned int queue_depth,
+ unsigned int set_flags);
int blk_mq_register_dev(struct device *, struct request_queue *);
void blk_mq_unregister_dev(struct device *, struct request_queue *);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _BLK_PM_H_
+#define _BLK_PM_H_
+
+struct device;
+struct request_queue;
+
+/*
+ * block layer runtime pm functions
+ */
+#ifdef CONFIG_PM
+extern void blk_pm_runtime_init(struct request_queue *q, struct device *dev);
+extern int blk_pre_runtime_suspend(struct request_queue *q);
+extern void blk_post_runtime_suspend(struct request_queue *q, int err);
+extern void blk_pre_runtime_resume(struct request_queue *q);
+extern void blk_post_runtime_resume(struct request_queue *q, int err);
+extern void blk_set_runtime_active(struct request_queue *q);
+#else
+static inline void blk_pm_runtime_init(struct request_queue *q,
+ struct device *dev) {}
+#endif
+
+#endif /* _BLK_PM_H_ */
* release. Read comment on top of bio_associate_current().
*/
struct io_context *bi_ioc;
- struct cgroup_subsys_state *bi_css;
struct blkcg_gq *bi_blkg;
struct bio_issue bi_issue;
#endif
#define RQF_QUIET ((__force req_flags_t)(1 << 11))
/* elevator private data attached */
#define RQF_ELVPRIV ((__force req_flags_t)(1 << 12))
-/* account I/O stat */
+/* account into disk and partition IO statistics */
#define RQF_IO_STAT ((__force req_flags_t)(1 << 13))
/* request came from our alloc pool */
#define RQF_ALLOCED ((__force req_flags_t)(1 << 14))
#define RQF_PM ((__force req_flags_t)(1 << 15))
/* on IO scheduler merge hash */
#define RQF_HASHED ((__force req_flags_t)(1 << 16))
-/* IO stats tracking on */
+/* track IO completion time */
#define RQF_STATS ((__force req_flags_t)(1 << 17))
/* Look at ->special_vec for the actual data payload instead of the
bio chain. */
* various queue flags, see QUEUE_* below
*/
unsigned long queue_flags;
+ /*
+ * Number of contexts that have called blk_set_pm_only(). If this
+ * counter is above zero then only RQF_PM and RQF_PREEMPT requests are
+ * processed.
+ */
+ atomic_t pm_only;
/*
* ida allocated id for this queue. Used to index queues from
#define QUEUE_FLAG_FAIL_IO 7 /* fake timeout */
#define QUEUE_FLAG_NONROT 9 /* non-rotational device (SSD) */
#define QUEUE_FLAG_VIRT QUEUE_FLAG_NONROT /* paravirt device */
-#define QUEUE_FLAG_IO_STAT 10 /* do IO stats */
+#define QUEUE_FLAG_IO_STAT 10 /* do disk/partitions IO accounting */
#define QUEUE_FLAG_DISCARD 11 /* supports DISCARD */
#define QUEUE_FLAG_NOXMERGES 12 /* No extended merges */
#define QUEUE_FLAG_ADD_RANDOM 13 /* Contributes to random pool */
#define QUEUE_FLAG_FUA 21 /* device supports FUA writes */
#define QUEUE_FLAG_FLUSH_NQ 22 /* flush not queueuable */
#define QUEUE_FLAG_DAX 23 /* device supports DAX */
-#define QUEUE_FLAG_STATS 24 /* track rq completion times */
+#define QUEUE_FLAG_STATS 24 /* track IO start and completion times */
#define QUEUE_FLAG_POLL_STATS 25 /* collecting stats for hybrid polling */
#define QUEUE_FLAG_REGISTERED 26 /* queue has been registered to a disk */
#define QUEUE_FLAG_SCSI_PASSTHROUGH 27 /* queue supports SCSI commands */
#define QUEUE_FLAG_QUIESCED 28 /* queue has been quiesced */
-#define QUEUE_FLAG_PREEMPT_ONLY 29 /* only process REQ_PREEMPT requests */
#define QUEUE_FLAG_DEFAULT ((1 << QUEUE_FLAG_IO_STAT) | \
(1 << QUEUE_FLAG_SAME_COMP) | \
((rq)->cmd_flags & (REQ_FAILFAST_DEV|REQ_FAILFAST_TRANSPORT| \
REQ_FAILFAST_DRIVER))
#define blk_queue_quiesced(q) test_bit(QUEUE_FLAG_QUIESCED, &(q)->queue_flags)
-#define blk_queue_preempt_only(q) \
- test_bit(QUEUE_FLAG_PREEMPT_ONLY, &(q)->queue_flags)
+#define blk_queue_pm_only(q) atomic_read(&(q)->pm_only)
#define blk_queue_fua(q) test_bit(QUEUE_FLAG_FUA, &(q)->queue_flags)
-extern int blk_set_preempt_only(struct request_queue *q);
-extern void blk_clear_preempt_only(struct request_queue *q);
+extern void blk_set_pm_only(struct request_queue *q);
+extern void blk_clear_pm_only(struct request_queue *q);
static inline int queue_in_flight(struct request_queue *q)
{
extern void blk_put_queue(struct request_queue *);
extern void blk_set_queue_dying(struct request_queue *);
-/*
- * block layer runtime pm functions
- */
-#ifdef CONFIG_PM
-extern void blk_pm_runtime_init(struct request_queue *q, struct device *dev);
-extern int blk_pre_runtime_suspend(struct request_queue *q);
-extern void blk_post_runtime_suspend(struct request_queue *q, int err);
-extern void blk_pre_runtime_resume(struct request_queue *q);
-extern void blk_post_runtime_resume(struct request_queue *q, int err);
-extern void blk_set_runtime_active(struct request_queue *q);
-#else
-static inline void blk_pm_runtime_init(struct request_queue *q,
- struct device *dev) {}
-static inline int blk_pre_runtime_suspend(struct request_queue *q)
-{
- return -ENOSYS;
-}
-static inline void blk_post_runtime_suspend(struct request_queue *q, int err) {}
-static inline void blk_pre_runtime_resume(struct request_queue *q) {}
-static inline void blk_post_runtime_resume(struct request_queue *q, int err) {}
-static inline void blk_set_runtime_active(struct request_queue *q) {}
-#endif
-
/*
* blk_plug permits building a queue of related requests by holding the I/O
* fragments for a short period. This allows merging of sequential requests
put_page(p.v);
}
-static inline bool __bvec_gap_to_prev(struct request_queue *q,
- struct bio_vec *bprv, unsigned int offset)
-{
- return offset ||
- ((bprv->bv_offset + bprv->bv_len) & queue_virt_boundary(q));
-}
-
-/*
- * Check if adding a bio_vec after bprv with offset would create a gap in
- * the SG list. Most drivers don't care about this, but some do.
- */
-static inline bool bvec_gap_to_prev(struct request_queue *q,
- struct bio_vec *bprv, unsigned int offset)
-{
- if (!queue_virt_boundary(q))
- return false;
- return __bvec_gap_to_prev(q, bprv, offset);
-}
-
-/*
- * Check if the two bvecs from two bios can be merged to one segment.
- * If yes, no need to check gap between the two bios since the 1st bio
- * and the 1st bvec in the 2nd bio can be handled in one segment.
- */
-static inline bool bios_segs_mergeable(struct request_queue *q,
- struct bio *prev, struct bio_vec *prev_last_bv,
- struct bio_vec *next_first_bv)
-{
- if (!BIOVEC_PHYS_MERGEABLE(prev_last_bv, next_first_bv))
- return false;
- if (!BIOVEC_SEG_BOUNDARY(q, prev_last_bv, next_first_bv))
- return false;
- if (prev->bi_seg_back_size + next_first_bv->bv_len >
- queue_max_segment_size(q))
- return false;
- return true;
-}
-
-static inline bool bio_will_gap(struct request_queue *q,
- struct request *prev_rq,
- struct bio *prev,
- struct bio *next)
-{
- if (bio_has_data(prev) && queue_virt_boundary(q)) {
- struct bio_vec pb, nb;
-
- /*
- * don't merge if the 1st bio starts with non-zero
- * offset, otherwise it is quite difficult to respect
- * sg gap limit. We work hard to merge a huge number of small
- * single bios in case of mkfs.
- */
- if (prev_rq)
- bio_get_first_bvec(prev_rq->bio, &pb);
- else
- bio_get_first_bvec(prev, &pb);
- if (pb.bv_offset)
- return true;
-
- /*
- * We don't need to worry about the situation that the
- * merged segment ends in unaligned virt boundary:
- *
- * - if 'pb' ends aligned, the merged segment ends aligned
- * - if 'pb' ends unaligned, the next bio must include
- * one single bvec of 'nb', otherwise the 'nb' can't
- * merge with 'pb'
- */
- bio_get_last_bvec(prev, &pb);
- bio_get_first_bvec(next, &nb);
-
- if (!bios_segs_mergeable(q, prev, &pb, &nb))
- return __bvec_gap_to_prev(q, &pb, nb.bv_offset);
- }
-
- return false;
-}
-
-static inline bool req_gap_back_merge(struct request *req, struct bio *bio)
-{
- return bio_will_gap(req->q, req, req->biotail, bio);
-}
-
-static inline bool req_gap_front_merge(struct request *req, struct bio *bio)
-{
- return bio_will_gap(req->q, NULL, bio, req->bio);
-}
-
int kblockd_schedule_work(struct work_struct *work);
int kblockd_schedule_work_on(int cpu, struct work_struct *work);
int kblockd_mod_delayed_work_on(int cpu, struct delayed_work *dwork, unsigned long delay);
return q->limits.max_integrity_segments;
}
-static inline bool integrity_req_gap_back_merge(struct request *req,
- struct bio *next)
-{
- struct bio_integrity_payload *bip = bio_integrity(req->bio);
- struct bio_integrity_payload *bip_next = bio_integrity(next);
-
- return bvec_gap_to_prev(req->q, &bip->bip_vec[bip->bip_vcnt - 1],
- bip_next->bip_vec[0].bv_offset);
-}
-
-static inline bool integrity_req_gap_front_merge(struct request *req,
- struct bio *bio)
-{
- struct bio_integrity_payload *bip = bio_integrity(bio);
- struct bio_integrity_payload *bip_next = bio_integrity(req->bio);
-
- return bvec_gap_to_prev(req->q, &bip->bip_vec[bip->bip_vcnt - 1],
- bip_next->bip_vec[0].bv_offset);
-}
-
/**
* bio_integrity_intervals - Return number of integrity intervals for a bio
* @bi: blk_integrity profile for device
return true;
}
-static inline bool integrity_req_gap_back_merge(struct request *req,
- struct bio *next)
-{
- return false;
-}
-static inline bool integrity_req_gap_front_merge(struct request *req,
- struct bio *bio)
-{
- return false;
-}
-
static inline unsigned int bio_integrity_intervals(struct blk_integrity *bi,
unsigned int sectors)
{
unsigned int bi_idx; /* current index into bvl_vec */
- unsigned int bi_done; /* number of bytes completed */
-
unsigned int bi_bvec_done; /* number of bytes completed in
current bvec */
};
bytes -= len;
iter->bi_size -= len;
iter->bi_bvec_done += len;
- iter->bi_done += len;
if (iter->bi_bvec_done == __bvec_iter_bvec(bv, *iter)->bv_len) {
iter->bi_bvec_done = 0;
* specific task are charged to the dom_cgrp.
*/
struct cgroup *dom_cgrp;
+ struct cgroup *old_dom_cgrp; /* used while enabling threaded */
/* per-cpu recursive resource statistics */
struct cgroup_rstat_cpu __percpu *rstat_cpu;
bool css_has_online_children(struct cgroup_subsys_state *css);
struct cgroup_subsys_state *css_from_id(int id, struct cgroup_subsys *ss);
+struct cgroup_subsys_state *cgroup_e_css(struct cgroup *cgroup,
+ struct cgroup_subsys *ss);
struct cgroup_subsys_state *cgroup_get_e_css(struct cgroup *cgroup,
struct cgroup_subsys *ss);
struct cgroup_subsys_state *css_tryget_online_from_dir(struct dentry *dentry,
compat_size_t ss_size;
} compat_stack_t;
#endif
+#ifndef COMPAT_MINSIGSTKSZ
+#define COMPAT_MINSIGSTKSZ MINSIGSTKSZ
+#endif
#define compat_jiffies_to_clock_t(x) \
(((unsigned long)(x) * COMPAT_USER_HZ) / HZ)
* Conflicts with inlining: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
*/
#define __no_sanitize_address __attribute__((no_sanitize_address))
+#ifdef CONFIG_KASAN
+#define __no_sanitize_address_or_inline \
+ __no_sanitize_address __maybe_unused notrace
+#else
+#define __no_sanitize_address_or_inline inline
+#endif
#endif
#if GCC_VERSION >= 50100
#if !defined(__no_sanitize_address)
#define __no_sanitize_address
+#define __no_sanitize_address_or_inline inline
#endif
/*
* unique, to convince GCC not to merge duplicate inline asm statements.
*/
#define annotate_reachable() ({ \
- asm volatile("%c0:\n\t" \
- ".pushsection .discard.reachable\n\t" \
- ".long %c0b - .\n\t" \
- ".popsection\n\t" : : "i" (__COUNTER__)); \
+ asm volatile("ANNOTATE_REACHABLE counter=%c0" \
+ : : "i" (__COUNTER__)); \
})
#define annotate_unreachable() ({ \
- asm volatile("%c0:\n\t" \
- ".pushsection .discard.unreachable\n\t" \
- ".long %c0b - .\n\t" \
- ".popsection\n\t" : : "i" (__COUNTER__)); \
+ asm volatile("ANNOTATE_UNREACHABLE counter=%c0" \
+ : : "i" (__COUNTER__)); \
})
-#define ASM_UNREACHABLE \
- "999:\n\t" \
- ".pushsection .discard.unreachable\n\t" \
- ".long 999b - .\n\t" \
- ".popsection\n\t"
#else
#define annotate_reachable()
#define annotate_unreachable()
return (void *)((unsigned long)off + *off);
}
+#else /* __ASSEMBLY__ */
+
+#ifdef __KERNEL__
+#ifndef LINKER_SCRIPT
+
+#ifdef CONFIG_STACK_VALIDATION
+.macro ANNOTATE_UNREACHABLE counter:req
+\counter:
+ .pushsection .discard.unreachable
+ .long \counter\()b -.
+ .popsection
+.endm
+
+.macro ANNOTATE_REACHABLE counter:req
+\counter:
+ .pushsection .discard.reachable
+ .long \counter\()b -.
+ .popsection
+.endm
+
+.macro ASM_UNREACHABLE
+999:
+ .pushsection .discard.unreachable
+ .long 999b - .
+ .popsection
+.endm
+#else /* CONFIG_STACK_VALIDATION */
+.macro ANNOTATE_UNREACHABLE counter:req
+.endm
+
+.macro ANNOTATE_REACHABLE counter:req
+.endm
+
+.macro ASM_UNREACHABLE
+.endm
+#endif /* CONFIG_STACK_VALIDATION */
+
+#endif /* LINKER_SCRIPT */
+#endif /* __KERNEL__ */
#endif /* __ASSEMBLY__ */
#ifndef __optimize
unsigned int registered:1;
unsigned int enabled:1;
unsigned int use_deepest_state:1;
+ unsigned int poll_time_limit:1;
unsigned int cpu;
int last_residency;
DECLARE_PER_CPU(struct cpuidle_device *, cpuidle_devices);
DECLARE_PER_CPU(struct cpuidle_device, cpuidle_dev);
-/**
- * cpuidle_get_last_residency - retrieves the last state's residency time
- * @dev: the target CPU
- */
-static inline int cpuidle_get_last_residency(struct cpuidle_device *dev)
-{
- return dev->last_residency;
-}
-
-
/****************************
* CPUIDLE DRIVER INTERFACE *
****************************/
struct task_struct;
-extern int debug_locks;
-extern int debug_locks_silent;
+extern int debug_locks __read_mostly;
+extern int debug_locks_silent __read_mostly;
static inline int __debug_locks_off(void)
extern int devfreq_suspend_device(struct devfreq *devfreq);
extern int devfreq_resume_device(struct devfreq *devfreq);
+/**
+ * update_devfreq() - Reevaluate the device and configure frequency
+ * @devfreq: the devfreq device
+ *
+ * Note: devfreq->lock must be held
+ */
+extern int update_devfreq(struct devfreq *devfreq);
+
/* Helper functions for devfreq user device driver with OPP. */
extern struct dev_pm_opp *devfreq_recommended_opp(struct device *dev,
unsigned long *freq, u32 flags);
* @offline: Set after successful invocation of bus type's .offline().
* @of_node_reused: Set if the device-tree node is shared with an ancestor
* device.
+ * @dma_coherent: this particular device is dma coherent, even if the
+ * architecture supports non-coherent devices.
*
* At the lowest level, every device in a Linux system is represented by an
* instance of struct device. The device structure contains the information
bool offline_disabled:1;
bool offline:1;
bool of_node_reused:1;
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+ defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+ defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+ bool dma_coherent:1;
+#endif
};
static inline struct device *kobj_to_dev(struct kobject *kobj)
extern int dma_debug_resize_entries(u32 num_entries);
+extern void debug_dma_map_single(struct device *dev, const void *addr,
+ unsigned long len);
+
extern void debug_dma_map_page(struct device *dev, struct page *page,
size_t offset, size_t size,
int direction, dma_addr_t dma_addr,
return 0;
}
+static inline void debug_dma_map_single(struct device *dev, const void *addr,
+ unsigned long len)
+{
+}
+
static inline void debug_dma_map_page(struct device *dev, struct page *page,
size_t offset, size_t size,
int direction, dma_addr_t dma_addr,
if (!dev->dma_mask)
return false;
- return addr + size - 1 <= *dev->dma_mask;
+ return addr + size - 1 <=
+ min_not_zero(*dev->dma_mask, dev->bus_dma_mask);
}
#endif /* !CONFIG_ARCH_HAS_PHYS_TO_DMA */
}
#endif /* CONFIG_ARCH_HAS_DMA_MARK_CLEAN */
+u64 dma_direct_get_required_mask(struct device *dev);
void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
gfp_t gfp, unsigned long attrs);
void dma_direct_free(struct device *dev, size_t size, void *cpu_addr,
dma_addr_t dma_addr, unsigned long attrs);
+void *dma_direct_alloc_pages(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs);
+void dma_direct_free_pages(struct device *dev, size_t size, void *cpu_addr,
+ dma_addr_t dma_addr, unsigned long attrs);
dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
unsigned long offset, size_t size, enum dma_data_direction dir,
unsigned long attrs);
enum dma_data_direction direction);
int (*mapping_error)(struct device *dev, dma_addr_t dma_addr);
int (*dma_supported)(struct device *dev, u64 mask);
-#ifdef ARCH_HAS_DMA_GET_REQUIRED_MASK
u64 (*get_required_mask)(struct device *dev);
-#endif
};
extern const struct dma_map_ops dma_direct_ops;
-extern const struct dma_map_ops dma_noncoherent_ops;
extern const struct dma_map_ops dma_virt_ops;
#define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
dma_addr_t addr;
BUG_ON(!valid_dma_direction(dir));
+ debug_dma_map_single(dev, ptr, size);
addr = ops->map_page(dev, virt_to_page(ptr),
offset_in_page(ptr), size,
dir, attrs);
}
extern int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,
- void *cpu_addr, dma_addr_t dma_addr, size_t size);
+ void *cpu_addr, dma_addr_t dma_addr, size_t size,
+ unsigned long attrs);
void *dma_common_contiguous_remap(struct page *page, size_t size,
unsigned long vm_flags,
BUG_ON(!ops);
if (ops->mmap)
return ops->mmap(dev, vma, cpu_addr, dma_addr, size, attrs);
- return dma_common_mmap(dev, vma, cpu_addr, dma_addr, size);
+ return dma_common_mmap(dev, vma, cpu_addr, dma_addr, size, attrs);
}
#define dma_mmap_coherent(d, v, c, h, s) dma_mmap_attrs(d, v, c, h, s, 0)
int
-dma_common_get_sgtable(struct device *dev, struct sg_table *sgt,
- void *cpu_addr, dma_addr_t dma_addr, size_t size);
+dma_common_get_sgtable(struct device *dev, struct sg_table *sgt, void *cpu_addr,
+ dma_addr_t dma_addr, size_t size, unsigned long attrs);
static inline int
dma_get_sgtable_attrs(struct device *dev, struct sg_table *sgt, void *cpu_addr,
if (ops->get_sgtable)
return ops->get_sgtable(dev, sgt, cpu_addr, dma_addr, size,
attrs);
- return dma_common_get_sgtable(dev, sgt, cpu_addr, dma_addr, size);
+ return dma_common_get_sgtable(dev, sgt, cpu_addr, dma_addr, size,
+ attrs);
}
#define dma_get_sgtable(d, t, v, h, s) dma_get_sgtable_attrs(d, t, v, h, s, 0)
}
static inline void *dma_alloc_coherent(struct device *dev, size_t size,
- dma_addr_t *dma_handle, gfp_t flag)
+ dma_addr_t *dma_handle, gfp_t gfp)
{
- return dma_alloc_attrs(dev, size, dma_handle, flag, 0);
+
+ return dma_alloc_attrs(dev, size, dma_handle, gfp,
+ (gfp & __GFP_NOWARN) ? DMA_ATTR_NO_WARN : 0);
}
static inline void dma_free_coherent(struct device *dev, size_t size,
}
#endif /* CONFIG_HAVE_GENERIC_DMA_COHERENT */
-#ifdef CONFIG_HAS_DMA
-int dma_configure(struct device *dev);
-void dma_deconfigure(struct device *dev);
-#else
-static inline int dma_configure(struct device *dev)
-{
- return 0;
-}
-
-static inline void dma_deconfigure(struct device *dev) {}
-#endif
-
/*
* Managed DMA API
*/
static inline void *dma_alloc_wc(struct device *dev, size_t size,
dma_addr_t *dma_addr, gfp_t gfp)
{
- return dma_alloc_attrs(dev, size, dma_addr, gfp,
- DMA_ATTR_WRITE_COMBINE);
+ unsigned long attrs = DMA_ATTR_NO_WARN;
+
+ if (gfp & __GFP_NOWARN)
+ attrs |= DMA_ATTR_NO_WARN;
+
+ return dma_alloc_attrs(dev, size, dma_addr, gfp, attrs);
}
#ifndef dma_alloc_writecombine
#define dma_alloc_writecombine dma_alloc_wc
#include <linux/dma-mapping.h>
+#ifdef CONFIG_ARCH_HAS_DMA_COHERENCE_H
+#include <asm/dma-coherence.h>
+#elif defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+ defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+ defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+static inline bool dev_is_dma_coherent(struct device *dev)
+{
+ return dev->dma_coherent;
+}
+#else
+static inline bool dev_is_dma_coherent(struct device *dev)
+{
+ return true;
+}
+#endif /* CONFIG_ARCH_HAS_DMA_COHERENCE_H */
+
void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
gfp_t gfp, unsigned long attrs);
void arch_dma_free(struct device *dev, size_t size, void *cpu_addr,
dma_addr_t dma_addr, unsigned long attrs);
+long arch_dma_coherent_to_pfn(struct device *dev, void *cpu_addr,
+ dma_addr_t dma_addr);
-#ifdef CONFIG_DMA_NONCOHERENT_MMAP
-int arch_dma_mmap(struct device *dev, struct vm_area_struct *vma,
- void *cpu_addr, dma_addr_t dma_addr, size_t size,
+#ifdef CONFIG_ARCH_HAS_DMA_MMAP_PGPROT
+pgprot_t arch_dma_mmap_pgprot(struct device *dev, pgprot_t prot,
unsigned long attrs);
#else
-#define arch_dma_mmap NULL
-#endif /* CONFIG_DMA_NONCOHERENT_MMAP */
+# define arch_dma_mmap_pgprot(dev, prot, attrs) pgprot_noncached(prot)
+#endif
#ifdef CONFIG_DMA_NONCOHERENT_CACHE_SYNC
void arch_dma_cache_sync(struct device *dev, void *vaddr, size_t size,
#define LINUX_EFI_LOADER_ENTRY_GUID EFI_GUID(0x4a67b082, 0x0a4c, 0x41cf, 0xb6, 0xc7, 0x44, 0x0b, 0x29, 0xbb, 0x8c, 0x4f)
#define LINUX_EFI_RANDOM_SEED_TABLE_GUID EFI_GUID(0x1ce1e5bc, 0x7ceb, 0x42f2, 0x81, 0xe5, 0x8a, 0xad, 0xf1, 0x80, 0xf5, 0x7b)
#define LINUX_EFI_TPM_EVENT_LOG_GUID EFI_GUID(0xb7799cb0, 0xeca2, 0x4943, 0x96, 0x67, 0x1f, 0xae, 0x07, 0xb7, 0x47, 0xfa)
+#define LINUX_EFI_MEMRESERVE_TABLE_GUID EFI_GUID(0x888eb0c6, 0x8ede, 0x4ff5, 0xa8, 0xf0, 0x9a, 0xee, 0x5c, 0xb9, 0x77, 0xc2)
typedef struct {
efi_guid_t guid;
unsigned long mem_attr_table; /* memory attributes table */
unsigned long rng_seed; /* UEFI firmware random seed */
unsigned long tpm_log; /* TPM2 Event Log table */
+ unsigned long mem_reserve; /* Linux EFI memreserve table */
efi_get_time_t *get_time;
efi_set_time_t *set_time;
efi_get_wakeup_time_t *get_wakeup_time;
extern u64 efi_mem_desc_end(efi_memory_desc_t *md);
extern int efi_mem_desc_lookup(u64 phys_addr, efi_memory_desc_t *out_md);
extern void efi_mem_reserve(phys_addr_t addr, u64 size);
+extern int efi_mem_reserve_persistent(phys_addr_t addr, u64 size);
extern void efi_initialize_iomem_resources(struct resource *code_resource,
struct resource *data_resource, struct resource *bss_resource);
extern void efi_reserve_boot_services(void);
extern int efi_tpm_eventlog_init(void);
+/*
+ * efi_runtime_service() function identifiers.
+ * "NONE" is used by efi_recover_from_page_fault() to check if the page
+ * fault happened while executing an efi runtime service.
+ */
+enum efi_rts_ids {
+ NONE,
+ GET_TIME,
+ SET_TIME,
+ GET_WAKEUP_TIME,
+ SET_WAKEUP_TIME,
+ GET_VARIABLE,
+ GET_NEXT_VARIABLE,
+ SET_VARIABLE,
+ QUERY_VARIABLE_INFO,
+ GET_NEXT_HIGH_MONO_COUNT,
+ RESET_SYSTEM,
+ UPDATE_CAPSULE,
+ QUERY_CAPSULE_CAPS,
+};
+
+/*
+ * efi_runtime_work: Details of EFI Runtime Service work
+ * @arg<1-5>: EFI Runtime Service function arguments
+ * @status: Status of executing EFI Runtime Service
+ * @efi_rts_id: EFI Runtime Service function identifier
+ * @efi_rts_comp: Struct used for handling completions
+ */
+struct efi_runtime_work {
+ void *arg1;
+ void *arg2;
+ void *arg3;
+ void *arg4;
+ void *arg5;
+ efi_status_t status;
+ struct work_struct work;
+ enum efi_rts_ids efi_rts_id;
+ struct completion efi_rts_comp;
+};
+
+extern struct efi_runtime_work efi_rts_work;
+
/* Workqueue to queue EFI Runtime Services */
extern struct workqueue_struct *efi_rts_wq;
+struct linux_efi_memreserve {
+ phys_addr_t next;
+ phys_addr_t base;
+ phys_addr_t size;
+};
+
#endif /* _LINUX_EFI_H */
void (*insert_requests)(struct blk_mq_hw_ctx *, struct list_head *, bool);
struct request *(*dispatch_request)(struct blk_mq_hw_ctx *);
bool (*has_work)(struct blk_mq_hw_ctx *);
- void (*completed_request)(struct request *);
+ void (*completed_request)(struct request *, u64);
void (*started_request)(struct request *);
void (*requeue_request)(struct request *);
struct request *(*former_request)(struct request_queue *, struct request *);
*/
/* Auto Boot Mode */
#define IFC_NAND_NCFGR_BOOT 0x80000000
+/* SRAM Initialization */
+#define IFC_NAND_NCFGR_SRAM_INIT_EN 0x20000000
/* Addressing Mode-ROW0+n/COL0 */
#define IFC_NAND_NCFGR_ADDR_MODE_RC0 0x00000000
/* Addressing Mode-ROW0+n/COL0+n */
extern void part_round_stats(struct request_queue *q, int cpu, struct hd_struct *part);
/* block/genhd.c */
-extern void device_add_disk(struct device *parent, struct gendisk *disk);
+extern void device_add_disk(struct device *parent, struct gendisk *disk,
+ const struct attribute_group **groups);
static inline void add_disk(struct gendisk *disk)
{
- device_add_disk(NULL, disk);
+ device_add_disk(NULL, disk, NULL);
}
extern void device_add_disk_no_queue_reg(struct device *parent, struct gendisk *disk);
static inline void add_disk_no_queue_reg(struct gendisk *disk)
*/
struct gpio_desc;
+/**
+ * Opaque descriptor for a structure of GPIO array attributes. This structure
+ * is attached to struct gpiod_descs obtained from gpiod_get_array() and can be
+ * passed back to get/set array functions in order to activate fast processing
+ * path if applicable.
+ */
+struct gpio_array;
+
/**
* Struct containing an array of descriptors that can be obtained using
* gpiod_get_array().
*/
struct gpio_descs {
+ struct gpio_array *info;
unsigned int ndescs;
struct gpio_desc *desc[];
};
#define GPIOD_FLAGS_BIT_DIR_OUT BIT(1)
#define GPIOD_FLAGS_BIT_DIR_VAL BIT(2)
#define GPIOD_FLAGS_BIT_OPEN_DRAIN BIT(3)
+#define GPIOD_FLAGS_BIT_NONEXCLUSIVE BIT(4)
/**
* Optional flags that can be passed to one of gpiod_* to configure direction
/* Value get/set from non-sleeping context */
int gpiod_get_value(const struct gpio_desc *desc);
int gpiod_get_array_value(unsigned int array_size,
- struct gpio_desc **desc_array, int *value_array);
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
void gpiod_set_value(struct gpio_desc *desc, int value);
-void gpiod_set_array_value(unsigned int array_size,
- struct gpio_desc **desc_array, int *value_array);
+int gpiod_set_array_value(unsigned int array_size,
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
int gpiod_get_raw_value(const struct gpio_desc *desc);
int gpiod_get_raw_array_value(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array);
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
void gpiod_set_raw_value(struct gpio_desc *desc, int value);
int gpiod_set_raw_array_value(unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array);
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
/* Value get/set from sleeping context */
int gpiod_get_value_cansleep(const struct gpio_desc *desc);
int gpiod_get_array_value_cansleep(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array);
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
void gpiod_set_value_cansleep(struct gpio_desc *desc, int value);
-void gpiod_set_array_value_cansleep(unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array);
+int gpiod_set_array_value_cansleep(unsigned int array_size,
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
int gpiod_get_raw_value_cansleep(const struct gpio_desc *desc);
int gpiod_get_raw_array_value_cansleep(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array);
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
void gpiod_set_raw_value_cansleep(struct gpio_desc *desc, int value);
int gpiod_set_raw_array_value_cansleep(unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array);
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap);
int gpiod_set_debounce(struct gpio_desc *desc, unsigned debounce);
int gpiod_set_transitory(struct gpio_desc *desc, bool transitory);
}
static inline int gpiod_get_array_value(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array)
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
/* GPIO can never have been requested */
WARN_ON(1);
/* GPIO can never have been requested */
WARN_ON(1);
}
-static inline void gpiod_set_array_value(unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array)
+static inline int gpiod_set_array_value(unsigned int array_size,
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
/* GPIO can never have been requested */
WARN_ON(1);
+ return 0;
}
static inline int gpiod_get_raw_value(const struct gpio_desc *desc)
{
}
static inline int gpiod_get_raw_array_value(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array)
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
/* GPIO can never have been requested */
WARN_ON(1);
WARN_ON(1);
}
static inline int gpiod_set_raw_array_value(unsigned int array_size,
- struct gpio_desc **desc_array,
- int *value_array)
+ struct gpio_desc **desc_array,
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
/* GPIO can never have been requested */
WARN_ON(1);
}
static inline int gpiod_get_array_value_cansleep(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array)
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
/* GPIO can never have been requested */
WARN_ON(1);
/* GPIO can never have been requested */
WARN_ON(1);
}
-static inline void gpiod_set_array_value_cansleep(unsigned int array_size,
+static inline int gpiod_set_array_value_cansleep(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array)
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
/* GPIO can never have been requested */
WARN_ON(1);
+ return 0;
}
static inline int gpiod_get_raw_value_cansleep(const struct gpio_desc *desc)
{
}
static inline int gpiod_get_raw_array_value_cansleep(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array)
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
/* GPIO can never have been requested */
WARN_ON(1);
}
static inline int gpiod_set_raw_array_value_cansleep(unsigned int array_size,
struct gpio_desc **desc_array,
- int *value_array)
+ struct gpio_array *array_info,
+ unsigned long *value_bitmap)
{
/* GPIO can never have been requested */
WARN_ON(1);
/**
* @lock_key:
*
- * Per GPIO IRQ chip lockdep classes.
+ * Per GPIO IRQ chip lockdep class for IRQ lock.
*/
struct lock_class_key *lock_key;
+
+ /**
+ * @request_key:
+ *
+ * Per GPIO IRQ chip lockdep class for IRQ request.
+ */
struct lock_class_key *request_key;
/**
*/
unsigned int num_parents;
+ /**
+ * @parent_irq:
+ *
+ * For use by gpiochip_set_cascaded_irqchip()
+ */
+ unsigned int parent_irq;
+
/**
* @parents:
*
* will allocate and map all IRQs during initialization.
*/
unsigned int first;
+
+ /**
+ * @irq_enable:
+ *
+ * Store old irq_chip irq_enable callback
+ */
+ void (*irq_enable)(struct irq_data *data);
+
+ /**
+ * @irq_disable:
+ *
+ * Store old irq_chip irq_disable callback
+ */
+ void (*irq_disable)(struct irq_data *data);
};
static inline struct gpio_irq_chip *to_gpio_irq_chip(struct irq_chip *chip)
* @free: optional hook for chip-specific deactivation, such as
* disabling module power and clock; may sleep
* @get_direction: returns direction for signal "offset", 0=out, 1=in,
- * (same as GPIOF_DIR_XXX), or negative error
+ * (same as GPIOF_DIR_XXX), or negative error.
+ * It is recommended to always implement this function, even on
+ * input-only or output-only gpio chips.
* @direction_input: configures signal "offset" as input, or returns error
+ * This can be omitted on input-only or output-only gpio chips.
* @direction_output: configures signal "offset" as output, or returns error
+ * This can be omitted on input-only or output-only gpio chips.
* @get: returns value for signal "offset", 0=low, 1=high, or negative error
* @get_multiple: reads values for multiple signals defined by "mask" and
* stores them in "bits", returns 0 on success or negative error
void (*dbg_show)(struct seq_file *s,
struct gpio_chip *chip);
+
+ int (*init_valid_mask)(struct gpio_chip *chip);
+
int base;
u16 ngpio;
const char *const *names;
/**
* @need_valid_mask:
*
- * If set core allocates @valid_mask with all bits set to one.
+ * If set core allocates @valid_mask with all its values initialized
+ * with init_valid_mask() or set to one if init_valid_mask() is not
+ * defined
*/
bool need_valid_mask;
int gpiochip_lock_as_irq(struct gpio_chip *chip, unsigned int offset);
void gpiochip_unlock_as_irq(struct gpio_chip *chip, unsigned int offset);
bool gpiochip_line_is_irq(struct gpio_chip *chip, unsigned int offset);
+int gpiochip_reqres_irq(struct gpio_chip *chip, unsigned int offset);
+void gpiochip_relres_irq(struct gpio_chip *chip, unsigned int offset);
+void gpiochip_disable_irq(struct gpio_chip *chip, unsigned int offset);
+void gpiochip_enable_irq(struct gpio_chip *chip, unsigned int offset);
/* Line status inquiry for drivers */
bool gpiochip_line_is_open_drain(struct gpio_chip *chip, unsigned int offset);
unsigned char *vec);
extern bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,
unsigned long new_addr, unsigned long old_end,
- pmd_t *old_pmd, pmd_t *new_pmd, bool *need_flush);
+ pmd_t *old_pmd, pmd_t *new_pmd);
extern int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd,
unsigned long addr, pgprot_t newprot,
int prot_numa);
hwmon_in_max_alarm,
hwmon_in_lcrit_alarm,
hwmon_in_crit_alarm,
+ hwmon_in_enable,
};
#define HWMON_I_INPUT BIT(hwmon_in_input)
#define HWMON_I_MAX_ALARM BIT(hwmon_in_max_alarm)
#define HWMON_I_LCRIT_ALARM BIT(hwmon_in_lcrit_alarm)
#define HWMON_I_CRIT_ALARM BIT(hwmon_in_crit_alarm)
+#define HWMON_I_ENABLE BIT(hwmon_in_enable)
enum hwmon_curr_attributes {
hwmon_curr_input,
* Opaque type for a IPMI message user. One of these is needed to
* send and receive messages.
*/
-typedef struct ipmi_user *ipmi_user_t;
+struct ipmi_user;
/*
* Stuff coming from the receive interface comes as one of these.
*/
/* Structure for the low-level drivers. */
-typedef struct ipmi_smi *ipmi_smi_t;
+struct ipmi_smi;
/*
* Messages to/from the lower layer. The smi interface will take one
void irq_matrix_assign_system(struct irq_matrix *m, unsigned int bit, bool replace);
int irq_matrix_reserve_managed(struct irq_matrix *m, const struct cpumask *msk);
void irq_matrix_remove_managed(struct irq_matrix *m, const struct cpumask *msk);
-int irq_matrix_alloc_managed(struct irq_matrix *m, unsigned int cpu);
+int irq_matrix_alloc_managed(struct irq_matrix *m, const struct cpumask *msk,
+ unsigned int *mapped_cpu);
void irq_matrix_reserve(struct irq_matrix *m);
void irq_matrix_remove_reserved(struct irq_matrix *m);
int irq_matrix_alloc(struct irq_matrix *m, const struct cpumask *msk,
#ifdef HAVE_JUMP_LABEL
#include <asm/jump_label.h>
+
+#ifndef __ASSEMBLY__
+#ifdef CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE
+
+struct jump_entry {
+ s32 code;
+ s32 target;
+ long key; // key may be far away from the core kernel under KASLR
+};
+
+static inline unsigned long jump_entry_code(const struct jump_entry *entry)
+{
+ return (unsigned long)&entry->code + entry->code;
+}
+
+static inline unsigned long jump_entry_target(const struct jump_entry *entry)
+{
+ return (unsigned long)&entry->target + entry->target;
+}
+
+static inline struct static_key *jump_entry_key(const struct jump_entry *entry)
+{
+ long offset = entry->key & ~3L;
+
+ return (struct static_key *)((unsigned long)&entry->key + offset);
+}
+
+#else
+
+static inline unsigned long jump_entry_code(const struct jump_entry *entry)
+{
+ return entry->code;
+}
+
+static inline unsigned long jump_entry_target(const struct jump_entry *entry)
+{
+ return entry->target;
+}
+
+static inline struct static_key *jump_entry_key(const struct jump_entry *entry)
+{
+ return (struct static_key *)((unsigned long)entry->key & ~3UL);
+}
+
+#endif
+
+static inline bool jump_entry_is_branch(const struct jump_entry *entry)
+{
+ return (unsigned long)entry->key & 1UL;
+}
+
+static inline bool jump_entry_is_init(const struct jump_entry *entry)
+{
+ return (unsigned long)entry->key & 2UL;
+}
+
+static inline void jump_entry_set_init(struct jump_entry *entry)
+{
+ entry->key |= 2;
+}
+
+#endif
#endif
#ifndef __ASSEMBLY__
extern struct jump_entry __stop___jump_table[];
extern void jump_label_init(void);
-extern void jump_label_invalidate_initmem(void);
extern void jump_label_lock(void);
extern void jump_label_unlock(void);
extern void arch_jump_label_transform(struct jump_entry *entry,
static_key_initialized = true;
}
-static inline void jump_label_invalidate_initmem(void) {}
-
static __always_inline bool static_key_false(struct static_key *key)
{
if (unlikely(static_key_count(key) > 0))
#include <linux/workqueue.h>
struct device;
+struct led_pattern;
/*
* LED Core
*/
unsigned long *delay_on,
unsigned long *delay_off);
+ int (*pattern_set)(struct led_classdev *led_cdev,
+ struct led_pattern *pattern, u32 len, int repeat);
+ int (*pattern_clear)(struct led_classdev *led_cdev);
+
struct device *dev;
const struct attribute_group **groups;
struct led_classdev *led_cdev, enum led_brightness brightness) { }
#endif
+/**
+ * struct led_pattern - pattern interval settings
+ * @delta_t: pattern interval delay, in milliseconds
+ * @brightness: pattern interval brightness
+ */
+struct led_pattern {
+ u32 delta_t;
+ int brightness;
+};
+
#endif /* __LINUX_LEDS_H_INCLUDED */
typedef int (nvm_id_fn)(struct nvm_dev *);
typedef int (nvm_op_bb_tbl_fn)(struct nvm_dev *, struct ppa_addr, u8 *);
typedef int (nvm_op_set_bb_fn)(struct nvm_dev *, struct ppa_addr *, int, int);
-typedef int (nvm_get_chk_meta_fn)(struct nvm_dev *, struct nvm_chk_meta *,
- sector_t, int);
+typedef int (nvm_get_chk_meta_fn)(struct nvm_dev *, sector_t, int,
+ struct nvm_chk_meta *);
typedef int (nvm_submit_io_fn)(struct nvm_dev *, struct nvm_rq *);
typedef int (nvm_submit_io_sync_fn)(struct nvm_dev *, struct nvm_rq *);
typedef void *(nvm_create_dma_pool_fn)(struct nvm_dev *, char *);
u64 ppa_status; /* ppa media status */
int error;
+ int is_seq; /* Sequential hint flag. 1.2 only */
+
void *private;
};
return rqdata + 1;
}
+static inline struct ppa_addr *nvm_rq_to_ppa_list(struct nvm_rq *rqd)
+{
+ return (rqd->nr_ppas > 1) ? rqd->ppa_list : &rqd->ppa_addr;
+}
+
enum {
NVM_BLK_ST_FREE = 0x1, /* Free block */
NVM_BLK_ST_TGT = 0x2, /* Block in use by target */
return l;
}
+static inline u64 dev_to_chunk_addr(struct nvm_dev *dev, void *addrf,
+ struct ppa_addr p)
+{
+ struct nvm_geo *geo = &dev->geo;
+ u64 caddr;
+
+ if (geo->version == NVM_OCSSD_SPEC_12) {
+ struct nvm_addrf_12 *ppaf = (struct nvm_addrf_12 *)addrf;
+
+ caddr = (u64)p.g.pg << ppaf->pg_offset;
+ caddr |= (u64)p.g.pl << ppaf->pln_offset;
+ caddr |= (u64)p.g.sec << ppaf->sec_offset;
+ } else {
+ caddr = p.m.sec;
+ }
+
+ return caddr;
+}
+
+static inline struct ppa_addr nvm_ppa32_to_ppa64(struct nvm_dev *dev,
+ void *addrf, u32 ppa32)
+{
+ struct ppa_addr ppa64;
+
+ ppa64.ppa = 0;
+
+ if (ppa32 == -1) {
+ ppa64.ppa = ADDR_EMPTY;
+ } else if (ppa32 & (1U << 31)) {
+ ppa64.c.line = ppa32 & ((~0U) >> 1);
+ ppa64.c.is_cached = 1;
+ } else {
+ struct nvm_geo *geo = &dev->geo;
+
+ if (geo->version == NVM_OCSSD_SPEC_12) {
+ struct nvm_addrf_12 *ppaf = addrf;
+
+ ppa64.g.ch = (ppa32 & ppaf->ch_mask) >>
+ ppaf->ch_offset;
+ ppa64.g.lun = (ppa32 & ppaf->lun_mask) >>
+ ppaf->lun_offset;
+ ppa64.g.blk = (ppa32 & ppaf->blk_mask) >>
+ ppaf->blk_offset;
+ ppa64.g.pg = (ppa32 & ppaf->pg_mask) >>
+ ppaf->pg_offset;
+ ppa64.g.pl = (ppa32 & ppaf->pln_mask) >>
+ ppaf->pln_offset;
+ ppa64.g.sec = (ppa32 & ppaf->sec_mask) >>
+ ppaf->sec_offset;
+ } else {
+ struct nvm_addrf *lbaf = addrf;
+
+ ppa64.m.grp = (ppa32 & lbaf->ch_mask) >>
+ lbaf->ch_offset;
+ ppa64.m.pu = (ppa32 & lbaf->lun_mask) >>
+ lbaf->lun_offset;
+ ppa64.m.chk = (ppa32 & lbaf->chk_mask) >>
+ lbaf->chk_offset;
+ ppa64.m.sec = (ppa32 & lbaf->sec_mask) >>
+ lbaf->sec_offset;
+ }
+ }
+
+ return ppa64;
+}
+
+static inline u32 nvm_ppa64_to_ppa32(struct nvm_dev *dev,
+ void *addrf, struct ppa_addr ppa64)
+{
+ u32 ppa32 = 0;
+
+ if (ppa64.ppa == ADDR_EMPTY) {
+ ppa32 = ~0U;
+ } else if (ppa64.c.is_cached) {
+ ppa32 |= ppa64.c.line;
+ ppa32 |= 1U << 31;
+ } else {
+ struct nvm_geo *geo = &dev->geo;
+
+ if (geo->version == NVM_OCSSD_SPEC_12) {
+ struct nvm_addrf_12 *ppaf = addrf;
+
+ ppa32 |= ppa64.g.ch << ppaf->ch_offset;
+ ppa32 |= ppa64.g.lun << ppaf->lun_offset;
+ ppa32 |= ppa64.g.blk << ppaf->blk_offset;
+ ppa32 |= ppa64.g.pg << ppaf->pg_offset;
+ ppa32 |= ppa64.g.pl << ppaf->pln_offset;
+ ppa32 |= ppa64.g.sec << ppaf->sec_offset;
+ } else {
+ struct nvm_addrf *lbaf = addrf;
+
+ ppa32 |= ppa64.m.grp << lbaf->ch_offset;
+ ppa32 |= ppa64.m.pu << lbaf->lun_offset;
+ ppa32 |= ppa64.m.chk << lbaf->chk_offset;
+ ppa32 |= ppa64.m.sec << lbaf->sec_offset;
+ }
+ }
+
+ return ppa32;
+}
+
+static inline int nvm_next_ppa_in_chk(struct nvm_tgt_dev *dev,
+ struct ppa_addr *ppa)
+{
+ struct nvm_geo *geo = &dev->geo;
+ int last = 0;
+
+ if (geo->version == NVM_OCSSD_SPEC_12) {
+ int sec = ppa->g.sec;
+
+ sec++;
+ if (sec == geo->ws_min) {
+ int pg = ppa->g.pg;
+
+ sec = 0;
+ pg++;
+ if (pg == geo->num_pg) {
+ int pl = ppa->g.pl;
+
+ pg = 0;
+ pl++;
+ if (pl == geo->num_pln)
+ last = 1;
+
+ ppa->g.pl = pl;
+ }
+ ppa->g.pg = pg;
+ }
+ ppa->g.sec = sec;
+ } else {
+ ppa->m.sec++;
+ if (ppa->m.sec == geo->clba)
+ last = 1;
+ }
+
+ return last;
+}
+
typedef blk_qc_t (nvm_tgt_make_rq_fn)(struct request_queue *, struct bio *);
typedef sector_t (nvm_tgt_capacity_fn)(void *);
typedef void *(nvm_tgt_init_fn)(struct nvm_tgt_dev *, struct gendisk *,
typedef int (nvm_tgt_sysfs_init_fn)(struct gendisk *);
typedef void (nvm_tgt_sysfs_exit_fn)(struct gendisk *);
+enum {
+ NVM_TGT_F_DEV_L2P = 0,
+ NVM_TGT_F_HOST_L2P = 1 << 0,
+};
+
struct nvm_tgt_type {
const char *name;
unsigned int version[3];
+ int flags;
/* target entry points */
nvm_tgt_make_rq_fn *make_rq;
extern int nvm_register(struct nvm_dev *);
extern void nvm_unregister(struct nvm_dev *);
-
-extern int nvm_get_chunk_meta(struct nvm_tgt_dev *tgt_dev,
- struct nvm_chk_meta *meta, struct ppa_addr ppa,
- int nchks);
-
-extern int nvm_set_tgt_bb_tbl(struct nvm_tgt_dev *, struct ppa_addr *,
+extern int nvm_get_chunk_meta(struct nvm_tgt_dev *, struct ppa_addr,
+ int, struct nvm_chk_meta *);
+extern int nvm_set_chunk_meta(struct nvm_tgt_dev *, struct ppa_addr *,
int, int);
extern int nvm_submit_io(struct nvm_tgt_dev *, struct nvm_rq *);
extern int nvm_submit_io_sync(struct nvm_tgt_dev *, struct nvm_rq *);
extern void nvm_end_io(struct nvm_rq *);
-extern int nvm_bb_tbl_fold(struct nvm_dev *, u8 *, int);
-extern int nvm_get_tgt_bb_tbl(struct nvm_tgt_dev *, struct ppa_addr, u8 *);
#else /* CONFIG_NVM */
struct nvm_dev_ops;
*/
unsigned int version;
- /*
- * Statistics counter:
- */
- unsigned long ops;
-
- const char *name;
int name_version;
+ const char *name;
#ifdef CONFIG_LOCK_STAT
unsigned long contention_point[LOCKSTAT_POINTS];
/* SPDX-License-Identifier: GPL-2.0-or-later */
/* Copyright (C) 2018 ROHM Semiconductors */
-#ifndef __LINUX_MFD_BD71837_H__
-#define __LINUX_MFD_BD71837_H__
+#ifndef __LINUX_MFD_BD718XX_H__
+#define __LINUX_MFD_BD718XX_H__
#include <linux/regmap.h>
enum {
- BD71837_BUCK1 = 0,
- BD71837_BUCK2,
- BD71837_BUCK3,
- BD71837_BUCK4,
- BD71837_BUCK5,
- BD71837_BUCK6,
- BD71837_BUCK7,
- BD71837_BUCK8,
- BD71837_LDO1,
- BD71837_LDO2,
- BD71837_LDO3,
- BD71837_LDO4,
- BD71837_LDO5,
- BD71837_LDO6,
- BD71837_LDO7,
- BD71837_REGULATOR_CNT,
+ BD718XX_TYPE_BD71837 = 0,
+ BD718XX_TYPE_BD71847,
+ BD718XX_TYPE_AMOUNT
};
-#define BD71837_BUCK1_VOLTAGE_NUM 0x40
-#define BD71837_BUCK2_VOLTAGE_NUM 0x40
-#define BD71837_BUCK3_VOLTAGE_NUM 0x40
-#define BD71837_BUCK4_VOLTAGE_NUM 0x40
+enum {
+ BD718XX_BUCK1 = 0,
+ BD718XX_BUCK2,
+ BD718XX_BUCK3,
+ BD718XX_BUCK4,
+ BD718XX_BUCK5,
+ BD718XX_BUCK6,
+ BD718XX_BUCK7,
+ BD718XX_BUCK8,
+ BD718XX_LDO1,
+ BD718XX_LDO2,
+ BD718XX_LDO3,
+ BD718XX_LDO4,
+ BD718XX_LDO5,
+ BD718XX_LDO6,
+ BD718XX_LDO7,
+ BD718XX_REGULATOR_AMOUNT,
+};
+
+/* Common voltage configurations */
+#define BD718XX_DVS_BUCK_VOLTAGE_NUM 0x3D
+#define BD718XX_4TH_NODVS_BUCK_VOLTAGE_NUM 0x3D
-#define BD71837_BUCK5_VOLTAGE_NUM 0x08
+#define BD718XX_LDO1_VOLTAGE_NUM 0x08
+#define BD718XX_LDO2_VOLTAGE_NUM 0x02
+#define BD718XX_LDO3_VOLTAGE_NUM 0x10
+#define BD718XX_LDO4_VOLTAGE_NUM 0x0A
+#define BD718XX_LDO6_VOLTAGE_NUM 0x0A
+
+/* BD71837 specific voltage configurations */
+#define BD71837_BUCK5_VOLTAGE_NUM 0x10
#define BD71837_BUCK6_VOLTAGE_NUM 0x04
#define BD71837_BUCK7_VOLTAGE_NUM 0x08
-#define BD71837_BUCK8_VOLTAGE_NUM 0x40
-
-#define BD71837_LDO1_VOLTAGE_NUM 0x04
-#define BD71837_LDO2_VOLTAGE_NUM 0x02
-#define BD71837_LDO3_VOLTAGE_NUM 0x10
-#define BD71837_LDO4_VOLTAGE_NUM 0x10
#define BD71837_LDO5_VOLTAGE_NUM 0x10
-#define BD71837_LDO6_VOLTAGE_NUM 0x10
#define BD71837_LDO7_VOLTAGE_NUM 0x10
+/* BD71847 specific voltage configurations */
+#define BD71847_BUCK3_VOLTAGE_NUM 0x18
+#define BD71847_BUCK4_VOLTAGE_NUM 0x08
+#define BD71847_LDO5_VOLTAGE_NUM 0x20
+
+/* Registers specific to BD71837 */
enum {
- BD71837_REG_REV = 0x00,
- BD71837_REG_SWRESET = 0x01,
- BD71837_REG_I2C_DEV = 0x02,
- BD71837_REG_PWRCTRL0 = 0x03,
- BD71837_REG_PWRCTRL1 = 0x04,
- BD71837_REG_BUCK1_CTRL = 0x05,
- BD71837_REG_BUCK2_CTRL = 0x06,
- BD71837_REG_BUCK3_CTRL = 0x07,
- BD71837_REG_BUCK4_CTRL = 0x08,
- BD71837_REG_BUCK5_CTRL = 0x09,
- BD71837_REG_BUCK6_CTRL = 0x0A,
- BD71837_REG_BUCK7_CTRL = 0x0B,
- BD71837_REG_BUCK8_CTRL = 0x0C,
- BD71837_REG_BUCK1_VOLT_RUN = 0x0D,
- BD71837_REG_BUCK1_VOLT_IDLE = 0x0E,
- BD71837_REG_BUCK1_VOLT_SUSP = 0x0F,
- BD71837_REG_BUCK2_VOLT_RUN = 0x10,
- BD71837_REG_BUCK2_VOLT_IDLE = 0x11,
- BD71837_REG_BUCK3_VOLT_RUN = 0x12,
- BD71837_REG_BUCK4_VOLT_RUN = 0x13,
- BD71837_REG_BUCK5_VOLT = 0x14,
- BD71837_REG_BUCK6_VOLT = 0x15,
- BD71837_REG_BUCK7_VOLT = 0x16,
- BD71837_REG_BUCK8_VOLT = 0x17,
- BD71837_REG_LDO1_VOLT = 0x18,
- BD71837_REG_LDO2_VOLT = 0x19,
- BD71837_REG_LDO3_VOLT = 0x1A,
- BD71837_REG_LDO4_VOLT = 0x1B,
- BD71837_REG_LDO5_VOLT = 0x1C,
- BD71837_REG_LDO6_VOLT = 0x1D,
- BD71837_REG_LDO7_VOLT = 0x1E,
- BD71837_REG_TRANS_COND0 = 0x1F,
- BD71837_REG_TRANS_COND1 = 0x20,
- BD71837_REG_VRFAULTEN = 0x21,
- BD718XX_REG_MVRFLTMASK0 = 0x22,
- BD718XX_REG_MVRFLTMASK1 = 0x23,
- BD718XX_REG_MVRFLTMASK2 = 0x24,
- BD71837_REG_RCVCFG = 0x25,
- BD71837_REG_RCVNUM = 0x26,
- BD71837_REG_PWRONCONFIG0 = 0x27,
- BD71837_REG_PWRONCONFIG1 = 0x28,
- BD71837_REG_RESETSRC = 0x29,
- BD71837_REG_MIRQ = 0x2A,
- BD71837_REG_IRQ = 0x2B,
- BD71837_REG_IN_MON = 0x2C,
- BD71837_REG_POW_STATE = 0x2D,
- BD71837_REG_OUT32K = 0x2E,
- BD71837_REG_REGLOCK = 0x2F,
- BD71837_REG_OTPVER = 0xFF,
- BD71837_MAX_REGISTER = 0x100,
+ BD71837_REG_BUCK3_CTRL = 0x07,
+ BD71837_REG_BUCK4_CTRL = 0x08,
+ BD71837_REG_BUCK3_VOLT_RUN = 0x12,
+ BD71837_REG_BUCK4_VOLT_RUN = 0x13,
+ BD71837_REG_LDO7_VOLT = 0x1E,
+};
+
+/* Registers common for BD71837 and BD71847 */
+enum {
+ BD718XX_REG_REV = 0x00,
+ BD718XX_REG_SWRESET = 0x01,
+ BD718XX_REG_I2C_DEV = 0x02,
+ BD718XX_REG_PWRCTRL0 = 0x03,
+ BD718XX_REG_PWRCTRL1 = 0x04,
+ BD718XX_REG_BUCK1_CTRL = 0x05,
+ BD718XX_REG_BUCK2_CTRL = 0x06,
+ BD718XX_REG_1ST_NODVS_BUCK_CTRL = 0x09,
+ BD718XX_REG_2ND_NODVS_BUCK_CTRL = 0x0A,
+ BD718XX_REG_3RD_NODVS_BUCK_CTRL = 0x0B,
+ BD718XX_REG_4TH_NODVS_BUCK_CTRL = 0x0C,
+ BD718XX_REG_BUCK1_VOLT_RUN = 0x0D,
+ BD718XX_REG_BUCK1_VOLT_IDLE = 0x0E,
+ BD718XX_REG_BUCK1_VOLT_SUSP = 0x0F,
+ BD718XX_REG_BUCK2_VOLT_RUN = 0x10,
+ BD718XX_REG_BUCK2_VOLT_IDLE = 0x11,
+ BD718XX_REG_1ST_NODVS_BUCK_VOLT = 0x14,
+ BD718XX_REG_2ND_NODVS_BUCK_VOLT = 0x15,
+ BD718XX_REG_3RD_NODVS_BUCK_VOLT = 0x16,
+ BD718XX_REG_4TH_NODVS_BUCK_VOLT = 0x17,
+ BD718XX_REG_LDO1_VOLT = 0x18,
+ BD718XX_REG_LDO2_VOLT = 0x19,
+ BD718XX_REG_LDO3_VOLT = 0x1A,
+ BD718XX_REG_LDO4_VOLT = 0x1B,
+ BD718XX_REG_LDO5_VOLT = 0x1C,
+ BD718XX_REG_LDO6_VOLT = 0x1D,
+ BD718XX_REG_TRANS_COND0 = 0x1F,
+ BD718XX_REG_TRANS_COND1 = 0x20,
+ BD718XX_REG_VRFAULTEN = 0x21,
+ BD718XX_REG_MVRFLTMASK0 = 0x22,
+ BD718XX_REG_MVRFLTMASK1 = 0x23,
+ BD718XX_REG_MVRFLTMASK2 = 0x24,
+ BD718XX_REG_RCVCFG = 0x25,
+ BD718XX_REG_RCVNUM = 0x26,
+ BD718XX_REG_PWRONCONFIG0 = 0x27,
+ BD718XX_REG_PWRONCONFIG1 = 0x28,
+ BD718XX_REG_RESETSRC = 0x29,
+ BD718XX_REG_MIRQ = 0x2A,
+ BD718XX_REG_IRQ = 0x2B,
+ BD718XX_REG_IN_MON = 0x2C,
+ BD718XX_REG_POW_STATE = 0x2D,
+ BD718XX_REG_OUT32K = 0x2E,
+ BD718XX_REG_REGLOCK = 0x2F,
+ BD718XX_REG_OTPVER = 0xFF,
+ BD718XX_MAX_REGISTER = 0x100,
};
#define REGLOCK_PWRSEQ 0x1
#define REGLOCK_VREG 0x10
/* Generic BUCK control masks */
-#define BD71837_BUCK_SEL 0x02
-#define BD71837_BUCK_EN 0x01
-#define BD71837_BUCK_RUN_ON 0x04
+#define BD718XX_BUCK_SEL 0x02
+#define BD718XX_BUCK_EN 0x01
+#define BD718XX_BUCK_RUN_ON 0x04
/* Generic LDO masks */
-#define BD71837_LDO_SEL 0x80
-#define BD71837_LDO_EN 0x40
+#define BD718XX_LDO_SEL 0x80
+#define BD718XX_LDO_EN 0x40
/* BD71837 BUCK ramp rate CTRL reg bits */
#define BUCK_RAMPRATE_MASK 0xC0
#define BUCK_RAMPRATE_2P50MV 0x2
#define BUCK_RAMPRATE_1P25MV 0x3
-/* BD71837_REG_BUCK1_VOLT_RUN bits */
-#define BUCK1_RUN_MASK 0x3F
-#define BUCK1_RUN_DEFAULT 0x14
-
-/* BD71837_REG_BUCK1_VOLT_SUSP bits */
-#define BUCK1_SUSP_MASK 0x3F
-#define BUCK1_SUSP_DEFAULT 0x14
+#define DVS_BUCK_RUN_MASK 0x3F
+#define DVS_BUCK_SUSP_MASK 0x3F
+#define DVS_BUCK_IDLE_MASK 0x3F
-/* BD71837_REG_BUCK1_VOLT_IDLE bits */
-#define BUCK1_IDLE_MASK 0x3F
-#define BUCK1_IDLE_DEFAULT 0x14
+#define BD718XX_1ST_NODVS_BUCK_MASK 0x07
+#define BD718XX_3RD_NODVS_BUCK_MASK 0x07
+#define BD718XX_4TH_NODVS_BUCK_MASK 0x3F
-/* BD71837_REG_BUCK2_VOLT_RUN bits */
-#define BUCK2_RUN_MASK 0x3F
-#define BUCK2_RUN_DEFAULT 0x1E
+#define BD71847_BUCK3_MASK 0x07
+#define BD71847_BUCK3_RANGE_MASK 0xC0
+#define BD71847_BUCK4_MASK 0x03
+#define BD71847_BUCK4_RANGE_MASK 0x40
-/* BD71837_REG_BUCK2_VOLT_IDLE bits */
-#define BUCK2_IDLE_MASK 0x3F
-#define BUCK2_IDLE_DEFAULT 0x14
+#define BD71837_BUCK5_MASK 0x07
+#define BD71837_BUCK5_RANGE_MASK 0x80
+#define BD71837_BUCK6_MASK 0x03
-/* BD71837_REG_BUCK3_VOLT_RUN bits */
-#define BUCK3_RUN_MASK 0x3F
-#define BUCK3_RUN_DEFAULT 0x1E
+#define BD718XX_LDO1_MASK 0x03
+#define BD718XX_LDO1_RANGE_MASK 0x20
+#define BD718XX_LDO2_MASK 0x20
+#define BD718XX_LDO3_MASK 0x0F
+#define BD718XX_LDO4_MASK 0x0F
+#define BD718XX_LDO6_MASK 0x0F
-/* BD71837_REG_BUCK4_VOLT_RUN bits */
-#define BUCK4_RUN_MASK 0x3F
-#define BUCK4_RUN_DEFAULT 0x1E
+#define BD71837_LDO5_MASK 0x0F
+#define BD71847_LDO5_MASK 0x0F
+#define BD71847_LDO5_RANGE_MASK 0x20
-/* BD71837_REG_BUCK5_VOLT bits */
-#define BUCK5_MASK 0x07
-#define BUCK5_DEFAULT 0x02
-
-/* BD71837_REG_BUCK6_VOLT bits */
-#define BUCK6_MASK 0x03
-#define BUCK6_DEFAULT 0x03
-
-/* BD71837_REG_BUCK7_VOLT bits */
-#define BUCK7_MASK 0x07
-#define BUCK7_DEFAULT 0x03
-
-/* BD71837_REG_BUCK8_VOLT bits */
-#define BUCK8_MASK 0x3F
-#define BUCK8_DEFAULT 0x1E
+#define BD71837_LDO7_MASK 0x0F
/* BD718XX Voltage monitoring masks */
#define BD718XX_BUCK1_VRMON80 0x1
#define BD71837_BUCK4_VRMON130 0x80
#define BD71837_LDO7_VRMON80 0x40
-/* BD71837_REG_IRQ bits */
+/* BD718XX_REG_IRQ bits */
#define IRQ_SWRST 0x40
#define IRQ_PWRON_S 0x20
#define IRQ_PWRON_L 0x10
#define IRQ_ON_REQ 0x02
#define IRQ_STBY_REQ 0x01
-/* BD71837_REG_OUT32K bits */
-#define BD71837_OUT32K_EN 0x01
+/* BD718XX_REG_OUT32K bits */
+#define BD718XX_OUT32K_EN 0x01
-/* BD71837 gated clock rate */
-#define BD71837_CLK_RATE 32768
+/* BD7183XX gated clock rate */
+#define BD718XX_CLK_RATE 32768
-/* ROHM BD71837 irqs */
+/* ROHM BD718XX irqs */
enum {
- BD71837_INT_STBY_REQ,
- BD71837_INT_ON_REQ,
- BD71837_INT_WDOG,
- BD71837_INT_PWRBTN,
- BD71837_INT_PWRBTN_L,
- BD71837_INT_PWRBTN_S,
- BD71837_INT_SWRST
+ BD718XX_INT_STBY_REQ,
+ BD718XX_INT_ON_REQ,
+ BD718XX_INT_WDOG,
+ BD718XX_INT_PWRBTN,
+ BD718XX_INT_PWRBTN_L,
+ BD718XX_INT_PWRBTN_S,
+ BD718XX_INT_SWRST
};
-/* ROHM BD71837 interrupt masks */
-#define BD71837_INT_SWRST_MASK 0x40
-#define BD71837_INT_PWRBTN_S_MASK 0x20
-#define BD71837_INT_PWRBTN_L_MASK 0x10
-#define BD71837_INT_PWRBTN_MASK 0x8
-#define BD71837_INT_WDOG_MASK 0x4
-#define BD71837_INT_ON_REQ_MASK 0x2
-#define BD71837_INT_STBY_REQ_MASK 0x1
-
-/* BD71837_REG_LDO1_VOLT bits */
-#define LDO1_MASK 0x03
-
-/* BD71837_REG_LDO1_VOLT bits */
-#define LDO2_MASK 0x20
-
-/* BD71837_REG_LDO3_VOLT bits */
-#define LDO3_MASK 0x0F
-
-/* BD71837_REG_LDO4_VOLT bits */
-#define LDO4_MASK 0x0F
-
-/* BD71837_REG_LDO5_VOLT bits */
-#define LDO5_MASK 0x0F
-
-/* BD71837_REG_LDO6_VOLT bits */
-#define LDO6_MASK 0x0F
-
-/* BD71837_REG_LDO7_VOLT bits */
-#define LDO7_MASK 0x0F
+/* ROHM BD718XX interrupt masks */
+#define BD718XX_INT_SWRST_MASK 0x40
+#define BD718XX_INT_PWRBTN_S_MASK 0x20
+#define BD718XX_INT_PWRBTN_L_MASK 0x10
+#define BD718XX_INT_PWRBTN_MASK 0x8
+#define BD718XX_INT_WDOG_MASK 0x4
+#define BD718XX_INT_ON_REQ_MASK 0x2
+#define BD718XX_INT_STBY_REQ_MASK 0x1
/* Register write induced reset settings */
* write 1 to it we will trigger the action. So always write 0 to it when
* changning SWRESET action - no matter what we read from it.
*/
-#define BD71837_SWRESET_TYPE_MASK 7
-#define BD71837_SWRESET_TYPE_DISABLED 0
-#define BD71837_SWRESET_TYPE_COLD 4
-#define BD71837_SWRESET_TYPE_WARM 6
+#define BD718XX_SWRESET_TYPE_MASK 7
+#define BD718XX_SWRESET_TYPE_DISABLED 0
+#define BD718XX_SWRESET_TYPE_COLD 4
+#define BD718XX_SWRESET_TYPE_WARM 6
-#define BD71837_SWRESET_RESET_MASK 1
-#define BD71837_SWRESET_RESET 1
+#define BD718XX_SWRESET_RESET_MASK 1
+#define BD718XX_SWRESET_RESET 1
/* Poweroff state transition conditions */
BD718XX_PWRBTN_LONG_PRESS_15S
};
-struct bd71837_pmic;
-struct bd71837_clk;
+struct bd718xx_clk;
-struct bd71837 {
+struct bd718xx {
+ unsigned int chip_type;
struct device *dev;
struct regmap *regmap;
unsigned long int id;
int chip_irq;
struct regmap_irq_chip_data *irq_data;
- struct bd71837_pmic *pmic;
- struct bd71837_clk *clk;
+ struct bd718xx_clk *clk;
};
-#endif /* __LINUX_MFD_BD71837_H__ */
+#endif /* __LINUX_MFD_BD718XX_H__ */
*/
#define TMIO_MMC_USE_GPIO_CD BIT(5)
-/*
- * Some controllers doesn't have over 0x100 register.
- * it is used to checking accessibility of
- * CTL_SD_CARD_CLK_CTL / CTL_CLK_AND_WAIT_CTL
- */
-#define TMIO_MMC_HAVE_HIGH_REG BIT(6)
-
/*
* Some controllers have CMD12 automatically
* issue/non-issue register
((fbc->frag_sz_m1 & ix) << fbc->log_stride);
}
+static inline u32
+mlx5_frag_buf_get_idx_last_contig_stride(struct mlx5_frag_buf_ctrl *fbc, u32 ix)
+{
+ u32 last_frag_stride_idx = (ix + fbc->strides_offset) | fbc->frag_sz_m1;
+
+ return min_t(u32, last_frag_stride_idx - fbc->strides_offset, fbc->sz_m1);
+}
+
int mlx5_cmd_init(struct mlx5_core_dev *dev);
void mlx5_cmd_cleanup(struct mlx5_core_dev *dev);
void mlx5_cmd_use_events(struct mlx5_core_dev *dev);
return host->can_retune == 1;
}
+static inline bool mmc_doing_retune(struct mmc_host *host)
+{
+ return host->doing_retune == 1;
+}
+
static inline enum dma_data_direction mmc_get_dma_dir(struct mmc_data *data)
{
return data->flags & MMC_DATA_WRITE ? DMA_TO_DEVICE : DMA_FROM_DEVICE;
enum zone_type kcompactd_classzone_idx;
wait_queue_head_t kcompactd_wait;
struct task_struct *kcompactd;
-#endif
-#ifdef CONFIG_NUMA_BALANCING
- /* Lock serializing the migrate rate limiting window */
- spinlock_t numabalancing_migrate_lock;
#endif
/*
* This is a per-node reserve of pages that are not available
#include <linux/export.h>
#include <linux/rbtree_latch.h>
#include <linux/error-injection.h>
+#include <linux/tracepoint-defs.h>
#include <linux/percpu.h>
#include <asm/module.h>
#ifdef CONFIG_TRACEPOINTS
unsigned int num_tracepoints;
- struct tracepoint * const *tracepoints_ptrs;
+ tracepoint_ptr_t *tracepoints_ptrs;
#endif
#ifdef HAVE_JUMP_LABEL
struct jump_entry *jump_entries;
#include <linux/mutex.h>
#include <linux/kref.h>
#include <linux/sysfs.h>
-#include <linux/workqueue.h>
struct hd_geometry;
struct mtd_info;
struct kref ref;
struct gendisk *disk;
struct attribute_group *disk_attributes;
- struct workqueue_struct *wq;
- struct work_struct work;
struct request_queue *rq;
+ struct list_head rq_list;
+ struct blk_mq_tag_set *tag_set;
spinlock_t queue_lock;
void *priv;
fmode_t file_mode;
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright © 2000-2010 David Woodhouse <dwmw2@infradead.org>
+ * Steven J. Hill <sjhill@realitydiluted.com>
+ * Thomas Gleixner <tglx@linutronix.de>
+ *
+ * Contains all JEDEC related definitions
+ */
+
+#ifndef __LINUX_MTD_JEDEC_H
+#define __LINUX_MTD_JEDEC_H
+
+struct jedec_ecc_info {
+ u8 ecc_bits;
+ u8 codeword_size;
+ __le16 bb_per_lun;
+ __le16 block_endurance;
+ u8 reserved[2];
+} __packed;
+
+/* JEDEC features */
+#define JEDEC_FEATURE_16_BIT_BUS (1 << 0)
+
+struct nand_jedec_params {
+ /* rev info and features block */
+ /* 'J' 'E' 'S' 'D' */
+ u8 sig[4];
+ __le16 revision;
+ __le16 features;
+ u8 opt_cmd[3];
+ __le16 sec_cmd;
+ u8 num_of_param_pages;
+ u8 reserved0[18];
+
+ /* manufacturer information block */
+ char manufacturer[12];
+ char model[20];
+ u8 jedec_id[6];
+ u8 reserved1[10];
+
+ /* memory organization block */
+ __le32 byte_per_page;
+ __le16 spare_bytes_per_page;
+ u8 reserved2[6];
+ __le32 pages_per_block;
+ __le32 blocks_per_lun;
+ u8 lun_count;
+ u8 addr_cycles;
+ u8 bits_per_cell;
+ u8 programs_per_page;
+ u8 multi_plane_addr;
+ u8 multi_plane_op_attr;
+ u8 reserved3[38];
+
+ /* electrical parameter block */
+ __le16 async_sdr_speed_grade;
+ __le16 toggle_ddr_speed_grade;
+ __le16 sync_ddr_speed_grade;
+ u8 async_sdr_features;
+ u8 toggle_ddr_features;
+ u8 sync_ddr_features;
+ __le16 t_prog;
+ __le16 t_bers;
+ __le16 t_r;
+ __le16 t_r_multi_plane;
+ __le16 t_ccs;
+ __le16 io_pin_capacitance_typ;
+ __le16 input_pin_capacitance_typ;
+ __le16 clk_pin_capacitance_typ;
+ u8 driver_strength_support;
+ __le16 t_adl;
+ u8 reserved4[36];
+
+ /* ECC and endurance block */
+ u8 guaranteed_good_blocks;
+ __le16 guaranteed_block_endurance;
+ struct jedec_ecc_info ecc_info[4];
+ u8 reserved5[29];
+
+ /* reserved */
+ u8 reserved6[148];
+
+ /* vendor */
+ __le16 vendor_rev_num;
+ u8 reserved7[88];
+
+ /* CRC for Parameter Page */
+ __le16 crc;
+} __packed;
+
+#endif /* __LINUX_MTD_JEDEC_H */
#define __MTD_NAND_BCH_H__
struct mtd_info;
+struct nand_chip;
struct nand_bch_control;
#if defined(CONFIG_MTD_NAND_ECC_BCH)
/*
* Calculate BCH ecc code
*/
-int nand_bch_calculate_ecc(struct mtd_info *mtd, const u_char *dat,
+int nand_bch_calculate_ecc(struct nand_chip *chip, const u_char *dat,
u_char *ecc_code);
/*
* Detect and correct bit errors
*/
-int nand_bch_correct_data(struct mtd_info *mtd, u_char *dat, u_char *read_ecc,
- u_char *calc_ecc);
+int nand_bch_correct_data(struct nand_chip *chip, u_char *dat,
+ u_char *read_ecc, u_char *calc_ecc);
/*
* Initialize BCH encoder/decoder
*/
static inline int mtd_nand_has_bch(void) { return 0; }
static inline int
-nand_bch_calculate_ecc(struct mtd_info *mtd, const u_char *dat,
+nand_bch_calculate_ecc(struct nand_chip *chip, const u_char *dat,
u_char *ecc_code)
{
return -1;
}
static inline int
-nand_bch_correct_data(struct mtd_info *mtd, unsigned char *buf,
+nand_bch_correct_data(struct nand_chip *chip, unsigned char *buf,
unsigned char *read_ecc, unsigned char *calc_ecc)
{
return -ENOTSUPP;
#ifndef __MTD_NAND_ECC_H__
#define __MTD_NAND_ECC_H__
-struct mtd_info;
+struct nand_chip;
/*
* Calculate 3 byte ECC code for eccsize byte block
*/
void __nand_calculate_ecc(const u_char *dat, unsigned int eccsize,
- u_char *ecc_code);
+ u_char *ecc_code, bool sm_order);
/*
* Calculate 3 byte ECC code for 256/512 byte block
*/
-int nand_calculate_ecc(struct mtd_info *mtd, const u_char *dat, u_char *ecc_code);
+int nand_calculate_ecc(struct nand_chip *chip, const u_char *dat,
+ u_char *ecc_code);
/*
* Detect and correct a 1 bit error for eccsize byte block
*/
int __nand_correct_data(u_char *dat, u_char *read_ecc, u_char *calc_ecc,
- unsigned int eccsize);
+ unsigned int eccsize, bool sm_order);
/*
* Detect and correct a 1 bit error for 256/512 byte block
*/
-int nand_correct_data(struct mtd_info *mtd, u_char *dat, u_char *read_ecc, u_char *calc_ecc);
+int nand_correct_data(struct nand_chip *chip, u_char *dat, u_char *read_ecc,
+ u_char *calc_ecc);
#endif /* __MTD_NAND_ECC_H__ */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright © 2000-2010 David Woodhouse <dwmw2@infradead.org>
+ * Steven J. Hill <sjhill@realitydiluted.com>
+ * Thomas Gleixner <tglx@linutronix.de>
+ *
+ * Contains all ONFI related definitions
+ */
+
+#ifndef __LINUX_MTD_ONFI_H
+#define __LINUX_MTD_ONFI_H
+
+#include <linux/types.h>
+
+/* ONFI version bits */
+#define ONFI_VERSION_1_0 BIT(1)
+#define ONFI_VERSION_2_0 BIT(2)
+#define ONFI_VERSION_2_1 BIT(3)
+#define ONFI_VERSION_2_2 BIT(4)
+#define ONFI_VERSION_2_3 BIT(5)
+#define ONFI_VERSION_3_0 BIT(6)
+#define ONFI_VERSION_3_1 BIT(7)
+#define ONFI_VERSION_3_2 BIT(8)
+#define ONFI_VERSION_4_0 BIT(9)
+
+/* ONFI features */
+#define ONFI_FEATURE_16_BIT_BUS (1 << 0)
+#define ONFI_FEATURE_EXT_PARAM_PAGE (1 << 7)
+
+/* ONFI timing mode, used in both asynchronous and synchronous mode */
+#define ONFI_TIMING_MODE_0 (1 << 0)
+#define ONFI_TIMING_MODE_1 (1 << 1)
+#define ONFI_TIMING_MODE_2 (1 << 2)
+#define ONFI_TIMING_MODE_3 (1 << 3)
+#define ONFI_TIMING_MODE_4 (1 << 4)
+#define ONFI_TIMING_MODE_5 (1 << 5)
+#define ONFI_TIMING_MODE_UNKNOWN (1 << 6)
+
+/* ONFI feature number/address */
+#define ONFI_FEATURE_NUMBER 256
+#define ONFI_FEATURE_ADDR_TIMING_MODE 0x1
+
+/* Vendor-specific feature address (Micron) */
+#define ONFI_FEATURE_ADDR_READ_RETRY 0x89
+#define ONFI_FEATURE_ON_DIE_ECC 0x90
+#define ONFI_FEATURE_ON_DIE_ECC_EN BIT(3)
+
+/* ONFI subfeature parameters length */
+#define ONFI_SUBFEATURE_PARAM_LEN 4
+
+/* ONFI optional commands SET/GET FEATURES supported? */
+#define ONFI_OPT_CMD_SET_GET_FEATURES (1 << 2)
+
+struct nand_onfi_params {
+ /* rev info and features block */
+ /* 'O' 'N' 'F' 'I' */
+ u8 sig[4];
+ __le16 revision;
+ __le16 features;
+ __le16 opt_cmd;
+ u8 reserved0[2];
+ __le16 ext_param_page_length; /* since ONFI 2.1 */
+ u8 num_of_param_pages; /* since ONFI 2.1 */
+ u8 reserved1[17];
+
+ /* manufacturer information block */
+ char manufacturer[12];
+ char model[20];
+ u8 jedec_id;
+ __le16 date_code;
+ u8 reserved2[13];
+
+ /* memory organization block */
+ __le32 byte_per_page;
+ __le16 spare_bytes_per_page;
+ __le32 data_bytes_per_ppage;
+ __le16 spare_bytes_per_ppage;
+ __le32 pages_per_block;
+ __le32 blocks_per_lun;
+ u8 lun_count;
+ u8 addr_cycles;
+ u8 bits_per_cell;
+ __le16 bb_per_lun;
+ __le16 block_endurance;
+ u8 guaranteed_good_blocks;
+ __le16 guaranteed_block_endurance;
+ u8 programs_per_page;
+ u8 ppage_attr;
+ u8 ecc_bits;
+ u8 interleaved_bits;
+ u8 interleaved_ops;
+ u8 reserved3[13];
+
+ /* electrical parameter block */
+ u8 io_pin_capacitance_max;
+ __le16 async_timing_mode;
+ __le16 program_cache_timing_mode;
+ __le16 t_prog;
+ __le16 t_bers;
+ __le16 t_r;
+ __le16 t_ccs;
+ __le16 src_sync_timing_mode;
+ u8 src_ssync_features;
+ __le16 clk_pin_capacitance_typ;
+ __le16 io_pin_capacitance_typ;
+ __le16 input_pin_capacitance_typ;
+ u8 input_pin_capacitance_max;
+ u8 driver_strength_support;
+ __le16 t_int_r;
+ __le16 t_adl;
+ u8 reserved4[8];
+
+ /* vendor */
+ __le16 vendor_revision;
+ u8 vendor[88];
+
+ __le16 crc;
+} __packed;
+
+#define ONFI_CRC_BASE 0x4F4E
+
+/* Extended ECC information Block Definition (since ONFI 2.1) */
+struct onfi_ext_ecc_info {
+ u8 ecc_bits;
+ u8 codeword_size;
+ __le16 bb_per_lun;
+ __le16 block_endurance;
+ u8 reserved[2];
+} __packed;
+
+#define ONFI_SECTION_TYPE_0 0 /* Unused section. */
+#define ONFI_SECTION_TYPE_1 1 /* for additional sections. */
+#define ONFI_SECTION_TYPE_2 2 /* for ECC information. */
+struct onfi_ext_section {
+ u8 type;
+ u8 length;
+} __packed;
+
+#define ONFI_EXT_SECTION_MAX 8
+
+/* Extended Parameter Page Definition (since ONFI 2.1) */
+struct onfi_ext_param_page {
+ __le16 crc;
+ u8 sig[4]; /* 'E' 'P' 'P' 'S' */
+ u8 reserved0[10];
+ struct onfi_ext_section sections[ONFI_EXT_SECTION_MAX];
+
+ /*
+ * The actual size of the Extended Parameter Page is in
+ * @ext_param_page_length of nand_onfi_params{}.
+ * The following are the variable length sections.
+ * So we do not add any fields below. Please see the ONFI spec.
+ */
+} __packed;
+
+/**
+ * struct onfi_params - ONFI specific parameters that will be reused
+ * @version: ONFI version (BCD encoded), 0 if ONFI is not supported
+ * @tPROG: Page program time
+ * @tBERS: Block erase time
+ * @tR: Page read time
+ * @tCCS: Change column setup time
+ * @async_timing_mode: Supported asynchronous timing mode
+ * @vendor_revision: Vendor specific revision number
+ * @vendor: Vendor specific data
+ */
+struct onfi_params {
+ int version;
+ u16 tPROG;
+ u16 tBERS;
+ u16 tR;
+ u16 tCCS;
+ u16 async_timing_mode;
+ u16 vendor_revision;
+ u8 vendor[88];
+};
+
+#endif /* __LINUX_MTD_ONFI_H */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright © 2000-2010 David Woodhouse <dwmw2@infradead.org>
+ * Steven J. Hill <sjhill@realitydiluted.com>
+ * Thomas Gleixner <tglx@linutronix.de>
+ *
+ * Contains all platform NAND related definitions.
+ */
+
+#ifndef __LINUX_MTD_PLATNAND_H
+#define __LINUX_MTD_PLATNAND_H
+
+#include <linux/mtd/partitions.h>
+#include <linux/mtd/rawnand.h>
+#include <linux/platform_device.h>
+
+/**
+ * struct platform_nand_chip - chip level device structure
+ * @nr_chips: max. number of chips to scan for
+ * @chip_offset: chip number offset
+ * @nr_partitions: number of partitions pointed to by partitions (or zero)
+ * @partitions: mtd partition list
+ * @chip_delay: R/B delay value in us
+ * @options: Option flags, e.g. 16bit buswidth
+ * @bbt_options: BBT option flags, e.g. NAND_BBT_USE_FLASH
+ * @part_probe_types: NULL-terminated array of probe types
+ */
+struct platform_nand_chip {
+ int nr_chips;
+ int chip_offset;
+ int nr_partitions;
+ struct mtd_partition *partitions;
+ int chip_delay;
+ unsigned int options;
+ unsigned int bbt_options;
+ const char **part_probe_types;
+};
+
+/**
+ * struct platform_nand_ctrl - controller level device structure
+ * @probe: platform specific function to probe/setup hardware
+ * @remove: platform specific function to remove/teardown hardware
+ * @dev_ready: platform specific function to read ready/busy pin
+ * @select_chip: platform specific chip select function
+ * @cmd_ctrl: platform specific function for controlling
+ * ALE/CLE/nCE. Also used to write command and address
+ * @write_buf: platform specific function for write buffer
+ * @read_buf: platform specific function for read buffer
+ * @priv: private data to transport driver specific settings
+ *
+ * All fields are optional and depend on the hardware driver requirements
+ */
+struct platform_nand_ctrl {
+ int (*probe)(struct platform_device *pdev);
+ void (*remove)(struct platform_device *pdev);
+ int (*dev_ready)(struct nand_chip *chip);
+ void (*select_chip)(struct nand_chip *chip, int cs);
+ void (*cmd_ctrl)(struct nand_chip *chip, int dat, unsigned int ctrl);
+ void (*write_buf)(struct nand_chip *chip, const uint8_t *buf, int len);
+ void (*read_buf)(struct nand_chip *chip, uint8_t *buf, int len);
+ void *priv;
+};
+
+/**
+ * struct platform_nand_data - container structure for platform-specific data
+ * @chip: chip level chip structure
+ * @ctrl: controller level device structure
+ */
+struct platform_nand_data {
+ struct platform_nand_chip chip;
+ struct platform_nand_ctrl ctrl;
+};
+
+#endif /* __LINUX_MTD_PLATNAND_H */
#include <linux/mtd/mtd.h>
#include <linux/mtd/flashchip.h>
#include <linux/mtd/bbm.h>
+#include <linux/mtd/jedec.h>
+#include <linux/mtd/onfi.h>
#include <linux/of.h>
#include <linux/types.h>
-struct nand_flash_dev;
-
-/* Scan and identify a NAND device */
-int nand_scan_with_ids(struct mtd_info *mtd, int max_chips,
- struct nand_flash_dev *ids);
-
-static inline int nand_scan(struct mtd_info *mtd, int max_chips)
-{
- return nand_scan_with_ids(mtd, max_chips, NULL);
-}
-
-/* Internal helper for board drivers which need to override command function */
-void nand_wait_ready(struct mtd_info *mtd);
+struct nand_chip;
/* The maximum number of NAND chips in an array */
#define NAND_MAX_CHIPS 8
#define NAND_ECC_GENERIC_ERASED_CHECK BIT(0)
#define NAND_ECC_MAXIMIZE BIT(1)
-/* Bit mask for flags passed to do_nand_read_ecc */
-#define NAND_GET_DEVICE 0x80
-
+/*
+ * When using software implementation of Hamming, we can specify which byte
+ * ordering should be used.
+ */
+#define NAND_ECC_SOFT_HAMMING_SM_ORDER BIT(2)
/*
* Option constants for bizarre disfunctionality and real
#define NAND_SAMSUNG_LP_OPTIONS NAND_CACHEPRG
/* Macros to identify the above */
-#define NAND_HAS_CACHEPROG(chip) ((chip->options & NAND_CACHEPRG))
#define NAND_HAS_SUBPAGE_READ(chip) ((chip->options & NAND_SUBPAGE_READ))
-#define NAND_HAS_SUBPAGE_WRITE(chip) !((chip)->options & NAND_NO_SUBPAGE_WRITE)
/* Non chip related options */
/* This option skips the bbt scan during initialization. */
#define NAND_USE_BOUNCE_BUFFER 0x00100000
/*
- * In case your controller is implementing ->cmd_ctrl() and is relying on the
- * default ->cmdfunc() implementation, you may want to let the core handle the
- * tCCS delay which is required when a column change (RNDIN or RNDOUT) is
- * requested.
+ * In case your controller is implementing ->legacy.cmd_ctrl() and is relying
+ * on the default ->cmdfunc() implementation, you may want to let the core
+ * handle the tCCS delay which is required when a column change (RNDIN or
+ * RNDOUT) is requested.
* If your controller already takes care of this delay, you don't need to set
* this flag.
*/
#define NAND_CI_CELLTYPE_MSK 0x0C
#define NAND_CI_CELLTYPE_SHIFT 2
-/* Keep gcc happy */
-struct nand_chip;
-
-/* ONFI version bits */
-#define ONFI_VERSION_1_0 BIT(1)
-#define ONFI_VERSION_2_0 BIT(2)
-#define ONFI_VERSION_2_1 BIT(3)
-#define ONFI_VERSION_2_2 BIT(4)
-#define ONFI_VERSION_2_3 BIT(5)
-#define ONFI_VERSION_3_0 BIT(6)
-#define ONFI_VERSION_3_1 BIT(7)
-#define ONFI_VERSION_3_2 BIT(8)
-#define ONFI_VERSION_4_0 BIT(9)
-
-/* ONFI features */
-#define ONFI_FEATURE_16_BIT_BUS (1 << 0)
-#define ONFI_FEATURE_EXT_PARAM_PAGE (1 << 7)
-
-/* ONFI timing mode, used in both asynchronous and synchronous mode */
-#define ONFI_TIMING_MODE_0 (1 << 0)
-#define ONFI_TIMING_MODE_1 (1 << 1)
-#define ONFI_TIMING_MODE_2 (1 << 2)
-#define ONFI_TIMING_MODE_3 (1 << 3)
-#define ONFI_TIMING_MODE_4 (1 << 4)
-#define ONFI_TIMING_MODE_5 (1 << 5)
-#define ONFI_TIMING_MODE_UNKNOWN (1 << 6)
-
-/* ONFI feature number/address */
-#define ONFI_FEATURE_NUMBER 256
-#define ONFI_FEATURE_ADDR_TIMING_MODE 0x1
-
-/* Vendor-specific feature address (Micron) */
-#define ONFI_FEATURE_ADDR_READ_RETRY 0x89
-#define ONFI_FEATURE_ON_DIE_ECC 0x90
-#define ONFI_FEATURE_ON_DIE_ECC_EN BIT(3)
-
-/* ONFI subfeature parameters length */
-#define ONFI_SUBFEATURE_PARAM_LEN 4
-
-/* ONFI optional commands SET/GET FEATURES supported? */
-#define ONFI_OPT_CMD_SET_GET_FEATURES (1 << 2)
-
-struct nand_onfi_params {
- /* rev info and features block */
- /* 'O' 'N' 'F' 'I' */
- u8 sig[4];
- __le16 revision;
- __le16 features;
- __le16 opt_cmd;
- u8 reserved0[2];
- __le16 ext_param_page_length; /* since ONFI 2.1 */
- u8 num_of_param_pages; /* since ONFI 2.1 */
- u8 reserved1[17];
-
- /* manufacturer information block */
- char manufacturer[12];
- char model[20];
- u8 jedec_id;
- __le16 date_code;
- u8 reserved2[13];
-
- /* memory organization block */
- __le32 byte_per_page;
- __le16 spare_bytes_per_page;
- __le32 data_bytes_per_ppage;
- __le16 spare_bytes_per_ppage;
- __le32 pages_per_block;
- __le32 blocks_per_lun;
- u8 lun_count;
- u8 addr_cycles;
- u8 bits_per_cell;
- __le16 bb_per_lun;
- __le16 block_endurance;
- u8 guaranteed_good_blocks;
- __le16 guaranteed_block_endurance;
- u8 programs_per_page;
- u8 ppage_attr;
- u8 ecc_bits;
- u8 interleaved_bits;
- u8 interleaved_ops;
- u8 reserved3[13];
-
- /* electrical parameter block */
- u8 io_pin_capacitance_max;
- __le16 async_timing_mode;
- __le16 program_cache_timing_mode;
- __le16 t_prog;
- __le16 t_bers;
- __le16 t_r;
- __le16 t_ccs;
- __le16 src_sync_timing_mode;
- u8 src_ssync_features;
- __le16 clk_pin_capacitance_typ;
- __le16 io_pin_capacitance_typ;
- __le16 input_pin_capacitance_typ;
- u8 input_pin_capacitance_max;
- u8 driver_strength_support;
- __le16 t_int_r;
- __le16 t_adl;
- u8 reserved4[8];
-
- /* vendor */
- __le16 vendor_revision;
- u8 vendor[88];
-
- __le16 crc;
-} __packed;
-
-#define ONFI_CRC_BASE 0x4F4E
-
-/* Extended ECC information Block Definition (since ONFI 2.1) */
-struct onfi_ext_ecc_info {
- u8 ecc_bits;
- u8 codeword_size;
- __le16 bb_per_lun;
- __le16 block_endurance;
- u8 reserved[2];
-} __packed;
-
-#define ONFI_SECTION_TYPE_0 0 /* Unused section. */
-#define ONFI_SECTION_TYPE_1 1 /* for additional sections. */
-#define ONFI_SECTION_TYPE_2 2 /* for ECC information. */
-struct onfi_ext_section {
- u8 type;
- u8 length;
-} __packed;
-
-#define ONFI_EXT_SECTION_MAX 8
-
-/* Extended Parameter Page Definition (since ONFI 2.1) */
-struct onfi_ext_param_page {
- __le16 crc;
- u8 sig[4]; /* 'E' 'P' 'P' 'S' */
- u8 reserved0[10];
- struct onfi_ext_section sections[ONFI_EXT_SECTION_MAX];
-
- /*
- * The actual size of the Extended Parameter Page is in
- * @ext_param_page_length of nand_onfi_params{}.
- * The following are the variable length sections.
- * So we do not add any fields below. Please see the ONFI spec.
- */
-} __packed;
-
-struct jedec_ecc_info {
- u8 ecc_bits;
- u8 codeword_size;
- __le16 bb_per_lun;
- __le16 block_endurance;
- u8 reserved[2];
-} __packed;
-
-/* JEDEC features */
-#define JEDEC_FEATURE_16_BIT_BUS (1 << 0)
-
-struct nand_jedec_params {
- /* rev info and features block */
- /* 'J' 'E' 'S' 'D' */
- u8 sig[4];
- __le16 revision;
- __le16 features;
- u8 opt_cmd[3];
- __le16 sec_cmd;
- u8 num_of_param_pages;
- u8 reserved0[18];
-
- /* manufacturer information block */
- char manufacturer[12];
- char model[20];
- u8 jedec_id[6];
- u8 reserved1[10];
-
- /* memory organization block */
- __le32 byte_per_page;
- __le16 spare_bytes_per_page;
- u8 reserved2[6];
- __le32 pages_per_block;
- __le32 blocks_per_lun;
- u8 lun_count;
- u8 addr_cycles;
- u8 bits_per_cell;
- u8 programs_per_page;
- u8 multi_plane_addr;
- u8 multi_plane_op_attr;
- u8 reserved3[38];
-
- /* electrical parameter block */
- __le16 async_sdr_speed_grade;
- __le16 toggle_ddr_speed_grade;
- __le16 sync_ddr_speed_grade;
- u8 async_sdr_features;
- u8 toggle_ddr_features;
- u8 sync_ddr_features;
- __le16 t_prog;
- __le16 t_bers;
- __le16 t_r;
- __le16 t_r_multi_plane;
- __le16 t_ccs;
- __le16 io_pin_capacitance_typ;
- __le16 input_pin_capacitance_typ;
- __le16 clk_pin_capacitance_typ;
- u8 driver_strength_support;
- __le16 t_adl;
- u8 reserved4[36];
-
- /* ECC and endurance block */
- u8 guaranteed_good_blocks;
- __le16 guaranteed_block_endurance;
- struct jedec_ecc_info ecc_info[4];
- u8 reserved5[29];
-
- /* reserved */
- u8 reserved6[148];
-
- /* vendor */
- __le16 vendor_rev_num;
- u8 reserved7[88];
-
- /* CRC for Parameter Page */
- __le16 crc;
-} __packed;
-
-/**
- * struct onfi_params - ONFI specific parameters that will be reused
- * @version: ONFI version (BCD encoded), 0 if ONFI is not supported
- * @tPROG: Page program time
- * @tBERS: Block erase time
- * @tR: Page read time
- * @tCCS: Change column setup time
- * @async_timing_mode: Supported asynchronous timing mode
- * @vendor_revision: Vendor specific revision number
- * @vendor: Vendor specific data
- */
-struct onfi_params {
- int version;
- u16 tPROG;
- u16 tBERS;
- u16 tR;
- u16 tCCS;
- u16 async_timing_mode;
- u16 vendor_revision;
- u8 vendor[88];
-};
-
/**
* struct nand_parameters - NAND generic parameters from the parameter page
* @model: Model name
void *priv;
u8 *calc_buf;
u8 *code_buf;
- void (*hwctl)(struct mtd_info *mtd, int mode);
- int (*calculate)(struct mtd_info *mtd, const uint8_t *dat,
- uint8_t *ecc_code);
- int (*correct)(struct mtd_info *mtd, uint8_t *dat, uint8_t *read_ecc,
- uint8_t *calc_ecc);
- int (*read_page_raw)(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page);
- int (*write_page_raw)(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page);
- int (*read_page)(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page);
- int (*read_subpage)(struct mtd_info *mtd, struct nand_chip *chip,
- uint32_t offs, uint32_t len, uint8_t *buf, int page);
- int (*write_subpage)(struct mtd_info *mtd, struct nand_chip *chip,
- uint32_t offset, uint32_t data_len,
- const uint8_t *data_buf, int oob_required, int page);
- int (*write_page)(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page);
- int (*write_oob_raw)(struct mtd_info *mtd, struct nand_chip *chip,
- int page);
- int (*read_oob_raw)(struct mtd_info *mtd, struct nand_chip *chip,
- int page);
- int (*read_oob)(struct mtd_info *mtd, struct nand_chip *chip, int page);
- int (*write_oob)(struct mtd_info *mtd, struct nand_chip *chip,
- int page);
+ void (*hwctl)(struct nand_chip *chip, int mode);
+ int (*calculate)(struct nand_chip *chip, const uint8_t *dat,
+ uint8_t *ecc_code);
+ int (*correct)(struct nand_chip *chip, uint8_t *dat, uint8_t *read_ecc,
+ uint8_t *calc_ecc);
+ int (*read_page_raw)(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page);
+ int (*write_page_raw)(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page);
+ int (*read_page)(struct nand_chip *chip, uint8_t *buf,
+ int oob_required, int page);
+ int (*read_subpage)(struct nand_chip *chip, uint32_t offs,
+ uint32_t len, uint8_t *buf, int page);
+ int (*write_subpage)(struct nand_chip *chip, uint32_t offset,
+ uint32_t data_len, const uint8_t *data_buf,
+ int oob_required, int page);
+ int (*write_page)(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page);
+ int (*write_oob_raw)(struct nand_chip *chip, int page);
+ int (*read_oob_raw)(struct nand_chip *chip, int page);
+ int (*read_oob)(struct nand_chip *chip, int page);
+ int (*write_oob)(struct nand_chip *chip, int page);
};
/**
return &conf->timings.sdr;
}
-/**
- * struct nand_manufacturer_ops - NAND Manufacturer operations
- * @detect: detect the NAND memory organization and capabilities
- * @init: initialize all vendor specific fields (like the ->read_retry()
- * implementation) if any.
- * @cleanup: the ->init() function may have allocated resources, ->cleanup()
- * is here to let vendor specific code release those resources.
- * @fixup_onfi_param_page: apply vendor specific fixups to the ONFI parameter
- * page. This is called after the checksum is verified.
- */
-struct nand_manufacturer_ops {
- void (*detect)(struct nand_chip *chip);
- int (*init)(struct nand_chip *chip);
- void (*cleanup)(struct nand_chip *chip);
- void (*fixup_onfi_param_page)(struct nand_chip *chip,
- struct nand_onfi_params *p);
-};
-
/**
* struct nand_op_cmd_instr - Definition of a command instruction
* @opcode: the command to issue in one cycle
const struct nand_op_parser *parser,
const struct nand_operation *op, bool check_only);
+/**
+ * struct nand_legacy - NAND chip legacy fields/hooks
+ * @IO_ADDR_R: address to read the 8 I/O lines of the flash device
+ * @IO_ADDR_W: address to write the 8 I/O lines of the flash device
+ * @read_byte: read one byte from the chip
+ * @write_byte: write a single byte to the chip on the low 8 I/O lines
+ * @write_buf: write data from the buffer to the chip
+ * @read_buf: read data from the chip into the buffer
+ * @cmd_ctrl: hardware specific function for controlling ALE/CLE/nCE. Also used
+ * to write command and address
+ * @cmdfunc: hardware specific function for writing commands to the chip.
+ * @dev_ready: hardware specific function for accessing device ready/busy line.
+ * If set to NULL no access to ready/busy is available and the
+ * ready/busy information is read from the chip status register.
+ * @waitfunc: hardware specific function for wait on ready.
+ * @block_bad: check if a block is bad, using OOB markers
+ * @block_markbad: mark a block bad
+ * @erase: erase function
+ * @set_features: set the NAND chip features
+ * @get_features: get the NAND chip features
+ * @chip_delay: chip dependent delay for transferring data from array to read
+ * regs (tR).
+ *
+ * If you look at this structure you're already wrong. These fields/hooks are
+ * all deprecated.
+ */
+struct nand_legacy {
+ void __iomem *IO_ADDR_R;
+ void __iomem *IO_ADDR_W;
+ u8 (*read_byte)(struct nand_chip *chip);
+ void (*write_byte)(struct nand_chip *chip, u8 byte);
+ void (*write_buf)(struct nand_chip *chip, const u8 *buf, int len);
+ void (*read_buf)(struct nand_chip *chip, u8 *buf, int len);
+ void (*cmd_ctrl)(struct nand_chip *chip, int dat, unsigned int ctrl);
+ void (*cmdfunc)(struct nand_chip *chip, unsigned command, int column,
+ int page_addr);
+ int (*dev_ready)(struct nand_chip *chip);
+ int (*waitfunc)(struct nand_chip *chip);
+ int (*block_bad)(struct nand_chip *chip, loff_t ofs);
+ int (*block_markbad)(struct nand_chip *chip, loff_t ofs);
+ int (*erase)(struct nand_chip *chip, int page);
+ int (*set_features)(struct nand_chip *chip, int feature_addr,
+ u8 *subfeature_para);
+ int (*get_features)(struct nand_chip *chip, int feature_addr,
+ u8 *subfeature_para);
+ int chip_delay;
+};
+
/**
* struct nand_chip - NAND Private Flash Chip Data
* @mtd: MTD device registered to the MTD framework
- * @IO_ADDR_R: [BOARDSPECIFIC] address to read the 8 I/O lines of the
- * flash device
- * @IO_ADDR_W: [BOARDSPECIFIC] address to write the 8 I/O lines of the
- * flash device.
- * @read_byte: [REPLACEABLE] read one byte from the chip
- * @read_word: [REPLACEABLE] read one word from the chip
- * @write_byte: [REPLACEABLE] write a single byte to the chip on the
- * low 8 I/O lines
- * @write_buf: [REPLACEABLE] write data from the buffer to the chip
- * @read_buf: [REPLACEABLE] read data from the chip into the buffer
+ * @legacy: All legacy fields/hooks. If you develop a new driver,
+ * don't even try to use any of these fields/hooks, and if
+ * you're modifying an existing driver that is using those
+ * fields/hooks, you should consider reworking the driver
+ * avoid using them.
* @select_chip: [REPLACEABLE] select chip nr
- * @block_bad: [REPLACEABLE] check if a block is bad, using OOB markers
- * @block_markbad: [REPLACEABLE] mark a block bad
- * @cmd_ctrl: [BOARDSPECIFIC] hardwarespecific function for controlling
- * ALE/CLE/nCE. Also used to write command and address
- * @dev_ready: [BOARDSPECIFIC] hardwarespecific function for accessing
- * device ready/busy line. If set to NULL no access to
- * ready/busy is available and the ready/busy information
- * is read from the chip status register.
- * @cmdfunc: [REPLACEABLE] hardwarespecific function for writing
- * commands to the chip.
- * @waitfunc: [REPLACEABLE] hardwarespecific function for wait on
- * ready.
* @exec_op: controller specific method to execute NAND operations.
* This method replaces ->cmdfunc(),
- * ->{read,write}_{buf,byte,word}(), ->dev_ready() and
- * ->waifunc().
+ * ->legacy.{read,write}_{buf,byte,word}(),
+ * ->legacy.dev_ready() and ->waifunc().
* @setup_read_retry: [FLASHSPECIFIC] flash (vendor) specific function for
* setting the read-retry mode. Mostly needed for MLC NAND.
* @ecc: [BOARDSPECIFIC] ECC control structure
* @buf_align: minimum buffer alignment required by a platform
* @dummy_controller: dummy controller implementation for drivers that can
* only control a single chip
- * @erase: [REPLACEABLE] erase function
- * @chip_delay: [BOARDSPECIFIC] chip dependent delay for transferring
- * data from array to read regs (tR).
* @state: [INTERN] the current state of the NAND device
* @oob_poi: "poison value buffer," used for laying out OOB data
* before writing
* @blocks_per_die: [INTERN] The number of PEBs in a die
* @data_interface: [INTERN] NAND interface timing information
* @read_retries: [INTERN] the number of read retry modes supported
- * @set_features: [REPLACEABLE] set the NAND chip features
- * @get_features: [REPLACEABLE] get the NAND chip features
* @setup_data_interface: [OPTIONAL] setup the data interface and timing. If
* chipnr is set to %NAND_DATA_IFACE_CHECK_ONLY this
* means the configuration should not be applied but
struct nand_chip {
struct mtd_info mtd;
- void __iomem *IO_ADDR_R;
- void __iomem *IO_ADDR_W;
- uint8_t (*read_byte)(struct mtd_info *mtd);
- u16 (*read_word)(struct mtd_info *mtd);
- void (*write_byte)(struct mtd_info *mtd, uint8_t byte);
- void (*write_buf)(struct mtd_info *mtd, const uint8_t *buf, int len);
- void (*read_buf)(struct mtd_info *mtd, uint8_t *buf, int len);
- void (*select_chip)(struct mtd_info *mtd, int chip);
- int (*block_bad)(struct mtd_info *mtd, loff_t ofs);
- int (*block_markbad)(struct mtd_info *mtd, loff_t ofs);
- void (*cmd_ctrl)(struct mtd_info *mtd, int dat, unsigned int ctrl);
- int (*dev_ready)(struct mtd_info *mtd);
- void (*cmdfunc)(struct mtd_info *mtd, unsigned command, int column,
- int page_addr);
- int(*waitfunc)(struct mtd_info *mtd, struct nand_chip *this);
+ struct nand_legacy legacy;
+
+ void (*select_chip)(struct nand_chip *chip, int cs);
int (*exec_op)(struct nand_chip *chip,
const struct nand_operation *op,
bool check_only);
- int (*erase)(struct mtd_info *mtd, int page);
- int (*set_features)(struct mtd_info *mtd, struct nand_chip *chip,
- int feature_addr, uint8_t *subfeature_para);
- int (*get_features)(struct mtd_info *mtd, struct nand_chip *chip,
- int feature_addr, uint8_t *subfeature_para);
- int (*setup_read_retry)(struct mtd_info *mtd, int retry_mode);
- int (*setup_data_interface)(struct mtd_info *mtd, int chipnr,
+ int (*setup_read_retry)(struct nand_chip *chip, int retry_mode);
+ int (*setup_data_interface)(struct nand_chip *chip, int chipnr,
const struct nand_data_interface *conf);
- int chip_delay;
unsigned int options;
unsigned int bbt_options;
return chip->manufacturer.priv;
}
-/*
- * NAND Flash Manufacturer ID Codes
- */
-#define NAND_MFR_TOSHIBA 0x98
-#define NAND_MFR_ESMT 0xc8
-#define NAND_MFR_SAMSUNG 0xec
-#define NAND_MFR_FUJITSU 0x04
-#define NAND_MFR_NATIONAL 0x8f
-#define NAND_MFR_RENESAS 0x07
-#define NAND_MFR_STMICRO 0x20
-#define NAND_MFR_HYNIX 0xad
-#define NAND_MFR_MICRON 0x2c
-#define NAND_MFR_AMD 0x01
-#define NAND_MFR_MACRONIX 0xc2
-#define NAND_MFR_EON 0x92
-#define NAND_MFR_SANDISK 0x45
-#define NAND_MFR_INTEL 0x89
-#define NAND_MFR_ATO 0x9b
-#define NAND_MFR_WINBOND 0xef
-
-
/*
* A helper for defining older NAND chips where the second ID byte fully
* defined the chip, including the geometry (chip size, eraseblock size, page
int onfi_timing_mode_default;
};
-/**
- * struct nand_manufacturer - NAND Flash Manufacturer structure
- * @name: Manufacturer name
- * @id: manufacturer ID code of device.
- * @ops: manufacturer operations
-*/
-struct nand_manufacturer {
- int id;
- char *name;
- const struct nand_manufacturer_ops *ops;
-};
-
-const struct nand_manufacturer *nand_get_manufacturer(u8 id);
-
-static inline const char *
-nand_manufacturer_name(const struct nand_manufacturer *manufacturer)
-{
- return manufacturer ? manufacturer->name : "Unknown";
-}
-
-extern struct nand_flash_dev nand_flash_ids[];
-
-extern const struct nand_manufacturer_ops toshiba_nand_manuf_ops;
-extern const struct nand_manufacturer_ops samsung_nand_manuf_ops;
-extern const struct nand_manufacturer_ops hynix_nand_manuf_ops;
-extern const struct nand_manufacturer_ops micron_nand_manuf_ops;
-extern const struct nand_manufacturer_ops amd_nand_manuf_ops;
-extern const struct nand_manufacturer_ops macronix_nand_manuf_ops;
-
int nand_create_bbt(struct nand_chip *chip);
-int nand_markbad_bbt(struct mtd_info *mtd, loff_t offs);
-int nand_isreserved_bbt(struct mtd_info *mtd, loff_t offs);
-int nand_isbad_bbt(struct mtd_info *mtd, loff_t offs, int allowbbt);
-int nand_erase_nand(struct mtd_info *mtd, struct erase_info *instr,
- int allowbbt);
-
-/**
- * struct platform_nand_chip - chip level device structure
- * @nr_chips: max. number of chips to scan for
- * @chip_offset: chip number offset
- * @nr_partitions: number of partitions pointed to by partitions (or zero)
- * @partitions: mtd partition list
- * @chip_delay: R/B delay value in us
- * @options: Option flags, e.g. 16bit buswidth
- * @bbt_options: BBT option flags, e.g. NAND_BBT_USE_FLASH
- * @part_probe_types: NULL-terminated array of probe types
- */
-struct platform_nand_chip {
- int nr_chips;
- int chip_offset;
- int nr_partitions;
- struct mtd_partition *partitions;
- int chip_delay;
- unsigned int options;
- unsigned int bbt_options;
- const char **part_probe_types;
-};
-
-/* Keep gcc happy */
-struct platform_device;
-
-/**
- * struct platform_nand_ctrl - controller level device structure
- * @probe: platform specific function to probe/setup hardware
- * @remove: platform specific function to remove/teardown hardware
- * @dev_ready: platform specific function to read ready/busy pin
- * @select_chip: platform specific chip select function
- * @cmd_ctrl: platform specific function for controlling
- * ALE/CLE/nCE. Also used to write command and address
- * @write_buf: platform specific function for write buffer
- * @read_buf: platform specific function for read buffer
- * @priv: private data to transport driver specific settings
- *
- * All fields are optional and depend on the hardware driver requirements
- */
-struct platform_nand_ctrl {
- int (*probe)(struct platform_device *pdev);
- void (*remove)(struct platform_device *pdev);
- int (*dev_ready)(struct mtd_info *mtd);
- void (*select_chip)(struct mtd_info *mtd, int chip);
- void (*cmd_ctrl)(struct mtd_info *mtd, int dat, unsigned int ctrl);
- void (*write_buf)(struct mtd_info *mtd, const uint8_t *buf, int len);
- void (*read_buf)(struct mtd_info *mtd, uint8_t *buf, int len);
- void *priv;
-};
-
-/**
- * struct platform_nand_data - container structure for platform-specific data
- * @chip: chip level chip structure
- * @ctrl: controller level device structure
- */
-struct platform_nand_data {
- struct platform_nand_chip chip;
- struct platform_nand_ctrl ctrl;
-};
-
-/* return the supported asynchronous timing mode. */
-static inline int onfi_get_async_timing_mode(struct nand_chip *chip)
-{
- if (!chip->parameters.onfi)
- return ONFI_TIMING_MODE_UNKNOWN;
-
- return chip->parameters.onfi->async_timing_mode;
-}
-
-int onfi_fill_data_interface(struct nand_chip *chip,
- enum nand_data_interface_type type,
- int timing_mode);
/*
* Check if it is a SLC nand.
return 0;
}
-/* get timing characteristics from ONFI timing mode. */
-const struct nand_sdr_timings *onfi_async_timing_mode_to_sdr_timings(int mode);
-
int nand_check_erased_ecc_chunk(void *data, int datalen,
void *ecc, int ecclen,
void *extraoob, int extraooblen,
const struct nand_ecc_caps *caps, int oobavail);
/* Default write_oob implementation */
-int nand_write_oob_std(struct mtd_info *mtd, struct nand_chip *chip, int page);
-
-/* Default write_oob syndrome implementation */
-int nand_write_oob_syndrome(struct mtd_info *mtd, struct nand_chip *chip,
- int page);
+int nand_write_oob_std(struct nand_chip *chip, int page);
/* Default read_oob implementation */
-int nand_read_oob_std(struct mtd_info *mtd, struct nand_chip *chip, int page);
+int nand_read_oob_std(struct nand_chip *chip, int page);
-/* Default read_oob syndrome implementation */
-int nand_read_oob_syndrome(struct mtd_info *mtd, struct nand_chip *chip,
- int page);
-
-/* Wrapper to use in order for controllers/vendors to GET/SET FEATURES */
-int nand_get_features(struct nand_chip *chip, int addr, u8 *subfeature_param);
-int nand_set_features(struct nand_chip *chip, int addr, u8 *subfeature_param);
/* Stub used by drivers that do not support GET/SET FEATURES operations */
-int nand_get_set_features_notsupp(struct mtd_info *mtd, struct nand_chip *chip,
- int addr, u8 *subfeature_param);
+int nand_get_set_features_notsupp(struct nand_chip *chip, int addr,
+ u8 *subfeature_param);
/* Default read_page_raw implementation */
-int nand_read_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- uint8_t *buf, int oob_required, int page);
-int nand_read_page_raw_notsupp(struct mtd_info *mtd, struct nand_chip *chip,
- u8 *buf, int oob_required, int page);
+int nand_read_page_raw(struct nand_chip *chip, uint8_t *buf, int oob_required,
+ int page);
/* Default write_page_raw implementation */
-int nand_write_page_raw(struct mtd_info *mtd, struct nand_chip *chip,
- const uint8_t *buf, int oob_required, int page);
-int nand_write_page_raw_notsupp(struct mtd_info *mtd, struct nand_chip *chip,
- const u8 *buf, int oob_required, int page);
+int nand_write_page_raw(struct nand_chip *chip, const uint8_t *buf,
+ int oob_required, int page);
/* Reset and initialize a NAND device */
int nand_reset(struct nand_chip *chip, int chipnr);
int nand_readid_op(struct nand_chip *chip, u8 addr, void *buf,
unsigned int len);
int nand_status_op(struct nand_chip *chip, u8 *status);
-int nand_exit_status_op(struct nand_chip *chip);
int nand_erase_op(struct nand_chip *chip, unsigned int eraseblock);
int nand_read_page_op(struct nand_chip *chip, unsigned int page,
unsigned int offset_in_page, void *buf, unsigned int len);
int nand_write_data_op(struct nand_chip *chip, const void *buf,
unsigned int len, bool force_8bit);
+/* Scan and identify a NAND device */
+int nand_scan_with_ids(struct nand_chip *chip, unsigned int max_chips,
+ struct nand_flash_dev *ids);
+
+static inline int nand_scan(struct nand_chip *chip, unsigned int max_chips)
+{
+ return nand_scan_with_ids(chip, max_chips, NULL);
+}
+
+/* Internal helper for board drivers which need to override command function */
+void nand_wait_ready(struct nand_chip *chip);
+
/*
* Free resources held by the NAND device, must be called on error after a
* sucessful nand_scan().
*/
void nand_cleanup(struct nand_chip *chip);
/* Unregister the MTD device and calls nand_cleanup() */
-void nand_release(struct mtd_info *mtd);
-
-/* Default extended ID decoding function */
-void nand_decode_ext_id(struct nand_chip *chip);
+void nand_release(struct nand_chip *chip);
/*
* External helper for controller drivers that have to implement the WAITRDY
SNOR_F_BROKEN_RESET = BIT(6),
};
+/**
+ * struct spi_nor_erase_type - Structure to describe a SPI NOR erase type
+ * @size: the size of the sector/block erased by the erase type.
+ * JEDEC JESD216B imposes erase sizes to be a power of 2.
+ * @size_shift: @size is a power of 2, the shift is stored in
+ * @size_shift.
+ * @size_mask: the size mask based on @size_shift.
+ * @opcode: the SPI command op code to erase the sector/block.
+ * @idx: Erase Type index as sorted in the Basic Flash Parameter
+ * Table. It will be used to synchronize the supported
+ * Erase Types with the ones identified in the SFDP
+ * optional tables.
+ */
+struct spi_nor_erase_type {
+ u32 size;
+ u32 size_shift;
+ u32 size_mask;
+ u8 opcode;
+ u8 idx;
+};
+
+/**
+ * struct spi_nor_erase_command - Used for non-uniform erases
+ * The structure is used to describe a list of erase commands to be executed
+ * once we validate that the erase can be performed. The elements in the list
+ * are run-length encoded.
+ * @list: for inclusion into the list of erase commands.
+ * @count: how many times the same erase command should be
+ * consecutively used.
+ * @size: the size of the sector/block erased by the command.
+ * @opcode: the SPI command op code to erase the sector/block.
+ */
+struct spi_nor_erase_command {
+ struct list_head list;
+ u32 count;
+ u32 size;
+ u8 opcode;
+};
+
+/**
+ * struct spi_nor_erase_region - Structure to describe a SPI NOR erase region
+ * @offset: the offset in the data array of erase region start.
+ * LSB bits are used as a bitmask encoding flags to
+ * determine if this region is overlaid, if this region is
+ * the last in the SPI NOR flash memory and to indicate
+ * all the supported erase commands inside this region.
+ * The erase types are sorted in ascending order with the
+ * smallest Erase Type size being at BIT(0).
+ * @size: the size of the region in bytes.
+ */
+struct spi_nor_erase_region {
+ u64 offset;
+ u64 size;
+};
+
+#define SNOR_ERASE_TYPE_MAX 4
+#define SNOR_ERASE_TYPE_MASK GENMASK_ULL(SNOR_ERASE_TYPE_MAX - 1, 0)
+
+#define SNOR_LAST_REGION BIT(4)
+#define SNOR_OVERLAID_REGION BIT(5)
+
+#define SNOR_ERASE_FLAGS_MAX 6
+#define SNOR_ERASE_FLAGS_MASK GENMASK_ULL(SNOR_ERASE_FLAGS_MAX - 1, 0)
+
+/**
+ * struct spi_nor_erase_map - Structure to describe the SPI NOR erase map
+ * @regions: array of erase regions. The regions are consecutive in
+ * address space. Walking through the regions is done
+ * incrementally.
+ * @uniform_region: a pre-allocated erase region for SPI NOR with a uniform
+ * sector size (legacy implementation).
+ * @erase_type: an array of erase types shared by all the regions.
+ * The erase types are sorted in ascending order, with the
+ * smallest Erase Type size being the first member in the
+ * erase_type array.
+ * @uniform_erase_type: bitmask encoding erase types that can erase the
+ * entire memory. This member is completed at init by
+ * uniform and non-uniform SPI NOR flash memories if they
+ * support at least one erase type that can erase the
+ * entire memory.
+ */
+struct spi_nor_erase_map {
+ struct spi_nor_erase_region *regions;
+ struct spi_nor_erase_region uniform_region;
+ struct spi_nor_erase_type erase_type[SNOR_ERASE_TYPE_MAX];
+ u8 uniform_erase_type;
+};
+
/**
* struct flash_info - Forward declaration of a structure used internally by
* spi_nor_scan()
* @write_proto: the SPI protocol for write operations
* @reg_proto the SPI protocol for read_reg/write_reg/erase operations
* @cmd_buf: used by the write_reg
+ * @erase_map: the erase map of the SPI NOR
* @prepare: [OPTIONAL] do some preparations for the
* read/write/erase/lock/unlock operations
* @unprepare: [OPTIONAL] do some post work after the
bool sst_write_second;
u32 flags;
u8 cmd_buf[SPI_NOR_MAX_CMD_SIZE];
+ struct spi_nor_erase_map erase_map;
int (*prepare)(struct spi_nor *nor, enum spi_nor_ops ops);
void (*unprepare)(struct spi_nor *nor, enum spi_nor_ops ops);
void *priv;
};
+static u64 __maybe_unused
+spi_nor_region_is_last(const struct spi_nor_erase_region *region)
+{
+ return region->offset & SNOR_LAST_REGION;
+}
+
+static u64 __maybe_unused
+spi_nor_region_end(const struct spi_nor_erase_region *region)
+{
+ return (region->offset & ~SNOR_ERASE_FLAGS_MASK) + region->size;
+}
+
+static void __maybe_unused
+spi_nor_region_mark_end(struct spi_nor_erase_region *region)
+{
+ region->offset |= SNOR_LAST_REGION;
+}
+
+static void __maybe_unused
+spi_nor_region_mark_overlay(struct spi_nor_erase_region *region)
+{
+ region->offset |= SNOR_OVERLAID_REGION;
+}
+
+static bool __maybe_unused spi_nor_has_uniform_erase(const struct spi_nor *nor)
+{
+ return !!nor->erase_map.uniform_erase_type;
+}
+
static inline void spi_nor_set_flash_node(struct spi_nor *nor,
struct device_node *np)
{
struct netlink_ext_ack *extack;
};
+struct netdev_notifier_info_ext {
+ struct netdev_notifier_info info; /* must be first */
+ union {
+ u32 mtu;
+ } ext;
+};
+
struct netdev_notifier_change_info {
struct netdev_notifier_info info; /* must be first */
unsigned int flags_changed;
NVME_SC_ANA_PERSISTENT_LOSS = 0x301,
NVME_SC_ANA_INACCESSIBLE = 0x302,
NVME_SC_ANA_TRANSITION = 0x303,
+ NVME_SC_HOST_PATH_ERROR = 0x370,
NVME_SC_DNR = 0x4000,
};
int of_dma_configure(struct device *dev,
struct device_node *np,
bool force_dma);
-void of_dma_deconfigure(struct device *dev);
#else /* CONFIG_OF */
static inline int of_driver_match_device(struct device *dev,
{
return 0;
}
-static inline void of_dma_deconfigure(struct device *dev)
-{}
#endif /* CONFIG_OF */
#endif /* _LINUX_OF_DEVICE_H */
unsigned long *out_hwirq,
unsigned int *out_type)
{ return -EINVAL; }
+
+static inline const struct pci_device_id *pci_match_id(const struct pci_device_id *ids,
+ struct pci_dev *dev)
+{ return NULL; }
#endif /* CONFIG_PCI */
/* Include architecture-dependent settings and functions */
#define PCI_CLASS_SERIAL_USB_DEVICE 0x0c03fe
#define PCI_CLASS_SERIAL_FIBER 0x0c04
#define PCI_CLASS_SERIAL_SMBUS 0x0c05
+#define PCI_CLASS_SERIAL_IPMI 0x0c07
+#define PCI_CLASS_SERIAL_IPMI_SMIC 0x0c0700
+#define PCI_CLASS_SERIAL_IPMI_KCS 0x0c0701
+#define PCI_CLASS_SERIAL_IPMI_BT 0x0c0702
#define PCI_BASE_CLASS_WIRELESS 0x0d
#define PCI_CLASS_WIRELESS_RF_CONTROLLER 0x0d10
#define PCI_VENDOR_ID_AMAZON 0x1d0f
+#define PCI_VENDOR_ID_HYGON 0x1d94
+
#define PCI_VENDOR_ID_TEKRAM 0x1de1
#define PCI_DEVICE_ID_TEKRAM_DC290 0xdc29
void percpu_ref_switch_to_percpu(struct percpu_ref *ref);
void percpu_ref_kill_and_confirm(struct percpu_ref *ref,
percpu_ref_func_t *confirm_kill);
+void percpu_ref_resurrect(struct percpu_ref *ref);
void percpu_ref_reinit(struct percpu_ref *ref);
/**
void (*stop)(struct arm_pmu *);
void (*reset)(void *);
int (*map_event)(struct perf_event *event);
+ int (*filter_match)(struct perf_event *event);
int num_events;
bool secure_access; /* 32-bit ARM only */
#define ARMV8_PMUV3_MAX_COMMON_EVENTS 0x40
#ifndef __DAVINCI_GPIO_PLATFORM_H
#define __DAVINCI_GPIO_PLATFORM_H
-#include <linux/io.h>
-#include <linux/spinlock.h>
-
-#include <asm-generic/gpio.h>
-
-#define MAX_REGS_BANKS 5
-#define MAX_INT_PER_BANK 32
-
struct davinci_gpio_platform_data {
u32 ngpio;
u32 gpio_unbanked;
};
-struct davinci_gpio_irq_data {
- void __iomem *regs;
- struct davinci_gpio_controller *chip;
- int bank_num;
-};
-
-struct davinci_gpio_controller {
- struct gpio_chip chip;
- struct irq_domain *irq_domain;
- /* Serialize access to GPIO registers */
- spinlock_t lock;
- void __iomem *regs[MAX_REGS_BANKS];
- int gpio_unbanked;
- int irqs[MAX_INT_PER_BANK];
- unsigned int base;
-};
-
-/*
- * basic gpio routines
- */
-#define GPIO(X) (X) /* 0 <= X <= (DAVINCI_N_GPIO - 1) */
-
/* Convert GPIO signal to GPIO pin number */
#define GPIO_TO_PIN(bank, gpio) (16 * (bank) + (gpio))
-static inline u32 __gpio_mask(unsigned gpio)
-{
- return 1 << (gpio % 32);
-}
#endif
bool is_mpuio; /* whether the bank is of type MPUIO */
u32 non_wakeup_gpios;
+ u32 quirks; /* Version specific quirks mask */
+
struct omap_gpio_reg_offs *regs;
/* Return context loss count due to PM states changing */
int (*get_context_loss_count)(struct device *dev);
};
-#if IS_BUILTIN(CONFIG_GPIO_OMAP)
-extern void omap2_gpio_prepare_for_idle(int off_mode);
-extern void omap2_gpio_resume_after_idle(void);
-#else
-static inline void omap2_gpio_prepare_for_idle(int off_mode)
-{
-}
-
-static inline void omap2_gpio_resume_after_idle(void)
-{
-}
-#endif
-
#endif
+++ /dev/null
-/*
- * GPIO (DIO) header for Technologic Systems TS-5500
- *
- * Copyright (c) 2012 Savoir-faire Linux Inc.
- * Vivien Didelot <vivien.didelot@savoirfairelinux.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#ifndef _PDATA_GPIO_TS5500_H
-#define _PDATA_GPIO_TS5500_H
-
-/**
- * struct ts5500_dio_platform_data - TS-5500 pin block configuration
- * @base: The GPIO base number to use.
- * @strap: The only pin connected to an interrupt in a block is input-only.
- * If you need a bidirectional line which can trigger an IRQ, you
- * may strap it with an in/out pin. This flag indicates this case.
- */
-struct ts5500_dio_platform_data {
- int base;
- bool strap;
-};
-
-#endif /* _PDATA_GPIO_TS5500_H */
/* string specifying a particular variant of hardware */
char *version;
- int gpio_cd; /* gpio (card detect) */
- int gpio_cod; /* gpio (cover detect) */
- int gpio_wp; /* gpio (write protect) */
/* if we have special card, init it using this callback */
void (*init_card)(struct mmc_card *card);
* 1: choose feedback clk + delay value
* 2: choose internal clk
* @clk_delay_enable: enable clk_delay or not, used on pxa910
- * @ext_cd_gpio: gpio pin used for external CD line
- * @ext_cd_gpio_invert: invert values for external CD gpio line
* @max_speed: the maximum speed supported
* @host_caps: Standard MMC host capabilities bit field.
* @quirks: quirks of platfrom
unsigned int clk_delay_cycles;
unsigned int clk_delay_sel;
bool clk_delay_enable;
- unsigned int ext_cd_gpio;
- bool ext_cd_gpio_invert;
unsigned int max_speed;
u32 host_caps;
u32 host_caps2;
* @num_chipselect: number of chipselects supported by this SPI master
* @intr_line: interrupt line used to connect the SPI IP to the ARM interrupt
* controller withn the SoC. Possible values are 0 and 1.
- * @chip_sel: list of GPIOs which can act as chip-selects for the SPI.
- * SPI_INTERN_CS denotes internal SPI chip-select. Not necessary
- * to populate if all chip-selects are internal.
* @cshold_bug: set this to true if the SPI controller on your chip requires
* a write to CSHOLD bit in between transfers (like in DM355).
* @dma_event_q: DMA event queue to use if SPI_IO_TYPE_DMA is used for any
u8 version;
u8 num_chipselect;
u8 intr_line;
- u8 *chip_sel;
u8 prescaler_limit;
bool cshold_bug;
enum dma_event_q dma_event_q;
#define platform_get_device_id(pdev) ((pdev)->id_entry)
+#define dev_is_platform(dev) ((dev)->bus == &platform_bus_type)
#define to_platform_device(x) container_of((x), struct platform_device, dev)
extern int platform_device_register(struct platform_device *);
#include <linux/notifier.h>
#include <linux/spinlock.h>
-/* Defines used for the flags field in the struct generic_pm_domain */
-#define GENPD_FLAG_PM_CLK (1U << 0) /* PM domain uses PM clk */
-#define GENPD_FLAG_IRQ_SAFE (1U << 1) /* PM domain operates in atomic */
-#define GENPD_FLAG_ALWAYS_ON (1U << 2) /* PM domain is always powered on */
-#define GENPD_FLAG_ACTIVE_WAKEUP (1U << 3) /* Keep devices active if wakeup */
+/*
+ * Flags to control the behaviour of a genpd.
+ *
+ * These flags may be set in the struct generic_pm_domain's flags field by a
+ * genpd backend driver. The flags must be set before it calls pm_genpd_init(),
+ * which initializes a genpd.
+ *
+ * GENPD_FLAG_PM_CLK: Instructs genpd to use the PM clk framework,
+ * while powering on/off attached devices.
+ *
+ * GENPD_FLAG_IRQ_SAFE: This informs genpd that its backend callbacks,
+ * ->power_on|off(), doesn't sleep. Hence, these
+ * can be invoked from within atomic context, which
+ * enables genpd to power on/off the PM domain,
+ * even when pm_runtime_is_irq_safe() returns true,
+ * for any of its attached devices. Note that, a
+ * genpd having this flag set, requires its
+ * masterdomains to also have it set.
+ *
+ * GENPD_FLAG_ALWAYS_ON: Instructs genpd to always keep the PM domain
+ * powered on.
+ *
+ * GENPD_FLAG_ACTIVE_WAKEUP: Instructs genpd to keep the PM domain powered
+ * on, in case any of its attached devices is used
+ * in the wakeup path to serve system wakeups.
+ */
+#define GENPD_FLAG_PM_CLK (1U << 0)
+#define GENPD_FLAG_IRQ_SAFE (1U << 1)
+#define GENPD_FLAG_ALWAYS_ON (1U << 2)
+#define GENPD_FLAG_ACTIVE_WAKEUP (1U << 3)
enum gpd_status {
GPD_STATE_ACTIVE = 0, /* PM domain is active */
#if defined(CONFIG_PM_OPP)
struct opp_table *dev_pm_opp_get_opp_table(struct device *dev);
+struct opp_table *dev_pm_opp_get_opp_table_indexed(struct device *dev, int index);
void dev_pm_opp_put_opp_table(struct opp_table *opp_table);
unsigned long dev_pm_opp_get_voltage(struct dev_pm_opp *opp);
return ERR_PTR(-ENOTSUPP);
}
+static inline struct opp_table *dev_pm_opp_get_opp_table_indexed(struct device *dev, int index)
+{
+ return ERR_PTR(-ENOTSUPP);
+}
+
static inline void dev_pm_opp_put_opp_table(struct opp_table *opp_table) {}
static inline unsigned long dev_pm_opp_get_voltage(struct dev_pm_opp *opp)
PXA27x_SSP,
PXA3xx_SSP,
PXA168_SSP,
+ MMP2_SSP,
PXA910_SSP,
CE4100_SSP,
QUARK_X1000_SSP,
const char *label;
int port_id;
- int type;
+ enum pxa_ssp_type type;
int use_count;
int irq;
#define HW_VER_MINOR_SHFT 16
#define HW_VER_STEP_MASK GENMASK(15, 0)
+#define GENI_SE_VERSION_MAJOR(ver) ((ver & HW_VER_MAJOR_MASK) >> HW_VER_MAJOR_SHFT)
+#define GENI_SE_VERSION_MINOR(ver) ((ver & HW_VER_MINOR_MASK) >> HW_VER_MINOR_SHFT)
+#define GENI_SE_VERSION_STEP(ver) (ver & HW_VER_STEP_MASK)
+
#if IS_ENABLED(CONFIG_QCOM_GENI_SE)
u32 geni_se_get_qup_hw_version(struct geni_se *se);
-#define geni_se_get_wrapper_version(se, major, minor, step) do { \
- u32 ver; \
-\
- ver = geni_se_get_qup_hw_version(se); \
- major = (ver & HW_VER_MAJOR_MASK) >> HW_VER_MAJOR_SHFT; \
- minor = (ver & HW_VER_MINOR_MASK) >> HW_VER_MINOR_SHFT; \
- step = version & HW_VER_STEP_MASK; \
-} while (0)
-
/**
* geni_se_read_proto() - Read the protocol configured for a serial engine
* @se: Pointer to the concerned serial engine.
* @list: the RCU-protected list to splice
* @prev: points to the last element of the existing list
* @next: points to the first element of the existing list
- * @sync: function to sync: synchronize_rcu(), synchronize_sched(), ...
+ * @sync: synchronize_rcu, synchronize_rcu_expedited, ...
*
* The list pointed to by @prev and @next can be RCU-read traversed
* concurrently with this function.
* designed for stacks.
* @list: the RCU-protected list to splice
* @head: the place in the existing list to splice the first list into
- * @sync: function to sync: synchronize_rcu(), synchronize_sched(), ...
+ * @sync: synchronize_rcu, synchronize_rcu_expedited, ...
*/
static inline void list_splice_init_rcu(struct list_head *list,
struct list_head *head,
* list, designed for queues.
* @list: the RCU-protected list to splice
* @head: the place in the existing list to splice the first list into
- * @sync: function to sync: synchronize_rcu(), synchronize_sched(), ...
+ * @sync: synchronize_rcu, synchronize_rcu_expedited, ...
*/
static inline void list_splice_tail_init_rcu(struct list_head *list,
struct list_head *head,
* @type: the type of the struct this is embedded in.
* @member: the name of the list_head within the struct.
*
- * This primitive may safely run concurrently with the _rcu list-mutation
- * primitives such as list_add_rcu(), but requires some implicit RCU
- * read-side guarding. One example is running within a special
- * exception-time environment where preemption is disabled and where
- * lockdep cannot be invoked (in which case updaters must use RCU-sched,
- * as in synchronize_sched(), call_rcu_sched(), and friends). Another
- * example is when items are added to the list, but never deleted.
+ * This primitive may safely run concurrently with the _rcu
+ * list-mutation primitives such as list_add_rcu(), but requires some
+ * implicit RCU read-side guarding. One example is running within a special
+ * exception-time environment where preemption is disabled and where lockdep
+ * cannot be invoked. Another example is when items are added to the list,
+ * but never deleted.
*/
#define list_entry_lockless(ptr, type, member) \
container_of((typeof(ptr))READ_ONCE(ptr), type, member)
* @head: the head for your list.
* @member: the name of the list_struct within the struct.
*
- * This primitive may safely run concurrently with the _rcu list-mutation
- * primitives such as list_add_rcu(), but requires some implicit RCU
- * read-side guarding. One example is running within a special
- * exception-time environment where preemption is disabled and where
- * lockdep cannot be invoked (in which case updaters must use RCU-sched,
- * as in synchronize_sched(), call_rcu_sched(), and friends). Another
- * example is when items are added to the list, but never deleted.
+ * This primitive may safely run concurrently with the _rcu
+ * list-mutation primitives such as list_add_rcu(), but requires some
+ * implicit RCU read-side guarding. One example is running within a special
+ * exception-time environment where preemption is disabled and where lockdep
+ * cannot be invoked. Another example is when items are added to the list,
+ * but never deleted.
*/
#define list_for_each_entry_lockless(pos, head, member) \
for (pos = list_entry_lockless((head)->next, typeof(*pos), member); \
#define ulong2long(a) (*(long *)(&(a)))
/* Exported common interfaces */
-
-#ifdef CONFIG_PREEMPT_RCU
void call_rcu(struct rcu_head *head, rcu_callback_t func);
-#else /* #ifdef CONFIG_PREEMPT_RCU */
-#define call_rcu call_rcu_sched
-#endif /* #else #ifdef CONFIG_PREEMPT_RCU */
-
-void call_rcu_bh(struct rcu_head *head, rcu_callback_t func);
-void call_rcu_sched(struct rcu_head *head, rcu_callback_t func);
-void synchronize_sched(void);
void rcu_barrier_tasks(void);
+void synchronize_rcu(void);
#ifdef CONFIG_PREEMPT_RCU
void __rcu_read_lock(void);
void __rcu_read_unlock(void);
-void synchronize_rcu(void);
/*
* Defined as a macro as it is a very low level header included from
preempt_enable();
}
-static inline void synchronize_rcu(void)
-{
- synchronize_sched();
-}
-
static inline int rcu_preempt_depth(void)
{
return 0;
/* Internal to kernel */
void rcu_init(void);
extern int rcu_scheduler_active __read_mostly;
-void rcu_sched_qs(void);
-void rcu_bh_qs(void);
void rcu_check_callbacks(int user);
void rcu_report_dead(unsigned int cpu);
void rcutree_migrate_callbacks(int cpu);
* RCU_NONIDLE - Indicate idle-loop code that needs RCU readers
* @a: Code that RCU needs to pay attention to.
*
- * RCU, RCU-bh, and RCU-sched read-side critical sections are forbidden
- * in the inner idle loop, that is, between the rcu_idle_enter() and
- * the rcu_idle_exit() -- RCU will happily ignore any such read-side
- * critical sections. However, things like powertop need tracepoints
- * in the inner idle loop.
+ * RCU read-side critical sections are forbidden in the inner idle loop,
+ * that is, between the rcu_idle_enter() and the rcu_idle_exit() -- RCU
+ * will happily ignore any such read-side critical sections. However,
+ * things like powertop need tracepoints in the inner idle loop.
*
* This macro provides the way out: RCU_NONIDLE(do_something_with_RCU())
* will tell RCU that it needs to pay attention, invoke its argument
if (READ_ONCE((t)->rcu_tasks_holdout)) \
WRITE_ONCE((t)->rcu_tasks_holdout, false); \
} while (0)
-#define rcu_note_voluntary_context_switch(t) \
- do { \
- rcu_all_qs(); \
- rcu_tasks_qs(t); \
- } while (0)
+#define rcu_note_voluntary_context_switch(t) rcu_tasks_qs(t)
void call_rcu_tasks(struct rcu_head *head, rcu_callback_t func);
void synchronize_rcu_tasks(void);
void exit_tasks_rcu_start(void);
void exit_tasks_rcu_finish(void);
#else /* #ifdef CONFIG_TASKS_RCU */
#define rcu_tasks_qs(t) do { } while (0)
-#define rcu_note_voluntary_context_switch(t) rcu_all_qs()
-#define call_rcu_tasks call_rcu_sched
-#define synchronize_rcu_tasks synchronize_sched
+#define rcu_note_voluntary_context_switch(t) do { } while (0)
+#define call_rcu_tasks call_rcu
+#define synchronize_rcu_tasks synchronize_rcu
static inline void exit_tasks_rcu_start(void) { }
static inline void exit_tasks_rcu_finish(void) { }
#endif /* #else #ifdef CONFIG_TASKS_RCU */
* Helper functions for rcu_dereference_check(), rcu_dereference_protected()
* and rcu_assign_pointer(). Some of these could be folded into their
* callers, but they are left separate in order to ease introduction of
- * multiple flavors of pointers to match the multiple flavors of RCU
- * (e.g., __rcu_bh, * __rcu_sched, and __srcu), should this make sense in
- * the future.
+ * multiple pointers markings to match different RCU implementations
+ * (e.g., __srcu), should this make sense in the future.
*/
#ifdef __CHECKER__
/**
* rcu_read_lock_bh() - mark the beginning of an RCU-bh critical section
*
- * This is equivalent of rcu_read_lock(), but to be used when updates
- * are being done using call_rcu_bh() or synchronize_rcu_bh(). Since
- * both call_rcu_bh() and synchronize_rcu_bh() consider completion of a
- * softirq handler to be a quiescent state, a process in RCU read-side
- * critical section must be protected by disabling softirqs. Read-side
- * critical sections in interrupt context can use just rcu_read_lock(),
- * though this should at least be commented to avoid confusing people
- * reading the code.
+ * This is equivalent of rcu_read_lock(), but also disables softirqs.
+ * Note that anything else that disables softirqs can also serve as
+ * an RCU read-side critical section.
*
* Note that rcu_read_lock_bh() and the matching rcu_read_unlock_bh()
* must occur in the same context, for example, it is illegal to invoke
/**
* rcu_read_lock_sched() - mark the beginning of a RCU-sched critical section
*
- * This is equivalent of rcu_read_lock(), but to be used when updates
- * are being done using call_rcu_sched() or synchronize_rcu_sched().
- * Read-side critical sections can also be introduced by anything that
- * disables preemption, including local_irq_disable() and friends.
+ * This is equivalent of rcu_read_lock(), but disables preemption.
+ * Read-side critical sections can also be introduced by anything else
+ * that disables preemption, including local_irq_disable() and friends.
*
* Note that rcu_read_lock_sched() and the matching rcu_read_unlock_sched()
* must occur in the same context, for example, it is illegal to invoke
#endif /* #else #ifdef CONFIG_ARCH_WEAK_RELEASE_ACQUIRE */
+/* Has the specified rcu_head structure been handed to call_rcu()? */
+
+/*
+ * rcu_head_init - Initialize rcu_head for rcu_head_after_call_rcu()
+ * @rhp: The rcu_head structure to initialize.
+ *
+ * If you intend to invoke rcu_head_after_call_rcu() to test whether a
+ * given rcu_head structure has already been passed to call_rcu(), then
+ * you must also invoke this rcu_head_init() function on it just after
+ * allocating that structure. Calls to this function must not race with
+ * calls to call_rcu(), rcu_head_after_call_rcu(), or callback invocation.
+ */
+static inline void rcu_head_init(struct rcu_head *rhp)
+{
+ rhp->func = (rcu_callback_t)~0L;
+}
+
+/*
+ * rcu_head_after_call_rcu - Has this rcu_head been passed to call_rcu()?
+ * @rhp: The rcu_head structure to test.
+ * @func: The function passed to call_rcu() along with @rhp.
+ *
+ * Returns @true if the @rhp has been passed to call_rcu() with @func,
+ * and @false otherwise. Emits a warning in any other case, including
+ * the case where @rhp has already been invoked after a grace period.
+ * Calls to this function must not race with callback invocation. One way
+ * to avoid such races is to enclose the call to rcu_head_after_call_rcu()
+ * in an RCU read-side critical section that includes a read-side fetch
+ * of the pointer to the structure containing @rhp.
+ */
+static inline bool
+rcu_head_after_call_rcu(struct rcu_head *rhp, rcu_callback_t f)
+{
+ if (READ_ONCE(rhp->func) == f)
+ return true;
+ WARN_ON_ONCE(READ_ONCE(rhp->func) != (rcu_callback_t)~0L);
+ return false;
+}
+
+
+/* Transitional pre-consolidation compatibility definitions. */
+
+static inline void synchronize_rcu_bh(void)
+{
+ synchronize_rcu();
+}
+
+static inline void synchronize_rcu_bh_expedited(void)
+{
+ synchronize_rcu_expedited();
+}
+
+static inline void call_rcu_bh(struct rcu_head *head, rcu_callback_t func)
+{
+ call_rcu(head, func);
+}
+
+static inline void rcu_barrier_bh(void)
+{
+ rcu_barrier();
+}
+
+static inline void synchronize_sched(void)
+{
+ synchronize_rcu();
+}
+
+static inline void synchronize_sched_expedited(void)
+{
+ synchronize_rcu_expedited();
+}
+
+static inline void call_rcu_sched(struct rcu_head *head, rcu_callback_t func)
+{
+ call_rcu(head, func);
+}
+
+static inline void rcu_barrier_sched(void)
+{
+ rcu_barrier();
+}
+
+static inline unsigned long get_state_synchronize_sched(void)
+{
+ return get_state_synchronize_rcu();
+}
+
+static inline void cond_synchronize_sched(unsigned long oldstate)
+{
+ cond_synchronize_rcu(oldstate);
+}
+
#endif /* __LINUX_RCUPDATE_H */
/**
* synchronize_rcu_mult - Wait concurrently for multiple grace periods
- * @...: List of call_rcu() functions for the flavors to wait on.
+ * @...: List of call_rcu() functions for different grace periods to wait on
*
- * This macro waits concurrently for multiple flavors of RCU grace periods.
- * For example, synchronize_rcu_mult(call_rcu, call_rcu_bh) would wait
- * on concurrent RCU and RCU-bh grace periods. Waiting on a give SRCU
+ * This macro waits concurrently for multiple types of RCU grace periods.
+ * For example, synchronize_rcu_mult(call_rcu, call_rcu_tasks) would wait
+ * on concurrent RCU and RCU-tasks grace periods. Waiting on a give SRCU
* domain requires you to write a wrapper function for that SRCU domain's
* call_srcu() function, supplying the corresponding srcu_struct.
*
- * If Tiny RCU, tell _wait_rcu_gp() not to bother waiting for RCU
- * or RCU-bh, given that anywhere synchronize_rcu_mult() can be called
- * is automatically a grace period.
+ * If Tiny RCU, tell _wait_rcu_gp() does not bother waiting for RCU,
+ * given that anywhere synchronize_rcu_mult() can be called is automatically
+ * a grace period.
*/
#define synchronize_rcu_mult(...) \
_wait_rcu_gp(IS_ENABLED(CONFIG_TINY_RCU), __VA_ARGS__)
#include <linux/ktime.h>
-struct rcu_dynticks;
-static inline int rcu_dynticks_snap(struct rcu_dynticks *rdtp)
-{
- return 0;
-}
-
/* Never flag non-existent other CPUs! */
static inline bool rcu_eqs_special_set(int cpu) { return false; }
might_sleep();
}
-static inline unsigned long get_state_synchronize_sched(void)
-{
- return 0;
-}
-
-static inline void cond_synchronize_sched(unsigned long oldstate)
-{
- might_sleep();
-}
-
-extern void rcu_barrier_bh(void);
-extern void rcu_barrier_sched(void);
+extern void rcu_barrier(void);
static inline void synchronize_rcu_expedited(void)
{
- synchronize_sched(); /* Only one CPU, so pretty fast anyway!!! */
+ synchronize_rcu();
}
-static inline void rcu_barrier(void)
+static inline void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func)
{
- rcu_barrier_sched(); /* Only one CPU, so only one list of callbacks! */
-}
-
-static inline void synchronize_rcu_bh(void)
-{
- synchronize_sched();
-}
-
-static inline void synchronize_rcu_bh_expedited(void)
-{
- synchronize_sched();
+ call_rcu(head, func);
}
-static inline void synchronize_sched_expedited(void)
-{
- synchronize_sched();
-}
+void rcu_qs(void);
-static inline void kfree_call_rcu(struct rcu_head *head,
- rcu_callback_t func)
+static inline void rcu_softirq_qs(void)
{
- call_rcu(head, func);
+ rcu_qs();
}
#define rcu_note_context_switch(preempt) \
do { \
- rcu_sched_qs(); \
+ rcu_qs(); \
rcu_tasks_qs(current); \
} while (0)
*/
static inline void rcu_virt_note_context_switch(int cpu) { }
static inline void rcu_cpu_stall_reset(void) { }
+static inline int rcu_jiffies_till_stall_check(void) { return 21 * HZ; }
static inline void rcu_idle_enter(void) { }
static inline void rcu_idle_exit(void) { }
static inline void rcu_irq_enter(void) { }
static inline void rcu_irq_enter_irqson(void) { }
static inline void rcu_irq_exit(void) { }
static inline void exit_rcu(void) { }
+static inline bool rcu_preempt_need_deferred_qs(struct task_struct *t)
+{
+ return false;
+}
+static inline void rcu_preempt_deferred_qs(struct task_struct *t) { }
#ifdef CONFIG_SRCU
void rcu_scheduler_starting(void);
#else /* #ifndef CONFIG_SRCU */
#ifndef __LINUX_RCUTREE_H
#define __LINUX_RCUTREE_H
+void rcu_softirq_qs(void);
void rcu_note_context_switch(bool preempt);
int rcu_needs_cpu(u64 basem, u64 *nextevt);
void rcu_cpu_stall_reset(void);
rcu_note_context_switch(false);
}
-void synchronize_rcu_bh(void);
-void synchronize_sched_expedited(void);
void synchronize_rcu_expedited(void);
-
void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func);
-/**
- * synchronize_rcu_bh_expedited - Brute-force RCU-bh grace period
- *
- * Wait for an RCU-bh grace period to elapse, but use a "big hammer"
- * approach to force the grace period to end quickly. This consumes
- * significant time on all CPUs and is unfriendly to real-time workloads,
- * so is thus not recommended for any sort of common-case code. In fact,
- * if you are using synchronize_rcu_bh_expedited() in a loop, please
- * restructure your code to batch your updates, and then use a single
- * synchronize_rcu_bh() instead.
- *
- * Note that it is illegal to call this function while holding any lock
- * that is acquired by a CPU-hotplug notifier. And yes, it is also illegal
- * to call this function from a CPU-hotplug notifier. Failing to observe
- * these restriction will result in deadlock.
- */
-static inline void synchronize_rcu_bh_expedited(void)
-{
- synchronize_sched_expedited();
-}
-
void rcu_barrier(void);
-void rcu_barrier_bh(void);
-void rcu_barrier_sched(void);
bool rcu_eqs_special_set(int cpu);
unsigned long get_state_synchronize_rcu(void);
void cond_synchronize_rcu(unsigned long oldstate);
-unsigned long get_state_synchronize_sched(void);
-void cond_synchronize_sched(unsigned long oldstate);
void rcu_idle_enter(void);
void rcu_idle_exit(void);
extern int rcu_scheduler_active __read_mostly;
void rcu_end_inkernel_boot(void);
bool rcu_is_watching(void);
+#ifndef CONFIG_PREEMPT
void rcu_all_qs(void);
+#endif
/* RCUtree hotplug events */
int rcutree_prepare_cpu(unsigned int cpu);
* field is NULL but precious_table (see below) is not, the
* check is performed on such table (a register is precious if
* it belongs to one of the ranges specified by precious_table).
+ * @writeable_noinc_reg: Optional callback returning true if the register
+ * supports multiple write operations without incrementing
+ * the register number. If this field is NULL but
+ * wr_noinc_table (see below) is not, the check is
+ * performed on such table (a register is no increment
+ * writeable if it belongs to one of the ranges specified
+ * by wr_noinc_table).
* @readable_noinc_reg: Optional callback returning true if the register
* supports multiple read operations without incrementing
* the register number. If this field is NULL but
* @rd_table: As above, for read access.
* @volatile_table: As above, for volatile registers.
* @precious_table: As above, for precious registers.
+ * @wr_noinc_table: As above, for no increment writeable registers.
* @rd_noinc_table: As above, for no increment readable registers.
* @reg_defaults: Power on reset values for registers (for use with
* register cache support).
* masks are used.
* @zero_flag_mask: If set, read_flag_mask and write_flag_mask are used even
* if they are both empty.
- * @use_single_rw: If set, converts the bulk read and write operations into
- * a series of single read and write operations. This is useful
- * for device that does not support bulk read and write.
+ * @use_single_read: If set, converts the bulk read operation into a series of
+ * single read operations. This is useful for a device that
+ * does not support bulk read.
+ * @use_single_write: If set, converts the bulk write operation into a series of
+ * single write operations. This is useful for a device that
+ * does not support bulk write.
* @can_multi_write: If set, the device supports the multi write mode of bulk
* write operations, if clear multi write requests will be
* split into individual write operations
bool (*readable_reg)(struct device *dev, unsigned int reg);
bool (*volatile_reg)(struct device *dev, unsigned int reg);
bool (*precious_reg)(struct device *dev, unsigned int reg);
+ bool (*writeable_noinc_reg)(struct device *dev, unsigned int reg);
bool (*readable_noinc_reg)(struct device *dev, unsigned int reg);
bool disable_locking;
const struct regmap_access_table *rd_table;
const struct regmap_access_table *volatile_table;
const struct regmap_access_table *precious_table;
+ const struct regmap_access_table *wr_noinc_table;
const struct regmap_access_table *rd_noinc_table;
const struct reg_default *reg_defaults;
unsigned int num_reg_defaults;
unsigned long write_flag_mask;
bool zero_flag_mask;
- bool use_single_rw;
+ bool use_single_read;
+ bool use_single_write;
bool can_multi_write;
enum regmap_endian reg_format_endian;
int regmap_write_async(struct regmap *map, unsigned int reg, unsigned int val);
int regmap_raw_write(struct regmap *map, unsigned int reg,
const void *val, size_t val_len);
+int regmap_noinc_write(struct regmap *map, unsigned int reg,
+ const void *val, size_t val_len);
int regmap_bulk_write(struct regmap *map, unsigned int reg, const void *val,
size_t val_count);
int regmap_multi_reg_write(struct regmap *map, const struct reg_sequence *regs,
return -EINVAL;
}
+static inline int regmap_noinc_write(struct regmap *map, unsigned int reg,
+ const void *val, size_t val_len)
+{
+ WARN_ONCE(1, "regmap API is disabled");
+ return -EINVAL;
+}
+
static inline int regmap_bulk_write(struct regmap *map, unsigned int reg,
const void *val, size_t val_count)
{
* @ramp_delay: Time to settle down after voltage change (unit: uV/us)
* @min_dropout_uV: The minimum dropout voltage this regulator can handle
* @linear_ranges: A constant table of possible voltage ranges.
- * @n_linear_ranges: Number of entries in the @linear_ranges table.
+ * @linear_range_selectors: A constant table of voltage range selectors.
+ * If pickable ranges are used each range must
+ * have corresponding selector here.
+ * @n_linear_ranges: Number of entries in the @linear_ranges (and in
+ * linear_range_selectors if used) table(s).
* @volt_table: Voltage mapping table (if table based mapping)
*
+ * @vsel_range_reg: Register for range selector when using pickable ranges
+ * and regulator_regmap_X_voltage_X_pickable functions.
+ * @vsel_range_mask: Mask for register bitfield used for range selector
* @vsel_reg: Register for selector when using regulator_regmap_X_voltage_
* @vsel_mask: Mask for register bitfield used for selector
* @csel_reg: Register for TPS65218 LS3 current regulator
int min_dropout_uV;
const struct regulator_linear_range *linear_ranges;
+ const unsigned int *linear_range_selectors;
+
int n_linear_ranges;
const unsigned int *volt_table;
+ unsigned int vsel_range_reg;
+ unsigned int vsel_range_mask;
unsigned int vsel_reg;
unsigned int vsel_mask;
unsigned int csel_reg;
int regulator_list_voltage_linear(struct regulator_dev *rdev,
unsigned int selector);
+int regulator_list_voltage_pickable_linear_range(struct regulator_dev *rdev,
+ unsigned int selector);
int regulator_list_voltage_linear_range(struct regulator_dev *rdev,
unsigned int selector);
int regulator_list_voltage_table(struct regulator_dev *rdev,
unsigned int selector);
int regulator_map_voltage_linear(struct regulator_dev *rdev,
int min_uV, int max_uV);
+int regulator_map_voltage_pickable_linear_range(struct regulator_dev *rdev,
+ int min_uV, int max_uV);
int regulator_map_voltage_linear_range(struct regulator_dev *rdev,
int min_uV, int max_uV);
int regulator_map_voltage_iterate(struct regulator_dev *rdev,
int min_uV, int max_uV);
int regulator_map_voltage_ascend(struct regulator_dev *rdev,
int min_uV, int max_uV);
+int regulator_get_voltage_sel_pickable_regmap(struct regulator_dev *rdev);
+int regulator_set_voltage_sel_pickable_regmap(struct regulator_dev *rdev,
+ unsigned int sel);
int regulator_get_voltage_sel_regmap(struct regulator_dev *rdev);
int regulator_set_voltage_sel_regmap(struct regulator_dev *rdev, unsigned sel);
int regulator_is_enabled_regmap(struct regulator_dev *rdev);
* @supply_name: Name of the regulator supply
* @input_supply: Name of the input regulator supply
* @microvolts: Output voltage of regulator
- * @gpio: GPIO to use for enable control
- * set to -EINVAL if not used
* @startup_delay: Start-up time in microseconds
* @gpio_is_open_drain: Gpio pin is open drain or normal type.
* If it is open drain type then HIGH will be set
const char *supply_name;
const char *input_supply;
int microvolts;
- int gpio;
unsigned startup_delay;
unsigned gpio_is_open_drain:1;
unsigned enable_high:1;
};
/*
- * Setting bit 0 of the owner field with other non-zero bits will indicate
+ * Setting bit 1 of the owner field but not bit 0 will indicate
* that the rwsem is writer-owned with an unknown owner.
*/
-#define RWSEM_OWNER_UNKNOWN ((struct task_struct *)-1L)
+#define RWSEM_OWNER_UNKNOWN ((struct task_struct *)-2L)
extern struct rw_semaphore *rwsem_down_read_failed(struct rw_semaphore *sem);
extern struct rw_semaphore *rwsem_down_read_failed_killable(struct rw_semaphore *sem);
struct {
u8 blocked;
u8 need_qs;
- u8 exp_need_qs;
-
- /* Otherwise the compiler can store garbage here: */
- u8 pad;
} b; /* Bits. */
- u32 s; /* Set of bits. */
+ u16 s; /* Set of bits. */
};
enum perf_event_task_context {
unsigned use_memdelay:1;
#endif
+ /*
+ * May usercopy functions fault on kernel addresses?
+ * This is not just a single bit because this can potentially nest.
+ */
+ unsigned int kernel_uaccess_faults_ok;
+
unsigned long atomic_flags; /* Flags requiring atomic access. */
struct restart_block restart_block;
#define SD_BALANCE_FORK 0x0008 /* Balance on fork, clone */
#define SD_BALANCE_WAKE 0x0010 /* Balance on wakeup */
#define SD_WAKE_AFFINE 0x0020 /* Wake task to waking CPU */
-#define SD_ASYM_CPUCAPACITY 0x0040 /* Groups have different max cpu capacities */
-#define SD_SHARE_CPUCAPACITY 0x0080 /* Domain members share cpu capacity */
+#define SD_ASYM_CPUCAPACITY 0x0040 /* Domain members have different CPU capacities */
+#define SD_SHARE_CPUCAPACITY 0x0080 /* Domain members share CPU capacity */
#define SD_SHARE_POWERDOMAIN 0x0100 /* Domain members share power domain */
-#define SD_SHARE_PKG_RESOURCES 0x0200 /* Domain members share cpu pkg resources */
+#define SD_SHARE_PKG_RESOURCES 0x0200 /* Domain members share CPU pkg resources */
#define SD_SERIALIZE 0x0400 /* Only a single load balancing instance */
#define SD_ASYM_PACKING 0x0800 /* Place busy groups earlier in the domain */
#define SD_PREFER_SIBLING 0x1000 /* Prefer to place tasks in a sibling domain */
-/*
- * Copyright (C) 2005 David Brownell
+/* SPDX-License-Identifier: GPL-2.0-or-later
*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
+ * Copyright (C) 2005 David Brownell
*/
#ifndef __LINUX_SPI_H
#define SPI_TX_QUAD 0x200 /* transmit with 4 wires */
#define SPI_RX_DUAL 0x400 /* receive with 2 wires */
#define SPI_RX_QUAD 0x800 /* receive with 4 wires */
+#define SPI_CS_WORD 0x1000 /* toggle cs after each word */
int irq;
void *controller_state;
void *controller_data;
char modalias[SPI_NAME_SIZE];
+ const char *driver_override;
int cs_gpio; /* chip select gpio */
/* the statistics */
* the controller talks to each chip, like:
* - memory packing (12 bit samples into low bits, others zeroed)
* - priority
- * - drop chipselect after each word
* - chipselect delays
* - ...
*/
* @delay_usecs: microseconds to delay after this transfer before
* (optionally) changing the chipselect status, then starting
* the next transfer or completing this @spi_message.
+ * @word_delay: clock cycles to inter word delay after each word size
+ * (set by bits_per_word) transmission.
* @transfer_list: transfers are sequenced through @spi_message.transfers
* @tx_sg: Scatterlist for transmit, currently not for client use
* @rx_sg: Scatterlist for receive, currently not for client use
u8 bits_per_word;
u16 delay_usecs;
u32 speed_hz;
+ u16 word_delay;
struct list_head transfer_list;
};
{ return 0; }
#endif
-
/* If you're hotplugging an adapter with devices (parport, usb, etc)
* use spi_new_device() to describe each device. You can also call
* spi_unregister_device() to start making that device vanish, but
return list_is_last(&xfer->transfer_list, &ctlr->cur_msg->transfers);
}
+/* OF support code */
+#if IS_ENABLED(CONFIG_OF)
+
+/* must call put_device() when done with returned spi_device device */
+extern struct spi_device *
+of_find_spi_device_by_node(struct device_node *node);
+
+#else
+
+static inline struct spi_device *
+of_find_spi_device_by_node(struct device_node *node)
+{
+ return NULL;
+}
+
+#endif /* IS_ENABLED(CONFIG_OF) */
/* Compatibility layer */
#define spi_master spi_controller
#define SRCU_STATE_SCAN2 2
#define __SRCU_STRUCT_INIT(name, pcpu_name) \
- { \
- .sda = &pcpu_name, \
- .lock = __SPIN_LOCK_UNLOCKED(name.lock), \
- .srcu_gp_seq_needed = 0 - 1, \
- __SRCU_DEP_MAP_INIT(name) \
- }
+{ \
+ .sda = &pcpu_name, \
+ .lock = __SPIN_LOCK_UNLOCKED(name.lock), \
+ .srcu_gp_seq_needed = -1UL, \
+ .work = __DELAYED_WORK_INITIALIZER(name.work, NULL, 0), \
+ __SRCU_DEP_MAP_INIT(name) \
+}
/*
* Define and initialize a srcu struct at build time.
up something else. */
extern asmlinkage void __init start_kernel(void);
+extern void __init arch_call_rest_init(void);
+extern void __ref rest_init(void);
#endif /* _LINUX_START_KERNEL_H */
return unlikely(s2idle_state == S2IDLE_STATE_ENTER);
}
+extern bool pm_suspend_via_s2idle(void);
extern void __init pm_states_init(void);
extern void s2idle_set_ops(const struct platform_s2idle_ops *ops);
extern void s2idle_wake(void);
static inline void pm_set_resume_via_firmware(void) {}
static inline bool pm_suspend_via_firmware(void) { return false; }
static inline bool pm_resume_via_firmware(void) { return false; }
+static inline bool pm_suspend_via_s2idle(void) { return false; }
static inline void suspend_set_ops(const struct platform_suspend_ops *ops) {}
static inline int pm_suspend(suspend_state_t state) { return -ENOSYS; }
int torture_shutdown_init(int ssecs, void (*cleanup)(void));
/* Task stuttering, which forces load/no-load transitions. */
-void stutter_wait(const char *title);
+bool stutter_wait(const char *title);
int torture_stutter_init(int s);
/* Initialization and cleanup. */
struct tracepoint_func __rcu *funcs;
};
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+typedef const int tracepoint_ptr_t;
+#else
+typedef struct tracepoint * const tracepoint_ptr_t;
+#endif
+
struct bpf_raw_event_map {
struct tracepoint *tp;
void *bpf_func;
#define TRACE_DEFINE_ENUM(x)
#define TRACE_DEFINE_SIZEOF(x)
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
+{
+ return offset_to_ptr(p);
+}
+
+#define __TRACEPOINT_ENTRY(name) \
+ asm(" .section \"__tracepoints_ptrs\", \"a\" \n" \
+ " .balign 4 \n" \
+ " .long __tracepoint_" #name " - . \n" \
+ " .previous \n")
+#else
+static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
+{
+ return *p;
+}
+
+#define __TRACEPOINT_ENTRY(name) \
+ static tracepoint_ptr_t __tracepoint_ptr_##name __used \
+ __attribute__((section("__tracepoints_ptrs"))) = \
+ &__tracepoint_##name
+#endif
+
#endif /* _LINUX_TRACEPOINT_H */
/*
return static_key_false(&__tracepoint_##name.key); \
}
-#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
-#define __TRACEPOINT_ENTRY(name) \
- asm(" .section \"__tracepoints_ptrs\", \"a\" \n" \
- " .balign 4 \n" \
- " .long __tracepoint_" #name " - . \n" \
- " .previous \n")
-#else
-#define __TRACEPOINT_ENTRY(name) \
- static struct tracepoint * const __tracepoint_ptr_##name __used \
- __attribute__((section("__tracepoints_ptrs"))) = \
- &__tracepoint_##name
-#endif
-
/*
* We have no guarantee that gcc and the linker won't up-align the tracepoint
* structures, so we create an array of pointers that will be used for iteration
*
* @bio is a part of the writeback in progress controlled by @wbc. Perform
* writeback specific initialization. This is used to apply the cgroup
- * writeback context.
+ * writeback context. Must be called after the bio has been associated with
+ * a device.
*/
static inline void wbc_init_bio(struct writeback_control *wbc, struct bio *bio)
{
* regular writeback instead of writing things out itself.
*/
if (wbc->wb)
- bio_associate_blkcg(bio, wbc->wb->blkcg_css);
+ bio_associate_blkg_from_css(bio, wbc->wb->blkcg_css);
}
#else /* CONFIG_CGROUP_WRITEBACK */
#define DEVLINK_RESOURCE_ID_PARENT_TOP 0
-#define DEVLINK_PARAM_MAX_STRING_VALUE 32
+#define __DEVLINK_PARAM_MAX_STRING_VALUE 32
enum devlink_param_type {
DEVLINK_PARAM_TYPE_U8,
DEVLINK_PARAM_TYPE_U16,
u8 vu8;
u16 vu16;
u32 vu32;
- const char *vstr;
+ char vstr[__DEVLINK_PARAM_MAX_STRING_VALUE];
bool vbool;
};
int devlink_param_driverinit_value_set(struct devlink *devlink, u32 param_id,
union devlink_param_value init_val);
void devlink_param_value_changed(struct devlink *devlink, u32 param_id);
+void devlink_param_value_str_fill(union devlink_param_value *dst_val,
+ const char *src);
struct devlink_region *devlink_region_create(struct devlink *devlink,
const char *region_name,
u32 region_max_snapshots,
{
}
+static inline void
+devlink_param_value_str_fill(union devlink_param_value *dst_val,
+ const char *src)
+{
+}
+
static inline struct devlink_region *
devlink_region_create(struct devlink *devlink,
const char *region_name,
dst->ops->update_pmtu(dst, NULL, skb, mtu);
}
+static inline void skb_tunnel_check_pmtu(struct sk_buff *skb,
+ struct dst_entry *encap_dst,
+ int headroom)
+{
+ u32 encap_mtu = dst_mtu(encap_dst);
+
+ if (skb->len > encap_mtu - headroom)
+ skb_dst_update_pmtu(skb, encap_mtu - headroom);
+}
+
#endif /* _NET_DST_H */
struct rt6_info * __percpu *rt6i_pcpu;
struct rt6_exception_bucket __rcu *rt6i_exception_bucket;
+#ifdef CONFIG_IPV6_ROUTER_PREF
+ unsigned long last_probe;
+#endif
+
u32 fib6_metric;
u8 fib6_protocol;
u8 fib6_type;
int fib_sync_down_dev(struct net_device *dev, unsigned long event, bool force);
int fib_sync_down_addr(struct net_device *dev, __be32 local);
int fib_sync_up(struct net_device *dev, unsigned int nh_flags);
+void fib_sync_mtu(struct net_device *dev, u32 orig_mtu);
#ifdef CONFIG_IP_ROUTE_MULTIPATH
int fib_multipath_hash(const struct net *net, const struct flowi4 *fl4,
__u16 size;
size = ntohs(chunk->chunk_hdr->length);
- size -= sctp_datahdr_len(&chunk->asoc->stream);
+ size -= sctp_datachk_len(&chunk->asoc->stream);
return size;
}
unsigned long sackdelay;
__u32 sackfreq;
+ atomic_t mtu_info;
+
/* When was the last time that we heard from this transport? We use
* this to pick new active and retran paths.
*/
*/
int bman_acquire(struct bman_pool *pool, struct bm_buffer *bufs, u8 num);
+/**
+ * bman_is_probed - Check if bman is probed
+ *
+ * Returns 1 if the bman driver successfully probed, -1 if the bman driver
+ * failed to probe or 0 if the bman driver did not probed yet.
+ */
+int bman_is_probed(void);
+
#endif /* __FSL_BMAN_H */
*/
int qman_release_cgrid(u32 id);
+/**
+ * qman_is_probed - Check if qman is probed
+ *
+ * Returns 1 if the qman driver successfully probed, -1 if the qman driver
+ * failed to probe or 0 if the qman driver did not probed yet.
+ */
+int qman_is_probed(void);
+
#endif /* __FSL_QMAN_H */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM hwmon
+
+#if !defined(_TRACE_HWMON_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_HWMON_H
+
+#include <linux/tracepoint.h>
+
+DECLARE_EVENT_CLASS(hwmon_attr_class,
+
+ TP_PROTO(int index, const char *attr_name, long val),
+
+ TP_ARGS(index, attr_name, val),
+
+ TP_STRUCT__entry(
+ __field(int, index)
+ __string(attr_name, attr_name)
+ __field(long, val)
+ ),
+
+ TP_fast_assign(
+ __entry->index = index;
+ __assign_str(attr_name, attr_name);
+ __entry->val = val;
+ ),
+
+ TP_printk("index=%d, attr_name=%s, val=%ld",
+ __entry->index, __get_str(attr_name), __entry->val)
+);
+
+DEFINE_EVENT(hwmon_attr_class, hwmon_attr_show,
+
+ TP_PROTO(int index, const char *attr_name, long val),
+
+ TP_ARGS(index, attr_name, val)
+);
+
+DEFINE_EVENT(hwmon_attr_class, hwmon_attr_store,
+
+ TP_PROTO(int index, const char *attr_name, long val),
+
+ TP_ARGS(index, attr_name, val)
+);
+
+TRACE_EVENT(hwmon_attr_show_string,
+
+ TP_PROTO(int index, const char *attr_name, const char *s),
+
+ TP_ARGS(index, attr_name, s),
+
+ TP_STRUCT__entry(
+ __field(int, index)
+ __string(attr_name, attr_name)
+ __string(label, s)
+ ),
+
+ TP_fast_assign(
+ __entry->index = index;
+ __assign_str(attr_name, attr_name);
+ __assign_str(label, s);
+ ),
+
+ TP_printk("index=%d, attr_name=%s, val=%s",
+ __entry->index, __get_str(attr_name), __get_str(label))
+);
+
+#endif /* _TRACE_HWMON_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM kyber
+
+#if !defined(_TRACE_KYBER_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_KYBER_H
+
+#include <linux/blkdev.h>
+#include <linux/tracepoint.h>
+
+#define DOMAIN_LEN 16
+#define LATENCY_TYPE_LEN 8
+
+TRACE_EVENT(kyber_latency,
+
+ TP_PROTO(struct request_queue *q, const char *domain, const char *type,
+ unsigned int percentile, unsigned int numerator,
+ unsigned int denominator, unsigned int samples),
+
+ TP_ARGS(q, domain, type, percentile, numerator, denominator, samples),
+
+ TP_STRUCT__entry(
+ __field( dev_t, dev )
+ __array( char, domain, DOMAIN_LEN )
+ __array( char, type, LATENCY_TYPE_LEN )
+ __field( u8, percentile )
+ __field( u8, numerator )
+ __field( u8, denominator )
+ __field( unsigned int, samples )
+ ),
+
+ TP_fast_assign(
+ __entry->dev = disk_devt(dev_to_disk(kobj_to_dev(q->kobj.parent)));
+ strlcpy(__entry->domain, domain, DOMAIN_LEN);
+ strlcpy(__entry->type, type, DOMAIN_LEN);
+ __entry->percentile = percentile;
+ __entry->numerator = numerator;
+ __entry->denominator = denominator;
+ __entry->samples = samples;
+ ),
+
+ TP_printk("%d,%d %s %s p%u %u/%u samples=%u",
+ MAJOR(__entry->dev), MINOR(__entry->dev), __entry->domain,
+ __entry->type, __entry->percentile, __entry->numerator,
+ __entry->denominator, __entry->samples)
+);
+
+TRACE_EVENT(kyber_adjust,
+
+ TP_PROTO(struct request_queue *q, const char *domain,
+ unsigned int depth),
+
+ TP_ARGS(q, domain, depth),
+
+ TP_STRUCT__entry(
+ __field( dev_t, dev )
+ __array( char, domain, DOMAIN_LEN )
+ __field( unsigned int, depth )
+ ),
+
+ TP_fast_assign(
+ __entry->dev = disk_devt(dev_to_disk(kobj_to_dev(q->kobj.parent)));
+ strlcpy(__entry->domain, domain, DOMAIN_LEN);
+ __entry->depth = depth;
+ ),
+
+ TP_printk("%d,%d %s %u",
+ MAJOR(__entry->dev), MINOR(__entry->dev), __entry->domain,
+ __entry->depth)
+);
+
+TRACE_EVENT(kyber_throttled,
+
+ TP_PROTO(struct request_queue *q, const char *domain),
+
+ TP_ARGS(q, domain),
+
+ TP_STRUCT__entry(
+ __field( dev_t, dev )
+ __array( char, domain, DOMAIN_LEN )
+ ),
+
+ TP_fast_assign(
+ __entry->dev = disk_devt(dev_to_disk(kobj_to_dev(q->kobj.parent)));
+ strlcpy(__entry->domain, domain, DOMAIN_LEN);
+ ),
+
+ TP_printk("%d,%d %s", MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->domain)
+);
+
+#define _TRACE_KYBER_H
+#endif /* _TRACE_KYBER_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
* Tracepoint for quiescent states detected by force_quiescent_state().
* These trace events include the type of RCU, the grace-period number
* that was blocked by the CPU, the CPU itself, and the type of quiescent
- * state, which can be "dti" for dyntick-idle mode, "kick" when kicking
- * a CPU that has been in dyntick-idle mode for too long, or "rqc" if the
- * CPU got a quiescent state via its rcu_qs_ctr.
+ * state, which can be "dti" for dyntick-idle mode or "kick" when kicking
+ * a CPU that has been in dyntick-idle mode for too long.
*/
TRACE_EVENT(rcu_fqs,
);
/*
- * Tracepoint for _rcu_barrier() execution. The string "s" describes
- * the _rcu_barrier phase:
- * "Begin": _rcu_barrier() started.
- * "EarlyExit": _rcu_barrier() piggybacked, thus early exit.
- * "Inc1": _rcu_barrier() piggyback check counter incremented.
- * "OfflineNoCB": _rcu_barrier() found callback on never-online CPU
- * "OnlineNoCB": _rcu_barrier() found online no-CBs CPU.
- * "OnlineQ": _rcu_barrier() found online CPU with callbacks.
- * "OnlineNQ": _rcu_barrier() found online CPU, no callbacks.
+ * Tracepoint for rcu_barrier() execution. The string "s" describes
+ * the rcu_barrier phase:
+ * "Begin": rcu_barrier() started.
+ * "EarlyExit": rcu_barrier() piggybacked, thus early exit.
+ * "Inc1": rcu_barrier() piggyback check counter incremented.
+ * "OfflineNoCB": rcu_barrier() found callback on never-online CPU
+ * "OnlineNoCB": rcu_barrier() found online no-CBs CPU.
+ * "OnlineQ": rcu_barrier() found online CPU with callbacks.
+ * "OnlineNQ": rcu_barrier() found online CPU, no callbacks.
* "IRQ": An rcu_barrier_callback() callback posted on remote CPU.
* "IRQNQ": An rcu_barrier_callback() callback found no callbacks.
* "CB": An rcu_barrier_callback() invoked a callback, not the last.
* "LastCB": An rcu_barrier_callback() invoked the last callback.
- * "Inc2": _rcu_barrier() piggyback check counter incremented.
+ * "Inc2": rcu_barrier() piggyback check counter incremented.
* The "cpu" argument is the CPU or -1 if meaningless, the "cnt" argument
* is the count of remaining callbacks, and "done" is the piggybacking count.
*/
TP_fast_assign(
__entry->call = call_id;
memcpy(&__entry->whdr, whdr, sizeof(__entry->whdr));
+ __entry->where = where;
),
TP_printk("c=%08x %08x:%08x:%08x:%04x %08x %08x %02x %02x %s %s",
(__entry->prev_state & (TASK_REPORT_MAX - 1)) ?
__print_flags(__entry->prev_state & (TASK_REPORT_MAX - 1), "|",
- { 0x01, "S" }, { 0x02, "D" }, { 0x04, "T" },
- { 0x08, "t" }, { 0x10, "X" }, { 0x20, "Z" },
- { 0x40, "P" }, { 0x80, "I" }) :
+ { TASK_INTERRUPTIBLE, "S" },
+ { TASK_UNINTERRUPTIBLE, "D" },
+ { __TASK_STOPPED, "T" },
+ { __TASK_TRACED, "t" },
+ { EXIT_DEAD, "X" },
+ { EXIT_ZOMBIE, "Z" },
+ { TASK_PARKED, "P" },
+ { TASK_DEAD, "I" }) :
"R",
__entry->prev_state & TASK_REPORT_MAX ? "+" : "",
/**
* struct gpiohandle_request - Information about a GPIO handle request
- * @lineoffsets: an array desired lines, specified by offset index for the
+ * @lineoffsets: an array of desired lines, specified by offset index for the
* associated GPIO device
* @flags: desired flags for the desired GPIO lines, such as
* GPIOHANDLE_REQUEST_OUTPUT, GPIOHANDLE_REQUEST_ACTIVE_LOW etc, OR:ed
SCTP_SACK_IMMEDIATELY = (1 << 3), /* SACK should be sent without delay. */
/* 2 bits here have been used by SCTP_PR_SCTP_MASK */
SCTP_SENDALL = (1 << 6),
+ SCTP_PR_SCTP_ALL = (1 << 7),
SCTP_NOTIFICATION = MSG_NOTIFICATION, /* Next message is not user msg but notification. */
SCTP_EOF = MSG_FIN, /* Initiate graceful shutdown process. */
};
* on the internal clcsock, and more SMC-related socket data
*/
struct smc_diag_msg {
- __u8 diag_family;
- __u8 diag_state;
- __u8 diag_mode;
- __u8 diag_shutdown;
+ __u8 diag_family;
+ __u8 diag_state;
+ union {
+ __u8 diag_mode;
+ __u8 diag_fallback; /* the old name of the field */
+ };
+ __u8 diag_shutdown;
struct inet_diag_sockid id;
- __u32 diag_uid;
- __u64 diag_inode;
+ __u32 diag_uid;
+ __aligned_u64 diag_inode;
};
/* Mode of a connection */
};
struct smcd_diag_dmbinfo { /* SMC-D Socket internals */
- __u32 linkid; /* Link identifier */
- __u64 peer_gid; /* Peer GID */
- __u64 my_gid; /* My GID */
- __u64 token; /* Token of DMB */
- __u64 peer_token; /* Token of remote DMBE */
+ __u32 linkid; /* Link identifier */
+ __aligned_u64 peer_gid; /* Peer GID */
+ __aligned_u64 my_gid; /* My GID */
+ __aligned_u64 token; /* Token of DMB */
+ __aligned_u64 peer_token; /* Token of remote DMBE */
};
#endif /* _UAPI_SMC_DIAG_H_ */
#define UDP_ENCAP_L2TPINUDP 3 /* rfc2661 */
#define UDP_ENCAP_GTP0 4 /* GSM TS 09.60 */
#define UDP_ENCAP_GTP1U 5 /* 3GPP TS 29.060 */
+#define UDP_ENCAP_RXRPC 6
#endif /* _UAPI_LINUX_UDP_H */
#define xen_initial_domain() (0)
#endif /* CONFIG_XEN_DOM0 */
+struct bio_vec;
+bool xen_biovec_phys_mergeable(const struct bio_vec *vec1,
+ const struct bio_vec *vec2);
+
#endif /* _XEN_XEN_H */
If in doubt, say N here.
+config HAVE_SCHED_AVG_IRQ
+ def_bool y
+ depends on IRQ_TIME_ACCOUNTING || PARAVIRT_TIME_ACCOUNTING
+ depends on SMP
+
config BSD_PROCESS_ACCT
bool "BSD Process Accounting"
depends on MULTIUSER
static __initdata DECLARE_COMPLETION(kthreadd_done);
-static noinline void __ref rest_init(void)
+noinline void __ref rest_init(void)
{
struct task_struct *tsk;
int pid;
pti_init();
}
+void __init __weak arch_call_rest_init(void)
+{
+ rest_init();
+}
+
asmlinkage __visible void __init start_kernel(void)
{
char *command_line;
}
/* Do the rest non-__init'ed, we're now alive */
- rest_init();
+ arch_call_rest_init();
}
/* Call all constructor functions linked into the kernel. */
/* need to finish all async __init code before freeing the memory */
async_synchronize_full();
ftrace_free_init_mem();
- jump_label_invalidate_initmem();
free_initmem();
mark_readonly();
sock_hold(sock->sk);
old_xs = xchg(&m->xsk_map[i], xs);
- if (old_xs) {
- /* Make sure we've flushed everything. */
- synchronize_net();
+ if (old_xs)
sock_put((struct sock *)old_xs);
- }
sockfd_put(sock);
return 0;
return -EINVAL;
old_xs = xchg(&m->xsk_map[k], NULL);
- if (old_xs) {
- /* Make sure we've flushed everything. */
- synchronize_net();
+ if (old_xs)
sock_put((struct sock *)old_xs);
- }
return 0;
}
}
/**
- * cgroup_e_css - obtain a cgroup's effective css for the specified subsystem
+ * cgroup_e_css_by_mask - obtain a cgroup's effective css for the specified ss
* @cgrp: the cgroup of interest
* @ss: the subsystem of interest (%NULL returns @cgrp->self)
*
* enabled. If @ss is associated with the hierarchy @cgrp is on, this
* function is guaranteed to return non-NULL css.
*/
-static struct cgroup_subsys_state *cgroup_e_css(struct cgroup *cgrp,
- struct cgroup_subsys *ss)
+static struct cgroup_subsys_state *cgroup_e_css_by_mask(struct cgroup *cgrp,
+ struct cgroup_subsys *ss)
{
lockdep_assert_held(&cgroup_mutex);
return cgroup_css(cgrp, ss);
}
+/**
+ * cgroup_e_css - obtain a cgroup's effective css for the specified subsystem
+ * @cgrp: the cgroup of interest
+ * @ss: the subsystem of interest
+ *
+ * Find and get the effective css of @cgrp for @ss. The effective css is
+ * defined as the matching css of the nearest ancestor including self which
+ * has @ss enabled. If @ss is not mounted on the hierarchy @cgrp is on,
+ * the root css is returned, so this function always returns a valid css.
+ *
+ * The returned css is not guaranteed to be online, and therefore it is the
+ * callers responsiblity to tryget a reference for it.
+ */
+struct cgroup_subsys_state *cgroup_e_css(struct cgroup *cgrp,
+ struct cgroup_subsys *ss)
+{
+ struct cgroup_subsys_state *css;
+
+ do {
+ css = cgroup_css(cgrp, ss);
+
+ if (css)
+ return css;
+ cgrp = cgroup_parent(cgrp);
+ } while (cgrp);
+
+ return init_css_set.subsys[ss->id];
+}
+
/**
* cgroup_get_e_css - get a cgroup's effective css for the specified subsystem
* @cgrp: the cgroup of interest
*
* Should be called under cgroup_[tree_]mutex.
*/
-#define for_each_e_css(css, ssid, cgrp) \
- for ((ssid) = 0; (ssid) < CGROUP_SUBSYS_COUNT; (ssid)++) \
- if (!((css) = cgroup_e_css(cgrp, cgroup_subsys[(ssid)]))) \
- ; \
+#define for_each_e_css(css, ssid, cgrp) \
+ for ((ssid) = 0; (ssid) < CGROUP_SUBSYS_COUNT; (ssid)++) \
+ if (!((css) = cgroup_e_css_by_mask(cgrp, \
+ cgroup_subsys[(ssid)]))) \
+ ; \
else
/**
* @ss is in this hierarchy, so we want the
* effective css from @cgrp.
*/
- template[i] = cgroup_e_css(cgrp, ss);
+ template[i] = cgroup_e_css_by_mask(cgrp, ss);
} else {
/*
* @ss is not in this hierarchy, so we don't want
}
/**
- * cgroup_save_control - save control masks of a subtree
+ * cgroup_save_control - save control masks and dom_cgrp of a subtree
* @cgrp: root of the target subtree
*
- * Save ->subtree_control and ->subtree_ss_mask to the respective old_
- * prefixed fields for @cgrp's subtree including @cgrp itself.
+ * Save ->subtree_control, ->subtree_ss_mask and ->dom_cgrp to the
+ * respective old_ prefixed fields for @cgrp's subtree including @cgrp
+ * itself.
*/
static void cgroup_save_control(struct cgroup *cgrp)
{
cgroup_for_each_live_descendant_pre(dsct, d_css, cgrp) {
dsct->old_subtree_control = dsct->subtree_control;
dsct->old_subtree_ss_mask = dsct->subtree_ss_mask;
+ dsct->old_dom_cgrp = dsct->dom_cgrp;
}
}
}
/**
- * cgroup_restore_control - restore control masks of a subtree
+ * cgroup_restore_control - restore control masks and dom_cgrp of a subtree
* @cgrp: root of the target subtree
*
- * Restore ->subtree_control and ->subtree_ss_mask from the respective old_
- * prefixed fields for @cgrp's subtree including @cgrp itself.
+ * Restore ->subtree_control, ->subtree_ss_mask and ->dom_cgrp from the
+ * respective old_ prefixed fields for @cgrp's subtree including @cgrp
+ * itself.
*/
static void cgroup_restore_control(struct cgroup *cgrp)
{
cgroup_for_each_live_descendant_post(dsct, d_css, cgrp) {
dsct->subtree_control = dsct->old_subtree_control;
dsct->subtree_ss_mask = dsct->old_subtree_ss_mask;
+ dsct->dom_cgrp = dsct->old_dom_cgrp;
}
}
return ret;
/*
- * At this point, cgroup_e_css() results reflect the new csses
+ * At this point, cgroup_e_css_by_mask() results reflect the new csses
* making the following cgroup_update_dfl_csses() properly update
* css associations of all tasks in the subtree.
*/
{
struct cgroup *parent = cgroup_parent(cgrp);
struct cgroup *dom_cgrp = parent->dom_cgrp;
+ struct cgroup *dsct;
+ struct cgroup_subsys_state *d_css;
int ret;
lockdep_assert_held(&cgroup_mutex);
*/
cgroup_save_control(cgrp);
- cgrp->dom_cgrp = dom_cgrp;
+ cgroup_for_each_live_descendant_pre(dsct, d_css, cgrp)
+ if (dsct == cgrp || cgroup_is_threaded(dsct))
+ dsct->dom_cgrp = dom_cgrp;
+
ret = cgroup_apply_control(cgrp);
if (!ret)
parent->nr_threaded_children++;
- else
- cgrp->dom_cgrp = cgrp;
cgroup_finalize_control(cgrp, ret);
return ret;
percpu_rwsem_assert_held(&cpu_hotplug_lock);
}
+static void lockdep_acquire_cpus_lock(void)
+{
+ rwsem_acquire(&cpu_hotplug_lock.rw_sem.dep_map, 0, 0, _THIS_IP_);
+}
+
+static void lockdep_release_cpus_lock(void)
+{
+ rwsem_release(&cpu_hotplug_lock.rw_sem.dep_map, 1, _THIS_IP_);
+}
+
/*
* Wait for currently running CPU hotplug operations to complete (if any) and
* disable future CPU hotplug (from sysfs). The 'cpu_add_remove_lock' protects
cpu_maps_update_done();
}
EXPORT_SYMBOL_GPL(cpu_hotplug_enable);
+
+#else
+
+static void lockdep_acquire_cpus_lock(void)
+{
+}
+
+static void lockdep_release_cpus_lock(void)
+{
+}
+
#endif /* CONFIG_HOTPLUG_CPU */
#ifdef CONFIG_HOTPLUG_SMT
pr_info("SMT: Force disabled\n");
cpu_smt_control = CPU_SMT_FORCE_DISABLED;
} else {
+ pr_info("SMT: disabled\n");
cpu_smt_control = CPU_SMT_DISABLED;
}
}
*/
smp_mb();
+ /*
+ * The BP holds the hotplug lock, but we're now running on the AP,
+ * ensure that anybody asserting the lock is held, will actually find
+ * it so.
+ */
+ lockdep_acquire_cpus_lock();
cpuhp_lock_acquire(bringup);
if (st->single) {
}
cpuhp_lock_release(bringup);
+ lockdep_release_cpus_lock();
if (!st->should_run)
complete_ap_thread(st, bringup);
config ARCH_DMA_ADDR_T_64BIT
def_bool 64BIT || PHYS_ADDR_T_64BIT
+config ARCH_HAS_DMA_COHERENCE_H
+ bool
+
config HAVE_GENERIC_DMA_COHERENT
bool
config ARCH_HAS_SYNC_DMA_FOR_CPU_ALL
bool
-config DMA_DIRECT_OPS
+config ARCH_HAS_DMA_COHERENT_TO_PFN
bool
- depends on HAS_DMA
-config DMA_NONCOHERENT_OPS
+config ARCH_HAS_DMA_MMAP_PGPROT
bool
- depends on HAS_DMA
- select DMA_DIRECT_OPS
-config DMA_NONCOHERENT_MMAP
+config DMA_DIRECT_OPS
bool
- depends on DMA_NONCOHERENT_OPS
+ depends on HAS_DMA
config DMA_NONCOHERENT_CACHE_SYNC
bool
- depends on DMA_NONCOHERENT_OPS
+ depends on DMA_DIRECT_OPS
config DMA_VIRT_OPS
bool
obj-$(CONFIG_DMA_CMA) += contiguous.o
obj-$(CONFIG_HAVE_GENERIC_DMA_COHERENT) += coherent.o
obj-$(CONFIG_DMA_DIRECT_OPS) += direct.o
-obj-$(CONFIG_DMA_NONCOHERENT_OPS) += noncoherent.o
obj-$(CONFIG_DMA_VIRT_OPS) += virt.o
obj-$(CONFIG_DMA_API_DEBUG) += debug.o
obj-$(CONFIG_SWIOTLB) += swiotlb.o
static int __init early_cma(char *p)
{
- pr_debug("%s(%s)\n", __func__, p);
+ if (!p) {
+ pr_err("Config string not provided\n");
+ return -EINVAL;
+ }
+
size_cmdline = memparse(p, &p);
if (*p != '@')
return 0;
#endif
}
+void debug_dma_map_single(struct device *dev, const void *addr,
+ unsigned long len)
+{
+ if (unlikely(dma_debug_disabled()))
+ return;
+
+ if (!virt_addr_valid(addr))
+ err_printk(dev, NULL, "DMA-API: device driver maps memory from invalid area [addr=%p] [len=%lu]\n",
+ addr, len);
+
+ if (is_vmalloc_addr(addr))
+ err_printk(dev, NULL, "DMA-API: device driver maps memory from vmalloc area [addr=%p] [len=%lu]\n",
+ addr, len);
+}
+EXPORT_SYMBOL(debug_dma_map_single);
+
void debug_dma_map_page(struct device *dev, struct page *page, size_t offset,
size_t size, int direction, dma_addr_t dma_addr,
bool map_single)
// SPDX-License-Identifier: GPL-2.0
/*
- * DMA operations that map physical memory directly without using an IOMMU or
- * flushing caches.
+ * Copyright (C) 2018 Christoph Hellwig.
+ *
+ * DMA operations that map physical memory directly without using an IOMMU.
*/
+#include <linux/bootmem.h> /* for max_pfn */
#include <linux/export.h>
#include <linux/mm.h>
#include <linux/dma-direct.h>
#include <linux/scatterlist.h>
#include <linux/dma-contiguous.h>
+#include <linux/dma-noncoherent.h>
#include <linux/pfn.h>
#include <linux/set_memory.h>
return false;
}
- if (*dev->dma_mask >= DMA_BIT_MASK(32)) {
+ if (*dev->dma_mask >= DMA_BIT_MASK(32) || dev->bus_dma_mask) {
dev_err(dev,
- "%s: overflow %pad+%zu of device mask %llx\n",
- caller, &dma_addr, size, *dev->dma_mask);
+ "%s: overflow %pad+%zu of device mask %llx bus mask %llx\n",
+ caller, &dma_addr, size,
+ *dev->dma_mask, dev->bus_dma_mask);
}
return false;
}
return true;
}
+static inline dma_addr_t phys_to_dma_direct(struct device *dev,
+ phys_addr_t phys)
+{
+ if (force_dma_unencrypted())
+ return __phys_to_dma(dev, phys);
+ return phys_to_dma(dev, phys);
+}
+
+u64 dma_direct_get_required_mask(struct device *dev)
+{
+ u64 max_dma = phys_to_dma_direct(dev, (max_pfn - 1) << PAGE_SHIFT);
+
+ if (dev->bus_dma_mask && dev->bus_dma_mask < max_dma)
+ max_dma = dev->bus_dma_mask;
+
+ return (1ULL << (fls64(max_dma) - 1)) * 2 - 1;
+}
+
+static gfp_t __dma_direct_optimal_gfp_mask(struct device *dev, u64 dma_mask,
+ u64 *phys_mask)
+{
+ if (dev->bus_dma_mask && dev->bus_dma_mask < dma_mask)
+ dma_mask = dev->bus_dma_mask;
+
+ if (force_dma_unencrypted())
+ *phys_mask = __dma_to_phys(dev, dma_mask);
+ else
+ *phys_mask = dma_to_phys(dev, dma_mask);
+
+ /*
+ * Optimistically try the zone that the physical address mask falls
+ * into first. If that returns memory that isn't actually addressable
+ * we will fallback to the next lower zone and try again.
+ *
+ * Note that GFP_DMA32 and GFP_DMA are no ops without the corresponding
+ * zones.
+ */
+ if (*phys_mask <= DMA_BIT_MASK(ARCH_ZONE_DMA_BITS))
+ return GFP_DMA;
+ if (*phys_mask <= DMA_BIT_MASK(32))
+ return GFP_DMA32;
+ return 0;
+}
+
static bool dma_coherent_ok(struct device *dev, phys_addr_t phys, size_t size)
{
- dma_addr_t addr = force_dma_unencrypted() ?
- __phys_to_dma(dev, phys) : phys_to_dma(dev, phys);
- return addr + size - 1 <= dev->coherent_dma_mask;
+ return phys_to_dma_direct(dev, phys) + size - 1 <=
+ min_not_zero(dev->coherent_dma_mask, dev->bus_dma_mask);
}
-void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
- gfp_t gfp, unsigned long attrs)
+void *dma_direct_alloc_pages(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs)
{
unsigned int count = PAGE_ALIGN(size) >> PAGE_SHIFT;
int page_order = get_order(size);
struct page *page = NULL;
+ u64 phys_mask;
void *ret;
+ if (attrs & DMA_ATTR_NO_WARN)
+ gfp |= __GFP_NOWARN;
+
/* we always manually zero the memory once we are done: */
gfp &= ~__GFP_ZERO;
-
- /* GFP_DMA32 and GFP_DMA are no ops without the corresponding zones: */
- if (dev->coherent_dma_mask <= DMA_BIT_MASK(ARCH_ZONE_DMA_BITS))
- gfp |= GFP_DMA;
- if (dev->coherent_dma_mask <= DMA_BIT_MASK(32) && !(gfp & GFP_DMA))
- gfp |= GFP_DMA32;
-
+ gfp |= __dma_direct_optimal_gfp_mask(dev, dev->coherent_dma_mask,
+ &phys_mask);
again:
/* CMA can be used only in the context which permits sleeping */
if (gfpflags_allow_blocking(gfp)) {
page = NULL;
if (IS_ENABLED(CONFIG_ZONE_DMA32) &&
- dev->coherent_dma_mask < DMA_BIT_MASK(64) &&
+ phys_mask < DMA_BIT_MASK(64) &&
!(gfp & (GFP_DMA32 | GFP_DMA))) {
gfp |= GFP_DMA32;
goto again;
}
if (IS_ENABLED(CONFIG_ZONE_DMA) &&
- dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
- !(gfp & GFP_DMA)) {
+ phys_mask < DMA_BIT_MASK(32) && !(gfp & GFP_DMA)) {
gfp = (gfp & ~GFP_DMA32) | GFP_DMA;
goto again;
}
* NOTE: this function must never look at the dma_addr argument, because we want
* to be able to use it as a helper for iommu implementations as well.
*/
-void dma_direct_free(struct device *dev, size_t size, void *cpu_addr,
+void dma_direct_free_pages(struct device *dev, size_t size, void *cpu_addr,
dma_addr_t dma_addr, unsigned long attrs)
{
unsigned int count = PAGE_ALIGN(size) >> PAGE_SHIFT;
free_pages((unsigned long)cpu_addr, page_order);
}
+void *dma_direct_alloc(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs)
+{
+ if (!dev_is_dma_coherent(dev))
+ return arch_dma_alloc(dev, size, dma_handle, gfp, attrs);
+ return dma_direct_alloc_pages(dev, size, dma_handle, gfp, attrs);
+}
+
+void dma_direct_free(struct device *dev, size_t size,
+ void *cpu_addr, dma_addr_t dma_addr, unsigned long attrs)
+{
+ if (!dev_is_dma_coherent(dev))
+ arch_dma_free(dev, size, cpu_addr, dma_addr, attrs);
+ else
+ dma_direct_free_pages(dev, size, cpu_addr, dma_addr, attrs);
+}
+
+static void dma_direct_sync_single_for_device(struct device *dev,
+ dma_addr_t addr, size_t size, enum dma_data_direction dir)
+{
+ if (dev_is_dma_coherent(dev))
+ return;
+ arch_sync_dma_for_device(dev, dma_to_phys(dev, addr), size, dir);
+}
+
+static void dma_direct_sync_sg_for_device(struct device *dev,
+ struct scatterlist *sgl, int nents, enum dma_data_direction dir)
+{
+ struct scatterlist *sg;
+ int i;
+
+ if (dev_is_dma_coherent(dev))
+ return;
+
+ for_each_sg(sgl, sg, nents, i)
+ arch_sync_dma_for_device(dev, sg_phys(sg), sg->length, dir);
+}
+
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+ defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+static void dma_direct_sync_single_for_cpu(struct device *dev,
+ dma_addr_t addr, size_t size, enum dma_data_direction dir)
+{
+ if (dev_is_dma_coherent(dev))
+ return;
+ arch_sync_dma_for_cpu(dev, dma_to_phys(dev, addr), size, dir);
+ arch_sync_dma_for_cpu_all(dev);
+}
+
+static void dma_direct_sync_sg_for_cpu(struct device *dev,
+ struct scatterlist *sgl, int nents, enum dma_data_direction dir)
+{
+ struct scatterlist *sg;
+ int i;
+
+ if (dev_is_dma_coherent(dev))
+ return;
+
+ for_each_sg(sgl, sg, nents, i)
+ arch_sync_dma_for_cpu(dev, sg_phys(sg), sg->length, dir);
+ arch_sync_dma_for_cpu_all(dev);
+}
+
+static void dma_direct_unmap_page(struct device *dev, dma_addr_t addr,
+ size_t size, enum dma_data_direction dir, unsigned long attrs)
+{
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ dma_direct_sync_single_for_cpu(dev, addr, size, dir);
+}
+
+static void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl,
+ int nents, enum dma_data_direction dir, unsigned long attrs)
+{
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ dma_direct_sync_sg_for_cpu(dev, sgl, nents, dir);
+}
+#endif
+
dma_addr_t dma_direct_map_page(struct device *dev, struct page *page,
unsigned long offset, size_t size, enum dma_data_direction dir,
unsigned long attrs)
{
- dma_addr_t dma_addr = phys_to_dma(dev, page_to_phys(page)) + offset;
+ phys_addr_t phys = page_to_phys(page) + offset;
+ dma_addr_t dma_addr = phys_to_dma(dev, phys);
if (!check_addr(dev, dma_addr, size, __func__))
return DIRECT_MAPPING_ERROR;
+
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ dma_direct_sync_single_for_device(dev, dma_addr, size, dir);
return dma_addr;
}
sg_dma_len(sg) = sg->length;
}
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ dma_direct_sync_sg_for_device(dev, sgl, nents, dir);
return nents;
}
+/*
+ * Because 32-bit DMA masks are so common we expect every architecture to be
+ * able to satisfy them - either by not supporting more physical memory, or by
+ * providing a ZONE_DMA32. If neither is the case, the architecture needs to
+ * use an IOMMU instead of the direct mapping.
+ */
int dma_direct_supported(struct device *dev, u64 mask)
{
-#ifdef CONFIG_ZONE_DMA
- if (mask < phys_to_dma(dev, DMA_BIT_MASK(ARCH_ZONE_DMA_BITS)))
- return 0;
-#else
- /*
- * Because 32-bit DMA masks are so common we expect every architecture
- * to be able to satisfy them - either by not supporting more physical
- * memory, or by providing a ZONE_DMA32. If neither is the case, the
- * architecture needs to use an IOMMU instead of the direct mapping.
- */
- if (mask < phys_to_dma(dev, DMA_BIT_MASK(32)))
- return 0;
-#endif
- /*
- * Upstream PCI/PCIe bridges or SoC interconnects may not carry
- * as many DMA address bits as the device itself supports.
- */
- if (dev->bus_dma_mask && mask > dev->bus_dma_mask)
- return 0;
- return 1;
+ u64 min_mask;
+
+ if (IS_ENABLED(CONFIG_ZONE_DMA))
+ min_mask = DMA_BIT_MASK(ARCH_ZONE_DMA_BITS);
+ else
+ min_mask = DMA_BIT_MASK(32);
+
+ min_mask = min_t(u64, min_mask, (max_pfn - 1) << PAGE_SHIFT);
+
+ return mask >= phys_to_dma(dev, min_mask);
}
int dma_direct_mapping_error(struct device *dev, dma_addr_t dma_addr)
.free = dma_direct_free,
.map_page = dma_direct_map_page,
.map_sg = dma_direct_map_sg,
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE)
+ .sync_single_for_device = dma_direct_sync_single_for_device,
+ .sync_sg_for_device = dma_direct_sync_sg_for_device,
+#endif
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+ defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+ .sync_single_for_cpu = dma_direct_sync_single_for_cpu,
+ .sync_sg_for_cpu = dma_direct_sync_sg_for_cpu,
+ .unmap_page = dma_direct_unmap_page,
+ .unmap_sg = dma_direct_unmap_sg,
+#endif
+ .get_required_mask = dma_direct_get_required_mask,
.dma_supported = dma_direct_supported,
.mapping_error = dma_direct_mapping_error,
+ .cache_sync = arch_dma_cache_sync,
};
EXPORT_SYMBOL(dma_direct_ops);
*/
#include <linux/acpi.h>
-#include <linux/dma-mapping.h>
+#include <linux/dma-noncoherent.h>
#include <linux/export.h>
#include <linux/gfp.h>
#include <linux/of_device.h>
* Create scatter-list for the already allocated DMA buffer.
*/
int dma_common_get_sgtable(struct device *dev, struct sg_table *sgt,
- void *cpu_addr, dma_addr_t handle, size_t size)
+ void *cpu_addr, dma_addr_t dma_addr, size_t size,
+ unsigned long attrs)
{
- struct page *page = virt_to_page(cpu_addr);
+ struct page *page;
int ret;
- ret = sg_alloc_table(sgt, 1, GFP_KERNEL);
- if (unlikely(ret))
- return ret;
+ if (!dev_is_dma_coherent(dev)) {
+ if (!IS_ENABLED(CONFIG_ARCH_HAS_DMA_COHERENT_TO_PFN))
+ return -ENXIO;
- sg_set_page(sgt->sgl, page, PAGE_ALIGN(size), 0);
- return 0;
+ page = pfn_to_page(arch_dma_coherent_to_pfn(dev, cpu_addr,
+ dma_addr));
+ } else {
+ page = virt_to_page(cpu_addr);
+ }
+
+ ret = sg_alloc_table(sgt, 1, GFP_KERNEL);
+ if (!ret)
+ sg_set_page(sgt->sgl, page, PAGE_ALIGN(size), 0);
+ return ret;
}
EXPORT_SYMBOL(dma_common_get_sgtable);
* Create userspace mapping for the DMA-coherent memory.
*/
int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,
- void *cpu_addr, dma_addr_t dma_addr, size_t size)
+ void *cpu_addr, dma_addr_t dma_addr, size_t size,
+ unsigned long attrs)
{
- int ret = -ENXIO;
#ifndef CONFIG_ARCH_NO_COHERENT_DMA_MMAP
unsigned long user_count = vma_pages(vma);
unsigned long count = PAGE_ALIGN(size) >> PAGE_SHIFT;
unsigned long off = vma->vm_pgoff;
+ unsigned long pfn;
+ int ret = -ENXIO;
- vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
+ vma->vm_page_prot = arch_dma_mmap_pgprot(dev, vma->vm_page_prot, attrs);
if (dma_mmap_from_dev_coherent(dev, vma, cpu_addr, size, &ret))
return ret;
- if (off < count && user_count <= (count - off))
- ret = remap_pfn_range(vma, vma->vm_start,
- page_to_pfn(virt_to_page(cpu_addr)) + off,
- user_count << PAGE_SHIFT,
- vma->vm_page_prot);
-#endif /* !CONFIG_ARCH_NO_COHERENT_DMA_MMAP */
+ if (off >= count || user_count > count - off)
+ return -ENXIO;
- return ret;
+ if (!dev_is_dma_coherent(dev)) {
+ if (!IS_ENABLED(CONFIG_ARCH_HAS_DMA_COHERENT_TO_PFN))
+ return -ENXIO;
+ pfn = arch_dma_coherent_to_pfn(dev, cpu_addr, dma_addr);
+ } else {
+ pfn = page_to_pfn(virt_to_page(cpu_addr));
+ }
+
+ return remap_pfn_range(vma, vma->vm_start, pfn + vma->vm_pgoff,
+ user_count << PAGE_SHIFT, vma->vm_page_prot);
+#else
+ return -ENXIO;
+#endif /* !CONFIG_ARCH_NO_COHERENT_DMA_MMAP */
}
EXPORT_SYMBOL(dma_common_mmap);
vunmap(cpu_addr);
}
#endif
-
-/*
- * enables DMA API use for a device
- */
-int dma_configure(struct device *dev)
-{
- if (dev->bus->dma_configure)
- return dev->bus->dma_configure(dev);
- return 0;
-}
-
-void dma_deconfigure(struct device *dev)
-{
- of_dma_deconfigure(dev);
- acpi_dma_deconfigure(dev);
-}
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Copyright (C) 2018 Christoph Hellwig.
- *
- * DMA operations that map physical memory directly without providing cache
- * coherence.
- */
-#include <linux/export.h>
-#include <linux/mm.h>
-#include <linux/dma-direct.h>
-#include <linux/dma-noncoherent.h>
-#include <linux/scatterlist.h>
-
-static void dma_noncoherent_sync_single_for_device(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir)
-{
- arch_sync_dma_for_device(dev, dma_to_phys(dev, addr), size, dir);
-}
-
-static void dma_noncoherent_sync_sg_for_device(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir)
-{
- struct scatterlist *sg;
- int i;
-
- for_each_sg(sgl, sg, nents, i)
- arch_sync_dma_for_device(dev, sg_phys(sg), sg->length, dir);
-}
-
-static dma_addr_t dma_noncoherent_map_page(struct device *dev, struct page *page,
- unsigned long offset, size_t size, enum dma_data_direction dir,
- unsigned long attrs)
-{
- dma_addr_t addr;
-
- addr = dma_direct_map_page(dev, page, offset, size, dir, attrs);
- if (!dma_mapping_error(dev, addr) && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
- arch_sync_dma_for_device(dev, page_to_phys(page) + offset,
- size, dir);
- return addr;
-}
-
-static int dma_noncoherent_map_sg(struct device *dev, struct scatterlist *sgl,
- int nents, enum dma_data_direction dir, unsigned long attrs)
-{
- nents = dma_direct_map_sg(dev, sgl, nents, dir, attrs);
- if (nents > 0 && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
- dma_noncoherent_sync_sg_for_device(dev, sgl, nents, dir);
- return nents;
-}
-
-#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
- defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
-static void dma_noncoherent_sync_single_for_cpu(struct device *dev,
- dma_addr_t addr, size_t size, enum dma_data_direction dir)
-{
- arch_sync_dma_for_cpu(dev, dma_to_phys(dev, addr), size, dir);
- arch_sync_dma_for_cpu_all(dev);
-}
-
-static void dma_noncoherent_sync_sg_for_cpu(struct device *dev,
- struct scatterlist *sgl, int nents, enum dma_data_direction dir)
-{
- struct scatterlist *sg;
- int i;
-
- for_each_sg(sgl, sg, nents, i)
- arch_sync_dma_for_cpu(dev, sg_phys(sg), sg->length, dir);
- arch_sync_dma_for_cpu_all(dev);
-}
-
-static void dma_noncoherent_unmap_page(struct device *dev, dma_addr_t addr,
- size_t size, enum dma_data_direction dir, unsigned long attrs)
-{
- if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
- dma_noncoherent_sync_single_for_cpu(dev, addr, size, dir);
-}
-
-static void dma_noncoherent_unmap_sg(struct device *dev, struct scatterlist *sgl,
- int nents, enum dma_data_direction dir, unsigned long attrs)
-{
- if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
- dma_noncoherent_sync_sg_for_cpu(dev, sgl, nents, dir);
-}
-#endif
-
-const struct dma_map_ops dma_noncoherent_ops = {
- .alloc = arch_dma_alloc,
- .free = arch_dma_free,
- .mmap = arch_dma_mmap,
- .sync_single_for_device = dma_noncoherent_sync_single_for_device,
- .sync_sg_for_device = dma_noncoherent_sync_sg_for_device,
- .map_page = dma_noncoherent_map_page,
- .map_sg = dma_noncoherent_map_sg,
-#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
- defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
- .sync_single_for_cpu = dma_noncoherent_sync_single_for_cpu,
- .sync_sg_for_cpu = dma_noncoherent_sync_sg_for_cpu,
- .unmap_page = dma_noncoherent_unmap_page,
- .unmap_sg = dma_noncoherent_unmap_sg,
-#endif
- .dma_supported = dma_direct_supported,
- .mapping_error = dma_direct_mapping_error,
- .cache_sync = arch_dma_cache_sync,
-};
-EXPORT_SYMBOL(dma_noncoherent_ops);
if (size || handle->aux_flags) {
/*
* Only send RECORD_AUX if we have something useful to communicate
+ *
+ * Note: the OVERWRITE records by themselves are not considered
+ * useful, as they don't communicate any *new* information,
+ * aside from the short-lived offset, that becomes history at
+ * the next event sched-in and therefore isn't useful.
+ * The userspace that needs to copy out AUX data in overwrite
+ * mode should know to use user_page::aux_head for the actual
+ * offset. So, from now on we don't output AUX records that
+ * have *only* OVERWRITE flag set.
*/
- perf_event_aux_event(handle->event, aux_head, size,
- handle->aux_flags);
+ if (handle->aux_flags & ~(u64)PERF_AUX_FLAG_OVERWRITE)
+ perf_event_aux_event(handle->event, aux_head, size,
+ handle->aux_flags);
}
rb->user_page->aux_head = rb->aux_head;
{
struct futex_hash_bucket *hb;
- if (WARN_ON_SMP(!q->lock_ptr || !spin_is_locked(q->lock_ptr))
- || WARN_ON(plist_node_empty(&q->list)))
+ if (WARN_ON_SMP(!q->lock_ptr) || WARN_ON(plist_node_empty(&q->list)))
return;
+ lockdep_assert_held(q->lock_ptr);
hb = container_of(q->lock_ptr, struct futex_hash_bucket, lock);
plist_del(&q->list, &hb->chain);
return area;
}
+/* Find the best CPU which has the lowest vector allocation count */
+static unsigned int matrix_find_best_cpu(struct irq_matrix *m,
+ const struct cpumask *msk)
+{
+ unsigned int cpu, best_cpu, maxavl = 0;
+ struct cpumap *cm;
+
+ best_cpu = UINT_MAX;
+
+ for_each_cpu(cpu, msk) {
+ cm = per_cpu_ptr(m->maps, cpu);
+
+ if (!cm->online || cm->available <= maxavl)
+ continue;
+
+ best_cpu = cpu;
+ maxavl = cm->available;
+ }
+ return best_cpu;
+}
+
/**
* irq_matrix_assign_system - Assign system wide entry in the matrix
* @m: Matrix pointer
* @m: Matrix pointer
* @cpu: On which CPU the interrupt should be allocated
*/
-int irq_matrix_alloc_managed(struct irq_matrix *m, unsigned int cpu)
+int irq_matrix_alloc_managed(struct irq_matrix *m, const struct cpumask *msk,
+ unsigned int *mapped_cpu)
{
- struct cpumap *cm = per_cpu_ptr(m->maps, cpu);
- unsigned int bit, end = m->alloc_end;
+ unsigned int bit, cpu, end = m->alloc_end;
+ struct cpumap *cm;
+
+ if (cpumask_empty(msk))
+ return -EINVAL;
+
+ cpu = matrix_find_best_cpu(m, msk);
+ if (cpu == UINT_MAX)
+ return -ENOSPC;
+ cm = per_cpu_ptr(m->maps, cpu);
+ end = m->alloc_end;
/* Get managed bit which are not allocated */
bitmap_andnot(m->scratch_map, cm->managed_map, cm->alloc_map, end);
bit = find_first_bit(m->scratch_map, end);
set_bit(bit, cm->alloc_map);
cm->allocated++;
m->total_allocated++;
+ *mapped_cpu = cpu;
trace_irq_matrix_alloc_managed(bit, cpu, m, cm);
return bit;
}
int irq_matrix_alloc(struct irq_matrix *m, const struct cpumask *msk,
bool reserved, unsigned int *mapped_cpu)
{
- unsigned int cpu, best_cpu, maxavl = 0;
+ unsigned int cpu, bit;
struct cpumap *cm;
- unsigned int bit;
-
- best_cpu = UINT_MAX;
- for_each_cpu(cpu, msk) {
- cm = per_cpu_ptr(m->maps, cpu);
- if (!cm->online || cm->available <= maxavl)
- continue;
+ cpu = matrix_find_best_cpu(m, msk);
+ if (cpu == UINT_MAX)
+ return -ENOSPC;
- best_cpu = cpu;
- maxavl = cm->available;
- }
+ cm = per_cpu_ptr(m->maps, cpu);
+ bit = matrix_alloc_area(m, cm, 1, false);
+ if (bit >= m->alloc_end)
+ return -ENOSPC;
+ cm->allocated++;
+ cm->available--;
+ m->total_allocated++;
+ m->global_available--;
+ if (reserved)
+ m->global_reserved--;
+ *mapped_cpu = cpu;
+ trace_irq_matrix_alloc(bit, cpu, m, cm);
+ return bit;
- if (maxavl) {
- cm = per_cpu_ptr(m->maps, best_cpu);
- bit = matrix_alloc_area(m, cm, 1, false);
- if (bit < m->alloc_end) {
- cm->allocated++;
- cm->available--;
- m->total_allocated++;
- m->global_available--;
- if (reserved)
- m->global_reserved--;
- *mapped_cpu = best_cpu;
- trace_irq_matrix_alloc(bit, best_cpu, m, cm);
- return bit;
- }
- }
- return -ENOSPC;
}
/**
const struct jump_entry *jea = a;
const struct jump_entry *jeb = b;
- if (jea->key < jeb->key)
+ if (jump_entry_key(jea) < jump_entry_key(jeb))
return -1;
- if (jea->key > jeb->key)
+ if (jump_entry_key(jea) > jump_entry_key(jeb))
return 1;
return 0;
}
+static void jump_label_swap(void *a, void *b, int size)
+{
+ long delta = (unsigned long)a - (unsigned long)b;
+ struct jump_entry *jea = a;
+ struct jump_entry *jeb = b;
+ struct jump_entry tmp = *jea;
+
+ jea->code = jeb->code - delta;
+ jea->target = jeb->target - delta;
+ jea->key = jeb->key - delta;
+
+ jeb->code = tmp.code + delta;
+ jeb->target = tmp.target + delta;
+ jeb->key = tmp.key + delta;
+}
+
static void
jump_label_sort_entries(struct jump_entry *start, struct jump_entry *stop)
{
unsigned long size;
+ void *swapfn = NULL;
+
+ if (IS_ENABLED(CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE))
+ swapfn = jump_label_swap;
size = (((unsigned long)stop - (unsigned long)start)
/ sizeof(struct jump_entry));
- sort(start, size, sizeof(struct jump_entry), jump_label_cmp, NULL);
+ sort(start, size, sizeof(struct jump_entry), jump_label_cmp, swapfn);
}
static void jump_label_update(struct static_key *key);
int v, v1;
STATIC_KEY_CHECK_USE(key);
+ lockdep_assert_cpus_held();
/*
* Careful if we get concurrent static_key_slow_inc() calls;
void static_key_enable_cpuslocked(struct static_key *key)
{
STATIC_KEY_CHECK_USE(key);
+ lockdep_assert_cpus_held();
if (atomic_read(&key->enabled) > 0) {
WARN_ON_ONCE(atomic_read(&key->enabled) != 1);
void static_key_disable_cpuslocked(struct static_key *key)
{
STATIC_KEY_CHECK_USE(key);
+ lockdep_assert_cpus_held();
if (atomic_read(&key->enabled) != 1) {
WARN_ON_ONCE(atomic_read(&key->enabled) != 0);
unsigned long rate_limit,
struct delayed_work *work)
{
+ lockdep_assert_cpus_held();
+
/*
* The negative count check is valid even when a negative
* key->enabled is in use by static_key_slow_inc(); a
static int addr_conflict(struct jump_entry *entry, void *start, void *end)
{
- if (entry->code <= (unsigned long)end &&
- entry->code + JUMP_LABEL_NOP_SIZE > (unsigned long)start)
+ if (jump_entry_code(entry) <= (unsigned long)end &&
+ jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE > (unsigned long)start)
return 1;
return 0;
key->type |= JUMP_TYPE_LINKED;
}
-static inline struct static_key *jump_entry_key(struct jump_entry *entry)
-{
- return (struct static_key *)((unsigned long)entry->key & ~1UL);
-}
-
-static bool jump_entry_branch(struct jump_entry *entry)
-{
- return (unsigned long)entry->key & 1UL;
-}
-
/***
* A 'struct static_key' uses a union such that it either points directly
* to a table of 'struct jump_entry' or to a linked list of modules which in
{
struct static_key *key = jump_entry_key(entry);
bool enabled = static_key_enabled(key);
- bool branch = jump_entry_branch(entry);
+ bool branch = jump_entry_is_branch(entry);
/* See the comment in linux/jump_label.h */
return enabled ^ branch;
static void __jump_label_update(struct static_key *key,
struct jump_entry *entry,
- struct jump_entry *stop)
+ struct jump_entry *stop,
+ bool init)
{
for (; (entry < stop) && (jump_entry_key(entry) == key); entry++) {
/*
* An entry->code of 0 indicates an entry which has been
* disabled because it was in an init text area.
*/
- if (entry->code) {
- if (kernel_text_address(entry->code))
+ if (init || !jump_entry_is_init(entry)) {
+ if (kernel_text_address(jump_entry_code(entry)))
arch_jump_label_transform(entry, jump_label_type(entry));
else
WARN_ONCE(1, "can't patch jump_label at %pS",
- (void *)(unsigned long)entry->code);
+ (void *)jump_entry_code(entry));
}
}
}
if (jump_label_type(iter) == JUMP_LABEL_NOP)
arch_jump_label_transform_static(iter, JUMP_LABEL_NOP);
+ if (init_section_contains((void *)jump_entry_code(iter), 1))
+ jump_entry_set_init(iter);
+
iterk = jump_entry_key(iter);
if (iterk == key)
continue;
cpus_read_unlock();
}
-/* Disable any jump label entries in __init/__exit code */
-void __init jump_label_invalidate_initmem(void)
-{
- struct jump_entry *iter_start = __start___jump_table;
- struct jump_entry *iter_stop = __stop___jump_table;
- struct jump_entry *iter;
-
- for (iter = iter_start; iter < iter_stop; iter++) {
- if (init_section_contains((void *)(unsigned long)iter->code, 1))
- iter->code = 0;
- }
-}
-
#ifdef CONFIG_MODULES
static enum jump_label_type jump_label_init_type(struct jump_entry *entry)
{
struct static_key *key = jump_entry_key(entry);
bool type = static_key_type(key);
- bool branch = jump_entry_branch(entry);
+ bool branch = jump_entry_is_branch(entry);
/* See the comment in linux/jump_label.h */
return type ^ branch;
static inline struct static_key_mod *static_key_mod(struct static_key *key)
{
- WARN_ON_ONCE(!(key->type & JUMP_TYPE_LINKED));
+ WARN_ON_ONCE(!static_key_linked(key));
return (struct static_key_mod *)(key->type & ~JUMP_TYPE_MASK);
}
stop = __stop___jump_table;
else
stop = m->jump_entries + m->num_jump_entries;
- __jump_label_update(key, mod->entries, stop);
+ __jump_label_update(key, mod->entries, stop,
+ m && m->state == MODULE_STATE_COMING);
}
}
for (iter = iter_start; iter < iter_stop; iter++) {
struct static_key *iterk;
+ if (within_module_init(jump_entry_code(iter), mod))
+ jump_entry_set_init(iter);
+
iterk = jump_entry_key(iter);
if (iterk == key)
continue;
key = iterk;
- if (within_module(iter->key, mod)) {
+ if (within_module((unsigned long)key, mod)) {
static_key_set_entries(key, iter);
continue;
}
/* Only update if we've changed from our initial state */
if (jump_label_type(iter) != jump_label_init_type(iter))
- __jump_label_update(key, iter, iter_stop);
+ __jump_label_update(key, iter, iter_stop, true);
}
return 0;
key = jump_entry_key(iter);
- if (within_module(iter->key, mod))
+ if (within_module((unsigned long)key, mod))
continue;
/* No memory during module load */
}
}
-/* Disable any jump label entries in module init code */
-static void jump_label_invalidate_module_init(struct module *mod)
-{
- struct jump_entry *iter_start = mod->jump_entries;
- struct jump_entry *iter_stop = iter_start + mod->num_jump_entries;
- struct jump_entry *iter;
-
- for (iter = iter_start; iter < iter_stop; iter++) {
- if (within_module_init(iter->code, mod))
- iter->code = 0;
- }
-}
-
static int
jump_label_module_notify(struct notifier_block *self, unsigned long val,
void *data)
case MODULE_STATE_GOING:
jump_label_del_module(mod);
break;
- case MODULE_STATE_LIVE:
- jump_label_invalidate_module_init(mod);
- break;
}
jump_label_unlock();
entry = static_key_entries(key);
/* if there are no users, entry can be NULL */
if (entry)
- __jump_label_update(key, entry, stop);
+ __jump_label_update(key, entry, stop,
+ system_state < SYSTEM_RUNNING);
}
#ifdef CONFIG_STATIC_KEYS_SELFTEST
struct optimized_kprobe *op, *tmp;
list_for_each_entry_safe(op, tmp, &freeing_list, list) {
- BUG_ON(!kprobe_unused(&op->kp));
list_del_init(&op->list);
+ if (WARN_ON_ONCE(!kprobe_unused(&op->kp))) {
+ /*
+ * This must not happen, but if there is a kprobe
+ * still in use, keep it on kprobes hash list.
+ */
+ continue;
+ }
free_aggr_kprobe(&op->kp);
}
}
}
/* Cancel unoptimizing for reusing */
-static void reuse_unused_kprobe(struct kprobe *ap)
+static int reuse_unused_kprobe(struct kprobe *ap)
{
struct optimized_kprobe *op;
+ int ret;
- BUG_ON(!kprobe_unused(ap));
/*
* Unused kprobe MUST be on the way of delayed unoptimizing (means
* there is still a relative jump) and disabled.
/* Enable the probe again */
ap->flags &= ~KPROBE_FLAG_DISABLED;
/* Optimize it again (remove from op->list) */
- BUG_ON(!kprobe_optready(ap));
+ ret = kprobe_optready(ap);
+ if (ret)
+ return ret;
+
optimize_kprobe(ap);
+ return 0;
}
/* Remove optimized instructions */
#define kprobe_disarmed(p) kprobe_disabled(p)
#define wait_for_kprobe_optimizer() do {} while (0)
-/* There should be no unused kprobes can be reused without optimization */
-static void reuse_unused_kprobe(struct kprobe *ap)
+static int reuse_unused_kprobe(struct kprobe *ap)
{
+ /*
+ * If the optimized kprobe is NOT supported, the aggr kprobe is
+ * released at the same time that the last aggregated kprobe is
+ * unregistered.
+ * Thus there should be no chance to reuse unused kprobe.
+ */
printk(KERN_ERR "Error: There should be no unused kprobe here.\n");
- BUG_ON(kprobe_unused(ap));
+ return -EINVAL;
}
static void free_aggr_kprobe(struct kprobe *p)
/* Add the new probe to ap->list */
static int add_new_kprobe(struct kprobe *ap, struct kprobe *p)
{
- BUG_ON(kprobe_gone(ap) || kprobe_gone(p));
-
if (p->post_handler)
unoptimize_kprobe(ap, true); /* Fall back to normal kprobe */
goto out;
}
init_aggr_kprobe(ap, orig_p);
- } else if (kprobe_unused(ap))
+ } else if (kprobe_unused(ap)) {
/* This probe is going to die. Rescue it */
- reuse_unused_kprobe(ap);
+ ret = reuse_unused_kprobe(ap);
+ if (ret)
+ goto out;
+ }
if (kprobe_gone(ap)) {
/*
return 0;
disarmed:
- BUG_ON(!kprobe_disarmed(ap));
hlist_del_rcu(&ap->hlist);
return 0;
}
* get freed - this significantly simplifies the debugging code.
*/
unsigned long nr_lock_classes;
-static struct lock_class lock_classes[MAX_LOCKDEP_KEYS];
+struct lock_class lock_classes[MAX_LOCKDEP_KEYS];
static inline struct lock_class *hlock_class(struct held_lock *hlock)
{
printk("%*s->", depth, "");
print_lock_name(class);
- printk(KERN_CONT " ops: %lu", class->ops);
+#ifdef CONFIG_DEBUG_LOCKDEP
+ printk(KERN_CONT " ops: %lu", debug_class_ops_read(class));
+#endif
printk(KERN_CONT " {\n");
for (bit = 0; bit < LOCK_USAGE_STATES; bit++) {
return 1;
}
-/*
- * This is for building a chain between just two different classes,
- * instead of adding a new hlock upon current, which is done by
- * add_chain_cache().
- *
- * This can be called in any context with two classes, while
- * add_chain_cache() must be done within the lock owener's context
- * since it uses hlock which might be racy in another context.
- */
-static inline int add_chain_cache_classes(unsigned int prev,
- unsigned int next,
- unsigned int irq_context,
- u64 chain_key)
-{
- struct hlist_head *hash_head = chainhashentry(chain_key);
- struct lock_chain *chain;
-
- /*
- * Allocate a new chain entry from the static array, and add
- * it to the hash:
- */
-
- /*
- * We might need to take the graph lock, ensure we've got IRQs
- * disabled to make this an IRQ-safe lock.. for recursion reasons
- * lockdep won't complain about its own locking errors.
- */
- if (DEBUG_LOCKS_WARN_ON(!irqs_disabled()))
- return 0;
-
- if (unlikely(nr_lock_chains >= MAX_LOCKDEP_CHAINS)) {
- if (!debug_locks_off_graph_unlock())
- return 0;
-
- print_lockdep_off("BUG: MAX_LOCKDEP_CHAINS too low!");
- dump_stack();
- return 0;
- }
-
- chain = lock_chains + nr_lock_chains++;
- chain->chain_key = chain_key;
- chain->irq_context = irq_context;
- chain->depth = 2;
- if (likely(nr_chain_hlocks + chain->depth <= MAX_LOCKDEP_CHAIN_HLOCKS)) {
- chain->base = nr_chain_hlocks;
- nr_chain_hlocks += chain->depth;
- chain_hlocks[chain->base] = prev - 1;
- chain_hlocks[chain->base + 1] = next -1;
- }
-#ifdef CONFIG_DEBUG_LOCKDEP
- /*
- * Important for check_no_collision().
- */
- else {
- if (!debug_locks_off_graph_unlock())
- return 0;
-
- print_lockdep_off("BUG: MAX_LOCKDEP_CHAIN_HLOCKS too low!");
- dump_stack();
- return 0;
- }
-#endif
-
- hlist_add_head_rcu(&chain->entry, hash_head);
- debug_atomic_inc(chain_lookup_misses);
- inc_chains();
-
- return 1;
-}
-
/*
* Adds a dependency chain into chain hashtable. And must be called with
* graph_lock held.
/*
* This gets called for every mutex_lock*()/spin_lock*() operation.
* We maintain the dependency maps and validate the locking attempt:
+ *
+ * The callers must make sure that IRQs are disabled before calling it,
+ * otherwise we could get an interrupt which would want to take locks,
+ * which would end up in lockdep again.
*/
static int __lock_acquire(struct lockdep_map *lock, unsigned int subclass,
int trylock, int read, int check, int hardirqs_off,
if (unlikely(!debug_locks))
return 0;
- /*
- * Lockdep should run with IRQs disabled, otherwise we could
- * get an interrupt which would want to take locks, which would
- * end up in lockdep and have you got a head-ache already?
- */
- if (DEBUG_LOCKS_WARN_ON(!irqs_disabled()))
- return 0;
-
if (!prove_locking || lock->key == &__lockdep_no_validate__)
check = 0;
if (!class)
return 0;
}
- atomic_inc((atomic_t *)&class->ops);
+
+ debug_class_ops_inc(class);
+
if (very_verbose(class)) {
printk("\nacquire class [%px] %s", class->key, class->name);
if (class->name_version > 1)
{
struct held_lock *hlock;
+ if (DEBUG_LOCKS_WARN_ON(!irqs_disabled()))
+ return 0;
+
for (hlock = curr->held_locks + idx; idx < depth; idx++, hlock++) {
if (!__lock_acquire(hlock->instance,
hlock_class(hlock)->subclass,
curr->lockdep_depth = i;
curr->curr_chain_key = hlock->prev_chain_key;
+ /*
+ * The most likely case is when the unlock is on the innermost
+ * lock. In this case, we are done!
+ */
+ if (i == depth-1)
+ return 1;
+
if (reacquire_held_locks(curr, depth, i + 1))
return 0;
* We had N bottles of beer on the wall, we drank one, but now
* there's not N-1 bottles of beer left on the wall...
*/
- if (DEBUG_LOCKS_WARN_ON(curr->lockdep_depth != depth - 1))
- return 0;
+ DEBUG_LOCKS_WARN_ON(curr->lockdep_depth != depth-1);
- return 1;
+ /*
+ * Since reacquire_held_locks() would have called check_chain_key()
+ * indirectly via __lock_acquire(), we don't need to do it again
+ * on return.
+ */
+ return 0;
}
static int __lock_is_held(const struct lockdep_map *lock, int read)
{
unsigned long flags;
- if (unlikely(!lock_stat))
+ if (unlikely(!lock_stat || !debug_locks))
return;
if (unlikely(current->lockdep_recursion))
{
unsigned long flags;
- if (unlikely(!lock_stat))
+ if (unlikely(!lock_stat || !debug_locks))
return;
if (unlikely(current->lockdep_recursion))
int nr_find_usage_forwards_recursions;
int nr_find_usage_backwards_checks;
int nr_find_usage_backwards_recursions;
+
+ /*
+ * Per lock class locking operation stat counts
+ */
+ unsigned long lock_class_ops[MAX_LOCKDEP_KEYS];
};
DECLARE_PER_CPU(struct lockdep_stats, lockdep_stats);
+extern struct lock_class lock_classes[MAX_LOCKDEP_KEYS];
#define __debug_atomic_inc(ptr) \
this_cpu_inc(lockdep_stats.ptr);
} \
__total; \
})
+
+static inline void debug_class_ops_inc(struct lock_class *class)
+{
+ int idx;
+
+ idx = class - lock_classes;
+ __debug_atomic_inc(lock_class_ops[idx]);
+}
+
+static inline unsigned long debug_class_ops_read(struct lock_class *class)
+{
+ int idx, cpu;
+ unsigned long ops = 0;
+
+ idx = class - lock_classes;
+ for_each_possible_cpu(cpu)
+ ops += per_cpu(lockdep_stats.lock_class_ops[idx], cpu);
+ return ops;
+}
+
#else
# define __debug_atomic_inc(ptr) do { } while (0)
# define debug_atomic_inc(ptr) do { } while (0)
# define debug_atomic_dec(ptr) do { } while (0)
# define debug_atomic_read(ptr) 0
+# define debug_class_ops_inc(ptr) do { } while (0)
#endif
seq_printf(m, "%p", class->key);
#ifdef CONFIG_DEBUG_LOCKDEP
- seq_printf(m, " OPS:%8ld", class->ops);
+ seq_printf(m, " OPS:%8ld", debug_class_ops_read(class));
#endif
#ifdef CONFIG_PROVE_LOCKING
seq_printf(m, " FD:%5ld", lockdep_count_forward_deps(class));
*/
#include "mcs_spinlock.h"
+#define MAX_NODES 4
+/*
+ * On 64-bit architectures, the mcs_spinlock structure will be 16 bytes in
+ * size and four of them will fit nicely in one 64-byte cacheline. For
+ * pvqspinlock, however, we need more space for extra data. To accommodate
+ * that, we insert two more long words to pad it up to 32 bytes. IOW, only
+ * two of them can fit in a cacheline in this case. That is OK as it is rare
+ * to have more than 2 levels of slowpath nesting in actual use. We don't
+ * want to penalize pvqspinlocks to optimize for a rare case in native
+ * qspinlocks.
+ */
+struct qnode {
+ struct mcs_spinlock mcs;
#ifdef CONFIG_PARAVIRT_SPINLOCKS
-#define MAX_NODES 8
-#else
-#define MAX_NODES 4
+ long reserved[2];
#endif
+};
/*
* The pending bit spinning loop count.
*
* PV doubles the storage and uses the second cacheline for PV state.
*/
-static DEFINE_PER_CPU_ALIGNED(struct mcs_spinlock, mcs_nodes[MAX_NODES]);
+static DEFINE_PER_CPU_ALIGNED(struct qnode, qnodes[MAX_NODES]);
/*
* We must be able to distinguish between no-tail and the tail at 0:0,
int cpu = (tail >> _Q_TAIL_CPU_OFFSET) - 1;
int idx = (tail & _Q_TAIL_IDX_MASK) >> _Q_TAIL_IDX_OFFSET;
- return per_cpu_ptr(&mcs_nodes[idx], cpu);
+ return per_cpu_ptr(&qnodes[idx].mcs, cpu);
+}
+
+static inline __pure
+struct mcs_spinlock *grab_mcs_node(struct mcs_spinlock *base, int idx)
+{
+ return &((struct qnode *)base + idx)->mcs;
}
#define _Q_LOCKED_PENDING_MASK (_Q_LOCKED_MASK | _Q_PENDING_MASK)
}
#endif /* _Q_PENDING_BITS == 8 */
+/**
+ * queued_fetch_set_pending_acquire - fetch the whole lock value and set pending
+ * @lock : Pointer to queued spinlock structure
+ * Return: The previous lock value
+ *
+ * *,*,* -> *,1,*
+ */
+#ifndef queued_fetch_set_pending_acquire
+static __always_inline u32 queued_fetch_set_pending_acquire(struct qspinlock *lock)
+{
+ return atomic_fetch_or_acquire(_Q_PENDING_VAL, &lock->val);
+}
+#endif
+
/**
* set_locked - Set the lock bit and own the lock
* @lock: Pointer to queued spinlock structure
/*
* trylock || pending
*
- * 0,0,0 -> 0,0,1 ; trylock
- * 0,0,1 -> 0,1,1 ; pending
+ * 0,0,* -> 0,1,* -> 0,0,1 pending, trylock
*/
- val = atomic_fetch_or_acquire(_Q_PENDING_VAL, &lock->val);
- if (!(val & ~_Q_LOCKED_MASK)) {
- /*
- * We're pending, wait for the owner to go away.
- *
- * *,1,1 -> *,1,0
- *
- * this wait loop must be a load-acquire such that we match the
- * store-release that clears the locked bit and create lock
- * sequentiality; this is because not all
- * clear_pending_set_locked() implementations imply full
- * barriers.
- */
- if (val & _Q_LOCKED_MASK) {
- atomic_cond_read_acquire(&lock->val,
- !(VAL & _Q_LOCKED_MASK));
- }
+ val = queued_fetch_set_pending_acquire(lock);
- /*
- * take ownership and clear the pending bit.
- *
- * *,1,0 -> *,0,1
- */
- clear_pending_set_locked(lock);
- qstat_inc(qstat_lock_pending, true);
- return;
+ /*
+ * If we observe contention, there is a concurrent locker.
+ *
+ * Undo and queue; our setting of PENDING might have made the
+ * n,0,0 -> 0,0,0 transition fail and it will now be waiting
+ * on @next to become !NULL.
+ */
+ if (unlikely(val & ~_Q_LOCKED_MASK)) {
+
+ /* Undo PENDING if we set it. */
+ if (!(val & _Q_PENDING_MASK))
+ clear_pending(lock);
+
+ goto queue;
}
/*
- * If pending was clear but there are waiters in the queue, then
- * we need to undo our setting of pending before we queue ourselves.
+ * We're pending, wait for the owner to go away.
+ *
+ * 0,1,1 -> 0,1,0
+ *
+ * this wait loop must be a load-acquire such that we match the
+ * store-release that clears the locked bit and create lock
+ * sequentiality; this is because not all
+ * clear_pending_set_locked() implementations imply full
+ * barriers.
+ */
+ if (val & _Q_LOCKED_MASK)
+ atomic_cond_read_acquire(&lock->val, !(VAL & _Q_LOCKED_MASK));
+
+ /*
+ * take ownership and clear the pending bit.
+ *
+ * 0,1,0 -> 0,0,1
*/
- if (!(val & _Q_PENDING_MASK))
- clear_pending(lock);
+ clear_pending_set_locked(lock);
+ qstat_inc(qstat_lock_pending, true);
+ return;
/*
* End of pending bit optimistic spinning and beginning of MCS
queue:
qstat_inc(qstat_lock_slowpath, true);
pv_queue:
- node = this_cpu_ptr(&mcs_nodes[0]);
+ node = this_cpu_ptr(&qnodes[0].mcs);
idx = node->count++;
tail = encode_tail(smp_processor_id(), idx);
- node += idx;
+ node = grab_mcs_node(node, idx);
+
+ /*
+ * Keep counts of non-zero index values:
+ */
+ qstat_inc(qstat_lock_idx1 + idx - 1, idx);
/*
* Ensure that we increment the head node->count before initialising
*/
/*
- * In the PV case we might already have _Q_LOCKED_VAL set.
+ * In the PV case we might already have _Q_LOCKED_VAL set, because
+ * of lock stealing; therefore we must also allow:
+ *
+ * n,0,1 -> 0,0,1
*
- * The atomic_cond_read_acquire() call above has provided the
- * necessary acquire semantics required for locking.
+ * Note: at this point: (val & _Q_PENDING_MASK) == 0, because of the
+ * above wait condition, therefore any concurrent setting of
+ * PENDING will make the uncontended transition fail.
*/
- if (((val & _Q_TAIL_MASK) == tail) &&
- atomic_try_cmpxchg_relaxed(&lock->val, &val, _Q_LOCKED_VAL))
- goto release; /* No contention */
+ if ((val & _Q_TAIL_MASK) == tail) {
+ if (atomic_try_cmpxchg_relaxed(&lock->val, &val, _Q_LOCKED_VAL))
+ goto release; /* No contention */
+ }
- /* Either somebody is queued behind us or _Q_PENDING_VAL is set */
+ /*
+ * Either somebody is queued behind us or _Q_PENDING_VAL got set
+ * which will then detect the remaining tail and queue behind us
+ * ensuring we'll see a @next.
+ */
set_locked(lock);
/*
/*
* release the node
*/
- __this_cpu_dec(mcs_nodes[0].count);
+ __this_cpu_dec(qnodes[0].mcs.count);
}
EXPORT_SYMBOL(queued_spin_lock_slowpath);
struct pv_node {
struct mcs_spinlock mcs;
- struct mcs_spinlock __res[3];
-
int cpu;
u8 state;
};
{
struct pv_node *pn = (struct pv_node *)node;
- BUILD_BUG_ON(sizeof(struct pv_node) > 5*sizeof(struct mcs_spinlock));
+ BUILD_BUG_ON(sizeof(struct pv_node) > sizeof(struct qnode));
pn->cpu = smp_processor_id();
pn->state = vcpu_running;
qstat_pv_wait_node,
qstat_lock_pending,
qstat_lock_slowpath,
+ qstat_lock_idx1,
+ qstat_lock_idx2,
+ qstat_lock_idx3,
qstat_num, /* Total number of statistical counters */
qstat_reset_cnts = qstat_num,
};
[qstat_pv_wait_node] = "pv_wait_node",
[qstat_lock_pending] = "lock_pending",
[qstat_lock_slowpath] = "lock_slowpath",
+ [qstat_lock_idx1] = "lock_index1",
+ [qstat_lock_idx2] = "lock_index2",
+ [qstat_lock_idx3] = "lock_index3",
[qstat_reset_cnts] = "reset_counters",
};
__rt_mutex_lock(lock, subclass);
}
EXPORT_SYMBOL_GPL(rt_mutex_lock_nested);
-#endif
-#ifndef CONFIG_DEBUG_LOCK_ALLOC
+#else /* !CONFIG_DEBUG_LOCK_ALLOC */
+
/**
* rt_mutex_lock - lock a rt_mutex
*
* but it gives the spinners an early indication that the
* readers now have the lock.
*/
- rwsem_set_reader_owned(sem);
+ __rwsem_set_reader_owned(sem, waiter->task);
}
/*
waiter.type = RWSEM_WAITING_FOR_READ;
raw_spin_lock_irq(&sem->wait_lock);
- if (list_empty(&sem->wait_list))
+ if (list_empty(&sem->wait_list)) {
+ /*
+ * In case the wait queue is empty and the lock isn't owned
+ * by a writer, this reader can exit the slowpath and return
+ * immediately as its RWSEM_ACTIVE_READ_BIAS has already
+ * been set in the count.
+ */
+ if (atomic_long_read(&sem->count) >= 0) {
+ raw_spin_unlock_irq(&sem->wait_lock);
+ return sem;
+ }
adjustment += RWSEM_WAITING_BIAS;
+ }
list_add_tail(&waiter.list, &sem->wait_list);
/* we're now waiting on the lock, but no longer actively locking */
void up_read(struct rw_semaphore *sem)
{
rwsem_release(&sem->dep_map, 1, _RET_IP_);
- DEBUG_RWSEMS_WARN_ON(sem->owner != RWSEM_READER_OWNED);
+ DEBUG_RWSEMS_WARN_ON(!((unsigned long)sem->owner & RWSEM_READER_OWNED));
+ rwsem_clear_reader_owned(sem);
__up_read(sem);
}
might_sleep();
__down_read(sem);
- rwsem_set_reader_owned(sem);
+ __rwsem_set_reader_owned(sem, NULL);
}
EXPORT_SYMBOL(down_read_non_owner);
void up_read_non_owner(struct rw_semaphore *sem)
{
- DEBUG_RWSEMS_WARN_ON(sem->owner != RWSEM_READER_OWNED);
+ DEBUG_RWSEMS_WARN_ON(!((unsigned long)sem->owner & RWSEM_READER_OWNED));
__up_read(sem);
}
/* SPDX-License-Identifier: GPL-2.0 */
/*
- * The owner field of the rw_semaphore structure will be set to
- * RWSEM_READER_OWNED when a reader grabs the lock. A writer will clear
- * the owner field when it unlocks. A reader, on the other hand, will
- * not touch the owner field when it unlocks.
+ * The least significant 2 bits of the owner value has the following
+ * meanings when set.
+ * - RWSEM_READER_OWNED (bit 0): The rwsem is owned by readers
+ * - RWSEM_ANONYMOUSLY_OWNED (bit 1): The rwsem is anonymously owned,
+ * i.e. the owner(s) cannot be readily determined. It can be reader
+ * owned or the owning writer is indeterminate.
*
- * In essence, the owner field now has the following 4 states:
- * 1) 0
- * - lock is free or the owner hasn't set the field yet
- * 2) RWSEM_READER_OWNED
- * - lock is currently or previously owned by readers (lock is free
- * or not set by owner yet)
- * 3) RWSEM_ANONYMOUSLY_OWNED bit set with some other bits set as well
- * - lock is owned by an anonymous writer, so spinning on the lock
- * owner should be disabled.
- * 4) Other non-zero value
- * - a writer owns the lock and other writers can spin on the lock owner.
+ * When a writer acquires a rwsem, it puts its task_struct pointer
+ * into the owner field. It is cleared after an unlock.
+ *
+ * When a reader acquires a rwsem, it will also puts its task_struct
+ * pointer into the owner field with both the RWSEM_READER_OWNED and
+ * RWSEM_ANONYMOUSLY_OWNED bits set. On unlock, the owner field will
+ * largely be left untouched. So for a free or reader-owned rwsem,
+ * the owner value may contain information about the last reader that
+ * acquires the rwsem. The anonymous bit is set because that particular
+ * reader may or may not still own the lock.
+ *
+ * That information may be helpful in debugging cases where the system
+ * seems to hang on a reader owned rwsem especially if only one reader
+ * is involved. Ideally we would like to track all the readers that own
+ * a rwsem, but the overhead is simply too big.
*/
-#define RWSEM_ANONYMOUSLY_OWNED (1UL << 0)
-#define RWSEM_READER_OWNED ((struct task_struct *)RWSEM_ANONYMOUSLY_OWNED)
+#define RWSEM_READER_OWNED (1UL << 0)
+#define RWSEM_ANONYMOUSLY_OWNED (1UL << 1)
#ifdef CONFIG_DEBUG_RWSEMS
# define DEBUG_RWSEMS_WARN_ON(c) DEBUG_LOCKS_WARN_ON(c)
WRITE_ONCE(sem->owner, NULL);
}
+/*
+ * The task_struct pointer of the last owning reader will be left in
+ * the owner field.
+ *
+ * Note that the owner value just indicates the task has owned the rwsem
+ * previously, it may not be the real owner or one of the real owners
+ * anymore when that field is examined, so take it with a grain of salt.
+ */
+static inline void __rwsem_set_reader_owned(struct rw_semaphore *sem,
+ struct task_struct *owner)
+{
+ unsigned long val = (unsigned long)owner | RWSEM_READER_OWNED
+ | RWSEM_ANONYMOUSLY_OWNED;
+
+ WRITE_ONCE(sem->owner, (struct task_struct *)val);
+}
+
static inline void rwsem_set_reader_owned(struct rw_semaphore *sem)
{
- /*
- * We check the owner value first to make sure that we will only
- * do a write to the rwsem cacheline when it is really necessary
- * to minimize cacheline contention.
- */
- if (READ_ONCE(sem->owner) != RWSEM_READER_OWNED)
- WRITE_ONCE(sem->owner, RWSEM_READER_OWNED);
+ __rwsem_set_reader_owned(sem, current);
}
/*
{
return (unsigned long)owner & RWSEM_ANONYMOUSLY_OWNED;
}
+
+#ifdef CONFIG_DEBUG_RWSEMS
+/*
+ * With CONFIG_DEBUG_RWSEMS configured, it will make sure that if there
+ * is a task pointer in owner of a reader-owned rwsem, it will be the
+ * real owner or one of the real owners. The only exception is when the
+ * unlock is done by up_read_non_owner().
+ */
+#define rwsem_clear_reader_owned rwsem_clear_reader_owned
+static inline void rwsem_clear_reader_owned(struct rw_semaphore *sem)
+{
+ unsigned long val = (unsigned long)current | RWSEM_READER_OWNED
+ | RWSEM_ANONYMOUSLY_OWNED;
+ if (READ_ONCE(sem->owner) == (struct task_struct *)val)
+ cmpxchg_relaxed((unsigned long *)&sem->owner, val,
+ RWSEM_READER_OWNED | RWSEM_ANONYMOUSLY_OWNED);
+}
+#endif
+
#else
static inline void rwsem_set_owner(struct rw_semaphore *sem)
{
{
}
+static inline void __rwsem_set_reader_owned(struct rw_semaphore *sem,
+ struct task_struct *owner)
+{
+}
+
static inline void rwsem_set_reader_owned(struct rw_semaphore *sem)
{
}
#endif
+
+#ifndef rwsem_clear_reader_owned
+static inline void rwsem_clear_reader_owned(struct rw_semaphore *sem)
+{
+}
+#endif
* Note: ro_after_init sections also have SHF_{WRITE,ALLOC} set.
*/
ndx = find_sec(info, ".data..ro_after_init");
+ if (ndx)
+ info->sechdrs[ndx].sh_flags |= SHF_RO_AFTER_INIT;
+ /*
+ * Mark the __jump_table section as ro_after_init as well: these data
+ * structures are never modified, with the exception of entries that
+ * refer to code in the __init section, which are annotated as such
+ * at module load time.
+ */
+ ndx = find_sec(info, "__jump_table");
if (ndx)
info->sechdrs[ndx].sh_flags |= SHF_RO_AFTER_INIT;
if (wq_busy)
show_workqueue_state();
- if (!wakeup) {
+ if (!wakeup || pm_debug_messages_on) {
read_lock(&tasklist_lock);
for_each_process_thread(g, p) {
if (p != current && !freezer_should_skip(p)
enum s2idle_states __read_mostly s2idle_state;
static DEFINE_RAW_SPINLOCK(s2idle_lock);
+bool pm_suspend_via_s2idle(void)
+{
+ return mem_sleep_current == PM_SUSPEND_TO_IDLE;
+}
+EXPORT_SYMBOL_GPL(pm_suspend_via_s2idle);
+
void s2idle_set_ops(const struct platform_s2idle_ops *ops)
{
lock_system_sleep();
This option boosts the priority of preempted RCU readers that
block the current preemptible RCU grace period for too long.
This option also prevents heavy loads from blocking RCU
- callback invocation for all flavors of RCU.
+ callback invocation.
Say Y here if you are working with real-time apps or heavy loads
Say N here if you are unsure.
callback invocation to energy-efficient CPUs in battery-powered
asymmetric multiprocessors.
- This option offloads callback invocation from the set of
- CPUs specified at boot time by the rcu_nocbs parameter.
- For each such CPU, a kthread ("rcuox/N") will be created to
- invoke callbacks, where the "N" is the CPU being offloaded,
- and where the "x" is "b" for RCU-bh, "p" for RCU-preempt, and
- "s" for RCU-sched. Nothing prevents this kthread from running
+ This option offloads callback invocation from the set of CPUs
+ specified at boot time by the rcu_nocbs parameter. For each
+ such CPU, a kthread ("rcuox/N") will be created to invoke
+ callbacks, where the "N" is the CPU being offloaded, and where
+ the "p" for RCU-preempt (PREEMPT kernels) and "s" for RCU-sched
+ (!PREEMPT kernels). Nothing prevents this kthread from running
on the specified CPUs, but (1) the kthreads may be preempted
between each callback, and (2) affinity or cgroups can be used
to force the kthreads to run on whatever set of CPUs is desired.
/*
* debug_rcu_head_queue()/debug_rcu_head_unqueue() are used internally
- * by call_rcu() and rcu callback execution, and are therefore not part of the
- * RCU API. Leaving in rcupdate.h because they are used by all RCU flavors.
+ * by call_rcu() and rcu callback execution, and are therefore not part
+ * of the RCU API. These are in rcupdate.h because they are used by all
+ * RCU implementations.
*/
#ifdef CONFIG_DEBUG_OBJECTS_RCU_HEAD
*/
static inline bool __rcu_reclaim(const char *rn, struct rcu_head *head)
{
+ rcu_callback_t f;
unsigned long offset = (unsigned long)head->func;
rcu_lock_acquire(&rcu_callback_map);
return true;
} else {
RCU_TRACE(trace_rcu_invoke_callback(rn, head);)
- head->func(head);
+ f = head->func;
+ WRITE_ONCE(head->func, (rcu_callback_t)0L);
+ f(head);
rcu_lock_release(&rcu_callback_map);
return false;
}
}
}
-/* Returns first leaf rcu_node of the specified RCU flavor. */
-#define rcu_first_leaf_node(rsp) ((rsp)->level[rcu_num_lvls - 1])
+/* Returns a pointer to the first leaf rcu_node structure. */
+#define rcu_first_leaf_node() (rcu_state.level[rcu_num_lvls - 1])
/* Is this rcu_node a leaf? */
#define rcu_is_leaf_node(rnp) ((rnp)->level == rcu_num_lvls - 1)
/* Is this rcu_node the last leaf? */
-#define rcu_is_last_leaf_node(rsp, rnp) ((rnp) == &(rsp)->node[rcu_num_nodes - 1])
+#define rcu_is_last_leaf_node(rnp) ((rnp) == &rcu_state.node[rcu_num_nodes - 1])
/*
- * Do a full breadth-first scan of the rcu_node structures for the
- * specified rcu_state structure.
+ * Do a full breadth-first scan of the {s,}rcu_node structures for the
+ * specified state structure (for SRCU) or the only rcu_state structure
+ * (for RCU).
*/
-#define rcu_for_each_node_breadth_first(rsp, rnp) \
- for ((rnp) = &(rsp)->node[0]; \
- (rnp) < &(rsp)->node[rcu_num_nodes]; (rnp)++)
+#define srcu_for_each_node_breadth_first(sp, rnp) \
+ for ((rnp) = &(sp)->node[0]; \
+ (rnp) < &(sp)->node[rcu_num_nodes]; (rnp)++)
+#define rcu_for_each_node_breadth_first(rnp) \
+ srcu_for_each_node_breadth_first(&rcu_state, rnp)
/*
- * Do a breadth-first scan of the non-leaf rcu_node structures for the
- * specified rcu_state structure. Note that if there is a singleton
- * rcu_node tree with but one rcu_node structure, this loop is a no-op.
+ * Scan the leaves of the rcu_node hierarchy for the rcu_state structure.
+ * Note that if there is a singleton rcu_node tree with but one rcu_node
+ * structure, this loop -will- visit the rcu_node structure. It is still
+ * a leaf node, even if it is also the root node.
*/
-#define rcu_for_each_nonleaf_node_breadth_first(rsp, rnp) \
- for ((rnp) = &(rsp)->node[0]; !rcu_is_leaf_node(rsp, rnp); (rnp)++)
-
-/*
- * Scan the leaves of the rcu_node hierarchy for the specified rcu_state
- * structure. Note that if there is a singleton rcu_node tree with but
- * one rcu_node structure, this loop -will- visit the rcu_node structure.
- * It is still a leaf node, even if it is also the root node.
- */
-#define rcu_for_each_leaf_node(rsp, rnp) \
- for ((rnp) = rcu_first_leaf_node(rsp); \
- (rnp) < &(rsp)->node[rcu_num_nodes]; (rnp)++)
+#define rcu_for_each_leaf_node(rnp) \
+ for ((rnp) = rcu_first_leaf_node(); \
+ (rnp) < &rcu_state.node[rcu_num_nodes]; (rnp)++)
/*
* Iterate over all possible CPUs in a leaf RCU node.
#endif /* #if defined(SRCU) || !defined(TINY_RCU) */
+#ifdef CONFIG_SRCU
+void srcu_init(void);
+#else /* #ifdef CONFIG_SRCU */
+static inline void srcu_init(void) { }
+#endif /* #else #ifdef CONFIG_SRCU */
+
#ifdef CONFIG_TINY_RCU
/* Tiny RCU doesn't expedite, as its purpose in life is instead to be tiny. */
static inline bool rcu_gp_is_normal(void) { return true; }
#ifdef CONFIG_TINY_RCU
static inline unsigned long rcu_get_gp_seq(void) { return 0; }
-static inline unsigned long rcu_bh_get_gp_seq(void) { return 0; }
-static inline unsigned long rcu_sched_get_gp_seq(void) { return 0; }
static inline unsigned long rcu_exp_batches_completed(void) { return 0; }
-static inline unsigned long rcu_exp_batches_completed_sched(void) { return 0; }
static inline unsigned long
srcu_batches_completed(struct srcu_struct *sp) { return 0; }
static inline void rcu_force_quiescent_state(void) { }
-static inline void rcu_bh_force_quiescent_state(void) { }
-static inline void rcu_sched_force_quiescent_state(void) { }
static inline void show_rcu_gp_kthreads(void) { }
static inline int rcu_get_gp_kthreads_prio(void) { return 0; }
#else /* #ifdef CONFIG_TINY_RCU */
unsigned long rcu_get_gp_seq(void);
-unsigned long rcu_bh_get_gp_seq(void);
-unsigned long rcu_sched_get_gp_seq(void);
unsigned long rcu_exp_batches_completed(void);
-unsigned long rcu_exp_batches_completed_sched(void);
unsigned long srcu_batches_completed(struct srcu_struct *sp);
void show_rcu_gp_kthreads(void);
int rcu_get_gp_kthreads_prio(void);
void rcu_force_quiescent_state(void);
-void rcu_bh_force_quiescent_state(void);
-void rcu_sched_force_quiescent_state(void);
extern struct workqueue_struct *rcu_gp_wq;
extern struct workqueue_struct *rcu_par_gp_wq;
#endif /* #else #ifdef CONFIG_TINY_RCU */
.name = "rcu"
};
-/*
- * Definitions for rcu_bh perf testing.
- */
-
-static int rcu_bh_perf_read_lock(void) __acquires(RCU_BH)
-{
- rcu_read_lock_bh();
- return 0;
-}
-
-static void rcu_bh_perf_read_unlock(int idx) __releases(RCU_BH)
-{
- rcu_read_unlock_bh();
-}
-
-static struct rcu_perf_ops rcu_bh_ops = {
- .ptype = RCU_BH_FLAVOR,
- .init = rcu_sync_perf_init,
- .readlock = rcu_bh_perf_read_lock,
- .readunlock = rcu_bh_perf_read_unlock,
- .get_gp_seq = rcu_bh_get_gp_seq,
- .gp_diff = rcu_seq_diff,
- .exp_completed = rcu_exp_batches_completed_sched,
- .async = call_rcu_bh,
- .gp_barrier = rcu_barrier_bh,
- .sync = synchronize_rcu_bh,
- .exp_sync = synchronize_rcu_bh_expedited,
- .name = "rcu_bh"
-};
-
/*
* Definitions for srcu perf testing.
*/
.name = "srcud"
};
-/*
- * Definitions for sched perf testing.
- */
-
-static int sched_perf_read_lock(void)
-{
- preempt_disable();
- return 0;
-}
-
-static void sched_perf_read_unlock(int idx)
-{
- preempt_enable();
-}
-
-static struct rcu_perf_ops sched_ops = {
- .ptype = RCU_SCHED_FLAVOR,
- .init = rcu_sync_perf_init,
- .readlock = sched_perf_read_lock,
- .readunlock = sched_perf_read_unlock,
- .get_gp_seq = rcu_sched_get_gp_seq,
- .gp_diff = rcu_seq_diff,
- .exp_completed = rcu_exp_batches_completed_sched,
- .async = call_rcu_sched,
- .gp_barrier = rcu_barrier_sched,
- .sync = synchronize_sched,
- .exp_sync = synchronize_sched_expedited,
- .name = "sched"
-};
-
/*
* Definitions for RCU-tasks perf testing.
*/
kfree(writer_n_durations);
}
- /* Do flavor-specific cleanup operations. */
+ /* Do torture-type-specific cleanup operations. */
if (cur_ops->cleanup != NULL)
cur_ops->cleanup();
long i;
int firsterr = 0;
static struct rcu_perf_ops *perf_ops[] = {
- &rcu_ops, &rcu_bh_ops, &srcu_ops, &srcud_ops, &sched_ops,
- &tasks_ops,
+ &rcu_ops, &srcu_ops, &srcud_ops, &tasks_ops,
};
if (!torture_init_begin(perf_type, verbose))
for (i = 0; i < ARRAY_SIZE(perf_ops); i++)
pr_cont(" %s", perf_ops[i]->name);
pr_cont("\n");
+ WARN_ON(!IS_MODULE(CONFIG_RCU_PERF_TEST));
firsterr = -EINVAL;
goto unwind;
}
/* Bits for ->extendables field, extendables param, and related definitions. */
#define RCUTORTURE_RDR_SHIFT 8 /* Put SRCU index in upper bits. */
#define RCUTORTURE_RDR_MASK ((1 << RCUTORTURE_RDR_SHIFT) - 1)
-#define RCUTORTURE_RDR_BH 0x1 /* Extend readers by disabling bh. */
-#define RCUTORTURE_RDR_IRQ 0x2 /* ... disabling interrupts. */
-#define RCUTORTURE_RDR_PREEMPT 0x4 /* ... disabling preemption. */
-#define RCUTORTURE_RDR_RCU 0x8 /* ... entering another RCU reader. */
-#define RCUTORTURE_RDR_NBITS 4 /* Number of bits defined above. */
-#define RCUTORTURE_MAX_EXTEND (RCUTORTURE_RDR_BH | RCUTORTURE_RDR_IRQ | \
- RCUTORTURE_RDR_PREEMPT)
+#define RCUTORTURE_RDR_BH 0x01 /* Extend readers by disabling bh. */
+#define RCUTORTURE_RDR_IRQ 0x02 /* ... disabling interrupts. */
+#define RCUTORTURE_RDR_PREEMPT 0x04 /* ... disabling preemption. */
+#define RCUTORTURE_RDR_RBH 0x08 /* ... rcu_read_lock_bh(). */
+#define RCUTORTURE_RDR_SCHED 0x10 /* ... rcu_read_lock_sched(). */
+#define RCUTORTURE_RDR_RCU 0x20 /* ... entering another RCU reader. */
+#define RCUTORTURE_RDR_NBITS 6 /* Number of bits defined above. */
+#define RCUTORTURE_MAX_EXTEND \
+ (RCUTORTURE_RDR_BH | RCUTORTURE_RDR_IRQ | RCUTORTURE_RDR_PREEMPT | \
+ RCUTORTURE_RDR_RBH | RCUTORTURE_RDR_SCHED)
#define RCUTORTURE_RDR_MAX_LOOPS 0x7 /* Maximum reader extensions. */
/* Must be power of two minus one. */
+#define RCUTORTURE_RDR_MAX_SEGS (RCUTORTURE_RDR_MAX_LOOPS + 3)
torture_param(int, cbflood_inter_holdoff, HZ,
"Holdoff between floods (jiffies)");
"Duration of fqs bursts (us), 0 to disable");
torture_param(int, fqs_holdoff, 0, "Holdoff time within fqs bursts (us)");
torture_param(int, fqs_stutter, 3, "Wait time between fqs bursts (s)");
+torture_param(bool, fwd_progress, 1, "Test grace-period forward progress");
+torture_param(int, fwd_progress_div, 4, "Fraction of CPU stall to wait");
+torture_param(int, fwd_progress_holdoff, 60,
+ "Time between forward-progress tests (s)");
+torture_param(bool, fwd_progress_need_resched, 1,
+ "Hide cond_resched() behind need_resched()");
torture_param(bool, gp_cond, false, "Use conditional/async GP wait primitives");
torture_param(bool, gp_exp, false, "Use expedited GP wait primitives");
torture_param(bool, gp_normal, false,
static char *torture_type = "rcu";
module_param(torture_type, charp, 0444);
-MODULE_PARM_DESC(torture_type, "Type of RCU to torture (rcu, rcu_bh, ...)");
+MODULE_PARM_DESC(torture_type, "Type of RCU to torture (rcu, srcu, ...)");
static int nrealreaders;
static int ncbflooders;
static struct task_struct *fqs_task;
static struct task_struct *boost_tasks[NR_CPUS];
static struct task_struct *stall_task;
+static struct task_struct *fwd_prog_task;
static struct task_struct **barrier_cbs_tasks;
static struct task_struct *barrier_task;
"RTWS_STOPPING",
};
+/* Record reader segment types and duration for first failing read. */
+struct rt_read_seg {
+ int rt_readstate;
+ unsigned long rt_delay_jiffies;
+ unsigned long rt_delay_ms;
+ unsigned long rt_delay_us;
+ bool rt_preempted;
+};
+static int err_segs_recorded;
+static struct rt_read_seg err_segs[RCUTORTURE_RDR_MAX_SEGS];
+static int rt_read_nsegs;
+
static const char *rcu_torture_writer_state_getname(void)
{
unsigned int i = READ_ONCE(rcu_torture_writer_state);
void (*init)(void);
void (*cleanup)(void);
int (*readlock)(void);
- void (*read_delay)(struct torture_random_state *rrsp);
+ void (*read_delay)(struct torture_random_state *rrsp,
+ struct rt_read_seg *rtrsp);
void (*readunlock)(int idx);
unsigned long (*get_gp_seq)(void);
unsigned long (*gp_diff)(unsigned long new, unsigned long old);
void (*cb_barrier)(void);
void (*fqs)(void);
void (*stats)(void);
+ int (*stall_dur)(void);
int irq_capable;
int can_boost;
int extendables;
return 0;
}
-static void rcu_read_delay(struct torture_random_state *rrsp)
+static void
+rcu_read_delay(struct torture_random_state *rrsp, struct rt_read_seg *rtrsp)
{
unsigned long started;
unsigned long completed;
const unsigned long shortdelay_us = 200;
- const unsigned long longdelay_ms = 50;
+ unsigned long longdelay_ms = 300;
unsigned long long ts;
/* We want a short delay sometimes to make a reader delay the grace
if (!(torture_random(rrsp) % (nrealreaders * 2000 * longdelay_ms))) {
started = cur_ops->get_gp_seq();
ts = rcu_trace_clock_local();
+ if (preempt_count() & (SOFTIRQ_MASK | HARDIRQ_MASK))
+ longdelay_ms = 5; /* Avoid triggering BH limits. */
mdelay(longdelay_ms);
+ rtrsp->rt_delay_ms = longdelay_ms;
completed = cur_ops->get_gp_seq();
do_trace_rcu_torture_read(cur_ops->name, NULL, ts,
started, completed);
}
- if (!(torture_random(rrsp) % (nrealreaders * 2 * shortdelay_us)))
+ if (!(torture_random(rrsp) % (nrealreaders * 2 * shortdelay_us))) {
udelay(shortdelay_us);
+ rtrsp->rt_delay_us = shortdelay_us;
+ }
if (!preempt_count() &&
- !(torture_random(rrsp) % (nrealreaders * 500)))
+ !(torture_random(rrsp) % (nrealreaders * 500))) {
torture_preempt_schedule(); /* QS only if preemptible. */
+ rtrsp->rt_preempted = true;
+ }
}
static void rcu_torture_read_unlock(int idx) __releases(RCU)
.cb_barrier = rcu_barrier,
.fqs = rcu_force_quiescent_state,
.stats = NULL,
+ .stall_dur = rcu_jiffies_till_stall_check,
.irq_capable = 1,
.can_boost = rcu_can_boost(),
+ .extendables = RCUTORTURE_MAX_EXTEND,
.name = "rcu"
};
-/*
- * Definitions for rcu_bh torture testing.
- */
-
-static int rcu_bh_torture_read_lock(void) __acquires(RCU_BH)
-{
- rcu_read_lock_bh();
- return 0;
-}
-
-static void rcu_bh_torture_read_unlock(int idx) __releases(RCU_BH)
-{
- rcu_read_unlock_bh();
-}
-
-static void rcu_bh_torture_deferred_free(struct rcu_torture *p)
-{
- call_rcu_bh(&p->rtort_rcu, rcu_torture_cb);
-}
-
-static struct rcu_torture_ops rcu_bh_ops = {
- .ttype = RCU_BH_FLAVOR,
- .init = rcu_sync_torture_init,
- .readlock = rcu_bh_torture_read_lock,
- .read_delay = rcu_read_delay, /* just reuse rcu's version. */
- .readunlock = rcu_bh_torture_read_unlock,
- .get_gp_seq = rcu_bh_get_gp_seq,
- .gp_diff = rcu_seq_diff,
- .deferred_free = rcu_bh_torture_deferred_free,
- .sync = synchronize_rcu_bh,
- .exp_sync = synchronize_rcu_bh_expedited,
- .call = call_rcu_bh,
- .cb_barrier = rcu_barrier_bh,
- .fqs = rcu_bh_force_quiescent_state,
- .stats = NULL,
- .irq_capable = 1,
- .extendables = (RCUTORTURE_RDR_BH | RCUTORTURE_RDR_IRQ),
- .ext_irq_conflict = RCUTORTURE_RDR_RCU,
- .name = "rcu_bh"
-};
-
/*
* Don't even think about trying any of these in real life!!!
* The names includes "busted", and they really means it!
return srcu_read_lock(srcu_ctlp);
}
-static void srcu_read_delay(struct torture_random_state *rrsp)
+static void
+srcu_read_delay(struct torture_random_state *rrsp, struct rt_read_seg *rtrsp)
{
long delay;
const long uspertick = 1000000 / HZ;
delay = torture_random(rrsp) %
(nrealreaders * 2 * longdelay * uspertick);
- if (!delay && in_task())
+ if (!delay && in_task()) {
schedule_timeout_interruptible(longdelay);
- else
- rcu_read_delay(rrsp);
+ rtrsp->rt_delay_jiffies = longdelay;
+ } else {
+ rcu_read_delay(rrsp, rtrsp);
+ }
}
static void srcu_torture_read_unlock(int idx) __releases(srcu_ctlp)
.name = "busted_srcud"
};
-/*
- * Definitions for sched torture testing.
- */
-
-static int sched_torture_read_lock(void)
-{
- preempt_disable();
- return 0;
-}
-
-static void sched_torture_read_unlock(int idx)
-{
- preempt_enable();
-}
-
-static void rcu_sched_torture_deferred_free(struct rcu_torture *p)
-{
- call_rcu_sched(&p->rtort_rcu, rcu_torture_cb);
-}
-
-static struct rcu_torture_ops sched_ops = {
- .ttype = RCU_SCHED_FLAVOR,
- .init = rcu_sync_torture_init,
- .readlock = sched_torture_read_lock,
- .read_delay = rcu_read_delay, /* just reuse rcu's version. */
- .readunlock = sched_torture_read_unlock,
- .get_gp_seq = rcu_sched_get_gp_seq,
- .gp_diff = rcu_seq_diff,
- .deferred_free = rcu_sched_torture_deferred_free,
- .sync = synchronize_sched,
- .exp_sync = synchronize_sched_expedited,
- .get_state = get_state_synchronize_sched,
- .cond_sync = cond_synchronize_sched,
- .call = call_rcu_sched,
- .cb_barrier = rcu_barrier_sched,
- .fqs = rcu_sched_force_quiescent_state,
- .stats = NULL,
- .irq_capable = 1,
- .extendables = RCUTORTURE_MAX_EXTEND,
- .name = "sched"
-};
-
/*
* Definitions for RCU-tasks torture testing.
*/
break;
}
}
- rcu_torture_current_version++;
+ WRITE_ONCE(rcu_torture_current_version,
+ rcu_torture_current_version + 1);
/* Cycle through nesting levels of rcu_expedite_gp() calls. */
if (can_expedite &&
!(torture_random(&rand) & 0xff & (!!expediting - 1))) {
!rcu_gp_is_normal();
}
rcu_torture_writer_state = RTWS_STUTTER;
- stutter_wait("rcu_torture_writer");
+ if (stutter_wait("rcu_torture_writer"))
+ for (i = 0; i < ARRAY_SIZE(rcu_tortures); i++)
+ if (list_empty(&rcu_tortures[i].rtort_free))
+ WARN_ON_ONCE(1);
} while (!torture_must_stop());
/* Reset expediting back to unexpedited. */
if (expediting > 0)
* change, do a ->read_delay().
*/
static void rcutorture_one_extend(int *readstate, int newstate,
- struct torture_random_state *trsp)
+ struct torture_random_state *trsp,
+ struct rt_read_seg *rtrsp)
{
int idxnew = -1;
int idxold = *readstate;
WARN_ON_ONCE(idxold < 0);
WARN_ON_ONCE((idxold >> RCUTORTURE_RDR_SHIFT) > 1);
+ rtrsp->rt_readstate = newstate;
/* First, put new protection in place to avoid critical-section gap. */
if (statesnew & RCUTORTURE_RDR_BH)
local_irq_disable();
if (statesnew & RCUTORTURE_RDR_PREEMPT)
preempt_disable();
+ if (statesnew & RCUTORTURE_RDR_RBH)
+ rcu_read_lock_bh();
+ if (statesnew & RCUTORTURE_RDR_SCHED)
+ rcu_read_lock_sched();
if (statesnew & RCUTORTURE_RDR_RCU)
idxnew = cur_ops->readlock() << RCUTORTURE_RDR_SHIFT;
local_bh_enable();
if (statesold & RCUTORTURE_RDR_PREEMPT)
preempt_enable();
+ if (statesold & RCUTORTURE_RDR_RBH)
+ rcu_read_unlock_bh();
+ if (statesold & RCUTORTURE_RDR_SCHED)
+ rcu_read_unlock_sched();
if (statesold & RCUTORTURE_RDR_RCU)
cur_ops->readunlock(idxold >> RCUTORTURE_RDR_SHIFT);
/* Delay if neither beginning nor end and there was a change. */
if ((statesnew || statesold) && *readstate && newstate)
- cur_ops->read_delay(trsp);
+ cur_ops->read_delay(trsp, rtrsp);
/* Update the reader state. */
if (idxnew == -1)
{
int mask = rcutorture_extend_mask_max();
unsigned long randmask1 = torture_random(trsp) >> 8;
- unsigned long randmask2 = randmask1 >> 1;
+ unsigned long randmask2 = randmask1 >> 3;
WARN_ON_ONCE(mask >> RCUTORTURE_RDR_SHIFT);
- /* Half the time lots of bits, half the time only one bit. */
- if (randmask1 & 0x1)
+ /* Most of the time lots of bits, half the time only one bit. */
+ if (!(randmask1 & 0x7))
mask = mask & randmask2;
else
mask = mask & (1 << (randmask2 % RCUTORTURE_RDR_NBITS));
+ /* Can't enable bh w/irq disabled. */
if ((mask & RCUTORTURE_RDR_IRQ) &&
- !(mask & RCUTORTURE_RDR_BH) &&
- (oldmask & RCUTORTURE_RDR_BH))
- mask |= RCUTORTURE_RDR_BH; /* Can't enable bh w/irq disabled. */
+ ((!(mask & RCUTORTURE_RDR_BH) && (oldmask & RCUTORTURE_RDR_BH)) ||
+ (!(mask & RCUTORTURE_RDR_RBH) && (oldmask & RCUTORTURE_RDR_RBH))))
+ mask |= RCUTORTURE_RDR_BH | RCUTORTURE_RDR_RBH;
if ((mask & RCUTORTURE_RDR_IRQ) &&
!(mask & cur_ops->ext_irq_conflict) &&
(oldmask & cur_ops->ext_irq_conflict))
* Do a randomly selected number of extensions of an existing RCU read-side
* critical section.
*/
-static void rcutorture_loop_extend(int *readstate,
- struct torture_random_state *trsp)
+static struct rt_read_seg *
+rcutorture_loop_extend(int *readstate, struct torture_random_state *trsp,
+ struct rt_read_seg *rtrsp)
{
int i;
+ int j;
int mask = rcutorture_extend_mask_max();
WARN_ON_ONCE(!*readstate); /* -Existing- RCU read-side critsect! */
if (!((mask - 1) & mask))
- return; /* Current RCU flavor not extendable. */
- i = (torture_random(trsp) >> 3) & RCUTORTURE_RDR_MAX_LOOPS;
- while (i--) {
+ return rtrsp; /* Current RCU reader not extendable. */
+ /* Bias towards larger numbers of loops. */
+ i = (torture_random(trsp) >> 3);
+ i = ((i | (i >> 3)) & RCUTORTURE_RDR_MAX_LOOPS) + 1;
+ for (j = 0; j < i; j++) {
mask = rcutorture_extend_mask(*readstate, trsp);
- rcutorture_one_extend(readstate, mask, trsp);
+ rcutorture_one_extend(readstate, mask, trsp, &rtrsp[j]);
}
+ return &rtrsp[j];
}
/*
*/
static bool rcu_torture_one_read(struct torture_random_state *trsp)
{
+ int i;
unsigned long started;
unsigned long completed;
int newstate;
struct rcu_torture *p;
int pipe_count;
int readstate = 0;
+ struct rt_read_seg rtseg[RCUTORTURE_RDR_MAX_SEGS] = { { 0 } };
+ struct rt_read_seg *rtrsp = &rtseg[0];
+ struct rt_read_seg *rtrsp1;
unsigned long long ts;
newstate = rcutorture_extend_mask(readstate, trsp);
- rcutorture_one_extend(&readstate, newstate, trsp);
+ rcutorture_one_extend(&readstate, newstate, trsp, rtrsp++);
started = cur_ops->get_gp_seq();
ts = rcu_trace_clock_local();
p = rcu_dereference_check(rcu_torture_current,
torturing_tasks());
if (p == NULL) {
/* Wait for rcu_torture_writer to get underway */
- rcutorture_one_extend(&readstate, 0, trsp);
+ rcutorture_one_extend(&readstate, 0, trsp, rtrsp);
return false;
}
if (p->rtort_mbtest == 0)
atomic_inc(&n_rcu_torture_mberror);
- rcutorture_loop_extend(&readstate, trsp);
+ rtrsp = rcutorture_loop_extend(&readstate, trsp, rtrsp);
preempt_disable();
pipe_count = p->rtort_pipe_count;
if (pipe_count > RCU_TORTURE_PIPE_LEN) {
}
__this_cpu_inc(rcu_torture_batch[completed]);
preempt_enable();
- rcutorture_one_extend(&readstate, 0, trsp);
+ rcutorture_one_extend(&readstate, 0, trsp, rtrsp);
WARN_ON_ONCE(readstate & RCUTORTURE_RDR_MASK);
+
+ /* If error or close call, record the sequence of reader protections. */
+ if ((pipe_count > 1 || completed > 1) && !xchg(&err_segs_recorded, 1)) {
+ i = 0;
+ for (rtrsp1 = &rtseg[0]; rtrsp1 < rtrsp; rtrsp1++)
+ err_segs[i++] = *rtrsp1;
+ rt_read_nsegs = i;
+ }
+
return true;
}
static int
rcu_torture_reader(void *arg)
{
+ unsigned long lastsleep = jiffies;
+ long myid = (long)arg;
+ int mynumonline = myid;
DEFINE_TORTURE_RANDOM(rand);
struct timer_list t;
}
if (!rcu_torture_one_read(&rand))
schedule_timeout_interruptible(HZ);
+ if (time_after(jiffies, lastsleep)) {
+ schedule_timeout_interruptible(1);
+ lastsleep = jiffies + 10;
+ }
+ while (num_online_cpus() < mynumonline && !torture_must_stop())
+ schedule_timeout_interruptible(HZ / 5);
stutter_wait("rcu_torture_reader");
} while (!torture_must_stop());
if (irqreader && cur_ops->irq_capable) {
return torture_create_kthread(rcu_torture_stall, NULL, stall_task);
}
+/* State structure for forward-progress self-propagating RCU callback. */
+struct fwd_cb_state {
+ struct rcu_head rh;
+ int stop;
+};
+
+/*
+ * Forward-progress self-propagating RCU callback function. Because
+ * callbacks run from softirq, this function is an implicit RCU read-side
+ * critical section.
+ */
+static void rcu_torture_fwd_prog_cb(struct rcu_head *rhp)
+{
+ struct fwd_cb_state *fcsp = container_of(rhp, struct fwd_cb_state, rh);
+
+ if (READ_ONCE(fcsp->stop)) {
+ WRITE_ONCE(fcsp->stop, 2);
+ return;
+ }
+ cur_ops->call(&fcsp->rh, rcu_torture_fwd_prog_cb);
+}
+
+/* Carry out grace-period forward-progress testing. */
+static int rcu_torture_fwd_prog(void *args)
+{
+ unsigned long cver;
+ unsigned long dur;
+ struct fwd_cb_state fcs;
+ unsigned long gps;
+ int idx;
+ int sd;
+ int sd4;
+ bool selfpropcb = false;
+ unsigned long stopat;
+ int tested = 0;
+ int tested_tries = 0;
+ static DEFINE_TORTURE_RANDOM(trs);
+
+ VERBOSE_TOROUT_STRING("rcu_torture_fwd_progress task started");
+ if (!IS_ENABLED(CONFIG_SMP) || !IS_ENABLED(CONFIG_RCU_BOOST))
+ set_user_nice(current, MAX_NICE);
+ if (cur_ops->call && cur_ops->sync && cur_ops->cb_barrier) {
+ init_rcu_head_on_stack(&fcs.rh);
+ selfpropcb = true;
+ }
+ do {
+ schedule_timeout_interruptible(fwd_progress_holdoff * HZ);
+ if (selfpropcb) {
+ WRITE_ONCE(fcs.stop, 0);
+ cur_ops->call(&fcs.rh, rcu_torture_fwd_prog_cb);
+ }
+ cver = READ_ONCE(rcu_torture_current_version);
+ gps = cur_ops->get_gp_seq();
+ sd = cur_ops->stall_dur() + 1;
+ sd4 = (sd + fwd_progress_div - 1) / fwd_progress_div;
+ dur = sd4 + torture_random(&trs) % (sd - sd4);
+ stopat = jiffies + dur;
+ while (time_before(jiffies, stopat) && !torture_must_stop()) {
+ idx = cur_ops->readlock();
+ udelay(10);
+ cur_ops->readunlock(idx);
+ if (!fwd_progress_need_resched || need_resched())
+ cond_resched();
+ }
+ tested_tries++;
+ if (!time_before(jiffies, stopat) && !torture_must_stop()) {
+ tested++;
+ cver = READ_ONCE(rcu_torture_current_version) - cver;
+ gps = rcutorture_seq_diff(cur_ops->get_gp_seq(), gps);
+ WARN_ON(!cver && gps < 2);
+ pr_alert("%s: Duration %ld cver %ld gps %ld\n", __func__, dur, cver, gps);
+ }
+ if (selfpropcb) {
+ WRITE_ONCE(fcs.stop, 1);
+ cur_ops->sync(); /* Wait for running CB to complete. */
+ cur_ops->cb_barrier(); /* Wait for queued callbacks. */
+ }
+ /* Avoid slow periods, better to test when busy. */
+ stutter_wait("rcu_torture_fwd_prog");
+ } while (!torture_must_stop());
+ if (selfpropcb) {
+ WARN_ON(READ_ONCE(fcs.stop) != 2);
+ destroy_rcu_head_on_stack(&fcs.rh);
+ }
+ /* Short runs might not contain a valid forward-progress attempt. */
+ WARN_ON(!tested && tested_tries >= 5);
+ pr_alert("%s: tested %d tested_tries %d\n", __func__, tested, tested_tries);
+ torture_kthread_stopping("rcu_torture_fwd_prog");
+ return 0;
+}
+
+/* If forward-progress checking is requested and feasible, spawn the thread. */
+static int __init rcu_torture_fwd_prog_init(void)
+{
+ if (!fwd_progress)
+ return 0; /* Not requested, so don't do it. */
+ if (!cur_ops->stall_dur || cur_ops->stall_dur() <= 0) {
+ VERBOSE_TOROUT_STRING("rcu_torture_fwd_prog_init: Disabled, unsupported by RCU flavor under test");
+ return 0;
+ }
+ if (stall_cpu > 0) {
+ VERBOSE_TOROUT_STRING("rcu_torture_fwd_prog_init: Disabled, conflicts with CPU-stall testing");
+ if (IS_MODULE(CONFIG_RCU_TORTURE_TESTS))
+ return -EINVAL; /* In module, can fail back to user. */
+ WARN_ON(1); /* Make sure rcutorture notices conflict. */
+ return 0;
+ }
+ if (fwd_progress_holdoff <= 0)
+ fwd_progress_holdoff = 1;
+ if (fwd_progress_div <= 0)
+ fwd_progress_div = 4;
+ return torture_create_kthread(rcu_torture_fwd_prog,
+ NULL, fwd_prog_task);
+}
+
/* Callback function for RCU barrier testing. */
static void rcu_torture_barrier_cbf(struct rcu_head *rcu)
{
static void
rcu_torture_cleanup(void)
{
+ int firsttime;
int flags = 0;
unsigned long gp_seq = 0;
int i;
}
rcu_torture_barrier_cleanup();
+ torture_stop_kthread(rcu_torture_fwd_prog, fwd_prog_task);
torture_stop_kthread(rcu_torture_stall, stall_task);
torture_stop_kthread(rcu_torture_writer, writer_task);
cpuhp_remove_state(rcutor_hp);
/*
- * Wait for all RCU callbacks to fire, then do flavor-specific
+ * Wait for all RCU callbacks to fire, then do torture-type-specific
* cleanup operations.
*/
if (cur_ops->cb_barrier != NULL)
rcu_torture_stats_print(); /* -After- the stats thread is stopped! */
+ if (err_segs_recorded) {
+ pr_alert("Failure/close-call rcutorture reader segments:\n");
+ if (rt_read_nsegs == 0)
+ pr_alert("\t: No segments recorded!!!\n");
+ firsttime = 1;
+ for (i = 0; i < rt_read_nsegs; i++) {
+ pr_alert("\t%d: %#x ", i, err_segs[i].rt_readstate);
+ if (err_segs[i].rt_delay_jiffies != 0) {
+ pr_cont("%s%ldjiffies", firsttime ? "" : "+",
+ err_segs[i].rt_delay_jiffies);
+ firsttime = 0;
+ }
+ if (err_segs[i].rt_delay_ms != 0) {
+ pr_cont("%s%ldms", firsttime ? "" : "+",
+ err_segs[i].rt_delay_ms);
+ firsttime = 0;
+ }
+ if (err_segs[i].rt_delay_us != 0) {
+ pr_cont("%s%ldus", firsttime ? "" : "+",
+ err_segs[i].rt_delay_us);
+ firsttime = 0;
+ }
+ pr_cont("%s\n",
+ err_segs[i].rt_preempted ? "preempted" : "");
+
+ }
+ }
if (atomic_read(&n_rcu_torture_error) || n_rcu_torture_barrier_error)
rcu_torture_print_module_parms(cur_ops, "End of test: FAILURE");
else if (torture_onoff_failures())
static int __init
rcu_torture_init(void)
{
- int i;
+ long i;
int cpu;
int firsterr = 0;
static struct rcu_torture_ops *torture_ops[] = {
- &rcu_ops, &rcu_bh_ops, &rcu_busted_ops, &srcu_ops, &srcud_ops,
- &busted_srcud_ops, &sched_ops, &tasks_ops,
+ &rcu_ops, &rcu_busted_ops, &srcu_ops, &srcud_ops,
+ &busted_srcud_ops, &tasks_ops,
};
if (!torture_init_begin(torture_type, verbose))
for (i = 0; i < ARRAY_SIZE(torture_ops); i++)
pr_cont(" %s", torture_ops[i]->name);
pr_cont("\n");
+ WARN_ON(!IS_MODULE(CONFIG_RCU_TORTURE_TEST));
firsterr = -EINVAL;
goto unwind;
}
per_cpu(rcu_torture_batch, cpu)[i] = 0;
}
}
+ err_segs_recorded = 0;
+ rt_read_nsegs = 0;
/* Start up the kthreads. */
goto unwind;
}
for (i = 0; i < nrealreaders; i++) {
- firsterr = torture_create_kthread(rcu_torture_reader, NULL,
+ firsterr = torture_create_kthread(rcu_torture_reader, (void *)i,
reader_tasks[i]);
if (firsterr)
goto unwind;
if (firsterr)
goto unwind;
firsterr = rcu_torture_stall_init();
+ if (firsterr)
+ goto unwind;
+ firsterr = rcu_torture_fwd_prog_init();
if (firsterr)
goto unwind;
firsterr = rcu_torture_barrier_init();
#include "rcu.h"
int rcu_scheduler_active __read_mostly;
+static LIST_HEAD(srcu_boot_list);
+static bool srcu_init_done;
static int init_srcu_struct_fields(struct srcu_struct *sp)
{
sp->srcu_gp_waiting = false;
sp->srcu_idx = 0;
INIT_WORK(&sp->srcu_work, srcu_drive_gp);
+ INIT_LIST_HEAD(&sp->srcu_work.entry);
return 0;
}
*sp->srcu_cb_tail = rhp;
sp->srcu_cb_tail = &rhp->next;
local_irq_restore(flags);
- if (!READ_ONCE(sp->srcu_gp_running))
- schedule_work(&sp->srcu_work);
+ if (!READ_ONCE(sp->srcu_gp_running)) {
+ if (likely(srcu_init_done))
+ schedule_work(&sp->srcu_work);
+ else if (list_empty(&sp->srcu_work.entry))
+ list_add(&sp->srcu_work.entry, &srcu_boot_list);
+ }
}
EXPORT_SYMBOL_GPL(call_srcu);
{
rcu_scheduler_active = RCU_SCHEDULER_RUNNING;
}
+
+/*
+ * Queue work for srcu_struct structures with early boot callbacks.
+ * The work won't actually execute until the workqueue initialization
+ * phase that takes place after the scheduler starts.
+ */
+void __init srcu_init(void)
+{
+ struct srcu_struct *sp;
+
+ srcu_init_done = true;
+ while (!list_empty(&srcu_boot_list)) {
+ sp = list_first_entry(&srcu_boot_list,
+ struct srcu_struct, srcu_work.entry);
+ list_del_init(&sp->srcu_work.entry);
+ schedule_work(&sp->srcu_work);
+ }
+}
static ulong counter_wrap_check = (ULONG_MAX >> 2);
module_param(counter_wrap_check, ulong, 0444);
+/* Early-boot callback-management, so early that no lock is required! */
+static LIST_HEAD(srcu_boot_list);
+static bool __read_mostly srcu_init_done;
+
static void srcu_invoke_callbacks(struct work_struct *work);
static void srcu_reschedule(struct srcu_struct *sp, unsigned long delay);
static void process_srcu(struct work_struct *work);
rcu_init_levelspread(levelspread, num_rcu_lvl);
/* Each pass through this loop initializes one srcu_node structure. */
- rcu_for_each_node_breadth_first(sp, snp) {
+ srcu_for_each_node_breadth_first(sp, snp) {
spin_lock_init(&ACCESS_PRIVATE(snp, lock));
WARN_ON_ONCE(ARRAY_SIZE(snp->srcu_have_cbs) !=
ARRAY_SIZE(snp->srcu_data_have_cbs));
{
unsigned long flags;
- WARN_ON_ONCE(rcu_scheduler_active == RCU_SCHEDULER_INIT);
/* The smp_load_acquire() pairs with the smp_store_release(). */
if (!rcu_seq_state(smp_load_acquire(&sp->srcu_gp_seq_needed))) /*^^^*/
return; /* Already initialized. */
/* Initiate callback invocation as needed. */
idx = rcu_seq_ctr(gpseq) % ARRAY_SIZE(snp->srcu_have_cbs);
- rcu_for_each_node_breadth_first(sp, snp) {
+ srcu_for_each_node_breadth_first(sp, snp) {
spin_lock_irq_rcu_node(snp);
cbs = false;
last_lvl = snp >= sp->level[rcu_num_lvls - 1];
rcu_seq_state(sp->srcu_gp_seq) == SRCU_STATE_IDLE) {
WARN_ON_ONCE(ULONG_CMP_GE(sp->srcu_gp_seq, sp->srcu_gp_seq_needed));
srcu_gp_start(sp);
- queue_delayed_work(rcu_gp_wq, &sp->work, srcu_get_delay(sp));
+ if (likely(srcu_init_done))
+ queue_delayed_work(rcu_gp_wq, &sp->work,
+ srcu_get_delay(sp));
+ else if (list_empty(&sp->work.work.entry))
+ list_add(&sp->work.work.entry, &srcu_boot_list);
}
spin_unlock_irqrestore_rcu_node(sp, flags);
}
* There are memory-ordering constraints implied by synchronize_srcu().
* On systems with more than one CPU, when synchronize_srcu() returns,
* each CPU is guaranteed to have executed a full memory barrier since
- * the end of its last corresponding SRCU-sched read-side critical section
+ * the end of its last corresponding SRCU read-side critical section
* whose beginning preceded the call to synchronize_srcu(). In addition,
* each CPU having an SRCU read-side critical section that extends beyond
* the return from synchronize_srcu() is guaranteed to have executed a
return 0;
}
early_initcall(srcu_bootup_announce);
+
+void __init srcu_init(void)
+{
+ struct srcu_struct *sp;
+
+ srcu_init_done = true;
+ while (!list_empty(&srcu_boot_list)) {
+ sp = list_first_entry(&srcu_boot_list, struct srcu_struct,
+ work.work.entry);
+ check_init_srcu_struct(sp);
+ list_del_init(&sp->work.work.entry);
+ queue_work(rcu_gp_wq, &sp->work.work);
+ }
+}
};
/* Definition for rcupdate control block. */
-static struct rcu_ctrlblk rcu_sched_ctrlblk = {
- .donetail = &rcu_sched_ctrlblk.rcucblist,
- .curtail = &rcu_sched_ctrlblk.rcucblist,
+static struct rcu_ctrlblk rcu_ctrlblk = {
+ .donetail = &rcu_ctrlblk.rcucblist,
+ .curtail = &rcu_ctrlblk.rcucblist,
};
-static struct rcu_ctrlblk rcu_bh_ctrlblk = {
- .donetail = &rcu_bh_ctrlblk.rcucblist,
- .curtail = &rcu_bh_ctrlblk.rcucblist,
-};
-
-void rcu_barrier_bh(void)
-{
- wait_rcu_gp(call_rcu_bh);
-}
-EXPORT_SYMBOL(rcu_barrier_bh);
-
-void rcu_barrier_sched(void)
-{
- wait_rcu_gp(call_rcu_sched);
-}
-EXPORT_SYMBOL(rcu_barrier_sched);
-
-/*
- * Helper function for rcu_sched_qs() and rcu_bh_qs().
- * Also irqs are disabled to avoid confusion due to interrupt handlers
- * invoking call_rcu().
- */
-static int rcu_qsctr_help(struct rcu_ctrlblk *rcp)
-{
- if (rcp->donetail != rcp->curtail) {
- rcp->donetail = rcp->curtail;
- return 1;
- }
-
- return 0;
-}
-
-/*
- * Record an rcu quiescent state. And an rcu_bh quiescent state while we
- * are at it, given that any rcu quiescent state is also an rcu_bh
- * quiescent state. Use "+" instead of "||" to defeat short circuiting.
- */
-void rcu_sched_qs(void)
+void rcu_barrier(void)
{
- unsigned long flags;
-
- local_irq_save(flags);
- if (rcu_qsctr_help(&rcu_sched_ctrlblk) +
- rcu_qsctr_help(&rcu_bh_ctrlblk))
- raise_softirq(RCU_SOFTIRQ);
- local_irq_restore(flags);
+ wait_rcu_gp(call_rcu);
}
+EXPORT_SYMBOL(rcu_barrier);
-/*
- * Record an rcu_bh quiescent state.
- */
-void rcu_bh_qs(void)
+/* Record an rcu quiescent state. */
+void rcu_qs(void)
{
unsigned long flags;
local_irq_save(flags);
- if (rcu_qsctr_help(&rcu_bh_ctrlblk))
+ if (rcu_ctrlblk.donetail != rcu_ctrlblk.curtail) {
+ rcu_ctrlblk.donetail = rcu_ctrlblk.curtail;
raise_softirq(RCU_SOFTIRQ);
+ }
local_irq_restore(flags);
}
*/
void rcu_check_callbacks(int user)
{
- if (user)
- rcu_sched_qs();
- if (user || !in_softirq())
- rcu_bh_qs();
+ if (user) {
+ rcu_qs();
+ } else if (rcu_ctrlblk.donetail != rcu_ctrlblk.curtail) {
+ set_tsk_need_resched(current);
+ set_preempt_need_resched();
+ }
}
-/*
- * Invoke the RCU callbacks on the specified rcu_ctrlkblk structure
- * whose grace period has elapsed.
- */
-static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp)
+/* Invoke the RCU callbacks whose grace period has elapsed. */
+static __latent_entropy void rcu_process_callbacks(struct softirq_action *unused)
{
struct rcu_head *next, *list;
unsigned long flags;
/* Move the ready-to-invoke callbacks to a local list. */
local_irq_save(flags);
- if (rcp->donetail == &rcp->rcucblist) {
+ if (rcu_ctrlblk.donetail == &rcu_ctrlblk.rcucblist) {
/* No callbacks ready, so just leave. */
local_irq_restore(flags);
return;
}
- list = rcp->rcucblist;
- rcp->rcucblist = *rcp->donetail;
- *rcp->donetail = NULL;
- if (rcp->curtail == rcp->donetail)
- rcp->curtail = &rcp->rcucblist;
- rcp->donetail = &rcp->rcucblist;
+ list = rcu_ctrlblk.rcucblist;
+ rcu_ctrlblk.rcucblist = *rcu_ctrlblk.donetail;
+ *rcu_ctrlblk.donetail = NULL;
+ if (rcu_ctrlblk.curtail == rcu_ctrlblk.donetail)
+ rcu_ctrlblk.curtail = &rcu_ctrlblk.rcucblist;
+ rcu_ctrlblk.donetail = &rcu_ctrlblk.rcucblist;
local_irq_restore(flags);
/* Invoke the callbacks on the local list. */
}
}
-static __latent_entropy void rcu_process_callbacks(struct softirq_action *unused)
-{
- __rcu_process_callbacks(&rcu_sched_ctrlblk);
- __rcu_process_callbacks(&rcu_bh_ctrlblk);
-}
-
/*
* Wait for a grace period to elapse. But it is illegal to invoke
- * synchronize_sched() from within an RCU read-side critical section.
- * Therefore, any legal call to synchronize_sched() is a quiescent
- * state, and so on a UP system, synchronize_sched() need do nothing.
- * Ditto for synchronize_rcu_bh(). (But Lai Jiangshan points out the
- * benefits of doing might_sleep() to reduce latency.)
+ * synchronize_rcu() from within an RCU read-side critical section.
+ * Therefore, any legal call to synchronize_rcu() is a quiescent
+ * state, and so on a UP system, synchronize_rcu() need do nothing.
+ * (But Lai Jiangshan points out the benefits of doing might_sleep()
+ * to reduce latency.)
*
* Cool, huh? (Due to Josh Triplett.)
*/
-void synchronize_sched(void)
+void synchronize_rcu(void)
{
RCU_LOCKDEP_WARN(lock_is_held(&rcu_bh_lock_map) ||
lock_is_held(&rcu_lock_map) ||
lock_is_held(&rcu_sched_lock_map),
- "Illegal synchronize_sched() in RCU read-side critical section");
+ "Illegal synchronize_rcu() in RCU read-side critical section");
}
-EXPORT_SYMBOL_GPL(synchronize_sched);
+EXPORT_SYMBOL_GPL(synchronize_rcu);
/*
- * Helper function for call_rcu() and call_rcu_bh().
+ * Post an RCU callback to be invoked after the end of an RCU grace
+ * period. But since we have but one CPU, that would be after any
+ * quiescent state.
*/
-static void __call_rcu(struct rcu_head *head,
- rcu_callback_t func,
- struct rcu_ctrlblk *rcp)
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
{
unsigned long flags;
head->next = NULL;
local_irq_save(flags);
- *rcp->curtail = head;
- rcp->curtail = &head->next;
+ *rcu_ctrlblk.curtail = head;
+ rcu_ctrlblk.curtail = &head->next;
local_irq_restore(flags);
if (unlikely(is_idle_task(current))) {
- /* force scheduling for rcu_sched_qs() */
+ /* force scheduling for rcu_qs() */
resched_cpu(0);
}
}
-
-/*
- * Post an RCU callback to be invoked after the end of an RCU-sched grace
- * period. But since we have but one CPU, that would be after any
- * quiescent state.
- */
-void call_rcu_sched(struct rcu_head *head, rcu_callback_t func)
-{
- __call_rcu(head, func, &rcu_sched_ctrlblk);
-}
-EXPORT_SYMBOL_GPL(call_rcu_sched);
-
-/*
- * Post an RCU bottom-half callback to be invoked after any subsequent
- * quiescent state.
- */
-void call_rcu_bh(struct rcu_head *head, rcu_callback_t func)
-{
- __call_rcu(head, func, &rcu_bh_ctrlblk);
-}
-EXPORT_SYMBOL_GPL(call_rcu_bh);
+EXPORT_SYMBOL_GPL(call_rcu);
void __init rcu_init(void)
{
open_softirq(RCU_SOFTIRQ, rcu_process_callbacks);
rcu_early_boot_tests();
+ srcu_init();
}
#include <linux/trace_events.h>
#include <linux/suspend.h>
#include <linux/ftrace.h>
+#include <linux/tick.h>
#include "tree.h"
#include "rcu.h"
/* Data structures. */
/*
- * In order to export the rcu_state name to the tracing tools, it
- * needs to be added in the __tracepoint_string section.
- * This requires defining a separate variable tp_<sname>_varname
- * that points to the string being used, and this will allow
- * the tracing userspace tools to be able to decipher the string
- * address to the matching string.
+ * Steal a bit from the bottom of ->dynticks for idle entry/exit
+ * control. Initially this is for TLB flushing.
*/
-#ifdef CONFIG_TRACING
-# define DEFINE_RCU_TPS(sname) \
-static char sname##_varname[] = #sname; \
-static const char *tp_##sname##_varname __used __tracepoint_string = sname##_varname;
-# define RCU_STATE_NAME(sname) sname##_varname
-#else
-# define DEFINE_RCU_TPS(sname)
-# define RCU_STATE_NAME(sname) __stringify(sname)
+#define RCU_DYNTICK_CTRL_MASK 0x1
+#define RCU_DYNTICK_CTRL_CTR (RCU_DYNTICK_CTRL_MASK + 1)
+#ifndef rcu_eqs_special_exit
+#define rcu_eqs_special_exit() do { } while (0)
#endif
-#define RCU_STATE_INITIALIZER(sname, sabbr, cr) \
-DEFINE_RCU_TPS(sname) \
-static DEFINE_PER_CPU_SHARED_ALIGNED(struct rcu_data, sname##_data); \
-struct rcu_state sname##_state = { \
- .level = { &sname##_state.node[0] }, \
- .rda = &sname##_data, \
- .call = cr, \
- .gp_state = RCU_GP_IDLE, \
- .gp_seq = (0UL - 300UL) << RCU_SEQ_CTR_SHIFT, \
- .barrier_mutex = __MUTEX_INITIALIZER(sname##_state.barrier_mutex), \
- .name = RCU_STATE_NAME(sname), \
- .abbr = sabbr, \
- .exp_mutex = __MUTEX_INITIALIZER(sname##_state.exp_mutex), \
- .exp_wake_mutex = __MUTEX_INITIALIZER(sname##_state.exp_wake_mutex), \
- .ofl_lock = __SPIN_LOCK_UNLOCKED(sname##_state.ofl_lock), \
-}
-
-RCU_STATE_INITIALIZER(rcu_sched, 's', call_rcu_sched);
-RCU_STATE_INITIALIZER(rcu_bh, 'b', call_rcu_bh);
-
-static struct rcu_state *const rcu_state_p;
-LIST_HEAD(rcu_struct_flavors);
+static DEFINE_PER_CPU_SHARED_ALIGNED(struct rcu_data, rcu_data) = {
+ .dynticks_nesting = 1,
+ .dynticks_nmi_nesting = DYNTICK_IRQ_NONIDLE,
+ .dynticks = ATOMIC_INIT(RCU_DYNTICK_CTRL_CTR),
+};
+struct rcu_state rcu_state = {
+ .level = { &rcu_state.node[0] },
+ .gp_state = RCU_GP_IDLE,
+ .gp_seq = (0UL - 300UL) << RCU_SEQ_CTR_SHIFT,
+ .barrier_mutex = __MUTEX_INITIALIZER(rcu_state.barrier_mutex),
+ .name = RCU_NAME,
+ .abbr = RCU_ABBR,
+ .exp_mutex = __MUTEX_INITIALIZER(rcu_state.exp_mutex),
+ .exp_wake_mutex = __MUTEX_INITIALIZER(rcu_state.exp_wake_mutex),
+ .ofl_lock = __RAW_SPIN_LOCK_UNLOCKED(rcu_state.ofl_lock),
+};
/* Dump rcu_node combining tree at boot to verify correct setup. */
static bool dump_tree;
*/
static int rcu_scheduler_fully_active __read_mostly;
-static void
-rcu_report_qs_rnp(unsigned long mask, struct rcu_state *rsp,
- struct rcu_node *rnp, unsigned long gps, unsigned long flags);
+static void rcu_report_qs_rnp(unsigned long mask, struct rcu_node *rnp,
+ unsigned long gps, unsigned long flags);
static void rcu_init_new_rnp(struct rcu_node *rnp_leaf);
static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf);
static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu);
static void invoke_rcu_core(void);
-static void invoke_rcu_callbacks(struct rcu_state *rsp, struct rcu_data *rdp);
-static void rcu_report_exp_rdp(struct rcu_state *rsp,
- struct rcu_data *rdp, bool wake);
+static void invoke_rcu_callbacks(struct rcu_data *rdp);
+static void rcu_report_exp_rdp(struct rcu_data *rdp);
static void sync_sched_exp_online_cleanup(int cpu);
/* rcuc/rcub kthread realtime priority */
static int gp_cleanup_delay;
module_param(gp_cleanup_delay, int, 0444);
-/* Retreive RCU kthreads priority for rcutorture */
+/* Retrieve RCU kthreads priority for rcutorture */
int rcu_get_gp_kthreads_prio(void)
{
return kthread_prio;
* permit this function to be invoked without holding the root rcu_node
* structure's ->lock, but of course results can be subject to change.
*/
-static int rcu_gp_in_progress(struct rcu_state *rsp)
+static int rcu_gp_in_progress(void)
{
- return rcu_seq_state(rcu_seq_current(&rsp->gp_seq));
-}
-
-/*
- * Note a quiescent state. Because we do not need to know
- * how many quiescent states passed, just if there was at least
- * one since the start of the grace period, this just sets a flag.
- * The caller must have disabled preemption.
- */
-void rcu_sched_qs(void)
-{
- RCU_LOCKDEP_WARN(preemptible(), "rcu_sched_qs() invoked with preemption enabled!!!");
- if (!__this_cpu_read(rcu_sched_data.cpu_no_qs.s))
- return;
- trace_rcu_grace_period(TPS("rcu_sched"),
- __this_cpu_read(rcu_sched_data.gp_seq),
- TPS("cpuqs"));
- __this_cpu_write(rcu_sched_data.cpu_no_qs.b.norm, false);
- if (!__this_cpu_read(rcu_sched_data.cpu_no_qs.b.exp))
- return;
- __this_cpu_write(rcu_sched_data.cpu_no_qs.b.exp, false);
- rcu_report_exp_rdp(&rcu_sched_state,
- this_cpu_ptr(&rcu_sched_data), true);
+ return rcu_seq_state(rcu_seq_current(&rcu_state.gp_seq));
}
-void rcu_bh_qs(void)
+void rcu_softirq_qs(void)
{
- RCU_LOCKDEP_WARN(preemptible(), "rcu_bh_qs() invoked with preemption enabled!!!");
- if (__this_cpu_read(rcu_bh_data.cpu_no_qs.s)) {
- trace_rcu_grace_period(TPS("rcu_bh"),
- __this_cpu_read(rcu_bh_data.gp_seq),
- TPS("cpuqs"));
- __this_cpu_write(rcu_bh_data.cpu_no_qs.b.norm, false);
- }
+ rcu_qs();
+ rcu_preempt_deferred_qs(current);
}
-/*
- * Steal a bit from the bottom of ->dynticks for idle entry/exit
- * control. Initially this is for TLB flushing.
- */
-#define RCU_DYNTICK_CTRL_MASK 0x1
-#define RCU_DYNTICK_CTRL_CTR (RCU_DYNTICK_CTRL_MASK + 1)
-#ifndef rcu_eqs_special_exit
-#define rcu_eqs_special_exit() do { } while (0)
-#endif
-
-static DEFINE_PER_CPU(struct rcu_dynticks, rcu_dynticks) = {
- .dynticks_nesting = 1,
- .dynticks_nmi_nesting = DYNTICK_IRQ_NONIDLE,
- .dynticks = ATOMIC_INIT(RCU_DYNTICK_CTRL_CTR),
-};
-
/*
* Record entry into an extended quiescent state. This is only to be
* called when not already in an extended quiescent state.
*/
static void rcu_dynticks_eqs_enter(void)
{
- struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
int seq;
/*
* critical sections, and we also must force ordering with the
* next idle sojourn.
*/
- seq = atomic_add_return(RCU_DYNTICK_CTRL_CTR, &rdtp->dynticks);
+ seq = atomic_add_return(RCU_DYNTICK_CTRL_CTR, &rdp->dynticks);
/* Better be in an extended quiescent state! */
WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) &&
(seq & RCU_DYNTICK_CTRL_CTR));
*/
static void rcu_dynticks_eqs_exit(void)
{
- struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
int seq;
/*
* and we also must force ordering with the next RCU read-side
* critical section.
*/
- seq = atomic_add_return(RCU_DYNTICK_CTRL_CTR, &rdtp->dynticks);
+ seq = atomic_add_return(RCU_DYNTICK_CTRL_CTR, &rdp->dynticks);
WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) &&
!(seq & RCU_DYNTICK_CTRL_CTR));
if (seq & RCU_DYNTICK_CTRL_MASK) {
- atomic_andnot(RCU_DYNTICK_CTRL_MASK, &rdtp->dynticks);
+ atomic_andnot(RCU_DYNTICK_CTRL_MASK, &rdp->dynticks);
smp_mb__after_atomic(); /* _exit after clearing mask. */
/* Prefer duplicate flushes to losing a flush. */
rcu_eqs_special_exit();
*/
static void rcu_dynticks_eqs_online(void)
{
- struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
- if (atomic_read(&rdtp->dynticks) & RCU_DYNTICK_CTRL_CTR)
+ if (atomic_read(&rdp->dynticks) & RCU_DYNTICK_CTRL_CTR)
return;
- atomic_add(RCU_DYNTICK_CTRL_CTR, &rdtp->dynticks);
+ atomic_add(RCU_DYNTICK_CTRL_CTR, &rdp->dynticks);
}
/*
*/
bool rcu_dynticks_curr_cpu_in_eqs(void)
{
- struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
- return !(atomic_read(&rdtp->dynticks) & RCU_DYNTICK_CTRL_CTR);
+ return !(atomic_read(&rdp->dynticks) & RCU_DYNTICK_CTRL_CTR);
}
/*
* Snapshot the ->dynticks counter with full ordering so as to allow
* stable comparison of this counter with past and future snapshots.
*/
-int rcu_dynticks_snap(struct rcu_dynticks *rdtp)
+int rcu_dynticks_snap(struct rcu_data *rdp)
{
- int snap = atomic_add_return(0, &rdtp->dynticks);
+ int snap = atomic_add_return(0, &rdp->dynticks);
return snap & ~RCU_DYNTICK_CTRL_MASK;
}
}
/*
- * Return true if the CPU corresponding to the specified rcu_dynticks
+ * Return true if the CPU corresponding to the specified rcu_data
* structure has spent some time in an extended quiescent state since
* rcu_dynticks_snap() returned the specified snapshot.
*/
-static bool rcu_dynticks_in_eqs_since(struct rcu_dynticks *rdtp, int snap)
+static bool rcu_dynticks_in_eqs_since(struct rcu_data *rdp, int snap)
{
- return snap != rcu_dynticks_snap(rdtp);
+ return snap != rcu_dynticks_snap(rdp);
}
/*
{
int old;
int new;
- struct rcu_dynticks *rdtp = &per_cpu(rcu_dynticks, cpu);
+ struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
do {
- old = atomic_read(&rdtp->dynticks);
+ old = atomic_read(&rdp->dynticks);
if (old & RCU_DYNTICK_CTRL_CTR)
return false;
new = old | RCU_DYNTICK_CTRL_MASK;
- } while (atomic_cmpxchg(&rdtp->dynticks, old, new) != old);
+ } while (atomic_cmpxchg(&rdp->dynticks, old, new) != old);
return true;
}
*
* The caller must have disabled interrupts and must not be idle.
*/
-static void rcu_momentary_dyntick_idle(void)
+static void __maybe_unused rcu_momentary_dyntick_idle(void)
{
- struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
int special;
- raw_cpu_write(rcu_dynticks.rcu_need_heavy_qs, false);
- special = atomic_add_return(2 * RCU_DYNTICK_CTRL_CTR, &rdtp->dynticks);
+ raw_cpu_write(rcu_data.rcu_need_heavy_qs, false);
+ special = atomic_add_return(2 * RCU_DYNTICK_CTRL_CTR,
+ &this_cpu_ptr(&rcu_data)->dynticks);
/* It is illegal to call this from idle state. */
WARN_ON_ONCE(!(special & RCU_DYNTICK_CTRL_CTR));
+ rcu_preempt_deferred_qs(current);
}
-/*
- * Note a context switch. This is a quiescent state for RCU-sched,
- * and requires special handling for preemptible RCU.
- * The caller must have disabled interrupts.
- */
-void rcu_note_context_switch(bool preempt)
-{
- barrier(); /* Avoid RCU read-side critical sections leaking down. */
- trace_rcu_utilization(TPS("Start context switch"));
- rcu_sched_qs();
- rcu_preempt_note_context_switch(preempt);
- /* Load rcu_urgent_qs before other flags. */
- if (!smp_load_acquire(this_cpu_ptr(&rcu_dynticks.rcu_urgent_qs)))
- goto out;
- this_cpu_write(rcu_dynticks.rcu_urgent_qs, false);
- if (unlikely(raw_cpu_read(rcu_dynticks.rcu_need_heavy_qs)))
- rcu_momentary_dyntick_idle();
- this_cpu_inc(rcu_dynticks.rcu_qs_ctr);
- if (!preempt)
- rcu_tasks_qs(current);
-out:
- trace_rcu_utilization(TPS("End context switch"));
- barrier(); /* Avoid RCU read-side critical sections leaking up. */
-}
-EXPORT_SYMBOL_GPL(rcu_note_context_switch);
-
-/*
- * Register a quiescent state for all RCU flavors. If there is an
- * emergency, invoke rcu_momentary_dyntick_idle() to do a heavy-weight
- * dyntick-idle quiescent state visible to other CPUs (but only for those
- * RCU flavors in desperate need of a quiescent state, which will normally
- * be none of them). Either way, do a lightweight quiescent state for
- * all RCU flavors.
- *
- * The barrier() calls are redundant in the common case when this is
- * called externally, but just in case this is called from within this
- * file.
+/**
+ * rcu_is_cpu_rrupt_from_idle - see if idle or immediately interrupted from idle
*
+ * If the current CPU is idle or running at a first-level (not nested)
+ * interrupt from idle, return true. The caller must have at least
+ * disabled preemption.
*/
-void rcu_all_qs(void)
+static int rcu_is_cpu_rrupt_from_idle(void)
{
- unsigned long flags;
-
- if (!raw_cpu_read(rcu_dynticks.rcu_urgent_qs))
- return;
- preempt_disable();
- /* Load rcu_urgent_qs before other flags. */
- if (!smp_load_acquire(this_cpu_ptr(&rcu_dynticks.rcu_urgent_qs))) {
- preempt_enable();
- return;
- }
- this_cpu_write(rcu_dynticks.rcu_urgent_qs, false);
- barrier(); /* Avoid RCU read-side critical sections leaking down. */
- if (unlikely(raw_cpu_read(rcu_dynticks.rcu_need_heavy_qs))) {
- local_irq_save(flags);
- rcu_momentary_dyntick_idle();
- local_irq_restore(flags);
- }
- if (unlikely(raw_cpu_read(rcu_sched_data.cpu_no_qs.b.exp)))
- rcu_sched_qs();
- this_cpu_inc(rcu_dynticks.rcu_qs_ctr);
- barrier(); /* Avoid RCU read-side critical sections leaking up. */
- preempt_enable();
+ return __this_cpu_read(rcu_data.dynticks_nesting) <= 0 &&
+ __this_cpu_read(rcu_data.dynticks_nmi_nesting) <= 1;
}
-EXPORT_SYMBOL_GPL(rcu_all_qs);
#define DEFAULT_RCU_BLIMIT 10 /* Maximum callbacks per rcu_do_batch. */
static long blimit = DEFAULT_RCU_BLIMIT;
static ulong jiffies_till_next_fqs = ULONG_MAX;
static bool rcu_kick_kthreads;
+/*
+ * How long the grace period must be before we start recruiting
+ * quiescent-state help from rcu_note_context_switch().
+ */
+static ulong jiffies_till_sched_qs = ULONG_MAX;
+module_param(jiffies_till_sched_qs, ulong, 0444);
+static ulong jiffies_to_sched_qs; /* Adjusted version of above if not default */
+module_param(jiffies_to_sched_qs, ulong, 0444); /* Display only! */
+
+/*
+ * Make sure that we give the grace-period kthread time to detect any
+ * idle CPUs before taking active measures to force quiescent states.
+ * However, don't go below 100 milliseconds, adjusted upwards for really
+ * large systems.
+ */
+static void adjust_jiffies_till_sched_qs(void)
+{
+ unsigned long j;
+
+ /* If jiffies_till_sched_qs was specified, respect the request. */
+ if (jiffies_till_sched_qs != ULONG_MAX) {
+ WRITE_ONCE(jiffies_to_sched_qs, jiffies_till_sched_qs);
+ return;
+ }
+ j = READ_ONCE(jiffies_till_first_fqs) +
+ 2 * READ_ONCE(jiffies_till_next_fqs);
+ if (j < HZ / 10 + nr_cpu_ids / RCU_JIFFIES_FQS_DIV)
+ j = HZ / 10 + nr_cpu_ids / RCU_JIFFIES_FQS_DIV;
+ pr_info("RCU calculated value of scheduler-enlistment delay is %ld jiffies.\n", j);
+ WRITE_ONCE(jiffies_to_sched_qs, j);
+}
+
static int param_set_first_fqs_jiffies(const char *val, const struct kernel_param *kp)
{
ulong j;
int ret = kstrtoul(val, 0, &j);
- if (!ret)
+ if (!ret) {
WRITE_ONCE(*(ulong *)kp->arg, (j > HZ) ? HZ : j);
+ adjust_jiffies_till_sched_qs();
+ }
return ret;
}
ulong j;
int ret = kstrtoul(val, 0, &j);
- if (!ret)
+ if (!ret) {
WRITE_ONCE(*(ulong *)kp->arg, (j > HZ) ? HZ : (j ?: 1));
+ adjust_jiffies_till_sched_qs();
+ }
return ret;
}
module_param_cb(jiffies_till_next_fqs, &next_fqs_jiffies_ops, &jiffies_till_next_fqs, 0644);
module_param(rcu_kick_kthreads, bool, 0644);
-/*
- * How long the grace period must be before we start recruiting
- * quiescent-state help from rcu_note_context_switch().
- */
-static ulong jiffies_till_sched_qs = HZ / 10;
-module_param(jiffies_till_sched_qs, ulong, 0444);
-
-static void force_qs_rnp(struct rcu_state *rsp, int (*f)(struct rcu_data *rsp));
-static void force_quiescent_state(struct rcu_state *rsp);
+static void force_qs_rnp(int (*f)(struct rcu_data *rdp));
+static void force_quiescent_state(void);
static int rcu_pending(void);
/*
*/
unsigned long rcu_get_gp_seq(void)
{
- return READ_ONCE(rcu_state_p->gp_seq);
+ return READ_ONCE(rcu_state.gp_seq);
}
EXPORT_SYMBOL_GPL(rcu_get_gp_seq);
-/*
- * Return the number of RCU-sched GPs completed thus far for debug & stats.
- */
-unsigned long rcu_sched_get_gp_seq(void)
-{
- return READ_ONCE(rcu_sched_state.gp_seq);
-}
-EXPORT_SYMBOL_GPL(rcu_sched_get_gp_seq);
-
-/*
- * Return the number of RCU-bh GPs completed thus far for debug & stats.
- */
-unsigned long rcu_bh_get_gp_seq(void)
-{
- return READ_ONCE(rcu_bh_state.gp_seq);
-}
-EXPORT_SYMBOL_GPL(rcu_bh_get_gp_seq);
-
/*
* Return the number of RCU expedited batches completed thus far for
* debug & stats. Odd numbers mean that a batch is in progress, even
*/
unsigned long rcu_exp_batches_completed(void)
{
- return rcu_state_p->expedited_sequence;
+ return rcu_state.expedited_sequence;
}
EXPORT_SYMBOL_GPL(rcu_exp_batches_completed);
-/*
- * Return the number of RCU-sched expedited batches completed thus far
- * for debug & stats. Similar to rcu_exp_batches_completed().
- */
-unsigned long rcu_exp_batches_completed_sched(void)
-{
- return rcu_sched_state.expedited_sequence;
-}
-EXPORT_SYMBOL_GPL(rcu_exp_batches_completed_sched);
-
/*
* Force a quiescent state.
*/
void rcu_force_quiescent_state(void)
{
- force_quiescent_state(rcu_state_p);
+ force_quiescent_state();
}
EXPORT_SYMBOL_GPL(rcu_force_quiescent_state);
-/*
- * Force a quiescent state for RCU BH.
- */
-void rcu_bh_force_quiescent_state(void)
-{
- force_quiescent_state(&rcu_bh_state);
-}
-EXPORT_SYMBOL_GPL(rcu_bh_force_quiescent_state);
-
-/*
- * Force a quiescent state for RCU-sched.
- */
-void rcu_sched_force_quiescent_state(void)
-{
- force_quiescent_state(&rcu_sched_state);
-}
-EXPORT_SYMBOL_GPL(rcu_sched_force_quiescent_state);
-
/*
* Show the state of the grace-period kthreads.
*/
int cpu;
struct rcu_data *rdp;
struct rcu_node *rnp;
- struct rcu_state *rsp;
- for_each_rcu_flavor(rsp) {
- pr_info("%s: wait state: %d ->state: %#lx\n",
- rsp->name, rsp->gp_state, rsp->gp_kthread->state);
- rcu_for_each_node_breadth_first(rsp, rnp) {
- if (ULONG_CMP_GE(rsp->gp_seq, rnp->gp_seq_needed))
- continue;
- pr_info("\trcu_node %d:%d ->gp_seq %lu ->gp_seq_needed %lu\n",
- rnp->grplo, rnp->grphi, rnp->gp_seq,
- rnp->gp_seq_needed);
- if (!rcu_is_leaf_node(rnp))
+ pr_info("%s: wait state: %d ->state: %#lx\n", rcu_state.name,
+ rcu_state.gp_state, rcu_state.gp_kthread->state);
+ rcu_for_each_node_breadth_first(rnp) {
+ if (ULONG_CMP_GE(rcu_state.gp_seq, rnp->gp_seq_needed))
+ continue;
+ pr_info("\trcu_node %d:%d ->gp_seq %lu ->gp_seq_needed %lu\n",
+ rnp->grplo, rnp->grphi, rnp->gp_seq,
+ rnp->gp_seq_needed);
+ if (!rcu_is_leaf_node(rnp))
+ continue;
+ for_each_leaf_node_possible_cpu(rnp, cpu) {
+ rdp = per_cpu_ptr(&rcu_data, cpu);
+ if (rdp->gpwrap ||
+ ULONG_CMP_GE(rcu_state.gp_seq,
+ rdp->gp_seq_needed))
continue;
- for_each_leaf_node_possible_cpu(rnp, cpu) {
- rdp = per_cpu_ptr(rsp->rda, cpu);
- if (rdp->gpwrap ||
- ULONG_CMP_GE(rsp->gp_seq,
- rdp->gp_seq_needed))
- continue;
- pr_info("\tcpu %d ->gp_seq_needed %lu\n",
- cpu, rdp->gp_seq_needed);
- }
+ pr_info("\tcpu %d ->gp_seq_needed %lu\n",
+ cpu, rdp->gp_seq_needed);
}
- /* sched_show_task(rsp->gp_kthread); */
}
+ /* sched_show_task(rcu_state.gp_kthread); */
}
EXPORT_SYMBOL_GPL(show_rcu_gp_kthreads);
void rcutorture_get_gp_data(enum rcutorture_type test_type, int *flags,
unsigned long *gp_seq)
{
- struct rcu_state *rsp = NULL;
-
switch (test_type) {
case RCU_FLAVOR:
- rsp = rcu_state_p;
- break;
case RCU_BH_FLAVOR:
- rsp = &rcu_bh_state;
- break;
case RCU_SCHED_FLAVOR:
- rsp = &rcu_sched_state;
+ *flags = READ_ONCE(rcu_state.gp_flags);
+ *gp_seq = rcu_seq_current(&rcu_state.gp_seq);
break;
default:
break;
}
- if (rsp == NULL)
- return;
- *flags = READ_ONCE(rsp->gp_flags);
- *gp_seq = rcu_seq_current(&rsp->gp_seq);
}
EXPORT_SYMBOL_GPL(rcutorture_get_gp_data);
/*
- * Return the root node of the specified rcu_state structure.
+ * Return the root node of the rcu_state structure.
*/
-static struct rcu_node *rcu_get_root(struct rcu_state *rsp)
+static struct rcu_node *rcu_get_root(void)
{
- return &rsp->node[0];
+ return &rcu_state.node[0];
}
/*
*/
static void rcu_eqs_enter(bool user)
{
- struct rcu_state *rsp;
- struct rcu_data *rdp;
- struct rcu_dynticks *rdtp;
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
- rdtp = this_cpu_ptr(&rcu_dynticks);
- WRITE_ONCE(rdtp->dynticks_nmi_nesting, 0);
+ WARN_ON_ONCE(rdp->dynticks_nmi_nesting != DYNTICK_IRQ_NONIDLE);
+ WRITE_ONCE(rdp->dynticks_nmi_nesting, 0);
WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) &&
- rdtp->dynticks_nesting == 0);
- if (rdtp->dynticks_nesting != 1) {
- rdtp->dynticks_nesting--;
+ rdp->dynticks_nesting == 0);
+ if (rdp->dynticks_nesting != 1) {
+ rdp->dynticks_nesting--;
return;
}
lockdep_assert_irqs_disabled();
- trace_rcu_dyntick(TPS("Start"), rdtp->dynticks_nesting, 0, rdtp->dynticks);
+ trace_rcu_dyntick(TPS("Start"), rdp->dynticks_nesting, 0, rdp->dynticks);
WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
- for_each_rcu_flavor(rsp) {
- rdp = this_cpu_ptr(rsp->rda);
- do_nocb_deferred_wakeup(rdp);
- }
+ rdp = this_cpu_ptr(&rcu_data);
+ do_nocb_deferred_wakeup(rdp);
rcu_prepare_for_idle();
- WRITE_ONCE(rdtp->dynticks_nesting, 0); /* Avoid irq-access tearing. */
+ rcu_preempt_deferred_qs(current);
+ WRITE_ONCE(rdp->dynticks_nesting, 0); /* Avoid irq-access tearing. */
rcu_dynticks_eqs_enter();
rcu_dynticks_task_enter();
}
}
#endif /* CONFIG_NO_HZ_FULL */
-/**
- * rcu_nmi_exit - inform RCU of exit from NMI context
- *
+/*
* If we are returning from the outermost NMI handler that interrupted an
- * RCU-idle period, update rdtp->dynticks and rdtp->dynticks_nmi_nesting
+ * RCU-idle period, update rdp->dynticks and rdp->dynticks_nmi_nesting
* to let the RCU grace-period handling know that the CPU is back to
* being RCU-idle.
*
- * If you add or remove a call to rcu_nmi_exit(), be sure to test
+ * If you add or remove a call to rcu_nmi_exit_common(), be sure to test
* with CONFIG_RCU_EQS_DEBUG=y.
*/
-void rcu_nmi_exit(void)
+static __always_inline void rcu_nmi_exit_common(bool irq)
{
- struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
/*
* Check for ->dynticks_nmi_nesting underflow and bad ->dynticks.
* (We are exiting an NMI handler, so RCU better be paying attention
* to us!)
*/
- WARN_ON_ONCE(rdtp->dynticks_nmi_nesting <= 0);
+ WARN_ON_ONCE(rdp->dynticks_nmi_nesting <= 0);
WARN_ON_ONCE(rcu_dynticks_curr_cpu_in_eqs());
/*
* If the nesting level is not 1, the CPU wasn't RCU-idle, so
* leave it in non-RCU-idle state.
*/
- if (rdtp->dynticks_nmi_nesting != 1) {
- trace_rcu_dyntick(TPS("--="), rdtp->dynticks_nmi_nesting, rdtp->dynticks_nmi_nesting - 2, rdtp->dynticks);
- WRITE_ONCE(rdtp->dynticks_nmi_nesting, /* No store tearing. */
- rdtp->dynticks_nmi_nesting - 2);
+ if (rdp->dynticks_nmi_nesting != 1) {
+ trace_rcu_dyntick(TPS("--="), rdp->dynticks_nmi_nesting, rdp->dynticks_nmi_nesting - 2, rdp->dynticks);
+ WRITE_ONCE(rdp->dynticks_nmi_nesting, /* No store tearing. */
+ rdp->dynticks_nmi_nesting - 2);
return;
}
/* This NMI interrupted an RCU-idle CPU, restore RCU-idleness. */
- trace_rcu_dyntick(TPS("Startirq"), rdtp->dynticks_nmi_nesting, 0, rdtp->dynticks);
- WRITE_ONCE(rdtp->dynticks_nmi_nesting, 0); /* Avoid store tearing. */
+ trace_rcu_dyntick(TPS("Startirq"), rdp->dynticks_nmi_nesting, 0, rdp->dynticks);
+ WRITE_ONCE(rdp->dynticks_nmi_nesting, 0); /* Avoid store tearing. */
+
+ if (irq)
+ rcu_prepare_for_idle();
+
rcu_dynticks_eqs_enter();
+
+ if (irq)
+ rcu_dynticks_task_enter();
+}
+
+/**
+ * rcu_nmi_exit - inform RCU of exit from NMI context
+ * @irq: Is this call from rcu_irq_exit?
+ *
+ * If you add or remove a call to rcu_nmi_exit(), be sure to test
+ * with CONFIG_RCU_EQS_DEBUG=y.
+ */
+void rcu_nmi_exit(void)
+{
+ rcu_nmi_exit_common(false);
}
/**
*/
void rcu_irq_exit(void)
{
- struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
-
lockdep_assert_irqs_disabled();
- if (rdtp->dynticks_nmi_nesting == 1)
- rcu_prepare_for_idle();
- rcu_nmi_exit();
- if (rdtp->dynticks_nmi_nesting == 0)
- rcu_dynticks_task_enter();
+ rcu_nmi_exit_common(true);
}
/*
*/
static void rcu_eqs_exit(bool user)
{
- struct rcu_dynticks *rdtp;
+ struct rcu_data *rdp;
long oldval;
lockdep_assert_irqs_disabled();
- rdtp = this_cpu_ptr(&rcu_dynticks);
- oldval = rdtp->dynticks_nesting;
+ rdp = this_cpu_ptr(&rcu_data);
+ oldval = rdp->dynticks_nesting;
WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && oldval < 0);
if (oldval) {
- rdtp->dynticks_nesting++;
+ rdp->dynticks_nesting++;
return;
}
rcu_dynticks_task_exit();
rcu_dynticks_eqs_exit();
rcu_cleanup_after_idle();
- trace_rcu_dyntick(TPS("End"), rdtp->dynticks_nesting, 1, rdtp->dynticks);
+ trace_rcu_dyntick(TPS("End"), rdp->dynticks_nesting, 1, rdp->dynticks);
WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
- WRITE_ONCE(rdtp->dynticks_nesting, 1);
- WRITE_ONCE(rdtp->dynticks_nmi_nesting, DYNTICK_IRQ_NONIDLE);
+ WRITE_ONCE(rdp->dynticks_nesting, 1);
+ WARN_ON_ONCE(rdp->dynticks_nmi_nesting);
+ WRITE_ONCE(rdp->dynticks_nmi_nesting, DYNTICK_IRQ_NONIDLE);
}
/**
#endif /* CONFIG_NO_HZ_FULL */
/**
- * rcu_nmi_enter - inform RCU of entry to NMI context
+ * rcu_nmi_enter_common - inform RCU of entry to NMI context
+ * @irq: Is this call from rcu_irq_enter?
*
- * If the CPU was idle from RCU's viewpoint, update rdtp->dynticks and
- * rdtp->dynticks_nmi_nesting to let the RCU grace-period handling know
+ * If the CPU was idle from RCU's viewpoint, update rdp->dynticks and
+ * rdp->dynticks_nmi_nesting to let the RCU grace-period handling know
* that the CPU is active. This implementation permits nested NMIs, as
* long as the nesting level does not overflow an int. (You will probably
* run out of stack space first.)
*
- * If you add or remove a call to rcu_nmi_enter(), be sure to test
+ * If you add or remove a call to rcu_nmi_enter_common(), be sure to test
* with CONFIG_RCU_EQS_DEBUG=y.
*/
-void rcu_nmi_enter(void)
+static __always_inline void rcu_nmi_enter_common(bool irq)
{
- struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
long incby = 2;
/* Complain about underflow. */
- WARN_ON_ONCE(rdtp->dynticks_nmi_nesting < 0);
+ WARN_ON_ONCE(rdp->dynticks_nmi_nesting < 0);
/*
* If idle from RCU viewpoint, atomically increment ->dynticks
* period (observation due to Andy Lutomirski).
*/
if (rcu_dynticks_curr_cpu_in_eqs()) {
+
+ if (irq)
+ rcu_dynticks_task_exit();
+
rcu_dynticks_eqs_exit();
+
+ if (irq)
+ rcu_cleanup_after_idle();
+
incby = 1;
}
trace_rcu_dyntick(incby == 1 ? TPS("Endirq") : TPS("++="),
- rdtp->dynticks_nmi_nesting,
- rdtp->dynticks_nmi_nesting + incby, rdtp->dynticks);
- WRITE_ONCE(rdtp->dynticks_nmi_nesting, /* Prevent store tearing. */
- rdtp->dynticks_nmi_nesting + incby);
+ rdp->dynticks_nmi_nesting,
+ rdp->dynticks_nmi_nesting + incby, rdp->dynticks);
+ WRITE_ONCE(rdp->dynticks_nmi_nesting, /* Prevent store tearing. */
+ rdp->dynticks_nmi_nesting + incby);
barrier();
}
+/**
+ * rcu_nmi_enter - inform RCU of entry to NMI context
+ */
+void rcu_nmi_enter(void)
+{
+ rcu_nmi_enter_common(false);
+}
+
/**
* rcu_irq_enter - inform RCU that current CPU is entering irq away from idle
*
*/
void rcu_irq_enter(void)
{
- struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
-
lockdep_assert_irqs_disabled();
- if (rdtp->dynticks_nmi_nesting == 0)
- rcu_dynticks_task_exit();
- rcu_nmi_enter();
- if (rdtp->dynticks_nmi_nesting == 1)
- rcu_cleanup_after_idle();
+ rcu_nmi_enter_common(true);
}
/*
cpu = task_cpu(t);
if (!task_curr(t))
return; /* This task is not running on that CPU. */
- smp_store_release(per_cpu_ptr(&rcu_dynticks.rcu_urgent_qs, cpu), true);
+ smp_store_release(per_cpu_ptr(&rcu_data.rcu_urgent_qs, cpu), true);
}
#if defined(CONFIG_PROVE_RCU) && defined(CONFIG_HOTPLUG_CPU)
* Disable preemption to avoid false positives that could otherwise
* happen due to the current CPU number being sampled, this task being
* preempted, its old CPU being taken offline, resuming on some other CPU,
- * then determining that its old CPU is now offline. Because there are
- * multiple flavors of RCU, and because this function can be called in the
- * midst of updating the flavors while a given CPU coming online or going
- * offline, it is necessary to check all flavors. If any of the flavors
- * believe that given CPU is online, it is considered to be online.
+ * then determining that its old CPU is now offline.
*
* Disable checking if in an NMI handler because we cannot safely
* report errors from NMI handlers anyway. In addition, it is OK to use
{
struct rcu_data *rdp;
struct rcu_node *rnp;
- struct rcu_state *rsp;
+ bool ret = false;
if (in_nmi() || !rcu_scheduler_fully_active)
return true;
preempt_disable();
- for_each_rcu_flavor(rsp) {
- rdp = this_cpu_ptr(rsp->rda);
- rnp = rdp->mynode;
- if (rdp->grpmask & rcu_rnp_online_cpus(rnp)) {
- preempt_enable();
- return true;
- }
- }
+ rdp = this_cpu_ptr(&rcu_data);
+ rnp = rdp->mynode;
+ if (rdp->grpmask & rcu_rnp_online_cpus(rnp))
+ ret = true;
preempt_enable();
- return false;
+ return ret;
}
EXPORT_SYMBOL_GPL(rcu_lockdep_current_cpu_online);
#endif /* #if defined(CONFIG_PROVE_RCU) && defined(CONFIG_HOTPLUG_CPU) */
-/**
- * rcu_is_cpu_rrupt_from_idle - see if idle or immediately interrupted from idle
- *
- * If the current CPU is idle or running at a first-level (not nested)
- * interrupt from idle, return true. The caller must have at least
- * disabled preemption.
- */
-static int rcu_is_cpu_rrupt_from_idle(void)
-{
- return __this_cpu_read(rcu_dynticks.dynticks_nesting) <= 0 &&
- __this_cpu_read(rcu_dynticks.dynticks_nmi_nesting) <= 1;
-}
-
/*
* We are reporting a quiescent state on behalf of some other CPU, so
* it is our responsibility to check for and handle potential overflow
*/
static int dyntick_save_progress_counter(struct rcu_data *rdp)
{
- rdp->dynticks_snap = rcu_dynticks_snap(rdp->dynticks);
+ rdp->dynticks_snap = rcu_dynticks_snap(rdp);
if (rcu_dynticks_in_eqs(rdp->dynticks_snap)) {
- trace_rcu_fqs(rdp->rsp->name, rdp->gp_seq, rdp->cpu, TPS("dti"));
+ trace_rcu_fqs(rcu_state.name, rdp->gp_seq, rdp->cpu, TPS("dti"));
rcu_gpnum_ovf(rdp->mynode, rdp);
return 1;
}
* read-side critical section that started before the beginning
* of the current RCU grace period.
*/
- if (rcu_dynticks_in_eqs_since(rdp->dynticks, rdp->dynticks_snap)) {
- trace_rcu_fqs(rdp->rsp->name, rdp->gp_seq, rdp->cpu, TPS("dti"));
- rdp->dynticks_fqs++;
- rcu_gpnum_ovf(rnp, rdp);
- return 1;
- }
-
- /*
- * Has this CPU encountered a cond_resched() since the beginning
- * of the grace period? For this to be the case, the CPU has to
- * have noticed the current grace period. This might not be the
- * case for nohz_full CPUs looping in the kernel.
- */
- jtsq = jiffies_till_sched_qs;
- ruqp = per_cpu_ptr(&rcu_dynticks.rcu_urgent_qs, rdp->cpu);
- if (time_after(jiffies, rdp->rsp->gp_start + jtsq) &&
- READ_ONCE(rdp->rcu_qs_ctr_snap) != per_cpu(rcu_dynticks.rcu_qs_ctr, rdp->cpu) &&
- rcu_seq_current(&rdp->gp_seq) == rnp->gp_seq && !rdp->gpwrap) {
- trace_rcu_fqs(rdp->rsp->name, rdp->gp_seq, rdp->cpu, TPS("rqc"));
+ if (rcu_dynticks_in_eqs_since(rdp, rdp->dynticks_snap)) {
+ trace_rcu_fqs(rcu_state.name, rdp->gp_seq, rdp->cpu, TPS("dti"));
rcu_gpnum_ovf(rnp, rdp);
return 1;
- } else if (time_after(jiffies, rdp->rsp->gp_start + jtsq)) {
- /* Load rcu_qs_ctr before store to rcu_urgent_qs. */
- smp_store_release(ruqp, true);
}
/* If waiting too long on an offline CPU, complain. */
if (!(rdp->grpmask & rcu_rnp_online_cpus(rnp)) &&
- time_after(jiffies, rdp->rsp->gp_start + HZ)) {
+ time_after(jiffies, rcu_state.gp_start + HZ)) {
bool onl;
struct rcu_node *rnp1;
/*
* A CPU running for an extended time within the kernel can
- * delay RCU grace periods. When the CPU is in NO_HZ_FULL mode,
- * even context-switching back and forth between a pair of
- * in-kernel CPU-bound tasks cannot advance grace periods.
- * So if the grace period is old enough, make the CPU pay attention.
- * Note that the unsynchronized assignments to the per-CPU
- * rcu_need_heavy_qs variable are safe. Yes, setting of
- * bits can be lost, but they will be set again on the next
- * force-quiescent-state pass. So lost bit sets do not result
- * in incorrect behavior, merely in a grace period lasting
- * a few jiffies longer than it might otherwise. Because
- * there are at most four threads involved, and because the
- * updates are only once every few jiffies, the probability of
- * lossage (and thus of slight grace-period extension) is
- * quite low.
+ * delay RCU grace periods: (1) At age jiffies_to_sched_qs,
+ * set .rcu_urgent_qs, (2) At age 2*jiffies_to_sched_qs, set
+ * both .rcu_need_heavy_qs and .rcu_urgent_qs. Note that the
+ * unsynchronized assignments to the per-CPU rcu_need_heavy_qs
+ * variable are safe because the assignments are repeated if this
+ * CPU failed to pass through a quiescent state. This code
+ * also checks .jiffies_resched in case jiffies_to_sched_qs
+ * is set way high.
*/
- rnhqp = &per_cpu(rcu_dynticks.rcu_need_heavy_qs, rdp->cpu);
+ jtsq = READ_ONCE(jiffies_to_sched_qs);
+ ruqp = per_cpu_ptr(&rcu_data.rcu_urgent_qs, rdp->cpu);
+ rnhqp = &per_cpu(rcu_data.rcu_need_heavy_qs, rdp->cpu);
if (!READ_ONCE(*rnhqp) &&
- (time_after(jiffies, rdp->rsp->gp_start + jtsq) ||
- time_after(jiffies, rdp->rsp->jiffies_resched))) {
+ (time_after(jiffies, rcu_state.gp_start + jtsq * 2) ||
+ time_after(jiffies, rcu_state.jiffies_resched))) {
WRITE_ONCE(*rnhqp, true);
/* Store rcu_need_heavy_qs before rcu_urgent_qs. */
smp_store_release(ruqp, true);
- rdp->rsp->jiffies_resched += jtsq; /* Re-enable beating. */
+ } else if (time_after(jiffies, rcu_state.gp_start + jtsq)) {
+ WRITE_ONCE(*ruqp, true);
}
/*
- * If more than halfway to RCU CPU stall-warning time, do a
- * resched_cpu() to try to loosen things up a bit. Also check to
- * see if the CPU is getting hammered with interrupts, but only
- * once per grace period, just to keep the IPIs down to a dull roar.
+ * NO_HZ_FULL CPUs can run in-kernel without rcu_check_callbacks!
+ * The above code handles this, but only for straight cond_resched().
+ * And some in-kernel loops check need_resched() before calling
+ * cond_resched(), which defeats the above code for CPUs that are
+ * running in-kernel with scheduling-clock interrupts disabled.
+ * So hit them over the head with the resched_cpu() hammer!
*/
- if (jiffies - rdp->rsp->gp_start > rcu_jiffies_till_stall_check() / 2) {
+ if (tick_nohz_full_cpu(rdp->cpu) &&
+ time_after(jiffies,
+ READ_ONCE(rdp->last_fqs_resched) + jtsq * 3)) {
resched_cpu(rdp->cpu);
+ WRITE_ONCE(rdp->last_fqs_resched, jiffies);
+ }
+
+ /*
+ * If more than halfway to RCU CPU stall-warning time, invoke
+ * resched_cpu() more frequently to try to loosen things up a bit.
+ * Also check to see if the CPU is getting hammered with interrupts,
+ * but only once per grace period, just to keep the IPIs down to
+ * a dull roar.
+ */
+ if (time_after(jiffies, rcu_state.jiffies_resched)) {
+ if (time_after(jiffies,
+ READ_ONCE(rdp->last_fqs_resched) + jtsq)) {
+ resched_cpu(rdp->cpu);
+ WRITE_ONCE(rdp->last_fqs_resched, jiffies);
+ }
if (IS_ENABLED(CONFIG_IRQ_WORK) &&
!rdp->rcu_iw_pending && rdp->rcu_iw_gp_seq != rnp->gp_seq &&
(rnp->ffmask & rdp->grpmask)) {
return 0;
}
-static void record_gp_stall_check_time(struct rcu_state *rsp)
+static void record_gp_stall_check_time(void)
{
unsigned long j = jiffies;
unsigned long j1;
- rsp->gp_start = j;
+ rcu_state.gp_start = j;
j1 = rcu_jiffies_till_stall_check();
/* Record ->gp_start before ->jiffies_stall. */
- smp_store_release(&rsp->jiffies_stall, j + j1); /* ^^^ */
- rsp->jiffies_resched = j + j1 / 2;
- rsp->n_force_qs_gpstart = READ_ONCE(rsp->n_force_qs);
+ smp_store_release(&rcu_state.jiffies_stall, j + j1); /* ^^^ */
+ rcu_state.jiffies_resched = j + j1 / 2;
+ rcu_state.n_force_qs_gpstart = READ_ONCE(rcu_state.n_force_qs);
}
/*
/*
* Complain about starvation of grace-period kthread.
*/
-static void rcu_check_gp_kthread_starvation(struct rcu_state *rsp)
+static void rcu_check_gp_kthread_starvation(void)
{
- unsigned long gpa;
+ struct task_struct *gpk = rcu_state.gp_kthread;
unsigned long j;
- j = jiffies;
- gpa = READ_ONCE(rsp->gp_activity);
- if (j - gpa > 2 * HZ) {
+ j = jiffies - READ_ONCE(rcu_state.gp_activity);
+ if (j > 2 * HZ) {
pr_err("%s kthread starved for %ld jiffies! g%ld f%#x %s(%d) ->state=%#lx ->cpu=%d\n",
- rsp->name, j - gpa,
- (long)rcu_seq_current(&rsp->gp_seq),
- rsp->gp_flags,
- gp_state_getname(rsp->gp_state), rsp->gp_state,
- rsp->gp_kthread ? rsp->gp_kthread->state : ~0,
- rsp->gp_kthread ? task_cpu(rsp->gp_kthread) : -1);
- if (rsp->gp_kthread) {
+ rcu_state.name, j,
+ (long)rcu_seq_current(&rcu_state.gp_seq),
+ rcu_state.gp_flags,
+ gp_state_getname(rcu_state.gp_state), rcu_state.gp_state,
+ gpk ? gpk->state : ~0, gpk ? task_cpu(gpk) : -1);
+ if (gpk) {
pr_err("RCU grace-period kthread stack dump:\n");
- sched_show_task(rsp->gp_kthread);
- wake_up_process(rsp->gp_kthread);
+ sched_show_task(gpk);
+ wake_up_process(gpk);
}
}
}
* that don't support NMI-based stack dumps. The NMI-triggered stack
* traces are more accurate because they are printed by the target CPU.
*/
-static void rcu_dump_cpu_stacks(struct rcu_state *rsp)
+static void rcu_dump_cpu_stacks(void)
{
int cpu;
unsigned long flags;
struct rcu_node *rnp;
- rcu_for_each_leaf_node(rsp, rnp) {
+ rcu_for_each_leaf_node(rnp) {
raw_spin_lock_irqsave_rcu_node(rnp, flags);
for_each_leaf_node_possible_cpu(rnp, cpu)
if (rnp->qsmask & leaf_node_cpu_bit(rnp, cpu))
* If too much time has passed in the current grace period, and if
* so configured, go kick the relevant kthreads.
*/
-static void rcu_stall_kick_kthreads(struct rcu_state *rsp)
+static void rcu_stall_kick_kthreads(void)
{
unsigned long j;
if (!rcu_kick_kthreads)
return;
- j = READ_ONCE(rsp->jiffies_kick_kthreads);
- if (time_after(jiffies, j) && rsp->gp_kthread &&
- (rcu_gp_in_progress(rsp) || READ_ONCE(rsp->gp_flags))) {
- WARN_ONCE(1, "Kicking %s grace-period kthread\n", rsp->name);
+ j = READ_ONCE(rcu_state.jiffies_kick_kthreads);
+ if (time_after(jiffies, j) && rcu_state.gp_kthread &&
+ (rcu_gp_in_progress() || READ_ONCE(rcu_state.gp_flags))) {
+ WARN_ONCE(1, "Kicking %s grace-period kthread\n",
+ rcu_state.name);
rcu_ftrace_dump(DUMP_ALL);
- wake_up_process(rsp->gp_kthread);
- WRITE_ONCE(rsp->jiffies_kick_kthreads, j + HZ);
+ wake_up_process(rcu_state.gp_kthread);
+ WRITE_ONCE(rcu_state.jiffies_kick_kthreads, j + HZ);
}
}
panic("RCU Stall\n");
}
-static void print_other_cpu_stall(struct rcu_state *rsp, unsigned long gp_seq)
+static void print_other_cpu_stall(unsigned long gp_seq)
{
int cpu;
unsigned long flags;
unsigned long gpa;
unsigned long j;
int ndetected = 0;
- struct rcu_node *rnp = rcu_get_root(rsp);
+ struct rcu_node *rnp = rcu_get_root();
long totqlen = 0;
/* Kick and suppress, if so configured. */
- rcu_stall_kick_kthreads(rsp);
+ rcu_stall_kick_kthreads();
if (rcu_cpu_stall_suppress)
return;
* See Documentation/RCU/stallwarn.txt for info on how to debug
* RCU CPU stall warnings.
*/
- pr_err("INFO: %s detected stalls on CPUs/tasks:", rsp->name);
+ pr_err("INFO: %s detected stalls on CPUs/tasks:", rcu_state.name);
print_cpu_stall_info_begin();
- rcu_for_each_leaf_node(rsp, rnp) {
+ rcu_for_each_leaf_node(rnp) {
raw_spin_lock_irqsave_rcu_node(rnp, flags);
ndetected += rcu_print_task_stall(rnp);
if (rnp->qsmask != 0) {
for_each_leaf_node_possible_cpu(rnp, cpu)
if (rnp->qsmask & leaf_node_cpu_bit(rnp, cpu)) {
- print_cpu_stall_info(rsp, cpu);
+ print_cpu_stall_info(cpu);
ndetected++;
}
}
print_cpu_stall_info_end();
for_each_possible_cpu(cpu)
- totqlen += rcu_segcblist_n_cbs(&per_cpu_ptr(rsp->rda,
+ totqlen += rcu_segcblist_n_cbs(&per_cpu_ptr(&rcu_data,
cpu)->cblist);
pr_cont("(detected by %d, t=%ld jiffies, g=%ld, q=%lu)\n",
- smp_processor_id(), (long)(jiffies - rsp->gp_start),
- (long)rcu_seq_current(&rsp->gp_seq), totqlen);
+ smp_processor_id(), (long)(jiffies - rcu_state.gp_start),
+ (long)rcu_seq_current(&rcu_state.gp_seq), totqlen);
if (ndetected) {
- rcu_dump_cpu_stacks(rsp);
+ rcu_dump_cpu_stacks();
/* Complain about tasks blocking the grace period. */
- rcu_print_detail_task_stall(rsp);
+ rcu_print_detail_task_stall();
} else {
- if (rcu_seq_current(&rsp->gp_seq) != gp_seq) {
+ if (rcu_seq_current(&rcu_state.gp_seq) != gp_seq) {
pr_err("INFO: Stall ended before state dump start\n");
} else {
j = jiffies;
- gpa = READ_ONCE(rsp->gp_activity);
+ gpa = READ_ONCE(rcu_state.gp_activity);
pr_err("All QSes seen, last %s kthread activity %ld (%ld-%ld), jiffies_till_next_fqs=%ld, root ->qsmask %#lx\n",
- rsp->name, j - gpa, j, gpa,
- jiffies_till_next_fqs,
- rcu_get_root(rsp)->qsmask);
+ rcu_state.name, j - gpa, j, gpa,
+ READ_ONCE(jiffies_till_next_fqs),
+ rcu_get_root()->qsmask);
/* In this case, the current CPU might be at fault. */
sched_show_task(current);
}
}
/* Rewrite if needed in case of slow consoles. */
- if (ULONG_CMP_GE(jiffies, READ_ONCE(rsp->jiffies_stall)))
- WRITE_ONCE(rsp->jiffies_stall,
+ if (ULONG_CMP_GE(jiffies, READ_ONCE(rcu_state.jiffies_stall)))
+ WRITE_ONCE(rcu_state.jiffies_stall,
jiffies + 3 * rcu_jiffies_till_stall_check() + 3);
- rcu_check_gp_kthread_starvation(rsp);
+ rcu_check_gp_kthread_starvation();
panic_on_rcu_stall();
- force_quiescent_state(rsp); /* Kick them all. */
+ force_quiescent_state(); /* Kick them all. */
}
-static void print_cpu_stall(struct rcu_state *rsp)
+static void print_cpu_stall(void)
{
int cpu;
unsigned long flags;
- struct rcu_data *rdp = this_cpu_ptr(rsp->rda);
- struct rcu_node *rnp = rcu_get_root(rsp);
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
+ struct rcu_node *rnp = rcu_get_root();
long totqlen = 0;
/* Kick and suppress, if so configured. */
- rcu_stall_kick_kthreads(rsp);
+ rcu_stall_kick_kthreads();
if (rcu_cpu_stall_suppress)
return;
* See Documentation/RCU/stallwarn.txt for info on how to debug
* RCU CPU stall warnings.
*/
- pr_err("INFO: %s self-detected stall on CPU", rsp->name);
+ pr_err("INFO: %s self-detected stall on CPU", rcu_state.name);
print_cpu_stall_info_begin();
raw_spin_lock_irqsave_rcu_node(rdp->mynode, flags);
- print_cpu_stall_info(rsp, smp_processor_id());
+ print_cpu_stall_info(smp_processor_id());
raw_spin_unlock_irqrestore_rcu_node(rdp->mynode, flags);
print_cpu_stall_info_end();
for_each_possible_cpu(cpu)
- totqlen += rcu_segcblist_n_cbs(&per_cpu_ptr(rsp->rda,
+ totqlen += rcu_segcblist_n_cbs(&per_cpu_ptr(&rcu_data,
cpu)->cblist);
pr_cont(" (t=%lu jiffies g=%ld q=%lu)\n",
- jiffies - rsp->gp_start,
- (long)rcu_seq_current(&rsp->gp_seq), totqlen);
+ jiffies - rcu_state.gp_start,
+ (long)rcu_seq_current(&rcu_state.gp_seq), totqlen);
- rcu_check_gp_kthread_starvation(rsp);
+ rcu_check_gp_kthread_starvation();
- rcu_dump_cpu_stacks(rsp);
+ rcu_dump_cpu_stacks();
raw_spin_lock_irqsave_rcu_node(rnp, flags);
/* Rewrite if needed in case of slow consoles. */
- if (ULONG_CMP_GE(jiffies, READ_ONCE(rsp->jiffies_stall)))
- WRITE_ONCE(rsp->jiffies_stall,
+ if (ULONG_CMP_GE(jiffies, READ_ONCE(rcu_state.jiffies_stall)))
+ WRITE_ONCE(rcu_state.jiffies_stall,
jiffies + 3 * rcu_jiffies_till_stall_check() + 3);
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
* progress and it could be we're stuck in kernel space without context
* switches for an entirely unreasonable amount of time.
*/
- resched_cpu(smp_processor_id());
+ set_tsk_need_resched(current);
+ set_preempt_need_resched();
}
-static void check_cpu_stall(struct rcu_state *rsp, struct rcu_data *rdp)
+static void check_cpu_stall(struct rcu_data *rdp)
{
unsigned long gs1;
unsigned long gs2;
struct rcu_node *rnp;
if ((rcu_cpu_stall_suppress && !rcu_kick_kthreads) ||
- !rcu_gp_in_progress(rsp))
+ !rcu_gp_in_progress())
return;
- rcu_stall_kick_kthreads(rsp);
+ rcu_stall_kick_kthreads();
j = jiffies;
/*
* Lots of memory barriers to reject false positives.
*
- * The idea is to pick up rsp->gp_seq, then rsp->jiffies_stall,
- * then rsp->gp_start, and finally another copy of rsp->gp_seq.
- * These values are updated in the opposite order with memory
- * barriers (or equivalent) during grace-period initialization
- * and cleanup. Now, a false positive can occur if we get an new
- * value of rsp->gp_start and a old value of rsp->jiffies_stall.
- * But given the memory barriers, the only way that this can happen
- * is if one grace period ends and another starts between these
- * two fetches. This is detected by comparing the second fetch
- * of rsp->gp_seq with the previous fetch from rsp->gp_seq.
+ * The idea is to pick up rcu_state.gp_seq, then
+ * rcu_state.jiffies_stall, then rcu_state.gp_start, and finally
+ * another copy of rcu_state.gp_seq. These values are updated in
+ * the opposite order with memory barriers (or equivalent) during
+ * grace-period initialization and cleanup. Now, a false positive
+ * can occur if we get an new value of rcu_state.gp_start and a old
+ * value of rcu_state.jiffies_stall. But given the memory barriers,
+ * the only way that this can happen is if one grace period ends
+ * and another starts between these two fetches. This is detected
+ * by comparing the second fetch of rcu_state.gp_seq with the
+ * previous fetch from rcu_state.gp_seq.
*
- * Given this check, comparisons of jiffies, rsp->jiffies_stall,
- * and rsp->gp_start suffice to forestall false positives.
+ * Given this check, comparisons of jiffies, rcu_state.jiffies_stall,
+ * and rcu_state.gp_start suffice to forestall false positives.
*/
- gs1 = READ_ONCE(rsp->gp_seq);
+ gs1 = READ_ONCE(rcu_state.gp_seq);
smp_rmb(); /* Pick up ->gp_seq first... */
- js = READ_ONCE(rsp->jiffies_stall);
+ js = READ_ONCE(rcu_state.jiffies_stall);
smp_rmb(); /* ...then ->jiffies_stall before the rest... */
- gps = READ_ONCE(rsp->gp_start);
+ gps = READ_ONCE(rcu_state.gp_start);
smp_rmb(); /* ...and finally ->gp_start before ->gp_seq again. */
- gs2 = READ_ONCE(rsp->gp_seq);
+ gs2 = READ_ONCE(rcu_state.gp_seq);
if (gs1 != gs2 ||
ULONG_CMP_LT(j, js) ||
ULONG_CMP_GE(gps, js))
return; /* No stall or GP completed since entering function. */
rnp = rdp->mynode;
jn = jiffies + 3 * rcu_jiffies_till_stall_check() + 3;
- if (rcu_gp_in_progress(rsp) &&
+ if (rcu_gp_in_progress() &&
(READ_ONCE(rnp->qsmask) & rdp->grpmask) &&
- cmpxchg(&rsp->jiffies_stall, js, jn) == js) {
+ cmpxchg(&rcu_state.jiffies_stall, js, jn) == js) {
/* We haven't checked in, so go dump stack. */
- print_cpu_stall(rsp);
+ print_cpu_stall();
- } else if (rcu_gp_in_progress(rsp) &&
+ } else if (rcu_gp_in_progress() &&
ULONG_CMP_GE(j, js + RCU_STALL_RAT_DELAY) &&
- cmpxchg(&rsp->jiffies_stall, js, jn) == js) {
+ cmpxchg(&rcu_state.jiffies_stall, js, jn) == js) {
/* They had a few time units to dump stack, so complain. */
- print_other_cpu_stall(rsp, gs2);
+ print_other_cpu_stall(gs2);
}
}
*/
void rcu_cpu_stall_reset(void)
{
- struct rcu_state *rsp;
-
- for_each_rcu_flavor(rsp)
- WRITE_ONCE(rsp->jiffies_stall, jiffies + ULONG_MAX / 2);
+ WRITE_ONCE(rcu_state.jiffies_stall, jiffies + ULONG_MAX / 2);
}
/* Trace-event wrapper function for trace_rcu_future_grace_period. */
static void trace_rcu_this_gp(struct rcu_node *rnp, struct rcu_data *rdp,
unsigned long gp_seq_req, const char *s)
{
- trace_rcu_future_grace_period(rdp->rsp->name, rnp->gp_seq, gp_seq_req,
+ trace_rcu_future_grace_period(rcu_state.name, rnp->gp_seq, gp_seq_req,
rnp->level, rnp->grplo, rnp->grphi, s);
}
unsigned long gp_seq_req)
{
bool ret = false;
- struct rcu_state *rsp = rdp->rsp;
struct rcu_node *rnp;
/*
}
/* If GP already in progress, just leave, otherwise start one. */
- if (rcu_gp_in_progress(rsp)) {
+ if (rcu_gp_in_progress()) {
trace_rcu_this_gp(rnp, rdp, gp_seq_req, TPS("Startedleafroot"));
goto unlock_out;
}
trace_rcu_this_gp(rnp, rdp, gp_seq_req, TPS("Startedroot"));
- WRITE_ONCE(rsp->gp_flags, rsp->gp_flags | RCU_GP_FLAG_INIT);
- rsp->gp_req_activity = jiffies;
- if (!rsp->gp_kthread) {
+ WRITE_ONCE(rcu_state.gp_flags, rcu_state.gp_flags | RCU_GP_FLAG_INIT);
+ rcu_state.gp_req_activity = jiffies;
+ if (!rcu_state.gp_kthread) {
trace_rcu_this_gp(rnp, rdp, gp_seq_req, TPS("NoGPkthread"));
goto unlock_out;
}
- trace_rcu_grace_period(rsp->name, READ_ONCE(rsp->gp_seq), TPS("newreq"));
+ trace_rcu_grace_period(rcu_state.name, READ_ONCE(rcu_state.gp_seq), TPS("newreq"));
ret = true; /* Caller must wake GP kthread. */
unlock_out:
/* Push furthest requested GP to leaf node and rcu_data structure. */
* Clean up any old requests for the just-ended grace period. Also return
* whether any additional grace periods have been requested.
*/
-static bool rcu_future_gp_cleanup(struct rcu_state *rsp, struct rcu_node *rnp)
+static bool rcu_future_gp_cleanup(struct rcu_node *rnp)
{
bool needmore;
- struct rcu_data *rdp = this_cpu_ptr(rsp->rda);
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
needmore = ULONG_CMP_LT(rnp->gp_seq, rnp->gp_seq_needed);
if (!needmore)
}
/*
- * Awaken the grace-period kthread for the specified flavor of RCU.
- * Don't do a self-awaken, and don't bother awakening when there is
- * nothing for the grace-period kthread to do (as in several CPUs
- * raced to awaken, and we lost), and finally don't try to awaken
- * a kthread that has not yet been created.
+ * Awaken the grace-period kthread. Don't do a self-awaken, and don't
+ * bother awakening when there is nothing for the grace-period kthread
+ * to do (as in several CPUs raced to awaken, and we lost), and finally
+ * don't try to awaken a kthread that has not yet been created.
*/
-static void rcu_gp_kthread_wake(struct rcu_state *rsp)
+static void rcu_gp_kthread_wake(void)
{
- if (current == rsp->gp_kthread ||
- !READ_ONCE(rsp->gp_flags) ||
- !rsp->gp_kthread)
+ if (current == rcu_state.gp_kthread ||
+ !READ_ONCE(rcu_state.gp_flags) ||
+ !rcu_state.gp_kthread)
return;
- swake_up_one(&rsp->gp_wq);
+ swake_up_one(&rcu_state.gp_wq);
}
/*
*
* The caller must hold rnp->lock with interrupts disabled.
*/
-static bool rcu_accelerate_cbs(struct rcu_state *rsp, struct rcu_node *rnp,
- struct rcu_data *rdp)
+static bool rcu_accelerate_cbs(struct rcu_node *rnp, struct rcu_data *rdp)
{
unsigned long gp_seq_req;
bool ret = false;
* accelerating callback invocation to an earlier grace-period
* number.
*/
- gp_seq_req = rcu_seq_snap(&rsp->gp_seq);
+ gp_seq_req = rcu_seq_snap(&rcu_state.gp_seq);
if (rcu_segcblist_accelerate(&rdp->cblist, gp_seq_req))
ret = rcu_start_this_gp(rnp, rdp, gp_seq_req);
/* Trace depending on how much we were able to accelerate. */
if (rcu_segcblist_restempty(&rdp->cblist, RCU_WAIT_TAIL))
- trace_rcu_grace_period(rsp->name, rdp->gp_seq, TPS("AccWaitCB"));
+ trace_rcu_grace_period(rcu_state.name, rdp->gp_seq, TPS("AccWaitCB"));
else
- trace_rcu_grace_period(rsp->name, rdp->gp_seq, TPS("AccReadyCB"));
+ trace_rcu_grace_period(rcu_state.name, rdp->gp_seq, TPS("AccReadyCB"));
return ret;
}
* that a new grace-period request be made, invokes rcu_accelerate_cbs()
* while holding the leaf rcu_node structure's ->lock.
*/
-static void rcu_accelerate_cbs_unlocked(struct rcu_state *rsp,
- struct rcu_node *rnp,
+static void rcu_accelerate_cbs_unlocked(struct rcu_node *rnp,
struct rcu_data *rdp)
{
unsigned long c;
bool needwake;
lockdep_assert_irqs_disabled();
- c = rcu_seq_snap(&rsp->gp_seq);
+ c = rcu_seq_snap(&rcu_state.gp_seq);
if (!rdp->gpwrap && ULONG_CMP_GE(rdp->gp_seq_needed, c)) {
/* Old request still live, so mark recent callbacks. */
(void)rcu_segcblist_accelerate(&rdp->cblist, c);
return;
}
raw_spin_lock_rcu_node(rnp); /* irqs already disabled. */
- needwake = rcu_accelerate_cbs(rsp, rnp, rdp);
+ needwake = rcu_accelerate_cbs(rnp, rdp);
raw_spin_unlock_rcu_node(rnp); /* irqs remain disabled. */
if (needwake)
- rcu_gp_kthread_wake(rsp);
+ rcu_gp_kthread_wake();
}
/*
*
* The caller must hold rnp->lock with interrupts disabled.
*/
-static bool rcu_advance_cbs(struct rcu_state *rsp, struct rcu_node *rnp,
- struct rcu_data *rdp)
+static bool rcu_advance_cbs(struct rcu_node *rnp, struct rcu_data *rdp)
{
raw_lockdep_assert_held_rcu_node(rnp);
rcu_segcblist_advance(&rdp->cblist, rnp->gp_seq);
/* Classify any remaining callbacks. */
- return rcu_accelerate_cbs(rsp, rnp, rdp);
+ return rcu_accelerate_cbs(rnp, rdp);
}
/*
* structure corresponding to the current CPU, and must have irqs disabled.
* Returns true if the grace-period kthread needs to be awakened.
*/
-static bool __note_gp_changes(struct rcu_state *rsp, struct rcu_node *rnp,
- struct rcu_data *rdp)
+static bool __note_gp_changes(struct rcu_node *rnp, struct rcu_data *rdp)
{
bool ret;
bool need_gp;
/* Handle the ends of any preceding grace periods first. */
if (rcu_seq_completed_gp(rdp->gp_seq, rnp->gp_seq) ||
unlikely(READ_ONCE(rdp->gpwrap))) {
- ret = rcu_advance_cbs(rsp, rnp, rdp); /* Advance callbacks. */
- trace_rcu_grace_period(rsp->name, rdp->gp_seq, TPS("cpuend"));
+ ret = rcu_advance_cbs(rnp, rdp); /* Advance callbacks. */
+ trace_rcu_grace_period(rcu_state.name, rdp->gp_seq, TPS("cpuend"));
} else {
- ret = rcu_accelerate_cbs(rsp, rnp, rdp); /* Recent callbacks. */
+ ret = rcu_accelerate_cbs(rnp, rdp); /* Recent callbacks. */
}
/* Now handle the beginnings of any new-to-this-CPU grace periods. */
* set up to detect a quiescent state, otherwise don't
* go looking for one.
*/
- trace_rcu_grace_period(rsp->name, rnp->gp_seq, TPS("cpustart"));
+ trace_rcu_grace_period(rcu_state.name, rnp->gp_seq, TPS("cpustart"));
need_gp = !!(rnp->qsmask & rdp->grpmask);
rdp->cpu_no_qs.b.norm = need_gp;
- rdp->rcu_qs_ctr_snap = __this_cpu_read(rcu_dynticks.rcu_qs_ctr);
rdp->core_needs_qs = need_gp;
zero_cpu_stall_ticks(rdp);
}
return ret;
}
-static void note_gp_changes(struct rcu_state *rsp, struct rcu_data *rdp)
+static void note_gp_changes(struct rcu_data *rdp)
{
unsigned long flags;
bool needwake;
local_irq_restore(flags);
return;
}
- needwake = __note_gp_changes(rsp, rnp, rdp);
+ needwake = __note_gp_changes(rnp, rdp);
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
if (needwake)
- rcu_gp_kthread_wake(rsp);
+ rcu_gp_kthread_wake();
}
-static void rcu_gp_slow(struct rcu_state *rsp, int delay)
+static void rcu_gp_slow(int delay)
{
if (delay > 0 &&
- !(rcu_seq_ctr(rsp->gp_seq) %
+ !(rcu_seq_ctr(rcu_state.gp_seq) %
(rcu_num_nodes * PER_RCU_NODE_PERIOD * delay)))
schedule_timeout_uninterruptible(delay);
}
/*
* Initialize a new grace period. Return false if no grace period required.
*/
-static bool rcu_gp_init(struct rcu_state *rsp)
+static bool rcu_gp_init(void)
{
unsigned long flags;
unsigned long oldmask;
unsigned long mask;
struct rcu_data *rdp;
- struct rcu_node *rnp = rcu_get_root(rsp);
+ struct rcu_node *rnp = rcu_get_root();
- WRITE_ONCE(rsp->gp_activity, jiffies);
+ WRITE_ONCE(rcu_state.gp_activity, jiffies);
raw_spin_lock_irq_rcu_node(rnp);
- if (!READ_ONCE(rsp->gp_flags)) {
+ if (!READ_ONCE(rcu_state.gp_flags)) {
/* Spurious wakeup, tell caller to go back to sleep. */
raw_spin_unlock_irq_rcu_node(rnp);
return false;
}
- WRITE_ONCE(rsp->gp_flags, 0); /* Clear all flags: New grace period. */
+ WRITE_ONCE(rcu_state.gp_flags, 0); /* Clear all flags: New GP. */
- if (WARN_ON_ONCE(rcu_gp_in_progress(rsp))) {
+ if (WARN_ON_ONCE(rcu_gp_in_progress())) {
/*
* Grace period already in progress, don't start another.
* Not supposed to be able to happen.
}
/* Advance to a new grace period and initialize state. */
- record_gp_stall_check_time(rsp);
+ record_gp_stall_check_time();
/* Record GP times before starting GP, hence rcu_seq_start(). */
- rcu_seq_start(&rsp->gp_seq);
- trace_rcu_grace_period(rsp->name, rsp->gp_seq, TPS("start"));
+ rcu_seq_start(&rcu_state.gp_seq);
+ trace_rcu_grace_period(rcu_state.name, rcu_state.gp_seq, TPS("start"));
raw_spin_unlock_irq_rcu_node(rnp);
/*
* for subsequent online CPUs, and that quiescent-state forcing
* will handle subsequent offline CPUs.
*/
- rsp->gp_state = RCU_GP_ONOFF;
- rcu_for_each_leaf_node(rsp, rnp) {
- spin_lock(&rsp->ofl_lock);
+ rcu_state.gp_state = RCU_GP_ONOFF;
+ rcu_for_each_leaf_node(rnp) {
+ raw_spin_lock(&rcu_state.ofl_lock);
raw_spin_lock_irq_rcu_node(rnp);
if (rnp->qsmaskinit == rnp->qsmaskinitnext &&
!rnp->wait_blkd_tasks) {
/* Nothing to do on this leaf rcu_node structure. */
raw_spin_unlock_irq_rcu_node(rnp);
- spin_unlock(&rsp->ofl_lock);
+ raw_spin_unlock(&rcu_state.ofl_lock);
continue;
}
}
raw_spin_unlock_irq_rcu_node(rnp);
- spin_unlock(&rsp->ofl_lock);
+ raw_spin_unlock(&rcu_state.ofl_lock);
}
- rcu_gp_slow(rsp, gp_preinit_delay); /* Races with CPU hotplug. */
+ rcu_gp_slow(gp_preinit_delay); /* Races with CPU hotplug. */
/*
* Set the quiescent-state-needed bits in all the rcu_node
- * structures for all currently online CPUs in breadth-first order,
- * starting from the root rcu_node structure, relying on the layout
- * of the tree within the rsp->node[] array. Note that other CPUs
- * will access only the leaves of the hierarchy, thus seeing that no
- * grace period is in progress, at least until the corresponding
- * leaf node has been initialized.
+ * structures for all currently online CPUs in breadth-first
+ * order, starting from the root rcu_node structure, relying on the
+ * layout of the tree within the rcu_state.node[] array. Note that
+ * other CPUs will access only the leaves of the hierarchy, thus
+ * seeing that no grace period is in progress, at least until the
+ * corresponding leaf node has been initialized.
*
* The grace period cannot complete until the initialization
* process finishes, because this kthread handles both.
*/
- rsp->gp_state = RCU_GP_INIT;
- rcu_for_each_node_breadth_first(rsp, rnp) {
- rcu_gp_slow(rsp, gp_init_delay);
+ rcu_state.gp_state = RCU_GP_INIT;
+ rcu_for_each_node_breadth_first(rnp) {
+ rcu_gp_slow(gp_init_delay);
raw_spin_lock_irqsave_rcu_node(rnp, flags);
- rdp = this_cpu_ptr(rsp->rda);
- rcu_preempt_check_blocked_tasks(rsp, rnp);
+ rdp = this_cpu_ptr(&rcu_data);
+ rcu_preempt_check_blocked_tasks(rnp);
rnp->qsmask = rnp->qsmaskinit;
- WRITE_ONCE(rnp->gp_seq, rsp->gp_seq);
+ WRITE_ONCE(rnp->gp_seq, rcu_state.gp_seq);
if (rnp == rdp->mynode)
- (void)__note_gp_changes(rsp, rnp, rdp);
+ (void)__note_gp_changes(rnp, rdp);
rcu_preempt_boost_start_gp(rnp);
- trace_rcu_grace_period_init(rsp->name, rnp->gp_seq,
+ trace_rcu_grace_period_init(rcu_state.name, rnp->gp_seq,
rnp->level, rnp->grplo,
rnp->grphi, rnp->qsmask);
/* Quiescent states for tasks on any now-offline CPUs. */
mask = rnp->qsmask & ~rnp->qsmaskinitnext;
rnp->rcu_gp_init_mask = mask;
if ((mask || rnp->wait_blkd_tasks) && rcu_is_leaf_node(rnp))
- rcu_report_qs_rnp(mask, rsp, rnp, rnp->gp_seq, flags);
+ rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags);
else
raw_spin_unlock_irq_rcu_node(rnp);
cond_resched_tasks_rcu_qs();
- WRITE_ONCE(rsp->gp_activity, jiffies);
+ WRITE_ONCE(rcu_state.gp_activity, jiffies);
}
return true;
* Helper function for swait_event_idle_exclusive() wakeup at force-quiescent-state
* time.
*/
-static bool rcu_gp_fqs_check_wake(struct rcu_state *rsp, int *gfp)
+static bool rcu_gp_fqs_check_wake(int *gfp)
{
- struct rcu_node *rnp = rcu_get_root(rsp);
+ struct rcu_node *rnp = rcu_get_root();
/* Someone like call_rcu() requested a force-quiescent-state scan. */
- *gfp = READ_ONCE(rsp->gp_flags);
+ *gfp = READ_ONCE(rcu_state.gp_flags);
if (*gfp & RCU_GP_FLAG_FQS)
return true;
/*
* Do one round of quiescent-state forcing.
*/
-static void rcu_gp_fqs(struct rcu_state *rsp, bool first_time)
+static void rcu_gp_fqs(bool first_time)
{
- struct rcu_node *rnp = rcu_get_root(rsp);
+ struct rcu_node *rnp = rcu_get_root();
- WRITE_ONCE(rsp->gp_activity, jiffies);
- rsp->n_force_qs++;
+ WRITE_ONCE(rcu_state.gp_activity, jiffies);
+ rcu_state.n_force_qs++;
if (first_time) {
/* Collect dyntick-idle snapshots. */
- force_qs_rnp(rsp, dyntick_save_progress_counter);
+ force_qs_rnp(dyntick_save_progress_counter);
} else {
/* Handle dyntick-idle and offline CPUs. */
- force_qs_rnp(rsp, rcu_implicit_dynticks_qs);
+ force_qs_rnp(rcu_implicit_dynticks_qs);
}
/* Clear flag to prevent immediate re-entry. */
- if (READ_ONCE(rsp->gp_flags) & RCU_GP_FLAG_FQS) {
+ if (READ_ONCE(rcu_state.gp_flags) & RCU_GP_FLAG_FQS) {
raw_spin_lock_irq_rcu_node(rnp);
- WRITE_ONCE(rsp->gp_flags,
- READ_ONCE(rsp->gp_flags) & ~RCU_GP_FLAG_FQS);
+ WRITE_ONCE(rcu_state.gp_flags,
+ READ_ONCE(rcu_state.gp_flags) & ~RCU_GP_FLAG_FQS);
raw_spin_unlock_irq_rcu_node(rnp);
}
}
+/*
+ * Loop doing repeated quiescent-state forcing until the grace period ends.
+ */
+static void rcu_gp_fqs_loop(void)
+{
+ bool first_gp_fqs;
+ int gf;
+ unsigned long j;
+ int ret;
+ struct rcu_node *rnp = rcu_get_root();
+
+ first_gp_fqs = true;
+ j = READ_ONCE(jiffies_till_first_fqs);
+ ret = 0;
+ for (;;) {
+ if (!ret) {
+ rcu_state.jiffies_force_qs = jiffies + j;
+ WRITE_ONCE(rcu_state.jiffies_kick_kthreads,
+ jiffies + 3 * j);
+ }
+ trace_rcu_grace_period(rcu_state.name,
+ READ_ONCE(rcu_state.gp_seq),
+ TPS("fqswait"));
+ rcu_state.gp_state = RCU_GP_WAIT_FQS;
+ ret = swait_event_idle_timeout_exclusive(
+ rcu_state.gp_wq, rcu_gp_fqs_check_wake(&gf), j);
+ rcu_state.gp_state = RCU_GP_DOING_FQS;
+ /* Locking provides needed memory barriers. */
+ /* If grace period done, leave loop. */
+ if (!READ_ONCE(rnp->qsmask) &&
+ !rcu_preempt_blocked_readers_cgp(rnp))
+ break;
+ /* If time for quiescent-state forcing, do it. */
+ if (ULONG_CMP_GE(jiffies, rcu_state.jiffies_force_qs) ||
+ (gf & RCU_GP_FLAG_FQS)) {
+ trace_rcu_grace_period(rcu_state.name,
+ READ_ONCE(rcu_state.gp_seq),
+ TPS("fqsstart"));
+ rcu_gp_fqs(first_gp_fqs);
+ first_gp_fqs = false;
+ trace_rcu_grace_period(rcu_state.name,
+ READ_ONCE(rcu_state.gp_seq),
+ TPS("fqsend"));
+ cond_resched_tasks_rcu_qs();
+ WRITE_ONCE(rcu_state.gp_activity, jiffies);
+ ret = 0; /* Force full wait till next FQS. */
+ j = READ_ONCE(jiffies_till_next_fqs);
+ } else {
+ /* Deal with stray signal. */
+ cond_resched_tasks_rcu_qs();
+ WRITE_ONCE(rcu_state.gp_activity, jiffies);
+ WARN_ON(signal_pending(current));
+ trace_rcu_grace_period(rcu_state.name,
+ READ_ONCE(rcu_state.gp_seq),
+ TPS("fqswaitsig"));
+ ret = 1; /* Keep old FQS timing. */
+ j = jiffies;
+ if (time_after(jiffies, rcu_state.jiffies_force_qs))
+ j = 1;
+ else
+ j = rcu_state.jiffies_force_qs - j;
+ }
+ }
+}
+
/*
* Clean up after the old grace period.
*/
-static void rcu_gp_cleanup(struct rcu_state *rsp)
+static void rcu_gp_cleanup(void)
{
unsigned long gp_duration;
bool needgp = false;
unsigned long new_gp_seq;
struct rcu_data *rdp;
- struct rcu_node *rnp = rcu_get_root(rsp);
+ struct rcu_node *rnp = rcu_get_root();
struct swait_queue_head *sq;
- WRITE_ONCE(rsp->gp_activity, jiffies);
+ WRITE_ONCE(rcu_state.gp_activity, jiffies);
raw_spin_lock_irq_rcu_node(rnp);
- gp_duration = jiffies - rsp->gp_start;
- if (gp_duration > rsp->gp_max)
- rsp->gp_max = gp_duration;
+ gp_duration = jiffies - rcu_state.gp_start;
+ if (gp_duration > rcu_state.gp_max)
+ rcu_state.gp_max = gp_duration;
/*
* We know the grace period is complete, but to everyone else
* the rcu_node structures before the beginning of the next grace
* period is recorded in any of the rcu_node structures.
*/
- new_gp_seq = rsp->gp_seq;
+ new_gp_seq = rcu_state.gp_seq;
rcu_seq_end(&new_gp_seq);
- rcu_for_each_node_breadth_first(rsp, rnp) {
+ rcu_for_each_node_breadth_first(rnp) {
raw_spin_lock_irq_rcu_node(rnp);
if (WARN_ON_ONCE(rcu_preempt_blocked_readers_cgp(rnp)))
- dump_blkd_tasks(rsp, rnp, 10);
+ dump_blkd_tasks(rnp, 10);
WARN_ON_ONCE(rnp->qsmask);
WRITE_ONCE(rnp->gp_seq, new_gp_seq);
- rdp = this_cpu_ptr(rsp->rda);
+ rdp = this_cpu_ptr(&rcu_data);
if (rnp == rdp->mynode)
- needgp = __note_gp_changes(rsp, rnp, rdp) || needgp;
+ needgp = __note_gp_changes(rnp, rdp) || needgp;
/* smp_mb() provided by prior unlock-lock pair. */
- needgp = rcu_future_gp_cleanup(rsp, rnp) || needgp;
+ needgp = rcu_future_gp_cleanup(rnp) || needgp;
sq = rcu_nocb_gp_get(rnp);
raw_spin_unlock_irq_rcu_node(rnp);
rcu_nocb_gp_cleanup(sq);
cond_resched_tasks_rcu_qs();
- WRITE_ONCE(rsp->gp_activity, jiffies);
- rcu_gp_slow(rsp, gp_cleanup_delay);
+ WRITE_ONCE(rcu_state.gp_activity, jiffies);
+ rcu_gp_slow(gp_cleanup_delay);
}
- rnp = rcu_get_root(rsp);
- raw_spin_lock_irq_rcu_node(rnp); /* GP before rsp->gp_seq update. */
+ rnp = rcu_get_root();
+ raw_spin_lock_irq_rcu_node(rnp); /* GP before ->gp_seq update. */
/* Declare grace period done. */
- rcu_seq_end(&rsp->gp_seq);
- trace_rcu_grace_period(rsp->name, rsp->gp_seq, TPS("end"));
- rsp->gp_state = RCU_GP_IDLE;
+ rcu_seq_end(&rcu_state.gp_seq);
+ trace_rcu_grace_period(rcu_state.name, rcu_state.gp_seq, TPS("end"));
+ rcu_state.gp_state = RCU_GP_IDLE;
/* Check for GP requests since above loop. */
- rdp = this_cpu_ptr(rsp->rda);
+ rdp = this_cpu_ptr(&rcu_data);
if (!needgp && ULONG_CMP_LT(rnp->gp_seq, rnp->gp_seq_needed)) {
trace_rcu_this_gp(rnp, rdp, rnp->gp_seq_needed,
TPS("CleanupMore"));
needgp = true;
}
/* Advance CBs to reduce false positives below. */
- if (!rcu_accelerate_cbs(rsp, rnp, rdp) && needgp) {
- WRITE_ONCE(rsp->gp_flags, RCU_GP_FLAG_INIT);
- rsp->gp_req_activity = jiffies;
- trace_rcu_grace_period(rsp->name, READ_ONCE(rsp->gp_seq),
+ if (!rcu_accelerate_cbs(rnp, rdp) && needgp) {
+ WRITE_ONCE(rcu_state.gp_flags, RCU_GP_FLAG_INIT);
+ rcu_state.gp_req_activity = jiffies;
+ trace_rcu_grace_period(rcu_state.name,
+ READ_ONCE(rcu_state.gp_seq),
TPS("newreq"));
} else {
- WRITE_ONCE(rsp->gp_flags, rsp->gp_flags & RCU_GP_FLAG_INIT);
+ WRITE_ONCE(rcu_state.gp_flags,
+ rcu_state.gp_flags & RCU_GP_FLAG_INIT);
}
raw_spin_unlock_irq_rcu_node(rnp);
}
/*
* Body of kthread that handles grace periods.
*/
-static int __noreturn rcu_gp_kthread(void *arg)
+static int __noreturn rcu_gp_kthread(void *unused)
{
- bool first_gp_fqs;
- int gf;
- unsigned long j;
- int ret;
- struct rcu_state *rsp = arg;
- struct rcu_node *rnp = rcu_get_root(rsp);
-
rcu_bind_gp_kthread();
for (;;) {
/* Handle grace-period start. */
for (;;) {
- trace_rcu_grace_period(rsp->name,
- READ_ONCE(rsp->gp_seq),
+ trace_rcu_grace_period(rcu_state.name,
+ READ_ONCE(rcu_state.gp_seq),
TPS("reqwait"));
- rsp->gp_state = RCU_GP_WAIT_GPS;
- swait_event_idle_exclusive(rsp->gp_wq, READ_ONCE(rsp->gp_flags) &
- RCU_GP_FLAG_INIT);
- rsp->gp_state = RCU_GP_DONE_GPS;
+ rcu_state.gp_state = RCU_GP_WAIT_GPS;
+ swait_event_idle_exclusive(rcu_state.gp_wq,
+ READ_ONCE(rcu_state.gp_flags) &
+ RCU_GP_FLAG_INIT);
+ rcu_state.gp_state = RCU_GP_DONE_GPS;
/* Locking provides needed memory barrier. */
- if (rcu_gp_init(rsp))
+ if (rcu_gp_init())
break;
cond_resched_tasks_rcu_qs();
- WRITE_ONCE(rsp->gp_activity, jiffies);
+ WRITE_ONCE(rcu_state.gp_activity, jiffies);
WARN_ON(signal_pending(current));
- trace_rcu_grace_period(rsp->name,
- READ_ONCE(rsp->gp_seq),
+ trace_rcu_grace_period(rcu_state.name,
+ READ_ONCE(rcu_state.gp_seq),
TPS("reqwaitsig"));
}
/* Handle quiescent-state forcing. */
- first_gp_fqs = true;
- j = jiffies_till_first_fqs;
- ret = 0;
- for (;;) {
- if (!ret) {
- rsp->jiffies_force_qs = jiffies + j;
- WRITE_ONCE(rsp->jiffies_kick_kthreads,
- jiffies + 3 * j);
- }
- trace_rcu_grace_period(rsp->name,
- READ_ONCE(rsp->gp_seq),
- TPS("fqswait"));
- rsp->gp_state = RCU_GP_WAIT_FQS;
- ret = swait_event_idle_timeout_exclusive(rsp->gp_wq,
- rcu_gp_fqs_check_wake(rsp, &gf), j);
- rsp->gp_state = RCU_GP_DOING_FQS;
- /* Locking provides needed memory barriers. */
- /* If grace period done, leave loop. */
- if (!READ_ONCE(rnp->qsmask) &&
- !rcu_preempt_blocked_readers_cgp(rnp))
- break;
- /* If time for quiescent-state forcing, do it. */
- if (ULONG_CMP_GE(jiffies, rsp->jiffies_force_qs) ||
- (gf & RCU_GP_FLAG_FQS)) {
- trace_rcu_grace_period(rsp->name,
- READ_ONCE(rsp->gp_seq),
- TPS("fqsstart"));
- rcu_gp_fqs(rsp, first_gp_fqs);
- first_gp_fqs = false;
- trace_rcu_grace_period(rsp->name,
- READ_ONCE(rsp->gp_seq),
- TPS("fqsend"));
- cond_resched_tasks_rcu_qs();
- WRITE_ONCE(rsp->gp_activity, jiffies);
- ret = 0; /* Force full wait till next FQS. */
- j = jiffies_till_next_fqs;
- } else {
- /* Deal with stray signal. */
- cond_resched_tasks_rcu_qs();
- WRITE_ONCE(rsp->gp_activity, jiffies);
- WARN_ON(signal_pending(current));
- trace_rcu_grace_period(rsp->name,
- READ_ONCE(rsp->gp_seq),
- TPS("fqswaitsig"));
- ret = 1; /* Keep old FQS timing. */
- j = jiffies;
- if (time_after(jiffies, rsp->jiffies_force_qs))
- j = 1;
- else
- j = rsp->jiffies_force_qs - j;
- }
- }
+ rcu_gp_fqs_loop();
/* Handle grace-period end. */
- rsp->gp_state = RCU_GP_CLEANUP;
- rcu_gp_cleanup(rsp);
- rsp->gp_state = RCU_GP_CLEANED;
+ rcu_state.gp_state = RCU_GP_CLEANUP;
+ rcu_gp_cleanup();
+ rcu_state.gp_state = RCU_GP_CLEANED;
}
}
/*
- * Report a full set of quiescent states to the specified rcu_state data
- * structure. Invoke rcu_gp_kthread_wake() to awaken the grace-period
- * kthread if another grace period is required. Whether we wake
- * the grace-period kthread or it awakens itself for the next round
- * of quiescent-state forcing, that kthread will clean up after the
- * just-completed grace period. Note that the caller must hold rnp->lock,
- * which is released before return.
+ * Report a full set of quiescent states to the rcu_state data structure.
+ * Invoke rcu_gp_kthread_wake() to awaken the grace-period kthread if
+ * another grace period is required. Whether we wake the grace-period
+ * kthread or it awakens itself for the next round of quiescent-state
+ * forcing, that kthread will clean up after the just-completed grace
+ * period. Note that the caller must hold rnp->lock, which is released
+ * before return.
*/
-static void rcu_report_qs_rsp(struct rcu_state *rsp, unsigned long flags)
- __releases(rcu_get_root(rsp)->lock)
+static void rcu_report_qs_rsp(unsigned long flags)
+ __releases(rcu_get_root()->lock)
{
- raw_lockdep_assert_held_rcu_node(rcu_get_root(rsp));
- WARN_ON_ONCE(!rcu_gp_in_progress(rsp));
- WRITE_ONCE(rsp->gp_flags, READ_ONCE(rsp->gp_flags) | RCU_GP_FLAG_FQS);
- raw_spin_unlock_irqrestore_rcu_node(rcu_get_root(rsp), flags);
- rcu_gp_kthread_wake(rsp);
+ raw_lockdep_assert_held_rcu_node(rcu_get_root());
+ WARN_ON_ONCE(!rcu_gp_in_progress());
+ WRITE_ONCE(rcu_state.gp_flags,
+ READ_ONCE(rcu_state.gp_flags) | RCU_GP_FLAG_FQS);
+ raw_spin_unlock_irqrestore_rcu_node(rcu_get_root(), flags);
+ rcu_gp_kthread_wake();
}
/*
* disabled. This allows propagating quiescent state due to resumed tasks
* during grace-period initialization.
*/
-static void
-rcu_report_qs_rnp(unsigned long mask, struct rcu_state *rsp,
- struct rcu_node *rnp, unsigned long gps, unsigned long flags)
+static void rcu_report_qs_rnp(unsigned long mask, struct rcu_node *rnp,
+ unsigned long gps, unsigned long flags)
__releases(rnp->lock)
{
unsigned long oldmask = 0;
WARN_ON_ONCE(!rcu_is_leaf_node(rnp) &&
rcu_preempt_blocked_readers_cgp(rnp));
rnp->qsmask &= ~mask;
- trace_rcu_quiescent_state_report(rsp->name, rnp->gp_seq,
+ trace_rcu_quiescent_state_report(rcu_state.name, rnp->gp_seq,
mask, rnp->qsmask, rnp->level,
rnp->grplo, rnp->grphi,
!!rnp->gp_tasks);
* state for this grace period. Invoke rcu_report_qs_rsp()
* to clean up and start the next grace period if one is needed.
*/
- rcu_report_qs_rsp(rsp, flags); /* releases rnp->lock. */
+ rcu_report_qs_rsp(flags); /* releases rnp->lock. */
}
/*
* Record a quiescent state for all tasks that were previously queued
* on the specified rcu_node structure and that were blocking the current
- * RCU grace period. The caller must hold the specified rnp->lock with
+ * RCU grace period. The caller must hold the corresponding rnp->lock with
* irqs disabled, and this lock is released upon return, but irqs remain
* disabled.
*/
static void __maybe_unused
-rcu_report_unblock_qs_rnp(struct rcu_state *rsp,
- struct rcu_node *rnp, unsigned long flags)
+rcu_report_unblock_qs_rnp(struct rcu_node *rnp, unsigned long flags)
__releases(rnp->lock)
{
unsigned long gps;
struct rcu_node *rnp_p;
raw_lockdep_assert_held_rcu_node(rnp);
- if (WARN_ON_ONCE(rcu_state_p == &rcu_sched_state) ||
- WARN_ON_ONCE(rsp != rcu_state_p) ||
+ if (WARN_ON_ONCE(!IS_ENABLED(CONFIG_PREEMPT)) ||
WARN_ON_ONCE(rcu_preempt_blocked_readers_cgp(rnp)) ||
rnp->qsmask != 0) {
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
* Only one rcu_node structure in the tree, so don't
* try to report up to its nonexistent parent!
*/
- rcu_report_qs_rsp(rsp, flags);
+ rcu_report_qs_rsp(flags);
return;
}
mask = rnp->grpmask;
raw_spin_unlock_rcu_node(rnp); /* irqs remain disabled. */
raw_spin_lock_rcu_node(rnp_p); /* irqs already disabled. */
- rcu_report_qs_rnp(mask, rsp, rnp_p, gps, flags);
+ rcu_report_qs_rnp(mask, rnp_p, gps, flags);
}
/*
* structure. This must be called from the specified CPU.
*/
static void
-rcu_report_qs_rdp(int cpu, struct rcu_state *rsp, struct rcu_data *rdp)
+rcu_report_qs_rdp(int cpu, struct rcu_data *rdp)
{
unsigned long flags;
unsigned long mask;
* within the current grace period.
*/
rdp->cpu_no_qs.b.norm = true; /* need qs for new gp. */
- rdp->rcu_qs_ctr_snap = __this_cpu_read(rcu_dynticks.rcu_qs_ctr);
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
return;
}
* This GP can't end until cpu checks in, so all of our
* callbacks can be processed during the next GP.
*/
- needwake = rcu_accelerate_cbs(rsp, rnp, rdp);
+ needwake = rcu_accelerate_cbs(rnp, rdp);
- rcu_report_qs_rnp(mask, rsp, rnp, rnp->gp_seq, flags);
+ rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags);
/* ^^^ Released rnp->lock */
if (needwake)
- rcu_gp_kthread_wake(rsp);
+ rcu_gp_kthread_wake();
}
}
* quiescent state for this grace period, and record that fact if so.
*/
static void
-rcu_check_quiescent_state(struct rcu_state *rsp, struct rcu_data *rdp)
+rcu_check_quiescent_state(struct rcu_data *rdp)
{
/* Check for grace-period ends and beginnings. */
- note_gp_changes(rsp, rdp);
+ note_gp_changes(rdp);
/*
* Does this CPU still need to do its part for current grace period?
* Tell RCU we are done (but rcu_report_qs_rdp() will be the
* judge of that).
*/
- rcu_report_qs_rdp(rdp->cpu, rsp, rdp);
+ rcu_report_qs_rdp(rdp->cpu, rdp);
}
/*
- * Trace the fact that this CPU is going offline.
+ * Near the end of the offline process. Trace the fact that this CPU
+ * is going offline.
*/
-static void rcu_cleanup_dying_cpu(struct rcu_state *rsp)
+int rcutree_dying_cpu(unsigned int cpu)
{
RCU_TRACE(bool blkd;)
- RCU_TRACE(struct rcu_data *rdp = this_cpu_ptr(rsp->rda);)
+ RCU_TRACE(struct rcu_data *rdp = this_cpu_ptr(&rcu_data);)
RCU_TRACE(struct rcu_node *rnp = rdp->mynode;)
if (!IS_ENABLED(CONFIG_HOTPLUG_CPU))
- return;
+ return 0;
RCU_TRACE(blkd = !!(rnp->qsmask & rdp->grpmask);)
- trace_rcu_grace_period(rsp->name, rnp->gp_seq,
+ trace_rcu_grace_period(rcu_state.name, rnp->gp_seq,
blkd ? TPS("cpuofl") : TPS("cpuofl-bgp"));
+ return 0;
}
/*
* There can only be one CPU hotplug operation at a time, so no need for
* explicit locking.
*/
-static void rcu_cleanup_dead_cpu(int cpu, struct rcu_state *rsp)
+int rcutree_dead_cpu(unsigned int cpu)
{
- struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
+ struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */
if (!IS_ENABLED(CONFIG_HOTPLUG_CPU))
- return;
+ return 0;
/* Adjust any no-longer-needed kthreads. */
rcu_boost_kthread_setaffinity(rnp, -1);
+ /* Do any needed no-CB deferred wakeups from this CPU. */
+ do_nocb_deferred_wakeup(per_cpu_ptr(&rcu_data, cpu));
+ return 0;
}
/*
* Invoke any RCU callbacks that have made it to the end of their grace
* period. Thottle as specified by rdp->blimit.
*/
-static void rcu_do_batch(struct rcu_state *rsp, struct rcu_data *rdp)
+static void rcu_do_batch(struct rcu_data *rdp)
{
unsigned long flags;
struct rcu_head *rhp;
/* If no callbacks are ready, just return. */
if (!rcu_segcblist_ready_cbs(&rdp->cblist)) {
- trace_rcu_batch_start(rsp->name,
+ trace_rcu_batch_start(rcu_state.name,
rcu_segcblist_n_lazy_cbs(&rdp->cblist),
rcu_segcblist_n_cbs(&rdp->cblist), 0);
- trace_rcu_batch_end(rsp->name, 0,
+ trace_rcu_batch_end(rcu_state.name, 0,
!rcu_segcblist_empty(&rdp->cblist),
need_resched(), is_idle_task(current),
rcu_is_callbacks_kthread());
local_irq_save(flags);
WARN_ON_ONCE(cpu_is_offline(smp_processor_id()));
bl = rdp->blimit;
- trace_rcu_batch_start(rsp->name, rcu_segcblist_n_lazy_cbs(&rdp->cblist),
+ trace_rcu_batch_start(rcu_state.name,
+ rcu_segcblist_n_lazy_cbs(&rdp->cblist),
rcu_segcblist_n_cbs(&rdp->cblist), bl);
rcu_segcblist_extract_done_cbs(&rdp->cblist, &rcl);
local_irq_restore(flags);
rhp = rcu_cblist_dequeue(&rcl);
for (; rhp; rhp = rcu_cblist_dequeue(&rcl)) {
debug_rcu_head_unqueue(rhp);
- if (__rcu_reclaim(rsp->name, rhp))
+ if (__rcu_reclaim(rcu_state.name, rhp))
rcu_cblist_dequeued_lazy(&rcl);
/*
* Stop only if limit reached and CPU has something to do.
local_irq_save(flags);
count = -rcl.len;
- trace_rcu_batch_end(rsp->name, count, !!rcl.head, need_resched(),
+ trace_rcu_batch_end(rcu_state.name, count, !!rcl.head, need_resched(),
is_idle_task(current), rcu_is_callbacks_kthread());
/* Update counts and requeue any remaining callbacks. */
/* Reset ->qlen_last_fqs_check trigger if enough CBs have drained. */
if (count == 0 && rdp->qlen_last_fqs_check != 0) {
rdp->qlen_last_fqs_check = 0;
- rdp->n_force_qs_snap = rsp->n_force_qs;
+ rdp->n_force_qs_snap = rcu_state.n_force_qs;
} else if (count < rdp->qlen_last_fqs_check - qhimark)
rdp->qlen_last_fqs_check = count;
void rcu_check_callbacks(int user)
{
trace_rcu_utilization(TPS("Start scheduler-tick"));
- increment_cpu_stall_ticks();
- if (user || rcu_is_cpu_rrupt_from_idle()) {
-
- /*
- * Get here if this CPU took its interrupt from user
- * mode or from the idle loop, and if this is not a
- * nested interrupt. In this case, the CPU is in
- * a quiescent state, so note it.
- *
- * No memory barrier is required here because both
- * rcu_sched_qs() and rcu_bh_qs() reference only CPU-local
- * variables that other CPUs neither access nor modify,
- * at least not while the corresponding CPU is online.
- */
-
- rcu_sched_qs();
- rcu_bh_qs();
- rcu_note_voluntary_context_switch(current);
-
- } else if (!in_softirq()) {
-
- /*
- * Get here if this CPU did not take its interrupt from
- * softirq, in other words, if it is not interrupting
- * a rcu_bh read-side critical section. This is an _bh
- * critical section, so note it.
- */
-
- rcu_bh_qs();
+ raw_cpu_inc(rcu_data.ticks_this_gp);
+ /* The load-acquire pairs with the store-release setting to true. */
+ if (smp_load_acquire(this_cpu_ptr(&rcu_data.rcu_urgent_qs))) {
+ /* Idle and userspace execution already are quiescent states. */
+ if (!rcu_is_cpu_rrupt_from_idle() && !user) {
+ set_tsk_need_resched(current);
+ set_preempt_need_resched();
+ }
+ __this_cpu_write(rcu_data.rcu_urgent_qs, false);
}
- rcu_preempt_check_callbacks();
+ rcu_flavor_check_callbacks(user);
if (rcu_pending())
invoke_rcu_core();
*
* The caller must have suppressed start of new grace periods.
*/
-static void force_qs_rnp(struct rcu_state *rsp, int (*f)(struct rcu_data *rsp))
+static void force_qs_rnp(int (*f)(struct rcu_data *rdp))
{
int cpu;
unsigned long flags;
unsigned long mask;
struct rcu_node *rnp;
- rcu_for_each_leaf_node(rsp, rnp) {
+ rcu_for_each_leaf_node(rnp) {
cond_resched_tasks_rcu_qs();
mask = 0;
raw_spin_lock_irqsave_rcu_node(rnp, flags);
if (rnp->qsmask == 0) {
- if (rcu_state_p == &rcu_sched_state ||
- rsp != rcu_state_p ||
+ if (!IS_ENABLED(CONFIG_PREEMPT) ||
rcu_preempt_blocked_readers_cgp(rnp)) {
/*
* No point in scanning bits because they
for_each_leaf_node_possible_cpu(rnp, cpu) {
unsigned long bit = leaf_node_cpu_bit(rnp, cpu);
if ((rnp->qsmask & bit) != 0) {
- if (f(per_cpu_ptr(rsp->rda, cpu)))
+ if (f(per_cpu_ptr(&rcu_data, cpu)))
mask |= bit;
}
}
if (mask != 0) {
/* Idle/offline CPUs, report (releases rnp->lock). */
- rcu_report_qs_rnp(mask, rsp, rnp, rnp->gp_seq, flags);
+ rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags);
} else {
/* Nothing to do here, so just drop the lock. */
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
* Force quiescent states on reluctant CPUs, and also detect which
* CPUs are in dyntick-idle mode.
*/
-static void force_quiescent_state(struct rcu_state *rsp)
+static void force_quiescent_state(void)
{
unsigned long flags;
bool ret;
struct rcu_node *rnp_old = NULL;
/* Funnel through hierarchy to reduce memory contention. */
- rnp = __this_cpu_read(rsp->rda->mynode);
+ rnp = __this_cpu_read(rcu_data.mynode);
for (; rnp != NULL; rnp = rnp->parent) {
- ret = (READ_ONCE(rsp->gp_flags) & RCU_GP_FLAG_FQS) ||
+ ret = (READ_ONCE(rcu_state.gp_flags) & RCU_GP_FLAG_FQS) ||
!raw_spin_trylock(&rnp->fqslock);
if (rnp_old != NULL)
raw_spin_unlock(&rnp_old->fqslock);
return;
rnp_old = rnp;
}
- /* rnp_old == rcu_get_root(rsp), rnp == NULL. */
+ /* rnp_old == rcu_get_root(), rnp == NULL. */
/* Reached the root of the rcu_node tree, acquire lock. */
raw_spin_lock_irqsave_rcu_node(rnp_old, flags);
raw_spin_unlock(&rnp_old->fqslock);
- if (READ_ONCE(rsp->gp_flags) & RCU_GP_FLAG_FQS) {
+ if (READ_ONCE(rcu_state.gp_flags) & RCU_GP_FLAG_FQS) {
raw_spin_unlock_irqrestore_rcu_node(rnp_old, flags);
return; /* Someone beat us to it. */
}
- WRITE_ONCE(rsp->gp_flags, READ_ONCE(rsp->gp_flags) | RCU_GP_FLAG_FQS);
+ WRITE_ONCE(rcu_state.gp_flags,
+ READ_ONCE(rcu_state.gp_flags) | RCU_GP_FLAG_FQS);
raw_spin_unlock_irqrestore_rcu_node(rnp_old, flags);
- rcu_gp_kthread_wake(rsp);
+ rcu_gp_kthread_wake();
}
/*
* RCU to come out of its idle mode.
*/
static void
-rcu_check_gp_start_stall(struct rcu_state *rsp, struct rcu_node *rnp,
- struct rcu_data *rdp)
+rcu_check_gp_start_stall(struct rcu_node *rnp, struct rcu_data *rdp)
{
const unsigned long gpssdelay = rcu_jiffies_till_stall_check() * HZ;
unsigned long flags;
unsigned long j;
- struct rcu_node *rnp_root = rcu_get_root(rsp);
+ struct rcu_node *rnp_root = rcu_get_root();
static atomic_t warned = ATOMIC_INIT(0);
- if (!IS_ENABLED(CONFIG_PROVE_RCU) || rcu_gp_in_progress(rsp) ||
+ if (!IS_ENABLED(CONFIG_PROVE_RCU) || rcu_gp_in_progress() ||
ULONG_CMP_GE(rnp_root->gp_seq, rnp_root->gp_seq_needed))
return;
j = jiffies; /* Expensive access, and in common case don't get here. */
- if (time_before(j, READ_ONCE(rsp->gp_req_activity) + gpssdelay) ||
- time_before(j, READ_ONCE(rsp->gp_activity) + gpssdelay) ||
+ if (time_before(j, READ_ONCE(rcu_state.gp_req_activity) + gpssdelay) ||
+ time_before(j, READ_ONCE(rcu_state.gp_activity) + gpssdelay) ||
atomic_read(&warned))
return;
raw_spin_lock_irqsave_rcu_node(rnp, flags);
j = jiffies;
- if (rcu_gp_in_progress(rsp) ||
+ if (rcu_gp_in_progress() ||
ULONG_CMP_GE(rnp_root->gp_seq, rnp_root->gp_seq_needed) ||
- time_before(j, READ_ONCE(rsp->gp_req_activity) + gpssdelay) ||
- time_before(j, READ_ONCE(rsp->gp_activity) + gpssdelay) ||
+ time_before(j, READ_ONCE(rcu_state.gp_req_activity) + gpssdelay) ||
+ time_before(j, READ_ONCE(rcu_state.gp_activity) + gpssdelay) ||
atomic_read(&warned)) {
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
return;
if (rnp_root != rnp)
raw_spin_lock_rcu_node(rnp_root); /* irqs already disabled. */
j = jiffies;
- if (rcu_gp_in_progress(rsp) ||
+ if (rcu_gp_in_progress() ||
ULONG_CMP_GE(rnp_root->gp_seq, rnp_root->gp_seq_needed) ||
- time_before(j, rsp->gp_req_activity + gpssdelay) ||
- time_before(j, rsp->gp_activity + gpssdelay) ||
+ time_before(j, rcu_state.gp_req_activity + gpssdelay) ||
+ time_before(j, rcu_state.gp_activity + gpssdelay) ||
atomic_xchg(&warned, 1)) {
raw_spin_unlock_rcu_node(rnp_root); /* irqs remain disabled. */
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
return;
}
pr_alert("%s: g%ld->%ld gar:%lu ga:%lu f%#x gs:%d %s->state:%#lx\n",
- __func__, (long)READ_ONCE(rsp->gp_seq),
+ __func__, (long)READ_ONCE(rcu_state.gp_seq),
(long)READ_ONCE(rnp_root->gp_seq_needed),
- j - rsp->gp_req_activity, j - rsp->gp_activity,
- rsp->gp_flags, rsp->gp_state, rsp->name,
- rsp->gp_kthread ? rsp->gp_kthread->state : 0x1ffffL);
+ j - rcu_state.gp_req_activity, j - rcu_state.gp_activity,
+ rcu_state.gp_flags, rcu_state.gp_state, rcu_state.name,
+ rcu_state.gp_kthread ? rcu_state.gp_kthread->state : 0x1ffffL);
WARN_ON(1);
if (rnp_root != rnp)
raw_spin_unlock_rcu_node(rnp_root);
}
/*
- * This does the RCU core processing work for the specified rcu_state
- * and rcu_data structures. This may be called only from the CPU to
- * whom the rdp belongs.
+ * This does the RCU core processing work for the specified rcu_data
+ * structures. This may be called only from the CPU to whom the rdp
+ * belongs.
*/
-static void
-__rcu_process_callbacks(struct rcu_state *rsp)
+static __latent_entropy void rcu_process_callbacks(struct softirq_action *unused)
{
unsigned long flags;
- struct rcu_data *rdp = raw_cpu_ptr(rsp->rda);
+ struct rcu_data *rdp = raw_cpu_ptr(&rcu_data);
struct rcu_node *rnp = rdp->mynode;
+ if (cpu_is_offline(smp_processor_id()))
+ return;
+ trace_rcu_utilization(TPS("Start RCU core"));
WARN_ON_ONCE(!rdp->beenonline);
+ /* Report any deferred quiescent states if preemption enabled. */
+ if (!(preempt_count() & PREEMPT_MASK)) {
+ rcu_preempt_deferred_qs(current);
+ } else if (rcu_preempt_need_deferred_qs(current)) {
+ set_tsk_need_resched(current);
+ set_preempt_need_resched();
+ }
+
/* Update RCU state based on any recent quiescent states. */
- rcu_check_quiescent_state(rsp, rdp);
+ rcu_check_quiescent_state(rdp);
/* No grace period and unregistered callbacks? */
- if (!rcu_gp_in_progress(rsp) &&
+ if (!rcu_gp_in_progress() &&
rcu_segcblist_is_enabled(&rdp->cblist)) {
local_irq_save(flags);
if (!rcu_segcblist_restempty(&rdp->cblist, RCU_NEXT_READY_TAIL))
- rcu_accelerate_cbs_unlocked(rsp, rnp, rdp);
+ rcu_accelerate_cbs_unlocked(rnp, rdp);
local_irq_restore(flags);
}
- rcu_check_gp_start_stall(rsp, rnp, rdp);
+ rcu_check_gp_start_stall(rnp, rdp);
/* If there are callbacks ready, invoke them. */
if (rcu_segcblist_ready_cbs(&rdp->cblist))
- invoke_rcu_callbacks(rsp, rdp);
+ invoke_rcu_callbacks(rdp);
/* Do any needed deferred wakeups of rcuo kthreads. */
do_nocb_deferred_wakeup(rdp);
-}
-
-/*
- * Do RCU core processing for the current CPU.
- */
-static __latent_entropy void rcu_process_callbacks(struct softirq_action *unused)
-{
- struct rcu_state *rsp;
-
- if (cpu_is_offline(smp_processor_id()))
- return;
- trace_rcu_utilization(TPS("Start RCU core"));
- for_each_rcu_flavor(rsp)
- __rcu_process_callbacks(rsp);
trace_rcu_utilization(TPS("End RCU core"));
}
/*
- * Schedule RCU callback invocation. If the specified type of RCU
- * does not support RCU priority boosting, just do a direct call,
- * otherwise wake up the per-CPU kernel kthread. Note that because we
- * are running on the current CPU with softirqs disabled, the
- * rcu_cpu_kthread_task cannot disappear out from under us.
+ * Schedule RCU callback invocation. If the running implementation of RCU
+ * does not support RCU priority boosting, just do a direct call, otherwise
+ * wake up the per-CPU kernel kthread. Note that because we are running
+ * on the current CPU with softirqs disabled, the rcu_cpu_kthread_task
+ * cannot disappear out from under us.
*/
-static void invoke_rcu_callbacks(struct rcu_state *rsp, struct rcu_data *rdp)
+static void invoke_rcu_callbacks(struct rcu_data *rdp)
{
if (unlikely(!READ_ONCE(rcu_scheduler_fully_active)))
return;
- if (likely(!rsp->boost)) {
- rcu_do_batch(rsp, rdp);
+ if (likely(!rcu_state.boost)) {
+ rcu_do_batch(rdp);
return;
}
invoke_rcu_callbacks_kthread();
/*
* Handle any core-RCU processing required by a call_rcu() invocation.
*/
-static void __call_rcu_core(struct rcu_state *rsp, struct rcu_data *rdp,
- struct rcu_head *head, unsigned long flags)
+static void __call_rcu_core(struct rcu_data *rdp, struct rcu_head *head,
+ unsigned long flags)
{
/*
* If called from an extended quiescent state, invoke the RCU
rdp->qlen_last_fqs_check + qhimark)) {
/* Are we ignoring a completed grace period? */
- note_gp_changes(rsp, rdp);
+ note_gp_changes(rdp);
/* Start a new grace period if one not already started. */
- if (!rcu_gp_in_progress(rsp)) {
- rcu_accelerate_cbs_unlocked(rsp, rdp->mynode, rdp);
+ if (!rcu_gp_in_progress()) {
+ rcu_accelerate_cbs_unlocked(rdp->mynode, rdp);
} else {
/* Give the grace period a kick. */
rdp->blimit = LONG_MAX;
- if (rsp->n_force_qs == rdp->n_force_qs_snap &&
+ if (rcu_state.n_force_qs == rdp->n_force_qs_snap &&
rcu_segcblist_first_pend_cb(&rdp->cblist) != head)
- force_quiescent_state(rsp);
- rdp->n_force_qs_snap = rsp->n_force_qs;
+ force_quiescent_state();
+ rdp->n_force_qs_snap = rcu_state.n_force_qs;
rdp->qlen_last_fqs_check = rcu_segcblist_n_cbs(&rdp->cblist);
}
}
/*
* Helper function for call_rcu() and friends. The cpu argument will
* normally be -1, indicating "currently running CPU". It may specify
- * a CPU only if that CPU is a no-CBs CPU. Currently, only _rcu_barrier()
+ * a CPU only if that CPU is a no-CBs CPU. Currently, only rcu_barrier()
* is expected to specify a CPU.
*/
static void
-__call_rcu(struct rcu_head *head, rcu_callback_t func,
- struct rcu_state *rsp, int cpu, bool lazy)
+__call_rcu(struct rcu_head *head, rcu_callback_t func, int cpu, bool lazy)
{
unsigned long flags;
struct rcu_data *rdp;
head->func = func;
head->next = NULL;
local_irq_save(flags);
- rdp = this_cpu_ptr(rsp->rda);
+ rdp = this_cpu_ptr(&rcu_data);
/* Add the callback to our list. */
if (unlikely(!rcu_segcblist_is_enabled(&rdp->cblist)) || cpu != -1) {
int offline;
if (cpu != -1)
- rdp = per_cpu_ptr(rsp->rda, cpu);
+ rdp = per_cpu_ptr(&rcu_data, cpu);
if (likely(rdp->mynode)) {
/* Post-boot, so this should be for a no-CBs CPU. */
offline = !__call_rcu_nocb(rdp, head, lazy, flags);
rcu_idle_count_callbacks_posted();
if (__is_kfree_rcu_offset((unsigned long)func))
- trace_rcu_kfree_callback(rsp->name, head, (unsigned long)func,
+ trace_rcu_kfree_callback(rcu_state.name, head,
+ (unsigned long)func,
rcu_segcblist_n_lazy_cbs(&rdp->cblist),
rcu_segcblist_n_cbs(&rdp->cblist));
else
- trace_rcu_callback(rsp->name, head,
+ trace_rcu_callback(rcu_state.name, head,
rcu_segcblist_n_lazy_cbs(&rdp->cblist),
rcu_segcblist_n_cbs(&rdp->cblist));
/* Go handle any RCU core processing required. */
- __call_rcu_core(rsp, rdp, head, flags);
+ __call_rcu_core(rdp, head, flags);
local_irq_restore(flags);
}
/**
- * call_rcu_sched() - Queue an RCU for invocation after sched grace period.
- * @head: structure to be used for queueing the RCU updates.
- * @func: actual callback function to be invoked after the grace period
- *
- * The callback function will be invoked some time after a full grace
- * period elapses, in other words after all currently executing RCU
- * read-side critical sections have completed. call_rcu_sched() assumes
- * that the read-side critical sections end on enabling of preemption
- * or on voluntary preemption.
- * RCU read-side critical sections are delimited by:
- *
- * - rcu_read_lock_sched() and rcu_read_unlock_sched(), OR
- * - anything that disables preemption.
- *
- * These may be nested.
- *
- * See the description of call_rcu() for more detailed information on
- * memory ordering guarantees.
- */
-void call_rcu_sched(struct rcu_head *head, rcu_callback_t func)
-{
- __call_rcu(head, func, &rcu_sched_state, -1, 0);
-}
-EXPORT_SYMBOL_GPL(call_rcu_sched);
-
-/**
- * call_rcu_bh() - Queue an RCU for invocation after a quicker grace period.
+ * call_rcu() - Queue an RCU callback for invocation after a grace period.
* @head: structure to be used for queueing the RCU updates.
* @func: actual callback function to be invoked after the grace period
*
* The callback function will be invoked some time after a full grace
- * period elapses, in other words after all currently executing RCU
- * read-side critical sections have completed. call_rcu_bh() assumes
- * that the read-side critical sections end on completion of a softirq
- * handler. This means that read-side critical sections in process
- * context must not be interrupted by softirqs. This interface is to be
- * used when most of the read-side critical sections are in softirq context.
- * RCU read-side critical sections are delimited by:
- *
- * - rcu_read_lock() and rcu_read_unlock(), if in interrupt context, OR
- * - rcu_read_lock_bh() and rcu_read_unlock_bh(), if in process context.
- *
- * These may be nested.
- *
- * See the description of call_rcu() for more detailed information on
- * memory ordering guarantees.
- */
-void call_rcu_bh(struct rcu_head *head, rcu_callback_t func)
-{
- __call_rcu(head, func, &rcu_bh_state, -1, 0);
-}
-EXPORT_SYMBOL_GPL(call_rcu_bh);
+ * period elapses, in other words after all pre-existing RCU read-side
+ * critical sections have completed. However, the callback function
+ * might well execute concurrently with RCU read-side critical sections
+ * that started after call_rcu() was invoked. RCU read-side critical
+ * sections are delimited by rcu_read_lock() and rcu_read_unlock(), and
+ * may be nested. In addition, regions of code across which interrupts,
+ * preemption, or softirqs have been disabled also serve as RCU read-side
+ * critical sections. This includes hardware interrupt handlers, softirq
+ * handlers, and NMI handlers.
+ *
+ * Note that all CPUs must agree that the grace period extended beyond
+ * all pre-existing RCU read-side critical section. On systems with more
+ * than one CPU, this means that when "func()" is invoked, each CPU is
+ * guaranteed to have executed a full memory barrier since the end of its
+ * last RCU read-side critical section whose beginning preceded the call
+ * to call_rcu(). It also means that each CPU executing an RCU read-side
+ * critical section that continues beyond the start of "func()" must have
+ * executed a memory barrier after the call_rcu() but before the beginning
+ * of that RCU read-side critical section. Note that these guarantees
+ * include CPUs that are offline, idle, or executing in user mode, as
+ * well as CPUs that are executing in the kernel.
+ *
+ * Furthermore, if CPU A invoked call_rcu() and CPU B invoked the
+ * resulting RCU callback function "func()", then both CPU A and CPU B are
+ * guaranteed to execute a full memory barrier during the time interval
+ * between the call to call_rcu() and the invocation of "func()" -- even
+ * if CPU A and CPU B are the same CPU (but again only if the system has
+ * more than one CPU).
+ */
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
+{
+ __call_rcu(head, func, -1, 0);
+}
+EXPORT_SYMBOL_GPL(call_rcu);
/*
* Queue an RCU callback for lazy invocation after a grace period.
* callbacks in the list of pending callbacks. Until then, this
* function may only be called from __kfree_rcu().
*/
-void kfree_call_rcu(struct rcu_head *head,
- rcu_callback_t func)
+void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func)
{
- __call_rcu(head, func, rcu_state_p, -1, 1);
+ __call_rcu(head, func, -1, 1);
}
EXPORT_SYMBOL_GPL(kfree_call_rcu);
-/*
- * Because a context switch is a grace period for RCU-sched and RCU-bh,
- * any blocking grace-period wait automatically implies a grace period
- * if there is only one CPU online at any point time during execution
- * of either synchronize_sched() or synchronize_rcu_bh(). It is OK to
- * occasionally incorrectly indicate that there are multiple CPUs online
- * when there was in fact only one the whole time, as this just adds
- * some overhead: RCU still operates correctly.
- */
-static int rcu_blocking_is_gp(void)
-{
- int ret;
-
- might_sleep(); /* Check for RCU read-side critical section. */
- preempt_disable();
- ret = num_online_cpus() <= 1;
- preempt_enable();
- return ret;
-}
-
-/**
- * synchronize_sched - wait until an rcu-sched grace period has elapsed.
- *
- * Control will return to the caller some time after a full rcu-sched
- * grace period has elapsed, in other words after all currently executing
- * rcu-sched read-side critical sections have completed. These read-side
- * critical sections are delimited by rcu_read_lock_sched() and
- * rcu_read_unlock_sched(), and may be nested. Note that preempt_disable(),
- * local_irq_disable(), and so on may be used in place of
- * rcu_read_lock_sched().
- *
- * This means that all preempt_disable code sequences, including NMI and
- * non-threaded hardware-interrupt handlers, in progress on entry will
- * have completed before this primitive returns. However, this does not
- * guarantee that softirq handlers will have completed, since in some
- * kernels, these handlers can run in process context, and can block.
- *
- * Note that this guarantee implies further memory-ordering guarantees.
- * On systems with more than one CPU, when synchronize_sched() returns,
- * each CPU is guaranteed to have executed a full memory barrier since the
- * end of its last RCU-sched read-side critical section whose beginning
- * preceded the call to synchronize_sched(). In addition, each CPU having
- * an RCU read-side critical section that extends beyond the return from
- * synchronize_sched() is guaranteed to have executed a full memory barrier
- * after the beginning of synchronize_sched() and before the beginning of
- * that RCU read-side critical section. Note that these guarantees include
- * CPUs that are offline, idle, or executing in user mode, as well as CPUs
- * that are executing in the kernel.
- *
- * Furthermore, if CPU A invoked synchronize_sched(), which returned
- * to its caller on CPU B, then both CPU A and CPU B are guaranteed
- * to have executed a full memory barrier during the execution of
- * synchronize_sched() -- even if CPU A and CPU B are the same CPU (but
- * again only if the system has more than one CPU).
- */
-void synchronize_sched(void)
-{
- RCU_LOCKDEP_WARN(lock_is_held(&rcu_bh_lock_map) ||
- lock_is_held(&rcu_lock_map) ||
- lock_is_held(&rcu_sched_lock_map),
- "Illegal synchronize_sched() in RCU-sched read-side critical section");
- if (rcu_blocking_is_gp())
- return;
- if (rcu_gp_is_expedited())
- synchronize_sched_expedited();
- else
- wait_rcu_gp(call_rcu_sched);
-}
-EXPORT_SYMBOL_GPL(synchronize_sched);
-
-/**
- * synchronize_rcu_bh - wait until an rcu_bh grace period has elapsed.
- *
- * Control will return to the caller some time after a full rcu_bh grace
- * period has elapsed, in other words after all currently executing rcu_bh
- * read-side critical sections have completed. RCU read-side critical
- * sections are delimited by rcu_read_lock_bh() and rcu_read_unlock_bh(),
- * and may be nested.
- *
- * See the description of synchronize_sched() for more detailed information
- * on memory ordering guarantees.
- */
-void synchronize_rcu_bh(void)
-{
- RCU_LOCKDEP_WARN(lock_is_held(&rcu_bh_lock_map) ||
- lock_is_held(&rcu_lock_map) ||
- lock_is_held(&rcu_sched_lock_map),
- "Illegal synchronize_rcu_bh() in RCU-bh read-side critical section");
- if (rcu_blocking_is_gp())
- return;
- if (rcu_gp_is_expedited())
- synchronize_rcu_bh_expedited();
- else
- wait_rcu_gp(call_rcu_bh);
-}
-EXPORT_SYMBOL_GPL(synchronize_rcu_bh);
-
/**
* get_state_synchronize_rcu - Snapshot current RCU state
*
* before the load from ->gp_seq.
*/
smp_mb(); /* ^^^ */
- return rcu_seq_snap(&rcu_state_p->gp_seq);
+ return rcu_seq_snap(&rcu_state.gp_seq);
}
EXPORT_SYMBOL_GPL(get_state_synchronize_rcu);
*/
void cond_synchronize_rcu(unsigned long oldstate)
{
- if (!rcu_seq_done(&rcu_state_p->gp_seq, oldstate))
+ if (!rcu_seq_done(&rcu_state.gp_seq, oldstate))
synchronize_rcu();
else
smp_mb(); /* Ensure GP ends before subsequent accesses. */
}
EXPORT_SYMBOL_GPL(cond_synchronize_rcu);
-/**
- * get_state_synchronize_sched - Snapshot current RCU-sched state
- *
- * Returns a cookie that is used by a later call to cond_synchronize_sched()
- * to determine whether or not a full grace period has elapsed in the
- * meantime.
- */
-unsigned long get_state_synchronize_sched(void)
-{
- /*
- * Any prior manipulation of RCU-protected data must happen
- * before the load from ->gp_seq.
- */
- smp_mb(); /* ^^^ */
- return rcu_seq_snap(&rcu_sched_state.gp_seq);
-}
-EXPORT_SYMBOL_GPL(get_state_synchronize_sched);
-
-/**
- * cond_synchronize_sched - Conditionally wait for an RCU-sched grace period
- *
- * @oldstate: return value from earlier call to get_state_synchronize_sched()
- *
- * If a full RCU-sched grace period has elapsed since the earlier call to
- * get_state_synchronize_sched(), just return. Otherwise, invoke
- * synchronize_sched() to wait for a full grace period.
- *
- * Yes, this function does not take counter wrap into account. But
- * counter wrap is harmless. If the counter wraps, we have waited for
- * more than 2 billion grace periods (and way more on a 64-bit system!),
- * so waiting for one additional grace period should be just fine.
- */
-void cond_synchronize_sched(unsigned long oldstate)
-{
- if (!rcu_seq_done(&rcu_sched_state.gp_seq, oldstate))
- synchronize_sched();
- else
- smp_mb(); /* Ensure GP ends before subsequent accesses. */
-}
-EXPORT_SYMBOL_GPL(cond_synchronize_sched);
-
/*
- * Check to see if there is any immediate RCU-related work to be done
- * by the current CPU, for the specified type of RCU, returning 1 if so.
- * The checks are in order of increasing expense: checks that can be
- * carried out against CPU-local state are performed first. However,
- * we must check for CPU stalls first, else we might not get a chance.
+ * Check to see if there is any immediate RCU-related work to be done by
+ * the current CPU, returning 1 if so and zero otherwise. The checks are
+ * in order of increasing expense: checks that can be carried out against
+ * CPU-local state are performed first. However, we must check for CPU
+ * stalls first, else we might not get a chance.
*/
-static int __rcu_pending(struct rcu_state *rsp, struct rcu_data *rdp)
+static int rcu_pending(void)
{
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
struct rcu_node *rnp = rdp->mynode;
/* Check for CPU stalls, if enabled. */
- check_cpu_stall(rsp, rdp);
+ check_cpu_stall(rdp);
/* Is this CPU a NO_HZ_FULL CPU that should ignore RCU? */
- if (rcu_nohz_full_cpu(rsp))
+ if (rcu_nohz_full_cpu())
return 0;
/* Is the RCU core waiting for a quiescent state from this CPU? */
return 1;
/* Has RCU gone idle with this CPU needing another grace period? */
- if (!rcu_gp_in_progress(rsp) &&
+ if (!rcu_gp_in_progress() &&
rcu_segcblist_is_enabled(&rdp->cblist) &&
!rcu_segcblist_restempty(&rdp->cblist, RCU_NEXT_READY_TAIL))
return 1;
return 0;
}
-/*
- * Check to see if there is any immediate RCU-related work to be done
- * by the current CPU, returning 1 if so. This function is part of the
- * RCU implementation; it is -not- an exported member of the RCU API.
- */
-static int rcu_pending(void)
-{
- struct rcu_state *rsp;
-
- for_each_rcu_flavor(rsp)
- if (__rcu_pending(rsp, this_cpu_ptr(rsp->rda)))
- return 1;
- return 0;
-}
-
/*
* Return true if the specified CPU has any callback. If all_lazy is
* non-NULL, store an indication of whether all callbacks are lazy.
bool al = true;
bool hc = false;
struct rcu_data *rdp;
- struct rcu_state *rsp;
- for_each_rcu_flavor(rsp) {
- rdp = this_cpu_ptr(rsp->rda);
- if (rcu_segcblist_empty(&rdp->cblist))
- continue;
+ rdp = this_cpu_ptr(&rcu_data);
+ if (!rcu_segcblist_empty(&rdp->cblist)) {
hc = true;
- if (rcu_segcblist_n_nonlazy_cbs(&rdp->cblist) || !all_lazy) {
+ if (rcu_segcblist_n_nonlazy_cbs(&rdp->cblist))
al = false;
- break;
- }
}
if (all_lazy)
*all_lazy = al;
}
/*
- * Helper function for _rcu_barrier() tracing. If tracing is disabled,
+ * Helper function for rcu_barrier() tracing. If tracing is disabled,
* the compiler is expected to optimize this away.
*/
-static void _rcu_barrier_trace(struct rcu_state *rsp, const char *s,
- int cpu, unsigned long done)
+static void rcu_barrier_trace(const char *s, int cpu, unsigned long done)
{
- trace_rcu_barrier(rsp->name, s, cpu,
- atomic_read(&rsp->barrier_cpu_count), done);
+ trace_rcu_barrier(rcu_state.name, s, cpu,
+ atomic_read(&rcu_state.barrier_cpu_count), done);
}
/*
- * RCU callback function for _rcu_barrier(). If we are last, wake
- * up the task executing _rcu_barrier().
+ * RCU callback function for rcu_barrier(). If we are last, wake
+ * up the task executing rcu_barrier().
*/
static void rcu_barrier_callback(struct rcu_head *rhp)
{
- struct rcu_data *rdp = container_of(rhp, struct rcu_data, barrier_head);
- struct rcu_state *rsp = rdp->rsp;
-
- if (atomic_dec_and_test(&rsp->barrier_cpu_count)) {
- _rcu_barrier_trace(rsp, TPS("LastCB"), -1,
- rsp->barrier_sequence);
- complete(&rsp->barrier_completion);
+ if (atomic_dec_and_test(&rcu_state.barrier_cpu_count)) {
+ rcu_barrier_trace(TPS("LastCB"), -1,
+ rcu_state.barrier_sequence);
+ complete(&rcu_state.barrier_completion);
} else {
- _rcu_barrier_trace(rsp, TPS("CB"), -1, rsp->barrier_sequence);
+ rcu_barrier_trace(TPS("CB"), -1, rcu_state.barrier_sequence);
}
}
/*
* Called with preemption disabled, and from cross-cpu IRQ context.
*/
-static void rcu_barrier_func(void *type)
+static void rcu_barrier_func(void *unused)
{
- struct rcu_state *rsp = type;
- struct rcu_data *rdp = raw_cpu_ptr(rsp->rda);
+ struct rcu_data *rdp = raw_cpu_ptr(&rcu_data);
- _rcu_barrier_trace(rsp, TPS("IRQ"), -1, rsp->barrier_sequence);
+ rcu_barrier_trace(TPS("IRQ"), -1, rcu_state.barrier_sequence);
rdp->barrier_head.func = rcu_barrier_callback;
debug_rcu_head_queue(&rdp->barrier_head);
if (rcu_segcblist_entrain(&rdp->cblist, &rdp->barrier_head, 0)) {
- atomic_inc(&rsp->barrier_cpu_count);
+ atomic_inc(&rcu_state.barrier_cpu_count);
} else {
debug_rcu_head_unqueue(&rdp->barrier_head);
- _rcu_barrier_trace(rsp, TPS("IRQNQ"), -1,
- rsp->barrier_sequence);
+ rcu_barrier_trace(TPS("IRQNQ"), -1,
+ rcu_state.barrier_sequence);
}
}
-/*
- * Orchestrate the specified type of RCU barrier, waiting for all
- * RCU callbacks of the specified type to complete.
+/**
+ * rcu_barrier - Wait until all in-flight call_rcu() callbacks complete.
+ *
+ * Note that this primitive does not necessarily wait for an RCU grace period
+ * to complete. For example, if there are no RCU callbacks queued anywhere
+ * in the system, then rcu_barrier() is within its rights to return
+ * immediately, without waiting for anything, much less an RCU grace period.
*/
-static void _rcu_barrier(struct rcu_state *rsp)
+void rcu_barrier(void)
{
int cpu;
struct rcu_data *rdp;
- unsigned long s = rcu_seq_snap(&rsp->barrier_sequence);
+ unsigned long s = rcu_seq_snap(&rcu_state.barrier_sequence);
- _rcu_barrier_trace(rsp, TPS("Begin"), -1, s);
+ rcu_barrier_trace(TPS("Begin"), -1, s);
/* Take mutex to serialize concurrent rcu_barrier() requests. */
- mutex_lock(&rsp->barrier_mutex);
+ mutex_lock(&rcu_state.barrier_mutex);
/* Did someone else do our work for us? */
- if (rcu_seq_done(&rsp->barrier_sequence, s)) {
- _rcu_barrier_trace(rsp, TPS("EarlyExit"), -1,
- rsp->barrier_sequence);
+ if (rcu_seq_done(&rcu_state.barrier_sequence, s)) {
+ rcu_barrier_trace(TPS("EarlyExit"), -1,
+ rcu_state.barrier_sequence);
smp_mb(); /* caller's subsequent code after above check. */
- mutex_unlock(&rsp->barrier_mutex);
+ mutex_unlock(&rcu_state.barrier_mutex);
return;
}
/* Mark the start of the barrier operation. */
- rcu_seq_start(&rsp->barrier_sequence);
- _rcu_barrier_trace(rsp, TPS("Inc1"), -1, rsp->barrier_sequence);
+ rcu_seq_start(&rcu_state.barrier_sequence);
+ rcu_barrier_trace(TPS("Inc1"), -1, rcu_state.barrier_sequence);
/*
* Initialize the count to one rather than to zero in order to
* (or preemption of this task). Exclude CPU-hotplug operations
* to ensure that no offline CPU has callbacks queued.
*/
- init_completion(&rsp->barrier_completion);
- atomic_set(&rsp->barrier_cpu_count, 1);
+ init_completion(&rcu_state.barrier_completion);
+ atomic_set(&rcu_state.barrier_cpu_count, 1);
get_online_cpus();
/*
for_each_possible_cpu(cpu) {
if (!cpu_online(cpu) && !rcu_is_nocb_cpu(cpu))
continue;
- rdp = per_cpu_ptr(rsp->rda, cpu);
+ rdp = per_cpu_ptr(&rcu_data, cpu);
if (rcu_is_nocb_cpu(cpu)) {
- if (!rcu_nocb_cpu_needs_barrier(rsp, cpu)) {
- _rcu_barrier_trace(rsp, TPS("OfflineNoCB"), cpu,
- rsp->barrier_sequence);
+ if (!rcu_nocb_cpu_needs_barrier(cpu)) {
+ rcu_barrier_trace(TPS("OfflineNoCB"), cpu,
+ rcu_state.barrier_sequence);
} else {
- _rcu_barrier_trace(rsp, TPS("OnlineNoCB"), cpu,
- rsp->barrier_sequence);
+ rcu_barrier_trace(TPS("OnlineNoCB"), cpu,
+ rcu_state.barrier_sequence);
smp_mb__before_atomic();
- atomic_inc(&rsp->barrier_cpu_count);
+ atomic_inc(&rcu_state.barrier_cpu_count);
__call_rcu(&rdp->barrier_head,
- rcu_barrier_callback, rsp, cpu, 0);
+ rcu_barrier_callback, cpu, 0);
}
} else if (rcu_segcblist_n_cbs(&rdp->cblist)) {
- _rcu_barrier_trace(rsp, TPS("OnlineQ"), cpu,
- rsp->barrier_sequence);
- smp_call_function_single(cpu, rcu_barrier_func, rsp, 1);
+ rcu_barrier_trace(TPS("OnlineQ"), cpu,
+ rcu_state.barrier_sequence);
+ smp_call_function_single(cpu, rcu_barrier_func, NULL, 1);
} else {
- _rcu_barrier_trace(rsp, TPS("OnlineNQ"), cpu,
- rsp->barrier_sequence);
+ rcu_barrier_trace(TPS("OnlineNQ"), cpu,
+ rcu_state.barrier_sequence);
}
}
put_online_cpus();
* Now that we have an rcu_barrier_callback() callback on each
* CPU, and thus each counted, remove the initial count.
*/
- if (atomic_dec_and_test(&rsp->barrier_cpu_count))
- complete(&rsp->barrier_completion);
+ if (atomic_dec_and_test(&rcu_state.barrier_cpu_count))
+ complete(&rcu_state.barrier_completion);
/* Wait for all rcu_barrier_callback() callbacks to be invoked. */
- wait_for_completion(&rsp->barrier_completion);
+ wait_for_completion(&rcu_state.barrier_completion);
/* Mark the end of the barrier operation. */
- _rcu_barrier_trace(rsp, TPS("Inc2"), -1, rsp->barrier_sequence);
- rcu_seq_end(&rsp->barrier_sequence);
+ rcu_barrier_trace(TPS("Inc2"), -1, rcu_state.barrier_sequence);
+ rcu_seq_end(&rcu_state.barrier_sequence);
/* Other rcu_barrier() invocations can now safely proceed. */
- mutex_unlock(&rsp->barrier_mutex);
-}
-
-/**
- * rcu_barrier_bh - Wait until all in-flight call_rcu_bh() callbacks complete.
- */
-void rcu_barrier_bh(void)
-{
- _rcu_barrier(&rcu_bh_state);
-}
-EXPORT_SYMBOL_GPL(rcu_barrier_bh);
-
-/**
- * rcu_barrier_sched - Wait for in-flight call_rcu_sched() callbacks.
- */
-void rcu_barrier_sched(void)
-{
- _rcu_barrier(&rcu_sched_state);
+ mutex_unlock(&rcu_state.barrier_mutex);
}
-EXPORT_SYMBOL_GPL(rcu_barrier_sched);
+EXPORT_SYMBOL_GPL(rcu_barrier);
/*
* Propagate ->qsinitmask bits up the rcu_node tree to account for the
* Do boot-time initialization of a CPU's per-CPU RCU data.
*/
static void __init
-rcu_boot_init_percpu_data(int cpu, struct rcu_state *rsp)
+rcu_boot_init_percpu_data(int cpu)
{
- struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
+ struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
/* Set up local state, ensuring consistent view of global state. */
rdp->grpmask = leaf_node_cpu_bit(rdp->mynode, cpu);
- rdp->dynticks = &per_cpu(rcu_dynticks, cpu);
- WARN_ON_ONCE(rdp->dynticks->dynticks_nesting != 1);
- WARN_ON_ONCE(rcu_dynticks_in_eqs(rcu_dynticks_snap(rdp->dynticks)));
- rdp->rcu_ofl_gp_seq = rsp->gp_seq;
+ WARN_ON_ONCE(rdp->dynticks_nesting != 1);
+ WARN_ON_ONCE(rcu_dynticks_in_eqs(rcu_dynticks_snap(rdp)));
+ rdp->rcu_ofl_gp_seq = rcu_state.gp_seq;
rdp->rcu_ofl_gp_flags = RCU_GP_CLEANED;
- rdp->rcu_onl_gp_seq = rsp->gp_seq;
+ rdp->rcu_onl_gp_seq = rcu_state.gp_seq;
rdp->rcu_onl_gp_flags = RCU_GP_CLEANED;
rdp->cpu = cpu;
- rdp->rsp = rsp;
rcu_boot_init_nocb_percpu_data(rdp);
}
/*
- * Initialize a CPU's per-CPU RCU data. Note that only one online or
+ * Invoked early in the CPU-online process, when pretty much all services
+ * are available. The incoming CPU is not present.
+ *
+ * Initializes a CPU's per-CPU RCU data. Note that only one online or
* offline event can be happening at a given time. Note also that we can
* accept some slop in the rsp->gp_seq access due to the fact that this
* CPU cannot possibly have any RCU callbacks in flight yet.
*/
-static void
-rcu_init_percpu_data(int cpu, struct rcu_state *rsp)
+int rcutree_prepare_cpu(unsigned int cpu)
{
unsigned long flags;
- struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
- struct rcu_node *rnp = rcu_get_root(rsp);
+ struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
+ struct rcu_node *rnp = rcu_get_root();
/* Set up local state, ensuring consistent view of global state. */
raw_spin_lock_irqsave_rcu_node(rnp, flags);
rdp->qlen_last_fqs_check = 0;
- rdp->n_force_qs_snap = rsp->n_force_qs;
+ rdp->n_force_qs_snap = rcu_state.n_force_qs;
rdp->blimit = blimit;
if (rcu_segcblist_empty(&rdp->cblist) && /* No early-boot CBs? */
!init_nocb_callback_list(rdp))
rcu_segcblist_init(&rdp->cblist); /* Re-enable callbacks. */
- rdp->dynticks->dynticks_nesting = 1; /* CPU not up, no tearing. */
+ rdp->dynticks_nesting = 1; /* CPU not up, no tearing. */
rcu_dynticks_eqs_online();
raw_spin_unlock_rcu_node(rnp); /* irqs remain disabled. */
rdp->gp_seq = rnp->gp_seq;
rdp->gp_seq_needed = rnp->gp_seq;
rdp->cpu_no_qs.b.norm = true;
- rdp->rcu_qs_ctr_snap = per_cpu(rcu_dynticks.rcu_qs_ctr, cpu);
rdp->core_needs_qs = false;
rdp->rcu_iw_pending = false;
rdp->rcu_iw_gp_seq = rnp->gp_seq - 1;
- trace_rcu_grace_period(rsp->name, rdp->gp_seq, TPS("cpuonl"));
+ trace_rcu_grace_period(rcu_state.name, rdp->gp_seq, TPS("cpuonl"));
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
-}
-
-/*
- * Invoked early in the CPU-online process, when pretty much all
- * services are available. The incoming CPU is not present.
- */
-int rcutree_prepare_cpu(unsigned int cpu)
-{
- struct rcu_state *rsp;
-
- for_each_rcu_flavor(rsp)
- rcu_init_percpu_data(cpu, rsp);
-
rcu_prepare_kthreads(cpu);
rcu_spawn_all_nocb_kthreads(cpu);
*/
static void rcutree_affinity_setting(unsigned int cpu, int outgoing)
{
- struct rcu_data *rdp = per_cpu_ptr(rcu_state_p->rda, cpu);
+ struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
rcu_boost_kthread_setaffinity(rdp->mynode, outgoing);
}
unsigned long flags;
struct rcu_data *rdp;
struct rcu_node *rnp;
- struct rcu_state *rsp;
- for_each_rcu_flavor(rsp) {
- rdp = per_cpu_ptr(rsp->rda, cpu);
- rnp = rdp->mynode;
- raw_spin_lock_irqsave_rcu_node(rnp, flags);
- rnp->ffmask |= rdp->grpmask;
- raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
- }
+ rdp = per_cpu_ptr(&rcu_data, cpu);
+ rnp = rdp->mynode;
+ raw_spin_lock_irqsave_rcu_node(rnp, flags);
+ rnp->ffmask |= rdp->grpmask;
+ raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
if (IS_ENABLED(CONFIG_TREE_SRCU))
srcu_online_cpu(cpu);
if (rcu_scheduler_active == RCU_SCHEDULER_INACTIVE)
unsigned long flags;
struct rcu_data *rdp;
struct rcu_node *rnp;
- struct rcu_state *rsp;
- for_each_rcu_flavor(rsp) {
- rdp = per_cpu_ptr(rsp->rda, cpu);
- rnp = rdp->mynode;
- raw_spin_lock_irqsave_rcu_node(rnp, flags);
- rnp->ffmask &= ~rdp->grpmask;
- raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
- }
+ rdp = per_cpu_ptr(&rcu_data, cpu);
+ rnp = rdp->mynode;
+ raw_spin_lock_irqsave_rcu_node(rnp, flags);
+ rnp->ffmask &= ~rdp->grpmask;
+ raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
rcutree_affinity_setting(cpu, cpu);
if (IS_ENABLED(CONFIG_TREE_SRCU))
return 0;
}
-/*
- * Near the end of the offline process. We do only tracing here.
- */
-int rcutree_dying_cpu(unsigned int cpu)
-{
- struct rcu_state *rsp;
-
- for_each_rcu_flavor(rsp)
- rcu_cleanup_dying_cpu(rsp);
- return 0;
-}
-
-/*
- * The outgoing CPU is gone and we are running elsewhere.
- */
-int rcutree_dead_cpu(unsigned int cpu)
-{
- struct rcu_state *rsp;
-
- for_each_rcu_flavor(rsp) {
- rcu_cleanup_dead_cpu(cpu, rsp);
- do_nocb_deferred_wakeup(per_cpu_ptr(rsp->rda, cpu));
- }
- return 0;
-}
-
static DEFINE_PER_CPU(int, rcu_cpu_started);
/*
unsigned long oldmask;
struct rcu_data *rdp;
struct rcu_node *rnp;
- struct rcu_state *rsp;
if (per_cpu(rcu_cpu_started, cpu))
return;
per_cpu(rcu_cpu_started, cpu) = 1;
- for_each_rcu_flavor(rsp) {
- rdp = per_cpu_ptr(rsp->rda, cpu);
- rnp = rdp->mynode;
- mask = rdp->grpmask;
- raw_spin_lock_irqsave_rcu_node(rnp, flags);
- rnp->qsmaskinitnext |= mask;
- oldmask = rnp->expmaskinitnext;
- rnp->expmaskinitnext |= mask;
- oldmask ^= rnp->expmaskinitnext;
- nbits = bitmap_weight(&oldmask, BITS_PER_LONG);
- /* Allow lockless access for expedited grace periods. */
- smp_store_release(&rsp->ncpus, rsp->ncpus + nbits); /* ^^^ */
- rcu_gpnum_ovf(rnp, rdp); /* Offline-induced counter wrap? */
- rdp->rcu_onl_gp_seq = READ_ONCE(rsp->gp_seq);
- rdp->rcu_onl_gp_flags = READ_ONCE(rsp->gp_flags);
- if (rnp->qsmask & mask) { /* RCU waiting on incoming CPU? */
- /* Report QS -after- changing ->qsmaskinitnext! */
- rcu_report_qs_rnp(mask, rsp, rnp, rnp->gp_seq, flags);
- } else {
- raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
- }
+ rdp = per_cpu_ptr(&rcu_data, cpu);
+ rnp = rdp->mynode;
+ mask = rdp->grpmask;
+ raw_spin_lock_irqsave_rcu_node(rnp, flags);
+ rnp->qsmaskinitnext |= mask;
+ oldmask = rnp->expmaskinitnext;
+ rnp->expmaskinitnext |= mask;
+ oldmask ^= rnp->expmaskinitnext;
+ nbits = bitmap_weight(&oldmask, BITS_PER_LONG);
+ /* Allow lockless access for expedited grace periods. */
+ smp_store_release(&rcu_state.ncpus, rcu_state.ncpus + nbits); /* ^^^ */
+ rcu_gpnum_ovf(rnp, rdp); /* Offline-induced counter wrap? */
+ rdp->rcu_onl_gp_seq = READ_ONCE(rcu_state.gp_seq);
+ rdp->rcu_onl_gp_flags = READ_ONCE(rcu_state.gp_flags);
+ if (rnp->qsmask & mask) { /* RCU waiting on incoming CPU? */
+ /* Report QS -after- changing ->qsmaskinitnext! */
+ rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags);
+ } else {
+ raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
}
smp_mb(); /* Ensure RCU read-side usage follows above initialization. */
}
#ifdef CONFIG_HOTPLUG_CPU
/*
- * The CPU is exiting the idle loop into the arch_cpu_idle_dead()
- * function. We now remove it from the rcu_node tree's ->qsmaskinitnext
- * bit masks.
+ * The outgoing function has no further need of RCU, so remove it from
+ * the rcu_node tree's ->qsmaskinitnext bit masks.
+ *
+ * Note that this function is special in that it is invoked directly
+ * from the outgoing CPU rather than from the cpuhp_step mechanism.
+ * This is because this function must be invoked at a precise location.
*/
-static void rcu_cleanup_dying_idle_cpu(int cpu, struct rcu_state *rsp)
+void rcu_report_dead(unsigned int cpu)
{
unsigned long flags;
unsigned long mask;
- struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
+ struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */
+ /* QS for any half-done expedited grace period. */
+ preempt_disable();
+ rcu_report_exp_rdp(this_cpu_ptr(&rcu_data));
+ preempt_enable();
+ rcu_preempt_deferred_qs(current);
+
/* Remove outgoing CPU from mask in the leaf rcu_node structure. */
mask = rdp->grpmask;
- spin_lock(&rsp->ofl_lock);
+ raw_spin_lock(&rcu_state.ofl_lock);
raw_spin_lock_irqsave_rcu_node(rnp, flags); /* Enforce GP memory-order guarantee. */
- rdp->rcu_ofl_gp_seq = READ_ONCE(rsp->gp_seq);
- rdp->rcu_ofl_gp_flags = READ_ONCE(rsp->gp_flags);
+ rdp->rcu_ofl_gp_seq = READ_ONCE(rcu_state.gp_seq);
+ rdp->rcu_ofl_gp_flags = READ_ONCE(rcu_state.gp_flags);
if (rnp->qsmask & mask) { /* RCU waiting on outgoing CPU? */
/* Report quiescent state -before- changing ->qsmaskinitnext! */
- rcu_report_qs_rnp(mask, rsp, rnp, rnp->gp_seq, flags);
+ rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags);
raw_spin_lock_irqsave_rcu_node(rnp, flags);
}
rnp->qsmaskinitnext &= ~mask;
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
- spin_unlock(&rsp->ofl_lock);
-}
-
-/*
- * The outgoing function has no further need of RCU, so remove it from
- * the list of CPUs that RCU must track.
- *
- * Note that this function is special in that it is invoked directly
- * from the outgoing CPU rather than from the cpuhp_step mechanism.
- * This is because this function must be invoked at a precise location.
- */
-void rcu_report_dead(unsigned int cpu)
-{
- struct rcu_state *rsp;
-
- /* QS for any half-done expedited RCU-sched GP. */
- preempt_disable();
- rcu_report_exp_rdp(&rcu_sched_state,
- this_cpu_ptr(rcu_sched_state.rda), true);
- preempt_enable();
- for_each_rcu_flavor(rsp)
- rcu_cleanup_dying_idle_cpu(cpu, rsp);
+ raw_spin_unlock(&rcu_state.ofl_lock);
per_cpu(rcu_cpu_started, cpu) = 0;
}
-/* Migrate the dead CPU's callbacks to the current CPU. */
-static void rcu_migrate_callbacks(int cpu, struct rcu_state *rsp)
+/*
+ * The outgoing CPU has just passed through the dying-idle state, and we
+ * are being invoked from the CPU that was IPIed to continue the offline
+ * operation. Migrate the outgoing CPU's callbacks to the current CPU.
+ */
+void rcutree_migrate_callbacks(int cpu)
{
unsigned long flags;
struct rcu_data *my_rdp;
- struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
- struct rcu_node *rnp_root = rcu_get_root(rdp->rsp);
+ struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
+ struct rcu_node *rnp_root = rcu_get_root();
bool needwake;
if (rcu_is_nocb_cpu(cpu) || rcu_segcblist_empty(&rdp->cblist))
return; /* No callbacks to migrate. */
local_irq_save(flags);
- my_rdp = this_cpu_ptr(rsp->rda);
+ my_rdp = this_cpu_ptr(&rcu_data);
if (rcu_nocb_adopt_orphan_cbs(my_rdp, rdp, flags)) {
local_irq_restore(flags);
return;
}
raw_spin_lock_rcu_node(rnp_root); /* irqs already disabled. */
/* Leverage recent GPs and set GP for new callbacks. */
- needwake = rcu_advance_cbs(rsp, rnp_root, rdp) ||
- rcu_advance_cbs(rsp, rnp_root, my_rdp);
+ needwake = rcu_advance_cbs(rnp_root, rdp) ||
+ rcu_advance_cbs(rnp_root, my_rdp);
rcu_segcblist_merge(&my_rdp->cblist, &rdp->cblist);
WARN_ON_ONCE(rcu_segcblist_empty(&my_rdp->cblist) !=
!rcu_segcblist_n_cbs(&my_rdp->cblist));
raw_spin_unlock_irqrestore_rcu_node(rnp_root, flags);
if (needwake)
- rcu_gp_kthread_wake(rsp);
+ rcu_gp_kthread_wake();
WARN_ONCE(rcu_segcblist_n_cbs(&rdp->cblist) != 0 ||
!rcu_segcblist_empty(&rdp->cblist),
"rcu_cleanup_dead_cpu: Callbacks on offline CPU %d: qlen=%lu, 1stCB=%p\n",
cpu, rcu_segcblist_n_cbs(&rdp->cblist),
rcu_segcblist_first_cb(&rdp->cblist));
}
-
-/*
- * The outgoing CPU has just passed through the dying-idle state,
- * and we are being invoked from the CPU that was IPIed to continue the
- * offline operation. We need to migrate the outgoing CPU's callbacks.
- */
-void rcutree_migrate_callbacks(int cpu)
-{
- struct rcu_state *rsp;
-
- for_each_rcu_flavor(rsp)
- rcu_migrate_callbacks(cpu, rsp);
-}
#endif
/*
}
/*
- * Spawn the kthreads that handle each RCU flavor's grace periods.
+ * Spawn the kthreads that handle RCU's grace periods.
*/
static int __init rcu_spawn_gp_kthread(void)
{
unsigned long flags;
int kthread_prio_in = kthread_prio;
struct rcu_node *rnp;
- struct rcu_state *rsp;
struct sched_param sp;
struct task_struct *t;
kthread_prio, kthread_prio_in);
rcu_scheduler_fully_active = 1;
- for_each_rcu_flavor(rsp) {
- t = kthread_create(rcu_gp_kthread, rsp, "%s", rsp->name);
- BUG_ON(IS_ERR(t));
- rnp = rcu_get_root(rsp);
- raw_spin_lock_irqsave_rcu_node(rnp, flags);
- rsp->gp_kthread = t;
- if (kthread_prio) {
- sp.sched_priority = kthread_prio;
- sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
- }
- raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
- wake_up_process(t);
+ t = kthread_create(rcu_gp_kthread, NULL, "%s", rcu_state.name);
+ BUG_ON(IS_ERR(t));
+ rnp = rcu_get_root();
+ raw_spin_lock_irqsave_rcu_node(rnp, flags);
+ rcu_state.gp_kthread = t;
+ if (kthread_prio) {
+ sp.sched_priority = kthread_prio;
+ sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
}
+ raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
+ wake_up_process(t);
rcu_spawn_nocb_kthreads();
rcu_spawn_boost_kthreads();
return 0;
}
/*
- * Helper function for rcu_init() that initializes one rcu_state structure.
+ * Helper function for rcu_init() that initializes the rcu_state structure.
*/
-static void __init rcu_init_one(struct rcu_state *rsp)
+static void __init rcu_init_one(void)
{
static const char * const buf[] = RCU_NODE_NAME_INIT;
static const char * const fqs[] = RCU_FQS_NAME_INIT;
/* Initialize the level-tracking arrays. */
for (i = 1; i < rcu_num_lvls; i++)
- rsp->level[i] = rsp->level[i - 1] + num_rcu_lvl[i - 1];
+ rcu_state.level[i] =
+ rcu_state.level[i - 1] + num_rcu_lvl[i - 1];
rcu_init_levelspread(levelspread, num_rcu_lvl);
/* Initialize the elements themselves, starting from the leaves. */
for (i = rcu_num_lvls - 1; i >= 0; i--) {
cpustride *= levelspread[i];
- rnp = rsp->level[i];
+ rnp = rcu_state.level[i];
for (j = 0; j < num_rcu_lvl[i]; j++, rnp++) {
raw_spin_lock_init(&ACCESS_PRIVATE(rnp, lock));
lockdep_set_class_and_name(&ACCESS_PRIVATE(rnp, lock),
raw_spin_lock_init(&rnp->fqslock);
lockdep_set_class_and_name(&rnp->fqslock,
&rcu_fqs_class[i], fqs[i]);
- rnp->gp_seq = rsp->gp_seq;
- rnp->gp_seq_needed = rsp->gp_seq;
- rnp->completedqs = rsp->gp_seq;
+ rnp->gp_seq = rcu_state.gp_seq;
+ rnp->gp_seq_needed = rcu_state.gp_seq;
+ rnp->completedqs = rcu_state.gp_seq;
rnp->qsmask = 0;
rnp->qsmaskinit = 0;
rnp->grplo = j * cpustride;
rnp->parent = NULL;
} else {
rnp->grpnum = j % levelspread[i - 1];
- rnp->grpmask = 1UL << rnp->grpnum;
- rnp->parent = rsp->level[i - 1] +
+ rnp->grpmask = BIT(rnp->grpnum);
+ rnp->parent = rcu_state.level[i - 1] +
j / levelspread[i - 1];
}
rnp->level = i;
}
}
- init_swait_queue_head(&rsp->gp_wq);
- init_swait_queue_head(&rsp->expedited_wq);
- rnp = rcu_first_leaf_node(rsp);
+ init_swait_queue_head(&rcu_state.gp_wq);
+ init_swait_queue_head(&rcu_state.expedited_wq);
+ rnp = rcu_first_leaf_node();
for_each_possible_cpu(i) {
while (i > rnp->grphi)
rnp++;
- per_cpu_ptr(rsp->rda, i)->mynode = rnp;
- rcu_boot_init_percpu_data(i, rsp);
+ per_cpu_ptr(&rcu_data, i)->mynode = rnp;
+ rcu_boot_init_percpu_data(i);
}
- list_add(&rsp->flavors, &rcu_struct_flavors);
}
/*
jiffies_till_first_fqs = d;
if (jiffies_till_next_fqs == ULONG_MAX)
jiffies_till_next_fqs = d;
+ if (jiffies_till_sched_qs == ULONG_MAX)
+ adjust_jiffies_till_sched_qs();
/* If the compile-time values are accurate, just leave. */
if (rcu_fanout_leaf == RCU_FANOUT_LEAF &&
/*
* Dump out the structure of the rcu_node combining tree associated
- * with the rcu_state structure referenced by rsp.
+ * with the rcu_state structure.
*/
-static void __init rcu_dump_rcu_node_tree(struct rcu_state *rsp)
+static void __init rcu_dump_rcu_node_tree(void)
{
int level = 0;
struct rcu_node *rnp;
pr_info("rcu_node tree layout dump\n");
pr_info(" ");
- rcu_for_each_node_breadth_first(rsp, rnp) {
+ rcu_for_each_node_breadth_first(rnp) {
if (rnp->level != level) {
pr_cont("\n");
pr_info(" ");
rcu_bootup_announce();
rcu_init_geometry();
- rcu_init_one(&rcu_bh_state);
- rcu_init_one(&rcu_sched_state);
+ rcu_init_one();
if (dump_tree)
- rcu_dump_rcu_node_tree(&rcu_sched_state);
- __rcu_init_preempt();
+ rcu_dump_rcu_node_tree();
open_softirq(RCU_SOFTIRQ, rcu_process_callbacks);
/*
WARN_ON(!rcu_gp_wq);
rcu_par_gp_wq = alloc_workqueue("rcu_par_gp", WQ_MEM_RECLAIM, 0);
WARN_ON(!rcu_par_gp_wq);
+ srcu_init();
}
#include "tree_exp.h"
#include "rcu_segcblist.h"
-/*
- * Dynticks per-CPU state.
- */
-struct rcu_dynticks {
- long dynticks_nesting; /* Track process nesting level. */
- long dynticks_nmi_nesting; /* Track irq/NMI nesting level. */
- atomic_t dynticks; /* Even value for idle, else odd. */
- bool rcu_need_heavy_qs; /* GP old, need heavy quiescent state. */
- unsigned long rcu_qs_ctr; /* Light universal quiescent state ctr. */
- bool rcu_urgent_qs; /* GP old need light quiescent state. */
-#ifdef CONFIG_RCU_FAST_NO_HZ
- bool all_lazy; /* Are all CPU's CBs lazy? */
- unsigned long nonlazy_posted;
- /* # times non-lazy CBs posted to CPU. */
- unsigned long nonlazy_posted_snap;
- /* idle-period nonlazy_posted snapshot. */
- unsigned long last_accelerate;
- /* Last jiffy CBs were accelerated. */
- unsigned long last_advance_all;
- /* Last jiffy CBs were all advanced. */
- int tick_nohz_enabled_snap; /* Previously seen value from sysfs. */
-#endif /* #ifdef CONFIG_RCU_FAST_NO_HZ */
-};
-
/* Communicate arguments to a workqueue handler. */
struct rcu_exp_work {
smp_call_func_t rew_func;
- struct rcu_state *rew_rsp;
unsigned long rew_s;
struct work_struct rew_work;
};
* are indexed relative to this interval rather than the global CPU ID space.
* This generates the bit for a CPU in node-local masks.
*/
-#define leaf_node_cpu_bit(rnp, cpu) (1UL << ((cpu) - (rnp)->grplo))
+#define leaf_node_cpu_bit(rnp, cpu) (BIT((cpu) - (rnp)->grplo))
/*
* Union to allow "aggregate OR" operation on the need for a quiescent
/* 1) quiescent-state and grace-period handling : */
unsigned long gp_seq; /* Track rsp->rcu_gp_seq counter. */
unsigned long gp_seq_needed; /* Track rsp->rcu_gp_seq_needed ctr. */
- unsigned long rcu_qs_ctr_snap;/* Snapshot of rcu_qs_ctr to check */
- /* for rcu_all_qs() invocations. */
union rcu_noqs cpu_no_qs; /* No QSes yet for this CPU. */
bool core_needs_qs; /* Core waits for quiesc state. */
bool beenonline; /* CPU online at least once. */
bool gpwrap; /* Possible ->gp_seq wrap. */
+ bool deferred_qs; /* This CPU awaiting a deferred QS? */
struct rcu_node *mynode; /* This CPU's leaf of hierarchy */
unsigned long grpmask; /* Mask to apply to leaf qsmask. */
unsigned long ticks_this_gp; /* The number of scheduling-clock */
long blimit; /* Upper limit on a processed batch */
/* 3) dynticks interface. */
- struct rcu_dynticks *dynticks; /* Shared per-CPU dynticks state. */
int dynticks_snap; /* Per-GP tracking for dynticks. */
-
- /* 4) reasons this CPU needed to be kicked by force_quiescent_state */
- unsigned long dynticks_fqs; /* Kicked due to dynticks idle. */
- unsigned long cond_resched_completed;
- /* Grace period that needs help */
- /* from cond_resched(). */
-
- /* 5) _rcu_barrier(), OOM callbacks, and expediting. */
- struct rcu_head barrier_head;
+ long dynticks_nesting; /* Track process nesting level. */
+ long dynticks_nmi_nesting; /* Track irq/NMI nesting level. */
+ atomic_t dynticks; /* Even value for idle, else odd. */
+ bool rcu_need_heavy_qs; /* GP old, so heavy quiescent state! */
+ bool rcu_urgent_qs; /* GP old need light quiescent state. */
#ifdef CONFIG_RCU_FAST_NO_HZ
- struct rcu_head oom_head;
+ bool all_lazy; /* Are all CPU's CBs lazy? */
+ unsigned long nonlazy_posted; /* # times non-lazy CB posted to CPU. */
+ unsigned long nonlazy_posted_snap;
+ /* Nonlazy_posted snapshot. */
+ unsigned long last_accelerate; /* Last jiffy CBs were accelerated. */
+ unsigned long last_advance_all; /* Last jiffy CBs were all advanced. */
+ int tick_nohz_enabled_snap; /* Previously seen value from sysfs. */
#endif /* #ifdef CONFIG_RCU_FAST_NO_HZ */
+
+ /* 4) rcu_barrier(), OOM callbacks, and expediting. */
+ struct rcu_head barrier_head;
int exp_dynticks_snap; /* Double-check need for IPI. */
- /* 6) Callback offloading. */
+ /* 5) Callback offloading. */
#ifdef CONFIG_RCU_NOCB_CPU
struct rcu_head *nocb_head; /* CBs waiting for kthread. */
struct rcu_head **nocb_tail;
/* Leader CPU takes GP-end wakeups. */
#endif /* #ifdef CONFIG_RCU_NOCB_CPU */
- /* 7) Diagnostic data, including RCU CPU stall warnings. */
+ /* 6) Diagnostic data, including RCU CPU stall warnings. */
unsigned int softirq_snap; /* Snapshot of softirq activity. */
/* ->rcu_iw* fields protected by leaf rcu_node ->lock. */
struct irq_work rcu_iw; /* Check for non-irq activity. */
short rcu_ofl_gp_flags; /* ->gp_flags at last offline. */
unsigned long rcu_onl_gp_seq; /* ->gp_seq at last online. */
short rcu_onl_gp_flags; /* ->gp_flags at last online. */
+ unsigned long last_fqs_resched; /* Time of last rcu_resched(). */
int cpu;
- struct rcu_state *rsp;
};
/* Values for nocb_defer_wakeup field in struct rcu_data. */
struct rcu_node *level[RCU_NUM_LVLS + 1];
/* Hierarchy levels (+1 to */
/* shut bogus gcc warning) */
- struct rcu_data __percpu *rda; /* pointer of percu rcu_data. */
- call_rcu_func_t call; /* call_rcu() flavor. */
int ncpus; /* # CPUs seen so far. */
/* The following fields are guarded by the root rcu_node's lock. */
atomic_t barrier_cpu_count; /* # CPUs waiting on. */
struct completion barrier_completion; /* Wake at barrier end. */
unsigned long barrier_sequence; /* ++ at start and end of */
- /* _rcu_barrier(). */
+ /* rcu_barrier(). */
/* End of fields guarded by barrier_mutex. */
struct mutex exp_mutex; /* Serialize expedited GP. */
/* jiffies. */
const char *name; /* Name of structure. */
char abbr; /* Abbreviated name. */
- struct list_head flavors; /* List of RCU flavors. */
- spinlock_t ofl_lock ____cacheline_internodealigned_in_smp;
+ raw_spinlock_t ofl_lock ____cacheline_internodealigned_in_smp;
/* Synchronize offline with */
/* GP pre-initialization. */
};
#define RCU_GP_CLEANUP 7 /* Grace-period cleanup started. */
#define RCU_GP_CLEANED 8 /* Grace-period cleanup complete. */
-#ifndef RCU_TREE_NONCORE
static const char * const gp_state_names[] = {
"RCU_GP_IDLE",
"RCU_GP_WAIT_GPS",
"RCU_GP_CLEANUP",
"RCU_GP_CLEANED",
};
-#endif /* #ifndef RCU_TREE_NONCORE */
-
-extern struct list_head rcu_struct_flavors;
-/* Sequence through rcu_state structures for each RCU flavor. */
-#define for_each_rcu_flavor(rsp) \
- list_for_each_entry((rsp), &rcu_struct_flavors, flavors)
+/*
+ * In order to export the rcu_state name to the tracing tools, it
+ * needs to be added in the __tracepoint_string section.
+ * This requires defining a separate variable tp_<sname>_varname
+ * that points to the string being used, and this will allow
+ * the tracing userspace tools to be able to decipher the string
+ * address to the matching string.
+ */
+#ifdef CONFIG_PREEMPT_RCU
+#define RCU_ABBR 'p'
+#define RCU_NAME_RAW "rcu_preempt"
+#else /* #ifdef CONFIG_PREEMPT_RCU */
+#define RCU_ABBR 's'
+#define RCU_NAME_RAW "rcu_sched"
+#endif /* #else #ifdef CONFIG_PREEMPT_RCU */
+#ifndef CONFIG_TRACING
+#define RCU_NAME RCU_NAME_RAW
+#else /* #ifdef CONFIG_TRACING */
+static char rcu_name[] = RCU_NAME_RAW;
+static const char *tp_rcu_varname __used __tracepoint_string = rcu_name;
+#define RCU_NAME rcu_name
+#endif /* #else #ifdef CONFIG_TRACING */
/*
* RCU implementation internal declarations:
extern struct rcu_state rcu_preempt_state;
#endif /* #ifdef CONFIG_PREEMPT_RCU */
-int rcu_dynticks_snap(struct rcu_dynticks *rdtp);
+int rcu_dynticks_snap(struct rcu_data *rdp);
#ifdef CONFIG_RCU_BOOST
DECLARE_PER_CPU(unsigned int, rcu_cpu_kthread_status);
DECLARE_PER_CPU(char, rcu_cpu_has_work);
#endif /* #ifdef CONFIG_RCU_BOOST */
-#ifndef RCU_TREE_NONCORE
-
/* Forward declarations for rcutree_plugin.h */
static void rcu_bootup_announce(void);
-static void rcu_preempt_note_context_switch(bool preempt);
+static void rcu_qs(void);
static int rcu_preempt_blocked_readers_cgp(struct rcu_node *rnp);
#ifdef CONFIG_HOTPLUG_CPU
static bool rcu_preempt_has_tasks(struct rcu_node *rnp);
#endif /* #ifdef CONFIG_HOTPLUG_CPU */
-static void rcu_print_detail_task_stall(struct rcu_state *rsp);
+static void rcu_print_detail_task_stall(void);
static int rcu_print_task_stall(struct rcu_node *rnp);
static int rcu_print_task_exp_stall(struct rcu_node *rnp);
-static void rcu_preempt_check_blocked_tasks(struct rcu_state *rsp,
- struct rcu_node *rnp);
-static void rcu_preempt_check_callbacks(void);
+static void rcu_preempt_check_blocked_tasks(struct rcu_node *rnp);
+static void rcu_flavor_check_callbacks(int user);
void call_rcu(struct rcu_head *head, rcu_callback_t func);
-static void __init __rcu_init_preempt(void);
-static void dump_blkd_tasks(struct rcu_state *rsp, struct rcu_node *rnp,
- int ncheck);
+static void dump_blkd_tasks(struct rcu_node *rnp, int ncheck);
static void rcu_initiate_boost(struct rcu_node *rnp, unsigned long flags);
static void rcu_preempt_boost_start_gp(struct rcu_node *rnp);
static void invoke_rcu_callbacks_kthread(void);
static bool rcu_is_callbacks_kthread(void);
-#ifdef CONFIG_RCU_BOOST
-static int rcu_spawn_one_boost_kthread(struct rcu_state *rsp,
- struct rcu_node *rnp);
-#endif /* #ifdef CONFIG_RCU_BOOST */
static void __init rcu_spawn_boost_kthreads(void);
static void rcu_prepare_kthreads(int cpu);
static void rcu_cleanup_after_idle(void);
static void rcu_prepare_for_idle(void);
static void rcu_idle_count_callbacks_posted(void);
static bool rcu_preempt_has_tasks(struct rcu_node *rnp);
+static bool rcu_preempt_need_deferred_qs(struct task_struct *t);
+static void rcu_preempt_deferred_qs(struct task_struct *t);
static void print_cpu_stall_info_begin(void);
-static void print_cpu_stall_info(struct rcu_state *rsp, int cpu);
+static void print_cpu_stall_info(int cpu);
static void print_cpu_stall_info_end(void);
static void zero_cpu_stall_ticks(struct rcu_data *rdp);
-static void increment_cpu_stall_ticks(void);
-static bool rcu_nocb_cpu_needs_barrier(struct rcu_state *rsp, int cpu);
+static bool rcu_nocb_cpu_needs_barrier(int cpu);
static struct swait_queue_head *rcu_nocb_gp_get(struct rcu_node *rnp);
static void rcu_nocb_gp_cleanup(struct swait_queue_head *sq);
static void rcu_init_one_nocb(struct rcu_node *rnp);
static void rcu_spawn_all_nocb_kthreads(int cpu);
static void __init rcu_spawn_nocb_kthreads(void);
#ifdef CONFIG_RCU_NOCB_CPU
-static void __init rcu_organize_nocb_kthreads(struct rcu_state *rsp);
+static void __init rcu_organize_nocb_kthreads(void);
#endif /* #ifdef CONFIG_RCU_NOCB_CPU */
static bool init_nocb_callback_list(struct rcu_data *rdp);
static void rcu_bind_gp_kthread(void);
-static bool rcu_nohz_full_cpu(struct rcu_state *rsp);
+static bool rcu_nohz_full_cpu(void);
static void rcu_dynticks_task_enter(void);
static void rcu_dynticks_task_exit(void);
void srcu_online_cpu(unsigned int cpu) { }
void srcu_offline_cpu(unsigned int cpu) { }
#endif /* #else #ifdef CONFIG_SRCU */
-
-#endif /* #ifndef RCU_TREE_NONCORE */
/*
* Record the start of an expedited grace period.
*/
-static void rcu_exp_gp_seq_start(struct rcu_state *rsp)
+static void rcu_exp_gp_seq_start(void)
{
- rcu_seq_start(&rsp->expedited_sequence);
+ rcu_seq_start(&rcu_state.expedited_sequence);
}
/*
* Return then value that expedited-grace-period counter will have
* at the end of the current grace period.
*/
-static __maybe_unused unsigned long rcu_exp_gp_seq_endval(struct rcu_state *rsp)
+static __maybe_unused unsigned long rcu_exp_gp_seq_endval(void)
{
- return rcu_seq_endval(&rsp->expedited_sequence);
+ return rcu_seq_endval(&rcu_state.expedited_sequence);
}
/*
* Record the end of an expedited grace period.
*/
-static void rcu_exp_gp_seq_end(struct rcu_state *rsp)
+static void rcu_exp_gp_seq_end(void)
{
- rcu_seq_end(&rsp->expedited_sequence);
+ rcu_seq_end(&rcu_state.expedited_sequence);
smp_mb(); /* Ensure that consecutive grace periods serialize. */
}
/*
* Take a snapshot of the expedited-grace-period counter.
*/
-static unsigned long rcu_exp_gp_seq_snap(struct rcu_state *rsp)
+static unsigned long rcu_exp_gp_seq_snap(void)
{
unsigned long s;
smp_mb(); /* Caller's modifications seen first by other CPUs. */
- s = rcu_seq_snap(&rsp->expedited_sequence);
- trace_rcu_exp_grace_period(rsp->name, s, TPS("snap"));
+ s = rcu_seq_snap(&rcu_state.expedited_sequence);
+ trace_rcu_exp_grace_period(rcu_state.name, s, TPS("snap"));
return s;
}
* if a full expedited grace period has elapsed since that snapshot
* was taken.
*/
-static bool rcu_exp_gp_seq_done(struct rcu_state *rsp, unsigned long s)
+static bool rcu_exp_gp_seq_done(unsigned long s)
{
- return rcu_seq_done(&rsp->expedited_sequence, s);
+ return rcu_seq_done(&rcu_state.expedited_sequence, s);
}
/*
* ever been online. This means that this function normally takes its
* no-work-to-do fastpath.
*/
-static void sync_exp_reset_tree_hotplug(struct rcu_state *rsp)
+static void sync_exp_reset_tree_hotplug(void)
{
bool done;
unsigned long flags;
unsigned long mask;
unsigned long oldmask;
- int ncpus = smp_load_acquire(&rsp->ncpus); /* Order against locking. */
+ int ncpus = smp_load_acquire(&rcu_state.ncpus); /* Order vs. locking. */
struct rcu_node *rnp;
struct rcu_node *rnp_up;
/* If no new CPUs onlined since last time, nothing to do. */
- if (likely(ncpus == rsp->ncpus_snap))
+ if (likely(ncpus == rcu_state.ncpus_snap))
return;
- rsp->ncpus_snap = ncpus;
+ rcu_state.ncpus_snap = ncpus;
/*
* Each pass through the following loop propagates newly onlined
* CPUs for the current rcu_node structure up the rcu_node tree.
*/
- rcu_for_each_leaf_node(rsp, rnp) {
+ rcu_for_each_leaf_node(rnp) {
raw_spin_lock_irqsave_rcu_node(rnp, flags);
if (rnp->expmaskinit == rnp->expmaskinitnext) {
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
* Reset the ->expmask values in the rcu_node tree in preparation for
* a new expedited grace period.
*/
-static void __maybe_unused sync_exp_reset_tree(struct rcu_state *rsp)
+static void __maybe_unused sync_exp_reset_tree(void)
{
unsigned long flags;
struct rcu_node *rnp;
- sync_exp_reset_tree_hotplug(rsp);
- rcu_for_each_node_breadth_first(rsp, rnp) {
+ sync_exp_reset_tree_hotplug();
+ rcu_for_each_node_breadth_first(rnp) {
raw_spin_lock_irqsave_rcu_node(rnp, flags);
WARN_ON_ONCE(rnp->expmask);
rnp->expmask = rnp->expmaskinit;
*
* Caller must hold the specified rcu_node structure's ->lock.
*/
-static void __rcu_report_exp_rnp(struct rcu_state *rsp, struct rcu_node *rnp,
+static void __rcu_report_exp_rnp(struct rcu_node *rnp,
bool wake, unsigned long flags)
__releases(rnp->lock)
{
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
if (wake) {
smp_mb(); /* EGP done before wake_up(). */
- swake_up_one(&rsp->expedited_wq);
+ swake_up_one(&rcu_state.expedited_wq);
}
break;
}
* Report expedited quiescent state for specified node. This is a
* lock-acquisition wrapper function for __rcu_report_exp_rnp().
*/
-static void __maybe_unused rcu_report_exp_rnp(struct rcu_state *rsp,
- struct rcu_node *rnp, bool wake)
+static void __maybe_unused rcu_report_exp_rnp(struct rcu_node *rnp, bool wake)
{
unsigned long flags;
raw_spin_lock_irqsave_rcu_node(rnp, flags);
- __rcu_report_exp_rnp(rsp, rnp, wake, flags);
+ __rcu_report_exp_rnp(rnp, wake, flags);
}
/*
* Report expedited quiescent state for multiple CPUs, all covered by the
* specified leaf rcu_node structure.
*/
-static void rcu_report_exp_cpu_mult(struct rcu_state *rsp, struct rcu_node *rnp,
+static void rcu_report_exp_cpu_mult(struct rcu_node *rnp,
unsigned long mask, bool wake)
{
unsigned long flags;
return;
}
rnp->expmask &= ~mask;
- __rcu_report_exp_rnp(rsp, rnp, wake, flags); /* Releases rnp->lock. */
+ __rcu_report_exp_rnp(rnp, wake, flags); /* Releases rnp->lock. */
}
/*
* Report expedited quiescent state for specified rcu_data (CPU).
*/
-static void rcu_report_exp_rdp(struct rcu_state *rsp, struct rcu_data *rdp,
- bool wake)
+static void rcu_report_exp_rdp(struct rcu_data *rdp)
{
- rcu_report_exp_cpu_mult(rsp, rdp->mynode, rdp->grpmask, wake);
+ WRITE_ONCE(rdp->deferred_qs, false);
+ rcu_report_exp_cpu_mult(rdp->mynode, rdp->grpmask, true);
}
-/* Common code for synchronize_{rcu,sched}_expedited() work-done checking. */
-static bool sync_exp_work_done(struct rcu_state *rsp, unsigned long s)
+/* Common code for work-done checking. */
+static bool sync_exp_work_done(unsigned long s)
{
- if (rcu_exp_gp_seq_done(rsp, s)) {
- trace_rcu_exp_grace_period(rsp->name, s, TPS("done"));
+ if (rcu_exp_gp_seq_done(s)) {
+ trace_rcu_exp_grace_period(rcu_state.name, s, TPS("done"));
/* Ensure test happens before caller kfree(). */
smp_mb__before_atomic(); /* ^^^ */
return true;
* with the mutex held, indicating that the caller must actually do the
* expedited grace period.
*/
-static bool exp_funnel_lock(struct rcu_state *rsp, unsigned long s)
+static bool exp_funnel_lock(unsigned long s)
{
- struct rcu_data *rdp = per_cpu_ptr(rsp->rda, raw_smp_processor_id());
+ struct rcu_data *rdp = per_cpu_ptr(&rcu_data, raw_smp_processor_id());
struct rcu_node *rnp = rdp->mynode;
- struct rcu_node *rnp_root = rcu_get_root(rsp);
+ struct rcu_node *rnp_root = rcu_get_root();
/* Low-contention fastpath. */
if (ULONG_CMP_LT(READ_ONCE(rnp->exp_seq_rq), s) &&
(rnp == rnp_root ||
ULONG_CMP_LT(READ_ONCE(rnp_root->exp_seq_rq), s)) &&
- mutex_trylock(&rsp->exp_mutex))
+ mutex_trylock(&rcu_state.exp_mutex))
goto fastpath;
/*
* Each pass through the following loop works its way up
* the rcu_node tree, returning if others have done the work or
- * otherwise falls through to acquire rsp->exp_mutex. The mapping
+ * otherwise falls through to acquire ->exp_mutex. The mapping
* from CPU to rcu_node structure can be inexact, as it is just
* promoting locality and is not strictly needed for correctness.
*/
for (; rnp != NULL; rnp = rnp->parent) {
- if (sync_exp_work_done(rsp, s))
+ if (sync_exp_work_done(s))
return true;
/* Work not done, either wait here or go up. */
/* Someone else doing GP, so wait for them. */
spin_unlock(&rnp->exp_lock);
- trace_rcu_exp_funnel_lock(rsp->name, rnp->level,
+ trace_rcu_exp_funnel_lock(rcu_state.name, rnp->level,
rnp->grplo, rnp->grphi,
TPS("wait"));
wait_event(rnp->exp_wq[rcu_seq_ctr(s) & 0x3],
- sync_exp_work_done(rsp, s));
+ sync_exp_work_done(s));
return true;
}
rnp->exp_seq_rq = s; /* Followers can wait on us. */
spin_unlock(&rnp->exp_lock);
- trace_rcu_exp_funnel_lock(rsp->name, rnp->level, rnp->grplo,
- rnp->grphi, TPS("nxtlvl"));
+ trace_rcu_exp_funnel_lock(rcu_state.name, rnp->level,
+ rnp->grplo, rnp->grphi, TPS("nxtlvl"));
}
- mutex_lock(&rsp->exp_mutex);
+ mutex_lock(&rcu_state.exp_mutex);
fastpath:
- if (sync_exp_work_done(rsp, s)) {
- mutex_unlock(&rsp->exp_mutex);
+ if (sync_exp_work_done(s)) {
+ mutex_unlock(&rcu_state.exp_mutex);
return true;
}
- rcu_exp_gp_seq_start(rsp);
- trace_rcu_exp_grace_period(rsp->name, s, TPS("start"));
+ rcu_exp_gp_seq_start();
+ trace_rcu_exp_grace_period(rcu_state.name, s, TPS("start"));
return false;
}
-/* Invoked on each online non-idle CPU for expedited quiescent state. */
-static void sync_sched_exp_handler(void *data)
-{
- struct rcu_data *rdp;
- struct rcu_node *rnp;
- struct rcu_state *rsp = data;
-
- rdp = this_cpu_ptr(rsp->rda);
- rnp = rdp->mynode;
- if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) ||
- __this_cpu_read(rcu_sched_data.cpu_no_qs.b.exp))
- return;
- if (rcu_is_cpu_rrupt_from_idle()) {
- rcu_report_exp_rdp(&rcu_sched_state,
- this_cpu_ptr(&rcu_sched_data), true);
- return;
- }
- __this_cpu_write(rcu_sched_data.cpu_no_qs.b.exp, true);
- /* Store .exp before .rcu_urgent_qs. */
- smp_store_release(this_cpu_ptr(&rcu_dynticks.rcu_urgent_qs), true);
- resched_cpu(smp_processor_id());
-}
-
-/* Send IPI for expedited cleanup if needed at end of CPU-hotplug operation. */
-static void sync_sched_exp_online_cleanup(int cpu)
-{
- struct rcu_data *rdp;
- int ret;
- struct rcu_node *rnp;
- struct rcu_state *rsp = &rcu_sched_state;
-
- rdp = per_cpu_ptr(rsp->rda, cpu);
- rnp = rdp->mynode;
- if (!(READ_ONCE(rnp->expmask) & rdp->grpmask))
- return;
- ret = smp_call_function_single(cpu, sync_sched_exp_handler, rsp, 0);
- WARN_ON_ONCE(ret);
-}
-
/*
* Select the CPUs within the specified rcu_node that the upcoming
* expedited grace period needs to wait for.
struct rcu_exp_work *rewp =
container_of(wp, struct rcu_exp_work, rew_work);
struct rcu_node *rnp = container_of(rewp, struct rcu_node, rew);
- struct rcu_state *rsp = rewp->rew_rsp;
func = rewp->rew_func;
raw_spin_lock_irqsave_rcu_node(rnp, flags);
mask_ofl_test = 0;
for_each_leaf_node_cpu_mask(rnp, cpu, rnp->expmask) {
unsigned long mask = leaf_node_cpu_bit(rnp, cpu);
- struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
- struct rcu_dynticks *rdtp = per_cpu_ptr(&rcu_dynticks, cpu);
+ struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
int snap;
if (raw_smp_processor_id() == cpu ||
!(rnp->qsmaskinitnext & mask)) {
mask_ofl_test |= mask;
} else {
- snap = rcu_dynticks_snap(rdtp);
+ snap = rcu_dynticks_snap(rdp);
if (rcu_dynticks_in_eqs(snap))
mask_ofl_test |= mask;
else
/* IPI the remaining CPUs for expedited quiescent state. */
for_each_leaf_node_cpu_mask(rnp, cpu, rnp->expmask) {
unsigned long mask = leaf_node_cpu_bit(rnp, cpu);
- struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
+ struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
if (!(mask_ofl_ipi & mask))
continue;
retry_ipi:
- if (rcu_dynticks_in_eqs_since(rdp->dynticks,
- rdp->exp_dynticks_snap)) {
+ if (rcu_dynticks_in_eqs_since(rdp, rdp->exp_dynticks_snap)) {
mask_ofl_test |= mask;
continue;
}
- ret = smp_call_function_single(cpu, func, rsp, 0);
+ ret = smp_call_function_single(cpu, func, NULL, 0);
if (!ret) {
mask_ofl_ipi &= ~mask;
continue;
(rnp->expmask & mask)) {
/* Online, so delay for a bit and try again. */
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
- trace_rcu_exp_grace_period(rsp->name, rcu_exp_gp_seq_endval(rsp), TPS("selectofl"));
+ trace_rcu_exp_grace_period(rcu_state.name, rcu_exp_gp_seq_endval(), TPS("selectofl"));
schedule_timeout_uninterruptible(1);
goto retry_ipi;
}
/* Report quiescent states for those that went offline. */
mask_ofl_test |= mask_ofl_ipi;
if (mask_ofl_test)
- rcu_report_exp_cpu_mult(rsp, rnp, mask_ofl_test, false);
+ rcu_report_exp_cpu_mult(rnp, mask_ofl_test, false);
}
/*
* Select the nodes that the upcoming expedited grace period needs
* to wait for.
*/
-static void sync_rcu_exp_select_cpus(struct rcu_state *rsp,
- smp_call_func_t func)
+static void sync_rcu_exp_select_cpus(smp_call_func_t func)
{
int cpu;
struct rcu_node *rnp;
- trace_rcu_exp_grace_period(rsp->name, rcu_exp_gp_seq_endval(rsp), TPS("reset"));
- sync_exp_reset_tree(rsp);
- trace_rcu_exp_grace_period(rsp->name, rcu_exp_gp_seq_endval(rsp), TPS("select"));
+ trace_rcu_exp_grace_period(rcu_state.name, rcu_exp_gp_seq_endval(), TPS("reset"));
+ sync_exp_reset_tree();
+ trace_rcu_exp_grace_period(rcu_state.name, rcu_exp_gp_seq_endval(), TPS("select"));
/* Schedule work for each leaf rcu_node structure. */
- rcu_for_each_leaf_node(rsp, rnp) {
+ rcu_for_each_leaf_node(rnp) {
rnp->exp_need_flush = false;
if (!READ_ONCE(rnp->expmask))
continue; /* Avoid early boot non-existent wq. */
rnp->rew.rew_func = func;
- rnp->rew.rew_rsp = rsp;
if (!READ_ONCE(rcu_par_gp_wq) ||
rcu_scheduler_active != RCU_SCHEDULER_RUNNING ||
- rcu_is_last_leaf_node(rsp, rnp)) {
+ rcu_is_last_leaf_node(rnp)) {
/* No workqueues yet or last leaf, do direct call. */
sync_rcu_exp_select_node_cpus(&rnp->rew.rew_work);
continue;
}
/* Wait for workqueue jobs (if any) to complete. */
- rcu_for_each_leaf_node(rsp, rnp)
+ rcu_for_each_leaf_node(rnp)
if (rnp->exp_need_flush)
flush_work(&rnp->rew.rew_work);
}
-static void synchronize_sched_expedited_wait(struct rcu_state *rsp)
+static void synchronize_sched_expedited_wait(void)
{
int cpu;
unsigned long jiffies_stall;
unsigned long mask;
int ndetected;
struct rcu_node *rnp;
- struct rcu_node *rnp_root = rcu_get_root(rsp);
+ struct rcu_node *rnp_root = rcu_get_root();
int ret;
- trace_rcu_exp_grace_period(rsp->name, rcu_exp_gp_seq_endval(rsp), TPS("startwait"));
+ trace_rcu_exp_grace_period(rcu_state.name, rcu_exp_gp_seq_endval(), TPS("startwait"));
jiffies_stall = rcu_jiffies_till_stall_check();
jiffies_start = jiffies;
for (;;) {
ret = swait_event_timeout_exclusive(
- rsp->expedited_wq,
+ rcu_state.expedited_wq,
sync_rcu_preempt_exp_done_unlocked(rnp_root),
jiffies_stall);
if (ret > 0 || sync_rcu_preempt_exp_done_unlocked(rnp_root))
continue;
panic_on_rcu_stall();
pr_err("INFO: %s detected expedited stalls on CPUs/tasks: {",
- rsp->name);
+ rcu_state.name);
ndetected = 0;
- rcu_for_each_leaf_node(rsp, rnp) {
+ rcu_for_each_leaf_node(rnp) {
ndetected += rcu_print_task_exp_stall(rnp);
for_each_leaf_node_possible_cpu(rnp, cpu) {
struct rcu_data *rdp;
if (!(rnp->expmask & mask))
continue;
ndetected++;
- rdp = per_cpu_ptr(rsp->rda, cpu);
+ rdp = per_cpu_ptr(&rcu_data, cpu);
pr_cont(" %d-%c%c%c", cpu,
"O."[!!cpu_online(cpu)],
"o."[!!(rdp->grpmask & rnp->expmaskinit)],
}
}
pr_cont(" } %lu jiffies s: %lu root: %#lx/%c\n",
- jiffies - jiffies_start, rsp->expedited_sequence,
+ jiffies - jiffies_start, rcu_state.expedited_sequence,
rnp_root->expmask, ".T"[!!rnp_root->exp_tasks]);
if (ndetected) {
pr_err("blocking rcu_node structures:");
- rcu_for_each_node_breadth_first(rsp, rnp) {
+ rcu_for_each_node_breadth_first(rnp) {
if (rnp == rnp_root)
continue; /* printed unconditionally */
if (sync_rcu_preempt_exp_done_unlocked(rnp))
}
pr_cont("\n");
}
- rcu_for_each_leaf_node(rsp, rnp) {
+ rcu_for_each_leaf_node(rnp) {
for_each_leaf_node_possible_cpu(rnp, cpu) {
mask = leaf_node_cpu_bit(rnp, cpu);
if (!(rnp->expmask & mask))
* grace period. Also update all the ->exp_seq_rq counters as needed
* in order to avoid counter-wrap problems.
*/
-static void rcu_exp_wait_wake(struct rcu_state *rsp, unsigned long s)
+static void rcu_exp_wait_wake(unsigned long s)
{
struct rcu_node *rnp;
- synchronize_sched_expedited_wait(rsp);
- rcu_exp_gp_seq_end(rsp);
- trace_rcu_exp_grace_period(rsp->name, s, TPS("end"));
+ synchronize_sched_expedited_wait();
+ rcu_exp_gp_seq_end();
+ trace_rcu_exp_grace_period(rcu_state.name, s, TPS("end"));
/*
* Switch over to wakeup mode, allowing the next GP, but -only- the
* next GP, to proceed.
*/
- mutex_lock(&rsp->exp_wake_mutex);
+ mutex_lock(&rcu_state.exp_wake_mutex);
- rcu_for_each_node_breadth_first(rsp, rnp) {
+ rcu_for_each_node_breadth_first(rnp) {
if (ULONG_CMP_LT(READ_ONCE(rnp->exp_seq_rq), s)) {
spin_lock(&rnp->exp_lock);
/* Recheck, avoid hang in case someone just arrived. */
spin_unlock(&rnp->exp_lock);
}
smp_mb(); /* All above changes before wakeup. */
- wake_up_all(&rnp->exp_wq[rcu_seq_ctr(rsp->expedited_sequence) & 0x3]);
+ wake_up_all(&rnp->exp_wq[rcu_seq_ctr(rcu_state.expedited_sequence) & 0x3]);
}
- trace_rcu_exp_grace_period(rsp->name, s, TPS("endwake"));
- mutex_unlock(&rsp->exp_wake_mutex);
+ trace_rcu_exp_grace_period(rcu_state.name, s, TPS("endwake"));
+ mutex_unlock(&rcu_state.exp_wake_mutex);
}
/*
* Common code to drive an expedited grace period forward, used by
* workqueues and mid-boot-time tasks.
*/
-static void rcu_exp_sel_wait_wake(struct rcu_state *rsp,
- smp_call_func_t func, unsigned long s)
+static void rcu_exp_sel_wait_wake(smp_call_func_t func, unsigned long s)
{
/* Initialize the rcu_node tree in preparation for the wait. */
- sync_rcu_exp_select_cpus(rsp, func);
+ sync_rcu_exp_select_cpus(func);
/* Wait and clean up, including waking everyone. */
- rcu_exp_wait_wake(rsp, s);
+ rcu_exp_wait_wake(s);
}
/*
struct rcu_exp_work *rewp;
rewp = container_of(wp, struct rcu_exp_work, rew_work);
- rcu_exp_sel_wait_wake(rewp->rew_rsp, rewp->rew_func, rewp->rew_s);
+ rcu_exp_sel_wait_wake(rewp->rew_func, rewp->rew_s);
}
/*
- * Given an rcu_state pointer and a smp_call_function() handler, kick
- * off the specified flavor of expedited grace period.
+ * Given a smp_call_function() handler, kick off the specified
+ * implementation of expedited grace period.
*/
-static void _synchronize_rcu_expedited(struct rcu_state *rsp,
- smp_call_func_t func)
+static void _synchronize_rcu_expedited(smp_call_func_t func)
{
struct rcu_data *rdp;
struct rcu_exp_work rew;
/* If expedited grace periods are prohibited, fall back to normal. */
if (rcu_gp_is_normal()) {
- wait_rcu_gp(rsp->call);
+ wait_rcu_gp(call_rcu);
return;
}
/* Take a snapshot of the sequence number. */
- s = rcu_exp_gp_seq_snap(rsp);
- if (exp_funnel_lock(rsp, s))
+ s = rcu_exp_gp_seq_snap();
+ if (exp_funnel_lock(s))
return; /* Someone else did our work for us. */
/* Ensure that load happens before action based on it. */
if (unlikely(rcu_scheduler_active == RCU_SCHEDULER_INIT)) {
/* Direct call during scheduler init and early_initcalls(). */
- rcu_exp_sel_wait_wake(rsp, func, s);
+ rcu_exp_sel_wait_wake(func, s);
} else {
/* Marshall arguments & schedule the expedited grace period. */
rew.rew_func = func;
- rew.rew_rsp = rsp;
rew.rew_s = s;
INIT_WORK_ONSTACK(&rew.rew_work, wait_rcu_exp_gp);
queue_work(rcu_gp_wq, &rew.rew_work);
}
/* Wait for expedited grace period to complete. */
- rdp = per_cpu_ptr(rsp->rda, raw_smp_processor_id());
- rnp = rcu_get_root(rsp);
+ rdp = per_cpu_ptr(&rcu_data, raw_smp_processor_id());
+ rnp = rcu_get_root();
wait_event(rnp->exp_wq[rcu_seq_ctr(s) & 0x3],
- sync_exp_work_done(rsp, s));
+ sync_exp_work_done(s));
smp_mb(); /* Workqueue actions happen before return. */
/* Let the next expedited grace period start. */
- mutex_unlock(&rsp->exp_mutex);
-}
-
-/**
- * synchronize_sched_expedited - Brute-force RCU-sched grace period
- *
- * Wait for an RCU-sched grace period to elapse, but use a "big hammer"
- * approach to force the grace period to end quickly. This consumes
- * significant time on all CPUs and is unfriendly to real-time workloads,
- * so is thus not recommended for any sort of common-case code. In fact,
- * if you are using synchronize_sched_expedited() in a loop, please
- * restructure your code to batch your updates, and then use a single
- * synchronize_sched() instead.
- *
- * This implementation can be thought of as an application of sequence
- * locking to expedited grace periods, but using the sequence counter to
- * determine when someone else has already done the work instead of for
- * retrying readers.
- */
-void synchronize_sched_expedited(void)
-{
- struct rcu_state *rsp = &rcu_sched_state;
-
- RCU_LOCKDEP_WARN(lock_is_held(&rcu_bh_lock_map) ||
- lock_is_held(&rcu_lock_map) ||
- lock_is_held(&rcu_sched_lock_map),
- "Illegal synchronize_sched_expedited() in RCU read-side critical section");
-
- /* If only one CPU, this is automatically a grace period. */
- if (rcu_blocking_is_gp())
- return;
-
- _synchronize_rcu_expedited(rsp, sync_sched_exp_handler);
+ mutex_unlock(&rcu_state.exp_mutex);
}
-EXPORT_SYMBOL_GPL(synchronize_sched_expedited);
#ifdef CONFIG_PREEMPT_RCU
* ->expmask fields in the rcu_node tree. Otherwise, immediately
* report the quiescent state.
*/
-static void sync_rcu_exp_handler(void *info)
+static void sync_rcu_exp_handler(void *unused)
{
- struct rcu_data *rdp;
- struct rcu_state *rsp = info;
+ unsigned long flags;
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
+ struct rcu_node *rnp = rdp->mynode;
struct task_struct *t = current;
/*
- * Within an RCU read-side critical section, request that the next
- * rcu_read_unlock() report. Unless this RCU read-side critical
- * section has already blocked, in which case it is already set
- * up for the expedited grace period to wait on it.
+ * First, the common case of not being in an RCU read-side
+ * critical section. If also enabled or idle, immediately
+ * report the quiescent state, otherwise defer.
*/
- if (t->rcu_read_lock_nesting > 0 &&
- !t->rcu_read_unlock_special.b.blocked) {
- t->rcu_read_unlock_special.b.exp_need_qs = true;
+ if (!t->rcu_read_lock_nesting) {
+ if (!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)) ||
+ rcu_dynticks_curr_cpu_in_eqs()) {
+ rcu_report_exp_rdp(rdp);
+ } else {
+ rdp->deferred_qs = true;
+ set_tsk_need_resched(t);
+ set_preempt_need_resched();
+ }
return;
}
/*
- * We are either exiting an RCU read-side critical section (negative
- * values of t->rcu_read_lock_nesting) or are not in one at all
- * (zero value of t->rcu_read_lock_nesting). Or we are in an RCU
- * read-side critical section that blocked before this expedited
- * grace period started. Either way, we can immediately report
- * the quiescent state.
+ * Second, the less-common case of being in an RCU read-side
+ * critical section. In this case we can count on a future
+ * rcu_read_unlock(). However, this rcu_read_unlock() might
+ * execute on some other CPU, but in that case there will be
+ * a future context switch. Either way, if the expedited
+ * grace period is still waiting on this CPU, set ->deferred_qs
+ * so that the eventual quiescent state will be reported.
+ * Note that there is a large group of race conditions that
+ * can have caused this quiescent state to already have been
+ * reported, so we really do need to check ->expmask.
+ */
+ if (t->rcu_read_lock_nesting > 0) {
+ raw_spin_lock_irqsave_rcu_node(rnp, flags);
+ if (rnp->expmask & rdp->grpmask)
+ rdp->deferred_qs = true;
+ raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
+ }
+
+ /*
+ * The final and least likely case is where the interrupted
+ * code was just about to or just finished exiting the RCU-preempt
+ * read-side critical section, and no, we can't tell which.
+ * So either way, set ->deferred_qs to flag later code that
+ * a quiescent state is required.
+ *
+ * If the CPU is fully enabled (or if some buggy RCU-preempt
+ * read-side critical section is being used from idle), just
+ * invoke rcu_preempt_defer_qs() to immediately report the
+ * quiescent state. We cannot use rcu_read_unlock_special()
+ * because we are in an interrupt handler, which will cause that
+ * function to take an early exit without doing anything.
+ *
+ * Otherwise, force a context switch after the CPU enables everything.
*/
- rdp = this_cpu_ptr(rsp->rda);
- rcu_report_exp_rdp(rsp, rdp, true);
+ rdp->deferred_qs = true;
+ if (!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)) ||
+ WARN_ON_ONCE(rcu_dynticks_curr_cpu_in_eqs())) {
+ rcu_preempt_deferred_qs(t);
+ } else {
+ set_tsk_need_resched(t);
+ set_preempt_need_resched();
+ }
+}
+
+/* PREEMPT=y, so no PREEMPT=n expedited grace period to clean up after. */
+static void sync_sched_exp_online_cleanup(int cpu)
+{
}
/**
* you are using synchronize_rcu_expedited() in a loop, please restructure
* your code to batch your updates, and then Use a single synchronize_rcu()
* instead.
+ *
+ * This has the same semantics as (but is more brutal than) synchronize_rcu().
*/
void synchronize_rcu_expedited(void)
{
- struct rcu_state *rsp = rcu_state_p;
-
RCU_LOCKDEP_WARN(lock_is_held(&rcu_bh_lock_map) ||
lock_is_held(&rcu_lock_map) ||
lock_is_held(&rcu_sched_lock_map),
if (rcu_scheduler_active == RCU_SCHEDULER_INACTIVE)
return;
- _synchronize_rcu_expedited(rsp, sync_rcu_exp_handler);
+ _synchronize_rcu_expedited(sync_rcu_exp_handler);
}
EXPORT_SYMBOL_GPL(synchronize_rcu_expedited);
#else /* #ifdef CONFIG_PREEMPT_RCU */
+/* Invoked on each online non-idle CPU for expedited quiescent state. */
+static void sync_sched_exp_handler(void *unused)
+{
+ struct rcu_data *rdp;
+ struct rcu_node *rnp;
+
+ rdp = this_cpu_ptr(&rcu_data);
+ rnp = rdp->mynode;
+ if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) ||
+ __this_cpu_read(rcu_data.cpu_no_qs.b.exp))
+ return;
+ if (rcu_is_cpu_rrupt_from_idle()) {
+ rcu_report_exp_rdp(this_cpu_ptr(&rcu_data));
+ return;
+ }
+ __this_cpu_write(rcu_data.cpu_no_qs.b.exp, true);
+ /* Store .exp before .rcu_urgent_qs. */
+ smp_store_release(this_cpu_ptr(&rcu_data.rcu_urgent_qs), true);
+ set_tsk_need_resched(current);
+ set_preempt_need_resched();
+}
+
+/* Send IPI for expedited cleanup if needed at end of CPU-hotplug operation. */
+static void sync_sched_exp_online_cleanup(int cpu)
+{
+ struct rcu_data *rdp;
+ int ret;
+ struct rcu_node *rnp;
+
+ rdp = per_cpu_ptr(&rcu_data, cpu);
+ rnp = rdp->mynode;
+ if (!(READ_ONCE(rnp->expmask) & rdp->grpmask))
+ return;
+ ret = smp_call_function_single(cpu, sync_sched_exp_handler, NULL, 0);
+ WARN_ON_ONCE(ret);
+}
+
/*
- * Wait for an rcu-preempt grace period, but make it happen quickly.
- * But because preemptible RCU does not exist, map to rcu-sched.
+ * Because a context switch is a grace period for !PREEMPT, any
+ * blocking grace-period wait automatically implies a grace period if
+ * there is only one CPU online at any point time during execution of
+ * either synchronize_rcu() or synchronize_rcu_expedited(). It is OK to
+ * occasionally incorrectly indicate that there are multiple CPUs online
+ * when there was in fact only one the whole time, as this just adds some
+ * overhead: RCU still operates correctly.
*/
+static int rcu_blocking_is_gp(void)
+{
+ int ret;
+
+ might_sleep(); /* Check for RCU read-side critical section. */
+ preempt_disable();
+ ret = num_online_cpus() <= 1;
+ preempt_enable();
+ return ret;
+}
+
+/* PREEMPT=n implementation of synchronize_rcu_expedited(). */
void synchronize_rcu_expedited(void)
{
- synchronize_sched_expedited();
+ RCU_LOCKDEP_WARN(lock_is_held(&rcu_bh_lock_map) ||
+ lock_is_held(&rcu_lock_map) ||
+ lock_is_held(&rcu_sched_lock_map),
+ "Illegal synchronize_rcu_expedited() in RCU read-side critical section");
+
+ /* If only one CPU, this is automatically a grace period. */
+ if (rcu_blocking_is_gp())
+ return;
+
+ _synchronize_rcu_expedited(sync_sched_exp_handler);
}
EXPORT_SYMBOL_GPL(synchronize_rcu_expedited);
#include "../locking/rtmutex_common.h"
/*
- * Control variables for per-CPU and per-rcu_node kthreads. These
- * handle all flavors of RCU.
+ * Control variables for per-CPU and per-rcu_node kthreads.
*/
static DEFINE_PER_CPU(struct task_struct *, rcu_cpu_kthread_task);
DEFINE_PER_CPU(unsigned int, rcu_cpu_kthread_status);
pr_info("\tBoot-time adjustment of first FQS scan delay to %ld jiffies.\n", jiffies_till_first_fqs);
if (jiffies_till_next_fqs != ULONG_MAX)
pr_info("\tBoot-time adjustment of subsequent FQS scan delay to %ld jiffies.\n", jiffies_till_next_fqs);
+ if (jiffies_till_sched_qs != ULONG_MAX)
+ pr_info("\tBoot-time adjustment of scheduler-enlistment delay to %ld jiffies.\n", jiffies_till_sched_qs);
if (rcu_kick_kthreads)
pr_info("\tKick kthreads if too-long grace period.\n");
if (IS_ENABLED(CONFIG_DEBUG_OBJECTS_RCU_HEAD))
#ifdef CONFIG_PREEMPT_RCU
-RCU_STATE_INITIALIZER(rcu_preempt, 'p', call_rcu);
-static struct rcu_state *const rcu_state_p = &rcu_preempt_state;
-static struct rcu_data __percpu *const rcu_data_p = &rcu_preempt_data;
-
-static void rcu_report_exp_rnp(struct rcu_state *rsp, struct rcu_node *rnp,
- bool wake);
+static void rcu_report_exp_rnp(struct rcu_node *rnp, bool wake);
static void rcu_read_unlock_special(struct task_struct *t);
/*
* no need to check for a subsequent expedited GP. (Though we are
* still in a quiescent state in any case.)
*/
- if (blkd_state & RCU_EXP_BLKD &&
- t->rcu_read_unlock_special.b.exp_need_qs) {
- t->rcu_read_unlock_special.b.exp_need_qs = false;
- rcu_report_exp_rdp(rdp->rsp, rdp, true);
- } else {
- WARN_ON_ONCE(t->rcu_read_unlock_special.b.exp_need_qs);
- }
+ if (blkd_state & RCU_EXP_BLKD && rdp->deferred_qs)
+ rcu_report_exp_rdp(rdp);
+ else
+ WARN_ON_ONCE(rdp->deferred_qs);
}
/*
*
* Callers to this function must disable preemption.
*/
-static void rcu_preempt_qs(void)
+static void rcu_qs(void)
{
- RCU_LOCKDEP_WARN(preemptible(), "rcu_preempt_qs() invoked with preemption enabled!!!\n");
- if (__this_cpu_read(rcu_data_p->cpu_no_qs.s)) {
+ RCU_LOCKDEP_WARN(preemptible(), "rcu_qs() invoked with preemption enabled!!!\n");
+ if (__this_cpu_read(rcu_data.cpu_no_qs.s)) {
trace_rcu_grace_period(TPS("rcu_preempt"),
- __this_cpu_read(rcu_data_p->gp_seq),
+ __this_cpu_read(rcu_data.gp_seq),
TPS("cpuqs"));
- __this_cpu_write(rcu_data_p->cpu_no_qs.b.norm, false);
- barrier(); /* Coordinate with rcu_preempt_check_callbacks(). */
+ __this_cpu_write(rcu_data.cpu_no_qs.b.norm, false);
+ barrier(); /* Coordinate with rcu_flavor_check_callbacks(). */
current->rcu_read_unlock_special.b.need_qs = false;
}
}
*
* Caller must disable interrupts.
*/
-static void rcu_preempt_note_context_switch(bool preempt)
+void rcu_note_context_switch(bool preempt)
{
struct task_struct *t = current;
- struct rcu_data *rdp;
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
struct rcu_node *rnp;
+ barrier(); /* Avoid RCU read-side critical sections leaking down. */
+ trace_rcu_utilization(TPS("Start context switch"));
lockdep_assert_irqs_disabled();
WARN_ON_ONCE(!preempt && t->rcu_read_lock_nesting > 0);
if (t->rcu_read_lock_nesting > 0 &&
!t->rcu_read_unlock_special.b.blocked) {
/* Possibly blocking in an RCU read-side critical section. */
- rdp = this_cpu_ptr(rcu_state_p->rda);
rnp = rdp->mynode;
raw_spin_lock_rcu_node(rnp);
t->rcu_read_unlock_special.b.blocked = true;
*/
WARN_ON_ONCE((rdp->grpmask & rcu_rnp_online_cpus(rnp)) == 0);
WARN_ON_ONCE(!list_empty(&t->rcu_node_entry));
- trace_rcu_preempt_task(rdp->rsp->name,
+ trace_rcu_preempt_task(rcu_state.name,
t->pid,
(rnp->qsmask & rdp->grpmask)
? rnp->gp_seq
* behalf of preempted instance of __rcu_read_unlock().
*/
rcu_read_unlock_special(t);
+ rcu_preempt_deferred_qs(t);
+ } else {
+ rcu_preempt_deferred_qs(t);
}
/*
* grace period, then the fact that the task has been enqueued
* means that we continue to block the current grace period.
*/
- rcu_preempt_qs();
+ rcu_qs();
+ if (rdp->deferred_qs)
+ rcu_report_exp_rdp(rdp);
+ trace_rcu_utilization(TPS("End context switch"));
+ barrier(); /* Avoid RCU read-side critical sections leaking up. */
}
+EXPORT_SYMBOL_GPL(rcu_note_context_switch);
/*
* Check for preempted RCU readers blocking the current grace period
}
/*
- * Handle special cases during rcu_read_unlock(), such as needing to
- * notify RCU core processing or task having blocked during the RCU
- * read-side critical section.
+ * Report deferred quiescent states. The deferral time can
+ * be quite short, for example, in the case of the call from
+ * rcu_read_unlock_special().
*/
-static void rcu_read_unlock_special(struct task_struct *t)
+static void
+rcu_preempt_deferred_qs_irqrestore(struct task_struct *t, unsigned long flags)
{
bool empty_exp;
bool empty_norm;
bool empty_exp_now;
- unsigned long flags;
struct list_head *np;
bool drop_boost_mutex = false;
struct rcu_data *rdp;
struct rcu_node *rnp;
union rcu_special special;
- /* NMI handlers cannot block and cannot safely manipulate state. */
- if (in_nmi())
- return;
-
- local_irq_save(flags);
-
/*
* If RCU core is waiting for this CPU to exit its critical section,
* report the fact that it has exited. Because irqs are disabled,
* t->rcu_read_unlock_special cannot change.
*/
special = t->rcu_read_unlock_special;
+ rdp = this_cpu_ptr(&rcu_data);
+ if (!special.s && !rdp->deferred_qs) {
+ local_irq_restore(flags);
+ return;
+ }
if (special.b.need_qs) {
- rcu_preempt_qs();
+ rcu_qs();
t->rcu_read_unlock_special.b.need_qs = false;
- if (!t->rcu_read_unlock_special.s) {
+ if (!t->rcu_read_unlock_special.s && !rdp->deferred_qs) {
local_irq_restore(flags);
return;
}
}
/*
- * Respond to a request for an expedited grace period, but only if
- * we were not preempted, meaning that we were running on the same
- * CPU throughout. If we were preempted, the exp_need_qs flag
- * would have been cleared at the time of the first preemption,
- * and the quiescent state would be reported when we were dequeued.
+ * Respond to a request by an expedited grace period for a
+ * quiescent state from this CPU. Note that requests from
+ * tasks are handled when removing the task from the
+ * blocked-tasks list below.
*/
- if (special.b.exp_need_qs) {
- WARN_ON_ONCE(special.b.blocked);
- t->rcu_read_unlock_special.b.exp_need_qs = false;
- rdp = this_cpu_ptr(rcu_state_p->rda);
- rcu_report_exp_rdp(rcu_state_p, rdp, true);
+ if (rdp->deferred_qs) {
+ rcu_report_exp_rdp(rdp);
if (!t->rcu_read_unlock_special.s) {
local_irq_restore(flags);
return;
}
}
- /* Hardware IRQ handlers cannot block, complain if they get here. */
- if (in_irq() || in_serving_softirq()) {
- lockdep_rcu_suspicious(__FILE__, __LINE__,
- "rcu_read_unlock() from irq or softirq with blocking in critical section!!!\n");
- pr_alert("->rcu_read_unlock_special: %#x (b: %d, enq: %d nq: %d)\n",
- t->rcu_read_unlock_special.s,
- t->rcu_read_unlock_special.b.blocked,
- t->rcu_read_unlock_special.b.exp_need_qs,
- t->rcu_read_unlock_special.b.need_qs);
- local_irq_restore(flags);
- return;
- }
-
/* Clean up if blocked during RCU read-side critical section. */
if (special.b.blocked) {
t->rcu_read_unlock_special.b.blocked = false;
rnp->grplo,
rnp->grphi,
!!rnp->gp_tasks);
- rcu_report_unblock_qs_rnp(rcu_state_p, rnp, flags);
+ rcu_report_unblock_qs_rnp(rnp, flags);
} else {
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
}
* then we need to report up the rcu_node hierarchy.
*/
if (!empty_exp && empty_exp_now)
- rcu_report_exp_rnp(rcu_state_p, rnp, true);
+ rcu_report_exp_rnp(rnp, true);
} else {
local_irq_restore(flags);
}
}
+/*
+ * Is a deferred quiescent-state pending, and are we also not in
+ * an RCU read-side critical section? It is the caller's responsibility
+ * to ensure it is otherwise safe to report any deferred quiescent
+ * states. The reason for this is that it is safe to report a
+ * quiescent state during context switch even though preemption
+ * is disabled. This function cannot be expected to understand these
+ * nuances, so the caller must handle them.
+ */
+static bool rcu_preempt_need_deferred_qs(struct task_struct *t)
+{
+ return (this_cpu_ptr(&rcu_data)->deferred_qs ||
+ READ_ONCE(t->rcu_read_unlock_special.s)) &&
+ t->rcu_read_lock_nesting <= 0;
+}
+
+/*
+ * Report a deferred quiescent state if needed and safe to do so.
+ * As with rcu_preempt_need_deferred_qs(), "safe" involves only
+ * not being in an RCU read-side critical section. The caller must
+ * evaluate safety in terms of interrupt, softirq, and preemption
+ * disabling.
+ */
+static void rcu_preempt_deferred_qs(struct task_struct *t)
+{
+ unsigned long flags;
+ bool couldrecurse = t->rcu_read_lock_nesting >= 0;
+
+ if (!rcu_preempt_need_deferred_qs(t))
+ return;
+ if (couldrecurse)
+ t->rcu_read_lock_nesting -= INT_MIN;
+ local_irq_save(flags);
+ rcu_preempt_deferred_qs_irqrestore(t, flags);
+ if (couldrecurse)
+ t->rcu_read_lock_nesting += INT_MIN;
+}
+
+/*
+ * Handle special cases during rcu_read_unlock(), such as needing to
+ * notify RCU core processing or task having blocked during the RCU
+ * read-side critical section.
+ */
+static void rcu_read_unlock_special(struct task_struct *t)
+{
+ unsigned long flags;
+ bool preempt_bh_were_disabled =
+ !!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK));
+ bool irqs_were_disabled;
+
+ /* NMI handlers cannot block and cannot safely manipulate state. */
+ if (in_nmi())
+ return;
+
+ local_irq_save(flags);
+ irqs_were_disabled = irqs_disabled_flags(flags);
+ if ((preempt_bh_were_disabled || irqs_were_disabled) &&
+ t->rcu_read_unlock_special.b.blocked) {
+ /* Need to defer quiescent state until everything is enabled. */
+ raise_softirq_irqoff(RCU_SOFTIRQ);
+ local_irq_restore(flags);
+ return;
+ }
+ rcu_preempt_deferred_qs_irqrestore(t, flags);
+}
+
/*
* Dump detailed information for all tasks blocking the current RCU
* grace period on the specified rcu_node structure.
* Dump detailed information for all tasks blocking the current RCU
* grace period.
*/
-static void rcu_print_detail_task_stall(struct rcu_state *rsp)
+static void rcu_print_detail_task_stall(void)
{
- struct rcu_node *rnp = rcu_get_root(rsp);
+ struct rcu_node *rnp = rcu_get_root();
rcu_print_detail_task_stall_rnp(rnp);
- rcu_for_each_leaf_node(rsp, rnp)
+ rcu_for_each_leaf_node(rnp)
rcu_print_detail_task_stall_rnp(rnp);
}
* Also, if there are blocked tasks on the list, they automatically
* block the newly created grace period, so set up ->gp_tasks accordingly.
*/
-static void
-rcu_preempt_check_blocked_tasks(struct rcu_state *rsp, struct rcu_node *rnp)
+static void rcu_preempt_check_blocked_tasks(struct rcu_node *rnp)
{
struct task_struct *t;
RCU_LOCKDEP_WARN(preemptible(), "rcu_preempt_check_blocked_tasks() invoked with preemption enabled!!!\n");
if (WARN_ON_ONCE(rcu_preempt_blocked_readers_cgp(rnp)))
- dump_blkd_tasks(rsp, rnp, 10);
+ dump_blkd_tasks(rnp, 10);
if (rcu_preempt_has_tasks(rnp) &&
(rnp->qsmaskinit || rnp->wait_blkd_tasks)) {
rnp->gp_tasks = rnp->blkd_tasks.next;
*
* Caller must disable hard irqs.
*/
-static void rcu_preempt_check_callbacks(void)
+static void rcu_flavor_check_callbacks(int user)
{
- struct rcu_state *rsp = &rcu_preempt_state;
struct task_struct *t = current;
- if (t->rcu_read_lock_nesting == 0) {
- rcu_preempt_qs();
+ if (user || rcu_is_cpu_rrupt_from_idle()) {
+ rcu_note_voluntary_context_switch(current);
+ }
+ if (t->rcu_read_lock_nesting > 0 ||
+ (preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK))) {
+ /* No QS, force context switch if deferred. */
+ if (rcu_preempt_need_deferred_qs(t)) {
+ set_tsk_need_resched(t);
+ set_preempt_need_resched();
+ }
+ } else if (rcu_preempt_need_deferred_qs(t)) {
+ rcu_preempt_deferred_qs(t); /* Report deferred QS. */
+ return;
+ } else if (!t->rcu_read_lock_nesting) {
+ rcu_qs(); /* Report immediate QS. */
return;
}
+
+ /* If GP is oldish, ask for help from rcu_read_unlock_special(). */
if (t->rcu_read_lock_nesting > 0 &&
- __this_cpu_read(rcu_data_p->core_needs_qs) &&
- __this_cpu_read(rcu_data_p->cpu_no_qs.b.norm) &&
+ __this_cpu_read(rcu_data.core_needs_qs) &&
+ __this_cpu_read(rcu_data.cpu_no_qs.b.norm) &&
!t->rcu_read_unlock_special.b.need_qs &&
- time_after(jiffies, rsp->gp_start + HZ))
+ time_after(jiffies, rcu_state.gp_start + HZ))
t->rcu_read_unlock_special.b.need_qs = true;
}
-/**
- * call_rcu() - Queue an RCU callback for invocation after a grace period.
- * @head: structure to be used for queueing the RCU updates.
- * @func: actual callback function to be invoked after the grace period
- *
- * The callback function will be invoked some time after a full grace
- * period elapses, in other words after all pre-existing RCU read-side
- * critical sections have completed. However, the callback function
- * might well execute concurrently with RCU read-side critical sections
- * that started after call_rcu() was invoked. RCU read-side critical
- * sections are delimited by rcu_read_lock() and rcu_read_unlock(),
- * and may be nested.
- *
- * Note that all CPUs must agree that the grace period extended beyond
- * all pre-existing RCU read-side critical section. On systems with more
- * than one CPU, this means that when "func()" is invoked, each CPU is
- * guaranteed to have executed a full memory barrier since the end of its
- * last RCU read-side critical section whose beginning preceded the call
- * to call_rcu(). It also means that each CPU executing an RCU read-side
- * critical section that continues beyond the start of "func()" must have
- * executed a memory barrier after the call_rcu() but before the beginning
- * of that RCU read-side critical section. Note that these guarantees
- * include CPUs that are offline, idle, or executing in user mode, as
- * well as CPUs that are executing in the kernel.
- *
- * Furthermore, if CPU A invoked call_rcu() and CPU B invoked the
- * resulting RCU callback function "func()", then both CPU A and CPU B are
- * guaranteed to execute a full memory barrier during the time interval
- * between the call to call_rcu() and the invocation of "func()" -- even
- * if CPU A and CPU B are the same CPU (but again only if the system has
- * more than one CPU).
- */
-void call_rcu(struct rcu_head *head, rcu_callback_t func)
-{
- __call_rcu(head, func, rcu_state_p, -1, 0);
-}
-EXPORT_SYMBOL_GPL(call_rcu);
-
/**
* synchronize_rcu - wait until a grace period has elapsed.
*
* concurrently with new RCU read-side critical sections that began while
* synchronize_rcu() was waiting. RCU read-side critical sections are
* delimited by rcu_read_lock() and rcu_read_unlock(), and may be nested.
+ * In addition, regions of code across which interrupts, preemption, or
+ * softirqs have been disabled also serve as RCU read-side critical
+ * sections. This includes hardware interrupt handlers, softirq handlers,
+ * and NMI handlers.
+ *
+ * Note that this guarantee implies further memory-ordering guarantees.
+ * On systems with more than one CPU, when synchronize_rcu() returns,
+ * each CPU is guaranteed to have executed a full memory barrier since
+ * the end of its last RCU read-side critical section whose beginning
+ * preceded the call to synchronize_rcu(). In addition, each CPU having
+ * an RCU read-side critical section that extends beyond the return from
+ * synchronize_rcu() is guaranteed to have executed a full memory barrier
+ * after the beginning of synchronize_rcu() and before the beginning of
+ * that RCU read-side critical section. Note that these guarantees include
+ * CPUs that are offline, idle, or executing in user mode, as well as CPUs
+ * that are executing in the kernel.
*
- * See the description of synchronize_sched() for more detailed
- * information on memory-ordering guarantees. However, please note
- * that -only- the memory-ordering guarantees apply. For example,
- * synchronize_rcu() is -not- guaranteed to wait on things like code
- * protected by preempt_disable(), instead, synchronize_rcu() is -only-
- * guaranteed to wait on RCU read-side critical sections, that is, sections
- * of code protected by rcu_read_lock().
+ * Furthermore, if CPU A invoked synchronize_rcu(), which returned
+ * to its caller on CPU B, then both CPU A and CPU B are guaranteed
+ * to have executed a full memory barrier during the execution of
+ * synchronize_rcu() -- even if CPU A and CPU B are the same CPU (but
+ * again only if the system has more than one CPU).
*/
void synchronize_rcu(void)
{
}
EXPORT_SYMBOL_GPL(synchronize_rcu);
-/**
- * rcu_barrier - Wait until all in-flight call_rcu() callbacks complete.
- *
- * Note that this primitive does not necessarily wait for an RCU grace period
- * to complete. For example, if there are no RCU callbacks queued anywhere
- * in the system, then rcu_barrier() is within its rights to return
- * immediately, without waiting for anything, much less an RCU grace period.
- */
-void rcu_barrier(void)
-{
- _rcu_barrier(rcu_state_p);
-}
-EXPORT_SYMBOL_GPL(rcu_barrier);
-
-/*
- * Initialize preemptible RCU's state structures.
- */
-static void __init __rcu_init_preempt(void)
-{
- rcu_init_one(rcu_state_p);
-}
-
/*
* Check for a task exiting while in a preemptible-RCU read-side
* critical section, clean up if so. No need to issue warnings,
barrier();
t->rcu_read_unlock_special.b.blocked = true;
__rcu_read_unlock();
+ rcu_preempt_deferred_qs(current);
}
/*
* specified number of elements.
*/
static void
-dump_blkd_tasks(struct rcu_state *rsp, struct rcu_node *rnp, int ncheck)
+dump_blkd_tasks(struct rcu_node *rnp, int ncheck)
{
int cpu;
int i;
}
pr_cont("\n");
for (cpu = rnp->grplo; cpu <= rnp->grphi; cpu++) {
- rdp = per_cpu_ptr(rsp->rda, cpu);
+ rdp = per_cpu_ptr(&rcu_data, cpu);
onl = !!(rdp->grpmask & rcu_rnp_online_cpus(rnp));
pr_info("\t%d: %c online: %ld(%d) offline: %ld(%d)\n",
cpu, ".o"[onl],
#else /* #ifdef CONFIG_PREEMPT_RCU */
-static struct rcu_state *const rcu_state_p = &rcu_sched_state;
-
/*
* Tell them what RCU they are running.
*/
}
/*
- * Because preemptible RCU does not exist, we never have to check for
- * CPUs being in quiescent states.
+ * Note a quiescent state for PREEMPT=n. Because we do not need to know
+ * how many quiescent states passed, just if there was at least one since
+ * the start of the grace period, this just sets a flag. The caller must
+ * have disabled preemption.
*/
-static void rcu_preempt_note_context_switch(bool preempt)
+static void rcu_qs(void)
{
+ RCU_LOCKDEP_WARN(preemptible(), "rcu_qs() invoked with preemption enabled!!!");
+ if (!__this_cpu_read(rcu_data.cpu_no_qs.s))
+ return;
+ trace_rcu_grace_period(TPS("rcu_sched"),
+ __this_cpu_read(rcu_data.gp_seq), TPS("cpuqs"));
+ __this_cpu_write(rcu_data.cpu_no_qs.b.norm, false);
+ if (!__this_cpu_read(rcu_data.cpu_no_qs.b.exp))
+ return;
+ __this_cpu_write(rcu_data.cpu_no_qs.b.exp, false);
+ rcu_report_exp_rdp(this_cpu_ptr(&rcu_data));
}
+/*
+ * Register an urgently needed quiescent state. If there is an
+ * emergency, invoke rcu_momentary_dyntick_idle() to do a heavy-weight
+ * dyntick-idle quiescent state visible to other CPUs, which will in
+ * some cases serve for expedited as well as normal grace periods.
+ * Either way, register a lightweight quiescent state.
+ *
+ * The barrier() calls are redundant in the common case when this is
+ * called externally, but just in case this is called from within this
+ * file.
+ *
+ */
+void rcu_all_qs(void)
+{
+ unsigned long flags;
+
+ if (!raw_cpu_read(rcu_data.rcu_urgent_qs))
+ return;
+ preempt_disable();
+ /* Load rcu_urgent_qs before other flags. */
+ if (!smp_load_acquire(this_cpu_ptr(&rcu_data.rcu_urgent_qs))) {
+ preempt_enable();
+ return;
+ }
+ this_cpu_write(rcu_data.rcu_urgent_qs, false);
+ barrier(); /* Avoid RCU read-side critical sections leaking down. */
+ if (unlikely(raw_cpu_read(rcu_data.rcu_need_heavy_qs))) {
+ local_irq_save(flags);
+ rcu_momentary_dyntick_idle();
+ local_irq_restore(flags);
+ }
+ rcu_qs();
+ barrier(); /* Avoid RCU read-side critical sections leaking up. */
+ preempt_enable();
+}
+EXPORT_SYMBOL_GPL(rcu_all_qs);
+
+/*
+ * Note a PREEMPT=n context switch. The caller must have disabled interrupts.
+ */
+void rcu_note_context_switch(bool preempt)
+{
+ barrier(); /* Avoid RCU read-side critical sections leaking down. */
+ trace_rcu_utilization(TPS("Start context switch"));
+ rcu_qs();
+ /* Load rcu_urgent_qs before other flags. */
+ if (!smp_load_acquire(this_cpu_ptr(&rcu_data.rcu_urgent_qs)))
+ goto out;
+ this_cpu_write(rcu_data.rcu_urgent_qs, false);
+ if (unlikely(raw_cpu_read(rcu_data.rcu_need_heavy_qs)))
+ rcu_momentary_dyntick_idle();
+ if (!preempt)
+ rcu_tasks_qs(current);
+out:
+ trace_rcu_utilization(TPS("End context switch"));
+ barrier(); /* Avoid RCU read-side critical sections leaking up. */
+}
+EXPORT_SYMBOL_GPL(rcu_note_context_switch);
+
/*
* Because preemptible RCU does not exist, there are never any preempted
* RCU readers.
return false;
}
+/*
+ * Because there is no preemptible RCU, there can be no deferred quiescent
+ * states.
+ */
+static bool rcu_preempt_need_deferred_qs(struct task_struct *t)
+{
+ return false;
+}
+static void rcu_preempt_deferred_qs(struct task_struct *t) { }
+
/*
* Because preemptible RCU does not exist, we never have to check for
* tasks blocked within RCU read-side critical sections.
*/
-static void rcu_print_detail_task_stall(struct rcu_state *rsp)
+static void rcu_print_detail_task_stall(void)
{
}
* so there is no need to check for blocked tasks. So check only for
* bogus qsmask values.
*/
-static void
-rcu_preempt_check_blocked_tasks(struct rcu_state *rsp, struct rcu_node *rnp)
+static void rcu_preempt_check_blocked_tasks(struct rcu_node *rnp)
{
WARN_ON_ONCE(rnp->qsmask);
}
/*
- * Because preemptible RCU does not exist, it never has any callbacks
- * to check.
+ * Check to see if this CPU is in a non-context-switch quiescent state
+ * (user mode or idle loop for rcu, non-softirq execution for rcu_bh).
+ * Also schedule RCU core processing.
+ *
+ * This function must be called from hardirq context. It is normally
+ * invoked from the scheduling-clock interrupt.
*/
-static void rcu_preempt_check_callbacks(void)
+static void rcu_flavor_check_callbacks(int user)
{
-}
+ if (user || rcu_is_cpu_rrupt_from_idle()) {
-/*
- * Because preemptible RCU does not exist, rcu_barrier() is just
- * another name for rcu_barrier_sched().
- */
-void rcu_barrier(void)
-{
- rcu_barrier_sched();
+ /*
+ * Get here if this CPU took its interrupt from user
+ * mode or from the idle loop, and if this is not a
+ * nested interrupt. In this case, the CPU is in
+ * a quiescent state, so note it.
+ *
+ * No memory barrier is required here because rcu_qs()
+ * references only CPU-local variables that other CPUs
+ * neither access nor modify, at least not while the
+ * corresponding CPU is online.
+ */
+
+ rcu_qs();
+ }
}
-EXPORT_SYMBOL_GPL(rcu_barrier);
-/*
- * Because preemptible RCU does not exist, it need not be initialized.
- */
-static void __init __rcu_init_preempt(void)
+/* PREEMPT=n implementation of synchronize_rcu(). */
+void synchronize_rcu(void)
{
+ RCU_LOCKDEP_WARN(lock_is_held(&rcu_bh_lock_map) ||
+ lock_is_held(&rcu_lock_map) ||
+ lock_is_held(&rcu_sched_lock_map),
+ "Illegal synchronize_rcu() in RCU read-side critical section");
+ if (rcu_blocking_is_gp())
+ return;
+ if (rcu_gp_is_expedited())
+ synchronize_rcu_expedited();
+ else
+ wait_rcu_gp(call_rcu);
}
+EXPORT_SYMBOL_GPL(synchronize_rcu);
/*
* Because preemptible RCU does not exist, tasks cannot possibly exit
* Dump the guaranteed-empty blocked-tasks state. Trust but verify.
*/
static void
-dump_blkd_tasks(struct rcu_state *rsp, struct rcu_node *rnp, int ncheck)
+dump_blkd_tasks(struct rcu_node *rnp, int ncheck)
{
WARN_ON_ONCE(!list_empty(&rnp->blkd_tasks));
}
* already exist. We only create this kthread for preemptible RCU.
* Returns zero if all is well, a negated errno otherwise.
*/
-static int rcu_spawn_one_boost_kthread(struct rcu_state *rsp,
- struct rcu_node *rnp)
+static int rcu_spawn_one_boost_kthread(struct rcu_node *rnp)
{
- int rnp_index = rnp - &rsp->node[0];
+ int rnp_index = rnp - rcu_get_root();
unsigned long flags;
struct sched_param sp;
struct task_struct *t;
- if (rcu_state_p != rsp)
+ if (!IS_ENABLED(CONFIG_PREEMPT_RCU))
return 0;
if (!rcu_scheduler_fully_active || rcu_rnp_online_cpus(rnp) == 0)
return 0;
- rsp->boost = 1;
+ rcu_state.boost = 1;
if (rnp->boost_kthread_task != NULL)
return 0;
t = kthread_create(rcu_boost_kthread, (void *)rnp,
static void rcu_kthread_do_work(void)
{
- rcu_do_batch(&rcu_sched_state, this_cpu_ptr(&rcu_sched_data));
- rcu_do_batch(&rcu_bh_state, this_cpu_ptr(&rcu_bh_data));
- rcu_do_batch(&rcu_preempt_state, this_cpu_ptr(&rcu_preempt_data));
+ rcu_do_batch(this_cpu_ptr(&rcu_data));
}
static void rcu_cpu_kthread_setup(unsigned int cpu)
}
/*
- * Per-CPU kernel thread that invokes RCU callbacks. This replaces the
- * RCU softirq used in flavors and configurations of RCU that do not
- * support RCU priority boosting.
+ * Per-CPU kernel thread that invokes RCU callbacks. This replaces
+ * the RCU softirq used in configurations of RCU that do not support RCU
+ * priority boosting.
*/
static void rcu_cpu_kthread(unsigned int cpu)
{
for_each_possible_cpu(cpu)
per_cpu(rcu_cpu_has_work, cpu) = 0;
BUG_ON(smpboot_register_percpu_thread(&rcu_cpu_thread_spec));
- rcu_for_each_leaf_node(rcu_state_p, rnp)
- (void)rcu_spawn_one_boost_kthread(rcu_state_p, rnp);
+ rcu_for_each_leaf_node(rnp)
+ (void)rcu_spawn_one_boost_kthread(rnp);
}
static void rcu_prepare_kthreads(int cpu)
{
- struct rcu_data *rdp = per_cpu_ptr(rcu_state_p->rda, cpu);
+ struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
struct rcu_node *rnp = rdp->mynode;
/* Fire up the incoming CPU's kthread and leaf rcu_node kthread. */
if (rcu_scheduler_fully_active)
- (void)rcu_spawn_one_boost_kthread(rcu_state_p, rnp);
+ (void)rcu_spawn_one_boost_kthread(rnp);
}
#else /* #ifdef CONFIG_RCU_BOOST */
* 1 if so. This function is part of the RCU implementation; it is -not-
* an exported member of the RCU API.
*
- * Because we not have RCU_FAST_NO_HZ, just check whether this CPU needs
- * any flavor of RCU.
+ * Because we not have RCU_FAST_NO_HZ, just check whether or not this
+ * CPU has RCU callbacks queued.
*/
int rcu_needs_cpu(u64 basemono, u64 *nextevt)
{
module_param(rcu_idle_lazy_gp_delay, int, 0644);
/*
- * Try to advance callbacks for all flavors of RCU on the current CPU, but
- * only if it has been awhile since the last time we did so. Afterwards,
- * if there are any callbacks ready for immediate invocation, return true.
+ * Try to advance callbacks on the current CPU, but only if it has been
+ * awhile since the last time we did so. Afterwards, if there are any
+ * callbacks ready for immediate invocation, return true.
*/
static bool __maybe_unused rcu_try_advance_all_cbs(void)
{
bool cbs_ready = false;
- struct rcu_data *rdp;
- struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
struct rcu_node *rnp;
- struct rcu_state *rsp;
/* Exit early if we advanced recently. */
- if (jiffies == rdtp->last_advance_all)
+ if (jiffies == rdp->last_advance_all)
return false;
- rdtp->last_advance_all = jiffies;
+ rdp->last_advance_all = jiffies;
- for_each_rcu_flavor(rsp) {
- rdp = this_cpu_ptr(rsp->rda);
- rnp = rdp->mynode;
+ rnp = rdp->mynode;
- /*
- * Don't bother checking unless a grace period has
- * completed since we last checked and there are
- * callbacks not yet ready to invoke.
- */
- if ((rcu_seq_completed_gp(rdp->gp_seq,
- rcu_seq_current(&rnp->gp_seq)) ||
- unlikely(READ_ONCE(rdp->gpwrap))) &&
- rcu_segcblist_pend_cbs(&rdp->cblist))
- note_gp_changes(rsp, rdp);
-
- if (rcu_segcblist_ready_cbs(&rdp->cblist))
- cbs_ready = true;
- }
+ /*
+ * Don't bother checking unless a grace period has
+ * completed since we last checked and there are
+ * callbacks not yet ready to invoke.
+ */
+ if ((rcu_seq_completed_gp(rdp->gp_seq,
+ rcu_seq_current(&rnp->gp_seq)) ||
+ unlikely(READ_ONCE(rdp->gpwrap))) &&
+ rcu_segcblist_pend_cbs(&rdp->cblist))
+ note_gp_changes(rdp);
+
+ if (rcu_segcblist_ready_cbs(&rdp->cblist))
+ cbs_ready = true;
return cbs_ready;
}
*/
int rcu_needs_cpu(u64 basemono, u64 *nextevt)
{
- struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
unsigned long dj;
lockdep_assert_irqs_disabled();
/* Snapshot to detect later posting of non-lazy callback. */
- rdtp->nonlazy_posted_snap = rdtp->nonlazy_posted;
+ rdp->nonlazy_posted_snap = rdp->nonlazy_posted;
/* If no callbacks, RCU doesn't need the CPU. */
- if (!rcu_cpu_has_callbacks(&rdtp->all_lazy)) {
+ if (!rcu_cpu_has_callbacks(&rdp->all_lazy)) {
*nextevt = KTIME_MAX;
return 0;
}
invoke_rcu_core();
return 1;
}
- rdtp->last_accelerate = jiffies;
+ rdp->last_accelerate = jiffies;
/* Request timer delay depending on laziness, and round. */
- if (!rdtp->all_lazy) {
+ if (!rdp->all_lazy) {
dj = round_up(rcu_idle_gp_delay + jiffies,
rcu_idle_gp_delay) - jiffies;
} else {
static void rcu_prepare_for_idle(void)
{
bool needwake;
- struct rcu_data *rdp;
- struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
+ struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
struct rcu_node *rnp;
- struct rcu_state *rsp;
int tne;
lockdep_assert_irqs_disabled();
/* Handle nohz enablement switches conservatively. */
tne = READ_ONCE(tick_nohz_active);
- if (tne != rdtp->tick_nohz_enabled_snap) {
+ if (tne != rdp->tick_nohz_enabled_snap) {
if (rcu_cpu_has_callbacks(NULL))
invoke_rcu_core(); /* force nohz to see update. */
- rdtp->tick_nohz_enabled_snap = tne;
+ rdp->tick_nohz_enabled_snap = tne;
return;
}
if (!tne)
* callbacks, invoke RCU core for the side-effect of recalculating
* idle duration on re-entry to idle.
*/
- if (rdtp->all_lazy &&
- rdtp->nonlazy_posted != rdtp->nonlazy_posted_snap) {
- rdtp->all_lazy = false;
- rdtp->nonlazy_posted_snap = rdtp->nonlazy_posted;
+ if (rdp->all_lazy &&
+ rdp->nonlazy_posted != rdp->nonlazy_posted_snap) {
+ rdp->all_lazy = false;
+ rdp->nonlazy_posted_snap = rdp->nonlazy_posted;
invoke_rcu_core();
return;
}
* If we have not yet accelerated this jiffy, accelerate all
* callbacks on this CPU.
*/
- if (rdtp->last_accelerate == jiffies)
+ if (rdp->last_accelerate == jiffies)
return;
- rdtp->last_accelerate = jiffies;
- for_each_rcu_flavor(rsp) {
- rdp = this_cpu_ptr(rsp->rda);
- if (!rcu_segcblist_pend_cbs(&rdp->cblist))
- continue;
+ rdp->last_accelerate = jiffies;
+ if (rcu_segcblist_pend_cbs(&rdp->cblist)) {
rnp = rdp->mynode;
raw_spin_lock_rcu_node(rnp); /* irqs already disabled. */
- needwake = rcu_accelerate_cbs(rsp, rnp, rdp);
+ needwake = rcu_accelerate_cbs(rnp, rdp);
raw_spin_unlock_rcu_node(rnp); /* irqs remain disabled. */
if (needwake)
- rcu_gp_kthread_wake(rsp);
+ rcu_gp_kthread_wake();
}
}
*/
static void rcu_idle_count_callbacks_posted(void)
{
- __this_cpu_add(rcu_dynticks.nonlazy_posted, 1);
-}
-
-/*
- * Data for flushing lazy RCU callbacks at OOM time.
- */
-static atomic_t oom_callback_count;
-static DECLARE_WAIT_QUEUE_HEAD(oom_callback_wq);
-
-/*
- * RCU OOM callback -- decrement the outstanding count and deliver the
- * wake-up if we are the last one.
- */
-static void rcu_oom_callback(struct rcu_head *rhp)
-{
- if (atomic_dec_and_test(&oom_callback_count))
- wake_up(&oom_callback_wq);
-}
-
-/*
- * Post an rcu_oom_notify callback on the current CPU if it has at
- * least one lazy callback. This will unnecessarily post callbacks
- * to CPUs that already have a non-lazy callback at the end of their
- * callback list, but this is an infrequent operation, so accept some
- * extra overhead to keep things simple.
- */
-static void rcu_oom_notify_cpu(void *unused)
-{
- struct rcu_state *rsp;
- struct rcu_data *rdp;
-
- for_each_rcu_flavor(rsp) {
- rdp = raw_cpu_ptr(rsp->rda);
- if (rcu_segcblist_n_lazy_cbs(&rdp->cblist)) {
- atomic_inc(&oom_callback_count);
- rsp->call(&rdp->oom_head, rcu_oom_callback);
- }
- }
-}
-
-/*
- * If low on memory, ensure that each CPU has a non-lazy callback.
- * This will wake up CPUs that have only lazy callbacks, in turn
- * ensuring that they free up the corresponding memory in a timely manner.
- * Because an uncertain amount of memory will be freed in some uncertain
- * timeframe, we do not claim to have freed anything.
- */
-static int rcu_oom_notify(struct notifier_block *self,
- unsigned long notused, void *nfreed)
-{
- int cpu;
-
- /* Wait for callbacks from earlier instance to complete. */
- wait_event(oom_callback_wq, atomic_read(&oom_callback_count) == 0);
- smp_mb(); /* Ensure callback reuse happens after callback invocation. */
-
- /*
- * Prevent premature wakeup: ensure that all increments happen
- * before there is a chance of the counter reaching zero.
- */
- atomic_set(&oom_callback_count, 1);
-
- for_each_online_cpu(cpu) {
- smp_call_function_single(cpu, rcu_oom_notify_cpu, NULL, 1);
- cond_resched_tasks_rcu_qs();
- }
-
- /* Unconditionally decrement: no need to wake ourselves up. */
- atomic_dec(&oom_callback_count);
-
- return NOTIFY_OK;
+ __this_cpu_add(rcu_data.nonlazy_posted, 1);
}
-static struct notifier_block rcu_oom_nb = {
- .notifier_call = rcu_oom_notify
-};
-
-static int __init rcu_register_oom_notifier(void)
-{
- register_oom_notifier(&rcu_oom_nb);
- return 0;
-}
-early_initcall(rcu_register_oom_notifier);
-
#endif /* #else #if !defined(CONFIG_RCU_FAST_NO_HZ) */
#ifdef CONFIG_RCU_FAST_NO_HZ
static void print_cpu_stall_fast_no_hz(char *cp, int cpu)
{
- struct rcu_dynticks *rdtp = &per_cpu(rcu_dynticks, cpu);
- unsigned long nlpd = rdtp->nonlazy_posted - rdtp->nonlazy_posted_snap;
+ struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
+ unsigned long nlpd = rdp->nonlazy_posted - rdp->nonlazy_posted_snap;
sprintf(cp, "last_accelerate: %04lx/%04lx, nonlazy_posted: %ld, %c%c",
- rdtp->last_accelerate & 0xffff, jiffies & 0xffff,
+ rdp->last_accelerate & 0xffff, jiffies & 0xffff,
ulong2long(nlpd),
- rdtp->all_lazy ? 'L' : '.',
- rdtp->tick_nohz_enabled_snap ? '.' : 'D');
+ rdp->all_lazy ? 'L' : '.',
+ rdp->tick_nohz_enabled_snap ? '.' : 'D');
}
#else /* #ifdef CONFIG_RCU_FAST_NO_HZ */
/*
* Print out diagnostic information for the specified stalled CPU.
*
- * If the specified CPU is aware of the current RCU grace period
- * (flavor specified by rsp), then print the number of scheduling
- * clock interrupts the CPU has taken during the time that it has
- * been aware. Otherwise, print the number of RCU grace periods
- * that this CPU is ignorant of, for example, "1" if the CPU was
- * aware of the previous grace period.
+ * If the specified CPU is aware of the current RCU grace period, then
+ * print the number of scheduling clock interrupts the CPU has taken
+ * during the time that it has been aware. Otherwise, print the number
+ * of RCU grace periods that this CPU is ignorant of, for example, "1"
+ * if the CPU was aware of the previous grace period.
*
* Also print out idle and (if CONFIG_RCU_FAST_NO_HZ) idle-entry info.
*/
-static void print_cpu_stall_info(struct rcu_state *rsp, int cpu)
+static void print_cpu_stall_info(int cpu)
{
unsigned long delta;
char fast_no_hz[72];
- struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
- struct rcu_dynticks *rdtp = rdp->dynticks;
+ struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
char *ticks_title;
unsigned long ticks_value;
*/
touch_nmi_watchdog();
- ticks_value = rcu_seq_ctr(rsp->gp_seq - rdp->gp_seq);
+ ticks_value = rcu_seq_ctr(rcu_state.gp_seq - rdp->gp_seq);
if (ticks_value) {
ticks_title = "GPs behind";
} else {
rdp->rcu_iw_pending ? (int)min(delta, 9UL) + '0' :
"!."[!delta],
ticks_value, ticks_title,
- rcu_dynticks_snap(rdtp) & 0xfff,
- rdtp->dynticks_nesting, rdtp->dynticks_nmi_nesting,
+ rcu_dynticks_snap(rdp) & 0xfff,
+ rdp->dynticks_nesting, rdp->dynticks_nmi_nesting,
rdp->softirq_snap, kstat_softirqs_cpu(RCU_SOFTIRQ, cpu),
- READ_ONCE(rsp->n_force_qs) - rsp->n_force_qs_gpstart,
+ READ_ONCE(rcu_state.n_force_qs) - rcu_state.n_force_qs_gpstart,
fast_no_hz);
}
pr_err("\t");
}
-/* Zero ->ticks_this_gp for all flavors of RCU. */
+/* Zero ->ticks_this_gp and snapshot the number of RCU softirq handlers. */
static void zero_cpu_stall_ticks(struct rcu_data *rdp)
{
rdp->ticks_this_gp = 0;
rdp->softirq_snap = kstat_softirqs_cpu(RCU_SOFTIRQ, smp_processor_id());
-}
-
-/* Increment ->ticks_this_gp for all flavors of RCU. */
-static void increment_cpu_stall_ticks(void)
-{
- struct rcu_state *rsp;
-
- for_each_rcu_flavor(rsp)
- raw_cpu_inc(rsp->rda->ticks_this_gp);
+ WRITE_ONCE(rdp->last_fqs_resched, jiffies);
}
#ifdef CONFIG_RCU_NOCB_CPU
if (rdp->nocb_defer_wakeup == RCU_NOCB_WAKE_NOT)
mod_timer(&rdp->nocb_timer, jiffies + 1);
WRITE_ONCE(rdp->nocb_defer_wakeup, waketype);
- trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu, reason);
+ trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, reason);
raw_spin_unlock_irqrestore(&rdp->nocb_lock, flags);
}
/*
- * Does the specified CPU need an RCU callback for the specified flavor
+ * Does the specified CPU need an RCU callback for this invocation
* of rcu_barrier()?
*/
-static bool rcu_nocb_cpu_needs_barrier(struct rcu_state *rsp, int cpu)
+static bool rcu_nocb_cpu_needs_barrier(int cpu)
{
- struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
+ struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
unsigned long ret;
#ifdef CONFIG_PROVE_RCU
struct rcu_head *rhp;
* There needs to be a barrier before this function is called,
* but associated with a prior determination that no more
* callbacks would be posted. In the worst case, the first
- * barrier in _rcu_barrier() suffices (but the caller cannot
+ * barrier in rcu_barrier() suffices (but the caller cannot
* necessarily rely on this, not a substitute for the caller
* getting the concurrency design right!). There must also be
* a barrier between the following load an posting of a callback
/* If we are not being polled and there is a kthread, awaken it ... */
t = READ_ONCE(rdp->nocb_kthread);
if (rcu_nocb_poll || !t) {
- trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu,
+ trace_rcu_nocb_wake(rcu_state.name, rdp->cpu,
TPS("WakeNotPoll"));
return;
}
if (!irqs_disabled_flags(flags)) {
/* ... if queue was empty ... */
wake_nocb_leader(rdp, false);
- trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu,
+ trace_rcu_nocb_wake(rcu_state.name, rdp->cpu,
TPS("WakeEmpty"));
} else {
wake_nocb_leader_defer(rdp, RCU_NOCB_WAKE,
/* ... or if many callbacks queued. */
if (!irqs_disabled_flags(flags)) {
wake_nocb_leader(rdp, true);
- trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu,
+ trace_rcu_nocb_wake(rcu_state.name, rdp->cpu,
TPS("WakeOvf"));
} else {
wake_nocb_leader_defer(rdp, RCU_NOCB_WAKE_FORCE,
}
rdp->qlen_last_fqs_check = LONG_MAX / 2;
} else {
- trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu, TPS("WakeNot"));
+ trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("WakeNot"));
}
return;
}
return false;
__call_rcu_nocb_enqueue(rdp, rhp, &rhp->next, 1, lazy, flags);
if (__is_kfree_rcu_offset((unsigned long)rhp->func))
- trace_rcu_kfree_callback(rdp->rsp->name, rhp,
+ trace_rcu_kfree_callback(rcu_state.name, rhp,
(unsigned long)rhp->func,
-atomic_long_read(&rdp->nocb_q_count_lazy),
-atomic_long_read(&rdp->nocb_q_count));
else
- trace_rcu_callback(rdp->rsp->name, rhp,
+ trace_rcu_callback(rcu_state.name, rhp,
-atomic_long_read(&rdp->nocb_q_count_lazy),
-atomic_long_read(&rdp->nocb_q_count));
struct rcu_node *rnp = rdp->mynode;
local_irq_save(flags);
- c = rcu_seq_snap(&rdp->rsp->gp_seq);
+ c = rcu_seq_snap(&rcu_state.gp_seq);
if (!rdp->gpwrap && ULONG_CMP_GE(rdp->gp_seq_needed, c)) {
local_irq_restore(flags);
} else {
needwake = rcu_start_this_gp(rnp, rdp, c);
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
if (needwake)
- rcu_gp_kthread_wake(rdp->rsp);
+ rcu_gp_kthread_wake();
}
/*
/* Wait for callbacks to appear. */
if (!rcu_nocb_poll) {
- trace_rcu_nocb_wake(my_rdp->rsp->name, my_rdp->cpu, TPS("Sleep"));
+ trace_rcu_nocb_wake(rcu_state.name, my_rdp->cpu, TPS("Sleep"));
swait_event_interruptible_exclusive(my_rdp->nocb_wq,
!READ_ONCE(my_rdp->nocb_leader_sleep));
raw_spin_lock_irqsave(&my_rdp->nocb_lock, flags);
raw_spin_unlock_irqrestore(&my_rdp->nocb_lock, flags);
} else if (firsttime) {
firsttime = false; /* Don't drown trace log with "Poll"! */
- trace_rcu_nocb_wake(my_rdp->rsp->name, my_rdp->cpu, TPS("Poll"));
+ trace_rcu_nocb_wake(rcu_state.name, my_rdp->cpu, TPS("Poll"));
}
/*
if (rcu_nocb_poll) {
schedule_timeout_interruptible(1);
} else {
- trace_rcu_nocb_wake(my_rdp->rsp->name, my_rdp->cpu,
+ trace_rcu_nocb_wake(rcu_state.name, my_rdp->cpu,
TPS("WokeEmpty"));
}
goto wait_again;
static void nocb_follower_wait(struct rcu_data *rdp)
{
for (;;) {
- trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu, TPS("FollowerSleep"));
+ trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("FollowerSleep"));
swait_event_interruptible_exclusive(rdp->nocb_wq,
READ_ONCE(rdp->nocb_follower_head));
if (smp_load_acquire(&rdp->nocb_follower_head)) {
return;
}
WARN_ON(signal_pending(current));
- trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu, TPS("WokeEmpty"));
+ trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("WokeEmpty"));
}
}
rdp->nocb_follower_tail = &rdp->nocb_follower_head;
raw_spin_unlock_irqrestore(&rdp->nocb_lock, flags);
BUG_ON(!list);
- trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu, TPS("WokeNonEmpty"));
+ trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("WokeNonEmpty"));
/* Each pass through the following loop invokes a callback. */
- trace_rcu_batch_start(rdp->rsp->name,
+ trace_rcu_batch_start(rcu_state.name,
atomic_long_read(&rdp->nocb_q_count_lazy),
atomic_long_read(&rdp->nocb_q_count), -1);
c = cl = 0;
next = list->next;
/* Wait for enqueuing to complete, if needed. */
while (next == NULL && &list->next != tail) {
- trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu,
+ trace_rcu_nocb_wake(rcu_state.name, rdp->cpu,
TPS("WaitQueue"));
schedule_timeout_interruptible(1);
- trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu,
+ trace_rcu_nocb_wake(rcu_state.name, rdp->cpu,
TPS("WokeQueue"));
next = list->next;
}
debug_rcu_head_unqueue(list);
local_bh_disable();
- if (__rcu_reclaim(rdp->rsp->name, list))
+ if (__rcu_reclaim(rcu_state.name, list))
cl++;
c++;
local_bh_enable();
cond_resched_tasks_rcu_qs();
list = next;
}
- trace_rcu_batch_end(rdp->rsp->name, c, !!list, 0, 0, 1);
+ trace_rcu_batch_end(rcu_state.name, c, !!list, 0, 0, 1);
smp_mb__before_atomic(); /* _add after CB invocation. */
atomic_long_add(-c, &rdp->nocb_q_count);
atomic_long_add(-cl, &rdp->nocb_q_count_lazy);
ndw = READ_ONCE(rdp->nocb_defer_wakeup);
WRITE_ONCE(rdp->nocb_defer_wakeup, RCU_NOCB_WAKE_NOT);
__wake_nocb_leader(rdp, ndw == RCU_NOCB_WAKE_FORCE, flags);
- trace_rcu_nocb_wake(rdp->rsp->name, rdp->cpu, TPS("DeferredWake"));
+ trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("DeferredWake"));
}
/* Do a deferred wakeup of rcu_nocb_kthread() from a timer handler. */
{
int cpu;
bool need_rcu_nocb_mask = false;
- struct rcu_state *rsp;
#if defined(CONFIG_NO_HZ_FULL)
if (tick_nohz_full_running && cpumask_weight(tick_nohz_full_mask))
if (rcu_nocb_poll)
pr_info("\tPoll for callbacks from no-CBs CPUs.\n");
- for_each_rcu_flavor(rsp) {
- for_each_cpu(cpu, rcu_nocb_mask)
- init_nocb_callback_list(per_cpu_ptr(rsp->rda, cpu));
- rcu_organize_nocb_kthreads(rsp);
- }
+ for_each_cpu(cpu, rcu_nocb_mask)
+ init_nocb_callback_list(per_cpu_ptr(&rcu_data, cpu));
+ rcu_organize_nocb_kthreads();
}
/* Initialize per-rcu_data variables for no-CBs CPUs. */
/*
* If the specified CPU is a no-CBs CPU that does not already have its
- * rcuo kthread for the specified RCU flavor, spawn it. If the CPUs are
- * brought online out of order, this can require re-organizing the
- * leader-follower relationships.
+ * rcuo kthread, spawn it. If the CPUs are brought online out of order,
+ * this can require re-organizing the leader-follower relationships.
*/
-static void rcu_spawn_one_nocb_kthread(struct rcu_state *rsp, int cpu)
+static void rcu_spawn_one_nocb_kthread(int cpu)
{
struct rcu_data *rdp;
struct rcu_data *rdp_last;
struct rcu_data *rdp_old_leader;
- struct rcu_data *rdp_spawn = per_cpu_ptr(rsp->rda, cpu);
+ struct rcu_data *rdp_spawn = per_cpu_ptr(&rcu_data, cpu);
struct task_struct *t;
/*
rdp_spawn->nocb_next_follower = rdp_old_leader;
}
- /* Spawn the kthread for this CPU and RCU flavor. */
+ /* Spawn the kthread for this CPU. */
t = kthread_run(rcu_nocb_kthread, rdp_spawn,
- "rcuo%c/%d", rsp->abbr, cpu);
+ "rcuo%c/%d", rcu_state.abbr, cpu);
BUG_ON(IS_ERR(t));
WRITE_ONCE(rdp_spawn->nocb_kthread, t);
}
*/
static void rcu_spawn_all_nocb_kthreads(int cpu)
{
- struct rcu_state *rsp;
-
if (rcu_scheduler_fully_active)
- for_each_rcu_flavor(rsp)
- rcu_spawn_one_nocb_kthread(rsp, cpu);
+ rcu_spawn_one_nocb_kthread(cpu);
}
/*
/*
* Initialize leader-follower relationships for all no-CBs CPU.
*/
-static void __init rcu_organize_nocb_kthreads(struct rcu_state *rsp)
+static void __init rcu_organize_nocb_kthreads(void)
{
int cpu;
int ls = rcu_nocb_leader_stride;
* we will spawn the needed set of rcu_nocb_kthread() kthreads.
*/
for_each_cpu(cpu, rcu_nocb_mask) {
- rdp = per_cpu_ptr(rsp->rda, cpu);
+ rdp = per_cpu_ptr(&rcu_data, cpu);
if (rdp->cpu >= nl) {
/* New leader, set up for followers & next leader. */
nl = DIV_ROUND_UP(rdp->cpu + 1, ls) * ls;
#else /* #ifdef CONFIG_RCU_NOCB_CPU */
-static bool rcu_nocb_cpu_needs_barrier(struct rcu_state *rsp, int cpu)
+static bool rcu_nocb_cpu_needs_barrier(int cpu)
{
WARN_ON_ONCE(1); /* Should be dead code. */
return false;
* This code relies on the fact that all NO_HZ_FULL CPUs are also
* CONFIG_RCU_NOCB_CPU CPUs.
*/
-static bool rcu_nohz_full_cpu(struct rcu_state *rsp)
+static bool rcu_nohz_full_cpu(void)
{
#ifdef CONFIG_NO_HZ_FULL
if (tick_nohz_full_cpu(smp_processor_id()) &&
- (!rcu_gp_in_progress(rsp) ||
- ULONG_CMP_LT(jiffies, READ_ONCE(rsp->gp_start) + HZ)))
+ (!rcu_gp_in_progress() ||
+ ULONG_CMP_LT(jiffies, READ_ONCE(rcu_state.gp_start) + HZ)))
return true;
#endif /* #ifdef CONFIG_NO_HZ_FULL */
return false;
if (!IS_ENABLED(CONFIG_PROVE_RCU))
return;
synchronize_rcu();
- synchronize_rcu_bh();
- synchronize_sched();
synchronize_rcu_expedited();
- synchronize_rcu_bh_expedited();
- synchronize_sched_expedited();
}
#if !defined(CONFIG_TINY_RCU) || defined(CONFIG_SRCU)
*
* Check debug_lockdep_rcu_enabled() to prevent false positives during boot.
*
- * Note that rcu_read_lock() is disallowed if the CPU is either idle or
+ * Note that rcu_read_lock_bh() is disallowed if the CPU is either idle or
* offline from an RCU perspective, so check for those as well.
*/
int rcu_read_lock_bh_held(void)
int i;
int j;
- /* Initialize and register callbacks for each flavor specified. */
+ /* Initialize and register callbacks for each crcu_array element. */
for (i = 0; i < n; i++) {
if (checktiny &&
(crcu_array[i] == call_rcu ||
}
return till_stall_check * HZ + RCU_STALL_DELAY_DELTA;
}
+EXPORT_SYMBOL_GPL(rcu_jiffies_till_stall_check);
void rcu_sysrq_start(void)
{
/*
* Wait for all pre-existing t->on_rq and t->nvcsw
- * transitions to complete. Invoking synchronize_sched()
+ * transitions to complete. Invoking synchronize_rcu()
* suffices because all these transitions occur with
- * interrupts disabled. Without this synchronize_sched(),
+ * interrupts disabled. Without this synchronize_rcu(),
* a read-side critical section that started before the
* grace period might be incorrectly seen as having started
* after the grace period.
*
- * This synchronize_sched() also dispenses with the
+ * This synchronize_rcu() also dispenses with the
* need for a memory barrier on the first store to
* ->rcu_tasks_holdout, as it forces the store to happen
* after the beginning of the grace period.
*/
- synchronize_sched();
+ synchronize_rcu();
/*
* There were callbacks, so we need to wait for an
* This does only part of the job, ensuring that all
* tasks that were previously exiting reach the point
* where they have disabled preemption, allowing the
- * later synchronize_sched() to finish the job.
+ * later synchronize_rcu() to finish the job.
*/
synchronize_srcu(&tasks_rcu_exit_srcu);
* cause their RCU-tasks read-side critical sections to
* extend past the end of the grace period. However,
* because these ->nvcsw updates are carried out with
- * interrupts disabled, we can use synchronize_sched()
+ * interrupts disabled, we can use synchronize_rcu()
* to force the needed ordering on all such CPUs.
*
- * This synchronize_sched() also confines all
+ * This synchronize_rcu() also confines all
* ->rcu_tasks_holdout accesses to be within the grace
* period, avoiding the need for memory barriers for
* ->rcu_tasks_holdout accesses.
*
- * In addition, this synchronize_sched() waits for exiting
+ * In addition, this synchronize_rcu() waits for exiting
* tasks to complete their final preempt_disable() region
* of execution, cleaning up after the synchronize_srcu()
* above.
*/
- synchronize_sched();
+ synchronize_rcu();
/* Invoke the callbacks. */
while (list) {
#ifdef CONFIG_PROVE_RCU
/*
- * Early boot self test parameters, one for each flavor
+ * Early boot self test parameters.
*/
static bool rcu_self_test;
-static bool rcu_self_test_bh;
-static bool rcu_self_test_sched;
-
module_param(rcu_self_test, bool, 0444);
-module_param(rcu_self_test_bh, bool, 0444);
-module_param(rcu_self_test_sched, bool, 0444);
static int rcu_self_test_counter;
pr_info("RCU test callback executed %d\n", rcu_self_test_counter);
}
+DEFINE_STATIC_SRCU(early_srcu);
+
static void early_boot_test_call_rcu(void)
{
static struct rcu_head head;
+ static struct rcu_head shead;
call_rcu(&head, test_callback);
-}
-
-static void early_boot_test_call_rcu_bh(void)
-{
- static struct rcu_head head;
-
- call_rcu_bh(&head, test_callback);
-}
-
-static void early_boot_test_call_rcu_sched(void)
-{
- static struct rcu_head head;
-
- call_rcu_sched(&head, test_callback);
+ if (IS_ENABLED(CONFIG_SRCU))
+ call_srcu(&early_srcu, &shead, test_callback);
}
void rcu_early_boot_tests(void)
if (rcu_self_test)
early_boot_test_call_rcu();
- if (rcu_self_test_bh)
- early_boot_test_call_rcu_bh();
- if (rcu_self_test_sched)
- early_boot_test_call_rcu_sched();
rcu_test_sync_prims();
}
if (rcu_self_test) {
early_boot_test_counter++;
rcu_barrier();
+ if (IS_ENABLED(CONFIG_SRCU)) {
+ early_boot_test_counter++;
+ srcu_barrier(&early_srcu);
+ }
}
- if (rcu_self_test_bh) {
- early_boot_test_counter++;
- rcu_barrier_bh();
- }
- if (rcu_self_test_sched) {
- early_boot_test_counter++;
- rcu_barrier_sched();
- }
-
if (rcu_self_test_counter != early_boot_test_counter) {
WARN_ON(1);
ret = -1;
*/
void (*pm_power_off_prepare)(void);
+EXPORT_SYMBOL_GPL(pm_power_off_prepare);
/**
* emergency_restart - reboot the system
* In theory, the compile should just see 0 here, and optimize out the call
* to sched_rt_avg_update. But I don't trust it...
*/
-#if defined(CONFIG_IRQ_TIME_ACCOUNTING) || defined(CONFIG_PARAVIRT_TIME_ACCOUNTING)
- s64 steal = 0, irq_delta = 0;
-#endif
+ s64 __maybe_unused steal = 0, irq_delta = 0;
+
#ifdef CONFIG_IRQ_TIME_ACCOUNTING
irq_delta = irq_time_read(cpu_of(rq)) - rq->prev_irq_time;
rq->clock_task += delta;
-#ifdef HAVE_SCHED_AVG_IRQ
+#ifdef CONFIG_HAVE_SCHED_AVG_IRQ
if ((irq_delta + steal) && sched_feat(NONTASK_CAPACITY))
update_irq_load_avg(rq, irq_delta + steal);
#endif
if (idle_policy(p->policy)) {
load->weight = scale_load(WEIGHT_IDLEPRIO);
load->inv_weight = WMULT_IDLEPRIO;
+ p->se.runnable_weight = load->weight;
return;
}
} else {
load->weight = scale_load(sched_prio_to_weight[prio]);
load->inv_weight = sched_prio_to_wmult[prio];
+ p->se.runnable_weight = load->weight;
}
}
}
/*
- * Consumers of these two interfaces, like for example the cpufreq menu
- * governor are using nonsensical data. Boosting frequency for a CPU that has
- * IO-wait which might not even end up running the task when it does become
- * runnable.
+ * Consumers of these two interfaces, like for example the cpuidle menu
+ * governor, are using nonsensical data. Preferring shallow idle state selection
+ * for a CPU that has IO-wait which might not even end up running the task when
+ * it does become runnable.
*/
unsigned long nr_iowait_cpu(int cpu)
static int select_idle_sibling(struct task_struct *p, int prev_cpu, int cpu);
static unsigned long task_h_load(struct task_struct *p);
+static unsigned long capacity_of(int cpu);
/* Give new sched_entity start runnable values to heavy its load in infant time */
void init_entity_runnable_average(struct sched_entity *se)
static unsigned long weighted_cpuload(struct rq *rq);
static unsigned long source_load(int cpu, int type);
static unsigned long target_load(int cpu, int type);
-static unsigned long capacity_of(int cpu);
/* Cached statistics for all CPUs within a node */
struct numa_stats {
/* Total compute capacity of CPUs on a node */
unsigned long compute_capacity;
-
- unsigned int nr_running;
};
/*
*/
static void update_numa_stats(struct numa_stats *ns, int nid)
{
- int smt, cpu, cpus = 0;
- unsigned long capacity;
+ int cpu;
memset(ns, 0, sizeof(*ns));
for_each_cpu(cpu, cpumask_of_node(nid)) {
struct rq *rq = cpu_rq(cpu);
- ns->nr_running += rq->nr_running;
ns->load += weighted_cpuload(rq);
ns->compute_capacity += capacity_of(cpu);
-
- cpus++;
}
- /*
- * If we raced with hotplug and there are no CPUs left in our mask
- * the @ns structure is NULL'ed and task_numa_compare() will
- * not find this node attractive.
- *
- * We'll detect a huge imbalance and bail there.
- */
- if (!cpus)
- return;
-
- /* smt := ceil(cpus / capacity), assumes: 1 < smt_power < 2 */
- smt = DIV_ROUND_UP(SCHED_CAPACITY_SCALE * cpus, ns->compute_capacity);
- capacity = cpus / smt; /* cores */
-
- capacity = min_t(unsigned, capacity,
- DIV_ROUND_CLOSEST(ns->compute_capacity, SCHED_CAPACITY_SCALE));
}
struct task_numa_env {
WRITE_ONCE(p->se.avg.util_est, ue);
}
+static inline int task_fits_capacity(struct task_struct *p, long capacity)
+{
+ return capacity * 1024 > task_util_est(p) * capacity_margin;
+}
+
+static inline void update_misfit_status(struct task_struct *p, struct rq *rq)
+{
+ if (!static_branch_unlikely(&sched_asym_cpucapacity))
+ return;
+
+ if (!p) {
+ rq->misfit_task_load = 0;
+ return;
+ }
+
+ if (task_fits_capacity(p, capacity_of(cpu_of(rq)))) {
+ rq->misfit_task_load = 0;
+ return;
+ }
+
+ rq->misfit_task_load = task_h_load(p);
+}
+
#else /* CONFIG_SMP */
#define UPDATE_TG 0x0
static inline void
util_est_dequeue(struct cfs_rq *cfs_rq, struct task_struct *p,
bool task_sleep) {}
+static inline void update_misfit_status(struct task_struct *p, struct rq *rq) {}
#endif /* CONFIG_SMP */
* put back on, and if we advance min_vruntime, we'll be placed back
* further than we started -- ie. we'll be penalized.
*/
- if ((flags & (DEQUEUE_SAVE | DEQUEUE_MOVE)) == DEQUEUE_SAVE)
+ if ((flags & (DEQUEUE_SAVE | DEQUEUE_MOVE)) != DEQUEUE_SAVE)
update_min_vruntime(cfs_rq);
}
/*
* Add to the _head_ of the list, so that an already-started
- * distribute_cfs_runtime will not see us
+ * distribute_cfs_runtime will not see us. If disribute_cfs_runtime is
+ * not running add to the tail so that later runqueues don't get starved.
*/
- list_add_rcu(&cfs_rq->throttled_list, &cfs_b->throttled_cfs_rq);
+ if (cfs_b->distribute_running)
+ list_add_rcu(&cfs_rq->throttled_list, &cfs_b->throttled_cfs_rq);
+ else
+ list_add_tail_rcu(&cfs_rq->throttled_list, &cfs_b->throttled_cfs_rq);
/*
* If we're the first throttled task, make sure the bandwidth
* in us over-using our runtime if it is all used during this loop, but
* only by limited amounts in that extreme case.
*/
- while (throttled && cfs_b->runtime > 0) {
+ while (throttled && cfs_b->runtime > 0 && !cfs_b->distribute_running) {
runtime = cfs_b->runtime;
+ cfs_b->distribute_running = 1;
raw_spin_unlock(&cfs_b->lock);
/* we can't nest cfs_b->lock while distributing bandwidth */
runtime = distribute_cfs_runtime(cfs_b, runtime,
runtime_expires);
raw_spin_lock(&cfs_b->lock);
+ cfs_b->distribute_running = 0;
throttled = !list_empty(&cfs_b->throttled_cfs_rq);
cfs_b->runtime -= min(runtime, cfs_b->runtime);
/* confirm we're still not at a refresh boundary */
raw_spin_lock(&cfs_b->lock);
+ if (cfs_b->distribute_running) {
+ raw_spin_unlock(&cfs_b->lock);
+ return;
+ }
+
if (runtime_refresh_within(cfs_b, min_bandwidth_expiration)) {
raw_spin_unlock(&cfs_b->lock);
return;
runtime = cfs_b->runtime;
expires = cfs_b->runtime_expires;
+ if (runtime)
+ cfs_b->distribute_running = 1;
+
raw_spin_unlock(&cfs_b->lock);
if (!runtime)
raw_spin_lock(&cfs_b->lock);
if (expires == cfs_b->runtime_expires)
cfs_b->runtime -= min(runtime, cfs_b->runtime);
+ cfs_b->distribute_running = 0;
raw_spin_unlock(&cfs_b->lock);
}
cfs_b->period_timer.function = sched_cfs_period_timer;
hrtimer_init(&cfs_b->slack_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
cfs_b->slack_timer.function = sched_cfs_slack_timer;
+ cfs_b->distribute_running = 0;
}
static void init_cfs_rq_runtime(struct cfs_rq *cfs_rq)
{
long min_cap, max_cap;
+ if (!static_branch_unlikely(&sched_asym_cpucapacity))
+ return 0;
+
min_cap = min(capacity_orig_of(prev_cpu), capacity_orig_of(cpu));
max_cap = cpu_rq(cpu)->rd->max_cpu_capacity;
/* Bring task utilization in sync with prev_cpu */
sync_entity_load_avg(&p->se);
- return min_cap * 1024 < task_util(p) * capacity_margin;
+ return !task_fits_capacity(p, min_cap);
}
/*
if (hrtick_enabled(rq))
hrtick_start_fair(rq, p);
+ update_misfit_status(p, rq);
+
return p;
idle:
+ update_misfit_status(NULL, rq);
new_tasks = idle_balance(rq, rf);
/*
enum fbq_type { regular, remote, all };
+enum group_type {
+ group_other = 0,
+ group_misfit_task,
+ group_imbalanced,
+ group_overloaded,
+};
+
#define LBF_ALL_PINNED 0x01
#define LBF_NEED_BREAK 0x02
#define LBF_DST_PINNED 0x04
unsigned int loop_max;
enum fbq_type fbq_type;
+ enum group_type src_grp_type;
struct list_head tasks;
};
if (READ_ONCE(rq->avg_dl.util_avg))
return true;
-#if defined(CONFIG_IRQ_TIME_ACCOUNTING) || defined(CONFIG_PARAVIRT_TIME_ACCOUNTING)
+#ifdef CONFIG_HAVE_SCHED_AVG_IRQ
if (READ_ONCE(rq->avg_irq.util_avg))
return true;
#endif
/********** Helpers for find_busiest_group ************************/
-enum group_type {
- group_other = 0,
- group_imbalanced,
- group_overloaded,
-};
-
/*
* sg_lb_stats - stats of a sched_group required for load_balancing
*/
unsigned int group_weight;
enum group_type group_type;
int group_no_capacity;
+ unsigned long group_misfit_task_load; /* A CPU has a task too big for its capacity */
#ifdef CONFIG_NUMA_BALANCING
unsigned int nr_numa_running;
unsigned int nr_preferred_running;
cpu_rq(cpu)->cpu_capacity = capacity;
sdg->sgc->capacity = capacity;
sdg->sgc->min_capacity = capacity;
+ sdg->sgc->max_capacity = capacity;
}
void update_group_capacity(struct sched_domain *sd, int cpu)
{
struct sched_domain *child = sd->child;
struct sched_group *group, *sdg = sd->groups;
- unsigned long capacity, min_capacity;
+ unsigned long capacity, min_capacity, max_capacity;
unsigned long interval;
interval = msecs_to_jiffies(sd->balance_interval);
capacity = 0;
min_capacity = ULONG_MAX;
+ max_capacity = 0;
if (child->flags & SD_OVERLAP) {
/*
}
min_capacity = min(capacity, min_capacity);
+ max_capacity = max(capacity, max_capacity);
}
} else {
/*
capacity += sgc->capacity;
min_capacity = min(sgc->min_capacity, min_capacity);
+ max_capacity = max(sgc->max_capacity, max_capacity);
group = group->next;
} while (group != child->groups);
}
sdg->sgc->capacity = capacity;
sdg->sgc->min_capacity = min_capacity;
+ sdg->sgc->max_capacity = max_capacity;
}
/*
}
/*
- * group_smaller_cpu_capacity: Returns true if sched_group sg has smaller
+ * group_smaller_min_cpu_capacity: Returns true if sched_group sg has smaller
* per-CPU capacity than sched_group ref.
*/
static inline bool
-group_smaller_cpu_capacity(struct sched_group *sg, struct sched_group *ref)
+group_smaller_min_cpu_capacity(struct sched_group *sg, struct sched_group *ref)
{
return sg->sgc->min_capacity * capacity_margin <
ref->sgc->min_capacity * 1024;
}
+/*
+ * group_smaller_max_cpu_capacity: Returns true if sched_group sg has smaller
+ * per-CPU capacity_orig than sched_group ref.
+ */
+static inline bool
+group_smaller_max_cpu_capacity(struct sched_group *sg, struct sched_group *ref)
+{
+ return sg->sgc->max_capacity * capacity_margin <
+ ref->sgc->max_capacity * 1024;
+}
+
static inline enum
group_type group_classify(struct sched_group *group,
struct sg_lb_stats *sgs)
if (sg_imbalanced(group))
return group_imbalanced;
+ if (sgs->group_misfit_task_load)
+ return group_misfit_task;
+
return group_other;
}
* @load_idx: Load index of sched_domain of this_cpu for load calc.
* @local_group: Does group contain this_cpu.
* @sgs: variable to hold the statistics for this group.
- * @overload: Indicate more than one runnable task for any CPU.
+ * @overload: Indicate pullable load (e.g. >1 runnable task).
*/
static inline void update_sg_lb_stats(struct lb_env *env,
struct sched_group *group, int load_idx,
*/
if (!nr_running && idle_cpu(i))
sgs->idle_cpus++;
+
+ if (env->sd->flags & SD_ASYM_CPUCAPACITY &&
+ sgs->group_misfit_task_load < rq->misfit_task_load) {
+ sgs->group_misfit_task_load = rq->misfit_task_load;
+ *overload = 1;
+ }
}
/* Adjust by relative CPU capacity of the group */
{
struct sg_lb_stats *busiest = &sds->busiest_stat;
+ /*
+ * Don't try to pull misfit tasks we can't help.
+ * We can use max_capacity here as reduction in capacity on some
+ * CPUs in the group should either be possible to resolve
+ * internally or be covered by avg_load imbalance (eventually).
+ */
+ if (sgs->group_type == group_misfit_task &&
+ (!group_smaller_max_cpu_capacity(sg, sds->local) ||
+ !group_has_capacity(env, &sds->local_stat)))
+ return false;
+
if (sgs->group_type > busiest->group_type)
return true;
* power/energy consequences are not considered.
*/
if (sgs->sum_nr_running <= sgs->group_weight &&
- group_smaller_cpu_capacity(sds->local, sg))
+ group_smaller_min_cpu_capacity(sds->local, sg))
+ return false;
+
+ /*
+ * If we have more than one misfit sg go with the biggest misfit.
+ */
+ if (sgs->group_type == group_misfit_task &&
+ sgs->group_misfit_task_load < busiest->group_misfit_task_load)
return false;
asym_packing:
struct sched_group *sg = env->sd->groups;
struct sg_lb_stats *local = &sds->local_stat;
struct sg_lb_stats tmp_sgs;
- int load_idx, prefer_sibling = 0;
+ int load_idx;
bool overload = false;
-
- if (child && child->flags & SD_PREFER_SIBLING)
- prefer_sibling = 1;
+ bool prefer_sibling = child && child->flags & SD_PREFER_SIBLING;
#ifdef CONFIG_NO_HZ_COMMON
if (env->idle == CPU_NEWLY_IDLE && READ_ONCE(nohz.has_blocked))
if (!env->sd->parent) {
/* update overload indicator if we are at root domain */
- if (env->dst_rq->rd->overload != overload)
- env->dst_rq->rd->overload = overload;
+ if (READ_ONCE(env->dst_rq->rd->overload) != overload)
+ WRITE_ONCE(env->dst_rq->rd->overload, overload);
}
}
* factors in sg capacity and sgs with smaller group_type are
* skipped when updating the busiest sg:
*/
- if (busiest->avg_load <= sds->avg_load ||
- local->avg_load >= sds->avg_load) {
+ if (busiest->group_type != group_misfit_task &&
+ (busiest->avg_load <= sds->avg_load ||
+ local->avg_load >= sds->avg_load)) {
env->imbalance = 0;
return fix_small_imbalance(env, sds);
}
(sds->avg_load - local->avg_load) * local->group_capacity
) / SCHED_CAPACITY_SCALE;
+ /* Boost imbalance to allow misfit task to be balanced. */
+ if (busiest->group_type == group_misfit_task) {
+ env->imbalance = max_t(long, env->imbalance,
+ busiest->group_misfit_task_load);
+ }
+
/*
* if *imbalance is less than the average load per runnable task
* there is no guarantee that any tasks will be moved so we'll have
busiest->group_no_capacity)
goto force_balance;
+ /* Misfit tasks should be dealt with regardless of the avg load */
+ if (busiest->group_type == group_misfit_task)
+ goto force_balance;
+
/*
* If the local group is busier than the selected busiest group
* don't try and pull any tasks.
force_balance:
/* Looks like there is an imbalance. Compute it */
+ env->src_grp_type = busiest->group_type;
calculate_imbalance(env, &sds);
return env->imbalance ? sds.busiest : NULL;
if (rt > env->fbq_type)
continue;
+ /*
+ * For ASYM_CPUCAPACITY domains with misfit tasks we simply
+ * seek the "biggest" misfit task.
+ */
+ if (env->src_grp_type == group_misfit_task) {
+ if (rq->misfit_task_load > busiest_load) {
+ busiest_load = rq->misfit_task_load;
+ busiest = rq;
+ }
+
+ continue;
+ }
+
capacity = capacity_of(i);
+ /*
+ * For ASYM_CPUCAPACITY domains, don't pick a CPU that could
+ * eventually lead to active_balancing high->low capacity.
+ * Higher per-CPU capacity is considered better than balancing
+ * average load.
+ */
+ if (env->sd->flags & SD_ASYM_CPUCAPACITY &&
+ capacity_of(env->dst_cpu) < capacity &&
+ rq->nr_running == 1)
+ continue;
+
wl = weighted_cpuload(rq);
/*
return 1;
}
+ if (env->src_grp_type == group_misfit_task)
+ return 1;
+
return unlikely(sd->nr_balance_failed > sd->cache_nice_tries+2);
}
if (time_before(now, nohz.next_balance))
goto out;
- if (rq->nr_running >= 2) {
+ if (rq->nr_running >= 2 || rq->misfit_task_load) {
flags = NOHZ_KICK_MASK;
goto out;
}
rq_unpin_lock(this_rq, rf);
if (this_rq->avg_idle < sysctl_sched_migration_cost ||
- !this_rq->rd->overload) {
+ !READ_ONCE(this_rq->rd->overload)) {
rcu_read_lock();
sd = rcu_dereference_check_sched_domain(this_rq->sd);
if (static_branch_unlikely(&sched_numa_balancing))
task_tick_numa(rq, curr);
+
+ update_misfit_status(curr, rq);
}
/*
SCHED_FEAT(HRTICK, false)
SCHED_FEAT(DOUBLE_TICK, false)
-SCHED_FEAT(LB_BIAS, true)
+SCHED_FEAT(LB_BIAS, false)
/*
* Decrement CPU capacity based on time not spent running tasks
int __update_load_avg_blocked_se(u64 now, int cpu, struct sched_entity *se)
{
- if (entity_is_task(se))
- se->runnable_weight = se->load.weight;
-
if (___update_load_sum(now, cpu, &se->avg, 0, 0, 0)) {
___update_load_avg(&se->avg, se_weight(se), se_runnable(se));
return 1;
int __update_load_avg_se(u64 now, int cpu, struct cfs_rq *cfs_rq, struct sched_entity *se)
{
- if (entity_is_task(se))
- se->runnable_weight = se->load.weight;
-
if (___update_load_sum(now, cpu, &se->avg, !!se->on_rq, !!se->on_rq,
cfs_rq->curr == se)) {
return 0;
}
-#if defined(CONFIG_IRQ_TIME_ACCOUNTING) || defined(CONFIG_PARAVIRT_TIME_ACCOUNTING)
+#ifdef CONFIG_HAVE_SCHED_AVG_IRQ
/*
* irq:
*
int update_rt_rq_load_avg(u64 now, struct rq *rq, int running);
int update_dl_rq_load_avg(u64 now, struct rq *rq, int running);
-#if defined(CONFIG_IRQ_TIME_ACCOUNTING) || defined(CONFIG_PARAVIRT_TIME_ACCOUNTING)
+#ifdef CONFIG_HAVE_SCHED_AVG_IRQ
int update_irq_load_avg(struct rq *rq, u64 running);
#else
static inline int
int nr_periods;
int nr_throttled;
u64 throttled_time;
+
+ bool distribute_running;
#endif
};
cpumask_var_t span;
cpumask_var_t online;
- /* Indicate more than one runnable task for any CPU */
- bool overload;
+ /*
+ * Indicate pullable load on at least one CPU, e.g:
+ * - More than one runnable task
+ * - Running task is misfit
+ */
+ int overload;
/*
* The bit corresponding to a CPU gets set here if such CPU has more
unsigned char idle_balance;
+ unsigned long misfit_task_load;
+
/* For active balancing */
int active_balance;
int push_cpu;
struct sched_avg avg_rt;
struct sched_avg avg_dl;
-#if defined(CONFIG_IRQ_TIME_ACCOUNTING) || defined(CONFIG_PARAVIRT_TIME_ACCOUNTING)
-#define HAVE_SCHED_AVG_IRQ
+#ifdef CONFIG_HAVE_SCHED_AVG_IRQ
struct sched_avg avg_irq;
#endif
u64 idle_stamp;
DECLARE_PER_CPU(struct sched_domain_shared *, sd_llc_shared);
DECLARE_PER_CPU(struct sched_domain *, sd_numa);
DECLARE_PER_CPU(struct sched_domain *, sd_asym);
+extern struct static_key_false sched_asym_cpucapacity;
struct sched_group_capacity {
atomic_t ref;
*/
unsigned long capacity;
unsigned long min_capacity; /* Min per-CPU capacity in group */
+ unsigned long max_capacity; /* Max per-CPU capacity in group */
unsigned long next_update;
int imbalance; /* XXX unrelated to capacity but shared group state */
0;
#undef SCHED_FEAT
-#define sched_feat(x) (sysctl_sched_features & (1UL << __SCHED_FEAT_##x))
+#define sched_feat(x) !!(sysctl_sched_features & (1UL << __SCHED_FEAT_##x))
#endif /* SCHED_DEBUG && HAVE_JUMP_LABEL */
if (prev_nr < 2 && rq->nr_running >= 2) {
#ifdef CONFIG_SMP
- if (!rq->rd->overload)
- rq->rd->overload = true;
+ if (!READ_ONCE(rq->rd->overload))
+ WRITE_ONCE(rq->rd->overload, 1);
#endif
}
}
#endif
-#ifdef HAVE_SCHED_AVG_IRQ
+#ifdef CONFIG_HAVE_SCHED_AVG_IRQ
static inline unsigned long cpu_util_irq(struct rq *rq)
{
return rq->avg_irq.util_avg;
DEFINE_MUTEX(sched_domains_mutex);
/* Protected by sched_domains_mutex: */
-cpumask_var_t sched_domains_tmpmask;
-cpumask_var_t sched_domains_tmpmask2;
+static cpumask_var_t sched_domains_tmpmask;
+static cpumask_var_t sched_domains_tmpmask2;
#ifdef CONFIG_SCHED_DEBUG
DEFINE_PER_CPU(struct sched_domain_shared *, sd_llc_shared);
DEFINE_PER_CPU(struct sched_domain *, sd_numa);
DEFINE_PER_CPU(struct sched_domain *, sd_asym);
+DEFINE_STATIC_KEY_FALSE(sched_asym_cpucapacity);
static void update_top_cache_domain(int cpu)
{
sg_span = sched_group_span(sg);
sg->sgc->capacity = SCHED_CAPACITY_SCALE * cpumask_weight(sg_span);
sg->sgc->min_capacity = SCHED_CAPACITY_SCALE;
+ sg->sgc->max_capacity = SCHED_CAPACITY_SCALE;
}
static int
sg->sgc->capacity = SCHED_CAPACITY_SCALE * cpumask_weight(sched_group_span(sg));
sg->sgc->min_capacity = SCHED_CAPACITY_SCALE;
+ sg->sgc->max_capacity = SCHED_CAPACITY_SCALE;
return sg;
}
* SD_SHARE_PKG_RESOURCES - describes shared caches
* SD_NUMA - describes NUMA topologies
* SD_SHARE_POWERDOMAIN - describes shared power domain
- * SD_ASYM_CPUCAPACITY - describes mixed capacity topologies
*
* Odd one out, which beside describing the topology has a quirk also
* prescribes the desired behaviour that goes along with it:
SD_SHARE_PKG_RESOURCES | \
SD_NUMA | \
SD_ASYM_PACKING | \
- SD_ASYM_CPUCAPACITY | \
SD_SHARE_POWERDOMAIN)
static struct sched_domain *
sd_init(struct sched_domain_topology_level *tl,
const struct cpumask *cpu_map,
- struct sched_domain *child, int cpu)
+ struct sched_domain *child, int dflags, int cpu)
{
struct sd_data *sdd = &tl->data;
struct sched_domain *sd = *per_cpu_ptr(sdd->sd, cpu);
"wrong sd_flags in topology description\n"))
sd_flags &= ~TOPOLOGY_SD_FLAGS;
+ /* Apply detected topology flags */
+ sd_flags |= dflags;
+
*sd = (struct sched_domain){
.min_interval = sd_weight,
.max_interval = 2*sd_weight,
| 0*SD_SHARE_CPUCAPACITY
| 0*SD_SHARE_PKG_RESOURCES
| 0*SD_SERIALIZE
- | 0*SD_PREFER_SIBLING
+ | 1*SD_PREFER_SIBLING
| 0*SD_NUMA
| sd_flags
,
if (sd->flags & SD_ASYM_CPUCAPACITY) {
struct sched_domain *t = sd;
+ /*
+ * Don't attempt to spread across CPUs of different capacities.
+ */
+ if (sd->child)
+ sd->child->flags &= ~SD_PREFER_SIBLING;
+
for_each_lower_domain(t)
t->flags |= SD_BALANCE_WAKE;
}
if (sd->flags & SD_SHARE_CPUCAPACITY) {
- sd->flags |= SD_PREFER_SIBLING;
sd->imbalance_pct = 110;
sd->smt_gain = 1178; /* ~15% */
} else if (sd->flags & SD_SHARE_PKG_RESOURCES) {
- sd->flags |= SD_PREFER_SIBLING;
sd->imbalance_pct = 117;
sd->cache_nice_tries = 1;
sd->busy_idx = 2;
sd->busy_idx = 3;
sd->idle_idx = 2;
+ sd->flags &= ~SD_PREFER_SIBLING;
sd->flags |= SD_SERIALIZE;
if (sched_domains_numa_distance[tl->numa_level] > RECLAIM_DISTANCE) {
sd->flags &= ~(SD_BALANCE_EXEC |
#endif
} else {
- sd->flags |= SD_PREFER_SIBLING;
sd->cache_nice_tries = 1;
sd->busy_idx = 2;
sd->idle_idx = 1;
static struct sched_domain *build_sched_domain(struct sched_domain_topology_level *tl,
const struct cpumask *cpu_map, struct sched_domain_attr *attr,
- struct sched_domain *child, int cpu)
+ struct sched_domain *child, int dflags, int cpu)
{
- struct sched_domain *sd = sd_init(tl, cpu_map, child, cpu);
+ struct sched_domain *sd = sd_init(tl, cpu_map, child, dflags, cpu);
if (child) {
sd->level = child->level + 1;
return sd;
}
+/*
+ * Find the sched_domain_topology_level where all CPU capacities are visible
+ * for all CPUs.
+ */
+static struct sched_domain_topology_level
+*asym_cpu_capacity_level(const struct cpumask *cpu_map)
+{
+ int i, j, asym_level = 0;
+ bool asym = false;
+ struct sched_domain_topology_level *tl, *asym_tl = NULL;
+ unsigned long cap;
+
+ /* Is there any asymmetry? */
+ cap = arch_scale_cpu_capacity(NULL, cpumask_first(cpu_map));
+
+ for_each_cpu(i, cpu_map) {
+ if (arch_scale_cpu_capacity(NULL, i) != cap) {
+ asym = true;
+ break;
+ }
+ }
+
+ if (!asym)
+ return NULL;
+
+ /*
+ * Examine topology from all CPU's point of views to detect the lowest
+ * sched_domain_topology_level where a highest capacity CPU is visible
+ * to everyone.
+ */
+ for_each_cpu(i, cpu_map) {
+ unsigned long max_capacity = arch_scale_cpu_capacity(NULL, i);
+ int tl_id = 0;
+
+ for_each_sd_topology(tl) {
+ if (tl_id < asym_level)
+ goto next_level;
+
+ for_each_cpu_and(j, tl->mask(i), cpu_map) {
+ unsigned long capacity;
+
+ capacity = arch_scale_cpu_capacity(NULL, j);
+
+ if (capacity <= max_capacity)
+ continue;
+
+ max_capacity = capacity;
+ asym_level = tl_id;
+ asym_tl = tl;
+ }
+next_level:
+ tl_id++;
+ }
+ }
+
+ return asym_tl;
+}
+
+
/*
* Build sched domains for a given set of CPUs and attach the sched domains
* to the individual CPUs
struct s_data d;
struct rq *rq = NULL;
int i, ret = -ENOMEM;
+ struct sched_domain_topology_level *tl_asym;
+ bool has_asym = false;
alloc_state = __visit_domain_allocation_hell(&d, cpu_map);
if (alloc_state != sa_rootdomain)
goto error;
+ tl_asym = asym_cpu_capacity_level(cpu_map);
+
/* Set up domains for CPUs specified by the cpu_map: */
for_each_cpu(i, cpu_map) {
struct sched_domain_topology_level *tl;
sd = NULL;
for_each_sd_topology(tl) {
- sd = build_sched_domain(tl, cpu_map, attr, sd, i);
+ int dflags = 0;
+
+ if (tl == tl_asym) {
+ dflags |= SD_ASYM_CPUCAPACITY;
+ has_asym = true;
+ }
+
+ sd = build_sched_domain(tl, cpu_map, attr, sd, dflags, i);
+
if (tl == sched_domain_topology)
*per_cpu_ptr(d.sd, i) = sd;
if (tl->flags & SDTL_OVERLAP)
}
rcu_read_unlock();
+ if (has_asym)
+ static_branch_enable_cpuslocked(&sched_asym_cpucapacity);
+
if (rq && sched_debug_enabled) {
pr_info("root domain span: %*pbl (max cpu_capacity = %lu)\n",
cpumask_pr_args(cpu_map), rq->rd->max_cpu_capacity);
}
static int
-do_sigaltstack (const stack_t *ss, stack_t *oss, unsigned long sp)
+do_sigaltstack (const stack_t *ss, stack_t *oss, unsigned long sp,
+ size_t min_ss_size)
{
struct task_struct *t = current;
ss_size = 0;
ss_sp = NULL;
} else {
- if (unlikely(ss_size < MINSIGSTKSZ))
+ if (unlikely(ss_size < min_ss_size))
return -ENOMEM;
}
if (uss && copy_from_user(&new, uss, sizeof(stack_t)))
return -EFAULT;
err = do_sigaltstack(uss ? &new : NULL, uoss ? &old : NULL,
- current_user_stack_pointer());
+ current_user_stack_pointer(),
+ MINSIGSTKSZ);
if (!err && uoss && copy_to_user(uoss, &old, sizeof(stack_t)))
err = -EFAULT;
return err;
stack_t new;
if (copy_from_user(&new, uss, sizeof(stack_t)))
return -EFAULT;
- (void)do_sigaltstack(&new, NULL, current_user_stack_pointer());
+ (void)do_sigaltstack(&new, NULL, current_user_stack_pointer(),
+ MINSIGSTKSZ);
/* squash all but EFAULT for now */
return 0;
}
uss.ss_size = uss32.ss_size;
}
ret = do_sigaltstack(uss_ptr ? &uss : NULL, &uoss,
- compat_user_stack_pointer());
+ compat_user_stack_pointer(),
+ COMPAT_MINSIGSTKSZ);
if (ret >= 0 && uoss_ptr) {
compat_stack_t old;
memset(&old, 0, sizeof(old));
pending >>= softirq_bit;
}
- rcu_bh_qs();
+ if (__this_cpu_read(ksoftirqd) == current)
+ rcu_softirq_qs();
local_irq_disable();
pending = local_softirq_pending();
* Block until the stutter interval ends. This must be called periodically
* by all running kthreads that need to be subject to stuttering.
*/
-void stutter_wait(const char *title)
+bool stutter_wait(const char *title)
{
int spt;
}
torture_shutdown_absorb(title);
}
+ return !!spt;
}
EXPORT_SYMBOL_GPL(stutter_wait);
if (!bt || !(blk_tracer_flags.val & TRACE_BLK_OPT_CGROUP))
return NULL;
- if (!bio->bi_css)
+ if (!bio->bi_blkg)
return NULL;
- return cgroup_get_kernfs_id(bio->bi_css->cgroup);
+ return cgroup_get_kernfs_id(bio_blkcg(bio)->css.cgroup);
}
#else
static union kernfs_node_id *
* Copyright (C) 2018 Joel Fernandes (Google) <joel@joelfernandes.org>
*/
+#include <linux/trace_clock.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/kernel.h>
#include <linux/kthread.h>
-#include <linux/ktime.h>
#include <linux/module.h>
#include <linux/printk.h>
#include <linux/string.h>
static void busy_wait(ulong time)
{
- ktime_t start, end;
- start = ktime_get();
+ u64 start, end;
+ start = trace_clock_local();
do {
- end = ktime_get();
+ end = trace_clock_local();
if (kthread_should_stop())
break;
- } while (ktime_to_ns(ktime_sub(end, start)) < (time * 1000));
+ } while ((end - start) < (time * 1000));
}
static int preemptirq_delay_run(void *data)
kfree(field);
}
-static struct synth_field *parse_synth_field(char *field_type,
- char *field_name)
+static struct synth_field *parse_synth_field(int argc, char **argv,
+ int *consumed)
{
struct synth_field *field;
+ const char *prefix = NULL;
+ char *field_type = argv[0], *field_name;
int len, ret = 0;
char *array;
if (field_type[0] == ';')
field_type++;
+ if (!strcmp(field_type, "unsigned")) {
+ if (argc < 3)
+ return ERR_PTR(-EINVAL);
+ prefix = "unsigned ";
+ field_type = argv[1];
+ field_name = argv[2];
+ *consumed = 3;
+ } else {
+ field_name = argv[1];
+ *consumed = 2;
+ }
+
len = strlen(field_name);
if (field_name[len - 1] == ';')
field_name[len - 1] = '\0';
array = strchr(field_name, '[');
if (array)
len += strlen(array);
+ if (prefix)
+ len += strlen(prefix);
field->type = kzalloc(len, GFP_KERNEL);
if (!field->type) {
ret = -ENOMEM;
goto free;
}
+ if (prefix)
+ strcat(field->type, prefix);
strcat(field->type, field_type);
if (array) {
strcat(field->type, array);
struct synth_field *field, *fields[SYNTH_FIELDS_MAX];
struct synth_event *event = NULL;
bool delete_event = false;
- int i, n_fields = 0, ret = 0;
+ int i, consumed = 0, n_fields = 0, ret = 0;
char *name;
mutex_lock(&synth_event_mutex);
goto err;
}
- field = parse_synth_field(argv[i], argv[i + 1]);
+ field = parse_synth_field(argc - i, &argv[i], &consumed);
if (IS_ERR(field)) {
ret = PTR_ERR(field);
goto err;
}
- fields[n_fields] = field;
- i++; n_fields++;
+ fields[n_fields++] = field;
+ i += consumed - 1;
}
- if (i < argc) {
+ if (i < argc && strcmp(argv[i], ";") != 0) {
ret = -EINVAL;
goto err;
}
#include <linux/sched/task.h>
#include <linux/static_key.h>
-extern struct tracepoint * const __start___tracepoints_ptrs[];
-extern struct tracepoint * const __stop___tracepoints_ptrs[];
+extern tracepoint_ptr_t __start___tracepoints_ptrs[];
+extern tracepoint_ptr_t __stop___tracepoints_ptrs[];
DEFINE_SRCU(tracepoint_srcu);
EXPORT_SYMBOL_GPL(tracepoint_srcu);
}
EXPORT_SYMBOL_GPL(tracepoint_probe_unregister);
-static void for_each_tracepoint_range(struct tracepoint * const *begin,
- struct tracepoint * const *end,
+static void for_each_tracepoint_range(
+ tracepoint_ptr_t *begin, tracepoint_ptr_t *end,
void (*fct)(struct tracepoint *tp, void *priv),
void *priv)
{
+ tracepoint_ptr_t *iter;
+
if (!begin)
return;
-
- if (IS_ENABLED(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS)) {
- const int *iter;
-
- for (iter = (const int *)begin; iter < (const int *)end; iter++)
- fct(offset_to_ptr(iter), priv);
- } else {
- struct tracepoint * const *iter;
-
- for (iter = begin; iter < end; iter++)
- fct(*iter, priv);
- }
+ for (iter = begin; iter < end; iter++)
+ fct(tracepoint_ptr_deref(iter), priv);
}
#ifdef CONFIG_MODULES
endchoice
+config KASAN_S390_4_LEVEL_PAGING
+ bool "KASan: use 4-level paging"
+ depends on KASAN && S390
+ help
+ Compiling the kernel with KASan disables automatic 3-level vs
+ 4-level paging selection. 3-level paging is used by default (up
+ to 3TB of RAM with KASan enabled). This options allows to force
+ 4-level paging instead.
+
config TEST_KASAN
tristate "Module for testing kasan for bug detection"
depends on m && KASAN
obj-$(CONFIG_ZLIB_DEFLATE) += zlib_deflate/
obj-$(CONFIG_REED_SOLOMON) += reed_solomon/
obj-$(CONFIG_BCH) += bch.o
-CFLAGS_bch.o := $(call cc-option,-Wframe-larger-than=4500)
obj-$(CONFIG_LZO_COMPRESS) += lzo/
obj-$(CONFIG_LZO_DECOMPRESS) += lzo/
obj-$(CONFIG_LZ4_COMPRESS) += lz4/
#define GF_T(_p) (CONFIG_BCH_CONST_T)
#define GF_N(_p) ((1 << (CONFIG_BCH_CONST_M))-1)
#define BCH_MAX_M (CONFIG_BCH_CONST_M)
+#define BCH_MAX_T (CONFIG_BCH_CONST_T)
#else
#define GF_M(_p) ((_p)->m)
#define GF_T(_p) ((_p)->t)
#define GF_N(_p) ((_p)->n)
-#define BCH_MAX_M 15
+#define BCH_MAX_M 15 /* 2KB */
+#define BCH_MAX_T 64 /* 64 bit correction */
#endif
-#define BCH_MAX_T (((1 << BCH_MAX_M) - 1) / BCH_MAX_M)
-
#define BCH_ECC_WORDS(_p) DIV_ROUND_UP(GF_M(_p)*GF_T(_p), 32)
#define BCH_ECC_BYTES(_p) DIV_ROUND_UP(GF_M(_p)*GF_T(_p), 8)
#define BCH_ECC_MAX_WORDS DIV_ROUND_UP(BCH_MAX_M * BCH_MAX_T, 32)
-#define BCH_ECC_MAX_BYTES DIV_ROUND_UP(BCH_MAX_M * BCH_MAX_T, 8)
#ifndef dbg
#define dbg(_fmt, args...) do {} while (0)
const uint32_t * const tab3 = tab2 + 256*(l+1);
const uint32_t *pdata, *p0, *p1, *p2, *p3;
+ if (WARN_ON(r_bytes > sizeof(r)))
+ return;
+
if (ecc) {
/* load ecc parity bytes into internal 32-bit buffer */
load_ecc8(bch, bch->ecc_buf, ecc);
*/
goto fail;
+ if (t > BCH_MAX_T)
+ /*
+ * we can support larger than 64 bits if necessary, at the
+ * cost of higher stack usage.
+ */
+ goto fail;
+
/* sanity checks */
if ((t < 1) || (m*t >= ((1 << m)-1)))
/* invalid t value */
}
#if CRC_LE_BITS == 1
-u32 __pure crc32_le(u32 crc, unsigned char const *p, size_t len)
+u32 __pure __weak crc32_le(u32 crc, unsigned char const *p, size_t len)
{
return crc32_le_generic(crc, p, len, NULL, CRC32_POLY_LE);
}
-u32 __pure __crc32c_le(u32 crc, unsigned char const *p, size_t len)
+u32 __pure __weak __crc32c_le(u32 crc, unsigned char const *p, size_t len)
{
return crc32_le_generic(crc, p, len, NULL, CRC32C_POLY_LE);
}
#else
-u32 __pure crc32_le(u32 crc, unsigned char const *p, size_t len)
+u32 __pure __weak crc32_le(u32 crc, unsigned char const *p, size_t len)
{
return crc32_le_generic(crc, p, len,
(const u32 (*)[256])crc32table_le, CRC32_POLY_LE);
}
-u32 __pure __crc32c_le(u32 crc, unsigned char const *p, size_t len)
+u32 __pure __weak __crc32c_le(u32 crc, unsigned char const *p, size_t len)
{
return crc32_le_generic(crc, p, len,
(const u32 (*)[256])crc32ctable_le, CRC32C_POLY_LE);
EXPORT_SYMBOL(crc32_le);
EXPORT_SYMBOL(__crc32c_le);
+u32 crc32_le_base(u32, unsigned char const *, size_t) __alias(crc32_le);
+u32 __crc32c_le_base(u32, unsigned char const *, size_t) __alias(__crc32c_le);
+
/*
* This multiplies the polynomials x and y modulo the given modulus.
* This follows the "little-endian" CRC convention that the lsbit
* that would just muddy the log. So we report the first one and
* shut up after that.
*/
-int debug_locks = 1;
+int debug_locks __read_mostly = 1;
EXPORT_SYMBOL_GPL(debug_locks);
/*
* 'silent failure': nothing is printed to the console when
* a locking bug is detected.
*/
-int debug_locks_silent;
+int debug_locks_silent __read_mostly;
EXPORT_SYMBOL_GPL(debug_locks_silent);
/*
*/
int debug_locks_off(void)
{
- if (__debug_locks_off()) {
+ if (debug_locks && __debug_locks_off()) {
if (!debug_locks_silent) {
console_verbose();
return 1;
*/
void percpu_ref_reinit(struct percpu_ref *ref)
{
+ WARN_ON_ONCE(!percpu_ref_is_zero(ref));
+
+ percpu_ref_resurrect(ref);
+}
+EXPORT_SYMBOL_GPL(percpu_ref_reinit);
+
+/**
+ * percpu_ref_resurrect - modify a percpu refcount from dead to live
+ * @ref: perpcu_ref to resurrect
+ *
+ * Modify @ref so that it's in the same state as before percpu_ref_kill() was
+ * called. @ref must be dead but must not yet have exited.
+ *
+ * If @ref->release() frees @ref then the caller is responsible for
+ * guaranteeing that @ref->release() does not get called while this
+ * function is in progress.
+ *
+ * Note that percpu_ref_tryget[_live]() are safe to perform on @ref while
+ * this function is in progress.
+ */
+void percpu_ref_resurrect(struct percpu_ref *ref)
+{
+ unsigned long __percpu *percpu_count;
unsigned long flags;
spin_lock_irqsave(&percpu_ref_switch_lock, flags);
- WARN_ON_ONCE(!percpu_ref_is_zero(ref));
+ WARN_ON_ONCE(!(ref->percpu_count_ptr & __PERCPU_REF_DEAD));
+ WARN_ON_ONCE(__ref_is_percpu(ref, &percpu_count));
ref->percpu_count_ptr &= ~__PERCPU_REF_DEAD;
percpu_ref_get(ref);
spin_unlock_irqrestore(&percpu_ref_switch_lock, flags);
}
-EXPORT_SYMBOL_GPL(percpu_ref_reinit);
+EXPORT_SYMBOL_GPL(percpu_ref_resurrect);
IDA_BUG_ON(ida, !ida_is_empty(ida));
}
+static DEFINE_IDA(ida);
+
static int ida_checks(void)
{
- DEFINE_IDA(ida);
-
IDA_BUG_ON(&ida, !ida_is_empty(&ida));
ida_check_alloc(&ida);
ida_check_destroy(&ida);
copy = end - str;
memcpy(str, args, copy);
str += len;
- args += len;
+ args += len + 1;
}
}
if (process)
mmu-y := nommu.o
mmu-$(CONFIG_MMU) := gup.o highmem.o memory.o mincore.o \
- mlock.o mmap.o mprotect.o mremap.o msync.o \
- page_vma_mapped.o pagewalk.o pgtable-generic.o \
- rmap.o vmalloc.o
+ mlock.o mmap.o mmu_gather.o mprotect.o mremap.o \
+ msync.o page_vma_mapped.o pagewalk.o \
+ pgtable-generic.o rmap.o vmalloc.o
ifdef CONFIG_CROSS_MEMORY_ATTACH
bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,
unsigned long new_addr, unsigned long old_end,
- pmd_t *old_pmd, pmd_t *new_pmd, bool *need_flush)
+ pmd_t *old_pmd, pmd_t *new_pmd)
{
spinlock_t *old_ptl, *new_ptl;
pmd_t pmd;
if (new_ptl != old_ptl)
spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
pmd = pmdp_huge_get_and_clear(mm, old_addr, old_pmd);
- if (pmd_present(pmd) && pmd_dirty(pmd))
+ if (pmd_present(pmd))
force_flush = true;
VM_BUG_ON(!pmd_none(*new_pmd));
}
pmd = move_soft_dirty_pmd(pmd);
set_pmd_at(mm, new_addr, new_pmd, pmd);
- if (new_ptl != old_ptl)
- spin_unlock(new_ptl);
if (force_flush)
flush_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
- else
- *need_flush = true;
+ if (new_ptl != old_ptl)
+ spin_unlock(new_ptl);
spin_unlock(old_ptl);
return true;
}
if (!(pvmw->pmd && !pvmw->pte))
return;
- mmu_notifier_invalidate_range_start(mm, address,
- address + HPAGE_PMD_SIZE);
-
flush_cache_range(vma, address, address + HPAGE_PMD_SIZE);
pmdval = *pvmw->pmd;
pmdp_invalidate(vma, address, pvmw->pmd);
set_pmd_at(mm, address, pvmw->pmd, pmdswp);
page_remove_rmap(page, true);
put_page(page);
-
- mmu_notifier_invalidate_range_end(mm, address,
- address + HPAGE_PMD_SIZE);
}
void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)
set_fs(KERNEL_DS);
pagefault_disable();
+ current->kernel_uaccess_faults_ok++;
ret = __copy_from_user_inatomic(dst,
(__force const void __user *)src, size);
+ current->kernel_uaccess_faults_ok--;
pagefault_enable();
set_fs(old_fs);
set_fs(KERNEL_DS);
pagefault_disable();
+ current->kernel_uaccess_faults_ok++;
ret = __copy_to_user_inatomic((__force void __user *)dst, src, size);
+ current->kernel_uaccess_faults_ok--;
pagefault_enable();
set_fs(old_fs);
set_fs(KERNEL_DS);
pagefault_disable();
+ current->kernel_uaccess_faults_ok++;
do {
ret = __get_user(*dst++, (const char __user __force *)src++);
} while (dst[-1] && ret == 0 && src - unsafe_addr < count);
+ current->kernel_uaccess_faults_ok--;
dst[-1] = '\0';
pagefault_enable();
set_fs(old_fs);
#endif /* SPLIT_RSS_COUNTING */
-#ifdef HAVE_GENERIC_MMU_GATHER
-
-static bool tlb_next_batch(struct mmu_gather *tlb)
-{
- struct mmu_gather_batch *batch;
-
- batch = tlb->active;
- if (batch->next) {
- tlb->active = batch->next;
- return true;
- }
-
- if (tlb->batch_count == MAX_GATHER_BATCH_COUNT)
- return false;
-
- batch = (void *)__get_free_pages(GFP_NOWAIT | __GFP_NOWARN, 0);
- if (!batch)
- return false;
-
- tlb->batch_count++;
- batch->next = NULL;
- batch->nr = 0;
- batch->max = MAX_GATHER_BATCH;
-
- tlb->active->next = batch;
- tlb->active = batch;
-
- return true;
-}
-
-void arch_tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
- unsigned long start, unsigned long end)
-{
- tlb->mm = mm;
-
- /* Is it from 0 to ~0? */
- tlb->fullmm = !(start | (end+1));
- tlb->need_flush_all = 0;
- tlb->local.next = NULL;
- tlb->local.nr = 0;
- tlb->local.max = ARRAY_SIZE(tlb->__pages);
- tlb->active = &tlb->local;
- tlb->batch_count = 0;
-
-#ifdef CONFIG_HAVE_RCU_TABLE_FREE
- tlb->batch = NULL;
-#endif
- tlb->page_size = 0;
-
- __tlb_reset_range(tlb);
-}
-
-static void tlb_flush_mmu_free(struct mmu_gather *tlb)
-{
- struct mmu_gather_batch *batch;
-
-#ifdef CONFIG_HAVE_RCU_TABLE_FREE
- tlb_table_flush(tlb);
-#endif
- for (batch = &tlb->local; batch && batch->nr; batch = batch->next) {
- free_pages_and_swap_cache(batch->pages, batch->nr);
- batch->nr = 0;
- }
- tlb->active = &tlb->local;
-}
-
-void tlb_flush_mmu(struct mmu_gather *tlb)
-{
- tlb_flush_mmu_tlbonly(tlb);
- tlb_flush_mmu_free(tlb);
-}
-
-/* tlb_finish_mmu
- * Called at the end of the shootdown operation to free up any resources
- * that were required.
- */
-void arch_tlb_finish_mmu(struct mmu_gather *tlb,
- unsigned long start, unsigned long end, bool force)
-{
- struct mmu_gather_batch *batch, *next;
-
- if (force)
- __tlb_adjust_range(tlb, start, end - start);
-
- tlb_flush_mmu(tlb);
-
- /* keep the page table cache within bounds */
- check_pgt_cache();
-
- for (batch = tlb->local.next; batch; batch = next) {
- next = batch->next;
- free_pages((unsigned long)batch, 0);
- }
- tlb->local.next = NULL;
-}
-
-/* __tlb_remove_page
- * Must perform the equivalent to __free_pte(pte_get_and_clear(ptep)), while
- * handling the additional races in SMP caused by other CPUs caching valid
- * mappings in their TLBs. Returns the number of free page slots left.
- * When out of page slots we must call tlb_flush_mmu().
- *returns true if the caller should flush.
- */
-bool __tlb_remove_page_size(struct mmu_gather *tlb, struct page *page, int page_size)
-{
- struct mmu_gather_batch *batch;
-
- VM_BUG_ON(!tlb->end);
- VM_WARN_ON(tlb->page_size != page_size);
-
- batch = tlb->active;
- /*
- * Add the page and check if we are full. If so
- * force a flush.
- */
- batch->pages[batch->nr++] = page;
- if (batch->nr == batch->max) {
- if (!tlb_next_batch(tlb))
- return true;
- batch = tlb->active;
- }
- VM_BUG_ON_PAGE(batch->nr > batch->max, page);
-
- return false;
-}
-
-#endif /* HAVE_GENERIC_MMU_GATHER */
-
-#ifdef CONFIG_HAVE_RCU_TABLE_FREE
-
-/*
- * See the comment near struct mmu_table_batch.
- */
-
-/*
- * If we want tlb_remove_table() to imply TLB invalidates.
- */
-static inline void tlb_table_invalidate(struct mmu_gather *tlb)
-{
-#ifdef CONFIG_HAVE_RCU_TABLE_INVALIDATE
- /*
- * Invalidate page-table caches used by hardware walkers. Then we still
- * need to RCU-sched wait while freeing the pages because software
- * walkers can still be in-flight.
- */
- tlb_flush_mmu_tlbonly(tlb);
-#endif
-}
-
-static void tlb_remove_table_smp_sync(void *arg)
-{
- /* Simply deliver the interrupt */
-}
-
-static void tlb_remove_table_one(void *table)
-{
- /*
- * This isn't an RCU grace period and hence the page-tables cannot be
- * assumed to be actually RCU-freed.
- *
- * It is however sufficient for software page-table walkers that rely on
- * IRQ disabling. See the comment near struct mmu_table_batch.
- */
- smp_call_function(tlb_remove_table_smp_sync, NULL, 1);
- __tlb_remove_table(table);
-}
-
-static void tlb_remove_table_rcu(struct rcu_head *head)
-{
- struct mmu_table_batch *batch;
- int i;
-
- batch = container_of(head, struct mmu_table_batch, rcu);
-
- for (i = 0; i < batch->nr; i++)
- __tlb_remove_table(batch->tables[i]);
-
- free_page((unsigned long)batch);
-}
-
-void tlb_table_flush(struct mmu_gather *tlb)
-{
- struct mmu_table_batch **batch = &tlb->batch;
-
- if (*batch) {
- tlb_table_invalidate(tlb);
- call_rcu_sched(&(*batch)->rcu, tlb_remove_table_rcu);
- *batch = NULL;
- }
-}
-
-void tlb_remove_table(struct mmu_gather *tlb, void *table)
-{
- struct mmu_table_batch **batch = &tlb->batch;
-
- if (*batch == NULL) {
- *batch = (struct mmu_table_batch *)__get_free_page(GFP_NOWAIT | __GFP_NOWARN);
- if (*batch == NULL) {
- tlb_table_invalidate(tlb);
- tlb_remove_table_one(table);
- return;
- }
- (*batch)->nr = 0;
- }
-
- (*batch)->tables[(*batch)->nr++] = table;
- if ((*batch)->nr == MAX_TABLE_BATCH)
- tlb_table_flush(tlb);
-}
-
-#endif /* CONFIG_HAVE_RCU_TABLE_FREE */
-
-/**
- * tlb_gather_mmu - initialize an mmu_gather structure for page-table tear-down
- * @tlb: the mmu_gather structure to initialize
- * @mm: the mm_struct of the target address space
- * @start: start of the region that will be removed from the page-table
- * @end: end of the region that will be removed from the page-table
- *
- * Called to initialize an (on-stack) mmu_gather structure for page-table
- * tear-down from @mm. The @start and @end are set to 0 and -1
- * respectively when @mm is without users and we're going to destroy
- * the full address space (exit/execve).
- */
-void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
- unsigned long start, unsigned long end)
-{
- arch_tlb_gather_mmu(tlb, mm, start, end);
- inc_tlb_flush_pending(tlb->mm);
-}
-
-void tlb_finish_mmu(struct mmu_gather *tlb,
- unsigned long start, unsigned long end)
-{
- /*
- * If there are parallel threads are doing PTE changes on same range
- * under non-exclusive lock(e.g., mmap_sem read-side) but defer TLB
- * flush by batching, a thread has stable TLB entry can fail to flush
- * the TLB by observing pte_none|!pte_dirty, for example so flush TLB
- * forcefully if we detect parallel PTE batching threads.
- */
- bool force = mm_tlb_flush_nested(tlb->mm);
-
- arch_tlb_finish_mmu(tlb, start, end, force);
- dec_tlb_flush_pending(tlb->mm);
-}
-
/*
* Note: this doesn't free the actual pages themselves. That
* has been handled earlier when unmapping all the memory regions.
if (flags & MAP_FIXED_NOREPLACE) {
struct vm_area_struct *vma = find_vma(mm, addr);
- if (vma && vma->vm_start <= addr)
+ if (vma && vma->vm_start < addr + len)
return -EEXIST;
}
--- /dev/null
+#include <linux/gfp.h>
+#include <linux/highmem.h>
+#include <linux/kernel.h>
+#include <linux/mmdebug.h>
+#include <linux/mm_types.h>
+#include <linux/pagemap.h>
+#include <linux/rcupdate.h>
+#include <linux/smp.h>
+#include <linux/swap.h>
+
+#include <asm/pgalloc.h>
+#include <asm/tlb.h>
+
+#ifdef HAVE_GENERIC_MMU_GATHER
+
+static bool tlb_next_batch(struct mmu_gather *tlb)
+{
+ struct mmu_gather_batch *batch;
+
+ batch = tlb->active;
+ if (batch->next) {
+ tlb->active = batch->next;
+ return true;
+ }
+
+ if (tlb->batch_count == MAX_GATHER_BATCH_COUNT)
+ return false;
+
+ batch = (void *)__get_free_pages(GFP_NOWAIT | __GFP_NOWARN, 0);
+ if (!batch)
+ return false;
+
+ tlb->batch_count++;
+ batch->next = NULL;
+ batch->nr = 0;
+ batch->max = MAX_GATHER_BATCH;
+
+ tlb->active->next = batch;
+ tlb->active = batch;
+
+ return true;
+}
+
+void arch_tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
+ unsigned long start, unsigned long end)
+{
+ tlb->mm = mm;
+
+ /* Is it from 0 to ~0? */
+ tlb->fullmm = !(start | (end+1));
+ tlb->need_flush_all = 0;
+ tlb->local.next = NULL;
+ tlb->local.nr = 0;
+ tlb->local.max = ARRAY_SIZE(tlb->__pages);
+ tlb->active = &tlb->local;
+ tlb->batch_count = 0;
+
+#ifdef CONFIG_HAVE_RCU_TABLE_FREE
+ tlb->batch = NULL;
+#endif
+ tlb->page_size = 0;
+
+ __tlb_reset_range(tlb);
+}
+
+void tlb_flush_mmu_free(struct mmu_gather *tlb)
+{
+ struct mmu_gather_batch *batch;
+
+#ifdef CONFIG_HAVE_RCU_TABLE_FREE
+ tlb_table_flush(tlb);
+#endif
+ for (batch = &tlb->local; batch && batch->nr; batch = batch->next) {
+ free_pages_and_swap_cache(batch->pages, batch->nr);
+ batch->nr = 0;
+ }
+ tlb->active = &tlb->local;
+}
+
+void tlb_flush_mmu(struct mmu_gather *tlb)
+{
+ tlb_flush_mmu_tlbonly(tlb);
+ tlb_flush_mmu_free(tlb);
+}
+
+/* tlb_finish_mmu
+ * Called at the end of the shootdown operation to free up any resources
+ * that were required.
+ */
+void arch_tlb_finish_mmu(struct mmu_gather *tlb,
+ unsigned long start, unsigned long end, bool force)
+{
+ struct mmu_gather_batch *batch, *next;
+
+ if (force) {
+ __tlb_reset_range(tlb);
+ __tlb_adjust_range(tlb, start, end - start);
+ }
+
+ tlb_flush_mmu(tlb);
+
+ /* keep the page table cache within bounds */
+ check_pgt_cache();
+
+ for (batch = tlb->local.next; batch; batch = next) {
+ next = batch->next;
+ free_pages((unsigned long)batch, 0);
+ }
+ tlb->local.next = NULL;
+}
+
+/* __tlb_remove_page
+ * Must perform the equivalent to __free_pte(pte_get_and_clear(ptep)), while
+ * handling the additional races in SMP caused by other CPUs caching valid
+ * mappings in their TLBs. Returns the number of free page slots left.
+ * When out of page slots we must call tlb_flush_mmu().
+ *returns true if the caller should flush.
+ */
+bool __tlb_remove_page_size(struct mmu_gather *tlb, struct page *page, int page_size)
+{
+ struct mmu_gather_batch *batch;
+
+ VM_BUG_ON(!tlb->end);
+ VM_WARN_ON(tlb->page_size != page_size);
+
+ batch = tlb->active;
+ /*
+ * Add the page and check if we are full. If so
+ * force a flush.
+ */
+ batch->pages[batch->nr++] = page;
+ if (batch->nr == batch->max) {
+ if (!tlb_next_batch(tlb))
+ return true;
+ batch = tlb->active;
+ }
+ VM_BUG_ON_PAGE(batch->nr > batch->max, page);
+
+ return false;
+}
+
+#endif /* HAVE_GENERIC_MMU_GATHER */
+
+#ifdef CONFIG_HAVE_RCU_TABLE_FREE
+
+/*
+ * See the comment near struct mmu_table_batch.
+ */
+
+/*
+ * If we want tlb_remove_table() to imply TLB invalidates.
+ */
+static inline void tlb_table_invalidate(struct mmu_gather *tlb)
+{
+#ifdef CONFIG_HAVE_RCU_TABLE_INVALIDATE
+ /*
+ * Invalidate page-table caches used by hardware walkers. Then we still
+ * need to RCU-sched wait while freeing the pages because software
+ * walkers can still be in-flight.
+ */
+ tlb_flush_mmu_tlbonly(tlb);
+#endif
+}
+
+static void tlb_remove_table_smp_sync(void *arg)
+{
+ /* Simply deliver the interrupt */
+}
+
+static void tlb_remove_table_one(void *table)
+{
+ /*
+ * This isn't an RCU grace period and hence the page-tables cannot be
+ * assumed to be actually RCU-freed.
+ *
+ * It is however sufficient for software page-table walkers that rely on
+ * IRQ disabling. See the comment near struct mmu_table_batch.
+ */
+ smp_call_function(tlb_remove_table_smp_sync, NULL, 1);
+ __tlb_remove_table(table);
+}
+
+static void tlb_remove_table_rcu(struct rcu_head *head)
+{
+ struct mmu_table_batch *batch;
+ int i;
+
+ batch = container_of(head, struct mmu_table_batch, rcu);
+
+ for (i = 0; i < batch->nr; i++)
+ __tlb_remove_table(batch->tables[i]);
+
+ free_page((unsigned long)batch);
+}
+
+void tlb_table_flush(struct mmu_gather *tlb)
+{
+ struct mmu_table_batch **batch = &tlb->batch;
+
+ if (*batch) {
+ tlb_table_invalidate(tlb);
+ call_rcu_sched(&(*batch)->rcu, tlb_remove_table_rcu);
+ *batch = NULL;
+ }
+}
+
+void tlb_remove_table(struct mmu_gather *tlb, void *table)
+{
+ struct mmu_table_batch **batch = &tlb->batch;
+
+ if (*batch == NULL) {
+ *batch = (struct mmu_table_batch *)__get_free_page(GFP_NOWAIT | __GFP_NOWARN);
+ if (*batch == NULL) {
+ tlb_table_invalidate(tlb);
+ tlb_remove_table_one(table);
+ return;
+ }
+ (*batch)->nr = 0;
+ }
+
+ (*batch)->tables[(*batch)->nr++] = table;
+ if ((*batch)->nr == MAX_TABLE_BATCH)
+ tlb_table_flush(tlb);
+}
+
+#endif /* CONFIG_HAVE_RCU_TABLE_FREE */
+
+/**
+ * tlb_gather_mmu - initialize an mmu_gather structure for page-table tear-down
+ * @tlb: the mmu_gather structure to initialize
+ * @mm: the mm_struct of the target address space
+ * @start: start of the region that will be removed from the page-table
+ * @end: end of the region that will be removed from the page-table
+ *
+ * Called to initialize an (on-stack) mmu_gather structure for page-table
+ * tear-down from @mm. The @start and @end are set to 0 and -1
+ * respectively when @mm is without users and we're going to destroy
+ * the full address space (exit/execve).
+ */
+void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm,
+ unsigned long start, unsigned long end)
+{
+ arch_tlb_gather_mmu(tlb, mm, start, end);
+ inc_tlb_flush_pending(tlb->mm);
+}
+
+void tlb_finish_mmu(struct mmu_gather *tlb,
+ unsigned long start, unsigned long end)
+{
+ /*
+ * If there are parallel threads are doing PTE changes on same range
+ * under non-exclusive lock(e.g., mmap_sem read-side) but defer TLB
+ * flush by batching, a thread has stable TLB entry can fail to flush
+ * the TLB by observing pte_none|!pte_dirty, for example so flush TLB
+ * forcefully if we detect parallel PTE batching threads.
+ */
+ bool force = mm_tlb_flush_nested(tlb->mm);
+
+ arch_tlb_finish_mmu(tlb, start, end, force);
+ dec_tlb_flush_pending(tlb->mm);
+}
static void move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd,
unsigned long old_addr, unsigned long old_end,
struct vm_area_struct *new_vma, pmd_t *new_pmd,
- unsigned long new_addr, bool need_rmap_locks, bool *need_flush)
+ unsigned long new_addr, bool need_rmap_locks)
{
struct mm_struct *mm = vma->vm_mm;
pte_t *old_pte, *new_pte, pte;
pte = ptep_get_and_clear(mm, old_addr, old_pte);
/*
- * If we are remapping a dirty PTE, make sure
+ * If we are remapping a valid PTE, make sure
* to flush TLB before we drop the PTL for the
- * old PTE or we may race with page_mkclean().
+ * PTE.
*
- * This check has to be done after we removed the
- * old PTE from page tables or another thread may
- * dirty it after the check and before the removal.
+ * NOTE! Both old and new PTL matter: the old one
+ * for racing with page_mkclean(), the new one to
+ * make sure the physical page stays valid until
+ * the TLB entry for the old mapping has been
+ * flushed.
*/
- if (pte_present(pte) && pte_dirty(pte))
+ if (pte_present(pte))
force_flush = true;
pte = move_pte(pte, new_vma->vm_page_prot, old_addr, new_addr);
pte = move_soft_dirty_pte(pte);
}
arch_leave_lazy_mmu_mode();
+ if (force_flush)
+ flush_tlb_range(vma, old_end - len, old_end);
if (new_ptl != old_ptl)
spin_unlock(new_ptl);
pte_unmap(new_pte - 1);
- if (force_flush)
- flush_tlb_range(vma, old_end - len, old_end);
- else
- *need_flush = true;
pte_unmap_unlock(old_pte - 1, old_ptl);
if (need_rmap_locks)
drop_rmap_locks(vma);
{
unsigned long extent, next, old_end;
pmd_t *old_pmd, *new_pmd;
- bool need_flush = false;
unsigned long mmun_start; /* For mmu_notifiers */
unsigned long mmun_end; /* For mmu_notifiers */
if (need_rmap_locks)
take_rmap_locks(vma);
moved = move_huge_pmd(vma, old_addr, new_addr,
- old_end, old_pmd, new_pmd,
- &need_flush);
+ old_end, old_pmd, new_pmd);
if (need_rmap_locks)
drop_rmap_locks(vma);
if (moved)
if (extent > next - new_addr)
extent = next - new_addr;
move_ptes(vma, old_pmd, old_addr, old_addr + extent, new_vma,
- new_pmd, new_addr, need_rmap_locks, &need_flush);
+ new_pmd, new_addr, need_rmap_locks);
}
- if (need_flush)
- flush_tlb_range(vma, old_end-len, old_addr);
mmu_notifier_invalidate_range_end(vma->vm_mm, mmun_start, mmun_end);
return PAGE_ALIGN(pages * sizeof(struct page)) >> PAGE_SHIFT;
}
-#ifdef CONFIG_NUMA_BALANCING
-static void pgdat_init_numabalancing(struct pglist_data *pgdat)
-{
- spin_lock_init(&pgdat->numabalancing_migrate_lock);
-}
-#else
-static void pgdat_init_numabalancing(struct pglist_data *pgdat) {}
-#endif
-
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
static void pgdat_init_split_queue(struct pglist_data *pgdat)
{
{
pgdat_resize_init(pgdat);
- pgdat_init_numabalancing(pgdat);
pgdat_init_split_queue(pgdat);
pgdat_init_kcompactd(pgdat);
goto out;
}
bio->bi_opf = REQ_OP_WRITE | REQ_SWAP | wbc_to_write_flags(wbc);
- bio_associate_blkcg_from_page(bio, page);
+ bio_associate_blkg_from_page(bio, page);
count_swpout_vm_event(page);
set_page_writeback(page);
unlock_page(page);
{
if (!chunk)
return;
+ pcpu_mem_free(chunk->md_blocks);
pcpu_mem_free(chunk->bound_map);
pcpu_mem_free(chunk->alloc_map);
pcpu_mem_free(chunk);
if (!info->pid)
return;
- tsk = pid_task(find_vpid(info->pid), PIDTYPE_PID);
- if (tsk)
+ tsk = get_pid_task(find_vpid(info->pid), PIDTYPE_PID);
+ if (tsk) {
force_sig(SIGKILL, tsk);
+ put_task_struct(tsk);
+ }
fput(info->pipe_to_umh);
fput(info->pipe_from_umh);
info->pid = 0;
}
EXPORT_SYMBOL(call_netdevice_notifiers);
+/**
+ * call_netdevice_notifiers_mtu - call all network notifier blocks
+ * @val: value passed unmodified to notifier function
+ * @dev: net_device pointer passed unmodified to notifier function
+ * @arg: additional u32 argument passed to the notifier function
+ *
+ * Call all network notifier blocks. Parameters and return value
+ * are as for raw_notifier_call_chain().
+ */
+static int call_netdevice_notifiers_mtu(unsigned long val,
+ struct net_device *dev, u32 arg)
+{
+ struct netdev_notifier_info_ext info = {
+ .info.dev = dev,
+ .ext.mtu = arg,
+ };
+
+ BUILD_BUG_ON(offsetof(struct netdev_notifier_info_ext, info) != 0);
+
+ return call_netdevice_notifiers_info(val, &info.info);
+}
+
#ifdef CONFIG_NET_INGRESS
static DEFINE_STATIC_KEY_FALSE(ingress_needed_key);
err = __dev_set_mtu(dev, new_mtu);
if (!err) {
- err = call_netdevice_notifiers(NETDEV_CHANGEMTU, dev);
+ err = call_netdevice_notifiers_mtu(NETDEV_CHANGEMTU, dev,
+ orig_mtu);
err = notifier_to_errno(err);
if (err) {
/* setting mtu back and notifying everyone again,
* so that they have a chance to revert changes.
*/
__dev_set_mtu(dev, orig_mtu);
- call_netdevice_notifiers(NETDEV_CHANGEMTU, dev);
+ call_netdevice_notifiers_mtu(NETDEV_CHANGEMTU, dev,
+ new_mtu);
}
}
return err;
struct genl_info *info,
union devlink_param_value *value)
{
+ int len;
+
if (param->type != DEVLINK_PARAM_TYPE_BOOL &&
!info->attrs[DEVLINK_ATTR_PARAM_VALUE_DATA])
return -EINVAL;
value->vu32 = nla_get_u32(info->attrs[DEVLINK_ATTR_PARAM_VALUE_DATA]);
break;
case DEVLINK_PARAM_TYPE_STRING:
- if (nla_len(info->attrs[DEVLINK_ATTR_PARAM_VALUE_DATA]) >
- DEVLINK_PARAM_MAX_STRING_VALUE)
+ len = strnlen(nla_data(info->attrs[DEVLINK_ATTR_PARAM_VALUE_DATA]),
+ nla_len(info->attrs[DEVLINK_ATTR_PARAM_VALUE_DATA]));
+ if (len == nla_len(info->attrs[DEVLINK_ATTR_PARAM_VALUE_DATA]) ||
+ len >= __DEVLINK_PARAM_MAX_STRING_VALUE)
return -EINVAL;
- value->vstr = nla_data(info->attrs[DEVLINK_ATTR_PARAM_VALUE_DATA]);
+ strcpy(value->vstr,
+ nla_data(info->attrs[DEVLINK_ATTR_PARAM_VALUE_DATA]));
break;
case DEVLINK_PARAM_TYPE_BOOL:
value->vbool = info->attrs[DEVLINK_ATTR_PARAM_VALUE_DATA] ?
return -EOPNOTSUPP;
if (cmode == DEVLINK_PARAM_CMODE_DRIVERINIT) {
- param_item->driverinit_value = value;
+ if (param->type == DEVLINK_PARAM_TYPE_STRING)
+ strcpy(param_item->driverinit_value.vstr, value.vstr);
+ else
+ param_item->driverinit_value = value;
param_item->driverinit_value_valid = true;
} else {
if (!param->set)
DEVLINK_PARAM_CMODE_DRIVERINIT))
return -EOPNOTSUPP;
- *init_val = param_item->driverinit_value;
+ if (param_item->param->type == DEVLINK_PARAM_TYPE_STRING)
+ strcpy(init_val->vstr, param_item->driverinit_value.vstr);
+ else
+ *init_val = param_item->driverinit_value;
return 0;
}
DEVLINK_PARAM_CMODE_DRIVERINIT))
return -EOPNOTSUPP;
- param_item->driverinit_value = init_val;
+ if (param_item->param->type == DEVLINK_PARAM_TYPE_STRING)
+ strcpy(param_item->driverinit_value.vstr, init_val.vstr);
+ else
+ param_item->driverinit_value = init_val;
param_item->driverinit_value_valid = true;
devlink_param_notify(devlink, param_item, DEVLINK_CMD_PARAM_NEW);
}
EXPORT_SYMBOL_GPL(devlink_param_value_changed);
+/**
+ * devlink_param_value_str_fill - Safely fill-up the string preventing
+ * from overflow of the preallocated buffer
+ *
+ * @dst_val: destination devlink_param_value
+ * @src: source buffer
+ */
+void devlink_param_value_str_fill(union devlink_param_value *dst_val,
+ const char *src)
+{
+ size_t len;
+
+ len = strlcpy(dst_val->vstr, src, __DEVLINK_PARAM_MAX_STRING_VALUE);
+ WARN_ON(len >= __DEVLINK_PARAM_MAX_STRING_VALUE);
+}
+EXPORT_SYMBOL_GPL(devlink_param_value_str_fill);
+
/**
* devlink_region_create - create a new address region
*
return -EINVAL;
}
+ if (info.cmd != cmd)
+ return -EINVAL;
+
if (info.cmd == ETHTOOL_GRXCLSRLALL) {
if (info.rule_cnt > 0) {
if (info.rule_cnt <= KMALLOC_MAX_SIZE / sizeof(u32))
return ret;
}
-static int ethtool_set_per_queue(struct net_device *dev, void __user *useraddr)
+static int ethtool_set_per_queue(struct net_device *dev,
+ void __user *useraddr, u32 sub_cmd)
{
struct ethtool_per_queue_op per_queue_opt;
if (copy_from_user(&per_queue_opt, useraddr, sizeof(per_queue_opt)))
return -EFAULT;
+ if (per_queue_opt.sub_command != sub_cmd)
+ return -EINVAL;
+
switch (per_queue_opt.sub_command) {
case ETHTOOL_GCOALESCE:
return ethtool_get_per_queue_coalesce(dev, useraddr, &per_queue_opt);
rc = ethtool_get_phy_stats(dev, useraddr);
break;
case ETHTOOL_PERQUEUE:
- rc = ethtool_set_per_queue(dev, useraddr);
+ rc = ethtool_set_per_queue(dev, useraddr, sub_cmd);
break;
case ETHTOOL_GLINKSETTINGS:
rc = ethtool_get_link_ksettings(dev, useraddr);
neigh->nud_state = new;
err = 0;
notify = old & NUD_VALID;
- if (((old & (NUD_INCOMPLETE | NUD_PROBE)) ||
- (flags & NEIGH_UPDATE_F_ADMIN)) &&
+ if ((old & (NUD_INCOMPLETE | NUD_PROBE)) &&
(new & NUD_FAILED)) {
neigh_invalidate(neigh);
notify = 1;
/* It is up to the caller to keep npinfo alive. */
struct netpoll_info *npinfo;
- rcu_read_lock_bh();
lockdep_assert_irqs_disabled();
npinfo = rcu_dereference_bh(np->dev->npinfo);
skb_queue_tail(&npinfo->txq, skb);
schedule_delayed_work(&npinfo->tx_work,0);
}
- rcu_read_unlock_bh();
}
EXPORT_SYMBOL(netpoll_send_skb_on_dev);
if (skb->ip_summed == CHECKSUM_COMPLETE) {
int delta = skb->len - len;
- skb->csum = csum_sub(skb->csum,
- skb_checksum(skb, len, delta, 0));
+ skb->csum = csum_block_sub(skb->csum,
+ skb_checksum(skb, len, delta, 0),
+ len);
}
return __pskb_trim(skb, len);
}
*/
bool skb_partial_csum_set(struct sk_buff *skb, u16 start, u16 off)
{
- if (unlikely(start > skb_headlen(skb)) ||
- unlikely((int)start + off > skb_headlen(skb) - 2)) {
- net_warn_ratelimited("bad partial csum: csum=%u/%u len=%u\n",
- start, off, skb_headlen(skb));
+ u32 csum_end = (u32)start + (u32)off + sizeof(__sum16);
+ u32 csum_start = skb_headroom(skb) + (u32)start;
+
+ if (unlikely(csum_start > U16_MAX || csum_end > skb_headlen(skb))) {
+ net_warn_ratelimited("bad partial csum: csum=%u/%u headroom=%u headlen=%u\n",
+ start, off, skb_headroom(skb), skb_headlen(skb));
return false;
}
skb->ip_summed = CHECKSUM_PARTIAL;
- skb->csum_start = skb_headroom(skb) + start;
+ skb->csum_start = csum_start;
skb->csum_offset = off;
skb_set_transport_header(skb, start);
return true;
static int fib_netdev_event(struct notifier_block *this, unsigned long event, void *ptr)
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- struct netdev_notifier_changeupper_info *info;
+ struct netdev_notifier_changeupper_info *upper_info = ptr;
+ struct netdev_notifier_info_ext *info_ext = ptr;
struct in_device *in_dev;
struct net *net = dev_net(dev);
unsigned int flags;
fib_sync_up(dev, RTNH_F_LINKDOWN);
else
fib_sync_down_dev(dev, event, false);
- /* fall through */
+ rt_cache_flush(net);
+ break;
case NETDEV_CHANGEMTU:
+ fib_sync_mtu(dev, info_ext->ext.mtu);
rt_cache_flush(net);
break;
case NETDEV_CHANGEUPPER:
- info = ptr;
+ upper_info = ptr;
/* flush all routes if dev is linked to or unlinked from
* an L3 master device (e.g., VRF)
*/
- if (info->upper_dev && netif_is_l3_master(info->upper_dev))
+ if (upper_info->upper_dev &&
+ netif_is_l3_master(upper_info->upper_dev))
fib_disable_ip(dev, NETDEV_DOWN, true);
break;
}
return NOTIFY_DONE;
}
+/* Update the PMTU of exceptions when:
+ * - the new MTU of the first hop becomes smaller than the PMTU
+ * - the old MTU was the same as the PMTU, and it limited discovery of
+ * larger MTUs on the path. With that limit raised, we can now
+ * discover larger MTUs
+ * A special case is locked exceptions, for which the PMTU is smaller
+ * than the minimal accepted PMTU:
+ * - if the new MTU is greater than the PMTU, don't make any change
+ * - otherwise, unlock and set PMTU
+ */
+static void nh_update_mtu(struct fib_nh *nh, u32 new, u32 orig)
+{
+ struct fnhe_hash_bucket *bucket;
+ int i;
+
+ bucket = rcu_dereference_protected(nh->nh_exceptions, 1);
+ if (!bucket)
+ return;
+
+ for (i = 0; i < FNHE_HASH_SIZE; i++) {
+ struct fib_nh_exception *fnhe;
+
+ for (fnhe = rcu_dereference_protected(bucket[i].chain, 1);
+ fnhe;
+ fnhe = rcu_dereference_protected(fnhe->fnhe_next, 1)) {
+ if (fnhe->fnhe_mtu_locked) {
+ if (new <= fnhe->fnhe_pmtu) {
+ fnhe->fnhe_pmtu = new;
+ fnhe->fnhe_mtu_locked = false;
+ }
+ } else if (new < fnhe->fnhe_pmtu ||
+ orig == fnhe->fnhe_pmtu) {
+ fnhe->fnhe_pmtu = new;
+ }
+ }
+ }
+}
+
+void fib_sync_mtu(struct net_device *dev, u32 orig_mtu)
+{
+ unsigned int hash = fib_devindex_hashfn(dev->ifindex);
+ struct hlist_head *head = &fib_info_devhash[hash];
+ struct fib_nh *nh;
+
+ hlist_for_each_entry(nh, head, nh_hash) {
+ if (nh->nh_dev == dev)
+ nh_update_mtu(nh, dev->mtu, orig_mtu);
+ }
+}
+
/* Event force Flags Description
* NETDEV_CHANGE 0 LINKDOWN Carrier OFF, not for scope host
* NETDEV_DOWN 0 LINKDOWN|DEAD Link down, not for scope host
next_entry:
e++;
}
- e = 0;
- s_e = 0;
spin_lock_bh(lock);
list_for_each_entry(mfc, &mrt->mfc_unres_queue, list) {
static void __ip_rt_update_pmtu(struct rtable *rt, struct flowi4 *fl4, u32 mtu)
{
struct dst_entry *dst = &rt->dst;
+ u32 old_mtu = ipv4_mtu(dst);
struct fib_result res;
bool lock = false;
if (ip_mtu_locked(dst))
return;
- if (ipv4_mtu(dst) < mtu)
+ if (old_mtu < mtu)
return;
if (mtu < ip_rt_min_pmtu) {
lock = true;
- mtu = ip_rt_min_pmtu;
+ mtu = min(old_mtu, ip_rt_min_pmtu);
}
- if (rt->rt_pmtu == mtu &&
+ if (rt->rt_pmtu == mtu && !lock &&
time_before(jiffies, dst->expires - ip_rt_mtu_expires / 2))
return;
*err = error;
return NULL;
}
-EXPORT_SYMBOL_GPL(__skb_recv_udp);
+EXPORT_SYMBOL(__skb_recv_udp);
/*
* This should be easy, if there is something there we
/* unicast address incl. temp addr */
list_for_each_entry(ifa, &idev->addr_list, if_list) {
- if (++ip_idx < s_ip_idx)
- continue;
+ if (ip_idx < s_ip_idx)
+ goto next;
err = inet6_fill_ifaddr(skb, ifa,
NETLINK_CB(cb->skb).portid,
cb->nlh->nlmsg_seq,
if (err < 0)
break;
nl_dump_check_consistent(cb, nlmsg_hdr(skb));
+next:
+ ip_idx++;
}
break;
}
*ppcpu_rt = NULL;
}
}
+
+ free_percpu(f6i->rt6i_pcpu);
}
lwtstate_put(f6i->fib6_nh.nh_lwtstate);
}
skb_dst_set(skb, dst);
- if (encap_limit >= 0) {
- init_tel_txopt(&opt, encap_limit);
- ipv6_push_frag_opts(skb, &opt.ops, &proto);
- }
-
if (hop_limit == 0) {
if (skb->protocol == htons(ETH_P_IP))
hop_limit = ip_hdr(skb)->ttl;
if (err)
return err;
+ if (encap_limit >= 0) {
+ init_tel_txopt(&opt, encap_limit);
+ ipv6_push_frag_opts(skb, &opt.ops, &proto);
+ }
+
skb_push(skb, sizeof(struct ipv6hdr));
skb_reset_network_header(skb);
ipv6h = ipv6_hdr(skb);
{
int err;
- /* callers have the socket lock and rtnl lock
- * so no other readers or writers of iml or its sflist
- */
+ write_lock_bh(&iml->sflock);
if (!iml->sflist) {
/* any-source empty exclude case */
- return ip6_mc_del_src(idev, &iml->addr, iml->sfmode, 0, NULL, 0);
+ err = ip6_mc_del_src(idev, &iml->addr, iml->sfmode, 0, NULL, 0);
+ } else {
+ err = ip6_mc_del_src(idev, &iml->addr, iml->sfmode,
+ iml->sflist->sl_count, iml->sflist->sl_addr, 0);
+ sock_kfree_s(sk, iml->sflist, IP6_SFLSIZE(iml->sflist->sl_max));
+ iml->sflist = NULL;
}
- err = ip6_mc_del_src(idev, &iml->addr, iml->sfmode,
- iml->sflist->sl_count, iml->sflist->sl_addr, 0);
- sock_kfree_s(sk, iml->sflist, IP6_SFLSIZE(iml->sflist->sl_max));
- iml->sflist = NULL;
+ write_unlock_bh(&iml->sflock);
return err;
}
static void rt6_probe(struct fib6_info *rt)
{
- struct __rt6_probe_work *work;
+ struct __rt6_probe_work *work = NULL;
const struct in6_addr *nh_gw;
struct neighbour *neigh;
struct net_device *dev;
+ struct inet6_dev *idev;
/*
* Okay, this does not seem to be appropriate
nh_gw = &rt->fib6_nh.nh_gw;
dev = rt->fib6_nh.nh_dev;
rcu_read_lock_bh();
+ idev = __in6_dev_get(dev);
neigh = __ipv6_neigh_lookup_noref(dev, nh_gw);
if (neigh) {
- struct inet6_dev *idev;
-
if (neigh->nud_state & NUD_VALID)
goto out;
- idev = __in6_dev_get(dev);
- work = NULL;
write_lock(&neigh->lock);
if (!(neigh->nud_state & NUD_VALID) &&
time_after(jiffies,
__neigh_set_probe_once(neigh);
}
write_unlock(&neigh->lock);
- } else {
+ } else if (time_after(jiffies, rt->last_probe +
+ idev->cnf.rtr_probe_interval)) {
work = kmalloc(sizeof(*work), GFP_ATOMIC);
}
if (work) {
+ rt->last_probe = jiffies;
INIT_WORK(&work->work, rt6_probe_deferred);
work->target = *nh_gw;
dev_hold(dev);
ret = udpv6_queue_rcv_skb(sk, skb);
- /* a return value > 0 means to resubmit the input, but
- * it wants the return to be -protocol, or 0
- */
+ /* a return value > 0 means to resubmit the input */
if (ret > 0)
- return -ret;
+ return ret;
return 0;
}
fl6->daddr = reverse ? hdr->saddr : hdr->daddr;
fl6->saddr = reverse ? hdr->daddr : hdr->saddr;
- while (nh + offset + 1 < skb->data ||
- pskb_may_pull(skb, nh + offset + 1 - skb->data)) {
+ while (nh + offset + sizeof(*exthdr) < skb->data ||
+ pskb_may_pull(skb, nh + offset + sizeof(*exthdr) - skb->data)) {
nh = skb_network_header(skb);
exthdr = (struct ipv6_opt_hdr *)(nh + offset);
llc_sk(sk)->sap = sap;
spin_lock_bh(&sap->sk_lock);
+ sock_set_flag(sk, SOCK_RCU_FREE);
sap->sk_count++;
sk_nulls_add_node_rcu(sk, laddr_hb);
hlist_add_head(&llc->dev_hash_node, dev_hb);
return ret;
}
-static int rds_send_mprds_hash(struct rds_sock *rs, struct rds_connection *conn)
+static int rds_send_mprds_hash(struct rds_sock *rs,
+ struct rds_connection *conn, int nonblock)
{
int hash;
* used. But if we are interrupted, we have to use the zero
* c_path in case the connection ends up being non-MP capable.
*/
- if (conn->c_npaths == 0)
+ if (conn->c_npaths == 0) {
+ /* Cannot wait for the connection be made, so just use
+ * the base c_path.
+ */
+ if (nonblock)
+ return 0;
if (wait_event_interruptible(conn->c_hs_waitq,
conn->c_npaths != 0))
hash = 0;
+ }
if (conn->c_npaths == 1)
hash = 0;
}
}
if (conn->c_trans->t_mp_capable)
- cpath = &conn->c_path[rds_send_mprds_hash(rs, conn)];
+ cpath = &conn->c_path[rds_send_mprds_hash(rs, conn, nonblock)];
else
cpath = &conn->c_path[0];
/* calculated RTT cache */
#define RXRPC_RTT_CACHE_SIZE 32
+ spinlock_t rtt_input_lock; /* RTT lock for input routine */
ktime_t rtt_last_req; /* Time of last RTT request */
u64 rtt; /* Current RTT estimate (in nS) */
u64 rtt_sum; /* Sum of cache contents */
spinlock_t state_lock; /* state-change lock */
enum rxrpc_conn_cache_state cache_state;
enum rxrpc_conn_proto_state state; /* current state of connection */
- u32 local_abort; /* local abort code */
- u32 remote_abort; /* remote abort code */
+ u32 abort_code; /* Abort code of connection abort */
int debug_id; /* debug ID for printks */
atomic_t serial; /* packet serial number counter */
unsigned int hi_serial; /* highest serial number received */
u32 security_nonce; /* response re-use preventer */
- u16 service_id; /* Service ID, possibly upgraded */
+ u32 service_id; /* Service ID, possibly upgraded */
u8 size_align; /* data size alignment (for security) */
u8 security_size; /* security header size */
u8 security_ix; /* security type */
u8 out_clientflag; /* RXRPC_CLIENT_INITIATED if we are client */
+ short error; /* Local error code */
};
static inline bool rxrpc_to_server(const struct rxrpc_skb_priv *sp)
bool tx_phase; /* T if transmission phase, F if receive phase */
u8 nr_jumbo_bad; /* Number of jumbo dups/exceeds-windows */
+ spinlock_t input_lock; /* Lock for packet input to this call */
+
/* receive-phase ACK management */
u8 ackr_reason; /* reason to ACK */
u16 ackr_skew; /* skew on packet being ACK'd */
void rxrpc_discard_prealloc(struct rxrpc_sock *);
struct rxrpc_call *rxrpc_new_incoming_call(struct rxrpc_local *,
struct rxrpc_sock *,
- struct rxrpc_peer *,
- struct rxrpc_connection *,
struct sk_buff *);
void rxrpc_accept_incoming_calls(struct rxrpc_local *);
struct rxrpc_call *rxrpc_accept_call(struct rxrpc_sock *, unsigned long,
extern struct idr rxrpc_client_conn_ids;
void rxrpc_destroy_client_conn_ids(void);
-int rxrpc_connect_call(struct rxrpc_call *, struct rxrpc_conn_parameters *,
- struct sockaddr_rxrpc *, gfp_t);
+int rxrpc_connect_call(struct rxrpc_sock *, struct rxrpc_call *,
+ struct rxrpc_conn_parameters *, struct sockaddr_rxrpc *,
+ gfp_t);
void rxrpc_expose_client_call(struct rxrpc_call *);
void rxrpc_disconnect_client_call(struct rxrpc_call *);
void rxrpc_put_client_conn(struct rxrpc_connection *);
/*
* input.c
*/
-void rxrpc_data_ready(struct sock *);
+int rxrpc_input_packet(struct sock *, struct sk_buff *);
/*
* insecure.c
*/
struct rxrpc_peer *rxrpc_lookup_peer_rcu(struct rxrpc_local *,
const struct sockaddr_rxrpc *);
-struct rxrpc_peer *rxrpc_lookup_peer(struct rxrpc_local *,
+struct rxrpc_peer *rxrpc_lookup_peer(struct rxrpc_sock *, struct rxrpc_local *,
struct sockaddr_rxrpc *, gfp_t);
struct rxrpc_peer *rxrpc_alloc_peer(struct rxrpc_local *, gfp_t);
-void rxrpc_new_incoming_peer(struct rxrpc_local *, struct rxrpc_peer *);
+void rxrpc_new_incoming_peer(struct rxrpc_sock *, struct rxrpc_local *,
+ struct rxrpc_peer *);
void rxrpc_destroy_all_peers(struct rxrpc_net *);
struct rxrpc_peer *rxrpc_get_peer(struct rxrpc_peer *);
struct rxrpc_peer *rxrpc_get_peer_maybe(struct rxrpc_peer *);
(peer_tail + 1) &
(RXRPC_BACKLOG_MAX - 1));
- rxrpc_new_incoming_peer(local, peer);
+ rxrpc_new_incoming_peer(rx, local, peer);
}
/* Now allocate and set up the connection */
*/
struct rxrpc_call *rxrpc_new_incoming_call(struct rxrpc_local *local,
struct rxrpc_sock *rx,
- struct rxrpc_peer *peer,
- struct rxrpc_connection *conn,
struct sk_buff *skb)
{
struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
+ struct rxrpc_connection *conn;
+ struct rxrpc_peer *peer = NULL;
struct rxrpc_call *call;
_enter("");
goto out;
}
+ /* The peer, connection and call may all have sprung into existence due
+ * to a duplicate packet being handled on another CPU in parallel, so
+ * we have to recheck the routing. However, we're now holding
+ * rx->incoming_lock, so the values should remain stable.
+ */
+ conn = rxrpc_find_connection_rcu(local, skb, &peer);
+
call = rxrpc_alloc_incoming_call(rx, local, peer, conn, skb);
if (!call) {
skb->mark = RXRPC_SKB_MARK_REJECT_BUSY;
case RXRPC_CONN_SERVICE:
write_lock(&call->state_lock);
- if (rx->discard_new_call)
- call->state = RXRPC_CALL_SERVER_RECV_REQUEST;
- else
- call->state = RXRPC_CALL_SERVER_ACCEPTING;
+ if (call->state < RXRPC_CALL_COMPLETE) {
+ if (rx->discard_new_call)
+ call->state = RXRPC_CALL_SERVER_RECV_REQUEST;
+ else
+ call->state = RXRPC_CALL_SERVER_ACCEPTING;
+ }
write_unlock(&call->state_lock);
break;
case RXRPC_CONN_REMOTELY_ABORTED:
rxrpc_set_call_completion(call, RXRPC_CALL_REMOTELY_ABORTED,
- conn->remote_abort, -ECONNABORTED);
+ conn->abort_code, conn->error);
break;
case RXRPC_CONN_LOCALLY_ABORTED:
rxrpc_abort_call("CON", call, sp->hdr.seq,
- conn->local_abort, -ECONNABORTED);
+ conn->abort_code, conn->error);
break;
default:
BUG();
init_waitqueue_head(&call->waitq);
spin_lock_init(&call->lock);
spin_lock_init(&call->notify_lock);
+ spin_lock_init(&call->input_lock);
rwlock_init(&call->state_lock);
atomic_set(&call->usage, 1);
call->debug_id = debug_id;
/* Set up or get a connection record and set the protocol parameters,
* including channel number and call ID.
*/
- ret = rxrpc_connect_call(call, cp, srx, gfp);
+ ret = rxrpc_connect_call(rx, call, cp, srx, gfp);
if (ret < 0)
goto error;
/* Set up or get a connection record and set the protocol parameters,
* including channel number and call ID.
*/
- ret = rxrpc_connect_call(call, cp, srx, gfp);
+ ret = rxrpc_connect_call(rx, call, cp, srx, gfp);
if (ret < 0)
goto error;
* If we return with a connection, the call will be on its waiting list. It's
* left to the caller to assign a channel and wake up the call.
*/
-static int rxrpc_get_client_conn(struct rxrpc_call *call,
+static int rxrpc_get_client_conn(struct rxrpc_sock *rx,
+ struct rxrpc_call *call,
struct rxrpc_conn_parameters *cp,
struct sockaddr_rxrpc *srx,
gfp_t gfp)
_enter("{%d,%lx},", call->debug_id, call->user_call_ID);
- cp->peer = rxrpc_lookup_peer(cp->local, srx, gfp);
+ cp->peer = rxrpc_lookup_peer(rx, cp->local, srx, gfp);
if (!cp->peer)
goto error;
* find a connection for a call
* - called in process context with IRQs enabled
*/
-int rxrpc_connect_call(struct rxrpc_call *call,
+int rxrpc_connect_call(struct rxrpc_sock *rx,
+ struct rxrpc_call *call,
struct rxrpc_conn_parameters *cp,
struct sockaddr_rxrpc *srx,
gfp_t gfp)
rxrpc_discard_expired_client_conns(&rxnet->client_conn_reaper);
rxrpc_cull_active_client_conns(rxnet);
- ret = rxrpc_get_client_conn(call, cp, srx, gfp);
+ ret = rxrpc_get_client_conn(rx, call, cp, srx, gfp);
if (ret < 0)
goto out;
switch (chan->last_type) {
case RXRPC_PACKET_TYPE_ABORT:
- _proto("Tx ABORT %%%u { %d } [re]", serial, conn->local_abort);
+ _proto("Tx ABORT %%%u { %d } [re]", serial, conn->abort_code);
break;
case RXRPC_PACKET_TYPE_ACK:
trace_rxrpc_tx_ack(chan->call_debug_id, serial,
* pass a connection-level abort onto all calls on that connection
*/
static void rxrpc_abort_calls(struct rxrpc_connection *conn,
- enum rxrpc_call_completion compl,
- u32 abort_code, int error)
+ enum rxrpc_call_completion compl)
{
struct rxrpc_call *call;
int i;
- _enter("{%d},%x", conn->debug_id, abort_code);
+ _enter("{%d},%x", conn->debug_id, conn->abort_code);
spin_lock(&conn->channel_lock);
trace_rxrpc_abort(call->debug_id,
"CON", call->cid,
call->call_id, 0,
- abort_code, error);
+ conn->abort_code,
+ conn->error);
if (rxrpc_set_call_completion(call, compl,
- abort_code, error))
+ conn->abort_code,
+ conn->error))
rxrpc_notify_socket(call);
}
}
return 0;
}
+ conn->error = error;
+ conn->abort_code = abort_code;
conn->state = RXRPC_CONN_LOCALLY_ABORTED;
spin_unlock_bh(&conn->state_lock);
- rxrpc_abort_calls(conn, RXRPC_CALL_LOCALLY_ABORTED, abort_code, error);
+ rxrpc_abort_calls(conn, RXRPC_CALL_LOCALLY_ABORTED);
msg.msg_name = &conn->params.peer->srx.transport;
msg.msg_namelen = conn->params.peer->srx.transport_len;
whdr._rsvd = 0;
whdr.serviceId = htons(conn->service_id);
- word = htonl(conn->local_abort);
+ word = htonl(conn->abort_code);
iov[0].iov_base = &whdr;
iov[0].iov_len = sizeof(whdr);
serial = atomic_inc_return(&conn->serial);
whdr.serial = htonl(serial);
- _proto("Tx CONN ABORT %%%u { %d }", serial, conn->local_abort);
+ _proto("Tx CONN ABORT %%%u { %d }", serial, conn->abort_code);
ret = kernel_sendmsg(conn->params.local->socket, &msg, iov, 2, len);
if (ret < 0) {
abort_code = ntohl(wtmp);
_proto("Rx ABORT %%%u { ac=%d }", sp->hdr.serial, abort_code);
+ conn->error = -ECONNABORTED;
+ conn->abort_code = abort_code;
conn->state = RXRPC_CONN_REMOTELY_ABORTED;
- rxrpc_abort_calls(conn, RXRPC_CALL_REMOTELY_ABORTED,
- abort_code, -ECONNABORTED);
+ rxrpc_abort_calls(conn, RXRPC_CALL_REMOTELY_ABORTED);
return -ECONNABORTED;
case RXRPC_PACKET_TYPE_CHALLENGE:
/*
* Apply a hard ACK by advancing the Tx window.
*/
-static void rxrpc_rotate_tx_window(struct rxrpc_call *call, rxrpc_seq_t to,
+static bool rxrpc_rotate_tx_window(struct rxrpc_call *call, rxrpc_seq_t to,
struct rxrpc_ack_summary *summary)
{
struct sk_buff *skb, *list = NULL;
+ bool rot_last = false;
int ix;
u8 annotation;
skb->next = list;
list = skb;
- if (annotation & RXRPC_TX_ANNO_LAST)
+ if (annotation & RXRPC_TX_ANNO_LAST) {
set_bit(RXRPC_CALL_TX_LAST, &call->flags);
+ rot_last = true;
+ }
if ((annotation & RXRPC_TX_ANNO_MASK) != RXRPC_TX_ANNO_ACK)
summary->nr_rot_new_acks++;
}
spin_unlock(&call->lock);
- trace_rxrpc_transmit(call, (test_bit(RXRPC_CALL_TX_LAST, &call->flags) ?
+ trace_rxrpc_transmit(call, (rot_last ?
rxrpc_transmit_rotate_last :
rxrpc_transmit_rotate));
wake_up(&call->waitq);
skb->next = NULL;
rxrpc_free_skb(skb, rxrpc_skb_tx_freed);
}
+
+ return rot_last;
}
/*
static bool rxrpc_end_tx_phase(struct rxrpc_call *call, bool reply_begun,
const char *abort_why)
{
+ unsigned int state;
ASSERT(test_bit(RXRPC_CALL_TX_LAST, &call->flags));
write_lock(&call->state_lock);
- switch (call->state) {
+ state = call->state;
+ switch (state) {
case RXRPC_CALL_CLIENT_SEND_REQUEST:
case RXRPC_CALL_CLIENT_AWAIT_REPLY:
if (reply_begun)
- call->state = RXRPC_CALL_CLIENT_RECV_REPLY;
+ call->state = state = RXRPC_CALL_CLIENT_RECV_REPLY;
else
- call->state = RXRPC_CALL_CLIENT_AWAIT_REPLY;
+ call->state = state = RXRPC_CALL_CLIENT_AWAIT_REPLY;
break;
case RXRPC_CALL_SERVER_AWAIT_ACK:
__rxrpc_call_completed(call);
rxrpc_notify_socket(call);
+ state = call->state;
break;
default:
}
write_unlock(&call->state_lock);
- if (call->state == RXRPC_CALL_CLIENT_AWAIT_REPLY) {
+ if (state == RXRPC_CALL_CLIENT_AWAIT_REPLY)
trace_rxrpc_transmit(call, rxrpc_transmit_await_reply);
- } else {
+ else
trace_rxrpc_transmit(call, rxrpc_transmit_end);
- }
_leave(" = ok");
return true;
trace_rxrpc_timer(call, rxrpc_timer_init_for_reply, now);
}
- if (!test_bit(RXRPC_CALL_TX_LAST, &call->flags))
- rxrpc_rotate_tx_window(call, top, &summary);
if (!test_bit(RXRPC_CALL_TX_LAST, &call->flags)) {
- rxrpc_proto_abort("TXL", call, top);
- return false;
+ if (!rxrpc_rotate_tx_window(call, top, &summary)) {
+ rxrpc_proto_abort("TXL", call, top);
+ return false;
+ }
}
if (!rxrpc_end_tx_phase(call, true, "ETD"))
return false;
}
}
+ spin_lock(&call->input_lock);
+
/* Received data implicitly ACKs all of the request packets we sent
* when we're acting as a client.
*/
if ((state == RXRPC_CALL_CLIENT_SEND_REQUEST ||
state == RXRPC_CALL_CLIENT_AWAIT_REPLY) &&
!rxrpc_receiving_reply(call))
- return;
+ goto unlock;
call->ackr_prev_seq = seq;
if (flags & RXRPC_LAST_PACKET) {
if (test_bit(RXRPC_CALL_RX_LAST, &call->flags) &&
- seq != call->rx_top)
- return rxrpc_proto_abort("LSN", call, seq);
+ seq != call->rx_top) {
+ rxrpc_proto_abort("LSN", call, seq);
+ goto unlock;
+ }
} else {
if (test_bit(RXRPC_CALL_RX_LAST, &call->flags) &&
- after_eq(seq, call->rx_top))
- return rxrpc_proto_abort("LSA", call, seq);
+ after_eq(seq, call->rx_top)) {
+ rxrpc_proto_abort("LSA", call, seq);
+ goto unlock;
+ }
}
trace_rxrpc_rx_data(call->debug_id, seq, serial, flags, annotation);
skip:
offset += len;
if (flags & RXRPC_JUMBO_PACKET) {
- if (skb_copy_bits(skb, offset, &flags, 1) < 0)
- return rxrpc_proto_abort("XJF", call, seq);
+ if (skb_copy_bits(skb, offset, &flags, 1) < 0) {
+ rxrpc_proto_abort("XJF", call, seq);
+ goto unlock;
+ }
offset += sizeof(struct rxrpc_jumbo_header);
seq++;
serial++;
trace_rxrpc_notify_socket(call->debug_id, serial);
rxrpc_notify_socket(call);
}
+
+unlock:
+ spin_unlock(&call->input_lock);
_leave(" [queued]");
}
ping_time = call->ping_time;
smp_rmb();
- ping_serial = call->ping_serial;
+ ping_serial = READ_ONCE(call->ping_serial);
if (orig_serial == call->acks_lost_ping)
rxrpc_input_check_for_lost_ack(call);
- if (!test_bit(RXRPC_CALL_PINGING, &call->flags) ||
- before(orig_serial, ping_serial))
+ if (before(orig_serial, ping_serial) ||
+ !test_and_clear_bit(RXRPC_CALL_PINGING, &call->flags))
return;
- clear_bit(RXRPC_CALL_PINGING, &call->flags);
if (after(orig_serial, ping_serial))
return;
rxrpc_propose_ack_respond_to_ack);
}
+ /* Discard any out-of-order or duplicate ACKs. */
+ if (before_eq(sp->hdr.serial, call->acks_latest))
+ return;
+
+ buf.info.rxMTU = 0;
ioffset = offset + nr_acks + 3;
- if (skb->len >= ioffset + sizeof(buf.info)) {
- if (skb_copy_bits(skb, ioffset, &buf.info, sizeof(buf.info)) < 0)
- return rxrpc_proto_abort("XAI", call, 0);
+ if (skb->len >= ioffset + sizeof(buf.info) &&
+ skb_copy_bits(skb, ioffset, &buf.info, sizeof(buf.info)) < 0)
+ return rxrpc_proto_abort("XAI", call, 0);
+
+ spin_lock(&call->input_lock);
+
+ /* Discard any out-of-order or duplicate ACKs. */
+ if (before_eq(sp->hdr.serial, call->acks_latest))
+ goto out;
+ call->acks_latest_ts = skb->tstamp;
+ call->acks_latest = sp->hdr.serial;
+
+ /* Parse rwind and mtu sizes if provided. */
+ if (buf.info.rxMTU)
rxrpc_input_ackinfo(call, skb, &buf.info);
- }
- if (first_soft_ack == 0)
- return rxrpc_proto_abort("AK0", call, 0);
+ if (first_soft_ack == 0) {
+ rxrpc_proto_abort("AK0", call, 0);
+ goto out;
+ }
/* Ignore ACKs unless we are or have just been transmitting. */
switch (READ_ONCE(call->state)) {
case RXRPC_CALL_SERVER_AWAIT_ACK:
break;
default:
- return;
- }
-
- /* Discard any out-of-order or duplicate ACKs. */
- if (before_eq(sp->hdr.serial, call->acks_latest)) {
- _debug("discard ACK %d <= %d",
- sp->hdr.serial, call->acks_latest);
- return;
+ goto out;
}
- call->acks_latest_ts = skb->tstamp;
- call->acks_latest = sp->hdr.serial;
if (before(hard_ack, call->tx_hard_ack) ||
- after(hard_ack, call->tx_top))
- return rxrpc_proto_abort("AKW", call, 0);
- if (nr_acks > call->tx_top - hard_ack)
- return rxrpc_proto_abort("AKN", call, 0);
+ after(hard_ack, call->tx_top)) {
+ rxrpc_proto_abort("AKW", call, 0);
+ goto out;
+ }
+ if (nr_acks > call->tx_top - hard_ack) {
+ rxrpc_proto_abort("AKN", call, 0);
+ goto out;
+ }
- if (after(hard_ack, call->tx_hard_ack))
- rxrpc_rotate_tx_window(call, hard_ack, &summary);
+ if (after(hard_ack, call->tx_hard_ack)) {
+ if (rxrpc_rotate_tx_window(call, hard_ack, &summary)) {
+ rxrpc_end_tx_phase(call, false, "ETA");
+ goto out;
+ }
+ }
if (nr_acks > 0) {
- if (skb_copy_bits(skb, offset, buf.acks, nr_acks) < 0)
- return rxrpc_proto_abort("XSA", call, 0);
+ if (skb_copy_bits(skb, offset, buf.acks, nr_acks) < 0) {
+ rxrpc_proto_abort("XSA", call, 0);
+ goto out;
+ }
rxrpc_input_soft_acks(call, buf.acks, first_soft_ack, nr_acks,
&summary);
}
- if (test_bit(RXRPC_CALL_TX_LAST, &call->flags)) {
- rxrpc_end_tx_phase(call, false, "ETA");
- return;
- }
-
if (call->rxtx_annotations[call->tx_top & RXRPC_RXTX_BUFF_MASK] &
RXRPC_TX_ANNO_LAST &&
summary.nr_acks == call->tx_top - hard_ack &&
false, true,
rxrpc_propose_ack_ping_for_lost_reply);
- return rxrpc_congestion_management(call, skb, &summary, acked_serial);
+ rxrpc_congestion_management(call, skb, &summary, acked_serial);
+out:
+ spin_unlock(&call->input_lock);
}
/*
_proto("Rx ACKALL %%%u", sp->hdr.serial);
- rxrpc_rotate_tx_window(call, call->tx_top, &summary);
- if (test_bit(RXRPC_CALL_TX_LAST, &call->flags))
+ spin_lock(&call->input_lock);
+
+ if (rxrpc_rotate_tx_window(call, call->tx_top, &summary))
rxrpc_end_tx_phase(call, false, "ETL");
+
+ spin_unlock(&call->input_lock);
}
/*
}
/*
- * Handle a new call on a channel implicitly completing the preceding call on
- * that channel.
+ * Handle a new service call on a channel implicitly completing the preceding
+ * call on that channel. This does not apply to client conns.
*
* TODO: If callNumber > call_id + 1, renegotiate security.
*/
-static void rxrpc_input_implicit_end_call(struct rxrpc_connection *conn,
+static void rxrpc_input_implicit_end_call(struct rxrpc_sock *rx,
+ struct rxrpc_connection *conn,
struct rxrpc_call *call)
{
switch (READ_ONCE(call->state)) {
case RXRPC_CALL_SERVER_AWAIT_ACK:
rxrpc_call_completed(call);
- break;
+ /* Fall through */
case RXRPC_CALL_COMPLETE:
break;
default:
set_bit(RXRPC_CALL_EV_ABORT, &call->events);
rxrpc_queue_call(call);
}
+ trace_rxrpc_improper_term(call);
break;
}
- trace_rxrpc_improper_term(call);
+ spin_lock(&rx->incoming_lock);
__rxrpc_disconnect_call(conn, call);
+ spin_unlock(&rx->incoming_lock);
rxrpc_notify_socket(call);
}
* The socket is locked by the caller and this prevents the socket from being
* shut down and the local endpoint from going away, thus sk_user_data will not
* be cleared until this function returns.
+ *
+ * Called with the RCU read lock held from the IP layer via UDP.
*/
-void rxrpc_data_ready(struct sock *udp_sk)
+int rxrpc_input_packet(struct sock *udp_sk, struct sk_buff *skb)
{
struct rxrpc_connection *conn;
struct rxrpc_channel *chan;
struct rxrpc_local *local = udp_sk->sk_user_data;
struct rxrpc_peer *peer = NULL;
struct rxrpc_sock *rx = NULL;
- struct sk_buff *skb;
unsigned int channel;
- int ret, skew = 0;
+ int skew = 0;
_enter("%p", udp_sk);
- ASSERT(!irqs_disabled());
-
- skb = skb_recv_udp(udp_sk, 0, 1, &ret);
- if (!skb) {
- if (ret == -EAGAIN)
- return;
- _debug("UDP socket error %d", ret);
- return;
- }
-
if (skb->tstamp == 0)
skb->tstamp = ktime_get_real();
rxrpc_new_skb(skb, rxrpc_skb_rx_received);
- _net("recv skb %p", skb);
-
- /* we'll probably need to checksum it (didn't call sock_recvmsg) */
- if (skb_checksum_complete(skb)) {
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
- __UDP_INC_STATS(&init_net, UDP_MIB_INERRORS, 0);
- _leave(" [CSUM failed]");
- return;
- }
-
- __UDP_INC_STATS(&init_net, UDP_MIB_INDATAGRAMS, 0);
+ skb_pull(skb, sizeof(struct udphdr));
/* The UDP protocol already released all skb resources;
* we are free to add our own data there.
static int lose;
if ((lose++ & 7) == 7) {
trace_rxrpc_rx_lose(sp);
- rxrpc_lose_skb(skb, rxrpc_skb_rx_lost);
- return;
+ rxrpc_free_skb(skb, rxrpc_skb_rx_lost);
+ return 0;
}
}
+ if (skb->tstamp == 0)
+ skb->tstamp = ktime_get_real();
trace_rxrpc_rx_packet(sp);
switch (sp->hdr.type) {
if (sp->hdr.serviceId == 0)
goto bad_message;
- rcu_read_lock();
-
if (rxrpc_to_server(sp)) {
/* Weed out packets to services we're not offering. Packets
* that would begin a call are explicitly rejected and the rest
if (sp->hdr.type == RXRPC_PACKET_TYPE_DATA &&
sp->hdr.seq == 1)
goto unsupported_service;
- goto discard_unlock;
+ goto discard;
}
}
goto wrong_security;
if (sp->hdr.serviceId != conn->service_id) {
- if (!test_bit(RXRPC_CONN_PROBING_FOR_UPGRADE, &conn->flags) ||
- conn->service_id != conn->params.service_id)
+ int old_id;
+
+ if (!test_bit(RXRPC_CONN_PROBING_FOR_UPGRADE, &conn->flags))
+ goto reupgrade;
+ old_id = cmpxchg(&conn->service_id, conn->params.service_id,
+ sp->hdr.serviceId);
+
+ if (old_id != conn->params.service_id &&
+ old_id != sp->hdr.serviceId)
goto reupgrade;
- conn->service_id = sp->hdr.serviceId;
}
if (sp->hdr.callNumber == 0) {
/* Connection-level packet */
_debug("CONN %p {%d}", conn, conn->debug_id);
rxrpc_post_packet_to_conn(conn, skb);
- goto out_unlock;
+ goto out;
}
/* Note the serial number skew here */
/* Ignore really old calls */
if (sp->hdr.callNumber < chan->last_call)
- goto discard_unlock;
+ goto discard;
if (sp->hdr.callNumber == chan->last_call) {
if (chan->call ||
sp->hdr.type == RXRPC_PACKET_TYPE_ABORT)
- goto discard_unlock;
+ goto discard;
/* For the previous service call, if completed
* successfully, we discard all further packets.
*/
if (rxrpc_conn_is_service(conn) &&
chan->last_type == RXRPC_PACKET_TYPE_ACK)
- goto discard_unlock;
+ goto discard;
/* But otherwise we need to retransmit the final packet
* from data cached in the connection record.
sp->hdr.serial,
sp->hdr.flags, 0);
rxrpc_post_packet_to_conn(conn, skb);
- goto out_unlock;
+ goto out;
}
call = rcu_dereference(chan->call);
if (sp->hdr.callNumber > chan->call_id) {
- if (rxrpc_to_client(sp)) {
- rcu_read_unlock();
+ if (rxrpc_to_client(sp))
goto reject_packet;
- }
if (call)
- rxrpc_input_implicit_end_call(conn, call);
+ rxrpc_input_implicit_end_call(rx, conn, call);
call = NULL;
}
if (!call || atomic_read(&call->usage) == 0) {
if (rxrpc_to_client(sp) ||
sp->hdr.type != RXRPC_PACKET_TYPE_DATA)
- goto bad_message_unlock;
+ goto bad_message;
if (sp->hdr.seq != 1)
- goto discard_unlock;
- call = rxrpc_new_incoming_call(local, rx, peer, conn, skb);
- if (!call) {
- rcu_read_unlock();
+ goto discard;
+ call = rxrpc_new_incoming_call(local, rx, skb);
+ if (!call)
goto reject_packet;
- }
rxrpc_send_ping(call, skb, skew);
mutex_unlock(&call->user_mutex);
}
rxrpc_input_call_packet(call, skb, skew);
- goto discard_unlock;
+ goto discard;
-discard_unlock:
- rcu_read_unlock();
discard:
rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
out:
trace_rxrpc_rx_done(0, 0);
- return;
-
-out_unlock:
- rcu_read_unlock();
- goto out;
+ return 0;
wrong_security:
- rcu_read_unlock();
trace_rxrpc_abort(0, "SEC", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
RXKADINCONSISTENCY, EBADMSG);
skb->priority = RXKADINCONSISTENCY;
goto post_abort;
unsupported_service:
- rcu_read_unlock();
trace_rxrpc_abort(0, "INV", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
RX_INVALID_OPERATION, EOPNOTSUPP);
skb->priority = RX_INVALID_OPERATION;
goto post_abort;
reupgrade:
- rcu_read_unlock();
trace_rxrpc_abort(0, "UPG", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
RX_PROTOCOL_ERROR, EBADMSG);
goto protocol_error;
-bad_message_unlock:
- rcu_read_unlock();
bad_message:
trace_rxrpc_abort(0, "BAD", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
RX_PROTOCOL_ERROR, EBADMSG);
trace_rxrpc_rx_done(skb->mark, skb->priority);
rxrpc_reject_packet(local, skb);
_leave(" [badmsg]");
+ return 0;
}
#include <linux/ip.h>
#include <linux/hashtable.h>
#include <net/sock.h>
+#include <net/udp.h>
#include <net/af_rxrpc.h>
#include "ar-internal.h"
*/
static int rxrpc_open_socket(struct rxrpc_local *local, struct net *net)
{
- struct sock *sock;
+ struct sock *usk;
int ret, opt;
_enter("%p{%d,%d}",
return ret;
}
+ /* set the socket up */
+ usk = local->socket->sk;
+ inet_sk(usk)->mc_loop = 0;
+
+ /* Enable CHECKSUM_UNNECESSARY to CHECKSUM_COMPLETE conversion */
+ inet_inc_convert_csum(usk);
+
+ rcu_assign_sk_user_data(usk, local);
+
+ udp_sk(usk)->encap_type = UDP_ENCAP_RXRPC;
+ udp_sk(usk)->encap_rcv = rxrpc_input_packet;
+ udp_sk(usk)->encap_destroy = NULL;
+ udp_sk(usk)->gro_receive = NULL;
+ udp_sk(usk)->gro_complete = NULL;
+
+ udp_encap_enable();
+#if IS_ENABLED(CONFIG_AF_RXRPC_IPV6)
+ if (local->srx.transport.family == AF_INET6)
+ udpv6_encap_enable();
+#endif
+ usk->sk_error_report = rxrpc_error_report;
+
/* if a local address was supplied then bind it */
if (local->srx.transport_len > sizeof(sa_family_t)) {
_debug("bind");
BUG();
}
- /* set the socket up */
- sock = local->socket->sk;
- sock->sk_user_data = local;
- sock->sk_data_ready = rxrpc_data_ready;
- sock->sk_error_report = rxrpc_error_report;
_leave(" = 0");
return 0;
whdr.flags ^= RXRPC_CLIENT_INITIATED;
whdr.flags &= RXRPC_CLIENT_INITIATED;
- ret = kernel_sendmsg(local->socket, &msg, iov, 2, size);
+ ret = kernel_sendmsg(local->socket, &msg,
+ iov, ioc, size);
if (ret < 0)
trace_rxrpc_tx_fail(local->debug_id, 0, ret,
rxrpc_tx_point_reject);
rxrpc_store_error(peer, serr);
rcu_read_unlock();
rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_put_peer(peer);
_leave("");
}
if (rtt < 0)
return;
+ spin_lock(&peer->rtt_input_lock);
+
/* Replace the oldest datum in the RTT buffer */
sum -= peer->rtt_cache[cursor];
sum += rtt;
peer->rtt_usage = usage;
}
+ spin_unlock(&peer->rtt_input_lock);
+
/* Now recalculate the average */
if (usage == RXRPC_RTT_CACHE_SIZE) {
avg = sum / RXRPC_RTT_CACHE_SIZE;
do_div(avg, usage);
}
+ /* Don't need to update this under lock */
peer->rtt = avg;
trace_rxrpc_rtt_rx(call, why, send_serial, resp_serial, rtt,
usage, avg);
* assess the MTU size for the network interface through which this peer is
* reached
*/
-static void rxrpc_assess_MTU_size(struct rxrpc_peer *peer)
+static void rxrpc_assess_MTU_size(struct rxrpc_sock *rx,
+ struct rxrpc_peer *peer)
{
+ struct net *net = sock_net(&rx->sk);
struct dst_entry *dst;
struct rtable *rt;
struct flowi fl;
switch (peer->srx.transport.family) {
case AF_INET:
rt = ip_route_output_ports(
- &init_net, fl4, NULL,
+ net, fl4, NULL,
peer->srx.transport.sin.sin_addr.s_addr, 0,
htons(7000), htons(7001), IPPROTO_UDP, 0, 0);
if (IS_ERR(rt)) {
sizeof(struct in6_addr));
fl6->fl6_dport = htons(7001);
fl6->fl6_sport = htons(7000);
- dst = ip6_route_output(&init_net, NULL, fl6);
+ dst = ip6_route_output(net, NULL, fl6);
if (dst->error) {
_leave(" [route err %d]", dst->error);
return;
peer->service_conns = RB_ROOT;
seqlock_init(&peer->service_conn_lock);
spin_lock_init(&peer->lock);
+ spin_lock_init(&peer->rtt_input_lock);
peer->debug_id = atomic_inc_return(&rxrpc_debug_id);
if (RXRPC_TX_SMSS > 2190)
/*
* Initialise peer record.
*/
-static void rxrpc_init_peer(struct rxrpc_peer *peer, unsigned long hash_key)
+static void rxrpc_init_peer(struct rxrpc_sock *rx, struct rxrpc_peer *peer,
+ unsigned long hash_key)
{
peer->hash_key = hash_key;
- rxrpc_assess_MTU_size(peer);
+ rxrpc_assess_MTU_size(rx, peer);
peer->mtu = peer->if_mtu;
peer->rtt_last_req = ktime_get_real();
/*
* Set up a new peer.
*/
-static struct rxrpc_peer *rxrpc_create_peer(struct rxrpc_local *local,
+static struct rxrpc_peer *rxrpc_create_peer(struct rxrpc_sock *rx,
+ struct rxrpc_local *local,
struct sockaddr_rxrpc *srx,
unsigned long hash_key,
gfp_t gfp)
peer = rxrpc_alloc_peer(local, gfp);
if (peer) {
memcpy(&peer->srx, srx, sizeof(*srx));
- rxrpc_init_peer(peer, hash_key);
+ rxrpc_init_peer(rx, peer, hash_key);
}
_leave(" = %p", peer);
* since we've already done a search in the list from the non-reentrant context
* (the data_ready handler) that is the only place we can add new peers.
*/
-void rxrpc_new_incoming_peer(struct rxrpc_local *local, struct rxrpc_peer *peer)
+void rxrpc_new_incoming_peer(struct rxrpc_sock *rx, struct rxrpc_local *local,
+ struct rxrpc_peer *peer)
{
struct rxrpc_net *rxnet = local->rxnet;
unsigned long hash_key;
hash_key = rxrpc_peer_hash_key(local, &peer->srx);
peer->local = local;
- rxrpc_init_peer(peer, hash_key);
+ rxrpc_init_peer(rx, peer, hash_key);
spin_lock(&rxnet->peer_hash_lock);
hash_add_rcu(rxnet->peer_hash, &peer->hash_link, hash_key);
/*
* obtain a remote transport endpoint for the specified address
*/
-struct rxrpc_peer *rxrpc_lookup_peer(struct rxrpc_local *local,
+struct rxrpc_peer *rxrpc_lookup_peer(struct rxrpc_sock *rx,
+ struct rxrpc_local *local,
struct sockaddr_rxrpc *srx, gfp_t gfp)
{
struct rxrpc_peer *peer, *candidate;
/* The peer is not yet present in hash - create a candidate
* for a new record and then redo the search.
*/
- candidate = rxrpc_create_peer(local, srx, hash_key, gfp);
+ candidate = rxrpc_create_peer(rx, local, srx, hash_key, gfp);
if (!candidate) {
_leave(" = NULL [nomem]");
return NULL;
#include <net/pkt_sched.h>
#include <net/pkt_cls.h>
+extern const struct nla_policy rtm_tca_policy[TCA_MAX + 1];
+
/* The list of all installed classifier types */
static LIST_HEAD(tcf_proto_base);
replay:
tp_created = 0;
- err = nlmsg_parse(n, sizeof(*t), tca, TCA_MAX, NULL, extack);
+ err = nlmsg_parse(n, sizeof(*t), tca, TCA_MAX, rtm_tca_policy, extack);
if (err < 0)
return err;
if (!netlink_ns_capable(skb, net->user_ns, CAP_NET_ADMIN))
return -EPERM;
- err = nlmsg_parse(n, sizeof(*t), tca, TCA_MAX, NULL, extack);
+ err = nlmsg_parse(n, sizeof(*t), tca, TCA_MAX, rtm_tca_policy, extack);
if (err < 0)
return err;
void *fh = NULL;
int err;
- err = nlmsg_parse(n, sizeof(*t), tca, TCA_MAX, NULL, extack);
+ err = nlmsg_parse(n, sizeof(*t), tca, TCA_MAX, rtm_tca_policy, extack);
if (err < 0)
return err;
return -EPERM;
replay:
- err = nlmsg_parse(n, sizeof(*t), tca, TCA_MAX, NULL, extack);
+ err = nlmsg_parse(n, sizeof(*t), tca, TCA_MAX, rtm_tca_policy, extack);
if (err < 0)
return err;
if (nlmsg_len(cb->nlh) < sizeof(*tcm))
return skb->len;
- err = nlmsg_parse(cb->nlh, sizeof(*tcm), tca, TCA_MAX, NULL, NULL);
+ err = nlmsg_parse(cb->nlh, sizeof(*tcm), tca, TCA_MAX, rtm_tca_policy,
+ NULL);
if (err)
return err;
rcu_assign_pointer(tp_c->hlist, root_ht);
root_ht->tp_c = tp_c;
+ root_ht->refcnt++;
rcu_assign_pointer(tp->root, root_ht);
tp->data = tp_c;
return 0;
struct tc_u_hnode __rcu **hn;
struct tc_u_hnode *phn;
- WARN_ON(ht->refcnt);
+ WARN_ON(--ht->refcnt);
u32_clear_hnode(tp, ht, extack);
WARN_ON(root_ht == NULL);
- if (root_ht && --root_ht->refcnt == 0)
+ if (root_ht && --root_ht->refcnt == 1)
u32_destroy_hnode(tp, root_ht, extack);
if (--tp_c->refcnt == 0) {
}
if (ht->refcnt == 1) {
- ht->refcnt--;
u32_destroy_hnode(tp, ht, extack);
} else {
NL_SET_ERR_MSG_MOD(extack, "Can not delete in-use filter");
out:
*last = true;
if (root_ht) {
- if (root_ht->refcnt > 1) {
+ if (root_ht->refcnt > 2) {
*last = false;
goto ret;
}
- if (root_ht->refcnt == 1) {
+ if (root_ht->refcnt == 2) {
if (!ht_empty(root_ht)) {
*last = false;
goto ret;
return 0;
}
-/*
- * Delete/get qdisc.
- */
-
const struct nla_policy rtm_tca_policy[TCA_MAX + 1] = {
[TCA_KIND] = { .type = NLA_STRING },
[TCA_OPTIONS] = { .type = NLA_NESTED },
[TCA_EGRESS_BLOCK] = { .type = NLA_U32 },
};
+/*
+ * Delete/get qdisc.
+ */
+
static int tc_get_qdisc(struct sk_buff *skb, struct nlmsghdr *n,
struct netlink_ext_ack *extack)
{
if (tcm->tcm_parent) {
q = qdisc_match_from_root(root, TC_H_MAJ(tcm->tcm_parent));
- if (q && tc_dump_tclass_qdisc(q, skb, tcm, cb, t_p, s_t) < 0)
+ if (q && q != root &&
+ tc_dump_tclass_qdisc(q, skb, tcm, cb, t_p, s_t) < 0)
return -1;
return 0;
}
for (i = 1; i <= CAKE_QUEUES; i++)
quantum_div[i] = 65535 / i;
- q->tins = kvzalloc(CAKE_MAX_TINS * sizeof(struct cake_tin_data),
+ q->tins = kvcalloc(CAKE_MAX_TINS, sizeof(struct cake_tin_data),
GFP_KERNEL);
if (!q->tins)
goto nomem;
/* Get the lowest pmtu of all the transports. */
list_for_each_entry(t, &asoc->peer.transport_addr_list, transports) {
if (t->pmtu_pending && t->dst) {
- sctp_transport_update_pmtu(t, sctp_dst_mtu(t->dst));
+ sctp_transport_update_pmtu(t,
+ atomic_read(&t->mtu_info));
t->pmtu_pending = 0;
}
if (!pmtu || (t->pathmtu < pmtu))
return;
if (sock_owned_by_user(sk)) {
+ atomic_set(&t->mtu_info, pmtu);
asoc->pmtu_pending = 1;
t->pmtu_pending = 1;
return;
sctp_assoc_sync_pmtu(asoc);
}
+ if (asoc->pmtu_pending) {
+ if (asoc->param_flags & SPP_PMTUD_ENABLE)
+ sctp_assoc_sync_pmtu(asoc);
+ asoc->pmtu_pending = 0;
+ }
+
/* If there a is a prepend chunk stick it on the list before
* any other chunks get appended.
*/
spin_lock_bh(&sctp_assocs_id_lock);
asoc = (struct sctp_association *)idr_find(&sctp_assocs_id, (int)id);
+ if (asoc && (asoc->base.sk != sk || asoc->base.dead))
+ asoc = NULL;
spin_unlock_bh(&sctp_assocs_id_lock);
- if (!asoc || (asoc->base.sk != sk) || asoc->base.dead)
- return NULL;
-
return asoc;
}
if (sp->strm_interleave) {
timeo = sock_sndtimeo(sk, 0);
err = sctp_wait_for_connect(asoc, &timeo);
- if (err)
+ if (err) {
+ err = -ESRCH;
goto err;
+ }
} else {
wait_connect = true;
}
}
policy = params.sprstat_policy;
- if (policy & ~SCTP_PR_SCTP_MASK)
+ if (!policy || (policy & ~(SCTP_PR_SCTP_MASK | SCTP_PR_SCTP_ALL)))
goto out;
asoc = sctp_id2assoc(sk, params.sprstat_assoc_id);
if (!asoc)
goto out;
- if (policy == SCTP_PR_SCTP_NONE) {
+ if (policy & SCTP_PR_SCTP_ALL) {
params.sprstat_abandoned_unsent = 0;
params.sprstat_abandoned_sent = 0;
for (policy = 0; policy <= SCTP_PR_INDEX(MAX); policy++) {
}
policy = params.sprstat_policy;
- if (policy & ~SCTP_PR_SCTP_MASK)
+ if (!policy || (policy & ~(SCTP_PR_SCTP_MASK | SCTP_PR_SCTP_ALL)))
goto out;
asoc = sctp_id2assoc(sk, params.sprstat_assoc_id);
goto out;
}
- if (policy == SCTP_PR_SCTP_NONE) {
+ if (policy == SCTP_PR_SCTP_ALL) {
params.sprstat_abandoned_unsent = 0;
params.sprstat_abandoned_sent = 0;
for (policy = 0; policy <= SCTP_PR_INDEX(MAX); policy++) {
copy_in_user(&rxnfc->fs.ring_cookie,
&compat_rxnfc->fs.ring_cookie,
(void __user *)(&rxnfc->fs.location + 1) -
- (void __user *)&rxnfc->fs.ring_cookie) ||
- copy_in_user(&rxnfc->rule_cnt, &compat_rxnfc->rule_cnt,
- sizeof(rxnfc->rule_cnt)))
+ (void __user *)&rxnfc->fs.ring_cookie))
+ return -EFAULT;
+ if (ethcmd == ETHTOOL_GRXCLSRLALL) {
+ if (put_user(rule_cnt, &rxnfc->rule_cnt))
+ return -EFAULT;
+ } else if (copy_in_user(&rxnfc->rule_cnt,
+ &compat_rxnfc->rule_cnt,
+ sizeof(rxnfc->rule_cnt)))
return -EFAULT;
}
struct sk_buff *skb;
struct tipc_msg *hdr;
+ memset(&evt, 0, sizeof(evt));
evt.event = event;
evt.found_lower = m->instance;
evt.found_upper = m->instance;
l->in_session = false;
l->bearer_id = bearer_id;
l->tolerance = tolerance;
+ if (bc_rcvlink)
+ bc_rcvlink->tolerance = tolerance;
l->net_plane = net_plane;
l->advertised_mtu = mtu;
l->mtu = mtu;
void tipc_link_reset(struct tipc_link *l)
{
+ struct sk_buff_head list;
+
+ __skb_queue_head_init(&list);
+
l->in_session = false;
l->session++;
l->mtu = l->advertised_mtu;
+
spin_lock_bh(&l->wakeupq.lock);
+ skb_queue_splice_init(&l->wakeupq, &list);
+ spin_unlock_bh(&l->wakeupq.lock);
+
spin_lock_bh(&l->inputq->lock);
- skb_queue_splice_init(&l->wakeupq, l->inputq);
+ skb_queue_splice_init(&list, l->inputq);
spin_unlock_bh(&l->inputq->lock);
- spin_unlock_bh(&l->wakeupq.lock);
__skb_queue_purge(&l->transmq);
__skb_queue_purge(&l->deferdq);
/* Detect repeated retransmit failures on same packet */
if (r->last_retransm != buf_seqno(skb)) {
r->last_retransm = buf_seqno(skb);
- r->stale_limit = jiffies + msecs_to_jiffies(l->tolerance);
+ r->stale_limit = jiffies + msecs_to_jiffies(r->tolerance);
+ r->stale_cnt = 0;
} else if (++r->stale_cnt > 99 && time_after(jiffies, r->stale_limit)) {
link_retransmit_failure(l, skb);
if (link_is_bc_sndlink(l))
strncpy(if_name, data, TIPC_MAX_IF_NAME);
/* Update own tolerance if peer indicates a non-zero value */
- if (in_range(peers_tol, TIPC_MIN_LINK_TOL, TIPC_MAX_LINK_TOL))
+ if (in_range(peers_tol, TIPC_MIN_LINK_TOL, TIPC_MAX_LINK_TOL)) {
l->tolerance = peers_tol;
-
+ l->bc_rcvlink->tolerance = peers_tol;
+ }
/* Update own priority if peer's priority is higher */
if (in_range(peers_prio, l->priority + 1, TIPC_MAX_LINK_PRI))
l->priority = peers_prio;
l->rcv_nxt_state = msg_seqno(hdr) + 1;
/* Update own tolerance if peer indicates a non-zero value */
- if (in_range(peers_tol, TIPC_MIN_LINK_TOL, TIPC_MAX_LINK_TOL))
+ if (in_range(peers_tol, TIPC_MIN_LINK_TOL, TIPC_MAX_LINK_TOL)) {
l->tolerance = peers_tol;
-
+ l->bc_rcvlink->tolerance = peers_tol;
+ }
/* Update own prio if peer indicates a different value */
if ((peers_prio != l->priority) &&
in_range(peers_prio, 1, TIPC_MAX_LINK_PRI)) {
struct sk_buff_head *xmitq)
{
l->tolerance = tol;
+ if (l->bc_rcvlink)
+ l->bc_rcvlink->tolerance = tol;
if (link_is_up(l))
tipc_link_build_proto_msg(l, STATE_MSG, 0, 0, 0, tol, 0, xmitq);
}
struct sk_buff *buf;
struct distr_item *item;
- list_del(&publ->binding_node);
+ list_del_rcu(&publ->binding_node);
if (publ->scope == TIPC_NODE_SCOPE)
return NULL;
ITEM_SIZE) * ITEM_SIZE;
u32 msg_rem = msg_dsz;
- list_for_each_entry(publ, pls, binding_node) {
+ list_for_each_entry_rcu(publ, pls, binding_node) {
/* Prepare next buffer: */
if (!skb) {
skb = named_prepare_buf(net, PUBLICATION, msg_rem,
* @skb: pointer to message buffer.
*/
static void tipc_sk_conn_proto_rcv(struct tipc_sock *tsk, struct sk_buff *skb,
+ struct sk_buff_head *inputq,
struct sk_buff_head *xmitq)
{
struct tipc_msg *hdr = buf_msg(skb);
tipc_node_remove_conn(sock_net(sk), tsk_peer_node(tsk),
tsk_peer_port(tsk));
sk->sk_state_change(sk);
- goto exit;
+
+ /* State change is ignored if socket already awake,
+ * - convert msg to abort msg and add to inqueue
+ */
+ msg_set_user(hdr, TIPC_CRITICAL_IMPORTANCE);
+ msg_set_type(hdr, TIPC_CONN_MSG);
+ msg_set_size(hdr, BASIC_H_SIZE);
+ msg_set_hdr_sz(hdr, BASIC_H_SIZE);
+ __skb_queue_tail(inputq, skb);
+ return;
}
tsk->probe_unacked = false;
switch (msg_user(hdr)) {
case CONN_MANAGER:
- tipc_sk_conn_proto_rcv(tsk, skb, xmitq);
+ tipc_sk_conn_proto_rcv(tsk, skb, inputq, xmitq);
return;
case SOCK_WAKEUP:
tipc_dest_del(&tsk->cong_links, msg_orignode(hdr), 0);
sk->sk_destruct = xsk_destruct;
sk_refcnt_debug_inc(sk);
+ sock_set_flag(sk, SOCK_RCU_FREE);
+
xs = xdp_sk(sk);
mutex_init(&xs->mutex);
spin_lock_init(&xs->tx_completion_lock);
static void xfrmi_dev_free(struct net_device *dev)
{
+ struct xfrm_if *xi = netdev_priv(dev);
+
+ gro_cells_destroy(&xi->gro_cells);
free_percpu(dev->tstats);
}
break;
}
if (newpos)
- hlist_add_behind(&policy->bydst, newpos);
+ hlist_add_behind_rcu(&policy->bydst, newpos);
else
- hlist_add_head(&policy->bydst, chain);
+ hlist_add_head_rcu(&policy->bydst, chain);
}
spin_unlock_bh(&net->xfrm.xfrm_policy_lock);
break;
}
if (newpos)
- hlist_add_behind(&policy->bydst, newpos);
+ hlist_add_behind_rcu(&policy->bydst, newpos);
else
- hlist_add_head(&policy->bydst, chain);
+ hlist_add_head_rcu(&policy->bydst, chain);
__xfrm_policy_link(policy, dir);
/* After previous checking, family can either be AF_INET or AF_INET6 */
menuconfig SAMPLES
bool "Sample kernel code"
+ depends on !UML
help
You can build and test sample kernel code here.
# Do not attempt to build with gcc plugins during cc-option tests.
# (And this uses delayed resolution so the flags will be up to date.)
-CC_OPTION_CFLAGS = $(filter-out $(GCC_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
+# In addition, do not include the asm macros which are built later.
+CC_OPTION_FILTERED = $(GCC_PLUGINS_CFLAGS) $(ASM_MACRO_FLAGS)
+CC_OPTION_CFLAGS = $(filter-out $(CC_OPTION_FILTERED),$(KBUILD_CFLAGS))
# cc-option
# Usage: cflags-y += $(call cc-option,-march=winchip-c6,-march=i586)
sub_cmd_record_mcount = set -e ; perl $(srctree)/scripts/recordmcount.pl "$(ARCH)" \
"$(if $(CONFIG_CPU_BIG_ENDIAN),big,little)" \
"$(if $(CONFIG_64BIT),64,32)" \
- "$(OBJDUMP)" "$(OBJCOPY)" "$(CC) $(KBUILD_CFLAGS)" \
+ "$(OBJDUMP)" "$(OBJCOPY)" "$(CC) $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS)" \
"$(LD) $(KBUILD_LDFLAGS)" "$(NM)" "$(RM)" "$(MV)" \
"$(if $(part-of-module),1,0)" "$(@)";
recordmcount_source := $(srctree)/scripts/recordmcount.pl
hostprogs-y := modpost mk_elfconfig
always := $(hostprogs-y) empty.o
+CFLAGS_REMOVE_empty.o := $(ASM_MACRO_FLAGS)
+
modpost-objs := modpost.o file2alias.o sumversion.o
devicetable-offsets-file := devicetable-offsets.h
.cache_type = REGCACHE_RBTREE,
.reg_read = hda_reg_read,
.reg_write = hda_reg_write,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
/**
.readable_reg = cs35l33_readable_register,
.writeable_reg = cs35l33_writeable_register,
.cache_type = REGCACHE_RBTREE,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
static int __maybe_unused cs35l33_runtime_resume(struct device *dev)
.readable_reg = cs35l35_readable_register,
.precious_reg = cs35l35_precious_register,
.cache_type = REGCACHE_RBTREE,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
static irqreturn_t cs35l35_irq(int irq, void *data)
.precious_reg = cs43130_precious_register,
.volatile_reg = cs43130_volatile_register,
.cache_type = REGCACHE_RBTREE,
- .use_single_rw = true, /* needed for regcache_sync */
+ /* needed for regcache_sync */
+ .use_single_read = true,
+ .use_single_write = true,
};
static u16 const cs43130_dc_threshold[CS43130_DC_THRESHOLD] = {
.val_bits = 8,
.max_register = ES8328_REG_MAX,
.cache_type = REGCACHE_RBTREE,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
EXPORT_SYMBOL_GPL(es8328_regmap_config);
.num_reg_defaults = ARRAY_SIZE(rt1305_reg),
.ranges = rt1305_ranges,
.num_ranges = ARRAY_SIZE(rt1305_ranges),
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
#if defined(CONFIG_OF)
.cache_type = REGCACHE_RBTREE,
.reg_defaults = rt5514_reg,
.num_reg_defaults = ARRAY_SIZE(rt5514_reg),
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
static const struct i2c_device_id rt5514_i2c_id[] = {
static const struct regmap_config rt5616_regmap = {
.reg_bits = 8,
.val_bits = 16,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.max_register = RT5616_DEVICE_ID + 1 + (ARRAY_SIZE(rt5616_ranges) *
RT5616_PR_SPACING),
.volatile_reg = rt5616_volatile_register,
static const struct regmap_config rt5640_regmap = {
.reg_bits = 8,
.val_bits = 16,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.max_register = RT5640_VENDOR_ID2 + 1 + (ARRAY_SIZE(rt5640_ranges) *
RT5640_PR_SPACING),
static const struct regmap_config rt5645_regmap = {
.reg_bits = 8,
.val_bits = 16,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.max_register = RT5645_VENDOR_ID2 + 1 + (ARRAY_SIZE(rt5645_ranges) *
RT5645_PR_SPACING),
.volatile_reg = rt5645_volatile_register,
static const struct regmap_config rt5650_regmap = {
.reg_bits = 8,
.val_bits = 16,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.max_register = RT5645_VENDOR_ID2 + 1 + (ARRAY_SIZE(rt5645_ranges) *
RT5645_PR_SPACING),
.volatile_reg = rt5645_volatile_register,
.name="nocache",
.reg_bits = 8,
.val_bits = 16,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.max_register = RT5645_VENDOR_ID2 + 1,
.cache_type = REGCACHE_NONE,
};
.num_reg_defaults = ARRAY_SIZE(rt5651_reg),
.ranges = rt5651_ranges,
.num_ranges = ARRAY_SIZE(rt5651_ranges),
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
#if defined(CONFIG_OF)
static const struct regmap_config rt5660_regmap = {
.reg_bits = 8,
.val_bits = 16,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.max_register = RT5660_VENDOR_ID2 + 1 + (ARRAY_SIZE(rt5660_ranges) *
RT5660_PR_SPACING),
static const struct regmap_config rt5663_v2_regmap = {
.reg_bits = 16,
.val_bits = 16,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.max_register = 0x07fa,
.volatile_reg = rt5663_v2_volatile_register,
.readable_reg = rt5663_v2_readable_register,
static const struct regmap_config rt5663_regmap = {
.reg_bits = 16,
.val_bits = 16,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.max_register = 0x03f3,
.volatile_reg = rt5663_volatile_register,
.readable_reg = rt5663_readable_register,
.name = "nocache",
.reg_bits = 16,
.val_bits = 16,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.max_register = 0x03f3,
.cache_type = REGCACHE_NONE,
};
.cache_type = REGCACHE_RBTREE,
.reg_defaults = rt5665_reg,
.num_reg_defaults = ARRAY_SIZE(rt5665_reg),
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
static const struct i2c_device_id rt5665_i2c_id[] = {
.cache_type = REGCACHE_RBTREE,
.reg_defaults = rt5668_reg,
.num_reg_defaults = ARRAY_SIZE(rt5668_reg),
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
static const struct i2c_device_id rt5668_i2c_id[] = {
static const struct regmap_config rt5670_regmap = {
.reg_bits = 8,
.val_bits = 16,
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
.max_register = RT5670_VENDOR_ID2 + 1 + (ARRAY_SIZE(rt5670_ranges) *
RT5670_PR_SPACING),
.volatile_reg = rt5670_volatile_register,
.cache_type = REGCACHE_RBTREE,
.reg_defaults = rt5682_reg,
.num_reg_defaults = ARRAY_SIZE(rt5682_reg),
- .use_single_rw = true,
+ .use_single_read = true,
+ .use_single_write = true,
};
static const struct i2c_device_id rt5682_i2c_id[] = {
};
static const struct x86_cpu_id baytrail_cpu_ids[] = {
- { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT1 }, /* Valleyview */
+ { X86_VENDOR_INTEL, 6, INTEL_FAM6_ATOM_SILVERMONT }, /* Valleyview */
{}
};
#define KVM_X86_QUIRK_LINT0_REENABLED (1 << 0)
#define KVM_X86_QUIRK_CD_NW_CLEARED (1 << 1)
+#define KVM_X86_QUIRK_LAPIC_MMIO_HOLE (1 << 2)
#define KVM_STATE_NESTED_GUEST_MODE 0x00000001
#define KVM_STATE_NESTED_RUN_PENDING 0x00000002
#define _TOOLS_LINUX_BITOPS_H_
#include <asm/types.h>
-#include <linux/compiler.h>
-
#ifndef __WORDSIZE
#define __WORDSIZE (__SIZEOF_LONG__ * 8)
#endif
#ifndef BITS_PER_LONG
# define BITS_PER_LONG __WORDSIZE
#endif
+#include <linux/bits.h>
+#include <linux/compiler.h>
-#define BIT_MASK(nr) (1UL << ((nr) % BITS_PER_LONG))
-#define BIT_WORD(nr) ((nr) / BITS_PER_LONG)
-#define BITS_PER_BYTE 8
#define BITS_TO_LONGS(nr) DIV_ROUND_UP(nr, BITS_PER_BYTE * sizeof(long))
#define BITS_TO_U64(nr) DIV_ROUND_UP(nr, BITS_PER_BYTE * sizeof(u64))
#define BITS_TO_U32(nr) DIV_ROUND_UP(nr, BITS_PER_BYTE * sizeof(u32))
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LINUX_BITS_H
+#define __LINUX_BITS_H
+#include <asm/bitsperlong.h>
+
+#define BIT(nr) (1UL << (nr))
+#define BIT_ULL(nr) (1ULL << (nr))
+#define BIT_MASK(nr) (1UL << ((nr) % BITS_PER_LONG))
+#define BIT_WORD(nr) ((nr) / BITS_PER_LONG)
+#define BIT_ULL_MASK(nr) (1ULL << ((nr) % BITS_PER_LONG_LONG))
+#define BIT_ULL_WORD(nr) ((nr) / BITS_PER_LONG_LONG)
+#define BITS_PER_BYTE 8
+
+/*
+ * Create a contiguous bitmask starting at bit position @l and ending at
+ * position @h. For example
+ * GENMASK_ULL(39, 21) gives us the 64bit vector 0x000000ffffe00000.
+ */
+#define GENMASK(h, l) \
+ (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
+
+#define GENMASK_ULL(h, l) \
+ (((~0ULL) - (1ULL << (l)) + 1) & \
+ (~0ULL >> (BITS_PER_LONG_LONG - 1 - (h))))
+
+#endif /* __LINUX_BITS_H */
return unlikely(!ptr) || IS_ERR_VALUE((unsigned long)ptr);
}
+static inline int __must_check PTR_ERR_OR_ZERO(__force const void *ptr)
+{
+ if (IS_ERR(ptr))
+ return PTR_ERR(ptr);
+ else
+ return 0;
+}
#endif /* _LINUX_ERR_H */
#define KVM_CAP_S390_HPAGE_1M 156
#define KVM_CAP_NESTED_STATE 157
#define KVM_CAP_ARM_INJECT_SERROR_ESR 158
+#define KVM_CAP_MSR_PLATFORM_INFO 159
#ifdef KVM_CAP_IRQ_ROUTING
__tracing_path_set("", mnt);
- return mnt;
+ return tracing_path;
}
static const char *tracing_path_debugfs_mount(void)
__tracing_path_set("tracing/", mnt);
- return mnt;
+ return tracing_path;
}
const char *tracing_path_mount(void)
subcmd_config.pager_env = pager_env;
}
+static const char *forced_pager;
+
+void force_pager(const char *pager)
+{
+ forced_pager = pager;
+}
+
static void pager_preexec(void)
{
/*
const char *pager = getenv(subcmd_config.pager_env);
struct winsize sz;
- if (!isatty(1))
+ if (forced_pager)
+ pager = forced_pager;
+ if (!isatty(1) && !forced_pager)
return;
if (ioctl(1, TIOCGWINSZ, &sz) == 0)
pager_columns = sz.ws_col;
extern void setup_pager(void);
extern int pager_in_use(void);
extern int pager_get_columns(void);
+extern void force_pager(const char *);
#endif /* __SUBCMD_PAGER_H */
libtraceevent-y += parse-filter.o
libtraceevent-y += parse-utils.o
libtraceevent-y += kbuffer-parse.o
+libtraceevent-y += tep_strerror.o
+libtraceevent-y += event-parse-api.o
plugin_jbd2-y += plugin_jbd2.o
plugin_hrtimer-y += plugin_hrtimer.o
--- /dev/null
+// SPDX-License-Identifier: LGPL-2.1
+/*
+ * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
+ *
+ */
+
+#include "event-parse.h"
+#include "event-parse-local.h"
+#include "event-utils.h"
+
+/**
+ * tep_get_first_event - returns the first event in the events array
+ * @tep: a handle to the tep_handle
+ *
+ * This returns pointer to the first element of the events array
+ * If @tep is NULL, NULL is returned.
+ */
+struct tep_event_format *tep_get_first_event(struct tep_handle *tep)
+{
+ if (tep && tep->events)
+ return tep->events[0];
+
+ return NULL;
+}
+
+/**
+ * tep_get_events_count - get the number of defined events
+ * @tep: a handle to the tep_handle
+ *
+ * This returns number of elements in event array
+ * If @tep is NULL, 0 is returned.
+ */
+int tep_get_events_count(struct tep_handle *tep)
+{
+ if(tep)
+ return tep->nr_events;
+ return 0;
+}
+
+/**
+ * tep_set_flag - set event parser flag
+ * @tep: a handle to the tep_handle
+ * @flag: flag, or combination of flags to be set
+ * can be any combination from enum tep_flag
+ *
+ * This sets a flag or mbination of flags from enum tep_flag
+ */
+void tep_set_flag(struct tep_handle *tep, int flag)
+{
+ if(tep)
+ tep->flags |= flag;
+}
+
+unsigned short __tep_data2host2(struct tep_handle *pevent, unsigned short data)
+{
+ unsigned short swap;
+
+ if (!pevent || pevent->host_bigendian == pevent->file_bigendian)
+ return data;
+
+ swap = ((data & 0xffULL) << 8) |
+ ((data & (0xffULL << 8)) >> 8);
+
+ return swap;
+}
+
+unsigned int __tep_data2host4(struct tep_handle *pevent, unsigned int data)
+{
+ unsigned int swap;
+
+ if (!pevent || pevent->host_bigendian == pevent->file_bigendian)
+ return data;
+
+ swap = ((data & 0xffULL) << 24) |
+ ((data & (0xffULL << 8)) << 8) |
+ ((data & (0xffULL << 16)) >> 8) |
+ ((data & (0xffULL << 24)) >> 24);
+
+ return swap;
+}
+
+unsigned long long
+__tep_data2host8(struct tep_handle *pevent, unsigned long long data)
+{
+ unsigned long long swap;
+
+ if (!pevent || pevent->host_bigendian == pevent->file_bigendian)
+ return data;
+
+ swap = ((data & 0xffULL) << 56) |
+ ((data & (0xffULL << 8)) << 40) |
+ ((data & (0xffULL << 16)) << 24) |
+ ((data & (0xffULL << 24)) << 8) |
+ ((data & (0xffULL << 32)) >> 8) |
+ ((data & (0xffULL << 40)) >> 24) |
+ ((data & (0xffULL << 48)) >> 40) |
+ ((data & (0xffULL << 56)) >> 56);
+
+ return swap;
+}
+
+/**
+ * tep_get_header_page_size - get size of the header page
+ * @pevent: a handle to the tep_handle
+ *
+ * This returns size of the header page
+ * If @pevent is NULL, 0 is returned.
+ */
+int tep_get_header_page_size(struct tep_handle *pevent)
+{
+ if(pevent)
+ return pevent->header_page_size_size;
+ return 0;
+}
+
+/**
+ * tep_get_cpus - get the number of CPUs
+ * @pevent: a handle to the tep_handle
+ *
+ * This returns the number of CPUs
+ * If @pevent is NULL, 0 is returned.
+ */
+int tep_get_cpus(struct tep_handle *pevent)
+{
+ if(pevent)
+ return pevent->cpus;
+ return 0;
+}
+
+/**
+ * tep_set_cpus - set the number of CPUs
+ * @pevent: a handle to the tep_handle
+ *
+ * This sets the number of CPUs
+ */
+void tep_set_cpus(struct tep_handle *pevent, int cpus)
+{
+ if(pevent)
+ pevent->cpus = cpus;
+}
+
+/**
+ * tep_get_long_size - get the size of a long integer on the current machine
+ * @pevent: a handle to the tep_handle
+ *
+ * This returns the size of a long integer on the current machine
+ * If @pevent is NULL, 0 is returned.
+ */
+int tep_get_long_size(struct tep_handle *pevent)
+{
+ if(pevent)
+ return pevent->long_size;
+ return 0;
+}
+
+/**
+ * tep_set_long_size - set the size of a long integer on the current machine
+ * @pevent: a handle to the tep_handle
+ * @size: size, in bytes, of a long integer
+ *
+ * This sets the size of a long integer on the current machine
+ */
+void tep_set_long_size(struct tep_handle *pevent, int long_size)
+{
+ if(pevent)
+ pevent->long_size = long_size;
+}
+
+/**
+ * tep_get_page_size - get the size of a memory page on the current machine
+ * @pevent: a handle to the tep_handle
+ *
+ * This returns the size of a memory page on the current machine
+ * If @pevent is NULL, 0 is returned.
+ */
+int tep_get_page_size(struct tep_handle *pevent)
+{
+ if(pevent)
+ return pevent->page_size;
+ return 0;
+}
+
+/**
+ * tep_set_page_size - set the size of a memory page on the current machine
+ * @pevent: a handle to the tep_handle
+ * @_page_size: size of a memory page, in bytes
+ *
+ * This sets the size of a memory page on the current machine
+ */
+void tep_set_page_size(struct tep_handle *pevent, int _page_size)
+{
+ if(pevent)
+ pevent->page_size = _page_size;
+}
+
+/**
+ * tep_is_file_bigendian - get if the file is in big endian order
+ * @pevent: a handle to the tep_handle
+ *
+ * This returns if the file is in big endian order
+ * If @pevent is NULL, 0 is returned.
+ */
+int tep_is_file_bigendian(struct tep_handle *pevent)
+{
+ if(pevent)
+ return pevent->file_bigendian;
+ return 0;
+}
+
+/**
+ * tep_set_file_bigendian - set if the file is in big endian order
+ * @pevent: a handle to the tep_handle
+ * @endian: non zero, if the file is in big endian order
+ *
+ * This sets if the file is in big endian order
+ */
+void tep_set_file_bigendian(struct tep_handle *pevent, enum tep_endian endian)
+{
+ if(pevent)
+ pevent->file_bigendian = endian;
+}
+
+/**
+ * tep_is_host_bigendian - get if the order of the current host is big endian
+ * @pevent: a handle to the tep_handle
+ *
+ * This gets if the order of the current host is big endian
+ * If @pevent is NULL, 0 is returned.
+ */
+int tep_is_host_bigendian(struct tep_handle *pevent)
+{
+ if(pevent)
+ return pevent->host_bigendian;
+ return 0;
+}
+
+/**
+ * tep_set_host_bigendian - set the order of the local host
+ * @pevent: a handle to the tep_handle
+ * @endian: non zero, if the local host has big endian order
+ *
+ * This sets the order of the local host
+ */
+void tep_set_host_bigendian(struct tep_handle *pevent, enum tep_endian endian)
+{
+ if(pevent)
+ pevent->host_bigendian = endian;
+}
+
+/**
+ * tep_is_latency_format - get if the latency output format is configured
+ * @pevent: a handle to the tep_handle
+ *
+ * This gets if the latency output format is configured
+ * If @pevent is NULL, 0 is returned.
+ */
+int tep_is_latency_format(struct tep_handle *pevent)
+{
+ if(pevent)
+ return pevent->latency_format;
+ return 0;
+}
+
+/**
+ * tep_set_latency_format - set the latency output format
+ * @pevent: a handle to the tep_handle
+ * @lat: non zero for latency output format
+ *
+ * This sets the latency output format
+ */
+void tep_set_latency_format(struct tep_handle *pevent, int lat)
+{
+ if(pevent)
+ pevent->latency_format = lat;
+}
--- /dev/null
+// SPDX-License-Identifier: LGPL-2.1
+/*
+ * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
+ *
+ */
+
+#ifndef _PARSE_EVENTS_INT_H
+#define _PARSE_EVENTS_INT_H
+
+struct cmdline;
+struct cmdline_list;
+struct func_map;
+struct func_list;
+struct event_handler;
+struct func_resolver;
+
+struct tep_handle {
+ int ref_count;
+
+ int header_page_ts_offset;
+ int header_page_ts_size;
+ int header_page_size_offset;
+ int header_page_size_size;
+ int header_page_data_offset;
+ int header_page_data_size;
+ int header_page_overwrite;
+
+ enum tep_endian file_bigendian;
+ enum tep_endian host_bigendian;
+
+ int latency_format;
+
+ int old_format;
+
+ int cpus;
+ int long_size;
+ int page_size;
+
+ struct cmdline *cmdlines;
+ struct cmdline_list *cmdlist;
+ int cmdline_count;
+
+ struct func_map *func_map;
+ struct func_resolver *func_resolver;
+ struct func_list *funclist;
+ unsigned int func_count;
+
+ struct printk_map *printk_map;
+ struct printk_list *printklist;
+ unsigned int printk_count;
+
+
+ struct tep_event_format **events;
+ int nr_events;
+ struct tep_event_format **sort_events;
+ enum tep_event_sort_type last_type;
+
+ int type_offset;
+ int type_size;
+
+ int pid_offset;
+ int pid_size;
+
+ int pc_offset;
+ int pc_size;
+
+ int flags_offset;
+ int flags_size;
+
+ int ld_offset;
+ int ld_size;
+
+ int print_raw;
+
+ int test_filters;
+
+ int flags;
+
+ struct tep_format_field *bprint_ip_field;
+ struct tep_format_field *bprint_fmt_field;
+ struct tep_format_field *bprint_buf_field;
+
+ struct event_handler *handlers;
+ struct tep_function_handler *func_handlers;
+
+ /* cache */
+ struct tep_event_format *last_event;
+
+ char *trace_clock;
+};
+
+#endif /* _PARSE_EVENTS_INT_H */
#include <errno.h>
#include <stdint.h>
#include <limits.h>
-#include <linux/string.h>
#include <linux/time64.h>
#include <netinet/in.h>
#include "event-parse.h"
+
+#include "event-parse-local.h"
#include "event-utils.h"
+#include "trace-seq.h"
static const char *input_buf;
static unsigned long long input_buf_ptr;
static unsigned long long
process_defined_func(struct trace_seq *s, void *data, int size,
- struct event_format *event, struct print_arg *arg);
+ struct tep_event_format *event, struct tep_print_arg *arg);
static void free_func_handle(struct tep_function_handler *func);
x++;
}
-struct print_arg *alloc_arg(void)
+struct tep_print_arg *alloc_arg(void)
{
- return calloc(1, sizeof(struct print_arg));
+ return calloc(1, sizeof(struct tep_print_arg));
}
struct cmdline {
}
}
-static struct event_format *alloc_event(void)
+static struct tep_event_format *alloc_event(void)
{
- return calloc(1, sizeof(struct event_format));
+ return calloc(1, sizeof(struct tep_event_format));
}
-static int add_event(struct tep_handle *pevent, struct event_format *event)
+static int add_event(struct tep_handle *pevent, struct tep_event_format *event)
{
int i;
- struct event_format **events = realloc(pevent->events, sizeof(event) *
- (pevent->nr_events + 1));
+ struct tep_event_format **events = realloc(pevent->events, sizeof(event) *
+ (pevent->nr_events + 1));
if (!events)
return -1;
return 0;
}
-static int event_item_type(enum event_type type)
+static int event_item_type(enum tep_event_type type)
{
switch (type) {
- case EVENT_ITEM ... EVENT_SQUOTE:
+ case TEP_EVENT_ITEM ... TEP_EVENT_SQUOTE:
return 1;
- case EVENT_ERROR ... EVENT_DELIM:
+ case TEP_EVENT_ERROR ... TEP_EVENT_DELIM:
default:
return 0;
}
}
-static void free_flag_sym(struct print_flag_sym *fsym)
+static void free_flag_sym(struct tep_print_flag_sym *fsym)
{
- struct print_flag_sym *next;
+ struct tep_print_flag_sym *next;
while (fsym) {
next = fsym->next;
}
}
-static void free_arg(struct print_arg *arg)
+static void free_arg(struct tep_print_arg *arg)
{
- struct print_arg *farg;
+ struct tep_print_arg *farg;
if (!arg)
return;
switch (arg->type) {
- case PRINT_ATOM:
+ case TEP_PRINT_ATOM:
free(arg->atom.atom);
break;
- case PRINT_FIELD:
+ case TEP_PRINT_FIELD:
free(arg->field.name);
break;
- case PRINT_FLAGS:
+ case TEP_PRINT_FLAGS:
free_arg(arg->flags.field);
free(arg->flags.delim);
free_flag_sym(arg->flags.flags);
break;
- case PRINT_SYMBOL:
+ case TEP_PRINT_SYMBOL:
free_arg(arg->symbol.field);
free_flag_sym(arg->symbol.symbols);
break;
- case PRINT_HEX:
- case PRINT_HEX_STR:
+ case TEP_PRINT_HEX:
+ case TEP_PRINT_HEX_STR:
free_arg(arg->hex.field);
free_arg(arg->hex.size);
break;
- case PRINT_INT_ARRAY:
+ case TEP_PRINT_INT_ARRAY:
free_arg(arg->int_array.field);
free_arg(arg->int_array.count);
free_arg(arg->int_array.el_size);
break;
- case PRINT_TYPE:
+ case TEP_PRINT_TYPE:
free(arg->typecast.type);
free_arg(arg->typecast.item);
break;
- case PRINT_STRING:
- case PRINT_BSTRING:
+ case TEP_PRINT_STRING:
+ case TEP_PRINT_BSTRING:
free(arg->string.string);
break;
- case PRINT_BITMASK:
+ case TEP_PRINT_BITMASK:
free(arg->bitmask.bitmask);
break;
- case PRINT_DYNAMIC_ARRAY:
- case PRINT_DYNAMIC_ARRAY_LEN:
+ case TEP_PRINT_DYNAMIC_ARRAY:
+ case TEP_PRINT_DYNAMIC_ARRAY_LEN:
free(arg->dynarray.index);
break;
- case PRINT_OP:
+ case TEP_PRINT_OP:
free(arg->op.op);
free_arg(arg->op.left);
free_arg(arg->op.right);
break;
- case PRINT_FUNC:
+ case TEP_PRINT_FUNC:
while (arg->func.args) {
farg = arg->func.args;
arg->func.args = farg->next;
}
break;
- case PRINT_NULL:
+ case TEP_PRINT_NULL:
default:
break;
}
free(arg);
}
-static enum event_type get_type(int ch)
+static enum tep_event_type get_type(int ch)
{
if (ch == '\n')
- return EVENT_NEWLINE;
+ return TEP_EVENT_NEWLINE;
if (isspace(ch))
- return EVENT_SPACE;
+ return TEP_EVENT_SPACE;
if (isalnum(ch) || ch == '_')
- return EVENT_ITEM;
+ return TEP_EVENT_ITEM;
if (ch == '\'')
- return EVENT_SQUOTE;
+ return TEP_EVENT_SQUOTE;
if (ch == '"')
- return EVENT_DQUOTE;
+ return TEP_EVENT_DQUOTE;
if (!isprint(ch))
- return EVENT_NONE;
+ return TEP_EVENT_NONE;
if (ch == '(' || ch == ')' || ch == ',')
- return EVENT_DELIM;
+ return TEP_EVENT_DELIM;
- return EVENT_OP;
+ return TEP_EVENT_OP;
}
static int __read_char(void)
return 0;
}
-static enum event_type force_token(const char *str, char **tok);
+static enum tep_event_type force_token(const char *str, char **tok);
-static enum event_type __read_token(char **tok)
+static enum tep_event_type __read_token(char **tok)
{
char buf[BUFSIZ];
int ch, last_ch, quote_ch, next_ch;
int i = 0;
int tok_size = 0;
- enum event_type type;
+ enum tep_event_type type;
*tok = NULL;
ch = __read_char();
if (ch < 0)
- return EVENT_NONE;
+ return TEP_EVENT_NONE;
type = get_type(ch);
- if (type == EVENT_NONE)
+ if (type == TEP_EVENT_NONE)
return type;
buf[i++] = ch;
switch (type) {
- case EVENT_NEWLINE:
- case EVENT_DELIM:
+ case TEP_EVENT_NEWLINE:
+ case TEP_EVENT_DELIM:
if (asprintf(tok, "%c", ch) < 0)
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
return type;
- case EVENT_OP:
+ case TEP_EVENT_OP:
switch (ch) {
case '-':
next_ch = __peek_char();
buf[i++] = __read_char();
goto out;
- case EVENT_DQUOTE:
- case EVENT_SQUOTE:
+ case TEP_EVENT_DQUOTE:
+ case TEP_EVENT_SQUOTE:
/* don't keep quotes */
i--;
quote_ch = ch;
tok_size += BUFSIZ;
if (extend_token(tok, buf, tok_size) < 0)
- return EVENT_NONE;
+ return TEP_EVENT_NONE;
i = 0;
}
last_ch = ch;
* For strings (double quotes) check the next token.
* If it is another string, concatinate the two.
*/
- if (type == EVENT_DQUOTE) {
+ if (type == TEP_EVENT_DQUOTE) {
unsigned long long save_input_buf_ptr = input_buf_ptr;
do {
goto out;
- case EVENT_ERROR ... EVENT_SPACE:
- case EVENT_ITEM:
+ case TEP_EVENT_ERROR ... TEP_EVENT_SPACE:
+ case TEP_EVENT_ITEM:
default:
break;
}
tok_size += BUFSIZ;
if (extend_token(tok, buf, tok_size) < 0)
- return EVENT_NONE;
+ return TEP_EVENT_NONE;
i = 0;
}
ch = __read_char();
out:
buf[i] = 0;
if (extend_token(tok, buf, tok_size + i + 1) < 0)
- return EVENT_NONE;
+ return TEP_EVENT_NONE;
- if (type == EVENT_ITEM) {
+ if (type == TEP_EVENT_ITEM) {
/*
* Older versions of the kernel has a bug that
* creates invalid symbols and will break the mac80211
return type;
}
-static enum event_type force_token(const char *str, char **tok)
+static enum tep_event_type force_token(const char *str, char **tok)
{
const char *save_input_buf;
unsigned long long save_input_buf_ptr;
unsigned long long save_input_buf_siz;
- enum event_type type;
+ enum tep_event_type type;
/* save off the current input pointers */
save_input_buf = input_buf;
free(tok);
}
-static enum event_type read_token(char **tok)
+static enum tep_event_type read_token(char **tok)
{
- enum event_type type;
+ enum tep_event_type type;
for (;;) {
type = __read_token(tok);
- if (type != EVENT_SPACE)
+ if (type != TEP_EVENT_SPACE)
return type;
free_token(*tok);
/* not reached */
*tok = NULL;
- return EVENT_NONE;
+ return TEP_EVENT_NONE;
}
/**
*
* Returns the token type.
*/
-enum event_type tep_read_token(char **tok)
+enum tep_event_type tep_read_token(char **tok)
{
return read_token(tok);
}
}
/* no newline */
-static enum event_type read_token_item(char **tok)
+static enum tep_event_type read_token_item(char **tok)
{
- enum event_type type;
+ enum tep_event_type type;
for (;;) {
type = __read_token(tok);
- if (type != EVENT_SPACE && type != EVENT_NEWLINE)
+ if (type != TEP_EVENT_SPACE && type != TEP_EVENT_NEWLINE)
return type;
free_token(*tok);
*tok = NULL;
/* not reached */
*tok = NULL;
- return EVENT_NONE;
+ return TEP_EVENT_NONE;
}
-static int test_type(enum event_type type, enum event_type expect)
+static int test_type(enum tep_event_type type, enum tep_event_type expect)
{
if (type != expect) {
do_warning("Error: expected type %d but read %d",
return 0;
}
-static int test_type_token(enum event_type type, const char *token,
- enum event_type expect, const char *expect_tok)
+static int test_type_token(enum tep_event_type type, const char *token,
+ enum tep_event_type expect, const char *expect_tok)
{
if (type != expect) {
do_warning("Error: expected type %d but read %d",
return 0;
}
-static int __read_expect_type(enum event_type expect, char **tok, int newline_ok)
+static int __read_expect_type(enum tep_event_type expect, char **tok, int newline_ok)
{
- enum event_type type;
+ enum tep_event_type type;
if (newline_ok)
type = read_token(tok);
return test_type(type, expect);
}
-static int read_expect_type(enum event_type expect, char **tok)
+static int read_expect_type(enum tep_event_type expect, char **tok)
{
return __read_expect_type(expect, tok, 1);
}
-static int __read_expected(enum event_type expect, const char *str,
+static int __read_expected(enum tep_event_type expect, const char *str,
int newline_ok)
{
- enum event_type type;
+ enum tep_event_type type;
char *token;
int ret;
return ret;
}
-static int read_expected(enum event_type expect, const char *str)
+static int read_expected(enum tep_event_type expect, const char *str)
{
return __read_expected(expect, str, 1);
}
-static int read_expected_item(enum event_type expect, const char *str)
+static int read_expected_item(enum tep_event_type expect, const char *str)
{
return __read_expected(expect, str, 0);
}
{
char *token;
- if (read_expected(EVENT_ITEM, "name") < 0)
+ if (read_expected(TEP_EVENT_ITEM, "name") < 0)
return NULL;
- if (read_expected(EVENT_OP, ":") < 0)
+ if (read_expected(TEP_EVENT_OP, ":") < 0)
return NULL;
- if (read_expect_type(EVENT_ITEM, &token) < 0)
+ if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
goto fail;
return token;
char *token;
int id;
- if (read_expected_item(EVENT_ITEM, "ID") < 0)
+ if (read_expected_item(TEP_EVENT_ITEM, "ID") < 0)
return -1;
- if (read_expected(EVENT_OP, ":") < 0)
+ if (read_expected(TEP_EVENT_OP, ":") < 0)
return -1;
- if (read_expect_type(EVENT_ITEM, &token) < 0)
+ if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
goto fail;
id = strtoul(token, NULL, 0);
return -1;
}
-static int field_is_string(struct format_field *field)
+static int field_is_string(struct tep_format_field *field)
{
- if ((field->flags & FIELD_IS_ARRAY) &&
+ if ((field->flags & TEP_FIELD_IS_ARRAY) &&
(strstr(field->type, "char") || strstr(field->type, "u8") ||
strstr(field->type, "s8")))
return 1;
return 0;
}
-static int field_is_dynamic(struct format_field *field)
+static int field_is_dynamic(struct tep_format_field *field)
{
if (strncmp(field->type, "__data_loc", 10) == 0)
return 1;
return 0;
}
-static int field_is_long(struct format_field *field)
+static int field_is_long(struct tep_format_field *field)
{
/* includes long long */
if (strstr(field->type, "long"))
static unsigned int type_size(const char *name)
{
- /* This covers all FIELD_IS_STRING types. */
+ /* This covers all TEP_FIELD_IS_STRING types. */
static struct {
const char *type;
unsigned int size;
return 0;
}
-static int event_read_fields(struct event_format *event, struct format_field **fields)
+static int event_read_fields(struct tep_event_format *event, struct tep_format_field **fields)
{
- struct format_field *field = NULL;
- enum event_type type;
+ struct tep_format_field *field = NULL;
+ enum tep_event_type type;
char *token;
char *last_token;
int count = 0;
unsigned int size_dynamic = 0;
type = read_token(&token);
- if (type == EVENT_NEWLINE) {
+ if (type == TEP_EVENT_NEWLINE) {
free_token(token);
return count;
}
count++;
- if (test_type_token(type, token, EVENT_ITEM, "field"))
+ if (test_type_token(type, token, TEP_EVENT_ITEM, "field"))
goto fail;
free_token(token);
* The ftrace fields may still use the "special" name.
* Just ignore it.
*/
- if (event->flags & EVENT_FL_ISFTRACE &&
- type == EVENT_ITEM && strcmp(token, "special") == 0) {
+ if (event->flags & TEP_EVENT_FL_ISFTRACE &&
+ type == TEP_EVENT_ITEM && strcmp(token, "special") == 0) {
free_token(token);
type = read_token(&token);
}
- if (test_type_token(type, token, EVENT_OP, ":") < 0)
+ if (test_type_token(type, token, TEP_EVENT_OP, ":") < 0)
goto fail;
free_token(token);
- if (read_expect_type(EVENT_ITEM, &token) < 0)
+ if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
goto fail;
last_token = token;
/* read the rest of the type */
for (;;) {
type = read_token(&token);
- if (type == EVENT_ITEM ||
- (type == EVENT_OP && strcmp(token, "*") == 0) ||
+ if (type == TEP_EVENT_ITEM ||
+ (type == TEP_EVENT_OP && strcmp(token, "*") == 0) ||
/*
* Some of the ftrace fields are broken and have
* an illegal "." in them.
*/
- (event->flags & EVENT_FL_ISFTRACE &&
- type == EVENT_OP && strcmp(token, ".") == 0)) {
+ (event->flags & TEP_EVENT_FL_ISFTRACE &&
+ type == TEP_EVENT_OP && strcmp(token, ".") == 0)) {
if (strcmp(token, "*") == 0)
- field->flags |= FIELD_IS_POINTER;
+ field->flags |= TEP_FIELD_IS_POINTER;
if (field->type) {
char *new_type;
}
field->name = field->alias = last_token;
- if (test_type(type, EVENT_OP))
+ if (test_type(type, TEP_EVENT_OP))
goto fail;
if (strcmp(token, "[") == 0) {
- enum event_type last_type = type;
+ enum tep_event_type last_type = type;
char *brackets = token;
char *new_brackets;
int len;
- field->flags |= FIELD_IS_ARRAY;
+ field->flags |= TEP_FIELD_IS_ARRAY;
type = read_token(&token);
- if (type == EVENT_ITEM)
+ if (type == TEP_EVENT_ITEM)
field->arraylen = strtoul(token, NULL, 0);
else
field->arraylen = 0;
while (strcmp(token, "]") != 0) {
- if (last_type == EVENT_ITEM &&
- type == EVENT_ITEM)
+ if (last_type == TEP_EVENT_ITEM &&
+ type == TEP_EVENT_ITEM)
len = 2;
else
len = 1;
field->arraylen = strtoul(token, NULL, 0);
free_token(token);
type = read_token(&token);
- if (type == EVENT_NONE) {
+ if (type == TEP_EVENT_NONE) {
do_warning_event(event, "failed to find token");
goto fail;
}
* If the next token is not an OP, then it is of
* the format: type [] item;
*/
- if (type == EVENT_ITEM) {
+ if (type == TEP_EVENT_ITEM) {
char *new_type;
new_type = realloc(field->type,
strlen(field->type) +
}
if (field_is_string(field))
- field->flags |= FIELD_IS_STRING;
+ field->flags |= TEP_FIELD_IS_STRING;
if (field_is_dynamic(field))
- field->flags |= FIELD_IS_DYNAMIC;
+ field->flags |= TEP_FIELD_IS_DYNAMIC;
if (field_is_long(field))
- field->flags |= FIELD_IS_LONG;
+ field->flags |= TEP_FIELD_IS_LONG;
- if (test_type_token(type, token, EVENT_OP, ";"))
+ if (test_type_token(type, token, TEP_EVENT_OP, ";"))
goto fail;
free_token(token);
- if (read_expected(EVENT_ITEM, "offset") < 0)
+ if (read_expected(TEP_EVENT_ITEM, "offset") < 0)
goto fail_expect;
- if (read_expected(EVENT_OP, ":") < 0)
+ if (read_expected(TEP_EVENT_OP, ":") < 0)
goto fail_expect;
- if (read_expect_type(EVENT_ITEM, &token))
+ if (read_expect_type(TEP_EVENT_ITEM, &token))
goto fail;
field->offset = strtoul(token, NULL, 0);
free_token(token);
- if (read_expected(EVENT_OP, ";") < 0)
+ if (read_expected(TEP_EVENT_OP, ";") < 0)
goto fail_expect;
- if (read_expected(EVENT_ITEM, "size") < 0)
+ if (read_expected(TEP_EVENT_ITEM, "size") < 0)
goto fail_expect;
- if (read_expected(EVENT_OP, ":") < 0)
+ if (read_expected(TEP_EVENT_OP, ":") < 0)
goto fail_expect;
- if (read_expect_type(EVENT_ITEM, &token))
+ if (read_expect_type(TEP_EVENT_ITEM, &token))
goto fail;
field->size = strtoul(token, NULL, 0);
free_token(token);
- if (read_expected(EVENT_OP, ";") < 0)
+ if (read_expected(TEP_EVENT_OP, ";") < 0)
goto fail_expect;
type = read_token(&token);
- if (type != EVENT_NEWLINE) {
+ if (type != TEP_EVENT_NEWLINE) {
/* newer versions of the kernel have a "signed" type */
- if (test_type_token(type, token, EVENT_ITEM, "signed"))
+ if (test_type_token(type, token, TEP_EVENT_ITEM, "signed"))
goto fail;
free_token(token);
- if (read_expected(EVENT_OP, ":") < 0)
+ if (read_expected(TEP_EVENT_OP, ":") < 0)
goto fail_expect;
- if (read_expect_type(EVENT_ITEM, &token))
+ if (read_expect_type(TEP_EVENT_ITEM, &token))
goto fail;
if (strtoul(token, NULL, 0))
- field->flags |= FIELD_IS_SIGNED;
+ field->flags |= TEP_FIELD_IS_SIGNED;
free_token(token);
- if (read_expected(EVENT_OP, ";") < 0)
+ if (read_expected(TEP_EVENT_OP, ";") < 0)
goto fail_expect;
- if (read_expect_type(EVENT_NEWLINE, &token))
+ if (read_expect_type(TEP_EVENT_NEWLINE, &token))
goto fail;
}
free_token(token);
- if (field->flags & FIELD_IS_ARRAY) {
+ if (field->flags & TEP_FIELD_IS_ARRAY) {
if (field->arraylen)
field->elementsize = field->size / field->arraylen;
- else if (field->flags & FIELD_IS_DYNAMIC)
+ else if (field->flags & TEP_FIELD_IS_DYNAMIC)
field->elementsize = size_dynamic;
- else if (field->flags & FIELD_IS_STRING)
+ else if (field->flags & TEP_FIELD_IS_STRING)
field->elementsize = 1;
- else if (field->flags & FIELD_IS_LONG)
+ else if (field->flags & TEP_FIELD_IS_LONG)
field->elementsize = event->pevent ?
event->pevent->long_size :
sizeof(long);
return -1;
}
-static int event_read_format(struct event_format *event)
+static int event_read_format(struct tep_event_format *event)
{
char *token;
int ret;
- if (read_expected_item(EVENT_ITEM, "format") < 0)
+ if (read_expected_item(TEP_EVENT_ITEM, "format") < 0)
return -1;
- if (read_expected(EVENT_OP, ":") < 0)
+ if (read_expected(TEP_EVENT_OP, ":") < 0)
return -1;
- if (read_expect_type(EVENT_NEWLINE, &token))
+ if (read_expect_type(TEP_EVENT_NEWLINE, &token))
goto fail;
free_token(token);
return -1;
}
-static enum event_type
-process_arg_token(struct event_format *event, struct print_arg *arg,
- char **tok, enum event_type type);
+static enum tep_event_type
+process_arg_token(struct tep_event_format *event, struct tep_print_arg *arg,
+ char **tok, enum tep_event_type type);
-static enum event_type
-process_arg(struct event_format *event, struct print_arg *arg, char **tok)
+static enum tep_event_type
+process_arg(struct tep_event_format *event, struct tep_print_arg *arg, char **tok)
{
- enum event_type type;
+ enum tep_event_type type;
char *token;
type = read_token(&token);
return process_arg_token(event, arg, tok, type);
}
-static enum event_type
-process_op(struct event_format *event, struct print_arg *arg, char **tok);
+static enum tep_event_type
+process_op(struct tep_event_format *event, struct tep_print_arg *arg, char **tok);
/*
* For __print_symbolic() and __print_flags, we need to completely
* evaluate the first argument, which defines what to print next.
*/
-static enum event_type
-process_field_arg(struct event_format *event, struct print_arg *arg, char **tok)
+static enum tep_event_type
+process_field_arg(struct tep_event_format *event, struct tep_print_arg *arg, char **tok)
{
- enum event_type type;
+ enum tep_event_type type;
type = process_arg(event, arg, tok);
- while (type == EVENT_OP) {
+ while (type == TEP_EVENT_OP) {
type = process_op(event, arg, tok);
}
return type;
}
-static enum event_type
-process_cond(struct event_format *event, struct print_arg *top, char **tok)
+static enum tep_event_type
+process_cond(struct tep_event_format *event, struct tep_print_arg *top, char **tok)
{
- struct print_arg *arg, *left, *right;
- enum event_type type;
+ struct tep_print_arg *arg, *left, *right;
+ enum tep_event_type type;
char *token = NULL;
arg = alloc_arg();
goto out_free;
}
- arg->type = PRINT_OP;
+ arg->type = TEP_PRINT_OP;
arg->op.left = left;
arg->op.right = right;
type = process_arg(event, left, &token);
again:
- if (type == EVENT_ERROR)
+ if (type == TEP_EVENT_ERROR)
goto out_free;
/* Handle other operations in the arguments */
- if (type == EVENT_OP && strcmp(token, ":") != 0) {
+ if (type == TEP_EVENT_OP && strcmp(token, ":") != 0) {
type = process_op(event, left, &token);
goto again;
}
- if (test_type_token(type, token, EVENT_OP, ":"))
+ if (test_type_token(type, token, TEP_EVENT_OP, ":"))
goto out_free;
arg->op.op = token;
top->op.right = NULL;
free_token(token);
free_arg(arg);
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_array(struct event_format *event, struct print_arg *top, char **tok)
+static enum tep_event_type
+process_array(struct tep_event_format *event, struct tep_print_arg *top, char **tok)
{
- struct print_arg *arg;
- enum event_type type;
+ struct tep_print_arg *arg;
+ enum tep_event_type type;
char *token = NULL;
arg = alloc_arg();
do_warning_event(event, "%s: not enough memory!", __func__);
/* '*tok' is set to top->op.op. No need to free. */
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
*tok = NULL;
type = process_arg(event, arg, &token);
- if (test_type_token(type, token, EVENT_OP, "]"))
+ if (test_type_token(type, token, TEP_EVENT_OP, "]"))
goto out_free;
top->op.right = arg;
out_free:
free_token(token);
free_arg(arg);
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
static int get_op_prio(char *op)
}
}
-static int set_op_prio(struct print_arg *arg)
+static int set_op_prio(struct tep_print_arg *arg)
{
/* single ops are the greatest */
- if (!arg->op.left || arg->op.left->type == PRINT_NULL)
+ if (!arg->op.left || arg->op.left->type == TEP_PRINT_NULL)
arg->op.prio = 0;
else
arg->op.prio = get_op_prio(arg->op.op);
}
/* Note, *tok does not get freed, but will most likely be saved */
-static enum event_type
-process_op(struct event_format *event, struct print_arg *arg, char **tok)
+static enum tep_event_type
+process_op(struct tep_event_format *event, struct tep_print_arg *arg, char **tok)
{
- struct print_arg *left, *right = NULL;
- enum event_type type;
+ struct tep_print_arg *left, *right = NULL;
+ enum tep_event_type type;
char *token;
/* the op is passed in via tok */
token = *tok;
- if (arg->type == PRINT_OP && !arg->op.left) {
+ if (arg->type == TEP_PRINT_OP && !arg->op.left) {
/* handle single op */
if (token[1]) {
do_warning_event(event, "bad op token %s", token);
if (!left)
goto out_warn_free;
- left->type = PRINT_NULL;
+ left->type = TEP_PRINT_NULL;
arg->op.left = left;
right = alloc_arg();
/* copy the top arg to the left */
*left = *arg;
- arg->type = PRINT_OP;
+ arg->type = TEP_PRINT_OP;
arg->op.op = token;
arg->op.left = left;
arg->op.prio = 0;
/* copy the top arg to the left */
*left = *arg;
- arg->type = PRINT_OP;
+ arg->type = TEP_PRINT_OP;
arg->op.op = token;
arg->op.left = left;
arg->op.right = NULL;
if (set_op_prio(arg) == -1) {
- event->flags |= EVENT_FL_FAILED;
+ event->flags |= TEP_EVENT_FL_FAILED;
/* arg->op.op (= token) will be freed at out_free */
arg->op.op = NULL;
goto out_free;
/* could just be a type pointer */
if ((strcmp(arg->op.op, "*") == 0) &&
- type == EVENT_DELIM && (strcmp(token, ")") == 0)) {
+ type == TEP_EVENT_DELIM && (strcmp(token, ")") == 0)) {
char *new_atom;
- if (left->type != PRINT_ATOM) {
+ if (left->type != TEP_PRINT_ATOM) {
do_warning_event(event, "bad pointer type");
goto out_free;
}
goto out_warn_free;
type = process_arg_token(event, right, tok, type);
- if (type == EVENT_ERROR) {
+ if (type == TEP_EVENT_ERROR) {
free_arg(right);
/* token was freed in process_arg_token() via *tok */
token = NULL;
goto out_free;
}
- if (right->type == PRINT_OP &&
+ if (right->type == TEP_PRINT_OP &&
get_op_prio(arg->op.op) < get_op_prio(right->op.op)) {
- struct print_arg tmp;
+ struct tep_print_arg tmp;
/* rotate ops according to the priority */
arg->op.right = right->op.left;
*left = *arg;
- arg->type = PRINT_OP;
+ arg->type = TEP_PRINT_OP;
arg->op.op = token;
arg->op.left = left;
} else {
do_warning_event(event, "unknown op '%s'", token);
- event->flags |= EVENT_FL_FAILED;
+ event->flags |= TEP_EVENT_FL_FAILED;
/* the arg is now the left side */
goto out_free;
}
- if (type == EVENT_OP && strcmp(*tok, ":") != 0) {
+ if (type == TEP_EVENT_OP && strcmp(*tok, ":") != 0) {
int prio;
/* higher prios need to be closer to the root */
out_free:
free_token(token);
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_entry(struct event_format *event __maybe_unused, struct print_arg *arg,
+static enum tep_event_type
+process_entry(struct tep_event_format *event __maybe_unused, struct tep_print_arg *arg,
char **tok)
{
- enum event_type type;
+ enum tep_event_type type;
char *field;
char *token;
- if (read_expected(EVENT_OP, "->") < 0)
+ if (read_expected(TEP_EVENT_OP, "->") < 0)
goto out_err;
- if (read_expect_type(EVENT_ITEM, &token) < 0)
+ if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
goto out_free;
field = token;
- arg->type = PRINT_FIELD;
+ arg->type = TEP_PRINT_FIELD;
arg->field.name = field;
if (is_flag_field) {
arg->field.field = tep_find_any_field(event, arg->field.name);
- arg->field.field->flags |= FIELD_IS_FLAG;
+ arg->field.field->flags |= TEP_FIELD_IS_FLAG;
is_flag_field = 0;
} else if (is_symbolic_field) {
arg->field.field = tep_find_any_field(event, arg->field.name);
- arg->field.field->flags |= FIELD_IS_SYMBOLIC;
+ arg->field.field->flags |= TEP_FIELD_IS_SYMBOLIC;
is_symbolic_field = 0;
}
free_token(token);
out_err:
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static int alloc_and_process_delim(struct event_format *event, char *next_token,
- struct print_arg **print_arg)
+static int alloc_and_process_delim(struct tep_event_format *event, char *next_token,
+ struct tep_print_arg **print_arg)
{
- struct print_arg *field;
- enum event_type type;
+ struct tep_print_arg *field;
+ enum tep_event_type type;
char *token;
int ret = 0;
type = process_arg(event, field, &token);
- if (test_type_token(type, token, EVENT_DELIM, next_token)) {
+ if (test_type_token(type, token, TEP_EVENT_DELIM, next_token)) {
errno = EINVAL;
ret = -1;
free_arg(field);
return ret;
}
-static char *arg_eval (struct print_arg *arg);
+static char *arg_eval (struct tep_print_arg *arg);
static unsigned long long
eval_type_str(unsigned long long val, const char *type, int pointer)
* Try to figure out the type.
*/
static unsigned long long
-eval_type(unsigned long long val, struct print_arg *arg, int pointer)
+eval_type(unsigned long long val, struct tep_print_arg *arg, int pointer)
{
- if (arg->type != PRINT_TYPE) {
+ if (arg->type != TEP_PRINT_TYPE) {
do_warning("expected type argument");
return 0;
}
return eval_type_str(val, arg->typecast.type, pointer);
}
-static int arg_num_eval(struct print_arg *arg, long long *val)
+static int arg_num_eval(struct tep_print_arg *arg, long long *val)
{
long long left, right;
int ret = 1;
switch (arg->type) {
- case PRINT_ATOM:
+ case TEP_PRINT_ATOM:
*val = strtoll(arg->atom.atom, NULL, 0);
break;
- case PRINT_TYPE:
+ case TEP_PRINT_TYPE:
ret = arg_num_eval(arg->typecast.item, val);
if (!ret)
break;
*val = eval_type(*val, arg, 0);
break;
- case PRINT_OP:
+ case TEP_PRINT_OP:
switch (arg->op.op[0]) {
case '|':
ret = arg_num_eval(arg->op.left, &left);
break;
case '-':
/* check for negative */
- if (arg->op.left->type == PRINT_NULL)
+ if (arg->op.left->type == TEP_PRINT_NULL)
left = 0;
else
ret = arg_num_eval(arg->op.left, &left);
*val = left - right;
break;
case '+':
- if (arg->op.left->type == PRINT_NULL)
+ if (arg->op.left->type == TEP_PRINT_NULL)
left = 0;
else
ret = arg_num_eval(arg->op.left, &left);
}
break;
- case PRINT_NULL:
- case PRINT_FIELD ... PRINT_SYMBOL:
- case PRINT_STRING:
- case PRINT_BSTRING:
- case PRINT_BITMASK:
+ case TEP_PRINT_NULL:
+ case TEP_PRINT_FIELD ... TEP_PRINT_SYMBOL:
+ case TEP_PRINT_STRING:
+ case TEP_PRINT_BSTRING:
+ case TEP_PRINT_BITMASK:
default:
do_warning("invalid eval type %d", arg->type);
ret = 0;
return ret;
}
-static char *arg_eval (struct print_arg *arg)
+static char *arg_eval (struct tep_print_arg *arg)
{
long long val;
static char buf[20];
switch (arg->type) {
- case PRINT_ATOM:
+ case TEP_PRINT_ATOM:
return arg->atom.atom;
- case PRINT_TYPE:
+ case TEP_PRINT_TYPE:
return arg_eval(arg->typecast.item);
- case PRINT_OP:
+ case TEP_PRINT_OP:
if (!arg_num_eval(arg, &val))
break;
sprintf(buf, "%lld", val);
return buf;
- case PRINT_NULL:
- case PRINT_FIELD ... PRINT_SYMBOL:
- case PRINT_STRING:
- case PRINT_BSTRING:
- case PRINT_BITMASK:
+ case TEP_PRINT_NULL:
+ case TEP_PRINT_FIELD ... TEP_PRINT_SYMBOL:
+ case TEP_PRINT_STRING:
+ case TEP_PRINT_BSTRING:
+ case TEP_PRINT_BITMASK:
default:
do_warning("invalid eval type %d", arg->type);
break;
return NULL;
}
-static enum event_type
-process_fields(struct event_format *event, struct print_flag_sym **list, char **tok)
+static enum tep_event_type
+process_fields(struct tep_event_format *event, struct tep_print_flag_sym **list, char **tok)
{
- enum event_type type;
- struct print_arg *arg = NULL;
- struct print_flag_sym *field;
+ enum tep_event_type type;
+ struct tep_print_arg *arg = NULL;
+ struct tep_print_flag_sym *field;
char *token = *tok;
char *value;
do {
free_token(token);
type = read_token_item(&token);
- if (test_type_token(type, token, EVENT_OP, "{"))
+ if (test_type_token(type, token, TEP_EVENT_OP, "{"))
break;
arg = alloc_arg();
free_token(token);
type = process_arg(event, arg, &token);
- if (type == EVENT_OP)
+ if (type == TEP_EVENT_OP)
type = process_op(event, arg, &token);
- if (type == EVENT_ERROR)
+ if (type == TEP_EVENT_ERROR)
goto out_free;
- if (test_type_token(type, token, EVENT_DELIM, ","))
+ if (test_type_token(type, token, TEP_EVENT_DELIM, ","))
goto out_free;
field = calloc(1, sizeof(*field));
free_token(token);
type = process_arg(event, arg, &token);
- if (test_type_token(type, token, EVENT_OP, "}"))
+ if (test_type_token(type, token, TEP_EVENT_OP, "}"))
goto out_free_field;
value = arg_eval(arg);
free_token(token);
type = read_token_item(&token);
- } while (type == EVENT_DELIM && strcmp(token, ",") == 0);
+ } while (type == TEP_EVENT_DELIM && strcmp(token, ",") == 0);
*tok = token;
return type;
free_token(token);
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_flags(struct event_format *event, struct print_arg *arg, char **tok)
+static enum tep_event_type
+process_flags(struct tep_event_format *event, struct tep_print_arg *arg, char **tok)
{
- struct print_arg *field;
- enum event_type type;
+ struct tep_print_arg *field;
+ enum tep_event_type type;
char *token = NULL;
memset(arg, 0, sizeof(*arg));
- arg->type = PRINT_FLAGS;
+ arg->type = TEP_PRINT_FLAGS;
field = alloc_arg();
if (!field) {
type = process_field_arg(event, field, &token);
/* Handle operations in the first argument */
- while (type == EVENT_OP)
+ while (type == TEP_EVENT_OP)
type = process_op(event, field, &token);
- if (test_type_token(type, token, EVENT_DELIM, ","))
+ if (test_type_token(type, token, TEP_EVENT_DELIM, ","))
goto out_free_field;
free_token(token);
type = read_token_item(&token);
}
- if (test_type_token(type, token, EVENT_DELIM, ","))
+ if (test_type_token(type, token, TEP_EVENT_DELIM, ","))
goto out_free;
type = process_fields(event, &arg->flags.flags, &token);
- if (test_type_token(type, token, EVENT_DELIM, ")"))
+ if (test_type_token(type, token, TEP_EVENT_DELIM, ")"))
goto out_free;
free_token(token);
out_free:
free_token(token);
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_symbols(struct event_format *event, struct print_arg *arg, char **tok)
+static enum tep_event_type
+process_symbols(struct tep_event_format *event, struct tep_print_arg *arg, char **tok)
{
- struct print_arg *field;
- enum event_type type;
+ struct tep_print_arg *field;
+ enum tep_event_type type;
char *token = NULL;
memset(arg, 0, sizeof(*arg));
- arg->type = PRINT_SYMBOL;
+ arg->type = TEP_PRINT_SYMBOL;
field = alloc_arg();
if (!field) {
type = process_field_arg(event, field, &token);
- if (test_type_token(type, token, EVENT_DELIM, ","))
+ if (test_type_token(type, token, TEP_EVENT_DELIM, ","))
goto out_free_field;
arg->symbol.field = field;
type = process_fields(event, &arg->symbol.symbols, &token);
- if (test_type_token(type, token, EVENT_DELIM, ")"))
+ if (test_type_token(type, token, TEP_EVENT_DELIM, ")"))
goto out_free;
free_token(token);
out_free:
free_token(token);
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_hex_common(struct event_format *event, struct print_arg *arg,
- char **tok, enum print_arg_type type)
+static enum tep_event_type
+process_hex_common(struct tep_event_format *event, struct tep_print_arg *arg,
+ char **tok, enum tep_print_arg_type type)
{
memset(arg, 0, sizeof(*arg));
arg->type = type;
arg->hex.field = NULL;
out:
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_hex(struct event_format *event, struct print_arg *arg, char **tok)
+static enum tep_event_type
+process_hex(struct tep_event_format *event, struct tep_print_arg *arg, char **tok)
{
- return process_hex_common(event, arg, tok, PRINT_HEX);
+ return process_hex_common(event, arg, tok, TEP_PRINT_HEX);
}
-static enum event_type
-process_hex_str(struct event_format *event, struct print_arg *arg,
+static enum tep_event_type
+process_hex_str(struct tep_event_format *event, struct tep_print_arg *arg,
char **tok)
{
- return process_hex_common(event, arg, tok, PRINT_HEX_STR);
+ return process_hex_common(event, arg, tok, TEP_PRINT_HEX_STR);
}
-static enum event_type
-process_int_array(struct event_format *event, struct print_arg *arg, char **tok)
+static enum tep_event_type
+process_int_array(struct tep_event_format *event, struct tep_print_arg *arg, char **tok)
{
memset(arg, 0, sizeof(*arg));
- arg->type = PRINT_INT_ARRAY;
+ arg->type = TEP_PRINT_INT_ARRAY;
if (alloc_and_process_delim(event, ",", &arg->int_array.field))
goto out;
arg->int_array.field = NULL;
out:
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_dynamic_array(struct event_format *event, struct print_arg *arg, char **tok)
+static enum tep_event_type
+process_dynamic_array(struct tep_event_format *event, struct tep_print_arg *arg, char **tok)
{
- struct format_field *field;
- enum event_type type;
+ struct tep_format_field *field;
+ enum tep_event_type type;
char *token;
memset(arg, 0, sizeof(*arg));
- arg->type = PRINT_DYNAMIC_ARRAY;
+ arg->type = TEP_PRINT_DYNAMIC_ARRAY;
/*
* The item within the parenthesis is another field that holds
*/
type = read_token(&token);
*tok = token;
- if (type != EVENT_ITEM)
+ if (type != TEP_EVENT_ITEM)
goto out_free;
/* Find the field */
arg->dynarray.field = field;
arg->dynarray.index = 0;
- if (read_expected(EVENT_DELIM, ")") < 0)
+ if (read_expected(TEP_EVENT_DELIM, ")") < 0)
goto out_free;
free_token(token);
type = read_token_item(&token);
*tok = token;
- if (type != EVENT_OP || strcmp(token, "[") != 0)
+ if (type != TEP_EVENT_OP || strcmp(token, "[") != 0)
return type;
free_token(token);
if (!arg) {
do_warning_event(event, "%s: not enough memory!", __func__);
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
type = process_arg(event, arg, &token);
- if (type == EVENT_ERROR)
+ if (type == TEP_EVENT_ERROR)
goto out_free_arg;
- if (!test_type_token(type, token, EVENT_OP, "]"))
+ if (!test_type_token(type, token, TEP_EVENT_OP, "]"))
goto out_free_arg;
free_token(token);
out_free:
free_token(token);
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_dynamic_array_len(struct event_format *event, struct print_arg *arg,
+static enum tep_event_type
+process_dynamic_array_len(struct tep_event_format *event, struct tep_print_arg *arg,
char **tok)
{
- struct format_field *field;
- enum event_type type;
+ struct tep_format_field *field;
+ enum tep_event_type type;
char *token;
- if (read_expect_type(EVENT_ITEM, &token) < 0)
+ if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
goto out_free;
- arg->type = PRINT_DYNAMIC_ARRAY_LEN;
+ arg->type = TEP_PRINT_DYNAMIC_ARRAY_LEN;
/* Find the field */
field = tep_find_field(event, token);
arg->dynarray.field = field;
arg->dynarray.index = 0;
- if (read_expected(EVENT_DELIM, ")") < 0)
+ if (read_expected(TEP_EVENT_DELIM, ")") < 0)
goto out_err;
type = read_token(&token);
free_token(token);
out_err:
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_paren(struct event_format *event, struct print_arg *arg, char **tok)
+static enum tep_event_type
+process_paren(struct tep_event_format *event, struct tep_print_arg *arg, char **tok)
{
- struct print_arg *item_arg;
- enum event_type type;
+ struct tep_print_arg *item_arg;
+ enum tep_event_type type;
char *token;
type = process_arg(event, arg, &token);
- if (type == EVENT_ERROR)
+ if (type == TEP_EVENT_ERROR)
goto out_free;
- if (type == EVENT_OP)
+ if (type == TEP_EVENT_OP)
type = process_op(event, arg, &token);
- if (type == EVENT_ERROR)
+ if (type == TEP_EVENT_ERROR)
goto out_free;
- if (test_type_token(type, token, EVENT_DELIM, ")"))
+ if (test_type_token(type, token, TEP_EVENT_DELIM, ")"))
goto out_free;
free_token(token);
* this was a typecast.
*/
if (event_item_type(type) ||
- (type == EVENT_DELIM && strcmp(token, "(") == 0)) {
+ (type == TEP_EVENT_DELIM && strcmp(token, "(") == 0)) {
/* make this a typecast and contine */
/* prevous must be an atom */
- if (arg->type != PRINT_ATOM) {
- do_warning_event(event, "previous needed to be PRINT_ATOM");
+ if (arg->type != TEP_PRINT_ATOM) {
+ do_warning_event(event, "previous needed to be TEP_PRINT_ATOM");
goto out_free;
}
goto out_free;
}
- arg->type = PRINT_TYPE;
+ arg->type = TEP_PRINT_TYPE;
arg->typecast.type = arg->atom.atom;
arg->typecast.item = item_arg;
type = process_arg_token(event, item_arg, &token, type);
out_free:
free_token(token);
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_str(struct event_format *event __maybe_unused, struct print_arg *arg,
+static enum tep_event_type
+process_str(struct tep_event_format *event __maybe_unused, struct tep_print_arg *arg,
char **tok)
{
- enum event_type type;
+ enum tep_event_type type;
char *token;
- if (read_expect_type(EVENT_ITEM, &token) < 0)
+ if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
goto out_free;
- arg->type = PRINT_STRING;
+ arg->type = TEP_PRINT_STRING;
arg->string.string = token;
arg->string.offset = -1;
- if (read_expected(EVENT_DELIM, ")") < 0)
+ if (read_expected(TEP_EVENT_DELIM, ")") < 0)
goto out_err;
type = read_token(&token);
free_token(token);
out_err:
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_bitmask(struct event_format *event __maybe_unused, struct print_arg *arg,
- char **tok)
+static enum tep_event_type
+process_bitmask(struct tep_event_format *event __maybe_unused, struct tep_print_arg *arg,
+ char **tok)
{
- enum event_type type;
+ enum tep_event_type type;
char *token;
- if (read_expect_type(EVENT_ITEM, &token) < 0)
+ if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
goto out_free;
- arg->type = PRINT_BITMASK;
+ arg->type = TEP_PRINT_BITMASK;
arg->bitmask.bitmask = token;
arg->bitmask.offset = -1;
- if (read_expected(EVENT_DELIM, ")") < 0)
+ if (read_expected(TEP_EVENT_DELIM, ")") < 0)
goto out_err;
type = read_token(&token);
free_token(token);
out_err:
*tok = NULL;
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
static struct tep_function_handler *
}
}
-static enum event_type
-process_func_handler(struct event_format *event, struct tep_function_handler *func,
- struct print_arg *arg, char **tok)
+static enum tep_event_type
+process_func_handler(struct tep_event_format *event, struct tep_function_handler *func,
+ struct tep_print_arg *arg, char **tok)
{
- struct print_arg **next_arg;
- struct print_arg *farg;
- enum event_type type;
+ struct tep_print_arg **next_arg;
+ struct tep_print_arg *farg;
+ enum tep_event_type type;
char *token;
int i;
- arg->type = PRINT_FUNC;
+ arg->type = TEP_PRINT_FUNC;
arg->func.func = func;
*tok = NULL;
if (!farg) {
do_warning_event(event, "%s: not enough memory!",
__func__);
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
type = process_arg(event, farg, &token);
if (i < (func->nr_args - 1)) {
- if (type != EVENT_DELIM || strcmp(token, ",") != 0) {
+ if (type != TEP_EVENT_DELIM || strcmp(token, ",") != 0) {
do_warning_event(event,
"Error: function '%s()' expects %d arguments but event %s only uses %d",
func->name, func->nr_args,
goto err;
}
} else {
- if (type != EVENT_DELIM || strcmp(token, ")") != 0) {
+ if (type != TEP_EVENT_DELIM || strcmp(token, ")") != 0) {
do_warning_event(event,
"Error: function '%s()' only expects %d arguments but event %s has more",
func->name, func->nr_args, event->name);
err:
free_arg(farg);
free_token(token);
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_function(struct event_format *event, struct print_arg *arg,
+static enum tep_event_type
+process_function(struct tep_event_format *event, struct tep_print_arg *arg,
char *token, char **tok)
{
struct tep_function_handler *func;
do_warning_event(event, "function %s not defined", token);
free_token(token);
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
-static enum event_type
-process_arg_token(struct event_format *event, struct print_arg *arg,
- char **tok, enum event_type type)
+static enum tep_event_type
+process_arg_token(struct tep_event_format *event, struct tep_print_arg *arg,
+ char **tok, enum tep_event_type type)
{
char *token;
char *atom;
token = *tok;
switch (type) {
- case EVENT_ITEM:
+ case TEP_EVENT_ITEM:
if (strcmp(token, "REC") == 0) {
free_token(token);
type = process_entry(event, arg, &token);
* If the next token is a parenthesis, then this
* is a function.
*/
- if (type == EVENT_DELIM && strcmp(token, "(") == 0) {
+ if (type == TEP_EVENT_DELIM && strcmp(token, "(") == 0) {
free_token(token);
token = NULL;
/* this will free atom. */
break;
}
/* atoms can be more than one token long */
- while (type == EVENT_ITEM) {
+ while (type == TEP_EVENT_ITEM) {
char *new_atom;
new_atom = realloc(atom,
strlen(atom) + strlen(token) + 2);
free(atom);
*tok = NULL;
free_token(token);
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
atom = new_atom;
strcat(atom, " ");
type = read_token_item(&token);
}
- arg->type = PRINT_ATOM;
+ arg->type = TEP_PRINT_ATOM;
arg->atom.atom = atom;
break;
- case EVENT_DQUOTE:
- case EVENT_SQUOTE:
- arg->type = PRINT_ATOM;
+ case TEP_EVENT_DQUOTE:
+ case TEP_EVENT_SQUOTE:
+ arg->type = TEP_PRINT_ATOM;
arg->atom.atom = token;
type = read_token_item(&token);
break;
- case EVENT_DELIM:
+ case TEP_EVENT_DELIM:
if (strcmp(token, "(") == 0) {
free_token(token);
type = process_paren(event, arg, &token);
break;
}
- case EVENT_OP:
+ case TEP_EVENT_OP:
/* handle single ops */
- arg->type = PRINT_OP;
+ arg->type = TEP_PRINT_OP;
arg->op.op = token;
arg->op.left = NULL;
type = process_op(event, arg, &token);
/* On error, the op is freed */
- if (type == EVENT_ERROR)
+ if (type == TEP_EVENT_ERROR)
arg->op.op = NULL;
/* return error type if errored */
break;
- case EVENT_ERROR ... EVENT_NEWLINE:
+ case TEP_EVENT_ERROR ... TEP_EVENT_NEWLINE:
default:
do_warning_event(event, "unexpected type %d", type);
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
*tok = token;
return type;
}
-static int event_read_print_args(struct event_format *event, struct print_arg **list)
+static int event_read_print_args(struct tep_event_format *event, struct tep_print_arg **list)
{
- enum event_type type = EVENT_ERROR;
- struct print_arg *arg;
+ enum tep_event_type type = TEP_EVENT_ERROR;
+ struct tep_print_arg *arg;
char *token;
int args = 0;
do {
- if (type == EVENT_NEWLINE) {
+ if (type == TEP_EVENT_NEWLINE) {
type = read_token_item(&token);
continue;
}
type = process_arg(event, arg, &token);
- if (type == EVENT_ERROR) {
+ if (type == TEP_EVENT_ERROR) {
free_token(token);
free_arg(arg);
return -1;
*list = arg;
args++;
- if (type == EVENT_OP) {
+ if (type == TEP_EVENT_OP) {
type = process_op(event, arg, &token);
free_token(token);
- if (type == EVENT_ERROR) {
+ if (type == TEP_EVENT_ERROR) {
*list = NULL;
free_arg(arg);
return -1;
continue;
}
- if (type == EVENT_DELIM && strcmp(token, ",") == 0) {
+ if (type == TEP_EVENT_DELIM && strcmp(token, ",") == 0) {
free_token(token);
*list = arg;
list = &arg->next;
continue;
}
break;
- } while (type != EVENT_NONE);
+ } while (type != TEP_EVENT_NONE);
- if (type != EVENT_NONE && type != EVENT_ERROR)
+ if (type != TEP_EVENT_NONE && type != TEP_EVENT_ERROR)
free_token(token);
return args;
}
-static int event_read_print(struct event_format *event)
+static int event_read_print(struct tep_event_format *event)
{
- enum event_type type;
+ enum tep_event_type type;
char *token;
int ret;
- if (read_expected_item(EVENT_ITEM, "print") < 0)
+ if (read_expected_item(TEP_EVENT_ITEM, "print") < 0)
return -1;
- if (read_expected(EVENT_ITEM, "fmt") < 0)
+ if (read_expected(TEP_EVENT_ITEM, "fmt") < 0)
return -1;
- if (read_expected(EVENT_OP, ":") < 0)
+ if (read_expected(TEP_EVENT_OP, ":") < 0)
return -1;
- if (read_expect_type(EVENT_DQUOTE, &token) < 0)
+ if (read_expect_type(TEP_EVENT_DQUOTE, &token) < 0)
goto fail;
concat:
/* ok to have no arg */
type = read_token_item(&token);
- if (type == EVENT_NONE)
+ if (type == TEP_EVENT_NONE)
return 0;
/* Handle concatenation of print lines */
- if (type == EVENT_DQUOTE) {
+ if (type == TEP_EVENT_DQUOTE) {
char *cat;
if (asprintf(&cat, "%s%s", event->print_fmt.format, token) < 0)
goto concat;
}
- if (test_type_token(type, token, EVENT_DELIM, ","))
+ if (test_type_token(type, token, TEP_EVENT_DELIM, ","))
goto fail;
free_token(token);
* Returns a common field from the event by the given @name.
* This only searchs the common fields and not all field.
*/
-struct format_field *
-tep_find_common_field(struct event_format *event, const char *name)
+struct tep_format_field *
+tep_find_common_field(struct tep_event_format *event, const char *name)
{
- struct format_field *format;
+ struct tep_format_field *format;
for (format = event->format.common_fields;
format; format = format->next) {
* Returns a non-common field by the given @name.
* This does not search common fields.
*/
-struct format_field *
-tep_find_field(struct event_format *event, const char *name)
+struct tep_format_field *
+tep_find_field(struct tep_event_format *event, const char *name)
{
- struct format_field *format;
+ struct tep_format_field *format;
for (format = event->format.fields;
format; format = format->next) {
* This searchs the common field names first, then
* the non-common ones if a common one was not found.
*/
-struct format_field *
-tep_find_any_field(struct event_format *event, const char *name)
+struct tep_format_field *
+tep_find_any_field(struct tep_event_format *event, const char *name)
{
- struct format_field *format;
+ struct tep_format_field *format;
format = tep_find_common_field(event, name);
if (format)
case 1:
return *(unsigned char *)ptr;
case 2:
- return data2host2(pevent, ptr);
+ return tep_data2host2(pevent, ptr);
case 4:
- return data2host4(pevent, ptr);
+ return tep_data2host4(pevent, ptr);
case 8:
- return data2host8(pevent, ptr);
+ return tep_data2host8(pevent, ptr);
default:
/* BUG! */
return 0;
*
* Returns 0 on success, -1 otherwise.
*/
-int tep_read_number_field(struct format_field *field, const void *data,
+int tep_read_number_field(struct tep_format_field *field, const void *data,
unsigned long long *value)
{
if (!field)
static int get_common_info(struct tep_handle *pevent,
const char *type, int *offset, int *size)
{
- struct event_format *event;
- struct format_field *field;
+ struct tep_event_format *event;
+ struct tep_format_field *field;
/*
* All events should have the same common elements.
*
* Returns an event that has a given @id.
*/
-struct event_format *tep_find_event(struct tep_handle *pevent, int id)
+struct tep_event_format *tep_find_event(struct tep_handle *pevent, int id)
{
- struct event_format **eventptr;
- struct event_format key;
- struct event_format *pkey = &key;
+ struct tep_event_format **eventptr;
+ struct tep_event_format key;
+ struct tep_event_format *pkey = &key;
/* Check cache first */
if (pevent->last_event && pevent->last_event->id == id)
* This returns an event with a given @name and under the system
* @sys. If @sys is NULL the first event with @name is returned.
*/
-struct event_format *
+struct tep_event_format *
tep_find_event_by_name(struct tep_handle *pevent,
const char *sys, const char *name)
{
- struct event_format *event;
+ struct tep_event_format *event;
int i;
if (pevent->last_event &&
}
static unsigned long long
-eval_num_arg(void *data, int size, struct event_format *event, struct print_arg *arg)
+eval_num_arg(void *data, int size, struct tep_event_format *event, struct tep_print_arg *arg)
{
struct tep_handle *pevent = event->pevent;
unsigned long long val = 0;
unsigned long long left, right;
- struct print_arg *typearg = NULL;
- struct print_arg *larg;
+ struct tep_print_arg *typearg = NULL;
+ struct tep_print_arg *larg;
unsigned long offset;
unsigned int field_size;
switch (arg->type) {
- case PRINT_NULL:
+ case TEP_PRINT_NULL:
/* ?? */
return 0;
- case PRINT_ATOM:
+ case TEP_PRINT_ATOM:
return strtoull(arg->atom.atom, NULL, 0);
- case PRINT_FIELD:
+ case TEP_PRINT_FIELD:
if (!arg->field.field) {
arg->field.field = tep_find_any_field(event, arg->field.name);
if (!arg->field.field)
val = tep_read_number(pevent, data + arg->field.field->offset,
arg->field.field->size);
break;
- case PRINT_FLAGS:
- case PRINT_SYMBOL:
- case PRINT_INT_ARRAY:
- case PRINT_HEX:
- case PRINT_HEX_STR:
+ case TEP_PRINT_FLAGS:
+ case TEP_PRINT_SYMBOL:
+ case TEP_PRINT_INT_ARRAY:
+ case TEP_PRINT_HEX:
+ case TEP_PRINT_HEX_STR:
break;
- case PRINT_TYPE:
+ case TEP_PRINT_TYPE:
val = eval_num_arg(data, size, event, arg->typecast.item);
return eval_type(val, arg, 0);
- case PRINT_STRING:
- case PRINT_BSTRING:
- case PRINT_BITMASK:
+ case TEP_PRINT_STRING:
+ case TEP_PRINT_BSTRING:
+ case TEP_PRINT_BITMASK:
return 0;
- case PRINT_FUNC: {
+ case TEP_PRINT_FUNC: {
struct trace_seq s;
trace_seq_init(&s);
val = process_defined_func(&s, data, size, event, arg);
trace_seq_destroy(&s);
return val;
}
- case PRINT_OP:
+ case TEP_PRINT_OP:
if (strcmp(arg->op.op, "[") == 0) {
/*
* Arrays are special, since we don't want
/* handle typecasts */
larg = arg->op.left;
- while (larg->type == PRINT_TYPE) {
+ while (larg->type == TEP_PRINT_TYPE) {
if (!typearg)
typearg = larg;
larg = larg->typecast.item;
field_size = pevent->long_size;
switch (larg->type) {
- case PRINT_DYNAMIC_ARRAY:
+ case TEP_PRINT_DYNAMIC_ARRAY:
offset = tep_read_number(pevent,
data + larg->dynarray.field->offset,
larg->dynarray.field->size);
offset &= 0xffff;
offset += right;
break;
- case PRINT_FIELD:
+ case TEP_PRINT_FIELD:
if (!larg->field.field) {
larg->field.field =
tep_find_any_field(event, larg->field.name);
goto out_warning_op;
}
break;
- case PRINT_DYNAMIC_ARRAY_LEN:
+ case TEP_PRINT_DYNAMIC_ARRAY_LEN:
offset = tep_read_number(pevent,
data + arg->dynarray.field->offset,
arg->dynarray.field->size);
*/
val = (unsigned long long)(offset >> 16);
break;
- case PRINT_DYNAMIC_ARRAY:
+ case TEP_PRINT_DYNAMIC_ARRAY:
/* Without [], we pass the address to the dynamic data */
offset = tep_read_number(pevent,
data + arg->dynarray.field->offset,
}
static void print_str_arg(struct trace_seq *s, void *data, int size,
- struct event_format *event, const char *format,
- int len_arg, struct print_arg *arg)
+ struct tep_event_format *event, const char *format,
+ int len_arg, struct tep_print_arg *arg)
{
struct tep_handle *pevent = event->pevent;
- struct print_flag_sym *flag;
- struct format_field *field;
+ struct tep_print_flag_sym *flag;
+ struct tep_format_field *field;
struct printk_map *printk;
long long val, fval;
unsigned long long addr;
int i, len;
switch (arg->type) {
- case PRINT_NULL:
+ case TEP_PRINT_NULL:
/* ?? */
return;
- case PRINT_ATOM:
+ case TEP_PRINT_ATOM:
print_str_to_seq(s, format, len_arg, arg->atom.atom);
return;
- case PRINT_FIELD:
+ case TEP_PRINT_FIELD:
field = arg->field.field;
if (!field) {
field = tep_find_any_field(event, arg->field.name);
* and the size is the same as long_size, assume that it
* is a pointer.
*/
- if (!(field->flags & FIELD_IS_ARRAY) &&
+ if (!(field->flags & TEP_FIELD_IS_ARRAY) &&
field->size == pevent->long_size) {
/* Handle heterogeneous recording and processing
print_str_to_seq(s, format, len_arg, str);
free(str);
break;
- case PRINT_FLAGS:
+ case TEP_PRINT_FLAGS:
val = eval_num_arg(data, size, event, arg->flags.field);
print = 0;
for (flag = arg->flags.flags; flag; flag = flag->next) {
trace_seq_printf(s, "0x%llx", val);
}
break;
- case PRINT_SYMBOL:
+ case TEP_PRINT_SYMBOL:
val = eval_num_arg(data, size, event, arg->symbol.field);
for (flag = arg->symbol.symbols; flag; flag = flag->next) {
fval = eval_flag(flag->value);
if (!flag)
trace_seq_printf(s, "0x%llx", val);
break;
- case PRINT_HEX:
- case PRINT_HEX_STR:
- if (arg->hex.field->type == PRINT_DYNAMIC_ARRAY) {
+ case TEP_PRINT_HEX:
+ case TEP_PRINT_HEX_STR:
+ if (arg->hex.field->type == TEP_PRINT_DYNAMIC_ARRAY) {
unsigned long offset;
offset = tep_read_number(pevent,
data + arg->hex.field->dynarray.field->offset,
}
len = eval_num_arg(data, size, event, arg->hex.size);
for (i = 0; i < len; i++) {
- if (i && arg->type == PRINT_HEX)
+ if (i && arg->type == TEP_PRINT_HEX)
trace_seq_putc(s, ' ');
trace_seq_printf(s, "%02x", hex[i]);
}
break;
- case PRINT_INT_ARRAY: {
+ case TEP_PRINT_INT_ARRAY: {
void *num;
int el_size;
- if (arg->int_array.field->type == PRINT_DYNAMIC_ARRAY) {
+ if (arg->int_array.field->type == TEP_PRINT_DYNAMIC_ARRAY) {
unsigned long offset;
- struct format_field *field =
+ struct tep_format_field *field =
arg->int_array.field->dynarray.field;
offset = tep_read_number(pevent,
data + field->offset,
}
break;
}
- case PRINT_TYPE:
+ case TEP_PRINT_TYPE:
break;
- case PRINT_STRING: {
+ case TEP_PRINT_STRING: {
int str_offset;
if (arg->string.offset == -1) {
- struct format_field *f;
+ struct tep_format_field *f;
f = tep_find_any_field(event, arg->string.string);
arg->string.offset = f->offset;
}
- str_offset = data2host4(pevent, data + arg->string.offset);
+ str_offset = tep_data2host4(pevent, data + arg->string.offset);
str_offset &= 0xffff;
print_str_to_seq(s, format, len_arg, ((char *)data) + str_offset);
break;
}
- case PRINT_BSTRING:
+ case TEP_PRINT_BSTRING:
print_str_to_seq(s, format, len_arg, arg->string.string);
break;
- case PRINT_BITMASK: {
+ case TEP_PRINT_BITMASK: {
int bitmask_offset;
int bitmask_size;
if (arg->bitmask.offset == -1) {
- struct format_field *f;
+ struct tep_format_field *f;
f = tep_find_any_field(event, arg->bitmask.bitmask);
arg->bitmask.offset = f->offset;
}
- bitmask_offset = data2host4(pevent, data + arg->bitmask.offset);
+ bitmask_offset = tep_data2host4(pevent, data + arg->bitmask.offset);
bitmask_size = bitmask_offset >> 16;
bitmask_offset &= 0xffff;
print_bitmask_to_seq(pevent, s, format, len_arg,
data + bitmask_offset, bitmask_size);
break;
}
- case PRINT_OP:
+ case TEP_PRINT_OP:
/*
* The only op for string should be ? :
*/
print_str_arg(s, data, size, event,
format, len_arg, arg->op.right->op.right);
break;
- case PRINT_FUNC:
+ case TEP_PRINT_FUNC:
process_defined_func(s, data, size, event, arg);
break;
default:
static unsigned long long
process_defined_func(struct trace_seq *s, void *data, int size,
- struct event_format *event, struct print_arg *arg)
+ struct tep_event_format *event, struct tep_print_arg *arg)
{
struct tep_function_handler *func_handle = arg->func.func;
struct func_params *param;
unsigned long long *args;
unsigned long long ret;
- struct print_arg *farg;
+ struct tep_print_arg *farg;
struct trace_seq str;
struct save_str {
struct save_str *next;
return ret;
}
-static void free_args(struct print_arg *args)
+static void free_args(struct tep_print_arg *args)
{
- struct print_arg *next;
+ struct tep_print_arg *next;
while (args) {
next = args->next;
}
}
-static struct print_arg *make_bprint_args(char *fmt, void *data, int size, struct event_format *event)
+static struct tep_print_arg *make_bprint_args(char *fmt, void *data, int size, struct tep_event_format *event)
{
struct tep_handle *pevent = event->pevent;
- struct format_field *field, *ip_field;
- struct print_arg *args, *arg, **next;
+ struct tep_format_field *field, *ip_field;
+ struct tep_print_arg *args, *arg, **next;
unsigned long long ip, val;
char *ptr;
void *bptr;
arg->next = NULL;
next = &arg->next;
- arg->type = PRINT_ATOM;
+ arg->type = TEP_PRINT_ATOM;
if (asprintf(&arg->atom.atom, "%lld", ip) < 0)
goto out_free;
goto out_free;
}
arg->next = NULL;
- arg->type = PRINT_ATOM;
+ arg->type = TEP_PRINT_ATOM;
if (asprintf(&arg->atom.atom, "%lld", val) < 0) {
free(arg);
goto out_free;
goto out_free;
}
arg->next = NULL;
- arg->type = PRINT_BSTRING;
+ arg->type = TEP_PRINT_BSTRING;
arg->string.string = strdup(bptr);
if (!arg->string.string)
goto out_free;
static char *
get_bprint_format(void *data, int size __maybe_unused,
- struct event_format *event)
+ struct tep_event_format *event)
{
struct tep_handle *pevent = event->pevent;
unsigned long long addr;
- struct format_field *field;
+ struct tep_format_field *field;
struct printk_map *printk;
char *format;
}
static void print_mac_arg(struct trace_seq *s, int mac, void *data, int size,
- struct event_format *event, struct print_arg *arg)
+ struct tep_event_format *event, struct tep_print_arg *arg)
{
unsigned char *buf;
const char *fmt = "%.2x:%.2x:%.2x:%.2x:%.2x:%.2x";
- if (arg->type == PRINT_FUNC) {
+ if (arg->type == TEP_PRINT_FUNC) {
process_defined_func(s, data, size, event, arg);
return;
}
- if (arg->type != PRINT_FIELD) {
+ if (arg->type != TEP_PRINT_FIELD) {
trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d",
arg->type);
return;
* %pISpc print an IP address based on sockaddr; p adds port.
*/
static int print_ipv4_arg(struct trace_seq *s, const char *ptr, char i,
- void *data, int size, struct event_format *event,
- struct print_arg *arg)
+ void *data, int size, struct tep_event_format *event,
+ struct tep_print_arg *arg)
{
unsigned char *buf;
- if (arg->type == PRINT_FUNC) {
+ if (arg->type == TEP_PRINT_FUNC) {
process_defined_func(s, data, size, event, arg);
return 0;
}
- if (arg->type != PRINT_FIELD) {
+ if (arg->type != TEP_PRINT_FIELD) {
trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d", arg->type);
return 0;
}
}
static int print_ipv6_arg(struct trace_seq *s, const char *ptr, char i,
- void *data, int size, struct event_format *event,
- struct print_arg *arg)
+ void *data, int size, struct tep_event_format *event,
+ struct tep_print_arg *arg)
{
char have_c = 0;
unsigned char *buf;
rc++;
}
- if (arg->type == PRINT_FUNC) {
+ if (arg->type == TEP_PRINT_FUNC) {
process_defined_func(s, data, size, event, arg);
return rc;
}
- if (arg->type != PRINT_FIELD) {
+ if (arg->type != TEP_PRINT_FIELD) {
trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d", arg->type);
return rc;
}
}
static int print_ipsa_arg(struct trace_seq *s, const char *ptr, char i,
- void *data, int size, struct event_format *event,
- struct print_arg *arg)
+ void *data, int size, struct tep_event_format *event,
+ struct tep_print_arg *arg)
{
char have_c = 0, have_p = 0;
unsigned char *buf;
}
}
- if (arg->type == PRINT_FUNC) {
+ if (arg->type == TEP_PRINT_FUNC) {
process_defined_func(s, data, size, event, arg);
return rc;
}
- if (arg->type != PRINT_FIELD) {
+ if (arg->type != TEP_PRINT_FIELD) {
trace_seq_printf(s, "ARG TYPE NOT FIELD BUT %d", arg->type);
return rc;
}
}
static int print_ip_arg(struct trace_seq *s, const char *ptr,
- void *data, int size, struct event_format *event,
- struct print_arg *arg)
+ void *data, int size, struct tep_event_format *event,
+ struct tep_print_arg *arg)
{
char i = *ptr; /* 'i' or 'I' */
char ver;
}
void tep_print_field(struct trace_seq *s, void *data,
- struct format_field *field)
+ struct tep_format_field *field)
{
unsigned long long val;
unsigned int offset, len, i;
struct tep_handle *pevent = field->event->pevent;
- if (field->flags & FIELD_IS_ARRAY) {
+ if (field->flags & TEP_FIELD_IS_ARRAY) {
offset = field->offset;
len = field->size;
- if (field->flags & FIELD_IS_DYNAMIC) {
+ if (field->flags & TEP_FIELD_IS_DYNAMIC) {
val = tep_read_number(pevent, data + offset, len);
offset = val;
len = offset >> 16;
offset &= 0xffff;
}
- if (field->flags & FIELD_IS_STRING &&
+ if (field->flags & TEP_FIELD_IS_STRING &&
is_printable_array(data + offset, len)) {
trace_seq_printf(s, "%s", (char *)data + offset);
} else {
*((unsigned char *)data + offset + i));
}
trace_seq_putc(s, ']');
- field->flags &= ~FIELD_IS_STRING;
+ field->flags &= ~TEP_FIELD_IS_STRING;
}
} else {
val = tep_read_number(pevent, data + field->offset,
field->size);
- if (field->flags & FIELD_IS_POINTER) {
+ if (field->flags & TEP_FIELD_IS_POINTER) {
trace_seq_printf(s, "0x%llx", val);
- } else if (field->flags & FIELD_IS_SIGNED) {
+ } else if (field->flags & TEP_FIELD_IS_SIGNED) {
switch (field->size) {
case 4:
/*
* If field is long then print it in hex.
* A long usually stores pointers.
*/
- if (field->flags & FIELD_IS_LONG)
+ if (field->flags & TEP_FIELD_IS_LONG)
trace_seq_printf(s, "0x%x", (int)val);
else
trace_seq_printf(s, "%d", (int)val);
trace_seq_printf(s, "%lld", val);
}
} else {
- if (field->flags & FIELD_IS_LONG)
+ if (field->flags & TEP_FIELD_IS_LONG)
trace_seq_printf(s, "0x%llx", val);
else
trace_seq_printf(s, "%llu", val);
}
void tep_print_fields(struct trace_seq *s, void *data,
- int size __maybe_unused, struct event_format *event)
+ int size __maybe_unused, struct tep_event_format *event)
{
- struct format_field *field;
+ struct tep_format_field *field;
field = event->format.fields;
while (field) {
}
}
-static void pretty_print(struct trace_seq *s, void *data, int size, struct event_format *event)
+static void pretty_print(struct trace_seq *s, void *data, int size, struct tep_event_format *event)
{
struct tep_handle *pevent = event->pevent;
- struct print_fmt *print_fmt = &event->print_fmt;
- struct print_arg *arg = print_fmt->args;
- struct print_arg *args = NULL;
+ struct tep_print_fmt *print_fmt = &event->print_fmt;
+ struct tep_print_arg *arg = print_fmt->args;
+ struct tep_print_arg *args = NULL;
const char *ptr = print_fmt->format;
unsigned long long val;
struct func_map *func;
int len;
int ls;
- if (event->flags & EVENT_FL_FAILED) {
+ if (event->flags & TEP_EVENT_FL_FAILED) {
trace_seq_printf(s, "[FAILED TO PARSE]");
tep_print_fields(s, data, size, event);
return;
}
- if (event->flags & EVENT_FL_ISBPRINT) {
+ if (event->flags & TEP_EVENT_FL_ISBPRINT) {
bprint_fmt = get_bprint_format(data, size, event);
args = make_bprint_args(bprint_fmt, data, size, event);
arg = args;
/* The argument is the length. */
if (!arg) {
do_warning_event(event, "no argument match");
- event->flags |= EVENT_FL_FAILED;
+ event->flags |= TEP_EVENT_FL_FAILED;
goto out_failed;
}
len_arg = eval_num_arg(data, size, event, arg);
if (isalnum(ptr[1]))
ptr++;
- if (arg->type == PRINT_BSTRING) {
+ if (arg->type == TEP_PRINT_BSTRING) {
trace_seq_puts(s, arg->string.string);
break;
}
case 'u':
if (!arg) {
do_warning_event(event, "no argument match");
- event->flags |= EVENT_FL_FAILED;
+ event->flags |= TEP_EVENT_FL_FAILED;
goto out_failed;
}
/* should never happen */
if (len > 31) {
do_warning_event(event, "bad format!");
- event->flags |= EVENT_FL_FAILED;
+ event->flags |= TEP_EVENT_FL_FAILED;
len = 31;
}
break;
default:
do_warning_event(event, "bad count (%d)", ls);
- event->flags |= EVENT_FL_FAILED;
+ event->flags |= TEP_EVENT_FL_FAILED;
}
break;
case 's':
if (!arg) {
do_warning_event(event, "no matching argument");
- event->flags |= EVENT_FL_FAILED;
+ event->flags |= TEP_EVENT_FL_FAILED;
goto out_failed;
}
/* should never happen */
if (len > 31) {
do_warning_event(event, "bad format!");
- event->flags |= EVENT_FL_FAILED;
+ event->flags |= TEP_EVENT_FL_FAILED;
len = 31;
}
trace_seq_putc(s, *ptr);
}
- if (event->flags & EVENT_FL_FAILED) {
+ if (event->flags & TEP_EVENT_FL_FAILED) {
out_failed:
trace_seq_printf(s, "[FAILED TO PARSE]");
}
*
* This returns the event form a given @type;
*/
-struct event_format *tep_data_event_from_type(struct tep_handle *pevent, int type)
+struct tep_event_format *tep_data_event_from_type(struct tep_handle *pevent, int type)
{
return tep_find_event(pevent, type);
}
* This parses the raw @data using the given @event information and
* writes the print format into the trace_seq.
*/
-void tep_event_info(struct trace_seq *s, struct event_format *event,
+void tep_event_info(struct trace_seq *s, struct tep_event_format *event,
struct tep_record *record)
{
int print_pretty = 1;
- if (event->pevent->print_raw || (event->flags & EVENT_FL_PRINTRAW))
+ if (event->pevent->print_raw || (event->flags & TEP_EVENT_FL_PRINTRAW))
tep_print_fields(s, record->data, record->size, event);
else {
- if (event->handler && !(event->flags & EVENT_FL_NOHANDLE))
+ if (event->handler && !(event->flags & TEP_EVENT_FL_NOHANDLE))
print_pretty = event->handler(s, record, event,
event->context);
* Returns the associated event for a given record, or NULL if non is
* is found.
*/
-struct event_format *
+struct tep_event_format *
tep_find_event_by_record(struct tep_handle *pevent, struct tep_record *record)
{
int type;
* Writes the tasks comm, pid and CPU to @s.
*/
void tep_print_event_task(struct tep_handle *pevent, struct trace_seq *s,
- struct event_format *event,
+ struct tep_event_format *event,
struct tep_record *record)
{
void *data = record->data;
* Writes the timestamp of the record into @s.
*/
void tep_print_event_time(struct tep_handle *pevent, struct trace_seq *s,
- struct event_format *event,
+ struct tep_event_format *event,
struct tep_record *record,
bool use_trace_clock)
{
* Writes the parsing of the record's data to @s.
*/
void tep_print_event_data(struct tep_handle *pevent, struct trace_seq *s,
- struct event_format *event,
+ struct tep_event_format *event,
struct tep_record *record)
{
static const char *spaces = " "; /* 20 spaces */
void tep_print_event(struct tep_handle *pevent, struct trace_seq *s,
struct tep_record *record, bool use_trace_clock)
{
- struct event_format *event;
+ struct tep_event_format *event;
event = tep_find_event_by_record(pevent, record);
if (!event) {
static int events_id_cmp(const void *a, const void *b)
{
- struct event_format * const * ea = a;
- struct event_format * const * eb = b;
+ struct tep_event_format * const * ea = a;
+ struct tep_event_format * const * eb = b;
if ((*ea)->id < (*eb)->id)
return -1;
static int events_name_cmp(const void *a, const void *b)
{
- struct event_format * const * ea = a;
- struct event_format * const * eb = b;
+ struct tep_event_format * const * ea = a;
+ struct tep_event_format * const * eb = b;
int res;
res = strcmp((*ea)->name, (*eb)->name);
static int events_system_cmp(const void *a, const void *b)
{
- struct event_format * const * ea = a;
- struct event_format * const * eb = b;
+ struct tep_event_format * const * ea = a;
+ struct tep_event_format * const * eb = b;
int res;
res = strcmp((*ea)->system, (*eb)->system);
return events_id_cmp(a, b);
}
-struct event_format **tep_list_events(struct tep_handle *pevent, enum event_sort_type sort_type)
+struct tep_event_format **tep_list_events(struct tep_handle *pevent, enum tep_event_sort_type sort_type)
{
- struct event_format **events;
+ struct tep_event_format **events;
int (*sort)(const void *a, const void *b);
events = pevent->sort_events;
pevent->sort_events = events;
/* the internal events are sorted by id */
- if (sort_type == EVENT_SORT_ID) {
+ if (sort_type == TEP_EVENT_SORT_ID) {
pevent->last_type = sort_type;
return events;
}
}
switch (sort_type) {
- case EVENT_SORT_ID:
+ case TEP_EVENT_SORT_ID:
sort = events_id_cmp;
break;
- case EVENT_SORT_NAME:
+ case TEP_EVENT_SORT_NAME:
sort = events_name_cmp;
break;
- case EVENT_SORT_SYSTEM:
+ case TEP_EVENT_SORT_SYSTEM:
sort = events_system_cmp;
break;
default:
return events;
}
-static struct format_field **
+static struct tep_format_field **
get_event_fields(const char *type, const char *name,
- int count, struct format_field *list)
+ int count, struct tep_format_field *list)
{
- struct format_field **fields;
- struct format_field *field;
+ struct tep_format_field **fields;
+ struct tep_format_field *field;
int i = 0;
fields = malloc(sizeof(*fields) * (count + 1));
* Returns an allocated array of fields. The last item in the array is NULL.
* The array must be freed with free().
*/
-struct format_field **tep_event_common_fields(struct event_format *event)
+struct tep_format_field **tep_event_common_fields(struct tep_event_format *event)
{
return get_event_fields("common", event->name,
event->format.nr_common,
* Returns an allocated array of fields. The last item in the array is NULL.
* The array must be freed with free().
*/
-struct format_field **tep_event_fields(struct event_format *event)
+struct tep_format_field **tep_event_fields(struct tep_event_format *event)
{
return get_event_fields("event", event->name,
event->format.nr_fields,
event->format.fields);
}
-static void print_fields(struct trace_seq *s, struct print_flag_sym *field)
+static void print_fields(struct trace_seq *s, struct tep_print_flag_sym *field)
{
trace_seq_printf(s, "{ %s, %s }", field->value, field->str);
if (field->next) {
}
/* for debugging */
-static void print_args(struct print_arg *args)
+static void print_args(struct tep_print_arg *args)
{
int print_paren = 1;
struct trace_seq s;
switch (args->type) {
- case PRINT_NULL:
+ case TEP_PRINT_NULL:
printf("null");
break;
- case PRINT_ATOM:
+ case TEP_PRINT_ATOM:
printf("%s", args->atom.atom);
break;
- case PRINT_FIELD:
+ case TEP_PRINT_FIELD:
printf("REC->%s", args->field.name);
break;
- case PRINT_FLAGS:
+ case TEP_PRINT_FLAGS:
printf("__print_flags(");
print_args(args->flags.field);
printf(", %s, ", args->flags.delim);
trace_seq_destroy(&s);
printf(")");
break;
- case PRINT_SYMBOL:
+ case TEP_PRINT_SYMBOL:
printf("__print_symbolic(");
print_args(args->symbol.field);
printf(", ");
trace_seq_destroy(&s);
printf(")");
break;
- case PRINT_HEX:
+ case TEP_PRINT_HEX:
printf("__print_hex(");
print_args(args->hex.field);
printf(", ");
print_args(args->hex.size);
printf(")");
break;
- case PRINT_HEX_STR:
+ case TEP_PRINT_HEX_STR:
printf("__print_hex_str(");
print_args(args->hex.field);
printf(", ");
print_args(args->hex.size);
printf(")");
break;
- case PRINT_INT_ARRAY:
+ case TEP_PRINT_INT_ARRAY:
printf("__print_array(");
print_args(args->int_array.field);
printf(", ");
print_args(args->int_array.el_size);
printf(")");
break;
- case PRINT_STRING:
- case PRINT_BSTRING:
+ case TEP_PRINT_STRING:
+ case TEP_PRINT_BSTRING:
printf("__get_str(%s)", args->string.string);
break;
- case PRINT_BITMASK:
+ case TEP_PRINT_BITMASK:
printf("__get_bitmask(%s)", args->bitmask.bitmask);
break;
- case PRINT_TYPE:
+ case TEP_PRINT_TYPE:
printf("(%s)", args->typecast.type);
print_args(args->typecast.item);
break;
- case PRINT_OP:
+ case TEP_PRINT_OP:
if (strcmp(args->op.op, ":") == 0)
print_paren = 0;
if (print_paren)
save_input_buf_ptr = input_buf_ptr;
save_input_buf_siz = input_buf_siz;
- if (read_expected(EVENT_ITEM, "field") < 0)
+ if (read_expected(TEP_EVENT_ITEM, "field") < 0)
return;
- if (read_expected(EVENT_OP, ":") < 0)
+ if (read_expected(TEP_EVENT_OP, ":") < 0)
return;
/* type */
- if (read_expect_type(EVENT_ITEM, &token) < 0)
+ if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
goto fail;
free_token(token);
* If this is not a mandatory field, then test it first.
*/
if (mandatory) {
- if (read_expected(EVENT_ITEM, field) < 0)
+ if (read_expected(TEP_EVENT_ITEM, field) < 0)
return;
} else {
- if (read_expect_type(EVENT_ITEM, &token) < 0)
+ if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
goto fail;
if (strcmp(token, field) != 0)
goto discard;
free_token(token);
}
- if (read_expected(EVENT_OP, ";") < 0)
+ if (read_expected(TEP_EVENT_OP, ";") < 0)
return;
- if (read_expected(EVENT_ITEM, "offset") < 0)
+ if (read_expected(TEP_EVENT_ITEM, "offset") < 0)
return;
- if (read_expected(EVENT_OP, ":") < 0)
+ if (read_expected(TEP_EVENT_OP, ":") < 0)
return;
- if (read_expect_type(EVENT_ITEM, &token) < 0)
+ if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
goto fail;
*offset = atoi(token);
free_token(token);
- if (read_expected(EVENT_OP, ";") < 0)
+ if (read_expected(TEP_EVENT_OP, ";") < 0)
return;
- if (read_expected(EVENT_ITEM, "size") < 0)
+ if (read_expected(TEP_EVENT_ITEM, "size") < 0)
return;
- if (read_expected(EVENT_OP, ":") < 0)
+ if (read_expected(TEP_EVENT_OP, ":") < 0)
return;
- if (read_expect_type(EVENT_ITEM, &token) < 0)
+ if (read_expect_type(TEP_EVENT_ITEM, &token) < 0)
goto fail;
*size = atoi(token);
free_token(token);
- if (read_expected(EVENT_OP, ";") < 0)
+ if (read_expected(TEP_EVENT_OP, ";") < 0)
return;
type = read_token(&token);
- if (type != EVENT_NEWLINE) {
+ if (type != TEP_EVENT_NEWLINE) {
/* newer versions of the kernel have a "signed" type */
- if (type != EVENT_ITEM)
+ if (type != TEP_EVENT_ITEM)
goto fail;
if (strcmp(token, "signed") != 0)
free_token(token);
- if (read_expected(EVENT_OP, ":") < 0)
+ if (read_expected(TEP_EVENT_OP, ":") < 0)
return;
- if (read_expect_type(EVENT_ITEM, &token))
+ if (read_expect_type(TEP_EVENT_ITEM, &token))
goto fail;
free_token(token);
- if (read_expected(EVENT_OP, ";") < 0)
+ if (read_expected(TEP_EVENT_OP, ";") < 0)
return;
- if (read_expect_type(EVENT_NEWLINE, &token))
+ if (read_expect_type(TEP_EVENT_NEWLINE, &token))
goto fail;
}
fail:
return 0;
}
-static int event_matches(struct event_format *event,
+static int event_matches(struct tep_event_format *event,
int id, const char *sys_name,
const char *event_name)
{
free(handle);
}
-static int find_event_handle(struct tep_handle *pevent, struct event_format *event)
+static int find_event_handle(struct tep_handle *pevent, struct tep_event_format *event)
{
struct event_handler *handle, **next;
*
* /sys/kernel/debug/tracing/events/.../.../format
*/
-enum tep_errno __tep_parse_format(struct event_format **eventp,
+enum tep_errno __tep_parse_format(struct tep_event_format **eventp,
struct tep_handle *pevent, const char *buf,
unsigned long size, const char *sys)
{
- struct event_format *event;
+ struct tep_event_format *event;
int ret;
init_input_buf(buf, size);
}
if (strcmp(sys, "ftrace") == 0) {
- event->flags |= EVENT_FL_ISFTRACE;
+ event->flags |= TEP_EVENT_FL_ISFTRACE;
if (strcmp(event->name, "bprint") == 0)
- event->flags |= EVENT_FL_ISBPRINT;
+ event->flags |= TEP_EVENT_FL_ISBPRINT;
}
event->id = event_read_id();
goto event_parse_failed;
}
- if (!ret && (event->flags & EVENT_FL_ISFTRACE)) {
- struct format_field *field;
- struct print_arg *arg, **list;
+ if (!ret && (event->flags & TEP_EVENT_FL_ISFTRACE)) {
+ struct tep_format_field *field;
+ struct tep_print_arg *arg, **list;
/* old ftrace had no args */
list = &event->print_fmt.args;
for (field = event->format.fields; field; field = field->next) {
arg = alloc_arg();
if (!arg) {
- event->flags |= EVENT_FL_FAILED;
+ event->flags |= TEP_EVENT_FL_FAILED;
return TEP_ERRNO__OLD_FTRACE_ARG_FAILED;
}
- arg->type = PRINT_FIELD;
+ arg->type = TEP_PRINT_FIELD;
arg->field.name = strdup(field->name);
if (!arg->field.name) {
- event->flags |= EVENT_FL_FAILED;
+ event->flags |= TEP_EVENT_FL_FAILED;
free_arg(arg);
return TEP_ERRNO__OLD_FTRACE_ARG_FAILED;
}
return 0;
event_parse_failed:
- event->flags |= EVENT_FL_FAILED;
+ event->flags |= TEP_EVENT_FL_FAILED;
return ret;
event_alloc_failed:
static enum tep_errno
__parse_event(struct tep_handle *pevent,
- struct event_format **eventp,
+ struct tep_event_format **eventp,
const char *buf, unsigned long size,
const char *sys)
{
int ret = __tep_parse_format(eventp, pevent, buf, size, sys);
- struct event_format *event = *eventp;
+ struct tep_event_format *event = *eventp;
if (event == NULL)
return ret;
* /sys/kernel/debug/tracing/events/.../.../format
*/
enum tep_errno tep_parse_format(struct tep_handle *pevent,
- struct event_format **eventp,
+ struct tep_event_format **eventp,
const char *buf,
unsigned long size, const char *sys)
{
enum tep_errno tep_parse_event(struct tep_handle *pevent, const char *buf,
unsigned long size, const char *sys)
{
- struct event_format *event = NULL;
+ struct tep_event_format *event = NULL;
return __parse_event(pevent, &event, buf, size, sys);
}
-#undef _PE
-#define _PE(code, str) str
-static const char * const tep_error_str[] = {
- TEP_ERRORS
-};
-#undef _PE
-
-int tep_strerror(struct tep_handle *pevent __maybe_unused,
- enum tep_errno errnum, char *buf, size_t buflen)
-{
- int idx;
- const char *msg;
-
- if (errnum >= 0) {
- str_error_r(errnum, buf, buflen);
- return 0;
- }
-
- if (errnum <= __TEP_ERRNO__START ||
- errnum >= __TEP_ERRNO__END)
- return -1;
-
- idx = errnum - __TEP_ERRNO__START - 1;
- msg = tep_error_str[idx];
- snprintf(buf, buflen, "%s", msg);
-
- return 0;
-}
-
-int get_field_val(struct trace_seq *s, struct format_field *field,
+int get_field_val(struct trace_seq *s, struct tep_format_field *field,
const char *name, struct tep_record *record,
unsigned long long *val, int err)
{
*
* On failure, it returns NULL.
*/
-void *tep_get_field_raw(struct trace_seq *s, struct event_format *event,
+void *tep_get_field_raw(struct trace_seq *s, struct tep_event_format *event,
const char *name, struct tep_record *record,
int *len, int err)
{
- struct format_field *field;
+ struct tep_format_field *field;
void *data = record->data;
unsigned offset;
int dummy;
len = &dummy;
offset = field->offset;
- if (field->flags & FIELD_IS_DYNAMIC) {
+ if (field->flags & TEP_FIELD_IS_DYNAMIC) {
offset = tep_read_number(event->pevent,
data + offset, field->size);
*len = offset >> 16;
*
* Returns 0 on success -1 on field not found.
*/
-int tep_get_field_val(struct trace_seq *s, struct event_format *event,
+int tep_get_field_val(struct trace_seq *s, struct tep_event_format *event,
const char *name, struct tep_record *record,
unsigned long long *val, int err)
{
- struct format_field *field;
+ struct tep_format_field *field;
if (!event)
return -1;
*
* Returns 0 on success -1 on field not found.
*/
-int tep_get_common_field_val(struct trace_seq *s, struct event_format *event,
+int tep_get_common_field_val(struct trace_seq *s, struct tep_event_format *event,
const char *name, struct tep_record *record,
unsigned long long *val, int err)
{
- struct format_field *field;
+ struct tep_format_field *field;
if (!event)
return -1;
*
* Returns 0 on success -1 on field not found.
*/
-int tep_get_any_field_val(struct trace_seq *s, struct event_format *event,
+int tep_get_any_field_val(struct trace_seq *s, struct tep_event_format *event,
const char *name, struct tep_record *record,
unsigned long long *val, int err)
{
- struct format_field *field;
+ struct tep_format_field *field;
if (!event)
return -1;
* Returns: 0 on success, -1 field not found, or 1 if buffer is full.
*/
int tep_print_num_field(struct trace_seq *s, const char *fmt,
- struct event_format *event, const char *name,
+ struct tep_event_format *event, const char *name,
struct tep_record *record, int err)
{
- struct format_field *field = tep_find_field(event, name);
+ struct tep_format_field *field = tep_find_field(event, name);
unsigned long long val;
if (!field)
* Returns: 0 on success, -1 field not found, or 1 if buffer is full.
*/
int tep_print_func_field(struct trace_seq *s, const char *fmt,
- struct event_format *event, const char *name,
+ struct tep_event_format *event, const char *name,
struct tep_record *record, int err)
{
- struct format_field *field = tep_find_field(event, name);
+ struct tep_format_field *field = tep_find_field(event, name);
struct tep_handle *pevent = event->pevent;
unsigned long long val;
struct func_map *func;
return -1;
}
-static struct event_format *search_event(struct tep_handle *pevent, int id,
+static struct tep_event_format *search_event(struct tep_handle *pevent, int id,
const char *sys_name,
const char *event_name)
{
- struct event_format *event;
+ struct tep_event_format *event;
if (id >= 0) {
/* search by id */
const char *sys_name, const char *event_name,
tep_event_handler_func func, void *context)
{
- struct event_format *event;
+ struct tep_event_format *event;
struct event_handler *handle;
event = search_event(pevent, id, sys_name, event_name);
const char *sys_name, const char *event_name,
tep_event_handler_func func, void *context)
{
- struct event_format *event;
+ struct tep_event_format *event;
struct event_handler *handle;
struct event_handler **next;
pevent->ref_count++;
}
-void tep_free_format_field(struct format_field *field)
+void tep_free_format_field(struct tep_format_field *field)
{
free(field->type);
if (field->alias != field->name)
free(field);
}
-static void free_format_fields(struct format_field *field)
+static void free_format_fields(struct tep_format_field *field)
{
- struct format_field *next;
+ struct tep_format_field *next;
while (field) {
next = field->next;
}
}
-static void free_formats(struct format *format)
+static void free_formats(struct tep_format *format)
{
free_format_fields(format->common_fields);
free_format_fields(format->fields);
}
-void tep_free_format(struct event_format *event)
+void tep_free_format(struct tep_event_format *event)
{
free(event->name);
free(event->system);
#include <regex.h>
#include <string.h>
+#include "trace-seq.h"
+
#ifndef __maybe_unused
#define __maybe_unused __attribute__((unused))
#endif
-/* ----------------------- trace_seq ----------------------- */
-
-
-#ifndef TRACE_SEQ_BUF_SIZE
-#define TRACE_SEQ_BUF_SIZE 4096
-#endif
-
#ifndef DEBUG_RECORD
#define DEBUG_RECORD 0
#endif
#endif
};
-enum trace_seq_fail {
- TRACE_SEQ__GOOD,
- TRACE_SEQ__BUFFER_POISONED,
- TRACE_SEQ__MEM_ALLOC_FAILED,
-};
-
-/*
- * Trace sequences are used to allow a function to call several other functions
- * to create a string of data to use (up to a max of PAGE_SIZE).
- */
-
-struct trace_seq {
- char *buffer;
- unsigned int buffer_size;
- unsigned int len;
- unsigned int readpos;
- enum trace_seq_fail state;
-};
-
-void trace_seq_init(struct trace_seq *s);
-void trace_seq_reset(struct trace_seq *s);
-void trace_seq_destroy(struct trace_seq *s);
-
-extern int trace_seq_printf(struct trace_seq *s, const char *fmt, ...)
- __attribute__ ((format (printf, 2, 3)));
-extern int trace_seq_vprintf(struct trace_seq *s, const char *fmt, va_list args)
- __attribute__ ((format (printf, 2, 0)));
-
-extern int trace_seq_puts(struct trace_seq *s, const char *str);
-extern int trace_seq_putc(struct trace_seq *s, unsigned char c);
-
-extern void trace_seq_terminate(struct trace_seq *s);
-
-extern int trace_seq_do_fprintf(struct trace_seq *s, FILE *fp);
-extern int trace_seq_do_printf(struct trace_seq *s);
-
-
-/* ----------------------- pevent ----------------------- */
+/* ----------------------- tep ----------------------- */
struct tep_handle;
-struct event_format;
+struct tep_event_format;
typedef int (*tep_event_handler_func)(struct trace_seq *s,
struct tep_record *record,
- struct event_format *event,
+ struct tep_event_format *event,
void *context);
typedef int (*tep_plugin_load_func)(struct tep_handle *pevent);
#define TEP_PLUGIN_OPTIONS_NAME MAKE_STR(TEP_PLUGIN_OPTIONS)
#define TEP_PLUGIN_ALIAS_NAME MAKE_STR(TEP_PLUGIN_ALIAS)
-enum format_flags {
- FIELD_IS_ARRAY = 1,
- FIELD_IS_POINTER = 2,
- FIELD_IS_SIGNED = 4,
- FIELD_IS_STRING = 8,
- FIELD_IS_DYNAMIC = 16,
- FIELD_IS_LONG = 32,
- FIELD_IS_FLAG = 64,
- FIELD_IS_SYMBOLIC = 128,
+enum tep_format_flags {
+ TEP_FIELD_IS_ARRAY = 1,
+ TEP_FIELD_IS_POINTER = 2,
+ TEP_FIELD_IS_SIGNED = 4,
+ TEP_FIELD_IS_STRING = 8,
+ TEP_FIELD_IS_DYNAMIC = 16,
+ TEP_FIELD_IS_LONG = 32,
+ TEP_FIELD_IS_FLAG = 64,
+ TEP_FIELD_IS_SYMBOLIC = 128,
};
-struct format_field {
- struct format_field *next;
- struct event_format *event;
+struct tep_format_field {
+ struct tep_format_field *next;
+ struct tep_event_format *event;
char *type;
char *name;
char *alias;
unsigned long flags;
};
-struct format {
+struct tep_format {
int nr_common;
int nr_fields;
- struct format_field *common_fields;
- struct format_field *fields;
+ struct tep_format_field *common_fields;
+ struct tep_format_field *fields;
};
-struct print_arg_atom {
+struct tep_print_arg_atom {
char *atom;
};
-struct print_arg_string {
+struct tep_print_arg_string {
char *string;
int offset;
};
-struct print_arg_bitmask {
+struct tep_print_arg_bitmask {
char *bitmask;
int offset;
};
-struct print_arg_field {
+struct tep_print_arg_field {
char *name;
- struct format_field *field;
+ struct tep_format_field *field;
};
-struct print_flag_sym {
- struct print_flag_sym *next;
- char *value;
- char *str;
+struct tep_print_flag_sym {
+ struct tep_print_flag_sym *next;
+ char *value;
+ char *str;
};
-struct print_arg_typecast {
+struct tep_print_arg_typecast {
char *type;
- struct print_arg *item;
+ struct tep_print_arg *item;
};
-struct print_arg_flags {
- struct print_arg *field;
- char *delim;
- struct print_flag_sym *flags;
+struct tep_print_arg_flags {
+ struct tep_print_arg *field;
+ char *delim;
+ struct tep_print_flag_sym *flags;
};
-struct print_arg_symbol {
- struct print_arg *field;
- struct print_flag_sym *symbols;
+struct tep_print_arg_symbol {
+ struct tep_print_arg *field;
+ struct tep_print_flag_sym *symbols;
};
-struct print_arg_hex {
- struct print_arg *field;
- struct print_arg *size;
+struct tep_print_arg_hex {
+ struct tep_print_arg *field;
+ struct tep_print_arg *size;
};
-struct print_arg_int_array {
- struct print_arg *field;
- struct print_arg *count;
- struct print_arg *el_size;
+struct tep_print_arg_int_array {
+ struct tep_print_arg *field;
+ struct tep_print_arg *count;
+ struct tep_print_arg *el_size;
};
-struct print_arg_dynarray {
- struct format_field *field;
- struct print_arg *index;
+struct tep_print_arg_dynarray {
+ struct tep_format_field *field;
+ struct tep_print_arg *index;
};
-struct print_arg;
+struct tep_print_arg;
-struct print_arg_op {
+struct tep_print_arg_op {
char *op;
int prio;
- struct print_arg *left;
- struct print_arg *right;
+ struct tep_print_arg *left;
+ struct tep_print_arg *right;
};
struct tep_function_handler;
-struct print_arg_func {
+struct tep_print_arg_func {
struct tep_function_handler *func;
- struct print_arg *args;
-};
-
-enum print_arg_type {
- PRINT_NULL,
- PRINT_ATOM,
- PRINT_FIELD,
- PRINT_FLAGS,
- PRINT_SYMBOL,
- PRINT_HEX,
- PRINT_INT_ARRAY,
- PRINT_TYPE,
- PRINT_STRING,
- PRINT_BSTRING,
- PRINT_DYNAMIC_ARRAY,
- PRINT_OP,
- PRINT_FUNC,
- PRINT_BITMASK,
- PRINT_DYNAMIC_ARRAY_LEN,
- PRINT_HEX_STR,
-};
-
-struct print_arg {
- struct print_arg *next;
- enum print_arg_type type;
+ struct tep_print_arg *args;
+};
+
+enum tep_print_arg_type {
+ TEP_PRINT_NULL,
+ TEP_PRINT_ATOM,
+ TEP_PRINT_FIELD,
+ TEP_PRINT_FLAGS,
+ TEP_PRINT_SYMBOL,
+ TEP_PRINT_HEX,
+ TEP_PRINT_INT_ARRAY,
+ TEP_PRINT_TYPE,
+ TEP_PRINT_STRING,
+ TEP_PRINT_BSTRING,
+ TEP_PRINT_DYNAMIC_ARRAY,
+ TEP_PRINT_OP,
+ TEP_PRINT_FUNC,
+ TEP_PRINT_BITMASK,
+ TEP_PRINT_DYNAMIC_ARRAY_LEN,
+ TEP_PRINT_HEX_STR,
+};
+
+struct tep_print_arg {
+ struct tep_print_arg *next;
+ enum tep_print_arg_type type;
union {
- struct print_arg_atom atom;
- struct print_arg_field field;
- struct print_arg_typecast typecast;
- struct print_arg_flags flags;
- struct print_arg_symbol symbol;
- struct print_arg_hex hex;
- struct print_arg_int_array int_array;
- struct print_arg_func func;
- struct print_arg_string string;
- struct print_arg_bitmask bitmask;
- struct print_arg_op op;
- struct print_arg_dynarray dynarray;
+ struct tep_print_arg_atom atom;
+ struct tep_print_arg_field field;
+ struct tep_print_arg_typecast typecast;
+ struct tep_print_arg_flags flags;
+ struct tep_print_arg_symbol symbol;
+ struct tep_print_arg_hex hex;
+ struct tep_print_arg_int_array int_array;
+ struct tep_print_arg_func func;
+ struct tep_print_arg_string string;
+ struct tep_print_arg_bitmask bitmask;
+ struct tep_print_arg_op op;
+ struct tep_print_arg_dynarray dynarray;
};
};
-struct print_fmt {
+struct tep_print_fmt {
char *format;
- struct print_arg *args;
+ struct tep_print_arg *args;
};
-struct event_format {
+struct tep_event_format {
struct tep_handle *pevent;
char *name;
int id;
int flags;
- struct format format;
- struct print_fmt print_fmt;
+ struct tep_format format;
+ struct tep_print_fmt print_fmt;
char *system;
tep_event_handler_func handler;
void *context;
};
enum {
- EVENT_FL_ISFTRACE = 0x01,
- EVENT_FL_ISPRINT = 0x02,
- EVENT_FL_ISBPRINT = 0x04,
- EVENT_FL_ISFUNCENT = 0x10,
- EVENT_FL_ISFUNCRET = 0x20,
- EVENT_FL_NOHANDLE = 0x40,
- EVENT_FL_PRINTRAW = 0x80,
+ TEP_EVENT_FL_ISFTRACE = 0x01,
+ TEP_EVENT_FL_ISPRINT = 0x02,
+ TEP_EVENT_FL_ISBPRINT = 0x04,
+ TEP_EVENT_FL_ISFUNCENT = 0x10,
+ TEP_EVENT_FL_ISFUNCRET = 0x20,
+ TEP_EVENT_FL_NOHANDLE = 0x40,
+ TEP_EVENT_FL_PRINTRAW = 0x80,
- EVENT_FL_FAILED = 0x80000000
+ TEP_EVENT_FL_FAILED = 0x80000000
};
-enum event_sort_type {
- EVENT_SORT_ID,
- EVENT_SORT_NAME,
- EVENT_SORT_SYSTEM,
+enum tep_event_sort_type {
+ TEP_EVENT_SORT_ID,
+ TEP_EVENT_SORT_NAME,
+ TEP_EVENT_SORT_SYSTEM,
};
-enum event_type {
- EVENT_ERROR,
- EVENT_NONE,
- EVENT_SPACE,
- EVENT_NEWLINE,
- EVENT_OP,
- EVENT_DELIM,
- EVENT_ITEM,
- EVENT_DQUOTE,
- EVENT_SQUOTE,
+enum tep_event_type {
+ TEP_EVENT_ERROR,
+ TEP_EVENT_NONE,
+ TEP_EVENT_SPACE,
+ TEP_EVENT_NEWLINE,
+ TEP_EVENT_OP,
+ TEP_EVENT_DELIM,
+ TEP_EVENT_ITEM,
+ TEP_EVENT_DQUOTE,
+ TEP_EVENT_SQUOTE,
};
typedef unsigned long long (*tep_func_handler)(struct trace_seq *s,
};
#undef _PE
-struct plugin_list;
+struct tep_plugin_list;
#define INVALID_PLUGIN_LIST_OPTION ((char **)((unsigned long)-1))
-struct plugin_list *tep_load_plugins(struct tep_handle *pevent);
-void tep_unload_plugins(struct plugin_list *plugin_list,
+struct tep_plugin_list *tep_load_plugins(struct tep_handle *pevent);
+void tep_unload_plugins(struct tep_plugin_list *plugin_list,
struct tep_handle *pevent);
char **tep_plugin_list_options(void);
void tep_plugin_free_options_list(char **list);
void tep_plugin_remove_options(struct tep_plugin_option *options);
void tep_print_plugins(struct trace_seq *s,
const char *prefix, const char *suffix,
- const struct plugin_list *list);
-
-struct cmdline;
-struct cmdline_list;
-struct func_map;
-struct func_list;
-struct event_handler;
-struct func_resolver;
+ const struct tep_plugin_list *list);
+/* tep_handle */
typedef char *(tep_func_resolver_t)(void *priv,
unsigned long long *addrp, char **modp);
+void tep_set_flag(struct tep_handle *tep, int flag);
+unsigned short __tep_data2host2(struct tep_handle *pevent, unsigned short data);
+unsigned int __tep_data2host4(struct tep_handle *pevent, unsigned int data);
+unsigned long long
+__tep_data2host8(struct tep_handle *pevent, unsigned long long data);
-struct tep_handle {
- int ref_count;
-
- int header_page_ts_offset;
- int header_page_ts_size;
- int header_page_size_offset;
- int header_page_size_size;
- int header_page_data_offset;
- int header_page_data_size;
- int header_page_overwrite;
-
- int file_bigendian;
- int host_bigendian;
-
- int latency_format;
-
- int old_format;
-
- int cpus;
- int long_size;
- int page_size;
-
- struct cmdline *cmdlines;
- struct cmdline_list *cmdlist;
- int cmdline_count;
-
- struct func_map *func_map;
- struct func_resolver *func_resolver;
- struct func_list *funclist;
- unsigned int func_count;
-
- struct printk_map *printk_map;
- struct printk_list *printklist;
- unsigned int printk_count;
-
-
- struct event_format **events;
- int nr_events;
- struct event_format **sort_events;
- enum event_sort_type last_type;
-
- int type_offset;
- int type_size;
-
- int pid_offset;
- int pid_size;
-
- int pc_offset;
- int pc_size;
-
- int flags_offset;
- int flags_size;
-
- int ld_offset;
- int ld_size;
-
- int print_raw;
-
- int test_filters;
-
- int flags;
-
- struct format_field *bprint_ip_field;
- struct format_field *bprint_fmt_field;
- struct format_field *bprint_buf_field;
-
- struct event_handler *handlers;
- struct tep_function_handler *func_handlers;
-
- /* cache */
- struct event_format *last_event;
-
- char *trace_clock;
-};
-
-static inline void tep_set_flag(struct tep_handle *pevent, int flag)
-{
- pevent->flags |= flag;
-}
-
-static inline unsigned short
-__data2host2(struct tep_handle *pevent, unsigned short data)
-{
- unsigned short swap;
-
- if (pevent->host_bigendian == pevent->file_bigendian)
- return data;
-
- swap = ((data & 0xffULL) << 8) |
- ((data & (0xffULL << 8)) >> 8);
-
- return swap;
-}
-
-static inline unsigned int
-__data2host4(struct tep_handle *pevent, unsigned int data)
-{
- unsigned int swap;
-
- if (pevent->host_bigendian == pevent->file_bigendian)
- return data;
-
- swap = ((data & 0xffULL) << 24) |
- ((data & (0xffULL << 8)) << 8) |
- ((data & (0xffULL << 16)) >> 8) |
- ((data & (0xffULL << 24)) >> 24);
-
- return swap;
-}
-
-static inline unsigned long long
-__data2host8(struct tep_handle *pevent, unsigned long long data)
-{
- unsigned long long swap;
-
- if (pevent->host_bigendian == pevent->file_bigendian)
- return data;
-
- swap = ((data & 0xffULL) << 56) |
- ((data & (0xffULL << 8)) << 40) |
- ((data & (0xffULL << 16)) << 24) |
- ((data & (0xffULL << 24)) << 8) |
- ((data & (0xffULL << 32)) >> 8) |
- ((data & (0xffULL << 40)) >> 24) |
- ((data & (0xffULL << 48)) >> 40) |
- ((data & (0xffULL << 56)) >> 56);
-
- return swap;
-}
-
-#define data2host2(pevent, ptr) __data2host2(pevent, *(unsigned short *)(ptr))
-#define data2host4(pevent, ptr) __data2host4(pevent, *(unsigned int *)(ptr))
-#define data2host8(pevent, ptr) \
+#define tep_data2host2(pevent, ptr) __tep_data2host2(pevent, *(unsigned short *)(ptr))
+#define tep_data2host4(pevent, ptr) __tep_data2host4(pevent, *(unsigned int *)(ptr))
+#define tep_data2host8(pevent, ptr) \
({ \
unsigned long long __val; \
\
memcpy(&__val, (ptr), sizeof(unsigned long long)); \
- __data2host8(pevent, __val); \
+ __tep_data2host8(pevent, __val); \
})
static inline int tep_host_bigendian(void)
int tep_pid_is_registered(struct tep_handle *pevent, int pid);
void tep_print_event_task(struct tep_handle *pevent, struct trace_seq *s,
- struct event_format *event,
+ struct tep_event_format *event,
struct tep_record *record);
void tep_print_event_time(struct tep_handle *pevent, struct trace_seq *s,
- struct event_format *event,
+ struct tep_event_format *event,
struct tep_record *record,
bool use_trace_clock);
void tep_print_event_data(struct tep_handle *pevent, struct trace_seq *s,
- struct event_format *event,
+ struct tep_event_format *event,
struct tep_record *record);
void tep_print_event(struct tep_handle *pevent, struct trace_seq *s,
struct tep_record *record, bool use_trace_clock);
enum tep_errno tep_parse_event(struct tep_handle *pevent, const char *buf,
unsigned long size, const char *sys);
enum tep_errno tep_parse_format(struct tep_handle *pevent,
- struct event_format **eventp,
+ struct tep_event_format **eventp,
const char *buf,
unsigned long size, const char *sys);
-void tep_free_format(struct event_format *event);
-void tep_free_format_field(struct format_field *field);
+void tep_free_format(struct tep_event_format *event);
+void tep_free_format_field(struct tep_format_field *field);
-void *tep_get_field_raw(struct trace_seq *s, struct event_format *event,
+void *tep_get_field_raw(struct trace_seq *s, struct tep_event_format *event,
const char *name, struct tep_record *record,
int *len, int err);
-int tep_get_field_val(struct trace_seq *s, struct event_format *event,
+int tep_get_field_val(struct trace_seq *s, struct tep_event_format *event,
const char *name, struct tep_record *record,
unsigned long long *val, int err);
-int tep_get_common_field_val(struct trace_seq *s, struct event_format *event,
+int tep_get_common_field_val(struct trace_seq *s, struct tep_event_format *event,
const char *name, struct tep_record *record,
unsigned long long *val, int err);
-int tep_get_any_field_val(struct trace_seq *s, struct event_format *event,
+int tep_get_any_field_val(struct trace_seq *s, struct tep_event_format *event,
const char *name, struct tep_record *record,
unsigned long long *val, int err);
int tep_print_num_field(struct trace_seq *s, const char *fmt,
- struct event_format *event, const char *name,
- struct tep_record *record, int err);
+ struct tep_event_format *event, const char *name,
+ struct tep_record *record, int err);
int tep_print_func_field(struct trace_seq *s, const char *fmt,
- struct event_format *event, const char *name,
+ struct tep_event_format *event, const char *name,
struct tep_record *record, int err);
int tep_register_event_handler(struct tep_handle *pevent, int id,
int tep_unregister_print_function(struct tep_handle *pevent,
tep_func_handler func, char *name);
-struct format_field *tep_find_common_field(struct event_format *event, const char *name);
-struct format_field *tep_find_field(struct event_format *event, const char *name);
-struct format_field *tep_find_any_field(struct event_format *event, const char *name);
+struct tep_format_field *tep_find_common_field(struct tep_event_format *event, const char *name);
+struct tep_format_field *tep_find_field(struct tep_event_format *event, const char *name);
+struct tep_format_field *tep_find_any_field(struct tep_event_format *event, const char *name);
const char *tep_find_function(struct tep_handle *pevent, unsigned long long addr);
unsigned long long
tep_find_function_address(struct tep_handle *pevent, unsigned long long addr);
unsigned long long tep_read_number(struct tep_handle *pevent, const void *ptr, int size);
-int tep_read_number_field(struct format_field *field, const void *data,
+int tep_read_number_field(struct tep_format_field *field, const void *data,
unsigned long long *value);
-struct event_format *tep_find_event(struct tep_handle *pevent, int id);
+struct tep_event_format *tep_get_first_event(struct tep_handle *tep);
+int tep_get_events_count(struct tep_handle *tep);
+struct tep_event_format *tep_find_event(struct tep_handle *pevent, int id);
-struct event_format *
+struct tep_event_format *
tep_find_event_by_name(struct tep_handle *pevent, const char *sys, const char *name);
-
-struct event_format *
+struct tep_event_format *
tep_find_event_by_record(struct tep_handle *pevent, struct tep_record *record);
void tep_data_lat_fmt(struct tep_handle *pevent,
struct trace_seq *s, struct tep_record *record);
int tep_data_type(struct tep_handle *pevent, struct tep_record *rec);
-struct event_format *tep_data_event_from_type(struct tep_handle *pevent, int type);
+struct tep_event_format *tep_data_event_from_type(struct tep_handle *pevent, int type);
int tep_data_pid(struct tep_handle *pevent, struct tep_record *rec);
int tep_data_preempt_count(struct tep_handle *pevent, struct tep_record *rec);
int tep_data_flags(struct tep_handle *pevent, struct tep_record *rec);
int tep_cmdline_pid(struct tep_handle *pevent, struct cmdline *cmdline);
void tep_print_field(struct trace_seq *s, void *data,
- struct format_field *field);
+ struct tep_format_field *field);
void tep_print_fields(struct trace_seq *s, void *data,
- int size __maybe_unused, struct event_format *event);
-void tep_event_info(struct trace_seq *s, struct event_format *event,
+ int size __maybe_unused, struct tep_event_format *event);
+void tep_event_info(struct trace_seq *s, struct tep_event_format *event,
struct tep_record *record);
int tep_strerror(struct tep_handle *pevent, enum tep_errno errnum,
char *buf, size_t buflen);
-struct event_format **tep_list_events(struct tep_handle *pevent, enum event_sort_type);
-struct format_field **tep_event_common_fields(struct event_format *event);
-struct format_field **tep_event_fields(struct event_format *event);
-
-static inline int tep_get_cpus(struct tep_handle *pevent)
-{
- return pevent->cpus;
-}
-
-static inline void tep_set_cpus(struct tep_handle *pevent, int cpus)
-{
- pevent->cpus = cpus;
-}
-
-static inline int tep_get_long_size(struct tep_handle *pevent)
-{
- return pevent->long_size;
-}
-
-static inline void tep_set_long_size(struct tep_handle *pevent, int long_size)
-{
- pevent->long_size = long_size;
-}
-
-static inline int tep_get_page_size(struct tep_handle *pevent)
-{
- return pevent->page_size;
-}
-
-static inline void tep_set_page_size(struct tep_handle *pevent, int _page_size)
-{
- pevent->page_size = _page_size;
-}
-
-static inline int tep_is_file_bigendian(struct tep_handle *pevent)
-{
- return pevent->file_bigendian;
-}
-
-static inline void tep_set_file_bigendian(struct tep_handle *pevent, int endian)
-{
- pevent->file_bigendian = endian;
-}
-
-static inline int tep_is_host_bigendian(struct tep_handle *pevent)
-{
- return pevent->host_bigendian;
-}
-
-static inline void tep_set_host_bigendian(struct tep_handle *pevent, int endian)
-{
- pevent->host_bigendian = endian;
-}
-
-static inline int tep_is_latency_format(struct tep_handle *pevent)
-{
- return pevent->latency_format;
-}
-
-static inline void tep_set_latency_format(struct tep_handle *pevent, int lat)
-{
- pevent->latency_format = lat;
-}
+struct tep_event_format **tep_list_events(struct tep_handle *pevent, enum tep_event_sort_type);
+struct tep_format_field **tep_event_common_fields(struct tep_event_format *event);
+struct tep_format_field **tep_event_fields(struct tep_event_format *event);
+
+enum tep_endian {
+ TEP_LITTLE_ENDIAN = 0,
+ TEP_BIG_ENDIAN
+};
+int tep_get_cpus(struct tep_handle *pevent);
+void tep_set_cpus(struct tep_handle *pevent, int cpus);
+int tep_get_long_size(struct tep_handle *pevent);
+void tep_set_long_size(struct tep_handle *pevent, int long_size);
+int tep_get_page_size(struct tep_handle *pevent);
+void tep_set_page_size(struct tep_handle *pevent, int _page_size);
+int tep_is_file_bigendian(struct tep_handle *pevent);
+void tep_set_file_bigendian(struct tep_handle *pevent, enum tep_endian endian);
+int tep_is_host_bigendian(struct tep_handle *pevent);
+void tep_set_host_bigendian(struct tep_handle *pevent, enum tep_endian endian);
+int tep_is_latency_format(struct tep_handle *pevent);
+void tep_set_latency_format(struct tep_handle *pevent, int lat);
+int tep_get_header_page_size(struct tep_handle *pevent);
struct tep_handle *tep_alloc(void);
void tep_free(struct tep_handle *pevent);
/* access to the internal parser */
void tep_buffer_init(const char *buf, unsigned long long size);
-enum event_type tep_read_token(char **tok);
+enum tep_event_type tep_read_token(char **tok);
void tep_free_token(char *token);
int tep_peek_char(void);
const char *tep_get_input_buf(void);
/* ----------------------- filtering ----------------------- */
-enum filter_boolean_type {
- FILTER_FALSE,
- FILTER_TRUE,
+enum tep_filter_boolean_type {
+ TEP_FILTER_FALSE,
+ TEP_FILTER_TRUE,
};
-enum filter_op_type {
- FILTER_OP_AND = 1,
- FILTER_OP_OR,
- FILTER_OP_NOT,
+enum tep_filter_op_type {
+ TEP_FILTER_OP_AND = 1,
+ TEP_FILTER_OP_OR,
+ TEP_FILTER_OP_NOT,
};
-enum filter_cmp_type {
- FILTER_CMP_NONE,
- FILTER_CMP_EQ,
- FILTER_CMP_NE,
- FILTER_CMP_GT,
- FILTER_CMP_LT,
- FILTER_CMP_GE,
- FILTER_CMP_LE,
- FILTER_CMP_MATCH,
- FILTER_CMP_NOT_MATCH,
- FILTER_CMP_REGEX,
- FILTER_CMP_NOT_REGEX,
+enum tep_filter_cmp_type {
+ TEP_FILTER_CMP_NONE,
+ TEP_FILTER_CMP_EQ,
+ TEP_FILTER_CMP_NE,
+ TEP_FILTER_CMP_GT,
+ TEP_FILTER_CMP_LT,
+ TEP_FILTER_CMP_GE,
+ TEP_FILTER_CMP_LE,
+ TEP_FILTER_CMP_MATCH,
+ TEP_FILTER_CMP_NOT_MATCH,
+ TEP_FILTER_CMP_REGEX,
+ TEP_FILTER_CMP_NOT_REGEX,
};
-enum filter_exp_type {
- FILTER_EXP_NONE,
- FILTER_EXP_ADD,
- FILTER_EXP_SUB,
- FILTER_EXP_MUL,
- FILTER_EXP_DIV,
- FILTER_EXP_MOD,
- FILTER_EXP_RSHIFT,
- FILTER_EXP_LSHIFT,
- FILTER_EXP_AND,
- FILTER_EXP_OR,
- FILTER_EXP_XOR,
- FILTER_EXP_NOT,
+enum tep_filter_exp_type {
+ TEP_FILTER_EXP_NONE,
+ TEP_FILTER_EXP_ADD,
+ TEP_FILTER_EXP_SUB,
+ TEP_FILTER_EXP_MUL,
+ TEP_FILTER_EXP_DIV,
+ TEP_FILTER_EXP_MOD,
+ TEP_FILTER_EXP_RSHIFT,
+ TEP_FILTER_EXP_LSHIFT,
+ TEP_FILTER_EXP_AND,
+ TEP_FILTER_EXP_OR,
+ TEP_FILTER_EXP_XOR,
+ TEP_FILTER_EXP_NOT,
};
-enum filter_arg_type {
- FILTER_ARG_NONE,
- FILTER_ARG_BOOLEAN,
- FILTER_ARG_VALUE,
- FILTER_ARG_FIELD,
- FILTER_ARG_EXP,
- FILTER_ARG_OP,
- FILTER_ARG_NUM,
- FILTER_ARG_STR,
+enum tep_filter_arg_type {
+ TEP_FILTER_ARG_NONE,
+ TEP_FILTER_ARG_BOOLEAN,
+ TEP_FILTER_ARG_VALUE,
+ TEP_FILTER_ARG_FIELD,
+ TEP_FILTER_ARG_EXP,
+ TEP_FILTER_ARG_OP,
+ TEP_FILTER_ARG_NUM,
+ TEP_FILTER_ARG_STR,
};
-enum filter_value_type {
- FILTER_NUMBER,
- FILTER_STRING,
- FILTER_CHAR
+enum tep_filter_value_type {
+ TEP_FILTER_NUMBER,
+ TEP_FILTER_STRING,
+ TEP_FILTER_CHAR
};
-struct fliter_arg;
+struct tep_filter_arg;
-struct filter_arg_boolean {
- enum filter_boolean_type value;
+struct tep_filter_arg_boolean {
+ enum tep_filter_boolean_type value;
};
-struct filter_arg_field {
- struct format_field *field;
+struct tep_filter_arg_field {
+ struct tep_format_field *field;
};
-struct filter_arg_value {
- enum filter_value_type type;
+struct tep_filter_arg_value {
+ enum tep_filter_value_type type;
union {
char *str;
unsigned long long val;
};
};
-struct filter_arg_op {
- enum filter_op_type type;
- struct filter_arg *left;
- struct filter_arg *right;
+struct tep_filter_arg_op {
+ enum tep_filter_op_type type;
+ struct tep_filter_arg *left;
+ struct tep_filter_arg *right;
};
-struct filter_arg_exp {
- enum filter_exp_type type;
- struct filter_arg *left;
- struct filter_arg *right;
+struct tep_filter_arg_exp {
+ enum tep_filter_exp_type type;
+ struct tep_filter_arg *left;
+ struct tep_filter_arg *right;
};
-struct filter_arg_num {
- enum filter_cmp_type type;
- struct filter_arg *left;
- struct filter_arg *right;
+struct tep_filter_arg_num {
+ enum tep_filter_cmp_type type;
+ struct tep_filter_arg *left;
+ struct tep_filter_arg *right;
};
-struct filter_arg_str {
- enum filter_cmp_type type;
- struct format_field *field;
- char *val;
- char *buffer;
- regex_t reg;
+struct tep_filter_arg_str {
+ enum tep_filter_cmp_type type;
+ struct tep_format_field *field;
+ char *val;
+ char *buffer;
+ regex_t reg;
};
-struct filter_arg {
- enum filter_arg_type type;
+struct tep_filter_arg {
+ enum tep_filter_arg_type type;
union {
- struct filter_arg_boolean boolean;
- struct filter_arg_field field;
- struct filter_arg_value value;
- struct filter_arg_op op;
- struct filter_arg_exp exp;
- struct filter_arg_num num;
- struct filter_arg_str str;
+ struct tep_filter_arg_boolean boolean;
+ struct tep_filter_arg_field field;
+ struct tep_filter_arg_value value;
+ struct tep_filter_arg_op op;
+ struct tep_filter_arg_exp exp;
+ struct tep_filter_arg_num num;
+ struct tep_filter_arg_str str;
};
};
-struct filter_type {
+struct tep_filter_type {
int event_id;
- struct event_format *event;
- struct filter_arg *filter;
+ struct tep_event_format *event;
+ struct tep_filter_arg *filter;
};
#define TEP_FILTER_ERROR_BUFSZ 1024
-struct event_filter {
+struct tep_event_filter {
struct tep_handle *pevent;
int filters;
- struct filter_type *event_filters;
+ struct tep_filter_type *event_filters;
char error_buffer[TEP_FILTER_ERROR_BUFSZ];
};
-struct event_filter *tep_filter_alloc(struct tep_handle *pevent);
+struct tep_event_filter *tep_filter_alloc(struct tep_handle *pevent);
/* for backward compatibility */
#define FILTER_NONE TEP_ERRNO__NO_FILTER
#define FILTER_MISS TEP_ERRNO__FILTER_MISS
#define FILTER_MATCH TEP_ERRNO__FILTER_MATCH
-enum filter_trivial_type {
- FILTER_TRIVIAL_FALSE,
- FILTER_TRIVIAL_TRUE,
- FILTER_TRIVIAL_BOTH,
+enum tep_filter_trivial_type {
+ TEP_FILTER_TRIVIAL_FALSE,
+ TEP_FILTER_TRIVIAL_TRUE,
+ TEP_FILTER_TRIVIAL_BOTH,
};
-enum tep_errno tep_filter_add_filter_str(struct event_filter *filter,
+enum tep_errno tep_filter_add_filter_str(struct tep_event_filter *filter,
const char *filter_str);
-enum tep_errno tep_filter_match(struct event_filter *filter,
+enum tep_errno tep_filter_match(struct tep_event_filter *filter,
struct tep_record *record);
-int tep_filter_strerror(struct event_filter *filter, enum tep_errno err,
+int tep_filter_strerror(struct tep_event_filter *filter, enum tep_errno err,
char *buf, size_t buflen);
-int tep_event_filtered(struct event_filter *filter,
+int tep_event_filtered(struct tep_event_filter *filter,
int event_id);
-void tep_filter_reset(struct event_filter *filter);
+void tep_filter_reset(struct tep_event_filter *filter);
-int tep_filter_clear_trivial(struct event_filter *filter,
- enum filter_trivial_type type);
+int tep_filter_clear_trivial(struct tep_event_filter *filter,
+ enum tep_filter_trivial_type type);
-void tep_filter_free(struct event_filter *filter);
+void tep_filter_free(struct tep_event_filter *filter);
-char *tep_filter_make_string(struct event_filter *filter, int event_id);
+char *tep_filter_make_string(struct tep_event_filter *filter, int event_id);
-int tep_filter_remove_event(struct event_filter *filter,
+int tep_filter_remove_event(struct tep_event_filter *filter,
int event_id);
-int tep_filter_event_has_trivial(struct event_filter *filter,
+int tep_filter_event_has_trivial(struct tep_event_filter *filter,
int event_id,
- enum filter_trivial_type type);
+ enum tep_filter_trivial_type type);
-int tep_filter_copy(struct event_filter *dest, struct event_filter *source);
+int tep_filter_copy(struct tep_event_filter *dest, struct tep_event_filter *source);
-int tep_update_trivial(struct event_filter *dest, struct event_filter *source,
- enum filter_trivial_type type);
+int tep_update_trivial(struct tep_event_filter *dest, struct tep_event_filter *source,
+ enum tep_filter_trivial_type type);
-int tep_filter_compare(struct event_filter *filter1, struct event_filter *filter2);
+int tep_filter_compare(struct tep_event_filter *filter1, struct tep_event_filter *filter2);
#endif /* _PARSE_EVENTS_H */
#include <unistd.h>
#include <dirent.h>
#include "event-parse.h"
+#include "event-parse-local.h"
#include "event-utils.h"
+#include "trace-seq.h"
#define LOCAL_PLUGIN_DIR ".traceevent/plugins"
char *value;
} *trace_plugin_options;
-struct plugin_list {
- struct plugin_list *next;
+struct tep_plugin_list {
+ struct tep_plugin_list *next;
char *name;
void *handle;
};
*/
void tep_print_plugins(struct trace_seq *s,
const char *prefix, const char *suffix,
- const struct plugin_list *list)
+ const struct tep_plugin_list *list)
{
while (list) {
trace_seq_printf(s, "%s%s%s", prefix, list->name, suffix);
load_plugin(struct tep_handle *pevent, const char *path,
const char *file, void *data)
{
- struct plugin_list **plugin_list = data;
+ struct tep_plugin_list **plugin_list = data;
tep_plugin_load_func func;
- struct plugin_list *list;
+ struct tep_plugin_list *list;
const char *alias;
char *plugin;
void *handle;
free(path);
}
-struct plugin_list*
+struct tep_plugin_list*
tep_load_plugins(struct tep_handle *pevent)
{
- struct plugin_list *list = NULL;
+ struct tep_plugin_list *list = NULL;
load_plugins(pevent, ".so", load_plugin, &list);
return list;
}
void
-tep_unload_plugins(struct plugin_list *plugin_list, struct tep_handle *pevent)
+tep_unload_plugins(struct tep_plugin_list *plugin_list, struct tep_handle *pevent)
{
tep_plugin_unload_func func;
- struct plugin_list *list;
+ struct tep_plugin_list *list;
while (plugin_list) {
list = plugin_list;
#include <sys/types.h>
#include "event-parse.h"
+#include "event-parse-local.h"
#include "event-utils.h"
#define COMM "COMM"
#define CPU "CPU"
-static struct format_field comm = {
+static struct tep_format_field comm = {
.name = "COMM",
};
-static struct format_field cpu = {
+static struct tep_format_field cpu = {
.name = "CPU",
};
struct event_list {
struct event_list *next;
- struct event_format *event;
+ struct tep_event_format *event;
};
static void show_error(char *error_buf, const char *fmt, ...)
tep_free_token(token);
}
-static enum event_type read_token(char **tok)
+static enum tep_event_type read_token(char **tok)
{
- enum event_type type;
+ enum tep_event_type type;
char *token = NULL;
do {
free_token(token);
type = tep_read_token(&token);
- } while (type == EVENT_NEWLINE || type == EVENT_SPACE);
+ } while (type == TEP_EVENT_NEWLINE || type == TEP_EVENT_SPACE);
/* If token is = or ! check to see if the next char is ~ */
if (token &&
*tok = malloc(3);
if (*tok == NULL) {
free_token(token);
- return EVENT_ERROR;
+ return TEP_EVENT_ERROR;
}
sprintf(*tok, "%c%c", *token, '~');
free_token(token);
static int filter_cmp(const void *a, const void *b)
{
- const struct filter_type *ea = a;
- const struct filter_type *eb = b;
+ const struct tep_filter_type *ea = a;
+ const struct tep_filter_type *eb = b;
if (ea->event_id < eb->event_id)
return -1;
return 0;
}
-static struct filter_type *
-find_filter_type(struct event_filter *filter, int id)
+static struct tep_filter_type *
+find_filter_type(struct tep_event_filter *filter, int id)
{
- struct filter_type *filter_type;
- struct filter_type key;
+ struct tep_filter_type *filter_type;
+ struct tep_filter_type key;
key.event_id = id;
return filter_type;
}
-static struct filter_type *
-add_filter_type(struct event_filter *filter, int id)
+static struct tep_filter_type *
+add_filter_type(struct tep_event_filter *filter, int id)
{
- struct filter_type *filter_type;
+ struct tep_filter_type *filter_type;
int i;
filter_type = find_filter_type(filter, id);
* tep_filter_alloc - create a new event filter
* @pevent: The pevent that this filter is associated with
*/
-struct event_filter *tep_filter_alloc(struct tep_handle *pevent)
+struct tep_event_filter *tep_filter_alloc(struct tep_handle *pevent)
{
- struct event_filter *filter;
+ struct tep_event_filter *filter;
filter = malloc(sizeof(*filter));
if (filter == NULL)
return filter;
}
-static struct filter_arg *allocate_arg(void)
+static struct tep_filter_arg *allocate_arg(void)
{
- return calloc(1, sizeof(struct filter_arg));
+ return calloc(1, sizeof(struct tep_filter_arg));
}
-static void free_arg(struct filter_arg *arg)
+static void free_arg(struct tep_filter_arg *arg)
{
if (!arg)
return;
switch (arg->type) {
- case FILTER_ARG_NONE:
- case FILTER_ARG_BOOLEAN:
+ case TEP_FILTER_ARG_NONE:
+ case TEP_FILTER_ARG_BOOLEAN:
break;
- case FILTER_ARG_NUM:
+ case TEP_FILTER_ARG_NUM:
free_arg(arg->num.left);
free_arg(arg->num.right);
break;
- case FILTER_ARG_EXP:
+ case TEP_FILTER_ARG_EXP:
free_arg(arg->exp.left);
free_arg(arg->exp.right);
break;
- case FILTER_ARG_STR:
+ case TEP_FILTER_ARG_STR:
free(arg->str.val);
regfree(&arg->str.reg);
free(arg->str.buffer);
break;
- case FILTER_ARG_VALUE:
- if (arg->value.type == FILTER_STRING ||
- arg->value.type == FILTER_CHAR)
+ case TEP_FILTER_ARG_VALUE:
+ if (arg->value.type == TEP_FILTER_STRING ||
+ arg->value.type == TEP_FILTER_CHAR)
free(arg->value.str);
break;
- case FILTER_ARG_OP:
+ case TEP_FILTER_ARG_OP:
free_arg(arg->op.left);
free_arg(arg->op.right);
default:
}
static int add_event(struct event_list **events,
- struct event_format *event)
+ struct tep_event_format *event)
{
struct event_list *list;
return 0;
}
-static int event_match(struct event_format *event,
+static int event_match(struct tep_event_format *event,
regex_t *sreg, regex_t *ereg)
{
if (sreg) {
find_event(struct tep_handle *pevent, struct event_list **events,
char *sys_name, char *event_name)
{
- struct event_format *event;
+ struct tep_event_format *event;
regex_t ereg;
regex_t sreg;
int match = 0;
}
static enum tep_errno
-create_arg_item(struct event_format *event, const char *token,
- enum event_type type, struct filter_arg **parg, char *error_str)
+create_arg_item(struct tep_event_format *event, const char *token,
+ enum tep_event_type type, struct tep_filter_arg **parg, char *error_str)
{
- struct format_field *field;
- struct filter_arg *arg;
+ struct tep_format_field *field;
+ struct tep_filter_arg *arg;
arg = allocate_arg();
if (arg == NULL) {
switch (type) {
- case EVENT_SQUOTE:
- case EVENT_DQUOTE:
- arg->type = FILTER_ARG_VALUE;
+ case TEP_EVENT_SQUOTE:
+ case TEP_EVENT_DQUOTE:
+ arg->type = TEP_FILTER_ARG_VALUE;
arg->value.type =
- type == EVENT_DQUOTE ? FILTER_STRING : FILTER_CHAR;
+ type == TEP_EVENT_DQUOTE ? TEP_FILTER_STRING : TEP_FILTER_CHAR;
arg->value.str = strdup(token);
if (!arg->value.str) {
free_arg(arg);
return TEP_ERRNO__MEM_ALLOC_FAILED;
}
break;
- case EVENT_ITEM:
+ case TEP_EVENT_ITEM:
/* if it is a number, then convert it */
if (isdigit(token[0])) {
- arg->type = FILTER_ARG_VALUE;
- arg->value.type = FILTER_NUMBER;
+ arg->type = TEP_FILTER_ARG_VALUE;
+ arg->value.type = TEP_FILTER_NUMBER;
arg->value.val = strtoull(token, NULL, 0);
break;
}
field = &cpu;
} else {
/* not a field, Make it false */
- arg->type = FILTER_ARG_BOOLEAN;
- arg->boolean.value = FILTER_FALSE;
+ arg->type = TEP_FILTER_ARG_BOOLEAN;
+ arg->boolean.value = TEP_FILTER_FALSE;
break;
}
}
- arg->type = FILTER_ARG_FIELD;
+ arg->type = TEP_FILTER_ARG_FIELD;
arg->field.field = field;
break;
default:
return 0;
}
-static struct filter_arg *
-create_arg_op(enum filter_op_type btype)
+static struct tep_filter_arg *
+create_arg_op(enum tep_filter_op_type btype)
{
- struct filter_arg *arg;
+ struct tep_filter_arg *arg;
arg = allocate_arg();
if (!arg)
return NULL;
- arg->type = FILTER_ARG_OP;
+ arg->type = TEP_FILTER_ARG_OP;
arg->op.type = btype;
return arg;
}
-static struct filter_arg *
-create_arg_exp(enum filter_exp_type etype)
+static struct tep_filter_arg *
+create_arg_exp(enum tep_filter_exp_type etype)
{
- struct filter_arg *arg;
+ struct tep_filter_arg *arg;
arg = allocate_arg();
if (!arg)
return NULL;
- arg->type = FILTER_ARG_EXP;
+ arg->type = TEP_FILTER_ARG_EXP;
arg->exp.type = etype;
return arg;
}
-static struct filter_arg *
-create_arg_cmp(enum filter_cmp_type ctype)
+static struct tep_filter_arg *
+create_arg_cmp(enum tep_filter_cmp_type ctype)
{
- struct filter_arg *arg;
+ struct tep_filter_arg *arg;
arg = allocate_arg();
if (!arg)
return NULL;
/* Use NUM and change if necessary */
- arg->type = FILTER_ARG_NUM;
+ arg->type = TEP_FILTER_ARG_NUM;
arg->num.type = ctype;
return arg;
}
static enum tep_errno
-add_right(struct filter_arg *op, struct filter_arg *arg, char *error_str)
+add_right(struct tep_filter_arg *op, struct tep_filter_arg *arg, char *error_str)
{
- struct filter_arg *left;
+ struct tep_filter_arg *left;
char *str;
int op_type;
int ret;
switch (op->type) {
- case FILTER_ARG_EXP:
+ case TEP_FILTER_ARG_EXP:
if (op->exp.right)
goto out_fail;
op->exp.right = arg;
break;
- case FILTER_ARG_OP:
+ case TEP_FILTER_ARG_OP:
if (op->op.right)
goto out_fail;
op->op.right = arg;
break;
- case FILTER_ARG_NUM:
+ case TEP_FILTER_ARG_NUM:
if (op->op.right)
goto out_fail;
/*
* The arg must be num, str, or field
*/
switch (arg->type) {
- case FILTER_ARG_VALUE:
- case FILTER_ARG_FIELD:
+ case TEP_FILTER_ARG_VALUE:
+ case TEP_FILTER_ARG_FIELD:
break;
default:
show_error(error_str, "Illegal rvalue");
* convert this to a string or regex.
*/
switch (arg->value.type) {
- case FILTER_CHAR:
+ case TEP_FILTER_CHAR:
/*
* A char should be converted to number if
* the string is 1 byte, and the compare
* is not a REGEX.
*/
if (strlen(arg->value.str) == 1 &&
- op->num.type != FILTER_CMP_REGEX &&
- op->num.type != FILTER_CMP_NOT_REGEX) {
- arg->value.type = FILTER_NUMBER;
+ op->num.type != TEP_FILTER_CMP_REGEX &&
+ op->num.type != TEP_FILTER_CMP_NOT_REGEX) {
+ arg->value.type = TEP_FILTER_NUMBER;
goto do_int;
}
/* fall through */
- case FILTER_STRING:
+ case TEP_FILTER_STRING:
/* convert op to a string arg */
op_type = op->num.type;
* If left arg was a field not found then
* NULL the entire op.
*/
- if (left->type == FILTER_ARG_BOOLEAN) {
+ if (left->type == TEP_FILTER_ARG_BOOLEAN) {
free_arg(left);
free_arg(arg);
- op->type = FILTER_ARG_BOOLEAN;
- op->boolean.value = FILTER_FALSE;
+ op->type = TEP_FILTER_ARG_BOOLEAN;
+ op->boolean.value = TEP_FILTER_FALSE;
break;
}
/* Left arg must be a field */
- if (left->type != FILTER_ARG_FIELD) {
+ if (left->type != TEP_FILTER_ARG_FIELD) {
show_error(error_str,
"Illegal lvalue for string comparison");
return TEP_ERRNO__ILLEGAL_LVALUE;
/* Make sure this is a valid string compare */
switch (op_type) {
- case FILTER_CMP_EQ:
- op_type = FILTER_CMP_MATCH;
+ case TEP_FILTER_CMP_EQ:
+ op_type = TEP_FILTER_CMP_MATCH;
break;
- case FILTER_CMP_NE:
- op_type = FILTER_CMP_NOT_MATCH;
+ case TEP_FILTER_CMP_NE:
+ op_type = TEP_FILTER_CMP_NOT_MATCH;
break;
- case FILTER_CMP_REGEX:
- case FILTER_CMP_NOT_REGEX:
+ case TEP_FILTER_CMP_REGEX:
+ case TEP_FILTER_CMP_NOT_REGEX:
ret = regcomp(&op->str.reg, str, REG_ICASE|REG_NOSUB);
if (ret) {
show_error(error_str,
return TEP_ERRNO__ILLEGAL_STRING_CMP;
}
- op->type = FILTER_ARG_STR;
+ op->type = TEP_FILTER_ARG_STR;
op->str.type = op_type;
op->str.field = left->field.field;
op->str.val = strdup(str);
break;
- case FILTER_NUMBER:
+ case TEP_FILTER_NUMBER:
do_int:
switch (op->num.type) {
- case FILTER_CMP_REGEX:
- case FILTER_CMP_NOT_REGEX:
+ case TEP_FILTER_CMP_REGEX:
+ case TEP_FILTER_CMP_NOT_REGEX:
show_error(error_str,
"Op not allowed with integers");
return TEP_ERRNO__ILLEGAL_INTEGER_CMP;
return TEP_ERRNO__SYNTAX_ERROR;
}
-static struct filter_arg *
-rotate_op_right(struct filter_arg *a, struct filter_arg *b)
+static struct tep_filter_arg *
+rotate_op_right(struct tep_filter_arg *a, struct tep_filter_arg *b)
{
- struct filter_arg *arg;
+ struct tep_filter_arg *arg;
arg = a->op.right;
a->op.right = b;
return arg;
}
-static enum tep_errno add_left(struct filter_arg *op, struct filter_arg *arg)
+static enum tep_errno add_left(struct tep_filter_arg *op, struct tep_filter_arg *arg)
{
switch (op->type) {
- case FILTER_ARG_EXP:
- if (arg->type == FILTER_ARG_OP)
+ case TEP_FILTER_ARG_EXP:
+ if (arg->type == TEP_FILTER_ARG_OP)
arg = rotate_op_right(arg, op);
op->exp.left = arg;
break;
- case FILTER_ARG_OP:
+ case TEP_FILTER_ARG_OP:
op->op.left = arg;
break;
- case FILTER_ARG_NUM:
- if (arg->type == FILTER_ARG_OP)
+ case TEP_FILTER_ARG_NUM:
+ if (arg->type == TEP_FILTER_ARG_OP)
arg = rotate_op_right(arg, op);
/* left arg of compares must be a field */
- if (arg->type != FILTER_ARG_FIELD &&
- arg->type != FILTER_ARG_BOOLEAN)
+ if (arg->type != TEP_FILTER_ARG_FIELD &&
+ arg->type != TEP_FILTER_ARG_BOOLEAN)
return TEP_ERRNO__INVALID_ARG_TYPE;
op->num.left = arg;
break;
};
static enum op_type process_op(const char *token,
- enum filter_op_type *btype,
- enum filter_cmp_type *ctype,
- enum filter_exp_type *etype)
+ enum tep_filter_op_type *btype,
+ enum tep_filter_cmp_type *ctype,
+ enum tep_filter_exp_type *etype)
{
- *btype = FILTER_OP_NOT;
- *etype = FILTER_EXP_NONE;
- *ctype = FILTER_CMP_NONE;
+ *btype = TEP_FILTER_OP_NOT;
+ *etype = TEP_FILTER_EXP_NONE;
+ *ctype = TEP_FILTER_CMP_NONE;
if (strcmp(token, "&&") == 0)
- *btype = FILTER_OP_AND;
+ *btype = TEP_FILTER_OP_AND;
else if (strcmp(token, "||") == 0)
- *btype = FILTER_OP_OR;
+ *btype = TEP_FILTER_OP_OR;
else if (strcmp(token, "!") == 0)
return OP_NOT;
- if (*btype != FILTER_OP_NOT)
+ if (*btype != TEP_FILTER_OP_NOT)
return OP_BOOL;
/* Check for value expressions */
if (strcmp(token, "+") == 0) {
- *etype = FILTER_EXP_ADD;
+ *etype = TEP_FILTER_EXP_ADD;
} else if (strcmp(token, "-") == 0) {
- *etype = FILTER_EXP_SUB;
+ *etype = TEP_FILTER_EXP_SUB;
} else if (strcmp(token, "*") == 0) {
- *etype = FILTER_EXP_MUL;
+ *etype = TEP_FILTER_EXP_MUL;
} else if (strcmp(token, "/") == 0) {
- *etype = FILTER_EXP_DIV;
+ *etype = TEP_FILTER_EXP_DIV;
} else if (strcmp(token, "%") == 0) {
- *etype = FILTER_EXP_MOD;
+ *etype = TEP_FILTER_EXP_MOD;
} else if (strcmp(token, ">>") == 0) {
- *etype = FILTER_EXP_RSHIFT;
+ *etype = TEP_FILTER_EXP_RSHIFT;
} else if (strcmp(token, "<<") == 0) {
- *etype = FILTER_EXP_LSHIFT;
+ *etype = TEP_FILTER_EXP_LSHIFT;
} else if (strcmp(token, "&") == 0) {
- *etype = FILTER_EXP_AND;
+ *etype = TEP_FILTER_EXP_AND;
} else if (strcmp(token, "|") == 0) {
- *etype = FILTER_EXP_OR;
+ *etype = TEP_FILTER_EXP_OR;
} else if (strcmp(token, "^") == 0) {
- *etype = FILTER_EXP_XOR;
+ *etype = TEP_FILTER_EXP_XOR;
} else if (strcmp(token, "~") == 0)
- *etype = FILTER_EXP_NOT;
+ *etype = TEP_FILTER_EXP_NOT;
- if (*etype != FILTER_EXP_NONE)
+ if (*etype != TEP_FILTER_EXP_NONE)
return OP_EXP;
/* Check for compares */
if (strcmp(token, "==") == 0)
- *ctype = FILTER_CMP_EQ;
+ *ctype = TEP_FILTER_CMP_EQ;
else if (strcmp(token, "!=") == 0)
- *ctype = FILTER_CMP_NE;
+ *ctype = TEP_FILTER_CMP_NE;
else if (strcmp(token, "<") == 0)
- *ctype = FILTER_CMP_LT;
+ *ctype = TEP_FILTER_CMP_LT;
else if (strcmp(token, ">") == 0)
- *ctype = FILTER_CMP_GT;
+ *ctype = TEP_FILTER_CMP_GT;
else if (strcmp(token, "<=") == 0)
- *ctype = FILTER_CMP_LE;
+ *ctype = TEP_FILTER_CMP_LE;
else if (strcmp(token, ">=") == 0)
- *ctype = FILTER_CMP_GE;
+ *ctype = TEP_FILTER_CMP_GE;
else if (strcmp(token, "=~") == 0)
- *ctype = FILTER_CMP_REGEX;
+ *ctype = TEP_FILTER_CMP_REGEX;
else if (strcmp(token, "!~") == 0)
- *ctype = FILTER_CMP_NOT_REGEX;
+ *ctype = TEP_FILTER_CMP_NOT_REGEX;
else
return OP_NONE;
return OP_CMP;
}
-static int check_op_done(struct filter_arg *arg)
+static int check_op_done(struct tep_filter_arg *arg)
{
switch (arg->type) {
- case FILTER_ARG_EXP:
+ case TEP_FILTER_ARG_EXP:
return arg->exp.right != NULL;
- case FILTER_ARG_OP:
+ case TEP_FILTER_ARG_OP:
return arg->op.right != NULL;
- case FILTER_ARG_NUM:
+ case TEP_FILTER_ARG_NUM:
return arg->num.right != NULL;
- case FILTER_ARG_STR:
+ case TEP_FILTER_ARG_STR:
/* A string conversion is always done */
return 1;
- case FILTER_ARG_BOOLEAN:
+ case TEP_FILTER_ARG_BOOLEAN:
/* field not found, is ok */
return 1;
};
static enum tep_errno
-reparent_op_arg(struct filter_arg *parent, struct filter_arg *old_child,
- struct filter_arg *arg, char *error_str)
+reparent_op_arg(struct tep_filter_arg *parent, struct tep_filter_arg *old_child,
+ struct tep_filter_arg *arg, char *error_str)
{
- struct filter_arg *other_child;
- struct filter_arg **ptr;
+ struct tep_filter_arg *other_child;
+ struct tep_filter_arg **ptr;
- if (parent->type != FILTER_ARG_OP &&
- arg->type != FILTER_ARG_OP) {
+ if (parent->type != TEP_FILTER_ARG_OP &&
+ arg->type != TEP_FILTER_ARG_OP) {
show_error(error_str, "can not reparent other than OP");
return TEP_ERRNO__REPARENT_NOT_OP;
}
}
/* Returns either filter_vals (success) or tep_errno (failfure) */
-static int test_arg(struct filter_arg *parent, struct filter_arg *arg,
+static int test_arg(struct tep_filter_arg *parent, struct tep_filter_arg *arg,
char *error_str)
{
int lval, rval;
switch (arg->type) {
/* bad case */
- case FILTER_ARG_BOOLEAN:
+ case TEP_FILTER_ARG_BOOLEAN:
return FILTER_VAL_FALSE + arg->boolean.value;
/* good cases: */
- case FILTER_ARG_STR:
- case FILTER_ARG_VALUE:
- case FILTER_ARG_FIELD:
+ case TEP_FILTER_ARG_STR:
+ case TEP_FILTER_ARG_VALUE:
+ case TEP_FILTER_ARG_FIELD:
return FILTER_VAL_NORM;
- case FILTER_ARG_EXP:
+ case TEP_FILTER_ARG_EXP:
lval = test_arg(arg, arg->exp.left, error_str);
if (lval != FILTER_VAL_NORM)
return lval;
return rval;
return FILTER_VAL_NORM;
- case FILTER_ARG_NUM:
+ case TEP_FILTER_ARG_NUM:
lval = test_arg(arg, arg->num.left, error_str);
if (lval != FILTER_VAL_NORM)
return lval;
return rval;
return FILTER_VAL_NORM;
- case FILTER_ARG_OP:
- if (arg->op.type != FILTER_OP_NOT) {
+ case TEP_FILTER_ARG_OP:
+ if (arg->op.type != TEP_FILTER_OP_NOT) {
lval = test_arg(arg, arg->op.left, error_str);
switch (lval) {
case FILTER_VAL_NORM:
break;
case FILTER_VAL_TRUE:
- if (arg->op.type == FILTER_OP_OR)
+ if (arg->op.type == TEP_FILTER_OP_OR)
return FILTER_VAL_TRUE;
rval = test_arg(arg, arg->op.right, error_str);
if (rval != FILTER_VAL_NORM)
error_str);
case FILTER_VAL_FALSE:
- if (arg->op.type == FILTER_OP_AND)
+ if (arg->op.type == TEP_FILTER_OP_AND)
return FILTER_VAL_FALSE;
rval = test_arg(arg, arg->op.right, error_str);
if (rval != FILTER_VAL_NORM)
break;
case FILTER_VAL_TRUE:
- if (arg->op.type == FILTER_OP_OR)
+ if (arg->op.type == TEP_FILTER_OP_OR)
return FILTER_VAL_TRUE;
- if (arg->op.type == FILTER_OP_NOT)
+ if (arg->op.type == TEP_FILTER_OP_NOT)
return FILTER_VAL_FALSE;
return reparent_op_arg(parent, arg, arg->op.left,
error_str);
case FILTER_VAL_FALSE:
- if (arg->op.type == FILTER_OP_AND)
+ if (arg->op.type == TEP_FILTER_OP_AND)
return FILTER_VAL_FALSE;
- if (arg->op.type == FILTER_OP_NOT)
+ if (arg->op.type == TEP_FILTER_OP_NOT)
return FILTER_VAL_TRUE;
return reparent_op_arg(parent, arg, arg->op.left,
}
/* Remove any unknown event fields */
-static int collapse_tree(struct filter_arg *arg,
- struct filter_arg **arg_collapsed, char *error_str)
+static int collapse_tree(struct tep_filter_arg *arg,
+ struct tep_filter_arg **arg_collapsed, char *error_str)
{
int ret;
free_arg(arg);
arg = allocate_arg();
if (arg) {
- arg->type = FILTER_ARG_BOOLEAN;
+ arg->type = TEP_FILTER_ARG_BOOLEAN;
arg->boolean.value = ret == FILTER_VAL_TRUE;
} else {
show_error(error_str, "Failed to allocate filter arg");
}
static enum tep_errno
-process_filter(struct event_format *event, struct filter_arg **parg,
+process_filter(struct tep_event_format *event, struct tep_filter_arg **parg,
char *error_str, int not)
{
- enum event_type type;
+ enum tep_event_type type;
char *token = NULL;
- struct filter_arg *current_op = NULL;
- struct filter_arg *current_exp = NULL;
- struct filter_arg *left_item = NULL;
- struct filter_arg *arg = NULL;
+ struct tep_filter_arg *current_op = NULL;
+ struct tep_filter_arg *current_exp = NULL;
+ struct tep_filter_arg *left_item = NULL;
+ struct tep_filter_arg *arg = NULL;
enum op_type op_type;
- enum filter_op_type btype;
- enum filter_exp_type etype;
- enum filter_cmp_type ctype;
+ enum tep_filter_op_type btype;
+ enum tep_filter_exp_type etype;
+ enum tep_filter_cmp_type ctype;
enum tep_errno ret;
*parg = NULL;
free(token);
type = read_token(&token);
switch (type) {
- case EVENT_SQUOTE:
- case EVENT_DQUOTE:
- case EVENT_ITEM:
+ case TEP_EVENT_SQUOTE:
+ case TEP_EVENT_DQUOTE:
+ case TEP_EVENT_ITEM:
ret = create_arg_item(event, token, type, &arg, error_str);
if (ret < 0)
goto fail;
arg = NULL;
break;
- case EVENT_DELIM:
+ case TEP_EVENT_DELIM:
if (*token == ',') {
show_error(error_str, "Illegal token ','");
ret = TEP_ERRNO__ILLEGAL_TOKEN;
}
break;
- case EVENT_OP:
+ case TEP_EVENT_OP:
op_type = process_op(token, &btype, &ctype, &etype);
/* All expect a left arg except for NOT */
if (ret < 0)
goto fail_syntax;
break;
- case EVENT_NONE:
+ case TEP_EVENT_NONE:
break;
- case EVENT_ERROR:
+ case TEP_EVENT_ERROR:
goto fail_alloc;
default:
goto fail_syntax;
}
- } while (type != EVENT_NONE);
+ } while (type != TEP_EVENT_NONE);
if (!current_op && !current_exp)
goto fail_syntax;
}
static enum tep_errno
-process_event(struct event_format *event, const char *filter_str,
- struct filter_arg **parg, char *error_str)
+process_event(struct tep_event_format *event, const char *filter_str,
+ struct tep_filter_arg **parg, char *error_str)
{
int ret;
if (*parg == NULL)
return TEP_ERRNO__MEM_ALLOC_FAILED;
- (*parg)->type = FILTER_ARG_BOOLEAN;
- (*parg)->boolean.value = FILTER_FALSE;
+ (*parg)->type = TEP_FILTER_ARG_BOOLEAN;
+ (*parg)->boolean.value = TEP_FILTER_FALSE;
}
return 0;
}
static enum tep_errno
-filter_event(struct event_filter *filter, struct event_format *event,
+filter_event(struct tep_event_filter *filter, struct tep_event_format *event,
const char *filter_str, char *error_str)
{
- struct filter_type *filter_type;
- struct filter_arg *arg;
+ struct tep_filter_type *filter_type;
+ struct tep_filter_arg *arg;
enum tep_errno ret;
if (filter_str) {
if (arg == NULL)
return TEP_ERRNO__MEM_ALLOC_FAILED;
- arg->type = FILTER_ARG_BOOLEAN;
- arg->boolean.value = FILTER_TRUE;
+ arg->type = TEP_FILTER_ARG_BOOLEAN;
+ arg->boolean.value = TEP_FILTER_TRUE;
}
filter_type = add_filter_type(filter, event->id);
return 0;
}
-static void filter_init_error_buf(struct event_filter *filter)
+static void filter_init_error_buf(struct tep_event_filter *filter)
{
/* clear buffer to reset show error */
tep_buffer_init("", 0);
* negative error code. Use tep_filter_strerror() to see
* actual error message in case of error.
*/
-enum tep_errno tep_filter_add_filter_str(struct event_filter *filter,
+enum tep_errno tep_filter_add_filter_str(struct tep_event_filter *filter,
const char *filter_str)
{
struct tep_handle *pevent = filter->pevent;
return rtn;
}
-static void free_filter_type(struct filter_type *filter_type)
+static void free_filter_type(struct tep_filter_type *filter_type)
{
free_arg(filter_type->filter);
}
*
* Returns 0 if message was filled successfully, -1 if error
*/
-int tep_filter_strerror(struct event_filter *filter, enum tep_errno err,
+int tep_filter_strerror(struct tep_event_filter *filter, enum tep_errno err,
char *buf, size_t buflen)
{
if (err <= __TEP_ERRNO__START || err >= __TEP_ERRNO__END)
* Returns 1: if an event was removed
* 0: if the event was not found
*/
-int tep_filter_remove_event(struct event_filter *filter,
+int tep_filter_remove_event(struct tep_event_filter *filter,
int event_id)
{
- struct filter_type *filter_type;
+ struct tep_filter_type *filter_type;
unsigned long len;
if (!filter->filters)
*
* Removes all filters from a filter and resets it.
*/
-void tep_filter_reset(struct event_filter *filter)
+void tep_filter_reset(struct tep_event_filter *filter)
{
int i;
filter->event_filters = NULL;
}
-void tep_filter_free(struct event_filter *filter)
+void tep_filter_free(struct tep_event_filter *filter)
{
tep_unref(filter->pevent);
free(filter);
}
-static char *arg_to_str(struct event_filter *filter, struct filter_arg *arg);
+static char *arg_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg);
-static int copy_filter_type(struct event_filter *filter,
- struct event_filter *source,
- struct filter_type *filter_type)
+static int copy_filter_type(struct tep_event_filter *filter,
+ struct tep_event_filter *source,
+ struct tep_filter_type *filter_type)
{
- struct filter_arg *arg;
- struct event_format *event;
+ struct tep_filter_arg *arg;
+ struct tep_event_format *event;
const char *sys;
const char *name;
char *str;
if (arg == NULL)
return -1;
- arg->type = FILTER_ARG_BOOLEAN;
+ arg->type = TEP_FILTER_ARG_BOOLEAN;
if (strcmp(str, "TRUE") == 0)
arg->boolean.value = 1;
else
*
* Returns 0 on success and -1 if not all filters were copied
*/
-int tep_filter_copy(struct event_filter *dest, struct event_filter *source)
+int tep_filter_copy(struct tep_event_filter *dest, struct tep_event_filter *source)
{
int ret = 0;
int i;
* Returns 0 on success and -1 if there was a problem updating, but
* events may have still been updated on error.
*/
-int tep_update_trivial(struct event_filter *dest, struct event_filter *source,
- enum filter_trivial_type type)
+int tep_update_trivial(struct tep_event_filter *dest, struct tep_event_filter *source,
+ enum tep_filter_trivial_type type)
{
struct tep_handle *src_pevent;
struct tep_handle *dest_pevent;
- struct event_format *event;
- struct filter_type *filter_type;
- struct filter_arg *arg;
+ struct tep_event_format *event;
+ struct tep_filter_type *filter_type;
+ struct tep_filter_arg *arg;
char *str;
int i;
for (i = 0; i < dest->filters; i++) {
filter_type = &dest->event_filters[i];
arg = filter_type->filter;
- if (arg->type != FILTER_ARG_BOOLEAN)
+ if (arg->type != TEP_FILTER_ARG_BOOLEAN)
continue;
- if ((arg->boolean.value && type == FILTER_TRIVIAL_FALSE) ||
- (!arg->boolean.value && type == FILTER_TRIVIAL_TRUE))
+ if ((arg->boolean.value && type == TEP_FILTER_TRIVIAL_FALSE) ||
+ (!arg->boolean.value && type == TEP_FILTER_TRIVIAL_TRUE))
continue;
event = filter_type->event;
*
* Returns 0 on success and -1 if there was a problem.
*/
-int tep_filter_clear_trivial(struct event_filter *filter,
- enum filter_trivial_type type)
+int tep_filter_clear_trivial(struct tep_event_filter *filter,
+ enum tep_filter_trivial_type type)
{
- struct filter_type *filter_type;
+ struct tep_filter_type *filter_type;
int count = 0;
int *ids = NULL;
int i;
int *new_ids;
filter_type = &filter->event_filters[i];
- if (filter_type->filter->type != FILTER_ARG_BOOLEAN)
+ if (filter_type->filter->type != TEP_FILTER_ARG_BOOLEAN)
continue;
switch (type) {
- case FILTER_TRIVIAL_FALSE:
+ case TEP_FILTER_TRIVIAL_FALSE:
if (filter_type->filter->boolean.value)
continue;
break;
- case FILTER_TRIVIAL_TRUE:
+ case TEP_FILTER_TRIVIAL_TRUE:
if (!filter_type->filter->boolean.value)
continue;
default:
* Returns 1 if the event contains a matching trivial type
* otherwise 0.
*/
-int tep_filter_event_has_trivial(struct event_filter *filter,
+int tep_filter_event_has_trivial(struct tep_event_filter *filter,
int event_id,
- enum filter_trivial_type type)
+ enum tep_filter_trivial_type type)
{
- struct filter_type *filter_type;
+ struct tep_filter_type *filter_type;
if (!filter->filters)
return 0;
if (!filter_type)
return 0;
- if (filter_type->filter->type != FILTER_ARG_BOOLEAN)
+ if (filter_type->filter->type != TEP_FILTER_ARG_BOOLEAN)
return 0;
switch (type) {
- case FILTER_TRIVIAL_FALSE:
+ case TEP_FILTER_TRIVIAL_FALSE:
return !filter_type->filter->boolean.value;
- case FILTER_TRIVIAL_TRUE:
+ case TEP_FILTER_TRIVIAL_TRUE:
return filter_type->filter->boolean.value;
default:
return 1;
}
}
-static int test_filter(struct event_format *event, struct filter_arg *arg,
+static int test_filter(struct tep_event_format *event, struct tep_filter_arg *arg,
struct tep_record *record, enum tep_errno *err);
static const char *
-get_comm(struct event_format *event, struct tep_record *record)
+get_comm(struct tep_event_format *event, struct tep_record *record)
{
const char *comm;
int pid;
}
static unsigned long long
-get_value(struct event_format *event,
- struct format_field *field, struct tep_record *record)
+get_value(struct tep_event_format *event,
+ struct tep_format_field *field, struct tep_record *record)
{
unsigned long long val;
tep_read_number_field(field, record->data, &val);
- if (!(field->flags & FIELD_IS_SIGNED))
+ if (!(field->flags & TEP_FIELD_IS_SIGNED))
return val;
switch (field->size) {
}
static unsigned long long
-get_arg_value(struct event_format *event, struct filter_arg *arg,
+get_arg_value(struct tep_event_format *event, struct tep_filter_arg *arg,
struct tep_record *record, enum tep_errno *err);
static unsigned long long
-get_exp_value(struct event_format *event, struct filter_arg *arg,
+get_exp_value(struct tep_event_format *event, struct tep_filter_arg *arg,
struct tep_record *record, enum tep_errno *err)
{
unsigned long long lval, rval;
}
switch (arg->exp.type) {
- case FILTER_EXP_ADD:
+ case TEP_FILTER_EXP_ADD:
return lval + rval;
- case FILTER_EXP_SUB:
+ case TEP_FILTER_EXP_SUB:
return lval - rval;
- case FILTER_EXP_MUL:
+ case TEP_FILTER_EXP_MUL:
return lval * rval;
- case FILTER_EXP_DIV:
+ case TEP_FILTER_EXP_DIV:
return lval / rval;
- case FILTER_EXP_MOD:
+ case TEP_FILTER_EXP_MOD:
return lval % rval;
- case FILTER_EXP_RSHIFT:
+ case TEP_FILTER_EXP_RSHIFT:
return lval >> rval;
- case FILTER_EXP_LSHIFT:
+ case TEP_FILTER_EXP_LSHIFT:
return lval << rval;
- case FILTER_EXP_AND:
+ case TEP_FILTER_EXP_AND:
return lval & rval;
- case FILTER_EXP_OR:
+ case TEP_FILTER_EXP_OR:
return lval | rval;
- case FILTER_EXP_XOR:
+ case TEP_FILTER_EXP_XOR:
return lval ^ rval;
- case FILTER_EXP_NOT:
+ case TEP_FILTER_EXP_NOT:
default:
if (!*err)
*err = TEP_ERRNO__INVALID_EXP_TYPE;
}
static unsigned long long
-get_arg_value(struct event_format *event, struct filter_arg *arg,
+get_arg_value(struct tep_event_format *event, struct tep_filter_arg *arg,
struct tep_record *record, enum tep_errno *err)
{
switch (arg->type) {
- case FILTER_ARG_FIELD:
+ case TEP_FILTER_ARG_FIELD:
return get_value(event, arg->field.field, record);
- case FILTER_ARG_VALUE:
- if (arg->value.type != FILTER_NUMBER) {
+ case TEP_FILTER_ARG_VALUE:
+ if (arg->value.type != TEP_FILTER_NUMBER) {
if (!*err)
*err = TEP_ERRNO__NOT_A_NUMBER;
}
return arg->value.val;
- case FILTER_ARG_EXP:
+ case TEP_FILTER_ARG_EXP:
return get_exp_value(event, arg, record, err);
default:
return 0;
}
-static int test_num(struct event_format *event, struct filter_arg *arg,
+static int test_num(struct tep_event_format *event, struct tep_filter_arg *arg,
struct tep_record *record, enum tep_errno *err)
{
unsigned long long lval, rval;
}
switch (arg->num.type) {
- case FILTER_CMP_EQ:
+ case TEP_FILTER_CMP_EQ:
return lval == rval;
- case FILTER_CMP_NE:
+ case TEP_FILTER_CMP_NE:
return lval != rval;
- case FILTER_CMP_GT:
+ case TEP_FILTER_CMP_GT:
return lval > rval;
- case FILTER_CMP_LT:
+ case TEP_FILTER_CMP_LT:
return lval < rval;
- case FILTER_CMP_GE:
+ case TEP_FILTER_CMP_GE:
return lval >= rval;
- case FILTER_CMP_LE:
+ case TEP_FILTER_CMP_LE:
return lval <= rval;
default:
}
}
-static const char *get_field_str(struct filter_arg *arg, struct tep_record *record)
+static const char *get_field_str(struct tep_filter_arg *arg, struct tep_record *record)
{
- struct event_format *event;
+ struct tep_event_format *event;
struct tep_handle *pevent;
unsigned long long addr;
const char *val = NULL;
char hex[64];
/* If the field is not a string convert it */
- if (arg->str.field->flags & FIELD_IS_STRING) {
+ if (arg->str.field->flags & TEP_FIELD_IS_STRING) {
val = record->data + arg->str.field->offset;
size = arg->str.field->size;
- if (arg->str.field->flags & FIELD_IS_DYNAMIC) {
+ if (arg->str.field->flags & TEP_FIELD_IS_DYNAMIC) {
addr = *(unsigned int *)val;
val = record->data + (addr & 0xffff);
size = addr >> 16;
pevent = event->pevent;
addr = get_value(event, arg->str.field, record);
- if (arg->str.field->flags & (FIELD_IS_POINTER | FIELD_IS_LONG))
+ if (arg->str.field->flags & (TEP_FIELD_IS_POINTER | TEP_FIELD_IS_LONG))
/* convert to a kernel symbol */
val = tep_find_function(pevent, addr);
return val;
}
-static int test_str(struct event_format *event, struct filter_arg *arg,
+static int test_str(struct tep_event_format *event, struct tep_filter_arg *arg,
struct tep_record *record, enum tep_errno *err)
{
const char *val;
val = get_field_str(arg, record);
switch (arg->str.type) {
- case FILTER_CMP_MATCH:
+ case TEP_FILTER_CMP_MATCH:
return strcmp(val, arg->str.val) == 0;
- case FILTER_CMP_NOT_MATCH:
+ case TEP_FILTER_CMP_NOT_MATCH:
return strcmp(val, arg->str.val) != 0;
- case FILTER_CMP_REGEX:
+ case TEP_FILTER_CMP_REGEX:
/* Returns zero on match */
return !regexec(&arg->str.reg, val, 0, NULL, 0);
- case FILTER_CMP_NOT_REGEX:
+ case TEP_FILTER_CMP_NOT_REGEX:
return regexec(&arg->str.reg, val, 0, NULL, 0);
default:
}
}
-static int test_op(struct event_format *event, struct filter_arg *arg,
+static int test_op(struct tep_event_format *event, struct tep_filter_arg *arg,
struct tep_record *record, enum tep_errno *err)
{
switch (arg->op.type) {
- case FILTER_OP_AND:
+ case TEP_FILTER_OP_AND:
return test_filter(event, arg->op.left, record, err) &&
test_filter(event, arg->op.right, record, err);
- case FILTER_OP_OR:
+ case TEP_FILTER_OP_OR:
return test_filter(event, arg->op.left, record, err) ||
test_filter(event, arg->op.right, record, err);
- case FILTER_OP_NOT:
+ case TEP_FILTER_OP_NOT:
return !test_filter(event, arg->op.right, record, err);
default:
}
}
-static int test_filter(struct event_format *event, struct filter_arg *arg,
+static int test_filter(struct tep_event_format *event, struct tep_filter_arg *arg,
struct tep_record *record, enum tep_errno *err)
{
if (*err) {
}
switch (arg->type) {
- case FILTER_ARG_BOOLEAN:
+ case TEP_FILTER_ARG_BOOLEAN:
/* easy case */
return arg->boolean.value;
- case FILTER_ARG_OP:
+ case TEP_FILTER_ARG_OP:
return test_op(event, arg, record, err);
- case FILTER_ARG_NUM:
+ case TEP_FILTER_ARG_NUM:
return test_num(event, arg, record, err);
- case FILTER_ARG_STR:
+ case TEP_FILTER_ARG_STR:
return test_str(event, arg, record, err);
- case FILTER_ARG_EXP:
- case FILTER_ARG_VALUE:
- case FILTER_ARG_FIELD:
+ case TEP_FILTER_ARG_EXP:
+ case TEP_FILTER_ARG_VALUE:
+ case TEP_FILTER_ARG_FIELD:
/*
* Expressions, fields and values evaluate
* to true if they return non zero
* Returns 1 if filter found for @event_id
* otherwise 0;
*/
-int tep_event_filtered(struct event_filter *filter, int event_id)
+int tep_event_filtered(struct tep_event_filter *filter, int event_id)
{
- struct filter_type *filter_type;
+ struct tep_filter_type *filter_type;
if (!filter->filters)
return 0;
* NO_FILTER - if no filters exist
* otherwise - error occurred during test
*/
-enum tep_errno tep_filter_match(struct event_filter *filter,
+enum tep_errno tep_filter_match(struct tep_event_filter *filter,
struct tep_record *record)
{
struct tep_handle *pevent = filter->pevent;
- struct filter_type *filter_type;
+ struct tep_filter_type *filter_type;
int event_id;
int ret;
enum tep_errno err = 0;
return ret ? TEP_ERRNO__FILTER_MATCH : TEP_ERRNO__FILTER_MISS;
}
-static char *op_to_str(struct event_filter *filter, struct filter_arg *arg)
+static char *op_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
{
char *str = NULL;
char *left = NULL;
int val;
switch (arg->op.type) {
- case FILTER_OP_AND:
+ case TEP_FILTER_OP_AND:
op = "&&";
/* fall through */
- case FILTER_OP_OR:
+ case TEP_FILTER_OP_OR:
if (!op)
op = "||";
right_val = 0;
if (left_val >= 0) {
- if ((arg->op.type == FILTER_OP_AND && !left_val) ||
- (arg->op.type == FILTER_OP_OR && left_val)) {
+ if ((arg->op.type == TEP_FILTER_OP_AND && !left_val) ||
+ (arg->op.type == TEP_FILTER_OP_OR && left_val)) {
/* Just return left value */
str = left;
left = NULL;
/* just evaluate this. */
val = 0;
switch (arg->op.type) {
- case FILTER_OP_AND:
+ case TEP_FILTER_OP_AND:
val = left_val && right_val;
break;
- case FILTER_OP_OR:
+ case TEP_FILTER_OP_OR:
val = left_val || right_val;
break;
default:
}
}
if (right_val >= 0) {
- if ((arg->op.type == FILTER_OP_AND && !right_val) ||
- (arg->op.type == FILTER_OP_OR && right_val)) {
+ if ((arg->op.type == TEP_FILTER_OP_AND && !right_val) ||
+ (arg->op.type == TEP_FILTER_OP_OR && right_val)) {
/* Just return right value */
str = right;
right = NULL;
asprintf(&str, "(%s) %s (%s)", left, op, right);
break;
- case FILTER_OP_NOT:
+ case TEP_FILTER_OP_NOT:
op = "!";
right = arg_to_str(filter, arg->op.right);
if (!right)
return str;
}
-static char *val_to_str(struct event_filter *filter, struct filter_arg *arg)
+static char *val_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
{
char *str = NULL;
return str;
}
-static char *field_to_str(struct event_filter *filter, struct filter_arg *arg)
+static char *field_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
{
return strdup(arg->field.field->name);
}
-static char *exp_to_str(struct event_filter *filter, struct filter_arg *arg)
+static char *exp_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
{
char *lstr;
char *rstr;
goto out;
switch (arg->exp.type) {
- case FILTER_EXP_ADD:
+ case TEP_FILTER_EXP_ADD:
op = "+";
break;
- case FILTER_EXP_SUB:
+ case TEP_FILTER_EXP_SUB:
op = "-";
break;
- case FILTER_EXP_MUL:
+ case TEP_FILTER_EXP_MUL:
op = "*";
break;
- case FILTER_EXP_DIV:
+ case TEP_FILTER_EXP_DIV:
op = "/";
break;
- case FILTER_EXP_MOD:
+ case TEP_FILTER_EXP_MOD:
op = "%";
break;
- case FILTER_EXP_RSHIFT:
+ case TEP_FILTER_EXP_RSHIFT:
op = ">>";
break;
- case FILTER_EXP_LSHIFT:
+ case TEP_FILTER_EXP_LSHIFT:
op = "<<";
break;
- case FILTER_EXP_AND:
+ case TEP_FILTER_EXP_AND:
op = "&";
break;
- case FILTER_EXP_OR:
+ case TEP_FILTER_EXP_OR:
op = "|";
break;
- case FILTER_EXP_XOR:
+ case TEP_FILTER_EXP_XOR:
op = "^";
break;
default:
return str;
}
-static char *num_to_str(struct event_filter *filter, struct filter_arg *arg)
+static char *num_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
{
char *lstr;
char *rstr;
goto out;
switch (arg->num.type) {
- case FILTER_CMP_EQ:
+ case TEP_FILTER_CMP_EQ:
op = "==";
/* fall through */
- case FILTER_CMP_NE:
+ case TEP_FILTER_CMP_NE:
if (!op)
op = "!=";
/* fall through */
- case FILTER_CMP_GT:
+ case TEP_FILTER_CMP_GT:
if (!op)
op = ">";
/* fall through */
- case FILTER_CMP_LT:
+ case TEP_FILTER_CMP_LT:
if (!op)
op = "<";
/* fall through */
- case FILTER_CMP_GE:
+ case TEP_FILTER_CMP_GE:
if (!op)
op = ">=";
/* fall through */
- case FILTER_CMP_LE:
+ case TEP_FILTER_CMP_LE:
if (!op)
op = "<=";
return str;
}
-static char *str_to_str(struct event_filter *filter, struct filter_arg *arg)
+static char *str_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
{
char *str = NULL;
char *op = NULL;
switch (arg->str.type) {
- case FILTER_CMP_MATCH:
+ case TEP_FILTER_CMP_MATCH:
op = "==";
/* fall through */
- case FILTER_CMP_NOT_MATCH:
+ case TEP_FILTER_CMP_NOT_MATCH:
if (!op)
op = "!=";
/* fall through */
- case FILTER_CMP_REGEX:
+ case TEP_FILTER_CMP_REGEX:
if (!op)
op = "=~";
/* fall through */
- case FILTER_CMP_NOT_REGEX:
+ case TEP_FILTER_CMP_NOT_REGEX:
if (!op)
op = "!~";
return str;
}
-static char *arg_to_str(struct event_filter *filter, struct filter_arg *arg)
+static char *arg_to_str(struct tep_event_filter *filter, struct tep_filter_arg *arg)
{
char *str = NULL;
switch (arg->type) {
- case FILTER_ARG_BOOLEAN:
+ case TEP_FILTER_ARG_BOOLEAN:
asprintf(&str, arg->boolean.value ? "TRUE" : "FALSE");
return str;
- case FILTER_ARG_OP:
+ case TEP_FILTER_ARG_OP:
return op_to_str(filter, arg);
- case FILTER_ARG_NUM:
+ case TEP_FILTER_ARG_NUM:
return num_to_str(filter, arg);
- case FILTER_ARG_STR:
+ case TEP_FILTER_ARG_STR:
return str_to_str(filter, arg);
- case FILTER_ARG_VALUE:
+ case TEP_FILTER_ARG_VALUE:
return val_to_str(filter, arg);
- case FILTER_ARG_FIELD:
+ case TEP_FILTER_ARG_FIELD:
return field_to_str(filter, arg);
- case FILTER_ARG_EXP:
+ case TEP_FILTER_ARG_EXP:
return exp_to_str(filter, arg);
default:
* NULL is returned if no filter is found or allocation failed.
*/
char *
-tep_filter_make_string(struct event_filter *filter, int event_id)
+tep_filter_make_string(struct tep_event_filter *filter, int event_id)
{
- struct filter_type *filter_type;
+ struct tep_filter_type *filter_type;
if (!filter->filters)
return NULL;
* 1 if the two filters hold the same content.
* 0 if they do not.
*/
-int tep_filter_compare(struct event_filter *filter1, struct event_filter *filter2)
+int tep_filter_compare(struct tep_event_filter *filter1, struct tep_event_filter *filter2)
{
- struct filter_type *filter_type1;
- struct filter_type *filter_type2;
+ struct tep_filter_type *filter_type1;
+ struct tep_filter_type *filter_type2;
char *str1, *str2;
int result;
int i;
if (filter_type1->filter->type != filter_type2->filter->type)
break;
switch (filter_type1->filter->type) {
- case FILTER_TRIVIAL_FALSE:
- case FILTER_TRIVIAL_TRUE:
+ case TEP_FILTER_TRIVIAL_FALSE:
+ case TEP_FILTER_TRIVIAL_TRUE:
/* trivial types just need the type compared */
continue;
default:
#include "event-parse.h"
#include "event-utils.h"
+#include "trace-seq.h"
static struct func_stack {
int size;
}
static int function_handler(struct trace_seq *s, struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
struct tep_handle *pevent = event->pevent;
unsigned long long function;
#include <string.h>
#include "event-parse.h"
+#include "trace-seq.h"
static int timer_expire_handler(struct trace_seq *s,
struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
trace_seq_printf(s, "hrtimer=");
static int timer_start_handler(struct trace_seq *s,
struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
trace_seq_printf(s, "hrtimer=");
#include <string.h>
#include "event-parse.h"
+#include "trace-seq.h"
#define MINORBITS 20
#define MINORMASK ((1U << MINORBITS) - 1)
#include <string.h>
#include "event-parse.h"
+#include "trace-seq.h"
static int call_site_handler(struct trace_seq *s, struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
- struct format_field *field;
+ struct tep_format_field *field;
unsigned long long val, addr;
void *data = record->data;
const char *func;
#include <stdint.h>
#include "event-parse.h"
+#include "trace-seq.h"
#ifdef HAVE_UDIS86
}
static int print_exit_reason(struct trace_seq *s, struct tep_record *record,
- struct event_format *event, const char *field)
+ struct tep_event_format *event, const char *field)
{
unsigned long long isa;
unsigned long long val;
}
static int kvm_exit_handler(struct trace_seq *s, struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
unsigned long long info1 = 0, info2 = 0;
static int kvm_emulate_insn_handler(struct trace_seq *s,
struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
unsigned long long rip, csbase, len, flags, failed;
int llen;
static int kvm_nested_vmexit_inject_handler(struct trace_seq *s, struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
if (print_exit_reason(s, record, event, "exit_code") < 0)
return -1;
}
static int kvm_nested_vmexit_handler(struct trace_seq *s, struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
tep_print_num_field(s, "rip %llx ", event, "rip", record, 1);
};
static int kvm_mmu_print_role(struct trace_seq *s, struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
unsigned long long val;
static const char *access_str[] = {
static int kvm_mmu_get_page_handler(struct trace_seq *s,
struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
unsigned long long val;
#include <string.h>
#include "event-parse.h"
+#include "trace-seq.h"
#define INDENT 65
-static void print_string(struct trace_seq *s, struct event_format *event,
+static void print_string(struct trace_seq *s, struct tep_event_format *event,
const char *name, const void *data)
{
- struct format_field *f = tep_find_field(event, name);
+ struct tep_format_field *f = tep_find_field(event, name);
int offset;
int length;
static int drv_bss_info_changed(struct trace_seq *s,
struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
void *data = record->data;
#include <string.h>
#include "event-parse.h"
+#include "trace-seq.h"
static void write_state(struct trace_seq *s, int val)
{
trace_seq_putc(s, 'R');
}
-static void write_and_save_comm(struct format_field *field,
+static void write_and_save_comm(struct tep_format_field *field,
struct tep_record *record,
struct trace_seq *s, int pid)
{
static int sched_wakeup_handler(struct trace_seq *s,
struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
- struct format_field *field;
+ struct tep_format_field *field;
unsigned long long val;
if (tep_get_field_val(s, event, "pid", record, &val, 1))
static int sched_switch_handler(struct trace_seq *s,
struct tep_record *record,
- struct event_format *event, void *context)
+ struct tep_event_format *event, void *context)
{
- struct format_field *field;
+ struct tep_format_field *field;
unsigned long long val;
if (tep_get_field_val(s, event, "prev_pid", record, &val, 1))
#include <string.h>
#include <inttypes.h>
#include "event-parse.h"
+#include "trace-seq.h"
typedef unsigned long sector_t;
typedef uint64_t u64;
#include <stdlib.h>
#include <string.h>
#include "event-parse.h"
+#include "trace-seq.h"
#define __HYPERVISOR_set_trap_table 0
#define __HYPERVISOR_mmu_update 1
--- /dev/null
+// SPDX-License-Identifier: LGPL-2.1
+#undef _GNU_SOURCE
+#include <string.h>
+#include <stdio.h>
+
+#include "event-parse.h"
+
+#undef _PE
+#define _PE(code, str) str
+static const char * const tep_error_str[] = {
+ TEP_ERRORS
+};
+#undef _PE
+
+/*
+ * The tools so far have been using the strerror_r() GNU variant, that returns
+ * a string, be it the buffer passed or something else.
+ *
+ * But that, besides being tricky in cases where we expect that the function
+ * using strerror_r() returns the error formatted in a provided buffer (we have
+ * to check if it returned something else and copy that instead), breaks the
+ * build on systems not using glibc, like Alpine Linux, where musl libc is
+ * used.
+ *
+ * So, introduce yet another wrapper, str_error_r(), that has the GNU
+ * interface, but uses the portable XSI variant of strerror_r(), so that users
+ * rest asured that the provided buffer is used and it is what is returned.
+ */
+int tep_strerror(struct tep_handle *tep __maybe_unused,
+ enum tep_errno errnum, char *buf, size_t buflen)
+{
+ const char *msg;
+ int idx;
+
+ if (!buflen)
+ return 0;
+
+ if (errnum >= 0) {
+ int err = strerror_r(errnum, buf, buflen);
+ buf[buflen - 1] = 0;
+ return err;
+ }
+
+ if (errnum <= __TEP_ERRNO__START ||
+ errnum >= __TEP_ERRNO__END)
+ return -1;
+
+ idx = errnum - __TEP_ERRNO__START - 1;
+ msg = tep_error_str[idx];
+ snprintf(buf, buflen, "%s", msg);
+
+ return 0;
+}
* Copyright (C) 2009 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
*
*/
+#include "trace-seq.h"
+
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
--- /dev/null
+// SPDX-License-Identifier: LGPL-2.1
+/*
+ * Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
+ *
+ */
+
+#ifndef _TRACE_SEQ_H
+#define _TRACE_SEQ_H
+
+#include <stdarg.h>
+#include <stdio.h>
+
+/* ----------------------- trace_seq ----------------------- */
+
+#ifndef TRACE_SEQ_BUF_SIZE
+#define TRACE_SEQ_BUF_SIZE 4096
+#endif
+
+enum trace_seq_fail {
+ TRACE_SEQ__GOOD,
+ TRACE_SEQ__BUFFER_POISONED,
+ TRACE_SEQ__MEM_ALLOC_FAILED,
+};
+
+/*
+ * Trace sequences are used to allow a function to call several other functions
+ * to create a string of data to use (up to a max of PAGE_SIZE).
+ */
+
+struct trace_seq {
+ char *buffer;
+ unsigned int buffer_size;
+ unsigned int len;
+ unsigned int readpos;
+ enum trace_seq_fail state;
+};
+
+void trace_seq_init(struct trace_seq *s);
+void trace_seq_reset(struct trace_seq *s);
+void trace_seq_destroy(struct trace_seq *s);
+
+extern int trace_seq_printf(struct trace_seq *s, const char *fmt, ...)
+ __attribute__ ((format (printf, 2, 3)));
+extern int trace_seq_vprintf(struct trace_seq *s, const char *fmt, va_list args)
+ __attribute__ ((format (printf, 2, 0)));
+
+extern int trace_seq_puts(struct trace_seq *s, const char *str);
+extern int trace_seq_putc(struct trace_seq *s, unsigned char c);
+
+extern void trace_seq_terminate(struct trace_seq *s);
+
+extern int trace_seq_do_fprintf(struct trace_seq *s, FILE *fp);
+extern int trace_seq_do_printf(struct trace_seq *s);
+
+#endif /* _TRACE_SEQ_H */
20. THE HAPPENS-BEFORE RELATION: hb
21. THE PROPAGATES-BEFORE RELATION: pb
22. RCU RELATIONS: rcu-link, gp, rscs, rcu-fence, and rb
- 23. ODDS AND ENDS
+ 23. LOCKING
+ 24. ODDS AND ENDS
violating the write-write coherence rule by requiring the CPU not to
send the W write to the memory subsystem at all!)
-There is one last example of preserved program order in the LKMM: when
-a load-acquire reads from an earlier store-release. For example:
-
- smp_store_release(&x, 123);
- r1 = smp_load_acquire(&x);
-
-If the smp_load_acquire() ends up obtaining the 123 value that was
-stored by the smp_store_release(), the LKMM says that the load must be
-executed after the store; the store cannot be forwarded to the load.
-This requirement does not arise from the operational model, but it
-yields correct predictions on all architectures supported by the Linux
-kernel, although for differing reasons.
-
-On some architectures, including x86 and ARMv8, it is true that the
-store cannot be forwarded to the load. On others, including PowerPC
-and ARMv7, smp_store_release() generates object code that starts with
-a fence and smp_load_acquire() generates object code that ends with a
-fence. The upshot is that even though the store may be forwarded to
-the load, it is still true that any instruction preceding the store
-will be executed before the load or any following instructions, and
-the store will be executed before any instruction following the load.
-
AND THEN THERE WAS ALPHA
------------------------
grace period does and ends after it does.
+LOCKING
+-------
+
+The LKMM includes locking. In fact, there is special code for locking
+in the formal model, added in order to make tools run faster.
+However, this special code is intended to be more or less equivalent
+to concepts we have already covered. A spinlock_t variable is treated
+the same as an int, and spin_lock(&s) is treated almost the same as:
+
+ while (cmpxchg_acquire(&s, 0, 1) != 0)
+ cpu_relax();
+
+This waits until s is equal to 0 and then atomically sets it to 1,
+and the read part of the cmpxchg operation acts as an acquire fence.
+An alternate way to express the same thing would be:
+
+ r = xchg_acquire(&s, 1);
+
+along with a requirement that at the end, r = 0. Similarly,
+spin_trylock(&s) is treated almost the same as:
+
+ return !cmpxchg_acquire(&s, 0, 1);
+
+which atomically sets s to 1 if it is currently equal to 0 and returns
+true if it succeeds (the read part of the cmpxchg operation acts as an
+acquire fence only if the operation is successful). spin_unlock(&s)
+is treated almost the same as:
+
+ smp_store_release(&s, 0);
+
+The "almost" qualifiers above need some explanation. In the LKMM, the
+store-release in a spin_unlock() and the load-acquire which forms the
+first half of the atomic rmw update in a spin_lock() or a successful
+spin_trylock() -- we can call these things lock-releases and
+lock-acquires -- have two properties beyond those of ordinary releases
+and acquires.
+
+First, when a lock-acquire reads from a lock-release, the LKMM
+requires that every instruction po-before the lock-release must
+execute before any instruction po-after the lock-acquire. This would
+naturally hold if the release and acquire operations were on different
+CPUs, but the LKMM says it holds even when they are on the same CPU.
+For example:
+
+ int x, y;
+ spinlock_t s;
+
+ P0()
+ {
+ int r1, r2;
+
+ spin_lock(&s);
+ r1 = READ_ONCE(x);
+ spin_unlock(&s);
+ spin_lock(&s);
+ r2 = READ_ONCE(y);
+ spin_unlock(&s);
+ }
+
+ P1()
+ {
+ WRITE_ONCE(y, 1);
+ smp_wmb();
+ WRITE_ONCE(x, 1);
+ }
+
+Here the second spin_lock() reads from the first spin_unlock(), and
+therefore the load of x must execute before the load of y. Thus we
+cannot have r1 = 1 and r2 = 0 at the end (this is an instance of the
+MP pattern).
+
+This requirement does not apply to ordinary release and acquire
+fences, only to lock-related operations. For instance, suppose P0()
+in the example had been written as:
+
+ P0()
+ {
+ int r1, r2, r3;
+
+ r1 = READ_ONCE(x);
+ smp_store_release(&s, 1);
+ r3 = smp_load_acquire(&s);
+ r2 = READ_ONCE(y);
+ }
+
+Then the CPU would be allowed to forward the s = 1 value from the
+smp_store_release() to the smp_load_acquire(), executing the
+instructions in the following order:
+
+ r3 = smp_load_acquire(&s); // Obtains r3 = 1
+ r2 = READ_ONCE(y);
+ r1 = READ_ONCE(x);
+ smp_store_release(&s, 1); // Value is forwarded
+
+and thus it could load y before x, obtaining r2 = 0 and r1 = 1.
+
+Second, when a lock-acquire reads from a lock-release, and some other
+stores W and W' occur po-before the lock-release and po-after the
+lock-acquire respectively, the LKMM requires that W must propagate to
+each CPU before W' does. For example, consider:
+
+ int x, y;
+ spinlock_t x;
+
+ P0()
+ {
+ spin_lock(&s);
+ WRITE_ONCE(x, 1);
+ spin_unlock(&s);
+ }
+
+ P1()
+ {
+ int r1;
+
+ spin_lock(&s);
+ r1 = READ_ONCE(x);
+ WRITE_ONCE(y, 1);
+ spin_unlock(&s);
+ }
+
+ P2()
+ {
+ int r2, r3;
+
+ r2 = READ_ONCE(y);
+ smp_rmb();
+ r3 = READ_ONCE(x);
+ }
+
+If r1 = 1 at the end then the spin_lock() in P1 must have read from
+the spin_unlock() in P0. Hence the store to x must propagate to P2
+before the store to y does, so we cannot have r2 = 1 and r3 = 0.
+
+These two special requirements for lock-release and lock-acquire do
+not arise from the operational model. Nevertheless, kernel developers
+have come to expect and rely on them because they do hold on all
+architectures supported by the Linux kernel, albeit for various
+differing reasons.
+
+
ODDS AND ENDS
-------------
events and the events preceding them against all po-later
events.
-The LKMM includes locking. In fact, there is special code for locking
-in the formal model, added in order to make tools run faster.
-However, this special code is intended to be exactly equivalent to
-concepts we have already covered. A spinlock_t variable is treated
-the same as an int, and spin_lock(&s) is treated the same as:
-
- while (cmpxchg_acquire(&s, 0, 1) != 0)
- cpu_relax();
-
-which waits until s is equal to 0 and then atomically sets it to 1,
-and where the read part of the atomic update is also an acquire fence.
-An alternate way to express the same thing would be:
-
- r = xchg_acquire(&s, 1);
-
-along with a requirement that at the end, r = 0. spin_unlock(&s) is
-treated the same as:
-
- smp_store_release(&s, 0);
-
Interestingly, RCU and locking each introduce the possibility of
deadlock. When faced with code sequences such as:
smp_rmb() macro orders prior loads against later loads. Therefore, if
the final value of r0 is 1, the final value of r1 must also be 1.
-The the xlog_state_switch_iclogs() function in fs/xfs/xfs_log.c contains
+The xlog_state_switch_iclogs() function in fs/xfs/xfs_log.c contains
the following write-side code fragment:
log->l_curr_block -= log->l_logBBsize;
particular, the "THE PROGRAM ORDER RELATION: po AND po-loc"
and "A WARNING" sections).
+ Note that this limitation in turn limits LKMM's ability to
+ accurately model address, control, and data dependencies.
+ For example, if the compiler can deduce the value of some variable
+ carrying a dependency, then the compiler can break that dependency
+ by substituting a constant of that value.
+
2. Multiple access sizes for a single variable are not supported,
and neither are misaligned or partially overlapping accesses.
However, a substantial amount of support is provided for these
operations, as shown in the linux-kernel.def file.
+ a. When rcu_assign_pointer() is passed NULL, the Linux
+ kernel provides no ordering, but LKMM models this
+ case as a store release.
+
+ b. The "unless" RMW operations are not currently modeled:
+ atomic_long_add_unless(), atomic_add_unless(),
+ atomic_inc_unless_negative(), and
+ atomic_dec_unless_positive(). These can be emulated
+ in litmus tests, for example, by using atomic_cmpxchg().
+
+ c. The call_rcu() function is not modeled. It can be
+ emulated in litmus tests by adding another process that
+ invokes synchronize_rcu() and the body of the callback
+ function, with (for example) a release-acquire from
+ the site of the emulated call_rcu() to the beginning
+ of the additional process.
+
+ d. The rcu_barrier() function is not modeled. It can be
+ emulated in litmus tests emulating call_rcu() via
+ (for example) a release-acquire from the end of each
+ additional call_rcu() process to the site of the
+ emulated rcu-barrier().
+
+ e. Sleepable RCU (SRCU) is not modeled. It can be
+ emulated, but perhaps not simply.
+
+ f. Reader-writer locking is not modeled. It can be
+ emulated in litmus tests using atomic read-modify-write
+ operations.
+
The "herd7" tool has some additional limitations of its own, apart from
the memory model:
Some of these limitations may be overcome in the future, but others are
more likely to be addressed by incorporating the Linux-kernel memory model
into other tools.
+
+Finally, please note that LKMM is subject to change as hardware, use cases,
+and compilers evolve.
(* Release Acquire *)
let acq-po = [Acquire] ; po ; [M]
let po-rel = [M] ; po ; [Release]
-let rfi-rel-acq = [Release] ; rfi ; [Acquire]
+let po-unlock-rf-lock-po = po ; [UL] ; rf ; [LKR] ; po
(**********************************)
(* Fundamental coherence ordering *)
let rwdep = (dep | ctrl) ; [W]
let overwrite = co | fr
let to-w = rwdep | (overwrite & int)
-let to-r = addr | (dep ; rfi) | rfi-rel-acq
+let to-r = addr | (dep ; rfi)
let fence = strong-fence | wmb | po-rel | rmb | acq-po
-let ppo = to-r | to-w | fence
+let ppo = to-r | to-w | fence | (po-unlock-rf-lock-po & int)
(* Propagation: Ordering from release operations and strong fences. *)
let A-cumul(r) = rfe? ; r
-let cumul-fence = A-cumul(strong-fence | po-rel) | wmb
+let cumul-fence = A-cumul(strong-fence | po-rel) | wmb | po-unlock-rf-lock-po
let prop = (overwrite & ext)? ; cumul-fence* ; rfe?
(*
C ISA2+pooncelock+pooncelock+pombonce
(*
- * Result: Sometimes
+ * Result: Never
*
- * This test shows that the ordering provided by a lock-protected S
- * litmus test (P0() and P1()) are not visible to external process P2().
- * This is likely to change soon.
+ * This test shows that write-write ordering provided by locks
+ * (in P0() and P1()) is visible to external process P2().
*)
{}
-This directory contains the following litmus tests:
+============
+LITMUS TESTS
+============
CoRR+poonceonce+Once.litmus
Test of read-read coherence, that is, whether or not two
ISA2+pooncelock+pooncelock+pombonce.litmus
Tests whether the ordering provided by a lock-protected S
litmus test is visible to an external process whose accesses are
- separated by smp_mb(). This addition of an external process to
+ separated by smp_mb(). This addition of an external process to
S is otherwise known as ISA2.
ISA2+poonceonces.litmus
A great many more litmus tests are available here:
https://github.com/paulmckrcu/litmus
+
+==================
+LITMUS TEST NAMING
+==================
+
+Litmus tests are usually named based on their contents, which means that
+looking at the name tells you what the litmus test does. The naming
+scheme covers litmus tests having a single cycle that passes through
+each process exactly once, so litmus tests not fitting this description
+are named on an ad-hoc basis.
+
+The structure of a litmus-test name is the litmus-test class, a plus
+sign ("+"), and one string for each process, separated by plus signs.
+The end of the name is ".litmus".
+
+The litmus-test classes may be found in the infamous test6.pdf:
+https://www.cl.cam.ac.uk/~pes20/ppc-supplemental/test6.pdf
+Each class defines the pattern of accesses and of the variables accessed.
+For example, if the one process writes to a pair of variables, and
+the other process reads from these same variables, the corresponding
+litmus-test class is "MP" (message passing), which may be found on the
+left-hand end of the second row of tests on page one of test6.pdf.
+
+The strings used to identify the actions carried out by each process are
+complex due to a desire to have short(er) names. Thus, there is a tool to
+generate these strings from a given litmus test's actions. For example,
+consider the processes from SB+rfionceonce-poonceonces.litmus:
+
+ P0(int *x, int *y)
+ {
+ int r1;
+ int r2;
+
+ WRITE_ONCE(*x, 1);
+ r1 = READ_ONCE(*x);
+ r2 = READ_ONCE(*y);
+ }
+
+ P1(int *x, int *y)
+ {
+ int r3;
+ int r4;
+
+ WRITE_ONCE(*y, 1);
+ r3 = READ_ONCE(*y);
+ r4 = READ_ONCE(*x);
+ }
+
+The next step is to construct a space-separated list of descriptors,
+interleaving descriptions of the relation between a pair of consecutive
+accesses with descriptions of the second access in the pair.
+
+P0()'s WRITE_ONCE() is read by its first READ_ONCE(), which is a
+reads-from link (rf) and internal to the P0() process. This is
+"rfi", which is an abbreviation for "reads-from internal". Because
+some of the tools string these abbreviations together with space
+characters separating processes, the first character is capitalized,
+resulting in "Rfi".
+
+P0()'s second access is a READ_ONCE(), as opposed to (for example)
+smp_load_acquire(), so next is "Once". Thus far, we have "Rfi Once".
+
+P0()'s third access is also a READ_ONCE(), but to y rather than x.
+This is related to P0()'s second access by program order ("po"),
+to a different variable ("d"), and both accesses are reads ("RR").
+The resulting descriptor is "PodRR". Because P0()'s third access is
+READ_ONCE(), we add another "Once" descriptor.
+
+A from-read ("fre") relation links P0()'s third to P1()'s first
+access, and the resulting descriptor is "Fre". P1()'s first access is
+WRITE_ONCE(), which as before gives the descriptor "Once". The string
+thus far is thus "Rfi Once PodRR Once Fre Once".
+
+The remainder of P1() is similar to P0(), which means we add
+"Rfi Once PodRR Once". Another fre links P1()'s last access to
+P0()'s first access, which is WRITE_ONCE(), so we add "Fre Once".
+The full string is thus:
+
+ Rfi Once PodRR Once Fre Once Rfi Once PodRR Once Fre Once
+
+This string can be given to the "norm7" and "classify7" tools to
+produce the name:
+
+ $ norm7 -bell linux-kernel.bell \
+ Rfi Once PodRR Once Fre Once Rfi Once PodRR Once Fre Once | \
+ sed -e 's/:.*//g'
+ SB+rfionceonce-poonceonces
+
+Adding the ".litmus" suffix: SB+rfionceonce-poonceonces.litmus
+
+The descriptors that describe connections between consecutive accesses
+within the cycle through a given litmus test can be provided by the herd
+tool (Rfi, Po, Fre, and so on) or by the linux-kernel.bell file (Once,
+Release, Acquire, and so on).
+
+To see the full list of descriptors, execute the following command:
+
+ $ diyone7 -bell linux-kernel.bell -show edges
#define EX_ORIG_OFFSET 0
#define EX_NEW_OFFSET 4
-#define JUMP_ENTRY_SIZE 24
+#define JUMP_ENTRY_SIZE 16
#define JUMP_ORIG_OFFSET 0
-#define JUMP_NEW_OFFSET 8
+#define JUMP_NEW_OFFSET 4
#define ALT_ENTRY_SIZE 13
#define ALT_ORIG_OFFSET 0
JDIR=$(shell /usr/sbin/update-java-alternatives -l | head -1 | awk '{print $$3}')
else
ifneq (,$(wildcard /usr/sbin/alternatives))
- JDIR=$(shell alternatives --display java | tail -1 | cut -d' ' -f 5 | sed 's%/jre/bin/java.%%g')
+ JDIR=$(shell /usr/sbin/alternatives --display java | tail -1 | cut -d' ' -f 5 | sed 's%/jre/bin/java.%%g')
endif
endif
ifndef JDIR
$(LIB_FILE): $(LIBPERF_IN)
$(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $(LIBPERF_IN) $(LIB_OBJS)
-LIBTRACEEVENT_FLAGS += plugin_dir=$(plugindir_SQ)
+LIBTRACEEVENT_FLAGS += plugin_dir=$(plugindir_SQ) 'EXTRA_CFLAGS=$(EXTRA_CFLAGS)' 'LDFLAGS=$(LDFLAGS)'
$(LIBTRACEEVENT): FORCE
$(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) $(OUTPUT)libtraceevent.a
ifndef NO_LIBBPF
$(call QUIET_INSTALL, bpf-headers) \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf'; \
- $(INSTALL) include/bpf/*.h -t '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf'
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf/linux'; \
+ $(INSTALL) include/bpf/*.h -t '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf'; \
+ $(INSTALL) include/bpf/linux/*.h -t '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf/linux'
$(call QUIET_INSTALL, bpf-examples) \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perf_examples_instdir_SQ)/bpf'; \
$(INSTALL) examples/bpf/*.c -t '$(DESTDIR_SQ)$(perf_examples_instdir_SQ)/bpf'
jump_insn;
};
+static int arm64_mov__parse(struct arch *arch __maybe_unused,
+ struct ins_operands *ops,
+ struct map_symbol *ms __maybe_unused)
+{
+ char *s = strchr(ops->raw, ','), *target, *endptr;
+
+ if (s == NULL)
+ return -1;
+
+ *s = '\0';
+ ops->source.raw = strdup(ops->raw);
+ *s = ',';
+
+ if (ops->source.raw == NULL)
+ return -1;
+
+ target = ++s;
+ ops->target.raw = strdup(target);
+ if (ops->target.raw == NULL)
+ goto out_free_source;
+
+ ops->target.addr = strtoull(target, &endptr, 16);
+ if (endptr == target)
+ goto out_free_target;
+
+ s = strchr(endptr, '<');
+ if (s == NULL)
+ goto out_free_target;
+ endptr = strchr(s + 1, '>');
+ if (endptr == NULL)
+ goto out_free_target;
+
+ *endptr = '\0';
+ *s = ' ';
+ ops->target.name = strdup(s);
+ *s = '<';
+ *endptr = '>';
+ if (ops->target.name == NULL)
+ goto out_free_target;
+
+ return 0;
+
+out_free_target:
+ zfree(&ops->target.raw);
+out_free_source:
+ zfree(&ops->source.raw);
+ return -1;
+}
+
+static int mov__scnprintf(struct ins *ins, char *bf, size_t size,
+ struct ins_operands *ops);
+
+static struct ins_ops arm64_mov_ops = {
+ .parse = arm64_mov__parse,
+ .scnprintf = mov__scnprintf,
+};
+
static struct ins_ops *arm64__associate_instruction_ops(struct arch *arch, const char *name)
{
struct arm64_annotate *arm = arch->priv;
else if (!strcmp(name, "ret"))
ops = &ret_ops;
else
- return NULL;
+ ops = &arm64_mov_ops;
arch__associate_ins_ops(arch, name, ops);
return ops;
return -1;
}
-static int mov__scnprintf(struct ins *ins, char *bf, size_t size,
- struct ins_operands *ops);
static struct ins_ops s390_mov_ops = {
.parse = s390_mov__parse,
return ret;
}
-static int process_feature_event(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session)
+static int process_feature_event(struct perf_session *session,
+ union perf_event *event)
{
if (event->feat.feat_id < HEADER_LAST_FEATURE)
- return perf_event__process_feature(tool, event, session);
+ return perf_event__process_feature(session, event);
return 0;
}
}
#endif
-static int perf_event__repipe_op2_synth(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session
- __maybe_unused)
+static int perf_event__repipe_op2_synth(struct perf_session *session,
+ union perf_event *event)
{
- return perf_event__repipe_synth(tool, event);
+ return perf_event__repipe_synth(session->tool, event);
}
static int perf_event__repipe_attr(struct perf_tool *tool,
return 0;
}
-static s64 perf_event__repipe_auxtrace(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session)
+static s64 perf_event__repipe_auxtrace(struct perf_session *session,
+ union perf_event *event)
{
+ struct perf_tool *tool = session->tool;
struct perf_inject *inject = container_of(tool, struct perf_inject,
tool);
int ret;
#else
static s64
-perf_event__repipe_auxtrace(struct perf_tool *tool __maybe_unused,
- union perf_event *event __maybe_unused,
- struct perf_session *session __maybe_unused)
+perf_event__repipe_auxtrace(struct perf_session *session __maybe_unused,
+ union perf_event *event __maybe_unused)
{
pr_err("AUX area tracing not supported\n");
return -EINVAL;
return err;
}
-static int perf_event__repipe_tracing_data(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session)
+static int perf_event__repipe_tracing_data(struct perf_session *session,
+ union perf_event *event)
{
int err;
- perf_event__repipe_synth(tool, event);
- err = perf_event__process_tracing_data(tool, event, session);
+ perf_event__repipe_synth(session->tool, event);
+ err = perf_event__process_tracing_data(session, event);
return err;
}
-static int perf_event__repipe_id_index(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session)
+static int perf_event__repipe_id_index(struct perf_session *session,
+ union perf_event *event)
{
int err;
- perf_event__repipe_synth(tool, event);
- err = perf_event__process_id_index(tool, event, session);
+ perf_event__repipe_synth(session->tool, event);
+ err = perf_event__process_id_index(session, event);
return err;
}
"kallsyms pathname"),
OPT_BOOLEAN('f', "force", &data.force, "don't complain, do it"),
OPT_CALLBACK_OPTARG(0, "itrace", &inject.itrace_synth_opts,
- NULL, "opts", "Instruction Tracing options",
+ NULL, "opts", "Instruction Tracing options\n"
+ ITRACE_HELP,
itrace_parse_synth_opts),
OPT_BOOLEAN(0, "strip", &inject.strip,
"strip non-synthesized events (use with --itrace)"),
trigger_is_ready(&switch_output_trigger);
}
-static int record__write(struct record *rec, void *bf, size_t size)
+static int record__write(struct record *rec, struct perf_mmap *map __maybe_unused,
+ void *bf, size_t size)
{
- if (perf_data__write(rec->session->data, bf, size) < 0) {
+ struct perf_data_file *file = &rec->session->data->file;
+
+ if (perf_data_file__write(file, bf, size) < 0) {
pr_err("failed to write perf data, error: %m\n");
return -1;
}
struct machine *machine __maybe_unused)
{
struct record *rec = container_of(tool, struct record, tool);
- return record__write(rec, event, event->header.size);
+ return record__write(rec, NULL, event, event->header.size);
}
-static int record__pushfn(void *to, void *bf, size_t size)
+static int record__pushfn(struct perf_mmap *map, void *to, void *bf, size_t size)
{
struct record *rec = to;
rec->samples++;
- return record__write(rec, bf, size);
+ return record__write(rec, map, bf, size);
}
static volatile int done;
#ifdef HAVE_AUXTRACE_SUPPORT
static int record__process_auxtrace(struct perf_tool *tool,
+ struct perf_mmap *map,
union perf_event *event, void *data1,
size_t len1, void *data2, size_t len2)
{
if (padding)
padding = 8 - padding;
- record__write(rec, event, event->header.size);
- record__write(rec, data1, len1);
+ record__write(rec, map, event, event->header.size);
+ record__write(rec, map, data1, len1);
if (len2)
- record__write(rec, data2, len2);
- record__write(rec, &pad, padding);
+ record__write(rec, map, data2, len2);
+ record__write(rec, map, &pad, padding);
return 0;
}
static int record__auxtrace_mmap_read(struct record *rec,
- struct auxtrace_mmap *mm)
+ struct perf_mmap *map)
{
int ret;
- ret = auxtrace_mmap__read(mm, rec->itr, &rec->tool,
+ ret = auxtrace_mmap__read(map, rec->itr, &rec->tool,
record__process_auxtrace);
if (ret < 0)
return ret;
}
static int record__auxtrace_mmap_read_snapshot(struct record *rec,
- struct auxtrace_mmap *mm)
+ struct perf_mmap *map)
{
int ret;
- ret = auxtrace_mmap__read_snapshot(mm, rec->itr, &rec->tool,
+ ret = auxtrace_mmap__read_snapshot(map, rec->itr, &rec->tool,
record__process_auxtrace,
rec->opts.auxtrace_snapshot_size);
if (ret < 0)
int rc = 0;
for (i = 0; i < rec->evlist->nr_mmaps; i++) {
- struct auxtrace_mmap *mm =
- &rec->evlist->mmap[i].auxtrace_mmap;
+ struct perf_mmap *map = &rec->evlist->mmap[i];
- if (!mm->base)
+ if (!map->auxtrace_mmap.base)
continue;
- if (record__auxtrace_mmap_read_snapshot(rec, mm) != 0) {
+ if (record__auxtrace_mmap_read_snapshot(rec, map) != 0) {
rc = -1;
goto out;
}
static inline
int record__auxtrace_mmap_read(struct record *rec __maybe_unused,
- struct auxtrace_mmap *mm __maybe_unused)
+ struct perf_mmap *map __maybe_unused)
{
return 0;
}
return 0;
for (i = 0; i < evlist->nr_mmaps; i++) {
- struct auxtrace_mmap *mm = &maps[i].auxtrace_mmap;
+ struct perf_mmap *map = &maps[i];
- if (maps[i].base) {
- if (perf_mmap__push(&maps[i], rec, record__pushfn) != 0) {
+ if (map->base) {
+ if (perf_mmap__push(map, rec, record__pushfn) != 0) {
rc = -1;
goto out;
}
}
- if (mm->base && !rec->opts.auxtrace_snapshot_mode &&
- record__auxtrace_mmap_read(rec, mm) != 0) {
+ if (map->auxtrace_mmap.base && !rec->opts.auxtrace_snapshot_mode &&
+ record__auxtrace_mmap_read(rec, map) != 0) {
rc = -1;
goto out;
}
* at least one event.
*/
if (bytes_written != rec->bytes_written)
- rc = record__write(rec, &finished_round_event, sizeof(finished_round_event));
+ rc = record__write(rec, NULL, &finished_round_event, sizeof(finished_round_event));
if (overwrite)
perf_evlist__toggle_bkw_mmap(evlist, BKW_MMAP_EMPTY);
* We need to synthesize events first, because some
* features works on top of them (on report side).
*/
- err = perf_event__synthesize_attrs(tool, session,
+ err = perf_event__synthesize_attrs(tool, rec->evlist,
process_synthesized_event);
if (err < 0) {
pr_err("Couldn't synthesize attrs.\n");
perf_evlist__force_leader(evlist);
}
-static int process_feature_event(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session __maybe_unused)
+static int process_feature_event(struct perf_session *session,
+ union perf_event *event)
{
- struct report *rep = container_of(tool, struct report, tool);
+ struct report *rep = container_of(session->tool, struct report, tool);
if (event->feat.feat_id < HEADER_LAST_FEATURE)
- return perf_event__process_feature(tool, event, session);
+ return perf_event__process_feature(session, event);
if (event->feat.feat_id != HEADER_LAST_FEATURE) {
pr_err("failed: wrong feature ID: %" PRIu64 "\n",
.id_index = perf_event__process_id_index,
.auxtrace_info = perf_event__process_auxtrace_info,
.auxtrace = perf_event__process_auxtrace,
+ .event_update = perf_event__process_event_update,
.feature = process_feature_event,
.ordered_events = true,
.ordering_requires_timestamps = true,
OPT_CALLBACK(0, "percentage", NULL, "relative|absolute",
"how to display percentage of filtered entries", parse_filter_percentage),
OPT_CALLBACK_OPTARG(0, "itrace", &itrace_synth_opts, NULL, "opts",
- "Instruction Tracing options",
+ "Instruction Tracing options\n" ITRACE_HELP,
itrace_parse_synth_opts),
OPT_BOOLEAN(0, "full-source-path", &srcline_full_filename,
"Show full source file name path for source lines"),
PERF_OUTPUT_WEIGHT))
return -EINVAL;
- if (PRINT_FIELD(SYM) && !PRINT_FIELD(IP) && !PRINT_FIELD(ADDR)) {
+ if (PRINT_FIELD(SYM) &&
+ !(evsel->attr.sample_type & (PERF_SAMPLE_IP|PERF_SAMPLE_ADDR))) {
pr_err("Display of symbols requested but neither sample IP nor "
- "sample address\nis selected. Hence, no addresses to convert "
+ "sample address\navailable. Hence, no addresses to convert "
"to symbols.\n");
return -EINVAL;
}
"selected.\n");
return -EINVAL;
}
- if (PRINT_FIELD(DSO) && !PRINT_FIELD(IP) && !PRINT_FIELD(ADDR) &&
- !PRINT_FIELD(BRSTACK) && !PRINT_FIELD(BRSTACKSYM) && !PRINT_FIELD(BRSTACKOFF)) {
- pr_err("Display of DSO requested but no address to convert. Select\n"
- "sample IP, sample address, brstack, brstacksym, or brstackoff.\n");
+ if (PRINT_FIELD(DSO) &&
+ !(evsel->attr.sample_type & (PERF_SAMPLE_IP|PERF_SAMPLE_ADDR))) {
+ pr_err("Display of DSO requested but no address to convert.\n");
return -EINVAL;
}
if (PRINT_FIELD(SRCLINE) && !PRINT_FIELD(IP)) {
const char *name = NULL;
static int spacing;
int len = 0;
+ int dlen = 0;
u64 ip = 0;
/*
ip = sample->ip;
}
+ if (PRINT_FIELD(DSO) && !(PRINT_FIELD(IP) || PRINT_FIELD(ADDR))) {
+ dlen += fprintf(fp, "(");
+ dlen += map__fprintf_dsoname(al->map, fp);
+ dlen += fprintf(fp, ")\t");
+ }
+
if (name)
len = fprintf(fp, "%*s%s", (int)depth * 4, "", name);
else if (ip)
if (len < spacing)
len += fprintf(fp, "%*s", spacing - len, "");
- return len;
+ return len + dlen;
}
static int perf_sample__fprintf_insn(struct perf_sample *sample,
{0, NULL}
};
+static const char *sample_flags_to_name(u32 flags)
+{
+ int i;
+
+ for (i = 0; sample_flags[i].name ; i++) {
+ if (sample_flags[i].flags == flags)
+ return sample_flags[i].name;
+ }
+
+ return NULL;
+}
+
static int perf_sample__fprintf_flags(u32 flags, FILE *fp)
{
const char *chars = PERF_IP_FLAG_CHARS;
char str[33];
int i, pos = 0;
- for (i = 0; sample_flags[i].name ; i++) {
- if (sample_flags[i].flags == (flags & ~PERF_IP_FLAG_IN_TX)) {
- name = sample_flags[i].name;
- break;
- }
+ name = sample_flags_to_name(flags & ~PERF_IP_FLAG_IN_TX);
+ if (name)
+ return fprintf(fp, " %-15s%4s ", name, in_tx ? "(x)" : "");
+
+ if (flags & PERF_IP_FLAG_TRACE_BEGIN) {
+ name = sample_flags_to_name(flags & ~(PERF_IP_FLAG_IN_TX | PERF_IP_FLAG_TRACE_BEGIN));
+ if (name)
+ return fprintf(fp, " tr strt %-7s%4s ", name, in_tx ? "(x)" : "");
+ }
+
+ if (flags & PERF_IP_FLAG_TRACE_END) {
+ name = sample_flags_to_name(flags & ~(PERF_IP_FLAG_IN_TX | PERF_IP_FLAG_TRACE_END));
+ if (name)
+ return fprintf(fp, " tr end %-7s%4s ", name, in_tx ? "(x)" : "");
}
for (i = 0; i < n; i++, flags >>= 1) {
}
str[pos] = 0;
- if (name)
- return fprintf(fp, " %-7s%4s ", name, in_tx ? "(x)" : "");
-
- return fprintf(fp, " %-11s ", str);
+ return fprintf(fp, " %-19s ", str);
}
struct printer_data {
FILE *fp;
};
-static void script_print_metric(void *ctx, const char *color,
+static void script_print_metric(struct perf_stat_config *config __maybe_unused,
+ void *ctx, const char *color,
const char *fmt,
const char *unit, double val)
{
fprintf(mctx->fp, " %s\n", unit);
}
-static void script_new_line(void *ctx)
+static void script_new_line(struct perf_stat_config *config __maybe_unused,
+ void *ctx)
{
struct metric_ctx *mctx = ctx;
evsel_script(evsel)->val = val;
if (evsel_script(evsel->leader)->gnum == evsel->leader->nr_members) {
for_each_group_member (ev2, evsel->leader) {
- perf_stat__print_shadow_stats(ev2,
+ perf_stat__print_shadow_stats(&stat_config, ev2,
evsel_script(ev2)->val,
sample->cpu,
&ctx,
output[j].fields &= ~all_output_options[i].field;
else
output[j].fields |= all_output_options[i].field;
+ output[j].user_set = true;
+ output[j].wildcard_set = true;
}
}
} else {
rc = -EINVAL;
goto out;
}
- output[type].fields |= all_output_options[i].field;
+ output[type].user_set = true;
+ output[type].wildcard_set = true;
}
}
}
}
-static int process_stat_round_event(struct perf_tool *tool __maybe_unused,
- union perf_event *event,
- struct perf_session *session)
+static int process_stat_round_event(struct perf_session *session,
+ union perf_event *event)
{
struct stat_round_event *round = &event->stat_round;
struct perf_evsel *counter;
return 0;
}
-static int process_stat_config_event(struct perf_tool *tool __maybe_unused,
- union perf_event *event,
- struct perf_session *session __maybe_unused)
+static int process_stat_config_event(struct perf_session *session __maybe_unused,
+ union perf_event *event)
{
perf_event__read_stat_config(&stat_config, &event->stat_config);
return 0;
}
static
-int process_thread_map_event(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session __maybe_unused)
+int process_thread_map_event(struct perf_session *session,
+ union perf_event *event)
{
+ struct perf_tool *tool = session->tool;
struct perf_script *script = container_of(tool, struct perf_script, tool);
if (script->threads) {
}
static
-int process_cpu_map_event(struct perf_tool *tool __maybe_unused,
- union perf_event *event,
- struct perf_session *session __maybe_unused)
+int process_cpu_map_event(struct perf_session *session,
+ union perf_event *event)
{
+ struct perf_tool *tool = session->tool;
struct perf_script *script = container_of(tool, struct perf_script, tool);
if (script->cpus) {
return set_maps(script);
}
-static int process_feature_event(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session)
+static int process_feature_event(struct perf_session *session,
+ union perf_event *event)
{
if (event->feat.feat_id < HEADER_LAST_FEATURE)
- return perf_event__process_feature(tool, event, session);
+ return perf_event__process_feature(session, event);
return 0;
}
#ifdef HAVE_AUXTRACE_SUPPORT
-static int perf_script__process_auxtrace_info(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session)
+static int perf_script__process_auxtrace_info(struct perf_session *session,
+ union perf_event *event)
{
- int ret = perf_event__process_auxtrace_info(tool, event, session);
+ struct perf_tool *tool = session->tool;
+
+ int ret = perf_event__process_auxtrace_info(session, event);
if (ret == 0) {
struct perf_script *script = container_of(tool, struct perf_script, tool);
OPT_BOOLEAN(0, "ns", &nanosecs,
"Use 9 decimal places when displaying time"),
OPT_CALLBACK_OPTARG(0, "itrace", &itrace_synth_opts, NULL, "opts",
- "Instruction Tracing options",
+ "Instruction Tracing options\n" ITRACE_HELP,
itrace_parse_synth_opts),
OPT_BOOLEAN(0, "full-source-path", &srcline_full_filename,
"Show full source file name path for source lines"),
#include "sane_ctype.h"
#define DEFAULT_SEPARATOR " "
-#define CNTR_NOT_SUPPORTED "<not supported>"
-#define CNTR_NOT_COUNTED "<not counted>"
#define FREEZE_ON_SMI_PATH "devices/cpu/freeze_on_smi"
static void print_counters(struct timespec *ts, int argc, const char **argv);
static struct perf_evlist *evsel_list;
-static struct rblist metric_events;
-
static struct target target = {
.uid = UINT_MAX,
};
-typedef int (*aggr_get_id_t)(struct cpu_map *m, int cpu);
-
#define METRIC_ONLY_LEN 20
-static int run_count = 1;
-static bool no_inherit = false;
static volatile pid_t child_pid = -1;
-static bool null_run = false;
static int detailed_run = 0;
static bool transaction_run;
static bool topdown_run = false;
static bool smi_cost = false;
static bool smi_reset = false;
-static bool big_num = true;
static int big_num_opt = -1;
-static const char *csv_sep = NULL;
-static bool csv_output = false;
static bool group = false;
static const char *pre_cmd = NULL;
static const char *post_cmd = NULL;
static bool sync_run = false;
-static unsigned int initial_delay = 0;
-static unsigned int unit_width = 4; /* strlen("unit") */
static bool forever = false;
-static bool metric_only = false;
static bool force_metric_only = false;
-static bool no_merge = false;
-static bool walltime_run_table = false;
static struct timespec ref_time;
-static struct cpu_map *aggr_map;
-static aggr_get_id_t aggr_get_id;
static bool append_file;
static bool interval_count;
-static bool interval_clear;
static const char *output_name;
static int output_fd;
-static int print_free_counters_hint;
-static int print_mixed_hw_group_error;
-static u64 *walltime_run;
-static bool ru_display = false;
-static struct rusage ru_data;
-static unsigned int metric_only_len = METRIC_ONLY_LEN;
struct perf_stat {
bool record;
static volatile int done = 0;
static struct perf_stat_config stat_config = {
- .aggr_mode = AGGR_GLOBAL,
- .scale = true,
+ .aggr_mode = AGGR_GLOBAL,
+ .scale = true,
+ .unit_width = 4, /* strlen("unit") */
+ .run_count = 1,
+ .metric_only_len = METRIC_ONLY_LEN,
+ .walltime_nsecs_stats = &walltime_nsecs_stats,
+ .big_num = true,
};
-static bool is_duration_time(struct perf_evsel *evsel)
-{
- return !strcmp(evsel->name, "duration_time");
-}
-
static inline void diff_timespec(struct timespec *r, struct timespec *a,
struct timespec *b)
{
perf_stat__reset_shadow_per_stat(&stat_config.stats[i]);
}
-static int create_perf_stat_counter(struct perf_evsel *evsel)
-{
- struct perf_event_attr *attr = &evsel->attr;
- struct perf_evsel *leader = evsel->leader;
-
- if (stat_config.scale) {
- attr->read_format = PERF_FORMAT_TOTAL_TIME_ENABLED |
- PERF_FORMAT_TOTAL_TIME_RUNNING;
- }
-
- /*
- * The event is part of non trivial group, let's enable
- * the group read (for leader) and ID retrieval for all
- * members.
- */
- if (leader->nr_members > 1)
- attr->read_format |= PERF_FORMAT_ID|PERF_FORMAT_GROUP;
-
- attr->inherit = !no_inherit;
-
- /*
- * Some events get initialized with sample_(period/type) set,
- * like tracepoints. Clear it up for counting.
- */
- attr->sample_period = 0;
-
- /*
- * But set sample_type to PERF_SAMPLE_IDENTIFIER, which should be harmless
- * while avoiding that older tools show confusing messages.
- *
- * However for pipe sessions we need to keep it zero,
- * because script's perf_evsel__check_attr is triggered
- * by attr->sample_type != 0, and we can't run it on
- * stat sessions.
- */
- if (!(STAT_RECORD && perf_stat.data.is_pipe))
- attr->sample_type = PERF_SAMPLE_IDENTIFIER;
-
- /*
- * Disabling all counters initially, they will be enabled
- * either manually by us or by kernel via enable_on_exec
- * set later.
- */
- if (perf_evsel__is_group_leader(evsel)) {
- attr->disabled = 1;
-
- /*
- * In case of initial_delay we enable tracee
- * events manually.
- */
- if (target__none(&target) && !initial_delay)
- attr->enable_on_exec = 1;
- }
-
- if (target__has_cpu(&target) && !target__has_per_thread(&target))
- return perf_evsel__open_per_cpu(evsel, perf_evsel__cpus(evsel));
-
- return perf_evsel__open_per_thread(evsel, evsel_list->threads);
-}
-
static int process_synthesized_event(struct perf_tool *tool __maybe_unused,
union perf_event *event,
struct perf_sample *sample __maybe_unused,
static void enable_counters(void)
{
- if (initial_delay)
- usleep(initial_delay * USEC_PER_MSEC);
+ if (stat_config.initial_delay)
+ usleep(stat_config.initial_delay * USEC_PER_MSEC);
/*
* We need to enable counters only if:
* - we don't have tracee (attaching to task or cpu)
* - we have initial delay configured
*/
- if (!target__none(&target) || initial_delay)
+ if (!target__none(&target) || stat_config.initial_delay)
perf_evlist__enable(evsel_list);
}
workload_exec_errno = info->si_value.sival_int;
}
-static int perf_stat_synthesize_config(bool is_pipe)
-{
- int err;
-
- if (is_pipe) {
- err = perf_event__synthesize_attrs(NULL, perf_stat.session,
- process_synthesized_event);
- if (err < 0) {
- pr_err("Couldn't synthesize attrs.\n");
- return err;
- }
- }
-
- err = perf_event__synthesize_extra_attr(NULL,
- evsel_list,
- process_synthesized_event,
- is_pipe);
-
- err = perf_event__synthesize_thread_map2(NULL, evsel_list->threads,
- process_synthesized_event,
- NULL);
- if (err < 0) {
- pr_err("Couldn't synthesize thread map.\n");
- return err;
- }
-
- err = perf_event__synthesize_cpu_map(NULL, evsel_list->cpus,
- process_synthesized_event, NULL);
- if (err < 0) {
- pr_err("Couldn't synthesize thread map.\n");
- return err;
- }
-
- err = perf_event__synthesize_stat_config(NULL, &stat_config,
- process_synthesized_event, NULL);
- if (err < 0) {
- pr_err("Couldn't synthesize config.\n");
- return err;
- }
-
- return 0;
-}
-
-#define FD(e, x, y) (*(int *)xyarray__entry(e->fd, x, y))
-
-static int __store_counter_ids(struct perf_evsel *counter)
-{
- int cpu, thread;
-
- for (cpu = 0; cpu < xyarray__max_x(counter->fd); cpu++) {
- for (thread = 0; thread < xyarray__max_y(counter->fd);
- thread++) {
- int fd = FD(counter, cpu, thread);
-
- if (perf_evlist__id_add_fd(evsel_list, counter,
- cpu, thread, fd) < 0)
- return -1;
- }
- }
-
- return 0;
-}
-
-static int store_counter_ids(struct perf_evsel *counter)
-{
- struct cpu_map *cpus = counter->cpus;
- struct thread_map *threads = counter->threads;
-
- if (perf_evsel__alloc_id(counter, cpus->nr, threads->nr))
- return -ENOMEM;
-
- return __store_counter_ids(counter);
-}
-
static bool perf_evsel__should_store_id(struct perf_evsel *counter)
{
return STAT_RECORD || counter->attr.read_format & PERF_FORMAT_ID;
evlist__for_each_entry(evsel_list, counter) {
try_again:
- if (create_perf_stat_counter(counter) < 0) {
+ if (create_perf_stat_counter(counter, &stat_config, &target) < 0) {
/* Weak group failed. Reset the group. */
if ((errno == EINVAL || errno == EBADF) &&
counter->supported = true;
l = strlen(counter->unit);
- if (l > unit_width)
- unit_width = l;
+ if (l > stat_config.unit_width)
+ stat_config.unit_width = l;
if (perf_evsel__should_store_id(counter) &&
- store_counter_ids(counter))
+ perf_evsel__store_ids(counter, evsel_list))
return -1;
}
if (err < 0)
return err;
- err = perf_stat_synthesize_config(is_pipe);
+ err = perf_stat_synthesize_config(&stat_config, NULL, evsel_list,
+ process_synthesized_event, is_pipe);
if (err < 0)
return err;
}
break;
}
}
- wait4(child_pid, &status, 0, &ru_data);
+ wait4(child_pid, &status, 0, &stat_config.ru_data);
if (workload_exec_errno) {
const char *emsg = str_error_r(workload_exec_errno, msg, sizeof(msg));
t1 = rdclock();
- if (walltime_run_table)
- walltime_run[run_idx] = t1 - t0;
+ if (stat_config.walltime_run_table)
+ stat_config.walltime_run[run_idx] = t1 - t0;
update_stats(&walltime_nsecs_stats, t1 - t0);
return ret;
}
-static void print_running(u64 run, u64 ena)
-{
- if (csv_output) {
- fprintf(stat_config.output, "%s%" PRIu64 "%s%.2f",
- csv_sep,
- run,
- csv_sep,
- ena ? 100.0 * run / ena : 100.0);
- } else if (run != ena) {
- fprintf(stat_config.output, " (%.2f%%)", 100.0 * run / ena);
- }
-}
-
-static void print_noise_pct(double total, double avg)
-{
- double pct = rel_stddev_stats(total, avg);
-
- if (csv_output)
- fprintf(stat_config.output, "%s%.2f%%", csv_sep, pct);
- else if (pct)
- fprintf(stat_config.output, " ( +-%6.2f%% )", pct);
-}
-
-static void print_noise(struct perf_evsel *evsel, double avg)
-{
- struct perf_stat_evsel *ps;
-
- if (run_count == 1)
- return;
-
- ps = evsel->stats;
- print_noise_pct(stddev_stats(&ps->res_stats[0]), avg);
-}
-
-static void aggr_printout(struct perf_evsel *evsel, int id, int nr)
-{
- switch (stat_config.aggr_mode) {
- case AGGR_CORE:
- fprintf(stat_config.output, "S%d-C%*d%s%*d%s",
- cpu_map__id_to_socket(id),
- csv_output ? 0 : -8,
- cpu_map__id_to_cpu(id),
- csv_sep,
- csv_output ? 0 : 4,
- nr,
- csv_sep);
- break;
- case AGGR_SOCKET:
- fprintf(stat_config.output, "S%*d%s%*d%s",
- csv_output ? 0 : -5,
- id,
- csv_sep,
- csv_output ? 0 : 4,
- nr,
- csv_sep);
- break;
- case AGGR_NONE:
- fprintf(stat_config.output, "CPU%*d%s",
- csv_output ? 0 : -4,
- perf_evsel__cpus(evsel)->map[id], csv_sep);
- break;
- case AGGR_THREAD:
- fprintf(stat_config.output, "%*s-%*d%s",
- csv_output ? 0 : 16,
- thread_map__comm(evsel->threads, id),
- csv_output ? 0 : -8,
- thread_map__pid(evsel->threads, id),
- csv_sep);
- break;
- case AGGR_GLOBAL:
- case AGGR_UNSET:
- default:
- break;
- }
-}
-
-struct outstate {
- FILE *fh;
- bool newline;
- const char *prefix;
- int nfields;
- int id, nr;
- struct perf_evsel *evsel;
-};
-
-#define METRIC_LEN 35
-
-static void new_line_std(void *ctx)
-{
- struct outstate *os = ctx;
-
- os->newline = true;
-}
-
-static void do_new_line_std(struct outstate *os)
-{
- fputc('\n', os->fh);
- fputs(os->prefix, os->fh);
- aggr_printout(os->evsel, os->id, os->nr);
- if (stat_config.aggr_mode == AGGR_NONE)
- fprintf(os->fh, " ");
- fprintf(os->fh, " ");
-}
-
-static void print_metric_std(void *ctx, const char *color, const char *fmt,
- const char *unit, double val)
-{
- struct outstate *os = ctx;
- FILE *out = os->fh;
- int n;
- bool newline = os->newline;
-
- os->newline = false;
-
- if (unit == NULL || fmt == NULL) {
- fprintf(out, "%-*s", METRIC_LEN, "");
- return;
- }
-
- if (newline)
- do_new_line_std(os);
-
- n = fprintf(out, " # ");
- if (color)
- n += color_fprintf(out, color, fmt, val);
- else
- n += fprintf(out, fmt, val);
- fprintf(out, " %-*s", METRIC_LEN - n - 1, unit);
-}
-
-static void new_line_csv(void *ctx)
-{
- struct outstate *os = ctx;
- int i;
-
- fputc('\n', os->fh);
- if (os->prefix)
- fprintf(os->fh, "%s%s", os->prefix, csv_sep);
- aggr_printout(os->evsel, os->id, os->nr);
- for (i = 0; i < os->nfields; i++)
- fputs(csv_sep, os->fh);
-}
-
-static void print_metric_csv(void *ctx,
- const char *color __maybe_unused,
- const char *fmt, const char *unit, double val)
-{
- struct outstate *os = ctx;
- FILE *out = os->fh;
- char buf[64], *vals, *ends;
-
- if (unit == NULL || fmt == NULL) {
- fprintf(out, "%s%s", csv_sep, csv_sep);
- return;
- }
- snprintf(buf, sizeof(buf), fmt, val);
- ends = vals = ltrim(buf);
- while (isdigit(*ends) || *ends == '.')
- ends++;
- *ends = 0;
- while (isspace(*unit))
- unit++;
- fprintf(out, "%s%s%s%s", csv_sep, vals, csv_sep, unit);
-}
-
-/* Filter out some columns that don't work well in metrics only mode */
-
-static bool valid_only_metric(const char *unit)
-{
- if (!unit)
- return false;
- if (strstr(unit, "/sec") ||
- strstr(unit, "hz") ||
- strstr(unit, "Hz") ||
- strstr(unit, "CPUs utilized"))
- return false;
- return true;
-}
-
-static const char *fixunit(char *buf, struct perf_evsel *evsel,
- const char *unit)
-{
- if (!strncmp(unit, "of all", 6)) {
- snprintf(buf, 1024, "%s %s", perf_evsel__name(evsel),
- unit);
- return buf;
- }
- return unit;
-}
-
-static void print_metric_only(void *ctx, const char *color, const char *fmt,
- const char *unit, double val)
-{
- struct outstate *os = ctx;
- FILE *out = os->fh;
- char buf[1024], str[1024];
- unsigned mlen = metric_only_len;
-
- if (!valid_only_metric(unit))
- return;
- unit = fixunit(buf, os->evsel, unit);
- if (mlen < strlen(unit))
- mlen = strlen(unit) + 1;
-
- if (color)
- mlen += strlen(color) + sizeof(PERF_COLOR_RESET) - 1;
-
- color_snprintf(str, sizeof(str), color ?: "", fmt, val);
- fprintf(out, "%*s ", mlen, str);
-}
-
-static void print_metric_only_csv(void *ctx, const char *color __maybe_unused,
- const char *fmt,
- const char *unit, double val)
-{
- struct outstate *os = ctx;
- FILE *out = os->fh;
- char buf[64], *vals, *ends;
- char tbuf[1024];
-
- if (!valid_only_metric(unit))
- return;
- unit = fixunit(tbuf, os->evsel, unit);
- snprintf(buf, sizeof buf, fmt, val);
- ends = vals = ltrim(buf);
- while (isdigit(*ends) || *ends == '.')
- ends++;
- *ends = 0;
- fprintf(out, "%s%s", vals, csv_sep);
-}
-
-static void new_line_metric(void *ctx __maybe_unused)
-{
-}
-
-static void print_metric_header(void *ctx, const char *color __maybe_unused,
- const char *fmt __maybe_unused,
- const char *unit, double val __maybe_unused)
-{
- struct outstate *os = ctx;
- char tbuf[1024];
-
- if (!valid_only_metric(unit))
- return;
- unit = fixunit(tbuf, os->evsel, unit);
- if (csv_output)
- fprintf(os->fh, "%s%s", unit, csv_sep);
- else
- fprintf(os->fh, "%*s ", metric_only_len, unit);
-}
-
-static int first_shadow_cpu(struct perf_evsel *evsel, int id)
-{
- int i;
-
- if (!aggr_get_id)
- return 0;
-
- if (stat_config.aggr_mode == AGGR_NONE)
- return id;
-
- if (stat_config.aggr_mode == AGGR_GLOBAL)
- return 0;
-
- for (i = 0; i < perf_evsel__nr_cpus(evsel); i++) {
- int cpu2 = perf_evsel__cpus(evsel)->map[i];
-
- if (aggr_get_id(evsel_list->cpus, cpu2) == id)
- return cpu2;
- }
- return 0;
-}
-
-static void abs_printout(int id, int nr, struct perf_evsel *evsel, double avg)
-{
- FILE *output = stat_config.output;
- double sc = evsel->scale;
- const char *fmt;
-
- if (csv_output) {
- fmt = floor(sc) != sc ? "%.2f%s" : "%.0f%s";
- } else {
- if (big_num)
- fmt = floor(sc) != sc ? "%'18.2f%s" : "%'18.0f%s";
- else
- fmt = floor(sc) != sc ? "%18.2f%s" : "%18.0f%s";
- }
-
- aggr_printout(evsel, id, nr);
-
- fprintf(output, fmt, avg, csv_sep);
-
- if (evsel->unit)
- fprintf(output, "%-*s%s",
- csv_output ? 0 : unit_width,
- evsel->unit, csv_sep);
-
- fprintf(output, "%-*s", csv_output ? 0 : 25, perf_evsel__name(evsel));
-
- if (evsel->cgrp)
- fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
-}
-
-static bool is_mixed_hw_group(struct perf_evsel *counter)
-{
- struct perf_evlist *evlist = counter->evlist;
- u32 pmu_type = counter->attr.type;
- struct perf_evsel *pos;
-
- if (counter->nr_members < 2)
- return false;
-
- evlist__for_each_entry(evlist, pos) {
- /* software events can be part of any hardware group */
- if (pos->attr.type == PERF_TYPE_SOFTWARE)
- continue;
- if (pmu_type == PERF_TYPE_SOFTWARE) {
- pmu_type = pos->attr.type;
- continue;
- }
- if (pmu_type != pos->attr.type)
- return true;
- }
-
- return false;
-}
-
-static void printout(int id, int nr, struct perf_evsel *counter, double uval,
- char *prefix, u64 run, u64 ena, double noise,
- struct runtime_stat *st)
-{
- struct perf_stat_output_ctx out;
- struct outstate os = {
- .fh = stat_config.output,
- .prefix = prefix ? prefix : "",
- .id = id,
- .nr = nr,
- .evsel = counter,
- };
- print_metric_t pm = print_metric_std;
- void (*nl)(void *);
-
- if (metric_only) {
- nl = new_line_metric;
- if (csv_output)
- pm = print_metric_only_csv;
- else
- pm = print_metric_only;
- } else
- nl = new_line_std;
-
- if (csv_output && !metric_only) {
- static int aggr_fields[] = {
- [AGGR_GLOBAL] = 0,
- [AGGR_THREAD] = 1,
- [AGGR_NONE] = 1,
- [AGGR_SOCKET] = 2,
- [AGGR_CORE] = 2,
- };
-
- pm = print_metric_csv;
- nl = new_line_csv;
- os.nfields = 3;
- os.nfields += aggr_fields[stat_config.aggr_mode];
- if (counter->cgrp)
- os.nfields++;
- }
- if (run == 0 || ena == 0 || counter->counts->scaled == -1) {
- if (metric_only) {
- pm(&os, NULL, "", "", 0);
- return;
- }
- aggr_printout(counter, id, nr);
-
- fprintf(stat_config.output, "%*s%s",
- csv_output ? 0 : 18,
- counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
- csv_sep);
-
- if (counter->supported) {
- print_free_counters_hint = 1;
- if (is_mixed_hw_group(counter))
- print_mixed_hw_group_error = 1;
- }
-
- fprintf(stat_config.output, "%-*s%s",
- csv_output ? 0 : unit_width,
- counter->unit, csv_sep);
-
- fprintf(stat_config.output, "%*s",
- csv_output ? 0 : -25,
- perf_evsel__name(counter));
-
- if (counter->cgrp)
- fprintf(stat_config.output, "%s%s",
- csv_sep, counter->cgrp->name);
-
- if (!csv_output)
- pm(&os, NULL, NULL, "", 0);
- print_noise(counter, noise);
- print_running(run, ena);
- if (csv_output)
- pm(&os, NULL, NULL, "", 0);
- return;
- }
-
- if (!metric_only)
- abs_printout(id, nr, counter, uval);
-
- out.print_metric = pm;
- out.new_line = nl;
- out.ctx = &os;
- out.force_header = false;
-
- if (csv_output && !metric_only) {
- print_noise(counter, noise);
- print_running(run, ena);
- }
-
- perf_stat__print_shadow_stats(counter, uval,
- first_shadow_cpu(counter, id),
- &out, &metric_events, st);
- if (!csv_output && !metric_only) {
- print_noise(counter, noise);
- print_running(run, ena);
- }
-}
-
-static void aggr_update_shadow(void)
-{
- int cpu, s2, id, s;
- u64 val;
- struct perf_evsel *counter;
-
- for (s = 0; s < aggr_map->nr; s++) {
- id = aggr_map->map[s];
- evlist__for_each_entry(evsel_list, counter) {
- val = 0;
- for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
- s2 = aggr_get_id(evsel_list->cpus, cpu);
- if (s2 != id)
- continue;
- val += perf_counts(counter->counts, cpu, 0)->val;
- }
- perf_stat__update_shadow_stats(counter, val,
- first_shadow_cpu(counter, id),
- &rt_stat);
- }
- }
-}
-
-static void uniquify_event_name(struct perf_evsel *counter)
-{
- char *new_name;
- char *config;
-
- if (counter->uniquified_name ||
- !counter->pmu_name || !strncmp(counter->name, counter->pmu_name,
- strlen(counter->pmu_name)))
- return;
-
- config = strchr(counter->name, '/');
- if (config) {
- if (asprintf(&new_name,
- "%s%s", counter->pmu_name, config) > 0) {
- free(counter->name);
- counter->name = new_name;
- }
- } else {
- if (asprintf(&new_name,
- "%s [%s]", counter->name, counter->pmu_name) > 0) {
- free(counter->name);
- counter->name = new_name;
- }
- }
-
- counter->uniquified_name = true;
-}
-
-static void collect_all_aliases(struct perf_evsel *counter,
- void (*cb)(struct perf_evsel *counter, void *data,
- bool first),
- void *data)
-{
- struct perf_evsel *alias;
-
- alias = list_prepare_entry(counter, &(evsel_list->entries), node);
- list_for_each_entry_continue (alias, &evsel_list->entries, node) {
- if (strcmp(perf_evsel__name(alias), perf_evsel__name(counter)) ||
- alias->scale != counter->scale ||
- alias->cgrp != counter->cgrp ||
- strcmp(alias->unit, counter->unit) ||
- perf_evsel__is_clock(alias) != perf_evsel__is_clock(counter))
- break;
- alias->merged_stat = true;
- cb(alias, data, false);
- }
-}
-
-static bool collect_data(struct perf_evsel *counter,
- void (*cb)(struct perf_evsel *counter, void *data,
- bool first),
- void *data)
-{
- if (counter->merged_stat)
- return false;
- cb(counter, data, true);
- if (no_merge)
- uniquify_event_name(counter);
- else if (counter->auto_merge_stats)
- collect_all_aliases(counter, cb, data);
- return true;
-}
-
-struct aggr_data {
- u64 ena, run, val;
- int id;
- int nr;
- int cpu;
-};
-
-static void aggr_cb(struct perf_evsel *counter, void *data, bool first)
-{
- struct aggr_data *ad = data;
- int cpu, s2;
-
- for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
- struct perf_counts_values *counts;
-
- s2 = aggr_get_id(perf_evsel__cpus(counter), cpu);
- if (s2 != ad->id)
- continue;
- if (first)
- ad->nr++;
- counts = perf_counts(counter->counts, cpu, 0);
- /*
- * When any result is bad, make them all to give
- * consistent output in interval mode.
- */
- if (counts->ena == 0 || counts->run == 0 ||
- counter->counts->scaled == -1) {
- ad->ena = 0;
- ad->run = 0;
- break;
- }
- ad->val += counts->val;
- ad->ena += counts->ena;
- ad->run += counts->run;
- }
-}
-
-static void print_aggr(char *prefix)
-{
- FILE *output = stat_config.output;
- struct perf_evsel *counter;
- int s, id, nr;
- double uval;
- u64 ena, run, val;
- bool first;
-
- if (!(aggr_map || aggr_get_id))
- return;
-
- aggr_update_shadow();
-
- /*
- * With metric_only everything is on a single line.
- * Without each counter has its own line.
- */
- for (s = 0; s < aggr_map->nr; s++) {
- struct aggr_data ad;
- if (prefix && metric_only)
- fprintf(output, "%s", prefix);
-
- ad.id = id = aggr_map->map[s];
- first = true;
- evlist__for_each_entry(evsel_list, counter) {
- if (is_duration_time(counter))
- continue;
-
- ad.val = ad.ena = ad.run = 0;
- ad.nr = 0;
- if (!collect_data(counter, aggr_cb, &ad))
- continue;
- nr = ad.nr;
- ena = ad.ena;
- run = ad.run;
- val = ad.val;
- if (first && metric_only) {
- first = false;
- aggr_printout(counter, id, nr);
- }
- if (prefix && !metric_only)
- fprintf(output, "%s", prefix);
-
- uval = val * counter->scale;
- printout(id, nr, counter, uval, prefix, run, ena, 1.0,
- &rt_stat);
- if (!metric_only)
- fputc('\n', output);
- }
- if (metric_only)
- fputc('\n', output);
- }
-}
-
-static int cmp_val(const void *a, const void *b)
-{
- return ((struct perf_aggr_thread_value *)b)->val -
- ((struct perf_aggr_thread_value *)a)->val;
-}
-
-static struct perf_aggr_thread_value *sort_aggr_thread(
- struct perf_evsel *counter,
- int nthreads, int ncpus,
- int *ret)
-{
- int cpu, thread, i = 0;
- double uval;
- struct perf_aggr_thread_value *buf;
-
- buf = calloc(nthreads, sizeof(struct perf_aggr_thread_value));
- if (!buf)
- return NULL;
-
- for (thread = 0; thread < nthreads; thread++) {
- u64 ena = 0, run = 0, val = 0;
-
- for (cpu = 0; cpu < ncpus; cpu++) {
- val += perf_counts(counter->counts, cpu, thread)->val;
- ena += perf_counts(counter->counts, cpu, thread)->ena;
- run += perf_counts(counter->counts, cpu, thread)->run;
- }
-
- uval = val * counter->scale;
-
- /*
- * Skip value 0 when enabling --per-thread globally,
- * otherwise too many 0 output.
- */
- if (uval == 0.0 && target__has_per_thread(&target))
- continue;
-
- buf[i].counter = counter;
- buf[i].id = thread;
- buf[i].uval = uval;
- buf[i].val = val;
- buf[i].run = run;
- buf[i].ena = ena;
- i++;
- }
-
- qsort(buf, i, sizeof(struct perf_aggr_thread_value), cmp_val);
-
- if (ret)
- *ret = i;
-
- return buf;
-}
-
-static void print_aggr_thread(struct perf_evsel *counter, char *prefix)
-{
- FILE *output = stat_config.output;
- int nthreads = thread_map__nr(counter->threads);
- int ncpus = cpu_map__nr(counter->cpus);
- int thread, sorted_threads, id;
- struct perf_aggr_thread_value *buf;
-
- buf = sort_aggr_thread(counter, nthreads, ncpus, &sorted_threads);
- if (!buf) {
- perror("cannot sort aggr thread");
- return;
- }
-
- for (thread = 0; thread < sorted_threads; thread++) {
- if (prefix)
- fprintf(output, "%s", prefix);
-
- id = buf[thread].id;
- if (stat_config.stats)
- printout(id, 0, buf[thread].counter, buf[thread].uval,
- prefix, buf[thread].run, buf[thread].ena, 1.0,
- &stat_config.stats[id]);
- else
- printout(id, 0, buf[thread].counter, buf[thread].uval,
- prefix, buf[thread].run, buf[thread].ena, 1.0,
- &rt_stat);
- fputc('\n', output);
- }
-
- free(buf);
-}
-
-struct caggr_data {
- double avg, avg_enabled, avg_running;
-};
-
-static void counter_aggr_cb(struct perf_evsel *counter, void *data,
- bool first __maybe_unused)
-{
- struct caggr_data *cd = data;
- struct perf_stat_evsel *ps = counter->stats;
-
- cd->avg += avg_stats(&ps->res_stats[0]);
- cd->avg_enabled += avg_stats(&ps->res_stats[1]);
- cd->avg_running += avg_stats(&ps->res_stats[2]);
-}
-
-/*
- * Print out the results of a single counter:
- * aggregated counts in system-wide mode
- */
-static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
-{
- FILE *output = stat_config.output;
- double uval;
- struct caggr_data cd = { .avg = 0.0 };
-
- if (!collect_data(counter, counter_aggr_cb, &cd))
- return;
-
- if (prefix && !metric_only)
- fprintf(output, "%s", prefix);
-
- uval = cd.avg * counter->scale;
- printout(-1, 0, counter, uval, prefix, cd.avg_running, cd.avg_enabled,
- cd.avg, &rt_stat);
- if (!metric_only)
- fprintf(output, "\n");
-}
-
-static void counter_cb(struct perf_evsel *counter, void *data,
- bool first __maybe_unused)
-{
- struct aggr_data *ad = data;
-
- ad->val += perf_counts(counter->counts, ad->cpu, 0)->val;
- ad->ena += perf_counts(counter->counts, ad->cpu, 0)->ena;
- ad->run += perf_counts(counter->counts, ad->cpu, 0)->run;
-}
-
-/*
- * Print out the results of a single counter:
- * does not use aggregated count in system-wide
- */
-static void print_counter(struct perf_evsel *counter, char *prefix)
-{
- FILE *output = stat_config.output;
- u64 ena, run, val;
- double uval;
- int cpu;
-
- for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
- struct aggr_data ad = { .cpu = cpu };
-
- if (!collect_data(counter, counter_cb, &ad))
- return;
- val = ad.val;
- ena = ad.ena;
- run = ad.run;
-
- if (prefix)
- fprintf(output, "%s", prefix);
-
- uval = val * counter->scale;
- printout(cpu, 0, counter, uval, prefix, run, ena, 1.0,
- &rt_stat);
-
- fputc('\n', output);
- }
-}
-
-static void print_no_aggr_metric(char *prefix)
-{
- int cpu;
- int nrcpus = 0;
- struct perf_evsel *counter;
- u64 ena, run, val;
- double uval;
-
- nrcpus = evsel_list->cpus->nr;
- for (cpu = 0; cpu < nrcpus; cpu++) {
- bool first = true;
-
- if (prefix)
- fputs(prefix, stat_config.output);
- evlist__for_each_entry(evsel_list, counter) {
- if (is_duration_time(counter))
- continue;
- if (first) {
- aggr_printout(counter, cpu, 0);
- first = false;
- }
- val = perf_counts(counter->counts, cpu, 0)->val;
- ena = perf_counts(counter->counts, cpu, 0)->ena;
- run = perf_counts(counter->counts, cpu, 0)->run;
-
- uval = val * counter->scale;
- printout(cpu, 0, counter, uval, prefix, run, ena, 1.0,
- &rt_stat);
- }
- fputc('\n', stat_config.output);
- }
-}
-
-static int aggr_header_lens[] = {
- [AGGR_CORE] = 18,
- [AGGR_SOCKET] = 12,
- [AGGR_NONE] = 6,
- [AGGR_THREAD] = 24,
- [AGGR_GLOBAL] = 0,
-};
-
-static const char *aggr_header_csv[] = {
- [AGGR_CORE] = "core,cpus,",
- [AGGR_SOCKET] = "socket,cpus",
- [AGGR_NONE] = "cpu,",
- [AGGR_THREAD] = "comm-pid,",
- [AGGR_GLOBAL] = ""
-};
-
-static void print_metric_headers(const char *prefix, bool no_indent)
-{
- struct perf_stat_output_ctx out;
- struct perf_evsel *counter;
- struct outstate os = {
- .fh = stat_config.output
- };
-
- if (prefix)
- fprintf(stat_config.output, "%s", prefix);
-
- if (!csv_output && !no_indent)
- fprintf(stat_config.output, "%*s",
- aggr_header_lens[stat_config.aggr_mode], "");
- if (csv_output) {
- if (stat_config.interval)
- fputs("time,", stat_config.output);
- fputs(aggr_header_csv[stat_config.aggr_mode],
- stat_config.output);
- }
-
- /* Print metrics headers only */
- evlist__for_each_entry(evsel_list, counter) {
- if (is_duration_time(counter))
- continue;
- os.evsel = counter;
- out.ctx = &os;
- out.print_metric = print_metric_header;
- out.new_line = new_line_metric;
- out.force_header = true;
- os.evsel = counter;
- perf_stat__print_shadow_stats(counter, 0,
- 0,
- &out,
- &metric_events,
- &rt_stat);
- }
- fputc('\n', stat_config.output);
-}
-
-static void print_interval(char *prefix, struct timespec *ts)
-{
- FILE *output = stat_config.output;
- static int num_print_interval;
-
- if (interval_clear)
- puts(CONSOLE_CLEAR);
-
- sprintf(prefix, "%6lu.%09lu%s", ts->tv_sec, ts->tv_nsec, csv_sep);
-
- if ((num_print_interval == 0 && !csv_output) || interval_clear) {
- switch (stat_config.aggr_mode) {
- case AGGR_SOCKET:
- fprintf(output, "# time socket cpus");
- if (!metric_only)
- fprintf(output, " counts %*s events\n", unit_width, "unit");
- break;
- case AGGR_CORE:
- fprintf(output, "# time core cpus");
- if (!metric_only)
- fprintf(output, " counts %*s events\n", unit_width, "unit");
- break;
- case AGGR_NONE:
- fprintf(output, "# time CPU ");
- if (!metric_only)
- fprintf(output, " counts %*s events\n", unit_width, "unit");
- break;
- case AGGR_THREAD:
- fprintf(output, "# time comm-pid");
- if (!metric_only)
- fprintf(output, " counts %*s events\n", unit_width, "unit");
- break;
- case AGGR_GLOBAL:
- default:
- fprintf(output, "# time");
- if (!metric_only)
- fprintf(output, " counts %*s events\n", unit_width, "unit");
- case AGGR_UNSET:
- break;
- }
- }
-
- if ((num_print_interval == 0 || interval_clear) && metric_only)
- print_metric_headers(" ", true);
- if (++num_print_interval == 25)
- num_print_interval = 0;
-}
-
-static void print_header(int argc, const char **argv)
-{
- FILE *output = stat_config.output;
- int i;
-
- fflush(stdout);
-
- if (!csv_output) {
- fprintf(output, "\n");
- fprintf(output, " Performance counter stats for ");
- if (target.system_wide)
- fprintf(output, "\'system wide");
- else if (target.cpu_list)
- fprintf(output, "\'CPU(s) %s", target.cpu_list);
- else if (!target__has_task(&target)) {
- fprintf(output, "\'%s", argv ? argv[0] : "pipe");
- for (i = 1; argv && (i < argc); i++)
- fprintf(output, " %s", argv[i]);
- } else if (target.pid)
- fprintf(output, "process id \'%s", target.pid);
- else
- fprintf(output, "thread id \'%s", target.tid);
-
- fprintf(output, "\'");
- if (run_count > 1)
- fprintf(output, " (%d runs)", run_count);
- fprintf(output, ":\n\n");
- }
-}
-
-static int get_precision(double num)
-{
- if (num > 1)
- return 0;
-
- return lround(ceil(-log10(num)));
-}
-
-static void print_table(FILE *output, int precision, double avg)
-{
- char tmp[64];
- int idx, indent = 0;
-
- scnprintf(tmp, 64, " %17.*f", precision, avg);
- while (tmp[indent] == ' ')
- indent++;
-
- fprintf(output, "%*s# Table of individual measurements:\n", indent, "");
-
- for (idx = 0; idx < run_count; idx++) {
- double run = (double) walltime_run[idx] / NSEC_PER_SEC;
- int h, n = 1 + abs((int) (100.0 * (run - avg)/run) / 5);
-
- fprintf(output, " %17.*f (%+.*f) ",
- precision, run, precision, run - avg);
-
- for (h = 0; h < n; h++)
- fprintf(output, "#");
-
- fprintf(output, "\n");
- }
-
- fprintf(output, "\n%*s# Final result:\n", indent, "");
-}
-
-static double timeval2double(struct timeval *t)
-{
- return t->tv_sec + (double) t->tv_usec/USEC_PER_SEC;
-}
-
-static void print_footer(void)
-{
- double avg = avg_stats(&walltime_nsecs_stats) / NSEC_PER_SEC;
- FILE *output = stat_config.output;
- int n;
-
- if (!null_run)
- fprintf(output, "\n");
-
- if (run_count == 1) {
- fprintf(output, " %17.9f seconds time elapsed", avg);
-
- if (ru_display) {
- double ru_utime = timeval2double(&ru_data.ru_utime);
- double ru_stime = timeval2double(&ru_data.ru_stime);
-
- fprintf(output, "\n\n");
- fprintf(output, " %17.9f seconds user\n", ru_utime);
- fprintf(output, " %17.9f seconds sys\n", ru_stime);
- }
- } else {
- double sd = stddev_stats(&walltime_nsecs_stats) / NSEC_PER_SEC;
- /*
- * Display at most 2 more significant
- * digits than the stddev inaccuracy.
- */
- int precision = get_precision(sd) + 2;
-
- if (walltime_run_table)
- print_table(output, precision, avg);
-
- fprintf(output, " %17.*f +- %.*f seconds time elapsed",
- precision, avg, precision, sd);
-
- print_noise_pct(sd, avg);
- }
- fprintf(output, "\n\n");
-
- if (print_free_counters_hint &&
- sysctl__read_int("kernel/nmi_watchdog", &n) >= 0 &&
- n > 0)
- fprintf(output,
-"Some events weren't counted. Try disabling the NMI watchdog:\n"
-" echo 0 > /proc/sys/kernel/nmi_watchdog\n"
-" perf stat ...\n"
-" echo 1 > /proc/sys/kernel/nmi_watchdog\n");
-
- if (print_mixed_hw_group_error)
- fprintf(output,
- "The events in group usually have to be from "
- "the same PMU. Try reorganizing the group.\n");
-}
-
static void print_counters(struct timespec *ts, int argc, const char **argv)
{
- int interval = stat_config.interval;
- struct perf_evsel *counter;
- char buf[64], *prefix = NULL;
-
/* Do not print anything if we record to the pipe. */
if (STAT_RECORD && perf_stat.data.is_pipe)
return;
- if (interval)
- print_interval(prefix = buf, ts);
- else
- print_header(argc, argv);
-
- if (metric_only) {
- static int num_print_iv;
-
- if (num_print_iv == 0 && !interval)
- print_metric_headers(prefix, false);
- if (num_print_iv++ == 25)
- num_print_iv = 0;
- if (stat_config.aggr_mode == AGGR_GLOBAL && prefix)
- fprintf(stat_config.output, "%s", prefix);
- }
-
- switch (stat_config.aggr_mode) {
- case AGGR_CORE:
- case AGGR_SOCKET:
- print_aggr(prefix);
- break;
- case AGGR_THREAD:
- evlist__for_each_entry(evsel_list, counter) {
- if (is_duration_time(counter))
- continue;
- print_aggr_thread(counter, prefix);
- }
- break;
- case AGGR_GLOBAL:
- evlist__for_each_entry(evsel_list, counter) {
- if (is_duration_time(counter))
- continue;
- print_counter_aggr(counter, prefix);
- }
- if (metric_only)
- fputc('\n', stat_config.output);
- break;
- case AGGR_NONE:
- if (metric_only)
- print_no_aggr_metric(prefix);
- else {
- evlist__for_each_entry(evsel_list, counter) {
- if (is_duration_time(counter))
- continue;
- print_counter(counter, prefix);
- }
- }
- break;
- case AGGR_UNSET:
- default:
- break;
- }
-
- if (!interval && !csv_output)
- print_footer();
-
- fflush(stat_config.output);
+ perf_evlist__print_counters(evsel_list, &stat_config, &target,
+ ts, argc, argv);
}
static volatile int signr = -1;
const char *s __maybe_unused, int unset)
{
force_metric_only = true;
- metric_only = !unset;
+ stat_config.metric_only = !unset;
return 0;
}
const char *str,
int unset __maybe_unused)
{
- return metricgroup__parse_groups(opt, str, &metric_events);
+ return metricgroup__parse_groups(opt, str, &stat_config.metric_events);
}
static const struct option stat_options[] = {
parse_events_option),
OPT_CALLBACK(0, "filter", &evsel_list, "filter",
"event filter", parse_filter),
- OPT_BOOLEAN('i', "no-inherit", &no_inherit,
+ OPT_BOOLEAN('i', "no-inherit", &stat_config.no_inherit,
"child tasks do not inherit counters"),
OPT_STRING('p', "pid", &target.pid, "pid",
"stat events on existing process id"),
OPT_BOOLEAN('c', "scale", &stat_config.scale, "scale/normalize counters"),
OPT_INCR('v', "verbose", &verbose,
"be more verbose (show counter open errors, etc)"),
- OPT_INTEGER('r', "repeat", &run_count,
+ OPT_INTEGER('r', "repeat", &stat_config.run_count,
"repeat command and print average + stddev (max: 100, forever: 0)"),
- OPT_BOOLEAN(0, "table", &walltime_run_table,
+ OPT_BOOLEAN(0, "table", &stat_config.walltime_run_table,
"display details about each run (only with -r option)"),
- OPT_BOOLEAN('n', "null", &null_run,
+ OPT_BOOLEAN('n', "null", &stat_config.null_run,
"null run - dont start any counters"),
OPT_INCR('d', "detailed", &detailed_run,
"detailed run - start a lot of events"),
"list of cpus to monitor in system-wide"),
OPT_SET_UINT('A', "no-aggr", &stat_config.aggr_mode,
"disable CPU count aggregation", AGGR_NONE),
- OPT_BOOLEAN(0, "no-merge", &no_merge, "Do not merge identical named events"),
- OPT_STRING('x', "field-separator", &csv_sep, "separator",
+ OPT_BOOLEAN(0, "no-merge", &stat_config.no_merge, "Do not merge identical named events"),
+ OPT_STRING('x', "field-separator", &stat_config.csv_sep, "separator",
"print counts with custom separator"),
OPT_CALLBACK('G', "cgroup", &evsel_list, "name",
"monitor event in cgroup name only", parse_cgroups),
"(overhead is possible for values <= 100ms)"),
OPT_INTEGER(0, "interval-count", &stat_config.times,
"print counts for fixed number of times"),
- OPT_BOOLEAN(0, "interval-clear", &interval_clear,
+ OPT_BOOLEAN(0, "interval-clear", &stat_config.interval_clear,
"clear screen in between new interval"),
OPT_UINTEGER(0, "timeout", &stat_config.timeout,
"stop workload and print counts after a timeout period in ms (>= 10ms)"),
"aggregate counts per physical processor core", AGGR_CORE),
OPT_SET_UINT(0, "per-thread", &stat_config.aggr_mode,
"aggregate counts per thread", AGGR_THREAD),
- OPT_UINTEGER('D', "delay", &initial_delay,
+ OPT_UINTEGER('D', "delay", &stat_config.initial_delay,
"ms to wait before starting measurement after program start"),
- OPT_CALLBACK_NOOPT(0, "metric-only", &metric_only, NULL,
+ OPT_CALLBACK_NOOPT(0, "metric-only", &stat_config.metric_only, NULL,
"Only print computed metrics. No raw values", enable_metric_only),
OPT_BOOLEAN(0, "topdown", &topdown_run,
"measure topdown level 1 statistics"),
OPT_END()
};
-static int perf_stat__get_socket(struct cpu_map *map, int cpu)
+static int perf_stat__get_socket(struct perf_stat_config *config __maybe_unused,
+ struct cpu_map *map, int cpu)
{
return cpu_map__get_socket(map, cpu, NULL);
}
-static int perf_stat__get_core(struct cpu_map *map, int cpu)
+static int perf_stat__get_core(struct perf_stat_config *config __maybe_unused,
+ struct cpu_map *map, int cpu)
{
return cpu_map__get_core(map, cpu, NULL);
}
return max;
}
-static struct cpu_map *cpus_aggr_map;
-
-static int perf_stat__get_aggr(aggr_get_id_t get_id, struct cpu_map *map, int idx)
+static int perf_stat__get_aggr(struct perf_stat_config *config,
+ aggr_get_id_t get_id, struct cpu_map *map, int idx)
{
int cpu;
cpu = map->map[idx];
- if (cpus_aggr_map->map[cpu] == -1)
- cpus_aggr_map->map[cpu] = get_id(map, idx);
+ if (config->cpus_aggr_map->map[cpu] == -1)
+ config->cpus_aggr_map->map[cpu] = get_id(config, map, idx);
- return cpus_aggr_map->map[cpu];
+ return config->cpus_aggr_map->map[cpu];
}
-static int perf_stat__get_socket_cached(struct cpu_map *map, int idx)
+static int perf_stat__get_socket_cached(struct perf_stat_config *config,
+ struct cpu_map *map, int idx)
{
- return perf_stat__get_aggr(perf_stat__get_socket, map, idx);
+ return perf_stat__get_aggr(config, perf_stat__get_socket, map, idx);
}
-static int perf_stat__get_core_cached(struct cpu_map *map, int idx)
+static int perf_stat__get_core_cached(struct perf_stat_config *config,
+ struct cpu_map *map, int idx)
{
- return perf_stat__get_aggr(perf_stat__get_core, map, idx);
+ return perf_stat__get_aggr(config, perf_stat__get_core, map, idx);
}
static int perf_stat_init_aggr_mode(void)
switch (stat_config.aggr_mode) {
case AGGR_SOCKET:
- if (cpu_map__build_socket_map(evsel_list->cpus, &aggr_map)) {
+ if (cpu_map__build_socket_map(evsel_list->cpus, &stat_config.aggr_map)) {
perror("cannot build socket map");
return -1;
}
- aggr_get_id = perf_stat__get_socket_cached;
+ stat_config.aggr_get_id = perf_stat__get_socket_cached;
break;
case AGGR_CORE:
- if (cpu_map__build_core_map(evsel_list->cpus, &aggr_map)) {
+ if (cpu_map__build_core_map(evsel_list->cpus, &stat_config.aggr_map)) {
perror("cannot build core map");
return -1;
}
- aggr_get_id = perf_stat__get_core_cached;
+ stat_config.aggr_get_id = perf_stat__get_core_cached;
break;
case AGGR_NONE:
case AGGR_GLOBAL:
* the aggregation translate cpumap.
*/
nr = cpu_map__get_max(evsel_list->cpus);
- cpus_aggr_map = cpu_map__empty_new(nr + 1);
- return cpus_aggr_map ? 0 : -ENOMEM;
+ stat_config.cpus_aggr_map = cpu_map__empty_new(nr + 1);
+ return stat_config.cpus_aggr_map ? 0 : -ENOMEM;
}
static void perf_stat__exit_aggr_mode(void)
{
- cpu_map__put(aggr_map);
- cpu_map__put(cpus_aggr_map);
- aggr_map = NULL;
- cpus_aggr_map = NULL;
+ cpu_map__put(stat_config.aggr_map);
+ cpu_map__put(stat_config.cpus_aggr_map);
+ stat_config.aggr_map = NULL;
+ stat_config.cpus_aggr_map = NULL;
}
static inline int perf_env__get_cpu(struct perf_env *env, struct cpu_map *map, int idx)
return cpu_map__build_map(cpus, corep, perf_env__get_core, env);
}
-static int perf_stat__get_socket_file(struct cpu_map *map, int idx)
+static int perf_stat__get_socket_file(struct perf_stat_config *config __maybe_unused,
+ struct cpu_map *map, int idx)
{
return perf_env__get_socket(map, idx, &perf_stat.session->header.env);
}
-static int perf_stat__get_core_file(struct cpu_map *map, int idx)
+static int perf_stat__get_core_file(struct perf_stat_config *config __maybe_unused,
+ struct cpu_map *map, int idx)
{
return perf_env__get_core(map, idx, &perf_stat.session->header.env);
}
switch (stat_config.aggr_mode) {
case AGGR_SOCKET:
- if (perf_env__build_socket_map(env, evsel_list->cpus, &aggr_map)) {
+ if (perf_env__build_socket_map(env, evsel_list->cpus, &stat_config.aggr_map)) {
perror("cannot build socket map");
return -1;
}
- aggr_get_id = perf_stat__get_socket_file;
+ stat_config.aggr_get_id = perf_stat__get_socket_file;
break;
case AGGR_CORE:
- if (perf_env__build_core_map(env, evsel_list->cpus, &aggr_map)) {
+ if (perf_env__build_core_map(env, evsel_list->cpus, &stat_config.aggr_map)) {
perror("cannot build core map");
return -1;
}
- aggr_get_id = perf_stat__get_core_file;
+ stat_config.aggr_get_id = perf_stat__get_core_file;
break;
case AGGR_NONE:
case AGGR_GLOBAL:
struct parse_events_error errinfo;
/* Set attrs if no event is selected and !null_run: */
- if (null_run)
+ if (stat_config.null_run)
return 0;
if (transaction_run) {
struct option opt = { .value = &evsel_list };
return metricgroup__parse_groups(&opt, "transaction",
- &metric_events);
+ &stat_config.metric_events);
}
if (pmu_have_event("cpu", "cycles-ct") &&
if (pmu_have_event("msr", "aperf") &&
pmu_have_event("msr", "smi")) {
if (!force_metric_only)
- metric_only = true;
+ stat_config.metric_only = true;
err = parse_events(evsel_list, smi_cost_attrs, &errinfo);
} else {
fprintf(stderr, "To measure SMI cost, it needs "
}
if (!force_metric_only)
- metric_only = true;
+ stat_config.metric_only = true;
if (topdown_filter_events(topdown_attrs, &str,
arch_topdown_check_group(&warn)) < 0) {
pr_err("Out of memory\n");
if (output_name)
data->file.path = output_name;
- if (run_count != 1 || forever) {
+ if (stat_config.run_count != 1 || forever) {
pr_err("Cannot use -r option with perf stat record.\n");
return -1;
}
return argc;
}
-static int process_stat_round_event(struct perf_tool *tool __maybe_unused,
- union perf_event *event,
- struct perf_session *session)
+static int process_stat_round_event(struct perf_session *session,
+ union perf_event *event)
{
struct stat_round_event *stat_round = &event->stat_round;
struct perf_evsel *counter;
}
static
-int process_stat_config_event(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session __maybe_unused)
+int process_stat_config_event(struct perf_session *session,
+ union perf_event *event)
{
+ struct perf_tool *tool = session->tool;
struct perf_stat *st = container_of(tool, struct perf_stat, tool);
perf_event__read_stat_config(&stat_config, &event->stat_config);
}
static
-int process_thread_map_event(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session __maybe_unused)
+int process_thread_map_event(struct perf_session *session,
+ union perf_event *event)
{
+ struct perf_tool *tool = session->tool;
struct perf_stat *st = container_of(tool, struct perf_stat, tool);
if (st->threads) {
}
static
-int process_cpu_map_event(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session __maybe_unused)
+int process_cpu_map_event(struct perf_session *session,
+ union perf_event *event)
{
+ struct perf_tool *tool = session->tool;
struct perf_stat *st = container_of(tool, struct perf_stat, tool);
struct cpu_map *cpus;
perf_stat__collect_metric_expr(evsel_list);
perf_stat__init_shadow_stats();
- if (csv_sep) {
- csv_output = true;
- if (!strcmp(csv_sep, "\\t"))
- csv_sep = "\t";
+ if (stat_config.csv_sep) {
+ stat_config.csv_output = true;
+ if (!strcmp(stat_config.csv_sep, "\\t"))
+ stat_config.csv_sep = "\t";
} else
- csv_sep = DEFAULT_SEPARATOR;
+ stat_config.csv_sep = DEFAULT_SEPARATOR;
if (argc && !strncmp(argv[0], "rec", 3)) {
argc = __cmd_record(argc, argv);
goto out;
}
- if (metric_only && stat_config.aggr_mode == AGGR_THREAD) {
+ if (stat_config.metric_only && stat_config.aggr_mode == AGGR_THREAD) {
fprintf(stderr, "--metric-only is not supported with --per-thread\n");
goto out;
}
- if (metric_only && run_count > 1) {
+ if (stat_config.metric_only && stat_config.run_count > 1) {
fprintf(stderr, "--metric-only is not supported with -r\n");
goto out;
}
- if (walltime_run_table && run_count <= 1) {
+ if (stat_config.walltime_run_table && stat_config.run_count <= 1) {
fprintf(stderr, "--table is only supported with -r\n");
parse_options_usage(stat_usage, stat_options, "r", 1);
parse_options_usage(NULL, stat_options, "table", 0);
/*
* let the spreadsheet do the pretty-printing
*/
- if (csv_output) {
+ if (stat_config.csv_output) {
/* User explicitly passed -B? */
if (big_num_opt == 1) {
fprintf(stderr, "-B option not supported with -x\n");
parse_options_usage(NULL, stat_options, "x", 1);
goto out;
} else /* Nope, so disable big number formatting */
- big_num = false;
+ stat_config.big_num = false;
} else if (big_num_opt == 0) /* User passed --no-big-num */
- big_num = false;
+ stat_config.big_num = false;
setup_system_wide(argc);
* Display user/system times only for single
* run and when there's specified tracee.
*/
- if ((run_count == 1) && target__none(&target))
- ru_display = true;
+ if ((stat_config.run_count == 1) && target__none(&target))
+ stat_config.ru_display = true;
- if (run_count < 0) {
+ if (stat_config.run_count < 0) {
pr_err("Run count must be a positive number\n");
parse_options_usage(stat_usage, stat_options, "r", 1);
goto out;
- } else if (run_count == 0) {
+ } else if (stat_config.run_count == 0) {
forever = true;
- run_count = 1;
+ stat_config.run_count = 1;
}
- if (walltime_run_table) {
- walltime_run = zalloc(run_count * sizeof(walltime_run[0]));
- if (!walltime_run) {
+ if (stat_config.walltime_run_table) {
+ stat_config.walltime_run = zalloc(stat_config.run_count * sizeof(stat_config.walltime_run[0]));
+ if (!stat_config.walltime_run) {
pr_err("failed to setup -r option");
goto out;
}
if (perf_stat_init_aggr_mode())
goto out;
+ /*
+ * Set sample_type to PERF_SAMPLE_IDENTIFIER, which should be harmless
+ * while avoiding that older tools show confusing messages.
+ *
+ * However for pipe sessions we need to keep it zero,
+ * because script's perf_evsel__check_attr is triggered
+ * by attr->sample_type != 0, and we can't run it on
+ * stat sessions.
+ */
+ stat_config.identifier = !(STAT_RECORD && perf_stat.data.is_pipe);
+
/*
* We dont want to block the signals - that would cause
* child tasks to inherit that and Ctrl-C would not work.
signal(SIGABRT, skip_signal);
status = 0;
- for (run_idx = 0; forever || run_idx < run_count; run_idx++) {
- if (run_count != 1 && verbose > 0)
+ for (run_idx = 0; forever || run_idx < stat_config.run_count; run_idx++) {
+ if (stat_config.run_count != 1 && verbose > 0)
fprintf(output, "[ perf stat: executing run #%d ... ]\n",
run_idx + 1);
perf_stat__exit_aggr_mode();
perf_evlist__free_stats(evsel_list);
out:
- free(walltime_run);
+ free(stat_config.walltime_run);
if (smi_cost && smi_reset)
sysfs__write_int(FREEZE_ON_SMI_PATH, 0);
return 0;
}
-static int tp_field__init_uint(struct tp_field *field, struct format_field *format_field, bool needs_swap)
+static int tp_field__init_uint(struct tp_field *field, struct tep_format_field *format_field, bool needs_swap)
{
return __tp_field__init_uint(field, format_field->size, format_field->offset, needs_swap);
}
return 0;
}
-static int tp_field__init_ptr(struct tp_field *field, struct format_field *format_field)
+static int tp_field__init_ptr(struct tp_field *field, struct tep_format_field *format_field)
{
return __tp_field__init_ptr(field, format_field->offset);
}
struct tp_field *field,
const char *name)
{
- struct format_field *format_field = perf_evsel__field(evsel, name);
+ struct tep_format_field *format_field = perf_evsel__field(evsel, name);
if (format_field == NULL)
return -1;
struct tp_field *field,
const char *name)
{
- struct format_field *format_field = perf_evsel__field(evsel, name);
+ struct tep_format_field *format_field = perf_evsel__field(evsel, name);
if (format_field == NULL)
return -1;
return __tp_field__init_ptr(&sc->args, sc->id.offset + sizeof(u64));
}
+static int perf_evsel__init_augmented_syscall_tp_ret(struct perf_evsel *evsel)
+{
+ struct syscall_tp *sc = evsel->priv;
+
+ return __tp_field__init_uint(&sc->ret, sizeof(u64), sc->id.offset + sizeof(u64), evsel->needs_swap);
+}
+
static int perf_evsel__init_raw_syscall_tp(struct perf_evsel *evsel, void *handler)
{
evsel->priv = malloc(sizeof(struct syscall_tp));
};
static DEFINE_STRARRAY(clockid);
-static const char *socket_families[] = {
- "UNSPEC", "LOCAL", "INET", "AX25", "IPX", "APPLETALK", "NETROM",
- "BRIDGE", "ATMPVC", "X25", "INET6", "ROSE", "DECnet", "NETBEUI",
- "SECURITY", "KEY", "NETLINK", "PACKET", "ASH", "ECONET", "ATMSVC",
- "RDS", "SNA", "IRDA", "PPPOX", "WANPIPE", "LLC", "IB", "CAN", "TIPC",
- "BLUETOOTH", "IUCV", "RXRPC", "ISDN", "PHONET", "IEEE802154", "CAIF",
- "ALG", "NFC", "VSOCK",
-};
-static DEFINE_STRARRAY(socket_families);
-
static size_t syscall_arg__scnprintf_access_mode(char *bf, size_t size,
struct syscall_arg *arg)
{
} syscall_fmts[] = {
{ .name = "access",
.arg = { [1] = { .scnprintf = SCA_ACCMODE, /* mode */ }, }, },
+ { .name = "bind",
+ .arg = { [1] = { .scnprintf = SCA_SOCKADDR, /* umyaddr */ }, }, },
{ .name = "bpf",
.arg = { [0] = STRARRAY(cmd, bpf_cmd), }, },
{ .name = "brk", .hexret = true,
[4] = { .name = "tls", .scnprintf = SCA_HEX, }, }, },
{ .name = "close",
.arg = { [0] = { .scnprintf = SCA_CLOSE_FD, /* fd */ }, }, },
+ { .name = "connect",
+ .arg = { [1] = { .scnprintf = SCA_SOCKADDR, /* servaddr */ }, }, },
{ .name = "epoll_ctl",
.arg = { [1] = STRARRAY(op, epoll_ctl_ops), }, },
{ .name = "eventfd2",
{ .name = "sendmsg",
.arg = { [2] = { .scnprintf = SCA_MSG_FLAGS, /* flags */ }, }, },
{ .name = "sendto",
- .arg = { [3] = { .scnprintf = SCA_MSG_FLAGS, /* flags */ }, }, },
+ .arg = { [3] = { .scnprintf = SCA_MSG_FLAGS, /* flags */ },
+ [4] = { .scnprintf = SCA_SOCKADDR, /* addr */ }, }, },
{ .name = "set_tid_address", .errpid = true, },
{ .name = "setitimer",
.arg = { [0] = STRARRAY(which, itimers), }, },
.arg = { [2] = { .scnprintf = SCA_SIGNUM, /* sig */ }, }, },
{ .name = "tkill",
.arg = { [1] = { .scnprintf = SCA_SIGNUM, /* sig */ }, }, },
+ { .name = "umount2", .alias = "umount", },
{ .name = "uname", .alias = "newuname", },
{ .name = "unlinkat",
.arg = { [0] = { .scnprintf = SCA_FDAT, /* dfd */ }, }, },
/*
* is_exit: is this "exit" or "exit_group"?
* is_open: is this "open" or "openat"? To associate the fd returned in sys_exit with the pathname in sys_enter.
+ * args_size: sum of the sizes of the syscall arguments, anything after that is augmented stuff: pathname for openat, etc.
*/
struct syscall {
- struct event_format *tp_format;
+ struct tep_event_format *tp_format;
int nr_args;
+ int args_size;
bool is_exit;
bool is_open;
- struct format_field *args;
+ struct tep_format_field *args;
const char *name;
struct syscall_fmt *fmt;
struct syscall_arg_fmt *arg_fmt;
ttrace->filename.entry_str_pos = bf - ttrace->entry_str;
}
+static size_t syscall_arg__scnprintf_augmented_string(struct syscall_arg *arg, char *bf, size_t size)
+{
+ struct augmented_arg *augmented_arg = arg->augmented.args;
+
+ return scnprintf(bf, size, "%.*s", augmented_arg->size, augmented_arg->value);
+}
+
static size_t syscall_arg__scnprintf_filename(char *bf, size_t size,
struct syscall_arg *arg)
{
unsigned long ptr = arg->val;
+ if (arg->augmented.args)
+ return syscall_arg__scnprintf_augmented_string(arg, bf, size);
+
if (!arg->trace->vfs_getname)
return scnprintf(bf, size, "%#x", ptr);
interrupted = sig == SIGINT;
}
-static size_t trace__fprintf_entry_head(struct trace *trace, struct thread *thread,
- u64 duration, bool duration_calculated, u64 tstamp, FILE *fp)
+static size_t trace__fprintf_comm_tid(struct trace *trace, struct thread *thread, FILE *fp)
{
- size_t printed = trace__fprintf_tstamp(trace, tstamp, fp);
- printed += fprintf_duration(duration, duration_calculated, fp);
+ size_t printed = 0;
if (trace->multiple_threads) {
if (trace->show_comm)
return printed;
}
+static size_t trace__fprintf_entry_head(struct trace *trace, struct thread *thread,
+ u64 duration, bool duration_calculated, u64 tstamp, FILE *fp)
+{
+ size_t printed = trace__fprintf_tstamp(trace, tstamp, fp);
+ printed += fprintf_duration(duration, duration_calculated, fp);
+ return printed + trace__fprintf_comm_tid(trace, thread, fp);
+}
+
static int trace__process_event(struct trace *trace, struct machine *machine,
union perf_event *event, struct perf_sample *sample)
{
static int syscall__set_arg_fmts(struct syscall *sc)
{
- struct format_field *field;
+ struct tep_format_field *field, *last_field = NULL;
int idx = 0, len;
for (field = sc->args; field; field = field->next, ++idx) {
+ last_field = field;
+
if (sc->fmt && sc->fmt->arg[idx].scnprintf)
continue;
strcmp(field->name, "path") == 0 ||
strcmp(field->name, "pathname") == 0))
sc->arg_fmt[idx].scnprintf = SCA_FILENAME;
- else if (field->flags & FIELD_IS_POINTER)
+ else if (field->flags & TEP_FIELD_IS_POINTER)
sc->arg_fmt[idx].scnprintf = syscall_arg__scnprintf_hex;
else if (strcmp(field->type, "pid_t") == 0)
sc->arg_fmt[idx].scnprintf = SCA_PID;
}
}
+ if (last_field)
+ sc->args_size = last_field->offset + last_field->size;
+
return 0;
}
}
static size_t syscall__scnprintf_args(struct syscall *sc, char *bf, size_t size,
- unsigned char *args, struct trace *trace,
- struct thread *thread)
+ unsigned char *args, void *augmented_args, int augmented_args_size,
+ struct trace *trace, struct thread *thread)
{
size_t printed = 0;
unsigned long val;
u8 bit = 1;
struct syscall_arg arg = {
.args = args,
+ .augmented = {
+ .size = augmented_args_size,
+ .args = augmented_args,
+ },
.idx = 0,
.mask = 0,
.trace = trace,
ttrace->ret_scnprintf = NULL;
if (sc->args != NULL) {
- struct format_field *field;
+ struct tep_format_field *field;
for (field = sc->args; field;
field = field->next, ++arg.idx, bit <<= 1) {
return printed;
}
+static void *syscall__augmented_args(struct syscall *sc, struct perf_sample *sample, int *augmented_args_size)
+{
+ void *augmented_args = NULL;
+
+ *augmented_args_size = sample->raw_size - sc->args_size;
+ if (*augmented_args_size > 0)
+ augmented_args = sample->raw_data + sc->args_size;
+
+ return augmented_args;
+}
+
static int trace__sys_enter(struct trace *trace, struct perf_evsel *evsel,
union perf_event *event __maybe_unused,
struct perf_sample *sample)
size_t printed = 0;
struct thread *thread;
int id = perf_evsel__sc_tp_uint(evsel, id, sample), err = -1;
+ int augmented_args_size = 0;
+ void *augmented_args = NULL;
struct syscall *sc = trace__syscall_info(trace, evsel, id);
struct thread_trace *ttrace;
if (!(trace->duration_filter || trace->summary_only || trace->min_stack))
trace__printf_interrupted_entry(trace);
-
+ /*
+ * If this is raw_syscalls.sys_enter, then it always comes with the 6 possible
+ * arguments, even if the syscall being handled, say "openat", uses only 4 arguments
+ * this breaks syscall__augmented_args() check for augmented args, as we calculate
+ * syscall->args_size using each syscalls:sys_enter_NAME tracefs format file,
+ * so when handling, say the openat syscall, we end up getting 6 args for the
+ * raw_syscalls:sys_enter event, when we expected just 4, we end up mistakenly
+ * thinking that the extra 2 u64 args are the augmented filename, so just check
+ * here and avoid using augmented syscalls when the evsel is the raw_syscalls one.
+ */
+ if (evsel != trace->syscalls.events.sys_enter)
+ augmented_args = syscall__augmented_args(sc, sample, &augmented_args_size);
ttrace->entry_time = sample->time;
msg = ttrace->entry_str;
printed += scnprintf(msg + printed, trace__entry_str_size - printed, "%s(", sc->name);
printed += syscall__scnprintf_args(sc, msg + printed, trace__entry_str_size - printed,
- args, trace, thread);
+ args, augmented_args, augmented_args_size, trace, thread);
if (sc->is_exit) {
if (!(trace->duration_filter || trace->summary_only || trace->failure_only || trace->min_stack)) {
int id = perf_evsel__sc_tp_uint(evsel, id, sample), err = -1;
struct syscall *sc = trace__syscall_info(trace, evsel, id);
char msg[1024];
- void *args;
+ void *args, *augmented_args = NULL;
+ int augmented_args_size;
if (sc == NULL)
return -1;
goto out_put;
args = perf_evsel__sc_tp_ptr(evsel, args, sample);
- syscall__scnprintf_args(sc, msg, sizeof(msg), args, trace, thread);
+ augmented_args = syscall__augmented_args(sc, sample, &augmented_args_size);
+ syscall__scnprintf_args(sc, msg, sizeof(msg), args, augmented_args, augmented_args_size, trace, thread);
fprintf(trace->output, "%s", msg);
err = 0;
out_put:
union perf_event *event __maybe_unused,
struct perf_sample *sample)
{
+ struct thread *thread = machine__findnew_thread(trace->host, sample->pid, sample->tid);
int callchain_ret = 0;
if (sample->callchain) {
if (trace->trace_syscalls)
fprintf(trace->output, "( ): ");
+ if (thread)
+ trace__fprintf_comm_tid(trace, thread, trace->output);
+
+ if (evsel == trace->syscalls.events.augmented) {
+ int id = perf_evsel__sc_tp_uint(evsel, id, sample);
+ struct syscall *sc = trace__syscall_info(trace, evsel, id);
+
+ if (sc) {
+ fprintf(trace->output, "%s(", sc->name);
+ trace__fprintf_sys_enter(trace, evsel, sample);
+ fputc(')', trace->output);
+ goto newline;
+ }
+
+ /*
+ * XXX: Not having the associated syscall info or not finding/adding
+ * the thread should never happen, but if it does...
+ * fall thru and print it as a bpf_output event.
+ */
+ }
+
fprintf(trace->output, "%s:", evsel->name);
if (perf_evsel__is_bpf_output(evsel)) {
- if (evsel == trace->syscalls.events.augmented)
- trace__fprintf_sys_enter(trace, evsel, sample);
- else
- bpf_output__fprintf(trace, sample);
+ bpf_output__fprintf(trace, sample);
} else if (evsel->tp_format) {
if (strncmp(evsel->tp_format->name, "sys_enter_", 10) ||
trace__fprintf_sys_enter(trace, evsel, sample)) {
}
}
+newline:
fprintf(trace->output, "\n");
if (callchain_ret > 0)
trace__fprintf_callchain(trace, sample);
else if (callchain_ret < 0)
pr_err("Problem processing %s callchain, skipping...\n", perf_evsel__name(evsel));
+ thread__put(thread);
out:
return 0;
}
goto out;
}
- if (evsel) {
- if (perf_evsel__init_augmented_syscall_tp(evsel) ||
- perf_evsel__init_augmented_syscall_tp_args(evsel))
- goto out;
+ if (evsel)
trace.syscalls.events.augmented = evsel;
- }
err = bpf__setup_stdout(trace.evlist);
if (err) {
}
}
+ /*
+ * If we are augmenting syscalls, then combine what we put in the
+ * __augmented_syscalls__ BPF map with what is in the
+ * syscalls:sys_exit_FOO tracepoints, i.e. just like we do without BPF,
+ * combining raw_syscalls:sys_enter with raw_syscalls:sys_exit.
+ *
+ * We'll switch to look at two BPF maps, one for sys_enter and the
+ * other for sys_exit when we start augmenting the sys_exit paths with
+ * buffers that are being copied from kernel to userspace, think 'read'
+ * syscall.
+ */
+ if (trace.syscalls.events.augmented) {
+ evsel = trace.syscalls.events.augmented;
+
+ if (perf_evsel__init_augmented_syscall_tp(evsel) ||
+ perf_evsel__init_augmented_syscall_tp_args(evsel))
+ goto out;
+ evsel->handler = trace__sys_enter;
+
+ evlist__for_each_entry(trace.evlist, evsel) {
+ if (strstarts(perf_evsel__name(evsel), "syscalls:sys_exit_")) {
+ perf_evsel__init_augmented_syscall_tp(evsel);
+ perf_evsel__init_augmented_syscall_tp_ret(evsel);
+ evsel->handler = trace__sys_exit;
+ }
+ }
+ }
+
if ((argc >= 1) && (strcmp(argv[0], "record") == 0))
return trace__record(&trace, argc-1, &argv[1]);
include/uapi/linux/stat.h
include/uapi/linux/vhost.h
include/uapi/sound/asound.h
+include/linux/bits.h
include/linux/hash.h
include/uapi/linux/hw_breakpoint.h
arch/x86/include/asm/disabled-features.h
perf-timechart mainporcelain common
perf-top mainporcelain common
perf-trace mainporcelain audit
+perf-version mainporcelain common
// SPDX-License-Identifier: GPL-2.0
/*
- * Augment the openat syscall with the contents of the filename pointer argument.
+ * Augment syscalls with the contents of the pointer arguments.
*
* Test it with:
*
* the last one should be the one for '/etc/passwd'.
*
* This matches what is marshalled into the raw_syscall:sys_enter payload
- * expected by the 'perf trace' beautifiers, and can be used by them unmodified,
- * which will be done as that feature is implemented in the next csets, for now
- * it will appear in a dump done by the default tracepoint handler in 'perf trace',
- * that uses bpf_output__fprintf() to just dump those contents, as done with
- * the bpf-output event associated with the __bpf_output__ map declared in
- * tools/perf/include/bpf/stdio.h.
+ * expected by the 'perf trace' beautifiers, and can be used by them, that will
+ * check if perf_sample->raw_data is more than what is expected for each
+ * syscalls:sys_{enter,exit}_SYSCALL tracepoint, uing the extra data as the
+ * contents of pointer arguments.
*/
#include <stdio.h>
+#include <linux/socket.h>
struct bpf_map SEC("maps") __augmented_syscalls__ = {
.type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
.max_entries = __NR_CPUS__,
};
+struct syscall_exit_args {
+ unsigned long long common_tp_fields;
+ long syscall_nr;
+ long ret;
+};
+
+struct augmented_filename {
+ unsigned int size;
+ int reserved;
+ char value[256];
+};
+
+#define augmented_filename_syscall(syscall) \
+struct augmented_enter_##syscall##_args { \
+ struct syscall_enter_##syscall##_args args; \
+ struct augmented_filename filename; \
+}; \
+int syscall_enter(syscall)(struct syscall_enter_##syscall##_args *args) \
+{ \
+ struct augmented_enter_##syscall##_args augmented_args = { .filename.reserved = 0, }; \
+ unsigned int len = sizeof(augmented_args); \
+ probe_read(&augmented_args.args, sizeof(augmented_args.args), args); \
+ augmented_args.filename.size = probe_read_str(&augmented_args.filename.value, \
+ sizeof(augmented_args.filename.value), \
+ args->filename_ptr); \
+ if (augmented_args.filename.size < sizeof(augmented_args.filename.value)) { \
+ len -= sizeof(augmented_args.filename.value) - augmented_args.filename.size; \
+ len &= sizeof(augmented_args.filename.value) - 1; \
+ } \
+ perf_event_output(args, &__augmented_syscalls__, BPF_F_CURRENT_CPU, \
+ &augmented_args, len); \
+ return 0; \
+} \
+int syscall_exit(syscall)(struct syscall_exit_args *args) \
+{ \
+ return 1; /* 0 as soon as we start copying data returned by the kernel, e.g. 'read' */ \
+}
+
struct syscall_enter_openat_args {
unsigned long long common_tp_fields;
long syscall_nr;
long mode;
};
-struct augmented_enter_openat_args {
- struct syscall_enter_openat_args args;
- char filename[64];
+augmented_filename_syscall(openat);
+
+struct syscall_enter_open_args {
+ unsigned long long common_tp_fields;
+ long syscall_nr;
+ char *filename_ptr;
+ long flags;
+ long mode;
+};
+
+augmented_filename_syscall(open);
+
+struct syscall_enter_inotify_add_watch_args {
+ unsigned long long common_tp_fields;
+ long syscall_nr;
+ long fd;
+ char *filename_ptr;
+ long mask;
+};
+
+augmented_filename_syscall(inotify_add_watch);
+
+struct statbuf;
+
+struct syscall_enter_newstat_args {
+ unsigned long long common_tp_fields;
+ long syscall_nr;
+ char *filename_ptr;
+ struct stat *statbuf;
};
-int syscall_enter(openat)(struct syscall_enter_openat_args *args)
-{
- struct augmented_enter_openat_args augmented_args;
+augmented_filename_syscall(newstat);
+
+#ifndef _K_SS_MAXSIZE
+#define _K_SS_MAXSIZE 128
+#endif
- probe_read(&augmented_args.args, sizeof(augmented_args.args), args);
- probe_read_str(&augmented_args.filename, sizeof(augmented_args.filename), args->filename_ptr);
- perf_event_output(args, &__augmented_syscalls__, BPF_F_CURRENT_CPU,
- &augmented_args, sizeof(augmented_args));
- return 1;
+#define augmented_sockaddr_syscall(syscall) \
+struct augmented_enter_##syscall##_args { \
+ struct syscall_enter_##syscall##_args args; \
+ struct sockaddr_storage addr; \
+}; \
+int syscall_enter(syscall)(struct syscall_enter_##syscall##_args *args) \
+{ \
+ struct augmented_enter_##syscall##_args augmented_args; \
+ unsigned long addrlen = sizeof(augmented_args.addr); \
+ probe_read(&augmented_args.args, sizeof(augmented_args.args), args); \
+/* FIXME_CLANG_OPTIMIZATION_THAT_ACCESSES_USER_CONTROLLED_ADDRLEN_DESPITE_THIS_CHECK */ \
+/* if (addrlen > augmented_args.args.addrlen) */ \
+/* addrlen = augmented_args.args.addrlen; */ \
+/* */ \
+ probe_read(&augmented_args.addr, addrlen, args->addr_ptr); \
+ perf_event_output(args, &__augmented_syscalls__, BPF_F_CURRENT_CPU, \
+ &augmented_args, \
+ sizeof(augmented_args) - sizeof(augmented_args.addr) + addrlen); \
+ return 0; \
+} \
+int syscall_exit(syscall)(struct syscall_exit_args *args) \
+{ \
+ return 1; /* 0 as soon as we start copying data returned by the kernel, e.g. 'read' */ \
}
+struct sockaddr;
+
+struct syscall_enter_bind_args {
+ unsigned long long common_tp_fields;
+ long syscall_nr;
+ long fd;
+ struct sockaddr *addr_ptr;
+ unsigned long addrlen;
+};
+
+augmented_sockaddr_syscall(bind);
+
+struct syscall_enter_connect_args {
+ unsigned long long common_tp_fields;
+ long syscall_nr;
+ long fd;
+ struct sockaddr *addr_ptr;
+ unsigned long addrlen;
+};
+
+augmented_sockaddr_syscall(connect);
+
+struct syscall_enter_sendto_args {
+ unsigned long long common_tp_fields;
+ long syscall_nr;
+ long fd;
+ void *buff;
+ long len;
+ unsigned long flags;
+ struct sockaddr *addr_ptr;
+ long addr_len;
+};
+
+augmented_sockaddr_syscall(sendto);
+
license(GPL);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Augment the filename syscalls with the contents of the filename pointer argument
+ * filtering only those that do not start with /etc/.
+ *
+ * Test it with:
+ *
+ * perf trace -e tools/perf/examples/bpf/augmented_syscalls.c cat /etc/passwd > /dev/null
+ *
+ * It'll catch some openat syscalls related to the dynamic linked and
+ * the last one should be the one for '/etc/passwd'.
+ *
+ * This matches what is marshalled into the raw_syscall:sys_enter payload
+ * expected by the 'perf trace' beautifiers, and can be used by them unmodified,
+ * which will be done as that feature is implemented in the next csets, for now
+ * it will appear in a dump done by the default tracepoint handler in 'perf trace',
+ * that uses bpf_output__fprintf() to just dump those contents, as done with
+ * the bpf-output event associated with the __bpf_output__ map declared in
+ * tools/perf/include/bpf/stdio.h.
+ */
+
+#include <stdio.h>
+
+struct bpf_map SEC("maps") __augmented_syscalls__ = {
+ .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
+ .key_size = sizeof(int),
+ .value_size = sizeof(u32),
+ .max_entries = __NR_CPUS__,
+};
+
+struct augmented_filename {
+ int size;
+ int reserved;
+ char value[64];
+};
+
+#define augmented_filename_syscall_enter(syscall) \
+struct augmented_enter_##syscall##_args { \
+ struct syscall_enter_##syscall##_args args; \
+ struct augmented_filename filename; \
+}; \
+int syscall_enter(syscall)(struct syscall_enter_##syscall##_args *args) \
+{ \
+ char etc[6] = "/etc/"; \
+ struct augmented_enter_##syscall##_args augmented_args = { .filename.reserved = 0, }; \
+ probe_read(&augmented_args.args, sizeof(augmented_args.args), args); \
+ augmented_args.filename.size = probe_read_str(&augmented_args.filename.value, \
+ sizeof(augmented_args.filename.value), \
+ args->filename_ptr); \
+ if (__builtin_memcmp(augmented_args.filename.value, etc, 4) != 0) \
+ return 0; \
+ perf_event_output(args, &__augmented_syscalls__, BPF_F_CURRENT_CPU, \
+ &augmented_args, \
+ (sizeof(augmented_args) - sizeof(augmented_args.filename.value) + \
+ augmented_args.filename.size)); \
+ return 0; \
+}
+
+struct syscall_enter_openat_args {
+ unsigned long long common_tp_fields;
+ long syscall_nr;
+ long dfd;
+ char *filename_ptr;
+ long flags;
+ long mode;
+};
+
+augmented_filename_syscall_enter(openat);
+
+struct syscall_enter_open_args {
+ unsigned long long common_tp_fields;
+ long syscall_nr;
+ char *filename_ptr;
+ long flags;
+ long mode;
+};
+
+augmented_filename_syscall_enter(open);
+
+license(GPL);
#define syscall_enter(name) \
SEC("syscalls:sys_enter_" #name) syscall_enter_ ## name
+#define syscall_exit(name) \
+ SEC("syscalls:sys_exit_" #name) syscall_exit_ ## name
+
#define license(name) \
char _license[] SEC("license") = #name; \
int _version SEC("version") = LINUX_VERSION_CODE;
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _UAPI_LINUX_SOCKET_H
+#define _UAPI_LINUX_SOCKET_H
+
+/*
+ * Desired design of maximum size and alignment (see RFC2553)
+ */
+#define _K_SS_MAXSIZE 128 /* Implementation specific max size */
+#define _K_SS_ALIGNSIZE (__alignof__ (struct sockaddr *))
+ /* Implementation specific desired alignment */
+
+typedef unsigned short __kernel_sa_family_t;
+
+struct __kernel_sockaddr_storage {
+ __kernel_sa_family_t ss_family; /* address family */
+ /* Following field(s) are implementation specific */
+ char __data[_K_SS_MAXSIZE - sizeof(unsigned short)];
+ /* space to achieve desired size, */
+ /* _SS_MAXSIZE value minus size of ss_family */
+} __attribute__ ((aligned(_K_SS_ALIGNSIZE))); /* force desired alignment */
+
+#define sockaddr_storage __kernel_sockaddr_storage
+
+#endif /* _UAPI_LINUX_SOCKET_H */
--- /dev/null
+[
+ {
+ "ArchStdEvent": "BR_IMMED_SPEC",
+ },
+ {
+ "ArchStdEvent": "BR_RETURN_SPEC",
+ },
+ {
+ "ArchStdEvent": "BR_INDIRECT_SPEC",
+ },
+ {
+ "PublicDescription": "Mispredicted or not predicted branch speculatively executed",
+ "EventCode": "0x10",
+ "EventName": "BR_MIS_PRED",
+ "BriefDescription": "Branch mispredicted"
+ },
+ {
+ "PublicDescription": "Predictable branch speculatively executed",
+ "EventCode": "0x12",
+ "EventName": "BR_PRED",
+ "BriefDescription": "Predictable branch"
+ },
+]
--- /dev/null
+[
+ {
+ "ArchStdEvent": "BUS_ACCESS_RD",
+ },
+ {
+ "ArchStdEvent": "BUS_ACCESS_WR",
+ },
+ {
+ "ArchStdEvent": "BUS_ACCESS_SHARED",
+ },
+ {
+ "ArchStdEvent": "BUS_ACCESS_NOT_SHARED",
+ },
+ {
+ "ArchStdEvent": "BUS_ACCESS_NORMAL",
+ },
+ {
+ "ArchStdEvent": "BUS_ACCESS_PERIPH",
+ },
+ {
+ "PublicDescription": "Bus access",
+ "EventCode": "0x19",
+ "EventName": "BUS_ACCESS",
+ "BriefDescription": "Bus access"
+ },
+]
--- /dev/null
+[
+ {
+ "ArchStdEvent": "L1D_CACHE_RD",
+ },
+ {
+ "ArchStdEvent": "L1D_CACHE_WR",
+ },
+ {
+ "ArchStdEvent": "L1D_CACHE_REFILL_RD",
+ },
+ {
+ "ArchStdEvent": "L1D_CACHE_INVAL",
+ },
+ {
+ "ArchStdEvent": "L1D_TLB_REFILL_RD",
+ },
+ {
+ "ArchStdEvent": "L1D_TLB_REFILL_WR",
+ },
+ {
+ "ArchStdEvent": "L2D_CACHE_RD",
+ },
+ {
+ "ArchStdEvent": "L2D_CACHE_WR",
+ },
+ {
+ "ArchStdEvent": "L2D_CACHE_REFILL_RD",
+ },
+ {
+ "ArchStdEvent": "L2D_CACHE_REFILL_WR",
+ },
+ {
+ "ArchStdEvent": "L2D_CACHE_WB_VICTIM",
+ },
+ {
+ "ArchStdEvent": "L2D_CACHE_WB_CLEAN",
+ },
+ {
+ "ArchStdEvent": "L2D_CACHE_INVAL",
+ },
+ {
+ "PublicDescription": "Level 1 instruction cache refill",
+ "EventCode": "0x01",
+ "EventName": "L1I_CACHE_REFILL",
+ "BriefDescription": "L1I cache refill"
+ },
+ {
+ "PublicDescription": "Level 1 instruction TLB refill",
+ "EventCode": "0x02",
+ "EventName": "L1I_TLB_REFILL",
+ "BriefDescription": "L1I TLB refill"
+ },
+ {
+ "PublicDescription": "Level 1 data cache refill",
+ "EventCode": "0x03",
+ "EventName": "L1D_CACHE_REFILL",
+ "BriefDescription": "L1D cache refill"
+ },
+ {
+ "PublicDescription": "Level 1 data cache access",
+ "EventCode": "0x04",
+ "EventName": "L1D_CACHE_ACCESS",
+ "BriefDescription": "L1D cache access"
+ },
+ {
+ "PublicDescription": "Level 1 data TLB refill",
+ "EventCode": "0x05",
+ "EventName": "L1D_TLB_REFILL",
+ "BriefDescription": "L1D TLB refill"
+ },
+ {
+ "PublicDescription": "Level 1 instruction cache access",
+ "EventCode": "0x14",
+ "EventName": "L1I_CACHE_ACCESS",
+ "BriefDescription": "L1I cache access"
+ },
+ {
+ "PublicDescription": "Level 2 data cache access",
+ "EventCode": "0x16",
+ "EventName": "L2D_CACHE_ACCESS",
+ "BriefDescription": "L2D cache access"
+ },
+ {
+ "PublicDescription": "Level 2 data refill",
+ "EventCode": "0x17",
+ "EventName": "L2D_CACHE_REFILL",
+ "BriefDescription": "L2D cache refill"
+ },
+ {
+ "PublicDescription": "Level 2 data cache, Write-Back",
+ "EventCode": "0x18",
+ "EventName": "L2D_CACHE_WB",
+ "BriefDescription": "L2D cache Write-Back"
+ },
+ {
+ "PublicDescription": "Level 1 data TLB access. This event counts any load or store operation which accesses the data L1 TLB",
+ "EventCode": "0x25",
+ "EventName": "L1D_TLB_ACCESS",
+ "BriefDescription": "L1D TLB access"
+ },
+ {
+ "PublicDescription": "Level 1 instruction TLB access. This event counts any instruction fetch which accesses the instruction L1 TLB",
+ "EventCode": "0x26",
+ "EventName": "L1I_TLB_ACCESS",
+ "BriefDescription": "L1I TLB access"
+ },
+ {
+ "PublicDescription": "Level 2 access to data TLB that caused a page table walk. This event counts on any data access which causes L2D_TLB_REFILL to count",
+ "EventCode": "0x34",
+ "EventName": "L2D_TLB_ACCESS",
+ "BriefDescription": "L2D TLB access"
+ },
+ {
+ "PublicDescription": "Level 2 access to instruciton TLB that caused a page table walk. This event counts on any instruciton access which causes L2I_TLB_REFILL to count",
+ "EventCode": "0x35",
+ "EventName": "L2I_TLB_ACCESS",
+ "BriefDescription": "L2D TLB access"
+ },
+ {
+ "PublicDescription": "Branch target buffer misprediction",
+ "EventCode": "0x102",
+ "EventName": "BTB_MIS_PRED",
+ "BriefDescription": "BTB misprediction"
+ },
+ {
+ "PublicDescription": "ITB miss",
+ "EventCode": "0x103",
+ "EventName": "ITB_MISS",
+ "BriefDescription": "ITB miss"
+ },
+ {
+ "PublicDescription": "DTB miss",
+ "EventCode": "0x104",
+ "EventName": "DTB_MISS",
+ "BriefDescription": "DTB miss"
+ },
+ {
+ "PublicDescription": "Level 1 data cache late miss",
+ "EventCode": "0x105",
+ "EventName": "L1D_CACHE_LATE_MISS",
+ "BriefDescription": "L1D cache late miss"
+ },
+ {
+ "PublicDescription": "Level 1 data cache prefetch request",
+ "EventCode": "0x106",
+ "EventName": "L1D_CACHE_PREFETCH",
+ "BriefDescription": "L1D cache prefetch"
+ },
+ {
+ "PublicDescription": "Level 2 data cache prefetch request",
+ "EventCode": "0x107",
+ "EventName": "L2D_CACHE_PREFETCH",
+ "BriefDescription": "L2D cache prefetch"
+ },
+ {
+ "PublicDescription": "Level 1 stage 2 TLB refill",
+ "EventCode": "0x111",
+ "EventName": "L1_STAGE2_TLB_REFILL",
+ "BriefDescription": "L1 stage 2 TLB refill"
+ },
+ {
+ "PublicDescription": "Page walk cache level-0 stage-1 hit",
+ "EventCode": "0x112",
+ "EventName": "PAGE_WALK_L0_STAGE1_HIT",
+ "BriefDescription": "Page walk, L0 stage-1 hit"
+ },
+ {
+ "PublicDescription": "Page walk cache level-1 stage-1 hit",
+ "EventCode": "0x113",
+ "EventName": "PAGE_WALK_L1_STAGE1_HIT",
+ "BriefDescription": "Page walk, L1 stage-1 hit"
+ },
+ {
+ "PublicDescription": "Page walk cache level-2 stage-1 hit",
+ "EventCode": "0x114",
+ "EventName": "PAGE_WALK_L2_STAGE1_HIT",
+ "BriefDescription": "Page walk, L2 stage-1 hit"
+ },
+ {
+ "PublicDescription": "Page walk cache level-1 stage-2 hit",
+ "EventCode": "0x115",
+ "EventName": "PAGE_WALK_L1_STAGE2_HIT",
+ "BriefDescription": "Page walk, L1 stage-2 hit"
+ },
+ {
+ "PublicDescription": "Page walk cache level-2 stage-2 hit",
+ "EventCode": "0x116",
+ "EventName": "PAGE_WALK_L2_STAGE2_HIT",
+ "BriefDescription": "Page walk, L2 stage-2 hit"
+ },
+]
--- /dev/null
+[
+ {
+ "PublicDescription": "The number of core clock cycles",
+ "EventCode": "0x11",
+ "EventName": "CPU_CYCLES",
+ "BriefDescription": "Clock cycles"
+ },
+ {
+ "PublicDescription": "FSU clocking gated off cycle",
+ "EventCode": "0x101",
+ "EventName": "FSU_CLOCK_OFF_CYCLES",
+ "BriefDescription": "FSU clocking gated off cycle"
+ },
+ {
+ "PublicDescription": "Wait state cycle",
+ "EventCode": "0x110",
+ "EventName": "Wait_CYCLES",
+ "BriefDescription": "Wait state cycle"
+ },
+]
+++ /dev/null
-[
- {
- "ArchStdEvent": "L1D_CACHE_RD",
- },
- {
- "ArchStdEvent": "L1D_CACHE_WR",
- },
- {
- "ArchStdEvent": "L1D_CACHE_REFILL_RD",
- },
- {
- "ArchStdEvent": "L1D_CACHE_REFILL_WR",
- },
- {
- "ArchStdEvent": "L1D_TLB_REFILL_RD",
- },
- {
- "ArchStdEvent": "L1D_TLB_REFILL_WR",
- },
- {
- "ArchStdEvent": "L1D_TLB_RD",
- },
- {
- "ArchStdEvent": "L1D_TLB_WR",
- },
- {
- "ArchStdEvent": "BUS_ACCESS_RD",
- },
- {
- "ArchStdEvent": "BUS_ACCESS_WR",
- }
-]
--- /dev/null
+[
+ {
+ "ArchStdEvent": "EXC_UNDEF",
+ },
+ {
+ "ArchStdEvent": "EXC_SVC",
+ },
+ {
+ "ArchStdEvent": "EXC_PABORT",
+ },
+ {
+ "ArchStdEvent": "EXC_DABORT",
+ },
+ {
+ "ArchStdEvent": "EXC_IRQ",
+ },
+ {
+ "ArchStdEvent": "EXC_FIQ",
+ },
+ {
+ "ArchStdEvent": "EXC_HVC",
+ },
+ {
+ "ArchStdEvent": "EXC_TRAP_PABORT",
+ },
+ {
+ "ArchStdEvent": "EXC_TRAP_DABORT",
+ },
+ {
+ "ArchStdEvent": "EXC_TRAP_OTHER",
+ },
+ {
+ "ArchStdEvent": "EXC_TRAP_IRQ",
+ },
+ {
+ "ArchStdEvent": "EXC_TRAP_FIQ",
+ },
+ {
+ "PublicDescription": "Exception taken",
+ "EventCode": "0x09",
+ "EventName": "EXC_TAKEN",
+ "BriefDescription": "Exception taken"
+ },
+ {
+ "PublicDescription": "Instruction architecturally executed, condition check pass, exception return",
+ "EventCode": "0x0a",
+ "EventName": "EXC_RETURN",
+ "BriefDescription": "Exception return"
+ },
+]
--- /dev/null
+[
+ {
+ "ArchStdEvent": "LD_SPEC",
+ },
+ {
+ "ArchStdEvent": "ST_SPEC",
+ },
+ {
+ "ArchStdEvent": "LDST_SPEC",
+ },
+ {
+ "ArchStdEvent": "DP_SPEC",
+ },
+ {
+ "ArchStdEvent": "ASE_SPEC",
+ },
+ {
+ "ArchStdEvent": "VFP_SPEC",
+ },
+ {
+ "ArchStdEvent": "PC_WRITE_SPEC",
+ },
+ {
+ "ArchStdEvent": "CRYPTO_SPEC",
+ },
+ {
+ "ArchStdEvent": "ISB_SPEC",
+ },
+ {
+ "ArchStdEvent": "DSB_SPEC",
+ },
+ {
+ "ArchStdEvent": "DMB_SPEC",
+ },
+ {
+ "ArchStdEvent": "RC_LD_SPEC",
+ },
+ {
+ "ArchStdEvent": "RC_ST_SPEC",
+ },
+ {
+ "PublicDescription": "Instruction architecturally executed, software increment",
+ "EventCode": "0x00",
+ "EventName": "SW_INCR",
+ "BriefDescription": "Software increment"
+ },
+ {
+ "PublicDescription": "Instruction architecturally executed",
+ "EventCode": "0x08",
+ "EventName": "INST_RETIRED",
+ "BriefDescription": "Instruction retired"
+ },
+ {
+ "PublicDescription": "Instruction architecturally executed, condition code check pass, write to CONTEXTIDR",
+ "EventCode": "0x0b",
+ "EventName": "CID_WRITE_RETIRED",
+ "BriefDescription": "Write to CONTEXTIDR"
+ },
+ {
+ "PublicDescription": "Operation speculatively executed",
+ "EventCode": "0x1b",
+ "EventName": "INST_SPEC",
+ "BriefDescription": "Speculatively executed"
+ },
+ {
+ "PublicDescription": "Instruction architecturally executed (condition check pass), write to TTBR",
+ "EventCode": "0x1c",
+ "EventName": "TTBR_WRITE_RETIRED",
+ "BriefDescription": "Instruction executed, TTBR write"
+ },
+ {
+ "PublicDescription": "Instruction architecturally executed, branch. This event counts all branches, taken or not. This excludes exception entries, debug entries and CCFAIL branches",
+ "EventCode": "0x21",
+ "EventName": "BR_RETIRED",
+ "BriefDescription": "Branch retired"
+ },
+ {
+ "PublicDescription": "Instruction architecturally executed, mispredicted branch. This event counts any branch counted by BR_RETIRED which is not correctly predicted and causes a pipeline flush",
+ "EventCode": "0x22",
+ "EventName": "BR_MISPRED_RETIRED",
+ "BriefDescription": "Mispredicted branch retired"
+ },
+ {
+ "PublicDescription": "Operation speculatively executed, NOP",
+ "EventCode": "0x100",
+ "EventName": "NOP_SPEC",
+ "BriefDescription": "Speculatively executed, NOP"
+ },
+]
--- /dev/null
+[
+ {
+ "ArchStdEvent": "LDREX_SPEC",
+ },
+ {
+ "ArchStdEvent": "STREX_PASS_SPEC",
+ },
+ {
+ "ArchStdEvent": "STREX_FAIL_SPEC",
+ },
+ {
+ "ArchStdEvent": "STREX_SPEC",
+ },
+]
--- /dev/null
+[
+ {
+ "ArchStdEvent": "MEM_ACCESS_RD",
+ },
+ {
+ "ArchStdEvent": "MEM_ACCESS_WR",
+ },
+ {
+ "ArchStdEvent": "UNALIGNED_LD_SPEC",
+ },
+ {
+ "ArchStdEvent": "UNALIGNED_ST_SPEC",
+ },
+ {
+ "ArchStdEvent": "UNALIGNED_LDST_SPEC",
+ },
+ {
+ "PublicDescription": "Data memory access",
+ "EventCode": "0x13",
+ "EventName": "MEM_ACCESS",
+ "BriefDescription": "Memory access"
+ },
+ {
+ "PublicDescription": "Local memory error. This event counts any correctable or uncorrectable memory error (ECC or parity) in the protected core RAMs",
+ "EventCode": "0x1a",
+ "EventName": "MEM_ERROR",
+ "BriefDescription": "Memory error"
+ },
+]
--- /dev/null
+[
+ {
+ "PublicDescription": "Decode starved for instruction cycle",
+ "EventCode": "0x108",
+ "EventName": "DECODE_STALL",
+ "BriefDescription": "Decode starved"
+ },
+ {
+ "PublicDescription": "Op dispatch stalled cycle",
+ "EventCode": "0x109",
+ "EventName": "DISPATCH_STALL",
+ "BriefDescription": "Dispatch stalled"
+ },
+ {
+ "PublicDescription": "IXA Op non-issue",
+ "EventCode": "0x10a",
+ "EventName": "IXA_STALL",
+ "BriefDescription": "IXA stalled"
+ },
+ {
+ "PublicDescription": "IXB Op non-issue",
+ "EventCode": "0x10b",
+ "EventName": "IXB_STALL",
+ "BriefDescription": "IXB stalled"
+ },
+ {
+ "PublicDescription": "BX Op non-issue",
+ "EventCode": "0x10c",
+ "EventName": "BX_STALL",
+ "BriefDescription": "BX stalled"
+ },
+ {
+ "PublicDescription": "LX Op non-issue",
+ "EventCode": "0x10d",
+ "EventName": "LX_STALL",
+ "BriefDescription": "LX stalled"
+ },
+ {
+ "PublicDescription": "SX Op non-issue",
+ "EventCode": "0x10e",
+ "EventName": "SX_STALL",
+ "BriefDescription": "SX stalled"
+ },
+ {
+ "PublicDescription": "FX Op non-issue",
+ "EventCode": "0x10f",
+ "EventName": "FX_STALL",
+ "BriefDescription": "FX stalled"
+ },
+]
"Counter": "0,1,2,3",
"EventCode": "0xb",
"EventName": "UNC_P_FREQ_GE_1200MHZ_CYCLES",
- "Filter": "filter_band0=1200",
+ "Filter": "filter_band0=12",
"MetricExpr": "(UNC_P_FREQ_GE_1200MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_1200mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xc",
"EventName": "UNC_P_FREQ_GE_2000MHZ_CYCLES",
- "Filter": "filter_band1=2000",
+ "Filter": "filter_band1=20",
"MetricExpr": "(UNC_P_FREQ_GE_2000MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_2000mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xd",
"EventName": "UNC_P_FREQ_GE_3000MHZ_CYCLES",
- "Filter": "filter_band2=3000",
+ "Filter": "filter_band2=30",
"MetricExpr": "(UNC_P_FREQ_GE_3000MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_3000mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xe",
"EventName": "UNC_P_FREQ_GE_4000MHZ_CYCLES",
- "Filter": "filter_band3=4000",
+ "Filter": "filter_band3=40",
"MetricExpr": "(UNC_P_FREQ_GE_4000MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_4000mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xb",
"EventName": "UNC_P_FREQ_GE_1200MHZ_TRANSITIONS",
- "Filter": "edge=1,filter_band0=1200",
+ "Filter": "edge=1,filter_band0=12",
"MetricExpr": "(UNC_P_FREQ_GE_1200MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_1200mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xc",
"EventName": "UNC_P_FREQ_GE_2000MHZ_TRANSITIONS",
- "Filter": "edge=1,filter_band1=2000",
+ "Filter": "edge=1,filter_band1=20",
"MetricExpr": "(UNC_P_FREQ_GE_2000MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_2000mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xd",
"EventName": "UNC_P_FREQ_GE_3000MHZ_TRANSITIONS",
- "Filter": "edge=1,filter_band2=4000",
+ "Filter": "edge=1,filter_band2=30",
"MetricExpr": "(UNC_P_FREQ_GE_3000MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_3000mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xe",
"EventName": "UNC_P_FREQ_GE_4000MHZ_TRANSITIONS",
- "Filter": "edge=1,filter_band3=4000",
+ "Filter": "edge=1,filter_band3=40",
"MetricExpr": "(UNC_P_FREQ_GE_4000MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_4000mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xb",
"EventName": "UNC_P_FREQ_GE_1200MHZ_CYCLES",
- "Filter": "filter_band0=1200",
+ "Filter": "filter_band0=12",
"MetricExpr": "(UNC_P_FREQ_GE_1200MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_1200mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xc",
"EventName": "UNC_P_FREQ_GE_2000MHZ_CYCLES",
- "Filter": "filter_band1=2000",
+ "Filter": "filter_band1=20",
"MetricExpr": "(UNC_P_FREQ_GE_2000MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_2000mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xd",
"EventName": "UNC_P_FREQ_GE_3000MHZ_CYCLES",
- "Filter": "filter_band2=3000",
+ "Filter": "filter_band2=30",
"MetricExpr": "(UNC_P_FREQ_GE_3000MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_3000mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xe",
"EventName": "UNC_P_FREQ_GE_4000MHZ_CYCLES",
- "Filter": "filter_band3=4000",
+ "Filter": "filter_band3=40",
"MetricExpr": "(UNC_P_FREQ_GE_4000MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_4000mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xb",
"EventName": "UNC_P_FREQ_GE_1200MHZ_TRANSITIONS",
- "Filter": "edge=1,filter_band0=1200",
+ "Filter": "edge=1,filter_band0=12",
"MetricExpr": "(UNC_P_FREQ_GE_1200MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_1200mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xc",
"EventName": "UNC_P_FREQ_GE_2000MHZ_TRANSITIONS",
- "Filter": "edge=1,filter_band1=2000",
+ "Filter": "edge=1,filter_band1=20",
"MetricExpr": "(UNC_P_FREQ_GE_2000MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_2000mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xd",
"EventName": "UNC_P_FREQ_GE_3000MHZ_TRANSITIONS",
- "Filter": "edge=1,filter_band2=4000",
+ "Filter": "edge=1,filter_band2=30",
"MetricExpr": "(UNC_P_FREQ_GE_3000MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_3000mhz_cycles %",
"PerPkg": "1",
"Counter": "0,1,2,3",
"EventCode": "0xe",
"EventName": "UNC_P_FREQ_GE_4000MHZ_TRANSITIONS",
- "Filter": "edge=1,filter_band3=4000",
+ "Filter": "edge=1,filter_band3=40",
"MetricExpr": "(UNC_P_FREQ_GE_4000MHZ_CYCLES / UNC_P_CLOCKTICKS) * 100.",
"MetricName": "freq_ge_4000mhz_cycles %",
"PerPkg": "1",
libpq = CDLL("libpq.so.5")
PQconnectdb = libpq.PQconnectdb
PQconnectdb.restype = c_void_p
+PQconnectdb.argtypes = [ c_char_p ]
PQfinish = libpq.PQfinish
+PQfinish.argtypes = [ c_void_p ]
PQstatus = libpq.PQstatus
+PQstatus.restype = c_int
+PQstatus.argtypes = [ c_void_p ]
PQexec = libpq.PQexec
PQexec.restype = c_void_p
+PQexec.argtypes = [ c_void_p, c_char_p ]
PQresultStatus = libpq.PQresultStatus
+PQresultStatus.restype = c_int
+PQresultStatus.argtypes = [ c_void_p ]
PQputCopyData = libpq.PQputCopyData
+PQputCopyData.restype = c_int
PQputCopyData.argtypes = [ c_void_p, c_void_p, c_int ]
PQputCopyEnd = libpq.PQputCopyEnd
+PQputCopyEnd.restype = c_int
PQputCopyEnd.argtypes = [ c_void_p, c_void_p ]
sys.path.append(os.environ['PERF_EXEC_PATH'] + \
def sample_table(*x):
if branches:
- bind_exec(sample_query, 18, x)
+ for xx in x[0:15]:
+ sample_query.addBindValue(str(xx))
+ for xx in x[19:22]:
+ sample_query.addBindValue(str(xx))
+ do_query_(sample_query)
else:
bind_exec(sample_query, 22, x)
perf-y += bp_signal.o
perf-y += bp_signal_overflow.o
perf-y += bp_account.o
+perf-y += wp.o
perf-y += task-exit.o
perf-y += sw-clock.o
perf-y += mmap-thread-lookup.o
.func = test__bp_accounting,
.is_supported = test__bp_signal_is_supported,
},
+ {
+ .desc = "Watchpoint",
+ .func = test__wp,
+ .is_supported = test__wp_is_supported,
+ .subtest = {
+ .skip_if_fail = false,
+ .get_nr = test__wp_subtest_get_nr,
+ .get_desc = test__wp_subtest_get_desc,
+ },
+ },
{
.desc = "Number of exit events of a simple workload",
.func = test__task_exit,
static int perf_evsel__test_field(struct perf_evsel *evsel, const char *name,
int size, bool should_be_signed)
{
- struct format_field *field = perf_evsel__field(evsel, name);
+ struct tep_format_field *field = perf_evsel__field(evsel, name);
int is_signed;
int ret = 0;
return -1;
}
- is_signed = !!(field->flags | FIELD_IS_SIGNED);
+ is_signed = !!(field->flags | TEP_FIELD_IS_SIGNED);
if (should_be_signed && !is_signed) {
pr_debug("%s: \"%s\" signedness(%d) is wrong, should be %d\n",
evsel->name, name, is_signed, should_be_signed);
*)
eventattr='max-stack=3'
echo "getaddrinfo\+0x[[:xdigit:]]+[[:space:]]\($libc\)$" >> $expected
- echo ".*\+0x[[:xdigit:]]+[[:space:]]\(.*/bin/ping.*\)$" >> $expected
+ echo ".*(\+0x[[:xdigit:]]+|\[unknown\])[[:space:]]\(.*/bin/ping.*\)$" >> $expected
;;
esac
int test__bp_signal(struct test *test, int subtest);
int test__bp_signal_overflow(struct test *test, int subtest);
int test__bp_accounting(struct test *test, int subtest);
+int test__wp(struct test *test, int subtest);
+const char *test__wp_subtest_get_desc(int subtest);
+int test__wp_subtest_get_nr(void);
int test__task_exit(struct test *test, int subtest);
int test__mem(struct test *test, int subtest);
int test__sw_clock_freq(struct test *test, int subtest);
int test__mem2node(struct test *t, int subtest);
bool test__bp_signal_is_supported(void);
+bool test__wp_is_supported(void);
#if defined(__arm__) || defined(__aarch64__)
#ifdef HAVE_DWARF_UNWIND_SUPPORT
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <stdlib.h>
+#include <sys/ioctl.h>
+#include <linux/hw_breakpoint.h>
+#include "tests.h"
+#include "debug.h"
+#include "cloexec.h"
+
+#define WP_TEST_ASSERT_VAL(fd, text, val) \
+do { \
+ long long count; \
+ wp_read(fd, &count, sizeof(long long)); \
+ TEST_ASSERT_VAL(text, count == val); \
+} while (0)
+
+volatile u64 data1;
+volatile u8 data2[3];
+
+static int wp_read(int fd, long long *count, int size)
+{
+ int ret = read(fd, count, size);
+
+ if (ret != size) {
+ pr_debug("failed to read: %d\n", ret);
+ return -1;
+ }
+ return 0;
+}
+
+static void get__perf_event_attr(struct perf_event_attr *attr, int wp_type,
+ void *wp_addr, unsigned long wp_len)
+{
+ memset(attr, 0, sizeof(struct perf_event_attr));
+ attr->type = PERF_TYPE_BREAKPOINT;
+ attr->size = sizeof(struct perf_event_attr);
+ attr->config = 0;
+ attr->bp_type = wp_type;
+ attr->bp_addr = (unsigned long)wp_addr;
+ attr->bp_len = wp_len;
+ attr->sample_period = 1;
+ attr->sample_type = PERF_SAMPLE_IP;
+ attr->exclude_kernel = 1;
+ attr->exclude_hv = 1;
+}
+
+static int __event(int wp_type, void *wp_addr, unsigned long wp_len)
+{
+ int fd;
+ struct perf_event_attr attr;
+
+ get__perf_event_attr(&attr, wp_type, wp_addr, wp_len);
+ fd = sys_perf_event_open(&attr, 0, -1, -1,
+ perf_event_open_cloexec_flag());
+ if (fd < 0)
+ pr_debug("failed opening event %x\n", attr.bp_type);
+
+ return fd;
+}
+
+static int wp_ro_test(void)
+{
+ int fd;
+ unsigned long tmp, tmp1 = rand();
+
+ fd = __event(HW_BREAKPOINT_R, (void *)&data1, sizeof(data1));
+ if (fd < 0)
+ return -1;
+
+ tmp = data1;
+ WP_TEST_ASSERT_VAL(fd, "RO watchpoint", 1);
+
+ data1 = tmp1 + tmp;
+ WP_TEST_ASSERT_VAL(fd, "RO watchpoint", 1);
+
+ close(fd);
+ return 0;
+}
+
+static int wp_wo_test(void)
+{
+ int fd;
+ unsigned long tmp, tmp1 = rand();
+
+ fd = __event(HW_BREAKPOINT_W, (void *)&data1, sizeof(data1));
+ if (fd < 0)
+ return -1;
+
+ tmp = data1;
+ WP_TEST_ASSERT_VAL(fd, "WO watchpoint", 0);
+
+ data1 = tmp1 + tmp;
+ WP_TEST_ASSERT_VAL(fd, "WO watchpoint", 1);
+
+ close(fd);
+ return 0;
+}
+
+static int wp_rw_test(void)
+{
+ int fd;
+ unsigned long tmp, tmp1 = rand();
+
+ fd = __event(HW_BREAKPOINT_R | HW_BREAKPOINT_W, (void *)&data1,
+ sizeof(data1));
+ if (fd < 0)
+ return -1;
+
+ tmp = data1;
+ WP_TEST_ASSERT_VAL(fd, "RW watchpoint", 1);
+
+ data1 = tmp1 + tmp;
+ WP_TEST_ASSERT_VAL(fd, "RW watchpoint", 2);
+
+ close(fd);
+ return 0;
+}
+
+static int wp_modify_test(void)
+{
+ int fd, ret;
+ unsigned long tmp = rand();
+ struct perf_event_attr new_attr;
+
+ fd = __event(HW_BREAKPOINT_W, (void *)&data1, sizeof(data1));
+ if (fd < 0)
+ return -1;
+
+ data1 = tmp;
+ WP_TEST_ASSERT_VAL(fd, "Modify watchpoint", 1);
+
+ /* Modify watchpoint with disabled = 1 */
+ get__perf_event_attr(&new_attr, HW_BREAKPOINT_W, (void *)&data2[0],
+ sizeof(u8) * 2);
+ new_attr.disabled = 1;
+ ret = ioctl(fd, PERF_EVENT_IOC_MODIFY_ATTRIBUTES, &new_attr);
+ if (ret < 0) {
+ pr_debug("ioctl(PERF_EVENT_IOC_MODIFY_ATTRIBUTES) failed\n");
+ close(fd);
+ return ret;
+ }
+
+ data2[1] = tmp; /* Not Counted */
+ WP_TEST_ASSERT_VAL(fd, "Modify watchpoint", 1);
+
+ /* Enable the event */
+ ioctl(fd, PERF_EVENT_IOC_ENABLE, 0);
+ if (ret < 0) {
+ pr_debug("Failed to enable event\n");
+ close(fd);
+ return ret;
+ }
+
+ data2[1] = tmp; /* Counted */
+ WP_TEST_ASSERT_VAL(fd, "Modify watchpoint", 2);
+
+ data2[2] = tmp; /* Not Counted */
+ WP_TEST_ASSERT_VAL(fd, "Modify watchpoint", 2);
+
+ close(fd);
+ return 0;
+}
+
+static bool wp_ro_supported(void)
+{
+#if defined (__x86_64__) || defined (__i386__)
+ return false;
+#else
+ return true;
+#endif
+}
+
+static void wp_ro_skip_msg(void)
+{
+#if defined (__x86_64__) || defined (__i386__)
+ pr_debug("Hardware does not support read only watchpoints.\n");
+#endif
+}
+
+static struct {
+ const char *desc;
+ int (*target_func)(void);
+ bool (*is_supported)(void);
+ void (*skip_msg)(void);
+} wp_testcase_table[] = {
+ {
+ .desc = "Read Only Watchpoint",
+ .target_func = &wp_ro_test,
+ .is_supported = &wp_ro_supported,
+ .skip_msg = &wp_ro_skip_msg,
+ },
+ {
+ .desc = "Write Only Watchpoint",
+ .target_func = &wp_wo_test,
+ },
+ {
+ .desc = "Read / Write Watchpoint",
+ .target_func = &wp_rw_test,
+ },
+ {
+ .desc = "Modify Watchpoint",
+ .target_func = &wp_modify_test,
+ },
+};
+
+int test__wp_subtest_get_nr(void)
+{
+ return (int)ARRAY_SIZE(wp_testcase_table);
+}
+
+const char *test__wp_subtest_get_desc(int i)
+{
+ if (i < 0 || i >= (int)ARRAY_SIZE(wp_testcase_table))
+ return NULL;
+ return wp_testcase_table[i].desc;
+}
+
+int test__wp(struct test *test __maybe_unused, int i)
+{
+ if (i < 0 || i >= (int)ARRAY_SIZE(wp_testcase_table))
+ return TEST_FAIL;
+
+ if (wp_testcase_table[i].is_supported &&
+ !wp_testcase_table[i].is_supported()) {
+ wp_testcase_table[i].skip_msg();
+ return TEST_SKIP;
+ }
+
+ return !wp_testcase_table[i].target_func() ? TEST_OK : TEST_FAIL;
+}
+
+/* The s390 so far does not have support for
+ * instruction breakpoint using the perf_event_open() system call.
+ */
+bool test__wp_is_supported(void)
+{
+#if defined(__s390x__)
+ return false;
+#else
+ return true;
+#endif
+}
libperf-y += kcmp.o
libperf-y += pkey_alloc.o
libperf-y += prctl.o
+libperf-y += sockaddr.o
libperf-y += socket.o
libperf-y += statx.o
size_t pid__scnprintf_fd(struct trace *trace, pid_t pid, int fd, char *bf, size_t size);
+extern struct strarray strarray__socket_families;
+
+/**
+ * augmented_arg: extra payload for syscall pointer arguments
+
+ * If perf_sample->raw_size is more than what a syscall sys_enter_FOO puts,
+ * then its the arguments contents, so that we can show more than just a
+ * pointer. This will be done initially with eBPF, the start of that is at the
+ * tools/perf/examples/bpf/augmented_syscalls.c example for the openat, but
+ * will eventually be done automagically caching the running kernel tracefs
+ * events data into an eBPF C script, that then gets compiled and its .o file
+ * cached for subsequent use. For char pointers like the ones for 'open' like
+ * syscalls its easy, for the rest we should use DWARF or better, BTF, much
+ * more compact.
+ *
+ * @size: 8 if all we need is an integer, otherwise all of the augmented arg.
+ * @int_arg: will be used for integer like pointer contents, like 'accept's 'upeer_addrlen'
+ * @value: u64 aligned, for structs, pathnames
+ */
+struct augmented_arg {
+ int size;
+ int int_arg;
+ u64 value[];
+};
+
/**
* @val: value of syscall argument being formatted
* @args: All the args, use syscall_args__val(arg, nth) to access one
+ * @augmented_args: Extra data that can be collected, for instance, with eBPF for expanding the pathname for open, etc
+ * @augmented_args_size: augmented_args total payload size
* @thread: tid state (maps, pid, tid, etc)
* @trace: 'perf trace' internals: all threads, etc
* @parm: private area, may be an strarray, for instance
struct syscall_arg {
unsigned long val;
unsigned char *args;
+ struct {
+ struct augmented_arg *args;
+ int size;
+ } augmented;
struct thread *thread;
struct trace *trace;
void *parm;
size_t syscall_arg__scnprintf_prctl_arg3(char *bf, size_t size, struct syscall_arg *arg);
#define SCA_PRCTL_ARG3 syscall_arg__scnprintf_prctl_arg3
+size_t syscall_arg__scnprintf_sockaddr(char *bf, size_t size, struct syscall_arg *arg);
+#define SCA_SOCKADDR syscall_arg__scnprintf_sockaddr
+
size_t syscall_arg__scnprintf_socket_protocol(char *bf, size_t size, struct syscall_arg *arg);
#define SCA_SK_PROTO syscall_arg__scnprintf_socket_protocol
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018, Red Hat Inc, Arnaldo Carvalho de Melo <acme@redhat.com>
+
+#include "trace/beauty/beauty.h"
+#include <sys/socket.h>
+#include <sys/types.h>
+#include <sys/un.h>
+#include <arpa/inet.h>
+
+static const char *socket_families[] = {
+ "UNSPEC", "LOCAL", "INET", "AX25", "IPX", "APPLETALK", "NETROM",
+ "BRIDGE", "ATMPVC", "X25", "INET6", "ROSE", "DECnet", "NETBEUI",
+ "SECURITY", "KEY", "NETLINK", "PACKET", "ASH", "ECONET", "ATMSVC",
+ "RDS", "SNA", "IRDA", "PPPOX", "WANPIPE", "LLC", "IB", "CAN", "TIPC",
+ "BLUETOOTH", "IUCV", "RXRPC", "ISDN", "PHONET", "IEEE802154", "CAIF",
+ "ALG", "NFC", "VSOCK",
+};
+DEFINE_STRARRAY(socket_families);
+
+static size_t af_inet__scnprintf(struct sockaddr *sa, char *bf, size_t size)
+{
+ struct sockaddr_in *sin = (struct sockaddr_in *)sa;
+ char tmp[16];
+ return scnprintf(bf, size, ", port: %d, addr: %s", ntohs(sin->sin_port),
+ inet_ntop(sin->sin_family, &sin->sin_addr, tmp, sizeof(tmp)));
+}
+
+static size_t af_inet6__scnprintf(struct sockaddr *sa, char *bf, size_t size)
+{
+ struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)sa;
+ u32 flowinfo = ntohl(sin6->sin6_flowinfo);
+ char tmp[512];
+ size_t printed = scnprintf(bf, size, ", port: %d, addr: %s", ntohs(sin6->sin6_port),
+ inet_ntop(sin6->sin6_family, &sin6->sin6_addr, tmp, sizeof(tmp)));
+ if (flowinfo != 0)
+ printed += scnprintf(bf + printed, size - printed, ", flowinfo: %lu", flowinfo);
+ if (sin6->sin6_scope_id != 0)
+ printed += scnprintf(bf + printed, size - printed, ", scope_id: %lu", sin6->sin6_scope_id);
+
+ return printed;
+}
+
+static size_t af_local__scnprintf(struct sockaddr *sa, char *bf, size_t size)
+{
+ struct sockaddr_un *sun = (struct sockaddr_un *)sa;
+ return scnprintf(bf, size, ", path: %s", sun->sun_path);
+}
+
+static size_t (*af_scnprintfs[])(struct sockaddr *sa, char *bf, size_t size) = {
+ [AF_LOCAL] = af_local__scnprintf,
+ [AF_INET] = af_inet__scnprintf,
+ [AF_INET6] = af_inet6__scnprintf,
+};
+
+static size_t syscall_arg__scnprintf_augmented_sockaddr(struct syscall_arg *arg, char *bf, size_t size)
+{
+ struct sockaddr *sa = (struct sockaddr *)arg->augmented.args;
+ char family[32];
+ size_t printed;
+
+ strarray__scnprintf(&strarray__socket_families, family, sizeof(family), "%d", sa->sa_family);
+ printed = scnprintf(bf, size, "{ .family: %s", family);
+
+ if (sa->sa_family < ARRAY_SIZE(af_scnprintfs) && af_scnprintfs[sa->sa_family])
+ printed += af_scnprintfs[sa->sa_family](sa, bf + printed, size - printed);
+
+ return printed + scnprintf(bf + printed, size - printed, " }");
+}
+
+size_t syscall_arg__scnprintf_sockaddr(char *bf, size_t size, struct syscall_arg *arg)
+{
+ if (arg->augmented.args)
+ return syscall_arg__scnprintf_augmented_sockaddr(arg, bf, size);
+
+ return scnprintf(bf, size, "%#x", arg->val);
+}
libperf-y += counts.o
libperf-y += stat.o
libperf-y += stat-shadow.o
+libperf-y += stat-display.o
libperf-y += record.o
libperf-y += srcline.o
libperf-y += data.o
return err;
}
-int perf_event__process_auxtrace_info(struct perf_tool *tool __maybe_unused,
- union perf_event *event,
- struct perf_session *session)
+int perf_event__process_auxtrace_info(struct perf_session *session,
+ union perf_event *event)
{
enum auxtrace_type type = event->auxtrace_info.type;
}
}
-s64 perf_event__process_auxtrace(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session)
+s64 perf_event__process_auxtrace(struct perf_session *session,
+ union perf_event *event)
{
s64 err;
if (!session->auxtrace || event->header.type != PERF_RECORD_AUXTRACE)
return -EINVAL;
- err = session->auxtrace->process_auxtrace_event(session, event, tool);
+ err = session->auxtrace->process_auxtrace_event(session, event, session->tool);
if (err < 0)
return err;
}
}
-int perf_event__process_auxtrace_error(struct perf_tool *tool __maybe_unused,
- union perf_event *event,
- struct perf_session *session)
+int perf_event__process_auxtrace_error(struct perf_session *session,
+ union perf_event *event)
{
if (auxtrace__dont_decode(session))
return 0;
return 0;
}
-static int __auxtrace_mmap__read(struct auxtrace_mmap *mm,
+static int __auxtrace_mmap__read(struct perf_mmap *map,
struct auxtrace_record *itr,
struct perf_tool *tool, process_auxtrace_t fn,
bool snapshot, size_t snapshot_size)
{
+ struct auxtrace_mmap *mm = &map->auxtrace_mmap;
u64 head, old = mm->prev, offset, ref;
unsigned char *data = mm->base;
size_t size, head_off, old_off, len1, len2, padding;
ev.auxtrace.tid = mm->tid;
ev.auxtrace.cpu = mm->cpu;
- if (fn(tool, &ev, data1, len1, data2, len2))
+ if (fn(tool, map, &ev, data1, len1, data2, len2))
return -1;
mm->prev = head;
return 1;
}
-int auxtrace_mmap__read(struct auxtrace_mmap *mm, struct auxtrace_record *itr,
+int auxtrace_mmap__read(struct perf_mmap *map, struct auxtrace_record *itr,
struct perf_tool *tool, process_auxtrace_t fn)
{
- return __auxtrace_mmap__read(mm, itr, tool, fn, false, 0);
+ return __auxtrace_mmap__read(map, itr, tool, fn, false, 0);
}
-int auxtrace_mmap__read_snapshot(struct auxtrace_mmap *mm,
+int auxtrace_mmap__read_snapshot(struct perf_mmap *map,
struct auxtrace_record *itr,
struct perf_tool *tool, process_auxtrace_t fn,
size_t snapshot_size)
{
- return __auxtrace_mmap__read(mm, itr, tool, fn, true, snapshot_size);
+ return __auxtrace_mmap__read(map, itr, tool, fn, true, snapshot_size);
}
/**
#include <linux/list.h>
#include <linux/perf_event.h>
#include <linux/types.h>
+#include <asm/bitsperlong.h>
#include "../perf.h"
#include "event.h"
struct perf_session;
struct perf_evlist;
struct perf_tool;
+struct perf_mmap;
struct option;
struct record_opts;
struct auxtrace_info_event;
bool per_cpu);
typedef int (*process_auxtrace_t)(struct perf_tool *tool,
+ struct perf_mmap *map,
union perf_event *event, void *data1,
size_t len1, void *data2, size_t len2);
-int auxtrace_mmap__read(struct auxtrace_mmap *mm, struct auxtrace_record *itr,
+int auxtrace_mmap__read(struct perf_mmap *map, struct auxtrace_record *itr,
struct perf_tool *tool, process_auxtrace_t fn);
-int auxtrace_mmap__read_snapshot(struct auxtrace_mmap *mm,
+int auxtrace_mmap__read_snapshot(struct perf_mmap *map,
struct auxtrace_record *itr,
struct perf_tool *tool, process_auxtrace_t fn,
size_t snapshot_size);
struct perf_tool *tool,
struct perf_session *session,
perf_event__handler_t process);
-int perf_event__process_auxtrace_info(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session);
-s64 perf_event__process_auxtrace(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session);
-int perf_event__process_auxtrace_error(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session);
+int perf_event__process_auxtrace_info(struct perf_session *session,
+ union perf_event *event);
+s64 perf_event__process_auxtrace(struct perf_session *session,
+ union perf_event *event);
+int perf_event__process_auxtrace_error(struct perf_session *session,
+ union perf_event *event);
int itrace_parse_synth_opts(const struct option *opt, const char *str,
int unset);
void itrace_synth_opts__set_default(struct itrace_synth_opts *synth_opts);
return session->auxtrace->free(session);
}
+#define ITRACE_HELP \
+" i: synthesize instructions events\n" \
+" b: synthesize branches events\n" \
+" c: synthesize branches events (calls only)\n" \
+" r: synthesize branches events (returns only)\n" \
+" x: synthesize transactions events\n" \
+" w: synthesize ptwrite events\n" \
+" p: synthesize power events\n" \
+" e: synthesize error events\n" \
+" d: create a debug log\n" \
+" g[len]: synthesize a call chain (use with i or x)\n" \
+" l[len]: synthesize last branch entries (use with i or x)\n" \
+" sNUMBER: skip initial number of events\n" \
+" PERIOD[ns|us|ms|i|t]: specify period to sample stream\n" \
+" concatenate multiple options. Default is ibxwpe or cewp\n"
+
+
#else
static inline struct auxtrace_record *
struct perf_evlist *evlist, int idx,
bool per_cpu);
+#define ITRACE_HELP ""
+
#endif
#endif
int bpf__setup_stdout(struct perf_evlist *evlist)
{
struct perf_evsel *evsel = bpf__setup_output_event(evlist, "__bpf_stdout__");
- return IS_ERR(evsel) ? PTR_ERR(evsel) : 0;
+ return PTR_ERR_OR_ZERO(evsel);
}
#define ERRNO_OFFSET(e) ((e) - __BPF_LOADER_ERRNO__START)
}
static struct bt_ctf_field_type*
-get_tracepoint_field_type(struct ctf_writer *cw, struct format_field *field)
+get_tracepoint_field_type(struct ctf_writer *cw, struct tep_format_field *field)
{
unsigned long flags = field->flags;
- if (flags & FIELD_IS_STRING)
+ if (flags & TEP_FIELD_IS_STRING)
return cw->data.string;
- if (!(flags & FIELD_IS_SIGNED)) {
+ if (!(flags & TEP_FIELD_IS_SIGNED)) {
/* unsigned long are mostly pointers */
- if (flags & FIELD_IS_LONG || flags & FIELD_IS_POINTER)
+ if (flags & TEP_FIELD_IS_LONG || flags & TEP_FIELD_IS_POINTER)
return cw->data.u64_hex;
}
- if (flags & FIELD_IS_SIGNED) {
+ if (flags & TEP_FIELD_IS_SIGNED) {
if (field->size == 8)
return cw->data.s64;
else
struct bt_ctf_event_class *event_class,
struct bt_ctf_event *event,
struct perf_sample *sample,
- struct format_field *fmtf)
+ struct tep_format_field *fmtf)
{
struct bt_ctf_field_type *type;
struct bt_ctf_field *array_field;
name = fmtf->alias;
offset = fmtf->offset;
len = fmtf->size;
- if (flags & FIELD_IS_STRING)
- flags &= ~FIELD_IS_ARRAY;
+ if (flags & TEP_FIELD_IS_STRING)
+ flags &= ~TEP_FIELD_IS_ARRAY;
- if (flags & FIELD_IS_DYNAMIC) {
+ if (flags & TEP_FIELD_IS_DYNAMIC) {
unsigned long long tmp_val;
tmp_val = tep_read_number(fmtf->event->pevent,
offset &= 0xffff;
}
- if (flags & FIELD_IS_ARRAY) {
+ if (flags & TEP_FIELD_IS_ARRAY) {
type = bt_ctf_event_class_get_field_by_name(
event_class, name);
type = get_tracepoint_field_type(cw, fmtf);
for (i = 0; i < n_items; i++) {
- if (flags & FIELD_IS_ARRAY)
+ if (flags & TEP_FIELD_IS_ARRAY)
field = bt_ctf_field_array_get_field(array_field, i);
else
field = bt_ctf_field_create(type);
return -1;
}
- if (flags & FIELD_IS_STRING)
+ if (flags & TEP_FIELD_IS_STRING)
ret = string_set_value(field, data + offset + i * len);
else {
unsigned long long value_int;
fmtf->event->pevent,
data + offset + i * len, len);
- if (!(flags & FIELD_IS_SIGNED))
+ if (!(flags & TEP_FIELD_IS_SIGNED))
ret = bt_ctf_field_unsigned_integer_set_value(
field, value_int);
else
pr_err("failed to set file value %s\n", name);
goto err_put_field;
}
- if (!(flags & FIELD_IS_ARRAY)) {
+ if (!(flags & TEP_FIELD_IS_ARRAY)) {
ret = bt_ctf_event_set_payload(event, name, field);
if (ret) {
pr_err("failed to set payload %s\n", name);
}
bt_ctf_field_put(field);
}
- if (flags & FIELD_IS_ARRAY) {
+ if (flags & TEP_FIELD_IS_ARRAY) {
ret = bt_ctf_event_set_payload(event, name, array_field);
if (ret) {
pr_err("Failed add payload array %s\n", name);
static int add_tracepoint_fields_values(struct ctf_writer *cw,
struct bt_ctf_event_class *event_class,
struct bt_ctf_event *event,
- struct format_field *fields,
+ struct tep_format_field *fields,
struct perf_sample *sample)
{
- struct format_field *field;
+ struct tep_format_field *field;
int ret;
for (field = fields; field; field = field->next) {
struct perf_evsel *evsel,
struct perf_sample *sample)
{
- struct format_field *common_fields = evsel->tp_format->format.common_fields;
- struct format_field *fields = evsel->tp_format->format.fields;
+ struct tep_format_field *common_fields = evsel->tp_format->format.common_fields;
+ struct tep_format_field *fields = evsel->tp_format->format.fields;
int ret;
ret = add_tracepoint_fields_values(cw, event_class, event,
static int event_class_add_field(struct bt_ctf_event_class *event_class,
struct bt_ctf_field_type *type,
- struct format_field *field)
+ struct tep_format_field *field)
{
struct bt_ctf_field_type *t = NULL;
char *name;
}
static int add_tracepoint_fields_types(struct ctf_writer *cw,
- struct format_field *fields,
+ struct tep_format_field *fields,
struct bt_ctf_event_class *event_class)
{
- struct format_field *field;
+ struct tep_format_field *field;
int ret;
for (field = fields; field; field = field->next) {
* type and don't care that it is an array. What we don't
* support is an array of strings.
*/
- if (flags & FIELD_IS_STRING)
- flags &= ~FIELD_IS_ARRAY;
+ if (flags & TEP_FIELD_IS_STRING)
+ flags &= ~TEP_FIELD_IS_ARRAY;
- if (flags & FIELD_IS_ARRAY)
+ if (flags & TEP_FIELD_IS_ARRAY)
type = bt_ctf_field_type_array_create(type, field->arraylen);
ret = event_class_add_field(event_class, type, field);
- if (flags & FIELD_IS_ARRAY)
+ if (flags & TEP_FIELD_IS_ARRAY)
bt_ctf_field_type_put(type);
if (ret) {
struct perf_evsel *evsel,
struct bt_ctf_event_class *class)
{
- struct format_field *common_fields = evsel->tp_format->format.common_fields;
- struct format_field *fields = evsel->tp_format->format.fields;
+ struct tep_format_field *common_fields = evsel->tp_format->format.common_fields;
+ struct tep_format_field *fields = evsel->tp_format->format.fields;
int ret;
ret = add_tracepoint_fields_types(cw, common_fields, class);
{
struct perf_session *session;
struct perf_data data = {
- .file.path = input,
+ .file = { .path = input, .fd = -1 },
.mode = PERF_DATA_MODE_READ,
.force = opts->force,
};
if (err)
break;
}
+
+ /* Add trace begin / end variants */
+ for (i = 0; branch_types[i].name ; i++) {
+ const char *name = branch_types[i].name;
+ u32 type = branch_types[i].branch_type;
+ char buf[64];
+
+ if (type == PERF_IP_FLAG_BRANCH ||
+ (type & (PERF_IP_FLAG_TRACE_BEGIN | PERF_IP_FLAG_TRACE_END)))
+ continue;
+
+ snprintf(buf, sizeof(buf), "trace begin / %s", name);
+ err = db_export__branch_type(dbe, type | PERF_IP_FLAG_TRACE_BEGIN, buf);
+ if (err)
+ break;
+
+ snprintf(buf, sizeof(buf), "%s / trace end", name);
+ err = db_export__branch_type(dbe, type | PERF_IP_FLAG_TRACE_END, buf);
+ if (err)
+ break;
+ }
+
return err;
}
}
*size += sizeof(struct cpu_map_data);
+ *size = PERF_ALIGN(*size, sizeof(u64));
return zalloc(*size);
}
return NULL;
}
-try_again:
+
al->map = map_groups__find(mg, al->addr);
- if (al->map == NULL) {
- /*
- * If this is outside of all known maps, and is a negative
- * address, try to look it up in the kernel dso, as it might be
- * a vsyscall or vdso (which executes in user-mode).
- *
- * XXX This is nasty, we should have a symbol list in the
- * "[vdso]" dso, but for now lets use the old trick of looking
- * in the whole kernel symbol list.
- */
- if (cpumode == PERF_RECORD_MISC_USER && machine &&
- mg != &machine->kmaps &&
- machine__kernel_ip(machine, al->addr)) {
- mg = &machine->kmaps;
- load_map = true;
- goto try_again;
- }
- } else {
+ if (al->map != NULL) {
/*
* Kernel maps might be changed when loading symbols so loading
* must be done prior to using kernel maps.
attr->exclude_user = 1;
}
+ if (evsel->own_cpus)
+ evsel->attr.read_format |= PERF_FORMAT_ID;
+
/*
* Apply event specific term settings,
* it overloads any global configuration.
return 0;
}
-struct format_field *perf_evsel__field(struct perf_evsel *evsel, const char *name)
+struct tep_format_field *perf_evsel__field(struct perf_evsel *evsel, const char *name)
{
return tep_find_field(evsel->tp_format, name);
}
void *perf_evsel__rawptr(struct perf_evsel *evsel, struct perf_sample *sample,
const char *name)
{
- struct format_field *field = perf_evsel__field(evsel, name);
+ struct tep_format_field *field = perf_evsel__field(evsel, name);
int offset;
if (!field)
offset = field->offset;
- if (field->flags & FIELD_IS_DYNAMIC) {
+ if (field->flags & TEP_FIELD_IS_DYNAMIC) {
offset = *(int *)(sample->raw_data + field->offset);
offset &= 0xffff;
}
return sample->raw_data + offset;
}
-u64 format_field__intval(struct format_field *field, struct perf_sample *sample,
+u64 format_field__intval(struct tep_format_field *field, struct perf_sample *sample,
bool needs_swap)
{
u64 value;
u64 perf_evsel__intval(struct perf_evsel *evsel, struct perf_sample *sample,
const char *name)
{
- struct format_field *field = perf_evsel__field(evsel, name);
+ struct tep_format_field *field = perf_evsel__field(evsel, name);
if (!field)
return 0;
return evsel->evlist->env;
return NULL;
}
+
+static int store_evsel_ids(struct perf_evsel *evsel, struct perf_evlist *evlist)
+{
+ int cpu, thread;
+
+ for (cpu = 0; cpu < xyarray__max_x(evsel->fd); cpu++) {
+ for (thread = 0; thread < xyarray__max_y(evsel->fd);
+ thread++) {
+ int fd = FD(evsel, cpu, thread);
+
+ if (perf_evlist__id_add_fd(evlist, evsel,
+ cpu, thread, fd) < 0)
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+int perf_evsel__store_ids(struct perf_evsel *evsel, struct perf_evlist *evlist)
+{
+ struct cpu_map *cpus = evsel->cpus;
+ struct thread_map *threads = evsel->threads;
+
+ if (perf_evsel__alloc_id(evsel, cpus->nr, threads->nr))
+ return -ENOMEM;
+
+ return store_evsel_ids(evsel, evlist);
+}
char *name;
double scale;
const char *unit;
- struct event_format *tp_format;
+ struct tep_event_format *tp_format;
off_t id_offset;
struct perf_stat_evsel *stats;
void *priv;
struct perf_evsel *perf_evsel__new_cycles(bool precise);
-struct event_format *event_format__new(const char *sys, const char *name);
+struct tep_event_format *event_format__new(const char *sys, const char *name);
void perf_evsel__init(struct perf_evsel *evsel,
struct perf_event_attr *attr, int idx);
return perf_evsel__rawptr(evsel, sample, name);
}
-struct format_field;
+struct tep_format_field;
-u64 format_field__intval(struct format_field *field, struct perf_sample *sample, bool needs_swap);
+u64 format_field__intval(struct tep_format_field *field, struct perf_sample *sample, bool needs_swap);
-struct format_field *perf_evsel__field(struct perf_evsel *evsel, const char *name);
+struct tep_format_field *perf_evsel__field(struct perf_evsel *evsel, const char *name);
#define perf_evsel__match(evsel, t, c) \
(evsel->attr.type == PERF_TYPE_##t && \
struct perf_env *perf_evsel__env(struct perf_evsel *evsel);
+int perf_evsel__store_ids(struct perf_evsel *evsel, struct perf_evlist *evlist);
#endif /* __PERF_EVSEL_H */
}
if (details->trace_fields) {
- struct format_field *field;
+ struct tep_format_field *field;
if (evsel->attr.type != PERF_TYPE_TRACEPOINT) {
printed += comma_fprintf(fp, &first, " (not a tracepoint)");
static int perf_evsel__prepare_tracepoint_event(struct perf_evsel *evsel,
struct tep_handle *pevent)
{
- struct event_format *event;
+ struct tep_event_format *event;
char bf[128];
/* already prepared */
return ret;
}
-int perf_event__process_feature(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session __maybe_unused)
+int perf_event__process_feature(struct perf_session *session,
+ union perf_event *event)
{
+ struct perf_tool *tool = session->tool;
struct feat_fd ff = { .fd = 0 };
struct feature_event *fe = (struct feature_event *)event;
int type = fe->header.type;
}
int perf_event__synthesize_attrs(struct perf_tool *tool,
- struct perf_session *session,
- perf_event__handler_t process)
+ struct perf_evlist *evlist,
+ perf_event__handler_t process)
{
struct perf_evsel *evsel;
int err = 0;
- evlist__for_each_entry(session->evlist, evsel) {
+ evlist__for_each_entry(evlist, evsel) {
err = perf_event__synthesize_attr(tool, &evsel->attr, evsel->ids,
evsel->id, process);
if (err) {
return aligned_size;
}
-int perf_event__process_tracing_data(struct perf_tool *tool __maybe_unused,
- union perf_event *event,
- struct perf_session *session)
+int perf_event__process_tracing_data(struct perf_session *session,
+ union perf_event *event)
{
ssize_t size_read, padding, size = event->tracing_data.size;
int fd = perf_data__fd(session->data);
return err;
}
-int perf_event__process_build_id(struct perf_tool *tool __maybe_unused,
- union perf_event *event,
- struct perf_session *session)
+int perf_event__process_build_id(struct perf_session *session,
+ union perf_event *event)
{
__event_process_build_id(&event->build_id,
event->build_id.filename,
perf_event__handler_t process,
bool is_pipe);
-int perf_event__process_feature(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session);
+int perf_event__process_feature(struct perf_session *session,
+ union perf_event *event);
int perf_event__synthesize_attr(struct perf_tool *tool,
struct perf_event_attr *attr, u32 ids, u64 *id,
perf_event__handler_t process);
int perf_event__synthesize_attrs(struct perf_tool *tool,
- struct perf_session *session,
+ struct perf_evlist *evlist,
perf_event__handler_t process);
int perf_event__synthesize_event_update_unit(struct perf_tool *tool,
struct perf_evsel *evsel,
int perf_event__synthesize_tracing_data(struct perf_tool *tool,
int fd, struct perf_evlist *evlist,
perf_event__handler_t process);
-int perf_event__process_tracing_data(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session);
+int perf_event__process_tracing_data(struct perf_session *session,
+ union perf_event *event);
int perf_event__synthesize_build_id(struct perf_tool *tool,
struct dso *pos, u16 misc,
perf_event__handler_t process,
struct machine *machine);
-int perf_event__process_build_id(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session);
+int perf_event__process_build_id(struct perf_session *session,
+ union perf_event *event);
bool is_perf_magic(u64 magic);
#define NAME_ALIGN 64
decoder->pge = false;
decoder->continuous_period = false;
decoder->pkt_state = INTEL_PT_STATE_IN_SYNC;
- decoder->state.to_ip = 0;
+ decoder->state.type |= INTEL_PT_TRACE_END;
return 0;
}
if (err == INTEL_PT_RETURN)
decoder->continuous_period = false;
decoder->pkt_state = INTEL_PT_STATE_IN_SYNC;
decoder->state.from_ip = decoder->ip;
- decoder->state.to_ip = 0;
- if (decoder->packet.count != 0)
+ if (decoder->packet.count == 0) {
+ decoder->state.to_ip = 0;
+ } else {
+ decoder->state.to_ip = decoder->last_ip;
decoder->ip = decoder->last_ip;
+ }
+ decoder->state.type |= INTEL_PT_TRACE_END;
} else {
decoder->pkt_state = INTEL_PT_STATE_IN_SYNC;
decoder->state.from_ip = decoder->ip;
decoder->pkt_state = INTEL_PT_STATE_IN_SYNC;
decoder->ip = to_ip;
decoder->state.from_ip = decoder->ip;
- decoder->state.to_ip = 0;
+ decoder->state.to_ip = to_ip;
+ decoder->state.type |= INTEL_PT_TRACE_END;
return 0;
}
intel_pt_log_at("ERROR: Conditional branch when expecting indirect branch",
case INTEL_PT_TIP_PGD:
decoder->state.from_ip = decoder->ip;
- decoder->state.to_ip = 0;
- if (decoder->packet.count != 0) {
+ if (decoder->packet.count == 0) {
+ decoder->state.to_ip = 0;
+ } else {
intel_pt_set_ip(decoder);
- intel_pt_log("Omitting PGD ip " x64_fmt "\n",
- decoder->ip);
+ decoder->state.to_ip = decoder->ip;
}
decoder->pge = false;
decoder->continuous_period = false;
+ decoder->state.type |= INTEL_PT_TRACE_END;
return 0;
case INTEL_PT_TIP_PGE:
intel_pt_set_ip(decoder);
decoder->state.to_ip = decoder->ip;
}
+ decoder->state.type |= INTEL_PT_TRACE_BEGIN;
return 0;
case INTEL_PT_TIP:
intel_pt_set_ip(decoder);
decoder->state.from_ip = 0;
decoder->state.to_ip = decoder->ip;
+ decoder->state.type |= INTEL_PT_TRACE_BEGIN;
return 0;
}
decoder->pge = decoder->packet.type != INTEL_PT_TIP_PGD;
if (intel_pt_have_ip(decoder))
intel_pt_set_ip(decoder);
- if (decoder->ip)
- return 0;
- break;
+ if (!decoder->ip)
+ break;
+ if (decoder->packet.type == INTEL_PT_TIP_PGE)
+ decoder->state.type |= INTEL_PT_TRACE_BEGIN;
+ if (decoder->packet.type == INTEL_PT_TIP_PGD)
+ decoder->state.type |= INTEL_PT_TRACE_END;
+ return 0;
case INTEL_PT_FUP:
if (intel_pt_have_ip(decoder))
INTEL_PT_EX_STOP = 1 << 6,
INTEL_PT_PWR_EXIT = 1 << 7,
INTEL_PT_CBR_CHG = 1 << 8,
+ INTEL_PT_TRACE_BEGIN = 1 << 9,
+ INTEL_PT_TRACE_END = 1 << 10,
};
enum intel_pt_period_type {
ptq->insn_len = ptq->state->insn_len;
memcpy(ptq->insn, ptq->state->insn, INTEL_PT_INSN_BUF_SZ);
}
+
+ if (ptq->state->type & INTEL_PT_TRACE_BEGIN)
+ ptq->flags |= PERF_IP_FLAG_TRACE_BEGIN;
+ if (ptq->state->type & INTEL_PT_TRACE_END)
+ ptq->flags |= PERF_IP_FLAG_TRACE_END;
}
static int intel_pt_setup_queue(struct intel_pt *pt,
#define CLANG_BPF_CMD_DEFAULT_TEMPLATE \
"$CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS "\
"-DLINUX_VERSION_CODE=$LINUX_VERSION_CODE " \
- "$CLANG_OPTIONS $KERNEL_INC_OPTIONS $PERF_BPF_INC_OPTIONS " \
+ "$CLANG_OPTIONS $PERF_BPF_INC_OPTIONS $KERNEL_INC_OPTIONS " \
"-Wno-unused-value -Wno-pointer-sign " \
"-working-directory $WORKING_DIR " \
"-c \"$CLANG_SOURCE\" -target bpf $CLANG_EMIT_LLVM -O2 -o - $LLVM_OPTIONS_PIPE"
if (!symbol_conf.inline_name || !map || !sym)
return ret;
- addr = map__rip_2objdump(map, ip);
+ addr = map__map_ip(map, ip);
+ addr = map__rip_2objdump(map, addr);
inline_node = inlines__tree_find(&map->dso->inlined_nodes, addr);
if (!inline_node) {
{
struct callchain_cursor *cursor = arg;
const char *srcline = NULL;
- u64 addr;
+ u64 addr = entry->ip;
if (symbol_conf.hide_unresolved && entry->sym == NULL)
return 0;
* Convert entry->ip from a virtual address to an offset in
* its corresponding binary.
*/
- addr = map__map_ip(entry->map, entry->ip);
+ if (entry->map)
+ addr = map__map_ip(entry->map, entry->ip);
srcline = callchain_srcline(entry->map, entry->sym, addr);
return callchain_cursor_append(cursor, entry->ip,
build_id__sprintf(map->dso->build_id,
sizeof(map->dso->build_id),
sbuild_id);
- pr_warning("%s with build id %s not found",
- name, sbuild_id);
+ pr_debug("%s with build id %s not found", name, sbuild_id);
} else
- pr_warning("Failed to open %s", name);
+ pr_debug("Failed to open %s", name);
- pr_warning(", continuing without symbols\n");
+ pr_debug(", continuing without symbols\n");
return -1;
} else if (nr == 0) {
#ifdef HAVE_LIBELF_SUPPORT
if (len > sizeof(DSO__DELETED) &&
strcmp(name + real_len + 1, DSO__DELETED) == 0) {
- pr_warning("%.*s was updated (is prelink enabled?). "
+ pr_debug("%.*s was updated (is prelink enabled?). "
"Restart the long running apps that use it!\n",
(int)real_len, name);
} else {
- pr_warning("no symbols found in %s, maybe install "
- "a debug package?\n", name);
+ pr_debug("no symbols found in %s, maybe install a debug package?\n", name);
}
#endif
return -1;
if (verbose >= 2) {
if (use_browser) {
- pr_warning("overlapping maps in %s "
- "(disable tui for more info)\n",
+ pr_debug("overlapping maps in %s (disable tui for more info)\n",
map->dso->name);
} else {
fputs("overlapping maps:\n", fp);
}
int perf_mmap__push(struct perf_mmap *md, void *to,
- int push(void *to, void *buf, size_t size))
+ int push(struct perf_mmap *map, void *to, void *buf, size_t size))
{
u64 head = perf_mmap__read_head(md);
unsigned char *data = md->base + page_size;
size = md->mask + 1 - (md->start & md->mask);
md->start += size;
- if (push(to, buf, size) < 0) {
+ if (push(md, to, buf, size) < 0) {
rc = -1;
goto out;
}
size = md->end - md->start;
md->start += size;
- if (push(to, buf, size) < 0) {
+ if (push(md, to, buf, size) < 0) {
rc = -1;
goto out;
}
union perf_event *perf_mmap__read_event(struct perf_mmap *map);
int perf_mmap__push(struct perf_mmap *md, void *to,
- int push(void *to, void *buf, size_t size));
+ int push(struct perf_mmap *map, void *to, void *buf, size_t size));
size_t perf_mmap__mmap_len(struct perf_mmap *map);
return oe->copy_on_queue ? __dup_event(oe, event) : event;
}
-static void free_dup_event(struct ordered_events *oe, union perf_event *event)
+static void __free_dup_event(struct ordered_events *oe, union perf_event *event)
{
- if (event && oe->copy_on_queue) {
+ if (event) {
oe->cur_alloc_size -= event->header.size;
free(event);
}
}
+static void free_dup_event(struct ordered_events *oe, union perf_event *event)
+{
+ if (oe->copy_on_queue)
+ __free_dup_event(oe, event);
+}
+
#define MAX_SAMPLE_BUFFER (64 * 1024 / sizeof(struct ordered_event))
static struct ordered_event *alloc_event(struct ordered_events *oe,
union perf_event *event)
struct list_head *cache = &oe->cache;
struct ordered_event *new = NULL;
union perf_event *new_event;
+ size_t size;
new_event = dup_event(oe, event);
if (!new_event)
return NULL;
+ /*
+ * We maintain the following scheme of buffers for ordered
+ * event allocation:
+ *
+ * to_free list -> buffer1 (64K)
+ * buffer2 (64K)
+ * ...
+ *
+ * Each buffer keeps an array of ordered events objects:
+ * buffer -> event[0]
+ * event[1]
+ * ...
+ *
+ * Each allocated ordered event is linked to one of
+ * following lists:
+ * - time ordered list 'events'
+ * - list of currently removed events 'cache'
+ *
+ * Allocation of the ordered event uses the following order
+ * to get the memory:
+ * - use recently removed object from 'cache' list
+ * - use available object in current allocation buffer
+ * - allocate new buffer if the current buffer is full
+ *
+ * Removal of ordered event object moves it from events to
+ * the cache list.
+ */
+ size = sizeof(*oe->buffer) + MAX_SAMPLE_BUFFER * sizeof(*new);
+
if (!list_empty(cache)) {
new = list_entry(cache->next, struct ordered_event, list);
list_del(&new->list);
} else if (oe->buffer) {
- new = oe->buffer + oe->buffer_idx;
+ new = &oe->buffer->event[oe->buffer_idx];
if (++oe->buffer_idx == MAX_SAMPLE_BUFFER)
oe->buffer = NULL;
- } else if (oe->cur_alloc_size < oe->max_alloc_size) {
- size_t size = MAX_SAMPLE_BUFFER * sizeof(*new);
-
+ } else if ((oe->cur_alloc_size + size) < oe->max_alloc_size) {
oe->buffer = malloc(size);
if (!oe->buffer) {
free_dup_event(oe, new_event);
oe->cur_alloc_size += size;
list_add(&oe->buffer->list, &oe->to_free);
- /* First entry is abused to maintain the to_free list. */
- oe->buffer_idx = 2;
- new = oe->buffer + 1;
+ oe->buffer_idx = 1;
+ new = &oe->buffer->event[0];
} else {
pr("allocation limit reached %" PRIu64 "B\n", oe->max_alloc_size);
+ return NULL;
}
new->event = new_event;
oe->deliver = deliver;
}
+static void
+ordered_events_buffer__free(struct ordered_events_buffer *buffer,
+ unsigned int max, struct ordered_events *oe)
+{
+ if (oe->copy_on_queue) {
+ unsigned int i;
+
+ for (i = 0; i < max; i++)
+ __free_dup_event(oe, buffer->event[i].event);
+ }
+
+ free(buffer);
+}
+
void ordered_events__free(struct ordered_events *oe)
{
- while (!list_empty(&oe->to_free)) {
- struct ordered_event *event;
+ struct ordered_events_buffer *buffer, *tmp;
- event = list_entry(oe->to_free.next, struct ordered_event, list);
- list_del(&event->list);
- free_dup_event(oe, event->event);
- free(event);
+ if (list_empty(&oe->to_free))
+ return;
+
+ /*
+ * Current buffer might not have all the events allocated
+ * yet, we need to free only allocated ones ...
+ */
+ list_del(&oe->buffer->list);
+ ordered_events_buffer__free(oe->buffer, oe->buffer_idx, oe);
+
+ /* ... and continue with the rest */
+ list_for_each_entry_safe(buffer, tmp, &oe->to_free, list) {
+ list_del(&buffer->list);
+ ordered_events_buffer__free(buffer, MAX_SAMPLE_BUFFER, oe);
}
}
typedef int (*ordered_events__deliver_t)(struct ordered_events *oe,
struct ordered_event *event);
+struct ordered_events_buffer {
+ struct list_head list;
+ struct ordered_event event[0];
+};
+
struct ordered_events {
- u64 last_flush;
- u64 next_flush;
- u64 max_timestamp;
- u64 max_alloc_size;
- u64 cur_alloc_size;
- struct list_head events;
- struct list_head cache;
- struct list_head to_free;
- struct ordered_event *buffer;
- struct ordered_event *last;
- ordered_events__deliver_t deliver;
- int buffer_idx;
- unsigned int nr_events;
- enum oe_flush last_flush_type;
- u32 nr_unordered_events;
- bool copy_on_queue;
+ u64 last_flush;
+ u64 next_flush;
+ u64 max_timestamp;
+ u64 max_alloc_size;
+ u64 cur_alloc_size;
+ struct list_head events;
+ struct list_head cache;
+ struct list_head to_free;
+ struct ordered_events_buffer *buffer;
+ struct ordered_event *last;
+ ordered_events__deliver_t deliver;
+ int buffer_idx;
+ unsigned int nr_events;
+ enum oe_flush last_flush_type;
+ u32 nr_unordered_events;
+ bool copy_on_queue;
};
int ordered_events__queue(struct ordered_events *oe, union perf_event *event,
static __u64 pmu_format_max_value(const unsigned long *format)
{
- __u64 w = 0;
- int fbit;
-
- for_each_set_bit(fbit, format, PERF_PMU_FORMAT_BITS)
- w |= (1ULL << fbit);
+ int w;
- return w;
+ w = bitmap_weight(format, PERF_PMU_FORMAT_BITS);
+ if (!w)
+ return 0;
+ if (w < 64)
+ return (1ULL << w) - 1;
+ return -1;
}
/*
}
static PyObject*
-tracepoint_field(struct pyrf_event *pe, struct format_field *field)
+tracepoint_field(struct pyrf_event *pe, struct tep_format_field *field)
{
struct tep_handle *pevent = field->event->pevent;
void *data = pe->sample.raw_data;
unsigned long long val;
unsigned int offset, len;
- if (field->flags & FIELD_IS_ARRAY) {
+ if (field->flags & TEP_FIELD_IS_ARRAY) {
offset = field->offset;
len = field->size;
- if (field->flags & FIELD_IS_DYNAMIC) {
+ if (field->flags & TEP_FIELD_IS_DYNAMIC) {
val = tep_read_number(pevent, data + offset, len);
offset = val;
len = offset >> 16;
offset &= 0xffff;
}
- if (field->flags & FIELD_IS_STRING &&
+ if (field->flags & TEP_FIELD_IS_STRING &&
is_printable_array(data + offset, len)) {
ret = _PyUnicode_FromString((char *)data + offset);
} else {
ret = PyByteArray_FromStringAndSize((const char *) data + offset, len);
- field->flags &= ~FIELD_IS_STRING;
+ field->flags &= ~TEP_FIELD_IS_STRING;
}
} else {
val = tep_read_number(pevent, data + field->offset,
field->size);
- if (field->flags & FIELD_IS_POINTER)
+ if (field->flags & TEP_FIELD_IS_POINTER)
ret = PyLong_FromUnsignedLong((unsigned long) val);
- else if (field->flags & FIELD_IS_SIGNED)
+ else if (field->flags & TEP_FIELD_IS_SIGNED)
ret = PyLong_FromLong((long) val);
else
ret = PyLong_FromUnsignedLong((unsigned long) val);
{
const char *str = _PyUnicode_AsString(PyObject_Str(attr_name));
struct perf_evsel *evsel = pevent->evsel;
- struct format_field *field;
+ struct tep_format_field *field;
if (!evsel->tp_format) {
- struct event_format *tp_format;
+ struct tep_event_format *tp_format;
tp_format = trace_event__tp_format_id(evsel->attr.config);
if (!tp_format)
static PyObject *pyrf__tracepoint(struct pyrf_evsel *pevsel,
PyObject *args, PyObject *kwargs)
{
- struct event_format *tp_format;
+ struct tep_event_format *tp_format;
static char *kwlist[] = { "sys", "name", NULL };
char *sys = NULL;
char *name = NULL;
#include <linux/bitops.h>
#include <linux/log2.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+
#include "cpumap.h"
#include "color.h"
#include "evsel.h"
#include "auxtrace.h"
#include "s390-cpumsf.h"
#include "s390-cpumsf-kernel.h"
+#include "config.h"
struct s390_cpumsf {
struct auxtrace auxtrace;
u32 pmu_type;
u16 machine_type;
bool data_queued;
+ bool use_logfile;
+ char *logdir;
};
struct s390_cpumsf_queue {
unsigned int queue_nr;
struct auxtrace_buffer *buffer;
int cpu;
+ FILE *logfile;
};
/* Display s390 CPU measurement facility basic-sampling data entry */
buffer->use_size = buffer->size;
buffer->use_data = buffer->data;
}
+ if (sfq->logfile) { /* Write into log file */
+ size_t rc = fwrite(buffer->data, buffer->size, 1,
+ sfq->logfile);
+ if (rc != 1)
+ pr_err("Failed to write auxiliary data\n");
+ }
} else
buffer = sfq->buffer;
return -ENOMEM;
buffer->use_size = buffer->size;
buffer->use_data = buffer->data;
+
+ if (sfq->logfile) { /* Write into log file */
+ size_t rc = fwrite(buffer->data, buffer->size, 1,
+ sfq->logfile);
+ if (rc != 1)
+ pr_err("Failed to write auxiliary data\n");
+ }
}
pr_debug4("%s queue_nr:%d buffer:%" PRId64 " offset:%#" PRIx64 " size:%#zx rest:%#zx\n",
__func__, sfq->queue_nr, buffer->buffer_nr, buffer->offset,
sfq->sf = sf;
sfq->queue_nr = queue_nr;
sfq->cpu = -1;
+ if (sf->use_logfile) {
+ char *name;
+ int rc;
+
+ rc = (sf->logdir)
+ ? asprintf(&name, "%s/aux.smp.%02x",
+ sf->logdir, queue_nr)
+ : asprintf(&name, "aux.smp.%02x", queue_nr);
+ if (rc > 0)
+ sfq->logfile = fopen(name, "w");
+ if (sfq->logfile == NULL) {
+ pr_err("Failed to open auxiliary log file %s,"
+ "continue...\n", name);
+ sf->use_logfile = false;
+ }
+ free(name);
+ }
return sfq;
}
struct auxtrace_queues *queues = &sf->queues;
unsigned int i;
- for (i = 0; i < queues->nr_queues; i++)
+ for (i = 0; i < queues->nr_queues; i++) {
+ struct s390_cpumsf_queue *sfq = (struct s390_cpumsf_queue *)
+ queues->queue_array[i].priv;
+
+ if (sfq != NULL && sfq->logfile) {
+ fclose(sfq->logfile);
+ sfq->logfile = NULL;
+ }
zfree(&queues->queue_array[i].priv);
+ }
auxtrace_queues__free(queues);
}
auxtrace_heap__free(&sf->heap);
s390_cpumsf_free_queues(session);
session->auxtrace = NULL;
+ free(sf->logdir);
free(sf);
}
/* Check itrace options set on perf report command.
* Return true, if none are set or all options specified can be
- * handled on s390.
+ * handled on s390 (currently only option 'd' for logging.
* Return false otherwise.
*/
static bool check_auxtrace_itrace(struct itrace_synth_opts *itops)
{
+ bool ison = false;
+
if (!itops || !itops->set)
return true;
- pr_err("No --itrace options supported\n");
+ ison = itops->inject || itops->instructions || itops->branches ||
+ itops->transactions || itops->ptwrites ||
+ itops->pwr_events || itops->errors ||
+ itops->dont_decode || itops->calls || itops->returns ||
+ itops->callchain || itops->thread_stack ||
+ itops->last_branch;
+ if (!ison)
+ return true;
+ pr_err("Unsupported --itrace options specified\n");
return false;
}
+/* Check for AUXTRACE dump directory if it is needed.
+ * On failure print an error message but continue.
+ * Return 0 on wrong keyword in config file and 1 otherwise.
+ */
+static int s390_cpumsf__config(const char *var, const char *value, void *cb)
+{
+ struct s390_cpumsf *sf = cb;
+ struct stat stbuf;
+ int rc;
+
+ if (strcmp(var, "auxtrace.dumpdir"))
+ return 0;
+ sf->logdir = strdup(value);
+ if (sf->logdir == NULL) {
+ pr_err("Failed to find auxtrace log directory %s,"
+ " continue with current directory...\n", value);
+ return 1;
+ }
+ rc = stat(sf->logdir, &stbuf);
+ if (rc == -1 || !S_ISDIR(stbuf.st_mode)) {
+ pr_err("Missing auxtrace log directory %s,"
+ " continue with current directory...\n", value);
+ free(sf->logdir);
+ sf->logdir = NULL;
+ }
+ return 1;
+}
+
int s390_cpumsf_process_auxtrace_info(union perf_event *event,
struct perf_session *session)
{
err = -EINVAL;
goto err_free;
}
+ sf->use_logfile = session->itrace_synth_opts->log;
+ if (sf->use_logfile)
+ perf_config(s390_cpumsf__config, sf);
err = auxtrace_queues__init(&sf->queues);
if (err)
auxtrace_queues__free(&sf->queues);
session->auxtrace = NULL;
err_free:
+ free(sf->logdir);
free(sf);
return err;
}
LEAVE;
}
-static void define_symbolic_values(struct print_flag_sym *field,
+static void define_symbolic_values(struct tep_print_flag_sym *field,
const char *ev_name,
const char *field_name)
{
LEAVE;
}
-static void define_flag_values(struct print_flag_sym *field,
+static void define_flag_values(struct tep_print_flag_sym *field,
const char *ev_name,
const char *field_name)
{
LEAVE;
}
-static void define_event_symbols(struct event_format *event,
+static void define_event_symbols(struct tep_event_format *event,
const char *ev_name,
- struct print_arg *args)
+ struct tep_print_arg *args)
{
if (args == NULL)
return;
switch (args->type) {
- case PRINT_NULL:
+ case TEP_PRINT_NULL:
break;
- case PRINT_ATOM:
+ case TEP_PRINT_ATOM:
define_flag_value(ev_name, cur_field_name, "0",
args->atom.atom);
zero_flag_atom = 0;
break;
- case PRINT_FIELD:
+ case TEP_PRINT_FIELD:
free(cur_field_name);
cur_field_name = strdup(args->field.name);
break;
- case PRINT_FLAGS:
+ case TEP_PRINT_FLAGS:
define_event_symbols(event, ev_name, args->flags.field);
define_flag_field(ev_name, cur_field_name, args->flags.delim);
define_flag_values(args->flags.flags, ev_name, cur_field_name);
break;
- case PRINT_SYMBOL:
+ case TEP_PRINT_SYMBOL:
define_event_symbols(event, ev_name, args->symbol.field);
define_symbolic_field(ev_name, cur_field_name);
define_symbolic_values(args->symbol.symbols, ev_name,
cur_field_name);
break;
- case PRINT_HEX:
- case PRINT_HEX_STR:
+ case TEP_PRINT_HEX:
+ case TEP_PRINT_HEX_STR:
define_event_symbols(event, ev_name, args->hex.field);
define_event_symbols(event, ev_name, args->hex.size);
break;
- case PRINT_INT_ARRAY:
+ case TEP_PRINT_INT_ARRAY:
define_event_symbols(event, ev_name, args->int_array.field);
define_event_symbols(event, ev_name, args->int_array.count);
define_event_symbols(event, ev_name, args->int_array.el_size);
break;
- case PRINT_BSTRING:
- case PRINT_DYNAMIC_ARRAY:
- case PRINT_DYNAMIC_ARRAY_LEN:
- case PRINT_STRING:
- case PRINT_BITMASK:
+ case TEP_PRINT_BSTRING:
+ case TEP_PRINT_DYNAMIC_ARRAY:
+ case TEP_PRINT_DYNAMIC_ARRAY_LEN:
+ case TEP_PRINT_STRING:
+ case TEP_PRINT_BITMASK:
break;
- case PRINT_TYPE:
+ case TEP_PRINT_TYPE:
define_event_symbols(event, ev_name, args->typecast.item);
break;
- case PRINT_OP:
+ case TEP_PRINT_OP:
if (strcmp(args->op.op, ":") == 0)
zero_flag_atom = 1;
define_event_symbols(event, ev_name, args->op.left);
define_event_symbols(event, ev_name, args->op.right);
break;
- case PRINT_FUNC:
+ case TEP_PRINT_FUNC:
default:
pr_err("Unsupported print arg type\n");
/* we should warn... */
struct addr_location *al)
{
struct thread *thread = al->thread;
- struct event_format *event = evsel->tp_format;
- struct format_field *field;
+ struct tep_event_format *event = evsel->tp_format;
+ struct tep_format_field *field;
static char handler[256];
unsigned long long val;
unsigned long s, ns;
/* common fields other than pid can be accessed via xsub fns */
for (field = event->format.fields; field; field = field->next) {
- if (field->flags & FIELD_IS_STRING) {
+ if (field->flags & TEP_FIELD_IS_STRING) {
int offset;
- if (field->flags & FIELD_IS_DYNAMIC) {
+ if (field->flags & TEP_FIELD_IS_DYNAMIC) {
offset = *(int *)(data + field->offset);
offset &= 0xffff;
} else
} else { /* FIELD_IS_NUMERIC */
val = read_size(event, data + field->offset,
field->size);
- if (field->flags & FIELD_IS_SIGNED) {
+ if (field->flags & TEP_FIELD_IS_SIGNED) {
XPUSHs(sv_2mortal(newSViv(val)));
} else {
XPUSHs(sv_2mortal(newSVuv(val)));
static int perl_generate_script(struct tep_handle *pevent, const char *outfile)
{
- struct event_format *event = NULL;
- struct format_field *f;
+ struct tep_event_format *event = NULL;
+ struct tep_format_field *f;
char fname[PATH_MAX];
int not_first, count;
FILE *ofp;
count++;
fprintf(ofp, "%s=", f->name);
- if (f->flags & FIELD_IS_STRING ||
- f->flags & FIELD_IS_FLAG ||
- f->flags & FIELD_IS_SYMBOLIC)
+ if (f->flags & TEP_FIELD_IS_STRING ||
+ f->flags & TEP_FIELD_IS_FLAG ||
+ f->flags & TEP_FIELD_IS_SYMBOLIC)
fprintf(ofp, "%%s");
- else if (f->flags & FIELD_IS_SIGNED)
+ else if (f->flags & TEP_FIELD_IS_SIGNED)
fprintf(ofp, "%%d");
else
fprintf(ofp, "%%u");
if (++count % 5 == 0)
fprintf(ofp, "\n\t ");
- if (f->flags & FIELD_IS_FLAG) {
+ if (f->flags & TEP_FIELD_IS_FLAG) {
if ((count - 1) % 5 != 0) {
fprintf(ofp, "\n\t ");
count = 4;
event->name);
fprintf(ofp, "\"%s\", $%s)", f->name,
f->name);
- } else if (f->flags & FIELD_IS_SYMBOLIC) {
+ } else if (f->flags & TEP_FIELD_IS_SYMBOLIC) {
if ((count - 1) % 5 != 0) {
fprintf(ofp, "\n\t ");
count = 4;
call_object(handler, args, handler_name);
}
-static void define_value(enum print_arg_type field_type,
+static void define_value(enum tep_print_arg_type field_type,
const char *ev_name,
const char *field_name,
const char *field_value,
unsigned long long value;
unsigned n = 0;
- if (field_type == PRINT_SYMBOL)
+ if (field_type == TEP_PRINT_SYMBOL)
handler_name = "define_symbolic_value";
t = PyTuple_New(4);
Py_DECREF(t);
}
-static void define_values(enum print_arg_type field_type,
- struct print_flag_sym *field,
+static void define_values(enum tep_print_arg_type field_type,
+ struct tep_print_flag_sym *field,
const char *ev_name,
const char *field_name)
{
define_values(field_type, field->next, ev_name, field_name);
}
-static void define_field(enum print_arg_type field_type,
+static void define_field(enum tep_print_arg_type field_type,
const char *ev_name,
const char *field_name,
const char *delim)
PyObject *t;
unsigned n = 0;
- if (field_type == PRINT_SYMBOL)
+ if (field_type == TEP_PRINT_SYMBOL)
handler_name = "define_symbolic_field";
- if (field_type == PRINT_FLAGS)
+ if (field_type == TEP_PRINT_FLAGS)
t = PyTuple_New(3);
else
t = PyTuple_New(2);
PyTuple_SetItem(t, n++, _PyUnicode_FromString(ev_name));
PyTuple_SetItem(t, n++, _PyUnicode_FromString(field_name));
- if (field_type == PRINT_FLAGS)
+ if (field_type == TEP_PRINT_FLAGS)
PyTuple_SetItem(t, n++, _PyUnicode_FromString(delim));
try_call_object(handler_name, t);
Py_DECREF(t);
}
-static void define_event_symbols(struct event_format *event,
+static void define_event_symbols(struct tep_event_format *event,
const char *ev_name,
- struct print_arg *args)
+ struct tep_print_arg *args)
{
if (args == NULL)
return;
switch (args->type) {
- case PRINT_NULL:
+ case TEP_PRINT_NULL:
break;
- case PRINT_ATOM:
- define_value(PRINT_FLAGS, ev_name, cur_field_name, "0",
+ case TEP_PRINT_ATOM:
+ define_value(TEP_PRINT_FLAGS, ev_name, cur_field_name, "0",
args->atom.atom);
zero_flag_atom = 0;
break;
- case PRINT_FIELD:
+ case TEP_PRINT_FIELD:
free(cur_field_name);
cur_field_name = strdup(args->field.name);
break;
- case PRINT_FLAGS:
+ case TEP_PRINT_FLAGS:
define_event_symbols(event, ev_name, args->flags.field);
- define_field(PRINT_FLAGS, ev_name, cur_field_name,
+ define_field(TEP_PRINT_FLAGS, ev_name, cur_field_name,
args->flags.delim);
- define_values(PRINT_FLAGS, args->flags.flags, ev_name,
+ define_values(TEP_PRINT_FLAGS, args->flags.flags, ev_name,
cur_field_name);
break;
- case PRINT_SYMBOL:
+ case TEP_PRINT_SYMBOL:
define_event_symbols(event, ev_name, args->symbol.field);
- define_field(PRINT_SYMBOL, ev_name, cur_field_name, NULL);
- define_values(PRINT_SYMBOL, args->symbol.symbols, ev_name,
+ define_field(TEP_PRINT_SYMBOL, ev_name, cur_field_name, NULL);
+ define_values(TEP_PRINT_SYMBOL, args->symbol.symbols, ev_name,
cur_field_name);
break;
- case PRINT_HEX:
- case PRINT_HEX_STR:
+ case TEP_PRINT_HEX:
+ case TEP_PRINT_HEX_STR:
define_event_symbols(event, ev_name, args->hex.field);
define_event_symbols(event, ev_name, args->hex.size);
break;
- case PRINT_INT_ARRAY:
+ case TEP_PRINT_INT_ARRAY:
define_event_symbols(event, ev_name, args->int_array.field);
define_event_symbols(event, ev_name, args->int_array.count);
define_event_symbols(event, ev_name, args->int_array.el_size);
break;
- case PRINT_STRING:
+ case TEP_PRINT_STRING:
break;
- case PRINT_TYPE:
+ case TEP_PRINT_TYPE:
define_event_symbols(event, ev_name, args->typecast.item);
break;
- case PRINT_OP:
+ case TEP_PRINT_OP:
if (strcmp(args->op.op, ":") == 0)
zero_flag_atom = 1;
define_event_symbols(event, ev_name, args->op.left);
break;
default:
/* gcc warns for these? */
- case PRINT_BSTRING:
- case PRINT_DYNAMIC_ARRAY:
- case PRINT_DYNAMIC_ARRAY_LEN:
- case PRINT_FUNC:
- case PRINT_BITMASK:
+ case TEP_PRINT_BSTRING:
+ case TEP_PRINT_DYNAMIC_ARRAY:
+ case TEP_PRINT_DYNAMIC_ARRAY_LEN:
+ case TEP_PRINT_FUNC:
+ case TEP_PRINT_BITMASK:
/* we should warn... */
return;
}
define_event_symbols(event, ev_name, args->next);
}
-static PyObject *get_field_numeric_entry(struct event_format *event,
- struct format_field *field, void *data)
+static PyObject *get_field_numeric_entry(struct tep_event_format *event,
+ struct tep_format_field *field, void *data)
{
- bool is_array = field->flags & FIELD_IS_ARRAY;
+ bool is_array = field->flags & TEP_FIELD_IS_ARRAY;
PyObject *obj = NULL, *list = NULL;
unsigned long long val;
unsigned int item_size, n_items, i;
val = read_size(event, data + field->offset + i * item_size,
item_size);
- if (field->flags & FIELD_IS_SIGNED) {
+ if (field->flags & TEP_FIELD_IS_SIGNED) {
if ((long long)val >= LONG_MIN &&
(long long)val <= LONG_MAX)
obj = _PyLong_FromLong(val);
struct perf_evsel *evsel,
struct addr_location *al)
{
- struct event_format *event = evsel->tp_format;
+ struct tep_event_format *event = evsel->tp_format;
PyObject *handler, *context, *t, *obj = NULL, *callchain;
PyObject *dict = NULL, *all_entries_dict = NULL;
static char handler_name[256];
- struct format_field *field;
+ struct tep_format_field *field;
unsigned long s, ns;
unsigned n = 0;
int pid;
unsigned int offset, len;
unsigned long long val;
- if (field->flags & FIELD_IS_ARRAY) {
+ if (field->flags & TEP_FIELD_IS_ARRAY) {
offset = field->offset;
len = field->size;
- if (field->flags & FIELD_IS_DYNAMIC) {
+ if (field->flags & TEP_FIELD_IS_DYNAMIC) {
val = tep_read_number(scripting_context->pevent,
data + offset, len);
offset = val;
len = offset >> 16;
offset &= 0xffff;
}
- if (field->flags & FIELD_IS_STRING &&
+ if (field->flags & TEP_FIELD_IS_STRING &&
is_printable_array(data + offset, len)) {
obj = _PyUnicode_FromString((char *) data + offset);
} else {
obj = PyByteArray_FromStringAndSize((const char *) data + offset, len);
- field->flags &= ~FIELD_IS_STRING;
+ field->flags &= ~TEP_FIELD_IS_STRING;
}
} else { /* FIELD_IS_NUMERIC */
obj = get_field_numeric_entry(event, field, data);
static int python_generate_script(struct tep_handle *pevent, const char *outfile)
{
- struct event_format *event = NULL;
- struct format_field *f;
+ struct tep_event_format *event = NULL;
+ struct tep_format_field *f;
char fname[PATH_MAX];
int not_first, count;
FILE *ofp;
count++;
fprintf(ofp, "%s=", f->name);
- if (f->flags & FIELD_IS_STRING ||
- f->flags & FIELD_IS_FLAG ||
- f->flags & FIELD_IS_ARRAY ||
- f->flags & FIELD_IS_SYMBOLIC)
+ if (f->flags & TEP_FIELD_IS_STRING ||
+ f->flags & TEP_FIELD_IS_FLAG ||
+ f->flags & TEP_FIELD_IS_ARRAY ||
+ f->flags & TEP_FIELD_IS_SYMBOLIC)
fprintf(ofp, "%%s");
- else if (f->flags & FIELD_IS_SIGNED)
+ else if (f->flags & TEP_FIELD_IS_SIGNED)
fprintf(ofp, "%%d");
else
fprintf(ofp, "%%u");
if (++count % 5 == 0)
fprintf(ofp, "\n\t\t");
- if (f->flags & FIELD_IS_FLAG) {
+ if (f->flags & TEP_FIELD_IS_FLAG) {
if ((count - 1) % 5 != 0) {
fprintf(ofp, "\n\t\t");
count = 4;
event->name);
fprintf(ofp, "\"%s\", %s)", f->name,
f->name);
- } else if (f->flags & FIELD_IS_SYMBOLIC) {
+ } else if (f->flags & TEP_FIELD_IS_SYMBOLIC) {
if ((count - 1) % 5 != 0) {
fprintf(ofp, "\n\t\t");
count = 4;
free(session);
}
-static int process_event_synth_tracing_data_stub(struct perf_tool *tool
+static int process_event_synth_tracing_data_stub(struct perf_session *session
__maybe_unused,
union perf_event *event
- __maybe_unused,
- struct perf_session *session
- __maybe_unused)
+ __maybe_unused)
{
dump_printf(": unhandled!\n");
return 0;
return 0;
}
-static s64 process_event_auxtrace_stub(struct perf_tool *tool __maybe_unused,
- union perf_event *event,
- struct perf_session *session
- __maybe_unused)
+static s64 process_event_auxtrace_stub(struct perf_session *session __maybe_unused,
+ union perf_event *event)
{
dump_printf(": unhandled!\n");
if (perf_data__is_pipe(session->data))
return event->auxtrace.size;
}
-static int process_event_op2_stub(struct perf_tool *tool __maybe_unused,
- union perf_event *event __maybe_unused,
- struct perf_session *session __maybe_unused)
+static int process_event_op2_stub(struct perf_session *session __maybe_unused,
+ union perf_event *event __maybe_unused)
{
dump_printf(": unhandled!\n");
return 0;
static
-int process_event_thread_map_stub(struct perf_tool *tool __maybe_unused,
- union perf_event *event __maybe_unused,
- struct perf_session *session __maybe_unused)
+int process_event_thread_map_stub(struct perf_session *session __maybe_unused,
+ union perf_event *event __maybe_unused)
{
if (dump_trace)
perf_event__fprintf_thread_map(event, stdout);
}
static
-int process_event_cpu_map_stub(struct perf_tool *tool __maybe_unused,
- union perf_event *event __maybe_unused,
- struct perf_session *session __maybe_unused)
+int process_event_cpu_map_stub(struct perf_session *session __maybe_unused,
+ union perf_event *event __maybe_unused)
{
if (dump_trace)
perf_event__fprintf_cpu_map(event, stdout);
}
static
-int process_event_stat_config_stub(struct perf_tool *tool __maybe_unused,
- union perf_event *event __maybe_unused,
- struct perf_session *session __maybe_unused)
+int process_event_stat_config_stub(struct perf_session *session __maybe_unused,
+ union perf_event *event __maybe_unused)
{
if (dump_trace)
perf_event__fprintf_stat_config(event, stdout);
return 0;
}
-static int process_stat_stub(struct perf_tool *tool __maybe_unused,
- union perf_event *event __maybe_unused,
- struct perf_session *perf_session
- __maybe_unused)
+static int process_stat_stub(struct perf_session *perf_session __maybe_unused,
+ union perf_event *event)
{
if (dump_trace)
perf_event__fprintf_stat(event, stdout);
return 0;
}
-static int process_stat_round_stub(struct perf_tool *tool __maybe_unused,
- union perf_event *event __maybe_unused,
- struct perf_session *perf_session
- __maybe_unused)
+static int process_stat_round_stub(struct perf_session *perf_session __maybe_unused,
+ union perf_event *event)
{
if (dump_trace)
perf_event__fprintf_stat_round(event, stdout);
case PERF_RECORD_HEADER_TRACING_DATA:
/* setup for reading amidst mmap */
lseek(fd, file_offset, SEEK_SET);
- return tool->tracing_data(tool, event, session);
+ return tool->tracing_data(session, event);
case PERF_RECORD_HEADER_BUILD_ID:
- return tool->build_id(tool, event, session);
+ return tool->build_id(session, event);
case PERF_RECORD_FINISHED_ROUND:
return tool->finished_round(tool, event, oe);
case PERF_RECORD_ID_INDEX:
- return tool->id_index(tool, event, session);
+ return tool->id_index(session, event);
case PERF_RECORD_AUXTRACE_INFO:
- return tool->auxtrace_info(tool, event, session);
+ return tool->auxtrace_info(session, event);
case PERF_RECORD_AUXTRACE:
/* setup for reading amidst mmap */
lseek(fd, file_offset + event->header.size, SEEK_SET);
- return tool->auxtrace(tool, event, session);
+ return tool->auxtrace(session, event);
case PERF_RECORD_AUXTRACE_ERROR:
perf_session__auxtrace_error_inc(session, event);
- return tool->auxtrace_error(tool, event, session);
+ return tool->auxtrace_error(session, event);
case PERF_RECORD_THREAD_MAP:
- return tool->thread_map(tool, event, session);
+ return tool->thread_map(session, event);
case PERF_RECORD_CPU_MAP:
- return tool->cpu_map(tool, event, session);
+ return tool->cpu_map(session, event);
case PERF_RECORD_STAT_CONFIG:
- return tool->stat_config(tool, event, session);
+ return tool->stat_config(session, event);
case PERF_RECORD_STAT:
- return tool->stat(tool, event, session);
+ return tool->stat(session, event);
case PERF_RECORD_STAT_ROUND:
- return tool->stat_round(tool, event, session);
+ return tool->stat_round(session, event);
case PERF_RECORD_TIME_CONV:
session->time_conv = event->time_conv;
- return tool->time_conv(tool, event, session);
+ return tool->time_conv(session, event);
case PERF_RECORD_HEADER_FEATURE:
- return tool->feature(tool, event, session);
+ return tool->feature(session, event);
default:
return -EINVAL;
}
return err;
}
-int perf_event__process_id_index(struct perf_tool *tool __maybe_unused,
- union perf_event *event,
- struct perf_session *session)
+int perf_event__process_id_index(struct perf_session *session,
+ union perf_event *event)
{
struct perf_evlist *evlist = session->evlist;
struct id_index_event *ie = &event->id_index;
union perf_event *event,
struct perf_sample *sample);
-int perf_event__process_id_index(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session);
+int perf_event__process_id_index(struct perf_session *session,
+ union perf_event *event);
int perf_event__synthesize_id_index(struct perf_tool *tool,
perf_event__handler_t process,
from re import sub
def clang_has_option(option):
- return [o for o in Popen(['clang', option], stderr=PIPE).stderr.readlines() if "unknown argument" in o] == [ ]
+ return [o for o in Popen(['clang', option], stderr=PIPE).stderr.readlines() if b"unknown argument" in o] == [ ]
cc = getenv("CC")
if cc == "clang":
- from _sysconfigdata import build_time_vars
- build_time_vars["CFLAGS"] = sub("-specs=[^ ]+", "", build_time_vars["CFLAGS"])
- if not clang_has_option("-mcet"):
- build_time_vars["CFLAGS"] = sub("-mcet", "", build_time_vars["CFLAGS"])
- if not clang_has_option("-fcf-protection"):
- build_time_vars["CFLAGS"] = sub("-fcf-protection", "", build_time_vars["CFLAGS"])
+ from distutils.sysconfig import get_config_vars
+ vars = get_config_vars()
+ for var in ('CFLAGS', 'OPT'):
+ vars[var] = sub("-specs=[^ ]+", "", vars[var])
+ if not clang_has_option("-mcet"):
+ vars[var] = sub("-mcet", "", vars[var])
+ if not clang_has_option("-fcf-protection"):
+ vars[var] = sub("-fcf-protection", "", vars[var])
from distutils.core import setup, Extension
cflags = getenv('CFLAGS', '').split()
# switch off several checks (need to be at the end of cflags list)
-cflags += ['-fno-strict-aliasing', '-Wno-write-strings', '-Wno-unused-parameter' ]
+cflags += ['-fno-strict-aliasing', '-Wno-write-strings', '-Wno-unused-parameter', '-Wno-redundant-decls' ]
if cc != "clang":
cflags += ['-Wno-cast-function-type' ]
struct hpp_dynamic_entry {
struct perf_hpp_fmt hpp;
struct perf_evsel *evsel;
- struct format_field *field;
+ struct tep_format_field *field;
unsigned dynamic_len;
bool raw_trace;
};
if (namelen > len)
len = namelen;
- if (!(hde->field->flags & FIELD_IS_STRING)) {
+ if (!(hde->field->flags & TEP_FIELD_IS_STRING)) {
/* length for print hex numbers */
fieldlen = hde->field->size * 2 + 2;
}
struct hist_entry *he)
{
char *str, *pos;
- struct format_field *field = hde->field;
+ struct tep_format_field *field = hde->field;
size_t namelen;
bool last = false;
struct hpp_dynamic_entry *hde;
size_t len = fmt->user_len;
char *str, *pos;
- struct format_field *field;
+ struct tep_format_field *field;
size_t namelen;
bool last = false;
int ret;
struct hist_entry *a, struct hist_entry *b)
{
struct hpp_dynamic_entry *hde;
- struct format_field *field;
+ struct tep_format_field *field;
unsigned offset, size;
hde = container_of(fmt, struct hpp_dynamic_entry, hpp);
}
field = hde->field;
- if (field->flags & FIELD_IS_DYNAMIC) {
+ if (field->flags & TEP_FIELD_IS_DYNAMIC) {
unsigned long long dyn;
tep_read_number_field(field, a->raw_data, &dyn);
}
static struct hpp_dynamic_entry *
-__alloc_dynamic_entry(struct perf_evsel *evsel, struct format_field *field,
+__alloc_dynamic_entry(struct perf_evsel *evsel, struct tep_format_field *field,
int level)
{
struct hpp_dynamic_entry *hde;
}
static int __dynamic_dimension__add(struct perf_evsel *evsel,
- struct format_field *field,
+ struct tep_format_field *field,
bool raw_trace, int level)
{
struct hpp_dynamic_entry *hde;
static int add_evsel_fields(struct perf_evsel *evsel, bool raw_trace, int level)
{
int ret;
- struct format_field *field;
+ struct tep_format_field *field;
field = evsel->tp_format->format.fields;
while (field) {
{
int ret = -ESRCH;
struct perf_evsel *evsel;
- struct format_field *field;
+ struct tep_format_field *field;
evlist__for_each_entry(evlist, evsel) {
if (evsel->attr.type != PERF_TYPE_TRACEPOINT)
{
char *str, *event_name, *field_name, *opt_name;
struct perf_evsel *evsel;
- struct format_field *field;
+ struct tep_format_field *field;
bool raw_trace = symbol_conf.raw_trace;
int ret = 0;
struct symbol *inline_sym;
char *demangled = NULL;
+ if (!funcname)
+ funcname = "??";
+
if (dso) {
demangled = dso__demangle_sym(dso, 0, funcname);
if (demangled)
--- /dev/null
+#include <stdio.h>
+#include <inttypes.h>
+#include <linux/time64.h>
+#include <math.h>
+#include "evlist.h"
+#include "evsel.h"
+#include "stat.h"
+#include "top.h"
+#include "thread_map.h"
+#include "cpumap.h"
+#include "string2.h"
+#include "sane_ctype.h"
+#include "cgroup.h"
+#include <math.h>
+#include <api/fs/fs.h>
+
+#define CNTR_NOT_SUPPORTED "<not supported>"
+#define CNTR_NOT_COUNTED "<not counted>"
+
+static bool is_duration_time(struct perf_evsel *evsel)
+{
+ return !strcmp(evsel->name, "duration_time");
+}
+
+static void print_running(struct perf_stat_config *config,
+ u64 run, u64 ena)
+{
+ if (config->csv_output) {
+ fprintf(config->output, "%s%" PRIu64 "%s%.2f",
+ config->csv_sep,
+ run,
+ config->csv_sep,
+ ena ? 100.0 * run / ena : 100.0);
+ } else if (run != ena) {
+ fprintf(config->output, " (%.2f%%)", 100.0 * run / ena);
+ }
+}
+
+static void print_noise_pct(struct perf_stat_config *config,
+ double total, double avg)
+{
+ double pct = rel_stddev_stats(total, avg);
+
+ if (config->csv_output)
+ fprintf(config->output, "%s%.2f%%", config->csv_sep, pct);
+ else if (pct)
+ fprintf(config->output, " ( +-%6.2f%% )", pct);
+}
+
+static void print_noise(struct perf_stat_config *config,
+ struct perf_evsel *evsel, double avg)
+{
+ struct perf_stat_evsel *ps;
+
+ if (config->run_count == 1)
+ return;
+
+ ps = evsel->stats;
+ print_noise_pct(config, stddev_stats(&ps->res_stats[0]), avg);
+}
+
+static void aggr_printout(struct perf_stat_config *config,
+ struct perf_evsel *evsel, int id, int nr)
+{
+ switch (config->aggr_mode) {
+ case AGGR_CORE:
+ fprintf(config->output, "S%d-C%*d%s%*d%s",
+ cpu_map__id_to_socket(id),
+ config->csv_output ? 0 : -8,
+ cpu_map__id_to_cpu(id),
+ config->csv_sep,
+ config->csv_output ? 0 : 4,
+ nr,
+ config->csv_sep);
+ break;
+ case AGGR_SOCKET:
+ fprintf(config->output, "S%*d%s%*d%s",
+ config->csv_output ? 0 : -5,
+ id,
+ config->csv_sep,
+ config->csv_output ? 0 : 4,
+ nr,
+ config->csv_sep);
+ break;
+ case AGGR_NONE:
+ fprintf(config->output, "CPU%*d%s",
+ config->csv_output ? 0 : -4,
+ perf_evsel__cpus(evsel)->map[id], config->csv_sep);
+ break;
+ case AGGR_THREAD:
+ fprintf(config->output, "%*s-%*d%s",
+ config->csv_output ? 0 : 16,
+ thread_map__comm(evsel->threads, id),
+ config->csv_output ? 0 : -8,
+ thread_map__pid(evsel->threads, id),
+ config->csv_sep);
+ break;
+ case AGGR_GLOBAL:
+ case AGGR_UNSET:
+ default:
+ break;
+ }
+}
+
+struct outstate {
+ FILE *fh;
+ bool newline;
+ const char *prefix;
+ int nfields;
+ int id, nr;
+ struct perf_evsel *evsel;
+};
+
+#define METRIC_LEN 35
+
+static void new_line_std(struct perf_stat_config *config __maybe_unused,
+ void *ctx)
+{
+ struct outstate *os = ctx;
+
+ os->newline = true;
+}
+
+static void do_new_line_std(struct perf_stat_config *config,
+ struct outstate *os)
+{
+ fputc('\n', os->fh);
+ fputs(os->prefix, os->fh);
+ aggr_printout(config, os->evsel, os->id, os->nr);
+ if (config->aggr_mode == AGGR_NONE)
+ fprintf(os->fh, " ");
+ fprintf(os->fh, " ");
+}
+
+static void print_metric_std(struct perf_stat_config *config,
+ void *ctx, const char *color, const char *fmt,
+ const char *unit, double val)
+{
+ struct outstate *os = ctx;
+ FILE *out = os->fh;
+ int n;
+ bool newline = os->newline;
+
+ os->newline = false;
+
+ if (unit == NULL || fmt == NULL) {
+ fprintf(out, "%-*s", METRIC_LEN, "");
+ return;
+ }
+
+ if (newline)
+ do_new_line_std(config, os);
+
+ n = fprintf(out, " # ");
+ if (color)
+ n += color_fprintf(out, color, fmt, val);
+ else
+ n += fprintf(out, fmt, val);
+ fprintf(out, " %-*s", METRIC_LEN - n - 1, unit);
+}
+
+static void new_line_csv(struct perf_stat_config *config, void *ctx)
+{
+ struct outstate *os = ctx;
+ int i;
+
+ fputc('\n', os->fh);
+ if (os->prefix)
+ fprintf(os->fh, "%s%s", os->prefix, config->csv_sep);
+ aggr_printout(config, os->evsel, os->id, os->nr);
+ for (i = 0; i < os->nfields; i++)
+ fputs(config->csv_sep, os->fh);
+}
+
+static void print_metric_csv(struct perf_stat_config *config __maybe_unused,
+ void *ctx,
+ const char *color __maybe_unused,
+ const char *fmt, const char *unit, double val)
+{
+ struct outstate *os = ctx;
+ FILE *out = os->fh;
+ char buf[64], *vals, *ends;
+
+ if (unit == NULL || fmt == NULL) {
+ fprintf(out, "%s%s", config->csv_sep, config->csv_sep);
+ return;
+ }
+ snprintf(buf, sizeof(buf), fmt, val);
+ ends = vals = ltrim(buf);
+ while (isdigit(*ends) || *ends == '.')
+ ends++;
+ *ends = 0;
+ while (isspace(*unit))
+ unit++;
+ fprintf(out, "%s%s%s%s", config->csv_sep, vals, config->csv_sep, unit);
+}
+
+/* Filter out some columns that don't work well in metrics only mode */
+
+static bool valid_only_metric(const char *unit)
+{
+ if (!unit)
+ return false;
+ if (strstr(unit, "/sec") ||
+ strstr(unit, "hz") ||
+ strstr(unit, "Hz") ||
+ strstr(unit, "CPUs utilized"))
+ return false;
+ return true;
+}
+
+static const char *fixunit(char *buf, struct perf_evsel *evsel,
+ const char *unit)
+{
+ if (!strncmp(unit, "of all", 6)) {
+ snprintf(buf, 1024, "%s %s", perf_evsel__name(evsel),
+ unit);
+ return buf;
+ }
+ return unit;
+}
+
+static void print_metric_only(struct perf_stat_config *config,
+ void *ctx, const char *color, const char *fmt,
+ const char *unit, double val)
+{
+ struct outstate *os = ctx;
+ FILE *out = os->fh;
+ char buf[1024], str[1024];
+ unsigned mlen = config->metric_only_len;
+
+ if (!valid_only_metric(unit))
+ return;
+ unit = fixunit(buf, os->evsel, unit);
+ if (mlen < strlen(unit))
+ mlen = strlen(unit) + 1;
+
+ if (color)
+ mlen += strlen(color) + sizeof(PERF_COLOR_RESET) - 1;
+
+ color_snprintf(str, sizeof(str), color ?: "", fmt, val);
+ fprintf(out, "%*s ", mlen, str);
+}
+
+static void print_metric_only_csv(struct perf_stat_config *config __maybe_unused,
+ void *ctx, const char *color __maybe_unused,
+ const char *fmt,
+ const char *unit, double val)
+{
+ struct outstate *os = ctx;
+ FILE *out = os->fh;
+ char buf[64], *vals, *ends;
+ char tbuf[1024];
+
+ if (!valid_only_metric(unit))
+ return;
+ unit = fixunit(tbuf, os->evsel, unit);
+ snprintf(buf, sizeof buf, fmt, val);
+ ends = vals = ltrim(buf);
+ while (isdigit(*ends) || *ends == '.')
+ ends++;
+ *ends = 0;
+ fprintf(out, "%s%s", vals, config->csv_sep);
+}
+
+static void new_line_metric(struct perf_stat_config *config __maybe_unused,
+ void *ctx __maybe_unused)
+{
+}
+
+static void print_metric_header(struct perf_stat_config *config,
+ void *ctx, const char *color __maybe_unused,
+ const char *fmt __maybe_unused,
+ const char *unit, double val __maybe_unused)
+{
+ struct outstate *os = ctx;
+ char tbuf[1024];
+
+ if (!valid_only_metric(unit))
+ return;
+ unit = fixunit(tbuf, os->evsel, unit);
+ if (config->csv_output)
+ fprintf(os->fh, "%s%s", unit, config->csv_sep);
+ else
+ fprintf(os->fh, "%*s ", config->metric_only_len, unit);
+}
+
+static int first_shadow_cpu(struct perf_stat_config *config,
+ struct perf_evsel *evsel, int id)
+{
+ struct perf_evlist *evlist = evsel->evlist;
+ int i;
+
+ if (!config->aggr_get_id)
+ return 0;
+
+ if (config->aggr_mode == AGGR_NONE)
+ return id;
+
+ if (config->aggr_mode == AGGR_GLOBAL)
+ return 0;
+
+ for (i = 0; i < perf_evsel__nr_cpus(evsel); i++) {
+ int cpu2 = perf_evsel__cpus(evsel)->map[i];
+
+ if (config->aggr_get_id(config, evlist->cpus, cpu2) == id)
+ return cpu2;
+ }
+ return 0;
+}
+
+static void abs_printout(struct perf_stat_config *config,
+ int id, int nr, struct perf_evsel *evsel, double avg)
+{
+ FILE *output = config->output;
+ double sc = evsel->scale;
+ const char *fmt;
+
+ if (config->csv_output) {
+ fmt = floor(sc) != sc ? "%.2f%s" : "%.0f%s";
+ } else {
+ if (config->big_num)
+ fmt = floor(sc) != sc ? "%'18.2f%s" : "%'18.0f%s";
+ else
+ fmt = floor(sc) != sc ? "%18.2f%s" : "%18.0f%s";
+ }
+
+ aggr_printout(config, evsel, id, nr);
+
+ fprintf(output, fmt, avg, config->csv_sep);
+
+ if (evsel->unit)
+ fprintf(output, "%-*s%s",
+ config->csv_output ? 0 : config->unit_width,
+ evsel->unit, config->csv_sep);
+
+ fprintf(output, "%-*s", config->csv_output ? 0 : 25, perf_evsel__name(evsel));
+
+ if (evsel->cgrp)
+ fprintf(output, "%s%s", config->csv_sep, evsel->cgrp->name);
+}
+
+static bool is_mixed_hw_group(struct perf_evsel *counter)
+{
+ struct perf_evlist *evlist = counter->evlist;
+ u32 pmu_type = counter->attr.type;
+ struct perf_evsel *pos;
+
+ if (counter->nr_members < 2)
+ return false;
+
+ evlist__for_each_entry(evlist, pos) {
+ /* software events can be part of any hardware group */
+ if (pos->attr.type == PERF_TYPE_SOFTWARE)
+ continue;
+ if (pmu_type == PERF_TYPE_SOFTWARE) {
+ pmu_type = pos->attr.type;
+ continue;
+ }
+ if (pmu_type != pos->attr.type)
+ return true;
+ }
+
+ return false;
+}
+
+static void printout(struct perf_stat_config *config, int id, int nr,
+ struct perf_evsel *counter, double uval,
+ char *prefix, u64 run, u64 ena, double noise,
+ struct runtime_stat *st)
+{
+ struct perf_stat_output_ctx out;
+ struct outstate os = {
+ .fh = config->output,
+ .prefix = prefix ? prefix : "",
+ .id = id,
+ .nr = nr,
+ .evsel = counter,
+ };
+ print_metric_t pm = print_metric_std;
+ new_line_t nl;
+
+ if (config->metric_only) {
+ nl = new_line_metric;
+ if (config->csv_output)
+ pm = print_metric_only_csv;
+ else
+ pm = print_metric_only;
+ } else
+ nl = new_line_std;
+
+ if (config->csv_output && !config->metric_only) {
+ static int aggr_fields[] = {
+ [AGGR_GLOBAL] = 0,
+ [AGGR_THREAD] = 1,
+ [AGGR_NONE] = 1,
+ [AGGR_SOCKET] = 2,
+ [AGGR_CORE] = 2,
+ };
+
+ pm = print_metric_csv;
+ nl = new_line_csv;
+ os.nfields = 3;
+ os.nfields += aggr_fields[config->aggr_mode];
+ if (counter->cgrp)
+ os.nfields++;
+ }
+ if (run == 0 || ena == 0 || counter->counts->scaled == -1) {
+ if (config->metric_only) {
+ pm(config, &os, NULL, "", "", 0);
+ return;
+ }
+ aggr_printout(config, counter, id, nr);
+
+ fprintf(config->output, "%*s%s",
+ config->csv_output ? 0 : 18,
+ counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
+ config->csv_sep);
+
+ if (counter->supported) {
+ config->print_free_counters_hint = 1;
+ if (is_mixed_hw_group(counter))
+ config->print_mixed_hw_group_error = 1;
+ }
+
+ fprintf(config->output, "%-*s%s",
+ config->csv_output ? 0 : config->unit_width,
+ counter->unit, config->csv_sep);
+
+ fprintf(config->output, "%*s",
+ config->csv_output ? 0 : -25,
+ perf_evsel__name(counter));
+
+ if (counter->cgrp)
+ fprintf(config->output, "%s%s",
+ config->csv_sep, counter->cgrp->name);
+
+ if (!config->csv_output)
+ pm(config, &os, NULL, NULL, "", 0);
+ print_noise(config, counter, noise);
+ print_running(config, run, ena);
+ if (config->csv_output)
+ pm(config, &os, NULL, NULL, "", 0);
+ return;
+ }
+
+ if (!config->metric_only)
+ abs_printout(config, id, nr, counter, uval);
+
+ out.print_metric = pm;
+ out.new_line = nl;
+ out.ctx = &os;
+ out.force_header = false;
+
+ if (config->csv_output && !config->metric_only) {
+ print_noise(config, counter, noise);
+ print_running(config, run, ena);
+ }
+
+ perf_stat__print_shadow_stats(config, counter, uval,
+ first_shadow_cpu(config, counter, id),
+ &out, &config->metric_events, st);
+ if (!config->csv_output && !config->metric_only) {
+ print_noise(config, counter, noise);
+ print_running(config, run, ena);
+ }
+}
+
+static void aggr_update_shadow(struct perf_stat_config *config,
+ struct perf_evlist *evlist)
+{
+ int cpu, s2, id, s;
+ u64 val;
+ struct perf_evsel *counter;
+
+ for (s = 0; s < config->aggr_map->nr; s++) {
+ id = config->aggr_map->map[s];
+ evlist__for_each_entry(evlist, counter) {
+ val = 0;
+ for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
+ s2 = config->aggr_get_id(config, evlist->cpus, cpu);
+ if (s2 != id)
+ continue;
+ val += perf_counts(counter->counts, cpu, 0)->val;
+ }
+ perf_stat__update_shadow_stats(counter, val,
+ first_shadow_cpu(config, counter, id),
+ &rt_stat);
+ }
+ }
+}
+
+static void uniquify_event_name(struct perf_evsel *counter)
+{
+ char *new_name;
+ char *config;
+
+ if (counter->uniquified_name ||
+ !counter->pmu_name || !strncmp(counter->name, counter->pmu_name,
+ strlen(counter->pmu_name)))
+ return;
+
+ config = strchr(counter->name, '/');
+ if (config) {
+ if (asprintf(&new_name,
+ "%s%s", counter->pmu_name, config) > 0) {
+ free(counter->name);
+ counter->name = new_name;
+ }
+ } else {
+ if (asprintf(&new_name,
+ "%s [%s]", counter->name, counter->pmu_name) > 0) {
+ free(counter->name);
+ counter->name = new_name;
+ }
+ }
+
+ counter->uniquified_name = true;
+}
+
+static void collect_all_aliases(struct perf_stat_config *config, struct perf_evsel *counter,
+ void (*cb)(struct perf_stat_config *config, struct perf_evsel *counter, void *data,
+ bool first),
+ void *data)
+{
+ struct perf_evlist *evlist = counter->evlist;
+ struct perf_evsel *alias;
+
+ alias = list_prepare_entry(counter, &(evlist->entries), node);
+ list_for_each_entry_continue (alias, &evlist->entries, node) {
+ if (strcmp(perf_evsel__name(alias), perf_evsel__name(counter)) ||
+ alias->scale != counter->scale ||
+ alias->cgrp != counter->cgrp ||
+ strcmp(alias->unit, counter->unit) ||
+ perf_evsel__is_clock(alias) != perf_evsel__is_clock(counter))
+ break;
+ alias->merged_stat = true;
+ cb(config, alias, data, false);
+ }
+}
+
+static bool collect_data(struct perf_stat_config *config, struct perf_evsel *counter,
+ void (*cb)(struct perf_stat_config *config, struct perf_evsel *counter, void *data,
+ bool first),
+ void *data)
+{
+ if (counter->merged_stat)
+ return false;
+ cb(config, counter, data, true);
+ if (config->no_merge)
+ uniquify_event_name(counter);
+ else if (counter->auto_merge_stats)
+ collect_all_aliases(config, counter, cb, data);
+ return true;
+}
+
+struct aggr_data {
+ u64 ena, run, val;
+ int id;
+ int nr;
+ int cpu;
+};
+
+static void aggr_cb(struct perf_stat_config *config,
+ struct perf_evsel *counter, void *data, bool first)
+{
+ struct aggr_data *ad = data;
+ int cpu, s2;
+
+ for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
+ struct perf_counts_values *counts;
+
+ s2 = config->aggr_get_id(config, perf_evsel__cpus(counter), cpu);
+ if (s2 != ad->id)
+ continue;
+ if (first)
+ ad->nr++;
+ counts = perf_counts(counter->counts, cpu, 0);
+ /*
+ * When any result is bad, make them all to give
+ * consistent output in interval mode.
+ */
+ if (counts->ena == 0 || counts->run == 0 ||
+ counter->counts->scaled == -1) {
+ ad->ena = 0;
+ ad->run = 0;
+ break;
+ }
+ ad->val += counts->val;
+ ad->ena += counts->ena;
+ ad->run += counts->run;
+ }
+}
+
+static void print_aggr(struct perf_stat_config *config,
+ struct perf_evlist *evlist,
+ char *prefix)
+{
+ bool metric_only = config->metric_only;
+ FILE *output = config->output;
+ struct perf_evsel *counter;
+ int s, id, nr;
+ double uval;
+ u64 ena, run, val;
+ bool first;
+
+ if (!(config->aggr_map || config->aggr_get_id))
+ return;
+
+ aggr_update_shadow(config, evlist);
+
+ /*
+ * With metric_only everything is on a single line.
+ * Without each counter has its own line.
+ */
+ for (s = 0; s < config->aggr_map->nr; s++) {
+ struct aggr_data ad;
+ if (prefix && metric_only)
+ fprintf(output, "%s", prefix);
+
+ ad.id = id = config->aggr_map->map[s];
+ first = true;
+ evlist__for_each_entry(evlist, counter) {
+ if (is_duration_time(counter))
+ continue;
+
+ ad.val = ad.ena = ad.run = 0;
+ ad.nr = 0;
+ if (!collect_data(config, counter, aggr_cb, &ad))
+ continue;
+ nr = ad.nr;
+ ena = ad.ena;
+ run = ad.run;
+ val = ad.val;
+ if (first && metric_only) {
+ first = false;
+ aggr_printout(config, counter, id, nr);
+ }
+ if (prefix && !metric_only)
+ fprintf(output, "%s", prefix);
+
+ uval = val * counter->scale;
+ printout(config, id, nr, counter, uval, prefix,
+ run, ena, 1.0, &rt_stat);
+ if (!metric_only)
+ fputc('\n', output);
+ }
+ if (metric_only)
+ fputc('\n', output);
+ }
+}
+
+static int cmp_val(const void *a, const void *b)
+{
+ return ((struct perf_aggr_thread_value *)b)->val -
+ ((struct perf_aggr_thread_value *)a)->val;
+}
+
+static struct perf_aggr_thread_value *sort_aggr_thread(
+ struct perf_evsel *counter,
+ int nthreads, int ncpus,
+ int *ret,
+ struct target *_target)
+{
+ int cpu, thread, i = 0;
+ double uval;
+ struct perf_aggr_thread_value *buf;
+
+ buf = calloc(nthreads, sizeof(struct perf_aggr_thread_value));
+ if (!buf)
+ return NULL;
+
+ for (thread = 0; thread < nthreads; thread++) {
+ u64 ena = 0, run = 0, val = 0;
+
+ for (cpu = 0; cpu < ncpus; cpu++) {
+ val += perf_counts(counter->counts, cpu, thread)->val;
+ ena += perf_counts(counter->counts, cpu, thread)->ena;
+ run += perf_counts(counter->counts, cpu, thread)->run;
+ }
+
+ uval = val * counter->scale;
+
+ /*
+ * Skip value 0 when enabling --per-thread globally,
+ * otherwise too many 0 output.
+ */
+ if (uval == 0.0 && target__has_per_thread(_target))
+ continue;
+
+ buf[i].counter = counter;
+ buf[i].id = thread;
+ buf[i].uval = uval;
+ buf[i].val = val;
+ buf[i].run = run;
+ buf[i].ena = ena;
+ i++;
+ }
+
+ qsort(buf, i, sizeof(struct perf_aggr_thread_value), cmp_val);
+
+ if (ret)
+ *ret = i;
+
+ return buf;
+}
+
+static void print_aggr_thread(struct perf_stat_config *config,
+ struct target *_target,
+ struct perf_evsel *counter, char *prefix)
+{
+ FILE *output = config->output;
+ int nthreads = thread_map__nr(counter->threads);
+ int ncpus = cpu_map__nr(counter->cpus);
+ int thread, sorted_threads, id;
+ struct perf_aggr_thread_value *buf;
+
+ buf = sort_aggr_thread(counter, nthreads, ncpus, &sorted_threads, _target);
+ if (!buf) {
+ perror("cannot sort aggr thread");
+ return;
+ }
+
+ for (thread = 0; thread < sorted_threads; thread++) {
+ if (prefix)
+ fprintf(output, "%s", prefix);
+
+ id = buf[thread].id;
+ if (config->stats)
+ printout(config, id, 0, buf[thread].counter, buf[thread].uval,
+ prefix, buf[thread].run, buf[thread].ena, 1.0,
+ &config->stats[id]);
+ else
+ printout(config, id, 0, buf[thread].counter, buf[thread].uval,
+ prefix, buf[thread].run, buf[thread].ena, 1.0,
+ &rt_stat);
+ fputc('\n', output);
+ }
+
+ free(buf);
+}
+
+struct caggr_data {
+ double avg, avg_enabled, avg_running;
+};
+
+static void counter_aggr_cb(struct perf_stat_config *config __maybe_unused,
+ struct perf_evsel *counter, void *data,
+ bool first __maybe_unused)
+{
+ struct caggr_data *cd = data;
+ struct perf_stat_evsel *ps = counter->stats;
+
+ cd->avg += avg_stats(&ps->res_stats[0]);
+ cd->avg_enabled += avg_stats(&ps->res_stats[1]);
+ cd->avg_running += avg_stats(&ps->res_stats[2]);
+}
+
+/*
+ * Print out the results of a single counter:
+ * aggregated counts in system-wide mode
+ */
+static void print_counter_aggr(struct perf_stat_config *config,
+ struct perf_evsel *counter, char *prefix)
+{
+ bool metric_only = config->metric_only;
+ FILE *output = config->output;
+ double uval;
+ struct caggr_data cd = { .avg = 0.0 };
+
+ if (!collect_data(config, counter, counter_aggr_cb, &cd))
+ return;
+
+ if (prefix && !metric_only)
+ fprintf(output, "%s", prefix);
+
+ uval = cd.avg * counter->scale;
+ printout(config, -1, 0, counter, uval, prefix, cd.avg_running, cd.avg_enabled,
+ cd.avg, &rt_stat);
+ if (!metric_only)
+ fprintf(output, "\n");
+}
+
+static void counter_cb(struct perf_stat_config *config __maybe_unused,
+ struct perf_evsel *counter, void *data,
+ bool first __maybe_unused)
+{
+ struct aggr_data *ad = data;
+
+ ad->val += perf_counts(counter->counts, ad->cpu, 0)->val;
+ ad->ena += perf_counts(counter->counts, ad->cpu, 0)->ena;
+ ad->run += perf_counts(counter->counts, ad->cpu, 0)->run;
+}
+
+/*
+ * Print out the results of a single counter:
+ * does not use aggregated count in system-wide
+ */
+static void print_counter(struct perf_stat_config *config,
+ struct perf_evsel *counter, char *prefix)
+{
+ FILE *output = config->output;
+ u64 ena, run, val;
+ double uval;
+ int cpu;
+
+ for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
+ struct aggr_data ad = { .cpu = cpu };
+
+ if (!collect_data(config, counter, counter_cb, &ad))
+ return;
+ val = ad.val;
+ ena = ad.ena;
+ run = ad.run;
+
+ if (prefix)
+ fprintf(output, "%s", prefix);
+
+ uval = val * counter->scale;
+ printout(config, cpu, 0, counter, uval, prefix, run, ena, 1.0,
+ &rt_stat);
+
+ fputc('\n', output);
+ }
+}
+
+static void print_no_aggr_metric(struct perf_stat_config *config,
+ struct perf_evlist *evlist,
+ char *prefix)
+{
+ int cpu;
+ int nrcpus = 0;
+ struct perf_evsel *counter;
+ u64 ena, run, val;
+ double uval;
+
+ nrcpus = evlist->cpus->nr;
+ for (cpu = 0; cpu < nrcpus; cpu++) {
+ bool first = true;
+
+ if (prefix)
+ fputs(prefix, config->output);
+ evlist__for_each_entry(evlist, counter) {
+ if (is_duration_time(counter))
+ continue;
+ if (first) {
+ aggr_printout(config, counter, cpu, 0);
+ first = false;
+ }
+ val = perf_counts(counter->counts, cpu, 0)->val;
+ ena = perf_counts(counter->counts, cpu, 0)->ena;
+ run = perf_counts(counter->counts, cpu, 0)->run;
+
+ uval = val * counter->scale;
+ printout(config, cpu, 0, counter, uval, prefix, run, ena, 1.0,
+ &rt_stat);
+ }
+ fputc('\n', config->output);
+ }
+}
+
+static int aggr_header_lens[] = {
+ [AGGR_CORE] = 18,
+ [AGGR_SOCKET] = 12,
+ [AGGR_NONE] = 6,
+ [AGGR_THREAD] = 24,
+ [AGGR_GLOBAL] = 0,
+};
+
+static const char *aggr_header_csv[] = {
+ [AGGR_CORE] = "core,cpus,",
+ [AGGR_SOCKET] = "socket,cpus",
+ [AGGR_NONE] = "cpu,",
+ [AGGR_THREAD] = "comm-pid,",
+ [AGGR_GLOBAL] = ""
+};
+
+static void print_metric_headers(struct perf_stat_config *config,
+ struct perf_evlist *evlist,
+ const char *prefix, bool no_indent)
+{
+ struct perf_stat_output_ctx out;
+ struct perf_evsel *counter;
+ struct outstate os = {
+ .fh = config->output
+ };
+
+ if (prefix)
+ fprintf(config->output, "%s", prefix);
+
+ if (!config->csv_output && !no_indent)
+ fprintf(config->output, "%*s",
+ aggr_header_lens[config->aggr_mode], "");
+ if (config->csv_output) {
+ if (config->interval)
+ fputs("time,", config->output);
+ fputs(aggr_header_csv[config->aggr_mode], config->output);
+ }
+
+ /* Print metrics headers only */
+ evlist__for_each_entry(evlist, counter) {
+ if (is_duration_time(counter))
+ continue;
+ os.evsel = counter;
+ out.ctx = &os;
+ out.print_metric = print_metric_header;
+ out.new_line = new_line_metric;
+ out.force_header = true;
+ os.evsel = counter;
+ perf_stat__print_shadow_stats(config, counter, 0,
+ 0,
+ &out,
+ &config->metric_events,
+ &rt_stat);
+ }
+ fputc('\n', config->output);
+}
+
+static void print_interval(struct perf_stat_config *config,
+ struct perf_evlist *evlist,
+ char *prefix, struct timespec *ts)
+{
+ bool metric_only = config->metric_only;
+ unsigned int unit_width = config->unit_width;
+ FILE *output = config->output;
+ static int num_print_interval;
+
+ if (config->interval_clear)
+ puts(CONSOLE_CLEAR);
+
+ sprintf(prefix, "%6lu.%09lu%s", ts->tv_sec, ts->tv_nsec, config->csv_sep);
+
+ if ((num_print_interval == 0 && !config->csv_output) || config->interval_clear) {
+ switch (config->aggr_mode) {
+ case AGGR_SOCKET:
+ fprintf(output, "# time socket cpus");
+ if (!metric_only)
+ fprintf(output, " counts %*s events\n", unit_width, "unit");
+ break;
+ case AGGR_CORE:
+ fprintf(output, "# time core cpus");
+ if (!metric_only)
+ fprintf(output, " counts %*s events\n", unit_width, "unit");
+ break;
+ case AGGR_NONE:
+ fprintf(output, "# time CPU ");
+ if (!metric_only)
+ fprintf(output, " counts %*s events\n", unit_width, "unit");
+ break;
+ case AGGR_THREAD:
+ fprintf(output, "# time comm-pid");
+ if (!metric_only)
+ fprintf(output, " counts %*s events\n", unit_width, "unit");
+ break;
+ case AGGR_GLOBAL:
+ default:
+ fprintf(output, "# time");
+ if (!metric_only)
+ fprintf(output, " counts %*s events\n", unit_width, "unit");
+ case AGGR_UNSET:
+ break;
+ }
+ }
+
+ if ((num_print_interval == 0 || config->interval_clear) && metric_only)
+ print_metric_headers(config, evlist, " ", true);
+ if (++num_print_interval == 25)
+ num_print_interval = 0;
+}
+
+static void print_header(struct perf_stat_config *config,
+ struct target *_target,
+ int argc, const char **argv)
+{
+ FILE *output = config->output;
+ int i;
+
+ fflush(stdout);
+
+ if (!config->csv_output) {
+ fprintf(output, "\n");
+ fprintf(output, " Performance counter stats for ");
+ if (_target->system_wide)
+ fprintf(output, "\'system wide");
+ else if (_target->cpu_list)
+ fprintf(output, "\'CPU(s) %s", _target->cpu_list);
+ else if (!target__has_task(_target)) {
+ fprintf(output, "\'%s", argv ? argv[0] : "pipe");
+ for (i = 1; argv && (i < argc); i++)
+ fprintf(output, " %s", argv[i]);
+ } else if (_target->pid)
+ fprintf(output, "process id \'%s", _target->pid);
+ else
+ fprintf(output, "thread id \'%s", _target->tid);
+
+ fprintf(output, "\'");
+ if (config->run_count > 1)
+ fprintf(output, " (%d runs)", config->run_count);
+ fprintf(output, ":\n\n");
+ }
+}
+
+static int get_precision(double num)
+{
+ if (num > 1)
+ return 0;
+
+ return lround(ceil(-log10(num)));
+}
+
+static void print_table(struct perf_stat_config *config,
+ FILE *output, int precision, double avg)
+{
+ char tmp[64];
+ int idx, indent = 0;
+
+ scnprintf(tmp, 64, " %17.*f", precision, avg);
+ while (tmp[indent] == ' ')
+ indent++;
+
+ fprintf(output, "%*s# Table of individual measurements:\n", indent, "");
+
+ for (idx = 0; idx < config->run_count; idx++) {
+ double run = (double) config->walltime_run[idx] / NSEC_PER_SEC;
+ int h, n = 1 + abs((int) (100.0 * (run - avg)/run) / 5);
+
+ fprintf(output, " %17.*f (%+.*f) ",
+ precision, run, precision, run - avg);
+
+ for (h = 0; h < n; h++)
+ fprintf(output, "#");
+
+ fprintf(output, "\n");
+ }
+
+ fprintf(output, "\n%*s# Final result:\n", indent, "");
+}
+
+static double timeval2double(struct timeval *t)
+{
+ return t->tv_sec + (double) t->tv_usec/USEC_PER_SEC;
+}
+
+static void print_footer(struct perf_stat_config *config)
+{
+ double avg = avg_stats(config->walltime_nsecs_stats) / NSEC_PER_SEC;
+ FILE *output = config->output;
+ int n;
+
+ if (!config->null_run)
+ fprintf(output, "\n");
+
+ if (config->run_count == 1) {
+ fprintf(output, " %17.9f seconds time elapsed", avg);
+
+ if (config->ru_display) {
+ double ru_utime = timeval2double(&config->ru_data.ru_utime);
+ double ru_stime = timeval2double(&config->ru_data.ru_stime);
+
+ fprintf(output, "\n\n");
+ fprintf(output, " %17.9f seconds user\n", ru_utime);
+ fprintf(output, " %17.9f seconds sys\n", ru_stime);
+ }
+ } else {
+ double sd = stddev_stats(config->walltime_nsecs_stats) / NSEC_PER_SEC;
+ /*
+ * Display at most 2 more significant
+ * digits than the stddev inaccuracy.
+ */
+ int precision = get_precision(sd) + 2;
+
+ if (config->walltime_run_table)
+ print_table(config, output, precision, avg);
+
+ fprintf(output, " %17.*f +- %.*f seconds time elapsed",
+ precision, avg, precision, sd);
+
+ print_noise_pct(config, sd, avg);
+ }
+ fprintf(output, "\n\n");
+
+ if (config->print_free_counters_hint &&
+ sysctl__read_int("kernel/nmi_watchdog", &n) >= 0 &&
+ n > 0)
+ fprintf(output,
+"Some events weren't counted. Try disabling the NMI watchdog:\n"
+" echo 0 > /proc/sys/kernel/nmi_watchdog\n"
+" perf stat ...\n"
+" echo 1 > /proc/sys/kernel/nmi_watchdog\n");
+
+ if (config->print_mixed_hw_group_error)
+ fprintf(output,
+ "The events in group usually have to be from "
+ "the same PMU. Try reorganizing the group.\n");
+}
+
+void
+perf_evlist__print_counters(struct perf_evlist *evlist,
+ struct perf_stat_config *config,
+ struct target *_target,
+ struct timespec *ts,
+ int argc, const char **argv)
+{
+ bool metric_only = config->metric_only;
+ int interval = config->interval;
+ struct perf_evsel *counter;
+ char buf[64], *prefix = NULL;
+
+ if (interval)
+ print_interval(config, evlist, prefix = buf, ts);
+ else
+ print_header(config, _target, argc, argv);
+
+ if (metric_only) {
+ static int num_print_iv;
+
+ if (num_print_iv == 0 && !interval)
+ print_metric_headers(config, evlist, prefix, false);
+ if (num_print_iv++ == 25)
+ num_print_iv = 0;
+ if (config->aggr_mode == AGGR_GLOBAL && prefix)
+ fprintf(config->output, "%s", prefix);
+ }
+
+ switch (config->aggr_mode) {
+ case AGGR_CORE:
+ case AGGR_SOCKET:
+ print_aggr(config, evlist, prefix);
+ break;
+ case AGGR_THREAD:
+ evlist__for_each_entry(evlist, counter) {
+ if (is_duration_time(counter))
+ continue;
+ print_aggr_thread(config, _target, counter, prefix);
+ }
+ break;
+ case AGGR_GLOBAL:
+ evlist__for_each_entry(evlist, counter) {
+ if (is_duration_time(counter))
+ continue;
+ print_counter_aggr(config, counter, prefix);
+ }
+ if (metric_only)
+ fputc('\n', config->output);
+ break;
+ case AGGR_NONE:
+ if (metric_only)
+ print_no_aggr_metric(config, evlist, prefix);
+ else {
+ evlist__for_each_entry(evlist, counter) {
+ if (is_duration_time(counter))
+ continue;
+ print_counter(config, counter, prefix);
+ }
+ }
+ break;
+ case AGGR_UNSET:
+ default:
+ break;
+ }
+
+ if (!interval && !config->csv_output)
+ print_footer(config);
+
+ fflush(config->output);
+}
return v->stats.n;
}
-static void print_stalled_cycles_frontend(int cpu,
+static void print_stalled_cycles_frontend(struct perf_stat_config *config,
+ int cpu,
struct perf_evsel *evsel, double avg,
struct perf_stat_output_ctx *out,
struct runtime_stat *st)
color = get_ratio_color(GRC_STALLED_CYCLES_FE, ratio);
if (ratio)
- out->print_metric(out->ctx, color, "%7.2f%%", "frontend cycles idle",
+ out->print_metric(config, out->ctx, color, "%7.2f%%", "frontend cycles idle",
ratio);
else
- out->print_metric(out->ctx, NULL, NULL, "frontend cycles idle", 0);
+ out->print_metric(config, out->ctx, NULL, NULL, "frontend cycles idle", 0);
}
-static void print_stalled_cycles_backend(int cpu,
+static void print_stalled_cycles_backend(struct perf_stat_config *config,
+ int cpu,
struct perf_evsel *evsel, double avg,
struct perf_stat_output_ctx *out,
struct runtime_stat *st)
color = get_ratio_color(GRC_STALLED_CYCLES_BE, ratio);
- out->print_metric(out->ctx, color, "%7.2f%%", "backend cycles idle", ratio);
+ out->print_metric(config, out->ctx, color, "%7.2f%%", "backend cycles idle", ratio);
}
-static void print_branch_misses(int cpu,
+static void print_branch_misses(struct perf_stat_config *config,
+ int cpu,
struct perf_evsel *evsel,
double avg,
struct perf_stat_output_ctx *out,
color = get_ratio_color(GRC_CACHE_MISSES, ratio);
- out->print_metric(out->ctx, color, "%7.2f%%", "of all branches", ratio);
+ out->print_metric(config, out->ctx, color, "%7.2f%%", "of all branches", ratio);
}
-static void print_l1_dcache_misses(int cpu,
+static void print_l1_dcache_misses(struct perf_stat_config *config,
+ int cpu,
struct perf_evsel *evsel,
double avg,
struct perf_stat_output_ctx *out,
color = get_ratio_color(GRC_CACHE_MISSES, ratio);
- out->print_metric(out->ctx, color, "%7.2f%%", "of all L1-dcache hits", ratio);
+ out->print_metric(config, out->ctx, color, "%7.2f%%", "of all L1-dcache hits", ratio);
}
-static void print_l1_icache_misses(int cpu,
+static void print_l1_icache_misses(struct perf_stat_config *config,
+ int cpu,
struct perf_evsel *evsel,
double avg,
struct perf_stat_output_ctx *out,
ratio = avg / total * 100.0;
color = get_ratio_color(GRC_CACHE_MISSES, ratio);
- out->print_metric(out->ctx, color, "%7.2f%%", "of all L1-icache hits", ratio);
+ out->print_metric(config, out->ctx, color, "%7.2f%%", "of all L1-icache hits", ratio);
}
-static void print_dtlb_cache_misses(int cpu,
+static void print_dtlb_cache_misses(struct perf_stat_config *config,
+ int cpu,
struct perf_evsel *evsel,
double avg,
struct perf_stat_output_ctx *out,
ratio = avg / total * 100.0;
color = get_ratio_color(GRC_CACHE_MISSES, ratio);
- out->print_metric(out->ctx, color, "%7.2f%%", "of all dTLB cache hits", ratio);
+ out->print_metric(config, out->ctx, color, "%7.2f%%", "of all dTLB cache hits", ratio);
}
-static void print_itlb_cache_misses(int cpu,
+static void print_itlb_cache_misses(struct perf_stat_config *config,
+ int cpu,
struct perf_evsel *evsel,
double avg,
struct perf_stat_output_ctx *out,
ratio = avg / total * 100.0;
color = get_ratio_color(GRC_CACHE_MISSES, ratio);
- out->print_metric(out->ctx, color, "%7.2f%%", "of all iTLB cache hits", ratio);
+ out->print_metric(config, out->ctx, color, "%7.2f%%", "of all iTLB cache hits", ratio);
}
-static void print_ll_cache_misses(int cpu,
+static void print_ll_cache_misses(struct perf_stat_config *config,
+ int cpu,
struct perf_evsel *evsel,
double avg,
struct perf_stat_output_ctx *out,
ratio = avg / total * 100.0;
color = get_ratio_color(GRC_CACHE_MISSES, ratio);
- out->print_metric(out->ctx, color, "%7.2f%%", "of all LL-cache hits", ratio);
+ out->print_metric(config, out->ctx, color, "%7.2f%%", "of all LL-cache hits", ratio);
}
/*
return sanitize_val(1.0 - sum);
}
-static void print_smi_cost(int cpu, struct perf_evsel *evsel,
+static void print_smi_cost(struct perf_stat_config *config,
+ int cpu, struct perf_evsel *evsel,
struct perf_stat_output_ctx *out,
struct runtime_stat *st)
{
if (cost > 10)
color = PERF_COLOR_RED;
- out->print_metric(out->ctx, color, "%8.1f%%", "SMI cycles%", cost);
- out->print_metric(out->ctx, NULL, "%4.0f", "SMI#", smi_num);
+ out->print_metric(config, out->ctx, color, "%8.1f%%", "SMI cycles%", cost);
+ out->print_metric(config, out->ctx, NULL, "%4.0f", "SMI#", smi_num);
}
-static void generic_metric(const char *metric_expr,
+static void generic_metric(struct perf_stat_config *config,
+ const char *metric_expr,
struct perf_evsel **metric_events,
char *name,
const char *metric_name,
const char *p = metric_expr;
if (expr__parse(&ratio, &pctx, &p) == 0)
- print_metric(ctxp, NULL, "%8.1f",
+ print_metric(config, ctxp, NULL, "%8.1f",
metric_name ?
metric_name :
out->force_header ? name : "",
ratio);
else
- print_metric(ctxp, NULL, NULL,
+ print_metric(config, ctxp, NULL, NULL,
out->force_header ?
(metric_name ? metric_name : name) : "", 0);
} else
- print_metric(ctxp, NULL, NULL, "", 0);
+ print_metric(config, ctxp, NULL, NULL, "", 0);
}
-void perf_stat__print_shadow_stats(struct perf_evsel *evsel,
+void perf_stat__print_shadow_stats(struct perf_stat_config *config,
+ struct perf_evsel *evsel,
double avg, int cpu,
struct perf_stat_output_ctx *out,
struct rblist *metric_events,
if (total) {
ratio = avg / total;
- print_metric(ctxp, NULL, "%7.2f ",
+ print_metric(config, ctxp, NULL, "%7.2f ",
"insn per cycle", ratio);
} else {
- print_metric(ctxp, NULL, NULL, "insn per cycle", 0);
+ print_metric(config, ctxp, NULL, NULL, "insn per cycle", 0);
}
total = runtime_stat_avg(st, STAT_STALLED_CYCLES_FRONT,
ctx, cpu));
if (total && avg) {
- out->new_line(ctxp);
+ out->new_line(config, ctxp);
ratio = total / avg;
- print_metric(ctxp, NULL, "%7.2f ",
+ print_metric(config, ctxp, NULL, "%7.2f ",
"stalled cycles per insn",
ratio);
} else if (have_frontend_stalled) {
- print_metric(ctxp, NULL, NULL,
+ print_metric(config, ctxp, NULL, NULL,
"stalled cycles per insn", 0);
}
} else if (perf_evsel__match(evsel, HARDWARE, HW_BRANCH_MISSES)) {
if (runtime_stat_n(st, STAT_BRANCHES, ctx, cpu) != 0)
- print_branch_misses(cpu, evsel, avg, out, st);
+ print_branch_misses(config, cpu, evsel, avg, out, st);
else
- print_metric(ctxp, NULL, NULL, "of all branches", 0);
+ print_metric(config, ctxp, NULL, NULL, "of all branches", 0);
} else if (
evsel->attr.type == PERF_TYPE_HW_CACHE &&
evsel->attr.config == ( PERF_COUNT_HW_CACHE_L1D |
((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
if (runtime_stat_n(st, STAT_L1_DCACHE, ctx, cpu) != 0)
- print_l1_dcache_misses(cpu, evsel, avg, out, st);
+ print_l1_dcache_misses(config, cpu, evsel, avg, out, st);
else
- print_metric(ctxp, NULL, NULL, "of all L1-dcache hits", 0);
+ print_metric(config, ctxp, NULL, NULL, "of all L1-dcache hits", 0);
} else if (
evsel->attr.type == PERF_TYPE_HW_CACHE &&
evsel->attr.config == ( PERF_COUNT_HW_CACHE_L1I |
((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
if (runtime_stat_n(st, STAT_L1_ICACHE, ctx, cpu) != 0)
- print_l1_icache_misses(cpu, evsel, avg, out, st);
+ print_l1_icache_misses(config, cpu, evsel, avg, out, st);
else
- print_metric(ctxp, NULL, NULL, "of all L1-icache hits", 0);
+ print_metric(config, ctxp, NULL, NULL, "of all L1-icache hits", 0);
} else if (
evsel->attr.type == PERF_TYPE_HW_CACHE &&
evsel->attr.config == ( PERF_COUNT_HW_CACHE_DTLB |
((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
if (runtime_stat_n(st, STAT_DTLB_CACHE, ctx, cpu) != 0)
- print_dtlb_cache_misses(cpu, evsel, avg, out, st);
+ print_dtlb_cache_misses(config, cpu, evsel, avg, out, st);
else
- print_metric(ctxp, NULL, NULL, "of all dTLB cache hits", 0);
+ print_metric(config, ctxp, NULL, NULL, "of all dTLB cache hits", 0);
} else if (
evsel->attr.type == PERF_TYPE_HW_CACHE &&
evsel->attr.config == ( PERF_COUNT_HW_CACHE_ITLB |
((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
if (runtime_stat_n(st, STAT_ITLB_CACHE, ctx, cpu) != 0)
- print_itlb_cache_misses(cpu, evsel, avg, out, st);
+ print_itlb_cache_misses(config, cpu, evsel, avg, out, st);
else
- print_metric(ctxp, NULL, NULL, "of all iTLB cache hits", 0);
+ print_metric(config, ctxp, NULL, NULL, "of all iTLB cache hits", 0);
} else if (
evsel->attr.type == PERF_TYPE_HW_CACHE &&
evsel->attr.config == ( PERF_COUNT_HW_CACHE_LL |
((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
if (runtime_stat_n(st, STAT_LL_CACHE, ctx, cpu) != 0)
- print_ll_cache_misses(cpu, evsel, avg, out, st);
+ print_ll_cache_misses(config, cpu, evsel, avg, out, st);
else
- print_metric(ctxp, NULL, NULL, "of all LL-cache hits", 0);
+ print_metric(config, ctxp, NULL, NULL, "of all LL-cache hits", 0);
} else if (perf_evsel__match(evsel, HARDWARE, HW_CACHE_MISSES)) {
total = runtime_stat_avg(st, STAT_CACHEREFS, ctx, cpu);
ratio = avg * 100 / total;
if (runtime_stat_n(st, STAT_CACHEREFS, ctx, cpu) != 0)
- print_metric(ctxp, NULL, "%8.3f %%",
+ print_metric(config, ctxp, NULL, "%8.3f %%",
"of all cache refs", ratio);
else
- print_metric(ctxp, NULL, NULL, "of all cache refs", 0);
+ print_metric(config, ctxp, NULL, NULL, "of all cache refs", 0);
} else if (perf_evsel__match(evsel, HARDWARE, HW_STALLED_CYCLES_FRONTEND)) {
- print_stalled_cycles_frontend(cpu, evsel, avg, out, st);
+ print_stalled_cycles_frontend(config, cpu, evsel, avg, out, st);
} else if (perf_evsel__match(evsel, HARDWARE, HW_STALLED_CYCLES_BACKEND)) {
- print_stalled_cycles_backend(cpu, evsel, avg, out, st);
+ print_stalled_cycles_backend(config, cpu, evsel, avg, out, st);
} else if (perf_evsel__match(evsel, HARDWARE, HW_CPU_CYCLES)) {
total = runtime_stat_avg(st, STAT_NSECS, 0, cpu);
if (total) {
ratio = avg / total;
- print_metric(ctxp, NULL, "%8.3f", "GHz", ratio);
+ print_metric(config, ctxp, NULL, "%8.3f", "GHz", ratio);
} else {
- print_metric(ctxp, NULL, NULL, "Ghz", 0);
+ print_metric(config, ctxp, NULL, NULL, "Ghz", 0);
}
} else if (perf_stat_evsel__is(evsel, CYCLES_IN_TX)) {
total = runtime_stat_avg(st, STAT_CYCLES, ctx, cpu);
if (total)
- print_metric(ctxp, NULL,
+ print_metric(config, ctxp, NULL,
"%7.2f%%", "transactional cycles",
100.0 * (avg / total));
else
- print_metric(ctxp, NULL, NULL, "transactional cycles",
+ print_metric(config, ctxp, NULL, NULL, "transactional cycles",
0);
} else if (perf_stat_evsel__is(evsel, CYCLES_IN_TX_CP)) {
total = runtime_stat_avg(st, STAT_CYCLES, ctx, cpu);
if (total2 < avg)
total2 = avg;
if (total)
- print_metric(ctxp, NULL, "%7.2f%%", "aborted cycles",
+ print_metric(config, ctxp, NULL, "%7.2f%%", "aborted cycles",
100.0 * ((total2-avg) / total));
else
- print_metric(ctxp, NULL, NULL, "aborted cycles", 0);
+ print_metric(config, ctxp, NULL, NULL, "aborted cycles", 0);
} else if (perf_stat_evsel__is(evsel, TRANSACTION_START)) {
total = runtime_stat_avg(st, STAT_CYCLES_IN_TX,
ctx, cpu);
ratio = total / avg;
if (runtime_stat_n(st, STAT_CYCLES_IN_TX, ctx, cpu) != 0)
- print_metric(ctxp, NULL, "%8.0f",
+ print_metric(config, ctxp, NULL, "%8.0f",
"cycles / transaction", ratio);
else
- print_metric(ctxp, NULL, NULL, "cycles / transaction",
+ print_metric(config, ctxp, NULL, NULL, "cycles / transaction",
0);
} else if (perf_stat_evsel__is(evsel, ELISION_START)) {
total = runtime_stat_avg(st, STAT_CYCLES_IN_TX,
if (avg)
ratio = total / avg;
- print_metric(ctxp, NULL, "%8.0f", "cycles / elision", ratio);
+ print_metric(config, ctxp, NULL, "%8.0f", "cycles / elision", ratio);
} else if (perf_evsel__is_clock(evsel)) {
if ((ratio = avg_stats(&walltime_nsecs_stats)) != 0)
- print_metric(ctxp, NULL, "%8.3f", "CPUs utilized",
+ print_metric(config, ctxp, NULL, "%8.3f", "CPUs utilized",
avg / (ratio * evsel->scale));
else
- print_metric(ctxp, NULL, NULL, "CPUs utilized", 0);
+ print_metric(config, ctxp, NULL, NULL, "CPUs utilized", 0);
} else if (perf_stat_evsel__is(evsel, TOPDOWN_FETCH_BUBBLES)) {
double fe_bound = td_fe_bound(ctx, cpu, st);
if (fe_bound > 0.2)
color = PERF_COLOR_RED;
- print_metric(ctxp, color, "%8.1f%%", "frontend bound",
+ print_metric(config, ctxp, color, "%8.1f%%", "frontend bound",
fe_bound * 100.);
} else if (perf_stat_evsel__is(evsel, TOPDOWN_SLOTS_RETIRED)) {
double retiring = td_retiring(ctx, cpu, st);
if (retiring > 0.7)
color = PERF_COLOR_GREEN;
- print_metric(ctxp, color, "%8.1f%%", "retiring",
+ print_metric(config, ctxp, color, "%8.1f%%", "retiring",
retiring * 100.);
} else if (perf_stat_evsel__is(evsel, TOPDOWN_RECOVERY_BUBBLES)) {
double bad_spec = td_bad_spec(ctx, cpu, st);
if (bad_spec > 0.1)
color = PERF_COLOR_RED;
- print_metric(ctxp, color, "%8.1f%%", "bad speculation",
+ print_metric(config, ctxp, color, "%8.1f%%", "bad speculation",
bad_spec * 100.);
} else if (perf_stat_evsel__is(evsel, TOPDOWN_SLOTS_ISSUED)) {
double be_bound = td_be_bound(ctx, cpu, st);
if (be_bound > 0.2)
color = PERF_COLOR_RED;
if (td_total_slots(ctx, cpu, st) > 0)
- print_metric(ctxp, color, "%8.1f%%", name,
+ print_metric(config, ctxp, color, "%8.1f%%", name,
be_bound * 100.);
else
- print_metric(ctxp, NULL, NULL, name, 0);
+ print_metric(config, ctxp, NULL, NULL, name, 0);
} else if (evsel->metric_expr) {
- generic_metric(evsel->metric_expr, evsel->metric_events, evsel->name,
+ generic_metric(config, evsel->metric_expr, evsel->metric_events, evsel->name,
evsel->metric_name, avg, cpu, out, st);
} else if (runtime_stat_n(st, STAT_NSECS, 0, cpu) != 0) {
char unit = 'M';
unit = 'K';
}
snprintf(unit_buf, sizeof(unit_buf), "%c/sec", unit);
- print_metric(ctxp, NULL, "%8.3f", unit_buf, ratio);
+ print_metric(config, ctxp, NULL, "%8.3f", unit_buf, ratio);
} else if (perf_stat_evsel__is(evsel, SMI_NUM)) {
- print_smi_cost(cpu, evsel, out, st);
+ print_smi_cost(config, cpu, evsel, out, st);
} else {
num = 0;
}
list_for_each_entry (mexp, &me->head, nd) {
if (num++ > 0)
- out->new_line(ctxp);
- generic_metric(mexp->metric_expr, mexp->metric_events,
+ out->new_line(config, ctxp);
+ generic_metric(config, mexp->metric_expr, mexp->metric_events,
evsel->name, mexp->metric_name,
avg, cpu, out, st);
}
}
if (num == 0)
- print_metric(ctxp, NULL, NULL, NULL, 0);
+ print_metric(config, ctxp, NULL, NULL, NULL, 0);
}
return 0;
}
-int perf_event__process_stat_event(struct perf_tool *tool __maybe_unused,
- union perf_event *event,
- struct perf_session *session)
+int perf_event__process_stat_event(struct perf_session *session,
+ union perf_event *event)
{
struct perf_counts_values count;
struct stat_event *st = &event->stat;
return ret;
}
+
+int create_perf_stat_counter(struct perf_evsel *evsel,
+ struct perf_stat_config *config,
+ struct target *target)
+{
+ struct perf_event_attr *attr = &evsel->attr;
+ struct perf_evsel *leader = evsel->leader;
+
+ if (config->scale) {
+ attr->read_format = PERF_FORMAT_TOTAL_TIME_ENABLED |
+ PERF_FORMAT_TOTAL_TIME_RUNNING;
+ }
+
+ /*
+ * The event is part of non trivial group, let's enable
+ * the group read (for leader) and ID retrieval for all
+ * members.
+ */
+ if (leader->nr_members > 1)
+ attr->read_format |= PERF_FORMAT_ID|PERF_FORMAT_GROUP;
+
+ attr->inherit = !config->no_inherit;
+
+ /*
+ * Some events get initialized with sample_(period/type) set,
+ * like tracepoints. Clear it up for counting.
+ */
+ attr->sample_period = 0;
+
+ if (config->identifier)
+ attr->sample_type = PERF_SAMPLE_IDENTIFIER;
+
+ /*
+ * Disabling all counters initially, they will be enabled
+ * either manually by us or by kernel via enable_on_exec
+ * set later.
+ */
+ if (perf_evsel__is_group_leader(evsel)) {
+ attr->disabled = 1;
+
+ /*
+ * In case of initial_delay we enable tracee
+ * events manually.
+ */
+ if (target__none(target) && !config->initial_delay)
+ attr->enable_on_exec = 1;
+ }
+
+ if (target__has_cpu(target) && !target__has_per_thread(target))
+ return perf_evsel__open_per_cpu(evsel, perf_evsel__cpus(evsel));
+
+ return perf_evsel__open_per_thread(evsel, evsel->threads);
+}
+
+int perf_stat_synthesize_config(struct perf_stat_config *config,
+ struct perf_tool *tool,
+ struct perf_evlist *evlist,
+ perf_event__handler_t process,
+ bool attrs)
+{
+ int err;
+
+ if (attrs) {
+ err = perf_event__synthesize_attrs(tool, evlist, process);
+ if (err < 0) {
+ pr_err("Couldn't synthesize attrs.\n");
+ return err;
+ }
+ }
+
+ err = perf_event__synthesize_extra_attr(tool, evlist, process,
+ attrs);
+
+ err = perf_event__synthesize_thread_map2(tool, evlist->threads,
+ process, NULL);
+ if (err < 0) {
+ pr_err("Couldn't synthesize thread map.\n");
+ return err;
+ }
+
+ err = perf_event__synthesize_cpu_map(tool, evlist->cpus,
+ process, NULL);
+ if (err < 0) {
+ pr_err("Couldn't synthesize thread map.\n");
+ return err;
+ }
+
+ err = perf_event__synthesize_stat_config(tool, config, process, NULL);
+ if (err < 0) {
+ pr_err("Couldn't synthesize config.\n");
+ return err;
+ }
+
+ return 0;
+}
#include <linux/types.h>
#include <stdio.h>
+#include <sys/types.h>
+#include <sys/time.h>
+#include <sys/resource.h>
+#include <sys/wait.h>
#include "xyarray.h"
#include "rblist.h"
+#include "perf.h"
+#include "event.h"
struct stats {
double n, mean, M2;
struct rblist value_list;
};
+typedef int (*aggr_get_id_t)(struct perf_stat_config *config,
+ struct cpu_map *m, int cpu);
+
struct perf_stat_config {
- enum aggr_mode aggr_mode;
- bool scale;
- FILE *output;
- unsigned int interval;
- unsigned int timeout;
- int times;
- struct runtime_stat *stats;
- int stats_num;
+ enum aggr_mode aggr_mode;
+ bool scale;
+ bool no_inherit;
+ bool identifier;
+ bool csv_output;
+ bool interval_clear;
+ bool metric_only;
+ bool null_run;
+ bool ru_display;
+ bool big_num;
+ bool no_merge;
+ bool walltime_run_table;
+ FILE *output;
+ unsigned int interval;
+ unsigned int timeout;
+ unsigned int initial_delay;
+ unsigned int unit_width;
+ unsigned int metric_only_len;
+ int times;
+ int run_count;
+ int print_free_counters_hint;
+ int print_mixed_hw_group_error;
+ struct runtime_stat *stats;
+ int stats_num;
+ const char *csv_sep;
+ struct stats *walltime_nsecs_stats;
+ struct rusage ru_data;
+ struct cpu_map *aggr_map;
+ aggr_get_id_t aggr_get_id;
+ struct cpu_map *cpus_aggr_map;
+ u64 *walltime_run;
+ struct rblist metric_events;
};
void update_stats(struct stats *stats, u64 val);
extern struct runtime_stat rt_stat;
extern struct stats walltime_nsecs_stats;
-typedef void (*print_metric_t)(void *ctx, const char *color, const char *unit,
+typedef void (*print_metric_t)(struct perf_stat_config *config,
+ void *ctx, const char *color, const char *unit,
const char *fmt, double val);
-typedef void (*new_line_t )(void *ctx);
+typedef void (*new_line_t)(struct perf_stat_config *config, void *ctx);
void runtime_stat__init(struct runtime_stat *st);
void runtime_stat__exit(struct runtime_stat *st);
bool force_header;
};
-void perf_stat__print_shadow_stats(struct perf_evsel *evsel,
+void perf_stat__print_shadow_stats(struct perf_stat_config *config,
+ struct perf_evsel *evsel,
double avg, int cpu,
struct perf_stat_output_ctx *out,
struct rblist *metric_events,
struct perf_tool;
union perf_event;
struct perf_session;
-int perf_event__process_stat_event(struct perf_tool *tool,
- union perf_event *event,
- struct perf_session *session);
+int perf_event__process_stat_event(struct perf_session *session,
+ union perf_event *event);
size_t perf_event__fprintf_stat(union perf_event *event, FILE *fp);
size_t perf_event__fprintf_stat_round(union perf_event *event, FILE *fp);
size_t perf_event__fprintf_stat_config(union perf_event *event, FILE *fp);
+
+int create_perf_stat_counter(struct perf_evsel *evsel,
+ struct perf_stat_config *config,
+ struct target *target);
+int perf_stat_synthesize_config(struct perf_stat_config *config,
+ struct perf_tool *tool,
+ struct perf_evlist *evlist,
+ perf_event__handler_t process,
+ bool attrs);
+void
+perf_evlist__print_counters(struct perf_evlist *evlist,
+ struct perf_stat_config *config,
+ struct target *_target,
+ struct timespec *ts,
+ int argc, const char **argv);
#endif
va_copy(ap_saved, ap);
len = vsnprintf(sb->buf + sb->len, sb->alloc - sb->len, fmt, ap);
- if (len < 0)
+ if (len < 0) {
+ va_end(ap_saved);
return len;
+ }
if (len > strbuf_avail(sb)) {
ret = strbuf_grow(sb, len);
- if (ret)
+ if (ret) {
+ va_end(ap_saved);
return ret;
+ }
len = vsnprintf(sb->buf + sb->len, sb->alloc - sb->len, fmt, ap_saved);
va_end(ap_saved);
if (len > strbuf_avail(sb)) {
pr_debug("this should not happen, your vsnprintf is broken");
+ va_end(ap_saved);
return -EINVAL;
}
}
+ va_end(ap_saved);
return strbuf_setlen(sb, sb->len + len);
}
* @branch_count: the branch count when the entry was created
* @cp: call path
* @no_call: a 'call' was not seen
+ * @trace_end: a 'call' but trace ended
*/
struct thread_stack_entry {
u64 ret_addr;
u64 branch_count;
struct call_path *cp;
bool no_call;
+ bool trace_end;
};
/**
return ts;
}
-static int thread_stack__push(struct thread_stack *ts, u64 ret_addr)
+static int thread_stack__push(struct thread_stack *ts, u64 ret_addr,
+ bool trace_end)
{
int err = 0;
}
}
+ ts->stack[ts->cnt].trace_end = trace_end;
ts->stack[ts->cnt++].ret_addr = ret_addr;
return err;
}
}
+static void thread_stack__pop_trace_end(struct thread_stack *ts)
+{
+ size_t i;
+
+ for (i = ts->cnt; i; ) {
+ if (ts->stack[--i].trace_end)
+ ts->cnt = i;
+ else
+ return;
+ }
+}
+
static bool thread_stack__in_kernel(struct thread_stack *ts)
{
if (!ts->cnt)
ret_addr = from_ip + insn_len;
if (ret_addr == to_ip)
return 0; /* Zero-length calls are excluded */
- return thread_stack__push(thread->ts, ret_addr);
- } else if (flags & PERF_IP_FLAG_RETURN) {
- if (!from_ip)
- return 0;
+ return thread_stack__push(thread->ts, ret_addr,
+ flags & PERF_IP_FLAG_TRACE_END);
+ } else if (flags & PERF_IP_FLAG_TRACE_BEGIN) {
+ /*
+ * If the caller did not change the trace number (which would
+ * have flushed the stack) then try to make sense of the stack.
+ * Possibly, tracing began after returning to the current
+ * address, so try to pop that. Also, do not expect a call made
+ * when the trace ended, to return, so pop that.
+ */
+ thread_stack__pop(thread->ts, to_ip);
+ thread_stack__pop_trace_end(thread->ts);
+ } else if ((flags & PERF_IP_FLAG_RETURN) && from_ip) {
thread_stack__pop(thread->ts, to_ip);
}
static int thread_stack__push_cp(struct thread_stack *ts, u64 ret_addr,
u64 timestamp, u64 ref, struct call_path *cp,
- bool no_call)
+ bool no_call, bool trace_end)
{
struct thread_stack_entry *tse;
int err;
tse->branch_count = ts->branch_count;
tse->cp = cp;
tse->no_call = no_call;
+ tse->trace_end = trace_end;
return 0;
}
return -ENOMEM;
return thread_stack__push_cp(thread->ts, ip, sample->time, ref, cp,
- true);
+ true, false);
}
static int thread_stack__no_call_return(struct thread *thread,
if (!cp)
return -ENOMEM;
return thread_stack__push_cp(ts, 0, sample->time, ref,
- cp, true);
+ cp, true, false);
}
} else if (thread_stack__in_kernel(ts) && sample->ip < ks) {
/* Return to userspace, so pop all kernel addresses */
return -ENOMEM;
err = thread_stack__push_cp(ts, sample->addr, sample->time, ref, cp,
- true);
+ true, false);
if (err)
return err;
/* Pop trace end */
tse = &ts->stack[ts->cnt - 1];
- if (tse->cp->sym == NULL && tse->cp->ip == 0) {
+ if (tse->trace_end) {
err = thread_stack__call_return(thread, ts, --ts->cnt,
timestamp, ref, false);
if (err)
ret_addr = sample->ip + sample->insn_len;
return thread_stack__push_cp(ts, ret_addr, sample->time, ref, cp,
- false);
+ false, true);
}
int thread_stack__process(struct thread *thread, struct comm *comm,
ts->last_time = sample->time;
if (sample->flags & PERF_IP_FLAG_CALL) {
+ bool trace_end = sample->flags & PERF_IP_FLAG_TRACE_END;
struct call_path_root *cpr = ts->crp->cpr;
struct call_path *cp;
u64 ret_addr;
if (!cp)
return -ENOMEM;
err = thread_stack__push_cp(ts, ret_addr, sample->time, ref,
- cp, false);
+ cp, false, trace_end);
} else if (sample->flags & PERF_IP_FLAG_RETURN) {
if (!sample->ip || !sample->addr)
return 0;
union perf_event *event,
struct perf_evlist **pevlist);
-typedef int (*event_op2)(struct perf_tool *tool, union perf_event *event,
- struct perf_session *session);
+typedef int (*event_op2)(struct perf_session *session, union perf_event *event);
+typedef s64 (*event_op3)(struct perf_session *session, union perf_event *event);
typedef int (*event_oe)(struct perf_tool *tool, union perf_event *event,
struct ordered_events *oe);
-typedef s64 (*event_op3)(struct perf_tool *tool, union perf_event *event,
- struct perf_session *session);
-
enum show_feature_header {
SHOW_FEAT_NO_HEADER = 0,
SHOW_FEAT_HEADER,
"/tmp/perf-XXXXXX");
if (!mkstemp(tdata->temp_file)) {
pr_debug("Can't make temp file");
+ free(tdata);
return NULL;
}
temp_fd = open(tdata->temp_file, O_RDWR);
if (temp_fd < 0) {
pr_debug("Can't read '%s'", tdata->temp_file);
+ free(tdata);
return NULL;
}
int *offset, int *size, const char *type)
{
struct tep_handle *pevent = context->pevent;
- struct event_format *event;
- struct format_field *field;
+ struct tep_event_format *event;
+ struct tep_format_field *field;
if (!*size) {
- if (!pevent->events)
+
+ event = tep_get_first_event(pevent);
+ if (!event)
return 0;
- event = pevent->events[0];
field = tep_find_common_field(event, type);
if (!field)
return 0;
}
unsigned long long
-raw_field_value(struct event_format *event, const char *name, void *data)
+raw_field_value(struct tep_event_format *event, const char *name, void *data)
{
- struct format_field *field;
+ struct tep_format_field *field;
unsigned long long val;
field = tep_find_any_field(event, name);
return val;
}
-unsigned long long read_size(struct event_format *event, void *ptr, int size)
+unsigned long long read_size(struct tep_event_format *event, void *ptr, int size)
{
return tep_read_number(event->pevent, ptr, size);
}
-void event_format__fprintf(struct event_format *event,
+void event_format__fprintf(struct tep_event_format *event,
int cpu, void *data, int size, FILE *fp)
{
struct tep_record record;
trace_seq_destroy(&s);
}
-void event_format__print(struct event_format *event,
+void event_format__print(struct tep_event_format *event,
int cpu, void *data, int size)
{
return event_format__fprintf(event, cpu, data, size, stdout);
printk = strdup(fmt+1);
line = strtok_r(NULL, "\n", &next);
tep_register_print_string(pevent, printk, addr);
+ free(printk);
}
}
return tep_parse_event(pevent, buf, size, sys);
}
-struct event_format *trace_find_next_event(struct tep_handle *pevent,
- struct event_format *event)
+struct tep_event_format *trace_find_next_event(struct tep_handle *pevent,
+ struct tep_event_format *event)
{
static int idx;
+ int events_count;
+ struct tep_event_format *all_events;
- if (!pevent || !pevent->events)
+ all_events = tep_get_first_event(pevent);
+ events_count = tep_get_events_count(pevent);
+ if (!pevent || !all_events || events_count < 1)
return NULL;
if (!event) {
idx = 0;
- return pevent->events[0];
+ return all_events;
}
- if (idx < pevent->nr_events && event == pevent->events[idx]) {
+ if (idx < events_count && event == (all_events + idx)) {
idx++;
- if (idx == pevent->nr_events)
+ if (idx == events_count)
return NULL;
- return pevent->events[idx];
+ return (all_events + idx);
}
- for (idx = 1; idx < pevent->nr_events; idx++) {
- if (event == pevent->events[idx - 1])
- return pevent->events[idx];
+ for (idx = 1; idx < events_count; idx++) {
+ if (event == (all_events + (idx - 1)))
+ return (all_events + idx);
}
return NULL;
}
if (do_read(&data, 4) < 0)
return 0;
- return __data2host4(pevent, data);
+ return __tep_data2host4(pevent, data);
}
static unsigned long long read8(struct tep_handle *pevent)
if (do_read(&data, 8) < 0)
return 0;
- return __data2host8(pevent, data);
+ return __tep_data2host8(pevent, data);
}
static char *read_string(void)
* The commit field in the page is of type long,
* use that instead, since it represents the kernel.
*/
- tep_set_long_size(pevent, pevent->header_page_size_size);
+ tep_set_long_size(pevent, tep_get_header_page_size(pevent));
}
free(header_page);
}
ret = do_read(buf, size);
- if (ret < 0) {
- free(buf);
+ if (ret < 0)
goto out;
- }
ret = parse_event_file(pevent, buf, size, sys);
if (ret < 0)
for (x=0; x < count; x++) {
size = read8(pevent);
ret = read_event_file(pevent, sys, size);
- if (ret)
+ if (ret) {
+ free(sys);
return ret;
+ }
}
+ free(sys);
}
return 0;
}
/*
* Returns pointer with encoded error via <linux/err.h> interface.
*/
-static struct event_format*
+static struct tep_event_format*
tp_format(const char *sys, const char *name)
{
char *tp_dir = get_events_file(sys);
struct tep_handle *pevent = tevent.pevent;
- struct event_format *event = NULL;
+ struct tep_event_format *event = NULL;
char path[PATH_MAX];
size_t size;
char *data;
/*
* Returns pointer with encoded error via <linux/err.h> interface.
*/
-struct event_format*
+struct tep_event_format*
trace_event__tp_format(const char *sys, const char *name)
{
if (!tevent_initialized && trace_event__init2())
return tp_format(sys, name);
}
-struct event_format *trace_event__tp_format_id(int id)
+struct tep_event_format *trace_event__tp_format_id(int id)
{
if (!tevent_initialized && trace_event__init2())
return ERR_PTR(-ENOMEM);
#define _PERF_UTIL_TRACE_EVENT_H
#include <traceevent/event-parse.h>
+#include <traceevent/trace-seq.h>
#include "parse-events.h"
struct machine;
union perf_event;
struct perf_tool;
struct thread;
-struct plugin_list;
+struct tep_plugin_list;
struct trace_event {
struct tep_handle *pevent;
- struct plugin_list *plugin_list;
+ struct tep_plugin_list *plugin_list;
};
int trace_event__init(struct trace_event *t);
void trace_event__cleanup(struct trace_event *t);
int trace_event__register_resolver(struct machine *machine,
tep_func_resolver_t *func);
-struct event_format*
+struct tep_event_format*
trace_event__tp_format(const char *sys, const char *name);
-struct event_format *trace_event__tp_format_id(int id);
+struct tep_event_format *trace_event__tp_format_id(int id);
int bigendian(void);
-void event_format__fprintf(struct event_format *event,
+void event_format__fprintf(struct tep_event_format *event,
int cpu, void *data, int size, FILE *fp);
-void event_format__print(struct event_format *event,
+void event_format__print(struct tep_event_format *event,
int cpu, void *data, int size);
int parse_ftrace_file(struct tep_handle *pevent, char *buf, unsigned long size);
char *buf, unsigned long size, char *sys);
unsigned long long
-raw_field_value(struct event_format *event, const char *name, void *data);
+raw_field_value(struct tep_event_format *event, const char *name, void *data);
void parse_proc_kallsyms(struct tep_handle *pevent, char *file, unsigned int size);
void parse_ftrace_printk(struct tep_handle *pevent, char *file, unsigned int size);
ssize_t trace_report(int fd, struct trace_event *tevent, bool repipe);
-struct event_format *trace_find_next_event(struct tep_handle *pevent,
- struct event_format *event);
-unsigned long long read_size(struct event_format *event, void *ptr, int size);
+struct tep_event_format *trace_find_next_event(struct tep_handle *pevent,
+ struct tep_event_format *event);
+unsigned long long read_size(struct tep_event_format *event, void *ptr, int size);
unsigned long long eval_flag(const char *flag);
int read_tracing_data(int fd, struct list_head *pattrs);
return err;
}
-static int copyfile_offset(int ifd, loff_t off_in, int ofd, loff_t off_out, u64 size)
+int copyfile_offset(int ifd, loff_t off_in, int ofd, loff_t off_out, u64 size)
{
void *ptr;
loff_t pgoff;
/* glibc 2.20 deprecates _BSD_SOURCE in favour of _DEFAULT_SOURCE */
#define _DEFAULT_SOURCE 1
+#include <fcntl.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdlib.h>
int copyfile(const char *from, const char *to);
int copyfile_mode(const char *from, const char *to, mode_t mode);
int copyfile_ns(const char *from, const char *to, struct nsinfo *nsi);
+int copyfile_offset(int ifd, loff_t off_in, int ofd, loff_t off_out, u64 size);
ssize_t readn(int fd, void *buf, size_t n);
ssize_t writen(int fd, const void *buf, size_t n);
config->cpu = 0;
config->prio = SCHED_HIGH;
config->verbose = 0;
- strncpy(config->governor, "ondemand", 8);
+ strncpy(config->governor, "ondemand", sizeof(config->governor));
config->output = stdout;
unsigned long pstates[MAX_HW_PSTATES] = {0,};
if (cpupower_cpu_info.vendor != X86_VENDOR_AMD &&
+ cpupower_cpu_info.vendor != X86_VENDOR_HYGON &&
cpupower_cpu_info.vendor != X86_VENDOR_INTEL)
return 0;
printf(_(" Supported: %s\n"), support ? _("yes") : _("no"));
printf(_(" Active: %s\n"), active ? _("yes") : _("no"));
- if (cpupower_cpu_info.vendor == X86_VENDOR_AMD &&
- cpupower_cpu_info.family >= 0x10) {
+ if ((cpupower_cpu_info.vendor == X86_VENDOR_AMD &&
+ cpupower_cpu_info.family >= 0x10) ||
+ cpupower_cpu_info.vendor == X86_VENDOR_HYGON) {
ret = decode_pstates(cpu, cpupower_cpu_info.family, b_states,
pstates, &pstate_no);
if (ret)
printf(_(" Boost States: %d\n"), b_states);
printf(_(" Total States: %d\n"), pstate_no);
for (i = 0; i < pstate_no; i++) {
+ if (!pstates[i])
+ continue;
if (i < b_states)
printf(_(" Pstate-Pb%d: %luMHz (boost state)"
"\n"), i, pstates[i]);
unsigned vid:8;
unsigned iddval:8;
unsigned idddiv:2;
- unsigned res1:30;
+ unsigned res1:31;
unsigned en:1;
} fam17h_bits;
unsigned long long val;
if (family == 0x12)
t = pstate.val & 0xf;
- else if (family == 0x17)
+ else if (family == 0x17 || family == 0x18)
t = pstate.fam17h_bits.did;
else
t = pstate.bits.did;
int fid, did, cof;
did = get_did(family, pstate);
- if (family == 0x17) {
+ if (family == 0x17 || family == 0x18) {
fid = pstate.fam17h_bits.fid;
cof = 200 * fid / did;
} else {
}
if (read_msr(cpu, MSR_AMD_PSTATE + i, &pstate.val))
return -1;
+ if ((cpu_family == 0x17) && (!pstate.fam17h_bits.en))
+ continue;
+ else if (!pstate.bits.en)
+ continue;
+
pstates[i] = get_cof(cpu_family, pstate);
}
*no = i;
#include "helpers/helpers.h"
static const char *cpu_vendor_table[X86_VENDOR_MAX] = {
- "Unknown", "GenuineIntel", "AuthenticAMD",
+ "Unknown", "GenuineIntel", "AuthenticAMD", "HygonGenuine",
};
#if defined(__i386__) || defined(__x86_64__)
fclose(fp);
/* Get some useful CPU capabilities from cpuid */
if (cpu_info->vendor != X86_VENDOR_AMD &&
+ cpu_info->vendor != X86_VENDOR_HYGON &&
cpu_info->vendor != X86_VENDOR_INTEL)
return ret;
if (cpuid_level >= 6 && (cpuid_ecx(6) & 0x1))
cpu_info->caps |= CPUPOWER_CAP_APERF;
- /* AMD Boost state enable/disable register */
- if (cpu_info->vendor == X86_VENDOR_AMD) {
+ /* AMD or Hygon Boost state enable/disable register */
+ if (cpu_info->vendor == X86_VENDOR_AMD ||
+ cpu_info->vendor == X86_VENDOR_HYGON) {
if (ext_cpuid_level >= 0x80000007 &&
(cpuid_edx(0x80000007) & (1 << 9)))
cpu_info->caps |= CPUPOWER_CAP_AMD_CBP;
/* cpuid and cpuinfo helpers **************************/
enum cpupower_cpu_vendor {X86_VENDOR_UNKNOWN = 0, X86_VENDOR_INTEL,
- X86_VENDOR_AMD, X86_VENDOR_MAX};
+ X86_VENDOR_AMD, X86_VENDOR_HYGON, X86_VENDOR_MAX};
#define CPUPOWER_CAP_INV_TSC 0x00000001
#define CPUPOWER_CAP_APERF 0x00000002
* has Hardware determined variable increments instead.
*/
- if (cpu_info.family == 0x17) {
+ if (cpu_info.family == 0x17 || cpu_info.family == 0x18) {
if (!read_msr(cpu, MSR_AMD_HWCR, &val)) {
if (!(val & CPUPOWER_AMD_CPBDIS))
*active = 1;
if (!(cpupower_cpu_info.caps & CPUPOWER_CAP_INV_TSC))
goto use_sysfs;
- if (cpupower_cpu_info.vendor == X86_VENDOR_AMD) {
+ if (cpupower_cpu_info.vendor == X86_VENDOR_AMD ||
+ cpupower_cpu_info.vendor == X86_VENDOR_HYGON) {
/* MSR_AMD_HWCR tells us whether TSC runs at P0/mperf
* freq.
* A test whether hwcr is accessable/available would be:
install -m 644 config/suspend-x2-proc.cfg $(DESTDIR)$(PREFIX)/lib/pm-graph/config
install -d $(DESTDIR)$(PREFIX)/bin
- ln -s $(DESTDIR)$(PREFIX)/lib/pm-graph/bootgraph.py $(DESTDIR)$(PREFIX)/bin/bootgraph
- ln -s $(DESTDIR)$(PREFIX)/lib/pm-graph/sleepgraph.py $(DESTDIR)$(PREFIX)/bin/sleepgraph
+ ln -s ../lib/pm-graph/bootgraph.py $(DESTDIR)$(PREFIX)/bin/bootgraph
+ ln -s ../lib/pm-graph/sleepgraph.py $(DESTDIR)$(PREFIX)/bin/sleepgraph
install -d $(DESTDIR)$(PREFIX)/share/man/man8
install bootgraph.8 $(DESTDIR)$(PREFIX)/share/man/man8
from subprocess import call, Popen, PIPE
import sleepgraph as aslib
+def pprint(msg):
+ print(msg)
+ sys.stdout.flush()
+
# ----------------- CLASSES --------------------
# Class: SystemValues
return cmdline
def manualRebootRequired(self):
cmdline = self.kernelParams()
- print 'To generate a new timeline manually, follow these steps:\n'
- print '1. Add the CMDLINE string to your kernel command line.'
- print '2. Reboot the system.'
- print '3. After reboot, re-run this tool with the same arguments but no command (w/o -reboot or -manual).\n'
- print 'CMDLINE="%s"' % cmdline
+ pprint('To generate a new timeline manually, follow these steps:\n\n'\
+ '1. Add the CMDLINE string to your kernel command line.\n'\
+ '2. Reboot the system.\n'\
+ '3. After reboot, re-run this tool with the same arguments but no command (w/o -reboot or -manual).\n\n'\
+ 'CMDLINE="%s"' % cmdline)
sys.exit()
def blGrub(self):
blcmd = ''
if len(cg.list) < 1 or cg.invalid or (cg.end - cg.start == 0):
continue
if(not cg.postProcess()):
- print('Sanity check failed for %s-%d' % (proc, pid))
+ pprint('Sanity check failed for %s-%d' % (proc, pid))
continue
# match cg data to devices
devname = data.deviceMatch(pid, cg)
sysvals.vprint('%s callgraph found for %s %s-%d [%f - %f]' %\
(kind, cg.name, proc, pid, cg.start, cg.end))
elif len(cg.list) > 1000000:
- print 'WARNING: the callgraph found for %s is massive! (%d lines)' %\
- (devname, len(cg.list))
+ pprint('WARNING: the callgraph found for %s is massive! (%d lines)' %\
+ (devname, len(cg.list)))
# Function: retrieveLogs
# Description:
tMax = data.end
tTotal = tMax - t0
if(tTotal == 0):
- print('ERROR: No timeline data')
+ pprint('ERROR: No timeline data')
return False
user_mode = '%.0f'%(data.tUserMode*1000)
last_init = '%.0f'%(tTotal*1000)
op.close()
res = call([cmd, cronfile])
except Exception, e:
- print 'Exception: %s' % str(e)
+ pprint('Exception: %s' % str(e))
shutil.move(backfile, cronfile)
res = -1
if res != 0:
call(sysvals.blexec, stderr=PIPE, stdout=PIPE,
env={'PATH': '.:/sbin:/usr/sbin:/usr/bin:/sbin:/bin'})
except Exception, e:
- print 'Exception: %s\n' % str(e)
+ pprint('Exception: %s\n' % str(e))
return
# extract the option and create a grub config without it
sysvals.rootUser(True)
res = call(sysvals.blexec)
os.remove(grubfile)
except Exception, e:
- print 'Exception: %s' % str(e)
+ pprint('Exception: %s' % str(e))
res = -1
# cleanup
shutil.move(tempfile, grubfile)
def doError(msg, help=False):
if help == True:
printHelp()
- print 'ERROR: %s\n' % msg
+ pprint('ERROR: %s\n' % msg)
sysvals.outputResult({'error':msg})
sys.exit()
# Description:
# print out the help text
def printHelp():
- print('')
- print('%s v%s' % (sysvals.title, sysvals.version))
- print('Usage: bootgraph <options> <command>')
- print('')
- print('Description:')
- print(' This tool reads in a dmesg log of linux kernel boot and')
- print(' creates an html representation of the boot timeline up to')
- print(' the start of the init process.')
- print('')
- print(' If no specific command is given the tool reads the current dmesg')
- print(' and/or ftrace log and creates a timeline')
- print('')
- print(' Generates output files in subdirectory: boot-yymmdd-HHMMSS')
- print(' HTML output: <hostname>_boot.html')
- print(' raw dmesg output: <hostname>_boot_dmesg.txt')
- print(' raw ftrace output: <hostname>_boot_ftrace.txt')
- print('')
- print('Options:')
- print(' -h Print this help text')
- print(' -v Print the current tool version')
- print(' -verbose Print extra information during execution and analysis')
- print(' -addlogs Add the dmesg log to the html output')
- print(' -result fn Export a results table to a text file for parsing.')
- print(' -o name Overrides the output subdirectory name when running a new test')
- print(' default: boot-{date}-{time}')
- print(' [advanced]')
- print(' -fstat Use ftrace to add function detail and statistics (default: disabled)')
- print(' -f/-callgraph Add callgraph detail, can be very large (default: disabled)')
- print(' -maxdepth N limit the callgraph data to N call levels (default: 2)')
- print(' -mincg ms Discard all callgraphs shorter than ms milliseconds (e.g. 0.001 for us)')
- print(' -timeprec N Number of significant digits in timestamps (0:S, 3:ms, [6:us])')
- print(' -expandcg pre-expand the callgraph data in the html output (default: disabled)')
- print(' -func list Limit ftrace to comma-delimited list of functions (default: do_one_initcall)')
- print(' -cgfilter S Filter the callgraph output in the timeline')
- print(' -cgskip file Callgraph functions to skip, off to disable (default: cgskip.txt)')
- print(' -bl name Use the following boot loader for kernel params (default: grub)')
- print(' -reboot Reboot the machine automatically and generate a new timeline')
- print(' -manual Show the steps to generate a new timeline manually (used with -reboot)')
- print('')
- print('Other commands:')
- print(' -flistall Print all functions capable of being captured in ftrace')
- print(' -sysinfo Print out system info extracted from BIOS')
- print(' [redo]')
- print(' -dmesg file Create HTML output using dmesg input (used with -ftrace)')
- print(' -ftrace file Create HTML output using ftrace input (used with -dmesg)')
- print('')
+ pprint('\n%s v%s\n'\
+ 'Usage: bootgraph <options> <command>\n'\
+ '\n'\
+ 'Description:\n'\
+ ' This tool reads in a dmesg log of linux kernel boot and\n'\
+ ' creates an html representation of the boot timeline up to\n'\
+ ' the start of the init process.\n'\
+ '\n'\
+ ' If no specific command is given the tool reads the current dmesg\n'\
+ ' and/or ftrace log and creates a timeline\n'\
+ '\n'\
+ ' Generates output files in subdirectory: boot-yymmdd-HHMMSS\n'\
+ ' HTML output: <hostname>_boot.html\n'\
+ ' raw dmesg output: <hostname>_boot_dmesg.txt\n'\
+ ' raw ftrace output: <hostname>_boot_ftrace.txt\n'\
+ '\n'\
+ 'Options:\n'\
+ ' -h Print this help text\n'\
+ ' -v Print the current tool version\n'\
+ ' -verbose Print extra information during execution and analysis\n'\
+ ' -addlogs Add the dmesg log to the html output\n'\
+ ' -result fn Export a results table to a text file for parsing.\n'\
+ ' -o name Overrides the output subdirectory name when running a new test\n'\
+ ' default: boot-{date}-{time}\n'\
+ ' [advanced]\n'\
+ ' -fstat Use ftrace to add function detail and statistics (default: disabled)\n'\
+ ' -f/-callgraph Add callgraph detail, can be very large (default: disabled)\n'\
+ ' -maxdepth N limit the callgraph data to N call levels (default: 2)\n'\
+ ' -mincg ms Discard all callgraphs shorter than ms milliseconds (e.g. 0.001 for us)\n'\
+ ' -timeprec N Number of significant digits in timestamps (0:S, 3:ms, [6:us])\n'\
+ ' -expandcg pre-expand the callgraph data in the html output (default: disabled)\n'\
+ ' -func list Limit ftrace to comma-delimited list of functions (default: do_one_initcall)\n'\
+ ' -cgfilter S Filter the callgraph output in the timeline\n'\
+ ' -cgskip file Callgraph functions to skip, off to disable (default: cgskip.txt)\n'\
+ ' -bl name Use the following boot loader for kernel params (default: grub)\n'\
+ ' -reboot Reboot the machine automatically and generate a new timeline\n'\
+ ' -manual Show the steps to generate a new timeline manually (used with -reboot)\n'\
+ '\n'\
+ 'Other commands:\n'\
+ ' -flistall Print all functions capable of being captured in ftrace\n'\
+ ' -sysinfo Print out system info extracted from BIOS\n'\
+ ' [redo]\n'\
+ ' -dmesg file Create HTML output using dmesg input (used with -ftrace)\n'\
+ ' -ftrace file Create HTML output using ftrace input (used with -dmesg)\n'\
+ '' % (sysvals.title, sysvals.version))
return True
# ----------------- MAIN --------------------
printHelp()
sys.exit()
elif(arg == '-v'):
- print("Version %s" % sysvals.version)
+ pprint("Version %s" % sysvals.version)
sys.exit()
elif(arg == '-verbose'):
sysvals.verbose = True
print f
elif cmd == 'checkbl':
sysvals.getBootLoader()
- print 'Boot Loader: %s\n%s' % (sysvals.bootloader, sysvals.blexec)
+ pprint('Boot Loader: %s\n%s' % (sysvals.bootloader, sysvals.blexec))
elif(cmd == 'sysinfo'):
sysvals.printSystemInfo(True)
sys.exit()
# console calls
printk
dev_printk
+__dev_printk
console_unlock
# memory handling
# example: [color=#CC00CC]
#
# arglist: A list of arguments from registers/stack addresses. See URL:
-# https://www.kernel.org/doc/Documentation/trace/kprobetrace.rst
+# https://www.kernel.org/doc/Documentation/trace/kprobetrace.txt
#
# example: cpu=%di:s32
#
# example: [color=#CC00CC]
#
# arglist: A list of arguments from registers/stack addresses. See URL:
-# https://www.kernel.org/doc/Documentation/trace/kprobetrace.rst
+# https://www.kernel.org/doc/Documentation/trace/kprobetrace.txt
#
# example: port=+36(%di):s32
#
by 5 seconds to allow runtime suspend changes to occur. The settings are restored
after the test is complete.
.TP
-\fB-display \fIon/off\fR
-Turn the display on or off for the test using the xset command. This helps
-maintain the consistecy of test data for better comparison.
+\fB-display \fIon/off/standby/suspend\fR
+Switch the display to the requested mode for the test using the xset command.
+This helps maintain the consistency of test data for better comparison.
.TP
\fB-skiphtml\fR
Run the test and capture the trace logs, but skip the timeline generation.
\fB-battery\fR
Print out battery status and current charge.
.TP
+\fB-xon/-xoff/-xstandby/-xsuspend\fR
+Test xset by attempting to switch the display to the given mode. This
+is the same command which will be issued by \fB-display \fImode\fR.
+.TP
+\fB-xstat\fR
+Get the current DPMS display mode.
+.TP
\fB-sysinfo\fR
Print out system info extracted from BIOS. Reads /dev/mem directly instead of going through dmidecode.
.TP
import string
import re
import platform
+import signal
from datetime import datetime
import struct
import ConfigParser
from threading import Thread
from subprocess import call, Popen, PIPE
+def pprint(msg):
+ print(msg)
+ sys.stdout.flush()
+
# ----------------- CLASSES --------------------
# Class: SystemValues
# store system values and test parameters
class SystemValues:
title = 'SleepGraph'
- version = '5.1'
+ version = '5.2'
ansi = False
rs = 0
- display = 0
+ display = ''
gzip = False
sync = False
verbose = False
tpath = '/sys/kernel/debug/tracing/'
fpdtpath = '/sys/firmware/acpi/tables/FPDT'
epath = '/sys/kernel/debug/tracing/events/power/'
+ pmdpath = '/sys/power/pm_debug_messages'
traceevents = [
'suspend_resume',
'device_pm_callback_end',
mempath = '/dev/mem'
powerfile = '/sys/power/state'
mempowerfile = '/sys/power/mem_sleep'
+ diskpowerfile = '/sys/power/disk'
suspendmode = 'mem'
memmode = ''
+ diskmode = ''
hostname = 'localhost'
prefix = 'test'
teststamp = ''
useprocmon = False
notestrun = False
cgdump = False
+ devdump = False
mixedphaseheight = True
devprops = dict()
predelay = 0
postdelay = 0
- procexecfmt = 'ps - (?P<ps>.*)$'
- devpropfmt = '# Device Properties: .*'
- tracertypefmt = '# tracer: (?P<t>.*)'
- firmwarefmt = '# fwsuspend (?P<s>[0-9]*) fwresume (?P<r>[0-9]*)$'
+ pmdebug = ''
tracefuncs = {
'sys_sync': {},
+ 'ksys_sync': {},
'__pm_notifier_call_chain': {},
'pm_prepare_console': {},
'pm_notifier_call_chain': {},
dev_tracefuncs = {
# general wait/delay/sleep
'msleep': { 'args_x86_64': {'time':'%di:s32'}, 'ub': 1 },
- 'schedule_timeout_uninterruptible': { 'args_x86_64': {'timeout':'%di:s32'}, 'ub': 1 },
'schedule_timeout': { 'args_x86_64': {'timeout':'%di:s32'}, 'ub': 1 },
'udelay': { 'func':'__const_udelay', 'args_x86_64': {'loops':'%di:s32'}, 'ub': 1 },
'usleep_range': { 'args_x86_64': {'min':'%di:s32', 'max':'%si:s32'}, 'ub': 1 },
# filesystem
'ext4_sync_fs': {},
# 80211
+ 'ath10k_bmi_read_memory': { 'args_x86_64': {'length':'%cx:s32'} },
+ 'ath10k_bmi_write_memory': { 'args_x86_64': {'length':'%cx:s32'} },
+ 'ath10k_bmi_fast_download': { 'args_x86_64': {'length':'%cx:s32'} },
'iwlagn_mac_start': {},
'iwlagn_alloc_bcast_station': {},
'iwl_trans_pcie_start_hw': {},
timeformat = '%.3f'
cmdline = '%s %s' % \
(os.path.basename(sys.argv[0]), ' '.join(sys.argv[1:]))
+ sudouser = ''
def __init__(self):
self.archargs = 'args_'+platform.machine()
self.hostname = platform.node()
if (hasattr(sys.stdout, 'isatty') and sys.stdout.isatty()):
self.ansi = True
self.testdir = datetime.now().strftime('suspend-%y%m%d-%H%M%S')
+ if os.getuid() == 0 and 'SUDO_USER' in os.environ and \
+ os.environ['SUDO_USER']:
+ self.sudouser = os.environ['SUDO_USER']
def vprint(self, msg):
self.logmsg += msg+'\n'
- if(self.verbose):
- print(msg)
+ if self.verbose or msg.startswith('WARNING:'):
+ pprint(msg)
+ def signalHandler(self, signum, frame):
+ if not self.result:
+ return
+ signame = self.signames[signum] if signum in self.signames else 'UNKNOWN'
+ msg = 'Signal %s caused a tool exit, line %d' % (signame, frame.f_lineno)
+ sysvals.outputResult({'error':msg})
+ sys.exit(3)
+ def signalHandlerInit(self):
+ capture = ['BUS', 'SYS', 'XCPU', 'XFSZ', 'PWR', 'HUP', 'INT', 'QUIT',
+ 'ILL', 'ABRT', 'FPE', 'SEGV', 'TERM', 'TSTP']
+ self.signames = dict()
+ for i in capture:
+ s = 'SIG'+i
+ try:
+ signum = getattr(signal, s)
+ signal.signal(signum, self.signalHandler)
+ except:
+ continue
+ self.signames[signum] = s
def rootCheck(self, fatal=True):
if(os.access(self.powerfile, os.W_OK)):
return True
if fatal:
msg = 'This command requires sysfs mount and root access'
- print('ERROR: %s\n') % msg
+ pprint('ERROR: %s\n' % msg)
self.outputResult({'error':msg})
- sys.exit()
+ sys.exit(1)
return False
def rootUser(self, fatal=False):
if 'USER' in os.environ and os.environ['USER'] == 'root':
return True
if fatal:
msg = 'This command must be run as root'
- print('ERROR: %s\n') % msg
+ pprint('ERROR: %s\n' % msg)
self.outputResult({'error':msg})
- sys.exit()
+ sys.exit(1)
return False
def getExec(self, cmd):
dirlist = ['/sbin', '/bin', '/usr/sbin', '/usr/bin',
ktime = m.group('ktime')
fp.close()
self.dmesgstart = float(ktime)
- def getdmesg(self, fwdata=[]):
- op = self.writeDatafileHeader(sysvals.dmesgfile, fwdata)
+ def getdmesg(self, testdata):
+ op = self.writeDatafileHeader(sysvals.dmesgfile, testdata)
# store all new dmesg lines since initdmesg was called
fp = Popen('dmesg', stdout=PIPE).stdout
for line in fp:
if len(self.kprobes) < 1:
return
if output:
- print(' kprobe functions in this kernel:')
+ pprint(' kprobe functions in this kernel:')
# first test each kprobe
rejects = []
# sort kprobes: trace, ub-dev, custom, dev
else:
kpl[2].append(name)
if output:
- print(' %s: %s' % (name, res))
+ pprint(' %s: %s' % (name, res))
kplist = kpl[0] + kpl[1] + kpl[2] + kpl[3]
# remove all failed ones from the list
for name in rejects:
if output:
check = self.fgetVal('kprobe_events')
linesack = (len(check.split('\n')) - 1) / 2
- print(' kprobe functions enabled: %d/%d' % (linesack, linesout))
+ pprint(' kprobe functions enabled: %d/%d' % (linesack, linesout))
self.fsetVal('1', 'events/kprobes/enable')
def testKprobe(self, kname, kprobe):
self.fsetVal('0', 'events/kprobes/enable')
self.fsetVal('0', 'events/kprobes/enable')
self.fsetVal('', 'kprobe_events')
self.fsetVal('1024', 'buffer_size_kb')
+ if self.pmdebug:
+ self.setVal(self.pmdebug, self.pmdpath)
def setupAllKprobes(self):
for name in self.tracefuncs:
self.defaultKprobe(name, self.tracefuncs[name])
return False
def initFtrace(self):
self.printSystemInfo(False)
- print('INITIALIZING FTRACE...')
+ pprint('INITIALIZING FTRACE...')
# turn trace off
self.fsetVal('0', 'tracing_on')
self.cleanupFtrace()
+ # pm debug messages
+ pv = self.getVal(self.pmdpath)
+ if pv != '1':
+ self.setVal('1', self.pmdpath)
+ self.pmdebug = pv
# set the trace clock to global
self.fsetVal('global', 'trace_clock')
self.fsetVal('nop', 'current_tracer')
if self.bufsize > 0:
tgtsize = self.bufsize
elif self.usecallgraph or self.usedevsrc:
- tgtsize = min(self.memfree, 3*1024*1024)
+ bmax = (1*1024*1024) if self.suspendmode == 'disk' else (3*1024*1024)
+ tgtsize = min(self.memfree, bmax)
else:
tgtsize = 65536
while not self.fsetVal('%d' % (tgtsize / cpus), 'buffer_size_kb'):
if tgtsize < 65536:
tgtsize = int(self.fgetVal('buffer_size_kb')) * cpus
break
- print 'Setting trace buffers to %d kB (%d kB per cpu)' % (tgtsize, tgtsize/cpus)
+ pprint('Setting trace buffers to %d kB (%d kB per cpu)' % (tgtsize, tgtsize/cpus))
# initialize the callgraph trace
if(self.usecallgraph):
# set trace type
if self.usedevsrc:
for name in self.dev_tracefuncs:
self.defaultKprobe(name, self.dev_tracefuncs[name])
- print('INITIALIZING KPROBES...')
+ pprint('INITIALIZING KPROBES...')
self.addKprobes(self.verbose)
if(self.usetraceevents):
# turn trace events on
if not self.ansi:
return str
return '\x1B[%d;40m%s\x1B[m' % (color, str)
- def writeDatafileHeader(self, filename, fwdata=[]):
+ def writeDatafileHeader(self, filename, testdata):
fp = self.openlog(filename, 'w')
fp.write('%s\n%s\n# command | %s\n' % (self.teststamp, self.sysstamp, self.cmdline))
- if(self.suspendmode == 'mem' or self.suspendmode == 'command'):
- for fw in fwdata:
+ for test in testdata:
+ if 'fw' in test:
+ fw = test['fw']
if(fw):
fp.write('# fwsuspend %u fwresume %u\n' % (fw[0], fw[1]))
+ if 'bat' in test:
+ (a1, c1), (a2, c2) = test['bat']
+ fp.write('# battery %s %d %s %d\n' % (a1, c1, a2, c2))
+ if test['error'] or len(testdata) > 1:
+ fp.write('# enter_sleep_error %s\n' % test['error'])
return fp
- def sudouser(self, dir):
- if os.path.exists(dir) and os.getuid() == 0 and \
- 'SUDO_USER' in os.environ:
+ def sudoUserchown(self, dir):
+ if os.path.exists(dir) and self.sudouser:
cmd = 'chown -R {0}:{0} {1} > /dev/null 2>&1'
- call(cmd.format(os.environ['SUDO_USER'], dir), shell=True)
+ call(cmd.format(self.sudouser, dir), shell=True)
def outputResult(self, testdata, num=0):
if not self.result:
return
if 'bugurl' in testdata:
fp.write('url%s: %s\n' % (n, testdata['bugurl']))
fp.close()
- self.sudouser(self.result)
+ self.sudoUserchown(self.result)
def configFile(self, file):
dir = os.path.dirname(os.path.realpath(__file__))
if os.path.exists(file):
# Simple class which holds property values collected
# for all the devices used in the timeline.
class DevProps:
- syspath = ''
- altname = ''
- async = True
- xtraclass = ''
- xtrainfo = ''
+ def __init__(self):
+ self.syspath = ''
+ self.altname = ''
+ self.async = True
+ self.xtraclass = ''
+ self.xtrainfo = ''
def out(self, dev):
return '%s,%s,%d;' % (dev, self.altname, self.async)
def debug(self, dev):
- print '%s:\n\taltname = %s\n\t async = %s' % (dev, self.altname, self.async)
+ pprint('%s:\n\taltname = %s\n\t async = %s' % (dev, self.altname, self.async))
def altName(self, dev):
if not self.altname or self.altname == dev:
return dev
# A container used to create a device hierachy, with a single root node
# and a tree of child nodes. Used by Data.deviceTopology()
class DeviceNode:
- name = ''
- children = 0
- depth = 0
def __init__(self, nodename, nodedepth):
self.name = nodename
self.children = []
# }
#
class Data:
- dmesg = {} # root data structure
- phases = [] # ordered list of phases
- start = 0.0 # test start
- end = 0.0 # test end
- tSuspended = 0.0 # low-level suspend start
- tResumed = 0.0 # low-level resume start
- tKernSus = 0.0 # kernel level suspend start
- tKernRes = 0.0 # kernel level resume end
- tLow = 0.0 # time spent in low-level suspend (standby/freeze)
- fwValid = False # is firmware data available
- fwSuspend = 0 # time spent in firmware suspend
- fwResume = 0 # time spent in firmware resume
- dmesgtext = [] # dmesg text file in memory
- pstl = 0 # process timeline
- testnumber = 0
- idstr = ''
- html_device_id = 0
- stamp = 0
- outfile = ''
- devpids = []
- kerror = False
+ phasedef = {
+ 'suspend_prepare': {'order': 0, 'color': '#CCFFCC'},
+ 'suspend': {'order': 1, 'color': '#88FF88'},
+ 'suspend_late': {'order': 2, 'color': '#00AA00'},
+ 'suspend_noirq': {'order': 3, 'color': '#008888'},
+ 'suspend_machine': {'order': 4, 'color': '#0000FF'},
+ 'resume_machine': {'order': 5, 'color': '#FF0000'},
+ 'resume_noirq': {'order': 6, 'color': '#FF9900'},
+ 'resume_early': {'order': 7, 'color': '#FFCC00'},
+ 'resume': {'order': 8, 'color': '#FFFF88'},
+ 'resume_complete': {'order': 9, 'color': '#FFFFCC'},
+ }
+ errlist = {
+ 'HWERROR' : '.*\[ *Hardware Error *\].*',
+ 'FWBUG' : '.*\[ *Firmware Bug *\].*',
+ 'BUG' : '.*BUG.*',
+ 'ERROR' : '.*ERROR.*',
+ 'WARNING' : '.*WARNING.*',
+ 'IRQ' : '.*genirq: .*',
+ 'TASKFAIL': '.*Freezing of tasks failed.*',
+ }
def __init__(self, num):
idchar = 'abcdefghij'
- self.pstl = dict()
+ self.start = 0.0 # test start
+ self.end = 0.0 # test end
+ self.tSuspended = 0.0 # low-level suspend start
+ self.tResumed = 0.0 # low-level resume start
+ self.tKernSus = 0.0 # kernel level suspend start
+ self.tKernRes = 0.0 # kernel level resume end
+ self.fwValid = False # is firmware data available
+ self.fwSuspend = 0 # time spent in firmware suspend
+ self.fwResume = 0 # time spent in firmware resume
+ self.html_device_id = 0
+ self.stamp = 0
+ self.outfile = ''
+ self.kerror = False
+ self.battery = 0
+ self.enterfail = ''
+ self.currphase = ''
+ self.pstl = dict() # process timeline
self.testnumber = num
self.idstr = idchar[num]
- self.dmesgtext = []
- self.phases = []
- self.dmesg = { # fixed list of 10 phases
- 'suspend_prepare': {'list': dict(), 'start': -1.0, 'end': -1.0,
- 'row': 0, 'color': '#CCFFCC', 'order': 0},
- 'suspend': {'list': dict(), 'start': -1.0, 'end': -1.0,
- 'row': 0, 'color': '#88FF88', 'order': 1},
- 'suspend_late': {'list': dict(), 'start': -1.0, 'end': -1.0,
- 'row': 0, 'color': '#00AA00', 'order': 2},
- 'suspend_noirq': {'list': dict(), 'start': -1.0, 'end': -1.0,
- 'row': 0, 'color': '#008888', 'order': 3},
- 'suspend_machine': {'list': dict(), 'start': -1.0, 'end': -1.0,
- 'row': 0, 'color': '#0000FF', 'order': 4},
- 'resume_machine': {'list': dict(), 'start': -1.0, 'end': -1.0,
- 'row': 0, 'color': '#FF0000', 'order': 5},
- 'resume_noirq': {'list': dict(), 'start': -1.0, 'end': -1.0,
- 'row': 0, 'color': '#FF9900', 'order': 6},
- 'resume_early': {'list': dict(), 'start': -1.0, 'end': -1.0,
- 'row': 0, 'color': '#FFCC00', 'order': 7},
- 'resume': {'list': dict(), 'start': -1.0, 'end': -1.0,
- 'row': 0, 'color': '#FFFF88', 'order': 8},
- 'resume_complete': {'list': dict(), 'start': -1.0, 'end': -1.0,
- 'row': 0, 'color': '#FFFFCC', 'order': 9}
- }
- self.phases = self.sortedPhases()
+ self.dmesgtext = [] # dmesg text file in memory
+ self.dmesg = dict() # root data structure
+ self.errorinfo = {'suspend':[],'resume':[]}
+ self.tLow = [] # time spent in low-level suspends (standby/freeze)
+ self.devpids = []
+ self.devicegroups = 0
+ def sortedPhases(self):
+ return sorted(self.dmesg, key=lambda k:self.dmesg[k]['order'])
+ def initDevicegroups(self):
+ # called when phases are all finished being added
+ for phase in self.dmesg.keys():
+ if '*' in phase:
+ p = phase.split('*')
+ pnew = '%s%d' % (p[0], len(p))
+ self.dmesg[pnew] = self.dmesg.pop(phase)
self.devicegroups = []
- for phase in self.phases:
+ for phase in self.sortedPhases():
self.devicegroups.append([phase])
- self.errorinfo = {'suspend':[],'resume':[]}
+ def nextPhase(self, phase, offset):
+ order = self.dmesg[phase]['order'] + offset
+ for p in self.dmesg:
+ if self.dmesg[p]['order'] == order:
+ return p
+ return ''
+ def lastPhase(self):
+ plist = self.sortedPhases()
+ if len(plist) < 1:
+ return ''
+ return plist[-1]
def extractErrorInfo(self):
- elist = {
- 'HWERROR' : '.*\[ *Hardware Error *\].*',
- 'FWBUG' : '.*\[ *Firmware Bug *\].*',
- 'BUG' : '.*BUG.*',
- 'ERROR' : '.*ERROR.*',
- 'WARNING' : '.*WARNING.*',
- 'IRQ' : '.*genirq: .*',
- 'TASKFAIL': '.*Freezing of tasks failed.*',
- }
lf = sysvals.openlog(sysvals.dmesgfile, 'r')
i = 0
list = []
continue
dir = 'suspend' if t < self.tSuspended else 'resume'
msg = m.group('msg')
- for err in elist:
- if re.match(elist[err], msg):
+ for err in self.errlist:
+ if re.match(self.errlist[err], msg):
list.append((err, dir, t, i, i))
self.kerror = True
break
def setEnd(self, time):
self.end = time
def isTraceEventOutsideDeviceCalls(self, pid, time):
- for phase in self.phases:
+ for phase in self.sortedPhases():
list = self.dmesg[phase]['list']
for dev in list:
d = list[dev]
time < d['end']):
return False
return True
- def phaseCollision(self, phase, isbegin, line):
- key = 'end'
- if isbegin:
- key = 'start'
- if self.dmesg[phase][key] >= 0:
- sysvals.vprint('IGNORE: %s' % line.strip())
- return True
- return False
def sourcePhase(self, start):
- for phase in self.phases:
+ for phase in self.sortedPhases():
+ if 'machine' in phase:
+ continue
pend = self.dmesg[phase]['end']
if start <= pend:
return phase
return tgtdev
def addDeviceFunctionCall(self, displayname, kprobename, proc, pid, start, end, cdata, rdata):
# try to place the call in a device
- tgtdev = self.sourceDevice(self.phases, start, end, pid, 'device')
+ phases = self.sortedPhases()
+ tgtdev = self.sourceDevice(phases, start, end, pid, 'device')
# calls with device pids that occur outside device bounds are dropped
# TODO: include these somehow
if not tgtdev and pid in self.devpids:
return False
# try to place the call in a thread
if not tgtdev:
- tgtdev = self.sourceDevice(self.phases, start, end, pid, 'thread')
+ tgtdev = self.sourceDevice(phases, start, end, pid, 'thread')
# create new thread blocks, expand as new calls are found
if not tgtdev:
if proc == '<...>':
def overflowDevices(self):
# get a list of devices that extend beyond the end of this test run
devlist = []
- for phase in self.phases:
+ for phase in self.sortedPhases():
list = self.dmesg[phase]['list']
for devname in list:
dev = list[devname]
# merge any devices that overlap devlist
for dev in devlist:
devname = dev['name']
- for phase in self.phases:
+ for phase in self.sortedPhases():
list = self.dmesg[phase]['list']
if devname not in list:
continue
del list[devname]
def usurpTouchingThread(self, name, dev):
# the caller test has priority of this thread, give it to him
- for phase in self.phases:
+ for phase in self.sortedPhases():
list = self.dmesg[phase]['list']
if name in list:
tdev = list[name]
break
def stitchTouchingThreads(self, testlist):
# merge any threads between tests that touch
- for phase in self.phases:
+ for phase in self.sortedPhases():
list = self.dmesg[phase]['list']
for devname in list:
dev = list[devname]
data.usurpTouchingThread(devname, dev)
def optimizeDevSrc(self):
# merge any src call loops to reduce timeline size
- for phase in self.phases:
+ for phase in self.sortedPhases():
list = self.dmesg[phase]['list']
for dev in list:
if 'src' not in list[dev]:
self.tKernSus = self.trimTimeVal(self.tKernSus, t0, dT, left)
self.tKernRes = self.trimTimeVal(self.tKernRes, t0, dT, left)
self.end = self.trimTimeVal(self.end, t0, dT, left)
- for phase in self.phases:
+ for phase in self.sortedPhases():
p = self.dmesg[phase]
p['start'] = self.trimTimeVal(p['start'], t0, dT, left)
p['end'] = self.trimTimeVal(p['end'], t0, dT, left)
d = list[name]
d['start'] = self.trimTimeVal(d['start'], t0, dT, left)
d['end'] = self.trimTimeVal(d['end'], t0, dT, left)
+ d['length'] = d['end'] - d['start']
if('ftrace' in d):
cg = d['ftrace']
cg.start = self.trimTimeVal(cg.start, t0, dT, left)
tm = self.trimTimeVal(tm, t0, dT, left)
list.append((type, tm, idx1, idx2))
self.errorinfo[dir] = list
- def normalizeTime(self, tZero):
+ def trimFreezeTime(self, tZero):
# trim out any standby or freeze clock time
- if(self.tSuspended != self.tResumed):
- if(self.tResumed > tZero):
- self.trimTime(self.tSuspended, \
- self.tResumed-self.tSuspended, True)
- else:
- self.trimTime(self.tSuspended, \
- self.tResumed-self.tSuspended, False)
+ lp = ''
+ for phase in self.sortedPhases():
+ if 'resume_machine' in phase and 'suspend_machine' in lp:
+ tS, tR = self.dmesg[lp]['end'], self.dmesg[phase]['start']
+ tL = tR - tS
+ if tL > 0:
+ left = True if tR > tZero else False
+ self.trimTime(tS, tL, left)
+ self.tLow.append('%.0f'%(tL*1000))
+ lp = phase
def getTimeValues(self):
- sktime = (self.dmesg['suspend_machine']['end'] - \
- self.tKernSus) * 1000
- rktime = (self.dmesg['resume_complete']['end'] - \
- self.dmesg['resume_machine']['start']) * 1000
+ sktime = (self.tSuspended - self.tKernSus) * 1000
+ rktime = (self.tKernRes - self.tResumed) * 1000
return (sktime, rktime)
- def setPhase(self, phase, ktime, isbegin):
+ def setPhase(self, phase, ktime, isbegin, order=-1):
if(isbegin):
+ # phase start over current phase
+ if self.currphase:
+ if 'resume_machine' not in self.currphase:
+ sysvals.vprint('WARNING: phase %s failed to end' % self.currphase)
+ self.dmesg[self.currphase]['end'] = ktime
+ phases = self.dmesg.keys()
+ color = self.phasedef[phase]['color']
+ count = len(phases) if order < 0 else order
+ # create unique name for every new phase
+ while phase in phases:
+ phase += '*'
+ self.dmesg[phase] = {'list': dict(), 'start': -1.0, 'end': -1.0,
+ 'row': 0, 'color': color, 'order': count}
self.dmesg[phase]['start'] = ktime
+ self.currphase = phase
else:
+ # phase end without a start
+ if phase not in self.currphase:
+ if self.currphase:
+ sysvals.vprint('WARNING: %s ended instead of %s, ftrace corruption?' % (phase, self.currphase))
+ else:
+ sysvals.vprint('WARNING: %s ended without a start, ftrace corruption?' % phase)
+ return phase
+ phase = self.currphase
self.dmesg[phase]['end'] = ktime
- def dmesgSortVal(self, phase):
- return self.dmesg[phase]['order']
- def sortedPhases(self):
- return sorted(self.dmesg, key=self.dmesgSortVal)
+ self.currphase = ''
+ return phase
def sortedDevices(self, phase):
list = self.dmesg[phase]['list']
slist = []
for devname in phaselist:
dev = phaselist[devname]
if(dev['end'] < 0):
- for p in self.phases:
+ for p in self.sortedPhases():
if self.dmesg[p]['end'] > dev['start']:
dev['end'] = self.dmesg[p]['end']
break
sysvals.vprint('%s (%s): callback didnt return' % (devname, phase))
def deviceFilter(self, devicefilter):
- for phase in self.phases:
+ for phase in self.sortedPhases():
list = self.dmesg[phase]['list']
rmlist = []
for name in list:
del list[name]
def fixupInitcallsThatDidntReturn(self):
# if any calls never returned, clip them at system resume end
- for phase in self.phases:
+ for phase in self.sortedPhases():
self.fixupInitcalls(phase)
def phaseOverlap(self, phases):
rmgroups = []
self.devicegroups.append(newgroup)
def newActionGlobal(self, name, start, end, pid=-1, color=''):
# which phase is this device callback or action in
+ phases = self.sortedPhases()
targetphase = 'none'
htmlclass = ''
overlap = 0.0
- phases = []
- for phase in self.phases:
+ myphases = []
+ for phase in phases:
pstart = self.dmesg[phase]['start']
pend = self.dmesg[phase]['end']
# see if the action overlaps this phase
o = max(0, min(end, pend) - max(start, pstart))
if o > 0:
- phases.append(phase)
+ myphases.append(phase)
# set the target phase to the one that overlaps most
if o > overlap:
if overlap > 0 and phase == 'post_resume':
overlap = o
# if no target phase was found, pin it to the edge
if targetphase == 'none':
- p0start = self.dmesg[self.phases[0]]['start']
+ p0start = self.dmesg[phases[0]]['start']
if start <= p0start:
- targetphase = self.phases[0]
+ targetphase = phases[0]
else:
- targetphase = self.phases[-1]
+ targetphase = phases[-1]
if pid == -2:
htmlclass = ' bg'
elif pid == -3:
htmlclass = ' ps'
- if len(phases) > 1:
+ if len(myphases) > 1:
htmlclass = ' bg'
- self.phaseOverlap(phases)
- if targetphase in self.phases:
+ self.phaseOverlap(myphases)
+ if targetphase in phases:
newname = self.newAction(targetphase, name, pid, '', start, end, '', htmlclass, color)
return (targetphase, newname)
return False
if(list[child]['par'] == devname):
devlist.append(child)
return devlist
+ def maxDeviceNameSize(self, phase):
+ size = 0
+ for name in self.dmesg[phase]['list']:
+ if len(name) > size:
+ size = len(name)
+ return size
def printDetails(self):
sysvals.vprint('Timeline Details:')
sysvals.vprint(' test start: %f' % self.start)
sysvals.vprint('kernel suspend start: %f' % self.tKernSus)
- for phase in self.phases:
- dc = len(self.dmesg[phase]['list'])
- sysvals.vprint(' %16s: %f - %f (%d devices)' % (phase, \
- self.dmesg[phase]['start'], self.dmesg[phase]['end'], dc))
+ tS = tR = False
+ for phase in self.sortedPhases():
+ devlist = self.dmesg[phase]['list']
+ dc, ps, pe = len(devlist), self.dmesg[phase]['start'], self.dmesg[phase]['end']
+ if not tS and ps >= self.tSuspended:
+ sysvals.vprint(' machine suspended: %f' % self.tSuspended)
+ tS = True
+ if not tR and ps >= self.tResumed:
+ sysvals.vprint(' machine resumed: %f' % self.tResumed)
+ tR = True
+ sysvals.vprint('%20s: %f - %f (%d devices)' % (phase, ps, pe, dc))
+ if sysvals.devdump:
+ sysvals.vprint(''.join('-' for i in range(80)))
+ maxname = '%d' % self.maxDeviceNameSize(phase)
+ fmt = '%3d) %'+maxname+'s - %f - %f'
+ c = 1
+ for name in devlist:
+ s = devlist[name]['start']
+ e = devlist[name]['end']
+ sysvals.vprint(fmt % (c, name, s, e))
+ c += 1
+ sysvals.vprint(''.join('-' for i in range(80)))
sysvals.vprint(' kernel resume end: %f' % self.tKernRes)
sysvals.vprint(' test end: %f' % self.end)
def deviceChildrenAllPhases(self, devname):
devlist = []
- for phase in self.phases:
+ for phase in self.sortedPhases():
list = self.deviceChildren(devname, phase)
for dev in list:
if dev not in devlist:
if node.name:
info = ''
drv = ''
- for phase in self.phases:
+ for phase in self.sortedPhases():
list = self.dmesg[phase]['list']
if node.name in list:
s = list[node.name]['start']
c = self.addProcessUsageEvent(ps, tres)
if c > 0:
sysvals.vprint('%25s (res): %d' % (ps, c))
+ def handleEndMarker(self, time):
+ dm = self.dmesg
+ self.setEnd(time)
+ self.initDevicegroups()
+ # give suspend_prepare an end if needed
+ if 'suspend_prepare' in dm and dm['suspend_prepare']['end'] < 0:
+ dm['suspend_prepare']['end'] = time
+ # assume resume machine ends at next phase start
+ if 'resume_machine' in dm and dm['resume_machine']['end'] < 0:
+ np = self.nextPhase('resume_machine', 1)
+ if np:
+ dm['resume_machine']['end'] = dm[np]['start']
+ # if kernel resume end not found, assume its the end marker
+ if self.tKernRes == 0.0:
+ self.tKernRes = time
+ # if kernel suspend start not found, assume its the end marker
+ if self.tKernSus == 0.0:
+ self.tKernSus = time
+ # set resume complete to end at end marker
+ if 'resume_complete' in dm:
+ dm['resume_complete']['end'] = time
def debugPrint(self):
- for p in self.phases:
+ for p in self.sortedPhases():
list = self.dmesg[p]['list']
for devname in list:
dev = list[devname]
# Description:
# A container for kprobe function data we want in the dev timeline
class DevFunction:
- row = 0
- count = 1
def __init__(self, name, args, caller, ret, start, end, u, proc, pid, color):
+ self.row = 0
+ self.count = 1
self.name = name
self.args = args
self.caller = caller
# suspend_resume: phase or custom exec block data
# device_pm_callback: device callback info
class FTraceLine:
- time = 0.0
- length = 0.0
- fcall = False
- freturn = False
- fevent = False
- fkprobe = False
- depth = 0
- name = ''
- type = ''
def __init__(self, t, m='', d=''):
+ self.length = 0.0
+ self.fcall = False
+ self.freturn = False
+ self.fevent = False
+ self.fkprobe = False
+ self.depth = 0
+ self.name = ''
+ self.type = ''
self.time = float(t)
if not m and not d:
return
return len(str)/2
def debugPrint(self, info=''):
if self.isLeaf():
- print(' -- %12.6f (depth=%02d): %s(); (%.3f us) %s' % (self.time, \
+ pprint(' -- %12.6f (depth=%02d): %s(); (%.3f us) %s' % (self.time, \
self.depth, self.name, self.length*1000000, info))
elif self.freturn:
- print(' -- %12.6f (depth=%02d): %s} (%.3f us) %s' % (self.time, \
+ pprint(' -- %12.6f (depth=%02d): %s} (%.3f us) %s' % (self.time, \
self.depth, self.name, self.length*1000000, info))
else:
- print(' -- %12.6f (depth=%02d): %s() { (%.3f us) %s' % (self.time, \
+ pprint(' -- %12.6f (depth=%02d): %s() { (%.3f us) %s' % (self.time, \
self.depth, self.name, self.length*1000000, info))
def startMarker(self):
# Is this the starting line of a suspend?
# Each instance is tied to a single device in a single phase, and is
# comprised of an ordered list of FTraceLine objects
class FTraceCallGraph:
- id = ''
- start = -1.0
- end = -1.0
- list = []
- invalid = False
- depth = 0
- pid = 0
- name = ''
- partial = False
vfname = 'missing_function_name'
- ignore = False
- sv = 0
def __init__(self, pid, sv):
+ self.id = ''
+ self.invalid = False
+ self.name = ''
+ self.partial = False
+ self.ignore = False
self.start = -1.0
self.end = -1.0
self.list = []
if warning and ('[make leaf]', line) not in info:
info.append(('', line))
if warning:
- print 'WARNING: ftrace data missing, corrections made:'
+ pprint('WARNING: ftrace data missing, corrections made:')
for i in info:
t, obj = i
if obj:
id = 'task %s' % (self.pid)
window = '(%f - %f)' % (self.start, line.time)
if(self.depth < 0):
- print('Data misalignment for '+id+\
+ pprint('Data misalignment for '+id+\
' (buffer overflow), ignoring this callback')
else:
- print('Too much data for '+id+\
+ pprint('Too much data for '+id+\
' '+window+', ignoring this callback')
def slice(self, dev):
minicg = FTraceCallGraph(dev['pid'], self.sv)
elif l.isReturn():
if(l.depth not in stack):
if self.sv.verbose:
- print 'Post Process Error: Depth missing'
+ pprint('Post Process Error: Depth missing')
l.debugPrint()
return False
# calculate call length from call/return lines
return True
elif(cnt < 0):
if self.sv.verbose:
- print 'Post Process Error: Depth is less than 0'
+ pprint('Post Process Error: Depth is less than 0')
return False
# trace ended before call tree finished
return self.repair(cnt)
dev['ftrace'] = cg
found = devname
return found
- for p in data.phases:
+ for p in data.sortedPhases():
if(data.dmesg[p]['start'] <= self.start and
self.start <= data.dmesg[p]['end']):
list = data.dmesg[p]['list']
if fs < data.start or fe > data.end:
return
phase = ''
- for p in data.phases:
+ for p in data.sortedPhases():
if(data.dmesg[p]['start'] <= self.start and
self.start < data.dmesg[p]['end']):
phase = p
phase, myname = out
data.dmesg[phase]['list'][myname]['ftrace'] = self
def debugPrint(self, info=''):
- print('%s pid=%d [%f - %f] %.3f us') % \
+ pprint('%s pid=%d [%f - %f] %.3f us' % \
(self.name, self.pid, self.start, self.end,
- (self.end - self.start)*1000000)
+ (self.end - self.start)*1000000))
for l in self.list:
if l.isLeaf():
- print('%f (%02d): %s(); (%.3f us)%s' % (l.time, \
+ pprint('%f (%02d): %s(); (%.3f us)%s' % (l.time, \
l.depth, l.name, l.length*1000000, info))
elif l.freturn:
- print('%f (%02d): %s} (%.3f us)%s' % (l.time, \
+ pprint('%f (%02d): %s} (%.3f us)%s' % (l.time, \
l.depth, l.name, l.length*1000000, info))
else:
- print('%f (%02d): %s() { (%.3f us)%s' % (l.time, \
+ pprint('%f (%02d): %s() { (%.3f us)%s' % (l.time, \
l.depth, l.name, l.length*1000000, info))
- print(' ')
+ pprint(' ')
class DevItem:
def __init__(self, test, phase, dev):
# A container for a device timeline which calculates
# all the html properties to display it correctly
class Timeline:
- html = ''
- height = 0 # total timeline height
- scaleH = 20 # timescale (top) row height
- rowH = 30 # device row height
- bodyH = 0 # body height
- rows = 0 # total timeline rows
- rowlines = dict()
- rowheight = dict()
html_tblock = '<div id="block{0}" class="tblock" style="left:{1}%;width:{2}%;"><div class="tback" style="height:{3}px"></div>\n'
html_device = '<div id="{0}" title="{1}" class="thread{7}" style="left:{2}%;top:{3}px;height:{4}px;width:{5}%;{8}">{6}</div>\n'
html_phase = '<div class="phase" style="left:{0}%;width:{1}%;top:{2}px;height:{3}px;background:{4}">{5}</div>\n'
html_phaselet = '<div id="{0}" class="phaselet" style="left:{1}%;width:{2}%;background:{3}"></div>\n'
html_legend = '<div id="p{3}" class="square" style="left:{0}%;background:{1}"> {2}</div>\n'
def __init__(self, rowheight, scaleheight):
- self.rowH = rowheight
- self.scaleH = scaleheight
self.html = ''
+ self.height = 0 # total timeline height
+ self.scaleH = scaleheight # timescale (top) row height
+ self.rowH = rowheight # device row height
+ self.bodyH = 0 # body height
+ self.rows = 0 # total timeline rows
+ self.rowlines = dict()
+ self.rowheight = dict()
def createHeader(self, sv, stamp):
if(not stamp['time']):
return
# Description:
# A list of values describing the properties of these test runs
class TestProps:
- stamp = ''
- sysinfo = ''
- cmdline = ''
- kparams = ''
- S0i3 = False
- fwdata = []
stampfmt = '# [a-z]*-(?P<m>[0-9]{2})(?P<d>[0-9]{2})(?P<y>[0-9]{2})-'+\
'(?P<H>[0-9]{2})(?P<M>[0-9]{2})(?P<S>[0-9]{2})'+\
' (?P<host>.*) (?P<mode>.*) (?P<kernel>.*)$'
+ batteryfmt = '^# battery (?P<a1>\w*) (?P<c1>\d*) (?P<a2>\w*) (?P<c2>\d*)'
+ testerrfmt = '^# enter_sleep_error (?P<e>.*)'
sysinfofmt = '^# sysinfo .*'
cmdlinefmt = '^# command \| (?P<cmd>.*)'
kparamsfmt = '^# kparams \| (?P<kp>.*)'
+ devpropfmt = '# Device Properties: .*'
+ tracertypefmt = '# tracer: (?P<t>.*)'
+ firmwarefmt = '# fwsuspend (?P<s>[0-9]*) fwresume (?P<r>[0-9]*)$'
+ procexecfmt = 'ps - (?P<ps>.*)$'
ftrace_line_fmt_fg = \
'^ *(?P<time>[0-9\.]*) *\| *(?P<cpu>[0-9]*)\)'+\
' *(?P<proc>.*)-(?P<pid>[0-9]*) *\|'+\
' *(?P<proc>.*)-(?P<pid>[0-9]*) *\[(?P<cpu>[0-9]*)\] *'+\
'(?P<flags>.{4}) *(?P<time>[0-9\.]*): *'+\
'(?P<msg>.*)'
- ftrace_line_fmt = ftrace_line_fmt_nop
- cgformat = False
- data = 0
- ktemp = dict()
def __init__(self):
+ self.stamp = ''
+ self.sysinfo = ''
+ self.cmdline = ''
+ self.kparams = ''
+ self.testerror = []
+ self.battery = []
+ self.fwdata = []
+ self.ftrace_line_fmt = self.ftrace_line_fmt_nop
+ self.cgformat = False
+ self.data = 0
self.ktemp = dict()
def setTracerType(self, tracer):
if(tracer == 'function_graph'):
else:
doError('Invalid tracer format: [%s]' % tracer)
def parseStamp(self, data, sv):
+ # global test data
m = re.match(self.stampfmt, self.stamp)
data.stamp = {'time': '', 'host': '', 'mode': ''}
dt = datetime(int(m.group('y'))+2000, int(m.group('m')),
sv.kparams = m.group('kp')
if not sv.stamp:
sv.stamp = data.stamp
+ # firmware data
+ if sv.suspendmode == 'mem' and len(self.fwdata) > data.testnumber:
+ data.fwSuspend, data.fwResume = self.fwdata[data.testnumber]
+ if(data.fwSuspend > 0 or data.fwResume > 0):
+ data.fwValid = True
+ # battery data
+ if len(self.battery) > data.testnumber:
+ m = re.match(self.batteryfmt, self.battery[data.testnumber])
+ if m:
+ data.battery = m.groups()
+ # sleep mode enter errors
+ if len(self.testerror) > data.testnumber:
+ m = re.match(self.testerrfmt, self.testerror[data.testnumber])
+ if m:
+ data.enterfail = m.group('e')
# Class: TestRun
# Description:
# A container for a suspend/resume test run. This is necessary as
# there could be more than one, and they need to be separate.
class TestRun:
- ftemp = dict()
- ttemp = dict()
- data = 0
def __init__(self, dataobj):
self.data = dataobj
self.ftemp = dict()
self.ttemp = dict()
class ProcessMonitor:
- proclist = dict()
- running = False
+ def __init__(self):
+ self.proclist = dict()
+ self.running = False
def procstat(self):
c = ['cat /proc/[1-9]*/stat 2>/dev/null']
process = Popen(c, shell=True, stdout=PIPE)
# markers, and/or kprobes required for primary parsing.
def doesTraceLogHaveTraceEvents():
kpcheck = ['_cal: (', '_cpu_down()']
- techeck = ['suspend_resume']
- tmcheck = ['SUSPEND START', 'RESUME COMPLETE']
+ techeck = ['suspend_resume', 'device_pm_callback']
+ tmcheck = ['tracing_mark_write']
sysvals.usekprobes = False
fp = sysvals.openlog(sysvals.ftracefile, 'r')
for line in fp:
check.remove(i)
tmcheck = check
fp.close()
- if len(techeck) == 0:
- sysvals.usetraceevents = True
- else:
- sysvals.usetraceevents = False
- if len(tmcheck) == 0:
- sysvals.usetracemarkers = True
- else:
- sysvals.usetracemarkers = False
+ sysvals.usetraceevents = True if len(techeck) < 2 else False
+ sysvals.usetracemarkers = True if len(tmcheck) == 0 else False
# Function: appendIncompleteTraceLog
# Description:
# [deprecated for kernel 3.15 or newer]
-# Legacy support of ftrace outputs that lack the device_pm_callback
-# and/or suspend_resume trace events. The primary data should be
-# taken from dmesg, and this ftrace is used only for callgraph data
-# or custom actions in the timeline. The data is appended to the Data
-# objects provided.
+# Adds callgraph data which lacks trace event data. This is only
+# for timelines generated from 3.15 or older
# Arguments:
# testruns: the array of Data objects obtained from parseKernelLog
def appendIncompleteTraceLog(testruns):
elif re.match(tp.cmdlinefmt, line):
tp.cmdline = line
continue
+ elif re.match(tp.batteryfmt, line):
+ tp.battery.append(line)
+ continue
+ elif re.match(tp.testerrfmt, line):
+ tp.testerror.append(line)
+ continue
# determine the trace data type (required for further parsing)
- m = re.match(sysvals.tracertypefmt, line)
+ m = re.match(tp.tracertypefmt, line)
if(m):
tp.setTracerType(m.group('t'))
continue
# device properties line
- if(re.match(sysvals.devpropfmt, line)):
+ if(re.match(tp.devpropfmt, line)):
devProps(line)
continue
# parse only valid lines, if this is not one move on
continue
# trace event processing
if(t.fevent):
- # general trace events have two types, begin and end
- if(re.match('(?P<name>.*) begin$', t.name)):
- isbegin = True
- elif(re.match('(?P<name>.*) end$', t.name)):
- isbegin = False
- else:
- continue
- m = re.match('(?P<name>.*)\[(?P<val>[0-9]*)\] .*', t.name)
- if(m):
- val = m.group('val')
- if val == '0':
- name = m.group('name')
- else:
- name = m.group('name')+'['+val+']'
- else:
- m = re.match('(?P<name>.*) .*', t.name)
- name = m.group('name')
- # special processing for trace events
- if re.match('dpm_prepare\[.*', name):
- continue
- elif re.match('machine_suspend.*', name):
- continue
- elif re.match('suspend_enter\[.*', name):
- if(not isbegin):
- data.dmesg['suspend_prepare']['end'] = t.time
- continue
- elif re.match('dpm_suspend\[.*', name):
- if(not isbegin):
- data.dmesg['suspend']['end'] = t.time
- continue
- elif re.match('dpm_suspend_late\[.*', name):
- if(isbegin):
- data.dmesg['suspend_late']['start'] = t.time
- else:
- data.dmesg['suspend_late']['end'] = t.time
- continue
- elif re.match('dpm_suspend_noirq\[.*', name):
- if(isbegin):
- data.dmesg['suspend_noirq']['start'] = t.time
- else:
- data.dmesg['suspend_noirq']['end'] = t.time
- continue
- elif re.match('dpm_resume_noirq\[.*', name):
- if(isbegin):
- data.dmesg['resume_machine']['end'] = t.time
- data.dmesg['resume_noirq']['start'] = t.time
- else:
- data.dmesg['resume_noirq']['end'] = t.time
- continue
- elif re.match('dpm_resume_early\[.*', name):
- if(isbegin):
- data.dmesg['resume_early']['start'] = t.time
- else:
- data.dmesg['resume_early']['end'] = t.time
- continue
- elif re.match('dpm_resume\[.*', name):
- if(isbegin):
- data.dmesg['resume']['start'] = t.time
- else:
- data.dmesg['resume']['end'] = t.time
- continue
- elif re.match('dpm_complete\[.*', name):
- if(isbegin):
- data.dmesg['resume_complete']['start'] = t.time
- else:
- data.dmesg['resume_complete']['end'] = t.time
- continue
- # skip trace events inside devices calls
- if(not data.isTraceEventOutsideDeviceCalls(pid, t.time)):
- continue
- # global events (outside device calls) are simply graphed
- if(isbegin):
- # store each trace event in ttemp
- if(name not in testrun[testidx].ttemp):
- testrun[testidx].ttemp[name] = []
- testrun[testidx].ttemp[name].append(\
- {'begin': t.time, 'end': t.time})
- else:
- # finish off matching trace event in ttemp
- if(name in testrun[testidx].ttemp):
- testrun[testidx].ttemp[name][-1]['end'] = t.time
+ continue
# call/return processing
elif sysvals.usecallgraph:
# create a callgraph object for the data
tf.close()
for test in testrun:
- # add the traceevent data to the device hierarchy
- if(sysvals.usetraceevents):
- for name in test.ttemp:
- for event in test.ttemp[name]:
- test.data.newActionGlobal(name, event['begin'], event['end'])
-
# add the callgraph data to the device hierarchy
for pid in test.ftemp:
for cg in test.ftemp[pid]:
continue
callstart = cg.start
callend = cg.end
- for p in test.data.phases:
+ for p in test.data.sortedPhases():
if(test.data.dmesg[p]['start'] <= callstart and
callstart <= test.data.dmesg[p]['end']):
list = test.data.dmesg[p]['list']
doError('%s does not exist' % sysvals.ftracefile)
if not live:
sysvals.setupAllKprobes()
+ ksuscalls = ['pm_prepare_console']
+ krescalls = ['pm_restore_console']
tracewatch = []
if sysvals.usekprobes:
tracewatch += ['sync_filesystems', 'freeze_processes', 'syscore_suspend',
- 'syscore_resume', 'resume_console', 'thaw_processes', 'CPU_ON', 'CPU_OFF']
+ 'syscore_resume', 'resume_console', 'thaw_processes', 'CPU_ON',
+ 'CPU_OFF', 'timekeeping_freeze', 'acpi_suspend']
# extract the callgraph and traceevent data
tp = TestProps()
elif re.match(tp.cmdlinefmt, line):
tp.cmdline = line
continue
+ elif re.match(tp.batteryfmt, line):
+ tp.battery.append(line)
+ continue
+ elif re.match(tp.testerrfmt, line):
+ tp.testerror.append(line)
+ continue
# firmware line: pull out any firmware data
- m = re.match(sysvals.firmwarefmt, line)
+ m = re.match(tp.firmwarefmt, line)
if(m):
tp.fwdata.append((int(m.group('s')), int(m.group('r'))))
continue
# tracer type line: determine the trace data type
- m = re.match(sysvals.tracertypefmt, line)
+ m = re.match(tp.tracertypefmt, line)
if(m):
tp.setTracerType(m.group('t'))
continue
# device properties line
- if(re.match(sysvals.devpropfmt, line)):
+ if(re.match(tp.devpropfmt, line)):
devProps(line)
continue
# ignore all other commented lines
continue
# find the start of suspend
if(t.startMarker()):
- phase = 'suspend_prepare'
data = Data(len(testdata))
testdata.append(data)
testrun = TestRun(data)
testruns.append(testrun)
tp.parseStamp(data, sysvals)
data.setStart(t.time)
- data.tKernSus = t.time
+ data.first_suspend_prepare = True
+ phase = data.setPhase('suspend_prepare', t.time, True)
continue
if(not data):
continue
# process cpu exec line
if t.type == 'tracing_mark_write':
- m = re.match(sysvals.procexecfmt, t.name)
+ m = re.match(tp.procexecfmt, t.name)
if(m):
proclist = dict()
for ps in m.group('ps').split(','):
continue
# find the end of resume
if(t.endMarker()):
- data.setEnd(t.time)
- if data.tKernRes == 0.0:
- data.tKernRes = t.time
- if data.dmesg['resume_complete']['end'] < 0:
- data.dmesg['resume_complete']['end'] = t.time
- if sysvals.suspendmode == 'mem' and len(tp.fwdata) > data.testnumber:
- data.fwSuspend, data.fwResume = tp.fwdata[data.testnumber]
- if(data.tSuspended != 0 and data.tResumed != 0 and \
- (data.fwSuspend > 0 or data.fwResume > 0)):
- data.fwValid = True
+ data.handleEndMarker(t.time)
if(not sysvals.usetracemarkers):
# no trace markers? then quit and be sure to finish recording
# the event we used to trigger resume end
- if(len(testrun.ttemp['thaw_processes']) > 0):
+ if('thaw_processes' in testrun.ttemp and len(testrun.ttemp['thaw_processes']) > 0):
# if an entry exists, assume this is its end
testrun.ttemp['thaw_processes'][-1]['end'] = t.time
break
continue
# trace event processing
if(t.fevent):
- if(phase == 'post_resume'):
- data.setEnd(t.time)
if(t.type == 'suspend_resume'):
# suspend_resume trace events have two types, begin and end
if(re.match('(?P<name>.*) begin$', t.name)):
# -- phase changes --
# start of kernel suspend
if(re.match('suspend_enter\[.*', t.name)):
- if(isbegin and data.start == data.tKernSus):
- data.dmesg[phase]['start'] = t.time
+ if(isbegin):
data.tKernSus = t.time
continue
# suspend_prepare start
elif(re.match('dpm_prepare\[.*', t.name)):
- phase = 'suspend_prepare'
- if(not isbegin):
- data.dmesg[phase]['end'] = t.time
- if data.dmesg[phase]['start'] < 0:
- data.dmesg[phase]['start'] = data.start
+ if isbegin and data.first_suspend_prepare:
+ data.first_suspend_prepare = False
+ if data.tKernSus == 0:
+ data.tKernSus = t.time
+ continue
+ phase = data.setPhase('suspend_prepare', t.time, isbegin)
continue
# suspend start
elif(re.match('dpm_suspend\[.*', t.name)):
- phase = 'suspend'
- data.setPhase(phase, t.time, isbegin)
+ phase = data.setPhase('suspend', t.time, isbegin)
continue
# suspend_late start
elif(re.match('dpm_suspend_late\[.*', t.name)):
- phase = 'suspend_late'
- data.setPhase(phase, t.time, isbegin)
+ phase = data.setPhase('suspend_late', t.time, isbegin)
continue
# suspend_noirq start
elif(re.match('dpm_suspend_noirq\[.*', t.name)):
- if data.phaseCollision('suspend_noirq', isbegin, line):
- continue
- phase = 'suspend_noirq'
- data.setPhase(phase, t.time, isbegin)
- if(not isbegin):
- phase = 'suspend_machine'
- data.dmesg[phase]['start'] = t.time
+ phase = data.setPhase('suspend_noirq', t.time, isbegin)
continue
# suspend_machine/resume_machine
elif(re.match('machine_suspend\[.*', t.name)):
if(isbegin):
- phase = 'suspend_machine'
- data.dmesg[phase]['end'] = t.time
- data.tSuspended = t.time
+ lp = data.lastPhase()
+ phase = data.setPhase('suspend_machine', data.dmesg[lp]['end'], True)
+ data.setPhase(phase, t.time, False)
+ if data.tSuspended == 0:
+ data.tSuspended = t.time
else:
- if(sysvals.suspendmode in ['mem', 'disk'] and not tp.S0i3):
- data.dmesg['suspend_machine']['end'] = t.time
+ phase = data.setPhase('resume_machine', t.time, True)
+ if(sysvals.suspendmode in ['mem', 'disk']):
+ susp = phase.replace('resume', 'suspend')
+ if susp in data.dmesg:
+ data.dmesg[susp]['end'] = t.time
data.tSuspended = t.time
- phase = 'resume_machine'
- data.dmesg[phase]['start'] = t.time
data.tResumed = t.time
- data.tLow = data.tResumed - data.tSuspended
- continue
- # acpi_suspend
- elif(re.match('acpi_suspend\[.*', t.name)):
- # acpi_suspend[0] S0i3
- if(re.match('acpi_suspend\[0\] begin', t.name)):
- if(sysvals.suspendmode == 'mem'):
- tp.S0i3 = True
- data.dmesg['suspend_machine']['end'] = t.time
- data.tSuspended = t.time
continue
# resume_noirq start
elif(re.match('dpm_resume_noirq\[.*', t.name)):
- if data.phaseCollision('resume_noirq', isbegin, line):
- continue
- phase = 'resume_noirq'
- data.setPhase(phase, t.time, isbegin)
- if(isbegin):
- data.dmesg['resume_machine']['end'] = t.time
+ phase = data.setPhase('resume_noirq', t.time, isbegin)
continue
# resume_early start
elif(re.match('dpm_resume_early\[.*', t.name)):
- phase = 'resume_early'
- data.setPhase(phase, t.time, isbegin)
+ phase = data.setPhase('resume_early', t.time, isbegin)
continue
# resume start
elif(re.match('dpm_resume\[.*', t.name)):
- phase = 'resume'
- data.setPhase(phase, t.time, isbegin)
+ phase = data.setPhase('resume', t.time, isbegin)
continue
# resume complete start
elif(re.match('dpm_complete\[.*', t.name)):
- phase = 'resume_complete'
- if(isbegin):
- data.dmesg[phase]['start'] = t.time
+ phase = data.setPhase('resume_complete', t.time, isbegin)
continue
# skip trace events inside devices calls
if(not data.isTraceEventOutsideDeviceCalls(pid, t.time)):
if(len(testrun.ttemp[name]) > 0):
# if an entry exists, assume this is its end
testrun.ttemp[name][-1]['end'] = t.time
- elif(phase == 'post_resume'):
- # post resume events can just have ends
- testrun.ttemp[name].append({
- 'begin': data.dmesg[phase]['start'],
- 'end': t.time})
# device callback start
elif(t.type == 'device_pm_callback_start'):
+ if phase not in data.dmesg:
+ continue
m = re.match('(?P<drv>.*) (?P<d>.*), parent: *(?P<p>.*), .*',\
t.name);
if(not m):
data.devpids.append(pid)
# device callback finish
elif(t.type == 'device_pm_callback_end'):
+ if phase not in data.dmesg:
+ continue
m = re.match('(?P<drv>.*) (?P<d>.*), err.*', t.name);
if(not m):
continue
'cdata': kprobedata,
'proc': m_proc,
})
+ # start of kernel resume
+ if(phase == 'suspend_prepare' and kprobename in ksuscalls):
+ data.tKernSus = t.time
elif(t.freturn):
if(key not in tp.ktemp) or len(tp.ktemp[key]) < 1:
continue
e['end'] = t.time
e['rdata'] = kprobedata
# end of kernel resume
- if(kprobename == 'pm_notifier_call_chain' or \
- kprobename == 'pm_restore_console'):
- data.dmesg[phase]['end'] = t.time
+ if(phase != 'suspend_prepare' and kprobename in krescalls):
+ if phase in data.dmesg:
+ data.dmesg[phase]['end'] = t.time
data.tKernRes = t.time
# callgraph processing
if(res == -1):
testrun.ftemp[key][-1].addLine(t)
tf.close()
+ if data and not data.devicegroups:
+ sysvals.vprint('WARNING: end marker is missing')
+ data.handleEndMarker(t.time)
if sysvals.suspendmode == 'command':
for test in testruns:
- for p in test.data.phases:
+ for p in test.data.sortedPhases():
if p == 'suspend_prepare':
test.data.dmesg[p]['start'] = test.data.start
test.data.dmesg[p]['end'] = test.data.end
test.data.dmesg[p]['end'] = test.data.end
test.data.tSuspended = test.data.end
test.data.tResumed = test.data.end
- test.data.tLow = 0
test.data.fwValid = False
# dev source and procmon events can be unreadable with mixed phase height
if sysvals.usedevsrc or sysvals.useprocmon:
sysvals.mixedphaseheight = False
+ # expand phase boundaries so there are no gaps
+ for data in testdata:
+ lp = data.sortedPhases()[0]
+ for p in data.sortedPhases():
+ if(p != lp and not ('machine' in p and 'machine' in lp)):
+ data.dmesg[lp]['end'] = data.dmesg[p]['start']
+ lp = p
+
for i in range(len(testruns)):
test = testruns[i]
data = test.data
sortkey = '%f%f%d' % (cg.start, cg.end, pid)
sortlist[sortkey] = cg
elif len(cg.list) > 1000000:
- print 'WARNING: the callgraph for %s is massive (%d lines)' %\
- (devname, len(cg.list))
+ sysvals.vprint('WARNING: the callgraph for %s is massive (%d lines)' %\
+ (devname, len(cg.list)))
# create blocks for orphan cg data
for sortkey in sorted(sortlist):
cg = sortlist[sortkey]
for data in testdata:
tn = '' if len(testdata) == 1 else ('%d' % (data.testnumber + 1))
terr = ''
- lp = data.phases[0]
- for p in data.phases:
- if(data.dmesg[p]['start'] < 0 and data.dmesg[p]['end'] < 0):
+ phasedef = data.phasedef
+ lp = 'suspend_prepare'
+ for p in sorted(phasedef, key=lambda k:phasedef[k]['order']):
+ if p not in data.dmesg:
if not terr:
- print 'TEST%s FAILED: %s failed in %s phase' % (tn, sysvals.suspendmode, lp)
+ pprint('TEST%s FAILED: %s failed in %s phase' % (tn, sysvals.suspendmode, lp))
terr = '%s%s failed in %s phase' % (sysvals.suspendmode, tn, lp)
error.append(terr)
+ if data.tSuspended == 0:
+ data.tSuspended = data.dmesg[lp]['end']
+ if data.tResumed == 0:
+ data.tResumed = data.dmesg[lp]['end']
+ data.fwValid = False
sysvals.vprint('WARNING: phase "%s" is missing!' % p)
- if(data.dmesg[p]['start'] < 0):
- data.dmesg[p]['start'] = data.dmesg[lp]['end']
- if(p == 'resume_machine'):
- data.tSuspended = data.dmesg[lp]['end']
- data.tResumed = data.dmesg[lp]['end']
- data.tLow = 0
- if(data.dmesg[p]['end'] < 0):
- data.dmesg[p]['end'] = data.dmesg[p]['start']
- if(p != lp and not ('machine' in p and 'machine' in lp)):
- data.dmesg[lp]['end'] = data.dmesg[p]['start']
lp = p
+ if not terr and data.enterfail:
+ pprint('test%s FAILED: enter %s failed with %s' % (tn, sysvals.suspendmode, data.enterfail))
+ terr = 'test%s failed to enter %s mode' % (tn, sysvals.suspendmode)
+ error.append(terr)
+ if data.tSuspended == 0:
+ data.tSuspended = data.tKernRes
+ if data.tResumed == 0:
+ data.tResumed = data.tSuspended
if(len(sysvals.devicefilter) > 0):
data.deviceFilter(sysvals.devicefilter)
elif re.match(tp.cmdlinefmt, line):
tp.cmdline = line
continue
- m = re.match(sysvals.firmwarefmt, line)
+ elif re.match(tp.batteryfmt, line):
+ tp.battery.append(line)
+ continue
+ elif re.match(tp.testerrfmt, line):
+ tp.testerror.append(line)
+ continue
+ m = re.match(tp.firmwarefmt, line)
if(m):
tp.fwdata.append((int(m.group('s')), int(m.group('r'))))
continue
testruns.append(data)
data = Data(len(testruns))
tp.parseStamp(data, sysvals)
- if len(tp.fwdata) > data.testnumber:
- data.fwSuspend, data.fwResume = tp.fwdata[data.testnumber]
- if(data.fwSuspend > 0 or data.fwResume > 0):
- data.fwValid = True
if(not data):
continue
m = re.match('.* *(?P<k>[0-9]\.[0-9]{2}\.[0-9]-.*) .*', msg)
if data:
testruns.append(data)
if len(testruns) < 1:
- print('ERROR: dmesg log has no suspend/resume data: %s' \
+ pprint('ERROR: dmesg log has no suspend/resume data: %s' \
% sysvals.dmesgfile)
# fix lines with same timestamp/function with the call and return swapped
# dmesg phase match table
dm = {
- 'suspend_prepare': 'PM: Syncing filesystems.*',
- 'suspend': 'PM: Entering [a-z]* sleep.*',
- 'suspend_late': 'PM: suspend of devices complete after.*',
- 'suspend_noirq': 'PM: late suspend of devices complete after.*',
- 'suspend_machine': 'PM: noirq suspend of devices complete after.*',
- 'resume_machine': 'ACPI: Low-level resume complete.*',
- 'resume_noirq': 'ACPI: Waking up from system sleep state.*',
- 'resume_early': 'PM: noirq resume of devices complete after.*',
- 'resume': 'PM: early resume of devices complete after.*',
- 'resume_complete': 'PM: resume of devices complete after.*',
- 'post_resume': '.*Restarting tasks \.\.\..*',
+ 'suspend_prepare': ['PM: Syncing filesystems.*'],
+ 'suspend': ['PM: Entering [a-z]* sleep.*', 'Suspending console.*'],
+ 'suspend_late': ['PM: suspend of devices complete after.*'],
+ 'suspend_noirq': ['PM: late suspend of devices complete after.*'],
+ 'suspend_machine': ['PM: noirq suspend of devices complete after.*'],
+ 'resume_machine': ['ACPI: Low-level resume complete.*'],
+ 'resume_noirq': ['ACPI: Waking up from system sleep state.*'],
+ 'resume_early': ['PM: noirq resume of devices complete after.*'],
+ 'resume': ['PM: early resume of devices complete after.*'],
+ 'resume_complete': ['PM: resume of devices complete after.*'],
+ 'post_resume': ['.*Restarting tasks \.\.\..*'],
}
if(sysvals.suspendmode == 'standby'):
- dm['resume_machine'] = 'PM: Restoring platform NVS memory'
+ dm['resume_machine'] = ['PM: Restoring platform NVS memory']
elif(sysvals.suspendmode == 'disk'):
- dm['suspend_late'] = 'PM: freeze of devices complete after.*'
- dm['suspend_noirq'] = 'PM: late freeze of devices complete after.*'
- dm['suspend_machine'] = 'PM: noirq freeze of devices complete after.*'
- dm['resume_machine'] = 'PM: Restoring platform NVS memory'
- dm['resume_early'] = 'PM: noirq restore of devices complete after.*'
- dm['resume'] = 'PM: early restore of devices complete after.*'
- dm['resume_complete'] = 'PM: restore of devices complete after.*'
+ dm['suspend_late'] = ['PM: freeze of devices complete after.*']
+ dm['suspend_noirq'] = ['PM: late freeze of devices complete after.*']
+ dm['suspend_machine'] = ['PM: noirq freeze of devices complete after.*']
+ dm['resume_machine'] = ['PM: Restoring platform NVS memory']
+ dm['resume_early'] = ['PM: noirq restore of devices complete after.*']
+ dm['resume'] = ['PM: early restore of devices complete after.*']
+ dm['resume_complete'] = ['PM: restore of devices complete after.*']
elif(sysvals.suspendmode == 'freeze'):
- dm['resume_machine'] = 'ACPI: resume from mwait'
+ dm['resume_machine'] = ['ACPI: resume from mwait']
# action table (expected events that occur and show up in dmesg)
at = {
else:
continue
+ # check for a phase change line
+ phasechange = False
+ for p in dm:
+ for s in dm[p]:
+ if(re.match(s, msg)):
+ phasechange, phase = True, p
+ break
+
# hack for determining resume_machine end for freeze
if(not sysvals.usetraceevents and sysvals.suspendmode == 'freeze' \
and phase == 'resume_machine' and \
re.match('calling (?P<f>.*)\+ @ .*, parent: .*', msg)):
- data.dmesg['resume_machine']['end'] = ktime
- phase = 'resume_noirq'
- data.dmesg[phase]['start'] = ktime
-
- # suspend start
- if(re.match(dm['suspend_prepare'], msg)):
- phase = 'suspend_prepare'
- data.dmesg[phase]['start'] = ktime
- data.setStart(ktime)
- data.tKernSus = ktime
- # suspend start
- elif(re.match(dm['suspend'], msg)):
- data.dmesg['suspend_prepare']['end'] = ktime
- phase = 'suspend'
- data.dmesg[phase]['start'] = ktime
- # suspend_late start
- elif(re.match(dm['suspend_late'], msg)):
- data.dmesg['suspend']['end'] = ktime
- phase = 'suspend_late'
- data.dmesg[phase]['start'] = ktime
- # suspend_noirq start
- elif(re.match(dm['suspend_noirq'], msg)):
- data.dmesg['suspend_late']['end'] = ktime
- phase = 'suspend_noirq'
- data.dmesg[phase]['start'] = ktime
- # suspend_machine start
- elif(re.match(dm['suspend_machine'], msg)):
- data.dmesg['suspend_noirq']['end'] = ktime
- phase = 'suspend_machine'
- data.dmesg[phase]['start'] = ktime
- # resume_machine start
- elif(re.match(dm['resume_machine'], msg)):
- if(sysvals.suspendmode in ['freeze', 'standby']):
- data.tSuspended = prevktime
- data.dmesg['suspend_machine']['end'] = prevktime
- else:
- data.tSuspended = ktime
- data.dmesg['suspend_machine']['end'] = ktime
- phase = 'resume_machine'
- data.tResumed = ktime
- data.tLow = data.tResumed - data.tSuspended
- data.dmesg[phase]['start'] = ktime
- # resume_noirq start
- elif(re.match(dm['resume_noirq'], msg)):
- data.dmesg['resume_machine']['end'] = ktime
+ data.setPhase(phase, ktime, False)
phase = 'resume_noirq'
- data.dmesg[phase]['start'] = ktime
- # resume_early start
- elif(re.match(dm['resume_early'], msg)):
- data.dmesg['resume_noirq']['end'] = ktime
- phase = 'resume_early'
- data.dmesg[phase]['start'] = ktime
- # resume start
- elif(re.match(dm['resume'], msg)):
- data.dmesg['resume_early']['end'] = ktime
- phase = 'resume'
- data.dmesg[phase]['start'] = ktime
- # resume complete start
- elif(re.match(dm['resume_complete'], msg)):
- data.dmesg['resume']['end'] = ktime
- phase = 'resume_complete'
- data.dmesg[phase]['start'] = ktime
- # post resume start
- elif(re.match(dm['post_resume'], msg)):
- data.dmesg['resume_complete']['end'] = ktime
- data.setEnd(ktime)
- data.tKernRes = ktime
- break
+ data.setPhase(phase, ktime, True)
+
+ if phasechange:
+ if phase == 'suspend_prepare':
+ data.setPhase(phase, ktime, True)
+ data.setStart(ktime)
+ data.tKernSus = ktime
+ elif phase == 'suspend':
+ lp = data.lastPhase()
+ if lp:
+ data.setPhase(lp, ktime, False)
+ data.setPhase(phase, ktime, True)
+ elif phase == 'suspend_late':
+ lp = data.lastPhase()
+ if lp:
+ data.setPhase(lp, ktime, False)
+ data.setPhase(phase, ktime, True)
+ elif phase == 'suspend_noirq':
+ lp = data.lastPhase()
+ if lp:
+ data.setPhase(lp, ktime, False)
+ data.setPhase(phase, ktime, True)
+ elif phase == 'suspend_machine':
+ lp = data.lastPhase()
+ if lp:
+ data.setPhase(lp, ktime, False)
+ data.setPhase(phase, ktime, True)
+ elif phase == 'resume_machine':
+ lp = data.lastPhase()
+ if(sysvals.suspendmode in ['freeze', 'standby']):
+ data.tSuspended = prevktime
+ if lp:
+ data.setPhase(lp, prevktime, False)
+ else:
+ data.tSuspended = ktime
+ if lp:
+ data.setPhase(lp, prevktime, False)
+ data.tResumed = ktime
+ data.setPhase(phase, ktime, True)
+ elif phase == 'resume_noirq':
+ lp = data.lastPhase()
+ if lp:
+ data.setPhase(lp, ktime, False)
+ data.setPhase(phase, ktime, True)
+ elif phase == 'resume_early':
+ lp = data.lastPhase()
+ if lp:
+ data.setPhase(lp, ktime, False)
+ data.setPhase(phase, ktime, True)
+ elif phase == 'resume':
+ lp = data.lastPhase()
+ if lp:
+ data.setPhase(lp, ktime, False)
+ data.setPhase(phase, ktime, True)
+ elif phase == 'resume_complete':
+ lp = data.lastPhase()
+ if lp:
+ data.setPhase(lp, ktime, False)
+ data.setPhase(phase, ktime, True)
+ elif phase == 'post_resume':
+ lp = data.lastPhase()
+ if lp:
+ data.setPhase(lp, ktime, False)
+ data.setEnd(ktime)
+ data.tKernRes = ktime
+ break
# -- device callbacks --
- if(phase in data.phases):
+ if(phase in data.sortedPhases()):
# device init call
if(re.match('calling (?P<f>.*)\+ @ .*, parent: .*', msg)):
sm = re.match('calling (?P<f>.*)\+ @ '+\
actions[cpu].append({'begin': cpu_start, 'end': ktime})
cpu_start = ktime
prevktime = ktime
+ data.initDevicegroups()
# fill in any missing phases
- lp = data.phases[0]
- for p in data.phases:
- if(data.dmesg[p]['start'] < 0 and data.dmesg[p]['end'] < 0):
- print('WARNING: phase "%s" is missing, something went wrong!' % p)
- print(' In %s, this dmesg line denotes the start of %s:' % \
- (sysvals.suspendmode, p))
- print(' "%s"' % dm[p])
- if(data.dmesg[p]['start'] < 0):
- data.dmesg[p]['start'] = data.dmesg[lp]['end']
- if(p == 'resume_machine'):
- data.tSuspended = data.dmesg[lp]['end']
- data.tResumed = data.dmesg[lp]['end']
- data.tLow = 0
- if(data.dmesg[p]['end'] < 0):
- data.dmesg[p]['end'] = data.dmesg[p]['start']
+ phasedef = data.phasedef
+ terr, lp = '', 'suspend_prepare'
+ for p in sorted(phasedef, key=lambda k:phasedef[k]['order']):
+ if p not in data.dmesg:
+ if not terr:
+ pprint('TEST FAILED: %s failed in %s phase' % (sysvals.suspendmode, lp))
+ terr = '%s failed in %s phase' % (sysvals.suspendmode, lp)
+ if data.tSuspended == 0:
+ data.tSuspended = data.dmesg[lp]['end']
+ if data.tResumed == 0:
+ data.tResumed = data.dmesg[lp]['end']
+ sysvals.vprint('WARNING: phase "%s" is missing!' % p)
lp = p
+ lp = data.sortedPhases()[0]
+ for p in data.sortedPhases():
+ if(p != lp and not ('machine' in p and 'machine' in lp)):
+ data.dmesg[lp]['end'] = data.dmesg[p]['start']
+ lp = p
+ if data.tSuspended == 0:
+ data.tSuspended = data.tKernRes
+ if data.tResumed == 0:
+ data.tResumed = data.tSuspended
# fill in any actions we've found
for name in actions:
hf.write('<section id="callgraphs" class="callgraph">\n')
# write out the ftrace data converted to html
num = 0
- for p in data.phases:
+ for p in data.sortedPhases():
if sv.cgphase and p != sv.cgphase:
continue
list = data.dmesg[p]['list']
# Create summary html file for a series of tests
# Arguments:
# testruns: array of Data objects from parseTraceLog
-def createHTMLSummarySimple(testruns, htmlfile, folder):
+def createHTMLSummarySimple(testruns, htmlfile, title):
# write the html header first (html head, css code, up to body start)
html = '<!DOCTYPE html>\n<html>\n<head>\n\
<meta http-equiv="content-type" content="text/html; charset=UTF-8">\n\
table {width:100%;border-collapse: collapse;}\n\
.summary {border:1px solid;}\n\
th {border: 1px solid black;background:#222;color:white;}\n\
- td {font: 16px "Times New Roman";text-align: center;}\n\
+ td {font: 14px "Times New Roman";text-align: center;}\n\
tr.head td {border: 1px solid black;background:#aaa;}\n\
tr.alt {background-color:#ddd;}\n\
tr.notice {color:red;}\n\
iMin, iMed, iMax = [0, 0], [0, 0], [0, 0]
num = 0
lastmode = ''
- cnt = {'pass':0, 'fail':0, 'hang':0}
+ cnt = dict()
for data in sorted(testruns, key=lambda v:(v['mode'], v['host'], v['kernel'], v['time'])):
mode = data['mode']
if mode not in list:
tVal = [float(data['suspend']), float(data['resume'])]
list[mode]['data'].append([data['host'], data['kernel'],
data['time'], tVal[0], tVal[1], data['url'], data['result'],
- data['issues']])
+ data['issues'], data['sus_worst'], data['sus_worsttime'],
+ data['res_worst'], data['res_worsttime']])
idx = len(list[mode]['data']) - 1
+ if data['result'] not in cnt:
+ cnt[data['result']] = 1
+ else:
+ cnt[data['result']] += 1
if data['result'] == 'pass':
- cnt['pass'] += 1
for i in range(2):
tMed[i].append(tVal[i])
tAvg[i] += tVal[i]
iMax[i] = idx
tMax[i] = tVal[i]
num += 1
- elif data['result'] == 'hang':
- cnt['hang'] += 1
- elif data['result'] == 'fail':
- cnt['fail'] += 1
lastmode = mode
if lastmode and num > 0:
for i in range(2):
for ilk in sorted(cnt, reverse=True):
if cnt[ilk] > 0:
desc.append('%d %s' % (cnt[ilk], ilk))
- html += '<div class="stamp">%s (%d tests: %s)</div>\n' % (folder, len(testruns), ', '.join(desc))
+ html += '<div class="stamp">%s (%d tests: %s)</div>\n' % (title, len(testruns), ', '.join(desc))
th = '\t<th>{0}</th>\n'
td = '\t<td>{0}</td>\n'
tdh = '\t<td{1}>{0}</td>\n'
html += '<table class="summary">\n<tr>\n' + th.format('#') +\
th.format('Mode') + th.format('Host') + th.format('Kernel') +\
th.format('Test Time') + th.format('Result') + th.format('Issues') +\
- th.format('Suspend') + th.format('Resume') + th.format('Detail') + '</tr>\n'
+ th.format('Suspend') + th.format('Resume') +\
+ th.format('Worst Suspend Device') + th.format('SD Time') +\
+ th.format('Worst Resume Device') + th.format('RD Time') +\
+ th.format('Detail') + '</tr>\n'
# export list into html
head = '<tr class="head"><td>{0}</td><td>{1}</td>'+\
- '<td colspan=8 class="sus">Suspend Avg={2} '+\
+ '<td colspan=12 class="sus">Suspend Avg={2} '+\
'<span class=minval><a href="#s{10}min">Min={3}</a></span> '+\
'<span class=medval><a href="#s{10}med">Med={4}</a></span> '+\
'<span class=maxval><a href="#s{10}max">Max={5}</a></span> '+\
'<span class=medval><a href="#r{10}med">Med={8}</a></span> '+\
'<span class=maxval><a href="#r{10}max">Max={9}</a></span></td>'+\
'</tr>\n'
- headnone = '<tr class="head"><td>{0}</td><td>{1}</td><td colspan=8></td></tr>\n'
+ headnone = '<tr class="head"><td>{0}</td><td>{1}</td><td colspan=12></td></tr>\n'
for mode in list:
# header line for each suspend mode
num = 0
html += td.format(d[7]) # issues
html += tdh.format('%.3f ms' % d[3], tHigh[0]) if d[3] else td.format('') # suspend
html += tdh.format('%.3f ms' % d[4], tHigh[1]) if d[4] else td.format('') # resume
+ html += td.format(d[8]) # sus_worst
+ html += td.format('%.3f ms' % d[9]) if d[9] else td.format('') # sus_worst time
+ html += td.format(d[10]) # res_worst
+ html += td.format('%.3f ms' % d[11]) if d[11] else td.format('') # res_worst time
html += tdlink.format(d[5]) if d[5] else td.format('') # url
html += '</tr>\n'
num += 1
# True if the html file was created, false if it failed
def createHTML(testruns, testfail):
if len(testruns) < 1:
- print('ERROR: Not enough test data to build a timeline')
+ pprint('ERROR: Not enough test data to build a timeline')
return
kerror = False
for data in testruns:
if data.kerror:
kerror = True
- data.normalizeTime(testruns[-1].tSuspended)
+ if(sysvals.suspendmode in ['freeze', 'standby']):
+ data.trimFreezeTime(testruns[-1].tSuspended)
# html function templates
html_error = '<div id="{1}" title="kernel error/warning" class="err" style="right:{0}%">{2}→</div>\n'
sktime, rktime = data.getTimeValues()
if(tTotal == 0):
doError('No timeline data')
- if(data.tLow > 0):
- low_time = '%.0f'%(data.tLow*1000)
+ if(len(data.tLow) > 0):
+ low_time = '|'.join(data.tLow)
if sysvals.suspendmode == 'command':
run_time = '%.0f'%((data.end-data.start)*1000)
if sysvals.testcommand:
if(len(testruns) > 1):
testdesc1 = testdesc2 = ordinal(data.testnumber+1)
testdesc2 += ' '
- if(data.tLow == 0):
+ if(len(data.tLow) == 0):
thtml = html_timetotal.format(suspend_time, \
resume_time, testdesc1, stitle, rtitle)
else:
rtitle = 'time from firmware mode to return from kernel enter_state(%s) [kernel time only]' % sysvals.suspendmode
if(len(testruns) > 1):
testdesc = ordinal(data.testnumber+1)+' '+testdesc
- if(data.tLow == 0):
+ if(len(data.tLow) == 0):
thtml = html_timetotal.format(suspend_time, \
resume_time, testdesc, stitle, rtitle)
else:
# draw the full timeline
devtl.createZoomBox(sysvals.suspendmode, len(testruns))
- phases = {'suspend':[],'resume':[]}
- for phase in data.dmesg:
- if 'resume' in phase:
- phases['resume'].append(phase)
- else:
- phases['suspend'].append(phase)
-
- # draw each test run chronologically
for data in testruns:
+ # draw each test run and block chronologically
+ phases = {'suspend':[],'resume':[]}
+ for phase in data.sortedPhases():
+ if data.dmesg[phase]['start'] >= data.tSuspended:
+ phases['resume'].append(phase)
+ else:
+ phases['suspend'].append(phase)
# now draw the actual timeline blocks
for dir in phases:
# draw suspend and resume blocks separately
continue
width = '%f' % (((mTotal*100.0)-sysvals.srgap/2)/tTotal)
devtl.html += devtl.html_tblock.format(bname, left, width, devtl.scaleH)
- for b in sorted(phases[dir]):
+ for b in phases[dir]:
# draw the phase color background
phase = data.dmesg[b]
length = phase['end']-phase['start']
id = '%d_%d' % (idx1, idx2)
right = '%f' % (((mMax-t)*100.0)/mTotal)
devtl.html += html_error.format(right, id, type)
- for b in sorted(phases[dir]):
+ for b in phases[dir]:
# draw the devices for this phase
phaselist = data.dmesg[b]['list']
for d in data.tdevlist[b]:
# draw a legend which describes the phases by color
if sysvals.suspendmode != 'command':
- data = testruns[-1]
+ phasedef = testruns[-1].phasedef
devtl.html += '<div class="legend">\n'
- pdelta = 100.0/len(data.phases)
+ pdelta = 100.0/len(phasedef.keys())
pmargin = pdelta / 4.0
- for phase in data.phases:
- tmp = phase.split('_')
- id = tmp[0][0]
- if(len(tmp) > 1):
- id += tmp[1][0]
- order = '%.2f' % ((data.dmesg[phase]['order'] * pdelta) + pmargin)
+ for phase in sorted(phasedef, key=lambda k:phasedef[k]['order']):
+ id, p = '', phasedef[phase]
+ for word in phase.split('_'):
+ id += word[0]
+ order = '%.2f' % ((p['order'] * pdelta) + pmargin)
name = string.replace(phase, '_', ' ')
- devtl.html += devtl.html_legend.format(order, \
- data.dmesg[phase]['color'], name, id)
+ devtl.html += devtl.html_legend.format(order, p['color'], name, id)
devtl.html += '</div>\n'
hf = open(sysvals.htmlfile, 'w')
pscolor = 'linear-gradient(to top left, #ccc, #eee)'
hf.write(devtl.html_phaselet.format('pre_suspend_process', \
'0', '0', pscolor))
- for b in data.phases:
+ for b in data.sortedPhases():
phase = data.dmesg[b]
length = phase['end']-phase['start']
left = '%.3f' % (((phase['start']-t0)*100.0)/tTotal)
sv.rstgt, sv.rsval, sv.rsdir = 'on', 'auto', 'enabled'
else:
sv.rstgt, sv.rsval, sv.rsdir = 'auto', 'on', 'disabled'
- print('CONFIGURING RUNTIME SUSPEND...')
+ pprint('CONFIGURING RUNTIME SUSPEND...')
sv.rslist = deviceInfo(sv.rstgt)
for i in sv.rslist:
sv.setVal(sv.rsval, i)
- print('runtime suspend %s on all devices (%d changed)' % (sv.rsdir, len(sv.rslist)))
- print('waiting 5 seconds...')
+ pprint('runtime suspend %s on all devices (%d changed)' % (sv.rsdir, len(sv.rslist)))
+ pprint('waiting 5 seconds...')
time.sleep(5)
else:
# runtime suspend re-enable or re-disable
for i in sv.rslist:
sv.setVal(sv.rstgt, i)
- print('runtime suspend settings restored on %d devices' % len(sv.rslist))
+ pprint('runtime suspend settings restored on %d devices' % len(sv.rslist))
# Function: executeSuspend
# Description:
def executeSuspend():
pm = ProcessMonitor()
tp = sysvals.tpath
- fwdata = []
+ testdata = []
+ battery = True if getBattery() else False
# run these commands to prepare the system for suspend
if sysvals.display:
- if sysvals.display > 0:
- print('TURN DISPLAY ON')
- call('xset -d :0.0 dpms force suspend', shell=True)
- call('xset -d :0.0 dpms force on', shell=True)
- else:
- print('TURN DISPLAY OFF')
- call('xset -d :0.0 dpms force suspend', shell=True)
+ pprint('SET DISPLAY TO %s' % sysvals.display.upper())
+ displayControl(sysvals.display)
time.sleep(1)
if sysvals.sync:
- print('SYNCING FILESYSTEMS')
+ pprint('SYNCING FILESYSTEMS')
call('sync', shell=True)
# mark the start point in the kernel ring buffer just as we start
sysvals.initdmesg()
# start ftrace
if(sysvals.usecallgraph or sysvals.usetraceevents):
- print('START TRACING')
+ pprint('START TRACING')
sysvals.fsetVal('1', 'tracing_on')
if sysvals.useprocmon:
pm.start()
sysvals.fsetVal('WAIT END', 'trace_marker')
# start message
if sysvals.testcommand != '':
- print('COMMAND START')
+ pprint('COMMAND START')
else:
if(sysvals.rtcwake):
- print('SUSPEND START')
+ pprint('SUSPEND START')
else:
- print('SUSPEND START (press a key to resume)')
+ pprint('SUSPEND START (press a key to resume)')
+ bat1 = getBattery() if battery else False
# set rtcwake
if(sysvals.rtcwake):
- print('will issue an rtcwake in %d seconds' % sysvals.rtcwaketime)
+ pprint('will issue an rtcwake in %d seconds' % sysvals.rtcwaketime)
sysvals.rtcWakeAlarmOn()
# start of suspend trace marker
if(sysvals.usecallgraph or sysvals.usetraceevents):
time.sleep(sysvals.predelay/1000.0)
sysvals.fsetVal('WAIT END', 'trace_marker')
# initiate suspend or command
+ tdata = {'error': ''}
if sysvals.testcommand != '':
- call(sysvals.testcommand+' 2>&1', shell=True);
+ res = call(sysvals.testcommand+' 2>&1', shell=True);
+ if res != 0:
+ tdata['error'] = 'cmd returned %d' % res
else:
mode = sysvals.suspendmode
if sysvals.memmode and os.path.exists(sysvals.mempowerfile):
pf = open(sysvals.mempowerfile, 'w')
pf.write(sysvals.memmode)
pf.close()
+ if sysvals.diskmode and os.path.exists(sysvals.diskpowerfile):
+ mode = 'disk'
+ pf = open(sysvals.diskpowerfile, 'w')
+ pf.write(sysvals.diskmode)
+ pf.close()
pf = open(sysvals.powerfile, 'w')
pf.write(mode)
# execution will pause here
try:
pf.close()
- except:
- pass
+ except Exception as e:
+ tdata['error'] = str(e)
if(sysvals.rtcwake):
sysvals.rtcWakeAlarmOff()
# postdelay delay
time.sleep(sysvals.postdelay/1000.0)
sysvals.fsetVal('WAIT END', 'trace_marker')
# return from suspend
- print('RESUME COMPLETE')
+ pprint('RESUME COMPLETE')
if(sysvals.usecallgraph or sysvals.usetraceevents):
sysvals.fsetVal('RESUME COMPLETE', 'trace_marker')
if(sysvals.suspendmode == 'mem' or sysvals.suspendmode == 'command'):
- fwdata.append(getFPDT(False))
+ tdata['fw'] = getFPDT(False)
+ bat2 = getBattery() if battery else False
+ if battery and bat1 and bat2:
+ tdata['bat'] = (bat1, bat2)
+ testdata.append(tdata)
# stop ftrace
if(sysvals.usecallgraph or sysvals.usetraceevents):
if sysvals.useprocmon:
pm.stop()
sysvals.fsetVal('0', 'tracing_on')
- print('CAPTURING TRACE')
- op = sysvals.writeDatafileHeader(sysvals.ftracefile, fwdata)
+ # grab a copy of the dmesg output
+ pprint('CAPTURING DMESG')
+ sysvals.getdmesg(testdata)
+ # grab a copy of the ftrace output
+ if(sysvals.usecallgraph or sysvals.usetraceevents):
+ pprint('CAPTURING TRACE')
+ op = sysvals.writeDatafileHeader(sysvals.ftracefile, testdata)
fp = open(tp+'trace', 'r')
for line in fp:
op.write(line)
op.close()
sysvals.fsetVal('', 'trace')
devProps()
- # grab a copy of the dmesg output
- print('CAPTURING DMESG')
- sysvals.getdmesg(fwdata)
def readFile(file):
if os.path.islink(file):
# a list of USB device names to sysvals for better timeline readability
def deviceInfo(output=''):
if not output:
- print('LEGEND')
- print('---------------------------------------------------------------------------------------------')
- print(' A = async/sync PM queue (A/S) C = runtime active children')
- print(' R = runtime suspend enabled/disabled (E/D) rACTIVE = runtime active (min/sec)')
- print(' S = runtime status active/suspended (A/S) rSUSPEND = runtime suspend (min/sec)')
- print(' U = runtime usage count')
- print('---------------------------------------------------------------------------------------------')
- print('DEVICE NAME A R S U C rACTIVE rSUSPEND')
- print('---------------------------------------------------------------------------------------------')
+ pprint('LEGEND\n'\
+ '---------------------------------------------------------------------------------------------\n'\
+ ' A = async/sync PM queue (A/S) C = runtime active children\n'\
+ ' R = runtime suspend enabled/disabled (E/D) rACTIVE = runtime active (min/sec)\n'\
+ ' S = runtime status active/suspended (A/S) rSUSPEND = runtime suspend (min/sec)\n'\
+ ' U = runtime usage count\n'\
+ '---------------------------------------------------------------------------------------------\n'\
+ 'DEVICE NAME A R S U C rACTIVE rSUSPEND\n'\
+ '---------------------------------------------------------------------------------------------')
res = []
tgtval = 'runtime_status'
alreadystamped = True
continue
# determine the trace data type (required for further parsing)
- m = re.match(sysvals.tracertypefmt, line)
+ m = re.match(tp.tracertypefmt, line)
if(m):
tp.setTracerType(m.group('t'))
continue
fp.close()
if 'mem' in modes and not deep:
modes.remove('mem')
+ if('disk' in modes and os.path.exists(sysvals.diskpowerfile)):
+ fp = open(sysvals.diskpowerfile, 'r')
+ for m in string.split(fp.read()):
+ modes.append('disk-%s' % m.strip('[]'))
+ fp.close()
return modes
# Function: dmidecode
return out
def getBattery():
- p = '/sys/class/power_supply'
- bat = dict()
+ p, charge, bat = '/sys/class/power_supply', 0, {}
+ if not os.path.exists(p):
+ return False
for d in os.listdir(p):
type = sysvals.getVal(os.path.join(p, d, 'type')).strip().lower()
if type != 'battery':
for v in ['status', 'energy_now', 'capacity_now']:
bat[v] = sysvals.getVal(os.path.join(p, d, v)).strip().lower()
break
- ac = True
- if 'status' in bat and 'discharging' in bat['status']:
- ac = False
- charge = 0
+ if 'status' not in bat:
+ return False
+ ac = False if 'discharging' in bat['status'] else True
for v in ['energy_now', 'capacity_now']:
if v in bat and bat[v]:
charge = int(bat[v])
return (ac, charge)
+def displayControl(cmd):
+ xset, ret = 'xset -d :0.0 {0}', 0
+ if sysvals.sudouser:
+ xset = 'sudo -u %s %s' % (sysvals.sudouser, xset)
+ if cmd == 'init':
+ ret = call(xset.format('dpms 0 0 0'), shell=True)
+ if not ret:
+ ret = call(xset.format('s off'), shell=True)
+ elif cmd == 'reset':
+ ret = call(xset.format('s reset'), shell=True)
+ elif cmd in ['on', 'off', 'standby', 'suspend']:
+ b4 = displayControl('stat')
+ ret = call(xset.format('dpms force %s' % cmd), shell=True)
+ if not ret:
+ curr = displayControl('stat')
+ sysvals.vprint('Display Switched: %s -> %s' % (b4, curr))
+ if curr != cmd:
+ sysvals.vprint('WARNING: Display failed to change to %s' % cmd)
+ if ret:
+ sysvals.vprint('WARNING: Display failed to change to %s with xset' % cmd)
+ return ret
+ elif cmd == 'stat':
+ fp = Popen(xset.format('q').split(' '), stdout=PIPE).stdout
+ ret = 'unknown'
+ for line in fp:
+ m = re.match('[\s]*Monitor is (?P<m>.*)', line)
+ if(m and len(m.group('m')) >= 2):
+ out = m.group('m').lower()
+ ret = out[3:] if out[0:2] == 'in' else out
+ break
+ fp.close()
+ return ret
+
# Function: getFPDT
# Description:
# Read the acpi bios tables and pull out FPDT, the firmware data
table = struct.unpack('4sIBB6s8sI4sI', buf[0:36])
if(output):
- print('')
- print('Firmware Performance Data Table (%s)' % table[0])
- print(' Signature : %s' % table[0])
- print(' Table Length : %u' % table[1])
- print(' Revision : %u' % table[2])
- print(' Checksum : 0x%x' % table[3])
- print(' OEM ID : %s' % table[4])
- print(' OEM Table ID : %s' % table[5])
- print(' OEM Revision : %u' % table[6])
- print(' Creator ID : %s' % table[7])
- print(' Creator Revision : 0x%x' % table[8])
- print('')
+ pprint('\n'\
+ 'Firmware Performance Data Table (%s)\n'\
+ ' Signature : %s\n'\
+ ' Table Length : %u\n'\
+ ' Revision : %u\n'\
+ ' Checksum : 0x%x\n'\
+ ' OEM ID : %s\n'\
+ ' OEM Table ID : %s\n'\
+ ' OEM Revision : %u\n'\
+ ' Creator ID : %s\n'\
+ ' Creator Revision : 0x%x\n'\
+ '' % (table[0], table[0], table[1], table[2], table[3],
+ table[4], table[5], table[6], table[7], table[8]))
if(table[0] != 'FPDT'):
if(output):
first = fp.read(8)
except:
if(output):
- print('Bad address 0x%x in %s' % (addr, sysvals.mempath))
+ pprint('Bad address 0x%x in %s' % (addr, sysvals.mempath))
return [0, 0]
rechead = struct.unpack('4sI', first)
recdata = fp.read(rechead[1]-8)
if(rechead[0] == 'FBPT'):
record = struct.unpack('HBBIQQQQQ', recdata)
if(output):
- print('%s (%s)' % (rectype[header[0]], rechead[0]))
- print(' Reset END : %u ns' % record[4])
- print(' OS Loader LoadImage Start : %u ns' % record[5])
- print(' OS Loader StartImage Start : %u ns' % record[6])
- print(' ExitBootServices Entry : %u ns' % record[7])
- print(' ExitBootServices Exit : %u ns' % record[8])
+ pprint('%s (%s)\n'\
+ ' Reset END : %u ns\n'\
+ ' OS Loader LoadImage Start : %u ns\n'\
+ ' OS Loader StartImage Start : %u ns\n'\
+ ' ExitBootServices Entry : %u ns\n'\
+ ' ExitBootServices Exit : %u ns'\
+ '' % (rectype[header[0]], rechead[0], record[4], record[5],
+ record[6], record[7], record[8]))
elif(rechead[0] == 'S3PT'):
if(output):
- print('%s (%s)' % (rectype[header[0]], rechead[0]))
+ pprint('%s (%s)' % (rectype[header[0]], rechead[0]))
j = 0
while(j < len(recdata)):
prechead = struct.unpack('HBB', recdata[j:j+4])
record = struct.unpack('IIQQ', recdata[j:j+prechead[1]])
fwData[1] = record[2]
if(output):
- print(' %s' % prectype[prechead[0]])
- print(' Resume Count : %u' % \
- record[1])
- print(' FullResume : %u ns' % \
- record[2])
- print(' AverageResume : %u ns' % \
- record[3])
+ pprint(' %s\n'\
+ ' Resume Count : %u\n'\
+ ' FullResume : %u ns\n'\
+ ' AverageResume : %u ns'\
+ '' % (prectype[prechead[0]], record[1],
+ record[2], record[3]))
elif(prechead[0] == 1):
record = struct.unpack('QQ', recdata[j+4:j+prechead[1]])
fwData[0] = record[1] - record[0]
if(output):
- print(' %s' % prectype[prechead[0]])
- print(' SuspendStart : %u ns' % \
- record[0])
- print(' SuspendEnd : %u ns' % \
- record[1])
- print(' SuspendTime : %u ns' % \
- fwData[0])
+ pprint(' %s\n'\
+ ' SuspendStart : %u ns\n'\
+ ' SuspendEnd : %u ns\n'\
+ ' SuspendTime : %u ns'\
+ '' % (prectype[prechead[0]], record[0],
+ record[1], fwData[0]))
+
j += prechead[1]
if(output):
- print('')
+ pprint('')
i += header[1]
fp.close()
return fwData
# Output:
# True if the test will work, False if not
def statusCheck(probecheck=False):
- status = True
+ status = ''
- print('Checking this system (%s)...' % platform.node())
+ pprint('Checking this system (%s)...' % platform.node())
# check we have root access
res = sysvals.colorText('NO (No features of this tool will work!)')
if(sysvals.rootCheck(False)):
res = 'YES'
- print(' have root access: %s' % res)
+ pprint(' have root access: %s' % res)
if(res != 'YES'):
- print(' Try running this script with sudo')
- return False
+ pprint(' Try running this script with sudo')
+ return 'missing root access'
# check sysfs is mounted
res = sysvals.colorText('NO (No features of this tool will work!)')
if(os.path.exists(sysvals.powerfile)):
res = 'YES'
- print(' is sysfs mounted: %s' % res)
+ pprint(' is sysfs mounted: %s' % res)
if(res != 'YES'):
- return False
+ return 'sysfs is missing'
# check target mode is a valid mode
if sysvals.suspendmode != 'command':
if(sysvals.suspendmode in modes):
res = 'YES'
else:
- status = False
- print(' is "%s" a valid power mode: %s' % (sysvals.suspendmode, res))
+ status = '%s mode is not supported' % sysvals.suspendmode
+ pprint(' is "%s" a valid power mode: %s' % (sysvals.suspendmode, res))
if(res == 'NO'):
- print(' valid power modes are: %s' % modes)
- print(' please choose one with -m')
+ pprint(' valid power modes are: %s' % modes)
+ pprint(' please choose one with -m')
# check if ftrace is available
res = sysvals.colorText('NO')
if(ftgood):
res = 'YES'
elif(sysvals.usecallgraph):
- status = False
- print(' is ftrace supported: %s' % res)
+ status = 'ftrace is not properly supported'
+ pprint(' is ftrace supported: %s' % res)
# check if kprobes are available
res = sysvals.colorText('NO')
res = 'YES'
else:
sysvals.usedevsrc = False
- print(' are kprobes supported: %s' % res)
+ pprint(' are kprobes supported: %s' % res)
# what data source are we using
res = 'DMESG'
sysvals.usetraceevents = False
if(sysvals.usetraceevents):
res = 'FTRACE (all trace events found)'
- print(' timeline data source: %s' % res)
+ pprint(' timeline data source: %s' % res)
# check if rtcwake
res = sysvals.colorText('NO')
if(sysvals.rtcpath != ''):
res = 'YES'
elif(sysvals.rtcwake):
- status = False
- print(' is rtcwake supported: %s' % res)
+ status = 'rtcwake is not properly supported'
+ pprint(' is rtcwake supported: %s' % res)
if not probecheck:
return status
def doError(msg, help=False):
if(help == True):
printHelp()
- print('ERROR: %s\n') % msg
+ pprint('ERROR: %s\n' % msg)
sysvals.outputResult({'error':msg})
- sys.exit()
+ sys.exit(1)
# Function: getArgInt
# Description:
return val
def processData(live=False):
- print('PROCESSING DATA')
+ pprint('PROCESSING DATA')
error = ''
if(sysvals.usetraceevents):
testruns, error = parseTraceLog(live)
appendIncompleteTraceLog(testruns)
sysvals.vprint('Command:\n %s' % sysvals.cmdline)
for data in testruns:
+ if data.battery:
+ a1, c1, a2, c2 = data.battery
+ s = 'Battery:\n Before - AC: %s, Charge: %d\n After - AC: %s, Charge: %d' % \
+ (a1, int(c1), a2, int(c2))
+ sysvals.vprint(s)
data.printDetails()
if sysvals.cgdump:
for data in testruns:
data.debugPrint()
- sys.exit()
+ sys.exit(0)
if len(testruns) < 1:
+ pprint('ERROR: Not enough test data to build a timeline')
return (testruns, {'error': 'timeline generation failed'})
sysvals.vprint('Creating the html timeline (%s)...' % sysvals.htmlfile)
createHTML(testruns, error)
- print('DONE')
+ pprint('DONE')
data = testruns[0]
stamp = data.stamp
stamp['suspend'], stamp['resume'] = data.getTimeValues()
elif not os.access(sysvals.htmlfile, os.W_OK):
doError('missing permission to write to %s' % sysvals.htmlfile)
testruns, stamp = processData(False)
+ sysvals.logmsg = ''
return stamp
# Function: runTest
executeSuspend()
sysvals.cleanupFtrace()
if sysvals.skiphtml:
- sysvals.sudouser(sysvals.testdir)
+ sysvals.sudoUserchown(sysvals.testdir)
return
testruns, stamp = processData(True)
for data in testruns:
del data
- sysvals.sudouser(sysvals.testdir)
+ sysvals.sudoUserchown(sysvals.testdir)
sysvals.outputResult(stamp, n)
+ if 'error' in stamp:
+ return 2
+ return 0
def find_in_html(html, start, end, firstonly=True):
n, out = 0, []
return ''
return out
+def data_from_html(file, outpath, devlist=False):
+ html = open(file, 'r').read()
+ suspend = find_in_html(html, 'Kernel Suspend', 'ms')
+ resume = find_in_html(html, 'Kernel Resume', 'ms')
+ line = find_in_html(html, '<div class="stamp">', '</div>')
+ stmp = line.split()
+ if not suspend or not resume or len(stmp) != 8:
+ return False
+ try:
+ dt = datetime.strptime(' '.join(stmp[3:]), '%B %d %Y, %I:%M:%S %p')
+ except:
+ return False
+ tstr = dt.strftime('%Y/%m/%d %H:%M:%S')
+ error = find_in_html(html, '<table class="testfail"><tr><td>', '</td>')
+ if error:
+ m = re.match('[a-z]* failed in (?P<p>[a-z0-9_]*) phase', error)
+ if m:
+ result = 'fail in %s' % m.group('p')
+ else:
+ result = 'fail'
+ else:
+ result = 'pass'
+ ilist = []
+ e = find_in_html(html, 'class="err"[\w=":;\.%\- ]*>', '→</div>', False)
+ for i in list(set(e)):
+ ilist.append('%sx%d' % (i, e.count(i)) if e.count(i) > 1 else i)
+ low = find_in_html(html, 'freeze time: <b>', ' ms</b>')
+ if low and '|' in low:
+ ilist.append('FREEZEx%d' % len(low.split('|')))
+ devices = dict()
+ for line in html.split('\n'):
+ m = re.match(' *<div id=\"[a,0-9]*\" *title=\"(?P<title>.*)\" class=\"thread.*', line)
+ if not m or 'thread kth' in line or 'thread sec' in line:
+ continue
+ m = re.match('(?P<n>.*) \((?P<t>[0-9,\.]*) ms\) (?P<p>.*)', m.group('title'))
+ if not m:
+ continue
+ name, time, phase = m.group('n'), m.group('t'), m.group('p')
+ if ' async' in name or ' sync' in name:
+ name = ' '.join(name.split(' ')[:-1])
+ d = phase.split('_')[0]
+ if d not in devices:
+ devices[d] = dict()
+ if name not in devices[d]:
+ devices[d][name] = 0.0
+ devices[d][name] += float(time)
+ worst = {'suspend': {'name':'', 'time': 0.0},
+ 'resume': {'name':'', 'time': 0.0}}
+ for d in devices:
+ if d not in worst:
+ worst[d] = dict()
+ dev = devices[d]
+ if len(dev.keys()) > 0:
+ n = sorted(dev, key=dev.get, reverse=True)[0]
+ worst[d]['name'], worst[d]['time'] = n, dev[n]
+ data = {
+ 'mode': stmp[2],
+ 'host': stmp[0],
+ 'kernel': stmp[1],
+ 'time': tstr,
+ 'result': result,
+ 'issues': ' '.join(ilist),
+ 'suspend': suspend,
+ 'resume': resume,
+ 'sus_worst': worst['suspend']['name'],
+ 'sus_worsttime': worst['suspend']['time'],
+ 'res_worst': worst['resume']['name'],
+ 'res_worsttime': worst['resume']['time'],
+ 'url': os.path.relpath(file, outpath),
+ }
+ if devlist:
+ data['devlist'] = devices
+ return data
+
# Function: runSummary
# Description:
# create a summary of tests in a sub-directory
def runSummary(subdir, local=True, genhtml=False):
inpath = os.path.abspath(subdir)
- outpath = inpath
- if local:
- outpath = os.path.abspath('.')
- print('Generating a summary of folder "%s"' % inpath)
+ outpath = os.path.abspath('.') if local else inpath
+ pprint('Generating a summary of folder "%s"' % inpath)
if genhtml:
for dirname, dirnames, filenames in os.walk(subdir):
sysvals.dmesgfile = sysvals.ftracefile = sysvals.htmlfile = ''
sysvals.setOutputFile()
if sysvals.ftracefile and sysvals.htmlfile and \
not os.path.exists(sysvals.htmlfile):
- print('FTRACE: %s' % sysvals.ftracefile)
+ pprint('FTRACE: %s' % sysvals.ftracefile)
if sysvals.dmesgfile:
- print('DMESG : %s' % sysvals.dmesgfile)
+ pprint('DMESG : %s' % sysvals.dmesgfile)
rerunTest()
testruns = []
+ desc = {'host':[],'mode':[],'kernel':[]}
for dirname, dirnames, filenames in os.walk(subdir):
for filename in filenames:
if(not re.match('.*.html', filename)):
continue
- file = os.path.join(dirname, filename)
- html = open(file, 'r').read()
- suspend = find_in_html(html, 'Kernel Suspend', 'ms')
- resume = find_in_html(html, 'Kernel Resume', 'ms')
- line = find_in_html(html, '<div class="stamp">', '</div>')
- stmp = line.split()
- if not suspend or not resume or len(stmp) != 8:
+ data = data_from_html(os.path.join(dirname, filename), outpath)
+ if(not data):
continue
- try:
- dt = datetime.strptime(' '.join(stmp[3:]), '%B %d %Y, %I:%M:%S %p')
- except:
- continue
- tstr = dt.strftime('%Y/%m/%d %H:%M:%S')
- error = find_in_html(html, '<table class="testfail"><tr><td>', '</td>')
- result = 'fail' if error else 'pass'
- ilist = []
- e = find_in_html(html, 'class="err"[\w=":;\.%\- ]*>', '→</div>', False)
- for i in list(set(e)):
- ilist.append('%sx%d' % (i, e.count(i)) if e.count(i) > 1 else i)
- data = {
- 'mode': stmp[2],
- 'host': stmp[0],
- 'kernel': stmp[1],
- 'time': tstr,
- 'result': result,
- 'issues': ','.join(ilist),
- 'suspend': suspend,
- 'resume': resume,
- 'url': os.path.relpath(file, outpath),
- }
testruns.append(data)
+ for key in desc:
+ if data[key] not in desc[key]:
+ desc[key].append(data[key])
outfile = os.path.join(outpath, 'summary.html')
- print('Summary file: %s' % outfile)
- createHTMLSummarySimple(testruns, outfile, inpath)
+ pprint('Summary file: %s' % outfile)
+ if len(desc['host']) == len(desc['mode']) == len(desc['kernel']) == 1:
+ title = '%s %s %s' % (desc['host'][0], desc['kernel'][0], desc['mode'][0])
+ else:
+ title = inpath
+ createHTMLSummarySimple(testruns, outfile, title)
# Function: checkArgBool
# Description:
else:
doError('invalid value --> (%s: %s), use "enable/disable"' % (option, value), True)
elif(option == 'display'):
- if value in switchvalues:
- if value in switchoff:
- sysvals.display = -1
- else:
- sysvals.display = 1
- else:
- doError('invalid value --> (%s: %s), use "on/off"' % (option, value), True)
+ disopt = ['on', 'off', 'standby', 'suspend']
+ if value not in disopt:
+ doError('invalid value --> (%s: %s), use %s' % (option, value, disopt), True)
+ sysvals.display = value
elif(option == 'gzip'):
sysvals.gzip = checkArgBool(option, value)
elif(option == 'cgfilter'):
sysvals.cgtest = getArgInt('cgtest', value, 0, 1, False)
elif(option == 'cgphase'):
d = Data(0)
- if value not in d.phases:
+ if value not in d.sortedPhases():
doError('invalid phase --> (%s: %s), valid phases are %s'\
- % (option, value, d.phases), True)
+ % (option, value, d.sortedPhases()), True)
sysvals.cgphase = value
elif(option == 'fadd'):
file = sysvals.configFile(value)
# Description:
# print out the help text
def printHelp():
- print('')
- print('%s v%s' % (sysvals.title, sysvals.version))
- print('Usage: sudo sleepgraph <options> <commands>')
- print('')
- print('Description:')
- print(' This tool is designed to assist kernel and OS developers in optimizing')
- print(' their linux stack\'s suspend/resume time. Using a kernel image built')
- print(' with a few extra options enabled, the tool will execute a suspend and')
- print(' capture dmesg and ftrace data until resume is complete. This data is')
- print(' transformed into a device timeline and an optional callgraph to give')
- print(' a detailed view of which devices/subsystems are taking the most')
- print(' time in suspend/resume.')
- print('')
- print(' If no specific command is given, the default behavior is to initiate')
- print(' a suspend/resume and capture the dmesg/ftrace output as an html timeline.')
- print('')
- print(' Generates output files in subdirectory: suspend-yymmdd-HHMMSS')
- print(' HTML output: <hostname>_<mode>.html')
- print(' raw dmesg output: <hostname>_<mode>_dmesg.txt')
- print(' raw ftrace output: <hostname>_<mode>_ftrace.txt')
- print('')
- print('Options:')
- print(' -h Print this help text')
- print(' -v Print the current tool version')
- print(' -config fn Pull arguments and config options from file fn')
- print(' -verbose Print extra information during execution and analysis')
- print(' -m mode Mode to initiate for suspend (default: %s)') % (sysvals.suspendmode)
- print(' -o name Overrides the output subdirectory name when running a new test')
- print(' default: suspend-{date}-{time}')
- print(' -rtcwake t Wakeup t seconds after suspend, set t to "off" to disable (default: 15)')
- print(' -addlogs Add the dmesg and ftrace logs to the html output')
- print(' -srgap Add a visible gap in the timeline between sus/res (default: disabled)')
- print(' -skiphtml Run the test and capture the trace logs, but skip the timeline (default: disabled)')
- print(' -result fn Export a results table to a text file for parsing.')
- print(' [testprep]')
- print(' -sync Sync the filesystems before starting the test')
- print(' -rs on/off Enable/disable runtime suspend for all devices, restore all after test')
- print(' -display on/off Turn the display on or off for the test')
- print(' [advanced]')
- print(' -gzip Gzip the trace and dmesg logs to save space')
- print(' -cmd {s} Run the timeline over a custom command, e.g. "sync -d"')
- print(' -proc Add usermode process info into the timeline (default: disabled)')
- print(' -dev Add kernel function calls and threads to the timeline (default: disabled)')
- print(' -x2 Run two suspend/resumes back to back (default: disabled)')
- print(' -x2delay t Include t ms delay between multiple test runs (default: 0 ms)')
- print(' -predelay t Include t ms delay before 1st suspend (default: 0 ms)')
- print(' -postdelay t Include t ms delay after last resume (default: 0 ms)')
- print(' -mindev ms Discard all device blocks shorter than ms milliseconds (e.g. 0.001 for us)')
- print(' -multi n d Execute <n> consecutive tests at <d> seconds intervals. The outputs will')
- print(' be created in a new subdirectory with a summary page.')
- print(' [debug]')
- print(' -f Use ftrace to create device callgraphs (default: disabled)')
- print(' -maxdepth N limit the callgraph data to N call levels (default: 0=all)')
- print(' -expandcg pre-expand the callgraph data in the html output (default: disabled)')
- print(' -fadd file Add functions to be graphed in the timeline from a list in a text file')
- print(' -filter "d1,d2,..." Filter out all but this comma-delimited list of device names')
- print(' -mincg ms Discard all callgraphs shorter than ms milliseconds (e.g. 0.001 for us)')
- print(' -cgphase P Only show callgraph data for phase P (e.g. suspend_late)')
- print(' -cgtest N Only show callgraph data for test N (e.g. 0 or 1 in an x2 run)')
- print(' -timeprec N Number of significant digits in timestamps (0:S, [3:ms], 6:us)')
- print(' -cgfilter S Filter the callgraph output in the timeline')
- print(' -cgskip file Callgraph functions to skip, off to disable (default: cgskip.txt)')
- print(' -bufsize N Set trace buffer size to N kilo-bytes (default: all of free memory)')
- print('')
- print('Other commands:')
- print(' -modes List available suspend modes')
- print(' -status Test to see if the system is enabled to run this tool')
- print(' -fpdt Print out the contents of the ACPI Firmware Performance Data Table')
- print(' -battery Print out battery info (if available)')
- print(' -sysinfo Print out system info extracted from BIOS')
- print(' -devinfo Print out the pm settings of all devices which support runtime suspend')
- print(' -flist Print the list of functions currently being captured in ftrace')
- print(' -flistall Print all functions capable of being captured in ftrace')
- print(' -summary dir Create a summary of tests in this dir [-genhtml builds missing html]')
- print(' [redo]')
- print(' -ftrace ftracefile Create HTML output using ftrace input (used with -dmesg)')
- print(' -dmesg dmesgfile Create HTML output using dmesg (used with -ftrace)')
- print('')
+ pprint('\n%s v%s\n'\
+ 'Usage: sudo sleepgraph <options> <commands>\n'\
+ '\n'\
+ 'Description:\n'\
+ ' This tool is designed to assist kernel and OS developers in optimizing\n'\
+ ' their linux stack\'s suspend/resume time. Using a kernel image built\n'\
+ ' with a few extra options enabled, the tool will execute a suspend and\n'\
+ ' capture dmesg and ftrace data until resume is complete. This data is\n'\
+ ' transformed into a device timeline and an optional callgraph to give\n'\
+ ' a detailed view of which devices/subsystems are taking the most\n'\
+ ' time in suspend/resume.\n'\
+ '\n'\
+ ' If no specific command is given, the default behavior is to initiate\n'\
+ ' a suspend/resume and capture the dmesg/ftrace output as an html timeline.\n'\
+ '\n'\
+ ' Generates output files in subdirectory: suspend-yymmdd-HHMMSS\n'\
+ ' HTML output: <hostname>_<mode>.html\n'\
+ ' raw dmesg output: <hostname>_<mode>_dmesg.txt\n'\
+ ' raw ftrace output: <hostname>_<mode>_ftrace.txt\n'\
+ '\n'\
+ 'Options:\n'\
+ ' -h Print this help text\n'\
+ ' -v Print the current tool version\n'\
+ ' -config fn Pull arguments and config options from file fn\n'\
+ ' -verbose Print extra information during execution and analysis\n'\
+ ' -m mode Mode to initiate for suspend (default: %s)\n'\
+ ' -o name Overrides the output subdirectory name when running a new test\n'\
+ ' default: suspend-{date}-{time}\n'\
+ ' -rtcwake t Wakeup t seconds after suspend, set t to "off" to disable (default: 15)\n'\
+ ' -addlogs Add the dmesg and ftrace logs to the html output\n'\
+ ' -srgap Add a visible gap in the timeline between sus/res (default: disabled)\n'\
+ ' -skiphtml Run the test and capture the trace logs, but skip the timeline (default: disabled)\n'\
+ ' -result fn Export a results table to a text file for parsing.\n'\
+ ' [testprep]\n'\
+ ' -sync Sync the filesystems before starting the test\n'\
+ ' -rs on/off Enable/disable runtime suspend for all devices, restore all after test\n'\
+ ' -display m Change the display mode to m for the test (on/off/standby/suspend)\n'\
+ ' [advanced]\n'\
+ ' -gzip Gzip the trace and dmesg logs to save space\n'\
+ ' -cmd {s} Run the timeline over a custom command, e.g. "sync -d"\n'\
+ ' -proc Add usermode process info into the timeline (default: disabled)\n'\
+ ' -dev Add kernel function calls and threads to the timeline (default: disabled)\n'\
+ ' -x2 Run two suspend/resumes back to back (default: disabled)\n'\
+ ' -x2delay t Include t ms delay between multiple test runs (default: 0 ms)\n'\
+ ' -predelay t Include t ms delay before 1st suspend (default: 0 ms)\n'\
+ ' -postdelay t Include t ms delay after last resume (default: 0 ms)\n'\
+ ' -mindev ms Discard all device blocks shorter than ms milliseconds (e.g. 0.001 for us)\n'\
+ ' -multi n d Execute <n> consecutive tests at <d> seconds intervals. The outputs will\n'\
+ ' be created in a new subdirectory with a summary page.\n'\
+ ' [debug]\n'\
+ ' -f Use ftrace to create device callgraphs (default: disabled)\n'\
+ ' -maxdepth N limit the callgraph data to N call levels (default: 0=all)\n'\
+ ' -expandcg pre-expand the callgraph data in the html output (default: disabled)\n'\
+ ' -fadd file Add functions to be graphed in the timeline from a list in a text file\n'\
+ ' -filter "d1,d2,..." Filter out all but this comma-delimited list of device names\n'\
+ ' -mincg ms Discard all callgraphs shorter than ms milliseconds (e.g. 0.001 for us)\n'\
+ ' -cgphase P Only show callgraph data for phase P (e.g. suspend_late)\n'\
+ ' -cgtest N Only show callgraph data for test N (e.g. 0 or 1 in an x2 run)\n'\
+ ' -timeprec N Number of significant digits in timestamps (0:S, [3:ms], 6:us)\n'\
+ ' -cgfilter S Filter the callgraph output in the timeline\n'\
+ ' -cgskip file Callgraph functions to skip, off to disable (default: cgskip.txt)\n'\
+ ' -bufsize N Set trace buffer size to N kilo-bytes (default: all of free memory)\n'\
+ ' -devdump Print out all the raw device data for each phase\n'\
+ ' -cgdump Print out all the raw callgraph data\n'\
+ '\n'\
+ 'Other commands:\n'\
+ ' -modes List available suspend modes\n'\
+ ' -status Test to see if the system is enabled to run this tool\n'\
+ ' -fpdt Print out the contents of the ACPI Firmware Performance Data Table\n'\
+ ' -battery Print out battery info (if available)\n'\
+ ' -x<mode> Test xset by toggling the given mode (on/off/standby/suspend)\n'\
+ ' -sysinfo Print out system info extracted from BIOS\n'\
+ ' -devinfo Print out the pm settings of all devices which support runtime suspend\n'\
+ ' -flist Print the list of functions currently being captured in ftrace\n'\
+ ' -flistall Print all functions capable of being captured in ftrace\n'\
+ ' -summary dir Create a summary of tests in this dir [-genhtml builds missing html]\n'\
+ ' [redo]\n'\
+ ' -ftrace ftracefile Create HTML output using ftrace input (used with -dmesg)\n'\
+ ' -dmesg dmesgfile Create HTML output using dmesg (used with -ftrace)\n'\
+ '' % (sysvals.title, sysvals.version, sysvals.suspendmode))
return True
# ----------------- MAIN --------------------
if __name__ == '__main__':
genhtml = False
cmd = ''
- simplecmds = ['-sysinfo', '-modes', '-fpdt', '-flist', '-flistall', '-devinfo', '-status', '-battery']
+ simplecmds = ['-sysinfo', '-modes', '-fpdt', '-flist', '-flistall',
+ '-devinfo', '-status', '-battery', '-xon', '-xoff', '-xstandby',
+ '-xsuspend', '-xinit', '-xreset', '-xstat']
if '-f' in sys.argv:
sysvals.cgskip = sysvals.configFile('cgskip.txt')
# loop through the command line arguments
cmd = arg[1:]
elif(arg == '-h'):
printHelp()
- sys.exit()
+ sys.exit(0)
elif(arg == '-v'):
- print("Version %s" % sysvals.version)
- sys.exit()
+ pprint("Version %s" % sysvals.version)
+ sys.exit(0)
elif(arg == '-x2'):
sysvals.execcount = 2
elif(arg == '-x2delay'):
sysvals.skiphtml = True
elif(arg == '-cgdump'):
sysvals.cgdump = True
+ elif(arg == '-devdump'):
+ sysvals.devdump = True
elif(arg == '-genhtml'):
genhtml = True
elif(arg == '-addlogs'):
sysvals.dmesglog = sysvals.ftracelog = True
+ elif(arg == '-addlogdmesg'):
+ sysvals.dmesglog = True
+ elif(arg == '-addlogftrace'):
+ sysvals.ftracelog = True
elif(arg == '-verbose'):
sysvals.verbose = True
elif(arg == '-proc'):
try:
val = args.next()
except:
- doError('-display requires "on" or "off"', True)
- if val.lower() in switchvalues:
- if val.lower() in switchoff:
- sysvals.display = -1
- else:
- sysvals.display = 1
- else:
- doError('invalid option: %s, use "on/off"' % val, True)
+ doError('-display requires an mode value', True)
+ disopt = ['on', 'off', 'standby', 'suspend']
+ if val.lower() not in disopt:
+ doError('valid display mode values are %s' % disopt, True)
+ sysvals.display = val.lower()
elif(arg == '-maxdepth'):
sysvals.max_graph_depth = getArgInt('-maxdepth', args, 0, 1000)
elif(arg == '-rtcwake'):
except:
doError('No phase name supplied', True)
d = Data(0)
- if val not in d.phases:
+ if val not in d.phasedef:
doError('invalid phase --> (%s: %s), valid phases are %s'\
- % (arg, val, d.phases), True)
+ % (arg, val, d.phasedef.keys()), True)
sysvals.cgphase = val
elif(arg == '-cgfilter'):
try:
except:
doError('No result file supplied', True)
sysvals.result = val
+ sysvals.signalHandlerInit()
else:
doError('Invalid argument: '+arg, True)
# just run a utility command and exit
if(cmd != ''):
+ ret = 0
if(cmd == 'status'):
- statusCheck(True)
+ if not statusCheck(True):
+ ret = 1
elif(cmd == 'fpdt'):
- getFPDT(True)
+ if not getFPDT(True):
+ ret = 1
elif(cmd == 'battery'):
- print 'AC Connect: %s\nCharge: %d' % getBattery()
+ out = getBattery()
+ if out:
+ pprint('AC Connect : %s\nBattery Charge: %d' % out)
+ else:
+ pprint('no battery found')
+ ret = 1
elif(cmd == 'sysinfo'):
sysvals.printSystemInfo(True)
elif(cmd == 'devinfo'):
sysvals.getFtraceFilterFunctions(False)
elif(cmd == 'summary'):
runSummary(sysvals.outdir, True, genhtml)
- sys.exit()
+ elif(cmd in ['xon', 'xoff', 'xstandby', 'xsuspend', 'xinit', 'xreset']):
+ sysvals.verbose = True
+ ret = displayControl(cmd[1:])
+ elif(cmd == 'xstat'):
+ pprint('Display Status: %s' % displayControl('stat').upper())
+ sys.exit(ret)
# if instructed, re-analyze existing data files
if(sysvals.notestrun):
stamp = rerunTest()
sysvals.outputResult(stamp)
- sys.exit()
+ sys.exit(0)
# verify that we can run a test
- if(not statusCheck()):
- doError('Check FAILED, aborting the test run!')
+ error = statusCheck()
+ if(error):
+ doError(error)
- # extract mem modes and convert
+ # extract mem/disk extra modes and convert
mode = sysvals.suspendmode
- if 'mem' == mode[:3]:
- if '-' in mode:
- memmode = mode.split('-')[-1]
- else:
- memmode = 'deep'
+ if mode.startswith('mem'):
+ memmode = mode.split('-', 1)[-1] if '-' in mode else 'deep'
if memmode == 'shallow':
mode = 'standby'
elif memmode == 's2idle':
mode = 'mem'
sysvals.memmode = memmode
sysvals.suspendmode = mode
+ if mode.startswith('disk-'):
+ sysvals.diskmode = mode.split('-', 1)[-1]
+ sysvals.suspendmode = 'disk'
sysvals.systemInfo(dmidecode(sysvals.mempath))
setRuntimeSuspend(True)
if sysvals.display:
- call('xset -d :0.0 dpms 0 0 0', shell=True)
- call('xset -d :0.0 s off', shell=True)
+ displayControl('init')
+ ret = 0
if sysvals.multitest['run']:
# run multiple tests in a separate subdirectory
if not sysvals.outdir:
os.mkdir(sysvals.outdir)
for i in range(sysvals.multitest['count']):
if(i != 0):
- print('Waiting %d seconds...' % (sysvals.multitest['delay']))
+ pprint('Waiting %d seconds...' % (sysvals.multitest['delay']))
time.sleep(sysvals.multitest['delay'])
- print('TEST (%d/%d) START' % (i+1, sysvals.multitest['count']))
+ pprint('TEST (%d/%d) START' % (i+1, sysvals.multitest['count']))
fmt = 'suspend-%y%m%d-%H%M%S'
sysvals.testdir = os.path.join(sysvals.outdir, datetime.now().strftime(fmt))
- runTest(i+1)
- print('TEST (%d/%d) COMPLETE' % (i+1, sysvals.multitest['count']))
+ ret = runTest(i+1)
+ pprint('TEST (%d/%d) COMPLETE' % (i+1, sysvals.multitest['count']))
sysvals.logmsg = ''
if not sysvals.skiphtml:
runSummary(sysvals.outdir, False, False)
- sysvals.sudouser(sysvals.outdir)
+ sysvals.sudoUserchown(sysvals.outdir)
else:
if sysvals.outdir:
sysvals.testdir = sysvals.outdir
# run the test in the current directory
- runTest()
+ ret = runTest()
if sysvals.display:
- call('xset -d :0.0 s reset', shell=True)
+ displayControl('reset')
setRuntimeSuspend(False)
+ sys.exit(ret)
switch (model) {
case INTEL_FAM6_ATOM_GOLDMONT:
case INTEL_FAM6_SKYLAKE_X:
- case INTEL_FAM6_ATOM_DENVERTON:
+ case INTEL_FAM6_ATOM_GOLDMONT_X:
return 1;
}
return 0;
pkg_cstate_limits = skx_pkg_cstate_limits;
has_misc_feature_control = 1;
break;
- case INTEL_FAM6_ATOM_SILVERMONT1: /* BYT */
+ case INTEL_FAM6_ATOM_SILVERMONT: /* BYT */
no_MSR_MISC_PWR_MGMT = 1;
- case INTEL_FAM6_ATOM_SILVERMONT2: /* AVN */
+ case INTEL_FAM6_ATOM_SILVERMONT_X: /* AVN */
pkg_cstate_limits = slv_pkg_cstate_limits;
break;
case INTEL_FAM6_ATOM_AIRMONT: /* AMT */
pkg_cstate_limits = phi_pkg_cstate_limits;
break;
case INTEL_FAM6_ATOM_GOLDMONT: /* BXT */
- case INTEL_FAM6_ATOM_GEMINI_LAKE:
- case INTEL_FAM6_ATOM_DENVERTON: /* DNV */
+ case INTEL_FAM6_ATOM_GOLDMONT_PLUS:
+ case INTEL_FAM6_ATOM_GOLDMONT_X: /* DNV */
pkg_cstate_limits = bxt_pkg_cstate_limits;
break;
default:
return 0;
switch (model) {
- case INTEL_FAM6_ATOM_SILVERMONT1:
- case INTEL_FAM6_ATOM_MERRIFIELD:
- case INTEL_FAM6_ATOM_MOOREFIELD:
+ case INTEL_FAM6_ATOM_SILVERMONT:
+ case INTEL_FAM6_ATOM_SILVERMONT_MID:
+ case INTEL_FAM6_ATOM_AIRMONT_MID:
return 1;
}
return 0;
return 0;
switch (model) {
- case INTEL_FAM6_ATOM_DENVERTON:
+ case INTEL_FAM6_ATOM_GOLDMONT_X:
return 1;
}
return 0;
return ((msr >> 0) & RAPL_POWER_GRANULARITY) * rapl_power_units;
switch (model) {
- case INTEL_FAM6_ATOM_SILVERMONT1:
- case INTEL_FAM6_ATOM_SILVERMONT2:
+ case INTEL_FAM6_ATOM_SILVERMONT:
+ case INTEL_FAM6_ATOM_SILVERMONT_X:
return 30.0;
default:
return 135.0;
}
break;
case INTEL_FAM6_ATOM_GOLDMONT: /* BXT */
- case INTEL_FAM6_ATOM_GEMINI_LAKE:
+ case INTEL_FAM6_ATOM_GOLDMONT_PLUS:
do_rapl = RAPL_PKG | RAPL_PKG_POWER_INFO;
if (rapl_joules)
BIC_PRESENT(BIC_Pkg_J);
BIC_PRESENT(BIC_RAMWatt);
}
break;
- case INTEL_FAM6_ATOM_SILVERMONT1: /* BYT */
- case INTEL_FAM6_ATOM_SILVERMONT2: /* AVN */
+ case INTEL_FAM6_ATOM_SILVERMONT: /* BYT */
+ case INTEL_FAM6_ATOM_SILVERMONT_X: /* AVN */
do_rapl = RAPL_PKG | RAPL_CORES;
if (rapl_joules) {
BIC_PRESENT(BIC_Pkg_J);
BIC_PRESENT(BIC_CorWatt);
}
break;
- case INTEL_FAM6_ATOM_DENVERTON: /* DNV */
+ case INTEL_FAM6_ATOM_GOLDMONT_X: /* DNV */
do_rapl = RAPL_PKG | RAPL_DRAM | RAPL_DRAM_POWER_INFO | RAPL_DRAM_PERF_STATUS | RAPL_PKG_PERF_STATUS | RAPL_PKG_POWER_INFO | RAPL_CORES_ENERGY_STATUS;
BIC_PRESENT(BIC_PKG__);
BIC_PRESENT(BIC_RAM__);
return;
rapl_power_units = 1.0 / (1 << (msr & 0xF));
- if (model == INTEL_FAM6_ATOM_SILVERMONT1)
+ if (model == INTEL_FAM6_ATOM_SILVERMONT)
rapl_energy_units = 1.0 * (1 << (msr >> 8 & 0x1F)) / 1000000;
else
rapl_energy_units = 1.0 / (1 << (msr >> 8 & 0x1F));
case INTEL_FAM6_CANNONLAKE_MOBILE: /* CNL */
case INTEL_FAM6_SKYLAKE_X: /* SKX */
case INTEL_FAM6_ATOM_GOLDMONT: /* BXT */
- case INTEL_FAM6_ATOM_GEMINI_LAKE:
- case INTEL_FAM6_ATOM_DENVERTON: /* DNV */
+ case INTEL_FAM6_ATOM_GOLDMONT_PLUS:
+ case INTEL_FAM6_ATOM_GOLDMONT_X: /* DNV */
return 1;
}
return 0;
case INTEL_FAM6_KABYLAKE_DESKTOP: /* KBL */
case INTEL_FAM6_CANNONLAKE_MOBILE: /* CNL */
case INTEL_FAM6_ATOM_GOLDMONT: /* BXT */
- case INTEL_FAM6_ATOM_GEMINI_LAKE:
+ case INTEL_FAM6_ATOM_GOLDMONT_PLUS:
return 1;
}
return 0;
if (!genuine_intel)
return 0;
switch (model) {
- case INTEL_FAM6_ATOM_SILVERMONT1: /* BYT */
- case INTEL_FAM6_ATOM_SILVERMONT2: /* AVN */
+ case INTEL_FAM6_ATOM_SILVERMONT: /* BYT */
+ case INTEL_FAM6_ATOM_SILVERMONT_X: /* AVN */
return 1;
}
return 0;
case INTEL_FAM6_KABYLAKE_DESKTOP: /* KBL */
crystal_hz = 24000000; /* 24.0 MHz */
break;
- case INTEL_FAM6_ATOM_DENVERTON: /* DNV */
+ case INTEL_FAM6_ATOM_GOLDMONT_X: /* DNV */
crystal_hz = 25000000; /* 25.0 MHz */
break;
case INTEL_FAM6_ATOM_GOLDMONT: /* BXT */
- case INTEL_FAM6_ATOM_GEMINI_LAKE:
+ case INTEL_FAM6_ATOM_GOLDMONT_PLUS:
crystal_hz = 19200000; /* 19.2 MHz */
break;
default:
while (i++ % line_size)
printf("__ ");
}
- printf(" | "); /* right close */
+ printf(" |");
while (line < address) {
c = *line++;
- printf("%c", (c < 33 || c == 255) ? 0x2E : c);
+ printf("%c", (c < 32 || c > 126) ? '.' : c);
}
- printf("\n");
+ printf("|\n");
if (length > 0)
printf("%s | ", prefix);
}
src/usbip attach -r localhost -b $busid;
echo "=============================================================="
+# Wait for sysfs file to be updated. Without this sleep, usbip port
+# shows no imported devices.
+sleep 3;
+
echo "List imported devices - expect to see imported devices";
src/usbip port;
echo "=============================================================="
--- /dev/null
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test synthetic_events syntax parser
+
+do_reset() {
+ reset_trigger
+ echo > set_event
+ clear_trace
+}
+
+fail() { #msg
+ do_reset
+ echo $1
+ exit_fail
+}
+
+if [ ! -f set_event ]; then
+ echo "event tracing is not supported"
+ exit_unsupported
+fi
+
+if [ ! -f synthetic_events ]; then
+ echo "synthetic event is not supported"
+ exit_unsupported
+fi
+
+reset_tracer
+do_reset
+
+echo "Test synthetic_events syntax parser"
+
+echo > synthetic_events
+
+# synthetic event must have a field
+! echo "myevent" >> synthetic_events
+echo "myevent u64 var1" >> synthetic_events
+
+# synthetic event must be found in synthetic_events
+grep "myevent[[:space:]]u64 var1" synthetic_events
+
+# it is not possible to add same name event
+! echo "myevent u64 var2" >> synthetic_events
+
+# Non-append open will cleanup all events and add new one
+echo "myevent u64 var2" > synthetic_events
+
+# multiple fields with different spaces
+echo "myevent u64 var1; u64 var2;" > synthetic_events
+grep "myevent[[:space:]]u64 var1; u64 var2" synthetic_events
+echo "myevent u64 var1 ; u64 var2 ;" > synthetic_events
+grep "myevent[[:space:]]u64 var1; u64 var2" synthetic_events
+echo "myevent u64 var1 ;u64 var2" > synthetic_events
+grep "myevent[[:space:]]u64 var1; u64 var2" synthetic_events
+
+# test field types
+echo "myevent u32 var" > synthetic_events
+echo "myevent u16 var" > synthetic_events
+echo "myevent u8 var" > synthetic_events
+echo "myevent s64 var" > synthetic_events
+echo "myevent s32 var" > synthetic_events
+echo "myevent s16 var" > synthetic_events
+echo "myevent s8 var" > synthetic_events
+
+echo "myevent char var" > synthetic_events
+echo "myevent int var" > synthetic_events
+echo "myevent long var" > synthetic_events
+echo "myevent pid_t var" > synthetic_events
+
+echo "myevent unsigned char var" > synthetic_events
+echo "myevent unsigned int var" > synthetic_events
+echo "myevent unsigned long var" > synthetic_events
+grep "myevent[[:space:]]unsigned long var" synthetic_events
+
+# test string type
+echo "myevent char var[10]" > synthetic_events
+grep "myevent[[:space:]]char\[10\] var" synthetic_events
+
+do_reset
+
+exit 0
}
}
-static struct rlimit rlim_old, rlim_new;
+static struct rlimit rlim_old;
static __attribute__((constructor)) void main_ctor(void)
{
getrlimit(RLIMIT_MEMLOCK, &rlim_old);
- rlim_new.rlim_cur = rlim_old.rlim_cur + (1UL << 20);
- rlim_new.rlim_max = rlim_old.rlim_max + (1UL << 20);
- setrlimit(RLIMIT_MEMLOCK, &rlim_new);
+
+ if (rlim_old.rlim_cur != RLIM_INFINITY) {
+ struct rlimit rlim_new;
+
+ rlim_new.rlim_cur = rlim_old.rlim_cur + (1UL << 20);
+ rlim_new.rlim_max = rlim_old.rlim_max + (1UL << 20);
+ setrlimit(RLIMIT_MEMLOCK, &rlim_new);
+ }
}
static __attribute__((destructor)) void main_dtor(void)
-#!/bin/sh
+#!/bin/bash
#
# This test is for checking rtnetlink callpaths, and get as much coverage as possible.
#
-#!/bin/sh
+#!/bin/bash
# SPDX-License-Identifier: GPL-2.0
#
# Run a series of udpgso benchmarks
parse-build.sh $resdir/Make.out $title
else
# Build failed.
- cp $builddir/Make*.out $resdir
cp $builddir/.config $resdir || :
echo Build failed, not running KVM, see $resdir.
if test -f $builddir.wait
TREE03
TREE04
TREE05
-TREE06
TREE07
-TREE08
TREE09
SRCU-N
SRCU-P
rcutorture.torture_type=srcud
+rcupdate.rcu_self_test=1
rcutorture.torture_type=srcud
+rcupdate.rcu_self_test=1
rcupdate.rcu_self_test=1
-rcupdate.rcu_self_test_bh=1
-rcutorture.torture_type=rcu_bh
-rcutorture.torture_type=rcu_bh maxcpus=8 nr_cpus=43
+maxcpus=8 nr_cpus=43
rcutree.gp_preinit_delay=3
rcutree.gp_init_delay=3
rcutree.gp_cleanup_delay=3
-rcutorture.torture_type=rcu_bh rcutree.rcu_fanout_leaf=4 nohz_full=1-7
+rcutree.rcu_fanout_leaf=4 nohz_full=1-7
-rcutorture.torture_type=sched
-rcupdate.rcu_self_test_sched=1
rcutree.gp_preinit_delay=3
rcutree.gp_init_delay=3
rcutree.gp_cleanup_delay=3
+rcupdate.rcu_self_test=1
rcupdate.rcu_self_test=1
-rcupdate.rcu_self_test_bh=1
-rcupdate.rcu_self_test_sched=1
rcutree.rcu_fanout_exact=1
rcutree.gp_preinit_delay=3
rcutree.gp_init_delay=3
-rcutorture.torture_type=sched
rcupdate.rcu_self_test=1
-rcupdate.rcu_self_test_sched=1
rcutree.rcu_fanout_exact=1
rcu_nocbs=0-7
static void update_vttbr(struct kvm *kvm)
{
phys_addr_t pgd_phys;
- u64 vmid;
+ u64 vmid, cnp = kvm_cpu_has_cnp() ? VTTBR_CNP_BIT : 0;
bool new_gen;
read_lock(&kvm_vmid_lock);
pgd_phys = virt_to_phys(kvm->arch.pgd);
BUG_ON(pgd_phys & ~VTTBR_BADDR_MASK);
vmid = ((u64)(kvm->arch.vmid) << VTTBR_VMID_SHIFT) & VTTBR_VMID_MASK(kvm_vmid_bits);
- kvm->arch.vttbr = kvm_phys_to_vttbr(pgd_phys) | vmid;
+ kvm->arch.vttbr = kvm_phys_to_vttbr(pgd_phys) | vmid | cnp;
write_unlock(&kvm_vmid_lock);
}
git.samba.org / sfrench / cifs-2.6.git / commitdiff