linux-stable/drivers/clocksource
Samuel Holland e19ca3fe6c clocksource/drivers/arch_timer: Workaround for Allwinner A64 timer instability
commit c950ca8c35 upstream.

The Allwinner A64 SoC is known[1] to have an unstable architectural
timer, which manifests itself most obviously in the time jumping forward
a multiple of 95 years[2][3]. This coincides with 2^56 cycles at a
timer frequency of 24 MHz, implying that the time went slightly backward
(and this was interpreted by the kernel as it jumping forward and
wrapping around past the epoch).

Investigation revealed instability in the low bits of CNTVCT at the
point a high bit rolls over. This leads to power-of-two cycle forward
and backward jumps. (Testing shows that forward jumps are about twice as
likely as backward jumps.) Since the counter value returns to normal
after an indeterminate read, each "jump" really consists of both a
forward and backward jump from the software perspective.

Unless the kernel is trapping CNTVCT reads, a userspace program is able
to read the register in a loop faster than it changes. A test program
running on all 4 CPU cores that reported jumps larger than 100 ms was
run for 13.6 hours and reported the following:

 Count | Event
-------+---------------------------
  9940 | jumped backward      699ms
   268 | jumped backward     1398ms
     1 | jumped backward     2097ms
 16020 | jumped forward       175ms
  6443 | jumped forward       699ms
  2976 | jumped forward      1398ms
     9 | jumped forward    356516ms
     9 | jumped forward    357215ms
     4 | jumped forward    714430ms
     1 | jumped forward   3578440ms

This works out to a jump larger than 100 ms about every 5.5 seconds on
each CPU core.

The largest jump (almost an hour!) was the following sequence of reads:
    0x0000007fffffffff → 0x00000093feffffff → 0x0000008000000000

Note that the middle bits don't necessarily all read as all zeroes or
all ones during the anomalous behavior; however the low 10 bits checked
by the function in this patch have never been observed with any other
value.

Also note that smaller jumps are much more common, with backward jumps
of 2048 (2^11) cycles observed over 400 times per second on each core.
(Of course, this is partially explained by lower bits rolling over more
frequently.) Any one of these could have caused the 95 year time skip.

Similar anomalies were observed while reading CNTPCT (after patching the
kernel to allow reads from userspace). However, the CNTPCT jumps are
much less frequent, and only small jumps were observed. The same program
as before (except now reading CNTPCT) observed after 72 hours:

 Count | Event
-------+---------------------------
    17 | jumped backward      699ms
    52 | jumped forward       175ms
  2831 | jumped forward       699ms
     5 | jumped forward      1398ms

Further investigation showed that the instability in CNTPCT/CNTVCT also
affected the respective timer's TVAL register. The following values were
observed immediately after writing CNVT_TVAL to 0x10000000:

 CNTVCT             | CNTV_TVAL  | CNTV_CVAL          | CNTV_TVAL Error
--------------------+------------+--------------------+-----------------
 0x000000d4a2d8bfff | 0x10003fff | 0x000000d4b2d8bfff | +0x00004000
 0x000000d4a2d94000 | 0x0fffffff | 0x000000d4b2d97fff | -0x00004000
 0x000000d4a2d97fff | 0x10003fff | 0x000000d4b2d97fff | +0x00004000
 0x000000d4a2d9c000 | 0x0fffffff | 0x000000d4b2d9ffff | -0x00004000

The pattern of errors in CNTV_TVAL seemed to depend on exactly which
value was written to it. For example, after writing 0x10101010:

 CNTVCT             | CNTV_TVAL  | CNTV_CVAL          | CNTV_TVAL Error
--------------------+------------+--------------------+-----------------
 0x000001ac3effffff | 0x1110100f | 0x000001ac4f10100f | +0x1000000
 0x000001ac40000000 | 0x1010100f | 0x000001ac5110100f | -0x1000000
 0x000001ac58ffffff | 0x1110100f | 0x000001ac6910100f | +0x1000000
 0x000001ac66000000 | 0x1010100f | 0x000001ac7710100f | -0x1000000
 0x000001ac6affffff | 0x1110100f | 0x000001ac7b10100f | +0x1000000
 0x000001ac6e000000 | 0x1010100f | 0x000001ac7f10100f | -0x1000000

I was also twice able to reproduce the issue covered by Allwinner's
workaround[4], that writing to TVAL sometimes fails, and both CVAL and
TVAL are left with entirely bogus values. One was the following values:

 CNTVCT             | CNTV_TVAL  | CNTV_CVAL
--------------------+------------+--------------------------------------
 0x000000d4a2d6014c | 0x8fbd5721 | 0x000000d132935fff (615s in the past)
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>

========================================================================

Because the CPU can read the CNTPCT/CNTVCT registers faster than they
change, performing two reads of the register and comparing the high bits
(like other workarounds) is not a workable solution. And because the
timer can jump both forward and backward, no pair of reads can
distinguish a good value from a bad one. The only way to guarantee a
good value from consecutive reads would be to read _three_ times, and
take the middle value only if the three values are 1) each unique and
2) increasing. This takes at minimum 3 counter cycles (125 ns), or more
if an anomaly is detected.

However, since there is a distinct pattern to the bad values, we can
optimize the common case (1022/1024 of the time) to a single read by
simply ignoring values that match the error pattern. This still takes no
more than 3 cycles in the worst case, and requires much less code. As an
additional safety check, we still limit the loop iteration to the number
of max-frequency (1.2 GHz) CPU cycles in three 24 MHz counter periods.

For the TVAL registers, the simple solution is to not use them. Instead,
read or write the CVAL and calculate the TVAL value in software.

Although the manufacturer is aware of at least part of the erratum[4],
there is no official name for it. For now, use the kernel-internal name
"UNKNOWN1".

[1]: https://github.com/armbian/build/commit/a08cd6fe7ae9
[2]: https://forum.armbian.com/topic/3458-a64-datetime-clock-issue/
[3]: https://irclog.whitequark.org/linux-sunxi/2018-01-26
[4]: https://github.com/Allwinner-Homlet/H6-BSP4.9-linux/blob/master/drivers/clocksource/arm_arch_timer.c#L272

Acked-by: Maxime Ripard <maxime.ripard@bootlin.com>
Tested-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Samuel Holland <samuel@sholland.org>
Cc: stable@vger.kernel.org
Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-03-23 20:09:58 +01:00
..
acpi_pm.c clocksource: Use a plain u64 instead of cycle_t 2016-12-25 11:04:12 +01:00
arc_timer.c clocksource/drivers/arc_timer: Utilize generic sched_clock 2019-01-09 17:38:42 +01:00
arm_arch_timer.c clocksource/drivers/arch_timer: Workaround for Allwinner A64 timer instability 2019-03-23 20:09:58 +01:00
arm_global_timer.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
armv7m_systick.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
asm9260_timer.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
bcm2835_timer.c locking/atomics: COCCINELLE/treewide: Convert trivial ACCESS_ONCE() patterns to READ_ONCE()/WRITE_ONCE() 2017-10-25 11:01:08 +02:00
bcm_kona_timer.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
cadence_ttc_timer.c Merge branch 'timers/urgent' into timers/core 2017-06-21 09:07:52 +02:00
clksrc-dbx500-prcmu.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
clksrc_st_lpc.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
clps711x-timer.c clocksource/drivers: Rename CLKSRC_OF to TIMER_OF 2017-06-14 12:01:03 +02:00
cs5535-clockevt.c clockevents/drivers/cs5535: Improve resilience to spurious interrupts 2017-10-20 13:41:52 +02:00
dummy_timer.c cpu/hotplug: Cleanup state names 2016-12-25 10:47:44 +01:00
dw_apb_timer.c Merge branch 'clockevents/4.12' of https://git.linaro.org/people/daniel.lezcano/linux into timers/core 2017-04-17 10:55:14 +02:00
dw_apb_timer_of.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
em_sti.c clocksource/drivers/em_sti: Fix error return codes in em_sti_probe() 2017-08-10 14:48:18 +02:00
exynos_mct.c clocksource/drivers/exynos_mct: Clear timer interrupt when shutdown 2019-03-23 20:09:58 +01:00
fsl_ftm_timer.c clocksource/drivers/fsl_ftm_timer: Fix error return checking 2018-02-26 13:56:05 +01:00
h8300_timer8.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
h8300_timer16.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
h8300_tpu.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
i8253.c clockevents/drivers/i8253: Add support for PIT shutdown quirk 2018-11-21 09:19:20 +01:00
jcore-pit.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
Kconfig clocksource/drivers/arch_timer: Workaround for Allwinner A64 timer instability 2019-03-23 20:09:58 +01:00
Makefile RISC-V Updates for the 4.19 Merge Window 2018-08-19 09:56:38 -07:00
meson6_timer.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
mips-gic-timer.c clocksource/drivers/mips-gic-timer: Add pr_fmt and reword pr_* messages 2018-05-18 22:53:09 +02:00
mmio.c clocksource: Use a plain u64 instead of cycle_t 2016-12-25 11:04:12 +01:00
mps2-timer.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
mxs_timer.c clocksource/drivers/mxs_timer: Switch to SPDX identifier 2018-05-23 07:39:09 +02:00
nomadik-mtu.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
numachip.c x86/numachip: Add const and __initconst to numachip2_clockevent 2017-09-25 09:36:15 +02:00
owl-timer.c clocksource/drivers/owl: Add the S700 timer 2018-01-08 17:57:23 +01:00
pxa_timer.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
qcom-timer.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
renesas-ostm.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
riscv_timer.c clocksource: new RISC-V SBI timer driver 2018-08-13 08:31:31 -07:00
rockchip_timer.c clocksource/drivers/rockchip: pr_err() strings should end with newlines 2017-10-19 23:49:02 +02:00
samsung_pwm_timer.c clocksource/drivers: Rename CLKSRC_OF to TIMER_OF 2017-06-14 12:01:03 +02:00
scx200_hrt.c clocksource: Use a plain u64 instead of cycle_t 2016-12-25 11:04:12 +01:00
sh_cmt.c treewide: kzalloc() -> kcalloc() 2018-06-12 16:19:22 -07:00
sh_mtu2.c treewide: kzalloc() -> kcalloc() 2018-06-12 16:19:22 -07:00
sh_tmu.c treewide: kzalloc() -> kcalloc() 2018-06-12 16:19:22 -07:00
sun4i_timer.c clocksource/drivers/sun4i: Switch to the timer-of common init 2017-06-26 18:32:04 +02:00
tango_xtal.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
tcb_clksrc.c clocksource/drivers/tcb_clksrc: Fix clock speed message 2018-01-08 17:57:23 +01:00
tegra20_timer.c Merge branch 'x86-timers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-08-13 18:28:19 -07:00
time-armada-370-xp.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
time-efm32.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
time-lpc32xx.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
time-orion.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
time-pistachio.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
timer-atcpit100.c clocksource/drivers: Set clockevent device cpumask to cpu_possible_mask 2018-07-26 11:26:30 +02:00
timer-atlas7.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
timer-atmel-pit.c clocksource/drivers/timer-atmel-pit: Properly handle error cases 2018-09-27 12:01:45 +02:00
timer-atmel-st.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
timer-digicolor.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
timer-fttmr010.c clocksource/drivers/fttmr010: Fix set_next_event handler 2018-09-24 06:13:31 +02:00
timer-imx-gpt.c clocksource/drivers/timer-imx-gpt: Switch to SPDX identifier 2018-05-23 07:38:52 +02:00
timer-imx-tpm.c clocksource/drivers/timer-imx-tpm: Switch to SPDX identifier 2018-05-23 07:39:01 +02:00
timer-integrator-ap.c clocksource/drivers/integrator-ap: Add missing of_node_put() 2019-01-26 09:32:42 +01:00
timer-keystone.c clocksource/drivers: Set clockevent device cpumask to cpu_possible_mask 2018-07-26 11:26:30 +02:00
timer-mediatek.c clocksource/drivers/timer-mediatek: Add support for system timer 2018-07-26 11:26:34 +02:00
timer-npcm7xx.c clocksource/drivers/npcm: Add NPCM7xx timer driver 2018-03-30 22:44:09 +02:00
timer-nps.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
timer-of.c clocksource/drivers/timer-of: Don't request the resource by name 2018-01-08 17:57:24 +01:00
timer-of.h clocksource/drivers/timer-of: Store the device node pointer in 'struct timer_of' 2018-01-08 17:57:24 +01:00
timer-oxnas-rps.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
timer-prima2.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
timer-probe.c clocksource: Convert to using %pOF instead of full_name 2017-08-31 15:56:17 +02:00
timer-sp.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
timer-sp804.c ARM: timer-sp: Use of_clk_get_parent_count() instead of open coding 2018-05-02 08:32:57 -07:00
timer-sprd.c clocksource/drivers/sprd: Register one always-on timer to compensate suspend time 2018-07-26 11:26:34 +02:00
timer-stm32.c clocksource/drivers/stm32: Fix error return code 2018-06-12 23:54:01 +02:00
timer-sun5i.c treewide/trivial: Remove ';;$' typo noise 2018-02-22 10:59:33 +01:00
timer-ti-32k.c clocksource/drivers/ti-32k: Add CLOCK_SOURCE_SUSPEND_NONSTOP flag for non-am43 SoCs 2018-09-13 15:23:35 +02:00
timer-ti-dm.c clocksource: timer-ti-dm: Fix pwm dmtimer usage of fck reparenting 2019-03-23 20:09:42 +01:00
timer-u300.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
versatile.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
vf_pit_timer.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
vt8500_timer.c clocksource/drivers: Rename CLOCKSOURCE_OF_DECLARE to TIMER_OF_DECLARE 2017-06-14 11:58:45 +02:00
zevio-timer.c clocksource/drivers: Set clockevent device cpumask to cpu_possible_mask 2018-07-26 11:26:30 +02:00