Commit f45709df77 ("serial: 8250: Don't touch RTS modem control while
in rs485 mode") sought to prevent user space from interfering with rs485
communication by ignoring a TIOCMSET ioctl() which changes RTS polarity.
It did so in serial8250_do_set_mctrl(), which turns out to be too deep
in the call stack: When a uart_port is opened, RTS polarity is set by
the rs485-aware function uart_port_dtr_rts(). It calls down to
serial8250_do_set_mctrl() and that particular RTS polarity change should
*not* be ignored.
The user-visible result is that on 8250_omap ports which use rs485 with
inverse polarity (RTS bit in MCR register is 1 to receive, 0 to send),
a newly opened port initially sets up RTS for sending instead of
receiving. That's because omap_8250_startup() sets the cached value
up->mcr to 0 and omap_8250_restore_regs() subsequently writes it to the
MCR register. Due to the commit, serial8250_do_set_mctrl() preserves
that incorrect register value:
do_sys_openat2
do_filp_open
path_openat
vfs_open
do_dentry_open
chrdev_open
tty_open
uart_open
tty_port_open
uart_port_activate
uart_startup
uart_port_startup
serial8250_startup
omap_8250_startup # up->mcr = 0
uart_change_speed
serial8250_set_termios
omap_8250_set_termios
omap_8250_restore_regs
serial8250_out_MCR # up->mcr written
tty_port_block_til_ready
uart_dtr_rts
uart_port_dtr_rts
serial8250_set_mctrl
omap8250_set_mctrl
serial8250_do_set_mctrl # mcr[1] = 1 ignored
Fix by intercepting RTS changes from user space in uart_tiocmset()
instead.
Link: https://lore.kernel.org/linux-serial/20211027111644.1996921-1-baocheng.su@siemens.com/
Fixes: f45709df77 ("serial: 8250: Don't touch RTS modem control while in rs485 mode")
Cc: Chao Zeng <chao.zeng@siemens.com>
Cc: stable@vger.kernel.org # v5.7+
Reported-by: Su Bao Cheng <baocheng.su@siemens.com>
Reported-by: Jan Kiszka <jan.kiszka@siemens.com>
Tested-by: Su Bao Cheng <baocheng.su@siemens.com>
Signed-off-by: Lukas Wunner <lukas@wunner.de>
Link: https://lore.kernel.org/r/21170e622a1aaf842a50b32146008b5374b3dd1d.1637596432.git.lukas@wunner.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
f85e04503f