Branch rpi-4.14.y-rt hanging with "Disabling IRQ#59"

[zavorra]

On branch rpi-4.14.y-rt with serial console enabled ( "enable_uart=1" in config.txt ) the pi hangs after some (a few) minute printing "Disabling IRQ#59" on the console.

After some discussion here, with the suggested option "dtoverlay=pi3-disable-bt" the PI was able to run fine for a couple of hours.

[pelwell]

It's a problem with the shared interrupt line for the AUX devices. Reworking the AUX interrupt driver to mask all interrupts when any one is masked may fix it, although it isn't ideal.

[TiejunChina]

We should not thread irq if this irq is shared, so please add this flag, IRQF_NO_THREAD. @pelwell Will you take this?

[pelwell]

I'm working on it now, but stalled because of getting interrupts wrong while working remotely. Will pick this up later.

Lets keep the discussion here rather than duplicating to the Forum.

[TiejunChina]

@pelwell @zavorra Does this work for you?

diff --git a/drivers/spi/spi-bcm2835aux.c b/drivers/spi/spi-bcm2835aux.c
index 7428091d3f5b..2f2c7caa9246 100644
--- a/drivers/spi/spi-bcm2835aux.c
+++ b/drivers/spi/spi-bcm2835aux.c
@@ -484,7 +484,11 @@ static int bcm2835aux_spi_probe(struct platform_device *pdev)
 
        err = devm_request_irq(&pdev->dev, bs->irq,
                               bcm2835aux_spi_interrupt,
+#ifdef CONFIG_PREEMPT_RT_FULL
+                              IRQF_SHARED | IRQF_NO_THREAD,
+#else
                               IRQF_SHARED,
+#endif
                               dev_name(&pdev->dev), master);
        if (err) {
                dev_err(&pdev->dev, "could not request IRQ: %d\n", err);
diff --git a/drivers/tty/serial/8250/8250_bcm2835aux.c b/drivers/tty/serial/8250/8250_bcm2835aux.c
index 4f8b8c971868..8cb418c7be48 100644
--- a/drivers/tty/serial/8250/8250_bcm2835aux.c
+++ b/drivers/tty/serial/8250/8250_bcm2835aux.c
@@ -66,6 +66,9 @@ static int bcm2835aux_serial_probe(struct platform_device *pdev)
                return ret;
        }
        data->uart.port.irq = ret;
+#ifdef CONFIG_PREEMPT_RT_FULL
+       data->uart.port.irqflags = IRQF_NO_THREAD;
+#endif
 
        /* map the main registers */
        res = platform_get_resource(pdev, IORESOURCE_MEM, 0);

[zavorra]

I won't have access to my PI until the weekend, sorry

[TiejunChina]

@pelwell Phil, are you still working on this?

[pelwell]

Yes, but it keeps getting preempted by higher priority tasks.

[MaoniinoaM]

Hi, @pelwell,
My task needs run in real time state. So I try to port rpi-4.14.y-rt to raspberry PI 3B. And now I have compiled the kernel and port the kernel to PI as the indication from PI home-page. But when I reboot PI. it prints :
INFO: rcu_preemp detected stalls on CPUs/tasks:
o0-...: (0 ticks this GP) isle=ae2/14000000001/0 softirq=0/0 fqs=0

I have also update the fireware to the newest master branch.
So, please help me figure out what's wrong.
Thank you!

[TiejunChina]

@elitezhu Did you try the latest rpi-4.14.y-rt? I can boot that with my pi 3B.

[MaoniinoaM]

@TiejunChina Oh, I hold the version downloaded one week ago.
So you mean I'd better to have a try the latest version you push 19 hours ago?
Thanks. And keep in touch.

[MaoniinoaM]

@TiejunChina Thanks for your reminding. Now I can port the RT kernel to PI. But my APP can't run in RT kernel successfully. The APP can run in both Ubuntu 16.04 and PI 3B (with normal kernel) normally. Here is the issue:
1->when call socket(), it will fail. And the log shows:
socket () at ../sysdeps/unix/syscall-template.S:84
84 ../sysdeps/unix/syscall-template.S: No such file or directory.

2->the app will break with segment fault. And the log shows:
Thread 7 "PN_LITE" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x74623470 (LWP 7682)]
0x76fbaf4c in memcmp () from /usr/lib/arm-linux-gnueabihf/libarmmem.so

These two issues don't look like my app causes.

Could you give me some help? Thanks!

[MaoniinoaM]

@TiejunChina, Thank you for help. I have closed my issues. Thanks again!

[JamesH65]

What is the current status of this? Its a bit difficult to determine from the email chain.

[TiejunChina]

@JamesH65 I just committed this to work around our current problem but ideally, we should consider implementing threaded handler while registering irq_handler_t thread_fn by means of request_threaded_irq().

[JamesH65]

OK, will leave open for the moment.

[10ne1]

I think commit torvalds/linux@bc519d9 ("spi: bcm2835aux: ensure interrupts are enabled for shared handler") which landed in upstream since 4.17 may fix this.

@pelwell You were added to review on ML and also did the initial upstreaming work for the bcm2835 aux interrupt controller. Can you confirm if this is fixed?

[pablogs9]

I just have compiled the rpi-4.14.y-rt branch at a42048c, and seems to keep warning about Disabling IRQ#59.

I'm using SPI2 for EN28J60 Ethernet driver (this gets blocked when Disabling IRQ#59 appears) and

dtoverlay=uart1,txd1_pin=32,rxd1_pin=33

to have an PPP modem connection. The issue only appears when I use /dev/ttyS0 (called from PPP daemon), if I don't execute the PPP daemon it seems that Disabling IRQ#59 error don't appear.

@10ne1 , torvalds/linux@bc519d9 seems to be implemented in the current stage of the rpi-4.14.y-rt branch.

Please let us know if someone knows what is happening.