armv8-m: userspace: syscall return sequence needs to be atomic

[ioannisg]

Describe the bug
The sequence of operations when returning from a Cortex-M system call is as follows:

1. Stack Limit Register is cleared (if we are on ARMv8-M)
2. Stack Pointer is set back to nPRIV (user) stack
3. thread.mode variable is set to nPRIV
4. CONTROL register nPRIV bit is set

The above sequence is executed in thread PRIV mode (cannot be executed in nPRIV mode, actually, as it leads to fault).

Now, if swap occurs between (1) and (2), or between (2) and (3) the PSPLIM will be set again to guard the privilege thread stack. This will cause a fault when the thread attempts to push to the user stack, which lies below the PRIV thread stack.

Note: ARMv7-M does not have this problem because it does not have stack pointer limit registers, so only steps (2), (3), (4) are executed.

For example: if swap occurs between (2) and (4) the processor returns to nPRIV mode, but the attempt to (over)write to the CONTROL Register is silently ignored; this is not a problem since the value is the correct one.

Impact
Undefined behavior and possible processor fault during system calls.

Solution

We cannot reverse the order of operations, because the PSP, PSPLIM cannot be written from nPRIV mode.

As a result, we must ensure SWAP cannot occur during this sequence of operations.

[ioannisg]

FYI @andrewboie @jhedberg