Add "ready" signal to incoming data

[tomeroberts]

As part of the icache development work, it was proposed to add a ready signal from the core to stall incoming data on instr_rdata_i. Creating this issue to gather thoughts on whether this would be a good idea.

Upsides

• Would allow an area reduction in the icache (and also probably in the normal prefetch buffer) by reducing the required amount of data buffering.
• Would be a good way of fixing Fetch FIFO blocks instruction request when full. #574.

Downsides

• Changes pin compatibility at the top level.
• Would be annoying / painful to add as a parameterized option.
• As noted by @rswarbrick back-pressure could have an adverse effect on system performance where there is system bus contention.

Compatibility

• Easily compatible with TL-UL interfaces, which already have that signal.
• Easy enough for AXI which already has similar signalling.
• Should be doable with AHB.
• Wishbone I think should support this with it's retry signaling.

[imphil]

Sounds reasonable to me. How does that relate to the concerns others had about our bus interface?
@davideschiavone for the wider picture on the other implementations using this style of bus
@eunchan for TL-UL

[GregAC]

I would note there's a credit based flow control discussion on the OT repo: lowRISC/opentitan#1644

Credit based could be nicer but again makes the interface more complex.

[imphil]

@Silabs-ArjanB just posted a doc to a better specified/revised version of the bus standard we use at https://github.com/openhwgroup/core-v-docs/blob/master/cores/cv32e40p/OBI-v1.0.pdf, and it contains a rready signal. That's what you're looking for here as well, @tomroberts-lowrisc?

[Silabs-ArjanB]

To me rready seems a logical addition (and can indeed be used to safe buffering space). If not implemented (as on RI5CY/CV32E40P) then its value is assumed 1'b1 always. Of course if a CPU uses such an rready signal, then it puts a not-backward compatible requirement on its environment (and therefore it would be nice if such addition would be parameterized).

[eunchan]

I always prefer sending request only when the requester (TL master) has enough space to accept the request to sending predictably. It needs more space in case of DMA engine But it prevents any deadlock issue. I'd rather have bigger core area to be safe.

[GregAC]

There are some wider questions around Ibex's memory interfacing to resolve, this can be considered as part of them. However for the foreseeable future the top-level interface will remain as it is.