Wishbone constraint random testing

[christiaanb]

This issue is meant to start a discussion, not as a way to dictate how things should be.

Wishbone master testing

In full wishbone, a slave can respond to a master in four ways:

1. ACK, complete the request
2. ERR, indicate the request is faulty
3. RTY (optional), indicates the device is not ready and the cycle should be retried
4. STALL (optional), stop taking in new requests

This means a slave can exist in four configurations:

• Minimal: ACK and ERR
• RTY only: ACK, ERR, and RTY
• Stall only: ACK, ERR, and STALL
• Full: ACK, ERR, RTY, and STALL

import Data.Type.Bool
import Data.Type.Equality

data SlaveConf = Minimal | RTY_Only | Stall_Only | Full

data SSlaveConf (conf :: SlaveConf) where
  SMinimal :: SSlaveConf 'Minimal
  SRTY_Only :: SSlaveConf 'RTY_Only
  SStall_Only :: SSlaveConf 'Stall_Only
  SFull :: SSlaveConf 'Full

data WishboneS2M (conf :: SlaveConf) bytes =
  WishboneM2S
    { dat_o :: BitVector (8 * bytes)
    , ack_o :: Bool
    , err_o :: Bool
    , rty_o :: If (conf == 'RTY_Only || conf == 'Full) Bool ()
    , stall_o :: If (conf == 'Stall_Only || conf == 'Full) Bool ()
    }

data Wait = Stall Word | DelayResp Word

genWaitCycles :: SSlaveConf conf -> Gen [Wait]
genWaitCycles = -- only generate stalls for slaves that support them

data M2SResp = ACK | ERR | RTY

m2sResp :: SlaveConf conf -> [Wait] -> Gen [M2SResp]
m2sResp = 
  -- generate valid responses according to the slave configuration
  --
  -- To figure out:
  -- When a slave supports stalls, and we delay our response
  -- i.e. accept transactions, do we then need to ACK them all?
  -- or can some be ERR or RTY?


wishBoneSlave ::
  -- | Slave configuration (fixed/given)
  SSlaveConf conf ->
  -- | Responses to CYC + STB high (generated)
  [M2SResp] ->
  -- | Data to return on ACK high (generated)
  [BitVector (bytes * 8) ->
  -- | Wait cycle duration when CYC + STB are high (generated)
  [Wait] ->
  -- | M2S
  Signal dom (WishboneM2S conf bytes addrWidth) ->
  -- | S2M
  Signal dom (WishboneS2M conf bytes)
wishBoneSlave ... = ...
  -- ACK, RTY, and ERR are `undefined` when there's no active cycle
  --
  -- When the slave supports stalls, and we're told to delay the response
  -- Collect sufficient responses and data

To then generate a constraint random slave I imagine:

createSlave slaveConf genData = do
  waitCycles <- genWaitCycles slaveConf 
  -- Ensure we generate enough responses for the number of wait cycles
  -- in case stalls are supported
  responses <- m2sResp slaveConf waitCyles
  -- Generate the data for all ACKs
  respData <- genData responses
  return (wishBoneSlave slaveConf responses respData waitCycles)

[martijnbastiaan]

I've got a few points of discussion, overall I think this is the way to go. I'll prefix them with a number for ease of communication :-).

1. I had expected to see Circuit in your types. Any reason:

  -- | M2S
  Signal dom (WishboneM2S conf bytes addrWidth) ->
  -- | S2M
  Signal dom (WishboneS2M conf bytes)

isn't:

  Circuit (Wishbone conf bytes addrWidth) ()

?

2. For Df I've written a stall component, which can sit in between two other components. I.e., StallInfo -> Circuit (Df a) (Df a). My gut feeling says it's prettier to separate the "drive" and "stall" concerns into two separate functions, but then again it's probably easier to just write a separate test master / test slave.

3. For Df I wrote type classes which allowed me to implement (a highly generic) propWithModel. In the end, I somewhat regret doing this as the type class hierarchy is complex, probably too Df-specific, hard to implement, and hard to debug. On top of that it doesn't really work with memory mapped components - although this is somewhat simple to fix. What I'm trying to say is: if we're going with the non type class approach for Wishbone, we probably want to use the same test approach for Df.

   • To not completely burn down my own work though :-): it does lead to very readable test cases, its impossible to forget to test certain aspects, and some care has been taken that the error messages it displays clearly indicate what went wrong.

4. With the current type signatures one needs to determine the data to return up front (i.e., in the test case). It might be more natural to test with a stateful function instead (this could be specialized to the one mentioned in OP):

wishBoneSlave ::
  -- | Slave configuration (fixed/given)
  SSlaveConf conf ->
  -- | Responses to CYC + STB high (generated)
  [M2SResp] ->
  -- | Data to return on ACK high
  (WishboneM2S conf bytes addrWidth -> State s (BitVector (bytes * 8))) ->
  -- | Wait cycle duration when CYC + STB are high (generated)
  [Wait] ->
  -- | M2S
  Signal dom (WishboneM2S conf bytes addrWidth) ->
  -- | S2M
  Signal dom (WishboneS2M conf bytes)

5. We should think about reset testing. I believe we should design (most, but almost all) components such that they behave properly when their resets are asserted. Where properly is:
   • They don't generate valid bus cycles (i.e., supply data). [This one is optional, if (2) is followed.]
   • They don't acknowledge bus cycles (i.e., receive data).

[martijnbastiaan]

One more thing I don't immediately find obvious: how would you compose the proposed function to an actual test case? I imagine that -once designs under test start to be more complex- we'd use the Plugin to compose designs. However, using wishBoneSlave would hide all data you'd want to test. I.e., you'd end up with a Circuit () ().