HIL testing

Development archive
1

Hi everyone,

in today's developer meeting the discussion passed by the topic of HIL testing once again, this time in the context of the upcoming re-modeling of the I2C interface. Some weeks ago I had an idea for a simple HIL setup for automatically testing perihp/i2c drivers, and I promised to share it (although it is in very very early state). So here it is.

So the basic idea is to connect the I2C pins of the 'board-under-test' (i2c master for now) to a 2nd board (lets call it 'test peer'), running a fully controllable software i2c slave implementation. On the device-under-test we simply run the 'tests/periph_i2c' app, that lets us control the I2C master using shell commands (init/read/write incl parameters). On the 'test peer' we run a programmable soft i2c client, that offers shell commands expressing expectations, e.g. "expect 4 byter [abcd] written to addr 0x23".

Now for creating an automated I2C test-suite, I imagine something similar to our existing pyexpect scripts (testrunner), just with the added capability of handling two clients. So the test script could look something like this (mostly pseudocode...):

`PORT=/dev/ttyACM0 TESTPEER=/dev/ttyACM1 make hil-test`

triggers:

...
def testfunc(testie, peer):
     ...
    # tell the 'test peer' to expect something to be written to addr 0x23 and respond with a NACK
     peer.sendline("expect_addr 0x23")
    # write a random byte the addr 0x23
     testie.sendline("w 0x23 a")
     # the 'test peer' will tell if it sees the expected address
     peer.expect("[SUCCESS]")
     # the 'device-under-test' should recognize the NACK
     testie.expect("error: no device with that address found on bus")

     # test writing (expect 4 bytes [affe] written to address 0x42)
     peer.sendline("expect_write 0x42 a f f e")
     testie.sendline("w 0x42 a f f e")
     peer.expect("[SUCCESS]")
     testie.expect("I2C_DEV(0): successfully wrote 4 bytes to the bus\n")

    ...
     # many many more test cases here, including simulation of badly behaving slaves and bus recovery...

I have sketched a I2C slave implementation to run on the 'test peer' in [1] (under `tests/softi2ccli`). It is in no means production code, but a quick prove of concept. But IMHO this would be a very easy and straight forward way to have an automatic test-suite that would work on someones desk for now.

Integration into the bigger picture would hopefully be easy doable, but this not in the scope of this particular proposal here... For now my goal would simply be to build a test suite for assisting with the upcoming I2C remodeling to get some hands-on experience and speed up the testing efforts for that task.

Cheers, Hauke

[1] https://github.com/haukepetersen/RIOT/tree/add_softi2cli

2

Hi Hauke,

thanks for you efforts! This is great and really useful work. IMO it is the right direction, even though I always hoped to find a way around implementing our own I2C slave device. That's why I wanted to take this opportunity to ask around for other experiences with I2C (slave) test devices. Anyone, anything?

Best Peter

3

Hi Peter,

I think definition/implementation of a I2C slave interface/drivers is still wanted. But I found, that for the presented HIL use-case, a custom pin-banging i2c slave mock-up implementation is better suited, as it further allows us to intentionally trigger faulty bus behavior (not releasing the clock line, etc), which gives us even better control than using a hardware I2C slave unit for the test peer. When getting to testing the actual I2C slave implementations at some point, we would IMHO then also need a pin-banging version of a I2C master for testing...

But before jumping into the I2C slave discussion (and same goes of course for the SPI slave), I think it makes sense to lead all our efforts into the I2C master API remodeling first.

Cheers, Hauke