Complex behaving Hardware with CircuitPython - Part IV

The first three parts of this guide dealt with the development of software to control hardware that is more complex and demanding than typical hobbyist projects.

With software of this kind, it isn't obvious that it works correctly. Therefore, testing is essential - but manual testing is a lot of work. So let's automate it.

That raises two problems - how to specify tests and how to simulate hardware.

Specifying Tests with Gherkin

Gherkin is a language for specifying test cases from the perspective of software behavior.

Gherkin divides test cases into features that consist of one or more scenarios. Each scenario contains multiple test steps. These describe the initial situation (using the Given keyword) and then interactions with the software (When keyword) and their expected results (Then keyword).

Here is a simple Gherkin feature testing a requirement from the example from the third part of this guide ("If the cart is moving from one end position to the other and does not reach the destination after 15 seconds, the motor should stop.").

Feature: Example 3 fail

  Scenario: Failure on going left

        # in between
    Given the analog input A4 has the value 20000
      And the analog input A5 has the value 20000
      And the digital input A6 has the value True
      And the program in the file "transfer_machine.py" is started
     When 100 milliseconds elapse
        # drive to left end position
     Then the digital output A1 has the value True
      And the digital output A2 has the value True
      And the digital output A3 has the value False
     When 16 seconds elapse
        # stop
     Then the digital output A1 has the value False
      And the digital output A2 has the value False
      And the digital output A3 has the value False
      And the program terminated

How does Gherkin know how to test CircuitPython programs using the cooperative_multitasking library? It doesn't. The test steps must be created individually. This is where the simulator comes into play.

Simulating Hardware

The simulator implements the CircuitPython APIs so that test steps can set and check simulated hardware states. All of this is interleaved with the execution of the CircuitPython program.

Here is a list of possible test steps.

Set up the initial state of your hardware

• Given the analog input {name} has the value {value}
• Given the digital input {name} has the value True
• Given the digital input {name} has the value False

Specify the CircuitPython file that should be tested

• Given the program in the file "{circuit_python_file_name}" is started

Change the state of your hardware

• When the analog input {name} changes its value to {value}
• When the digital input {name} changes its value to True
• When the digital input {name} changes its value to False

Run the CircuitPython program for a specified time

• When {value} milliseconds elapse
• When {value} seconds elapse
• When {value} minutes elapse

Check the expected state of your hardware

• Then the digital output {name} has the value True
• Then the digital output {name} has the value False
• Then the {index}. of the neopixels {name} shows the color {expected_value}
• Then the neopixel {name} shows the color {expected_value}
• Then the neopixels {name} have the brightness {expected_value}
• Then the neopixel {name} has the brightness {expected_value}
• Then the {index}. of the dotstars ({name_1}, {name_2}) shows the color {expected_value}
• Then the dotstar ({name_1}, {name_2}) shows the color {expected_value}
• Then the dotstars ({name_1}, {name_2}) have the brightness {expected_brightness}
• Then the dotstar ({name_1}, {name_2}) has the brightness {expected_brightness}

Check exceptions or termination of your software

• Then the program terminated

Abstraction and repetition of steps

• Given the steps in the file "{file_name}" are executed
• When the steps in the file "{file_name}" are executed
• When the steps in the file "{file_name}" are repeated {count} times

Try it yourself

There are at least two possible ways in using the CircuitPython simulator.

The repository https://bitbucket.org/amotzek/circuit-python/src/master/ uses the Docker image amotzek/circuit-python-simulator in its pipeline to test some example files. See the bitbucket-pipelines.yml file to see how this works.

If you want to use the simulator locally:

• Make sure you have Python 3 installed.
• Install Behave with pip3 install behave.
• Download features.zip from the code section of this project.
• Create a directory for your tests and unzip features.zip into it. Your directory will now have a subdirectory named features.
• Write your own test cases and place your feature files into the features subdirectory.
• Open a shell and navigate to your test directory.
• If you execute behave, it will run your tests.

What's next

Gherkin has more functionality than was mentioned here. The complete language is described in the relevant parts of the Behave documentation or the Cucumber documentation.

The simulator only knows about AnalogIn, board, DigitalInOut, DotStar and NeoPixel, and it only works if the cooperative_multitasking library is used - because that's what I need. If you want to simulate and test other hardware, you will have to implement the additional classes and step definitions.