Vehicle App Integration Testing

To be sure that a newly created Vehicle App will run together with the KUKSA Data Broker and potentially other dependant Vehicle Services or Vehicle Apps, it’s essential to write integration tests along with developing the app.

To execute an integration test, the dependant components need to be running and be accessible from the test runner. This guide will describe how integration tests can be written and integrated in the CI pipeline so that they are executed automatically when building the application.

Writing Test Cases

To write an integration test, you should check the sample that comes with the template ( /app/tests/integration/integration_test.py ). To support interacting with the MQTT broker and the KUKSA Data Broker (to get and set values for data points), there are two classes present in Python SDK that will help:

• MqttClient: this class provides methods for interacting with the MQTT broker. Currently, the following methods are available:

  • publish_and_wait_for_response: publishes the specified payload to the given request topic and waits (till timeout) for a message to the response topic. The payload of the first message that arrives in the response topic will be returned. If the timeout expires before, an empty string ("") is returned.

  • publish_and_wait_for_property: publishes the specified payload to the given request topic and waits (till timeout) until the given property value is found in an incoming message to the response topic. The path describes the property location within the response message, the value the property value to look for.

    Example:

    {
        "status": "success",
        "result": {
            "responsecode": 10
        }
    }
    

    If the responsecode property should be checked for the value 10, the path would be ["result", "responsecode"], property value would be 10. When the requested value has been found in a response message, the payload of that message will be returned. If the timeout expires before receiving a matching message, an empty string ("") is returned.

  This class can be initialized with a given port. If no port is specified, the environment variable MQTT_PORT will be checked. If this is not possible either, the default value of 1883 will be used. It’s recommended to specify no port when initializing that class as it will locally use the default port 1883 and in CI the port is set by the environment variable MQTT_PORT. This will prevent a check-in in the wrong port during local development.

• IntTestHelper: this class provides functionality to interact with the KUKSA Data Broker.

  • register_datapoint: registers a new data point with given name and type ( here you can find more information about the available types)
  • set_..._datapoint: set the given value for the data point with the given name (with given type). If the data point does not exist, it will be registered.

  This class can be initialized with a given port. If no port is specified, the environment variable VDB_PORT will be checked. If this is not possible either, the default value of 55555 will be used. It’s recommended to specify no port when initializing that class as it will locally use the default port 55555 and in CI the port is set by the environment variable VDB_PORT. This will prevent a check-in in the wrong port during local development.

Please make sure that you don’t check in the test classes with using local ports because then the execution in the CI workflow will fail (as the CI workflow uses Kubernetes execution for running integration tests).

Runtime components

To be able to test the Vehicle App in an integrated way, the following components should be running:

• Dapr
• Mosquitto
• Data Broker
• Vehicle Services

We distinguish between two environments for executing the Vehicle App and the runtime components:

• Local execution: components are running locally in the development environment
• Kubernetes execution: components (and application) are deployed and running in a Kubernetes control plane (e.g. K3D)
• Kanto execution: components (and application) are deployed and running in a Kanto control plane

Local execution

First, make sure that the runtime services are configured and running like described here .

The application itself can be executed by using a Visual Studio Launch Config (by pressing F5) or by executing the provided task Local Runtime - Run VehicleApp.

When the runtime services and the application are running, integration tests can be executed locally via

  pytest ./app/tests/integration

or using the testing tab in the sidebar to the left.

Kanto runtime

First, make sure that the runtime and the services are up and running, like described here .

The application itself can be deployed by executing the provided task Kanto Runtime - Deploy VehicleApp or Kanto Runtime - Deploy VehicleApp (without rebuild). Depending on whether your app is already available as a container or not.

When the runtime services and the application are running, integration tests can be executed locally via

  pytest ./app/tests/integration

or using the testing tab in the sidebar to the left.

Kubernetes execution (K3D)

If you want to execute the integration tests in Kubernetes mode, make sure that K3D is up and running according to the documentation . Local ports for Mosquitto and KUKSA Data Broker are 1883 and 55555. In Kubernetes mode, these ports are exposed as nodeports 31883 and 30555 respectively (see here for details). To ensure that the tests connect to correct ports of the containers, please execute the following commands in a new bash terminal:

  export MQTT_PORT=31883 && export VDB_PORT=30555
  pytest ./app/tests/integration

Integration Tests in CI pipeline

The tests will be discovered and executed automatically in the provided CI pipeline . The job Run Integration Tests contains all steps to set up and execute all integration tests in Kubernetes mode. Basically it is doing the same steps as you saw above:

1. start the K3D runtime
2. deploy the Vehicle App container
3. set the correct MQTT and Databroker ports
4. execute the integration tests

Finally the test results are collected and published as artifacts of the workflow.

Troubleshooting

Check if the services are registered correctly in Dapr

• Dapr extension is available as VS Code extension and gives overview of all running Dapr services.

Troubleshoot IntTestHelper

• Make sure that the KUKSA Data Broker is up and running by checking the task log.
• Make sure that you are using the right ports for local/Kubernetes execution.
• Make sure that you installed the correct version of the SDK (SDV-package).

Troubleshoot Mosquitto (MQTT Broker)

• Make sure that Mosquitto is up and running by checking the task log.
• Make sure that you are using the right ports for local/Kubernetes execution.
• Use VsMqtt extension to connect to MQTT broker locally (localhost:1883) or in a Kubernetes cluster (localhost:31883) to monitor topics in MQTT broker by subscribing to all topics using #.

Next steps

• Concept: Deployment Model
• Concept: Build and release process
• Tutorial: Deploy a Python Vehicle App with Helm