Skip to content
This repository has been archived by the owner on Dec 18, 2024. It is now read-only.

Release Testing

Jump to bottom
Erik Jaegervall edited this page Jul 20, 2023 · 22 revisions

This page intends to list the tests that we shall/must do before we do a release for this repository. The intention of the release test is to do a reasonable regression test for components listed as Beta/Production/Stable/Mature in the KUKSA Component Maturity Overview. It is assumed that new features of individual components have been thoroughly tested, the focus of the release test is to verify that there are no regressions. For capacity reasons the tests have no intention to cover all possible use cases.

Prerequisites

  • No outstanding Pull Requests that may affect component behavior.
  • CI Docker builds on latest master has succeeded
  • Pre-release of kuksa-client exists on PyPI

Tests

Basic functionality, TLS and Tokens

Verify that reading and writing Vehicle.Speed works for the following combinations

Abbreviations

  • DB = Databroker
  • S = Server
  • TOK = JWT Tokens
  • TLS = TLS
  • D = DOCKER (If not mantioned native shall be used)
  • DBC = Databroker Client
  • PC = Python Client
  • GC = Go Client

Left of "equal" denotes Server, right of "equal" denotes client side "Minus" used to separate parts

Basic testing with Databroker (Insecure, No TLS)

erik@debian3:~/kuksa.val/kuksa_databroker$ cargo run --bin databroker -- --metadata ../data/vss-core/vss_release_4.0.json --insecure

Python Client test

(kuksa-client) erik@debian3:~/kuksa.val/kuksa-client$ kuksa-client --ip 127.0.0.1 --port 55555 --protocol grpc  --insecure


Test Client> setValue Vehicle.Speed 33
OK

Test Client> getValue Vehicle.Speed

Databroker CLI

erik@debian3:~/kuksa.val/kuksa_databroker$ cargo run --bin databroker-cli
...
sdv.databroker.v1 > feed Vehicle.Speed 44
[feed]  OK  
sdv.databroker.v1 > get Vehicle.Speed
[get]  OK  
Vehicle.Speed: 44.00

GO Client

Start Client

erik@debian3:~/kuksa.val/kuksa_go_client$ go run .

Start Python client (or Databroker CLI) and verify that DTCList has been set by Go client:

Test Client> getValue Vehicle.OBD.DTCList
{
    "path": "Vehicle.OBD.DTCList",
    "value": {
        "value": {
            "values": [
                "dtc1, dtc2",
                "dtc2",
                "dtc3, dtc3",
                "dtc4"
            ]
        },
        "timestamp": "2023-07-20T13:03:46.314802+00:00"
    }
}

Set Vehicle.Speed in Python client and verify that Go client gets a notification

Test Client> setValue Vehicle.Speed 65

2023/07/20 15:04:14 Vehicle.Speed Subscribed: timestamp:{seconds:1689858254 nanos:509989582} float:65

Feeder smoke tests

DBC2VAL

They rely on the client, so some testing is necessary. Two options exist, either build and install kuksa-client locally, or upload an alpha package to pypi. Testing with an uploaded package is preferable as that also tests that Pypi packaging works as expected.

Start DBC Feeder using default settings

erik@debian3:~/kuksa.val.feeders/dbc2val$ ./dbcfeeder.py

Let it run for a while, verify with Python client (or databroker CLI) that Vehicle.Speed change now and then.

Test Client> getValue Vehicle.Speed                          
{
    "path": "Vehicle.Speed",
    "value": {
        "value": 77.0,
        "timestamp": "2023-07-20T13:17:26.946733+00:00"
    }
}

Test Client> getValue Vehicle.Speed
{
    "path": "Vehicle.Speed",
    "value": {
        "value": 54.0,
        "timestamp": "2023-07-20T13:17:33.924064+00:00"
    }
}

GPS2VAL

This is somewhat tricky. the trick seems to be to make sure that you use the default Python version of your distribution, i.e. not something you have sideloaded. Also using a different port is needed. To test:

Run gpsfake:

erik@debian3:~/kuksa.val.feeders/gps2val$ gpsfake -t -P 2949 simplelog_example.nmea
Processing simplelog_example.nmea
gpsfake: log cycle of simplelog_example.nmea begins.
gpsfake: log cycle of simplelog_example.nmea begins.

Run gps2val:

erik@debian3:~/kuksa.val.feeders/gps2val$ python gpsd_feeder.py --protocol grpc --port 55555 --insecure true --gpsd_port 2949
2023-07-20 15:26:32,525 INFO gpsfeeder: No configuration file found. Using default values.
2023-07-20 15:26:32,526 INFO gpsfeeder: Init kuksa client...
2023-07-20 15:26:32,578 INFO gpsfeeder: No token information provided, subsequent errors expected if Server/Databroker requires authentication!
2023-07-20 15:26:32,579 INFO gpsfeeder: Connection status is: False
2023-07-20 15:26:32,579 INFO kuksa_client.grpc: No Root CA present, it will not be posible to use a secure connection!
2023-07-20 15:26:32,579 INFO kuksa_client.grpc.aio: Establishing insecure channel
2023-07-20 15:26:32,579 INFO gpsfeeder: Init gpsd client...
2023-07-20 15:26:32,580 INFO gpsfeeder: Trying to connect gpsd at 127.0.0.1 port 2949
2023-07-20 15:26:32,580 INFO gpsfeeder: Using mapping
2023-07-20 15:26:32,582 INFO gpsfeeder: gpsd receive loop started
gRPC channel connected.
2023-07-20 15:26:32,601 INFO gpsfeeder: Number of VSS messages sent so far: 1
2023-07-20 15:26:32,620 INFO gpsfeeder: Number of VSS messages sent so far: 2
2023-07-20 15:26:33,640 INFO gpsfeeder: Number of VSS messages sent so far: 4

verify with KUKSA Client that lat and/or lon is updated

Test Client> getValue Vehicle.CurrentLocation.Latitude
{
    "path": "Vehicle.CurrentLocation.Latitude",
    "value": {
        "value": 48.779966667,
        "timestamp": "2023-07-20T13:27:09.618384+00:00"
    }
}

Test Client> getValue Vehicle.CurrentLocation.Latitude
{
    "path": "Vehicle.CurrentLocation.Latitude",
    "value": {
        "value": 48.780183333,
        "timestamp": "2023-07-20T13:27:13.012102+00:00"
    }
}
  • DB-TOK=PC
  • DB-TLS=PC
  • DB-TOK-TLS-D=PC-D
  • DB-TOK-TLS=DBC
  • DB=DBC-D
  • DB=GC
  • DBC
Clone this wiki locally