Local test execution setup

[WilliamRoebuck]

Add a basic setup to run system-level tests for lifecycle on the host machine. Resolves eclipse-score/score#2403

The minimal example shows a GTEST executable launched by LM that tests simple state transitions.

[github-actions]

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run //:license-check

Status: ⚠️ Needs Review

Click to expand output

[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.4.2) and connecting to it...
INFO: Invocation ID: 6cf46ad5-1e3e-49c8-afd7-1f3c4cf07bfa
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)

Analyzing: target //:license-check (34 packages loaded, 9 targets configured)

Analyzing: target //:license-check (41 packages loaded, 9 targets configured)

Analyzing: target //:license-check (77 packages loaded, 9 targets configured)

Analyzing: target //:license-check (118 packages loaded, 429 targets configured)

Analyzing: target //:license-check (133 packages loaded, 3984 targets configured)

Analyzing: target //:license-check (134 packages loaded, 5373 targets configured)

Analyzing: target //:license-check (140 packages loaded, 7218 targets configured)

Analyzing: target //:license-check (145 packages loaded, 7267 targets configured)

Analyzing: target //:license-check (145 packages loaded, 7267 targets configured)

Analyzing: target //:license-check (149 packages loaded, 9401 targets configured)

INFO: Analyzed target //:license-check (150 packages loaded, 9527 targets configured).
[1 / 11] Expanding template license.check.license_check; 0s local
[9 / 13] JavaToolchainCompileClasses external/rules_java+/toolchains/platformclasspath_classes; 0s disk-cache, processwrapper-sandbox ... (2 actions running)
[11 / 13] JavaToolchainCompileBootClasspath external/rules_java+/toolchains/platformclasspath.jar; 0s disk-cache, processwrapper-sandbox
INFO: Found 1 target...
Target //:license.check.license_check up-to-date:
  bazel-bin/license.check.license_check
  bazel-bin/license.check.license_check.jar
INFO: Elapsed time: 29.174s, Critical Path: 2.46s
INFO: 13 processes: 9 internal, 3 processwrapper-sandbox, 1 worker.
INFO: Build completed successfully, 13 total actions
INFO: Running command line: bazel-bin/license.check.license_check ./formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

[github-actions]

The created documentation from the pull request is available at: docu-html

[pawelrutkaq]

This shall be synced with @PiotrKorkus hwoto approach tests. There are score tools for it. We shall make it consistent.

[NicolasFussberger]

This shall be synced with @PiotrKorkus hwoto approach tests. There are score tools for it. We shall make it consistent.

I had a brief chat on slack with Piotr and as far as I understood there are 2 options in Score.
The framework from tooling repo which only allows local execution and the ITF framework which allows execution on qemu machine.

I believe, since the launch manager has some qnx-specific functionality (like e.g. security policies), that testing on QNX is a must -have. But always running tests in QEMU is also a pain, which slows down development.
So the idea was to have the best of both worlds, to allow just executing tests locally (on linux), but also allow to run the tests in qemu using ITF. So this PR adds a minimal abstraction that lets you start something locally or via ssh on qemu machine.

This is only the first step to get some basic smoke tests running. Maybe we can agree on a first solution just to get some integration tests working soon. I feel like there is no consistent solution in SCore at this point.

[pawelrutkaq]

This shall be synced with @PiotrKorkus hwoto approach tests. There are score tools for it. We shall make it consistent.

I had a brief chat on slack with Piotr and as far as I understood there are 2 options in Score. The framework from tooling repo which only allows local execution and the ITF framework which allows execution on qemu machine.

I believe, since the launch manager has some qnx-specific functionality (like e.g. security policies), that testing on QNX is a must -have. But always running tests in QEMU is also a pain, which slows down development. So the idea was to have the best of both worlds, to allow just executing tests locally (on linux), but also allow to run the tests in qemu using ITF. So this PR adds a minimal abstraction that lets you start something locally or via ssh on qemu machine.

This is only the first step to get some basic smoke tests running. Maybe we can agree on a first solution just to get some integration tests working soon. I feel like there is no consistent solution in SCore at this point.

itf can run docker, I am bit wordied that not we doing simply third thing becasue its faster to do some few python code. I will talk to Piotr, but we shall still not merge it to main before release as first we need other fixes - i dont see this to hurry

[MaciejKaszynski]

This shall be synced with @PiotrKorkus hwoto approach tests. There are score tools for it. We shall make it consistent.

I had a brief chat on slack with Piotr and as far as I understood there are 2 options in Score. The framework from tooling repo which only allows local execution and the ITF framework which allows execution on qemu machine.
I believe, since the launch manager has some qnx-specific functionality (like e.g. security policies), that testing on QNX is a must -have. But always running tests in QEMU is also a pain, which slows down development. So the idea was to have the best of both worlds, to allow just executing tests locally (on linux), but also allow to run the tests in qemu using ITF. So this PR adds a minimal abstraction that lets you start something locally or via ssh on qemu machine.
This is only the first step to get some basic smoke tests running. Maybe we can agree on a first solution just to get some integration tests working soon. I feel like there is no consistent solution in SCore at this point.

itf can run docker, I am bit wordied that not we doing simply third thing becasue its faster to do some few python code. I will talk to Piotr, but we shall still not merge it to main before release as first we need other fixes - i dont see this to hurry

So we are also working on a ITF integration and that is where the real testing is done (Draft PR #75). Right now we are using QEMU targets but I had a brief talk with an ITF person, and the goal seems to be for different targets to be transparent to the test, meaning that using docker vs QEMU can be changed with just a argument.
(I know currently there is some re-writing happening in ITF and the APIs aren't final. There are still some issues for example with docker vs QEMU as running a command on the target requires different python code. But it sounds like at some point it would be the same.)

This PR allows for native testing which is just a improvement for developers. I don't think there is anything extra needed to allow for this. There is a wrapper here for the testing framework which is a good idea anyways to have a more consistent interface. The PR will just add an implementation for the interface for native support and the previously mentioned branch would add an implementation for ITF support.