Metadata-Version: 2.3
Name: test-automation-suite
Version: 2.7.0
Summary: Test automation suite for TUXEDO hard- and software.
Requires-Dist: argcomplete>=3.6.3
Requires-Dist: dbus-fast>=4.0.0
Requires-Dist: jinja2>=3.1.6
Requires-Dist: junit-xml>=1.9
Requires-Dist: pillow>=12.1.1
Requires-Dist: pydantic>=2.12.5
Requires-Dist: pygelf>=0.4.3
Requires-Dist: pyyaml>=6.0.3
Requires-Python: >=3.11
Description-Content-Type: text/markdown

# Test Automation Suite (TAS)

Test automation suite for TUXEDO hard- and software.

## Usage

**The user should be set to auto login, as many tests need to reboot.
Without auto login, these reboots will interrupt the testing process.**

Do not run the TAS with `sudo`, as it will cause issues with screenshots.

Example for running a full testing procedure:

```sh
# Install dependencies, then start the tests immediately
test-automation-suite run test_security_kernel
# Install dependencies, then reboot, then start the tests
test-automation-suite run --prepare-only test_security_kernel && systemctl reboot
```

Example for running individual tests:

```sh
test-automation-suite check network
test-automation-suite benchmark edge_gaming
```

The log files and screenshots are stored in `$HOME/test-automation-suite/[TIMESTAMP]/`.

### Command Line Options

- `--prepare-only`: Dependencies will be installed and the auto start file created, but no tests will be run. This
  should **always** be passed when running the TAS from within the FAI.
- `--config path/to/config.yaml`: Override the configurations specified in [
  `config.py`](src/test_automation_suite/core/config.py).
- `--verbose`: Enables verbose logging.

## Development

All commands needed for development can be found in the [justfile](./justfile).

### Remote Deployment

Usually, TAS development requires running the TAS on a test device.
To simplify doing so, the `just deploy [LAST_IP_OCTET]` command can be used.
It will automatically sync the TAS project to the device under `10.10.20.LAST_IP_OCTET`, which means the test device must be connected to the testing LAN network (`10.10.20.*`), and the dev device must be on the `10.10.11.*` network.

The TAS can then be found under `$HOME/test-automation-suite` and can be run using `just sync-dependencies` followed by `just run [TAS PARAMETERS]`.

Note that the `test-automation-suite` package should be installed first, to ensure that all package dependencies are installed as well.

### Adding new Tasks

To add a new check, you have to:

1. Create a new file `src/test_automation_suite/checks/check_NAME.py`
2. Then create the check class following this template:

```py
class CheckNAME(Check):
    some_parameter: int
    
    @override
    def steps(self) -> list[Step]:
        return [*super().steps(), self.check_NAME]

    def check_NAME(self, _context: Context, logger: Logger) -> StepResult:
        result, error = run_cmd_t(f"do-magic {self.some_parameter}")
        if error:
            logger.error("Failed to do magic: %s", error)
            return StepResult(success=False)
            
        return StepResult(success=True)
```

3. Add the check to the [`checks/__init__.py`](src/test_automation_suite/checks/__init__.py) file

For benchmarks, you can just replace every `check` with `benchmark` and it should work.

### Adding or Updating Procedures

Procedures are defined in [`files/etc/test-automation-suite`](files/etc/test-automation-suite) as YAML files.

Checks and benchmarks will be executed in the order in which they are listed there.

## Design Decisions

### Checks and Benchmarks

The TAS automates two kinds of tasks: `Check`s and `Benchmark`s.

Checks simply test whether something is working as expected, returning either success or an error report.
New checks can be registered in the [`checks/__init__.py`](src/test_automation_suite/checks/__init__.py) file.

Benchmarks only fail when they can not run.
If they run, they are considered success and return the benchmark results without judging them.
New benchmarks can be registered in the [`benchmarks/__init__.py`](src/test_automation_suite/benchmarks/__init__.py)
file.

### Defining Test Procedures

While individual checks and benchmarks can be triggered, the default approach to using the TAS is running one of the
testing procedures defined under `/etc/test-automation-suite/`.

### Reboots, Persistence File and Auto Start

The TAS will save its state to `$HOME/test-automation-suite/state.json` before every task and reboot.
This allows the TAS to resume the test where it left of in case a test needs to reboot or fails critically.

Before rebooting, the TAS will also create a desktop file to automatically re-start itself afterwards.

## TAS Changelog

All fixes and new features for TAS should be recorded in the changelog located in `package.yml`.

### Versioning Rules:

Fixes/patches: increment the last digit of the version number. For example `2.0.3`to `2.0.4`
New features: increment the second digit of the version number. For example: `2.0.4` to `2.1.0`

### Important

After updating the changelog version, make sure to also update the version in `pyproject.toml` and `uv.lock`.
This helps prevent pipeline errors caused by version mismatches
