Pattern-Based Validation of SysML~v2 Models with Refinery

# SysML v2 Validator

An automated, tool-independent validation framework for [SysML v2](https://www.omg.org/spec/SysML/2.0) models using the [Refinery](https://refinery.tools/) graph solver.

## Overview

Modern cyber-physical and safety-critical systems involve engineers from multiple disciplines, increasing the risk of modeling errors. While effective validation tools exist for SysML v1, similar capabilities are not yet mature for SysML v2.

This project provides an **automated validation pipeline** that:

1. Connects to any [SysML v2 API](https://github.com/Systems-Modeling/SysML-v2-API-Services)-compatible server

2. Transforms the model into a [Refinery](https://refinery.tools/)-compatible representation

3. Evaluates user-defined **validation patterns** (structural pitfalls, flow analysis, multiplicity checks, etc.)

4. Returns **human-readable** interpreted results

An expressive pattern language allows organizations to define and execute their own rules. An initial set of patterns demonstrates common SysML v2 pitfalls.

## Repository Structure

```

├── scripts/ # Python validation pipeline

│ ├── run_pipeline.py # Main entry point

│ ├── model_transformer.py # Fetch & transform models from the API

│ ├── build_refinery_problem.py # Assemble .problem files

│ ├── check_with_refinery.py # Run Refinery CLI via Docker

│ ├── inconsistency_interpreter.py # Human-readable output

│ ├── validation_pattern_utils.py # Pattern file parsing

│ ├── sync_projects_commits.py # CSV batch sync

│ ├── sysml_api.py # Shared SysML v2 API layer

│ ├── inconsistency_templates.json # Interpretation templates

│ └── sysml_models.csv # Example CSV for batch mode

├── config/ # Pipeline configuration files

│ ├── Whitelist.txt # Metamodel element type whitelist

│ ├── OppositePairs.txt # Opposite relation pairs

│ └── AdditionalRefineryAttributes.txt

├── metamodels/

│ └── metamodel.txt # SysML v2 metamodel in Refinery format

├── validation_patterns/ # Validation pattern definitions (.pat)

│ ├── P1.pat – P4.pat # Common pitfall patterns

│ ├── FlowValidationPatterns.pat

│ ├── StartAndDoneNodeIdentification.pat

│ ├── SuccessionMultiplicity.pat

│ └── LiteralValueCheck.pat

├── example_models/ # SysML v2 example models (.sysml)

├── showcase/ # Interactive web showcase (Docker Compose)

│ ├── backend/ # Flask API backend

│ ├── notebooks/ # Jupyter SysML notebook with examples

│ ├── css/, js/ # Frontend assets

│ ├── index.html # Showcase web UI

│ ├── api.Dockerfile # SysML v2 API Services image

│ ├── jupyter.Dockerfile # JupyterLab with SysML kernel

│ └── backend/Dockerfile # Backend + pipeline image

├── docker-compose.yml # Full-stack showcase (pre-built images)

├── docker-compose.build.yml # Override for local builds

├── .github/workflows/ # CI/CD: Docker image publishing

├── LICENSE # Apache License 2.0

└── CONTRIBUTORS.md

```

## Prerequisites

- **Python 3.10+** with `requests` (`pip install requests`)

- **Docker** (for running the Refinery CLI and the showcase)

- A running **SysML v2 API** instance (or use the included Docker Compose setup)

## Quick Start — CLI Pipeline

### 1. Run the full pipeline for a single model

```bash

cd scripts/

# By project name (resolves latest commit automatically)

python run_pipeline.py --project-name MyModel --api-base http://localhost:9000

# By project ID

python run_pipeline.py --project-id 728419a3-1c5e-4b57-8578-544ff6855fd4

# With a specific commit

python run_pipeline.py --project-name MyModel --commit-id <commit-uuid>

```

This will:

- Fetch the model from the SysML v2 API

- Transform it into Refinery format (saved to `transformed/`)

- Build the `.problem` file with all validation patterns (saved to `problems/`)

### 2. Run the validation

```bash

python check_with_refinery.py MyModel

```

This runs the [Refinery CLI](https://refinery.tools/) in a Docker container. Results include:

- A consistency summary

- Human-readable interpreted issues (if any)

You can also point directly to a `.problem` file or directory:

```bash

python check_with_refinery.py problems/MyModel/MyModel.problem

python check_with_refinery.py problems/MyModel # checks all .problem files

```

### 3. Run pipeline + validation in one step

```bash

python run_pipeline.py --project-name MyModel --check

```

### CLI Arguments Reference

| Script | Key Arguments |

|--------|--------------|

| `run_pipeline.py` | `--project-name`, `--project-id`, `--commit-id`, `--api-base`, `--check`, `--split-patterns`, `--validation-pattern`, `--use-cached-model` |

| `check_with_refinery.py` | `<problem_file>`, `--jobs`, `--timeout`, `--api-base`, `--show-all` |

| `model_transformer.py` | `--base-url`, `--project-id`, `--commit-id`, `--output`, `--page-size` |

| `build_refinery_problem.py` | `--output`, `--metamodel-file`, `--transformed-file`, `--validation-dir`, `--split-patterns` |

| `sync_projects_commits.py` | `--api-base`, `--csv`, `--discover` |

Run any script with `--help` for the full list of options.

## Quick Start — Docker Compose Showcase

The showcase provides a self-contained environment with a web UI, a SysML v2 API, a JupyterLab instance, and the validation backend.

### Using pre-built images (recommended)

The default `docker-compose.yml` pulls published images from GitHub Container Registry:

```bash

docker compose up -d

```

### Building locally

A separate override file (`docker-compose.build.yml`) provides build directives for local development:

```bash

docker compose -f docker-compose.yml -f docker-compose.build.yml up -d --build

```

### Services

| Service | URL | Description |

|---------|-----|-------------|

| **Web UI** | http://localhost:8080 | Interactive validation showcase |

| **SysML v2 API** | http://localhost:9000 | SysML v2 API Services |

| **JupyterLab** | http://localhost:8888 | SysML v2 notebook environment |

| **PostgreSQL** | localhost:5432 | Database for the SysML v2 API |

**JupyterLab access:** The default token is `sysmlv2-validator`. Open http://localhost:8888/?token=sysmlv2-validator or set a custom token via the `JUPYTER_TOKEN` environment variable in `docker-compose.yml`.

The Jupyter instance comes with pre-loaded SysML v2 example notebooks containing all validation test models. Use the `%publish` magic to push them to the API, then validate through the web UI.

### Workflow

1. Open JupyterLab and run the example notebook cells to publish models to the API

2. Open the Web UI at http://localhost:8080

3. Select a project and commit

4. Choose validation patterns (or load the built-in examples)

5. Click **Run Validation** and review the results

## Validation Patterns

Patterns are written in [Refinery's](https://refinery.tools/) pattern language. Each `.pat` file in `validation_patterns/` defines one or more `pred` (predicate) or `error` (inconsistency) rules.

### Included Patterns

| File | Description |

|------|-------------|

| `P1.pat` | Package-level structural usage (SysML v2 Common Pitfalls Part 1) |

| `P2.pat` | Message/flow endpoints not bound to event occurrences (Part 2) |

| `P3.pat` | Subsetting usage without explicit multiplicity (Part 3) |

| `P4.pat` | The `then` trap — shorthand succession resolution (Part 4) |

| `FlowValidationPatterns.pat` | Multiple outgoing/incoming successions on non-control nodes |

| `StartAndDoneNodeIdentification.pat` | Identifies likely start/done nodes in action flows |

| `SuccessionMultiplicity.pat` | Missing explicit succession end multiplicities |

| `LiteralValueCheck.pat` | Literal value checks (placeholder pattern) |

### Writing Custom Patterns

Create a new `.pat` file in `validation_patterns/`. Example:

```

error MyCustomRule(element) <->

PartUsage(element),

Element::owner(element, pkg),

Package(pkg).

```

The pattern will be automatically included in the next pipeline run.

## Third-Party Dependencies

### Python (pipeline scripts)

| Package | License |

|---------|---------|

| [requests](https://pypi.org/project/requests/) | Apache 2.0 |

### Python (showcase backend)

| Package | License |

|---------|---------|

| [Flask](https://pypi.org/project/Flask/) | BSD-3 |

| [flask-cors](https://pypi.org/project/Flask-Cors/) | MIT |

| [requests](https://pypi.org/project/requests/) | Apache 2.0 |

| [gunicorn](https://pypi.org/project/gunicorn/) | MIT |

## Citation

If you use this tool in your research, please cite:

```bibtex

% COMING SOON

```

## License

This project is licensed under the [Apache License 2.0](LICENSE).

## Acknowledgments

This work was developed at the [Critical Systems Research Group (ftsrg)](https://ftsrg.mit.bme.hu/) of the [Budapest University of Technology and Economics (BME)](https://www.bme.hu/).