Supplementary Material for paper "Shaking Up Quantum Simulators with Fuzzing and Rigour"

This repository provides tools for constructing quantum circuits and validating quantum validators using the Alloy modeling language to encode standard quantum gates. We include our Alloy model along with setup and usage instructions. Additionally, the repository contains: xml_seeds/ – XML-based test seeds generated with Alloy, used to obtain the results presented in our paper. execution_logs/ – Logs from simulating the test cases. post-processing/ – Data visualised in our paper, organised as follows: chi_js/ – Chi-square and Jensen-Shannon Divergence analysis (Fig. 6–7). memory/ – Seeds and post-processing data for plotting memory-related results (Fig. 8). shots/ – Data for analysing the optimal number of shots (Fig. 6). invariants/ – Demonstrations of invariants discussed in Section 5. For detailed instructions, see the corresponding folders. Repository Structure .├── Alloy-model│ ├── README.md│ └── standard_gate_library.als├── README.md├── execution_logs├── fuzzQ.py├── post-processing│ ├── chi_js│ ├── invariants│ │ ├── GHZ│ │ ├── Involution│ │ └── Symmetry│ ├── memory│ │ └── xml_instances│ ├── shots│ └── state_vectors_mismatch_Accumulation_of_Numerical_Errors│ └── 3q15g_noSwap└── xml_seeds Parsing Standard Gates The Alloy model, Alloy-model/standard_gate_library.als, encodes standard quantum gates such as XGate, HGate, and CXGate. Using the Alloy tool, we generate XML-based test seeds representing valid quantum circuits that adhere to the model’s constraints. The Python script fuzzQ.py processes these XML instances, converting them into executable Qiskit quantum circuits. It then simulates the circuits across various backends, providing detailed measurement results (outcome probabilities) to facilitate the assessment and validation of quantum simulators. Requirements Using a Virtual Environment (Recommended) While not mandatory, using a virtual environment ensures that your Python packages are isolated from your system’s Python installation. This prevents any version conflicts and keeps your environment clean and manageable. Steps to Set Up: Create a Virtual Environment Open your terminal and run: python3 -m venv myenv This will create a virtual environment named myenv in your current directory. Activate the Virtual Environment Once the environment is created, activate it: source myenv/bin/activate # On macOS/Linux myenv\Scripts\activate # On Windows Once activated, your terminal prompt should display (myenv). Install Dependencies The fuzzQ.py script is designed to automatically install the necessary libraries. However, if you prefer to have full control over the installation process, you can do so manually by following these steps: Install pip (if not already installed) sudo apt install python3-pip Other Dependencies With the environment activated, install the required packages: pip install argparse qiskit qiskit-aer cirq quimb numpy seaborn matplotlib prettytable scipy tqdm psutil Run the Script Execute the program while the virtual environment is active: python fuzzQ.py <path_to_xml_file> Deactivate the Virtual Environment When done, deactivate the virtual environment: deactivate Note: You only need to create the virtual environment and install the dependencies once. After that, you can reactivate the environment whenever you want to run the program. Usage Run the script with: python fuzzQ.py <path_to_xml_file> The XML file should be an Alloy-generated instance representing a quantum circuit. Configuration These variables can be modified inside fuzzQ.py to customise execution settings, including iteration count, circuit scaling, and backend parameters. shots: Sets the number of times the quantum backend repeats the execution of the circuit per submission. Defaults to 10,000 but can be adjusted as needed. num_iterations: Specifies how many times the script submits the circuit to the backend. This acts as an external wrapper, repeatedly passing the same circuit for execution. multiply_circuit: Controls how many times the gate sequence is concatenated, effectively repeating the circuit’s operations and increasing its depth. Test Mode: If no XML file is provided, the script runs in test mode. It will call test_circuits(), allowing you to set up an ad-hoc circuit instead of passing one via an XML file. Simulation Metrics Output The fuzzQ.py script generates detailed simulation metrics and stores them in the simulation_results.csv file. Below is an example of the output format: Metric Value num_qubits 110 num_gates 10 depth 6 num_iterations 1 shots 1000 alpha 0.05 chi_reject_qiskit 1 chi_reject_cirq 0 chi_qiskit [0.0381] chi_cirq [0.2087] js_divergences [0.0195] mem[MiB] 513.15625 Metrics Explanation: num_qubits: Number of qubits in the quantum circuit. num_gates: Total number of quantum gates used in the circuit. depth: The depth of the circuit, representing the number of layers of operations. num_iterations: The number of times the simulation was repeated. shots: Number of measurement shots performed to get statistical data. alpha: The significance level used in statistical tests for the simulation. chi_reject_qiskit/chi_reject_cirq: Chi-square test result for rejecting the null hypothesis, specific to Qiskit or Cirq. chi_qiskit/chi_cirq: The calculated chi-square values for the simulation results from Qiskit and Cirq, respectively. js_divergences: The Jensen-Shannon divergence between Qiskit and Cirq outputs, used to compare distributions. mem[MiB]: The memory usage in MiB during the simulation. These metrics are essential for analysing the performance, statistical significance, and resource consumption of quantum circuits, providing insights into simulations across different platforms. Batch Processing of Quantum Circuits While the fuzzQ.py script is designed to parse and simulate/measure one quantum circuit at a time, the process_xml.sh script offers a streamlined way to process multiple circuits. It scans all .xml files within the folder specified as an argument and runs simulations/measures for each file. Example Usage ./process_xml.sh <path_to_xml_folder>