Structured quantum benchmarks¶
MQT Core defines each structured quantum benchmark by benchmark-specific
parameters and an analytic or verification reference. A benchmark instance can
produce a structured QC program, a resolved manifest, and a stable case ID. The
generated program returns one classical register named result. Outcome strings
are big-endian: the highest-index result bit is the leftmost character.
For an end-to-end device example, start with Compile and execute Shor’s algorithm, which factors 21 with Shor’s algorithm. The repeat-until-success example below demonstrates an adaptive retry loop and a phase-sensitive readout.
Discover the catalog¶
The command-line registry is the current list of available families. Each family has its own instance specification schema. These cells execute the CLI and fail if it exits unsuccessfully; JSON formatting only makes its output easier to read.
1import json
2import subprocess
3
4catalog = subprocess.run(["mqt-core-bench", "list"], check=True, capture_output=True, text=True)
5print(json.dumps(json.loads(catalog.stdout), indent=2))
{
"benchmarks": [
{
"definition_version": 1,
"id": "bv"
},
{
"definition_version": 1,
"id": "ghz"
},
{
"definition_version": 1,
"id": "grover"
},
{
"definition_version": 1,
"id": "grover-weak-measurement"
},
{
"definition_version": 1,
"id": "magic-state-distillation"
},
{
"definition_version": 1,
"id": "modular-multiplier"
},
{
"definition_version": 1,
"id": "multiplexer"
},
{
"definition_version": 1,
"id": "qft"
},
{
"definition_version": 1,
"id": "qft-adder"
},
{
"definition_version": 1,
"id": "qpe"
},
{
"definition_version": 1,
"id": "repeat-until-success"
},
{
"definition_version": 1,
"id": "shor"
},
{
"definition_version": 1,
"id": "teleportation"
},
{
"definition_version": 1,
"id": "w-state"
}
],
"schema_version": 1
}
1description = subprocess.run(["mqt-core-bench", "describe", "qft"], check=True, capture_output=True, text=True)
2print(json.dumps(json.loads(description.stdout), indent=2))
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"benchmark": {
"const": "qft"
},
"parameters": {
"additionalProperties": false,
"properties": {
"method": {
"default": "standard",
"enum": [
"standard",
"semiclassical"
]
},
"period_exponent": {
"maximum": 1074,
"minimum": 0,
"type": "integer"
},
"qubits": {
"maximum": 1000000,
"minimum": 1,
"type": "integer"
}
},
"required": [
"qubits",
"period_exponent"
],
"type": "object"
},
"schema_version": {
"const": 1
}
},
"required": [
"schema_version",
"benchmark",
"parameters"
],
"type": "object",
"x-mqt-definition-version": 1
}
Configure a typed instance¶
Python exposes each benchmark through a family-specific type. Parameterized families also expose option types. The QFT input below is the uniform superposition of multiples of two. Both circuit methods use the same logical output and reference.
1from mqt.core.bench import qft
2
3
4benchmark = qft.QFT(
5 qft.Options(
6 qubits=3,
7 period_exponent=1,
8 method=qft.Method.SEMICLASSICAL,
9 )
10)
11print("Method:", benchmark.options.method)
12print("Output:", benchmark.output.name)
13print("Width:", benchmark.output.width)
Method: Method.SEMICLASSICAL
Output: result
Width: 3
Each family validates its instance when it creates one. Fixed families need no options.
Inspect the canonical instance specification and manifest¶
A canonical instance specification records every resolved default. A manifest also binds the logical output, reference descriptor, family-definition version, and case ID.
1import json
2
3
4instance_specification = json.loads(benchmark.instance_specification_json)
5manifest = json.loads(benchmark.manifest_json)
6print("Instance specification:")
7print(json.dumps(instance_specification, indent=2))
8print("\nManifest summary:")
9print(
10 json.dumps(
11 {
12 "case_id": manifest["case_id"],
13 "outputs": manifest["outputs"],
14 "reference": manifest["reference"],
15 },
16 indent=2,
17 )
18)
Instance specification:
{
"benchmark": "qft",
"parameters": {
"method": "semiclassical",
"period_exponent": 1,
"qubits": 3
},
"schema_version": 1
}
Manifest summary:
{
"case_id": "sha256-ceed1b83b1051851f192e0144524d231c8ecd1a441cb363ce1f26babc4bcaaa7",
"outputs": [
{
"name": "result",
"width": 3
}
],
"reference": {
"kind": "analytic",
"model": "qft_power_of_two_period",
"outcome_order": "big_endian",
"output": "result",
"version": 1
}
}
Query and evaluate the reference¶
For three output bits and period exponent one, QFT has two equal peaks.
1probabilities = {
2 outcome: benchmark.probability(outcome) for outcome in ("000", "100", "010")
3}
4assert probabilities == {"000": 0.5, "100": 0.5, "010": 0.0}
5print(json.dumps(probabilities, indent=2))
{
"000": 0.5,
"100": 0.5,
"010": 0.0
}
1evaluation = benchmark.evaluate({"000": 500, "100": 500})
2print(
3 json.dumps(
4 {
5 "total_variation_distance": evaluation.total_variation_distance,
6 "squared_hellinger_fidelity": evaluation.squared_hellinger_fidelity,
7 "success_probability": evaluation.success_probability,
8 },
9 indent=2,
10 )
11)
{
"total_variation_distance": 0.0,
"squared_hellinger_fidelity": 1.0,
"success_probability": null
}
Total variation distance zero and squared Hellinger fidelity one identify an exact distribution. Some benchmark families also report a success probability for a distinguished success outcome.
Generate structured IR¶
Generation returns a QCProgram, the program type used
by the MQT Compiler Collection. The program
can enter the normal compiler pipeline.
QPE keeps phase residues as exact unsigned integers and converts them to f64
angles after modular reduction. Constant QFT addition and modular multiplication
consume bits from least to most significant using
\(\theta \gets \theta/2 \pm \pi b\), with angles in \([-2\pi, 2\pi]\). Constant
addition stores the input bits; Shor stores modular powers and their inverses.
Iterative QPE, semiclassical QFT, and Shor carry one measurement correction:
\(c \gets c/2 \pm \pi b/2\). The correction stays in \([-\pi, \pi]\) and needs
constant work per measurement. This scalar arithmetic executes with the quantum
operations, so measure compilation and sampling costs separately.
1program = benchmark.generate()
2assert program.is_valid
3structured_ir = program.ir
4assert program.copy().to_qco().is_valid
5print(structured_ir)
module {
func.func @main() -> !cbit.reg<3> attributes {mqt.entry_point} {
%cst = arith.constant 5.000000e-01 : f64
%cst_0 = arith.constant 1.5707963267948966 : f64
%cst_1 = arith.constant 0.000000e+00 : f64
%c2 = arith.constant 2 : index
%c1 = arith.constant 1 : index
%c0 = arith.constant 0 : index
%q = qc.alloc : !qc.qubit
%result = cbit.alloc(#cbit.init<zero>) {mqt.register_name = "result"} : !cbit.reg<3>
%0 = scf.for %arg0 = %c0 to %c2 step %c1 iter_args(%arg1 = %cst_1) -> (f64) {
qc.h %q : !qc.qubit
qc.p(%arg1) %q : !qc.qubit
qc.h %q : !qc.qubit
%b_2 = qc.measure %q : !qc.qubit -> i1
cbit.store %b_2, %result[%arg0] : !cbit.reg<3>
qc.reset %q : !qc.qubit
%1 = arith.mulf %arg1, %cst : f64
%2 = arith.uitofp %b_2 : i1 to f64
%3 = arith.mulf %2, %cst_0 : f64
%4 = arith.addf %1, %3 : f64
scf.yield %4 : f64
}
qc.p(%0) %q : !qc.qubit
qc.h %q : !qc.qubit
%b = qc.measure %q : !qc.qubit -> i1
cbit.store %b, %result[%c2] : !cbit.reg<3>
qc.reset %q : !qc.qubit
qc.dealloc %q : !qc.qubit
return %result : !cbit.reg<3>
}
}
Run the command-line workflow¶
The CLI writes the program first and its manifest last. A manifest is therefore the completion marker. Existing output files always cause an error.
1instance_specification_path.write_text(
2 benchmark.instance_specification_json, encoding="utf-8"
3)
4counts_path.write_text(
5 json.dumps({"schema_version": 1, "counts": {"000": 5, "100": 5}}),
6 encoding="utf-8",
7)
1generation = subprocess.run(
2 ["mqt-core-bench", "generate", "--instance-specification", str(instance_specification_path),
3 "--format", "qc", "--output", str(output_directory)],
4 check=True, capture_output=True, text=True,
5)
6generated = json.loads(generation.stdout)
7print("Generated", generated["benchmark"], "as", generated["format"])
Generated qft as qc
1manifest_path = next(output_directory.glob("*.manifest.json"))
2program_path = next(output_directory.glob("*.qc.mlir"))
3print("Program:", program_path.name)
4print("Manifest:", manifest_path.name)
Program: qft-sha256-ceed1b83b1051851f192e0144524d231c8ecd1a441cb363ce1f26babc4bcaaa7.qc.mlir
Manifest: qft-sha256-ceed1b83b1051851f192e0144524d231c8ecd1a441cb363ce1f26babc4bcaaa7.qc.manifest.json
1evaluation_result = subprocess.run(
2 ["mqt-core-bench", "evaluate", "--manifest", str(manifest_path), "--counts", str(counts_path)],
3 check=True, capture_output=True, text=True,
4)
5metrics = json.loads(evaluation_result.stdout)["metrics"]
6assert metrics["total_variation_distance"] == 0
7print(json.dumps(metrics, indent=2))
{
"squared_hellinger_fidelity": 1.0,
"success_probability": null,
"total_variation_distance": 0.0
}
Use --format jeff instead of --format qc to write a binary jeff program.
The output format changes the file name, but not the semantic case ID.
C++ API¶
A source installation provides the MQT::CoreBench target with typed
parameters, references, evaluation, instances, instance specifications, and
manifests.
#include "bench/Grover.hpp"
#include <cassert>
int main() {
const mqt::bench::Grover benchmark{{.markedBitstring = "101"}};
const auto evaluation = benchmark.evaluate({{"101", 1000}});
assert(evaluation.successProbability == 1.0);
}
find_package(mqt-core CONFIG REQUIRED COMPONENTS Development)
target_link_libraries(my-benchmark PRIVATE MQT::CoreBench)
The source build also provides MQT::CoreBenchGenerate. It exposes typed
mqt::bench::generate(...) overloads from mqt/bench/Generate.h and returns a
llvm::FailureOr<mlir::QCProgram>. This target is only available in source
builds.
Add a benchmark¶
Adding a family requires five extension points:
Add one
(TYPE, STEM, ID, DEFINITION_VERSION)row toinclude/mqt-core/bench/BenchmarkFamilies.inc. Its expansions provide the public JSON declarations and the synchronized semantic and MLIR registry glue.Add the typed instance, any options and validation, a reference, and evaluation under
include/mqt-core/bench/andsrc/bench/. Add the family-specific parameter JSON, reference JSON, parser, and schema body tosrc/bench/JSON.cpp.Declare and implement the structured emitter under
mlir/bench/, add its source to the program library, and declare the typedgenerate(...)overload. The catalog supplies the generation wrapper and JSON dispatch row.Add the explicit Python types in a family registration source under
bindings/bench/, register its direct submodule inregister_bench.cpp, and add the source tobindings/bench/CMakeLists.txt.Test the reference, strict instance specification JSON, emitter structure,
jeffconversion, and Python generation.
BenchmarkFamilies.inc is the sole family catalog. Do not add a private family
list, a generic option map, or a public base class.
Reproducibility contract¶
Instance specifications reject unknown fields and invalid values. The case ID
does not depend on a path or output format. Parsing a manifest checks its
resolved parameters, logical output, reference, definition version, and case ID.
Before evaluation, normalize backend results to the manifest’s big-endian
result order.
Benchmark families¶
Grover search with weak measurement¶
The grover-weak-measurement family implements the weakly measured Grover
search from Andrés-Martínez and Heunen.
It prepares the uniform state and applies one Grover iteration before each
\(\kappa\)-measurement. The measurement computes the marked-state predicate with
\(O_\chi\), applies controlled \(R_\kappa\) to a probe, uncomputes \(O_\chi\), and
measures the probe. The loop continues on \(0\) and exits on \(1\); at exit, the
search register is the marked state.
For \(n\) search qubits, the default measurement strength is
The big-endian marked bitstring has 2–2044 bits; at this limit, the default strength remains a normal double. The benchmark accepts \(0<\kappa\leq 2^{-n/2}\), the range in which the paper proves the Grover step robust to the measurements. A smaller value disturbs the search state less but makes each successful probe outcome less likely. The analytic reference scores the marked search state after the loop terminates; it does not score the number of iterations.
1import math
2
3from mqt.core import mlir
4from mqt.core.bench import grover_weak_measurement
5
6benchmark = grover_weak_measurement.Grover(
7 grover_weak_measurement.Options(marked_bitstring="101")
8)
9assert math.isclose(benchmark.options.measurement_strength, 2 ** (-3 / 2))
10assert mlir.sample(benchmark.generate(), shots=128, seed=17) == {"101": 128}
Magic-state distillation¶
The magic-state-distillation family implements concatenated 15-to-1
distillation with the five-qubit circuit in
Litinski, Figure 3.
Each block starts in \(|+\rangle^{\otimes 5}\) and applies the figure’s 15 Pauli
rotations \(P_{\pi/8}=e^{-i\pi P/8}\) in order. It measures the last four qubits
in the X basis and accepts only when all four outcomes are \(+1\). The first qubit
retains \(T^\dagger|+\rangle\) in the ideal case.
Set levels to a positive integer (default 1). The first level uses ideal
rotations; higher levels consume lower-level output states through Clifford
gates and measurement feedback. Resetting and reusing workspace requires
5 * levels qubits, while each shot executes 15**levels leaf rotations. The
qubit count must fit signed 64-bit circuit dimensions. Generation, compilation,
and execution remain subject to available resources and backend limits; backends
that inline function calls can expand the circuit exponentially.
The two-bit result reports any rejected block in bit 1 and a failed root-state
check in bit 0. The root check applies \(T\) then \(H\) before measurement. Ideal
execution returns 00. All blocks execute once, regardless of rejection. The
benchmark models ideal logical circuits without input noise, physical error
correction, or retries.
Sample the five-qubit circuit directly¶
This example samples the deterministic ideal output with 16 shots.
1from mqt.core.bench import magic_state_distillation
2from mqt.core.mlir import sample
3
4distillation = magic_state_distillation.MagicStateDistillation()
5direct_counts = sample(distillation.generate(), shots=16, seed=17)
6assert direct_counts == {"00": 16}
7print(direct_counts)
{'00': 16}
For device execution, see MQT Core DD-based Simulator QDMI Device.
Modular multiplier¶
The modular-multiplier family uses the controlled modular arithmetic circuit
from Figures 5 and 6 of
Beauregard’s circuit for Shor’s algorithm.
It computes control || multiplicand || product, where
The product register starts at zero and retains its leading overflow bit; a work qubit must return to zero. This is an out-of-place multiplier.
The classical multiplier and canonical modulus are equal-width binary
strings with \(0 < \mathtt{multiplier} < \mathtt{modulus}\). The required
multiplicand has the same width and accepts 0, 1, and +, as in the QFT
adder. A + prepares an independent \(|+\rangle\) qubit. The control accepts
"0", "1", or "+", and defaults to "1". Widths range from 2 to 63 bits.
from mqt.core.bench import modular_multiplier
benchmark = modular_multiplier.ModularMultiplier(
modular_multiplier.Options(multiplier="011", modulus="101", multiplicand="111")
)
assert benchmark.expected_result == "11110001" # control=1, input=7, product=1
assert benchmark.evaluate({"11110001": 100}).success_probability == 1.0
Basis inputs have one exact expected_result, so TVD and success probability
provide a direct check independent of width. An all-zero output fails for this
nonzero example. Test different inputs and both control values to exercise
wraparound and the inactive path.
For superposed inputs, expected_result is None. The reference assigns
probability \(2^{-k}\) to each allowed input and its correct product, where \(k\) is
the number of + input bits, including the control. success_probability is
the shot-weighted fraction matching both the configured inputs and the
arithmetic relation. With \(S\) shots, empirical TVD is at least
\(\max(0,1-S/2^k)\), even for ideal execution. Keep \(k\) small for sampling-based
distribution checks at large widths.
Computational-basis measurements cannot detect arbitrary relative-phase errors. Native tests therefore also compare complete coherent states and require clean work-qubit recovery. A single correct basis result does not certify a unitary on every input.
QFT addition¶
The qft-adder family adds two equal-width operands. REGISTER stores the
addend in qubits and applies controlled phases; CONSTANT combines the known
addend into one phase per accumulator qubit. Both use the same exact QFT and
inverse QFT. WRAP keeps an \(n\)-bit sum, while CARRY keeps an extra sum bit.
1from mqt.core import mlir
2from mqt.core.bench import qft_adder
3
4adder = qft_adder.QFTAdder(
5 qft_adder.Options(
6 addend="110",
7 accumulator="011",
8 method=qft_adder.Method.CONSTANT,
9 overflow=qft_adder.Overflow.CARRY,
10 )
11)
12assert mlir.sample(adder.generate(), shots=128, seed=17) == {"1001": 128}
Operands are big-endian strings; leading zeros set their common width. The
accumulator and constant addends must be binary. Register addends may also use
+ for independently prepared \(|+\rangle\) qubits, such as addend="1+0".
Register results concatenate the addend and sum so their correlation remains
observable. Constant results contain only the sum. expected_result is the
unique logical outcome for basis inputs and None for a superposed addend. The
total sum width, including an optional carry bit, is limited to 1024.
Quantum phase estimation¶
The qpe family estimates a supplied phase using a phase gate and a known
eigenstate. Standard QPE uses a query register and inverse QFT; iterative QPE
measures, resets, and reuses one query qubit with measurement feedback. At
eight-bit precision they use nine and two qubits, respectively.
1from fractions import Fraction
2
3from mqt.core.bench import qpe
4from mqt.core.mlir import compile_program, submit_program
5from mqt.core.qdmi import open_device
6
7device = open_device("mqt.ddsim.default")
8for method in (qpe.Method.STANDARD, qpe.Method.ITERATIVE):
9 phase_estimation = qpe.QPE(qpe.Options(precision=8, phase=Fraction(3, 8), method=method))
10 compiled = compile_program(phase_estimation.generate(), target=device)
11 job = submit_program(compiled, target=device, num_shots=64, custom1=17)
12 job.wait()
13 assert job.get_counts() == {"01100000": 64}
14 assert phase_estimation.evaluate(job.get_counts()).total_variation_distance < 1e-12
A phase such as \(1/3\) lies between eight-bit estimates, giving a distribution over nearby values. TVD compares the sampled distribution with this benchmark’s analytic reference; finite samples generally have nonzero distance.
1approximate = qpe.QPE(qpe.Options(precision=8, phase=Fraction(1, 3), method=qpe.Method.ITERATIVE))
2compiled = compile_program(approximate.generate(), target=device)
3job = submit_program(compiled, target=device, num_shots=4096, custom1=17)
4job.wait()
5evaluation = approximate.evaluate(job.get_counts())
6assert evaluation.total_variation_distance < 0.08
7print(f"Total variation distance: {evaluation.total_variation_distance:.3f}")
Total variation distance: 0.016
Repeat until success¶
A repeat-until-success (RUS) circuit measures an ancilla to decide whether an operation succeeded. On failure, it restores the ancilla and retries. RUS decompositions can reduce the expected number of costly non-Clifford gates in fault-tolerant quantum computing.
The repeat-until-success family generalizes the two-\(T\)-gate circuit in Figure
8 of Paetznick and Svore. It applies
to \(n\) data qubits initially in \(|0^n\rangle\). Each attempt succeeds with probability \(3/4\); failure leaves the data unchanged up to global phase. The expected number of attempts is \(4/3\), giving \(8/3\) \(T\) gates on average before target synthesis. Two controlled Pauli strings use \(2n\) CNOT gates per attempt.
Set data_qubits to \(n\) (default 1, range 1–1,000,000). The circuit uses one
additional ancilla, initially zero, and retries through a while loop with no
fixed attempt limit. Canonical instance specifications include data_qubits,
and case IDs distinguish widths:
{"schema_version":1,"benchmark":"repeat-until-success","parameters":{"data_qubits":32}}
Compile and execute the retry loop¶
Four data qubits and one ancilla give five qubits in this example. DDSIM executes the generated program through Adaptive QIR. See Compile and execute Shor’s algorithm for the basic compile-and-submit workflow and Installation for setup.
1from mqt.core.bench import repeat_until_success
2from mqt.core.mlir import compile_program, submit_program
3from mqt.core.qdmi import open_device
4
5rus = repeat_until_success.RepeatUntilSuccess(
6 repeat_until_success.Options(data_qubits=4)
7)
8device = open_device("mqt.ddsim.default")
9compiled = compile_program(rus.generate(), target=device)
10job = submit_program(compiled, target=device, num_shots=4096, custom1=17)
11job.wait()
12counts = job.get_counts()
13print(f"Counts: {counts}")
Counts: {'0': 3962, '1': 134}
Check the relative phase¶
Successful execution prepares
Measuring only in the computational basis would miss the relative phase. The
benchmark instead measures \(Y\otimes X^{\otimes(n-1)}\) by changing basis and
accumulating parity on the first data qubit. The one-bit result has ideal
probabilities
The result is a final parity measurement, not the ancilla’s success flag or the number of retry attempts.
1evaluation = rus.evaluate(counts)
2assert sum(counts.values()) == 4096
3assert evaluation.total_variation_distance < 0.02
4for bit in ("0", "1"):
5 observed = counts.get(bit, 0) / sum(counts.values())
6 print(f"P({bit}): observed {observed:.3f}, ideal {rus.probability(bit):.3f}")
7print(f"Total variation distance: {evaluation.total_variation_distance:.3f}")
P(0): observed 0.967, ideal 0.971
P(1): observed 0.033, ideal 0.029
Total variation distance: 0.004
The analytic probabilities are independent of width. This readout checks the relative phase without an exponentially large output distribution. Structured loops keep the program compact; execution still takes linear work per attempt. The input width limit does not guarantee that a device or compiler supports that many qubits. See Compile for a QDMI device for device and control-flow limits. The benchmark simulates an ideal adaptive circuit; it does not model error correction, magic-state distillation, or cultivation.
Shor order finding¶
The shor family implements the semiclassical circuit in Sections 2.3–2.4 of
Beauregard’s algorithm. An odd
number from \(3\) through \(2^{31}-1\) and a coprime base with
\(1 < \mathtt{base} < \mathtt{number}\) (default 2) define an instance. Prime
moduli are valid. For an \(n\)-bit modulus, the circuit uses \(2n+3\) qubits and
returns \(2n\) phase bits.
1from mqt.core.bench import shor
2
3order_finding = shor.Shor(shor.Options(number=21))
4assert order_finding.output.width == 10
5assert order_finding.evaluate({"0010101011": 64}).factors == (3, 7)
The manifest uses a verification reference with model shor_factors.
Evaluation reports a sorted factor pair, when found, and the fraction of shots
that independently yield verified factors. It does not report an ideal phase
distribution, TVD, or Hellinger fidelity. An unsuccessful phase is valid data;
invalid outcomes and empty counts are rejected. The input limit bounds classical
recovery, not simulation cost. See Compile and execute Shor’s algorithm for phase
interpretation, execution, and the callback-based factoring workflow.
W-state preparation¶
The w-state family prepares the equal, positive-amplitude superposition of all
single-excitation states:
qubits must be positive. The ideal probability is \(1/n\) for each
single-excitation bitstring and zero otherwise.
1from mqt.core import mlir
2from mqt.core.bench import w_state
3
4w = w_state.WState(w_state.Options(qubits=3))
5counts = mlir.sample(w.generate(), shots=4096, seed=17)
6assert set(counts) == {"001", "010", "100"}
7assert w.evaluate(counts).total_variation_distance < 0.03
8assert w.probability("010") == 1 / 3
9
10large_w = w_state.WState(w_state.Options(qubits=256))
11large_counts = mlir.sample(large_w.generate(), shots=64, seed=17)
12assert sum(large_counts.values()) == 64
13assert all(len(outcome) == 256 and outcome.count("1") == 1 for outcome in large_counts)
DD sampling needs no dense statevector, but intermediate DDs determine cost.