How to Benchmark a Quantum Circuit: Depth, Width, Fidelity, and Runtime Basics

Overview

Here is the short version: a useful quantum circuit benchmark answers four questions.

1. How large is the circuit? Usually measured by width, or the number of qubits used.
2. How long is the circuit logically? Usually measured by depth, or the number of sequential gate layers after accounting for parallel operations.
3. How close is the output to what you intended? Often approximated with fidelity-related checks, success probability, or distance from an expected distribution.
4. How expensive was execution? Measured with runtime, shot count, queue delay, and sometimes compilation overhead.

Those four categories sound simple, but many developers get confused because they are reported differently across SDKs, simulators, and hardware platforms. A circuit that looks shallow before compilation can become much deeper after routing. A high-fidelity result on a noiseless simulator says little about device behavior. A fast wall-clock run can hide long queue times or repeated retries.

That is why quantum circuit benchmarking needs a framework, not just a number. The goal is not to crown a universal winner. The goal is to create a benchmark record that another developer can reproduce and interpret.

As a rule, benchmark at three levels:

• Abstract circuit level: the algorithmic circuit before device-specific constraints.
• Compiled circuit level: the transpiled or transformed circuit for a target backend.
• Execution result level: the observed outputs, runtime behavior, and quality metrics.

If you separate those three levels, your comparisons become much cleaner. This is especially useful in quantum programming for beginners, where it is easy to confuse algorithm design issues with backend limitations.

Core framework

This section gives you a reusable framework for quantum circuit benchmarking. You can apply it in a qiskit tutorial workflow, adapt it to a cirq tutorial, or use the same logic in PennyLane and other SDKs.

1. Define the benchmark purpose before you collect metrics

Do not start by asking, “What numbers can I get?” Start by asking, “What am I comparing?” Common benchmark goals include:

• Comparing two implementations of the same algorithm
• Comparing a hand-written circuit with an SDK-generated one
• Comparing simulator output with hardware output
• Checking whether a transpiler setting improves execution viability
• Tracking performance drift over time on the same backend

Your purpose determines which metrics matter. If your goal is algorithm design, abstract depth and gate count matter more than queue time. If your goal is production-style execution on a device, compilation and runtime details become central.

2. Measure width carefully

Width is usually the number of qubits required by the circuit. In practice, you should record at least two versions:

• Logical width: qubits required by the algorithm itself
• Physical or mapped width: qubits actually occupied after mapping to a backend, if ancilla or routing overhead increases usage

Width matters because larger circuits are generally harder to execute reliably. More qubits usually means more exposure to noise, more calibration sensitivity, and a bigger state space to simulate classically.

When reporting width, also note whether classical registers or mid-circuit measurement logic are involved. They may not increase qubit count, but they affect execution complexity and portability.

3. Measure depth at the right stage

Circuit depth is one of the most common metrics in quantum circuit performance, but it is also one of the easiest to misuse. Depth should be measured at both:

• Pre-compilation depth: useful for understanding algorithmic structure
• Post-compilation depth: useful for understanding real execution cost on a chosen backend

A circuit with excellent theoretical depth can become much deeper after hardware-aware compilation due to limited qubit connectivity, basis gate translation, or inserted swap operations. If you only report pre-compilation depth, you may hide the actual device burden.

Depth alone is still incomplete. Record these supporting metrics as well:

• Total gate count
• Two-qubit gate count
• Measurement count
• Critical-path depth for two-qubit operations, if your SDK exposes it

Two-qubit gates deserve special attention because they are often noisier and more costly than single-qubit operations on many platforms. In many real workflows, reducing entangling-gate count matters more than trimming a few single-qubit layers.

4. Treat fidelity as a family of checks, not one universal number

Fidelity sounds precise, but in everyday developer workflows it often stands in for several different ideas:

• State fidelity against an ideal target state
• Process fidelity for a compiled operation
• Success probability for the expected answer
• Similarity between observed and expected output distributions
• Task-level quality, such as energy error in VQE or approximation quality in QAOA

That means you should be explicit. Instead of writing “fidelity = 0.91” without context, write what was measured and how. For example:

• Distribution similarity against noiseless simulation
• Probability of measuring the expected bitstring
• Distance from the target expectation value

For many developers, especially outside research-heavy workflows, a practical substitute for full fidelity analysis is an accuracy proxy: did the circuit produce the correct answer often enough to be useful for the task? That may be more actionable than a formal metric if you are benchmarking application behavior rather than low-level control performance.

If you want a simple fidelity-oriented checklist, use:

1. Define the ideal reference result
2. Choose a task-relevant similarity metric
3. Use the same shot count across comparable runs
4. Repeat trials enough times to detect instability
5. Record whether noise was simulated, inferred, or observed on hardware

5. Break runtime into separate components

Runtime in quantum systems is not just one clock reading. For a meaningful benchmark, separate it into:

• Circuit build time: time to construct the circuit in code
• Compilation or transpilation time: time spent preparing the circuit for a backend
• Submission delay: time from submission to actual execution start
• Execution time: backend processing time for the shots requested
• Total wall-clock time: end-to-end elapsed time seen by the user

This matters because different platforms optimize different parts of the stack. A simulator may have near-zero queue delay but expensive statevector simulation. A hardware backend may execute quickly once scheduled but sit in queue for much longer. If you combine all of that into one runtime number, you lose the detail needed for a fair comparison.

6. Document the environment and assumptions

No quantum computing tutorial on benchmarking is complete without metadata. Include:

• SDK and version
• Backend type: statevector simulator, shot-based simulator, noisy simulator, or hardware
• Target basis gates if relevant
• Qubit topology or connectivity assumptions
• Shot count
• Optimization or transpiler level
• Noise model, if any
• Random seed, if supported

This is the difference between a benchmark and a screenshot. Good metadata makes your result reusable.

7. Use a compact benchmark template

For each circuit, save a record like this:

• Benchmark goal
• Problem size
• Logical width
• Compiled width
• Pre-compilation depth
• Post-compilation depth
• Total gate count
• Two-qubit gate count
• Reference output definition
• Quality metric used
• Shot count
• Compilation time
• Execution time
• Total wall-clock time
• Backend and SDK details

Once you adopt a template like this, benchmarking becomes much less subjective.

Practical examples

Let us make the framework concrete with a few realistic scenarios.

Example 1: Comparing two Bell-state circuits

Suppose you write two versions of a simple Bell-state preparation circuit. Both use two qubits and aim for the same output distribution. At first glance, benchmarking feels trivial. But even here, a useful comparison includes:

• Logical width: 2 for both circuits
• Pre-compilation depth: perhaps identical
• Post-compilation depth: may differ if one version uses gates that decompose differently on the target backend
• Two-qubit gate count: usually the key cost driver
• Output distribution similarity: compare observed counts with the ideal 50/50 pattern on the relevant outcomes
• Runtime: compilation and shot execution time

In a noiseless simulator, both may appear equivalent. On hardware, one decomposition may introduce more error after basis translation. This is a simple example of why abstract correctness is not enough.

Example 2: Benchmarking a QAOA layer

Variational circuits are a good stress test because performance depends on both circuit design and repeated execution. If you benchmark a QAOA layer, record not only the usual depth and width but also:

• Parameter count
• Cost of repeated evaluations across optimization steps
• Average quality metric across runs, not just the best run
• Sensitivity to shot count

This matters because a circuit with lower single-run depth may still be more expensive overall if the optimizer needs many more evaluations. If you want context for where this fits, see Variational Quantum Algorithms Explained: Why VQE and QAOA Keep Showing Up and QAOA Tutorial: From Cost Hamiltonian to a Working Python Example.

Example 3: Simulator versus hardware benchmark

A common developer workflow is to validate a circuit on a simulator, then run it on hardware. This is where many informal benchmarks break down.

A sound comparison should preserve:

• The same algorithmic circuit definition
• A documented compilation pathway for each target
• A comparable shot count where meaningful
• A clear quality metric tied to the intended task

Then record the differences separately:

• Simulator depth after compilation
• Hardware depth after compilation
• Simulator execution time
• Hardware queue plus execution time
• Simulator output quality
• Hardware output quality

If the hardware result degrades badly, do not jump straight to “the device is bad.” Check whether routing overhead inflated depth or whether the entangling-gate count increased. A backend-specific compilation issue may explain more than the raw device noise. For background on noise itself, this companion guide helps: Quantum Noise Models Explained: Depolarizing, Bit-Flip, Phase-Flip, and More.

Example 4: Cross-SDK comparison

If you are comparing the same circuit idea across Qiskit, Cirq, and PennyLane, be careful not to compare the surface syntax instead of the compiled result. The fair workflow is:

1. Define the same abstract circuit intent
2. Compile each version for a comparable backend model
3. Record basis-gate differences and optimizer settings
4. Compare post-compilation metrics, not just source-code brevity

This approach is especially useful if you are working through a broader quantum computing roadmap or choosing a toolchain for a team. Related reading: Quantum Programming Roadmap: What to Learn After Python if You Want to Build with Qubits and Quantum Machine Learning Framework Comparison: PennyLane vs Qiskit Machine Learning vs TensorFlow Quantum.

Common mistakes

If your benchmark feels inconsistent or hard to trust, one of these issues is usually the cause.

Using only one metric

A shallow circuit is not automatically better. A faster run is not automatically more accurate. A higher success probability with ten times the runtime may or may not be worthwhile depending on the use case. Always pair structural metrics with quality and cost metrics.

Comparing pre- and post-compilation circuits as if they were the same object

This is one of the biggest benchmarking errors in quantum computing for developers. The algorithmic circuit is not the execution-ready circuit. Report both explicitly.

Ignoring two-qubit gate cost

Total depth is useful, but two-qubit gate count often predicts practical difficulty better than total gate count alone. If you skip it, your benchmark may miss the main bottleneck.

Hiding shot count and repetition count

Results from 100 shots and 10,000 shots should not be treated as directly comparable. Likewise, a single lucky run should not define performance. Record repetitions and variability.

Using “fidelity” without definition

If the metric is really success probability, say that. If it is distribution overlap, say that. Precision in naming improves benchmark quality immediately.

Confusing queue delay with algorithmic inefficiency

Hardware access conditions can change independently of circuit quality. End-to-end runtime is useful, but it should not be the only runtime number you report.

Comparing different problem sizes

A benchmark only makes sense if the task is held constant. If one circuit solves a larger instance or uses more expressive structure, raw comparisons are misleading unless normalized carefully.

Forgetting the hardware model

Connectivity, native gates, and qubit quality influence compiled depth and error exposure. A benchmark that omits backend assumptions is hard to reuse. If your work touches hardware differences more directly, it also helps to understand platform tradeoffs such as those discussed in Trapped Ion Quantum Computers Explained: Strengths, Tradeoffs, and Use Cases and Quantum Annealing vs Gate-Based Quantum Computing: Which Problems Fit Each Model?.

When to revisit

A benchmarking method is only useful if you update it when the surrounding assumptions change. This is the part many teams skip. They create one notebook, one chart, and then keep reusing it long after the toolchain or backend has changed.

Revisit your benchmark when any of the following happens:

• The primary method changes. For example, you move from ideal simulation to noisy simulation, or from one transpilation strategy to another.
• New tools or standards appear. SDKs often add better compilation passes, scheduling options, or new metric APIs.
• You switch backend families. A circuit optimized for one hardware style may benchmark differently on another.
• Your application objective changes. A benchmark for state preparation is not the same as a benchmark for optimization quality or kernel evaluation.
• You scale problem size. Small toy examples can hide depth blowups and routing costs that appear at larger sizes.

To keep benchmarking practical, use this action-oriented refresh checklist:

1. Pick one representative circuit from your workflow.
2. Run it at the abstract, compiled, and execution levels.
3. Record width, depth, two-qubit gates, quality metric, and runtime split.
4. Save backend metadata and SDK versions.
5. Repeat after any major compiler, backend, or noise-model change.
6. Keep old results so you can track improvement or regression over time.

If you are building a personal learning path, this benchmark habit also improves how you read algorithms. It forces you to connect theory with execution reality. For example, when studying the Quantum Fourier Transform or Shor's Algorithm, benchmarking helps you distinguish elegant circuit structure from hardware practicality.

The durable takeaway is simple: benchmark quantum circuits with a small, explicit set of metrics, and always report the stage at which each metric was measured. Width tells you how many qubits you are asking for. Depth and two-qubit count tell you how much sequential and entangling work the circuit demands. Fidelity-oriented checks tell you whether the answer is still meaningful. Runtime tells you what the workflow costs in practice. Put together, those metrics give you a benchmark you can compare today and revisit when methods, tools, or standards evolve.