Metric profiles (opendpd/core/metrics/, protected path)

Every score in an EvaluationResult names the profile and version that produced it. A profile fixes the algorithm, the band layout, the PSD estimator, the aggregation, the unit and the "better" direction. Changing any of these is a scientific-semantics change: bump the profile version (or add a profile) and keep the old one reachable. opendpd profiles prints the registry; GET /metrics/profiles serves it to the GUI.

opendpd-spectral-v2 (Studio default, frozen)

Studio training and final evaluation use the same valid-sample calculation. Evaluation tensors may contain padding in their final segment; that padding is excluded before estimating the spectrum or error. The old padded segment introduces a discontinuity that can dominate adjacent-channel power even when the original signal has very little leakage.

The PSD is a two-sided Welch estimate: periodic Hann window, dataset nperseg, 50% overlap, no detrending, density scaling. Band power is the sum of bins with centre frequencies in [low, high), multiplied by fs/nperseg. With occupied bandwidth B and N = n_sub_ch, each carrier has width W = B/N. Left and right adjacent bands are [-B/2-W, -B/2) and [B/2, B/2+W). The reference is the strongest of the N in-band carriers. ACLR_L/R = 10 log10(P_adjacent / P_reference) are negative dBc (lower is better); ACLR_AVG is their arithmetic mean in dB. Missing metadata, insufficient samples or bands outside the capture produce an unavailable metric, never an invented score.

NMSE is pooled over valid samples. IBE is the in-band error power ratio, using the same definition as general-spectral-v1; it is not demodulated constellation EVM. PA checkpoints use validation NMSE; DPD checkpoints use validation carrier ACLR_AVG. Validation/test history contains full-split evaluations only. Short live signal excerpts do not produce metric scores.

Carrier ACLR differs from general-spectral-v1 ACPR, which integrates an adjacent band as wide as the whole occupied signal and uses the whole main band as its reference. These are explicit baseband protocols, not NR, LTE or Wi-Fi transmitter conformance measurements. Compare results using the same profile, bandwidth, carrier count and sample rate.

tests/unit/test_metrics_spectral_v2.py checks known adjacent-tone powers, bandwidths from 5–320 MHz, carrier counts, odd FFT sizes, incomplete segments and an NR 20 MHz padding regression against a separately written FFT periodogram implementation. Historical CLI/paper reproduction scripts retain legacy-opendpd-v1; pass --metric_profile opendpd-spectral-v2 to the legacy CLI to opt into the new training calculation.

The general and symbol-EVM profiles remain alternative report scorers. They retain the historical training objective; the exported historical CLI command therefore names legacy-opendpd-v1, while final reports keep their selected scoring profile. The CLI training flag offers only the two training protocols.

legacy-opendpd-v1 (frozen, historical reproduction)

The historical OpenDPD numbers, computed by utils/metrics.py exactly as the trainer calls it. Values are pinned by tests/golden/legacy_metrics_v1.json (rel/abs 1e-7) and the registry reproduces them (tests/unit/test_metrics_registry.py).

Metric	Unit	Definition	Aggregation
NMSE	dB	mean over segments of 10·log10(Σ|e|² / Σ|y|²)	mean of per-segment dB, not pooled
EVM	dB	20·log10 of the mean sub-channel spectral error ratio	repo-specific; not a demodulated EVM
ACLR_L / ACLR_R	dBc	10·log10(P_adjacent / max sub-channel power)	leakage convention, negative, lower is better
ACLR_AVG	dBc	(ACLR_L + ACLR_R)/2	arithmetic mean of dB values

Segments are nperseg samples; the last one is zero-padded and included.

general-spectral-v1

Explicit conventions, analytic reference tests in tests/unit/test_metrics_general.py.

Metric	Unit	Definition	Reference check
NMSE	dB	10·log10(Σ|y−r|² / Σ|r|²) pooled over the valid range	complex gain g: exactly 20·log10|g−1|; white noise: its SNR ±0.1 dB
IBE in-band error	dB	10·log10(P_main(y−r) / P_main(r)) from the Welch PSD	gain error exact; noise: SNR + 10·log10(BW/fs) ±0.15 dB
ACPR_L / ACPR_R	dBc	10·log10(P_adjacent(y) / P_main(y)) of the evaluated signal	a tone of known power in the adjacent channel ±0.05 dB

Parameters (all recorded in the profile): Hann window, nperseg from the dataset, 50 % overlap, two-sided density PSD, band power = Σ bins·(fs/nperseg); main channel [−BW/2, +BW/2), adjacent channels of the same width immediately next to it; no normalisation; pooled aggregation; floor −300 dB instead of −∞ when the error is exactly zero.

Conventions kept apart. ACPR is leakage (adjacent/main, negative dBc, lower is better). The positive suppression convention (main/adjacent, higher is better) equals −ACPR and is deliberately not reported so that name, formula and direction never disagree. IBE is a power ratio in dB; the amplitude ratio is 10^(IBE/20) and the percentage 100·10^(IBE/20); the three forms are documented, never mixed in one field. Pooled NMSE and the legacy mean-of-segment-dB NMSE are different numbers (test_pooled_nmse_is_not_the_mean_of_per_segment_db).

Explicit statuses, never fake numbers. MetricValue.status is ok only with a finite value; otherwise the reason is stored:

Situation	Status
no reference signal	missing_reference (NMSE, IBE)
reference has zero energy / no in-band power	invalid
NaN/Inf in the signals	invalid
sample_rate_hz, bandwidth_hz or nperseg missing	not_applicable (spectral metrics)
a band lies outside the captured ±fs/2	not_applicable, reason names the band
fewer valid samples than one PSD segment / no bin in a band	not_applicable
computation raised	failed with the exception

ofdm-lte20-evm-v1 (pending cross-validation, hidden in the GUI)

Data-aided RMS EVM and E-UTRA-style ACLR of a capture bound to the ofdm-lte20-v1 reference waveform (CP-OFDM with the LTE 20 MHz numerology and known 64QAM symbols). Procedure, deviations from a standard's EVM definition, error budget and the cross-validation protocol are in docs/protocols/waveform-profiles.md.

Metric	Unit	Definition
EVM_RMS	%	100·sqrt(Σ|Ŝ − S|² / Σ|S|²) over every occupied subcarrier of every complete OFDM symbol, after timing, frequency-offset and per-subcarrier least-squares equalisation from the known symbols
EVM_DB	dB	20·log10(EVM_RMS / 100)
ACLR_L / ACLR_R	dBc	10·log10(P[±20 MHz ± 9 MHz] / P[−9, +9 MHz]) from the Welch PSD at the capture rate (leakage, negative)

Statuses: missing_reference without a waveform binding or when the signal does not correlate with the waveform; not_applicable when the capture rate cannot be converted exactly to 30.72 MS/s, is below it, or (ACLR) does not contain the first adjacent channel (< 58 MS/s). The built-in ten-carrier captures are missing_reference under this profile by design.

Validation status of a profile

MetricProfile.validation says how a profile's numbers have been checked: golden (frozen references of the historical code: legacy-opendpd-v1), analytic (closed-form reference tests: general-spectral-v1), pending_cross_validation (implemented and tested, the independent-backend comparison has not passed: ofdm-lte20-evm-v1) or cross_validated. The service computes and stores every profile; the GUI offers only profiles past pending_cross_validation; opendpd profiles prints the status and opendpd evaluate says so before scoring under a pending profile. Every result of a pending profile carries the limitation "pending cross-validation".

Where the numbers come from

The worker scores the best checkpoint over the test split once at the end of a run (opendpd.services.evaluation.evaluate_all): predictions are computed once and every registered profile is stored under runs/<id>/results/<profile>.json; the configured profile (evaluation.profile_id, default opendpd-spectral-v2) is the primary result.json. Under the legacy profile this reproduces the trainer's own TEST_* log values (test_result_metrics_come_from_the_registry_and_agree_with_the_training_log). opendpd evaluate RUN --profile P re-scores later from the same checkpoint. Display decimation, zoom or chart filtering never touch these files: charts are produced from separate, decimated data and formal metrics are only ever computed here.

Comparability

opendpd.core.metrics.incompatibilities(a, b) lists why two results must not be ranked against each other: different profile/version, evidence type (pa_modeling vs dpd_surrogate vs dpd_measured), dataset, preprocessing version, split protocol, evaluated split, reference kind, reference gain (the operating point), PA surrogate (for simulated DPD results: the weight hash the DPD was scored through), execution semantics or mock status. Results with an empty list may be ranked; others may only be shown side by side with the reasons (GET /results/compare, the Compare page).

Validated LTE v2 application profiles

ofdm-lte20-evm-v2 uses dataset-sized symmetric Hann segments and ofdm-lte20-evm-auto-v2 uses full-record symmetric Hamming. Both use enclosing FFT-bin integration and guarded frequency-tolerant acquisition. Their 115-item registered comparison passed in five recorded runs against independent MATLAB implementations at the unchanged budgets; evidence. They require a binding to the known periodic waveform and do not establish 3GPP conformance. The old v1 profile stays pending with its original failed record.

Studio, CLI and SDK resolve profiles through opendpd.core.metric_profiles. It extends the application catalogue while delegating every existing profile to the unchanged implementation. Arena continues to import its frozen catalogue from opendpd.core.metrics; none of its fingerprinted source files, sealed reference artifacts or scoring definitions changes.