Skip to content

Fixed validation depth diagnostics

evaluate_depth_baselines scores several predictions on the same validation patches and reports MAE, RMSE, signed bias, and valid support independently for each depth. Configured evaluations include physical depth coordinates, sampled indices, and the seed in their report. Invalid GLORYS cells are excluded; configured evaluations fail if predictions are nonfinite on valid targets. The normal validation dataloader remains shuffled; this command creates a separate deterministic Subset using --seed.

For already exported arrays:

/work/envs/depth/bin/python -m depth_recon.scripts.evaluate_depth_baselines \
  --target-npz target.npz --mask-npz target_mask.npz \
  --method climatology=climatology.npz --method unet=unet.npz \
  --output-json depth_metrics.json

For configured checkpoints, use NAME|CONFIG|CHECKPOINT and the evaluator builds the configured validation dataset and model without training:

/work/envs/depth/bin/python -m depth_recon.scripts.evaluate_depth_baselines \
  --configured-method 'unet|unet_training.yaml|unet.ckpt' \
  --configured-method 'depthdif|diffusion_training.yaml|depthdif.ckpt' \
  --sample-count 32 --seed 7 --batch-size 4 --device cuda --variable temperature \
  --output-json depth_metrics.json --output-npz depth_arrays.npz

The configured mode also scores the dataset's normalized climatology channel when present. All methods use the same sampled validation indices and output temperature metrics in degrees Celsius. Use --variable salinity for salinity models or the salinity output of a joint model. Supply the saved training super-config corresponding to each checkpoint; weights load strictly. Target values and masks must match across methods. This command performs inference only.

Fit the optional background from the training years, explicitly excluding the validation year:

/work/envs/depth/bin/python -m depth_recon.data.fit_climatology \
  --root /work/data/OceanVariableReconstruction \
  --output /absolute/path/to/climatology.npz --val-year 2016 \
  --spatial-stride 4 --field both

This streams dates into monthly spatial means. --spatial-stride reduces the stored spatial resolution; nearest-neighbour sampling restores patch dimensions. An unobserved monthly cell falls back to the same cell's annual training mean, then the training mean at that depth. Entirely unsupported depths retain zero observation counts and an explicitly recorded placeholder to preserve channel alignment. The loader rejects valid targets at those depths, so placeholders cannot silently become supervised climatology predictions. Source dates, grid, depth coordinates, and excluded year are stored with the artifact and validated when the dataset opens it. Temperature and salinity artifacts are supported individually or together. Full-resolution fitting and large joint artifacts require more memory; the fitter does not train a model.

For a fresh diffusion model, add:

data:
  dataset:
    wet_domain:
      enabled: true
      reference_date: null  # earliest GLORYS reference mask; no target values used
    climatology:
      enabled: true
      path: /absolute/path/to/climatology.npz
model:
  climatology_residual: true

The model subtracts the background in the existing normalized units, conditions on it, and adds it back to its predictions. There is no per-depth rescaling or new loss term. Physical wet masks, observation masks, and supervised target masks remain distinct. The nine-profile plot now uses the dataset's actual depths; without physical coordinates, it explicitly labels the axis as depth bands.