Machine learning / snow depth  ·  leave-one-site-out

SnowEx Model Results.

Every model, scored on ground it never saw.
Training, results and predictions for each run.

RUNS
4
SITES PREDICTED
7
BEST UNSEEN-SITE RMSE · 30 MRandom forest · mean of 7 held-out sites
0.49 m
  • Baselines4 runs

Run / 03  ·  2026-10-11

Random Forest Baseline

Random forest
+ gradient boosting

Random forest, with gradient boosting for comparison, predicting LiDAR snow depth from UAVSAR, terrain and canopy at seven SnowEx sites. Each site is scored by models that never saw it. Grand Mesa's 2017 surveys are excluded. This is the reference every later model has to beat.

Unseen sites RMSE · 30 m
0.49 m
Error vs predicting the mean
−12%
Unseen sites median R² · 30 m
+0.09
Old-style R² · 30 m
0.82
RMSE (m) · random forest · lower is better0.00.10.20.30.40.50.60.73m6m15m30m60m99mblock sizePredicting the mean, 3m blocks: RMSE 0.63 mPredicting the mean, 6m blocks: RMSE 0.61 mPredicting the mean, 15m blocks: RMSE 0.58 mPredicting the mean, 30m blocks: RMSE 0.56 mPredicting the mean, 60m blocks: RMSE 0.53 mPredicting the mean, 99m blocks: RMSE 0.51 mUnseen sites, 3m blocks: RMSE 0.57 mUnseen sites, 6m blocks: RMSE 0.54 mUnseen sites, 15m blocks: RMSE 0.51 mUnseen sites, 30m blocks: RMSE 0.49 mUnseen sites, 60m blocks: RMSE 0.47 mUnseen sites, 99m blocks: RMSE 0.45 mRandom pixel split, 3m blocks: RMSE 0.39 mRandom pixel split, 6m blocks: RMSE 0.34 mRandom pixel split, 15m blocks: RMSE 0.29 mRandom pixel split, 30m blocks: RMSE 0.23 mRandom pixel split, 60m blocks: RMSE 0.17 mRandom pixel split, 99m blocks: RMSE 0.15 mPredicting the meanUnseen sitesRandom pixel split
Typical error in metres at each block size. Scored the old way, on a random pixel split, the error looks much smaller than it is on sites the model never saw. The dashed line is the error of predicting the training sites' mean depth, which a model has to beat. Same rows, same model.

Key findings

  • Random pixel splits overstate skill, and more so at coarser blocks: scored the old way, R² rises from 0.63 at 3 m to 0.90 at 99 m, while on unseen sites the median R² stays between −0.14 and +0.09.
  • On unseen sites the random forest's RMSE is 0.45–0.57 m, 10–13% lower than predicting the training sites' mean depth.
  • Most error is each site's overall snow amount: in the gradient-boosting maps the bias runs from −0.40 m (Little Cottonwood) to +0.66 m (Grand Mesa). The spatial pattern is partly captured: correlation 0.82 at Mores Creek, 0.69 at Dry Creek and 0.64 at Banner Summit, about 0.2 at Fraser and Little Cottonwood.
  • The models barely use the radar: at 30 m terrain carries 0.75 of the importance, canopy 0.18 and radar 0.07, and unwrapped phase almost none. Removing every radar input does not raise the random forest's error (0.48 m without, 0.49 m with). One radar pair measures about a week of change, while the labels are absolute depth.
  • Gradient boosting is worse than the random forest in this run (RMSE 0.55 m vs 0.49 m at 30 m), mostly at Grand Mesa, so its extra complexity is not justified.
  • Grand Mesa's five 2017 surveys are excluded. About 70% of those rows have no canopy input and most of their radar scenes are HH-only; scored on them, the models showed no skill (pattern correlation 0.05 in a 6,000-row check). Excluding them slightly raised the error at the other six sites (0.49 to 0.51 m).

Leave-one-site-out.
Every number and map comes from models that never saw that site.

Data: SnowEx Field Atlas v1.0.05 ↗

About these results

Why leave-one-site-out.

Neighbouring pixels share information, so a random train/test split lets a model memorise each site and look far better than it is. Holding out a whole site asks the real question: does the model work somewhere new? Earlier evaluations in this project used random splits; every run here is scored on unseen sites.

What the numbers mean.

R² compares a model with the held-out site's own mean depth, so a negative value means it would have been better to know that site's average. "Error vs predicting the mean" compares RMSE with predicting the training sites' average, which is all a model without site knowledge could do. Pattern r is the correlation between predicted and LiDAR depth across the site. R² is very sensitive to a site's overall level, so the headline numbers are RMSE and the error against predicting the mean.