Machine learning / snow depth · leave-one-site-out
SnowEx Model Results.
Every model, scored on ground it never saw.
Training, results and predictions for each run.
- RUNS
- 4
- SITES PREDICTED
- 7
- BEST UNSEEN-SITE RMSE · 30 MRandom forest · mean of 7 held-out sites
- 0.49 m
Baselines4 runs
Run / 01 · 2026-10-11
Linear regression
Ordinary least squares, the simplest model on the ladder. Same data and evaluation as the Random Forest Baseline: each site is scored by a model trained only on the other six, Grand Mesa's 2017 surveys are excluded.
- Unseen sites RMSE · 30 m
- 0.58 m
- Error vs predicting the mean
- +7%
- Unseen sites median R² · 30 m
- -0.41
- Old-style R² · 30 m
- 0.34
Key findings
- On unseen sites at 30 m the RMSE is 0.58 m, 7% higher than predicting the training sites' mean depth (0.56 m). It beats the mean at 4 of 7 sites.
- Across block sizes the unseen-site RMSE runs from 0.52 to 0.64 m.
- Scored the old way, on a random pixel split, R² is 0.34 at 30 m.
- With all 31 inputs, least squares is unstable: each amplitude ratio is exactly amplitude 2 minus amplitude 1, and the fitted model gave errors of thousands of metres at a new site. The three ratio inputs are left out for this model.
One map per site, from models that never saw that site (random forest). Sites are listed from lowest to highest error. RMSE is the typical error. Bias is the average error: right of centre means the model predicts too much snow. Pattern r is how well the deep and shallow places line up (1 is perfect). Open 3D shows LiDAR, the predictions and the error on the terrain.
| Held-out site | RMSE | Bias | Pattern r |
|---|
Each input’s share of the model’s total importance (0–1; all inputs sum to 1). Measured on the held-out sites by shuffling one input and seeing how much the error rises, averaged over linear regression. Bars are scaled to the most important input.
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Evaluation
Leave-one-site-out: each of the 7 labelled sites is predicted by models trained only on the other 6, so every number and map shows generalisation to unseen ground. Reynolds Creek has no snow-depth labels and is not used.
Data
SnowEx Field Atlas v1.0.05 (enrichment 3.3.0). One sample = one block of 3 m cells (3, 6, 15, 30, 60 or 99 m; cell means, kept when >=80% of cells have snow depth and radar geometry) x one LiDAR snow-depth date x one UAVSAR interferometric pair within 5 days. 10 site-surveys on 8 LiDAR dates across 7 sites. Grand Mesa's five 2017 surveys are excluded (about 70% of those rows have no canopy input and most of their radar scenes are HH-only), so Grand Mesa is trained and scored on its 2020 survey only.
This version of the archive on GitHub ↗
Sampling
Up to 60,000 training rows per site, sites weighted equally so Grand Mesa cannot dominate. The held-out site is never used for fitting, tuning or early stopping.
Reference
Every model is compared with predicting the training mean; the old evaluation style (random 80/20 pixel split, all sites pooled) is run on the same rows for contrast.
Inputs
- Radar, per polarisation HH / HV / VV: amplitude pass 1 and 2 (dB), amplitude ratio (dB), coherence, unwrapped phase re-referenced to each scene's median, wrapped phase as cos/sin
- Geometry: local and flat incidence angle
- Terrain: elevation, slope, aspect as sin/cos
- Canopy: vegetation height, canopy fraction (nearest vegetation date)
- Timing: radar pair length, LiDAR-radar gap (days)
Models
- Linear regression: ordinary least squares on 28 inputs (the three amplitude ratios are left out because each equals amplitude 2 minus amplitude 1). Missing inputs are filled with the training median; inputs are standardised.
scripts/ml/ in the cryogars-atlas-results repository.
RMSE by block size · metres, lower is better
Unseen sites, averaged over the held-out sites. The lowest unseen-site error in each row is marked.
| Block | Predicting the mean | Unseen sites | Random pixel split (old style) | ||
|---|---|---|---|---|---|
| Linear regression | without radar | without elevation | |||
| 3m | 0.63 | 0.64 | 0.62 | 0.60 | 0.55 |
| 6m | 0.61 | 0.62 | 0.60 | 0.58 | 0.51 |
| 15m | 0.58 | 0.59 | 0.57 | 0.55 | 0.47 |
| 30m | 0.56 | 0.58 | 0.55 | 0.53 | 0.44 |
| 60m | 0.53 | 0.54 | 0.52 | 0.50 | 0.39 |
| 99m | 0.51 | 0.52 | 0.50 | 0.48 | 0.37 |
RMSE at each held-out site · 30 m
Sorted from lowest to highest linear regression error. Bias is the average error; R² compares with that site's own mean depth.
| Held-out site | Predicting the mean | RMSE m | Bias m | R² |
|---|---|---|---|---|
| Linear regression | Linear regression | Linear regression | ||
| Banner Summit | 0.46 | 0.40 | -0.07 | +0.11 |
| Dry Creek | 0.51 | 0.50 | +0.33 | -0.51 |
| Cameron Pass | 0.46 | 0.50 | -0.13 | -0.25 |
| Mores Creek | 0.58 | 0.53 | -0.38 | -0.41 |
| Little Cottonwood Canyon | 0.84 | 0.68 | -0.28 | -0.14 |
| Grand Mesa | 0.42 | 0.71 | +0.61 | -33.43 |
| Fraser | 0.62 | 0.72 | +0.64 | -3.14 |
Other scores by block size
| Block | Linear regression · unseen sites | Random split R² | ||
|---|---|---|---|---|
| vs mean | median R² | mean R² | ||
| 3m | +4% | -0.20 | -2.21 | 0.25 |
| 6m | +4% | -0.24 | -2.61 | 0.27 |
| 15m | +5% | -0.32 | -3.70 | 0.30 |
| 30m | +7% | -0.41 | -5.40 | 0.34 |
| 60m | +5% | -0.49 | -6.48 | 0.40 |
| 99m | +6% | -0.60 | -9.73 | 0.42 |
Limits
- Labels are absolute snow depth while one radar pair measures about a week of change, so terrain and canopy dominate these baselines.
- Mean R² is not a headline number here. R² compares a model with the held-out site's own mean depth, so a site with a narrow range of depths and an error in overall level scores far below zero (Grand Mesa 2020 ≈ −11, Fraser ≈ −2) and dominates the average. RMSE, error vs predicting the mean and median R² are reported instead; mean R² is listed in the full table.
- Values are block means of the 3 m archive, not native-resolution radar.
Run / 02 · 2026-10-11
Decision tree
One regression tree, the building block of the random forest. Same data and evaluation as the Random Forest Baseline: each site is scored by a model trained only on the other six, Grand Mesa's 2017 surveys are excluded.
- Unseen sites RMSE · 30 m
- 0.59 m
- Error vs predicting the mean
- +7%
- Unseen sites median R² · 30 m
- -0.53
- Old-style R² · 30 m
- 0.65
Key findings
- On unseen sites at 30 m the RMSE is 0.59 m, 7% higher than predicting the training sites' mean depth (0.56 m). It beats the mean at 2 of 7 sites.
- Across block sizes the unseen-site RMSE runs from 0.52 to 0.64 m.
- Scored the old way, on a random pixel split, R² is 0.65 at 30 m.
- A single tree is clearly worse than the random forest's 300 trees on the same inputs (0.49 m at 30 m).
One map per site, from models that never saw that site (random forest). Sites are listed from lowest to highest error. RMSE is the typical error. Bias is the average error: right of centre means the model predicts too much snow. Pattern r is how well the deep and shallow places line up (1 is perfect). Open 3D shows LiDAR, the predictions and the error on the terrain.
| Held-out site | RMSE | Bias | Pattern r |
|---|
Each input’s share of the model’s total importance (0–1; all inputs sum to 1). Measured on the held-out sites by shuffling one input and seeing how much the error rises, averaged over decision tree. Bars are scaled to the most important input.
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Evaluation
Leave-one-site-out: each of the 7 labelled sites is predicted by models trained only on the other 6, so every number and map shows generalisation to unseen ground. Reynolds Creek has no snow-depth labels and is not used.
Data
SnowEx Field Atlas v1.0.05 (enrichment 3.3.0). One sample = one block of 3 m cells (3, 6, 15, 30, 60 or 99 m; cell means, kept when >=80% of cells have snow depth and radar geometry) x one LiDAR snow-depth date x one UAVSAR interferometric pair within 5 days. 10 site-surveys on 8 LiDAR dates across 7 sites. Grand Mesa's five 2017 surveys are excluded (about 70% of those rows have no canopy input and most of their radar scenes are HH-only), so Grand Mesa is trained and scored on its 2020 survey only.
This version of the archive on GitHub ↗
Sampling
Up to 60,000 training rows per site, sites weighted equally so Grand Mesa cannot dominate. The held-out site is never used for fitting, tuning or early stopping.
Reference
Every model is compared with predicting the training mean; the old evaluation style (random 80/20 pixel split, all sites pooled) is run on the same rows for contrast.
Inputs
- Radar, per polarisation HH / HV / VV: amplitude pass 1 and 2 (dB), amplitude ratio (dB), coherence, unwrapped phase re-referenced to each scene's median, wrapped phase as cos/sin
- Geometry: local and flat incidence angle
- Terrain: elevation, slope, aspect as sin/cos
- Canopy: vegetation height, canopy fraction (nearest vegetation date)
- Timing: radar pair length, LiDAR-radar gap (days)
Models
- Decision tree: one regression tree, depth at most 10, at least 10 rows per leaf, all 31 inputs.
scripts/ml/ in the cryogars-atlas-results repository.
RMSE by block size · metres, lower is better
Unseen sites, averaged over the held-out sites. The lowest unseen-site error in each row is marked.
| Block | Predicting the mean | Unseen sites | Random pixel split (old style) | ||
|---|---|---|---|---|---|
| Decision tree | without radar | without elevation | |||
| 3m | 0.63 | 0.64 | 0.69 | 0.62 | 0.46 |
| 6m | 0.61 | 0.61 | 0.61 | 0.60 | 0.41 |
| 15m | 0.58 | 0.60 | 0.57 | 0.59 | 0.36 |
| 30m | 0.56 | 0.59 | 0.57 | 0.57 | 0.32 |
| 60m | 0.53 | 0.54 | 0.54 | 0.57 | 0.26 |
| 99m | 0.51 | 0.52 | 0.54 | 0.52 | 0.23 |
RMSE at each held-out site · 30 m
Sorted from lowest to highest decision tree error. Bias is the average error; R² compares with that site's own mean depth.
| Held-out site | Predicting the mean | RMSE m | Bias m | R² |
|---|---|---|---|---|
| Decision tree | Decision tree | Decision tree | ||
| Dry Creek | 0.51 | 0.42 | +0.24 | -0.09 |
| Mores Creek | 0.58 | 0.44 | -0.32 | +0.01 |
| Banner Summit | 0.46 | 0.47 | +0.11 | -0.22 |
| Cameron Pass | 0.46 | 0.56 | -0.19 | -0.53 |
| Grand Mesa | 0.42 | 0.59 | +0.39 | -22.78 |
| Fraser | 0.62 | 0.70 | +0.52 | -2.85 |
| Little Cottonwood Canyon | 0.84 | 0.98 | -0.46 | -1.38 |
Other scores by block size
| Block | Decision tree · unseen sites | Random split R² | ||
|---|---|---|---|---|
| vs mean | median R² | mean R² | ||
| 3m | +1% | -0.21 | -1.71 | 0.47 |
| 6m | 0% | -0.21 | -1.76 | 0.53 |
| 15m | +2% | -0.46 | -2.01 | 0.59 |
| 30m | +7% | -0.53 | -3.98 | 0.65 |
| 60m | 0% | -0.41 | -3.33 | 0.74 |
| 99m | +1% | -0.77 | -4.71 | 0.77 |
Limits
- Labels are absolute snow depth while one radar pair measures about a week of change, so terrain and canopy dominate these baselines.
- Mean R² is not a headline number here. R² compares a model with the held-out site's own mean depth, so a site with a narrow range of depths and an error in overall level scores far below zero (Grand Mesa 2020 ≈ −11, Fraser ≈ −2) and dominates the average. RMSE, error vs predicting the mean and median R² are reported instead; mean R² is listed in the full table.
- Values are block means of the 3 m archive, not native-resolution radar.
Run / 03 · 2026-10-11
Random Forest Baseline
+ gradient boosting
Random forest, with gradient boosting for comparison, predicting LiDAR snow depth from UAVSAR, terrain and canopy at seven SnowEx sites. Each site is scored by models that never saw it. Grand Mesa's 2017 surveys are excluded. This is the reference every later model has to beat.
- Unseen sites RMSE · 30 m
- 0.49 m
- Error vs predicting the mean
- −12%
- Unseen sites median R² · 30 m
- +0.09
- Old-style R² · 30 m
- 0.82
Key findings
- Random pixel splits overstate skill, and more so at coarser blocks: scored the old way, R² rises from 0.63 at 3 m to 0.90 at 99 m, while on unseen sites the median R² stays between −0.14 and +0.09.
- On unseen sites the random forest's RMSE is 0.45–0.57 m, 10–13% lower than predicting the training sites' mean depth.
- Most error is each site's overall snow amount: in the gradient-boosting maps the bias runs from −0.40 m (Little Cottonwood) to +0.66 m (Grand Mesa). The spatial pattern is partly captured: correlation 0.82 at Mores Creek, 0.69 at Dry Creek and 0.64 at Banner Summit, about 0.2 at Fraser and Little Cottonwood.
- The models barely use the radar: at 30 m terrain carries 0.75 of the importance, canopy 0.18 and radar 0.07, and unwrapped phase almost none. Removing every radar input does not raise the random forest's error (0.48 m without, 0.49 m with). One radar pair measures about a week of change, while the labels are absolute depth.
- Gradient boosting is worse than the random forest in this run (RMSE 0.55 m vs 0.49 m at 30 m), mostly at Grand Mesa, so its extra complexity is not justified.
- Grand Mesa's five 2017 surveys are excluded. About 70% of those rows have no canopy input and most of their radar scenes are HH-only; scored on them, the models showed no skill (pattern correlation 0.05 in a 6,000-row check). Excluding them slightly raised the error at the other six sites (0.49 to 0.51 m).
One map per site, from models that never saw that site (random forest). Sites are listed from lowest to highest error. RMSE is the typical error. Bias is the average error: right of centre means the model predicts too much snow. Pattern r is how well the deep and shallow places line up (1 is perfect). Open 3D shows LiDAR, the predictions and the error on the terrain.
| Held-out site | RMSE | Bias | Pattern r | ||
|---|---|---|---|---|---|
| 01 | Dry CreekLiDAR 2020-02-19 · 41,742 blocks | 0.38 m | +0.23 m | 0.67 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
| 02 | Banner SummitLiDAR 2021-03-15 · 185,165 blocks | 0.38 m | +0.22 m | 0.70 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
| 03 | Grand MesaLiDAR 2020-02-01 · 63,006 blocks | 0.40 m | +0.38 m | 0.46 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
| 04 | Mores CreekLiDAR 2021-03-15 · 38,319 blocks | 0.40 m | -0.30 m | 0.86 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
| 05 | Cameron PassLiDAR 2021-03-19 · 23,528 blocks | 0.41 m | -0.14 m | 0.51 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
| 06 | FraserLiDAR 2021-03-19 · 69,190 blocks | 0.63 m | +0.53 m | 0.28 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
| 07 | Little Cottonwood CanyonLiDAR 2021-03-18 · 30,970 blocks | 0.77 m | -0.41 m | 0.17 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
Each input’s share of the model’s total importance (0–1; all inputs sum to 1). Measured on the held-out sites by shuffling one input and seeing how much the error rises, averaged over random forest and gradient boosting. Bars are scaled to the most important input.
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Evaluation
Leave-one-site-out: each of the 7 labelled sites is predicted by models trained only on the other 6, so every number and map shows generalisation to unseen ground. Reynolds Creek has no snow-depth labels and is not used.
Data
SnowEx Field Atlas v1.0.05 (enrichment 3.3.0). One sample = one block of 3 m cells (3, 6, 15, 30, 60 or 99 m; cell means, kept when >=80% of cells have snow depth and radar geometry) x one LiDAR snow-depth date x one UAVSAR interferometric pair within 5 days. 10 site-surveys on 8 LiDAR dates across 7 sites. Grand Mesa's five 2017 surveys are excluded (about 70% of those rows have no canopy input and most of their radar scenes are HH-only), so Grand Mesa is trained and scored on its 2020 survey only.
This version of the archive on GitHub ↗
Sampling
Up to 60,000 training rows per site, sites weighted equally so Grand Mesa cannot dominate. The held-out site is never used for fitting, tuning or early stopping.
Reference
Every model is compared with predicting the training mean; the old evaluation style (random 80/20 pixel split, all sites pooled) is run on the same rows for contrast.
Inputs
- Radar, per polarisation HH / HV / VV: amplitude pass 1 and 2 (dB), amplitude ratio (dB), coherence, unwrapped phase re-referenced to each scene's median, wrapped phase as cos/sin
- Geometry: local and flat incidence angle
- Terrain: elevation, slope, aspect as sin/cos
- Canopy: vegetation height, canopy fraction (nearest vegetation date)
- Timing: radar pair length, LiDAR-radar gap (days)
Models
- Random forest: 300 trees, min_samples_leaf 10, max_features 0.33 (fixed settings)
- Gradient boosting (scikit-learn HistGradientBoosting, 300 iterations, L2 1.0): learning rate 0.05/0.1 and 15/31 leaves chosen by grouped cross-validation over the training sites only
scripts/ml/ in the cryogars-atlas-results repository.
RMSE by block size · metres, lower is better
Unseen sites, averaged over the held-out sites. The lowest unseen-site error in each row is marked.
| Block | Predicting the mean | Unseen sites | Random pixel split (old style) | |||
|---|---|---|---|---|---|---|
| Random forest | Gradient boosting | without radar | without elevation | |||
| 3m | 0.63 | 0.57 | 0.60 | 0.57 | 0.57 | 0.39 |
| 6m | 0.61 | 0.54 | 0.56 | 0.54 | 0.55 | 0.34 |
| 15m | 0.58 | 0.51 | 0.57 | 0.51 | 0.52 | 0.29 |
| 30m | 0.56 | 0.49 | 0.55 | 0.48 | 0.50 | 0.23 |
| 60m | 0.53 | 0.47 | 0.50 | 0.46 | 0.48 | 0.17 |
| 99m | 0.51 | 0.45 | 0.47 | 0.44 | 0.46 | 0.15 |
RMSE at each held-out site · 30 m
Sorted from lowest to highest random forest error. Bias is the average error; R² compares with that site's own mean depth.
| Held-out site | Predicting the mean | RMSE m | Bias m | R² | |||
|---|---|---|---|---|---|---|---|
| Random forest | Gradient boosting | Random forest | Gradient boosting | Random forest | Gradient boosting | ||
| Dry Creek | 0.51 | 0.38 | 0.33 | +0.23 | +0.14 | +0.11 | +0.35 |
| Banner Summit | 0.46 | 0.38 | 0.39 | +0.15 | +0.17 | +0.18 | +0.12 |
| Grand Mesa | 0.42 | 0.41 | 0.75 | +0.38 | +0.66 | -10.71 | -37.81 |
| Mores Creek | 0.58 | 0.42 | 0.43 | -0.32 | -0.31 | +0.10 | +0.07 |
| Cameron Pass | 0.46 | 0.43 | 0.47 | -0.14 | -0.24 | +0.09 | -0.11 |
| Fraser | 0.62 | 0.62 | 0.67 | +0.51 | +0.54 | -2.05 | -2.50 |
| Little Cottonwood Canyon | 0.84 | 0.80 | 0.79 | -0.41 | -0.40 | -0.59 | -0.58 |
Other scores by block size
| Block | Random forest · unseen sites | Gradient boosting · unseen sites | Random split R² | ||||
|---|---|---|---|---|---|---|---|
| vs mean | median R² | mean R² | vs mean | median R² | mean R² | ||
| 3m | −9% | +0.09 | -1.15 | −3% | +0.04 | -2.34 | 0.63 |
| 6m | −11% | +0.09 | -1.16 | −8% | +0.01 | -1.76 | 0.68 |
| 15m | −12% | +0.09 | -1.39 | 0% | -0.01 | -4.62 | 0.75 |
| 30m | −12% | +0.09 | -1.84 | +1% | -0.11 | -5.78 | 0.82 |
| 60m | −13% | +0.03 | -2.51 | −4% | -0.18 | -5.65 | 0.89 |
| 99m | −12% | -0.14 | -3.13 | −8% | -0.27 | -5.15 | 0.90 |
Limits
- Labels are absolute snow depth while one radar pair measures about a week of change, so terrain and canopy dominate these baselines.
- Mean R² is not a headline number here. R² compares a model with the held-out site's own mean depth, so a site with a narrow range of depths and an error in overall level scores far below zero (Grand Mesa 2020 ≈ −11, Fraser ≈ −2) and dominates the average. RMSE, error vs predicting the mean and median R² are reported instead; mean R² is listed in the full table.
- Each block appears once per matched radar pair; prediction maps average those pairs.
- Values are block means of the 3 m archive, not native-resolution radar.
Run / 04 · 2026-10-08
Random Forest Baseline · first run
+ gradient boosting
The first run of this baseline, from 2026-10-08. Same models and inputs as the current run, but the data still included Grand Mesa's five 2017 surveys. The 2026-10-11 run replaces it; this one is kept as a record.
- Unseen sites RMSE · 30 m
- 0.53 m
- Error vs predicting the mean
- −12%
- Unseen sites median R² · 30 m
- +0.09
- Old-style R² · 30 m
- 0.71
Key findings
- Random pixel splits overstate skill, and more so at coarser blocks: R² rises from 0.54 to 0.78 the old way while it falls on unseen sites.
- Most error is each site's overall snow amount (bias up to ±0.5 m). The spatial pattern is partly captured: correlation 0.5–0.75 at Mores Creek, Dry Creek and Banner Summit, about 0.1 at Grand Mesa.
- The models barely use the radar: terrain and canopy carry almost all of the importance, and re-referenced unwrapped phase almost none. One radar pair measures about a week of change, while the labels are absolute depth.
- Elevation does not transfer between sites: at Banner Summit the model over-predicts valleys by 0.3 m and under-predicts ridges, because each site has its own snow-elevation gradient.
- Random forest and gradient boosting are statistically tied, so the extra complexity of boosting is not justified here.
One map per site, from models that never saw that site (random forest). Sites are listed from lowest to highest error. RMSE is the typical error. Bias is the average error: right of centre means the model predicts too much snow. Pattern r is how well the deep and shallow places line up (1 is perfect). Open 3D shows LiDAR, the predictions and the error on the terrain.
| Held-out site | RMSE | Bias | Pattern r | ||
|---|---|---|---|---|---|
| 01 | Cameron PassLiDAR 2021-03-19 · 23,528 blocks | 0.37 m | -0.04 m | 0.61 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
| 02 | Banner SummitLiDAR 2021-03-15 · 185,165 blocks | 0.38 m | +0.21 m | 0.70 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
| 03 | Dry CreekLiDAR 2020-02-19 · 41,742 blocks | 0.38 m | +0.23 m | 0.66 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
| 04 | Grand MesaLiDAR 2020-02-01 · 63,006 blocks | 0.40 m | +0.38 m | 0.46 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
| 05 | Mores CreekLiDAR 2021-03-15 · 38,319 blocks | 0.41 m | -0.31 m | 0.86 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
| 06 | FraserLiDAR 2021-03-19 · 69,190 blocks | 0.63 m | +0.53 m | 0.29 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
| 07 | Little Cottonwood CanyonLiDAR 2021-03-18 · 30,970 blocks | 0.72 m | -0.38 m | 0.29 | Open 3D GeoTIFF 30 mGeoTIFF 99 m |
Each input’s share of the model’s total importance (0–1; all inputs sum to 1). Measured on the held-out sites by shuffling one input and seeing how much the error rises, averaged over random forest and gradient boosting. Bars are scaled to the most important input.
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Share by input type
Terrain
Canopy
Radar geometry
Radar · HH
Radar · HV
Radar · VV
Timing
Evaluation
Leave-one-site-out: each of the 7 labelled sites is predicted by models trained only on the other 6, so every number and map shows generalisation to unseen ground. Reynolds Creek has no snow-depth labels and is not used.
Data
SnowEx Field Atlas v1.0.05 (enrichment 3.3.0). One sample = one block of 3 m cells (3, 6, 15, 30, 60 or 99 m; cell means, kept when >=80% of cells have snow depth and radar geometry) x one LiDAR snow-depth date x one UAVSAR interferometric pair within 5 days. 13 LiDAR dates across 7 sites.
This version of the archive on GitHub ↗
Sampling
Up to 60,000 training rows per site, sites weighted equally so Grand Mesa cannot dominate. The held-out site is never used for fitting, tuning or early stopping.
Reference
Every model is compared with predicting the training mean; the old evaluation style (random 80/20 pixel split, all sites pooled) is run on the same rows for contrast.
Inputs
- Radar, per polarisation HH / HV / VV: amplitude pass 1 and 2 (dB), amplitude ratio (dB), coherence, unwrapped phase re-referenced to each scene's median, wrapped phase as cos/sin
- Geometry: local and flat incidence angle
- Terrain: elevation, slope, aspect as sin/cos
- Canopy: vegetation height, canopy fraction (nearest vegetation date)
- Timing: radar pair length, LiDAR-radar gap (days)
Models
- Random forest: 300 trees, min_samples_leaf 10, max_features 0.33 (fixed settings)
- Gradient boosting (scikit-learn HistGradientBoosting, 300 iterations, L2 1.0): learning rate 0.05/0.1 and 15/31 leaves chosen by grouped cross-validation over the training sites only
scripts/ml/ in the cryogars-atlas-results repository.
RMSE by block size · metres, lower is better
Unseen sites, averaged over the held-out sites. The lowest unseen-site error in each row is marked.
| Block | Predicting the mean | Unseen sites | Random pixel split (old style) | |||
|---|---|---|---|---|---|---|
| Random forest | Gradient boosting | without radar | without elevation | |||
| 3m | 0.69 | 0.61 | 0.62 | 0.62 | 0.65 | 0.46 |
| 6m | 0.66 | 0.60 | 0.60 | 0.60 | 0.63 | 0.42 |
| 15m | 0.63 | 0.56 | 0.57 | 0.57 | 0.59 | 0.37 |
| 30m | 0.60 | 0.53 | 0.54 | 0.54 | 0.56 | 0.31 |
| 60m | 0.56 | 0.50 | 0.50 | 0.51 | 0.53 | 0.25 |
| 99m | 0.53 | 0.47 | 0.47 | 0.49 | 0.50 | 0.25 |
RMSE at each held-out site · 30 m
Sorted from lowest to highest random forest error. Bias is the average error; R² compares with that site's own mean depth.
| Held-out site | Predicting the mean | RMSE m | Bias m | R² | |||
|---|---|---|---|---|---|---|---|
| Random forest | Gradient boosting | Random forest | Gradient boosting | Random forest | Gradient boosting | ||
| Banner Summit | 0.45 | 0.38 | 0.39 | +0.14 | +0.11 | +0.20 | +0.16 |
| Dry Creek | 0.54 | 0.38 | 0.31 | +0.23 | +0.12 | +0.12 | +0.41 |
| Cameron Pass | 0.45 | 0.41 | 0.41 | -0.04 | -0.07 | +0.17 | +0.15 |
| Mores Creek | 0.55 | 0.42 | 0.43 | -0.33 | -0.31 | +0.09 | +0.05 |
| Fraser | 0.66 | 0.62 | 0.62 | +0.51 | +0.49 | -2.03 | -2.00 |
| Little Cottonwood Canyon | 0.82 | 0.75 | 0.76 | -0.38 | -0.39 | -0.41 | -0.44 |
| Grand Mesa | 0.71 | 0.77 | 0.83 | +0.27 | +0.36 | -0.22 | -0.42 |
Other scores by block size
| Block | Random forest · unseen sites | Gradient boosting · unseen sites | Random split R² | ||||
|---|---|---|---|---|---|---|---|
| vs mean | median R² | mean R² | vs mean | median R² | mean R² | ||
| 3m | −12% | +0.12 | -0.11 | −12% | +0.13 | -0.11 | 0.54 |
| 6m | −11% | +0.10 | -0.17 | −11% | +0.15 | -0.16 | 0.58 |
| 15m | −12% | +0.11 | -0.20 | −11% | +0.12 | -0.23 | 0.65 |
| 30m | −12% | +0.09 | -0.30 | −12% | +0.05 | -0.30 | 0.71 |
| 60m | −12% | +0.06 | -0.41 | −11% | -0.07 | -0.44 | 0.78 |
| 99m | −12% | +0.06 | -0.51 | −13% | -0.05 | -0.44 | 0.77 |
Limits
- Labels are absolute snow depth while one radar pair measures about a week of change, so terrain and canopy dominate these baselines.
- Mean R² is pulled down by Fraser (≈ −2); medians and bias-removed errors give the fuller picture.
- Each block appears once per matched radar pair; prediction maps average those pairs.
- Values are block means of the 3 m archive, not native-resolution radar.
Leave-one-site-out.
Every number and map comes from models that never saw that site.
About these results
Why leave-one-site-out.
Neighbouring pixels share information, so a random train/test split lets a model memorise each site and look far better than it is. Holding out a whole site asks the real question: does the model work somewhere new? Earlier evaluations in this project used random splits; every run here is scored on unseen sites.
What the numbers mean.
R² compares a model with the held-out site's own mean depth, so a negative value means it would have been better to know that site's average. "Error vs predicting the mean" compares RMSE with predicting the training sites' average, which is all a model without site knowledge could do. Pattern r is the correlation between predicted and LiDAR depth across the site. R² is very sensitive to a site's overall level, so the headline numbers are RMSE and the error against predicting the mean.