Every satellite agriculture company runs into the same wall: the sensor that reads your field only works when the sky is clear. This is our engineering plan for getting a trustworthy crop-health signal to a farmer even in a Finish November — from squeezing every clear pixel out of optical imagery, through the time-series methods that reconstruct the missing days, to radar that ignores clouds entirely.
Why clouds are the whole problem
The optical indices we build our reports on — NDVI, NDWI, NDRE, SAVI and the rest — are all ratios of reflected sunlight in specific wavelengths. A cloud sits between the crop and the satellite and reflects that sunlight straight back before it ever touches the field. There is no crop signal underneath a cloud to recover; the pixel is simply blind.
Sentinel-2 revisits the same field about every 5 days with its two satellites. In Mediterranean summer that is plenty — most passes are clear. In Northern Europe in winter, cloud cover runs 60–90% of days, so a whole month of 5-day passes can arrive with every single image unusable. That is the reliability gap that quietly kills subscription trust: a farmer paying monthly does not care why the image is white, only that it is.
The fix is not one silver bullet. It's a cascade of techniques, each catching the cases the previous one missed, ending with radar that never sees a cloud at all.
Each rung catches the cloudy cases the previous one can't — ending with radar, which never sees a cloud at all.
The four ways to beat the clouds
Here are the four approaches that matter most, in the order a good pipeline reaches for them.
Look more often
Combine several satellites into one stream so a clear sky is far likelier on any given date.
The simplest defence against clouds is more chances at a clear sky. One satellite passing every five days gives you only a handful of looks per month; if the cloudy days line up with the pass days, you get nothing. So providers combine several sources: Sentinel-2 and the free American Landsat satellites see the field in a similar way, and stacking them roughly doubles how often it's photographed. For near-certainty, commercial constellations such as Planet image the same spot every single day, at a cost.
There's a small catch: two satellites measure the same field slightly differently, because their sensors respond to colour in subtly different ways. Before mixing them you harmonise with a simple per-band correction — a straight-line rescale of one sensor to match the other:
where a and b are a published gain and offset per band (NASA's Harmonized Landsat–Sentinel, "HLS", provides them). After this, the two sensors line up to within a rounding error and can be treated as one continuous record.
pystac-client + stackstac (or odc-stac), handle rasters with rioxarray/xarray, and reach Sentinel data via sentinelhub-py. In MATLAB, the Mapping Toolbox (readgeoraster) reads the imagery and the Image Processing Toolbox aligns the grids so both sensors overlay pixel-for-pixel.Don't throw away the half-cloudy pictures
Mask out only the cloudy pixels, then stitch the clear pieces of several partial images into one clean map.
A picture that's 40% cloud usually gets discarded — but the other 60% is perfectly clear, and clouds drift, so the clear half of today's image rarely sits over the same spot as next week's. The fix is compositing. Two steps make it work. First, masking: Sentinel-2 ships a "scene classification" layer that labels every pixel — vegetation, soil, water, cloud, shadow, snow — so you drop anything tagged cloud or shadow. Second, choosing a value for each spot from the clear observations that remain. The classic rule for a vegetation index is to keep the highest reading in the window:
This "maximum-value composite" works because leftover haze and thin cloud always drag a vegetation index down, so the highest clear reading rejects the contaminated ones — the same rule behind NASA's MODIS composites. When you'd rather have the freshest data than the greenest, swap it for a weighted average favouring the most recent, cleanest pixels.
xarray stack — mask with .where(), reduce with .max("time") or .median("time"); s2cloudless adds a cloud-probability mask when the built-in one misses thin cloud. In MATLAB, stack the scenes into a 3-D array and reduce with max/median along the third dimension, tidying masks with the Image Processing Toolbox.Fill the gaps using the season's curve
A crop's health follows a smooth seasonal arc, so a clouded-out day can be estimated from the clear days around it.
Even after combining satellites and compositing, some dates will have no clear view at all. But a crop doesn't change randomly day to day — it follows a smooth arc: greening up, peaking, then ripening and fading. When a day is clouded out, you estimate what its health would have been from the clear readings on either side and the shape of the whole season. A handful of named methods do this, in rising order of sophistication:
- Linear interpolation — draw a straight line between the last clear day before the gap and the first clear day after. Honest for short gaps, blind to curvature.
- Savitzky–Golay smoothing — slide a small window along the season and fit a gentle curve. Unlike a moving average it keeps the height and timing of the peak, so it doesn't flatten the greenup a farmer is paying to catch.
- The Whittaker smoother — fills gaps and smooths in one step, balancing "stay close to the readings" against "keep the curve gentle." A weight of zero on cloudy days lets them be reconstructed while clear days anchor the shape.
- Harmonic (Fourier) regression — fit the season's rise and fall as a wave, then read off an expected value for any date at all, even one that was never clear.
That last idea is worth showing, because it directly answers "it's cloudy today — what should the reading be?" You model the seasonal curve as a sum of waves and fit it through the clear points:
Here T is the season length and just two or three terms (k = 1, 2, 3) capture the greenup-peak-senescence shape. Fit on the clear days, evaluate on the cloudy one.
scipy.signal.savgol_filter for Savitzky–Golay, pandas.interpolate() for gaps, the whittaker-eilers package for the Whittaker smoother, and numpy.polyfit/statsmodels for the harmonic fit. In MATLAB, sgolayfilt (Signal Processing Toolbox), fillmissing/interp1, and the Curve Fitting Toolbox's Fourier model (fit(t, y, 'fourier2')); the Whittaker smoother is a short sparse-matrix solve in plain MATLAB.When the sky won't clear, switch to radar
Radar sends its own signal through the clouds — then a trained model translates it back into a familiar crop-health estimate.
Sometimes there's a genuinely unbroken stretch of overcast — two solid weeks of grey — and no amount of stitching helps, because there was never a clear look to begin with. This is where the second kind of satellite from that earlier post earns its place. Radar satellites like the European Sentinel-1 are active instruments: rather than waiting for sunlight, they beam their own microwave signal down and measure what bounces back. Those microwaves pass straight through clouds, so a radar satellite sees the field just as clearly in a storm as on a bright day, and it works at night too. It's free, from the same European programme as Sentinel-2.
Radar reads the physical structure and moisture of the canopy rather than colour, through the strength of the echo in two channels called VV and VH. From those you can build a radar version of a vegetation index — most commonly the Radar Vegetation Index:
which climbs from near 0 over bare soil toward 1 over a dense canopy. Raw radar needs a preprocessing chain first (calibration, a speckle filter to tame its grainy texture, terrain correction) — but that heavy lifting can largely be handed to the data provider.
On its own that's harder for a farmer to read. So the smart move, and one the leading agtech providers use, is to translate: on the many days when both a clear optical image and a radar image exist, you train a model to learn the relationship between them — roughly,
Once trained, on a hopelessly cloudy day you feed it the always-available radar and get a familiar-looking crop-health estimate. It's an estimate, not a measurement, so it should be labelled that way in the report — but "we still give you a signal after two cloudy weeks" is exactly the promise optical-only competitors can't make.
esa-snappy) does the full preprocessing if you want control. For the optical-from-radar model, Python's scikit-learn, xgboost or lightgbm cover it; in MATLAB the Statistics and Machine Learning Toolbox (fitrensemble, fitrgp, TreeBagger) does the same, with the Image Processing Toolbox for speckle filtering.Optical vs. radar, at a glance
| Property | Optical Sentinel-2 | Radar / SAR Sentinel-1 |
|---|---|---|
| How it sees | Reads reflected sunlight (passive) | Sends its own signal, reads the echo (active) |
| Clouds | Blocked — no data underneath | Sees straight through them |
| Night | Needs daylight | Works day or night |
| Ease of reading | Intuitive green health maps | Harder — reads structure & moisture |
The four approaches, side by side
| Approach | What it does | Best for |
|---|---|---|
| Look more often | More passes, more chances at a clear sky | Occasional clouds |
| Compositing | Stitches the clear pieces of partial images together | Scattered, patchy cloud |
| Gap-filling | Estimates cloudy days from the seasonal trend | Isolated missing dates |
| Radar (SAR) | Sees through cloud entirely, in any weather | Long overcast stretches |
Honesty is part of the toolkit
One last piece is less technical but just as important: being straight with the farmer. When conditions are genuinely bad, a good service says so — showing the most recent reliable reading with a clear note on its date, rather than dressing up a guess as fresh fact. Trust survives a cloudy fortnight; a silent white image does not.
Put together, these approaches turn clouds from a threat that can sink the service into a manageable, graceful trade-off. On a clear day you get a crisp map. On a patchy one, a composite. On an isolated cloudy date, a well-founded estimate. And through a solid grey winter, radar still keeps the field in view. The sky stops being the thing that decides whether a farmer gets answers.