Skip to main content

Monthly Median Composites, One Machine per Month

Cloud-free monthly composites are usually an overnight batch job. This canvas runs every month in your date range on its own machine, all at once, and fills in a month grid as each one finishes. In the run shown below, 37 monthly composites finished in 6.2 seconds of wall time from 61.5 seconds of total work, spread across 37 machines.

Try it out​

Open the Canvas →

Start small

A 12-month range over a few square kilometres returns in a couple of seconds. Widening to five years barely changes the wall time, since each extra month gets its own machine.

1 · Request​

The widget is a MapLibre map with a Draw box tool. If you don't draw a box, the pipeline uses the current map view. The other inputs are:

  • Start and end date
  • Output: NDVI median or RGB median
  • Max cloud cover (%)
  • Scenes / month: 0 composites every scene in the month; a small number samples that many evenly across it.

Run pipeline sends these as one JSON pipeline_request parameter to the three nodes below, with a run_id timestamp so each run is distinct from the last.

2 · Scene search​

This UDF queries Fused's partitioned Sentinel-2 L2A STAC index (s3://fused-asset/stac/sentinel-2-l2a/partitioned_2015_2025/) for scenes that intersect your AOI, filters by date range and cloud cover, and drops duplicate products.

It builds COG URLs directly from each product URI (mission, date and MGRS tile map to a sentinel-cogs.s3.us-west-2.amazonaws.com path) instead of resolving STAC assets. The output is a (scene_id, datetime, band, url, cloud_cover) table with Red, NIR and SCL for NDVI, plus Green and Blue for RGB.

3 · Monthly fan-out​

This node plans the run from the request alone. It lists the YYYY-MM months in the date range, sets the output size (10 m/px, at most 640 px on the long edge), and hashes the request into a 16-character key. Results are stored under fd://monthly_median/<key>, so the same request always maps to the same location.

Scenes are grouped by month, and each month becomes one call to .map():

# doctest: skip
result = month_udf.map(
    args,                        # one dict per calendar month
    engine="remote",
    max_workers=len(args),       # one machine per month, all at once
    worker_concurrency=1,
    cache_max_age="30d",
    max_retry=1,
).df()

Setting max_workers=len(args) with worker_concurrency=1 gives each month its own machine, so 37 months run on 37 machines in parallel. Each machine:

  1. Reads the month's scenes with windowed rasterio reads on a 12-thread pool.
  2. Masks clouds using the Sentinel-2 SCL band.
  3. Computes NDVI as (NIR − Red) / (NIR + Red).
  4. Takes a nanmedian across the scenes, so a pixel clouded in some passes still gets a value from the others.

It then writes three objects to storage: a colorized PNG for display, a grayscale PNG with the quantized NDVI values, and a JSON status file (scene count, valid-pixel coverage, run time, worker). Nothing is returned through the pipeline; the view reads results straight from storage.

4 · Month grid​

The output node recomputes the same plan, so it knows every month's storage path before any composite exists. It signs a URL for each month and returns a page that polls storage once a second, replacing each "queued" card with its image as soon as that month finishes.

Click a month to enlarge it, and hover over the image to read the median NDVI at that pixel. When the last month lands, the footer shows wall time against total worker time.

note

This node uses cache_max_age="0s" on purpose. The signed URLs expire an hour after signing, so a cached page would eventually show broken images.

About the data​

PropertyValue
ImagerySentinel-2 L2A COGs, AWS Open Data (anonymous)
Scene catalogFused STAC index, partitioned 2015–2025
BandsRed (B04), NIR (B08), SCL, plus Green (B03) and Blue (B02) in RGB mode
Cloud maskingSCL classes 0, 1, 3, 7–11
NDVI display range−0.2 (built-up / bare soil) to 0.8 (dense vegetation)

See also​