Skip to main content

H3 datasets

Once a dataset is partitioned and indexed with fused.h3, any HTTP client can read it with GET /h3/query: a point, a bounding box, an H3 cell or a time window comes back as a Parquet file, without running a UDF. fused.h3.query() is a client of this endpoint.

A public dataset needs no credentials, so a browser page can call the endpoint directly. The AIS map loads every ship position report on screen this way, one /h3/query request per map tile.

Query a dataset​

GET /h3/query

Scope: public or authorized. With no credentials, only a public dataset at a literal s3:// path whose files are publicly readable can be read (403 otherwise). With credentials, path resolves to your environment's own dataset, else to the public one.

Returns the matching rows as a Parquet file.

curl -G "https://www.fused.io/server/v1/h3/query" \
  --data-urlencode "path=s3://fused-asset/data/ais/ais_week_h3_v2/" \
  -d min_lng=-74.10 -d min_lat=40.55 -d max_lng=-73.90 -d max_lat=40.72 \
  -d time_min=2024-07-03T12:00:00 -d time_max=2024-07-03T12:59:59 \
  -d columns=MMSI -d columns=VesselName -d columns=BaseDateTime \
  -d columns=LAT -d columns=LON -d columns=SOG \
  -o harbor.parquet

This reads the 6,214 position reports in New York Harbor between 12:00 and 13:00 UTC on July 3, 2024, from the public AIS dataset.

Query parameters

ParameterTypeRequiredDescription
pathstringYesThe indexed dataset, as passed to fused.h3.index()
min_lng, min_lat, max_lng, max_latnumberNoA bounding box. Give all four.
lat, lngnumberNoA point. Give both.
cellstringNoAn H3 cell id as a hex string. Returns every row in that cell, or in its descendants when cell is coarser than the dataset.
hex_min, hex_maxstringNoAn inclusive range of raw H3 cell ids, each as a hex string
time_min, time_maxstringNoAn inclusive ISO datetime window. Omitted, the whole time range is read.
partitionstringNoJSON object mapping a partition column to a value, or a list of values that any may match. See Filter by partition.
partition_rangestringNoJSON object mapping a partition column to an inclusive [min, max], with null for an open end
columnsstringNoA column to return. Repeat the parameter for several. Omitted, every column is returned.
formatstringNoWhat to return. Leave it out to get the matching rows. Use stats or files to inspect a query without downloading any rows: stats returns JSON with the min and max of each column listed in columns (required with stats), for each row group (block of rows in a file) the query matches. See Stats format. files returns JSON listing the files the query would read, with their row counts. See Files format.
max_row_groupsintegerNoRow groups per page, from 1 to 16,384. At most 4,096 when returning rows. Default 1,024.
cursorstringNoThe cursor from the previous page. See Page through large results.

Pass at most one spatial selector: the bounding box, the point, cell, or hex_min and hex_max. With none, the query reads the whole dataset, narrowed only by time and partition.

Filter by partition​

partition and partition_range select folders written with partition_by in fused.h3.partition(), for example partition={"date": "2024-07-03"} or partition_range={"date": ["2024-07-01", "2024-07-07"]}. A folder must match every column named. A date column matches a folder whose period (YYYY, YYYY-MM or YYYY-MM-DD) overlaps the value or range.

Page through large results​

A large result comes back one page at a time. When more remain, the response carries a cursor: the X-Fused-Next-Cursor header when returning rows, or the next_cursor field of the body for format=stats. Send it back as cursor, with the same query, until a response has none. format=files always answers in one page.

Stats format​

format=stats returns JSON with min and max values per row group, without reading any rows. columns is required, and each column must be a number, date, timestamp or string column of the matched files, other than the H3 column. The body has h3_resolution, partitions (the folders of this page's row groups), next_cursor, and row_groups: parallel arrays h3_min and h3_max (hex strings), partition (an index into partitions) and stats ({column: {min: [...], max: [...]}}), one entry per row group.

Files format​

format=files returns JSON describing the indexed files that overlap the query, without opening any. The body has level ("file"), h3_resolution (the resolution of the data cells), cell_resolution (the resolution of the H3 cells that name the files), cells (the H3 cell, in lowercase hex, that names each file) and row_counts: parallel arrays total (each file's rows in the matched folders) and window (only the rows in folders the time window overlaps). columns, max_row_groups and cursor are ignored. time_min and time_max narrow the answer only when the time column is a date partition column.

Errors​

StatusWhen
400More than one spatial selector, an incomplete one (for example lat without lng), or an invalid cell id
400time_min or time_max on a dataset with no time column
400partition or partition_range that is not a JSON object, names a column that is not a partition column, or holds a value of the wrong type
400max_row_groups above 4,096 when returning rows
400format=stats with no columns, with the H3 column, with a column that no matched file has, or with a column type it cannot summarize
400format=files selecting more than 2,048 folders
400A malformed cursor, or one from a different query
403No credentials, and path is not a public s3:// dataset
404You can read no dataset at path
409The dataset has never finished indexing, or the cursor predates a re-index
422A name in columns is missing from the indexed files, max_row_groups is above 16,384, or (format=files) an indexed file is not named by an H3 cell

See also​