Skip to main content

fused.h3

delete_index​

delete_index(path: str, *, base_url: str | None = None) -> None

Delete the index registered for path. The data files are untouched.

Parameters:

  • path (str) – The location an earlier fused.h3.index call registered.
  • base_url (str | None) – Base URL for the API. If None, uses the current environment.

index​

index(
    path: str,
    hex_column: str = "hex",
    time_column: str | None = None,
    *,
    partition_columns: str | list[str] | None = None,
    space: str | None = None,
    time: str | None = None,
    instance_type: InstanceType = "small",
    wait: bool = True,
    overwrite: bool = False,
    base_url: str | None = None,
    **submit_kwargs: Any
) -> dict[str, Any] | JobPool

Register a hive-partitioned parquet tree as a queryable H3 location.

Runs a batch job that walks path and streams its row-group index into the server, via fused.submit.

Parameters:

  • path (str) – Path to the parquet tree (e.g. "s3://my-bucket/my-dataset/"). Every .parquet file under it, at any depth, must carry an integer H3 cell column named hex_column at a single resolution.
  • hex_column (str) – Name of the H3 cell column present in every file. Defaults to "hex", what fused.h3.partition writes.
  • time_column (str | None) – Optional. Names either an in-file timestamp/date column, or a folder level written key=value whose value is a YYYY, YYYY-MM, or YYYY-MM-DD period covering every row under that folder, or an ISO timestamp (an instant). If None, the only date partition column filters by time, and a location without one is timeless.
  • partition_columns (str | list[str] | None) – The key=value folder levels (e.g. scenario=ssp245/date=2050-01/) whose values query can filter by, as a column name or a list of names in folder order. Defaults to the partition_by columns in the tree's _manifest.json (written by fused.h3.partition), else none. Each column is typed "string", "number" or "date" from the manifest, else from its values (all numbers: number; all YYYY, YYYY-MM, YYYY-MM-DD periods or ISO instants: date; otherwise string). Each leaf folder must have exactly one folder per column, in that order; a folder that does not fails the run.
  • space (str | None) – Deprecated alias of hex_column.
  • time (str | None) – Deprecated alias of time_column.
  • instance_type (InstanceType) – The batch instance type the indexing job runs on. Defaults to "small".
  • wait (bool) – If True (the default), tails the job (printing status lines) until it finishes and returns its result. Inside a realtime UDF it gives up watching after ~90 s and raises TimeoutError; the batch job keeps running, so poll it with index_status. If False, returns the JobPool immediately without waiting, so the caller can poll or .wait() on it themselves. The batch job id is printed as soon as the job is submitted either way.
  • overwrite (bool) – If False (the default), re-running is incremental: folders unchanged since the last run are skipped, and only new, changed and removed files are indexed or dropped. A run that was stopped or partly failed resumes where it left off. If True, the whole tree is indexed again from scratch, replacing the previous index.
  • base_url (str | None) – Base URL for the API. If None, uses the current environment.
  • **submit_kwargs (Any) – Additional keyword arguments passed through to fused.submit. max_retry defaults to 0 here (unlike fused.submit's own default) -- pass it explicitly to override.

Returns:

  • dict[str, Any] | JobPool – If wait=True, a dictionary with the indexing results:
  • path: The indexed path.
  • status: "ready".
  • files: Number of parquet files indexed.
  • row_groups: Number of row groups indexed.
  • resolution: The H3 resolution found across the tree.
  • time_min: Earliest time found, or None for a timeless location.
  • time_max: Latest time found, or None for a timeless location.
  • hex_column, time_column: The columns the location was indexed with.
  • partition_columns: Per partition column, its name, type, the number of distinct values and their min and max.
  • dict[str, Any] | JobPool – If wait=False, the JobPool running the indexing job.

Raises:

  • TimeoutError – wait=True inside a realtime UDF and the job was still running after ~90 s. The job is not cancelled; the message names the job id.
  • RuntimeError – wait=True and the indexing job failed. If only some files failed to index, everything else is indexed and queryable; the error names the failed paths, and re-running retries them.

Example:

```py
fused.h3.index(
    path="s3://my-bucket/era5_h3/",
    hex_column="hex",
    partition_columns=["scenario", "date"],
)

index_status​

index_status(path: str, *, base_url: str | None = None) -> dict[str, Any]

Describe the index registered for path.

Parameters:

  • path (str) – The location an earlier fused.h3.index call registered.
  • base_url (str | None) – Base URL for the API. If None, uses the current environment.

Returns:

  • dict[str, Any] – A dictionary with path, status, indexing (True while an index
  • dict[str, Any] – job is running), files, row_groups, resolution, hex_column,
  • dict[str, Any] – time_column, partition_columns (per column: name, type,
  • dict[str, Any] – distinct, min, max), time_min, time_max, indexed_as_of (files modified after this may not be
  • dict[str, Any] – indexed yet), last_run_mode ("incremental" or "overwrite"),
  • dict[str, Any] – last_run_finished_at, and failures (a list of {path, error}
  • dict[str, Any] – for files the last run could not index).

indexed_files​

indexed_files(
    path: str, *, prefix: str | None = None, base_url: str | None = None
) -> Iterator[str]

Iterate the files indexed under path, in path order.

Parameters:

  • path (str) – The location an earlier fused.h3.index call registered.
  • prefix (str | None) – Only yield files under this folder or path prefix.
  • base_url (str | None) – Base URL for the API. If None, uses the current environment.

Yields:

  • str – The full path of each indexed file.

partition​

partition(
    data: Any,
    output: str | Path,
    *,
    hex_column: str | None = None,
    point_column: str | tuple[str, str] | None = None,
    h3_res: int = 7,
    file_res: int = 1,
    fill: str | None = None,
    partition_by: str | list[str] | None = None,
    sort_by: str | list[str] | None = None,
    overwrite: bool = False,
    append: bool = False,
    row_group_size: int = 65536,
    compression: str = "zstd",
    compression_level: int = 5,
    key_columns: list[str] | None = None
) -> None

Write data as hex-sorted H3 parquet files with a _manifest.json.

Parameters:

  • data (Any) – An iterator of pyarrow.RecordBatch, or a pandas.DataFrame, pyarrow.Table or pyarrow.RecordBatch. Everything runs in this process. Paths are not accepted: wrap one with fused.h3.sources.parquet(path).
  • output (str | Path) – Output folder. Files are written as <output>/<col>=<value>/.../<H3 parent cell hex>.parquet, one col=value folder per partition_by column in that order (for example scenario=ssp245/date=2050-01-01/), or <output>/<H3 parent cell hex>.parquet when partition_by is None.
  • hex_column (str | None) – Name of an existing H3 cell column (uint64). Give exactly one of hex_column and point_column.
  • point_column (str | tuple[str, str] | None) – Where each row is, so the cell at h3_res can be computed. Either ("lat_column", "lng_column"), two numeric columns (lat first), or the name of one geometry column holding WKB (binary), such as point_column="geometry". A Point geometry hexes itself; any other geometry gets the cell covering its centroid. Nothing is polyfilled: one row in, one row out. Null and empty geometries raise. The centroid is planar, taken on the coordinates as stored, which must be lon/lat degrees (EPSG:4326); a centroid outside lon -180..180 / lat -90..90 (for example projected coordinates) raises. Geometries that cross the antimeridian or wrap a pole get a centroid on the wrong side, so split them or pass lat/lng columns for those rows. The point columns stay in the files. A geopandas.GeoDataFrame passed as data arrives with its geometry as WKB, and GeoParquet read with fused.h3.sources.parquet is already WKB.
  • h3_res (int) – Resolution of the cell each row gets; this is the hex column.
  • file_res (int) – Resolution of the H3 parent cell that groups rows into files: each row's cell is walked up to its ancestor at this resolution and all rows under one ancestor are written to one file named after it (default 1, about 117k res-7 cells per file; raise it for more, smaller files).
  • fill (str | None) – "nearest" fills every cell at h3_res from the nearest point of a regular lat/lng raster, instead of hexing each row as it is (the default). Each input batch must then hold a complete raster per key tuple, one row per raster point, as fused.h3.sources.era5 yields. Key columns are key_columns, partition_by and every column that is not floating point (such as date or hour); the floating point columns are the values to fill, and none is an error. Needs point_column=("lat", "lng"). One filled batch becomes one batch per H3 parent cell.
  • partition_by (str | list[str] | None) – Column, or list of columns, whose values become nested col=value folders, or None (the default) for no folders. Values are written by type: dates as YYYY-MM-DD, timestamps as ISO 8601 (2050-01-01T06:00:00), integers as plain digits, floats and strings as they print. The type of each column is recorded in _manifest.json. Values may not be null, empty, start with _ or contain /. The columns stay in the files.
  • sort_by (str | list[str] | None) – Extra column(s) to sort by after hex.
  • overwrite (bool) – Replace existing output. Without it, an any existing file in output (a dataset, or leftovers of an aborted run that has no _manifest.json) raises.
  • append (bool) – Add to an existing partitioned dataset instead of failing on it. Every leaf partition folder this run writes replaces that folder in output (so re-running a day is idempotent) and the other folders are left alone; _manifest.json is updated to match. Needs partition_by and an existing _manifest.json whose h3_res, file_res, fill, partition_by, sort_by and partition column types match this call, checked before anything is written. Cannot be combined with overwrite. A run that dies midway leaves the folders it already replaced rewritten only in part, and the manifest unchanged, until it is run again.
  • row_group_size (int) – Rows per parquet row group.
  • compression (str) – Parquet compression codec.
  • compression_level (int) – Compression level (zstd only).
  • key_columns (list[str] | None) – Columns that get dictionary encoding (hex never does). Defaults to the partition_by columns when set, else none; name other low-cardinality columns here. With fill="nearest" the non-floating columns are added automatically.

Raises:

  • TypeError – data is a path or another unsupported type.
  • ValueError – An inconsistent argument, a named column missing from the input (checked on the first batch), input with no rows, an existing output without overwrite or append, append without partition_by, with overwrite, without an existing manifest or with settings that differ from it, or a problem found while bucketing (null coordinates, null hex, null or folder-unsafe partition values).

A file is written as soon as its rows are complete: the first time a (partition values, H3 parent cell) shows up its file is written directly. If a later batch has rows for the same file, the file is read back and merged with them through a temporary folder, so readers may yield any mix of parent cells in any order and only files hit twice are written twice.

Output is touched as late as possible, so a failed run never wipes a good dataset. Any existing file in output without overwrite raises before any work. Otherwise existing output is deleted (overwrite=True) just before the first file is written, after the first batch is proven to have the named columns and a batch with at least one row has been bucketed; files are written under a hidden name and moved into place, and _manifest.json comes last.

query​

query(
    path: str,
    cell: str | None = None,
    *,
    lat: float | None = None,
    lng: float | None = None,
    bbox: tuple[float, float, float, float] | None = None,
    hex_range: tuple[str, str] | None = None,
    time_min: str | None = None,
    time_max: str | None = None,
    partition: dict[str, Any] | None = None,
    partition_range: dict[str, Any] | None = None,
    columns: list[str] | None = None,
    format: str = "parquet",
    max_row_groups: int | None = None,
    base_url: str | None = None
) -> pd.DataFrame | dict[str, Any]

Read the rows of a previously indexed H3 location.

At most one spatial selector may be given: cell (an H3 cell id, as a hex string -- matches every row filed under a descendant of cell, or under cell itself if it is already at the location's resolution), lat+lng (a point), bbox (a (min_lng, min_lat, max_lng, max_lat) tuple), or hex_range (an inclusive raw H3 cell-id range, as a (hex_min, hex_max) tuple of hex strings). Omitting all of them returns every row at the location.

Parameters:

  • path (str) – The location an earlier fused.h3.index call registered.
  • cell (str | None) – An H3 cell id (as a hex string, e.g. "8928308280fffff").
  • lat (float | None) – Latitude of a point selector. Requires lng.
  • lng (float | None) – Longitude of a point selector. Requires lat.
  • bbox (tuple[float, float, float, float] | None) – (min_lng, min_lat, max_lng, max_lat) bounding box selector.
  • hex_range (tuple[str, str] | None) – (hex_min, hex_max) inclusive raw H3 cell-id range.
  • time_min (str | None) – Inclusive lower bound (ISO datetime). Only valid against a location with a time column (a time_column, or a single date partition column).
  • time_max (str | None) – Inclusive upper bound (ISO datetime). Only valid against a location with a time column.
  • partition (dict[str, Any] | None) – Keep only the folders whose partition values match, as {column: value}. A list or tuple of values matches any of them: {"scenario": ["ssp245", "ssp585"], "date": "2050-01"}. A folder must match every column named. A date column matches by overlap: "2050-01" matches a folder holding January 2050.
  • partition_range (dict[str, Any] | None) – Keep only the folders whose partition value lies in an inclusive (min, max) range, as {column: (min, max)}; None leaves an end open: {"date": ("2050-01", None)}. Works for string, number and date columns. A range on the location's time column also narrows the time window.
  • columns (list[str] | None) – Which columns to read. Defaults to all columns. Ignored when format="index".
  • format (str) – "parquet" (the default) returns the matching rows as a DataFrame. "index" instead returns a dictionary describing the matching row groups without reading any row data (ranges, footers, row_groups -- see the server's /h3/query docs).
  • max_row_groups (int | None) – Caps the number of matching row groups the query may read/describe; a query matching more than that raises a 413 before any row data is read. If None (the default), the server's own default applies. Raise this for a query that legitimately spans many row groups (e.g. a wide time window over many cells).
  • base_url (str | None) – Base URL for the API. If None, uses the current environment.

Returns:

  • DataFrame | dict[str, Any] – A pandas DataFrame with the matching rows (format="parquet"), or a
  • DataFrame | dict[str, Any] – dictionary describing the matching row groups (format="index").

Raises:

  • HTTPError – If the API request fails (404 if path has not been indexed, 409 if it is not yet "ready", 400 for an invalid or conflicting selector, against time bounds on a timeless location, or for a partition or partition_range naming a column that is not a partition column or holding a value that does not fit it, 413 if the query matches more row groups than limit and/or max_row_groups allow).

Example:

```py
df = fused.h3.query(
    path="s3://my-bucket/era5_h3/",
    cell="8928308280fffff",
    time_min="2024-01-01T00:00:00",
    time_max="2024-01-31T23:59:59",
)

df = fused.h3.query(
    path="s3://my-bucket/climate_h3/",
    bbox=(-122.5, 37.7, -122.4, 37.8),
    partition={"scenario": "ssp245"},
    partition_range={"date": ("2050-01", "2050-06")},
)

run_ingest_raster_to_h3​

run_ingest_raster_to_h3(
    input_path: str | list[str],
    output_path: str,
    metrics: str | list[str] = "cnt",
    res: int | None = None,
    k_ring: int = 1,
    res_offset: int = 1,
    chunk_res: int | None = None,
    file_res: int | None = None,
    overview_res: list[int] | None = None,
    overview_chunk_res: int | list[int] | None = None,
    max_rows_per_chunk: int = 0,
    include_source_url: bool = True,
    target_chunk_size: int | None = None,
    nodata: int | float | None = None,
    debug_mode: bool = False,
    remove_tmp_files: bool = True,
    tmp_path: str | None = None,
    overwrite: bool = False,
    steps: list[str] | None = None,
    extract_engine: str | None = None,
    extract_max_workers: int = 256,
    partition_engine: str | None = None,
    partition_max_workers: int = 256,
    overview_engine: str | None = None,
    overview_max_workers: int = 256,
    **kwargs: int
)

Run the raster to H3 ingestion process.

This process involves multiple steps:

  • extract pixels values and assign to H3 cells in chunks (extract step)
  • combine the chunks per partition (file) and prepare metadata (partition step)
  • create the metadata _sample file and overviews files

Parameters:

  • input_path (str, list) – Path(s) to the input raster data. When this is a single path, the file is chunked up for processing based on target_chunk_size. When this is a list of paths, each file is processed as one chunk.
  • output_path (str) – Path for the resulting Parquet dataset.
  • metrics (str or list of str) – The metrics to compute per H3 cell. Supported metrics are either "cnt" or a list containing any of "avg", "min", "max", "stddev", "mode" (i.e. most common value), and "sum".
  • res (int) – The resolution of the H3 cells in the output data. The pixel values are assigned to H3 cells at resolution res + res_offset and then aggregated to res. By default, this is inferred based on the resolution of the input data ensuring the H3 cell size is close to the pixel size (e.g. for a raster with pixel size of 30x30m, a resolution of 11 is inferred).
  • k_ring (int) – The k-ring distance at resolution res + res_offset to which the pixel value is assigned (in addition to the center cell). Defaults to 1.
  • res_offset (int) – Offset to child resolution (relative to res) at which to assign the raw pixel values to H3 cells.
  • file_res (int) – The H3 resolution to chunk the resulting files of the Parquet dataset. By default will be inferred based on the target resolution res. You can specify file_res=-1 to have a single output file.
  • chunk_res (int) – The H3 resolution to chunk the row groups within each file of the Parquet dataset (ignored when max_rows_per_chunk is specified). By default will be inferred based on the target resolution res.
  • overview_res (list of int) – The H3 resolutions for which to create overview files. By default, overviews are created for resolutions 3 to 7 (or capped at a lower value if the res of the output dataset is lower).
  • overview_chunk_res (int or list of int) – The H3 resolution(s) to chunk the row groups within each overview file of the Parquet dataset. By default, each overview file is chunked at the overview resolution minus 5 (clamped between 0 and the res of the output dataset).
  • max_rows_per_chunk (int) – The maximum number of rows per chunk in the resulting data and overview files. If 0 (the default), chunk_res and overview_chunk_res are used to determine the chunking.
  • include_source_url (bool) – If True, include a "source_url" column in the output dataset that contains a list of source URLs that contributed data to each H3 cell. Defaults to True, set to False to omit this column.
  • target_chunk_size (int) – The approximate number of pixel values to process per chunk in the first "extract" step. Defaults to 10,000,000 for ingesting a single file or a few files. If ingesting more than 20 files, each file is processed as a single chunk by default, but you can override this by specifying a specific target_chunk_size value, or by specifying target_chunk_size=0 to always process each file as a single chunk.
  • nodata (int or float) – By default, the "nodata" value indicated in the raster metadata will be used to ignore pixels with this value when ingesting the data. If this information is missing or wrong in the raster metadata, you can override the value by specifying this keyword.
  • debug_mode (bool) – If True, run only the first two chunks for debugging purposes. Defaults to False.
  • remove_tmp_files (bool) – If True, remove the temporary files after ingestion is complete. Defaults to True.
  • tmp_path (str) – Optional path to use for the temporary files. If specified, the extract step is skipped and it is assumed that the temporary files are already present at this path.
  • overwrite (bool) – If True, overwrite the output path if it already exists, by first removing the existing content before writing the new files. Defaults to False, in which case an error is raised if the output_path is not empty.
  • steps (list of str) – The processing steps to run. Can include "extract", "partition", "metadata", and "overview". By default, all steps are run.
  • extract_engine (str) – The engine to use for the extract step. By default, tries first with "realtime" and then falls back to "large".
  • extract_max_workers (int) – The maximum number of workers to use for the extract step. Defaults to 256.
  • partition_engine (str) – The engine to use for the partition step. By default, tries first with "realtime" and then falls back to "large".
  • partition_max_workers (int) – The maximum number of workers to use for the partition step. Defaults to 256.
  • overview_engine (str) – The engine to use for the overview step. By default, tries first with "realtime" and then falls back to "large".
  • overview_max_workers (int) – The maximum number of workers to use for the overview step. Defaults to 256.

The extract, partition and overview steps are run in parallel using fused.submit(). By default, the function will first attempt to run this using "realtime" instances, and retry any failed runs using "large" instances.

You can override this behavior by specifying the engine, max_workers, worker_concurrency, etc parameters as additional keyword arguments to this function (**kwargs). If you want to specify those per step, use extract_engine, partition_engine, etc. For example, to run everything locally on the same machine where this function runs, use:

run_ingest_raster_to_h3(..., engine="local")

sources.era5​

era5(
    year: int,
    month: int,
    *,
    outputs: Sequence[str],
    cadence: str = "daily",
    days: Sequence[int] | None = None,
    bbox: Sequence[float] | None = None,
    read_workers: int = 16
) -> Iterator[pa.RecordBatch]

Read a month of ERA5 weather from the public ARCO store as lat/lng rows.

ARCO ERA5 is Google's public copy of the ECMWF ERA5 reanalysis: hourly global weather on a 0.25 degree latitude/longitude raster, 1940 to now. Reading needs network access to storage.googleapis.com. Decoding the zarr chunks needs numcodecs, which the Fused runtime and job2 environments provide; anywhere else, install it yourself.

Yields record batches with columns lat, lng, date (date32), hour (uint8, hourly only) and one float32 column per output, in the order given. Longitudes are converted to [-180, 180) for a tidy output column; the nearest-point fill does not need it. Cells are not computed here: hex it with fused.h3.partition(..., point_column=("lat", "lng"), fill="nearest"), which fills every cell from its nearest raster point.

A daily call yields the whole month one lat/lng tile at a time (90 by 90 degrees, so 8 tiles over the globe). A tile batch holds every requested day, day-major: one complete raster of the tile's 0.25 degree points per date. Tiles are disjoint sets of raster points, so each cell is filled from exactly one tile. Without partition_by, partition therefore writes one monthly file per H3 parent cell directly; only parents that straddle a tile edge are hit by two tiles and merged. partition_by="date" still works and writes one folder per day instead.

An hourly call yields one batch per day (a month of hourly rasters is about 3 GB per column, too much to hold at once). A batch holds all 24 hours, hour-major: 24 complete rasters, one per hour 0 to 23. Pass partition_by="date" so each day becomes a folder (output/date=<date>/<parent>.parquet) of one file per H3 parent cell.

This function is the template for a new datasource: any generator that yields batches of this shape (a complete regular lat/lng raster per key tuple, plus key and value columns) works with partition. partition treats every column that is not floating point (date, hour) as a key and fills the float32 outputs.

Parameters:

  • year (int) – Calendar year.
  • month (int) – Month, 1 to 12.
  • outputs (Sequence[str]) – Output columns to read. Names depend on cadence: daily has statistics such as t2m_min, t2m_max, t2m_mean, tp (precipitation, mm) and ssrd (radiation, W/m2); hourly has one column per variable such as t2m and tp.
  • cadence (str) – "daily" reduces each day's hours to one row per point per day and yields the month in lat/lng tiles. "hourly" keeps every hour (24 times the rows) in one batch per day and reads one day at a time to bound memory.
  • days (Sequence[int] | None) – Days of the month to read. Default is every day.
  • bbox (Sequence[float] | None) – (min_lng, min_lat, max_lng, max_lat) in degrees, longitude in [-180, 180], to keep only the raster points inside the box (edges inclusive). min_lng > max_lng means the box crosses the antimeridian. Default None reads the whole globe. The store's chunks hold the whole globe per hour, so bbox saves memory and output, not download. Cells emitted by partition are those whose nearest raster point is in the box. The box must keep at least two latitude rows and two longitude columns (span at least two 0.25 degree points on each axis).
  • read_workers (int) – Concurrent chunk downloads.

Raises:

  • ValueError – Unknown cadence or outputs, a bad month, days outside the month, or a bbox that is malformed, out of range, selects no raster point, or spans fewer than two points on an axis.
  • Era5SourceError – Raised during iteration, not at the call, when the ARCO store cannot be read or numcodecs is missing.

Example:

from fused import h3
batches = h3.sources.era5(
    2024, 1, outputs=["t2m_mean", "tp"], days=[1, 2]
)
h3.partition(
    batches, "s3://bucket/era5/", point_column=("lat", "lng"),
    fill="nearest", h3_res=7, file_res=1,
)  # doctest: +SKIP

# Hourly takes the same call plus ``partition_by="date"``; each day's
# file holds all 24 hours.

batches = h3.sources.era5(
    2024, 1, outputs=["t2m"], cadence="hourly", days=[1, 2]
)
h3.partition(
    batches, "s3://bucket/era5_hourly/", point_column=("lat", "lng"),
    fill="nearest", partition_by="date",
    h3_res=7, file_res=1,
)  # doctest: +SKIP

sources.parquet​

parquet(
    path: str | Path,
    *,
    columns: Sequence[str] | None = None,
    batch_size: int = 65536
) -> Iterator[pa.RecordBatch]

Stream a parquet file, or every .parquet file under a folder.

Files are read in sorted path order. path can be a local path or any fsspec URL (fd://, file://, s3://, gs://, az://, memory://). Remote schemes need their fsspec backend installed (s3fs, gcsfs, adlfs).

Parameters:

  • path (str | Path) – A parquet file or a folder of them (searched recursively).
  • columns (Sequence[str] | None) – Read only these columns. Default is all columns.
  • batch_size (int) – Maximum rows per yielded batch.

Raises:

  • ValueError – No parquet files were found under path.

sources.table​

table(data: Any) -> Iterator[pa.RecordBatch]

Yield the batches of a pandas.DataFrame, pyarrow.Table or pyarrow.RecordBatch.

A DataFrame index becomes a column only when every level is named; otherwise (a RangeIndex or an unnamed index, such as after filtering) it is dropped. A geopandas.GeoDataFrame's geometry columns arrive as WKB (binary), the form fused.h3.partition(..., point_column=...) expects; GeoParquet read with parquet is already WKB. The centroid is planar in the coordinates as stored (lon/lat degrees expected), so geometries that cross the antimeridian or wrap a pole are mis-hexed; split them or pass lat/lng columns as point_column= for those rows.

Raises:

  • TypeError – data is none of those.