Skip to main content

File Formats

This page specifies which File Formats for both raster & vector data we prefer working with at Fused, and why

For rasters (images)​

For images (like satellite images) we recommend using Cloud Optimized GeoTiffs (COGs). To paraphrase the Cloud Native Geo guide on them:

Cloud-Optimized GeoTIFF (COG), a raster format, is a variant of the TIFF image format that specifies a particular layout of internal data in the GeoTIFF specification to allow for optimized (subsetted or aggregated) access over a network for display or data reading

warning

Fused does not (yet) have a build-in tool to ingest raster data. We suggest you create COGs yourself, for example by using gdal's built-in options or rio-cogeo

Cloud Optimized GeoTiffs have multiple different features making them particularly interesting for cloud native applications, namely:

  • Tiling: Images are split into smaller tiles that can be individually accessed, making getting only parts of data a lot faster.
  • Overviews: Pre-rendered images of lower zoom levels of images. This makes displaying images at different zoom levels a lot faster
A simple overview of Geoparquet benefits

A simple visual of COG tiling: If we only need the top left part of the image we can fetch only those tiles (green arrows). Image courtesy of Element 84's blog on COGs

For vectors (tables)​

To handle vector data such as pandas DataFrames or geopandas GeoDataFrames we recommend using GeoParquet files. To (once again) paraphrase the Cloud Native Geo guide:

GeoParquet is an encoding for how to store geospatial vector data (point, lines, polygons) in Apache Parquet, a popular columnar storage format for tabular data.

A simple overview of Geoparquet benefits

Image credit from the Cloud Native Geo slideshow

tip

Refer to the next section to see all the details of how to ingest your data with Fused's built-in fused.ingest() to make the most out of geoparquet

Additional resources​