Google Earth Engine

Fused interfaces Google Earth Engine with the Python earthengine-api library. This example shows how to load data from GEE datasets into Fused UDFs and read it with xarray.

1. Authenticate with a Google Service Account

Create a UDF to set your Google Service Account credentials in your Fused runtime disk in a file in the /mnt/cache directory.

@fused.udf
def udf():

    import os, json

    # Google Key as JSON
    data = {
        'type': 'service_account',
        'project_id': 'MYPROJECT',
        'private_key_id': '1234',
        'private_key': '-----BEGIN PRIVATE KEY-----...\n-----END PRIVATE KEY-----\n',
        'client_email': 'fused-account@MYPROJECT.iam.gserviceaccount.com',
        'client_id': '1234567',
        'auth_uri': 'https://accounts.google.com/o/oauth2/auth',
        'token_uri': 'https://oauth2.googleapis.com/token',
        'auth_provider_x509_cert_url': 'https://www.googleapis.com/oauth2/v1/certs',
        'client_x509_cert_url': 'https://www.googleapis.com/robot/v1/metadata/x509/fused-pg%40MYPROJECT.iam.gserviceaccount.com',
        'universe_domain': 'googleapis.com'
    }

    # Define the target path for the new GEE credentials file
    key_path = '/mnt/cache/gee_creds.json'

    # Write the loaded JSON data to the new file
    with open(key_path, 'w') as file:
        json.dump(data, file)

2. Load data from Google Earth Engine

Create a UDF to load data from a GEE ImageCollection and open it with xarray. Authenticate by passing the key file path to ee.ServiceAccountCredentials.

@fused.udf
def udf(bbox: fused.types.TileGDF=None, n=10):
    import ee
    import xarray

    utils = fused.load("https://github.com/fusedio/udfs/tree/f928ee1/public/common/").utils

    # Authenticate GEE
    key_path = '/mnt/cache/gee_creds.json'
    credentials = ee.ServiceAccountCredentials("fused-account@fused-gee.iam.gserviceaccount.com", key_path)
    ee.Initialize(opt_url="https://earthengine-highvolume.googleapis.com", credentials=credentials)

    # Generate GEE bounding box for spatial filter
    geom = ee.Geometry.Rectangle(*bbox.total_bounds)
    scale = 1 / 2 ** max(0, bbox.z[0])  # A larger scale will increase your resolution per z but slow down the loading

    # Load data from a GEE ImageCollection
    ic = ee.ImageCollection("MODIS/061/MOD13A2").filter(
        ee.Filter.date("2023-01-01", "2023-06-01")
    )

    # Open with xarray (the `xee` package must be present for engine="ee" to work)
    ds = xarray.open_dataset(ic, engine="ee", geometry=geom, scale=scale).isel(time=0)

    # Transform image color with a utility function
    arr = utils.arr_to_plasma(ds["NDVI"].values.squeeze().T, min_max=(0, 8000))
    return arr