Environment variables

Save constants to an .env file to make them available to UDFs as environment variables. You should use the secrets manager for sensitive information like API keys.

First, run a File UDF that sets variables in an .env file in the /mnt/cache/ directory.

@fused.udf
def udf():
    env_vars = """
    MY_ENV_VAR=123
    """

    # Path to .env file in disk file system
    env_file_path = '/mnt/cache/.env'

    # Write the environment variables to the .env file
    with open(env_file_path, 'w') as file:
        file.write(env_vars)

Now, any UDF can load the values from .env as environment variables with the load_dotenv and access them with os.getenv.

@fused.udf
def udf():
    from dotenv import load_dotenv

    # Load environment variable
    env_file_path = '/mnt/cache/.env'
    load_dotenv(env_file_path, override=True)

    # Access environment variable
    print(f"Updated MY_ENV_VAR: {os.getenv('MY_ENV_VAR')}")