Skip to main content

Storage options

Fused provides two file systems to make files accessible to all UDFs: an S3-backed home directory and a disk. Access is scoped at the organization level.

Home directory​

Fused provisions a private home directory in S3 for your account. Paths look like s3://fused-users/fused/aman/ and are ideal for large-scale, cloud-native, or globally accessible datasets such as ingested tables, GeoTIFFs, and files that need to be read outside of Fused.

Use the File explorer to browse your home directory and confirm the exact path attached to your account.

file explorer

job = fused.ingest(
    input="https://www2.census.gov/geo/tiger/TIGER_RD18/STATE/06_CALIFORNIA/06/tl_rd22_06_bg.zip",
    output="s3://fused-users/fused/aman/census/ca_bg_2022/",  # change to your own path
).run_batch()

/mnt/cache disk​

/mnt/cache is the path to a mounted disk to store files shared between UDFs. This is where @fused.cache and fused.download write data. It's ideal for files that UDFs need to read with low-latency, downloaded files, the output of cached functions, access keys, .env, and ML model weights.

UDFs may interact with the disk as with a local file system. For example, to list files in the directory:

@fused.udf
def udf():
    import os
    for each in os.listdir('/mnt/cache/'):
        print(each)

Troubleshooting​

If you encounter the following error, it means /mnt/cache is not yet configured for your environment. To resolve this issue, please contact the Fused team to enable it.

Error: No such file or directory: '/mnt/cache/'