Tilebox Articles

GeoAI Can't Scale On Models Alone: It Needs An Operating Layer

Megan Van Patten — Tue, 23 Jun 2026 00:00:00 GMT

GeoAI needs to get grounded

GeoAI is moving fast because the opportunity is real. Satellite imagery, SAR, hyperspectral data, foundation models, and AI agents are making it possible to ask operational questions about the physical world and get answers in minutes.

The market is no longer theoretical. Planet is turning daily Earth observation into monitoring contracts. ICEYE has become one of the clearest signals of demand for persistent, all-weather intelligence. BlackSky’s Spectra AI platform points to the shift from imagery to real-time monitoring. Pixxel is adding hyperspectral data to the stack. Kayrros turns satellite observations into climate and energy intelligence. Xoople is framing the next phase directly: mapping the Earth for AI. Google, Microsoft, and IBM are pushing geospatial foundation models closer to the tools analysts already use.

While there is ample opportunity, the limitation is that more models, more satellites, and more data do not automatically create grounded workflows.

The stack can infer. It still needs to verify.

The leading GeoAI companies make different parts of the stack stronger: better imagery, richer sensors, faster revisit, stronger analytics, easier model access, and more verticalized signals.

But the hard part is no longer proving that GeoAI can produce an answer. The hard part is proving that the answer can be trusted, reproduced, and operationalized.

An AI agent can sound right without being right.

Geospatial data is numeric, spatial, and consequential. A wrong field name, stale dataset, bad geometry predicate, or unverified flood extent can affect a risk model, infrastructure plan, supply-chain forecast, or public-sector response.

Three major limitations show up across the market.

The context is fragmented. Every provider has its own catalog, schema, access pattern, license, format, and cloud footprint. A GeoAI team can have excellent data and still spend too much time translating between systems.

The workflows are brittle. Agents can generate plausible-looking code that references fields that do not exist, assumes the wrong time column, or breaks when moved to a new region. Foundation models can perform well on familiar data and degrade when the geography, sensor, or local context changes.

The outputs are hard to verify. A model output is not an operational record. For high-stakes use cases, teams need to know what ran, which data it used, what parameters changed, and what the system produced.

The market is scaling its promises faster than it is scaling its ability to ground them.

GeoAI needs an operating layer

The next phase of GeoAI will not be won by the model alone. It will be won by teams that turn AI outputs into governed, repeatable workflows.

That requires an operating layer with guardrails for agents. And that’s where the Tilebox agentic framework fits in. It gives coding agents the operational context and engineering tools they need to work with geospatial data safely and repeatably.

Live context. Agents inspect real dataset schemas, collections, query options, workflow state, jobs, logs, and spans before acting. They do not have to guess whether the field is cloudcover or cloud_cover.

Deterministic tools. Agents use the Tilebox CLI to run real commands with machine-readable inputs and outputs. They can query data, submit jobs, deploy workflows, inspect failures, and iterate from the terminal.

Agent skills. Tilebox skills teach agents how to manage datasets, monitor jobs, write workflows, release code, and inspect automations. The agent gets the operating pattern once and applies it across projects.

Distributed compute. Tilebox lets agents run geospatial workflows closer to the data, reducing data movement and one-off infrastructure work as teams expand into new regions or products.

GeoAI will transform global, regional, and local intelligence. But it needs more than signals and models. It needs agents that can act through governed systems, inspect what happened, and reproduce the result.

Tilebox is how GeoAI gets grounded.

Climate Tech's Most Unlikely Ally

Megan Van Patten — Thu, 18 Jun 2026 00:00:00 GMT

Tilebox-skilled agents offer efficient, verified, reproducible geospatial workflows to diversify and scale climate analytics for timely action.

The climate tech conundrum

Climate risk analytics, extreme-weather forecasting, water and nature solutions was the fastest-growing segment in climate tech in 2025, its share of funding climbing from 5.0% to 8.2% in a single year on a projected $5.5 billion (Net Zero Insights, State of Climate Tech 2025). Insurers, asset managers, and corporates filing TCFD and IFRS S2 disclosures are all hungry for asset-level risk insight.

And yet, while US climate-tech investment reached $29 billion in 2025, a record 52% of venture-backed climate-tech companies cut their net burn year-over-year just to stay resilient (Silicon Valley Bank, Future of Climate Tech 2026). The promise is enormous, but the economics are unforgiving. The difference between the two is almost always engineering overhead.

Fragile scaffolding

The geospatial market sorts into three tiers. At the bottom sit the EO satellite operators (Planet, ICEYE, Vantor) plus public archives from NASA and ESA, supplying raw imagery. In the middle, hyperscalers like Google, Microsoft, and AWS store the petabytes and ship powerful geospatial foundation models. At the top is the intelligence layer: the companies that fuse EO data with weather records, economic data, and proprietary models to deliver something a risk manager can actually act on.

That top layer is where most of the value, and complexity, concentrates. It’s also where the engineering burden is heaviest, because intelligence-layer companies inherit every messy seam between the layers below them: incompatible formats, authentication to a dozen archives, and enormous files that cost real money every time they cross a cloud region.

The egress math is not hypothetical — at standard cloud pricing of roughly $0.09/GB, moving a single large embedding dataset out of one provider has run teams tens of thousands of dollars in transfer fees alone, and bespoke formats compound that tax across every downstream product. Not to mention, value-killing latency which adds to the challenge of these climate orgs making enough to keep the lights on.

Where foundation models stop and local intelligence begins

Geospatial foundation models from Google, IBM (Prithvi), Microsoft (Aurora), and Clay have been a genuine leap, compressing raw imagery into representations downstream apps can use at a fraction of the previous compute cost. But even as these models get richer, the value in climate intelligence is rarely global. It’s regional and specific: this watershed, this county’s flood exposure, this cooperative’s fields.

On independent benchmarks, a model that performs competitively on familiar data can collapse to under 20% accuracy when moved to a new region (PANGAEA). Parametric insurance runs into the same wall from the other direction: the events it pays out on are inherently local, so a model that’s only right on average is wrong exactly where a claim is filed.

Delivering actionable value takes local sensor data, regional context, validation against ground truth, and constant re-tuning, without a custom build each time. The hard part isn’t running the model. It’s feeding it the right local data continuously, at the granularity a region demands. And that is an orchestration problem.

Fixing what’s broken

Three failure modes recur across climate data companies, and none of them are about the science. These are the common challenges across geospatial development that Tilebox was founded to solve.

Pipeline sprawl. Every new product pulls from different sources in different formats, so each becomes a fragile custom pipeline to maintain, and the cost of a product line is dominated not by the model but by the connectors feeding it.
The geospatial data tax. Imagery files are huge, moving them between regions burns egress fees, and wiring up connectors takes weeks of engineering before a single insight ships — overhead that, for a small team, is the difference between profitability and another bridge round.
The last-mile workflow gap. Since finance runs on tables and feeds rather than maps, insight that can’t drop into the format and cadence a client already works in doesn’t matter how good the underlying data is.

To fix these problems, you need both interoperability and speed. That’s why we released Tilebox for agentic engineering, and built it to support any data source, environment, algorithm, and coding agent. More importantly, when using Tilebox, agents work more efficiently.

Agents that run the pipeline, not just the model

The difference between an AI answer and an operational workflow is whether you can inspect, reproduce, and trust the result. Agents on Tilebox use the same operational context human developers do, through three primitives:

MCP server exposes live dataset schemas, query tools, and job status. The agent reads the real schema (exact fields like cloud_cover , precise_time ) instead of guessing from stale docs, so its code runs first time rather than failing on an incorrect field.
CLI machine-readable descriptions of every command and return. The agent runs real, governed actions (discover data, trigger a workflow, inspect execution) then parses the result and repeats the loop. No human stitching steps together; the run is reviewable from the terminal.
Agent skills a single-line install drops them into the universal skills directory your coding agent already reads (Claude Code, Cursor, Codex). No special prompt prefix: install once, describe the task in plain language.

The result: agents act through real commands against governed data, leaving a record of what ran, where, on which data, and what it produced.

Grounded that way, a climate analytics company’s data operations can run largely autonomously: agents query open (or custom) datasets, deploy risk and monitoring pipelines, adjust model thresholds, and answer “are the GPU tasks stuck on the cloud cluster?” from live task state instead of a manual log dig.

And because Tilebox’s architecture is inherently built for distributed computing, agents catalog and transform data without moving it. For a marketplace model, that autonomy extends to the catalog itself: agents can onboard and validate new datasets far faster than a manual review queue allows.

Start building

Here’s a change detection tracker we built with an agent. Watch the demo video to see how to build the workflow underneath then run it yourself or fork the source code.

Parametric Insurance That Scales

Megan Van Patten — Tue, 16 Jun 2026 00:00:00 GMT

The parametric insurance model promises instant payouts the moment a hurricane crosses a windspeed threshold or a region misses its rainfall benchmark. This new era of insurance is poised to provide greater clarity for customers, but the back-end operations haven’t been so straight forward — or financially sufficient.

Bottlenecks and bottom lines

Investments in machine learning have sharpened the algorithms and replaced latent claims processing. However, a processing bottleneck persists, just with a different input: multi-source geospatial data. A mess of customized, fragile pipelines stitching together several packages that never work as seamlessly as stakeholders demand they do.

Tileboxchanges the economics of that workload by letting AI agents run the geospatial pipeline themselves, using an ultra-efficient and agnostic framework built for humans and agents.

The “geospatial data tax”

The engineering overhead required to monitor global assets is often referred to as a geospatial data tax. Satellite imagery files are enormous. Moving them between cloud regions burns through egress fees. Building custom connectors to public archives and private constellations requires weeks, if not months, of complex cloud and data engineering. From drought coverage for Kenyan farmers to wildfires for California vineyards, every new product means another custom build. Another custom build requires more manual maintenance.

The ability to scale parametric algorithms for optimal proficiency is fumbled by the fragmentation of each pipeline. Tilebox removes these burdens and offers a simplified entry point to geospatial data processing for new and evolving markets.

Agent see, agent do

With Tilebox, AI agents access the same operational context developers use: live documentation, authenticated access to Tilebox resources, and deterministic tools for running real actions. This means an insurance company’s data operations can become largely autonomous. Agents automate new event data ingestion, deploy new risk pipelines, and adjust thresholds without human DevOps in the loop.

The roadmap to real value

Efficiency. Parametric insurers don’t need to build entire space-data engineering departments. Agents handle ingestion, cataloging, and pipeline maintenance on Tilebox’s process-where-it-sits architecture, avoiding costly data movement, vendor dependencies, and wasted engineering resources.

Market growth. When launching a new parametric product no longer requires six months of engineering work, insurers can serve markets that were previously uneconomic: smallholder farmers, SMB flood cover, hyper-local windstorm policies.

Faster payouts. Threshold breaches calculated and verified during the satellite pass itself can automate near-instant settlement. Talk with our team about Tilebox On Orbit for edge computing payloads.

As the majority of the EO sector races to hammer out siloed GeoAI use cases, agents running on Tilebox can deliver repeatable workflows with deterministic results across markets.

Are You A Platform Integrator Or An EO Developer?

Megan Van Patten — Tue, 19 May 2026 00:00:00 GMT

Every geospatial data pipeline needs the same foundational capabilities:

A way to query metadata across multiple providers
A workflow engine that handles parallel execution and resumes from failure
Triggers that fire when new data lands
Code that’s easy enough to write so you can iterate quickly
As hyperspectral data adoption rises, a way to retrieve only the bands you actually need

None of these requirements are exotic. The problem is that no single tool has historically delivered all of them; teams build their stack by stitching together a STAC client, a PostGIS database, an Airflow or Temporal instance, custom storage retrieval scripts per provider, and homegrown deduplication logic on top.

That’s five tools, five sets of credentials, five places where things can break, and five ongoing maintenance burdens. Before a team writes a single line of the algorithm that actually differentiates their product, they’ve signed up to be a platform integrator.

The hidden costs of fragmentation

The compounding cost of this fragmented stack shows up in three places.

Engineering time goes to integration work that no customer ever sees.
New team members spend weeks learning a custom stack instead of a standard one.
Every provider API change or breaking update means rewriting integration code your team never budgeted time for.

Every hour spent gluing tools together is an hour not spent on the algorithm, the user experience, or the customer. Teams under deadline pressure inevitably make trade-offs against infrastructure quality, and over time, those trade-offs compound into slower iteration, harder onboarding, and your competitive position becomes defined by what the team can maintain rather than what they could be building.

A different architecture

Tilebox combines those capabilities into a single framework. We created a developer experience that makes building faster with an execution architecture that makes running pipelines more cost-efficient and more reliable.

Spatio-temporal metadata querying across open data and proprietary datasets through one API
Storage clients that retrieve files directly from provider infrastructure with partial product downloads built in
A workflow engine with automatic parallelization, retries, and idempotency
Storage event and cron-based automations for near-real-time processing
Multi-language SDKs (Python, Go) and an MCP server for AI-assisted development

Each of these capabilities exists somewhere else. None of them exist together, built specifically for EO, in a single integrated framework - until we launched Tilebox.

Focus on the work that matters

When the infrastructure stack collapses from five tools to one, the work that disappears is the undifferentiated work. What’s left is the work that actually matters: the algorithms, the analytics, the products.

Teams using Tilebox work faster because the data infrastructure already exists. They’re building a product, not a product and the infrastructure underneath it. They’re spending their time on the things customers actually pay for, and deploying production-grade EO products in months rather than years.

How to Simplify Multi-Source EO Data Pipelines

Megan Van Patten — Thu, 14 May 2026 00:00:00 GMT

No single satellite sensor can do everything. Optical cameras go blind in clouds or at night. Some sensors can detect crop stress but can’t explain why. Others see through clouds but miss important visual detail. As the idiom goes, more sensors are better than one.

Combining data from multiple satellites is technically hard and expensive. Teams spend months on custom pipeline “plumbing” just to handle different data formats, failed downloads, and scaling issues, leaving little time for real science. Many end up cutting scope entirely, limiting themselves to a single data source to avoid the headache.

Evolving the Geospatial Workflow

Given the exponential growth in satellite data and relatively low ROI for downstream geospatial applications, it’s important to point out that the bottleneck is not sales. It’s the pipeline. Between the total acquisition cost of data, the engineering resources it takes to build and maintain pipelines, plus growing competition - its critical to focus on differentiation and offering the best available product using the best available data, no matter the source(s).

We’re working to evolve the geospatial sector by solving for traditional obstacles and drastically improving data efficiency.

Here are a few of the things we’re changing:

One framework for any environment(s). If you need to process Sentinel-2 data from an Open Telekom Cloud, Landsat data from AWS US-West-2, and proprietary data in a Google Cloud for a multi-source pipeline, Tilebox enables you to execute the same workflow in each environment.
Parallel processing at scale. One job can automatically split into thousands of smaller tasks, no manual batching required.
Automatic error recovery. If a download fails, Tilebox retries it. A 10,000-scene job that breaks at scene 8,743 resumes from there, not from the beginning.
Runs on your infrastructure. You control the compute and cost; Tilebox handles the coordination.

Adding a new data source used to mean weeks of integration work. With Tilebox, it’s just a new dataset name and one additional line of code.

client.dataset("<slug>")

For EO teams, that’s the difference between multi-source data being a real advantage, or just a project that never gets built.

Check out this video for a sample workflow.

Watch on YouTube

3 Ways to Start Building With Tilebox: From Query to Code

Megan Van Patten — Wed, 22 Apr 2026 00:00:00 GMT

Whether you are exploring geospatial data for the first time, or a veteran engineer, here are three ways Tilebox makes it easier to get from query to code.

1. Export as Code in the Console

The Tilebox Console is a visual dataset explorer. Browse available datasets, draw your area of interest, set a time window, apply quality filters, and then hit Export as Code. You get a ready-to-run Python script, pre-filled with the correct dataset slug, collection name, and storage client for that provider. Copy, paste, run.

Best for: first-time users, analysts who want to prototype quickly, or anyone exploring a dataset they haven’t worked with before.

2. Write It Directly with the SDK

Once you’re comfortable with Tilebox’s patterns, you can write queries directly. Install tilebox-datasets and tilebox-storage, initialize a client with your API key, and you’re querying scene metadata with a spatio-temporal filter in a few lines. The same query structure works across every dataset in your catalog (Sentinel, Landsat, Umbra SAR, Wyvern). Write once, deploy anywhere.

Best for: developers building pipelines, automating workflows, or integrating Tilebox into existing codebases.

3. Ask an Agent

After connecting the Tilebox MCP directly to your AI coding assistant, like Claude or Cursor, the agent can query your live catalog in real time. It knows your dataset slugs, collection names, and schemas before it writes a single line, so you can prompt it in plain English: “Write me a script to pull all Sentinel-2 scenes over Berlin with less than 10% cloud cover for Q1 2025.” The agent handles the lookup problem entirely.

Best for: developers working in AI-assisted IDEs, and anyone who wants to skip from intent to working code in one step.

Whichever path fits your workflow, the underlying result is the same: less time on data access infrastructure, more time on the analysis that actually matters.

Start prototyping or validate your current workflows with Tilebox Labs | $0/mo - no credit card required. Sign up here.

How to Query Sentinel-2 Data in Python (No Setup Required)

Stefan Amberger — Mon, 06 Apr 2026 00:00:00 GMT

In just five lines of Python, you can query every Sentinel-2 scene over any location on Earth – filtered by time, geography, and cloud cover — with the Tilebox Open Data catalog.

No ESA account, no Copernicus API credentials, no OAuth dance required. Install one package, point it at the Tilebox Open Data catalog, and get an xarray Dataset back.

The output is an interactive folium map showing Sentinel-2 scene footprints over your area of interest, colored by cloud cover percentage. Open in Google Colab here.

Install Tilebox

uv add tilebox    # or: pip install tilebox

That’s the only dependency you need for querying. We’ll add folium and matplotlib later for visualization.

Query Sentinel-2 scenes over a location

Pick a location and a time range. This example queries all Sentinel-2 Level 2A scenes over Paris during September 2025.

from shapely import Polygon
from tilebox.datasets import Client

# Polygon around Paris
paris = Polygon([
    (2.245958, 48.8311913), (2.2436426, 48.8577043),
    (2.2843926, 48.896382), (2.3172704, 48.9088622),
    (2.4006226, 48.9055141), (2.4186822, 48.8829853),
    (2.4182191, 48.8238749), (2.3520005, 48.8058842),
    (2.245958, 48.8311913),
])

client = Client()
dataset = client.dataset("open_data.copernicus.sentinel2_msi")
data = dataset.query(
    collections=["S2A_S2MSI2A", "S2B_S2MSI2A", "S2C_S2MSI2A"],
    temporal_extent=("2025-09-01", "2025-10-01"),
    spatial_extent=paris,
    show_progress=True,
)
print(data)

No API key required. The Client() constructor connects to the Tilebox open data catalog by default when querying public datasets. The dataset.query() call queries across all three Sentinel-2 satellites (2A, 2B, 2C) and returns the results as an xarray.Dataset.

The spatial extent here is a polygon around central Paris, but it accepts any Polygon or MultiPolygon — country outlines, watersheds, coastlines, or any arbitrary shape you can define with Shapely. Tilebox also handles antimeridian crossings and pole-covering geometries correctly out of the box.

<xarray.Dataset> Size: 19kB
Dimensions:                (time: 42)
Coordinates:
  * time                   (time) datetime64[ns] ...
Data variables: (12/23)
    id                     (time) <U36 ...
    geometry               (time) object ...
    granule_name           (time) object ...
    cloud_cover            (time) float64 ...
    thumbnail              (time) object ...
    ...

Each row is one Sentinel-2 scene. The dataset includes the scene geometry, cloud cover percentage, thumbnail URL, granule name, and 20+ other metadata fields — all indexed by acquisition time.

Explore the results with xarray

The output is a standard xarray Dataset. You can filter, slice, and analyze it with any operation xarray supports.

Check how many scenes were acquired and what the cloud cover distribution looks like:

print(f"Found {data.sizes['time']} Sentinel-2 scenes over Paris in September 2025")
print(f"Cloud cover: min={data.cloud_cover.min().item():.1f}%, "
      f"max={data.cloud_cover.max().item():.1f}%, "
      f"mean={data.cloud_cover.mean().item():.1f}%")

Found 42 Sentinel-2 scenes over Berlin in September 2025
Cloud cover: min=0.2%, max=99.8%, mean=48.3%

Filter by cloud cover

Most workflows start with filtering out cloudy scenes. xarray makes this straightforward:

clear_scenes = data.where(data.cloud_cover < 20, drop=True)
print(f"{clear_scenes.sizes['time']} scenes with less than 20% cloud cover")

You can also sort by cloud cover to find the clearest acquisitions:

import numpy as np

sorted_indices = np.argsort(data.cloud_cover.values)
clearest = data.isel(time=sorted_indices[:5])
for i in range(clearest.sizes['time']):
    scene = clearest.isel(time=i)
    print(f"{scene.time.dt.strftime('%Y-%m-%d').item()} — "
          f"{scene.cloud_cover.item():.1f}% cloud cover — "
          f"{scene.granule_name.item()}")

Plot scene footprints on an interactive map

Install the visualization dependencies:

uv add folium matplotlib    # or: pip install folium matplotlib

Now plot every scene footprint on a folium map, colored by cloud cover percentage. Green means clear skies, red means heavy cloud cover.

import folium
import matplotlib
import matplotlib.cm as cm

# Create a map centered on our area of interest
center_lat = (52.3 + 52.7) / 2
center_lon = (13.1 + 13.8) / 2
m = folium.Map(location=[center_lat, center_lon], zoom_start=10)

# Color scale: green (clear) to red (cloudy)
norm = matplotlib.colors.Normalize(vmin=0, vmax=100)
colormap = cm.RdYlGn_r

for i in range(data.sizes['time']):
    scene = data.isel(time=i)
    cloud = scene.cloud_cover.item()
    rgba = colormap(norm(cloud))
    color = matplotlib.colors.to_hex(rgba)

    # Extract polygon coordinates from the geometry
    geom = scene.geometry.item()
    coords = [(lat, lon) for lon, lat in geom.exterior.coords]

    folium.Polygon(
        locations=coords,
        color=color,
        fill=True,
        fill_color=color,
        fill_opacity=0.4,
        popup=f"{scene.time.dt.strftime('%Y-%m-%d').item()}<br>"
              f"Cloud cover: {cloud:.1f}%<br>"
              f"{scene.granule_name.item()}",
    ).add_to(m)

# Add a bounding box for the area of interest
folium.Rectangle(
    bounds=[(52.3, 13.1), (52.7, 13.8)],
    color="blue",
    fill=False,
    weight=2,
).add_to(m)

m

Open the map in a Jupyter notebook and click on any footprint to see the acquisition date, cloud cover, and granule name.

Scene footprint

Preview a scene thumbnail

Each Sentinel-2 scene in the Tilebox catalog includes a thumbnail URL. You can display it directly in a notebook:

from IPython.display import Image, display

# Pick the clearest scene
sorted_indices = np.argsort(data.cloud_cover.values)
best_scene = data.isel(time=sorted_indices[0])

print(f"Clearest scene: {best_scene.time.dt.strftime('%Y-%m-%d').item()}, "
      f"{best_scene.cloud_cover.item():.1f}% cloud cover")
display(Image(url=best_scene.thumbnail.item(), width=400))

Scene thumbnail

What you just did

In under 30 lines of code, you:

Connected to Tilebox’s open data catalog — no API key, no account
Queried all Sentinel-2 Level 2A scenes over Paris for a full month
Filtered scenes by cloud cover using standard xarray operations
Plotted every scene footprint on an interactive map, color-coded by cloud cover
Previewed a scene thumbnail

The query returned structured metadata as an xarray Dataset: scene geometries, acquisition timestamps, cloud cover percentages, thumbnail URLs, and granule identifiers. From here you can feed the results into any processing pipeline, download the actual data from Copernicus Data Space, or build automated monitoring workflows with Tilebox.

Downloading the actual scene data

Tilebox indexes satellite metadata — the full scene data (bands, reflectance values) lives in the Copernicus Data Space Ecosystem. Once you’ve identified the scenes you want, you can download them using the Tilebox storage client:

from tilebox.storage import CopernicusStorageClient

storage_client = CopernicusStorageClient(
    access_key="YOUR_CDSE_ACCESS_KEY",
    secret_access_key="YOUR_CDSE_SECRET_KEY",
)

for i in range(clear_scenes.sizes['time']):
    scene = clear_scenes.isel(time=i)
    path = storage_client.download(scene)
    print(f"Downloaded to {path}")

This requires free Copernicus Data Space credentials — but querying and filtering metadata through Tilebox is completely open.

Full script

Here’s the complete script in one block. Copy it into a Jupyter notebook or Python file and run it top to bottom.

from shapely import Polygon
from tilebox.datasets import Client
import numpy as np
import folium
import matplotlib
import matplotlib.cm as cm
from IPython.display import Image, display

# 1. Define your area of interest
berlin = Polygon((
    (13.1, 52.3),
    (13.1, 52.7),
    (13.8, 52.7),
    (13.8, 52.3),
    (13.1, 52.3),
))

# 2. Query Sentinel-2 scenes (no API key needed)
client = Client()
dataset = client.dataset("open_data.copernicus.sentinel2_msi")
data = dataset.query(
    collections=["S2A_S2MSI2A", "S2B_S2MSI2A", "S2C_S2MSI2A"],
    temporal_extent=("2025-09-01", "2025-10-01"),
    spatial_extent=berlin,
    show_progress=True,
)

# 3. Print summary
print(f"Found {data.sizes['time']} Sentinel-2 scenes")
print(f"Cloud cover: min={data.cloud_cover.min().item():.1f}%, "
      f"max={data.cloud_cover.max().item():.1f}%, "
      f"mean={data.cloud_cover.mean().item():.1f}%")

# 4. Filter by cloud cover
clear_scenes = data.where(data.cloud_cover < 20, drop=True)
print(f"{clear_scenes.sizes['time']} scenes with less than 20% cloud cover")

# 5. Plot footprints on an interactive map
center_lat = (52.3 + 52.7) / 2
center_lon = (13.1 + 13.8) / 2
m = folium.Map(location=[center_lat, center_lon], zoom_start=10)

norm = matplotlib.colors.Normalize(vmin=0, vmax=100)
colormap = cm.RdYlGn_r

for i in range(data.sizes['time']):
    scene = data.isel(time=i)
    cloud = scene.cloud_cover.item()
    rgba = colormap(norm(cloud))
    color = matplotlib.colors.to_hex(rgba)
    geom = scene.geometry.item()
    coords = [(lat, lon) for lon, lat in geom.exterior.coords]

    folium.Polygon(
        locations=coords,
        color=color,
        fill=True,
        fill_color=color,
        fill_opacity=0.4,
        popup=f"{scene.time.dt.strftime('%Y-%m-%d').item()}<br>"
              f"Cloud cover: {cloud:.1f}%<br>"
              f"{scene.granule_name.item()}",
    ).add_to(m)

folium.Rectangle(
    bounds=[(52.3, 13.1), (52.7, 13.8)],
    color="blue",
    fill=False,
    weight=2,
).add_to(m)

m

# Display the map in a Jupyter notebook

# 6. Preview the clearest scene thumbnail
sorted_indices = np.argsort(data.cloud_cover.values)
best_scene = data.isel(time=sorted_indices[0])
print(f"\nClearest scene: {best_scene.time.dt.strftime('%Y-%m-%d').item()}, "
      f"{best_scene.cloud_cover.item():.1f}% cloud cover")
display(Image(url=best_scene.thumbnail.item(), width=400))

Next steps

To run this yourself, install Tilebox with uv add tilebox (or pip install tilebox) and open the full notebook from the tilebox/examples repository. Swap out the Paris polygon for your own coordinates and you’ll have a map of your area’s Sentinel-2 coverage in under five minutes.

Connecting AI Assistants with Tilebox: Grounding LLMs in Live Space Data

Stefan Amberger — Tue, 13 Jan 2026 00:00:00 GMT

Tilebox now supports MCP Servers to bridge the gap between Large Language Models and your dynamic space data. Built on the open Model Context Protocol, this integration allows Large Language Models (LLMs) to interact directly with your live Tilebox environment. This provides responses grounded in your actual data schemas and job statuses.

Static documentation is often insufficient for high velocity space data engineering. To build reliable pipelines, your AI assistants need more than general knowledge. They require real time context from your specific datasets and workflows.

What is the Tilebox MCP Server?

The Model Context Protocol (MCP) establishes a standardized connection between AI applications and external data. The Tilebox MCP server exposes tools that enable your assistant to:

Introspect Schemas: Retrieve the precise, up-to-the-minute schema of your custom datasets (including field types and descriptions).
Query Datasets: Assistants can list and filter data points directly from the chat interface.
Monitor Workflows: Check the status of Jobs, visualize task dependencies, and identify failures in your Task Runners in real time.

When an AI tool like Claude Code, Cursor, Amp, or OpenAI Codex is configured with the Tilebox MCP server, it can proactively invoke these relevant tools. For example, when you ask a question or request to write code about a specific dataset, the AI tool can query the Tilebox MCP server for the precise, current dataset schema, using that accurate information to generate the code and ground its response.

How the Tilebox MCP Server Works

Tilebox offers one comprehensive MCP server to cover both operational data (datasets and workflows) and documentation: <https://mcp.tilebox.com/mcp>

Purpose: Provides tools for accessing and interacting with Tilebox datasets and workflows. It also provides access to the official Tilebox documentation.
Requirement: Requires authentication using a Tilebox API key.

This single server allows an AI assistant to learn the general syntax and usage of Tilebox APIs and then customize its responses or generated code based on the actual available datasets and their exact schema.

Customized Coding Assistant

A developer is using an AI coding assistant (like Claude or Cursor) to write a script that interacts with a specific Tilebox dataset.

API Syntax Learning: The assistant uses the docs MCP server (https://docs.tilebox.com/mcp) to learn the general syntax and usage of the Tilebox APIs.
Crucially, it then queries the data/workflow MCP server (https://mcp.tilebox.com/) to retrieve the current, precise schema of the target dataset.
Benefit: By grounding documentation syntax with live data schemas, the AI assistant generates code that is:
- Syntactically correct and follows the latest Tilebox SDK patterns for Go or Python.
- Contextually accurate and uses the exact field names and data types (e.g., cloud_cover, precise_time) from your specific collection.
- Error-proof by preventing common runtime errors caused by typos in field strings or mismatched schema expectations.

Dynamic and Grounded Q&A

An analyst needs a quick answer to a question that requires live data context, such as “What was the total workflow run time yesterday?”

The AI assistant leverages the data/workflow MCP server’s tools to query the live datasets and workflows in Tilebox.
Benefit: The AI provides an accurate, up-to-the-minute answer that is explicitly grounded in the live Tilebox data, allowing the user to trust the output without manual verification.

Configuration & Setup

You can connect your AI tools to Tilebox using the http transport.

1. JSON Configuration Snippet

For the data/workflow server, remember to include an Authorization header with your Tilebox API key as the bearer token. Use this for tools like Cursor or the Claude Desktop app. Replace the placeholder with your Tilebox API Key created in the Console.

{
  "mcpServers": {
    "tilebox": {
      "url": "https://mcp.tilebox.com/",
      "headers": {
        "Authorization": "Bearer <YOUR_TILEBOX_API_KEY>"
      }
    }
  }
}

2. For Claude Code CLI

Add these servers directly via your terminal:

{
  "mcpServers": {
    "tilebox": {
      "url": "https://mcp.tilebox.com/",
      "headers": {
        "Authorization": "Bearer <YOUR_TILEBOX_API_KEY>"
      }
    }
  }
}

Note: You can still download our full documentation as a markdown file at docs.tilebox.com/llms-full.txt and upload it manually to any LLM.

Watch on YouTube

A practical demo of grounding LLMs in live operational data using MCP

MCP in Action

The following scenarios illustrate how the Tilebox MCP server transforms your AI assistant into a proactive operations partner.

1. Precision Spatio Temporal Filtering

Traditional AI assistants often hallucinate field names for geospatial data. With MCP, the assistant queries your live collection schema before writing any code.

User Prompt: “Write a Python script to find all Sentinel 2 granules from last week that fully contain the city of Denver.”
AI Action: The assistant uses the Tilebox MCP tool to retrieve the live schema for the Sentinel 2 dataset.
AI Output: It generates a script using spatial_extent with mode: contains for a polygon around Denver. This ensures the query is technically valid on the first run.

2. Multi Cluster Job Monitoring

Monitoring distributed execution across heterogeneous environments is often complex. The MCP server allows your AI to act as a mission control interface.

User Prompt: “Check the status of the Mosaic job. Are the GPU intensive tasks stuck on the cloud cluster?”
AI Action: The assistant invokes tools to find the Job ID and filters tasks by their assigned cluster_slug.
AI Output: “The root task is running on your on prem cluster. However, 12 subtasks assigned to the gpu cloud cluster are currently in the QUEUED state because no Task Runners are active in that cluster.”

3. Automated Error Diagnostics

When a workflow fails, you can use the AI to identify the exact point of failure without digging through logs manually.

User Prompt: “Why did my last data ingestion job fail?”
AI Action: The assistant queries the most recent Job with a FAILED state and retrieves the error message reported to the orchestrator.
AI Output: “The task LoadCSV failed on Task Runner node 7 with a ValueError: Invalid UUID. One of the IDs in your source file does not match the required schema for the id field.”

By adopting an open protocol approach, Tilebox ensures your AI assistant remains a peer in your infrastructure while staying agnostic, scalable, and secure. This is the crucial step in reducing downtime and accelerating payload-to-platform revenue.

The 6 Workflow Archetypes: A guide from discovery to services

Stefan Amberger — Mon, 25 Aug 2025 00:00:00 GMT

Bottom Line Up Front: This post outlines six powerful workflow patterns in Tilebox, from processing data directly in-orbit to cut downlink costs, to building on-demand data products for your customers. Whether you’re developing algorithms, operating in near real-time, or automating routine tasks, these examples demonstrate how to transform raw geospatial data into actionable intelligence more efficiently.

Your main focus is on delivering products and insights. Tilebox gives you the power to do that, and without frustrating limitations. We’ve compiled a list of our most useful workflow types with examples to help you improve your current processes. Learn how to scale up, expedite, and diversify your products and services to transform raw geospatial data into actionable intelligence.

1. In-Orbit Workflows: Process Data at the Edge

Downlinking raw satellite data is like shipping ore from a mine; it’s bulky and expensive. Processing it in-orbit is like refining it at the source and shipping only the gold. This approach dramatically reduces downlink costs, a substantial part of satellite operations.

For monitoring use cases requiring extremely low latency, or for hyperspectral providers who need to downlink only specific bands, on-orbit processing is a necessity. It moves beyond simple object detection demos to enable operational, decision-making AI in space.

Tilebox Highlights

Lightweight and Efficient: Tilebox is designed to be resource-efficient, consuming minimal power and compute from precious on-orbit resources. Work can be queued and executed only when compute capacity is available, allowing modules to be scaled up or down to fulfill power budgets.
Ground-to-Space API Parity: The same APIs for workflow orchestration and management are used on-orbit and on the ground. This allows you to create a “digital twin” of your on-orbit processing state for rapid testing, validation, and iteration of algorithms on the ground before uplinking.
Integrated Observability for Attribution: Collect critical attribution data and logs from in-orbit processes. This provides a tight feedback loop with ground operations, which is essential for operationalizing AI and decision-making in space.
Parallel Processing in Space: Fully utilize on-orbit compute capabilities by running tasks in parallel, ensuring that you get the most out of your hardware.

What you’ll need

On-orbit compute hardware with Tilebox deployed (c.f. from one of our pre-integrated compute module partners).
A ground-based environment for algorithm development and testing.

Next Steps

Start developing your algorithms on the hosted version of Tilebox today; they will be fully compatible with the on-board daemon.
Contact our team to discuss how to implement Tilebox for your on-orbit processing needs.

2. Distributed Workflows: Process Data Across Multiple Environments

As remote sensing data is often distributed across various locations, workflows that span multiple clouds and on-premise compute environments are becoming increasingly common. Tilebox simplifies the implementation of these complex workflows, such as data fusion use-cases like wildfire detection using both Landsat and Sentinel data. All workflows may be distributed.

Tilebox Highlights

Process Data at the Source: Minimize costly data egress fees and reduce latency by running compute tasks directly where your data resides, whether on-premise or across multiple cloud providers.
Natively Heterogeneous: Orchestrate workflows that span on-premise servers, multiple public clouds, and even on-orbit compute, without requiring complex Kubernetes setups.
Utilize Specialized Hardware: Route specific tasks to clusters with specialized hardware, such as GPUs or high-memory nodes, ensuring optimal performance for every step of your pipeline.
Simplified Multi-Cluster Dependencies: Declare dependencies between tasks in different environments as easily as if they were on the same machine. Tilebox handles the complex orchestration.
Resilience Across Environments: The failure of a node in one environment doesn’t cascade. The workflow’s built-in fault tolerance ensures that the overall process can continue even with localized infrastructure issues.

What you’ll need

Access to two or more distinct compute environments (e.g., on-premise and public cloud).
Tilebox Task Runners deployed in each of these environments.
Tasks designed to run in specific environments, orchestrated via Tilebox Clusters.

Next Steps

Read about how to manage clusters for distributed workflows.
Create a cluster in the console
Start two runners, one in each cluster
Schedule work from a task running on one cluster to the other

3. Near Real-Time Workflows: Trigger Processing on Incoming Data

When speed is critical, workflows can be triggered in near real-time as new data becomes available. This is necessary for applications like wildfire detection or maritime surveillance, which require immediate processing of newly published data.

For instance, when a satellite downlinks new payload data, it can automatically trigger the L0 to L1 processing chain.

Tilebox Highlights

Low-Overhead Processing: The time it takes to start a task is in the millisecond range, making Tilebox highly effective for latency-sensitive applications with frequent, small workloads.
Diverse Triggering Mechanisms: Initiate workflows automatically from a variety of sources, including Storage Events in cloud buckets (GCS, S3) and local file systems (both in closed beta)
Robust Backpressure Management: Gracefully handles sudden bursts of trigger events, ensuring system stability and preventing your infrastructure from being overloaded during data storms.
Full Observability: Monitor the health and performance of your automated pipelines with integrated Tracing and Logging, which is essential for production environments.

What you’ll need

An event source configured to trigger workflows (e.g., a cloud storage bucket with notifications enabled).
A Task defined to handle the incoming event data.
A continuously running Tilebox Task Runner listening for these jobs.

Next Steps

Learn how to set up automations to trigger workflows from events.

4. Customer-Triggered Workflows: Enable On-Demand Processing

Empower your customers by allowing them to trigger asynchronous jobs directly from a customer portal or API. This is ideal for generating custom reports, visualizations, or performing exploratory processing of specific areas of interest.

Tilebox Highlights

API-Driven Automation: Expose your processing capabilities directly to customers through a secure API, allowing them to request custom products on-demand.
Asynchronous Execution: Jobs run in the background, allowing your customer-facing portal or API to remain responsive. You can notify customers upon completion.
Secure by Design: The underlying architecture allows compute nodes to run in isolated environments without internet exposure, protecting your core infrastructure and algorithms.

What you’ll need

A customer-facing application (e.g., web portal, mobile app) with an API.
Backend logic that translates customer requests into Tilebox API calls to submit jobs.
A scalable set of Task Runners to handle potentially bursty customer demand.

Next Steps

Expose an API endpoint that your customers can use to trigger a Tilebox workflow.

5. Batch Workflows: Develop and Backtest Algorithms at Scale

Batch processing is fundamental for developing and backtesting algorithms. It allows you to reprocess entire missions or apply new algorithms to large volumes of historical data, which is critical for creating and validating new products.

These workflows are ideal for large-scale processing that covers vast areas of interest or long time periods, such as creating quarterly cloud-free mosaics or running crop yield estimates.

Tilebox Highlights

Dynamic Workflows: Unlike systems that use static DAGs, a Tilebox workflow can adapt at runtime. Tasks can determine subsequent steps based on intermediate results, enabling more complex and data-dependent processing.
Cost-Efficient Retries: For long-running processes, failures can be expensive. Tilebox allows workflows to be restarted from the point of failure after a bug is fixed, saving significant time and infrastructure costs.
Simplified Infrastructure: Tilebox simplifies deployment by not requiring complex cluster managers like Kubernetes. Since compute nodes operate independently without direct communication, network setup is significantly easier.
Flexibility and Scalability: Implement tasks in your preferred language—not just Python—and scale your jobs to handle millions of tasks.

What you’ll need

An algorithm or processing logic defined in Tilebox Tasks.
A dataset to process (e.g., a large data archive or a custom Tilebox Dataset).
Compute infrastructure (even a local machine) to run a Tilebox Task Runner.

Next Steps

Explore our workflows hello world to get started with your first batch workflow.
Read the documentation on submitting jobs.

6. Scheduled Workflows: Automate Repetitive Tasks

Automate recurring tasks to ensure consistency and reduce manual effort. This is perfect for operations that need to run at regular intervals, such as scraping tasks that operate hourly or workflows that generate daily mosaics and weekly reports.

Tilebox Highlights

First-Class CRON Support: Scheduled automations are a core feature, allowing you to reliably run tasks at any interval.
Seamless Updates with Versioning: Deploy updated task logic at any time. The next scheduled run will automatically use the new version, enabling rolling updates to your jobs without downtime or manual changes.
Flexible Management: Manage your scheduled automations programmatically via the SDK or through the intuitive Tilebox Console.
Code Reusability: The framework for batch and scheduled workflows is the same, allowing for quick operationalization by reusing code between different modes.

What you’ll need

A recurring task defined as a Tilebox Task (e.g., generating a daily report).
An automation configured with a CRON schedule in the Tilebox Console or via the SDK.
A continuously running Tilebox Task Runner to execute the scheduled jobs.

Next Steps

See how to configure CRON triggers in the documentation.
Explore our CRON example on Github

Want to explore something else? Join us on Discord for community support, ideation, and more.

Cloud-Free, Country-Scale Mosaic in Under 3 Hours: A Tilebox Workflow

Lukas Bindreiter — Tue, 15 Jul 2025 00:00:00 GMT

Creating large-scale, cloud-free satellite imagery mosaics demands efficient data handling and powerful processing. Inspired by a data scientist’s approach to generating a quarterly cloud-free Sentinel-2 10m resolution RGB mosaic over Ireland, we replicated this process using Tilebox to demonstrate how use cases like this can be streamlined. This post highlights how Tilebox streamlines complex geospatial workflows, leveraging multi-environment execution and parallel writes to a Zarr datacube for optimal performance.

Our workflow involves four key steps:

Locating relevant Sentinel-2 input granules over Ireland within a three-month period
Reading the Red, Green, and Blue (RGB) bands, plus the scene classification layer (cloud mask) from each located Sentinel-2 product
Reprojecting every product onto a common grid
Aggregating data across the time dimension to produce a single cloud-free measurement for every pixel

If you want to skip ahead and see our results for yourself, check out the interactive visualization down below.

Efficient Granule Discovery with Tilebox Datasets

The first step, locating the necessary Sentinel-2 granules, is straightforward with Tilebox’s spatio-temporal query capabilities. Our tilebox.datasets client allows for rapid discovery of relevant data.

from tilebox.datasets import Client
from shapely.geometry import box

# Initialize the Tilebox datasets client
client = Client()
sentinel_2 = client.dataset("open_data.copernicus.sentinel2_msi")
sentinel_2a = sentinel_2.collection("S2A_S2MSI2A")

# Define a rectangular area over Ireland
area = box(-10.68234795, 51.36473433, -5.34679566, 55.44704815)

# Query for Sentinel-2 granules within the specified temporal and spatial extent
granules = sentinel_2a.query(
    temporal_extent=("2025-03-01", "2025-06-01"),
    spatial_extent=area
)

print(f"Located {granules.sizes['time']} Sentinel-2 granules.")

In just milliseconds, we located around 700 Sentinel-2A granules needed for our mosaic. Tilebox Open Data provides instant access to public datasets, and its robust spatial indexing handles complex geometries like antimeridian crossings seamlessly.

Managing Data Volumes

For our true-color mosaic, we need the Red, Green, and Blue bands at 10m resolution, plus the 20m resolution cloud mask for filtering. A single Sentinel-2 L2A granule, with these four bands, amounts to around 315MB. The 700 granules we located amount to a total of approximately 155GB of data. Efficiently handling this volume requires a smart approach to storage and processing.

To avoid costly data transfers and manage large intermediate products, we chose Zarr as our intermediate storage format. Zarr is a highly efficient, chunked array format ideal for parallel I/O and cloud-native workflows. It allows us to persist reprojected data in a way that supports easy spatial chunking and parallel access across the time dimension.

We initialized an empty Zarr cube for each band, with dimensions corresponding to the full spatial extent over our area over Ireland and a time dimension corresponding to one time layer for each of our 700 products. This resulted in a data cube shape of time=716, y=37151, x=45419. By setting the time dimension chunk size to 1, we enable parallel writing of individual timestamps. Furthermore, configuring a spatial chunk size (in our case we chose 2048x2048 pixels) allows Zarr to automatically skip writing empty chunks for each time layer, providing immediate efficiency gains, especially when reprojecting smaller granules onto a large target grid. Additionally this is also what allows us to process individual, smaller spatial chunks across the entire time dimension in the required subsequent temporal aggregation.

Orchestrating Multi-Environment Workflows

The process of reading, reprojecting, and writing each Sentinel-2 product to a Zarr cube is inherently parallel. Tilebox Workflows are designed to leverage this parallelism by allowing us to define individual tasks that can be automatically parallelized. All that is required to achieve that is to transform our processing logic into tasks.

import rasterio
from tilebox.workflows import Task, ExecutionContext

class GranuleProductToZarr(Task):
    """
    Processes a single Sentinel-2 product (e.g., B02, B03, B04, SCL)
    reprojects it, and writes it to the Zarr datacube at a specific time index.
    """

    product_location: str
    """A concrete Sentinel 2 product to convert to Zarr"""

    time_index: int
    """The time index of the granule in the output Zarr datacube"""

    def execute(self, context: ExecutionContext) -> None:
        # Open the JPEG2000 product using rasterio
        with rasterio.open(self.product_location, driver="JP2OpenJPEG") as product:
             with rasterio.open(self.product_location, driver="JP2OpenJPEG") as product:
             arr = product.read(1)
             src_grid = GeoBox(shape=arr.shape, affine=product.transform, crs=product.crs)

         # .... continue with reprojecting the array, and writing it to zarr at the given time index

class GranuleToZarr(Task):
    """
    Orchestrates the processing of a single Sentinel-2 granule by
    submitting subtasks for each relevant band.
    """
    granule_location: str
    time_index: int

    def execute(self, context: ExecutionContext) -> None:
        products = list_products(  # glob the given directory for objects matching a pattern
            self.granule_location,
            filter=["R10m/*B02*", "R10m/*B03*", "R10m/*B04*", "R20m/*SCL*"]
        )
        for product in products:
            context.submit_subtask(product, self.time_index)

The GranuleToZarr task spawns multiple GranuleProductToZarr subtasks, one for each band. These subtasks can then execute in parallel on available task runners. A higher-level task then iterates through the list of all 700 Sentinel-2 granules in a similar fashion, submitting a GranuleToZarr task for each.

Co-locating Compute with Multi-Environment Capabilities

As you may have noticed, the above GranuleToZarr task assumes Sentinel-2 product files are available as part of the local file system. Traditionally, we would need to adapt this logic to also support reading products via an S3 compatible object store interface. However, a significant advantage of Tilebox is its ability to run workflows across diverse compute environments. To avoid downloading 155GB of Sentinel-2 data from the Copernicus archive (hosted on CloudFerro), we instead executed our workflow in a multi-environment fashion.

Distributing work across various compute environments is a built-in feature of our workflow orchestrator, so the only requirement for setting this up was to start task runners in the right locations, no adaptations to the source code are necessary.

Data Access & Reprojection (CloudFerro): We deployed a Tilebox task runner on a virtual machine within CloudFerro’s infrastructure. This runner’s sole responsibility is to read Copernicus data, reproject it, and write the intermediate Zarr cube directly to Google Cloud Storage. This minimizes egress from CloudFerro. And it allows us to read the Sentinel products directly from a filesystem, since VMs on CloudFerro have the whole Copernicus archive mounted at /eodata.
Temporal Aggregation (Local Cluster): For the final temporal aggregation (producing the cloud-free mosaic), we utilized a makeshift cluster using three of our developer MacBooks. Each MacBook’s task runner fetches a specific 2048x2048 spatial chunk of the Zarr cube across the entire time range, performs the aggregation, and writes the output for that chunk back into the final mosaic layer of our Zarr store. This is made possible by Zarr’s powerful rechunking capabilities and Tilebox’s flexible task distribution. This strategy enabled us to avoid setting up multiple expensive VMs on Cloudferro for compute. Alternatively, we could have just as well used cheap spot instances, which would be the ideal solution for even larger processings, such as a mosaic of the entire globe, and to minimize AWS egress cost.

This multi-environment approach demonstrates Tilebox’s flexibility: seamlessly integrating specialized compute resources (CloudFerro for data proximity, MacBooks for distributed final processing) into a single, cohesive workflow without rewriting code.

Figure 1: Architecture diagram depicting a multi-environment workflow, and Zarr chunking mechanics to enable parallelization. Depicted are task runners deployed on a CloudFerro VM as well as locally on developer notebooks.

Visualizing the Results

The final output is a mosaic, which we converted from Zarr to GeoTIFF and then uploaded to Ellipsis Drive for visualization.

Explore the interactive mosaic on Tilebox

This workflow showcases how Tilebox simplifies complex, large-scale geospatial processing by combining efficient data access, parallel processing with Zarr, and flexible multi-environment workflow orchestration. The full code for this example is available on GitHub, and you can try it out yourself using our free Community Access tier.

Skip the Respawn Delay: Space Data Workflows with Built-in Resilience

Stefan Amberger — Tue, 25 Mar 2025 00:00:00 GMT

Natural checkpoints, versioned tasking, and other smart workflow design decisions

During large processing workflows, things can and will go wrong. Memory errors, missing data, network issues, just to name a few. These processing errors cost time and money, resulting in inefficient compute resource utilization, reduced developer productivity, and increased egress costs from redundantly fetched external data dependencies.

Many workflow orchestrators don’t account for re-entrant execution at all. If something goes wrong, the whole processing has to be re-done. Or, a new workflow has to be developed for every failure that is able to resume from that particular partial state, and explicit checkpoints have to be added to the workflow to persist partial states. Not only does this take valuable time, it should be retired as a historical problem – there is no need to settle for this model.

Accounting for all possible errors beforehand is not possible, so it’s vital to be ready to act with the right resources to thoroughly investigate, rewrite, and deploy.

This is precisely why we designed our workflow orchestrator differently. Tilebox Tasks form natural checkpoints, and the idempotent nature of them enables efficient re-entrant execution. If something goes wrong, full observability enables developers to quickly dive in and figure out the problem. And it’s all ready-to-go out of the box.

A long running workflow may fail after a lot of the processing work is already completed.

One approach to recover from such failures is to just re-run the whole processing, essentially duplicating all the work that has already been computed.

Another approach to handle such failures is to develop a new workflow capable of picking up partial results from a previously failed run. While this preserves compute resources, it does take valuable development time.

Tilebox re-entrant execution eliminates the necessity for manual recovery tasks and reduces downtime while simplifying the workflow.

After the issue is identified and fixed, versioned tasks allow gradual rollout without disrupting existing clusters / workflows. Then the workflow orchestrator is able to resume the failed job from the state it was in previously to the failure. Re-entrant processing can be manually triggered or configured using automated retries, giving developers another layer of operational resiliency.

Join us on discord for the latest releases, challenges, and Tilebox tips.

Beyond General-Purpose: The Need for Space-Data Native Frameworks

Megan Van Patten — Mon, 17 Feb 2025 00:00:00 GMT

In-house space data pipelines are finicky, to say the least. A patchwork that is often rebuilt for every major upgrade. Workflows that are impossible to iterate because they are baked into Infrastructure as Code. Infrastructure redundancies that consume unnecessary time and resources, like fixed cluster sizes, the overhead of booting up Docker containers for every task, or difficulties in rolling out updates while keeping production up. The inefficiencies are many, but they are all a result of incompatible design across environments, temporary fixes, and data type limitations.

There has been no standard or best practices for space data pipelines. To reach profitability and meet the expectations of real world applications it needs a space-data native framework.

Here’s how we designed ours:

Datasets

Let’s start with the critical foundation of data access and storage. Common database models such as Postgres and even specialized time-series databases like InfluxDB are amazing pieces of engineering, but on their own come with constraints that become real obstacles for space data pipelines, particularly regarding Earth Observation. One lacks support for textual data, the other supports geometries but has limited support for spatial indexing, a critical capability of any data catalog, and a very common reason for performance limitations of the data catalogs of the world.

Instead of wasting time waiting for results or consuming all your CPU on table scans, a framework designed intentionally for these pipelines needs to have:

Support for real-time metadata indexing with customizable data types
Support for arbitrary data types, including strings, polygons, coordinates
Fast, reliable spatio-temporal indexing
A very high performance API to support large queries of telemetry, metadata, or lightweight payload data
Backwards compatible typing and customizable datasets, or catalogs
Standards compliant (STAC) output interfaces where desired

Tilebox affords engineers the freedom to create and edit their own data types – instead of the database schema – without breaking existing software and datasets. Spatio-temporal queries are orders of magnitudes faster than comparable systems. This is essential for streamlined scalability. As operations expand, so will capabilities.

Workflows

While many companies are still figuring it out on the ground, others are preparing for edge computing. Any space-data native framework has to work in orbit. And, we think, everywhere else. Across clouds, mixed environments, and multiple clusters. Data fusion requires this level of flexibility, plus it empowers any team to work with the exact data they need no matter where it’s stored. This expedites processing and analysis, which creates increased revenue opportunities for time-sensitive data.

Efficiency is a key requirement for space data, as its source is far away and its volume is massive. Reducing downlink, transfer, and storage costs is one side of the equation, the other is job resilience. Executing workflows on Spot Instances is a necessary cost-savings for large-scale satellite data processing. But what happens when that Spot Instance goes down? What if a job breaks?

Resiliency affects everything: compute costs, latency, data security.

Manually developed monitoring tools are one of those inefficient redundancies software teams shouldn’t be wasting their time on building and managing. Even Tilebox leverages a service for this, Axiom.co; pre-integrated with Tilebox to deliver the fastest, most thorough distributed observability across your workflows, with a generic OpenTelemetry exporter available as well.

Identifying the break is the first step, re-entrant processing is the second step. Tilebox saves all work up to the breakpoint and automatically initiates a retry, keeping you up and running without manual intervention and rescheduling. And for large workflows, Tilebox supports auto-scaling clusters.

These are not conveniences, they are vital functionalities for a successful space data pipeline.

Future-Proof Your Pipeline

The right tooling makes all the difference. Space data engineers need flexibility, resilience, and purpose-built functionality to stay focused on delivering rather than pipeline maintenance and repair. We are building Tilebox for you and the future of space data services. So, create an account and explore our work – and give us feedback on your ideal tooling. It’s time to evolve space data management with high-performance software made for space data.

Join our discord community for updates on our latest releases.