Monitoring AO through the API

Each of AO's services answers a health check that needs no sign-in, so an uptime monitor or dashboard, such as Uptime Kuma, Grafana or Nagios, can watch AO without an AO account. With an Admin's sign-in, a dashboard can also show sign-ins, people, storage by project and Remote Sync transfers.

Health checks

Send GET https://<your AO address>/api/v1/serverstates/<service>/heartbeat. No sign-in is needed. Each answer also gives the build's commit and date, which My Account > Version shows under the same names.

The same services in My Account > Version, with each one's code and build date.

For example:

curl -s https://ao.example.com/api/v1/serverstates/geo/heartbeat

answers:

{
  "apiVersion": "1.12.0",
  "applicationStatus": "ok",
  "builtDateTime": "2026-10-21T16:32:58+00:00",
  "builtEpoch": "1792600378",
  "dbConnectivity": "ok",
  "gitCommit": "49312e5624a9ca3536cdf0074212b2fc432fd5f6",
  "status": "alive"
}

What to alert on

• Check the answer, not just the status code. geo, survey and file answer 200 even when AO's database is down: dbConnectivity then says error. In Uptime Kuma, use the monitor type HTTP(s) - Json Query with the expression dbConnectivity and the expected value ok.
• Watch the database through geo, survey or file. message always reports dbConnectivity as ok.
• Check toolchain and cesium less often, such as every 5 minutes. Each check asks the worker for an answer and can take up to 5 seconds.
• Don't alert on cesium alone. The Cesium service doesn't always run. When it isn't running, its check answers 504 and the Version tab shows Unavailable (Version information is available when the Cesium service is active.) Look into it if visualizations also stop finishing.
• asset-server checks its storage about once a minute, so its answer can be up to a minute old.
• Poll the other checks every 1 to 5 minutes, with a timeout of about 15 seconds.

A check from a script

This Python script (download check_ao.py) checks every service except cesium and exits with 1 if any check fails, so cron, Nagios or a CI job can run it. It needs only Python 3, with no extra packages, and works without internet access on your own network. It names itself in the User-Agent header, because some proxies, such as Cloudflare, refuse Python's default.

#!/usr/bin/env python3
"""Check AO's health checks. Usage: python3 check_ao.py https://ao.example.com"""
import json
import sys
import urllib.error
import urllib.request

ao = sys.argv[1].rstrip("/")
failed = []
for service in ["geo", "survey", "file", "asset-server", "toolchain", "message"]:
    url = f"{ao}/api/v1/serverstates/{service}/heartbeat"
    request = urllib.request.Request(url, headers={"User-Agent": "ao-health-check"})
    try:
        with urllib.request.urlopen(request, timeout=15) as response:
            answer = json.load(response)
        database, built = answer.get("dbConnectivity"), answer.get("builtDateTime")
        healthy = database in (None, "ok")
        print(f"{service:13} {response.status}  database {database}  built {built}")
    except urllib.error.HTTPError as error:
        healthy = False
        print(f"{service:13} {error.code}")
    except (urllib.error.URLError, TimeoutError) as error:
        healthy = False
        print(f"{service:13} no answer ({error})")
    except ValueError:
        healthy = False
        print(f"{service:13} the answer isn't a health check (a sign-in page or proxy in the way?)")
    if not healthy:
        failed.append(service)

if failed:
    print("Failed:", ", ".join(failed))
    sys.exit(1)

Activity and usage

These calls need an AO sign-in: see Getting started with the AO API. Use your organization's uuid from GET /api/v1/users/me, under items.organization_details.

Good to know

• Health checks need no account, so a monitor that only checks health never stores an AO password.
• For the calls that need a sign-in, sign in once and renew with the refresh token, rather than signing in on every poll. Each sign-in starts a new session, an account keeps at most 10, and the oldest is signed out. On MCP-enabled AO, a Read only API token is simpler: it isn't a session and can't change anything. See Ways to sign in in Getting started with the AO API.
• Poll storage by project at most every hour or so. It adds up every file, visualization and object, which takes a while on a large AO. It answers 404 when the organization has no projects.
• The sign-in record has a logout row for every session, timed at the session's last activity, even when nobody signed out.
• AO's checks don't cover the server itself. On self-hosted AO and AO OnBoard, also watch disk space and the server with your usual tools. Everything on this page works on your own network without internet access.
• AO in the cloud and AO for government: Terradepth watches these. Their status page is status.ao.terradepth.com.

Related: Getting started with the AO API · Upload and download files with a script · Troubleshooting and getting help