Integrating with Aranea

§ 01 · ACCESS

Getting access

Access is granted per client, per source. Apply from the form on the overview page or directly:

curl -X POST https://your-aranea-host/v1/register \
  -H 'content-type: application/json' \
  -d '{
        "name": "Jane Doe",
        "contact_email": "jane@example.com",
        "company": "Example d.o.o.",
        "intended_use": "Price analytics for the Belgrade rental market"
      }'

The response is always 202 Accepted — including for an address that already applied, so a 202 is not confirmation that a new account was created. An operator reviews every application and, on approval, sends your API key out of band. No endpoint ever returns a key.

Fill in intended_use. It is what the operator actually reads, and it determines which sources you are granted. Applications without it take longer.

§ 02 · AUTHENTICATION

Authenticating

Every endpoint except POST /v1/register takes your key as a bearer token. Keys look like spk_….

curl https://your-aranea-host/v1/sources \
  -H "Authorization: Bearer $ARANEA_KEY"

Keep the key server-side. It is not scoped to an origin and it is not revocable by you — a leaked key means asking the operator to revoke and reissue.

Every authentication failure returns a byte-identical 401: missing header, malformed token, unknown key, revoked key, or an account not yet approved. This is deliberate — the response reveals nothing about which accounts or keys exist. Do not build branching logic on top of it; on a 401, stop and check the key.

§ 03 · SOURCES

Discovering your sources and their schemas

Start here, always. GET /v1/sources is the authoritative statement of what your licence covers, and it carries the JSON Schema that every listing from each source conforms to.

GET /v1/sources

[
  {
    "code": "halooglasi",
    "base_url": "https://www.halooglasi.com",
    "schema": { "type": "object", "properties": { "price_eur": … } },
    "watched_fields": ["price_eur", "area_m2", "status"]
  }
]

code

The source identifier. It is what you pass to ?source= and what appears as listing.source.

schema

A JSON Schema describing the payload of every listing from this source. Generate your types from it, or validate against it on your side. It is null only while a source has no active crawler configuration.

Payload shape differs per source, and can gain fields. Aranea passes each source's validated payload through verbatim rather than flattening every source into one lowest-common-denominator record. Re-read /v1/sources on deploy and treat unknown payload fields as additive — never reject a payload for carrying a field you do not know.

§ 04 · DATA MODEL

The data model

Every read endpoint returns the same listing object. Only payload varies by source.

{
  "id": "0198f3c2-…",              // Aranea's stable id — your primary key
  "source": "halooglasi",
  "source_listing_id": "5425331",  // the id on the source portal
  "url": "https://www.halooglasi.com/…",
  "status": "active",
  "first_seen_at": "2026-06-02T09:14:03Z",
  "updated_at":    "2026-08-09T11:42:51Z",
  "payload": { … }                 // conforms to the source's schema
}

Two fields carry the whole integration:

• id — stable for the life of the listing, including across expiry and return. Key your own store on it and every write becomes an idempotent upsert. Do not key on source_listing_id: it is only unique within a source.
• updated_at — bumped whenever the listing's content or status changes. It is what updated_since filters on and what pages are ordered by, and it is the checkpoint your sync loop persists.

Status

A listing can go from expired back to active if it reappears on the portal, and that transition bumps updated_at like any other change. Treat any status you do not recognise as not-active rather than failing — gone is reserved and currently unused.

The same values filter a search: ?status=active. Anything else is a 400 rather than an empty page, so a typo in a sync job surfaces immediately instead of looking like a quiet feed.

§ 05 · BACKFILL

Step one — the initial backfill

Page through GET /v1/listings with no updated_since to pull everything you are licensed for. Results are ordered by updated_at descending, then id descending, and paginated with an opaque cursor.

GET /v1/listings?limit=100

{
  "items": [ … ],
  "next_cursor": "MjAyNi0wOC0wOVQxMTo0Mjo1MS4…"
}

Pass next_cursor back as ?cursor= and repeat until it comes back null. Treat the cursor as opaque — its encoding is not part of the contract.

The last request legitimately returns an empty page. A page that exactly fills limit always carries a cursor, so a run whose final page is exactly full will make one more request that returns items: [] and a null cursor. That is the terminator, not an error — loop on next_cursor, never on whether items is empty.

Use limit=100, the maximum. Values outside 1–100 are clamped rather than rejected, so an out-of-range limit fails silently instead of loudly — pass a valid one.

§ 06 · DELTA SYNC

Step two — staying in sync

After the backfill, never pull the world again. Persist a checkpoint and pass it as updated_since; you get back only what changed at or after that instant.

GET /v1/listings?updated_since=2026-08-09T11:42:51Z&limit=100

The same loop serves both phases — the backfill is just a sync with no checkpoint yet:

import os, time, requests

BASE = "https://your-aranea-host"
session = requests.Session()
session.headers["Authorization"] = f"Bearer {os.environ['ARANEA_KEY']}"

def get_page(params):
    while True:
        r = session.get(f"{BASE}/v1/listings", params=params, timeout=30)
        if r.status_code == 429:
            time.sleep(int(r.headers.get("Retry-After", "60")))
            continue
        r.raise_for_status()
        return r.json()

def sync(since=None):
    """Pulls everything changed at or after `since`; returns the next
    checkpoint. Pass None for the initial backfill."""
    params = {"limit": 100}
    if since:
        params["updated_since"] = since
    checkpoint, first = since, True

    while True:
        page = get_page(params)
        if first and page["items"]:
            # Newest-first ordering makes page one's head the high-water
            # mark. Take it before consuming: anything that changes during
            # the run then lands above it and is caught by the next sync
            # rather than silently skipped.
            checkpoint = page["items"][0]["updated_at"]
        first = False

        for listing in page["items"]:
            upsert(listing)          # keyed on listing["id"] — idempotent

        if not page["next_cursor"]:
            return checkpoint
        params["cursor"] = page["next_cursor"]

Three properties of this loop matter, and skipping any of them produces an integration that loses records:

• The checkpoint comes from page one, not the last page. Pages descend, so page one holds the newest updated_at in the run. A listing modified while you are paginating gets an updated_at above that mark and is picked up next time.
• updated_since is inclusive. Every sync re-delivers the listing(s) sitting exactly on the checkpoint. This is intentional — an exclusive bound loses records that share a timestamp — so writes must be idempotent upserts keyed on id.
• Persist the checkpoint only after the run commits. If the loop dies halfway, re-running from the old checkpoint replays harmlessly; advancing it early loses whatever was not written.

Poll as often as your rate limit comfortably allows — every few minutes is typical. There is no penalty for a poll that returns nothing.

§ 07 · HISTORY

Version history

Every client-visible change is versioned. GET /v1/listings/{id}/history returns the full timeline, newest first:

{
  "items": [
    { "fetched_at": "2026-08-09T11:42:51Z", "payload": { "price_eur": 132000, … } },
    { "fetched_at": "2026-07-14T08:03:12Z", "payload": { "price_eur": 139000, … } }
  ]
}

Each entry is a whole payload, not a diff — compare consecutive entries to derive what changed, e.g. to build a price curve. The timeline is not paginated, so fetch it per listing on demand rather than sweeping it for every record.

GET /v1/listings/{id} returns a single current listing. Both endpoints answer 404 for an unknown id and for one outside your grants alike — the API does not distinguish them.

§ 08 · OPERATIONS

Rate limits, errors and retries

Rate limits

Requests are counted per client in a fixed one-minute window; the ceiling is set on your licence. Exceeding it returns 429 with a Retry-After header holding the seconds remaining in the current window — wait exactly that long and your next request is admitted. Registration is limited separately, by IP, per hour, and answers the same way.

Status codes

The error envelope

Every failure answers with the same body — including ones raised before your request reaches a handler, like a malformed JSON body, an unparseable parameter, an unknown path or a wrong method:

{ "error": { "message": "malformed cursor" } }

Branch on the status code, not the message. message is diagnostic text for your logs, not a stable machine identifier — it may be reworded at any time. It is also deliberately constant for 401 and 404, so neither can be used to probe for which keys, accounts or listings exist.

§ 09 · BEFORE YOU GO LIVE

Integration checklist

• Key held server-side, in a secret store — never in a browser or a repository.
• Types generated from, or payloads validated against, each source's schema from /v1/sources.
• Unknown payload fields tolerated, not rejected.
• Local store keyed on listing id; every write an idempotent upsert.
• Sync checkpoint taken from page one and persisted only after the run commits.
• Pagination loops on next_cursor, and tolerates a final empty page.
• 429 handled by honouring Retry-After; 5xx by backoff with jitter.
• Delta sync running on its own schedule, checkpointed off the newest updated_at of the first page.

Field-level detail for every endpoint lives in the API reference, and the machine-readable contract is at /v1/openapi.json — point your client generator straight at it.