A resilient client for CoreLogic property data

Enterprise property data is some of the most valuable information a real-estate product can carry, and some of the most awkward to fetch. This is a look at how a property platform built a small, resilient client over CoreLogic (the data provider now branded Cotality), pulling ownership, building, tax, sale, and roof-age facts for any address. The interesting part is not the data model; it is everything the client has to survive to get the data at all. Tokens expire mid-request, an edge WAF sometimes answers with an HTML block page where JSON was expected, and the two API realms the platform depends on authenticate as two different clients. The writeup covers what the data provides, why a finicky enterprise API needs a dedicated resilience layer, how that layer is built, how it behaves across real requests, and the compromises baked into the design.

The interesting part is not the property data model. It is everything the client has to survive to get the data at all.

What does CoreLogic property data actually provide?

CoreLogic (Cotality) aggregates public-record and derived property data into APIs a product can query per address, instead of scraping county registers one jurisdiction at a time. For this platform, an address is enriched with a property's ownership history, building characteristics, tax assessment, most recent transfer, last market sale, and an estimated roof age. Those facts feed a property report and a downstream data-completeness classification that decides how much the platform can say about a given home.

The data arrives across two distinct realms. The Property API resolves an address to a CLIP - CoreLogic's stable property identifier - and then returns the structured detail (buildings, ownership, site location, tax assessment, transfers, sales) for that CLIP. The Spatial API answers a narrower, geospatial question: given a full address, roughly how old is the roof? The two realms are separate products with separate endpoints, and, as it turns out, separate credentials.

The signal is valuable precisely because it is external and authoritative. A user can mistype their own square footage or forget when they bought the house; a public-record-backed API is far harder to argue with. That is what makes it worth the integration effort, and worth building a client that keeps working when the provider's edge is having a bad day. For a SaaS product that sells trustworthy property reports, the enrichment layer is not a nice-to-have; it is the product.

Why does one enterprise data API need a resilient client?

Enterprise data APIs fail in ways consumer APIs usually do not, and they fail in the middle of otherwise healthy traffic. Three failure modes drove the design here. Access tokens are short-lived and expire without warning, so a request that authenticated a minute ago can come back 401. The provider sits behind an edge WAF (web application firewall, the security layer that screens traffic before it reaches the API) that occasionally answers a legitimate request with an HTML block page, which turns a naive res.json() into a hard parse error. And the service rate-limits, so a burst of lookups can draw a 429 that a single-shot client would simply surface as a failure.

None of these are exotic, but together they make a thin fetch(url).then(r => r.json()) client unreliable in production. The cost of getting it wrong is not just a failed request; it is a half-built property report, or a lookup that fails on a transient token expiry that a single retry would have fixed. The resilient client exists so those failure modes are handled once, in one place, rather than being rediscovered at every call site.

There is a second, quieter reason: two realms. Because Property and Spatial authenticate as different OAuth clients against different token endpoints, the naive approach would spread two credential handshakes and two token lifecycles across the codebase. Centralizing them behind one client keeps that complexity contained. This is the kind of API integration work where most of the value is in the edges, not the happy path.

How does the client talk to two API realms at once?

The two realms are modeled explicitly as a small config record, and every request names the realm it targets. There is one generic fetch function; the realm is its first argument. That is the whole trick that makes two independently-authenticated APIs feel like one client.

The layered architecture: one resilient fetch layer sits between the orchestrator and the two realms, and a database row caches the merged result so the provider is queried as rarely as possible.

The config carries a token URL, a base URL, and a client id and secret per realm, all overridable by environment variable so the same code points at UAT (the vendor's user-acceptance-testing sandbox) or production without a change. Note the two different apex domains: Property lives on corelogicapi.com, Spatial on corelogic.com. Naming them side by side in one record is what keeps the "two realms" reality honest and visible rather than smeared across the app.

type Realm = 'property' | 'spatial';

interface RealmConfig {
  name: Realm;
  tokenUrl: string;
  baseUrl: string;
  key: string | undefined;
  secret: string | undefined;
}

const realms: Record<Realm, RealmConfig> = {
  property: {
    name: 'property',
    tokenUrl: process.env.COTALITY_PROPERTY_TOKEN_URL ??
      'https://property-uat.corelogicapi.com/oauth/token?grant_type=client_credentials',
    baseUrl: process.env.COTALITY_PROPERTY_BASE_URL ?? 'https://property-uat.corelogicapi.com',
    key: process.env.COTALITY_PROPERTY_KEY,
    secret: process.env.COTALITY_PROPERTY_SECRET,
  },
  spatial: {
    name: 'spatial',
    tokenUrl: process.env.COTALITY_SPATIAL_TOKEN_URL ??
      'https://api-uat.corelogic.com/oauth/token?grant_type=client_credentials',
    baseUrl: process.env.COTALITY_SPATIAL_BASE_URL ?? 'https://api-uat.corelogic.com',
    key: process.env.COTALITY_SPATIAL_KEY,
    secret: process.env.COTALITY_SPATIAL_SECRET,
  },
};

It is worth being precise about the phrase "one OAuth2 client". The rest of the application sees one client, but under the hood there is no shared credential: each realm runs its own client_credentials grant, with its own key and secret, against its own token URL. The unification is a code-path unification, not a credential one - and that distinction is exactly what the token cache below has to respect. According to the OAuth 2.0 Authorization Framework (RFC 6749, published October 2012), the client credentials grant is designed for machine-to-machine access where no end user is present, which is exactly the shape of a server calling a data provider.

The two realms are similar in how they authenticate but differ sharply in how much resilience wraps them, which is easiest to read side by side.

How does token caching work?

Each realm keeps its own access token in an in-memory cache, tagged with the expiry the token endpoint returned, and reuses it until shortly before it expires. The grant is client_credentials; the credentials go up as HTTP Basic auth, and the grant type rides on the token URL's query string rather than a request body.

The one detail that matters most is the refresh buffer. A cached token is only reused while the current time is more than five minutes before its expiry. That buffer avoids a nasty race: a token that looks valid at the moment you check it can expire in the milliseconds between the check and the server receiving the request. Refreshing five minutes early makes that class of 401 almost impossible.

const TOKEN_REFRESH_BUFFER_S = 300; // refresh 5 minutes early

interface TokenCacheEntry { token: string | null; expiresAt: number; }
const tokenCache: Record<Realm, TokenCacheEntry> = {
  property: { token: null, expiresAt: 0 },
  spatial: { token: null, expiresAt: 0 },
};

async function getAccessToken(realm: Realm): Promise<string> {
  const cfg = realms[realm];
  const cache = tokenCache[realm];
  const now = Math.floor(Date.now() / 1000);

  if (cache.token && now < cache.expiresAt - TOKEN_REFRESH_BUFFER_S) {
    return cache.token; // still comfortably valid
  }

  const basic = Buffer.from(`${cfg.key}:${cfg.secret}`).toString('base64');
  const res = await fetch(cfg.tokenUrl, {
    method: 'POST',
    headers: {
      Authorization: `Basic ${basic}`,
      'Content-Type': 'application/x-www-form-urlencoded',
      Accept: 'application/json',
    },
  });

  const data = (await res.json()) as { access_token: string; expires_in: number };
  cache.token = data.access_token;
  cache.expiresAt = now + data.expires_in;
  return cache.token;
}

Because the cache is keyed by realm, a Property token and a Spatial token live and expire independently. Clearing one (which the 401 path below does) never disturbs the other. The cache is also process-local, which is fine for this workload but is a scaling assumption worth naming - more on that in the gotchas.

How does the client retry, back off, and re-authenticate?

All of the retry, backoff, and re-auth logic lives in the one generic fetch function, so every Property call inherits it for free. The shape is a bounded loop: up to two retries on top of the first attempt, an exponential delay before each retry, and two special cases - 401 and 429 - that feed back into the same loop instead of failing.

A 401 mid-request, re-authenticated in place: the cache is cleared, a fresh token is exchanged after one backoff, and the original request is replayed. The caller only sees a slightly slower success.

A 401 is treated as a stale token, not a dead end. The client clears that realm's cached token, then continues; the next iteration calls getAccessToken, finds an empty cache, exchanges fresh credentials, and replays the original request with the new bearer token. A 429 simply backs off and retries. Backoff is a clean exponential (500 ms, then 1000 ms) with no jitter, and client errors other than 401/429 are not retried at all - there is no point retrying a 400.

There is a known shortcut here worth calling out early. According to MDN's HTTP reference, a 429 Too Many Requests response should carry a Retry-After header telling the client exactly how long to wait, and a well-behaved client reads it. This client does not: it applies the same fixed exponential backoff to a 429 as to any other retry and ignores Retry-After entirely. At the current request volume that is harmless, but it is the first thing to change if throughput grows (see the gotchas below).

const MAX_RETRIES = 2;
const INITIAL_BACKOFF_MS = 500;

async function cotalityFetch<T>(realm: Realm, path: string): Promise<T> {
  let lastError: unknown;

  for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
    if (attempt > 0) {
      const delay = INITIAL_BACKOFF_MS * Math.pow(2, attempt - 1); // 500ms, 1000ms
      await new Promise((r) => setTimeout(r, delay));
    }

    const token = await getAccessToken(realm);
    const res = await fetch(`${realms[realm].baseUrl}${path}`, {
      headers: { Authorization: `Bearer ${token}`, Accept: 'application/json' },
    });

    if (res.ok) return (await res.json()) as T;

    if (res.status === 401 && attempt < MAX_RETRIES) {
      tokenCache[realm].token = null; // force a fresh token, then replay
      tokenCache[realm].expiresAt = 0;
      lastError = new CotalityError('Token expired, retrying', 401);
      continue;
    }

    if (res.status === 429 && attempt < MAX_RETRIES) {
      lastError = new CotalityError('Rate limited', 429);
      continue; // note: does not read Retry-After
    }

    throw new CotalityError(await readErrorBody(res), res.status);
  }

  throw lastError ?? new CotalityError('Request failed', 500);
}

The readErrorBody call on the final throw is where WAF detection happens, which is the next piece.

How does the client detect a WAF block page?

The client inspects every non-OK response body before it trusts it to be JSON, and recognizes the classic "expected JSON, got an HTML block page" case. When a WAF or CDN edge decides a request looks like a bot, it does not return a tidy 403 { "error": ... }; it returns a full HTML page with a 200-ish shell and an incident id. A client that blindly calls res.json() on that gets an opaque parse error and no idea why.

Instead, the error reader checks the Content-Type and sniffs the start of the body for <!doctype or <html>. If it looks like HTML, it fingerprints the vendor from telltale strings (Imperva/Incapsula, Cloudflare, or a generic CDN edge otherwise), pulls out the incident id if one is present, and returns a human-readable diagnostic that gets attached to the thrown error and logged. The message says, in effect, "this was a WAF block page, here is the incident id, the URL is probably wrong or you are being treated as a bot" - which is worth far more at 2 a.m. than a SyntaxError: Unexpected token <. According to Imperva, its Incapsula block pages embed an incident id precisely so support can trace why a specific request was stopped, so capturing that id turns an opaque block into a ticketable event.

async function readErrorBody(res: Response): Promise<string> {
  const contentType = res.headers.get('content-type') ?? '';
  const text = await res.text();
  const isHtml = contentType.includes('html') || /^\s*<(!doctype|html)/i.test(text);

  if (isHtml) {
    const incident = text.match(/incident_id=([^&"]+)/i)?.[1];
    const waf = /incapsula/i.test(text)
      ? 'Incapsula/Imperva WAF'
      : /cloudflare/i.test(text)
        ? 'Cloudflare WAF'
        : 'WAF/CDN edge';
    return `<HTML response - looks like ${waf} block page${incident ? ` (incident=${incident})` : ''}. ` +
      `The URL is probably wrong or the request is being treated as a bot.>`;
  }

  return text.length > 500 ? `${text.slice(0, 500)}...` : text;
}

The client does not try to programmatically defeat the WAF; that would be a losing game. The actual mitigation is upstream: the token and API requests go out with a browser-like User-Agent so a legitimate server-to-server call is less likely to be flagged as automated in the first place. WAF detection is the safety net for when that is not enough, turning a silent, confusing failure into a labeled, actionable one.

What happens when the Spatial realm is down?

The Spatial roof-age call is intentionally held to a lower standard than the Property calls: it is fail-soft. It shares the token cache (it still calls getAccessToken('spatial')), but it does not go through the resilient fetch layer at all - no retries, no backoff, no 401 replay. On a 204 No Content, any non-OK status, or a thrown exception, it simply returns null and logs a warning.

That is a deliberate priority call. Roof age is one enriching field on a report, not the report itself. The orchestrator runs the property-detail call and the roof-age call in parallel with Promise.all, then merges whatever came back. If Spatial times out or returns a block page, roof age is null and the rest of the report is unaffected.

The data flow: search resolves a CLIP, then detail and roof-age run in parallel. The Property calls get the full resilience budget; the Spatial call fails soft to null, so an outage there costs one field, not the report. The expensive, must-have data (ownership, building, tax, sale) gets the full resilience budget; the nice-to-have data gets a graceful shrug. Choosing which calls deserve which treatment is a core part of designing a backend integration that stays up when a dependency does not.

How does it behave across three real scenarios?

The design is easiest to understand through the requests it was built for. Three cases cover the range.

Scenario 1: a warm cache and a clean lookup

An address comes in and both realm tokens are already cached and comfortably inside their windows. The Property search resolves a CLIP on the first attempt, the detail and roof-age calls fire in parallel, and everything returns 200. No token exchange, no retry, no backoff. This is the common case, and it is fast because the five-minute buffer means the cache is almost always warm across a burst of lookups. The merged result is written to the property row, so the very next lookup for that address skips the API entirely.

Scenario 2: a token expires mid-request

A detail call goes out with a token that was valid at check time but has just expired, and the server returns 401. The client nulls the Property token, waits the 500 ms backoff, calls getAccessToken again (which now exchanges fresh credentials), and replays the request with the new bearer token, which succeeds. The Spatial token is untouched throughout. The caller never sees the 401; it sees a slightly slower success. This is exactly the failure the buffer is meant to prevent, caught by the re-auth path on the rare occasion it slips through.

Scenario 3: a WAF block page and a Spatial outage at once

A detail request is answered by the edge WAF with an HTML block page carrying an Imperva incident id, and at the same time the Spatial API is timing out. On the Property side, the client reads the body, recognizes the block page, and after exhausting its retries throws a CotalityError whose message names the WAF and the incident id - so the log line is actionable rather than a cryptic parse error. On the Spatial side, the fail-soft call returns null. The call site (covered next) catches the Property error and proceeds with whatever it has, so the outcome is a degraded report plus a precise breadcrumb for whoever investigates.

Where is the client actually called?

The client is consumed server-side through one orchestrator, and the call sites treat a failure as recoverable rather than fatal. The report and classification flows both prefer data already stored on the property row and only reach for the API when there is nothing cached. When they do call, they wrap it in a try/catch, log the CotalityError status and message, and continue with null so a provider outage never blocks a report from being created.

let cotalityData: CotalityData | null = null;
try {
  cotalityData = await lookupCotalityData(addressComponents);
} catch (err) {
  const status = err instanceof CotalityError ? err.status : '-';
  console.warn(`[classify] cotality lookup failed status=${status} - ${(err as Error).message}`);
}
const { tier, reasons } = classifyPropertyCompleteness(cotalityData);

This is the second layer of caching, and it mirrors the pattern we reach for across our case studies: compute an expensive external signal once, store it where the product already reads, and treat the provider as a source of truth to be consulted sparingly rather than a dependency on the hot path. The in-memory token cache keeps auth cheap within a process; the database row keeps the whole enriched result cheap across processes and restarts. Once an address has been enriched, the platform recomputes its classification from the stored JSON without touching CoreLogic again, which both saves quota and insulates the product from the provider's uptime.

What are the risks and gotchas?

The design makes several deliberate compromises, and knowing them matters for anyone auditing the integration or extending it.

Retry-After is not honored. A 429 is retried on the same fixed exponential schedule as everything else; the server's Retry-After header is ignored. At low volume this is harmless, but under real rate-limit pressure it means the client can retry sooner than the provider asked, which is exactly when you least want to.

There is no concurrency throttle. Nothing caps in-flight requests. A batch job that enriches many addresses at once can fan out unbounded parallel calls, walk straight into the provider's rate limit, and then hammer it with jitter-free retries. A client-side token bucket would prevent the 429s the backoff is currently mopping up.

Backoff has no jitter. Retries are deterministic (500 ms, 1000 ms). If several requests are rate-limited at the same instant, they all back off in lockstep and retry in the same instant - a small thundering-herd effect. Adding randomized jitter is a one-line fix that meaningfully smooths retry storms.

The Spatial call has no resilience. Roof age gets no retries, no backoff, and no 401 replay. That is acceptable because it is fail-soft, but it also means a transient Spatial blip that a single retry would have fixed silently drops the field. The fail-soft default quietly hides recoverable errors as missing data.

The token response is logged in full. On a successful token exchange the raw token JSON is logged, which puts a live access token in the logs. The config dump masks the key and secret, but the token itself should be redacted the same way. It is the one place the otherwise careful secret handling slips.

Defaults point at UAT. Every base and token URL defaults to the UAT environment; production is reached only by setting the environment variables. That is a safe default for development but a sharp edge in deployment - a missing env var fails closed to test data rather than loudly erroring, so a misconfigured production deploy can silently serve UAT results.

What would we do differently?

Before the changes, it helps to weigh what the design already gets right against what it still leaves open, because the two lists decide which fixes are worth the effort.

The highest-value change is to honor Retry-After and add a client-side concurrency limit. Together they would turn rate-limit handling from reactive (back off after a 429) into proactive (never exceed the limit in the first place), which matters the moment enrichment moves from one-at-a-time to batch. Adding jitter to the backoff is a trivial companion change that removes the lockstep-retry risk.

Three smaller changes follow. Redact the access token in the success log the same way the key and secret are masked. Give the Spatial call at least one retry before it falls back to null, so a single transient error stops costing a field. And make the environment explicit - fail loudly when production credentials are missing rather than silently defaulting to UAT. None of these change the architecture; they close the gaps a resilient client can least afford to leave open. This is the kind of hardening a short software development pass handles well before an integration carries real traffic.

Frequently asked questions

Is this really one OAuth2 client shared across both API realms?

Not literally. Each realm - Property and Spatial - has its own client id and secret, its own token URL, and its own token cache. What is shared is one generic fetch function that takes the realm as an argument, so the two realms look like one client to the rest of the app even though they authenticate independently.

How does the client avoid re-authenticating on every request?

It caches each realm's access token in memory with the expiry the token endpoint returns, and reuses it until five minutes before it expires. That early-refresh buffer means a token is never used right up to the edge of its lifetime, which avoids a race where a token that looked valid at send time has expired by the time it arrives.

What happens when a request comes back 401 mid-flight?

The client treats a 401 as a stale token rather than a hard failure. It nulls that realm's cached token, forces a fresh client-credentials exchange on the next loop iteration, and replays the original request with the new bearer token, all within the same retry budget. Only that realm's cache is cleared, so the other realm keeps its token.

Does the client respect the Retry-After header on a 429?

No, and this is a known gap. A 429 is retried, but on the same fixed exponential schedule (500 ms then 1000 ms) that every other retry uses; the Retry-After header the server sends is not read. For the current low request volume that is acceptable, but honoring Retry-After would be the first change if throughput grew.

What happens if the Spatial roof-age API is down?

The report is still produced. The roof-age call runs in parallel with the property-detail call and is deliberately fail-soft: on a 204, any non-OK status, or a thrown error it returns null instead of throwing. A Spatial outage therefore costs one optional field, not the whole lookup.

Sources and further reading

• OAuth 2.0 client credentials grant (RFC 6749, section 4.4): https://datatracker.ietf.org/doc/html/rfc6749#section-4.4
• MDN - the HTTP Retry-After response header: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Retry-After
• MDN - using the Fetch API: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch
• CoreLogic (Cotality) - property data and analytics provider: https://www.corelogic.com/
• Imperva - Incapsula WAF and incident ids: https://www.imperva.com/products/web-application-firewall-waf/

What's next

The client does one well-defined job today: it turns two finicky, separately-authenticated enterprise data realms into one typed lookup that survives token expiry, rate limits, and edge block pages, and it caches the result so the provider is queried as rarely as possible. The clearest next steps are to honor Retry-After with a client-side rate limit, add jitter to the backoff, and redact the token in logs. If you are wiring a temperamental third-party data API into a product and want the client to be both resilient and honest about its own failure modes, get in touch - it is the kind of integration work we do often.