To sync SMS history into your database, call GET /v1/messages and walk backward through it with before_id: each request returns a page, and you pass the id of the oldest message you've stored to get the next older page. Stop when a page comes back empty or reaches messages you already hold. That's the whole mechanism, and the rest of this post covers the parts that cause bugs: dedupe, gaps, backfill versus catch-up and what the history won't tell you.
We make Ready, so the endpoint shapes below are Ready's. The design applies to any API that pages with a cursor. Check the response fields in the messages reference before you build the schema, since this post doesn't repeat them.
Why mirror history at all
A platform that texts for its customers usually wants message history inside its own product: a thread view in a CRM, an audit trail for a support team, usage reporting or a timeline on a contact record. Reading from the provider on every page load is slow, adds a dependency to a screen your customer opens ten times a day and makes search impossible. A local copy gives you joins, search and an answer when the provider is slow.
The cost is that the copy can drift. So decide up front what's authoritative. A good default is that the provider is the system of record for delivery outcomes and your database is a cache you can rebuild from it.
How before_id pagination works
GET /v1/messages pages with before_id. That's a cursor, not a page number, and the difference matters. With page numbers, a new message arriving during your sync shifts every later page by one, so you skip or repeat a row. With a cursor, "older than this id" stays true no matter what arrives.
The loop looks like this:
- Request the newest page with no
before_id. - Store every message on it, then take the id of the oldest one.
- Request again with
before_idset to that id. - Repeat until a page is empty.
# First page: the newest messages
curl -s https://api.tryready.com/v1/messages \
-H "X-API-Key: $READY_API_KEY"
# Next page: messages older than the last one you stored
curl -s "https://api.tryready.com/v1/messages?before_id=$OLDEST_ID" \
-H "X-API-Key: $READY_API_KEY" The ordering is newest first, so confirm that in the reference too. Treat the response's field names and page size as whatever the reference says, and read them from there instead of guessing. Don't hard-code an assumed page size either. Loop until the page is empty.
Backfill once, then catch up
You'll run two different jobs, and mixing them causes most sync bugs.
| Job | Direction | Stops when | How often |
|---|---|---|---|
| Backfill | Newest to oldest | A page is empty | Once per account, resumable |
| Catch-up | Newest to your newest stored row | A page reaches an id you already hold | Every few minutes, or on a trigger |
Backfill walks all the way back. Save the cursor after each page, so a crash at page 400 restarts at page 400 and not page 1. Run it as a background job, one account at a time, and throttle it. A backfill for a large account is a lot of requests, and you don't want it competing with live sends for your own worker capacity or your API headroom. SMS API rate limits explains why a client should pace itself.
Catch-up starts at the newest page and stops as soon as it meets an id you already store. Because the walk is newest first, a catch-up usually fetches one page.
A subtle point is that catch-up and backfill meet in the middle. A backfill that's half done must not be confused with "I have everything older than my newest row". Track the backfill cursor separately from the catch-up high-water mark, and mark an account history_complete only when backfill ends on an empty page.
Make every write idempotent
Sync code gets rerun: after a crash, after a deploy, after someone clicks "resync". If a rerun inserts duplicates, your thread view shows every text twice.
Use the provider's message id as a unique key on your table and write with an upsert (INSERT ... ON CONFLICT DO UPDATE in Postgres). Two things follow:
- Rerunning a page is harmless. The same ids overwrite themselves with identical data.
- Updates land for free. A message you stored as queued earlier can come back as delivered. The upsert updates the row, so your copy catches up with the delivery outcome.
That second point is why a catch-up that stops at "the first id I know" can leave stale rows behind. If you show delivery status, re-read a window of recent history on each catch-up, say the last day, rather than stopping at the first known id. Or use delivery webhooks for status changes and let history sync handle the rest. Webhook delivery reports covers that side, and receiving texts with webhooks shows how to ack fast and process later.
Polling versus webhooks
History sync by polling is simple and self-healing. Webhooks are fast but can be missed, because your endpoint was down, a deploy dropped a request, or a retry was exhausted.
The pairing that holds up is both. Let webhooks update your database in near real time. Then run the catch-up poll on a schedule as the safety net that repairs anything a webhook missed. Nobody has to remember to run it, and a gap closes itself in a few minutes.
A workable schedule, with stated assumptions:
- Assumption: an account sends and receives a few hundred texts a day.
- A catch-up every 5 minutes fetches one small page almost every time, so the cost is one request per account per run.
- A nightly job re-reads the last 24 hours to refresh delivery status on anything that was still in flight.
If you run this for hundreds of sub-accounts, spread the runs out with random offsets so they don't all hit at the top of the minute.
What history won't tell you
A message log isn't a compliance record, and building one on top of it can mislead you.
- Opt-outs.
GET /v1/opt-outslists only the contacts opted out on this account. A STOP from a deleted contact, a dialer do-not-call entry and a STOP held on another account in the same family are left out. So it isn't a complete mirror. Before a send from another tool, or after a provider migration, checkGET /v1/opt-outs/:phone. The opt-out API post has more. - Consent. Ready records an attestation per API send that sets
consent_attested, not a per-contact consent record. The sender keeps the recipient's opt-in in its own form or CRM. A hosted Ready opt-in page is the exception, and it stores sign-ups per contact. - PHI. Message bodies are copied into your database. Ready isn't HIPAA compliant and doesn't sign BAAs, so keep diagnoses, medications and test results out of texts, and think about retention before you mirror bodies at all.
If your product stores bodies, give your own customers a way to delete a contact's messages. A mirror that you can't prune from will outlast its welcome.
Sync checklist for a multi-tenant platform
- Store the tenant id next to every message, and keep one cursor per sub-account.
- Persist the backfill cursor after every page, not at the end.
- Use the provider's message id as the unique key, and upsert.
- Re-read a recent window to pick up status changes.
- Alert when an account's last successful catch-up is older than your threshold.
- Treat a sudden empty first page on a busy account as a failure to investigate, not a success.
Also keep sync reads out of the way of live sends. Run them from a separate worker pool with a lower concurrency cap, so a large backfill can't slow down the queue that sends your customers' texts.
The practical takeaway
Page backward with before_id, upsert on the provider's message id and keep backfill and catch-up as two separate jobs with two separate cursors. Let webhooks give you speed and a scheduled poll give you correctness. Then be honest about what the copy is: a convenience cache of delivery history, not your consent or opt-out ledger. When you're ready to build it, sign up, send a few test texts and walk your own history with the commands above.