A Kafka consumer that sends texts needs three rules. Key messages by recipient so one person's texts stay in order, use the event id as the Idempotency-Key so a redelivery never texts twice, and commit the offset only after the SMS API has answered. Kafka gives you at-least-once delivery, and these rules turn that into one text per event.
We make Ready, so the example calls our REST API with the one request shape we document. The consumer code uses KafkaJS as a familiar example; the same logic applies in any client. It assumes Node 18 or later.
Choose the partition key on purpose
Kafka orders messages within a partition, not across a topic. The key decides the partition, so the key decides what stays in order.
For texting, the usual choices are:
| Key | Ordering you get | Risk |
|---|---|---|
| Recipient phone number (E.164) | One person's texts arrive in order | Fine for most loads |
| Customer or tenant id | All of a tenant's texts in order | One busy tenant overloads one partition |
| No key | None | A reminder can overtake the booking it refers to |
Key by recipient unless you have a reason not to. A confirmation followed by a change notice should never swap places, and keying by phone number guarantees one consumer handles both one after the other. Keying by tenant sounds tidy for rate limits, but it turns your largest customer into a hot partition. If you need per-tenant fairness, handle it in the consumer instead, as covered in our guide to SMS queue design for platforms.
Put the idempotency key in the event
Kafka may deliver the same record twice after a rebalance or a crash. The SMS API protects you only if the second attempt carries the same Idempotency-Key as the first.
Don't derive the key from the partition and offset. A replay into a new topic changes the offset. Mint a UUID when the event is produced, store it in the payload, and send it as the header on every attempt. Our post on deterministic idempotency keys for SMS covers when to derive one from the business event instead, such as order id plus message type.
await producer.send({
topic: 'sms.requests',
messages: [{
key: '+12125550199',
value: JSON.stringify({
event_id: crypto.randomUUID(),
to: '+12125550199',
body: 'Your order #1042 has shipped.',
}),
}],
}); Commit after the send, not before
With auto-commit on, the client may mark a record done before your code has sent anything, and a crash then loses the text. Turn it off and commit by hand.
await consumer.run({
autoCommit: false,
eachMessage: async ({ topic, partition, message }) => {
const event = JSON.parse(message.value.toString());
await sendWithRetry(event);
await consumer.commitOffsets([
{ topic, partition, offset: (Number(message.offset) + 1).toString() },
]);
},
}); The offset you commit is the next one to read, so add one. If the process dies between the send and the commit, the record comes back and the same Idempotency-Key makes the repeat harmless. That window is the whole reason the key exists.
Retry by status code
Not every failure deserves another try, and a blocked partition holds up every text behind it.
async function sendWithRetry(event) {
for (let attempt = 1; attempt <= 4; attempt++) {
let res;
try {
res = await fetch('https://api.tryready.com/v1/messages', {
method: 'POST',
headers: {
'X-API-Key': process.env.READY_API_KEY,
'Content-Type': 'application/json',
'Idempotency-Key': event.event_id,
},
body: JSON.stringify({ to: event.to, body: event.body, consent_attested: true }),
signal: AbortSignal.timeout(35000) // the API may take up to 30 s to answer,
});
} catch {
await sleep(2 ** attempt * 500);
continue;
}
if (res.ok) return;
if (res.status === 429 || (res.status === 409 && (await res.text()).includes('idempotency_in_progress'))) {
await sleep(2 ** attempt * 500);
continue;
}
return sendToDeadLetter(event, res.status);
}
return sendToRetryTopic(event);
} Read consent_attested from your consent record for that contact rather than hardcoding it. Until January 20, 2027 (UTC) a send without it still goes out with a warning and no attestation; from that date the API refuses it with 403 api_consent_attested_required.
The branches work like this:
- 2xx: done, commit.
- 429, a timeout, or a 409
idempotency_in_progress: wait and retry with the same key, with a small ceiling. The 409 means the first try is still running, so the timeout has to be longer than the API's 30 seconds. - 503: nothing was sent or charged, and the same key would replay the stored error. Dead-letter it with the status, and a replay job republishes it with a new key.
- 500 or 504: check
GET /v1/messagesbefore sending again, because the message may have gone out. The dead letter topic holds it until that check is done. - Other 4xx: the request is wrong and will fail the same way on every attempt. Send it to a dead letter topic and move on.
- Retries exhausted: publish to a retry topic with a delay, then commit, so the partition keeps moving.
A 409 QUIET_HOURS_BLOCK is a special case. The API holds a text only when the sending window opens within a short time. Otherwise it refuses, and the sender resends once the window opens, so route that event to the delayed retry topic rather than the dead letter topic. See handling the quiet hours 409 for the details, and the API error reference for the other codes.
Keep the consumer from overloading the API
A consumer can read far faster than a carrier-facing API should be called. Set partitionsConsumedConcurrently low, put a shared limiter in front of the fetch, and let consumer lag absorb the burst. Lag is a healthy sign here: it means the topic is doing its job as a buffer. Check your own rate limits in the messages API docs before you pick a number, and run a test at your expected peak using our notes on load testing before launch.
Takeaways
Key by recipient, carry an event id from producer to API, commit after the send, and sort failures into retry, delay or dead letter. If your texts begin as database writes, put an outbox in front of the producer so a text is never lost between the commit and the publish; our post on the transactional outbox pattern shows how. When you want to wire a consumer to a real number, sign up and send a first message.