To stop one tenant's big campaign from starving the others, put a token bucket in front of each tenant and drain the tenants through a weighted round robin queue, so the shared send rate gets divided instead of won by whoever enqueued first. A single FIFO queue is the thing that fails: one customer queues 50,000 texts, and every appointment reminder behind them waits.
This is a guide for platforms that text on behalf of their own customers. The shared pipe is finite, so fairness is your job, not the carrier's. We make Ready, so the one API call in the sketch is ours, but the scheduling logic works with any sending API.
Why a plain queue starves small tenants
Say your platform sends through one registered campaign with a send rate of about 10 segments per second. That figure is an assumption for this example. Carriers publish no official rate card, and a fresh standard campaign with no vetting is closer to 1 to 4 segments per second, while higher use-case tiers run from roughly 10 to 75. Whatever your number is, it's one number for the whole pipe.
Now one tenant enqueues 36,000 texts at 9:00 AM. At 10 segments per second that's an hour of work. A second tenant enqueues a single reminder at 9:01. In a FIFO queue it goes out at 10:00, long after the appointment it was about.
Nothing is broken. The queue did exactly what a queue does. The fix is to stop treating the pipe as one line.
Two layers, two jobs
Fair sharing takes two separate mechanisms. People often build one and expect it to do both.
| Layer | Question it answers | Mechanism |
|---|---|---|
| Admission | Is this tenant allowed to add more work right now? | Per-tenant token bucket |
| Dispatch | Whose message goes out next? | Weighted round robin over per-tenant queues |
The token bucket caps how fast a tenant can send. The round robin makes sure that when several tenants have work waiting, each gets its share of the pipe. Without the bucket, a tenant can hold the pipe at full speed whenever it's alone. Without the round robin, the bucket does nothing for the tenants stuck behind a long backlog.
The architecture side of this, including where tenant IDs live, is in our multi-tenant SMS architecture guide. The queue basics are in SMS queue design for platforms.
Per-tenant token buckets
A token bucket holds up to burst tokens and refills at rate tokens per second. Sending one segment costs one token. An empty bucket means the message waits, it isn't dropped.
Set rate as a fraction of the pipe, not the whole pipe. If the pipe is 10 segments per second and you have five active tenants, a reasonable default is a rate of 4 per second each and a burst of 20. The rates add up to more than 10 on purpose. Buckets are ceilings, and the dispatch layer enforces the real total.
Charge the bucket in segments, not messages. A 140-character text with one emoji is 3 segments, not 1, so a tenant that sends emoji-heavy copy uses three times the pipe per message. Billing meters segments too, which keeps the two views consistent.
Weighted fair dispatch with deficit round robin
Plain round robin is fair per message. Deficit round robin (DRR) is fair per segment, which is what the pipe actually carries. Each tenant has its own queue and a deficit counter. On each pass, a tenant's deficit grows by quantum * weight, and the tenant sends messages while the head-of-line cost fits in its deficit.
Weights let you give a tenant on a bigger plan a larger share without ever letting it take everything. A weight of 1 is the default.
import time
from collections import deque
from dataclasses import dataclass, field
PIPE_SEGMENTS_PER_SEC = 10 # assumption for this example, use your own measured rate
QUANTUM = 4 # segments added to each tenant's deficit per pass
class TokenBucket:
def __init__(self, rate: float, burst: float):
self.rate, self.burst = rate, burst
self.tokens, self.last = burst, time.monotonic()
def take(self, n: int) -> bool:
now = time.monotonic()
self.tokens = min(self.burst, self.tokens + (now - self.last) * self.rate)
self.last = now
if self.tokens >= n:
self.tokens -= n
return True
return False
@dataclass
class Tenant:
weight: int = 1
bucket: TokenBucket = field(default_factory=lambda: TokenBucket(4, 20))
queue: deque = field(default_factory=deque) # items: (message, segments)
deficit: float = 0.0
def drr_pass(tenants: dict[str, Tenant], send) -> int:
"""One round over all tenants. Returns segments dispatched."""
sent = 0
for t in tenants.values():
if not t.queue:
t.deficit = 0.0 # idle tenants don't bank credit
continue
t.deficit += QUANTUM * t.weight
while t.queue and t.queue[0][1] <= t.deficit:
message, segs = t.queue[0]
if not t.bucket.take(segs):
break # tenant is over its own ceiling, move on
t.queue.popleft()
t.deficit -= segs
send(message)
sent += segs
return sent Run drr_pass in a loop and sleep just enough that total dispatch stays under PIPE_SEGMENTS_PER_SEC. The line that resets an idle tenant's deficit matters. Without it, a tenant that was quiet all morning would come back with a huge stored credit and burst past everyone.
Sending without double sends
Dispatch workers crash and restart, and a message pulled from the queue may or may not have gone out. Give every message a stable key and send it with the Idempotency-Key header, so a retry never texts twice.
import os
import requests
def send(message: dict) -> None:
resp = requests.post(
"https://api.tryready.com/v1/messages",
json={
"to": message["to"],
"body": message["body"],
"consent_attested": True,
},
headers={
"X-API-Key": os.environ["READY_API_KEY"],
"Idempotency-Key": message["id"], # your stable per-message ID
},
timeout=15,
)
resp.raise_for_status() How to build those keys is covered in How to stop duplicate texts from your SMS API with idempotency keys. For a 429 or a timeout, requeue at the front of that tenant's queue and don't charge its bucket twice. The SMS API rate limits post covers the retry behavior.
Choosing weights and watching for drift
Start with equal weights. Add weights only when you have a reason a customer can understand, such as a paid tier. Fairness you can't explain to a customer is a support ticket.
Three numbers tell you whether it's working:
- Queue age per tenant at the 95th percentile. A small tenant's oldest waiting message should stay in seconds even while a big tenant has hours of backlog.
- Share of dispatched segments per tenant over a five-minute window, compared with its weight.
- Bucket empty rate. A tenant whose bucket is empty most of the time is hitting its own ceiling, and that conversation belongs with the customer, not the scheduler.
Time-sensitive traffic deserves its own lane. Appointment reminders and one-time codes shouldn't sit in the same per-tenant queue as a marketing campaign. Give them a second queue per tenant, drain it first, and keep it small. Whether that lane is worth building depends on how often your tenants run both kinds of traffic at once.
What this doesn't fix
Fair sharing divides the pipe, it doesn't enlarge it. If the total demand is above your registered campaign's rate all day, every tenant is slower. The remedy there is more throughput, either a higher use-case tier or splitting heavy tenants onto their own sub-accounts with their own registration. Brand vetting raises the daily cap when it crosses a trust tier, but it doesn't change the per-second rate.
Quiet hours are a separate gate. A campaign text can be refused with a 409 QUIET_HOURS_BLOCK, so treat that as "not sent, retry in the window", never as a failure that burns the bucket's tokens.
The short version
Cap each tenant with a token bucket in segments, divide the pipe with deficit round robin, send with a stable Idempotency-Key, and measure queue age per tenant. State your assumed pipe rate in the code and revisit it when your registration changes. The sub-account model that makes per-tenant limits easy is described in the sub-accounts docs.