Rate limits and quotas
The limits on this page apply to the PHI Identification endpoint (POST /entityextraction/pipelines/imo-phi-identification), which runs an LLM-based Named Entity Recognition model on GPU and is the throughput-constrained step of the De-Identification flow. The PHI Obfuscation endpoint (POST /transformation/v1/pipelines/imo-phi-obfuscation) is rules-based, runs on CPU, and operates with separate, substantially higher limits — it is not typically a bottleneck in a chained Identification → Obfuscation workflow.
Limits are tuned per customer and can be raised based on observed usage — contact our support team if you need higher throughput. For general guidance on handling 429 responses, see Best Practices → Handling Limits.
Throughput (PHI Identification)
| Limit | Value | Notes |
|---|---|---|
| Sustained request rate | 100 requests per minute | Per customer; guaranteed under nominal fleet capacity |
| Short-burst rate | 200 requests per minute for up to 5 minutes | Service autoscales to absorb spikes |
| Maximum concurrent in-flight requests | 128 | Beyond this, requests queue and may receive a 429 |
Per-request limits (PHI Identification)
| Limit | Value | Notes |
|---|---|---|
Maximum input length (text field) |
~1,024 model tokens (≈ 700–1,000 characters of clinical text) | Larger documents must be split client-side |
| Per-request output cap | 300 tokens | Server-enforced; not customer-configurable |
| Per-request timeout | 15 seconds | Requests exceeding this return 504 Gateway Timeout |
Latency expectations (PHI Identification)
Measured against the production configuration during the April 2026 load-testing campaign on NVIDIA A10G hardware:
| Customer load | p95 response time |
|---|---|
| Up to 100 RPM | < 2.5 seconds |
| 100 – 200 RPM (within autoscaling envelope) | < 4 seconds |
| Brief bursts up to 300 RPM (≤ 5 minutes) | < 5 seconds |
Traffic sustained above 200 RPM for longer than 5 minutes should be coordinated with our support team in advance so we can pre-scale capacity.