Performance Tuning

PII Eraser is designed for high throughput, but client-side implementation details significantly impact total processing speed. This page covers how to optimize your application's interaction with the PII Eraser API.

For hardware selection, CPU requirements, and throughput benchmarks, see Benchmarks & Hardware Selection.

Concurrency is Key

The benchmarks demonstrate that concurrent processing delivers a significant boost to processing throughput at the expense of request latency. Batch processing applications in particular should always use concurrency.

If you have multiple PII Eraser instances behind a load balancer, maintain 4 × [Number of instances] concurrent requests to fully saturate the hardware. Even 2 concurrent requests per instance delivers most of the throughput gain.

Text Endpoints: Batching vs. Single Requests

The text API endpoints accept lists of strings like["text1", "text2", ...]. Batching multiple texts into a single request reduces network overhead from TCP/TLS handshakes and HTTP framing, but does not noticeably impact processing throughput on the server side. It also increases the risk of exceeding the maximum token limit, which works per API request, not per string.

When to batch: Low-latency networks or when you have many short texts. Sending 10–50 texts per request is a reasonable default.

When to use single requests: Any latency-sensitive application such as detecting PCI in LLM chat interfaces, or when individual texts are very large.

Keep-Alive Connections

Ensure your HTTP client uses persistent connections (Keep-Alive). Re-establishing TCP/TLS handshakes for every API call can add overall stack latency.