AIForum.

Idempotency Keys for AI API Requests: Preventing Duplicate Charges and Outputs

On this page

THE SHORT ANSWER

An idempotency key is a client-generated string that lets an API server recognize a retry as the same logical operation instead of a new one. For AI APIs, where one request can start a paid generation or a multi-minute job, that distinction decides whether a dropped connection costs you one output or two. This guide covers how the mechanism works, how key scoping and retention differ between providers, and how to test that your retries are actually deduplicated.
Quick facts

An idempotency key is a client-generated unique string that lets an API server recognize a retry as the same logical operation rather than a new one. For AI APIs, where a single request can start a paid generation, a long-running job, or both, that distinction decides whether a dropped connection costs you one output or two. This article explains the mechanism as documented by two providers with different semantics, shows how to choose and scope keys, and gives a repeatable test protocol for confirming that your retries are deduplicated.

What an idempotency key does at the API boundary

The key is produced by the client, not the server. As Stripe’s API reference puts it, a client generates an idempotency key, which is a unique key that the server uses to recognize subsequent retries of the same request. Stripe suggests V4 UUIDs or another random string with enough entropy to avoid collisions, allows keys up to 255 characters, and warns against using sensitive data such as email addresses or personal identifiers as keys.

The server’s job is to remember the key and, depending on its implementation, either the outcome of the first attempt or the fact that an attempt is already in progress. When a second request arrives carrying a key the server has already seen, the server does not execute the operation again. What it returns instead varies — and that variation is the part most client code gets wrong.

Three documented duplicate behaviors, not one

It is tempting to treat “idempotency key” as a single well-defined feature. In practice, providers implement at least three distinct responses to a duplicate, and your retry logic has to handle whichever one it meets.

Replay the saved response. Stripe’s API v1 saves the status code and body of the first request made with a given idempotency key after endpoint execution begins. Retries with the same key return that saved response, including saved 500 errors. The idempotency layer also compares the incoming parameters against those of the original request and returns an error if they differ, which prevents a key from being reused for a different operation.

Reject with a conflict. FLORA’s platform documentation describes an Idempotency-Key header that works on every mutating request (POST, PUT, PATCH, DELETE). Keys are remembered for two hours, scoped to the workspace, the endpoint, and its path parameters. A repeat inside that window is rejected with 409 idempotency_duplicate. FLORA does not replay the first response and does not compare bodies: a different body sent with the same key is still treated as a duplicate. If the first request fails with any non-2xx response, the key is released and a retry runs normally.

Return the existing object. FLORA also supports an idempotency_key body field, but only on POST /techniques/{slug}/runs. This variant is scoped to the workspace and remembered for as long as the run exists; a repeat returns the original run with its current status rather than an error. A failed request creates no run, so there is nothing to return.

The practical consequence: a client that assumes “duplicate means I get the same body back” will break against a 409-style API, and a client that assumes “duplicate means 409” will throw away a perfectly valid replayed response.

BehaviorStripe API v1FLORA Idempotency-Key headerFLORA run idempotency_key field
RetentionKeys can be pruned automatically once at least 24 hours old2 hoursAs long as the run exists
Response to a duplicateSaved status code and body of the first request409 idempotency_duplicateThe original run, with its current status
Request comparisonIncoming parameters are compared to the original and an error is returned if they differBodies are not compared; a different body with the same key is still a duplicateScoped to the workspace
If the first request failsResult is not saved when parameter validation fails or a concurrent request conflicts, so the request can be retriedKey is released on any non-2xx responseA failed request creates no run
Concurrent requests, same keyResult is not saved if the request conflicts with another executing concurrentlyOne proceeds; the other gets 409 idempotency_duplicateNot specified in the documentation

Why AI requests are unusually exposed to duplicates

FLORA’s documentation states the problem plainly: when a POST fails mid-flight, you cannot tell whether the API received it, and a blind retry can start the same run twice and bill it twice. That ambiguity is worse for AI workloads than for typical CRUD calls because generation requests are slow and expensive relative to a database write. A client timeout, a proxy idle-connection limit, or a mobile network handoff can all sever the connection after the server has already accepted the work.

Duplicates also compound across layers. A retry inside your SDK, a retry in an HTTP gateway, a redelivered webhook, and a queue worker that restarts mid-job can each independently re-issue the same logical operation. Without a shared key that survives all of those layers, each layer deduplicates only its own traffic.

Choosing and scoping keys

Two decisions matter more than the string format.

Random or derived. A random V4 UUID is the simplest option and matches Stripe’s suggestion of a random string with enough entropy. The alternative is a deterministic key derived from business identity — for example a job record ID plus a step name. Deterministic keys let independent processes arrive at the same key without coordinating, which is useful when a scheduler and a manual retry both touch the same job. The tradeoff is that any change to the derivation changes the key, so the derivation rule must be stable and versioned.

Where the key is created and stored. The key must be generated once per logical operation and persisted before the first attempt. If it lives only in a local variable inside a retry loop, a process restart produces a new key and the protection disappears. Storing it next to the job record — a database row, a queue message attribute, a WordPress post meta field — makes it survive restarts and lets a different worker resume the same operation safely.

Scope and retention then bound what the key can do. FLORA’s header keys are scoped to workspace, endpoint, and path parameters, so the same string used against a different endpoint or a different project is a different request. Stripe compares incoming parameters against the original and errors on a mismatch. On retention, Stripe notes that API v1 keys can be removed automatically once they are at least 24 hours old, and that reusing a key after the original has been pruned generates a new request. FLORA’s two-hour window is much shorter. If your retry policy can outlive the provider’s retention window, the key stops protecting you at that boundary — by design, not by bug.

A retry pattern that holds up

  1. Generate the key once, before the first attempt, and persist it with the job record.
  2. Send the key on every attempt of that job, including manual re-runs triggered by an operator.
  3. Retry on network errors, timeouts, and 5xx responses using the same key, with exponential backoff.
  4. On 409 idempotency_duplicate, stop retrying and look up the existing run or job instead. The operation is already in flight or finished; a further retry will not help.
  5. On a replayed success response, treat it as a normal success and record the returned identifier.
  6. Log the key alongside the provider request ID so that a duplicate investigation has a join point.

The following is an illustrative sketch of that loop, not a tested implementation. It assumes conflict semantics and a separate lookup path for the existing job.

import uuid, time, httpx

def generate(job, client):
    # The key is created once per logical job and persisted with it.
    key = job.get("idempotency_key") or str(uuid.uuid4())
    job["idempotency_key"] = key

    for attempt in range(5):
        r = client.post(
            "https://api.example.com/v1/generate",
            headers={"Idempotency-Key": key},
            json={"prompt": job["prompt"]},
        )
        if r.status_code == 409:
            return lookup_existing(job, client)   # already in flight or done
        if r.status_code < 500:
            return r
        time.sleep(2 ** attempt)
    raise RuntimeError("retries exhausted")

What idempotency keys do not fix

Keys deduplicate one thing: repeated delivery of the same request under the same key. They do not make an endpoint idempotent, and they do not cover non-idempotent side effects that happen after the API boundary. If your handler enqueues a follow-up job, sends a notification, or writes a row before the idempotency record is committed, a retry can still duplicate that work. Stripe’s note that API v1 saves results only after endpoint execution begins is a reminder that failures occurring before execution are not recorded — those requests remain safely retryable, but anything your own code did beforehand is your responsibility.

Keys also do not help if the client generates a fresh key on every attempt. That is the most common implementation bug and it is invisible in testing unless you specifically assert on the key value across attempts. Finally, webhook redelivery and streaming disconnects sit outside the request path: a stream that ends early may still have completed server-side, and a redelivered webhook needs its own deduplication on the consumer side.

Verifying deduplication with a repeatable test

The checks below are a protocol to run against your own integration, not measured results. Record the key, the response status, the response body, and the number of objects or runs created at each step.

  1. Happy-path duplicate. Send the same request twice with the same key and an unchanged body. Under replay semantics you should receive the identical saved status and body; under conflict semantics you should receive 409. Confirm that only one object or run exists.
  2. Changed body, same key. Send the same key with a modified parameter. Stripe documents that this errors; FLORA documents that it is still treated as a duplicate rather than a new request. Either way, no second operation should be created.
  3. Failure then retry. Force a validation error (for example, a missing required field), then resend with the same key and a valid body. Under Stripe API v1 semantics the failed attempt was not saved, so the retry executes; FLORA releases the key on any non-2xx response, so the retry also runs.
  4. Concurrency. Fire two requests with the same key at the same time. FLORA documents that one proceeds and the other gets 409 idempotency_duplicate; Stripe documents that the result is not saved when a request conflicts with another executing concurrently.
  5. Retention boundary. Retry after the provider’s window has elapsed and confirm that a new operation is created. This is expected behavior and worth documenting for your support team.

Local inference and WordPress automation

Self-hosted inference does not remove the duplicate problem; it changes the cost. A duplicated local generation does not produce a per-token bill, but it consumes GPU time, may occupy a queue slot, and can produce a second output that diverges from the first and confuses whatever consumes it. Many self-hosted inference servers do not implement an idempotency header, so the deduplication belongs in the layer in front of them: a gateway or job table with a unique constraint on the key, checked before the request is dispatched.

The same pattern applies to WordPress and automation stacks. Scheduled tasks, form submissions, and webhook handlers can fire more than once — an overlapping cron run, a retried HTTP request from an automation platform, or a user double-submitting a form. Persisting a generated key with the job record before the first outbound call, and enforcing uniqueness in a custom table or a post meta lookup, gives you the same protection the hosted APIs offer. For a fuller treatment of retry timing, see the related reading suggestions below.

What to do next

Add a nullable key column with a unique constraint to the table that tracks your AI jobs, generate the key before the first attempt rather than inside the retry loop, and log it next to the provider’s request ID. Then run the five verification checks above against a non-production workspace. The most valuable one is the failure-then-retry case, because it tells you whether your provider releases keys on errors or holds them — a difference that determines whether a user can recover from a validation mistake without manual intervention.

If you have already shipped an idempotency layer, sharing the exact duplicate behavior you observed — provider, endpoint, status code, and whether a second object was created — is the kind of detail that helps other practitioners pick the right semantics for their own retry code.

FAQ

What happens if I reuse an idempotency key after it expires?

Stripe states that API v1 keys can be pruned automatically after at least 24 hours, and that reusing a key after the original is pruned generates a new request. FLORA remembers Idempotency-Key header values for two hours, so a retry after that window is a new request as well.

Should the idempotency key be a UUID or a hash of the request body?

Both work, but they behave differently. Stripe suggests V4 UUIDs or another random string with enough entropy, and warns against using sensitive data as keys. A hash of the body is deterministic, so independent processes can derive the same key — but any change to any field produces a different key, which can silently create a second operation.

Do idempotency keys guarantee I am not charged twice?

They prevent duplicate operations only when the retry carries the same key. If your client generates a new key per attempt, or if the retry happens after the provider's retention window, the server treats it as a new request. Keys also do not cover side effects your own code performs outside the API boundary.

Does adding an idempotency key make my endpoint idempotent?

No. The key is a deduplication layer at the API boundary. Downstream effects such as enqueuing follow-up jobs, sending notifications, or writing rows still need their own safeguards, because they may execute before or independently of the idempotency record.

References

  1. Stripe — Idempotent requests, API Reference: https://docs.stripe.com/api/idempotent_requests
  2. FLORA — Idempotency, Platform Docs: https://docs.flora.ai/api/platform/idempotency

Related terms

Idempotency
The property that performing an operation more than once has the same effect as performing it once.
Idempotency key
A client-generated unique string sent with a request so the server can identify retries of the same logical operation.
Replay semantics
A duplicate-handling behavior where the server returns the saved status code and body from the first request made with that key, as Stripe API v1 does.
Conflict semantics
A duplicate-handling behavior where the server rejects a repeated key with an error such as 409 idempotency_duplicate instead of replaying the original response, as FLORA's Idempotency-Key header does.
Deduplication window
The period during which a server remembers a key. Retries after this window are treated as new requests.

Leave a Reply

Your email address will not be published. Required fields are marked *