Shitty Coders

Transfer Family IdP throttled at reserved 5

What

AWS Transfer Family is the managed SFTP front door onto S3 here. Uploads go to a bucket.

If you use a custom identity provider (IdP) like Keycloak, Transfer Family delegates authentication to an AWS Lambda function. In this case, it passes the credentials, and the Lambda returns a shared IAM role and a bucket prefix as the home directory /.

It’s all simple and beautiful until this function errors, times out, or gets throttled. Transfer Family then helpfully spits out a useless generic message: Received error from identity provider.

Naturally, when a partner’s SFTP clients started failing, everyone immediately blamed Keycloak. But the real culprit was buried in the infrastructure-as-code:

ReservedConcurrentExecutions: 5

That is not a guaranteed capacity reservation. It is a hard, merciless cap. It takes 5 from your account’s concurrency pool (1000 per region by default), and absolutely no more than five instances can run simultaneously. Six logins hit at once? Five process, and the sixth gets a 429 “Too Many Requests” which Transfer Family disguises as an IdP error.

To be fair, five isn’t inherently a small number. Plenty of high-traffic Lambdas get away with a cap of five because they execute in milliseconds. But this function is a slow-motion disaster. Keycloak actually does its job: it intentionally delays bad password responses to punish brute-force attacks. Then, just to really twist the knife, the Lambda sometimes adds an additional five-second delay of its own. Thanks to this compounding time-out trap, a random internet bot guessing admin/root or a disabled account login can hold a single execution open for an eternity.

Then a partner hammered the system with simultaneous SFTP sessions for a bulk upload. When that burst collided with seats already held hostage by agonizingly slow authentication attempts, the limit exhausted instantly. The best part? The Lambda had actually been throttling occasionally in the past and was just never noticed. Automated clients politely retried, conveniently masking the underlying bottleneck until this massive burst finally broke the illusion.

Why it hurts

Transfer Family failing closed on an IdP error is technically correct, but this configuration creates a highly efficient, self-inflicted Denial of Service (DoS) machine.

A tiny concurrency limit combined with an IdP penalty delay means background internet noise completely fills your five chairs. Real users get locked out. Because the logs point the finger at the identity provider, you waste hours debugging a perfectly healthy Keycloak while AWS Lambda is quietly dropping 429s. Automated clients will retry, but eventually give up. Files simply vanish into the void.

What to do

  • Do the real math: Lambda concurrency equals your requests per second multiplied by the duration of the request. Stop planning for the happy path where a successful login is fast. Calculate your limit based on the worst-case scenario: peak login attempts multiplied by that agonizing 5-10 second penalty delay. If you don’t account for how long bad actors stall your system, you will run out of chairs. Give it room to breathe, but leave enough in your account pool for other functions.
  • Monitor the real bottleneck: Stop staring exclusively at IdP logs and hoping client retries will save you. Set CloudWatch alarms for Lambda Throttles alongside Transfer Family IdP errors.
  • Fail fast on garbage: If the username is an obvious bot target (root, admin, test) or a known suspended account, reject it instantly inside the Lambda. Don’t pay AWS to sit around and wait for compounding penalty delays.
  • Rate-limit at the edge: If you have an AWS WAF in front, rate-limit repeated failures by IP address. Do not try to be clever by writing code inside your Auth Lambda to dynamically update the WAF. If all five of your concurrency seats are full of bots, your function gets throttled before it can even run the code to ban them. Don’t make the drowning doorman responsible for calling the bouncer. Either let WAF handle the rate limits automatically, or use a completely separate Lambda triggered by CloudWatch logs to ban bad IPs.