Rate-Limit Logging

Failsafe's retry and circuit-breaker policies, from earlier in the course, protect the valuation service from a provider that is unreliable. A rate limiter protects against a different failure mode entirely: volume, not reliability. Black Book, Carfax, and S&P VIS each enforce their own call quotas, and your own `/vehicles/{vin}/valuation` endpoint needs the same protection in the other direction, so a single caller retrying aggressively cannot starve every other caller of capacity. The classic, well-understood algorithm for this is a token bucket: a bucket holds a fixed number of tokens, refills at a steady rate, and every request consumes one token — when the bucket is empty, the request is throttled instead of served.

Bucket4j is the standard library for this on the JVM, and wiring it up to guard calls to, say, Carfax, is a matter of defining a bandwidth — a capacity and a refill rate — per provider and consulting the bucket before every outbound call. `bucket.tryConsumeAndReturnRemaining(1)` returns a probe object telling you whether the token was granted and, if not, exactly when the next one will be available, which is the information a throttle log line actually needs.

What belongs in that log line is specific: which limiter was hit (which provider, or which caller's API key), the bucket's current state (tokens remaining, time until refill), and enough identity to trace the request — a VIN, a caller id, a trace id — so that whoever is looking at this during an incident can answer "who got throttled, by which limiter, and when will it recover" from that one line alone.

What does not belong is a WARN line for every single rejected request once the system is actually under load. A retry storm hitting a closed bucket can produce hundreds or thousands of rejections per second, and logging each one individually does two things at once: it costs real money in ingestion, and it buries the handful of log lines that would have actually helped you diagnose the spike underneath a wall of identical noise. The fix is not silence — it is sampling or aggregation: log one representative line out of every N rejections, or emit a periodic summary ("throttled 4,812 requests from client X in the last 60 seconds") instead of one line per request.

The metric is what carries the real signal, and it should exist independently of whatever sampling rate the logs use. Incrementing a `valuation.ratelimit.throttled` counter, tagged by limiter and caller or provider, on every single throttle event — using the same Micrometer pattern from the last lesson — means the dashboard sees the true rejection rate in real time even while the logs are deliberately throttling themselves down to a manageable trickle.

Picture the concrete scenario: a nightly batch job re-pricing an entire inventory kicks off, bursts far past Carfax's configured rate, and the limiter starts rejecting. The Datadog counter spikes immediately and a monitor fires. The sampled logs give you one or two representative lines naming the caller and the limiter — enough to identify the batch job as the culprit in under a minute — instead of either fifty thousand identical log lines or, worse, nothing at all because someone decided logging every rejection was too noisy and turned the whole thing off.

This is the same discipline the whole module has been building toward: a guardrail is only half the job. Failsafe protects against an unreliable downstream and pairs with metrics that show when it trips; the rate limiter protects against volume and pairs with metrics and disciplined logging that show when it trips. Neither guardrail is worth much if nobody can see it working.

@Component
class ProviderRateLimiters {

    private val buckets = ConcurrentHashMap<String, Bucket>()

    fun bucketFor(providerName: String): Bucket = buckets.computeIfAbsent(providerName) {
        Bucket.builder()
            .addLimit(Bandwidth.classic(50, Refill.greedy(50, Duration.ofSeconds(60))))
            .build()
    }
}

class RateLimitedValuationProviderClient(
    private val providerName: String,
    private val delegate: ValuationProviderClient,
    private val rateLimiters: ProviderRateLimiters,
    private val metrics: ProviderMetrics,
    private val throttleLogSampler: ThrottleLogSampler,
) : ValuationProviderClient {

    override suspend fun fetchValuation(vin: String): ProviderQuote {
        val bucket = rateLimiters.bucketFor(providerName)
        val probe = bucket.tryConsumeAndReturnRemaining(1)

        if (!probe.isConsumed) {
            metrics.recordThrottle(providerName)
            if (throttleLogSampler.shouldLog(providerName)) {
                logger.warn(
                    "Throttled call to provider={} vin={} tokensRemaining={} nextRefillMs={}",
                    providerName, vin, probe.remainingTokens, probe.nanosToWaitForRefill / 1_000_000
                )
            }
            throw ProviderRateLimitExceededException(providerName)
        }

        return delegate.fetchValuation(vin)
    }
}

class ThrottleLogSampler(private val logEvery: Int = 100) {

    private val counters = ConcurrentHashMap<String, AtomicLong>()

    fun shouldLog(key: String): Boolean {
        val count = counters.computeIfAbsent(key) { AtomicLong(0) }.incrementAndGet()
        return count % logEvery == 1L
    }
}