Implements a token-bucket based rate limiting mechanism with optional
probabilistic dropping inspired by the BLUE queue management algorithm .
The throttle provides the
method which provides
a binary yes/no decision.
The core token bucket algorithm starts with an initial set of tokens based
setting. Tokens are dispensed each
call, which fails if there are not enough tokens to
satisfy a given request.
The token bucket refills over time, providing
milliseconds, capping at
This design allows the throttle to allow short bursts to pass, while still
capping the total number of requests per time interval.
One issue with a pure token bucket approach for something like request or
connection throttling is that the wall clock arrival time of requests affects
the probability of a request being allowed to pass or not. Under constant
load this can lead to request starvation for requests that constantly arrive
later than the majority.
In an attempt to combat this, this throttle can also provide probabilistic
dropping. This is enabled anytime
is set to a value
The probabilistic algorithm starts with an initial drop probability of 0, and
adjusts this probability roughly every
The first request after
, the algorithm checks the
token bucket. If the token bucket is empty, the drop probability is increased
up to a maximum of
. Otherwise, if
the bucket has a token deficit less than
decreasePoint * maxTokens
the probability is decreased by
Given a call to
, requests are first dropped randomly
based on the current drop probability, and only surviving requests are then
checked against the token bucket.
When under constant load, the probabilistic algorithm will adapt to a drop
frequency that should keep requests within the token limit. When load drops,
the drop probability will decrease, eventually returning to zero if possible.
 "BLUE: A New Class of Active Queue Management Algorithms"