[RADIATOR] add Attributes when retrying to a new Host in AuthROUNDROBIN (radiator Digest, Vol 63, Issue 14)

[David Zych]

On 08/25/2014 02:51 PM, Heikki Vatiainen wrote:
> There is now ReplyTimeoutHook in Radiator 4.13 patches. The arguments it
> is passed are the same as the arguments of NoReplyHook except that there
> are only two arguments. If you need to access the reply, 3rd argument
> with NoReplyHook, it's available via $p->{rp} as usually.
> 
> This hook is called when there is no reply from the current Host. It is
> not called for every retransmission, only when the current Host is
> deemed to have failed for the request. After the hook is called, the
> next Host is tried, if there is one.
> 
> Please let me know how it works.
> 
> Thanks,
> Heikki
> 

Hi Heikki,

It's taken me longer than I had hoped to circle back around to this, but
I wanted to say thanks very much for the new patches! I am using them
now to cope much more gracefully if one of my back-end "worker"
processes gets stalled by an external dependency (i.e. ntlm_auth).

Here are the key pieces, for the benefit of anyone else trying to
accomplish something similar.

Thanks,
David


In main instance...

# Helper AuthBy to proxy stateless inner authentication requests to
# worker instances using simple round robin.
<AuthBy ROUNDROBIN>
  Identifier workerproxy
  include %D/private/localhost.secret
  # give up if worker doesn't respond within this time
  RetryTimeout 3
  # do not retry against the same worker
  Retries 0
  # transparently work around problems affecting only a single worker,
  # but limit our attempts per request (by default, ROUNDROBIN tries
  # every Host once unless already marked down by FailureBackoffTime)
  # for two reasons: 1) eventually it will be too late for a reply to
  # be useful to the client, and 2) there might be something special
  # about this particular request that's triggering the problem.
  MaxTargetHosts 2
  FailureBackoffTime 1
  # enable workers to identify and proactively IGNORE any proxied
  # request old enough that this main instance has already given up on
  # it (note: same system, so no worries about clock skew).  This
  # ensures that we won't exacerbate an external bottleneck scenario
  # by wasting our limited resources processing stale requests.
  StripFromRequest X-Proxy-Timestamp,X-Proxy-Timeout
  AddToRequest X-Proxy-Timestamp=%t,X-Proxy-Timeout=3
  ReplyTimeoutHook sub { CITES::proxy_updateTimestamp(@_) }

  <Host 127.0.0.1>
    AuthPort %{GlobalVar:radius.authworker1.port}
  </Host>
  <Host 127.0.0.1>
    AuthPort %{GlobalVar:radius.authworker2.port}
  </Host>
  <Host 127.0.0.1>
    AuthPort %{GlobalVar:radius.authworker3.port}
  </Host>
  <Host 127.0.0.1>
    AuthPort %{GlobalVar:radius.authworker4.port}
  </Host>

  # no accounting requests should end up here
  IgnoreAccounting
</AuthBy>


# AuthRADIUS ReplyTimeoutHook: update X-Proxy-Timestamp before forwarding to
# next Host
sub proxy_updateTimestamp
{
  my $attrname = "X-Proxy-Timestamp";
  my ($p,$fp) = @_;

  $$fp->change_attr($attrname, time);
  &main::log($main::LOG_DEBUG, "Set $attrname = ".$$fp->get_attr($attrname), $$p);
}


In worker instance...

<Client 127.0.0.1>
  Identifier localhost
  include %D/private/localhost.secret
  # always handle "duplicate" requests from self
  DupInterval 0
  # identify (and log) proxied requests too old to be worth answering
  PreHandlerHook sub { CITES::ignore_expired(@_) }
</Client>

# ignore requests flagged as expired by the above hook
<Handler Client-Identifier=localhost, IGNORE_EXPIRED=1>
  Identifier localhost-ignoreExpired
  <AuthBy INTERNAL>
    DefaultResult IGNORE
  </AuthBy>
</Handler>


# PreHandlerHook for localhost Client: if the request was proxied too long
# ago to be worth answering now, log an error and set IGNORE_EXPIRED to
# trigger the Handler.
sub ignore_expired {
  my ($tsattr, $windowattr) = qw(X-Proxy-Timestamp X-Proxy-Timeout);
  my $p = shift;

  my ($ts, $window) = map { $$p->get_attr($_) || return } ($tsattr, $windowattr);

  my $age = time - $ts;
  if ($age < 0 or $age > $window) {
    $$p->add_attr('IGNORE_EXPIRED',1);
    &main::log($main::LOG_ERR, "Ignoring expired request for ".$$p->getUserName().": $tsattr is ${age}s old", $$p);
  }
}