(RADIATOR) Redback is sending too many Access-Requests

Wed Feb 1 03:49:12 CST 2006

Thanks so much for the detailed help and description of your very similar
(almost identical) problem I'm having.

I will try to see if we can configure "delay" on PPP connections. Maybe it
will help. not sure if Redback has a similar setting like your Cisco. I'll
also consider blocking the bad users to force them to call support and we
can identify their equipment or something.

However, the reason I'm trying to make this hook work is to make the radius
"safe" from individual problems. A "few" bad apples should not cause the
whole system to stop working.

But thanks again, you've been a great help.

On 2/1/06, Martin Wallner <Martin.Wallner at eunet.co.at> wrote:
>
> Mishar, Hugh,
>
> We had similar Problems with CISCO Equipment here (and it was not CISCO
> which caused the problem). It took a while to find out, but normally the
> accounts that 'hammer' down the pipe are trying to get in with a wrong
> password and/or wrong username. Customer side equipment not deactivated is
> most of the troublebringers.
>
> Most of the xDSL-Gear on client side has what we start to call here 'crazy
> hammers' (older ZyXEL P310, P324 do this, and ZyXEL did take precautions for
> the newer models against this 'hammering'). If they are declined, they
> relentlessly try again, without pause between the attempts, with this
> attemps efffectively draining the resources on the NAS, every one of them
> doing - in a way - his own DOS attack. If you have said you have 50+ users,
> you have a nice DDOS-Attack running, by your own people (or ex-customers).
>
> A (partial) remedy for this is to make the delay time on the PPP in the
> NAS longer, so effectively not ALLOWING another start of a new PPP-Session.
> We have it on 15 seconds, and it seems to work fine (we have about the same
> amount of DSL-Customers on the box like you have). This also helps if the
> box has to be rebooted or DSLAM's are taken down and up again, and the whole
> population wants to get back in...
>
> Another way is to spread the load, so you have your accounting on the
> OracleDB and the authentication on another DB, provided by the main DB...
>
> Another way is also to change the type of Authentication... LDAP comes to
> mind, 2 LDAP server are MUCH MORE responsive than any DB with authenticating
> (you are pulling too much overhead with DB-Authenticating. It's ok to do
> Authentication by SQL, but the responsiveness has a sharp drop if you are
> over a distinct amounts of transactions per second...)
>
> The ultimate Remedy (but your Support won't love you for this) is, to find
> out WHO and WHY these 50+ 'bad' clients are there hammering the NAS... In
> my case it was the ultimate 'cleanup', Support didn't talk with Systems for
> 2 weeks :-), but they called all and everyone who's equipment was hammering
> in ... contact all ex-users and bring them to deactivate the still running
> equipment. Bring the Telco to take down the DSL and so on...  poof.... no
> more hammering...
>
> Martin Wallner
>
>  ------------------------------
> *From:* owner-radiator at open.com.au [mailto:owner-radiator at open.com.au] *On
> Behalf Of *Mishari Al-Faris
> *Sent:* Mittwoch, 01. Februar 2006 08:20
> *To:* Hugh Irvine
> *Cc:* radiator at open.com.au
> *Subject:* Re: (RADIATOR) Redback is sending too many Access-Requests
>
> Hello Hugh,
>
> I understand that it will "accept" and spread them randomly over 1-20
> mins. That is fine with me, as long as I treat the offending users
> differently than the normal ones. My problem is that my oracle DB behind
> radiator can not handle all the requests, the good and the bad ones. The bad
> requests are relentless and can constitute over +95% of the incoming
> requests. I have about 7500+ DSL customers and they stop complaining once I
> manually do "AuthBy AllowAll" temporarily, however, I have around 200+
> customers who require static IPs obtained from the DB, and once I turn off
> authentication those static-IP customers start getting dynamic IPs and start
> complaining, so I can't stop authentication for long.
>
> The amount of usernames that produce the bad requests is very small
> compared to the total amount of users, say around 50+. And those are the
> ones I want to exclude, also, I'm not sure what's common between them that
> causes them to send so many requests. So far what I see from these users is
> that they send 1 Access-Request every 1-5 seconds, consistently and without
> stopping, regardless of what response I send them, Accept or Reject doesn't
> matter, they keep on sending. And even when I use AllowAll, which is the
> fastest response I can achieve, they still send the requests, so I'm ruling
> out that the radius is slow to respond to them.
>
> Also, the requests that come from the same user each has a different
> "Acct-Session-Id". And the Redback is configured for 60 seconds timeout and
> only 1 retry. So this makes me rule out that the Redback is retrying for the
> same user. Instead what I think is happening is that the user connection to
> the Redback is unstable and disconnecting and connecting again quickly,
> which causes the Redback to create a new request for him.
>
> So, I'm willing to try this solution even if the per-user counters take up
> alot of memory, if it doesn't work I'll stop using it, but in the meantime
> I'm in trouble and am willing to try anything :(
>
> Ofcourse it goes without saying that I'm not holding any guarantees
> against anyone either. If it works it works.
>
> So what I want to do is to exclude the few users that send that many
> requests and either Accept them, or Ignore them, or Reject them, doesnt
> really matter, as long as I don't go to the AuthBy PLSQL clause for these
> guys. hence protecting my DB from the onslaught.
>
> On 2/1/06, Hugh Irvine <hugh at open.com.au> wrote:
> >
> >
> > Hello Mishari -
> >
> > You should note first of all that this hook code will not "ignore"
> > the access requests - the hook is designed to "accept" all access
> > requests over a certain number with a variable session timeout that
> > will cause the resulting temporary session to drop after some random
> > time. The idea being to spread the requests over a longer period of
> > time. Also note that this is an idea only - I make no guarantees
> > about the success or otherwise of using this code.
> >
> > I am also not sure about maintaining per-user counters, as this will
> > lead to greatly increased memory usage.
> >
> > Can you tell me exactly what you are wanting to do?
> >
> > regards
> >
> > Hugh
> >
> >
> >
> > On 31 Jan 2006, at 22:53, Mishari Al-Faris wrote:
> >
> > > Dear Hugh,
> > >
> > > This is an example that you suggested a while back to mitigate
> > > excessive requests coming from DSL NASes.
> > > I've been trying to modify it to our needs but have been getting
> > > compilation errors. Let me just explain what I wish to do instead
> > > of going through what I did wrong.
> > >
> > > I'd like to count the access trials per "user" not per "NAS". If a
> > > certain username is seen trying more than say 1 request per 5
> > > seconds, I want to ignore the request, and not go through my
> > > AuthPLSQL AuthBy clause. Is this possible? thanks.
> > >
> > > # RequestHook for AuthBy INTERNAL
> > > # This hook counts the number of access requests that are received
> > > for a
> > > # particular NAS, and returns an ACCEPT if there are more than 100
> > > per second.
> > > # A Session-Timeout reply attribute is added to the reply with a
> > > random
> > > # value between 1 and 1200 seconds(20 minutes).
> > > #
> > > # Note: these values should be altered as required.
> > > #
> > > # Hugh Irvine, Open System Consultants, 20050829
> > >
> > > sub
> > > {
> > > my $p = $_[0];
> > > my $time = time;
> > > my $code = $p->code;
> > > my $nas = $p->{Client};
> > > if ($time == $nas->{last_throttle_time} && $code eq 'Access- Request')
> >
> > > {
> > > if (++$nas->{throttle_count} > 100)
> > > {
> > > $p->{rp}->add_attr('Session-Timeout', int (rand(1200) + 1));
> > > return ($main::ACCEPT, 'Conditional flood control');
> > > }
> > > }
> > > else
> > > {
> > > $nas->{throttle_count} = 0;
> > > }
> > > $nas->{last_throttle_time} = $time;
> > > return ($main::IGNORE, 'Continue to proxy');
> > > }
> > >
> > >
> > >
> > > Here is an example of how to use the hook.
> > >
> > >
> > > <Handler .....>
> > >
> > > AuthByPolicy ContinueWhileIgnore
> > >
> > > <AuthBy INTERNAL>
> > > RequestHook file:"throttle.pl"
> > > AddToReply .....
> > > </AuthBy>
> > > # normal AuthBy
> > > <AuthBy .....>
> > > .....
> > > </AuthBy>
> > > </Handler>
> >
> >
> > NB:
> >
> > Have you read the reference manual ("doc/ref.html")?
> > Have you searched the mailing list archive ( www.open.com.au/archives/
> > radiator)?
> > Have you had a quick look on Google (www.google.com)?
> > Have you included a copy of your configuration file (no secrets),
> > together with a trace 4 debug showing what is happening?
> >
> > --
> > Radiator: the most portable, flexible and configurable RADIUS server
> > anywhere. Available on *NIX, *BSD, Windows, MacOS X.
> > -
> > Nets: internetwork inventory and management - graphical, extensible,
> > flexible with hardware, software, platform and database independence.
> > -
> > CATool: Private Certificate Authority for Unix and Unix-like systems.
> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.open.com.au/pipermail/radiator/attachments/20060201/e3c73f37/attachment.html>