(RADIATOR) Redback is sending too many Access-Requests
Hugh Irvine
hugh at open.com.au
Wed Feb 1 15:56:33 CST 2006
Hello Toomas -
Many thanks for the description of your solution.
This is exactly what I had suggested originally with the hook code
that is now in "goodies/hooks.txt".
regards
Hugh
On 1 Feb 2006, at 22:06, Toomas Kärner wrote:
> Hello Mishari,
>
> I'm currently rebuilding our DSL authentication aswell and I have
> solved this problem in a totally different way. Instead of denying
> access to these hammering clients, I always let them in but apply a
> set of parameters that either give it private ip (to save ip space)
> and set it into special context in RB (not to have private and public
> ip's in same context) and do "deny ip any any" OR apply a set of
> parameters that redirect any http traffic to your "portal" that gives
> them proper explanation why they have ended up there (invoice not
> paid etc.) and deny any other type of traffic.
>
> This will reduce load on your auth systems and users will also
> understand why their connection does not "work". It gives you a new
> medium to reach your customers that have a problem (get deny from your
> old auth system) and they have to call your support to find out reason
> for deny. Now, when most customers run their PPPoE client in CPE the
> Reply-Message is useless as a tool for telling customers that "bad
> password" or "pay your bill" because they will not see it. Very rear
> PPPoE client softwares for PC showed that also. But if a
> web-page pops up saying "Pay your bill" regardless "where they want to
> click that day" then it's way more effective.
>
> Rgds.
> Toomas
>
> Wednesday, February 1, 2006, 11:49:12 AM, you wrote:
>
>> Thanks so much for the detailed help and description of your very
>> similar (almost identical) problem I'm having.
>>
>> I will try to see if we can configure "delay" on PPP connections.
>> Maybe it will help. not sure if Redback has a similar setting like
>> your Cisco. I'll also consider blocking the bad users to force them
>> to call support and we can identify their equipment or something.
>>
>> However, the reason I'm trying to make this hook work is to make
>> the radius "safe" from individual problems. A "few" bad apples
>> should not cause the whole system to stop working.
>>
>> But thanks again, you've been a great help.
>
>> On 2/1/06, Martin Wallner <Martin.Wallner at eunet.co.at> wrote:
>>
>> Mishar, Hugh,
>>
>>
>>
>> We had similar Problems with CISCO Equipment here (and it was not
>> CISCO which caused the problem). It took a while to find out, but
>> normally the accounts that 'hammer' down the pipe are trying to get
>> in with a wrong password and/or wrong username. Customer side
>> equipment not deactivated is most of the troublebringers.
>>
>>
>>
>> Most of the xDSL-Gear on client side has what we start to call
>> here 'crazy hammers' (older ZyXEL P310, P324 do this, and ZyXEL did
>> take precautions for the newer models against this 'hammering'). If
>> they are declined, they relentlessly try again, without pause
>> between the attempts, with this attemps efffectively draining the
>> resources on the NAS, every one of them doing - in a way - his own
>> DOS attack. If you have said you have 50+ users, you have a nice
>> DDOS-Attack running, by your own people (or ex-customers).
>>
>>
>>
>> A (partial) remedy for this is to make the delay time on the PPP
>> in the NAS longer, so effectively not ALLOWING another start of a
>> new PPP-Session. We have it on 15 seconds, and it seems to work
>> fine (we have about the same amount of DSL-Customers on the box
>> like you have). This also helps if the box has to be rebooted or
>> DSLAM's are taken down and up again, and the whole population
>> wants to get back in...
>>
>>
>>
>> Another way is to spread the load, so you have your accounting
>> on the OracleDB and the authentication on another DB, provided by
>> the main DB...
>>
>>
>>
>> Another way is also to change the type of Authentication... LDAP
>> comes to mind, 2 LDAP server are MUCH MORE responsive than any DB
>> with authenticating (you are pulling too much overhead with
>> DB-Authenticating. It's ok to do Authentication by SQL,
>> but the responsiveness has a sharp drop if you are over a distinct
>> amounts of transactions per second...)
>>
>>
>>
>> The ultimate Remedy (but your Support won't love you for this) is,
>> to find out WHO and WHY these 50+ 'bad' clients are there hammering
>> the NAS... In my case it was the ultimate 'cleanup', Support didn't
>> talk with Systems for 2 weeks :-), but they called all and everyone
>> who's equipment was hammering in ... contact all ex-users and bring
>> them to deactivate the still running equipment. Bring the Telco to
>> take down the DSL and so on... poof.... no more hammering...
>>
>>
>>
>> Martin Wallner
>
>>
>>
>>
>
>> From: owner-radiator at open.com.au
>> [mailto:owner-radiator at open.com.au] On Behalf Of Mishari Al-Faris
>> Sent: Mittwoch, 01. Februar 2006 08:20
>> To: Hugh Irvine
>> Cc: radiator at open.com.au
>> Subject: Re: (RADIATOR) Redback is sending too many Access-
>> Requests
>
>>
>> Hello Hugh,
>
>> I understand that it will "accept" and spread them randomly over
>> 1-20 mins. That is fine with me, as long as I treat the offending
>> users differently than the normal ones. My problem is that my oracle
>> DB behind radiator can not handle all the requests, the good and the
>> bad ones. The bad requests are relentless and can constitute over
>> +95% of the incoming requests. I have about 7500+ DSL customers
>> and they stop complaining once I manually do "AuthBy AllowAll"
>> temporarily, however, I have around 200+ customers who require
>> static IPs obtained from the DB, and once I turn off
>> authentication those static-IP customers start getting dynamic IPs
>> and start complaining, so I can't stop authentication for long.
>
>> The amount of usernames that produce the bad requests is very
>> small compared to the total amount of users, say around 50+. And
>> those are the ones I want to exclude, also, I'm not sure what's
>> common between them that causes them to send so many requests. So
>> far what I see from these users is that they send 1
>> Access-Request every 1-5 seconds, consistently and without stopping,
>> regardless of what response I send them, Accept or Reject doesn't
>> matter, they keep on sending. And even when I use AllowAll, which
>> is the fastest response I can achieve, they still send the
>> requests, so I'm ruling out that the radius is slow to respond
>> to them.
>
>> Also, the requests that come from the same user each has a
>> different "Acct-Session-Id". And the Redback is configured for 60
>> seconds timeout and only 1 retry. So this makes me rule out that the
>> Redback is retrying for the same user. Instead what I think is
>> happening is that the user connection to the Redback is unstable
>> and disconnecting and connecting again quickly, which causes the
>> Redback to create a new request for him.
>
>> So, I'm willing to try this solution even if the per-user counters
>> take up alot of memory, if it doesn't work I'll stop using it, but
>> in the meantime I'm in trouble and am willing to try anything :(
>
>> Ofcourse it goes without saying that I'm not holding any
>> guarantees against anyone either. If it works it works.
>
>> So what I want to do is to exclude the few users that send that
>> many requests and either Accept them, or Ignore them, or Reject
>> them, doesnt really matter, as long as I don't go to the AuthBy
>> PLSQL clause for these guys. hence protecting my DB from the
>> onslaught.
>
>>
>> On 2/1/06, Hugh Irvine <hugh at open.com.au> wrote:
>
>> Hello Mishari -
>
>> You should note first of all that this hook code will not
>> "ignore"
>> the access requests - the hook is designed to "accept" all
>> access
>> requests over a certain number with a variable session
>> timeout that
>> will cause the resulting temporary session to drop after some random
>> time. The idea being to spread the requests over a longer
>> period of
>> time. Also note that this is an idea only - I make no guarantees
>> about the success or otherwise of using this code.
>
>> I am also not sure about maintaining per-user counters, as
>> this will
>> lead to greatly increased memory usage.
>
>> Can you tell me exactly what you are wanting to do?
>
>> regards
>
>> Hugh
>
>
>
>> On 31 Jan 2006, at 22:53, Mishari Al-Faris wrote:
>
>>> Dear Hugh,
>>>
>>> This is an example that you suggested a while back to mitigate
>>> excessive requests coming from DSL NASes.
>>> I've been trying to modify it to our needs but have been
>>> getting
>>> compilation errors. Let me just explain what I wish to do
>>> instead
>>> of going through what I did wrong.
>>>
>>> I'd like to count the access trials per "user" not per
>>> "NAS". If a
>>> certain username is seen trying more than say 1 request per 5
>>> seconds, I want to ignore the request, and not go through my
>>> AuthPLSQL AuthBy clause. Is this possible? thanks.
>>>
>>> # RequestHook for AuthBy INTERNAL
>>> # This hook counts the number of access requests that are
>>> received
>>> for a
>>> # particular NAS, and returns an ACCEPT if there are more than 100
>>> per second.
>>> # A Session-Timeout reply attribute is added to the reply
>>> with a
>>> random
>>> # value between 1 and 1200 seconds(20 minutes).
>>> #
>>> # Note: these values should be altered as required.
>>> #
>>> # Hugh Irvine, Open System Consultants, 20050829
>>>
>>> sub
>>> {
>>> my $p = $_[0];
>>> my $time = time;
>>> my $code = $p->code;
>>> my $nas = $p->{Client};
>>> if ($time == $nas->{last_throttle_time} && $code eq 'Access-
>>> Request')
>>> {
>>> if (++$nas->{throttle_count} > 100)
>>> {
>>> $p->{rp}->add_attr('Session-Timeout', int (rand(1200) + 1));
>>> return ($main::ACCEPT, 'Conditional flood control');
>>> }
>>> }
>>> else
>>> {
>>> $nas->{throttle_count} = 0;
>>> }
>>> $nas->{last_throttle_time} = $time;
>>> return ($main::IGNORE, 'Continue to proxy');
>>> }
>>>
>>>
>>>
>>> Here is an example of how to use the hook.
>>>
>>>
>>> <Handler .....>
>>>
>>> AuthByPolicy ContinueWhileIgnore
>>>
>>> <AuthBy INTERNAL>
>>> RequestHook file:"throttle.pl"
>>> AddToReply .....
>>> </AuthBy>
>>> # normal AuthBy
>>> <AuthBy .....>
>>> .....
>>> </AuthBy>
>>> </Handler>
>
>
>> NB:
>
>> Have you read the reference manual ("doc/ref.html")?
>> Have you searched the mailing list archive ( www.open.com.au/
>> archives/
>> radiator)?
>> Have you had a quick look on Google (www.google.com)?
>> Have you included a copy of your configuration file (no
>> secrets),
>> together with a trace 4 debug showing what is happening?
>
>> --
>> Radiator: the most portable, flexible and configurable RADIUS
>> server
>> anywhere. Available on *NIX, *BSD, Windows, MacOS X.
>> -
>> Nets: internetwork inventory and management - graphical,
>> extensible,
>> flexible with hardware, software, platform and database
>> independence.
>> -
>> CATool: Private Certificate Authority for Unix and Unix-like
>> systems.
>
>
>
>
>
>>
>
>>
>
>
> --
> Archive at http://www.open.com.au/archives/radiator/
> Announcements on radiator-announce at open.com.au
> To unsubscribe, email 'majordomo at open.com.au' with
> 'unsubscribe radiator' in the body of the message.
NB:
Have you read the reference manual ("doc/ref.html")?
Have you searched the mailing list archive (www.open.com.au/archives/
radiator)?
Have you had a quick look on Google (www.google.com)?
Have you included a copy of your configuration file (no secrets),
together with a trace 4 debug showing what is happening?
--
Radiator: the most portable, flexible and configurable RADIUS server
anywhere. Available on *NIX, *BSD, Windows, MacOS X.
-
Nets: internetwork inventory and management - graphical, extensible,
flexible with hardware, software, platform and database independence.
-
CATool: Private Certificate Authority for Unix and Unix-like systems.
--
Archive at http://www.open.com.au/archives/radiator/
Announcements on radiator-announce at open.com.au
To unsubscribe, email 'majordomo at open.com.au' with
'unsubscribe radiator' in the body of the message.
More information about the radiator
mailing list