[RADIATOR] Authorization delay problem SQL

Fri Nov 23 16:16:31 CST 2012

Hello Ricardo -

Problems with the inbound queue are almost always due to the slow response from some external resource that Radiator is using - usually an LDAP or SQL database.

To see exactly what is happening you need to be running trace 4 debug with LogMicroseconds (requires Time:Hires from CPAN).

LogMicroseconds adds the microsecond offset in the current second on every DEBUG line so you can see exactly how long each processing step is taking (the six digit number after the timestamp).

If for example you have an SQL query or an LDAP lookup taking 0.1 second (one tenth of a second), then it follows that only a maximum of 10 requests per second can be processed and any more than 10 requests per second will result in the UDP queue filling up and eventually overflowing. 

For high-volume applications you can use either a hardware or software loadbalancer in front of multiple Radiator hosts, each Radiator host set up with "frontend" and "backend" processes as previously described.

We (OSC) are available on a contract basis for design and re-engineering consulting if desired.

regards

Hugh

On 24 Nov 2012, at 05:26, Ricardo Martinez <rmartinez at redvoiss.net> wrote:

> Hello Hugh.
> Thanks for the tips.  Right now we have move from One instance of Radiator
> two 4 instances attending the same number of NAS, so far so good.  I have
> one more question regarding to this issue.  If you have a single RADIUS
> used as a frontend, Aren't you going to have the same problem with one udp
> socket receiving all the requests?
> 
> Ricardo.-
> 
> -----Mensaje original-----
> De: Hugh Irvine [mailto:hugh at open.com.au]
> Enviado el: jueves, 22 de noviembre de 2012 19:33
> Para: Ricardo Martinez
> CC: Heikki Vatiainen; radiator at open.com.au
> Asunto: Re: [RADIATOR] Authorization delay problem SQL
> 
> 
> Hello Ricardo -
> 
> In my consulting practice I almost always find that splitting the
> configuration up into multiple separate Radiator instances is the
> preferred approach.
> 
> I generally end up with a single RADIUS "frontend" authentication process
> (and a single TACACS "frontend" process if required), that classifies the
> requests according to whatever makes sense in the particular environment
> and then proxies the requests to individual "backend" processes for each
> class of request. I also use a separate process for RADIUS accounting.
> 
> This approach is *much* easier to understand and manage, and it is very
> easy to add new classes of processing, simply by adding a Handler to the
> "frontend" instance and the corresponding "backend" instance.
> 
> Common elements of the configuration can be kept in single files that are
> included in each instance that requires them (using the "include"
> directive).
> 
> If you have any questions feel free to contact me.
> 
> regards
> 
> Hugh
> 
> 
> 
> On 23 Nov 2012, at 03:33, Ricardo Martinez <rmartinez at redvoiss.net> wrote:
> 
>> Hello Heikki.
>> Just now we have another delay issue, and we have checked that the delay
>> is in the queue.   The numbers of  Recv-Q queue are as follows :
>> 
>> Time     --  Recv-Q
>> 12:31:30 -- 47928
>> 12:31:32 -- 48752
>> 12:31:34 -- 47608
>> 12:31:36 -- 64528
>> 12:31:38 -- 61984
>> 12:31:40 -- 55360
>> 12:31:42 -- 51616
>> 12:31:44 -- 59216
>> 12:31:46 -- 52608
>> 12:31:48 -- 50912
>> 12:31:50 -- 75640
>> 12:31:52 -- 49016
>> 12:31:54 -- 50848
>> 12:31:56 -- 50712
>> 12:31:58 -- 58168
>> 12:32:01 -- 59016
>> 12:32:03 -- 53024
>> 12:32:05 -- 65984
>> 12:32:07 -- 83320
>> 12:32:09 -- 66056
>> 12:32:11 -- 60072
>> 12:32:13 -- 71968
>> 12:32:15 -- 94736
>> 12:32:17 -- 67752
>> 12:32:19 -- 66480
>> 12:32:21 -- 77072
>> 12:32:23 -- 72320
>> 12:32:25 -- 65984
>> 12:32:27 -- 58728
>> 12:32:29 -- 59936
>> 12:32:31 -- 49008
>> 12:32:33 -- 45776
>> 12:32:36 -- 60136
>> 12:32:38 -- 43456
>> 12:32:40 -- 53168
>> 12:32:42 -- 54936
>> 12:32:44 -- 67200
>> 12:32:46 -- 68048
>> 12:32:48 -- 59872
>> 12:32:50 -- 61336
>> 12:32:52 -- 67888
>> 12:32:54 -- 51688
>> 12:32:56 -- 54448
>> 12:32:58 -- 71976
>> 12:33:00 -- 53840
>> 12:33:02 -- 52744
>> 12:33:04 -- 63536
>> 12:33:06 -- 77400
>> 12:33:08 -- 60848
>> 12:33:11 -- 53104
>> 12:33:13 -- 63600
>> 12:33:15 -- 57128
>> 12:33:17 -- 80152
>> 12:33:19 -- 47176
>> 12:33:21 -- 67824
>> 12:33:23 -- 53872
>> 12:33:25 -- 64808
>> 12:33:27 -- 63432
>> 12:33:29 -- 59160
>> 12:33:31 -- 70512
>> 12:33:33 -- 57832
>> 12:33:35 -- 64160
>> 12:33:37 -- 52184
>> 
>> When there is "no delay" the numbers of the queue are between 0 and
> 3000.
>> So it is clear that the queue is delaying the requests.  From the
>> "simpliClient.pl" I added a TIMESTAMP for the moment it sends the
>> Request and when I see the reception in the debug from Radiator I also
>> see the delay.  Please check this :
>> 
>> Thu Nov 22 12:32:26 2012 781959: DEBUG: Packet dump:
>> *** Received from 64.76.154.202 port 57017 ....
>> Code:       Access-Request
>> Identifier: 0
>> Authentic:
>> <11>p<255><139><28><132><188><144><180><222><152>.<186>_M<26>
>> Attributes:
>>       User-Name = "5555848118 at mydomain.net"
>>       Service-Type = 18
>>       NAS-Port = 0
>>       NAS-IP-Address = 64.76.154.35
>>       cisco-h323-remote-address = "1353598345.50094 -- Thu Nov 22
>> 12:32:25 2012"
>> 
>> The packet was received at 12:32:26.781959  and it was send at
>> 12:32:25.50095, so there is 1,28 secs of delay in the queue.
>> I don't see any unusual logs in the Radiator.  WE have a normal rate
>> of 30 to 35 request per second but it grows near 50 request per second
>> when the delay start.
>> So.  We have One instance of Radiator to attend several NAS and the
>> configuration file it has 43 different Handlers (Could this be the
>> cause of the delay in the Radiator?).  Could be a good practice to
>> separate this instance in 4 or 5 instances with less Handlers?.  As
>> you mention the sockets are per instance, isn't?.  Or it is better to
>> have another server running the instantes?
>> Can you give me some hint of optimization?
>> 
>> Hope you can help me here.
>> Best Regards,
>> Ricardo.-
>> _______________________________________________
>> radiator mailing list
>> radiator at open.com.au
>> http://www.open.com.au/mailman/listinfo/radiator
> 
> 
> --
> 
> Hugh Irvine
> hugh at open.com.au
> 
> Radiator: the most portable, flexible and configurable RADIUS server
> anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
> Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS,
> TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER
> etc.
> Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.

--

Hugh Irvine
hugh at open.com.au

Radiator: the most portable, flexible and configurable RADIUS server 
anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, 
Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, 
TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP,
DIAMETER etc. 
Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.