(RADIATOR) Threading support (again)

Hugh Irvine hugh at open.com.au
Wed Sep 5 16:31:30 CDT 2007


Hello Alex -

Radiator only ever issues SNMP calls if you have the NasType set in  
the Client clauses, _and_ if there is a session limit exception. In  
other words, Radiator only ever checks the NAS if it has to when a  
session limit is exceeded. There are debug messages in the code, but  
you will only see them when and if Radiator needs to run a check on a  
client device.

We have looked at POE, however we tend to find using multiple  
instances of Radiator to be easier to design, build and maintain.

As has been pointed out by myself and others, it is almost never  
Radiator itself which is the problem - rather it is almost always the  
backend database that Radiator is waiting for. As I have stated  
before, there is no point in using multi-threading if all of the  
threads are going to be stalled waiting for the same slow resource.

regards

Hugh


On 5 Sep 2007, at 23:35, Hartmaier Alexander wrote:

> Hi Hugh!
>
> I was just curious if Radiator really makes snmp gets to the  
> various devices because I've never encountered any positive or  
> negative log entries.
> I don't even know if our firewalls are configured to let our radius  
> servers do snmp requests to the various devices.
>
> Because Radiator is blocking on DBI calls I assume it also is on  
> e.g. snmp checks which might time out if not configured on the  
> firewall or other problem like wrong snmp (read) community string.
>
> Our problems occur on high traffic where Radiator starts to delay  
> the replies so that the devices send multiple requests when their  
> set timeout kicks in.
> I wasn't able to figure out what exactly causes the problem but I  
> assume the accounting and/or radonline sql queries block Radiator.
>
> Have you thought about using POE to make Radiator non-blocking?
>
> -Alex
>
> -----Original Message-----
> From: Hugh Irvine [mailto:hugh at open.com.au]
> Sent: Wednesday, September 05, 2007 5:24 AM
> To: Hartmaier Alexander
> Subject: Re: (RADIATOR) Threading support (again)
>
>
> Hello Alex -
>
> Thanks - from what I can see there are log messages in most of the
> Radius/Nas/* modules.
>
> Can you give me an example of what you see happening currently and
> what you would like to see?
>
> regards
>
> Hugh
>
>
> On 5 Sep 2007, at 02:44, Hartmaier Alexander wrote:
>
>> Hi Hugh!
>>
>> You can configure the client type with the 'NasType' config  
>> parameter.
>> According to the docs this will be use to determine the way of
>> checking logging in session once in a while (e.g. ping, snmp,
>> finger,...).
>>
>> -Alex
>>
>>
>> -----Original Message-----
>> From: Hugh Irvine [mailto:hugh at open.com.au]
>> Sent: Tuesday, September 04, 2007 1:31 AM
>> To: Hartmaier Alexander
>> Cc: radiator at open.com.au
>> Subject: Re: (RADIATOR) Threading support (again)
>>
>>
>> Hello Alexander -
>>
>> Can you clarify what you mean by "snmp checks"?
>>
>> regards
>>
>> Hugh
>>
>>
>> On 3 Sep 2007, at 23:54, Hartmaier Alexander wrote:
>>
>>> Hi!
>>>
>>> We encountered similar problems in the last few month when traffic
>>> rises.
>>> Have you thought about using POE and various components (like
>>> asynchronous DBI
>>> calls) for Radiator?
>>> For us it seems that the accounting inserts block radiator  
>>> sometimes.
>>> How could be snmp checks be logged to verify that aren't the cause
>>> of our
>>> problems?
>>> I never saw a single log line for those even with a trace 5 log.
>>>
>>> With best regards
>>> Alexander Hartmaier
>>>
>>> T-Systems Austria GesmbH
>>> Rennweg 97-99
>>> A-1030 Vienna
>>>
>>> phone: +43-(0)57057-4320
>>> mobile: +43-(0)676-8642-4320
>>> -----Original Message-----
>>> From: owner-radiator at open.com.au [mailto:owner-
>>> radiator at open.com.au] On Behalf
>>> Of John Morrissey
>>> Sent: Friday, August 24, 2007 11:46 PM
>>> To: radiator at open.com.au
>>> Subject: (RADIATOR) Threading support (again)
>>>
>>> Looking through the archives, it's been a long time since someone's
>>> kicked
>>> the threading horse.
>>>
>>> We've been running Radiator on a fairly substantial RADIUS
>>> installation (a
>>> dozen or so machines in three locations across the United States)  
>>> for
>>> several years now, making use of an LDAP directory for account
>>> storage.
>>> (Hugh or Mike, feel free to e-mail me privately if you need more
>>> information.) Unfortunately, OpenLDAP libraries do not seem well
>>> behaved
>>> when Radiator forks, although it's been a while since we've tried.
>>>
>>> Our largest problem with a monolithic process is that we've had
>>> occasional
>>> problems where device outages cause thousands of subscribers to
>>> reauthenticate simultaneously. This overwhelms our authentication
>>> radiusd
>>> instances, causing UDP network queues on that port to skyrocket,
>>> eventually
>>> leading to a cascade failure that is obnoxious to recover from, to
>>> say the
>>> least. Our RADIUS servers are by no means underpowered, but it's
>>> just not
>>> cost-effective to maintain a huge (and otherwise idle) hardware
>>> pool just to
>>> handle occasional huge spikes like this.
>>>
>>> Additionally, we're starting to deploy machines with four CPU cores
>>> as their
>>> price has dropped substantially. We like sticking to a fairly
>>> standard
>>> configuration, so we'd like to avoid "wasting" hardware that
>>> Radiator will
>>> never use (we have auth and accounting split into two Radiator
>>> instances).
>>> We could run more radiusd processes on different ports or IP
>>> addresses, but
>>> this seems wasteful and complex.
>>>
>>> Lastly, we're looking at implementing simultaneous use restrictions
>>> and are
>>> concerned that the state refresh (SNMP in some cases, locally
>>> implemented
>>> interfaces to proprietary devices in others) will block radiusd
>>> excessively.
>>> We're also looking at proxying to more customers, and are concerned
>>> that
>>> unreachable or underperforming proxy destinations will cause the  
>>> same
>>> behavior.
>>>
>>> We had considered implementing rate limiting to handle our first
>>> concern
>>> above, but that doesn't get us anywhere with the others. To that
>>> end, I'm
>>> toying with the idea of adding a new process model to Radiator
>>> somewhat akin
>>> to Apache's prefork MPM, with a number of child processes (not
>>> threads)
>>> running round-robin on the incoming UDP port or handling work
>>> directed at
>>> them via some sort of IPC from a central request handler process.
>>>
>>> I haven't dived into the code yet to see how non-trivial this will
>>> be, and I
>>> thought I'd ask around here first to see what everyone's thoughts
>>> are. I
>>> know that a threaded model hasn't been implemented due to alpha-
>>> quality
>>> thread support in Perl, and I haven't looked into its current state
>>> yet.
>>> That's part of the reason I'm keen on using separate worker  
>>> processes
>>> instead of threads.
>>>
>>> thanks,
>>> john
>>> -- 
>>> John Morrissey          _o            /\         ----  __o
>>> jwm at horde.net        _-< \_          /  \       ----  <  \,
>>> www.horde.net/    __(_)/_(_)________/    \_______(_) /_(_)__
>>>
>>> --
>>> Archive at http://www.open.com.au/archives/radiator/
>>> Announcements on radiator-announce at open.com.au
>>> To unsubscribe, email 'majordomo at open.com.au' with
>>> 'unsubscribe radiator' in the body of the message.
>>>
>>
>>
>>
>> NB:
>>
>> Have you read the reference manual ("doc/ref.html")?
>> Have you searched the mailing list archive (www.open.com.au/archives/
>> radiator)?
>> Have you had a quick look on Google (www.google.com)?
>> Have you included a copy of your configuration file (no secrets),
>> together with a trace 4 debug showing what is happening?
>> Have you checked the RadiusExpert wiki:
>> http://www.open.com.au/wiki/index.php/Main_Page
>>
>> -- 
>> Radiator: the most portable, flexible and configurable RADIUS server
>> anywhere. Available on *NIX, *BSD, Windows, MacOS X.
>> Includes support for reliable RADIUS transport (RadSec),
>> and DIAMETER translation agent.
>> -
>> Nets: internetwork inventory and management - graphical, extensible,
>> flexible with hardware, software, platform and database independence.
>> -
>> CATool: Private Certificate Authority for Unix and Unix-like systems.
>>
>>
>>
>> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"* 
>> "
>> *"*"*"*
>> T-Systems Austria GesmbH   Rennweg 97-99, 1030 Wien
>> Handelsgericht Wien, FN 79340b
>> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"* 
>> "
>> *"*"*"*
>> Notice: This e-mail contains information that is confidential and
>> may be privileged.
>> If you are not the intended recipient, please notify the sender and
>> then delete this e-mail immediately.
>> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"* 
>> "
>> *"*"*"*
>
>
>
> NB:
>
> Have you read the reference manual ("doc/ref.html")?
> Have you searched the mailing list archive (www.open.com.au/archives/
> radiator)?
> Have you had a quick look on Google (www.google.com)?
> Have you included a copy of your configuration file (no secrets),
> together with a trace 4 debug showing what is happening?
> Have you checked the RadiusExpert wiki:
> http://www.open.com.au/wiki/index.php/Main_Page
>
> -- 
> Radiator: the most portable, flexible and configurable RADIUS server
> anywhere. Available on *NIX, *BSD, Windows, MacOS X.
> Includes support for reliable RADIUS transport (RadSec),
> and DIAMETER translation agent.
> -
> Nets: internetwork inventory and management - graphical, extensible,
> flexible with hardware, software, platform and database independence.
> -
> CATool: Private Certificate Authority for Unix and Unix-like systems.
>
>
>
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*" 
> *"*"*"*
> T-Systems Austria GesmbH   Rennweg 97-99, 1030 Wien
> Handelsgericht Wien, FN 79340b
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*" 
> *"*"*"*
> Notice: This e-mail contains information that is confidential and  
> may be privileged.
> If you are not the intended recipient, please notify the sender and  
> then delete this e-mail immediately.
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*" 
> *"*"*"*
>
> --
> Archive at http://www.open.com.au/archives/radiator/
> Announcements on radiator-announce at open.com.au
> To unsubscribe, email 'majordomo at open.com.au' with
> 'unsubscribe radiator' in the body of the message.



NB:

Have you read the reference manual ("doc/ref.html")?
Have you searched the mailing list archive (www.open.com.au/archives/ 
radiator)?
Have you had a quick look on Google (www.google.com)?
Have you included a copy of your configuration file (no secrets),
together with a trace 4 debug showing what is happening?
Have you checked the RadiusExpert wiki:
http://www.open.com.au/wiki/index.php/Main_Page

-- 
Radiator: the most portable, flexible and configurable RADIUS server
anywhere. Available on *NIX, *BSD, Windows, MacOS X.
Includes support for reliable RADIUS transport (RadSec),
and DIAMETER translation agent.
-
Nets: internetwork inventory and management - graphical, extensible,
flexible with hardware, software, platform and database independence.
-
CATool: Private Certificate Authority for Unix and Unix-like systems.


--
Archive at http://www.open.com.au/archives/radiator/
Announcements on radiator-announce at open.com.au
To unsubscribe, email 'majordomo at open.com.au' with
'unsubscribe radiator' in the body of the message.


More information about the radiator mailing list