[RADIATOR] Radiator performance

Francisco Rodrigo Cortinas Maseda francisco.cortinas at jazztel.com
Wed Feb 10 05:42:49 CST 2010


Hi Alex,

Try:

<Client DEFAULT>
        Secret  mysecret
#       DupInterval 0
#       NoIgnoreDuplicates Accounting-Request
</Client>


Francisco Rodrigo Cortiñas
Jazz Telecom SAU
Telefono: +34 91 183 9046
Email: francisco.cortinas at jazztel.com


-----Mensaje original-----
De: radiator-bounces at open.com.au [mailto:radiator-bounces at open.com.au] En nombre de Alex Massover
Enviado el: miércoles 10 de febrero de 2010 12:18
Para: Hugh Irvine
CC: radiator at open.com.au
Asunto: Re: [RADIATOR] Radiator performance


Hi,

And this still happens:
a) without syslog (with Trace 0)
b) without Farm
c) without SOAP (with AuthByInternal)

It really looks like every request came is not getting freed from the memory (I only test accounting requests). I'm using Radiator 4.5.1 on Red Hat Enterprise Linux Server release 5.2 (Tikanga) x86_64


Here's my full config:

FarmSize 32
SocketQueueLength 524288000
#Foreground
#LogStdout
LogDir          /var/log/radius
DbDir           /etc/radiator
DictionaryFile %D/dictionary, %D/dictionary.sip
# Use a low trace level in production systems. Increase
# it to 4 or 5 for debugging, or use the -trace flag to radiusd
Trace           0


# You will probably want to add other Clients to suit your site, # one for each NAS you want to work with <Client DEFAULT>
        Secret  mysecret
#       DupInterval 0
        NoIgnoreDuplicates Accounting-Request
</Client>
<Realm loadbalancer>
        <AuthBy INTERNAL>
                DefaultResult ACCEPT
        </AuthBy>
</Realm>

<Realm DEFAULT>
        <Log SYSLOG>
                Facility local2
                Trace 5
        </Log>

       <AuthBy SOAP>
                Timeout 1
               Endpoint http://MSG-LB-BES-STG:80/SoapHandler.ashx
              SOAPTrace all
       </AuthBy>

</Realm>

> -----Original Message-----
> From: Alex Massover
> Sent: יום ד 10 פברואר 2010 10:46
> To: 'Hugh Irvine'
> Cc: radiator at open.com.au
> Subject: RE: [RADIATOR] Radiator performance
>
> Hi!
>
> After a several hours of processing tens of requests per second the
> memory usage grows from 700M to 4G and the server hangs.
>
> I run Radiator with dispatcher with 32 instances doing AuthbySOAP.
>
> Does it need more than 4G memory? What are the requirements?
>
> If Radiator processes requests serially there should be no memory
> usage growth from request to request as far as I understand.
>
> Maybe it's a memory leak?
>
> > -----Original Message-----
> > From: Hugh Irvine [mailto:hugh at open.com.au]
> > Sent: יום ג 09 פברואר 2010 12:17
> > To: Alex Massover
> > Subject: Re: [RADIATOR] Radiator performance
> >
> >
> > Hello Alex -
> >
> > Yes it makes sense to do what you describe, and as RadSec is a TCP
> > protocol it will certainly help UDP problems.
> >
> > BTW - you should also run separate Radiator instances for
> > authentication and accounting which will improve performance.
> >
> > regards
> >
> > Hugh
> >
> >
> > On 9 Feb 2010, at 20:59, Alex Massover wrote:
> >
> > > Yes, I see. Also 'netstat -su' clearly shows dropped packets.
> > >
> > > I configured big UDP buffer (50M), and configured the farm of 32
> > instances - looks like it handles the designed load.
> > >
> > >
> > > I have another question to consult, please. Because of performance
> > issues NAS software does not wait for accounting reply and does not
> try
> > to resend it.
> > >
> > > So I afraid of dropped packets. Does RadSec supposed to help me
> with
> > it?
> > >
> > > I want to put RadSec locally on each NAS, NAS will send udp
> > accounting requests to RadSec via local interface, and RadSec will
> send
> > accounting requests to Radiator via TCP in reliable way.
> > >
> > > Does it make sense? Will it also help with UDP buffer queue in
> > Radiator?
> > >
> > >> -----Original Message-----
> > >> From: Hugh Irvine [mailto:hugh at open.com.au]
> > >> Sent: יום ו 05 פברואר 2010 00:58
> > >> To: Alex Massover
> > >> Cc: radiator at open.com.au
> > >> Subject: Re: [RADIATOR] Radiator performance
> > >>
> > >>
> > >> Hello Alex -
> > >>
> > >> This problem has been discussed numerous times on the mailing
> list.
> > >>
> > >> What tends to happen is Radiator is kept waiting by an external
> > >> resource such as a SOAP interface, the UDP buffer queue that
> > Radiator
> > >> is listening on fills up and overflows, and the client NAS
> equipment
> > >> see requests timing out and retransmit.
> > >>
> > >> The problem is almost always due to the delay caused by the
> external
> > >> resource.
> > >>
> > >> As mentioned in my previous mail, the first thing to do is look
> > >> at
> a
> > >> trace 4 debug with LogMicroseconds so you can see exactly how
> > >> long
> > each
> > >> processing step is taking.
> > >>
> > >> In your case it sounds like there is a ~50 millisecond response
> time
> > to
> > >> your SOAP queries (ie. 20 requests per second).
> > >>
> > >> Note this comment in the example "goodies/soap.cfg" configuration
> > file:
> > >>
> > >>
> > >>                # Caution: the overheads of creating and
> > >> processing SOAP requests
> > >>                # mean you will never get carrier class
> > >> performance from AuthBy SOAP
> > >>
> > >>
> > >> Also note that if Radiator is not seeing the requests at all,
> > >> this
> > is
> > >> confirmation that the system UDP buffer queue has overflowed.
> > >>
> > >> regards
> > >>
> > >> Hugh
> > >>
> > >>
> > >> On 5 Feb 2010, at 01:46, Alex Massover wrote:
> > >>
> > >>> Hi!
> > >>>
> > >>> But my problem is not about delay, it's about losing/not
> processing
> > >> request.
> > >>>
> > >>> For example I captured with wireshark an accounting request came
> to
> > >> the server but no evidence to this request in radiator log.
> Meaning
> > >> Radiator just silently ignored this request.
> > >>>
> > >>> Farm didn't help, I experience the same behavior.
> > >>>
> > >>> Radiator reports that there's no dropped request:
> > >>>
> > >>>         Reply-Message = "Radiator Radius server version 4.5.1"
> > >>>         Reply-Message = "Running on nj-radius-stg1.jajah.dublin
> since Thu
> > >> Feb  4 12:19:53 2010"
> > >>>         Reply-Message = "6 Requests in the last second"
> > >>>         Reply-Message = "0 Access accepts"
> > >>>         Reply-Message = "0 Access challenges"
> > >>>         Reply-Message = "0 Access rejects"
> > >>>         Reply-Message = "0 Access requests"
> > >>>         Reply-Message = "9143 Accounting requests"
> > >>>         Reply-Message = "9143 Accounting responses"
> > >>>         Reply-Message = "0 Bad authenticators in authentication
> requests"
> > >>>         Reply-Message = "0 Bad authenticators in accounting
> requests"
> > >>>         Reply-Message = "0 Total Bad authenticators in requests"
> > >>>         Reply-Message = "0 Dropped access requests"
> > >>>         Reply-Message = "0 Dropped accounting requests"
> > >>>         Reply-Message = "0 Total dropped requests"
> > >>>         Reply-Message = "0 Duplicate access requests"
> > >>>         Reply-Message = "0 Duplicate accounting requests"
> > >>>         Reply-Message = "0 Total duplicate requests"
> > >>>         Reply-Message = "0 Malformed access requests"
> > >>>         Reply-Message = "0 Malformed accounting requests"
> > >>>         Reply-Message = "0 Total proxied requests with no reply"
> > >>>         Reply-Message = "0 Total proxied requests"
> > >>>         Reply-Message = "9143 Total requests"
> > >>>         Reply-Message = "0.162143349699178 Average response
> > >>> time"
> > >>>
> > >>>
> > >>>> -----Original Message-----
> > >>>> From: Hugh Irvine [mailto:hugh at open.com.au]
> > >>>> Sent: יום ד 03 פברואר 2010 00:41
> > >>>> To: Alex Massover
> > >>>> Cc: radiator at open.com.au
> > >>>> Subject: Re: [RADIATOR] Radiator performance
> > >>>>
> > >>>>
> > >>>> Hello Alex -
> > >>>>
> > >>>> The best way to see what is happening with your authentications
> is
> > >> to
> > >>>> run Radiator with a Trace 4 debug and a LogMicroseconds logger
> > >>>> (requires Time-Hires from CPAN) so you can see how long each
> > >> processing
> > >>>> step is taking.
> > >>>>
> > >>>> Something like this in your configuration file:
> > >>>>
> > >>>> .....
> > >>>>
> > >>>> Trace 4
> > >>>>
> > >>>> <Log FILE>
> > >>>>        Filename %L/microseconds-%Y-%m-%d.log
> > >>>>        # requires Time-Hires from CPAN
> > >>>>        LogMicroseconds
> > >>>>        Trace 4
> > >>>> </Log>
> > >>>>
> > >>>> .....
> > >>>>
> > >>>> This will add a six digit microseconds offset in each debug
> > >> timestamp
> > >>>> so you can see where the delays are.
> > >>>>
> > >>>> Once we have the LogMicroseconds debug we can see exactly what
> is
> > >>>> happening.
> > >>>>
> > >>>> regards
> > >>>>
> > >>>> Hugh
> > >>>>
> > >>>>
> > >>>> On 3 Feb 2010, at 00:49, Alex Massover wrote:
> > >>>>
> > >>>>> Hi!
> > >>>>>
> > >>>>> What performance I can expect from single Radiator server?
> > >>>>>
> > >>>>> Currently I’m getting timeout from clients with more than 20
> > >> requests
> > >>>> per second, on local LAN, with very low CPU usage and load
> > average,
> > >> on
> > >>>> VMware ESX, RHEL 5.
> > >>>>>
> > >>>>> Is it OK and I should get more servers, or it supposed to
> handle
> > >> much
> > >>>> more? 20 requests per second doesn’t sound a lot for me.
> > >>>>>
> > >>>>> My configuration is also very simple and SOAP endpoint always
> > >> answers
> > >>>> fast, no timeouts/rejects from there.
> > >>>>>
> > >>>>> <Realm DEFAULT>
> > >>>>>               <Log SYSLOG>
> > >>>>>                               Facility local2
> > >>>>>                               Trace 5
> > >>>>>               </Log>
> > >>>>>               AuthByPolicy ContinueUntilAccept
> > >>>>>
> > >>>>>      <AuthBy SOAP>
> > >>>>> #              Fork
> > >>>>>                               Timeout 5
> > >>>>>              Endpoint http://MSG-LB-BES-STG:80/SoapHandler.ashx
> > >>>>>              SOAPTrace result
> > >>>>>              SOAPTrace all
> > >>>>>      </AuthBy>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Best Regards,
> > >>>>> Alex Massover
> > >>>>> VoIP R&D TL
> > >>>>> Jajah Inc.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> This mail was sent via Mail-SeCure System.
> > >>>>> _______________________________________________
> > >>>>> radiator mailing list
> > >>>>> radiator at open.com.au
> > >>>>> http://www.open.com.au/mailman/listinfo/radiator
> > >>>>
> > >>>>
> > >>>>
> > >>>> NB:
> > >>>>
> > >>>> Have you read the reference manual ("doc/ref.html")? Have you
> > >>>> searched the mailing list archive
> > >>>> (www.open.com.au/archives/radiator)?
> > >>>> Have you had a quick look on Google (www.google.com)? Have you
> > >>>> included a copy of your configuration file (no
> secrets),
> > >>>> together with a trace 4 debug showing what is happening?
> > >>>>
> > >>>> --
> > >>>> Radiator: the most portable, flexible and configurable RADIUS
> > server
> > >>>> anywhere. Available on *NIX, *BSD, Windows, MacOS X. Includes
> > >>>> support for reliable RADIUS transport (RadSec), and DIAMETER
> > >>>> translation agent.
> > >>>> -
> > >>>> Nets: internetwork inventory and management - graphical,
> > extensible,
> > >>>> flexible with hardware, software, platform and database
> > >> independence.
> > >>>> -
> > >>>> CATool: Private Certificate Authority for Unix and Unix-like
> > >> systems.
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> This mail was received via Mail-SeCure System.
> > >>>>
> > >>>
> > >>>
> > >>> This mail was sent via Mail-SeCure System.
> > >>>
> > >>>
> > >>> _______________________________________________
> > >>> radiator mailing list
> > >>> radiator at open.com.au
> > >>> http://www.open.com.au/mailman/listinfo/radiator
> > >>
> > >>
> > >>
> > >> NB:
> > >>
> > >> Have you read the reference manual ("doc/ref.html")? Have you
> > >> searched the mailing list archive
> > >> (www.open.com.au/archives/radiator)?
> > >> Have you had a quick look on Google (www.google.com)? Have you
> > >> included a copy of your configuration file (no secrets), together
> > >> with a trace 4 debug showing what is happening?
> > >>
> > >> --
> > >> Radiator: the most portable, flexible and configurable RADIUS
> server
> > >> anywhere. Available on *NIX, *BSD, Windows, MacOS X. Includes
> > >> support for reliable RADIUS transport (RadSec), and DIAMETER
> > >> translation agent.
> > >> -
> > >> Nets: internetwork inventory and management - graphical,
> extensible,
> > >> flexible with hardware, software, platform and database
> > independence.
> > >> -
> > >> CATool: Private Certificate Authority for Unix and Unix-like
> > systems.
> > >>
> > >>
> > >>
> > >>
> > >> This mail was received via Mail-SeCure System.
> > >>
> > >
> > >
> > > This mail was sent via Mail-SeCure System.
> >
> >
> >
> > NB:
> >
> > Have you read the reference manual ("doc/ref.html")?
> > Have you searched the mailing list archive
> > (www.open.com.au/archives/radiator)?
> > Have you had a quick look on Google (www.google.com)?
> > Have you included a copy of your configuration file (no secrets),
> > together with a trace 4 debug showing what is happening?
> >
> > --
> > Radiator: the most portable, flexible and configurable RADIUS server
> > anywhere. Available on *NIX, *BSD, Windows, MacOS X. Includes
> > support for reliable RADIUS transport (RadSec), and DIAMETER
> > translation agent.
> > -
> > Nets: internetwork inventory and management - graphical, extensible,
> > flexible with hardware, software, platform and database
> > independence.
> > -
> > CATool: Private Certificate Authority for Unix and Unix-like systems.
> >
> >
> >
> >
> > This mail was received via Mail-SeCure System.
> >


This mail was sent via Mail-SeCure System. _______________________________________________
radiator mailing list
radiator at open.com.au http://www.open.com.au/mailman/listinfo/radiator

Antes de imprimir este e-mail piense bien si es necesario hacerlo.


--------------------------------------------------------------------------------

Este mensaje es privado y CONFIDENCIAL y se dirige exclusivamente a su destinatario. Si usted ha recibido este mensaje por error, no debe revelar, copiar, distribuir o usarlo en ningún sentido. Le rogamos lo comunique al remitente y borre dicho mensaje y cualquier documento adjunto que pudiera contener. El correo electrónico via Internet no permite asegurar la confidencialidad de los mensajes que se transmiten ni su integridad o correcta recepción. JAZZTEL no asume responsabilidad por estas circunstancias. Si el destinatario de este mensaje no consintiera la utilización del correo electrónico via Internet y la grabación de los mensajes, rogamos lo ponga en nuestro conocimiento de forma inmediata.Cualquier opinión expresada en este mensaje pertenece únicamente al autor remitente, y no representa necesariamente la opinión de JAZZTEL, a no ser que expresamente se diga y el remitente esté autorizado para hacerlo.


--------------------------------------------------------------------------------


This message is private and CONFIDENTIAL and it is intended exclusively for its addressee. If you receive this message in error, you should not disclose, copy, distribute this e-mail or use it in any other way. Please inform the sender and delete the message and attachments from your system.Internet e-mail neither guarantees the confidentiality nor the integrity or proper receipt of the messages sent. JAZZTEL does not assume any liability for those circumstances. If the addressee of this message does not consent to the use of Internet e-mail and message recording, please notify us immediately.Any views or opinions contained in this message are solely those of the author, and do not necessarily represent those of JAZZTEL, unless otherwise specifically stated and the sender is authorised to do so.


--------------------------------------------------------------------------------


More information about the radiator mailing list