[RADIATOR] Radiator Crash. SQL Timeout.
Heikki Vatiainen
hvn at open.com.au
Fri Dec 21 04:36:44 CST 2012
On 12/20/2012 02:58 PM, Ricardo Martinez wrote:
> I ran Radiator as you mention
>
> ./radiusd -config_file radius_siptrunk_edge.cfg -foreground -log_stdout
>
> The process crashed again, unfortunately the only error is a "killed"
> message in the console :
See dmesg command and log messages in /var/log log files. See if you can
find anything related to OOM (out of memory) that corresponds to the
moment radiusd gets 'Killed'.
Then see which version of DBD::Oracle you have and see if it has a
connection timeout related memory leak which was fixed in 1.28. I have
seen increased memory use happen on some cases when there are connection
failures.
Please let us know how it goes.
Thanks,
Heikki
> Thu Dec 20 01:05:12 2012 450537: DEBUG: Packet dump:
> *** Received from XX.XX.XX.130 port 43744 ....
> Code: Access-Request
> Identifier: 128
> Authentic: <143><163><137><253><163><148>)<239><253><182><203><226>E`0w
> Attributes:
> User-Name = "XX.XX.XX.68"
> User-Password = <234>CK<252>
> <137><176><159>b<210><142>1<162><148><201><17>
> Calling-Station-Id = "18017269888"
> Called-Station-Id = "1820#0056322935804"
> cisco-h323-conf-id = "h323-conf-id=76AF9ED4 D481907D EB98F4DD
> A3B81088"
> cisco-avpair =
> "call-id=37994417-3564965111-273540 at Argentina-1.Latinode.com"
> cisco-h323-remote-address = "h323-remote-address=XX.XX.XX.68"
> cisco-avpair = "h323-session-protocol=sipv2"
> NAS-Port = 5060
> NAS-IP-Address = XX.XX.XX.130
>
> Thu Dec 20 01:05:12 2012 452269: DEBUG: Handling request with Handler
> 'Request-Type = Access-Request, NAS-IP-Address=XX.XX.XX.130', Identifier
> ''
> Thu Dec 20 01:05:12 2012 452994: DEBUG: Deleting session for
> 200.49.30.68, XX.XX.XX.130, 5060
> Thu Dec 20 01:05:12 2012 453982: DEBUG: Handling with Radius::AuthGROUP:
> Thu Dec 20 01:05:12 2012 454762: DEBUG: Handling with Radius::AuthSQL:
> Thu Dec 20 01:05:12 2012 455498: DEBUG: Handling with Radius::AuthSQL:
> Thu Dec 20 01:05:12 2012 457500: DEBUG: Query is: 'select
> 'Routing:'||radiator.reemplazadominio(radiator.traduce('sip:'||a.numero||'
> @sip1.hola.net',
> 'sip:'||substr('1820#0056322935804',nvl(length(b.adm_info),0)+1)),b.termin
> al_alias)|| ';cli='||a.numero || fg. glbl_pkg.dejanumeros('18017269888')
> || ';cnam='||'18017269888'||
> ';rid=-1;expires=45;np_expires=15;maxcc='||b.canales from
> fg.fc_mayo_numero a,fg.fc_mayo_gateway b, fg.fc_mayo_gateway_ip c where
> a.gate_cod=c.gate_cod and b.gate_cod=a.gate_cod and c.ip=XX.XX.XX.68' and
> a.orden=1 and b.mayo_esta_cod=1 and b.voiss_esta_cod=1 and
> c.tipo_servicio=1':
> Thu Dec 20 01:05:14 2012 458533: ERR: Execute failed for 'select
> 'Routing:'||radiator.reemplazadominio(radiator.traduce('sip:'||a.numero||'
> @sip1.hola.net',
> 'sip:'||substr('1820#0056322935804',nvl(length(b.adm_info),0)+1)),b.termin
> al_alias)|| ';cli='||a.numero || fg. glbl_pkg.dejanumeros('18017269888')
> || ';cnam='||'18017269888'||
> ';rid=-1;expires=45;np_expires=15;maxcc='||b.canales from
> fg.fc_mayo_numero a,fg.fc_mayo_gateway b, fg.fc_mayo_gateway_ip c where
> a.gate_cod=c.gate_cod and b.gate_cod=a.gate_cod and c.ip='XX.XX.XX.68' and
> a.orden=1 and b.mayo_esta_cod=1 and b.voiss_esta_cod=1 and
> c.tipo_servicio=1': SQL Timeout
> Killed
>
> I think this one is difficult to track. Is there anything more that I
> could do ?
>
> Regards,
> Ricardo Martinez.-
>
>
> -----Mensaje original-----
> De: radiator-bounces at open.com.au [mailto:radiator-bounces at open.com.au] En
> nombre de Heikki Vatiainen
> Enviado el: lunes, 17 de diciembre de 2012 19:11
> Para: radiator at open.com.au
> Asunto: Re: [RADIATOR] Radiator Crash. SQL Timeout.
>
> On 12/17/2012 04:34 PM, Ricardo Martinez wrote:
>
>> We're experiencing several crashes of our Radius service and we don't
>> have any clue why this could be happening.
>
> Could you try running radiusd with -foreground and -log_stdout options
> instead of starting it the normal way?
>
> There are very likely messages from the libraries Radiator uses which show
> up only in STDERR or STDOUT streams. These messages can not be logged by
> radiusd, but they should show up on the console.
>
> When it crashes, please see if there are any other messages than what
> radiusd normally logs.
>
> Thanks,
> Heikki
>
>> We're using an AuthBY SQL against an ORACLE database. We have 5 to 6
>> more instances running in the same server without any problem, but two
>> of them are crashing once a day. This is part of the configuration file
> :
>>
>>
>>
>> LogStdout
>>
>> DbDir /usr/src/Radiator-4.9
>>
>> LogDir /var/log/radiator
>>
>> DictionaryFile %D/dictionary,%D/dictionary.sip,%D/dictionary.kamailio
>>
>> AuthPort 1654
>>
>> AcctPort
>>
>> BindAddress XX.XX.XX.XX
>>
>>
>>
>> # This will log at DEBUG level: very verbose
>>
>> # User a lower trace level in production systems, typically use 3
>>
>> Trace 3
>>
>> LogMicroseconds
>>
>>
>>
>> <Handler Request-Type = Access-Request, NAS-IP-Address=XX.XX.XX.130>
>>
>> # Look up user details via SQL
>>
>>
>>
>> <AuthBy GROUP>
>>
>>
>>
>> AuthByPolicy ContinueUntilAccept
>>
>> <AuthBy SQL>
>>
>> DBSource dbi:Oracle:xxxxx
>>
>> DBUsername raxxxxxx
>>
>> DBAuth radxxxx
>>
>>
>>
>> Timeout 2
>>
>> FailureBackoffTime 60
>>
>> SQLRetries 1
>>
>> NoDefault
>>
>> #Incluye Info de Anexo en CLI
>>
>> AuthSelect select
>> 'Routing:'||radiator.reemplazadominio(radiator.traduce('sip:'||a.numer
>> o||'@sip1.hola.net
>> <http://sip1.hola.net>', \
>>
>>
>> 'sip:'||substr('%{Called-Station-Id}',nvl(length(b.adm_info),0)+1)),b.
>> terminal_alias)||
>> \
>>
>> ';cli='||a.numero || fg.
>> glbl_pkg.dejanumeros('%{Calling-Station-Id}') ||
>> ';cnam='||'%{Calling-Station-Id}'|| \
>>
>> ';rid=-1;expires=45;np_expires=15;maxcc='||b.canales
>> from fg.fc_mayo_numero a,fg.fc_mayo_gateway b, \
>>
>> fg.fc_mayo_gateway_ip c where a.gate_cod=c.gate_cod
>> and b.gate_cod=a.gate_cod and c.ip='%{User-Name}' and \
>>
>> a.orden=1 and b.mayo_esta_cod=1 and b.voiss_esta_cod=1
>> and c.tipo_servicio=1
>>
>>
>>
>> AuthColumnDef 0, h323-ivr-in, reply
>>
>>
>>
>> </AuthBy>
>>
>> <AuthBy SQL>
>>
>> DBSource dbi:Oracle:xxxxx
>>
>> DBUsername raxxxxxx
>>
>> DBAuth radxxxx
>>
>>
>>
>> Timeout 2
>>
>> FailureBackoffTime 60
>>
>> SQLRetries 1
>>
>> NoDefault
>>
>> AuthSelect select
>> 'Routing:'||radiator.reemplazadominio(radiator.traduce('sip:'||a.numer
>> o||'@sip2.hola.net
>> <http://sip2.hola.net>', \
>>
>>
>> 'sip:'||substr('%{Called-Station-Id}',nvl(length(b.adm_info),0)+1)),b.
>> ras_address)||
>> \
>>
>> ';cli='||a.numero|| fg.
>> glbl_pkg.dejanumeros('%{Calling-Station-Id}')|| \
>>
>>
>> ';cnam='||'%{Calling-Station-Id}'||';rid=-1;expires=45;np_expires=15;m
>> axcc='||b.puertas from fg.fc_gateway_numero a,fg.fc_gateway \
>>
>> b,fg.fc_gateway_ip c where a.gate_cod=c.gate_cod and
>> b.gate_cod=a.gate_cod and c.ip='%{User-Name}' \
>>
>> and b.admin_esta_cod=1 and a.orden=1 and
>> c.tipo_servicio=1
>>
>>
>>
>> AuthColumnDef 0, h323-ivr-in, reply
>>
>>
>>
>> </AuthBy>
>>
>> <AuthBy INTERNAL>
>>
>> DefaultResult REJECT
>>
>> </AuthBy>
>>
>> </AuthBy>
>>
>> PostAuthHook file:"%D/SCRIPT_POSTAUTHHOOK"
>>
>> </Handler>
>>
>>
>>
>>
>>
>> This is the debug before the crash :
>>
>>
>>
>> Sun Dec 16 14:03:22 2012 764008: DEBUG: Handling with Radius::AuthGROUP:
>>
>> Sun Dec 16 14:03:22 2012 764431: DEBUG: Handling with Radius::AuthSQL:
>>
>> Sun Dec 16 14:03:22 2012 764774: DEBUG: Handling with Radius::AuthSQL:
>>
>> Sun Dec 16 14:03:22 2012 765795: DEBUG: Query is: 'select
>>
> 'Routing:'||radiator.reemplazadominio(radiator.traduce('sip:'||a.numero||'
>>
>> @sip1.hola.net <http://sipmy.redvoiss.net>',
>> 'sip:'||substr('1795#005625383191',nvl(length(b.adm_info),0)+1)),b.ter
>> minal_alias)||
>> ';cli='||a.numero || fg.
>>
>> glbl_pkg.dejanumeros('14802321589') || ';cnam='||'14802321589'||
>> ';rid=-1;expires=45;np_expires=15;maxcc='||b.canales from fg.fc_may
>>
>> o_numero a,fg.fc_mayo_gateway b, fg.fc_mayo_gateway_ip c where
>> a.gate_cod=c.gate_cod and b.gate_cod=a.gate_cod and c.ip='66.165.170.
>>
>> 214' and a.orden=1 and b.mayo_esta_cod=1 and b.voiss_esta_cod=1 and
>> c.tipo_servicio=1':
>>
>> Sun Dec 16 14:03:24 2012 766859: ERR: Execute failed for 'select
>> 'Routing:'||radiator.reemplazadominio(radiator.traduce('sip:'||a.nu
>> <http://a.nu>
>>
>> mero||'@sip1.hola.net <http://sipmy.redvoiss.net>',
>> 'sip:'||substr('1795#005625383191',nvl(length(b.adm_info),0)+1)),b.ter
>> minal_alias)||
>> ';cli='||a.numero
>>
>> || fg. glbl_pkg.dejanumeros('14802321589') ||
>> || ';cnam='||'14802321589'||
>> ';rid=-1;expires=45;np_expires=15;maxcc='||b.canales from fg
>>
>> .fc_mayo_numero a,fg.fc_mayo_gateway b, fg.fc_mayo_gateway_ip c where
>> a.gate_cod=c.gate_cod and b.gate_cod=a.gate_cod and c.ip='66.1
>>
>> 65.170.214' and a.orden=1 and b.mayo_esta_cod=1 and b.voiss_esta_cod=1
>> and c.tipo_servicio=1': SQL Timeout
>>
>>
>>
>> And then the service crashes.
>>
>> Could be a problem with the Select?, or maybe a bug for the Timeout?.
>> I have made the same configuration file in a lab enviroment, without
>> the "select" and if the database give me SQL Timeout it goes to the
>> Next Policy without any problem.
>>
>>
>>
>> Radius Version : 4.9
>>
>> Perl : perl5 (revision 5 version 10 subversion 1)
>>
>> OS : CentOS 6.0
>>
>> Kernel : 2.6.32-71.el6.x86_64
>>
>>
>>
>>
>>
>> Hope someone could help me here.
>>
>>
>>
>> Regards,
>>
>> Ricardo Martinez.-
>>
>>
>>
>>
>>
>> _______________________________________________
>> radiator mailing list
>> radiator at open.com.au
>> http://www.open.com.au/mailman/listinfo/radiator
>>
>
>
> --
> Heikki Vatiainen <hvn at open.com.au>
>
> Radiator: the most portable, flexible and configurable RADIUS server
> anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
> Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS,
> TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER
> etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc.
> _______________________________________________
> radiator mailing list
> radiator at open.com.au
> http://www.open.com.au/mailman/listinfo/radiator
>
--
Heikki Vatiainen <hvn at open.com.au>
Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS,
TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS,
NetWare etc.
More information about the radiator
mailing list