[RADIATOR] ERROR: invalid byte sequence for encoding "UTF8"

Wed Feb 29 15:28:03 CST 2012

On 02/28/2012 04:44 PM, Traiano Welcome wrote:

> Recently, I've begun seeing the following error message in my radius logs
> at the point where radius accounting packet data is inserted to the
> postgresql database I'm using with radiator 4.9:
> 
> ---
> ERROR: invalid byte sequence for encoding "UTF8": 0x9e
> HINT: This error can also happen if the byte sequence does not match the
> encoding expected by the server, which is controlled by "client_encoding".

I have seen that too. In your case it seems to be that calledstationid
is probably of type text or varchar, and a binary value was received and
then tried to be inserted in the database.

http://tools.ietf.org/html/rfc2865#section-5.30

The RFC recommends UTF-8, so it might be possible to configure the NAS
to send a string instead of binary.

It this is a problem with a certain client, then a ClientHook can be
used to rewrite Called-Station-Id to a value accepted by the database.

I'd say this is very rare. I have only seen strings in Called-Station-Id
attributes so far.

Heikki

> It appears that strings based on a non-UTF8 character set are being
> included in the radius accounting data and sent on to my radius server by
> the NAS (an ACME SBC appliance).
> 
> Below is a more complete segment of a sample log:
> 
> ---
> Tue Feb 28 10:44:30 2012: DEBUG: Handling request with Handler
> 'NAS-Identifier=/^TRAITECH-.*$/', Identifier ''
> Tue Feb 28 10:44:30 2012: DEBUG:  Adding session for , 127.0.0.100, 5060
> Tue Feb 28 10:44:30 2012: DEBUG: Handling with Radius::AuthSQL:
> Tue Feb 28 10:44:30 2012: DEBUG: Handling accounting with Radius::AuthSQL
> 
> Tue Feb 28 10:44:30 2012: DEBUG: do query is: 'insert into acmevoiprecords
> (calledstationid,callingstationid,egresscallid,ingresscallid,nasidentifier,
> nasipaddress,sessionegresscallid,sessionforkedcallid,sessiongenericid,sessi
> oningresscallid,sessionprotocoltype,statustype,timestamp) values
> ('<9e>^N^F','Matroos,16319
> <sip:0873116319 at 172.32.0.61>;tag=9tBN868U1B42S^^
> ','23c4e3e9-dc8b-122f-088a-5cf3fc962edc','23c4e3e9-dc8b-122f-088a-5cf3fc962
> edc','TRAITECH-DBN-VPR','127.0.0.100','23c4e3e9-dc8b-122f-088a-5cf3fc962edc
> ','','','23c4e3e9-dc8b-122f-088a-5cf3fc962edc','SIP','Stop','1330418670')':
> 
> Tue Feb 28 10:44:30 2012: DEBUG: do query is: 'insert into acmevoiprecords
> (calledstationid,callingstationid,cdrsequencenumber,connecttime,customvsa20
> 0,customvsa201,customvsa202,customvsa203,customvsa204,customvsa205,customvs
> a206,customvsa207,customvsa208,customvsa209,customvsa210,customvsa211,custo
> mvsa212,customvsa213,customvsa214,customvsa215,customvsa216,customvsa217,cu
> stomvsa218,customvsa219,customvsa220,customvsa221,customvsa222,customvsa223
> ,customvsa224,customvsa225,customvsa226,customvsa227,customvsa228,customvsa
> 229,customvsa230,egresscallid,egressfinalroutingnumber,egresslocaladdr,egre
> ssnetworkinterfaceid,egressremoteaddr,egressvlantagvalue,firmwareversion,fl
> owindstaddr,flowinsrcaddr,flowoutdstaddr,flowoutsrcaddr,ingresscallid,ingre
> sslocaladdr,ingressnetworkinterfaceid,ingressrealm,ingressremoteaddr,ingres
> svlantagvalue,localtimezone,nasidentifier,nasipaddress,originatingtrunkcont
> ext,originatingtrunkgroup,passertedid,postdialdelay,primaryroutingnumber,se
> ssionchargingvector,sessionegresscallid,sessionegressrealm,sessionegressrph
> ,sessionforkedcallid,sessiongenericid,sessioningresscallid,sessioningressre
> alm,sessioningressrph,sessionprotocoltype,setuptime,sipdiversion,statustype
> ,terminatingtrunkcontext,terminatingtrunkgroup,timestamp) values
> ('<sip:+27838781684 at 192.168.0.236>','"+2721113641"
> <sip:+2721113641 at 192.168.0.38>;tag=as6b537178',10019633,'00:00:00.000 SAST
> JAN 01 
> 1970','','','','','','','','','','','','','','','','','','','','','','','',
> '','','','','','','','','4e25d24c4df4a88c57d0ed281ca3a638 at 192.168.0.38','',
> '0.0.0.0:0','','0.0.0.0:0','0','SCX6.2.0 MR-3 Patch 1 (Build
> 642)','0.0.0.0','0.0.0.0','0.0.0.0','0.0.0.0','4e25d24c4df4a88c57d0ed281ca3
> a638 at 192.168.0.38','192.168.0.236:5060','t3p-in0_0','t3_ecn','192.168.0.33:
> 5060','2027','GMT+02:00','TRAITECH-DBN-VPR','127.0.0.100','','','','0','sip
> :+27838781684 at 192.168.0.236','','4e25d24c4df4a88c57d0ed281ca3a638 at 192.168.0
> .38','','','','','4e25d24c4df4a88c57d0ed281ca3a638 at 192.168.0.38','t3_ecn','
> ','SIP','10:44:18.607 SAST FEB 28 2012','','Start','','','1330418670')':
> 
> Tue Feb 28 10:44:30 2012: ERR: do failed for 'insert into acmevoiprecords
> (calledstationid,callingstationid,egresscallid,ingresscallid,nasidentifier,
> nasipaddress,sessionegresscallid,sessionforkedcallid,sessiongenericid,sessi
> oningresscallid,sessionprotocoltype,statustype,timestamp) values
> ('<9e>^N^F','Matroos,16319
> <sip:0873116319 at 172.32.0.61>;tag=9tBN868U1B42S^^
> ','23c4e3e9-dc8b-122f-088a-5cf3fc962edc','23c4e3e9-dc8b-122f-088a-5cf3fc962
> edc','TRAITECH-DBN-VPR','127.0.0.100','23c4e3e9-dc8b-122f-088a-5cf3fc962edc
> ','','','23c4e3e9-dc8b-122f-088a-5cf3fc962edc','SIP','Stop','1330418670')':
>  ERROR:  invalid byte sequence for encoding "UTF8": 0x9e
> 
> HINT:  This error can also happen if the byte sequence does not match the
> encoding expected by the server, which is controlled by "client_encoding".
> 
> Tue Feb 28 10:44:30 2012: DEBUG: AuthBy SQL result: ACCEPT,
> Tue Feb 28 10:44:30 2012: DEBUG: Accounting accepted
> Tue Feb 28 10:44:30 2012: DEBUG: Packet dump:
> 
> *** Received from 192.168.93.3 port 1098 ....
> ----
> 
> 
> 
> Has anyone encountered this issue before ? Would this be something I
> should have fixed on the NAS end of things, or is there some way to work
> around it in Radiator or in my  postgresql database configuration?
> 	
> 
> Thanks in advance,
> Traiano
> 
> _______________________________________________
> radiator mailing list
> radiator at open.com.au
> http://www.open.com.au/mailman/listinfo/radiator

-- 
Heikki Vatiainen <hvn at open.com.au>

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS,
TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS,
NetWare etc.