(RADIATOR) Radiator+MySQL problem

Romain Vergniol romain.vergniol at cegedim.fr
Mon Jun 3 03:56:26 CDT 2002


I've upgraded to Radiator 3.1 and the problem is fixed.


After the upgrade, when I start radiusd from the console, lots of messages
apear as said in a previous mail
(http://www.open.com.au/archives/radiator/2002-05/msg00214.html).
Ex:
AuthLogSQL log Radius::AuthLogSQL=HASH(0x84982e0) 0
Radius::Radius=HASH(0x87691d4)
AuthLogSQL log Radius::AuthLogSQL=HASH(0x84982e0) 0
Radius::Radius=HASH(0x878052c)
AuthLogSQL log Radius::AuthLogSQL=HASH(0x84982e0) 0
Radius::Radius=HASH(0x8788be8)
AuthLogSQL log Radius::AuthLogSQL=HASH(0x84982e0) 0
Radius::Radius=HASH(0x8780660)
...

Is there a way to avoid this ?

Thanks again.

Romain


----- Original Message -----
From: "Mike McCauley" <mikem at open.com.au>
To: <hugh at open.com.au>; "Romain Vergniol" <romain.vergniol at cegedim.fr>
Cc: "Charles Delorme" <charles.delorme at cegedim.fr>; "Mailing-List Radiator"
<radiator at open.com.au>
Sent: Saturday, June 01, 2002 1:52 AM
Subject: Re: (RADIATOR) Radiator+MySQL problem


> Hello Romain,
>
> this is almost certainly the SQL Trace 4 infinite recursion problem that
was
> fixed in version 3.1.
>
> Cheers.
>
> On Sat, 1 Jun 2002 08:44, Hugh Irvine wrote:
> > Salut Romain -
> >
> > Quel plaisir de pouvoir regarder le tennis a Roland-Garros en ce moment!
> >
> > :-)
> >
> > (ayant vecu a deux pas du stade ....)
> >
> > I have copied Mike on this mail, but have you tested the same scenario
with
> > Radiator 3.1? I know there has been some work done in this area.
> >
> > Please let me know if 3.1 fixes the problem.
> >
> > If not, please send me the config and trace 4 again for the new version.
> >
> > A+
> >
> > Hugues
> >
> > On Fri, 31 May 2002 20:58, Romain Vergniol wrote:
> > > Hello Hugh, Mike, everyone,
> > >
> > > I use Radiator 3.0 together with MySQL on Slackware 8.0 servers.
> > > I noticed that Radiator badly manages the loss of MySQL server ; in
> > > certain cases,
> > > Radiator consumes all available memory then crashes down.
> > >
> > > I set up a test server and it was easy to reproduce the problem, here
is
> > > the scenario :
> > >
> > >     - start MySQL server
> > >     - start Radiator
> > >     - start radpwtst with thousands iterations to have the time to do
our
> > > tests
> > >     - shutdown MySQL server
> > >
> > > At this moment, Radiator starts to consume all available memory until
it
> > > crashes (restarting Mysql server doesn't change anything...).
> > > I tried to increase or reduce 'FailureBackoffTime' but it doesn't seem
to
> > > change anything.
> > >
> > > Shutting down MySQL when no radius request is sent doesn't  reproduce
the
> > > problem.
> > >
> > > Is my config file OK ?
> > >
> > > Here is a trace 4 debug with the config file
> > >
> > >
> > > Romain Vergniol
> > > CEGEDIM
> > > France
> > >
> > > PS : Radiator is a great product !    :)
> > >
> > >
> > > ======================================================================
> > >
> > > Trace 4 debug:
> > >
> > > *** Received from 172.xxxxxxx port 1074 ....
> > > Code:       Access-Request
> > > Identifier: 219
> > > Authentic:  1234567890123456
> > > Attributes:
> > >         User-Name = "test_rvergniol at cegedim.net"
> > >         Service-Type = Framed
> > >         NAS-IP-Address = 172.xxxxxx
> > >         NAS-Port = 1234
> > >         Called-Station-Id = "0000"
> > >         Calling-Station-Id = "987654321"
> > >         NAS-Port-Type = Async
> > >         Framed-IP-Address = 172.xxxxxxx
> > >         User-Password =
> > > "<170>n<243>=<28><251><161><238>4<31><131>"B<18><249><132>"
> > >
> > > Fri May 31 11:50:46 2002: DEBUG: Handling request with Handler
> > > 'Request-Type=Access-Request'
> > > Fri May 31 11:50:46 2002: DEBUG:  Deleting session for
> > > test_rvergniol at cegedim.net, 172.xxxxxx, 1234
> > > Fri May 31 11:50:46 2002: DEBUG: Handling with Radius::AuthGROUP
> > > Fri May 31 11:50:46 2002: DEBUG: Handling with Radius::AuthSQL
> > > Fri May 31 11:50:46 2002: DEBUG: Handling with Radius::AuthSQL:
> > > Fri May 31 11:50:46 2002: DEBUG: Query is: select S.PASSWORD,
> > > S.CHECKATTR, S.REPLYATTR from SUBSCRIBERS as S, REL_PROFCOM as P where
> > > S.USERNAME='test_rvergniol at cegedim.net' and
S.NASIDENTIFIER='172.xxxxxx'
> > > and S.PROFCOM=P.ID and P.NUM='0000'
> > >
> > > Fri May 31 11:50:46 2002: DEBUG: Radius::AuthSQL looks for match with
> > > test_rvergniol at cegedim.net
> > > Fri May 31 11:50:46 2002: DEBUG: Expiration date converted to:
1798758000
> > > Fri May 31 11:50:46 2002: DEBUG: Radius::AuthSQL ACCEPT:
> > > Fri May 31 11:50:46 2002: DEBUG: Access accepted for
> > > test_rvergniol at cegedim.net
> > > Fri May 31 11:50:46 2002: DEBUG: Packet dump:
> > > *** Sending to 172.xxxxxxxxport 1074 ....
> > > Code:       Access-Accept
> > > Identifier: 219
> > > Authentic:  1234567890123456
> > > Attributes:
> > >         Framed-IP-Address = 172.xxxxxxxxx
> > >         Service-Type = Framed
> > >         cisco-avpair = "ip:dns-servers=xxxxxxx yyyyyyyyyyyyyy"
> > >         cisco-avpair = "ip:inacl#1=permit ip any 172.xxxxxxxxx/16"
> > >         cisco-avpair = "ip:inacl#2=permit ip any host
xxxxxxxxxxxxxxxx"
> > >         cisco-avpair = "ip:inacl#3=permit ip any host yyyyyyyyyyyy"
> > >         cisco-avpair = "ip:inacl#4=deny all any any"
> > >         Framed-Protocol = PPP
> > >
> > > Fri May 31 11:50:46 2002: DEBUG: Packet dump:
> > >
> > > #------------------------------------------------
> > > # Up to here, everything is OK
> > > # MySQL shuts down now
> > > #------------------------------------------------
> > >
> > >
> > > *** Received from 172.xxxxxxxxx port 1074 ....
> > > Code:       Accounting-Request
> > > Identifier: 220
> > > Authentic:
> > > <212><29><155>$\<246><202><193><141><206><12><148><15><138><210>Q
> > > Attributes:
> > >         User-Name = "test_rvergniol at cegedim.net"
> > >         Service-Type = Framed
> > >         NAS-IP-Address = 172.xxxxxxx
> > >         NAS-Port = 1234
> > >         NAS-Port-Type = Async
> > >         Acct-Session-Id = "00001400"
> > >         Acct-Status-Type = Start
> > >         Called-Station-Id = "0000"
> > >         Calling-Station-Id = "987654321"
> > >         Framed-IP-Address = 172xxxxxxx
> > >         Acct-Delay-Time = 0
> > >
> > > Fri May 31 11:50:46 2002: DEBUG: Handling request with Handler
> > > 'Request-Type=Accounting-Request'
> > > Fri May 31 11:50:46 2002: DEBUG:  Adding session for
> > > test_rvergniol at cegedim.net, 172.xxxxxx, 1234
> > > Fri May 31 11:50:46 2002: DEBUG: Handling with Radius::AuthSQL
> > > Fri May 31 11:50:46 2002: DEBUG: Handling accounting with
Radius::AuthSQL
> > > Fri May 31 11:50:46 2002: DEBUG: do query is: insert into ACCOUNTING
> > >                 (USERNAME, TIME_STAMP, ACCTSTATUSTYPE, ACCTDELAYTIME,
> > > ACCTSESSIONID, NASIDENTIFIER, FRAMEDIPADDRESS,
ASCEND_CALLED_STATION_ID,
> > > ASCEND_CALLING_STATION_ID, FORMATTED_DATE)
> > >                 values
> > >                 ('test_rvergniol at cegedim.net', 1022838646, 'Start', 0,
> > > '00001400', '172.xxxxxxxx', '172.xxxxxxx', '0000', '987654321',
> > > '2002-05-31.11.50.46')
> > >
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > Fri May 31 11:50:46 2002: ERR: do failed for 'insert into ACCOUNTING
> > >                 (USERNAME, TIME_STAMP, ACCTSTATUSTYPE, ACCTDELAYTIME,
> > > ACCTSESSIONID, NASIDENTIFIER, FRAMEDIPADDRESS,
ASCEND_CALLED_STATION_ID,
> > > ASCEND_CALLING_STATION_ID, FORMATTED_DATE)
> > >                 values
> > >                 ('test_rvergniol at cegedim.net', 1022838646, 'Start', 0,
> > > '00001400', '172.xxxxxx', '172.xxxxxxxx', '0000', '987654321',
> > > '2002-05-31.11.50.46')': MySQL server has gone away
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > DBD::mysql::db do failed: MySQL server has gone away at
> > > /usr/lib/perl5/site_perl/Radius/SqlDb.pm line 218.
> > > Out of memory!
> > > Attempt to free unreferenced scalar during global destruction.
> > >
> > >
> > >
=========================================================================
> > >==
> > >
> > > radius.cfg of the test server
> > >
> > > Foreground
> > > LogStdout
> > > LogDir          /usr/local/etc/radius
> > > DbDir           /usr/local/etc/radius
> > >
> > > PidFile /usr/local/etc/radius/radius.pid
> > >
> > > BindAddress 172.xxxxxxxxxxx
> > > AuthPort 1812
> > > AcctPort 1813
> > >
> > > <ClientListSQL>
> > >         DBSource        dbi:mysql:xxxxxxxxxxxxxx
> > >         DBUsername      xxxxxxxxxxxxx
> > >         DBAuth          xxxxxxxx
> > >         FailureBackoffTime 20
> > >
> > > </ClientListSQL>
> > >
> > > <Client DEFAULT>
> > > Secret xxxxxxxxxxxxxxx
> > > </Client>
> > >
> > > #<AddressAllocator SQL>
> > > #       Identifier SQLAllocator
> > > #
> > > #       DBSource        dbi:mysql:xxxxxxxxxxxxxx
> > > #        DBUsername      xxxxxxxxxxxxx
> > > #        DBAuth          xxxxxxxx
> > > #       FailureBackoffTime 5
> > > #
> > > #       #DefaultLeasePeriod     86400
> > > #       #LeaseReclaimInterval 86400
> > > #       FindQuery select TIME_STAMP, YIADDR, SUBNETMASK, DNSSERVER
from
> > > RADPOOL where POOL='%0' and STATE=0 limit 1
> > > #
> > > #       AllocateQuery update RADPOOL set STATE=1, TIME_STAMP=%0,
> > > EXPIRY=%1, USERNAME='%2' where YIADDR='%3'
> > > #
> > > #       DeallocateQuery update RADPOOL set STATE=0, TIME_STAMP=%t
where
> > > YIADDR='%0'
> > > #
> > > #</AddressAllocator>
> > >
> > > <AuthBy GROUP>
> > >         Identifier AUTH_USER
> > > #       AuthByPolicy ContinueWhileAccept
> > >         <AuthBy SQL>
> > >                 Identifier AUTH_USER
> > >
> > >                 DBSource        dbi:mysql:xxxxxxxxxxxxxx
> > >             DBUsername      xxxxxxxxxxxxx
> > >             DBAuth          xxxxxxxx
> > >
> > >                 FailureBackoffTime 20
> > >
> > >                 AuthSelect select PASSWORD, CHECKATTR, REPLYATTR from
> > > SUBSCRIBERS where USERNAME='%n' limit 1
> > >                 AuthColumnDef 0, User-Password, check
> > >                 AuthColumnDef 1, GENERIC, check
> > >                 AuthColumnDef 2, GENERIC, reply
> > >
> > >         </AuthBy SQL>
> > > #       <AuthBy DYNADDRESS>
> > > #                Allocator       SQLAllocator
> > > #                #MapAttribute   yiaddr, Framed-IP-Address
> > > #                #MapAttribute   subnetmask, Framed-IP-Netmask
> > > #                StripFromReply  PoolHint
> > > #        </AuthBy>
> > > </AuthBy>
> > >
> > >
> > > <AuthBy GROUP>
> > >         Identifier AUTH_USER_NAS
> > > #        AuthByPolicy ContinueWhileAccept
> > >         <AuthBy SQL>
> > >              DBSource        dbi:mysql:xxxxxxxxxxxxxx
> > >             DBUsername      xxxxxxxxxxxxx
> > >             DBAuth          xxxxxxxx
> > >
> > >                 FailureBackoffTime 20
> > >
> > >                 #AuthSelect select PASSWORD, CHECKATTR, REPLYATTR from
> > > SUBSCRIBERS where USERNAME='%n' and NASIDENTIFIER='%N'
> > >
> > >                 AuthSelect select S.PASSWORD, S.CHECKATTR, S.REPLYATTR
\
> > >                 from SUBSCRIBERS as S, REL_PROFCOM as P \
> > >                 where S.USERNAME='%n' and S.NASIDENTIFIER='%N' \
> > >                 and S.PROFCOM=P.ID \
> > >                 and P.NUM='%{Called-Station-Id}'
> > >
> > >                 AuthColumnDef 0, User-Password, check
> > >                 AuthColumnDef 1, GENERIC, check
> > >                 AuthColumnDef 2, GENERIC, reply
> > >
> > >         </AuthBy SQL>
> > > #        <AuthBy DYNADDRESS>
> > > #                Allocator       SQLAllocator
> > > #               #MapAttribute   yiaddr, Ascend-Assign-IP-Pool
> > > #                MapAttribute   yiaddr, Framed-IP-Address
> > > #                MapAttribute   subnetmask, Framed-IP-Netmask
> > > #                StripFromReply  PoolHint
> > > #        </AuthBy>
> > > </AuthBy>
> > >
> > >
> > > <AuthBy GROUP>
> > >         Identifier AUTH_USER_NAS_CSID
> > >         NoDefault
> > >         AuthByPolicy ContinueWhileAccept
> > >         <AuthBy SQL>
> > >                DBSource        dbi:mysql:xxxxxxxxxxxxxx
> > >             DBUsername      xxxxxxxxxxxxx
> > >             DBAuth          xxxxxxxx
> > >
> > >                 FailureBackoffTime 20
> > >
> > >                 AuthSelect select S.PASSWORD, S.CHECKATTR, S.REPLYATTR
\
> > >                 from SUBSCRIBERS as S, REL_PROFCOM as P \
> > >                 where S.USERNAME='%n' and S.NASIDENTIFIER='%N' \
> > >                 and S.PROFCOM=P.ID \
> > >                 and P.NUM='%{Called-Station-Id}'
> > >
> > >                 AuthColumnDef 0, User-Password, check
> > >                 AuthColumnDef 1, GENERIC, check
> > >                 AuthColumnDef 2, GENERIC, reply
> > >
> > >          </AuthBy>
> > >
> > > </AuthBy>
> > >
> > >
> > >
> > >
> > > <AuthBy SQL>
> > >
> > >         Identifier ACCT
> > >
> > >         DBSource        dbi:mysql:xxxxxxxxxxxxxx
> > >         DBUsername      xxxxxxxxxxxxx
> > >         DBAuth          xxxxxxxx
> > >
> > >
> > >         AccountingTable ACCOUNTING
> > >         AcctColumnDef   USERNAME,User-Name
> > >         AcctColumnDef   REALM,User-Realm
> > >         AcctColumnDef   TIME_STAMP,Timestamp,integer
> > >         AcctColumnDef   ACCTSTATUSTYPE,Acct-Status-Type
> > >         AcctColumnDef   ACCTDELAYTIME,Acct-Delay-Time,integer
> > >         AcctColumnDef   ACCTINPUTOCTETS,Acct-Input-Octets,integer
> > >         AcctColumnDef   ACCTOUTPUTOCTETS,Acct-Output-Octets,integer
> > >         AcctColumnDef   ACCTSESSIONID,Acct-Session-Id
> > >         AcctColumnDef   ACCTSESSIONTIME,Acct-Session-Time,integer
> > >         AcctColumnDef
> > > ASCEND_DISCONNECT_CAUSE,Ascend-Disconnect-Cause,integer
> > >         AcctColumnDef   ACCTTERMINATECAUSE,Acct-Terminate-Cause
> > >         AcctColumnDef   NASIDENTIFIER,%N,formatted
> > >         AcctColumnDef   FRAMEDIPADDRESS,Framed-IP-Address
> > >         AcctColumnDef
> > > ASCEND_CONNECT_PROGRESS,Ascend-Connect-Progress,integer
> > >         AcctColumnDef   ASCEND_DATA_RATE,Ascend-Data-Rate,integer
> > >         AcctColumnDef
> > > ASCEND_PRE_INPUT_OCTETS,Ascend-Pre-Input-Octets,integer
> > >         AcctColumnDef
> > > ASCEND_PRE_OUTPUT_OCTETS,Ascend-Pre-Output-Octets,integer
> > >         AcctColumnDef
> > > ASCEND_PRESESSION_TIME,Ascend-PreSession-Time,integer
> > >         AcctColumnDef   ASCEND_XMIT_RATE,Ascend-Xmit-Rate,integer
> > >         AcctColumnDef   ASCEND_CALLED_STATION_ID,Called-Station-Id
> > >         AcctColumnDef   ASCEND_CALLING_STATION_ID,Calling-Station-Id
> > >         AcctColumnDef   FRAMED_PROTOCOL,Framed-Protocol,integer
> > >         AcctColumnDef   FORMATTED_DATE,%Y-%m-%d.%H.%M.%S,formatted
> > >
> > >         AcctFailedLogFileName /usr/local/etc/radius/missedaccounting
> > >
> > >
> > > </AuthBy>
> > >
> > > <AuthLog SQL>
> > >         Identifier AUTHLOG
> > >
> > >         DBSource        dbi:mysql:xxxxxxxxxxxxx
> > >         DBUsername      xxxxxxxxx
> > >         DBAuth          xxxxxxxxxxxxxxxxx
> > >         FailureBackoffTime 20
> > >
> > >         LogSuccess 1
> > >         LogFailure 1
> > >         Table RADAUTHLOG_1
> > >         SuccessQuery INSERT into RADAUTHLOG_1 values("%t","%n" , "%N"
,
> > > "Radius2: OK. %1" )
> > >         FailureQuery INSERT into RADAUTHLOG_1 values("%t","%n" , "%N"
,
> > > "Radius2: NOK. %1(C-S-Id:%{Called-Station-Id})" )
> > > </AuthLog>
> > >
> > > # Scripts de Fail-Over
> > > <Handler Request-Type=Access-Request,Realm=test.fr>
> > >         AuthBy AUTH_USER
> > > </Handler>
> > >
> > > # TEST
> > > <Handler Request-Type=Access-Request,Realm=test.cegedim>
> > >         #AuthBy AUTH_USER_NAS
> > >         AuthBy AUTH_USER_NAS_CSID
> > >         AuthLog AUTHLOG
> > > </Handler>
> > >
> > > #####################################################
> > > # OTHER.USERS
> > > ####################################################
> > >
> > >
> > > <Handler Request-Type=Accounting-Request>
> > >         AuthBy ACCT
> > > </Handler>
> > >
> > > <Handler Request-Type=Access-Request,Realm=teams.bak>
> > >         RejectHasReason
> > >         AuthBy AUTH_USER
> > >         AuthLog AUTHLOG
> > > </Handler>
> > >
> > >
> > > <Handler Request-Type=Access-Request>
> > >         RejectHasReason
> > >         AuthBy AUTH_USER_NAS_CSID
> > >         AuthLog AUTHLOG
> > > </Handler>
> > >
> > > ######################################################
> > >
> > >
> > >
> > > <Log SQL>
> > >         DBSource        dbi:mysql:xxxxxxxxxxxxxx
> > >         DBUsername      xxxxxxxxxxxxx
> > >         DBAuth          xxxxxxxx
> > >         FailureBackoffTime 20
> > >
> > >         Trace 3
> > > </Log>
> > >
> > > ===
> > > Archive at http://www.open.com.au/archives/radiator/
> > > Announcements on radiator-announce at open.com.au
> > > To unsubscribe, email 'majordomo at open.com.au' with
> > > 'unsubscribe radiator' in the body of the message.
>
> --
> Mike McCauley                               mikem at open.com.au
> Open System Consultants Pty. Ltd            Unix, Perl, Motif, C++, WWW
> 24 Bateman St Hampton, VIC 3188 Australia   http://www.open.com.au
> Phone +61 3 9598-0985                       Fax   +61 3 9598-0955
>
> Radiator: the most portable, flexible and configurable RADIUS server
> anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
> Platypus, Freeside, TACACS+, PAM, external, Active Directory etc etc
> on Unix, Win95/8, 2000, NT, MacOS 9, MacOS X etc etc
>
> ===
> Archive at http://www.open.com.au/archives/radiator/
> Announcements on radiator-announce at open.com.au
> To unsubscribe, email 'majordomo at open.com.au' with
> 'unsubscribe radiator' in the body of the message.

===
Archive at http://www.open.com.au/archives/radiator/
Announcements on radiator-announce at open.com.au
To unsubscribe, email 'majordomo at open.com.au' with
'unsubscribe radiator' in the body of the message.


More information about the radiator mailing list