[RADIATOR] Memory Leak on RHEL 8.5

[Heikki Vatiainen]

On 6.4.2022 14.17, Wolfgang Breyha wrote:

> That proves that Rocky/Alma are indeed "100% bug-for-bug" compatible;-)

Indeed, and fortunately adds possibility there's a way to find the 
reason. Unfortunately it seems not to be that easy. I used 
openssl-1.1.1k-5.el8_5.src.rpm and tried to compile a local copy. After 
applying the patches and compiling with parameters from openssl.spec, I 
was able to create a version that works.

Problem is that it works too well. It does not leak. Compiler flags, see 
openssl version -a, match the system version and everything looks equal 
(except of built date, etc. changes that are expected).

It might be that it's not a problem caused by patches the source rpm 
contains, but something else. The build I did was manual done by calling 
./Configure ... after patching the OpenSSL source.

Stefan suggested raising a ticket at RedHat. Even if it may not be 
patches, at least we know have more information.

> As a first step I tried to reduce my test config to yours. But this raised
> some questions...
> 
>>> I then start eapol_test (from wpa_supplicant RPM) with a config of
>>> network={
>>> eap=PEAP
>>> eapol_flags=0
>>> key_mgmt=IEEE8021X
>>> identity="testuser"
>>> anonymous_identity="anonymous"
>>> password="testpass"
>>> ca_cert="/etc/pki/tls/cert.pem"
>>> phase2="auth=MSCHAPV2"
>>> }
>>> in a loop and can watch radiusd eating memory.
>>
>> I used exactly the same config with my testing. I even used eapol_test that
>> comes with 'yum install wpa_supplicant', but I don't think eapol_test
>> version matters.
> 
> Did you really use this unmodified and if yes, was cert.pem the system file
> our the test CA? I was not able to successfully AUTH without the test CA here.

Good catch. I did update certificates in eapol_test conf file to use 
test certificates that come with Radiator:

network={
eap=PEAP
eapol_flags=0
key_mgmt=IEEE8021X
identity="testuser"
anonymous_identity="anonymous"
password="testpass"
#ca_cert="/etc/pki/tls/cert.pem"
ca_cert="demoCA/cacert.pem"
phase2="auth=MSCHAPV2"
}

File key.pem in Radiator configuration, as you correclty noted, contains 
the decrypted private key from cert-srv.pem. I decrypted it to avoid 
extra OpenSSL API calls that would otherwise be needed to decrypt the 
key from within Radiator. In other words, the configuration shouldn't 
have anything special in it.

>>      EAPTLS_PrivateKeyFile %D/key.pem
> 
> I assumed that this is a copy of the key in crt-serv.pem without
> passphrase. Otherwise radiusd complains about the key and can't do TLS
> handshakes at all.

Yes, that's exactly correct. Here's the OpenSSL related part of Radiator 
configuration (%D expands to .):

<AuthBy FILE>
     Identifier AuthTEST
     Filename %D/users
     EAPType PEAP,MSCHAP-V2
     EAPTLS_CAFile %D/demoCA/cacert.pem
     EAPTLS_CertificateFile %D/cert-srv.pem
     EAPTLS_PrivateKeyFile %D/key.pem
     EAPTLS_CertificateType PEM
     EAPTLS_MaxFragmentSize 1000
     EAPTLS_SessionResumption 0
     AutoMPPEKeys
</AuthBy>

Radiator and eapol_test were both run from the same directory. I also 
turned off session resumption so that there's no need to maintain 
session cache and Radius related context information.

> With these changes I'm able to use eapol_test successfully and the leaks
> occur fast enough. And valgrind reports a lot of leaks in SSL_context.

Is that a new location or are the reports from the same two places as 
earlier?

> I'm not that experienced using valgrind and did just what most "how-to"s
> suggest;-)
> 
> I'm using the RHEL8 valgrind RPM and start radiusd with:
> # valgrind --log-file=/tmp/val.log --leak-check=yes perl /opt/radiator
> /radiator/radiusd -foreground -no_pid_file -config_file leak_test.cfg
> 
> Then I call eapol_test in a bash for loop 1..1000. After stopping radiusd
> val.log contains several references to SSL_, X509, ASN1_.

Hmm, multiple different places. Thanks for the quick start guide. I'll 
see if I can get something useful too.

Thanks,
heikki

-- 
Heikki Vatiainen
OSC, makers of Radiator
Visit radiatorsoftware.com for Radiator AAA server software