<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Helvetica Neue", sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
hello</div>
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Helvetica Neue", sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Helvetica Neue", sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
i have the following setup</div>
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Helvetica Neue", sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Helvetica Neue", sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<Handler Suffix = @realm, NAS-Identifier = sshd, NAS-Port-Type = Virtual, Service-Type = Authenticate-Only>
<div> <AuthBy LDAP2></div>
<div> Identifier PROXY_realm</div>
<div> Host realm.example.com</div>
Port 636</div>
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Helvetica Neue", sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
BaseDN OU=realm,DC=example,DC=com
<div> SearchFilter (cn=%1)</div>
<div> ServerChecksPassword</div>
<div> Version 3</div>
<div> </AuthBy></div>
</Handler><br>
</div>
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Helvetica Neue", sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Helvetica Neue", sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
now security requires a client certificate (which i have) to authenticate the ldap connection</div>
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Helvetica Neue", sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Helvetica Neue", sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
how would i configure the certificate for authenticating the ldap connection ?<br>
</div>
<div>
<div style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Helvetica Neue", sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature">
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)" class="elementToProof">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><span style="font-family:"Century Gothic",sans-serif;font-size:10pt">Yours sincerely</span></span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="font-family:Arial,Helvetica,sans-serif"></span>
<div>
<div>
<div>
<div>
<div dir="ltr">
<div>
<div dir="ltr" style="font-family:Arial,Helvetica,sans-serif;font-size:10pt">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr" style="font-size:10.5pt">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr"><span><br>
</span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<span style="font-family:"Century Gothic",sans-serif; font-size:10pt">Alfred Reibenschuh</span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div>
<div>
<div>
<div>
<div dir="ltr">
<div>
<div dir="ltr" style="font-family:Arial,Helvetica,sans-serif; font-size:10pt">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr" style="font-size:10.5pt">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr"><br>
<span style="font-family:"Century Gothic",sans-serif; font-size:10pt"></span></div>
<span dir="ltr"><span style="font-family:"Century Gothic",sans-serif;font-size:10pt"># Network System Management Platform Lead<br>
</span></span>
<div dir="ltr"><span style="font-family:"Century Gothic",sans-serif;font-size:10pt"># Network Device Backup & Restore Process Manager<br>
</span></div>
<span dir="ltr"></span>
<div dir="ltr"><br>
</div>
<span style="font-family:"Century Gothic",sans-serif; font-size:10pt">Value Transformation Services GmbH</span><span style="font-family:Arial,Helvetica,sans-serif"></span><br>
<span style="font-family:Arial,Helvetica,sans-serif"></span>
<div dir="ltr"><span style="font-family:Arial,Helvetica,sans-serif"></span><span style="font-family:"Century Gothic",sans-serif; font-size:10pt">An IBM / KYNDRYL Company</span><span style="font-family:Arial,Helvetica,sans-serif"><br>
</span><span style="font-family:"Century Gothic",sans-serif; font-size:10pt">Obere Donaustrasse 95</span><span style="font-family:Arial,Helvetica,sans-serif"><br>
</span><span style="font-family:"Century Gothic",sans-serif; font-size:10pt">1020 Wien</span></div>
<div dir="ltr"><br>
<span style="font-family:"Century Gothic",sans-serif; font-size:10pt">Phone: +43-1-2056320-143</span><span style="font-family:Arial,Helvetica,sans-serif"><br>
</span><span style="font-family:"Century Gothic",sans-serif; font-size:10pt">Mobile: +43-664-3523820</span></div>
<div dir="ltr"><span style="font-family:Arial,Helvetica,sans-serif"><u><a href="mailto:Alfred.Reibenschuh_v-tservices@at.ibm.com" tabindex="-1"><span style="font-family:"Century Gothic",sans-serif; font-size:10pt">mail: Alfred.Reibenschuh@kyndryl.com</span></a></u><br>
</span><span style="font-family:"Century Gothic",sans-serif; font-size:10pt">webex:
</span><span style="font-family:Arial,Helvetica,sans-serif"><span><span><span tabindex="0" title="https://kyndryl.webex.com/meet/alfred.reibenschuh"><a href="https://kyndryl.webex.com/meet/alfred.reibenschuh"><span style="font-family:"Century Gothic",sans-serif; font-size:10pt">https://kyndryl.webex.com/meet/alfred.reibenschuh</span></a></span></span></span><br>
<br>
</span><span style="font-family:"Century Gothic",sans-serif; font-size:10pt">Please consider the environment before printing this e-mail.</span><span style="font-family:Arial,Helvetica,sans-serif"><br>
<br>
</span><span style="font-family:"Century Gothic",sans-serif; font-size:10pt">This e-mail is confidential and may also contain privileged information. If you are not the intended recipient you are not authorized to read, print, save, process or disclose this
message. If you have received this message by mistake, please inform the sender immediately and delete this e-mail, its attachments and any copies.</span><span style="font-family:Arial,Helvetica,sans-serif"><br>
</span><span style="font-family:"Century Gothic",sans-serif; font-size:10pt">Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur penalties.</span><br>
<span style="font-family:"Century Gothic",sans-serif; font-size:10pt"> </span></div>
<div dir="ltr"><span style="font-family:"Century Gothic",sans-serif; font-size:10pt">Thank you!</span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> radiator <radiator-bounces@lists.open.com.au> on behalf of radiator-request@lists.open.com.au <radiator-request@lists.open.com.au><br>
<b>Sent:</b> Thursday, April 7, 2022 2:00 PM<br>
<b>To:</b> radiator@lists.open.com.au <radiator@lists.open.com.au><br>
<b>Subject:</b> [EXTERNAL] radiator Digest, Vol 155, Issue 4</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">Send radiator mailing list submissions to<br>
radiator@lists.open.com.au<br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="https://lists.open.com.au/mailman/listinfo/radiator">https://lists.open.com.au/mailman/listinfo/radiator</a>
<br>
or, via email, send a message with subject or body 'help' to<br>
radiator-request@lists.open.com.au<br>
<br>
You can reach the person managing the list at<br>
radiator-owner@lists.open.com.au<br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of radiator digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: Memory Leak on RHEL 8.5 (Heikki Vatiainen)<br>
2. Re: Memory Leak on RHEL 8.5 (Wolfgang Breyha)<br>
3. Re: Memory Leak on RHEL 8.5 (Wolfgang Breyha)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Wed, 6 Apr 2022 19:27:59 +0300<br>
From: Heikki Vatiainen <hvn@open.com.au><br>
To: radiator@lists.open.com.au<br>
Subject: Re: [RADIATOR] Memory Leak on RHEL 8.5<br>
Message-ID: <f2e72eb1-3d39-9ec3-e9a1-ed4b4d41c5d3@open.com.au><br>
Content-Type: text/plain; charset=UTF-8; format=flowed<br>
<br>
On 6.4.2022 14.17, Wolfgang Breyha wrote:<br>
<br>
> That proves that Rocky/Alma are indeed "100% bug-for-bug" compatible;-)<br>
<br>
Indeed, and fortunately adds possibility there's a way to find the <br>
reason. Unfortunately it seems not to be that easy. I used <br>
openssl-1.1.1k-5.el8_5.src.rpm and tried to compile a local copy. After <br>
applying the patches and compiling with parameters from openssl.spec, I <br>
was able to create a version that works.<br>
<br>
Problem is that it works too well. It does not leak. Compiler flags, see <br>
openssl version -a, match the system version and everything looks equal <br>
(except of built date, etc. changes that are expected).<br>
<br>
It might be that it's not a problem caused by patches the source rpm <br>
contains, but something else. The build I did was manual done by calling <br>
./Configure ... after patching the OpenSSL source.<br>
<br>
Stefan suggested raising a ticket at RedHat. Even if it may not be <br>
patches, at least we know have more information.<br>
<br>
> As a first step I tried to reduce my test config to yours. But this raised<br>
> some questions...<br>
> <br>
>>> I then start eapol_test (from wpa_supplicant RPM) with a config of<br>
>>> network={<br>
>>> eap=PEAP<br>
>>> eapol_flags=0<br>
>>> key_mgmt=IEEE8021X<br>
>>> identity="testuser"<br>
>>> anonymous_identity="anonymous"<br>
>>> password="testpass"<br>
>>> ca_cert="/etc/pki/tls/cert.pem"<br>
>>> phase2="auth=MSCHAPV2"<br>
>>> }<br>
>>> in a loop and can watch radiusd eating memory.<br>
>><br>
>> I used exactly the same config with my testing. I even used eapol_test that<br>
>> comes with 'yum install wpa_supplicant', but I don't think eapol_test<br>
>> version matters.<br>
> <br>
> Did you really use this unmodified and if yes, was cert.pem the system file<br>
> our the test CA? I was not able to successfully AUTH without the test CA here.<br>
<br>
Good catch. I did update certificates in eapol_test conf file to use <br>
test certificates that come with Radiator:<br>
<br>
network={<br>
eap=PEAP<br>
eapol_flags=0<br>
key_mgmt=IEEE8021X<br>
identity="testuser"<br>
anonymous_identity="anonymous"<br>
password="testpass"<br>
#ca_cert="/etc/pki/tls/cert.pem"<br>
ca_cert="demoCA/cacert.pem"<br>
phase2="auth=MSCHAPV2"<br>
}<br>
<br>
File key.pem in Radiator configuration, as you correclty noted, contains <br>
the decrypted private key from cert-srv.pem. I decrypted it to avoid <br>
extra OpenSSL API calls that would otherwise be needed to decrypt the <br>
key from within Radiator. In other words, the configuration shouldn't <br>
have anything special in it.<br>
<br>
>> ??? EAPTLS_PrivateKeyFile %D/key.pem<br>
> <br>
> I assumed that this is a copy of the key in crt-serv.pem without<br>
> passphrase. Otherwise radiusd complains about the key and can't do TLS<br>
> handshakes at all.<br>
<br>
Yes, that's exactly correct. Here's the OpenSSL related part of Radiator <br>
configuration (%D expands to .):<br>
<br>
<AuthBy FILE><br>
Identifier AuthTEST<br>
Filename %D/users<br>
EAPType PEAP,MSCHAP-V2<br>
EAPTLS_CAFile %D/demoCA/cacert.pem<br>
EAPTLS_CertificateFile %D/cert-srv.pem<br>
EAPTLS_PrivateKeyFile %D/key.pem<br>
EAPTLS_CertificateType PEM<br>
EAPTLS_MaxFragmentSize 1000<br>
EAPTLS_SessionResumption 0<br>
AutoMPPEKeys<br>
</AuthBy><br>
<br>
Radiator and eapol_test were both run from the same directory. I also <br>
turned off session resumption so that there's no need to maintain <br>
session cache and Radius related context information.<br>
<br>
> With these changes I'm able to use eapol_test successfully and the leaks<br>
> occur fast enough. And valgrind reports a lot of leaks in SSL_context.<br>
<br>
Is that a new location or are the reports from the same two places as <br>
earlier?<br>
<br>
> I'm not that experienced using valgrind and did just what most "how-to"s<br>
> suggest;-)<br>
> <br>
> I'm using the RHEL8 valgrind RPM and start radiusd with:<br>
> # valgrind --log-file=/tmp/val.log --leak-check=yes perl /opt/radiator<br>
> /radiator/radiusd -foreground -no_pid_file -config_file leak_test.cfg<br>
> <br>
> Then I call eapol_test in a bash for loop 1..1000. After stopping radiusd<br>
> val.log contains several references to SSL_, X509, ASN1_.<br>
<br>
Hmm, multiple different places. Thanks for the quick start guide. I'll <br>
see if I can get something useful too.<br>
<br>
Thanks,<br>
heikki<br>
<br>
-- <br>
Heikki Vatiainen<br>
OSC, makers of Radiator<br>
Visit radiatorsoftware.com for Radiator AAA server software<br>
<br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Thu, 7 Apr 2022 01:15:40 +0200<br>
From: Wolfgang Breyha <radiator@blafasel.at><br>
To: radiator@lists.open.com.au<br>
Subject: Re: [RADIATOR] Memory Leak on RHEL 8.5<br>
Message-ID: <8b7953af-4f2c-9a8b-ee54-510c0308a43a@breyha.at><br>
Content-Type: text/plain; charset=UTF-8<br>
<br>
Hi!<br>
<br>
A question left from your previous answer:<br>
Which version does el9 beta use? I see they switched to OpenSSL 3 already<br>
searching the net, right?<br>
<br>
On 06/04/2022 18:27, Heikki Vatiainen wrote:<br>
> Unfortunately it seems not to be that easy. I used<br>
> openssl-1.1.1k-5.el8_5.src.rpm and tried to compile a local copy. After<br>
> applying the patches and compiling with parameters from openssl.spec, I was<br>
> able to create a version that works.<br>
> <br>
> Problem is that it works too well. It does not leak. Compiler flags, see<br>
> openssl version -a, match the system version and everything looks equal<br>
> (except of built date, etc. changes that are expected).<br>
<br>
Really? That's interesting, because meanwhile I tried the following things:<br>
*) rebuild 1.1.1k from el8 src RPM<br>
*) rebuild the current openssl-1.1.1n-1.fc35.src.rpm for el8<br>
*) remove several patches from the 1.1.1n spec in steps to the point of<br>
nearly destroying the host ending up with broken ssh/dnf due to missing<br>
krb5_evp symbols. Luckily wget still worked with https to fetch the<br>
original RPMS;-)<br>
<br>
But even then ... no change in memory leaks for radiusd.<br>
<br>
*) tried to build the rpm with all patches but with the "unhobbled" full<br>
openssl-1.1.1n tarball.<br>
<br>
No difference as well. It leaks.<br>
<br>
I also tried another tool for memleak detection called memleax ...<br>
<a href="https://github.com/WuBingzheng/memleax">https://github.com/WuBingzheng/memleax</a>
<br>
... which is able to attach to a running PID. This detects the same leak as<br>
well:<br>
CallStack[24]: may-leak=6 (1536 bytes)<br>
expired=6 (1536 bytes), free_expired=0 (0 bytes)<br>
alloc=33 (8448 bytes), free=0 (0 bytes)<br>
freed memory live time: min=0 max=0 average=0<br>
un-freed memory live time: max=11<br>
0x00007fbbc4301a90 libc-2.28.so __malloc()+0<br>
0x00007fbbc211b90d libcrypto.so CRYPTO_zalloc()+13<br>
0x00007fbbc2107ac4 libcrypto.so EVP_PKEY_meth_new()+36<br>
0x00007fbbc154ead8 pkcs11.so<br>
0x00007fbbc20e98e5 libcrypto.so ENGINE_get_pkey_meth()+53<br>
0x00007fbbc2107ea5 libcrypto.so<br>
0x00007fbbc2103544 libcrypto.so<br>
0x00007fbbc24d7a42 libssl.so<br>
0x00007fbbc24ca33f libssl.so<br>
0x00007fbbc24b5c98 libssl.so SSL_do_handshake()+88<br>
0x00007fbbc2767e84 SSLeay.so<br>
0x00007fbbc55144b9 libperl.so Perl_pp_entersub()+505<br>
0x00007fbbc550c325 libperl.so Perl_runops_standard()+53<br>
0x00007fbbc548bffd libperl.so perl_run()+797<br>
0x0000562b6f5f7eda perl<br>
0x00007fbbc429f493 libc-2.28.so __libc_start_main()+243<br>
0x0000562b6f5f7f1e perl<br>
<br>
<br>
Then I used my locally built openssl 1.1.1n RPMs I use on other el8 hosts.<br>
These use a clean openssl source without RH patches, slightly different<br>
Configure and builds to /usr/local/openssl. I then rebuilt the Net::SSLeay<br>
1.92 RPM I already built before with OPENSSL_PREFIX to use my custom OpenSSL.<br>
<br>
This way I can easily switch between original and custom OpenSSL by<br>
downgrading/updating Net::SSLeay.<br>
<br>
And with the custom OpenSSL RPMs it seems radiusd doesn't leak. At least<br>
valgrind/memleax do not detect the SSL_do_handshake leak and radiusd memory<br>
footprint gets stable after the first batch of requests floating at<br>
~110-120MB. I put this setup on one of our production hosts now to see what<br>
happens as soon as people start connecting their WLAN devices in the morning.<br>
<br>
> It might be that it's not a problem caused by patches the source rpm<br>
> contains, but something else. The build I did was manual done by calling<br>
> ./Configure ... after patching the OpenSSL source.<br>
<br>
Hmmm, I can't confirm this since all the RPMs I used in my tests are built<br>
on Rocky 8.5 as well. The Fedora 35 SPEC doesn't differ in any important<br>
aspect.<br>
<br>
I rebuilt 1.1.1k with the RHEL8 spec. So, this should be the exact same<br>
result as your local build except the rpmbuild environment (which sets a<br>
lot of compiler/linker defaults which you compared).<br>
<br>
But only the version without the RH hobble/fips/patches-stuff works which<br>
uses the same rpmbuild env and the SPEC file I use is a modification of the<br>
original one.<br>
<br>
So my guess for now is that the issue is caused by the patches (but not the<br>
hobbled part). Since wide parts of the patches depend on each other I see<br>
no easy way to pin it to one of them.<br>
<br>
Greetings, Wolfgang<br>
<br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Thu, 7 Apr 2022 12:57:47 +0200<br>
From: Wolfgang Breyha <radiator@blafasel.at><br>
To: radiator@lists.open.com.au<br>
Subject: Re: [RADIATOR] Memory Leak on RHEL 8.5<br>
Message-ID: <eb208f6b-9bf0-a1fa-617c-5a6a139d2749@breyha.at><br>
Content-Type: text/plain; charset=UTF-8<br>
<br>
On 07/04/2022 01:15, Wolfgang Breyha wrote:<br>
> And with the custom OpenSSL RPMs it seems radiusd doesn't leak. At least<br>
> valgrind/memleax do not detect the SSL_do_handshake leak and radiusd memory<br>
> footprint gets stable after the first batch of requests floating at<br>
> ~110-120MB. I put this setup on one of our production hosts now to see what<br>
> happens as soon as people start connecting their WLAN devices in the morning.<br>
<br>
Meanwhile I can confirm that my custom RPMs do not leak and radiusd memory<br>
usage only depends on the load it has to handle.<br>
<br>
So I filed:<br>
<a href="https://bugzilla.redhat.com/show_bug.cgi?id=2072962">https://bugzilla.redhat.com/show_bug.cgi?id=2072962</a>
<br>
<br>
Greetings, Wolfgang<br>
-- <br>
Wolfgang Breyha <wbreyha@gmx.net> | <a href="https://www.blafasel.at/">https://www.blafasel.at/</a>
<br>
Vienna University Computer Center | Austria<br>
<br>
<br>
------------------------------<br>
<br>
Subject: Digest Footer<br>
<br>
_______________________________________________<br>
radiator mailing list<br>
radiator@lists.open.com.au<br>
<a href="https://lists.open.com.au/mailman/listinfo/radiator">https://lists.open.com.au/mailman/listinfo/radiator</a>
<br>
<br>
------------------------------<br>
<br>
End of radiator Digest, Vol 155, Issue 4<br>
****************************************<br>
</div>
</span></font></div>
<DIV>
Unless stated otherwise above:<BR>
Kyndryl Austria GmbH<BR>
Sitz: Wien<BR>
Firmenbuchgericht: Handelsgericht Wien, FN 554717 k<BR>
</DIV></body>
</html>