[RADIATOR] Unicode in nthash passwords

[Heikki Vatiainen]

On 5.5.2026 15.43, Hosek, Ondrej via radiator wrote:

> On Wed, 2026-04-15 at 10:35 +0000, Hosek, Ondrej via radiator wrote:
>> The easiest option would be to rewrite ASCIItoUnicode to
>> interpret its argument as UTF-8 instead of ISO 8859-1, but this is
>> guaranteed to break compatibility with existing setups.

It might, I agree. The RADIUS RFCs use UTF-8 for Text data type, but do 
not explictily require UTF-8 with User-Password. Even if it were 
required, the various clients may accept any input from the user.

The server of course has to be very careful handling the input. Encoding 
mismatches will trigger authentication failures even if the password is 
good.

The good point with the current method is that it's very simple since it 
does encoding octet by octet. When accepting octets that are supposed to 
be encoded as UTF-8 and need to be converted to UTF-16LE encoding, the 
server may find that the input isn't UTF-8 and has to recover from that. 
That may be surprising for the user since they know that their password 
is correct. The encoding confusion may happen, for example, if there's a 
captive portal for network access or some other system that doesn't 
accept UTF-8.

These are likely not common these days, but I think I've seen non-UTF-8 
from WLAN captive portals. Localisation settings especially may cause 
this, if I remember correctly.

>> A more flexible alternative would be to allow administrators to
>> specify the encoding of incoming passwords for the purpose of NT
>> hashing; I am not sure whether this option would be best placed on
>> a <Handler>, <Client>, <AuthBy> or different level, but if the
>> default is kept ISO 8859-1 and can be reconfigured to UTF-8, this
>> will probably solve the issue.

That's a good idea, it can be made configurable. A small correction: The 
current behaviour is to convert octets, a binary stream, which includes 
octets not defined in 8859-1. One option to get started could simply be 
to make it a global option.

> Can I perhaps get input from a Radiator Software developer on this?

Sorry for taking this long. I was away in April and have been looking at 
this now. I did some prototyping to look for various encoding options 
and I'm considering using the modules that come with Perl itself.

What I'm considering is to use Encode to see that the supposed UTF-8 
encoding is correct. If it is, use Unicode::Normalize to transform the 
UTF-8 encoding with Normalization Form Canonical Decomposition (NFD) 
[1]. The octets would then be ready for UTF-16LE encoding and further 
hashing.

Encoding errors would likely need logging. This would help 
troubleshooting the cases when the password is correct but the encoding 
is not UTF-8, or whatever the expected encoding is.

Normalisation is needed, for example, when the user has letter 'ö' 
(lowercase o with the two dots above it) in their password. In this case 
the incoming UTF-8 encoding could have one of these two ways to encode 
'ö' (uses one or two unicode codepoints):

     U+00F6 LATIN SMALL LETTER O WITH DIAERESIS
or
     U+006F LATIN SMALL LETTER O
     U+0308 COMBINING DIAERESIS

The NFD would use the latter. NFD also does any reordering that's 
needed. This ensures the octet by octet processing is always done in the 
same order when the eventual hashing is done.

Ondra, can you check if NFD is used with your network password? It might 
be needed to make the normalization method configurable too so that it 
matches the processing that's done when the users set their password.

To summarise: can be done, but needs to match the processing done when 
the password is set. Encoding problems also need debug logging.

[1] https://en.wikipedia.org/wiki/Unicode_equivalence#Normal_forms


Thanks,
Heikki


-- 
Heikki Vatiainen
Radiator Software, makers of Radiator
Visit radiatorsoftware.com for Radiator AAA server software