TLS/SNI: Alert.UNRECOGNIZED_NAME unreachable because sniMatchers is never set

[DigitalMediageek]

We like the use the feature returning Alert.UNRECOGNIZED_NAME during TLS handshake, when requested server name is unknown. AFAIK, this should work when using SSL bundle configuration. However, debugging the code, I'm pretty convinced, that this behaviour cannot work, currently.

Pre-requisites:

• tested with Spring Boot V3.5.3
• tested with Java 24

Demo-case:

There is no special demo required. Just generate a simple plain Spring Boot project on start.spring.io and configure it like follows:

spring:
  application:
    name: sni-demo
  ssl:
    bundle:
      jks:
        web:
          keystore:
            location: "classpath:fakeserver.pfx"
            password: ""
            type: "PKCS12"
          truststore:
            location: "classpath:fake-ca.pfx"
            password: ""
            type: "PKCS12"
server:
  port: 8443
  ssl:
    bundle: "web"
    server-name-bundles:
      - server-name: "fake.server.com"
        bundle: "web"

Create fakeserver pfx and ca to your needs.

When debugging the code, I found the following:

• returning Alert.UNRECOGNIZED_NAME is done in ServerNameExtension.java here:

if (!shc.sslConfig.sniMatchers.isEmpty()) {
    sni = chooseSni(shc.sslConfig.sniMatchers, spec.serverNames);
    if (sni != null) {
        if (SSLLogger.isOn && SSLLogger.isOn("ssl,handshake")) {
            SSLLogger.fine(
                    "server name indication (" +
                    sni + ") is accepted");
        }
    } else {
        // We do not reject client without SNI extension currently.
        throw shc.conContext.fatal(Alert.UNRECOGNIZED_NAME,
                "Unrecognized server name indication");
    }
}

This code requires sslConfig.sniMatchers to be set.

• sslConfig is set inside HandshakeContext.java constructor:

this.sslConfig = (SSLConfiguration)conContext.sslConfig.clone();

• and the enclosing SSLParameters is in turn coming from AbstractJsseEndpoint.java:

SSLEngine engine = sslContext.createSSLEngine();
engine.setUseClientMode(false);
engine.setEnabledCipherSuites(sslHostConfig.getEnabledCiphers());
engine.setEnabledProtocols(sslHostConfig.getEnabledProtocols());
SSLParameters sslParameters = engine.getSSLParameters();
sslParameters.setUseCipherSuitesOrder(sslHostConfig.getHonorCipherOrder());
if (clientRequestedApplicationProtocols != null && clientRequestedApplicationProtocols.size() > 0 && this.negotiableProtocols.size() > 0) {
    List<String> commonProtocols = new ArrayList(this.negotiableProtocols);
    commonProtocols.retainAll(clientRequestedApplicationProtocols);
    if (commonProtocols.size() > 0) {
        String[] commonProtocolsArray = (String[])commonProtocols.toArray(new String[0]);
        sslParameters.setApplicationProtocols(commonProtocolsArray);
    }
}

switch (sslHostConfig.getCertificateVerification()) {
    case NONE:
        sslParameters.setNeedClientAuth(false);
        sslParameters.setWantClientAuth(false);
        break;
    case OPTIONAL:
    case OPTIONAL_NO_CA:
        sslParameters.setWantClientAuth(true);
        break;
    case REQUIRED:
        sslParameters.setNeedClientAuth(true);
}

engine.setSSLParameters(sslParameters);
return engine;

As one can see, an empty SSLEngine is created here and SNIMatchers of SSLParameters are never set. Hence, Alert.UNRECOGNIZED_NAME can never occur, because sslConfig.sniMatchers is always empty.

Please review.

Best regards
Harry

See also: https://stackoverflow.com/questions/79706211/how-to-set-snimatcher-when-using-spring-boot-correctly/79708736#79708736

[mhalbritter]

Hello! AbstractJsseEndpoint is a class in Tomcat, Spring Boot is merely using it. Are you saying they are failing to set the SNIMatchers? If so, then this is the wrong issue tracker.

Or is there something wrong in Boot? I can only find one code ___location which is working with SSLEngine but this is related to Kafka, not web servers.

[DigitalMediageek]

Thanks! Your comment brought me on another track. I modified AbstractJsseEndpoint by implementing a custom protocol handler. The solution is documented on stackoverflow. I still think that this is a bug, but as you said by tomcat, not by spring boot. This issue can be closed.