Configuring an encrypted connection between WAS 6.1 and Oracle 10g with server authentication

I stumbled upon some obstacles while configuring a secured connection between WebSphere Application Server 6.1.0.17 and Oracle 10g Release 2 over TCPS. The requirement was to establish an encrypted connection with server (i.e. database listener) authentication. It is realized in the second scenario described in Oracle's guide for enabling SSL on JDBC thin driver. It seems all pretty straightforward and in fact it is — on the Sun's JVM the following code works like a charm (tested with ojdbc14.jar for Oracle 10g Release 2):

import java.util.Properties;
import java.sql.*;

class OracleSSLTest {
  public static void main(String[] args) throws Exception {
    DriverManager.registerDriver(new oracle.jdbc.driver.OracleDriver());
    String url = "jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=oracle)(PORT=2484)) (CONNECT_DATA=(SERVICE_NAME=TESTSSL)(SERVER=DEDICATED)))";
    Properties props = new Properties();
    props.setProperty("user", "testssl");
    props.setProperty("password", "testssl");
    props.setProperty("javax.net.ssl.trustStore", "client.jks");
    props.setProperty("javax.net.ssl.trustStoreType", "JKS");
    props.setProperty("javax.net.ssl.trustStorePassword", "password");

    Connection conn = DriverManager.getConnection(url, props);
    System.out.println("Great success!");
  }
}

However, the same code launched on IBM JVM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64 j9vmxa6423-20080315) generates the following exception:

Exception in thread "main" java.sql.SQLException: I/O Exception: The Network Adapter could not establish the connection
  at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
  at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146)
  at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:255)
  at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:387)
  at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:441)
  at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:165)
  at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:35)
  at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:801)
  at java.sql.DriverManager.getConnection(DriverManager.java:562)
  at java.sql.DriverManager.getConnection(DriverManager.java:155)
  at OracleSSLTest.main(Test.java:15)

I've checked that GIJ behaves similar. Running the code with a newer JDBC driver, namely ojdbc5.jar, gives more descriptive messages (the improved verbosity of the new drivers can save you a lot of time):

Exception in thread "main" java.sql.SQLException: The Network Adapter could not establish the connection
  at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:412)
  at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:531)
  at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:221)
  at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
  at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:503)
  at java.sql.DriverManager.getConnection(DriverManager.java:562)
  at java.sql.DriverManager.getConnection(DriverManager.java:155)
  at OracleSSLTest.main(Test.java:15)
Caused by: oracle.net.ns.NetException: The Network Adapter could not establish the connection
  at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:359)
  at oracle.net.resolver.AddrResolution.resolveAndExecute(AddrResolution.java:422)
  at oracle.net.ns.NSProtocol.establishConnection(NSProtocol.java:672)
  at oracle.net.ns.NSProtocol.connect(NSProtocol.java:237)
  at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1042)
  at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:301)
  ... 7 more
Caused by: oracle.net.ns.NetException: The ssl protocol specified is not supported.
  at oracle.net.nt.TcpsConfigure.configureVersion(TcpsConfigure.java:181)
  at oracle.net.nt.TcpsNTAdapter.setSSLSocketOptions(TcpsNTAdapter.java:146)
  at oracle.net.nt.TcpsNTAdapter.connect(TcpsNTAdapter.java:121)
  at oracle.net.nt.ConnOption.connect(ConnOption.java:123)
  at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:337)
  ... 12 more
Caused by: java.lang.IllegalArgumentException: SSLv2Hello
  at com.ibm.jsse2.mb.a(mb.java:6)
  at com.ibm.jsse2.lb.<init>(lb.java:16)
  at com.ibm.jsse2.jc.setEnabledProtocols(jc.java:245)
  at oracle.net.nt.TcpsConfigure.configureVersion(TcpsConfigure.java:177)
  ... 16 more

This difference between Sun's and IBM's JVMs is documented. I don't want to use SSLv2 (which is considered insecure now) in any form anyway, so I added another property:

props.setProperty("oracle.net.ssl_version", "3.0");

Enforcing SSL or TSL on the listener side in listener.ora doesn't do the trick. I've written some enhanced version of the test to simulate more accurately what I believe WAS is doing while establishing the connection:

import java.util.Properties;
import java.sql.*;
import javax.sql.*;
import oracle.jdbc.pool.OracleConnectionPoolDataSource;

class OracleSSLTest {
  public static void main(String[] args) throws Exception {
    DriverManager.registerDriver (new oracle.jdbc.driver.OracleDriver());
    String url = "jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=oracle)(PORT=2484)) (CONNECT_DATA=(SERVICE_NAME=TESTSSL)(SERVER=DEDICATED))" + 
    "(SECURITY=(SSL_SERVER_CERT_DN=\"CN=testssl,C=PL\")))";
    Properties props = new Properties();
    props.setProperty("user", "testssl");
    props.setProperty("password", "testssl");
    props.setProperty("javax.net.ssl.trustStore", "client.jks");
    props.setProperty("javax.net.ssl.trustStoreType", "JKS");
    props.setProperty("javax.net.ssl.trustStorePassword", "password");
    props.setProperty("oracle.net.ssl_server_dn_match", "true");

    props.setProperty("oracle.net.ssl_version", "3.0");
    Connection conn = DriverManager.getConnection(url, props);

    System.out.println("Great success!");

    OracleConnectionPoolDataSource ds = new OracleConnectionPoolDataSource();
    ds.setURL(url);
    // setting username and password once again through datasource methods seems to be required as exception is thrown otherwise
    ds.setUser("testssl");
    ds.setPassword("testssl");
    ds.setConnectionProperties(props);
    PooledConnection pc = ds.getPooledConnection();

    System.out.println("Another great success!");
  }
}

Since the test run without an error also with ojdbc14.jar, I switched to the Integrated Solutions Console to create the data source with the above properties. One thing to keep in mind is that the properties to be passed to the OracleConnectionPoolDataSource factory must be defined as a value of a newly created custom property connectionProperties in the following form:

javax.net.ssl.trustStore=client.jks; javax.net.ssl.trustStoreType=JKS; javax.net.ssl.trustStorePassword=password; oracle.net.ssl_server_dn_match=true; oracle.net.ssl_version=3.0

Another problem one may encounter is using an anonymous cipher. Trying to limit ciphers probed during the handshake with the statement:

props.setProperty("oracle.net.ssl_cipher_suites", "(SSL_DH_anon_WITH_3DES_EDE_CBC_SHA, SSL_DH_anon_WITH_RC4_128_MD5, SSL_DH_anon_WITH_DES_CBC_SHA)");

Effects in the following exception:

Exception in thread "main" java.sql.SQLException: I/O Exception: IBM's Client TrustManager does not allow anonymous cipher suites: SSL_DH_anon_WITH_3DES_EDE_CBC_SHA
  at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:421)
  at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:531)
  at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:221)
  at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
  at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:503)
  at java.sql.DriverManager.getConnection(DriverManager.java:562)
  at java.sql.DriverManager.getConnection(DriverManager.java:155)
  at OracleSSLTest.main(Test.java:15)
Caused by: javax.net.ssl.SSLHandshakeException: IBM's Client TrustManager does not allow anonymous cipher suites: SSL_DH_anon_WITH_3DES_EDE_CBC_SHA
  at com.ibm.jsse2.eb.serverHello(eb.java:199)
  at com.ibm.jsse2.eb.a(eb.java:54)
  at com.ibm.jsse2.db.m(db.java:208)
  at com.ibm.jsse2.db.a(db.java:259)
  at com.ibm.jsse2.jc.a(jc.java:271)
  at com.ibm.jsse2.jc.g(jc.java:403)
  at com.ibm.jsse2.jc.a(jc.java:401)
  at com.ibm.jsse2.j.write(j.java:10)
  at oracle.net.ns.Packet.send(Packet.java:385)
  at oracle.net.ns.ConnectPacket.send(ConnectPacket.java:173)
  at oracle.net.ns.NSProtocol.connect(NSProtocol.java:283)
  at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1042)
  at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:301)
  ... 7 more

IBM documents a way to implement own TrustManager and override the default with SSLContext.init() method for your own code, but it obviously doesn't hit the spot. There is another solution in the documentation that suggests to add the line com.ibm.ssl.skipDefaultTrustManagerWhenCustomDefined=true at the top of the ssl.client.props file and define the custom trust manager with com.ibm.ssl.customKeyManager thereabouts, but it doesn't work for me.