Fixing an error 400 after logging into Document Builder

If the web browser displays an Error 400: RPENG license checkout failed after logging in to Document Builder, your application server might be configured to support the Transport Layer Security (TLS) 1.2 protocol. You must configure extra server settings to run Document Builder with an application server in TLS 1.2 mode.

Before you begin

Configure your application server with the TLSv1.2 protocol for the Secure Sockets Layer (SSL) context to enforce SP800-131a requirements. For more information, see one of the following links that depends on your application server: Deploy Document Builder on your application server:

About this task

The following are examples of the error 400 messages that display in the web browser after logging into Document Builder:
  • Error 400:  RPENG license checkout failed.  Error is: javax.net.ssl SSLHandshakeException:  Remote host closed connection during handshake.
  • Error 400:  RPENG license checkout failed.  Error is:  java.lang.IllegalArgumentException: Only TLS1.2 protocol can be enabled in SP800131 strict mode.

Procedure

  1. Define the following Java virtual machine (JVM) settings for your application server:
    # The Java Secure Socket Extension (JSSE) is enabled to run in SP800-131 strict mode by setting the system property com.ibm.jsse2.sp800-131 to strict
    -Dcom.ibm.jsse2.sp800-131=strict
    -Dcom.ibm.team.repository.transport.client.protocol=TLSv1.2
    -Dcom.ibm.rational.rpe.tls12only=true
    Tip: See the following link or information about how to set JVM properties, which depend on your application server:
  2. Edit the server.xml file to add the following SSL and keystore service object entries:
    <ssl id="defaultSSLConfig" keyStoreRef="defaultKeyStore" sslProtocol="TLSv1.2" 
    enabledCiphers="TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 
    TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 
    TLS_RSA_WITH_AES_128_CBC_SHA256 
    TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA 
    TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA 
    TLS_RSA_WITH_AES_128_CBC_SHA 
    TLS_DHE_RSA_WITH_AES_128_CBC_SHA 
    TLS_DHE_RSA_WITH_AES_128_CBC_SHA256"/>
    				  
    <!--  keystore service object entry -->
    <keyStore id="defaultKeyStore" password="RPEt3st@123"  type="JCEKS" location="ibm-team-ssl.keystore"/>
    The RPEt3st@123 password and ibm-team-ssl.keystore location are examples, the administrator must replace these examples with the proper password and keystore location.
  3. Add the environment variable:
    JVM_ARGS=-Dhttps.protocols=TLSv1.2

Feedback