It's all about the answers!

Ask a question

RTC Build agent on z/OS SSL communication RC:410 and RC:420


Sathya moorthy (352813) | asked Jun 14 '14, 2:51 p.m.
retagged Sep 01 '17, 11:41 a.m. by Ken Tessier (84117)

Hi,

We have installed RTC Build Agent on z/OS. now we are planning to do SSL settings for Build agent.

This is Our SSL part of bfagent.conf

     May use a gskkyman key database instead of SAF keyring                 


gsk_ssl_key_location /etc/jazz405/ccm/bfagentssl.kdb                         
gsk_ssl_kdb_password 93a2686a707787d400f7f2d401194145b4e0c3cb695d0e1b58ec    
gsk_keyring_label bfacert                                                    
gsk_ssl_protocol ALL                                                         
gsk_ssl_cipher_v2 6321                                                       
gsk_ssl_cipher_v3 0906030201                                                 
gsk_ssl_client_authentication false                                          
gsk_password_encrypt true                                                    

We are getting the below error, when we test the connection on the bfagent.log debug.

Sat Jun 14 18:32:33 2014 [16842915] main.c          : 519: === STARTING DAEMON PROCESS ===
Sat Jun 14 18:32:34 2014 [16842915] bf_ipv6.c       : 780: bf_listener_new: [0.0.0.0/5555] listening
Sat Jun 14 18:32:34 2014 [16842915] daemon.c        :  53: background: orphaning daemon
Sat Jun 14 18:32:34 2014 [16842915] daemon.c        :  45: --- EXITING ---  (create orphan)
Sat Jun 14 18:32:34 2014 [   65700] daemon.c        :  45: --- EXITING ---  (create orphan)
Sat Jun 14 18:32:41 2014 [33620131] daemon.c        : 131: [/5555] accepting [161.178.198.170/49830] (161.178.198.170)
Sat Jun 14 18:32:41 2014 [33620131] daemon.c        : 156: [161.178.198.170/49830]: attached to pid 16842916
Sat Jun 14 18:32:41 2014 [16842916] daemon.c        : 149: === NEW AGENT ===
Sat Jun 14 18:32:41 2014 [16842916] platform.c      : 192: ICONV ok [IBM-1047]
Sat Jun 14 18:32:41 2014 [16842916] platform.c      :2132: LOCALE ok [] -> [C]
Sat Jun 14 18:32:41 2014 [16842916] io.c            : 482: In start_SSL : initialize system ssl environment
Sat Jun 14 18:32:41 2014 [16842916] io.c            : 291: gsk_ssl_kdb_password : 93a2686a707787d400f7f2d401194145b4e0c3cb695d0e1b58ec

Sat Jun 14 18:32:41 2014 [16842916] bfcryptloader.c : 569: Password decoded.
Sat Jun 14 18:32:41 2014 [16842916] io.c            : 300: decode_password :***
Sat Jun 14 18:32:42 2014 [16842916] io.c            : 365: SSL ssl_key_location=/etc/jazz405/ccm/bfagentssl.kdb
Sat Jun 14 18:32:42 2014 [16842916] io.c            : 378: SSL decode_password=

Sat Jun 14 18:32:42 2014 [16842916] io.c            : 415: Server: Setting protocol to all.

Sat Jun 14 18:32:42 2014 [16842916] io.c            : 418: SSL ssl_key_ring_label=bfacert
Sat Jun 14 18:32:42 2014 [16842916] io.c            : 430: SYSTEM SSL env init ok
Sat Jun 14 18:32:42 2014 [16842916] io.c            : 496: Calling secure socket open
Sat Jun 14 18:32:42 2014 [16842916] io.c            : 517: SSL GSK_SESSION_TYPE=GSK_SERVER_SESSION
Sat Jun 14 18:32:42 2014 [16842916] io.c            : 524: SSL cipher_specs_V2=6321
Sat Jun 14 18:32:42 2014 [16842916] io.c            : 529: SSL cipher_specs_V3=0906030201
Sat Jun 14 18:32:42 2014 [16842916] io.c            : 533: Calling gsk_secure_socket_init() to start ssl handshake with client.
Sat Jun 14 18:34:05 2014 [33620131] daemon.c        : 131: [*/5555] accepting [10.26.125.254/17502] (10.26.125.254)
Sat Jun 14 18:34:06 2014 [33620131] daemon.c        : 156: [10.26.125.254/17502]: attached to pid 65702
Sat Jun 14 18:34:06 2014 [   65702] daemon.c        : 149: === NEW AGENT ===
Sat Jun 14 18:34:06 2014 [   65702] platform.c      : 192: ICONV ok [IBM-1047]
Sat Jun 14 18:34:06 2014 [   65702] platform.c      :2132: LOCALE ok [] -> [C]
Sat Jun 14 18:34:06 2014 [   65702] io.c            : 482: In start_SSL : initialize system ssl environment
Sat Jun 14 18:34:06 2014 [   65702] io.c            : 291: gsk_ssl_kdb_password : 93a2686a707787d400f7f2d401194145b4e0c3cb695d0e1b58ec

Sat Jun 14 18:34:06 2014 [   65702] bfcryptloader.c : 569: Password decoded.
Sat Jun 14 18:34:06 2014 [   65702] io.c            : 300: decode_password :

Sat Jun 14 18:34:06 2014 [   65702] io.c            : 365: SSL ssl_key_location=/etc/jazz405/ccm/bfagentssl.kdb
Sat Jun 14 18:34:06 2014 [   65702] io.c            : 378: SSL decode_password=

Sat Jun 14 18:34:06 2014 [   65702] io.c            : 415: Server: Setting protocol to all.

Sat Jun 14 18:34:06 2014 [   65702] io.c            : 418: SSL ssl_key_ring_label=bfacert
Sat Jun 14 18:34:06 2014 [   65702] io.c            : 430: SYSTEM SSL env init ok
Sat Jun 14 18:34:06 2014 [   65702] io.c            : 496: Calling secure socket open
Sat Jun 14 18:34:06 2014 [   65702] io.c            : 517: SSL GSK_SESSION_TYPE=GSK_SERVER_SESSION
Sat Jun 14 18:34:06 2014 [   65702] io.c            : 524: SSL cipher_specs_V2=6321
Sat Jun 14 18:34:06 2014 [   65702] io.c            : 529: SSL cipher_specs_V3=0906030201
Sat Jun 14 18:34:06 2014 [   65702] io.c            : 533: Calling gsk_secure_socket_init() to start ssl handshake with client.
Sat Jun 14 18:37:41 2014 [33620131] daemon.c        : 131: [*/5555] accepting [10.26.125.254/17660] (10.26.125.254)
Sat Jun 14 18:37:41 2014 [16842916] io.c            : 536: SSL_accept rc=410 (0 is good, >=1 is handshake failure or fatal)
Sat Jun 14 18:37:41 2014 [16842916] io.c            : 539:  SSLErrorHandshake: failed to Call gsk_secure_socket_init() to.
Sat Jun 14 18:37:41 2014 [16842916] agent.c         : 761: cleanup
Sat Jun 14 18:37:41 2014 [33620131] platform.c      : 260: SIGCHLD(20) 16842916 -> bfdaemon: status=0
Sat Jun 14 18:37:41 2014 [33620131] daemon.c        : 156: [10.26.125.254/17660]: attached to pid 65705
Sat Jun 14 18:37:41 2014 [   65705] daemon.c        : 149: === NEW AGENT ===
Sat Jun 14 18:37:41 2014 [   65705] platform.c      : 192: ICONV ok [IBM-1047]
Sat Jun 14 18:37:41 2014 [   65705] platform.c      :2132: LOCALE ok [] -> [C]
Sat Jun 14 18:37:41 2014 [   65705] io.c            : 482: In start_SSL : initialize system ssl environment
Sat Jun 14 18:37:41 2014 [   65705] io.c            : 291: gsk_ssl_kdb_password : 93a2686a707787d400f7f2d401194145b4e0c3cb695d0e1b58ec

Sat Jun 14 18:37:41 2014 [   65705] bfcryptloader.c : 569: Password decoded.
Sat Jun 14 18:37:41 2014 [   65705] io.c            : 300: decode_password :

Sat Jun 14 18:37:42 2014 [   65705] io.c            : 365: SSL ssl_key_location=/etc/jazz405/ccm/bfagentssl.kdb
Sat Jun 14 18:37:42 2014 [   65705] io.c            : 378: SSL decode_password=
**
Sat Jun 14 18:37:42 2014 [   65705] io.c            : 415: Server: Setting protocol to all.

Sat Jun 14 18:37:42 2014 [   65705] io.c            : 418: SSL ssl_key_ring_label=bfacert
Sat Jun 14 18:37:42 2014 [   65705] io.c            : 430: SYSTEM SSL env init ok
Sat Jun 14 18:37:42 2014 [   65705] io.c            : 496: Calling secure socket open
Sat Jun 14 18:37:42 2014 [   65705] io.c            : 517: SSL GSK_SESSION_TYPE=GSK_SERVER_SESSION
Sat Jun 14 18:37:42 2014 [   65705] io.c            : 524: SSL cipher_specs_V2=6321
Sat Jun 14 18:37:42 2014 [   65705] io.c            : 529: SSL cipher_specs_V3=0906030201
Sat Jun 14 18:37:42 2014 [   65705] io.c            : 533: Calling gsk_secure_socket_init() to start ssl handshake with client.
Sat Jun 14 18:37:42 2014 [   65705] io.c            : 536: SSL_accept rc=420 (0 is good, >=1 is handshake failure or fatal)
Sat Jun 14 18:37:42 2014 [   65705] io.c            : 539:  SSLErrorHandshake: failed to Call gsk_secure_socket_init() to.
Sat Jun 14 18:37:42 2014 [   65705] agent.c         : 761: cleanup
Sat Jun 14 18:37:42 2014 [33620131] platform.c      : 260: SIGCHLD(20) 65705 -> bfdaemon: status=0


Please let us know what we are missing.

Regards,
Sathya


Comments
Donald Nong commented Jun 15 '14, 11:18 p.m.

Check the corresponding error messages for the two return code.
http://publib.boulder.ibm.com/infocenter/zvm/v5r4/topic/com.ibm.zvm.v54.kijl0/hcsk7b3040.htm#wq211
RC 410 means "incorrectly-formatted message received from peer application". Maybe you can use tcpdump to capture the network traffic and see what was being transferred.


Sathya moorthy commented Jun 16 '14, 2:08 a.m.

Thanks Donald Nong. Yes i had found the reason code what it means for RC 410 and RC 420, but not sure why its happening. I'll check on tcpdump.

Regards,
Sathya


Eric Kung commented Aug 24 '17, 7:22 p.m. | edited Aug 24 '17, 9:10 p.m.

 Hi,  Sathya,

Bring this old discussion back again,  we are struggling with same setup.  Trying to get z/OS Rational build agent connect to RTC server with TLS1.2,  got exactly same error as you had. 
Did you end up with working solution ?  are you able to share if you did.


Sathya moorthy commented Aug 25 '17, 1:25 a.m.

Hi Eric,

Yes, it was resolved and we were able to use TLS1.2. I'm not able to recollect them now. can you share your log and conf details.

Regards,
Sathya


Eric Kung commented Aug 27 '17, 6:00 p.m. | edited Aug 27 '17, 6:44 p.m.

 Hi, Sathya,


Thanks for your reply. 

I followed this link to set it up, use kdb file as certificate

SSL section of bfagent.conf
# Note: If using SSL, create or obtain PEM keystores from engine and set the proper key password.
#ssl_key_location /usr/local/bin/buildForgeKey.pem
#ssl_key_password <password>
#ssl_cert_location /usr/local/bin/buildForgeCert.pem
#ssl_ca_location /usr/local/bin/buildForgeCA.pem
# Note: The ssl_protocol can be TLSv1,TLSv1.1 or TLSv1.2. You select only one.
#ssl_protocol TLSv1
#ssl_protocol TLSv1.1
ssl_protocol TLSv1.2
#ssl_cipher_group ALL
#ssl_cipher_override <cipher_list>
#ssl_client_authentication true
#ssl_client_auth_sufficient true
fips_enabled true

# Note: If using password encryption, see "Export Key File" in console to obtain keys in bfpwcrypt.conf.
#password_encrypt_module ./bfcrypt.dll;./bfpwcrypt.conf
#digest_algorithm  SHA2

#     System SSL support using a SAF keyring, just like  [userid]/[keyring] SAF
#gsk_ssl_key_location root/cert SAF
#                           OR
#     May use a gskkyman key database instead of SAF keyring
gsk_ssl_key_location /etc/jazz603/ccm/ek_rba.kdb
gsk_ssl_kdb_password 9b16d9abab33fcb200f7fa63be9a9c5289d522e5058157f5e738
gsk_keyring_label EKCERT01
gsk_ssl_protocol ALL
#gsk_ssl_protocol TLSV1_2
#gsk_ssl_cipher_v2 6321
#gsk_ssl_cipher_v3 0906030201
gsk_ssl_client_authentication true
gsk_password_encrypt true

bfagent.log :
Fri Aug 25 05:13:33 2017 [   65881] platform.c      : 192: ICONV ok [IBM-1047]
Fri Aug 25 05:13:33 2017 [   65881] platform.c      :2154: LOCALE ok [] -> [C]
Fri Aug 25 05:13:33 2017 [   65881] main.c          : 519: === STARTING DAEMON PROCESS ===
Fri Aug 25 05:13:33 2017 [   65881] bf_ipv6.c       : 780: bf_listener_new: [0.0.0.0/5555] listening
Fri Aug 25 05:13:33 2017 [   65881] daemon.c        :  53: background: orphaning daemon
Fri Aug 25 05:13:33 2017 [   65881] daemon.c        :  45: --- EXITING ---  (create orphan)
Fri Aug 25 05:13:33 2017 [   65882] daemon.c        :  45: --- EXITING ---  (create orphan)
Fri Aug 25 05:14:51 2017 [16843097] daemon.c        : 131: [/5555] accepting [9.32.157.211/57246] (igartctrain02.swg.usma.ibm.com)
Fri Aug 25 05:14:51 2017 [16843097] daemon.c        : 156: [9.32.157.211/57246]: attached to pid 33620315
Fri Aug 25 05:14:51 2017 [33620315] daemon.c        : 149: === NEW AGENT ===
Fri Aug 25 05:14:51 2017 [33620315] platform.c      : 192: ICONV ok [IBM-1047]
Fri Aug 25 05:14:51 2017 [33620315] platform.c      :2154: LOCALE ok [] -> [C]
Fri Aug 25 05:14:51 2017 [33620315] io.c            : 482: In start_SSL : initialize system ssl environment
Fri Aug 25 05:14:51 2017 [33620315] io.c            : 291: gsk_ssl_kdb_password : 9b16d9abab33fcb200f7fa63be9a9c5289d522e5058157f5e738

Fri Aug 25 05:14:51 2017 [33620315] bfcryptloader.c : 569: Password decoded.
Fri Aug 25 05:14:51 2017 [33620315] io.c            : 300: decode_password :***
Fri Aug 25 05:14:52 2017 [33620315] io.c            : 365: SSL ssl_key_location=/etc/jazz603/ccm/ek_rba.kdb
Fri Aug 25 05:14:52 2017 [33620315] io.c            : 378: SSL decode_password=**
Fri Aug 25 05:14:52 2017 [33620315] io.c            : 415: Server: Setting protocol to all.

Fri Aug 25 05:14:52 2017 [33620315] io.c            : 418: SSL ssl_key_ring_label=EKCERT01
Fri Aug 25 05:14:52 2017 [33620315] io.c            : 430: SYSTEM SSL env init ok
Fri Aug 25 05:14:52 2017 [33620315] io.c            : 496: Calling secure socket open 
Fri Aug 25 05:14:52 2017 [33620315] io.c            : 514: SSL GSK_SESSION_TYPE=GSK_SERVER_SESSION_WITH_CL_AUTH
Fri Aug 25 05:14:52 2017 [33620315] io.c            : 524: SSL cipher_specs_V2=631
Fri Aug 25 05:14:52 2017 [33620315] io.c            : 529: SSL cipher_specs_V3=090201
Fri Aug 25 05:14:52 2017 [33620315] io.c            : 533: Calling gsk_secure_socket_init() to start ssl handshake with client.
Fri Aug 25 05:19:51 2017 [33620315] io.c            : 536: SSL_accept rc=410 (0 is good, >=1 is handshake failure or fatal)
Fri Aug 25 05:19:51 2017 [33620315] io.c            : 539:  SSLErrorHandshake: failed to Call gsk_secure_socket_init() to.
Fri Aug 25 05:19:51 2017 [33620315] agent.c         : 761: cleanup
Fri Aug 25 05:19:51 2017 [33620315] agent.c         : 763: cleanup, then unmap_drives

One other thing I don't understand.
If I issue a bfagent -v command in z/OS USS shell,  it show system ssl v1

UISC196:/u/jazz603/ccm: >bfagent -v  
IBM Rational Build Forge Agent       
8.0.0.3 GA (Build 0007)              
Platform: Unix                       
SYSTEM SSL: TLSv1                    
UISC196:/u/jazz603/ccm: >            
                                     
Our z/OS environment has system SSL TLS1.2


Sathya moorthy commented Aug 28 '17, 11:14 a.m.

I had a copy my conf file. I have enabled trace where you can get more info, reg the issue.



gsk_trace_file /tmp/rtc/gskssl.trc
gsk_trace 0xFF

gsk_ssl_key_location /etc/jazz405/ccm/bfagentssl.kdb
gsk_ssl_kdb_password 93a2686a707787d400f7f2d401194145b4e0c3cb695d0e1b58ec
gsk_keyring_label bfacert
gsk_ssl_protocol ALL
gsk_ssl_cipher_v2 6321
gsk_ssl_cipher_v3 0906030201
gsk_ssl_client_authentication false

gsk_password_encrypt true



You can comment this ssl_protocol TLSv1.2.

After the change, can you restart the build agent and check the logs and trace.

Regards,
Sathya


Eric Kung commented Aug 28 '17, 7:12 p.m. | edited Aug 28 '17, 7:13 p.m.

 Hi, Sathya,

The 2 bfagent.conf lines gsk_trace_file and gsk_trace has no effect.  therefore I added this line to startbfa.sh

The gsk trace shows system ssl v3,  although a PTF OA39422 has been installed to enable TLS1.2
08/28/2017-17:21:26 Thd-0 INFO gsk_svc_init(): System SSL Version 3, Release 23, Service level OA39422
08/28/2017-17:21:26 Thd-0 INFO gsk_svc_init(): LE runtime level 0x410d0000, 31-bit addressing mode
08/28/2017-17:21:26 Thd-0 INFO gsk_svc_init(): STDOUT handle=1, STDERR handle=2, TRACE handle=4
08/28/2017-17:21:26 Thd-0 INFO gsk_dll_init_once(): Using variant character table for code set IBM-1047
08/28/2017-17:21:26 Thd-0 INFO gsk_dll_init_once(): Using local code page IBM-1047
08/28/2017-17:21:26 Thd-0 INFO gsk_dll_init_once(): Using ISO8859-1 for TELETEX string
08/28/2017-17:21:27 Thd-0 ERROR gsk_dll_init_once(): Unable to load crypto DLL
    EDC5205S DLL module not found.
08/28/2017-17:21:27 Thd-0 INFO crypto_init(): SHA-1 crypto assist is available
08/28/2017-17:21:27 Thd-0 INFO crypto_init(): SHA-224 crypto assist is available
08/28/2017-17:21:27 Thd-0 INFO crypto_init(): SHA-256 crypto assist is available
08/28/2017-17:21:27 Thd-0 INFO crypto_init(): SHA-384 crypto assist is available
08/28/2017-17:21:27 Thd-0 INFO crypto_init(): SHA-512 crypto assist is available
08/28/2017-17:21:27 Thd-0 INFO crypto_init(): DES crypto assist is available
08/28/2017-17:21:27 Thd-0 INFO crypto_init(): DES3 crypto assist is available
08/28/2017-17:21:27 Thd-0 INFO crypto_init(): AES 128-bit crypto assist is available
08/28/2017-17:21:27 Thd-0 INFO crypto_init(): AES 256-bit crypto assist is available
08/28/2017-17:21:27 Thd-0 INFO crypto_init(): ICSF services are not available
08/28/2017-17:21:27 Thd-0 INFO gsk_dll_init_once(): Job name BLZBFA4, Process 05010114
08/28/2017-17:21:27 Thd-0 INFO gsk_dll_init_once(): GSKSRVR communication area at 00000000  

Do you still have a gsk trace file available for us to compare ? 
Can you access your z/OS USS shell to issue the command bfagent -v ?  would like to see what SYSTEM SSL level in the command output. 

Thank you for your great help. 


Sathya moorthy commented Aug 29 '17, 6:56 a.m.

I don't have anymore access to the RTC Build agent. so i can't check the logs or issue commands. I'm answering to your questions based on my experience here.

I see on gsktrace shows the following:
ERROR gsk_dll_init_once(): Unable to load crypto DLL EDC5205S DLL module not found.

This means that the Security Level 3 FMID JCPT3D1 is not installed and that means you will only be able to use Base Security Level ciphers.

You will need to order FMID JCPT3D1 to use the stronger ciphers.

Are you doing everything on Mainframe or ADCD package?

Regards,
Sathya


Eric Kung commented Aug 29 '17, 8:46 p.m. | edited Aug 29 '17, 8:46 p.m.

 Hi, Sathya

We are testing this on ADCD package, before move it to production mainframe.
Unfortunately ADCD package only have z/OS 1.13, there are many difference from a real mainframe environment. 
Our sysprog also identified the problem with missing FMID. he has installed it. 
We progressed a little, still have cipher issue though. 
Thank you for your help :)

showing 5 of 9 show 4 more comments

4 answers



permanent link
Jean-Bernard Curmi (14162) | answered Aug 31 '17, 12:01 p.m.
JAZZ DEVELOPER

Could you please add

gsk_ssl_cipher_v3 0a

in your bfagent.conf file


Comments
Eric Kung commented Aug 31 '17, 7:11 p.m. | edited Aug 31 '17, 7:11 p.m.

 Hi, Jean

We have resolved cipher issue, but stuck with RTC side can not validate Build agent certificate. 
SSL trace message
 ERROR read_v3_alert(): SSL V3 alert 46 received from 9.32.157.211[56180]

Do you think RTC server need to be restarted once a cerficate is imported according to this article https://jazz.net/library/article/607


permanent link
Jean-Bernard Curmi (14162) | answered Sep 01 '17, 5:55 a.m.
JAZZ DEVELOPER

The RTC server doesn't need to be restarted.

I am afraid the doc you followed to create the certificate is not up to date,

The certificate must be generated with a Rsa Key Size of 2048 and  Signature Digest Type of SHA-256   

ie  


permanent link
Jean-Bernard Curmi (14162) | answered Sep 01 '17, 5:56 a.m.
JAZZ DEVELOPER

                                                                  
       RSA Key Size                                               
                                                                  
   1 - 1024-bit key                                               
   2 - 2048-bit key                                               
   3 - 4096-bit key                                               
                                                                  
Select RSA key size (press ENTER to return to menu): 2            
                                                                  
       Signature Digest Type                                      
                                                                  
   1 - SHA-1                                                      
   2 - SHA-224                                                    
   3 - SHA-256                                                    
   4 - SHA-384                                                    
   5 - SHA-512                                                    
                                                                  
Select digest type (press ENTER to return to menu): 3             


permanent link
Eric Kung (11) | answered Sep 01 '17, 6:09 a.m.

Yes, I am aware about the newer digest type and stronger key length is required.

I used exactly what you listed.

This is part of the certificate details

 Signature algorithm: sha256WithRsaEncryption 
    Issuer unique ID: None                    
   Subject unique ID: None                    
Public key algorithm: rsaEncryption           
     Public key size: 2048                    

I still think the cert isn't imported in RTC server side correctly. 

Your answer


Register or to post your answer.