It's all about the answers!

Ask a question

Help to rescue RTC/RRC in POC env, CRJAZ1505E, CRJAZ1061I, CRJAZ1972E, ...


long TRUONG (365496133) | asked Mar 21 '16, 2:45 a.m.
edited Mar 21 '16, 2:48 a.m.
 On Windows/Tomcat RTC/RRC 5.0.2: We are switching SQLserver DB ID to comply with annual passwd change requirement (passwd will be changed for idle ID when it's not in use, this way without down time).

We started with jtsDB in JTS on POC, The save of new ID/passwd took so long (compared to the IDs switch before, though this time not from ID with invalid passwd to one with valid passwd), we made the mistake of refreshing the webpage mid-save, the save hung and never completed, everything went haywire. Pls see error messages below.

Same ID switch was carried out in PIT, the first one (ccmDB in CCM) took a long time, but all 6 (ccmDB & dw in CCM, jtsDB & dw in JTS, rmDB & dw in RM) went through OK.

We checked everything we can think of such as the teamserver.properties files, env var SQLSERVER_JDBC_DRIVER_FILE. Completed the ID switch in all 6 locations. Now at the end of day 3, we found that all Consumer (Inbound) Authorized keys have been gone. Wonder if can do a .../jts/reset to bring the keys back?. Or what else do we need to do?




Accepted answer


permanent link
long TRUONG (365496133) | answered Mar 30 '16, 6:55 p.m.
Issue resolved, PMR closed: After all there was no RTC breakdown, no issue with the procedure switching to another ID/passwd set. The whole brouhaha was due to (SQLserver) DB permissions not properly set for the switchToID and too shallow independent verification not revealing the flaw on the DB side.

Support got us running  onlineverify on jts and ccm, it was found that typically:

The SELECT permission was denied on the object 'ALL_TYPES', database 'jts', schema 'REPOSITORY'.  

com.ibm.team.repository.common.InternalRepositoryException: CRJAZ0398I Unable to find dbid for ItemType "com.ibm.team.repository.auth#AppCreds"

  It had used this teamserver.properties:

Loading the configuration from "file:conf/jts/teamserver.properties".

  And had accessed this DB:

jdbc:sqlserver://N01DSW227:1433;databaseName=jts;user=xxxxxxxx;password=xxxxxxxx

That brought us out of tunnel vision -- thinking that RTC completely broke down, and thinking that our independent verification (via SQLserver studio) of login and drilling down several levels from the top of the DB's, sufficient proof of credentials and permissions -- and reverted to the original switchFromID/switchFromPasswd: Everything was hunky dory again.

DBA's helped us in granting  db_owner permissions to switchToID on POC, and we are up and running with an effective passwd change.

We would humbly recommend our below passwd-change procedure 

  • to minimize risk to the point of no interruption required
  • to remove the necessity of tight cooperation and schedule sync between the SCM and DBA teams
Password Change Procedure Adopted:

  • At their convenience, DBA team creates a second set of bona fide ID/passwd, switchToID/switchToPasswd (assuming the same set used for all DB's) ahead of passwd-change D-day.
  • SCM team changes the passwd (via thus created set of ID/passwd or another existing set of ID/to-be-used-newPasswd)
    • Independently verify login, credentials, permissions ( by direct access to the DB's) rigorously (to avoid the brouhaha we got in due to permissions to the switchToID, and too shallow check on this post)
    • Change the database DB/passwd on the Data Connection web page. Note that for each app (JTS, RTC, RRC, RQM) there are two sets of ID/passwd on the web page, one for the app DB, one for datawarehouse DB: Switch from switchFromID/switchFromPasswd to switchToID/switchToPasswd. 
    • Save the web page: Be patient, the save of the web page may take time to complete
    • Verify access from the Jazz tools.
    • Optionally verify the DB log(s).
  • Again at their convenience, post passwd change completion, DBA team changes the switchFromPasswd, of the now idle ID switchFromID, to an agreed-to new passwd for the next passwd change.
On subsequent passwd change, the first step of creating a new set of ID/passwd will be skipped, the passwd change procedure will start with the SCM team performing the passwd change on the waiting set of ID/passwd.
Ralph Schoon selected this answer as the correct answer

One other answer



permanent link
Ralph Schoon (57.2k23642) | answered Mar 21 '16, 3:58 a.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
I would suggest to get in contact with support to try to resolve your situation. If you need to change the DB Password, I would assume you have to use the DB Tools to do so, while the server is down, and then also change the password in the teamserver.properties before bringing up the server again.

Comments
long TRUONG commented Mar 24 '16, 9:57 a.m.

 Thx Ralph,

 [PMR 32232,227,000] : Help to rescue RTC/RRC in POC env, CRJAZ1505E, for those interested and have access: In progress, server still down.

Your answer


Register or to post your answer.