Data Collection Job Status fails
Running RTC version 4.0 and Derby on a Windows 2008 R2 server...
Finally got enough data inputted but Data Collection Job status form is showing all jobs failed. Where do I start my search for an answer? I'm opening a PMR but we need an answer fast! thanks. |
12 answers
Benjamin Silverman (4.1k●6●10)
| answered Oct 22 '12, 3:45 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
Hi Rodney,
The best place to start looking is in the ETL logs. These should be located in the server install directory in the logs folder (ie - C:\IBM\JazzTeamServer\server\logs). You should see the following logs: ccm-etl.log for CCM qm-et.log for QM jts-etl.log for JTS and RM You can also run the ISALite Data Collector and provide it in your PMR by going to C:\IBM\JazzTeamServer\support\ISALite and running "runISALite.bat" which will collect all the logs from the server as well as some other helpful information for support. Hope it helps, Comments Thanks. I'll look there. Anything in particular that I should look for?
|
|
Benjamin Silverman (4.1k●6●10)
| answered Oct 22 '12, 3:57 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
The log captures "ERROR" messages by default, but can be re-configured in the ETL section of the log4j.properties if you need to finer tune the logging to capture more information. To start, I'd look for any "ERROR" type messages and provide the details here if you can. Support however may ask you re-run the ETL with the log configured for "DEBUG" if there are no errors, or the errors do not reveal what the problem is. It is also possible that exceptions are thrown in the application log (ie - rm.log, qm.log, etc...), which is why I thought the ISALite Data Collection would be helpful, although I certainly understand your policy may prohibit you from sharing the details.
Comments (ccm-etl.log)Lots of ERRORS...
|
|
Benjamin Silverman (4.1k●6●10)
| answered Oct 22 '12, 4:21 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
Does the user that is configured to run ETL have the "Data Collector" license? Is that the same exception you see in the jts-etl.log, or is there more info there? That might explain why only the STAR job works, as this data is collected from the data in the database rather than from the applications,
Comments Yes it is the same error, but more.
|
|
Benjamin Silverman (4.1k●6●10)
| answered Oct 22 '12, 4:48 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
That might be the problem. The ETL is configured to run with a certain user if you go to https://<server>:<port>/jts/admin -> Reports -> Data Collection Jobs, and check the "User ID" who is running them. That user is created during jts/setup, so if you've attached the Data Collector license to any other user, you'll need to change the user that runs the ETL to make sure it's the user that has that license. You'll need to do that for each application.
Comments How do I check the user id who is running them?
|
|
Benjamin Silverman (4.1k●6●10)
| answered Oct 22 '12, 5:02 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
If you go to https://<server>:<port>/jts/admin -> Reports -> Data Collection Jobs, you'll see each job listed for the JTS/RM applications, and there is a "User ID" field that indicates the user configured to run the ETL. Again, you'll have to update this under each admin page (ie - qm/admin -> Reports -> Data Collection Jobs, ccm/admin -> Reports -> Data Collection Jobs) for the change to be made for each application.
Comments Not Seeing User ID. I have a screenshot but an unable to upload at this time.
If I select the Data Warehouse Connection, I do see some configurations settings like Data Collection User or Consumer key. But this field is not changeable.
|
|
Benjamin Silverman (4.1k●6●10)
| answered Oct 22 '12, 6:00 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
Sure, you can email it to me at bsilverm<at>us<dot>ibm<dot>com. Or I can attach a screenshot of mine here if it helps:
|
|
Benjamin Silverman (4.1k●6●10)
| answered Oct 22 '12, 6:26 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
I'm using 4.0 and the screenshot I showed is the default. Is your XDC Authentication type set to "oauth" instead of "FORM"? You can check under Reports -> Data Warehouse Connection -> XDC Authentication, or under jts/admin -> Server -> Advanced Properties -> XDC Authentication. I'd try changing it to FORM and specifying the username/password for your ETL user, as using a consumer key/secret is not the default configuration.
Comments It was set to JTS. I changed it to Form and now on the Data Collection Jobs screen the field indicates User ID and Password but it's still showing the consumer key not the User ID.
Is the ETL user the same as the Data Collection User that's shown on the Warehouse Connections screen or should it be the User who is trying to run the reports?
|
|
Benjamin Silverman (4.1k●6●10)
| answered Oct 22 '12, 6:37 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
You should be able to update the User ID/password on that page to reflect the user who has the Data Collector license. Are you not able to modify the values?
Comments Okay. I changed the User ID to my login User ID and the jobs are now running. The first few have already passed.
|
|
Benjamin Silverman (4.1k●6●10)
| answered Oct 22 '12, 6:50 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
Yes, you only need one Data Collector license since you only need one functional ETL user on the server. These jobs run by default at midnight, so the users should find that reports are updated with the latest data on the following day without you having to run them manually.
You can re-configure the time they run by entering an integer value under Reports -> Data Warehouse Connection -> Data Warehouse Snapshot Time, but midnight (off hours) is best (as it does put some stress on the applications), unless you have some scheduled maintenance that you know will interfere with the job (ie - database goes down at midnight for some maintenance task). I'm glad the jobs are running now! Comments So the jobs will run automatically at midnight? Do i need to set the account that it uses or is that controlled automatically by RTC?
Can the ID be reset to one of the 'dummy' user IDs ? (e.g. ccm_user) We just moved to 4.0.1 and our dw jobs (in ccm ) are failing with "peer not authenticated". The ID being used doesn't have any collector license and it's an LDAP ID that we've setup for lots of other purposes. Having its password in lots of places makes me twitchy (not for security, but just the possibility of overlooking if the password should ever be reset).
Benjamin Silverman
commented Dec 12 '12, 1:52 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
I would advise against using one of the other functional users (ie - ccm_user) as the ETL user. Those users are dedicated to other tasks and should not be designated for other activities.
|
|
Benjamin Silverman (4.1k●6●10)
| answered Oct 22 '12, 7:13 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
Yes, they'll run automatically at midnight every night. You can check the job status some time after midnight to confirm they've executed. The account that it uses is the one you have set up under Reports -> Data Collection Jobs, which I believe you said is your account for now. You may want to create a functional user (ie - ETLUSER) and assign it the data collector license as to not interfere with the ETL (ie - you change your password, then the ETL's will start failing).
|
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.