Data Collection Job Status fails

0

permanent link

Benjamin Silverman (4.1k●6●10) | answered Oct 22 '12, 3:45 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

Hi Rodney,

The best place to start looking is in the ETL logs. These should be located in the server install directory in the logs folder (ie - C:\IBM\JazzTeamServer\server\logs). You should see the following logs:

ccm-etl.log for CCM
qm-et.log for QM
jts-etl.log for JTS and RM

You can also run the ISALite Data Collector and provide it in your PMR by going to C:\IBM\JazzTeamServer\support\ISALite and running "runISALite.bat" which will collect all the logs from the server as well as some other helpful information for support. Hope it helps,

Comments

Rodney Haymore commented Oct 22 '12, 3:52 p.m.

Thanks. I'll look there. Anything in particular that I should look for?

Unfortunately our RTC is installed on a closed network and the output of the runISALite batch file produces a ton of files that makes it practically impossible to get those files off the network and to IBM without going through a lengthy extraction process (classification).

0

permanent link

Benjamin Silverman (4.1k●6●10) | answered Oct 22 '12, 3:57 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

The log captures "ERROR" messages by default, but can be re-configured in the ETL section of the log4j.properties if you need to finer tune the logging to capture more information. To start, I'd look for any "ERROR" type messages and provide the details here if you can. Support however may ask you re-run the ETL with the log configured for "DEBUG" if there are no errors, or the errors do not reveal what the problem is. It is also possible that exceptions are thrown in the application log (ie - rm.log, qm.log, etc...), which is why I thought the ISALite Data Collection would be helpful, although I certainly understand your policy may prohibit you from sharing the details.

Comments

Rodney Haymore commented Oct 22 '12, 4:11 p.m.

(ccm-etl.log)Lots of ERRORS...
Authentecation Exception CRRRE1417E the HTTP status '403' and the masseage is forbidden...java SQLException.

One note...The STAR job is successful. The Requirements and Common jobs fails.

0

permanent link

Benjamin Silverman (4.1k●6●10) | answered Oct 22 '12, 4:21 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

Does the user that is configured to run ETL have the "Data Collector" license? Is that the same exception you see in the jts-etl.log, or is there more info there? That might explain why only the STAR job works, as this data is collected from the data in the database rather than from the applications,

Comments

Rodney Haymore commented Oct 22 '12, 4:37 p.m.

Yes it is the same error, but more.

Datawarehouse ...RemoteRRCSnapshotservice has failed. Aborting all the jobs.

We have one Data Collector license and I've set attached it to my Admin account. I'll try setting it to he user's to see what luck he has.

0

permanent link

Benjamin Silverman (4.1k●6●10) | answered Oct 22 '12, 4:48 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

That might be the problem. The ETL is configured to run with a certain user if you go to https://<server>:<port>/jts/admin -> Reports -> Data Collection Jobs, and check the "User ID" who is running them. That user is created during jts/setup, so if you've attached the Data Collector license to any other user, you'll need to change the user that runs the ETL to make sure it's the user that has that license. You'll need to do that for each application.

Comments

Rodney Haymore commented Oct 22 '12, 5:00 p.m.

How do I check the user id who is running them?

All I see is: Data Collection Job, State, Comsumer Key, Consumer Secret, Actions

Do I have to decode the Consumer Key?

0

permanent link

Benjamin Silverman (4.1k●6●10) | answered Oct 22 '12, 5:02 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

If you go to https://<server>:<port>/jts/admin -> Reports -> Data Collection Jobs, you'll see each job listed for the JTS/RM applications, and there is a "User ID" field that indicates the user configured to run the ETL. Again, you'll have to update this under each admin page (ie - qm/admin -> Reports -> Data Collection Jobs, ccm/admin -> Reports -> Data Collection Jobs) for the change to be made for each application.

Comments

Rodney Haymore commented Oct 22 '12, 5:37 p.m.

Not Seeing User ID. I have a screenshot but an unable to upload at this time.

"sorry, file uploading requires reputation >60"

Is there some other way to get the file to you?

Rodney Haymore commented Oct 22 '12, 5:47 p.m.

If I select the Data Warehouse Connection, I do see some configurations settings like Data Collection User or Consumer key. But this field is not changeable.

0	permanent link Benjamin Silverman (4.1k●6●10) \| answered Oct 22 '12, 6:00 p.m. FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Sure, you can email it to me at bsilverm<at>us<dot>ibm<dot>com. Or I can attach a screenshot of mine here if it helps:

0

permanent link

Benjamin Silverman (4.1k●6●10) | answered Oct 22 '12, 6:26 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

I'm using 4.0 and the screenshot I showed is the default. Is your XDC Authentication type set to "oauth" instead of "FORM"? You can check under Reports -> Data Warehouse Connection -> XDC Authentication, or under jts/admin -> Server -> Advanced Properties -> XDC Authentication. I'd try changing it to FORM and specifying the username/password for your ETL user, as using a consumer key/secret is not the default configuration.

Comments

Rodney Haymore commented Oct 22 '12, 6:32 p.m.

It was set to JTS. I changed it to Form and now on the Data Collection Jobs screen the field indicates User ID and Password but it's still showing the consumer key not the User ID.

Rodney Haymore commented Oct 22 '12, 6:35 p.m.

Is the ETL user the same as the Data Collection User that's shown on the Warehouse Connections screen or should it be the User who is trying to run the reports?

Is the Warehouse Connection user (ADMIN) the account that automatically runs the Jobs everyday as opposed to the user who manually runs the jobs on demand?

0

permanent link

Benjamin Silverman (4.1k●6●10) | answered Oct 22 '12, 6:37 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

You should be able to update the User ID/password on that page to reflect the user who has the Data Collector license. Are you not able to modify the values?

Comments

Rodney Haymore commented Oct 22 '12, 6:41 p.m.

Okay. I changed the User ID to my login User ID and the jobs are now running. The first few have already passed.

How is this supposed to work in real life? Will I have to run the jobs whenever the user needs reports or is there some way to allow users to run the jobs themselves?

We only have one Data Collector license.

In any case, Thank you very much for your support.

0

permanent link

Benjamin Silverman (4.1k●6●10) | answered Oct 22 '12, 6:50 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

Yes, you only need one Data Collector license since you only need one functional ETL user on the server. These jobs run by default at midnight, so the users should find that reports are updated with the latest data on the following day without you having to run them manually.

You can re-configure the time they run by entering an integer value under Reports -> Data Warehouse Connection -> Data Warehouse Snapshot Time, but midnight (off hours) is best (as it does put some stress on the applications), unless you have some scheduled maintenance that you know will interfere with the job (ie - database goes down at midnight for some maintenance task).

I'm glad the jobs are running now!

Comments

Rodney Haymore commented Oct 22 '12, 6:59 p.m.

So the jobs will run automatically at midnight? Do i need to set the account that it uses or is that controlled automatically by RTC?

Kevin Ramer commented Dec 12 '12, 1:48 p.m.

Can the ID be reset to one of the 'dummy' user IDs ? (e.g. ccm_user) We just moved to 4.0.1 and our dw jobs (in ccm ) are failing with "peer not authenticated". The ID being used doesn't have any collector license and it's an LDAP ID that we've setup for lots of other purposes. Having its password in lots of places makes me twitchy (not for security, but just the possibility of overlooking if the password should ever be reset).

TIA -- Kevin

Benjamin Silverman commented Dec 12 '12, 1:52 p.m.

FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

I would advise against using one of the other functional users (ie - ccm_user) as the ETL user. Those users are dedicated to other tasks and should not be designated for other activities.

0

permanent link

Benjamin Silverman (4.1k●6●10) | answered Oct 22 '12, 7:13 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

Yes, they'll run automatically at midnight every night. You can check the job status some time after midnight to confirm they've executed. The account that it uses is the one you have set up under Reports -> Data Collection Jobs, which I believe you said is your account for now. You may want to create a functional user (ie - ETLUSER) and assign it the data collector license as to not interfere with the ETL (ie - you change your password, then the ETL's will start failing).

It's all about the answers!

Related questions

Data Collection Job Status fails

12 answers

Your answer