ETL Data update problem

VK L (817●7●148●159) | asked May 10 '13, 9:22 a.m.
retagged May 10 '13, 9:40 a.m. by Krzysztof Kaźmierczyk (7.4k●3●56●99)

Noticed that data is not getting updated into the DW past 2 days. Till then it was running fine.

Jobs are completing and no errors - but the changes made in RTC are not reflecting in the Warehouse tables. Even when queried from DW with modified_since parameter, it is giving 0 rows. What could be the issue?

How to check whether any changes have gone in?

Thanks.

3 answers

Most liked answers ↑|Newest answers|Oldest answers

permanent link

Indradri Basu (1.8k●1●5●14) | answered May 10 '13, 10:40 a.m.

Hi Valli, you should check the <app>_etl.log file. If data is coming over to the DW database then you should find some similar entries in the log as shown below

ETL: Records Selected: 7
ETL: Records Inserted: 7
ETL: Records Updated: 0
ETL: Records Ignored: 0

permanent link

VK L (817●7●148●159) | answered May 10 '13, 10:44 a.m.

Hi Indrani,

The Logs are showing 0 inserts, 0 updates. But the problem is that there is data in the source, modified in the past-2 days for example, but somehow the ETL is not picking up these. It was working well earlier though.

Scenario:

The values for some attributes are changed in RTC Workitem. Insight DW is still showing data as of May 7th. the last 2 days Changes are not getting updated through the ETL

Comments

Indradri Basu commented May 10 '13, 10:58 a.m.

The next thing I would probably check is if anything has changed in the Datawarehouse connection page. I would also check the systemout.log and etl logs for any errors. See if the password of the etl_user has expired or not. Try running the Full Data collection jobs manually and check the logs again and see if you can spot an error or a problem

VK L commented May 10 '13, 11:08 a.m.

Are you mentioning about datawarehouse connection page on RTC-Admin part?

Indradri Basu commented May 10 '13, 1:13 p.m.

Yes, and you can also see if your teamserver.properties files has been modified recently.

VK L commented May 11 '13, 4:43 a.m.

Full Load is also not making any updates. There are a few connection errors on the database log - but not sure if its related to this. I am able to connect to the database through the DB2 Admin center.

There are no related errors in systemout.log, cognos logs. no changes to teamserver file, dw connections.

permanent link

VK L (817●7●148●159) | answered May 10 '13, 10:48 a.m.

ri_jdbc.log doesnt give any errors as such - but, when i try in data manager -> querying on data source, its retrieving o rows suddenly for past dates as well. Not sure what has changed. Any ideas on what could be the cause and how to go about it?

Comments

Indradri Basu commented May 10 '13, 11:00 a.m.

If you can't spot any problem based on my comment above, I would suggest to check your database logs for any errors.

VK L commented May 13 '13, 12:55 a.m.

Is it required that clmdwadmin id should be present in JazzAdmin group for the ETL Jobs to be able to retrieve data from RTC? Recently i had removed the mapping from JazzAdmin group on the assumption that only JazzDWAdmin privilege is required. As per my checks, this looks to have caused the issue.

Mapped it to JazzAdmin group again and its working fine now.

Indradri Basu commented May 13 '13, 2:36 a.m.

You got it right. the etl_user must be in JazzAdmin group and I suppose subscription in JazzDWAdmin is no longer mandatory. Check this infocenter link
However this should have reflected in the logs but glad to know things are on track now.

VK L commented May 13 '13, 2:40 a.m.

Thanks. Is this the case in 3.0.1.1 as well? I was of the idea that JazzDWAdmin is the group related to Data warehouse [Jobs,etc].

It's all about the answers!

Related questions

ETL Data update problem

3 answers

Your answer