DCC not collecting all requirements / missing data?
Hello,
DCC v5.0.2 iFix 014, Oracle db, DCC and JRS on their own WAS/RHEL server, RTC/DNG/RQM each on their own RHEL/WAS servers.
Client was having some serious data issues, with items duplicated and missing and all kinds of corruption. To remedy with a fresh start, we dropped the entire Data Warehouse from Oracle and set DCC to do a Full job. This had a few results:
a) The IDs for projects (and likely other things) changed, meaning all JRS reports had to have the projects re-selected and any manually run SQL data warehouse pulls had to have their IDs updated. I expected this sort of behaviour from prior experience with Insight, just noting it for others. Any ID that isn't directly from CLM is not guaranteed to be static during a refresh operation.
b) Some larger jobs like History were failing; we think this has been corrected by increasing the temp space in Oracle. Perhaps this impacted other jobs in a non-failing error?
c) There are lots of artifacts missing from the data warehouse now. A DNG query that returns ~1500 requirements only returns a couple hundred from the data warehouse. If I just pull from VW_REQUIREMENTS where the ID < 100, I get only a handful instead of our first hundred artifacts. I'm requesting DB access now to see if the ROIDS.REQUIREMENTS table has all our data and it somehow isn't making it into the view (I've seen issues with the soft_deleted value being incorrectly set to 1 before, though I thought that was just passed to VW_REQUIREMENTS anyways).
C is the real problem here. CCM seems fine, I'll check QM, DNG is just wildly under-collected. I am considering setting DNG to a Full ETL and running them all again; perhaps now that the temp space is expanded it won't encounter such issues? However, I have fears that maybe it's improperly processing some custom attributes or something like that. There's usually more delivered than processed in the DCC result overview for DNG jobs, but I don't know how significant that is. I am aware v6 improves DCC but unfortunately we cannot quite yet upgrade, it'll be a month or two.
4 answers
Hello June,
one thing that can explain the difference in C could be the presence of archived project areas / artifacts.
For example if you have a project A and you archive it, it will be marked as soft deleted in the data warehouse.
However if you drop the data warehouse and recreate it from scratch the data from the archived elements won't be loaded again.
So I would recommend to check what you have in RIODS.REQUIREMENT against what you have in DNG active projects, to understand if everything is there.
Best Regards,
Francesco Chiossi
one thing that can explain the difference in C could be the presence of archived project areas / artifacts.
For example if you have a project A and you archive it, it will be marked as soft deleted in the data warehouse.
However if you drop the data warehouse and recreate it from scratch the data from the archived elements won't be loaded again.
So I would recommend to check what you have in RIODS.REQUIREMENT against what you have in DNG active projects, to understand if everything is there.
Best Regards,
Francesco Chiossi
Hi,
I'm having a similar issue. DNG requirements are present in the RIODS.REQUIREMENT table, but RRDI does not see the requirements or project area. JRS shows the active project, but nothing returns when running the report (1942 known requirements). RRDI query studio and report studio do not list the active project (Reporting Data Model (DW) -> Operational Data Store -> Requirement Area -> Requirement -> Project Name).
Setup: DCC v5.0.2 iFix 015, Oracle DB (11.2.0.4), DCC and RRDI are on their own WAS (8.5.5) / RHEL (6.7) server, RTC/DNG/RQM on one RHEL/WAS server (JTS).
History: Upgraded from version 3.0.1.1 to 5.0.2 on 15 June 2016.
Any help is greatly appreciated.
v/r
James Freeman
Comments
showing 5 of 6
show 1 more comments
Hello James,
if the requirements are present in the RIODS.REQUIREMENT table but not visible in the reports (JRS or RRDI) there are 2 possible causes that comes to my mind:
1) The requirements are masked as deleted. Check what is the value of the ISSOFTDELETED column in RIODS.REQUIREMENT; if it's 1 then the requirement is marked as deleted and will not show up in the reports.
2) Your user don't have the project access control permissions to see the data.
This can happen if you have project access control enabled and your user:
- is not a member of the project area
- does not have a valid email address in CLM
Best Regards
Francesco Chiossi
if the requirements are present in the RIODS.REQUIREMENT table but not visible in the reports (JRS or RRDI) there are 2 possible causes that comes to my mind:
1) The requirements are masked as deleted. Check what is the value of the ISSOFTDELETED column in RIODS.REQUIREMENT; if it's 1 then the requirement is marked as deleted and will not show up in the reports.
2) Your user don't have the project access control permissions to see the data.
This can happen if you have project access control enabled and your user:
- is not a member of the project area
- does not have a valid email address in CLM
Best Regards
Francesco Chiossi
Comments
We have similar problems after upgrade from 502 ix14 to 602 ifix 3. Single CLM install on RH w/Oracle backend.
In 2 of our projects, we have missing/duplicate data or no data at all. These projects were not deleted/archived (although we have a number of archived RM projects.) We had to wait for Ifix 2 to address migration problems due to archived projects.
I have a PMR in support on our RM problems post upgrade but have yet to hear back from them.
One suspicion I have is that there was some data corruption in 502 usage resulting from stalled doc imports. One problem we had as causing stalled imports was lack of disk space for the index dir. The only recourse to get this working again was to restart JAZZ. I think this is were the data corruption occurred. The app however appeared to function without issue. Support also had us run a bunch of utilities on rm (verify, validate module etc..) just prior to the upgrade and noted no issues.
Not much help in resolving your issue but your problem does not appear to be unique.
Norm
In 2 of our projects, we have missing/duplicate data or no data at all. These projects were not deleted/archived (although we have a number of archived RM projects.) We had to wait for Ifix 2 to address migration problems due to archived projects.
I have a PMR in support on our RM problems post upgrade but have yet to hear back from them.
One suspicion I have is that there was some data corruption in 502 usage resulting from stalled doc imports. One problem we had as causing stalled imports was lack of disk space for the index dir. The only recourse to get this working again was to restart JAZZ. I think this is were the data corruption occurred. The app however appeared to function without issue. Support also had us run a bunch of utilities on rm (verify, validate module etc..) just prior to the upgrade and noted no issues.
Not much help in resolving your issue but your problem does not appear to be unique.
Norm
Comments
