What to look for when Scheduled Jobs quit running ?
QM 4.0.7 with build integration to RTC 4.0.7. User reports that late last week ( week of Jan 19 ) completed builds do not trigger a QM Scheduled job. Build results are coming into QM in a timely fashion. The only error I'm seeing on the QM application are attempts to insert into EXECUTION.EXECUTION_VARIABLES with what is claimed to be an existing NAME_COL.
Looking at the DDL for this table shows this alter command:
I'm searching old logs for the first appearance of this failure and will update accordingly. But meantime, is the above possibly related to failing of scheduled jobs ?. Executing the test "by hand" completes.
Thanks
APPEND: The itemId UUID changes with every logging of the INSERT failure.
APPEND2: ran the onlineVerify. The only thing flagged is:
Caused by: com.ibm.team.repository.common.InternalRepositoryException: CRJAZ0343I Executing executeBatch failed. The number of results returned was "2".SQL: INSERTINTO EXECUTION.VARIABLE_NAME (STATE_ID, ITEM_ID, CONTEXT_ID, MODIFIED, MODIFIED_BY_ITEM_ID, NAME_COL) VALUES (?, ?, ?, ?, ?, ?)SQL Exception #1SQL Message: [jcc][t4][102][10040][4.14.121] Batch failure. The batch was submitted, but at least one exception occurred on an individual member of the batch.Use getNextException() to retrieve the exceptions for specific batched elements. ERRORCODE=-4229, SQLSTATE=nullSQL State: nullError Code: -4229SQL Exception #2SQL Message: Error for batch element #2: DB2 SQL Error: SQLCODE=-803, SQLSTATE=23505, SQLERRMC=6;EXECUTION.VARIABLE_NAME, DRIVER=4.14.121SQL State: 23505Error Code: -803Exception Details: CRJAZ0250I A statement attempted to insert a duplicate value in a table that requires unique values. For more information about the statement that failed, check the server logs.Class: com.ibm.team.repository.service.internal.dataaccess.write2.RowItem Handle: com.ibm.rqm.execution.common.model.impl.VariableNameImpl@d01a7c4b (stateId: [UUID _u1be1aILEeSEJPzDk4R2rA], itemId: [UUID _u1be1KILEeSEJPzDk4R2rA], origin: <unset>, immutable: true) (contextId: [UUID _8lNyYNwSEd2pIJ5QVwgQGg], modified: 2015-01-22 02:53:19.177, workingCopy: false) (predecessor: <unset>) (name: NOTEST)
Looking at the DDL for this table shows this alter command:
ALTER TABLE "EXECUTION"."VARIABLE_NAME"_u1be1KILEeSEJPzDk4R2rA does not exist for any ITEM_ID in the aforementioned table.
ADD CONSTRAINT "VARIABLE_NAME_PK6" PRIMARY KEY ("ITEM_ID");
I'm searching old logs for the first appearance of this failure and will update accordingly. But meantime, is the above possibly related to failing of scheduled jobs ?. Executing the test "by hand" completes.
Thanks
APPEND: The itemId UUID changes with every logging of the INSERT failure.
APPEND2: ran the onlineVerify. The only thing flagged is:
2015-01-28 12:18:54,713 8.060345%: Verifying the queryable tables for item type: com.ibm.team.markers#Marker2015-01-28 12:18:56,879 The Item with ItemID "_UpxIsKV4EeSJn72oZyeeGA" and StateId "_vJ5voqcREeSJoL2oZyeeGA" was found in the ITEM_CURRENTS table, but the same Item had a different StateId, "_vFiNBacREeSJoL2oZyeeGA", in the Item query table2015-01-28 12:18:56,879 8.060345%: Verifying the queryable tables for item type: com.ibm.team.markers#Marker2015-01-28 12:19:09,176 8.103448%: Verifying the queryable tables for item type: com.ibm.team.markers#Marker
Accepted answer
Please open a PMR.