6.0.6 iFix009 - Throwable: Callee Path Warning
Ever since applying iFix009, we are seeing numerous instances of this warning. What does this mean? Is there something we should do in response?
[ WebContainer : 10] WARN server.core.jpi.internal.JAFBulkPartitionerFactory - lockGraphRetryHandler: Failure of lock bulk retry operation that was initiated by exception of type class com.ibm.team.jfs.sdk.common.ResourceNotFoundException caught while handling resource p_99RZkLSQEemG78d2uNB8xwg91 in storage area com.ibm.rdm.locks; Retry Count: 6; Current Stack Trace of Callee: java.lang.Throwable: Callee Path
at com.ibm.rdm.fronting.server.core.jpi.internal.JAFBulkPartitionerFactory$6.call(JAFBulkPartitionerFactory.java:359) at com.ibm.rdm.fronting.server.core.jpi.internal.JAFBulkPartitionerFactory$6.call(JAFBulkPartitionerFactory.java:1) at com.ibm.rdm.fronting.server.core.jpi.internal.JAFBulkPartitionerFactory$1.handleException(JAFBulkPartitionerFactory.java:145) at com.ibm.team.jfs.sdk.storage.internal.StorageService.handleBulkException(StorageService.java:582) at com.ibm.team.jfs.sdk.storage.internal.StorageService$3.handleException(StorageService.java:552) at com.ibm.team.jfs.resource.service.internal.ResourceService.bulkFetch(ResourceService.java:790) at sun.reflect.GeneratedMethodAccessor735.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56) at java.lang.reflect.Method.invoke(Method.java:620) at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:361) at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:347) at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56) at com.sun.proxy.$Proxy760.bulkFetch(Unknown Source) at com.ibm.team.jfs.sdk.storage.internal.StorageService.fetchResources(StorageService.java:511) at com.ibm.team.jfs.sdk.storage.internal.StorageService.bulkRetrieveResources(StorageService.java:700) at sun.reflect.GeneratedMethodAccessor736.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56) at java.lang.reflect.Method.invoke(Method.java:620) at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:361) at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:347) at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56) at com.sun.proxy.$Proxy811.bulkRetrieveResources(Unknown Source) at com.ibm.rdm.fronting.server.core.jpi.internal.JAFBulkPartitionerFactory.handleBulkRequests(JAFBulkPartitionerFactory.java:222) at com.ibm.rdm.fronting.server.core.jpi.internal.OrdinaryBulkPartitionerFactory.execute(OrdinaryBulkPartitionerFactory.java:72) at com.ibm.rdm.fronting.server.core.jpi.internal.OrdinaryBulkPartitionerFactory$1OrdinaryBulkPartition.call(OrdinaryBulkPartitionerFactory.java:58) at com.ibm.rdm.fronting.server.core.jpi.internal.OrdinaryBulkPartitionerFactory$1OrdinaryBulkPartition.call(OrdinaryBulkPartitionerFactory.java:1) at java.util.concurrent.FutureTask.run(FutureTask.java:274) at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:144) at com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:58) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.handleOrdinaryBulkables(InternalCoreServices.java:796) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.handleExecutables(InternalCoreServices.java:733) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:409) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.executeRequestSequences(InternalCoreServices.java:1149) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.handleRequestSequences(InternalCoreServices.java:1097) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:364) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.executeRequestSequences(InternalCoreServices.java:1149) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.handleRequestSequences(InternalCoreServices.java:1097) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:364) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.executeRequestSequences(InternalCoreServices.java:1149) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.handlePreProcessing(InternalCoreServices.java:649) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:360) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:260) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:253) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.bulk(InternalCoreServices.java:235) at com.ibm.rdm.fronting.server.core.CoreServices.bulk(CoreServices.java:148) at com.ibm.rdm.fronting.server.core.CoreServices.bulk(CoreServices.java:107) at com.ibm.rdm.fronting.server.services.multiRequest.MultiRequestService.processBulkHttpRequests(MultiRequestService.java:504) at com.ibm.rdm.fronting.server.services.multiRequest.MultiRequestService.processRdfXmlRequest(MultiRequestService.java:202) at com.ibm.rdm.fronting.server.services.multiRequest.MultiRequestService.doPost(MultiRequestService.java:161) at com.ibm.rdm.fronting.server.services.RMRestService.doPostInternal(RMRestService.java:95) at com.ibm.rdm.fronting.server.services.RMRestService.runInternalMethod(RMRestService.java:484) at com.ibm.rdm.fronting.server.services.RMRestService.handleMethod(RMRestService.java:412) at com.ibm.rdm.fronting.server.services.RMRestService.performService(RMRestService.java:348) at com.ibm.rdm.fronting.server.services.RMRestServiceBase.service(RMRestServiceBase.java:230) at com.ibm.team.jfs.app.servlet.AppContainerServlet.dispatchRequest(AppContainerServlet.java:155) at com.ibm.team.jfs.app.servlet.AppContainerServlet.service(AppContainerServlet.java:282) at javax.servlet.http.HttpServlet.service(HttpServlet.java:668) at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleAppSdkRestServiceRequest2(AbstractTeamServerServlet.java:2110) at com.ibm.team.repository.servlet.AbstractTeamServerServlet.access$2(AbstractTeamServerServlet.java:2055) at com.ibm.team.repository.servlet.AbstractTeamServerServlet$2.service(AbstractTeamServerServlet.java:271) at com.ibm.team.repository.service.internal.compatibility.auth.DelegatedAuthHandlerService$1.run(DelegatedAuthHandlerService.java:264) at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleAuthentication(AbstractTeamServerServlet.java:2166) at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleAppSdkRestServiceRequest(AbstractTeamServerServlet.java:2031) at com.ibm.team.repository.servlet.AbstractTeamServerServlet.service(AbstractTeamServerServlet.java:1768) at javax.servlet.http.HttpServlet.service(HttpServlet.java:668) at org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180) at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61) at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128) at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:76) at javax.servlet.http.HttpServlet.service(HttpServlet.java:668) at org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:152) at com.ibm.team.repository.server.servletbridge.JazzServlet.service(JazzServlet.java:74) at javax.servlet.http.HttpServlet.service(HttpServlet.java:668) at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1232) at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:781) at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:480) at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178) at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136) at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97) at com.ibm.team.repository.server.servletbridge.BridgeFilter.processDelegate(BridgeFilter.java:165) at com.ibm.team.repository.server.servletbridge.BridgeFilter.doFilter(BridgeFilter.java:198) at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195) at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91) at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:967) at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1107) at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:87) at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:947) at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1817) at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:200) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:463) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:530) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:316) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:287) at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1084) at com.ibm.ws.ssl.channel.impl.SSLConnectionLink$MyReadCompletedCallback.complete(SSLConnectionLink.java:658) at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1820) at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175) at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217) at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161) at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138) at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204) at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775) at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905) at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1892) |
Accepted answer
2 other answers
Hello,
After looking into this exception a bit, it was determined that it is most likely unnecessary logging content. This message does not necessarily mean that there is an issue. Most likely the exception will occur in absence of any actual product defect.
There are further details in the defect, but I think you can safely ignore it. In short, there are cases where this warning message may be logged when there is no issue and the product is working as designed. If this exception is spamming your logs consistently, you may want to turn it off by adding the following line to the DOORS Next log4j.properties file and restarting the server:
log4j.logger.com.ibm.rdm.fronting.server.core.jpi.internal.JAFBulkPartitionerFactory=ERROR
This will cause that logger to only produce log lines when the message is at severity-level ERROR or higher (like FATAL). In this case, the logger only has other DEBUG-level messages that are not needed, so the noise should go away without losing any important logging.
Alternatively, If you do not want to restart the server and you're a JazzAdmin, you should be able to set the log4j setting immediately by going to:
Gearbox drop-down -> "Manage Server Administration" -> Debug tab -> Select the "Logging" option in the left panel
-> click "Configure loggers".
Scroll to the bottom of that page and there should be a text-box form under the text "Text-based log configuration". From here, you can add the line as well and push the submit button. This should update the log4j.properties file and apply the settings immediately without a server restart.
Although the defect RM log filled with ResourceNotFoundException from lockGraphRetryHandler: Failure of lock bulk retry operation... (131600) at comment 9 explains a way to cause this log message to spam the logs unnecessarily, there may exist a case where the warning is an issue, but I'm not aware of one at the moment. If you think you have found one, feel free to open an escalation for a more detailed analysis.
I'm not sure if this message will be disabled (by default) in a future ifix, but we will investigate that option. The workaround mentioned above should turn it off for you for now.
Comments Thanks much Chase! I'm going to try that. I've been having issues lately with the log being so large it takes a really long time to load. Maybe this will help. |
|
We closed the defect RM log filled with ResourceNotFoundException from lockGraphRetryHandler: Failure of lock bulk retry operation... (131600). This issue is not applicable to the 7.x stream.
The log4j.properties change that we recommended to prevent this message from spamming your logs will not be necessary in a future ifix release. We are delivering a fix to 6.0.6.1 ifix and 6.0.6 ifix streams. If you are curious to see which ifix this gets delivered in, follow:
We are reusing a pre-existing maintenance item since this log message was introduced in an ifix change.
Comments Thanks Chase. I should mention we do not have CM turned on in our application, but I'm assuming that iFix014 will fix the problem regardless.
Carol
|
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.