It's all about the answers!

Ask a question

6.0.6 iFix009 - Throwable: Callee Path Warning


Carol Watson (5966) | asked Sep 05 '19, 9:59 a.m.

 Ever since applying iFix009, we are seeing numerous instances of this warning. What does this mean?  Is there something we should do in response? 


[ WebContainer : 10] WARN server.core.jpi.internal.JAFBulkPartitionerFactory - lockGraphRetryHandler: Failure of lock bulk retry operation that was initiated by exception of type class com.ibm.team.jfs.sdk.common.ResourceNotFoundException caught while handling resource p_99RZkLSQEemG78d2uNB8xwg91 in storage area com.ibm.rdm.locks; Retry Count: 6; Current Stack Trace of Callee: java.lang.Throwable: Callee Path
at com.ibm.rdm.fronting.server.core.jpi.internal.JAFBulkPartitionerFactory$6.call(JAFBulkPartitionerFactory.java:359)
at com.ibm.rdm.fronting.server.core.jpi.internal.JAFBulkPartitionerFactory$6.call(JAFBulkPartitionerFactory.java:1)
at com.ibm.rdm.fronting.server.core.jpi.internal.JAFBulkPartitionerFactory$1.handleException(JAFBulkPartitionerFactory.java:145)
at com.ibm.team.jfs.sdk.storage.internal.StorageService.handleBulkException(StorageService.java:582)
at com.ibm.team.jfs.sdk.storage.internal.StorageService$3.handleException(StorageService.java:552)
at com.ibm.team.jfs.resource.service.internal.ResourceService.bulkFetch(ResourceService.java:790)
at sun.reflect.GeneratedMethodAccessor735.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:361)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:347)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
at com.sun.proxy.$Proxy760.bulkFetch(Unknown Source)
at com.ibm.team.jfs.sdk.storage.internal.StorageService.fetchResources(StorageService.java:511)
at com.ibm.team.jfs.sdk.storage.internal.StorageService.bulkRetrieveResources(StorageService.java:700)
at sun.reflect.GeneratedMethodAccessor736.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:361)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:347)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
at com.sun.proxy.$Proxy811.bulkRetrieveResources(Unknown Source)
at com.ibm.rdm.fronting.server.core.jpi.internal.JAFBulkPartitionerFactory.handleBulkRequests(JAFBulkPartitionerFactory.java:222)
at com.ibm.rdm.fronting.server.core.jpi.internal.OrdinaryBulkPartitionerFactory.execute(OrdinaryBulkPartitionerFactory.java:72)
at com.ibm.rdm.fronting.server.core.jpi.internal.OrdinaryBulkPartitionerFactory$1OrdinaryBulkPartition.call(OrdinaryBulkPartitionerFactory.java:58)
at com.ibm.rdm.fronting.server.core.jpi.internal.OrdinaryBulkPartitionerFactory$1OrdinaryBulkPartition.call(OrdinaryBulkPartitionerFactory.java:1)
at java.util.concurrent.FutureTask.run(FutureTask.java:274)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:144)
at com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:58)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.handleOrdinaryBulkables(InternalCoreServices.java:796)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.handleExecutables(InternalCoreServices.java:733)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:409)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.executeRequestSequences(InternalCoreServices.java:1149)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.handleRequestSequences(InternalCoreServices.java:1097)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:364)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.executeRequestSequences(InternalCoreServices.java:1149)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.handleRequestSequences(InternalCoreServices.java:1097)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:364)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.executeRequestSequences(InternalCoreServices.java:1149)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.handlePreProcessing(InternalCoreServices.java:649)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:360)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:260)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:253)
at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.bulk(InternalCoreServices.java:235)
at com.ibm.rdm.fronting.server.core.CoreServices.bulk(CoreServices.java:148)
at com.ibm.rdm.fronting.server.core.CoreServices.bulk(CoreServices.java:107)
at com.ibm.rdm.fronting.server.services.multiRequest.MultiRequestService.processBulkHttpRequests(MultiRequestService.java:504)
at com.ibm.rdm.fronting.server.services.multiRequest.MultiRequestService.processRdfXmlRequest(MultiRequestService.java:202)
at com.ibm.rdm.fronting.server.services.multiRequest.MultiRequestService.doPost(MultiRequestService.java:161)
at com.ibm.rdm.fronting.server.services.RMRestService.doPostInternal(RMRestService.java:95)
at com.ibm.rdm.fronting.server.services.RMRestService.runInternalMethod(RMRestService.java:484)
at com.ibm.rdm.fronting.server.services.RMRestService.handleMethod(RMRestService.java:412)
at com.ibm.rdm.fronting.server.services.RMRestService.performService(RMRestService.java:348)
at com.ibm.rdm.fronting.server.services.RMRestServiceBase.service(RMRestServiceBase.java:230)
at com.ibm.team.jfs.app.servlet.AppContainerServlet.dispatchRequest(AppContainerServlet.java:155)
at com.ibm.team.jfs.app.servlet.AppContainerServlet.service(AppContainerServlet.java:282)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleAppSdkRestServiceRequest2(AbstractTeamServerServlet.java:2110)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.access$2(AbstractTeamServerServlet.java:2055)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet$2.service(AbstractTeamServerServlet.java:271)
at com.ibm.team.repository.service.internal.compatibility.auth.DelegatedAuthHandlerService$1.run(DelegatedAuthHandlerService.java:264)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleAuthentication(AbstractTeamServerServlet.java:2166)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleAppSdkRestServiceRequest(AbstractTeamServerServlet.java:2031)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.service(AbstractTeamServerServlet.java:1768)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:76)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:152)
at com.ibm.team.repository.server.servletbridge.JazzServlet.service(JazzServlet.java:74)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1232)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:781)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:480)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97)
at com.ibm.team.repository.server.servletbridge.BridgeFilter.processDelegate(BridgeFilter.java:165)
at com.ibm.team.repository.server.servletbridge.BridgeFilter.doFilter(BridgeFilter.java:198)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:967)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1107)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:87)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:947)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1817)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:200)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:463)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:530)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:316)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:287)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1084)
at com.ibm.ws.ssl.channel.impl.SSLConnectionLink$MyReadCompletedCallback.complete(SSLConnectionLink.java:658)
at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1820)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1892)

Accepted answer


permanent link
Rosa Naranjo (2.8k11420) | answered Sep 11 '19, 7:32 a.m.
FORUM MODERATOR / JAZZ DEVELOPER
Hello
I opened a defect for this. Read the description for more details

Carol Watson selected this answer as the correct answer

Comments
Carol Watson commented Sep 11 '19, 9:18 a.m.

Thank you Rose.  I read through the work item and am not quite clear on what's meant by a "cloned module".  Where I'm seeing this occur is in modules that were migrated from DOORS to DNG, and which have numerous (custom) links to other modules in the same Project. 

Regards,
Carol


Rosa Naranjo commented Sep 11 '19, 9:23 a.m.
FORUM MODERATOR / JAZZ DEVELOPER
Cloning is described here


We support cloning and copying.

If you can write this up with specific steps to reproduce, Expected results, Actual results, you can open a support case for further investigation.

You can include a ReqIF and a project template as well for us to try to reproduce internally.

2 other answers



permanent link
Chase Patterson (362) | answered Jan 07, 10:45 a.m.
edited Jan 07, 10:54 a.m.

Hello, 


After looking into this exception a bit, it was determined that it is most likely unnecessary logging content. This message does not necessarily mean that there is an issue. Most likely the exception will occur in absence of any actual product defect.

There are further details in the defect, but I think you can safely ignore it. In short, there are cases where this warning message may be logged when there is no issue and the product is working as designed. If this exception is spamming your logs consistently, you may want to turn it off by adding the following line to the DOORS Next log4j.properties file and restarting the server:

log4j.logger.com.ibm.rdm.fronting.server.core.jpi.internal.JAFBulkPartitionerFactory=ERROR

This will cause that logger to only produce log lines when the message is at severity-level ERROR or higher (like FATAL). In this case, the logger only has other DEBUG-level messages that are not needed, so the noise should go away without losing any important logging.

Alternatively, If you do not want to restart the server and you're a JazzAdmin, you should be able to set the log4j setting immediately by going to:
 Gearbox drop-down ->  "Manage Server Administration" -> Debug tab -> Select the "Logging" option in the left panel 
-> click "Configure loggers". 

Scroll to the bottom of that page and there should be a text-box form under the text "Text-based log configuration". From here, you can add the line as well and push the submit button. This should update the log4j.properties file and apply the settings immediately without a server restart.

Although the defect RM log filled with ResourceNotFoundException from lockGraphRetryHandler: Failure of lock bulk retry operation... (131600) at comment 9 explains a way to cause this log message to spam the logs unnecessarily, there may exist a case where the warning is an issue, but I'm not aware of one at the moment. If you think you have found one, feel free to open an escalation for a more detailed analysis. 

I'm not sure if this message will be disabled (by default) in a future ifix, but we will investigate that option. The workaround mentioned above should turn it off for you for now. 


Comments
Carol Watson commented Jan 07, 10:50 a.m.

 Thanks much Chase!  I'm going to try that.  I've been having issues lately with the log being so large it takes a really long time to load.  Maybe this will help.


permanent link
Chase Patterson (362) | answered Jan 08, 10:36 a.m.

We closed the defect RM log filled with ResourceNotFoundException from lockGraphRetryHandler: Failure of lock bulk retry operation... (131600). This issue is not applicable to the 7.x stream. 


The log4j.properties change that we recommended to prevent this message from spamming your logs will not be necessary in a future ifix release. We are delivering a fix to 6.0.6.1 ifix and 6.0.6 ifix streams. If you are curious to see which ifix this gets delivered in, follow: 
We are reusing a pre-existing maintenance item since this log message was introduced in an ifix change. 


Comments
Carol Watson commented Jan 08, 10:47 a.m.

 Thanks Chase. I should mention we do not have CM turned on in our application, but I'm assuming that iFix014 will fix the problem regardless.


Carol

Your answer


Register or to post your answer.