Why does WebSphere Application Server hang on stopping CCM application?
When I stop the CCM application on WAS console, WAS hangs and the CCM application is not stopped.
I can see errors in WAS log repeatedly. Anyone has the same experience?
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R java.lang.NullPointerException
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R at org.eclipse.equinox.weaving.hooks.WeavingLoaderDelegateHook.postFindResource(WeavingLoaderDelegateHook.java:131)
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R at org.eclipse.osgi.internal.loader.BundleLoader.searchHooks(BundleLoader.java:454)
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R at org.eclipse.osgi.internal.loader.BundleLoader.findResource(BundleLoader.java:592)
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R at org.eclipse.osgi.internal.loader.ModuleClassLoader.getResource(ModuleClassLoader.java:201)
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R at org.eclipse.osgi.internal.framework.ContextFinder.getResource(ContextFinder.java:144)
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R at com.ibm.db2.jcc.am.vd.run(vd.java:49)
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R at java.security.AccessController.doPrivileged(AccessController.java:678)
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R at com.ibm.db2.jcc.am.GlobalProperties.a(GlobalProperties.java:146)
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R at com.ibm.db2.jcc.am.GlobalProperties.d(GlobalProperties.java:100)
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R at com.ibm.db2.jcc.am.ar.run(ar.java:124)
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R at java.util.TimerThread.mainLoop(Timer.java:566)
[2020/3/18 15:55:27:950 CST] 000000fa SystemErr R at java.util.TimerThread.run(Timer.java:516)
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R java.lang.NullPointerException
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R at org.eclipse.equinox.weaving.hooks.WeavingLoaderDelegateHook.postFindResource(WeavingLoaderDelegateHook.java:131)
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R at org.eclipse.osgi.internal.loader.BundleLoader.searchHooks(BundleLoader.java:454)
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R at org.eclipse.osgi.internal.loader.BundleLoader.findResource(BundleLoader.java:592)
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R at org.eclipse.osgi.internal.loader.ModuleClassLoader.getResource(ModuleClassLoader.java:201)
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R at org.eclipse.osgi.internal.framework.ContextFinder.getResource(ContextFinder.java:144)
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R at com.ibm.db2.jcc.am.vd.run(vd.java:49)
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R at java.security.AccessController.doPrivileged(AccessController.java:678)
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R at com.ibm.db2.jcc.am.GlobalProperties.a(GlobalProperties.java:146)
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R at com.ibm.db2.jcc.am.GlobalProperties.d(GlobalProperties.java:100)
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R at com.ibm.db2.jcc.am.ar.run(ar.java:124)
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R at java.util.TimerThread.mainLoop(Timer.java:566)
[2020/3/18 15:55:44:583 CST] 00000121 SystemErr R at java.util.TimerThread.run(Timer.java:516)
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R java.lang.NullPointerException
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R at org.eclipse.equinox.weaving.hooks.WeavingLoaderDelegateHook.postFindResource(WeavingLoaderDelegateHook.java:131)
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R at org.eclipse.osgi.internal.loader.BundleLoader.searchHooks(BundleLoader.java:454)
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R at org.eclipse.osgi.internal.loader.BundleLoader.findResource(BundleLoader.java:592)
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R at org.eclipse.osgi.internal.loader.ModuleClassLoader.getResource(ModuleClassLoader.java:201)
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R at org.eclipse.osgi.internal.framework.ContextFinder.getResource(ContextFinder.java:144)
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R at com.ibm.db2.jcc.am.vd.run(vd.java:49)
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R at java.security.AccessController.doPrivileged(AccessController.java:678)
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R at com.ibm.db2.jcc.am.GlobalProperties.a(GlobalProperties.java:146)
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R at com.ibm.db2.jcc.am.GlobalProperties.d(GlobalProperties.java:100)
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R at com.ibm.db2.jcc.am.ar.run(ar.java:124)
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R at java.util.TimerThread.mainLoop(Timer.java:566)
[2020/3/18 15:56:27:951 CST] 000000fa SystemErr R at java.util.TimerThread.run(Timer.java:516)
RTC version is 6.0.6.1
WAS 8.5.5.17
Database is Db2 11.1
|
One answer
Hi Ian,
Thanks for your response. About your questions. I detailed the answers as below:
1. Are you able to stop the server by some other means?
Ans: No. When the CCM application hang, I can't stop the WAS service either. I have to kill the process from the task manager.
2. Are you able to stop other ELM applications successfully?
Ans: I can stop RS、DCC、JTS applications in the integrated console of WAS.
3. What happens if you stop RTC from command line by going into the profile itself? Is it successful?
Ans: No. The server is unable to stop due to the RTC application is hung during the stop process.
4. What OS are you using? If Windows, do you have a Windows service to start and stop the server? Does that successfully stop the server?
Ans: The OS is Windows. And WAS is installed as a service.
Comments
Ian Wark
commented Mar 22 '20, 10:30 p.m.
That information is quite helpful. I think it shows that the hang is happening specifically with the CCM application.
My first suggestion would be to clear the WebSphere caches. Particular the temp, wstemp, logs and class caches. But you need to stop the server and take care before doing so. I found these tech notes.
https://www.ibm.com/support/pages/things-know-deleting-temporary-cache-and-log-files-websphere-application-server
If that doesn't work, I think you should obtain 5 javacores on shutdown on the CCM application server once every 30 seconds. Obtain the ISADC during that period to cross-reference too. If you can clear logs while the server is stopped that would help. Raise a case to customer support, (if you can) to get that analyzed.
Ian Wark
commented Mar 23 '20, 4:02 a.m.
You may want to also see if this applies:
|
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.
Comments