It's all about the answers!

Ask a question

Any idea why hundreds of WARN's of "CRJAZ2199W Internal error" & of "Unhandled Exception" with socket write errors in the ccm.log ?


long TRUONG (3654104140) | asked Oct 21 '14, 9:27 p.m.
 On RTC 4.0.6 now, but same issues while we were on 4.0.3:

We got hundreds of pairs of WARN's, with each followed by at least a pair of "socket write error"s, which amount to the thousands in the ccm.log. Always a bone of contention to IBM support when we submitted any PMR requiring poring thru ccm.log, yet they have never been tied to any issues we encountered. Any idea ?

Line 1: 2014-10-20 07:00:28,228 [http-bio-443-exec-8308 @@ 07:00 svc_DemandManagement /ccm/oslc/contexts/_D-LXEOJ9EeKfHbAZQnNPkg/workitems]  WARN bm.team.repository.common.transport.ServerHttpUtil  - CRJAZ2199W Internal error occurred while trying to to send exception information to client - response in committed state.

Line 2: ClientAbortException:  java.net.SocketException: Connection reset by peer: socket write error
Line 55: Caused by: java.net.SocketException: Connection reset by peer: socket write error

Line 73: 2014-10-20 07:00:31,551 [http-bio-443-exec-8356 @@ 07:00 svc_DemandManagement /ccm/oslc/contexts/_D-LXEOJ9EeKfHbAZQnNPkg/workitems]  WARN com.ibm.team.workitem                               - Unhandled Exception

Line 74: ClientAbortException:  java.net.SocketException: Connection reset by peer: socket write error
Line 127: Caused by: java.net.SocketException: Connection reset by peer: socket write error

Line 145: 2014-10-20 07:00:31,551 [http-bio-443-exec-8356 @@ 07:00 svc_DemandManagement /ccm/oslc/contexts/_D-LXEOJ9EeKfHbAZQnNPkg/workitems]  WARN bm.team.repository.common.transport.ServerHttpUtil  - CRJAZ2199W Internal error occurred while trying to to send exception information to client - response in committed state.

Line 74: ClientAbortException:  java.net.SocketException: Connection reset by peer: socket write error
Line 127: Caused by: java.net.SocketException: Connection reset by peer: socket write error

Line 217: 2014-10-20 07:00:35,311 [http-bio-443-exec-8212 @@ 07:00 svc_DemandManagement /ccm/oslc/contexts/_D-LXEOJ9EeKfHbAZQnNPkg/workitems]  WARN com.ibm.team.workitem                               - Unhandled Exception

Line 218: ClientAbortException:  java.net.SocketException: Connection reset by peer: socket write error
Line 271: Caused by: java.net.SocketException: Connection reset by peer: socket write error
...
...
...
Line 112820: 2014-10-21 17:15:34,894 [http-bio-443-exec-9101 @@ 17:14 gp4denf /ccm/service/com.ibm.team.workitem.common.internal.rest.IWorkItemRestService/allValues]  WARN bm.team.repository.common.transport.ServerHttpUtil  - CRJAZ2199W Internal error occurred while trying to to send exception information to client - response in committed state.

Line 112821: com.ibm.team.repository.common.internal.marshal.MarshallingException: ClientAbortException:  java.net.SocketException: Connection reset by peer: socket write error
Line 112860: Caused by: ClientAbortException:  java.net.SocketException: Connection reset by peer: socket write error
Line 112898: Caused by: java.net.SocketException: Connection reset by peer: socket write error
Line 112917: ClientAbortException:  java.net.SocketException: Connection reset by peer: socket write error
Line 112970: Caused by: java.net.SocketException: Connection reset by peer: socket write error

Line 112988: 2014-10-21 17:30:01,495 [http-bio-443-exec-8794 @@ 17:29 svc_DemandManagement /ccm/oslc/contexts/_D-LXEOJ9EeKfHbAZQnNPkg/workitems]  WARN bm.team.repository.common.transport.ServerHttpUtil  - CRJAZ2199W Internal error occurred while trying to to send exception information to client - response in committed state.

Line 112989: ClientAbortException:  java.net.SocketException: Connection reset by peer: socket write error
Line 113042: Caused by: java.net.SocketException: Connection reset by peer: socket write error
Line 113154: ClientAbortException:  java.net.SocketException: Connection reset by peer: socket write error
Line 113207: Caused by: java.net.SocketException: Connection reset by peer: socket write error

Line 113225: 2014-10-21 17:45:00,529 [http-bio-443-exec-9206 @@ 17:44 svc_DemandManagement /ccm/oslc/contexts/_D-LXEOJ9EeKfHbAZQnNPkg/workitems]  WARN bm.team.repository.common.transport.ServerHttpUtil  - CRJAZ2199W Internal error occurred while trying to to send exception information to client - response in committed state.

Line 113226: ClientAbortException:  java.net.SocketException: Connection reset by peer: socket write error
Line 113279: Caused by: java.net.SocketException: Connection reset by peer: socket write error


Comments
long TRUONG commented Jan 24 '15, 7:35 p.m.

updated: This issue is still on an active long running PMR, and we have not got closer to the cause. They have not been impacting into any disruption or interruption yet. So far have not manifested into any related, visible symptoms. 


long TRUONG commented Mar 18 '15, 5:35 p.m.

Thx (a late one) Don,


Not until your colleague Isabel mentioning your comments here that we started to understand you were talking about the RTC service account svc_DemandManagement in the log we posted (duh!).
And without the coincidence that the DMS team is joining our team, we may not have related the account to possibly the DMS app, even if we had understood so then.

We may be a lot closer to the answer of most of the write socket errors now.


long TRUONG commented Mar 31 '15, 2:00 p.m.

DMS is  our home grown light business issue/demand tracking app. It is making calls to the RTC every 15 minutes to sync with RTC WI, via the non-privileged service account svc_DemandManagement, only with repo permission jazzUsers.


DMS calls time out at 30 secs, hence causing the majority of the errors when RTC server slow to respond.

We also have TasktopSync, trying to sync QC defects with RTC WI Bugs every 10 minutes, via the admin privileged service account svc_rtc, with all the repo permissions.

TTsync are making more sophisticated/complicated/involved syncs, yet it is generating a lot less, if any, write socket errors.

Accepted answer


permanent link
Donald Nong (14.4k314) | answered Jan 27 '15, 7:06 p.m.
I doubt that you will "see" any symptoms. By the look of it, the _client_ disconnects before the _server_ is still trying to send exceptions out, so you will not see the intended exception on the client. The user names related to the exceptions appear to be service accounts, are they? If so, are they related to any custom applications that have their own error handling causing this behavior?
long TRUONG selected this answer as the correct answer

Your answer


Register or to post your answer.