Is there any fix for this : CRJAZ1165I Marshalling exception processing request for service
We are noticing that our RTC server is going down very often. The reason was, the DB user is getting locked, due to multiple number of unsuccessful logins. What I presume is, until this exception throws all the RTC users are trying to revoke the session, which eventually gets failed after this exception is thrown. This is what I understood, please correct me if I am wrong here.
I searched for this, I could not find any fix for this. Please let me know how to fix this issue. Thanks!
CRJAZ1165I Marshalling exception processing request for service "com.ibm.team.repository.common.service.IQueryService".
CRJAZ1170I The request was made by user "xxxx@yy.zz.com" from "xxx.xxx.xxx.xxx".CRJAZ1166I The stack trace hash is 3601A6B65EE2D4314B71A07F3F1E7F90B91A0D97.
com.ibm.team.repository.common.internal.marshal.MarshallingException: Async operation timed out
at com.ibm.team.repository.common.internal.marshal.impl.EObjectMarshaller.demarshalInputStreamToObject(EObjectMarshaller.java:579)
at com.ibm.team.repository.common.internal.marshal.impl.EObjectMarshaller.demarshalInputStreamToObject(EObjectMarshaller.java:599)
at com.ibm.team.repository.common.internal.marshal.impl.WebServicesMarshaller.demarshalInputStreamToServiceRequest(WebServicesMarshaller.java:139)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.doPost(AbstractTeamServerServlet.java:690)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:738)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest2(AbstractTeamServerServlet.java:2358)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest(AbstractTeamServerServlet.java:2155)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.access$0(AbstractTeamServerServlet.java:2140)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet$1.service(AbstractTeamServerServlet.java:224)
at com.ibm.team.repository.internal.service.auth.impl.JAuthHandler$1.run(JAuthHandler.java:109)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.service(AbstractTeamServerServlet.java:1813)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
at org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:126)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:76)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
at org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:120)
at com.ibm.team.repository.server.servletbridge.JazzServlet.service(JazzServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1646)
Caused by: java.net.SocketTimeoutException: Async operation timed out
at com.ibm.ws.tcp.channel.impl.AioTCPReadRequestContextImpl.processSyncReadRequest(AioTCPReadRequestContextImpl.java:189)
at com.ibm.ws.tcp.channel.impl.TCPReadRequestContextImpl.read(TCPReadRequestContextImpl.java:111)
at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext.read(SSLReadServiceContext.java:265)
at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.fillABuffer(HttpServiceContextImpl.java:4171)
at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.readSingleBlock(HttpServiceContextImpl.java:3403)
at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.readBodyBuffer(HttpServiceContextImpl.java:3509)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundServiceContextImpl.getRequestBodyBuffer(HttpInboundServiceContextImpl.java:1782)
at com.ibm.ws.webcontainer.channel.WCCByteBufferInputStream.bufferIsGood(WCCByteBufferInputStream.java:373)
at com.ibm.ws.webcontainer.channel.WCCByteBufferInputStream.read(WCCByteBufferInputStream.java:266)
at com.ibm.ws.webcontainer.srt.http.HttpInputStream.read(HttpInputStream.java:325)
at java.io.FilterInputStream.read(FilterInputStream.java:127)
at com.ibm.team.repository.servlet.internal.CountingInputStream.read(CountingInputStream.java:80)
at com.ibm.team.repository.common.internal.marshal.impl.EObjectMarshaller.demarshalInputStreamToObject(EObjectMarshaller.java:567)
... 48 more
Caused by: com.ibm.io.async.AsyncTimeoutException(Async operation timed out, [Timeout, rc=0])
at com.ibm.io.async.AbstractAsyncFuture.waitForCompletion(AbstractAsyncFuture.java:359)
at com.ibm.io.async.AsyncFuture.getByteCount(AsyncFuture.java:218)
at com.ibm.ws.tcp.channel.impl.AioSocketIOChannel.readAIOSync(AioSocketIOChannel.java:215)
at com.ibm.ws.tcp.channel.impl.AioTCPReadRequestContextImpl.processSyncReadRequest(AioTCPReadRequestContextImpl.java:182)
I searched for this, I could not find any fix for this. Please let me know how to fix this issue. Thanks!
CRJAZ1165I Marshalling exception processing request for service "com.ibm.team.repository.common.service.IQueryService".
CRJAZ1170I The request was made by user "xxxx@yy.zz.com" from "xxx.xxx.xxx.xxx".CRJAZ1166I The stack trace hash is 3601A6B65EE2D4314B71A07F3F1E7F90B91A0D97.
com.ibm.team.repository.common.internal.marshal.MarshallingException: Async operation timed out
at com.ibm.team.repository.common.internal.marshal.impl.EObjectMarshaller.demarshalInputStreamToObject(EObjectMarshaller.java:579)
at com.ibm.team.repository.common.internal.marshal.impl.EObjectMarshaller.demarshalInputStreamToObject(EObjectMarshaller.java:599)
at com.ibm.team.repository.common.internal.marshal.impl.WebServicesMarshaller.demarshalInputStreamToServiceRequest(WebServicesMarshaller.java:139)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.doPost(AbstractTeamServerServlet.java:690)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:738)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest2(AbstractTeamServerServlet.java:2358)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest(AbstractTeamServerServlet.java:2155)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.access$0(AbstractTeamServerServlet.java:2140)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet$1.service(AbstractTeamServerServlet.java:224)
at com.ibm.team.repository.internal.service.auth.impl.JAuthHandler$1.run(JAuthHandler.java:109)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.service(AbstractTeamServerServlet.java:1813)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
at org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:126)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:76)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
at org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:120)
at com.ibm.team.repository.server.servletbridge.JazzServlet.service(JazzServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1646)
Caused by: java.net.SocketTimeoutException: Async operation timed out
at com.ibm.ws.tcp.channel.impl.AioTCPReadRequestContextImpl.processSyncReadRequest(AioTCPReadRequestContextImpl.java:189)
at com.ibm.ws.tcp.channel.impl.TCPReadRequestContextImpl.read(TCPReadRequestContextImpl.java:111)
at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext.read(SSLReadServiceContext.java:265)
at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.fillABuffer(HttpServiceContextImpl.java:4171)
at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.readSingleBlock(HttpServiceContextImpl.java:3403)
at com.ibm.ws.http.channel.impl.HttpServiceContextImpl.readBodyBuffer(HttpServiceContextImpl.java:3509)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundServiceContextImpl.getRequestBodyBuffer(HttpInboundServiceContextImpl.java:1782)
at com.ibm.ws.webcontainer.channel.WCCByteBufferInputStream.bufferIsGood(WCCByteBufferInputStream.java:373)
at com.ibm.ws.webcontainer.channel.WCCByteBufferInputStream.read(WCCByteBufferInputStream.java:266)
at com.ibm.ws.webcontainer.srt.http.HttpInputStream.read(HttpInputStream.java:325)
at java.io.FilterInputStream.read(FilterInputStream.java:127)
at com.ibm.team.repository.servlet.internal.CountingInputStream.read(CountingInputStream.java:80)
at com.ibm.team.repository.common.internal.marshal.impl.EObjectMarshaller.demarshalInputStreamToObject(EObjectMarshaller.java:567)
... 48 more
Caused by: com.ibm.io.async.AsyncTimeoutException(Async operation timed out, [Timeout, rc=0])
at com.ibm.io.async.AbstractAsyncFuture.waitForCompletion(AbstractAsyncFuture.java:359)
at com.ibm.io.async.AsyncFuture.getByteCount(AsyncFuture.java:218)
at com.ibm.ws.tcp.channel.impl.AioSocketIOChannel.readAIOSync(AioSocketIOChannel.java:215)
at com.ibm.ws.tcp.channel.impl.AioTCPReadRequestContextImpl.processSyncReadRequest(AioTCPReadRequestContextImpl.java:182)
One answer
Hi Donald, Yes the DB user gets locked, whenever RTC server goes down. As and when RTC goes down, we first see whether the DB user is locked or not, and yes it gets locked, then we unlock that user, from where RTC comes up. When we checked the log, it mentioned that teamserver.properties is having issues, but we didnt see any issue or change there.
And also xxxx@yy.zz.com is the RTC admin user which is configured in LDAP. It is just an admin user which is used for rtc build related purposes.
And also xxxx@yy.zz.com is the RTC admin user which is configured in LDAP. It is just an admin user which is used for rtc build related purposes.
Comments
Donald Nong
Mar 23 '15, 7:10 p.m.1 vote
Naveen Krishnan
Mar 25 '15, 9:09 a.m.Donald Nong
Mar 26 '15, 12:58 a.m.