I have a JTS running under WAS 7.0.0.19 and this morning I noticed some mild performance slowdown and found that the WAS is clipping along at 25% cpu all by itself. Looking at the Active Services page I see 3 of these
com.ibm.team.repository.service.internal.auth.proxy.AuthProxy.findAppCreds
running for 4hr now sometimes showing this stack:
org.eclipse.emf.ecore.util.EcoreUtil$UUID.generate(EcoreUtil.java:3634)
org.eclipse.emf.ecore.util.EcoreUtil.generateUUID(EcoreUtil.java:3616)
com.ibm.team.repository.common.UUID.generate(UUID.java:51)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$Invalidator.(TransactionalCache.java:1735)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$CacheEntry.(TransactionalCache.java:1328)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$DistributedTransactionalBackingMap.mergeEntries(TransactionalCache.java:2515)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$DistributedTransactionalBackingMap.get(TransactionalCache.java:2569)
I tried to find the so-called Smokin' Gun (props to Robert Cray) in the jts logs but don't see a lot. I do see this often:
2012-04-11 09:27:59,204 [ WebContainer : 25] ERROR ibm.team.dashboard.internal.web.IFrameProxyServlet - POST https://jts02
:9444/jts/_iframeproxy/https%253A%252F%252Frtp-rtc10.tivlab.raleigh.ibm.com%253A9443%252Fjazz/service/com.ibm.te
am.workitem.common.internal.rest.IQueryRestService/getResultSet
HEADER: Host = jts02:9444
HEADER: User-Agent = Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.28) Gecko/20120306 AskTbFTB/3.14.1.20007 (CK-IBM) Firefox
/3.6.28 ( .NET CLR 3.5.30729; .NET4.0E)
HEADER: Accept = text/json
HEADER: Accept-Language = en-us,en;q=0.5
HEADER: Accept-Encoding = gzip,deflate
HEADER: Accept-Charset = ISO-8859-1,utf-8;q=0.7,*;q=0.7
HEADER: Keep-Alive = 115
HEADER: Connection = keep-alive
HEADER: X-com-ibm-team-configuration-versions = LATEST
HEADER: X-Jazz-CSRF-Prevent = 0000TM3TFlcd6syis3BZebh7B_E:-1
HEADER: Content-Type = application/x-www-form-urlencoded; charset=utf-8
HEADER: X-Requested-With = XMLHttpRequest
HEADER: Referer = https://jts02:9444/jts/_iframeproxy/https%253A%252F%252Frtp-rtc10.tivlab.raleigh.ibm.com%253A9
443%252Fjazz/_ajax-modules/com.ibm.team.dashboard.web.iframe/com.ibm.team.workitem.viewlets.web.ui.internal.WorkItemsViewlet?_proxyURL=
%2Fjts%2F_iframeproxy%2Fhttps%25253A%25252F%25252Frtp-rtc10.tivlab.raleigh.ibm.com%25253A9443%25252Fjazz
HEADER: Content-Length = 453
HEADER: Cookie = net-jazz-ajax-cookie-rememberUserId=; JazzFormAuth=Form; UnicaNIODID=lsTrro7DH4o-Wo3f6lv; JSESSIONID=0000TM3TFlcd6syis
3BZebh7B_E:-1; LtpaToken2=/NRU+y+bCAWbaoZoDn1+1mxXJymqiohJU4tLy+nkybiKhgSIppRuNmpGQU0K+FOSfC2OpRRkBiUXLVZPD/mNZ5aCEG/R6X9kc4U6U4W+d5jLI
Fx6bu6a7J2DWU+AVuKUMfHNudLIhuYeeVdRCRKJ7XmGZWx+cj0XB/gsOsh6KQeVtyjDW2JialJtToNK8RP8ILfSPBT06SiKhRS/egKuB06L6ro+6XmBKy9t0zNCVyza5y7Bt5FG
zY6TtCw6eXIGhp8vAOCFRl4Lqrctu2eDhkTSTx3pmg7CG7CTeDgUH+wiTI9lNiMCevrQhMfqO54AYZNzwQs1ISQHHdd1C0Uq/Mhh5iR2+7FpbJNCkFzVpGYt1XKpEMhYKxZV8Z4
h4rJn6IuYAZzxGFNQhTnC1gKy9t14pEoS5Y3OgqR13PBBYBHDnIJuRqITZuDGNnS9pfn817biEgTv4MA4nYrlY3XexKn1K9HtCnpTXemgAKJXrZ8IjknlQSwQWAtw7k+451oMuJ
27AnrpZUao93uWkaHcYQye+AkcCpek05VlGaMR2OcEsbC1I50Bg9kgYa29mKPyMNrUr5TFwjpV+czbc6EwryITLABJ9ta3K8c5z4f0CuZ+8RHDX5Ytp0nL3FOJxzB1TEhGt1qiM
6A5VAUQd6KYYXMtpVuZch3VY47Xb4g4FalcLVHQFuB6teKAjgcs/cS9yaqF2e1Uxz+fl24wLZgjHw==; WASReqURL=https://:9444/jts/authenticated/identity?red
irectPath=%252Fjts%252Fwhoami%253FitemId%253D_4EDxUF1DEeG3x-nGmNdbWw%2526projectAreaItemId%253Dtrue
HEADER: Pragma = no-cache
HEADER: Cache-Control = no-cache
Parameter: columnIdentifiers = [workItemType, summary, creator, owner, creationDate, duration, category, target, projectArea, internalT
ags, internalState]
Parameter: filterAttribute = []
Parameter: maxResults = [15]
Parameter: skipOAuth = [true]
Parameter: absoluteURIs = [true]
Parameter: filterValue = []
Parameter: projectAreaItemId = [true]
Parameter: itemId = [_GF8wUM5NEeCP58g2HCYqew]
Parameter: startIndex = [0]
java.lang.StringIndexOutOfBoundsException: String index out of range: 13
at java.lang.String.substring(String.java:1072)
at org.eclipse.equinox.http.servlet.internal.HttpServletRequestAdaptor.getPathInfo(HttpServletRequestAdaptor.java:60)
at javax.servlet.http.HttpServletRequestWrapper.getPathInfo(HttpServletRequestWrapper.java:146)
at org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:89)
I know I can probably rectify the CPU load by bouncing the JTS, but it has a number of dependent CCM.
Any suggestions ?