Jazz Forum Welcome to the Jazz Community Forum Connect and collaborate with IBM Engineering experts and users

Issues with connector performance benchmarking .

We are trying to benchmark the performance of RTC-CMVC connector and do the necessary optimizations . After addressing all the know issues we are unable to figure out the reason for one particular case .

We use interop service to import and synchronize the work item . To increase the throughput we call "IncomingSynchornizeAndWait" through an executor service with 20threads .

Through java profiling we have observed that import goes on very fast initially and but towards the end (last 20-30 defects - irrespective of total defect count) , one or two threads goes into blocked or deadlocked state and the thread stack trace shows two methods most of the time :
a) findProxyByUri
b) findProxyByExternalState

During this time , the interop DEBUG logs also halts completely (no output at all) .

This deadlock gets resolved after some time automatically .

We tried to find a way to unblock this situaiton by :
- Changing async task values for total and interop in admin UI ,
- Monitored the CPU and Disk usage to ensure that RTC server CPU is not in 'wait" state for I/O (Moved Db2 to another server)
- Tuned the AIX servers for no limits on files and memory.
- Turned off logging
- Turned off indexing by suspending the indexer
- Turned on Logging at DEBUG level for ccm, jts , interop for all items in the log4.properties .

but could not figure out anything in particular and thus posting here.

During the deadlock CPU remain busy doing something . This is noticed even after we create lot of work items through java api itself .

We suspect that some RTC processes gets kicked in towards last leg of import leading to blocking calls on "findItem" method and once RTC finishes its own work , the import thread unblocks to finish the remaining import .

Is there a way to curcumvent this issue since the impact for us is more than 60% in import time and this seems to increase with increase in number of defects for import.

So to summarize :
1) What is RTC doing after import has started or when a bulk workitem creation has finished
2) What is the way to delay that process so that import can finish asap and RTC can continue with its work later on ?

0 votes



4 answers

Permanent link

Team - Any pointers on this ?

0 votes


Permanent link
It's hard to say what might be going on; I'm not familiar with what might be happening in RTC when lots of work items are created quickly. I suggest the standard approaches to debugging Java performance and threading issues. The first thing I would look at is a Java dump when you think things are hung up or slow, to try and see what all the threads are doing.

0 votes


Permanent link
Hi John - We thought of profiling RTC server as well , but stopped on this forum for quick help at its not the RTC server which gets blocked but one of the client threads (mainly in findProxyByUri , findProxyByExternalState ) .
Is there anyone else i can chat with on ST for some quick pointers from server side ?

0 votes


Permanent link
The "find" methods in the client just issue queries to the RTC server, so it sounds like the server gets into a state where it is not responding to those queries very readily. That's why it would be interesting to see thread dumps from the RTC server when it is in that state.

0 votes

Your answer

Register or log in to post your answer.

Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.

Search context
Follow this question

By Email: 

Once you sign in you will be able to subscribe for any updates here.

By RSS:

Answers
Answers and Comments
Question details
× 10,938

Question asked: Sep 09 '11, 7:22 a.m.

Question was seen: 6,073 times

Last updated: Sep 09 '11, 7:22 a.m.

Confirmation Cancel Confirm