It's all about the answers!

Ask a question

Share your top 5 pain points using RQM


4
5
Pramod Chandoria (2.1k11120) | asked Jun 21 '12, 8:26 a.m.
JAZZ DEVELOPER
edited Nov 08 '12, 12:37 a.m.
Please read on if you have used Rational Quality Manager (RQM) and want to provide feedback or want to share success story..

Please help us to serve you better by providing quality feedback on 5 top pain points (or things you like) specific to Rational Quality Manager or it's integration.

You can tell about e.g.
  • Feature Gap
  • Day to day Usability issues
  • Performance issues
  • Reliability issues
  • Complex feature to understand
  • Anything else
  • I would like if RQM could...
You can also share your success story using RQM.

Thanks for your valuable responses which helps us to serve you better

Comments
Anandeep Sidana commented Jun 20 '13, 4:19 a.m. | edited Jun 20 '13, 4:21 a.m.

Hi,
1.Test reporting is biggest challenge, creating custom reports on diff attributes there is no easy way. This is posing to be a show stopper to adopt RQM across group.
2.Test case execution when done individually in test suite, there is no way to filter already executed test cases nor any filters
3.Same test case can be added multiple times in one test suite.when add from selected test cases to be added,user never know that test case is being added more than once, Quality center creates another copy and there are better filters.
4.Getting folks adapt using RQM is very hard to convince especially when its transition from one tool ( QC) to RQM being not as very user friendly.


Harish Kumar commented Jul 21 '15, 7:18 a.m. | edited Jul 21 '15, 7:19 a.m.

Is there a way to approve a test suite and by doing so, all the test cases and test scripts under that test suite get approved automatically?

Regards

Harish KN

22 answers



permanent link
Michelle Crane (12322) | answered Jul 11 '12, 9:09 a.m.
JAZZ DEVELOPER
1. Continually having to log in every so often.  I'm on RQM all day, and I continually have to log in.  The fact that I keep several tabs open into RQM makes this even more frustrating; I have to re-log in to every tab.

2. Printing is useless.  Sure, I can print a test case, but if I try printing out a list of test cases in a test plan, I get too much information and wasted paper.  Plus, printing doesn't scale - it takes so long to create a big printout.

3. The concept of test scripts per test case.  It took us over a year to realize that we were using RQM wrong - we had multiple scripts per test case, in the assumption that if you ran a TER, the scripts would be strung together.  Not so.  We were actually missing scripts when we ran the TERs.

4. Test scripts are a nice concept, but they are really hard to create/edit.  I'm talking here about the table used for the script steps.  Due to point 3 above, I had to do a bunch of editing, copying steps from one script and pasting them into another.  This was a serious annoyance.  I got into a rhythm, with two browsers open of Tab, Ctrl+C, Atl+Tab, Tab, Ctrl+V.  In general, I love how easy scripts are to run but I absolutely HATE editing them.  I wish there were some way to use a desktop tool, or something. 

5. When I create TERs, I would love to know who ran them last.  Sometimes I want to have the same testers run the same tests; sometimes I want to avoid that.  Unless I actually do a bunch of clicking and navigating, I can't know who ran the last TER for a particular test case.  It takes too long when you're assigning dozens of cases at a time.

Comments
Pramod Chandoria commented Jul 11 '12, 11:52 a.m.
JAZZ DEVELOPER

Hi Michelle, This is very nice feedback, I will follow it up. Thank you



Kevin Ramer commented Apr 23 '13, 8:02 a.m.

On #1, this can be alleviated by getting the admin for your QM to make the length of the web sessions longer than the default 30 minutes.


permanent link
Colin Thorne (22421619) | answered Jul 18 '12, 10:51 a.m.
We have been using RQM 3.0.1.1 extensively to help us test our product development. In particular we are using the RQMCommandLineAdapter to automate the running of our tests. From this use case we are finding the following problems:
  1. So slow to navigate - this seems to be a combination of:
    1. lack of responsiveness when there are large numbers of results (e..g the previous results list of a test suite execution record) which can take up to a few minutes to respond
    2. the tabs inside the browser window - when you are scrolled down to the bottom of a page of results you can no longer see the tabs to choose another one, and when you switch between tabs you are always taken back to the top again, not where you were viewing. Not a big problem when you are using it occasionally, but when we are using it all day, every day, it becomes very frustrating. I get lost amongst all the tabs. I think RQM 4.0 could solve this problem for us, which is great when we start using it.
  2. Problems with reliably using execution schedules. Sometimes an execution schedule with fail with an 'Error' state which has no more information to suggest a problem. Trial and error suggests that any change to a test suite will stop an existing execution schedule (that uses the test suites's execution record) from working. The solution is that every time a test suite is modified, any execution schedule has be modified by removing and re-adding the same test suite execution record. Which leads me to my next point
  3. The slowest part of the whole process: modifying an execution schedule. We have test suites with about 30 tests. Whenever I need to change an execution schedule with one or more test suite execution schedule I have to wait for all the individual test cases to find a machine to run on. It takes 2 or 3 seconds per test case at best. I have spent many minutes several times a day waiting for the execution schedule to refresh itself before I can make a small change. By the way, the ability to re-order test suite execution records in a test schedule is broken although I haven't raised a bug yet.
  4. The way that blocking defects are associated with test case results is good, but I would like to automatically remove the blocking defect when the defect is verified. It is a slow process to individually remove all blocking defects (is there any way to do this other than one test case at a time?)
  5. No way of seeing the test output until the test has finished. The RQMCommandLineAdapter does not have any way of shown the test output while the test is running. Unlike an RTC build using a jazz build engine, there is no log file that I can tail on the target server. 
  6. Am I allowed a number 6? I have just remembered what is probably just a niggle compared to the other problems. When I am in a test suite I would like to add it to a test plan, but I cannot add the test plan from the test suite. Instead I have to go to the list of test plans, open it, then find the test suite to add. 
Sorry this sounds so negative, there are many positives about using RQM but we are finding it harder and harder to use (mainly because of performance and navigation problems) as we add more and more test cases.

Comments
Pramod Chandoria commented Nov 12 '12, 11:21 a.m. | edited Nov 12 '12, 11:22 a.m.
JAZZ DEVELOPER

Check following workitems
So slow to navigate previous results list of a test suite execution record (76059)
Problems with reliably using execution schedules (76060)
https://jazz.net/jazz02/resource/itemName/com.ibm.team.workitem.WorkItem/76061
Automatically remove the blocking defect when the defect is verified (76062)
Usability issue adding test suite to test plan (76063)


permanent link
Rene Meyer (42913134) | answered Jun 21 '12, 9:14 a.m.
JAZZ DEVELOPER
Hello Pramod,

1. No screenshot utility when authoring manual tests.
2. Product Structures like a category tree would be more convenient to create in a tree like editor view on testcases for average users of the tool.
3. Test data are bound to a test script which makes it difficult to parametrize the same test with different data. You have 2 test cases A and B and would like to test with the same test script but different data set for A and B. Test data should be bound to TCER and test script should only use variables, in the TCER there would be a mapping from test data columns to test script variables.
4. Test data will not remember which test data rows where already used in tests before.

Best Regards,
René


Comments
Pramod Chandoria commented Jun 21 '12, 9:17 a.m.
JAZZ DEVELOPER

Thanks Rene, Above mentioned points are good inputs.



permanent link
Ranjith Parameswaran (12311215) | answered Jul 05 '12, 8:10 a.m.


Hello Pramod,

 

In future will RQM have the customization available for its test artifacts as RTC offers for its work item?


Comments
1
Pramod Chandoria commented Jul 12 '12, 9:24 a.m.
JAZZ DEVELOPER

RQM 4.0 provides customization for test artifact state similar to RTC. See Test artifact workflow customization (https://jazz.net/downloads/rational-quality-manager/releases/4.0?p=news#workflow) for more details


permanent link
luciano adamiak (36111) | answered Aug 10 '12, 4:04 p.m.
  For large execution teams, this is REALLY a problem:

  • Datasets are associated with Test Scripts and not TCER.
It makes no sense. It should be associated with the execution!
Imagine I have a really large dataset that has to be executed manually and I want to split the work among several executors.
I could than create several Suite Execution Records, set one dataset for each and have several employees working simultaneously. 

permanent link
Michael Walker (99213190157) | answered Jun 25 '12, 7:01 p.m.
Here's 3 I can think of off the top of my head

1.  Inability to create dynamic test suites

2. When creating Manual Tests we can't put screenshots in for steps.

3. Inconsistency with RFT adapters



Comments
Pramod Chandoria commented Jun 26 '12, 2:23 a.m.
JAZZ DEVELOPER

Thanks Mike, Could you also provide little more information on #3 Inconsistency with RFT adapters


1
Michael Walker commented Jul 02 '12, 2:55 a.m.

After talking to the users, the inconsistency was actually with executing RFT scripts using shared resources. Sometimes it would work and other times nothing would happen when they try and execute and there were no error messages. When not using shared resources it worked every time.


1
Pramod Chandoria commented Jul 04 '12, 4:10 a.m.
JAZZ DEVELOPER

Mike, Make sure that shared location is accessible from the RQM Server location. If RQM is running as service, the user account used to run the service, the same user should have access to the shared location.


1
Aaron Shepherd commented Jul 18 '12, 10:03 a.m. | edited Jul 18 '12, 10:17 a.m.

When executing from a shared resource, the RFT adapter copies a limited number of files associated with a script. Non-project files, like jars, are expected to already be on the client in "C:\Documents and Settings\All Users\Application Data\IBM\RFT\customization" If a script which requires additional libraries is executed, and those libraries are not in the customization folder, you will see the behavior you are describing. We found this to go against the idea of having a "shared resource" so we mount the share on the client and run from RQM as if it is a local project.


Pramod Chandoria commented Nov 12 '12, 11:27 a.m.
JAZZ DEVELOPER

permanent link
Michael te Wierik (21) | answered Jul 12 '12, 2:50 a.m.
  1. Creating and maintaining links to RFT test Scripts via the adapter is a painful exercise. If you could use a standard "Browse" with the adapter, or even paste a string in this would go a long way to improving my workload in maintenance of Test Scripts. The other option available which downloads the scripts locally is not an option in our environment.
  2. If status of a test script/case/suite is Approved some changes are not available to be implemented until you reopen the the status. Negates the use of approved as we just leave it in an earlier status state so that we can still edit.
  3. Unable to debug RFT script runs from RQM unless extensive logging created in the script (console not active, RFT not active - just utilised in background etc.). This has caused us particular pain in debugging errors found when using the Execution Variables.
  4. Every time we make an Adapter connection, even from the same PC, we get another entry in the Adapter Hosts list, with the previous version of this same PC adapter being "red" and the new one being "green". Clearing this automatically would help with maintenance.
  5. If I tell someone what to do in QC they can execute a damn sight quicker than if I get them to execute the same method in RQM. This is due to:
  • So many clicks to do anything! Just navigating to a specific test script in a Test Suite requires:
    Open RQM > Select View Suites > filter for suite or search > Select Suite > Select Test Cases on Suite > Select Test script on Suite
    By comparison:
    Open QC > Navigate to TestSet (usually just a flick open of folder then select, depending on the project) > Test Case Instance is visible and ready for editing
  • Each click is a delay! Delivery of the RQM screens in our environment does not seem to be anywhere near as quick as presentations I have seen from IBM. Yes, I can see that its the environment, not the application, but users don't care and that reflects badly on the product.
  • Views are maintained in QC but not in RQM. If I am working in an area in QC and close the app, I can go back at any time and it will be the same view I left.
    In RQM I would need to follow a number of clicks before I get my "workspace" back
That's it for today... ;)

Comments
Aaron Shepherd commented Jul 18 '12, 10:09 a.m.

Re #3: RQM 4.0 added support for specifying additional logs/attachments for execution results generated by the command line adapter. An enhancement request has been submitted to implement a similar solution for the RFT adapter, but I do not know when or if it will be added.



permanent link
Frank Ning (50023111131) | answered Jul 24 '12, 6:10 a.m.

Overall RAM V4 is quite nicer than V3. Besides what other expertes mentioned, I would like to dream the following  features:

1) Allow dynamic query of test artifacts like what we can do with RTC work items by providing certain values of certain attributes on the fly.

2) Have "RQM client" (eclipse and Visual studio based) in the same way as RTC client to also manage test artifact creation, update, execution and query etc.

3) There can be a "Review" and "Approval" status attributes for test artifacts so that we can easily tell which test artifacts have been reviewed or approved. If not, it would show current status (Pending, In Progress, Not Opened, etc)  Currently we have to open the test artifacts to see the information in the formal review section, unless there is this kind of query I don't know.


permanent link
Robert Ducharme (93128) | answered Jul 24 '12, 8:29 a.m.

1 - editing test scripts is a pain. We find it easier to create test scripts in excel and then import them instead of using the current editor.

2 - auto generation of manual test scripts from description. Needs to be improved to make it easier to separate steps and add expected column in the scripts. Define a nice interface or way to write the description so conversion is better.

3 - provide ability to tie the execution of RQM test cases(?) back to work items. There are references but if a test case is passed, the total time it took to execute (through all the runs) and results should be fed back and update a linked work item (task or defect). Eliminates the need for a user to go into CCM and manual update that information.


permanent link
Bharat Patel (81113542) | answered Jul 26 '12, 12:53 p.m.

Following features can be incorporate in RQM

1) exporting of Test Case in Word document

2) If any requirement change , status of linked test case need to be suspect automatically. Requirement Reconcillation feature is there but it is not automated.

3) Versioning of Test Artifacts as other ALM tools offering this feature in market.

4) customization jazz login page for e.g. to textual change in JTS Login Page, client logo etc.

5) Reporting can be imnproved.

Your answer


Register or to post your answer.