I'm trying to figure out the best way to do this:

We're improving our automated regression testing and are contemplating using RQM to do it.  There's two avenues we can take - a single automated suite in the automated testing tool, outside RQM, vs an RQM suite with a sequence of test cases whos scripts are automated.

Currently, we have a regression suite that is partially automated using the later design - the system is started up automatically and the most rigorous and time consuming test cases have been automated, and the most difficult to automate or yet to be automated test script are still manual.  Intermitantly through the sprint, we kick the regression suite, wait an hour or so as it tests everything, and then head to the lab to click a few buttons once it reaches a manual test script, and once finished, let the automation resume, come back when the next manual test is ready, repeat.  Our Product Owner has prioritized automating the entire thing.

I would prefer to simply automate all our existing manual tests, and have our build server (in RTC) execute the test suite in the sprints test plan.  We use the 'Tested By Test Case' relation for test coverage of our stories (but should we be?) - the team reviews these test cases to confirm that we test enough, and is part of our definition of done (only when they are all executed and pass is the story done).

It would be neat if, as part of our work on a story, we could 'Tested By Test Case' to one or more of the regression test cases that will be automatically executed in our nightly regression tests, but then we would have one test cases relating to multiple stories, many of which would be 'Done'.  When that test case fails, and we make a defect towards it's result, it's going to 'Affect Plan Items' on all completed stories, and we'd have to uncheck all of them, because it doesn't affect those plan items anymore (only the one we're working on now).  We have this problem now, despite our full automation we're striving for - and we get around it by executing a copy of the test case manually.

So I'm thinking, I could copy the test case and 'tested by test case' to the copied test case - meaning, that test case was created to test this story - if the test case fails, it only affects that plan item.  But, to put it back in the regression suite is a pain, so much so that we probably won't do it. (see here).  What I mean is, we would modify the suite to replace it with the copy, run the regression suite, and when it all passes, put the old case back in (which doesn't have the Tested By Test Case link), so that if the regression suite fails in some future sprint, it doesn't 'Affect plan items' for stories already done.  But modifying the cases in a suite is hard.

 Is anyone else doing this?  If we could easily replace a test case in a suite, this would be ok.  Or if we could reordering test suites easier (drag and drop).  Or, if Affects Plan Item links are only created when the related work item is not done (this design would be best I think, because we wouldn't need to copy test cases).  Any other ideas to make this easier?

If I do the prior (outside of RQM), we'd need some way to describe, in our stories, which parts of our regression tests cover the story and be able to show that it was executed, and it would look different from manual tests (because not all our tests are automated - there would still be manual ones).