Memory Issues in DOORS 9.x?

pete.kowalski - Mon May 23 13:01:07 EDT 2011
Are you using regxp?

David_G_Bond - Mon May 23 13:09:49 EDT 2011
There are half a dozen global Regexp definitions.

• David Bond

pete.kowalski - Mon May 23 13:34:37 EDT 2011
The regexp is your problem then probably.

The regular expression engine was rewritten somewhat in DOORS 9.X and now it consumes more memory compared to previous DOORS versions.

I believe the "delete" function was overloaded to now support regexp in DOORS 9.X so if you are not deleting your regular expressions try that.

David_G_Bond - Mon May 23 15:18:07 EDT 2011
OK, so I moved the global Regexp definitions into the subroutines that use them and delete them at the end of the subroutine. I'll let you know how that works when they are defined locally and then deleted.

We've always made Regexp's global to minimize memory leaks, but we'll try it this way. It's a bit of a shame, though, because now we will have the same definition in multiple places.

• David Bond

David_G_Bond - Mon May 23 16:29:10 EDT 2011
The changes to the Regular Expression handling made no difference. The script failed in exactly the same place.

• David Bond

Mathias Mamsch - Tue May 24 07:50:30 EDT 2011
I did some work on DOORS memory managament in DOORS 8.2 recently - I hooked into the memory management to find out exactly where in the code the memory leaks are happening. I would like to test that with your scripts on DOORS 9. If you want, we can get in touch and see if the tools will be able to determine the cause of the memory leak, so we can file a bug report or correct a possible bug in the scripts. Mail me if you are interested m_m_amsch@googlemail.com (remove underscores).

Regards, Mathias

jdoucement - Thu May 26 10:30:44 EDT 2011
Hi everyone,

Just like Mathias I experienced memory management troubles those days with DOORS 8.3. I didn't use any regexp, I'm just using DXL attributes with objects having up to 2000 incoming links. Anyway, I think the way I got out of this issue could help you.
Give a try to pragmas.
Like:
pragma runLimit, 0 // actually no limit
pragma stack, 10000000 // it looks huge, doesn't it? In fact, not so huge if you consider all the recursive loops that might be caused by regexps.

Let us know!
Cheers,

Jean

jdoucement - Thu May 26 10:30:44 EDT 2011
Hi everyone,

Just like Mathias I experienced memory management troubles those days with DOORS 8.3. I didn't use any regexp, I'm just using DXL attributes with objects having up to 2000 incoming links. Anyway, I think the way I got out of this issue could help you.
Give a try to pragmas.
Like:
pragma runLimit, 0 // actually no limit
pragma stack, 10000000 // it looks huge, doesn't it? In fact, not so huge if you consider all the recursive loops that might be caused by regexps.

Let us know!
Cheers,

Jean

David_G_Bond - Mon May 23 16:29:10 EDT 2011
The changes to the Regular Expression handling made no difference. The script failed in exactly the same place.

• David Bond

SystemAdmin - Thu May 26 21:10:11 EDT 2011
Without seeing the code we're starting to clutch at straws and guess as to which library functions IBM may have modified in 9.3 that have become memory hungry.

An outside chance - one potential source of memory hogging is when opening a module - is the script opening modules invisible or visible (invisible uses less memory)?

W.r.t. the two modules that you mentioned had managed to open before failure set in - do either of these modules have triggers running DXL code that maybe using Regexp, or maybe a default view with DXL layout or DXL attributes that maybe the source of memory hogging? A possible knock on effect is that if there are any DXL layout's in the default view that trace to incoming\outgoing links, then the modules at the other end of these links will be opened invisible as well and will consume memory.

To help exclude the modules from being a part of the problem - keep a watch on memory usage in the Windows Performance Monitor as you open and close the modules manually, if any large memory blips appear and don't disappear after closing then maybe this is where to concentrate further.

David_G_Bond - Fri May 27 16:16:07 EDT 2011
Paul,

Thanks for your thoughts. I can't provide the script because it is both very big and filled with proprietary information.

We open the modules invisible and in standard view. There are triggers on some of the modules, but they are only activated if the modules are open in edit mode. We open the modules in read mode since data is gathered for a report and no changes to any modules are made. Standard view ensures that there are no layout dxls.

I use the Windows task manager and I see the memory use go up as modules are opened and then go down as they are closed. Whey the memory use goes down, it never goes to the same level as before the modules are processed. I know there are memory leaks - I just don't know where they might be or why they should be more severe than with DOORS 8.3.

• David Bond

SamLund - Mon May 30 10:53:00 EDT 2011
Quite recently, we had similar experiences, including significant "residual" reservation for memory.

Situation got a bit improved with strict adherence to following:

• delete any skip lists after the operation
• use preferably integer indexed skip lists
• setempty and delete any buffers after the operation
• set any string variables to null after the operation
• not a single variable what so ever to be declared inside any sort of loop.
• global string variables (only, or mainly). Minimize number of string variables declared.

• AND check attribute DXL for the same! Module misbehavior counts!

David_G_Bond - Tue May 31 15:15:03 EDT 2011
Sam,

Thanks for your suggestions. The script incorporates all of your suggestions. We never declare any variables inside of loops. We properly restore dynamic memory involving skip lists, buffers, and regular expressions. String variables are handled with great care.

To ensure that layout dxls are not an issue, all modules are opened (read-only, invisible) in standard view. Very few of our attributes are dxl attributes and none of those are involved in the script. If and when I discover the causes I will be sure to share that information here.

Thanks,

• David Bond