Deleting structures

Mathias Mamsch - Fri Jun 01 16:08:39 EDT 2012

Well the stuff that can be created and deleted is all the internal data structures of DXL (that are not bound to persisting data in DOORS)
 

create/delete            : Array, Buffer, Skip, DxlObject, Regexp, OleAutoArgs, Stat
create/oleCloseAutoObject: OleAutoObj 
create / destroy         : DB
client, server / delete  : IPC
Probably also: HTTPBody, HTTPHeader, HTTPHeaderEntry,

A special case is Stream, which you can close, but that will not free the Stream, i.e. a Stream is always leaked.

Then there is stuff that is bound persistently to DOORS Data:

Module, Item, Project, View, ...

Since they are bound to a specific DOORS instance, they are managed by DOORS internally and would need no destructor. They will also not create an entry in the allocated objects list. However there are special cases, for which this is not the case. They have no destructors, but will still create an entry in the allocated object list (what I consider a bug):

Baseline, Column, (possibly more?), ...

ModuleVersion is a very bad case, because this one actually has a destructor (delete (ModuleVersion)) but it also has a copy-assignment operator that will leak ModuleVersions uncontrolled unless you are very careful. Worst thing is, that DXL will call the copy-assignment operator only when the assignment is not part of the declaration. Example:

// targetVersion creates a new ModuleVersion 
ModuleVersion mv = targetVersion theLink  // assignment in declaration, will not create a copy of the ModuleVersion
delete mv // free the ModuleVersion, everything is fine 
 
 
ModuleVersion mv = null 
// this creates a copy of the targetVersion result, and will leak the targetVersion result
mv = targetVersion theLink   // calls the ::=(ModuleVersion &, ModuleVersion) copy-assignment operator!
delete mv // delete only the copy

Regarding the question, which DOORS Structures can be deleted without actually modifying the database is a good question. I did not try many of them, but it would surprise me if it were many. I am not sure if ViewDef creates an allocated object list entry. But ViewDefs are kind of complicated anyway, do you know any other structures that can be deleted this way?

Regards, Mathias

Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

llandale - Mon Jun 04 17:24:25 EDT 2012

I see ModuleProperties didn't make your lists. Are there just a couple, or perhaps many others that we need to deal with? Can we -- meaning you -- create a "void delete(&ModuleProperties)" function?

I'm sure you have some evidence of your "ModuleVersion" anomaly, but do you have an explanation and/or some way to deal with it? Does this solve the problem?

ModuleVersion targetVersionOfLink(Link lnk)
{  // Get the ModuleVersion of the target of the link
   // Caller should delete the mv when finished with it
   ModuleVersion mv = targetVersion(lnk)
   return(mv)
}  // end targetVersionOfLink()
 
ModuleVersion sourceVersionOfLink(Link lnk) ditto

-Louie

Mathias Mamsch - Thu Jun 07 10:16:12 EDT 2012

I answered the ModuleVersion thing here: https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14832712&#14832712

Regarding the ModuleProperties thing I guess I forgot about that. But I really do not know how to find out what types we need to deal with, other than trying. Using the allocatedObjects function from the above post one can determine if an object needs to be freed. For ViewDef for example the code looks like that:
 

print "Allocated Objects Before Declaration: " allocatedObjects() "\n"
ViewDef vd = createPublic() 
print "Allocated Objects After Declaration: " allocatedObjects() "\n"

If you run this you see, that there is no entry in the allocated objects list. So does this mean that it will permanently leak memory? No. There is in fact another category of objects in DXL, the most known one possibly the Attr__ type.

if (null current Object) error "Please run this script from a module with an object!"
 
print "Allocated Objects Before Declaration: " allocatedObjects() "\n"
Attr__ x = (current Object)."Object Text"  // allocate the Attr__ 
print "Allocated Objects After Declaration: " allocatedObjects() "\n"
string s = x   // assignment will free the Attr__
print "Allocated Objects After Assignment: " allocatedObjects() "\n"

My guess is that the same is true for ViewDef objects. Creating a ViewDef only makes sense with saving a view, therefore hopefully DOORS will not leak the ViewDef memory, but free it as soon as you save the view definition. The only way to test this is to try to assign the ViewDef after saving the view, and see if eventually you get a crash (sign of View Definition being freed). The same will happen if you try to assign the Attr__ twice.

So what types exactly are you worried about?

Regards, Mathias

llandale - Sat Nov 10 12:54:57 EST 2012
I've got a batch script that finds all recently modified modules, opens them Exclusive, then saves and closes without making changes. The idea is to improve database access on the several huge Shared modules we have; periodically saving them Exclusive is supposed to consolidate database files and improve open times.

I use "ModuleProperties" to query the "Last Modified On" date of the module, figuring to ignore it if more than 8 days old. I notice memory useage increases rather dramatically and I presume it to be the large number of ModuleProperties allocated units.

Can you create a "void delete(&ModuleProperties)" function?

-Louie

Mathias Mamsch - Sat Nov 10 13:36:15 EST 2012
Well since the ModuleProperties go to the allocation list it seems to be the best way instead of a very bad hack to try to call the internal cleanup function to use eval_ to read the ModuleProperties to a buffer. I showed here how to do it:

http://www.ibm.com/developerworks/forums/click.jspa?searchID=-1&messageID=14849772

Regards, Mathias

llandale - Sun Nov 11 14:34:18 EST 2012

You know I have an unreasonable resistance to outside-the-box voodoo. Never-the-less I ran your code in that post; wrote my own "string" instead of "Buffer" version; and wrote a test to see how long each takes along with using the normal non-eval version.

I was rather shocked at the results:

• all versions take about the same amount of time, 150ms
• 150ms seems like a whopping amount of time to me, getProperties() seems to be the culprit
• "eval_" didn't add any significant amount of time.
• creating a bunch of structures and running the code increased the Buffer and normal versions somewhat.

I'm inclined to use the String version because I would expect the Buffer version to deteriorate in time in large programs that create a lot of other structures.

Thoughts? Like perhaps the Buffer version allows distinction between results and errors and that matters.

-Louie

string    NameMod_Target        = "/Testing/Document1"
string  Results_Expected        = "Testing_"  // Prefix of module
pragma runLim, 0
 
//******************************************
string  fModAttr_GetValue(ModName_ in_mn, string in_NameAttr) 
{     // Get the module attr value without opening the module
        // This is done using ModuleProperties but since there is no delete for
        //      that structure, this function uses "eval_" to do it
        //      presuming the structure is removed after eval_() ends
        if (null in_mn  or null in_NameAttr)            then return("")       // Program input error
 
        string  ErrMess, EvalResult = ""
        ModuleVersion   mv = moduleVersion (in_mn) 
        if (null mv)
        {  print "fModAttr_GetValue(); bad moduleVersion\n"
           return("")
        }
        string  EvalCode = "
                ModName_        mn = addr_ " (int (addr_ in_mn)) "
                ModuleVersion   mv = addr_ " (int (addr_ mv))    "
                ModuleProperties mp
                getProperties (mv, mp) 
                return_(mp.\"" in_NameAttr "\" \"\")
        "  // end string EvalCode
        // print "Evaluating EvalCode: " EvalCode "\n\n"
        ErrMess = checkDXL(EvalCode)
        if (!null ErrMess)
        {  print "fModAttr_GetValue(); checkDXL() error [" ErrMess "\t]... for code:" EvalCode "\n"
        }
        else
        {  noError()
           EvalResult = eval_(EvalCode)
           ErrMess      = lastError() 
           if (!null ErrMess)
           {  print "fModAttr_GetValue() eval_() error [" ErrMess "\t]... for code:" EvalCode "\n"
              EvalResult        = ""
           }
        }
        delete(mv)
        // print "EvalResult [" EvalResult "]\n"
        return EvalResult
}    // end fModAttr_GetValue()
 
//******************************************
Buffer readModuleProperty (ModName_ mn, string aName) {
        Buffer buf = create() 
        string sCode = "
                ModName_ mn = addr_ " ((addr_ mn)  int) "
                Buffer buf  = addr_ " ((addr_ buf) int) "
                ModuleVersion mv = moduleVersion mn 
                ModuleProperties mp
                getProperties (mv, mp) 
                buf = mp.\"" aName "\"
        "
        // print "Evaluating sCode: " sCode "\n\n"
        noError(); string result = eval_ sCode; string errMsg = lastError() 
        if (!null result || !null errMsg) { delete buf; buf = null Buffer; print errMsg }
        return buf
}
 
 
void    DoTest_Buffer(int in_NumLoops)
{
        Buffer  x
        ModName_        mn = module(NameMod_Target)
        int     i, TimeStart = getTickCount_()
        for (i=0; i<in_NumLoops; i++)
        {  x = readModuleProperty(mn, "Prefix") 
           if (!null x) delete x
        }
        print "\t" (getTickCount_() - TimeStart) "ms\tDoTest_Buffer\n" 
        x = readModuleProperty(mn, "Prefix") 
        if (!null x) 
        {  if (tempStringOf(x) != Results_Expected) print "\t\tBAD results [" x "]\n"
           delete x     
        }
        else print "\t\tNull buff results\n"
}
 
 
void    DoTest_String(int in_NumLoops)
{
        string  x
        ModName_        mn = module(NameMod_Target)
        int     i, TimeStart = getTickCount_()
        for (i=0; i<in_NumLoops; i++)
        {  x = fModAttr_GetValue(mn, "Prefix") 
        }
        print "\t" (getTickCount_() - TimeStart) "ms\tDoTest_String\n" 
        if (x != Results_Expected) print "\t\tBAD results [" x "]\n"
}
 
 
void    DoTest_Normal(int in_NumLoops)
{
        string  x
        ModName_        mn = module(NameMod_Target)
        ModuleVersion   mv
        ModuleProperties mp
                //      mv = moduleVersion mn 
                //      getProperties (mv, mp) 
        int     i, TimeStart = getTickCount_()
        for (i=0; i<in_NumLoops; i++)
        {  mv = moduleVersion mn 
           getProperties (mv, mp) 
           x = mp."Prefix"
        }
        print "\t" (getTickCount_() - TimeStart) "ms\tDoTest_Normal\n" 
        if (x != Results_Expected) print "\t\tBAD results [" x "]\n"
}
 
void    DoAllTests(int in_NumLoops)
{
        print "Test: #Loops: " in_NumLoops "\n"
        DoTest_Normal(in_NumLoops)
        DoTest_Buffer(in_NumLoops)
        DoTest_String(in_NumLoops)
        DoTest_Normal(in_NumLoops)
}
 
if (false)
{    // Create a bunch of structures.  100,000 took about 2 minutes
        int i
        Buffer x
        for (i=0; i<30000; i++) x = create()
}
 
DoAllTests(100)         // 100 loops

Results:
 
Test: #Loops: 100
    1547ms      DoTest_Normal
        1453ms  DoTest_Buffer
        1469ms  DoTest_String
        1594ms  DoTest_Normal

Mathias Mamsch - Sun Nov 11 15:41:42 EST 2012
Well - i am not surprised by 150ms - after all getProperties is a network operation. It will read the module attribute file from the server and evaluate it. Regarding the timing I am really surprised, that there is no caching in the DOORS client. I would expect, that getProperties takes a long time on the first run and no time on the next runs. All in all it seems there is no need to any optimization in DXL as long as you are not dealing with stuff, that takes more than 100ms.

Of course if you expect DXL programs with lots of allocations the string version will exceed the buffer version in its performance. The error checking could either be done in the eval or you put some string in front of the result and check for that string. So that is no real reason to use the buffer variant. So I guess you will be fine using the string variant, as long as you don't query very long richText values (if that is even possible).

Regards, Mathias