DXL Script for Automatic Linking

Hello,

I need a DXL script that automatically links acronyms in a source module to acronyms defintion in a target module. Basically, it would read the object heading which is the acronym definition in my target module, then searches each object  in the source module for the acronym then automatically links the two objects if a match was found. Is there a script that would get this accomplished?

Thank,

~ Hanna

Re: DXL Script for Automatic Linking
smarti.sj - Mon Mar 17 13:08:43 EDT 2014

Hi Hanna

I may not understand fully what you are trying to do, but what is the goal of these links? To be able to easily get to the acronym definition?  If so may I suggest you create a little script that prompts the user for the unknown acronym and then looks in the Acronym module for the definition then spits it back to the user?  further, assign it a keyboard shortcut that makes it even easier to use.  I am worried these links may detract from other more important links if you have many acronyms.

Sara

Re: DXL Script for Automatic Linking
hyaldo - Mon Mar 17 13:23:53 EDT 2014

Re: DXL Script for Automatic Linking
llandale - Mon Mar 17 13:30:33 EDT 2014

• mAcronum = read(accronym module)
• Skip skpAcs = createString()  // KEY 'string' Acronym, DATA 'Object' handle
• for objAcr in mAcronum do
• {  if this is not an acronym object then continue
•    put(skpAcs, Acronym, objAcr
• }
• mSource = edit(Source module)
• buffer bufText = create()
• for oSource in mSource do
• {  if oSource doesn't qualify then continue
•    setempty(bufText)
•    bufText = oSource."Object Heading"
•    for Acronym in skpAcs do
•    {  if (contains(bufText, Acronym))
•       { oAcr = (Object key skpAcs)
•          if there is no such link then create link from oSource to oAcr using the Acronym Link Module
•       }
•       else
•       {  if there is indeed such a link then delete it
•       }
•    }
• }
• flushDeletions
• save(mSource)

The buffer "contains" function, I think, will find the acronym e.g. "ABC" but not when it is in some other string e.g. "sABC" or "ABCD".  You will need to test that.

-Louie

Re: DXL Script for Automatic Linking
hyaldo - Mon Mar 17 17:40:21 EDT 2014

Thanks for the sample code. I tried your function but for some reason I am still having trouble getting it to work. The 'contains' function doesn't seem to work properly as it's constantly returning -1 and when I try to print the Acronyms from inside the for loop in  "for Acronym in skpAcs do" , I am getting garbage data...and it seems like it's looping on all objects instead of the Acronyms only that are in the skip list. I just found some code on Smart DXL website which has a simple GUI that I'd like to use. I modified it so that it works for my specific needs and replaced the "doCreateLinks" with your sample code which I am attaching here. If you please look at the "doCreateLinks" function and see what I am missing, I would appreciate it.

 Thanks!

 ~ Hanna

Re: DXL Script for Automatic Linking
M_vdLaan - Wed Mar 19 10:58:50 EDT 2014

Louie, Hanna,

Looking at your code, Louie, it looks like you may have mixed up your key and object in the skpAcs; you're placing the object in the Skip with the acronym as the key, but int the loop " for Acronym in skpAcs do" you try to get the value of the Skip in the Acronym (which will probably not work since it's a string). Also you want to get the key using " oAcr = (Object key skpAcs)" but this is actually the string item. Am I making sense?

You probably want something like:

for oAcr in skpACs do {
    Acronym = (string key skpAcs)
    
    ...
    
}

This is probably the same thing you're getting mixed up, Hanna. Try swapping the key and value in the loop round line 139.

Sorry, cannot test it now, I'm away from DOORS, currently.

Good luck!

Marcel

Re: DXL Script for Automatic Linking
hyaldo - Thu Mar 20 17:54:15 EDT 2014

for oAcr in skpACs do {
    Acronym = (string key skpAcs)
    
    ...
    
}

Thank you all for the responses. This was actually my issue and I did come up with a work-around which works perfect with smaller number of objects. However, I seem to run into a bigger issue now.  DOORS is timing out when I run my code with this warning: "DXL Execution timeout" because I am running my DXL against very large modules with hundreds of objects. I did disable the timer but my DXL is running extremely slow and I don't know how to optimize it. This is the code that's causing it to run slow:

Any help is very much appreciated!!

~ Hanna

for oSource in mSource do
 { 

  // check for references in the Object Text attribute
  if (ObjectTextVal == true)
  {
   for objAcr in mReference do
   { 
    //if this is not an reference object then continue   
    //string Reference= trg."Object Heading"
    
    Reference = objAcr."Object Heading"
   
    if (objAcr."Definition" "" != "")
    {
     put(skpAcs, Reference, objAcr)
       
     string bufText1 =  oSource."Object Text"
   
     //string Scenario1 = "( )" Reference "( )"
     string Scenario2 = "([ ]|[.]|[:]|[;]|[,]|[(])"Reference"([ ]|[.]|[:]|[;]|[,]|[)])"

     Regexp FindReference = regexp Scenario2
     match1 = matches (Reference, bufText1)
         
     if (FindReference bufText1)
     {
      //print Reference"\n"
      //print bufText1 "\n"
      oSource ->LinkModName->objAcr
     }

    
    }
   }
 
  }

Re: DXL Script for Automatic Linking
M_vdLaan - Fri Mar 21 15:26:25 EDT 2014

Hello Hanna,

I can indeed see a few places where you can optimize:

1. You can optimize the loops;  you now how two nested loops where this is not really necessary. This is probably the number one resource hog. The inner loop can be moved out into its own. This will create the reference cache. In the second loop you can use this cache to link the objects
2. You are creating a lot of regexp's which may not be necessary. Perhaps you can optimize this. But I don't know how many acronyms you have.

I've throw together the following code as a framework for what I mean. This also incorporates a few other improvements.

Buffer bufAllRefs = create

// First build a cache of all objects that have a definition
Skip skpAcs = createString
string sReference
Object objAcr
for objAcr in mReference do { 
  if (objAcr."Definition" "" != "") {
    sReference = objAcr."Object Heading"
    put(skpAcs, sReference, objAcr)
    bufAllRefs += sReference 
    bufAllRefs += "|"
  }
}
bufAllRefs = tempStringOf(bufAllRefs)[0:length(bufAllRefs)-2]

// Take the Regexp outside the loop since redefining thisevery loop iteration
// is inefficient
Regexp reReference = regexp "([ .:;,\\()])(" bufAllRefs ")([ .:;,\\)])"

// Now we have te regular expression and the cache, iterate over the objects in 
// the 'source' module mSource
Object oSource
string sText
for oSource in mSource do {
  // Not sure what this next condition is meant to do.
  sText = oSource."Object Text"
  if (reReference sText) {
    sReference = sText[match 2]
    if (find(skpAcs, sReference, objAcr)) {
      // Assuming LinkModName is already defined...
      oSource ->LinkModName->objAcr
    }
  }
}

// clean up the Buffer
delete bufAllRefs

Hope this helps. Let me know how this works out for you.

Good luck,

Marcel

Re: DXL Script for Automatic Linking
hyaldo - Mon Mar 24 14:29:49 EDT 2014

Buffer bufAllRefs = create

// First build a cache of all objects that have a definition
Skip skpAcs = createString
string sReference
Object objAcr
for objAcr in mReference do { 
  if (objAcr."Definition" "" != "") {
    sReference = objAcr."Object Heading"
    put(skpAcs, sReference, objAcr)
    bufAllRefs += sReference 
    bufAllRefs += "|"
  }
}
bufAllRefs = tempStringOf(bufAllRefs)[0:length(bufAllRefs)-2]

// Take the Regexp outside the loop since redefining thisevery loop iteration
// is inefficient
Regexp reReference = regexp "([ .:;,\\()])(" bufAllRefs ")([ .:;,\\)])"

// Now we have te regular expression and the cache, iterate over the objects in 
// the 'source' module mSource
Object oSource
string sText
for oSource in mSource do {
  // Not sure what this next condition is meant to do.
  sText = oSource."Object Text"
  if (reReference sText) {
    sReference = sText[match 2]
    if (find(skpAcs, sReference, objAcr)) {
      // Assuming LinkModName is already defined...
      oSource ->LinkModName->objAcr
    }
  }
}

// clean up the Buffer
delete bufAllRefs

Thank you Marcel! I tried your code and for some reason, I am getting an error on this line:

bufAllRefs = tempStringOf(bufAllRefs) [0:length(bufAllRefs)-2]. [0:length(bufAllRefs)-2] seems to be invalid syntax and when I try to look up the tempStringOf function, I can't seem to find it in DXL reference manual? I tried the code without that portion, and I am having trouble now finding the acronyms. Also the \\) does not seem to be valid either. This code seems to only find one acronym...If I have more than one acronym in my object text, it won't link it. Please assist.

Thank you so much for your help!!

~ Hanna

Re: DXL Script for Automatic Linking
M_vdLaan - Mon Mar 24 16:19:58 EDT 2014

Hanna,

Are you getting a compile-time error or run-time? Which version of DOORS are you running?

Okay, you will need to take into account that that piece of code will fail if the no acronyms are found (in which case length(bufAllRefs) will be 0 and the substring extraction will indeed fail. Add a check before you continue or print out the buffer to check it's value. The tempStringOf function is an undocumented one. There are a quite a few of these. 

Also, to find multiple acronyms in the second loop, you'll need to change the if to a while loop that finds the first one, extracts it, and reassigns the sText to be the remaining text. I think it's along the lines of:

while (reReference sText) {
  sReference = sText[match 2]
  sText = sText[(end 2) + 1:] // Check the syntax here ...
  
  if (find(skpAcs, sReference, objAcr)) {
    // Assuming LinkModName is already defined...
    oSource ->LinkModName->objAcr
  }
  
}

You might need to check what happens when the same acronym is used in the text multiple times, though.

What's not valid with the \\)? It might be a DOORS v8 vs v9 thing.

Sorry I can't be more specific at the moment. No access to DOORS currently.

Re: DXL Script for Automatic Linking
hyaldo - Mon Mar 24 17:42:23 EDT 2014

while (reReference sText) {
  sReference = sText[match 2]
  sText = sText[(end 2) + 1:] // Check the syntax here ...
  
  if (find(skpAcs, sReference, objAcr)) {
    // Assuming LinkModName is already defined...
    oSource ->LinkModName->objAcr
  }
  
}

I am getting a compile-time error. I use DOORS version 9.3. The length of the buffer is not 0 as there are acronyms in my object text that I was able to print. I added the code you suggested above with the while loop and I am still running into performance issues. It seems to work a little faster though but I have more than 800 acronyms that I am trying to auto link which is still causing my script to run failrly slow. Is there anything else that I am missing or should try? is there a way that you can only loop on the acronyms that you find and link those instead of looping on the entire acrony list? Here is the updated code that I have right now:

Buffer bufAllRefs = create
 // build a cache of all objects that have a definitions
 string sReference
  
 for objAcr in mReference do
 { 
  if (objAcr."Definition" "" != "")
  {
   
   sReference = objAcr."Object Heading"
   put(skpAcs, sReference, objAcr)
   bufAllRefs +=sReference
   bufAllRefs +="|"
  }
 }

 bufAllRefs = tempStringOf(bufAllRefs)
 //[0:length(bufAllRefs)-2]

 string Scenario2 = "([ .:;,\\()])("bufAllRefs")([ .:;,\\)])"

 Regexp reReference = regexp Scenario2

 string sText
 for oSource in mSource do
 { 
  
  //match1 = matches (Reference, sText)
  sText = oSource."Object Text"
  match1 = matches (sReference, sText)
  print sReference"\n"
  print match1 "\n"
  while (reReference sText)
  {
   sReference = sText[match 2]
   sText = sText[(end 2) + 1:]

   if(find(skpAcs, sReference, objAcr))
   {
    print sReference"\n"
    print bufAllRefs "\n"
    oSource ->LinkModName->objAcr
   }
  }
 }

 delete bufAllRefs

 Thank you so much for your help!!

~ Hanna 

Re: DXL Script for Automatic Linking
M_vdLaan - Mon Mar 24 18:15:58 EDT 2014

Hanna,

Well, yes, a regular expression with 800 acronyms is not going to be fast...

"is there a way that you can only loop on the acronyms that you find and link those instead of looping on the entire acrony list?"

What do you mean by "acronyms that you find"? Are acronyms identifiable in some way in the Object Texts? If, for example, they are marked using braces or square brackets (e.g. [API]) then that would be a lot easier. Alternatively, you could look for words in ALL CAPS. But this very much depends on how you would identify an acronym in the Object Text...

Regards,

Marcel

Re: DXL Script for Automatic Linking
hyaldo - Tue Mar 25 14:05:26 EDT 2014

What I meant is basically only looping on acronyms that are in the Object Text instead of looping on the 800 acronyms to find a match. The problem with that is my acronyms list is a mix of everything...(all caps, no caps, with braces, etc...), so it's hard to identify. Can we use another method other than regular expressions that may speed it up?

Thanks!

~ Hanna

Re: DXL Script for Automatic Linking
MaltePlath - Mon Mar 31 08:21:44 EDT 2014

I am puzzled by the line

match1 = matches (sReference, sText)

As far as I could see, sReference holds the last acronym you found in mReference, and then the last acronym you matched.  Why do you want to match that at all, when you are applying your big compound regexp anyway?

One thing that definitely slows down the script that you posted are the print statements.  (I know that you need them for debugging!).

As for the regexp, you could optimize that, but that would be another adventure in DXL: Don't look for (AAA|AAB|ABC|ABD...) but for (A(A(A|B)|(B(C|D))...) - and then character classes will be faster than disjunctions ('|') - for the final letters.

If you notice that it works fast at first and then slows down (use you can use a progressBar to observe that), I would take a look at your usage of strings.  The line

sText = sText[(end 2) + 1:]

in the while loop generates a lot of strings, which will use a lot of memory, esp. if your source module is large as you write.  You should try using a buffer for that purpose.  (Use setempty(Buffer) if you find that assigning your text to the buffer does not properly overwrite the previous content.)  setempty() is faster than deleting and creating a buffer.

I hope that this gives you some ideas for further optimisations.

Re: DXL Script for Automatic Linking
hyaldo - Wed Apr 09 14:05:34 EDT 2014

match1 = matches (sReference, sText)

sText = sText[(end 2) + 1:]

Thank you all for the tips! I was able to optimize it based on the few suggestions and now it runs in less than 1 minute instead of hours which is awesome! Couple of things though, The script does not link to the following acronyms:

 CBM+

ABD (P)

(T)

(O)

 All of the above are acronyms in my acronyms list and I tried to change the regular expressions to handle those cases but my methods didn't seem to work. Any ideas?

 Also I am trying to expand this script further to make it link to definitions also which I assumed this would be the exact same logic we have for acronyms which partially works. It would only link to definitions that contains one word. If my definitions contain 2 words or more, it won't link them. Any ideas?

 Thanks in advance for all the help!!

 ~ Hanna

bool ObjectTextVal = get(ObjectTextControl)
Buffer bufAllRefs = create
 
 // build a cache of all objects that have a definitions
 string sReference
  
 for objAcr in mReference do
 { 
  if (objAcr."Definition" "" != "")
  {
   
   sReference = objAcr."Object Heading"
    
   // don't link every in to "inches" and every T to Tons...
   if (sReference == "T" || sReference == "in" || sReference == "s" || sReference == "g" || sReference == "A")

   {
    print sReference "\n"
    // do nothing with it
   }
   
   // add it to the sReference list
   else
   {
    put(skpAcs, sReference, objAcr)
    bufAllRefs +=sReference
    bufAllRefs +="|"
   }
  }
 }

 bufAllRefs = tempStringOf(bufAllRefs)
 //[0:length(bufAllRefs)-2]

  string Scenario2 = "([ .:;,\\()])("bufAllRefs")([ .:;,\\()])"
 

 Regexp reReference = regexp Scenario2
 string sText
 
 // progress bar loop
 int nos = 0
 for oSource in mSource do nos++

 progressStart(addDB, "Linking References", "Something", nos)
 nos = 0
 
 // Linking Object text
 if (ObjectTextVal == true)
 { 
  for oSource in mSource do
  { 
   
   sText = oSource."Object Text"
   match1 = matches (sReference, sText)
 
  
   // to find multiple acronyms, find the first match, extract it,
   // and reassigns the sText to be the remaining text.
  
   bufText1 = null
   string n = number(oSource)
   string h = oSource."Object Heading"
    
   string message = "Linking Object Text" "\n" n " " h

   progressStep ++nos
   
   if (null h) h = "null heading"
          progressMessage message
          if (progressCancelled)
   {
    if (confirm("Exit loop?"))
    {
            progressStop
                   halt
    }
 
   }
 
   while (reReference sText)
   {
    sReference = sText[match 2]
    string sReference1 = sText[match 0]

    sText = sText[(end 2) + 1:]
    string h = oSource."Object Heading"
      
    if (find(skpAcs, sReference, objAcr) || find(skpAcs, sReference1, objAcr))
    {
         oSource ->LinkModName->objAcr
    }
   }
  }
 }

Re: DXL Script for Automatic Linking
MaltePlath - Thu Apr 10 06:03:10 EDT 2014

If you have special characters, ie. ones that have a special meaning in regular expressions, you need to escape them with '\' (Backslash) before you add them to bufAllRefs. For example "CBM+" must become "CBM\+" and "ABD (P)" should become "ABD \(P\)".

For a list of characters to escape look at this thread: https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014958442&ps=25 and the one linked from one of the first posts.

We have established a convention that certain entity names have to be written in bold face, which makes it possible (with some margin for error) to check the requirement text for the entities mentioned by looking for the Rich Text mark-up for bold face ("{\b (.+)}") and to validate the strings found against another list.  So if you could require your authors to use bold, italics, or underline for acronyms, that would solve your problem with ambiguous words (like "in") by turning the initial problem on its head: You would not be looking for given acronyms in your text but for "acronym mark-up".  You could use your current approach to convert the existing text to the appropriately marked up text.

Re: DXL Script for Automatic Linking
hyaldo - Thu Apr 10 16:24:03 EDT 2014

I added the backslash to my acronoyms just to test it out and my bufAllRefs now looks like:

Acronym1|CBM\+|Acronym2|

and that still didn't seem to link it. Is there anything else that I am missing? Thanks!

Re: DXL Script for Automatic Linking
MaltePlath - Fri Apr 11 07:27:54 EDT 2014

If understand you correctly, you have change the Object Heading in the acronyms module.

Now your escaped acronym (CBM\+) does not match what you find in the Object Text, because if you added the backslash in the acronym module, you are looking for "CBM+" (sText[match 2]) in your skip list but that contains "CBM\+".  You need to escape the special characters in the regexp but not in the skip list.

For a trial run, you could do a literal comparison for "CBM+" (like you have for "T", "in", etc.) and put the escaped version in the buffer for the regexp.  For production, you would definitely want to pass all acronyms through a function for escaping special characters.