Trigger to prevent excessive copy and paste RTF characters

Hi there,

I am looking at ways I can encourage some of my users to insert Office sourced content (think Excel) as OLEs rather than the more intuitive way which is for them to copy and paste a table from excel/word/whatever.

The prompt for this is that we've been finding that we are getting some objects with excessive amounts of Rich Text Format characters which cause other problems -particularly with clarity of the obejcts when published to word via RPE and in some cases the hidden RTF characters can even crash an RPE report entirely.

I have created a trigger on object modification so that if a user modifies an existing object which has a large ratio of RTF characters vs plain text characters they're asked to insert it as an OLE instead, however I can't seem to set anything up on object creation.

It sure would be nice to be able to parse a newly created object to ensure the content isn't going to cause other problems, but interestingly, doors doesn't appear to have the option to create a trigger on object creation.

I have considered doing something on object save or module close, but I think the performance impact would make this option unfeasible -particularly if needing to trawl through the history.

Does anyone have any other ideas on how I can configure the tool to become more proactive with respect to RTF characters?

Best Regards,

Jack
JackSuss - Wed Oct 19 18:52:15 EDT 2011

Re: Trigger to prevent excessive copy and paste RTF characters
OurGuest - Thu Oct 20 07:57:53 EDT 2011

You can clean remove unwanted rtf characters.

Seems like smartdxl or galactic-solutions has a script for this.

Re: Trigger to prevent excessive copy and paste RTF characters
JackSuss - Thu Oct 20 17:15:56 EDT 2011

OurGuest - Thu Oct 20 07:57:53 EDT 2011
You can clean remove unwanted rtf characters.

Seems like smartdxl or galactic-solutions has a script for this.

Our Guest,

Thanks for the response. I'm not too concerned about the payload, what I need to be able to do is to check an object when it is created so these sorts of scripts won't be necessary.

The problem with these scripts is that if someone has copied a table from excel, it will strip the table down. My trigger will just ask the user to embed the object as an OLE instead and explain the reasons why.

Best Regards,

Jack

Re: Trigger to prevent excessive copy and paste RTF characters
Mathias Mamsch - Thu Oct 20 19:00:55 EDT 2011

JackSuss - Thu Oct 20 17:15:56 EDT 2011
Our Guest,

Thanks for the response. I'm not too concerned about the payload, what I need to be able to do is to check an object when it is created so these sorts of scripts won't be necessary.

The problem with these scripts is that if someone has copied a table from excel, it will strip the table down. My trigger will just ask the user to embed the object as an OLE instead and explain the reasons why.

Best Regards,

Jack

Well, there have been some experiments using sync triggers (when you create a new object the current object will change) and checking in history if that object was just created, but this will not work in every case.

What I do not get is: When you copy / paste from Office you will paste to an already existing object and you would be able to have an attribute change trigger react on the pasting of the content. Why would you want to trigger on object creation if your problem is copy pasting from Excel/Word? Did I miss some DOORS 9+ feature, where copy pasting from office will automagically create a new object?

Regards, Mathias

Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

Re: Trigger to prevent excessive copy and paste RTF characters
JackSuss - Thu Oct 20 22:36:05 EDT 2011

Mathias Mamsch - Thu Oct 20 19:00:55 EDT 2011
Well, there have been some experiments using sync triggers (when you create a new object the current object will change) and checking in history if that object was just created, but this will not work in every case.

What I do not get is: When you copy / paste from Office you will paste to an already existing object and you would be able to have an attribute change trigger react on the pasting of the content. Why would you want to trigger on object creation if your problem is copy pasting from Excel/Word? Did I miss some DOORS 9+ feature, where copy pasting from office will automagically create a new object?

Regards, Mathias


Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

Hi Mathias,

The concern I have with the sync trigger approach is performance.

I find the users do the following:

1) Create the Object (without saving)
2) Paste Content in
3) When finished, they save the module

So it only gets recorded as a create rather than a modify. For the modify trigger to fire, step 2 would have to be preceded with a save.

BTW- thanks for the reponses on this forum, being a lurker for a while I have found it very useful.

Best Regards,

Jack

Re: Trigger to prevent excessive copy and paste RTF characters
Mathias Mamsch - Thu Oct 20 22:59:12 EDT 2011

JackSuss - Thu Oct 20 22:36:05 EDT 2011
Hi Mathias,

The concern I have with the sync trigger approach is performance.

I find the users do the following:

1) Create the Object (without saving)
2) Paste Content in
3) When finished, they save the module

So it only gets recorded as a create rather than a modify. For the modify trigger to fire, step 2 would have to be preceded with a save.

BTW- thanks for the reponses on this forum, being a lurker for a while I have found it very useful.

Best Regards,

Jack

If you create a pre attribute modification trigger it will fire, just before the attribute value is written to the attribute, e.g. just before the value is pasted. I am not talking about a pre module modification trigger. The attribute modification trigger should suit your need of checking the contents before letting the user paste.

See for example that post:
https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14478138&#14478138

Regards, Mathias

Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

Re: Trigger to prevent excessive copy and paste RTF characters
JackSuss - Fri Oct 21 01:00:37 EDT 2011

Mathias Mamsch - Thu Oct 20 22:59:12 EDT 2011
If you create a pre attribute modification trigger it will fire, just before the attribute value is written to the attribute, e.g. just before the value is pasted. I am not talking about a pre module modification trigger. The attribute modification trigger should suit your need of checking the contents before letting the user paste.

See for example that post:
https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14478138&#14478138

Regards, Mathias


Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

Mathias,

Thanks for the pointer. I got it going as a post modification trigger whcih does fire with the scenario given.. The code follows:

string DXLCode =
" int intThreshold = 500 // CHANGE HERE TO RAISE OR LOWER THE TOLERANCE THRESHOLD

set(trigPreConPass)
Trigger trg = current; if (null trg) halt() // Error?

AttrDef ad = attrdef(trg); if (null ad) halt() // Error?
Object obj = object(trg); if (null obj) halt()
string Name = ad.name

int richLen = length richText obj.\"Object Text\"
int plainLen = length obj.\"Object Text\" \"\"
if (Name == \"Object Text\") {
if ((richLen-plainLen) > intThreshold) {
infoBox(\"This text has more than \" intThreshold \" RTF characters than plain text characters. Please insert this as an OLE object instead\")
}
}
" // end string DXLCode

string NameTrig = "DetectRTFTextChange"
Trigger trg
for trg in (current Project) do
{ if (name(trg) == NameTrig and
confirm("Delete trig '" NameTrig "'??"))
{ delete(trg)
halt // or break
}
}
if (confirm("Create trigger '" NameTrig "' in project??"))
{ trigger(NameTrig, project->module->all->attribute,
post, modify, 5, DXLCode)
}

Best Regards,

Jack

Re: Trigger to prevent excessive copy and paste RTF characters
Mathias Mamsch - Fri Oct 21 06:47:09 EDT 2011

JackSuss - Fri Oct 21 01:00:37 EDT 2011
Mathias,

Thanks for the pointer. I got it going as a post modification trigger whcih does fire with the scenario given.. The code follows:

string DXLCode =
" int intThreshold = 500 // CHANGE HERE TO RAISE OR LOWER THE TOLERANCE THRESHOLD

set(trigPreConPass)
Trigger trg = current; if (null trg) halt() // Error?

AttrDef ad = attrdef(trg); if (null ad) halt() // Error?
Object obj = object(trg); if (null obj) halt()
string Name = ad.name

int richLen = length richText obj.\"Object Text\"
int plainLen = length obj.\"Object Text\" \"\"
if (Name == \"Object Text\") {
if ((richLen-plainLen) > intThreshold) {
infoBox(\"This text has more than \" intThreshold \" RTF characters than plain text characters. Please insert this as an OLE object instead\")
}
}
" // end string DXLCode

string NameTrig = "DetectRTFTextChange"
Trigger trg
for trg in (current Project) do
{ if (name(trg) == NameTrig and
confirm("Delete trig '" NameTrig "'??"))
{ delete(trg)
halt // or break
}
}
if (confirm("Create trigger '" NameTrig "' in project??"))
{ trigger(NameTrig, project->module->all->attribute,
post, modify, 5, DXLCode)
}

Best Regards,

Jack

I am not sure if your approach is a good one. If you look at the richText then you will see that the richtext markup can vary largely.

  • You have a font table at the beginning of the richText.
  • You have a lot of markup for certain formattings (like bullet points)
  • If you have an OLE object inside, you will have LOTS of not printed richText


Setting a fixed threshold like 500 seems very arbitrary. You should think about looking for special keywords in the richText that come with some unwanted formatting. One technique I regularly use is make a layout DXL column:

 

display richText (obj."Object Text")



This way you can manually produce the unwanted results, and then try to find the richText patterns that come with such a result (e.g. for RTF tables this would be '\trowd', '\cell', etc.). Regards, Mathias



 

 


Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

 

Re: Trigger to prevent excessive copy and paste RTF characters
llandale - Fri Oct 21 14:24:24 EDT 2011

JackSuss - Fri Oct 21 01:00:37 EDT 2011
Mathias,

Thanks for the pointer. I got it going as a post modification trigger whcih does fire with the scenario given.. The code follows:

string DXLCode =
" int intThreshold = 500 // CHANGE HERE TO RAISE OR LOWER THE TOLERANCE THRESHOLD

set(trigPreConPass)
Trigger trg = current; if (null trg) halt() // Error?

AttrDef ad = attrdef(trg); if (null ad) halt() // Error?
Object obj = object(trg); if (null obj) halt()
string Name = ad.name

int richLen = length richText obj.\"Object Text\"
int plainLen = length obj.\"Object Text\" \"\"
if (Name == \"Object Text\") {
if ((richLen-plainLen) > intThreshold) {
infoBox(\"This text has more than \" intThreshold \" RTF characters than plain text characters. Please insert this as an OLE object instead\")
}
}
" // end string DXLCode

string NameTrig = "DetectRTFTextChange"
Trigger trg
for trg in (current Project) do
{ if (name(trg) == NameTrig and
confirm("Delete trig '" NameTrig "'??"))
{ delete(trg)
halt // or break
}
}
if (confirm("Create trigger '" NameTrig "' in project??"))
{ trigger(NameTrig, project->module->all->attribute,
post, modify, 5, DXLCode)
}

Best Regards,

Jack

Like the structure of the code!

So 'modify' is same as 'save' trigger. This code won't fire if you create an object with various forms of copy-object-past-object since the attr values seem to be pre-set. But it will fire in your scenario, create the object then copy-paste the text.

And it will fire when you import. That's probably NOT what you want as the user will need to sit there and Confirm your warning over and over. Maybe the trigger doesn't say anything unless inplaceEditing(Module) is true (user is actually editing).

If you only care about changes to Object Text then you should halt when Name is not "Object Text".
if (Name != \"Object Text\") halt()

You may also want to not warn them when they ARE indeed pasting an OLE diagram, looping through the RTPs and then RTs looking for one with rt.isOle. That sort of reasoning is why my DXL is always very long, but I digress.

The "set(trigPreConPass)" does nothing in a Post trigger. A Pre trigger could not query the obj.Name value (since it hasn't happened yet) and would need this:
... string ValueNominated = value(trg)
But I haven't thought about the raw text/rich text implications of that.

This is the perfect situation to use my fSplashMessage functions that display a message for x seconds without interfering with other DOORS work; too bad I never got that working. I wonder if you could 'realize' a display dialog, sleep_ for 5 seconds, then hide the dialog.

  • Louie

Re: Trigger to prevent excessive copy and paste RTF characters
JackSuss - Tue Oct 25 17:48:54 EDT 2011

Mathias Mamsch - Fri Oct 21 06:47:09 EDT 2011

I am not sure if your approach is a good one. If you look at the richText then you will see that the richtext markup can vary largely.

  • You have a font table at the beginning of the richText.
  • You have a lot of markup for certain formattings (like bullet points)
  • If you have an OLE object inside, you will have LOTS of not printed richText


Setting a fixed threshold like 500 seems very arbitrary. You should think about looking for special keywords in the richText that come with some unwanted formatting. One technique I regularly use is make a layout DXL column:

 

display richText (obj."Object Text")



This way you can manually produce the unwanted results, and then try to find the richText patterns that come with such a result (e.g. for RTF tables this would be '\trowd', '\cell', etc.). Regards, Mathias



 

 


Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

 

Hi mathias,

You're right, it is an arbitrary number. It sure would be handly to have a list of invalid RTF characters to search for under these circumstances, but as you have alluded to, this woul be largely trial and error.
Best Regards,

Jack

Re: Trigger to prevent excessive copy and paste RTF characters
JackSuss - Tue Oct 25 17:59:25 EDT 2011

llandale - Fri Oct 21 14:24:24 EDT 2011
Like the structure of the code!

So 'modify' is same as 'save' trigger. This code won't fire if you create an object with various forms of copy-object-past-object since the attr values seem to be pre-set. But it will fire in your scenario, create the object then copy-paste the text.

And it will fire when you import. That's probably NOT what you want as the user will need to sit there and Confirm your warning over and over. Maybe the trigger doesn't say anything unless inplaceEditing(Module) is true (user is actually editing).

If you only care about changes to Object Text then you should halt when Name is not "Object Text".
if (Name != \"Object Text\") halt()

You may also want to not warn them when they ARE indeed pasting an OLE diagram, looping through the RTPs and then RTs looking for one with rt.isOle. That sort of reasoning is why my DXL is always very long, but I digress.

The "set(trigPreConPass)" does nothing in a Post trigger. A Pre trigger could not query the obj.Name value (since it hasn't happened yet) and would need this:
... string ValueNominated = value(trg)
But I haven't thought about the raw text/rich text implications of that.

This is the perfect situation to use my fSplashMessage functions that display a message for x seconds without interfering with other DOORS work; too bad I never got that working. I wonder if you could 'realize' a display dialog, sleep_ for 5 seconds, then hide the dialog.

  • Louie

Hi Louie,

Thanks for the tips. Maybe there might be some way of changing the color of the Doors Window to indicate a possible issue?

I am finding that this appears to be skipping the embedded OLEs - probably because most of our OLES would have either come in via an import or via the insert OLE method.

I am finding that the trigger currently is nagging users -even with them using valid RTF characters, so i will need to identify the valid ones and strip them out -efficiently- somehow.

It's looking to me like to do this properly is going to end up with an adverse performance impact. The key with triggers is of course that they execute as quickly as possible. If I end up searching to stings in object text things could get really slow really fast -especially if I use regex.

Maybe there might be scope to list all the invalid RTF characters and search on them. It sure would be good if doors could strip out invalid RTF characters on a paste from the clipboard, but I digress.

One way or another, I need to be able to prevent the introduction of RTF characters that can cause an RPE report run to fail.

I will keep posting until we conclude something here.
Best Regards,

Jack

Re: Trigger to prevent excessive copy and paste RTF characters
llandale - Tue Oct 25 18:37:26 EDT 2011

JackSuss - Tue Oct 25 17:59:25 EDT 2011
Hi Louie,

Thanks for the tips. Maybe there might be some way of changing the color of the Doors Window to indicate a possible issue?

I am finding that this appears to be skipping the embedded OLEs - probably because most of our OLES would have either come in via an import or via the insert OLE method.

I am finding that the trigger currently is nagging users -even with them using valid RTF characters, so i will need to identify the valid ones and strip them out -efficiently- somehow.

It's looking to me like to do this properly is going to end up with an adverse performance impact. The key with triggers is of course that they execute as quickly as possible. If I end up searching to stings in object text things could get really slow really fast -especially if I use regex.

Maybe there might be scope to list all the invalid RTF characters and search on them. It sure would be good if doors could strip out invalid RTF characters on a paste from the clipboard, but I digress.

One way or another, I need to be able to prevent the introduction of RTF characters that can cause an RPE report run to fail.

I will keep posting until we conclude something here.
Best Regards,

Jack

I don't do much with richText, but these commands look promising:
string removeUnlistedRichText(string s)
string richTextFragment(string richString)
string richTextFragment(string richString, string fontTable)
string richTextFragment(string richText [, string fontTable [, bool
inTable]])

Your notion of searching the entire string doesn't seem realistic, but it would be more realistic if you could get a list of all known valid ones, and reject the ones you don't recognize. Thus you don't need a list of the invalid ones.

I wonder if a "rich text tag" is a string of an even number of back-slashes, followed by a string of alpha, optionally a single zero, followed by a space.

If it were me, I'd write that sloppy program, post it here, and let folks with more knowledge fill in the gaps.

  • Louie

Re: Trigger to prevent excessive copy and paste RTF characters
JackSuss - Tue Oct 25 18:58:57 EDT 2011

llandale - Tue Oct 25 18:37:26 EDT 2011
I don't do much with richText, but these commands look promising:
string removeUnlistedRichText(string s)
string richTextFragment(string richString)
string richTextFragment(string richString, string fontTable)
string richTextFragment(string richText [, string fontTable [, bool
inTable]])

Your notion of searching the entire string doesn't seem realistic, but it would be more realistic if you could get a list of all known valid ones, and reject the ones you don't recognize. Thus you don't need a list of the invalid ones.

I wonder if a "rich text tag" is a string of an even number of back-slashes, followed by a string of alpha, optionally a single zero, followed by a space.

If it were me, I'd write that sloppy program, post it here, and let folks with more knowledge fill in the gaps.

  • Louie

Hi Louie,

Yep, the removeUnlistedRichText function is what I used initially, but it actually strips out all RTF - including the valid Doors Bulletpoints etc. I logged a call with support and they came up wiht the idea of comparing RTFs with plain text to help identify and resolve the issue.

I still maintain that the removeUnlistedRichText function should strip out the invalid RTF characters for an object and leave the doors RTF characters in place.

If removeUnlistedRichText did as the documentation implies, I would just do a compare and then run the object text through it if the numbers were getting out of hand.

Thanks for the pointers to the other functions.

Best Regards,

Jack

Re: Trigger to prevent excessive copy and paste RTF characters
llandale - Tue Oct 25 19:27:38 EDT 2011

JackSuss - Tue Oct 25 18:58:57 EDT 2011
Hi Louie,

Yep, the removeUnlistedRichText function is what I used initially, but it actually strips out all RTF - including the valid Doors Bulletpoints etc. I logged a call with support and they came up wiht the idea of comparing RTFs with plain text to help identify and resolve the issue.

I still maintain that the removeUnlistedRichText function should strip out the invalid RTF characters for an object and leave the doors RTF characters in place.

If removeUnlistedRichText did as the documentation implies, I would just do a compare and then run the object text through it if the numbers were getting out of hand.

Thanks for the pointers to the other functions.

Best Regards,

Jack

Here's an incomplete scribbled thought; lets just rebuild the entire rich text string.

if (oleIsObject(obj)) return
sText = richTextNoOle(obj."Object Text")
RichTextParagraph    rtp
RichText                rt
Buffer  bufPara = create
Buffer  bufNew  = create
for rtp in sText do
{  bufPara    = ""
   for rt in rtp do
   {  bufPara += "\\{"
      if (rt.bold)              bufPara += "\\b "
      if (rt.italic)            bufPara += "\\italic "
      if (rt.strikethru)        butPara += "\\strike "
      if (rt.subscript)         bufPara += "\\subscript "
      if (rt.superscript)       bufPara += "\\superscript "
      if (rt.underline)         bufPara += "\\ul "
      bufPara += rt.text
      bufPara += "\\}
      if (rt.newline and !rt.last) bufPara += "\\par "
   }
        
   IndentLevel = rtp.indentLevel
   HasBullets  = rtp.isBullet
   bufNew       += applyTextFormattingToParagraph(stringOf(bufPara), HasBullets, IndentLevel, 0)
}
obj."Object Text Stripped" = tempStringOf(bufNew)

 

  • Louie