function string trim(string) no longer works with V9.5

I had a function that worked for years.  It seems that starting with DOORS 9.5, DXL started not passing a string parameter as a string unless I explicitly cast it as string or rename the function "trim" to something else like "trim1".  The prior version was 9.3.

Are there any new DXL built-in functions (aka PERMS) that could be causing a name clash?

It is pretty easy to reproduce.  See code below.  Sample data output follows.

// Bug Demo

/*
  Bug Demo
*/


string rightTrim(string s) {
    if ( null(s) ) return "";
    int FIRST = 0;            // constant left edge
    int last = length(s) - 1; // initial right edge
    // Starting from the right edge, hop over white space (if any)
    // Just don't fall off the left edge.
    while ( (last > -1) && (isspace(s[last])) ) {
       last--;
    }
    return s[FIRST:last]; // return substring with (possibly) modified last
}

string leftTrim(string s) {
    if (null(s)) return "";
    int first = 0;            // initial left edge
    int LAST = length(s) - 1; // constant right edge
    // Starting from the left edge, hop over white space (if any)
    // Just don't fall off the right edge.
    while ( (isspace(s[first])) && (first < LAST) ) {
       first++;
    }
    return s[first:LAST]; // return substring with (possibly) modified first
}

//------------------------------------------------------------------------------
// This is the old "string trim(string)" function that worked for years
//------------------------------------------------------------------------------
//string trim(string s) {
//    print("DEBUG: Here is what I'm supposed to trim [" s "]\n");
//    return leftTrim( rightTrim(s) );
//}

string trimNonRef(string s) {
    print("DEBUG: Here is what I'm supposed to trim [" s "]\n");
    return leftTrim( rightTrim(s) );
}

//------------------------------------------------------------------------------
// Starting with DOORS 9.5, for some unknown reason DXL started passing strings
// to trim(string) as something else unless string variables were explicitly
// typecast in-line like str = trim(string str);
// String literals did not have this problem str = trim (" trim me  ");
// but who does this anyway?
// If I change the function to: string trim1(string s)
// the problem goes away.  But I don't care to typecast every call to trim
// or change the name in all the code so changing tim's parameter type
// to string& (reference to string) thwarts the issue.
//------------------------------------------------------------------------------
// Here is the hokey "string trim(string&)" used as alternative to get the
// string data (even though I want string (not a refererence to string))
//------------------------------------------------------------------------------
string trim(string& s) {
    return trimNonRef(s); // call renamed original trim function
}

string myStr;
myStr = "                   test \n \r \t ";
print("original myStr is ~" (myStr) "~" "\n");
myStr = trim(myStr);
print("trim(myStr) is %" myStr "%\n");

//------------------------------------------------------------------------------
// OUTPUT 1 (as coded above):
//------------------------------------------------------------------------------
// original myStr is ~                   test
//       ~
// DEBUG: Here is what I'm supposed to trim [                   test
//       ]
// trim(myStr) is %test%
//------------------------------------------------------------------------------

//------------------------------------------------------------------------------
// OUTPUT 2 (with trimNonRef renamed to trim and trim(string&) commented out:
//------------------------------------------------------------------------------
// original myStr is ~                   test
//       ~
// DEBUG: Here is what I'm supposed to trim [K  !Ce!Ce      ZK      K   ]
// trim(myStr) is %K    !Ce!%
//------------------------------------------------------------------------------

//------------------------------------------------------------------------------
// OUTPUT 3 (only with myStr typecast as (string) during call to trim function.
//              myStr = trim( string myStr); // cast myStr as (string)
//          This works but why do I have to typecast a string to a string?
//------------------------------------------------------------------------------
// original myStr is ~                   test
//       ~
// DEBUG: Here is what I'm supposed to trim [                   test
//       ]
// trim(myStr) is %test%
//------------------------------------------------------------------------------


 

Re: function string trim(string) no longer works with V9.5
Adamarla - Thu Oct 17 02:45:24 EDT 2013

I don't have 9.5 so I this is a guess.

I suspect there is a new "trim" function in 9.5 which has different parameter/return types. You are overloading the function, and the parser is picking the wrong one when you do not tell it the type.

Search through the 9.5 %DOORSHOME%/lib/ folder. Maybe it is there someplace.

-Adam

Re: function string trim(string) no longer works with V9.5
Fu Lin Yiu - Thu Oct 17 12:11:40 EDT 2013

Good guess, but there are no functions in the %DOORSHOME%/lib/ sub-tree with name "trim" regardless of parameter lists or return type.

Reading reveals there is no documented trim function recently introduced in the dxl reference manual.

NOTE:

• I am not including any example folders.
• CLUE: If there was a new "native" trim() function I believe I could just call it without include file.  Right? I can't.
• Do get the same results as I posted?

Re: function string trim(string) no longer works with V9.5
llandale - Thu Oct 17 14:08:57 EDT 2013

You say it works when you declare "string myStr" but doesn't work when you don't?  Yet another reason to turn autodeclare off.

Adamarla has my guess, where "function" doesn't mean "perm" but some written function in the default loaded include files.  Renaming the function to make it works strongly supports this guess.

It is also possible, I suppose, that auto-declared "string"s are not quite right in v9.5

-Louie

Re: function string trim(string) no longer works with V9.5
Fu Lin Yiu - Thu Oct 17 23:40:24 EDT 2013

Turning auto-declare off was on my checklist of things to do when there is a new release.

DXL Reference says put it here.  Still true?

********

In DXL there is a mechanism called auto-declare, which means that a user need not specify a type for a variable. For example, in the script:
i=5
print i
the interpreter declares a new variable and deduces from the assignment that its type is int.
Because DXL is case-sensitive, there is a potential hazard when relying on this mechanism to type variables. If you make a mistake when typing a variable name, the interpreter assumes that a new variable is being used, which creates errors that are hard to find.
This feature can be disabled by adding the line:
XFLAGS_ &=~AutoDeclare_
to the bottom of the file $DOORSHOME/lib/dxl/startup.dxl.

********

Anything is possible with this silly putty of a language, but it appears auto-declare is disabled but it scews up the string parameter nevertheless.

*** New simpler demo ***

// OUTPUT:

Original (set in $DOORSHOME/lib/dxl/startup.dxl) XFLAGS_ is 8
Hammered-in (set above for fun) XFLAGS_ is 8
original myStr1 is ~bad~
Hi k%    o%    Pe    Pe    R%    Pe    V%    
trim(myStr1) is %Back from trim [k%    o%    Pe    Pe    X%    Pe    i%    ]%
original myStr2 is ~boys~
Hi boys
trim(myStr2) is %Back from trim [boys]%
 

Re: function string trim(string) no longer works with V9.5
Adamarla - Fri Oct 18 01:34:21 EDT 2013

I always turn AutoDeclare off at the top of every script with:

XFLAGS_ &= ~AutoDeclare_

That way you don't need to modify the client install, and if a script has turned it on(!), you have it off again :~/

-Adam

Re: function string trim(string) no longer works with V9.5
Adamarla - Fri Oct 18 01:59:47 EDT 2013

Well, this works for me on 9.3.0.3 [I don't have 9.5]:

pragma runLim, 0

pragma encoding, "UTF-8"

XFLAGS_ &= ~AutoDeclare_

 

string rightTrim(string s) {

    if ( null(s) ) return "";

    int FIRST = 0;            // constant left edge

    int last = length(s) - 1; // initial right edge

    // Starting from the right edge, hop over white space (if any)

    // Just don't fall off the left edge.

    while ( (last > -1) && (isspace(s[last])) ) {

       last--;

    }

    return s[FIRST:last]; // return substring with (possibly) modified last

}

string leftTrim(string s) {

    if (null(s)) return "";

    int first = 0;            // initial left edge

    int LAST = length(s) - 1; // constant right edge

    // Starting from the left edge, hop over white space (if any)

    // Just don't fall off the right edge.

    while ( (isspace(s[first])) && (first < LAST) ) {

       first++;

    }

    return s[first:LAST]; // return substring with (possibly) modified first

}

string trim(string &s) {

    // stringsDeRef = s // Maybe 9.5 needs it explicitly dereferenced???

    // return leftTrim( rightTrim(sDeRef) );

    return leftTrim( rightTrim(s) );

}

 

string sOriginal = "                   test \n \r \t ";

string &sReference = sOriginal

 

string sTrimmed1 = trim(sOriginal);

string sTrimmed2 = trim(sReference);

 

print("sOriginal: [" sOriginal "]\n");

print("sTrimmed1: {" sTrimmed1 "}\n");

print("sTrimmed2: {" sTrimmed2 "}\n");

-Adam

Re: function string trim(string) no longer works with V9.5
Fu Lin Yiu - Fri Oct 18 03:48:21 EDT 2013

I appreciate your reply.

• Unless it is some coincidence I have yet to find, all code I posted worked fine in DOORS 8.2 and 9.3.  I only had a problem with 9.5 (or so it seems).

• re your comment "One that takes references will take non references too and auto pass the address." -- this seemed correct at first.  I had trouble when the argument to function "string trim (stream&)" was a function that returned a string.  As a silly example stick a call to this as trim(string&)'s argument.
  
  string addTabOnLeftAndNewLineOnRight(string str) {
      return "\t" str "\n";
  }
  
  string trim(string& s) {
      print "Hi " s "\n";
      return "Back from trim [" s "]";
  }
  
  string str3= "  text ";
  
  print("str3='" str3 "\n");
  print("trim(addTabOnLeftAndNewLineOnRight(str3))='" trim(addTabOnLeftAndNewLineOnRight(str3)) "'\n")
   

• I find when a function has a string& argument a string variable will be accepted.
• When arg is a literal like " trim me ", the func(string&) won't like it.
• When another function returns a string (not string&) the trim(string&) will not like it.

Anyone have V9.5 that can give the samples a test drive (your car insurance may not cover it).???

Re: function string trim(string) no longer works with V9.5
Adamarla - Fri Oct 18 04:00:10 EDT 2013

Ah, yes. That is true.

A function expecting a reference will not take a literal so you will need a non reference version.

But, why have a reference version anyway? String variables are already pointers to the string table, and strings are imutable anyway so when you modify it you get a new entry in the string table with a different pointer.

Unless you want to return via byRef parameter and not the normal return type, just use a plain byVal string parameter.

-Adam

Re: function string trim(string) no longer works with V9.5
Mathias Mamsch - Fri Oct 18 05:16:54 EDT 2013

Never include that statement in scripts! Realize that this statement is executed at "Parsing" time, but will become active only AFTER the current script has been executed. So if you first execute

XFLAGS_ |= AutoDeclare_

And then execute:

XFLAGS_ &= ~AutoDeclare_
a = 1

You will see that this will run the first time and only on the second time yield an error! It is also clear, that you should never turn off autodeclare on a production system, because it might affect other scripts that made use of autodeclare (which you did not do).

So turning off autodeclare is something that should be done only locally on the machine that you develop with. So incorporate it in your start link with -C if you do not want to modify your installation.

Regards, Mathias

Re: function string trim(string) no longer works with V9.5
Adamarla - Fri Oct 18 05:25:25 EDT 2013

Fair enough, I had never checked that.

I have it turned off and expect it never to be turned on, but surely having it in scripts does no harm?

Even if it has no effect on the current execution of the script, it is not screwing other stuff up right?

-Adam

Re: function string trim(string) no longer works with V9.5
Mathias Mamsch - Fri Oct 18 06:28:34 EDT 2013

It will screw up EVERY dxl script running in the current client, that does not respect the "declare all variables" development guideline, until you turn it on again (it applies to all DXL: DXL attributes, layouts, built in, foreign scripts, ...)  But if you do :

XFLAGS_ &= ~AutoDeclare_

// ... your script ...
a = 10

XFLAGS_ |= AutoDeclare_

You will not have any effect at all, because the autodeclare statements cancel themselfes at runtime and have NO effect on the current script, so its basically the same as

a = 10
XFLAGS_ |= AutoDeclare_

When you run it in an autodeclare enabled environment it will work, if you run it in an non autodeclared enabled environment the script will not even execute and the autodeclare statements will also not be executed. So as a correction of my prior statement:

The autodeclare statement will be executed at runtime, but the current autodeclare setting will be checked at parse time (before execution), so that is why it has no effect on the current script.

Regards, Mathias

Re: function string trim(string) no longer works with V9.5
Adamarla - Fri Oct 18 07:27:17 EDT 2013

Yeah, I get that, but I see it as fine because I want all scripts to fail if they rely on AutoDeclare.

They would fail if you just turn it on once in StartUp anyway.

[Sorry Fu Lin Yiu for going off topic]

Re: function string trim(string) no longer works with V9.5
GothSloth - Fri Oct 18 08:21:23 EDT 2013

string test = "test string  "

trim( test )

print "-->" test "<--"
 

I ran the above code on 9.5 and it prints the following:

-->test string<--

Running the above code on 9.3 results in the following:

-E- DXL: <Line:3> incorrectly concatenated tokens
-E- DXL: <Line:3> undeclared variable (trim)
-I- DXL: All done. Errors reported: 2. Warnings reported: 0.

So it indeed looks like 9.5 has declared trim as a new function somewhere. I ran a strings on the 9.5 doors.exe and I did not find any occurrence of the word "trim".

Re: function string trim(string) no longer works with V9.5
Fu Lin Yiu - Fri Oct 18 11:48:01 EDT 2013

re:

"But, why have a reference version anyway? String variables are already pointers to the string table, and strings are imutable anyway so when you modify it you get a new entry in the string table with a different pointer."

I completely agree.  The only purpose of making a function signature of "string trim(string&)" was a last ditch kludge to avoid having to rename trim() to something else in all my code.  This would cover arguments that are string variables.  Arguments that are in-line calls to functions that return a string still would not work. 

This is a mess.  That is why I posted this issue.  I want to get to the root of the problem and not add a bunch of kludge code to cover up the new undocumented behavior.

Is there a new "string trim(string)" function in DXL V9.5?  It is such a common need that it should have been in DXL all along.  If so, I could comment out my trim code and go native. 

Is there any documentation indicating:

• Is it a new perm?
• What are the usage/signature variations
• Is it here to stay (or exposed by accident and will be removed soon)?

Re: function string trim(string) no longer works with V9.5
llandale - Fri Oct 18 12:26:33 EDT 2013

I agree that all scripts <SHOULD> be written with AutoDeclare off; but many scripts <ARE NOT>.  So if you routinely turn off AutoDeclare you will have difficulty running these other folks' sloppy scripts; which will be particularly relevant if they are attr-DXL, Layouts, or Triggers.

But you can have your cake and eat it too.  See attached.

-Louie

I wrote it in the steriotypically style of "Louie", rediculously verbose!

Re: function string trim(string) no longer works with V9.5
llandale - Fri Oct 18 12:33:14 EDT 2013

OK, so now I see you are casting myStr2 in the call to trim but not for myStr1.  Thus, this is not an AutoDeclare issue.

• myStr2 = trim(string myStr2)

I see no error in v9306, but that is what you said.

I don't know.  

[1] Try staging results in a new variable, myStr3:

• myStr3 = trim(myStr1)

[2] Try separating the declare from the assignment:

• string myStr1
• myStr1 = "bad"

-Louie

Re: function string trim(string) no longer works with V9.5
Fu Lin Yiu - Fri Oct 18 13:37:54 EDT 2013

PARTIAL CONCLUSION: GothSloth is on to something.  My further experiments reveal that there is a new (undocumented) perm out there. Apparently it has signature "void trim(string&)".  Given this, the undocumented new DXL perm "void trim(string&)" will only work with variables because it takes a string reference.  It won't work if you pass a function that returns a string.  For example myStr = trim(upper(someVar)) won't work.  Looks like we have an Unidentified Flying Perm (UFP) in V9.5

// OUTPUT:

Original (set in $DOORSHOME/lib/dxl/startup.dxl) XFLAGS_ is 8
Hammered-in (set above for fun) XFLAGS_ is 8
Let's see if we find an address for signature void trim(string&)
Address(base10) of void trim(string&) is [SIGNATURE[void function(string&)]: 71918848
Looks like we have an Unidentified Flying Perm (UFP) in V9.5
-->test string<--
Let's see if we find an address for signature string trim(string)
Address(base10) of string trim(string) is [SIGNATURE[string function(string)]: 167695256
The above is our local trim address
TEST1: var myStr1 is ~ from a variable  ~
Hi  from a variable  
TEST1: trim(string myStr1) is %Back from trim [ from a variable  ]%
TEST2: var myStr1 is ~ from a variable  ~
Hi  FROM A VARIABLE  
TEST2: trim(upper(myStr1)) is %Back from trim [ FROM A VARIABLE  ]%
TEST3: var myStr1 is ~ from a variable  ~
Hi l        
TEST3 FAIL: trim(myStr1) is %Back from trim [l        ]%
TEST3 FAIL: myStr1 is --> from a variable  <--
 

Re: function string trim(string) no longer works with V9.5
GothSloth - Fri Oct 18 14:50:22 EDT 2013

The trim() function is not defined in the DOORS executable. It doesn't show up by running strings on the DOORS exe. I'm a little stumped about where it is coming from. I added the following code to <doors-root>/lib/dxl/init.dxl:

string testString = "test string   \n"      // added at the top of the file

trim( testString ); print "-->" testString "<--\n"  //kept moving this line down the file until no error occured

The encrypted DXL file standard/itemProperties/properties.dxl causes the function to become defined. I ran a DXL trace by calling startDXLTracing_() before the include of properties.dxl and calling stopDXLTracing() after the include. The word "trim" does not show up in the DXL trace. I've attached the trace in case anyone want's to look at it.

Re: function string trim(string) no longer works with V9.5
Fu Lin Yiu - Fri Oct 18 16:04:33 EDT 2013

GothSloth -- Thanks for looking into this. 

<IBM_Rant>

It looks like the DOORS DXL trim function is in IBM's Area 51.

Why all the secrecy for a simple trim function that exists in almost any programming language (or library)?

I really wouldn't care except the introduction of this "top secret" feature is forcing me to rewrite code that has been working fine for years.

Hey IBM, how about saying what it is and how it works?  Is that too much to ask?  If trim() is a proprietary secret, how about hiding it better so it does not break existing user's code?  Add a few underscores to it at least.

</IBM_Rant>

Re: function string trim(string) no longer works with V9.5
Mike.Scharnow - Sat Oct 19 04:58:16 EDT 2013

GothSloth & Fu Lin Yu: Which DOORS version do you exactly work with? I have DOORS 9.5.0.0 installed without any patches, and when I start a clean DOORS without any include directory parameters and registry settings, I cannot reproduce your problem.

Re: function string trim(string) no longer works with V9.5
Fu Lin Yiu - Sat Oct 19 12:10:45 EDT 2013

Thanks giving this matter a spin Mike.Scharnow.

My client is:

Client version:     9.5.1.2
Home directory:     C:\Program Files\IBM\Rational\DOORS\9.5\
Build number:     95088
Build date:     Tuesday, August 13, 2013 10:09:47 PM

Results follow this simplified test code:

Auto-Declare setting (set in $DOORSHOME/lib/dxl/startup.dxl) XFLAGS_ is 8
Let's see if we find an address for signature void trim(string&)
If an address is reported, we have an Unidentified Flying Perm (UFP)
Address(base10) of void trim(string&) is [SIGNATURE[void function(string&)]: 71918848
-->test string<--
 

Re: function string trim(string) no longer works with V9.5
GothSloth - Mon Oct 21 13:58:50 EDT 2013

I too am using 9.5.1.2 build 95088.

Re: function string trim(string) no longer works with V9.5
GothSloth - Mon Oct 21 18:28:02 EDT 2013

For kicks and giggles I wanted to see what was getting passed to the function "string trim(string sOrig)". I have this function interpret sOrig as a string and print the results. I also interpret sOrig as a pointer to a string and print the results. Here is the code:

void printHexNibble( int nibble )
{
   print charOf( nibble + (nibble < 10 ? intOf('0') : intOf('A') - 10 ) )
}

void printHexByte( int byte )
{
   printHexNibble( ( byte & 0xF0 ) / 16 )
   printHexNibble( byte & 0x0F )
}

void printHexWord( int word )
{
      printHexByte( (word / 0x1000000 ) & 0xFF )
      printHexByte( (word / 0x0010000 ) & 0xFF )
      printHexByte( (word / 0x0000100 ) & 0xFF )
      printHexByte( (word             ) & 0xFF )
}

int peek32( int * address, int offset )
{
   address += offset
   return (*address) | 0
}

int peek8( int * address, int offset )
{
   address += offset
   return *address & 255
}

void printMemCharByte( int * address, int size )
{
   int i

   for( i = 0; i < size; i++ )
   {
      print charOf( peek8( address, i ) )
   }
}

string trim(string sOrig)
{
   //get address sOrig contains
   int *ptr    = addr_( sOrig )
   print "ptr: "; printHexWord( (int addr_( ptr ) ) ); print "\n"

   // get value located at the memory address specified by sOrig
   // as a 32bit pointer
   int *str = addr_( peek32( ptr, 0 ) )
   print "str ( *ptr ): "; printHexWord( (int addr_( str ) ) ); print "\n"

   // print memory located at the memory address specified by sOrig
   // as a string.  (c parlance: print *ptr)
   print "*ptr-->"; printMemCharByte( ptr, 16 ); print "<--\n"

   // print memory location specified by the address at the memory
   // location specified by sOrig. (c parlance: print **ptr)
   print "*str (**ptr)-->"; printMemCharByte( str, 16 ); print "<--\n"

   return sOrig
}



string testString = "test string    "
string resultString

resultString = trim( testString )




The results:

ptr: 0AD22960
str ( *ptr ): 0ACFD32B
*ptr-->+ÓÏ
!Ce‡!Ce‡!Ce‡<--
*str (**ptr)-->test string    <--



So it appears that the local trim() function is being passed a pointer to a string instead of a string. The DXL parser is confused. Typecasting the parameter to a string seems to give the parser enough of a hint so that it passes the parameter correctly.

Re: function string trim(string) no longer works with V9.5
llandale - Tue Oct 22 11:58:53 EDT 2013

Can you translate all this VooDoo into something practical we can actually do to resolve it?

Off the top of my head, I would be tempted to try this:

• string trim(string Orig){blablabla} // the real funtion
• string trim_Temp(string Orig){  return trim(Orig)}
• string tim(string &Orig){return(trim_Temp(Orig)}

-Louie

"VooDoo" because I really don't understand all this pointer/addr/reference stuff.   lol

Re: function string trim(string) no longer works with V9.5
Adamarla - Wed Oct 23 02:31:26 EDT 2013

Well, the simplest wat to resolve it is to rename the custom function to something other than trim.

Of course that is no good if you have baselined code...

-Adam

Re: function string trim(string) no longer works with V9.5
GothSloth - Wed Oct 23 09:05:56 EDT 2013

Louie,

  I agree with Adam. That would be a good approach. Type casting the parameters in calls to the locally defined trim function seem to work also. It looks like we have to modify our code to resolve this. I've opened a PMR requesting this function to be renamed trim_()  to follow the convention used by other IBM DXL scripts. I'll let you know how it goes.

Re: function string trim(string) no longer works with V9.5
GothSloth - Thu Nov 21 15:59:30 EST 2013

I am told this is now PM99549 and that it is due to be resolved in version 9.5.2.1.

Re: function string trim(string) no longer works with V9.5
Fu Lin Yiu - Tue Aug 01 22:07:46 EDT 2017

Recently we upgraded to DOORS 9.6.1.8.  I found that this http://www-01.ibm.com/support/docview.wss?uid=swg1PM99549 must have been implemented.

Now it is safe to revert to user-defined trim functions with signature:

string trim(string); // The typical signature for most programming languages

In a test I found the undocumented perm

void trim(&string);

was replaced with undocumented perm

void trim_(&string);