DXL Objects in 64 bit Environment

Has anybody checked Dxl objects on a 64 bit client, yet?

I'm trying to store callback addresses but following code fails and leads to a memory violation without further information in the DXL interaction window.

// Store 32 bit memory address in DXL Object
DxlObject x = new()

Addr64_ a = 2147483647 // max. int 32 bit
print a "" "\n"        // displays 2147483647

x->"address64" = a

The same applies for higher integer numbers. Also the following code crashes DOORS:

// Store real memory address in DXL object
DxlObject x = new

x->"address64" = addr_ x

But following code snippet works perfectly:

// Store 64 bit memory addresses in skip list
DxlObject x = new()

Skip sl = createString
Addr64_ a = null

print a "" "\n"     // prints 0
put(sl, "empty", a)
a = addr_ x
print a "" "\n"     // prints memory address of x
put(sl, "myDxlObject", addr_ x)

for a in sl do
{
    print (string key sl)
    print ": "
    if (a==null) print "N/A\n"
    else
    {
        print a "" "\n"
    }
}

It seems that skip lists can store the 64 bit type Addr64_ or return values of function addr_ whereas DxlObject can't. This is strange as I remember a post in this forum that explained DxlObjects to be internally handled as skip lists. So what would be the preferred way in 64 bit environments to store the address of callback functions for later reuse by a function proxy?

All tests have been performed with DOORS client version 9.6.1.8.

Re: DXL Objects in 64 bit Environment
Mike.Scharnow - Fri Feb 08 06:24:50 EST 2019

You can store the string representation in the DXLObject. We use the following code, tested on 9.6.1.11, inspired by https://www.ibm.com/developerworks/community/forums/html/topic?id=d7274c76-77e4-4932-ab29-829cb132dd4e. This is no explanation why DXL Objects don't do what they should do, so may be someone should submit a bug report with IBM.

string literal (Addr64_ val) {
    int iVal = (addr_ val) int
    if (iVal == ~0x7FFFFFFFFFFFFFFF) return "(~0x7FFFFFFFFFFFFFFF)"
    // if it's not the problem value, then use literal
    Addr64_ ad = addr_ val
    return "(" ad ")"
} 

void restore_value ( string sIn, _y &data ) {
    // Assignment directly during initialization. No clipping, no warning, adData gets the correct 64bit value
    Addr64_ adData = addr_ data;
    
    string sCode = "
    Addr64_ ad = addr_ " sIn "
    Addr64_ &ref = addr_ " literal (adData) "
    ref = ad
    "

    eval_ sCode    
}



// Store 32 bit memory address in DXL Object
DxlObject x = new()

Addr64_ a = 2147483647 // max. int 32 bit
print a "" "\n"        // displays 2147483647

x->"address64" = literal(a)



Addr64_ b 
restore_value (x->"address64", b)
print b "" "\n"        // displays 2147483647

Re: DXL Objects in 64 bit Environment
Mathias Mamsch - Fri Feb 08 11:33:21 EST 2019

There is nothing wrong with Skip lists or DXL objects. It seems to be rather a problem of the Addr64_ type.

// Store 32 bit memory address in DXL Object
DxlObject x = new()

// Addr64_ a;
int a = ~1 // Make a 64 bit int 0xFFFFFFFF(-1) 0xFFFFFFFE(-2)

// verify with a pointer that we really have 64 bits stored
int *ptr = &a;
ptr += 4

print "HIGH DWORD:" (*ptr) "\n"  // displays -1
print "Low Byte: " a "" "\n"     // displays -2
print (a == -2) "\n"

x->"address64" = a int; // store to skip

a = 0;
print "HIGH DWORD AFTER CLEAR: " (*ptr) "\n"; // verify high word has been cleared

a = (x->"address64") int
print "HIGH DWORD AFTER RESTORE: " (*ptr) "\n"// verify high word has been restored correctly

The same works also with Addr64_ types:

// Store 32 bit memory address in DXL Object
DxlObject x = new()

Addr64_ a = 0xBADC0FFEE  // ==> 50159747054. Note that assignments only work in declaration!
print a // 50159747054
// x->"address" = a;  // Crash
x->"address" = (addr_ a) int // Works

Addr64_ b = (x->"address") Addr64_
print b "" // 50159747054

So this is not a problem with the Skip/DxlObject rather than a problem with the Addr64_ type / with the DXL grammar. Regards, Mathias

Re: DXL Objects in 64 bit Environment
rdratlos - Fri Feb 08 17:45:55 EST 2019

// Store 32 bit memory address in DXL Object
DxlObject x = new()

// Addr64_ a;
int a = ~1 // Make a 64 bit int 0xFFFFFFFF(-1) 0xFFFFFFFE(-2)

// verify with a pointer that we really have 64 bits stored
int *ptr = &a;
ptr += 4

print "HIGH DWORD:" (*ptr) "\n"  // displays -1
print "Low Byte: " a "" "\n"     // displays -2
print (a == -2) "\n"

x->"address64" = a int; // store to skip

a = 0;
print "HIGH DWORD AFTER CLEAR: " (*ptr) "\n"; // verify high word has been cleared

a = (x->"address64") int
print "HIGH DWORD AFTER RESTORE: " (*ptr) "\n"// verify high word has been restored correctly

// Store 32 bit memory address in DXL Object
DxlObject x = new()

Addr64_ a = 0xBADC0FFEE  // ==> 50159747054. Note that assignments only work in declaration!
print a // 50159747054
// x->"address" = a;  // Crash
x->"address" = (addr_ a) int // Works

Addr64_ b = (x->"address") Addr64_
print b "" // 50159747054

What should we rely on?

// Next snippet
DxlObject dx = new()

Addr64_ a = -1  // prints -1 but internally stored as 64 bit int
print a "" "\n"

a = addr_ dx
print a "" "\n"  // prints memory address of dx

Addr64_ b = addr_ dx
print b "" "\n"  // prints memory address of dx

a = 0xBADC0FFEF
print "Address dx? " a "" "\n  // prints 50159747055

a = 50159747056
print a "" "\n"  // prints 50159747056

a = null
print a "" "\n"  // prints 0

Line 13 and 16 trigger warning "Assigning 64 bit value to 32 bit int". Such warnings caused us to investigate our callback storage code.

And when we try to assign (-1) in line 16 we get error: incorrect use of identifier (a).

On the other hand Mathias' example shows us that if we cast a 64 bit address to the originally 32 bit int type we store a 64 bit value.

// Test casts
int a = ~1 // Make a 64 bit int 0xFFFFFFFF(-1) 0xFFFFFFFE(-2)

// verify with a pointer that we really have 64 bits stored
int *ptr = &a;
ptr += 4

//print "HIGH DWORD:" (*ptr) "\n"  // displays -1
//print "Low Byte: " a "" "\n"     // displays -2
//print (a == -2) "\n"

Addr64_ b = 0x100000000  // 4294967296
a = (addr_ b) int

print "HIGH DWORD:" (*ptr) "\n"  // displays 1
print "Low Byte: " a "" "\n"     // displays 0
print a  "\n"                    // displays 0

Addr64_ c = a
print b "" " - " (b == c) "\n"   // displays 4294967296 - true

Again lines 12, 13 and 18 cause the Assigning 64 bit value to 32 bit int warning.

Are these warnings also displayed in version 9.6.1.11?

Re: DXL Objects in 64 bit Environment
Mike.Scharnow - Sat Feb 09 04:46:00 EST 2019

// Next snippet
DxlObject dx = new()

Addr64_ a = -1  // prints -1 but internally stored as 64 bit int
print a "" "\n"

a = addr_ dx
print a "" "\n"  // prints memory address of dx

Addr64_ b = addr_ dx
print b "" "\n"  // prints memory address of dx

a = 0xBADC0FFEF
print "Address dx? " a "" "\n  // prints 50159747055

a = 50159747056
print a "" "\n"  // prints 50159747056

a = null
print a "" "\n"  // prints 0

// Test casts
int a = ~1 // Make a 64 bit int 0xFFFFFFFF(-1) 0xFFFFFFFE(-2)

// verify with a pointer that we really have 64 bits stored
int *ptr = &a;
ptr += 4

//print "HIGH DWORD:" (*ptr) "\n"  // displays -1
//print "Low Byte: " a "" "\n"     // displays -2
//print (a == -2) "\n"

Addr64_ b = 0x100000000  // 4294967296
a = (addr_ b) int

print "HIGH DWORD:" (*ptr) "\n"  // displays 1
print "Low Byte: " a "" "\n"     // displays 0
print a  "\n"                    // displays 0

Addr64_ c = a
print b "" " - " (b == c) "\n"   // displays 4294967296 - true

Don't know if this is feasible for you, but IBM made several, ehm, let's call them "bug fixes", especially for handling the Addr64_ data type, in 9.6.1.9, 9.6.1.10 and  9.6.1.11. So, if possible, you should consider migrating to the newest DOORS version, which will work a bit more consistently.

In 9.6.1.11, your "Next snippet" produces the output

-1
2090793266160
2090793266160
Address dx? 50159747055
50159747056
0

without any warning.

a = -1
print a "" "\n"  // prints -1

Your "test casts" produces

-R-W- DXL: <Line:6> Argument 1 (32 bit int) was passed with a 64 bit value
-R-W- DXL: <Line:13> Assigning 64 bit value to 32 bit int
HIGH DWORD:1
Low Byte: 0
0
4294967296 - true

The warning in line 13 is clear, as an int is expected to be 32 bit (even though it *could* hold the value in reality). IBM tries to detect more and more "loopholes" where developers rely on internal data structures and try to prevent such "deeds"...

If you are stuck to 9.6.1.8, you will have to tweak your code until it works for this version and expect to have the same effort when migrating to the next version.

Edit: added "a = -1" and result

Re: DXL Objects in 64 bit Environment
Mathias Mamsch - Sat Feb 09 06:35:15 EST 2019

// Next snippet
DxlObject dx = new()

Addr64_ a = -1  // prints -1 but internally stored as 64 bit int
print a "" "\n"

a = addr_ dx
print a "" "\n"  // prints memory address of dx

Addr64_ b = addr_ dx
print b "" "\n"  // prints memory address of dx

a = 0xBADC0FFEF
print "Address dx? " a "" "\n  // prints 50159747055

a = 50159747056
print a "" "\n"  // prints 50159747056

a = null
print a "" "\n"  // prints 0

// Test casts
int a = ~1 // Make a 64 bit int 0xFFFFFFFF(-1) 0xFFFFFFFE(-2)

// verify with a pointer that we really have 64 bits stored
int *ptr = &a;
ptr += 4

//print "HIGH DWORD:" (*ptr) "\n"  // displays -1
//print "Low Byte: " a "" "\n"     // displays -2
//print (a == -2) "\n"

Addr64_ b = 0x100000000  // 4294967296
a = (addr_ b) int

print "HIGH DWORD:" (*ptr) "\n"  // displays 1
print "Low Byte: " a "" "\n"     // displays 0
print a  "\n"                    // displays 0

Addr64_ c = a
print b "" " - " (b == c) "\n"   // displays 4294967296 - true

Ok, lets sort this out! There are two or three things two notice:

• Integer Assignment and Math Operators is not suitable for 64 bit values anymore and produces warnings!
• Everything else (Type Casting, other types) works fine!
• 64 bit ints are deceiving (since print does clip)! They still exist, but you should not use them

So the following code should be perfectly valid and produce no warnings on any version:

Skip sk = addr_ 0xBADC0FFEE
DxlObject dx = new();
dx->"address" = (addr_ sk) int; // We are doing type casting, not int assignment

So there is not problem to put 64 addresses into a skip list or DXL objects.

To pass addresses to eval_ code, you can use Addr64_! You should note that the Addr64_ assignment warnings that you got only occur if you assign a 64 bit integer (which is artificially produced) to an Addr64_ directly via the  ::=(Addr64_, int) perm:

Skip sk = addr_ 0xBADC0FFEE

Addr64_ a = 0xBADC0FFEE    // Produces warning! --> Addr64_ ::= (Addr64_&, int)
Addr64_ b = (addr_ sk) int // Produces warning! --> Addr64_ ::= (Addr64_&, int)
Addr64_ c = addr_ sk       // No warning!       --> _k ::= (_k&, _k)

print "Address of sk is: " c "\n"

If you want to convert a string literal address inside an eval_ back to normal, just embed it in the string:

Skip sk = addr_ 0xBADC0FFEE
Addr64_ ad = addr_ sk

eval_ "
    Skip sk = addr_ " ad "
    Addr64_ ad = addr_ sk
    print ad
"

So actually it is pretty easy:

• Do not use int variables for storing 64 bit addresses
• Use addr64 for making a string literal for a 64 bit address, but use direct addr_ assignment, do not type cast!
• To convert string literals back to addresses in eval use a simple addr_

Hope that sorts things out! Regards, Mathias

by the way I tested on DOORS 9.6.1.7.

Re: DXL Objects in 64 bit Environment
Mike.Scharnow - Sat Feb 09 09:33:53 EST 2019

Skip sk = addr_ 0xBADC0FFEE
DxlObject dx = new();
dx->"address" = (addr_ sk) int; // We are doing type casting, not int assignment

Skip sk = addr_ 0xBADC0FFEE

Addr64_ a = 0xBADC0FFEE    // Produces warning! --> Addr64_ ::= (Addr64_&, int)
Addr64_ b = (addr_ sk) int // Produces warning! --> Addr64_ ::= (Addr64_&, int)
Addr64_ c = addr_ sk       // No warning!       --> _k ::= (_k&, _k)

print "Address of sk is: " c "\n"

Skip sk = addr_ 0xBADC0FFEE
Addr64_ ad = addr_ sk

eval_ "
    Skip sk = addr_ " ad "
    Addr64_ ad = addr_ sk
    print ad
"

You're right. Seems to work for callback function as well.

void bla (int a) {
   print a
}

Addr64_ ad = addr_ bla
print ad "\n"

DxlObject d = new()

d->"address" = (addr_ ad) int

Addr64_ ad2 = d->"address"
print ad2 "\n"

void callit (void callme(int), int param) {
   callme(param)
}


callit (addr_ ad2 , 99)

2095213434608
2095213434608
99

But I admit that I still have some doubts about the future-proofness of 

dx->"address" = (addr_ sk) int; // works
dx->"address" = (addr_ sk) Addr64_; // produces DOORS crash

I wonder whether the second line will probably work in a future version of DOORS and the first line will cease to work, as I think that the first line is not really "correct" in the sense that the address of something should be transported in an Addr64_ variable, not in an int variable, or did I not understand this correctly? But anyway, at least we have code that works for some of the next DOORS versions...

Re: DXL Objects in 64 bit Environment
Mathias Mamsch - Sat Feb 09 11:09:33 EST 2019

void bla (int a) {
   print a
}

Addr64_ ad = addr_ bla
print ad "\n"

DxlObject d = new()

d->"address" = (addr_ ad) int

Addr64_ ad2 = d->"address"
print ad2 "\n"

void callit (void callme(int), int param) {
   callme(param)
}


callit (addr_ ad2 , 99)

2095213434608
2095213434608
99

dx->"address" = (addr_ sk) int; // works
dx->"address" = (addr_ sk) Addr64_; // produces DOORS crash

There is no "int variable" ... The type casting just helps the interpreter to choose the right ::= operator ...I am not sure what exactly happens in both cases, but I am pretty sure that for some reason in the Addr64_ variant the wrong ::= operator is called. ... One can try to verify by progPrint:

DxlObject dx = new()
Addr64_ ad = addr_ 0xBADC0FFEE
if (false) dx->"address" = (addr_ ad) Addr64_
// if (false) dx->"address" = (addr_ ad) int

progPrint_

If you comment in the int version you get:

....
1.i59(BoolQuest, 1.i64, 1.i69, <>, 1.i93)
1.i64( PushVarC, false, -1,  43)^
1.i68(      Nop)
1.i68(      Nop)

1.i69(setFieldValue, 1.i73, 1.i86, <>)
1.i73(getFieldLHS, 1.i77, 1.i81, <>)

1.i77( PushVarC, dx, 0,  2)^
....

If you comment in the Add64_ version:

1.i59(BoolQuest, 1.i64, 1.i69, <>, 1.i96)
1.i64( PushVarC, false, -1,  43)^
1.i68(      Nop)
1.i68(      Nop)

1.i69( Int32Ass, 1.i73, 1.i86, 1.i93)
1.i73(getFieldValue, 1.i77, 1.i81, <>)

1.i77( PushVarC, dx, 0,  2)^

Here you can see, that in Addr64_ case the Interpreter calls "Int32Ass"  while in the other case it seems to call correctly the setFieldValue perm (which seems to be the correct one for DxlObject left Hand assignments). If you are unsure about the Syntax feel free to use the following form:

DxlObject dx = new()
Addr64_ a = addr_ 0xBADC0FFEE
put((addr_ dx) Skip, "address", a)  // Treat as Skip

Addr64_ b = dx->"address"; 
print b

After all DxlObjects and Skip objects are still the same (with disregard of memory allocations, DxlObjects seemed not automatically to be freed when the DXL ends anymore). Regards, Mathias