Beware of using negative hex literals?

DXL will limit all literal constants outside the range 0x00000000 to 0x7FFFFFFF.  It will make them into 0x7FFFFFFF.

However (and here's the bad part), if you ever give it a hex literal to process that lies outside the expected range, then it appears that strange "random" side-effect bugs can start to occur.  My suspicion is that something gets corrupted somewhere deep within the bowels of DXL, possibly related to (or affecting) either buffer, string or skip processing. 

You can however make a negative hex literal by using something in the range -0x00000000 to -0x7FFFFFFF.  It will not however cope with positive or negative 0x80000000.  This too it limits to 0x7FFFFFFF.  And it too can then have after-effects.

Bottom line?  Avoid hex literals unless they are definitely in the range  0x00000000 to 0x7FFFFFFF.

P.S.  If you read this thread through, you will find that I am corrected about the "bug" part.   Good - learned something!   :-)

Re: Beware of using negtaive hex literals
Mathias Mamsch - Sat Nov 22 14:09:46 EST 2014

This behaviour has actually been discussed in the forum already. The behaviour is not buggy or random. Since DXL uses 32 Bit signed integers, DXL will clip all literals that are lower then -MAX_INT  to  -MAX_INT and literals that are larger than MAX_INT to MAX_INT (MAX_INT being 2147483647 or 0x7FFFFFFF, the largest positive signed integer value)... While this behaviour makes some sense, it can of course lead to confusion and seemingly random behaviour.

Note that this behaviour only occurs on "parsing" that means, when DXL is executed the normal overflow rules apply, i.e. MAX_INT + 1 = MIN_INT

Of course (who would have guessed?) there is also a bug on integer parsing. The smallest signed integer number (MIN_INT) is actually -214783648. However DXL can calculate this number, but it cannot parse it, since it will parse the literal  214783648 (which will be clipped to 214783647) and then negate it (-214783647). 

print 0x7FFFFFFF        // prints  214783647    =   MAX_INT

print (-2147483648)     // prints -2147483647   = - MAX_INT !!

print (-2147483647 - 1) // prints -2147483648   =   MIN_INT

Regards, Mathias

Re: Beware of using negtaive hex literals
AlexTidmarsh - Sat Nov 22 14:21:25 EST 2014

No - I didn't mean the clipping itself is a bug - once I realised what it was doing, that was fine.

The buggy behaviour I was referring to was that after executing a statement like:

     int x = 0xFFFFFFFF

     print x

other totally unrelated code elsewhere (that was using Buffers, Skips and strings) started to behave strangely.   It stopped the strange behaviour when I used any non-negative literal in the above statement, but started again whenever I tried any negative literal.   Like I said, the other (affected) code was TOTALLY unrelated:  It didn't use 'x'. 

P.S.  Of course I can't entirely rule out something else going on that I am not aware of.  But the sudden behavioural anomaly that occurred for me was that a string I was printing (in a long list of strings) was turned to garbage.   I certainly couldn't see any dependence.  Anyways the code behaves normally even when I remove the above statements entirely!  Incidentally, the above statements were also inserted well BEFORE the declarations and usage of the code affected.  This is why I think something gets "damaged" when the interpreter hits an out-of-range hex literal.

Re: Beware of using negtaive hex literals
Mathias Mamsch - Sat Nov 22 17:49:41 EST 2014

I am pretty sure, that there is no relationship between negative hex literals and any data corruption. The data corruption you are mentioning usually occurs due to incorrect tempStringOf usage or returning local function data on the stack from a function, or returning stuff from an eval context, or incorrectly using buffers after freing them, etc. If you can show however a piece of code, that will show this kind of behavior and have one of us reproduce it, this will be an entirely different issue.

But I would not sign a petition to avoid negative hex literals unless I see such a piece of code myself... ;-) In the end this makes no sense. As I already said - hex literals are always parsed as "positive" and then negated in a separate step (as the min_int, max_int problem above shows). Therefore not only hex literals, but ANY negative number should trigger the same behaviour, which is not experienced.

Regards, Mathias

Re: Beware of using negtaive hex literals
AlexTidmarsh - Sat Nov 22 17:55:48 EST 2014

hmmm.  You may be on to something there:  the corrupted code does use tempStringOf...

I shall review the code that appeared to get affected - just in case I'm "abusing" a tempStringOf somewhere.

Re: Beware of using negtaive hex literals
AlexTidmarsh - Sat Nov 22 18:00:48 EST 2014

print 0x7FFFFFFF        // prints  214783647    =   MAX_INT

print (-2147483648)     // prints -2147483647   = - MAX_INT !!

print (-2147483647 - 1) // prints -2147483648   =   MIN_INT

My experience was, as you say, that INTEGER literals are indeed clipped both +ve and -ve.

However, HEX literals appear to be clipped differently - a negative HEX literal gets clipped to +MAX_INT as far as I can tell.

   print  0xFFFFFFFF  // prints 2147483647

It was THIS clipping (of what should be a NEGATIVE HEX literal) that appeared to cause strange behaviour in later code.

Of course, one can use:

     print  (-0x00000001)  // prints -1

which is how I worked around the problem - at which point the later (unrelated) strange behaviour stopped too.

Re: Beware of using negtaive hex literals
AlexTidmarsh - Sat Nov 22 18:26:58 EST 2014

Huh.

I can no-longer recreate the strange behaviour!

It would appear that I am working too many hours in a row!

So ok - there does not appear to be an knock-on bug (anymore?).   However, what I said about entering 0xFFFFFFFF stands:

you have to enter it as (-0x00000001).

Re: Beware of using negtaive hex literals
AlexTidmarsh - Sat Nov 22 18:44:40 EST 2014

Aha.   It happened again (it was random!  Every 2nd or 3rd run, and then it stopped happening)

However, I appear to have stopped its effect by changing the following lines in the affected routine:

// b is a Buffer ...

   string s=tempStringOf(b)

   delete b

   return s}

Which I changed to:

   string s=stringOf(b)

   delete b

   return s}

Although, to be honest, I can't quite see what the problem should be.  But it stops when I make that change.

Re: Beware of using negtaive hex literals
llandale - Sun Nov 23 16:33:44 EST 2014

You can only use tempStringOf(buf) when you immediately dispose of the results; such as setting an attr-value or writing to a file or setting a dialog. 

As the name suggests, it does not put the string in the string table but instead addresses the Buffer dirrectly.  When you delete the buffer its memory can get used up and no surprise your string will get corrupted.

• Buffer buf = create()
• buf = "ABC"
• string s1 = tempStringOf(buf)
• print "s1 = [" s1 "]\n"
• set(buf, 1, 'b')   // change the Buffer
• print "s1 = [" s1 "]\n"    // the string changes also
• buf += "D"
• print "s1 = [" s1 "]\n"

Result:

• s1 = [ABC]
• s1 = [AbC]
• s1 = [AbCD]

-Louie

Re: Beware of using negtaive hex literals
AlexTidmarsh - Mon Nov 24 02:08:08 EST 2014

Yeh, I knew that as such - but the brain didn't really compute the extent of the volatility. I originally thought (somewhat lazily) that once I had assigned it to the string variable 's', and returned 's' it would have been put into the string table at that very point.  But that's because of my 'Delphi' mind set!  But of course, as you rightly point out, Louie:

     string s = tempStringOf (b)    // s points into the buffer, not the string table

     return s   // Which returns the still volatile 's' and stores it in the variable addressed by the function result pointer

Which means that the string eventually loaded with that same 's'  is also still volatile, - unless you perhaps do this:

     string s = (tempStringOf (b)) ""    // The concatenation creates a string table entry.

which then means that you may as well use:

     string s = stringOf (b)    // s points into string table

Missing this (now obvious) behaviour is something that is bound to happen when constantly switching between Delphi programming (reference-counted string management) and DXL (string under-management). 

:-)