badly formed token (EOF)

For those of us that have been around for the earlier versions of DOORS, perpaps there is someone out there that knows the answer to this. I have a custom DXL menu item that works just find in DOORS 8.x+ but when I use the same menu item in DOORS 7.1 I get this error in the DXL window and needless to say it doesn't work:

-E- DXL: <Line:1> badly formed token (EOF)
-I- DXL: all done with 1 error and 0 warnings

Strangely enough if I use the Tools > Edit DXL... and load the script that sits on my computer (where the addins are pointing to) and click Run it then works. Has anyone seen this? What can I do to get it to work from the menu?
DOORSWizard - Thu Mar 22 13:37:58 EDT 2012

Re: badly formed token (EOF)
llandale - Thu Mar 22 15:08:21 EDT 2012

I guess encoding; try loading it into the DXL window and saving it, then trying it from the menu.

Re: badly formed token (EOF)
DOORSWizard - Thu Mar 22 16:00:06 EDT 2012

llandale - Thu Mar 22 15:08:21 EDT 2012
I guess encoding; try loading it into the DXL window and saving it, then trying it from the menu.

Louie, your suggestion worked like a charm. Thank you!

Re: badly formed token (EOF)
llandale - Thu Mar 22 16:03:07 EDT 2012

DOORSWizard - Thu Mar 22 16:00:06 EDT 2012
Louie, your suggestion worked like a charm. Thank you!

Lets hope the folks that understand encoding jump in here and explain it.

Re: badly formed token (EOF)
SystemAdmin - Thu Mar 22 20:04:48 EDT 2012

llandale - Thu Mar 22 16:03:07 EDT 2012
Lets hope the folks that understand encoding jump in here and explain it.

DOORS 8.0 introduced features that recognised the locale in which it is installed in e.g date formats, language, font sets and so on. In order to do this, data needed to be encoded in the Unicode format.

Unicode effectively defines a unique number for every character. So no matter what type of PC your using or what program is viewing the characters, and no matter what the local language is, it should be universally recognised - well, that's the theory anyway. I guess the DXL interpreter in version 8.x required the DXL to be in unicode format and was able to do the conversion on legacy DXL, but any DXL created in version 8 would be stored using the unicode format which would not be backward compatible.

Older versions of DOORS suffered from the problem where it would conflict with the default locale encoding format on the PC, or an application like MSWord - so a bullet symbol in DOORS may map to a smiley face or something else when exported and viewed in another application - the worse case I ever experienced of this was a raised degrees symbol for temperature (not a superscript lwr case letter "o", but an actual degrees symbol) - which when exported to MSWord via the now defunct DocExpress tool it was mapped to a capital letter O, so 30 degrees centigrade looked like 300 degrees centigrade. I avoided using specific symbols after that.
Paul Miller
Melbourne, Australia

Re: badly formed token (EOF)
llandale - Fri Mar 23 10:33:56 EDT 2012

SystemAdmin - Thu Mar 22 20:04:48 EDT 2012
DOORS 8.0 introduced features that recognised the locale in which it is installed in e.g date formats, language, font sets and so on. In order to do this, data needed to be encoded in the Unicode format.

Unicode effectively defines a unique number for every character. So no matter what type of PC your using or what program is viewing the characters, and no matter what the local language is, it should be universally recognised - well, that's the theory anyway. I guess the DXL interpreter in version 8.x required the DXL to be in unicode format and was able to do the conversion on legacy DXL, but any DXL created in version 8 would be stored using the unicode format which would not be backward compatible.

Older versions of DOORS suffered from the problem where it would conflict with the default locale encoding format on the PC, or an application like MSWord - so a bullet symbol in DOORS may map to a smiley face or something else when exported and viewed in another application - the worse case I ever experienced of this was a raised degrees symbol for temperature (not a superscript lwr case letter "o", but an actual degrees symbol) - which when exported to MSWord via the now defunct DocExpress tool it was mapped to a capital letter O, so 30 degrees centigrade looked like 300 degrees centigrade. I avoided using specific symbols after that.

Paul Miller
Melbourne, Australia

Never ran into this issue, all my code works in all versions of DOORS. I edit with Crimson Editor.

Do I infer correctly? Files saved with the DXL editor in v8+ won't work for v7-; so its best to use an editor to save the DXL files?

Perhaps there is some other practical advise?

-Louie

Re: badly formed token (EOF)
SystemAdmin - Fri Mar 23 21:25:51 EDT 2012

llandale - Fri Mar 23 10:33:56 EDT 2012
Never ran into this issue, all my code works in all versions of DOORS. I edit with Crimson Editor.

Do I infer correctly? Files saved with the DXL editor in v8+ won't work for v7-; so its best to use an editor to save the DXL files?

Perhaps there is some other practical advise?

-Louie

As I no longer have version 7.1 installed, I can't test if DXL saved in version 8 and beyond will work - sounds like you have and it does work - kind of makes sense if it does because text editors would typically only need to support the single byte wide ASCII encoding format and would have no real need for the multi byte Unicode encoding formats which accommodate thousands of symbols. It was really an assumption on my part based on the original reported problem as the support for Unicode was a key difference between version 7.1 & 8.0 and the Unicode concept is not as simple as it might sound.

Paul Miller
Melbourne, Australia