The Preprocessor and tokens

[Karl Heuer]

In article <15606 at smoke.brl.mil> gwyn at smoke.brl.mil (Doug Gwyn) writes:
>Many alternative proposals were offered, including several from members of
>the public during the public review process, but ALL of them were
>technically flawed.

It may be that D. Hugh Redelmeier's proposal (X3J11/88-083, letter P24,
item 33/36) had some technical flaw, but if so, it wasn't the flaw that the
Committee asserted in their response.  (Hugh's proposal did not misparse
"1e4+6", nor any other valid construct as far as I can see.)

In case anyone's interested, here are the relevant patterns in egrep form.
X3J11:	[.]?[0-9]([0-9a-zA-Z_.]|[eE][-+])*
Karl:	[.]?[0-9][0-9.]*([eE][-+])?[0-9a-zA-Z_.]*
Hugh:	[.]?[0-9]([0-9.]|[eE][-+])*[0-9a-zA-Z_.]*

There's not much point in continuing to argue this, since the current
Standard is frozen.  On the bright side, we have:

[0] At least it's not a Quiet Change; any code that's broken will cause a
diagnostic rather than getting the wrong answer.

[1] A helpful compiler can always issue the diagnostic, and then guess what
you meant and do the right thing anyway.

[2] Those of us who normally put whitespace around binary operators won't be
hit by it anyway.

[3] If the C-2001 Committee decides it was a bad idea, they can still fix it,
since no working code can depend on this glitch.

[4] If, at the C-2001 Standardization Committee meeting, somebody puts LSD
into the drinking water and they decide to allow hexadecimal floating-point
constants, it'll be relatively easy to add.

[5] The experience may be helpful in designing a new language from scratch.
Using "1.0E2" and "1.0E_2" is perhaps a better idea than allowing "+" and "-"
to appear in the middle of a token.

Karl W. Z. Heuer (karl at ima.isc.com or uunet!ima!karl), The Walking Lint