Another error of a code generation in the "fblite" dialect

[monochromator]

I have written earlier about one error in code generator of FreeBASIC compiler in this forum topic: http://www.freebasic.net/forum/viewtopi ... =4&t=21009
Error was corrected by dkl, for which I am extremely grateful to him.
But now I'm faced with another error, very similar to the previous one.
Only it is associated not with the statement SELECT, but with statement FOR.
As an example, I offer the same program as before. Its new version can
download at this link: http://zalil.ru/34418586
Way to compile and run it is described in the previously mentioned forum thread.
This bug has no striking manifestations such as exceptions, but only leads to a calculation error.
The error occurs in the subroutine TraceLiner, which contains some nested FOR loops.

In lines 449-455 there are following loops:

FOR StartY% = UseMinY% TO UseMaxY% STEP (UseMaxY% - UseMinY%) \ 3
FOR StartX% = UseMinX% TO UseMaxX% STEP (UseMaxX% - UseMinX%) \ 3
.................
.................
.................
NEXT StartX%, StartY%

I shall call this loop as "external loop".

In his body in line 454 the subroutine call is located

GOSUB TraceEQV

In this subroutine in lines 491-498 also there is a loop:

FOR TestY% = ScreenY% - 1 TO ScreenY% + 1
FOR TestX% = ScreenX% - 1 TO ScreenX% + 1
................
................
................
NEXT TestX%, TestY%

I shall call this loop as "internal loop".

Code generator of the compiler emits an incorrect code for these loops.
The error lies in the fact that for both loops (for both external and
internal) the same temporary variables are used for storage of limit
values of the counters.

Here is the disassembly of relevant source lines.
First, the external loop:

449 FOR StartY% = UseMinY% TO UseMaxY% STEP (UseMaxY% - UseMinY%) \ 3
0x00003775 <+613>: mov ebx,DWORD PTR [ebp-0x4]
0x00003778 <+616>: mov DWORD PTR [ebp-0x44],ebx
0x0000377b <+619>: mov ebx,DWORD PTR [ebp-0x8]
0x0000377e <+622>: mov DWORD PTR [ebp-0x9c],ebx ;Used variable at [ebp-0x9c]
0x00003784 <+628>: mov ebx,DWORD PTR [ebp-0x8]
0x00003787 <+631>: sub ebx,DWORD PTR [ebp-0x4]
0x0000378a <+634>: mov ecx,0x3
0x0000378f <+639>: mov eax,ebx
0x00003791 <+641>: cdq
0x00003792 <+642>: idiv ecx
0x00003794 <+644>: mov ebx,eax
0x00003796 <+646>: mov DWORD PTR [ebp-0xa0],ebx ;Used variable at [ebp-0xa0]
0x0000379c <+652>: mov ebx,DWORD PTR [ebp-0xa0]
0x000037a2 <+658>: test ebx,ebx
0x000037a4 <+660>: setge bl
0x000037a7 <+663>: shr ebx,1
0x000037a9 <+665>: sbb ebx,ebx
0x000037ab <+667>: mov DWORD PTR [ebp-0x48],ebx
0x000037ae <+670>: jmp 0x3870 <TRACELINER+864>

455 NEXT StartX%, StartY%
0x00003840 <+816>: mov eax,DWORD PTR [ebp-0xa8]
0x00003846 <+822>: add DWORD PTR [ebp-0x4c],eax
0x00003849 <+825>: cmp DWORD PTR [ebp-0x50],0x0
0x0000384d <+829>: jne 0x385c <TRACELINER+844>
0x0000384f <+831>: mov eax,DWORD PTR [ebp-0xa4]
0x00003855 <+837>: cmp DWORD PTR [ebp-0x4c],eax
0x00003858 <+840>: jge 0x37ee <TRACELINER+734>
0x0000385a <+842>: jmp 0x3867 <TRACELINER+855>
0x0000385c <+844>: mov eax,DWORD PTR [ebp-0xa4]
0x00003862 <+850>: cmp DWORD PTR [ebp-0x4c],eax
0x00003865 <+853>: jle 0x37ee <TRACELINER+734>
0x00003867 <+855>: mov eax,DWORD PTR [ebp-0xa0] ;Used variable at [ebp-0xa0]
0x0000386d <+861>: add DWORD PTR [ebp-0x44],eax
0x00003870 <+864>: cmp DWORD PTR [ebp-0x48],0x0
0x00003874 <+868>: jne 0x3887 <TRACELINER+887>
0x00003876 <+870>: mov eax,DWORD PTR [ebp-0x9c] ;Used variable at [ebp-0x9c]
0x0000387c <+876>: cmp DWORD PTR [ebp-0x44],eax
0x0000387f <+879>: jge 0x37b3 <TRACELINER+675>
0x00003885 <+885>: jmp 0x3896 <TRACELINER+902>
0x00003887 <+887>: mov eax,DWORD PTR [ebp-0x9c] ;Used variable at [ebp-0x9c]
0x0000388d <+893>: cmp DWORD PTR [ebp-0x44],eax
0x00003890 <+896>: jle 0x37b3 <TRACELINER+675>

Now, the internal loop:

491 FOR TestY% = ScreenY% - 1 TO ScreenY% + 1
0x00003add <+1485>: mov eax,DWORD PTR [ebp-0x88]
0x00003ae3 <+1491>: dec eax
0x00003ae4 <+1492>: mov DWORD PTR [ebp-0x8c],eax
0x00003aea <+1498>: mov eax,DWORD PTR [ebp-0x88]
0x00003af0 <+1504>: inc eax
0x00003af1 <+1505>: mov DWORD PTR [ebp-0x9c],eax ;Used variable at [ebp-0x9c]
0x00003af7 <+1511>: jmp 0x3b8a <TRACELINER+1658>

492 FOR TestX% = ScreenX% - 1 TO ScreenX% + 1
0x00003afc <+1516>: mov eax,DWORD PTR [ebp-0x84]
0x00003b02 <+1522>: dec eax
0x00003b03 <+1523>: mov DWORD PTR [ebp-0x90],eax
0x00003b09 <+1529>: mov eax,DWORD PTR [ebp-0x84]
0x00003b0f <+1535>: inc eax
0x00003b10 <+1536>: mov DWORD PTR [ebp-0xa0],eax ;Used variable at [ebp-0xa0]
0x00003b16 <+1542>: jmp 0x3b76 <TRACELINER+1638>

498 NEXT TestX%, TestY%
0x00003b70 <+1632>: inc DWORD PTR [ebp-0x90]
0x00003b76 <+1638>: mov ebx,DWORD PTR [ebp-0xa0] ;Used variable at [ebp-0xa0]
0x00003b7c <+1644>: cmp DWORD PTR [ebp-0x90],ebx
0x00003b82 <+1650>: jle 0x3b18 <TRACELINER+1544>
0x00003b84 <+1652>: inc DWORD PTR [ebp-0x8c]
0x00003b8a <+1658>: mov ebx,DWORD PTR [ebp-0x9c] ;Used variable at [ebp-0x9c]
0x00003b90 <+1664>: cmp DWORD PTR [ebp-0x8c],ebx
0x00003b96 <+1670>: jle 0x3afc <TRACELINER+1516>

As it is possible to see, in both loops the same variables with addresses [ebp-0x9c]
and [ebp-0xa0] are used.
Since this error is not met anyone before, it is possible to assume that
it is specifically associated with "fblite" dialect used in the program
and occurs due to the specific scoping rules of variables.
Again have to ask the developers to fix the bug.

[monochromator]

I apologize, I was completely wrong about the causes of the error.

Error is due to incorrect handling by the compiler of the loops within routines called by GOSUB.

This is simple case:

Code: Select all

'$LANG: "fblite"

OPTION GOSUB

UseMinX% = 0: UseMaxX% = 520: UseMinY% = 0: UseMaxY% = 240

FOR StartY% = UseMinY% TO UseMaxY% STEP (UseMaxY% - UseMinY%) \ 3
 FOR StartX% = UseMinX% TO UseMaxX% STEP (UseMaxX% - UseMinX%) \ 3
  PRINT "External:", StartX%, StartY%
  IF COMMAND$ = "1" THEN GOSUB TraceEQV2 ELSE GOSUB TraceEQV
NEXT StartX%, StartY%
END

TraceEQV:
FOR TestY% = StartY% - 1 TO StartY% + 1
 FOR TestX% = StartX% - 1 TO StartX% + 1
  PRINT "Internal:", TestX%, TestY%
NEXT TestX%, TestY%
RETURN

TraceEQV2:
PRINT "Without internal loop"
RETURN

Compare the output of the program when you call it with the "1" on the command line and without.
Pay attention to the lines that contain "External"

[counting_pine]

Looks like this is happening because the memory holding the temporary step/end counters is being clobbered, because it is assigned to different variables after the loop:

Code: Select all

'' #lang "fblite" ' not needed

dim as integer i, j, iend = 10, jend = 20

for i = 1 to iend
    print i
    goto goaway
    comeback:
next i
assert(i = iend+1)


goto fin

goaway:
for j = 1 to jend: next j
goto comeback

fin:

[dkl]

The For block uses two implicit scopes, so in #lang fb at least those will capture the temp var created to hold the end value, of course then vars from parallel scopes will use the same memory, and goto-jumping between scopes is a problem.

In #lang fblite it worked fine in 0.24, because symbAddTempVar() unscoped the temp var in that case, but my recent change to it to use symbAddImplicitVar() instead of symbAddTempVar() broke that too. At least know I understand why temp vars were being unscoped in the first place.

I'm not sure yet how to fix it, except that the #lang fblite regression should be fixed, while for #lang fb it will probably be impossible. 0.24 in #lang fb showed a "branch crossing" warning, which is gone in Git currently, that at least should be restored to help detect such goto issues in #lang fb. Besides that Select Case and With blocks can potentially have the same issues (it's always those three...).

[monochromator]

The situation is much worse than originally thought.
Bug is not only in the "fblite" dialect.
Here is the same example for dialect "fb".
Only instead of GOSUB GOTO here is used.

Code: Select all

DIM AS INTEGER UseMinX = 0, UseMaxX = 520, UseMinY = 0, UseMaxY = 240
DIM AS INTEGER StartX, StartY, TestX, TestY

FOR StartY = UseMinY TO UseMaxY STEP (UseMaxY - UseMinY) \ 3
 FOR StartX = UseMinX TO UseMaxX STEP (UseMaxX - UseMinX) \ 3
  PRINT "External:", StartX, StartY
  IF COMMAND$ = "1" THEN GOTO TraceEQV1 ELSE GOTO TraceEQV2
TraceEQVEx:
NEXT StartX, StartY
END

TraceEQV1:
FOR TestY = StartY - 1 TO StartY + 1
 FOR TestX = StartX - 1 TO StartX + 1
  PRINT "Internal:", TestX, TestY
NEXT TestX, TestY
GOTO TraceEQVEx

TraceEQV2:
PRINT "Without internal loop"
GOTO TraceEQVEx

Again compare the output of the program when you call it with the "1" on the command line and without.
Pay attention to the lines that contain "External"

Thus, the error is not related to a particular dialect and scoping rules, but with the presence of unconditional jump statements themselves (such as GOTO and GOSUB).

Because of their availability FOR loop can be called from any place and on any level of nesting.

Reliably track it at compile time, in my opinion, is not possible.
Therefore, there exists only one solution - for every such structure (for structures FOR and WITH) have a special separate temporary variables set that will be used only for this particular structure and nowhere more.

[dodicat]

I suppose you could bludgeon your way through this with asm call.
Whether it is perfectly safe or not is another question.

Code: Select all

DIM AS INTEGER UseMinX = 0, UseMaxX = 520, UseMinY = 0, UseMaxY = 240
DIM AS INTEGER StartX, StartY, TestX, TestY

FOR StartY = UseMinY TO UseMaxY STEP (UseMaxY - UseMinY) \ 3
FOR StartX = UseMinX TO UseMaxX STEP (UseMaxX - UseMinX) \ 3
  PRINT "External:", StartX, StartY
  IF COMMAND$ = "1" THEN GOTO TraceEQV1 ELSE GOTO TraceEQV2
asm TraceEQVEx:
NEXT StartX, StartY
sleep
END

TraceEQV1:
FOR TestY = StartY - 1 TO StartY + 1
FOR TestX = StartX - 1 TO StartX + 1
  PRINT "Internal:", TestX, TestY
NEXT TestX, TestY
asm call TraceEQVEx
'GOTO TraceEQVEx

TraceEQV2:
PRINT "Without internal loop"
'GOTO TraceEQVEx
asm call TraceEQVEx

 

[monochromator]

This will not change anything
The problem is not in the way of the call, but that the loops use the same temporary variables, and thus the internal loop disrupts the external loop.

[TJF]

Here is the same example for dialect "fb".
Only instead of GOSUB GOTO here is used.

When I compile this code I get the compiler output (fbc-0.24 on Ubuntu32-12.04)

fbc -exx -w all "FOR2.bas" (im Verzeichnis: ...)
FOR2.bas(17) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_000D
FOR2.bas(17) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_000E
FOR2.bas(17) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_0006
FOR2.bas(17) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_0007
FOR2.bas(21) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_000D
FOR2.bas(21) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_000E
FOR2.bas(21) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_0006
FOR2.bas(21) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_0007
Kompilierung erfolgreich beendet.

These warnings should be sufficiant to tell you that the code is in bad style.

Why don't you use code like the following?

Code: Select all

#MACRO TraceEQV1()
  FOR TestY = StartY - 1 TO StartY + 1
    FOR TestX = StartX - 1 TO StartX + 1
      PRINT "Internal:", TestX, TestY
    NEXT
  NEXT
#ENDMACRO

#MACRO TraceEQV2()
  PRINT "Without internal loop"
#ENDMACRO

DIM AS INTEGER UseMinX = 0, UseMaxX = 520, UseMinY = 0, UseMaxY = 240
DIM AS INTEGER StartX, StartY, TestX, TestY

FOR StartY = UseMinY TO UseMaxY STEP (UseMaxY - UseMinY) \ 3
  FOR StartX = UseMinX TO UseMaxX STEP (UseMaxX - UseMinX) \ 3
    PRINT "External:", StartX, StartY

    SELECT CASE COMMAND$
    CASE "1"  : TraceEQV1()
    CASE ELSE : TraceEQV2()
    END SELECT
  NEXT
NEXT

[fxm]

When I compile this code I get the compiler output (fbc-0.24 on Ubuntu32-12.04)

fbc -exx -w all "FOR2.bas" (im Verzeichnis: ...)
FOR2.bas(17) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_000D
FOR2.bas(17) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_000E
FOR2.bas(17) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_0006
FOR2.bas(17) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_0007
FOR2.bas(21) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_000D
FOR2.bas(21) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_000E
FOR2.bas(21) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_0006
FOR2.bas(21) warning 14(1): Branch crossing local variable definition, to label: TRACEEQVEX, variable: LT_0007
Kompilierung erfolgreich beendet.

Same warnings with fbc-0.24 on Windows, but no warnings with fbc-0.25 (last build of MOD), yet with the same behavior!
Why?

[dodicat]

FB24 here.
That's why I forced a goto with asm call, I had a stack of compile errors.
I didn't realize that 25 was out.

There is an an amazing choice of compilers now, all running concurrently.
The trick is getting the right one.

[fxm]

I just wanted to point out that with FBC-025, the warnings disappear but the behavior of the program is exactly the same as with FBC-024.

[counting_pine]

I think dkl's post from yesterday says pretty much everything there is to be said about this bug.
Until it's fixed, I think all we can do is to suggest using While loops as a workaround.

[dkl]

The issue in -lang qb/fblite modes should be fixed now; for -lang fb I have restored the "branch crossing" warning. Overall this matches how DIM and other temporary variables work: in -lang qb/fblite, every variable is moved to procedure-level if it was declared in a nested block; while -lang fb has scopes and does not move anything (the commit message has some more detail).