@fxm, detecting fixed size arrays vs. dynamic arrays

[Juergen Kuehlwein]

Hi fxm,

i refer to this post

Maybe a cookie (var/fix length array flag) could be stored just before the descriptor structure:

- a "hidden" Integer at address = @Cptr(Integer Ptr, @descriptor)[-1]



This is better than my proposal."Old" code (e.g. a dll) would definitely get into trouble with my "new" descriptor, even if the size didn´t change, because it would still interpret the bitmask as a value and therefore read wrong values.

Taking a closer look at the RTL i can see that the descriptor isn´t saved to disk when saving an array, it´s only the array´s data. So the only remaining case i can think of when "new" code is confronted with an "old" descriptor, would be a call (an "old" dll calls an exported sub/function of a "new" executable, passing an array) from "old" code (do you agree, or did i miss something?). This could be a problem, because how can we be sure, that the integer before the descriptor is valid memory? And if it is valid, how can we decide, if it´s our hidden flag or something else (random)?

The other way round ("old" code receiving a "new" descriptor) it should work quite well, because the "old" code gets exactly, what it expects. It doesn´t know and it doesn´t care about the hidden flag.

your idea was to prepend the additional data, but what about appending it? E.g. just append space for one more dimension and fill in some key word (like "ArrayExt" in order to know it´s an extended descriptor) and our additional data. This can be coded quite easily for the the compiler, it´s just a few lines to change/add. According to my tests it works. Even the compiler still compiles itself with this modified descriptor. Everything seems to work still.

I need someone as experienced as you for further testing (of course everyone else wanting to help is welcome too!).

Especially the old code problem (see above) needs to be tested extensively. As i see it, old code shouldn´t have problems with the new descriptor, because it doesn´t even know about it. And as long as it´s written to valid memory (which the "new" compiler takes care about) nothing should happen. An old descriptor doen´t have the additional dimension, so such an array would be seen as a dynamic one (which is what we have now anyway). The only problem i still see, is that the memory, where we look for an additional dimension, might not be valid. This is very unlikely (if it is appended, prepending would be more problematic, at least i can tell this for Windows due to the way Windows memory mangement works), but still possible. If the memory is valid, we can check for our key word and thus can decide between an old and a new descriptor.

In Windows i can test, if memory is valid or not with SEH, is there a similar way in the `NIX world?. Or (even better) is there a corresponding C runtime function?

If someone wants to help, i can supply a new fbc.exe for testing or compiler code changes necessary for this, in case you want to compile the compiler yourself.


JK

[fxm]

The advantage of my solution is that this new information is always in the same place relative to the descriptor pointer (at "address - 1").
I propose the same principle than for the followings in presence of a data block of variable length:
- NEW[]: with the number of elements to destroy at "address - 1" compared to the allocated memory seen from the user.
- The "VTBL" (table of virtual member procedures): with the address of the RTTI block information at "address -1" compared the first virtual member procedure address referenced by the "VPTR" (virtual pointer).

But I think you should not start seriously without the advice of coderJeff (or dkl or counting_pine).

[Juergen Kuehlwein]

fxm,

The advantage of my solution is that this new information is always in the same place relative to the descriptor pointer (at "address - 1").

- granted, in my case it needs a simple multiplication to get the offest.

I think the difference between the examples you gave and the current situation with arrays is, that in these cases it was designed to be as it is from the start. With arrays we already have a different approach, the size of the dimension table is a multiple of the valid dimensions. Adding one more for our additional information is quite easy to code, i fear prepending this information will need more changes in more places in the compiler´s code.

Anyway, in principle your proposal is as good as mine. In theory it should work with new and old code (e.g dll compiled with the old descriptor), but only tests will show. The necessary changes in the RTL are not yet implemented, it´s almost the same in both cases and absolutely doable. So the question is, do the already applied changes break existing code and are there problems with the combination of new and old code and binaries?

Before bothering coderJeff i wanted to be sure, that it actually works - not only in what i tested. You seem to have done a lot with arrays, therefore i was asking, if you were willing to help me test.


JK

[Juergen Kuehlwein]

I ran some tests with combinations of old and new code and binaries (e.g. a .dll created with V1.05 and an .exe created with the changed compiler version and vice versa). So far i couldn´t find any problems.

This a proposal, a demonstration of how it could be done without breaking compatibility between older versions and this one. Furthermore it makes fixing the problems with ERASE and REDIM possible. If Jeff decides to do it some other way - fine. The important thing is that there will be a solution for it at all.


JK