Freebasic 1.20.0 Development

[fxm]

Assignment of an old/new fix-len string through a function by a 'Byref' parameter or a 'Byref return

The behavior of this type of assignment seems even more (last test) degraded for the 'Byref' return case:

Code: Select all

Function test1(Byref s As String) Byref As String
    s = "1"
    Return s
End Function

Print "Byref As String:"
Print

Dim As String s1
test1(s1)
Print "'" & s1 & "'"
(test1(s1)) = "12"
Print "'" & s1 & "'"
Print

Dim As String * 2 s2
test1(s2)
Print "'" & s2 & "'"
(test1(s2)) = "12"
Print "'" & s2 & "'"
Print

Function test2(Byref s As Zstring) Byref As Zstring
    s = "1"
    Return s
End Function

Print "Byref As Zstring:"
Print

Dim As Zstring * 3 s3
test2(s3)
Print "'" & s3 & "'"
(test2(s3)) = "12"
Print "'" & s3 & "'"
Print

Dim As String * 2 s4
test2(s4)
Print "'" & s4 & "'"
(test2(s4)) = "12"
Print "'" & s4 & "'"
Print

Sleep

With fbc version 1.10.0:

Byref As String:

'1'
'12'

'1'
'1'

Byref As Zstring:

'1'
'12'

'1'
'12'

With new fbc version 1.20.0:

Byref As String:

'1'
'12'

'1 '
'1 '

Byref As Zstring:

'1'
'12'

'1 '
'1 '

Can this behavior be improved ?

[coderJeff]

Assignment of an old/new fix-len string through a function by a 'Byref' parameter or a 'Byref return
...
Can this behavior be improved ?

Good question.

'byref zstring' and 'zstring ptr' are handled nearly the same under the hood.

For string*n passed to byref zstring, should probably just disallow it. If your test given is converted to use all pointers, then at least we get a 'passing different pointer types'

[coderJeff]

I have a simple suggestion: wouldn't be possible to return the address of a fixed length string not as a string pointer (it wouldn't work, since it would not contain the terminator, nor the string descriptor), but as an Unsigned Byte pointer?

After all, a fixed length string with no zero at the end is perfectly equivalent to an array of unsigned bytes

By dereferencing such a pointer, you would achieve a single byte, and you'd avoid the pollution issue. It would be up to the programmer to use pointer math to access other characters.

I think worth investigating because I'm not sure at the moment; the compiler sometimes uses constructions internally involving`*(@(expression))` where it's probably expected that it doesn't immediately convert types.

Another approach would be to at least warn (or even error) about things that certainly don't work, alerting the user that some cast() or other handling is needed for the given scenario.

[fxm]

Initialization/Assignment of an old/new fix-len string through a string literal expression

The behavior seems to have improved:

Code: Select all

Dim As String * 20 s1 = "Alpha" + Chr(0) + "Beta"
Print "'" & s1 & "'"

Dim As String * 20 s2
s2 = "Alpha" + Chr(0) + "Beta"
Print "'" & s2 & "'"

Sleep

With fbc version 1.10.0:

Code: Select all

'Alpha'
'AlphaBeta'

With new fbc version 1.20.0:

Code: Select all

'Alpha Beta          '
'Alpha               '

• New fix-len string:
  - For initialization, the supplied string literal expression is taken into account in a raw way, character by character.
  - For assignment, the supplied string literal expression is first evaluated as a zstring, before application.

[angros47]

I think worth investigating because I'm not sure at the moment; the compiler sometimes uses constructions internally involving`*(@(expression))` where it's probably expected that it doesn't immediately convert types.

Another approach would be to at least warn (or even error) about things that certainly don't work, alerting the user that some cast() or other handling is needed for the given scenario.

That gives me an idea, but I am not sure if it is even possible: having pointers for fixed length strings, but pointers are considered of different types depending of the string length.

Something like:

Code: Select all

Dim s as string*10
Dim p as string*10 ptr
dim p2 as string*5 ptr

s="Test string"

p=@s
print *p  'Should work and print "Test string"

p2=@s  'this should produce a warning during compiling
print *p2 'Should print "Test "

All string functions need to know how long is the string: for regular FreeBasic string that information is stored in the descriptor, so they can work as long as a pointer to the descriptor is passed. For Zstring, the string ends at the first null character, so the function can figure it from the string itself. For fixed length string, such information is missing, so there is no way to acquire it at runtime. During compile time the compiler knows it, so it can pass the string length to the function, but only if it knows that the variable is a fixed length string, and how long it is. Maybe such a behavior can be applied to pointers, too?
It would also make possible stuff like:

Code: Select all

sub Procedure (t as string*4)
     print t
end sub

Procedure "abcdefghil"  'should print "abcd"

(not sure if it would be useful to anyone, but would make sense)

[coderJeff]

That gives me an idea, but I am not sure if it is even possible: having pointers for fixed length strings, but pointers are considered of different types depending of the string length.

Indeed, this is what I would want. The challenge is designing it to be done efficiently within the compiler. I am always concerned about adding logic that slows down the whole compiler.

For interest, some information about fbc internals and type information:

Type information for most data types including multi level pointer and const qualifier variants is fully encoded in one 32-bit field. This allows fbc to efficiently store, pass, and manipulate type information within the compiler for most data types. This includes: any, boolean, enum, [unsigned] byte|short|long|integer|longint, zstring|wstring, single, double, var-len string, and va_list.

With UDT's (types and unions) there is more type information needed than would ever fit in one 32-bit field. So, we must always also know the name of the TYPE and the compiler is designed accordingly to look up additional information whenever we work with UDT data types.

To have a length for fixed length strings (which could potentially include zstring|wstring) we have more information to store than could be encoded in the existing 32-bit datatype field, however, "fixed-length string*N" is also not necessarily a named type. So currently lacking is 1) where to store the length so can retrieve later (i.e. some anonymous type naming scheme), and 2) all the logic to check and act on the stored length

For Zstring, the string ends at the first null character, so the function can figure it from the string itself.

nitpick: only if the data pointed to is well-formed / initialized.

[Lost Zergling]

@Jeff
There is indeed a complexity to the idea. The problem that the developer will be confronted with, in particular, is the reliability of the length of the strings being shorter than defined: this requires filtering the blanks on the right before each assignment, or assuming the fact that the information on the original length will be systematically lost in this case, which will induce non-linear operation from the user's point of view.
Instead of padding with spaces, dynamic use of 0.1 or two Chr(0) could address the issue.
- If string*N is instantiated over N characters (wich is preeminent use case), no Chr(0) would be required.

Following the idea, the user would therefore have the possibility of favoring either full use of memory by instantiating the corresponding length, or implicitly the speed of zstrings compatibility (by declaring for example string*N+1), depending on his need/algorithmic problem.

There would therefore always be a non-linearity, but in this case in the performance and not in the behavior.
On the other hand, when using zstring pointers, it would be non-linear because it would be necessary to check whether the last character located at +*N is a chr(0) or not, but this is not a priori too problematic if the size is fixed and known in advance.
I understand that this is easier said than done and I truly apologize for this silly remark, anyway, for example (and from experience), string*N or related solution using blank paddings would not work algorithmically neutrally on justified data sources (can indeed be a feature).

Added :
I understand main question is : can string*N returns less than N characters or shall it return always N characters ?
One hand string*N is seen as a "memory slot facilities", versus other hand a "new datatype".
I must admit answer is difficult.

Added :
I just realised user maybe can set String*N adding a chr(0) or not while setting, in wich case he/she assume the zstring compliancy or not depending on his/her requirements (in wich case zstring direct access possibility could be a feature).

[fxm]

@Jeff,

limit WSTRING*N, ZSTRING*N, STRING*N types to 2^31-1 and show an 'invalid size' error if the size given is less than 1 or greater than the limit

Could you confirm the theoretical limits for a var-len string:
between 0, and 2^31-1 (for 32-bit) or 2^63-1 (for 64-bit) ?

[fxm]

Change : STRING*N occupies N bytes and has no terminating null character

At first glance, list of pages that will need to be updated:

• KeyPgType
  LangQB
  KeyPgString
  KeyPgOpStrptr
  ProPgStringsTypes
  KeyPgSizeof
  KeyPgSwap
  KeyPgDim
  TblVarTypes

• +
  ProPgInitialization
  KeyPgUnion
  KeyPgByref

I think I will start updating the documentation tomorrow.

It is very likely that I miss documentation pages (see list above) that need to be updated.
Let me know it please.

[fxm]

@Jeff,

- internal: types containing only STRING*N fields or child types containing only STRING*N fields will initialize to spaces using a memory fill operation, and where other data types are present will induce the creation of an implicit constructor

Maybe simpler to always create at least one implicit constructor whenever a STRING*N field exists ?
(this is the only case of data type where one must fill the byte field with 32 and not 0 like all others)

[fxm]

BUG #1 with new STRING*N
(which I had difficulty identifying in a large code which no longer works)

Assigning a STRING * N variable, by an expression which is a concatenation including at least one string variable (whatever its type), does not work.

Example:

Code: Select all

Dim As String * 10 s
Dim As String s0 = "test"

s = "1 " & "test" & " 2"
Print "'" & s & "'"
s = "1 " & s0 & " 2"
Print "'" & s & "'"
s = s0 & " 3"
Print "'" & s & "'"

Sleep

Code: Select all

'1 test 2  '
'1         '
'test      '

Same problem with operators '=' (equality), '>', '<', '<>'.

[fxm]

BUG #2 with new STRING*N
(which I had difficulty identifying in a large code which no longer works)

The data of a var-len array of STRING*N are not initialized to space, but always to 0.
(OK for fix-len array of STRING*N)

Example:

Code: Select all

Dim As String * 1 s1(0)
Print s1(0)[0]

Redim As String * 1 s2(0)
Print s2(0)[0]

Dim As String * 1 s3()
Redim s3(0)
Print s3(0)[0]

Sleep

Code: Select all

32
0
0

[PeterHu]

Sorry for stepping in--

When try to compile below examples,the recent fbc64.exe version 1.2.0 crashed during compiling under windows 10 64bit,but fbc64 version 1.1 is fine.
https://github.com/jepalza/FB_DOOM_LIKE

I have no idea what happed,so just report here.

By the way,I am really wondering,wehere the language has decided that it will NOT add semantic macro system or even generics(template stuffs)?

Thanks and best regards.

[SARG]

When try to compile below examples,the recent fbc64.exe version 1.2.0 crashed during compiling under windows 10 64bit,but fbc64 version 1.1 is fine.

Are you sure of the working version ? with Version 1.10.1 (2023-12-24), I get the crash.

The issue seems to come with these 2 files #include "textures/T_NUMBERS.bi" and #include "textures/T_VIEW2D.bi"
here 00 is ouput but not here01 and not here02 when first include is commented

Code: Select all

'textures
#print "here 00"
#include "textures/T_NUMBERS.bi"
#print "here 01"
#include "textures/T_VIEW2D.bi"
#print "here 02"

They contain only a very long line :

Code: Select all

Dim Shared As Byte T_NUMBERS(...)  = { _
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, _
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, _
...............

Edit : also with Version 1.10.0 (2023-05-14),

[fxm]

Compatibility between old/new STRING*N (apart from remaining bugs)

When upgrading existing codes (to also compile and run with fbc version 1.20.0), the largest number of modifications is when printing, assigning, or comparing strings to each other.
To ignore the padding spaces added at the end of any STRING*N, one must apply RTRIM() on each STRING*N variable in the expression.

It is a big job !
I started doing this on a 5000 line code that uses a lot of STRING*N arrays.
One really have to go through the code line by line (that is how I found the two bugs above), but everything is not yet operational !