Fixed Length String handling

General FreeBASIC programming questions.
Munair
Posts: 834
Joined: Oct 19, 2017 15:00
Location: 't Zand, NL
Contact:

Fixed Length String handling

Postby Munair » Nov 23, 2018 12:30

Because the other thread (viewtopic.php?f=3&t=27169) went totally off topic, this one is specifically about fixed length string handling. Consider the following code that could be part of a comm or binary stream library to convert numbers in string format (back) to numbers:

Code: Select all

type TT1
   s as string * 8
end type

type TT2
   v as longint
end type

function vs(v as longint) as string
   dim t1 as TT1
   dim t2 as TT2
   t2.v = v
   lset t1, t2
   return t1.s
end function

function sv(s as string) as longint
   dim t1 as TT1
   dim t2 as TT2
   t1.s = s
   lset t2, t1
   return t2.v
end function

dim v as longint = 3044630875795881984
   print v
dim s as string = vs(v)
   v = sv(s)
   print v

end
The conversion from numeric to string and back won't take place with all numbers because FB shortcuts the fixed length string as soon as it finds a null character. So it won't copy the rest of the string to the (variable length) string even though there is valid numerical data. This means that some values will and others won't be converted back:

Code: Select all

dim v as longint
v = 3044630875795881984 ' won't convert back because the first three bytes are null characters
v = 3044630875795881983 ' converts back because the first bytes have value 255
v = 3044630875795881985 ' won't convert back because the first byte with value 1 is followed by a null character

The only way to get around this, as we have seen in the other thread, is to use a string function when converting to variable length string:

Code: Select all

function vs(v as longint) as string
   dim t1 as TT1
   dim t2 as TT2
   t2.v = v
   lset t1, t2
   return left(t1.s, 8) ' return all bytes because they represent a number
end function

Here the function LEFT() is used, but TRIM, LTRIM, RTRIM and even MID() and RIGHT() would do as well. Apparently these functions consider the entire string regardless of content, effectively returning the number of bytes specified to the variable length string.

Even though the FBWiki warns for unexpected behaviour with fixed-length strings, the question is whether resorting to an arbitrary string function in order to retrieve all bytes is intended or desired or undesired.

With fixed length strings the LEN() function returns the true length independent of null characters. A more logical approach might be that upon conversion from fixed length string to variable length string the entire fixed length string is tested. As soon as a character is found that is not a null character, the entire string should be returned as the content might be intended. If all characters are null characters the string could be considered empty and a null string could be returned.
Last edited by Munair on Nov 26, 2018 9:03, edited 1 time in total.
MrSwiss
Posts: 3025
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: Fixed Length String handling

Postby MrSwiss » Nov 23, 2018 12:51

Munair wrote:With fixed length strings the LEN() function returns the true length independent of null characters.
This seems to be an assumption by you, which the following snippet clearly shows.
As long as the ZString Ptr (a sort of fixed len string) is allocated, but not initialized,
Len() returms a arbitrary number ... (after init, Len() returns correctly, until zero).

Code: Select all

Dim As ZString Ptr  psz = Allocate(20 + 1)  ' useful len = 20

Print "allocate 20 + 1 byte, to ZString Ptr"
Print "Len(): "; Len(*psz)
Print
Print "initialize *psz"
Print
*psz = "hello from FreeBASIC"
Print "Len(): "; Len(*psz)
Print *psz
Print
DeAllocate(psz) : psz = 0
sleep
allocate 20 + 1 byte, to ZString Ptr
Len(): 3

initialize *psz

Len(): 20
hello from FreeBASIC

Munair
Posts: 834
Joined: Oct 19, 2017 15:00
Location: 't Zand, NL
Contact:

Re: Fixed Length String handling

Postby Munair » Nov 23, 2018 13:12

MrSwiss wrote:
Munair wrote:With fixed length strings the LEN() function returns the true length independent of null characters.
This seems to be an assumption by you, which the following snippet clearly shows.
As long as the ZString Ptr (a sort of fixed len string) is allocated, but not initialized,
Len() returms a arbitrary number ... (after init, Len() returns correctly, until zero).

You're confusing ZString Ptr with Zstring * Size. The first is a pointer to a string that may or may not yet have been assigned data of arbitrary length to be terminated by a null character. The second is fixed length upon declaration and LEN() will always return its predefined size - 1. Data longer than this size will be truncated.

My assumption is correct within the provided context.
Last edited by Munair on Nov 24, 2018 11:47, edited 1 time in total.
MrSwiss
Posts: 3025
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: Fixed Length String handling

Postby MrSwiss » Nov 23, 2018 13:22

No, the confusion seems to be on your end ...
A uninitialized ZString's len = 0 (even if allocated 21 bytes), since it hit's the Terminator
(NULL / ZERO) straight away! (initialized probably with: CAllocate)

Code: Select all

Dim As ZString * 21  sz  ' useful len = 20

Print "ZString * 21"
Print "Len(): "; Len(sz)
Print
Print "initialize *psz"
Print
sz = "hello from FreeBASIC"
Print "Len(): "; Len(sz)
Print sz
Print

sleep
Last edited by MrSwiss on Nov 23, 2018 13:25, edited 1 time in total.
jj2007
Posts: 1131
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Fixed Length String handling

Postby jj2007 » Nov 23, 2018 13:24

Taking your example from the other thread:

Code: Select all

type TPerson
   firstname as string * 16
end type

dim p as TPerson
mid(p.firstname, 6) = "jon"
dim s as string = p.firstname
print "part of type: [";s;"] with len=";len(s)

dim fixstring as string * 16
print "  standalone: [";fixstring;"] with len=";len(fixstring)   
   
Sleep

Output:

Code: Select all

part of type: [] with len= 0
  standalone: [                ] with len= 16

It seems that the wrong behaviour occurs only when the string is "embedded" in a TYPE.
Munair
Posts: 834
Joined: Oct 19, 2017 15:00
Location: 't Zand, NL
Contact:

Re: Fixed Length String handling

Postby Munair » Nov 23, 2018 13:29

MrSwiss wrote:No, the confusion seems to be on your end ...

Nope. You're first example is NOT fixed length upon declaration as I explained in my previous post. Again, Zstring Ptr is a data type which upon declaration has no length at all. So its size will be determined later and can be arbitrary:

Code: Select all

function ZStringPtrSize(byref s as zstring ptr) as uinteger
   return len(*s)
end function

dim buffer as string

buffer = "Here is something for you."
   print len(buffer)
   print ZstringPtrSize(buffer)
   
buffer = "Something totally different."
   print len(buffer)
   print ZstringPtrSize(buffer)

end

This thread is about fixed length strings upon declaration. Please stay on topic.
Last edited by Munair on Nov 23, 2018 13:44, edited 1 time in total.
MrSwiss
Posts: 3025
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: Fixed Length String handling

Postby MrSwiss » Nov 23, 2018 13:33

Sorry, but ZString * 21 IS a fixed size (don't rely on comments/print out's, in such quick hacks).

Please read the code more carefully ...
Munair
Posts: 834
Joined: Oct 19, 2017 15:00
Location: 't Zand, NL
Contact:

Re: Fixed Length String handling

Postby Munair » Nov 23, 2018 13:38

jj2007 wrote:Taking your example from the other thread:

Code: Select all

type TPerson
   firstname as string * 16
end type

dim p as TPerson
mid(p.firstname, 6) = "jon"
dim s as string = p.firstname
print "part of type: [";s;"] with len=";len(s)

dim fixstring as string * 16
print "  standalone: [";fixstring;"] with len=";len(fixstring)   
   
Sleep

Output:

Code: Select all

part of type: [] with len= 0
  standalone: [                ] with len= 16

It seems that the wrong behaviour occurs only when the string is "embedded" in a TYPE.

No, it occurs when converting fixed length string to variable length string:

Code: Select all

dim s as string * 16
dim t as string

mid(s, 6) = "jon"

t = s
print t; len(t)
Munair
Posts: 834
Joined: Oct 19, 2017 15:00
Location: 't Zand, NL
Contact:

Re: Fixed Length String handling

Postby Munair » Nov 23, 2018 13:47

MrSwiss wrote:Sorry, but ZString * 21 IS a fixed size (don't rely on comments/print out's, in such quick hacks).

Please read the code more carefully ...

Please read my earlier statement:
You're confusing ZString Ptr with Zstring * Size.
Last edited by Munair on Nov 23, 2018 22:08, edited 1 time in total.
MrSwiss
Posts: 3025
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: Fixed Length String handling

Postby MrSwiss » Nov 23, 2018 13:54

Munair wrote:It seems you still do not understand the difference.
I do, but it seems you ???
Since you are so brilliant, I'll leave it to you, to fix your problem
(others don't seem to have it).
jj2007
Posts: 1131
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Fixed Length String handling

Postby jj2007 » Nov 23, 2018 13:59

Munair wrote:
jj2007 wrote:It seems that the wrong behaviour occurs only when the string is "embedded" in a TYPE.

No, it occurs when converting fixed length string to variable length string:

You are right, I stand corrected!

Code: Select all

type TPerson
   firstname as string * 16
end type

dim p as TPerson

print "  original p: [";p.firstname;"] with len=";len(p.firstname)
mid(p.firstname, 6) = "jon"
print " modifiedl p: [";p.firstname;"] with len=";len(p.firstname)

dim s as string = p.firstname
print "converted->s: [";s;"] with len=";len(s)

dim fixstring as string * 16
print "  standalone: [";fixstring;"] with len=";len(fixstring)   
   
Sleep
Output:

Code: Select all

  original p: [                ] with len= 16
 modifiedl p: [     jon        ] with len= 16
converted->s: [] with len= 0
  standalone: [                ] with len= 16

And there is a nice surprise when using dim s as string *16 = p.firstname
Munair
Posts: 834
Joined: Oct 19, 2017 15:00
Location: 't Zand, NL
Contact:

Re: Fixed Length String handling

Postby Munair » Nov 23, 2018 14:04

Yes, FB simply assumes that when it encounters a null character the string ends. I consider that unpreferred compiler policy. The string might as well be terminated with any character < 32. In my opinion the compiler should leave it up to the programmer to consider what is relevant and only assume an empty string when all characters are null.

Furthermore it would be nice if a null assignment buffer = "" would replace all (null) characters with spaces.
Munair
Posts: 834
Joined: Oct 19, 2017 15:00
Location: 't Zand, NL
Contact:

Re: Fixed Length String handling

Postby Munair » Nov 23, 2018 14:10

jj2007 wrote:And there is a nice surprise when using dim s as string *16 = p.firstname

S is now also 16 bytes in size, but upon assignment still the copy ends when a null character is found, so S will be empty.
dodicat
Posts: 5697
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Fixed Length String handling

Postby dodicat » Nov 23, 2018 15:03

As with many things in fb, a workaround is required, so, getting the best one is the immediate quest.

I think mid() is the easiest cast from fixed string to var string.
Munair's previous example(fiddled with unfix a bit)

Code: Select all

type TT1
   s as string * 8
   static as string result
end type
dim shared as string tt1.result
tt1.result=space(8)

type TT2
   v as longint
end type

Function framecounter As long
    dim as double t2=timer
    Static As Double t3,frames,answer
    frames=frames+1
    If (t2-t3)>=1 Then
        t3=t2
        answer=frames
        frames=0
    End If
    Return answer
End Function

function unfix(byref s as zstring) byref as string
   for i as uinteger = 0 to 7
      TT1.result[i] = (s)[i]
   next
   return TT1.result
end function

function vs(v as longint) as string
    dim as string tmp
   
   dim t1 as TT1
   dim t2 as TT2
   t2.v = v
   lset t1, t2
 
  ' return chr(asc(t1.s,1),asc(t1.s,2),asc(t1.s,3),asc(t1.s,4),asc(t1.s,5),asc(t1.s,6),asc(t1.s,7),asc(t1.s,8))
   'return chr(t1.s[0],t1.s[1],t1.s[2],t1.s[3],t1.s[4],t1.s[5],t1.s[6],t1.s[7])
   return mid(t1.s,1)
   'return right(t1.s,8)
   'return unfix(t1.s)
end function

function sv(s as string) as longint
   dim t1 as TT1
   dim t2 as TT2
   t1.s = s
   lset t2, t1
   return t2.v
end function

screen 12
do
    locate 1,,0
dim x as longint = rnd*3044630875795881983
   'print x
dim s as string = vs(x)
  var v = sv(s)
   'print v
   if v<>x then beep:sleep
   locate 5
   print "fPS ";framecounter
 '  sleep
   loop until inkey=chr(27)

end 

But is this this a fair comparison?
Munair
Posts: 834
Joined: Oct 19, 2017 15:00
Location: 't Zand, NL
Contact:

Re: Fixed Length String handling

Postby Munair » Nov 23, 2018 15:05

A simple way to get the copy across is by means of a string function (I prefer LEFT()):

Code: Select all

type TPerson
   firstname as string * 16
end type

dim p as TPerson

mid(p.firstname, 6) = "jon"
print p.firstname
dim s as string * 16 = left(p.firstname, 16)
print "converted->s: [";s;"] with len=";len(s)
   
Sleep

Return to “General”

Who is online

Users browsing this forum: No registered users and 5 guests