Undocumented function

General FreeBASIC programming questions.
sancho3
Posts: 358
Joined: Sep 30, 2017 3:22

Undocumented function

Postby sancho3 » Jan 02, 2019 1:32

I was searching around for some specific code and came across this thread
In it chaos presets a function using a FB function called fb_StrAllocTempDescF
Here is the posted use of it (by chaos a little later in the thread):

Code: Select all

Function Cva (File() As uByte, ByVal Length As Integer = -1, ByVal Start As Integer = -1) As String
 
    If Start < LBound(File) Or Start > UBound(File) Then Start = LBound(File)
    If Length = -1 Or Length > UBound(File) - Start + 1 Then Length = UBound(File) - Start + 1
    Return fb_StrAllocTempDescF( @File(Start), Length )

End Function

I can't find any information about this at all. Is there any?
Are there any more undocumented functions?
fxm
Posts: 8907
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Undocumented function

Postby fxm » Jan 02, 2019 5:54

/* temp string descriptor allocation for fixed-len strings */
https://www.google.com/url?sa=t&rct=j&q ... cokbAfihEt

Should be equivalent to:

Code: Select all

Function Cva (File() As uByte, ByVal Length As Integer = -1, ByVal Start As Integer = -1) As String
 
    If Start < LBound(File) Or Start > UBound(File) Then Start = LBound(File)
    If Length = -1 Or Length > UBound(File) - Start + 1 Then Length = UBound(File) - Start + 1
    'Return fb_StrAllocTempDescF( @File(Start), Length )
    Return Left( *Cptr(Zstring Ptr, @File(Start)), Length )

End Function
counting_pine
Site Admin
Posts: 6166
Joined: Jul 05, 2005 17:32
Location: Manchester, Lancs

Re: Undocumented function

Postby counting_pine » Jan 02, 2019 9:08

The file containing it is here: https://github.com/freebasic/fbc/blob/m ... empdescf.c
If you search the Github project, you can also find instances where it's used internally.

Its behaviour could be better documented, but it wouldn't be part of the "public" documentation (e.g. the wiki), because it's part of FB's internal implementation, and it's not designed for end users. It could (in theory) change at any time, so it shouldn't be relied upon.


A caveat on left(*zptr, n) - I can't say for certain how this behaves.
The most desirable behaviour here would be that it reads the string up to the first n characters, or the null terminator, whichever comes first.

But in reality, when converting *zptr to string, I suspect it might read up to the null terminator to first determine the length, then convert to string, then truncate to n characters.

This means it might potentially have to read through megabytes of data, just to return a string with a few characters. Or alternatively, it might run out of bounds and cause a segfault.
fxm
Posts: 8907
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Undocumented function

Postby fxm » Jan 02, 2019 9:31

counting_pine wrote:A caveat on left(*zptr, n) - I can't say for certain how this behaves.
The most desirable behaviour here would be that it reads the string up to the first n characters, or the null terminator, whichever comes first.

But in reality, when converting *zptr to string, I suspect it might read up to the null terminator to first determine the length, then convert to string, then truncate to n characters.

This means it might potentially have to read through megabytes of data, just to return a string with a few characters. Or alternatively, it might run out of bounds and cause a segfault.

I thought about it, but I was too optimistic about the performance of the compiler.
After a test, I see that it does indeed what you feared:

Code: Select all

Redim As Ubyte array(1024*1024*1024)
Dim As Double t
Dim As String s

For I As Integer = Lbound(array) To Ubound(array) - 1
  array(I) = 65
Next I

t = Timer
s = Left(*Cptr(Zstring Ptr, @array(0)), 3)
t = Timer - t
Print t
Print "'" & s & "'"
Print

array(5) = 0
t = Timer
s = Left(*Cptr(Zstring Ptr, @array(0)), 3)
t = Timer - t
Print t
Print "'" & s & "'"
Print

Sleep

Code: Select all

 0.1623539000473784
'AAA'

 1.099977168905753e-006
'AAA'
fxm
Posts: 8907
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Undocumented function

Postby fxm » Jan 02, 2019 10:15

So, the optimized solution is either to create its own temporary string descriptor or to use this internal function of FB.

Warning:
- When I tested these two solutions, I discovered that the first argument to pass to 'fb_StrAllocTempDescF()' would not be an address but a reference (and the second argument would be the length+1)!
- Therefore, the function 'Cva()' of the first post would be bugged?


Code: Select all

Redim As Ubyte array(1024*1024*1024)
Dim As Ubyte u = 65
For I As Integer = Lbound(array) To Ubound(array) - 1
  array(I) = u
  u += 1
  If u = 0 Then u = 1
Next I
Dim As Double t
Dim As String s


t = Timer
s = Left(*Cptr(Zstring Ptr, @array(0)), 3)
t = Timer - t
Print t
Print "'" & s & "'"
Print

t = Timer
Scope
  Dim As String s0
  Cptr(Ubyte Ptr Ptr, @s0)[0] = @array(0)
  Cptr(Integer Ptr, @s0)[1] = 3
  s = S0
  Cptr(Ubyte Ptr Ptr, @s0)[0] = 0
  Cptr(Integer Ptr, @s0)[1] = 0
End Scope
t = Timer - t
Print t
Print "'" & s & "'"
Print

t = Timer
s = fb_StrAllocTempDescF(array(0), 3+1)  '' or s = fb_StrAllocTempDescF(Byval @array(0), 3+1)
t = Timer - t
Print t
Print "'" & s & "'"
Print


Sleep

Code: Select all

 0.1780833000470352
'ABC'

 2.000356289499905e-007
'ABC'

 3.000266133312834e-007
'ABC'
Last edited by fxm on Jan 02, 2019 17:25, edited 1 time in total.
fxm
Posts: 8907
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Undocumented function

Postby fxm » Jan 02, 2019 17:21

My own 'Cva()' code body:

Code: Select all

Function Cva (File() As uByte, ByVal Length As Integer = -1, ByVal Start As Integer = -1) As String
 
    If Start < LBound(File) Or Start > UBound(File) Then Start = LBound(File)
    If Length = -1 Or Length > UBound(File) - Start + 1 Then Length = UBound(File) - Start + 1
    'Return fb_StrAllocTempDescF( @File(Start), Length )
    Scope
        Dim As String s0
        Cptr(Ubyte Ptr Ptr, @s0)[0] = @File(Start)
        Cptr(Integer Ptr, @s0)[1] = Length
        Function = s0
        Cptr(Ubyte Ptr Ptr, @s0)[0] = 0
        Cptr(Integer Ptr, @s0)[1] = 0
    End Scope

End Function
dodicat
Posts: 5700
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Undocumented function

Postby dodicat » Jan 02, 2019 19:02

Many undocumented string functions can be seen in the fb source code rtl-strings.bas.
I have tried some speed tests, mostly these functions are slower than the front end counterparts.

Here I have started the same as fxm but deviated.
You could probably use a union for this also, but I shall leave that as an exercise for somebody else.

Code: Select all

Function Cva (File() As Ubyte, Byval Length As Integer = -1, Byval Start As Integer = -1) As String
   
    If Start < Lbound(File) Or Start > Ubound(File) Then Start = Lbound(File)
    If Length = -1 Or Length > Ubound(File) - Start + 1 Then Length = Ubound(File) - Start + 1
    'Return fb_StrAllocTempDescF( @File(Start), Length )
    Scope
        Dim As String s0
        Cptr(Ubyte Ptr Ptr, @s0)[0] = @File(Start)
        Cptr(Integer Ptr, @s0)[1] = Length
        Function = s0
        Cptr(Ubyte Ptr Ptr, @s0)[0] = 0
        Cptr(Integer Ptr, @s0)[1] = 0
    End Scope
   
End Function

Sub memcopy(dest As Any Ptr,src As Any Ptr,count As Long)
    Dim As Ubyte Ptr newdst=dest
    Dim As Ubyte Ptr newsrc=src
    While count
        count-=1
        *newdst=*newsrc
        newdst+=1
        newsrc+=1
    Wend
End Sub

Function Cva2 (File() As Ubyte, Byval Length As Integer = -1, Byval Start As Integer = -1) As String
    If Start < Lbound(File) Or Start > Ubound(File) Then Start = Lbound(File)
    If Length = -1 Or Length > Ubound(File) - Start + 1 Then Length = Ubound(File) - Start + 1
    Dim As String s=String(length,0)
    Dim As Ubyte Ptr z=@s[0]
    memcopy(z,@file(start),length)
    Return s
End Function



Redim As Ubyte u(5000000)
For n As Long=0 To 5000000
    u(n)=48+Rnd*9
Next

For z As Long=1 To 5
    Dim As Double t
    t=Timer
    Print Left(cva(u(),100000,436201),50)
    Print Timer-t,"fxm"
   
    t=Timer
    Print Left(cva2(u(),100000,436201),50)
    Print Timer-t,"dodicat"
    Print
Next z
Sleep

 
coderJeff
Site Admin
Posts: 2841
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Undocumented function

Postby coderJeff » Jan 04, 2019 16:53

Here is the full list (or should be as of 2019-01-04) of "undocumented" rtlib functions: rtl-decl.bas

There are 500+ functions & procedures that fbc compiler expects to be available.

I used a variation of this dump of declarations to create this fbc test master/tests/warnings/rtl-prototypes.bas to check that we could get PROCPTR() of all rtlib functions. In fbc 1.06 I made a change to function pointers to allow this.

Currently, best source of documentation is to read the src/rtlib/*.c files. Most are not likely to change any time soon, so it's probably OK to use them. We just don't advertise it as a public API because we really don't want to guarantee that the API will never change.

Return to “General”

Who is online

Users browsing this forum: No registered users and 3 guests