[SOLVED] turning working KNN code into a static library

New to FreeBASIC? Post your questions here.
Post Reply
ron77
Posts: 212
Joined: Feb 21, 2019 19:24

Re: [help] converting code from python to FB

Post by ron77 »

hello badidea :)

okay then what should i use instead? how do i loop over all values in x()?

btw you are invited to join the project.

:)
ron77
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: [help] converting code from python to FB

Post by Tourist Trap »

So first thing first.

The program starts here:

Code: Select all

'python
dataset = [[1, 1], [2, 3], [4, 3], [3, 2], [5, 5]]
If we want to keep it close to Python we can translate to this:

Code: Select all

'base 0 array assumed
#define definedataset(rowcount, init...)  dim datasetdefined(rowcount - 1 ,1) as double = init
#define dataset(r,i) datasetdefined(r ,i)

definedataset(5, {{1, 1}, {2, 3}, {4, 3}, {3, 2}, {5, 5}})    'close to "dataset = [[1, 1], [2, 3], [4, 3], [3, 2], [5, 5]]"

Code: Select all

for i as integer = 0 to ubound(datasetdefined, 1)
    ? dataset(i,0), dataset(i,1)
next i
'output:
' 1             1
' 2             3
' 4             3
' 3             2
' 5             5
Do you agree so far?
ron77
Posts: 212
Joined: Feb 21, 2019 19:24

Re: [help] converting code from python to FB

Post by ron77 »

hi Tourist Trap...

as for the datasets i have been given an OOP code that loads the data in CVS files...

here is the code of it:

Code: Select all

#include once "file.bi"

/'
  Number of claims
  Total payment for all the claims in thousands of Swedish Kronor
  for geographical zones in Sweden
'/
type InsuranceData
  as single _
    numberOfClaims, _
    totalPayment
end type

type InsuranceTable
  declare operator []( byval as uinteger ) byref as InsuranceData
  as InsuranceData row( any )
  as uinteger count
end type

operator InsuranceTable.[]( byval index as uinteger ) byref as InsuranceData
  return( row( index ) )
end operator

sub add overload( byref t as InsuranceTable, byref d as InsuranceData )
  t.count += 1
  redim preserve t.row( 0 to t.count - 1 )
  
  t.row( t.count - 1 ) = d
end sub

function loadDataset( byref path as const string ) as InsuranceTable
  dim as InsuranceTable t
  
  if( fileExists( path ) ) then
    dim as long f = freeFile()
    
    open path for input as f
    
    do while( not eof( f ) )
      dim as InsuranceData d
      
      input #f, d.numberOfClaims
      input #f, d.totalPayment
      
      add( t, d )
    loop
  end if
  
  return( t )
end function

var t = loadDataset( "dataset.csv" )

for i as integer = 0 to t.count - 1
  with t[ i ]
    ? .numberOfClaims, .totalPayment
  end with
next

sleep()
and the cvs file:

Code: Select all

1, 1
2, 3
4, 3
3, 2
5, 5
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: [help] converting code from python to FB

Post by Tourist Trap »

ron77 wrote:hi Tourist Trap...

as for the datasets i have been given an OOP code that loads the data in CVS files...
Ok. That's great job. But it won't help dealing with translating the Python code. Once we have this first one in freebasic form, dealing with the CSV is something else.

Of course we could do all the work in the CSV reader and even do the linear regression ourselves there. Indeed, I'm not sure why you grabbed this python program for this job.

So here is my proposal. We can keep translating 1 to 1 the python program. Or we can talk about linear regression. In the second case, we would find out very fastly that there are already math libs in C that do the job more than greatly, and that we would be able to link to FB easily. Then your left work would only be the part where you read the data, prepare it, manage it, from a user interface point of view. FB is very good at this.

So this is important that you evaluate if you still need this piece of pythonic stuff here :)
ron77
Posts: 212
Joined: Feb 21, 2019 19:24

Re: [help] converting code from python to FB

Post by ron77 »

hi Tourist Trap...

i'll tell you honestly that i'm willing to accept your offer and solution to convert the python code 1 to 1 your way... right now i'm only interested in translation the python code into FB and make it work - that's the MAIN GOUL of mine right now and i'm not strict about the ways of how to code it - i'm open to any suggestion or idea - even it i already have a half beaked code if some one or any one comes and says "look we need to start from scratch and do it like this" then FINE BY ME...

thanks and welcome aboard the project's ship... :)

ron77
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: [help] converting code from python to FB

Post by Tourist Trap »

ron77 wrote:right now i'm only interested in translation the python code into FB and make it work - that's the MAIN GOUL of mine right now and i'm not strict about the ways of how to code it - i'm open to any suggestion or idea - even it i already have a half beaked code if some one or any one comes and says "look we need to start from scratch and do it like this" then FINE BY ME...
Hi ron,

Then I would say that for me there are 2 ways here. Or deal with the code like pseudo-code, and try to figure out what in general terms it does and means. Then we implement it in FB. Or, we try to do 1 to 1 translation thanks to FB's macros.

I'm more inclined to the second option because it's the funny one :)

We already know how to pass a function as parameter and things like that. So far I've identified 3 kinds of constructs that we still have to deal with:
1 - loops through intervals
for i in range(len(actual)):
for i in range(len(x)):

2 - "for each" loops
for row in dataset:
for row in test:

3 - inline loops
actual = [row[-1] for row in dataset]
x = [row[0] for row in dataset]
I'm still thinking on how to deal with that without having to do too much parsing.
ron77
Posts: 212
Joined: Feb 21, 2019 19:24

Re: [help] converting code from python to FB

Post by ron77 »

hi tourist trap...

i've looked at the first draft code i was working on and i will now give here what i think we can extract from the old code to the new... i mean mostly equations function i have already successfully converted to FB... so here is what i got so far:

Code: Select all

python code
# Calculate the mean value of a list of numbers
def mean(values):
	return sum(values) / float(len(values))

Code: Select all

'fb code
' Calculate the mean value of a list of numbers
function sum(x() as double) as double
  dim as single result
  for i as integer = 0 to ubound(x) - 1
    result = result + x(i)
  next i
  return result
end FUNCTION

Code: Select all

python code
# Calculate the variance of a list of numbers
def variance(values, mean):
	return sum([(x-mean)**2 for x in values])

Code: Select all

'fb code
' Calculate the variance of a list of numbers
function variance(values() AS double, BYVAL means AS DOUBLE) AS DOUBLE
   DIM resalt AS DOUBLE = 0
   FOR i AS INTEGER = LBOUND(values) TO UBOUND(values)
      resalt = resalt + (values(i) - means) * (values(i) - means)
   NEXT i
   Return resalt
END FUNCTION

Code: Select all

python code
# Calculate covariance between x and y
def covariance(x, mean_x, y, mean_y):
	covar = 0.0
	for i in range(len(x)):
		covar += (x[i] - mean_x) * (y[i] - mean_y)
	return covar

Code: Select all

'fb code
FUNCTION covariance(x()as double, mean_x as double, y() as double, mean_y as double) as Double
    dim covar as Double
    for i as integer = 0 to UBOUND(x) - 1
        covar += (x(i) - mean_x) * (y(i) - mean_y)
    next
    return covar
end FUNCTION

Code: Select all

'fb code
'calculate mean
function mean(x() as double) as double
  return sum(x()) / cdbl(ubound(x) + 1)
end FUNCTION
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: [help] converting code from python to FB

Post by Tourist Trap »

ron77 wrote:hi tourist trap...

i've looked at the first draft code i was working on and i will now give here what i think we can extract from the old code to the new... i mean mostly equations function i have already successfully converted to FB... so here is what i got so far:

Code: Select all

'fb code
' Calculate the mean value of a list of numbers
function sum(x() as double) as double

Code: Select all

'fb code
' Calculate the variance of a list of numbers
function variance(values() AS double, BYVAL means AS DOUBLE) AS DOUBLE

Code: Select all

'fb code
FUNCTION covariance(x()as double, mean_x as double, y() as double, mean_y as double) as Double

Code: Select all

'fb code
'calculate mean
function mean(x() as double) as double
Fine. Those at least won't cause us trouble any more.

From my part, I have extended a little the translation of the first line of the original program. So will be able to have more realistic choices for testing matter:

Code: Select all

function CharCount(byref S as const string) as integer
    dim as integer returnValue = 0
    for cursorIndex as integer = 1 to len(S)
        if chr(S[cursorIndex])="," then returnValue += 1
    next cursorIndex
    return returnValue
end function
#macro definedataset(init...)
    dim datasetdefined(CharCount(#init)\2,1) as double
    scope
        dim as string ss = #init        
        dim as boolean toggleValue
        dim as integer indexer = 0
        for i as integer = 0 to len(ss)
            if asc(mid(ss, i + 1, 1))>47 andAlso asc(mid(ss, i + 1, 1))<58 then
                toggleValue = not toggleValue
                dim as integer startPos = i + 1
                dim as integer endPos = startPos
                do
                    if (asc(mid(ss, endPos, 1))>47 andAlso asc(mid(ss, endPos, 1))<58) orElse (mid(ss, endPos, 1)=".") then
                        endPos += 1
                    else
                        exit do
                    end if
                loop
                if toggleValue then
                    datasetdefined(indexer,0) = cDbl(mid(ss, i + 1, (endPos - startPos)))
                else
                    datasetdefined(indexer,1) = cDbl(mid(ss, i + 1, (endPos - startPos)))
                    indexer += 1
                end if
                i += (endPos - startPos)
            end if
        next i
    end scope
#endMacro

#define dataset(r,i) datasetdefined(r ,i)

'-------------------------------------------------------------------------------

definedataset({{100, 11.333}, {2, 3}, {4, 3}, {3, 2}, {51223.3221, 9999.9999}, {1.8888, 8.11111})

for i as integer = 0 to ubound(datasetdefined, 1)
    ? dataset(i,0),, dataset(i,1)
next i
The important part is :
definedataset({{100, 11.333}, {2, 3}, {4, 3}, {3, 2}, {51223.3221, 9999.9999}, {1.8888, 8.11111}),
that's as easy to type as
dataset = [[1, 1], [2, 3], [4, 3], [3, 2], [5, 5]].

Then now the second line requires to set up the proc pointer. That won't be difficult. Then we'll have to mimic the loops.
ron77
Posts: 212
Joined: Feb 21, 2019 19:24

Re: [help] converting code from python to FB

Post by ron77 »

hi tourist trap...

i would like to add you as a collaborator to the github project repository - do you have a github account and if so by what nick name?

:)
ron77
(ronen blumberg)
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: [help] converting code from python to FB

Post by Tourist Trap »

ron77 wrote:do you have a github account and if so by what nick name?
Thanks. Then you can find my Github account under the name : @trapmania. Should be still alive.
ron77
Posts: 212
Joined: Feb 21, 2019 19:24

Re: [help] converting code from python to FB

Post by ron77 »

hi tourist trap..

added you to the repository as collaborator check your e-mail :)

ron77
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: [help] converting code from python to FB

Post by Tourist Trap »

ron77 wrote:hi tourist trap..

added you to the repository as collaborator check your e-mail :)

ron77
Done, thanks.
I'm right now looking at overloading the operator [ ] from inside a UDT. Maybe it will be the best way to mimic python syntax when passing and using the dataset variable.
https://www.freebasic.net/wiki/ProPgOpe ... ing#MEMBER

@fxm,
is there not a mistake in the doc:
Declare Operator [] ( ByRef lhs As T Pointer, ByRef rhs As Integer ) ByRef As T
The compiler doesn't accept any rhs argument. You do not use this argument neither it seems from here : viewtopic.php?f=7&t=28720&p=274383&hilit=array#p274383.
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: [help] converting code from python to FB

Post by MrSwiss »

ron77, there are some problems in your math-code "as-is" (especially for a library).

You are making assumptions about the array structure, which will lead to errors later.
The typical user (of a lib) will throw just about anything at your code and you'll have to deal with it.
(Maybe better explained in actual FB code)

Code: Select all

Function Sum(a() As Double) As Double   ' sum up 1D-arrays contents
    Var r = 0.0                         ' r = double (by assignment)
    For i As Integer = LBound(a) To UBound(a)   ' NO assumtions made (about array)
        r += a(i)
    Next
    Return r
End Function

Function Ave(a() As Double) As Double   ' average (NOT to be confused with 'statistical' mean!)
    Var n = (UBound(a) - LBound(a)) + 1 ' n = integer (by assignment)
    Return Sum(a()) / n                 ' n = implicitly converted to double
End Function
Before going into math coding there are a few vital things to be aware of:
  • Variables (and their limits)
    Operator Precedence
    Conversion and Coersion
The list may be incomplete but OK for starters ... see: FB-Documentation.
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: [help] converting code from python to FB

Post by Tourist Trap »

Ok so this below is something that should work to get the square brackets instead of the rounded ones:

Code: Select all

type P2
    declare constructor()
    declare destructor()
    declare operator []( byval index As integer ) byref As double
    as double ptr _dptr(1 to 2)
end type

constructor P2()
    this._dptr(1) = new double(1)
    this._dptr(2) = new double(1)
end constructor
destructor P2()
    delete this._dptr(1)
    delete this._dptr(2)
end destructor

operator P2.[]( byval index As integer ) byref As double
    return *This._dptr(index)
end operator

dim as P2 pp

*pp._dptr(1) = 998
*pp._dptr(2) = 887

pp[1] = 666.779

? pp[1], pp[2]
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: [help] converting code from python to FB

Post by fxm »

Ok, but under the hood, we can just use an array of Double (instead of an array of pointer + New/Delete), inducing a structure simpler than above, while keeping the same interfacing for user with square brackets:

Code: Select all

type P2
    declare operator []( byval index As integer ) byref As double
    as double _d(1 to 2) = {1, 1}
end type

operator P2.[]( byval index As integer ) byref As double
    return _d(index)
end operator

dim as P2 pp

pp._d(1) = 998
pp._d(2) = 887

pp[1] = 666.779

? pp[1], pp[2]
Post Reply