Making an ANN

User projects written in or related to FreeBASIC.
BasicCoder2
Posts: 3359
Joined: Jan 01, 2009 7:03

Making an ANN

Postby BasicCoder2 » Jul 03, 2018 0:40

For some time I have always wanted to code an ANN black box but couldn't find a suitable tutorial that could show a math challenged reader the arithmetic involved in backpropagation instead explaining it with pure calculus.
This code is based on the tutorial at,
https://mattmazur.com/2015/03/17/a-step ... n-example/
Because it gives the actual numbers it is possible to check that you have coded correctly when your numbers come out the same.
The computations in the example below have been expanded out rather than each layer of the ANN having its own for/next loop. Hopefully I will fix that in the near future. I still don't understand it enough to add more hidden layers but ultimately I would like to end up with a generic ANN that I know how to use in my own AI examples.

Code: Select all

' This code is based on the tutorial at,
' https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
' It prints out the new values of weights after one forward and back pass

screenres 1280,600,32
color rgb(0,0,0),rgb(255,255,255):cls

function sigmoid(X as double) as double
    return 1/( 1 + EXP((-1)*X))
end function

dim shared as double i1,i2                      'inputs
dim shared as double w1,w2,w3,w4,w5,w6,w7,w8    'weights
dim shared as double nw1,nw2,nw3,nw4,nw5,nw6,nw7,nw8  'updated weights
dim shared as double netH1,netH2,neto1,neto2    'sum of weights * inputs
dim shared as double outH1,outH2,outo1,outo2    'sigmoid outputs
dim shared as double targeto1,targeto2          'targets
dim shared as double b1,b2                      'bias

dim shared as double Etotal,Eo1,Eo2             'total error

dim shared as double learningRate
learningRate = 0.5

'intialize inputs
i1 = 0.05
i2 = 0.10
'intialize bias
b1 = 0.35
b2 = 0.60
'initialize weights
w1 = 0.15
w2 = 0.20
w3 = 0.25
w4 = 0.30
w5 = 0.40
w6 = 0.45
w7 = 0.50
w8 = 0.55
'target values
targeto1 = 0.01
targeto2 = 0.99

'=============================================================================
'        THE FORWARD PASS
'=============================================================================
netH1 = w1*i1 + w2*i2 + b1 * 1
outH1 = sigmoid(netH1)

netH2 = w3*i1 + w4*i2 + b1 * 1
outH2 = sigmoid(netH2)

neto1 = w5*outH1 + w6*outH2 + b2 * 1
outo1 = sigmoid(neto1)

neto2 = w7*outH1 + w8*outH2 + b2 * 1
outo2 = sigmoid(neto2)

'COMPUTE ERROR
'=============
Eo1 = (targeto1 - outo1)^2 * 0.5
Eo2 = (targeto2 - outo2)^2 * 0.5
Etotal = Eo1 + Eo2

'=============================================================================
'          THE BACKWARD PASS
'=============================================================================
dim as single pX,pA,pB,pC   'used to hold results
'=============================================================================
' COMPUTE NEW WEIGHT VALUES IN OUTPUT LAYER
'=============================================================================
pA = -(targeto1-outo1)
pB = outo1 * (1 - outo1)
pC = outH1
pX = pA * pB * pC
nw5 = w5 - learningRate * pX
'=============================================================================
pA = -(targeto1-outo1)
pB = outo1 * (1 - outo1)
pC = outH2                    '<--- change
pX = pA * pB * pC
nw6 = w6 - learningRate * pX
'=============================================================================
pA = -(targeto2-outo2)
pB = outo2 * (1 - outo2)
pC = outH1
pX = pA * pB * pC
nw7 = w7 - learningRate * pX
'=============================================================================
pA = -(targeto2-outo2)
pB = outo2 * (1 - outo2)
pC = outH2
pX = pA * pB * pC
nw8 = w8 - learningRate * pX
'=============================================================================
dim as single temp,J,K,L,M,N   'used to hold results
'=============================================================================
'  COMPUTE NEW WEIGHT VALUES IN HIDDEN LAYER
'=============================================================================
'      NEW w1
'=============================================================================
pA = -(targeto1-outo1)
pB = outo1 * (1 - outo1)
temp = pA * pB
J = temp * w5
pA = -(targeto2-outo2)
pB = outo2 * (1 - outo2)
temp = pA * pB
K = temp * w7
L = J + K
M = outH1 * (1- outH1)
N = L * M * i1
nw1 = w1 - learningRate * N
'==============================================================================
'       NEW w2
'==============================================================================
pA = -(targeto1-outo1)
pB = outo1 * (1 - outo1)
temp = pA * pB
J = temp * w5
pA = -(targeto2-outo2)
pB = outo2 * (1 - outo2)
temp = pA * pB
K = temp * w7
L = J + K
M = outH2 * (1- outH2)
N = L * M * i2
nw2 = w2 - learningRate * N
'=============================================================================
'     NEW w3
'=============================================================================
pA = -(targeto1-outo1)
pB = outo1 * (1 - outo1)
temp = pA * pB
J = temp * w5
pA = -(targeto2-outo2)
pB = outo2 * (1 - outo2)
temp = pA * pB
K = temp * w7
L = J + K
M = outH2 * (1- outH2)
N = L * M * i2
nw3 = w3 - learningRate * N
'==============================================================================
'             NEW w4
'==============================================================================
pA = -(targeto1-outo1)
pB = outo1 * (1 - outo1)
temp = pA * pB
J = temp * w5
pA = -(targeto2-outo2)
pB = outo2 * (1 - outo2)
temp = pA * pB
K = temp * w7
L = J + K
M = outH2 * (1- outH2)
N = L * M * i2
nw4 = w4 - learningRate * N
'=============================================================================
' these should have the same values as given in the tutorial
print "new w1 = ";nw1
print "new w2 = ";nw2
print "new w3 = ";nw3
print "new w4 = ";nw4
print "new w5 = ";nw5
print "new w6 = ";nw6
print "new w7 = ";nw7
print "new w8 = ";nw8


sleep
Last edited by BasicCoder2 on Jul 09, 2018 5:52, edited 1 time in total.
dafhi
Posts: 1238
Joined: Jun 04, 2005 9:51

Re: Making an ANN

Postby dafhi » Jul 09, 2018 0:54

I highly recommend Luis Serrano's videos, especially this one

Return to “Projects”

Who is online

Users browsing this forum: No registered users and 1 guest