[S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post your FreeBASIC source, examples, tips and tricks here. Please don’t post code without including an explanation.
Post Reply
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by D.J.Peters »

Roland Chastain wrote:Is it possible to call a web service which returns JSON data...
I don't see that operation in the examples.
A JSON file is a text file so it's easy to download
but I don`t know what you mean with "web service"

Joshy

Code: Select all

{
  "date":"2018-04-05",
  "season":"easter",
  "season_week":1,
  "celebrations":[{
    "title":"Pâques Jeudi",
    "colour":"white",
    "rank":"Jours liturgiques primaires",
    "rank_num":1.2}],
  "weekday":"thursday"
}
Is the same code as in the other examples you have only to set the right http request

Code: Select all

' [S]imple [N]etwork [C]onnection
#include once "snc.bi"
#include once "snc_utility.bi"

'http://calapi.inadiutorium.cz/api/v0/fr/calendars/general-fr/today

' test of a client connection 
const as string ServerName = "calapi.inadiutorium.cz"
const as string ServerPath = "/api/v0/fr/calendars/general-fr/"
const as string ServerFile = "today"
const as string FileType   = MIME_TXT

' connect to web server at port 80
dim as NetworkClient client=type(ServerName,80)
' get a connection from ConnectionFactory
var connection = client.GetConnection()
' build an HTTP GET request
dim as string request =HTTPGet(ServerName,ServerPath & ServerFile,FileType)

' ready to send ?
while connection->CanPut()<>1
  sleep 100
wend
' put data on the connection
connection->PutData(strptr(request),len(request))
' ready to receive ?
while connection->CanGet()<>1
  sleep 100
wend

dim as zstring ptr buffer
var nBytes = connection->GetData(buffer)
print "number of received bytes " & nBytes
' get last char position of the HTTP asci header
var LastChar=instr(*buffer,HeaderEnd)-1
var Header  =left(*buffer,LastChar)
' is it a OK answer ?
if instr(Header,"200 OK")<1 then
  print *buffer
  print "can't get " & ServerName & ServerPath & ServerFile & " !"
  beep:sleep:end
end if
' get first byte behind the HTTP asci header
var DataStart=LastChar+4
' save it
open "test.json" for binary access write as #1
dim as ubyte ptr FileBuffer=@buffer[DataStart]
nBytes-=DataStart
put #1,,*FileBuffer,nBytes
close #1
print "file saved ..."
' free the buffer (allocate by snc.bi)
deallocate buffer
sleep
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by D.J.Peters »

I updated the included examples to my new server the results should be right now.

Joshy
Roland Chastain
Posts: 992
Joined: Nov 24, 2011 19:49
Location: France
Contact:

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by Roland Chastain »

Perfect. Thank you Joshy.
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by D.J.Peters »

(Sorry about my bad English)
Next time isn't easy to get data from HTTP GET response !

All the years in the past in the response header was the property

Content-Length: ##### (<- the size of the data body after the header)

But yesterday I got the first response from WEB server without this property.

Instead I found

Transfer-Encoding: chunked

That means the following body of data are separated in chunks.

The simple string "FreeBASIC is fun." can be separated in 4 chunks !

Code: Select all

9\r\n
FreeBASIC\r\n
3\r\n
 is\r\n
5\r\n
 fun.\r\n
0\r\n
\r\n
chunk 1 size 9
chunk 2 size 3
chunk 3 size 5
chunk 4 size 0 (end chunk)

the size of a chunk are decoded in hex notation

Code: Select all

B\r\n <- a chunk with 0xB = 11 chars
12345678910\r\n
0\r\n
\r\n
This all isn't fun and the chunks can be compressed too.

Transfer-Encoding: gzip, chunked

So a simple HTTP reader/down loader needs a decompression library also.

I wrote ip.php on my server

Code: Select all

<?php
$ip = getenv("REMOTE_ADDR");
echo $ip;
?>
And wondered where the extra signs before and after the IP string xx.xx.xx.xx come from.

Joshy
Last edited by D.J.Peters on Apr 06, 2018 12:44, edited 1 time in total.
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by D.J.Peters »

Code: Select all

HTTP/1.1 200 OK
Server: nginx/1.12.2
Date: Fri, 06 Apr 2018 12:39:16 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: close
X-Powered-By: PHP/5.4.45
Vary: Accept-Encoding
file: "test.bas"

Code: Select all

' [S]imple [N]etwork [C]onnection

#include once "snc.bi"
#include once "snc_utility.bi"
#include once "crt/string.bi"

#define ASSTRING(x) *cptr(zstring ptr,(x))

' test of a client connection 
const as string ServerName = "shiny3d.de"
const as string ServerPath = "/public/"
const as string ServerFile = "ip.php"
const as string FileType   = MIME_TXT

' connect to web server at port 80
dim as NetworkClient client=type(ServerName,80)
' get a connection from ConnectionFactory
var connection = client.GetConnection()
' build an HTTP GET request
dim as string request = HTTPGet(ServerName,ServerPath & ServerFile,FileType)
'print "<request>"
'print request
'print "</request>"

' ready to send ?
while connection->CanPut()<>1
  sleep 100
wend
' put data on the connection
connection->PutData(strptr(request),len(request))
' ready to receive ?
while connection->CanGet()<>1
  sleep 100
wend

print "receive data ..."
dim as zstring ptr response
var nBytes = connection->GetData(response)
if nBytes<=0 then
  print "error: GetData()" 
  beep : sleep : end 1
end if  
print "got " & nBytes & " bytes."
print
'print "<response>"
'print ASSTRING(response)
'print "</response>"
var index=instr(ASSTRING(response),HEADEREND)-1
var header = left(ASSTRING(response),index)
index+=5

print "<header>"
print header
print "</header>"
print

if instr(header,"200 OK")<=0 then
  print "error: no OK response 1" 
  deallocate response
  beep : sleep : end 1
end if  
var chunked = iif(instr(header,"chunked")>0,true,false)
var msg = mid(ASSTRING(response),index,nBytes-index)
var IP = ""
if chunked then
  var token = strtok(strptr(msg), LINEEND)
  while (token<>0)
    var strHex = "&H" & *token
    var chunksize = val(strHex)
    if chunksize>0 then ip &= *strtok(0,LINEEND)
    token = strtok(0,LINEEND)
  wend
else
  ip=msg
end if  
print "Your IP: " & ip
deallocate response
print "done ..."
sleep
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by D.J.Peters »

file: sncGetGoogleMaps.bas does not work anymore.
You have to pay for a google static map API key since July 16, 2018.
https://developers.google.com/maps/docu ... nd-billing

Joshy
leopardpm
Posts: 1795
Joined: Feb 28, 2009 20:58

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by leopardpm »

Joshy,
I downloaded your ZIP file and ran 2 of the example programs, downloadTEXT and downloadIMAGE and they both work perfectly (nice image, btw!!!)...

forgive my very basic questions regarding networking:

in older FB (0.21 etc) builds, I thought there was a simple network chat example where two users on two diff computers could connect and type to each other...

Do you have an example of such a chat program using your library?

also, to connect to another computer out in the world, I am guessing that a client would need the IP address, right? How does the server get the IP address of the client?

networking is a strange beast to me....

NOTE: such a chat is not my final goal, but from that code I could create what I am looking for... I just need to understand how to send data back n forth between 2 puters...
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by D.J.Peters »

@leopardpm I can't send you an working example since Monday I'm in hospital
I use only the first post as reference to show you pseudo code for a "simple" chat server.

May be other users used s.n.c successful and can help you also.

Joshy

chat_server.bas NOTE "PSEUDO CODE" I write it here in the browser and pre compile it only in my head :-)

Code: Select all

#include once "snc.bi"

const as ushort port = 12345
var Server = NetworkServer(port)

dim as NetworkConnection ptr clients(63)  ' max 64 clients

print "server is running ! .. press any key to quit ..."
while inkey()=""
  var newClient = Server.GetConnection()
  if newClient<>0 then
    dim as integer freeIndex=-1
    for i as integer = 0 to 63
      if clients(i)=0 then freeIndex=i:exit for
    next
    if freeIndex<>-1 then
      ' add new client
      clients(foundIndex)=newClient
      print "new client connected IP: " & newClient->ClientIP
    else
      ' report a warning "max clients are connected !"
      print "warning no free client !"
    end if
  else
    ' no new connection show for any incoming message.
    for i as integer = 0 to 63
      ' if this client ae connected and has new data send
      if clients(i)<>0 andalso clients(i)->canGet()=1 then 
        ' receive it (max 32 chars only as an example)
        dim as any ptr buffer
        dim as integer nBytes = clients(i)->GetData(buffer,32)
        ' send the message to all other clients (if any)
        if nBytes>0 then
          for j as integer = 0 to 63
            if i<>j andalso clients(j)<>0 andalso clients(j)->canPut()=1 then
              var status = clients(i)->PutData(buffer,nBytes)
              if status=0 then
                ' client is disconnected mark it as free
                print "client IP: " & clients(j)->ClientIP & " disconnected !"
                clients(j)=0
              elseif status=-1 then
                ' handle error here
              end if  
            end if
          next
          ' buffer was allocated by s.n.c so free it here
          deallocate buffer
        end if
      end if  
    next
  end if
  sleep(10)
wend
chat_client.bas

Code: Select all

#include once "snc.bi"

const as ushort port = 12345
var client = NetworkClient("127.0.0.1",port) ' local net in this case

if client=0 then
  print "error: no running chat server found ATM."
  print "try later again ..."
  beep : sleep: end 1
end if

print "connected to chat server ..."

var msg=""
while msg<>"quit"
  ' any new incoming message ? 
  if client->canGet()=1 then
    ' get message
    dim as any ptr buffer
    var nBytes=client->GetData(buffer)
    if nBytes>0 then
      print "msg: " & *cptr(zstring ptr,buffer)
    else
      ' handle error here
    end if  
    
  else
    ' fill var msg here use input or what ever
    ' your code
    ' ....
    var nChars=len(msg)
    if nChars>0 andalso msg<>"quit" then
      ' send it to the chat server or wait if server are bussy !!!
      if client->canPut()=1 then
        var nBytesSended = client->PutData(strptr(msg),nChars+1)) ' plus one for the "0" string terminator
      end if
      
    end if  
  
  end if
  sleep(10)
wend  
  
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by Tourist Trap »

D.J.Peters wrote:@leopardpm I can't send you an working example since Monday I'm in hospital
I use only the first post as reference to show you pseudo code for a "simple" chat server.
Hi D.J.Peters,
if the nurse sees you coding even pseudo-coding and compiling, she will not be happy.
You have to run this right now:

Code: Select all

do
       sleep
until escape(hospital)
Have a good rest!
leopardpm
Posts: 1795
Joined: Feb 28, 2009 20:58

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by leopardpm »

D.J.Peters wrote:@leopardpm I can't send you an working example since Monday I'm in hospital
LOL! on wed of last weeek I got in a roll over car accident... just got home from hospital too
bcohio2001
Posts: 556
Joined: Mar 10, 2007 15:44
Location: Ohio, USA
Contact:

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by bcohio2001 »

Is there a way to share just the connection.
I would like to open just one connection and be able to use it anywhere in the program.

Code: Select all

var shared connection = client.GetConnection()
error 11: Expected constant in 'var shared connection = client.GetConnection()'
Update:
Did a little bit of digging and a little luck too.

Code: Select all

dim shared as NetworkConnection ptr connection '= client.GetConnection()
connection = client.GetConnection()
badidea
Posts: 2586
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by badidea »

I have added a 'noWait' parameter to the GetData() function. This reduces the reply time by nearly 2 second for replies that are less then 8kB. E.g. for communication with this device: TP-Link HS110.

Code: Select all

declare function GetData(byref pData as any ptr, byval maxSize as integer = 0, byval noWait as integer = 0) as integer

Code: Select all

function NetworkConnection.GetData(byref pData as any ptr, byval maxSize as integer, byval noWait as integer) as integer
	const CHUNK_SIZE = 1024 * 8
	static as ubyte chunk(CHUNK_SIZE - 1)
	if sock = SNC_SOCK_ERROR then lasterror = ERROR_SOCKET : return -1
	dim as integer dSize = 0 'totalizer
	if pData then deallocate pData : pData = 0
	do
		'receive in buffer: chuck
		dim as integer nBytes = recv(sock, @chunk(0), CHUNK_SIZE, 0)

		'nothing reveiced?
		if nBytes = 0 then
			'maxSize should be minSize?
			if dSize < maxSize then lasterror = ERROR_DISCONNECTED : return -1
			exit do 'we are done
		end if
		if nBytes < 0 then lasterror = ERROR_GET : return -1

		'add reveived chunk to data buffer
		pData = reallocate(pData, dSize + nBytes)
		dim as ubyte ptr pWrite = pData + dSize
		memcpy pWrite, @chunk(0), nBytes
		dSize += nBytes

		if noWait <> 0 then return dSize

		'more data received then was asked for?
		if maxSize > 0 andalso dSize >= maxSize then 
			lasterror = ERROR_NO_ERROR
			exit do
		end if

		'why this delay?
		dim as integer _timeout = 2000
		sleep 20
		while CanGet() <> 1 andalso _timeout > 0
			sleep 10 : _timeout -= 10 'note: sleep 10 can be longer on some systems
		wend
		if _timeout <= 0 then
			lasterror = ERROR_DISCONNECTED 'but no error returned!
			exit do
		end if
	loop
	return dSize
end function
[/s]
Edit: The 'noWait' parameter is not needed, one can set the maxSize parameter below the expected data size for the same result with fast response.

Is there a way to specify the timeout on NetworkClient(ServerIP, ServerPort)?
Here (on linux) it seems 75.3 seconds which is quite long.
Last edited by badidea on Oct 15, 2019 15:20, edited 2 times in total.
badidea
Posts: 2586
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by badidea »

I changed the constructor NetworkClient() to allow setting a timeout. Probably GNU/linux only.
Better error handling to be done still.

Code: Select all

' connection_maker(port)
constructor NetworkClient(address as string, byval port as ushort, timeout as integer)
	base()
	if port = SNC_SOCK_ERROR then 
		LastError = ERROR_SOCKET
		return
	end if
	' server address
	dim as hostent ptr he = gethostbyname(strptr(address))
	if (he = 0) then 
		LastError = ERROR_RESOLVE
		return
	end if
	addr.sin_family = AF_INET
	addr.sin_port = htons(port)
	addr.sin_addr = *cptr(in_addr ptr, he->h_addr_list[0])

	if timeout <= 0 then
		'original code, connect in blocking mode
		if connect(sock, cptr(sockaddr ptr, @addr), sizeof(sockaddr)) = SNC_SOCK_ERROR then
			LastError = ERROR_CONNECT
		end if
	else
		'non-blocking with timeout 
		const EINPROGRESS = 115
		LastError = ERROR_NO_ERROR
		'Set non-blocking (without error checking)
		dim as long flags = fcntl(sock, F_GETFL, NULL)
		'if flags < 0 then ...

		fcntl(sock, F_SETFL, flags or O_NONBLOCK)
		'fcntl(...) < 0 then ...
		dim as integer res = connect(sock, cptr(sockaddr ptr, @addr), sizeof(sockaddr))
		if (res < 0) and (errno <> EINPROGRESS) then
			LastError = ERROR_CONNECT
			exit constructor
		end if
		if res = 0 then
			'immediate success
		else
			'in progress
			dim as timeval tv
			FD_ZERO(@read_fd) '<----------- is this read_fd() ok to use?
			FD_SET_(sock, @read_fd)
			tv.tv_sec = timeout
			tv.tv_usec = 0
			if select_(sock + 1, NULL, @read_fd, NULL, @tv) = 1 then
				dim as long optval
				dim as socklen_t optlen = sizeof(optval)
				getsockopt(sock, SOL_SOCKET, SO_ERROR, @optval, @optlen)
				'if optval <> 0 then ...
			else
				'timeout
				lastError = ERROR_TIMEOUT
			end if
		end if
		
		'Set to blocking mode again...
		flags = fcntl(sock, F_GETFL, NULL)
		'if flags < 0 then ...
		fcntl(sock, F_SETFL, flags and not(O_NONBLOCK))
		'fcntl(...) < 0 then ...
	end if
end constructor

function NetworkClient.GetConnection() as NetworkConnection ptr
	return new NetworkConnection(sock)
end function
Other changes:

Code: Select all

#include "crt.bi"
...
enum NETWORK_ERROR
	ERROR_TIMEOUT
end enum
...
function GetNetworkErrorText(byval errorcode as NETWORK_ERROR) as string
	case ERROR_TIMEOUT      : return "error: timeout !"
...
declare constructor (address as string, byval port as ushort, byval timeout as integer = 0)
Use example:

Code: Select all

print "Waiting for connection..."
dim as networkClient client = NetworkClient(ServerIP, ServerPort, 3)

if client.GetLastError() <> ERROR_NO_ERROR then
	print GetNetworkErrorText(client.GetLastError())
	end(EXIT_NOK)
end if

dim as networkConnection ptr pConnection = client.GetConnection()
Dinosaur
Posts: 1478
Joined: Jul 24, 2005 1:13
Location: Hervey Bay (.au)

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by Dinosaur »

Hi All

Adding rfid tag sensing to my SmartPlug application, but have struck a problem when testing.
The problem resides in the Client program being unable to verify server availability.
The code below is as usual stripped of all error checking to simplify my trouble shooting. (I can hear Joshy already)

Everything works perfectly when my server program is running on the other cpu, but if there is a failure on the server side, then the client
quits the program with "Exit code 141" , No server connection.
Now I expected the following statement to fail, thus preventing the execution of the Connection->PutData(.TxPtr,40) statement.

Code: Select all

If Connection->CanPut() Then

But this test passes and therefore executes the putdata statement.
Does it pass because it was able to connect to the other cpu, even though the Server program is not running ?
When it does, the program ends with the Exit code 141, instead of providing this code in Status for error checking.

Code: Select all

     'file: Ethclient.bas
    #include once "snc.bi"
Type Ethernets
    RxPort          as  UShort  = 12346
    TxPort          as  UShort  = 12345
    TxIP            as  String  = "192.168.0.3"
    MyIP            as  String  = "192.168.0.4"
    TxPtr           as  ZString Ptr
    TxData          as  ZString * 40
End Type
Dim Shared Ethernet as Ethernets
Declare Sub EthTx()
Dim Shared Keys as String * 1
Dim Shared RfidStr as String
Dim Shared as Integer Status
Sub EthTx()
        With Ethernet
            If Left(.TxData,1) <> " " Then
                var Client = NetworkClient(.TxIP,.TxPort)
                var Connection = Client.GetConnection()
                .TxPtr = @.TxData
                If Connection > 0 Then
                    If Connection->CanPut() Then
                        Status = Connection->PutData(.TxPtr,40)
                        'I cant check Status because program ends with the above statement.
                    Endif
                    Delete Connection
                EndIf
            Endif
        End With
End Sub

    Print "Press [q] to Quit"
    Do
        Keys = InKey
        If Keys = "q" then exit do
        RfidStr += Keys
        If Keys = chr(&h0d) Then
            Ethernet.TxData = Ethernet.MyIP + "," + RfidStr
            EthTx
            RfidStr = ""
        EndIf
    Loop
    End

Am I missing something fundamental here ?

Regards
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: [S]imple [N]etwork [C]onnection win and lin 32/64-bit.

Post by D.J.Peters »

[S]imple [N]etwork [C]onnection wrote:

Code: Select all

' Sends any data
  ' NOTE: send a string like this: PutData(strptr(txt),len(txt)+1)
  ' returns number of bytes sended
  ' returns -1 as error signal
  declare function PutData(byval pData as any ptr, byval dataSize as integer) as integer

Code: Select all

function NetworkConnection.PutData(byval pData as any ptr,byval dataSize as integer) as integer
  if sock=SNC_SOCK_ERROR then lasterror=ERROR_SOCKET:return -1
  if pData=0    then lasterror=ERROR_PUT:return -1
  if dataSize<1 then lasterror=ERROR_PUT:return -1
  dim as integer size,dSize=dataSize
  ' as long as not all data has been send
  while (size<dataSize)
    dim as integer nBytes = send(sock,pData,dSize, 0)
    if nBytes<0 then lasterror=ERROR_PUT:return -1
    pData+=nBytes:dSize-=nBytes
    size+=nBytes
  wend
  return size
end function
My code looks good :-)
May be it's a kind of pointer on pointer if you use zstring * 40 !!!

Why not using real strings ?

Code: Select all

Status = Connection->PutData( .aString, len(.aString)+1)
does .TxPtr points to the chars "the right way" or does it points to another "pointer of zstring * 40" ?

Joshy
Post Reply