OPEN CONS FOR BINARY????

General FreeBASIC programming questions.
Post Reply
Julcar
Posts: 141
Joined: Oct 19, 2010 18:52
Contact:

OPEN CONS FOR BINARY????

Post by Julcar »

Hi, I wonder if is possible in any way to do something like OPEN CONS FOR BINARY because I need to read a binary file from STDIN and I am having problems with FOR INPUT due to this:
Many file format standards (e.g. PNG or GIF) include the SUB character in their headers to perform precisely this function.
Some programming languages (e.g. Visual Basic) will not read past a "soft" EOF when using the built-in text file reading primitives (INPUT, LINE INPUT etc.), and alternate methods must be adopted, e.g. opening the file in binary mode or using the File System Object to progress beyond it.
As I want to read images from STDIN (uploading files to a webserver) at the first CHR$(26) the INPUT$ function stops reading the STDIN and the string returns incomplete.

I am doing some research but maybe an experimented programmer can help me.

Regards
SARG
Posts: 1756
Joined: May 27, 2005 7:15
Location: FRANCE

Re: OPEN CONS FOR BINARY????

Post by SARG »

You could try with "open pipe"

Short extract from user's help :

Code: Select all

Open Pipe Opens an external process' standard input (stdin) or output (stdout) stream for file operations.

Syntax

Open Pipe shell_command For Input As [#]filenumber
Open Pipe shell_command For Output As [#]filenumber
Open Pipe shell_command For Binary access_type [#]filenumber
......
The Binary file mode opens the external process' stdin or stdout streams - depending on the access type specified (see description of the access_type parameter above) - for random-access reading or writing of arbitrarily sized and interpreted raw data. Simple data type values, like Byte and LongInt, and whole chunks of memory can be read from or written to the streams with binary-mode file operations like Get # and Put #.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: OPEN CONS FOR BINARY????

Post by caseih »

No, open pipe is not what he wants. That's for running a child command and reading it's output. In this case he just was wants to read from standard in.

Is it not possible to open cons in binary mode?

If not I suppose another method would be to use the c library functions to do the job.
Julcar
Posts: 141
Joined: Oct 19, 2010 18:52
Contact:

Re: OPEN CONS FOR BINARY????

Post by Julcar »

caseih wrote:No, open pipe is not what he wants. That's for running a child command and reading it's output. In this case he just was wants to read from standard in.

Is it not possible to open cons in binary mode?

If not I suppose another method would be to use the c library functions to do the job.
That gave me another idea, using the shell keyword to execute a command in the prompt to redirect the stdin content to a temp file, and then reading the file in binary, is possible that?

something like this

Code: Select all

shell(">tempfile.bin")
open for binary "tempfile.bin" as #1
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: OPEN CONS FOR BINARY????

Post by MrSwiss »

I don't see the reason to do shell, just open the file in "binary" mode ...
but, to be on the safe side, check if the file exists, before opening it:

Code: Select all

#Include "file.bi"  ' needed for FileExists()

Function LoadFile( _
    ByVal filen As String _
    ) As String

    Dim As String text = ""

    If FileExists(filen) Then       ' crash, if not existing!
        Var f = FreeFile
        If Open (filen For Binary Access Read As #f) = 0 Then
            If Lof(f) > 0 Then
                text = String(Lof(f), 0)
                Get #f,, text
            End If
            Close(f)
        Else
            Print "Unable to load: " + filen
        End If
    Else
        Print "File not existing: " + filen
    End If

    Return text
End Function
Julcar
Posts: 141
Joined: Oct 19, 2010 18:52
Contact:

Re: OPEN CONS FOR BINARY????

Post by Julcar »

MrSwiss wrote:I don't see the reason to do shell, just open the file in "binary" mode ...
but, to be on the safe side, check if the file exists, before opening it:

Code: Select all

#Include "file.bi"  ' needed for FileExists()

Function LoadFile( _
    ByVal filen As String _
    ) As String

    Dim As String text = ""

    If FileExists(filen) Then       ' crash, if not existing!
        Var f = FreeFile
        If Open (filen For Binary Access Read As #f) = 0 Then
            If Lof(f) > 0 Then
                text = String(Lof(f), 0)
                Get #f,, text
            End If
            Close(f)
        Else
            Print "Unable to load: " + filen
        End If
    Else
        Print "File not existing: " + filen
    End If

    Return text
End Function
???

The file I want to read really does not exists, because is sent through a remote connection to a webserver from a client browser, so the stream to send the file is indeed Standard Input, as the file is sent as binary, I cannot open it as plain text, because freebasic stops reading the stdin at chr(26)
Julcar
Posts: 141
Joined: Oct 19, 2010 18:52
Contact:

Re: OPEN CONS FOR BINARY????

Post by Julcar »

This works partially:

Code: Select all

shell("more > tempfile.bin")
the more command reads the stdin and redirect the output to the tempfile, the problem: more remains opened and must be killed from taskmgr, but is a good advance!
counting_pine
Site Admin
Posts: 6323
Joined: Jul 05, 2005 17:32
Location: Manchester, Lancs

Re: OPEN CONS FOR BINARY????

Post by counting_pine »

Probably if OPEN doesn't do what you want, it would be easier to use the CRT library and fread(). I took partial inspiration from the answer at https://stackoverflow.com/a/21036020/446106, and came up with this:

Code: Select all

#include "crt.bi"
const blocksize = 1048576

dim block as string = space(BLOCKSIZE)
dim content as string
dim readlen as integer

_setmode(_fileno(stdin), _O_BINARY)

do
  readlen = fread(@block[0], 1, BLOCKSIZE, stdin)
  if readlen = 0 then exit do

  content &= left(block, readlen)

  '' print hex of each byte
  for i as integer = 0 to readlen-1
    print hex(block[i], 2) & " ";
  next i
  print

  'if instr(left(block, readlen), "EOF") then exit do
loop

print len(content) & " bytes read"
(If you lower the block size and uncomment the "EOF" test at the end of the loop, you can test manually on the console for small text inputs.)
Julcar
Posts: 141
Joined: Oct 19, 2010 18:52
Contact:

Re: OPEN CONS FOR BINARY????

Post by Julcar »

Sounds good using crt, but I guess FreeBasic should have a native way to access to stdin in other mode than plain text, or maybe doing a workaround for the bug of CHR(26)
counting_pine
Site Admin
Posts: 6323
Joined: Jul 05, 2005 17:32
Location: Manchester, Lancs

Re: OPEN CONS FOR BINARY????

Post by counting_pine »

Would be ideal, yes, but still, it's Better Than SHELL.
It would take some investment of time to investigate into how OPEN works, and adapt/rewrite its STDIN support. I guess, best case, the fix might be as simple as using _setmode().
To be honest, apart from the need for block/memory management, the C method will probably more efficient and robust, since it doesn't need to support any legacy QB I/O features/quirks.
counting_pine
Site Admin
Posts: 6323
Joined: Jul 05, 2005 17:32
Location: Manchester, Lancs

Re: OPEN CONS FOR BINARY????

Post by counting_pine »

OK, the rtlib code for OPEN CONS is here: https://sourceforge.net/p/fbc/code/ci/m ... ons_open.c

We can actually see that it explicitly only supports INPUT, OUTPUT, and APPEND modes. And it does use the 'stdin' handle directly.
So, something very much like '_setmode(_fileno(stdin), _O_BINARY)' could potentially be put in the code.

So what would need to be done is:
- update the code to use binary for stdin (or stdout)
- adapt the code to allow BINARY mode for CONS.
- test on Windows/DOS/Linux

But it's not clear to me what the disadvantages of binary mode are over text mode. It's also not clear to me whether the code should favour stdin or stdout for BINARY, or whether support for both should be implemented. Also with support for graceful failures if the programmer tries to seek within the stream.

It's also not clear to me that code would be massively clearer in BASIC. There'd still be no one-line method for GET'ing stdin into a String. It would still probably have to be read in blocks.

But I'm still open to suggestions.
Julcar
Posts: 141
Joined: Oct 19, 2010 18:52
Contact:

Re: OPEN CONS FOR BINARY????

Post by Julcar »

counting_pine wrote:OK, the rtlib code for OPEN CONS is here: https://sourceforge.net/p/fbc/code/ci/m ... ons_open.c

We can actually see that it explicitly only supports INPUT, OUTPUT, and APPEND modes. And it does use the 'stdin' handle directly.
So, something very much like '_setmode(_fileno(stdin), _O_BINARY)' could potentially be put in the code.

So what would need to be done is:
- update the code to use binary for stdin (or stdout)
- adapt the code to allow BINARY mode for CONS.
- test on Windows/DOS/Linux

But it's not clear to me what the disadvantages of binary mode are over text mode. It's also not clear to me whether the code should favour stdin or stdout for BINARY, or whether support for both should be implemented. Also with support for graceful failures if the programmer tries to seek within the stream.

It's also not clear to me that code would be massively clearer in BASIC. There'd still be no one-line method for GET'ing stdin into a String. It would still probably have to be read in blocks.

But I'm still open to suggestions.
I consider more important the fact that functions like INPUT$ stops reading at CHR(26) and I guess should be more easy to create a function like INPUT$ but that reads the console in binary mode (would be a great improvement over other basic dialects)

In my project right now I am using this function to read the stdin in text plain

Code: Select all

FUNCTION ReadStdin$
  DIM numChars, postQuery$
  IF ENVIRON$("REQUEST_METHOD") = "POST" THEN
    numChars = VAL(ENVIRON$("CONTENT_LENGTH"))
    postQuery$ = INPUT$(numChars)
    ReadStdin$ = postQuery$
  ELSE
    ReadStdin$ = ""
  END IF
END FUNCTION

COMMON SHARED StdinQuery$
StdinQuery$ = ReadStdin$

FUNCTION Post$ (arg$)
  Post$ = UnEscape$(SplitVar$(StdinQuery$, arg$, CHR$(38)))
END FUNCTION
These code allows me to parse correctly the <form method="post">, as you can see there is no need to use OPEN CONS because INPUT$ does the same task, so a function called INPUTB$ should be enough.
Julcar
Posts: 141
Joined: Oct 19, 2010 18:52
Contact:

Re: OPEN CONS FOR BINARY????

Post by Julcar »

finally this code did the work

Code: Select all

#include "crt.bi"
dim as integer length = val(environ("content_length"))

dim block as string = space(length)
dim content as string
dim readlen as integer

_setmode(_fileno(stdin), _O_BINARY)

readlen = fread(@block[0], 1, length, stdin)

for i as integer = 0 to readlen-1
  content &= chr(block[i])
next

print content
Post Reply