problem with binary read/write

New to FreeBASIC? Post your questions here.
sero
Posts: 59
Joined: Mar 06, 2018 13:26
Location: USA

problem with binary read/write

Post by sero »

Here's an example writing 3 bytes to a file. Then writing an additional 3 bytes erases the first 3 bytes. If I rewrite the first 3 then it erases the additional bytes. How can I avoid erasion? I tried to make a straight forward example since I'm not sure exactly how to explain what is going on.

Code: Select all

type udt_STUFF
  as byte X,Y,Z
end type

dim as long ff, size, position
dim as udt_STUFF STUFF( 1 to 2 )

STUFF( 1 ).X = 3
STUFF( 1 ).Y = 17
STUFF( 1 ).Z = 4
STUFF( 2 ).X = 94
STUFF( 2 ).Y = 87
STUFF( 2 ).Z = 61

ff = freefile
size = sizeof( udt_STUFF )
position = 1

open "stuff.bin" for binary access write as #ff
  put #ff, position, STUFF( 1 )
close #ff

open "stuff.bin" for binary access read as #ff
  get #ff, position, STUFF( 1 )
close #ff

PRINT "1ST 3 BYTES WRITTEN AND READ"
PRINT "STUFF( 1 ).X " & STUFF( 1 ).X
PRINT "STUFF( 1 ).Y " & STUFF( 1 ).Y
PRINT "STUFF( 1 ).Z " & STUFF( 1 ).Z
PRINT

position += size

open "stuff.bin" for binary access write as #ff
  put #ff, position, STUFF( 2 )
close #ff

open "stuff.bin" for binary access read as #ff
  get #ff, position, STUFF( 2 )
close #ff

PRINT "2ND 3 BYTES WRITTEN AND READ"
PRINT "STUFF( 2 ).X " & STUFF( 2 ).X
PRINT "STUFF( 2 ).Y " & STUFF( 2 ).Y
PRINT "STUFF( 2 ).Z " & STUFF( 2 ).Z
PRINT

position = 1

open "stuff.bin" for binary access read as #ff
  get #ff, position, STUFF( 1 )
close #ff

PRINT "1ST 3 BYTES READ ONLY"
PRINT "STUFF( 1 ).X " & STUFF( 1 ).X
PRINT "STUFF( 1 ).Y " & STUFF( 1 ).Y
PRINT "STUFF( 1 ).Z " & STUFF( 1 ).Z
PRINT
position = 1

open "stuff.bin" for binary access write as #ff
  put #ff, position, STUFF( 1 )
close #ff

position += size

open "stuff.bin" for binary access read as #ff
  get #ff, position, STUFF( 2 )
close #ff

PRINT "1ST 3 BYTES WRITTEN ONLY"
PRINT "2ND 3 BYTES READ ONLY"
PRINT "STUFF( 2 ).X " & STUFF( 2 ).X
PRINT "STUFF( 2 ).Y " & STUFF( 2 ).Y
PRINT "STUFF( 2 ).Z " & STUFF( 2 ).Z
PRINT

sleep
Last edited by sero on Apr 24, 2018 14:52, edited 1 time in total.
lizard
Posts: 440
Joined: Oct 17, 2017 11:35
Location: Germany

Re: problem with binary read/write

Post by lizard »

Manual:
https://www.freebasic.net/wiki/wikka.ph ... =KeyPgOpen

If you want to append at the end of a file use append. Simple write rewrites whole file.

open "stuff.bin" for binary access append as #ff
Last edited by lizard on Apr 24, 2018 4:36, edited 1 time in total.
paul doe
Moderator
Posts: 1730
Joined: Jul 25, 2017 17:22
Location: Argentina

Re: problem with binary read/write

Post by paul doe »

Just let FreeBasic do it's stuff ;)

Code: Select all

type myData
	as integer x
	as integer y
	as integer z
end type

dim as myData stuff( 0 to 2 )

with stuff( 0 )
	.x = 1
	.y = 2
	.z = 3
end with

with stuff( 1 )
	.x = 4
	.y = 5
	.z = 6
end with

with stuff( 2 )
	.x = 7
	.y = 8
	.z = 9
end with

dim as integer fileHandle = freeFile()
open "whatever.bin" for binary access write as fileHandle
	put #fileHandle, , stuff()
close( fileHandle )

? "Press a key to load data..."

sleep()

dim as myData loadedStuff( 0 to 2 )

fileHandle = freeFile()
open "whatever.bin" for binary access read as fileHandle
	get #fileHandle, , loadedStuff()
close( fileHandle )

for i as integer = 0 to 2
	? str( i ) & ": " & str( loadedStuff( i ).x ) & ", " & _
		str( loadedStuff( i ).y ) & ", " & str( loadedStuff( i ).z )
next

sleep()
If you want to update the 2nd record in the file, the code is this:

Code: Select all

with stuff( 1 )
	.x = 34324
	.y = 23
	.z = 787
end with

fileHandle = freeFile()
open "whatever.bin" for binary access write as fileHandle
	put #fileHandle, 1 * sizeOf( myData ) + 1, stuff( 1 )
close( fileHandle )
When you read the file again, you'll see it's updated. The general formula for computing the offset in bytes would be:

Code: Select all

offset = ( recordNumber - 1 ) * sizeOf( record ) + 1
Last edited by paul doe on Apr 24, 2018 4:52, edited 2 times in total.
lizard
Posts: 440
Joined: Oct 17, 2017 11:35
Location: Germany

Re: problem with binary read/write

Post by lizard »

lizard wrote:open "stuff.bin" for binary access append as #ff
No, false. it must be:

Open filename For Append As filenumber
paul doe
Moderator
Posts: 1730
Joined: Jul 25, 2017 17:22
Location: Argentina

Re: problem with binary read/write

Post by paul doe »

lizard wrote:Open filename For Append As filenumber
Note that sero's writting the file in binary mode, not sequential.
Last edited by paul doe on Apr 24, 2018 5:11, edited 1 time in total.
lizard
Posts: 440
Joined: Oct 17, 2017 11:35
Location: Germany

Re: problem with binary read/write

Post by lizard »

Yes if have seen. but he don't need to use binary, afaik.
badidea
Posts: 2586
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: problem with binary read/write

Post by badidea »

adele
Posts: 47
Joined: Jun 13, 2015 19:33

Re: problem with binary read/write

Post by adele »

Hi,

i came across a similar problem trying to write/append binary data to a file w/o fixed structure.
The thing nowhere mentioned, at least not at first glance, is: You have to write data you want to
append to a position beyond the end of file.

Observing this, you can use
"Open "stuff.bin" for binary access READ WRITE as #ff"

It is hard to explain, just look at the code and watch it running. Please mind the wrong file size
after the first write of the second record.

Code: Select all

#Include "file.bi"

type udt_STUFF
  as byte X,Y,Z
end type

dim as long ff, size, position
dim as udt_STUFF STUFF( 1 to 3 )

STUFF( 1 ).X = 3
STUFF( 1 ).Y = 17
STUFF( 1 ).Z = 4

STUFF( 2 ).X = 94
STUFF( 2 ).Y = 87
STUFF( 2 ).Z = 61

stuff(3).x=0
stuff(3).y=0
stuff(3).z=0

ff=FreeFile()
'zap file
Open "stuff.bin" for binary access write as #ff
'  put #ff, position, STUFF( 1 )
close #ff
Print "zapped, len=";filelen("stuff.bin")

Open "stuff.bin" for binary access Read write as #ff
  put #ff, position, STUFF( 1 )
close #ff
Print "stuff #1 written,len=";filelen("stuff.bin")

Open "stuff.bin" for binary access Read write As #ff
  Seek ff,Lof(ff)
  position=Seek(ff) ' causes misfunction w/0 "+" 
  put #ff, position, STUFF( 2 )
Close #ff
Print "stuff #2 written first time,len=";filelen("stuff.bin")

Open "stuff.bin" for binary access Read write As #ff
  position=position + 1 ' caused misfunction w/0 "+" 
  put #ff, position, STUFF( 2 )
Close #ff
Print "stuff #2 written 2nd time,len=";filelen("stuff.bin")
sleep
STOP
Good Luck

Adi
jj2007
Posts: 2326
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: problem with binary read/write

Post by jj2007 »

We have treated these issues in the Set EOF thread just a few weeks ago.
sero
Posts: 59
Joined: Mar 06, 2018 13:26
Location: USA

Re: problem with binary read/write

Post by sero »

lizard wrote:Open filename For Append As filenumber
@lizard - Works for me. By replacing "binary" with append it successfully adds data on to the end of the file. I'm still able to read binary as hoped.

This solution fixes half my problem. I want to alter existing data in the file but don't want to rewrite the entire file. Specifically because I won't have all the variables loaded into memory. Maybe the mistake is that I'm wanting to treat hard drive space like memory space. I don't see why I'm not able to open a file, go to a certain position, write data _replacing_ what is already there, without the whole rest of the file becoming zeros.
adele wrote:Open "stuff.bin" for binary access READ WRITE as #ff
@adele - I wouldn't have guessed but READ WRITE "puts" the data without erasion whereas WRITE only "puts" data with erasion to the rest of the data. Thank you for helping me see this.
paul doe wrote:Just let FreeBasic do it's stuff ;)
@paul doe - I can't get your solution to act any differently than my own problem. Below is your code showing that it is suffering from the same issue I'm having.

Code: Select all

type myData
   as integer x
   as integer y
   as integer z
end type

dim as myData stuff( 0 to 2 )

with stuff( 0 )
   .x = 1
   .y = 2
   .z = 3
end with

with stuff( 1 )
   .x = 4
   .y = 5
   .z = 6
end with

with stuff( 2 )
   .x = 7
   .y = 8
   .z = 9
end with

dim as integer fileHandle = freeFile()
open "whatever.bin" for binary access write as fileHandle
   put #fileHandle, , stuff()
close( fileHandle )

? "Press a key to load data..."

sleep()

dim as myData loadedStuff( 0 to 2 )

fileHandle = freeFile()
open "whatever.bin" for binary access read as fileHandle
   get #fileHandle, , loadedStuff()
close( fileHandle )

for i as integer = 0 to 2
   ? str( i ) & ": " & str( loadedStuff( i ).x ) & ", " & _
      str( loadedStuff( i ).y ) & ", " & str( loadedStuff( i ).z )
next


with stuff( 1 )
   .x = 34324
   .y = 23
   .z = 787
end with

fileHandle = freeFile()
open "whatever.bin" for binary access write as fileHandle
   put #fileHandle, 1 * sizeOf( myData ) + 1, stuff( 1 )
close( fileHandle )


fileHandle = freeFile()
open "whatever.bin" for binary access read as fileHandle
   get #fileHandle, , loadedStuff()
close( fileHandle )

print
print "now read in the data after having written to the the altered stuff( 1 )"
for i as integer = 0 to 2
   ? str( i ) & ": " & str( loadedStuff( i ).x ) & ", " & _
      str( loadedStuff( i ).y ) & ", " & str( loadedStuff( i ).z )
next

sleep()
@ badidea - Whoa, this works... Below is my code updated to take advantage of seek. What a wonderful tool

Code: Select all

type udt_STUFF
  as byte X,Y,Z
end type

dim as long ff, size, position
dim as udt_STUFF STUFF( 1 to 2 )

STUFF( 1 ).X = 3
STUFF( 1 ).Y = 17
STUFF( 1 ).Z = 4
STUFF( 2 ).X = 94
STUFF( 2 ).Y = 87
STUFF( 2 ).Z = 61

ff = freefile
size = sizeof( udt_STUFF )
position = 1

open "stuff.bin" for binary as #ff
  seek #ff, position
  put #ff,, STUFF( 1 )
close #ff

open "stuff.bin" for binary access read as #ff
  get #ff, position, STUFF( 1 )
close #ff

PRINT "1ST 3 BYTES WRITTEN USING SEEK AND READ WITH BINARY"
PRINT "STUFF( 1 ).X " & STUFF( 1 ).X
PRINT "STUFF( 1 ).Y " & STUFF( 1 ).Y
PRINT "STUFF( 1 ).Z " & STUFF( 1 ).Z
PRINT

position += size

open "stuff.bin" for binary as #ff
  seek #ff, position
  put #ff,, STUFF( 2 )
close #ff

open "stuff.bin" for binary access read as #ff
  get #ff, position, STUFF( 2 )
close #ff

PRINT "2ND 3 BYTES WRITTEN USING SEEK AND READ USING BINARY"
PRINT "HERE I PROVE SEEK DOES WHAT APPEND IS INTENDED TO DO"
PRINT "STUFF( 2 ).X " & STUFF( 2 ).X
PRINT "STUFF( 2 ).Y " & STUFF( 2 ).Y
PRINT "STUFF( 2 ).Z " & STUFF( 2 ).Z
PRINT

position = 1

open "stuff.bin" for binary access read as #ff
  get #ff, position, STUFF( 1 )
close #ff

PRINT "1ST 3 BYTES READ WITH BINARY"
PRINT "NOTICE THAT THEY HAVEN'T BEEN ERASED"
PRINT "STUFF( 1 ).X " & STUFF( 1 ).X
PRINT "STUFF( 1 ).Y " & STUFF( 1 ).Y
PRINT "STUFF( 1 ).Z " & STUFF( 1 ).Z
PRINT

STUFF( 1 ).X = 100
STUFF( 1 ).Y = 101
STUFF( 1 ).Z = 102

position = 1

open "stuff.bin" for binary as #ff
  seek #ff, position
  put #ff,, STUFF( 1 )
close #ff

open "stuff.bin" for binary access read as #ff
  get #ff, position, STUFF( 1 )
close #ff

PRINT "CHANGE STUFF( 1 ) DATA AND WRITE BACK IN USING SEEK"
PRINT "UPDATED 1ST 3 BYTES THEN READ USING BINARY"
PRINT "STUFF( 1 ).X " & STUFF( 1 ).X
PRINT "STUFF( 1 ).Y " & STUFF( 1 ).Y
PRINT "STUFF( 1 ).Z " & STUFF( 1 ).Z
PRINT

position += size

open "stuff.bin" for binary access read as #ff
  get #ff, position, STUFF( 2 )
close #ff

PRINT "2ND 3 BYTES READ ONLY USING BINARY"
PRINT "NOTICE THAT THEY HAVEN'T BEEN ERASED EITHER"
PRINT "STUFF( 2 ).X " & STUFF( 2 ).X
PRINT "STUFF( 2 ).Y " & STUFF( 2 ).Y
PRINT "STUFF( 2 ).Z " & STUFF( 2 ).Z
PRINT

PRINT "IT WORKS AS FAR AS I CAN TELL USING SEEK. APPEND WILL ONLY"
PRINT "ADD DATA ON TO THE END OF A FILE. BUT FOR ALTERING EXISTING"
PRINT "DATA IN A FILE I USED SEEK. THIS EXAMPLE PROVES IT WORKS."
PRINT "THIS EXAMPLE SHOWS THAT USING THIS SEEK METHOD INSTEAD OF"
PRINT "APPEND WORKS FOR WHAT APPEND IS INTENDED TO DO."
PRINT
PRINT "ALL THIS AND THE FILE REMAINED 6 BYTES AS HOPED"

sleep
jj2007 wrote:We have treated these issues in the Set EOF thread just a few weeks ago.
@jj2007 - Sorry I missed this when searching for existing problems/solutions. EOF wasn't and isn't what I was concerned about but maybe I should be. I will be sure to read through this whole other thread you recommend.
Last edited by sero on Apr 24, 2018 20:49, edited 1 time in total.
lizard
Posts: 440
Joined: Oct 17, 2017 11:35
Location: Germany

Re: problem with binary read/write

Post by lizard »

sero wrote:@lizard - Works for me.
Great. In practical use all the solutions are so quick one not even notices a difference. Mostly i prefer to keep things as easy as possible. Makes it easier from one self to read and for others. Easy is to hold the data in an array and then simply load and save the whole array each time.

Code: Select all

sub loadglyphs  ' open and read the file
   dim as integer filehandle
   filehandle = freefile
   open "glyphs.txt" for input as #filehandle
   for i = 0 to ubound(glyph)
     line input #filehandle, glyph(i) 
   next i         
   close #filehandle
end sub

sub saveglyphs  ' open and write the file
   dim as integer filehandle
   filehandle = freefile
   open "glyphs.txt" for output as #filehandle
   for i = 0 to ubound(glyph)
      print #filehandle, glyph(i)
   next i
   close #filehandle
end sub
That way one can save and load any data, whatever glyph() contains.
Last edited by lizard on Apr 24, 2018 15:35, edited 1 time in total.
paul doe
Moderator
Posts: 1730
Joined: Jul 25, 2017 17:22
Location: Argentina

Re: problem with binary read/write

Post by paul doe »

sero wrote:@paul doe - I can't get your solution to act any differently than my own problem. Below is your code showing that it is suffering from the same issue I'm having.
There, fixed (sorry, coded in a hurry):

Code: Select all

/'
	Disable field alignment when saving binary files, like this
'/
type myData field = 1
	as integer x
	as integer y
	as integer z
end type

dim as myData stuff( 0 to 2 )

with stuff( 0 )
	.x = 1
	.y = 2
	.z = 3
end with

with stuff( 1 )
	.x = 4
	.y = 5
	.z = 6
end with

with stuff( 2 )
	.x = 7
	.y = 8
	.z = 9
end with

dim as integer fileHandle = freeFile()
open "whatever.bin" for binary access write as fileHandle
	put #fileHandle, , stuff()
close( fileHandle )

? "Press a key to load data..."

with stuff( 1 )
	.x = 34324
	.y = 23
	.z = 787
end with

/'
	Update file like this
'/
fileHandle = freeFile()
open "whatever.bin" for binary access read write as fileHandle
	put #fileHandle, 1 * sizeOf( myData ) + 1, stuff( 1 ), sizeOf( myData )
close( fileHandle )

sleep()

dim as myData loadedStuff( 0 to 2 )

fileHandle = freeFile()
open "whatever.bin" for binary access read as fileHandle
	get #fileHandle, , loadedStuff()
close( fileHandle )

for i as integer = 0 to 2
	? str( i ) & ": " & str( loadedStuff( i ).x ) & ", " & _
		str( loadedStuff( i ).y ) & ", " & str( loadedStuff( i ).z )
next

sleep()
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: problem with binary read/write

Post by MrSwiss »

Sorry to butt in on this, but using Integer (in UDT), for file access (any method),
is actually *asking for troubles*, because SizeOf(Integer) varies, with the used
version of FBC (32/64 bit's), making the file unusable, when accessing with a
.exe compiled with the *other* FBC! In such a case, it is better, to use a fixed-
sized INT type, such as: Long ... (to avoid this sort of problem and, also saving
some memory, as well as file-size, in the process!)

Code: Select all

Type stuff  ' field allignment is 4
    As Long   x, y, z
End Type
sero
Posts: 59
Joined: Mar 06, 2018 13:26
Location: USA

Re: problem with binary read/write

Post by sero »

lizard wrote:Easy is to hold the data in an array and then simply load and save the whole array each time.
@lizard - For what I'm trying to achieve there would be a possibly large memory footprint with the user only ever accessing small amounts at a time. I'm going for small hits to the hard drive periodically and I'm hoping for _mostly_ unnoticeable lag. Maybe this isn't a preferred method but it does have the benefit of low memory footprint and a quick initial load time. My goal machine is an older computer with 5400rpm hard drive and low RAM.
paul doe wrote:type myData field = 1
@paul doe - I didn't know about field alignment until just now. I don't understand the purpose of this. The manual talks about padding which doesn't make any sense to have padding in a binary file. I changed the field = to 1, then 2, then 4 and the file size was not impacted at all. So what exactly does this do?
MrSwiss wrote:using Integer (in UDT), for file access (any method), is actually *asking for troubles*
@MrSwiss - good call
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: problem with binary read/write

Post by MrSwiss »

sero wrote:I didn't know about field alignment until just now. I don't understand the purpose of this.
Nowadays, its hardly worth, knowing about it (except you have to save every byte of RAM).
Padding is usually done, based on the *largest* numeric member, inside the UDT.

Code: Select all

Type MyNew
    As Longint  zx  ' 64 bit type
    As Short    w, h  ' 16 bit types (together 4 bytes)
    ' padding 4 bytes
    ' size of UDT = 16
End Type
Using Field = 1, reduces the size to 12 (but alignment suffers).
When using the type as array, speed reduction may happen (see: above).

(In old BASIC alignment was by default much larger, unless Field was specified.
afair: 128 bytes, default)
Post Reply