CSV to fixlength data files

General FreeBASIC programming questions.
badidea
Posts: 2586
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: CSV to fixlength data files

Post by badidea »

Gablea wrote:I have uploaded the full project to my server you can download it herehttp://www.algpos.co.uk/KPoS.7z

in the pos.ini file please change the PathToServerDatabase=c:\FromServer to the path of where you have saved the Folder DOSData for example

mine might be PathToServerDatabase=c:\DosDrive\KPoS\DOSData

this simpulates the Network shared folder that the PoS application would download from

also I am using a LCD display on Com1 for the user display you may want to reroute the output to the screen somehow
Be aware that there is an address and phone number in one of the files.

If you ever switch to linux, you need to pay attention to case sensitive file names.
Gablea
Posts: 1104
Joined: Apr 06, 2010 0:05
Location: Northampton, United Kingdom
Contact:

Re: CSV to fixlength data files

Post by Gablea »

in the r_header file yes that is the recipit header I have repalced that with a Example file now so it does not hold any information

and I was aware of the case sensativity of linux that is why i am really trying to get in the habbit of keeping everything lower case

to sign on you would need to use user ID 1105 and user password 1105 if it works you would see the "NEXT CUSTOMER PLEASE" screen
badidea
Posts: 2586
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: CSV to fixlength data files

Post by badidea »

Ok, I'll have a further look tomorrow. There is also some mixed use of tabs and spaces which makes the code harder to read here.
badidea
Posts: 2586
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: CSV to fixlength data files

Post by badidea »

This product line is a problem:
5000432013000,"L&B "0",0006,1,10,S,719,A,0,-,0,-,0,0,0,-,0,-,0,0,0,0
A " too much
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: CSV to fixlength data files

Post by MrSwiss »

Loading everything, before comparison, makes the "search" unbearably slow.
(that referres, to the current code ...)

Below a "simple" and therefore "fast" search Function returning simply found/
NOT found (all other processing, if it returns TRUE only | error otherwise):

Code: Select all

' since, Freefile returns a Long (parameter = Long)
Function SearchProduct( _
    ByVal ProductFileNumber As Long, _
    ByVal BarcodeNumber     As String _
    ) As Boolean    ' TRUE = found | FALSE = NOT found
    ' assuming ProductFileNumber is already "open", by caller ...
    While Not EOF(ProductFileNumber)
        Input #ProductFileNumber, Product_barcodenumber ' that's all we need for comparison!
    	 If Trim(BarcodeNumber) = Trim(Product_barcodenumber) Then Return TRUE
    Wend
    Return FALSE
End Function
Alternative: returning line number (found) or null (NOT found):

Code: Select all

' since, Freefile returns a Long (parameter = Long)
Function SearchProduct( _
    ByVal ProductFileNumber As Long, _
    ByVal BarcodeNumber     As String _
    ) As ULong      ' > 0 = line number | 0 = NOT found
    ' assuming ProductFileNumber is already "open"
    Dim As ULong cnt = 1
    While Not EOF(ProductFileNumber)
        Input #ProductFileNumber, Product_barcodenumber ' that's all we need for comparison!
    	 If Trim(BarcodeNumber) = Trim(Product_barcodenumber) Then Return cnt
    	 cnt += 1    ' increment line counter
    Wend
    Return 0        ' indicate error "NOT found"
End Function
That's all, as far as I'm wanting to get involved, in your code.
(I'm not about to lie, saying "I've no time" or similar, I've no interest, to get any
further involved.)
badidea
Posts: 2586
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: CSV to fixlength data files

Post by badidea »

Except for product "L&B "0" the file seems ok.

I just call FindProductinDatabase("9774024302051") form KPoS.bas and skip / ignore all cashier stuff. And removed all sleep calls. Works fine with corrected product above. Some other findings:

1) FindProductinDatabase does Print Product_barcodenumber, Product_posdescription for each line. This slows does finding the last product from 0.03 seconds to 0.34 seconds here.

2) CloseAllFiles is called nearly everywhere as if you have no idea which file is open when. When compiling with -w all -exx this throws runtime errors here when files were already closed.

1) The overall code formatting (and structure) is chaotic.
Gablea
Posts: 1104
Joined: Apr 06, 2010 0:05
Location: Northampton, United Kingdom
Contact:

Re: CSV to fixlength data files

Post by Gablea »

badidea wrote:Except for product "L&B "0" the file seems ok.

I just call FindProductinDatabase("9774024302051") form KPoS.bas and skip / ignore all cashier stuff. And removed all sleep calls. Works fine with corrected product above. Some other findings:

1) FindProductinDatabase does Print Product_barcodenumber, Product_posdescription for each line. This slows does finding the last product from 0.03 seconds to 0.34 seconds here.

2) CloseAllFiles is called nearly everywhere as if you have no idea which file is open when. When compiling with -w all -exx this throws runtime errors here when files were already closed.

1) The overall code formatting (and structure) is chaotic.
so do you think it could be the product "L&B "0" that is making it no find 50201600?

i have updated the code now just to Open the database files when the system starts up and to close then all once it exits the program (will write a update function soon that would close the files update them and then open them again
badidea
Posts: 2586
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: CSV to fixlength data files

Post by badidea »

Gablea wrote:so do you think it could be the product "L&B "0" that is making it no find 50201600?
Yep, 50201600 is at line 3588 and "L&B "0" is at line 2334. All after line 2334 will fail. Just call FindProductinDatabase() with print barcode and product name for each line and it will show barcodes "0" after "L&B "0". Make sure the " character cannot be in any string you write to csv-file to prevent this (not sure about single quote ' ).
jj2007
Posts: 2326
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: CSV to fixlength data files

Post by jj2007 »

badidea wrote:Make sure the " character cannot be in any string you write to csv-file to prevent this (not sure about single quote ' ).
Single quote should not be a problem, but caution with single double quotes:

Code: Select all

OK  00001019,STAMP 2ND CLASS,0010,0,-,S,050,A,0,-,0,-,0,0,0,-,0,-,0,0,0,0
OK  00001019,"STAMP 2ND CLASS",0010,0,-,S,050,A,0,-,0,-,0,0,0,-,0,-,0,0,0,0
OK  00001019,"STAMP "2ND" CLASS",0010,0,-,S,050,A,0,-,0,-,0,0,0,-,0,-,0,0,0,0
BAD 00001019,"STAMP "2ND CLASS",0010,0,-,S,050,A,0,-,0,-,0,0,0,-,0,-,0,0,0,0
BAD 00001019,"STAMP 2ND" CLASS",0010,0,-,S,050,A,0,-,0,-,0,0,0,-,0,-,0,0,0,0
Note that M$ Excel also has problems with such files. I know official statistical databases that have such lousy formatting errors, including linefeed characters inside a cell.
Gablea
Posts: 1104
Joined: Apr 06, 2010 0:05
Location: Northampton, United Kingdom
Contact:

Re: CSV to fixlength data files

Post by Gablea »

OK sounds like I would also need to do some update to the VB.net code that generates the csv file. that should not be a problem as I have the source code for that program.

thanks for all the input everyone this has help so much to see what i was doing wong.
Post Reply