badidea wrote:Maybe I should do the tests with a external USB mechanical disk.
I would be interested to see the timings.
badidea wrote:Maybe I should do the tests with a external USB mechanical disk.
This is roughly what I also measure on my machine. The file is in the cache, i.e. in RAM, and that's probably faster than the SSD. But I am not an expert for this hardware stuff...badidea wrote:I think that the OS is fooling me (or a mistake). Loading the whole file in 1 binary get() takes 0.14 seconds. This is ~10 x faster then the SSD read speed
Code: Select all
sudo sh -c "sync; echo 1 > /proc/sys/vm/drop_caches"
Code: Select all
function loadProductDB(inFileName as string, byref pData as ubyte ptr) as integer
dim as integer i, inFile = freefile()
dim as ulong fileSize
Dim result As Integer = Open(inFileName For Binary, Access Read, As #inFile)
fileSize = lof(inFile)
pData = allocate(fileSize)
if pData = 0 then return -1
print pData
get #inFile, , *pData, fileSize 'note FB is weird, dereference pointer first
Close #inFile
return fileSize
end function
dim as double t, dt
dim as ubyte ptr pData
dim as ulong fileSize
sleep 1,1
t = timer
fileSize = loadProductDB("products2.csv", pData)
print fileSize; " bytes,"; int(fileSize / (1024*1024)); " MB"
if pData <> 0 then deallocate(pData) else print "free error"
dt = timer - t
print dt
print int((fileSize / (1024*1024)) / dt); " MB/s"
I googled a bit and found this here:badidea wrote:... which corresponds to 460 MB/s. This could very well be the max. SSD read speed.
Even if you're already rocking a fast SSD (one of the best upgrades you can make), you can still improve your computer's performance by adding more memory and turning it into a RAM disk, which can be as much as 70 times faster than a regular hard drive or 20 times faster than an SSD.
Btw what is "extremely slow"? How many seconds, typically?Gablea wrote:I have a data file that has over 7,000 lines in it and it is extreamly slow at reading data from the product.dat file (this is a csv file)
jj2007 wrote:Btw what is "extremely slow"? How many seconds, typically?Gablea wrote:I have a data file that has over 7,000 lines in it and it is extreamly slow at reading data from the product.dat file (this is a csv file)
Code: Select all
Public Sub FindProductinDatabase(ByVal BarcodeNumber As String)
If RecipitClear = 1 Then
CreateRecipitHeadder
CreateRecipitFooter
'ResetPoSdataForNewSale
End If
Dim ProductFound As Integer = 0
CloseAllFiles
Open PathToProductDatabase For Input As #ProductFileNumber
Do until EOF(ProductFileNumber)
Input #ProductFileNumber, Product_barcodenumber, Product_posdescription, Product_salelocation, Product_agerestricted, Product_agelimit, Product_pricetype, Product_retailprice, Product_vatcode, Product_print_guarantee_message, Product_print_guarantee_code, Product_DisplayMessage, Product_DisplayMessage_code, Product_sendtoppr, Product_requestserial, Product_ItemNotAllowed, Product_ItemNotAllowedReason, Product_RestrictProductQty, Product_RestrictProductAllowed, Product_DiscountNotAllowed, Product_RefundNotAllowed, Product_AskQtyBeforeSelling, Product_HelhtlyStartVoucherOK
If Trim(BarcodeNumber) = Trim(Product_barcodenumber) Then
ProductFound = 1
CloseAllFiles
Exit Do
End If
Loop
CloseAllFiles
Select Case ProductFound
Case 0 ' Nothing Found
CloseAllFiles
ItemNotFound(BarcodeNumber)
Case 1 'Item was found in data file
Select Case Product_ItemNotAllowed
Case 1 'Item is NOT allowed to be sold
ItemNotallowedScreen
Case 0 'Item is allowed to be sold
CloseAllFiles
Dim LocalProduct As String = Product_posdescription
Product_posdescription = strReplace(LocalProduct, "''", """")
SaleLocationNumber = Trim(Product_salelocation)
PriceType = Trim(Product_pricetype)
PriceCheck = Val(Trim(Product_retailprice))
PriceCheck =(PriceCheck * 100)
Select Case Product_agerestricted
Case 0
'PriceTypeCheck 'Normal Item
Case 1
'AgeLimitDisplay(Val(Trim(Product_agelimit))) 'Age limited Item
End Select
Exit Sub
End Select
End Select
Exit Sub
End Sub
Sounds rather like a bug than like a bottleneck.Gablea wrote:So far it has been up to a minute to "look" for a item but then errors out and says Item not found BUT when i search for it manually inside the actual data file i can find it.
Code: Select all
Do until EOF(ProductFileNumber)
Input #ProductFileNumber, Product_....
If Trim(BarcodeNumber) = Trim(Product_barcodenumber) Then
ProductFound = 1
CloseAllFiles
Exit Do
End If
Loop
I have tested the "updated source code" above, the problem is it throws many errors:Gablea wrote:If you like kJ I can upload the source code
Code: Select all
... using \AllBasics\FreeBasic\fbc.exe -t 8000 -s console "\AllBasics\FreeBasic\tmp\TmpFile.bas"
____________________
\AllBasics\FreeBasic\tmp\TmpFile.bas(3) error 41: Variable not declared, RecipitClear
\AllBasics\FreeBasic\tmp\TmpFile.bas(3) error 3: Expected End-of-Line, found 'RecipitClear'
\AllBasics\FreeBasic\tmp\TmpFile.bas(4) error 41: Variable not declared, CreateRecipitHeadder
\AllBasics\FreeBasic\tmp\TmpFile.bas(5) error 41: Variable not declared, CreateRecipitFooter
\AllBasics\FreeBasic\tmp\TmpFile.bas(7) error 124: Expected 'END SUB', found 'End'
\AllBasics\FreeBasic\tmp\TmpFile.bas(11) error 41: Variable not declared, CloseAllFiles
\AllBasics\FreeBasic\tmp\TmpFile.bas(14) error 41: Variable not declared, PathToProductDatabase
\AllBasics\FreeBasic\tmp\TmpFile.bas(15) error 41: Variable not declared, ProductFileNumber
\AllBasics\FreeBasic\tmp\TmpFile.bas(16) error 9: Expected expression, found 'ProductFileNumber'
\AllBasics\FreeBasic\tmp\TmpFile.bas(18) error 9: Expected expression, found 'Product_barcodenumber'
\AllBasics\FreeBasic\tmp\TmpFile.bas(18) error 132: Too many errors, exiting
Users browsing this forum: No registered users and 1 guest