Again, this matters if you want to measure a handful of cycles. Here we are talking about Millions.srvaldez wrote:the CPUID instruction is used to serialize the instruction execution
Slow opening and closing of files
Re: Slow opening and closing of files
-
- Posts: 52
- Joined: Jan 13, 2018 8:47
- Location: Germany
- Contact:
Re: Slow opening and closing of files
Awesome, finally I can use ATT-syntax. :-)
Sadly this doesn't fix my problem with the slow Open and Close commands.
Sadly this doesn't fix my problem with the slow Open and Close commands.
Re: Slow opening and closing of files
Hi srvaldez.
I cannot get your code to run 32 or 64 bit.
Win 10
It compiles OK with -asm att.
But running produces a blank console which fizzles out after a second or so.
I cannot get your code to run 32 or 64 bit.
Win 10
It compiles OK with -asm att.
But running produces a blank console which fizzles out after a second or so.
Re: Slow opening and closing of files
How do you define "slow"? For me, your code runs very fast:IchMagBier wrote:Sadly this doesn't fix my problem with the slow Open and Close commands.
Code: Select all
dim as uinteger mess_time
'Is it possible to use AT&T-syntax instead of Intel?
#macro start_mess()
asm
rdtsc
mov qword ptr[mess_time],rax
end asm
#endmacro
#macro end_mess(strin)
asm
mov rbx, qword ptr[mess_time]
rdtsc
sub rax, rbx
mov qword ptr[mess_time], rax
end asm
print strin;
print mess_time
#endmacro
print "go" : getkey
start_mess()
var of=freefile
end_mess("Freefile: ")
start_mess()
open "test.txt" for output as #of
end_mess("Open: ")
start_mess()
print #of,!"Test\n"
end_mess("Print: ")
start_mess()
close #of
end_mess("Close: ")
sleep
On my Core i5, your code needs 2e6 cycles, i.e. roughly one millisecond. Is that slow?
-
- Posts: 52
- Joined: Jan 13, 2018 8:47
- Location: Germany
- Contact:
Re: Slow opening and closing of files
Well, it takes 198255 cycles for opening and 180859 for closing a file. Compared to only 6300 for actually writing a string to it. And I wonder if there is a faster way?
I have a program which processes some data and writes its output to a file. But the file-output (only Open and Close) is 10 times slower than the actual data processing. The program gets called a lot and I need it to be as fast as possible, because other programs depend on its output.
I have a program which processes some data and writes its output to a file. But the file-output (only Open and Close) is 10 times slower than the actual data processing. The program gets called a lot and I need it to be as fast as possible, because other programs depend on its output.
Re: Slow opening and closing of files
@IchMagBier
this is only my guess, I am no expert, could it be that when you close the file that at point the string is written to the file and then the file is closed ?, because the test string is small and only is printed to the file once, it's likely that the printing is done when you invoke the file close command, that is, it prints the string to the file and then closes the file.
@dodicat
I don't have time to adjust the code to support both 32 and 64-bit, but perhaps it's the RDTSCP instruction, could you test this code?
this is only my guess, I am no expert, could it be that when you close the file that at point the string is written to the file and then the file is closed ?, because the test string is small and only is printed to the file once, it's likely that the printing is done when you invoke the file close command, that is, it prints the string to the file and then closes the file.
@dodicat
I don't have time to adjust the code to support both 32 and 64-bit, but perhaps it's the RDTSCP instruction, could you test this code?
Code: Select all
dim as ulong time_high, time_low
dim as ulongint start_time, end_time
function if_rdtscp() as long
dim as ulong reg_edx = 0
asm
"movl $0x80000001, %%eax \n" _
"CPUID\n" _
"mov %%edx, %0\n" _
: "=r" (reg_edx) _
:: "%rax", "%rcx", "%rdx"
end asm
function = (reg_edx shr 27) and 1
end function
'Is it possible to use AT&T-syntax instead of Intel?
'answer: yes.
#macro start_mess()
asm
"CPUID\n" _
"RDTSC\n" _
"mov %%edx, %0\n" _
"mov %%eax, %1\n\t": "=r" (time_high), "=r" (time_low):: "%rax", "%rbx", "%rcx", "%rdx"
end asm
start_time = (culngint(time_high) shl 32) or time_low
#endmacro
#macro end_mess(strin)
asm
"RDTSCP\n" _
"mov %%edx, %0\n" _
"mov %%eax, %1\n" _
"CPUID\n\t": "=r" (time_high), "=r" (time_low):: "%rax", "%rbx", "%rcx", "%rdx"
end asm
end_time = (culngint(time_high) shl 32) or time_low
print strin;
print end_time - start_time
#endmacro
if if_rdtscp() then
start_mess()
var of=freefile
end_mess("Freefile: ")
start_mess()
open "test.txt" for output as #of
end_mess("Open: ")
start_mess()
print #of,!"Test\n"
end_mess("Print: ")
start_mess()
close #of
end_mess("Close: ")
else
print "Your processor does not support the RDTSCP instruction"
end if
sleep
Re: Slow opening and closing of files
No.IchMagBier wrote:Well, it takes 198255 cycles for opening and 180859 for closing a file. Compared to only 6300 for actually writing a string to it. And I wonder if there is a faster way?
Change the program's logic. If one millisecond for opening and closing a file matters in your case, then something is wrong with the design. If other programs depend on its output, this probably means that you are passing data using files. That is a hack that you can occasionally use for testing something, but there are much, much faster ways for passing data. Consider WM_COPYDATA if you are on Windows.I have a program which processes some data and writes its output to a file. But the file-output (only Open and Close) is 10 times slower than the actual data processing. The program gets called a lot and I need it to be as fast as possible, because other programs depend on its output.
Re: Slow opening and closing of files
Filesystem metadata (directories etc) often requires seeks. And seeks are still in the millisecond magnitude. Are you trying HDD or SSD?IchMagBier wrote:Well, it takes 198255 cycles for opening and 180859 for closing a file. Compared to only 6300 for actually writing a string to it. And I wonder if there is a faster way?
- Disable antivirus and other programs that listen on filesystem changes. They are often slow.
- Directory processing is (at least on Windows) quite slower in large (thousands+ files) directories. Keep it small.
- You might be able to fiddle with the sharing options to trigger exclusive mode, which might be faster than sharing compatible.
Re: Slow opening and closing of files
In my dos version of RadioTelphone Tutor for Android. (using a dos emulator),
I notice, when I would open a topic, it took a while.
What my program does when opening a topic, is opening files, and inputting data.
But this is dos.
Maybe I will need to convert more string functions to assembly.
Maybe string comparisons in assembly.
I notice, when I would open a topic, it took a while.
What my program does when opening a topic, is opening files, and inputting data.
But this is dos.
Maybe I will need to convert more string functions to assembly.
Maybe string comparisons in assembly.
Re: Slow opening and closing of files
Hi srvaldez.srvaldez wrote:@IchMagBier
this is only my guess, I am no expert, could it be that when you close the file that at point the string is written to the file and then the file is closed ?, because the test string is small and only is printed to the file once, it's likely that the printing is done when you invoke the file close command, that is, it prints the string to the file and then closes the file.
@dodicat
I don't have time to adjust the code to support both 32 and 64-bit, but perhaps it's the RDTSCP instruction, could you test this code?
message to me:
Your processor does not support the RDTSCP instruction
That's Ebay for you.
-
- Posts: 606
- Joined: Nov 28, 2012 1:27
- Location: CA, USA moving to WA, USA
- Contact:
Re: Slow opening and closing of files
I like the idea that metadata AND directory size would be the major reasons for the slow open/close operations.
If the the filesystem is the way you want to manage your data - for whatever reason - perhaps a ramdisk is your answer.
It is trivial in Linux, not hard in Windows.
You can do a mount with very little metadata in Linux if that is that is the actual reason for the slow open/close operations.
david
If the the filesystem is the way you want to manage your data - for whatever reason - perhaps a ramdisk is your answer.
It is trivial in Linux, not hard in Windows.
You can do a mount with very little metadata in Linux if that is that is the actual reason for the slow open/close operations.
david
Re: Slow opening and closing of files
Really, if one millisecond for opening + closing matters, then something is wrong in the design. I just made a little test sending repeatedly a 700k string from one program to another, and that takes typically about one millisecond. No big mess with the file system, just plain old InterProcess Communication.