Commandline parsing problem

General FreeBASIC programming questions.
Zippy
Posts: 1295
Joined: Feb 10, 2006 18:05

Commandline parsing problem

Postby Zippy » Mar 17, 2009 4:05

Edited to ADD: This turns out to NOT be a bug, see my 3rd post after this.

A leading double quote passed to an fb executable on Windows is dropped. This:

test.exe "C:\Documents and Settings\Zippy\"

is returned to command in code as this:

C:\Documents and Settings\Zippy\"

test code:

Code: Select all

dim as integer c
'
for c=1 to 5
    print using "# &";c,command(c)
next
'
sleep


and the return from this is wrong:

C:\"Documents and Settings"\Zippy\

- it should be one parameter, command(1).
Last edited by Zippy on Mar 18, 2009 5:24, edited 1 time in total.
MichaelW
Posts: 3500
Joined: May 16, 2006 22:34
Location: USA

Postby MichaelW » Mar 17, 2009 4:37

Using Version 0.21.0 (03-09-2009) this CL will preserve the double quotes:

\"C:\Program Files\FreeBASIC\My\_junk\text.txt\"

But it will still be split at the embedded space:
1 "C:\Program
2 Files\FreeBASIC\My\_junk\text.txt"

So it looks like there is more than one problem.
Zippy
Posts: 1295
Joined: Feb 10, 2006 18:05

Postby Zippy » Mar 17, 2009 22:40

I vaguely rem something about this, problems with quotes on the commandline, but can't find past refs. This is a show stopper, I can't figure out why it hasn't been a problem for me before this (the same behavior goes back to at least fb 0.15b).

Here's code to demo a few of the probs:

Code: Select all

'test for commandline argument/param bugs
' =====> creates "test.bat" in exepath
' compile as console-only
' DO NOT pass commandline params to this, it is self-propagating
'
#include once "windows.bi"
#include once "crt.bi"
'
dim as integer c,i,p,res
dim as string en,s(1 to 5),ts
dim as HANDLE ms
'
ts=command(0)
if right(ucase(ts),4)<>".EXE" then
    en=exepath & "\" & ts & ".exe"
else
    p=instrrev(ts,"\")
    if p>0 then
        ts=mid(ts,P+1)
        en=exepath & "\" & ts
    else
        print "Error.. bailing.."
        end
    end if
end if
'
'"C:\Documents and Settings\Zippy\"
s(1)="""C:\Documents and Settings\Zippy\"""

'C:\"Documents and Settings"\Zippy\
s(2)="C:\""Documents and Settings""\Zippy\"

'\"C:\Documents and Settings\Zippy\" 'MichaelW
s(3)="\""C:\Documents and Settings\Zippy\"""

'C:\Documents and Settings\Zippy\
s(4)="C:\Documents and Settings\Zippy\"
'
if command="" then
'
    open exepath & "\test.bat" for output as #1
    print #1,"@echo off"
    print #1,"echo IN test.bat: %1 %2 %3 %4 %5"
    for c=1 to 3
        print #1, using "echo # %#";c,c
    next
    close #1
    '
    for c=1 to 4
        color 15,0
        print "=================> Param:";c
        color 7,0
        shell(exepath & "\test.bat " & s(c))
        print
        shell(en & " " & s(c))
        print
    next
    '
    print "Sleeping to Exit.."
    sleep
    end
else
    print "IN fb code : ";
    for c=1 to 3
        print command(c);" ";
    next
    print
    for c=1 to 3
        print using "# &";c,command(c)
    next
    print using " CRT reports: # args including argv[0]";*__p___argc
'   
end if
'
'where is __fb_ctx.argv referenced here?
'http://fbc.svn.sourceforge.net/viewvc/fbc/trunk/FreeBASIC/src/rtlib/ +
'  libfb_sys_cmd.c?revision=4642&view=markup

I checked src code (see link at end of code), couldn't find/don't know where to look for __fb_ctx.argv
vdecampo
Posts: 2982
Joined: Aug 07, 2007 23:20
Location: Maryland, USA
Contact:

Postby vdecampo » Mar 17, 2009 23:29

Is there any way to retreive the complete command line and parse it manually?

-Vince
vdecampo
Posts: 2982
Joined: Aug 07, 2007 23:20
Location: Maryland, USA
Contact:

Postby vdecampo » Mar 17, 2009 23:35

You probably already know this but I'll just post how to get the raw command line with the WinAPI.

Code: Select all

#Include "windows.bi"

Dim a As ZString Ptr = GetCommandLine

Print *a

Sleep


-Vince
Zippy
Posts: 1295
Joined: Feb 10, 2006 18:05

Postby Zippy » Mar 18, 2009 0:38

vdecampo wrote:You probably already know this but I'll just post how to get the raw command line with the WinAPI.

Code: Select all

#Include "windows.bi"

Dim a As ZString Ptr = GetCommandLine

Print *a

Sleep


-Vince

Yes it does get the commandline; the complete exepath (if exe started outside exedir) + the exename + ".exe" if that was specified PLUS all of the commandline parameters/arguments to the executable all globbed together.

It confirms (the batch file results in test code) that the quote problem is inherent to fb. But it's then up to moi to parse it, unfortunately with all of the exepath exceptions, and the quotes, and the other special shell characters, and this doesn't resolve any other code without those parsed exceptions.

One of the tasks of the shell (Windows and Unix shells) is to parse commandline arguments. So that they are accesible via argv. So I don't have to reinvent freaking DOS.
Zippy
Posts: 1295
Joined: Feb 10, 2006 18:05

Postby Zippy » Mar 18, 2009 5:19

Sometimes I want to send Micro$oft et al to Venus. To the planet Venus, if that's not clear. To a hell it would be difficult to walk home from.

This \" thing is a feature not a bug. A Windows feature. A feature that applies only to commandline parameters fed to executables, not to/with batch files, not to/with the calling portion of commandlines.

So.. What it is.. is that a backslash is a backslash except when it precedes a quotation mark (I have a ca 1960 Random House dictionary that defines " as "a double quote") then it escapes the quote mark (the quote mark becomes a literal quote instead of a.. quote). but only in/with commandline parameters. To correct this you must escape the escape (use double backslashes).

So instead of this:

c:\>my.exe "C:\Documents and Settings\Zippy\"

one must use this:

c:\>my.exe "C:\Documents and Settings\Zippy\\"

which would almost make sense if it applied everywhere on the commandline.. But noooo.. See the following examples of what quoting is allowed where on the commandline.

E:\>e:\program files\temp\t6 "C:\Documents and Settings\Zippy\\"
'e:\program' is not recognized as an internal or external command,
operable program or batch file.

E:\>"e:\program files\temp\t6" "C:\Documents and Settings\Zippy\\"
IN fb code : C:\Documents and Settings\Zippy\

E:\>e:\"program files"\temp\t6 "C:\Documents and Settings\Zippy\\"
IN fb code : C:\Documents and Settings\Zippy\

E:\>e:\"program files\"temp\t6 "C:\Documents and Settings\Zippy\\"
IN fb code : C:\Documents and Settings\Zippy\

E:\>e:\"program files\temp\"t6 "C:\Documents and Settings\Zippy\\"
IN fb code : C:\Documents and Settings\Zippy\

E:\>e:\"program files\temp\t6" "C:\Documents and Settings\Zippy\\"
IN fb code : C:\Documents and Settings\Zippy\

Believe it or not there's an API function to parse the commandline.. But there's only a unicode version. There's ansi and unicode versions of GetCommandLine, but only a unicode version of the parser. So to use them together without invoking unicode is squirrely, like this:

Code: Select all

'get/parse commandline, Win API
' NOT UNICODE, but can't undef it.
#include once "windows.bi"
#include once "win/shellapi.bi"
'
#undef GetCommandLine
declare function GetCommandLine alias "GetCommandLineW" () as LPWSTR
#undef CommandLineToArgv
declare function CommandLineToArgv alias "CommandLineToArgvW" (byval as LPCWSTR, byval as integer ptr) as LPWSTR ptr
'
dim as integer c,nargs
dim as wstring ptr ptr szArglist
'
szArglist=CommandLineToArgv(GetCommandLine(),@nArgs)
'
print *GetCommandLine()
print
print "Argument Count: ";nargs
for c=0 to nargs-1
    print using "# = ";c;
    print *(szArglist[c])
next
'
localfree(szArglist)
sleep
end
marcov
Posts: 3010
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Postby marcov » Mar 18, 2009 8:19

Zippy wrote:Sometimes I want to send Micro$oft et al to Venus. To the planet Venus, if that's not clear. To a hell it would be difficult to walk home from.

This " thing is a feature not a bug. A Windows feature. A feature that applies only to commandline parameters fed to executables, not to/with batch files, not to/with the calling portion of commandlines.


Windows does non of the sort. Escaping chars is a C/Unix feature to compensate for defects in their string models which cannot contain all possible characters.

IOW it is probably the FB runtime (which is probably C based somewhere) that does this, not windows. Windows only knows quoting, and a backslash is a path separator.
marcov
Posts: 3010
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Postby marcov » Mar 18, 2009 8:55

Zippy wrote:One of the tasks of the shell (Windows and Unix shells) is to parse commandline arguments. So that they are accesible via argv. So I don't have to reinvent freaking DOS.


No, that is only on Unix, where the language runtime (libc) is tightly coupled to the system, and where the shell is more interpretive.

Windows is more open in this regard, the shell is more oriented towards batchjobs, and it gives the runtime straight access to the commandline. It's the runtime job to parse this within the bounderies set by the language definition and present it to the user.
fzabkar
Posts: 48
Joined: Sep 29, 2018 2:52

Re: Commandline parsing problem

Postby fzabkar » Aug 30, 2020 4:23

I'm seeing the same problem in FB ver 1.07.1.

I have posted some examples in this thread:

https://www.freebasic.net/forum/viewtopic.php?f=2&t=28783&p=275441#p275441

It appears to be a bug in FB, but nobody has reported it.

BTW, when I include "windows.bi", the compiler throws an error:

Code: Select all

Command executed:
"C:\Program Files\FreeBASIC\FreeBASIC-1.07.1-win32\fbc.exe" "C:\FZWork\FreeBASIC\test\windows_bi.bas"

Compiler output:
C:\Program Files\FreeBASIC\FreeBASIC-1.07.1-win32\inc\win\winbase.bi(2364) error 42: Variable not declared, _InterlockedCompareExchange64 in 'loop while InterlockedCompareExchange64(Destination, Old and Value, Old) <> Old'
C:\Program Files\FreeBASIC\FreeBASIC-1.07.1-win32\inc\win\winbase.bi(2372) error 9: Expected expression, found '_InterlockedCompareExchange64' in 'loop while InterlockedCompareExchange64(Destination, Old or Value, Old) <> Old'
C:\Program Files\FreeBASIC\FreeBASIC-1.07.1-win32\inc\win\winbase.bi(2380) error 9: Expected expression, found '_InterlockedCompareExchange64' in 'loop while InterlockedCompareExchange64(Destination, Old xor Value, Old) <> Old'
C:\Program Files\FreeBASIC\FreeBASIC-1.07.1-win32\inc\win\winbase.bi(2388) error 9: Expected expression, found '_InterlockedCompareExchange64' in 'loop while InterlockedCompareExchange64(Addend, Old + 1, Old) <> Old'
C:\Program Files\FreeBASIC\FreeBASIC-1.07.1-win32\inc\win\winbase.bi(2396) error 9: Expected expression, found '_InterlockedCompareExchange64' in 'loop while InterlockedCompareExchange64(Addend, Old - 1, Old) <> Old'
C:\Program Files\FreeBASIC\FreeBASIC-1.07.1-win32\inc\win\winbase.bi(2404) error 9: Expected expression, found '_InterlockedCompareExchange64' in 'loop while InterlockedCompareExchange64(Target, Value, Old) <> Old'
C:\Program Files\FreeBASIC\FreeBASIC-1.07.1-win32\inc\win\winbase.bi(2412) error 9: Expected expression, found '_InterlockedCompareExchange64' in 'loop while InterlockedCompareExchange64(Addend, Old + Value, Old) <> Old'

Results:
Compilation failed

System:
FBIde: 0.4.6
fbc:   FreeBASIC Compiler - Version 1.07.1 (2019-09-27), built for win32 (32bit)
OS:    Windows NT 6.2 (build 9200)
marcov
Posts: 3010
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Re: Commandline parsing problem

Postby marcov » Aug 30, 2020 10:28

Are you on a 32-bit windows? Or do you only generate 32-bit apps?
coderJeff
Site Admin
Posts: 3308
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Commandline parsing problem

Postby coderJeff » Aug 30, 2020 16:12

fzabkar wrote:It appears to be a bug in FB, but nobody has reported it.


Maybe is odd behaviour, but not only in FB. See:
https://docs.microsoft.com/en-us/cpp/cp ... -line-args
https://docs.microsoft.com/en-us/cpp/c- ... -arguments

A double quotation mark preceded by a backslash (\") is interpreted as a literal double quotation mark character (").
Backslashes are interpreted literally, unless they immediately precede a double quotation mark.
fzabkar
Posts: 48
Joined: Sep 29, 2018 2:52

Re: Commandline parsing problem

Postby fzabkar » Aug 30, 2020 19:18

I'm running under 32-bit Windows and generating 32-bit apps.

This behaviour is an annoyance. ISTM that anything on the command line should be interpreted as is, otherwise what is the point of the Command keyword?
marcov
Posts: 3010
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Re: Commandline parsing problem

Postby marcov » Aug 30, 2020 19:41

fzabkar wrote:I'm running under 32-bit Windows and generating 32-bit apps.

This behaviour is an annoyance. ISTM that anything on the command line should be interpreted as is, otherwise what is the point of the Command keyword?



InterlockedCompareExchange64 is afaik 64-bit only, so something in that header is either wrong or thinks you are on 64-bit.
fxm
Posts: 9938
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Commandline parsing problem

Postby fxm » Sep 03, 2020 6:37

coderJeff wrote:
fzabkar wrote:It appears to be a bug in FB, but nobody has reported it.


Maybe is odd behaviour, but not only in FB. See:
https://docs.microsoft.com/en-us/cpp/cp ... -line-args
https://docs.microsoft.com/en-us/cpp/c- ... -arguments

A double quotation mark preceded by a backslash (\") is interpreted as a literal double quotation mark character (").
Backslashes are interpreted literally, unless they immediately precede a double quotation mark.

In order to update the COMMAND documentation page:
Is this behavior specific to Windows or general (same behavior with Linux)?

Return to “General”

Who is online

Users browsing this forum: No registered users and 1 guest