List Files and Dirs of Path with Unicode character support ?

Post your FreeBASIC source, examples, tips and tricks here. Please don’t post code without including an explanation.
Vinion
Posts: 19
Joined: Sep 08, 2022 6:27

List Files and Dirs of Path with Unicode character support ?

Post by Vinion »

Hello,

I am trying to list Files and Directories of a Path using unicode support
My code works OK with Files and Dirs with English Names but not for Greek.

Any help would be appreciated

Code: Select all

#include once "vbcompat.bi"

Dim dirPath as String = "C:\Users\nsiat\Desktop" 

Dim filename as WString*128
Dim f as Integer = freefile ' Get a free file

Open dirPath For Output As #f
Dim mask as Integer = fbDirectory Or fbHidden Or fbSystem Or fbArchive Or fbReadOnly

filename = Dir(dirPath & "\" & "*", mask)				

Dim counter as Integer = 0

while len(filename)>0
	print filename
	
	filename = Dir()
wend

' Close the file
Close #f
dodicat
Posts: 7976
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: List Files and Dirs of Path with Unicode character support ?

Post by dodicat »

You could try changing your codepage during the session.
shell("chcp 65001") at the top
Vinion
Posts: 19
Joined: Sep 08, 2022 6:27

Re: List Files and Dirs of Path with Unicode character support ?

Post by Vinion »

@dodicat I tried the shell("chcp 65001")
I see "Active code page: 65001" on the output but the results are the same...
I can read only English characters from filenames
Josep Roca
Posts: 564
Joined: Sep 27, 2016 18:20
Location: Valencia, Spain

Re: List Files and Dirs of Path with Unicode character support ?

Post by Josep Roca »

FreeBasic's DIR function does not work with Unicode. You may need to use the FindFirstFileW, FindNext and FindClose API functions.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: List Files and Dirs of Path with Unicode character support ?

Post by caseih »

If the code page is set to UTF-8 65001, would dir() return a utf-8 encoded byte string from FindFirstfile() now that Windows 10 supports UTF-8 and in fact makes a default encoding for working in byte strings? Wine's source code does do that, but it does not seem to give me UTF-8, just ????? question marks in place of the non-ansi characters.
Last edited by caseih on Sep 18, 2022 3:40, edited 3 times in total.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: List Files and Dirs of Path with Unicode character support ?

Post by caseih »

Vinion wrote: Sep 17, 2022 17:15 @dodicat I tried the shell("chcp 65001")
I see "Active code page: 65001" on the output but the results are the same...
I can read only English characters from filenames
Is this because the string returned is blank, or does it return UTF-8 encoded data when using the 65001 code page? Or just ??????? instead of characters?

It's a bit funny that by being the first OS to embrace unicode, long before other standards were created, Windows kind of hobbled itself to a unicode encoding that, in hindsight, is less than ideal, especially when the rest of the OSes who were late to the party took the path of least resistance to choose UTF-8, which turned out to be the best choice.
Vinion
Posts: 19
Joined: Sep 08, 2022 6:27

Re: List Files and Dirs of Path with Unicode character support ?

Post by Vinion »

Hello,

I get questionmarks in place of non Unicode characters.
My OS is Windows 10

This is the output I get even by using the shell("chcp 65001")

```
File 0 C:\Users\user\Desktop\1.png
File 1 C:\Users\user\Desktop\AnyDesk.exe
File 2 C:\Users\user\Desktop\desktop.ini
File 3 C:\Users\user\Desktop\drawing.svg
File 4 C:\Users\user\Desktop\JavaApplication5
File 5 C:\Users\user\Desktop\My Drive.lnk
File 6 C:\Users\user\Desktop\New Text Document.txt
File 7 C:\Users\user\Desktop\Shiroi.zip
File 8 C:\Users\user\Desktop\SPACECLM__lmgrd__SSQ.lic
File 9 C:\Users\user\Desktop\Test.txt
File 10 C:\Users\user\Desktop\????µ?
File 11 C:\Users\user\Desktop\?????aµµat?sµ?? µe FreeBasic.pdf
```
dodicat
Posts: 7976
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: List Files and Dirs of Path with Unicode character support ?

Post by dodicat »

Just try shell then.

Code: Select all

 

var path="C:\Users\nsiat\Desktop"  
shell("dir  /b " + path)
sleep
 
Vinion
Posts: 19
Joined: Sep 08, 2022 6:27

Re: List Files and Dirs of Path with Unicode character support ?

Post by Vinion »

This actually works!
But how I can get the response from the shell("dir /b " + path) ?
dodicat wrote: Sep 18, 2022 9:33 Just try shell then.

Code: Select all

 

var path="C:\Users\nsiat\Desktop"  
shell("dir  /b " + path)
sleep
 
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: List Files and Dirs of Path with Unicode character support ?

Post by D.J.Peters »

You can read the output of a schell command with a PIPE

Joshy

Code: Select all

var UserPath = environ("USERPROFILE")
Open Pipe "dir /b " & UserPath For Input As #1
var aLine = ""
while not EOF(1)
  line input #1, aLine
  print aLine
wend
close #1
sleep
dodicat
Posts: 7976
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: List Files and Dirs of Path with Unicode character support ?

Post by dodicat »

If you want a hard copy of dir then try this

Code: Select all

#include "file.bi"
shell("chcp 65001")
cls


var path="C:\Users\computer\Desktop\fb\dirPath.txt" 
var folder="C:\Users\computer\Desktop\fb"
Function pipeout(Byval s As String="")  As String
    Var f=Freefile
    Dim As String tmp
    Open Pipe s For Input As #f 
    s=""
    Do Until Eof(f)
        Line Input #f,tmp
        s+=tmp+Chr(10)
    Loop
    Close #f
    Return s
End Function

 Open path For Output as #1
 var s=pipeout("dir  /b " +folder)
  Print #1,s
  Close #1
 print fileexists(path)
shell "notepad "+path
kill path
print fileexists(path)
sleep
 
be careful of your paths
path=the text file to write into
folder = the folder you want to dir.
Also note, for my convenience, I have killed the file at the end.
You may wish to keep it.
Vinion
Posts: 19
Joined: Sep 08, 2022 6:27

Re: List Files and Dirs of Path with Unicode character support ?

Post by Vinion »

Thank you for your replies. Still have not made it work...

The code I am using, after you suggested to use a pipe, is the following.
The problem occures while reading the pipe data

Code: Select all

Dim UserPath as String = "C:\users\nsiat\Desktop"
Open Pipe "dir /b " & UserPath For Input encoding "utf8" As #1

	Dim aLine as WString*64 = ""
	
	while not EOF(1)
	  line input #1, aLine
	  print aLine
	wend
close #1
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: List Files and Dirs of Path with Unicode character support ?

Post by D.J.Peters »

this works here but I don't have UTF8 file names on my gernan system.
use a string size of 1024 for filename or path 64 is often to small !

Joshy

Code: Select all

Dim UserPath as String = environ("USERPROFILE") & "\Desktop"
Open Pipe "dir /b " & UserPath For Input encoding "utf8" As #1
dim aLine as WString*64 = "" ' <-- make it bigger 
while not EOF(1)
  line input #1, aLine
  print aLine
wend
close #1
sleep
dodicat
Posts: 7976
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: List Files and Dirs of Path with Unicode character support ?

Post by dodicat »

You can try it all with shell.

Code: Select all



shell"chcp 65001"


var path="C:\Users\computer\Desktop\fb\dirPath.txt" 
var folder="C:\Users\computer\Desktop\fb"


shell "dir /b "+ folder +"> "+path

shell "notepad "+path

print iif(kill(path)=0,"The file has been deleted","Please felete the file manually")

'OR
shell "dir /b "+ folder 
sleep
 
Remember to close notepad to see the dir on the console.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: List Files and Dirs of Path with Unicode character support ?

Post by caseih »

I don't think FB lets you get anything back from Shell other than the process's exit value. And even if it did, you'd still have to deal with byte encodings to get things into a wide string. Since your goal is to get a list of file names in wide strings, you will have to use the win32 API directly: FindFirstFileW and FindNextFileW.

If under the hood FindFirstFileA and FindNextFileA called the wide versions and then converted to to multi-byte, of if you were on on Mac and Linux of course you could use standard dir() function, but you must include utf_conv.bi in your source and then call UTFToWChar() to convert the result from dir() to a WSTRING.

By the way on Windows 10 if I turn on the experimental option to provide UTF-8 support for all apps (which sets 65001 to be the system-wide default code page for non-unicode apps), dir() does appear to return multibyte characters, but they are just mojibake for some reason. Not sure why. According to MSDN, when UTF-8 is set as the default, all ANSI calls will actually call the wide versions under the hood, and then convert the result to multibyte according to the code page. That should mean dir() can return UTF-8 if you hold your nose just right. https://learn.microsoft.com/en-us/windo ... -code-page
Last edited by caseih on Sep 18, 2022 15:04, edited 1 time in total.
Post Reply