wxFBE, editor for both Windows and Linux

User projects written in or related to FreeBASIC.
Post Reply
VANYA
Posts: 1834
Joined: Oct 24, 2010 15:16
Location: Ярославль
Contact:

Re: wxFBE, editor for both Windows and Linux

Post by VANYA »

MOD wrote:Would you test it on a russian linux version?
I'm not good at Linux :) I tried but the constant error is missing a library, then the other ...

I'm a bad Linux-tester
MOD
Posts: 555
Joined: Jun 11, 2009 20:15

Re: wxFBE, editor for both Windows and Linux

Post by MOD »

Did you read the readme? Or do you know some linux users which could test it. I can't fix this issue for windows when I'm not sure, it will work on linux too.

I would test it but I don't speak russian and can't read any Cyrillic.
VANYA
Posts: 1834
Joined: Oct 24, 2010 15:16
Location: Ярославль
Contact:

Re: wxFBE, editor for both Windows and Linux

Post by VANYA »

MOD wrote:Did you read the readme? Or do you know some linux users which could test it. I can't fix this issue for windows when I'm not sure, it will work on linux too.

I would test it but I don't speak russian and can't read any Cyrillic.
I was able to run on Linux, but wxFBE it does not work with Cyrillic.
MOD
Posts: 555
Joined: Jun 11, 2009 20:15

Re: wxFBE, editor for both Windows and Linux

Post by MOD »

That's why I can't simply use WinAPI. If you can manage to get a correct string with just FB (or wx-c) in console, you could post it here. I really don't know how to handle this issue to get it work on all platforms.
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: wxFBE, editor for both Windows and Linux

Post by D.J.Peters »

Hello MOD
do you use the 0.24 fbc version from Mon Aug 20, 2012 to build wxFBE ?

Joshy
TJF
Posts: 3809
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: wxFBE, editor for both Windows and Linux

Post by TJF »

MOD wrote:That's why I can't simply use WinAPI.
That's no good idea for cross-platform code : )
MOD wrote:If you can manage to get a correct string with just FB (or wx-c) in console, you could post it here. I really don't know how to handle this issue to get it work on all platforms.
FB can handle it. But you need consoles on each platform that supports the character encoding used in FB.

For good I18N support in FB it's wise to use libintl. This works with UTF-8 encoding. UTF-8 is standard on most *NIX systems but it's hard to find a wondiws console that supports it.

Anyway, why do you need a solution for the console? I'd expect that you need a solution for the GUI (only).
marcov
Posts: 3452
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: wxFBE, editor for both Windows and Linux

Post by marcov »

TJF wrote: For good I18N support in FB it's wise to use libintl. This works with UTF-8 encoding. UTF-8 is standard on most *NIX systems but it's hard to find a wondiws console that supports it.
Don't you mean ncursesw? libintl doesn't handle platform specific output at all, but basic encoding conversions on *nix systems??!?


For more batch oriented jobs, one can set the Windows Console to utf8 using the command


CHCP 65001
MOD
Posts: 555
Joined: Jun 11, 2009 20:15

Re: wxFBE, editor for both Windows and Linux

Post by MOD »

do you use the 0.24 fbc version from Mon Aug 20, 2012 to build wxFBE ?
fbc -version wrote:FreeBASIC Compiler - Version 0.24.0 (08-19-2012)
That's no good idea for cross-platform code : )
Exactly. I don't need a console version but it would help. I tested with our german Umlaute and it works with just FB commands and with just wx-c, and due to the lack of russian skills, I can't test it.
It have to work with pure FB and wx-c, I won't use any additional library for this! And even if I would, I have to convert any string between FBString, wxString and that additional String-type and this is the next big issue with using another lib.
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: wxFBE, editor for both Windows and Linux

Post by D.J.Peters »

Hello MOD
i use the last 0.24 version from here http://www.freebasic.net/forum/viewtopi ... =1&t=20252
and if I rebuild wxFBE on windows I run in to trouble with the encode UTF statement from file open command !

By the way good job so far.

Joshy
TJF
Posts: 3809
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: wxFBE, editor for both Windows and Linux

Post by TJF »

marcov wrote:
TJF wrote: For good I18N support in FB it's wise to use libintl. This works with UTF-8 encoding. UTF-8 is standard on most *NIX systems but it's hard to find a wondiws console that supports it.
Don't you mean ncursesw? libintl doesn't handle platform specific output at all, but basic encoding conversions on *nix systems??!?
We're talking about russian language and cyrillic letters for the app mentioned in the subject.

I mean libintl. It supports I18N (= internationalisation). Have a look at Internationalisation (I18N) in FB code.

MOD wrote:It have to work with pure FB and wx-c, I won't use any additional library for this! And even if I would, I have to convert any string between FBString, wxString and that additional String-type and this is the next big issue with using another lib.
This is the point. I recomment not to use different character encoding! Go with a system that uses the same encoding in source code and in the GUI (console doesn't matter, it's for debugging only - use any language here).

In your mdlanguage project you should have learned that it needs more than a few strings to do I18N. I don't know how much wx-c supports I18N. But if it doesn't you will either implement a lousy I18N support or spend a lot of time for it (without using an additional library like libintl).
MOD
Posts: 555
Joined: Jun 11, 2009 20:15

Re: wxFBE, editor for both Windows and Linux

Post by MOD »

D.J.Peters: Solution

wxFBE uses mdLanguage and it's working with wx-c if the language files are UTF8 encoded. But indeed, it's a problem for normal console and fb-screen programs. Will libinlt work better with a console than mdLanguage?

wx-c supports a subset of what wxWidgets could do. I may have to extend wx-c under a new name, just need some more functions wrapped and get it tested.
Last edited by MOD on Sep 25, 2012 19:30, edited 1 time in total.
marcov
Posts: 3452
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: wxFBE, editor for both Windows and Linux

Post by marcov »

TJF wrote:
marcov wrote:
TJF wrote: For good I18N support in FB it's wise to use libintl. This works with UTF-8 encoding. UTF-8 is standard on most *NIX systems but it's hard to find a wondiws console that supports it.
Don't you mean ncursesw? libintl doesn't handle platform specific output at all, but basic encoding conversions on *nix systems??!?
We're talking about russian language and cyrillic letters for the app mentioned in the subject.
I got sidetracked by the "console" thinking it was about the console, but the basic problem still stands.

The language part of libintl will just select from various strings in different languages, returning it as UTF8.

The problem of this app however, is to hand it to feed it whatever API/library does the actual output.

It doesn't matter if you have an UTF8 string containing cyrillic, obtained from libintl, if you then pass it to an API that doesn't understand UTF8.

And on Windows, by default, no API understands UTF8.
TJF
Posts: 3809
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: wxFBE, editor for both Windows and Linux

Post by TJF »

MOD wrote:Will libinlt work better with a console than mdLanguage?
No. mdLanguage and also libintl serve STRINGs in different languages and the problem showing native characters in the console are the same. If the console supports UTF-8 there's no problem. But an FB screen doesn't support it.

The advantage of libintl is a bunch of additional features. Ie L10N support. Or the user doesn't need to select the language, libintl reads the systems locale setting and searches for the best matching translation in the data pool. Similar strings in the source code are translated only once. The xgettext tools have features making a translation easier. Ie you need not extract the strings manually, an autoscan does it for you. That way the source file name and line number of the original text are shown to the translator, so he can make context related translations. And these tools are commonly used, so it's easier to find somebody doing the translation. ...
marcov wrote:It doesn't matter if you have an UTF8 string containing cyrillic, obtained from libintl, if you then pass it to an API that doesn't understand UTF8.
iconv can be used to translate between different character sets. It's used by libintl on wodniws.
AGS
Posts: 1284
Joined: Sep 25, 2007 0:26
Location: the Netherlands

Post by AGS »

MOD wrote: My fault! The checkboxes exist already and the options are working but in code I did a 'If 0 Then' instead of the actual code. I fixed it and uploaded a new package (download at FreeBASIC-Portal, see first post). wxRegEx is not part of wx-c but the search engine can use some RegExp (that's why there's a checkbox for it and no need for pcre). Here a link for what works with it: RegExp in wxFBE.
I tried the regular expression engine as provided with scintilla and.... it's lightweight at best. No support for \w, \d etc...,
no lookahead/lookbehind assertions etc... With a hardcore pcre regular expression the entire C syntax can be recognized(including multiline macros). With the scinitilla regex engine I cannot even come close to that.

I've looked around on the internet for a solution (every project using scintilla runs into this regular expression problem).

Programmers notepad uses scintilla. Solution to the problem: create own regular expression engine and recompile scintilla (with homebrew regex engine built - in). Hmm....

Then I looked at Geany (uses scintilla) and it uses the regular expression engine provided by programmers notepad :) (geany uses it's own brand of scinitlla including the regular expression engine by programmers notepad).
Copy - paste is popular in the world of open - source programming :)

I think getting proper regular expression find/replace (proper as in better than the tiny regular expression engine provided by scintilla) working in wxfbe could be a non - trivial task. Replacing the engine used by scintilla involves recompiling wxwidgets (the regex engine in the scintilla library would need to be rewritten) and the wxwidgets wrapper.

The scintilla buffer itself can be used as input to some regular expression library(no need to recompile anything but wxfbe). The question is whether performance will be good enough. I think using an external regular expression library would work like so:
--> get regex pattern from user;
--> get copy of characters from user selected range (scintilla message: SCI_GETTEXT or similar);
--> match regex pattern against characters and replace whatever needs to be replaced;
--> replace text using scintilla message SCI_SETTEXT (or similar).

Getting the text takes as much memory as there are codepoints in the document. This could take quite a lot of memory (though in practice I don't think source code files are that big). Replacements are always nasty to perform as it involves string concatenation of some kind.

I am not sure whether styling will occur automagically after SCI_SETTEXT. SCI_SETTEXT replaces the current text with non - styled text (restyling is needed) but I am guessing scintilla will restyle automagically after SCI_SETTEXT.

I am guessing that exchanging the current scintilla regular expression engine with something more usable will be dead last on your to-do list (it will take quite a bit of time to implement proper regex find/replace). Which is a shame but I think the scintilla creators are the ones to blame here (why create an editing component that does not include a state-of-the-art regular expression engine?).
marcov
Posts: 3452
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: wxFBE, editor for both Windows and Linux

Post by marcov »

TJF wrote:
marcov wrote:It doesn't matter if you have an UTF8 string containing cyrillic, obtained from libintl, if you then pass it to an API that doesn't understand UTF8.
iconv can be used to translate between different character sets. It's used by libintl on wodniws.
I know. We support iconv as a fallback, on Windows even. (there are some problems with the missing of errno in some versions that require different allocation schemes though, since the E_TOOBIG (or something like that) can't be used to conservatively allocate)

But more importantly Windows itself can to some degree. But the basic problem that is the issue here is matching the right encoding to each API, not the conversion an sich.
Post Reply