WSTR Crash on linux 32-bit

Linux specific questions.
Post Reply
VANYA
Posts: 1834
Joined: Oct 24, 2010 15:16
Location: Ярославль
Contact:

WSTR Crash on linux 32-bit

Post by VANYA »

Hi all!

I found a problem on the system linux 32-bit. Linux 64-bit has no such problem.

If you save the original BAS code in UTF-8 encoding (without BOM) and compile with this content:

Code: Select all

dim as string s = chr(&hC4 , &hEE)

? lcase(Wstr(s))
then the compiled program crashes. Here is the content of the error:
free(): invalid pointer: 0x09778ea0 ***
======= Backtrace: =========
/lib/i386-linux-gnu/libc.so.6(+0x67387)[0xb751b387]
/lib/i386-linux-gnu/libc.so.6(+0x6d307)[0xb7521307]
/lib/i386-linux-gnu/libc.so.6(+0x6dc41)[0xb7521c41]
/home/user/FBTEMP[0x80494ca]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf7)[0xb74cc647]
======= Memory map: ========
08048000-0804e000 r-xp 00000000 08:01 22083 /home/user/FBTEMP (deleted)
0804e000-0804f000 rwxp 00005000 08:01 22083 /home/user/FBTEMP (deleted)
0804f000-08055000 rwxp 00000000 00:00 0
09778000-09799000 rwxp 00000000 00:00 0 [heap]
b7000000-b7021000 rwxp 00000000 00:00 0
b7021000-b7100000 ---p 00000000 00:00 0
b7147000-b7163000 r-xp 00000000 08:01 394602 /lib/i386-linux-gnu/libgcc_s.so.1
b7163000-b7164000 rwxp 0001b000 08:01 394602 /lib/i386-linux-gnu/libgcc_s.so.1
b7180000-b72b2000 r-xp 002d7000 08:01 11541 /usr/lib/locale/locale-archive
b72b2000-b74b2000 r-xp 00000000 08:01 11541 /usr/lib/locale/locale-archive
b74b2000-b74b4000 rwxp 00000000 00:00 0
b74b4000-b7664000 r-xp 00000000 08:01 458995 /lib/i386-linux-gnu/libc-2.23.so
b7664000-b7665000 ---p 001b0000 08:01 458995 /lib/i386-linux-gnu/libc-2.23.so
b7665000-b7667000 r-xp 001b0000 08:01 458995 /lib/i386-linux-gnu/libc-2.23.so
b7667000-b7668000 rwxp 001b2000 08:01 458995 /lib/i386-linux-gnu/libc-2.23.so
b7668000-b766b000 rwxp 00000000 00:00 0
b766b000-b7684000 r-xp 00000000 08:01 456338 /lib/i386-linux-gnu/libpthread-2.23.so
b7684000-b7685000 r-xp 00018000 08:01 456338 /lib/i386-linux-gnu/libpthread-2.23.so
b7685000-b7686000 rwxp 00019000 08:01 456338 /lib/i386-linux-gnu/libpthread-2.23.so
b7686000-b7688000 rwxp 00000000 00:00 0
b7688000-b768b000 r-xp 00000000 08:01 458983 /lib/i386-linux-gnu/libdl-2.23.so
b768b000-b768c000 r-xp 00002000 08:01 458983 /lib/i386-linux-gnu/libdl-2.23.so
b768c000-b768d000 rwxp 00003000 08:01 458983 /lib/i386-linux-gnu/libdl-2.23.so
b768d000-b76e0000 r-xp 00000000 08:01 458990 /lib/i386-linux-gnu/libm-2.23.so
b76e0000-b76e1000 r-xp 00052000 08:01 458990 /lib/i386-linux-gnu/libm-2.23.so
b76e1000-b76e2000 rwxp 00053000 08:01 458990 /lib/i386-linux-gnu/libm-2.23.so
b76e2000-b7702000 r-xp 00000000 08:01 394734 /lib/i386-linux-gnu/libtinfo.so.5.9
b7702000-b7704000 r-xp 0001f000 08:01 394734 /lib/i386-linux-gnu/libtinfo.so.5.9
b7704000-b7705000 rwxp 00021000 08:01 394734 /lib/i386-linux-gnu/libtinfo.so.5.9
b7719000-b771a000 rwxp 00000000 00:00 0
b771a000-b7721000 r-xs 00000000 08:01 5762 /usr/lib/i386-linux-gnu/gconv/gconv-modules.cache
b7721000-b7722000 rwxp 00000000 00:00 0
b7722000-b7724000 r--p 00000000 00:00 0 [vvar]
b7724000-b7725000 r-xp 00000000 00:00 0 [vdso]
b7725000-b7748000 r-xp 00000000 08:01 458987 /lib/i386-linux-gnu/ld-2.23.so
b7748000-b7749000 r-xp 00022000 08:01 458987 /lib/i386-linux-gnu/ld-2.23.so
b7749000-b774a000 rwxp 00023000 08:01 458987 /lib/i386-linux-gnu/ld-2.23.so
bf8d9000-bf8fa000 rwxp 00000000 00:00 0 [stack]

Aborting due to runtime error 14 ("abnormal termination" signal)
I certainly understand that the encoding of the file is UTF-8, but I am trying to send characters from an extended ASCII table. The encoding conversion is not correct, but shouldn't the WSTR and LCASE functions safe?
badidea
Posts: 2586
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: WSTR Crash on linux 32-bit

Post by badidea »

I booted my old netbook with 32-bit Xubuntu 14.04 and fbc 1.00.0, bit I could not reproduce the problem.
I'll update fbc and try again...

With fbc 1.08.1 (32-bit), the program crashes, but I also get a compile warning:

Code: Select all

fbc test.bas
fbc: /lib/i386-linux-gnu/libtinfo.so.5: no version information available (required by fbc)
badidea@N210:~/Desktop$ ./test 

*** Error in `./test': free(): invalid pointer: 0x08c1b5e8 ***
Aborted (core dumped)
I switched back to fbc 1.07.0 (32-bit) and both compiler warning and crash are gone.

Code: Select all

badidea@N210:~/Desktop$ fbc test.bas 
badidea@N210:~/Desktop$ ./test
??
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: WSTR Crash on linux 32-bit

Post by coderJeff »

The changed behaviour (new / different) mostly likely due to this fix by Skyfish for multi-byte character conversions:
https://github.com/freebasic/fbc/commit ... 1ddf9f26d5

In the runtime library, previous behaviour was to take a simple string length, but that would result in wrong buffer length if there were multibyte characters.

In the fix, the length calculation was changed to mbstowcs(), but if that fails and returns -1, it's not being handled causing a buffer allocation bug.

I have an idea for a fix that will revert to previous behaviour if invalid multi-byte characters are encountered.
VANYA
Posts: 1834
Joined: Oct 24, 2010 15:16
Location: Ярославль
Contact:

Re: WSTR Crash on linux 32-bit

Post by VANYA »

badidea , thanks for testing and confirming the problem.
coderJeff wrote:I have an idea for a fix that will revert to previous behaviour if invalid multi-byte characters are encountered.
I hope you can find a way to fix this problem. Thank you for paying attention to this topic.
Iczer
Posts: 99
Joined: Jul 04, 2017 18:09

Re: WSTR Crash on linux 32-bit

Post by Iczer »

It kind of of topic here, but may be related/helpful - i had crashes in fb-1.08.1 x64 in windows in similar, situation:

Code: Select all

dim as wstr * 64 wsQQQ= "*real unicode symbols, out of ascii table - asc("symbol") > 256*"

print "> string = ";str(wsQQQ)' <-- crush
with w-strings with have only asc("symbol") < 256 its work OK...
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: WSTR Crash on linux 32-bit

Post by coderJeff »

Iczer wrote:i had crashes in fb-1.08.1 x64 in windows in similar, situation:
Can you give a better example?
Is source file saved utf encoded with unicode points in the file?
Or is an ascii file with unicode escaped characters?

Much better for me to see an example that crashes fbc 1.08.1 so I can compare with current 1.09.0 development. Thanks.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: WSTR Crash on linux 32-bit

Post by coderJeff »

VANYA wrote:I hope you can find a way to fix this problem. Thank you for paying attention to this topic.
I pushed a fix for the problem to current fbc 1.09.0 development.
Bug ticket: #942 string to wstr conversion fails for invalid multibyte characters

Just note that string --> wstring conversions give different results on windows versus linux, at least for me. Maybe this due to operating system or locale, I don't know. I'm sure it's not the last we'll see for conversion bugs. At least the compiled program doesn't crash for this case.
VANYA
Posts: 1834
Joined: Oct 24, 2010 15:16
Location: Ярославль
Contact:

Re: WSTR Crash on linux 32-bit

Post by VANYA »

coderJeff wrote:
I pushed a fix for the problem to current fbc 1.09.0 development.
Bug ticket: #942 string to wstr conversion fails for invalid multibyte characters
Excellent thank you! When the next build comes out, I'll try and sign off on the test results.
Post Reply