Part 3: Action (hopefully)

General discussion for topics related to the FreeBASIC project or its community.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Part 3: Action (hopefully)

Post by angros47 »

marcov wrote: There are multiple directions to go with drawing. Directly on a canvas, via a widget set, or via opengl/directx/vulkan. And of course plain sad old framebuffer and even older VESA 1.x VGA.
Indeed. A strength of FreeBasic graphic mode is that it could use all these modes, and it happens under the hood: this helps portability, because when the developers use the ScreenRes command, they don't need to care if the screen mode will be set using a VGA BIOS call, or the most advanced API.

One of my patches in the source was to allow the use of OpenGL mode in the same way, so GPU shaders can be used on the FreeBasic graphic, too
Which one to chose? The one with the most future potential, Vulkan ?
Have you ever had a look at Vulkan? I really, really hope it won't go mainstream, at least not in its current form, because it looks terrible, for a developer. Look at one example:

https://github.com/krOoze/Hello_Triangl ... iangle.cpp

More than 1600 lines... just to draw one triangle! Something that in OpenGL 1.0 was done with three or four commands. OpenGL 2.0 already made it much harder (I saw it, when I adapted OpenB3D).... Vulkan seems to aim to make it impossible.
If you can generate it to play directly, you can generate it on install or first run too. No downloads and no hardware or API dependent mess.
No API dependent mess? And how are you supposed to play the generated files? You would still need to call the API to play a .WAV file (unless you try to play it with the SHELL command, that would be the worst solution). Both Windows and Linux sound API supports the option to play a buffer from memory, so there is no need to save the sound on the hard disk and reload it to play it, when you can play it directly.


@jj2007

Your syntax is too ambiguous: what if I want to play a sound file that is called "200" (a perfectly valid file name)?. Sound "200" would refer to a frequency, or to that file? Never use strings in place of numeric parameters.

My idea is something like:

Code: Select all

sound PulseWave(100),3 
sound TriangleWave (200),8
And, for a .WAV file, something like:

Code: Select all

Dim MyBuffer(200000) as short
LoadWav "File.wav", @MyBuffer(0)
playbuffer @MyBuffer(0),200000
LoadWav could be a native command, and it would be possible to have LoadMP3, LoadOGG and so on as external libraries.
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Part 3: Action (hopefully)

Post by marcov »

angros47 wrote:
marcov wrote: There are multiple directions to go with drawing. Directly on a canvas, via a widget set, or via opengl/directx/vulkan. And of course plain sad old framebuffer and even older VESA 1.x VGA.
Indeed. A strength of FreeBasic graphic mode is that it could use all these modes, and it happens under the hood: this helps portability, because when the developers use the ScreenRes command, they don't need to care if the screen mode will be set using a VGA BIOS call, or the most advanced API.
The graphic mode is a mix of old VGA drawing with some canvas drawing (GDI,Cairo) enhancements. You can of course try to emulate it using any technology, but that is not the same. QB64 is for emulations and freezing in old tech, not FB

And that is directly the weakness. You are forcing the modernly used technology through a roughly VGA interface that doesn't exist anymore. GPU and scaling windows are the norm, not old fixed form canvases, as well as rendering the image using compositing rather than old school drawing primitives.

That is already the case, but if you continue in this vein, it will only look more outdated say 3 years on.
One of my patches in the source was to allow the use of OpenGL mode in the same way, so GPU shaders can be used on the FreeBasic graphic, too
Fun, but IMHO irrelvevant, see above.
Which one to chose? The one with the most future potential, Vulkan ?
Have you ever had a look at Vulkan? I really, really hope it won't go mainstream, at least not in its current form, because it looks terrible, for a developer. Look at one example:
Yes I have. And I agree. But libraries will emerge in time to cover the worst. Hell, some might be coded in FB ! :-)
Moreover it has headers that are fairly machine translatable, so easier keeping up to date.

I do think it is early days yet for Vulkan, but I do think it IS coming long term. If only because most drivers will in the end support vulkan first, and opengl-over-vulkan second. And probably that second interface will get less efficient with time.

OTOH, if you define _any_ interface to a new graphics lib, you better pay attention to it. A bit of generalization might support both opengl and vulkan.
More than 1600 lines... just to draw one triangle! Something that in OpenGL 1.0 was done with three or four commands. OpenGL 2.0 already made it much harder (I saw it, when I adapted OpenB3D).... Vulkan seems to aim to make it impossible.
A few years ago I modernized my work codebase from opengl 1.x-early 2.x to 3.3 (it is mostly a zoomable bitmap with a multi layer canvas overlay). But in the end, I'm glad I did it. I learned quite some things about my old 1.x codebase too, and that it was much less perfect than I thought it to be. One of the reasons to migrate btw is that the old code became slower on DX10/GL3 hardware.

The only bummer is that I kept wglusebitmap font support in the end. It was faster than the shader eq that I had planned. I still have to simplify the shader stuff, but time ran out (or better, orders came in, and that is always a good thing :-)
If you can generate it to play directly, you can generate it on install or first run too. No downloads and no hardware or API dependent mess.
No API dependent mess? And how are you supposed to play the generated files?
I mean then a single interface to play a PCM WAV is enough. I assume that that is there of course (mmsystem, dxsound, gnome-sound or phonon and whatever Mac has).

If you use any other aspects of the audio card except that, it will be hell to implement on all those apis, and it will be more realtime, since you need to regularly feed more data or change settings.
St_W
Posts: 1619
Joined: Feb 11, 2009 14:24
Location: Austria
Contact:

Re: Part 3: Action (hopefully)

Post by St_W »

marcov wrote:
I didn't say that I don't like low level access, I just wanted to mention that it hasn't been the original philosophy of BASIC to do a lot of bit-twiddling, pointer-magic or direct hardware access. Of course people had to resort to those possibilities (as far as BASIC allowed) when the pre-made solutions of BASIC did not fit their needs anymore (maybe you remember DirectQB?).
I think you are more of a QB64 person. Emulations seem to suit you.
You must have got my point wrong. I was saying that simple things like drawing lines, rectangles, .. or using the PC beeper has been suitable in the 1980s or 90s when Basic was popular, but nowadays a lot more complex operations are considered simple operations. If you want to stay with PLAY and SOUND it's indeed better to go with QB64. But if you want to apply the original philosophy of BASIC to the current hardware and software you'll see that the API needs some changes to still fullfil the goal of simplicity for the user.
To get back to the example DirectQB: it wasn't written in BASIC because the available API and compiler were incapable and too restricted.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Part 3: Action (hopefully)

Post by angros47 »

marcov wrote:And that is directly the weakness. You are forcing the modernly used technology through a roughly VGA interface that doesn't exist anymore. GPU and scaling windows are the norm, not old fixed form canvases, as well as rendering the image using compositing rather than old school drawing primitives.
This is not the place to discuss FreeBasic weaknesses. There was a thread for that, some days ago

A few years ago I modernized my work codebase from opengl 1.x-early 2.x to 3.3 (it is mostly a zoomable bitmap with a multi layer canvas overlay). But in the end, I'm glad I did it. I learned quite some things about my old 1.x codebase too, and that it was much less perfect than I thought it to be. One of the reasons to migrate btw is that the old code became slower on DX10/GL3 hardware.

The only bummer is that I kept wglusebitmap font support in the end. It was faster than the shader eq that I had planned. I still have to simplify the shader stuff, but time ran out (or better, orders came in, and that is always a good thing :-)
I added shader support to OpenB3D, as well. But I waited, in doing that, and I am glad I did, because a lot of the features added in shaders were almost immediately deprecated. So, there is no point in supporting a new interface, until it is stable and offers real advantages. Using shaders for speed, for example, improves frame rate on some video cards... but on older ones, not supporting them, actually decreases frame rate, since the missing functions have to be emulated on the CPU (that would be slower, too, since we are talking about outdated hardware). So, unless you really need a feature that can be achieved only with shaders, you shouldn't use them just because it's the modern way to do things.

I mean then a single interface to play a PCM WAV is enough. I assume that that is there of course (mmsystem, dxsound, gnome-sound or phonon and whatever Mac has).

If you use any other aspects of the audio card except that, it will be hell to implement on all those apis, and it will be more realtime, since you need to regularly feed more data or change settings.
In fact, my suggestion was to build a simple, generic interface to feed raw PCM data to the sound card from a FreeBasic program. And such interface should be identical in DOS, Windows or Linux. Then, developers could write the code they want, portable from/to any operating system
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Part 3: Action (hopefully)

Post by marcov »

angros47 wrote:
marcov wrote:And that is directly the weakness. You are forcing the modernly used technology through a roughly VGA interface that doesn't exist anymore. GPU and scaling windows are the norm, not old fixed form canvases, as well as rendering the image using compositing rather than old school drawing primitives.
This is not the place to discuss FreeBasic weaknesses. There was a thread for that, some days ago
Only some heavily moderated one and for a limited time. Too limited a format, and the timing was inconvenient.

Anyway, sufficient to say, that most FB interfaces are stuck in a pre 2000 state, which shouldn't come as a surprise to anyone. Yes, FB is at least 32-bit, and the procedural system has evolved a bit beyond the ancient state, but the base principles of language and libraries are very old and thoroughly need overhaul. But it is probably possible to conserve both the old and put a set of new interfaces next to it. FPC did.
I added shader support to OpenB3D, as well. But I waited, in doing that, and I am glad I did, because a lot of the features added in shaders were almost immediately deprecated.
Yes. I maybe did the bulk of the work a bit later (say 3 years ago) and while I was doing it, the 3.3 core profile was introduced and I adapted to use that format as much as possible (with currently wglusebitmapfont as only exception. I do use an older 3.x system for uniforms in my shaders though, because it allows me to set uniforms on name, and not a number like the newer (layout(location = 2)) kind of syntax. Easier to keep in sync)

That said, I wouldn't know how to style a Opengl or vulkan interface. In my own code I did a bit of OO modeling to make it a bit more flexible and keep overview, but, while better, it is not the panacea I hoped it to be. Still too much interaction and object specific parameters that must propagate through.
So, there is no point in supporting a new interface, until it is stable and offers real advantages. Using shaders for speed, for example, improves frame rate on some video cards... but on older ones, not supporting them, actually decreases frame rate, since the missing functions have to be emulated on the CPU (that would be slower, too, since we are talking about outdated hardware). So, unless you really need a feature that can be achieved only with shaders, you shouldn't use them just because it's the modern way to do things.
Do you store all calls made in a canvas of sorts that you execute on draw? How does it work?
In fact, my suggestion was to build a simple, generic interface to feed raw PCM data to the sound card from a FreeBasic program. And such interface should be identical in DOS, Windows or Linux. Then, developers could write the code they want, portable from/to any operating system
Good. But make that the primary interface, also to the user, and not the old "sound hz". The few that want that, can make generators for it, but don't force users to obsolete interfaces.
Imortis
Moderator
Posts: 1923
Joined: Jun 02, 2005 15:10
Location: USA
Contact:

Re: Part 3: Action (hopefully)

Post by Imortis »

I feel like this is getting a bit off topic. I can see that most are okay with the idea of integrating a sound library into FB, but the details are up for debate. That is fine. Maybe we should start another thread to go more into detail on that?

What about the other idea of utilizing namespaces with Runtime Lib and FBGfx functions to keep the global namespace as clean as possible?
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Part 3: Action (hopefully)

Post by angros47 »

marcov wrote: Anyway, sufficient to say, that most FB interfaces are stuck in a pre 2000 state, which shouldn't come as a surprise to anyone. Yes, FB is at least 32-bit, and the procedural system has evolved a bit beyond the ancient state, but the base principles of language and libraries are very old and thoroughly need overhaul. But it is probably possible to conserve both the old and put a set of new interfaces next to it. FPC did.
True, but 2d raster graphic, nowaday works in the same way that 30 years ago: basically, a 2d matrix. Sure, it is not based on bit planes any more, but this happens under the hood. The only real change is that RGB color is used, instead of indexed color. Sure, new features could be added, like multiple layers, or scrolling, or rotation/scaling of pictures, but the buffer itself can be managed in the same way.

Another possible change would be adding support for HDR mode, but it is not a priority.
Do you store all calls made in a canvas of sorts that you execute on draw? How does it work?
OpenB3D? It creates a series of shaders when initialized, and then it stores all the meshes in the video memory. When the render function is called, first it checks for visibility, then the specific render routine is called for each mesh. The last shader used is kept in memory, to reduce switching
Good. But make that the primary interface, also to the user, and not the old "sound hz". The few that want that, can make generators for it, but don't force users to obsolete interfaces.
In fact my idea was that one. The "sound hz" would be only a legacy command for QB mode. But a "Sound TriangleWave(hz)", or similar, would work in a way similar to most modern soft synth used to program. Have you ever seen how they work?

http://cosmosis.co.uk/tips-and-tricks/m ... chitecture

basically, you combine one or more oscillators, one or more filters, modulators and you drive them with ADSR envelopes. The original Moog synthesizer worked in that way, and it is now almost a standard. With some tweaking it is possible to produce almost every timbre procedurally, and to simulate most of sound hardware
jj2007
Posts: 2326
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Part 3: Action (hopefully)

Post by jj2007 »

angros47 wrote:Your syntax is too ambiguous: what if I want to play a sound file that is called "200" (a perfectly valid file name)?. Sound "200" would refer to a frequency, or to that file? Never use strings in place of numeric parameters.

My idea is something like:

Code: Select all

sound PulseWave(100),3 
sound TriangleWave (200),8
And, for a .WAV file, something like:

Code: Select all

Dim MyBuffer(200000) as short
LoadWav "File.wav", @MyBuffer(0)
playbuffer @MyBuffer(0),200000
LoadWav could be a native command, and it would be possible to have LoadMP3, LoadOGG and so on as external libraries.
My syntax is simple. Of course, if somebody choses to use "200", without extension, as a name for a sound file, my syntax will miserably fail. Maybe it is too basic.
paul doe
Moderator
Posts: 1730
Joined: Jul 25, 2017 17:22
Location: Argentina

Re: Part 3: Action (hopefully)

Post by paul doe »

Imortis wrote:What about the other idea of utilizing namespaces with Runtime Lib and FBGfx functions to keep the global namespace as clean as possible?
I personally like the idea: if FB has namespace support, might as well use it as much as we can to expand it. I'm currently coding a framework -using VB.NET as a reference- to add a little functionality, as 'vanilla' FB is a little too barebones (read: low level by nowaday standards), that can be wrapped in a namespace also, and evolve from there.
lizard
Posts: 440
Joined: Oct 17, 2017 11:35
Location: Germany

Re: Part 3: Action (hopefully)

Post by lizard »

angros47 wrote:Is it enough, to start building a new sound library for FreeBasic?
While it maybe would not be senseful to add a native GUI to the core at the moment, loading and handling of multimedia files would be a good thing.

In order to integrate these functions smoothly into the core of FB with respect for the original philosophy and syntax of Basic this could happen with only a few commands like

result = LOAD( [ <buffer>, ]< filename> , <type> [ , <param> ] )

in usual manner like at the other commands, here type would describe the type of file like:

sound:
TYPE_WAV
TYPE_MP3
TYPE_OGG
...

graphics:
TYPE_BMP
TYPE_PNG
TYPE_GIF
...

text:
TYPE_TXT
TYPE_RTF
TYPE_DOC
...

others:
TYPE_XLS
TYPE_USER ' User defined
TYPE_ANY ' Any of the known, recognized by extension

Param would be optional further parameters. After loading data could be used depending on its type with commands like PLAY for soundfiles, SOUND and SPEAK for effects, waveforms and speechsynthesis like Angros47 mentioned.

Graphics can be smoothly integrated in the existing graphics system which ist already good now, BTW.

The rest of file types could be added over time, must not be all at once. Many file types and their handlers could be installed. One of the most interesting for me personally would be .rtf since this is my favourite textformat since long time.

This way a whole lot of functionality could be added in FB core with usage of only a few KEYWORDS.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Part 3: Action (hopefully)

Post by angros47 »

Just to make my point: playing an external audio file is the simplest thing in the world: it can be done even with a stupid piece of code like:

Code: Select all

SHELL “sndrec32.exe /embedding /play /close laser.wav”
A lot of libraries allow it: Allegro, for example.

But, as a programmer, I would want something better. I would want a way to build my own sounds, not to just load pre-made sounds. And building them with an external tool is not the same, because I might want to build the sound "on the fly", when the program is running. My examples about graphics were just analogies: like the graphic library, a sound library for FreeBasic should have some primitives to create new sounds.

We all agree that the commands SOUND and PLAY are too weak, for that. And there is a reason: they have been done for the PC speaker. Many 8-bit computers offered better sound commands: a Commodore 128, for example, offered control for timbre and ADSR envelope. An MSX offered polyphony for the PLAY command. All stuff that was not available on the PC speaker, so gw basic and Quick Basic never offered commands for that (not even volume was available).

So, we need to build a set of primitives that would control the sound like it was possible to do with the old computers running BASIC, but considering the sound features that are available on current operating systems
St_W
Posts: 1619
Joined: Feb 11, 2009 14:24
Location: Austria
Contact:

Re: Part 3: Action (hopefully)

Post by St_W »

angros47 wrote:But, as a programmer, I would want something better. I would want a way to build my own sounds, not to just load pre-made sounds. And building them with an external tool is not the same, because I might want to build the sound "on the fly", when the program is running. My examples about graphics were just analogies: like the graphic library, a sound library for FreeBasic should have some primitives to create new sounds.
I don't think that sound effects, music or graphics are typically created "on the fly" by the applications. Just like images or 3D models they are typically created using external tools like Photoshop, 3ds Max/Blender, Cubase/WaveLab/..., etc. and loaded by the application. The application typically only performs some transformations on the existing data, like picking an excerpt of it or applying effects on it.
angros47 wrote:We all agree that the commands SOUND and PLAY are too weak, for that. And there is a reason: they have been done for the PC speaker. Many 8-bit computers offered better sound commands: a Commodore 128, for example, offered control for timbre and ADSR envelope. An MSX offered polyphony for the PLAY command. All stuff that was not available on the PC speaker, so gw basic and Quick Basic never offered commands for that (not even volume was available).
So, we need to build a set of primitives that would control the sound like it was possible to do with the old computers running BASIC, but considering the sound features that are available on current operating systems
The question is where to start and where to end. It won't make a lot of sense to implement features equivalent to a large DAW software.

The features should be oriented on what is commonly needed and should be as universal and flexible as possible. Although WAV/MP3/OGG are specific file formats they are used so commonly that I'd add a loader for them. Once the audio/image data is loaded from a file the application may modify it and use it. You'll agree that using an external sound player allows less flexibility than a built-in player. For example you can't pause playback and it won't work platform independently.

IMHO the most important thing beside the API is to provide an abstraction from the platform dependent sound APIs. It should hide system specific details and complexity while still being as powerful and versatile as needed.

Regarding the API: if it should be designed similarily to the graphics API I'd avoid the introduction of new keywords as far as possible. e.g. BLOAD could be used to load the audio data (BLOAD WAV, "sound.wav"). We shouldn't repeat design mistakes like for BMP loading (into memory), which requires to read the dimensions of the image manually with knowledge of the BMP file format, because there's no API for it. Designing a good old-BASIC style API is possible for sure, however I'd prefer a new (object oriented) API in the FB (or its own) namespace.
lizard
Posts: 440
Joined: Oct 17, 2017 11:35
Location: Germany

Re: Part 3: Action (hopefully)

Post by lizard »

lizard wrote: TYPE_RTF
TYPE_DOC
TYPE_XLS
Naturally these formats are little strange here, have taken just any formats without thinking much. You know what i mean.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Part 3: Action (hopefully)

Post by angros47 »

St_W wrote:I don't think that sound effects, music or graphics are typically created "on the fly" by the applications. Just like images or 3D models they are typically created using external tools like Photoshop, 3ds Max/Blender, Cubase/WaveLab/..., etc. and loaded by the application. The application typically only performs some transformations on the existing data, like picking an excerpt of it or applying effects on it.
It depends on the application. A game, perhaps might work as you said, but what if the application written in FreeBasic is a graphic editor like Paint? Or a fractal explorer? Those programs need to create new graphics, not to load pictures. And one of the reason to use FreeBasic and not C is because it offers a very quick way to prototype similar applications, to build a picture from an algorithm, and so on.



The question is where to start and where to end. It won't make a lot of sense to implement features equivalent to a large DAW software
True. But it might make sense to implement the features of a small DAW software. The key of BASIC is to be All-Purpose. The commands available in the graphic library aren't supposed to be the equivalent of PhotoShop.
The features should be oriented on what is commonly needed and should be as universal and flexible as possible. Although WAV/MP3/OGG are specific file formats they are used so commonly that I'd add a loader for them.
I agree to add a loader for .WAV. The other formats should have an external library.

IMHO the most important thing beside the API is to provide an abstraction from the platform dependent sound APIs. It should hide system specific details and complexity while still being as powerful and versatile as needed.
That is exactly what I have done with my driver for PCM output: with two commands, you can output audio data on Linux, Windows and Dos using the same syntax
Regarding the API: if it should be designed similarily to the graphics API I'd avoid the introduction of new keywords as far as possible. e.g. BLOAD could be used to load the audio data (BLOAD WAV, "sound.wav").
Sounds good, but I wonder if it's possible without conflicts
St_W
Posts: 1619
Joined: Feb 11, 2009 14:24
Location: Austria
Contact:

Re: Part 3: Action (hopefully)

Post by St_W »

angros47 wrote:But it might make sense to implement the features of a small DAW software. The key of BASIC is to be All-Purpose. The commands available in the graphic library aren't supposed to be the equivalent of PhotoShop.
Sounds reasonable, could you mention some examples for such features?

I've started a new thread for finding a list of features and usecases for the new sound library:
https://freebasic.net/forum/viewtopic.p ... 6&p=241927

I'd like to encourage everyone interested to take part and suggest features there!
To keep that thread clean please continue (related and unrelated) discussions here for now.

If you'd like to suggest another process for finding important features and usecases for the sound library or if you think the procedure is stupid and useless please let's dicuss them here or in a new thread.
Post Reply