Audio library for FreeBasic - Features

General discussion for topics related to the FreeBASIC project or its community.
Post Reply
St_W
Posts: 1618
Joined: Feb 11, 2009 14:24
Location: Austria
Contact:

Audio library for FreeBasic - Features

Post by St_W »

One of the proposals that arose in the FreeBASIC Discussion December 2017 was to add a sound library to FreeBasic. The topic has been already discussed to some extent in this action thread, where several users showed interest in such a library:
https://freebasic.net/forum/viewtopic.php?f=17&t=26246
As the thread dicussed so many different aspects I though it could be a good idea to discuss a specific aspect of the design of that sound library in a new thread; this thread.

This is my proposal of a structured process for finding an appropriate sound library design:
  1. I think it could be a good start to compile a list of features the new sound library should provide from a (FreeBasic-)user perspective.
  2. Afterwards the importance of the features could be weighted, unimportant/unnecessary features could be removed, etc.
  3. As soon as a common set of features has been found, it would be easier to design an API that provides those features.
Let's start with the first point. I'd like to encourage everyone to list the features/usecases he/she wants to see the new FB audio library to support.

Please avoid to dicuss about features suggested by others in the current stage of the process to keep the thread clean. If you want to add/modify/remove features or usecases at a later time you can edit your posts rather than creating a new one.
Last edited by St_W on Jan 01, 2018 22:17, edited 1 time in total.
St_W
Posts: 1618
Joined: Feb 11, 2009 14:24
Location: Austria
Contact:

Re: Audio library for FreeBasic - Features

Post by St_W »

And here is my list of features (which could also serve as example for your posts):
  • Load a sound from a WAV file
  • Play the sound from memory
  • Play different sounds concurrently
  • Pause/Stop playback (for each played sound)
  • Change volume (ideally for each played sound)
  • Play from a certain position
  • Determine the playback position, state (playing/paused) and volume (for each played sound)
  • Multi-Platform: should support every platform supported by FB, that is mainly Windows & Linux. DOS support optional and maybe with limited features (not important for me)
Probably I've forgotten something, but that's my first sketch, I'll extend/update it over time.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Audio library for FreeBasic - Features

Post by angros47 »

I think a sound library should have both low and high level features

Low level:
Open a channel to sound output (setting frequency, mono or stereo mode, and bit depth)
Send raw PCM data to such channel
Open a MIDI channel
Send raw MIDI messages

High level:
Sound primitives to build different wave forms, and create several sound effects (it could also be used to emulate older platforms, or musical instruments)
Ability to save the created sounds as .WAV files
Ability to load .WAV files
Better interface to control MIDI output (perhaps through an improved version of PLAY)
(ability to load and play MIDI files?)

Speech synthesis (using SAPI under Windows, espeak under Linux)

Also, DOS should be supported at least partially, too.
oog
Posts: 124
Joined: Jul 08, 2011 20:34

Re: Audio library for FreeBasic - Features

Post by oog »

Another Feature Request:

PCM:
  • record PCM data into a buffer from a selectable source (like Line In, Microphone etc.)
MIDI:
  • Send MIDI Commands to internal Soundcard (Play Notes)
  • Receive MIDI Commands from other programs or physical MIDI Interface (Sequencer, MIDI Keyboard)
  • Send MIDI Commands to other Programs or physical MIDI Interface
badidea
Posts: 2586
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: Audio library for FreeBasic - Features

Post by badidea »

* Possibility to 'loop' a sound until stop command?
* Master volume?
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Audio library for FreeBasic - Features

Post by angros47 »

Master volume is usually set by the OS, individual applications are unable to modify it (otherwise, an application could set it to zero and mute all the others)
DamageX
Posts: 130
Joined: Nov 21, 2009 8:42

Re: Audio library for FreeBasic - Features

Post by DamageX »

I think a sound library should have both low and high level features

Low level:
Open a channel to sound output (setting frequency, mono or stereo mode, and bit depth)
Send raw PCM data to such channel
Open a MIDI channel
Send raw MIDI messages
I agree that these are fundamental capabilities which should be included. (Probably recording PCM and receiving MIDI also.)

I'm concerned about the details of handling PCM though. For instance, I can imagine a few ways one might implement this...
1) single buffer - load a WAV file, play from start to finish and then stop
2) multiple buffers, realtime - sound plays continuously while the application fills the buffers with data (need a callback function so the application knows when each buffer finishes playing)
3) multiple channels, using a software mixer in the FB sound library
4) multiple channels, using OS/hardware mixing

I think the first two are essential. While supporting multiple channels would be nice also, I worry that it requires more work, especially when aiming to support all platforms including DOS.
High level:
Sound primitives to build different wave forms, and create several sound effects (it could also be used to emulate older platforms, or musical instruments)
Ability to save the created sounds as .WAV files
Do you intend to create and modify sounds in memory only or with realtime playback?
Better interface to control MIDI output (perhaps through an improved version of PLAY)
MIDI can be very simple as it is. Playing music with the Win32 function midiOutShortMsg is pretty well equivalent to doing SOUND 0,100,10,15 in Atari BASIC :)
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Audio library for FreeBasic - Features

Post by angros47 »

Under Linux, the simplest way is to open a pipe toward programs like aplay or pacat (or opening a file toward "/dev/dsp"), and sending raw data, either with PRINT # or with PUT. So, raw PCM sound can be achieved even from the simplest code, and even scripting languages can do that. My idea is just to mimic this simple syntax: a SoundSet command to define the sampling frequency, the number of channels (mono or stereo), and the bit depth (8 or 16), and a PlayBuffer command, to send the number of samples and a pointer to the raw data, that could be in a simple numeric array, or in an UDT type (just like a graphic buffer: GET and PUT can use a specific image type, but also a simple byte array)

The command PlayBuffer could be used more than once, stacking more and more samples: the program would keep running, and if the command is used before the last buffer has finished being played, the new samples would just be queued (something similar, although much more primitive, happened even in the original QBASIC with the PLAY command). A SoundQueue() function could return how many samples are still to be played: when it runs too low, it's time to submit new samples, or the sound will stop.

So, a simple loader could be used to prepare a buffer from a WAV file (besides a small header, a WAV file consists just in raw data), and that buffer could be played with a single command. There would be no need to expose callback: if someone wants to play the same sound over and over, the only thing they would need is to check in the main loop when SoundQueue() is too low, and submit the sample again. This would allow both to play the same WAV file over and over, and to create sounds on the fly (simulating some PSG chip, mixing sound samples with DSP algorithm... or even running a .MP3 decoding library). Implementing support for other sound formats over a similar driver would be pretty easy, and portable.

About multiple channels: it could be done under ALSA, and under Windows Sound System, it would be more troublesome under DOS and under Open Sound System, so, to make the code portable, it would be easier not to include it, and letting the application taking care of the mixing, if needed (also, how would you deal with mixing an 8 bit mono sound with a 16 bit stereo one, with different sampling rate? It would make the library overcomplicated, to add a feature that might not even be used)
Do you intend to create and modify sounds in memory only or with realtime playback?
I intend to create sounds with realtime playback, to save memory, and to provide an experience closer to the classic basic sound command. Of course, it would be possible to send the created sound to a buffer, instead of playing it, and then play that buffer later, or save it as a new WAV file: the concept would be similar to current graphic commands: LINE, or CIRCLE can be executed immediately, displaying the result, or they can be used on a buffer made with ImageCreate, that can later be displayed, or saved.
MIDI can be very simple as it is. Playing music with the Win32 function midiOutShortMsg is pretty well equivalent to doing SOUND 0,100,10,15 in Atari BASIC
In the past, I have seen some examples to emulate SOUND command using midiOutShortMsg, under windows (the code was by Randy Keeling). It worked... but it was Windows only. Also, midi commands are so simple, and so standard, that perhaps it would be better to use them directly instead of building "wrappers" for them. The issue is to make such a command multiplatform, because under Windows if there is no midi-capable hardware a soft synthesizer is provided by default, under Linux on several distro it isn't, so you would have to install Timidity by hand (and it's a heavy download, if you want good sound fonts). Some sound libraries (allegro, or SDL) solved that issue by adding their own soft synth as fall-back solution if no midi is provided by the sound driver. So, the FreeBasic sound library will have to do the same (I already have ported the source of FMMidi to FreeBasic, and the license is compatible)
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Audio library for FreeBasic - Features

Post by angros47 »

Ok, after a long time, I finally managed to have a prototype of what I had in mind:

https://sourceforge.net/projects/freeba ... ary/files/

WAV and midi files are supported, they can be read and written, and the library provides some primitives to modify them, or to create new ones from scratch, using basic commands.
Imortis
Moderator
Posts: 1923
Joined: Jun 02, 2005 15:10
Location: USA
Contact:

Re: Audio library for FreeBasic - Features

Post by Imortis »

angros47 wrote:Ok, after a long time, I finally managed to have a prototype of what I had in mind:

https://sourceforge.net/projects/freeba ... ary/files/

WAV and midi files are supported, they can be read and written, and the library provides some primitives to modify them, or to create new ones from scratch, using basic commands.
Looks nice! When I get off work I will take a look at it in more detail.
Haubitze
Posts: 44
Joined: May 20, 2016 8:42

Re: Audio library for FreeBasic - Features

Post by Haubitze »

a simple SID playing routine where nice for my project, but i think open and play MP3 files easy where a must have if it is not to mutch work.
also a easy to use 3D sound positioning/playing where a good thing for 3D games.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Audio library for FreeBasic - Features

Post by angros47 »

Haubitze wrote:a simple SID playing routine where nice for my project, but i think open and play MP3 files easy where a must have if it is not to mutch work.
No. I think MP3 should stay outside of the SFX library, just as JPG are not included in the GFX library.
also a easy to use 3D sound positioning/playing where a good thing for 3D games.
The best solution would be to use OpenAL for 3d sound, just as OpenGL is used for 3d graphics.
Dr_D
Posts: 2451
Joined: May 27, 2005 4:59
Contact:

Re: Audio library for FreeBasic - Features

Post by Dr_D »

angros47 wrote:
Haubitze wrote:a simple SID playing routine where nice for my project, but i think open and play MP3 files easy where a must have if it is not to mutch work.
No. I think MP3 should stay outside of the SFX library, just as JPG are not included in the GFX library.
also a easy to use 3D sound positioning/playing where a good thing for 3D games.
The best solution would be to use OpenAL for 3d sound, just as OpenGL is used for 3d graphics.
Agreed. It's consistent that way. For instance, png images are normally loaded with FBPng. Also, the native FB primitive drawing set does not include any 3d functions. The community will always build some extended function/helper libraries.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Audio library for FreeBasic - Features

Post by angros47 »

Has anyone tested it? I'd like some feedbacks and opinions
Dr_D
Posts: 2451
Joined: May 27, 2005 4:59
Contact:

Re: Audio library for FreeBasic - Features

Post by Dr_D »

I'm sorry, I haven't tested anything related to sound for a while. I've been fatally addicted to working with circuits/serial communications lately. On top of that, my daughter just started college, and we've been talking like crazy. :) I will try to test after work tomorrow. Hopefully someone else has tested anything?
Post Reply