GUI design

[Psyche_Demon]

Hi,
I use the keyboard to navigate. For instance alt+tab/shift+tab to switch windows, CTRL+tab/shift+tab to switch tabs, tab/shift+tab to navigate controls on a window, arrow keys to navigate lists, menus and text, the alt key activates the menu bar, enter activates a menu item, space bar activates a button and toggles a checkbox, F1 for documentation (for the most part), Of course then you've got pretty standardised keyboard shortcuts (Alt+F4 to close applications, Ctrl+F4 to close single documents in an MDI, standard editing keys like Ctrl+Z/X/C/V etc), and of course keyboard shortcuts and hotkeys that might be linked to the software itself.
As for using FreeBASIC, I use a standard text editor. I don't bother with any of the IDE's, again because they are mostly visual.
For compilation, I either use the standard command line (Yay for text!), or I use my custom system that takes a single file with all the source files, libraries and compile options, invokes the compiler with the required parameters, traps the messages sent to stdout/stderr by the compiler and shows it in a message box.
Cheers.

[BasicCoder2]

Ok. So I take it using a mouse is not an option.

I see no reason why your suggestions could not be incorporated into one of the GUI's used by FreeBASIC but of course only the author or an advanced programmer with access to the source code could implement it. I could add any feature to one of the GUI designers I had written myself but modifying someone else's code would be beyond my abilities.

I assume as you would move through the controls using the keys for you the actual layout wouldn't be important however you are interested in making it nice for a sighted person to use? So your problem is that there is no audio feedback with current graphic user interface designers as to a controls actual dimensions and positions within a window?

.

[Psyche_Demon]

That's right. At the moment there are only numbers, which could mean absolutely anything. I could easily say, I want a window at 1000 1000, 2000 high and 2000 wide. But how on earth do I know how big that is physically? Whereas with some actual form of sensory representation, I would be able to imagine how that might look. Of course I couldn't visualise everything, but I could at least work out whether it would be readable, whether the controls fit etc. For instance, taking my idea to use panning (left/right in the stereo field) for the X axis, if my control was too far to the right (I.E. wouldn't fit on the screen), there'd be no sound. Same with pitch on the Y. If the frequency was unhearable (above, say, 15 to 20KHZ), you'd reach the sun instead of the screen, for want of a better analogy. If you could barely hear the sound because it was too short, you'd know the control was too small. If it was too low in volume, you'd know that the text was blurred or there was something obscuring it. If you heard two sounds at once, you know the controls overlap. The list of possibilities goes on.
Cheers.

[BasicCoder2]

Making a GUI layout of controls doesn't require a GUI designer. Such programs simply make it easy for a sighted person to build a GUI but you can also simply write lines of code to do the same thing. As for the size of the controls you can easily convert pixel size into inches or centimeters. Just get someone to measure it for you. In my case the screen is 1366 pixels wide or 13.5 inches or 345 cm or about 100 pixels to the inch.

Controls are just rectangles. What do you think of having flat rectangle pieces of different sizes that you can arrange as a layout on a surface where a grid is etched out so you can feel the actual positions? Then all you have to do is type in the numbers. The spatial arrangement of controls you can feel will then be displayed?

I really don't think you need all that audio feedback. I see that more as means for exploring the screen using a mouse or having a audio designer for laying out rectangles. I already have written a simple program to layout rectangles and could add an audio output and replace the mouse control with key controls. You could then have an audio output of the position and dimensions of the rectangle to type into the line of code of whatever control you want there.

Have you written any of the audio programs in FreeBASIC at this stage?

I have visual basic net 2010 installed although I haven't used it much.

Cheers

.

[OSchmidt]

At the moment there are only numbers, which could mean absolutely anything.
I could easily say, I want a window at 1000 1000, 2000 high and 2000 wide. But how on earth do I know how big that is physically?

Sorry, but since you mentioned that you wrote a lot of games (with special sounds), I thought that those would
also come with "some visuals" - and that you had some experience about Screen-Sizes or Sprite-Sizes and such.

Now it appears (from your recent remarks and reading again what you wrote about your games),
that these were "pure sound-output" (no graphics at all) - is that assumption correct?

If yes, then knowing about "typical ScreenSizes these days" (and the PixelDimensions of those Screens)
would be a requirement for you - so that you can think in "relations", which I guess will become "natural"
to you over some time, or perhaps it already is (in all likelihood better than average).

In short (but googling helps as well) - the "typical affordable Office-Monitor of a PC is a 24 inch Flat-Screen
these days,coming with "HD-resolution" (1920x1080 Pixels).

Not each and everybody has one of those - but IMO it's safe to say that this is some kind of "average".
Maybe to not loose Users who have Screens with lower Pixel-Resolution, you should restrict your
Form-Sizes to maybe 1024y768 maximium - with that Form-setting you will reach about 95% or more of
the current PC-Users (the Form in almost all cases then not being larger than the available ScreenSize).

After you established that FormSize as a "fixed PixelSize", a Base in your thinking and your Apps, you can take
these "fixed Dimensions" (the about 1000Pixels horizontal and 700 vertical) into account - and then start
learning (and later "thinking") in Pixel-Relations, to do the partioning of that available Form-Space in your head.

For that you will have "some homework" to do - e.g. by exploring Control-Boundaries in other Apps as e.g.:
- how wide (in Percent of the available Form-Width) is a typically left-hand-docked TreeView (it's about 20-30%)
- what's the Height in Pixels of a typically Top-Docked ToolBar (about 30-50 Pixels, depending on the Buttons and their Text-arrangement)
- what's the Height of a typical, Bottom-Docked StatusBar (it's about 25-35 Pixels)
- the typical Dimensions of a TextBox (about 120-200 in width, 22-30 Pixesl in Height)
- about the same size as the TextBoxes are the typical Labels, CheckBoxes, Buttons and DropDowns (dependend a bit of the TextLength they hold)

I still think, that a "manual adding" of each of these Controls (per Construction-CodeLine) in a Text-Editor
is to be preferred over any "Sound-Feedback" or something.

Some Overlap-Checks (as existing already in my last Demo-Snippet) are in order - but IMO only to
discover some "typos" you might have made whilst hacking in the Positioning-Coords in your Widget-creation-Lines.

Working this way has several advantages even for developers *with* the possibility "to see a certain Form-Layout"...
E.g. when defining Constants - for the LeftOffset, the Width and the Height e.g. for of a "whole Group of TextBoxes",
you can then copy and paste the first construction-line (with these three Coord-Constants) - and
copy it 5 times down the Code - after that only adjusting the Top-Coord and the Ctl-Names in each of these lines -
then you will have a perfectly arranged Group of 6-Text-Controls (each having the same LeftOffs, Width and Height).

If you try that within a Form-Editor (per Mouse, even with eye-sight) - you will have a hard time to get that Group
properly sized and aligned in the same time, an experienced Developer accomplishes that thing per Code-instructions only.

And as said, after your "first 5 Forms or so", you will know much better, how large a typical TextBox or Button is (in Pixels) -
or how to "align and dock a TreeView-Control at the Forms left side" (manually moving it in the Form_Resize-Event).

To achieve that kind of precision (in positioning and sizing) with some kind of Frequency-changing Sound-Feedback
is nearly impossible IMO - so why not rely on your Braille-reader and these nice and reliable "hard-numbers" for the
Control-Coords... GUIs look nice when "groups of controls are perfectly aligned, to the Pixel"... and for your first steps
(to get familiar with "what relation in percent of the available Form-Size" looks good on what Control-Class) - I would
ask a friend to go through the first simple (already working) Forms with you - giving you visual feedback, so that you're
able to build up some experience and knowledge about the whole thing, by adjusting your Coords to what he or she
suggests "looks good this way".

Just my two cents.

Olaf

[Psyche_Demon]

Hi,
Yes you are correct. My games are purely sound driven - no visuals whatsoever. So for me working in pixels is a brand new concept.
Somebody said searching on Google. This of course would be the obvious thing to do. But again, most websites describe visual elements in a way that a sighted person could understand them. I can find nothing that describes it to a totally blind person.
The other thing to note. Somebody mentioned a "braille" reader. My screen reader can work with braille displays. However I don't have such a display and so I am relying on text-to-speech only.
Cheers.

[BasicCoder2]

Have just visited your web page so I have a better idea where you are coming from.
Out of curiosity I have also just downloaded BGT and when I get time I will try and use it.
So at this stage if someone posts a FreeBASIC audio output, keyboard input program you would have no trouble compiling and running it?
.

[Psyche_Demon]

Hi there,
I could compile and use it fine.
Interesting you can still access my website - I have cancelled my accounts with the web host for various reasons and am in the process of moving to a VPS host rather than shared so I expected it to be down for a while.
I'd be interested to see what you think of BGT. I used to develop in that pretty much all the time (all it has is a show_game_window function so I don't even have to worry about positioning the window, just give it a title and you're good to go). The only caveat with BGT is that it is no longer maintained, hence the reason I decided to switch to another language. I've worked on a limited basis with C/C++, Python etc, but looks as though for the most part FB suits me just fine. Of course I'm still trying to get used to the facts that functions have to be declared or defined before they can be called, I have to get used to the difference of modules versus headers, learn the joys of pointers and memory management, but other than that it seems to have about 99% of what I need already packaged with it.
Cheers.

[BasicCoder2]

You say you know nothing about pixels. Think of them as being like the dots in braille but very small. They are so small that if you could run your finger over a line of them they would feel like a raised edge. Indeed if a screen were made of dots that could be raised or flat you would have a screen you could feel and be able to trace out the edges and raised areas.

You didn't respond to the idea of using a mouse to "see" the screen. After I googled the subject I see the only one experimental mouse which has a set of such dots on it that could read text under the mouse position and raise the dots required to make the braille text.

However it seems to me that using a mouse, or better still a touch screen, programs could be developed that told you with speech and or tones what you were touching. You could feel for objects as you might feel for objects on a desk. I have been experimenting with ideas using a blank screen to see if it would work for me.

Thus I move the mouse and it will verbally tell me where it is and what is under it. For example it might report the mouse as being in a particular area of the screen.

The code following is a simple example of using an auditory signal that the mouse is on an edge which can be imagined as a row of raised dots. You would need a mouse to test it. Just move the mouse slowly and whenever it is over an edge there will be a beep sound. Ideas keep coming as I think about it. Maybe the tone can signal how close the mouse is to an edge to make it easy to follow. Might be an auditory game in that.

Some of the programs require .wav data and like image files this forum doesn't provide storage for such files.

Also I have been thinking about how to just use the keyboard instead of the mouse.

Code: Select all

declare function sound alias"Beep"(frequency as integer,duration as integer) as integer

#define dot sound(900,50):sleep 100
#define dash sound(900,400):sleep 100

screenres 640,480,32
dim shared as integer mx,my,mb
dim shared as integer x,y,w,h
' fill with random rectangles
for i as integer = 0 to 20
    w = int(rnd(1)*100)+50
    h = int(rnd(1)*100)+50
    x = int(rnd(1)*(640-150))
    y = int(rnd(1)*(480-150))
    line (x,y)-(x+w,y+h),rgb(255,255,255),b
next i

setmouse 320,240  ' places mouse in center of window
do
    getmouse mx,my,,mb
    if point(mx,my)=rgb(255,255,255) and mb<>-1 then
        Dot
    end if
    sleep 2
loop until multikey(&H01)

.

[dodicat]

I was also thinking of a tone depending how far the mouse is from a certain point:

Code: Select all

#define UNICODE
#include Once "disphelper/disphelper.bi"
dhInitialize(TRUE)
dhToggleExceptions(True)

sub Speak(text As string,voice as string="Sapi.SpVoice")
  Dim As Wstring * 1024 tmp = voice
  DISPATCH_OBJ(IDptr)
  if dhCreateObject(@tmp,0,@IDptr)<>0 then exit sub
  tmp = text
  dhCallMethod(IDptr,".Speak(%S)",@tmp,1)
  SAFE_RELEASE(IDptr)
End sub



Declare Function sound Alias"Beep"(Byval f As long,Byval d As long) As long

dim as long mx,my,btn,flag
#define distance(x,y,px,py) sqr((px-x)*(px-x)+(py-y)*(py-y))
#define map(a,b,x,c,d) ((d)-(c))*((x)-(a))/((b)-(a))+(c)
#define incircle(cx,cy,radius,x,y) (cx-x)*(cx-x) +(cy-y)*(cy-y)<= radius*radius
 
screen 19
dim as long counter
speak "Press the left mouse button to hear the tone"
do
    getmouse mx,my,,btn
    counter+=1
    dim as integer d=distance(mx,my,400,300)
    dim as integer ptch=map(0,500,d,1000,50)
if btn =1 and flag=0 then
 flag=1:sound(ptch,50)
 if incircle(400,300,20,mx,my) then speak "In the circle"
 end if
screenlock
    cls
    locate 1,1
    print counter
    circle(400,300),20
screenunlock
sleep 1,1
flag=btn
loop until len(inkey)
print "done, press key"

sleep 

[Psyche_Demon]

Hi,
That brings up a very interesting question. If you don't actually have a screen, then how does the system work? I don't actually have a screen plugged in, yet my screen reader still reads text. So if I ran your code, how would the system define the edge of the window?
I haven't actually got a mouse so will have to test it after the new year when I can get somebody to take me out and get one. Even then I probably wouldn't have a clue what to do. In fact, I know so little about the mouse that I wonder how on earth it has its place in audio games (BGT has support for mouse movement and clicking and several audio games are starting to support the mouse as an input movement device).
As for your mouse query. I did respond - I said that my screen reader, NVDA, does have some mechanism for audio output when moving the mouse. Again I haven't been able to test this and I must admit I'm slightly scared to in case I end up doing something that I didn't intend - using a mouse is completely alien to me - it's like giving a baby a computer keyboard and they end up pressing the delete key - big oopsy!
Dodicat - I see you're using SAPI - that looks like the COM object API? I did not realise FB supported COM programming. That opens up a whole new set of doors to me. Unfortunately not perhaps in this area, but several other libraries that I used to use which are COM objects.
Cheers.

[dodicat]

You can use
setmouse ,,1,1
before a graphics loop.
This keeps the mouse visible (for me) the first 1
and keeps the mouse inside the window (the second 1)

I turned my monitor off and could locate the centre via the mouse.

including disphelper.bi lets you use VB script functions via the disphelper static library.

[BasicCoder2]

To Psyche Demon,

If you don't have a screen the system should still work. The screen image is in the computer, it is an array of numbers, and it is that array that is being used not the output display screen.

The code can be written to use the arrow keys instead of the mouse to move the cursor around.

Indeed I noticed a particular issue with using the mouse in that it is the visual position of the cursor on the screen that controls how the mouse is positioned. The mouse can be lifted and placed elsewhere without the cursor itself following the mouse. The position of the mouse on the mouse pad does not tell you the position of the mouse cursor on the screen area. This however is not the case with a touch screen. Where you touch it will determined the exact coordinates sent to the program.

The mouse query was about your interest in using the mouse not about NVDA having an audio output when it moves.

My understanding here is that you wanted a GUI for sighted people which you could use and or design yourself via some kind of auditory output. So I imagined some kind of audio white cane with sound and speech output could be used to build up a mental picture of where things were within the rectangle screen much as you might learn where things are in a room or table top and be able to move to them quickly and directly rather than in a linear fashion.

From some of your statements I see you and dodi cat have a lot more knowledge about the innards of Windows then I do. Mostly I am limited to simple BASIC code and depend on others to provide code for things not innate to FreeBASIC like sound. Would a stereo sound system be useful to know the relative positions of the objects. I imagine them having an auditory property like instruments in an orchestra.


Cheers.

[BasicCoder2]

To dodicat

It is very important for a sighted person to actually turn off the screen or wear a blindfold otherwise they will be using visual feedback without realizing it and simply observing that the frequency is changing.

I found it hard to find the circle, at least as quickly as I should be able to find it. Knowing the relative position above, below, left or right of the target would have been helpful.

With earphones I would suggest the ability for stereo sound could enable a much faster movement to some location without needing to change frequency?

Also the delay in the "you are in the circle" after clicking the mouse button was annoying. Instead I would suggest an instant "DING" would be better? Although I used your BEEP code example I think using .WAV sounds would be a much more useful and pleasant experience? The only reason I didn't use them is I can't post them to the forum.

My thoughts were to use the mouse or touch screen as a audio white cane so a mental picture of the GUI could be built up and items quickly found.

Cheers
.

[dodicat]

This one has four boxes.
use arrow keys and F1 to F4 and h to navigate to a box of your choice.
The relative position could be coded for, but this is a simple test for Psyche to try.
No mouse or screen required.
(small edit done)

Code: Select all

#define UNICODE
#include Once "disphelper/disphelper.bi"
#include "fbgfx.bi"
Using fb
dhInitialize(TRUE)
dhToggleExceptions(True)

Sub Speak(text As String,voice As String="Sapi.SpVoice")
    Dim As Wstring * 1024 tmp = voice
    DISPATCH_OBJ(IDptr)
    If dhCreateObject(@tmp,0,@IDptr)<>0 Then Exit Sub
    tmp = text
    dhCallMethod(IDptr,".Speak(%S)",@tmp,1)
    SAFE_RELEASE(IDptr)
End Sub

Declare Function sound Alias"Beep"(Byval f As Long,Byval d As Long) As Long
Type box
    As Long x,y,w,h
    As Long cx,cy
    As zstring * 10 c
    Declare Sub getcentre
    Declare Sub Draw
End Type
Sub box.getcentre
    cx=x+w/2
    cy=y+h/2
End Sub
Sub box.draw
    Line(x,y)-(x+w,y+h),,b
    ..draw String(cx-4,cy),c
End Sub
Dim As box b(1 To 4)
For n As Long=1 To 4
    b(n).x=200+n*70
    b(n).y=200
    b(n).w=50
    b(n).h=30
    b(n).c=Str(n)
    b(n).getcentre
Next n

#define inbox(x,y,b) x>b.x and x<b.x+b.w and y>b.y and y<b.y+b.h
#define distance(x,y,px,py) sqr((px-x)*(px-x)+(py-y)*(py-y))
#define map(a,b,x,c,d) ((d)-(c))*((x)-(a))/((b)-(a))+(c)

Screen 19,32
Dim As Long counter,x=120,y=120,flag
Dim As Long cx=b(1).cx,cy=b(1).cy
Dim As String focus="Box 1"
Setmouse ,,0,1

speak "Use the arrow keys to move"
speak "Press the space bar to hear a tone"
speak "press keys F1 to F4 to choose a box"
speak "Press h for a reminder"

Do
    'set the focus box
    If Multikey(SC_F1)Then cx=b(1).cx:cy=b(1).cy:focus="Box 1":speak  "Choose "+ focus
    If Multikey(SC_F2)Then cx=b(2).cx:cy=b(2).cy:focus="Box 2":speak  "Choose "+focus
    If Multikey(SC_F3)Then cx=b(3).cx:cy=b(3).cy:focus="Box 3":speak  "Choose "+focus
    If Multikey(SC_F4)Then cx=b(4).cx:cy=b(4).cy:focus="Box 4":speak  "Choose "+focus
    
    If Multikey(SC_H) Then speak "Focus" + focus 'help reminder
    
    ' Check arrow keys and update the (x, y) position accordingly
    If Multikey(SC_LEFT ) And x >   0 Then x = x - 1
    If Multikey(SC_RIGHT) And x < 800 Then x = x + 1
    If Multikey(SC_UP   ) And y >   0 Then y = y - 1
    If Multikey(SC_DOWN ) And y < 600 Then y = y + 1
    counter+=1
    Dim As Integer d=distance(x,y,cx,cy)
    Dim As Integer ptch=map(0,500,d,1000,50)
    If Multikey(SC_SPACE) And flag=0 Then   
        flag=1
        sound(ptch,50)
        For n As Long=1 To 4
            If inbox(x,y,b(n)) And focus="Box "+Str(n) Then speak "In" + focus
        Next n
    End If
    Screenlock
    Cls
    Locate 1,1
    Print counter,focus

    Circle(x,y),5,,,,,f
    For n As Long=1 To 4
        b(n).draw
    Next n
    Screenunlock
    flag=Len(Inkey)
    Sleep 1,1
    While Inkey <> "": Wend 
Loop Until Multikey(SC_ESCAPE)       
speak "done, press  any key"

Sleep