Simple shared memory [Linux]

[IchMagBier]

Good morning my friends

I wrote a simple function, which allows two or more programs to share a memory buffer. This can be quite useful for interprocess communication. For example you have one program which generates data and another program which processes it. It can be used like this:

Code: Select all

'program1.bas:
dim as ubyte ptr memory
memory=create_sharedmem(32)	'32 bytes
do
	memory[0]=rnd()*&hFF		'Write some random values in the buffer
	sleep 500
loop

'program2.bas:
dim as ubyte ptr memory
memory=create_sharedmem(32)	'32 bytes
do
	print memory[0]			'Print the values generated by "program1.bas"
	sleep 500
loop 

Linux and x86_64 only. It can be converted to 32bit quite easily, you just need to change the asm-code in the "syscall()"-function.

Here is the code for "create_sharedmem()". I didn't use header files, because I couldn't find any and I didn't want to translate the C-headers. It uses the "OPEN" and "MMAP" syscalls from Linux:

Code: Select all

dim shared as zstring*10 key = "SHM_TEST"

#define SYS_READ 		&h00
#define SYS_WRITE 		&h01
#define SYS_OPEN 		&h02
#define SYS_CLOSE		&h03
#define SYS_STAT 		&h04
#define SYS_FSTAT 		&h05
#define SYS_LSTAT 		&h06
#define SYS_POLL 		&h07
#define SYS_LSEEK 		&h08
#define SYS_MMAP 		&h09
#define SYS_MPROTECT 	&h0A
#define SYS_MUNMAP 		&h0B
#define SYS_BRK 		&h0C
'...
#define SYS_FTRUNCATE	&h4D

#define PROT_READ		&h01
#define PROT_WRITE		&h02 
#define PROT_EXEC		&h04    
#define PROT_NONE		&h00

#define O_RDONLY		&h00
#define O_WRONLY		&h01
#define O_RDWR			&h02
#define O_ACCMODE		&h03
#define O_CREAT			&h100

#define MAP_SHARED		&h01
#define MAP_PRIVATE		&h02
#define MAP_FIXED		&h10
#define MAP_FILE		&h00
#define MAP_ANONYMOUS	&h20

#define	S_ISUID	0004000
#define	S_ISGID	0002000
#define	S_IRWXU	0000700
#define	S_IRUSR	0000400	
#define	S_IWUSR	0000200
#define	S_IXUSR	0000100

function syscall(nummer as uinteger, par1 as uinteger=0, par2 as uinteger=0, par3 as uinteger=0, par4 as uinteger=0, par5 as uinteger=0, par6 as uinteger=0) as uinteger
	asm
		mov rdi, par1
		mov rsi, par2
		mov rdx, par3
		mov r10, par4
		mov r8,  par5
		mov r9,  par6
		mov rax, nummer
		syscall
		mov [function], rax
	end asm
end function

function create_sharedmem(size as uinteger)as ubyte ptr
	dim as ulong fd
	fd=syscall(SYS_OPEN, cast(uinteger,@key), O_RDWR or O_CREAT, S_IRUSR or S_IWUSR)
	syscall(SYS_FTRUNCATE, fd, size)
	return cast(ubyte ptr,syscall(SYS_MMAP,, size, PROT_READ or PROT_WRITE, MAP_SHARED, fd))
end function

The defines are stolen from various C-headers. The file in "key" needs to be the same for both programs. It is needed for the filenumber ('fd' in the code).

[TJF]

That method uses ASM code, and doesn't compile with -gcc option (like on ARM or Apple systems).

Why don't you use the libc functions? They also provide semaphores to protect the block. Ie. on 32 bit BeagleBone I use the header file (not tested on 64 bit)

Code: Select all

#DEFINE IPC_CREAT  &o1000
#DEFINE IPC_EXCL   &o2000
#DEFINE IPC_NOWAIT &o4000
#DEFINE IPC_RMID   0

#DEFINE SEM_UNDO &h1000
#DEFINE GETPID   11
#DEFINE GETVAL   12
#DEFINE GETALL   13
#DEFINE GETNCNT  14
#DEFINE GETZCNT  15
#DEFINE SETVAL   16
#DEFINE SETALL   17

TYPE AS __key_t key_t
TYPE sembuf
  AS USHORT sem_num
  AS SHORT _
    sem_op _
  , sem_flg
END TYPE

EXTERN "C" LIB "c"
DECLARE FUNCTION shmctl(BYVAL AS LONG, BYVAL AS LONG, BYVAL AS ANY PTR) AS LONG
DECLARE FUNCTION shmget(BYVAL AS key_t, BYVAL AS size_t, BYVAL AS LONG) AS LONG
DECLARE FUNCTION shmat(BYVAL AS LONG, BYVAL AS CONST ANY PTR, BYVAL AS LONG) AS ANY PTR
DECLARE FUNCTION shmdt(BYVAL AS CONST ANY PTR) AS LONG

DECLARE FUNCTION semctl(BYVAL AS LONG, BYVAL AS LONG, BYVAL AS LONG, ...) AS LONG
DECLARE FUNCTION semget(BYVAL AS key_t, BYVAL AS LONG, BYVAL AS LONG) AS LONG
DECLARE FUNCTION semop(BYVAL AS LONG, BYVAL AS ANY PTR, BYVAL AS size_t) AS LONG
END EXTERN

#DEFINE ID_SEM &Hdeadbeef

The code creating the shared block is

Code: Select all

VAR SEM_ID = semget(ID_SEM, 1, IPC_CREAT OR &o660)
IF -1 = SEM_ID                         THEN e = !"\n semget" : ErrorStop
IF -1 = semctl(SEM_ID, 0, SETVAL, 1)   THEN e = !"\n semctl" : ErrorStop

VAR SHM_ID = shmget(ID_SHM, shm_size, IPC_CREAT OR IPC_EXCL OR &o660)
IF -1 = SHM_ID                         THEN e = !"\n shmget" : ErrorStop
VAR shm_ptr = shmat(SHM_ID, 0, 0)
IF shm_ptr = CAST(ANY PTR, -1)          THEN e = !"\n shmat" : ErrorStop

VAR CSS = NEW(shm_ptr) Status                      '' create an instance of Status UDT in memory block
VAR PAR = NEW(shm_ptr + SIZEOF(Status)) Parameters '' create an instance of Parameters UDT behind the Status UDT

and in the other programms the same code acceses the block, but 'IPC_CREAT OR IPC_EXCL OR' is omitted.

[IchMagBier]

Why don't you use the libc functions?

I didn't use header files, because I couldn't find any and I didn't want to translate the C-headers.

I guess I was lazy. :p
I also don't like using more libraries than necessary, so I can keep my binary small.

You use shmget, while I use mmap, which is more "modern" or something. Shm is from the old System V API, as far as I know.

They also provide semaphores to protect the block.

When the program, you share the memory with, is a child program, you can use "MAP_ANONYMOUS", where mmap is called. Like this:

Code: Select all

return cast(ubyte ptr,syscall(SYS_MMAP,, size, PROT_READ or PROT_WRITE, MAP_SHARED or MAP_ANONYMOUS, fd))

In that case, the buffer can only be read and written by the child process and its parent.

[TJF]

I guess I was lazy. :p
I also don't like using more libraries than necessary, so I can keep my binary small.

You use shmget, while I use mmap, which is more "modern" or something. Shm is from the old System V API, as far as I know.

Your method uses libc, as well as mine. shm functions group is in the current API (and will be in future versions), mmap is used internally.

You don't keep your binary small. Instead you blow up your binary by re-coding libc functions.

Anyway, both of us like beer.

[nimdays]

Another way is memfd

[TJF]

Another way is memfd

Sure, but this needs a lot of slow seek, get and put operations. In contrast with shared memory there's no such overhead. You just create classes (UDTs) in that memory block and work with them as if they're in normal memory (see above):

Code: Select all

VAR CSS = NEW(shm_ptr) Status                      '' create an instance of Status UDT in memory block
VAR PAR = NEW(shm_ptr + SIZEOF(Status)) Parameters '' create an instance of Parameters UDT behind the Status UDT

[nimdays]

^^
Thanks