shm_attach

info at tristat dot org (2011-10-15 17:07:05)

You might expect the SHM functions to be significantly faster than building variables from MySQL operations. Unfortunately this is not the case regarding large multidimensional arrays. I've tested write, read, update and delete operations with with a two dimensional associative array directly with SHM and in comparison with a class using MySQL and a class using SHMOP functions (with mostly offset-accurate serialization). Both classes generated the same array as stored with SHM. Unlike SHMOP the performance of SHM decreases considerably with larger arrays. At 2000 instructions with an $array[$row][$key] of 200 rows and 5 keys even MySql is faster than SHM.This might be due to the fact, that SHM deals with any kind of arrays and makes internal use the PHPs powerful serialize() function.

nathanbruer at gmail dot com (2011-04-05 13:21:21)

I was playing around with these functions and made a class in the process. This will of course be slower than accessing a variable locally, but gives the ability to store variables in a shared environment and gives many running scripts the understanding that it should access them from the shared area. This should also auto destroy the shared memory area once no more scripts have a link to the data (when all scripts use this class).

<?php
class SharedMemory{
    private $nameToKey = array();
    private $key;
    private $id;
    function __construct($key = null){
        if($key === null){
            $tmp = tempnam('/tmp', 'PHP');
            $this->key = ftok($tmp, 'a');
            $this->id = shm_attach($this->key);
            $this->nameToKey[] = '';
            $this->nameToKey[] = '';
            $this->updateMemoryVarList();
            shm_put_var($this->id, 1, 1);
        }else{
            $this->key = $key;
            $this->id = sem_get($this->key);
            $this->refreshMemoryVarList();
            shm_put_var($this->id, 1, shm_get_var($this->id, 1) + 1);
        }
        if(!$this->id)
            die('Unable to create shared memory segment');
    }
    function __sleep(){
        shm_detach($this->id);
    }
    function __destruct(){
        if(shm_get_var($this->id, 1) == 1){
            // I am the last listener so kill shared memory space
            $this->remove();
        }else{
            shm_detach($this->id);
            shm_put_var($this->id, 1, shm_get_var($this->id, 1) - 1);
        }
    }
    function __wakeup(){
        $this->id = sem_get($this->key);
        shm_attach($this->id);
        $this->refreshMemoryVarList();
        shm_put_var($this->id, 1, shm_get_var($this->id, 1) + 1);
    }
    function getKey(){
        return $this->key;
    }
    function remove(){
        shm_remove($this->id);
    }
    function refreshMemoryVarList(){
        $this->nameToKey = shm_get_var($this->id, 0);
    }
    function updateMemoryVarList(){
        shm_put_var($this->id, 0, $this->nameToKey);
    }
    function __get($var){
        if(!in_array($var, $this->nameToKey)){
            $this->refreshMemoryVarList();
        }
        return shm_get_var($this->id, array_search($var, $this->nameToKey));
    }
    function __set($var, $val){
        if(!in_array($var, $this->nameToKey)){
            $this->refreshMemoryVarList();
            $this->nameToKey[] = $var;
            $this->updateMemoryVarList();
        }
        shm_put_var($this->id, array_search($var, $this->nameToKey), $val);
    }
}

// Example
$sharedMem = new SharedMemory();
$pid = pcntl_fork();
if($pid){
    //parent
    sleep(1);
    echo "Parent Says: " . $sharedMem->a . "\n";
    echo "Parent Changed to 0\n";
    $sharedMem->a = 0;
    //Parent just changed it to 0
    echo "Parent Says: " . $sharedMem->a . "\n";
    sleep(2);
    // Parent think's it's 0, but child has changed it to 1
    echo "Parent Says: " . $sharedMem->a . "\n";
}else{
    //child
    $sharedMem->a = 2;
    echo "Child Changed to 2\n";
    // Should be 2 as child just set it to 2
    echo "Child Says: " . $sharedMem->a . "\n";
    sleep(2);
    // Child think's it's 2, but the parent set it to 0.
    echo "Child Says: " . $sharedMem->a . "\n";
    echo "Child Added 1\n";
    $sharedMem->a++;
    echo "Child Says: " . $sharedMem->a . "\n";
}
?>

somepay at gmail dot com (2010-08-05 03:07:09)

In my log I have found string
"shm_attach(): failed for key 0x366f: No space left on device"
But have not found any suggestion for this at php.net and at google.
So question was how to make free memory used by shm_attach .
At first to view (Linux) how many segments allocated, use
:~# ipcs -mu
then
limits
:~# ipcs -ml
to remove segment use:
:~# ipcrm -m [shmid]
otherwise you can reboot server, or launch sh script based on commands above.
To avoid troubles with "No space left on device" ALWAYS use
shm_remove() & shm_detach() after call shm_attach().

pail dot luo at gmail dot com (2009-02-26 18:43:43)

A simple Sample to introduce SHM.

<?php
if ( sizeof($argv)<2 ) {
        echo "Usage: $argv[0] send|recv|rem|dele ID [msg] \n\n" ;
        echo "   EX: $argv[0] send 1 \"This is no 1\" \n" ;
        echo "       $argv[0] recv 1 \n" ;
        echo "       $argv[0] rem 1 \n" ;
        echo "       $argv[0] dele \n" ;
        exit;
}

// $SHMKey = ftok(__FILE__, "Z");
$SHMKey = "123456" ;

## Create/Open a shm
$seg = shm_attach( $SHMKey, 1024, 0666 ) ;

switch ( $argv[1] ) {
    case "send":
        shm_put_var($seg, $argv[2], $argv[3]);
        echo "send msg done...\n" ;
        break;
        
    case "recv":
        $data = shm_get_var($seg, $argv[2]);
        echo $data . "\n" ;
        break;
    
    case "rem":
        shm_remove_var($seg, $argv[2]);
        break;

    case "dele":
        shm_remove($seg);
        break;
    
    case "dele2":
        `/usr/bin/ipcrm -M 123456`;
        break;
}
?>

nkatz at yahooo dot com (2008-09-18 05:31:56)

Responding to jpeter1978 at yahoo dot com ... Assuming a character is typically two bytes, the size of a serialized variable (incl array or obj) is 2 * its strlen(). The 44 is 4 more than the 24 + 16 = 40 suggested by php at cytrax dot de plus 4 bytes for worst case 4-byte alignment. So this is probably a reliable formula if you are too lazy to align it using something similar to zeppelinux at comcast dot net as in:
($strlen($serialized_array_or_obj) /4 ) * 4 ) + 40;
The zeppelinux formula would use 20 (for 4-byte integer cpu) or 36 (for 64-bit or 8-byte cpu) for the ending constant so 40 or 44 is probably just achieving header padding but it certainly can't hurt.

zeppelinux at comcast dot net (2008-08-14 00:19:12)

<?php

//how to calculate the  minimum $memsize required to store the variable $foo where $foo='foobar'.

// when shm_attach() is called for the first time, PHP writes a header to the beginning of the shared memory. 
$shmHeaderSize = (PHP_INT_SIZE * 4) + 8;

// when shm_put_var() is called, the variable is serialized and a small header is placed in front of it before it is written to shared memory.
$shmVarSize = (((strlen(serialize($foo))+ (4 * PHP_INT_SIZE)) /4 ) * 4 ) + 4;

// now add the two together to get the total memory required. Of course, if you are storing more than one variable then you dont need to add $shmHeaderSize for each variable, only add it once.
$memsize = $shmHeaderSize + $shmVarSize;

//this will give you just enough memory to store the one variable using shm_put_var().
$shm_id = shm_attach ( $key, $memsize, 0666 ) ;
shm_put_var  ( $shm_id  , $variable_key  , $foo  );

any attempt to store another variable will result in a 'not enough memory' error.
Be aware that if you change the contents of $foo to a larger value and then you try to write it to shared memory again using shm_put_var(), then you will get a 'not enough memory' error. In this case, you will have to resize your shared memory segment and then write the new value.

If you are only storing variables that contain a single integer value, then you can avoid having to resize by always allocating the largest amount of memory that is required to store an int, which should be:
$shmIntVarSize = (((strlen(serialize(PHP_INT_MAX))+ (4 * PHP_INT_SIZE)) /4 ) * 4 ) + 4;

?>

hetii at poczta dot onet dot pl (2008-01-11 05:18:08)

Hi :)
I write small class for build bright message between my application.

<?
class Bright_Message
{

var $bright;
var $SHM_KEY;
var $my_pid;

        function Bright_Message($SHM_KEY=null)
        {
                $this->my_pid = getmypid();//Get own pid
                if (is_null($SHM_KEY)) $this->SHM_KEY = '123123123';
                $this->bright = shm_attach($this->SHM_KEY, 1024, 0666);
                $this->one_instance();
        }

        function get_msg($id,$remove=true)
        {
                if(@$msg=shm_get_var($this->bright,$id))
                {
                   if ($remove) @shm_remove_var($this->bright,$id);
                   return $msg;
                } else return false;
        }

        function snd_msg($id,$msg)
        {
                @shm_put_var($this->bright,$id,$msg);
                return true;
        }

        function one_instance()
        {
                $SHM_PID = $this->get_msg(1,false);
                if((strpos(exec('ps p'.$SHM_PID),$_SERVER['SCRIPT_FILENAME'])) === false)
                $this->snd_msg(1,$this->my_pid); else
                {
                        echo "This program exists on pid: $SHM_PID\r\n\r\n";
                        exit;
                }
        }

}
?>

send.php:
<?
include "bridge_message.class.php";
$shm = new Bright_Message();
$shm->snd_msg(2,'this is my simple message');
?>

receive.php:
<?
include "bridge_message.class.php";
$shm = new Bright_Message();
$msg = get_msg(2);
echo print_r($msg,1);
?>

jpeter1978 at yahoo dot com (2007-08-03 15:35:46)

I tried all the suggestions above for getting the object size (in bytes) for $memsize, but they didn't work universally for the two types of objects I tried (string and array of strings).
After doing some googling and experimenting, I've found the following magic formula:
$memsize = ( strlen( serialize( $object ) ) + 44 ) * 2;
I found this in someone else's code, so I can't explain it.

Uther Pendragon (2007-03-08 09:08:22)

As a follow-up to my last post regarding shm_attach and its limit capability for knowing how it was created....
for more control, use the shmop_* series of functions, as they have finer grained control than these.
and by the way: the SHMOP functions SHOULD BE listed under "see also" for all the SHM* wrapper functions (I assume they are wrappers to the SHMOP* functions).

Uther Pendragon (2007-01-24 08:26:09)

Since there aren't seperate functions for CREATING and ATTACHING shared memory (a mistake in my opinion), you might want to do some testing to check whether you've created it or not, as you may not want the slave of a master/slave pair to ever create the shared memory.
One way you can test this is by having your slave set the size to something small, then testing the size by putting a variable too large to fit, e.g.:
function get_shmem() {
return shm_attach(ftok('somefile.txt', 'T'), 100, 0644);
}
$shm = get_shmem();
while (!@shm_put_var($shm, 1, str_repeat('.....', 20))) {
shm_remove($shm);
sleep(1);
//we created it, so we'll remove it and sleep to wait for the master to create it, then try again.
$shm = get_shmem();
}
shm_remove_var($shm, 1);
//here we know the shared memory was already created, because we could put a variable in bigger than the size requested.
Another way you can handle it is to have all programs initialize the shared memory with the same parameters... I had a problem with this when my clients starting too fast and created the shmem without passing a memsize value, so it was defaulting to 10k which was too small.

Katzenmeier (2006-11-04 15:36:36)

The limit for one SHM block seems to be 32 MB, but you can split your data in several SHM blocks if you need to. The total SHM limit seems to be about 8 GB.
I'm not sure whether this is true for all configurations.

muytoloco at yahoo dot com dot br (2006-02-23 09:33:32)

If one process make a shm_attach to one inexistent memory area, this area will be created under the same priviliegies of the script running user. If another process will try to create or acces the same area, runnig by other user with different privileges of the first user, an error will occur.

rch at todo dot com dot uy (2005-03-28 04:17:45)

Cecil, the key of a var is an integer (not the name ). You can put multiples vars in the same share.

#!/usr/local/bin/php -q
<?PHP

$SHM_KEY = ftok(__FILE__, chr( 4 ) ); 

$data =  shm_attach($SHM_KEY, 1024, 0666);

$test1 = array("hello","world","1","2","3");
$test2 = array("hello","world","4","5","6");
$test3 = array("hello","world","7","8","9");

shm_put_var($data, 1, $test1);
shm_put_var($data, 2,$test2);
shm_put_var($data, 3,$test3);

print_r(shm_get_var($data, 1));
print_r(shm_get_var($data, 2));
print_r(shm_get_var($data, 3));

shm_detach($data);
?>

andreyKEINSPAM at php dot net (2004-04-24 09:31:43)

As far as I see from the sources of ext/sysvshm, it's not needed to do arithmetic (bit) OR (|) of "perm" with IPC_CREAT (and IPC_EXCL). The extension will do it for you. It tries to attach to the memory segment and if the try did not succeed it tries to attach but with flags set to user_flag | IPC_CREAT | IPC_EXCL.
The exact code (shm_flag is the third param to the function) :
if ((shm_id = shmget(shm_key, 0, 0)) < 0) {
if (shm_size < sizeof(sysvshm_chunk_head)) {
php_error_docref(NULL TSRMLS_CC, E_WARNING, "failed for key 0x%x: memorysize too small", shm_key);
efree(shm_list_ptr);
RETURN_FALSE;
}
if ((shm_id = shmget(shm_key, shm_size, shm_flag | IPC_CREAT | IPC_EXCL)) < 0) {
php_error_docref(NULL TSRMLS_CC, E_WARNING, "failed for key 0x%x: %s", shm_key, strerror(errno));
efree(shm_list_ptr);
RETURN_FALSE;
}
}

Cecil (2004-03-18 00:37:54)

Here is an example of how to use one shared memory block to store multiple variables or arrays.. unfortunetly in order to store more than ONE variable, you have to use sem_get() multiple times.. same goes for shm_attach(), shm_put_var() and shm_get_var()

#!/usr/local/bin/php -q
<?PHP
// test.php

$SHM_KEY = ftok(__FILE__,'A');

$shmid  = sem_get($SHM_KEY, 1024, 0644 | IPC_CREAT);
$shmid2 = sem_get($SHM_KEY, 1024, 0644 | IPC_CREAT);
$shmid3 = sem_get($SHM_KEY, 1024, 0644 | IPC_CREAT);

$data =  shm_attach($shmid, 1024);
$data2 = shm_attach($shmid2, 1024);
$data3 = shm_attach($shmid3, 1024);

$test = array("hello","world","1","2","3");
$test2 = array("hello","world","4","5","6");
$test3 = array("hello","world","7","8","9");

shm_put_var($data,$inmem,$test);
shm_put_var($data2,$inmem2,$test2);
shm_put_var($data3,$inmem3,$test3);

print_r(shm_get_var($data,$inmem));
print_r(shm_get_var($data2,$inmem2));
print_r(shm_get_var($data3,$inmem3));

shm_detach($data);
shm_detach($data2);
shm_detach($data2);
?>

to REALLY test it.. create a second script like so and run it..

#!/usr/local/bin/php -q
<?PHP
// test2.php

$SHM_KEY = ftok(__FILE__,'A');

$shmid  = sem_get($SHM_KEY, 1024, 0644 | IPC_CREAT);
$shmid2 = sem_get($SHM_KEY, 1024, 0644 | IPC_CREAT);
$shmid3 = sem_get($SHM_KEY, 1024, 0644 | IPC_CREAT);

$data =  shm_attach($shmid, 1024);
$data2 = shm_attach($shmid2, 1024);
$data3 = shm_attach($shmid3, 1024);

print_r(shm_get_var($data,$inmem));
print_r(shm_get_var($data2,$inmem2));
print_r(shm_get_var($data3,$inmem3));

shm_detach($data);
shm_detach($data2);
shm_detach($data2);
?>

As you can see, test2.php doesn't insert anything into shared memory.. yet it pulls out 3 totally different arrays already stored..

Hope that helps.. took me a bit to get it right.. everyone seems to have their own idea of how shm should be used. lol.

BTW, not sure how the ftok works to be honest, cause I didn't change the __FILE__ to match the file path of test.php or anything.. I would think that the file path out have to be the exact same to work correctly.. oh well, it worked as-is! haha..

- Cecil

php at cytrax dot de (2002-01-18 08:36:04)

The memsize needed for shared memory (tested on linux system) can be calculated with:
For each varialbe you want to store: 24 Bytes
+ serialized var-size (see below) alligned by 4 Bytes
+ 16 Bytes
For a update of a var with the same key, the memory for the old var will be needed extra.

koester at x-itec dot de (2001-03-21 12:41:46)

4194304 means 4 MB and NOT 4 GB for shared memory on FreeBSD. You can increase the shared memory at runtime if you like.

lew at persiankitty dot com (2001-01-25 04:30:35)

Finding out shared memory kernel settings in FreeBSD:
sys1# sysctl -a | grep -i SHM
kern.ipc.shmmax: 4194304
kern.ipc.shmmin: 1
kern.ipc.shmmni: 96
kern.ipc.shmseg: 64
kern.ipc.shmall: 1024
kern.ipc.shm_use_phys: 0
Shows us that we can allocate a total of 4GB (wow) of shared memory, etc...

webmaster at mail dot communityconnect dot com (1999-09-10 13:50:52)

With Sun Solaris 2.x, the MAXIMUM shared memory value allowed is 1,048,576. The maximum allowed value can be determined using the command /usr/sbin/sysdef. On Linux, there does not seem to be any system enforced maximum size. To change the maximum allowed size on Solaris 2.x, use set shmsys:shminfo_shmmax=[new value].

h dot raaf at i-k-c dot net (1999-03-25 05:40:16)

Notice that 'int key' for shared-memory is shared with the keys used for semaphores. So you get in trouble when you use the same key value for semaphores and shared memory!

eric at superstats dot com (1999-02-12 19:04:36)

Objects are stored serialized in shm_put_var, so to find a value for memsize, you can use strlen(serialize($object_to_store_in_shm)).