Low-Level Memcache() Interface

Introduction

This is a low-level interface to a group of memcache servers. This code tends to either return the requested data, or raises an exception if the data is not available or there is any sort of an error. If you want high level control, this is probably the interface for you. However, if you want something easy, like the old python-memcached module, you will want to wait for the higher level interfaces to be implemented.

Examples

Basic get() and exception example:

>>> import memcached2
>>> memcache = memcached2.Memcache(('memcached://localhost/',))
>>> try:
...    result = memcache.get('session_id')
...    print('Got cached results: {0}'.format(repr(result)))
... except memcached2.NoValue:
...    print('Cached value not available, need to recompute it')
...
Cached value not available, need to recompute it

Demonstrating set(), get() and ValueSuperStr:

>>> memcache.set('session_id', 'TEST SESSSION DATA')
>>> result = memcache.get('session_id')
>>> print('Got cached results: {0}'.format(repr(result)))
Got cached results: 'TEST SESSSION DATA'
>>> result.key
'session_id'
>>> result.flags
0

Example of get_multi() to retrieve multiple keys quickly:

>>> memcache.set('foo', '1')
>>> memcache.set('bar', '2')
>>> result = memcache.get_multi(['foo', 'bar', 'baz'])
>>> result.get('foo')
'1'
>>> result.get('bar')
'2'
>>> result.get('baz')
None

Usage of cache() to automatically cache values:

>>> numbers = range(10)
>>> calculate = lambda x: str(numbers.pop())
>>> memcache.flush_all()
>>> memcache.cache('foo', calculate)
'9'
>>> memcache.cache('foo', calculate)
'9'
>>> memcache.set('foo', 'hello')
>>> memcache.cache('foo', calculate)
'hello'
>>> memcache.flush_all()
>>> memcache.cache('foo', calculate)
'8'

Showing flags and expiration time and touch():

>>> memcache.set('foo', 'xXx', flags=12, exptime=30)
>>> result = memcache.get('foo')
>>> result
'xXx'
>>> result.key
'foo'
>>> result.flags
12
>>> import time
>>> time.sleep(30)
>>> result = memcache.get('foo')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "memcached2.py", line 411, in get
    raise NoValue()
memcached2.NoValue
>>> memcache.set('foo', 'bar', exptime=1)
>>> memcache.touch('foo', exptime=30)
>>> time.sleep(2)
>>> memcache.get('foo')
'bar'

Usage of replace(), append(), and prepend():

>>> memcache.replace('unset_key', 'xyzzy')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "memcached2.py", line 502, in replace
    self._storage_command(command, key)
  File "memcached2.py", line 945, in _storage_command
    raise NotStored()
memcached2.NotStored
>>> memcache.set('unset_key', 'old_data', exptime=30)
>>> memcache.replace('unset_key', 'xyzzy')
>>> memcache.get('unset_key')
'xyzzy'
>>> memcache.append('unset_key', '>>>')
>>> memcache.prepend('unset_key', '<<<')
>>> memcache.get('unset_key')
'<<<xyzzy>>>'

Example of using CAS (Check And Set) atomic operations:

>>> memcache.set('foo', 'bar')
>>> result = memcache.get('foo', get_cas=True)
>>> result.cas_unique
5625
>>> memcache.set('foo', 'baz', cas_unique=result.cas_unique)
>>> memcache.get('foo', get_cas=True)
'baz'
>>> memcache.set('foo', 'qux', cas_unique=result.cas_unique)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "memcached2.py", line 464, in set
    self._storage_command(command, key)
  File "memcached2.py", line 947, in _storage_command
    raise CASFailure()
memcached2.CASFailure
>>> memcache.get('foo', get_cas=True)
'baz'

Usage of incr()/decr():

>>> memcache.incr('incrtest', 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "memcached2.py", line 878, in incr
    return self._incrdecr_command(command, key)
  File "memcached2.py", line 915, in _incrdecr_command
    raise NotFound()
memcached2.NotFound
>>> memcache.set('incrtest', 'a')
>>> memcache.incr('incrtest', 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "memcached2.py", line 878, in incr
    return self._incrdecr_command(command, key)
  File "memcached2.py", line 919, in _incrdecr_command
    raise NonNumeric()
memcached2.NonNumeric
>>> memcache.set('incrtest', '1')
>>> memcache.incr('incrtest', 1)
2
>>> memcache.decr('incrtest', 1)
1
>>> memcache.get('incrtest')
'1'

Statistics sample information:

>>> import pprint
>>> pprint.pprint(memcache.stats())
[{'accepting_conns': '1',
  'auth_cmds': 0,
  'auth_errors': 0,
  'bytes': 201,
  'bytes_read': 173542,
  'bytes_written': 516341,
  'cas_badval': 49,
  'cas_hits': 49,
  'cas_misses': 0,
  'cmd_flush': 1154,
  'cmd_get': 880,
  'cmd_set': 5778,
  'cmd_touch': '148',
  'conn_yields': 0,
  'connection_structures': 9,
  'curr_connections': 5,
  'curr_items': 3,
  'decr_hits': 49,
  'decr_misses': 48,
  'delete_hits': 49,
  'delete_misses': 49,
  'evicted_unfetched': 0,
  'evictions': 0,
  'expired_unfetched': 0,
  'get_hits': '681',
  'get_misses': '199',
  'hash_bytes': 262144,
  'hash_is_expanding': '0',
  'hash_power_level': 16,
  'incr_hits': 49,
  'incr_misses': 49,
  'libevent': '2.0.19-stable',
  'limit_maxbytes': 67108864,
  'listen_disabled_num': '0',
  'pid': 22356,
  'pointer_size': 32,
  'reclaimed': 0,
  'reserved_fds': 20,
  'rusage_system': 7.568473,
  'rusage_user': 8.904556,
  'threads': 4,
  'time': 1366722131,
  'total_connections': 1545,
  'total_items': 5634,
  'touch_hits': 98,
  'touch_misses': 50,
  'uptime': 370393,
  'version': '1.4.14'}]
>>> pprint.pprint(memcache.stats_settings())
[{'auth_enabled_sasl': 'no',
  'binding_protocol': 'auto-negotiate',
  'cas_enabled': True,
  'chunk_size': 48,
  'detail_enabled': False,
  'domain_socket': 'NULL',
  'evictions': 'on',
  'growth_factor': 1.25,
  'hashpower_init': 0,
  'inter': '127.0.0.1',
  'item_size_max': 1048576,
  'maxbytes': 67108864,
  'maxconns': 1024,
  'maxconns_fast': False,
  'num_threads': 4,
  'num_threads_per_udp': 4,
  'oldest': 216734,
  'reqs_per_event': 20,
  'slab_automove': False,
  'slab_reassign': False,
  'stat_key_prefix': ':',
  'tcp_backlog': 1024,
  'tcpport': 11211,
  'udpport': 11211,
  'umask': 700,
  'verbosity': 0}]
>>> pprint.pprint(memcache.stats_items())
[{'1': {'age': 766,
        'evicted': 0,
        'evicted_nonzero': 0,
        'evicted_time': 0,
        'evicted_unfetched': 0,
        'expired_unfetched': 0,
        'number': 3,
        'outofmemory': 0,
        'reclaimed': 0,
        'tailrepairs': 0}}]
>>> pprint.pprint(memcache.stats_sizes())
[[(64, 1), (96, 2)]]
>>> pprint.pprint(memcache.stats_slabs())
[{'active_slabs': 1,
  'slabs': {'1': {'cas_badval': 49,
                  'cas_hits': 49,
                  'chunk_size': 80,
                  'chunks_per_page': 13107,
                  'cmd_set': 5778,
                  'decr_hits': 49,
                  'delete_hits': 49,
                  'free_chunks': 13104,
                  'free_chunks_end': 0,
                  'get_hits': 681,
                  'incr_hits': 49,
                  'mem_requested': 201,
                  'total_chunks': 13107,
                  'total_pages': 1,
                  'touch_hits': 98,
                  'used_chunks': 3}},
  'total_malloced': 1048560}]

How to delete(), flush_all(), and close() the connection:

>>> memcache.delete('foo')
>>> memcache.flush_all()
>>> memcache.close()

Object Documentation

class memcached2.Memcache(servers, value_wrapper=None, selector=None, hasher=None, server_pool=None)

Create a new memcache connection, to the specified servers.

The list of servers, specified by URL, are consulted based on the hash of the key, effectively “sharding” the key space.

This is a low-level memcache interface. This interface will raise exceptions when backend connections occur, allowing a program full control over handling of connection problems.

Example:

>>> from memcached2 import *                               # noqa
>>> mc = Memcache(['memcached://localhost:11211/'])
>>> mc.set('foo', 'bar')
>>> mc.get('foo')
'bar'

Extensive examples including demonstrations of the statistics output is available in the documentation for Memcache Examples

Parameters:
  • servers (list) – One or more server URIs of the form: “memcache://hostname[:port]/”
  • value_wrapper (ValueSuperStr or compatible object.) – (None) This causes values returned to be wrapped in the passed class before being returned. For example ValueSuperStr implements many useful additions to the string return.
  • selector (SelectorBase) – (None) This code implements the server selector logic. If not specified, the default is used. The default is to use SelectorFirst if only one server is specified, and SelectorRehashDownServers if multiple servers are given.
  • hasher (HasherBase) – (None) A “Hash” object which takes a key and returns a hash for persistent server selection. If not specified, it defaults to HasherZero if there is only one server specified, or HasherCMemcache otherwise.
  • server_pool (ServerPool object.) – (None) A server connection pool. If not specified, a global pool is used.
add(key, value, flags=0, exptime=0)

Store, but only if the server doesn’t already hold data for it.

Parameters:
  • key (str) – Key used to store value in memcache server and hashed to determine which server is used.
  • value (str) – Value stored in memcache server for this key.
  • flags (int (32 bits)) – If specified, the same value will be provided on get().
  • exptime (int) – If non-zero, it specifies the expriation time, in seconds, for this value.
append(key, value)

Store data after existing data associated with this key.

Parameters:
  • key (str) – Key used to store value in memcache server and hashed to determine which server is used.
  • value (str) – Value stored in memcache server for this key.
close()

Close the connection to all the backend servers.

decr(key, value=1)

Decrement the value for the key, treated as a 64-bit unsigned value.

Parameters:
  • key (str) – Key used to store value in memcache server and hashed to determine which server is used.
  • value (int (64 bit)) – A numeric value (default=1) to add to the existing value.
Returns:

int – (64 bits) The new value after the decrement.

Raises:

NotFound, NonNumeric, NotImplementedError

delete(key)

Delete the key if it exists.

Parameters:key (str) – Key used to store value in memcache server and hashed to determine which server is used.
Returns:Boolean indicating if key was deleted.
Raises:NotImplementedError, NoAvailableServers
flush_all()

Flush the memcache servers.

Note

An attempt is made to connect to all backend servers before running this command.

Raises:NotImplementedError
get(key, get_cas=False)

Retrieve the specified key from a memcache server.

Parameters:
  • key (str) – The key to lookup in the memcache server.
  • get_cas (bool) – If True, the “cas unique” is queried and the return object has the “cas_unique” attribute set.
Returns:

String, or “value_wrapper” as specified during object creation such as ~memcached2.ValueSuperStr.

Raises:

NoValue, NotImplementedError, NoAvailableServers

incr(key, value=1)

Increment the value for the key, treated as a 64-bit unsigned value.

Parameters:
  • key (str) – Key used to store value in memcache server and hashed to determine which server is used.
  • value (int (64 bit)) – A numeric value (default=1) to add to the existing value.
Returns:

int – (64 bits) The new value after the increment.

Raises:

NotFound, NonNumeric, NotImplementedError

prepend(key, value)

Store data before existing data associated with this key.

Parameters:
  • key (str) – Key used to store value in memcache server and hashed to determine which server is used.
  • value (str) – Value stored in memcache server for this key.
replace(key, value, flags=0, exptime=0)

Store data, but only if the server already holds data for it.

Parameters:
  • key (str) – Key used to store value in memcache server and hashed to determine which server is used.
  • value (str) – Value stored in memcache server for this key.
  • flags (int (32 bits)) – If specified, the same value will be provided on get().
  • exptime (int) – If non-zero, it specifies the expriation time, in seconds, for this value.
set(key, value, flags=0, exptime=0, cas_unique=None)

Set a key to the value in the memcache server.

Parameters:
  • key (str) – Key used to store value in memcache server and hashed to determine which server is used.
  • value (str) – Value stored in memcache server for this key.
  • flags (int (32 bits)) – If specified, the same value will be provided on get().
  • exptime (int) – If non-zero, it specifies the expriation time, in seconds, for this value.
  • cas_unique (int (64 bits)) – If specified as the 64-bit integer from get() with cas_unique=True, the value is only stored if the value has not been updated since the get() call.
stats()

Get general statistics about memcache servers.

Examples of the results of this function is available in the documentation as Memcache Statistics Examples

Note

An attempt is made to connect to all servers before issuing this command.

Returns:list – The statistics data is a dictionary of key/value pairs representing information about the server.

This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.

stats_items()

Get statistics about item storage per slab class from the memcache servers.

Examples of the results of this function is available in the documentation as Memcache Statistics Examples

Note

An attempt is made to connect to all servers before issuing this command.

Returns:list – The statistic information is a dictionary keyed by the “slab class”, with the value another dictionary of key/value pairs representing the slab information.

This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.

stats_settings()

Gets statistics about settings (primarily from processing command-line arguments).

Examples of the results of this function is available in the documentation as Memcache Statistics Examples

Note

An attempt is made to connect to all servers before issuing this command.

Returns:list – The statistic information is a dictionary of key/value pairs.

This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.

stats_sizes()

Get statistics about object sizes.

Examples of the results of this function is available in the documentation as Memcache Statistics Examples

Warning

This operation locks the cache while it iterates over all objects. Returns a list of (size,count) tuples received from the server.

Note

An attempt is made to connect to all servers before issuing this command.

Returns:list – Each statistic is a dictionary of of size:count where the size is rounded up to 32-byte ranges.

This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.

stats_slabs()

Gets information about each of the slabs created during memcached runtime. Returns a dictionary of slab IDs, each contains a dictionary of key/value pairs for that slab.

Examples of the results of this function is available in the documentation as Memcache Statistics Examples

Note

An attempt is made to connect to all servers before issuing this command.

Returns:list – The statistic information is a dictionary keyed by the “slab class”, with the value another dictionary of key/value pairs representing statistic information about each of the slabs created during the memcace runtime.

This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.

touch(key, exptime)

Update the expiration time on an item.

Parameters:
  • key (str) – Key used to store value in memcache server and hashed to determine which server is used.
  • exptime (int) – If non-zero, it specifies the expriation time, in seconds, for this value. Note that setting exptime=0 causes the item to not expire based on time.
Raises:

NotFound, NotImplementedError, NoAvailableServers