Low-Level Memcache() Interface¶
Introduction¶
This is a low-level interface to a group of memcache servers. This code tends to either return the requested data, or raises an exception if the data is not available or there is any sort of an error. If you want high level control, this is probably the interface for you. However, if you want something easy, like the old python-memcached module, you will want to wait for the higher level interfaces to be implemented.
Examples¶
Basic get()
and exception example:
>>> import memcached2
>>> memcache = memcached2.Memcache(('memcached://localhost/',))
>>> try:
... result = memcache.get('session_id')
... print('Got cached results: {0}'.format(repr(result)))
... except memcached2.NoValue:
... print('Cached value not available, need to recompute it')
...
Cached value not available, need to recompute it
Demonstrating set()
,
get()
and
ValueSuperStr
:
>>> memcache.set('session_id', 'TEST SESSSION DATA')
>>> result = memcache.get('session_id')
>>> print('Got cached results: {0}'.format(repr(result)))
Got cached results: 'TEST SESSSION DATA'
>>> result.key
'session_id'
>>> result.flags
0
Example of get_multi()
to retrieve multiple
keys quickly:
>>> memcache.set('foo', '1')
>>> memcache.set('bar', '2')
>>> result = memcache.get_multi(['foo', 'bar', 'baz'])
>>> result.get('foo')
'1'
>>> result.get('bar')
'2'
>>> result.get('baz')
None
Usage of
cache()
to automatically cache values:
>>> numbers = range(10)
>>> calculate = lambda x: str(numbers.pop())
>>> memcache.flush_all()
>>> memcache.cache('foo', calculate)
'9'
>>> memcache.cache('foo', calculate)
'9'
>>> memcache.set('foo', 'hello')
>>> memcache.cache('foo', calculate)
'hello'
>>> memcache.flush_all()
>>> memcache.cache('foo', calculate)
'8'
Showing flags and expiration time and touch()
:
>>> memcache.set('foo', 'xXx', flags=12, exptime=30)
>>> result = memcache.get('foo')
>>> result
'xXx'
>>> result.key
'foo'
>>> result.flags
12
>>> import time
>>> time.sleep(30)
>>> result = memcache.get('foo')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "memcached2.py", line 411, in get
raise NoValue()
memcached2.NoValue
>>> memcache.set('foo', 'bar', exptime=1)
>>> memcache.touch('foo', exptime=30)
>>> time.sleep(2)
>>> memcache.get('foo')
'bar'
Usage of replace()
,
append()
, and
prepend()
:
>>> memcache.replace('unset_key', 'xyzzy')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "memcached2.py", line 502, in replace
self._storage_command(command, key)
File "memcached2.py", line 945, in _storage_command
raise NotStored()
memcached2.NotStored
>>> memcache.set('unset_key', 'old_data', exptime=30)
>>> memcache.replace('unset_key', 'xyzzy')
>>> memcache.get('unset_key')
'xyzzy'
>>> memcache.append('unset_key', '>>>')
>>> memcache.prepend('unset_key', '<<<')
>>> memcache.get('unset_key')
'<<<xyzzy>>>'
Example of using CAS (Check And Set) atomic operations:
>>> memcache.set('foo', 'bar')
>>> result = memcache.get('foo', get_cas=True)
>>> result.cas_unique
5625
>>> memcache.set('foo', 'baz', cas_unique=result.cas_unique)
>>> memcache.get('foo', get_cas=True)
'baz'
>>> memcache.set('foo', 'qux', cas_unique=result.cas_unique)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "memcached2.py", line 464, in set
self._storage_command(command, key)
File "memcached2.py", line 947, in _storage_command
raise CASFailure()
memcached2.CASFailure
>>> memcache.get('foo', get_cas=True)
'baz'
>>> memcache.incr('incrtest', 1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "memcached2.py", line 878, in incr
return self._incrdecr_command(command, key)
File "memcached2.py", line 915, in _incrdecr_command
raise NotFound()
memcached2.NotFound
>>> memcache.set('incrtest', 'a')
>>> memcache.incr('incrtest', 1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "memcached2.py", line 878, in incr
return self._incrdecr_command(command, key)
File "memcached2.py", line 919, in _incrdecr_command
raise NonNumeric()
memcached2.NonNumeric
>>> memcache.set('incrtest', '1')
>>> memcache.incr('incrtest', 1)
2
>>> memcache.decr('incrtest', 1)
1
>>> memcache.get('incrtest')
'1'
Statistics sample information:
>>> import pprint
>>> pprint.pprint(memcache.stats())
[{'accepting_conns': '1',
'auth_cmds': 0,
'auth_errors': 0,
'bytes': 201,
'bytes_read': 173542,
'bytes_written': 516341,
'cas_badval': 49,
'cas_hits': 49,
'cas_misses': 0,
'cmd_flush': 1154,
'cmd_get': 880,
'cmd_set': 5778,
'cmd_touch': '148',
'conn_yields': 0,
'connection_structures': 9,
'curr_connections': 5,
'curr_items': 3,
'decr_hits': 49,
'decr_misses': 48,
'delete_hits': 49,
'delete_misses': 49,
'evicted_unfetched': 0,
'evictions': 0,
'expired_unfetched': 0,
'get_hits': '681',
'get_misses': '199',
'hash_bytes': 262144,
'hash_is_expanding': '0',
'hash_power_level': 16,
'incr_hits': 49,
'incr_misses': 49,
'libevent': '2.0.19-stable',
'limit_maxbytes': 67108864,
'listen_disabled_num': '0',
'pid': 22356,
'pointer_size': 32,
'reclaimed': 0,
'reserved_fds': 20,
'rusage_system': 7.568473,
'rusage_user': 8.904556,
'threads': 4,
'time': 1366722131,
'total_connections': 1545,
'total_items': 5634,
'touch_hits': 98,
'touch_misses': 50,
'uptime': 370393,
'version': '1.4.14'}]
>>> pprint.pprint(memcache.stats_settings())
[{'auth_enabled_sasl': 'no',
'binding_protocol': 'auto-negotiate',
'cas_enabled': True,
'chunk_size': 48,
'detail_enabled': False,
'domain_socket': 'NULL',
'evictions': 'on',
'growth_factor': 1.25,
'hashpower_init': 0,
'inter': '127.0.0.1',
'item_size_max': 1048576,
'maxbytes': 67108864,
'maxconns': 1024,
'maxconns_fast': False,
'num_threads': 4,
'num_threads_per_udp': 4,
'oldest': 216734,
'reqs_per_event': 20,
'slab_automove': False,
'slab_reassign': False,
'stat_key_prefix': ':',
'tcp_backlog': 1024,
'tcpport': 11211,
'udpport': 11211,
'umask': 700,
'verbosity': 0}]
>>> pprint.pprint(memcache.stats_items())
[{'1': {'age': 766,
'evicted': 0,
'evicted_nonzero': 0,
'evicted_time': 0,
'evicted_unfetched': 0,
'expired_unfetched': 0,
'number': 3,
'outofmemory': 0,
'reclaimed': 0,
'tailrepairs': 0}}]
>>> pprint.pprint(memcache.stats_sizes())
[[(64, 1), (96, 2)]]
>>> pprint.pprint(memcache.stats_slabs())
[{'active_slabs': 1,
'slabs': {'1': {'cas_badval': 49,
'cas_hits': 49,
'chunk_size': 80,
'chunks_per_page': 13107,
'cmd_set': 5778,
'decr_hits': 49,
'delete_hits': 49,
'free_chunks': 13104,
'free_chunks_end': 0,
'get_hits': 681,
'incr_hits': 49,
'mem_requested': 201,
'total_chunks': 13107,
'total_pages': 1,
'touch_hits': 98,
'used_chunks': 3}},
'total_malloced': 1048560}]
How to delete()
,
flush_all()
, and
close()
the connection:
>>> memcache.delete('foo')
>>> memcache.flush_all()
>>> memcache.close()
Object Documentation¶
-
class
memcached2.
Memcache
(servers, value_wrapper=None, selector=None, hasher=None, server_pool=None)¶ Create a new memcache connection, to the specified servers.
The list of servers, specified by URL, are consulted based on the hash of the key, effectively “sharding” the key space.
This is a low-level memcache interface. This interface will raise exceptions when backend connections occur, allowing a program full control over handling of connection problems.
Example:
>>> from memcached2 import * # noqa >>> mc = Memcache(['memcached://localhost:11211/']) >>> mc.set('foo', 'bar') >>> mc.get('foo') 'bar'
Extensive examples including demonstrations of the statistics output is available in the documentation for Memcache Examples
Parameters: - servers (list) – One or more server URIs of the form: “memcache://hostname[:port]/”
- value_wrapper (
ValueSuperStr
or compatible object.) – (None) This causes values returned to be wrapped in the passed class before being returned. For exampleValueSuperStr
implements many useful additions to the string return. - selector (
SelectorBase
) – (None) This code implements the server selector logic. If not specified, the default is used. The default is to useSelectorFirst
if only one server is specified, andSelectorRehashDownServers
if multiple servers are given. - hasher (
HasherBase
) – (None) A “Hash” object which takes a key and returns a hash for persistent server selection. If not specified, it defaults toHasherZero
if there is only one server specified, orHasherCMemcache
otherwise. - server_pool (
ServerPool
object.) – (None) A server connection pool. If not specified, a global pool is used.
-
add
(key, value, flags=0, exptime=0)¶ Store, but only if the server doesn’t already hold data for it.
Parameters: - key (str) – Key used to store value in memcache server and hashed to determine which server is used.
- value (str) – Value stored in memcache server for this key.
- flags (int (32 bits)) – If specified, the same value will be provided on
get()
. - exptime (int) – If non-zero, it specifies the expriation time, in seconds, for this value.
-
append
(key, value)¶ Store data after existing data associated with this key.
Parameters: - key (str) – Key used to store value in memcache server and hashed to determine which server is used.
- value (str) – Value stored in memcache server for this key.
-
close
()¶ Close the connection to all the backend servers.
-
decr
(key, value=1)¶ Decrement the value for the key, treated as a 64-bit unsigned value.
Parameters: - key (str) – Key used to store value in memcache server and hashed to determine which server is used.
- value (int (64 bit)) – A numeric value (default=1) to add to the existing value.
Returns: int – (64 bits) The new value after the decrement.
Raises: NotFound
,NonNumeric
,NotImplementedError
-
delete
(key)¶ Delete the key if it exists.
Parameters: key (str) – Key used to store value in memcache server and hashed to determine which server is used. Returns: Boolean indicating if key was deleted. Raises: NotImplementedError
,NoAvailableServers
-
flush_all
()¶ Flush the memcache servers.
Note
An attempt is made to connect to all backend servers before running this command.
Raises: NotImplementedError
-
get
(key, get_cas=False)¶ Retrieve the specified key from a memcache server.
Parameters: - key (str) – The key to lookup in the memcache server.
- get_cas (bool) – If True, the “cas unique” is queried and the return object has the “cas_unique” attribute set.
Returns: String, or “value_wrapper” as specified during object creation such as ~memcached2.ValueSuperStr.
Raises: NoValue
,NotImplementedError
,NoAvailableServers
-
incr
(key, value=1)¶ Increment the value for the key, treated as a 64-bit unsigned value.
Parameters: - key (str) – Key used to store value in memcache server and hashed to determine which server is used.
- value (int (64 bit)) – A numeric value (default=1) to add to the existing value.
Returns: int – (64 bits) The new value after the increment.
Raises: NotFound
,NonNumeric
,NotImplementedError
-
prepend
(key, value)¶ Store data before existing data associated with this key.
Parameters: - key (str) – Key used to store value in memcache server and hashed to determine which server is used.
- value (str) – Value stored in memcache server for this key.
-
replace
(key, value, flags=0, exptime=0)¶ Store data, but only if the server already holds data for it.
Parameters: - key (str) – Key used to store value in memcache server and hashed to determine which server is used.
- value (str) – Value stored in memcache server for this key.
- flags (int (32 bits)) – If specified, the same value will be provided on
get()
. - exptime (int) – If non-zero, it specifies the expriation time, in seconds, for this value.
-
set
(key, value, flags=0, exptime=0, cas_unique=None)¶ Set a key to the value in the memcache server.
Parameters: - key (str) – Key used to store value in memcache server and hashed to determine which server is used.
- value (str) – Value stored in memcache server for this key.
- flags (int (32 bits)) – If specified, the same value will be provided on
get()
. - exptime (int) – If non-zero, it specifies the expriation time, in seconds, for this value.
- cas_unique (int (64 bits)) – If specified as the 64-bit integer from
get()
with cas_unique=True, the value is only stored if the value has not been updated since theget()
call.
-
stats
()¶ Get general statistics about memcache servers.
Examples of the results of this function is available in the documentation as Memcache Statistics Examples
Note
An attempt is made to connect to all servers before issuing this command.
Returns: list – The statistics data is a dictionary of key/value pairs representing information about the server. This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.
-
stats_items
()¶ Get statistics about item storage per slab class from the memcache servers.
Examples of the results of this function is available in the documentation as Memcache Statistics Examples
Note
An attempt is made to connect to all servers before issuing this command.
Returns: list – The statistic information is a dictionary keyed by the “slab class”, with the value another dictionary of key/value pairs representing the slab information. This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.
-
stats_settings
()¶ Gets statistics about settings (primarily from processing command-line arguments).
Examples of the results of this function is available in the documentation as Memcache Statistics Examples
Note
An attempt is made to connect to all servers before issuing this command.
Returns: list – The statistic information is a dictionary of key/value pairs. This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.
-
stats_sizes
()¶ Get statistics about object sizes.
Examples of the results of this function is available in the documentation as Memcache Statistics Examples
Warning
This operation locks the cache while it iterates over all objects. Returns a list of (size,count) tuples received from the server.
Note
An attempt is made to connect to all servers before issuing this command.
Returns: list – Each statistic is a dictionary of of size:count where the size is rounded up to 32-byte ranges. This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.
-
stats_slabs
()¶ Gets information about each of the slabs created during memcached runtime. Returns a dictionary of slab IDs, each contains a dictionary of key/value pairs for that slab.
Examples of the results of this function is available in the documentation as Memcache Statistics Examples
Note
An attempt is made to connect to all servers before issuing this command.
Returns: list – The statistic information is a dictionary keyed by the “slab class”, with the value another dictionary of key/value pairs representing statistic information about each of the slabs created during the memcace runtime. This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.
-
touch
(key, exptime)¶ Update the expiration time on an item.
Parameters: - key (str) – Key used to store value in memcache server and hashed to determine which server is used.
- exptime (int) – If non-zero, it specifies the expriation time, in seconds, for this value. Note that setting exptime=0 causes the item to not expire based on time.
Raises: NotFound
,NotImplementedError
,NoAvailableServers