Selector Routines

class memcached2.SelectorBase

Select which server to use.

These classes implement a variety of algorithms for determining which server to use, based on the key being stored.

The selection is done based on a key_hash, as returned by the memcached2.HasherBase.hash() function.

Normally, the python-memcached2 classes will automatically pick a selector to use. However, for special circumstances you may wish to use a specific Selector or develop your own.

This is an abstract base class, here largely for documentation purposes. Selector sub-classes such as SelectorFirst and SelectorRehashDownServers, implement a select method which does all the work.

See select() for details of implementing a subclass.

select(server_uri_list, hasher, key, server_pool)

Select a server bashed on the key_hash.

Given the list of servers and a hash of of key, determine which of the servers this key is associated with on.

Parameters:
  • server_uri_list (list of server URIs) – A list of the server URIs to select among.
  • hasher (memcache2.HasherBase.hash().) – Hasher function, such as memcache2.HasherBase.hash().
  • key (str) – The key to hash.
  • server_pool (ServerPool object.) – (None) A server connection pool. If not specified, a global pool is used.
Returns:

string – The server_uri to use.

Raises:

NoAvailableServers

class memcached2.SelectorFirst

Server selector that only returns the first server. Useful when there is only one server to select amongst.

select(server_uri_list, hasher, key, server_pool)

See memcached2.SelectorBase.select() for details of this function.

class memcached2.SelectorRehashDownServers(hashing_retries=10)

Select a server, if it is down re-hash up to hashing_retries times.

This was the default in the original python-memcached module. If the server that a key is housed on is down, it will re-hash the key after adding an (ASCII) number of tries to the key and try that server.

This is most suitable if you want to inter-operate with the old python-memcache client.

If no up server is found after hashing_retries attempts, memcached2.NoAvailableServers is raised.

Parameters:hashing_retries (int) – Retry hashing the key looking for another server this many times.
select(server_uri_list, hasher, key, server_pool)

See memcached2.SelectorBase.select() for details of this function.

class memcached2.SelectorFractalSharding

On a down server, re-partition that servers keyspace to other servers.

This uses an algorithm that basically maps every key in the keyspace to a list of the servers that will answer queries for it. The first available server in that list will be used. The list is such that the keys that map to a server when it is up will get distributed across other servers evenly, stabally, and predictably.

I called it Fractal because when a server is down you dig deeper and see a new level of complexity in the keyspace mapping.

select(server_uri_list, hasher, key, server_pool)

See memcached2.SelectorBase.select() for details of this function.

class memcached2.SelectorConsistentHashing(total_buckets=None)

Predictably select a server, even if its normal server is down.

This implements the Consistent Hash algorithm as http://en.wikipedia.org/wiki/Consistent_hashing

This is done by splitting the key-space up into a number of buckets (more than the number of servers but probably no more than the number of servers squared). See Wikipedia for details on how this algorithm operates.

The downside of this mechanism is that it requires building a fairly large table at startup, so it is not suited to short lived code. It also is fairly expensive to add and remove servers from the pool (not implemented in this code). Note that it is NOT expensive to fail a server, only to completely remove it.

Parameters:total_buckets (int) – How many buckets to create. Smaller values decrease the startup overhead, but also mean that a down server will tend to not evenly redistribute it’s load across other servers. The default value of None means the default value of the number of servers squared.
select(server_uri_list, hasher, key, server_pool)

See memcached2.SelectorBase.select() for details of this function.