simplekv is an API for key-value store of binary data. Due to its basic interface, it is easy to implemented a large number of backends. simplekv‘s origins are in storing user-uploaded files on websites, but its low overhead and design should make it applicable for numerous other problems, an example is a session backend for the Flask framework.
Built upon the solid foundation are a few optional bells and whistles, such as automatic ID generation/hashing (in simplekv.idgen). A number of backends are available, ranging from FilesystemStore to support for Amazon S3 and Google Storage through BotoStore.
Faster in-memory stores suitable for session management are supported through the likes of RedisStore or MemcacheStore.
Here’s a simple example:
from simplekv.fs import FilesystemStore
store = FilesystemStore('./data')
store.put('key1', 'hello')
# will print "hello"
print store.get('key1')
# move the contents of a file to "key2" as efficiently as possible
store.put_file('key2', '/path/to/data')
Note that by changing the first two lines to:
from simplekv.memory.redisstore import RedisStore
import redis
store = RedisStore(redis.StrictRedis())
you could use the code exactly the same way, this time storing data inside a Redis database.
The smallest API supported by all backends.
Keys are ascii-strings with certain restrictions, guaranteed to be properly handled up to a length of at least 250 characters. Any function that takes a key as an argument raises a ValueError if the key is incorrect.
The regular expression for what constitutes a valid key is available as simplekv.VALID_KEY_REGEXP.
Checks if a key is present
| Parameters: | key – The key whose existence should be verified. |
|---|---|
| Raises: |
|
| Returns: | True if the key exists, False otherwise. |
Iterate over keys
| Raises IOError: | If there was an error accessing the store. |
|---|
Delete key and data associated with it.
If the key does not exist, no error is reported.
| Raises: |
|
|---|
Returns the key data as a string.
| Parameters: | key – Key to get |
|---|---|
| Raises: |
|
Write contents of key to file
Like put_file(), this method allows backends to implement a specialized function if data needs to be written to disk or streamed.
If file is a string, contents of key are written to a newly created file with the filename file. Otherwise, the data will be written using the write method of file.
| Parameters: |
|
|---|---|
| Raises: |
|
Return an Iterator over all keys currently in the store, in any order.
| Raises IOError: | If there was an error accessing the store. |
|---|
Return a list of keys currently in store, in any order
| Raises IOError: | If there was an error accessing the store. |
|---|
Open key for reading.
Returns a read-only file-like object for reading a key.
| Parameters: | key – Key to open |
|---|---|
| Raises: |
|
Store into key from file
Stores string data in key.
| Parameters: |
|
|---|---|
| Returns: | The key under which data was stored |
| Raises: |
|
Store into key from file on disk
Stores data from a source into key. file can either be a string, which will be interpretet as a filename, or an object with a read() method.
If the passed object has a fileno() method, it may be used to speed up the operation.
The file specified by file, if it is a filename, may be removed in the process, to avoid copying if possible. If you need to make a copy, pass the opened file instead.
| Parameters: |
|
|---|---|
| Returns: | The key under which data was stored |
| Raises: |
|
In addition to that, an extended base class is available that provides an extra method to support URL generation:
A KeyValueStore that supports getting a download URL for keys.
Returns a full external URL that can be used to retrieve key.
Does not perform any checks (such as if a key exists), other than whether or not key is a valid key.
| Parameters: | key – The key for which the url is to be generated |
|---|---|
| Raises ValueError: | |
| If the key is not valid. | |
| Returns: | A string containing a URL to access key |
This regular expression tests if a key is valid. Allowed are all alphanumeric characters, as well as !"`#$%&'()+,-.<=>?@[]^_{}~.
A compiled version of VALID_KEY_REGEXP.
Subclassing KeyValueStore is the fastest way to implement a new backend. It suffices to override the _delete(), iter_keys(), _open() and _put_file() methods, as all the other methods have default implementations that call these.
After that, you can override any number of underscore-prefixed methods with more specialized implementations to gain speed improvements.
Classes derived from KeyValueStore inherit a number of default implementations for the core API mehthods. Specifically, the delete(), get(), get_file(), keys(), open(), put(), put_file(), methods will each call the _check_valid_key() method if a key has been provided and then call one of the following protected methods:
Checks if a key is valid and raises a ValueError if its not.
When in need of checking a key for validity, always use this method if possible.
| Parameters: | key – The key to be checked |
|---|
Implementation for delete(). The default implementation will simply raise a NotImplementedError.
Implementation for get(). The default implementation will create a StringIO-buffer and then call _get_file().
| Parameters: | key – Key to be retrieved |
|---|
Write key to file-like object file. Either this method or _get_filename() will be called by get_file(). Note that this method does not accept strings.
| Parameters: |
|
|---|
Write key to file. Either this method or _get_file() will be called by get_file(). This method only accepts filenames and will open the file with a mode of wb, then call _get_file().
| Parameters: |
|
|---|
Default implementation for __contains__().
Determines whether or not a key exists by calling keys().
| Parameters: | key – Key to check existance of |
|---|
Open key for reading. Default implementation simply raises a NotImplementedError.
| Parameters: | key – Key to open |
|---|
Implementation for put(). The default implementation will create a StringIO-buffer and then call _put_file().
| Parameters: |
|
|---|
Store data from file-like object in key. Either this method or _put_filename() will be called by put_file(). Note that this method does not accept strings.
The default implementation will simply raise a NotImplementedError.
| Parameters: |
|
|---|
Store data from file in key. Either this method or _put_file() will be called by put_file(). Note that this method does not accept strings.
The default implementation will open the file in rb mode, then call _put_file().
| Parameters: |
|
|---|
Every call to a method on a KeyValueStore results in a single operation on the underlying backend. No guarantees are made above that, if you check if a key exists and then try to retrieve it, it may have already been deleted in between (instead, retrieve and catch the exception).