------------------------------------------------------------------------------- Redis - a fast in memory database ------------------------------------------------------------------------------- The command line interface is "redis-cli" redis-cli -h redis [-n db] [command...] For a comprehensive list of commands see https://redis.io/commands Data types Summery https://redis.io/topics/data-types ------------------------------------------------------------------------------- General... Is it alive ping # -> PONG ping sweetie # -> "sweetie" hello 2 # check the protocol version Simple "String" Value set some_key_name "some value" get some_key_name setnx some_key_name "default" # set if is does not exist yet Strings strlen some_key_name # -> 10 setrange some_key_name 4 "_" # replace the space with underscore (index from 0) Integers decr {key} decrby {key} {value} incr {key} incrby {key} {value} BitFields on "strings" bitcount {key} [start end] # count set bits in string bitfield {key} [GET type offset] # arbatry bitfield operations on string bitop {operation} {destkey} {key} # bit wise operation bitpop key bit [start] [end] # find ifrst set or clear bit in string Hash arrays hmset userinfo username anthony password mypwd sex male hkeys userinfo hexists userinfo username hgetall userinfo hget userinfo username # get just that field hstrlen userinfo username # length of value in field hset userinfo age 56 # add or update a field hsetnx userinfo age 56 # add field only if does not exist hincrby userinfo age 1 # increase an integer field hincrbyfloat {key} {field} {value} # increase an float field hdel userinfo password # remove a field Lists (also used for task-queues) lpush fruit apples lpush fruit oranges rpush fruit strawberry lpushx fruit banana # push on list, if list exists linsert fruit BEFORE apples grapes # -> 5 add grapes before apples reports list size # length of list llen fruit lrange fruit 0 -1 # list whole list # "banana" is at top (left most), "strawberry" at bottom (right most) # NOTE indexes can be negative, meaning count from right side (end) lindex fruit 0 # -> "banana" - the top element (no pop) lindex fruit -1 # -> "strawberry" - the last element (no pop) # from version 6.0.6 (current at time of writing is 5.0.9) lpos fruit apples # -> 3 position of the apple in the list lpop fruit # -> banana - removed from list rpop fruit # -> strawberry # Blocking pops, wait if list is empty # blpop brpop lrem fruit 1 apples # -> 1 remove 1 apple from list, return number removed ltrim fruit 1 3 # removes all except those 1 to 3 (remove 0 and 4th onwards) Sets sadd colors red sadd colors green sadd colors blue smembers colors Sorted sets (weighted sets) zadd pokemon 0 pikatu zadd pokemon 5 granbull zadd pokemon 3 meowth zcard pokemon # -> 3 0 number of elements in set zrange pokemon (0 5 BYSCORE # elements in with 0 < score <= 5 EG: exclude 0 zrangebyscore pokemon 0 10 # pikatu is first, meowth is in the middle bzpopmin # pop lowest score (pikatu), block if none available (priority TaskQ) bzpopmax # pop hightest score (meowth), block if none available (priority TaskQ) Geographic Location Sets geoadd {key} {lat} {long} geopos goedist georadius goeradiusbymemebr goesearch goesearchstore geohash --------------------- Key Handling Existance keys {glob pattern} exists {key} # return 0 or 1 rename {key} {newkey} renamex {key} {newkey} # only rename if new key does not exist! type {key} # string, hash, list, set, zset randomkey # get return the name of a key at random from redis Dump a key in a raw form dump fruit Delete elements del some_key_name userinfo fruit colors pokemon # return count of values removed Time limit expire {key} {age in seconds} pexpire {key} {age in ms} expireat {key} {unix timestamp} # unix time in seconds pexpireat {key} {unix timestamp in milliseconds} persist {key} # remove the expiry from key ttl {key} # get time to live in seconds -1 = forever pttl {key} # get time to live in ms setex {key} {time in sec} {value} # set key and expiry at same time psetex {key} {time in ms} {value} Databases a group of keys seperate to main area Databases are just a number from 1 to 16 (by default) move {key} {db} dbsize # number of keys in database flushdb # remove all keys from current DB flushall # remove all keys from all DB (reset system) ***** Transactions (make a sequence atomic) mutli # start a block of commands ... exec # execute the commands in the block discard # abort the commands, don't apply Commands command # list of available commands command count # count of availabel commands command getkeys # ??? command info {command} # help about a command Configs config get * # get all values config get port config get loglevel About server info # information about available resources info keyspace # number of databases and keys in use bgsave # ensure data is saved onto disk (after important updates) ------------------------------------------------------------------------------- General tutorials on redis Compare python data types with redis (examples of same thing in both) Part 1: Basically using it as a Distributed Database https://realpython.com/python-redis/ Part 2 : Expires and complete data data = json.dumps(variable) variable = json.loads(data) =============================================================================== General task queues This is designed to ensure all tasks get tracked and completed https://medium.com/faun/how-to-monitor-your-redis-based-job-queue-system-c518b34a7e7a Three queues (tasks are migrated across queue lists in sequence) waiting -> worker_processing -> failed Taskdata on the queue contain the data... uuid -> data for the job issue time exec time finish time requeue count Sequence... client Lpush waiting {taskdata} worker BRpopLpush waiting worker_processing {taskdata returned} monitor watches worker_processing and requeues tasks that don't progress if difference between issue and exec gets big, more workers needed if exec - finish difference means job is too big if requeue count gets large - job has problems. ------------------------------------------------------------------------------- redis as a TaskQ This just uses a list! PHP examples... phqueue (redis task queue) https://github.com/bryanhelmig/phqueue Enqueuing https://github.com/bryanhelmig/phqueue/blob/master/tasks.php (see add_task() Worker loop https://github.com/bryanhelmig/phqueue/blob/master/worker.php Using Predis https://github.com/predis/predis/tree/main Example redis (from php) https://github.com/predis/predis/tree/main/examples --- rpush queue json_data # client: push element on end json_data = blpop queue 0 # Worker: blocking pop of first element lrange queue 0 -1 # list the whole queue # Push a Task $queue = 'queue_name' $data = json_encode(Array('task_name'=> $task, 'params' => Array('arg1'=>$value1, 'arg2'=>$value2))); rpush( $queue, $data ); # Pop a Task class Tasks { public function email_friends($params) { $user_id = $params->user_id; $message = $params->message; ... } } $tasks = new Tasks(); $queue = 'queue_name' while (true) { $data = blpop( $queue, 0) # If no data... Loop if (!$data) { continue; } $json = json_decode($data[1]); $task_name = $json->task_name; $params = $json->params; # call the requested function call_user_func(array($tasks, $task_name), $params); } ------------------------------------------------------------------------------- Python Redis https://realpython.com/python-redis/ And more at bottom of page # python -m pip install redis import redis Pipelines Used to group commands into units with r.pipeline() as pipe: for h_id, hat in hats.items(): pipe.hmset(h_id, hat) pipe.execute() # python -m pip install hiredis # on top of redis import hiredis # faster, lower level redis protocol ------------------------------------------------------------------------------- Perl API See "store/perl/redis", for storing complex data, and subscriptions -------------------------------------------------------------------------------