Carrier DB beta the world's most efficient key-value database

is your database running? you better go catch it.

Background Details: Creating the Best Database Ever and More Implementation Details
Carrier DB [intel] 2019-07-01-beta
Note: not available for arm64 platforms natively yet. Email us for interest.
Double Note: these releases are obviously slightly old, so also email us if you'd like new releases.

Download for macOS

Download for Linux

Why Carrier DB?

Carrier DB is the most memory efficient, secure, production oriented key-value database server.

Memory efficiency is an often ignored aspect of data storage. When you have multiple terabytes of expensive RAM, your budget cries every time you waste resources on unnecessary bloated data storage overhead.

Carrier DB is built with data efficiency and security from the ground up. Carrier DB uses custom memory-efficient data structures to store your in-memory data 200% to 1000% more efficiently than other in-memory databases (space savings based on data size vs. storage size considering all accounting metadata and data structure overhead).

Carrier DB implements design goals other in-memory databases completely ignore:

  • non-blocking operations
    • Retrieve the middle 200,000 elements of a 3 million long list? No problem. Carrier DB remains responsive to other requests the entire time.
    • Check the intersection of a dozen sets each with 1 million items? Carrier DB will remain responsive the entire time.
    • Save the union of 40 sets each with 2 million items into a new set? Carrier DB remains responsive.
  • security first
    • create multiple isolated or shared security domains by listening on multiple IP addresses
    • serve unique TLS certificates on each listening virtual network
    • serve multiple TLS certificates on the same virtual network for serving both RSA and ECDSA clients
    • apply access restrictions per virtual network
    • retrieve independent network statistics for each listening IP address
    • configure protocol support per virtual network for accepting either legacy memcached or legacy redis text protocols
  • memory efficiency
    • store 1 billion values with only 5 GB to 10 GB internal metadata overhead
      • for comparison: memcached wastes about 100 GB in overhead per billion values (around 1 GB overhead per million values stored). redis wastes anywhere from 50 GB to 200 GB per billion values stored depending on which inefficiently written data structure(s) get used.
    • cache efficiency
      • Carrier DB is designed with physically optimal data structure overhead so your expensive CPU caches remain primed with as much usable user data as possible.
  • scale-up and scale-down efficiency
    • Carrier DB improves every modern data architecture by reducing your hardware costs. Carrier DB is created using the most space efficient data structures possible letting you host 200% to 1000% more data per server with Carrier DB than with legacy in-memory key-value databases.
    • Carrier DB supports multiple deployment scenarios including, but not limited to: massively distributed edge computing, IoT, 5G access point accounting, micro-POPs, CPE, and everything else from MB to GB-sized mobile platforms all the way up to multi-TB data platforms.

What does Carrier DB support?

legacy memcached protocol

Carrier DB is an extension of the amazing Carrier Cache memcached replacement. Carrier DB supports all memcached commands from Carrier Cache in addition to dozens more features outlined below.

legacy redis protocol

Carrier DB also supports the legacy redis protocol. We are currently increasing the number of supported built-in data structures accessible using legacy protocols.

As of right now Carrier DB supports strings, lists, HLLs, maps/dicts/hashes, sets, qsets, and qnsets. More data structures are showing up all the time, so join our mailing list to keep updated.

fully multi-threaded server

Carrier DB is an extremely multi-threaded database.

Carrier DB can efficiently be configured for hundreds or thousands of cores using terabytes of RAM all running under a single OS process (Note: values inside Carrier DB are currently limited to 281 terabytes maximum per item. Please let us know if you require higher limits.).

in-server atomic data structure operations

Carrier DB combines complex data structures, low latency atomic updates, and a massively multi-threaded architecture so your server doesn't collapse, stall, or become unresponsive due to large queries or greedy clients.

non-blocking transport encryption and decryption

Encryption is a must-have feature these days. Any servers not giving you the option of connecting over TLS are being professionally irresponsible and opening up your company to increased liability. (Note: this was a more biting critique in 2019 when redis had still refused to implement TLS for 10 years because of single-maintainer feature prejudice.)

If your in-memory databases stores user information and you are not using TLS for client connections, you are likely not GDPR compliant.

To ensure best performance, Carrier DB uses independent decryption thread pools and encryption thread pools. Carrier DB concurrently decrypts requests while also encrypting replies giving you the fastest TLS performance possible.

most efficient memory usage

Carrier DB has conquered the tyranny of 64-bit platforms by minimizing pointers inside all data structures.

Carrier DB stores all data in various succinct data structures we've created solely for the purpose of low overhead in-memory data storage capable of growing to dozens of terabytes of RAM.

Carrier DB actually implements the smallest data structures physically possible. No other systems approach the memory efficiency of Carrier DB.

multiple network virtual hosting

Carrier DB enables serving data from multiple IP addresses. Each IP address can be configured with independent security, encryption, protocol, and performance options.

Each Carrier DB virtual network can:

  • bind to an IPv4 or IPv6 address
  • enable per-IP TLS encryption
  • offer clients both RSA and ECDSA certificates at the same time
  • specify client protocol
    • legacy memcached protocol
    • legacy redis protocol
  • set per-network access levels
    • lock all clients to read-only mode
    • allow clients to run the stats command
    • allow clients full admin access
    • disable all data access and only allow stats—you can expose only the statistics interface to external monitoring services without risking any reads or writes to your database
  • create a per-virtual-network namespace so all clients automatically get restricted to an inescapable security boundary

Current Status

Carrier DBbeta is ready for evaluation and production use.

Beta releases expect you to have familiarity with legacy memcached/redis commands. Carrier DB also shares the same scalable server architecture as Carrier Cache so configuration documentation is at Carrier Cache Tech Specs.

If you run into any problems, let us know so we can prioritize fixes for you.

Current Features

Most features of Carrier DB have unique implementation details and have little to no equivalent in other databases when measured against memory efficiency and security guarantees.

Here's an overview of released and in-progress Carrier DB features:

Released

  • all Carrier Cache features are included in Carrier DB
    • legacy memcached protocol support (all commands)
  • legacy redis protocol support (including legacy and unique commands)
  • unlimited virtual network support for multi-hosting and security
    • virtual networks can limit clients to a namespace
    • virtual networks support access controls so you can restrict stats, admin, and/or data access per listening server
    • TLS can be enabled per-virtual-network
      • Includes support for RSA and ECDSA certificates
      • Virtual networks can serve RSA and ECDSA certificates concurrently for highest client performance
      • Carrier DB manages TLS encryption and decryption concurrently for the fastest encrypted connections possible
  • optimized low overhead string storage
  • support for cas in legacy redis protocol operations
  • a unique countutf8codepoints string command
  • incrby accepts signed integers, unsigned integers and floating point all in one command.
  • optimized low overhead set storage:
    • Also implements ssubset to check subset properties and sequal to check if two sets are exactly equal
    • qset — deduplicating set storage with fast intersections
    • qnset — compact integer set storage with fast intersections
  • optimized low overhead list storage:
    • lrange implemented as a non-blocking command
  • optimized low overhead HyperLogLog storage:
    • hlladd implemented as a non-blocking command
  • unified namespace architecture
    • Carrier DB namespaces replace legacy redis "hashes"
    • Carrier DB namespaces support unlimited nesting depth
    • Carrier DB namespaces support all data types, so you can use all types inside all namespaces
    • Carrier DB namespaces support members with expire times
    • Deleting a namespace deletes all sub-keys including sub-namespaces
    • Carrier DB namespaces can be configured as inescapable security boundaries
  • extreme memory efficiency and security as primary design priorities
  • fully multi-threaded data access
  • JSON stats reporting
  • JSON client info reporting
  • Built-in JSON reply syntax for all commands
  • optionally launch a per-network script at startup to notify your infrastructure about server details or run startup data operations at every restart.

In Progress, Unreleased

  • coming soon sorted sets
  • coming soon concurrent persistence
  • coming soon replication+failover
  • geo types
  • client blocking operations (brpop, blpop, etc)
  • pubsub operations
  • background LRU maint
  • clustering
  • more admin flexibility

If your favorite command is currently missing, let us know what you need. We prioritize features based on request popularity.

Carrier DB 2019-07-01

New Features

  • Startup notification script per-network
    • Network configuration now supports a notify [path-to-program] directive
    • Your notify executable receives 5 arguments: [networkName] [listenAddress] [listenPort] [tls|notls] [protocol]
    • Use cases:
      • If you configure a network with port 0 (so Carrier DB picks a random usable port on startup), your notify script can tell your infrastructure where your DB can be contacted.
      • Automatically load an initial dataset on startup. Just connect using the startup notification arguments and push a full dataset automatically when the server starts.
    • Also supports additional directives for running notifier as a different user: notifyUser [system username] and notifyGroup [group name].
  • Namespaces now usable across a wider variety of contexts
    • Responsibility for namespace security boundary is now delegated to a new command: nslock.
    • Previously, once entering a namespace (with ns, cs, or hs), the namespace could not be escaped unless you had admin permissions.
    • Now namespaces can be entered (ns et al) and exited freely (nsreset), but exiting a namespace is limited to the depth specified by nslock.
    • nslock has two forms:
      • nslock - set current namespace to the depth you get if you run nsreset in the future.
      • nslock [ns|cs|hs] namespaceDepth... - lock client connection to the namespace (or cs or hs) specified.
    • The second form of nslock ns a namespace to lock clients is useful for the per-network namespace configuration directive as namespace "nslock ns my private namespace" so clients will be unable to nsreset to any level above my private namespace.
    • nslock prevents clients from reaching higher namespaces. Clients can always access deeper namespaces and clients can further nslock deeper namespaces, but once nslock runs on a client (or when specified as part of a network configuration), the client will be unable to escape to any higher namespaces.

Improved Features

  • Improved JSON compliance
  • Improved collective output representation
    • Results for collective retrieval using KEYS, VALS, GETALL, etc now returns an array of arrays populated by each worker independently.
    • The new return layout is easier for the server to generate and is easier for future scalability since a single Carrier DB instance can span multiple threads and multiple servers. Now each thread across each server can generate its own collective output list then the final return value is an array of aggregated collective output arrays allowing for better logical data clustering along with enhanced protocol performance.

Carrier DB 2019-06-23

New Features

  • JSON output support for all commands
    • Carrier DB can now reply with JSON syntax natively for any command.
    • Enable JSON output per-connection by running command SYNTAX JSON
      • Every reply for your client will now be an efficient JSON string instead of legacy redis protocol output.
    • The new JSON output capability works with all legacy redis clients. The JSON output is given to clients as a legacy redis protocol string, so all legacy redis clients will just see a string response to all commands when requested, but your code can decode the string as valid json for all operations.
    • Instead of running SYNTAX JSON on each connection, you can set a default reply syntax on virtual networks with per-network config directive syntax json (even if a network has a default syntax, each connection can run SYNTAX to change reply syntax at any time).
    • In addition to JSON, you can also request SYNTAX PYTHON or configure a network with syntax python to return set-type output using Python set syntax (JSON has no 'set' built-in type, so in JSON, sets are printed as arrays).
    • JSON is actually a much more efficient data representation than the legacy redis protocol. Using the new JSON reply syntax mode leads to significant throughput improvements (especially over TLS) when making highly structured or multi-element data requests since fewer bytes need to be transferred (or encrypted) back to clients.

Improved Features

  • Improved Per-Network Permission Configuration
    • Previous releases had 4 different boolean permission config directives (enableadmin, disabledata, enablestats, readonly), but those are now removed in favor of a simpler access directive.
    • Using the new per-network access config, you can enable access using a string with any combination of:
      • admin
      • stats
      • read
      • write
    • Examples:
      • restrict a listening interface to read-only data access: access "read"
      • allow typical data access: access "read write"
      • allow an external monitoring service to only access statistics without being able to read or write anything inside your database at all: access "stats"
      • enable full access to all access levels: access "admin stats read write"
      • In keeping with Carrier DB and Carrier Cache's security-oriented architecture, access must be specified for each listening interface. If no access is defined, the default is to not allow any data access.
  • Improved random-with-memory LRU accuracy
  • Improved runtime stability under adverse conditions
    • This Carrier DB update includes, all together now: bug fixes and performance enhancements!

Carrier DB 2019-06-12

New Features

  • Improved TLS performance
    • Carrier DB and Carrier Cache now automatically manage the complete lifecycle of TLS session tickets for highest performance and lowest overhead encrypted reconnect.
    • Ticket keys are auto-rotated out every six hours to balance high performance connections with long term data integrity.
    • Automatic TLS ticket rotation is a feature unique to Carrier DB and Carrier Cache! No other system provides automatic TLS ticket phase out without external intervention, so your data is always safest in the hands of Carrier DB.
    • What's the performance difference?
      • An ideal, zero-latency TLS session takes about 9 milliseconds to negotiate encryption.
      • The initial TLS handshake transfers about 3,000 bytes to a client just in protocol overhead.
      • Using properly maintained TLS tickets, resuming a previous session takes 0.4 milliseconds and transfers about 350 bytes.
      • TLS ticket session resumption is 22x faster than a full TLS negotiation! (assuming no network latency)
      • TLS ticket session resumption transfers 10x less data for connecting than a full TLS negotiation!
      • Test TLS connect-vs-resume speed yourself with: echo | openssl s_client -connect [targetHost]:[targetPort] -reconnect
      • Carrier DB and Carrier Cache handle the full lifecycle of TLS ticket keys by auto-expiring older ticket keys on a rotating basis so your clients always have the highest performance combined with the highest integrity possible.
  • Multiple TLS certificates per virtual network
    • Carrier DB now supports offering multiple certificates to clients on each listening port.
    • You can retain compatibility with older clients by using RSA certificates while simultaneously offering modern, faster, and more efficient ECDSA certs to newer clients.
    • Just list multiple certchain and privatekey configure statements per network and Carrier DB takes care of the rest.
  • Restrict clients to per-network namespaces
    • Virtual networks now support a namespace directive describing the namespace all clients will be locked into before running any commands.
    • Example:
      • Network A: namespace "nslock ns users" — restrict all clients on Network A to the users namespace
      • Network B: namespace "nslock ns stats" — restrict all clients on Network B to a stats namespace
      • Clients on both networks above are 100% isolated from viewing, writing, modifying, or deleting any data from the other network since namespaces in Carrier DB are full security boundaries.
  • Two new data structures for specialized memory savings
    • qset — optimized deduplicating set storage
      • qset auto-indexes set elements to store elements only once on your entire server no matter how many times they are used in qsets.
      • qset helps when you have many medium or large size repeated values across many sets and you need super efficient space savings combined with fast set intersections.
    • qnset — optimized integer-only set storage
      • qnset stores integer values in native binary format within range [-9671406556917033397649407, 19342813113834066795298815].
      • qnset supports the smallest integer storage possible for sets with large value distributions as well as supporting fast numeric intersection capabilities for large sets.

Improved Features

  • Improved TLS reloading
    • All TLS certificates can be reloaded from an admin-enabled network with admin tls reload
  • Improved safe shutdown mechanics
    • Shutdown can now be initiated from an admin-enabled network with:
      • admin shutdown — clean shutdown — listening ports are closed but current in-flight requests run to completion.
      • admin shutdown fast — instant shutdown — stop the server without regard for clients or state or consistency.
    • Safe shutdown can also be initiated by sending SIGTERM to the Carrier DB parent process
  • Improved logging architecture
    • Logging is now more robust and reliable if the logger child process is interrupted by another system processes.
  • Improved runtime stability under adverse conditions
    • This Carrier DB update includes, all together now: bug fixes and performance enhancements!

 

Total Specs for Carrier DB

Feature Notes
legacy memcached protocol

enable by adding protocol dmp to a network config block

For server setup, see: Carrier Cache Tech Specs.

legacy redis protocol

enable by adding protocol drp to a network config block

For server setup, see: Carrier Cache Tech Specs.

Full JSON Output For All Commands

enable for all clients by adding syntax json to a network config block

or enable per-connection by running command SYNTAX JSON

you can return to legacy redis protocol by running SYNTAX DRP or you can even switch to Python-formatted replies with SYNTAX PYTHON

Carrier DB JSON output is available in all legacy redis clients with no changes necessary since the JSON output is returned as legacy redis protocol strings.

legacy redis protocol [namespaces]

Note: namespaces are unique to Carrier DB and have no legacy redis equivalent.

command deviations
ns | namespace

In Carrier DB, maps/hashes/dicts are fully formed namespaces.

A namespace is a key-value map where keys can be of any type.

In legacy redis parlance, this means you can have hashes with sub-hashes and sub-lists and sub-sets and sub-HLLs and any other data type inside of them.

All key-value maps in Carrier DB are namespaces:

  • You can set individual expires timeouts on any value inside any nested namespace.
  • You can use any operation on any value inside a namespace.
    • for example, legacy redis has a very, very limited number of "hash" commands.
    • In legacy redis, even though hash values are just strings, you can't perform all string operations on them. In Carrier DB, you can perform any operation on any value at any level of any namespace without needing to limit your data model based on lack of server features.

Other useful features of namespaces:

  • automatic prefix compression
    • Instead of storing a billion keys of the form prefix:subtype:subvalue:subsubvalue which would eat 30 GB RAM, you can use ns prefix subtype subvalue subsubvalue and only store the prefix values once for all final-level keys.
  • automatic multi-CPU socket NUMA domain locality
    • The first key in the namespace depth determines a single storage location for all sub-keys (including sub-namespaces).
    • On multi-processor machines, namespaces bind your data to a NUMA domain to reduce data access latency.
    • In cluster deployments, namespaces provide a way to guarantee data will be served from the same node for faster operations on related keys.
  • automatic cleanup
    • If you delete a namespace, all nested keys and values of all types are automatically recursively deleted too.
    • These self-cleaning namespace deletes enable easy management of user data. Give each user a unique namespace, then one command can wipe all their nested live data from your DB when needed, which is great for professional responsibility and GDPR compliance.
  • security boundary
    • Once a client enters a namespace, they can escape to higher level namespaces by running nsreset unless the namespace has been restricted using nslock.
    • Clients can always select or create deeper namespaces, but they can't go up the namespace hierarchy to parents or peers above their locked depth.

For backwards compatibility with legacy redis, hash commands still work too. Legacy redis hash commands are mapped to enable reading from one deeper namespace than you have selected without entering the namespace itself.

These usages are equivalent:

  • ns users
  • ns unregistered
  • set usersAreFrom unknown
  • ns users
  • hset unregistered usersAreFrom unknown
  • ns users unregistered
  • set usersAreFrom unknown

ns accepts multiple arguments to automatically enter a nested namespace.

If you are in a namespace and you run ns again, you enter a child namespace of your current namespace.

If your namespace can't logically exist because one of your requested elements is already a non-namespace type, you'll get an error when you attempt a data operation. Namespace validity is only checked when data access is attempted, not when ns commands are executed.

hs | hyperspace

hyperspaces are namespaces using the last namespace level to provide data placement.

Why do we need hyperspace and namespace?

Consider the namespace:

  • ns [tld] [domain] [access time]

You would end up with inefficient storage and performance because all your data will be placed at one location under the single namespace [tld].

hyperspace fixes this common inefficiency by allowing data placement based on the last (and hopefully most unique/specific) namespace element.

Consider the hyperspace:

  • hs [tld] [domain] [access time]

Now your data will be placed across nodes/domains/threads by [access time] instead of by [tld], which should provide better performance since you will have more distributed [access time] values than [tld] values.

cs | customspace

If namespace places data based on the first name entry and hyperspace places based on the last entry, what if we want to store data based on a different namespace position?

customspace allows you to pick any namespace entry as a placement key.

customspace is the general form of namespace selection.

You can implement namespace as customspace 0 to co-locate based on the first name provided.

You can implement hyperspace as customspace -1 to co-locate on the last name provided.

These pairs show equivalency of customspace:

  • ns [tld] [domain] [access time]
  • cs 0 [tld] [domain] [access time]
  • hs [tld] [domain] [access time]
  • cs 2 [tld] [domain] [access time]
  • hs [tld] [domain] [access time]
  • cs -1 [tld] [domain] [access time]
  • ns [tld] [domain] [access time]
  • cs -3 [tld] [domain] [access time]
  • hs [tld] [domain] [access time]
  • cs 9999 [tld] [domain] [access time]
  • ns [tld] [domain] [access time]
  • cs -100 [tld] [domain] [access time]

If you request values too big or too small, your customspace offset will be clamped to the start or end namespace depending on which direction you overran.

nsreset

Since Carrier DB namespaces can be security boundaries, a client can only "escape" from a namespace using nsreset, but nsreset will not escape above the boundary set by nslock.

You can run either:

  • nsreset
    • reset your client to the top (or locked depth) namespace
  • nsreset [ns1] [sub-ns2] ...
    • reset your client to a different namespace starting at the top (or locked) level

For locking:

  • nslock
    • lock client to the current namespace so nreset will never go higher.
  • nslock [ns|cs|hs] [ns1] [sub-ns2] ...
    • enter a namespace and lock to the full depth for all client operations.
hsreset

Same as nsreset except you are reset into a hyperspace.

csreset

Same as nsreset except you are reset into a customspace.

Takes argument of position before new reset:

  • csreset 0 ...
    • equivalent to nsreset
  • csreset -1 ...
    • equivalent to hsreset
whereami

namespace selection persists after you enter a namespace for the duration of your client connection.

Unless you are on an enableadmin network, you can't escape to a higher level namespace (though you can always create deeper level namespace maps).

If you need a reminder which namespace you're in, run whereami to get your current namespace depth along with the index of which namespace entry is being used for co-locating your data across nodes/domains/threads.

keys

Returns a list of all key names under your current namespace.

If run at the top level, this returns all keys on your server.

Equivalent to being outside of a namespace and running hkeys [namespace].

Order of result is not specified.

vals

Returns a list of all values under your current namespace.

If run at the top level, this returns all values on your server.

NOTE: vals returns all nested values for all types, including lists of any size and further nested namespaces of any depth and all their keys and values.

Equivalent to being outside of a namespace and running hvals [namespace].

Order of result is not specified. May not match keys result order.

getall | keysandvals

Returns alternating key, value list.

See caveats listed under vals so you don't explode your server.

Equivalent to hgetall used from one level outside a namespace.

keysandexpires

Returns alternating key, expires list.

NOTE: keysandexpires iterates all child namespaces and returns their expirations too.

Also available: keysandexpires nice to return readable date time strings.

keysandexpires nice currently has second resolution, but resolution may be increased in future releases.

keysandtypes

Returns alternating key, type list.

NOTE: keysandtypes iterates all child namespaces and returns their types too.

NOTE: Namespaces return nested values indicating they are namespaces.

legacy redis protocol [sets]
command deviations
sadd, srem, sismember, scard, spop [count], srandmember [count], sinter, sinterstore, sunion, sunionstore, sdiff, sdiffstore, smembers

Carrier DB supports all legacy redis set operations with the following changes:

  • smove can accept an unlimited number of elements to move instead of being limited to only one element
  • smembers is just an alias for get since, in Carrier DB, get can act on any type.
  • Carrier DB sorts membership results prior to client reply for more consistent data management.
ssubset

ssubset allows you to check if the first set is a subset of the second set directly in the server.

Returns 1 if the first key is a subset of the second key, otherwise 0.

sequal

sequal allows you to compare an unlimited number of sets for equality directly in the server.

Server-side compares are much more efficient than client-side checks when dealing with large sets.

Returns 1 if all sets have the same elements, otherwise 0.

Testing for equality is logically equivalent to getting true results for both ssubset A B and ssubset B A.

sequal is more efficient than double ssubset since sequal doesn't need to read all sets twice, plus sequal can compare an unbounded number of sets with just one command.

qsadd, qsrem, qsismember, qscard, qspop [count], qsrandmember [count], qsinter, qsinterstore, qsunion, qsunionstore, qsdiff, qsdiffstore, qsmembers, qssubset, qsequal

qsets provide the same interface as regular sets, but their in-server representation is optimized for repeated elements across many sets.

Elements inside all qsets are only stored once on the server, then references back to their values are used in each qset, potentially saving you an extreme amount of memory if you have many sets with medium to large duplicated elements.

qsets also provide very efficient set intersection calculations since elements can be compared by reference instead of iterating values for each intersection.

mnemonic: quantized sets

qnsadd, qnsrem, qnsismember, qnscard, qnspop [count], qnsrandmember [count], qnsinter, qnsinterstore, qnsunion, qnsunionstore, qnsdiff, qnsdiffstore, qnsmembers, qnssubset, qnsequal

qnsets provide the same interface as regular sets, but their in-server representation is optimized for storing integer-only values.

Elements inside all qnsets are stored as compressed integers on the server side. You can currently store numbers in range [-9671406556917033397649407, 19342813113834066795298815] fun fact: that's the range of 84-bit signed and unsigned integers. We may extend qnsets to support a 128-bit signed and unsigned range in a future release.

qnsets also provide very efficient set intersection calculations since the server is operating only on binary integer values directly.

mnemonic: quantized numeric sets

legacy redis protocol [string]
command deviations
cas

Carrier DB supports a compare-and-set cas command under legacy redis protocol even though redis has refused to add a compare-and-set operation for ten years.

Usage:

  • cas set compare [key] [new value] [old value]

Example of successful cas operation:

  • set hello world
  • cas set compare hello everybody world → return value is 1 on success
  • Now key hello has value everybody because world matched the current value of hello.
  • If you run cas set compare hello everybody world again, the command will return 0 and the value of key hello will not change because your compare value does not match the current value.
set

accepts legacy redis syntax options of:

  • set key value [EX seconds|PX milliseconds] [NX|XX]

Carrier DB does not implement the obsolete setex or psetex commands because they are just aliases for set key value EX sec and set key value PX ms.

Carrier DB does implement setnx because its return value is inconsistently different than set key value NX.

Additionally we match memcached commands by having:

  • add key value aliased to set key value NX
    • insert if key does not exist
  • replace key value aliased to set key value XX
    • insert if key does exist
get

Carrier DB converts all types to return values when using get (instead of only accepting strings).

get some-list returns the entire list content.

get some-hll returns the count of the HLL.

get some-map returns the key-value pairs of the namespace.

mset, msetnx

Works as expected, even across cluster nodes.

If failures encountered, send semi-reproducible test cases to concerns@carrierdb.cloud.

append, prepend

Carrier DB supports the prepend operation from memcached as well. legacy redis does not provide the operation.

strlen | countbytes, countutf8codepoints

strlen is also aliased to the more descriptive countbytes.

Carrier DB also provides a unique countutf8codepoints command to count variable length UTF8 code points.

A single printed UTF8 character can use anywhere from one to eight code points, and each code points is between one and four bytes long, so one UTF8 code point may not be a full printed character (due to combiners, zero width spaces, and zero width joiners (which is how emojis get "customized")).

del | delete remove key or namespace requested. namespace deletes recursively delete their entire contents.
incrby, decrby, incrbyfloat

Carrier DB does not implement the obsolete incr or decr commands.

Carrier DB has an intelligent type-based input system for number parsing, so incrbyfloat is just an alias for incrby. Both incrby and decrby will properly increment and convert values as large as 64-bit signed integers, 64-bit unsigned integers, and 64-bit doubles.

decrbyfloor decrbyfloor matches memcached's decr command. memcached's increment and decrement operations are always on unsigned 64-bit integers, so attempting to decrement below zero stores 0 as your result (hence, the floor of the decrement is zero).
legacy redis protocol [list]
command deviations
rpush, lpush, rpushx, lpushx, llen, rpop, lpop, lindex, ltrim, rpoplpush

Works as expected.

If failures encountered, send semi-reproducible test cases to concerns@carrierdb.cloud.

lrem, linsert, sort

Not implemented. These commands either don't seem useful or are needlessly complex.

If you think these list primitives should be included, let us know.

lrange

In Carrier DB, lrange is a non-blocking command.

If you have a list with 3 million elements and you want to use lrange to return the middle million elements, Carrier DB remains responsive to all other clients during your retrieval.

legacy redis protocol [hyperloglog]
command deviations
pfadd | hlladd

pfadd (also aliased to hlladd) is a non-blocking command. If a client adds a million elements using hlladd, Carrier DB will remain responsive without blocking other clients.

pfcount | hllcount

returns a unified count after temporarily merging all hll keys provided

pfmerge | hllmerge

pfmerge (also aliased to hllmerge) returns the count of the merged HLL target. This deviates from the legacy redis return value where merging HLLs just returns a useless OK status.

legacy redis protocol [hash, dict, map]

Note: redis hash commands are just protocol adapters. All maps in Carrier DB are fully formed namespaces supporting nested types.

command deviations
hset, hsetnx, hmset, hget, hmget, hdel, hkeys, hvals, hgetall

Each legacy hash command is mapped to Carrier DB namespace operations. Legacy hash commands just read values inside a namespace, but without a client needing to enter a namespace.

These two command sets are equivalent:

  • old hash version
    • hset a-map somekey somevalue
    • hget a-map somekey
  • namespace version
    • ns a-map
    • set somekey somevalue
    • get somekey
    • nsreset
    • hget a-map somekey

legacy redis protocol [server / admin]

command deviations
stats

Tired of legacy redis janky busted hand-written monitoring output where each line needs an independent parser because formatting is completely arbitrary? Have we got the solution for you: JSON statistics!

For details of Carrier Stats format, see Stats For The Modern Age.

If you are testing stats with redis-cli, the output of stats will be ugly because wobbly legacy redis-cli doesn't support printing newlines in results. You'll want to view pretty formatted output with this shell command:

  • printf "$(redis-cli -p PORT stats visual)"

For retrieval efficiency, individual subsections can be selected per stats request to avoid generating sections you don't need.

For example, you probably don't need to generate the license and startup sections for real time monitoring requests. You can specify zero or more of the following sections after the stats command:

  • license
  • process
  • cpu
  • memory
  • startup
  • os
  • log
  • keyspace
  • network
self stats

json output

returns statistics for your client connection.

also accepts the form self stats visual

admin tls reload

request a reload of all TLS certificates and keys currently configured.

Carrier DB will schedule a near-immediate reload of all certificates if they have changed. Unchanged certificates and keys are not reloaded.

admin loglevel set [level]

change logging verbosity live.

Levels are the standard range of (from least to most logging): EMERGENCY, ALERT, CRITICAL, ERROR, WARNING, NOTICE, INFO, DEBUG

admin shutdown [fast]

Schedule server shutdown immediately.

Server waits for all current client requests to complete before shutting down unless fast is provided.

You Have Reached The End Of The Page

Here at End Of The Page, we want to thank you for reading our page.

If you have any questions about Carrier DB, let us know and we can make the world a little bit better one program at a time.

For updates about new Carrier DB features and releases, email us or something?

This is the official site for all Carrier DB updates, releases, and feature notes. Check back often.


YOU CONSENT TO THE PLACEMENT OF COOKIES ON ANY SURFACE NEAR YOU FOR THE PURPOSE OF EASILY CONDUCTING CONSUMPTION OF AFOREMENTIONED COOKIES

NO WARRANTY WHATSOEVER, EXPRESS, STATUTORY, OR IMPLIED EXCEPT FOR AN EVERLASTING GUARANTEE OF BIG DATABASE ENERGY