A small compatibility layer for many KV stores like localStorage, Redis, FileSystem, etc:
import kv from "polystore";
const store = kv(new Map()); // in-memory
const store1 = kv(localStorage); // Persist in the browser
const store2 = kv(redisClient); // Use a Redis client for backend persistence
// etc.
This is the API with all of the methods (they are all async
):
-
.get(key): any
: retrieve a single value, ornull
if it doesn't exist or is expired. -
.set(key, value, options?)
: save a single value, which can be anything that is serializable. -
.add(value, options?)
: same as with.set()
, but auto-generate the key. -
.has(key): boolean
: check whether the key is in the store or not. -
.del(key)
: delete a single value from the store. -
.keys(prefix?): string[]
: get a list of all the available strings in the store. -
.values(prefix?): any[]
: get a list of all the values in the store. -
.entries(prefix?): [string, any][]
: get a list of all the key-value pairs in the store. -
.clear()
: delete ALL of the data in the store, effectively resetting it. -
.close()
: (only some stores) ends the connection to the store. -
.prefix(prefix): store
: create a new sub-instance of the store where all the keys have this prefix.
Available stores:
-
Memory
new Map()
(fe+be): an in-memory API to keep your KV store -
Local Storage
localStorage
(fe): persist the data in the browser's localStorage -
Session Storage
sessionStorage
(fe): persist the data in the browser's sessionStorage -
Cookies
"cookie"
(fe): persist the data using cookies -
LocalForage
localForage
(fe): persist the data on IndexedDB -
FS File
new URL('file:///...')
(be): store the data in a single JSON file -
Redis Client
redisClient
(be): use the Redis instance that you connect to -
Cloudflare KV
env.KV_NAMESPACE
(be): use Cloudflare's KV store - (WIP) Consul KV
new Consul()
(fe+be): use Hashicorp's Consul KV store (https://www.npmjs.com/package/consul#kv)
I made this library to be used as a "building block" of other libraries, so that your library can accept many cache stores effortlessly! It's isomorphic (Node.js and the Browser) and tiny (~2KB). For example, let's say you create an API library, then you can accept the stores from your client:
import MyApi from "my-api";
MyApi({ cache: new Map() }); // OR
MyApi({ cache: localStorage }); // OR
MyApi({ cache: redisClient }); // OR
MyApi({ cache: env.KV_NAMESPACE }); // OR
// ...
See how to initialize each store in the Stores list documentation. But basically for every store, it's like this:
import kv from "polystore";
// Initialize it; NO "new"; NO "await", just a plain function wrap:
const store = kv(MyClientOrStoreInstance);
// use the store
While you can keep a reference to the store and access it directly, we strongly recommend if you are going to use a store, to only access it through polystore
, since we do add custom serialization and extra properties for e.g. expiration time:
const map = new Map();
const store = kv(map);
// Works as expected
await store.set("a", "b");
console.log(await store.get("a"));
// DON'T DO THIS; this will break the app since we apply more
// advanced serialization to the values stored in memory
map.set("a", "b");
console.log(await store.get("a")); // THROWS ERROR
Retrieve a single value from the store. Will return null
if the value is not set in the store, or if it was set but has already expired:
const value = await store.get(key: string);
console.log(await store.get("key1")); // "Hello World"
console.log(await store.get("key2")); // ["my", "grocery", "list"]
console.log(await store.get("key3")); // { name: "Francisco" }
If the value is returned, it can be a simple type like boolean
, string
or number
, or it can be a plain Object or Array, or a combination of those.
The value cannot be more complex or non-serializable values like a
Date()
,Infinity
,undefined
(casted tonull
), a Symbol, etc.
Create or update a value in the store. Will return a promise that resolves with the key when the value has been saved. The value needs to be serializable:
await store.set(key: string, value: any, options?: { expires: number|string });
await store.set("key1", "Hello World");
await store.set("key2", ["my", "grocery", "list"], { expires: "1h" });
await store.set("key3", { name: "Francisco" }, { expires: 60 * 60 * 1000 });
The value can be a simple type like boolean
, string
or number
, or it can be a plain Object or Array, or a combination of those. It cannot be a more complex or non-serializable values like a Date()
, Infinity
, undefined
(casted to null
), a Symbol
, etc.
- By default the keys don't expire.
- Setting the
value
tonull
, or theexpires
to0
is the equivalent of deleting the key+value. - Conversely, setting
expires
tonull
orundefined
will make the value never to expire.
When the expires
option is set, it can be a number (seconds) or a string representing some time:
// Valid "expire" values:
0 - expire immediately (AKA delete it)
0.1 - expire after 100ms*
60 * 60 - expire after 1h
3_600 - expire after 1h
"10s" - expire after 10 seconds
"2minutes" - expire after 2 minutes
"5d" - expire after 5 days
* not all stores support sub-second expirations, notably Redis and Cookies don't, so it's safer to always use an integer or an amount larger than 1s
These are all the units available:
"ms", "millisecond", "s", "sec", "second", "m", "min", "minute", "h", "hr", "hour", "d", "day", "w", "wk", "week", "b" (month), "month", "y", "yr", "year"
Create a value in the store with a random key string. Will return a promise that resolves with the key when the value has been saved. The value needs to be serializable:
const key:string = await store.add(value: any, options?: { expires: number|string });
const key1 = await store.add("Hello World");
const key2 = await store.add(["my", "grocery", "list"], { expires: "1h" });
const key3 = await store.add({ name: "Francisco" }, { expires: 60 * 60 * 1000 });
The generated key is 24 AlphaNumeric characters (including upper and lower case) generated with random cryptography to make sure it's unguessable, high entropy and safe to use in most contexts like URLs, queries, etc. We use nanoid
with a custom dictionary, so you can check the entropy in this dictionary by removing the "_" and "-", and setting it to 24 characters.
Here is the safety: "If you generate 1 million keys/second, it will take ~14 million years in order to have a 1% probability of at least one collision."
Note: please make sure to read the
.set()
section for all the details, since.set()
and.add()
behave the same way except for the first argument.
Check whether the key is available in the store and not expired:
await store.has(key: string);
if (await store.has('cookie-consent')) {
loadCookies();
}
Remove a single key from the store:
await store.del(key: string);
Get all of the keys in the store, optionally filtered by a prefix:
await store.keys(filter?: string);
We ensure that all of the keys returned by this method are not expired, while discarding any potentially expired key. See expiration explained for more details.
Get all of the values in the store, optionally filtered by a key prefix:
await store.values(filter?: string);
This is useful specially when you already have the id/key within the value as an object, then you can just get a list of all of them:
const sessions = await store.values("session:");
// A list of all the sessions
const companies = await store.values("company:");
// A list of all the companies
We ensure that all of the values returned by this method are not expired, while discarding any potentially expired key. See expiration explained for more details.
Get all of the entries (key:value tuples) in the store, optionally filtered by a key prefix:
await store.entries(filter?: string);
It is in a format that you can easily build an object out of it:
const sessionEntries = await store.entries("session:");
const sessions = Object.fromEntries(sessionEntries);
We ensure that all of the entries returned by this method are not expired, while discarding any potentially expired key. See expiration explained for more details.
Remove all of the data from the store:
await store.clear();
Create a sub-store where all the operations use the given prefix. This is the only method of the store that is sync and you don't need to await:
const store = kv(new Map());
const session = store.prefix("session:");
Then all of the operations will be converted internally to add the prefix when reading, writing, etc:
const val = await session.get("key1"); // .get('session:key1');
await session.set("key2", "some data"); // .set('session:key2', ...);
const val = await session.has("key3"); // .has('session:key3');
await session.del("key4"); // .del('session:key4');
await session.keys(); // .keys('session:');
// ['key1', 'key2', ...] Note no prefix here
await session.clear(); // delete only keys with the prefix
This will probably never be stable given the nature of some engines, so as an alternative please consider using two stores instead of prefixes:
// Two in-memory stores
const store = kv(new Map());
const session = kv(new Map());
// Two file-stores
const users = kv(new URL(`file://${import.meta.dirname}/users.json`));
const books = kv(new URL(`file://${import.meta.dirname}/books.json`));
The main reason this is not stable is because some store engines don't allow for atomic deletion of keys given a prefix. While we do still clear them internally in those cases, that is a non-atomic operation and it could have some trouble if some other thread is reading/writing the data at the same time.
Accepts directly the store, or a promise that resolves into a store. All of the stores, including those that natively don't support it, are enhanced with Promises
and expires
times, so they all work the same way.
An in-memory KV store, with promises and expiration time:
import kv from "polystore";
// This already works, by default if there's nothing it'll use
// a new Map()
const store = kv();
await store.set("key1", "Hello world");
console.log(await store.get("key1"));
// Or you can be explicit:
const store = kv(new Map());
await store.set("key1", "Hello world");
console.log(await store.get("key1"));
The traditional localStorage that we all know and love, this time with a unified API, and promises:
import kv from "polystore";
const store = kv(localStorage);
await store.set("key1", "Hello world");
console.log(await store.get("key1"));
Same limitations as always apply to localStorage, if you think you are going to use too much storage try instead our integration with Local Forage!
Same as localStorage, but now for the session only:
import kv from "polystore";
const store = kv(sessionStorage);
await store.set("key1", "Hello world");
console.log(await store.get("key1"));
Supports native browser cookies, including setting the expire time:
import kv from "polystore";
const store = kv("cookie"); // yes, just a plain string
await store.set("key1", "Hello world");
console.log(await store.get("key1"));
It is fairly limited for how powerful cookies are, but in exchange it has the same API as any other method or KV store. It works with browser-side Cookies (no http-only).
Note: the cookie expire resolution is in the seconds, so times shorter than 1 second like
expires: 0.02
(20 ms) don't make sense for this storage method and won't properly save them.
Supports localForage (with any driver it uses) so that you have a unified API. It also adds the expires
option to the setters!
import kv from "polystore";
import localForage from "localforage";
const store = kv(localForage);
await store.set("key1", "Hello world", { expires: "1h" });
console.log(await store.get("key1"));
Supports the official Node Redis Client. You can pass either the client or the promise:
import kv from "polystore";
import { createClient } from "redis";
// Note: no need for await or similar
const store = kv(createClient().connect());
await store.set("key1", "Hello world");
console.log(await store.get("key1"));
Note: the Redis client expire resolution is in the seconds, so times shorter than 1 second like
expires: 0.02
(20 ms) don't make sense for this storage method and won't properly save them.
import kv from "polystore";
// Create a url with the file protocol:
const store = kv(new URL("file:///Users/me/project/cache.json"));
// Paths need to be absolute, but you can use process.cwd() to make
// it relative to the current process:
const store = kv(new URL(`file://${process.cwd()}/cache.json`));
const store2 = kv(new URL(`file://${import.meta.dirname}/data.json`));
Supports the official Cloudflare's KV stores. Follow the official guide, then load it like this:
import kv from "polystore";
export default {
async fetch(request, env, ctx) {
const store = kv(env.YOUR_KV_NAMESPACE);
await store.set("KEY", "VALUE");
const value = await store.get("KEY");
if (!value) {
return new Response("Value not found", { status: 404 });
}
return new Response(value);
},
};
Why? The Cloudflare native KV store only accepts strings and has you manually calculating timeouts, but as usual with polystore
you can set/get any serializable value and set the timeout in a familiar format:
// GOOD - with polystore
await store.set("user", { name: "Francisco" }, { expires: "2days" });
// COMPLEX - With native Cloudflare KV
const serialValue = JSON.stringify({ name: "Francisco" });
const twoDaysInSeconds = 2 * 24 * 3600;
await env.YOUR_KV_NAMESPACE.put("user", serialValue, {
expirationTtl: twoDaysInSeconds,
});
While different engines do expiration slightly differently internally, in creating polystore we want to ensure certain constrains, which can affect performance. For example, if you do this operation:
// in-memory store
const store = polystore(new Map());
await store.set("a", "b", { expires: "1s" });
// These checks of course work:
console.log(await store.keys()); // ['a']
console.log(await store.has("a")); // true
console.log(await store.get("a")); // 'b'
// Make sure the key is expired
await delay(2000); // 2s
// Not only the .get() is null, but `.has()` returns false, and .keys() ignores it
console.log(await store.keys()); // []
console.log(await store.has("a")); // false
console.log(await store.get("a")); // null
This is great because with polystore we do ensure that if a key has expired, it doesn't show up in .keys()
, .entries()
, .values()
, .has()
or .get()
.
However, in some stores this does come with some potential performance disadvantages. For example, both the in-memory example above and localStorage don't have a native expiration/eviction process, so we have to store that information as metadata, meaning that even to check if a key exists we need to read and decode its value. For one or few keys it's not a problem, but for large sets this can become an issue.
For other stores like Redis this is not a problem, because the low-level operations already do them natively, so we don't need to worry about this for performance at the user-level. Instead, Redis and cookies have the problem that they only have expiration resolution at the second level. Meaning that 800ms is not a valid Redis expiration time, it has to be 1s, 2s, etc.
A store needs at least 4 methods with these signatures:
const store = {};
store.get = (key: string) => Promise<any>;
store.set = (key: string, value: any, { expires: number }) => Promise<string>;
store.entries = (prefix: string = "") => Promise<[key:string, value:any][]>;
store.clear = () => Promise<null>;
All of the other methods will be implemented on top of these if not available, but you can provide those as well for optimizations, incompatible APIs, etc. For example, .set('a', null)
should delete the key a
, and for this you may provide a native implementation:
const native = myNativeStore();
const store = {};
store.get = (key) => native.getItem(key);
// ...
store.del = (key) => native.deleteItem(key);