Distributed Key/Value Store Abstraction Library
# Valkeyrie [](https://pkg.go.dev/github.com/kvtools/valkeyrie) [](https://github.com/kvtools/valkeyrie/actions/workflows/build.yml) [](https://goreportcard.com/report/github.com/kvtools/valkeyrie) `valkeyrie` provides a Go native library to store metadata using Distributed Key/Value stores (or common databases). Its goal is to abstract common store operations (`Get`, `Put`, `List`, etc.) for multiple Key/Value store backends. For example, you can easily implement a generic *Leader Election* algorithm on top of it (see the [docker/leadership](https://github.com/docker/leadership) repository). The benefit of `valkeyrie` is not to duplicate the code for programs that should support multiple distributed Key/Value stores such as `Consul`/`etcd`/`zookeeper`, etc. ## Examples of Usage You can refer to [Examples](https://github.com/kvtools/valkeyrie/blob/master/docs/examples.md) for a basic overview of the library. ```go package main import ( "context" "log" "time" "github.com/kvtools/consul" "github.com/kvtools/valkeyrie" ) func main() { ctx := context.Background() config := &consul.Config{ ConnectionTimeout: 10 * time.Second, } kv, err := valkeyrie.NewStore(ctx, consul.StoreName, []string{"localhost:8500"}, config) if err != nil { log.Fatal("Cannot create store consul") } key := "foo" err = kv.Put(ctx, key, []byte("bar"), nil) if err != nil { log.Fatalf("Error trying to put value at key: %v", key) } pair, err := kv.Get(ctx, key, nil) if err != nil { log.Fatalf("Error trying accessing value at key: %v", key) } log.Printf("value: %s", string(pair.Value)) err = kv.Delete(ctx, key) if err != nil { log.Fatalf("Error trying to delete key %v", key) } } ``` ## Compatibility A **storage backend** in `valkeyrie` implements (fully or partially) the [Store](https://github.com/kvtools/valkeyrie/blob/master/store/store.go#L69) interface. | Calls | Consul | Etcd | Zookeeper | Redis | BoltDB | DynamoDB | |-----------------------|:------:|:----:|:---------:|:-----:|:------:|:--------:| | Put | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | | Get | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | | Delete | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | | Exists | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | | Watch | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🔴 | 🔴 | | WatchTree | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🔴 | 🔴 | | NewLock (Lock/Unlock) | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🔴 | 🟢️ | | List | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | | DeleteTree | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | | AtomicPut | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | | AtomicDelete | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | 🟢️ | The store implementations: - [boltdb](https://github.com/kvtools/boltdb) - [consul](https://github.com/kvtools/consul) - [dynamodb](https://github.com/kvtools/dynamodb) - [etcdv2](https://github.com/kvtools/etcdv2) - [etcdv3](https://github.com/kvtools/etcdv3) - [redis](https://github.com/kvtools/redis) - [zookeeper](https://github.com/kvtools/zookeeper) The store template: - [template](https://github.com/kvtools/template) ## Limitations Distributed Key/Value stores often have different concepts for managing and formatting keys and their associated values. Even though `valkeyrie` tries to abstract those stores aiming for some consistency, in some cases it can't be applied easily. Calls like `WatchTree` may return different events (or number of events) depending on the backend (for now, `Etcd` and `Consul` will likely return more events than `Zookeeper` that you should triage properly). ## Contributing Want to contribute to `valkeyrie`? Take a look at the [Contribution Guidelines](https://github.com/kvtools/valkeyrie/blob/master/CONTRIBUTING.md). The [Maintainers](https://github.com/kvtools/valkeyrie/blob/master/maintainers.md). ## Copyright and License Apache License Version 2.0 Valkeyrie started as a hard fork of the unmaintained [libkv](https://github.com/docker/libkv). ================================================ FILE: store/errors.go ================================================ package store import ( "errors" "fmt" ) var ( // ErrCallNotSupported is thrown when a method is not implemented/supported by the current backend. ErrCallNotSupported = errors.New("the current call is not supported with this backend") // ErrNotReachable is thrown when the API cannot be reached for issuing common store operations. ErrNotReachable = errors.New("api not reachable") // ErrCannotLock is thrown when there is an error acquiring a lock on a key. ErrCannotLock = errors.New("error acquiring the lock") // ErrKeyModified is thrown during an atomic operation if the index does not match the one in the store. ErrKeyModified = errors.New("unable to complete atomic operation, key modified") // ErrKeyNotFound is thrown when the key is not found in the store during a Get operation. ErrKeyNotFound = errors.New("key not found in store") // ErrPreviousNotSpecified is thrown when the previous value is not specified for an atomic operation. ErrPreviousNotSpecified = errors.New("previous K/V pair should be provided for the Atomic operation") // ErrKeyExists is thrown when the previous value exists in the case of an AtomicPut. ErrKeyExists = errors.New("previous K/V pair exists, cannot complete Atomic operation") ) // InvalidConfigurationError is thrown when the type of the configuration is not supported by a store. type InvalidConfigurationError struct { Store string Config any } func (e *InvalidConfigurationError) Error() string { return fmt.Sprintf("%s: invalid configuration type: %T", e.Store, e.Config) } // UnknownConstructorError is thrown when a requested store is not register. type UnknownConstructorError struct { Store string } func (e UnknownConstructorError) Error() string { return fmt.Sprintf("unknown constructor %q (forgotten import?)", e.Store) } ================================================ FILE: store/helpers.go ================================================ package store import ( "strings" ) // CreateEndpoints creates a list of endpoints given the right scheme. func CreateEndpoints(addrs []string, scheme string) (entries []string) { for _, addr := range addrs { entries = append(entries, scheme+"://"+addr) } return entries } // SplitKey splits the key to extract path information. func SplitKey(key string) (path []string) { if strings.Contains(key, "/") { return strings.Split(key, "/") } return []string{key} } ================================================ FILE: store/store.go ================================================ // Package store contains KV store backends. package store import ( "context" "time" ) // Store represents the backend K/V storage. // Each store should support every call listed here. // Or it couldn't be implemented as a K/V backend for valkeyrie. type Store interface { // Put a value at the specified key. Put(ctx context.Context, key string, value []byte, opts *WriteOptions) error // Get a value given its key. Get(ctx context.Context, key string, opts *ReadOptions) (*KVPair, error) // Delete the value at the specified key. Delete(ctx context.Context, key string) error // Exists Verify if a Key exists in the store. Exists(ctx context.Context, key string, opts *ReadOptions) (bool, error) // Watch for changes on a key. Watch(ctx context.Context, key string, opts *ReadOptions) (<-chan *KVPair, error) // WatchTree watches for changes on child nodes under a given directory. WatchTree(ctx context.Context, directory string, opts *ReadOptions) (<-chan []*KVPair, error) // NewLock creates a lock for a given key. // The returned Locker is not held and must be acquired with `.Lock`. // The Value is optional. NewLock(ctx context.Context, key string, opts *LockOptions) (Locker, error) // List the content of a given prefix. List(ctx context.Context, directory string, opts *ReadOptions) ([]*KVPair, error) // DeleteTree deletes a range of keys under a given directory. DeleteTree(ctx context.Context, directory string) error // AtomicPut Atomic CAS operation on a single value. // Pass previous = nil to create a new key. AtomicPut(ctx context.Context, key string, value []byte, previous *KVPair, opts *WriteOptions) (bool, *KVPair, error) // AtomicDelete Atomic delete of a single value. AtomicDelete(ctx context.Context, key string, previous *KVPair) (bool, error) // Close the store connection. Close() error } // KVPair represents {Key, Value, LastIndex} tuple. type KVPair struct { Key string Value []byte LastIndex uint64 } // WriteOptions contains optional request parameters. type WriteOptions struct { IsDir bool TTL time.Duration // If true, the client will keep the lease alive in the background // for stores that are allowing it. KeepAlive bool } // ReadOptions contains optional request parameters. type ReadOptions struct { // Consistent defines if the behavior of a Get operation is linearizable or not. // Linearizability allows us to 'see' objects based on a real-time total order // as opposed to an arbitrary order or with stale values ('inconsistent' scenario). Consistent bool } // LockOptions contains optional request parameters. type LockOptions struct { Value []byte // Optional, value to associate with the lock. TTL time.Duration // Optional, expiration ttl associated with the lock. RenewLock chan struct{} // Optional, chan used to control and stop the session ttl renewal for the lock. DeleteOnUnlock bool // If true, the value will be deleted when the lock is unlocked or expires. } // Locker provides locking mechanism on top of the store. // Similar to sync.Locker except it may return errors. type Locker interface { Lock(ctx context.Context) (<-chan struct{}, error) Unlock(ctx context.Context) error } ================================================ FILE: testsuite/suite.go ================================================ // Package testsuite the valkeyrie tests suite. package testsuite import ( "context" "strconv" "strings" "testing" "time" "github.com/kvtools/valkeyrie/store" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" ) const testTimeout = 60 * time.Second // RunTestCommon tests the minimal required APIs which // should be supported by all K/V backends. func RunTestCommon(t *testing.T, kv store.Store) { t.Helper() testPutGetDeleteExists(t, kv) testList(t, kv) testDeleteTree(t, kv) } // RunTestListLock tests the list output for mutexes // and checks that internal side keys are not listed. func RunTestListLock(t *testing.T, kv store.Store) { t.Helper() testListLockKey(t, kv) } // RunTestAtomic tests the Atomic operations by the K/V // backends. func RunTestAtomic(t *testing.T, kv store.Store) { t.Helper() testAtomicPut(t, kv) testAtomicPutCreate(t, kv) testAtomicPutWithSlashSuffixKey(t, kv) testAtomicDelete(t, kv) } // RunTestWatch tests the watch/monitor APIs supported // by the K/V backends. func RunTestWatch(t *testing.T, kv store.Store) { t.Helper() testWatch(t, kv) testWatchTree(t, kv) } // RunTestLock tests the KV pair Lock/Unlock APIs supported // by the K/V backends. func RunTestLock(t *testing.T, kv store.Store) { t.Helper() testLockUnlock(t, kv) } // RunTestLockTTL tests the KV pair Lock with TTL APIs supported // by the K/V backends. func RunTestLockTTL(t *testing.T, kv store.Store, backup store.Store) { t.Helper() testLockTTL(t, kv, backup) } // RunTestTTL tests the TTL functionality of the K/V backend. func RunTestTTL(t *testing.T, kv store.Store, backup store.Store) { t.Helper() testPutTTL(t, kv, backup) } func checkPairNotNil(t *testing.T, pair *store.KVPair) { t.Helper() require.NotNilf(t, pair, "pair is nil") require.NotNilf(t, pair.Value, "value is nil") } func testPutGetDeleteExists(t *testing.T, kv store.Store) { t.Helper() ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() // Get a not exist key should return ErrKeyNotFound. _, err := kv.Get(ctx, "testPutGetDelete_not_exist_key", nil) assert.ErrorIs(t, err, store.ErrKeyNotFound) value := []byte("bar") for _, key := range []string{ "testPutGetDeleteExists", "testPutGetDeleteExists/", "testPutGetDeleteExists/testbar/", "testPutGetDeleteExists/testbar/testfoobar", } { // Put the key. err = kv.Put(ctx, key, value, nil) require.NoError(t, err) // Get should return the value and an incremented index. pair, err := kv.Get(ctx, key, nil) require.NoError(t, err) checkPairNotNil(t, pair) assert.Equal(t, pair.Value, value) assert.NotEqual(t, pair.LastIndex, 0) // Exists should return true. exists, err := kv.Exists(ctx, key, nil) require.NoError(t, err) assert.True(t, exists) // Delete the key. err = kv.Delete(ctx, key) require.NoError(t, err) // Get should fail. pair, err = kv.Get(ctx, key, nil) assert.Error(t, err) assert.Nil(t, pair) // Exists should return false. exists, err = kv.Exists(ctx, key, nil) require.NoError(t, err) assert.False(t, exists) } } func testWatch(t *testing.T, kv store.Store) { t.Helper() key := "testWatch" value := []byte("world") newValue := []byte("world!") ctx, cancel := context.WithTimeout(context.Background(), testTimeout) t.Cleanup(cancel) // Put the key. err := kv.Put(ctx, key, value, nil) require.NoError(t, err) events, err := kv.Watch(ctx, key, nil) require.NoError(t, err) require.NotNil(t, events) // Update loop. go func() { timeout := time.After(1 * time.Second) tick := time.NewTicker(250 * time.Millisecond) defer tick.Stop() for { select { case <-timeout: return case <-tick.C: err := kv.Put(ctx, key, newValue, nil) if assert.NoError(t, err) { continue } return } } }() // Check for updates. eventCount := 1 for { select { case event := <-events: assert.NotNil(t, event) assert.Equal(t, event.Key, key) if eventCount == 1 { assert.Equal(t, event.Value, value) } else { assert.Equal(t, event.Value, newValue) } eventCount++ // We received all the events we wanted to check. if eventCount >= 4 { return } case <-time.After(4 * time.Second): t.Fatal("Timeout reached") return } } } func testWatchTree(t *testing.T, kv store.Store) { t.Helper() dir := "testWatchTree" node1 := "testWatchTree/node1" value1 := []byte("node1") node2 := "testWatchTree/node2" value2 := []byte("node2") node3 := "testWatchTree/node3" value3 := []byte("node3") ctx, cancel := context.WithTimeout(context.Background(), testTimeout) t.Cleanup(cancel) err := kv.Put(ctx, node1, value1, nil) require.NoError(t, err) err = kv.Put(ctx, node2, value2, nil) require.NoError(t, err) err = kv.Put(ctx, node3, value3, nil) require.NoError(t, err) events, err := kv.WatchTree(ctx, dir, nil) require.NoError(t, err) require.NotNil(t, events) // Update loop. go func() { time.Sleep(500 * time.Millisecond) err := kv.Delete(ctx, node3) require.NoError(t, err) }() // Check for updates. eventCount := 1 for { select { case event := <-events: assert.NotNil(t, event) // We received the Delete event on a child node // Exit test successfully. if eventCount == 2 { return } eventCount++ case <-time.After(4 * time.Second): t.Fatal("Timeout reached") return } } } func testAtomicPut(t *testing.T, kv store.Store) { t.Helper() key := "testAtomicPut" value := []byte("world") ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() // Put the key. err := kv.Put(ctx, key, value, nil) require.NoError(t, err) // Get should return the value and an incremented index. pair, err := kv.Get(ctx, key, nil) require.NoError(t, err) checkPairNotNil(t, pair) assert.Equal(t, pair.Value, value) assert.NotEqual(t, pair.LastIndex, 0) // This CAS should fail: previous exists. success, _, err := kv.AtomicPut(ctx, key, []byte("WORLD"), nil, nil) assert.Error(t, err) assert.False(t, success) // This CAS should succeed. success, _, err = kv.AtomicPut(ctx, key, []byte("WORLD"), pair, nil) require.NoError(t, err) assert.True(t, success) // This CAS should fail, key has wrong index. pair.LastIndex = 6744 success, _, err = kv.AtomicPut(ctx, key, []byte("WORLDWORLD"), pair, nil) assert.Equal(t, err, store.ErrKeyModified) assert.False(t, success) } func testAtomicPutCreate(t *testing.T, kv store.Store) { t.Helper() ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() // Use a key in a new directory to ensure Stores will create directories // that don't yet exist. key := "testAtomicPutCreate/create" value := []byte("putcreate") // AtomicPut the key, previous = nil indicates create. success, _, err := kv.AtomicPut(ctx, key, value, nil, nil) require.NoError(t, err) assert.True(t, success) // Get should return the value and an incremented index. pair, err := kv.Get(ctx, key, nil) require.NoError(t, err) checkPairNotNil(t, pair) assert.Equal(t, pair.Value, value) // Attempting to create again should fail. success, _, err = kv.AtomicPut(ctx, key, value, nil, nil) assert.ErrorIs(t, err, store.ErrKeyExists) assert.False(t, success) // This CAS should succeed, since it has the value from Get(). success, _, err = kv.AtomicPut(ctx, key, []byte("PUTCREATE"), pair, nil) require.NoError(t, err) assert.True(t, success) } func testAtomicPutWithSlashSuffixKey(t *testing.T, kv store.Store) { t.Helper() ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() k1 := "testAtomicPutWithSlashSuffixKey/key/" success, _, err := kv.AtomicPut(ctx, k1, []byte{}, nil, nil) require.NoError(t, err) assert.True(t, success) } func testAtomicDelete(t *testing.T, kv store.Store) { t.Helper() key := "testAtomicDelete" value := []byte("world") ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() // Put the key. err := kv.Put(ctx, key, value, nil) require.NoError(t, err) // Get should return the value and an incremented index. pair, err := kv.Get(ctx, key, nil) require.NoError(t, err) checkPairNotNil(t, pair) assert.Equal(t, pair.Value, value) assert.NotEqual(t, pair.LastIndex, 0) tempIndex := pair.LastIndex // AtomicDelete should fail. pair.LastIndex = 6744 success, err := kv.AtomicDelete(ctx, key, pair) assert.Error(t, err) assert.False(t, success) // AtomicDelete should succeed. pair.LastIndex = tempIndex success, err = kv.AtomicDelete(ctx, key, pair) require.NoError(t, err) assert.True(t, success) // Delete a non-existent key; should fail. success, err = kv.AtomicDelete(ctx, key, pair) assert.ErrorIs(t, err, store.ErrKeyNotFound) assert.False(t, success) } func testLockUnlock(t *testing.T, kv store.Store) { t.Helper() ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() key := "testLockUnlock" value := []byte("bar") // We should be able to create a new lock on key. lock, err := kv.NewLock(ctx, key, &store.LockOptions{ Value: value, TTL: 2 * time.Second, DeleteOnUnlock: true, }) require.NoError(t, err) require.NotNil(t, lock) // Lock should successfully succeed or block. lockChan, err := lock.Lock(ctx) require.NoError(t, err) assert.NotNil(t, lockChan) // Get should work. pair, err := kv.Get(ctx, key, nil) require.NoError(t, err) checkPairNotNil(t, pair) assert.Equal(t, pair.Value, value) assert.NotEqual(t, pair.LastIndex, 0) // Unlock should succeed. err = lock.Unlock(ctx) require.NoError(t, err) // Lock should succeed again. lockChan, err = lock.Lock(ctx) require.NoError(t, err) assert.NotNil(t, lockChan) // Get should work. pair, err = kv.Get(ctx, key, nil) require.NoError(t, err) checkPairNotNil(t, pair) assert.Equal(t, pair.Value, value) assert.NotEqual(t, pair.LastIndex, 0) err = lock.Unlock(ctx) require.NoError(t, err) } func testLockTTL(t *testing.T, kv store.Store, otherConn store.Store) { t.Helper() ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() key := "testLockTTL" value := []byte("bar") renewCh := make(chan struct{}) // We should be able to create a new lock on key. lockOC, err := otherConn.NewLock(ctx, key, &store.LockOptions{ Value: value, TTL: 2 * time.Second, RenewLock: renewCh, }) require.NoError(t, err) require.NotNil(t, lockOC) // Lock should successfully succeed. lockChan, err := lockOC.Lock(ctx) require.NoError(t, err) assert.NotNil(t, lockChan) // Get should work. pair, err := otherConn.Get(ctx, key, nil) require.NoError(t, err) checkPairNotNil(t, pair) assert.Equal(t, pair.Value, value) assert.NotEqual(t, pair.LastIndex, 0) time.Sleep(3 * time.Second) value = []byte("foobar") // Create a new lock with another connection. lock, err := kv.NewLock(ctx, key, &store.LockOptions{ Value: value, TTL: 3 * time.Second, }) require.NoError(t, err) require.NotNil(t, lock) ctxLock, cancelLock := context.WithTimeout(ctx, 4*time.Second) defer cancelLock() // Lock should block, the session on the lock // is still active and renewed periodically. lockChan, _ = lock.Lock(ctxLock) require.Nil(t, lockChan) // Close the connection. _ = otherConn.Close() // Force to stop the session renewal for the lock. close(renewCh) // Let the session on the lock expire. time.Sleep(3 * time.Second) // Lock should now succeed for the other client. locked := make(chan struct{}) go func(<-chan struct{}) { lockChan, err = lock.Lock(ctx) require.NoError(t, err) assert.NotNil(t, lockChan) locked <- struct{}{} }(locked) select { case <-locked: break case <-time.After(4 * time.Second): t.Fatal("Unable to take the lock, timed out") } // Get should work with the new value. pair, err = kv.Get(ctx, key, nil) require.NoError(t, err) checkPairNotNil(t, pair) assert.Equal(t, pair.Value, value) assert.NotEqual(t, pair.LastIndex, 0) err = lock.Unlock(ctx) require.NoError(t, err) } func testPutTTL(t *testing.T, kv store.Store, otherConn store.Store) { t.Helper() firstKey := "testPutTTL" firstValue := []byte("foo") secondKey := "second" secondValue := []byte("bar") ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() // Put the first key with the Ephemeral flag. err := otherConn.Put(ctx, firstKey, firstValue, &store.WriteOptions{TTL: 2 * time.Second}) require.NoError(t, err) // Put a second key with the Ephemeral flag. err = otherConn.Put(ctx, secondKey, secondValue, &store.WriteOptions{TTL: 2 * time.Second}) require.NoError(t, err) // Get on firstKey should work. pair, err := kv.Get(ctx, firstKey, nil) require.NoError(t, err) checkPairNotNil(t, pair) // Get on secondKey should work. pair, err = kv.Get(ctx, secondKey, nil) require.NoError(t, err) checkPairNotNil(t, pair) // Close the connection. _ = otherConn.Close() // Let the session expire. time.Sleep(3 * time.Second) // Get on firstKey shouldn't work. pair, err = kv.Get(ctx, firstKey, nil) assert.Error(t, err) assert.Nil(t, pair) // Get on secondKey shouldn't work. pair, err = kv.Get(ctx, secondKey, nil) assert.Error(t, err) assert.Nil(t, pair) } func testList(t *testing.T, kv store.Store) { t.Helper() parentKey := "testList" childKey := "testList/child" subfolderKey := "testList/subfolder" ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() // Put the parent key. err := kv.Put(ctx, parentKey, nil, &store.WriteOptions{IsDir: true}) require.NoError(t, err) // Put the first child key. err = kv.Put(ctx, childKey, []byte("first"), nil) require.NoError(t, err) // Put the second child key which is also a directory. err = kv.Put(ctx, subfolderKey, []byte("second"), &store.WriteOptions{IsDir: true}) require.NoError(t, err) // Put child keys under secondKey. for i := 1; i <= 3; i++ { key := "testList/subfolder/key" + strconv.Itoa(i) err = kv.Put(ctx, key, []byte("value"), nil) require.NoError(t, err) } // List should work and return five child entries. for _, parent := range []string{parentKey, parentKey + "/"} { pairs, errList := kv.List(ctx, parent, nil) require.NoError(t, errList) assert.Len(t, pairs, 5) } // List on childKey should return 0 keys. pairs, err := kv.List(ctx, childKey, nil) require.NoError(t, err) assert.Empty(t, pairs) // List on subfolderKey should return 3 keys without the directory. pairs, err = kv.List(ctx, subfolderKey, nil) require.NoError(t, err) assert.Len(t, pairs, 3) // List should fail: the key does not exist. pairs, err = kv.List(ctx, "idontexist", nil) assert.ErrorIs(t, err, store.ErrKeyNotFound) assert.Nil(t, pairs) } func testListLockKey(t *testing.T, kv store.Store) { t.Helper() listKey := "testListLockSide" ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() err := kv.Put(ctx, listKey, []byte("val"), &store.WriteOptions{IsDir: true}) require.NoError(t, err) err = kv.Put(ctx, listKey+"/subfolder", []byte("val"), &store.WriteOptions{IsDir: true}) require.NoError(t, err) // Put keys under subfolder. for i := 1; i <= 3; i++ { key := listKey + "/subfolder/key" + strconv.Itoa(i) errPut := kv.Put(ctx, key, []byte("val"), nil) require.NoError(t, errPut) // We lock the child key. lock, errPut := kv.NewLock(ctx, key, &store.LockOptions{Value: []byte("locked"), TTL: 2 * time.Second}) require.NoError(t, errPut) require.NotNil(t, lock) lockChan, errPut := lock.Lock(ctx) require.NoError(t, errPut) assert.NotNil(t, lockChan) } // List children of the root directory (`listKey`), this should // not output any `___lock` entries and must contain 4 results. pairs, err := kv.List(ctx, listKey, nil) require.NoError(t, err) assert.Len(t, pairs, 4) for _, pair := range pairs { if strings.Contains(pair.Key, "___lock") { assert.FailNow(t, "tesListLockKey: found a key containing lock suffix '___lock'") } } } func testDeleteTree(t *testing.T, kv store.Store) { t.Helper() prefix := "testDeleteTree" firstKey := "testDeleteTree/first" firstValue := []byte("first") secondKey := "testDeleteTree/second" secondValue := []byte("second") ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() // Put the first key. err := kv.Put(ctx, firstKey, firstValue, nil) require.NoError(t, err) // Put the second key. err = kv.Put(ctx, secondKey, secondValue, nil) require.NoError(t, err) // Get should work on the first Key. pair, err := kv.Get(ctx, firstKey, nil) require.NoError(t, err) checkPairNotNil(t, pair) assert.Equal(t, pair.Value, firstValue) assert.NotEqual(t, 0, pair.LastIndex) // Get should work on the second Key. pair, err = kv.Get(ctx, secondKey, nil) require.NoError(t, err) checkPairNotNil(t, pair) assert.Equal(t, pair.Value, secondValue) assert.NotEqual(t, 0, pair.LastIndex) // Delete Values under directory `nodes`. err = kv.DeleteTree(ctx, prefix) require.NoError(t, err) // Get should fail on both keys. pair, err = kv.Get(ctx, firstKey, nil) assert.Error(t, err) assert.Nil(t, pair) pair, err = kv.Get(ctx, secondKey, nil) assert.Error(t, err) assert.Nil(t, pair) } // RunCleanup cleans up keys introduced by the tests. func RunCleanup(t *testing.T, kv store.Store) { t.Helper() ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() for _, key := range []string{ "testAtomicPutWithSlashSuffixKey", "testPutGetDeleteExists", "testWatch", "testWatchTree", "testAtomicPut", "testAtomicPutCreate", "testAtomicDelete", "testLockUnlock", "testLockTTL", "testPutTTL", "testList/subfolder", "testList", "testListLockSide/subfolder", "testListLockSide", "testDeleteTree", } { err := kv.DeleteTree(ctx, key) if err != nil { assert.ErrorIsf(t, err, store.ErrKeyNotFound, "failed to delete tree key %s", key) } err = kv.Delete(ctx, key) if err != nil { assert.ErrorIsf(t, err, store.ErrKeyNotFound, "failed to delete key %s", key) } } } ================================================ FILE: valkeyrie.go ================================================ // Package valkeyrie Distributed Key/Value Store Abstraction Library written in Go. package valkeyrie import ( "context" "sort" "sync" "github.com/kvtools/valkeyrie/store" ) var ( constructorsMu sync.RWMutex constructors = make(map[string]Constructor) ) // Config the raw type of the store configurations. type Config any // Constructor The signature of a store constructor. type Constructor func(ctx context.Context, endpoints []string, options Config) (store.Store, error) // Register makes a store constructor available by the provided name. // If Register is called twice with the same name or if constructor is nil, it panics. func Register(name string, cttr Constructor) { constructorsMu.Lock() defer constructorsMu.Unlock() if cttr == nil { panic("valkeyrie: Register constructor is nil") } if _, dup := constructors[name]; dup { panic("valkeyrie: Register called twice for constructor " + name) } constructors[name] = cttr } // Unregister Unregisters a store. func Unregister(storeName string) { constructorsMu.Lock() defer constructorsMu.Unlock() delete(constructors, storeName) } // UnregisterAllConstructors Unregisters all stores. func UnregisterAllConstructors() { constructorsMu.Lock() defer constructorsMu.Unlock() constructors = make(map[string]Constructor) } // Constructors returns a sorted list of the names of the registered constructors. func Constructors() []string { constructorsMu.RLock() defer constructorsMu.RUnlock() list := make([]string, 0, len(constructors)) for name := range constructors { list = append(list, name) } sort.Strings(list) return list } // NewStore creates a new store instance. func NewStore(ctx context.Context, storeName string, endpoints []string, options Config) (store.Store, error) { constructorsMu.RLock() construct, ok := constructors[storeName] constructorsMu.RUnlock() if !ok { return nil, &store.UnknownConstructorError{Store: storeName} } if construct == nil { return nil, &store.UnknownConstructorError{Store: storeName} } return construct(ctx, endpoints, options) } ================================================ FILE: valkeyrie_test.go ================================================ package valkeyrie import ( "context" "testing" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" ) func TestRegister(t *testing.T) { t.Cleanup(UnregisterAllConstructors) Register(testStoreName, newStore) assert.Len(t, constructors, 1) } func TestRegister_duplicate(t *testing.T) { t.Cleanup(UnregisterAllConstructors) Register(testStoreName, newStore) assert.Len(t, constructors, 1) assert.Panics(t, func() { Register(testStoreName, newStore) }) } func TestRegister_nil(t *testing.T) { t.Cleanup(UnregisterAllConstructors) assert.Panics(t, func() { Register(testStoreName, nil) }) } func TestUnregister(t *testing.T) { t.Cleanup(UnregisterAllConstructors) Register(testStoreName, newStore) assert.Len(t, constructors, 1) Unregister(testStoreName) constructorsMu.Lock() defer constructorsMu.Unlock() assert.Empty(t, constructors) } func TestConstructors(t *testing.T) { t.Cleanup(UnregisterAllConstructors) Register(testStoreName, newStore) assert.Len(t, constructors, 1) cttrs := Constructors() expected := []string{testStoreName} assert.Equal(t, expected, cttrs) } func TestNewStore(t *testing.T) { t.Cleanup(UnregisterAllConstructors) Register(testStoreName, newStore) assert.Len(t, constructors, 1) s, err := NewStore(context.Background(), testStoreName, nil, nil) require.NoError(t, err) assert.NotNil(t, s) assert.IsType(t, &Mock{}, s) }