Repository: aredington/schism Branch: master Commit: c4d042f7c5cf Files: 28 Total size: 134.9 KB Directory structure: gitextract_v04t0_24/ ├── .gitignore ├── CHANGELOG.md ├── LICENSE ├── README.md ├── project.clj ├── resources/ │ └── data_readers.cljc ├── src/ │ └── schism/ │ ├── core.cljc │ ├── impl/ │ │ ├── core.cljc │ │ ├── protocols.cljc │ │ ├── types/ │ │ │ ├── list.cljc │ │ │ ├── map.cljc │ │ │ ├── nested_map.cljc │ │ │ ├── nested_vector.cljc │ │ │ ├── nesting_util.cljc │ │ │ ├── set.cljc │ │ │ └── vector.cljc │ │ └── vector_clock.cljc │ └── node.cljc └── test/ └── schism/ ├── core_test.cljc ├── impl/ │ ├── types/ │ │ ├── list_test.cljc │ │ ├── map_test.cljc │ │ ├── nested_map_test.cljc │ │ ├── nested_vector_test.cljc │ │ ├── set_test.cljc │ │ └── vector_test.cljc │ └── vector_clock_test.cljc ├── node_test.cljc └── test.cljc ================================================ FILE CONTENTS ================================================ ================================================ FILE: .gitignore ================================================ /target /classes /checkouts pom.xml pom.xml.asc *.jar *.class /.lein-* /.nrepl-port .hgignore .hg/ node_modules/* ================================================ FILE: CHANGELOG.md ================================================ # Change Log All notable changes to this project will be documented in this file. This change log follows the conventions of [keepachangelog.com](http://keepachangelog.com/). ## [Unreleased] ### Changed - Add a new arity to `make-widget-async` to provide a different widget shape. ## [0.1.1] - 2018-01-05 ### Changed - Documentation on how to make the widgets. ### Removed - `make-widget-sync` - we're all async, all the time. ### Fixed - Fixed widget maker to keep working when daylight savings switches over. ## 0.1.0 - 2018-01-05 ### Added - Files from the new template. - Widget maker public API - `make-widget-sync`. [Unreleased]: https://github.com/your-name/schism/compare/0.1.1...HEAD [0.1.1]: https://github.com/your-name/schism/compare/0.1.0...0.1.1 ================================================ FILE: LICENSE ================================================ Copyright 2021 Alex Redington Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: README.md ================================================ # schism A batteries included library of CRDT implementations of Clojure's core data types: Sets, Maps, Vectors, and Lists with support for distributed modification and eventual consistency. ## Dependency Information Latest release: 0.1.2 [Leiningen](http://github.com/technomancy/leiningen/) and [Boot](http://boot-clj.com) dependency information: ``` [com.holychao/schism "0.1.2"] ``` ## Motivation Clojure is one of a handful of languages which can be authored and executed in both a high performance server environment and in a web browser. This strength affords many benefits, one of which the language does not cover is allowing for concurrent modification of data with rich synchronization semantics. There are some other efforts similar to this. Schism's aims are: - To minimize the locus of concerns outside of collection data structures. - To provide collection data structures with inferior performance to Clojure's own persistent data structures, but which only incur sub-linear cost increases, available through the same interfaces. - To provide collection data structures with greater storage, and serialization costs than Clojure's own persistent data structures, which grow with an upper bound of the number of elements in the collection, independent of the number of operations against that collection. - Allow for a Clojure and ClojureScript processes executing on separate nodes to maintain convergent data. ## Limitations Because schism refuses to use tombstones, some convergence operations will have different results than structures which will embrace the costs of tombstones. For example, Schism's set may drop a recently added element during convergence with certain vector clock states. While this quality is undesirable, I accept it more readily than monotonically increasing storage requirements for data explicitly intended for communication between nodes. Future work may pursue allowing for the convergence operation to have some configurability so that vector clocks will retain information more eagerly to reduce the incidence of this phenomena. ## Usage `schism.core` contains functions for generating new collections, e.g. `schism.core/convergent-set`. These functions accept arguments much like their Clojure core equivalents, so `(convergent-set :a :b :c)` creates a set equivalent to `#{:a :b :c}` These collections support Clojure's collection operations with the same semantic consequences (save for above mentioned performance and storage costs). Any divergence between a schism collection's behavior and a Clojure collection's behavior is a bug. Please file a report, as these should be relatively fast and easy to fix. String coercion of a schism collection is identical to that of a Clojure collection, e.g. a convergent set will coerce to a string as `#{:a :b :c}`. However, `pr-str` will generate a longer tagged literal containing all synchronization data for the structure to operate correctly. Schism eagerly enables edn readers for its structures, so synchronization can be as simple as sending the results of `pr-str` over the wire, reading on the receiving side and converging with the receiver's local copy. Convergence is done with `schism.core/converge`. It accepts two collections, which it assumes are both copies of the same replicated collection, and returns a single collection: the result of convergence. Converge replicates the metadata of its first argument into its return value, if present. Each CLJ and CLJS process working with any shared schism collection is a node in a distributed computing cluster. Each node should have an identity in order for synchronization to operate correctly. You may invoke `schism.core/initialize-node!` with no arguments to initialize the current process to have a randomly generated UUID as its node identifier. You may use any serializable value as the node identifier by passing that value to initialize-node. If you can create stable node identifiers, that can lead to some minor reduction in the storage requirements for schism's data structures. If two nodes operate on a replicated collection independently and do NOT have distinct node identifiers, schism's convergence behavior is undefined. (Don't do this). If you do not explicitly invoke `initialize-node!`, it is left at the value `nil`. ## Nested collections It is common to build up a tree of maps and vectors to create several addressable values that cohere to a common whole. If one built this collection up out of individual schism collections, a number of problems with convergence and serialization would present themselves. I have provided `schism.core/nested-map` and `schism.core/nested-vector` to address both these problems. While these provide best-least-surprise convergence, it's important to understand the limitations and leverages of best: - These collections incur substantially more CPU time to conduct simple operations as a isomorphic mirror of all modifications must be computed on each update. - While adding empty collections should work, please avoid doing so. - `clojure.core/assoc-in`, `clojure.core/update-in`, and friends should work with the convergent map without any special handling. - Vectors containing leaf nodes will compose tail insertions, so two nodes adding to the tail with `conj` will have both of their additions retained in chronological order. - Vectors at intermediate nodes will treat the child at the index as identity. - Collections must be isomorphic to converge: Do not allow one node to place a vector and another node to place a map at the same path in the tree. Given the additional complications I strongly encourage clients to pursue a strategy of retaining a shallow convergent-vector of entity ids and one convergent-map for each entity. For those cases where this combo is not sufficient, please wield `nested-map` and `nested-vector` with care. ## Further work - Configurable convergence - Other good ideas as the community provides them ## Contributing I don't use CA's or other such things. Bugfixes are welcome and appreciated. I reserve the right to dismiss feature requests in the guise of PRs. All work will be evaluated in light of conformance with motivations as stated in this document. ## License Copyright © 2020 Alex Redington Distributed under the MIT License ================================================ FILE: project.clj ================================================ (defproject com.holychao/schism "0.1.2" :description "First Class CRDTs for Clojure" :url "https://github.com/aredington/schism" :license {:name "MIT License" :url "https://opensource.org/licenses/MIT"} :dependencies [[org.clojure/clojure "1.10.0" :scope "provided"] [org.clojure/clojurescript "1.10.520" :scope "provided"]] :plugins [[lein-cljsbuild "1.1.7"] [lein-doo "0.1.11"]] :profiles {:dev {:dependencies [[doo "0.1.11"] [org.clojure/test.check "0.10.0-alpha4"]]}} :cljsbuild {:builds [{:id "test" :source-paths ["src" "test"] :compiler {:output-to "target/test.js" :main schism.test :output-dir "target" :optimizations :none :source-map true :pretty-print true :recompile-dependents false :parallel-build true :checked-arrays :warn}}]} :clean-targets ^{:protect false} ["target"] :aliases {"test-platforms" ["do" "clean," "test," "doo" "chrome-headless" "test" "once"]}) ================================================ FILE: resources/data_readers.cljc ================================================ {schism/set schism.impl.types.set/read-edn-set schism/map schism.impl.types.map/read-edn-map schism/list schism.impl.types.list/read-edn-list schism/vector schism.impl.types.vector/read-edn-vector schism/nested-map schism.impl.types.nested-map/read-edn-map schism/nested-vector schism.impl.types.nested-vector/read-edn-vector} ================================================ FILE: src/schism/core.cljc ================================================ (ns schism.core (:require [schism.impl.types.set :as sset] [schism.impl.types.map :as smap] [schism.impl.types.list :as slist] [schism.impl.types.vector :as svector] [schism.impl.types.nested-map :as nmap] [schism.impl.types.nested-vector :as nvector] [schism.impl.protocols :as sp] [schism.node :as sn])) (defn convergent-set "Create a new ORSWOT containing args. Each arg will be recorded as being added to the set by the current node at invocation time." [& args] (apply sset/new-set args)) (defn convergent-map "Create a new ORMWOT, establishing associations between each pair of key-value arguments. Each entry will be recorded as being added to the map by the current node at invocation time." [& args] (apply smap/new-map args)) (defn convergent-list "Create a new convergent list containing args. Args will be placed into the list in the order they appear during invocation, just as with `clojure.core/list`. Each entry will be recorded as being added to the list by the current node at invocation time." [& args] (apply slist/new-list args)) (defn convergent-vector "Create a new convergent vector containing args. Each entry will be recorded as being added to the list by the current node at invocation time, in the ordinal position it occupied during invocation." [& args] (apply svector/new-vector args)) (defn nested-map "Create a new convergent, nesting map containing args. Each non-collection value at any location in the map will be treated as a single atomic value, allowing for discrete modification of each terminal in the tree. Those collections returned by `nested-map` cannot perform as well as those returned by `convergent-map` and have higher computational and storage costs." [& args] (apply nmap/new-map args)) (defn nested-vector "Create a new convergent, nesting vector containing args. Each non-collection value at any location in the vector will be treated as a single atomic value, allowing for discrete modification of each terminal in the tree. Those collections returned by `nested-vector` cannot perform as well as those returned by `convergent-vector` and have higher computational and storage costs." [& args] (apply nvector/new-vector args)) (defn converge "Return a converged copy of `c1` containing the modifications of `c2`. Convergence is defined on a per-type basis. If `c1` has metadata, retain that metadata on the returned result. Convergence ticks the vector clock for the node on which convergence is occurring. `c1` and `c2` must be collections of the same type. The behavior of `converge` is not defined when either: - The current value of `schism.node/*current-node*` is nil - The current value of `schism.node/*current-node*` is shared with another node making modifications to the same logical collection." [c1 c2] (sp/synchronize c1 c2)) (def initialize-node! "Initialize the current node to an edn serializable value if provided. If invoked with no argument, initializes the current node to a random UUID." sn/initialize-node!) (defmacro with-node "Run `body` with the current node set to `node-id`" [id & body] `(sn/with-node ~id ~@body)) ================================================ FILE: src/schism/impl/core.cljc ================================================ (ns schism.impl.core (:require [schism.node :as node] [clojure.set :as set]) #?(:clj (:import (java.util Date)))) (def to-millis (memfn ^Date getTime)) (defn to-date [millis] #?(:clj (Date. millis) :cljs (js/Date. millis))) (defn now [] #?(:clj (Date.) :cljs (js/Date.))) (defn node-and-threshold [data] (->> data :vector-clock (reduce-kv (fn [[node time] candidate-node candidate-time] (if (< (to-millis time) (to-millis candidate-time)) [candidate-node candidate-time] [node time])) (-> data :vector-clock first)) (#(update % 1 to-millis)))) (defn retain-elements "Accepts two maps of the form {:vector-clock :elements :author-node :record-time }>} Returns a seq of elements to be retained using ORSWOT merge semantics." [own-data other-data] (let [other-threshold (-> own-data :vector-clock (get node/*current-node*) to-millis) [other-node own-threshold] (node-and-threshold other-data) own-vclock-for-other (-> own-data :vector-clock (get other-node)) other-vclock-limiter (if own-vclock-for-other (fn [{:keys [record-time] :as element}] (>= (to-millis own-vclock-for-other) (to-millis record-time))) (constantly true)) other-vclock-for-own (-> other-data :vector-clock (get node/*current-node*)) own-vclock-limiter (if other-vclock-for-own (fn [{:keys [record-time] :as element}] (>= (to-millis other-vclock-for-own) (to-millis record-time))) (constantly true)) other-additions (remove #(and (> other-threshold (to-millis (:record-time %))) (other-vclock-limiter %)) (:elements other-data)) own-additions (remove #(and (> own-threshold (to-millis (:record-time %))) (own-vclock-limiter %)) (:elements own-data))] (concat other-additions own-additions))) (defn common-elements "Accepts maps of the form {:vector-clock :elements :author-node :record-time }>} Returns a vector of the common elements." [& datasets] (apply set/intersection (map (comp set :elements) datasets))) (defn distinct-data "Accepts maps of the form {:vector-clock :elements :author-node :record-time }>} Returns a vector of the maps with the common elements entries removed." [& datasets] (let [common-elements (apply common-elements datasets)] (->> datasets (map #(update % :elements (fn [elements] (remove common-elements elements)))) (into [])))) (defn merged-clock "Accepts a collection of elements, and two or more datasets of the form {:vector-clock :elements :author-node :record-time }>} Returns a vector clock of the relevant nodes, that being the nodes referenced as :author-node in elements." [elements & datasets] (let [relevant-nodes (set (map :author-node elements))] (-> (apply merge-with (partial max-key to-millis) (map :vector-clock datasets)) (select-keys relevant-nodes)))) (def tail-insertion-sort-value "The value to use when sorting insertions by index, and the recorded index was -1, indicating the element was inserted at the tail." #?(:clj Long/MAX_VALUE :cljs (.-MAX_SAFE_INTEGER js/Number))) (defn assoc-n-with-tail-support "Assoc with support to place `v` at the tail of `a` when `n` is -1." [a n v] (if (= n -1) (conj a v) (assoc a n v))) ================================================ FILE: src/schism/impl/protocols.cljc ================================================ (ns schism.impl.protocols) (defprotocol Convergent (synchronize [convergent other] "Synchronizes `convergent` with `other` such that all changes incorporated into `other` will be represented in a new persistent structure derived from `convergent`.")) (defprotocol Vclocked "A protocol for obtaining the current vector clock of a value, and for deriving a new vector clock for a value." (get-clock [clocked] "Returns the current vector clock of `clocked`, a map of node IDs to timestamps.") (with-clock [clocked clock] "Returns a new structure derived from `clocked`, associating `clock` with the returned value.")) (extend-protocol Vclocked nil (get-clock [_] {}) (with-clock [_ clock] nil)) ================================================ FILE: src/schism/impl/types/list.cljc ================================================ (ns schism.impl.types.list "Definition and support for Schism's Convergent List type. The convergent list is a simple timestamped log of entries with a vector clock. Convergence places entries into the resultant list in insertion order. The vector clock conveys that an item has been removed from the list on another node." (:require [schism.impl.protocols :as proto] [schism.impl.vector-clock :as vc] [schism.impl.core :as ic] [schism.node :as node] #?(:cljs [cljs.reader :as reader])) #?(:cljs (:require-macros [schism.impl.vector-clock :as vc])) #?(:clj (:import (clojure.lang IPersistentCollection IPersistentStack IReduce Counted IHashEq Seqable IObj IMeta ISeq) (java.io Writer) (java.util Date Collection) (java.lang Object)))) ;; A CLJ & CLJS implementation of a convergent list ;; Each list maintains its own vector clock, and insertion times and ;; nodes for each element of the list. ConvergentList entries and ;; insertions are correlated positionally (as the list may contain the ;; same item multiple times.) Insertion times dictate ordering. The ;; vector clock determines if an entry has been removed. (declare clist-conj clist-rest clist-empty) #?(:clj (deftype ConvergentList [data vclock insertions] Counted (count [this] (.count ^Counted (.-data this))) IPersistentCollection (cons [this o] (clist-conj this o)) (empty [this] (clist-empty this)) (equiv [this other] (.equiv ^IPersistentCollection (.-data this) other)) Object (equals [this o] (.equals (.-data this) o)) (hashCode [this] (.hashCode (.-data this))) (toString [this] (.toString (.-data this))) IHashEq (hasheq [this] (.hasheq ^IHashEq (.-data this))) Seqable (seq [this] this) java.util.List (add [this o] (.add ^java.util.List (.-data this) o)) (add [this index o] (.add ^java.util.List (.-data this) index o)) (addAll [this c] (.addAll ^java.util.List (.-data this) c)) (clear [this] (.clear ^java.util.List (.-data this))) (contains [this o] (.contains ^java.util.List (.-data this) o)) (containsAll [this c] (.containsAll ^java.util.List (.-data this) c)) (get [this i] (.get ^java.util.List (.-data this) i)) (indexOf [this o] (.indexOf ^java.util.List (.-data this) o)) (isEmpty [this] (.isEmpty ^java.util.List (.-data this))) (iterator [this] (.iterator ^java.util.List (.-data this))) (lastIndexOf [this o] (.lastIndexOf ^java.util.List (.-data this) o)) (listIterator [this] (.listIterator ^java.util.List (.-data this))) (^Object remove [this ^int i] (.remove ^java.util.List (.-data this) i)) (^boolean remove [this ^Object o] (.remove ^java.util.List (.-data this) o)) (removeAll [this c] (.removeAll ^java.util.List (.-data this) c)) (replaceAll [this op] (.replaceAll ^java.util.List (.-data this) op)) (retainAll [this c] (.retainAll ^java.util.List (.-data this) c)) (set [this i e] (.set ^java.util.List (.-data this) i e)) (size [this] (.size ^java.util.List (.-data this))) (sort [this c] (.sort ^java.util.List (.-data this) c)) (spliterator [this] (.spliterator ^java.util.List (.-data this))) (subList [this i j] (.subList ^java.util.List (.-data this) i j)) (toArray [this] (.toArray ^java.util.List (.-data this))) (^"[Ljava.lang.Object;" toArray [this ^"[Ljava.lang.Object;" a] (.toArray ^java.util.List (.-data this) a)) IObj (withMeta [this meta] (ConvergentList. (with-meta ^IObj (.-data this) meta) (.-vclock this) (.-insertions this))) IMeta (meta [this] (.meta ^IMeta (.-data this))) IReduce (reduce [this f] (.reduce ^IReduce (.-data this) f)) IPersistentStack (peek [this] (.peek ^IPersistentStack (.-data this))) (pop [this] (clist-rest this)) ISeq (first [this] (.first ^ISeq (.-data this))) (next [this] (clist-rest this)) (more [this] (clist-rest this))) :cljs (deftype ConvergentList [data vclock insertions] ICounted (-count [this] (-count (.-data this))) IEmptyableCollection (-empty [this] (clist-empty this)) ICollection (-conj [this o] (clist-conj this o)) IEquiv (-equiv [this other] (-equiv (.-data this) other)) IPrintWithWriter (-pr-writer [o writer opts] (-write writer "#schism/list [") (-write writer (pr-str (.-data o))) (-write writer ", ") (-write writer (pr-str (.-vclock o))) (-write writer ", ") (-write writer (pr-str (.-insertions o))) (-write writer "]")) IHash (-hash [this] (-hash (.-data this))) ISeqable (-seq [this] this) Object (toString [this] (.toString (.-data this))) IMeta (-meta [this] (-meta (.-data this))) IWithMeta (-with-meta [this meta] (ConvergentList. (-with-meta (.-data this) meta) (.-vclock this) (.-insertions this))) ISeq (-first [this] (-first (.-data this))) (-rest [this] (clist-rest this)))) (defn clist-conj [^ConvergentList clist o] (vc/update-clock now clist (ConvergentList. (conj (.-data clist) o) (.-vclock clist) (conj (.-insertions clist) [node/*current-node* now])))) (defn clist-empty [^ConvergentList clist] (vc/update-clock _ clist (ConvergentList. (list) (hash-map) (list)))) (defn clist-rest [^ConvergentList clist] (vc/update-clock _ clist (ConvergentList. (rest (.-data clist)) (.-vclock clist) (rest (.-insertions clist))))) (defn- elemental-data [^ConvergentList l] {:vector-clock (.-vclock l) :elements (into [] (for [[datum [author-node record-time]] (map vector (.-data l) (.-insertions l))] {:data datum :author-node author-node :record-time record-time}))}) (extend-type ConvergentList proto/Vclocked (get-clock [this] (.-vclock this)) (with-clock [this new-clock] (ConvergentList. (.-data this) new-clock (.-insertions this))) proto/Convergent (synchronize [this ^ConvergentList other] (let [own-meta (-> this .-data meta) own-data (elemental-data this) other-data (elemental-data other) retain (->> (reverse (:elements other-data)) (map vector (reverse (:elements own-data))) (take-while (partial apply =)) reverse (map first)) completed-elements (concat (sort-by :record-time (apply ic/retain-elements (ic/distinct-data own-data other-data))) retain) completed-data (->> completed-elements (map :data) (into '()) reverse) completed-insertions (->> completed-elements (map (fn [{:keys [author-node record-time]}] [author-node record-time])) (into '()) reverse) completed-vclock (ic/merged-clock completed-elements own-data other-data)] (vc/update-clock _ this (ConvergentList. (with-meta completed-data own-meta) completed-vclock completed-insertions))))) #?(:clj (defmethod print-method ConvergentList [^ConvergentList l ^Writer writer] (.write writer "#schism/list [") (.write writer (pr-str (.-data l))) (.write writer ", ") (.write writer (pr-str (.-vclock l))) (.write writer ", ") (.write writer (pr-str (.-insertions l))) (.write writer "]"))) (defn read-edn-list [read-object] (let [[data vclock insertions] read-object] (ConvergentList. data vclock insertions))) #?(:cljs (cljs.reader/register-tag-parser! 'schism/list read-edn-list)) (defn new-list ([] (ConvergentList. (list) (hash-map) (list))) ([& args] (vc/update-clock now nil (ConvergentList. (apply list args) (hash-map) (apply list (repeat (count args) [node/*current-node* now])))))) ================================================ FILE: src/schism/impl/types/map.cljc ================================================ (ns schism.impl.types.map "Definition and support for Schism's Convergent Map type, an ORMWOT implemented on top of Clojure's persistent maps and a Schism Vector Clock." (:require [schism.impl.protocols :as proto] [schism.impl.vector-clock :as vc] [schism.impl.core :as ic] [schism.node :as node] [clojure.set :as set] #?(:cljs [cljs.reader :as reader])) #?(:cljs (:require-macros [schism.impl.vector-clock :as vc])) #?(:clj (:import (clojure.lang IPersistentCollection IPersistentMap IHashEq Associative ILookup Counted Seqable IMapIterable IKVReduce IFn IObj IMeta) (java.io Writer) (java.util Date Collection) (java.lang Object)))) ;; A CLJS and CLJ imeplementation of ORMWOT (Observed-Removed Map without Tombstones) ;; Each map maintains its own vector clock, and birth dots for each ;; map entry by key. A birth dot consists of the node adding the ;; entry, and the date at which it was added. The vector clock ;; determines if an entry has been removed: a compared map absent an ;; entry, but with a newer vector clock than the own birthdot on the ;; entry indicates that it was removed; a compared map absent an entry ;; with an older vector clock indicates that the birthdot was never ;; seen and should be in the merged map. Birthdots also arbitrate ;; entries: for each entry the last writer wins. (declare ormwot-conj ormwot-empty ormwot-assoc ormwot-dissoc) (def not-found-sym (gensym :not-found)) #?(:clj (deftype Map [data vclock birth-dots] Counted (count [this] (.count ^Counted (.-data this))) IPersistentCollection (cons [this o] (ormwot-conj this o)) (empty [this] (ormwot-empty this)) (equiv [this other] (.equiv ^IPersistentCollection (.-data this) other)) IPersistentMap (assoc [this k v] (ormwot-assoc this k v)) (assocEx [this k v] (when (not= (get this k not-found-sym) not-found-sym) (throw (ex-info "Attempt to assocEx on map with key" {:map this :key k :value v}))) (ormwot-assoc this k v)) (without [this k] (ormwot-dissoc this k)) Object (equals [this o] (.equals (.-data this) o)) (hashCode [this] (.hashCode (.-data this))) (toString [this] (.toString (.-data this))) ILookup (valAt [this k] (.valAt ^ILookup (.-data this) k)) (valAt [this k not-found] (.valAt ^ILookup (.-data this) k not-found)) IMapIterable (keyIterator [this] (.keyIterator ^IMapIterable (.-data this))) (valIterator [this] (.valIterator ^IMapIterable (.-data this))) IKVReduce (kvreduce [this f init] (.kvreduce ^IKVReduce (.-data this) f init)) IHashEq (hasheq [this] (.hasheq ^IHashEq (.-data this))) Seqable (seq [this] (.seq ^Seqable (.-data this))) java.util.Map (clear [this] (.clear ^java.util.Map (.-data this))) (compute [this k f] (.compute ^java.util.Map (.-data this) k f)) (computeIfAbsent [this k f] (.computeIfAbsent ^java.util.Map (.-data this) k f)) (computeIfPresent [this k f] (.computeIfPresent ^java.util.Map (.-data this) k f)) (containsKey [this k] (.containsKey ^java.util.Map (.-data this) k)) (containsValue [this v] (.containsValue ^java.util.Map (.-data this) v)) (entrySet [this] (.entrySet ^java.util.Map (.-data this))) (get [this k] (.get ^java.util.Map (.-data this) k)) (getOrDefault [this k not-found] (.getOrDefault ^java.util.Map (.-data this) k not-found)) (isEmpty [this] (.isEmpty ^java.util.Map (.-data this))) (keySet [this] (.keySet ^java.util.Map (.-data this))) (merge [this k v f] (.merge ^java.util.Map (.-data this) k v f)) (put [this k v] (.put ^java.util.Map (.-data this) k v)) (putAll [this m] (.putAll ^java.util.Map (.-data this) m)) (putIfAbsent [this k v] (.putIfAbsent ^java.util.Map (.-data this) k v)) (remove [this k] (.remove ^java.util.Map (.-data this) k)) (remove [this k v] (.remove ^java.util.Map (.-data this) k v)) (replace [this k v] (.replace ^java.util.Map (.-data this) k v)) (replace [this k ov nv] (.replace ^java.util.Map (.-data this) k ov nv)) (replaceAll [this f] (.replaceAll ^java.util.Map (.-data this) f)) (size [this] (.size ^java.util.Map (.-data this))) (values [this] (.values ^java.util.Map (.-data this))) IFn (invoke [this k] (.invoke ^IFn (.-data this) k)) (invoke [this k not-found] (.invoke ^IFn (.-data this) k not-found)) IObj (withMeta [this meta] (Map. (with-meta ^IObj (.-data this) meta) (.-vclock this) (.-birth-dots this))) IMeta (meta [this] (.meta ^IMeta (.-data this)))) :cljs (deftype Map [data vclock birth-dots] ICounted (-count [this] (-count (.-data this))) IEmptyableCollection (-empty [this] (ormwot-empty this)) ICollection (-conj [this o] (ormwot-conj this o)) IAssociative (-contains-key? [this k] (-contains-key? (.-data this) k)) (-assoc [this k v] (ormwot-assoc this k v)) IFind (-find [this k] (-find (.-data this) k)) IMap (-dissoc [this k] (ormwot-dissoc this k)) IKVReduce (-kv-reduce [this f init] (-kv-reduce (.-data this) f init)) IEquiv (-equiv [this other] (-equiv (.-data this) other)) ILookup (-lookup [this o] (-lookup (.-data this) o)) (-lookup [this o not-found] (-lookup (.-data this) o not-found)) IPrintWithWriter (-pr-writer [o writer opts] (-write writer "#schism/map [") (-write writer (pr-str (.-data o))) (-write writer ", ") (-write writer (pr-str (.-vclock o))) (-write writer ", ") (-write writer (pr-str (.-birth-dots o))) (-write writer "]")) IHash (-hash [this] (-hash (.-data this))) IFn (-invoke [this o] ((.-data this) o)) (-invoke [this o not-found] ((.-data this) o not-found)) ISeqable (-seq [this] (-seq (.-data this))) Object (toString [this] (.toString (.-data this))) IMeta (-meta [this] (-meta (.-data this))) IWithMeta (-with-meta [this meta] (Map. (-with-meta (.-data this) meta) (.-vclock this) (.-birth-dots this))))) (defn ormwot-conj [^Map ormwot pair] (vc/update-clock now ormwot (Map. (conj (.-data ormwot) pair) (.-vclock ormwot) (assoc (.-birth-dots ormwot) (first pair) [node/*current-node* now])))) (defn ormwot-empty [^Map ormwot] (vc/update-clock _ ormwot (Map. (hash-map) (hash-map) (hash-map)))) (defn ormwot-assoc [^Map ormwot k v] (vc/update-clock now ormwot (Map. (assoc (.-data ormwot) k v) (.-vclock ormwot) (assoc (.-birth-dots ormwot) k [node/*current-node* now])))) (defn ormwot-dissoc [^Map ormwot k] (vc/update-clock _ ormwot (Map. (dissoc (.-data ormwot) k) (.-vclock ormwot) (dissoc (.-birth-dots ormwot) k)))) (defn- elemental-data [^Map m] {:vector-clock (.-vclock m) :elements (into [] (for [datum (.-data m)] (let [dot (get (.-birth-dots m) (key datum))] {:data datum :author-node (first dot) :record-time (last dot)})))}) (extend-type Map proto/Vclocked (get-clock [this] (.-vclock this)) (with-clock [this new-clock] (Map. (.-data this) new-clock (.-birth-dots this))) proto/Convergent (synchronize [this ^Map other] (let [own-meta (-> this .-data meta) own-data (elemental-data this) other-data (elemental-data other) retain (filter (ic/common-elements own-data other-data) (:elements own-data)) completed-elements (->> (apply ic/retain-elements (ic/distinct-data own-data other-data)) (concat retain) (sort-by :record-time)) completed-data (into {} (map :data completed-elements)) completed-birth-dots (->> completed-elements (map (fn [{:keys [data author-node record-time]}] [(key data) [author-node record-time]])) (into {})) completed-vclock (ic/merged-clock completed-elements own-data other-data)] (vc/update-clock _ this (Map. (with-meta completed-data own-meta) completed-vclock completed-birth-dots))))) #?(:clj (defmethod print-method Map [^Map m ^Writer writer] (.write writer "#schism/map [") (.write writer (pr-str (.-data m))) (.write writer ", ") (.write writer (pr-str (.-vclock m))) (.write writer ", ") (.write writer (pr-str (.-birth-dots m))) (.write writer "]"))) (defn read-edn-map [read-object] (let [[data vclock birth-dots] read-object] (Map. data vclock birth-dots))) #?(:cljs (cljs.reader/register-tag-parser! 'schism/map read-edn-map)) (defn new-map ([] (Map. (hash-map) (hash-map) (hash-map))) ([& args] (vc/update-clock now nil (Map. (apply hash-map args) (hash-map) (apply hash-map (mapcat (fn [[k _]] [k [node/*current-node* now]]) (partition 2 args))))))) ================================================ FILE: src/schism/impl/types/nested_map.cljc ================================================ (ns schism.impl.types.nested-map "Definition and support for Schism's Deeply Convergent Map type, an ORMWOT implemented on top of Clojure's persistent maps and a Schism Vector Clock. Contrasted with schism.impl.types.map/Map, this incurs substantially greater computational costs for assoc type operations and cannot guarantee linear time results." (:require [schism.impl.types.nesting-util :as nu] [schism.impl.protocols :as proto] [schism.impl.vector-clock :as vc] [schism.impl.core :as ic] [schism.node :as node] [clojure.set :as set] [clojure.data :refer [diff]] #?(:cljs [cljs.reader :as reader])) #?(:cljs (:require-macros [schism.impl.vector-clock :as vc])) #?(:clj (:import (clojure.lang IPersistentCollection IPersistentMap IHashEq Associative ILookup Counted Seqable IMapIterable IKVReduce IFn IObj IMeta) (java.io Writer) (java.util Date Collection) (java.lang Object)))) (declare nested-map-conj nested-map-empty nested-map-assoc nested-map-dissoc) (def not-found-sym (gensym :not-found)) #?(:clj (deftype NestedMap [data vclock birth-dots] Counted (count [this] (.count ^Counted (.-data this))) IPersistentCollection (cons [this o] (nested-map-conj this o)) (empty [this] (nested-map-empty this)) (equiv [this other] (.equiv ^IPersistentCollection (.-data this) other)) IPersistentMap (assoc [this k v] (nested-map-assoc this k v)) (assocEx [this k v] (when (not= (get this k not-found-sym) not-found-sym) (throw (ex-info "Attempt to assocEx on map with key" {:map this :key k :value v}))) (nested-map-assoc this k v)) (without [this k] (nested-map-dissoc this k)) Object (equals [this o] (.equals (.-data this) o)) (hashCode [this] (.hashCode (.-data this))) (toString [this] (.toString (.-data this))) ILookup (valAt [this k] (.valAt ^ILookup (.-data this) k)) (valAt [this k not-found] (.valAt ^ILookup (.-data this) k not-found)) IMapIterable (keyIterator [this] (.keyIterator ^IMapIterable (.-data this))) (valIterator [this] (.valIterator ^IMapIterable (.-data this))) IKVReduce (kvreduce [this f init] (.kvreduce ^IKVReduce (.-data this) f init)) IHashEq (hasheq [this] (.hasheq ^IHashEq (.-data this))) Seqable (seq [this] (.seq ^Seqable (.-data this))) java.util.Map (clear [this] (.clear ^java.util.Map (.-data this))) (compute [this k f] (.compute ^java.util.Map (.-data this) k f)) (computeIfAbsent [this k f] (.computeIfAbsent ^java.util.Map (.-data this) k f)) (computeIfPresent [this k f] (.computeIfPresent ^java.util.Map (.-data this) k f)) (containsKey [this k] (.containsKey ^java.util.Map (.-data this) k)) (containsValue [this v] (.containsValue ^java.util.Map (.-data this) v)) (entrySet [this] (.entrySet ^java.util.Map (.-data this))) (get [this k] (.get ^java.util.Map (.-data this) k)) (getOrDefault [this k not-found] (.getOrDefault ^java.util.Map (.-data this) k not-found)) (isEmpty [this] (.isEmpty ^java.util.Map (.-data this))) (keySet [this] (.keySet ^java.util.Map (.-data this))) (merge [this k v f] (.merge ^java.util.Map (.-data this) k v f)) (put [this k v] (.put ^java.util.Map (.-data this) k v)) (putAll [this m] (.putAll ^java.util.Map (.-data this) m)) (putIfAbsent [this k v] (.putIfAbsent ^java.util.Map (.-data this) k v)) (remove [this k] (.remove ^java.util.Map (.-data this) k)) (remove [this k v] (.remove ^java.util.Map (.-data this) k v)) (replace [this k v] (.replace ^java.util.Map (.-data this) k v)) (replace [this k ov nv] (.replace ^java.util.Map (.-data this) k ov nv)) (replaceAll [this f] (.replaceAll ^java.util.Map (.-data this) f)) (size [this] (.size ^java.util.Map (.-data this))) (values [this] (.values ^java.util.Map (.-data this))) IFn (invoke [this k] (.invoke ^IFn (.-data this) k)) (invoke [this k not-found] (.invoke ^IFn (.-data this) k not-found)) IObj (withMeta [this meta] (NestedMap. (with-meta ^IObj (.-data this) meta) (.-vclock this) (.-birth-dots this))) IMeta (meta [this] (.meta ^IMeta (.-data this)))) :cljs (deftype NestedMap [data vclock birth-dots] ICounted (-count [this] (-count (.-data this))) IEmptyableCollection (-empty [this] (nested-map-empty this)) ICollection (-conj [this o] (nested-map-conj this o)) IAssociative (-contains-key? [this k] (-contains-key? (.-data this) k)) (-assoc [this k v] (nested-map-assoc this k v)) IFind (-find [this k] (-find (.-data this) k)) IMap (-dissoc [this k] (nested-map-dissoc this k)) IKVReduce (-kv-reduce [this f init] (-kv-reduce (.-data this) f init)) IEquiv (-equiv [this other] (-equiv (.-data this) other)) ILookup (-lookup [this o] (-lookup (.-data this) o)) (-lookup [this o not-found] (-lookup (.-data this) o not-found)) IPrintWithWriter (-pr-writer [o writer opts] (-write writer "#schism/nested-map [") (-write writer (pr-str (.-data o))) (-write writer ", ") (-write writer (pr-str (.-vclock o))) (-write writer ", ") (-write writer (pr-str (.-birth-dots o))) (-write writer "]")) IHash (-hash [this] (-hash (.-data this))) IFn (-invoke [this o] ((.-data this) o)) (-invoke [this o not-found] ((.-data this) o not-found)) ISeqable (-seq [this] (-seq (.-data this))) Object (toString [this] (.toString (.-data this))) IMeta (-meta [this] (-meta (.-data this))) IWithMeta (-with-meta [this meta] (NestedMap. (-with-meta (.-data this) meta) (.-vclock this) (.-birth-dots this))))) (defn nested-map-conj [^NestedMap nm pair] (vc/update-clock now nm (let [[updated updated-dots] (nu/nested-update (.-data nm) (.-birth-dots nm) #(conj % pair) node/*current-node* now)] (NestedMap. updated (.-vclock nm) updated-dots)))) (defn nested-map-empty [^NestedMap nm] (vc/update-clock _ nm (NestedMap. (hash-map) (hash-map) (hash-map)))) (defn nested-map-assoc [^NestedMap nm k v] (vc/update-clock now nm (let [[updated updated-dots] (nu/nested-update (.-data nm) (.-birth-dots nm) #(assoc % k v) node/*current-node* now)] (NestedMap. updated (.-vclock nm) updated-dots)))) (defn nested-map-dissoc [^NestedMap nm k] (vc/update-clock now nm (let [[updated updated-dots] (nu/nested-update (.-data nm) (.-birth-dots nm) #(dissoc % k) node/*current-node* now)] (NestedMap. updated (.-vclock nm) updated-dots)))) (defn- elemental-data [^NestedMap nm] (let [flat-data (nu/flat (.-data nm))] {:vector-clock (.-vclock nm) :elements (into [] (for [datum flat-data] (let [dot (get-in (.-birth-dots nm) (nu/access-path (key datum)))] {:data {:entry datum :insert-index (:i dot)} :author-node (:a dot) :record-time (:t dot)})))})) (extend-type NestedMap proto/Vclocked (get-clock [this] (.-vclock this)) (with-clock [this new-clock] (NestedMap. (.-data this) new-clock (.-birth-dots this))) proto/Convergent (synchronize [this ^NestedMap other] (let [own-meta (-> this .-data meta) own-data (elemental-data this) other-data (elemental-data other) retain (filter (ic/common-elements own-data other-data) (:elements own-data)) completed-elements (->> (apply ic/retain-elements (ic/distinct-data own-data other-data)) (concat retain) (sort-by :record-time) (map nu/finalize-projection-key)) completed-flat-data (map (comp :entry :data) completed-elements) completed-flat-birth-dots (map (fn [{:keys [author-node record-time] {:keys [insert-index entry] :as data} :data}] (let [dot {:a author-node :t record-time}] [(first entry) (if insert-index (assoc dot :i insert-index) dot)])) completed-elements) completed-vclock (ic/merged-clock completed-elements own-data other-data)] (vc/update-clock _ this (NestedMap. (with-meta (nu/project completed-flat-data) own-meta) completed-vclock (nu/project completed-flat-birth-dots)))))) #?(:clj (defmethod print-method NestedMap [^NestedMap nm ^Writer writer] (.write writer "#schism/nested-map [") (.write writer (pr-str (.-data nm))) (.write writer ", ") (.write writer (pr-str (.-vclock nm))) (.write writer ", ") (.write writer (pr-str (.-birth-dots nm))) (.write writer "]"))) (defn read-edn-map [read-object] (let [[data vclock birth-dots] read-object] (NestedMap. data vclock birth-dots))) #?(:cljs (cljs.reader/register-tag-parser! 'schism/map read-edn-map)) (defn new-map ([] (NestedMap. (hash-map) (hash-map) (hash-map))) ([& args] (vc/update-clock now nil (let [[updated updated-dots] (nu/nested-update {} {} (fn [_] (apply hash-map args)) node/*current-node* now)] (NestedMap. updated (hash-map) updated-dots))))) ================================================ FILE: src/schism/impl/types/nested_vector.cljc ================================================ (ns schism.impl.types.nested-vector "Definition and support for Schism's Deeply Convergent Vector type. The convergent vector is a timestamped log of entries with a vector clock & insertion index. Convergence places entries into the resultant vector in insertion order, with insertions occurring by replaying insertions operations in order. The vector clock conveys that an item has been removed from the vector on another node. This variant provides rich support for serialization and convergence of deeply nested structures, at the cost that all modification operations take linear time instead of constant or log time." (:require [schism.impl.types.nesting-util :as nu] [schism.impl.protocols :as proto] [schism.impl.vector-clock :as vc] [schism.impl.core :as ic] [schism.node :as node] [clojure.set :as set] [clojure.data :refer [diff]] #?(:cljs [cljs.reader :as reader])) #?(:cljs (:require-macros [schism.impl.vector-clock :as vc])) #?(:clj (:import (clojure.lang IPersistentCollection IPersistentStack IPersistentVector Reversible IReduce IKVReduce Indexed Associative Counted IHashEq Seqable IObj IMeta IFn ILookup) (java.io Writer) (java.util Date Collection) (java.lang Object Long)))) (declare nested-vector-conj nested-vector-pop nested-vector-empty nested-vector-assoc) #?(:clj (deftype NestedVector [data vclock insertions] Counted (count [this] (.count ^Counted (.-data this))) IPersistentCollection (cons [this o] (nested-vector-conj this o)) (empty [this] (nested-vector-empty this)) (equiv [this other] (.equiv ^IPersistentCollection (.-data this) other)) Object (equals [this o] (.equals (.-data this) o)) (hashCode [this] (.hashCode (.-data this))) (toString [this] (.toString (.-data this))) IHashEq (hasheq [this] (.hasheq ^IHashEq (.-data this))) Seqable (seq [this] (seq ^Seqable (.-data this))) java.util.List (add [this o] (.add ^java.util.List (.-data this) o)) (add [this index o] (.add ^java.util.List (.-data this) index o)) (addAll [this c] (.addAll ^java.util.List (.-data this) c)) (clear [this] (.clear ^java.util.List (.-data this))) (contains [this o] (.contains ^java.util.List (.-data this) o)) (containsAll [this c] (.containsAll ^java.util.List (.-data this) c)) (get [this i] (.get ^java.util.List (.-data this) i)) (indexOf [this o] (.indexOf ^java.util.List (.-data this) o)) (isEmpty [this] (.isEmpty ^java.util.List (.-data this))) (iterator [this] (.iterator ^java.util.List (.-data this))) (lastIndexOf [this o] (.lastIndexOf ^java.util.List (.-data this) o)) (listIterator [this] (.listIterator ^java.util.List (.-data this))) (^Object remove [this ^int i] (.remove ^java.util.List (.-data this) i)) (^boolean remove [this ^Object o] (.remove ^java.util.List (.-data this) o)) (removeAll [this c] (.removeAll ^java.util.List (.-data this) c)) (replaceAll [this op] (.replaceAll ^java.util.List (.-data this) op)) (retainAll [this c] (.retainAll ^java.util.List (.-data this) c)) (set [this i e] (.set ^java.util.List (.-data this) i e)) (size [this] (.size ^java.util.List (.-data this))) (sort [this c] (.sort ^java.util.List (.-data this) c)) (spliterator [this] (.spliterator ^java.util.List (.-data this))) (subList [this i j] (.subList ^java.util.List (.-data this) i j)) (toArray [this] (.toArray ^java.util.List (.-data this))) (^"[Ljava.lang.Object;" toArray [this ^"[Ljava.lang.Object;" a] (.toArray ^java.util.List (.-data this) a)) IObj (withMeta [this meta] (NestedVector. (with-meta ^IObj (.-data this) meta) (.-vclock this) (.-insertions this))) IMeta (meta [this] (.meta ^IMeta (.-data this))) IReduce (reduce [this f] (.reduce ^IReduce (.-data this) f)) IKVReduce (kvreduce [this f init] (.kvreduce ^IKVReduce (.-data this) f init)) IPersistentStack (peek [this] (.peek ^IPersistentStack (.-data this))) (pop [this] (nested-vector-pop this)) IPersistentVector (assocN [this i v] (nested-vector-assoc this i v)) ILookup (valAt [this k] (.valAt (.-data this) k)) (valAt [this k not-found] (.valAt (.-data this) k not-found)) Associative (containsKey [this k] (.containsKey ^Associative (.-data this) k)) (entryAt [this k] (.entryAt ^Associative (.-data this) k)) (assoc [this k v] (nested-vector-assoc this k v)) Indexed (nth [this i] (.indexed ^Indexed (.-data this) i)) (nth [this i not-found] (.indexed ^Indexed (.-data this) i not-found)) IFn (invoke [this k] (.invoke ^IFn (.-data this) k)) (invoke [this k not-found] (.invoke ^IFn (.-data this) k not-found))) :cljs (deftype NestedVector [data vclock insertions] ICounted (-count [this] (-count (.-data this))) IEmptyableCollection (-empty [this] (nested-vector-empty this)) ICollection (-conj [this o] (nested-vector-conj this o)) IEquiv (-equiv [this other] (-equiv (.-data this) other)) IPrintWithWriter (-pr-writer [o writer opts] (-write writer "#schism/nested-vector [") (-write writer (pr-str (.-data o))) (-write writer ", ") (-write writer (pr-str (.-vclock o))) (-write writer ", ") (-write writer (pr-str (.-insertions o))) (-write writer "]")) IHash (-hash [this] (-hash (.-data this))) ISeqable (-seq [this] (-seq (.-data this))) Object (toString [this] (.toString (.-data this))) IMeta (-meta [this] (-meta (.-data this))) IWithMeta (-with-meta [this meta] (NestedVector. (-with-meta (.-data this) meta) (.-vclock this) (.-insertions this))) IFn (-invoke [this k] ((.-data this) k)) (-invoke [this k not-found] ((.-data this) k not-found)) IIndexed (-nth [this n] (-nth (.-data this) n)) (-nth [this n not-found] (-nth (.-data this) n not-found)) ILookup (-lookup [this k] (-lookup (.-data this) k)) (-lookup [this k not-found] (-lookup (.-data this) k not-found)) IAssociative (-contains-key? [this k] (-contains-key? (.-data this) k)) (-assoc [this k v] (nested-vector-assoc this k v)) IFind (-find [this k] (-find (.-data this) k)) IStack (-peek [this] (-peek (.-data this))) (-pop [this] (nested-vector-pop this)) IVector (-assoc-n [this n v] (nested-vector-assoc this n v)) IReduce (-reduce [this f] (-reduce (.-data this) f)) (-reduce [this f start] (-reduce (.-data this) f start)) IKVReduce (-kv-reduce [this f init] (-kv-reduce (.-data this) f init)))) (defn nested-vector-conj [^NestedVector nvector o] (vc/update-clock now nvector (let [[updated updated-dots] (nu/nested-update (.-data nvector) (.-insertions nvector) #(conj % o) node/*current-node* now)] (NestedVector. updated (.-vclock nvector) updated-dots)))) (defn nested-vector-empty [^NestedVector nvector] (vc/update-clock _ nvector (NestedVector. (vector) (hash-map) (vector)))) (defn nested-vector-pop [^NestedVector nvector] (vc/update-clock _ nvector (NestedVector. (pop (.-data nvector)) (.-vclock nvector) (pop (.-insertions nvector))))) (defn nested-vector-assoc [^NestedVector nvector k v] (vc/update-clock now nvector (let [[updated updated-dots] (nu/nested-update (.-data nvector) (.-insertions nvector) #(assoc % k v) node/*current-node* now)] (NestedVector. updated (.-vclock nvector) updated-dots)))) (defn- elemental-data [^NestedVector v] (let [flat-data (nu/flat (.-data v))] {:vector-clock (.-vclock v) :elements (into [] (for [datum flat-data] (let [dot (get-in (.-insertions v) (nu/access-path (key datum)))] {:data {:entry datum :insert-index (:i dot)} :author-node (:a dot) :record-time (:t dot)})))})) (extend-type NestedVector proto/Vclocked (get-clock [this] (.-vclock this)) (with-clock [this new-clock] (NestedVector. (.-data this) new-clock (.-insertions this))) proto/Convergent (synchronize [this ^NestedVector other] (let [own-meta (-> this .-data meta) own-data (elemental-data this) other-data (elemental-data other) retain (filter (ic/common-elements own-data other-data) (:elements own-data)) completed-elements (->> (apply ic/retain-elements (ic/distinct-data own-data other-data)) (concat retain) (sort-by :record-time) (map nu/finalize-projection-key)) completed-flat-data (map (comp :entry :data) completed-elements) completed-flat-insertions (map (fn [{:keys [author-node record-time] {:keys [insert-index entry] :as data} :data}] (let [dot {:a author-node :t record-time}] [(first entry) (if insert-index (assoc dot :i insert-index) dot)])) completed-elements) completed-vclock (ic/merged-clock completed-elements own-data other-data)] (vc/update-clock _ this (NestedVector. (with-meta (nu/project completed-flat-data) own-meta) completed-vclock (nu/project completed-flat-insertions)))))) #?(:clj (defmethod print-method NestedVector [^NestedVector v ^Writer writer] (.write writer "#schism/nested-vector [") (.write writer (pr-str (.-data v))) (.write writer ", ") (.write writer (pr-str (.-vclock v))) (.write writer ", ") (.write writer (pr-str (.-insertions v))) (.write writer "]"))) (defn read-edn-vector [read-object] (let [[data vclock insertions] read-object] (NestedVector. data vclock insertions))) #?(:cljs (cljs.reader/register-tag-parser! 'schism/nested-vector read-edn-vector)) (defn new-vector ([] (NestedVector. (vector) (hash-map) (vector))) ([& args] (vc/update-clock now nil (let [[updated updated-dots] (nu/nested-update [] [] #(into % args) node/*current-node* now)] (NestedVector. updated (hash-map) updated-dots))))) ================================================ FILE: src/schism/impl/types/nesting_util.cljc ================================================ (ns schism.impl.types.nesting-util "Utility functions for supporting nested collections." (:require [clojure.data :refer [diff]] [schism.impl.core :as ic])) (defn compare-paths [[[a-type a-val :as first-a] & rest-a :as a] [[b-type b-val :as first-b] & rest-b :as b]] (cond (and (nil? a) (nil? b)) 0 (nil? a) -1 (nil? b) 1 (= first-a first-b) (recur rest-a rest-b) (and (= a-type 's) (= b-type 'a)) -1 (and (= b-type 's) (= a-type 'a)) 1 (and (= a-type 's) (= b-type 's)) (compare a-val b-val) (or rest-a rest-b) (recur rest-a rest-b) :else (compare (hash a-val) (hash b-val)))) (defn flat "Flattens a structure of associatives and sequentials to an associative of paths to leaf values. Each step in a path will retain both the type and the edge value, maps will be flattened to associatives. Vectors, lists, and other seqs will be flattened to sequentials." [c] (if (or (not (coll? c)) (empty? c)) c (let [marker (if (map? c) 'a 's) m (if (map? c) c (map-indexed (fn [i e] [i e]) c))] (into {} (mapcat (fn [[k v]] (let [child-flat (flat v)] (if (or (not (coll? child-flat)) (empty? child-flat)) [[[[marker k]] child-flat]] (for [[path element] child-flat] [(apply vector [marker k] path) element]))))) m)))) (defn access-path "`flat` returns a reconstitution path of both type and key. This is useful for reprojecting the flat version up into a nested structure, but is not amenable to `get-in`, `assoc-in`, et al. `access-path` will convert a reconstitution path into a more conventional access path." [reconstitution-path] (map second reconstitution-path)) (defn- assoc* [m [type key] v] (let [m (if m m (cond (= type 'a) {} (= type 's) [])) assoc-fn (if (vector? m) ic/assoc-n-with-tail-support assoc)] (assoc-fn m key v))) (defn- assoc-in* [m [[type key :as kspec] & kspecs] v] (if kspecs (assoc* m kspec (assoc-in* (get m key) kspecs v)) (assoc* m kspec v))) (defn project "Constitutes a structure as produced by `flat` up into a nested collection of maps and vectors. As all sequential items are coerced to vectors, this is not reflexive of `flat`." ([vals] (project vals nil)) ([vals basis] (reduce (fn [m [k v]] (assoc-in* m k v)) basis vals))) (defn clean* "Remove the key at k and any empty parents above it." [m [k & ks]] (if ks (let [cleaned (clean* (get m k) ks)] (if (empty? cleaned) (cond (vector? m) (pop m) (map? m) (dissoc m k)) (assoc m k cleaned))) (cond (vector? m) (pop m) (map? m) (dissoc m k)))) (defn nested-update "Does all of the book-keeping for nested map/vector combo data types. `original` is the original data structure, `provenance` is the original structure's provenance data, `update` is a update function to progress original. Returns positionally: the updated `original`, and, the updated `provenance`." [original provenance update author timestamp] (let [updated (update original) original-vals-flat (flat original) update-vals-flat (flat updated) [deletions additions common] (diff original-vals-flat update-vals-flat) provenance (reduce (fn [m [k v]] (if (contains? additions k) m (clean* m (access-path k)))) provenance deletions) addition-dots (for [[k v] (sort-by first compare-paths additions)] (let [to-vector? (= 's (first (last k))) distinct? (not (contains? deletions k)) basis {:a author :t timestamp}] [k (if to-vector? (merge basis {:i (if distinct? -1 (last (last k)))}) basis)]))] [updated (project addition-dots provenance)])) (defn finalize-projection-key [m] (let [{:keys [entry insert-index]} (:data m)] (if insert-index (assoc-in m [:data :entry] [(conj (pop (key entry)) ['s insert-index]) (val entry)]) m))) ================================================ FILE: src/schism/impl/types/set.cljc ================================================ (ns schism.impl.types.set "Definition and support for Schism's Convergent Set type, an ORSWOT implemented on top of Clojure's Persistent Set, Persistent Map and Schism's Vector Clock." (:require [schism.impl.protocols :as proto] [schism.impl.vector-clock :as vc] [schism.impl.core :as ic] [schism.node :as node] [clojure.set :as set] #?(:cljs [cljs.reader :as reader])) #?(:cljs (:require-macros [schism.impl.vector-clock :as vc])) #?(:clj (:import (clojure.lang IPersistentCollection IPersistentSet IHashEq Counted Seqable RT IFn IObj IMeta) (java.io Writer) (java.util Date Collection) (java.lang Object)))) ;; A CLJS and CLJ implementation of ORSWOT (Observed-Removed Set without Tombstones) ;; Each set maintains its own vector clock, and birth dots for each ;; member. A birth dot consists of the node adding the member, and the ;; date at which it was added. The vector clock determines if an ;; element has been removed: a compared set absent an element but with ;; a newer vector clock than the own birthdot on element indicates ;; that it was removed; a compared set absent an element with an older ;; vector clock indicates that the birthdot was never seen and should ;; be in the merged set. ;; Dot membership is always exactly the cardinality of the Set. Clock ;; membership is at most the cardinality of the set, but can correctly ;; synchronize with one more member than the total number of nodes in ;; the birth-dots; if a Vclock indicates an element was removed, the ;; node converging its changes can claim responsibility for removing ;; the element in the merged set. ;; TODO: ;; With vector clocks AND birthdots it is possible to disambiguate the ;; removal of an element from the post-replication addition of that ;; element. When merging other, examine it's vector clock for each ;; entry that is uniquely in own. If that entry's authoring node is ;; present in the vector clock, and the timestamp in other's vector ;; clock is less than the birth dot of own's entry, then retain the ;; entry, as other never saw it and could not have removed it. (declare orswot-conj orswot-empty orswot-disj) #?(:clj (deftype Set [data vclock birth-dots] Counted (count [this] (.count ^Counted (.-data this))) IPersistentCollection (cons [this o] (orswot-conj this o)) (empty [this] (orswot-empty this)) (equiv [this other] (.equiv ^IPersistentCollection (.-data this) other)) IPersistentSet (disjoin [this o] (orswot-disj this o)) (contains [this o] (.contains ^IPersistentSet (.-data this) o)) (get [this o] (.get ^IPersistentSet (.-data this) o)) Object (equals [this o] (.equals (.-data this) o)) (hashCode [this] (.hashCode (.-data this))) (toString [this] (.toString data)) IHashEq (hasheq [this] (.hasheq ^IHashEq (.-data this))) Seqable (seq [this] (.seq ^Seqable (.-data this))) java.util.Set (toArray [this] (.toArray (.-data this))) (^"[Ljava.lang.Object;" toArray [this ^"[Ljava.lang.Object;" a] (.toArray ^java.util.Set (.-data this) a)) (add [this o] (throw (UnsupportedOperationException.))) (remove [this o] (throw (UnsupportedOperationException.))) (addAll [this c] (throw (UnsupportedOperationException.))) (clear [this] (throw (UnsupportedOperationException.))) (retainAll [this c] (throw (UnsupportedOperationException.))) (removeAll [this c] (throw (UnsupportedOperationException.))) (containsAll [this c] (.containsAll ^java.util.Set (.-data this) c)) (size [this] (.size ^java.util.Set (.-data this))) (isEmpty [this] (.isEmpty ^java.util.Set (.-data this))) (iterator [this] (.iterator ^java.util.Set (.-data this))) IFn (invoke [this arg1] (.invoke ^IFn (.-data this) arg1)) IObj (withMeta [this meta] (Set. (.withMeta ^IObj (.-data this) meta) (.-vclock this) (.-birth-dots this))) IMeta (meta [this] (.meta ^IMeta (.-data this)))) :cljs (deftype Set [data vclock birth-dots] ICounted (-count [this] (-count (.-data this))) IEmptyableCollection (-empty [this] (orswot-empty this)) ICollection (-conj [this o] (orswot-conj this o)) ISet (-disjoin [this o] (orswot-disj this o)) IEquiv (-equiv [this other] (-equiv (.-data this) other)) ILookup (-lookup [this o] (-lookup (.-data this) o)) (-lookup [this o not-found] (-lookup (.-data this) o not-found)) IPrintWithWriter (-pr-writer [o writer opts] (-write writer "#schism/set [") (-write writer (pr-str (.-data o))) (-write writer ", ") (-write writer (pr-str (.-vclock o))) (-write writer ", ") (-write writer (pr-str (.-birth-dots o))) (-write writer "]")) IHash (-hash [this] (-hash (.-data this))) IFn (-invoke [this o] (-invoke (.-data this) o)) ISeqable (-seq [this] (-seq (.-data this))) Object (toString [this] (.toString (.-data this))) IMeta (-meta [this] (-meta (.-data this))) IWithMeta (-with-meta [this meta] (Set. (-with-meta (.-data this) meta) (.-vclock this) (.-birth-dots this))))) (defn orswot-conj [^Set orswot o] (vc/update-clock now orswot (Set. (conj (.-data orswot) o) (.-vclock orswot) (assoc (.-birth-dots orswot) o [node/*current-node* now])))) (defn orswot-empty [^Set orswot] (vc/update-clock _ orswot (Set. (hash-set) (hash-map) (hash-map)))) (defn orswot-disj [^Set orswot o] (vc/update-clock _ orswot (Set. (disj (.-data orswot) o) (.-vclock orswot) (dissoc (.-birth-dots orswot) o)))) (defn- elemental-data [^Set s] {:vector-clock (.-vclock s) :elements (into [] (for [datum (.-data s)] (let [dot (get (.-birth-dots s) datum)] {:data datum :author-node (first dot) :record-time (last dot)})))}) (extend-type Set proto/Vclocked (get-clock [this] (.-vclock this)) (with-clock [this new-clock] (Set. (.-data this) new-clock (.-birth-dots this))) proto/Convergent (synchronize [this ^Set other] (let [own-meta (-> this .-data meta) own-data (elemental-data this) other-data (elemental-data other) retain (filter (ic/common-elements own-data other-data) (:elements own-data)) completed-elements (->> (apply ic/retain-elements (ic/distinct-data own-data other-data)) (concat retain) (sort-by :record-time)) completed-data (into #{} (map :data completed-elements)) completed-birth-dots (->> completed-elements (map (fn [{:keys [data author-node record-time]}] [data [author-node record-time]])) (into {})) completed-vclock (ic/merged-clock completed-elements own-data other-data)] (vc/update-clock _ this (Set. (with-meta completed-data own-meta) completed-vclock completed-birth-dots))))) #?(:clj (defmethod print-method Set [^Set s ^Writer writer] (.write writer "#schism/set [") (.write writer (pr-str (.-data s))) (.write writer ", ") (.write writer (pr-str (.-vclock s))) (.write writer ", ") (.write writer (pr-str (.-birth-dots s))) (.write writer "]"))) (defn read-edn-set [read-object] (let [[data vclock birth-dots] read-object] (Set. data vclock birth-dots))) #?(:cljs (cljs.reader/register-tag-parser! 'schism/set read-edn-set)) (defn new-set ([] (Set. (hash-set) (hash-map) (hash-map))) ([& args] (vc/update-clock now nil (Set. (apply hash-set args) (hash-map) (apply hash-map (mapcat (fn [o] [o [node/*current-node* now]]) args)))))) ================================================ FILE: src/schism/impl/types/vector.cljc ================================================ (ns schism.impl.types.vector "Definition and support for Schism's Convergent vector type. The convergent vector is a timestamped log of entries with a vector clock & insertion index. Convergence places entries into the resultant vector in insertion order, with insertions occurring by replaying insertions operations in order. The vector clock conveys that an item has been removed from the vector on another node." (:require [schism.impl.protocols :as proto] [schism.impl.vector-clock :as vc] [schism.impl.core :as ic] [schism.node :as node] [clojure.set :as set] #?(:cljs [cljs.reader :as reader])) #?(:cljs (:require-macros [schism.impl.vector-clock :as vc])) #?(:clj (:import (clojure.lang IPersistentCollection IPersistentStack IPersistentVector Reversible IReduce IKVReduce Indexed Associative Counted IHashEq Seqable IObj IMeta IFn ILookup) (java.io Writer) (java.util Date Collection) (java.lang Object Long)))) ;; A CLJ & CLJS implementation of a convergent vector ;; Each vector maintains its own vector clock, and insertion times and ;; positions for each element of the vector. Vector entries and ;; insertions are correlated positionally (as the vector may contain ;; the same item multiple times.) Insertion times and indices dictate ;; ordering; elements inserted at the tail of the vector are recorded ;; as being inserted with index -1. The vector clock determines if an ;; entry has been removed. (declare cvector-conj cvector-pop cvector-empty cvector-assoc) #?(:clj (deftype ConvergentVector [data vclock insertions] Counted (count [this] (.count ^Counted (.-data this))) IPersistentCollection (cons [this o] (cvector-conj this o)) (empty [this] (cvector-empty this)) (equiv [this other] (.equiv ^IPersistentCollection (.-data this) other)) Object (equals [this o] (.equals (.-data this) o)) (hashCode [this] (.hashCode (.-data this))) (toString [this] (.toString (.-data this))) IHashEq (hasheq [this] (.hasheq ^IHashEq (.-data this))) Seqable (seq [this] (seq ^Seqable (.-data this))) java.util.List (add [this o] (.add ^java.util.List (.-data this) o)) (add [this index o] (.add ^java.util.List (.-data this) index o)) (addAll [this c] (.addAll ^java.util.List (.-data this) c)) (clear [this] (.clear ^java.util.List (.-data this))) (contains [this o] (.contains ^java.util.List (.-data this) o)) (containsAll [this c] (.containsAll ^java.util.List (.-data this) c)) (get [this i] (.get ^java.util.List (.-data this) i)) (indexOf [this o] (.indexOf ^java.util.List (.-data this) o)) (isEmpty [this] (.isEmpty ^java.util.List (.-data this))) (iterator [this] (.iterator ^java.util.List (.-data this))) (lastIndexOf [this o] (.lastIndexOf ^java.util.List (.-data this) o)) (listIterator [this] (.listIterator ^java.util.List (.-data this))) (^Object remove [this ^int i] (.remove ^java.util.List (.-data this) i)) (^boolean remove [this ^Object o] (.remove ^java.util.List (.-data this) o)) (removeAll [this c] (.removeAll ^java.util.List (.-data this) c)) (replaceAll [this op] (.replaceAll ^java.util.List (.-data this) op)) (retainAll [this c] (.retainAll ^java.util.List (.-data this) c)) (set [this i e] (.set ^java.util.List (.-data this) i e)) (size [this] (.size ^java.util.List (.-data this))) (sort [this c] (.sort ^java.util.List (.-data this) c)) (spliterator [this] (.spliterator ^java.util.List (.-data this))) (subList [this i j] (.subList ^java.util.List (.-data this) i j)) (toArray [this] (.toArray ^java.util.List (.-data this))) (^"[Ljava.lang.Object;" toArray [this ^"[Ljava.lang.Object;" a] (.toArray ^java.util.List (.-data this) a)) IObj (withMeta [this meta] (ConvergentVector. (with-meta ^IObj (.-data this) meta) (.-vclock this) (.-insertions this))) IMeta (meta [this] (.meta ^IMeta (.-data this))) IReduce (reduce [this f] (.reduce ^IReduce (.-data this) f)) IKVReduce (kvreduce [this f init] (.kvreduce ^IKVReduce (.-data this) f init)) IPersistentStack (peek [this] (.peek ^IPersistentStack (.-data this))) (pop [this] (cvector-pop this)) IPersistentVector (assocN [this i v] (cvector-assoc this i v)) ILookup (valAt [this k] (.valAt (.-data this) k)) (valAt [this k not-found] (.valAt (.-data this) k not-found)) Associative (containsKey [this k] (.containsKey ^Associative (.-data this) k)) (entryAt [this k] (.entryAt ^Associative (.-data this) k)) (assoc [this k v] (cvector-assoc this k v)) Indexed (nth [this i] (.indexed ^Indexed (.-data this) i)) (nth [this i not-found] (.indexed ^Indexed (.-data this) i not-found)) IFn (invoke [this k] (.invoke ^IFn (.-data this) k)) (invoke [this k not-found] (.invoke ^IFn (.-data this) k not-found))) :cljs (deftype ConvergentVector [data vclock insertions] ICounted (-count [this] (-count (.-data this))) IEmptyableCollection (-empty [this] (cvector-empty this)) ICollection (-conj [this o] (cvector-conj this o)) IEquiv (-equiv [this other] (-equiv (.-data this) other)) IPrintWithWriter (-pr-writer [o writer opts] (-write writer "#schism/vector [") (-write writer (pr-str (.-data o))) (-write writer ", ") (-write writer (pr-str (.-vclock o))) (-write writer ", ") (-write writer (pr-str (.-insertions o))) (-write writer "]")) IHash (-hash [this] (-hash (.-data this))) ISeqable (-seq [this] (-seq (.-data this))) Object (toString [this] (.toString (.-data this))) IMeta (-meta [this] (-meta (.-data this))) IWithMeta (-with-meta [this meta] (ConvergentVector. (-with-meta (.-data this) meta) (.-vclock this) (.-insertions this))) IFn (-invoke [this k] (-invoke (.-data this) k)) (-invoke [this k not-found] (-invoke (.-data this) k not-found)) IIndexed (-nth [this n] (-nth (.-data this) n)) (-nth [this n not-found] (-nth (.-data this) n not-found)) ILookup (-lookup [this k] (-lookup (.-data this) k)) (-lookup [this k not-found] (-lookup (.-data this) k not-found)) IAssociative (-contains-key? [this k] (-contains-key? (.-data this) k)) (-assoc [this k v] (cvector-assoc this k v)) IFind (-find [this k] (-find (.-data this) k)) IStack (-peek [this] (-peek (.-data this))) (-pop [this] (cvector-pop this)) IVector (-assoc-n [this n v] (cvector-assoc this n v)) IReduce (-reduce [this f] (-reduce (.-data this) f)) (-reduce [this f start] (-reduce (.-data this) f start)) IKVReduce (-kv-reduce [this f init] (-kv-reduce (.-data this) f init)))) (defn cvector-conj [^ConvergentVector cvector o] (vc/update-clock now cvector (ConvergentVector. (conj (.-data cvector) o) (.-vclock cvector) (conj (.-insertions cvector) [node/*current-node* -1 now])))) (defn cvector-empty [^ConvergentVector cvector] (vc/update-clock _ cvector (ConvergentVector. (vector) (hash-map) (vector)))) (defn cvector-pop [^ConvergentVector cvector] (vc/update-clock _ cvector (ConvergentVector. (pop (.-data cvector)) (.-vclock cvector) (pop (.-insertions cvector))))) (defn cvector-assoc [^ConvergentVector cvector k v] (vc/update-clock now cvector (ConvergentVector. (assoc (.-data cvector) k v) (.-vclock cvector) (assoc (.-insertions cvector) k [node/*current-node* k now])))) (defn- elemental-data [^ConvergentVector v] {:vector-clock (.-vclock v) :elements (into [] (for [[element [author-node insert-index record-time]] (map vector (.-data v) (.-insertions v))] {:data {:element element :insert-index insert-index} :author-node author-node :record-time record-time}))}) (extend-type ConvergentVector proto/Vclocked (get-clock [this] (.-vclock this)) (with-clock [this new-clock] (ConvergentVector. (.-data this) new-clock (.-insertions this))) proto/Convergent (synchronize [this ^ConvergentVector other] (let [own-meta (-> this .-data meta) own-data (elemental-data this) other-data (elemental-data other) retain (filter (ic/common-elements own-data other-data) (:elements own-data)) completed-elements (->> retain (concat (apply ic/retain-elements (ic/distinct-data own-data other-data))) (sort-by (fn [{:keys [author-node data record-time]}] (let [{:keys [insert-index]} data] [(if (= -1 insert-index) ic/tail-insertion-sort-value insert-index) record-time])))) ;; Given the potential for insertion at arbitrary indexes, ;; trying to find a common contiguous chunk is less fruitful ;; than taking our [element, node, index, timestamp] tuples ;; and treating them as discrete insertion ;; instructions. Removal is still indicated by vector clock, ;; AND it is reasonable to expect much of the vector to be ;; shared, so it's important to get to the right set of such ;; instructions. As time and index dictate the overall state ;; of convergence, it is not important to preserve ordering ;; through convergence, which affords using set logic to ;; find the common insertions. completed-data (reduce (fn [m element] (let [{:keys [element insert-index]} (:data element)] (ic/assoc-n-with-tail-support m insert-index element))) [] completed-elements) completed-insertions (reduce (fn [m {:keys [author-node record-time] {:keys [insert-index]} :data}] (ic/assoc-n-with-tail-support m insert-index [author-node insert-index record-time])) [] completed-elements) completed-vclock (ic/merged-clock completed-elements own-data other-data)] (vc/update-clock _ this (ConvergentVector. (with-meta completed-data own-meta) completed-vclock completed-insertions))))) #?(:clj (defmethod print-method ConvergentVector [^ConvergentVector v ^Writer writer] (.write writer "#schism/vector [") (.write writer (pr-str (.-data v))) (.write writer ", ") (.write writer (pr-str (.-vclock v))) (.write writer ", ") (.write writer (pr-str (.-insertions v))) (.write writer "]"))) (defn read-edn-vector [read-object] (let [[data vclock insertions] read-object] (ConvergentVector. data vclock insertions))) #?(:cljs (cljs.reader/register-tag-parser! 'schism/vector read-edn-vector)) (defn new-vector ([] (ConvergentVector. (vector) (hash-map) (vector))) ([& args] (vc/update-clock now nil (ConvergentVector. (apply vector args) (hash-map) (apply vector (for [i (range (count args))] [node/*current-node* i now])))))) ================================================ FILE: src/schism/impl/vector_clock.cljc ================================================ (ns schism.impl.vector-clock "Utility functions for working with the vector clock of a value that participates in the Vclocked protocol." (:require [schism.impl.protocols :as sp] [schism.impl.core :as ic] [schism.node :as node]) #?(:clj (:import (java.util Date)))) (defmacro update-clock "Binds the current time to `binding`, executes body, then updates the body's return value which participates in schism.impl.protocols/Vclocked, so that the vector clock contains the same time bound to `binding` for the current node." [binding clocked & body] `(let [now# (ic/to-date (max (ic/to-millis (ic/now)) (inc (apply max 0 (map ic/to-millis (vals (sp/get-clock ~clocked))))))) ~binding now# ret# (do ~@body) ret-clock# (sp/get-clock ret#)] (sp/with-clock ret# (assoc ret-clock# node/*current-node* now#)))) ================================================ FILE: src/schism/node.cljc ================================================ (ns schism.node #?(:clj (:import (java.util UUID)))) (def ^:dynamic *current-node* nil) (defn initialize-node! "Initialize `schism.node/*current-node*` to the passed in value, or a new random UUID. While any serializable value suffices as the current node, it should be unique within the cluster; a repeated node value may result in incorrect behaviors during convergence." ([] (initialize-node! #?(:clj (UUID/randomUUID) :cljs (random-uuid)))) ([id] #?(:clj (alter-var-root #'*current-node* (constantly id)) :cljs (set! *current-node* id)))) (defmacro with-node "Override the value of `schism.node/*current-node*` for the scope of `body`." [id & body] `(binding [*current-node* ~id] ~@body)) ================================================ FILE: test/schism/core_test.cljc ================================================ (ns schism.core-test (:require [schism.core :as schism :include-macros true] #?(:clj [clojure.test :refer [deftest testing is]] :cljs [cljs.test :refer [deftest testing is]]) [clojure.test.check.generators :as gen] [clojure.test.check.properties :as prop :include-macros true] [clojure.test.check.clojure-test #?(:clj :refer :cljs :refer-macros) [defspec]])) #?(:cljs (deftest collections-with-NaN (is (not= (hash-map js/NaN []) (hash-map js/NaN [])) "Because NaN cannot be compared for equality, two maps with NaN cannot be equal.") (is (not= (list js/NaN) (list js/NaN)) "Because NaN cannot be compared for equality, two lists with NaN cannot be equal.") (is (not= (hash-set js/NaN) (hash-set js/NaN)) "Because NaN cannot be compared for equality, two hash-sets with NaN cannot be equal.") (is (not= (vector js/NaN) (vector js/NaN)) "Because NaN cannot be compared for equality, two vectors with NaN cannot be equal."))) (def collection-any "CLJS any will sometimes return NaN, but property tests using equality of collections cannot pass with NaN as an element (see above), so explicitly filter out NaN" #?(:cljs (gen/such-that #(not (js/Number.isNaN %)) gen/any) :clj gen/any)) (defspec convergent-set-is-equivalent-to-hash-set 50 (prop/for-all [v (gen/vector collection-any)] (= (apply schism/convergent-set v) (apply hash-set v)))) (defspec convergent-map-is-equivalent-to-hash-map 50 (prop/for-all [entries (gen/fmap (comp (partial mapcat identity) seq) (gen/map collection-any collection-any))] (= (apply schism/convergent-map entries) (apply hash-map entries)))) (defspec nested-map-is-equivalent-to-hash-map 50 (prop/for-all [entries (gen/fmap (comp (partial mapcat identity) seq) (gen/map collection-any collection-any))] (= (apply schism/nested-map entries) (apply hash-map entries)))) (defspec convergent-vector-is-equivalent-to-vector 50 (prop/for-all [v (gen/vector collection-any)] (= (apply schism/convergent-vector v) (apply vector v)))) (defspec nested-vector-is-equivalent-to-vector 50 (prop/for-all [v (gen/vector collection-any)] (= (apply schism/nested-vector v) (apply vector v)))) (defspec convergent-list-is-equivalent-to-list 50 (prop/for-all [v (gen/list collection-any)] (= (apply schism/convergent-list v) (apply list v)))) (defspec conj-equivalence-for-sets 50 (prop/for-all [v (gen/vector collection-any) e collection-any] (= (conj (apply schism/convergent-set v) e) (conj (apply hash-set v) e)))) (defspec conj-equivalence-for-maps 30 (prop/for-all [entries (gen/fmap (comp (partial mapcat identity) seq) (gen/map collection-any collection-any)) e (gen/tuple collection-any collection-any)] (= (conj (apply schism/convergent-map entries) e) (conj (apply hash-map entries) e)))) (defspec conj-equivalence-for-nested-maps 30 (prop/for-all [entries (gen/fmap (comp (partial mapcat identity) seq) (gen/map collection-any collection-any)) e (gen/tuple collection-any collection-any)] (= (conj (apply schism/nested-map entries) e) (conj (apply hash-map entries) e)))) (defspec conj-equivalence-for-vectors 30 (prop/for-all [v (gen/vector collection-any) e collection-any] (= (conj (apply schism/convergent-vector v) e) (conj (apply vector v) e)))) (defspec conj-equivalence-for-nested-vectors 30 (prop/for-all [v (gen/vector collection-any) e collection-any] (= (conj (apply schism/nested-vector v) e) (conj (apply vector v) e)))) (defspec conj-equivalence-for-lists 30 (prop/for-all [v (gen/vector collection-any) e collection-any] (= (conj (apply schism/convergent-list v) e) (conj (apply list v) e)))) (defspec rest-equivalence-for-lists 30 (prop/for-all [v (gen/vector collection-any)] (= (rest (apply schism/convergent-list v)) (rest (apply list v))))) (defspec assoc-equivalence-for-maps 30 (prop/for-all [entries (gen/fmap (comp (partial mapcat identity) seq) (gen/map collection-any collection-any)) k collection-any v collection-any] (= (assoc (apply schism/convergent-map entries) k v) (assoc (apply hash-map entries) k v)))) (defspec assoc-equivalence-for-nested-maps 30 (prop/for-all [entries (gen/fmap (comp (partial mapcat identity) seq) (gen/map collection-any collection-any)) k collection-any v collection-any] (= (assoc (apply schism/nested-map entries) k v) (assoc (apply hash-map entries) k v)))) (defspec assoc-equivalence-for-vectors 30 (prop/for-all [v (gen/such-that #(< 0 (count %)) (gen/vector collection-any)) e collection-any] (gen/let [index (gen/choose 0 (dec (count v)))] (= (assoc (apply schism/convergent-vector v) index e) (assoc (apply vector v) index e))))) (defspec assoc-equivalence-for-nested-vectors 30 (prop/for-all [v (gen/such-that #(< 0 (count %)) (gen/vector collection-any)) e collection-any] (gen/let [index (gen/choose 0 (dec (count v)))] (= (assoc (apply schism/nested-vector v) index e) (assoc (apply vector v) index e))))) (defspec pop-equivalence-for-vectors 30 (prop/for-all [v (gen/such-that #(< 0 (count %)) (gen/vector collection-any))] (= (pop (apply schism/convergent-vector v)) (pop (apply vector v))))) (defspec pop-equivalence-for-nested-vectors 30 (prop/for-all [v (gen/such-that #(< 0 (count %)) (gen/vector collection-any))] (= (pop (apply schism/nested-vector v)) (pop (apply vector v))))) (defspec dissoc-present-key-for-maps 30 (prop/for-all [entries (gen/fmap (comp (partial mapcat identity) seq) (gen/map collection-any collection-any {:min-elements 1}))] (gen/let [key (gen/elements (keys (apply hash-map entries)))] (= (dissoc (apply schism/convergent-map entries) key) (dissoc (apply hash-map entries) key))))) (defspec dissoc-present-key-for-nested-maps 30 (prop/for-all [entries (gen/fmap (comp (partial mapcat identity) seq) (gen/map collection-any collection-any {:min-elements 1}))] (gen/let [key (gen/elements (keys (apply hash-map entries)))] (= (dissoc (apply schism/nested-map entries) key) (dissoc (apply hash-map entries) key))))) (defspec dissoc-random-value-for-maps 30 (prop/for-all [entries (gen/fmap (comp (partial mapcat identity) seq) (gen/map collection-any collection-any {:min-elements 1})) key gen/any] (= (dissoc (apply schism/convergent-map entries) key) (dissoc (apply hash-map entries) key)))) (defspec dissoc-random-value-for-nested-maps 30 (prop/for-all [entries (gen/fmap (comp (partial mapcat identity) seq) (gen/map collection-any collection-any {:min-elements 1})) key gen/any] (= (dissoc (apply schism/nested-map entries) key) (dissoc (apply hash-map entries) key)))) (defspec disj-included-element-for-sets 5 (prop/for-all [s (gen/set collection-any {:min-elements 1})] (gen/let [e (gen/elements (apply hash-set s))] (= (disj (apply schism/convergent-set s) e) (disj (apply hash-set s) e))))) (defspec disj-random-element-for-sets 5 (prop/for-all [s (gen/set collection-any {:min-elements 1}) e collection-any] (= (disj (apply schism/convergent-set s) e) (disj (apply hash-set s) e)))) (defspec converge-after-ops-equivalent-for-sets 50 (prop/for-all [s (gen/set collection-any {:min-elements 3}) ops (gen/let [operants (gen/list collection-any) operands (gen/vector (gen/elements [conj disj]) (count operants))] (map vector operands operants))] (let [basis-cset (schism/with-node :start (apply schism/convergent-set s)) vanilla-result (reduce (fn [memo [f operant]] (f memo operant)) s ops) schism-result (schism/with-node :end (reduce (fn [memo [f operant]] (f memo operant)) basis-cset ops)) converge-result (schism/with-node :start (schism/converge basis-cset schism-result))] (= vanilla-result schism-result converge-result)))) ================================================ FILE: test/schism/impl/types/list_test.cljc ================================================ (ns schism.impl.types.list-test (:require #?(:clj [clojure.test :refer [deftest testing is]] :cljs [cljs.test :refer [deftest testing is]]) #?(:cljs [cljs.reader :as reader]) [schism.impl.types.list :as slist] [schism.node :as node] [schism.impl.protocols :as proto]) #?(:clj (:import schism.impl.types.list.ConvergentList))) (deftest basic-IPC-ops (testing "Equiv for lists" (is (= (slist/new-list) '())) (is (= (slist/new-list :a true) '(:a true)))) (testing "Empty for lists" (is (= (empty (slist/new-list :a true)) (empty '(:a true))))) (testing "Count for lists" (is (= (count (slist/new-list :a true)) (count '(:a true))))) (testing "Conj for lists" (is (= (conj (slist/new-list) [:a true]) (conj '() [:a true]))) (is (= (conj (slist/new-list :a true) [:a false]) (conj '(:a true) [:a false]))))) (deftest basic-seq-ops (testing "conj" (is (= (conj (slist/new-list :a true) :a) (conj '(:a true) :a)))) (testing "rest" (is (= (rest (slist/new-list :a true)) (rest '(:a true)))))) (deftest converge-test (testing "Converge after concurrent additions on another node." (node/initialize-node! :converge-test-origin) ;; Can only rely on millisecond time scales, so sleep 1 second ;; between ops so that there's some non-zero passage of time (let [transfer (-> (slist/new-list true :a) (conj [:b 3])) other (node/with-node :converge-test-other-node (conj transfer [:c :d])) result (proto/synchronize transfer other)] (is (= result '([:c :d] [:b 3] true :a))))) (testing "Rest on another node mirrored locally after converge." (node/initialize-node! :converge-test-origin) (let [transfer (slist/new-list :a true :b 3 :c :d) other (node/with-node :converge-test-other-node (rest transfer)) result (proto/synchronize transfer other)] (is (= other '(true :b 3 :c :d))) (is (= result '(true :b 3 :c :d)))))) (deftest seqable-test (testing "Can turn a CLIST into a seq" (is (= (seq (slist/new-list :a true :b 3 :c :d)) (seq (list :a true :b 3 :c :d)))))) (deftest string-test (testing "Prints to console readably, even though edn is verbose" (is (= (str (slist/new-list :a true :b 3 :c :d)) (str (list :a true :b 3 :c :d)))))) (deftest serialization-test (testing "Round trip serialization generates the same structure." (let [^ConvergentList origin (-> (slist/new-list :a true :b 3 :c :d) (conj [:d :quux]) rest) ^ConvergentList round-tripped (-> origin pr-str #?(:clj read-string :cljs reader/read-string))] (is (= (.-data origin) (.-data round-tripped))) (is (= (.-vclock origin) (.-vclock round-tripped))) (is (= (.-insertions origin) (.-insertions round-tripped)))))) (deftest hashing-test (testing "Hashes to the same value as an equivalent list" (is (= (hash (into (slist/new-list) [[:a true] [:b 3] [:c :d]])) (hash (into '() [[:a true] [:b 3] [:c :d]])))))) (deftest meta-test (testing "Metadata on CLISTs" (is (= (meta (with-meta (slist/new-list) {:test :data})) {:test :data})))) ================================================ FILE: test/schism/impl/types/map_test.cljc ================================================ (ns schism.impl.types.map-test (:require #?(:clj [clojure.test :refer [deftest testing is]] :cljs [cljs.test :refer [deftest testing is]]) #?(:cljs [cljs.reader :as reader]) [schism.impl.types.map :as smap] [schism.node :as node] [schism.impl.protocols :as proto]) #?(:clj (:import schism.impl.types.map.Map))) (deftest basic-IPC-ops (testing "Equiv for maps" (is (= (smap/new-map) {})) (is (= (smap/new-map :a true) {:a true}))) (testing "Empty for maps" (is (= (empty (smap/new-map :a true)) (empty {:a true})))) (testing "Count for maps" (is (= (count (smap/new-map :a true)) (count {:a true})))) (testing "Conj for maps" (is (= (conj (smap/new-map) [:a true]) (conj {} [:a true]))) (is (= (conj (smap/new-map :a true) [:a false]) (conj {:a true} [:a false]))))) (deftest basic-IPS-ops (testing "dissoc" (is (= (dissoc (smap/new-map :a true) :a) (dissoc {:a true} :a)))) (testing "contains" (is (= (contains? (smap/new-map :a true) :a) (contains? {:a true} :a)))) (testing "get" (is (= (get (smap/new-map :a true) :a) (get {:a true} :a))))) (deftest converge-test (testing "Converge after concurrent additions on another node." (node/initialize-node! :converge-test-origin) ;; Can only rely on millisecond time scales, so sleep 1 second ;; between ops so that there's some non-zero passage of time (let [transfer (-> (smap/new-map :a true) (conj [:b 3])) other (node/with-node :converge-test-other-node (conj transfer [:c :d])) result (proto/synchronize transfer other)] (is (= result {:a true :b 3 :c :d})) (is (= {:a true :b 3 :c :d} (.-data result))) (doseq [[k v] (.-vclock result)] (is (#{:converge-test-origin :converge-test-other-node} k)) (is (instance? #?(:clj java.util.Date :cljs js/Date) v))) (is (= #{:converge-test-origin :converge-test-other-node} (set (keys (.-vclock result))))) (is (= (set (keys (.-data result))) (set (keys (.-birth-dots result))))) (doseq [[key [node time]] (.-birth-dots result)] (is (#{:a :b :c} key)) (is (#{:converge-test-origin :converge-test-other-node} node)) (is (instance? #?(:clj java.util.Date :cljs js/Date) time))))) (testing "Dissoc on another node mirrored locally after converge." (node/initialize-node! :converge-test-origin) (let [transfer (smap/new-map :a true :b 3 :c :d) other (node/with-node :converge-test-other-node (dissoc transfer :c)) result (proto/synchronize transfer other)] (is (= other {:a true :b 3})) (is (= result {:a true :b 3}))))) (deftest seqable-test (testing "Can turn an ORMWOT into a seq" (is (= (seq (smap/new-map :a true :b 3 :c :d)) (seq (hash-map :a true :b 3 :c :d)))))) (deftest ifn-test (testing "Can invoke an ORMWOT" (is (= ((smap/new-map :a true :b 3 :c :d) :c) ({:a true :b 3 :c :d} :c))))) (deftest string-test (testing "Prints to console readably, even though edn is verbose" (is (= (str (smap/new-map :a true :b 3 :c :d)) (str (hash-map :a true :b 3 :c :d)))))) (deftest serialization-test (testing "Round trip serialization generates the same structure." (let [^Map origin (-> (smap/new-map :a true :b 3 :c :d) (conj [:d :quux]) (dissoc :c)) ^Map round-tripped (-> origin pr-str #?(:clj read-string :cljs reader/read-string))] (is (= (.-data origin) (.-data round-tripped))) (is (= (.-vclock origin) (.-vclock round-tripped))) (is (= (.-birth-dots origin) (.-birth-dots round-tripped)))))) (deftest hashing-test (testing "Hashes to the same value as an equivalent hash-map" (is (= (hash (into (smap/new-map) [[:a true] [:b 3] [:c :d]])) (hash (into {} [[:a true] [:b 3] [:c :d]])))))) (deftest meta-test (testing "Metadata on ORMWOTs" (is (= (meta (with-meta (smap/new-map) {:test :data})) {:test :data})))) ================================================ FILE: test/schism/impl/types/nested_map_test.cljc ================================================ (ns schism.impl.types.nested-map-test (:require #?(:clj [clojure.test :refer [deftest testing is]] :cljs [cljs.test :refer [deftest testing is]]) #?(:cljs [cljs.reader :as reader]) [schism.impl.types.nested-map :as nmap] [schism.node :as node] [schism.impl.protocols :as proto]) #?(:clj (:import schism.impl.types.nested_map.NestedMap))) (deftest basic-IPC-ops (testing "Equiv for maps" (is (= (nmap/new-map) {})) (is (= (nmap/new-map :a true) {:a true}))) (testing "Empty for maps" (is (= (empty (nmap/new-map :a true)) (empty {:a true})))) (testing "Count for maps" (is (= (count (nmap/new-map :a true)) (count {:a true})))) (testing "Conj for maps" (is (= (conj (nmap/new-map) [:a true]) (conj {} [:a true]))) (is (= (conj (nmap/new-map :a true) [:a false]) (conj {:a true} [:a false]))))) (deftest basic-IPS-ops (testing "dissoc" (is (= (dissoc (nmap/new-map :a true) :a) (dissoc {:a true} :a)))) (testing "contains" (is (= (contains? (nmap/new-map :a true) :a) (contains? {:a true} :a)))) (testing "get" (is (= (get (nmap/new-map :a true) :a) (get {:a true} :a))))) (deftest converge-test (testing "Converge after concurrent additions on another node." (node/initialize-node! :converge-test-origin) ;; Can only rely on millisecond time scales, so sleep 1 second ;; between ops so that there's some non-zero passage of time (let [transfer (-> (nmap/new-map :a true) (conj [:b 3])) other (node/with-node :converge-test-other-node (conj transfer [:c :d])) result (proto/synchronize transfer other)] (is (= result {:a true :b 3 :c :d})) (is (= {:a true :b 3 :c :d} (.-data result))) (doseq [[k v] (.-vclock result)] (is (#{:converge-test-origin :converge-test-other-node} k)) (is (instance? #?(:clj java.util.Date :cljs js/Date) v))) (is (= #{:converge-test-origin :converge-test-other-node} (set (keys (.-vclock result))))) (is (= (set (keys (.-data result))) (set (keys (.-birth-dots result))))) (doseq [[key {node :a time :t} :as entry] (.-birth-dots result)] (is (#{:a :b :c} key)) (is (#{:converge-test-origin :converge-test-other-node} node)) (is (instance? #?(:clj java.util.Date :cljs js/Date) time))))) (testing "Dissoc on another node mirrored locally after converge." (node/initialize-node! :converge-test-origin) (let [transfer (nmap/new-map :a true :b 3 :c :d) other (node/with-node :converge-test-other-node (dissoc transfer :c)) result (proto/synchronize transfer other)] (is (= other {:a true :b 3})) (is (= result {:a true :b 3})) (is (= (contains? (.-birth-dots other) :c) false)) (is (= (contains? (.-birth-dots result) :c) false))))) (deftest seqable-test (testing "Can turn an ORMWOT into a seq" (is (= (seq (nmap/new-map :a true :b 3 :c :d)) (seq (hash-map :a true :b 3 :c :d)))))) (deftest ifn-test (testing "Can invoke an ORMWOT" (is (= ((nmap/new-map :a true :b 3 :c :d) :c) ({:a true :b 3 :c :d} :c))))) (deftest string-test (testing "Prints to console readably, even though edn is verbose" (is (= (str (nmap/new-map :a true :b 3 :c :d)) (str (hash-map :a true :b 3 :c :d)))))) (deftest serialization-test (testing "Round trip serialization generates the same structure." (let [^NestedMap origin (-> (nmap/new-map :a true :b 3 :c :d) (conj [:d :quux]) (dissoc :c)) ^NestedMap round-tripped (-> origin pr-str #?(:clj read-string :cljs reader/read-string))] (is (= (.-data origin) (.-data round-tripped))) (is (= (.-vclock origin) (.-vclock round-tripped))) (is (= (.-birth-dots origin) (.-birth-dots round-tripped)))))) (deftest hashing-test (testing "Hashes to the same value as an equivalent hash-map" (is (= (hash (into (nmap/new-map) [[:a true] [:b 3] [:c :d]])) (hash (into {} [[:a true] [:b 3] [:c :d]])))))) (deftest meta-test (testing "Metadata on ORMWOTs" (is (= (meta (with-meta (nmap/new-map) {:test :data})) {:test :data})))) (deftest path-atomicity-test (testing "Concurrent modification of a subtree by two nodes converges the atomic values" (node/initialize-node! :path-atomicity-origin) (let [original (-> (nmap/new-map :a true) (assoc-in [:b :c] true) (#(node/with-node :derivation-node-a (assoc-in % [:b :d] {:e 3 :f "frog"})))) derivation-a (node/with-node :derivation-node-a (assoc-in original [:b :d :f] "hog")) derivation-b (update-in original [:b :d :e] inc) result (proto/synchronize derivation-b derivation-a)] (is (= original {:a true :b {:c true :d {:e 3 :f "frog"}}})) (is (= result {:a true :b {:c true :d {:e 4 :f "hog"}}}))))) (deftest vector-conjs-compose (testing "Two conjs on different nodes yield both their conjs on convergence and do not overwrite" (node/initialize-node! :vector-conjs-origins) (let [original (-> (nmap/new-map :a [1]) (#(node/with-node :derivation-node-a (assoc-in % [:b] [2])))) derivation-a (node/with-node :derivation-node-a (update-in original [:a] conj 2)) derivation-b (node/with-node :derivation-node-b (update-in original [:a] conj 3)) result (proto/synchronize derivation-b derivation-a)] (is (= original {:a [1] :b [2]})) (is (= derivation-a {:a [1 2] :b [2]})) (is (= derivation-b {:a [1 3] :b [2]})) (is (= result {:a [1 2 3] :b [2]}))))) (deftest interesting-map-inits (testing "{-1 [0]}" (node/initialize-node! :interesting-map-inits) (let [v (nmap/new-map -1 [0])] (is (= {-1 [0]} (.-data v))) (is (= :interesting-map-inits (get-in (.-birth-dots v) [-1 0 :a]))) (is (= -1 (get-in (.-birth-dots v) [-1 0 :i]))) (is (instance? #?(:clj java.util.Date :cljs js/Date) (get-in (.-birth-dots v) [-1 0 :t]))))) (testing "{0 [[0] 0]}" (node/initialize-node! :interesting-map-inits) (let [v (nmap/new-map 0 [[0] 0])] (is (= {0 [[0] 0]} (.-data v))) (is (= :interesting-map-inits (get-in (.-birth-dots v) [0 0 0 :a]))) (is (= -1 (get-in (.-birth-dots v) [0 0 0 :i]))) (is (instance? #?(:clj java.util.Date :cljs js/Date) (get-in (.-birth-dots v) [0 0 0 :t]))))) (testing "{0 [0 0] -1 [0 0 0 0 0 0] -2 0}" (node/initialize-node! :interesting-map-inits) (let [v (nmap/new-map 0 [0 0] -1 [0 0 0 0 0 0] -2 0)] (is (= {0 [0 0] -1 [0 0 0 0 0 0] -2 0} (.-data v))) (is (= :interesting-map-inits (get-in (.-birth-dots v) [0 0 :a]))) (is (= -1 (get-in (.-birth-dots v) [0 0 :i]))) (is (instance? #?(:clj java.util.Date :cljs js/Date) (get-in (.-birth-dots v) [0 0 :t]))))) (testing "{0 [0 0] -1 [0 0 0 0 0 0] -2 0}" (node/initialize-node! :interesting-map-inits) (let [v (nmap/new-map 0 [0 0] -1 [0 0 0 0 0 0] -2 0)] (is (= {0 [0 0] -1 [0 0 0 0 0 0] -2 0} (.-data v))) (is (= :interesting-map-inits (get-in (.-birth-dots v) [0 0 :a]))) (is (= -1 (get-in (.-birth-dots v) [0 0 :i]))) (is (instance? #?(:clj java.util.Date :cljs js/Date) (get-in (.-birth-dots v) [0 0 :t]))))) (testing "{{} [0 0 0 0 0 0 0 0] #{} 0}" (node/initialize-node! :interesting-map-inits) (let [v (nmap/new-map {} [0 0 0 0 0 0 0 0] #{} 0)] (is (= {{} [0 0 0 0 0 0 0 0] #{} 0} (.-data v))) (is (= :interesting-map-inits (get-in (.-birth-dots v) [{} 0 :a]))) (is (= -1 (get-in (.-birth-dots v) [{} 0 :i]))) (is (instance? #?(:clj java.util.Date :cljs js/Date) (get-in (.-birth-dots v) [{} 0 :t])))))) ================================================ FILE: test/schism/impl/types/nested_vector_test.cljc ================================================ (ns schism.impl.types.nested-vector-test (:require #?(:clj [clojure.test :refer [deftest testing is]] :cljs [cljs.test :refer [deftest testing is]]) #?(:cljs [cljs.reader :as reader]) [schism.impl.types.nested-vector :as nvector] [schism.node :as node] [schism.impl.protocols :as proto]) #?(:clj (:import schism.impl.types.nested_vector.NestedVector))) (deftest basic-IPC-ops (testing "Equiv for maps" (is (= (nvector/new-vector) [])) (is (= (nvector/new-vector :a true) [:a true]))) (testing "Empty for maps" (is (= (empty (nvector/new-vector :a true)) (empty [:a true])))) (testing "Count for maps" (is (= (count (nvector/new-vector :a true)) (count [:a true])))) (testing "Conj for maps" (is (= (conj (nvector/new-vector) [:a true]) (conj [] [:a true]))) (is (= (conj (nvector/new-vector :a true) [:a false]) (conj [:a true] [:a false]))))) (deftest basic-vector-ops (testing "peek" (is (= (peek (nvector/new-vector :a true)) (peek [:a true])))) (testing "pop" (is (= (pop (nvector/new-vector :a true)) (pop [:a true]))))) (deftest converge-test (testing "Converge after concurrent additions on another node." (node/initialize-node! :converge-test-origin) ;; Can only rely on millisecond time scales, so sleep 1 second ;; between ops so that there's some non-zero passage of time (let [transfer (-> (nvector/new-vector :a true) (conj 3)) other (node/with-node :converge-test-other-node (conj transfer :d)) result (proto/synchronize transfer other)] (is (= result [:a true 3 :d])) (is (= [:a true 3 :d] (.-data result))) (doseq [[k v] (.-vclock result)] (is (#{:converge-test-origin :converge-test-other-node} k)) (is (instance? #?(:clj java.util.Date :cljs js/Date) v))) (is (= #{:converge-test-origin :converge-test-other-node} (set (keys (.-vclock result))))) (is (= (count (.-data result)) (count (.-insertions result)))) (doseq [{node :a time :t} (.-insertions result)] (is (#{:converge-test-origin :converge-test-other-node} node)) (is (instance? #?(:clj java.util.Date :cljs js/Date) time))))) (testing "Pop on another node mirrored locally after converge." (node/initialize-node! :converge-test-origin) (let [transfer (nvector/new-vector :a true :b 3 :c :d) other (node/with-node :converge-test-other-node (pop transfer)) result (proto/synchronize transfer other)] (is (= other [:a true :b 3 :c])) (is (= result [:a true :b 3 :c])) (is (= (count (.-insertions other)) 5)) (is (= (count (.-insertions result)) 5))))) (deftest seqable-test (testing "Can turn an nested vector into a seq" (is (= (seq (nvector/new-vector :a true :b 3 :c :d)) (seq (vector :a true :b 3 :c :d)))))) (deftest ifn-test (testing "Can invoke a vector" (is (= ((nvector/new-vector :a true :b 3 :c :d) 0) ([:a true :b 3 :c :d] 0) :a)))) (deftest string-test (testing "Prints to console readably, even though edn is verbose" (is (= (str (nvector/new-vector :a true :b 3 :c :d)) (str (vector :a true :b 3 :c :d)))))) (deftest serialization-test (testing "Round trip serialization generates the same structure." (let [^NestedVector origin (-> (nvector/new-vector :a true :b 3 :c :d) (conj [:d :quux]) pop) ^NestedVector round-tripped (-> origin pr-str #?(:clj read-string :cljs reader/read-string))] (is (= (.-data origin) (.-data round-tripped))) (is (= (.-vclock origin) (.-vclock round-tripped))) (is (= (.-insertions origin) (.-insertions round-tripped)))))) (deftest hashing-test (testing "Hashes to the same value as an equivalent vector" (is (= (hash (into (nvector/new-vector) [[:a true] [:b 3] [:c :d]])) (hash (into [] [[:a true] [:b 3] [:c :d]])))))) (deftest meta-test (testing "Metadata on nested vectors" (is (= (meta (with-meta (nvector/new-vector) {:test :data})) {:test :data})))) (deftest path-atomicity-test (testing "Concurrent modification of a subtree by two nodes converges the atomic values" (node/initialize-node! :path-atomicity-origin) (let [original (-> (nvector/new-vector :a true) (assoc-in [2 :b] true) (#(node/with-node :derivation-node-a (assoc-in % [3 :c] {:e 3 :f "frog"})))) derivation-a (node/with-node :derivation-node-a (assoc-in original [3 :c :f] "hog")) derivation-b (update-in original [3 :c :e] inc) result (proto/synchronize derivation-b derivation-a)] (is (= original [:a true {:b true} {:c {:e 3 :f "frog"}}])) (is (= result [:a true {:b true} {:c {:e 4 :f "hog"}}]))))) (deftest vector-conjs-compose (testing "Two conjs on different nodes yield both their conjs on convergence and do not overwrite" (node/initialize-node! :vector-conjs-origins) (let [original (-> (nvector/new-vector :a [1]) (#(node/with-node :derivation-node-a (assoc-in % [2] [2])))) derivation-a (node/with-node :derivation-node-a (update-in original [1] conj 2)) derivation-b (node/with-node :derivation-node-b (update-in original [1] conj 3)) result (proto/synchronize derivation-b derivation-a)] (is (= original [:a [1] [2]])) (is (= derivation-a [:a [1 2] [2]])) (is (= derivation-b [:a [1 3] [2]])) (is (= result [:a [1 2 3] [2]]))))) (deftest interesting-vector-inits (testing "Empty vector stacked 2 deep" (node/initialize-node! :interesting-vector-inits) (let [v (nvector/new-vector [[]])] (is (= [[[]]] (.-data v))) (is (= :interesting-vector-inits (get-in (.-insertions v) [0 0 :a]))) (is (= -1 (get-in (.-insertions v) [0 0 :i]))) (is (instance? #?(:clj java.util.Date :cljs js/Date) (get-in (.-insertions v) [0 0 :t])))))) ================================================ FILE: test/schism/impl/types/set_test.cljc ================================================ (ns schism.impl.types.set-test (:require #?(:clj [clojure.test :refer [deftest testing is are]] :cljs [cljs.test :refer [deftest testing is are]]) #?(:cljs [cljs.reader :as reader]) [schism.impl.types.set :as sset] [schism.node :as node] [schism.impl.protocols :as proto]) #?(:clj (:import schism.impl.types.set.Set))) (deftest basic-IPC-ops (testing "Equiv for sets" (is (= (sset/new-set) #{})) (is (= (sset/new-set :a) #{:a}))) (testing "Empty for sets" (is (= (empty (sset/new-set :a)) (empty #{:a})))) (testing "Count for sets" (is (= (count (sset/new-set :a)) (count #{:a})))) (testing "Conj for sets" (is (= (conj (sset/new-set) :a) (conj #{} :a))) (is (= (conj (sset/new-set :a) :a) (conj #{:a} :a))))) (deftest basic-IPS-ops (testing "disjoin" (is (= (disj (sset/new-set :a) :a) (disj #{:a} :a)))) (testing "contains" (is (= (contains? (sset/new-set :a) :a) (contains? #{:a} :a)))) (testing "get" (is (= (get (sset/new-set :a) :a) (get #{:a} :a))))) (deftest converge-test (testing "Converge after concurrent additions on another node." (node/initialize-node! :converge-test-origin) ;; Can only rely on millisecond time scales, so sleep 1 second ;; between ops so that there's some non-zero passage of time (let [transfer (-> (sset/new-set :a) (conj :b)) other (node/with-node :converge-test-other-node (conj transfer :c)) result (proto/synchronize transfer other)] (is (= result #{:a :b :c})) (is (= #{:a :b :c} (.-data result))) (doseq [[k v] (.-vclock result)] (is (#{:converge-test-origin :converge-test-other-node} k)) (is (instance? #?(:clj java.util.Date :cljs js/Date) v))) (is (= #{:converge-test-origin :converge-test-other-node} (set (keys (.-vclock result))))) (is (= (.-data result) (set (keys (.-birth-dots result))))) (doseq [[element [node time]] (.-birth-dots result)] (is (#{:a :b :c} element)) (is (#{:converge-test-origin :converge-test-other-node} node)) (is (instance? #?(:clj java.util.Date :cljs js/Date) time))))) (testing "Disj on another node mirrored locally after converge." (node/initialize-node! :converge-test-origin) (let [transfer (sset/new-set :a :b :c) other (node/with-node :converge-test-other-node (disj transfer :c)) result (proto/synchronize transfer other)] (is (= other #{:a :b})) (is (= result #{:a :b})))) (testing "Concurrent converges will not resolve to a mutually-exclusive addition when vector clocks will support it." (node/initialize-node! :converge-test-origin) (let [transfer (-> (sset/new-set :a) (conj :b)) iterate (conj transfer :d) other (node/with-node :converge-test-other-node (conj transfer :c)) result (proto/synchronize iterate other)] (is (= result #{:a :b :c :d}))))) (deftest seqable-test (testing "Can turn an ORSWOT into a seq" (is (= (seq (sset/new-set :a :b :c)) (seq (hash-set :a :b :c)))))) (deftest ifn-test (testing "Can invoke an ORSWOT" (is (= ((sset/new-set :a :b :c) :c) (#{:a :b :c} :c))))) (deftest string-test (testing "Prints to console readably, even though edn is verbose" (is (= (str (sset/new-set :a :b :c)) (str (hash-set :a :b :c)))))) (deftest serialization-test (testing "Round trip serialization generates the same structure." (let [^Set origin (-> (sset/new-set :a :b :c) (conj :d) (disj :c)) ^Set round-tripped (-> origin pr-str #?(:clj read-string :cljs reader/read-string))] (is (= (.-data origin) (.-data round-tripped))) (is (= (.-vclock origin) (.-vclock round-tripped))) (is (= (.-birth-dots origin) (.-birth-dots round-tripped)))))) (deftest hashing-test (testing "Hashes to the same value as an equivalent hash-set" (is (= (hash (into (sset/new-set) [:a :b :c])) (hash (into #{} [:a :b :c])))))) (deftest meta-test (testing "Metadata on ORSWOTs" (is (= (meta (with-meta (sset/new-set) {:test :data})) {:test :data})))) ================================================ FILE: test/schism/impl/types/vector_test.cljc ================================================ (ns schism.impl.types.vector-test (:require #?(:clj [clojure.test :refer [deftest testing is]] :cljs [cljs.test :refer [deftest testing is]]) #?(:cljs [cljs.reader :as reader]) [schism.impl.types.vector :as svector] [schism.node :as node] [schism.impl.protocols :as proto]) #?(:clj (:import schism.impl.types.vector.ConvergentVector))) (deftest basic-IPC-ops (testing "Equiv for vectors" (is (= (svector/new-vector) [])) (is (= (svector/new-vector :a true) [:a true]))) (testing "Empty for vectors" (is (= (empty (svector/new-vector :a true)) (empty [:a true])))) (testing "Count for vectors" (is (= (count (svector/new-vector :a true)) (count [:a true])))) (testing "Conj for vectors" (is (= (conj (svector/new-vector) [:a true]) (conj [] [:a true]))) (is (= (conj (svector/new-vector :a true) [:a false]) (conj [:a true] [:a false]))))) (deftest basic-vector-ops (testing "conj" (is (= (conj (svector/new-vector :a true) :a) (conj [:a true] :a)))) (testing "peek" (is (= (peek (svector/new-vector :a true)) (peek [:a true])))) (testing "pop" (is (= (pop (svector/new-vector :a true)) (pop [:a true]))))) (deftest converge-test (testing "Converge after concurrent additions on another node." (node/initialize-node! :converge-test-origin) ;; Can only rely on millisecond time scales, so sleep 1 second ;; between ops so that there's some non-zero passage of time (let [transfer (-> (svector/new-vector true :a) (conj [:b 3])) other (node/with-node :converge-test-other-node (conj transfer [:c :d])) result (proto/synchronize transfer other)] (is (= other [true :a [:b 3] [:c :d]])) (is (= result [true :a [:b 3] [:c :d]])))) (testing "Pop on another node mirrored locally after converge." (node/initialize-node! :converge-test-origin) (let [transfer (svector/new-vector :a true :b 3 :c :d) other (node/with-node :converge-test-other-node (pop transfer)) result (proto/synchronize transfer other)] (is (= other [:a true :b 3 :c])) (is (= result [:a true :b 3 :c]))))) (deftest seqable-test (testing "Can turn a CVECTOR into a seq" (is (= (seq (svector/new-vector :a true :b 3 :c :d)) (seq (vector :a true :b 3 :c :d)))))) (deftest string-test (testing "Prints to console readably, even though edn is verbose" (is (= (str (svector/new-vector :a true :b 3 :c :d)) (str (vector :a true :b 3 :c :d)))))) (deftest serialization-test (testing "Round trip serialization generates the same structure." (let [^ConvergentVector origin (-> (svector/new-vector :a true :b 3 :c :d) (conj [:d :quux]) pop) ^ConvergentVector round-tripped (-> origin pr-str #?(:clj read-string :cljs reader/read-string))] (is (= (.-data origin) (.-data round-tripped))) (is (= (.-vclock origin) (.-vclock round-tripped))) (is (= (.-insertions origin) (.-insertions round-tripped)))))) (deftest hashing-test (testing "Hashes to the same value as an equivalent vector" (is (= (hash (into (svector/new-vector) [[:a true] [:b 3] [:c :d]])) (hash (into [] [[:a true] [:b 3] [:c :d]])))))) (deftest meta-test (testing "Metadata on CVECTORs" (is (= (meta (with-meta (svector/new-vector) {:test :data})) {:test :data})))) ================================================ FILE: test/schism/impl/vector_clock_test.cljc ================================================ (ns schism.impl.vector-clock-test (:require #?(:clj [clojure.test :refer [deftest testing is]] :cljs [cljs.test :refer [deftest testing is]]) [schism.impl.vector-clock :as vc] [schism.impl.protocols :as proto] [schism.node :as node])) (defrecord SimpleClocked [last-time vclock] proto/Vclocked (get-clock [_] vclock) (with-clock [this new-clock] (assoc this :vclock new-clock))) (deftest update-clock-test (node/initialize-node! :clock-test-node) (let [test (->SimpleClocked nil {}) updated (vc/update-clock time test (assoc test :last-time time)) time (:last-time updated)] (is (= {:clock-test-node time} (proto/get-clock updated))) (is #?(:clj (instance? java.util.Date time) :cljs true)))) ================================================ FILE: test/schism/node_test.cljc ================================================ (ns schism.node-test (:require #?(:clj [clojure.test :refer [deftest testing is]] :cljs [cljs.test :refer [deftest testing is]]) [schism.node :as node]) #?(:cljs (:require-macros [schism.node :as node]))) (deftest initialize-node!-test (testing "With no arg" (node/initialize-node!) (is (some? node/*current-node*))) (testing "with an arg" (node/initialize-node! "a string node id") (is (= "a string node id" node/*current-node*)))) (deftest with-node-test (testing "with-node clobbers other set values for current-node" (node/initialize-node! :with-node-id) (is (= :with-node-id node/*current-node*)) (node/with-node :another-id (is (not= :with-node-id node/*current-node*)) (is (= :another-id node/*current-node*))))) ================================================ FILE: test/schism/test.cljc ================================================ (ns schism.test (:require #?(:cljs [doo.runner :refer-macros [doo-tests]]) schism.node-test schism.core-test schism.impl.vector-clock-test schism.impl.types.set-test schism.impl.types.map-test schism.impl.types.list-test schism.impl.types.vector-test schism.impl.types.nested-map-test schism.impl.types.nested-vector-test)) #?(:cljs (doo-tests 'schism.node-test 'schism.core-test 'schism.impl.vector-clock-test 'schism.impl.types.set-test 'schism.impl.types.map-test 'schism.impl.types.list-test 'schism.impl.types.vector-test 'schism.impl.types.nested-map-test 'schism.impl.types.nested-vector-test))