Repository: jonase/learndatalogtoday Branch: master Commit: acb59ffe591b Files: 35 Total size: 94.2 KB Directory structure: gitextract_zib_ertf/ ├── .gitignore ├── LICENSE.html ├── README.md ├── externs.js ├── project.clj ├── resources/ │ ├── chapters/ │ │ ├── TOC.md │ │ ├── chapter-0.edn │ │ ├── chapter-0.md │ │ ├── chapter-1.edn │ │ ├── chapter-1.md │ │ ├── chapter-2.edn │ │ ├── chapter-2.md │ │ ├── chapter-3.edn │ │ ├── chapter-3.md │ │ ├── chapter-4.edn │ │ ├── chapter-4.md │ │ ├── chapter-5.edn │ │ ├── chapter-5.md │ │ ├── chapter-6.edn │ │ ├── chapter-6.md │ │ ├── chapter-7.edn │ │ ├── chapter-7.md │ │ ├── chapter-8.edn │ │ ├── chapter-8.md │ │ ├── chapter-9.edn │ │ └── chapter-9.md │ ├── db/ │ │ ├── data.edn │ │ └── schema.edn │ ├── public/ │ │ └── style.css │ └── toc.md └── src/ ├── clj/ │ ├── learndatalogtoday/ │ │ ├── handler.clj │ │ └── views.clj │ └── tutorial/ │ └── fns.clj └── cljs/ └── learndatalogtoday/ ├── core.cljs └── reader.cljs ================================================ FILE CONTENTS ================================================ ================================================ FILE: .gitignore ================================================ /target /lib /classes /checkouts /resources/public/third-party /resources/public/app.js pom.xml *.jar *.class .lein-deps-sum .lein-failures .lein-plugins .lein-env *~ libpeerconnection.log .clj-kondo .lein-repl-history ================================================ FILE: LICENSE.html ================================================ Eclipse Public License - Version 1.0

Eclipse Public License - v 1.0

THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.

1. DEFINITIONS

"Contribution" means:

a) in the case of the initial Contributor, the initial code and documentation distributed under this Agreement, and

b) in the case of each subsequent Contributor:

i) changes to the Program, and

ii) additions to the Program;

where such changes and/or additions to the Program originate from and are distributed by that particular Contributor. A Contribution 'originates' from a Contributor if it was added to the Program by such Contributor itself or anyone acting on such Contributor's behalf. Contributions do not include additions to the Program which: (i) are separate modules of software distributed in conjunction with the Program under their own license agreement, and (ii) are not derivative works of the Program.

"Contributor" means any person or entity that distributes the Program.

"Licensed Patents" mean patent claims licensable by a Contributor which are necessarily infringed by the use or sale of its Contribution alone or when combined with the Program.

"Program" means the Contributions distributed in accordance with this Agreement.

"Recipient" means anyone who receives the Program under this Agreement, including all Contributors.

2. GRANT OF RIGHTS

a) Subject to the terms of this Agreement, each Contributor hereby grants Recipient a non-exclusive, worldwide, royalty-free copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, distribute and sublicense the Contribution of such Contributor, if any, and such derivative works, in source code and object code form.

b) Subject to the terms of this Agreement, each Contributor hereby grants Recipient a non-exclusive, worldwide, royalty-free patent license under Licensed Patents to make, use, sell, offer to sell, import and otherwise transfer the Contribution of such Contributor, if any, in source code and object code form. This patent license shall apply to the combination of the Contribution and the Program if, at the time the Contribution is added by the Contributor, such addition of the Contribution causes such combination to be covered by the Licensed Patents. The patent license shall not apply to any other combinations which include the Contribution. No hardware per se is licensed hereunder.

c) Recipient understands that although each Contributor grants the licenses to its Contributions set forth herein, no assurances are provided by any Contributor that the Program does not infringe the patent or other intellectual property rights of any other entity. Each Contributor disclaims any liability to Recipient for claims brought by any other entity based on infringement of intellectual property rights or otherwise. As a condition to exercising the rights and licenses granted hereunder, each Recipient hereby assumes sole responsibility to secure any other intellectual property rights needed, if any. For example, if a third party patent license is required to allow Recipient to distribute the Program, it is Recipient's responsibility to acquire that license before distributing the Program.

d) Each Contributor represents that to its knowledge it has sufficient copyright rights in its Contribution, if any, to grant the copyright license set forth in this Agreement.

3. REQUIREMENTS

A Contributor may choose to distribute the Program in object code form under its own license agreement, provided that:

a) it complies with the terms and conditions of this Agreement; and

b) its license agreement:

i) effectively disclaims on behalf of all Contributors all warranties and conditions, express and implied, including warranties or conditions of title and non-infringement, and implied warranties or conditions of merchantability and fitness for a particular purpose;

ii) effectively excludes on behalf of all Contributors all liability for damages, including direct, indirect, special, incidental and consequential damages, such as lost profits;

iii) states that any provisions which differ from this Agreement are offered by that Contributor alone and not by any other party; and

iv) states that source code for the Program is available from such Contributor, and informs licensees how to obtain it in a reasonable manner on or through a medium customarily used for software exchange.

When the Program is made available in source code form:

a) it must be made available under this Agreement; and

b) a copy of this Agreement must be included with each copy of the Program.

Contributors may not remove or alter any copyright notices contained within the Program.

Each Contributor must identify itself as the originator of its Contribution, if any, in a manner that reasonably allows subsequent Recipients to identify the originator of the Contribution.

4. COMMERCIAL DISTRIBUTION

Commercial distributors of software may accept certain responsibilities with respect to end users, business partners and the like. While this license is intended to facilitate the commercial use of the Program, the Contributor who includes the Program in a commercial product offering should do so in a manner which does not create potential liability for other Contributors. Therefore, if a Contributor includes the Program in a commercial product offering, such Contributor ("Commercial Contributor") hereby agrees to defend and indemnify every other Contributor ("Indemnified Contributor") against any losses, damages and costs (collectively "Losses") arising from claims, lawsuits and other legal actions brought by a third party against the Indemnified Contributor to the extent caused by the acts or omissions of such Commercial Contributor in connection with its distribution of the Program in a commercial product offering. The obligations in this section do not apply to any claims or Losses relating to any actual or alleged intellectual property infringement. In order to qualify, an Indemnified Contributor must: a) promptly notify the Commercial Contributor in writing of such claim, and b) allow the Commercial Contributor to control, and cooperate with the Commercial Contributor in, the defense and any related settlement negotiations. The Indemnified Contributor may participate in any such claim at its own expense.

For example, a Contributor might include the Program in a commercial product offering, Product X. That Contributor is then a Commercial Contributor. If that Commercial Contributor then makes performance claims, or offers warranties related to Product X, those performance claims and warranties are such Commercial Contributor's responsibility alone. Under this section, the Commercial Contributor would have to defend claims against the other Contributors related to those performance claims and warranties, and if a court requires any other Contributor to pay any damages as a result, the Commercial Contributor must pay those damages.

5. NO WARRANTY

EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Each Recipient is solely responsible for determining the appropriateness of using and distributing the Program and assumes all risks associated with its exercise of rights under this Agreement , including but not limited to the risks and costs of program errors, compliance with applicable laws, damage to or loss of data, programs or equipment, and unavailability or interruption of operations.

6. DISCLAIMER OF LIABILITY

EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR ANY CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

7. GENERAL

If any provision of this Agreement is invalid or unenforceable under applicable law, it shall not affect the validity or enforceability of the remainder of the terms of this Agreement, and without further action by the parties hereto, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable.

If Recipient institutes patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Program itself (excluding combinations of the Program with other software or hardware) infringes such Recipient's patent(s), then such Recipient's rights granted under Section 2(b) shall terminate as of the date such litigation is filed.

All Recipient's rights under this Agreement shall terminate if it fails to comply with any of the material terms or conditions of this Agreement and does not cure such failure in a reasonable period of time after becoming aware of such noncompliance. If all Recipient's rights under this Agreement terminate, Recipient agrees to cease use and distribution of the Program as soon as reasonably practicable. However, Recipient's obligations under this Agreement and any licenses granted by Recipient relating to the Program shall continue and survive.

Everyone is permitted to copy and distribute copies of this Agreement, but in order to avoid inconsistency the Agreement is copyrighted and may only be modified in the following manner. The Agreement Steward reserves the right to publish new versions (including revisions) of this Agreement from time to time. No one other than the Agreement Steward has the right to modify this Agreement. The Eclipse Foundation is the initial Agreement Steward. The Eclipse Foundation may assign the responsibility to serve as the Agreement Steward to a suitable separate entity. Each new version of the Agreement will be given a distinguishing version number. The Program (including Contributions) may always be distributed subject to the version of the Agreement under which it was received. In addition, after a new version of the Agreement is published, Contributor may elect to distribute the Program (including its Contributions) under the new version. Except as expressly stated in Sections 2(a) and 2(b) above, Recipient receives no rights or licenses to the intellectual property of any Contributor under this Agreement, whether expressly, by implication, estoppel or otherwise. All rights in the Program not expressly granted under this Agreement are reserved.

This Agreement is governed by the laws of the State of New York and the intellectual property laws of the United States of America. No party to this Agreement will bring a legal action under this Agreement more than one year after the cause of action arose. Each party waives its rights to a jury trial in any resulting litigation.

================================================ FILE: README.md ================================================ # [learndatalogtoday](https://www.learndatalogtoday.org) An interactive [Datalog tutorial](https://www.learndatalogtoday.org). ## Prerequisites You will need [Leiningen](https://github.com/technomancy/leiningen) and java installed. ## Run locally $ lein uberjar $ java -cp target/learndatalogtoday-standalone.jar clojure.main -m learndatalogtoday.handler Server is now running on `$PORT` (`http://localhost:8080` by default). ## License Copyright © 2013-2023 Jonas Enlund Distributed under the Eclipse Public License, the same as Clojure. ================================================ FILE: externs.js ================================================ var CodeMirror; function CodeMirror(){}; CodeMirror.fromTextArea = {}; CodeMirror.refresh = {}; CodeMirror.getValue = {}; CodeMirror.setValue = {}; var $; function $(){}; $.on = {}; ================================================ FILE: project.clj ================================================ (defproject learndatalogtoday "0.1.0" :description "Interactive Datalog Tutorial" :url "http://learndatalogtoday.org" :dependencies [[org.clojure/clojure "1.8.0"] [org.clojure/clojurescript "1.9.227"] [compojure "1.5.1"] [ring/ring-jetty-adapter "1.5.0"] [com.datomic/datomic-free "0.9.5394"] [datomic-query-helpers "0.1.1"] [hiccup "1.0.5"] [markdown-clj "0.9.89"] [fipp "0.6.6"] [com.taoensso/timbre "4.7.4"] [org.clojure/core.rrb-vector "0.1.2"] ;; cljs [hylla "0.2.0"] [hiccups "0.3.0"] [domina "1.0.3"]] :plugins [[lein-ring "0.9.7"] [lein-cljsbuild "1.1.4"]] :source-paths ["src/clj"] :ring {:handler learndatalogtoday.handler/app} :main learndatalogtoday.handler :aot :all :uberjar-name "learndatalogtoday-standalone.jar" :min-lein-version "2.0.0" :profiles {:dev {:dependencies [[ring-mock "0.1.5"]]}} :cljsbuild {:builds [{:source-paths ["src/cljs"] :compiler {:output-to "resources/public/app.js" :optimizations :advanced :externs ["externs.js"] :static-fns true}}]}) ================================================ FILE: resources/chapters/TOC.md ================================================ * Chapter 0: Introduction - Welcome - Datalog - Datomic - EDN - Copy/Paste a query as a simple exercise * Chapter 1: Basic Queries - About the example database - The Datalog data model - Queries as edn data - Example query - Wildcard pattern / elide values on right - Explenation of pattern variables and data patterns - 2-3 exercises * Find entity ids of movies made in 1987 * Find all movie names * Find the entity id's of all the directors * Chapter 2: Data patterns - Joins are implicit - Simple example - A second example - At least 4 exercises * Chapter 3: Input, part 1 - Leave $ and % for later - do all of * scalars: ?title * tuples: [?title ?name] * collections: [?name ...] * relations: [ [?title ?score] ] - Exercises for all of the above * Given a movie director, list all his movies. * Given two actors, list all movies they've performed in together. * Given a list of movie title, list all the title and the year the movie was released. * Chapter 4: More queries - query for attributes - query for transaction - ~3 examples * When was the sample data imported? * What attributes are associated with the movie "Die Hard" * Find all available attributes and there type. * Chapter 5: Predicates - ~3 exercises * Find movies older than ?year * Find movies older than ?title * Chapter 6: Transformation functions - Exercises * Given a namespace (as a string) Find all attributes, types and cardinality * Chapter 7: Aggregates * Chapter 8: Rules * Chapter 9: Querying multiple databases * Chapter 10: The End. ================================================ FILE: resources/chapters/chapter-0.edn ================================================ {:title "Welcome" :text-file "chapters/chapter-0.md" :exercises [{:question "Find all movies titles in the database." :hints [] :inputs [{:type :query :value [:find ...] :correct-value [:find ?title :where [_ :movie/title ?title]]}]}]} ================================================ FILE: resources/chapters/chapter-0.md ================================================ # Extensible Data Notation In Datomic, a Datalog query is written in [extensible data notation (edn)](http://edn-format.org). Edn is a data format similar to JSON, but it: * is extensible with user defined value types, * has more base types, * is a subset of [Clojure](http://clojure.org) data. Edn consists of: * Numbers: `42`, `3.14159` * Strings: `"This is a string"` * Keywords: `:kw`, `:namespaced/keyword`, `:foo.bar/baz` * Symbols: `max`, `+`, `?title` * Vectors: `[1 2 3]` `[:find ?foo ...]` * Lists: `(3.14 :foo [:bar :baz])`, `(+ 1 2 3 4)` * Instants: `#inst "2013-02-26"` * .. and a few other things which we will not need in this tutorial. Here is an example query that finds all movie titles in our example database: [:find ?title :where [_ :movie/title ?title]] Note that the query is a vector with four elements: * the keyword `:find` * the symbol `?title` * the keyword `:where` * the vector `[_ :movie/title ?title]` We'll go over the specific parts of the query later, but for now you should simply type the above query verbatim into the textbox below, press **Run Query**, and then continue to the next part of the tutorial. ================================================ FILE: resources/chapters/chapter-1.edn ================================================ {:title "Basic Queries" :text-file "chapters/chapter-1.md" :exercises [{:question "Find the entity ids of movies made in 1987" :hints ["You will need to use the `:movie/year` attribute" "The data pattern should look something like `[?e :movie/title 1987]`"] :inputs [{:type :query :value [:find ?e :where ...] :correct-value [:find ?e :where [?e :movie/year 1987]]}]} {:question "Find the entity-id **and** titles of movies in the database" :inputs [{:type :query :value [:find ?e ?title :where ...] :correct-value [:find ?e ?title :where [?e :movie/title ?title]]}]} {:question "Find the name of all people in the database" :inputs [{:type :query :value [:find] :correct-value [:find ?name :where [?p :person/name ?name]]}]}]} ================================================ FILE: resources/chapters/chapter-1.md ================================================ # Basic Queries The example database we'll use contains *movies* mostly, but not exclusively, from the 80s. You'll find information about movie titles, release year, directors, cast members etc. As the tutorial advances we'll learn more about the contents of the database and how it's organized. The data model in Datomic is based around atomic facts called **datoms**. A datom is a 4-tuple consisting of * Entity ID * Attribute * Value * Transaction ID You can think of the database as a flat **set of datoms** of the form: [ ] ... [ 167 :person/name "James Cameron" 102 ] [ 234 :movie/title "Die Hard" 102 ] [ 234 :movie/year 1987 102 ] [ 235 :movie/title "Terminator" 102 ] [ 235 :movie/director 167 102 ] ... Note that the last two datoms share the same entity ID, which means they are facts about the same movie. Note also that the last datom's value is the same as the first datom's entity ID, i.e. the value of the `:movie/director` attribute is itself an entity. All the datoms in the above set were added to the database in the same transaction, so they share the same transaction ID. A query is represented as a vector starting with the keyword `:find` followed by one or more **pattern variables** (symbols starting with `?`, e.g. `?title`). After the find clause comes the `:where` clause which restricts the query to datoms that match the given **data patterns**. For example, this query finds all entity-ids that have the attribute `:person/name` with a value of `"Ridley Scott"`: [:find ?e :where [?e :person/name "Ridley Scott"]] A data pattern is a datom with some parts replaced with pattern variables. It is the job of the query engine to figure out every possible value of each of the pattern variables and return the ones that are specified in the `:find` clause. The symbol `_` can be used as a wildcard for the parts of the data pattern that you wish to ignore. You can also elide trailing values in a data pattern. Therefore, the previous query is equivalent to this next query, because we ignore the transaction part of the datoms. [:find ?e :where [?e :person/name "Ridley Scott" _]] ================================================ FILE: resources/chapters/chapter-2.edn ================================================ {:title "Data Patterns" :text-file "chapters/chapter-2.md" :exercises [{:question "Find movie titles made in 1985" :inputs [{:type :query :value [:find ?title :where ...] :correct-value [:find ?title :where [?m :movie/title ?title] [?m :movie/year 1985]]}]} {:question "What year was \"Alien\" released?" :inputs [{:type :query :value [:find ?year :where ...] :correct-value [:find ?year :where [?m :movie/title "Alien"] [?m :movie/year ?year]]}]} {:question "Who directed RoboCop? You will need to use `[ :movie/director ]` to find the director for a movie." :inputs [{:type :query :value [:find ?name :where ...] :correct-value [:find ?name :where [?m :movie/title "RoboCop"] [?m :movie/director ?d] [?d :person/name ?name]]}]} {:question "Find directors who have directed Arnold Schwarzenegger in a movie." :inputs [{:type :query :value [:find ?name :where ...] :correct-value [:find ?name :where [?p :person/name "Arnold Schwarzenegger"] [?m :movie/cast ?p] [?m :movie/director ?d] [?d :person/name ?name]]}]}]} ================================================ FILE: resources/chapters/chapter-2.md ================================================ # Data patterns In the previous chapter, we looked at **data patterns**, i.e., vectors after the `:where` clause, such as `[?e :movie/title "Commando"]`. There can be many data patterns in a `:where` clause: [:find ?title :where [?e :movie/year 1987] [?e :movie/title ?title]] The important thing to note here is that the pattern variable `?e` is used in both data patterns. When a pattern variable is used in multiple places, the query engine requires it to be bound to the same value in each place. Therefore, this query will only find movie titles for movies made in 1987. The order of the data patterns does not matter (aside from performance considerations), so the previous query could just as well have been written this way: [:find ?title :where [?e :movie/title ?title] [?e :movie/year 1987]] In both cases, the result set will be exactly the same. Let's say we want to find out who starred in "Lethal Weapon". We will need three data patterns for this. The first one finds the entity ID of the movie with "Lethal Weapon" as the title: [?m :movie/title "Lethal Weapon"] Using the same entity ID at `?m`, we can find the cast members with the data pattern: [?m :movie/cast ?p] In this pattern, `?p` will now be (the entity ID of) a person entity, so we can grab the actual name with: [?p :person/name ?name] The query will therefore be: [:find ?name :where [?m :movie/title "Lethal Weapon"] [?m :movie/cast ?p] [?p :person/name ?name]] ================================================ FILE: resources/chapters/chapter-3.edn ================================================ {:title "Parameterized Queries" :text-file "chapters/chapter-3.md" :exercises [{:question "Find movie title by year" :inputs [{:type :query :value [:find ?title :in $ ?year :where ...] :correct-value [:find ?title :in $ ?year :where [?m :movie/year ?year] [?m :movie/title ?title]]} {:type :value :value 1988}]} {:question "Given a list of movie titles, find the title and the year that movie was released." :inputs [{:type :query :value [:find ?title ?year :in ... :where ...] :correct-value [:find ?title ?year :in $ [?title ...] :where [?m :movie/title ?title] [?m :movie/year ?year]]} {:type :value :value ["Lethal Weapon" "Lethal Weapon 2" "Lethal Weapon 3"]}]} {:question "Find all movie `?title`s where the `?actor` and the `?director` has worked together" :inputs [{:type :query :value [:find ?title :in $ ?actor ?director :where ...] :correct-value [:find ?title :in $ ?actor ?director :where [?a :person/name ?actor] [?d :person/name ?director] [?m :movie/cast ?a] [?m :movie/director ?d] [?m :movie/title ?title]]} {:type :value :value "Michael Biehn"} {:type :value :value "James Cameron"}]} {:question "Write a query that, given an actor name and a relation with movie-title/rating, finds the movie titles and corresponding rating for which that actor was a cast member." :inputs [{:type :query :value [:find ?title ?rating :in ... :where ...] :correct-value [:find ?title ?rating :in $ ?name [[?title ?rating]] :where [?p :person/name ?name] [?m :movie/cast ?p] [?m :movie/title ?title]]} {:type :value :value "Mel Gibson"} {:type :value :value [["Die Hard" 8.3] ["Alien" 8.5] ["Lethal Weapon" 7.6] ["Commando" 6.5] ["Mad Max Beyond Thunderdome" 6.1] ["Mad Max 2" 7.6] ["Rambo: First Blood Part II" 6.2] ["Braveheart" 8.4] ["Terminator 2: Judgment Day" 8.6] ["Predator 2" 6.1] ["First Blood" 7.6] ["Aliens" 8.5] ["Terminator 3: Rise of the Machines" 6.4] ["Rambo III" 5.4] ["Mad Max" 7.0] ["The Terminator" 8.1] ["Lethal Weapon 2" 7.1] ["Predator" 7.8] ["Lethal Weapon 3" 6.6] ["RoboCop" 7.5]]}]}]} ================================================ FILE: resources/chapters/chapter-3.md ================================================ # Parameterized queries Looking at this query: [:find ?title :where [?p :person/name "Sylvester Stallone"] [?m :movie/cast ?p] [?m :movie/title ?title]] It would be great if we could reuse this query to find movie titles for any actor and not just for "Sylvester Stallone". This is possible with an `:in` clause, which provides the query with input parameters, much in the same way that function or method arguments does in your programming language. Here's that query with an input parameter for the actor: [:find ?title :in $ ?name :where [?p :person/name ?name] [?m :movie/cast ?p] [?m :movie/title ?title]] This query takes two arguments: `$` is the database itself (implicit, if no `:in` clause is specified) and `?name` which presumably will be the name of some actor. The above query is executed like `(q query db "Sylvester Stallone")`, where `query` is the query we just saw, and `db` is a database value. You can have any number of inputs to a query. In the above query, the input pattern variable `?name` is bound to a scalar - a string in this case. There are four different kinds of input: scalars, tuples, collections and relations. ## A quick aside Hold on. Where does that `$` get used? In query, each of these data patterns is actually a **5 tuple**, of the form: [ ] It's just that the `database` part is implicit, much like the first parameter in the `:in` clause. This query is functionally identical to the previous one: [:find ?title :in $ ?name :where [$ ?p :person/name ?name] [$ ?m :movie/cast ?p] [$ ?m :movie/title ?title]] ## Tuples A tuple input is written as e.g. `[?name ?age]` and can be used when you want to destructure an input. Let's say you have the vector `["James Cameron" "Arnold Schwarzenegger"]` and you want to use this as input to find all movies where these two people collaborated: [:find ?title :in $ [?director ?actor] :where [?d :person/name ?director] [?a :person/name ?actor] [?m :movie/director ?d] [?m :movie/cast ?a] [?m :movie/title ?title]] Of course, in this case, you could just as well use two distinct inputs instead: :in $ ?director ?actor ## Collections You can use collection destructuring to implement a kind of logical **or** in your query. Say you want to find all movies directed by either James Cameron **or** Ridley Scott: [:find ?title :in $ [?director ...] :where [?p :person/name ?director] [?m :movie/director ?p] [?m :movie/title ?title]] Here, the `?director` pattern variable is initially bound to both "James Cameron" and "Ridley Scott". Note that the ellipsis following `?director` is a literal, not elided code. ## Relations Relations - a set of tuples - are the most interesting and powerful of input types, since you can join external relations with the datoms in your database. As a simple example, let's consider a relation with tuples `[movie-title box-office-earnings]`: [ ... ["Die Hard" 140700000] ["Alien" 104931801] ["Lethal Weapon" 120207127] ["Commando" 57491000] ... ] Let's use this data and the data in our database to find box office earnings for a particular director: [:find ?title ?box-office :in $ ?director [[?title ?box-office]] :where [?p :person/name ?director] [?m :movie/director ?p] [?m :movie/title ?title]] Note that the `?box-office` pattern variable does not appear in any of the data patterns in the `:where` clause. ================================================ FILE: resources/chapters/chapter-4.edn ================================================ {:text-file "chapters/chapter-4.md" :exercises [{:question "What attributes are associated with a given movie." :inputs [{:type :query :value [:find ?attr :in $ ?title :where ...] :correct-value [:find ?attr :in $ ?title :where [?m :movie/title ?title] [?m ?a] [?a :db/ident ?attr]]} {:type :value :value "Commando"}]} {:question "Find the names of all people associated with a particular movie (i.e. both the actors and the directors)" :inputs [{:type :query :value [:find ?name :in $ ?title [?attr ...] :where ...] :correct-value [:find ?name :in $ ?title [?attr ...] :where [?m :movie/title ?title] [?m ?attr ?p] [?p :person/name ?name]]} {:type :value :value "Die Hard"} {:type :value :value [:movie/cast :movie/director]}]} {:question "Find all available attributes, their type and their cardinality. This is essentially a query to find **the schema of the database**. To find all installed attributes you must use the `:db.install/attribute` attribute. You will also need to use the `:db/valueType` and `:db/cardinality` attributes as well as `:db/ident`." :inputs [{:type :query :value [:find ?attr ?type ?card :where ...] :correct-value [:find ?attr ?type ?card :where [_ :db.install/attribute ?a] [?a :db/valueType ?t] [?a :db/cardinality ?c] [?a :db/ident ?attr] [?t :db/ident ?type] [?c :db/ident ?card]]}]} {:question "When was the seed data imported into the database? Grab the transaction of any datom in the database, e.g., `[_ :movie/title _ ?tx]` and work from there." :inputs [{:type :query :value [:find ?inst :where ...] :correct-value [:find ?inst :where [_ :movie/title _ ?tx] [?tx :db/txInstant ?inst]]}]}]} ================================================ FILE: resources/chapters/chapter-4.md ================================================ # More queries A datom, as described earlier, is the 4-tuple `[eid attr val tx]`. So far, we have only asked questions about values and/or entity-ids. It's important to remember that it's also possible to ask questions about attributes and transactions. ## Attributes For example, say we want to find all attributes that are associated with person entities in our database. We know for certain that `:person/name` is one such attribute, but are there others we have not yet seen? [:find ?attr :where [?p :person/name] [?p ?attr]] The above query returns a set of entity ids referring to the attributes we are interested in. To get the actual keywords we need to look them up using the `:db/ident` attribute: [:find ?attr :where [?p :person/name] [?p ?a] [?a :db/ident ?attr]] This is because attributes are also entities in our database! ## Transactions It's also possible to run queries to find information about transactions, such as: * When was a fact asserted? * When was a fact retracted? * Which facts were part of a transaction? * Etc. The transaction entity is the fourth element in the datom vector. The only attribute associated with a transaction (by default) is `:db/txInstant` which is the instant in time when the transaction was committed to the database. Here's how we use the fourth element to find the time that "James Cameron" was set as the name for that person entity: [:find ?timestamp :where [?p :person/name "James Cameron" ?tx] [?tx :db/txInstant ?timestamp]] ================================================ FILE: resources/chapters/chapter-5.edn ================================================ {:text-file "chapters/chapter-5.md" :exercises [{:question "Find movies older than a certain year (inclusive)" :inputs [{:type :query :value [:find ?title ...] :correct-value [:find ?title :in $ ?year :where [?m :movie/title ?title] [?m :movie/year ?y] [(<= ?y ?year)]]} {:type :value :value 1979}]} {:question "Find **actors** older than Danny Glover" :inputs [{:type :query :value [:find ?actor ...] :correct-value [:find ?actor :where [?d :person/name "Danny Glover"] [?d :person/born ?b1] [?e :person/born ?b2] [_ :movie/cast ?e] [(< ?b2 ?b1)] [?e :person/name ?actor]]}]} {:question "Find movies newer than `?year` (inclusive) and has a `?rating` higher than the one supplied" :inputs [{:type :query :value [:find ?title :in ... :where ...] :correct-value [:find ?title :in $ ?year ?rating [[?title ?r]] :where [(< ?rating ?r)] [?m :movie/title ?title] [?m :movie/year ?y] [(<= ?year ?y)]]} {:type :value :value 1990} {:type :value :value 8.0} {:type :value :value [["Die Hard" 8.3] ["Alien" 8.5] ["Lethal Weapon" 7.6] ["Commando" 6.5] ["Mad Max Beyond Thunderdome" 6.1] ["Mad Max 2" 7.6] ["Rambo: First Blood Part II" 6.2] ["Braveheart" 8.4] ["Terminator 2: Judgment Day" 8.6] ["Predator 2" 6.1] ["First Blood" 7.6] ["Aliens" 8.5] ["Terminator 3: Rise of the Machines" 6.4] ["Rambo III" 5.4] ["Mad Max" 7.0] ["The Terminator" 8.1] ["Lethal Weapon 2" 7.1] ["Predator" 7.8] ["Lethal Weapon 3" 6.6] ["RoboCop" 7.5]]}]}]} ================================================ FILE: resources/chapters/chapter-5.md ================================================ # Predicates So far, we have only been dealing with **data patterns**: `[?m :movie/year ?year]`. We have not yet seen a proper way of handling questions like "*Find all movies released before 1984*". This is where **predicate clauses** come into play. Let's start with the query for the question above: [:find ?title :where [?m :movie/title ?title] [?m :movie/year ?year] [(< ?year 1984)]] The last clause, `[(< ?year 1984)]`, is a predicate clause. The predicate clause filters the result set to only include results for which the predicate returns a "truthy" (non-nil, non-false) value. You can use any Clojure function or Java method as a predicate function: [:find ?name :where [?p :person/name ?name] [(.startsWith ?name "M")]] Clojure functions must be fully namespace-qualified, so if you have defined your own predicate `awesome?` you must write it as `(my.namespace/awesome? ?movie)`. Some ubiquitous predicates can be used without namespace qualification: `<, >, <=, >=, =, not=` and so on. ================================================ FILE: resources/chapters/chapter-6.edn ================================================ {:text-file "chapters/chapter-6.md" :exercises [{:question "Find people by age. Use the function `tutorial.fns/age` to find the age given a birthday and a date representing \"today\"." :inputs [{:type :query :value [:find ?name :in $ ?age ?today :where ...] :correct-value [:find ?name :in $ ?age ?today :where [?p :person/name ?name] [?p :person/born ?born] [(tutorial.fns/age ?born ?today) ?age]]} {:type :value :value 63} {:type :value :value #inst "2013-08-02"}]} {:question "Find people younger than Bruce Willis and their ages." :inputs [{:type :query :value [:find ?name ?age :in $ ?today :where ...] :correct-value [:find ?name ?age :in $ ?today :where [?p :person/name "Bruce Willis"] [?p :person/born ?sborn] [?p2 :person/name ?name] [?p2 :person/born ?born] [(< ?sborn ?born)] [(tutorial.fns/age ?born ?today) ?age]]} {:type :value :value #inst "2013-08-02"}]} {:question "The birthday paradox states that in a room of 23 people there is a 50% chance that someone has the same birthday. Write a query to find who has the same birthday. Use the `<` predicate on the names to avoid duplicate answers. You can use (the deprecated) `.getDate` and `.getMonth` java `Date` methods." :inputs [{:type :query :value [:find ?name-1 ?name-2 :where ... [(< ?name-1 ?name-2)]] :correct-value [:find ?name-1 ?name-2 :where [?p1 :person/name ?name-1] [?p2 :person/name ?name-2] [?p1 :person/born ?born-1] [?p2 :person/born ?born-2] [(.getMonth ?born-1) ?m] [(.getMonth ?born-2) ?m] [(.getDate ?born-1) ?d] [(.getDate ?born-2) ?d] [(< ?name-1 ?name-2)]]}]}]} ================================================ FILE: resources/chapters/chapter-6.md ================================================ # Transformation functions **Transformation functions** are pure (= side-effect free) functions or methods which can be used in queries to transform values and bind their results to pattern variables. Say, for example, there exists an attribute `:person/born` with type `:db.type/instant`. Given the birthday, it's easy to calculate the (very approximate) age of a person: (defn age [birthday today] (quot (- (.getTime today) (.getTime birthday)) (* 1000 60 60 24 365))) with this function, we can now calculate the age of a person **inside the query itself**: [:find ?age :in $ ?name ?today :where [?p :person/name ?name] [?p :person/born ?born] [(tutorial.fns/age ?born ?today) ?age]] A transformation function clause has the shape `[( ...) ]` where `` can be the same binding forms as we saw in [chapter 3](/chapter/3): * Scalar: `?age` * Tuple: `[?foo ?bar ?baz]` * Collection: `[?name ...]` * Relation: `[[?title ?rating]]` One thing to be aware of is that transformation functions can't be nested. You can't write [(f (g ?x)) ?a] instead, you must bind intermediate results in temporary pattern variables [(g ?x) ?t] [(f ?t) ?a] ================================================ FILE: resources/chapters/chapter-7.edn ================================================ {:text-file "chapters/chapter-7.md" :exercises [{:question "`count` the number of movies in the database" :inputs [{:type :query :value [:find ... :where ...] :correct-value [:find (count ?m) :where [?m :movie/title]]}]} {:question "Find the birth date of the oldest person in the database." :inputs [{:type :query :value [:find ... :where ...] :correct-value [:find (min ?date) :where [_ :person/born ?date]]}]} {:question "Given a collection of actors and (the now familiar) ratings data. Find the average rating for each actor. The query should return the actor name and the `avg` rating." :inputs [{:type :query :value [:find ... :in ... :where ...] :correct-value [:find ?name (avg ?rating) :in $ [?name ...] [[?title ?rating]] :where [?p :person/name ?name] [?m :movie/cast ?p] [?m :movie/title ?title]]} {:type :value :value ["Sylvester Stallone" "Arnold Schwarzenegger" "Mel Gibson"]} {:type :value :value [["Die Hard" 8.3] ["Alien" 8.5] ["Lethal Weapon" 7.6] ["Commando" 6.5] ["Mad Max Beyond Thunderdome" 6.1] ["Mad Max 2" 7.6] ["Rambo: First Blood Part II" 6.2] ["Braveheart" 8.4] ["Terminator 2: Judgment Day" 8.6] ["Predator 2" 6.1] ["First Blood" 7.6] ["Aliens" 8.5] ["Terminator 3: Rise of the Machines" 6.4] ["Rambo III" 5.4] ["Mad Max" 7.0] ["The Terminator" 8.1] ["Lethal Weapon 2" 7.1] ["Predator" 7.8] ["Lethal Weapon 3" 6.6] ["RoboCop" 7.5]]}]}]} ================================================ FILE: resources/chapters/chapter-7.md ================================================ # Aggregates Aggregate functions such as `sum`, `max` etc. are readily available in Datomic's Datalog implementation. They are written in the `:find` clause in your query: [:find (max ?date) :where ...] An aggregate function collects values from multiple datoms and returns * A single value: `min`, `max`, `sum`, `avg`, etc. * A collection of values: `(min n ?d)` `(max n ?d)` `(sample n ?e)` etc. where `n` is an integer specifying the size of the collection. ================================================ FILE: resources/chapters/chapter-8.edn ================================================ {:text-file "chapters/chapter-8.md" :exercises [{:question "Write a rule `[movie-year ?title ?year]` where `?title` is the title of some movie and `?year` is that movies release year." :inputs [{:type :query :value [:find ?title :in $ % :where [movie-year ?title 1991]]} {:type :rule :value [[(movie-year ?title ?year) ...]] :correct-value [[(movie-year ?title ?year) [?m :movie/title ?title] [?m :movie/year ?year]]]}]} {:question "Two people are friends if they have worked together in a movie. Write a rule `[friends ?p1 ?p2]` where `p1` and `p2` are person entities. Try with a few different `?name` inputs to make sure you got it right. There might be some edge cases here." :inputs [{:type :query :value [:find ?friend :in $ % ?name :where [?p1 :person/name ?name] (friends ?p1 ?p2) [?p2 :person/name ?friend]]} {:type :rule :value [[(friends ?p1 ?p2) ...]] :correct-value [[(friends ?p1 ?p2) [?m :movie/cast ?p1] [?m :movie/cast ?p2] [(not= ?p1 ?p2)]] [(friends ?p1 ?p2) [?m :movie/cast ?p1] [?m :movie/director ?p2]] [(friends ?p1 ?p2) (friends ?p2 ?p1)]]} {:type :value :value "Sigourney Weaver"}]} {:question "Write a rule `[sequels ?m1 ?m2]` where `?m1` and `?m2` are movie entities. You'll need to use the attribute `:movie/sequel`. To implement this rule correctly you can think of the problem like this: A movie `?m2` is a sequel of `?m1` if either * `?m2` is the \"direct\" sequel of `m1` **or** * `?m2` is the sequel of some movie `?m` **and** that movie `?m` is the sequel to `?m1`. There are (at least) three different ways to write the above query. Try to find all three solutions." :inputs [{:type :query :value [:find ?sequel :in $ % ?title :where [?m :movie/title ?title] (sequels ?m ?s) [?s :movie/title ?sequel]]} {:type :rule :value [[(sequels ?m1 ?m2) ...]] :correct-value [[(sequels ?m1 ?m2) [?m1 :movie/sequel ?m2]] [(sequels ?m1 ?m2) [?m :movie/sequel ?m2] (sequels ?m1 ?m)]]} {:type :value :value "Mad Max"}]}]} ================================================ FILE: resources/chapters/chapter-8.md ================================================ # Rules Many times over the course of this tutorial, we have had to write the following three lines of repetitive query code: [?p :person/name ?name] [?m :movie/cast ?p] [?m :movie/title ?title] **Rules** are the means of abstraction in Datalog. You can abstract away reusable parts of your queries into rules, give them meaningful names and forget about the implementation details, just like you can with functions in your favorite programming language. Let's create a rule for the three lines above: [(actor-movie ?name ?title) [?p :person/name ?name] [?m :movie/cast ?p] [?m :movie/title ?title]] The first vector is called the *head* of the rule where the first symbol is the name of the rule. The rest of the rule is called the body. It is possible to use `(...)` or `[...]` to enclose it, but it is conventional to use `(...)` to aid the eye when distinguishing between the rule's head and its body, and also between rule invocations and normal data patterns, as we'll see below. You can think of a rule as a kind of function, but remember that this is logic programming, so we can use the same rule to: * find movie titles given an actor name, and * find actor names given a movie title. Put another way, we can use both `?name` and `?title` in `(actor-movie ?name ?title)` for input as well as for output. If we provide values for neither, we'll get all the possible combinations in the database. If we provide values for one or both, it'll constrain the result returned by the query as you'd expect. To use the above rule, you simply write the head of the rule instead of the data patterns. Any variable with values already bound will be input, the rest will be output. The query to find cast members of some movie, for which we previously had to write: [:find ?name :where [?p :person/name ?name] [?m :movie/cast ?p] [?m :movie/title "The Terminator"]] Now becomes: [:find ?name :in $ % (actor-movie ?name "The Terminator")] The `%` symbol in the `:in` clause represent the rules. You can write any number of rules, collect them in a vector, and pass them to the query engine like any other input: [[(rule-a ?a ?b) ...] [(rule-b ?a ?b) ...] ...] You can use [data patterns](/chapter/2), [predicates](/chapter/5), [transformation functions](/chapter/6) and calls to other rules in the body of a rule. Rules can also be used as another tool to write logical OR queries, as the same rule name can be used several times: [[(associated-with ?person ?movie) [?movie :movie/cast ?person]] [(associated-with ?person ?movie) [?movie :movie/director ?person]]] Subsequent rule definitions will only be used if the ones preceding it aren't satisfied. Using this rule, we can find both directors and cast members very easily: [:find ?name :in $ % :where [?m :movie/title "Predator"] (associated-with ?p ?m) [?p :person/name ?name]] Given the fact that rules can contain calls to other rules, what would happen if a rule called itself? Interesting things, it turns out, but let's find out in the exercises. ================================================ FILE: resources/chapters/chapter-9.edn ================================================ {:text-file "chapters/chapter-9.md"} ================================================ FILE: resources/chapters/chapter-9.md ================================================ # The End. ================================================ FILE: resources/db/data.edn ================================================ [ {:db/id #db/id [:db.part/user -100] :person/name "James Cameron" :person/born #inst "1954-08-16"} {:db/id #db/id [:db.part/user -101] :person/name "Arnold Schwarzenegger" :person/born #inst "1947-07-30"} {:db/id #db/id [:db.part/user -102] :person/name "Linda Hamilton" :person/born #inst "1956-09-26"} {:db/id #db/id [:db.part/user -103] :person/name "Michael Biehn" :person/born #inst "1956-07-31"} {:db/id #db/id [:db.part/user -104] :person/name "Ted Kotcheff" :person/born #inst "1931-04-07"} {:db/id #db/id [:db.part/user -105] :person/name "Sylvester Stallone" :person/born #inst "1946-07-06"} {:db/id #db/id [:db.part/user -106] :person/name "Richard Crenna" :person/born #inst "1926-11-30" :person/death #inst "2003-01-17"} {:db/id #db/id [:db.part/user -107] :person/name "Brian Dennehy" :person/born #inst "1938-07-09"} {:db/id #db/id [:db.part/user -108] :person/name "John McTiernan" :person/born #inst "1951-01-08"} {:db/id #db/id [:db.part/user -109] :person/name "Elpidia Carrillo" :person/born #inst "1961-08-16"} {:db/id #db/id [:db.part/user -110] :person/name "Carl Weathers" :person/born #inst "1948-01-14"} {:db/id #db/id [:db.part/user -111] :person/name "Richard Donner" :person/born #inst "1930-04-24"} {:db/id #db/id [:db.part/user -112] :person/name "Mel Gibson" :person/born #inst "1956-01-03"} {:db/id #db/id [:db.part/user -113] :person/name "Danny Glover" :person/born #inst "1946-07-22"} {:db/id #db/id [:db.part/user -114] :person/name "Gary Busey" :person/born #inst "1944-07-29"} {:db/id #db/id [:db.part/user -115] :person/name "Paul Verhoeven" :person/born #inst "1938-07-18"} {:db/id #db/id [:db.part/user -116] :person/name "Peter Weller" :person/born #inst "1947-06-24"} {:db/id #db/id [:db.part/user -117] :person/name "Nancy Allen" :person/born #inst "1950-06-24"} {:db/id #db/id [:db.part/user -118] :person/name "Ronny Cox" :person/born #inst "1938-07-23"} {:db/id #db/id [:db.part/user -119] :person/name "Mark L. Lester" :person/born #inst "1946-11-26"} {:db/id #db/id [:db.part/user -120] :person/name "Rae Dawn Chong" :person/born #inst "1961-02-28"} {:db/id #db/id [:db.part/user -121] :person/name "Alyssa Milano" :person/born #inst "1972-12-19"} {:db/id #db/id [:db.part/user -122] :person/name "Bruce Willis" :person/born #inst "1955-03-19"} {:db/id #db/id [:db.part/user -123] :person/name "Alan Rickman" :person/born #inst "1946-02-21"} {:db/id #db/id [:db.part/user -124] :person/name "Alexander Godunov" :person/born #inst "1949-11-28" :person/death #inst "1995-05-18"} {:db/id #db/id [:db.part/user -125] :person/name "Robert Patrick" :person/born #inst "1958-11-05"} {:db/id #db/id [:db.part/user -126] :person/name "Edward Furlong" :person/born #inst "1977-08-02"} {:db/id #db/id [:db.part/user -127] :person/name "Jonathan Mostow" :person/born #inst "1961-11-28"} {:db/id #db/id [:db.part/user -128] :person/name "Nick Stahl" :person/born #inst "1979-12-05"} {:db/id #db/id [:db.part/user -129] :person/name "Claire Danes" :person/born #inst "1979-04-12"} {:db/id #db/id [:db.part/user -130] :person/name "George P. Cosmatos" :person/born #inst "1941-01-04" :person/death #inst "2005-04-19"} {:db/id #db/id [:db.part/user -131] :person/name "Charles Napier" :person/born #inst "1936-04-12" :person/death #inst "2011-10-05"} {:db/id #db/id [:db.part/user -132] :person/name "Peter MacDonald"} {:db/id #db/id [:db.part/user -133] :person/name "Marc de Jonge" :person/born #inst "1949-02-16" :person/death #inst "1996-06-06"} {:db/id #db/id [:db.part/user -134] :person/name "Stephen Hopkins"} {:db/id #db/id [:db.part/user -135] :person/name "Ruben Blades" :person/born #inst "1948-07-16"} {:db/id #db/id [:db.part/user -136] :person/name "Joe Pesci" :person/born #inst "1943-02-09"} {:db/id #db/id [:db.part/user -137] :person/name "Ridley Scott" :person/born #inst "1937-11-30"} {:db/id #db/id [:db.part/user -138] :person/name "Tom Skerritt" :person/born #inst "1933-08-25"} {:db/id #db/id [:db.part/user -139] :person/name "Sigourney Weaver" :person/born #inst "1949-10-08"} {:db/id #db/id [:db.part/user -140] :person/name "Veronica Cartwright" :person/born #inst "1949-04-20"} {:db/id #db/id [:db.part/user -141] :person/name "Carrie Henn"} {:db/id #db/id [:db.part/user -142] :person/name "George Miller" :person/born #inst "1945-03-03"} {:db/id #db/id [:db.part/user -143] :person/name "Steve Bisley" :person/born #inst "1951-12-26"} {:db/id #db/id [:db.part/user -144] :person/name "Joanne Samuel"} {:db/id #db/id [:db.part/user -145] :person/name "Michael Preston" :person/born #inst "1938-05-14"} {:db/id #db/id [:db.part/user -146] :person/name "Bruce Spence" :person/born #inst "1945-09-17"} {:db/id #db/id [:db.part/user -147] :person/name "George Ogilvie" :person/born #inst "1931-03-05"} {:db/id #db/id [:db.part/user -148] :person/name "Tina Turner" :person/born #inst "1939-11-26"} {:db/id #db/id [:db.part/user -149] :person/name "Sophie Marceau" :person/born #inst "1966-11-17"} {:db/id #db/id [:db.part/user -200] :movie/title "The Terminator" :movie/year 1984 :movie/director #db/id [:db.part/user -100] :movie/cast [#db/id [:db.part/user -101] #db/id [:db.part/user -102] #db/id [:db.part/user -103]] :movie/sequel #db/id [:db.part/user -207]} {:db/id #db/id [:db.part/user -201] :movie/title "First Blood" :movie/year 1982 :movie/director #db/id [:db.part/user -104] :movie/cast [#db/id [:db.part/user -105] #db/id [:db.part/user -106] #db/id [:db.part/user -107]] :movie/sequel #db/id [:db.part/user -209]} {:db/id #db/id [:db.part/user -202] :movie/title "Predator" :movie/year 1987 :movie/director #db/id [:db.part/user -108] :movie/cast [#db/id [:db.part/user -101] #db/id [:db.part/user -109] #db/id [:db.part/user -110]] :movie/sequel #db/id [:db.part/user -211]} {:db/id #db/id [:db.part/user -203] :movie/title "Lethal Weapon" :movie/year 1987 :movie/director #db/id [:db.part/user -111] :movie/cast [#db/id [:db.part/user -112] #db/id [:db.part/user -113] #db/id [:db.part/user -114]] :movie/sequel #db/id [:db.part/user -212]} {:db/id #db/id [:db.part/user -204] :movie/title "RoboCop" :movie/year 1987 :movie/director #db/id [:db.part/user -115] :movie/cast [#db/id [:db.part/user -116] #db/id [:db.part/user -117] #db/id [:db.part/user -118]]} {:db/id #db/id [:db.part/user -205] :movie/title "Commando" :movie/year 1985 :movie/director #db/id [:db.part/user -119] :movie/cast [#db/id [:db.part/user -101] #db/id [:db.part/user -120] #db/id [:db.part/user -121]] :trivia "In 1986, a sequel was written with an eye to having John McTiernan direct. Schwarzenegger wasn't interested in reprising the role. The script was then reworked with a new central character, eventually played by Bruce Willis, and became Die Hard"} {:db/id #db/id [:db.part/user -206] :movie/title "Die Hard" :movie/year 1988 :movie/director #db/id [:db.part/user -108] :movie/cast [#db/id [:db.part/user -122] #db/id [:db.part/user -123] #db/id [:db.part/user -124]]} {:db/id #db/id [:db.part/user -207] :movie/title "Terminator 2: Judgment Day" :movie/year 1991 :movie/director #db/id [:db.part/user -100] :movie/cast [#db/id [:db.part/user -101] #db/id [:db.part/user -102] #db/id [:db.part/user -125] #db/id [:db.part/user -126]] :movie/sequel #db/id [:db.part/user -208]} {:db/id #db/id [:db.part/user -208] :movie/title "Terminator 3: Rise of the Machines" :movie/year 2003 :movie/director #db/id [:db.part/user -127] :movie/cast [#db/id [:db.part/user -101] #db/id [:db.part/user -128] #db/id [:db.part/user -129]]} {:db/id #db/id [:db.part/user -209] :movie/title "Rambo: First Blood Part II" :movie/year 1985 :movie/director #db/id [:db.part/user -130] :movie/cast [#db/id [:db.part/user -105] #db/id [:db.part/user -106] #db/id [:db.part/user -131]] :movie/sequel #db/id [:db.part/user -210]} {:db/id #db/id [:db.part/user -210] :movie/title "Rambo III" :movie/year 1988 :movie/director #db/id [:db.part/user -132] :movie/cast [#db/id [:db.part/user -105] #db/id [:db.part/user -106] #db/id [:db.part/user -133]]} {:db/id #db/id [:db.part/user -211] :movie/title "Predator 2" :movie/year 1990 :movie/director #db/id [:db.part/user -134] :movie/cast [#db/id [:db.part/user -113] #db/id [:db.part/user -114] #db/id [:db.part/user -135]]} {:db/id #db/id [:db.part/user -212] :movie/title "Lethal Weapon 2" :movie/year 1989 :movie/director #db/id [:db.part/user -111] :movie/cast [#db/id [:db.part/user -112] #db/id [:db.part/user -113] #db/id [:db.part/user -136]] :movie/sequel #db/id [:db.part/user -213]} {:db/id #db/id [:db.part/user -213] :movie/title "Lethal Weapon 3" :movie/year 1992 :movie/director #db/id [:db.part/user -111] :movie/cast [#db/id [:db.part/user -112] #db/id [:db.part/user -113] #db/id [:db.part/user -136]]} {:db/id #db/id [:db.part/user -214] :movie/title "Alien" :movie/year 1979 :movie/director #db/id [:db.part/user -137] :movie/cast [#db/id [:db.part/user -138] #db/id [:db.part/user -139] #db/id [:db.part/user -140]] :movie/sequel #db/id [:db.part/user -215]} {:db/id #db/id [:db.part/user -215] :movie/title "Aliens" :movie/year 1986 :movie/director #db/id [:db.part/user -100] :movie/cast [#db/id [:db.part/user -139] #db/id [:db.part/user -141] #db/id [:db.part/user -103]]} {:db/id #db/id [:db.part/user -216] :movie/title "Mad Max" :movie/year 1979 :movie/director #db/id [:db.part/user -142] :movie/cast [#db/id [:db.part/user -112] #db/id [:db.part/user -143] #db/id [:db.part/user -144]] :movie/sequel #db/id [:db.part/user -217]} {:db/id #db/id [:db.part/user -217] :movie/title "Mad Max 2" :movie/year 1981 :movie/director #db/id [:db.part/user -142] :movie/cast [#db/id [:db.part/user -112] #db/id [:db.part/user -145] #db/id [:db.part/user -146]] :movie/sequel #db/id [:db.part/user -218]} {:db/id #db/id [:db.part/user -218] :movie/title "Mad Max Beyond Thunderdome" :movie/year 1985 :movie/director [#db/id [:db.part/user -142] #db/id [:db.part/user -147]] :movie/cast [#db/id [:db.part/user -112] #db/id [:db.part/user -148]]} {:db/id #db/id [:db.part/user -219] :movie/title "Braveheart" :movie/year 1995 :movie/director [#db/id [:db.part/user -112]] :movie/cast [#db/id [:db.part/user -112] #db/id [:db.part/user -149]]} ] ================================================ FILE: resources/db/schema.edn ================================================ [ {:db/ident :movie/title :db/valueType :db.type/string :db/cardinality :db.cardinality/one :db/id #db/id [:db.part/db] :db.install/_attribute :db.part/db} {:db/ident :movie/year :db/valueType :db.type/long :db/cardinality :db.cardinality/one :db/id #db/id [:db.part/db] :db.install/_attribute :db.part/db} {:db/ident :movie/director :db/valueType :db.type/ref :db/cardinality :db.cardinality/many :db/id #db/id [:db.part/db] :db.install/_attribute :db.part/db} {:db/ident :movie/sequel :db/valueType :db.type/ref :db/cardinality :db.cardinality/one :db/id #db/id [:db.part/db] :db.install/_attribute :db.part/db} {:db/ident :movie/cast :db/valueType :db.type/ref :db/cardinality :db.cardinality/many :db/id #db/id [:db.part/db] :db.install/_attribute :db.part/db} {:db/ident :person/name :db/valueType :db.type/string :db/cardinality :db.cardinality/one :db/id #db/id [:db.part/db] :db.install/_attribute :db.part/db} {:db/ident :person/born :db/valueType :db.type/instant :db/cardinality :db.cardinality/one :db/id #db/id [:db.part/db] :db.install/_attribute :db.part/db} {:db/ident :person/death :db/valueType :db.type/instant :db/cardinality :db.cardinality/one :db/id #db/id [:db.part/db] :db.install/_attribute :db.part/db} {:db/ident :trivia :db/valueType :db.type/string :db/cardinality :db.cardinality/many :db/id #db/id [:db.part/db] :db.install/_attribute :db.part/db} ] ================================================ FILE: resources/public/style.css ================================================ .textcontent h1 { color: #08c; font-size: 18pt; } .textcontent h2 { color: #08c; font-size: 16pt; } .textcontent p { text-align: justify; line-height: 1.6em; } .CodeMirror { height: auto; border: 1px solid lightgrey; margin-bottom:10px; } .CodeMirror-scroll { overflow-y: hidden; overflow-x: auto; } table.resultset { font-family: monospace; } table.resultset thead { font-weight: bold; } ================================================ FILE: resources/toc.md ================================================ # Learn Datalog Today **Learn Datalog Today** is an interactive tutorial designed to teach you the [Datomic](https://datomic.com) dialect of [Datalog](https://en.wikipedia.org/wiki/Datalog). Datalog is a declarative **database query language** with roots in logic programming. Datalog has similar expressive power as [SQL](https://en.wikipedia.org/wiki/Sql). Datomic is a database with an interesting and novel architecture, giving its users a unique set of features. You can read more about Datomic at [https://datomic.com](http://datomic.com) and the architecture is described in some detail [in this InfoQ article](https://www.infoq.com/articles/Architecture-Datomic). ## Table of Contents - [Extensible Data Notation](/chapter/0) - [Basic Queries](/chapter/1) - [Data Patterns](/chapter/2) - [Parameterized Queries](/chapter/3) - [More Queries](/chapter/4) - [Predicates](/chapter/5) - [Transformation Functions](/chapter/6) - [Aggregates](/chapter/7) - [Rules](/chapter/8) This tutorial was written on rainy days for the [Lisp In Summer Projects](http://lispinsummerprojects.org) 2013. If you find bugs, or have suggestions on how to improve the tutorial, please visit the project on [github](https://github.com/jonase/learndatalogtoday). Many thanks to [Robert Stuttaford](https://twitter.com/RobStuttaford) for his careful proof reading/editing. I'd also like to thank everyone who has [contributed](https://github.com/jonase/learndatalogtoday/graphs/contributors) by fixing bugs and spelling mistakes. If you learn datalog today, you can consider [sponsoring](https://github.com/sponsors/jonase) the maintenance and running costs of this website. ================================================ FILE: src/clj/learndatalogtoday/handler.clj ================================================ (ns learndatalogtoday.handler (:gen-class) (:require [clojure.java.io :as io] [clojure.edn :as edn] [compojure.core :refer [routes GET POST]] [compojure.handler :as handler] [compojure.route :as route] [datomic-query-helpers.core :refer [check-query normalize pretty-query-string]] [datomic.api :as d] [fipp.edn :as fipp] [hiccup.page :refer [html5]] [learndatalogtoday.views :as views] [ring.adapter.jetty :as jetty] [taoensso.timbre :as log] [tutorial.fns]) (:import [java.util Date])) (def dev? (boolean (System/getenv "DEVMODE"))) (defn edn-response [edn-data] {:status 200 :headers {"Content-Type" "application/edn"} :body (pr-str edn-data)}) (defn read-file [s] (with-open [r (io/reader (io/resource s))] (read-string (slurp r)))) (defn read-chapter-data [chapter] (->> chapter (format "chapters/chapter-%s.edn") read-file)) (defn read-chapter "Returns a html string" [chapter] (let [chapter-data (read-chapter-data chapter)] (assoc chapter-data :html (views/chapter-response (assoc chapter-data :chapter chapter))))) (def whitelist '#{< > <= >= not= = tutorial.fns/age .getDate .getMonth movie-year sequels friends avg min max sum count}) (defn validate [[query & args]] (let [syms (check-query query args whitelist)] (if (empty? syms) (cons (normalize query) args) (throw (ex-info (str "Non-whitelist symbol used in query/args: " syms ". The symbol whitelist is " whitelist) {:syms syms}))))) (def toc (if dev? views/toc (memoize views/toc))) (def read-chapter-data (if dev? read-chapter-data (memoize read-chapter-data))) (def read-chapter (if dev? read-chapter (memoize read-chapter))) (defn app-routes [db chapters] (routes (GET "/" [] (toc)) (GET ["/chapter/:n" :n #"[0-9]+"] [n] (:html (read-chapter (Integer/parseInt n)))) (GET ["/query/:chapter/:exercise" :chapter #"[0-9]+" :exercise #"[0-9]+"] {{:keys [chapter exercise data] :as params} :params} (try (let [chapter (Integer/parseInt chapter) exercise (Integer/parseInt exercise) usr-input (edn/read-string data) ans-input (get-in (read-chapter-data chapter) [:exercises exercise :inputs]) [ans-query & ans-args] (validate (map #(or (:correct-value %1) %2) ans-input usr-input)) [usr-query & usr-args] (validate (edn/read-string data)) usr-result (apply d/q usr-query db usr-args) ans-result (apply d/q ans-query db ans-args)] (if (= usr-result ans-result) (do (log/info (format "Success query [%s,%s]: %s" chapter exercise (pr-str usr-input))) (edn-response {:status :success :result usr-result})) (do (log/info (format "Fail query [%s,%s]: %s" chapter exercise (pr-str usr-input))) (edn-response {:status :fail :result usr-result :correct-result ans-result})))) (catch Exception e (let [msg (.getMessage e)] (log/info (format "Error query [%s,%s]. Data: %s Message: %s" chapter exercise data msg)) (edn-response {:status :error :message (.getMessage e)}))))) (GET ["/answer/:chapter/:exercise" :chapter #"[0-9]+" :exercise #"[0-9]+"] [chapter exercise] (try (let [chapter (Integer/parseInt chapter) exercise (Integer/parseInt exercise) ans-input (get-in (read-chapter-data chapter) [:exercises exercise :inputs]) value #(or (:correct-value %) (:value %)) answer (map (fn [input] (condp = (:type input) :query (pretty-query-string (value input)) :rule (with-out-str (fipp/pprint (value input))) :value (with-out-str (fipp/pprint (value input))))) ans-input)] (do (log/info (format "Answer request [%s,%s]: %s" chapter exercise (seq answer))) (edn-response answer))))) (route/resources "/") (route/not-found "Not Found"))) (defn init-db [name schema seed-data] (let [uri (str "datomic:mem://" name) conn (do (d/delete-database uri) (d/create-database uri) (d/connect uri))] @(d/transact conn schema) @(d/transact conn seed-data) (d/db conn))) (def app (let [schema (read-file "db/schema.edn") seed-data (read-file "db/data.edn") db (init-db "movies" schema seed-data) chapters (mapv read-chapter (range 8))] ;; TODO 9 (handler/site (app-routes db chapters)))) (defn -main [] (let [port (Integer/parseInt (or (System/getenv "PORT") "8080"))] (jetty/run-jetty app {:port port :join? false}))) ================================================ FILE: src/clj/learndatalogtoday/views.clj ================================================ (ns learndatalogtoday.views (:require [clojure.java.io :as io] [datomic-query-helpers.core :refer [pretty-query-string]] [fipp.edn :as fipp] [hiccup.element :refer [javascript-tag]] [hiccup.page :refer [html5 include-js include-css]] [markdown.core :as md])) (defn footer [] [:footer.text-center {:style "border-top: 1px solid lightgrey; margin-top: 40px;padding:10px;"} [:small [:p [:a {:href "https://www.learndatalogtoday.org"} "www.learndatalogtoday.org"] " © 2013 - 2023 Jonas Enlund"] [:p [:a {:href "https://github.com/jonase/learndatalogtoday"} "github"] " | " [:a {:href "http://lispinsummerprojects.org/"} "lispinsummerprojects.org"]]]]) (defn row [& content] [:div.row [:div.offset2.span8 content]]) (defn base [chapter text exercises ecount] (list [:head (include-css "/third-party/bootstrap/css/bootstrap.css") (include-css "/third-party/codemirror-3.15/lib/codemirror.css") (include-css "/style.css") [:title "Learn Datalog Today!"]] [:body [:div.container (row [:div.textcontent text]) (row (when (> chapter 0) [:a {:href (str "/chapter/" (dec chapter))} "<< Previous chapter"]) (when (< chapter 8) [:a.pull-right {:href (str "/chapter/" (inc chapter))} "Next chapter >>"])) (row [:div.exercises {:style "margin-top: 14px"} exercises]) (row (footer))] (include-js "/third-party/jquery/jquery-1.10.1.min.js") (include-js "/third-party/codemirror-3.15/lib/codemirror.js") (include-js "/third-party/codemirror-3.15/mode/clojure/clojure.js") (include-js "/third-party/bootstrap/js/bootstrap.js") (include-js "/app.js") (javascript-tag (format "learndatalogtoday.core.init(%s, %s);" chapter ecount))])) (defn build-input [tab-n input-n input] (let [label (condp = (:type input) :query "Query:" :rule "Rules:" :value (str "Input #" input-n ":")) input-str (condp = (:type input) :query (pretty-query-string (:value input)) :rule (with-out-str (fipp/pprint (:value input))) :value (with-out-str (fipp/pprint (:value input))))] [:div.span8 [:div.row [:div.span8 [:p [:small [:strong label] (when (= :query (:type input)) [:span.pull-right "[ " [:a {:href "#" :class (str "show-ans-" tab-n)} "I give up!"] " ]"])]]]] [:div.row [:div.span8 [:textarea {:class (str "input-" tab-n)} input-str]]]])) (defn build-inputs [tab-n inputs] (map-indexed (partial build-input tab-n) inputs)) (defn build-exercise [tab-n exercise] (list [:div {:class (if (zero? tab-n) "tab-pane active" "tab-pane") :id (str "tab" tab-n)} (md/md-to-html-string (:question exercise)) [:div.row.inputs (build-inputs tab-n (:inputs exercise))] [:div.row [:div.span8 [:button.btn.btn-block {:id (str "run-query-" tab-n) :data-tab tab-n} "Run Query"]]] [:div.row [:div.span8 [:div.alerts] [:table.table.table-striped.resultset [:thead] [:tbody]]]]])) (defn build-exercises [exercises] (list [:div.tabbable [:ul.nav.nav-tabs (for [n (range (count exercises))] [:li (when (zero? n) {:class "active"}) [:a {:href (str "#tab" n) :data-toggle "tab"} [:span.label n]]])] [:div.tab-content (map-indexed build-exercise exercises)]])) (defn read-chapter [file] (with-open [r (io/reader (io/resource file))] (md/md-to-html-string (slurp r)))) (defn chapter-response [chapter-data] (let [text (-> chapter-data :text-file read-chapter) exercises (build-exercises (:exercises chapter-data)) ecount (count (:exercises chapter-data)) chapter (:chapter chapter-data)] (html5 (base chapter text exercises ecount)))) (defn toc [] (html5 [:head (include-css "/third-party/bootstrap/css/bootstrap.css") (include-css "/style.css") [:title "Learn Datalog Today!"]] [:body [:div.container (row [:div.textcontent (md/md-to-html-string (with-open [r (io/reader (io/resource "toc.md"))] (slurp r)))]) (row (footer))] (include-js "/third-party/jquery/jquery-1.10.1.min.js") (include-js "/third-party/bootstrap/js/bootstrap.js")])) ================================================ FILE: src/clj/tutorial/fns.clj ================================================ (ns tutorial.fns (:import [java.util Calendar Date])) (defn date-vector "Returns the vector [year month day]" [date] (let [c (Calendar/getInstance)] (.setTime c date) [(.get c Calendar/YEAR) (inc (.get c Calendar/MONTH)) (.get c Calendar/DAY_OF_MONTH)])) (defn age "Very crude." [^Date birthday ^Date today] (quot (- (.getTime today) (.getTime birthday)) (* 1000 60 60 24 365))) ================================================ FILE: src/cljs/learndatalogtoday/core.cljs ================================================ (ns learndatalogtoday.core (:require [learndatalogtoday.reader :refer [read-string]] [datomic-query-helpers.core :refer [normalize]] [hylla.remote :as remote] [domina :refer [by-id nodes]] [domina.css :refer [sel]] [domina.events :refer [listen! prevent-default]] [hiccups.runtime :refer [render-html]])) (defn find-clause [q] (:find (normalize q))) (defn render-row [row] (render-html [:tr (for [x row] [:td (pr-str x)])])) (defn render-error [msg] (render-html [:div.alert.alert-error [:strong "Oh snap!"] [:p msg]])) (defn render-fail [msg] (render-html [:div.alert [:p msg]])) (defn render-result [exercise query result-data] (let [alert (sel (str "#tab" exercise " .alerts")) thead (sel (str "#tab" exercise " thead")) tbody (sel (str "#tab" exercise " tbody"))] (domina/destroy-children! alert) (domina/destroy-children! thead) (domina/destroy-children! tbody) (if (= :error (:status result-data)) (domina/append! alert (render-error (:message result-data))) (do (if (= :fail (:status result-data)) (domina/append! alert (render-fail "Sorry, these results are not correct")) (domina/add-class! (sel ".active .label") "label-success")) (domina/append! thead (render-row (find-clause query))) (doseq [row (:result result-data)] (domina/append! tbody (render-row row))))))) (defn run-query-fn [chapter exercise editors] (fn [event] (try (let [input-strings (map #(.getValue %) editors) input-data (map read-string input-strings)] (remote/get (str "/query/" chapter "/" exercise) input-data #(render-result exercise (first input-data) %))) (catch js/Error e (render-result exercise nil {:status :error :message (.-message e)}))))) (defn show-ans-fn [chapter exercise editors] (fn [e] (prevent-default e) (remote/get (str "/answer/" chapter "/" exercise) nil (fn [ans] (mapv #(.setValue %1 %2) editors ans))))) (defn ^:export init [chapter ecount] (doseq [n (range ecount)] (let [button-id (str "#run-query-" n) input-class (str ".input-" n) editors (mapv #(.fromTextArea js/CodeMirror %) (nodes (sel input-class))) show-answer-class (str ".show-ans-" n)] ;; Need to refresh the codemirror editors when tab is shown. I'd ;; rather do this with domina but "shown" is a bootstrap ;; specific event. (.on (js/$ "a[data-toggle=\"tab\"]") "shown" (fn [e] (mapv #(.refresh %) editors))) (listen! (sel button-id) :click (run-query-fn chapter n editors)) (listen! (sel show-answer-class) :click (show-ans-fn chapter n editors))))) ================================================ FILE: src/cljs/learndatalogtoday/reader.cljs ================================================ ; Copyright (c) Rich Hickey. All rights reserved. ; The use and distribution terms for this software are covered by the ; Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php) ; which can be found in the file epl-v10.html at the root of this distribution. ; By using this software in any fashion, you are agreeing to be bound by ; the terms of this license. ; You must not remove this notice, or any other, from this software. ;; This is a patched version of the ClojureScript reader. It contains ;; bugfixes for CLJS-454, CLJS-564 and CLJS-565 as well as an ;; optimized StringPushbackReader. (ns learndatalogtoday.reader (:require [goog.string :as gstring])) (defprotocol PushbackReader (read-char [reader] "Returns the next char from the Reader, nil if the end of stream has been reached") (unread [reader ch] "Push back a single character on to the stream")) (deftype StringPushbackReader [s buffer ^:mutable idx] PushbackReader (read-char [reader] (if (zero? (.-length buffer)) (do (set! idx (inc idx)) (aget s idx)) (.pop buffer))) (unread [reader ch] (.push buffer ch))) (defn push-back-reader [s] "Creates a StringPushbackReader from a given string" (StringPushbackReader. s (array) -1)) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; predicates ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; (defn- ^boolean whitespace? "Checks whether a given character is whitespace" [ch] (or (gstring/isBreakingWhitespace ch) (identical? \, ch))) (defn- ^boolean numeric? "Checks whether a given character is numeric" [ch] (gstring/isNumeric ch)) (defn- ^boolean comment-prefix? "Checks whether the character begins a comment." [ch] (identical? \; ch)) (defn- ^boolean number-literal? "Checks whether the reader is at the start of a number literal" [reader initch] (or (numeric? initch) (and (or (identical? \+ initch) (identical? \- initch)) (numeric? (let [next-ch (read-char reader)] (unread reader next-ch) next-ch))))) (declare read macros dispatch-macros) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; read helpers ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; later will do e.g. line numbers... (defn reader-error [rdr & msg] (throw (js/Error. (apply str msg)))) (defn ^boolean macro-terminating? [ch] (and (not (identical? ch "#")) (not (identical? ch \')) (not (identical? ch ":")) (macros ch))) (defn read-token [rdr initch] (loop [sb (gstring/StringBuffer. initch) ch (read-char rdr)] (if (or (nil? ch) (whitespace? ch) (macro-terminating? ch)) (do (unread rdr ch) (. sb (toString))) (recur (do (.append sb ch) sb) (read-char rdr))))) (defn skip-line "Advances the reader to the end of a line. Returns the reader" [reader _] (loop [] (let [ch (read-char reader)] (if (or (identical? ch \n) (identical? ch \r) (nil? ch)) reader (recur))))) (def int-pattern (re-pattern "([-+]?)(?:(0)|([1-9][0-9]*)|0[xX]([0-9A-Fa-f]+)|0([0-7]+)|([1-9][0-9]?)[rR]([0-9A-Za-z]+)|0[0-9]+)(N)?")) (def ratio-pattern (re-pattern "([-+]?[0-9]+)/([0-9]+)")) (def float-pattern (re-pattern "([-+]?[0-9]+(\\.[0-9]*)?([eE][-+]?[0-9]+)?)(M)?")) (def symbol-pattern (re-pattern "[:]?([^0-9/].*/)?([^0-9/][^/]*)")) (defn- re-find* [re s] (let [matches (.exec re s)] (when-not (nil? matches) (if (== (alength matches) 1) (aget matches 0) matches)))) (defn- match-int [s] (let [groups (re-find* int-pattern s) group3 (aget groups 2)] (if-not (or (nil? group3) (< (alength group3) 1)) 0 (let [negate (if (identical? "-" (aget groups 1)) -1 1) a (cond (aget groups 3) (array (aget groups 3) 10) (aget groups 4) (array (aget groups 4) 16) (aget groups 5) (array (aget groups 5) 8) (aget groups 7) (array (aget groups 7) (js/parseInt (aget groups 7))) :default (array nil nil)) n (aget a 0) radix (aget a 1)] (if (nil? n) nil (* negate (js/parseInt n radix))))))) (defn- match-ratio [s] (let [groups (re-find* ratio-pattern s) numinator (aget groups 1) denominator (aget groups 2)] (/ (js/parseInt numinator) (js/parseInt denominator)))) (defn- match-float [s] (js/parseFloat s)) (defn- re-matches* [re s] (let [matches (.exec re s)] (when (and (not (nil? matches)) (identical? (aget matches 0) s)) (if (== (alength matches) 1) (aget matches 0) matches)))) (defn- match-number [s] (cond (re-matches* int-pattern s) (match-int s) (re-matches* ratio-pattern s) (match-ratio s) (re-matches* float-pattern s) (match-float s))) (defn escape-char-map [c] (cond (identical? c \t) "\t" (identical? c \r) "\r" (identical? c \n) "\n" (identical? c \\) \\ (identical? c \") \" (identical? c \b) "\b" (identical? c \f) "\f" :else nil)) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; unicode ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; (defn read-2-chars [reader] (.toString (gstring/StringBuffer. (read-char reader) (read-char reader)))) (defn read-4-chars [reader] (.toString (gstring/StringBuffer. (read-char reader) (read-char reader) (read-char reader) (read-char reader)))) (def unicode-2-pattern (re-pattern "[0-9A-Fa-f]{2}")) (def unicode-4-pattern (re-pattern "[0-9A-Fa-f]{4}")) (defn validate-unicode-escape [unicode-pattern reader escape-char unicode-str] (if (re-matches unicode-pattern unicode-str) unicode-str (reader-error reader "Unexpected unicode escape \\" escape-char unicode-str))) (defn make-unicode-char [code-str] (let [code (js/parseInt code-str 16)] (.fromCharCode js/String code))) (defn escape-char [buffer reader] (let [ch (read-char reader) mapresult (escape-char-map ch)] (if mapresult mapresult (cond (identical? ch \x) (->> (read-2-chars reader) (validate-unicode-escape unicode-2-pattern reader ch) (make-unicode-char)) (identical? ch \u) (->> (read-4-chars reader) (validate-unicode-escape unicode-4-pattern reader ch) (make-unicode-char)) (numeric? ch) (.fromCharCode js/String ch) :else (reader-error reader "Unexpected unicode escape \\" ch ))))) (defn read-past "Read until first character that doesn't match pred, returning char." [pred rdr] (loop [ch (read-char rdr)] (if (pred ch) (recur (read-char rdr)) ch))) (defn read-delimited-list [delim rdr recursive?] (loop [a (transient [])] (let [ch (read-past whitespace? rdr)] (when-not ch (reader-error rdr "EOF while reading")) (if (identical? delim ch) (persistent! a) (if-let [macrofn (macros ch)] (let [mret (macrofn rdr ch)] (recur (if (identical? mret rdr) a (conj! a mret)))) (do (unread rdr ch) (let [o (read rdr true nil recursive?)] (recur (if (identical? o rdr) a (conj! a o)))))))))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; data structure readers ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; (defn not-implemented [rdr ch] (reader-error rdr "Reader for " ch " not implemented yet")) (declare maybe-read-tagged-type) (defn read-dispatch [rdr _] (let [ch (read-char rdr) dm (dispatch-macros ch)] (if dm (dm rdr _) (if-let [obj (maybe-read-tagged-type rdr ch)] obj (reader-error rdr "No dispatch macro for " ch))))) (defn read-unmatched-delimiter [rdr ch] (reader-error rdr "Unmached delimiter " ch)) (defn read-list [rdr _] (apply list (read-delimited-list ")" rdr true))) (def read-comment skip-line) (defn read-vector [rdr _] (read-delimited-list "]" rdr true)) (defn read-map [rdr _] (let [l (read-delimited-list "}" rdr true)] (when (odd? (count l)) (reader-error rdr "Map literal must contain an even number of forms")) (apply hash-map l))) (defn read-number [reader initch] (loop [buffer (gstring/StringBuffer. initch) ch (read-char reader)] (if (or (nil? ch) (whitespace? ch) (macros ch)) (do (unread reader ch) (let [s (. buffer (toString))] (or (match-number s) (reader-error reader "Invalid number format [" s "]")))) (recur (do (.append buffer ch) buffer) (read-char reader))))) (defn read-string* [reader _] (loop [buffer (gstring/StringBuffer.) ch (read-char reader)] (cond (nil? ch) (reader-error reader "EOF while reading") (identical? "\\" ch) (recur (do (.append buffer (escape-char buffer reader)) buffer) (read-char reader)) (identical? \" ch) (. buffer (toString)) :default (recur (do (.append buffer ch) buffer) (read-char reader))))) (defn special-symbols [t not-found] (cond (identical? t "nil") nil (identical? t "true") true (identical? t "false") false :else not-found)) (defn read-symbol [reader initch] (let [token (read-token reader initch)] (if (gstring/contains token "/") (symbol (subs token 0 (.indexOf token "/")) (subs token (inc (.indexOf token "/")) (.-length token))) (special-symbols token (symbol token))))) (defn read-keyword [reader initch] (let [token (read-token reader (read-char reader)) a (re-matches* symbol-pattern token) token (aget a 0) ns (aget a 1) name (aget a 2)] (if (or (and (not (undefined? ns)) (identical? (. ns (substring (- (.-length ns) 2) (.-length ns))) ":/")) (identical? (aget name (dec (.-length name))) ":") (not (== (.indexOf token "::" 1) -1))) (reader-error reader "Invalid token: " token) (if (and (not (nil? ns)) (> (.-length ns) 0)) (keyword (.substring ns 0 (.indexOf ns "/")) name) (keyword token))))) (defn desugar-meta [f] (cond (symbol? f) {:tag f} (string? f) {:tag f} (keyword? f) {f true} :else f)) (defn wrapping-reader [sym] (fn [rdr _] (list sym (read rdr true nil true)))) (defn throwing-reader [msg] (fn [rdr _] (reader-error rdr msg))) (defn read-meta [rdr _] (let [m (desugar-meta (read rdr true nil true))] (when-not (map? m) (reader-error rdr "Metadata must be Symbol,Keyword,String or Map")) (let [o (read rdr true nil true)] (if (satisfies? IWithMeta o) (with-meta o (merge (meta o) m)) (reader-error rdr "Metadata can only be applied to IWithMetas"))))) (defn read-set [rdr _] (set (read-delimited-list "}" rdr true))) (defn read-regex [rdr ch] (-> (read-string* rdr ch) re-pattern)) (defn read-discard [rdr _] (read rdr true nil true) rdr) (defn macros [c] (cond (identical? c \") read-string* (identical? c \:) read-keyword (identical? c \;) not-implemented ;; never hit this (identical? c \') (wrapping-reader 'quote) (identical? c \@) (wrapping-reader 'deref) (identical? c \^) read-meta (identical? c \`) not-implemented (identical? c \~) not-implemented (identical? c \() read-list (identical? c \)) read-unmatched-delimiter (identical? c \[) read-vector (identical? c \]) read-unmatched-delimiter (identical? c \{) read-map (identical? c \}) read-unmatched-delimiter (identical? c \\) read-char (identical? c \#) read-dispatch :else nil)) ;; omitted by design: var reader, eval reader (defn dispatch-macros [s] (cond (identical? s "{") read-set (identical? s "<") (throwing-reader "Unreadable form") (identical? s "\"") read-regex (identical? s"!") read-comment (identical? s "_") read-discard :else nil)) (defn read "Reads the first object from a PushbackReader. Returns the object read. If EOF, throws if eof-is-error is true. Otherwise returns sentinel." [reader eof-is-error sentinel is-recursive] (let [ch (read-char reader)] (cond (nil? ch) (if eof-is-error (reader-error reader "EOF while reading") sentinel) (whitespace? ch) (recur reader eof-is-error sentinel is-recursive) (comment-prefix? ch) (recur (read-comment reader ch) eof-is-error sentinel is-recursive) :else (let [f (macros ch) res (cond f (f reader ch) (number-literal? reader ch) (read-number reader ch) :else (read-symbol reader ch))] (if (identical? res reader) (recur reader eof-is-error sentinel is-recursive) res))))) (defn read-string "Reads one object from the string s" [s] (let [r (push-back-reader s)] (read r true nil false))) ;; read instances (defn ^:private zero-fill-right-and-truncate [s width] (cond (= width (count s)) s (< width (count s)) (subs s 0 width) :else (loop [b (gstring/StringBuffer. s)] (if (< (.getLength b) width) (recur (.append b "0")) (.toString b))))) (defn ^:private divisible? [num div] (zero? (mod num div))) (defn ^:private indivisible? [num div] (not (divisible? num div))) (defn ^:private leap-year? [year] (and (divisible? year 4) (or (indivisible? year 100) (divisible? year 400)))) (def ^:private days-in-month (let [dim-norm [nil 31 28 31 30 31 30 31 31 30 31 30 31] dim-leap [nil 31 29 31 30 31 30 31 31 30 31 30 31]] (fn [month leap-year?] (get (if leap-year? dim-leap dim-norm) month)))) (def ^:private timestamp-regex #"(\d\d\d\d)(?:-(\d\d)(?:-(\d\d)(?:[T](\d\d)(?::(\d\d)(?::(\d\d)(?:[.](\d+))?)?)?)?)?)?(?:[Z]|([-+])(\d\d):(\d\d))?") (defn ^:private parse-int [s] (let [n (js/parseInt s)] (if-not (js/isNaN n) n))) (defn ^:private check [low n high msg] (when-not (<= low n high) (reader-error nil (str msg " Failed: " low "<=" n "<=" high))) n) (defn parse-and-validate-timestamp [s] (let [[_ years months days hours minutes seconds fraction offset-sign offset-hours offset-minutes :as v] (re-matches timestamp-regex s)] (if-not v (reader-error nil (str "Unrecognized date/time syntax: " s)) (let [years (parse-int years) months (or (parse-int months) 1) days (or (parse-int days) 1) hours (or (parse-int hours) 0) minutes (or (parse-int minutes) 0) seconds (or (parse-int seconds) 0) fraction (or (parse-int (zero-fill-right-and-truncate fraction 3)) 0) offset-sign (if (= offset-sign "-") -1 1) offset-hours (or (parse-int offset-hours) 0) offset-minutes (or (parse-int offset-minutes) 0) offset (* offset-sign (+ (* offset-hours 60) offset-minutes))] [years (check 1 months 12 "timestamp month field must be in range 1..12") (check 1 days (days-in-month months (leap-year? years)) "timestamp day field must be in range 1..last day in month") (check 0 hours 23 "timestamp hour field must be in range 0..23") (check 0 minutes 59 "timestamp minute field must be in range 0..59") (check 0 seconds (if (= minutes 59) 60 59) "timestamp second field must be in range 0..60") (check 0 fraction 999 "timestamp millisecond field must be in range 0..999") offset])))) (defn parse-timestamp [ts] (if-let [[years months days hours minutes seconds ms offset] (parse-and-validate-timestamp ts)] (js/Date. (- (.UTC js/Date years (dec months) days hours minutes seconds ms) (* offset 60 1000))) (reader-error nil (str "Unrecognized date/time syntax: " ts)))) (defn ^:private read-date [s] (if (string? s) (parse-timestamp s) (reader-error nil "Instance literal expects a string for its timestamp."))) (defn ^:private read-queue [elems] (if (vector? elems) (into cljs.core.PersistentQueue/EMPTY elems) (reader-error nil "Queue literal expects a vector for its elements."))) (defn ^:private read-uuid [uuid] (if (string? uuid) (UUID. uuid) (reader-error nil "UUID literal expects a string as its representation."))) (def *tag-table* (atom {"inst" read-date "uuid" read-uuid "queue" read-queue})) (def *default-data-reader-fn* (atom nil)) (defn maybe-read-tagged-type [rdr initch] (let [tag (read-symbol rdr initch) pfn (get @*tag-table* (str tag)) dfn @*default-data-reader-fn*] (cond pfn (pfn (read rdr true nil false)) dfn (dfn tag (read rdr true nil false)) :else (reader-error rdr "Could not find tag parser for " (str tag) " in " (pr-str (keys @*tag-table*)))))) (defn register-tag-parser! [tag f] (let [tag (str tag) old-parser (get @*tag-table* tag)] (swap! *tag-table* assoc tag f) old-parser)) (defn deregister-tag-parser! [tag] (let [tag (str tag) old-parser (get @*tag-table* tag)] (swap! *tag-table* dissoc tag) old-parser)) (defn register-default-tag-parser! [f] (let [old-parser @*default-data-reader-fn*] (swap! *default-data-reader-fn* (fn [_] f)) old-parser)) (defn deregister-default-tag-parser! [] (let [old-parser @*default-data-reader-fn*] (swap! *default-data-reader-fn* (fn [_] nil)) old-parser))