Showing preview only (521K chars total). Download the full file or copy to clipboard to get everything.
Repository: elastic/elasticsearch-formal-models
Branch: master
Commit: ca30663506a7
Files: 28
Total size: 503.0 KB
Directory structure:
gitextract_nf10lb83/
├── .gitignore
├── LICENSE
├── README.md
├── ReplicaEngine/
│ └── tla/
│ ├── ReplicaEngine.tla
│ └── ReplicaEngine.toolbox/
│ ├── .project
│ ├── .settings/
│ │ └── org.lamport.tla.toolbox.prefs
│ └── ReplicaEngine___model.launch
├── Storage/
│ └── tla/
│ ├── Storage.tla
│ └── Storage.toolbox/
│ └── Storage___model.launch
├── ZenWithTerms/
│ └── tla/
│ ├── ZenWithTerms.tla
│ └── ZenWithTerms.toolbox/
│ ├── .project
│ ├── .settings/
│ │ └── org.lamport.tla.toolbox.prefs
│ └── ZenWithTerms___model.launch
├── cluster/
│ ├── isabelle/
│ │ ├── Implementation.thy
│ │ ├── Monadic.thy
│ │ ├── OneSlot.thy
│ │ ├── Preliminaries.thy
│ │ ├── ROOT
│ │ ├── Zen.thy
│ │ └── document/
│ │ └── root.tex
│ └── tla/
│ ├── consensus.tla
│ └── consensus.toolbox/
│ ├── .project
│ ├── .settings/
│ │ └── org.lamport.tla.toolbox.prefs
│ └── consensus___model.launch
└── data/
└── tla/
├── replication.tla
└── replication.toolbox/
├── .project
├── .settings/
│ └── org.lamport.tla.toolbox.prefs
└── replication___model.launch
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
**/.DS_Store
**/tla/*.toolbox/model
**/tla/*.toolbox/*aux
**/tla/*.toolbox/*.log
**/tla/*.toolbox/*.pdf
**/tla/*.toolbox/*.tex
**/tla/*.toolbox/*___model_SnapShot*.launch
**/tla/*.toolbox/**/*.tla
**/tla/*.toolbox/**/*.out
**/tla/*.toolbox/**/MC.cfg
**/tla/*.pdf
**/tla/*.old
**/*~
cluster/isabelle/output
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: README.md
================================================
# Formal models of core Elasticsearch algorithms
This repository contains formal models of core [Elasticsearch](https://github.com/elastic/elasticsearch) algorithms and is directly related to implementation efforts around [data replication](https://github.com/elastic/elasticsearch/issues/10708) and [cluster coordination](https://github.com/elastic/elasticsearch/issues/32006). The models in this repository might represent past, current and future designs of Elasticsearch and can differ to their implementations in substantial ways. The formal models mainly serve to illustrate some of the high-level concepts and help to validate resiliency-related aspects.
## Models
### Cluster coordination model
The cluster coordination TLA+ model ensures the consistency of cluster state updates and represents the core [cluster coordination](https://github.com/elastic/elasticsearch/issues/32006) and metadata replication algorithm implemented in Elasticsearch 7.0. It consists of two files:
- [TLA+ specification](ZenWithTerms/tla/ZenWithTerms.tla) which has a [direct one-to-one implementation in Elasticsearch](https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/cluster/coordination/CoordinationState.java)
- [TLC model checking configuration](ZenWithTerms/tla/ZenWithTerms.toolbox/ZenWithTerms___model.launch)
### Data replication model
The data replication TLA+ model describes the Elasticsearch [sequence number](https://github.com/elastic/elasticsearch/issues/10708) based data replication approach, implemented since Elasticsearch 6.0, which consists of two files:
- [TLA+ specification](data/tla/replication.tla)
- [TLC model checking configuration](data/tla/replication.toolbox/replication___model.launch)
### Replica engine
A TLA+ model of how the
[engine](https://github.com/elastic/elasticsearch/blob/00fd73acc4a2991f96438f8c1948016c5b9eefb2/server/src/main/java/org/elasticsearch/index/engine/InternalEngine.java)
handles replication requests.
- [TLA+ specification](ReplicaEngine/tla/ReplicaEngine.tla)
- [TLC model checking configuration](ReplicaEngine/tla/ReplicaEngine.toolbox/ReplicaEngine___model.launch)
### Alternative cluster coordination model
The alternative cluster coordination TLA+ model consists of two files:
- [TLA+ specification](cluster/tla/consensus.tla)
- [TLC model checking configuration](cluster/tla/consensus.toolbox/consensus___model.launch)
The alternative cluster consensus Isabelle model consists of the following theories:
- [Basic definitions](cluster/isabelle/Preliminaries.thy)
- [An implementation in functional style](cluster/isabelle/Implementation.thy)
- [An implementation in monadic style, along with a proof it's equivalent to the previous](cluster/isabelle/Monadic.thy)
- [The proof that each slot is consistent, based on Lamport's Synod algorithm](cluster/isabelle/OneSlot.thy)
- [The proof that the implementation ensures consistency](cluster/isabelle/Zen.thy)
## How to edit/run TLA+:
- Install the [TLA Toolbox](http://research.microsoft.com/en-us/um/people/lamport/tla/toolbox.html)
- If on Mac OS, [move the downloaded app to the Applications folder first](https://groups.google.com/forum/#!topic/tlaplus/bL04c6BiYxo)
- Read some [documentation](http://research.microsoft.com/en-us/um/people/lamport/tla/book.html)
How to run the model checker in headless mode:
- Download [tla2tools.jar](http://research.microsoft.com/en-us/um/people/lamport/tla/tools.html)
- Run the model checker once in TLA+ Toolbox on desktop (can be aborted once started). This generates the folder `elasticsearch.toolbox/model/` that contains all model files that are required to run the model checker in headless mode.
- Copy the above folder and `tla2tools.jar` to the server running in headless mode.
- `cd` to the folder and run `java -Xmx30G -cp ../tla2tools.jar tlc2.TLC MC -deadlock -workers 12`. The setting `-Xmx30G` denotes the amount of memory to allocate to the model checker and `-workers 12` the number of worker threads (should be equal to the number of cores on machine). The setting `-deadlock` ensures that TLC explores the full reachable state space, not searching for deadlocks.
================================================
FILE: ReplicaEngine/tla/ReplicaEngine.tla
================================================
-------------------------- MODULE ReplicaEngine --------------------------
EXTENDS Naturals, FiniteSets, Sequences, TLC
(* Actions on the Lucene index *)
CONSTANTS Lucene_addDocuments, Lucene_updateDocuments, Lucene_deleteDocuments
CONSTANTS ADD, RETRY_ADD, UPDATE, DELETE, NULL
CONSTANTS DocContent
CONSTANTS DocAutoIdTimestamp
CONSTANTS DuplicationLimit
(* We model the activity of a single document, since distinct documents
(according to their IDs) are independent. Also each indexing operation
occurs under a lock for that document ID, so there is not much concurrency
to consider. *)
(* The set of individual requests that can occur on the document *)
Request(request_count)
(* ADD: An optimised append-only write can only occur as the first operation
on the document ID in seqno order. Any subsequent attempts to ADD the
document have the retry flag set and modelled as a RETRY_ADD. Other operations
on the document are also possible. *)
== [type : {ADD}, seqno : {1}, content : DocContent, autoIdTimeStamp : {DocAutoIdTimestamp}]
(* RETRY_ADD: A retry of a write that does involve an internally-generated
document ID. *)
\cup [type : {RETRY_ADD}, seqno : 1..request_count, content : DocContent, autoIdTimeStamp : {DocAutoIdTimestamp}]
(* UPDATE: A write that does not involve an internally-generated document ID. *)
\cup [type : {UPDATE}, seqno : 1..request_count, content : DocContent]
(* DELETE *)
\cup [type : {DELETE}, seqno : 1..request_count]
(* The set of sets of requests, which have distinct seqnos *)
RequestSet(request_count)
== { rs \in SUBSET Request(request_count):
/\ Cardinality(rs) = request_count
/\ Cardinality({r.seqno : r \in rs}) = request_count
/\ (* Also ADDs and RETRY_ADDs should have the same content *)
Cardinality({r.content: r \in { r \in rs: r.type \in {ADD, RETRY_ADD}}}) <= 1
}
(* Apply a set of operations to a document in seqno order *)
RECURSIVE ApplyOps(_, _, _)
ApplyOps(requests, nextSeqno, currentContent)
== IF \A r \in requests: r.seqno < nextSeqno
THEN currentContent
ELSE LET r == CHOOSE r \in requests: r.seqno = nextSeqno
IN IF r \in requests /\ r.seqno = nextSeqno
THEN ApplyOps(requests, nextSeqno + 1,
CASE r.type = DELETE -> NULL
[] r.type = ADD -> r.content
[] r.type = RETRY_ADD -> r.content
[] r.type = UPDATE -> r.content)
ELSE Assert(FALSE, "Bad sequence")
(* Calculate the final doc by applying all the requests in order *)
FinalDoc(requests) == ApplyOps(requests, 1, NULL)
(* Apply each the operation in the Lucene buffer, rejecting an
addDocuments when there is already a document present as this
would lead to duplication. *)
RECURSIVE ApplyBufferedOperations(_, _)
ApplyBufferedOperations(buffer, origDoc)
== IF buffer = <<>>
THEN origDoc
ELSE LET nextOp == Head(buffer)
IN ApplyBufferedOperations(Tail(buffer),
CASE nextOp.type = Lucene_deleteDocuments -> NULL
[] \/ nextOp.type = Lucene_updateDocuments
\/ /\ nextOp.type = Lucene_addDocuments
/\ origDoc = NULL -> [content |-> nextOp.content, seqno |-> nextOp.seqno]
[] OTHER -> Assert(FALSE, "Error: Lucene_addDocuments when origDoc /= NULL"))
Max(a,b) == IF a <= b THEN b ELSE a
(* --algorithm basic
variables
request_count \in 1..4,
replication_requests \in RequestSet(request_count),
expected_doc = FinalDoc(replication_requests),
versionMap_needsSafeAccess = FALSE,
versionMap_isUnsafe = FALSE,
versionMap_entry = NULL,
(* Other concurrent activity can flag that the version map needs to be safely accessed *)
process SafeAccessEnablerProcess = "SafeAccessEnabler"
begin
SafeAccessEnablerLoop:
while pc["Consumer"] /= "Done" do
versionMap_needsSafeAccess := (versionMap_needsSafeAccess = FALSE);
(* Technically the only way this can go back to FALSE is via a refresh, but
we should not need this fact, so model both kinds of change. *)
end while;
end process;
(* Other concurrent activity can make the version map become unsafe, if safe access mode is disabled *)
process UnsafePutterProcess = "UnsafePutter"
begin
UnsafePutterLoop:
while pc["Consumer"] /= "Done" do
await versionMap_needsSafeAccess = FALSE;
versionMap_isUnsafe := TRUE;
end while;
end process;
(* Other concurrent activity can increase the maxUnsafeAutoIdTimestamp *)
process MaxUnsafeAutoIdTimestampIncreaserProcess = "MaxUnsafeAutoIdTimestampIncreaser"
begin
MaxUnsafeAutoIdTimestampIncreaserLoop:
while pc["Consumer"] /= "Done" do
with newTimestamp \in {DocAutoIdTimestamp - 1, DocAutoIdTimestamp, DocAutoIdTimestamp + 1} do
await maxUnsafeAutoIdTimestamp < newTimestamp;
maxUnsafeAutoIdTimestamp := newTimestamp;
end with;
end while;
end process;
(* Lucene refreshes can happen at any time *)
process LuceneProcess = "ReplicaLucene"
variables
lucene_document = NULL,
lucene_buffer = <<>>,
begin
LuceneLoop:
while pc["Consumer"] /= "Done" \/ lucene_buffer /= <<>> do
lucene_document := ApplyBufferedOperations(lucene_buffer, lucene_document);
lucene_buffer := <<>>;
(* TODO Model the inner structure of the version map so this refresh can be
broken into the individual steps that occur concurrently with ongoing indexing. *)
versionMap_isUnsafe := FALSE;
versionMap_needsSafeAccess := FALSE;
if versionMap_entry /= NULL
then
if versionMap_entry.type = UPDATE
then
versionMap_entry := NULL;
else
assert versionMap_entry.type = DELETE;
versionMap_entry := [ versionMap_entry EXCEPT !.flushed = TRUE ];
end if;
end if;
end while;
end process;
(* Flushed deletes expire after a time and are cleaned up *)
process DeleteCollectorProcess = "DeleteCollector"
begin
DeleteCollectorLoop:
while pc["Consumer"] /= "Done" do
await /\ versionMap_entry /= NULL
/\ versionMap_entry.type = DELETE
/\ versionMap_entry.seqno <= localCheckPoint \* PR #28790
/\ versionMap_entry.flushed = TRUE;
versionMap_entry := NULL;
end while;
end process;
(* Local checkpoint advances as each operation is marked as completed *)
process LocalCheckpointTrackerProcess = "LocalCheckpointTracker"
variables
localCheckPoint = 0,
completedSeqnos = {}
begin
LocalCheckpointTrackerLoop:
while pc["Consumer"] /= "Done" do
await localCheckPoint + 1 \in completedSeqnos;
localCheckPoint := localCheckPoint + 1;
end while;
end process
process UnsafeSeqnoIncreaserProcess = "UnsafeSeqnoIncreaserProcess"
variables
maxSeqNoOfNonAppendOnlyOperations = 0,
begin
UnsafeSeqnoIncreaserProcessLoop:
while pc["Consumer"] /= "Done" /\ maxSeqNoOfNonAppendOnlyOperations < request_count + 1 do
maxSeqNoOfNonAppendOnlyOperations := maxSeqNoOfNonAppendOnlyOperations + 1;
end while;
end process
(* The process that consumes replication requests for a particular document ID, which
are processed in series because of the lock in the version map. *)
process ConsumerProcess = "Consumer"
variables
duplicationCount = 0,
maxUnsafeAutoIdTimestamp \in {0, DocAutoIdTimestamp - 1, DocAutoIdTimestamp, DocAutoIdTimestamp + 1},
req, plan,
deleteFromLucene, currentlyDeleted,
currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene,
begin
ConsumerLoop:
while replication_requests /= {} do
with replication_request \in replication_requests do
if replication_request.type = ADD
then
(* Never see two ADDs - if duplicated, one of them is a RETRY_ADD *)
either
(* Process ADD without duplication *)
replication_requests := replication_requests \ {replication_request};
req := replication_request;
or
await duplicationCount < DuplicationLimit;
duplicationCount := duplicationCount + 1;
(* Process ADD and leave a duplicate RETRY_ADD for later *)
replication_requests := (replication_requests \ {replication_request})
\cup {[replication_request EXCEPT !.type = RETRY_ADD]};
req := replication_request;
or
await duplicationCount < DuplicationLimit;
duplicationCount := duplicationCount + 1;
(* Process duplicate RETRY_ADD and leave the original ADD *)
req := [replication_request EXCEPT !.type = RETRY_ADD];
end either;
else
req := replication_request;
either
await duplicationCount < DuplicationLimit;
duplicationCount := duplicationCount + 1;
or
replication_requests := replication_requests \ {replication_request};
end either;
end if;
end with;
if req.type = DELETE
then
versionMap_needsSafeAccess := TRUE;
(* planDeletionAsNonPrimary *)
maxSeqNoOfNonAppendOnlyOperations := Max(maxSeqNoOfNonAppendOnlyOperations, req.seqno);
if req.seqno <= localCheckPoint
then
(* OP_STALE_OR_EQUAL *)
plan := "processButSkipLucene";
deleteFromLucene := FALSE;
currentlyDeleted := FALSE;
else
if versionMap_isUnsafe
then
(* Perform a Lucene refresh *)
AwaitRefreshOnDelete: \* Label here to allow for other concurrent activity
await lucene_buffer = <<>>;
versionMap_needsSafeAccess := TRUE;
end if;
compareDeleteOpToLuceneDocBasedOnSeqNo: \* Label needed because of AwaitRefreshOnDelete label
if versionMap_entry /= NULL
then
(* Doc is in version map *)
if req.seqno > versionMap_entry.seqno
then
(* OP_NEWER *)
plan := "processNormally";
deleteFromLucene := TRUE;
currentlyDeleted := FALSE;
else
(* OP_STALE_OR_EQUAL *)
plan := "processButSkipLucene";
deleteFromLucene := FALSE;
currentlyDeleted := FALSE;
end if;
else
(* Doc is not in version map - check Lucene *)
if lucene_document = NULL
then
(* LUCENE_DOC_NOT_FOUND *)
plan := "processNormallyExceptNotFound";
deleteFromLucene := TRUE;
currentlyDeleted := TRUE;
else
if req.seqno > lucene_document.seqno
then
(* OP_NEWER *)
plan := "processNormally";
deleteFromLucene := TRUE;
currentlyDeleted := FALSE;
else
(* OP_STALE_OR_EQUAL *)
plan := "processButSkipLucene";
deleteFromLucene := FALSE;
currentlyDeleted := FALSE;
end if;
end if;
end if;
end if;
ExecuteDeletePlan: \* Label needed because of AwaitRefreshOnDelete label
if deleteFromLucene
then
if currentlyDeleted = FALSE
then
lucene_buffer := Append(lucene_buffer, [ type |-> Lucene_deleteDocuments ]);
end if;
versionMap_entry := [ type |-> DELETE, seqno |-> req.seqno, flushed |-> FALSE ];
end if;
completedSeqnos := completedSeqnos \cup {req.seqno};
else
(* planIndexingAsNonPrimary *)
(* A RETRY_ADD has canOptimiseAddDocument = TRUE and
mayHaveBeenIndexedBefore = TRUE so is planned normally,
but also updates maxUnsafeAutoIdTimestamp within
mayHaveBeenIndexedBefore() *)
if req.type = RETRY_ADD
then
maxUnsafeAutoIdTimestamp := Max(maxUnsafeAutoIdTimestamp, req.autoIdTimeStamp);
end if;
(* An ADD can be optimized if mayHaveBeenIndexedBefore = FALSE
which is calculated by comparing timestamps. *)
if /\ req.type = ADD
/\ maxUnsafeAutoIdTimestamp < req.autoIdTimeStamp
/\ maxSeqNoOfNonAppendOnlyOperations < req.seqno \* PR #28787
then
plan := "optimisedAppendOnly";
currentNotFoundOrDeleted := TRUE;
useLuceneUpdateDocument := FALSE;
indexIntoLucene := TRUE;
else
if req.type \notin {ADD, RETRY_ADD}
then
maxSeqNoOfNonAppendOnlyOperations := Max(maxSeqNoOfNonAppendOnlyOperations, req.seqno);
end if;
(* All other operations are planned normally *)
versionMap_needsSafeAccess := TRUE;
if req.seqno <= localCheckPoint
then
(* OP_STALE_OR_EQUAL *)
plan := "processButSkipLucene";
currentNotFoundOrDeleted := FALSE;
useLuceneUpdateDocument := FALSE;
indexIntoLucene := FALSE;
else
if versionMap_isUnsafe
then
(* Perform a Lucene refresh *)
AwaitRefreshOnIndex: \* Label here to allow for other concurrent activity
await lucene_buffer = <<>>;
versionMap_needsSafeAccess := TRUE;
end if;
compareIndexOpToLuceneDocBasedOnSeqNo: \* Label needed because of AwaitRefreshOnIndex label
if req.seqno <= localCheckPoint \* PR #29276
then \* PR #29276
(* OP_STALE_OR_EQUAL *) \* PR #29276
plan := "processButSkipLucene"; \* PR #29276
currentNotFoundOrDeleted := FALSE; \* PR #29276
useLuceneUpdateDocument := FALSE; \* PR #29276
indexIntoLucene := FALSE; \* PR #29276
elsif versionMap_entry /= NULL
then
(* Doc is in version map *)
if req.seqno > versionMap_entry.seqno
then
(* OP_NEWER *)
plan := "processNormally";
currentNotFoundOrDeleted := FALSE;
useLuceneUpdateDocument := TRUE;
indexIntoLucene := TRUE;
else
(* OP_STALE_OR_EQUAL *)
plan := "processButSkipLucene";
currentNotFoundOrDeleted := FALSE;
useLuceneUpdateDocument := FALSE;
indexIntoLucene := FALSE;
end if;
else
(* Doc is not in version map - check Lucene *)
if lucene_document = NULL
then
(* LUCENE_DOC_NOT_FOUND *)
plan := "processNormallyExceptNotFound";
currentNotFoundOrDeleted := TRUE;
useLuceneUpdateDocument := FALSE;
indexIntoLucene := TRUE;
else
if req.seqno > lucene_document.seqno
then
(* OP_NEWER *)
plan := "processNormally";
currentNotFoundOrDeleted := FALSE;
useLuceneUpdateDocument := TRUE;
indexIntoLucene := TRUE;
else
(* OP_STALE_OR_EQUAL *)
plan := "processButSkipLucene";
currentNotFoundOrDeleted := FALSE;
useLuceneUpdateDocument := FALSE;
indexIntoLucene := FALSE;
end if;
end if;
end if;
end if;
end if;
(* planIndexingAsNonPrimary finished - now time to execute the plan *)
ExecuteIndexPlan: \* Label needed because of AwaitRefreshOnIndex label
if indexIntoLucene
then
lucene_buffer := Append(lucene_buffer,
[ type |-> IF useLuceneUpdateDocument THEN Lucene_updateDocuments ELSE Lucene_addDocuments
, seqno |-> req.seqno
, content |-> req.content
]);
if versionMap_needsSafeAccess
then
versionMap_entry := [ type |-> UPDATE, seqno |-> req.seqno ];
else
versionMap_isUnsafe := TRUE;
if /\ versionMap_entry /= NULL
/\ versionMap_entry.type = DELETE
/\ versionMap_entry.seqno < req.seqno
then
versionMap_entry := NULL; \* Desync bug #3 (no PR number yet)
end if;
end if;
end if;
completedSeqnos := completedSeqnos \cup {req.seqno}
end if;
end while;
end process
end algorithm
*)
\* BEGIN TRANSLATION
CONSTANT defaultInitValue
VARIABLES request_count, replication_requests, expected_doc,
versionMap_needsSafeAccess, versionMap_isUnsafe, versionMap_entry,
pc, lucene_document, lucene_buffer, localCheckPoint,
completedSeqnos, maxSeqNoOfNonAppendOnlyOperations,
duplicationCount, maxUnsafeAutoIdTimestamp, req, plan,
deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted,
useLuceneUpdateDocument, indexIntoLucene
vars == << request_count, replication_requests, expected_doc,
versionMap_needsSafeAccess, versionMap_isUnsafe, versionMap_entry,
pc, lucene_document, lucene_buffer, localCheckPoint,
completedSeqnos, maxSeqNoOfNonAppendOnlyOperations,
duplicationCount, maxUnsafeAutoIdTimestamp, req, plan,
deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted,
useLuceneUpdateDocument, indexIntoLucene >>
ProcSet == {"SafeAccessEnabler"} \cup {"UnsafePutter"} \cup {"MaxUnsafeAutoIdTimestampIncreaser"} \cup {"ReplicaLucene"} \cup {"DeleteCollector"} \cup {"LocalCheckpointTracker"} \cup {"UnsafeSeqnoIncreaserProcess"} \cup {"Consumer"}
Init == (* Global variables *)
/\ request_count \in 1..4
/\ replication_requests \in RequestSet(request_count)
/\ expected_doc = FinalDoc(replication_requests)
/\ versionMap_needsSafeAccess = FALSE
/\ versionMap_isUnsafe = FALSE
/\ versionMap_entry = NULL
(* Process LuceneProcess *)
/\ lucene_document = NULL
/\ lucene_buffer = <<>>
(* Process LocalCheckpointTrackerProcess *)
/\ localCheckPoint = 0
/\ completedSeqnos = {}
(* Process UnsafeSeqnoIncreaserProcess *)
/\ maxSeqNoOfNonAppendOnlyOperations = 0
(* Process ConsumerProcess *)
/\ duplicationCount = 0
/\ maxUnsafeAutoIdTimestamp \in {0, DocAutoIdTimestamp - 1, DocAutoIdTimestamp, DocAutoIdTimestamp + 1}
/\ req = defaultInitValue
/\ plan = defaultInitValue
/\ deleteFromLucene = defaultInitValue
/\ currentlyDeleted = defaultInitValue
/\ currentNotFoundOrDeleted = defaultInitValue
/\ useLuceneUpdateDocument = defaultInitValue
/\ indexIntoLucene = defaultInitValue
/\ pc = [self \in ProcSet |-> CASE self = "SafeAccessEnabler" -> "SafeAccessEnablerLoop"
[] self = "UnsafePutter" -> "UnsafePutterLoop"
[] self = "MaxUnsafeAutoIdTimestampIncreaser" -> "MaxUnsafeAutoIdTimestampIncreaserLoop"
[] self = "ReplicaLucene" -> "LuceneLoop"
[] self = "DeleteCollector" -> "DeleteCollectorLoop"
[] self = "LocalCheckpointTracker" -> "LocalCheckpointTrackerLoop"
[] self = "UnsafeSeqnoIncreaserProcess" -> "UnsafeSeqnoIncreaserProcessLoop"
[] self = "Consumer" -> "ConsumerLoop"]
SafeAccessEnablerLoop == /\ pc["SafeAccessEnabler"] = "SafeAccessEnablerLoop"
/\ IF pc["Consumer"] /= "Done"
THEN /\ versionMap_needsSafeAccess' = (versionMap_needsSafeAccess = FALSE)
/\ pc' = [pc EXCEPT !["SafeAccessEnabler"] = "SafeAccessEnablerLoop"]
ELSE /\ pc' = [pc EXCEPT !["SafeAccessEnabler"] = "Done"]
/\ UNCHANGED versionMap_needsSafeAccess
/\ UNCHANGED << request_count, replication_requests,
expected_doc, versionMap_isUnsafe,
versionMap_entry, lucene_document,
lucene_buffer, localCheckPoint,
completedSeqnos,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount,
maxUnsafeAutoIdTimestamp, req, plan,
deleteFromLucene, currentlyDeleted,
currentNotFoundOrDeleted,
useLuceneUpdateDocument,
indexIntoLucene >>
SafeAccessEnablerProcess == SafeAccessEnablerLoop
UnsafePutterLoop == /\ pc["UnsafePutter"] = "UnsafePutterLoop"
/\ IF pc["Consumer"] /= "Done"
THEN /\ versionMap_needsSafeAccess = FALSE
/\ versionMap_isUnsafe' = TRUE
/\ pc' = [pc EXCEPT !["UnsafePutter"] = "UnsafePutterLoop"]
ELSE /\ pc' = [pc EXCEPT !["UnsafePutter"] = "Done"]
/\ UNCHANGED versionMap_isUnsafe
/\ UNCHANGED << request_count, replication_requests,
expected_doc, versionMap_needsSafeAccess,
versionMap_entry, lucene_document,
lucene_buffer, localCheckPoint,
completedSeqnos,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount, maxUnsafeAutoIdTimestamp,
req, plan, deleteFromLucene,
currentlyDeleted, currentNotFoundOrDeleted,
useLuceneUpdateDocument, indexIntoLucene >>
UnsafePutterProcess == UnsafePutterLoop
MaxUnsafeAutoIdTimestampIncreaserLoop == /\ pc["MaxUnsafeAutoIdTimestampIncreaser"] = "MaxUnsafeAutoIdTimestampIncreaserLoop"
/\ IF pc["Consumer"] /= "Done"
THEN /\ \E newTimestamp \in {DocAutoIdTimestamp - 1, DocAutoIdTimestamp, DocAutoIdTimestamp + 1}:
/\ maxUnsafeAutoIdTimestamp < newTimestamp
/\ maxUnsafeAutoIdTimestamp' = newTimestamp
/\ pc' = [pc EXCEPT !["MaxUnsafeAutoIdTimestampIncreaser"] = "MaxUnsafeAutoIdTimestampIncreaserLoop"]
ELSE /\ pc' = [pc EXCEPT !["MaxUnsafeAutoIdTimestampIncreaser"] = "Done"]
/\ UNCHANGED maxUnsafeAutoIdTimestamp
/\ UNCHANGED << request_count,
replication_requests,
expected_doc,
versionMap_needsSafeAccess,
versionMap_isUnsafe,
versionMap_entry,
lucene_document,
lucene_buffer,
localCheckPoint,
completedSeqnos,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount, req,
plan,
deleteFromLucene,
currentlyDeleted,
currentNotFoundOrDeleted,
useLuceneUpdateDocument,
indexIntoLucene >>
MaxUnsafeAutoIdTimestampIncreaserProcess == MaxUnsafeAutoIdTimestampIncreaserLoop
LuceneLoop == /\ pc["ReplicaLucene"] = "LuceneLoop"
/\ IF pc["Consumer"] /= "Done" \/ lucene_buffer /= <<>>
THEN /\ lucene_document' = ApplyBufferedOperations(lucene_buffer, lucene_document)
/\ lucene_buffer' = <<>>
/\ versionMap_isUnsafe' = FALSE
/\ versionMap_needsSafeAccess' = FALSE
/\ IF versionMap_entry /= NULL
THEN /\ IF versionMap_entry.type = UPDATE
THEN /\ versionMap_entry' = NULL
ELSE /\ Assert(versionMap_entry.type = DELETE,
"Failure of assertion at line 147, column 17.")
/\ versionMap_entry' = [ versionMap_entry EXCEPT !.flushed = TRUE ]
ELSE /\ TRUE
/\ UNCHANGED versionMap_entry
/\ pc' = [pc EXCEPT !["ReplicaLucene"] = "LuceneLoop"]
ELSE /\ pc' = [pc EXCEPT !["ReplicaLucene"] = "Done"]
/\ UNCHANGED << versionMap_needsSafeAccess,
versionMap_isUnsafe, versionMap_entry,
lucene_document, lucene_buffer >>
/\ UNCHANGED << request_count, replication_requests,
expected_doc, localCheckPoint, completedSeqnos,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount, maxUnsafeAutoIdTimestamp, req,
plan, deleteFromLucene, currentlyDeleted,
currentNotFoundOrDeleted,
useLuceneUpdateDocument, indexIntoLucene >>
LuceneProcess == LuceneLoop
DeleteCollectorLoop == /\ pc["DeleteCollector"] = "DeleteCollectorLoop"
/\ IF pc["Consumer"] /= "Done"
THEN /\ /\ versionMap_entry /= NULL
/\ versionMap_entry.type = DELETE
/\ versionMap_entry.seqno <= localCheckPoint
/\ versionMap_entry.flushed = TRUE
/\ versionMap_entry' = NULL
/\ pc' = [pc EXCEPT !["DeleteCollector"] = "DeleteCollectorLoop"]
ELSE /\ pc' = [pc EXCEPT !["DeleteCollector"] = "Done"]
/\ UNCHANGED versionMap_entry
/\ UNCHANGED << request_count, replication_requests,
expected_doc,
versionMap_needsSafeAccess,
versionMap_isUnsafe, lucene_document,
lucene_buffer, localCheckPoint,
completedSeqnos,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount,
maxUnsafeAutoIdTimestamp, req, plan,
deleteFromLucene, currentlyDeleted,
currentNotFoundOrDeleted,
useLuceneUpdateDocument,
indexIntoLucene >>
DeleteCollectorProcess == DeleteCollectorLoop
LocalCheckpointTrackerLoop == /\ pc["LocalCheckpointTracker"] = "LocalCheckpointTrackerLoop"
/\ IF pc["Consumer"] /= "Done"
THEN /\ localCheckPoint + 1 \in completedSeqnos
/\ localCheckPoint' = localCheckPoint + 1
/\ pc' = [pc EXCEPT !["LocalCheckpointTracker"] = "LocalCheckpointTrackerLoop"]
ELSE /\ pc' = [pc EXCEPT !["LocalCheckpointTracker"] = "Done"]
/\ UNCHANGED localCheckPoint
/\ UNCHANGED << request_count,
replication_requests,
expected_doc,
versionMap_needsSafeAccess,
versionMap_isUnsafe,
versionMap_entry,
lucene_document, lucene_buffer,
completedSeqnos,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount,
maxUnsafeAutoIdTimestamp, req,
plan, deleteFromLucene,
currentlyDeleted,
currentNotFoundOrDeleted,
useLuceneUpdateDocument,
indexIntoLucene >>
LocalCheckpointTrackerProcess == LocalCheckpointTrackerLoop
UnsafeSeqnoIncreaserProcessLoop == /\ pc["UnsafeSeqnoIncreaserProcess"] = "UnsafeSeqnoIncreaserProcessLoop"
/\ IF pc["Consumer"] /= "Done" /\ maxSeqNoOfNonAppendOnlyOperations < request_count + 1
THEN /\ maxSeqNoOfNonAppendOnlyOperations' = maxSeqNoOfNonAppendOnlyOperations + 1
/\ pc' = [pc EXCEPT !["UnsafeSeqnoIncreaserProcess"] = "UnsafeSeqnoIncreaserProcessLoop"]
ELSE /\ pc' = [pc EXCEPT !["UnsafeSeqnoIncreaserProcess"] = "Done"]
/\ UNCHANGED maxSeqNoOfNonAppendOnlyOperations
/\ UNCHANGED << request_count,
replication_requests,
expected_doc,
versionMap_needsSafeAccess,
versionMap_isUnsafe,
versionMap_entry,
lucene_document,
lucene_buffer,
localCheckPoint,
completedSeqnos,
duplicationCount,
maxUnsafeAutoIdTimestamp,
req, plan, deleteFromLucene,
currentlyDeleted,
currentNotFoundOrDeleted,
useLuceneUpdateDocument,
indexIntoLucene >>
UnsafeSeqnoIncreaserProcess == UnsafeSeqnoIncreaserProcessLoop
ConsumerLoop == /\ pc["Consumer"] = "ConsumerLoop"
/\ IF replication_requests /= {}
THEN /\ \E replication_request \in replication_requests:
IF replication_request.type = ADD
THEN /\ \/ /\ replication_requests' = replication_requests \ {replication_request}
/\ req' = replication_request
/\ UNCHANGED duplicationCount
\/ /\ duplicationCount < DuplicationLimit
/\ duplicationCount' = duplicationCount + 1
/\ replication_requests' = (replication_requests \ {replication_request})
\cup {[replication_request EXCEPT !.type = RETRY_ADD]}
/\ req' = replication_request
\/ /\ duplicationCount < DuplicationLimit
/\ duplicationCount' = duplicationCount + 1
/\ req' = [replication_request EXCEPT !.type = RETRY_ADD]
/\ UNCHANGED replication_requests
ELSE /\ req' = replication_request
/\ \/ /\ duplicationCount < DuplicationLimit
/\ duplicationCount' = duplicationCount + 1
/\ UNCHANGED replication_requests
\/ /\ replication_requests' = replication_requests \ {replication_request}
/\ UNCHANGED duplicationCount
/\ IF req'.type = DELETE
THEN /\ versionMap_needsSafeAccess' = TRUE
/\ maxSeqNoOfNonAppendOnlyOperations' = Max(maxSeqNoOfNonAppendOnlyOperations, req'.seqno)
/\ IF req'.seqno <= localCheckPoint
THEN /\ plan' = "processButSkipLucene"
/\ deleteFromLucene' = FALSE
/\ currentlyDeleted' = FALSE
/\ pc' = [pc EXCEPT !["Consumer"] = "ExecuteDeletePlan"]
ELSE /\ IF versionMap_isUnsafe
THEN /\ pc' = [pc EXCEPT !["Consumer"] = "AwaitRefreshOnDelete"]
ELSE /\ pc' = [pc EXCEPT !["Consumer"] = "compareDeleteOpToLuceneDocBasedOnSeqNo"]
/\ UNCHANGED << plan,
deleteFromLucene,
currentlyDeleted >>
/\ UNCHANGED << maxUnsafeAutoIdTimestamp,
currentNotFoundOrDeleted,
useLuceneUpdateDocument,
indexIntoLucene >>
ELSE /\ IF req'.type = RETRY_ADD
THEN /\ maxUnsafeAutoIdTimestamp' = Max(maxUnsafeAutoIdTimestamp, req'.autoIdTimeStamp)
ELSE /\ TRUE
/\ UNCHANGED maxUnsafeAutoIdTimestamp
/\ IF /\ req'.type = ADD
/\ maxUnsafeAutoIdTimestamp' < req'.autoIdTimeStamp
/\ maxSeqNoOfNonAppendOnlyOperations < req'.seqno
THEN /\ plan' = "optimisedAppendOnly"
/\ currentNotFoundOrDeleted' = TRUE
/\ useLuceneUpdateDocument' = FALSE
/\ indexIntoLucene' = TRUE
/\ pc' = [pc EXCEPT !["Consumer"] = "ExecuteIndexPlan"]
/\ UNCHANGED << versionMap_needsSafeAccess,
maxSeqNoOfNonAppendOnlyOperations >>
ELSE /\ IF req'.type \notin {ADD, RETRY_ADD}
THEN /\ maxSeqNoOfNonAppendOnlyOperations' = Max(maxSeqNoOfNonAppendOnlyOperations, req'.seqno)
ELSE /\ TRUE
/\ UNCHANGED maxSeqNoOfNonAppendOnlyOperations
/\ versionMap_needsSafeAccess' = TRUE
/\ IF req'.seqno <= localCheckPoint
THEN /\ plan' = "processButSkipLucene"
/\ currentNotFoundOrDeleted' = FALSE
/\ useLuceneUpdateDocument' = FALSE
/\ indexIntoLucene' = FALSE
/\ pc' = [pc EXCEPT !["Consumer"] = "ExecuteIndexPlan"]
ELSE /\ IF versionMap_isUnsafe
THEN /\ pc' = [pc EXCEPT !["Consumer"] = "AwaitRefreshOnIndex"]
ELSE /\ pc' = [pc EXCEPT !["Consumer"] = "compareIndexOpToLuceneDocBasedOnSeqNo"]
/\ UNCHANGED << plan,
currentNotFoundOrDeleted,
useLuceneUpdateDocument,
indexIntoLucene >>
/\ UNCHANGED << deleteFromLucene,
currentlyDeleted >>
ELSE /\ pc' = [pc EXCEPT !["Consumer"] = "Done"]
/\ UNCHANGED << replication_requests,
versionMap_needsSafeAccess,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount,
maxUnsafeAutoIdTimestamp, req, plan,
deleteFromLucene, currentlyDeleted,
currentNotFoundOrDeleted,
useLuceneUpdateDocument,
indexIntoLucene >>
/\ UNCHANGED << request_count, expected_doc,
versionMap_isUnsafe, versionMap_entry,
lucene_document, lucene_buffer,
localCheckPoint, completedSeqnos >>
ExecuteDeletePlan == /\ pc["Consumer"] = "ExecuteDeletePlan"
/\ IF deleteFromLucene
THEN /\ IF currentlyDeleted = FALSE
THEN /\ lucene_buffer' = Append(lucene_buffer, [ type |-> Lucene_deleteDocuments ])
ELSE /\ TRUE
/\ UNCHANGED lucene_buffer
/\ versionMap_entry' = [ type |-> DELETE, seqno |-> req.seqno, flushed |-> FALSE ]
ELSE /\ TRUE
/\ UNCHANGED << versionMap_entry,
lucene_buffer >>
/\ completedSeqnos' = (completedSeqnos \cup {req.seqno})
/\ pc' = [pc EXCEPT !["Consumer"] = "ConsumerLoop"]
/\ UNCHANGED << request_count, replication_requests,
expected_doc, versionMap_needsSafeAccess,
versionMap_isUnsafe, lucene_document,
localCheckPoint,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount,
maxUnsafeAutoIdTimestamp, req, plan,
deleteFromLucene, currentlyDeleted,
currentNotFoundOrDeleted,
useLuceneUpdateDocument, indexIntoLucene >>
ExecuteIndexPlan == /\ pc["Consumer"] = "ExecuteIndexPlan"
/\ IF indexIntoLucene
THEN /\ lucene_buffer' = Append(lucene_buffer,
[ type |-> IF useLuceneUpdateDocument THEN Lucene_updateDocuments ELSE Lucene_addDocuments
, seqno |-> req.seqno
, content |-> req.content
])
/\ IF versionMap_needsSafeAccess
THEN /\ versionMap_entry' = [ type |-> UPDATE, seqno |-> req.seqno ]
/\ UNCHANGED versionMap_isUnsafe
ELSE /\ versionMap_isUnsafe' = TRUE
/\ IF /\ versionMap_entry /= NULL
/\ versionMap_entry.type = DELETE
/\ versionMap_entry.seqno < req.seqno
THEN /\ versionMap_entry' = NULL
ELSE /\ TRUE
/\ UNCHANGED versionMap_entry
ELSE /\ TRUE
/\ UNCHANGED << versionMap_isUnsafe,
versionMap_entry, lucene_buffer >>
/\ completedSeqnos' = (completedSeqnos \cup {req.seqno})
/\ pc' = [pc EXCEPT !["Consumer"] = "ConsumerLoop"]
/\ UNCHANGED << request_count, replication_requests,
expected_doc, versionMap_needsSafeAccess,
lucene_document, localCheckPoint,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount, maxUnsafeAutoIdTimestamp,
req, plan, deleteFromLucene,
currentlyDeleted, currentNotFoundOrDeleted,
useLuceneUpdateDocument, indexIntoLucene >>
compareDeleteOpToLuceneDocBasedOnSeqNo == /\ pc["Consumer"] = "compareDeleteOpToLuceneDocBasedOnSeqNo"
/\ IF versionMap_entry /= NULL
THEN /\ IF req.seqno > versionMap_entry.seqno
THEN /\ plan' = "processNormally"
/\ deleteFromLucene' = TRUE
/\ currentlyDeleted' = FALSE
ELSE /\ plan' = "processButSkipLucene"
/\ deleteFromLucene' = FALSE
/\ currentlyDeleted' = FALSE
ELSE /\ IF lucene_document = NULL
THEN /\ plan' = "processNormallyExceptNotFound"
/\ deleteFromLucene' = TRUE
/\ currentlyDeleted' = TRUE
ELSE /\ IF req.seqno > lucene_document.seqno
THEN /\ plan' = "processNormally"
/\ deleteFromLucene' = TRUE
/\ currentlyDeleted' = FALSE
ELSE /\ plan' = "processButSkipLucene"
/\ deleteFromLucene' = FALSE
/\ currentlyDeleted' = FALSE
/\ pc' = [pc EXCEPT !["Consumer"] = "ExecuteDeletePlan"]
/\ UNCHANGED << request_count,
replication_requests,
expected_doc,
versionMap_needsSafeAccess,
versionMap_isUnsafe,
versionMap_entry,
lucene_document,
lucene_buffer,
localCheckPoint,
completedSeqnos,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount,
maxUnsafeAutoIdTimestamp,
req,
currentNotFoundOrDeleted,
useLuceneUpdateDocument,
indexIntoLucene >>
AwaitRefreshOnDelete == /\ pc["Consumer"] = "AwaitRefreshOnDelete"
/\ lucene_buffer = <<>>
/\ versionMap_needsSafeAccess' = TRUE
/\ pc' = [pc EXCEPT !["Consumer"] = "compareDeleteOpToLuceneDocBasedOnSeqNo"]
/\ UNCHANGED << request_count, replication_requests,
expected_doc, versionMap_isUnsafe,
versionMap_entry, lucene_document,
lucene_buffer, localCheckPoint,
completedSeqnos,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount,
maxUnsafeAutoIdTimestamp, req, plan,
deleteFromLucene, currentlyDeleted,
currentNotFoundOrDeleted,
useLuceneUpdateDocument,
indexIntoLucene >>
compareIndexOpToLuceneDocBasedOnSeqNo == /\ pc["Consumer"] = "compareIndexOpToLuceneDocBasedOnSeqNo"
/\ IF req.seqno <= localCheckPoint
THEN /\ plan' = "processButSkipLucene"
/\ currentNotFoundOrDeleted' = FALSE
/\ useLuceneUpdateDocument' = FALSE
/\ indexIntoLucene' = FALSE
ELSE /\ IF versionMap_entry /= NULL
THEN /\ IF req.seqno > versionMap_entry.seqno
THEN /\ plan' = "processNormally"
/\ currentNotFoundOrDeleted' = FALSE
/\ useLuceneUpdateDocument' = TRUE
/\ indexIntoLucene' = TRUE
ELSE /\ plan' = "processButSkipLucene"
/\ currentNotFoundOrDeleted' = FALSE
/\ useLuceneUpdateDocument' = FALSE
/\ indexIntoLucene' = FALSE
ELSE /\ IF lucene_document = NULL
THEN /\ plan' = "processNormallyExceptNotFound"
/\ currentNotFoundOrDeleted' = TRUE
/\ useLuceneUpdateDocument' = FALSE
/\ indexIntoLucene' = TRUE
ELSE /\ IF req.seqno > lucene_document.seqno
THEN /\ plan' = "processNormally"
/\ currentNotFoundOrDeleted' = FALSE
/\ useLuceneUpdateDocument' = TRUE
/\ indexIntoLucene' = TRUE
ELSE /\ plan' = "processButSkipLucene"
/\ currentNotFoundOrDeleted' = FALSE
/\ useLuceneUpdateDocument' = FALSE
/\ indexIntoLucene' = FALSE
/\ pc' = [pc EXCEPT !["Consumer"] = "ExecuteIndexPlan"]
/\ UNCHANGED << request_count,
replication_requests,
expected_doc,
versionMap_needsSafeAccess,
versionMap_isUnsafe,
versionMap_entry,
lucene_document,
lucene_buffer,
localCheckPoint,
completedSeqnos,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount,
maxUnsafeAutoIdTimestamp,
req, deleteFromLucene,
currentlyDeleted >>
AwaitRefreshOnIndex == /\ pc["Consumer"] = "AwaitRefreshOnIndex"
/\ lucene_buffer = <<>>
/\ versionMap_needsSafeAccess' = TRUE
/\ pc' = [pc EXCEPT !["Consumer"] = "compareIndexOpToLuceneDocBasedOnSeqNo"]
/\ UNCHANGED << request_count, replication_requests,
expected_doc, versionMap_isUnsafe,
versionMap_entry, lucene_document,
lucene_buffer, localCheckPoint,
completedSeqnos,
maxSeqNoOfNonAppendOnlyOperations,
duplicationCount,
maxUnsafeAutoIdTimestamp, req, plan,
deleteFromLucene, currentlyDeleted,
currentNotFoundOrDeleted,
useLuceneUpdateDocument,
indexIntoLucene >>
ConsumerProcess == ConsumerLoop \/ ExecuteDeletePlan \/ ExecuteIndexPlan
\/ compareDeleteOpToLuceneDocBasedOnSeqNo
\/ AwaitRefreshOnDelete
\/ compareIndexOpToLuceneDocBasedOnSeqNo
\/ AwaitRefreshOnIndex
Next == SafeAccessEnablerProcess \/ UnsafePutterProcess
\/ MaxUnsafeAutoIdTimestampIncreaserProcess \/ LuceneProcess
\/ DeleteCollectorProcess \/ LocalCheckpointTrackerProcess
\/ UnsafeSeqnoIncreaserProcess \/ ConsumerProcess
\/ (* Disjunct to prevent deadlock on termination *)
((\A self \in ProcSet: pc[self] = "Done") /\ UNCHANGED vars)
Spec == Init /\ [][Next]_vars
Termination == <>(\A self \in ProcSet: pc[self] = "Done")
\* END TRANSLATION
Terminated == \A self \in ProcSet: pc[self] = "Done"
Invariant == Terminated => /\ expected_doc = NULL => lucene_document = NULL
/\ expected_doc /= NULL => lucene_document.content = expected_doc
=============================================================================
================================================
FILE: ReplicaEngine/tla/ReplicaEngine.toolbox/.project
================================================
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>ReplicaEngine</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
<buildCommand>
<name>toolbox.builder.TLAParserBuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>toolbox.builder.PCalAlgorithmSearchingBuilder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>toolbox.natures.TLANature</nature>
</natures>
<linkedResources>
<link>
<name>ReplicaEngine.tla</name>
<type>1</type>
<locationURI>PARENT-1-PROJECT_LOC/ReplicaEngine.tla</locationURI>
</link>
</linkedResources>
</projectDescription>
================================================
FILE: ReplicaEngine/tla/ReplicaEngine.toolbox/.settings/org.lamport.tla.toolbox.prefs
================================================
ProjectRootFile=PARENT-1-PROJECT_LOC/ReplicaEngine.tla
eclipse.preferences.version=1
================================================
FILE: ReplicaEngine/tla/ReplicaEngine.toolbox/ReplicaEngine___model.launch
================================================
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.lamport.tla.toolbox.tool.tlc.modelCheck">
<stringAttribute key="TLCCmdLineParameters" value=""/>
<stringAttribute key="configurationName" value="model"/>
<intAttribute key="dfidDepth" value="100"/>
<booleanAttribute key="dfidMode" value="false"/>
<intAttribute key="distributedFPSetCount" value="0"/>
<stringAttribute key="distributedNetworkInterface" value="192.168.1.39"/>
<intAttribute key="distributedNodesCount" value="1"/>
<stringAttribute key="distributedTLC" value="off"/>
<stringAttribute key="distributedTLCVMArgs" value=""/>
<intAttribute key="fpBits" value="1"/>
<intAttribute key="fpIndex" value="1"/>
<intAttribute key="maxHeapSize" value="25"/>
<intAttribute key="maxSetSize" value="1000000"/>
<booleanAttribute key="mcMode" value="true"/>
<stringAttribute key="modelBehaviorInit" value=""/>
<stringAttribute key="modelBehaviorNext" value=""/>
<stringAttribute key="modelBehaviorSpec" value="Spec"/>
<intAttribute key="modelBehaviorSpecType" value="1"/>
<stringAttribute key="modelBehaviorVars" value="versionMap_isUnsafe, lucene_buffer, maxSeqNoOfNonAppendOnlyOperations, request_count, expected_doc, duplicationCount, useLuceneUpdateDocument, pc, versionMap_entry, replication_requests, maxUnsafeAutoIdTimestamp, currentlyDeleted, req, lucene_document, indexIntoLucene, versionMap_needsSafeAccess, deleteFromLucene, currentNotFoundOrDeleted, plan, localCheckPoint, completedSeqnos"/>
<stringAttribute key="modelComments" value=""/>
<booleanAttribute key="modelCorrectnessCheckDeadlock" value="true"/>
<listAttribute key="modelCorrectnessInvariants">
<listEntry value="1Invariant"/>
</listAttribute>
<listAttribute key="modelCorrectnessProperties">
<listEntry value="0Termination"/>
<listEntry value="1Invariant"/>
</listAttribute>
<stringAttribute key="modelExpressionEval" value=""/>
<stringAttribute key="modelParameterActionConstraint" value=""/>
<listAttribute key="modelParameterConstants">
<listEntry value="UPDATE;;UPDATE;1;0"/>
<listEntry value="NULL;;NULL;1;0"/>
<listEntry value="ADD;;ADD;1;0"/>
<listEntry value="DELETE;;DELETE;1;0"/>
<listEntry value="DocContent;;{DocA, DocB};1;1"/>
<listEntry value="RETRY_ADD;;RETRY_ADD;1;0"/>
<listEntry value="Lucene_deleteDocuments;;Lucene_deleteDocuments;1;0"/>
<listEntry value="Lucene_addDocuments;;Lucene_addDocuments;1;0"/>
<listEntry value="Lucene_updateDocuments;;Lucene_updateDocuments;1;0"/>
<listEntry value="DocAutoIdTimestamp;;1000;0;0"/>
<listEntry value="defaultInitValue;;defaultInitValue;1;0"/>
<listEntry value="DuplicationLimit;;1;0;0"/>
</listAttribute>
<stringAttribute key="modelParameterContraint" value=""/>
<listAttribute key="modelParameterDefinitions"/>
<stringAttribute key="modelParameterModelValues" value="{}"/>
<stringAttribute key="modelParameterNewDefinitions" value=""/>
<stringAttribute key="modelPropertiesExpand" value=""/>
<intAttribute key="numberOfWorkers" value="4"/>
<booleanAttribute key="recover" value="false"/>
<stringAttribute key="result.mail.address" value=""/>
<intAttribute key="simuAril" value="-1"/>
<intAttribute key="simuDepth" value="100"/>
<intAttribute key="simuSeed" value="-1"/>
<stringAttribute key="specName" value="ReplicaEngine"/>
<stringAttribute key="view" value=""/>
</launchConfiguration>
================================================
FILE: Storage/tla/Storage.tla
================================================
------------------------------ MODULE Storage ------------------------------
EXTENDS Integers, FiniteSets, TLC
CONSTANTS
MaxNewMeta, \* maximum generation of newMeta to limit the state space
MetaDataContent \* content that is written to the metadata file
VARIABLES
metadata, \* metaData[i] = MetaDataContent if metadata of generation i is present
manifest, \* manifest[j] is generation of metadata j-th manifest is referencing
newMeta, \* generation of newly created metadata file
newManifest, \* generation of newly created manifest file
state, \* current state, describes what to do next
possibleStates \* set of generations of metadata that limits what can be read from disk
--------------------------------------------------------------------------
(*************************************************************************)
(* First we define some helper functions to work with files abstraction. *)
(* Files is a function from file generation to some content. *)
(*************************************************************************)
(*************************************************************************)
(* CurrentGeneration returns the maximum file generation. If there are *)
(* no files then -1 is returned. *)
(*************************************************************************)
CurrentGeneration(files) ==
IF DOMAIN files = {}
THEN -1
ELSE
CHOOSE gen \in DOMAIN files :
\A otherGen \in DOMAIN files : gen \geq otherGen
(*************************************************************************)
(* DeleteFile removes file with generation delGen. *)
(*************************************************************************)
DeleteFile(files, delGen) == [gen \in DOMAIN files \ {delGen} |-> files[gen]]
(*************************************************************************)
(* DeleteFilesExcept removes all files except keepGen. *)
(*************************************************************************)
DeleteFilesExcept(files, keepGen) == (keepGen :> files[keepGen])
(*************************************************************************)
(* WriteFile creates new file with specified generation and content. *)
(*************************************************************************)
WriteFile(files, gen, content) == (gen :> content) @@ files
--------------------------------------------------------------------------
(*************************************************************************)
(* Now we define functions to emulate write and cleanup of the metadata. *)
(*************************************************************************)
WriteMetaOk(gen) ==
/\ metadata' = WriteFile(metadata, gen, MetaDataContent)
/\ state' = "writeManifest"
WriteMetaFail(gen) ==
/\ metadata' = metadata
/\ state' = "writeMeta"
WriteMetaDirty(gen) ==
/\ \/ metadata' = WriteFile(metadata, gen, MetaDataContent)
\/ metadata' = metadata
/\ state' = "deleteNewMeta"
DeleteNewMeta ==
/\ \/ metadata' = DeleteFile(metadata, newMeta)
\/ metadata' = metadata
/\ state' = "writeMeta"
/\ UNCHANGED <<newMeta, newManifest, manifest, possibleStates>>
DeleteOldMeta ==
/\ \/ metadata' = DeleteFilesExcept(metadata, newMeta)
\/ metadata' = metadata
/\ state' = "writeMeta"
/\ UNCHANGED <<newMeta, newManifest, manifest, possibleStates>>
WriteMeta ==
LET gen == CurrentGeneration(metadata) + 1 IN
/\ newMeta' = gen
/\ \/ WriteMetaOk(gen)
\/ WriteMetaFail(gen)
\/ WriteMetaDirty(gen)
/\ UNCHANGED <<newManifest, manifest, possibleStates>>
--------------------------------------------------------------------------
(*************************************************************************)
(* Now we define functions to emulate write and cleanup of the manifest *)
(* file. *)
(*************************************************************************)
WriteManifestOk(gen) ==
/\ manifest' = WriteFile(manifest, gen, newMeta)
/\ state' = "deleteOldManifest"
/\ possibleStates' = {newMeta}
WriteManifestFail(gen) ==
/\ manifest' = manifest
/\ state' = "deleteNewMeta"
/\ possibleStates' = possibleStates
WriteManifestDirty(gen) ==
/\ \/ manifest' = WriteFile(manifest, gen, newMeta)
\/ manifest' = manifest
/\ state' = "deleteNewManifest"
/\ possibleStates' = possibleStates \union {newMeta}
WriteManifest ==
LET gen == CurrentGeneration(manifest) + 1 IN
/\ newManifest' = gen
/\ \/ WriteManifestOk(gen)
\/ WriteManifestFail(gen)
\/ WriteManifestDirty(gen)
/\ UNCHANGED <<newMeta, metadata>>
DeleteOldManifest ==
/\ \/ manifest' = DeleteFilesExcept(manifest, newManifest)
\/ manifest' = manifest
/\ state' = "deleteOldMeta"
/\ UNCHANGED <<newMeta, newManifest, metadata, possibleStates>>
--------------------------------------------------------------------------
(*************************************************************************)
(* Below are 3 versions of the same function, that is called when *)
(* manifest write was dirty. The buggy one was initially implemented and *)
(* was caught by https://github.com/elastic/elasticsearch/issues/39077. *)
(* Pick one and use in Next function. *)
(* https://github.com/elastic/elasticsearch/pull/40519 implements *)
(* DeleteNewManifestEasy. *)
(*************************************************************************)
DeleteNewManifestBuggy ==
/\ \/ manifest' = DeleteFile(manifest, newManifest)
\/ manifest' = manifest
/\ state' = "deleteNewMeta"
/\ UNCHANGED <<newMeta, newManifest, metadata, possibleStates>>
DeleteNewManifestEasy ==
/\ \/ manifest' = DeleteFile(manifest, newManifest)
\/ manifest' = manifest
/\ state' = "writeMeta"
/\ UNCHANGED <<newMeta, newManifest, possibleStates, metadata>>
DeleteNewManifestHard ==
/\ \/ /\ manifest' = DeleteFile(manifest, newManifest)
/\ state' = "deleteNewMeta"
\/ /\ manifest' = manifest
/\ state' = "writeMeta"
/\ UNCHANGED <<newMeta, newManifest, metadata, possibleStates>>
--------------------------------------------------------------------------
(*************************************************************************)
(* We can define Init and Next functions now. *)
(*************************************************************************)
Init ==
/\ metadata = <<>>
/\ manifest = <<>>
/\ newMeta = -1 \* no latest metadata file
/\ newManifest = -1 \* no latest manifest file
/\ state = "writeMeta" \* we start with writing metadata file
/\ possibleStates = {} \* no metadata can be read from disk
Next ==
\/ (state = "writeMeta" /\ WriteMeta)
\/ (state = "writeManifest" /\ WriteManifest)
\/ (state = "deleteOldManifest" /\ DeleteOldManifest)
\/ (state = "deleteOldMeta" /\ DeleteOldMeta)
\/ (state = "deleteNewManifest" /\ DeleteNewManifestEasy) \* try DeleteNewManifestBuggy and DeleteNewManifestHard
\/ (state = "deleteNewMeta" /\ DeleteNewMeta)
--------------------------------------------------------------------------
(*************************************************************************)
(* Our model has 2 invariants. *)
(*************************************************************************)
MetadataFileReferencedByManifestExists ==
CurrentGeneration(manifest) /= -1
=>
manifest[CurrentGeneration(manifest)] \in DOMAIN metadata
MetadataReferencedByManifestIsValid ==
CurrentGeneration(manifest) /= -1
=>
\E meta \in possibleStates : meta = manifest[CurrentGeneration(manifest)]
============
================================================
FILE: Storage/tla/Storage.toolbox/Storage___model.launch
================================================
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.lamport.tla.toolbox.tool.tlc.modelCheck">
<stringAttribute key="TLCCmdLineParameters" value=""/>
<stringAttribute key="configurationName" value="model"/>
<booleanAttribute key="deferLiveness" value="false"/>
<intAttribute key="dfidDepth" value="100"/>
<booleanAttribute key="dfidMode" value="false"/>
<intAttribute key="distributedFPSetCount" value="0"/>
<stringAttribute key="distributedNetworkInterface" value="10.2.42.180"/>
<intAttribute key="distributedNodesCount" value="1"/>
<stringAttribute key="distributedTLC" value="off"/>
<stringAttribute key="distributedTLCVMArgs" value=""/>
<intAttribute key="fpBits" value="1"/>
<intAttribute key="fpIndex" value="0"/>
<booleanAttribute key="fpIndexRandom" value="true"/>
<intAttribute key="maxHeapSize" value="25"/>
<intAttribute key="maxSetSize" value="1000000"/>
<booleanAttribute key="mcMode" value="true"/>
<stringAttribute key="modelBehaviorInit" value="Init"/>
<stringAttribute key="modelBehaviorNext" value="Next"/>
<stringAttribute key="modelBehaviorSpec" value=""/>
<intAttribute key="modelBehaviorSpecType" value="2"/>
<stringAttribute key="modelBehaviorVars" value="state, metadata, newManifest, possibleStates, newMeta, manifest"/>
<stringAttribute key="modelComments" value=""/>
<booleanAttribute key="modelCorrectnessCheckDeadlock" value="true"/>
<listAttribute key="modelCorrectnessInvariants">
<listEntry value="1MetadataFileReferencedByManifestExists"/>
<listEntry value="1MetadataReferencedByManifestIsValid"/>
</listAttribute>
<listAttribute key="modelCorrectnessProperties"/>
<stringAttribute key="modelExpressionEval" value=""/>
<stringAttribute key="modelParameterActionConstraint" value=""/>
<listAttribute key="modelParameterConstants">
<listEntry value="MaxNewMeta;;5;0;0"/>
<listEntry value="MetaDataContent;;MetaDataContent;1;0"/>
</listAttribute>
<stringAttribute key="modelParameterContraint" value="newMeta < MaxNewMeta"/>
<listAttribute key="modelParameterDefinitions"/>
<stringAttribute key="modelParameterModelValues" value="{}"/>
<stringAttribute key="modelParameterNewDefinitions" value=""/>
<intAttribute key="numberOfWorkers" value="6"/>
<booleanAttribute key="recover" value="false"/>
<stringAttribute key="result.mail.address" value=""/>
<intAttribute key="simuAril" value="-1"/>
<intAttribute key="simuDepth" value="100"/>
<intAttribute key="simuSeed" value="-1"/>
<stringAttribute key="specName" value="Storage"/>
<stringAttribute key="view" value=""/>
<booleanAttribute key="visualizeStateGraph" value="false"/>
</launchConfiguration>
================================================
FILE: ZenWithTerms/tla/ZenWithTerms.tla
================================================
-------------------------------------------------------------------------------------
-------------------------------- MODULE ZenWithTerms --------------------------------
\* Imported modules used in this specification
EXTENDS Naturals, FiniteSets, Sequences, TLC
----
CONSTANTS Values
\* Set of node ids (all master-eligible nodes)
CONSTANTS Nodes
\* RPC message types
CONSTANTS
Join,
PublishRequest,
PublishResponse,
Commit
----
\* Set of requests and responses sent between nodes.
VARIABLE messages
\* Transitive closure of value updates as done by leaders
VARIABLE descendant
\* Values to bootstrap the cluster
VARIABLE initialConfiguration
VARIABLE initialValue
VARIABLE initialAcceptedVersion
\* node state (map from node id to state)
VARIABLE currentTerm
VARIABLE lastCommittedConfiguration
VARIABLE lastAcceptedTerm
VARIABLE lastAcceptedVersion
VARIABLE lastAcceptedValue
VARIABLE lastAcceptedConfiguration
VARIABLE joinVotes
VARIABLE startedJoinSinceLastReboot
VARIABLE electionWon
VARIABLE lastPublishedVersion
VARIABLE lastPublishedConfiguration
VARIABLE publishVotes
----
Terms == Nat
Versions == Nat
\* set of valid configurations (i.e. the set of all non-empty subsets of Nodes)
ValidConfigs == SUBSET(Nodes) \ {{}}
\* cluster-state versions that might have come from older systems
InitialVersions == Nat
\* quorums correspond to majority of votes in a config
IsQuorum(votes, config) == Cardinality(votes \cap config) * 2 > Cardinality(config)
IsElectionQuorum(n, votes) ==
/\ IsQuorum(votes, lastCommittedConfiguration[n])
/\ IsQuorum(votes, lastAcceptedConfiguration[n])
IsPublishQuorum(n, votes) ==
/\ IsQuorum(votes, lastCommittedConfiguration[n])
/\ IsQuorum(votes, lastPublishedConfiguration[n])
\* initial model state
Init == /\ messages = {}
/\ descendant = {}
/\ initialConfiguration \in ValidConfigs
/\ initialValue \in Values
/\ initialAcceptedVersion \in [Nodes -> InitialVersions]
/\ currentTerm = [n \in Nodes |-> 0]
/\ lastCommittedConfiguration = [n \in Nodes |-> {}] \* empty config
/\ lastAcceptedTerm = [n \in Nodes |-> 0]
/\ lastAcceptedVersion = initialAcceptedVersion
/\ lastAcceptedValue \in {[n \in Nodes |-> v] : v \in Values} \* all agree on initial value
/\ lastAcceptedConfiguration = [n \in Nodes |-> lastCommittedConfiguration[n]]
/\ joinVotes = [n \in Nodes |-> {}]
/\ startedJoinSinceLastReboot = [n \in Nodes |-> FALSE]
/\ electionWon = [n \in Nodes |-> FALSE]
/\ lastPublishedVersion = [n \in Nodes |-> 0]
/\ lastPublishedConfiguration = [n \in Nodes |-> lastCommittedConfiguration[n]]
/\ publishVotes = [n \in Nodes |-> {}]
\* Bootstrap node n with the initial state and config
SetInitialState(n) ==
/\ lastAcceptedConfiguration[n] = {} \* not already bootstrapped
/\ Assert(lastAcceptedTerm[n] = 0, "lastAcceptedTerm should be 0")
/\ Assert(lastCommittedConfiguration[n] = {}, "lastCommittedConfiguration should be empty")
/\ Assert(lastPublishedVersion[n] = 0, "lastPublishedVersion should be 0")
/\ Assert(lastPublishedConfiguration[n] = {}, "lastPublishedConfiguration should be empty")
/\ Assert(electionWon[n] = FALSE, "electionWon should be FALSE")
/\ Assert(joinVotes[n] = {}, "joinVotes should be empty")
/\ Assert(publishVotes[n] = {}, "publishVotes should be empty")
/\ lastAcceptedConfiguration' = [lastAcceptedConfiguration EXCEPT ![n] = initialConfiguration]
/\ lastAcceptedValue' = [lastAcceptedValue EXCEPT ![n] = initialValue]
/\ lastCommittedConfiguration' = [lastCommittedConfiguration EXCEPT ![n] = initialConfiguration]
/\ Assert(lastAcceptedTerm[n] = 0, "lastAcceptedTerm should be 0")
/\ Assert(lastAcceptedConfiguration'[n] /= {}, "lastAcceptedConfiguration should be non-empty")
/\ Assert(lastCommittedConfiguration'[n] /= {}, "lastCommittedConfiguration should be non-empty")
/\ UNCHANGED <<descendant, initialConfiguration, initialValue, messages, lastAcceptedTerm, lastAcceptedVersion,
lastPublishedVersion, lastPublishedConfiguration, electionWon, joinVotes, publishVotes,
startedJoinSinceLastReboot, currentTerm, initialAcceptedVersion>>
\* Send join request from node n to node nm for term t
HandleStartJoin(n, nm, t) ==
/\ t > currentTerm[n]
/\ LET
joinRequest == [method |-> Join,
source |-> n,
dest |-> nm,
term |-> t,
laTerm |-> lastAcceptedTerm[n],
laVersion |-> lastAcceptedVersion[n]]
IN
/\ currentTerm' = [currentTerm EXCEPT ![n] = t]
/\ lastPublishedVersion' = [lastPublishedVersion EXCEPT ![n] = 0]
/\ lastPublishedConfiguration' = [lastPublishedConfiguration EXCEPT ![n] = lastAcceptedConfiguration[n]]
/\ startedJoinSinceLastReboot' = [startedJoinSinceLastReboot EXCEPT ![n] = TRUE]
/\ electionWon' = [electionWon EXCEPT ![n] = FALSE]
/\ joinVotes' = [joinVotes EXCEPT ![n] = {}]
/\ publishVotes' = [publishVotes EXCEPT ![n] = {}]
/\ messages' = messages \cup { joinRequest }
/\ UNCHANGED <<lastCommittedConfiguration, lastAcceptedConfiguration, lastAcceptedVersion,
lastAcceptedValue, lastAcceptedTerm, descendant, initialConfiguration, initialValue, initialAcceptedVersion>>
\* node n handles a join request and checks if it has received enough joins (= votes)
\* for its term to be elected as master
HandleJoin(n, m) ==
/\ m.method = Join
/\ m.term = currentTerm[n]
/\ startedJoinSinceLastReboot[n]
/\ \/ m.laTerm < lastAcceptedTerm[n]
\/ /\ m.laTerm = lastAcceptedTerm[n]
/\ m.laVersion <= lastAcceptedVersion[n]
/\ lastAcceptedConfiguration[n] /= {} \* must be bootstrapped
/\ joinVotes' = [joinVotes EXCEPT ![n] = @ \cup { m.source }]
/\ electionWon' = [electionWon EXCEPT ![n] = IsElectionQuorum(n, joinVotes'[n])]
/\ IF electionWon[n] = FALSE /\ electionWon'[n]
THEN
\* initiating publish version with last accepted version to enable client requests
/\ lastPublishedVersion' = [lastPublishedVersion EXCEPT ![n] = lastAcceptedVersion[n]]
ELSE
UNCHANGED <<lastPublishedVersion>>
/\ UNCHANGED <<lastCommittedConfiguration, currentTerm, publishVotes, messages, descendant,
lastAcceptedVersion, lastAcceptedValue, lastAcceptedConfiguration,
lastAcceptedTerm, startedJoinSinceLastReboot, lastPublishedConfiguration,
initialConfiguration, initialValue, initialAcceptedVersion>>
\* client causes a cluster state change val with configuration cfg
HandleClientValue(n, t, v, val, cfg) ==
/\ electionWon[n]
/\ lastPublishedVersion[n] = lastAcceptedVersion[n] \* means we have the last published value / config (useful for CAS operations, where we need to read the previous value first)
/\ t = currentTerm[n]
/\ v > lastPublishedVersion[n]
/\ cfg /= lastAcceptedConfiguration[n] => lastCommittedConfiguration[n] = lastAcceptedConfiguration[n] \* only allow reconfiguration if there is not already a reconfiguration in progress
/\ IsQuorum(joinVotes[n], cfg) \* only allow reconfiguration if we have a quorum of (join) votes for the new config
/\ LET
publishRequests == { [method |-> PublishRequest,
source |-> n,
dest |-> ns,
term |-> t,
version |-> v,
value |-> val,
config |-> cfg,
commConf |-> lastCommittedConfiguration[n]] : ns \in Nodes }
newEntry == [prevT |-> lastAcceptedTerm[n],
prevV |-> lastAcceptedVersion[n],
nextT |-> t,
nextV |-> v]
matchingElems == { e \in descendant :
/\ e.nextT = newEntry.prevT
/\ e.nextV = newEntry.prevV }
newTransitiveElems == { [prevT |-> e.prevT,
prevV |-> e.prevV,
nextT |-> newEntry.nextT,
nextV |-> newEntry.nextV] : e \in matchingElems }
IN
/\ descendant' = descendant \cup {newEntry} \cup newTransitiveElems
/\ lastPublishedVersion' = [lastPublishedVersion EXCEPT ![n] = v]
/\ lastPublishedConfiguration' = [lastPublishedConfiguration EXCEPT ![n] = cfg]
/\ publishVotes' = [publishVotes EXCEPT ![n] = {}] \* publishVotes are only counted per publish version
/\ messages' = messages \cup publishRequests
/\ UNCHANGED <<startedJoinSinceLastReboot, lastCommittedConfiguration, currentTerm, electionWon,
lastAcceptedVersion, lastAcceptedValue, lastAcceptedTerm, lastAcceptedConfiguration,
joinVotes, initialConfiguration, initialValue, initialAcceptedVersion>>
\* handle publish request m on node n
HandlePublishRequest(n, m) ==
/\ m.method = PublishRequest
/\ m.term = currentTerm[n]
/\ (m.term = lastAcceptedTerm[n]) => (m.version > lastAcceptedVersion[n])
/\ lastAcceptedTerm' = [lastAcceptedTerm EXCEPT ![n] = m.term]
/\ lastAcceptedVersion' = [lastAcceptedVersion EXCEPT ![n] = m.version]
/\ lastAcceptedValue' = [lastAcceptedValue EXCEPT ![n] = m.value]
/\ lastAcceptedConfiguration' = [lastAcceptedConfiguration EXCEPT ![n] = m.config]
/\ lastCommittedConfiguration' = [lastCommittedConfiguration EXCEPT ![n] = m.commConf]
/\ LET
response == [method |-> PublishResponse,
source |-> n,
dest |-> m.source,
term |-> m.term,
version |-> m.version]
IN
/\ messages' = messages \cup {response}
/\ UNCHANGED <<startedJoinSinceLastReboot, currentTerm, descendant, lastPublishedConfiguration,
electionWon, lastPublishedVersion, joinVotes, publishVotes, initialConfiguration,
initialValue, initialAcceptedVersion>>
\* node n commits a change
HandlePublishResponse(n, m) ==
/\ m.method = PublishResponse
/\ electionWon[n]
/\ m.term = currentTerm[n]
/\ m.version = lastPublishedVersion[n]
/\ publishVotes' = [publishVotes EXCEPT ![n] = @ \cup {m.source}]
/\ IF
IsPublishQuorum(n, publishVotes'[n])
THEN
LET
commitRequests == { [method |-> Commit,
source |-> n,
dest |-> ns,
term |-> currentTerm[n],
version |-> lastPublishedVersion[n]] : ns \in Nodes }
IN
/\ messages' = messages \cup commitRequests
ELSE
UNCHANGED <<messages>>
/\ UNCHANGED <<startedJoinSinceLastReboot, lastCommittedConfiguration, currentTerm, electionWon, descendant,
lastAcceptedVersion, lastAcceptedValue, lastAcceptedTerm, lastAcceptedConfiguration,
lastPublishedVersion, lastPublishedConfiguration, joinVotes, initialConfiguration,
initialValue, initialAcceptedVersion>>
\* apply committed configuration to node n
HandleCommit(n, m) ==
/\ m.method = Commit
/\ m.term = currentTerm[n]
/\ m.term = lastAcceptedTerm[n]
/\ m.version = lastAcceptedVersion[n]
/\ (electionWon[n] => lastAcceptedVersion[n] = lastPublishedVersion[n])
/\ lastCommittedConfiguration' = [lastCommittedConfiguration EXCEPT ![n] = lastAcceptedConfiguration[n]]
/\ UNCHANGED <<currentTerm, joinVotes, messages, lastAcceptedTerm, lastAcceptedValue, startedJoinSinceLastReboot, descendant,
electionWon, lastAcceptedConfiguration, lastAcceptedVersion, lastPublishedVersion, publishVotes,
lastPublishedConfiguration, initialConfiguration, initialValue, initialAcceptedVersion>>
\* crash/restart node n (loses ephemeral state)
RestartNode(n) ==
/\ joinVotes' = [joinVotes EXCEPT ![n] = {}]
/\ startedJoinSinceLastReboot' = [startedJoinSinceLastReboot EXCEPT ![n] = FALSE]
/\ electionWon' = [electionWon EXCEPT ![n] = FALSE]
/\ lastPublishedVersion' = [lastPublishedVersion EXCEPT ![n] = 0]
/\ lastPublishedConfiguration' = [lastPublishedConfiguration EXCEPT ![n] = lastAcceptedConfiguration[n]]
/\ publishVotes' = [publishVotes EXCEPT ![n] = {}]
/\ UNCHANGED <<messages, lastAcceptedVersion, currentTerm, lastCommittedConfiguration, descendant,
lastAcceptedTerm, lastAcceptedValue, lastAcceptedConfiguration, initialConfiguration,
initialValue, initialAcceptedVersion>>
\* next-step relation
Next ==
\/ \E n \in Nodes : SetInitialState(n)
\/ \E n, nm \in Nodes : \E t \in Terms : HandleStartJoin(n, nm, t)
\/ \E m \in messages : HandleJoin(m.dest, m)
\/ \E n \in Nodes : \E t \in Terms : \E v \in Versions : \E val \in Values : \E vs \in ValidConfigs : HandleClientValue(n, t, v, val, vs)
\/ \E m \in messages : HandlePublishRequest(m.dest, m)
\/ \E m \in messages : HandlePublishResponse(m.dest, m)
\/ \E m \in messages : HandleCommit(m.dest, m)
\/ \E n \in Nodes : RestartNode(n)
----
\* Invariants
SingleNodeInvariant ==
\A n \in Nodes :
/\ lastAcceptedTerm[n] <= currentTerm[n]
/\ electionWon[n] = IsElectionQuorum(n, joinVotes[n]) \* cached value is consistent
/\ IF electionWon[n] THEN lastPublishedVersion[n] >= lastAcceptedVersion[n] ELSE lastPublishedVersion[n] = 0
/\ electionWon[n] => startedJoinSinceLastReboot[n]
/\ publishVotes[n] /= {} => electionWon[n]
OneMasterPerTerm ==
\A m1, m2 \in messages:
/\ m1.method = PublishRequest
/\ m2.method = PublishRequest
/\ m1.term = m2.term
=> m1.source = m2.source
LogMatching ==
\A m1, m2 \in messages:
/\ m1.method = PublishRequest
/\ m2.method = PublishRequest
/\ m1.term = m2.term
/\ m1.version = m2.version
=> m1.value = m2.value
CommittedPublishRequest(mp) ==
/\ mp.method = PublishRequest
/\ \E mc \in messages:
/\ mc.method = Commit
/\ mp.term = mc.term
/\ mp.version = mc.version
DescendantRelationIsStrictlyOrdered ==
\A d \in descendant:
/\ d.prevT <= d.nextT
/\ d.prevV < d.nextV
DescendantRelationIsTransitive ==
\A d1, d2 \in descendant:
d1.nextT = d2.prevT /\ d1.nextV = d2.prevV
=> [prevT |-> d1.prevT, prevV |-> d1.prevV, nextT |-> d2.nextT, nextV |-> d2.nextV] \in descendant
NewerOpsBasedOnOlderCommittedOps ==
\A m1, m2 \in messages :
/\ CommittedPublishRequest(m1)
/\ m2.method = PublishRequest
/\ m2.term >= m1.term
/\ m2.version > m1.version
=> [prevT |-> m1.term, prevV |-> m1.version, nextT |-> m2.term, nextV |-> m2.version] \in descendant
\* main invariant (follows from NewerOpsBasedOnOlderCommittedOps):
CommittedValuesDescendantsFromCommittedValues ==
\A m1, m2 \in messages :
/\ CommittedPublishRequest(m1)
/\ CommittedPublishRequest(m2)
/\ \/ m1.term /= m2.term
\/ m1.version /= m2.version
=>
\/ [prevT |-> m1.term, prevV |-> m1.version, nextT |-> m2.term, nextV |-> m2.version] \in descendant
\/ [prevT |-> m2.term, prevV |-> m2.version, nextT |-> m1.term, nextV |-> m1.version] \in descendant
CommittedValuesDescendantsFromInitialValue ==
\E v \in InitialVersions :
/\ \E n \in Nodes : v = initialAcceptedVersion[n]
/\ \E votes \in SUBSET(initialConfiguration) :
/\ IsQuorum(votes, initialConfiguration)
/\ \A n \in votes : initialAcceptedVersion[n] <= v
/\ \A m \in messages :
CommittedPublishRequest(m)
=>
[prevT |-> 0, prevV |-> v, nextT |-> m.term, nextV |-> m.version] \in descendant
CommitHasQuorumVsPreviousCommittedConfiguration ==
\A mc \in messages: mc.method = Commit
=> (\A mprq \in messages: (/\ mprq.method = PublishRequest
/\ mprq.term = mc.term
/\ mprq.version = mc.version)
=> IsQuorum({mprs.source: mprs \in {mprs \in messages: /\ mprs.method = PublishResponse
/\ mprs.term = mprq.term
/\ mprs.version = mprq.version
}}, mprq.commConf))
P2bInvariant ==
\A mc \in messages: mc.method = Commit
=> (\A mprq \in messages: mprq.method = PublishRequest
=> (mprq.term > mc.term => mprq.version > mc.version))
\* State-exploration limits
StateConstraint ==
/\ \A n \in Nodes: IF currentTerm[n] <= 1 THEN lastPublishedVersion[n] <= 2 ELSE lastPublishedVersion[n] <= 3
/\ Cardinality(messages) <= 15
====================================================================================================
================================================
FILE: ZenWithTerms/tla/ZenWithTerms.toolbox/.project
================================================
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>ZenWithTerms</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
<buildCommand>
<name>toolbox.builder.TLAParserBuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>toolbox.builder.PCalAlgorithmSearchingBuilder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>toolbox.natures.TLANature</nature>
</natures>
<linkedResources>
<link>
<name>ZenWithTerms.tla</name>
<type>1</type>
<locationURI>PARENT-1-PROJECT_LOC/ZenWithTerms.tla</locationURI>
</link>
</linkedResources>
</projectDescription>
================================================
FILE: ZenWithTerms/tla/ZenWithTerms.toolbox/.settings/org.lamport.tla.toolbox.prefs
================================================
ProjectRootFile=PARENT-1-PROJECT_LOC/ZenWithTerms.tla
eclipse.preferences.version=1
================================================
FILE: ZenWithTerms/tla/ZenWithTerms.toolbox/ZenWithTerms___model.launch
================================================
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.lamport.tla.toolbox.tool.tlc.modelCheck">
<stringAttribute key="TLCCmdLineParameters" value=""/>
<stringAttribute key="configurationName" value="model"/>
<booleanAttribute key="deferLiveness" value="false"/>
<intAttribute key="dfidDepth" value="100"/>
<booleanAttribute key="dfidMode" value="false"/>
<intAttribute key="distributedFPSetCount" value="0"/>
<stringAttribute key="distributedNetworkInterface" value="192.168.178.34"/>
<intAttribute key="distributedNodesCount" value="1"/>
<stringAttribute key="distributedTLC" value="off"/>
<stringAttribute key="distributedTLCVMArgs" value=""/>
<intAttribute key="fpBits" value="1"/>
<intAttribute key="fpIndex" value="1"/>
<intAttribute key="maxHeapSize" value="25"/>
<intAttribute key="maxSetSize" value="1000000"/>
<booleanAttribute key="mcMode" value="true"/>
<stringAttribute key="modelBehaviorInit" value="Init"/>
<stringAttribute key="modelBehaviorNext" value="Next"/>
<stringAttribute key="modelBehaviorSpec" value=""/>
<intAttribute key="modelBehaviorSpecType" value="2"/>
<stringAttribute key="modelBehaviorVars" value="lastAcceptedTerm, initialConfiguration, messages, initialValue, lastPublishedConfiguration, lastPublishedVersion, electionWon, lastCommittedConfiguration, startedJoinSinceLastReboot, publishVotes, currentTerm, lastAcceptedVersion, descendant, joinVotes, lastAcceptedConfiguration, initialAcceptedVersion, lastAcceptedValue"/>
<stringAttribute key="modelComments" value=""/>
<booleanAttribute key="modelCorrectnessCheckDeadlock" value="true"/>
<listAttribute key="modelCorrectnessInvariants">
<listEntry value="1OneMasterPerTerm"/>
<listEntry value="1LogMatching"/>
<listEntry value="1SingleNodeInvariant"/>
<listEntry value="1CommittedValuesDescendantsFromCommittedValues"/>
<listEntry value="1CommittedValuesDescendantsFromInitialValue"/>
<listEntry value="1DescendantRelationIsStrictlyOrdered"/>
<listEntry value="1NewerOpsBasedOnOlderCommittedOps"/>
<listEntry value="1CommitHasQuorumVsPreviousCommittedConfiguration"/>
<listEntry value="1P2bInvariant"/>
<listEntry value="1DescendantRelationIsTransitive"/>
</listAttribute>
<listAttribute key="modelCorrectnessProperties"/>
<stringAttribute key="modelExpressionEval" value=""/>
<stringAttribute key="modelParameterActionConstraint" value=""/>
<listAttribute key="modelParameterConstants">
<listEntry value="Join;;Join;1;0"/>
<listEntry value="Nodes;;{n1, n2, n3};1;1"/>
<listEntry value="Commit;;Commit;1;0"/>
<listEntry value="PublishResponse;;PublishResponse;1;0"/>
<listEntry value="Values;;{v1, v2};1;1"/>
<listEntry value="PublishRequest;;PublishRequest;1;0"/>
</listAttribute>
<stringAttribute key="modelParameterContraint" value="StateConstraint"/>
<listAttribute key="modelParameterDefinitions">
<listEntry value="Terms;;{0,1,2};0;0"/>
<listEntry value="Versions;;{0,1,2,3};0;0"/>
<listEntry value="InitialVersions;;{0,1,2};0;0"/>
</listAttribute>
<stringAttribute key="modelParameterModelValues" value="{}"/>
<stringAttribute key="modelParameterNewDefinitions" value=""/>
<intAttribute key="numberOfWorkers" value="2"/>
<booleanAttribute key="recover" value="false"/>
<stringAttribute key="result.mail.address" value=""/>
<intAttribute key="simuAril" value="-1"/>
<intAttribute key="simuDepth" value="100"/>
<intAttribute key="simuSeed" value="-1"/>
<stringAttribute key="specName" value="ZenWithTerms"/>
<stringAttribute key="view" value=""/>
<booleanAttribute key="visualizeStateGraph" value="false"/>
</launchConfiguration>
================================================
FILE: cluster/isabelle/Implementation.thy
================================================
section \<open>Implementation\<close>
text \<open>This section presents the implementation of the algorithm.\<close>
theory Implementation
imports Preliminaries
begin
subsection \<open>Protocol messages\<close>
text \<open>The
proven-safe core of the protocol works by sending messages as described here. The remainder of the
protocol may send other messages too, and may drop, reorder or duplicate any of these messages, but
must not send these messages itself to ensure safety. Another way of thinking of these messages is
to consider them as ``fire-and-forget'' RPC invocations that, on receipt, call some local method, maybe
update the receiving node's state, and maybe yield some further messages. The @{type nat} parameter to each
message refers to a slot number.\<close>
datatype TermOption = NO_TERM | SomeTerm Term
instantiation TermOption :: linorder
begin
fun less_TermOption :: "TermOption \<Rightarrow> TermOption \<Rightarrow> bool"
where "t < NO_TERM = False"
| "NO_TERM < SomeTerm t = True"
| "SomeTerm t\<^sub>1 < SomeTerm t\<^sub>2 = (t\<^sub>1 < t\<^sub>2)"
definition less_eq_TermOption :: "TermOption \<Rightarrow> TermOption \<Rightarrow> bool"
where "(t\<^sub>1 :: TermOption) \<le> t\<^sub>2 \<equiv> t\<^sub>1 = t\<^sub>2 \<or> t\<^sub>1 < t\<^sub>2"
instance proof
fix x y z :: TermOption
show "(x < y) = (x \<le> y \<and> \<not> y \<le> x)" unfolding less_eq_TermOption_def apply auto
using less_TermOption.elims apply fastforce
by (metis less_TermOption.elims(2) less_TermOption.simps(3) less_not_sym)
show "x \<le> x" by (simp add: less_eq_TermOption_def)
show "x \<le> y \<Longrightarrow> y \<le> z \<Longrightarrow> x \<le> z" unfolding less_eq_TermOption_def apply auto
by (metis TermOption.distinct(1) TermOption.inject dual_order.strict_trans less_TermOption.elims(2) less_TermOption.elims(3))
show "x \<le> y \<Longrightarrow> y \<le> x \<Longrightarrow> x = y" unfolding less_eq_TermOption_def apply auto
using \<open>(x < y) = (x \<le> y \<and> \<not> y \<le> x)\<close> less_eq_TermOption_def by blast
show "x \<le> y \<or> y \<le> x" unfolding less_eq_TermOption_def apply auto
by (metis TermOption.distinct(1) TermOption.inject less_TermOption.elims(3) neqE)
qed
end
lemma NO_TERM_le [simp]: "NO_TERM \<le> t" by (cases t, simp_all add: less_eq_TermOption_def)
lemma le_NO_TERM [simp]: "(t \<le> NO_TERM) = (t = NO_TERM)" by (cases t, simp_all add: less_eq_TermOption_def)
lemma le_SomeTerm [simp]: "(SomeTerm t\<^sub>1 \<le> SomeTerm t\<^sub>2) = (t\<^sub>1 \<le> t\<^sub>2)" by (auto simp add: less_eq_TermOption_def)
datatype Message
= StartJoin Term
| Vote Slot Term TermOption
| ClientValue Value
| PublishRequest Slot Term Value
| PublishResponse Slot Term
| ApplyCommit Slot Term
| CatchUpRequest
| CatchUpResponse Slot "Node set" ClusterState
| DiscardJoinVotes
| Reboot
text \<open>Some prose descriptions of these messages follows, in order to give a bit more of an
intuitive understanding of their purposes.\<close>
text \<open>The message @{term "StartJoin t"} may be sent by any node to attempt to start a master
election in the given term @{term t}.\<close>
text \<open>The message @{term "Vote i t a"} may be sent by a node in response
to a @{term StartJoin} message. It indicates that the sender knows all committed values for slots
strictly below @{term i}, and that the sender will no longer vote (i.e. send an @{term
PublishResponse}) in any term prior to @{term t}. The field @{term a} is either @{term
None} or @{term "Some t'"}. In the former case this indicates that
the node has not yet sent any @{term PublishResponse} message in slot @{term i}, and in the latter
case it indicates that the largest term in which it has previously sent an @{term PublishResponse}
message is @{term t'}. All
nodes must avoid sending a @{term Vote} message to two different masters in the same term.\<close>
text \<open>The message @{term "ClientValue x"} may be sent by any node and indicates an attempt to
reach consensus on the value @{term x}.\<close>
text \<open>The message @{term "PublishRequest i t v"} may be sent by the elected master of term
@{term t} to request the other master-eligible nodes to vote for value @{term v} to be committed in
slot @{term i}.\<close>
text \<open>The message @{term "PublishResponse i t"} may be sent by node in response to
the corresponding @{term PublishRequest} message, indicating that the sender votes for the value
proposed by the master of term @{term t} to be committed in slot @{term i}.\<close>
text \<open>The message @{term "ApplyCommit i t"} indicates that the value proposed by the master of
term @{term t} in slot @{term i} received a quorum of votes and is therefore committed.\<close>
text \<open>The message @{term Reboot} may be sent by any node to represent the restart of a node, which
loses any ephemeral state.\<close>
text \<open>The abstract model of Zen keeps track of the set of all messages that have ever been
sent, and asserts that this set obeys certain invariants, listed below. Further below, it will be
shown that these invariants imply that each slot obeys the @{term oneSlot} invariants above and
hence that each slot cannot see inconsistent committed values.\<close>
datatype Destination = Broadcast | OneNode Node
record RoutedMessage =
sender :: Node
destination :: Destination
payload :: Message
text \<open>It will be useful to be able to choose the optional term with the greater term,
so here is a function that does that.\<close>
subsection \<open>Node implementation\<close>
text \<open>Each node holds the following local data.\<close>
record TermValue =
tvTerm :: Term
tvValue :: Value
record NodeData =
currentNode :: Node
currentTerm :: Term
(* committed state *)
firstUncommittedSlot :: Slot
currentVotingNodes :: "Node set"
currentClusterState :: ClusterState
(* accepted state *)
lastAcceptedData :: "TermValue option"
(* election state *)
joinVotes :: "Node set"
electionWon :: bool
(* publish state *)
publishPermitted :: bool
publishVotes :: "Node set"
definition lastAcceptedValue :: "NodeData \<Rightarrow> Value"
where "lastAcceptedValue nd \<equiv> tvValue (THE lad. lastAcceptedData nd = Some lad)"
definition lastAcceptedTerm :: "NodeData \<Rightarrow> TermOption"
where "lastAcceptedTerm nd \<equiv> case lastAcceptedData nd of None \<Rightarrow> NO_TERM | Some lad \<Rightarrow> SomeTerm (tvTerm lad)"
definition isQuorum :: "NodeData \<Rightarrow> Node set \<Rightarrow> bool"
where "isQuorum nd q \<equiv> q \<in> majorities (currentVotingNodes nd)"
lemma lastAcceptedValue_joinVotes_update[simp]: "lastAcceptedValue (joinVotes_update f nd) = lastAcceptedValue nd" by (simp add: lastAcceptedValue_def)
lemma lastAcceptedTerm_joinVotes_update[simp]: "lastAcceptedTerm (joinVotes_update f nd) = lastAcceptedTerm nd" by (simp add: lastAcceptedTerm_def)
lemma lastAcceptedValue_electionWon_update[simp]: "lastAcceptedValue (electionWon_update f nd) = lastAcceptedValue nd" by (simp add: lastAcceptedValue_def)
lemma lastAcceptedTerm_electionWon_update[simp]: "lastAcceptedTerm (electionWon_update f nd) = lastAcceptedTerm nd" by (simp add: lastAcceptedTerm_def)
text \<open>This method publishes a value via a @{term PublishRequest} message.\<close>
definition publishValue :: "Value \<Rightarrow> NodeData \<Rightarrow> (NodeData * Message option)"
where
"publishValue x nd \<equiv>
if electionWon nd \<and> publishPermitted nd
then ( nd \<lparr> publishPermitted := False \<rparr>
, Some (PublishRequest
(firstUncommittedSlot nd)
(currentTerm nd) x) )
else (nd, None)"
text \<open>This method updates the node's current term (if necessary) and discards any data associated
with the previous term.\<close>
definition ensureCurrentTerm :: "Term \<Rightarrow> NodeData \<Rightarrow> NodeData"
where
"ensureCurrentTerm t nd \<equiv>
if t \<le> currentTerm nd
then nd
else nd
\<lparr> joinVotes := {}
, currentTerm := t
, electionWon := False
, publishPermitted := True
, publishVotes := {} \<rparr>"
text \<open>This method updates the node's state on receipt of a vote (a @{term Vote}) in an election.\<close>
definition addElectionVote :: "Node \<Rightarrow> Slot => TermOption \<Rightarrow> NodeData \<Rightarrow> NodeData"
where
"addElectionVote s i a nd \<equiv> let newVotes = insert s (joinVotes nd)
in nd \<lparr> joinVotes := newVotes
, electionWon := isQuorum nd newVotes \<rparr>"
text \<open>Clients request the cluster to achieve consensus on certain values using the @{term ClientValue}
message which is handled as follows.\<close>
definition handleClientValue :: "Value \<Rightarrow> NodeData \<Rightarrow> (NodeData * Message option)"
where
"handleClientValue x nd \<equiv> if lastAcceptedTerm nd = NO_TERM then publishValue x nd else (nd, None)"
text \<open>A @{term StartJoin} message is checked for acceptability and then handled by updating the
node's term and yielding a @{term Vote} message as follows.\<close>
definition handleStartJoin :: "Term \<Rightarrow> NodeData \<Rightarrow> (NodeData * Message option)"
where
"handleStartJoin t nd \<equiv>
if currentTerm nd < t
then ( ensureCurrentTerm t nd
, Some (Vote (firstUncommittedSlot nd)
t
(lastAcceptedTerm nd)))
else (nd, None)"
text \<open>A @{term Vote} message is checked for acceptability and then handled as follows, perhaps
yielding a @{term PublishRequest} message.\<close>
definition handleVote :: "Node \<Rightarrow> Slot \<Rightarrow> Term \<Rightarrow> TermOption \<Rightarrow> NodeData \<Rightarrow> (NodeData * Message option)"
where
"handleVote s i t a nd \<equiv>
if t = currentTerm nd
\<and> (i < firstUncommittedSlot nd
\<or> (i = firstUncommittedSlot nd \<and> a \<le> lastAcceptedTerm nd))
then let nd1 = addElectionVote s i a nd
in (if lastAcceptedTerm nd = NO_TERM then (nd1, None) else publishValue (lastAcceptedValue nd1) nd1)
else (nd, None)"
text \<open>A @{term PublishRequest} message is checked for acceptability and then handled as follows,
yielding a @{term PublishResponse} message.\<close>
definition handlePublishRequest :: "Slot \<Rightarrow> Term \<Rightarrow> Value \<Rightarrow> NodeData \<Rightarrow> (NodeData * Message option)"
where
"handlePublishRequest i t x nd \<equiv>
if i = firstUncommittedSlot nd
\<and> t = currentTerm nd
then ( nd \<lparr> lastAcceptedData := Some \<lparr> tvTerm = t, tvValue = x \<rparr> \<rparr>
, Some (PublishResponse i t))
else (nd, None)"
text \<open>This method sends an @{term ApplyCommit} message if a quorum of votes has been received.\<close>
definition commitIfQuorate :: "NodeData \<Rightarrow> (NodeData * Message option)"
where
"commitIfQuorate nd = (nd, if isQuorum nd (publishVotes nd)
then Some (ApplyCommit (firstUncommittedSlot nd) (currentTerm nd)) else None)"
text \<open>A @{term PublishResponse} message is checked for acceptability and handled as follows. If
this message, together with the previously-received messages, forms a quorum of votes then the
value is committed, yielding an @{term ApplyCommit} message.\<close>
definition handlePublishResponse :: "Node \<Rightarrow> Slot \<Rightarrow> Term \<Rightarrow> NodeData \<Rightarrow> (NodeData * Message option)"
where
"handlePublishResponse s i t nd \<equiv>
if i = firstUncommittedSlot nd \<and> t = currentTerm nd
then commitIfQuorate (nd \<lparr> publishVotes := insert s (publishVotes nd) \<rparr>)
else (nd, None)"
text \<open>This method updates the node's state when a value is committed.\<close>
definition applyAcceptedValue :: "NodeData \<Rightarrow> NodeData"
where
"applyAcceptedValue nd \<equiv> case lastAcceptedValue nd of
NoOp \<Rightarrow> nd
| Reconfigure votingNodes \<Rightarrow> nd
\<lparr> currentVotingNodes := set votingNodes
, electionWon := joinVotes nd \<in> majorities (set votingNodes) \<rparr>
| ClusterStateDiff diff \<Rightarrow> nd \<lparr> currentClusterState := diff (currentClusterState nd) \<rparr>"
text \<open>An @{term ApplyCommit} message is applied to the current node's state, updating its configuration
and \texttt{ClusterState} via the @{term applyValue} method. It yields no messages.\<close>
definition handleApplyCommit :: "Slot \<Rightarrow> Term \<Rightarrow> NodeData \<Rightarrow> NodeData"
where
"handleApplyCommit i t nd \<equiv>
if i = firstUncommittedSlot nd \<and> lastAcceptedTerm nd = SomeTerm t
then (applyAcceptedValue nd)
\<lparr> firstUncommittedSlot := i + 1
, lastAcceptedData := None
, publishPermitted := True
, publishVotes := {} \<rparr>
else nd"
definition handleCatchUpRequest :: "NodeData \<Rightarrow> (NodeData * Message option)"
where
"handleCatchUpRequest nd = (nd, Some (CatchUpResponse (firstUncommittedSlot nd)
(currentVotingNodes nd) (currentClusterState nd)))"
definition handleCatchUpResponse :: "Slot \<Rightarrow> Node set \<Rightarrow> ClusterState \<Rightarrow> NodeData \<Rightarrow> NodeData"
where
"handleCatchUpResponse i conf cs nd \<equiv>
if firstUncommittedSlot nd < i
then nd \<lparr> firstUncommittedSlot := i
, publishPermitted := True
, publishVotes := {}
, currentVotingNodes := conf
, currentClusterState := cs
, lastAcceptedData := None
, joinVotes := {}
, electionWon := False \<rparr>
else nd"
text \<open>A @{term Reboot} message simulates the effect of a reboot, discarding any ephemeral state but
preserving the persistent state. It yields no messages.\<close>
definition handleReboot :: "NodeData \<Rightarrow> NodeData"
where
"handleReboot nd \<equiv>
\<lparr> currentNode = currentNode nd
, currentTerm = currentTerm nd
, firstUncommittedSlot = firstUncommittedSlot nd
, currentVotingNodes = currentVotingNodes nd
, currentClusterState = currentClusterState nd
, lastAcceptedData = lastAcceptedData nd
, joinVotes = {}
, electionWon = False
, publishPermitted = False
, publishVotes = {} \<rparr>"
text \<open>A @{term DiscardJoinVotes} message discards the votes received by a node. It yields
no messages.\<close>
definition handleDiscardJoinVotes :: "NodeData \<Rightarrow> NodeData"
where
"handleDiscardJoinVotes nd \<equiv> nd \<lparr> electionWon := False, joinVotes := {} \<rparr>"
text \<open>This function dispatches incoming messages to the appropriate handler method, and
routes any responses to the appropriate places. In particular, @{term Vote} messages
(sent by the @{term handleStartJoin} method) and
@{term PublishResponse} messages (sent by the @{term handlePublishRequest} method) are
only sent to a single node, whereas all other responses are broadcast to all nodes.\<close>
definition ProcessMessage :: "NodeData \<Rightarrow> RoutedMessage \<Rightarrow> (NodeData * RoutedMessage option)"
where
"ProcessMessage nd msg \<equiv>
let respondTo =
(\<lambda> d (nd, mmsg). case mmsg of
None \<Rightarrow> (nd, None)
| Some msg \<Rightarrow> (nd,
Some \<lparr> sender = currentNode nd, destination = d,
payload = msg \<rparr>));
respondToSender = respondTo (OneNode (sender msg));
respondToAll = respondTo Broadcast
in
if destination msg \<in> { Broadcast, OneNode (currentNode nd) }
then case payload msg of
StartJoin t
\<Rightarrow> respondToSender (handleStartJoin t nd)
| Vote i t a
\<Rightarrow> respondToAll (handleVote (sender msg) i t a nd)
| ClientValue x
\<Rightarrow> respondToAll (handleClientValue x nd)
| PublishRequest i t x
\<Rightarrow> respondToSender (handlePublishRequest i t x nd)
| PublishResponse i t
\<Rightarrow> respondToAll (handlePublishResponse (sender msg) i t nd)
| ApplyCommit i t
\<Rightarrow> (handleApplyCommit i t nd, None)
| CatchUpRequest
\<Rightarrow> respondToSender (handleCatchUpRequest nd)
| CatchUpResponse i conf cs
\<Rightarrow> (handleCatchUpResponse i conf cs nd, None)
| DiscardJoinVotes
\<Rightarrow> (handleDiscardJoinVotes nd, None)
| Reboot
\<Rightarrow> (handleReboot nd, None)
else (nd, None)"
text \<open>Nodes are initialised to this state. The data required is the initial configuration, @{term Q\<^sub>0}
and the initial \texttt{ClusterState}, here shown as @{term "ClusterState 0"}.\<close>
definition initialNodeState :: "Node \<Rightarrow> NodeData"
where "initialNodeState n =
\<lparr> currentNode = n
, currentTerm = 0
, firstUncommittedSlot = 0
, currentVotingNodes = V\<^sub>0
, currentClusterState = CS\<^sub>0
, lastAcceptedData = None
, joinVotes = {}
, electionWon = False
, publishPermitted = False
, publishVotes = {} \<rparr>"
(* Note: publishPermitted could be True initially, but in the actual implementation we call the
same constructor whether we're starting up from afresh or recovering from a reboot, and the value
is really unimportant as we need to run an election in a new term before becoming master anyway,
so it's hard to justify putting any effort into calculating different values for these two cases.
Instead just set it to False initially.*)
end
================================================
FILE: cluster/isabelle/Monadic.thy
================================================
theory Monadic
imports Implementation "~~/src/HOL/Library/Monad_Syntax"
begin
datatype Exception = IllegalArgumentException
datatype ('e,'a) Result = Success 'a | Exception 'e
datatype 'a Action = Action "NodeData \<Rightarrow> (NodeData * RoutedMessage list * (Exception,'a) Result)"
definition runM :: "'a Action \<Rightarrow> NodeData \<Rightarrow> (NodeData * RoutedMessage list * (Exception,'a) Result)"
where "runM ma \<equiv> case ma of Action unwrapped_ma \<Rightarrow> unwrapped_ma"
lemma runM_Action[simp]: "runM (Action f) = f" by (simp add: runM_def)
lemma runM_inject[intro]: "(\<And>nd. runM ma nd = runM mb nd) \<Longrightarrow> ma = mb" by (cases ma, cases mb, auto simp add: runM_def)
definition return :: "'a \<Rightarrow> 'a Action" where "return a \<equiv> Action (\<lambda> nd. (nd, [], Success a))"
lemma runM_return[simp]: "runM (return a) nd = (nd, [], Success a)" unfolding runM_def return_def by simp
definition Action_bind :: "'a Action \<Rightarrow> ('a \<Rightarrow> 'b Action) \<Rightarrow> 'b Action"
where "Action_bind ma mf \<equiv> Action (\<lambda> nd0. case runM ma nd0 of
(nd1, msgs1, result1) \<Rightarrow> (case result1 of
Exception e \<Rightarrow> (nd1, msgs1, Exception e)
| Success a \<Rightarrow> (case runM (mf a) nd1 of
(nd2, msgs2, result2) \<Rightarrow> (nd2, msgs1 @ msgs2, result2))))"
adhoc_overloading bind Action_bind
lemma runM_bind: "runM (a \<bind> f) nd0 = (case runM a nd0 of (nd1, msgs1, result1) \<Rightarrow> (case result1 of Exception e \<Rightarrow> (nd1, msgs1, Exception e) | Success b \<Rightarrow> (case runM (f b) nd1 of (nd2, msgs2, c) \<Rightarrow> (nd2, msgs1@msgs2, c))))"
unfolding Action_bind_def by auto
lemma return_bind[simp]: "do { a' <- return a; f a' } = f a"
apply (intro runM_inject) by (simp add: runM_bind)
lemma bind_return[simp]: "do { a' <- f; return a' } = f"
proof (intro runM_inject)
fix nd
obtain nd1 msgs1 result1 where result1: "runM f nd = (nd1, msgs1, result1)" by (cases "runM f nd", blast)
show "runM (f \<bind> return) nd = runM f nd"
by (cases result1, simp_all add: runM_bind result1)
qed
lemma bind_bind_assoc[simp]:
fixes f :: "'a Action"
shows "do { b <- do { a <- f; g a }; h b } = do { a <- f; b <- g a; h b }" (is "?LHS = ?RHS")
proof (intro runM_inject)
fix nd0
show "runM ?LHS nd0 = runM ?RHS nd0"
proof (cases "runM f nd0")
case fields1: (fields nd1 msgs1 result1)
show ?thesis
proof (cases result1)
case Exception show ?thesis by (simp add: runM_bind fields1 Exception)
next
case Success1: (Success b)
show ?thesis
proof (cases "runM (g b) nd1")
case fields2: (fields nd2 msgs2 result2)
show ?thesis
proof (cases result2)
case Exception show ?thesis by (simp add: runM_bind fields1 fields2 Success1 Exception)
next
case Success2: (Success c)
show ?thesis
by (cases "runM (h c) nd2", simp add: runM_bind fields1 Success1 fields2 Success2)
qed
qed
qed
qed
qed
definition getNodeData :: "NodeData Action" where "getNodeData \<equiv> Action (\<lambda>nd. (nd, [], Success nd))"
definition setNodeData :: "NodeData \<Rightarrow> unit Action" where "setNodeData nd \<equiv> Action (\<lambda>_. (nd, [], Success ()))"
lemma runM_getNodeData[simp]: "runM getNodeData nd = (nd, [], Success nd)" by (simp add: runM_def getNodeData_def)
lemma runM_setNodeData[simp]: "runM (setNodeData nd') nd = (nd', [], Success ())" by (simp add: runM_def setNodeData_def)
lemma runM_getNodeData_continue[simp]: "runM (do { nd' <- getNodeData; f nd' }) nd = runM (f nd) nd" by (simp add: runM_bind)
lemma runM_setNodeData_continue[simp]: "runM (do { setNodeData nd'; f }) nd = runM f nd'" by (simp add: runM_bind)
definition modifyNodeData :: "(NodeData \<Rightarrow> NodeData) \<Rightarrow> unit Action" where "modifyNodeData f = getNodeData \<bind> (setNodeData \<circ> f)"
lemma runM_modifyNodeData[simp]: "runM (modifyNodeData f) nd = (f nd, [], Success ())" by (simp add: modifyNodeData_def runM_bind)
lemma runM_modifyNodeData_continue[simp]: "runM (do { modifyNodeData f; a }) nd = runM a (f nd)" by (simp add: runM_bind)
definition tell :: "RoutedMessage list \<Rightarrow> unit Action" where "tell rms \<equiv> Action (\<lambda>nd. (nd, rms, Success ()))"
lemma runM_tell[simp]: "runM (tell rms) nd = (nd, rms, Success ())" by (simp add: runM_def tell_def)
lemma runM_tell_contiue[simp]: "runM (do { tell rms; a }) nd = (let (nd, rms', x) = runM a nd in (nd, rms@rms', x))" by (simp add: runM_bind tell_def)
definition send :: "RoutedMessage \<Rightarrow> unit Action" where "send rm = tell [rm]"
definition throw :: "Exception \<Rightarrow> 'a Action" where "throw e = Action (\<lambda>nd. (nd, [], Exception e))"
lemma runM_throw[simp]: "runM (throw e) nd = (nd, [], Exception e)" by (simp add: runM_def throw_def)
lemma throw_continue[simp]: "do { throw e; a } = throw e" by (intro runM_inject, simp add: runM_bind)
definition catch :: "'a Action \<Rightarrow> (Exception \<Rightarrow> 'a Action) \<Rightarrow> 'a Action"
where "catch go onException = Action (\<lambda>nd0. case runM go nd0 of (nd1, rms1, result1) \<Rightarrow> (case result1 of Success _ \<Rightarrow> (nd1, rms1, result1) | Exception e \<Rightarrow> runM (tell rms1 \<then> onException e) nd1))"
lemma catch_throw[simp]: "catch (throw e) handle = handle e" by (intro runM_inject, simp add: catch_def)
lemma catch_return[simp]: "catch (return a) handle = return a" by (intro runM_inject, simp add: catch_def)
lemma catch_getNodeData[simp]: "catch getNodeData handle = getNodeData" by (intro runM_inject, simp add: catch_def)
lemma catch_getNodeData_continue[simp]: "catch (do { nd <- getNodeData; f nd }) handle = do { nd <- getNodeData; catch (f nd) handle }" by (intro runM_inject, simp add: catch_def)
lemma catch_setNodeData[simp]: "catch (setNodeData nd) handle = setNodeData nd" by (intro runM_inject, simp add: catch_def)
lemma catch_setNodeData_continue[simp]: "catch (do { setNodeData nd; f }) handle = do { setNodeData nd; catch f handle }" by (intro runM_inject, simp add: catch_def)
lemma catch_modifyNodeData[simp]: "catch (modifyNodeData f) handle = modifyNodeData f" by (intro runM_inject, simp add: catch_def)
lemma catch_modifyNodeData_continue[simp]: "catch (do { modifyNodeData f; g }) handle = do { modifyNodeData f; catch g handle }" by (intro runM_inject, simp add: catch_def)
lemma catch_tell[simp]: "catch (tell rms) handle = tell rms" by (intro runM_inject, simp add: catch_def)
lemma catch_tell_continue[simp]: "catch (do { tell rms; f }) handle = do { tell rms; catch f handle }"
proof (intro runM_inject)
fix nd0
show "runM (catch (do { tell rms; f }) handle) nd0 = runM (do { tell rms; catch f handle }) nd0"
proof (cases "runM f nd0")
case fields1: (fields nd1 msgs1 result1)
show ?thesis
proof (cases result1)
case (Exception e) show ?thesis by (cases "runM (handle e) nd1", simp add: catch_def fields1 Exception)
next
case Success1: (Success b)
show ?thesis
by (simp add: catch_def fields1 Success1)
qed
qed
qed
lemma catch_send[simp]: "catch (send rm) handle = send rm" by (simp add: send_def)
lemma catch_send_continue[simp]: "catch (do { send rm; f }) handle = do { send rm; catch f handle }" by (simp add: send_def)
definition gets :: "(NodeData \<Rightarrow> 'a) \<Rightarrow> 'a Action" where "gets f \<equiv> do { nd <- getNodeData; return (f nd) }"
definition getCurrentClusterState where "getCurrentClusterState = gets currentClusterState"
definition getCurrentNode where "getCurrentNode = gets currentNode"
definition getCurrentTerm where "getCurrentTerm = gets currentTerm"
definition getCurrentVotingNodes where "getCurrentVotingNodes = gets currentVotingNodes"
definition getElectionWon where "getElectionWon = gets electionWon"
definition getFirstUncommittedSlot where "getFirstUncommittedSlot = gets firstUncommittedSlot"
definition getJoinVotes where "getJoinVotes = gets joinVotes"
definition getLastAcceptedData where "getLastAcceptedData = gets lastAcceptedData"
definition getPublishPermitted where "getPublishPermitted = gets publishPermitted"
definition getPublishVotes where "getPublishVotes = gets publishVotes"
definition sets where "sets f x = modifyNodeData (f (\<lambda>_. x))"
definition setCurrentClusterState where "setCurrentClusterState = sets currentClusterState_update"
definition setCurrentNode where "setCurrentNode = sets currentNode_update"
definition setCurrentTerm where "setCurrentTerm = sets currentTerm_update"
definition setCurrentVotingNodes where "setCurrentVotingNodes = sets currentVotingNodes_update"
definition setElectionWon where "setElectionWon = sets electionWon_update"
definition setFirstUncommittedSlot where "setFirstUncommittedSlot = sets firstUncommittedSlot_update"
definition setJoinVotes where "setJoinVotes = sets joinVotes_update"
definition setLastAcceptedData where "setLastAcceptedData = sets lastAcceptedData_update"
definition setPublishPermitted where "setPublishPermitted = sets publishPermitted_update"
definition setPublishVotes where "setPublishVotes = sets publishVotes_update"
definition modifies where "modifies f g = modifyNodeData (f g)"
definition modifyJoinVotes where "modifyJoinVotes = modifies joinVotes_update"
definition modifyPublishVotes where "modifyPublishVotes = modifies publishVotes_update"
definition modifyCurrentClusterState where "modifyCurrentClusterState = modifies currentClusterState_update"
definition "when" :: "bool \<Rightarrow> unit Action \<Rightarrow> unit Action" where "when c a \<equiv> if c then a else return ()"
definition unless :: "bool \<Rightarrow> unit Action \<Rightarrow> unit Action" where "unless \<equiv> when \<circ> Not"
lemma runM_when: "runM (when c a) nd = (if c then runM a nd else (nd, [], Success ()))"
by (auto simp add: when_def)
lemma runM_unless: "runM (unless c a) nd = (if c then (nd, [], Success ()) else runM a nd)"
by (auto simp add: unless_def when_def)
lemma runM_when_continue: "runM (do { when c a; b }) nd = (if c then runM (do {a;b}) nd else runM b nd)"
by (auto simp add: when_def)
lemma runM_unless_continue: "runM (do { unless c a; b }) nd = (if c then runM b nd else runM (do {a;b}) nd)"
by (auto simp add: unless_def when_def)
lemma catch_when[simp]: "catch (when c a) onException = when c (catch a onException)"
by (intro runM_inject, simp add: catch_def runM_when)
lemma catch_unless[simp]: "catch (unless c a) onException = unless c (catch a onException)"
by (intro runM_inject, simp add: catch_def runM_unless)
lemma catch_when_continue[simp]: "catch (do { when c a; b }) onException = (if c then catch (do {a;b}) onException else catch b onException)"
by (intro runM_inject, simp add: catch_def runM_when_continue)
lemma catch_unless_continue[simp]: "catch (do { unless c a; b }) onException = (if c then catch b onException else catch (do {a;b}) onException)"
by (intro runM_inject, simp add: catch_def runM_unless_continue)
definition ensureCorrectDestination :: "Destination \<Rightarrow> unit Action"
where "ensureCorrectDestination d \<equiv> do {
n <- getCurrentNode;
when (d \<notin> { Broadcast, OneNode n }) (throw IllegalArgumentException)
}"
lemma runM_ensureCorrectDestination_continue:
"runM (do { ensureCorrectDestination d; go }) nd = (if d \<in> { Broadcast, OneNode (currentNode nd) } then runM go nd else (nd, [], Exception IllegalArgumentException))"
by (simp add: ensureCorrectDestination_def getCurrentNode_def gets_def runM_when_continue)
definition broadcast :: "Message \<Rightarrow> unit Action"
where "broadcast msg \<equiv> do {
n <- getCurrentNode;
send \<lparr> sender = n, destination = Broadcast, payload = msg \<rparr>
}"
lemma runM_broadcast[simp]: "runM (broadcast msg) nd = (nd, [\<lparr> sender = currentNode nd, destination = Broadcast, payload = msg \<rparr>], Success ())"
by (simp add: broadcast_def getCurrentNode_def gets_def send_def)
definition sendTo :: "Node \<Rightarrow> Message \<Rightarrow> unit Action"
where "sendTo d msg \<equiv> do {
n <- getCurrentNode;
send \<lparr> sender = n, destination = OneNode d, payload = msg \<rparr>
}"
lemma runM_sendTo[simp]: "runM (sendTo d msg) nd = (nd, [\<lparr> sender = currentNode nd, destination = OneNode d, payload = msg \<rparr>], Success ())"
by (simp add: sendTo_def getCurrentNode_def gets_def send_def)
definition ignoringExceptions :: "unit Action \<Rightarrow> unit Action" where "ignoringExceptions go \<equiv> catch go (\<lambda>_. return ())"
lemma None_lt[simp]: "NO_TERM < t = (t \<noteq> NO_TERM)" by (cases t, simp_all)
definition getLastAcceptedTerm :: "TermOption Action"
where
"getLastAcceptedTerm \<equiv> do {
lastAcceptedData <- getLastAcceptedData;
case lastAcceptedData of
None \<Rightarrow> return NO_TERM
| Some tv \<Rightarrow> return (SomeTerm (tvTerm tv))
}"
definition doStartJoin :: "Node \<Rightarrow> Term \<Rightarrow> unit Action"
where
"doStartJoin newMaster newTerm \<equiv> do {
currentTerm <- getCurrentTerm;
when (newTerm \<le> currentTerm) (throw IllegalArgumentException);
setCurrentTerm newTerm;
setJoinVotes {};
setElectionWon False;
setPublishPermitted True;
setPublishVotes {};
firstUncommittedSlot <- getFirstUncommittedSlot;
lastAcceptedTerm <- getLastAcceptedTerm;
sendTo newMaster (Vote firstUncommittedSlot newTerm lastAcceptedTerm)
}"
definition doVote :: "Node \<Rightarrow> Slot \<Rightarrow> Term \<Rightarrow> TermOption \<Rightarrow> unit Action"
where
"doVote sourceNode voteFirstUncommittedSlot voteTerm voteLastAcceptedTerm \<equiv> do {
currentTerm <- getCurrentTerm;
when (voteTerm \<noteq> currentTerm) (throw IllegalArgumentException);
firstUncommittedSlot <- getFirstUncommittedSlot;
when (voteFirstUncommittedSlot > firstUncommittedSlot) (throw IllegalArgumentException);
lastAcceptedTerm <- getLastAcceptedTerm;
when (voteFirstUncommittedSlot = firstUncommittedSlot
\<and> voteLastAcceptedTerm > lastAcceptedTerm)
(throw IllegalArgumentException);
modifyJoinVotes (insert sourceNode);
joinVotes <- getJoinVotes;
currentVotingNodes <- getCurrentVotingNodes;
let electionWon' = card (joinVotes \<inter> currentVotingNodes) * 2 > card currentVotingNodes;
setElectionWon electionWon';
publishPermitted <- getPublishPermitted;
when (electionWon' \<and> publishPermitted \<and> lastAcceptedTerm \<noteq> NO_TERM) (do {
setPublishPermitted False;
lastAcceptedValue <- gets lastAcceptedValue; (* NB must be present since lastAcceptedTermInSlot \<noteq> NO_TERM *)
broadcast (PublishRequest firstUncommittedSlot currentTerm lastAcceptedValue)
})
}"
definition doPublishRequest :: "Node \<Rightarrow> Slot \<Rightarrow> TermValue \<Rightarrow> unit Action"
where
"doPublishRequest sourceNode requestSlot newAcceptedState \<equiv> do {
currentTerm <- getCurrentTerm;
when (tvTerm newAcceptedState \<noteq> currentTerm) (throw IllegalArgumentException);
firstUncommittedSlot <- getFirstUncommittedSlot;
when (requestSlot \<noteq> firstUncommittedSlot) (throw IllegalArgumentException);
setLastAcceptedData (Some newAcceptedState);
sendTo sourceNode (PublishResponse requestSlot (tvTerm newAcceptedState))
}"
record SlotTerm =
stSlot :: Slot
stTerm :: Term
definition ApplyCommitFromSlotTerm :: "SlotTerm \<Rightarrow> Message"
where "ApplyCommitFromSlotTerm st = ApplyCommit (stSlot st) (stTerm st)"
definition doPublishResponse :: "Node \<Rightarrow> SlotTerm \<Rightarrow> unit Action"
where
"doPublishResponse sourceNode slotTerm \<equiv> do {
currentTerm <- getCurrentTerm;
when (stTerm slotTerm \<noteq> currentTerm) (throw IllegalArgumentException);
firstUncommittedSlot <- getFirstUncommittedSlot;
when (stSlot slotTerm \<noteq> firstUncommittedSlot) (throw IllegalArgumentException);
modifyPublishVotes (insert sourceNode);
publishVotes <- getPublishVotes;
currentVotingNodes <- getCurrentVotingNodes;
when (card (publishVotes \<inter> currentVotingNodes) * 2 > card currentVotingNodes)
(broadcast (ApplyCommitFromSlotTerm slotTerm))
}"
definition doCommit :: "SlotTerm \<Rightarrow> unit Action"
where
"doCommit slotTerm \<equiv> do {
lastAcceptedTermInSlot <- getLastAcceptedTerm;
when (SomeTerm (stTerm slotTerm) \<noteq> lastAcceptedTermInSlot) (throw IllegalArgumentException);
firstUncommittedSlot <- getFirstUncommittedSlot;
when (stSlot slotTerm \<noteq> firstUncommittedSlot) (throw IllegalArgumentException);
lastAcceptedValue <- gets lastAcceptedValue; (* NB must be not None since lastAcceptedTerm = Some t *)
(case lastAcceptedValue of
ClusterStateDiff diff
\<Rightarrow> modifyCurrentClusterState diff
| Reconfigure votingNodes \<Rightarrow> do {
setCurrentVotingNodes (set votingNodes);
joinVotes <- getJoinVotes;
setElectionWon (card (joinVotes \<inter> (set votingNodes)) * 2 > card (set votingNodes))
}
| NoOp \<Rightarrow> return ());
setFirstUncommittedSlot (firstUncommittedSlot + 1);
setLastAcceptedData None;
setPublishPermitted True;
setPublishVotes {}
}"
definition generateCatchup :: "Node \<Rightarrow> unit Action"
where
"generateCatchup sourceNode \<equiv> do {
firstUncommittedSlot <- getFirstUncommittedSlot;
currentVotingNodes <- getCurrentVotingNodes;
currentClusterState <- getCurrentClusterState;
sendTo sourceNode (CatchUpResponse firstUncommittedSlot currentVotingNodes currentClusterState)
}"
definition applyCatchup :: "Slot \<Rightarrow> Node set \<Rightarrow> ClusterState \<Rightarrow> unit Action"
where
"applyCatchup catchUpSlot catchUpConfiguration catchUpState \<equiv> do {
firstUncommittedSlot <- getFirstUncommittedSlot;
when (catchUpSlot \<le> firstUncommittedSlot) (throw IllegalArgumentException);
setFirstUncommittedSlot catchUpSlot;
setCurrentVotingNodes catchUpConfiguration;
setCurrentClusterState catchUpState;
setLastAcceptedData None;
setJoinVotes {};
setElectionWon False;
setPublishVotes {};
setPublishPermitted True
}"
definition doClientValue :: "Value \<Rightarrow> unit Action"
where
"doClientValue x \<equiv> do {
electionWon <- getElectionWon;
when (\<not> electionWon) (throw IllegalArgumentException);
publishPermitted <- getPublishPermitted;
when (\<not> publishPermitted) (throw IllegalArgumentException);
lastAcceptedTermInSlot <- getLastAcceptedTerm;
when (lastAcceptedTermInSlot \<noteq> NO_TERM) (throw IllegalArgumentException);
setPublishPermitted False;
currentTerm <- getCurrentTerm;
firstUncommittedSlot <- getFirstUncommittedSlot;
broadcast (PublishRequest firstUncommittedSlot currentTerm x)
}"
definition doDiscardJoinVotes :: "unit Action"
where
"doDiscardJoinVotes \<equiv> do {
setJoinVotes {};
setElectionWon False
}"
definition doReboot :: "unit Action"
where
"doReboot \<equiv> modifyNodeData (\<lambda>nd.
(* persistent fields *)
\<lparr> currentNode = currentNode nd
, currentTerm = currentTerm nd
, firstUncommittedSlot = firstUncommittedSlot nd
, currentVotingNodes = currentVotingNodes nd
, currentClusterState = currentClusterState nd
, lastAcceptedData = lastAcceptedData nd
(* transient fields *)
, joinVotes = {}
, electionWon = False
, publishPermitted = False
, publishVotes = {} \<rparr>)"
definition ProcessMessageAction :: "RoutedMessage \<Rightarrow> unit Action"
where "ProcessMessageAction rm \<equiv> Action (\<lambda>nd. case ProcessMessage nd rm of (nd', messageOption) \<Rightarrow> (nd', case messageOption of None \<Rightarrow> [] | Some m \<Rightarrow> [m], Success ()))"
definition dispatchMessageInner :: "RoutedMessage \<Rightarrow> unit Action"
where "dispatchMessageInner m \<equiv> case payload m of
StartJoin t \<Rightarrow> doStartJoin (sender m) t
| Vote i t a \<Rightarrow> doVote (sender m) i t a
| ClientValue x \<Rightarrow> doClientValue x
| PublishRequest i t x \<Rightarrow> doPublishRequest (sender m) i \<lparr> tvTerm = t, tvValue = x \<rparr>
| PublishResponse i t \<Rightarrow> doPublishResponse (sender m) \<lparr> stSlot = i, stTerm = t \<rparr>
| ApplyCommit i t \<Rightarrow> doCommit \<lparr> stSlot = i, stTerm = t \<rparr>
| CatchUpRequest \<Rightarrow> generateCatchup (sender m)
| CatchUpResponse i conf cs \<Rightarrow> applyCatchup i conf cs
| DiscardJoinVotes \<Rightarrow> doDiscardJoinVotes
| Reboot \<Rightarrow> doReboot"
definition dispatchMessage :: "RoutedMessage \<Rightarrow> unit Action"
where "dispatchMessage m \<equiv> ignoringExceptions (do {
ensureCorrectDestination (destination m);
dispatchMessageInner m
})"
lemma getLastAcceptedTermInSlot_gets[simp]: "getLastAcceptedTerm = gets lastAcceptedTerm"
proof (intro runM_inject)
fix nd
show "runM getLastAcceptedTerm nd = runM (gets lastAcceptedTerm) nd"
by (cases "lastAcceptedData nd", simp_all add: gets_def getLastAcceptedTerm_def getLastAcceptedData_def
getFirstUncommittedSlot_def lastAcceptedTerm_def)
qed
lemma monadic_implementation_is_faithful:
"dispatchMessage = ProcessMessageAction"
proof (intro ext runM_inject)
fix rm nd
show "runM (dispatchMessage rm) nd = runM (ProcessMessageAction rm) nd" (is "?LHS = ?RHS")
proof (cases "destination rm \<in> {Broadcast, OneNode (currentNode nd)}")
case False
hence 1: "\<And>f. runM (do { ensureCorrectDestination (destination rm); f }) nd = (nd, [], Exception IllegalArgumentException)"
by (simp add: runM_ensureCorrectDestination_continue)
from False
show ?thesis
unfolding ProcessMessageAction_def dispatchMessage_def
by (simp add: ignoringExceptions_def catch_def 1 ProcessMessage_def)
next
case dest_ok: True
hence 1: "runM (dispatchMessage rm) nd = runM (ignoringExceptions (dispatchMessageInner rm)) nd"
by (simp add: dispatchMessage_def ignoringExceptions_def catch_def runM_ensureCorrectDestination_continue)
also have "... = runM (ProcessMessageAction rm) nd" (is "?LHS = ?RHS")
proof (cases "payload rm")
case (StartJoin t)
have "?LHS = runM (ignoringExceptions (doStartJoin (sender rm) t)) nd" (is "_ = ?STEP")
by (simp add: dispatchMessageInner_def StartJoin)
also consider
(a) "t \<le> currentTerm nd"
| (b) "currentTerm nd < t" "case lastAcceptedTerm nd of NO_TERM \<Rightarrow> False | SomeTerm x \<Rightarrow> t \<le> x"
| (c) "currentTerm nd < t" "case lastAcceptedTerm nd of NO_TERM \<Rightarrow> True | SomeTerm x \<Rightarrow> x < t"
proof (cases "t \<le> currentTerm nd")
case True thus ?thesis by (intro a)
next
case 1: False
with b c show ?thesis
by (cases "case lastAcceptedTerm nd of NO_TERM \<Rightarrow> False | SomeTerm x \<Rightarrow> t \<le> x", auto, cases "lastAcceptedTerm nd", auto)
qed
hence "?STEP = ?RHS"
proof cases
case a
thus ?thesis
by (simp add: StartJoin ProcessMessageAction_def dispatchMessage_def ProcessMessage_def Let_def runM_unless
doStartJoin_def getCurrentTerm_def gets_def setJoinVotes_def sets_def setCurrentTerm_def
setPublishPermitted_def setPublishVotes_def getFirstUncommittedSlot_def handleStartJoin_def ensureCurrentTerm_def setElectionWon_def
ignoringExceptions_def catch_def runM_when_continue)
next
case b
with StartJoin dest_ok show ?thesis
by (cases "lastAcceptedTerm nd ", simp_all add: ProcessMessageAction_def dispatchMessage_def ProcessMessage_def Let_def
doStartJoin_def getCurrentTerm_def gets_def setJoinVotes_def sets_def setCurrentTerm_def runM_unless lastAcceptedTerm_def
setPublishPermitted_def setPublishVotes_def getFirstUncommittedSlot_def handleStartJoin_def ensureCurrentTerm_def setElectionWon_def
ignoringExceptions_def catch_def runM_when_continue)
next
case c with StartJoin dest_ok show ?thesis
by (cases "lastAcceptedTerm nd", simp_all add: ProcessMessageAction_def dispatchMessage_def ProcessMessage_def Let_def
doStartJoin_def getCurrentTerm_def gets_def setJoinVotes_def sets_def setCurrentTerm_def runM_unless lastAcceptedTerm_def
setPublishPermitted_def setPublishVotes_def getFirstUncommittedSlot_def handleStartJoin_def ensureCurrentTerm_def setElectionWon_def
ignoringExceptions_def catch_def runM_when_continue)
qed
finally show ?thesis by simp
next
case (Vote i t a)
have "?LHS = runM (ignoringExceptions (doVote (sender rm) i t a)) nd" (is "_ = ?STEP")
by (simp add: dispatchMessageInner_def Vote)
also have "... = ?RHS"
proof (cases "firstUncommittedSlot nd < i")
case True
with Vote dest_ok show ?thesis
by (simp add: dispatchMessage_def runM_unless
doVote_def gets_def getFirstUncommittedSlot_def ProcessMessage_def
ProcessMessageAction_def handleVote_def ignoringExceptions_def getCurrentTerm_def)
next
case False hence le: "i \<le> firstUncommittedSlot nd" by simp
show ?thesis
proof (cases "t = currentTerm nd")
case False
with Vote dest_ok le show ?thesis
by (simp add: dispatchMessage_def runM_when runM_unless
doVote_def gets_def getFirstUncommittedSlot_def getCurrentTerm_def
ProcessMessage_def ProcessMessageAction_def handleVote_def ignoringExceptions_def)
next
case t: True
show ?thesis
proof (cases "i = firstUncommittedSlot nd")
case False
with Vote dest_ok le t show ?thesis
by (simp add: dispatchMessage_def Let_def runM_when_continue
doVote_def runM_when runM_unless
gets_def getFirstUncommittedSlot_def getCurrentTerm_def
getJoinVotes_def getCurrentVotingNodes_def
getPublishPermitted_def ignoringExceptions_def broadcast_def getCurrentNode_def
modifies_def modifyJoinVotes_def send_def
sets_def setElectionWon_def setPublishPermitted_def lastAcceptedValue_def
ProcessMessage_def ProcessMessageAction_def handleVote_def
addElectionVote_def publishValue_def isQuorum_def majorities_def)
next
case i: True
show ?thesis
proof (cases a)
case a: NO_TERM
show ?thesis
proof (cases "isQuorum nd (insert (sender rm) (joinVotes nd))")
case not_quorum: False
hence not_quorum_card: "\<not> card (currentVotingNodes nd) < card (insert (sender rm) (joinVotes nd) \<inter> currentVotingNodes nd) * 2"
by (simp add: isQuorum_def majorities_def)
have "?STEP = (nd\<lparr>electionWon := False, joinVotes := insert (sender rm) (joinVotes nd)\<rparr>, [], Success ())"
by (simp add: ignoringExceptions_def i t a doVote_def catch_def
gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def
modifyJoinVotes_def modifies_def getJoinVotes_def
getCurrentVotingNodes_def Let_def setElectionWon_def sets_def runM_when
not_quorum_card getPublishPermitted_def)
also from dest_ok have "... = ?RHS"
by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def
i t a addElectionVote_def not_quorum publishValue_def Let_def)
finally show ?thesis .
next
case quorum: True
hence quorum_card: "card (currentVotingNodes nd) < card (insert (sender rm) (joinVotes nd) \<inter> currentVotingNodes nd) * 2"
by (simp add: isQuorum_def majorities_def)
show ?thesis
proof (cases "publishPermitted nd \<and> lastAcceptedTerm nd \<noteq> NO_TERM")
case False
hence "?STEP = (nd\<lparr>electionWon := True, joinVotes := insert (sender rm) (joinVotes nd)\<rparr>, [], Success ())"
by (auto simp add: ignoringExceptions_def i t a doVote_def catch_def
gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def
modifyJoinVotes_def modifies_def getJoinVotes_def
getCurrentVotingNodes_def Let_def setElectionWon_def sets_def runM_when
quorum_card getPublishPermitted_def)
also from False dest_ok have "... = ?RHS"
by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def
i t a addElectionVote_def quorum publishValue_def Let_def)
finally show ?thesis .
next
case True
hence "?STEP = (nd\<lparr>electionWon := True, publishPermitted := False,
joinVotes := insert (sender rm) (joinVotes nd)\<rparr>,
[\<lparr>sender = currentNode nd, destination = Broadcast,
payload = PublishRequest (firstUncommittedSlot nd) (currentTerm nd)
(lastAcceptedValue nd) \<rparr>], Success ())"
by (auto simp add: ignoringExceptions_def i t a doVote_def catch_def
gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def
modifyJoinVotes_def modifies_def getJoinVotes_def
getCurrentVotingNodes_def Let_def setElectionWon_def sets_def runM_when
quorum_card getPublishPermitted_def setPublishPermitted_def lastAcceptedValue_def)
also from True dest_ok have "... = ?RHS"
by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def
i t a addElectionVote_def quorum publishValue_def Let_def lastAcceptedValue_def)
finally show ?thesis .
qed
qed
next
case a: (SomeTerm voteLastAcceptedTerm)
show ?thesis
proof (cases "lastAcceptedTerm nd")
case lat: NO_TERM
have "?STEP = (nd, [], Success ())"
by (auto simp add: ignoringExceptions_def i t a lat doVote_def catch_def
gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def)
also from dest_ok have "... = ?RHS"
by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def
i t a lat)
finally show ?thesis .
next
case lat: (SomeTerm nodeLastAcceptedTerm)
show ?thesis
proof (cases "voteLastAcceptedTerm \<le> nodeLastAcceptedTerm")
case False
hence "?STEP = (nd, [], Success ())"
by (simp add: ignoringExceptions_def i t a lat doVote_def catch_def
gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def)
also from False dest_ok have "... = ?RHS"
by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def
i t a lat max_def addElectionVote_def publishValue_def)
finally show ?thesis by simp
next
case True
show ?thesis
proof (cases "isQuorum nd (insert (sender rm) (joinVotes nd))")
case not_quorum: False
hence not_quorum_card: "\<not> card (currentVotingNodes nd) < card (insert (sender rm) (joinVotes nd) \<inter> currentVotingNodes nd) * 2"
by (simp add: isQuorum_def majorities_def)
from True
have "?STEP = (nd\<lparr>electionWon := False,
joinVotes := insert (sender rm) (joinVotes nd)\<rparr>, [], Success ())"
by (simp add: ignoringExceptions_def i t a lat doVote_def catch_def
gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def
modifyJoinVotes_def modifies_def getJoinVotes_def
getCurrentVotingNodes_def Let_def setElectionWon_def sets_def runM_when
not_quorum_card getPublishPermitted_def)
also from dest_ok True have "... = ?RHS"
by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def
i t a lat max_def addElectionVote_def not_quorum publishValue_def Let_def)
finally show ?thesis .
next
case quorum: True
hence quorum_card: "card (currentVotingNodes nd) < card (insert (sender rm) (joinVotes nd) \<inter> currentVotingNodes nd) * 2"
by (simp add: isQuorum_def majorities_def)
show ?thesis
proof (cases "publishPermitted nd")
case False
with True
have "?STEP = (nd\<lparr>electionWon := True,
joinVotes := insert (sender rm) (joinVotes nd)\<rparr>, [], Success ())"
by (simp add: ignoringExceptions_def i t a lat doVote_def catch_def
gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def
modifyJoinVotes_def modifies_def getJoinVotes_def
getCurrentVotingNodes_def Let_def setElectionWon_def sets_def runM_when
quorum_card getPublishPermitted_def setPublishPermitted_def)
also from False dest_ok have "... = ?RHS"
by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def
i t a lat True addElectionVote_def quorum publishValue_def Let_def)
finally show ?thesis .
next
case publishPermitted: True
have "?STEP = (nd\<lparr>electionWon := True, publishPermitted := False,
joinVotes := insert (sender rm) (joinVotes nd)\<rparr>,
[\<lparr>sender = currentNode nd, destination = Broadcast,
payload = PublishRequest (firstUncommittedSlot nd) (currentTerm nd)
(lastAcceptedValue nd) \<rparr>], Success ())"
apply (auto simp add: ignoringExceptions_def i t a lat True doVote_def catch_def
gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def
modifyJoinVotes_def modifies_def getJoinVotes_def
getCurrentVotingNodes_def Let_def setElectionWon_def sets_def runM_when
quorum_card getPublishPermitted_def setPublishPermitted_def lastAcceptedValue_def)
using True publishPermitted by auto
also from publishPermitted True dest_ok have "... = ?RHS"
by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def
i t a lat True addElectionVote_def quorum publishValue_def Let_def lastAcceptedValue_def)
finally show ?thesis .
qed
qed
qed
qed
qed
qed
qed
qed
finally show ?thesis .
next
case (ClientValue x)
with dest_ok show ?thesis
by (simp add: ProcessMessageAction_def dispatchMessageInner_def
doClientValue_def gets_def getElectionWon_def
runM_unless getPublishPermitted_def setPublishPermitted_def sets_def
getCurrentTerm_def getFirstUncommittedSlot_def ProcessMessage_def handleClientValue_def
publishValue_def runM_when ignoringExceptions_def ClientValue catch_def runM_when_continue)
next
case (PublishRequest i t x) with dest_ok show ?thesis
by (simp add: ProcessMessageAction_def dispatchMessageInner_def
doPublishRequest_def gets_def getCurrentTerm_def getFirstUncommittedSlot_def
sets_def setLastAcceptedData_def ignoringExceptions_def catch_def runM_when_continue
getCurrentNode_def runM_unless send_def
ProcessMessage_def handlePublishRequest_def runM_when)
next
case (PublishResponse i t) with dest_ok show ?thesis
by (simp add: ProcessMessageAction_def dispatchMessageInner_def
doPublishResponse_def gets_def getCurrentTerm_def getFirstUncommittedSlot_def
broadcast_def getCurrentNode_def runM_unless send_def
modifyPublishVotes_def modifies_def getPublishVotes_def getCurrentVotingNodes_def
runM_when ignoringExceptions_def catch_def runM_when_continue
ProcessMessage_def handlePublishResponse_def commitIfQuorate_def isQuorum_def majorities_def
ApplyCommitFromSlotTerm_def)
next
case (ApplyCommit i t)
show ?thesis
proof (cases "lastAcceptedValue nd")
case NoOp
with ApplyCommit dest_ok show ?thesis
by (simp add: ProcessMessageAction_def dispatchMessageInner_def
doCommit_def runM_unless runM_when
gets_def getFirstUncommittedSlot_def
sets_def setFirstUncommittedSlot_def setLastAcceptedData_def
setPublishPermitted_def setPublishVotes_def
ProcessMessage_def handleApplyCommit_def applyAcceptedValue_def
ignoringExceptions_def catch_def runM_when_continue)
next
case Reconfigure
with ApplyCommit dest_ok show ?thesis
by (simp add: ProcessMessageAction_def dispatchMessageInner_def
doCommit_def runM_unless runM_when
gets_def getFirstUncommittedSlot_def
getJoinVotes_def
sets_def setFirstUncommittedSlot_def setLastAcceptedData_def
setPublishPermitted_def setPublishVotes_def
setCurrentVotingNodes_def setElectionWon_def
ProcessMessage_def handleApplyCommit_def applyAcceptedValue_def majorities_def
ignoringExceptions_def catch_def runM_when_continue)
next
case ClusterStateDiff
with ApplyCommit dest_ok show ?thesis
by (simp add: ProcessMessageAction_def dispatchMessageInner_def
doCommit_def runM_unless runM_when
gets_def getFirstUncommittedSlot_def
sets_def setFirstUncommittedSlot_def
modifies_def modifyCurrentClusterState_def
setPublishPermitted_def setPublishVotes_def setLastAcceptedData_def
ProcessMessage_def handleApplyCommit_def applyAcceptedValue_def
ignoringExceptions_def catch_def runM_when_continue)
qed
next
case CatchUpRequest
with dest_ok show ?thesis
by (simp add: ProcessMessageAction_def dispatchMessageInner_def
generateCatchup_def
gets_def getFirstUncommittedSlot_def getCurrentVotingNodes_def getCurrentClusterState_def
ProcessMessage_def handleCatchUpRequest_def ignoringExceptions_def catch_def runM_when_continue)
next
case (CatchUpResponse i conf cs)
with dest_ok show ?thesis
by (simp add: ProcessMessageAction_def dispatchMessageInner_def
applyCatchup_def gets_def getFirstUncommittedSlot_def
sets_def setFirstUncommittedSlot_def
setPublishPermitted_def setPublishVotes_def setLastAcceptedData_def
setCurrentVotingNodes_def setCurrentClusterState_def setJoinVotes_def
setElectionWon_def runM_unless
ProcessMessage_def handleCatchUpResponse_def
ignoringExceptions_def catch_def runM_when_continue)
next
case Reboot
with dest_ok show ?thesis
by (simp add: ProcessMessageAction_def dispatchMessageInner_def
doReboot_def ProcessMessage_def handleReboot_def ignoringExceptions_def catch_def runM_when_continue)
next
case DiscardJoinVotes
with dest_ok show ?thesis
by (simp add: ProcessMessageAction_def dispatchMessageInner_def
doDiscardJoinVotes_def ProcessMessage_def handleDiscardJoinVotes_def ignoringExceptions_def catch_def
runM_when_continue setJoinVotes_def sets_def setElectionWon_def)
qed
finally show ?thesis .
qed
qed
end
================================================
FILE: cluster/isabelle/OneSlot.thy
================================================
section \<open>One-slot consistency\<close>
text \<open>The replicated state machine determines the values that are committed in each of a sequence
of \textit{slots}. Each slot runs a logically-separate consensus algorithm which is shown to be
consistent here. Further below, the protocol is shown to refine this slot-by-slot model correctly.\<close>
text \<open>Consistency is shown to follow from the invariants listed below. Further below, the protocol
is shown to preserve these invariants in each step, which means it is not enormously important
to understand these in detail.\<close>
theory OneSlot
imports Preliminaries
begin
locale oneSlot =
(* basic functions *)
fixes Q :: "Node set set"
fixes v :: "Term \<Rightarrow> Value"
(* message-sent predicates *)
fixes promised\<^sub>f :: "Node \<Rightarrow> Term \<Rightarrow> bool"
fixes promised\<^sub>b :: "Node \<Rightarrow> Term \<Rightarrow> Term \<Rightarrow> bool"
fixes proposed :: "Term \<Rightarrow> bool"
fixes accepted :: "Node \<Rightarrow> Term \<Rightarrow> bool"
fixes committed :: "Term \<Rightarrow> bool"
(* other definitions *)
fixes promised :: "Node \<Rightarrow> Term \<Rightarrow> bool"
defines "promised n t \<equiv> promised\<^sub>f n t \<or> (\<exists> t'. promised\<^sub>b n t t')"
fixes prevAccepted :: "Term \<Rightarrow> Node set \<Rightarrow> Term set"
defines "prevAccepted t ns \<equiv> {t'. \<exists> n \<in> ns. promised\<^sub>b n t t'}"
(* invariants *)
assumes Q_intersects: "Q \<frown> Q"
assumes promised\<^sub>f: "\<lbrakk> promised\<^sub>f n t; t' < t \<rbrakk> \<Longrightarrow> \<not> accepted n t'"
assumes promised\<^sub>b_lt: "promised\<^sub>b n t t' \<Longrightarrow> t' < t"
assumes promised\<^sub>b_accepted: "promised\<^sub>b n t t' \<Longrightarrow> accepted n t'"
assumes promised\<^sub>b_max: "\<lbrakk> promised\<^sub>b n t t'; t' < t''; t'' < t \<rbrakk>
\<Longrightarrow> \<not> accepted n t''"
assumes proposed: "proposed t
\<Longrightarrow> \<exists> q \<in> Q. (\<forall> n \<in> q. promised n t)
\<and> (prevAccepted t q = {}
\<or> (\<exists> t'. v t = v t' \<and> maxTerm (prevAccepted t q) \<le> t' \<and> proposed t' \<and> t' < t))"
assumes proposed_finite: "finite {t. proposed t}"
assumes accepted: "accepted n t \<Longrightarrow> proposed t"
assumes committed: "committed t \<Longrightarrow> \<exists> q \<in> Q. \<forall> n \<in> q. accepted n t"
lemma (in oneSlot) prevAccepted_proposed: "prevAccepted t ns \<subseteq> {t. proposed t}"
using accepted prevAccepted_def promised\<^sub>b_accepted by fastforce
lemma (in oneSlot) prevAccepted_finite: "finite (prevAccepted p ns)"
using prevAccepted_proposed proposed_finite by (meson rev_finite_subset)
lemma (in oneSlot) Q_nonempty: "\<And>q. q \<in> Q \<Longrightarrow> q \<noteq> {}"
using Q_intersects by (auto simp add: intersects_def)
text \<open>The heart of the consistency proof is property P2b from \textit{Paxos made simple} by Lamport:\<close>
lemma (in oneSlot) p2b:
assumes "proposed t\<^sub>1" and "committed t\<^sub>2" and "t\<^sub>2 < t\<^sub>1"
shows "v t\<^sub>1 = v t\<^sub>2"
using assms
proof (induct t\<^sub>1 rule: less_induct)
case (less t\<^sub>1)
hence hyp: "\<And> t\<^sub>1'. \<lbrakk> t\<^sub>1' < t\<^sub>1; proposed t\<^sub>1'; t\<^sub>2 \<le> t\<^sub>1' \<rbrakk> \<Longrightarrow> v t\<^sub>1' = v t\<^sub>2"
using le_imp_less_or_eq by blast
from `proposed t\<^sub>1` obtain q\<^sub>1 t\<^sub>1' where
q\<^sub>1_quorum: "q\<^sub>1 \<in> Q" and
q\<^sub>1_promised: "\<And>n. n \<in> q\<^sub>1 \<Longrightarrow> promised n t\<^sub>1" and
q\<^sub>1_value: "prevAccepted t\<^sub>1 q\<^sub>1 = {} \<or> (v t\<^sub>1 = v t\<^sub>1' \<and> maxTerm (prevAccepted t\<^sub>1 q\<^sub>1) \<le> t\<^sub>1' \<and> proposed t\<^sub>1' \<and> t\<^sub>1' < t\<^sub>1)"
by (meson proposed)
from `committed t\<^sub>2` obtain q\<^sub>2 where
q\<^sub>2_quorum: "q\<^sub>2 \<in> Q" and
q\<^sub>2_accepted: "\<And>n. n \<in> q\<^sub>2 \<Longrightarrow> accepted n t\<^sub>2"
using committed by force
have "q\<^sub>1 \<inter> q\<^sub>2 \<noteq> {}"
using Q_intersects intersects_def less.prems q\<^sub>1_quorum q\<^sub>2_quorum by auto
then obtain n where n\<^sub>1: "n \<in> q\<^sub>1" and n\<^sub>2: "n \<in> q\<^sub>2" by auto
from n\<^sub>1 q\<^sub>1_promised have "promised n t\<^sub>1" by simp
moreover from n\<^sub>2 q\<^sub>2_accepted have "accepted n t\<^sub>2" by simp
ultimately obtain t\<^sub>2' where t\<^sub>2': "promised\<^sub>b n t\<^sub>1 t\<^sub>2'"
using less.prems(3) promised\<^sub>f promised_def by blast
have q\<^sub>1_value: "v t\<^sub>1 = v t\<^sub>1'" "maxTerm (prevAccepted t\<^sub>1 q\<^sub>1) \<le> t\<^sub>1'" "proposed t\<^sub>1'" "t\<^sub>1' < t\<^sub>1"
using n\<^sub>1 prevAccepted_def q\<^sub>1_value t\<^sub>2' by auto
note `v t\<^sub>1 = v t\<^sub>1'`
also have "v t\<^sub>1' = v t\<^sub>2"
proof (intro hyp)
have p: "maxTerm (prevAccepted t\<^sub>1 q\<^sub>1) \<in> prevAccepted t\<^sub>1 q\<^sub>1"
apply (intro maxTerm_mem prevAccepted_finite)
using n\<^sub>1 prevAccepted_def t\<^sub>2' by auto
show "t\<^sub>1' < t\<^sub>1" "proposed t\<^sub>1'" using q\<^sub>1_value by simp_all
have "t\<^sub>2 \<le> t\<^sub>2'"
by (meson \<open>accepted n t\<^sub>2\<close> less.prems(3) not_le promised\<^sub>b_max t\<^sub>2')
also have "t\<^sub>2' \<le> maxTerm (prevAccepted t\<^sub>1 q\<^sub>1)"
using n\<^sub>1 prevAccepted_def t\<^sub>2' prevAccepted_finite by (intro maxTerm_max, auto)
also have "... \<le> t\<^sub>1'" using q\<^sub>1_value by simp
finally show "t\<^sub>2 \<le> t\<^sub>1'" .
qed
finally show ?case .
qed
text \<open>From this, it follows that any two committed values are equal as desired.\<close>
lemma (in oneSlot) consistent:
assumes "committed t\<^sub>1" and "committed t\<^sub>2"
shows "v t\<^sub>1 = v t\<^sub>2"
using assms by (metis Q_nonempty accepted all_not_in_conv committed not_less_iff_gr_or_eq p2b)
text \<open>It will be useful later to know the conditions under which a value in a term can be committed,
which is spelled out here:\<close>
lemma (in oneSlot) commit:
assumes q_quorum: "q \<in> Q"
assumes q_accepted: "\<And>n. n \<in> q \<Longrightarrow> accepted n t\<^sub>0"
defines "committed' t \<equiv> committed t \<or> t = t\<^sub>0"
shows "oneSlot Q v promised\<^sub>f promised\<^sub>b proposed accepted committed'"
by (smt committed'_def Q_intersects oneSlot_axioms oneSlot_def q_accepted q_quorum)
end
================================================
FILE: cluster/isabelle/Preliminaries.thy
================================================
section \<open>Preliminaries\<close>
text \<open>We start with some definitions of the types involved.\<close>
theory Preliminaries
imports Main
begin
subsection \<open>Slots\<close>
text \<open>Slots are identified by natural numbers.\<close>
type_synonym Slot = nat
subsection \<open>Terms\<close>
text \<open>Terms are identified by natural numbers.\<close>
type_synonym Term = nat
subsubsection \<open>Maximum term of a set\<close>
text \<open>A function for finding the maximum term in a set is as follows.\<close>
definition maxTerm :: "Term set \<Rightarrow> Term"
where "maxTerm S \<equiv> THE t. t \<in> S \<and> (\<forall> t' \<in> S. t' \<le> t)"
text \<open>It works correctly on finite and nonempty sets as follows:\<close>
theorem
fixes S :: "Term set"
assumes finite: "finite S"
shows maxTerm_mem: "S \<noteq> {} \<Longrightarrow> maxTerm S \<in> S"
and maxTerm_max: "\<And> t'. t' \<in> S \<Longrightarrow> t' \<le> maxTerm S"
proof -
presume "S \<noteq> {}"
with assms
obtain t where t: "t \<in> S" "\<And> t'. t' \<in> S \<Longrightarrow> t' \<le> t"
proof (induct arbitrary: thesis)
case empty
then show ?case by simp
next
case (insert t S)
show ?case
proof (cases "S = {}")
case True hence [simp]: "insert t S = {t}" by simp
from insert.prems show ?thesis by simp
next
case False
obtain t' where t': "t' \<in> S" "\<forall> t'' \<in> S. t'' \<le> t'"
by (meson False insert.hyps(3))
from t'
show ?thesis
proof (intro insert.prems ballI)
fix t'' assume t'': "t'' \<in> insert t S"
show "t'' \<le> (if t \<le> t' then t' else t)"
proof (cases "t'' = t")
case False
with t'' have "t'' \<in> S" by simp
with t' have "t'' \<le> t'" by simp
thus ?thesis by auto
qed simp
qed simp
qed
qed
from t have "maxTerm S = t"
by (unfold maxTerm_def, intro the_equality, simp_all add: eq_iff)
with t show "maxTerm S \<in> S" "\<And>t'. t' \<in> S \<Longrightarrow> t' \<le> maxTerm S" by simp_all
qed auto
lemma
assumes "\<And>t. t \<in> S \<Longrightarrow> t \<le> t'" "finite S" "S \<noteq> {}"
shows maxTerm_le: "maxTerm S \<le> t'" using assms maxTerm_mem by auto
subsection \<open>Configurations and quorums\<close>
text \<open>Nodes are simply identified by a natural number.\<close>
datatype Node = Node nat
definition natOfNode :: "Node \<Rightarrow> nat" where "natOfNode node \<equiv> case node of Node n \<Rightarrow> n"
lemma natOfNode_Node[simp]: "natOfNode (Node n) = n" by (simp add: natOfNode_def)
lemma Node_natOfNode[simp]: "Node (natOfNode n) = n" by (cases n, simp add: natOfNode_def)
lemma natOfNode_inj[simp]: "(natOfNode n\<^sub>1 = natOfNode n\<^sub>2) = (n\<^sub>1 = n\<^sub>2)" by (metis Node_natOfNode)
text \<open>It is useful to be able to talk about whether sets-of-sets-of nodes mutually intersect or not.\<close>
definition intersects :: "Node set set \<Rightarrow> Node set set \<Rightarrow> bool" (infixl "\<frown>" 50)
where "A \<frown> B \<equiv> \<forall> a \<in> A. \<forall> b \<in> B. a \<inter> b \<noteq> {}"
definition majorities :: "Node set \<Rightarrow> Node set set"
where "majorities votingNodes = { q. card votingNodes < card (q \<inter> votingNodes) * 2 }"
lemma majorities_nonempty: assumes "q \<in> majorities Q" shows "q \<noteq> {}"
using assms by (auto simp add: majorities_def)
lemma majorities_member: assumes "q \<in> majorities Q" obtains n where "n \<in> q"
using majorities_nonempty assms by fastforce
lemma majorities_intersect:
assumes "finite votingNodes"
shows "majorities votingNodes \<frown> majorities votingNodes"
unfolding intersects_def
proof (intro ballI notI)
fix q\<^sub>1 assume q\<^sub>1: "q\<^sub>1 \<in> majorities votingNodes"
fix q\<^sub>2 assume q\<^sub>2: "q\<^sub>2 \<in> majorities votingNodes"
assume disj: "q\<^sub>1 \<inter> q\<^sub>2 = {}"
have 1: "card ((q\<^sub>1 \<inter> votingNodes) \<union> (q\<^sub>2 \<inter> votingNodes)) = card (q\<^sub>1 \<inter> votingNodes) + card (q\<^sub>2 \<inter> votingNodes)"
proof (intro card_Un_disjoint)
from assms show "finite (q\<^sub>1 \<inter> votingNodes)" by simp
from assms show "finite (q\<^sub>2 \<inter> votingNodes)" by simp
from disj show "q\<^sub>1 \<inter> votingNodes \<inter> (q\<^sub>2 \<inter> votingNodes) = {}" by auto
qed
have "card ((q\<^sub>1 \<inter> votingNodes) \<union> (q\<^sub>2 \<inter> votingNodes)) \<le> card votingNodes" by (simp add: assms card_mono)
hence 2: "2 * card (q\<^sub>1 \<inter> votingNodes) + 2 * card (q\<^sub>2 \<inter> votingNodes) \<le> 2 * card votingNodes" by (simp add: 1)
from q\<^sub>1 q\<^sub>2 have 3: "card votingNodes + card votingNodes < 2 * card (q\<^sub>1 \<inter> votingNodes) + 2 * card (q\<^sub>2 \<inter> votingNodes)"
unfolding majorities_def by auto
from 2 3 show False by simp
qed
text \<open>A configuration of the system defines the sets of master-eligible nodes whose votes count when calculating quorums.
The initial configuration of the system is fixed to some arbitrary value.\<close>
consts Vs\<^sub>0 :: "Node list"
definition V\<^sub>0 :: "Node set" where "V\<^sub>0 \<equiv> set Vs\<^sub>0"
lemma finite_V\<^sub>0: "finite V\<^sub>0" unfolding V\<^sub>0_def by auto
lemma V\<^sub>0_intersects: "majorities V\<^sub>0 \<frown> majorities V\<^sub>0" using finite_V\<^sub>0 by (intro majorities_intersect)
subsection \<open>Values\<close>
text \<open>The model is a replicated state machine, with transitions that either do nothing, alter
the configuration of the system or set a new \texttt{ClusterState}. \texttt{ClusterState} values
are modelled simply as natural numbers.\<close>
datatype ClusterState = ClusterState nat
consts CS\<^sub>0 :: ClusterState
datatype Value
= NoOp
| Reconfigure "Node list" (* update the set of voting nodes. A list rather than a set to force it to be finite *)
| ClusterStateDiff "ClusterState \<Rightarrow> ClusterState" (* a ClusterState diff *)
text \<open>Some useful definitions and lemmas follow.\<close>
fun isReconfiguration :: "Value \<Rightarrow> bool"
where "isReconfiguration (Reconfigure _) = True"
| "isReconfiguration _ = False"
fun getConf :: "Value \<Rightarrow> Node set"
where "getConf (Reconfigure conf) = set conf"
| "getConf _ = {}"
lemma getConf_finite: "finite (getConf v)"
by (metis List.finite_set getConf.elims infinite_imp_nonempty)
lemma getConf_intersects: "majorities (getConf v) \<frown> majorities (getConf v)"
by (simp add: getConf_finite majorities_intersect)
end
================================================
FILE: cluster/isabelle/ROOT
================================================
session "elasticsearch-isabelle" = "HOL" +
options [document = pdf, document_output = "output", document_variants="document:outline=/proof"]
theories [document = false]
(* Foo *)
(* Bar *)
theories
Preliminaries
Implementation
Monadic
OneSlot
Zen
document_files
"root.tex"
================================================
FILE: cluster/isabelle/Zen.thy
================================================
section \<open>Safety Properties\<close>
text \<open>This section describes the invariants that hold in the system, shows that the implementation
preserves the invariants, and shows that the invariants imply the required safety properties.\<close>
theory Zen
imports Implementation OneSlot
begin
subsection \<open>Invariants on messages\<close>
text \<open>Firstly, a set of invariants that hold on the set of messages that
have ever been sent, without considering the state of any individual
node.\<close>
fun nat_inductive_def :: "'a \<Rightarrow> (nat \<Rightarrow> 'a \<Rightarrow> 'a) \<Rightarrow> nat \<Rightarrow> 'a"
where
"nat_inductive_def zeroCase sucCase 0 = zeroCase"
| "nat_inductive_def zeroCase sucCase (Suc i) = sucCase i (nat_inductive_def zeroCase sucCase i)"
locale zenMessages =
fixes messages :: "RoutedMessage set"
fixes isMessageFromTo :: "Node \<Rightarrow> Message \<Rightarrow> Destination \<Rightarrow> bool" ("(_) \<midarrow>\<langle> _ \<rangle>\<rightarrow> (_)" [1000,55,1000])
defines "s \<midarrow>\<langle> m \<rangle>\<rightarrow> d \<equiv> \<lparr> sender = s, destination = d, payload = m \<rparr> \<in> messages"
fixes isMessageFrom :: "Node \<Rightarrow> Message \<Rightarrow> bool" ("(_) \<midarrow>\<langle> _ \<rangle>\<leadsto>" [1000,55])
defines "s \<midarrow>\<langle> m \<rangle>\<leadsto> \<equiv> \<exists> d. s \<midarrow>\<langle> m \<rangle>\<rightarrow> d"
fixes isMessageTo :: "Message \<Rightarrow> Destination \<Rightarrow> bool" ("\<langle> _ \<rangle>\<rightarrow> (_)" [55,1000])
defines "\<langle> m \<rangle>\<rightarrow> d \<equiv> \<exists> s. s \<midarrow>\<langle> m \<rangle>\<rightarrow> d"
fixes isMessage :: "Message \<Rightarrow> bool" ("\<langle> _ \<rangle>\<leadsto>" [55])
defines "\<langle> m \<rangle>\<leadsto> \<equiv> \<exists> s. s \<midarrow>\<langle> m \<rangle>\<leadsto>"
(* value proposed in a slot & a term *)
fixes v :: "Slot \<Rightarrow> Term \<Rightarrow> Value"
defines "v i t \<equiv> THE x. \<langle> PublishRequest i t x \<rangle>\<leadsto>"
(* whether a slot is committed *)
fixes isCommitted :: "Slot \<Rightarrow> bool"
defines "isCommitted i \<equiv> \<exists> t. \<langle> ApplyCommit i t \<rangle>\<leadsto>"
(* whether all preceding slots are committed *)
fixes committedTo :: "Slot \<Rightarrow> bool" ("committed\<^sub><")
defines "committed\<^sub>< i \<equiv> \<forall> j < i. isCommitted j"
(* the committed value in a slot *)
fixes v\<^sub>c :: "Slot \<Rightarrow> Value"
defines "v\<^sub>c i \<equiv> v i (SOME t. \<langle> ApplyCommit i t \<rangle>\<leadsto>)"
(* the configuration of a slot *)
fixes V :: "Slot \<Rightarrow> Node set"
defines "V \<equiv> nat_inductive_def V\<^sub>0 (\<lambda>i Vi. if isReconfiguration (v\<^sub>c i) then getConf (v\<^sub>c i) else Vi)"
(* predicate to say whether an applicable Vote has been sent *)
fixes promised :: "Slot \<Rightarrow> Node \<Rightarrow> Node \<Rightarrow> Term \<Rightarrow> bool"
defines "promised i s dn t \<equiv> \<exists> i' \<le> i. \<exists> a. s \<midarrow>\<langle> Vote i' t a \<rangle>\<rightarrow> (OneNode dn)"
(* set of previously-accepted terms *)
fixes prevAccepted :: "Slot \<Rightarrow> Term \<Rightarrow> Node set \<Rightarrow> Term set"
defines "prevAccepted i t senders
\<equiv> {t'. \<exists> s \<in> senders. s \<midarrow>\<langle> Vote i t (SomeTerm t') \<rangle>\<leadsto> }"
fixes lastCommittedClusterStateBefore :: "Slot \<Rightarrow> ClusterState"
defines "lastCommittedClusterStateBefore \<equiv> nat_inductive_def CS\<^sub>0
(\<lambda>i CSi. case v\<^sub>c i of ClusterStateDiff diff \<Rightarrow> diff CSi | _ \<Rightarrow> CSi)"
(* ASSUMPTIONS *)
assumes Vote_future:
"\<And>i i' s t t' a.
\<lbrakk> s \<midarrow>\<langle> Vote i t a \<rangle>\<leadsto>; i < i'; t' < t \<rbrakk>
\<Longrightarrow> \<not> s \<midarrow>\<langle> PublishResponse i' t' \<rangle>\<leadsto>"
assumes Vote_None:
"\<And>i s t t'.
\<lbrakk> s \<midarrow>\<langle> Vote i t NO_TERM \<rangle>\<leadsto>; t' < t \<rbrakk>
\<Longrightarrow> \<not> s \<midarrow>\<langle> PublishResponse i t' \<rangle>\<leadsto>"
assumes Vote_Some_lt:
"\<And>i s t t'. s \<midarrow>\<langle> Vote i t (SomeTerm t') \<rangle>\<leadsto>
\<Longrightarrow> t' < t"
assumes Vote_Some_PublishResponse:
"\<And>i s t t'. s \<midarrow>\<langle> Vote i t (SomeTerm t') \<rangle>\<leadsto>
\<Longrightarrow> s \<midarrow>\<langle> PublishResponse i t' \<rangle>\<leadsto>"
assumes Vote_Some_max:
"\<And>i s t t' t''. \<lbrakk> s \<midarrow>\<langle> Vote i t (SomeTerm t') \<rangle>\<leadsto>; t' < t''; t'' < t \<rbrakk>
\<Longrightarrow> \<not> s \<midarrow>\<langle> PublishResponse i t'' \<rangle>\<leadsto>"
assumes Vote_not_broadcast:
"\<And>i t a d. \<langle> Vote i t a \<rangle>\<rightarrow> d \<Longrightarrow> d \<noteq> Broadcast"
assumes Vote_unique_destination:
"\<And>i s t a a' d d'. \<lbrakk> s \<midarrow>\<langle> Vote i t a \<rangle>\<rightarrow> d; s \<midarrow>\<langle> Vote i' t a' \<rangle>\<rightarrow> d' \<rbrakk>
\<Longrightarrow> d = d'"
assumes PublishRequest_committedTo:
"\<And>i t x. \<langle> PublishRequest i t x \<rangle>\<leadsto> \<Longrightarrow> committedTo i"
assumes PublishRequest_quorum:
"\<And>i s t x. s \<midarrow>\<langle> PublishRequest i t x \<rangle>\<leadsto>
\<Longrightarrow> \<exists> q \<in> majorities (V i). (\<forall> n \<in> q. promised i n s t) \<and>
(prevAccepted i t q = {}
\<or> (\<exists> t'. v i t = v i t' \<and> maxTerm (prevAccepted i t q) \<le> t'
\<and> \<langle> PublishResponse i t' \<rangle>\<leadsto> \<and> t' < t))"
assumes PublishRequest_function:
"\<And>i t x x'. \<lbrakk> \<langle> PublishRequest i t x \<rangle>\<leadsto>; \<langle> PublishRequest i t x' \<rangle>\<leadsto> \<rbrakk>
\<Longrightarrow> x = x'"
assumes finite_messages:
"finite messages"
assumes PublishResponse_PublishRequest:
"\<And>i s t. s \<midarrow>\<langle> PublishResponse i t \<rangle>\<leadsto> \<Longrightarrow> \<exists> x. \<langle> PublishRequest i t x \<rangle>\<leadsto>"
assumes ApplyCommit_quorum:
"\<And>i t. \<langle> ApplyCommit i t \<rangle>\<leadsto>
\<Longrightarrow> \<exists> q \<in> majorities (V i). \<forall> s \<in> q. s \<midarrow>\<langle> PublishResponse i t \<rangle>\<leadsto>"
assumes CatchUpResponse_committedTo:
"\<And>i conf cs. \<langle> CatchUpResponse i conf cs \<rangle>\<leadsto> \<Longrightarrow> committed\<^sub>< i"
assumes CatchUpResponse_V:
"\<And>i conf cs. \<langle> CatchUpResponse i conf cs \<rangle>\<leadsto> \<Longrightarrow> V i = conf"
assumes CatchUpResponse_lastCommittedClusterStateBefore:
"\<And>i conf cs. \<langle> CatchUpResponse i conf cs \<rangle>\<leadsto> \<Longrightarrow> lastCommittedClusterStateBefore i = cs"
definition (in zenMessages) votedFor :: "Node \<Rightarrow> Node \<Rightarrow> Term \<Rightarrow> bool"
where "votedFor n\<^sub>1 n\<^sub>2 t \<equiv> \<exists> i. promised i n\<^sub>1 n\<^sub>2 t"
lemma (in zenMessages) votedFor_unique:
assumes "votedFor n n\<^sub>1 t"
assumes "votedFor n n\<^sub>2 t"
shows "n\<^sub>1 = n\<^sub>2"
using assms unfolding votedFor_def by (meson Destination.inject Vote_unique_destination promised_def)
lemma (in zenMessages) V_simps[simp]:
"V 0 = V\<^sub>0"
"V (Suc i) = (if isReconfiguration (v\<^sub>c i) then getConf (v\<^sub>c i) else V i)"
unfolding V_def by simp_all
lemma (in zenMessages) lastCommittedClusterStateBefore_simps[simp]:
"lastCommittedClusterStateBefore 0 = CS\<^sub>0"
"lastCommittedClusterStateBefore (Suc i) = (case v\<^sub>c i of ClusterStateDiff diff \<Rightarrow> diff | _ \<Rightarrow> id) (lastCommittedClusterStateBefore i)"
unfolding lastCommittedClusterStateBefore_def by (simp, cases "v\<^sub>c i", auto)
declare [[goals_limit = 40]]
subsubsection \<open>Utility lemmas\<close>
text \<open>Some results that are useful later:\<close>
lemma (in zenMessages) V_finite: "finite (V i)"
by (induct i, simp_all add: finite_V\<^sub>0 getConf_finite)
lemma (in zenMessages) V_intersects: "majorities (V i) \<frown> majorities (V i)"
using V_finite majorities_intersect by simp
lemma (in zenMessages) ApplyCommit_PublishResponse:
assumes "\<langle> ApplyCommit i t \<rangle>\<leadsto>"
obtains s where "s \<midarrow>\<langle> PublishResponse i t \<rangle>\<leadsto>"
by (meson ApplyCommit_quorum majorities_member assms)
lemma (in zenMessages) ApplyCommit_PublishRequest:
assumes "\<langle> ApplyCommit i t \<rangle>\<leadsto>"
shows "\<langle> PublishRequest i t (v i t) \<rangle>\<leadsto>"
by (metis ApplyCommit_PublishResponse PublishResponse_PublishRequest assms the_equality v_def PublishRequest_function)
lemma (in zenMessages) PublishRequest_Vote:
assumes "s \<midarrow>\<langle> PublishRequest i t x \<rangle>\<leadsto>"
obtains i' n a where "i' \<le> i" "n \<midarrow>\<langle> Vote i' t a \<rangle>\<rightarrow> (OneNode s)"
by (meson PublishRequest_quorum majorities_member assms isMessage_def promised_def)
lemma (in zenMessages) finite_prevAccepted: "finite (prevAccepted i t ns)"
proof -
fix t\<^sub>0
define f :: "RoutedMessage \<Rightarrow> Term" where "f \<equiv> \<lambda> m. case payload m of Vote _ _ (SomeTerm t') \<Rightarrow> t' | _ \<Rightarrow> t\<^sub>0"
have "prevAccepted i t ns \<subseteq> f ` messages"
apply (simp add: prevAccepted_def f_def isMessageFrom_def isMessageFromTo_def, intro subsetI)
using image_iff by fastforce
with finite_messages show ?thesis using finite_surj by auto
qed
lemma (in zenMessages) promised_long_def: "\<exists>d. promised i s d t
\<equiv> (s \<midarrow>\<langle> Vote i t NO_TERM \<rangle>\<leadsto>
\<or> (\<exists>i'<i. \<exists>a. s \<midarrow>\<langle> Vote i' t a \<rangle>\<leadsto>))
\<or> (\<exists>t'. s \<midarrow>\<langle> Vote i t (SomeTerm t') \<rangle>\<leadsto>)"
(is "?LHS == ?RHS")
proof -
have "?LHS = ?RHS"
apply (intro iffI)
apply (metis TermOption.exhaust isMessageFrom_def nat_less_le promised_def)
by (metis Destination.exhaust Vote_not_broadcast isMessageFrom_def isMessageTo_def nat_less_le not_le promised_def)
thus "?LHS == ?RHS" by simp
qed
lemma (in zenMessages) Vote_value_function:
assumes "s \<midarrow>\<langle> Vote i t a\<^sub>1 \<rangle>\<leadsto>" and "s \<midarrow>\<langle> Vote i t a\<^sub>2 \<rangle>\<leadsto>"
shows "a\<^sub>1 = a\<^sub>2"
proof (cases a\<^sub>1)
case NO_TERM
with assms show ?thesis
by (metis TermOption.exhaust Vote_None Vote_Some_PublishResponse Vote_Some_lt)
next
case (SomeTerm t\<^sub>1)
with assms obtain t\<^sub>2 where a\<^sub>2: "a\<^sub>2 = SomeTerm t\<^sub>2"
using Vote_None Vote_Some_PublishResponse Vote_Some_lt TermOption.exhaust by metis
from SomeTerm a\<^sub>2 assms show ?thesis
by (metis Vote_Some_PublishResponse Vote_Some_lt less_linear Vote_Some_max)
qed
lemma (in zenMessages) shows finite_messages_insert: "finite (insert m messages)"
using finite_messages by auto
lemma (in zenMessages) isCommitted_committedTo:
assumes "isCommitted i"
shows "committed\<^sub>< i"
using ApplyCommit_PublishRequest PublishRequest_committedTo assms isCommitted_def by blast
lemma (in zenMessages) isCommitted_committedTo_Suc:
assumes "isCommitted i"
shows "committed\<^sub>< (Suc i)"
using assms committedTo_def isCommitted_committedTo less_antisym by blast
lemma (in zenMessages) promised_unique:
assumes "promised i s d t" and "promised i' s d' t"
shows "d = d'"
by (meson Destination.inject Vote_unique_destination assms promised_def)
lemma (in zenMessages) PublishResponse_PublishRequest_v:
assumes "\<langle> PublishResponse i t \<rangle>\<leadsto>"
shows "\<langle> PublishRequest i t (v i t) \<rangle>\<leadsto>"
proof -
from assms obtain s where "s \<midarrow>\<langle> PublishResponse i t \<rangle>\<leadsto>" unfolding isMessage_def by blast
with PublishResponse_PublishRequest
obtain x where x: "\<langle> PublishRequest i t x \<rangle>\<leadsto>" by blast
have "v i t = x" unfolding v_def using x by (intro the_equality PublishRequest_function)
with x show ?thesis by simp
qed
lemma (in zenMessages) Vote_PublishRequest_v:
assumes "\<langle> Vote i t (SomeTerm t') \<rangle>\<leadsto>"
shows "\<langle> PublishRequest i t' (v i t') \<rangle>\<leadsto>"
using assms Vote_Some_PublishResponse PublishResponse_PublishRequest_v
unfolding isMessage_def by metis
subsubsection \<open>Relationship to @{term oneSlot}\<close>
text \<open>This shows that each slot @{term i} in Zen satisfies the assumptions of the @{term
oneSlot} model above.\<close>
lemma (in zenMessages) zen_is_oneSlot:
fixes i
shows "oneSlot (majorities (V i)) (v i)
(\<lambda> s t. s \<midarrow>\<langle> Vote i t NO_TERM \<rangle>\<leadsto>
\<or> (\<exists> i' < i. \<exists> a. s \<midarrow>\<langle> Vote i' t a \<rangle>\<leadsto>))
(\<lambda> s t t'. s \<midarrow>\<langle> Vote i t (SomeTerm t') \<rangle>\<leadsto>)
(\<lambda> t. \<exists> x. \<langle> PublishRequest i t x \<rangle>\<leadsto>)
(\<lambda> s t. s \<midarrow>\<langle> PublishResponse i t \<rangle>\<leadsto>)
(\<lambda> t. \<langle> ApplyCommit i t \<rangle>\<leadsto>)"
proof (unfold_locales, fold prevAccepted_def promised_long_def)
from V_intersects show "majorities (V i) \<frown> majorities (V i)".
next
fix s t t'
assume "t' < t" "s \<midarrow>\<langle> Vote i t NO_TERM \<rangle>\<leadsto> \<or> (\<exists>i'<i. \<exists>a. s \<midarrow>\<langle> Vote i' t a \<rangle>\<leadsto>)"
thus "\<not> s \<midarrow>\<langle> PublishResponse i t' \<rangle>\<leadsto>"
using Vote_None Vote_future by auto
next
fix s t t'
assume j: "s \<midarrow>\<langle> Vote i t (SomeTerm t') \<rangle>\<leadsto>"
from j show "t' < t" using Vote_Some_lt by blast
from j show "s \<midarrow>\<langle> PublishResponse i t' \<rangle>\<leadsto>" using Vote_Some_PublishResponse by blast
fix t'' assume "t' < t''" "t'' < t"
with j show "\<not> s \<midarrow>\<langle> Pu
gitextract_nf10lb83/
├── .gitignore
├── LICENSE
├── README.md
├── ReplicaEngine/
│ └── tla/
│ ├── ReplicaEngine.tla
│ └── ReplicaEngine.toolbox/
│ ├── .project
│ ├── .settings/
│ │ └── org.lamport.tla.toolbox.prefs
│ └── ReplicaEngine___model.launch
├── Storage/
│ └── tla/
│ ├── Storage.tla
│ └── Storage.toolbox/
│ └── Storage___model.launch
├── ZenWithTerms/
│ └── tla/
│ ├── ZenWithTerms.tla
│ └── ZenWithTerms.toolbox/
│ ├── .project
│ ├── .settings/
│ │ └── org.lamport.tla.toolbox.prefs
│ └── ZenWithTerms___model.launch
├── cluster/
│ ├── isabelle/
│ │ ├── Implementation.thy
│ │ ├── Monadic.thy
│ │ ├── OneSlot.thy
│ │ ├── Preliminaries.thy
│ │ ├── ROOT
│ │ ├── Zen.thy
│ │ └── document/
│ │ └── root.tex
│ └── tla/
│ ├── consensus.tla
│ └── consensus.toolbox/
│ ├── .project
│ ├── .settings/
│ │ └── org.lamport.tla.toolbox.prefs
│ └── consensus___model.launch
└── data/
└── tla/
├── replication.tla
└── replication.toolbox/
├── .project
├── .settings/
│ └── org.lamport.tla.toolbox.prefs
└── replication___model.launch
Condensed preview — 28 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (541K chars).
[
{
"path": ".gitignore",
"chars": 306,
"preview": "**/.DS_Store\n**/tla/*.toolbox/model\n**/tla/*.toolbox/*aux\n**/tla/*.toolbox/*.log\n**/tla/*.toolbox/*.pdf\n**/tla/*.toolbox"
},
{
"path": "LICENSE",
"chars": 11357,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "README.md",
"chars": 4190,
"preview": "# Formal models of core Elasticsearch algorithms\n\nThis repository contains formal models of core [Elasticsearch](https:/"
},
{
"path": "ReplicaEngine/tla/ReplicaEngine.tla",
"chars": 56556,
"preview": "-------------------------- MODULE ReplicaEngine --------------------------\n\nEXTENDS Naturals, FiniteSets, Sequences, TLC"
},
{
"path": "ReplicaEngine/tla/ReplicaEngine.toolbox/.project",
"chars": 673,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<projectDescription>\n\t<name>ReplicaEngine</name>\n\t<comment></comment>\n\t<projects>"
},
{
"path": "ReplicaEngine/tla/ReplicaEngine.toolbox/.settings/org.lamport.tla.toolbox.prefs",
"chars": 85,
"preview": "ProjectRootFile=PARENT-1-PROJECT_LOC/ReplicaEngine.tla\neclipse.preferences.version=1\n"
},
{
"path": "ReplicaEngine/tla/ReplicaEngine.toolbox/ReplicaEngine___model.launch",
"chars": 3311,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<launchConfiguration type=\"org.lamport.tla.toolbox.tool.tlc.model"
},
{
"path": "Storage/tla/Storage.tla",
"chars": 8248,
"preview": "------------------------------ MODULE Storage ------------------------------\nEXTENDS Integers, FiniteSets, TLC\n\nCONSTANT"
},
{
"path": "Storage/tla/Storage.toolbox/Storage___model.launch",
"chars": 2820,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<launchConfiguration type=\"org.lamport.tla.toolbox.tool.tlc.model"
},
{
"path": "ZenWithTerms/tla/ZenWithTerms.tla",
"chars": 16957,
"preview": "-------------------------------------------------------------------------------------\n\n-------------------------------- "
},
{
"path": "ZenWithTerms/tla/ZenWithTerms.toolbox/.project",
"chars": 670,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<projectDescription>\n\t<name>ZenWithTerms</name>\n\t<comment></comment>\n\t<projects>\n"
},
{
"path": "ZenWithTerms/tla/ZenWithTerms.toolbox/.settings/org.lamport.tla.toolbox.prefs",
"chars": 84,
"preview": "ProjectRootFile=PARENT-1-PROJECT_LOC/ZenWithTerms.tla\neclipse.preferences.version=1\n"
},
{
"path": "ZenWithTerms/tla/ZenWithTerms.toolbox/ZenWithTerms___model.launch",
"chars": 3877,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<launchConfiguration type=\"org.lamport.tla.toolbox.tool.tlc.model"
},
{
"path": "cluster/isabelle/Implementation.thy",
"chars": 18283,
"preview": "section \\<open>Implementation\\<close>\n\ntext \\<open>This section presents the implementation of the algorithm.\\<close>\n\nt"
},
{
"path": "cluster/isabelle/Monadic.thy",
"chars": 41978,
"preview": "theory Monadic\n imports Implementation \"~~/src/HOL/Library/Monad_Syntax\"\nbegin\n\ndatatype Exception = IllegalArgumentExc"
},
{
"path": "cluster/isabelle/OneSlot.thy",
"chars": 6655,
"preview": "section \\<open>One-slot consistency\\<close>\n\ntext \\<open>The replicated state machine determines the values that are com"
},
{
"path": "cluster/isabelle/Preliminaries.thy",
"chars": 6666,
"preview": "section \\<open>Preliminaries\\<close>\n\ntext \\<open>We start with some definitions of the types involved.\\<close>\n\ntheory "
},
{
"path": "cluster/isabelle/ROOT",
"chars": 313,
"preview": "session \"elasticsearch-isabelle\" = \"HOL\" +\n options [document = pdf, document_output = \"output\", document_variants=\"doc"
},
{
"path": "cluster/isabelle/Zen.thy",
"chars": 268030,
"preview": "section \\<open>Safety Properties\\<close>\n\ntext \\<open>This section describes the invariants that hold in the system, sho"
},
{
"path": "cluster/isabelle/document/root.tex",
"chars": 1462,
"preview": "\\documentclass[11pt,a4paper]{article}\n\\usepackage{isabelle,isabellesym}\n\\usepackage{latexsym}\n\n% further packages requir"
},
{
"path": "cluster/tla/consensus.tla",
"chars": 13476,
"preview": "`^\\Large\\bf\nTLA+ Model of an improved Zen consensus algorithm with reconfiguration capabilities ^'\n---------------------"
},
{
"path": "cluster/tla/consensus.toolbox/.project",
"chars": 655,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<projectDescription>\n\t<name>zen</name>\n\t<comment></comment>\n\t<projects>\n\t</projec"
},
{
"path": "cluster/tla/consensus.toolbox/.settings/org.lamport.tla.toolbox.prefs",
"chars": 81,
"preview": "ProjectRootFile=PARENT-1-PROJECT_LOC/consensus.tla\neclipse.preferences.version=1\n"
},
{
"path": "cluster/tla/consensus.toolbox/consensus___model.launch",
"chars": 3499,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<launchConfiguration type=\"org.lamport.tla.toolbox.tool.tlc.model"
},
{
"path": "data/tla/replication.tla",
"chars": 40807,
"preview": "`^\\Large\\bf\nTLA+ Model of the Elasticsearch data replication approach ^'\n-----------------------------------------------"
},
{
"path": "data/tla/replication.toolbox/.project",
"chars": 663,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<projectDescription>\n\t<name>elastic</name>\n\t<comment></comment>\n\t<projects>\n\t</pr"
},
{
"path": "data/tla/replication.toolbox/.settings/org.lamport.tla.toolbox.prefs",
"chars": 83,
"preview": "ProjectRootFile=PARENT-1-PROJECT_LOC/replication.tla\neclipse.preferences.version=1\n"
},
{
"path": "data/tla/replication.toolbox/replication___model.launch",
"chars": 3260,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<launchConfiguration type=\"org.lamport.tla.toolbox.tool.tlc.model"
}
]
About this extraction
This page contains the full source code of the elastic/elasticsearch-formal-models GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 28 files (503.0 KB), approximately 137.4k tokens. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.