Repository: elastic/elasticsearch-formal-models Branch: master Commit: ca30663506a7 Files: 28 Total size: 503.0 KB Directory structure: gitextract_nf10lb83/ ├── .gitignore ├── LICENSE ├── README.md ├── ReplicaEngine/ │ └── tla/ │ ├── ReplicaEngine.tla │ └── ReplicaEngine.toolbox/ │ ├── .project │ ├── .settings/ │ │ └── org.lamport.tla.toolbox.prefs │ └── ReplicaEngine___model.launch ├── Storage/ │ └── tla/ │ ├── Storage.tla │ └── Storage.toolbox/ │ └── Storage___model.launch ├── ZenWithTerms/ │ └── tla/ │ ├── ZenWithTerms.tla │ └── ZenWithTerms.toolbox/ │ ├── .project │ ├── .settings/ │ │ └── org.lamport.tla.toolbox.prefs │ └── ZenWithTerms___model.launch ├── cluster/ │ ├── isabelle/ │ │ ├── Implementation.thy │ │ ├── Monadic.thy │ │ ├── OneSlot.thy │ │ ├── Preliminaries.thy │ │ ├── ROOT │ │ ├── Zen.thy │ │ └── document/ │ │ └── root.tex │ └── tla/ │ ├── consensus.tla │ └── consensus.toolbox/ │ ├── .project │ ├── .settings/ │ │ └── org.lamport.tla.toolbox.prefs │ └── consensus___model.launch └── data/ └── tla/ ├── replication.tla └── replication.toolbox/ ├── .project ├── .settings/ │ └── org.lamport.tla.toolbox.prefs └── replication___model.launch ================================================ FILE CONTENTS ================================================ ================================================ FILE: .gitignore ================================================ **/.DS_Store **/tla/*.toolbox/model **/tla/*.toolbox/*aux **/tla/*.toolbox/*.log **/tla/*.toolbox/*.pdf **/tla/*.toolbox/*.tex **/tla/*.toolbox/*___model_SnapShot*.launch **/tla/*.toolbox/**/*.tla **/tla/*.toolbox/**/*.out **/tla/*.toolbox/**/MC.cfg **/tla/*.pdf **/tla/*.old **/*~ cluster/isabelle/output ================================================ FILE: LICENSE ================================================ Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ================================================ FILE: README.md ================================================ # Formal models of core Elasticsearch algorithms This repository contains formal models of core [Elasticsearch](https://github.com/elastic/elasticsearch) algorithms and is directly related to implementation efforts around [data replication](https://github.com/elastic/elasticsearch/issues/10708) and [cluster coordination](https://github.com/elastic/elasticsearch/issues/32006). The models in this repository might represent past, current and future designs of Elasticsearch and can differ to their implementations in substantial ways. The formal models mainly serve to illustrate some of the high-level concepts and help to validate resiliency-related aspects. ## Models ### Cluster coordination model The cluster coordination TLA+ model ensures the consistency of cluster state updates and represents the core [cluster coordination](https://github.com/elastic/elasticsearch/issues/32006) and metadata replication algorithm implemented in Elasticsearch 7.0. It consists of two files: - [TLA+ specification](ZenWithTerms/tla/ZenWithTerms.tla) which has a [direct one-to-one implementation in Elasticsearch](https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/cluster/coordination/CoordinationState.java) - [TLC model checking configuration](ZenWithTerms/tla/ZenWithTerms.toolbox/ZenWithTerms___model.launch) ### Data replication model The data replication TLA+ model describes the Elasticsearch [sequence number](https://github.com/elastic/elasticsearch/issues/10708) based data replication approach, implemented since Elasticsearch 6.0, which consists of two files: - [TLA+ specification](data/tla/replication.tla) - [TLC model checking configuration](data/tla/replication.toolbox/replication___model.launch) ### Replica engine A TLA+ model of how the [engine](https://github.com/elastic/elasticsearch/blob/00fd73acc4a2991f96438f8c1948016c5b9eefb2/server/src/main/java/org/elasticsearch/index/engine/InternalEngine.java) handles replication requests. - [TLA+ specification](ReplicaEngine/tla/ReplicaEngine.tla) - [TLC model checking configuration](ReplicaEngine/tla/ReplicaEngine.toolbox/ReplicaEngine___model.launch) ### Alternative cluster coordination model The alternative cluster coordination TLA+ model consists of two files: - [TLA+ specification](cluster/tla/consensus.tla) - [TLC model checking configuration](cluster/tla/consensus.toolbox/consensus___model.launch) The alternative cluster consensus Isabelle model consists of the following theories: - [Basic definitions](cluster/isabelle/Preliminaries.thy) - [An implementation in functional style](cluster/isabelle/Implementation.thy) - [An implementation in monadic style, along with a proof it's equivalent to the previous](cluster/isabelle/Monadic.thy) - [The proof that each slot is consistent, based on Lamport's Synod algorithm](cluster/isabelle/OneSlot.thy) - [The proof that the implementation ensures consistency](cluster/isabelle/Zen.thy) ## How to edit/run TLA+: - Install the [TLA Toolbox](http://research.microsoft.com/en-us/um/people/lamport/tla/toolbox.html) - If on Mac OS, [move the downloaded app to the Applications folder first](https://groups.google.com/forum/#!topic/tlaplus/bL04c6BiYxo) - Read some [documentation](http://research.microsoft.com/en-us/um/people/lamport/tla/book.html) How to run the model checker in headless mode: - Download [tla2tools.jar](http://research.microsoft.com/en-us/um/people/lamport/tla/tools.html) - Run the model checker once in TLA+ Toolbox on desktop (can be aborted once started). This generates the folder `elasticsearch.toolbox/model/` that contains all model files that are required to run the model checker in headless mode. - Copy the above folder and `tla2tools.jar` to the server running in headless mode. - `cd` to the folder and run `java -Xmx30G -cp ../tla2tools.jar tlc2.TLC MC -deadlock -workers 12`. The setting `-Xmx30G` denotes the amount of memory to allocate to the model checker and `-workers 12` the number of worker threads (should be equal to the number of cores on machine). The setting `-deadlock` ensures that TLC explores the full reachable state space, not searching for deadlocks. ================================================ FILE: ReplicaEngine/tla/ReplicaEngine.tla ================================================ -------------------------- MODULE ReplicaEngine -------------------------- EXTENDS Naturals, FiniteSets, Sequences, TLC (* Actions on the Lucene index *) CONSTANTS Lucene_addDocuments, Lucene_updateDocuments, Lucene_deleteDocuments CONSTANTS ADD, RETRY_ADD, UPDATE, DELETE, NULL CONSTANTS DocContent CONSTANTS DocAutoIdTimestamp CONSTANTS DuplicationLimit (* We model the activity of a single document, since distinct documents (according to their IDs) are independent. Also each indexing operation occurs under a lock for that document ID, so there is not much concurrency to consider. *) (* The set of individual requests that can occur on the document *) Request(request_count) (* ADD: An optimised append-only write can only occur as the first operation on the document ID in seqno order. Any subsequent attempts to ADD the document have the retry flag set and modelled as a RETRY_ADD. Other operations on the document are also possible. *) == [type : {ADD}, seqno : {1}, content : DocContent, autoIdTimeStamp : {DocAutoIdTimestamp}] (* RETRY_ADD: A retry of a write that does involve an internally-generated document ID. *) \cup [type : {RETRY_ADD}, seqno : 1..request_count, content : DocContent, autoIdTimeStamp : {DocAutoIdTimestamp}] (* UPDATE: A write that does not involve an internally-generated document ID. *) \cup [type : {UPDATE}, seqno : 1..request_count, content : DocContent] (* DELETE *) \cup [type : {DELETE}, seqno : 1..request_count] (* The set of sets of requests, which have distinct seqnos *) RequestSet(request_count) == { rs \in SUBSET Request(request_count): /\ Cardinality(rs) = request_count /\ Cardinality({r.seqno : r \in rs}) = request_count /\ (* Also ADDs and RETRY_ADDs should have the same content *) Cardinality({r.content: r \in { r \in rs: r.type \in {ADD, RETRY_ADD}}}) <= 1 } (* Apply a set of operations to a document in seqno order *) RECURSIVE ApplyOps(_, _, _) ApplyOps(requests, nextSeqno, currentContent) == IF \A r \in requests: r.seqno < nextSeqno THEN currentContent ELSE LET r == CHOOSE r \in requests: r.seqno = nextSeqno IN IF r \in requests /\ r.seqno = nextSeqno THEN ApplyOps(requests, nextSeqno + 1, CASE r.type = DELETE -> NULL [] r.type = ADD -> r.content [] r.type = RETRY_ADD -> r.content [] r.type = UPDATE -> r.content) ELSE Assert(FALSE, "Bad sequence") (* Calculate the final doc by applying all the requests in order *) FinalDoc(requests) == ApplyOps(requests, 1, NULL) (* Apply each the operation in the Lucene buffer, rejecting an addDocuments when there is already a document present as this would lead to duplication. *) RECURSIVE ApplyBufferedOperations(_, _) ApplyBufferedOperations(buffer, origDoc) == IF buffer = <<>> THEN origDoc ELSE LET nextOp == Head(buffer) IN ApplyBufferedOperations(Tail(buffer), CASE nextOp.type = Lucene_deleteDocuments -> NULL [] \/ nextOp.type = Lucene_updateDocuments \/ /\ nextOp.type = Lucene_addDocuments /\ origDoc = NULL -> [content |-> nextOp.content, seqno |-> nextOp.seqno] [] OTHER -> Assert(FALSE, "Error: Lucene_addDocuments when origDoc /= NULL")) Max(a,b) == IF a <= b THEN b ELSE a (* --algorithm basic variables request_count \in 1..4, replication_requests \in RequestSet(request_count), expected_doc = FinalDoc(replication_requests), versionMap_needsSafeAccess = FALSE, versionMap_isUnsafe = FALSE, versionMap_entry = NULL, (* Other concurrent activity can flag that the version map needs to be safely accessed *) process SafeAccessEnablerProcess = "SafeAccessEnabler" begin SafeAccessEnablerLoop: while pc["Consumer"] /= "Done" do versionMap_needsSafeAccess := (versionMap_needsSafeAccess = FALSE); (* Technically the only way this can go back to FALSE is via a refresh, but we should not need this fact, so model both kinds of change. *) end while; end process; (* Other concurrent activity can make the version map become unsafe, if safe access mode is disabled *) process UnsafePutterProcess = "UnsafePutter" begin UnsafePutterLoop: while pc["Consumer"] /= "Done" do await versionMap_needsSafeAccess = FALSE; versionMap_isUnsafe := TRUE; end while; end process; (* Other concurrent activity can increase the maxUnsafeAutoIdTimestamp *) process MaxUnsafeAutoIdTimestampIncreaserProcess = "MaxUnsafeAutoIdTimestampIncreaser" begin MaxUnsafeAutoIdTimestampIncreaserLoop: while pc["Consumer"] /= "Done" do with newTimestamp \in {DocAutoIdTimestamp - 1, DocAutoIdTimestamp, DocAutoIdTimestamp + 1} do await maxUnsafeAutoIdTimestamp < newTimestamp; maxUnsafeAutoIdTimestamp := newTimestamp; end with; end while; end process; (* Lucene refreshes can happen at any time *) process LuceneProcess = "ReplicaLucene" variables lucene_document = NULL, lucene_buffer = <<>>, begin LuceneLoop: while pc["Consumer"] /= "Done" \/ lucene_buffer /= <<>> do lucene_document := ApplyBufferedOperations(lucene_buffer, lucene_document); lucene_buffer := <<>>; (* TODO Model the inner structure of the version map so this refresh can be broken into the individual steps that occur concurrently with ongoing indexing. *) versionMap_isUnsafe := FALSE; versionMap_needsSafeAccess := FALSE; if versionMap_entry /= NULL then if versionMap_entry.type = UPDATE then versionMap_entry := NULL; else assert versionMap_entry.type = DELETE; versionMap_entry := [ versionMap_entry EXCEPT !.flushed = TRUE ]; end if; end if; end while; end process; (* Flushed deletes expire after a time and are cleaned up *) process DeleteCollectorProcess = "DeleteCollector" begin DeleteCollectorLoop: while pc["Consumer"] /= "Done" do await /\ versionMap_entry /= NULL /\ versionMap_entry.type = DELETE /\ versionMap_entry.seqno <= localCheckPoint \* PR #28790 /\ versionMap_entry.flushed = TRUE; versionMap_entry := NULL; end while; end process; (* Local checkpoint advances as each operation is marked as completed *) process LocalCheckpointTrackerProcess = "LocalCheckpointTracker" variables localCheckPoint = 0, completedSeqnos = {} begin LocalCheckpointTrackerLoop: while pc["Consumer"] /= "Done" do await localCheckPoint + 1 \in completedSeqnos; localCheckPoint := localCheckPoint + 1; end while; end process process UnsafeSeqnoIncreaserProcess = "UnsafeSeqnoIncreaserProcess" variables maxSeqNoOfNonAppendOnlyOperations = 0, begin UnsafeSeqnoIncreaserProcessLoop: while pc["Consumer"] /= "Done" /\ maxSeqNoOfNonAppendOnlyOperations < request_count + 1 do maxSeqNoOfNonAppendOnlyOperations := maxSeqNoOfNonAppendOnlyOperations + 1; end while; end process (* The process that consumes replication requests for a particular document ID, which are processed in series because of the lock in the version map. *) process ConsumerProcess = "Consumer" variables duplicationCount = 0, maxUnsafeAutoIdTimestamp \in {0, DocAutoIdTimestamp - 1, DocAutoIdTimestamp, DocAutoIdTimestamp + 1}, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene, begin ConsumerLoop: while replication_requests /= {} do with replication_request \in replication_requests do if replication_request.type = ADD then (* Never see two ADDs - if duplicated, one of them is a RETRY_ADD *) either (* Process ADD without duplication *) replication_requests := replication_requests \ {replication_request}; req := replication_request; or await duplicationCount < DuplicationLimit; duplicationCount := duplicationCount + 1; (* Process ADD and leave a duplicate RETRY_ADD for later *) replication_requests := (replication_requests \ {replication_request}) \cup {[replication_request EXCEPT !.type = RETRY_ADD]}; req := replication_request; or await duplicationCount < DuplicationLimit; duplicationCount := duplicationCount + 1; (* Process duplicate RETRY_ADD and leave the original ADD *) req := [replication_request EXCEPT !.type = RETRY_ADD]; end either; else req := replication_request; either await duplicationCount < DuplicationLimit; duplicationCount := duplicationCount + 1; or replication_requests := replication_requests \ {replication_request}; end either; end if; end with; if req.type = DELETE then versionMap_needsSafeAccess := TRUE; (* planDeletionAsNonPrimary *) maxSeqNoOfNonAppendOnlyOperations := Max(maxSeqNoOfNonAppendOnlyOperations, req.seqno); if req.seqno <= localCheckPoint then (* OP_STALE_OR_EQUAL *) plan := "processButSkipLucene"; deleteFromLucene := FALSE; currentlyDeleted := FALSE; else if versionMap_isUnsafe then (* Perform a Lucene refresh *) AwaitRefreshOnDelete: \* Label here to allow for other concurrent activity await lucene_buffer = <<>>; versionMap_needsSafeAccess := TRUE; end if; compareDeleteOpToLuceneDocBasedOnSeqNo: \* Label needed because of AwaitRefreshOnDelete label if versionMap_entry /= NULL then (* Doc is in version map *) if req.seqno > versionMap_entry.seqno then (* OP_NEWER *) plan := "processNormally"; deleteFromLucene := TRUE; currentlyDeleted := FALSE; else (* OP_STALE_OR_EQUAL *) plan := "processButSkipLucene"; deleteFromLucene := FALSE; currentlyDeleted := FALSE; end if; else (* Doc is not in version map - check Lucene *) if lucene_document = NULL then (* LUCENE_DOC_NOT_FOUND *) plan := "processNormallyExceptNotFound"; deleteFromLucene := TRUE; currentlyDeleted := TRUE; else if req.seqno > lucene_document.seqno then (* OP_NEWER *) plan := "processNormally"; deleteFromLucene := TRUE; currentlyDeleted := FALSE; else (* OP_STALE_OR_EQUAL *) plan := "processButSkipLucene"; deleteFromLucene := FALSE; currentlyDeleted := FALSE; end if; end if; end if; end if; ExecuteDeletePlan: \* Label needed because of AwaitRefreshOnDelete label if deleteFromLucene then if currentlyDeleted = FALSE then lucene_buffer := Append(lucene_buffer, [ type |-> Lucene_deleteDocuments ]); end if; versionMap_entry := [ type |-> DELETE, seqno |-> req.seqno, flushed |-> FALSE ]; end if; completedSeqnos := completedSeqnos \cup {req.seqno}; else (* planIndexingAsNonPrimary *) (* A RETRY_ADD has canOptimiseAddDocument = TRUE and mayHaveBeenIndexedBefore = TRUE so is planned normally, but also updates maxUnsafeAutoIdTimestamp within mayHaveBeenIndexedBefore() *) if req.type = RETRY_ADD then maxUnsafeAutoIdTimestamp := Max(maxUnsafeAutoIdTimestamp, req.autoIdTimeStamp); end if; (* An ADD can be optimized if mayHaveBeenIndexedBefore = FALSE which is calculated by comparing timestamps. *) if /\ req.type = ADD /\ maxUnsafeAutoIdTimestamp < req.autoIdTimeStamp /\ maxSeqNoOfNonAppendOnlyOperations < req.seqno \* PR #28787 then plan := "optimisedAppendOnly"; currentNotFoundOrDeleted := TRUE; useLuceneUpdateDocument := FALSE; indexIntoLucene := TRUE; else if req.type \notin {ADD, RETRY_ADD} then maxSeqNoOfNonAppendOnlyOperations := Max(maxSeqNoOfNonAppendOnlyOperations, req.seqno); end if; (* All other operations are planned normally *) versionMap_needsSafeAccess := TRUE; if req.seqno <= localCheckPoint then (* OP_STALE_OR_EQUAL *) plan := "processButSkipLucene"; currentNotFoundOrDeleted := FALSE; useLuceneUpdateDocument := FALSE; indexIntoLucene := FALSE; else if versionMap_isUnsafe then (* Perform a Lucene refresh *) AwaitRefreshOnIndex: \* Label here to allow for other concurrent activity await lucene_buffer = <<>>; versionMap_needsSafeAccess := TRUE; end if; compareIndexOpToLuceneDocBasedOnSeqNo: \* Label needed because of AwaitRefreshOnIndex label if req.seqno <= localCheckPoint \* PR #29276 then \* PR #29276 (* OP_STALE_OR_EQUAL *) \* PR #29276 plan := "processButSkipLucene"; \* PR #29276 currentNotFoundOrDeleted := FALSE; \* PR #29276 useLuceneUpdateDocument := FALSE; \* PR #29276 indexIntoLucene := FALSE; \* PR #29276 elsif versionMap_entry /= NULL then (* Doc is in version map *) if req.seqno > versionMap_entry.seqno then (* OP_NEWER *) plan := "processNormally"; currentNotFoundOrDeleted := FALSE; useLuceneUpdateDocument := TRUE; indexIntoLucene := TRUE; else (* OP_STALE_OR_EQUAL *) plan := "processButSkipLucene"; currentNotFoundOrDeleted := FALSE; useLuceneUpdateDocument := FALSE; indexIntoLucene := FALSE; end if; else (* Doc is not in version map - check Lucene *) if lucene_document = NULL then (* LUCENE_DOC_NOT_FOUND *) plan := "processNormallyExceptNotFound"; currentNotFoundOrDeleted := TRUE; useLuceneUpdateDocument := FALSE; indexIntoLucene := TRUE; else if req.seqno > lucene_document.seqno then (* OP_NEWER *) plan := "processNormally"; currentNotFoundOrDeleted := FALSE; useLuceneUpdateDocument := TRUE; indexIntoLucene := TRUE; else (* OP_STALE_OR_EQUAL *) plan := "processButSkipLucene"; currentNotFoundOrDeleted := FALSE; useLuceneUpdateDocument := FALSE; indexIntoLucene := FALSE; end if; end if; end if; end if; end if; (* planIndexingAsNonPrimary finished - now time to execute the plan *) ExecuteIndexPlan: \* Label needed because of AwaitRefreshOnIndex label if indexIntoLucene then lucene_buffer := Append(lucene_buffer, [ type |-> IF useLuceneUpdateDocument THEN Lucene_updateDocuments ELSE Lucene_addDocuments , seqno |-> req.seqno , content |-> req.content ]); if versionMap_needsSafeAccess then versionMap_entry := [ type |-> UPDATE, seqno |-> req.seqno ]; else versionMap_isUnsafe := TRUE; if /\ versionMap_entry /= NULL /\ versionMap_entry.type = DELETE /\ versionMap_entry.seqno < req.seqno then versionMap_entry := NULL; \* Desync bug #3 (no PR number yet) end if; end if; end if; completedSeqnos := completedSeqnos \cup {req.seqno} end if; end while; end process end algorithm *) \* BEGIN TRANSLATION CONSTANT defaultInitValue VARIABLES request_count, replication_requests, expected_doc, versionMap_needsSafeAccess, versionMap_isUnsafe, versionMap_entry, pc, lucene_document, lucene_buffer, localCheckPoint, completedSeqnos, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene vars == << request_count, replication_requests, expected_doc, versionMap_needsSafeAccess, versionMap_isUnsafe, versionMap_entry, pc, lucene_document, lucene_buffer, localCheckPoint, completedSeqnos, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> ProcSet == {"SafeAccessEnabler"} \cup {"UnsafePutter"} \cup {"MaxUnsafeAutoIdTimestampIncreaser"} \cup {"ReplicaLucene"} \cup {"DeleteCollector"} \cup {"LocalCheckpointTracker"} \cup {"UnsafeSeqnoIncreaserProcess"} \cup {"Consumer"} Init == (* Global variables *) /\ request_count \in 1..4 /\ replication_requests \in RequestSet(request_count) /\ expected_doc = FinalDoc(replication_requests) /\ versionMap_needsSafeAccess = FALSE /\ versionMap_isUnsafe = FALSE /\ versionMap_entry = NULL (* Process LuceneProcess *) /\ lucene_document = NULL /\ lucene_buffer = <<>> (* Process LocalCheckpointTrackerProcess *) /\ localCheckPoint = 0 /\ completedSeqnos = {} (* Process UnsafeSeqnoIncreaserProcess *) /\ maxSeqNoOfNonAppendOnlyOperations = 0 (* Process ConsumerProcess *) /\ duplicationCount = 0 /\ maxUnsafeAutoIdTimestamp \in {0, DocAutoIdTimestamp - 1, DocAutoIdTimestamp, DocAutoIdTimestamp + 1} /\ req = defaultInitValue /\ plan = defaultInitValue /\ deleteFromLucene = defaultInitValue /\ currentlyDeleted = defaultInitValue /\ currentNotFoundOrDeleted = defaultInitValue /\ useLuceneUpdateDocument = defaultInitValue /\ indexIntoLucene = defaultInitValue /\ pc = [self \in ProcSet |-> CASE self = "SafeAccessEnabler" -> "SafeAccessEnablerLoop" [] self = "UnsafePutter" -> "UnsafePutterLoop" [] self = "MaxUnsafeAutoIdTimestampIncreaser" -> "MaxUnsafeAutoIdTimestampIncreaserLoop" [] self = "ReplicaLucene" -> "LuceneLoop" [] self = "DeleteCollector" -> "DeleteCollectorLoop" [] self = "LocalCheckpointTracker" -> "LocalCheckpointTrackerLoop" [] self = "UnsafeSeqnoIncreaserProcess" -> "UnsafeSeqnoIncreaserProcessLoop" [] self = "Consumer" -> "ConsumerLoop"] SafeAccessEnablerLoop == /\ pc["SafeAccessEnabler"] = "SafeAccessEnablerLoop" /\ IF pc["Consumer"] /= "Done" THEN /\ versionMap_needsSafeAccess' = (versionMap_needsSafeAccess = FALSE) /\ pc' = [pc EXCEPT !["SafeAccessEnabler"] = "SafeAccessEnablerLoop"] ELSE /\ pc' = [pc EXCEPT !["SafeAccessEnabler"] = "Done"] /\ UNCHANGED versionMap_needsSafeAccess /\ UNCHANGED << request_count, replication_requests, expected_doc, versionMap_isUnsafe, versionMap_entry, lucene_document, lucene_buffer, localCheckPoint, completedSeqnos, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> SafeAccessEnablerProcess == SafeAccessEnablerLoop UnsafePutterLoop == /\ pc["UnsafePutter"] = "UnsafePutterLoop" /\ IF pc["Consumer"] /= "Done" THEN /\ versionMap_needsSafeAccess = FALSE /\ versionMap_isUnsafe' = TRUE /\ pc' = [pc EXCEPT !["UnsafePutter"] = "UnsafePutterLoop"] ELSE /\ pc' = [pc EXCEPT !["UnsafePutter"] = "Done"] /\ UNCHANGED versionMap_isUnsafe /\ UNCHANGED << request_count, replication_requests, expected_doc, versionMap_needsSafeAccess, versionMap_entry, lucene_document, lucene_buffer, localCheckPoint, completedSeqnos, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> UnsafePutterProcess == UnsafePutterLoop MaxUnsafeAutoIdTimestampIncreaserLoop == /\ pc["MaxUnsafeAutoIdTimestampIncreaser"] = "MaxUnsafeAutoIdTimestampIncreaserLoop" /\ IF pc["Consumer"] /= "Done" THEN /\ \E newTimestamp \in {DocAutoIdTimestamp - 1, DocAutoIdTimestamp, DocAutoIdTimestamp + 1}: /\ maxUnsafeAutoIdTimestamp < newTimestamp /\ maxUnsafeAutoIdTimestamp' = newTimestamp /\ pc' = [pc EXCEPT !["MaxUnsafeAutoIdTimestampIncreaser"] = "MaxUnsafeAutoIdTimestampIncreaserLoop"] ELSE /\ pc' = [pc EXCEPT !["MaxUnsafeAutoIdTimestampIncreaser"] = "Done"] /\ UNCHANGED maxUnsafeAutoIdTimestamp /\ UNCHANGED << request_count, replication_requests, expected_doc, versionMap_needsSafeAccess, versionMap_isUnsafe, versionMap_entry, lucene_document, lucene_buffer, localCheckPoint, completedSeqnos, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> MaxUnsafeAutoIdTimestampIncreaserProcess == MaxUnsafeAutoIdTimestampIncreaserLoop LuceneLoop == /\ pc["ReplicaLucene"] = "LuceneLoop" /\ IF pc["Consumer"] /= "Done" \/ lucene_buffer /= <<>> THEN /\ lucene_document' = ApplyBufferedOperations(lucene_buffer, lucene_document) /\ lucene_buffer' = <<>> /\ versionMap_isUnsafe' = FALSE /\ versionMap_needsSafeAccess' = FALSE /\ IF versionMap_entry /= NULL THEN /\ IF versionMap_entry.type = UPDATE THEN /\ versionMap_entry' = NULL ELSE /\ Assert(versionMap_entry.type = DELETE, "Failure of assertion at line 147, column 17.") /\ versionMap_entry' = [ versionMap_entry EXCEPT !.flushed = TRUE ] ELSE /\ TRUE /\ UNCHANGED versionMap_entry /\ pc' = [pc EXCEPT !["ReplicaLucene"] = "LuceneLoop"] ELSE /\ pc' = [pc EXCEPT !["ReplicaLucene"] = "Done"] /\ UNCHANGED << versionMap_needsSafeAccess, versionMap_isUnsafe, versionMap_entry, lucene_document, lucene_buffer >> /\ UNCHANGED << request_count, replication_requests, expected_doc, localCheckPoint, completedSeqnos, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> LuceneProcess == LuceneLoop DeleteCollectorLoop == /\ pc["DeleteCollector"] = "DeleteCollectorLoop" /\ IF pc["Consumer"] /= "Done" THEN /\ /\ versionMap_entry /= NULL /\ versionMap_entry.type = DELETE /\ versionMap_entry.seqno <= localCheckPoint /\ versionMap_entry.flushed = TRUE /\ versionMap_entry' = NULL /\ pc' = [pc EXCEPT !["DeleteCollector"] = "DeleteCollectorLoop"] ELSE /\ pc' = [pc EXCEPT !["DeleteCollector"] = "Done"] /\ UNCHANGED versionMap_entry /\ UNCHANGED << request_count, replication_requests, expected_doc, versionMap_needsSafeAccess, versionMap_isUnsafe, lucene_document, lucene_buffer, localCheckPoint, completedSeqnos, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> DeleteCollectorProcess == DeleteCollectorLoop LocalCheckpointTrackerLoop == /\ pc["LocalCheckpointTracker"] = "LocalCheckpointTrackerLoop" /\ IF pc["Consumer"] /= "Done" THEN /\ localCheckPoint + 1 \in completedSeqnos /\ localCheckPoint' = localCheckPoint + 1 /\ pc' = [pc EXCEPT !["LocalCheckpointTracker"] = "LocalCheckpointTrackerLoop"] ELSE /\ pc' = [pc EXCEPT !["LocalCheckpointTracker"] = "Done"] /\ UNCHANGED localCheckPoint /\ UNCHANGED << request_count, replication_requests, expected_doc, versionMap_needsSafeAccess, versionMap_isUnsafe, versionMap_entry, lucene_document, lucene_buffer, completedSeqnos, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> LocalCheckpointTrackerProcess == LocalCheckpointTrackerLoop UnsafeSeqnoIncreaserProcessLoop == /\ pc["UnsafeSeqnoIncreaserProcess"] = "UnsafeSeqnoIncreaserProcessLoop" /\ IF pc["Consumer"] /= "Done" /\ maxSeqNoOfNonAppendOnlyOperations < request_count + 1 THEN /\ maxSeqNoOfNonAppendOnlyOperations' = maxSeqNoOfNonAppendOnlyOperations + 1 /\ pc' = [pc EXCEPT !["UnsafeSeqnoIncreaserProcess"] = "UnsafeSeqnoIncreaserProcessLoop"] ELSE /\ pc' = [pc EXCEPT !["UnsafeSeqnoIncreaserProcess"] = "Done"] /\ UNCHANGED maxSeqNoOfNonAppendOnlyOperations /\ UNCHANGED << request_count, replication_requests, expected_doc, versionMap_needsSafeAccess, versionMap_isUnsafe, versionMap_entry, lucene_document, lucene_buffer, localCheckPoint, completedSeqnos, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> UnsafeSeqnoIncreaserProcess == UnsafeSeqnoIncreaserProcessLoop ConsumerLoop == /\ pc["Consumer"] = "ConsumerLoop" /\ IF replication_requests /= {} THEN /\ \E replication_request \in replication_requests: IF replication_request.type = ADD THEN /\ \/ /\ replication_requests' = replication_requests \ {replication_request} /\ req' = replication_request /\ UNCHANGED duplicationCount \/ /\ duplicationCount < DuplicationLimit /\ duplicationCount' = duplicationCount + 1 /\ replication_requests' = (replication_requests \ {replication_request}) \cup {[replication_request EXCEPT !.type = RETRY_ADD]} /\ req' = replication_request \/ /\ duplicationCount < DuplicationLimit /\ duplicationCount' = duplicationCount + 1 /\ req' = [replication_request EXCEPT !.type = RETRY_ADD] /\ UNCHANGED replication_requests ELSE /\ req' = replication_request /\ \/ /\ duplicationCount < DuplicationLimit /\ duplicationCount' = duplicationCount + 1 /\ UNCHANGED replication_requests \/ /\ replication_requests' = replication_requests \ {replication_request} /\ UNCHANGED duplicationCount /\ IF req'.type = DELETE THEN /\ versionMap_needsSafeAccess' = TRUE /\ maxSeqNoOfNonAppendOnlyOperations' = Max(maxSeqNoOfNonAppendOnlyOperations, req'.seqno) /\ IF req'.seqno <= localCheckPoint THEN /\ plan' = "processButSkipLucene" /\ deleteFromLucene' = FALSE /\ currentlyDeleted' = FALSE /\ pc' = [pc EXCEPT !["Consumer"] = "ExecuteDeletePlan"] ELSE /\ IF versionMap_isUnsafe THEN /\ pc' = [pc EXCEPT !["Consumer"] = "AwaitRefreshOnDelete"] ELSE /\ pc' = [pc EXCEPT !["Consumer"] = "compareDeleteOpToLuceneDocBasedOnSeqNo"] /\ UNCHANGED << plan, deleteFromLucene, currentlyDeleted >> /\ UNCHANGED << maxUnsafeAutoIdTimestamp, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> ELSE /\ IF req'.type = RETRY_ADD THEN /\ maxUnsafeAutoIdTimestamp' = Max(maxUnsafeAutoIdTimestamp, req'.autoIdTimeStamp) ELSE /\ TRUE /\ UNCHANGED maxUnsafeAutoIdTimestamp /\ IF /\ req'.type = ADD /\ maxUnsafeAutoIdTimestamp' < req'.autoIdTimeStamp /\ maxSeqNoOfNonAppendOnlyOperations < req'.seqno THEN /\ plan' = "optimisedAppendOnly" /\ currentNotFoundOrDeleted' = TRUE /\ useLuceneUpdateDocument' = FALSE /\ indexIntoLucene' = TRUE /\ pc' = [pc EXCEPT !["Consumer"] = "ExecuteIndexPlan"] /\ UNCHANGED << versionMap_needsSafeAccess, maxSeqNoOfNonAppendOnlyOperations >> ELSE /\ IF req'.type \notin {ADD, RETRY_ADD} THEN /\ maxSeqNoOfNonAppendOnlyOperations' = Max(maxSeqNoOfNonAppendOnlyOperations, req'.seqno) ELSE /\ TRUE /\ UNCHANGED maxSeqNoOfNonAppendOnlyOperations /\ versionMap_needsSafeAccess' = TRUE /\ IF req'.seqno <= localCheckPoint THEN /\ plan' = "processButSkipLucene" /\ currentNotFoundOrDeleted' = FALSE /\ useLuceneUpdateDocument' = FALSE /\ indexIntoLucene' = FALSE /\ pc' = [pc EXCEPT !["Consumer"] = "ExecuteIndexPlan"] ELSE /\ IF versionMap_isUnsafe THEN /\ pc' = [pc EXCEPT !["Consumer"] = "AwaitRefreshOnIndex"] ELSE /\ pc' = [pc EXCEPT !["Consumer"] = "compareIndexOpToLuceneDocBasedOnSeqNo"] /\ UNCHANGED << plan, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> /\ UNCHANGED << deleteFromLucene, currentlyDeleted >> ELSE /\ pc' = [pc EXCEPT !["Consumer"] = "Done"] /\ UNCHANGED << replication_requests, versionMap_needsSafeAccess, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> /\ UNCHANGED << request_count, expected_doc, versionMap_isUnsafe, versionMap_entry, lucene_document, lucene_buffer, localCheckPoint, completedSeqnos >> ExecuteDeletePlan == /\ pc["Consumer"] = "ExecuteDeletePlan" /\ IF deleteFromLucene THEN /\ IF currentlyDeleted = FALSE THEN /\ lucene_buffer' = Append(lucene_buffer, [ type |-> Lucene_deleteDocuments ]) ELSE /\ TRUE /\ UNCHANGED lucene_buffer /\ versionMap_entry' = [ type |-> DELETE, seqno |-> req.seqno, flushed |-> FALSE ] ELSE /\ TRUE /\ UNCHANGED << versionMap_entry, lucene_buffer >> /\ completedSeqnos' = (completedSeqnos \cup {req.seqno}) /\ pc' = [pc EXCEPT !["Consumer"] = "ConsumerLoop"] /\ UNCHANGED << request_count, replication_requests, expected_doc, versionMap_needsSafeAccess, versionMap_isUnsafe, lucene_document, localCheckPoint, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> ExecuteIndexPlan == /\ pc["Consumer"] = "ExecuteIndexPlan" /\ IF indexIntoLucene THEN /\ lucene_buffer' = Append(lucene_buffer, [ type |-> IF useLuceneUpdateDocument THEN Lucene_updateDocuments ELSE Lucene_addDocuments , seqno |-> req.seqno , content |-> req.content ]) /\ IF versionMap_needsSafeAccess THEN /\ versionMap_entry' = [ type |-> UPDATE, seqno |-> req.seqno ] /\ UNCHANGED versionMap_isUnsafe ELSE /\ versionMap_isUnsafe' = TRUE /\ IF /\ versionMap_entry /= NULL /\ versionMap_entry.type = DELETE /\ versionMap_entry.seqno < req.seqno THEN /\ versionMap_entry' = NULL ELSE /\ TRUE /\ UNCHANGED versionMap_entry ELSE /\ TRUE /\ UNCHANGED << versionMap_isUnsafe, versionMap_entry, lucene_buffer >> /\ completedSeqnos' = (completedSeqnos \cup {req.seqno}) /\ pc' = [pc EXCEPT !["Consumer"] = "ConsumerLoop"] /\ UNCHANGED << request_count, replication_requests, expected_doc, versionMap_needsSafeAccess, lucene_document, localCheckPoint, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> compareDeleteOpToLuceneDocBasedOnSeqNo == /\ pc["Consumer"] = "compareDeleteOpToLuceneDocBasedOnSeqNo" /\ IF versionMap_entry /= NULL THEN /\ IF req.seqno > versionMap_entry.seqno THEN /\ plan' = "processNormally" /\ deleteFromLucene' = TRUE /\ currentlyDeleted' = FALSE ELSE /\ plan' = "processButSkipLucene" /\ deleteFromLucene' = FALSE /\ currentlyDeleted' = FALSE ELSE /\ IF lucene_document = NULL THEN /\ plan' = "processNormallyExceptNotFound" /\ deleteFromLucene' = TRUE /\ currentlyDeleted' = TRUE ELSE /\ IF req.seqno > lucene_document.seqno THEN /\ plan' = "processNormally" /\ deleteFromLucene' = TRUE /\ currentlyDeleted' = FALSE ELSE /\ plan' = "processButSkipLucene" /\ deleteFromLucene' = FALSE /\ currentlyDeleted' = FALSE /\ pc' = [pc EXCEPT !["Consumer"] = "ExecuteDeletePlan"] /\ UNCHANGED << request_count, replication_requests, expected_doc, versionMap_needsSafeAccess, versionMap_isUnsafe, versionMap_entry, lucene_document, lucene_buffer, localCheckPoint, completedSeqnos, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> AwaitRefreshOnDelete == /\ pc["Consumer"] = "AwaitRefreshOnDelete" /\ lucene_buffer = <<>> /\ versionMap_needsSafeAccess' = TRUE /\ pc' = [pc EXCEPT !["Consumer"] = "compareDeleteOpToLuceneDocBasedOnSeqNo"] /\ UNCHANGED << request_count, replication_requests, expected_doc, versionMap_isUnsafe, versionMap_entry, lucene_document, lucene_buffer, localCheckPoint, completedSeqnos, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> compareIndexOpToLuceneDocBasedOnSeqNo == /\ pc["Consumer"] = "compareIndexOpToLuceneDocBasedOnSeqNo" /\ IF req.seqno <= localCheckPoint THEN /\ plan' = "processButSkipLucene" /\ currentNotFoundOrDeleted' = FALSE /\ useLuceneUpdateDocument' = FALSE /\ indexIntoLucene' = FALSE ELSE /\ IF versionMap_entry /= NULL THEN /\ IF req.seqno > versionMap_entry.seqno THEN /\ plan' = "processNormally" /\ currentNotFoundOrDeleted' = FALSE /\ useLuceneUpdateDocument' = TRUE /\ indexIntoLucene' = TRUE ELSE /\ plan' = "processButSkipLucene" /\ currentNotFoundOrDeleted' = FALSE /\ useLuceneUpdateDocument' = FALSE /\ indexIntoLucene' = FALSE ELSE /\ IF lucene_document = NULL THEN /\ plan' = "processNormallyExceptNotFound" /\ currentNotFoundOrDeleted' = TRUE /\ useLuceneUpdateDocument' = FALSE /\ indexIntoLucene' = TRUE ELSE /\ IF req.seqno > lucene_document.seqno THEN /\ plan' = "processNormally" /\ currentNotFoundOrDeleted' = FALSE /\ useLuceneUpdateDocument' = TRUE /\ indexIntoLucene' = TRUE ELSE /\ plan' = "processButSkipLucene" /\ currentNotFoundOrDeleted' = FALSE /\ useLuceneUpdateDocument' = FALSE /\ indexIntoLucene' = FALSE /\ pc' = [pc EXCEPT !["Consumer"] = "ExecuteIndexPlan"] /\ UNCHANGED << request_count, replication_requests, expected_doc, versionMap_needsSafeAccess, versionMap_isUnsafe, versionMap_entry, lucene_document, lucene_buffer, localCheckPoint, completedSeqnos, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, deleteFromLucene, currentlyDeleted >> AwaitRefreshOnIndex == /\ pc["Consumer"] = "AwaitRefreshOnIndex" /\ lucene_buffer = <<>> /\ versionMap_needsSafeAccess' = TRUE /\ pc' = [pc EXCEPT !["Consumer"] = "compareIndexOpToLuceneDocBasedOnSeqNo"] /\ UNCHANGED << request_count, replication_requests, expected_doc, versionMap_isUnsafe, versionMap_entry, lucene_document, lucene_buffer, localCheckPoint, completedSeqnos, maxSeqNoOfNonAppendOnlyOperations, duplicationCount, maxUnsafeAutoIdTimestamp, req, plan, deleteFromLucene, currentlyDeleted, currentNotFoundOrDeleted, useLuceneUpdateDocument, indexIntoLucene >> ConsumerProcess == ConsumerLoop \/ ExecuteDeletePlan \/ ExecuteIndexPlan \/ compareDeleteOpToLuceneDocBasedOnSeqNo \/ AwaitRefreshOnDelete \/ compareIndexOpToLuceneDocBasedOnSeqNo \/ AwaitRefreshOnIndex Next == SafeAccessEnablerProcess \/ UnsafePutterProcess \/ MaxUnsafeAutoIdTimestampIncreaserProcess \/ LuceneProcess \/ DeleteCollectorProcess \/ LocalCheckpointTrackerProcess \/ UnsafeSeqnoIncreaserProcess \/ ConsumerProcess \/ (* Disjunct to prevent deadlock on termination *) ((\A self \in ProcSet: pc[self] = "Done") /\ UNCHANGED vars) Spec == Init /\ [][Next]_vars Termination == <>(\A self \in ProcSet: pc[self] = "Done") \* END TRANSLATION Terminated == \A self \in ProcSet: pc[self] = "Done" Invariant == Terminated => /\ expected_doc = NULL => lucene_document = NULL /\ expected_doc /= NULL => lucene_document.content = expected_doc ============================================================================= ================================================ FILE: ReplicaEngine/tla/ReplicaEngine.toolbox/.project ================================================ ReplicaEngine toolbox.builder.TLAParserBuilder toolbox.builder.PCalAlgorithmSearchingBuilder toolbox.natures.TLANature ReplicaEngine.tla 1 PARENT-1-PROJECT_LOC/ReplicaEngine.tla ================================================ FILE: ReplicaEngine/tla/ReplicaEngine.toolbox/.settings/org.lamport.tla.toolbox.prefs ================================================ ProjectRootFile=PARENT-1-PROJECT_LOC/ReplicaEngine.tla eclipse.preferences.version=1 ================================================ FILE: ReplicaEngine/tla/ReplicaEngine.toolbox/ReplicaEngine___model.launch ================================================ ================================================ FILE: Storage/tla/Storage.tla ================================================ ------------------------------ MODULE Storage ------------------------------ EXTENDS Integers, FiniteSets, TLC CONSTANTS MaxNewMeta, \* maximum generation of newMeta to limit the state space MetaDataContent \* content that is written to the metadata file VARIABLES metadata, \* metaData[i] = MetaDataContent if metadata of generation i is present manifest, \* manifest[j] is generation of metadata j-th manifest is referencing newMeta, \* generation of newly created metadata file newManifest, \* generation of newly created manifest file state, \* current state, describes what to do next possibleStates \* set of generations of metadata that limits what can be read from disk -------------------------------------------------------------------------- (*************************************************************************) (* First we define some helper functions to work with files abstraction. *) (* Files is a function from file generation to some content. *) (*************************************************************************) (*************************************************************************) (* CurrentGeneration returns the maximum file generation. If there are *) (* no files then -1 is returned. *) (*************************************************************************) CurrentGeneration(files) == IF DOMAIN files = {} THEN -1 ELSE CHOOSE gen \in DOMAIN files : \A otherGen \in DOMAIN files : gen \geq otherGen (*************************************************************************) (* DeleteFile removes file with generation delGen. *) (*************************************************************************) DeleteFile(files, delGen) == [gen \in DOMAIN files \ {delGen} |-> files[gen]] (*************************************************************************) (* DeleteFilesExcept removes all files except keepGen. *) (*************************************************************************) DeleteFilesExcept(files, keepGen) == (keepGen :> files[keepGen]) (*************************************************************************) (* WriteFile creates new file with specified generation and content. *) (*************************************************************************) WriteFile(files, gen, content) == (gen :> content) @@ files -------------------------------------------------------------------------- (*************************************************************************) (* Now we define functions to emulate write and cleanup of the metadata. *) (*************************************************************************) WriteMetaOk(gen) == /\ metadata' = WriteFile(metadata, gen, MetaDataContent) /\ state' = "writeManifest" WriteMetaFail(gen) == /\ metadata' = metadata /\ state' = "writeMeta" WriteMetaDirty(gen) == /\ \/ metadata' = WriteFile(metadata, gen, MetaDataContent) \/ metadata' = metadata /\ state' = "deleteNewMeta" DeleteNewMeta == /\ \/ metadata' = DeleteFile(metadata, newMeta) \/ metadata' = metadata /\ state' = "writeMeta" /\ UNCHANGED <> DeleteOldMeta == /\ \/ metadata' = DeleteFilesExcept(metadata, newMeta) \/ metadata' = metadata /\ state' = "writeMeta" /\ UNCHANGED <> WriteMeta == LET gen == CurrentGeneration(metadata) + 1 IN /\ newMeta' = gen /\ \/ WriteMetaOk(gen) \/ WriteMetaFail(gen) \/ WriteMetaDirty(gen) /\ UNCHANGED <> -------------------------------------------------------------------------- (*************************************************************************) (* Now we define functions to emulate write and cleanup of the manifest *) (* file. *) (*************************************************************************) WriteManifestOk(gen) == /\ manifest' = WriteFile(manifest, gen, newMeta) /\ state' = "deleteOldManifest" /\ possibleStates' = {newMeta} WriteManifestFail(gen) == /\ manifest' = manifest /\ state' = "deleteNewMeta" /\ possibleStates' = possibleStates WriteManifestDirty(gen) == /\ \/ manifest' = WriteFile(manifest, gen, newMeta) \/ manifest' = manifest /\ state' = "deleteNewManifest" /\ possibleStates' = possibleStates \union {newMeta} WriteManifest == LET gen == CurrentGeneration(manifest) + 1 IN /\ newManifest' = gen /\ \/ WriteManifestOk(gen) \/ WriteManifestFail(gen) \/ WriteManifestDirty(gen) /\ UNCHANGED <> DeleteOldManifest == /\ \/ manifest' = DeleteFilesExcept(manifest, newManifest) \/ manifest' = manifest /\ state' = "deleteOldMeta" /\ UNCHANGED <> -------------------------------------------------------------------------- (*************************************************************************) (* Below are 3 versions of the same function, that is called when *) (* manifest write was dirty. The buggy one was initially implemented and *) (* was caught by https://github.com/elastic/elasticsearch/issues/39077. *) (* Pick one and use in Next function. *) (* https://github.com/elastic/elasticsearch/pull/40519 implements *) (* DeleteNewManifestEasy. *) (*************************************************************************) DeleteNewManifestBuggy == /\ \/ manifest' = DeleteFile(manifest, newManifest) \/ manifest' = manifest /\ state' = "deleteNewMeta" /\ UNCHANGED <> DeleteNewManifestEasy == /\ \/ manifest' = DeleteFile(manifest, newManifest) \/ manifest' = manifest /\ state' = "writeMeta" /\ UNCHANGED <> DeleteNewManifestHard == /\ \/ /\ manifest' = DeleteFile(manifest, newManifest) /\ state' = "deleteNewMeta" \/ /\ manifest' = manifest /\ state' = "writeMeta" /\ UNCHANGED <> -------------------------------------------------------------------------- (*************************************************************************) (* We can define Init and Next functions now. *) (*************************************************************************) Init == /\ metadata = <<>> /\ manifest = <<>> /\ newMeta = -1 \* no latest metadata file /\ newManifest = -1 \* no latest manifest file /\ state = "writeMeta" \* we start with writing metadata file /\ possibleStates = {} \* no metadata can be read from disk Next == \/ (state = "writeMeta" /\ WriteMeta) \/ (state = "writeManifest" /\ WriteManifest) \/ (state = "deleteOldManifest" /\ DeleteOldManifest) \/ (state = "deleteOldMeta" /\ DeleteOldMeta) \/ (state = "deleteNewManifest" /\ DeleteNewManifestEasy) \* try DeleteNewManifestBuggy and DeleteNewManifestHard \/ (state = "deleteNewMeta" /\ DeleteNewMeta) -------------------------------------------------------------------------- (*************************************************************************) (* Our model has 2 invariants. *) (*************************************************************************) MetadataFileReferencedByManifestExists == CurrentGeneration(manifest) /= -1 => manifest[CurrentGeneration(manifest)] \in DOMAIN metadata MetadataReferencedByManifestIsValid == CurrentGeneration(manifest) /= -1 => \E meta \in possibleStates : meta = manifest[CurrentGeneration(manifest)] ============ ================================================ FILE: Storage/tla/Storage.toolbox/Storage___model.launch ================================================ ================================================ FILE: ZenWithTerms/tla/ZenWithTerms.tla ================================================ ------------------------------------------------------------------------------------- -------------------------------- MODULE ZenWithTerms -------------------------------- \* Imported modules used in this specification EXTENDS Naturals, FiniteSets, Sequences, TLC ---- CONSTANTS Values \* Set of node ids (all master-eligible nodes) CONSTANTS Nodes \* RPC message types CONSTANTS Join, PublishRequest, PublishResponse, Commit ---- \* Set of requests and responses sent between nodes. VARIABLE messages \* Transitive closure of value updates as done by leaders VARIABLE descendant \* Values to bootstrap the cluster VARIABLE initialConfiguration VARIABLE initialValue VARIABLE initialAcceptedVersion \* node state (map from node id to state) VARIABLE currentTerm VARIABLE lastCommittedConfiguration VARIABLE lastAcceptedTerm VARIABLE lastAcceptedVersion VARIABLE lastAcceptedValue VARIABLE lastAcceptedConfiguration VARIABLE joinVotes VARIABLE startedJoinSinceLastReboot VARIABLE electionWon VARIABLE lastPublishedVersion VARIABLE lastPublishedConfiguration VARIABLE publishVotes ---- Terms == Nat Versions == Nat \* set of valid configurations (i.e. the set of all non-empty subsets of Nodes) ValidConfigs == SUBSET(Nodes) \ {{}} \* cluster-state versions that might have come from older systems InitialVersions == Nat \* quorums correspond to majority of votes in a config IsQuorum(votes, config) == Cardinality(votes \cap config) * 2 > Cardinality(config) IsElectionQuorum(n, votes) == /\ IsQuorum(votes, lastCommittedConfiguration[n]) /\ IsQuorum(votes, lastAcceptedConfiguration[n]) IsPublishQuorum(n, votes) == /\ IsQuorum(votes, lastCommittedConfiguration[n]) /\ IsQuorum(votes, lastPublishedConfiguration[n]) \* initial model state Init == /\ messages = {} /\ descendant = {} /\ initialConfiguration \in ValidConfigs /\ initialValue \in Values /\ initialAcceptedVersion \in [Nodes -> InitialVersions] /\ currentTerm = [n \in Nodes |-> 0] /\ lastCommittedConfiguration = [n \in Nodes |-> {}] \* empty config /\ lastAcceptedTerm = [n \in Nodes |-> 0] /\ lastAcceptedVersion = initialAcceptedVersion /\ lastAcceptedValue \in {[n \in Nodes |-> v] : v \in Values} \* all agree on initial value /\ lastAcceptedConfiguration = [n \in Nodes |-> lastCommittedConfiguration[n]] /\ joinVotes = [n \in Nodes |-> {}] /\ startedJoinSinceLastReboot = [n \in Nodes |-> FALSE] /\ electionWon = [n \in Nodes |-> FALSE] /\ lastPublishedVersion = [n \in Nodes |-> 0] /\ lastPublishedConfiguration = [n \in Nodes |-> lastCommittedConfiguration[n]] /\ publishVotes = [n \in Nodes |-> {}] \* Bootstrap node n with the initial state and config SetInitialState(n) == /\ lastAcceptedConfiguration[n] = {} \* not already bootstrapped /\ Assert(lastAcceptedTerm[n] = 0, "lastAcceptedTerm should be 0") /\ Assert(lastCommittedConfiguration[n] = {}, "lastCommittedConfiguration should be empty") /\ Assert(lastPublishedVersion[n] = 0, "lastPublishedVersion should be 0") /\ Assert(lastPublishedConfiguration[n] = {}, "lastPublishedConfiguration should be empty") /\ Assert(electionWon[n] = FALSE, "electionWon should be FALSE") /\ Assert(joinVotes[n] = {}, "joinVotes should be empty") /\ Assert(publishVotes[n] = {}, "publishVotes should be empty") /\ lastAcceptedConfiguration' = [lastAcceptedConfiguration EXCEPT ![n] = initialConfiguration] /\ lastAcceptedValue' = [lastAcceptedValue EXCEPT ![n] = initialValue] /\ lastCommittedConfiguration' = [lastCommittedConfiguration EXCEPT ![n] = initialConfiguration] /\ Assert(lastAcceptedTerm[n] = 0, "lastAcceptedTerm should be 0") /\ Assert(lastAcceptedConfiguration'[n] /= {}, "lastAcceptedConfiguration should be non-empty") /\ Assert(lastCommittedConfiguration'[n] /= {}, "lastCommittedConfiguration should be non-empty") /\ UNCHANGED <> \* Send join request from node n to node nm for term t HandleStartJoin(n, nm, t) == /\ t > currentTerm[n] /\ LET joinRequest == [method |-> Join, source |-> n, dest |-> nm, term |-> t, laTerm |-> lastAcceptedTerm[n], laVersion |-> lastAcceptedVersion[n]] IN /\ currentTerm' = [currentTerm EXCEPT ![n] = t] /\ lastPublishedVersion' = [lastPublishedVersion EXCEPT ![n] = 0] /\ lastPublishedConfiguration' = [lastPublishedConfiguration EXCEPT ![n] = lastAcceptedConfiguration[n]] /\ startedJoinSinceLastReboot' = [startedJoinSinceLastReboot EXCEPT ![n] = TRUE] /\ electionWon' = [electionWon EXCEPT ![n] = FALSE] /\ joinVotes' = [joinVotes EXCEPT ![n] = {}] /\ publishVotes' = [publishVotes EXCEPT ![n] = {}] /\ messages' = messages \cup { joinRequest } /\ UNCHANGED <> \* node n handles a join request and checks if it has received enough joins (= votes) \* for its term to be elected as master HandleJoin(n, m) == /\ m.method = Join /\ m.term = currentTerm[n] /\ startedJoinSinceLastReboot[n] /\ \/ m.laTerm < lastAcceptedTerm[n] \/ /\ m.laTerm = lastAcceptedTerm[n] /\ m.laVersion <= lastAcceptedVersion[n] /\ lastAcceptedConfiguration[n] /= {} \* must be bootstrapped /\ joinVotes' = [joinVotes EXCEPT ![n] = @ \cup { m.source }] /\ electionWon' = [electionWon EXCEPT ![n] = IsElectionQuorum(n, joinVotes'[n])] /\ IF electionWon[n] = FALSE /\ electionWon'[n] THEN \* initiating publish version with last accepted version to enable client requests /\ lastPublishedVersion' = [lastPublishedVersion EXCEPT ![n] = lastAcceptedVersion[n]] ELSE UNCHANGED <> /\ UNCHANGED <> \* client causes a cluster state change val with configuration cfg HandleClientValue(n, t, v, val, cfg) == /\ electionWon[n] /\ lastPublishedVersion[n] = lastAcceptedVersion[n] \* means we have the last published value / config (useful for CAS operations, where we need to read the previous value first) /\ t = currentTerm[n] /\ v > lastPublishedVersion[n] /\ cfg /= lastAcceptedConfiguration[n] => lastCommittedConfiguration[n] = lastAcceptedConfiguration[n] \* only allow reconfiguration if there is not already a reconfiguration in progress /\ IsQuorum(joinVotes[n], cfg) \* only allow reconfiguration if we have a quorum of (join) votes for the new config /\ LET publishRequests == { [method |-> PublishRequest, source |-> n, dest |-> ns, term |-> t, version |-> v, value |-> val, config |-> cfg, commConf |-> lastCommittedConfiguration[n]] : ns \in Nodes } newEntry == [prevT |-> lastAcceptedTerm[n], prevV |-> lastAcceptedVersion[n], nextT |-> t, nextV |-> v] matchingElems == { e \in descendant : /\ e.nextT = newEntry.prevT /\ e.nextV = newEntry.prevV } newTransitiveElems == { [prevT |-> e.prevT, prevV |-> e.prevV, nextT |-> newEntry.nextT, nextV |-> newEntry.nextV] : e \in matchingElems } IN /\ descendant' = descendant \cup {newEntry} \cup newTransitiveElems /\ lastPublishedVersion' = [lastPublishedVersion EXCEPT ![n] = v] /\ lastPublishedConfiguration' = [lastPublishedConfiguration EXCEPT ![n] = cfg] /\ publishVotes' = [publishVotes EXCEPT ![n] = {}] \* publishVotes are only counted per publish version /\ messages' = messages \cup publishRequests /\ UNCHANGED <> \* handle publish request m on node n HandlePublishRequest(n, m) == /\ m.method = PublishRequest /\ m.term = currentTerm[n] /\ (m.term = lastAcceptedTerm[n]) => (m.version > lastAcceptedVersion[n]) /\ lastAcceptedTerm' = [lastAcceptedTerm EXCEPT ![n] = m.term] /\ lastAcceptedVersion' = [lastAcceptedVersion EXCEPT ![n] = m.version] /\ lastAcceptedValue' = [lastAcceptedValue EXCEPT ![n] = m.value] /\ lastAcceptedConfiguration' = [lastAcceptedConfiguration EXCEPT ![n] = m.config] /\ lastCommittedConfiguration' = [lastCommittedConfiguration EXCEPT ![n] = m.commConf] /\ LET response == [method |-> PublishResponse, source |-> n, dest |-> m.source, term |-> m.term, version |-> m.version] IN /\ messages' = messages \cup {response} /\ UNCHANGED <> \* node n commits a change HandlePublishResponse(n, m) == /\ m.method = PublishResponse /\ electionWon[n] /\ m.term = currentTerm[n] /\ m.version = lastPublishedVersion[n] /\ publishVotes' = [publishVotes EXCEPT ![n] = @ \cup {m.source}] /\ IF IsPublishQuorum(n, publishVotes'[n]) THEN LET commitRequests == { [method |-> Commit, source |-> n, dest |-> ns, term |-> currentTerm[n], version |-> lastPublishedVersion[n]] : ns \in Nodes } IN /\ messages' = messages \cup commitRequests ELSE UNCHANGED <> /\ UNCHANGED <> \* apply committed configuration to node n HandleCommit(n, m) == /\ m.method = Commit /\ m.term = currentTerm[n] /\ m.term = lastAcceptedTerm[n] /\ m.version = lastAcceptedVersion[n] /\ (electionWon[n] => lastAcceptedVersion[n] = lastPublishedVersion[n]) /\ lastCommittedConfiguration' = [lastCommittedConfiguration EXCEPT ![n] = lastAcceptedConfiguration[n]] /\ UNCHANGED <> \* crash/restart node n (loses ephemeral state) RestartNode(n) == /\ joinVotes' = [joinVotes EXCEPT ![n] = {}] /\ startedJoinSinceLastReboot' = [startedJoinSinceLastReboot EXCEPT ![n] = FALSE] /\ electionWon' = [electionWon EXCEPT ![n] = FALSE] /\ lastPublishedVersion' = [lastPublishedVersion EXCEPT ![n] = 0] /\ lastPublishedConfiguration' = [lastPublishedConfiguration EXCEPT ![n] = lastAcceptedConfiguration[n]] /\ publishVotes' = [publishVotes EXCEPT ![n] = {}] /\ UNCHANGED <> \* next-step relation Next == \/ \E n \in Nodes : SetInitialState(n) \/ \E n, nm \in Nodes : \E t \in Terms : HandleStartJoin(n, nm, t) \/ \E m \in messages : HandleJoin(m.dest, m) \/ \E n \in Nodes : \E t \in Terms : \E v \in Versions : \E val \in Values : \E vs \in ValidConfigs : HandleClientValue(n, t, v, val, vs) \/ \E m \in messages : HandlePublishRequest(m.dest, m) \/ \E m \in messages : HandlePublishResponse(m.dest, m) \/ \E m \in messages : HandleCommit(m.dest, m) \/ \E n \in Nodes : RestartNode(n) ---- \* Invariants SingleNodeInvariant == \A n \in Nodes : /\ lastAcceptedTerm[n] <= currentTerm[n] /\ electionWon[n] = IsElectionQuorum(n, joinVotes[n]) \* cached value is consistent /\ IF electionWon[n] THEN lastPublishedVersion[n] >= lastAcceptedVersion[n] ELSE lastPublishedVersion[n] = 0 /\ electionWon[n] => startedJoinSinceLastReboot[n] /\ publishVotes[n] /= {} => electionWon[n] OneMasterPerTerm == \A m1, m2 \in messages: /\ m1.method = PublishRequest /\ m2.method = PublishRequest /\ m1.term = m2.term => m1.source = m2.source LogMatching == \A m1, m2 \in messages: /\ m1.method = PublishRequest /\ m2.method = PublishRequest /\ m1.term = m2.term /\ m1.version = m2.version => m1.value = m2.value CommittedPublishRequest(mp) == /\ mp.method = PublishRequest /\ \E mc \in messages: /\ mc.method = Commit /\ mp.term = mc.term /\ mp.version = mc.version DescendantRelationIsStrictlyOrdered == \A d \in descendant: /\ d.prevT <= d.nextT /\ d.prevV < d.nextV DescendantRelationIsTransitive == \A d1, d2 \in descendant: d1.nextT = d2.prevT /\ d1.nextV = d2.prevV => [prevT |-> d1.prevT, prevV |-> d1.prevV, nextT |-> d2.nextT, nextV |-> d2.nextV] \in descendant NewerOpsBasedOnOlderCommittedOps == \A m1, m2 \in messages : /\ CommittedPublishRequest(m1) /\ m2.method = PublishRequest /\ m2.term >= m1.term /\ m2.version > m1.version => [prevT |-> m1.term, prevV |-> m1.version, nextT |-> m2.term, nextV |-> m2.version] \in descendant \* main invariant (follows from NewerOpsBasedOnOlderCommittedOps): CommittedValuesDescendantsFromCommittedValues == \A m1, m2 \in messages : /\ CommittedPublishRequest(m1) /\ CommittedPublishRequest(m2) /\ \/ m1.term /= m2.term \/ m1.version /= m2.version => \/ [prevT |-> m1.term, prevV |-> m1.version, nextT |-> m2.term, nextV |-> m2.version] \in descendant \/ [prevT |-> m2.term, prevV |-> m2.version, nextT |-> m1.term, nextV |-> m1.version] \in descendant CommittedValuesDescendantsFromInitialValue == \E v \in InitialVersions : /\ \E n \in Nodes : v = initialAcceptedVersion[n] /\ \E votes \in SUBSET(initialConfiguration) : /\ IsQuorum(votes, initialConfiguration) /\ \A n \in votes : initialAcceptedVersion[n] <= v /\ \A m \in messages : CommittedPublishRequest(m) => [prevT |-> 0, prevV |-> v, nextT |-> m.term, nextV |-> m.version] \in descendant CommitHasQuorumVsPreviousCommittedConfiguration == \A mc \in messages: mc.method = Commit => (\A mprq \in messages: (/\ mprq.method = PublishRequest /\ mprq.term = mc.term /\ mprq.version = mc.version) => IsQuorum({mprs.source: mprs \in {mprs \in messages: /\ mprs.method = PublishResponse /\ mprs.term = mprq.term /\ mprs.version = mprq.version }}, mprq.commConf)) P2bInvariant == \A mc \in messages: mc.method = Commit => (\A mprq \in messages: mprq.method = PublishRequest => (mprq.term > mc.term => mprq.version > mc.version)) \* State-exploration limits StateConstraint == /\ \A n \in Nodes: IF currentTerm[n] <= 1 THEN lastPublishedVersion[n] <= 2 ELSE lastPublishedVersion[n] <= 3 /\ Cardinality(messages) <= 15 ==================================================================================================== ================================================ FILE: ZenWithTerms/tla/ZenWithTerms.toolbox/.project ================================================ ZenWithTerms toolbox.builder.TLAParserBuilder toolbox.builder.PCalAlgorithmSearchingBuilder toolbox.natures.TLANature ZenWithTerms.tla 1 PARENT-1-PROJECT_LOC/ZenWithTerms.tla ================================================ FILE: ZenWithTerms/tla/ZenWithTerms.toolbox/.settings/org.lamport.tla.toolbox.prefs ================================================ ProjectRootFile=PARENT-1-PROJECT_LOC/ZenWithTerms.tla eclipse.preferences.version=1 ================================================ FILE: ZenWithTerms/tla/ZenWithTerms.toolbox/ZenWithTerms___model.launch ================================================ ================================================ FILE: cluster/isabelle/Implementation.thy ================================================ section \Implementation\ text \This section presents the implementation of the algorithm.\ theory Implementation imports Preliminaries begin subsection \Protocol messages\ text \The proven-safe core of the protocol works by sending messages as described here. The remainder of the protocol may send other messages too, and may drop, reorder or duplicate any of these messages, but must not send these messages itself to ensure safety. Another way of thinking of these messages is to consider them as ``fire-and-forget'' RPC invocations that, on receipt, call some local method, maybe update the receiving node's state, and maybe yield some further messages. The @{type nat} parameter to each message refers to a slot number.\ datatype TermOption = NO_TERM | SomeTerm Term instantiation TermOption :: linorder begin fun less_TermOption :: "TermOption \ TermOption \ bool" where "t < NO_TERM = False" | "NO_TERM < SomeTerm t = True" | "SomeTerm t\<^sub>1 < SomeTerm t\<^sub>2 = (t\<^sub>1 < t\<^sub>2)" definition less_eq_TermOption :: "TermOption \ TermOption \ bool" where "(t\<^sub>1 :: TermOption) \ t\<^sub>2 \ t\<^sub>1 = t\<^sub>2 \ t\<^sub>1 < t\<^sub>2" instance proof fix x y z :: TermOption show "(x < y) = (x \ y \ \ y \ x)" unfolding less_eq_TermOption_def apply auto using less_TermOption.elims apply fastforce by (metis less_TermOption.elims(2) less_TermOption.simps(3) less_not_sym) show "x \ x" by (simp add: less_eq_TermOption_def) show "x \ y \ y \ z \ x \ z" unfolding less_eq_TermOption_def apply auto by (metis TermOption.distinct(1) TermOption.inject dual_order.strict_trans less_TermOption.elims(2) less_TermOption.elims(3)) show "x \ y \ y \ x \ x = y" unfolding less_eq_TermOption_def apply auto using \(x < y) = (x \ y \ \ y \ x)\ less_eq_TermOption_def by blast show "x \ y \ y \ x" unfolding less_eq_TermOption_def apply auto by (metis TermOption.distinct(1) TermOption.inject less_TermOption.elims(3) neqE) qed end lemma NO_TERM_le [simp]: "NO_TERM \ t" by (cases t, simp_all add: less_eq_TermOption_def) lemma le_NO_TERM [simp]: "(t \ NO_TERM) = (t = NO_TERM)" by (cases t, simp_all add: less_eq_TermOption_def) lemma le_SomeTerm [simp]: "(SomeTerm t\<^sub>1 \ SomeTerm t\<^sub>2) = (t\<^sub>1 \ t\<^sub>2)" by (auto simp add: less_eq_TermOption_def) datatype Message = StartJoin Term | Vote Slot Term TermOption | ClientValue Value | PublishRequest Slot Term Value | PublishResponse Slot Term | ApplyCommit Slot Term | CatchUpRequest | CatchUpResponse Slot "Node set" ClusterState | DiscardJoinVotes | Reboot text \Some prose descriptions of these messages follows, in order to give a bit more of an intuitive understanding of their purposes.\ text \The message @{term "StartJoin t"} may be sent by any node to attempt to start a master election in the given term @{term t}.\ text \The message @{term "Vote i t a"} may be sent by a node in response to a @{term StartJoin} message. It indicates that the sender knows all committed values for slots strictly below @{term i}, and that the sender will no longer vote (i.e. send an @{term PublishResponse}) in any term prior to @{term t}. The field @{term a} is either @{term None} or @{term "Some t'"}. In the former case this indicates that the node has not yet sent any @{term PublishResponse} message in slot @{term i}, and in the latter case it indicates that the largest term in which it has previously sent an @{term PublishResponse} message is @{term t'}. All nodes must avoid sending a @{term Vote} message to two different masters in the same term.\ text \The message @{term "ClientValue x"} may be sent by any node and indicates an attempt to reach consensus on the value @{term x}.\ text \The message @{term "PublishRequest i t v"} may be sent by the elected master of term @{term t} to request the other master-eligible nodes to vote for value @{term v} to be committed in slot @{term i}.\ text \The message @{term "PublishResponse i t"} may be sent by node in response to the corresponding @{term PublishRequest} message, indicating that the sender votes for the value proposed by the master of term @{term t} to be committed in slot @{term i}.\ text \The message @{term "ApplyCommit i t"} indicates that the value proposed by the master of term @{term t} in slot @{term i} received a quorum of votes and is therefore committed.\ text \The message @{term Reboot} may be sent by any node to represent the restart of a node, which loses any ephemeral state.\ text \The abstract model of Zen keeps track of the set of all messages that have ever been sent, and asserts that this set obeys certain invariants, listed below. Further below, it will be shown that these invariants imply that each slot obeys the @{term oneSlot} invariants above and hence that each slot cannot see inconsistent committed values.\ datatype Destination = Broadcast | OneNode Node record RoutedMessage = sender :: Node destination :: Destination payload :: Message text \It will be useful to be able to choose the optional term with the greater term, so here is a function that does that.\ subsection \Node implementation\ text \Each node holds the following local data.\ record TermValue = tvTerm :: Term tvValue :: Value record NodeData = currentNode :: Node currentTerm :: Term (* committed state *) firstUncommittedSlot :: Slot currentVotingNodes :: "Node set" currentClusterState :: ClusterState (* accepted state *) lastAcceptedData :: "TermValue option" (* election state *) joinVotes :: "Node set" electionWon :: bool (* publish state *) publishPermitted :: bool publishVotes :: "Node set" definition lastAcceptedValue :: "NodeData \ Value" where "lastAcceptedValue nd \ tvValue (THE lad. lastAcceptedData nd = Some lad)" definition lastAcceptedTerm :: "NodeData \ TermOption" where "lastAcceptedTerm nd \ case lastAcceptedData nd of None \ NO_TERM | Some lad \ SomeTerm (tvTerm lad)" definition isQuorum :: "NodeData \ Node set \ bool" where "isQuorum nd q \ q \ majorities (currentVotingNodes nd)" lemma lastAcceptedValue_joinVotes_update[simp]: "lastAcceptedValue (joinVotes_update f nd) = lastAcceptedValue nd" by (simp add: lastAcceptedValue_def) lemma lastAcceptedTerm_joinVotes_update[simp]: "lastAcceptedTerm (joinVotes_update f nd) = lastAcceptedTerm nd" by (simp add: lastAcceptedTerm_def) lemma lastAcceptedValue_electionWon_update[simp]: "lastAcceptedValue (electionWon_update f nd) = lastAcceptedValue nd" by (simp add: lastAcceptedValue_def) lemma lastAcceptedTerm_electionWon_update[simp]: "lastAcceptedTerm (electionWon_update f nd) = lastAcceptedTerm nd" by (simp add: lastAcceptedTerm_def) text \This method publishes a value via a @{term PublishRequest} message.\ definition publishValue :: "Value \ NodeData \ (NodeData * Message option)" where "publishValue x nd \ if electionWon nd \ publishPermitted nd then ( nd \ publishPermitted := False \ , Some (PublishRequest (firstUncommittedSlot nd) (currentTerm nd) x) ) else (nd, None)" text \This method updates the node's current term (if necessary) and discards any data associated with the previous term.\ definition ensureCurrentTerm :: "Term \ NodeData \ NodeData" where "ensureCurrentTerm t nd \ if t \ currentTerm nd then nd else nd \ joinVotes := {} , currentTerm := t , electionWon := False , publishPermitted := True , publishVotes := {} \" text \This method updates the node's state on receipt of a vote (a @{term Vote}) in an election.\ definition addElectionVote :: "Node \ Slot => TermOption \ NodeData \ NodeData" where "addElectionVote s i a nd \ let newVotes = insert s (joinVotes nd) in nd \ joinVotes := newVotes , electionWon := isQuorum nd newVotes \" text \Clients request the cluster to achieve consensus on certain values using the @{term ClientValue} message which is handled as follows.\ definition handleClientValue :: "Value \ NodeData \ (NodeData * Message option)" where "handleClientValue x nd \ if lastAcceptedTerm nd = NO_TERM then publishValue x nd else (nd, None)" text \A @{term StartJoin} message is checked for acceptability and then handled by updating the node's term and yielding a @{term Vote} message as follows.\ definition handleStartJoin :: "Term \ NodeData \ (NodeData * Message option)" where "handleStartJoin t nd \ if currentTerm nd < t then ( ensureCurrentTerm t nd , Some (Vote (firstUncommittedSlot nd) t (lastAcceptedTerm nd))) else (nd, None)" text \A @{term Vote} message is checked for acceptability and then handled as follows, perhaps yielding a @{term PublishRequest} message.\ definition handleVote :: "Node \ Slot \ Term \ TermOption \ NodeData \ (NodeData * Message option)" where "handleVote s i t a nd \ if t = currentTerm nd \ (i < firstUncommittedSlot nd \ (i = firstUncommittedSlot nd \ a \ lastAcceptedTerm nd)) then let nd1 = addElectionVote s i a nd in (if lastAcceptedTerm nd = NO_TERM then (nd1, None) else publishValue (lastAcceptedValue nd1) nd1) else (nd, None)" text \A @{term PublishRequest} message is checked for acceptability and then handled as follows, yielding a @{term PublishResponse} message.\ definition handlePublishRequest :: "Slot \ Term \ Value \ NodeData \ (NodeData * Message option)" where "handlePublishRequest i t x nd \ if i = firstUncommittedSlot nd \ t = currentTerm nd then ( nd \ lastAcceptedData := Some \ tvTerm = t, tvValue = x \ \ , Some (PublishResponse i t)) else (nd, None)" text \This method sends an @{term ApplyCommit} message if a quorum of votes has been received.\ definition commitIfQuorate :: "NodeData \ (NodeData * Message option)" where "commitIfQuorate nd = (nd, if isQuorum nd (publishVotes nd) then Some (ApplyCommit (firstUncommittedSlot nd) (currentTerm nd)) else None)" text \A @{term PublishResponse} message is checked for acceptability and handled as follows. If this message, together with the previously-received messages, forms a quorum of votes then the value is committed, yielding an @{term ApplyCommit} message.\ definition handlePublishResponse :: "Node \ Slot \ Term \ NodeData \ (NodeData * Message option)" where "handlePublishResponse s i t nd \ if i = firstUncommittedSlot nd \ t = currentTerm nd then commitIfQuorate (nd \ publishVotes := insert s (publishVotes nd) \) else (nd, None)" text \This method updates the node's state when a value is committed.\ definition applyAcceptedValue :: "NodeData \ NodeData" where "applyAcceptedValue nd \ case lastAcceptedValue nd of NoOp \ nd | Reconfigure votingNodes \ nd \ currentVotingNodes := set votingNodes , electionWon := joinVotes nd \ majorities (set votingNodes) \ | ClusterStateDiff diff \ nd \ currentClusterState := diff (currentClusterState nd) \" text \An @{term ApplyCommit} message is applied to the current node's state, updating its configuration and \texttt{ClusterState} via the @{term applyValue} method. It yields no messages.\ definition handleApplyCommit :: "Slot \ Term \ NodeData \ NodeData" where "handleApplyCommit i t nd \ if i = firstUncommittedSlot nd \ lastAcceptedTerm nd = SomeTerm t then (applyAcceptedValue nd) \ firstUncommittedSlot := i + 1 , lastAcceptedData := None , publishPermitted := True , publishVotes := {} \ else nd" definition handleCatchUpRequest :: "NodeData \ (NodeData * Message option)" where "handleCatchUpRequest nd = (nd, Some (CatchUpResponse (firstUncommittedSlot nd) (currentVotingNodes nd) (currentClusterState nd)))" definition handleCatchUpResponse :: "Slot \ Node set \ ClusterState \ NodeData \ NodeData" where "handleCatchUpResponse i conf cs nd \ if firstUncommittedSlot nd < i then nd \ firstUncommittedSlot := i , publishPermitted := True , publishVotes := {} , currentVotingNodes := conf , currentClusterState := cs , lastAcceptedData := None , joinVotes := {} , electionWon := False \ else nd" text \A @{term Reboot} message simulates the effect of a reboot, discarding any ephemeral state but preserving the persistent state. It yields no messages.\ definition handleReboot :: "NodeData \ NodeData" where "handleReboot nd \ \ currentNode = currentNode nd , currentTerm = currentTerm nd , firstUncommittedSlot = firstUncommittedSlot nd , currentVotingNodes = currentVotingNodes nd , currentClusterState = currentClusterState nd , lastAcceptedData = lastAcceptedData nd , joinVotes = {} , electionWon = False , publishPermitted = False , publishVotes = {} \" text \A @{term DiscardJoinVotes} message discards the votes received by a node. It yields no messages.\ definition handleDiscardJoinVotes :: "NodeData \ NodeData" where "handleDiscardJoinVotes nd \ nd \ electionWon := False, joinVotes := {} \" text \This function dispatches incoming messages to the appropriate handler method, and routes any responses to the appropriate places. In particular, @{term Vote} messages (sent by the @{term handleStartJoin} method) and @{term PublishResponse} messages (sent by the @{term handlePublishRequest} method) are only sent to a single node, whereas all other responses are broadcast to all nodes.\ definition ProcessMessage :: "NodeData \ RoutedMessage \ (NodeData * RoutedMessage option)" where "ProcessMessage nd msg \ let respondTo = (\ d (nd, mmsg). case mmsg of None \ (nd, None) | Some msg \ (nd, Some \ sender = currentNode nd, destination = d, payload = msg \)); respondToSender = respondTo (OneNode (sender msg)); respondToAll = respondTo Broadcast in if destination msg \ { Broadcast, OneNode (currentNode nd) } then case payload msg of StartJoin t \ respondToSender (handleStartJoin t nd) | Vote i t a \ respondToAll (handleVote (sender msg) i t a nd) | ClientValue x \ respondToAll (handleClientValue x nd) | PublishRequest i t x \ respondToSender (handlePublishRequest i t x nd) | PublishResponse i t \ respondToAll (handlePublishResponse (sender msg) i t nd) | ApplyCommit i t \ (handleApplyCommit i t nd, None) | CatchUpRequest \ respondToSender (handleCatchUpRequest nd) | CatchUpResponse i conf cs \ (handleCatchUpResponse i conf cs nd, None) | DiscardJoinVotes \ (handleDiscardJoinVotes nd, None) | Reboot \ (handleReboot nd, None) else (nd, None)" text \Nodes are initialised to this state. The data required is the initial configuration, @{term Q\<^sub>0} and the initial \texttt{ClusterState}, here shown as @{term "ClusterState 0"}.\ definition initialNodeState :: "Node \ NodeData" where "initialNodeState n = \ currentNode = n , currentTerm = 0 , firstUncommittedSlot = 0 , currentVotingNodes = V\<^sub>0 , currentClusterState = CS\<^sub>0 , lastAcceptedData = None , joinVotes = {} , electionWon = False , publishPermitted = False , publishVotes = {} \" (* Note: publishPermitted could be True initially, but in the actual implementation we call the same constructor whether we're starting up from afresh or recovering from a reboot, and the value is really unimportant as we need to run an election in a new term before becoming master anyway, so it's hard to justify putting any effort into calculating different values for these two cases. Instead just set it to False initially.*) end ================================================ FILE: cluster/isabelle/Monadic.thy ================================================ theory Monadic imports Implementation "~~/src/HOL/Library/Monad_Syntax" begin datatype Exception = IllegalArgumentException datatype ('e,'a) Result = Success 'a | Exception 'e datatype 'a Action = Action "NodeData \ (NodeData * RoutedMessage list * (Exception,'a) Result)" definition runM :: "'a Action \ NodeData \ (NodeData * RoutedMessage list * (Exception,'a) Result)" where "runM ma \ case ma of Action unwrapped_ma \ unwrapped_ma" lemma runM_Action[simp]: "runM (Action f) = f" by (simp add: runM_def) lemma runM_inject[intro]: "(\nd. runM ma nd = runM mb nd) \ ma = mb" by (cases ma, cases mb, auto simp add: runM_def) definition return :: "'a \ 'a Action" where "return a \ Action (\ nd. (nd, [], Success a))" lemma runM_return[simp]: "runM (return a) nd = (nd, [], Success a)" unfolding runM_def return_def by simp definition Action_bind :: "'a Action \ ('a \ 'b Action) \ 'b Action" where "Action_bind ma mf \ Action (\ nd0. case runM ma nd0 of (nd1, msgs1, result1) \ (case result1 of Exception e \ (nd1, msgs1, Exception e) | Success a \ (case runM (mf a) nd1 of (nd2, msgs2, result2) \ (nd2, msgs1 @ msgs2, result2))))" adhoc_overloading bind Action_bind lemma runM_bind: "runM (a \ f) nd0 = (case runM a nd0 of (nd1, msgs1, result1) \ (case result1 of Exception e \ (nd1, msgs1, Exception e) | Success b \ (case runM (f b) nd1 of (nd2, msgs2, c) \ (nd2, msgs1@msgs2, c))))" unfolding Action_bind_def by auto lemma return_bind[simp]: "do { a' <- return a; f a' } = f a" apply (intro runM_inject) by (simp add: runM_bind) lemma bind_return[simp]: "do { a' <- f; return a' } = f" proof (intro runM_inject) fix nd obtain nd1 msgs1 result1 where result1: "runM f nd = (nd1, msgs1, result1)" by (cases "runM f nd", blast) show "runM (f \ return) nd = runM f nd" by (cases result1, simp_all add: runM_bind result1) qed lemma bind_bind_assoc[simp]: fixes f :: "'a Action" shows "do { b <- do { a <- f; g a }; h b } = do { a <- f; b <- g a; h b }" (is "?LHS = ?RHS") proof (intro runM_inject) fix nd0 show "runM ?LHS nd0 = runM ?RHS nd0" proof (cases "runM f nd0") case fields1: (fields nd1 msgs1 result1) show ?thesis proof (cases result1) case Exception show ?thesis by (simp add: runM_bind fields1 Exception) next case Success1: (Success b) show ?thesis proof (cases "runM (g b) nd1") case fields2: (fields nd2 msgs2 result2) show ?thesis proof (cases result2) case Exception show ?thesis by (simp add: runM_bind fields1 fields2 Success1 Exception) next case Success2: (Success c) show ?thesis by (cases "runM (h c) nd2", simp add: runM_bind fields1 Success1 fields2 Success2) qed qed qed qed qed definition getNodeData :: "NodeData Action" where "getNodeData \ Action (\nd. (nd, [], Success nd))" definition setNodeData :: "NodeData \ unit Action" where "setNodeData nd \ Action (\_. (nd, [], Success ()))" lemma runM_getNodeData[simp]: "runM getNodeData nd = (nd, [], Success nd)" by (simp add: runM_def getNodeData_def) lemma runM_setNodeData[simp]: "runM (setNodeData nd') nd = (nd', [], Success ())" by (simp add: runM_def setNodeData_def) lemma runM_getNodeData_continue[simp]: "runM (do { nd' <- getNodeData; f nd' }) nd = runM (f nd) nd" by (simp add: runM_bind) lemma runM_setNodeData_continue[simp]: "runM (do { setNodeData nd'; f }) nd = runM f nd'" by (simp add: runM_bind) definition modifyNodeData :: "(NodeData \ NodeData) \ unit Action" where "modifyNodeData f = getNodeData \ (setNodeData \ f)" lemma runM_modifyNodeData[simp]: "runM (modifyNodeData f) nd = (f nd, [], Success ())" by (simp add: modifyNodeData_def runM_bind) lemma runM_modifyNodeData_continue[simp]: "runM (do { modifyNodeData f; a }) nd = runM a (f nd)" by (simp add: runM_bind) definition tell :: "RoutedMessage list \ unit Action" where "tell rms \ Action (\nd. (nd, rms, Success ()))" lemma runM_tell[simp]: "runM (tell rms) nd = (nd, rms, Success ())" by (simp add: runM_def tell_def) lemma runM_tell_contiue[simp]: "runM (do { tell rms; a }) nd = (let (nd, rms', x) = runM a nd in (nd, rms@rms', x))" by (simp add: runM_bind tell_def) definition send :: "RoutedMessage \ unit Action" where "send rm = tell [rm]" definition throw :: "Exception \ 'a Action" where "throw e = Action (\nd. (nd, [], Exception e))" lemma runM_throw[simp]: "runM (throw e) nd = (nd, [], Exception e)" by (simp add: runM_def throw_def) lemma throw_continue[simp]: "do { throw e; a } = throw e" by (intro runM_inject, simp add: runM_bind) definition catch :: "'a Action \ (Exception \ 'a Action) \ 'a Action" where "catch go onException = Action (\nd0. case runM go nd0 of (nd1, rms1, result1) \ (case result1 of Success _ \ (nd1, rms1, result1) | Exception e \ runM (tell rms1 \ onException e) nd1))" lemma catch_throw[simp]: "catch (throw e) handle = handle e" by (intro runM_inject, simp add: catch_def) lemma catch_return[simp]: "catch (return a) handle = return a" by (intro runM_inject, simp add: catch_def) lemma catch_getNodeData[simp]: "catch getNodeData handle = getNodeData" by (intro runM_inject, simp add: catch_def) lemma catch_getNodeData_continue[simp]: "catch (do { nd <- getNodeData; f nd }) handle = do { nd <- getNodeData; catch (f nd) handle }" by (intro runM_inject, simp add: catch_def) lemma catch_setNodeData[simp]: "catch (setNodeData nd) handle = setNodeData nd" by (intro runM_inject, simp add: catch_def) lemma catch_setNodeData_continue[simp]: "catch (do { setNodeData nd; f }) handle = do { setNodeData nd; catch f handle }" by (intro runM_inject, simp add: catch_def) lemma catch_modifyNodeData[simp]: "catch (modifyNodeData f) handle = modifyNodeData f" by (intro runM_inject, simp add: catch_def) lemma catch_modifyNodeData_continue[simp]: "catch (do { modifyNodeData f; g }) handle = do { modifyNodeData f; catch g handle }" by (intro runM_inject, simp add: catch_def) lemma catch_tell[simp]: "catch (tell rms) handle = tell rms" by (intro runM_inject, simp add: catch_def) lemma catch_tell_continue[simp]: "catch (do { tell rms; f }) handle = do { tell rms; catch f handle }" proof (intro runM_inject) fix nd0 show "runM (catch (do { tell rms; f }) handle) nd0 = runM (do { tell rms; catch f handle }) nd0" proof (cases "runM f nd0") case fields1: (fields nd1 msgs1 result1) show ?thesis proof (cases result1) case (Exception e) show ?thesis by (cases "runM (handle e) nd1", simp add: catch_def fields1 Exception) next case Success1: (Success b) show ?thesis by (simp add: catch_def fields1 Success1) qed qed qed lemma catch_send[simp]: "catch (send rm) handle = send rm" by (simp add: send_def) lemma catch_send_continue[simp]: "catch (do { send rm; f }) handle = do { send rm; catch f handle }" by (simp add: send_def) definition gets :: "(NodeData \ 'a) \ 'a Action" where "gets f \ do { nd <- getNodeData; return (f nd) }" definition getCurrentClusterState where "getCurrentClusterState = gets currentClusterState" definition getCurrentNode where "getCurrentNode = gets currentNode" definition getCurrentTerm where "getCurrentTerm = gets currentTerm" definition getCurrentVotingNodes where "getCurrentVotingNodes = gets currentVotingNodes" definition getElectionWon where "getElectionWon = gets electionWon" definition getFirstUncommittedSlot where "getFirstUncommittedSlot = gets firstUncommittedSlot" definition getJoinVotes where "getJoinVotes = gets joinVotes" definition getLastAcceptedData where "getLastAcceptedData = gets lastAcceptedData" definition getPublishPermitted where "getPublishPermitted = gets publishPermitted" definition getPublishVotes where "getPublishVotes = gets publishVotes" definition sets where "sets f x = modifyNodeData (f (\_. x))" definition setCurrentClusterState where "setCurrentClusterState = sets currentClusterState_update" definition setCurrentNode where "setCurrentNode = sets currentNode_update" definition setCurrentTerm where "setCurrentTerm = sets currentTerm_update" definition setCurrentVotingNodes where "setCurrentVotingNodes = sets currentVotingNodes_update" definition setElectionWon where "setElectionWon = sets electionWon_update" definition setFirstUncommittedSlot where "setFirstUncommittedSlot = sets firstUncommittedSlot_update" definition setJoinVotes where "setJoinVotes = sets joinVotes_update" definition setLastAcceptedData where "setLastAcceptedData = sets lastAcceptedData_update" definition setPublishPermitted where "setPublishPermitted = sets publishPermitted_update" definition setPublishVotes where "setPublishVotes = sets publishVotes_update" definition modifies where "modifies f g = modifyNodeData (f g)" definition modifyJoinVotes where "modifyJoinVotes = modifies joinVotes_update" definition modifyPublishVotes where "modifyPublishVotes = modifies publishVotes_update" definition modifyCurrentClusterState where "modifyCurrentClusterState = modifies currentClusterState_update" definition "when" :: "bool \ unit Action \ unit Action" where "when c a \ if c then a else return ()" definition unless :: "bool \ unit Action \ unit Action" where "unless \ when \ Not" lemma runM_when: "runM (when c a) nd = (if c then runM a nd else (nd, [], Success ()))" by (auto simp add: when_def) lemma runM_unless: "runM (unless c a) nd = (if c then (nd, [], Success ()) else runM a nd)" by (auto simp add: unless_def when_def) lemma runM_when_continue: "runM (do { when c a; b }) nd = (if c then runM (do {a;b}) nd else runM b nd)" by (auto simp add: when_def) lemma runM_unless_continue: "runM (do { unless c a; b }) nd = (if c then runM b nd else runM (do {a;b}) nd)" by (auto simp add: unless_def when_def) lemma catch_when[simp]: "catch (when c a) onException = when c (catch a onException)" by (intro runM_inject, simp add: catch_def runM_when) lemma catch_unless[simp]: "catch (unless c a) onException = unless c (catch a onException)" by (intro runM_inject, simp add: catch_def runM_unless) lemma catch_when_continue[simp]: "catch (do { when c a; b }) onException = (if c then catch (do {a;b}) onException else catch b onException)" by (intro runM_inject, simp add: catch_def runM_when_continue) lemma catch_unless_continue[simp]: "catch (do { unless c a; b }) onException = (if c then catch b onException else catch (do {a;b}) onException)" by (intro runM_inject, simp add: catch_def runM_unless_continue) definition ensureCorrectDestination :: "Destination \ unit Action" where "ensureCorrectDestination d \ do { n <- getCurrentNode; when (d \ { Broadcast, OneNode n }) (throw IllegalArgumentException) }" lemma runM_ensureCorrectDestination_continue: "runM (do { ensureCorrectDestination d; go }) nd = (if d \ { Broadcast, OneNode (currentNode nd) } then runM go nd else (nd, [], Exception IllegalArgumentException))" by (simp add: ensureCorrectDestination_def getCurrentNode_def gets_def runM_when_continue) definition broadcast :: "Message \ unit Action" where "broadcast msg \ do { n <- getCurrentNode; send \ sender = n, destination = Broadcast, payload = msg \ }" lemma runM_broadcast[simp]: "runM (broadcast msg) nd = (nd, [\ sender = currentNode nd, destination = Broadcast, payload = msg \], Success ())" by (simp add: broadcast_def getCurrentNode_def gets_def send_def) definition sendTo :: "Node \ Message \ unit Action" where "sendTo d msg \ do { n <- getCurrentNode; send \ sender = n, destination = OneNode d, payload = msg \ }" lemma runM_sendTo[simp]: "runM (sendTo d msg) nd = (nd, [\ sender = currentNode nd, destination = OneNode d, payload = msg \], Success ())" by (simp add: sendTo_def getCurrentNode_def gets_def send_def) definition ignoringExceptions :: "unit Action \ unit Action" where "ignoringExceptions go \ catch go (\_. return ())" lemma None_lt[simp]: "NO_TERM < t = (t \ NO_TERM)" by (cases t, simp_all) definition getLastAcceptedTerm :: "TermOption Action" where "getLastAcceptedTerm \ do { lastAcceptedData <- getLastAcceptedData; case lastAcceptedData of None \ return NO_TERM | Some tv \ return (SomeTerm (tvTerm tv)) }" definition doStartJoin :: "Node \ Term \ unit Action" where "doStartJoin newMaster newTerm \ do { currentTerm <- getCurrentTerm; when (newTerm \ currentTerm) (throw IllegalArgumentException); setCurrentTerm newTerm; setJoinVotes {}; setElectionWon False; setPublishPermitted True; setPublishVotes {}; firstUncommittedSlot <- getFirstUncommittedSlot; lastAcceptedTerm <- getLastAcceptedTerm; sendTo newMaster (Vote firstUncommittedSlot newTerm lastAcceptedTerm) }" definition doVote :: "Node \ Slot \ Term \ TermOption \ unit Action" where "doVote sourceNode voteFirstUncommittedSlot voteTerm voteLastAcceptedTerm \ do { currentTerm <- getCurrentTerm; when (voteTerm \ currentTerm) (throw IllegalArgumentException); firstUncommittedSlot <- getFirstUncommittedSlot; when (voteFirstUncommittedSlot > firstUncommittedSlot) (throw IllegalArgumentException); lastAcceptedTerm <- getLastAcceptedTerm; when (voteFirstUncommittedSlot = firstUncommittedSlot \ voteLastAcceptedTerm > lastAcceptedTerm) (throw IllegalArgumentException); modifyJoinVotes (insert sourceNode); joinVotes <- getJoinVotes; currentVotingNodes <- getCurrentVotingNodes; let electionWon' = card (joinVotes \ currentVotingNodes) * 2 > card currentVotingNodes; setElectionWon electionWon'; publishPermitted <- getPublishPermitted; when (electionWon' \ publishPermitted \ lastAcceptedTerm \ NO_TERM) (do { setPublishPermitted False; lastAcceptedValue <- gets lastAcceptedValue; (* NB must be present since lastAcceptedTermInSlot \ NO_TERM *) broadcast (PublishRequest firstUncommittedSlot currentTerm lastAcceptedValue) }) }" definition doPublishRequest :: "Node \ Slot \ TermValue \ unit Action" where "doPublishRequest sourceNode requestSlot newAcceptedState \ do { currentTerm <- getCurrentTerm; when (tvTerm newAcceptedState \ currentTerm) (throw IllegalArgumentException); firstUncommittedSlot <- getFirstUncommittedSlot; when (requestSlot \ firstUncommittedSlot) (throw IllegalArgumentException); setLastAcceptedData (Some newAcceptedState); sendTo sourceNode (PublishResponse requestSlot (tvTerm newAcceptedState)) }" record SlotTerm = stSlot :: Slot stTerm :: Term definition ApplyCommitFromSlotTerm :: "SlotTerm \ Message" where "ApplyCommitFromSlotTerm st = ApplyCommit (stSlot st) (stTerm st)" definition doPublishResponse :: "Node \ SlotTerm \ unit Action" where "doPublishResponse sourceNode slotTerm \ do { currentTerm <- getCurrentTerm; when (stTerm slotTerm \ currentTerm) (throw IllegalArgumentException); firstUncommittedSlot <- getFirstUncommittedSlot; when (stSlot slotTerm \ firstUncommittedSlot) (throw IllegalArgumentException); modifyPublishVotes (insert sourceNode); publishVotes <- getPublishVotes; currentVotingNodes <- getCurrentVotingNodes; when (card (publishVotes \ currentVotingNodes) * 2 > card currentVotingNodes) (broadcast (ApplyCommitFromSlotTerm slotTerm)) }" definition doCommit :: "SlotTerm \ unit Action" where "doCommit slotTerm \ do { lastAcceptedTermInSlot <- getLastAcceptedTerm; when (SomeTerm (stTerm slotTerm) \ lastAcceptedTermInSlot) (throw IllegalArgumentException); firstUncommittedSlot <- getFirstUncommittedSlot; when (stSlot slotTerm \ firstUncommittedSlot) (throw IllegalArgumentException); lastAcceptedValue <- gets lastAcceptedValue; (* NB must be not None since lastAcceptedTerm = Some t *) (case lastAcceptedValue of ClusterStateDiff diff \ modifyCurrentClusterState diff | Reconfigure votingNodes \ do { setCurrentVotingNodes (set votingNodes); joinVotes <- getJoinVotes; setElectionWon (card (joinVotes \ (set votingNodes)) * 2 > card (set votingNodes)) } | NoOp \ return ()); setFirstUncommittedSlot (firstUncommittedSlot + 1); setLastAcceptedData None; setPublishPermitted True; setPublishVotes {} }" definition generateCatchup :: "Node \ unit Action" where "generateCatchup sourceNode \ do { firstUncommittedSlot <- getFirstUncommittedSlot; currentVotingNodes <- getCurrentVotingNodes; currentClusterState <- getCurrentClusterState; sendTo sourceNode (CatchUpResponse firstUncommittedSlot currentVotingNodes currentClusterState) }" definition applyCatchup :: "Slot \ Node set \ ClusterState \ unit Action" where "applyCatchup catchUpSlot catchUpConfiguration catchUpState \ do { firstUncommittedSlot <- getFirstUncommittedSlot; when (catchUpSlot \ firstUncommittedSlot) (throw IllegalArgumentException); setFirstUncommittedSlot catchUpSlot; setCurrentVotingNodes catchUpConfiguration; setCurrentClusterState catchUpState; setLastAcceptedData None; setJoinVotes {}; setElectionWon False; setPublishVotes {}; setPublishPermitted True }" definition doClientValue :: "Value \ unit Action" where "doClientValue x \ do { electionWon <- getElectionWon; when (\ electionWon) (throw IllegalArgumentException); publishPermitted <- getPublishPermitted; when (\ publishPermitted) (throw IllegalArgumentException); lastAcceptedTermInSlot <- getLastAcceptedTerm; when (lastAcceptedTermInSlot \ NO_TERM) (throw IllegalArgumentException); setPublishPermitted False; currentTerm <- getCurrentTerm; firstUncommittedSlot <- getFirstUncommittedSlot; broadcast (PublishRequest firstUncommittedSlot currentTerm x) }" definition doDiscardJoinVotes :: "unit Action" where "doDiscardJoinVotes \ do { setJoinVotes {}; setElectionWon False }" definition doReboot :: "unit Action" where "doReboot \ modifyNodeData (\nd. (* persistent fields *) \ currentNode = currentNode nd , currentTerm = currentTerm nd , firstUncommittedSlot = firstUncommittedSlot nd , currentVotingNodes = currentVotingNodes nd , currentClusterState = currentClusterState nd , lastAcceptedData = lastAcceptedData nd (* transient fields *) , joinVotes = {} , electionWon = False , publishPermitted = False , publishVotes = {} \)" definition ProcessMessageAction :: "RoutedMessage \ unit Action" where "ProcessMessageAction rm \ Action (\nd. case ProcessMessage nd rm of (nd', messageOption) \ (nd', case messageOption of None \ [] | Some m \ [m], Success ()))" definition dispatchMessageInner :: "RoutedMessage \ unit Action" where "dispatchMessageInner m \ case payload m of StartJoin t \ doStartJoin (sender m) t | Vote i t a \ doVote (sender m) i t a | ClientValue x \ doClientValue x | PublishRequest i t x \ doPublishRequest (sender m) i \ tvTerm = t, tvValue = x \ | PublishResponse i t \ doPublishResponse (sender m) \ stSlot = i, stTerm = t \ | ApplyCommit i t \ doCommit \ stSlot = i, stTerm = t \ | CatchUpRequest \ generateCatchup (sender m) | CatchUpResponse i conf cs \ applyCatchup i conf cs | DiscardJoinVotes \ doDiscardJoinVotes | Reboot \ doReboot" definition dispatchMessage :: "RoutedMessage \ unit Action" where "dispatchMessage m \ ignoringExceptions (do { ensureCorrectDestination (destination m); dispatchMessageInner m })" lemma getLastAcceptedTermInSlot_gets[simp]: "getLastAcceptedTerm = gets lastAcceptedTerm" proof (intro runM_inject) fix nd show "runM getLastAcceptedTerm nd = runM (gets lastAcceptedTerm) nd" by (cases "lastAcceptedData nd", simp_all add: gets_def getLastAcceptedTerm_def getLastAcceptedData_def getFirstUncommittedSlot_def lastAcceptedTerm_def) qed lemma monadic_implementation_is_faithful: "dispatchMessage = ProcessMessageAction" proof (intro ext runM_inject) fix rm nd show "runM (dispatchMessage rm) nd = runM (ProcessMessageAction rm) nd" (is "?LHS = ?RHS") proof (cases "destination rm \ {Broadcast, OneNode (currentNode nd)}") case False hence 1: "\f. runM (do { ensureCorrectDestination (destination rm); f }) nd = (nd, [], Exception IllegalArgumentException)" by (simp add: runM_ensureCorrectDestination_continue) from False show ?thesis unfolding ProcessMessageAction_def dispatchMessage_def by (simp add: ignoringExceptions_def catch_def 1 ProcessMessage_def) next case dest_ok: True hence 1: "runM (dispatchMessage rm) nd = runM (ignoringExceptions (dispatchMessageInner rm)) nd" by (simp add: dispatchMessage_def ignoringExceptions_def catch_def runM_ensureCorrectDestination_continue) also have "... = runM (ProcessMessageAction rm) nd" (is "?LHS = ?RHS") proof (cases "payload rm") case (StartJoin t) have "?LHS = runM (ignoringExceptions (doStartJoin (sender rm) t)) nd" (is "_ = ?STEP") by (simp add: dispatchMessageInner_def StartJoin) also consider (a) "t \ currentTerm nd" | (b) "currentTerm nd < t" "case lastAcceptedTerm nd of NO_TERM \ False | SomeTerm x \ t \ x" | (c) "currentTerm nd < t" "case lastAcceptedTerm nd of NO_TERM \ True | SomeTerm x \ x < t" proof (cases "t \ currentTerm nd") case True thus ?thesis by (intro a) next case 1: False with b c show ?thesis by (cases "case lastAcceptedTerm nd of NO_TERM \ False | SomeTerm x \ t \ x", auto, cases "lastAcceptedTerm nd", auto) qed hence "?STEP = ?RHS" proof cases case a thus ?thesis by (simp add: StartJoin ProcessMessageAction_def dispatchMessage_def ProcessMessage_def Let_def runM_unless doStartJoin_def getCurrentTerm_def gets_def setJoinVotes_def sets_def setCurrentTerm_def setPublishPermitted_def setPublishVotes_def getFirstUncommittedSlot_def handleStartJoin_def ensureCurrentTerm_def setElectionWon_def ignoringExceptions_def catch_def runM_when_continue) next case b with StartJoin dest_ok show ?thesis by (cases "lastAcceptedTerm nd ", simp_all add: ProcessMessageAction_def dispatchMessage_def ProcessMessage_def Let_def doStartJoin_def getCurrentTerm_def gets_def setJoinVotes_def sets_def setCurrentTerm_def runM_unless lastAcceptedTerm_def setPublishPermitted_def setPublishVotes_def getFirstUncommittedSlot_def handleStartJoin_def ensureCurrentTerm_def setElectionWon_def ignoringExceptions_def catch_def runM_when_continue) next case c with StartJoin dest_ok show ?thesis by (cases "lastAcceptedTerm nd", simp_all add: ProcessMessageAction_def dispatchMessage_def ProcessMessage_def Let_def doStartJoin_def getCurrentTerm_def gets_def setJoinVotes_def sets_def setCurrentTerm_def runM_unless lastAcceptedTerm_def setPublishPermitted_def setPublishVotes_def getFirstUncommittedSlot_def handleStartJoin_def ensureCurrentTerm_def setElectionWon_def ignoringExceptions_def catch_def runM_when_continue) qed finally show ?thesis by simp next case (Vote i t a) have "?LHS = runM (ignoringExceptions (doVote (sender rm) i t a)) nd" (is "_ = ?STEP") by (simp add: dispatchMessageInner_def Vote) also have "... = ?RHS" proof (cases "firstUncommittedSlot nd < i") case True with Vote dest_ok show ?thesis by (simp add: dispatchMessage_def runM_unless doVote_def gets_def getFirstUncommittedSlot_def ProcessMessage_def ProcessMessageAction_def handleVote_def ignoringExceptions_def getCurrentTerm_def) next case False hence le: "i \ firstUncommittedSlot nd" by simp show ?thesis proof (cases "t = currentTerm nd") case False with Vote dest_ok le show ?thesis by (simp add: dispatchMessage_def runM_when runM_unless doVote_def gets_def getFirstUncommittedSlot_def getCurrentTerm_def ProcessMessage_def ProcessMessageAction_def handleVote_def ignoringExceptions_def) next case t: True show ?thesis proof (cases "i = firstUncommittedSlot nd") case False with Vote dest_ok le t show ?thesis by (simp add: dispatchMessage_def Let_def runM_when_continue doVote_def runM_when runM_unless gets_def getFirstUncommittedSlot_def getCurrentTerm_def getJoinVotes_def getCurrentVotingNodes_def getPublishPermitted_def ignoringExceptions_def broadcast_def getCurrentNode_def modifies_def modifyJoinVotes_def send_def sets_def setElectionWon_def setPublishPermitted_def lastAcceptedValue_def ProcessMessage_def ProcessMessageAction_def handleVote_def addElectionVote_def publishValue_def isQuorum_def majorities_def) next case i: True show ?thesis proof (cases a) case a: NO_TERM show ?thesis proof (cases "isQuorum nd (insert (sender rm) (joinVotes nd))") case not_quorum: False hence not_quorum_card: "\ card (currentVotingNodes nd) < card (insert (sender rm) (joinVotes nd) \ currentVotingNodes nd) * 2" by (simp add: isQuorum_def majorities_def) have "?STEP = (nd\electionWon := False, joinVotes := insert (sender rm) (joinVotes nd)\, [], Success ())" by (simp add: ignoringExceptions_def i t a doVote_def catch_def gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def modifyJoinVotes_def modifies_def getJoinVotes_def getCurrentVotingNodes_def Let_def setElectionWon_def sets_def runM_when not_quorum_card getPublishPermitted_def) also from dest_ok have "... = ?RHS" by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def i t a addElectionVote_def not_quorum publishValue_def Let_def) finally show ?thesis . next case quorum: True hence quorum_card: "card (currentVotingNodes nd) < card (insert (sender rm) (joinVotes nd) \ currentVotingNodes nd) * 2" by (simp add: isQuorum_def majorities_def) show ?thesis proof (cases "publishPermitted nd \ lastAcceptedTerm nd \ NO_TERM") case False hence "?STEP = (nd\electionWon := True, joinVotes := insert (sender rm) (joinVotes nd)\, [], Success ())" by (auto simp add: ignoringExceptions_def i t a doVote_def catch_def gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def modifyJoinVotes_def modifies_def getJoinVotes_def getCurrentVotingNodes_def Let_def setElectionWon_def sets_def runM_when quorum_card getPublishPermitted_def) also from False dest_ok have "... = ?RHS" by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def i t a addElectionVote_def quorum publishValue_def Let_def) finally show ?thesis . next case True hence "?STEP = (nd\electionWon := True, publishPermitted := False, joinVotes := insert (sender rm) (joinVotes nd)\, [\sender = currentNode nd, destination = Broadcast, payload = PublishRequest (firstUncommittedSlot nd) (currentTerm nd) (lastAcceptedValue nd) \], Success ())" by (auto simp add: ignoringExceptions_def i t a doVote_def catch_def gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def modifyJoinVotes_def modifies_def getJoinVotes_def getCurrentVotingNodes_def Let_def setElectionWon_def sets_def runM_when quorum_card getPublishPermitted_def setPublishPermitted_def lastAcceptedValue_def) also from True dest_ok have "... = ?RHS" by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def i t a addElectionVote_def quorum publishValue_def Let_def lastAcceptedValue_def) finally show ?thesis . qed qed next case a: (SomeTerm voteLastAcceptedTerm) show ?thesis proof (cases "lastAcceptedTerm nd") case lat: NO_TERM have "?STEP = (nd, [], Success ())" by (auto simp add: ignoringExceptions_def i t a lat doVote_def catch_def gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def) also from dest_ok have "... = ?RHS" by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def i t a lat) finally show ?thesis . next case lat: (SomeTerm nodeLastAcceptedTerm) show ?thesis proof (cases "voteLastAcceptedTerm \ nodeLastAcceptedTerm") case False hence "?STEP = (nd, [], Success ())" by (simp add: ignoringExceptions_def i t a lat doVote_def catch_def gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def) also from False dest_ok have "... = ?RHS" by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def i t a lat max_def addElectionVote_def publishValue_def) finally show ?thesis by simp next case True show ?thesis proof (cases "isQuorum nd (insert (sender rm) (joinVotes nd))") case not_quorum: False hence not_quorum_card: "\ card (currentVotingNodes nd) < card (insert (sender rm) (joinVotes nd) \ currentVotingNodes nd) * 2" by (simp add: isQuorum_def majorities_def) from True have "?STEP = (nd\electionWon := False, joinVotes := insert (sender rm) (joinVotes nd)\, [], Success ())" by (simp add: ignoringExceptions_def i t a lat doVote_def catch_def gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def modifyJoinVotes_def modifies_def getJoinVotes_def getCurrentVotingNodes_def Let_def setElectionWon_def sets_def runM_when not_quorum_card getPublishPermitted_def) also from dest_ok True have "... = ?RHS" by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def i t a lat max_def addElectionVote_def not_quorum publishValue_def Let_def) finally show ?thesis . next case quorum: True hence quorum_card: "card (currentVotingNodes nd) < card (insert (sender rm) (joinVotes nd) \ currentVotingNodes nd) * 2" by (simp add: isQuorum_def majorities_def) show ?thesis proof (cases "publishPermitted nd") case False with True have "?STEP = (nd\electionWon := True, joinVotes := insert (sender rm) (joinVotes nd)\, [], Success ())" by (simp add: ignoringExceptions_def i t a lat doVote_def catch_def gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def modifyJoinVotes_def modifies_def getJoinVotes_def getCurrentVotingNodes_def Let_def setElectionWon_def sets_def runM_when quorum_card getPublishPermitted_def setPublishPermitted_def) also from False dest_ok have "... = ?RHS" by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def i t a lat True addElectionVote_def quorum publishValue_def Let_def) finally show ?thesis . next case publishPermitted: True have "?STEP = (nd\electionWon := True, publishPermitted := False, joinVotes := insert (sender rm) (joinVotes nd)\, [\sender = currentNode nd, destination = Broadcast, payload = PublishRequest (firstUncommittedSlot nd) (currentTerm nd) (lastAcceptedValue nd) \], Success ())" apply (auto simp add: ignoringExceptions_def i t a lat True doVote_def catch_def gets_def getCurrentTerm_def runM_when_continue getFirstUncommittedSlot_def modifyJoinVotes_def modifies_def getJoinVotes_def getCurrentVotingNodes_def Let_def setElectionWon_def sets_def runM_when quorum_card getPublishPermitted_def setPublishPermitted_def lastAcceptedValue_def) using True publishPermitted by auto also from publishPermitted True dest_ok have "... = ?RHS" by (simp add: ProcessMessageAction_def ProcessMessage_def Vote handleVote_def i t a lat True addElectionVote_def quorum publishValue_def Let_def lastAcceptedValue_def) finally show ?thesis . qed qed qed qed qed qed qed qed finally show ?thesis . next case (ClientValue x) with dest_ok show ?thesis by (simp add: ProcessMessageAction_def dispatchMessageInner_def doClientValue_def gets_def getElectionWon_def runM_unless getPublishPermitted_def setPublishPermitted_def sets_def getCurrentTerm_def getFirstUncommittedSlot_def ProcessMessage_def handleClientValue_def publishValue_def runM_when ignoringExceptions_def ClientValue catch_def runM_when_continue) next case (PublishRequest i t x) with dest_ok show ?thesis by (simp add: ProcessMessageAction_def dispatchMessageInner_def doPublishRequest_def gets_def getCurrentTerm_def getFirstUncommittedSlot_def sets_def setLastAcceptedData_def ignoringExceptions_def catch_def runM_when_continue getCurrentNode_def runM_unless send_def ProcessMessage_def handlePublishRequest_def runM_when) next case (PublishResponse i t) with dest_ok show ?thesis by (simp add: ProcessMessageAction_def dispatchMessageInner_def doPublishResponse_def gets_def getCurrentTerm_def getFirstUncommittedSlot_def broadcast_def getCurrentNode_def runM_unless send_def modifyPublishVotes_def modifies_def getPublishVotes_def getCurrentVotingNodes_def runM_when ignoringExceptions_def catch_def runM_when_continue ProcessMessage_def handlePublishResponse_def commitIfQuorate_def isQuorum_def majorities_def ApplyCommitFromSlotTerm_def) next case (ApplyCommit i t) show ?thesis proof (cases "lastAcceptedValue nd") case NoOp with ApplyCommit dest_ok show ?thesis by (simp add: ProcessMessageAction_def dispatchMessageInner_def doCommit_def runM_unless runM_when gets_def getFirstUncommittedSlot_def sets_def setFirstUncommittedSlot_def setLastAcceptedData_def setPublishPermitted_def setPublishVotes_def ProcessMessage_def handleApplyCommit_def applyAcceptedValue_def ignoringExceptions_def catch_def runM_when_continue) next case Reconfigure with ApplyCommit dest_ok show ?thesis by (simp add: ProcessMessageAction_def dispatchMessageInner_def doCommit_def runM_unless runM_when gets_def getFirstUncommittedSlot_def getJoinVotes_def sets_def setFirstUncommittedSlot_def setLastAcceptedData_def setPublishPermitted_def setPublishVotes_def setCurrentVotingNodes_def setElectionWon_def ProcessMessage_def handleApplyCommit_def applyAcceptedValue_def majorities_def ignoringExceptions_def catch_def runM_when_continue) next case ClusterStateDiff with ApplyCommit dest_ok show ?thesis by (simp add: ProcessMessageAction_def dispatchMessageInner_def doCommit_def runM_unless runM_when gets_def getFirstUncommittedSlot_def sets_def setFirstUncommittedSlot_def modifies_def modifyCurrentClusterState_def setPublishPermitted_def setPublishVotes_def setLastAcceptedData_def ProcessMessage_def handleApplyCommit_def applyAcceptedValue_def ignoringExceptions_def catch_def runM_when_continue) qed next case CatchUpRequest with dest_ok show ?thesis by (simp add: ProcessMessageAction_def dispatchMessageInner_def generateCatchup_def gets_def getFirstUncommittedSlot_def getCurrentVotingNodes_def getCurrentClusterState_def ProcessMessage_def handleCatchUpRequest_def ignoringExceptions_def catch_def runM_when_continue) next case (CatchUpResponse i conf cs) with dest_ok show ?thesis by (simp add: ProcessMessageAction_def dispatchMessageInner_def applyCatchup_def gets_def getFirstUncommittedSlot_def sets_def setFirstUncommittedSlot_def setPublishPermitted_def setPublishVotes_def setLastAcceptedData_def setCurrentVotingNodes_def setCurrentClusterState_def setJoinVotes_def setElectionWon_def runM_unless ProcessMessage_def handleCatchUpResponse_def ignoringExceptions_def catch_def runM_when_continue) next case Reboot with dest_ok show ?thesis by (simp add: ProcessMessageAction_def dispatchMessageInner_def doReboot_def ProcessMessage_def handleReboot_def ignoringExceptions_def catch_def runM_when_continue) next case DiscardJoinVotes with dest_ok show ?thesis by (simp add: ProcessMessageAction_def dispatchMessageInner_def doDiscardJoinVotes_def ProcessMessage_def handleDiscardJoinVotes_def ignoringExceptions_def catch_def runM_when_continue setJoinVotes_def sets_def setElectionWon_def) qed finally show ?thesis . qed qed end ================================================ FILE: cluster/isabelle/OneSlot.thy ================================================ section \One-slot consistency\ text \The replicated state machine determines the values that are committed in each of a sequence of \textit{slots}. Each slot runs a logically-separate consensus algorithm which is shown to be consistent here. Further below, the protocol is shown to refine this slot-by-slot model correctly.\ text \Consistency is shown to follow from the invariants listed below. Further below, the protocol is shown to preserve these invariants in each step, which means it is not enormously important to understand these in detail.\ theory OneSlot imports Preliminaries begin locale oneSlot = (* basic functions *) fixes Q :: "Node set set" fixes v :: "Term \ Value" (* message-sent predicates *) fixes promised\<^sub>f :: "Node \ Term \ bool" fixes promised\<^sub>b :: "Node \ Term \ Term \ bool" fixes proposed :: "Term \ bool" fixes accepted :: "Node \ Term \ bool" fixes committed :: "Term \ bool" (* other definitions *) fixes promised :: "Node \ Term \ bool" defines "promised n t \ promised\<^sub>f n t \ (\ t'. promised\<^sub>b n t t')" fixes prevAccepted :: "Term \ Node set \ Term set" defines "prevAccepted t ns \ {t'. \ n \ ns. promised\<^sub>b n t t'}" (* invariants *) assumes Q_intersects: "Q \ Q" assumes promised\<^sub>f: "\ promised\<^sub>f n t; t' < t \ \ \ accepted n t'" assumes promised\<^sub>b_lt: "promised\<^sub>b n t t' \ t' < t" assumes promised\<^sub>b_accepted: "promised\<^sub>b n t t' \ accepted n t'" assumes promised\<^sub>b_max: "\ promised\<^sub>b n t t'; t' < t''; t'' < t \ \ \ accepted n t''" assumes proposed: "proposed t \ \ q \ Q. (\ n \ q. promised n t) \ (prevAccepted t q = {} \ (\ t'. v t = v t' \ maxTerm (prevAccepted t q) \ t' \ proposed t' \ t' < t))" assumes proposed_finite: "finite {t. proposed t}" assumes accepted: "accepted n t \ proposed t" assumes committed: "committed t \ \ q \ Q. \ n \ q. accepted n t" lemma (in oneSlot) prevAccepted_proposed: "prevAccepted t ns \ {t. proposed t}" using accepted prevAccepted_def promised\<^sub>b_accepted by fastforce lemma (in oneSlot) prevAccepted_finite: "finite (prevAccepted p ns)" using prevAccepted_proposed proposed_finite by (meson rev_finite_subset) lemma (in oneSlot) Q_nonempty: "\q. q \ Q \ q \ {}" using Q_intersects by (auto simp add: intersects_def) text \The heart of the consistency proof is property P2b from \textit{Paxos made simple} by Lamport:\ lemma (in oneSlot) p2b: assumes "proposed t\<^sub>1" and "committed t\<^sub>2" and "t\<^sub>2 < t\<^sub>1" shows "v t\<^sub>1 = v t\<^sub>2" using assms proof (induct t\<^sub>1 rule: less_induct) case (less t\<^sub>1) hence hyp: "\ t\<^sub>1'. \ t\<^sub>1' < t\<^sub>1; proposed t\<^sub>1'; t\<^sub>2 \ t\<^sub>1' \ \ v t\<^sub>1' = v t\<^sub>2" using le_imp_less_or_eq by blast from `proposed t\<^sub>1` obtain q\<^sub>1 t\<^sub>1' where q\<^sub>1_quorum: "q\<^sub>1 \ Q" and q\<^sub>1_promised: "\n. n \ q\<^sub>1 \ promised n t\<^sub>1" and q\<^sub>1_value: "prevAccepted t\<^sub>1 q\<^sub>1 = {} \ (v t\<^sub>1 = v t\<^sub>1' \ maxTerm (prevAccepted t\<^sub>1 q\<^sub>1) \ t\<^sub>1' \ proposed t\<^sub>1' \ t\<^sub>1' < t\<^sub>1)" by (meson proposed) from `committed t\<^sub>2` obtain q\<^sub>2 where q\<^sub>2_quorum: "q\<^sub>2 \ Q" and q\<^sub>2_accepted: "\n. n \ q\<^sub>2 \ accepted n t\<^sub>2" using committed by force have "q\<^sub>1 \ q\<^sub>2 \ {}" using Q_intersects intersects_def less.prems q\<^sub>1_quorum q\<^sub>2_quorum by auto then obtain n where n\<^sub>1: "n \ q\<^sub>1" and n\<^sub>2: "n \ q\<^sub>2" by auto from n\<^sub>1 q\<^sub>1_promised have "promised n t\<^sub>1" by simp moreover from n\<^sub>2 q\<^sub>2_accepted have "accepted n t\<^sub>2" by simp ultimately obtain t\<^sub>2' where t\<^sub>2': "promised\<^sub>b n t\<^sub>1 t\<^sub>2'" using less.prems(3) promised\<^sub>f promised_def by blast have q\<^sub>1_value: "v t\<^sub>1 = v t\<^sub>1'" "maxTerm (prevAccepted t\<^sub>1 q\<^sub>1) \ t\<^sub>1'" "proposed t\<^sub>1'" "t\<^sub>1' < t\<^sub>1" using n\<^sub>1 prevAccepted_def q\<^sub>1_value t\<^sub>2' by auto note `v t\<^sub>1 = v t\<^sub>1'` also have "v t\<^sub>1' = v t\<^sub>2" proof (intro hyp) have p: "maxTerm (prevAccepted t\<^sub>1 q\<^sub>1) \ prevAccepted t\<^sub>1 q\<^sub>1" apply (intro maxTerm_mem prevAccepted_finite) using n\<^sub>1 prevAccepted_def t\<^sub>2' by auto show "t\<^sub>1' < t\<^sub>1" "proposed t\<^sub>1'" using q\<^sub>1_value by simp_all have "t\<^sub>2 \ t\<^sub>2'" by (meson \accepted n t\<^sub>2\ less.prems(3) not_le promised\<^sub>b_max t\<^sub>2') also have "t\<^sub>2' \ maxTerm (prevAccepted t\<^sub>1 q\<^sub>1)" using n\<^sub>1 prevAccepted_def t\<^sub>2' prevAccepted_finite by (intro maxTerm_max, auto) also have "... \ t\<^sub>1'" using q\<^sub>1_value by simp finally show "t\<^sub>2 \ t\<^sub>1'" . qed finally show ?case . qed text \From this, it follows that any two committed values are equal as desired.\ lemma (in oneSlot) consistent: assumes "committed t\<^sub>1" and "committed t\<^sub>2" shows "v t\<^sub>1 = v t\<^sub>2" using assms by (metis Q_nonempty accepted all_not_in_conv committed not_less_iff_gr_or_eq p2b) text \It will be useful later to know the conditions under which a value in a term can be committed, which is spelled out here:\ lemma (in oneSlot) commit: assumes q_quorum: "q \ Q" assumes q_accepted: "\n. n \ q \ accepted n t\<^sub>0" defines "committed' t \ committed t \ t = t\<^sub>0" shows "oneSlot Q v promised\<^sub>f promised\<^sub>b proposed accepted committed'" by (smt committed'_def Q_intersects oneSlot_axioms oneSlot_def q_accepted q_quorum) end ================================================ FILE: cluster/isabelle/Preliminaries.thy ================================================ section \Preliminaries\ text \We start with some definitions of the types involved.\ theory Preliminaries imports Main begin subsection \Slots\ text \Slots are identified by natural numbers.\ type_synonym Slot = nat subsection \Terms\ text \Terms are identified by natural numbers.\ type_synonym Term = nat subsubsection \Maximum term of a set\ text \A function for finding the maximum term in a set is as follows.\ definition maxTerm :: "Term set \ Term" where "maxTerm S \ THE t. t \ S \ (\ t' \ S. t' \ t)" text \It works correctly on finite and nonempty sets as follows:\ theorem fixes S :: "Term set" assumes finite: "finite S" shows maxTerm_mem: "S \ {} \ maxTerm S \ S" and maxTerm_max: "\ t'. t' \ S \ t' \ maxTerm S" proof - presume "S \ {}" with assms obtain t where t: "t \ S" "\ t'. t' \ S \ t' \ t" proof (induct arbitrary: thesis) case empty then show ?case by simp next case (insert t S) show ?case proof (cases "S = {}") case True hence [simp]: "insert t S = {t}" by simp from insert.prems show ?thesis by simp next case False obtain t' where t': "t' \ S" "\ t'' \ S. t'' \ t'" by (meson False insert.hyps(3)) from t' show ?thesis proof (intro insert.prems ballI) fix t'' assume t'': "t'' \ insert t S" show "t'' \ (if t \ t' then t' else t)" proof (cases "t'' = t") case False with t'' have "t'' \ S" by simp with t' have "t'' \ t'" by simp thus ?thesis by auto qed simp qed simp qed qed from t have "maxTerm S = t" by (unfold maxTerm_def, intro the_equality, simp_all add: eq_iff) with t show "maxTerm S \ S" "\t'. t' \ S \ t' \ maxTerm S" by simp_all qed auto lemma assumes "\t. t \ S \ t \ t'" "finite S" "S \ {}" shows maxTerm_le: "maxTerm S \ t'" using assms maxTerm_mem by auto subsection \Configurations and quorums\ text \Nodes are simply identified by a natural number.\ datatype Node = Node nat definition natOfNode :: "Node \ nat" where "natOfNode node \ case node of Node n \ n" lemma natOfNode_Node[simp]: "natOfNode (Node n) = n" by (simp add: natOfNode_def) lemma Node_natOfNode[simp]: "Node (natOfNode n) = n" by (cases n, simp add: natOfNode_def) lemma natOfNode_inj[simp]: "(natOfNode n\<^sub>1 = natOfNode n\<^sub>2) = (n\<^sub>1 = n\<^sub>2)" by (metis Node_natOfNode) text \It is useful to be able to talk about whether sets-of-sets-of nodes mutually intersect or not.\ definition intersects :: "Node set set \ Node set set \ bool" (infixl "\" 50) where "A \ B \ \ a \ A. \ b \ B. a \ b \ {}" definition majorities :: "Node set \ Node set set" where "majorities votingNodes = { q. card votingNodes < card (q \ votingNodes) * 2 }" lemma majorities_nonempty: assumes "q \ majorities Q" shows "q \ {}" using assms by (auto simp add: majorities_def) lemma majorities_member: assumes "q \ majorities Q" obtains n where "n \ q" using majorities_nonempty assms by fastforce lemma majorities_intersect: assumes "finite votingNodes" shows "majorities votingNodes \ majorities votingNodes" unfolding intersects_def proof (intro ballI notI) fix q\<^sub>1 assume q\<^sub>1: "q\<^sub>1 \ majorities votingNodes" fix q\<^sub>2 assume q\<^sub>2: "q\<^sub>2 \ majorities votingNodes" assume disj: "q\<^sub>1 \ q\<^sub>2 = {}" have 1: "card ((q\<^sub>1 \ votingNodes) \ (q\<^sub>2 \ votingNodes)) = card (q\<^sub>1 \ votingNodes) + card (q\<^sub>2 \ votingNodes)" proof (intro card_Un_disjoint) from assms show "finite (q\<^sub>1 \ votingNodes)" by simp from assms show "finite (q\<^sub>2 \ votingNodes)" by simp from disj show "q\<^sub>1 \ votingNodes \ (q\<^sub>2 \ votingNodes) = {}" by auto qed have "card ((q\<^sub>1 \ votingNodes) \ (q\<^sub>2 \ votingNodes)) \ card votingNodes" by (simp add: assms card_mono) hence 2: "2 * card (q\<^sub>1 \ votingNodes) + 2 * card (q\<^sub>2 \ votingNodes) \ 2 * card votingNodes" by (simp add: 1) from q\<^sub>1 q\<^sub>2 have 3: "card votingNodes + card votingNodes < 2 * card (q\<^sub>1 \ votingNodes) + 2 * card (q\<^sub>2 \ votingNodes)" unfolding majorities_def by auto from 2 3 show False by simp qed text \A configuration of the system defines the sets of master-eligible nodes whose votes count when calculating quorums. The initial configuration of the system is fixed to some arbitrary value.\ consts Vs\<^sub>0 :: "Node list" definition V\<^sub>0 :: "Node set" where "V\<^sub>0 \ set Vs\<^sub>0" lemma finite_V\<^sub>0: "finite V\<^sub>0" unfolding V\<^sub>0_def by auto lemma V\<^sub>0_intersects: "majorities V\<^sub>0 \ majorities V\<^sub>0" using finite_V\<^sub>0 by (intro majorities_intersect) subsection \Values\ text \The model is a replicated state machine, with transitions that either do nothing, alter the configuration of the system or set a new \texttt{ClusterState}. \texttt{ClusterState} values are modelled simply as natural numbers.\ datatype ClusterState = ClusterState nat consts CS\<^sub>0 :: ClusterState datatype Value = NoOp | Reconfigure "Node list" (* update the set of voting nodes. A list rather than a set to force it to be finite *) | ClusterStateDiff "ClusterState \ ClusterState" (* a ClusterState diff *) text \Some useful definitions and lemmas follow.\ fun isReconfiguration :: "Value \ bool" where "isReconfiguration (Reconfigure _) = True" | "isReconfiguration _ = False" fun getConf :: "Value \ Node set" where "getConf (Reconfigure conf) = set conf" | "getConf _ = {}" lemma getConf_finite: "finite (getConf v)" by (metis List.finite_set getConf.elims infinite_imp_nonempty) lemma getConf_intersects: "majorities (getConf v) \ majorities (getConf v)" by (simp add: getConf_finite majorities_intersect) end ================================================ FILE: cluster/isabelle/ROOT ================================================ session "elasticsearch-isabelle" = "HOL" + options [document = pdf, document_output = "output", document_variants="document:outline=/proof"] theories [document = false] (* Foo *) (* Bar *) theories Preliminaries Implementation Monadic OneSlot Zen document_files "root.tex" ================================================ FILE: cluster/isabelle/Zen.thy ================================================ section \Safety Properties\ text \This section describes the invariants that hold in the system, shows that the implementation preserves the invariants, and shows that the invariants imply the required safety properties.\ theory Zen imports Implementation OneSlot begin subsection \Invariants on messages\ text \Firstly, a set of invariants that hold on the set of messages that have ever been sent, without considering the state of any individual node.\ fun nat_inductive_def :: "'a \ (nat \ 'a \ 'a) \ nat \ 'a" where "nat_inductive_def zeroCase sucCase 0 = zeroCase" | "nat_inductive_def zeroCase sucCase (Suc i) = sucCase i (nat_inductive_def zeroCase sucCase i)" locale zenMessages = fixes messages :: "RoutedMessage set" fixes isMessageFromTo :: "Node \ Message \ Destination \ bool" ("(_) \\ _ \\ (_)" [1000,55,1000]) defines "s \\ m \\ d \ \ sender = s, destination = d, payload = m \ \ messages" fixes isMessageFrom :: "Node \ Message \ bool" ("(_) \\ _ \\" [1000,55]) defines "s \\ m \\ \ \ d. s \\ m \\ d" fixes isMessageTo :: "Message \ Destination \ bool" ("\ _ \\ (_)" [55,1000]) defines "\ m \\ d \ \ s. s \\ m \\ d" fixes isMessage :: "Message \ bool" ("\ _ \\" [55]) defines "\ m \\ \ \ s. s \\ m \\" (* value proposed in a slot & a term *) fixes v :: "Slot \ Term \ Value" defines "v i t \ THE x. \ PublishRequest i t x \\" (* whether a slot is committed *) fixes isCommitted :: "Slot \ bool" defines "isCommitted i \ \ t. \ ApplyCommit i t \\" (* whether all preceding slots are committed *) fixes committedTo :: "Slot \ bool" ("committed\<^sub><") defines "committed\<^sub>< i \ \ j < i. isCommitted j" (* the committed value in a slot *) fixes v\<^sub>c :: "Slot \ Value" defines "v\<^sub>c i \ v i (SOME t. \ ApplyCommit i t \\)" (* the configuration of a slot *) fixes V :: "Slot \ Node set" defines "V \ nat_inductive_def V\<^sub>0 (\i Vi. if isReconfiguration (v\<^sub>c i) then getConf (v\<^sub>c i) else Vi)" (* predicate to say whether an applicable Vote has been sent *) fixes promised :: "Slot \ Node \ Node \ Term \ bool" defines "promised i s dn t \ \ i' \ i. \ a. s \\ Vote i' t a \\ (OneNode dn)" (* set of previously-accepted terms *) fixes prevAccepted :: "Slot \ Term \ Node set \ Term set" defines "prevAccepted i t senders \ {t'. \ s \ senders. s \\ Vote i t (SomeTerm t') \\ }" fixes lastCommittedClusterStateBefore :: "Slot \ ClusterState" defines "lastCommittedClusterStateBefore \ nat_inductive_def CS\<^sub>0 (\i CSi. case v\<^sub>c i of ClusterStateDiff diff \ diff CSi | _ \ CSi)" (* ASSUMPTIONS *) assumes Vote_future: "\i i' s t t' a. \ s \\ Vote i t a \\; i < i'; t' < t \ \ \ s \\ PublishResponse i' t' \\" assumes Vote_None: "\i s t t'. \ s \\ Vote i t NO_TERM \\; t' < t \ \ \ s \\ PublishResponse i t' \\" assumes Vote_Some_lt: "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t" assumes Vote_Some_PublishResponse: "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\" assumes Vote_Some_max: "\i s t t' t''. \ s \\ Vote i t (SomeTerm t') \\; t' < t''; t'' < t \ \ \ s \\ PublishResponse i t'' \\" assumes Vote_not_broadcast: "\i t a d. \ Vote i t a \\ d \ d \ Broadcast" assumes Vote_unique_destination: "\i s t a a' d d'. \ s \\ Vote i t a \\ d; s \\ Vote i' t a' \\ d' \ \ d = d'" assumes PublishRequest_committedTo: "\i t x. \ PublishRequest i t x \\ \ committedTo i" assumes PublishRequest_quorum: "\i s t x. s \\ PublishRequest i t x \\ \ \ q \ majorities (V i). (\ n \ q. promised i n s t) \ (prevAccepted i t q = {} \ (\ t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))" assumes PublishRequest_function: "\i t x x'. \ \ PublishRequest i t x \\; \ PublishRequest i t x' \\ \ \ x = x'" assumes finite_messages: "finite messages" assumes PublishResponse_PublishRequest: "\i s t. s \\ PublishResponse i t \\ \ \ x. \ PublishRequest i t x \\" assumes ApplyCommit_quorum: "\i t. \ ApplyCommit i t \\ \ \ q \ majorities (V i). \ s \ q. s \\ PublishResponse i t \\" assumes CatchUpResponse_committedTo: "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i" assumes CatchUpResponse_V: "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf" assumes CatchUpResponse_lastCommittedClusterStateBefore: "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs" definition (in zenMessages) votedFor :: "Node \ Node \ Term \ bool" where "votedFor n\<^sub>1 n\<^sub>2 t \ \ i. promised i n\<^sub>1 n\<^sub>2 t" lemma (in zenMessages) votedFor_unique: assumes "votedFor n n\<^sub>1 t" assumes "votedFor n n\<^sub>2 t" shows "n\<^sub>1 = n\<^sub>2" using assms unfolding votedFor_def by (meson Destination.inject Vote_unique_destination promised_def) lemma (in zenMessages) V_simps[simp]: "V 0 = V\<^sub>0" "V (Suc i) = (if isReconfiguration (v\<^sub>c i) then getConf (v\<^sub>c i) else V i)" unfolding V_def by simp_all lemma (in zenMessages) lastCommittedClusterStateBefore_simps[simp]: "lastCommittedClusterStateBefore 0 = CS\<^sub>0" "lastCommittedClusterStateBefore (Suc i) = (case v\<^sub>c i of ClusterStateDiff diff \ diff | _ \ id) (lastCommittedClusterStateBefore i)" unfolding lastCommittedClusterStateBefore_def by (simp, cases "v\<^sub>c i", auto) declare [[goals_limit = 40]] subsubsection \Utility lemmas\ text \Some results that are useful later:\ lemma (in zenMessages) V_finite: "finite (V i)" by (induct i, simp_all add: finite_V\<^sub>0 getConf_finite) lemma (in zenMessages) V_intersects: "majorities (V i) \ majorities (V i)" using V_finite majorities_intersect by simp lemma (in zenMessages) ApplyCommit_PublishResponse: assumes "\ ApplyCommit i t \\" obtains s where "s \\ PublishResponse i t \\" by (meson ApplyCommit_quorum majorities_member assms) lemma (in zenMessages) ApplyCommit_PublishRequest: assumes "\ ApplyCommit i t \\" shows "\ PublishRequest i t (v i t) \\" by (metis ApplyCommit_PublishResponse PublishResponse_PublishRequest assms the_equality v_def PublishRequest_function) lemma (in zenMessages) PublishRequest_Vote: assumes "s \\ PublishRequest i t x \\" obtains i' n a where "i' \ i" "n \\ Vote i' t a \\ (OneNode s)" by (meson PublishRequest_quorum majorities_member assms isMessage_def promised_def) lemma (in zenMessages) finite_prevAccepted: "finite (prevAccepted i t ns)" proof - fix t\<^sub>0 define f :: "RoutedMessage \ Term" where "f \ \ m. case payload m of Vote _ _ (SomeTerm t') \ t' | _ \ t\<^sub>0" have "prevAccepted i t ns \ f ` messages" apply (simp add: prevAccepted_def f_def isMessageFrom_def isMessageFromTo_def, intro subsetI) using image_iff by fastforce with finite_messages show ?thesis using finite_surj by auto qed lemma (in zenMessages) promised_long_def: "\d. promised i s d t \ (s \\ Vote i t NO_TERM \\ \ (\i'a. s \\ Vote i' t a \\)) \ (\t'. s \\ Vote i t (SomeTerm t') \\)" (is "?LHS == ?RHS") proof - have "?LHS = ?RHS" apply (intro iffI) apply (metis TermOption.exhaust isMessageFrom_def nat_less_le promised_def) by (metis Destination.exhaust Vote_not_broadcast isMessageFrom_def isMessageTo_def nat_less_le not_le promised_def) thus "?LHS == ?RHS" by simp qed lemma (in zenMessages) Vote_value_function: assumes "s \\ Vote i t a\<^sub>1 \\" and "s \\ Vote i t a\<^sub>2 \\" shows "a\<^sub>1 = a\<^sub>2" proof (cases a\<^sub>1) case NO_TERM with assms show ?thesis by (metis TermOption.exhaust Vote_None Vote_Some_PublishResponse Vote_Some_lt) next case (SomeTerm t\<^sub>1) with assms obtain t\<^sub>2 where a\<^sub>2: "a\<^sub>2 = SomeTerm t\<^sub>2" using Vote_None Vote_Some_PublishResponse Vote_Some_lt TermOption.exhaust by metis from SomeTerm a\<^sub>2 assms show ?thesis by (metis Vote_Some_PublishResponse Vote_Some_lt less_linear Vote_Some_max) qed lemma (in zenMessages) shows finite_messages_insert: "finite (insert m messages)" using finite_messages by auto lemma (in zenMessages) isCommitted_committedTo: assumes "isCommitted i" shows "committed\<^sub>< i" using ApplyCommit_PublishRequest PublishRequest_committedTo assms isCommitted_def by blast lemma (in zenMessages) isCommitted_committedTo_Suc: assumes "isCommitted i" shows "committed\<^sub>< (Suc i)" using assms committedTo_def isCommitted_committedTo less_antisym by blast lemma (in zenMessages) promised_unique: assumes "promised i s d t" and "promised i' s d' t" shows "d = d'" by (meson Destination.inject Vote_unique_destination assms promised_def) lemma (in zenMessages) PublishResponse_PublishRequest_v: assumes "\ PublishResponse i t \\" shows "\ PublishRequest i t (v i t) \\" proof - from assms obtain s where "s \\ PublishResponse i t \\" unfolding isMessage_def by blast with PublishResponse_PublishRequest obtain x where x: "\ PublishRequest i t x \\" by blast have "v i t = x" unfolding v_def using x by (intro the_equality PublishRequest_function) with x show ?thesis by simp qed lemma (in zenMessages) Vote_PublishRequest_v: assumes "\ Vote i t (SomeTerm t') \\" shows "\ PublishRequest i t' (v i t') \\" using assms Vote_Some_PublishResponse PublishResponse_PublishRequest_v unfolding isMessage_def by metis subsubsection \Relationship to @{term oneSlot}\ text \This shows that each slot @{term i} in Zen satisfies the assumptions of the @{term oneSlot} model above.\ lemma (in zenMessages) zen_is_oneSlot: fixes i shows "oneSlot (majorities (V i)) (v i) (\ s t. s \\ Vote i t NO_TERM \\ \ (\ i' < i. \ a. s \\ Vote i' t a \\)) (\ s t t'. s \\ Vote i t (SomeTerm t') \\) (\ t. \ x. \ PublishRequest i t x \\) (\ s t. s \\ PublishResponse i t \\) (\ t. \ ApplyCommit i t \\)" proof (unfold_locales, fold prevAccepted_def promised_long_def) from V_intersects show "majorities (V i) \ majorities (V i)". next fix s t t' assume "t' < t" "s \\ Vote i t NO_TERM \\ \ (\i'a. s \\ Vote i' t a \\)" thus "\ s \\ PublishResponse i t' \\" using Vote_None Vote_future by auto next fix s t t' assume j: "s \\ Vote i t (SomeTerm t') \\" from j show "t' < t" using Vote_Some_lt by blast from j show "s \\ PublishResponse i t' \\" using Vote_Some_PublishResponse by blast fix t'' assume "t' < t''" "t'' < t" with j show "\ s \\ PublishResponse i t'' \\" using Vote_Some_max by blast next fix t assume "\x. \ PublishRequest i t x \\" then obtain x s where "s \\ PublishRequest i t x \\" by (auto simp add: isMessage_def) from PublishRequest_quorum [OF this] PublishResponse_PublishRequest show "\q \ majorities (V i). (\n\q. \d. promised i n d t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ (\x. \ PublishRequest i t' x \\) \ t' < t))" unfolding isMessageFrom_def by (meson PublishResponse_PublishRequest_v) next fix s t assume "s \\ PublishResponse i t \\" thus "\x. \ PublishRequest i t x \\" by (simp add: PublishResponse_PublishRequest) next fix t assume "\ ApplyCommit i t \\" thus "\q \ majorities (V i). \s\q. s \\ PublishResponse i t \\" by (simp add: ApplyCommit_quorum) next fix t\<^sub>0 define f :: "RoutedMessage \ Term" where "f \ \ m. case payload m of PublishRequest i t x \ t | _ \ t\<^sub>0" have "{t. \x. \ PublishRequest i t x \\} \ f ` messages" apply (unfold isMessage_def isMessageFrom_def isMessageFromTo_def) using f_def image_iff by fastforce moreover have "finite (f ` messages)" by (simp add: finite_messages) ultimately show "finite {t. \x. \ PublishRequest i t x \\}" using finite_subset by blast qed text \From this it follows that all committed values are equal.\ theorem (in zenMessages) consistent: assumes "\ ApplyCommit i t\<^sub>1 \\" assumes "\ ApplyCommit i t\<^sub>2 \\" assumes "\ PublishRequest i t\<^sub>1 v\<^sub>1 \\" assumes "\ PublishRequest i t\<^sub>2 v\<^sub>2 \\" shows "v\<^sub>1 = v\<^sub>2" proof - from oneSlot.consistent [OF zen_is_oneSlot] assms have "v i t\<^sub>1 = v i t\<^sub>2" by blast moreover have "v\<^sub>1 = v i t\<^sub>1" using ApplyCommit_PublishRequest assms PublishRequest_function by blast moreover have "v\<^sub>2 = v i t\<^sub>2" using ApplyCommit_PublishRequest assms PublishRequest_function by blast ultimately show ?thesis by simp qed lemma (in zenMessages) ApplyCommit_v\<^sub>c: assumes "\ ApplyCommit i t \\" shows "v\<^sub>c i = v i t" proof (unfold v\<^sub>c_def, intro someI2 [where Q = "\t'. v i t' = v i t"]) from assms show "\ ApplyCommit i t \\" . fix t' assume "\ ApplyCommit i t' \\" thus "v i t' = v i t" by (intro oneSlot.consistent [OF zen_is_oneSlot] assms) qed lemma (in zenMessages) ApplyCommit_PublishRequest_v\<^sub>c: assumes "\ ApplyCommit i t \\" shows "\ PublishRequest i t (v\<^sub>c i) \\" unfolding ApplyCommit_v\<^sub>c [OF assms] using ApplyCommit_PublishRequest assms . subsection \Invariants on node states\ text \A set of invariants which relate the states of the individual nodes to the set of messages sent.\ locale zen = zenMessages + fixes nodeState :: "Node \ NodeData" assumes currentNode_nodeState: "\n. currentNode (nodeState n) = n" assumes committedTo_firstUncommittedSlot: "\n. committed\<^sub>< (firstUncommittedSlot (nodeState n))" assumes currentVotingNodes_firstUncommittedSlot: "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))" assumes firstUncommittedSlot_PublishRequest: "\i n t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\" assumes firstUncommittedSlot_PublishResponse: "\i n t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\" assumes lastAcceptedTerm_None: "\n t. lastAcceptedTerm (nodeState n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\" assumes lastAcceptedTerm_Some_sent: "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\" assumes lastAcceptedTerm_Some_max: "\n t t'. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\ \ t' \ t" assumes lastAcceptedTerm_Some_currentTerm: "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ t \ currentTerm (nodeState n)" assumes lastAcceptedTerm_Some_value: "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState n)) \\" assumes Vote_currentTerm: "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)" assumes Vote_slot_function: "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'" assumes PublishRequest_currentTerm: "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ t \ currentTerm (nodeState n)" assumes PublishRequest_publishPermitted_currentTerm: "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ publishPermitted (nodeState n) \ t < currentTerm (nodeState n)" assumes joinVotes: "\ n n'. n' \ joinVotes (nodeState n) \ promised (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState n))" assumes electionWon_isQuorum: "\n. electionWon (nodeState n) \ isQuorum (nodeState n) (joinVotes (nodeState n))" assumes joinVotes_max: "\n n' a'. \ \ (\ x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) x \\); n' \ joinVotes (nodeState n); n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ \ a' \ lastAcceptedTerm (nodeState n)" assumes publishVotes: "\n n'. n' \ publishVotes (nodeState n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) \\" assumes currentClusterState_lastCommittedClusterStateBefore: "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))" lemma (in zen) joinVotes_votedFor: assumes "n' \ joinVotes (nodeState n)" shows "votedFor n' n (currentTerm (nodeState n))" using joinVotes assms unfolding votedFor_def by auto locale zenStep = zen + fixes messages' :: "RoutedMessage set" fixes nodeState' :: "Node \ NodeData" fixes n\<^sub>0 :: Node assumes messages_subset: "messages \ messages'" fixes nd :: NodeData defines "nd \ nodeState n\<^sub>0" fixes nd' :: NodeData defines "nd' \ nodeState' n\<^sub>0" assumes nodeState_unchanged: "\n. n \ n\<^sub>0 \ nodeState' n = nodeState n" (* updated definitions from zenMessages *) fixes isMessageFromTo' :: "Node \ Message \ Destination \ bool" ("(_) \\ _ \\' (_)" [1000,55,1000]) defines "s \\ m \\' d \ \ sender = s, destination = d, payload = m \ \ messages'" fixes isMessageFrom' :: "Node \ Message \ bool" ("(_) \\ _ \\'" [1000,55]) defines "s \\ m \\' \ \ d. s \\ m \\' d" fixes isMessageTo' :: "Message \ Destination \ bool" ("\ _ \\' (_)" [55,1000]) defines "\ m \\' d \ \ s. s \\ m \\' d" fixes isMessage' :: "Message \ bool" ("\ _ \\'" [55]) defines "\ m \\' \ \ s. s \\ m \\'" (* value proposed in a slot & a term *) fixes v' :: "nat \ Term \ Value" defines "v' i t \ THE x. \ PublishRequest i t x \\'" (* whether a slot is committed *) fixes isCommitted' :: "nat \ bool" defines "isCommitted' i \ \ t. \ ApplyCommit i t \\'" (* whether all preceding slots are committed *) fixes committedTo' :: "nat \ bool" ("committed\<^sub><'") defines "committed\<^sub><' i \ \ j < i. isCommitted' j" (* the committed value in a slot *) fixes v\<^sub>c' :: "nat \ Value" defines "v\<^sub>c' i \ v' i (SOME t. \ ApplyCommit i t \\')" (* the configuration of a slot *) fixes V' :: "Slot \ Node set" defines "V' \ nat_inductive_def V\<^sub>0 (\i Vi. if isReconfiguration (v\<^sub>c' i) then getConf (v\<^sub>c' i) else Vi)" (* predicate to say whether an applicable Vote has been sent *) fixes promised' :: "nat \ Node \ Node \ Term \ bool" defines "promised' i s dn t \ \ i' \ i. \ a. s \\ Vote i' t a \\' (OneNode dn)" (* set of previously-accepted terms *) fixes prevAccepted' :: "nat \ Term \ Node set \ Term set" defines "prevAccepted' i t senders \ {t'. \ s \ senders. s \\ Vote i t (SomeTerm t') \\' }" fixes lastCommittedClusterStateBefore' :: "nat \ ClusterState" defines "lastCommittedClusterStateBefore' \ nat_inductive_def CS\<^sub>0 (\i CSi. case v\<^sub>c' i of ClusterStateDiff diff \ diff CSi | _ \ CSi)" fixes sendTo :: "Destination \ (NodeData * Message option) \ RoutedMessage set" defines "sendTo d result \ case snd result of None \ messages | Some m \ insert \ sender = n\<^sub>0, destination = d, payload = m \ messages" lemma (in zenStep) nodeState'_def: "nodeState' n \ if n = n\<^sub>0 then nd' else nodeState n" using nodeState_unchanged nd'_def by presburger lemma (in zenStep) V'_simps[simp]: "V' 0 = V\<^sub>0" "V' (Suc i) = (if isReconfiguration (v\<^sub>c' i) then getConf (v\<^sub>c' i) else V' i)" unfolding V'_def by simp_all lemma (in zenStep) lastCommittedClusterStateBefore'_simps[simp]: "lastCommittedClusterStateBefore' 0 = CS\<^sub>0" "lastCommittedClusterStateBefore' (Suc i) = (case v\<^sub>c' i of ClusterStateDiff diff \ diff | _ \ id) (lastCommittedClusterStateBefore' i)" unfolding lastCommittedClusterStateBefore'_def by (simp, cases "v\<^sub>c' i", auto) lemma (in zenStep) sendTo_simps[simp]: "sendTo d (nd'', None) = messages" "sendTo d (nd'', Some m) = insert \ sender = n\<^sub>0, destination = d, payload = m \ messages" by (auto simp add: sendTo_def) lemma currentTerm_ensureCurrentTerm[simp]: "currentTerm nd \ t \ currentTerm (ensureCurrentTerm t nd) = t" by (auto simp add: ensureCurrentTerm_def) lemma (in zenStep) assumes "\i i' s t t' a. s \\ Vote i t a \\' \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\'" assumes "\i s t t'. s \\ Vote i t NO_TERM \\' \ t' < t \ \ s \\ PublishResponse i t' \\'" assumes "\i s t t'. s \\ Vote i t (SomeTerm t') \\' \ t' < t" assumes "\i s t t'. s \\ Vote i t (SomeTerm t') \\' \ s \\ PublishResponse i t' \\'" assumes "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\' \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\'" assumes "\i t a d. \ Vote i t a \\' d \ d \ Broadcast" assumes "\i' i s t a a' d d'. s \\ Vote i t a \\' d \ s \\ Vote i' t a' \\' d' \ d = d'" assumes "\i t x. \ PublishRequest i t x \\' \ committed\<^sub><' i" assumes "\i s t x. s \\ PublishRequest i t x \\' \ \q\majorities (V' i). (\n\q. promised' i n s t) \ (prevAccepted' i t q = {} \ (\t'. v' i t = v' i t' \ maxTerm (prevAccepted' i t q) \ t' \ \ PublishResponse i t' \\' \ t' < t))" assumes "\i t x x'. \ PublishRequest i t x \\' \ \ PublishRequest i t x' \\' \ x = x'" assumes "finite messages'" assumes "\i s t. s \\ PublishResponse i t \\' \ \x. \ PublishRequest i t x \\'" assumes "\i t. \ ApplyCommit i t \\' \ \q\majorities (V' i). \s\q. s \\ PublishResponse i t \\'" assumes "\n. currentNode (nodeState' n) = n" assumes "\n. committed\<^sub><' (firstUncommittedSlot (nodeState' n))" assumes "\n. currentVotingNodes (nodeState' n) = V' (firstUncommittedSlot (nodeState' n))" assumes "\i n t x. firstUncommittedSlot (nodeState' n) < i \ \ n \\ PublishRequest i t x \\'" assumes "\i n t. firstUncommittedSlot (nodeState' n) < i \ \ n \\ PublishResponse i t \\'" assumes "\n t. lastAcceptedTerm (nodeState' n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState' n)) t \\'" assumes "\n t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState' n)) t \\'" assumes "\n t t'. lastAcceptedTerm (nodeState' n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState' n)) t' \\' \ t' \ t" assumes "\n t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState' n)) t (lastAcceptedValue (nodeState' n)) \\'" assumes "\n t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ t \ currentTerm (nodeState' n)" assumes "\n i t a. n \\ Vote i t a \\' \ t \ currentTerm (nodeState' n)" assumes "\n i i' t a a'. n \\ Vote i t a \\' \ n \\ Vote i' t a' \\' \ i = i'" assumes "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState' n)) t x \\' \ t \ currentTerm (nodeState' n)" assumes "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState' n)) t x \\' \ publishPermitted (nodeState' n) \ t < currentTerm (nodeState' n)" assumes "\n n'. n' \ joinVotes (nodeState' n) \ promised' (firstUncommittedSlot (nodeState' n)) n' n (currentTerm (nodeState' n))" assumes "\n. electionWon (nodeState' n) \ isQuorum (nodeState' n) (joinVotes (nodeState' n))" assumes "\n n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState' n)) (currentTerm (nodeState' n)) x \\') \ n' \ joinVotes (nodeState' n) \ n' \\ Vote (firstUncommittedSlot (nodeState' n)) (currentTerm (nodeState' n)) a' \\' (OneNode n) \ a' \ lastAcceptedTerm (nodeState' n)" assumes "\n n'. n' \ publishVotes (nodeState' n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState' n)) (currentTerm (nodeState' n)) \\'" assumes "\n. currentClusterState (nodeState' n) = lastCommittedClusterStateBefore' (firstUncommittedSlot (nodeState' n))" assumes "\i conf cs. \ CatchUpResponse i conf cs \\' \ committed\<^sub><' i" assumes "\i conf cs. \ CatchUpResponse i conf cs \\' \ V' i = conf" assumes "\i conf cs. \ CatchUpResponse i conf cs \\' \ lastCommittedClusterStateBefore' i = cs" shows zenI: "zen messages' nodeState'" apply (unfold_locales) apply (fold isMessageFromTo'_def) apply (fold isMessageTo'_def) apply (fold isMessageFrom'_def) apply (fold isMessage'_def) apply (fold v'_def) apply (fold isCommitted'_def) apply (fold committedTo'_def) apply (fold v\<^sub>c'_def) apply (fold V'_def) apply (fold promised'_def) apply (fold prevAccepted'_def) apply (fold lastCommittedClusterStateBefore'_def) using assms proof - qed lemma (in zenStep) assumes "zen messages' nodeState'" "messages' \ messages''" "\n. n \ n\<^sub>0 \ nodeState' n = nodeState'' n" shows zenStepI1: "zenStep messages' nodeState' messages'' nodeState'' n\<^sub>0" proof (intro_locales) from `zen messages' nodeState'` show "zenMessages messages'" "zen_axioms messages' nodeState'" unfolding zen_def by simp_all from assms show "zenStep_axioms messages' nodeState' messages'' nodeState'' n\<^sub>0" by (intro zenStep_axioms.intro, auto) qed lemma (in zenStep) assumes "messages \ messages''" "\n. n \ n\<^sub>0 \ nodeState n = nodeState'' n" shows zenStepI2: "zenStep messages nodeState messages'' nodeState'' n\<^sub>0" proof (intro_locales) from assms show "zenStep_axioms messages nodeState messages'' nodeState'' n\<^sub>0" by (intro zenStep_axioms.intro, auto) qed lemma (in zenStep) Broadcast_to_OneNode: fixes x assumes nodeState': "nodeState' = nodeState" assumes sent: "n\<^sub>0 \\ m \\ Broadcast" assumes messages': "messages' = sendTo (OneNode d) (nd'', Some m)" shows "zen messages' nodeState'" proof - have messages': "messages' = insert \sender = n\<^sub>0, destination = OneNode d, payload = m \ messages" by (simp add: messages') from Vote_not_broadcast sent have not_Vote: "\i t a. m \ Vote i t a" unfolding isMessageTo_def by blast from sent not_Vote have message_simps: "\s m'. (s \\ m' \\') = (s \\ m' \\)" "\m'. (\ m' \\') = (\ m' \\)" "\s d i t a. (s \\ Vote i t a \\' d) = (s \\ Vote i t a \\ d)" "\d i t a. (\ Vote i t a \\' d) = (\ Vote i t a \\ d)" by (auto simp add: isMessageFromTo'_def isMessageTo'_def isMessage'_def isMessageFrom'_def, auto simp add: isMessageFromTo_def isMessageTo_def isMessage_def isMessageFrom_def messages') have property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. firstUncommittedSlot (nodeState' n) = firstUncommittedSlot (nodeState n)" "\n. currentVotingNodes (nodeState' n) = currentVotingNodes (nodeState n)" "\n q. isQuorum (nodeState' n) q = isQuorum (nodeState n) q" "\n. currentTerm (nodeState' n) = currentTerm (nodeState n)" "\n. publishPermitted (nodeState' n) = publishPermitted (nodeState n)" "\n. joinVotes (nodeState' n) = joinVotes (nodeState n)" "\n. electionWon (nodeState' n) = electionWon (nodeState n)" "\n. publishVotes (nodeState' n) = publishVotes (nodeState n)" "\n. currentClusterState (nodeState' n) = currentClusterState (nodeState n)" "\n. lastAcceptedTerm (nodeState' n) = lastAcceptedTerm (nodeState n)" "\n. lastAcceptedValue (nodeState' n) = lastAcceptedValue (nodeState n)" by (unfold nodeState', auto) have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def message_simps) have v\<^sub>c_eq[simp]: "v\<^sub>c' = v\<^sub>c" by (intro ext, auto simp add: v\<^sub>c'_def v\<^sub>c_def message_simps) have isCommitted_eq[simp]: "isCommitted' = isCommitted" by (intro ext, auto simp add: isCommitted'_def isCommitted_def message_simps) have committedTo_eq[simp]: "committed\<^sub><' = committed\<^sub><" by (intro ext, auto simp add: committedTo'_def committedTo_def) have V_eq[simp]: "V' = V" using v\<^sub>c_eq V'_def V_def by blast have lastCommittedClusterStateBefore_eq[simp]: "lastCommittedClusterStateBefore' = lastCommittedClusterStateBefore" unfolding lastCommittedClusterStateBefore_def lastCommittedClusterStateBefore'_def v\<^sub>c_eq .. have promised_eq[simp]: "promised' = promised" by (intro ext, auto simp add: promised'_def promised_def message_simps) have prevAccepted_eq[simp]: "prevAccepted' = prevAccepted" by (intro ext, auto simp add: prevAccepted'_def prevAccepted_def message_simps) show ?thesis apply (intro zenI) apply (unfold message_simps committedTo_eq V_eq v_eq lastCommittedClusterStateBefore_eq property_simps promised_eq prevAccepted_eq) proof - from finite_messages show "finite messages'" by (simp add: messages') from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from committedTo_firstUncommittedSlot show "\n. committed\<^sub>< (firstUncommittedSlot (nodeState n))" . from firstUncommittedSlot_PublishResponse show "\n i t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\" . from lastAcceptedTerm_None show "\n i t. lastAcceptedTerm (nodeState n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_sent show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_max show "\n t t'. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\ \ t' \ t". from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from electionWon_isQuorum show "\n. electionWon (nodeState n) \ isQuorum (nodeState n) (joinVotes (nodeState n))". from joinVotes_max show " \n n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) x \\) \ n' \ joinVotes (nodeState n) \ n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState n)". from publishVotes show "\n n'. n' \ publishVotes (nodeState n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) \\". from currentClusterState_lastCommittedClusterStateBefore show "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))". from joinVotes show "\n n'. n' \ joinVotes (nodeState n) \ promised (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState n))". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\". from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))". from firstUncommittedSlot_PublishRequest show "\i n t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\". from lastAcceptedTerm_Some_value show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState n)) \\". from PublishRequest_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ t \ currentTerm (nodeState n)". from PublishRequest_publishPermitted_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ publishPermitted (nodeState n) \ t < currentTerm (nodeState n)". from CatchUpResponse_committedTo show "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i". from CatchUpResponse_V show "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf". from CatchUpResponse_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs". from lastAcceptedTerm_Some_currentTerm show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ t \ currentTerm (nodeState n)". qed qed lemma (in zenStep) send_spontaneous_message: assumes messages': "messages' = sendTo d\<^sub>0 (nd, Some m)" assumes spontaneous: "case m of StartJoin _ \ True | ClientValue _ \ True | Reboot \ True | CatchUpRequest \ True | _ \ False" assumes nodeState': "nodeState' = nodeState" shows "zen messages' nodeState'" proof - have messages': "messages' = insert \sender = n\<^sub>0, destination = d\<^sub>0, payload = m \ messages" by (simp add: messages') from spontaneous have message_simps[simp]: "\s d i t a. (s \\ Vote i t a \\' d) = (s \\ Vote i t a \\ d)" "\s d i t. (s \\ PublishResponse i t \\' d) = (s \\ PublishResponse i t \\ d)" "\s d i t x. (s \\ PublishRequest i t x \\' d) = (s \\ PublishRequest i t x \\ d)" "\s d i t. (s \\ ApplyCommit i t \\' d) = (s \\ ApplyCommit i t \\ d)" "\d i t a. (\ Vote i t a \\' d) = (\ Vote i t a \\ d)" "\d i t. (\ PublishResponse i t \\' d) = (\ PublishResponse i t \\ d)" "\d i t x. (\ PublishRequest i t x \\' d) = (\ PublishRequest i t x \\ d)" "\d i t. (\ ApplyCommit i t \\' d) = (\ ApplyCommit i t \\ d)" "\s i t a. (s \\ Vote i t a \\') = (s \\ Vote i t a \\)" "\s i t. (s \\ PublishResponse i t \\') = (s \\ PublishResponse i t \\)" "\s i t x. (s \\ PublishRequest i t x \\') = (s \\ PublishRequest i t x \\)" "\s i t. (s \\ ApplyCommit i t \\') = (s \\ ApplyCommit i t \\)" "\d i t a. (\ Vote i t a \\') = (\ Vote i t a \\)" "\d i t. (\ PublishResponse i t \\') = (\ PublishResponse i t \\)" "\d i t x. (\ PublishRequest i t x \\') = (\ PublishRequest i t x \\)" "\d i t. (\ ApplyCommit i t \\') = (\ ApplyCommit i t \\)" "\s d i conf cs. (s \\ CatchUpResponse i conf cs \\' d) = (s \\ CatchUpResponse i conf cs \\ d)" "\d i conf cs. (\ CatchUpResponse i conf cs \\' d) = (\ CatchUpResponse i conf cs \\ d)" "\s i conf cs. (s \\ CatchUpResponse i conf cs \\') = (s \\ CatchUpResponse i conf cs \\)" "\d i conf cs. (\ CatchUpResponse i conf cs \\') = (\ CatchUpResponse i conf cs \\)" by (auto simp add: isMessageFromTo'_def isMessageTo'_def isMessage'_def isMessageFrom'_def, auto simp add: isMessageFromTo_def isMessageTo_def isMessage_def isMessageFrom_def messages') have property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. firstUncommittedSlot (nodeState' n) = firstUncommittedSlot (nodeState n)" "\n. currentVotingNodes (nodeState' n) = currentVotingNodes (nodeState n)" "\n q. isQuorum (nodeState' n) q = isQuorum (nodeState n) q" "\n. currentTerm (nodeState' n) = currentTerm (nodeState n)" "\n. publishPermitted (nodeState' n) = publishPermitted (nodeState n)" "\n. joinVotes (nodeState' n) = joinVotes (nodeState n)" "\n. electionWon (nodeState' n) = electionWon (nodeState n)" "\n. publishVotes (nodeState' n) = publishVotes (nodeState n)" "\n. currentClusterState (nodeState' n) = currentClusterState (nodeState n)" "\n. lastAcceptedTerm (nodeState' n) = lastAcceptedTerm (nodeState n)" "\n. lastAcceptedValue (nodeState' n) = lastAcceptedValue (nodeState n)" by (unfold nodeState', auto) have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def) have v\<^sub>c_eq[simp]: "v\<^sub>c' = v\<^sub>c" by (intro ext, auto simp add: v\<^sub>c'_def v\<^sub>c_def) have isCommitted_eq[simp]: "isCommitted' = isCommitted" by (intro ext, auto simp add: isCommitted'_def isCommitted_def) have committedTo_eq[simp]: "committed\<^sub><' = committed\<^sub><" by (intro ext, auto simp add: committedTo'_def committedTo_def) have V_eq[simp]: "V' = V" using v\<^sub>c_eq V'_def V_def by blast have lastCommittedClusterStateBefore_eq[simp]: "lastCommittedClusterStateBefore' = lastCommittedClusterStateBefore" unfolding lastCommittedClusterStateBefore_def lastCommittedClusterStateBefore'_def v\<^sub>c_eq .. have promised_eq[simp]: "promised' = promised" by (intro ext, auto simp add: promised'_def promised_def) have prevAccepted_eq[simp]: "prevAccepted' = prevAccepted" by (intro ext, auto simp add: prevAccepted'_def prevAccepted_def) show ?thesis apply (intro zenI) apply (unfold message_simps committedTo_eq V_eq v_eq lastCommittedClusterStateBefore_eq property_simps promised_eq prevAccepted_eq) proof - from finite_messages show "finite messages'" by (simp add: messages') from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from committedTo_firstUncommittedSlot show "\n. committed\<^sub>< (firstUncommittedSlot (nodeState n))" . from firstUncommittedSlot_PublishResponse show "\n i t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\" . from lastAcceptedTerm_None show "\n i t. lastAcceptedTerm (nodeState n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_sent show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_max show "\n t t'. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\ \ t' \ t". from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from electionWon_isQuorum show "\n. electionWon (nodeState n) \ isQuorum (nodeState n) (joinVotes (nodeState n))". from joinVotes_max show " \n n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) x \\) \ n' \ joinVotes (nodeState n) \ n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState n)". from publishVotes show "\n n'. n' \ publishVotes (nodeState n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) \\". from currentClusterState_lastCommittedClusterStateBefore show "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))". from joinVotes show "\n n'. n' \ joinVotes (nodeState n) \ promised (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState n))". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\". from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))". from firstUncommittedSlot_PublishRequest show "\i n t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\". from lastAcceptedTerm_Some_value show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState n)) \\". from PublishRequest_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ t \ currentTerm (nodeState n)". from PublishRequest_publishPermitted_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ publishPermitted (nodeState n) \ t < currentTerm (nodeState n)". from CatchUpResponse_committedTo show "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i". from CatchUpResponse_V show "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf". from CatchUpResponse_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs". from lastAcceptedTerm_Some_currentTerm show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ t \ currentTerm (nodeState n)". qed qed lemma (in zenStep) send_StartJoin: assumes messages': "messages' = sendTo d\<^sub>0 (nd, Some (StartJoin t\<^sub>0))" assumes nodeState': "nodeState' = nodeState" shows "zen messages' nodeState'" using assms by (intro send_spontaneous_message, auto) lemma (in zenStep) send_ClientValue: assumes messages': "messages' = sendTo d\<^sub>0 (nd, Some (ClientValue x\<^sub>0))" assumes nodeState': "nodeState' = nodeState" shows "zen messages' nodeState'" using assms by (intro send_spontaneous_message, auto) lemma (in zenStep) send_CatchUpRequest: assumes messages': "messages' = sendTo d\<^sub>0 (nd, Some CatchUpRequest)" assumes nodeState': "nodeState' = nodeState" shows "zen messages' nodeState'" using assms by (intro send_spontaneous_message, auto) lemma (in zenStep) send_Reboot: assumes messages': "messages' = sendTo d\<^sub>0 (nd, Some Reboot)" assumes nodeState': "nodeState' = nodeState" shows "zen messages' nodeState'" using assms by (intro send_spontaneous_message, auto) lemma (in zenStep) ensureCurrentTerm_invariants: assumes nd': "nd' = ensureCurrentTerm t nd" assumes messages': "messages' = messages" shows "zen messages' nodeState'" proof (cases "t \ currentTerm nd") case True hence "nodeState' = nodeState" by (intro ext, unfold nodeState'_def, auto simp add: nd' ensureCurrentTerm_def nd_def) with zen_axioms messages' show ?thesis by simp next case False hence t: "currentTerm nd < t" by simp have message_simps[simp]: "\s p d. (s \\ p \\' d) = (s \\ p \\ d)" "\p d. (\ p \\' d) = (\ p \\ d)" "\s p. (s \\ p \\') = (s \\ p \\)" "\p. (\ p \\') = (\ p \\)" by (unfold isMessageFromTo'_def isMessageTo'_def isMessageFrom'_def isMessage'_def, auto simp add: messages' isMessageFromTo_def isMessageTo_def isMessageFrom_def isMessage_def) have property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. firstUncommittedSlot (nodeState' n) = firstUncommittedSlot (nodeState n)" "\n. currentClusterState (nodeState' n) = currentClusterState (nodeState n)" "\n. currentVotingNodes (nodeState' n) = currentVotingNodes (nodeState n)" "\n q. isQuorum (nodeState' n) q = isQuorum (nodeState n) q" "\n. lastAcceptedTerm (nodeState' n) = lastAcceptedTerm (nodeState n)" "\n. lastAcceptedValue (nodeState' n) = lastAcceptedValue (nodeState n)" "currentTerm (nodeState' n\<^sub>0) = t" "electionWon (nodeState' n\<^sub>0) = False" "joinVotes (nodeState' n\<^sub>0) = {}" "publishPermitted (nodeState' n\<^sub>0) = True" "publishVotes (nodeState' n\<^sub>0) = {}" using t by (unfold nodeState'_def, auto simp add: nd_def isQuorum_def nd' ensureCurrentTerm_def lastAcceptedTerm_def lastAcceptedValue_def) have currentTerm_increases: "\n. currentTerm (nodeState n) \ currentTerm (nodeState' n)" using nd_def nodeState'_def property_simps t by auto have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def) have v\<^sub>c_eq[simp]: "v\<^sub>c' = v\<^sub>c" by (intro ext, auto simp add: v\<^sub>c'_def v\<^sub>c_def) have isCommitted_eq[simp]: "isCommitted' = isCommitted" by (intro ext, auto simp add: isCommitted'_def isCommitted_def) have committedTo_eq[simp]: "committed\<^sub><' = committed\<^sub><" by (intro ext, auto simp add: committedTo'_def committedTo_def) have V_eq[simp]: "V' = V" using v\<^sub>c_eq V'_def V_def by blast have lastCommittedClusterStateBefore_eq[simp]: "lastCommittedClusterStateBefore' = lastCommittedClusterStateBefore" unfolding lastCommittedClusterStateBefore_def lastCommittedClusterStateBefore'_def v\<^sub>c_eq .. have promised_eq: "\i n n' t. promised' i n n' t = promised i n n' t" by (simp add: promised_def promised'_def) have prevAccepted_eq: "\i t q. prevAccepted' i t q = prevAccepted i t q" by (simp add: prevAccepted_def prevAccepted'_def) show ?thesis apply (intro zenI) apply (unfold message_simps committedTo_eq V_eq v_eq lastCommittedClusterStateBefore_eq property_simps promised_eq prevAccepted_eq) proof - from finite_messages show "finite messages'" by (simp add: messages') from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from committedTo_firstUncommittedSlot show "\n. committed\<^sub>< (firstUncommittedSlot (nodeState n))" . from firstUncommittedSlot_PublishResponse show "\n i t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\" . from lastAcceptedTerm_None show "\n i t. lastAcceptedTerm (nodeState n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_sent show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_max show "\n t t'. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\ \ t' \ t". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from currentClusterState_lastCommittedClusterStateBefore show "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\". from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))". from firstUncommittedSlot_PublishRequest show "\i n t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\". from lastAcceptedTerm_Some_value show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState n)) \\". from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))". from CatchUpResponse_committedTo show "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i". from CatchUpResponse_V show "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf". from CatchUpResponse_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs". from Vote_currentTerm currentTerm_increases show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState' n)" using dual_order.trans by blast from PublishRequest_publishPermitted_currentTerm currentTerm_increases property_simps show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ publishPermitted (nodeState' n) \ t < currentTerm (nodeState' n)" by (metis False PublishRequest_currentTerm dual_order.trans leI nd_def nodeState_unchanged) from joinVotes show "\n n'. n' \ joinVotes (nodeState' n) \ promised (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState' n))" unfolding nodeState'_def by (auto simp add: nd'_def) from PublishRequest_currentTerm currentTerm_increases show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ t \ currentTerm (nodeState' n)" using dual_order.trans by blast from publishVotes show "\n n'. n' \ publishVotes (nodeState' n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState' n)) \\" unfolding nodeState'_def by (auto simp add: nd'_def) from electionWon_isQuorum show "\n. electionWon (nodeState' n) \ isQuorum (nodeState n) (joinVotes (nodeState' n))" unfolding nodeState'_def by (auto simp add: nd'_def) from lastAcceptedTerm_Some_currentTerm currentTerm_increases show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ t \ currentTerm (nodeState' n)" using dual_order.trans by blast show "\n n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState' n)) x \\) \ n' \ joinVotes (nodeState' n) \ n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState' n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState n)" by (metis ex_in_conv joinVotes_max nodeState_unchanged property_simps(10)) qed qed lemma (in zenStep) sendVote_invariants: assumes messages': "messages' = sendTo (OneNode d\<^sub>0) (nd'', Some (Vote (firstUncommittedSlot nd) (currentTerm nd) (lastAcceptedTerm nd)))" assumes nd': "nd' = nd" assumes not_sent: "\i a. \ n\<^sub>0 \\ Vote i (currentTerm nd) a \\" assumes not_accepted: "\t'. lastAcceptedTerm nd = SomeTerm t' \ t' < currentTerm nd" shows "zen messages' nodeState'" proof - have messages': "messages' = insert \sender = n\<^sub>0, destination = OneNode d\<^sub>0, payload = Vote (firstUncommittedSlot nd) (currentTerm nd) (lastAcceptedTerm nd)\ messages" by (simp add: messages') have nodeState'[simp]: "nodeState' = nodeState" by (intro ext, auto simp add: nodeState'_def nd' nd_def) have message_simps[simp]: "\s d i t. (s \\ ApplyCommit i t \\' d) = (s \\ ApplyCommit i t \\ d)" "\s d i t x. (s \\ PublishRequest i t x \\' d) = (s \\ PublishRequest i t x \\ d)" "\s d i t. (s \\ PublishResponse i t \\' d) = (s \\ PublishResponse i t \\ d)" "\d i t. (\ ApplyCommit i t \\' d) = (\ ApplyCommit i t \\ d)" "\d i t x. (\ PublishRequest i t x \\' d) = (\ PublishRequest i t x \\ d)" "\d i t. (\ PublishResponse i t \\' d) = (\ PublishResponse i t \\ d)" "\s i t. (s \\ ApplyCommit i t \\') = (s \\ ApplyCommit i t \\)" "\s i t x. (s \\ PublishRequest i t x \\') = (s \\ PublishRequest i t x \\)" "\s i t. (s \\ PublishResponse i t \\') = (s \\ PublishResponse i t \\)" "\i t. (\ ApplyCommit i t \\') = (\ ApplyCommit i t \\)" "\i t x. (\ PublishRequest i t x \\') = (\ PublishRequest i t x \\)" "\i t. (\ PublishResponse i t \\') = (\ PublishResponse i t \\)" "\s d i conf cs. (s \\ CatchUpResponse i conf cs \\' d) = (s \\ CatchUpResponse i conf cs \\ d)" "\d i conf cs. (\ CatchUpResponse i conf cs \\' d) = (\ CatchUpResponse i conf cs \\ d)" "\s i conf cs. (s \\ CatchUpResponse i conf cs \\') = (s \\ CatchUpResponse i conf cs \\)" "\d i conf cs. (\ CatchUpResponse i conf cs \\') = (\ CatchUpResponse i conf cs \\)" by (unfold isMessageFromTo'_def isMessageTo'_def isMessageFrom'_def isMessage'_def, auto simp add: messages' isMessageFromTo_def isMessageTo_def isMessageFrom_def isMessage_def) have Vote': "\s d i t' a. (s \\ Vote i t' a \\' d) = ((s \\ Vote i t' a \\ d) \ (s, i, t', a, d) = (n\<^sub>0, firstUncommittedSlot nd, currentTerm nd, lastAcceptedTerm nd, OneNode d\<^sub>0))" "\d i t' a. (\ Vote i t' a \\' d) = ((\ Vote i t' a \\ d) \ (i, t', a, d) = (firstUncommittedSlot nd, currentTerm nd, lastAcceptedTerm nd, OneNode d\<^sub>0))" "\s i t' a. (s \\ Vote i t' a \\') = ((s \\ Vote i t' a \\) \ (s, i, t', a) = (n\<^sub>0, firstUncommittedSlot nd, currentTerm nd, lastAcceptedTerm nd))" "\i t' a. (\ Vote i t' a \\') = ((\ Vote i t' a \\) \ (i, t', a) = (firstUncommittedSlot nd, currentTerm nd, lastAcceptedTerm nd))" by (unfold isMessageFromTo'_def isMessageFromTo_def isMessageTo'_def isMessageTo_def isMessageFrom'_def isMessageFrom_def isMessage'_def isMessage_def, auto simp add: messages') have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def) have v\<^sub>c_eq[simp]: "v\<^sub>c' = v\<^sub>c" by (intro ext, auto simp add: v\<^sub>c'_def v\<^sub>c_def) have isCommitted_eq[simp]: "isCommitted' = isCommitted" by (intro ext, auto simp add: isCommitted'_def isCommitted_def) have committedTo_eq[simp]: "committed\<^sub><' = committed\<^sub><" by (intro ext, auto simp add: committedTo'_def committedTo_def) have V_eq[simp]: "V' = V" using v\<^sub>c_eq V'_def V_def by blast have lastCommittedClusterStateBefore_eq[simp]: "lastCommittedClusterStateBefore' = lastCommittedClusterStateBefore" unfolding lastCommittedClusterStateBefore_def lastCommittedClusterStateBefore'_def v\<^sub>c_eq .. have promised_eq: "\i s dn t. promised' i s dn t = (promised i s dn t \ (firstUncommittedSlot nd \ i \ s = n\<^sub>0 \ dn = d\<^sub>0 \ t = currentTerm nd))" unfolding promised'_def promised_def Vote' by auto have prevAccepted_eq: "\i t q. prevAccepted' i t q = prevAccepted i t q \ {t'. n\<^sub>0 \ q \ i = firstUncommittedSlot nd \ t = currentTerm nd \ lastAcceptedTerm nd = SomeTerm t'}" unfolding prevAccepted_def prevAccepted'_def Vote' by auto have property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. firstUncommittedSlot (nodeState' n) = firstUncommittedSlot (nodeState n)" "\n. currentVotingNodes (nodeState' n) = currentVotingNodes (nodeState n)" "\n q. isQuorum (nodeState' n) q = isQuorum (nodeState n) q" "\n. lastAcceptedTerm (nodeState' n) = lastAcceptedTerm (nodeState n)" "\n. lastAcceptedValue (nodeState' n) = lastAcceptedValue (nodeState n)" "\n. currentTerm (nodeState' n) = currentTerm (nodeState n)" "\n. publishPermitted (nodeState' n) = publishPermitted (nodeState n)" "\n. publishVotes (nodeState' n) = publishVotes (nodeState n)" "\n. currentClusterState (nodeState' n) = currentClusterState (nodeState n)" "\n. electionWon (nodeState' n) = electionWon (nodeState n)" "\n. joinVotes (nodeState' n) = joinVotes (nodeState n)" by (unfold nodeState'_def, auto simp add: nd_def isQuorum_def nd' addElectionVote_def Let_def) show "zen messages' nodeState'" apply (intro zenI) apply (unfold message_simps property_simps committedTo_eq V_eq lastCommittedClusterStateBefore_eq v_eq) proof - from finite_messages show "finite messages'" by (simp add: messages') from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from committedTo_firstUncommittedSlot show "\n. committed\<^sub>< (firstUncommittedSlot (nodeState n))" . from firstUncommittedSlot_PublishResponse show "\n i t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\" . from lastAcceptedTerm_None show "\n i t. lastAcceptedTerm (nodeState n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_sent show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_max show "\n t t'. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\ \ t' \ t". from electionWon_isQuorum show "\n. electionWon (nodeState n) \ isQuorum (nodeState n) (joinVotes (nodeState n))". from publishVotes show "\n n'. n' \ publishVotes (nodeState n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) \\". from currentClusterState_lastCommittedClusterStateBefore show "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\". from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))". from firstUncommittedSlot_PublishRequest show "\i n t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\". from lastAcceptedTerm_Some_value show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState n)) \\". from PublishRequest_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ t \ currentTerm (nodeState n)". from PublishRequest_publishPermitted_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ publishPermitted (nodeState n) \ t < currentTerm (nodeState n)". from CatchUpResponse_committedTo show "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i". from CatchUpResponse_V show "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf". from CatchUpResponse_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs". from lastAcceptedTerm_Some_currentTerm show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ t \ currentTerm (nodeState n)". from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\' \ t \ currentTerm (nodeState n)" unfolding Vote' by (auto simp add: nd_def) from joinVotes show "\n n'. n' \ joinVotes (nodeState n) \ promised' (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState n))" unfolding promised_eq by simp from Vote_future firstUncommittedSlot_PublishResponse show "\i i' s t t' a. s \\ Vote i t a \\' \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\" unfolding Vote' nd_def by auto from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\' \ t' < t \ \ s \\ PublishResponse i t' \\" unfolding Vote' using Vote_future lastAcceptedTerm_None nd' nd'_def by auto from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\' \ t' < t" unfolding Vote' using not_accepted lastAcceptedTerm_def by (smt fst_conv TermOption.distinct(1) snd_conv) from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\' \ s \\ PublishResponse i t' \\" unfolding Vote' using lastAcceptedTerm_Some_sent nd' nd'_def nd_def by (smt Pair_inject TermOption.distinct(1)) from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\' \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\" unfolding Vote' nd_def by (smt Pair_inject TermOption.distinct(1) \\t' t na. \lastAcceptedTerm (nodeState na) = SomeTerm t; na \\ PublishResponse (firstUncommittedSlot (nodeState na)) t' \\\ \ t' \ t\ leD) from Vote_not_broadcast show "\i t a d. \ Vote i t a \\' d \ d \ Broadcast" unfolding Vote' by blast from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\' d \ s \\ Vote i' t a' \\' d' \ d = d'" unfolding Vote' using isMessageFrom_def not_sent by auto from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\' \ n \\ Vote i' t a' \\' \ i = i'" unfolding Vote' by (metis Vote'(3) Message.inject(2) RoutedMessage.ext_inject insert_iff isMessageFrom'_def isMessageFromTo'_def isMessageFromTo_def isMessageFrom_def messages' not_sent) from joinVotes_max show "\n n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) x \\) \ n' \ joinVotes (nodeState n) \ n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) a' \\' (OneNode n) \ a' \ lastAcceptedTerm (nodeState n)" unfolding Vote' by (smt Vote'(1) Message.inject(2) RoutedMessage.ext_inject insert_iff isMessageFromTo'_def isMessageFrom_def joinVotes messages' not_sent promised_def zen.Vote_slot_function zenMessages.Vote_value_function zenMessages_axioms zen_axioms) fix i s t x assume "s \\ PublishRequest i t x \\" from PublishRequest_quorum [OF this] obtain q where q_majority: "q \ majorities (V i)" and q_promised: "\n. n \ q \ promised i n s t" and q_value: "prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t)" by blast from not_sent q_promised have no_new_terms: "{t'. n\<^sub>0 \ q \ i = firstUncommittedSlot nd \ t = currentTerm nd \ lastAcceptedTerm nd = SomeTerm t'} = {}" unfolding isMessageFrom_def promised_def by blast hence prevAccepted_eq: "prevAccepted' i t q = prevAccepted i t q" unfolding prevAccepted_eq by auto show "\q\majorities (V i). (\n\q. promised' i n s t) \ (prevAccepted' i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted' i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))" proof (intro bexI conjI) from q_majority show "q \ majorities (V i)" . from q_promised show "\n\q. promised' i n s t" unfolding promised_eq by simp from q_value show "prevAccepted' i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted' i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t)" unfolding prevAccepted_eq by simp qed qed qed lemma (in zenStep) handleStartJoin_invariants: fixes t defines "result \ handleStartJoin t nd" assumes nd': "nd' = fst result" assumes sent: "s \\ StartJoin t \\" assumes messages': "messages' = sendTo (OneNode s) result" shows "zen messages' nodeState'" proof (cases "currentTerm nd < t") case False hence result: "result = (nd, None)" by (simp add: result_def handleStartJoin_def) have "messages' = messages" by (auto simp add: messages' result) moreover have "nodeState' = nodeState" by (intro ext, unfold nodeState'_def, simp add: nd' result nd_def) moreover note zen_axioms ultimately show ?thesis by simp next case True hence new_term: "currentTerm nd < t" by simp have result: "result = (ensureCurrentTerm t nd, Some (Vote (firstUncommittedSlot nd) t (lastAcceptedTerm nd)))" by (simp add: result_def handleStartJoin_def True) have nd': "nd' = ensureCurrentTerm t nd" by (simp add: nd' result) have zen1: "zen messages nodeState'" proof (intro zenStep.ensureCurrentTerm_invariants) show "messages = messages" .. from nd' show "nodeState' n\<^sub>0 = ensureCurrentTerm t (nodeState n\<^sub>0)" by (simp add: nd'_def nd_def) show "zenStep messages nodeState messages nodeState' n\<^sub>0" by (intro_locales, intro zenStep_axioms.intro nodeState_unchanged, simp_all) qed with nodeState_unchanged messages_subset have zenStep1: "zenStep messages nodeState' messages' nodeState' n\<^sub>0" by (intro_locales, simp add: zen_def, intro zenStep_axioms.intro, auto) have nodeState': "nodeState' = (\ n. if n = n\<^sub>0 then nd' else nodeState' n)" by (auto simp add: nodeState'_def) have nd'_eq: "nodeState' n\<^sub>0 = nd'" by (simp add: nodeState'_def) from True have currentTerm_nd': "currentTerm nd' = t" by (auto simp add: nd') have [simp]: "firstUncommittedSlot nd' = firstUncommittedSlot nd" "lastAcceptedTerm nd' = lastAcceptedTerm nd" by (auto simp add: nd' ensureCurrentTerm_def lastAcceptedTerm_def lastAcceptedValue_def) show "zen messages' nodeState'" proof (intro zenStep.sendVote_invariants [OF zenStep1], unfold nd'_eq currentTerm_nd', simp_all add: messages' result) show "\i a. \d. \sender = n\<^sub>0, destination = d, payload = Vote i t a\ \ messages" using nd_def new_term zen.Vote_currentTerm zen_axioms by fastforce show "\t'. lastAcceptedTerm nd = SomeTerm t' \ t' < t" using True lastAcceptedTerm_Some_currentTerm unfolding nd_def by fastforce qed qed lemma (in zenStep) addElectionVote_invariants: assumes nd': "nd' = addElectionVote s i a nd" assumes messages': "messages' = messages" assumes sent: "s \\ Vote i (currentTerm nd) a \\ (OneNode n\<^sub>0)" assumes precondition: "i < firstUncommittedSlot nd \ (i = firstUncommittedSlot nd \ a \ lastAcceptedTerm nd)" shows "zen messages' nodeState'" proof - from precondition have slot: "i \ firstUncommittedSlot nd" by auto have message_simps[simp]: "\s p d. (s \\ p \\' d) = (s \\ p \\ d)" "\p d. (\ p \\' d) = (\ p \\ d)" "\s p. (s \\ p \\') = (s \\ p \\)" "\p. (\ p \\') = (\ p \\)" by (unfold isMessageFromTo'_def isMessageTo'_def isMessageFrom'_def isMessage'_def, auto simp add: messages' isMessageFromTo_def isMessageTo_def isMessageFrom_def isMessage_def) have property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. firstUncommittedSlot (nodeState' n) = firstUncommittedSlot (nodeState n)" "\n. currentVotingNodes (nodeState' n) = currentVotingNodes (nodeState n)" "\n q. isQuorum (nodeState' n) q = isQuorum (nodeState n) q" "\n. lastAcceptedTerm (nodeState' n) = lastAcceptedTerm (nodeState n)" "\n. lastAcceptedValue (nodeState' n) = lastAcceptedValue (nodeState n)" "\n. currentTerm (nodeState' n) = currentTerm (nodeState n)" "\n. publishPermitted (nodeState' n) = publishPermitted (nodeState n)" "\n. publishVotes (nodeState' n) = publishVotes (nodeState n)" "\n. currentClusterState (nodeState' n) = currentClusterState (nodeState n)" by (unfold nodeState'_def, auto simp add: nd_def isQuorum_def nd' addElectionVote_def Let_def lastAcceptedValue_def lastAcceptedTerm_def) have updated_properties: "\n. joinVotes (nodeState' n) = joinVotes (nodeState n) \ (if n = n\<^sub>0 then {s} else {})" "\n. electionWon (nodeState' n) = (if n = n\<^sub>0 then isQuorum nd (insert s (joinVotes nd)) else electionWon (nodeState n))" unfolding nodeState'_def by (auto simp add: nd' addElectionVote_def Let_def nd_def) have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def) have v\<^sub>c_eq[simp]: "v\<^sub>c' = v\<^sub>c" by (intro ext, auto simp add: v\<^sub>c'_def v\<^sub>c_def) have isCommitted_eq[simp]: "isCommitted' = isCommitted" by (intro ext, auto simp add: isCommitted'_def isCommitted_def) have committedTo_eq[simp]: "committed\<^sub><' = committed\<^sub><" by (intro ext, auto simp add: committedTo'_def committedTo_def) have V_eq[simp]: "V' = V" using v\<^sub>c_eq V'_def V_def by blast have promised_eq[simp]: "promised' = promised" by (intro ext, auto simp add: promised'_def promised_def) have prevAccepted_eq[simp]: "prevAccepted' = prevAccepted" by (intro ext, auto simp add: prevAccepted'_def prevAccepted_def) have lastCommittedClusterStateBefore_eq[simp]: "lastCommittedClusterStateBefore' = lastCommittedClusterStateBefore" unfolding lastCommittedClusterStateBefore_def lastCommittedClusterStateBefore'_def v\<^sub>c_eq .. show "zen messages' nodeState'" apply (intro zenI) apply (unfold message_simps property_simps committedTo_eq v_eq V_eq promised_eq lastCommittedClusterStateBefore_eq prevAccepted_eq) proof - from finite_messages show "finite messages'" by (simp add: messages') from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from committedTo_firstUncommittedSlot show "\n. committed\<^sub>< (firstUncommittedSlot (nodeState n))" . from firstUncommittedSlot_PublishResponse show "\n i t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\" . from lastAcceptedTerm_None show "\n i t. lastAcceptedTerm (nodeState n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_sent show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_max show "\n t t'. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\ \ t' \ t". from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from publishVotes show "\n n'. n' \ publishVotes (nodeState n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) \\". from currentClusterState_lastCommittedClusterStateBefore show "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\". from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))". from firstUncommittedSlot_PublishRequest show "\i n t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\". from lastAcceptedTerm_Some_value show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState n)) \\". from PublishRequest_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ t \ currentTerm (nodeState n)". from PublishRequest_publishPermitted_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ publishPermitted (nodeState n) \ t < currentTerm (nodeState n)". from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))". from CatchUpResponse_committedTo show "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i". from CatchUpResponse_V show "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf". from CatchUpResponse_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs". from lastAcceptedTerm_Some_currentTerm show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ t \ currentTerm (nodeState n)". from electionWon_isQuorum show "\n. electionWon (nodeState' n) \ isQuorum (nodeState n) (joinVotes (nodeState' n))" unfolding updated_properties by (auto simp add: nd_def) fix n from joinVotes nd_def precondition sent show "\n'. n' \ joinVotes (nodeState' n) \ promised (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState n))" using precondition sent unfolding updated_properties nd_def apply (cases "n = n\<^sub>0", simp_all) using nd_def promised_def slot by blast fix n' a' assume fresh_request: "\ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) x \\)" and n'_joinVotes: "n' \ joinVotes (nodeState' n)" and n'_vote: "n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) a' \\ (OneNode n)" show "a' \ lastAcceptedTerm (nodeState n)" proof (cases "n = n\<^sub>0") case False with fresh_request n'_joinVotes n'_vote joinVotes_max show ?thesis by (simp add: updated_properties) next case n_eq: True show ?thesis proof (cases "n' = s") case n'_eq: True have i: "i = firstUncommittedSlot nd" proof (intro Vote_slot_function) from n'_vote show "n' \\ Vote (firstUncommittedSlot nd) (currentTerm nd) a' \\" unfolding isMessageFrom_def nd_def n_eq by auto from sent show "n' \\ Vote i (currentTerm nd) a \\" unfolding isMessageFrom_def nd_def n'_eq by auto qed have a: "a = a'" proof (intro Vote_value_function) from n'_vote show "n' \\ Vote (firstUncommittedSlot nd) (currentTerm nd) a' \\" unfolding isMessageFrom_def nd_def n_eq by auto from sent show "n' \\ Vote (firstUncommittedSlot nd) (currentTerm nd) a \\" unfolding isMessageFrom_def nd_def n'_eq i by auto qed from precondition show ?thesis by (auto simp add: i a n_eq nd_def) next case False with n'_joinVotes have n'_joinVotes: "n' \ joinVotes nd" by (simp add: nd_def n_eq updated_properties) with fresh_request n'_vote joinVotes_max show ?thesis unfolding nd_def n_eq by auto qed qed qed qed lemma (in zenStep) publishValue_invariants: fixes x defines "result \ publishValue x nd" assumes nd': "nd' = fst result" assumes messages': "messages' = sendTo Broadcast result" assumes x: "lastAcceptedTerm nd \ NO_TERM \ x = lastAcceptedValue nd" shows "zen messages' nodeState'" proof (cases "electionWon nd \ publishPermitted nd") case False hence result: "result = (nd, None)" by (simp add: result_def publishValue_def) have "messages' = messages" by (auto simp add: messages' result) moreover have "nodeState' = nodeState" by (intro ext, unfold nodeState'_def, simp add: nd' result nd_def) moreover note zen_axioms ultimately show ?thesis by simp next case won: True hence result: "result = (nd \ publishPermitted := False \, Some (PublishRequest (firstUncommittedSlot nd) (currentTerm nd) x))" by (simp add: result_def publishValue_def) have messages': "messages' = insert \sender = n\<^sub>0, destination = Broadcast, payload = PublishRequest (firstUncommittedSlot nd) (currentTerm nd) x\ messages" by (simp add: messages' result) have message_simps: "\s d i t a. (s \\ Vote i t a \\' d) = (s \\ Vote i t a \\ d)" "\s d i t. (s \\ PublishResponse i t \\' d) = (s \\ PublishResponse i t \\ d)" "\s d i t. (s \\ ApplyCommit i t \\' d) = (s \\ ApplyCommit i t \\ d)" "\d i t a. (\ Vote i t a \\' d) = (\ Vote i t a \\ d)" "\d i t. (\ PublishResponse i t \\' d) = (\ PublishResponse i t \\ d)" "\d i t. (\ ApplyCommit i t \\' d) = (\ ApplyCommit i t \\ d)" "\s i t a. (s \\ Vote i t a \\') = (s \\ Vote i t a \\)" "\s i t. (s \\ PublishResponse i t \\') = (s \\ PublishResponse i t \\)" "\s i t. (s \\ ApplyCommit i t \\') = (s \\ ApplyCommit i t \\)" "\d i t a. (\ Vote i t a \\') = (\ Vote i t a \\)" "\d i t. (\ PublishResponse i t \\') = (\ PublishResponse i t \\)" "\d i t. (\ ApplyCommit i t \\') = (\ ApplyCommit i t \\)" "\s d i conf cs. (s \\ CatchUpResponse i conf cs \\' d) = (s \\ CatchUpResponse i conf cs \\ d)" "\d i conf cs. (\ CatchUpResponse i conf cs \\' d) = (\ CatchUpResponse i conf cs \\ d)" "\s i conf cs. (s \\ CatchUpResponse i conf cs \\') = (s \\ CatchUpResponse i conf cs \\)" "\d i conf cs. (\ CatchUpResponse i conf cs \\') = (\ CatchUpResponse i conf cs \\)" by (auto simp add: isMessageFromTo'_def isMessageTo'_def isMessage'_def isMessageFrom'_def, auto simp add: isMessageFromTo_def isMessageTo_def isMessage_def isMessageFrom_def messages') have PublishRequest': "\s d i t x'. (s \\ PublishRequest i t x' \\' d) = ((s \\ PublishRequest i t x' \\ d) \ (s, d, i, t, x') = (n\<^sub>0, Broadcast, firstUncommittedSlot nd, currentTerm nd, x))" "\s i t x'. (s \\ PublishRequest i t x' \\') = ((s \\ PublishRequest i t x' \\) \ (s, i, t, x') = (n\<^sub>0, firstUncommittedSlot nd, currentTerm nd, x))" "\d i t x'. (\ PublishRequest i t x' \\' d) = ((\ PublishRequest i t x' \\ d) \ (d, i, t, x') = (Broadcast, firstUncommittedSlot nd, currentTerm nd, x))" "\i t x'. (\ PublishRequest i t x' \\') = ((\ PublishRequest i t x' \\) \ (i, t, x') = (firstUncommittedSlot nd, currentTerm nd, x))" by (auto simp add: isMessageFromTo'_def isMessageFrom'_def isMessageTo'_def isMessage'_def, auto simp add: messages' isMessageFromTo_def isMessageFrom_def isMessageTo_def isMessage_def) have fresh_request: "\x. \ \ PublishRequest (firstUncommittedSlot nd) (currentTerm nd) x \\" proof (intro notI) fix x' assume "\ PublishRequest (firstUncommittedSlot nd) (currentTerm nd) x' \\" with won obtain s d where s: "s \\ PublishRequest (firstUncommittedSlot nd) (currentTerm nd) x' \\ d" by (auto simp add: isMessage_def isMessageFrom_def) with PublishRequest_quorum [where s = s and i = "firstUncommittedSlot nd" and t = "currentTerm nd" and x = x'] obtain q where q: "q \ majorities (V (firstUncommittedSlot nd))" and promised: "\n. n \ q \ promised (firstUncommittedSlot nd) n s (currentTerm nd)" by (auto simp add: isMessageFrom_def, blast) from won have "isQuorum nd (joinVotes nd)" by (unfold nd_def, intro electionWon_isQuorum, simp) with currentVotingNodes_firstUncommittedSlot [of n\<^sub>0] have "joinVotes nd \ majorities (V (firstUncommittedSlot nd))" using nd_def isQuorum_def by auto from q this V_intersects have "q \ joinVotes nd \ {}" by (auto simp add: intersects_def) then obtain n where n: "n \ q" "n \ joinVotes nd" by auto from promised [OF `n \ q`] obtain i' a' where "n \\ Vote i' (currentTerm nd) a' \\ (OneNode s)" by (auto simp add: promised_def) moreover from joinVotes n obtain i'' a'' where "n \\ Vote i'' (currentTerm nd) a'' \\ (OneNode n\<^sub>0)" by (auto simp add: nd_def promised_def, blast) ultimately have "OneNode s = OneNode n\<^sub>0" by (intro Vote_unique_destination) with s have "n\<^sub>0 \\ PublishRequest (firstUncommittedSlot nd) (currentTerm nd) x' \\" by (auto simp add: isMessageFrom_def) hence "currentTerm nd < currentTerm (nodeState n\<^sub>0)" proof (intro PublishRequest_publishPermitted_currentTerm, fold nd_def) from won show "publishPermitted nd" by (simp add: nd_def) qed thus False by (simp add: nd_def) qed have v_eq: "\i t x. \ PublishRequest i t x \\ \ v' i t = v i t" proof - fix i t x' assume "\ PublishRequest i t x' \\" with fresh_request have ne: "(i, t) \ (firstUncommittedSlot nd, currentTerm nd)" by auto have sent_eq: "\x''. \ PublishRequest i t x'' \\' = \ PublishRequest i t x'' \\" proof (intro iffI) fix x'' show "\ PublishRequest i t x'' \\ \ \ PublishRequest i t x'' \\'" by (simp add: PublishRequest') assume "\ PublishRequest i t x'' \\'" with ne show "\ PublishRequest i t x'' \\" by (unfold PublishRequest', auto) qed show "v' i t = v i t" by (unfold v'_def v_def sent_eq, simp) qed have isCommitted_eq: "isCommitted' = isCommitted" by (intro ext, simp add: isCommitted_def isCommitted'_def message_simps) have committedTo_eq: "committed\<^sub><' = committed\<^sub><" by (intro ext, simp add: committedTo_def committedTo'_def isCommitted_eq) have v\<^sub>c_eq: "\i. isCommitted i \ v\<^sub>c' i = v\<^sub>c i" proof - fix i assume i: "isCommitted i" define t where "t \ SOME t. \ ApplyCommit i t \\" have t: "\ ApplyCommit i t \\" proof - from i obtain t where t: "\ ApplyCommit i t \\" by (auto simp add: isCommitted_def) thus ?thesis by (unfold t_def, intro someI) qed hence v_eq: "v' i t = v i t" by (intro v_eq [OF ApplyCommit_PublishRequest]) have "v\<^sub>c' i = v' i t" by (simp add: v\<^sub>c'_def t_def message_simps) also note v_eq also have "v i t = v\<^sub>c i" by (simp add: v\<^sub>c_def t_def) finally show "?thesis i" by simp qed have V_eq: "\i. committed\<^sub>< i \ V' i = V i" proof - fix i assume i: "committed\<^sub>< i" thus "?thesis i" proof (induct i) case (Suc i') hence prems: "committed\<^sub>< i'" "isCommitted i'" unfolding committedTo_def by auto thus ?case using Suc v\<^sub>c_eq by simp qed simp qed have lastCommittedClusterStateBefore_eq: "\i. committed\<^sub>< i \ lastCommittedClusterStateBefore' i = lastCommittedClusterStateBefore i" proof - fix i assume "committed\<^sub>< i" thus "?thesis i" proof (induct i) case (Suc i') hence prems: "committed\<^sub>< i'" "isCommitted i'" unfolding committedTo_def by auto thus ?case using Suc v\<^sub>c_eq by (cases "v\<^sub>c i'", simp_all) qed simp qed have promised_eq: "\i n n' t. promised' i n n' t = promised i n n' t" by (simp add: promised_def promised'_def message_simps) have prevAccepted_eq: "\i t q. prevAccepted' i t q = prevAccepted i t q" by (simp add: prevAccepted_def prevAccepted'_def message_simps) from committedTo_firstUncommittedSlot V_eq have V_slot_eq: "\n. V' (firstUncommittedSlot (nodeState n)) = V (firstUncommittedSlot (nodeState n))" by blast have lastCommittedClusterStateBefore_slot_eq: "\n. lastCommittedClusterStateBefore' (firstUncommittedSlot (nodeState n)) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))" by (intro lastCommittedClusterStateBefore_eq committedTo_firstUncommittedSlot) have v_prevAccepted_eq: "\i t q. prevAccepted i t q \ {} \ v' i (maxTerm (prevAccepted i t q)) = v i (maxTerm (prevAccepted i t q))" proof - fix i t q assume nonempty: "prevAccepted i t q \ {}" have "maxTerm (prevAccepted i t q) \ prevAccepted i t q" by (intro maxTerm_mem finite_prevAccepted nonempty) hence "\ Vote i t (SomeTerm (maxTerm (prevAccepted i t q))) \\" by (auto simp add: prevAccepted_def isMessage_def) hence "\ PublishResponse i (maxTerm (prevAccepted i t q)) \\" using Vote_Some_PublishResponse unfolding isMessage_def by blast then obtain x' where "\ PublishRequest i (maxTerm (prevAccepted i t q)) x' \\" using PublishResponse_PublishRequest unfolding isMessage_def by blast thus "?thesis i t q" by (intro v_eq) qed have nd': "nd' = nd \ publishPermitted := False \" by (simp add: nd' result) have property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. firstUncommittedSlot (nodeState' n) = firstUncommittedSlot (nodeState n)" "\n. currentVotingNodes (nodeState' n) = currentVotingNodes (nodeState n)" "\n q. isQuorum (nodeState' n) q = isQuorum (nodeState n) q" "\n. lastAcceptedValue (nodeState' n) = lastAcceptedValue (nodeState n)" "\n. lastAcceptedTerm (nodeState' n) = lastAcceptedTerm (nodeState n)" "\n. currentTerm (nodeState' n) = currentTerm (nodeState n)" "\n. publishVotes (nodeState' n) = publishVotes (nodeState n)" "\n. currentClusterState (nodeState' n) = currentClusterState (nodeState n)" "\n. electionWon (nodeState' n) = electionWon (nodeState n)" "\n. joinVotes (nodeState' n) = joinVotes (nodeState n)" by (unfold nodeState'_def, auto simp add: nd_def isQuorum_def nd' addElectionVote_def Let_def lastAcceptedTerm_def lastAcceptedValue_def) show ?thesis apply (intro zenI) apply (unfold message_simps committedTo_eq V_slot_eq lastCommittedClusterStateBefore_slot_eq property_simps promised_eq prevAccepted_eq) proof - from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from committedTo_firstUncommittedSlot show "\n. committed\<^sub>< (firstUncommittedSlot (nodeState n))" . from firstUncommittedSlot_PublishResponse show "\n i t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\" . from lastAcceptedTerm_None show "\n i t. lastAcceptedTerm (nodeState n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_sent show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_max show "\n t t'. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\ \ t' \ t". from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from electionWon_isQuorum show "\n. electionWon (nodeState n) \ isQuorum (nodeState n) (joinVotes (nodeState n))". from publishVotes show "\n n'. n' \ publishVotes (nodeState n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) \\". from currentClusterState_lastCommittedClusterStateBefore show "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))". from joinVotes show "\n n'. n' \ joinVotes (nodeState n) \ promised (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState n))". from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))". from lastAcceptedTerm_Some_currentTerm show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ t \ currentTerm (nodeState n)". { fix i conf cs assume a: "\ CatchUpResponse i conf cs \\" with CatchUpResponse_committedTo show "committed\<^sub>< i" . with a V_eq lastCommittedClusterStateBefore_eq CatchUpResponse_V CatchUpResponse_lastCommittedClusterStateBefore show "V' i = conf" "lastCommittedClusterStateBefore' i = cs" by auto } from lastAcceptedTerm_Some_value PublishRequest' show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState n)) \\'" by (meson isMessage'_def isMessageFrom'_def isMessageFrom_def isMessage_def) from PublishRequest_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\' \ t \ currentTerm (nodeState n)" using PublishRequest' isMessageFrom'_def isMessageFrom_def nd_def by fastforce from fresh_request PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\' \ \ PublishRequest i t x' \\' \ x = x'" unfolding PublishRequest' by auto from messages' finite_messages show "finite messages'" by simp from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\'" unfolding PublishRequest' by blast from ApplyCommit_quorum V_eq isCommitted_committedTo show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V' i). \s\q. s \\ PublishResponse i t \\" unfolding isCommitted_def by fastforce fix n from firstUncommittedSlot_PublishRequest show "\i t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\'" by (cases "n = n\<^sub>0", unfold nodeState'_def PublishRequest' isMessageFrom'_def isMessageFrom_def, auto simp add: nd' nd_def) from PublishRequest_publishPermitted_currentTerm show "\t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\' \ publishPermitted (nodeState' n) \ t < currentTerm (nodeState n)" unfolding isMessageFrom'_def PublishRequest' by (cases "n = n\<^sub>0", auto simp add: nodeState'_def nd' isMessageFrom_def) show PublishRequest_committedTo': "\i t x. \ PublishRequest i t x \\' \ committed\<^sub>< i" using PublishRequest_committedTo committedTo_firstUncommittedSlot by (auto simp add: committedTo_def PublishRequest' nd_def) from joinVotes_max show "\n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) x \\') \ n' \ joinVotes (nodeState n) \ n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState n)" by (cases "n = n\<^sub>0", auto simp add: nodeState'_def nd' PublishRequest') { fix i s t x' assume "s \\ PublishRequest i t x' \\'" thus "\q\majorities (V' i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v' i t = v' i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))" unfolding PublishRequest' proof (elim disjE) assume sent: "s \\ PublishRequest i t x' \\" from sent have [simp]: "V' i = V i" by (intro V_eq PublishRequest_committedTo, auto simp add: isMessage_def) from sent have [simp]: "v' i t = v i t" by (intro v_eq, auto simp add: isMessage_def) from PublishRequest_quorum [OF sent] obtain q where q_quorum: "q \ majorities (V i)" and q_promised: "\n. n \ q \ promised i n s t" and q_value: "prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t)" by blast show ?thesis proof (cases "prevAccepted i t q = {}") case True with q_quorum q_promised show ?thesis by auto next case False with q_value obtain t' where t': "v i t = v i t'" "maxTerm (prevAccepted i t q) \ t'" "\ PublishResponse i t' \\" "t' < t" by blast show ?thesis proof (intro bexI conjI disjI2 exI ballI) from q_promised show "\n. n \ q \ promised i n s t" . from t' show "maxTerm (prevAccepted i t q) \ t'" "\ PublishResponse i t' \\" "t' < t" by simp_all from q_quorum show "q \ majorities (V' i)" by simp have "v' i t = v i t" by simp also have "... = v i t'" using t' by simp also have "... = v' i t'" unfolding v_def v'_def PublishRequest' using t' by (metis PublishResponse_PublishRequest_v fresh_request fst_conv snd_conv) finally show "v' i t = v' i t'" . qed qed next assume "(s, i, t, x') = (n\<^sub>0, firstUncommittedSlot nd, currentTerm nd, x)" hence [simp]: "s = n\<^sub>0" "i = firstUncommittedSlot nd" "t = currentTerm nd" "x' = x" by simp_all have Vi: "V' i = V i" using committedTo_firstUncommittedSlot by (intro V_eq, simp add: nd_def) define q where "q \ joinVotes nd" show ?thesis proof (intro bexI conjI ballI) from won have "isQuorum nd q" unfolding nd_def q_def by (intro electionWon_isQuorum, simp) with currentVotingNodes_firstUncommittedSlot Vi show "q \ majorities (V' i)" by (auto simp add: nd_def isQuorum_def) fix n assume "n \ q" with joinVotes show "promised i n s t" by (simp add: nd_def q_def) next show "prevAccepted i t q = {} \ (\t'. v' i t = v' i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t)" proof (cases "prevAccepted i t q = {}") case True thus ?thesis by simp next case False then obtain t' n' where n'_vote: "n' \ q" and n'_sent: "n' \\ Vote i t (SomeTerm t') \\" unfolding prevAccepted_def by auto from n'_sent n'_vote have n'_sent: "n' \\ Vote i t (SomeTerm t') \\ (OneNode n\<^sub>0)" unfolding isMessageFrom_def q_def nd_def using \t = currentTerm nd\ isMessageFromTo_def nd_def zen.joinVotes zenMessages.Vote_unique_destination zenMessages_axioms zen_axioms by blast have t'_latis: "SomeTerm t' \ lastAcceptedTerm nd" unfolding nd_def proof (intro joinVotes_max) from fresh_request show "\ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n\<^sub>0)) (currentTerm (nodeState n\<^sub>0)) x \\)" unfolding nd_def by auto from n'_vote show "n' \ joinVotes (nodeState n\<^sub>0)" by (simp add: q_def nd_def) from n'_sent show "n' \\ Vote (firstUncommittedSlot (nodeState n\<^sub>0)) (currentTerm (nodeState n\<^sub>0)) (SomeTerm t') \\ (OneNode n\<^sub>0)" by (simp add: nd_def) qed then obtain t'' where t'': "lastAcceptedTerm nd = SomeTerm t''" by (cases "lastAcceptedTerm nd", auto) show ?thesis proof (intro disjI2 exI conjI) from t'' have slots: "firstUncommittedSlot nd = firstUncommittedSlot nd" by (cases "firstUncommittedSlot nd = firstUncommittedSlot nd", simp_all add: lastAcceptedTerm_def) from t'' have lat: "lastAcceptedTerm nd = SomeTerm t''" by (cases "firstUncommittedSlot nd = firstUncommittedSlot nd", simp_all add: lastAcceptedTerm_def) from lastAcceptedTerm_Some_sent lat slots show accepted: "\ PublishResponse i t'' \\" by (auto simp add: nd_def isMessage_def) from fresh_request have "\x'. \ PublishRequest i t x' \\' \ x' = x" unfolding PublishRequest' by simp hence "v' i t = x" unfolding v'_def using \s \\ PublishRequest i t x' \\'\ isMessage'_def by blast also from x have "x = lastAcceptedValue nd" by (simp add: t'') also have "... = v i t''" proof (intro PublishRequest_function) from accepted PublishResponse_PublishRequest_v show "\ PublishRequest i t'' (v i t'') \\" by simp from lat lastAcceptedTerm_Some_value slots show "\ PublishRequest i t'' (lastAcceptedValue nd) \\" by (simp add: nd_def) qed also have "... = v' i t''" proof (intro sym [OF v_eq]) from accepted PublishResponse_PublishRequest_v show "\ PublishRequest i t'' (v i t'') \\" by simp qed finally show "v' i t = v' i t''" . show "maxTerm (prevAccepted i t q) \ t''" proof (intro maxTerm_le False finite_prevAccepted) fix t''' assume "t''' \ prevAccepted i t q" thus "t''' \ t''" unfolding prevAccepted_def apply auto by (metis \\t s i' ia d' d a' a. \s \\ Vote ia t a \\ d; s \\ Vote i' t a' \\ d'\ \ d = d'\ fresh_request isMessageFrom_def joinVotes joinVotes_max le_SomeTerm nd_def promised_def q_def t'') qed have "t'' \ t" using \t = currentTerm nd\ lastAcceptedTerm_Some_currentTerm lat nd_def by blast moreover have "t'' \ t" apply (intro notI) using accepted fresh_request using lastAcceptedTerm_Some_value lat nd_def slots by auto ultimately show "t'' < t" by simp qed qed qed qed } qed qed lemma (in zenStep) handleVote_invariants: fixes s i t a defines "result \ handleVote s i t a nd" assumes nd': "nd' = fst result" assumes messages': "messages' = sendTo Broadcast result" assumes sent: "s \\ Vote i t a \\ (OneNode n\<^sub>0)" shows "zen messages' nodeState'" proof (cases "t = currentTerm nd \ (i < firstUncommittedSlot nd \ (i = firstUncommittedSlot nd \ (a \ lastAcceptedTerm nd)))") case False have [simp]: "result = (nd, None)" unfolding result_def handleVote_def by (simp add: False Let_def) have [simp]: "messages' = messages" by (simp add: messages') have [simp]: "nodeState' = nodeState" unfolding nodeState'_def by (intro ext, simp add: nd' nd_def) from zen_axioms show ?thesis by simp next case True hence True': "i \ firstUncommittedSlot nd" "t = currentTerm nd" "i < firstUncommittedSlot nd \ (i = firstUncommittedSlot nd \ a \ lastAcceptedTerm nd)" by auto define nd1 where "nd1 \ addElectionVote s i a nd" define nodeState1 where "\n. nodeState1 n \ if n = n\<^sub>0 then nd1 else nodeState n" have zenStep1: "zenStep messages nodeState messages nodeState1 n\<^sub>0" by (intro zenStepI2, auto simp add: nodeState1_def) have zenStep2: "zenStep messages nodeState1 messages' nodeState' n\<^sub>0" proof (intro zenStep.zenStepI1 [OF zenStep1] zenStep.addElectionVote_invariants [OF zenStep1] refl messages_subset, fold nd_def isMessageFromTo_def) show "nodeState1 n\<^sub>0 = addElectionVote s i a nd" by (simp add: nodeState1_def nd1_def) from True' sent show "s \\ Vote i (currentTerm nd) a \\ (OneNode n\<^sub>0)" by simp from True' show "i < firstUncommittedSlot nd \ (i = firstUncommittedSlot nd \ a \ lastAcceptedTerm nd)" by simp show "\n. n \ n\<^sub>0 \ nodeState1 n = nodeState' n" unfolding nodeState1_def nodeState'_def by simp qed note zenStep.publishValue_invariants have latis_eq: "lastAcceptedTerm nd1 = lastAcceptedTerm nd" by (simp add: nd1_def addElectionVote_def Let_def) show ?thesis proof (cases "lastAcceptedTerm nd1") case (SomeTerm t') with True' latis_eq have handleVote_eq: "handleVote s i t a nd = publishValue (lastAcceptedValue nd1) nd1" unfolding handleVote_def nd1_def by (simp add: addElectionVote_def Let_def) show ?thesis proof (intro zenStep.publishValue_invariants [OF zenStep2]) show "nodeState' n\<^sub>0 = fst (publishValue (lastAcceptedValue nd1) (nodeState1 n\<^sub>0))" by (simp add: nodeState1_def nd1_def nodeState'_def nd' result_def handleVote_eq) show "messages' = (case snd (publishValue (lastAcceptedValue nd1) (nodeState1 n\<^sub>0)) of None \ messages | Some m \ insert \sender = n\<^sub>0, destination = Broadcast, payload = m\ messages)" by (cases "publishValue (lastAcceptedValue nd1) nd1", cases "snd (publishValue (lastAcceptedValue nd1) nd1)", simp_all add: nodeState1_def messages' result_def handleVote_eq) show "lastAcceptedValue nd1 = lastAcceptedValue (nodeState1 n\<^sub>0)" by (simp add: nodeState1_def) qed next case NO_TERM with True' latis_eq have handleVote_eq: "handleVote s i t a nd = (nd1, None)" unfolding handleVote_def nd1_def by simp have [simp]: "messages' = messages" by (simp add: messages' result_def handleVote_eq) have [simp]: "nodeState' = nodeState1" by (intro ext, simp add: nodeState'_def nodeState1_def nd' result_def handleVote_eq) from zenStep2 show ?thesis by (simp add: zenStep_def) qed qed lemma (in zenStep) handleClientValue_invariants: fixes x defines "result \ handleClientValue x nd" assumes nd': "nd' = fst result" assumes messages': "messages' = sendTo Broadcast result" shows "zen messages' nodeState'" proof (cases "lastAcceptedTerm nd") case (SomeTerm t') hence "handleClientValue x nd = (nd, None)" by (auto simp add: handleClientValue_def) hence [simp]: "result = (nd, None)" by (simp add: result_def) have [simp]: "messages' = messages" by (simp add: messages') have [simp]: "nodeState' = nodeState" unfolding nodeState'_def by (intro ext, simp add: nd' nd_def) from zen_axioms show ?thesis by simp next case NO_TERM hence handleClientValue_eq[simp]: "handleClientValue x nd = publishValue x nd" by (auto simp add: handleClientValue_def) have result: "result = publishValue x nd" unfolding result_def by simp show ?thesis proof (intro publishValue_invariants) show "nd' = fst (publishValue x nd)" by (simp add: result nd') show "messages' = sendTo Broadcast (publishValue x nd)" by (simp_all add: messages' result) from NO_TERM show "lastAcceptedTerm nd \ NO_TERM \ x = lastAcceptedValue nd" by simp qed qed lemma (in zenStep) handlePublishRequest_invariants: fixes i t x defines "result \ handlePublishRequest i t x nd" assumes nd': "nd' = fst result" assumes messages': "messages' = sendTo (OneNode s) result" assumes sent: "s \\ PublishRequest i t x \\" shows "zen messages' nodeState'" proof (cases "i = firstUncommittedSlot nd \ t = currentTerm nd") case False hence [simp]: "result = (nd, None)" by (simp add: result_def handlePublishRequest_def) have [simp]: "messages' = messages" by (simp add: messages') have [simp]: "nodeState' = nodeState" unfolding nodeState'_def by (intro ext, simp add: nd' nd_def) from zen_axioms show ?thesis by simp next case precondition: True hence result: "result = (nd\lastAcceptedData := Some \ tvTerm = t, tvValue = x\ \, Some (PublishResponse i t))" by (auto simp add: result_def handlePublishRequest_def) have messages': "messages' = insert \ sender = n\<^sub>0, destination = OneNode s, payload = PublishResponse i t \ messages" by (simp add: messages' result) have message_simps[simp]: "\s d i t. (s \\ ApplyCommit i t \\' d) = (s \\ ApplyCommit i t \\ d)" "\s d i t x. (s \\ PublishRequest i t x \\' d) = (s \\ PublishRequest i t x \\ d)" "\s d i t a. (s \\ Vote i t a \\' d) = (s \\ Vote i t a \\ d)" "\d i t. (\ ApplyCommit i t \\' d) = (\ ApplyCommit i t \\ d)" "\d i t x. (\ PublishRequest i t x \\' d) = (\ PublishRequest i t x \\ d)" "\d i t a. (\ Vote i t a \\' d) = (\ Vote i t a \\ d)" "\s i t. (s \\ ApplyCommit i t \\') = (s \\ ApplyCommit i t \\)" "\s i t x. (s \\ PublishRequest i t x \\') = (s \\ PublishRequest i t x \\)" "\s i t a. (s \\ Vote i t a \\') = (s \\ Vote i t a \\)" "\i t. (\ ApplyCommit i t \\') = (\ ApplyCommit i t \\)" "\i t x. (\ PublishRequest i t x \\') = (\ PublishRequest i t x \\)" "\i t a. (\ Vote i t a \\') = (\ Vote i t a \\)" "\s d i conf cs. (s \\ CatchUpResponse i conf cs \\' d) = (s \\ CatchUpResponse i conf cs \\ d)" "\d i conf cs. (\ CatchUpResponse i conf cs \\' d) = (\ CatchUpResponse i conf cs \\ d)" "\s i conf cs. (s \\ CatchUpResponse i conf cs \\') = (s \\ CatchUpResponse i conf cs \\)" "\d i conf cs. (\ CatchUpResponse i conf cs \\') = (\ CatchUpResponse i conf cs \\)" by (unfold isMessageFromTo'_def isMessageTo'_def isMessageFrom'_def isMessage'_def, auto simp add: messages' isMessageFromTo_def isMessageTo_def isMessageFrom_def isMessage_def) have property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. firstUncommittedSlot (nodeState' n) = firstUncommittedSlot (nodeState n)" "\n. currentVotingNodes (nodeState' n) = currentVotingNodes (nodeState n)" "\n q. isQuorum (nodeState' n) q = isQuorum (nodeState n) q" "\n. currentTerm (nodeState' n) = currentTerm (nodeState n)" "\n. publishPermitted (nodeState' n) = publishPermitted (nodeState n)" "\n. joinVotes (nodeState' n) = joinVotes (nodeState n)" "\n. electionWon (nodeState' n) = electionWon (nodeState n)" "\n. publishVotes (nodeState' n) = publishVotes (nodeState n)" "\n. currentClusterState (nodeState' n) = currentClusterState (nodeState n)" "lastAcceptedTerm nd' = SomeTerm t" "lastAcceptedValue nd' = x" using precondition by (unfold nodeState'_def, auto simp add: result nd' nd_def isQuorum_def lastAcceptedTerm_def lastAcceptedValue_def) have updated_properties: "\n. lastAcceptedTerm (nodeState' n) = (if n = n\<^sub>0 then SomeTerm t else lastAcceptedTerm (nodeState n))" "\n. lastAcceptedValue (nodeState' n) = (if n = n\<^sub>0 then x else lastAcceptedValue (nodeState n))" by (unfold nodeState'_def, auto simp add: result nd' nd_def lastAcceptedTerm_def lastAcceptedValue_def) have PublishResponse': "\s' d i t'. (s' \\ PublishResponse i t' \\' d) = ((s' \\ PublishResponse i t' \\ d) \ (s', i, t', d) = (n\<^sub>0, firstUncommittedSlot nd, t, OneNode s))" "\d i t'. (\ PublishResponse i t' \\' d) = ((\ PublishResponse i t' \\ d) \ (i, t', d) = (firstUncommittedSlot nd, t, OneNode s))" "\s' i t'. (s' \\ PublishResponse i t' \\') = ((s' \\ PublishResponse i t' \\) \ (s', i, t') = (n\<^sub>0, firstUncommittedSlot nd, t))" "\i t'. (\ PublishResponse i t' \\') = ((\ PublishResponse i t' \\) \ (i, t') = (firstUncommittedSlot nd, t))" unfolding isMessageFromTo_def isMessageFromTo'_def isMessageFrom_def isMessageFrom'_def isMessageTo_def isMessageTo'_def isMessage_def isMessage'_def by (auto simp add: messages' precondition) have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def) have v\<^sub>c_eq[simp]: "v\<^sub>c' = v\<^sub>c" by (intro ext, auto simp add: v\<^sub>c'_def v\<^sub>c_def) have isCommitted_eq[simp]: "isCommitted' = isCommitted" by (intro ext, auto simp add: isCommitted'_def isCommitted_def) have committedTo_eq[simp]: "committed\<^sub><' = committed\<^sub><" by (intro ext, auto simp add: committedTo'_def committedTo_def) have V_eq[simp]: "V' = V" using v\<^sub>c_eq V'_def V_def by blast have lastCommittedClusterStateBefore_eq[simp]: "lastCommittedClusterStateBefore' = lastCommittedClusterStateBefore" unfolding lastCommittedClusterStateBefore_def lastCommittedClusterStateBefore'_def v\<^sub>c_eq .. have promised_eq[simp]: "promised' = promised" by (intro ext, auto simp add: promised'_def promised_def) have prevAccepted_eq[simp]: "prevAccepted' = prevAccepted" by (intro ext, auto simp add: prevAccepted'_def prevAccepted_def) show ?thesis apply (intro zenI) apply (unfold message_simps committedTo_eq V_eq v_eq lastCommittedClusterStateBefore_eq property_simps promised_eq prevAccepted_eq) proof - from finite_messages show "finite messages'" by (simp add: messages') from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from committedTo_firstUncommittedSlot show "\n. committed\<^sub>< (firstUncommittedSlot (nodeState n))" . from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from electionWon_isQuorum show "\n. electionWon (nodeState n) \ isQuorum (nodeState n) (joinVotes (nodeState n))". from currentClusterState_lastCommittedClusterStateBefore show "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))". from joinVotes show "\n n'. n' \ joinVotes (nodeState n) \ promised (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState n))". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))". from firstUncommittedSlot_PublishRequest show "\i n t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\". from PublishRequest_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ t \ currentTerm (nodeState n)". from PublishRequest_publishPermitted_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ publishPermitted (nodeState n) \ t < currentTerm (nodeState n)". from CatchUpResponse_committedTo show "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i". from CatchUpResponse_V show "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf". from CatchUpResponse_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs". from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\'" using precondition unfolding PublishResponse' nd_def by (metis Vote_currentTerm Pair_inject leD) from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\'" using precondition unfolding PublishResponse' nd_def by (metis Vote_currentTerm Pair_inject leD) from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\'" using precondition unfolding PublishResponse' nd_def by metis from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\'" using precondition unfolding PublishResponse' nd_def using Vote_currentTerm by fastforce from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\' \ t' < t))" unfolding PublishResponse' by meson fix n from firstUncommittedSlot_PublishResponse show "\i t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\'" using precondition property_simps unfolding PublishResponse' nd_def by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_None show "\i t. lastAcceptedTerm (nodeState' n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\'" using precondition property_simps updated_properties unfolding PublishResponse' nd_def by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_Some_sent show "\n t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\'" using precondition property_simps unfolding PublishResponse' nd_def by (metis nd'_def nodeState_unchanged TermOption.inject) from lastAcceptedTerm_Some_max show "\t t'. lastAcceptedTerm (nodeState' n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\' \ t' \ t" using precondition property_simps updated_properties unfolding PublishResponse' nd_def apply (cases "n = n\<^sub>0", auto) by (meson TermOption.exhaust lastAcceptedTerm_None lastAcceptedTerm_Some_currentTerm le_trans) from joinVotes_max show "\n n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) x \\) \ n' \ joinVotes (nodeState n) \ n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState' n)" using precondition property_simps unfolding PublishResponse' nd_def by (metis isMessage_def nodeState_unchanged sent) from publishVotes show "\n n'. n' \ publishVotes (nodeState n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) \\'" using precondition property_simps unfolding PublishResponse' nd_def by blast from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\' \ \x. \ PublishRequest i t x \\" using precondition property_simps unfolding PublishResponse' nd_def using isMessage_def sent by auto from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\'" using precondition property_simps unfolding PublishResponse' nd_def by meson from lastAcceptedTerm_Some_value show "\n t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState' n)) \\" using precondition property_simps unfolding PublishResponse' nd_def by (metis isMessage_def nd'_def nodeState_unchanged TermOption.inject sent) from lastAcceptedTerm_Some_currentTerm show "\n t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ t \ currentTerm (nodeState n)" using precondition property_simps unfolding PublishResponse' nd_def by (metis eq_imp_le TermOption.inject updated_properties(1)) qed qed lemma (in zenStep) addPublishVote_invariants: assumes nd': "nd' = nd \ publishVotes := insert s (publishVotes nd) \" assumes messages': "messages' = messages" assumes sent: "s \\ PublishResponse (firstUncommittedSlot nd) (currentTerm nd) \\" shows "zen messages' nodeState'" proof - have message_simps[simp]: "\s p d. (s \\ p \\' d) = (s \\ p \\ d)" "\p d. (\ p \\' d) = (\ p \\ d)" "\s p. (s \\ p \\') = (s \\ p \\)" "\p. (\ p \\') = (\ p \\)" by (unfold isMessageFromTo'_def isMessageTo'_def isMessageFrom'_def isMessage'_def, auto simp add: messages' isMessageFromTo_def isMessageTo_def isMessageFrom_def isMessage_def) have property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. firstUncommittedSlot (nodeState' n) = firstUncommittedSlot (nodeState n)" "\n. currentVotingNodes (nodeState' n) = currentVotingNodes (nodeState n)" "\n q. isQuorum (nodeState' n) q = isQuorum (nodeState n) q" "\n. lastAcceptedTerm (nodeState' n) = lastAcceptedTerm (nodeState n)" "\n. lastAcceptedValue (nodeState' n) = lastAcceptedValue (nodeState n)" "\n. currentTerm (nodeState' n) = currentTerm (nodeState n)" "\n. publishPermitted (nodeState' n) = publishPermitted (nodeState n)" "\n. joinVotes (nodeState' n) = joinVotes (nodeState n)" "\n. electionWon (nodeState' n) = electionWon (nodeState n)" "\n. currentClusterState (nodeState' n) = currentClusterState (nodeState n)" by (unfold nodeState'_def, auto simp add: nd_def isQuorum_def nd' addElectionVote_def Let_def lastAcceptedTerm_def lastAcceptedValue_def) have updated_properties[simp]: "\n. publishVotes (nodeState' n) = publishVotes (nodeState n) \ (if n = n\<^sub>0 then {s} else {})" by (unfold nodeState'_def, auto simp add: nd' nd_def isQuorum_def) have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def) have v\<^sub>c_eq[simp]: "v\<^sub>c' = v\<^sub>c" by (intro ext, auto simp add: v\<^sub>c'_def v\<^sub>c_def) have isCommitted_eq[simp]: "isCommitted' = isCommitted" by (intro ext, auto simp add: isCommitted'_def isCommitted_def) have committedTo_eq[simp]: "committed\<^sub><' = committed\<^sub><" by (intro ext, auto simp add: committedTo'_def committedTo_def) have V_eq[simp]: "V' = V" using v\<^sub>c_eq V'_def V_def by blast have promised_eq[simp]: "promised' = promised" by (intro ext, auto simp add: promised'_def promised_def) have lastCommittedClusterStateBefore_eq[simp]: "lastCommittedClusterStateBefore' = lastCommittedClusterStateBefore" unfolding lastCommittedClusterStateBefore_def lastCommittedClusterStateBefore'_def v\<^sub>c_eq .. have prevAccepted_eq[simp]: "prevAccepted' = prevAccepted" by (intro ext, auto simp add: prevAccepted'_def prevAccepted_def) show ?thesis apply (intro zenI) apply (unfold message_simps committedTo_eq V_eq v_eq lastCommittedClusterStateBefore_eq property_simps promised_eq prevAccepted_eq) proof - from finite_messages show "finite messages'" by (simp add: messages') from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from committedTo_firstUncommittedSlot show "\n. committed\<^sub>< (firstUncommittedSlot (nodeState n))" . from firstUncommittedSlot_PublishResponse show "\n i t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\" . from lastAcceptedTerm_None show "\n i t. lastAcceptedTerm (nodeState n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_sent show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_max show "\n t t'. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\ \ t' \ t". from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from electionWon_isQuorum show "\n. electionWon (nodeState n) \ isQuorum (nodeState n) (joinVotes (nodeState n))". from joinVotes_max show "\n n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) x \\) \ n' \ joinVotes (nodeState n) \ n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState n)". from currentClusterState_lastCommittedClusterStateBefore show "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))". from joinVotes show "\n n'. n' \ joinVotes (nodeState n) \ promised (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState n))". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\". from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))". from firstUncommittedSlot_PublishRequest show "\i n t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\". from lastAcceptedTerm_Some_value show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState n)) \\". from PublishRequest_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ t \ currentTerm (nodeState n)". from PublishRequest_publishPermitted_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ publishPermitted (nodeState n) \ t < currentTerm (nodeState n)". from CatchUpResponse_committedTo show "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i". from CatchUpResponse_V show "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf". from CatchUpResponse_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs". from lastAcceptedTerm_Some_currentTerm show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ t \ currentTerm (nodeState n)". from publishVotes show "\n n'. n' \ publishVotes (nodeState' n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) \\" unfolding updated_properties using sent apply auto by (metis empty_iff nd_def singleton_iff) qed qed lemma (in zenStep) commitIfQuorate_invariants: fixes s i t defines "result \ commitIfQuorate nd" assumes nd': "nd' = fst result" assumes messages': "messages' = sendTo Broadcast result" shows "zen messages' nodeState'" proof (cases "isQuorum nd (publishVotes nd)") case False hence [simp]: "result = (nd, None)" by (simp add: result_def commitIfQuorate_def) have [simp]: "messages' = messages" by (simp add: messages') have [simp]: "nodeState' = nodeState" unfolding nodeState'_def by (intro ext, simp add: nd' nd_def) from zen_axioms show ?thesis by simp next case isQuorum: True hence result: "result = (nd, Some (ApplyCommit (firstUncommittedSlot nd) (currentTerm nd)))" by (simp add: result_def commitIfQuorate_def) have nodeState': "nodeState' = nodeState" unfolding nodeState'_def by (intro ext, simp add: nd' nd_def result) have messages': "messages' = insert \ sender = n\<^sub>0, destination = Broadcast, payload = ApplyCommit (firstUncommittedSlot nd) (currentTerm nd) \ messages" by (simp add: messages' result) have message_simps[simp]: "\s d i t. (s \\ PublishResponse i t \\' d) = (s \\ PublishResponse i t \\ d)" "\s d i t x. (s \\ PublishRequest i t x \\' d) = (s \\ PublishRequest i t x \\ d)" "\s d i t a. (s \\ Vote i t a \\' d) = (s \\ Vote i t a \\ d)" "\d i t. (\ PublishResponse i t \\' d) = (\ PublishResponse i t \\ d)" "\d i t x. (\ PublishRequest i t x \\' d) = (\ PublishRequest i t x \\ d)" "\d i t a. (\ Vote i t a \\' d) = (\ Vote i t a \\ d)" "\s i t. (s \\ PublishResponse i t \\') = (s \\ PublishResponse i t \\)" "\s i t x. (s \\ PublishRequest i t x \\') = (s \\ PublishRequest i t x \\)" "\s i t a. (s \\ Vote i t a \\') = (s \\ Vote i t a \\)" "\i t. (\ PublishResponse i t \\') = (\ PublishResponse i t \\)" "\i t x. (\ PublishRequest i t x \\') = (\ PublishRequest i t x \\)" "\i t a. (\ Vote i t a \\') = (\ Vote i t a \\)" "\s d i conf cs. (s \\ CatchUpResponse i conf cs \\' d) = (s \\ CatchUpResponse i conf cs \\ d)" "\d i conf cs. (\ CatchUpResponse i conf cs \\' d) = (\ CatchUpResponse i conf cs \\ d)" "\s i conf cs. (s \\ CatchUpResponse i conf cs \\') = (s \\ CatchUpResponse i conf cs \\)" "\d i conf cs. (\ CatchUpResponse i conf cs \\') = (\ CatchUpResponse i conf cs \\)" by (unfold isMessageFromTo'_def isMessageTo'_def isMessageFrom'_def isMessage'_def, auto simp add: messages' isMessageFromTo_def isMessageTo_def isMessageFrom_def isMessage_def) have ApplyCommit': "\s d i t. (s \\ ApplyCommit i t \\' d) = ((s \\ ApplyCommit i t \\ d) \ (s, i, t, d) = (n\<^sub>0, firstUncommittedSlot nd, currentTerm nd, Broadcast))" "\s i t. (s \\ ApplyCommit i t \\') = ((s \\ ApplyCommit i t \\) \ (s, i, t) = (n\<^sub>0, firstUncommittedSlot nd, currentTerm nd))" "\d i t. (\ ApplyCommit i t \\' d) = ((\ ApplyCommit i t \\ d) \ (i, t, d) = (firstUncommittedSlot nd, currentTerm nd, Broadcast))" "\i t. (\ ApplyCommit i t \\') = ((\ ApplyCommit i t \\) \ (i, t) = (firstUncommittedSlot nd, currentTerm nd))" unfolding isMessageFromTo_def isMessageFromTo'_def isMessageFrom_def isMessageFrom'_def isMessageTo_def isMessageTo'_def isMessage_def isMessage'_def by (auto simp add: messages') from committedTo_firstUncommittedSlot have committedTo_current: "committed\<^sub>< (firstUncommittedSlot nd)" by (simp add: nd_def) have isCommitted_eq: "\i. isCommitted' i = (isCommitted i \ i = firstUncommittedSlot nd)" using isCommitted'_def isCommitted_def by (auto simp add: ApplyCommit') have committedTo_eq: "\i. committed\<^sub><' i = ((committed\<^sub>< i) \ (i = Suc (firstUncommittedSlot nd)))" proof - fix i show "?thesis i" proof (cases "isCommitted (firstUncommittedSlot nd)") case True with isCommitted_eq have 1: "isCommitted' = isCommitted" by (intro ext, auto) from True isCommitted_committedTo_Suc have 2: "committed\<^sub>< (Suc (firstUncommittedSlot nd))" by simp from 1 2 show ?thesis by (simp add: committedTo'_def committedTo_def, blast) next case False with committedTo_current isCommitted_committedTo have isCommitted_lt[simp]: "\j. isCommitted j = (j < firstUncommittedSlot nd)" using committedTo_def nat_neq_iff by blast have isCommitted'_le[simp]: "\j. isCommitted' j = (j \ firstUncommittedSlot nd)" by (auto simp add: isCommitted_eq) show ?thesis by (simp add: committedTo'_def committedTo_def, auto, presburger) qed qed have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def) note oneSlot.consistent [OF oneSlot.commit [OF zen_is_oneSlot]] have v\<^sub>c_eq: "\i. isCommitted i \ v\<^sub>c' i = v\<^sub>c i" proof - fix i assume "isCommitted i" then obtain t where t: "\ ApplyCommit i t \\" unfolding isCommitted_def by blast show "?thesis i" proof (cases "i = firstUncommittedSlot nd") case False thus ?thesis unfolding v\<^sub>c_def v\<^sub>c'_def v_eq ApplyCommit' by simp next case i: True show "v\<^sub>c' i = v\<^sub>c i" unfolding v\<^sub>c_def v\<^sub>c'_def v_eq proof (intro oneSlot.consistent [OF oneSlot.commit [OF zen_is_oneSlot]]) from isQuorum show "publishVotes nd \ majorities (V i)" unfolding nd_def i isQuorum_def using currentVotingNodes_firstUncommittedSlot by simp from publishVotes show "\n. n \ publishVotes nd \ n \\ PublishResponse i (currentTerm nd) \\" unfolding nd_def i . from t have "\ ApplyCommit i (SOME t. \ ApplyCommit i t \\) \\" by (intro someI) thus "\ ApplyCommit i (SOME t. \ ApplyCommit i t \\) \\ \ (SOME t. \ ApplyCommit i t \\) = currentTerm nd" by simp show "\ ApplyCommit i (SOME t. \ ApplyCommit i t \\') \\ \ (SOME t. \ ApplyCommit i t \\') = currentTerm nd" proof (rule someI2) from t show "\ ApplyCommit i t \\'" "\x. \ ApplyCommit i x \\' \ \ ApplyCommit i x \\ \ x = currentTerm nd" by (simp_all add: ApplyCommit' i) qed qed qed qed have V_eq: "\i. committed\<^sub>< i \ V' i = V i" proof - fix i assume i: "committed\<^sub>< i" thus "?thesis i" proof (induct i) case (Suc i') hence prems: "committed\<^sub>< i'" "isCommitted i'" unfolding committedTo_def by auto thus ?case using Suc v\<^sub>c_eq by simp qed simp qed hence V_era_eq: "\n. V' (firstUncommittedSlot (nodeState n)) = V (firstUncommittedSlot (nodeState n))" using committedTo_firstUncommittedSlot by blast have lastCommittedClusterStateBefore_eq: "\i. committed\<^sub>< i \ lastCommittedClusterStateBefore' i = lastCommittedClusterStateBefore i" proof - fix i assume "committed\<^sub>< i" thus "?thesis i" proof (induct i) case (Suc i') hence prems: "committed\<^sub>< i'" "isCommitted i'" unfolding committedTo_def by auto thus ?case using Suc v\<^sub>c_eq by (cases "v\<^sub>c i'", simp_all) qed simp qed hence lastCommittedClusterStateBefore_slot_eq: "\n. lastCommittedClusterStateBefore' (firstUncommittedSlot (nodeState n)) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))" using committedTo_firstUncommittedSlot by blast have promised_eq[simp]: "promised' = promised" by (intro ext, auto simp add: promised'_def promised_def) have prevAccepted_eq[simp]: "prevAccepted' = prevAccepted" by (intro ext, auto simp add: prevAccepted'_def prevAccepted_def) show ?thesis apply (intro zenI) apply (unfold nodeState' message_simps V_era_eq lastCommittedClusterStateBefore_slot_eq promised_eq prevAccepted_eq v_eq) proof - from finite_messages show "finite messages'" by (simp add: messages') from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from firstUncommittedSlot_PublishResponse show "\n i t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\" . from lastAcceptedTerm_None show "\n i t. lastAcceptedTerm (nodeState n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_sent show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_max show "\n t t'. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\ \ t' \ t". from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from electionWon_isQuorum show "\n. electionWon (nodeState n) \ isQuorum (nodeState n) (joinVotes (nodeState n))". from joinVotes_max show "\n n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) x \\) \ n' \ joinVotes (nodeState n) \ n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState n)". from publishVotes show "\n n'. n' \ publishVotes (nodeState n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) \\". from currentClusterState_lastCommittedClusterStateBefore show "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))". from joinVotes show "\n n'. n' \ joinVotes (nodeState n) \ promised (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState n))". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))". from firstUncommittedSlot_PublishRequest show "\i n t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\". from lastAcceptedTerm_Some_value show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState n)) \\". from PublishRequest_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ t \ currentTerm (nodeState n)". from PublishRequest_publishPermitted_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ publishPermitted (nodeState n) \ t < currentTerm (nodeState n)". from lastAcceptedTerm_Some_currentTerm show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ t \ currentTerm (nodeState n)". { fix i conf cs assume a: "\ CatchUpResponse i conf cs \\" with CatchUpResponse_committedTo have committedTo: "committed\<^sub>< i" . thus "committed\<^sub><' i" unfolding committedTo_eq by simp from a committedTo V_eq lastCommittedClusterStateBefore_eq CatchUpResponse_V CatchUpResponse_lastCommittedClusterStateBefore show "V' i = conf" "lastCommittedClusterStateBefore' i = cs" by auto } from committedTo_firstUncommittedSlot show "\n. committed\<^sub><' (firstUncommittedSlot (nodeState n))" unfolding committedTo_eq by simp from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub><' i" unfolding committedTo_eq by simp from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V' i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))" using V_eq PublishRequest_committedTo isMessage_def by metis from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\' \ \q\majorities (V' i). \s\q. s \\ PublishResponse i t \\" using V_eq committedTo_firstUncommittedSlot unfolding ApplyCommit' committedTo_def isCommitted_def by (metis ApplyCommit_PublishRequest_v\<^sub>c PublishRequest_committedTo V_eq isQuorum currentVotingNodes_firstUncommittedSlot nd_def prod.inject publishVotes isQuorum_def) qed qed lemma (in zenStep) handlePublishResponse_invariants: fixes s i t defines "result \ handlePublishResponse s i t nd" assumes nd': "nd' = fst result" assumes messages': "messages' = sendTo Broadcast result" assumes sent: "s \\ PublishResponse i t \\" shows "zen messages' nodeState'" proof (cases "i = firstUncommittedSlot nd \ t = currentTerm nd") case False hence [simp]: "result = (nd, None)" by (auto simp add: result_def handlePublishResponse_def Let_def) have [simp]: "messages' = messages" by (simp add: messages') have [simp]: "nodeState' = nodeState" unfolding nodeState'_def by (intro ext, simp add: nd' nd_def) from zen_axioms show ?thesis by simp next case True hence i: "i = firstUncommittedSlot nd" and t: "t = currentTerm nd" by simp_all define nd1 where "nd1 \ nd \publishVotes := insert s (publishVotes nd)\" define nodeState1 where "nodeState1 \ \n. if n = n\<^sub>0 then nd1 else nodeState n" have result: "result = commitIfQuorate nd1" by (simp add: result_def handlePublishResponse_def i t nd1_def) have zenStep1: "zenStep messages nodeState messages nodeState1 n\<^sub>0" by (intro zenStepI2, auto simp add: nodeState1_def) have zenStep2: "zenStep messages nodeState1 messages' nodeState' n\<^sub>0" proof (intro zenStep.zenStepI1 [OF zenStep1] zenStep.addPublishVote_invariants [OF zenStep1] refl messages_subset, fold isMessageFromTo_def isMessageFrom_def nd_def) show "nodeState1 n\<^sub>0 = nd\publishVotes := insert s (publishVotes nd)\" by (simp add: nodeState1_def nd1_def) from sent show "s \\ PublishResponse (firstUncommittedSlot nd) (currentTerm nd) \\" by (simp add: True) show "\n. n \ n\<^sub>0 \ nodeState1 n = nodeState' n" unfolding nodeState1_def nodeState'_def by simp qed show ?thesis proof (intro zenStep.commitIfQuorate_invariants [OF zenStep2]) show "nodeState' n\<^sub>0 = fst (commitIfQuorate (nodeState1 n\<^sub>0))" by (simp add: nodeState'_def nd' result nodeState1_def) show "messages' = (case snd (commitIfQuorate (nodeState1 n\<^sub>0)) of None \ messages | Some m \ insert \sender = n\<^sub>0, destination = Broadcast, payload = m\ messages)" by (cases "commitIfQuorate (nodeState1 n\<^sub>0)", cases "snd (commitIfQuorate (nodeState1 n\<^sub>0))", simp_all add: nodeState1_def messages' result_def handlePublishResponse_def i t nd1_def) qed qed lemma (in zenStep) handleApplyCommit_invariants: assumes nd': "nd' = handleApplyCommit i t nd" assumes messages'[simp]: "messages' = messages" assumes sent: "\ ApplyCommit i t \\" shows "zen messages' nodeState'" proof (cases "i = firstUncommittedSlot nd \ lastAcceptedTerm nd = SomeTerm t") case False hence nd'[simp]: "nd' = nd" by (auto simp add: nd' handleApplyCommit_def) have nodeState'[simp]: "nodeState' = nodeState" unfolding nodeState'_def by (intro ext, simp add: nd_def) from zen_axioms show ?thesis unfolding nodeState' by simp next case True hence i: "i = firstUncommittedSlot nd" and t: "lastAcceptedTerm nd = SomeTerm t" by simp_all have message_simps[simp]: "\s p d. (s \\ p \\' d) = (s \\ p \\ d)" "\p d. (\ p \\' d) = (\ p \\ d)" "\s p. (s \\ p \\') = (s \\ p \\)" "\p. (\ p \\') = (\ p \\)" by (unfold isMessageFromTo'_def isMessageTo'_def isMessageFrom'_def isMessage'_def, auto simp add: isMessageFromTo_def isMessageTo_def isMessageFrom_def isMessage_def) have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def) have v\<^sub>c_eq[simp]: "v\<^sub>c' = v\<^sub>c" by (intro ext, auto simp add: v\<^sub>c'_def v\<^sub>c_def) have isCommitted_eq[simp]: "isCommitted' = isCommitted" by (intro ext, auto simp add: isCommitted'_def isCommitted_def) have committedTo_eq[simp]: "committed\<^sub><' = committed\<^sub><" by (intro ext, auto simp add: committedTo'_def committedTo_def) have V_eq[simp]: "V' = V" using v\<^sub>c_eq V'_def V_def by blast have promised_eq[simp]: "promised' = promised" by (intro ext, auto simp add: promised'_def promised_def) have lastCommittedClusterStateBefore_eq[simp]: "lastCommittedClusterStateBefore' = lastCommittedClusterStateBefore" unfolding lastCommittedClusterStateBefore_def lastCommittedClusterStateBefore'_def v\<^sub>c_eq .. have prevAccepted_eq[simp]: "prevAccepted' = prevAccepted" by (intro ext, auto simp add: prevAccepted'_def prevAccepted_def) from sent t have lastAcceptedValue_eq: "v\<^sub>c i = lastAcceptedValue nd" unfolding i nd_def using lastAcceptedTerm_Some_value [of n\<^sub>0 t] by (intro PublishRequest_function [OF ApplyCommit_PublishRequest_v\<^sub>c]) show ?thesis proof (cases "isReconfiguration (v\<^sub>c i)") case False have "\n. currentNode (nodeState' n) = currentNode (nodeState n) \ currentTerm (nodeState' n) = currentTerm (nodeState n) \ currentVotingNodes (nodeState' n) = currentVotingNodes (nodeState n) \ joinVotes (nodeState' n) = joinVotes (nodeState n) \ electionWon (nodeState' n) = electionWon (nodeState n) \ isQuorum (nodeState' n) = isQuorum (nodeState n)" proof (cases "lastAcceptedValue nd") case NoOp thus "\n. ?thesis n" unfolding nodeState'_def nd' handleApplyCommit_def i t applyAcceptedValue_def isQuorum_def nd_def by simp next case ClusterStateDiff thus "\n. ?thesis n" unfolding nodeState'_def nd' handleApplyCommit_def i t applyAcceptedValue_def isQuorum_def nd_def by simp next case Reconfigure with False lastAcceptedValue_eq show "\n. ?thesis n" by simp qed hence property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. currentTerm (nodeState' n) = currentTerm (nodeState n)" "\n. currentVotingNodes (nodeState' n) = currentVotingNodes (nodeState n)" "\n. joinVotes (nodeState' n) = joinVotes (nodeState n)" "\n. electionWon (nodeState' n) = electionWon (nodeState n)" "\n. isQuorum (nodeState' n) = isQuorum (nodeState n)" by simp_all have "\n. firstUncommittedSlot (nodeState' n) = (if n = n\<^sub>0 then Suc (firstUncommittedSlot (nodeState n)) else firstUncommittedSlot (nodeState n)) \ publishPermitted (nodeState' n) = (publishPermitted (nodeState n) \ n = n\<^sub>0) \ publishVotes (nodeState' n) = (if n = n\<^sub>0 then {} else publishVotes (nodeState n)) \ lastAcceptedTerm (nodeState' n) = (if n = n\<^sub>0 then NO_TERM else lastAcceptedTerm (nodeState n))" proof (cases "lastAcceptedValue nd") case NoOp with i t show "\n. ?thesis n" by (unfold nodeState'_def, auto simp add: lastAcceptedValue_eq nd_def nd' applyAcceptedValue_def isQuorum_def handleApplyCommit_def lastAcceptedTerm_def) next case ClusterStateDiff with i t show "\n. ?thesis n" by (unfold nodeState'_def, auto simp add: lastAcceptedValue_eq nd_def nd' applyAcceptedValue_def isQuorum_def handleApplyCommit_def lastAcceptedTerm_def) next case Reconfigure with False lastAcceptedValue_eq show "\n. ?thesis n" by simp qed hence updated_properties: "\n. firstUncommittedSlot (nodeState' n) = (if n = n\<^sub>0 then Suc (firstUncommittedSlot (nodeState n)) else firstUncommittedSlot (nodeState n)) " "\n. publishPermitted (nodeState' n) = (publishPermitted (nodeState n) \ n = n\<^sub>0)" "\n. publishVotes (nodeState' n) = (if n = n\<^sub>0 then {} else publishVotes (nodeState n))" "\n. lastAcceptedTerm (nodeState' n) = (if n = n\<^sub>0 then NO_TERM else lastAcceptedTerm (nodeState n))" by simp_all show ?thesis apply (intro zenI) apply (unfold messages' message_simps committedTo_eq V_eq v_eq lastCommittedClusterStateBefore_eq property_simps promised_eq prevAccepted_eq) proof - from finite_messages show "finite messages" . from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from electionWon_isQuorum show "\n. electionWon (nodeState n) \ isQuorum (nodeState n) (joinVotes (nodeState n))". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\". from CatchUpResponse_committedTo show "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i". from CatchUpResponse_V show "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf". from CatchUpResponse_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs". fix n from firstUncommittedSlot_PublishResponse show "\i t. firstUncommittedSlot (nodeState' n) < i \ \ n \\ PublishResponse i t \\" using updated_properties by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_None show "\i t. lastAcceptedTerm (nodeState' n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState' n)) t \\" using updated_properties firstUncommittedSlot_PublishResponse by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_Some_sent show "\t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState' n)) t \\" using updated_properties by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_Some_max show "\t t'. lastAcceptedTerm (nodeState' n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState' n)) t' \\ \ t' \ t" using updated_properties by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_Some_value show "\t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState' n)) t (lastAcceptedValue (nodeState' n)) \\" using updated_properties by (cases "n = n\<^sub>0", auto, simp add: nodeState'_def) from lastAcceptedTerm_Some_currentTerm show "\t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ t \ currentTerm (nodeState n)" using updated_properties by (cases "n = n\<^sub>0", auto) from committedTo_firstUncommittedSlot show "committed\<^sub>< (firstUncommittedSlot (nodeState' n))" unfolding updated_properties committedTo_def apply (cases "n = n\<^sub>0", auto) using i isCommitted_def less_antisym nd_def sent by blast from joinVotes_max show "\n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState' n)) (currentTerm (nodeState n)) x \\) \ n' \ joinVotes (nodeState n) \ n' \\ Vote (firstUncommittedSlot (nodeState' n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState' n)" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) using isMessageFromTo_def zen.Vote_slot_function zen.joinVotes zen_axioms by fastforce from publishVotes show "\n'. n' \ publishVotes (nodeState' n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState' n)) (currentTerm (nodeState n)) \\" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) done from joinVotes show "\n'. n' \ joinVotes (nodeState n) \ promised (firstUncommittedSlot (nodeState' n)) n' n (currentTerm (nodeState n))" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) by (meson le_SucI promised_def) from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState' n))" unfolding updated_properties using False i nd_def by (cases "n = n\<^sub>0", auto) from firstUncommittedSlot_PublishRequest show "\i t x. firstUncommittedSlot (nodeState' n) < i \ \ n \\ PublishRequest i t x \\" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) done from PublishRequest_currentTerm show "\t x. n \\ PublishRequest (firstUncommittedSlot (nodeState' n)) t x \\ \ t \ currentTerm (nodeState n)" unfolding updated_properties apply (cases "n = n\<^sub>0", auto simp add: firstUncommittedSlot_PublishRequest) done from PublishRequest_publishPermitted_currentTerm show "\t x. n \\ PublishRequest (firstUncommittedSlot (nodeState' n)) t x \\ \ publishPermitted (nodeState' n) \ t < currentTerm (nodeState n)" unfolding updated_properties apply (cases "n = n\<^sub>0", auto simp add: firstUncommittedSlot_PublishRequest) done from currentClusterState_lastCommittedClusterStateBefore show "currentClusterState (nodeState' n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState' n))" proof (cases "n = n\<^sub>0") case False with currentClusterState_lastCommittedClusterStateBefore show ?thesis unfolding nodeState'_def by auto next case True show ?thesis proof (cases "v\<^sub>c i") case NoOp with currentClusterState_lastCommittedClusterStateBefore True lastAcceptedValue_eq i t show ?thesis unfolding nodeState'_def by (simp add: nd' nd_def applyAcceptedValue_def handleApplyCommit_def) next case Reconfigure with False show ?thesis by simp next case (ClusterStateDiff diff) with lastAcceptedValue_eq have "lastAcceptedValue nd = ClusterStateDiff diff" by simp with ClusterStateDiff True i t currentClusterState_lastCommittedClusterStateBefore show ?thesis unfolding nodeState'_def by (simp add: nd' nd_def applyAcceptedValue_def handleApplyCommit_def) qed qed qed next case True then obtain newConfig where Reconfigure: "v\<^sub>c i = Reconfigure newConfig" by (cases "v\<^sub>c i", auto) with lastAcceptedValue_eq have lastAcceptedValue_eq: "lastAcceptedValue nd = Reconfigure newConfig" by simp hence property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. currentTerm (nodeState' n) = currentTerm (nodeState n)" "\n. currentClusterState (nodeState' n) = currentClusterState (nodeState n)" "\n. joinVotes (nodeState' n) = joinVotes (nodeState n)" unfolding nodeState'_def by (auto simp add: nd' handleApplyCommit_def applyAcceptedValue_def nd_def lastAcceptedTerm_def firstUncommittedSlot_def lastAcceptedValue_def) have updated_properties: "\n. firstUncommittedSlot (nodeState' n) = (if n = n\<^sub>0 then Suc (firstUncommittedSlot (nodeState n)) else firstUncommittedSlot (nodeState n)) " "\n. publishPermitted (nodeState' n) = (publishPermitted (nodeState n) \ n = n\<^sub>0)" "\n. publishVotes (nodeState' n) = (if n = n\<^sub>0 then {} else publishVotes (nodeState n))" "\n. currentVotingNodes (nodeState' n) = (if n = n\<^sub>0 then set newConfig else currentVotingNodes (nodeState n))" "\n. electionWon (nodeState' n) = (if n = n\<^sub>0 then joinVotes nd \ majorities (set newConfig) else electionWon (nodeState n))" "\n. isQuorum (nodeState' n) = (if n = n\<^sub>0 then (\q. q \ majorities (set newConfig)) else isQuorum (nodeState n))" "\n. currentVotingNodes (nodeState' n) = (if n = n\<^sub>0 then set newConfig else currentVotingNodes (nodeState n))" "\n. lastAcceptedTerm (nodeState' n) = (if n = n\<^sub>0 then NO_TERM else lastAcceptedTerm (nodeState n))" "\n. n \ n\<^sub>0 \ lastAcceptedValue (nodeState' n) = lastAcceptedValue (nodeState n)" unfolding nodeState'_def using i t lastAcceptedValue_eq by (auto simp add: nd' handleApplyCommit_def applyAcceptedValue_def nd_def isQuorum_def lastAcceptedTerm_def) show ?thesis apply (intro zenI) apply (unfold messages' message_simps committedTo_eq V_eq v_eq lastCommittedClusterStateBefore_eq property_simps promised_eq prevAccepted_eq) proof - from finite_messages show "finite messages" . from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\". from CatchUpResponse_committedTo show "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i". from CatchUpResponse_V show "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf". from CatchUpResponse_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs". fix n from firstUncommittedSlot_PublishResponse show "\i t. firstUncommittedSlot (nodeState' n) < i \ \ n \\ PublishResponse i t \\" using updated_properties by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_None show "\i t. lastAcceptedTerm (nodeState' n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState' n)) t \\" using updated_properties firstUncommittedSlot_PublishResponse by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_Some_sent show "\t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState' n)) t \\" using updated_properties by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_Some_max show "\t t'. lastAcceptedTerm (nodeState' n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState' n)) t' \\ \ t' \ t" using updated_properties by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_Some_value show "\t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState' n)) t (lastAcceptedValue (nodeState' n)) \\" using updated_properties by (cases "n = n\<^sub>0", auto) from committedTo_firstUncommittedSlot show "committed\<^sub>< (firstUncommittedSlot (nodeState' n))" unfolding updated_properties committedTo_def apply (cases "n = n\<^sub>0", auto) using i isCommitted_def less_antisym nd_def sent by blast from electionWon_isQuorum show "\n. electionWon (nodeState' n) \ isQuorum (nodeState' n) (joinVotes (nodeState n))" unfolding updated_properties apply (cases "n = n\<^sub>0", auto simp add: nd_def) done from joinVotes_max show "\n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState' n)) (currentTerm (nodeState n)) x \\) \ n' \ joinVotes (nodeState n) \ n' \\ Vote (firstUncommittedSlot (nodeState' n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState' n)" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) using isMessageFromTo_def zen.Vote_slot_function zen.joinVotes zen_axioms by fastforce from publishVotes show "\n'. n' \ publishVotes (nodeState' n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState' n)) (currentTerm (nodeState n)) \\" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) done from joinVotes show "\n'. n' \ joinVotes (nodeState n) \ promised (firstUncommittedSlot (nodeState' n)) n' n (currentTerm (nodeState n))" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) by (meson le_SucI promised_def) from firstUncommittedSlot_PublishRequest show "\i t x. firstUncommittedSlot (nodeState' n) < i \ \ n \\ PublishRequest i t x \\" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) done from PublishRequest_currentTerm show "\t x. n \\ PublishRequest (firstUncommittedSlot (nodeState' n)) t x \\ \ t \ currentTerm (nodeState n)" unfolding updated_properties apply (cases "n = n\<^sub>0", auto simp add: firstUncommittedSlot_PublishRequest) done from PublishRequest_publishPermitted_currentTerm show "\t x. n \\ PublishRequest (firstUncommittedSlot (nodeState' n)) t x \\ \ publishPermitted (nodeState' n) \ t < currentTerm (nodeState n)" unfolding updated_properties apply (cases "n = n\<^sub>0", auto simp add: firstUncommittedSlot_PublishRequest) done from currentClusterState_lastCommittedClusterStateBefore Reconfigure i nd_def show "currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState' n))" unfolding updated_properties committedTo_def by (cases "n = n\<^sub>0", auto) from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState' n) = V (firstUncommittedSlot (nodeState' n))" unfolding updated_properties committedTo_def using Reconfigure i nd_def by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_Some_currentTerm show "\t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ t \ currentTerm (nodeState n)" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) done qed qed qed lemma (in zenStep) handleCatchUpRequest_invariants: fixes s i t defines "result \ handleCatchUpRequest nd" assumes nd': "nd' = fst result" assumes messages': "messages' = sendTo d\<^sub>0 result" shows "zen messages' nodeState'" proof - have result: "result = (nd, Some (CatchUpResponse (firstUncommittedSlot nd) (currentVotingNodes nd) (currentClusterState nd)))" by (simp add: result_def handleCatchUpRequest_def) have nd'[simp]: "nd' = nd" by (simp add: nd' result) have nodeState'[simp]: "nodeState' = nodeState" by (intro ext, simp add: nodeState'_def result nd_def) have messages': "messages' = insert \ sender = n\<^sub>0, destination = d\<^sub>0, payload = CatchUpResponse (firstUncommittedSlot nd) (currentVotingNodes nd) (currentClusterState nd) \ messages" by (simp add: messages' result) have message_simps[simp]: "\s d i t. (s \\ PublishResponse i t \\' d) = (s \\ PublishResponse i t \\ d)" "\s d i t. (s \\ ApplyCommit i t \\' d) = (s \\ ApplyCommit i t \\ d)" "\s d i t x. (s \\ PublishRequest i t x \\' d) = (s \\ PublishRequest i t x \\ d)" "\s d i t a. (s \\ Vote i t a \\' d) = (s \\ Vote i t a \\ d)" "\d i t. (\ PublishResponse i t \\' d) = (\ PublishResponse i t \\ d)" "\d i t. (\ ApplyCommit i t \\' d) = (\ ApplyCommit i t \\ d)" "\d i t x. (\ PublishRequest i t x \\' d) = (\ PublishRequest i t x \\ d)" "\d i t a. (\ Vote i t a \\' d) = (\ Vote i t a \\ d)" "\s i t. (s \\ PublishResponse i t \\') = (s \\ PublishResponse i t \\)" "\s i t. (s \\ ApplyCommit i t \\') = (s \\ ApplyCommit i t \\)" "\s i t x. (s \\ PublishRequest i t x \\') = (s \\ PublishRequest i t x \\)" "\s i t a. (s \\ Vote i t a \\') = (s \\ Vote i t a \\)" "\i t. (\ PublishResponse i t \\') = (\ PublishResponse i t \\)" "\i t. (\ ApplyCommit i t \\') = (\ ApplyCommit i t \\)" "\i t x. (\ PublishRequest i t x \\') = (\ PublishRequest i t x \\)" "\i t a. (\ Vote i t a \\') = (\ Vote i t a \\)" by (unfold isMessageFromTo'_def isMessageTo'_def isMessageFrom'_def isMessage'_def, auto simp add: messages' isMessageFromTo_def isMessageTo_def isMessageFrom_def isMessage_def) have CatchUpResponse': "\s d i conf cs. (s \\ CatchUpResponse i conf cs \\' d) = (s \\ CatchUpResponse i conf cs \\ d \ (s, i, conf, cs, d) = (n\<^sub>0, firstUncommittedSlot nd, currentVotingNodes nd, currentClusterState nd, d\<^sub>0))" "\d i conf cs. (\ CatchUpResponse i conf cs \\' d) = (\ CatchUpResponse i conf cs \\ d \ (i, conf, cs, d) = (firstUncommittedSlot nd, currentVotingNodes nd, currentClusterState nd, d\<^sub>0))" "\s i conf cs. (s \\ CatchUpResponse i conf cs \\') = (s \\ CatchUpResponse i conf cs \\ \ (s, i, conf, cs) = (n\<^sub>0, firstUncommittedSlot nd, currentVotingNodes nd, currentClusterState nd))" "\i conf cs. (\ CatchUpResponse i conf cs \\') = (\ CatchUpResponse i conf cs \\ \ (i, conf, cs) = (firstUncommittedSlot nd, currentVotingNodes nd, currentClusterState nd))" unfolding isMessageFromTo_def isMessageFromTo'_def isMessageFrom_def isMessageFrom'_def isMessageTo_def isMessageTo'_def isMessage_def isMessage'_def by (auto simp add: messages') have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def) have v\<^sub>c_eq[simp]: "v\<^sub>c' = v\<^sub>c" by (intro ext, auto simp add: v\<^sub>c'_def v\<^sub>c_def) have isCommitted_eq[simp]: "isCommitted' = isCommitted" by (intro ext, auto simp add: isCommitted'_def isCommitted_def) have committedTo_eq[simp]: "committed\<^sub><' = committed\<^sub><" by (intro ext, auto simp add: committedTo'_def committedTo_def) have V_eq[simp]: "V' = V" using v\<^sub>c_eq V'_def V_def by blast have promised_eq[simp]: "promised' = promised" by (intro ext, auto simp add: promised'_def promised_def) have lastCommittedClusterStateBefore_eq[simp]: "lastCommittedClusterStateBefore' = lastCommittedClusterStateBefore" unfolding lastCommittedClusterStateBefore_def lastCommittedClusterStateBefore'_def v\<^sub>c_eq .. have prevAccepted_eq[simp]: "prevAccepted' = prevAccepted" by (intro ext, auto simp add: prevAccepted'_def prevAccepted_def) show ?thesis apply (intro zenI) apply (unfold message_simps committedTo_eq V_eq v_eq lastCommittedClusterStateBefore_eq promised_eq prevAccepted_eq nodeState') proof - from finite_messages show "finite messages'" by (simp add: messages') from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from committedTo_firstUncommittedSlot show "\n. committed\<^sub>< (firstUncommittedSlot (nodeState n))" . from firstUncommittedSlot_PublishResponse show "\n i t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\" . from lastAcceptedTerm_None show "\n i t. lastAcceptedTerm (nodeState n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_sent show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_max show "\n t t'. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\ \ t' \ t". from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from electionWon_isQuorum show "\n. electionWon (nodeState n) \ isQuorum (nodeState n) (joinVotes (nodeState n))". from joinVotes_max show "\n n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) x \\) \ n' \ joinVotes (nodeState n) \ n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState n)". from publishVotes show "\n n'. n' \ publishVotes (nodeState n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) \\". from currentClusterState_lastCommittedClusterStateBefore show "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))". from joinVotes show "\n n'. n' \ joinVotes (nodeState n) \ promised (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState n))". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\". from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))". from firstUncommittedSlot_PublishRequest show "\i n t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\". from lastAcceptedTerm_Some_value show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState n)) \\". from PublishRequest_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ t \ currentTerm (nodeState n)". from PublishRequest_publishPermitted_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ publishPermitted (nodeState n) \ t < currentTerm (nodeState n)". from lastAcceptedTerm_Some_currentTerm show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ t \ currentTerm (nodeState n)". from CatchUpResponse_committedTo committedTo_firstUncommittedSlot show "\i conf cs. \ CatchUpResponse i conf cs \\' \ committed\<^sub>< i" unfolding CatchUpResponse' nd_def by blast from CatchUpResponse_V currentVotingNodes_firstUncommittedSlot show "\i conf cs. \ CatchUpResponse i conf cs \\' \ V i = conf" unfolding CatchUpResponse' nd_def isQuorum_def by auto from CatchUpResponse_lastCommittedClusterStateBefore currentClusterState_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\' \ lastCommittedClusterStateBefore i = cs" unfolding CatchUpResponse' nd_def by auto qed qed lemma (in zenStep) handleCatchUpResponse_invariants: assumes nd': "nd' = handleCatchUpResponse i conf cs nd" assumes messages'[simp]: "messages' = messages" assumes sent: "\ CatchUpResponse i conf cs \\" shows "zen messages' nodeState'" proof (cases "firstUncommittedSlot nd < i") case False hence nd'[simp]: "nd' = nd" by (auto simp add: nd' handleCatchUpResponse_def) have nodeState'[simp]: "nodeState' = nodeState" unfolding nodeState'_def by (intro ext, simp add: nd_def) from zen_axioms show ?thesis unfolding nodeState' by simp next case True hence nd': "nd' = nd \ firstUncommittedSlot := i , publishPermitted := True , publishVotes := {} , currentVotingNodes := conf , currentClusterState := cs , joinVotes := {} , electionWon := False , lastAcceptedData := None \" by (simp add: nd', simp add: handleCatchUpResponse_def) have message_simps[simp]: "\s p d. (s \\ p \\' d) = (s \\ p \\ d)" "\p d. (\ p \\' d) = (\ p \\ d)" "\s p. (s \\ p \\') = (s \\ p \\)" "\p. (\ p \\') = (\ p \\)" by (unfold isMessageFromTo'_def isMessageTo'_def isMessageFrom'_def isMessage'_def, auto simp add: isMessageFromTo_def isMessageTo_def isMessageFrom_def isMessage_def) have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def) have v\<^sub>c_eq[simp]: "v\<^sub>c' = v\<^sub>c" by (intro ext, auto simp add: v\<^sub>c'_def v\<^sub>c_def) have isCommitted_eq[simp]: "isCommitted' = isCommitted" by (intro ext, auto simp add: isCommitted'_def isCommitted_def) have committedTo_eq[simp]: "committed\<^sub><' = committed\<^sub><" by (intro ext, auto simp add: committedTo'_def committedTo_def) have V_eq[simp]: "V' = V" using v\<^sub>c_eq V'_def V_def by blast have promised_eq[simp]: "promised' = promised" by (intro ext, auto simp add: promised'_def promised_def) have lastCommittedClusterStateBefore_eq[simp]: "lastCommittedClusterStateBefore' = lastCommittedClusterStateBefore" unfolding lastCommittedClusterStateBefore_def lastCommittedClusterStateBefore'_def v\<^sub>c_eq .. have prevAccepted_eq[simp]: "prevAccepted' = prevAccepted" by (intro ext, auto simp add: prevAccepted'_def prevAccepted_def) have property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. currentTerm (nodeState' n) = currentTerm (nodeState n)" unfolding nodeState'_def by (auto simp add: nd' nd_def lastAcceptedTerm_def firstUncommittedSlot_def lastAcceptedValue_def) have updated_properties: "\n. firstUncommittedSlot (nodeState' n) = (if n = n\<^sub>0 then i else firstUncommittedSlot (nodeState n)) " "\n. publishPermitted (nodeState' n) = (publishPermitted (nodeState n) \ n = n\<^sub>0)" "\n. publishVotes (nodeState' n) = (if n = n\<^sub>0 then {} else publishVotes (nodeState n))" "\n. currentVotingNodes (nodeState' n) = (if n = n\<^sub>0 then conf else currentVotingNodes (nodeState n))" "\n. joinVotes (nodeState' n) = (if n = n\<^sub>0 then {} else joinVotes (nodeState n))" "\n. electionWon (nodeState' n) = (electionWon (nodeState n) \ n \ n\<^sub>0)" "\n. currentVotingNodes (nodeState' n) = (if n = n\<^sub>0 then conf else currentVotingNodes (nodeState n))" "\n. isQuorum (nodeState' n) = (if n = n\<^sub>0 then (\q. q \ majorities conf) else isQuorum (nodeState n))" "\n. currentClusterState (nodeState' n) = (if n = n\<^sub>0 then cs else currentClusterState (nodeState n))" unfolding nodeState'_def by (auto simp add: nd' nd_def isQuorum_def) have "\n. lastAcceptedTerm (nodeState' n) = (if n = n\<^sub>0 then NO_TERM else lastAcceptedTerm (nodeState n))" using True unfolding lastAcceptedTerm_def updated_properties property_simps nd_def using not_le by (simp add: nodeState'_def nd') note updated_properties = updated_properties this show ?thesis apply (intro zenI) apply (unfold messages' message_simps committedTo_eq V_eq v_eq lastCommittedClusterStateBefore_eq property_simps promised_eq prevAccepted_eq) proof - from finite_messages show "finite messages" . from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\". from CatchUpResponse_committedTo show "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i". from CatchUpResponse_V show "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf". from CatchUpResponse_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs". from committedTo_firstUncommittedSlot show "\n. committed\<^sub>< (firstUncommittedSlot (nodeState' n))" unfolding updated_properties apply auto using CatchUpResponse_committedTo sent by blast from electionWon_isQuorum show "\n. electionWon (nodeState' n) \ isQuorum (nodeState' n) (joinVotes (nodeState' n))" unfolding updated_properties apply auto done from joinVotes_max show "\n n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState' n)) (currentTerm (nodeState n)) x \\) \ n' \ joinVotes (nodeState' n) \ n' \\ Vote (firstUncommittedSlot (nodeState' n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState' n)" unfolding updated_properties apply auto done from publishVotes show "\n n'. n' \ publishVotes (nodeState' n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState' n)) (currentTerm (nodeState n)) \\" unfolding updated_properties apply auto done from joinVotes show "\n n'. n' \ joinVotes (nodeState' n) \ promised (firstUncommittedSlot (nodeState' n)) n' n (currentTerm (nodeState n))" unfolding updated_properties apply auto done fix n from firstUncommittedSlot_PublishRequest show "\i t x. firstUncommittedSlot (nodeState' n) < i \ \ n \\ PublishRequest i t x \\" unfolding updated_properties using True nd_def by (cases "n = n\<^sub>0", auto) from PublishRequest_currentTerm show "\t x. n \\ PublishRequest (firstUncommittedSlot (nodeState' n)) t x \\ \ t \ currentTerm (nodeState n)" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) using True firstUncommittedSlot_PublishRequest nd_def by blast from PublishRequest_publishPermitted_currentTerm show "\t x. n \\ PublishRequest (firstUncommittedSlot (nodeState' n)) t x \\ \ publishPermitted (nodeState' n) \ t < currentTerm (nodeState n)" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) using True firstUncommittedSlot_PublishRequest nd_def by blast from currentClusterState_lastCommittedClusterStateBefore show "currentClusterState (nodeState' n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState' n))" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) using CatchUpResponse_lastCommittedClusterStateBefore sent by blast from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState' n) = V (firstUncommittedSlot (nodeState' n))" unfolding updated_properties using CatchUpResponse_V sent by (cases "n = n\<^sub>0", auto) from firstUncommittedSlot_PublishResponse show "\i t. firstUncommittedSlot (nodeState' n) < i \ \ n \\ PublishResponse i t \\" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) using True dual_order.strict_trans nd_def by blast from lastAcceptedTerm_None show "\i t. lastAcceptedTerm (nodeState' n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState' n)) t \\" unfolding updated_properties apply (cases "n = n\<^sub>0", auto) using True firstUncommittedSlot_PublishResponse nd_def by blast from lastAcceptedTerm_Some_sent show "\t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState' n)) t \\" unfolding updated_properties by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_Some_max show "\t t'. lastAcceptedTerm (nodeState' n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState' n)) t' \\ \ t' \ t" unfolding updated_properties by (cases "n = n\<^sub>0", auto) from lastAcceptedTerm_Some_value show "\t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState' n)) t (lastAcceptedValue (nodeState' n)) \\" unfolding updated_properties lastAcceptedValue_def by (cases "n = n\<^sub>0", auto simp add: nodeState'_def) from lastAcceptedTerm_Some_currentTerm show "\t. lastAcceptedTerm (nodeState' n) = SomeTerm t \ t \ currentTerm (nodeState n)" unfolding updated_properties by (cases "n = n\<^sub>0", auto) qed qed lemma (in zenStep) handleDiscardJoinVotes_invariants: assumes nd': "nd' = handleDiscardJoinVotes nd" assumes messages': "messages' = messages" shows "zen messages' nodeState'" proof - have message_simps[simp]: "\s p d. (s \\ p \\' d) = (s \\ p \\ d)" "\p d. (\ p \\' d) = (\ p \\ d)" "\s p. (s \\ p \\') = (s \\ p \\)" "\p. (\ p \\') = (\ p \\)" by (unfold isMessageFromTo'_def isMessageTo'_def isMessageFrom'_def isMessage'_def, auto simp add: messages' isMessageFromTo_def isMessageTo_def isMessageFrom_def isMessage_def) have property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. firstUncommittedSlot (nodeState' n) = firstUncommittedSlot (nodeState n)" "\n. currentVotingNodes (nodeState' n) = currentVotingNodes (nodeState n)" "\n q. isQuorum (nodeState' n) q = isQuorum (nodeState n) q" "\n. lastAcceptedTerm (nodeState' n) = lastAcceptedTerm (nodeState n)" "\n. lastAcceptedValue (nodeState' n) = lastAcceptedValue (nodeState n)" "\n. currentTerm (nodeState' n) = currentTerm (nodeState n)" "\n. currentClusterState (nodeState' n) = currentClusterState (nodeState n)" "\n. publishPermitted (nodeState' n) = publishPermitted (nodeState n)" "\n. publishVotes (nodeState' n) = publishVotes (nodeState n)" by (unfold nodeState'_def, auto simp add: nd_def isQuorum_def nd' handleDiscardJoinVotes_def Let_def lastAcceptedValue_def lastAcceptedTerm_def) have updated_properties: "\n. joinVotes (nodeState' n) = (if n = n\<^sub>0 then {} else joinVotes (nodeState n))" "\n. electionWon (nodeState' n) = (electionWon (nodeState n) \ n \ n\<^sub>0)" by (unfold nodeState'_def, auto simp add: nd' nd_def handleDiscardJoinVotes_def) have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def) have v\<^sub>c_eq[simp]: "v\<^sub>c' = v\<^sub>c" by (intro ext, auto simp add: v\<^sub>c'_def v\<^sub>c_def) have isCommitted_eq[simp]: "isCommitted' = isCommitted" by (intro ext, auto simp add: isCommitted'_def isCommitted_def) have committedTo_eq[simp]: "committed\<^sub><' = committed\<^sub><" by (intro ext, auto simp add: committedTo'_def committedTo_def) have V_eq[simp]: "V' = V" using v\<^sub>c_eq V'_def V_def by blast have promised_eq[simp]: "promised' = promised" by (intro ext, auto simp add: promised'_def promised_def) have lastCommittedClusterStateBefore_eq[simp]: "lastCommittedClusterStateBefore' = lastCommittedClusterStateBefore" unfolding lastCommittedClusterStateBefore_def lastCommittedClusterStateBefore'_def v\<^sub>c_eq .. have prevAccepted_eq[simp]: "prevAccepted' = prevAccepted" by (intro ext, auto simp add: prevAccepted'_def prevAccepted_def) show ?thesis apply (intro zenI) apply (unfold message_simps committedTo_eq V_eq v_eq lastCommittedClusterStateBefore_eq property_simps promised_eq prevAccepted_eq) proof - from finite_messages show "finite messages'" by (simp add: messages') from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from committedTo_firstUncommittedSlot show "\n. committed\<^sub>< (firstUncommittedSlot (nodeState n))" . from firstUncommittedSlot_PublishResponse show "\n i t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\" . from lastAcceptedTerm_None show "\n i t. lastAcceptedTerm (nodeState n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_sent show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\" . from lastAcceptedTerm_Some_max show "\n t t'. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\ \ t' \ t" . from lastAcceptedTerm_Some_value show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState n)) \\" . from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from currentClusterState_lastCommittedClusterStateBefore show "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\". from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))". from firstUncommittedSlot_PublishRequest show "\i n t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\". from PublishRequest_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ t \ currentTerm (nodeState n)". from CatchUpResponse_committedTo show "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i". from CatchUpResponse_V show "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf". from CatchUpResponse_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs". from lastAcceptedTerm_Some_currentTerm show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ t \ currentTerm (nodeState n)". from PublishRequest_publishPermitted_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ publishPermitted (nodeState n) \ t < currentTerm (nodeState n)". from publishVotes show "\n n'. n' \ publishVotes (nodeState n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) \\". from electionWon_isQuorum show "\n. electionWon (nodeState' n) \ isQuorum (nodeState n) (joinVotes (nodeState' n))" unfolding updated_properties by simp fix n from joinVotes show "\n'. n' \ joinVotes (nodeState' n) \ promised (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState n))" unfolding updated_properties by (cases "n = n\<^sub>0", simp_all) from joinVotes_max show "\n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) x \\) \ n' \ joinVotes (nodeState' n) \ n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState n)" unfolding updated_properties by (cases "n = n\<^sub>0", simp_all) qed qed lemma (in zenStep) handleReboot_invariants: assumes nd': "nd' = handleReboot nd" assumes messages': "messages' = messages" shows "zen messages' nodeState'" proof - have message_simps[simp]: "\s p d. (s \\ p \\' d) = (s \\ p \\ d)" "\p d. (\ p \\' d) = (\ p \\ d)" "\s p. (s \\ p \\') = (s \\ p \\)" "\p. (\ p \\') = (\ p \\)" by (unfold isMessageFromTo'_def isMessageTo'_def isMessageFrom'_def isMessage'_def, auto simp add: messages' isMessageFromTo_def isMessageTo_def isMessageFrom_def isMessage_def) have property_simps[simp]: "\n. currentNode (nodeState' n) = currentNode (nodeState n)" "\n. firstUncommittedSlot (nodeState' n) = firstUncommittedSlot (nodeState n)" "\n. currentVotingNodes (nodeState' n) = currentVotingNodes (nodeState n)" "\n q. isQuorum (nodeState' n) q = isQuorum (nodeState n) q" "\n. lastAcceptedTerm (nodeState' n) = lastAcceptedTerm (nodeState n)" "\n. lastAcceptedValue (nodeState' n) = lastAcceptedValue (nodeState n)" "\n. currentTerm (nodeState' n) = currentTerm (nodeState n)" "\n. currentClusterState (nodeState' n) = currentClusterState (nodeState n)" by (unfold nodeState'_def, auto simp add: nd_def isQuorum_def nd' handleReboot_def Let_def lastAcceptedValue_def lastAcceptedTerm_def) have updated_properties: "\n. publishPermitted (nodeState' n) = (publishPermitted (nodeState n) \ n \ n\<^sub>0)" "\n. joinVotes (nodeState' n) = (if n = n\<^sub>0 then {} else joinVotes (nodeState n))" "\n. electionWon (nodeState' n) = (electionWon (nodeState n) \ n \ n\<^sub>0)" "\n. publishVotes (nodeState' n) = (if n = n\<^sub>0 then {} else publishVotes (nodeState n))" by (unfold nodeState'_def, auto simp add: nd' nd_def handleReboot_def) have v_eq[simp]: "v' = v" by (intro ext, auto simp add: v'_def v_def) have v\<^sub>c_eq[simp]: "v\<^sub>c' = v\<^sub>c" by (intro ext, auto simp add: v\<^sub>c'_def v\<^sub>c_def) have isCommitted_eq[simp]: "isCommitted' = isCommitted" by (intro ext, auto simp add: isCommitted'_def isCommitted_def) have committedTo_eq[simp]: "committed\<^sub><' = committed\<^sub><" by (intro ext, auto simp add: committedTo'_def committedTo_def) have V_eq[simp]: "V' = V" using v\<^sub>c_eq V'_def V_def by blast have promised_eq[simp]: "promised' = promised" by (intro ext, auto simp add: promised'_def promised_def) have lastCommittedClusterStateBefore_eq[simp]: "lastCommittedClusterStateBefore' = lastCommittedClusterStateBefore" unfolding lastCommittedClusterStateBefore_def lastCommittedClusterStateBefore'_def v\<^sub>c_eq .. have prevAccepted_eq[simp]: "prevAccepted' = prevAccepted" by (intro ext, auto simp add: prevAccepted'_def prevAccepted_def) show ?thesis apply (intro zenI) apply (unfold message_simps committedTo_eq V_eq v_eq lastCommittedClusterStateBefore_eq property_simps promised_eq prevAccepted_eq) proof - from finite_messages show "finite messages'" by (simp add: messages') from Vote_future show "\i i' s t t' a. s \\ Vote i t a \\ \ i < i' \ t' < t \ \ s \\ PublishResponse i' t' \\". from Vote_None show "\i s t t'. s \\ Vote i t NO_TERM \\ \ t' < t \ \ s \\ PublishResponse i t' \\". from Vote_Some_lt show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ t' < t". from Vote_Some_PublishResponse show "\i s t t'. s \\ Vote i t (SomeTerm t') \\ \ s \\ PublishResponse i t' \\". from Vote_Some_max show "\i s t t' t''. s \\ Vote i t (SomeTerm t') \\ \ t' < t'' \ t'' < t \ \ s \\ PublishResponse i t'' \\". from Vote_not_broadcast show "\i t a d. \ Vote i t a \\ d \ d \ Broadcast". from Vote_unique_destination show "\i' i s t a a' d d'. s \\ Vote i t a \\ d \ s \\ Vote i' t a' \\ d' \ d = d'". from currentNode_nodeState show "\n. currentNode (nodeState n) = n" . from committedTo_firstUncommittedSlot show "\n. committed\<^sub>< (firstUncommittedSlot (nodeState n))" . from firstUncommittedSlot_PublishResponse show "\n i t. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishResponse i t \\" . from lastAcceptedTerm_None show "\n i t. lastAcceptedTerm (nodeState n) = NO_TERM \ \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\". from lastAcceptedTerm_Some_sent show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t \\" . from lastAcceptedTerm_Some_max show "\n t t'. lastAcceptedTerm (nodeState n) = SomeTerm t \ n \\ PublishResponse (firstUncommittedSlot (nodeState n)) t' \\ \ t' \ t" . from lastAcceptedTerm_Some_value show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ \ PublishRequest (firstUncommittedSlot (nodeState n)) t (lastAcceptedValue (nodeState n)) \\" . from Vote_currentTerm show "\n i t a. n \\ Vote i t a \\ \ t \ currentTerm (nodeState n)". from Vote_slot_function show "\n i i' t a a'. n \\ Vote i t a \\ \ n \\ Vote i' t a' \\ \ i = i'". from currentClusterState_lastCommittedClusterStateBefore show "\n. currentClusterState (nodeState n) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n))". from PublishRequest_committedTo show "\i t x. \ PublishRequest i t x \\ \ committed\<^sub>< i". from PublishRequest_quorum show "\i s t x. s \\ PublishRequest i t x \\ \ \q\majorities (V i). (\n\q. promised i n s t) \ (prevAccepted i t q = {} \ (\t'. v i t = v i t' \ maxTerm (prevAccepted i t q) \ t' \ \ PublishResponse i t' \\ \ t' < t))". from PublishRequest_function show "\i t x x'. \ PublishRequest i t x \\ \ \ PublishRequest i t x' \\ \ x = x'". from PublishResponse_PublishRequest show "\i s t. s \\ PublishResponse i t \\ \ \x. \ PublishRequest i t x \\". from ApplyCommit_quorum show "\i t. \ ApplyCommit i t \\ \ \q\majorities (V i). \s\q. s \\ PublishResponse i t \\". from currentVotingNodes_firstUncommittedSlot show "\n. currentVotingNodes (nodeState n) = V (firstUncommittedSlot (nodeState n))". from firstUncommittedSlot_PublishRequest show "\i n t x. firstUncommittedSlot (nodeState n) < i \ \ n \\ PublishRequest i t x \\". from PublishRequest_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ t \ currentTerm (nodeState n)". from CatchUpResponse_committedTo show "\i conf cs. \ CatchUpResponse i conf cs \\ \ committed\<^sub>< i". from CatchUpResponse_V show "\i conf cs. \ CatchUpResponse i conf cs \\ \ V i = conf". from CatchUpResponse_lastCommittedClusterStateBefore show "\i conf cs. \ CatchUpResponse i conf cs \\ \ lastCommittedClusterStateBefore i = cs". from lastAcceptedTerm_Some_currentTerm show "\n t. lastAcceptedTerm (nodeState n) = SomeTerm t \ t \ currentTerm (nodeState n)". from electionWon_isQuorum show "\n. electionWon (nodeState' n) \ isQuorum (nodeState n) (joinVotes (nodeState' n))" unfolding updated_properties by simp from PublishRequest_publishPermitted_currentTerm show "\n t x. n \\ PublishRequest (firstUncommittedSlot (nodeState n)) t x \\ \ publishPermitted (nodeState' n) \ t < currentTerm (nodeState n)" unfolding updated_properties by simp fix n from joinVotes show "\n'. n' \ joinVotes (nodeState' n) \ promised (firstUncommittedSlot (nodeState n)) n' n (currentTerm (nodeState n))" unfolding updated_properties by (cases "n = n\<^sub>0", simp_all) from publishVotes show "\n'. n' \ publishVotes (nodeState' n) \ n' \\ PublishResponse (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) \\" unfolding updated_properties by (cases "n = n\<^sub>0", simp_all) from joinVotes_max show "\n' a'. \ (\x. \ PublishRequest (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) x \\) \ n' \ joinVotes (nodeState' n) \ n' \\ Vote (firstUncommittedSlot (nodeState n)) (currentTerm (nodeState n)) a' \\ (OneNode n) \ a' \ lastAcceptedTerm (nodeState n)" unfolding updated_properties by (cases "n = n\<^sub>0", simp_all) qed qed fun insertOption :: "RoutedMessage option \ RoutedMessage set \ RoutedMessage set" where "insertOption None = id" | "insertOption (Some m) = insert m" lemma currentNode_ensureCurrentTerm[simp]: "currentNode (ensureCurrentTerm t nd) = currentNode nd" by (auto simp add: ensureCurrentTerm_def) lemma currentNode_publishValue[simp]: "currentNode (fst (publishValue x nd)) = currentNode nd" by (auto simp add: publishValue_def) lemma currentNode_addElectionVote[simp]: "currentNode (addElectionVote s i a nd) = currentNode nd" by (auto simp add: addElectionVote_def Let_def) lemma currentNode_handleVote[simp]: "currentNode (fst (handleVote s i t a nd)) = currentNode nd" by (auto simp add: handleVote_def Let_def) text \\pagebreak\ lemma initial_state_satisfies_invariants: shows "zen {} initialNodeState" by (unfold_locales, simp_all add: initialNodeState_def isQuorum_def lastAcceptedTerm_def lastAcceptedValue_def) lemma (in zen) invariants_preserved_by_ProcessMessage: fixes n\<^sub>0 assumes "m \ messages" defines "nd \ nodeState n\<^sub>0" defines "result \ ProcessMessage nd m" defines "nodeState' \ \n. if n = n\<^sub>0 then (fst result) else nodeState n" defines "messages' \ insertOption (snd result) messages" shows "zen messages' nodeState'" proof - { assume r: "result = (nd, None)" from r have "nodeState' = nodeState" by (auto simp add: nodeState'_def nd_def) moreover from r have "messages' = messages" by (simp add: messages'_def) ultimately have "zen messages' nodeState'" using zen_axioms by blast } note noop = this from `m \ messages` have m: "(sender m) \\ payload m \\ (destination m)" by (cases m, auto simp add: isMessageFromTo_def) have currentNode[simp]: "currentNode nd = n\<^sub>0" "\n. currentNode (nodeState n) = n" by (simp_all add: nd_def currentNode_nodeState) have zenStep: "zenStep messages nodeState messages' nodeState' n\<^sub>0" proof (intro_locales, intro zenStep_axioms.intro) show "messages \ messages'" by (cases "snd result", auto simp add: messages'_def) qed (simp add: nodeState'_def) have [simp]: "nodeState n\<^sub>0 = nd" by (simp add: nd_def) define broadcast' :: "(NodeData * Message option) \ (NodeData * RoutedMessage option)" where "\p. broadcast' p \ case p of (nd, Some m') \ (nd, Some \sender = currentNode nd, destination = Broadcast, payload = m' \) | (nd, None) \ (nd, None)" define respond' :: "(NodeData * Message option) \ (NodeData * RoutedMessage option)" where "\p. respond' p \ case p of (nd, Some m') \ (nd, Some \sender = currentNode nd, destination = OneNode (sender m), payload = m' \) | (nd, None) \ (nd, None)" have fst_broadcast[simp]: "\p. fst (broadcast' p) = fst p" unfolding broadcast'_def by (simp add: case_prod_unfold option.case_eq_if) have fst_respond[simp]: "\p. fst (respond' p) = fst p" unfolding respond'_def by (simp add: case_prod_unfold option.case_eq_if) show ?thesis proof (cases "destination m = Broadcast \ destination m = OneNode (currentNode nd)") case False hence "result = (nd, None)" unfolding result_def ProcessMessage_def Let_def by simp thus ?thesis by (intro noop) next case dest_ok: True have dest_True: "destination m \ {Broadcast, OneNode (currentNode nd)} = True" using dest_ok by simp show ?thesis proof (cases "payload m") case (StartJoin t) have result: "result = respond' (handleStartJoin t nd)" unfolding result_def respond'_def ProcessMessage_def dest_True StartJoin by auto have currentNode_eq: "currentNode (nodeState' n\<^sub>0) = currentNode (nodeState n\<^sub>0)" unfolding nodeState'_def result by (auto simp add: handleStartJoin_def) show ?thesis proof (intro zenStep.handleStartJoin_invariants [OF zenStep]) show "nodeState' n\<^sub>0 = fst (handleStartJoin t (nodeState n\<^sub>0))" by (simp add: result nodeState'_def) from currentNode_eq show "messages' = (case snd (handleStartJoin t (nodeState n\<^sub>0)) of None \ messages | Some m' \ insert \sender = n\<^sub>0, destination = OneNode (sender m), payload = m'\ messages)" unfolding messages'_def result respond'_def nodeState'_def by (cases "handleStartJoin t nd", cases "snd (handleStartJoin t nd)", auto) from m show "\d. \sender = sender m, destination = d, payload = StartJoin t\ \ messages" by (auto simp add: isMessageFromTo_def StartJoin) qed next case (Vote i t a) from Vote_not_broadcast m Vote dest_ok have dest_m: "destination m = OneNode n\<^sub>0" apply (cases "destination m") using isMessageTo_def apply fastforce by auto from Vote_not_broadcast m Vote dest_ok have m: "(sender m) \\ Vote i t a \\ (OneNode n\<^sub>0)" apply (cases "destination m") using isMessageTo_def apply fastforce by auto have result: "result = broadcast' (handleVote (sender m) i t a nd)" unfolding result_def ProcessMessage_def Vote dest_True broadcast'_def by simp have currentNode_eq: "currentNode (nodeState' n\<^sub>0) = currentNode (nodeState n\<^sub>0)" unfolding nodeState'_def result by simp show ?thesis proof (intro zenStep.handleVote_invariants [OF zenStep]) show "nodeState' n\<^sub>0 = fst (handleVote (sender m) i t a (nodeState n\<^sub>0))" by (simp add: result nodeState'_def) from currentNode_eq show "messages' = (case snd (handleVote (sender m) i t a (nodeState n\<^sub>0)) of None \ messages | Some m \ insert \sender = n\<^sub>0, destination = Broadcast, payload = m\ messages)" unfolding messages'_def result broadcast'_def nodeState'_def by (cases "handleVote (sender m) i t a nd", cases "snd (handleVote (sender m) i t a nd)", auto) from m show "\sender = sender m, destination = OneNode n\<^sub>0, payload = Vote i t a\ \ messages" by (auto simp add: Vote isMessageFromTo_def) qed next case (ClientValue x) have result: "result = broadcast' (handleClientValue x nd)" unfolding result_def ProcessMessage_def ClientValue dest_True broadcast'_def by simp have currentNode_eq: "currentNode (nodeState' n\<^sub>0) = currentNode (nodeState n\<^sub>0)" unfolding nodeState'_def result by (simp add: handleClientValue_def) show ?thesis proof (intro zenStep.handleClientValue_invariants [OF zenStep]) show "nodeState' n\<^sub>0 = fst (handleClientValue x (nodeState n\<^sub>0))" by (simp add: result nodeState'_def) from currentNode_eq show "messages' = (case snd (handleClientValue x (nodeState n\<^sub>0)) of None \ messages | Some m \ insert \sender = n\<^sub>0, destination = Broadcast, payload = m\ messages)" unfolding messages'_def result broadcast'_def nodeState'_def by (cases "handleClientValue x (nodeState n\<^sub>0)", cases "snd (handleClientValue x (nodeState n\<^sub>0))", auto) qed next case (PublishRequest i t x) have result: "result = respond' (handlePublishRequest i t x nd)" unfolding result_def ProcessMessage_def PublishRequest dest_True by (simp add: respond'_def) have currentNode_eq: "currentNode (nodeState' n\<^sub>0) = currentNode (nodeState n\<^sub>0)" unfolding nodeState'_def result by (simp add: handlePublishRequest_def) show ?thesis proof (intro zenStep.handlePublishRequest_invariants [OF zenStep]) show "nodeState' n\<^sub>0 = fst (handlePublishRequest i t x (nodeState n\<^sub>0))" by (simp add: nodeState'_def result) from currentNode_eq show "messages' = (case snd (handlePublishRequest i t x (nodeState n\<^sub>0)) of None \ messages | Some m' \ insert \sender = n\<^sub>0, destination = OneNode (sender m), payload = m'\ messages)" unfolding messages'_def result respond'_def nodeState'_def by (cases "handlePublishRequest i t x (nodeState n\<^sub>0)", cases "snd (handlePublishRequest i t x (nodeState n\<^sub>0))", auto) from m show "\d. \sender = sender m, destination = d, payload = PublishRequest i t x\ \ messages" by (auto simp add: PublishRequest isMessageFromTo_def) qed next case (PublishResponse i t) have result: "result = broadcast' (handlePublishResponse (sender m) i t nd)" unfolding result_def ProcessMessage_def PublishResponse dest_True by (simp add: broadcast'_def) show ?thesis proof (intro zenStep.handlePublishResponse_invariants [OF zenStep]) show "nodeState' n\<^sub>0 = fst (handlePublishResponse (sender m) i t (nodeState n\<^sub>0))" by (simp add: result nodeState'_def) show "messages' = (case snd (handlePublishResponse (sender m) i t (nodeState n\<^sub>0)) of None \ messages | Some m \ insert \sender = n\<^sub>0, destination = Broadcast, payload = m\ messages)" by (simp_all add: messages'_def result broadcast'_def handlePublishResponse_def commitIfQuorate_def) from m show "\d. \sender = sender m, destination = d, payload = PublishResponse i t\ \ messages" by (auto simp add: PublishResponse isMessageFromTo_def) qed next case (ApplyCommit i t) have result: "result = (handleApplyCommit i t nd, None)" unfolding result_def ProcessMessage_def ApplyCommit dest_True by simp have currentNode_eq: "currentNode (nodeState' n\<^sub>0) = currentNode (nodeState n\<^sub>0)" unfolding nodeState'_def result by (cases "lastAcceptedValue nd", simp_all add: handleApplyCommit_def applyAcceptedValue_def) show ?thesis proof (intro zenStep.handleApplyCommit_invariants [OF zenStep]) show "nodeState' n\<^sub>0 = handleApplyCommit i t (nodeState n\<^sub>0)" by (simp add: result nodeState'_def) show "messages' = messages" by (simp add: result messages'_def) from m show "\s d. \sender = s, destination = d, payload = ApplyCommit i t\ \ messages" by (auto simp add: ApplyCommit isMessageFromTo_def) qed next case Reboot have result: "result = (handleReboot nd, None)" unfolding result_def ProcessMessage_def Reboot dest_True by simp show ?thesis proof (intro zenStep.handleReboot_invariants [OF zenStep]) show "nodeState' n\<^sub>0 = handleReboot (nodeState n\<^sub>0)" by (simp add: nodeState'_def result) show "messages' = messages" by (simp add: result messages'_def) qed next case DiscardJoinVotes have result: "result = (handleDiscardJoinVotes nd, None)" unfolding result_def ProcessMessage_def DiscardJoinVotes dest_True by simp show ?thesis proof (intro zenStep.handleDiscardJoinVotes_invariants [OF zenStep]) show "nodeState' n\<^sub>0 = handleDiscardJoinVotes (nodeState n\<^sub>0)" by (simp add: nodeState'_def result) show "messages' = messages" by (simp add: result messages'_def) qed next case CatchUpRequest have result: "result = respond' (handleCatchUpRequest nd)" unfolding result_def ProcessMessage_def CatchUpRequest dest_True by (simp add: respond'_def) have currentNode_eq: "currentNode (nodeState' n\<^sub>0) = currentNode (nodeState n\<^sub>0)" unfolding nodeState'_def result by (cases "lastAcceptedValue nd", simp_all add: handleCatchUpRequest_def) show ?thesis proof (intro zenStep.handleCatchUpRequest_invariants [OF zenStep]) show "nodeState' n\<^sub>0 = fst (handleCatchUpRequest (nodeState n\<^sub>0))" by (simp add: nodeState'_def result) show "messages' = (case snd (handleCatchUpRequest (nodeState n\<^sub>0)) of None \ messages | Some m' \ insert \sender = n\<^sub>0, destination = OneNode (sender m), payload = m'\ messages)" by (simp_all add: messages'_def result respond'_def handleCatchUpRequest_def) qed next case (CatchUpResponse i conf cs) have result: "result = (handleCatchUpResponse i conf cs nd, None)" unfolding result_def ProcessMessage_def CatchUpResponse dest_True by simp show ?thesis proof (intro zenStep.handleCatchUpResponse_invariants [OF zenStep]) show "nodeState' n\<^sub>0 = handleCatchUpResponse i conf cs (nodeState n\<^sub>0)" by (simp add: nodeState'_def result) show "messages' = messages" by (simp add: result messages'_def) from m show "\s d. \sender = s, destination = d, payload = CatchUpResponse i conf cs\ \ messages" unfolding CatchUpResponse isMessageFromTo_def by auto qed qed qed qed theorem (in zen) invariants_imply_consistent_states: assumes "firstUncommittedSlot (nodeState n\<^sub>1) = firstUncommittedSlot (nodeState n\<^sub>2)" shows "currentClusterState (nodeState n\<^sub>1) = currentClusterState (nodeState n\<^sub>2)" proof - have "currentClusterState (nodeState n\<^sub>1) = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n\<^sub>1))" using currentClusterState_lastCommittedClusterStateBefore . also have "... = lastCommittedClusterStateBefore (firstUncommittedSlot (nodeState n\<^sub>2))" using assms by simp also have "... = currentClusterState (nodeState n\<^sub>2)" using currentClusterState_lastCommittedClusterStateBefore .. finally show ?thesis . qed end ================================================ FILE: cluster/isabelle/document/root.tex ================================================ \documentclass[11pt,a4paper]{article} \usepackage{isabelle,isabellesym} \usepackage{latexsym} % further packages required for unusual symbols (see also % isabellesym.sty), use only when needed %\usepackage{amssymb} %for \, \, \, \, \, \, %\, \, \, \, \, %\, \, \ %\usepackage{eurosym} %for \ %\usepackage[only,bigsqcap]{stmaryrd} %for \ %\usepackage{eufrak} %for \ ... \, \ ... \ (also included in amssymb) %\usepackage{textcomp} %for \, \, \, \, \, %\ % this should be the last package used \usepackage{pdfsetup} % urls in roman style, theory text in math-similar italics \urlstyle{rm} \isabellestyle{it} % for uniform font size %\renewcommand{\isastyle}{\isastyleminor} \begin{document} \title{elasticsearch-isabelle} \author{David Turner} \maketitle \tableofcontents % sane default for proof documents \parindent 0pt\parskip 0.5ex \section{Introduction} This is a presentation of an Isabelle/HOL theory that describes, and proves correct, a protocol for sharing \texttt{ClusterState} updates in Elasticsearch. % generated text of all theories \input{session} % optional bibliography %\bibliographystyle{abbrv} %\bibliography{root} \end{document} %%% Local Variables: %%% mode: latex %%% TeX-master: t %%% End: ================================================ FILE: cluster/tla/consensus.tla ================================================ `^\Large\bf TLA+ Model of an improved Zen consensus algorithm with reconfiguration capabilities ^' ------------------------------------------------------------------------------------- -------------------------------- MODULE consensus ----------------------------------- \* Imported modules used in this specification EXTENDS Naturals, FiniteSets, Sequences, TLC ---- \* `^\Large\bf Constants ^' \* The specification first defines the constants of the model, which amount to values or sets of \* values that are fixed. CONSTANTS Values \* Set of node ids (all master-eligible nodes) CONSTANTS Nodes \* The constant "Nil" denotes a place-holder for a non-existing value CONSTANTS Nil \* RPC message types CONSTANTS Join, \* only request is modeled PublishRequest, PublishResponse, Commit, \* only request is modeled Catchup \* only response is modeled \* Publish request types CONSTANTS Reconfigure, ApplyCSDiff ---- \* `^\Large\bf Variables ^' \* The following describes the variable state of the model. \* Set of requests and responses sent between nodes. VARIABLE messages \* node state (map from node id to state) VARIABLE firstUncommittedSlot VARIABLE currentTerm VARIABLE currentConfiguration VARIABLE currentClusterState VARIABLE lastAcceptedTerm VARIABLE lastAcceptedValue VARIABLE joinVotes VARIABLE electionWon VARIABLE publishPermitted VARIABLE publishVotes ---- \* set of valid configurations (i.e. the set of all non-empty subsets of Nodes) ValidConfigs == SUBSET(Nodes) \ {{}} \* quorums correspond to majority of votes in a config IsQuorum(votes, config) == Cardinality(votes \cap config) * 2 > Cardinality(config) \* checks whether two configurations only have intersecting quorums IntersectingQuorums(config1, config2) == /\ \lnot IsQuorum(config1 \ config2, config1) /\ \lnot IsQuorum(config2 \ config1, config2) \* initial model state Init == /\ messages = {} /\ firstUncommittedSlot = [n \in Nodes |-> 0] /\ currentTerm = [n \in Nodes |-> 0] /\ currentConfiguration \in {[n \in Nodes |-> vc] : vc \in ValidConfigs} \* all agree on initial config /\ currentClusterState \in {[n \in Nodes |-> v] : v \in Values} \* all agree on initial value /\ lastAcceptedTerm = [n \in Nodes |-> Nil] /\ lastAcceptedValue = [n \in Nodes |-> Nil] /\ joinVotes = [n \in Nodes |-> {}] /\ electionWon = [n \in Nodes |-> FALSE] /\ publishPermitted = [n \in Nodes |-> FALSE] /\ publishVotes = [n \in Nodes |-> {}] \* Send join request from node n to node nm for term t HandleStartJoin(n, nm, t) == /\ t > currentTerm[n] /\ LET joinRequest == [method |-> Join, source |-> n, dest |-> nm, slot |-> firstUncommittedSlot[n], term |-> t, laTerm |-> lastAcceptedTerm[n]] IN /\ currentTerm' = [currentTerm EXCEPT ![n] = t] /\ publishPermitted' = [publishPermitted EXCEPT ![n] = TRUE] /\ electionWon' = [electionWon EXCEPT ![n] = FALSE] /\ joinVotes' = [joinVotes EXCEPT ![n] = {}] /\ publishVotes' = [publishVotes EXCEPT ![n] = {}] /\ messages' = messages \cup { joinRequest } /\ UNCHANGED <> \* node n handles a join request and checks if it has received enough joins (= votes) \* for its term to be elected as master HandleJoinRequest(n, m) == /\ m.method = Join /\ m.term = currentTerm[n] /\ \/ /\ m.slot < firstUncommittedSlot[n] \/ /\ m.slot = firstUncommittedSlot[n] /\ (m.laTerm /= Nil => lastAcceptedTerm[n] /= Nil /\ m.laTerm <= lastAcceptedTerm[n]) /\ joinVotes' = [joinVotes EXCEPT ![n] = @ \cup { m.source }] /\ electionWon' = [electionWon EXCEPT ![n] = IsQuorum(joinVotes'[n], currentConfiguration[n])] /\ IF electionWon'[n] /\ publishPermitted[n] /\ lastAcceptedTerm[n] /= Nil THEN LET publishRequests == { [method |-> PublishRequest, source |-> n, dest |-> ns, term |-> currentTerm[n], slot |-> firstUncommittedSlot[n], value |-> lastAcceptedValue[n]] : ns \in Nodes } IN /\ messages' = messages \cup publishRequests /\ publishPermitted' = [publishPermitted EXCEPT ![n] = FALSE] ELSE /\ UNCHANGED <> /\ UNCHANGED <> \* client causes a cluster state change v ClientRequest(n, v) == /\ electionWon[n] /\ publishPermitted[n] /\ LET publishRequests == { [method |-> PublishRequest, source |-> n, dest |-> ns, term |-> currentTerm[n], slot |-> firstUncommittedSlot[n], value |-> [type |-> ApplyCSDiff, val |-> (currentClusterState[n] :> v)] ] : ns \in Nodes } IN /\ publishPermitted' = [publishPermitted EXCEPT ![n] = FALSE] /\ messages' = messages \cup publishRequests /\ UNCHANGED <> \* change the set of voters ChangeVoters(n, vs) == /\ electionWon[n] /\ publishPermitted[n] /\ LET publishRequests == { [method |-> PublishRequest, source |-> n, dest |-> ns, term |-> currentTerm[n], slot |-> firstUncommittedSlot[n], value |-> [type |-> Reconfigure, val |-> vs]] : ns \in Nodes } IN /\ publishPermitted' = [publishPermitted EXCEPT ![n] = FALSE] /\ messages' = messages \cup publishRequests /\ UNCHANGED <> \* handle publish request m on node n HandlePublishRequest(n, m) == /\ m.method = PublishRequest /\ m.slot = firstUncommittedSlot[n] /\ m.term = currentTerm[n] /\ lastAcceptedTerm' = [lastAcceptedTerm EXCEPT ![n] = m.term] /\ lastAcceptedValue' = [lastAcceptedValue EXCEPT ![n] = m.value] /\ LET response == [method |-> PublishResponse, source |-> n, dest |-> m.source, success |-> TRUE, term |-> m.term, slot |-> m.slot] IN /\ messages' = messages \cup {response} /\ UNCHANGED <> \* node n commits a change HandlePublishResponse(n, m) == /\ m.method = PublishResponse /\ m.slot = firstUncommittedSlot[n] /\ m.term = currentTerm[n] /\ publishVotes' = [publishVotes EXCEPT ![n] = @ \cup {m.source}] /\ IF IsQuorum(publishVotes'[n], currentConfiguration[n]) THEN LET commitRequests == { [method |-> Commit, source |-> n, dest |-> ns, term |-> currentTerm[n], slot |-> firstUncommittedSlot[n]] : ns \in Nodes } IN /\ messages' = messages \cup commitRequests ELSE UNCHANGED <> /\ UNCHANGED <> \* apply committed change to node n HandleCommitRequest(n, m) == /\ m.method = Commit /\ m.slot = firstUncommittedSlot[n] /\ m.term = lastAcceptedTerm[n] /\ firstUncommittedSlot' = [firstUncommittedSlot EXCEPT ![n] = @ + 1] /\ lastAcceptedTerm' = [lastAcceptedTerm EXCEPT ![n] = Nil] /\ lastAcceptedValue' = [lastAcceptedValue EXCEPT ![n] = Nil] /\ publishPermitted' = [publishPermitted EXCEPT ![n] = TRUE] /\ publishVotes' = [publishVotes EXCEPT ![n] = {}] /\ IF lastAcceptedValue[n].type = Reconfigure THEN /\ currentConfiguration' = [currentConfiguration EXCEPT ![n] = lastAcceptedValue[n].val] /\ electionWon' = [electionWon EXCEPT ![n] = IsQuorum(joinVotes[n], currentConfiguration'[n])] /\ UNCHANGED <> ELSE /\ Assert(lastAcceptedValue[n].type = ApplyCSDiff, "unexpected type") /\ Assert(DOMAIN(lastAcceptedValue[n].val) = {currentClusterState[n]}, "diff mismatch") /\ currentClusterState' = [currentClusterState EXCEPT ![n] = lastAcceptedValue[n].val[@]] \* apply diff /\ UNCHANGED <> /\ UNCHANGED <> \* node n captures current state and sends a catch up message SendCatchupResponse(n) == /\ LET catchupMessage == [method |-> Catchup, slot |-> firstUncommittedSlot[n], config |-> currentConfiguration[n], state |-> currentClusterState[n]] IN /\ messages' = messages \cup { catchupMessage } /\ UNCHANGED <> \* node n handles a catchup message HandleCatchupResponse(n, m) == /\ m.method = Catchup /\ m.slot > firstUncommittedSlot[n] /\ firstUncommittedSlot' = [firstUncommittedSlot EXCEPT ![n] = m.slot] /\ lastAcceptedTerm' = [lastAcceptedTerm EXCEPT ![n] = Nil] /\ lastAcceptedValue' = [lastAcceptedValue EXCEPT ![n] = Nil] /\ publishPermitted' = [publishPermitted EXCEPT ![n] = TRUE] /\ electionWon' = [electionWon EXCEPT ![n] = FALSE] /\ currentConfiguration' = [currentConfiguration EXCEPT ![n] = m.config] /\ currentClusterState' = [currentClusterState EXCEPT ![n] = m.state] /\ joinVotes' = [joinVotes EXCEPT ![n] = {}] /\ publishVotes' = [publishVotes EXCEPT ![n] = {}] /\ UNCHANGED <> \* crash/restart node n (loses ephemeral state) RestartNode(n) == /\ electionWon' = [electionWon EXCEPT ![n] = FALSE] /\ publishPermitted' = [publishPermitted EXCEPT ![n] = FALSE] /\ joinVotes' = [joinVotes EXCEPT ![n] = {}] /\ publishVotes' = [publishVotes EXCEPT ![n] = {}] /\ UNCHANGED <> \* next-step relation Next == \/ \E n, nm \in Nodes : HandleStartJoin(n, nm, currentTerm[n] + 1) \/ \E m \in messages : HandleJoinRequest(m.dest, m) \/ \E n \in Nodes : \E v \in Values : ClientRequest(n, v) \/ \E m \in messages : HandlePublishRequest(m.dest, m) \/ \E m \in messages : HandlePublishResponse(m.dest, m) \/ \E m \in messages : HandleCommitRequest(m.dest, m) \/ \E n \in Nodes : RestartNode(n) \/ \E n \in Nodes : \E vs \in ValidConfigs : ChangeVoters(n, vs) \/ \E n \in Nodes : SendCatchupResponse(n) \/ \E n \in Nodes : \E m \in messages : HandleCatchupResponse(n, m) ---- \* main invariant: StateMachineSafety == \A n1, n2 \in Nodes : firstUncommittedSlot[n1] = firstUncommittedSlot[n2] => /\ currentClusterState[n1] = currentClusterState[n2] /\ currentConfiguration[n1] = currentConfiguration[n2] OneMasterPerTerm == \A n1, n2 \in Nodes : /\ electionWon[n1] /\ electionWon[n2] /\ currentTerm[n1] = currentTerm[n2] /\ IntersectingQuorums(currentConfiguration[n1], currentConfiguration[n2]) => n1 = n2 LogMatching == \A n1, n2 \in Nodes : /\ firstUncommittedSlot[n1] = firstUncommittedSlot[n2] /\ lastAcceptedTerm[n1] = lastAcceptedTerm[n2] => lastAcceptedValue[n1] = lastAcceptedValue[n2] SingleNodeInvariant == \A n \in Nodes : /\ (lastAcceptedTerm[n] = Nil) = (lastAcceptedValue[n] = Nil) /\ lastAcceptedTerm[n] /= Nil => (lastAcceptedTerm[n] <= currentTerm[n]) /\ electionWon[n] = IsQuorum(joinVotes[n], currentConfiguration[n]) \* cached value is consistent /\ electionWon[n] /\ publishPermitted[n] => lastAcceptedValue[n] = Nil LogMatchingMessages == \A m1, m2 \in messages: /\ m1.method = PublishRequest /\ m2.method = PublishRequest /\ m1.slot = m2.slot /\ m1.term = m2.term => m1.value = m2.value SafeCatchupMessages == \A m1, m2 \in messages: /\ m1.method = Catchup /\ m2.method = Catchup /\ m1.slot = m2.slot => m1.config = m2.config /\ m1.state = m2.state \* State-exploration limits StateConstraint == /\ \A n \in Nodes: currentTerm[n] <= 3 /\ \A n \in Nodes: firstUncommittedSlot[n] <= 2 /\ Cardinality(messages) <= 15 ==================================================================================================== ================================================ FILE: cluster/tla/consensus.toolbox/.project ================================================ zen toolbox.builder.TLAParserBuilder toolbox.builder.PCalAlgorithmSearchingBuilder toolbox.natures.TLANature consensus.tla 1 PARENT-1-PROJECT_LOC/consensus.tla ================================================ FILE: cluster/tla/consensus.toolbox/.settings/org.lamport.tla.toolbox.prefs ================================================ ProjectRootFile=PARENT-1-PROJECT_LOC/consensus.tla eclipse.preferences.version=1 ================================================ FILE: cluster/tla/consensus.toolbox/consensus___model.launch ================================================ ================================================ FILE: data/tla/replication.tla ================================================ `^\Large\bf TLA+ Model of the Elasticsearch data replication approach ^' ------------------------------------------------------------------------------------- This file provides a formal specification of how data replication will work in future versions of Elasticsearch. We consider this work-in-progress: Model as well as implementation are still evolving and might differ in substantial ways. Introduction ------------ An index, which is a collection of documents, can be divided into multiple pieces, known as shards, each of which can be stored on different machines. This approach of horizontal scaling enables Elasticsearch to store much larger indices than could fit on a single machine. To ensure high availability and scale out read access, shards are usually also replicated onto multiple machines. The main copy is called the primary, all other copies are simply called replicas. The number of primary shards for an index is fixed at creation time, which allows to deterministically route requests for specific documents to the right shard, based on a hash of the document key (called the document "_id"). In the context of data replication, shards do their work independently from each other. The specification therefore only considers a single primary shard together with its copies, the replicas. It assumes a fixed set of nodes, each of which can host either the primary or one of the replicas. Shard allocation (i.e. which node has which shard) is dynamic and determined by the master, a process in the cluster that is backed by a consensus module. The TLA+ specification does not model the consensus module, it assumes that this component is working correctly. It shows however how data replication integrates with the consensus module to achieve its guarantees. How data is replicated ---------------------- Clients send requests to an arbitrary node in the cluster. The node then routes the request to the node in the cluster that has the primary shard for the corresponding document id. The document is indexed at the primary and then replicated concurrently to the replicas. The replicas index the document and confirm successful replication to the primary. The primary then acknowledges successful replication to the client. What's covered / not covered in this model ------------------------------------------ Failures covered by the model: - node crashes - disconnects between nodes on a per request basis Also covered: - cluster state batching / asynchronous application (each node applies the cluster state, which is the state backed by the consensus model, at different points in time) - network delays: messages can arrive out-of-order and be delayed Limitations of the model: - shard initialization / recovery is not modeled: the model initially assumes shards to be started. When a shard fails, it is not reallocated / reassigned to another node but stays unassigned. When a primary shard fails, a random replica is promoted to primary (if replica exists). - only the transaction log is modeled. Lucene store as an optimistic consumer of the transaction log is not modeled. - adding nodes to the cluster is not modeled, whereas in Elasticsearch, nodes can dynamically be added to the cluster and those nodes will share in the hosting of shard data (shard data is moved to the new node through the process of recovery, mentioned above, which is also not modeled). Other differences between model and current Java implementation: - the Java implementation uses zero-based numbering for sequence numbers whereas the TLA+ model starts from one, as this is the natural way to access the first element of a sequence in TLA+. - ... ------------------------------ MODULE replication --------------------------------- \* Imported modules used in this specification EXTENDS Naturals, FiniteSets, Sequences, TLC ---- \* `^\Large\bf Constants ^' \* The specification first defines the constants of the model, which amount to values or sets of \* values that are fixed. \* Set of node ids uniquely representing each data node in the cluster CONSTANTS Nodes \* Set of possible document ids (i.e. values for "_id" field) CONSTANTS DocumentIds (* Communication between the nodes is done using an RPC-like mechanism. For each request there is an associated response. In-flight requests / responses are modeled using messages, which are represented in TLA+ as record types. To distinguish requests from responses, the message record type contains a field "request" that denotes whether the message is a request or a response (takes as value one of {TRUE, FALSE}). It also contains the node id of the sender and receiver of the call (in the fields "source" and "dest"). The procedure that is called is found under the field "method". For example, sending a replication request from node n1 to n2 would yield the following message: [request |-> TRUE, method |-> Replication, source |-> n1, dest |-> n2] Responses also have a field "success" which indicates whether the call was successful or not (one of {TRUE, FALSE}). The model currently supports two RPC methods, one to replicate data from the primary to the replicas and another one to trim the translog (explained later). The reroute logic for rerouting requests to the primary is not modeled as it has no impact on consistency/durability guarantees. *) CONSTANTS Replication, TrimTranslog (* Shard allocation is determined by the master and broadcasted to the data nodes in the form of a routing table, which is a map from node id to one of {Primary, Replica, Unassigned}, denoting whether the respective node has the primary shard, a replica shard or no shard assigned at all. *) CONSTANTS Primary, Replica, Unassigned \* The constant "Nil" denotes a non-existing value (e.g. in transaction log) or a place-holder for \* a value that is to be computed. CONSTANTS Nil ---- \* `^\Large\bf Variables ^' \* The following describes the variable state of the model. \* Set of in-flight requests and responses sent between data nodes. VARIABLE messages (* Beside managing and broadcasting the routing table, the master also tracks if a primary failed and/or a replica was promoted to primary, incrementing a number called the "primary term" whenever this happens. Each new primary operates under a new term, allowing nodes to reject replication requests that come from a primary that has been demoted by the master. The routing table and primary term together form the cluster state, which is simply a record type containing both: [routingTable |-> [n1 |-> Primary, n2 |-> Replica, n3 |-> Unassigned], primaryTerm |-> 1] The following variable represents the current cluster state on the master, which is not explicitly modeled as a node. *) VARIABLE clusterStateOnMaster \* For simplicity we assume that each client (index) request uses a new unique value to be indexed. \* This is just a natural number incremented on each client operation. VARIABLE nextClientValue \* The set of (acknowledged) client responses. Stored in a variable so that we can make assertions \* about successfully acknowledged requests (e.g. that they've been successfully stored). VARIABLE clientResponses \* To improve readibility of the specification, the following placeholder clientVars can be used \* instead of explicitly writing the two variables on the right hand side. clientVars == <> (* After indexing a document into the primary, it is replicated concurrently to the replicas. Writes are only acknowledged to the client after all replicas have acknowledged their writes to the primary. To correlate acknowledgements from multiple replicas for the same write on the primary, we use a unique request id that is shared by the concurrent replication requests going out to the replicas for each specific indexing operation on the primary. The responses carry the same request id as the requests they were originating from. This is just a natural number denoting the next available request id. It is incremented whenever a request id is used. *) VARIABLE nextRequestId \* The following variables capture state on a per-node basis (maps with domain nodes). (* Cluster states are determined by the master and broadcasted to nodes. Due to the distributed nature of the system, they can arrive and be applied at different times on the nodes. ES only guarantees that an older state is not applied on a node after a new one has been applied on the same node. It supports batching, however, so cluster states can be skipped if a newer has already arrived on a node before the old one has been processed on that node. Cluster states applied on each node are represented as a map from node id to cluster state that is currently applied on this node *) VARIABLE clusterStateOnNode (* The cluster state contains the current term number. Nodes might learn about the highest primary term number not only through cluster state updates, but also through other node-to-node communication such as replication requests. They store the most recent information (highest term they've heard about). The variable "currentTerm" is a map from node id to primary term number, representing the highest primary term number that is known to the node. *) VARIABLE currentTerm (* The transaction log is a history of operations. The primary shard determines the order in which operations occur by assigning consecutive sequence numbers to the operations that are indexed. The sequence number represents a slot in the transaction log that is occupied by the operation. When a write on a primary is replicated to replicas, the replication request contains the sequence number that was assigned to this operation on the primary. The replica then assigns the operation to the same slot in its transaction log. Due to the concurrent nature of replication, replicas might fill these slots out-of-order. If the primary crashes or some replication requests don't make it to the replica, the replica can end up in a state where its transaction log has holes in it (slots that are not filled while subsequent slots are filled). Example of a transaction log on the primary and a replica: `. --------------------- | 1 | 2 | 3 | 4 | 5 | primary |-------------------| | x | x | x | x | | --------------------- --------------------- | 1 | 2 | 3 | 4 | 5 | replica |-------------------| (request for slot 2 is still in-flight) | x | | x | x | | --------------------- .' The transaction log is modeled as a map from node id to map from sequence number to record type consisting of document id, value to be stored, primary term and "pending confirmation" marker (more on that later). *) VARIABLE tlog (* Having a transaction log in place, it is useful to know about the highest slot in the transaction log where all slots below it have been successfully replicated to all replicas, i.e. the common history shared by all in-sync shard copies. It is useful because in the case where a primary fails, a replica which is promoted to primary knows that it has to only worry about being out of sync with other replicas on slots that are beyond this slot. The primary is in charge of tracking this information, also called global checkpoint. For this, replica shards share information with the primary on the highest slot they have filled where all lower slots are filled as well, called the local checkpoint. The primary then establishes the global checkpoint as the minimum of the local checkpoint value received from all shard copies (including its own local checkpoint) and broadcasts this information to the replicas. The global checkpoint is modeled as a map from node id to sequence number. *) VARIABLE globalCheckPoint (* The local checkpoint is modeled as a map from node id (node that is doing the tracking) to node id (node for which the local checkpoint is being tracked) to sequence number. Only the primary maintains the local checkpoints from all replicas, but because of the possibility of the primary changing over time, and in order to separate the state for each node more clearly, we maintain a node id to local checkpoint map for each node in the cluster. *) VARIABLE localCheckPoint \* The placeholder "nodeVars" is used as a shorthand for all node variables nodeVars == <> ---- \* `^\Large\bf General helper functions ^' \* Return the minimum value from a set, or undefined if the set is empty. Min(s) == CHOOSE x \in s : \A y \in s : x <= y \* Return the maximum value from a set, or undefined if the set is empty. Max(s) == CHOOSE x \in s : \A y \in s : x >= y ---- \* `^\Large\bf Helper functions on routing table ^' (* Note, in this section, the terms "shard" and "node id" are conflated, because we are only considering one shard and all its copies, so each shard can be uniquely identified by the node it resides on. The term "shard" or "node" is chosen solely based on the context of what is being explained or specified. *) \* Returns shards that are marked as Primary in routing table Primaries(routingTable) == {n \in DOMAIN routingTable : routingTable[n] = Primary} \* Returns shards that are marked as Replica in routing table Replicas(routingTable) == {n \in DOMAIN routingTable : routingTable[n] = Replica} \* Returns shards that are marked as Primary or Replica in routing table Assigned(routingTable) == {n \in DOMAIN routingTable : routingTable[n] /= Unassigned} \* Determines whether the shard on node n was promoted to primary when a cluster state update occurs ShardWasPromotedToPrimary(n, incomingRoutingTable, localRoutingTable) == LET oldPrimaries == Primaries(localRoutingTable) newPrimaries == Primaries(incomingRoutingTable) IN /\ n \notin oldPrimaries /\ n \in newPrimaries \* Calculates new cluster state based on shard failure on node n FailShardOnMaster(n) == LET rt == clusterStateOnMaster.routingTable IN IF rt[n] = Unassigned THEN UNCHANGED <> ELSE \* increase primary term on primary failure LET newPt == IF rt[n] = Primary THEN clusterStateOnMaster.primaryTerm + 1 ELSE clusterStateOnMaster.primaryTerm IN IF rt[n] = Primary /\ Cardinality(Replicas(rt)) > 0 THEN \* promote replica to primary \E r \in Replicas(rt): clusterStateOnMaster' = [clusterStateOnMaster EXCEPT !.routingTable = [rt EXCEPT ![n] = Unassigned, ![r] = Primary], !.primaryTerm = newPt] ELSE clusterStateOnMaster' = [clusterStateOnMaster EXCEPT !.routingTable[n] = Unassigned, !.primaryTerm = newPt] ---- \* `^\Large\bf Helper functions for sending/receiving messages ^' \* Remove request from the set of messages and add response instead Reply(response, request) == messages' = {response} \cup (messages \ {request}) \* Generate default replication response based on replication request m \* Copies most of the fields over for convenience (to restore caller information upon return) DefaultReplicationResponse(m) == [request |-> FALSE, method |-> m.method, source |-> m.dest, dest |-> m.source, req |-> m.req, id |-> m.id, seq |-> m.seq, rterm |-> m.rterm, sterm |-> m.sterm, value |-> m.value, client |-> m.client, localCP |-> 0, success |-> TRUE] \* Generate default trim translog response based on trim translog request m DefaultTrimTranslogResponse(m) == [request |-> FALSE, method |-> m.method, source |-> m.dest, dest |-> m.source, req |-> m.req, maxseq |-> m.maxseq, \* trim above this sequence number term |-> m.term, \* trim entries with term lower than this success |-> TRUE] \* Generate default response based on request m DefaultResponse(m) == IF m.method = Replication THEN DefaultReplicationResponse(m) ELSE DefaultTrimTranslogResponse(m) \* Generate default failure response based on request m FailedResponse(m) == [DefaultResponse(m) EXCEPT !.success = FALSE] ---- \* `^\Large\bf Helper functions on translog ^' (* When a request comes to a primary, it has to select a slot (the sequence number) to put the request into. It corresponds to the slot in the transaction log right after the highest slot that's filled. The following function yields the highest slot that's filled in the transaction log, or 0 if no such slot exists. *) MaxSeq(ntlog) == Max(DOMAIN ntlog \cup {0}) (* `^\bf\large MaxConfirmedSeq ^' The local checkpoint is defined as the highest slot in the translog, where all lower slots are filled. It is not only holes in the translog that prevent the local checkpoint from moving foward. We actually want to prevent the local checkpoint from moving past slots which are filled but marked as pending confirmation. Pending entries in the translog are entries that appear after the global checkpoint that are not allowed to contribute to the advancement of the local checkpoint until a resyncing phase happens due to the primary shard changing nodes. Translog entries thus have a "pc" (pending confirmation) marker (\in {TRUE, FALSE}) that says whether the local checkpoint can move past them. This function yields highest sequence number which are not pending confirmation and where all lower slots are not pending confirmation either. Yields 0 if no such number exists. Examples: (T stands for TRUE, F for FALSE) `. --------------------- | 1 | 2 | 3 | 4 | 5 | |-------------------| MaxConfirmedSeq = 1 | x | x | x | x | | pc markers: | F | T | T | F | | --------------------- --------------------- | 1 | 2 | 3 | 4 | 5 | |-------------------| MaxConfirmedSeq = 2 | x | x | | x | | pc markers: | F | F | | F | | --------------------- .' *) MaxConfirmedSeq(ntlog) == LET ConfirmedTlogSlot(i) == i \in DOMAIN ntlog /\ ntlog[i].pc = FALSE IN CHOOSE i \in (DOMAIN ntlog \cup {0}) : /\ ConfirmedTlogSlot(i+1) = FALSE /\ \A j \in 1..i : ConfirmedTlogSlot(j) (* `^\bf\large MarkPC ^' Yields translog where all entries at position strictly larger than "globalCP" (representing the global checkpoint) are marked as "pending confirmation". Example: `. --------------------- MarkPC --------------------- | 1 | 2 | 3 | 4 | 5 | -----------> | 1 | 2 | 3 | 4 | 5 | |-------------------| globalCP = 1 |-------------------| | x | x | | x | | | x | x | | x | | | F | F | | F | | | F | T | | T | | --------------------- --------------------- .' *) MarkPC(ntlog, globalCP) == [j \in DOMAIN ntlog |-> [ntlog[j] EXCEPT !.pc = IF j > globalCP THEN TRUE ELSE @]] (* `^\bf\large FillAndUnmarkPC ^' Yields translog where all gaps are filled and the pending confirmation marker is set to FALSE for values at position strictly larger than "globalCP" representing the global checkpoint Example: `. --------------------- FillAndUnmarkPC --------------------- | 1 | 2 | 3 | 4 | 5 | -----------> | 1 | 2 | 3 | 4 | 5 | |-------------------| globalCP = 1 |-------------------| | x | x | | x | | | x | x | x | x | | | T | T | | F | | | T | F | F | F | | --------------------- --------------------- .' *) FillAndUnmarkPC(ntlog, storedTerm, globalCP) == [j \in 1..Max(DOMAIN ntlog \cup {0}) |-> IF j > globalCP THEN IF j \in DOMAIN ntlog THEN [ntlog[j] EXCEPT !.pc = FALSE] ELSE [id |-> Nil, term |-> storedTerm, value |-> Nil, pc |-> FALSE] ELSE ntlog[j]] (* `^\bf\large TrimTlog ^' Trim elements from translog with position strictly greater than maxseq and term strictly lower than minterm. Example: `. --------------------- TrimTlog --------------------- | 1 | 2 | 3 | 4 | 5 | -----------> | 1 | 2 | 3 | 4 | 5 | |-------------------| maxseq = 2 |-------------------| | x | x | x | x | x | minterm = 2 | x | x | x | | x | | T | T | T | F | T | | T | T | T | | F | terms: | 1 | 1 | 2 | 1 | 2 | | 1 | 1 | 2 | | 2 | --------------------- --------------------- .' *) TrimTlog(ntlog, maxseq, minterm) == [j \in {i \in DOMAIN ntlog : i <= maxseq \/ ntlog[i].term >= minterm} |-> ntlog[j]] ---- \* `^\Large\bf Initial states ^' \* All possible routing tables where there is one primary RoutingTablesWithPrimary == UNION { { [n \in {pn} |-> Primary] @@ \* pn has the primary [n \in rs |-> Replica] @@ \* rs is the subset of nodes having replicas [n \in ((Nodes \ rs) \ {pn}) |-> Unassigned] \* remaining nodes have unassigned shards : rs \in SUBSET (Nodes \ {pn}) } : pn \in Nodes } \* Possible initial routing tables are those which have a primary or where all shards are unassigned InitialRoutingTables == RoutingTablesWithPrimary \cup {[n \in Nodes |-> Unassigned]} \* The following constant denotes the set of possible initial cluster states that are to be \* considered for exploring the model, containing cluster states such as \* [ routingTable |-> [n1 |-> Primary, n2 |-> Replica, n3 |-> Replica], primaryTerm |-> 1 ] InitialClusterStates == { [routingTable |-> rt, primaryTerm |-> 1] : rt \in InitialRoutingTables } Init == /\ clusterStateOnMaster \in InitialClusterStates /\ messages = {} /\ nextClientValue = 1 /\ clientResponses = {} /\ nextRequestId = 1 /\ tlog = [n \in Nodes |-> << >>] /\ localCheckPoint = [n1 \in Nodes |-> [n2 \in Nodes |-> 0]] /\ globalCheckPoint = [n \in Nodes |-> 0] /\ clusterStateOnNode = [n \in Nodes |-> clusterStateOnMaster] /\ currentTerm = [n \in Nodes |-> clusterStateOnMaster.primaryTerm] ---- \* `^\Large\bf Next-step relations ^' \* Index request arrives on node n with document id docId ClientRequest(n, docId) == /\ clusterStateOnNode[n].routingTable[n] = Primary \* node believes itself to be the primary /\ LET replicas == Replicas(clusterStateOnNode[n].routingTable) primaryTerm == currentTerm[n] tlogEntry == [id |-> docId, term |-> primaryTerm, value |-> nextClientValue, pc |-> FALSE] seq == MaxSeq(tlog[n]) + 1 \* create replication requests for each replica that the primary knows about replRequests == {([request |-> TRUE, method |-> Replication, source |-> n, dest |-> rn, req |-> nextRequestId, id |-> docId, value |-> nextClientValue, seq |-> seq, rterm |-> primaryTerm, \* current term when issuing request sterm |-> primaryTerm, \* term to be stored (differs for fast resync) client |-> TRUE, \* it's a replication request initiated by client globalCP |-> globalCheckPoint[n]]) : rn \in replicas} IN \* put entry into translog /\ tlog' = [tlog EXCEPT ![n] = (seq :> tlogEntry) @@ @] \* Make sure that each client request uses a unique value /\ nextClientValue' = nextClientValue + 1 \* set next unique key to use for replication requests so that we can relate responses /\ nextRequestId' = nextRequestId + 1 \* update local checkpoint /\ localCheckPoint' = [localCheckPoint EXCEPT ![n][n] = seq] /\ Assert(localCheckPoint'[n][n] = localCheckPoint[n][n] + 1, "localCheckPoint incremented") \* send out replication requests /\ messages' = messages \cup replRequests /\ IF replicas = {} THEN \* no replicas, directly acknowledge to the client /\ clientResponses' = clientResponses \cup {[success |-> TRUE, id |-> docId, value |-> nextClientValue, seq |-> seq, term |-> primaryTerm]} ELSE \* replication requests sent out, wait for responses before acking to client /\ UNCHANGED <> /\ UNCHANGED <> \* Helper function for marking translog entries as pending confirmation if incoming term is higher \* than current term on node n. MaybeMarkPC(incomingTerm, n, globalCP) == IF incomingTerm > currentTerm[n] THEN \* there is a new primary, cannot safely advance local checkpoint \* before resync done, move it back to global checkpoint and add \* pending confirmation marker to all entries above global checkpoint MarkPC(tlog[n], globalCP) ELSE tlog[n] (* `^\bf Note on handling replication requests with higher primary terms^' If a replica receives a replication request with a primary term greater than its current primary term, it means that new primary was promoted. The `^na\"ive^' handling of this case would be to forget all translog operations above global checkpoint, reset local checkpoint to global checkpoint and await replay of operations above global checkpoint from the newly promoted primary. However, this approach does not work, because newly promoted primary might fail during resync process and if our replica is promoted to primary - it may miss operations above global checkpoint, effectively losing acknowledged writes. That's why we preserve existing entries in translog above global checkpoint but mark them as pending confirmation. During operations replay, replica replaces its translog entry with the entry received from the primary (which can be noop) and resets pending confirmation flag. *) \* Replication request arrives on node n with message m HandleReplicationRequest(n, m) == /\ m.request = TRUE /\ m.method = Replication /\ IF m.rterm < currentTerm[n] THEN \* don't accept replication requests with lower term than we have \* lower term means that it's coming from a primary that has since been demoted /\ Reply(FailedResponse(m), m) /\ UNCHANGED <> ELSE /\ LET tlogEntry == [id |-> m.id, term |-> m.sterm, value |-> m.value, pc |-> FALSE] newGlobalCP == Max({m.globalCP, globalCheckPoint[n]}) \* mark translog entries as pending if higher term and write request into translog newTlog == (m.seq :> tlogEntry) @@ MaybeMarkPC(m.rterm, n, newGlobalCP) \* recompute local checkpoint localCP == MaxConfirmedSeq(newTlog) IN /\ tlog' = [tlog EXCEPT ![n] = newTlog] /\ currentTerm' = [currentTerm EXCEPT ![n] = m.rterm] /\ globalCheckPoint' = [globalCheckPoint EXCEPT ![n] = newGlobalCP] /\ Reply([DefaultResponse(m) EXCEPT !.localCP = localCP], m) /\ UNCHANGED <> \* Trim translog request arrives on node n with message m HandleTrimTranslogRequest(n, m) == /\ m.request = TRUE /\ m.method = TrimTranslog /\ IF m.term < currentTerm[n] THEN \* don't handle requests with lower term than we have \* lower term means that it's coming from a primary that has since been demoted /\ Reply(FailedResponse(m), m) /\ UNCHANGED <> ELSE /\ LET newGlobalCP == Max({m.globalCP, globalCheckPoint[n]}) \* mark translog entries as pending if higher term and trim translog newTlog == TrimTlog(MaybeMarkPC(m.term, n, newGlobalCP), m.maxseq, m.term) IN /\ tlog' = [tlog EXCEPT ![n] = newTlog] /\ globalCheckPoint' = [globalCheckPoint EXCEPT ![n] = newGlobalCP] /\ currentTerm' = [currentTerm EXCEPT ![n] = m.term] /\ Reply(DefaultResponse(m), m) /\ UNCHANGED <> \* Helper function for handling replication responses FinishIfNeeded(m) == \* check if this is the last response we're waiting for IF /\ m.client /\ { ms \in messages : ms.req = m.req } = {m} \* check if the request has not been failed already to the client /\ { cr \in clientResponses : cr.success = FALSE /\ cr.req = m.req } = {} THEN clientResponses' = clientResponses \cup {[success |-> TRUE, id |-> m.id, value |-> m.value, seq |-> m.seq, term |-> m.rterm]} ELSE UNCHANGED <> \* Helper function for handling replication responses FinishAsFailed(m) == /\ clientResponses' = clientResponses \cup {[success |-> FALSE, req |-> m.req]} \* Replication response arrives on node n from node rn with message m HandleReplicationResponse(n, rn, m) == /\ m.request = FALSE /\ m.method = Replication \* are we still interested in the response or already marked the overall client request as failed? /\ IF m.success THEN \* is it a newer local checkpoint than we have? /\ IF m.localCP > localCheckPoint[n][rn] /\ \* is the shard still active on this node? clusterStateOnNode[n].routingTable[n] /= Unassigned THEN LET newLocalCheckPoint == [localCheckPoint EXCEPT ![n][rn] = m.localCP] assigned == Assigned(clusterStateOnNode[n].routingTable) computedGlobalCP == Min({newLocalCheckPoint[n][i] : i \in assigned}) IN /\ localCheckPoint' = newLocalCheckPoint \* also update global checkpoint if necessary /\ globalCheckPoint' = [globalCheckPoint EXCEPT ![n] = computedGlobalCP] ELSE UNCHANGED <> /\ UNCHANGED <> /\ FinishIfNeeded(m) ELSE \* replication failed, ask master to fail shard /\ IF m.rterm < clusterStateOnMaster.primaryTerm THEN \* term outdated, fail itself and don't ack client write /\ FinishAsFailed(m) /\ UNCHANGED <> ELSE \* fail shard and respond to client /\ FailShardOnMaster(rn) /\ FinishIfNeeded(m) /\ UNCHANGED <> /\ messages' = messages \ {m} /\ UNCHANGED <> \* Trim translog response arrives on node n from node rn with message m HandleTrimTranslogResponse(n, rn, m) == /\ m.request = FALSE /\ m.method = TrimTranslog /\ messages' = messages \ {m} /\ IF m.success = FALSE /\ m.term >= clusterStateOnMaster.primaryTerm THEN \* fail shard FailShardOnMaster(rn) ELSE UNCHANGED <> /\ UNCHANGED <> \* Cluster state propagated from master is applied to node n ApplyClusterStateFromMaster(n) == /\ clusterStateOnNode[n] /= clusterStateOnMaster /\ clusterStateOnNode' = [clusterStateOnNode EXCEPT ![n] = clusterStateOnMaster] /\ IF ShardWasPromotedToPrimary(n, clusterStateOnMaster.routingTable, clusterStateOnNode[n].routingTable) THEN \* shard promoted to primary, resync with replicas LET ntlog == tlog[n] globalCP == globalCheckPoint[n] newTerm == clusterStateOnMaster.primaryTerm \* fill gaps in tlog and remove pending confirmation marker newTlog == FillAndUnmarkPC(ntlog, newTerm, globalCP) replicas == Replicas(clusterStateOnMaster.routingTable) numReplicas == Cardinality(replicas) startSeq == globalCP + 1 endSeq == Max((DOMAIN ntlog) \cup {0}) numDocs == endSeq + 1 - startSeq \* resend all translog entries above global checkpoint to replicas replRequests == {([request |-> TRUE, method |-> Replication, source |-> n, dest |-> rn, req |-> nextRequestId + (i - startSeq), seq |-> i, rterm |-> newTerm, \* new term when issuing request sterm |-> newTlog[i].term, \* stored term for entry id |-> newTlog[i].id, value |-> newTlog[i].value, client |-> FALSE, \* request not initiated by client globalCP |-> globalCP]) : i \in startSeq..endSeq, rn \in replicas} \* send trim request to replicas trimRequests == {[request |-> TRUE, method |-> TrimTranslog, source |-> n, dest |-> rn, req |-> nextRequestId + numDocs, maxseq |-> endSeq, term |-> newTerm, client |-> FALSE, globalCP |-> globalCP] : rn \in replicas} IN /\ currentTerm' = [currentTerm EXCEPT ![n] = newTerm] /\ tlog' = [tlog EXCEPT ![n] = newTlog] /\ localCheckPoint' = [localCheckPoint EXCEPT ![n][n] = MaxConfirmedSeq(newTlog)] /\ messages' = messages \cup replRequests \cup trimRequests /\ nextRequestId' = nextRequestId + numDocs + 1 ELSE /\ UNCHANGED <> /\ UNCHANGED <> \* Fail request message FailRequestMessage(m) == /\ m.request = TRUE /\ Reply(FailedResponse(m), m) /\ UNCHANGED <> \* Fail response message FailResponseMessage(m) == /\ m.request = FALSE /\ m.success = TRUE /\ Reply([m EXCEPT !.success = FALSE], m) /\ UNCHANGED <> \* Node fault detection on master finds node n to be isolated from the cluster or crashed NodeFaultDetectionKicksNodeOut(n) == /\ clusterStateOnMaster.routingTable[n] /= Unassigned \* not already unassigned /\ FailShardOnMaster(n) /\ UNCHANGED <> \* Defines how the variables may transition. Next == \/ \E n \in Nodes : \E docId \in DocumentIds : ClientRequest(n, docId) \/ \E m \in messages : HandleReplicationRequest(m.dest, m) \/ \E m \in messages : HandleReplicationResponse(m.dest, m.source, m) \/ \E m \in messages : HandleTrimTranslogRequest(m.dest, m) \/ \E m \in messages : HandleTrimTranslogResponse(m.dest, m.source, m) \/ \E m \in messages : FailRequestMessage(m) \/ \E m \in messages : FailResponseMessage(m) \/ \E n \in Nodes : ApplyClusterStateFromMaster(n) \/ \E n \in Nodes : NodeFaultDetectionKicksNodeOut(n) ---- \* `^\Large\bf Helper functions for making assertions ^' \* no active messages NoActiveMessages == messages = {} \* shard that is considered active by the master ActiveShard(n) == clusterStateOnMaster.routingTable[n] /= Unassigned \* cluster state on master has been applied to all nodes that are still supposed to have an active shard ClusterStateAppliedOnAllNodesWithActiveShards == \A n \in Nodes : ActiveShard(n) => clusterStateOnNode[n] = clusterStateOnMaster \* everything in the translog up to and including slot i UpToSlot(ntlog, i) == [j \in 1..i |-> ntlog[j]] \* copy of translog, where we ignore the pending confirmation marker ExceptPC(ntlog) == [j \in DOMAIN ntlog |-> [r \in DOMAIN ntlog[j] \ {"pc"} |-> ntlog[j][r] ]] \* all shard copies contain same data AllCopiesSameContents == \A n1, n2 \in Nodes: /\ n1 /= n2 /\ ActiveShard(n1) /\ ActiveShard(n2) => ExceptPC(tlog[n1]) = ExceptPC(tlog[n2]) ---- \* `^\Large\bf Main invariants ^' \* checks if the translog for all nodes are equivalent up to their global checkpoint, only differing \* in the safety marker (which can be false sometimes if the global checkpoint on one shard is lower \* than on another one) SameTranslogUpToGlobalCheckPoint == \A n1, n2 \in Nodes: /\ n1 /= n2 /\ ActiveShard(n1) /\ ActiveShard(n2) => ExceptPC(UpToSlot(tlog[n1], globalCheckPoint[n1])) = ExceptPC(UpToSlot(tlog[n2], globalCheckPoint[n1])) \* checks if the translog for all nodes is eventually the same AllCopiesSameContentsOnQuietDown == (/\ NoActiveMessages /\ ClusterStateAppliedOnAllNodesWithActiveShards) => AllCopiesSameContents \* checks if all (acked) responses to client are successfully and correctly stored AllAckedResponsesStored == \A r \in clientResponses : \A n \in Nodes : /\ r.success = TRUE /\ ActiveShard(n) => /\ r.seq \in DOMAIN tlog[n] /\ tlog[n][r.seq].id = r.id /\ tlog[n][r.seq].value = r.value /\ tlog[n][r.seq].term = r.term \* checks that the global checkpoint is the same as or below the local checkpoint on each node GlobalCheckPointBelowLocalCheckPoints == \A n \in Nodes : globalCheckPoint[n] <= MaxConfirmedSeq(tlog[n]) \* local checkpoint always corresponds to MaxSeq and MaxConfirmedSeq on the primary node LocalCheckPointMatchesMaxConfirmedSeq == \A n \in Nodes : clusterStateOnNode[n].routingTable[n] = Primary => /\ localCheckPoint[n][n] = MaxConfirmedSeq(tlog[n]) /\ MaxSeq(tlog[n]) = MaxConfirmedSeq(tlog[n]) \* routing table is well-formed (has at most one primary, and has no replicas if no primaries) WellFormedRoutingTable(routingTable) == /\ Cardinality(Primaries(routingTable)) <= 1 /\ ( Cardinality(Primaries(routingTable)) = 0 => Cardinality(Assigned (routingTable)) = 0) StateConstraint == /\ nextClientValue <= 3 /\ Cardinality(messages) <= 5 /\ 0 < Cardinality(Assigned (clusterStateOnMaster.routingTable)) ============================================================================= ================================================ FILE: data/tla/replication.toolbox/.project ================================================ elastic toolbox.builder.TLAParserBuilder toolbox.builder.PCalAlgorithmSearchingBuilder toolbox.natures.TLANature replication.tla 1 PARENT-1-PROJECT_LOC/replication.tla ================================================ FILE: data/tla/replication.toolbox/.settings/org.lamport.tla.toolbox.prefs ================================================ ProjectRootFile=PARENT-1-PROJECT_LOC/replication.tla eclipse.preferences.version=1 ================================================ FILE: data/tla/replication.toolbox/replication___model.launch ================================================