Repository: blainehansen/magma Branch: main Commit: a6ced658d623 Files: 58 Total size: 591.0 KB Directory structure: gitextract_w5tkx20i/ ├── .editorconfig ├── .github/ │ └── FUNDING.yml ├── .gitignore ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── Cargo.toml ├── README.future.md ├── README.md ├── iris-notes.md ├── justfile ├── lab.ll ├── mg_examples/ │ └── main.mg ├── notes/ │ ├── 2019-popl-iron-final.md │ ├── assembly-proofs.md │ ├── category-theory-for-programmers.md │ ├── coq-coq-correct.md │ ├── coq-metacoq.md │ ├── indexing-foundational-proof-carrying-code.md │ ├── indexing-indexed-model.md │ ├── indexing-modal-model.md │ ├── iris-from-the-ground-up.md │ ├── iris-lecture-notes.md │ ├── jung-thesis.md │ ├── known_types.md │ ├── pony-reference-capabilities.md │ ├── tarjan/ │ │ ├── README.md │ │ ├── _CoqProject │ │ ├── extra_nocolors.v │ │ └── tarjan_nocolors.v │ └── tarjan.md ├── notes.md ├── old/ │ ├── checker.rs │ ├── inductive_serde.v │ ├── machine.md │ ├── machine.v │ ├── main.md │ ├── main.v │ └── parser_low.rs ├── posts/ │ ├── approachable-language-design.md │ ├── comparisons-with-other-projects.md │ ├── coq-for-engineers.md │ ├── crossing-no-mans-land.md │ ├── design-of-magmide.md │ ├── intro-verification-logic-in-magmide.md │ ├── iris-in-plain-terms.md │ ├── toward-termination-vcgen.md │ └── what-is-magmide.md ├── src/ │ ├── ast.rs │ ├── checker.rs │ ├── lib.rs │ ├── main.rs │ ├── old.md │ └── parser.rs └── theory/ ├── _CoqProject ├── list_assertions.v ├── main.v ├── playground.v └── utils.v ================================================ FILE CONTENTS ================================================ ================================================ FILE: .editorconfig ================================================ root = true [*] charset = utf-8 indent_style = tab end_of_line = lf insert_final_newline = true trim_trailing_whitespace = true [*.{md,yml,yaml}] indent_style = space indent_size = 2 ================================================ FILE: .github/FUNDING.yml ================================================ github: [blainehansen] ================================================ FILE: .gitignore ================================================ \#*.v\# *.glob *.vo *.vok *.vos *.aux *.d *.cmi *.cmo *.out _build Makefile Makefile.conf *.cache theorems/*.ml theorems/*.mli *.local.* *.bc # Added by cargo /target ================================================ FILE: CODE_OF_CONDUCT.md ================================================ # Code of Conduct We're using the exact same Code of Conduct as the Rust project, which [can be found online](https://www.rust-lang.org/conduct.html). ================================================ FILE: CONTRIBUTING.md ================================================ Hey there! Right now this project is optimized to be easy for me (Blaine Hansen) to work in. This means it might not be easy for anyone to jump right in, and syntax or workflow may disregard certain community standards if I find them inconvenient. I'm not really concerned with the different standards of different language communities, and if I feel a language community has made a standard choice that makes code more difficult to work with, I will completely ignore it. Although I will gladly accept pull requests that add conveniences for other setups, **I will deny any that disrupt my workflow**. I wish I had time to support other setups, but unfortunately working in Coq is very difficult and nit-picky, so the usual developer niceties such as using docker for local development aren't really practical. Here are the main things I can think of: - I run Ubuntu, so I have arranged all the scripts and build files to assume that. If you're interested in running on other systems, I'm afraid I have to leave you to your own devices. If a pull request makes a change that breaks the build on my system, I won't accept it. **I will gladly accept pull requests that make it possible to build everywhere!** However an important constraint is that [Coq interactive mode](https://packagecontrol.io/packages/Coq) must continue to work for me. If you can guide me toward a setup that allows other systems to run the build while working with Coq interactive mode, I'm happy to hear it. - [I only ever use tabs over spaces for indentation, always.](https://adamtuttle.codes/blog/2021/tabs-vs-spaces-its-an-accessibility-issue/) I will only use spaces if some irreplaceable piece of the system will literally not work if I don't (`yml` is an example). I'm more likely to simply [not use](https://github.com/avh4/elm-format/issues/158) a language if it requires spaces. You can see this choice being made in all the `dune` files throughout the project. The OCaml ecosystem seems to think that a *single* space is easy enough to read, whereas I find it extremely difficult to read (which highlights the real reason tabs are better, everyone can configure their own tab display width). - If some syntactic structure is "list-like" and supports one item per line, I will write it in a way that allows quickly adding and reordering lines without having to change the location of ending braces/parens. You can also see this in the `dune` files, where instead of using the lisp standard of placing closing parens on the same line as the last item, I place them on a new deindented line. These probably seem trite and nit-picky, and maybe they are. I just don't want to fight with this code more than is necessary. Thank you for your understanding! ================================================ FILE: Cargo.toml ================================================ [package] name = "magmide" version = "0.1.0" edition = "2021" # [lib] # name = "magmide" # path = "src/lib.rs" # crate-type = ["staticlib", "cdylib"] # [build] # # https://doc.rust-lang.org/cargo/reference/config.html # rustflags = ["-l", "LLVM-13", "-C", "link-args=-Wl,-undefined,dynamic_lookup"] # # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html [dependencies] nom = "7" # anyhow = "1.0.57" ================================================ FILE: README.future.md ================================================ # Magmide > Correct, Fast, Productive: pick three. Magmide is the first language built from the ground up to allow software engineers to productively write extremely high performance software for any computational environment, logically prove the software correct, and run/compile that code all within the same tool. The goal of the project is to spread the so-far purely academic knowledge of software verification and formal logic to a broad audience. It should be normal for engineers to create programs that are truly correct, safe, secure, robust, and performant. This file is a "by example" style reference for the features and interface of Magmide. It doesn't try to explain any of the underlying concepts, just document decisions, so you might want to read one of these other resources: - If you want to be convinced the goal of this project is both possible and necessary, please read [What is Magmide and Why is it Important?]() - If you want to learn about software verification and formal logic using Magmide, please read [Intro to Verification and Logic with Magmide](). - If you want to contribute and need the nitty-gritty technical details and current roadmap, please read [The Technical Design of Magmide](). ## Install and Use Magmide is heavily inspired by Rust and its commitment to ergonomic tooling and straightforward documentation. ```bash # install magmide and its tools curl --proto '=https' --tlsv1.2 -sSf https://sh.magmide.dev | sh # create a new project magmide new hello-world cd hello-world magmide check magmide run magmide build ``` ## Syntax Here's what we can do calling is just placing things next to each other with no commas. an *explicit* comma-separated list is always a tuple, which is why function arguments are always specified that way piping style calling uses `>functionname`. it seems that because of precedence and indentation rules which expressions are function names is always inferable? this works inline too, so `data>functionname` or `data >infix something` `>> arg arg2; expr` defines an anonymous function and immediately calls it in piping style. `>>;` is then the equivalent of your old `do` idea `--` is the "bumper" for an indented expression the sections of keywords are delimited by semicolons nested function calls are just indented since function calling is `/` is the *keyword continuation operator*, so all keywords, even possibly multi-line ones, can be defined metaprogramatically within the language ``` if yo; -- function_name arg arg >whatevs >another thing >> something; yo different something >> hm; abb >hm diff /elif yoyo; whatevs /else; dude if yo; yoyo /else; dude let thingy = if some >whatevs hmm; dude /else; yo ``` piping custom keywords can be done with a leading `;`? and standalone statement style ones are something else like `$`? custom keywords are called with a leading `;`? so something like `;route_get yoyo something; whatevs /err; dude` calling macros/known functions is indicated with something like a `~` or just the backtick thing? which means it can be done include the "backpassing" idea? or simplify it by somehow creating an "implicit callback defining pipe operator?" such as `>>>`? Magmide is whitespace/indentation sensitive. Anywhere a `;` can be used an opening indent can be used *additionally*. Anywhere a `,` can be used a newline can be used *instead*. The `:` operator is always used in some way to indicate type-like assertions. Precedence is decided using nesting with parentheses or indentation and never operator power. "Wrapping" delimiters are avoided. "Pipeability" is strongly valued. Operators are rarely used to represent actions that could be defined within the language, and instead prioritize adding new capabilities. ``` // defining computational types data Unit data Tuple; data Macro (S=undefined); | Block; BlockMacroFn | Function; FunctionMacroFn | Decorator; DecoratorMacroFn | Import; ImportMacroFn(S) alias SourceChannel S; Dict -> void fn non_existent_err macroName: str; str, str; return "Macro non-existent", "The macro "${macroName}" doesn't exist. fn incorrect_type_err macroName: str macroType: str expectedType: str ; str str ; return "Macro type mismatch", "The macro "${macroName}" is a ${macroType} type, but here it's being used as a ${expectedType} type." data CompileContext S; macros: Dict(Macro(S)) fileContext: FileContext sourceChannel: SourceChannel(S) handleScript: { path: str source: str } -> void readFile: str -> str | undefined joinPath: ..str -> str subsume: @T -> SpanResult -> Result Err: (ts.TextRange, str, str) -> Result macroCtx: MacroContext data MacroContext; Ok: @T, (T, SpanWarning[]?) -> SpanResult TsNodeErr: (ts.TextRange, str, ..str) -> SpanResult Err: (fileName: str, title: str, ..str) -> SpanResult tsNodeWarn: (node: ts.TextRange, str, ..str[]) -> void warn: (str, str, ..str[]) -> void subsume: @T, SpanResult T -> Result T, void data u8; bitarray(8) ideal Day; | monday | tuesday | wednesday | thursday | friday | saturday | sunday use Day.* rec next_weekday day: Day; match day; monday; tuesday, tuesday; wednesday, wednesday; thursday, thursday; friday friday; monday, saturday; monday, sunday; monday ideal Bool; | true | false use Bool.* rec negate b: Bool :: bool; match b; true; false false; true rec and b1: bool, b2: bool :: bool; match b1; true; b2 false; false rec or b1: bool, b2: bool :: bool; match b1; true; true false; b2 impl core.testable; rec test b: Bool :: bool; match b; true; testable.true, false; testable.false rec negate_using_test b: Bool :: bool; test b; false true ideal IndexList :: nat; | Nil :: IndexList(0) | Cons :: @n A IndexList(n) -> IndexList(n;next) rec append n1, ls1: IndexList(n1), n2, ls2: IndexList(n2) :: IndexList(n1 ;add n2); match ls1; Nil; ls2 Cons(_, x, ls1'); Cons(x, append(ls1', ls2)) prop even :: nat; | zero: even(0) | add_two: @n, even(n) -> even(n;next;next) use even.* thm four_is: even(4); prf; + add_two; + add_two; + zero thm four_is__next: even(4); prf; + (add_two 2 (add_two 0 zero)) thm plus_four: @n, even n -> even (4 ;add n); prf; => n; >>; => Hn; + add_two; + add_two; + Hn thm inversion: @n: nat, even n -> (n = 0) ;or (exists m; n = m;next;next ;and even m) ; prf; => n [| n' E'] left; _ -- right; exists n'; split _; + E' ``` ## Metaprogramming ## Interactive Tactic Mode ## Module system ``` // use a module whose location has been specified in the manifest // the manifest is essentially sugar for a handful of macros use lang{logic, compute} // the libraries 'lang', 'core', and 'std' are spoken for. perhaps though we can allow people to specify external packages with these names, we'll just give a warning that they're shadowing builtin modules // use a local module // files/directories/internal modules are all accessed with . // `__mod.mg` can act as an "module entry" for a directory, you can't shadow child files or directories // the `mod` keyword can create modules inside a file, you can't shadow sibling files or directories // `_file.mg` means that module is private, but since this is a verified language this is just a hint to not show the module in tooling, any true invariants should be fully specified with `&` use .local.nested{thing, further{nested.more, stuff}} // can do indented instead use .local.nested thing further{nested.more, stuff} whatever stuff.thingy // goes up to the project root use ~local.whatever // the module system allows full qualification of libraries, even to git repositories // the format 'name/something' defaults to namespaced libraries on the main package manager // a full git url obviously refers to that repo use person/lib.whatever // the above could be equivalent to: let person_lib = lang.pull_lib$(git: "https://github.com/person/lib") use person_lib.whatever ``` ``` use lang.{ logic, compute } // all inductive definitions use the `ind` keyword // the different kinds of types are included by default and automatically desugared to be the more "pure" versions of themselves // a union-like inductive ind Day | monday | tuesday | wednesday | thursday | friday | saturday | sunday // a record-like inductive ind Date year: logic.Nat month: logic.Nat & between(1, 12) day: logic.Nat // a tuple-like inductive ind IpAddress; logic.Byte, logic.Byte, logic.Byte, logic.Byte // the same as above but with a helper macro ind IpAddress; logic.tuple_repeat(logic.Byte, 4) // a unit-like inductive ind Unit rec next_weekday day // bring all the constructors of Day into scope use Day.* match day monday; tuesday, tuesday; wednesday, wednesday; thursday, thursday; friday friday; monday, saturday; monday, sunday; monday let next_weekday_computable = compute.logic_computable(next_weekday) let DayComputable = compute.type(next_weekday_computable).args[0].type dbg next_weekday_computable(DayComputable.monday) // outputs "Day.tuesday" // what if we were define the above types and function in the computable language? // it's as simple as changing "ind" to "type", "rec" to "fn", and ensuring all types are computable // all of these "creation" keywords are ultimately just some kind of sugar for a "let" type Day | monday | tuesday | wednesday | thursday | friday | saturday | sunday type Date year: u16 month: u8 & between(1, 12) day: u8 type Name; first: str, last: str type Pair U, T; U, T type IpAddress; u8, u8, u8, u8 type IpAddress; compute.tuple_repeat(u8, 4) type Unit fn next_weekday day use Day.* // a match implicitly takes discriminee, arms, proof of completeness match day monday; tuesday, tuesday; wednesday, wednesday; thursday, thursday; friday friday; monday, saturday; monday, sunday; monday // now no need to convert it first dbg next_weekday(Day.monday) // outputs "Day.tuesday" ``` In general, `;` is an inline delimiter between tuples, and `,` is an inline delimiter between tuple elements. Since basically every positional item in a programming language is a tuple (or the tuple equivalent record), the alteration of these two can delimit everything. Note these are only *inline* delimiters, indents are the equivalent of `;` and newlines are the equivalent of `,`. Why `;`? Because `:` is for type specification. `==` is for equality, and maps to the two different kinds of equality if it's used in a logical or computational context. ### trait system in host magmide don't need an orphan rule, just need explicit impl import and usage. the default impl is the bare one defined alongside the type, and either you always have to manually include/specify a different impl or its a semver violation to add a bare impl alongside a type that previously didn't have one ### example: converting a "logical" inductive type into an actual computable type ### example: adding an option to a computable discriminated union ### example: proving termination of a ## The embedded `core` language ## Testing talk about quickcheck and working up to a proof ## Metaprogramming Known strings given to a function Keyword macros ================================================ FILE: README.md ================================================ # :construction: Magmide is purely a research project at this point :construction: This repo is still very early and rough, it's mostly just notes, speculative writing, and exploratory theorem proving. Most of the files in this repo are just "mad scribblings" that I haven't refined enough to actually stand by! If you prefer video, this presentation talks about the core ideas that make formal verification and Magmide possible, and the design goals and intentions of the project: [![magmide talk](https://img.youtube.com/vi/Lf7ML_ErWvQ/0.jpg)](https://www.youtube.com/watch?v=Lf7ML_ErWvQ) In this readme I give a broad overview and answer a few possible questions. Enjoy! --- The goal of this project is to: **create a programming language capable of making formal verification and provably correct software practical and mainstream**. The language and its surrounding education/tooling ecosystem should provide a foundation strong enough to create verified software for any system or environment. Software is an increasingly critical component of our society, underpinning almost everything we do. It's also extremely vulnerable and unreliable. Software vulnerabilities and errors have likely caused humanity [trillions of dollars](https://www.it-cisq.org/pdf/CPSQ-2020-report.pdf) in damage, [social harm](https://findstack.com/hacking-statistics/), waste, and [lost growth opportunity](https://raygun.com/blog/cost-of-software-errors/) in the digital age (it seems clear [Tony Hoare's estimate](https://en.wikipedia.org/wiki/Tony_Hoare#Apologies_and_retractions) is way too conservative, especially if you include more than `null` errors). What would it look like if it was both possible and tractable for working software engineers to build and deploy software that was *provably correct*? Using [proof assistant languages](https://en.wikipedia.org/wiki/Proof_assistant) such as [Coq](https://en.wikipedia.org/wiki/Coq) it's possible to define logical assertions as code, and then write proofs of those assertions that can be automatically checked for consistency and correctness. Systems like this are extremely powerful, but have only been suited for niche academic applications until the fairly recent invention of [separation logic](http://www0.cs.ucl.ac.uk/staff/p.ohearn/papers/Marktoberdorf11LectureNotes.pdf). Separation logic isn't a tool, but a paradigm for making logical assertions about mutable and destructible state. The Rust ownership system was directly inspired by separation logic, which shows us that it really can be used to unlock revolutionary levels of productivity and excitement. Separation logic makes it possible to verify things about practical imperative code, rather than simply outlawing mutation and side effects as is done in functional languages. However Rust only exposes a simplified subset of separation logic, rather than exposing the full power of the paradigm. [The Iris separation logic](https://people.mpi-sws.org/~dreyer/papers/iris-ground-up/paper.pdf) was recently created by a team of academics to fully verify the correctness of the Rust type system and several core implementations that use `unsafe`. Iris is a fully powered separation logic, making it uniquely capable of verifying the kind of complex, concurrent, arbitrarily flexible assertions that could be implied by practical Rust code, even those that use `unsafe`. Iris could do the same for any other practical and realistic language. Isn't that amazing?!? A system that can prove completely and eternally that a use of `unsafe` isn't actually unsafe??!! You'd think the entire Rust and systems programming community would be over the moon! But as is common with academic projects, it's only being used to write papers rather than build real software systems. All the existing uses of Iris perform the proofs "on the side", analyzing [manual transcriptions of the source code as Coq notation](https://coq.inria.fr/refman/user-extensions/syntax-extensions.html) rather than directly reading the original source. And although the papers are more approachable than most academic papers, they're still academic papers, and so basically no working engineers have even heard of any of this. This is why I'm building Magmide, which is intended to be to Coq what Rust has been to C. There are quite a few proof languages capable of proving logical assertions in code, but none exist that are specifically designed to be used by working engineers to build real imperative programs. None have placed a full separation logic, particularly one as powerful as Iris, at the heart of their design, but instead are overly dogmatic about the pure functional paradigm. And all existing proof languages are hopelessly mired in the obtuse and unapproachable fog of [research debt](https://distill.pub/2017/research-debt/) created by the culture of academia. Even if formal verification is already capable of producing [provably safe and secure code](https://www.quantamagazine.org/formal-verification-creates-hacker-proof-code-20160920/), it isn't good enough if only professors have the time to gain the necessary expertise. We need to pull all this amazing knowledge out of the ivory tower and finally put it to work to make computing truly safe and robust. I strongly believe a world with mainstream formal verification would not only see a significant improvement in *magnitude* of social good produced by software, but a significant improvement in *kind* of social good. In the same way that Rust gave engineers much more capability to safely compose pieces of software therefore enabling them to confidently build much more ambitious systems, a language that gives them the ability to automatically check arbitrary conditions will make safe composition and ambitious design arbitrarily easier to do correctly. What kinds of ambitious software projects have been conceived but not pursued because getting them working would simply be too difficult? With machine checkable proofs in many more hands could we finally build *truly secure* operating systems, trustless networks, or electronic voting methods? How many people could be making previously unimagined contributions to computer science, mathematics, and even other logical fields such as economics and philosophy if only they had approachable tools to do so? I speculate about some possibilities at the end of this readme. To achieve this goal I've chosen an architecture I call the "split Logic/Host" architecture, where the two domains of software thinking are separated into two languages: - Logic, the dependently typed lambda calculus of constructions. This is where "imaginary" types are defined and proofs are conducted. - Host, the imperative language that actually runs on real machines. These two components must have a symbiotic relationship with one another: Logic is used to define and make assertions about Host, and Host computationally represents and implements both Logic and Host itself. ``` represents and implements +------------+------------+ | | | | | | v | | Logic +---------> Host | ^ | | | | +-------------------------+ logically defines and verifies ``` The easiest way to understand this is to think of Logic as the type system of Host. Logic is "imaginary" and only exists at compile time, and constrains/defines the behavior of Host. Logic just happens to itself be a dependently typed functional programming language! This design takes the concept of [self-hosting](https://en.wikipedia.org/wiki/Self-hosting_(compilers)) to its logical extreme. We intend to achieve this goal by [building Magmide as the Logic portion with Rust as Host, then defining the semantics of Rust *within* Magmide, and finally building a "reflective proof rule" into Magmide to allow it to use verified Rust code during proof checking.](https://github.com/magmide/magmide/blob/main/posts/design-of-magmide.md) This seems the most realistic way to bootstrap the project! I'm convinced this general architecture is the only one that can achieve Magmide's extremely ambitious goal. It feels like an optimal point in the design space, since I can't imagine another architecture that would allow all of the language components (proof checker, code compiler, target code being compiled) the possibility to be both bare metal and fully verified. But it's not good enough for the architecture to *allow* a great language design. Everything else about the design has to be chosen correctly as well. I claim that in order for the language to achieve its goal, it has to meet all these descriptions: ## Capable of arbitrary logic In order to really deliver the kind of truly transformative correctness guarantees that will inspire working engineers to learn and use a difficult new language, it doesn't make sense to stop short and only give them an "easy mode" verification tool. It should be possible to formalize and attempt to prove any proposition humanity is capable of representing logically, not only those that a fully automated tool like an [SMT solver](https://liquid.kosmikus.org/01-intro.html) can figure out. **A language with full logical expressiveness and manual proofs can still use convenient automation as well**, but the opposite isn't true. To meet this description, the language will be fully dependently typed and use the [Calculus of Constructions](https://en.wikipedia.org/wiki/Calculus_of_constructions) much like [Coq](https://en.wikipedia.org/wiki/Coq). I find [Adam Chlipala's "Why Coq?"](http://adam.chlipala.net/cpdt/html/Cpdt.Intro.html) arguments convincing in regard to this choice. Coq will also be used to bootstrap the first version of the compiler, allowing it to be self-hosting and even self-verifying using a minimally small trusted theory base. Read more about the design and bootstrapping plan in [`posts/design-of-magmide.md`](./posts/design-of-magmide.md). The [metacoq](https://github.com/MetaCoq/metacoq) and ["Coq Coq Correct!"](https://metacoq.github.io/coqcoqcorrect) projects have already done the work of formalizing and verifying Coq using Coq, so they will be very helpful. It's absolutely possible for mainstream engineers to learn and use these powerful logical concepts. The core ideas of formal verification (dependent types, proof objects, higher order logic, separation logic) aren't actually that complicated. They just haven't ever been properly explained because of [research debt](https://distill.pub/2017/research-debt/), and they weren't even all that practical before separation logic and Iris. I've been working on better explanations in the (extremely rough and early) [`posts/intro-verification-logic-in-magmide.md`](./posts/intro-verification-logic-in-magmide.md) and [`posts/coq-for-engineers.md`](./posts/coq-for-engineers.md). ## Capable of bare metal performance Software needs to perform well! Not all software has the same requirements, but often performance is intrinsically tied to correct execution. Very often the software that most importantly needs to be correct also most importantly needs to perform well. **If the language is capable of truly bare metal performance, it can still choose to create easy abstractions that sacrifice performance where that makes sense.** To meet this description Magmide will be built in and deeply integrated with Rust. Excitingly because of the inherent power and flexibility of a proof assistant this integration with Rust doesn't have to be permanent, and we could build other languages to act as Host as long as we can specify their semantics and make them interoperable! Because of separation logic and Iris, it is finally possible to verify code as low-level as Rust and more! ## Gradually verifiable Just because it's *possible* to fully verify all code, doesn't mean it should be *required*. It simply isn't practical to try to completely rewrite a legacy system in order to verify it. **We must be able to write code without needing to prove it's perfectly correct**, otherwise iteration and incremental adoption are impossible. Existing languages with goals of increased rigor such as Rust and Typescript strategically use concessions in the language such as `unsafe` and `any` to allow more rigorous code to coexist with legacy code as it's incrementally replaced. The only problem is that these concessions introduce genuine soundness gaps into the language, and it's often difficult or impossible to really understand how exposed your program is to these safety gaps. We can get both practical incremental adoption and complete understanding of the current safety of our program by leveraging work done in the [Iron obligation management logic](https://iris-project.org/pdfs/2019-popl-iron-final.pdf) built using Iris. We can use a concept of trackable effects to allow some safety conditions to be *optional*. Trackable effects will work by requiring a piece of some "correctness token" to be forever given up in order to perform a dangerous operation without justifying its safety with a proof. This would infect the violating code block with an effect type that will bubble up through any parent blocks. Defining effects in this way makes them completely composable *resources* rather than *wrappers*, meaning that they're more flexible and powerful than existing effect systems. Systems like algebraic effects or effect monads could be implemented using this resource paradigm, but the opposite isn't true. If the trackable effect system is defined in a sufficiently generic way then custom trackable effects could be created, allowing different projects to introduce new kinds of safety and correctness tracking, such as ensuring asynchronous code doesn't block the executor, or a web app doesn't render raw untrusted input, or a server doesn't leak secrets. Even if a project chooses to ignore some effects, they'll always know those effects are there, which means other possible users of the project will know as well. Project teams could choose to fail compilation if their program isn't memory safe or could panic, while others could tolerate some possible effects or write proofs to assert they only happen in certain well-defined circumstances. It would even be possible to create code that provably sandboxes an effect by ensuring it can't be detected at any higher level if contained within the sandbox. With all these systems in place, we can finally have a genuinely secure software ecosystem! ## Fully reusable We can't write all software in assembly language! Including first-class support for powerful metaprogramming, alongside a [query-based compiler](https://ollef.github.io/blog/posts/query-based-compilers.html), will allow users of this language to build verified abstractions that "combine upward" into higher levels, while still allowing the possibility for those higher levels to "drop down" back into the lower levels. Being a proof assistant, these escape hatches don't have to be unsafe, as higher level code can provide proofs to the lower level to justify its actions. This ability to create fully verifiable higher level abstractions means we can create a "verification pyramid", with excruciatingly verified software forming a foundation for a spectrum of software that decreases in importance and rigor. **Not all software has the same constraints, and it would be dumb to to verify a recipe app as rigorously as a cryptography function.** But even a recipe app would benefit from its foundations removing the need to worry about whole classes of safety and soundness conditions. And wouldn't it be great to prove your app will never leak memory or throw exceptions or enter an infinite loop/recursion? Magmide *itself* doesn't have to achieve mainstream success to massively improve the quality of all downstream software, but merely some sub-language. Many engineers have never heard of LLVM, but they still implicitly rely on it every day. Magmide would seek to do the same. We don't have to make formal verification fully mainstream, we just have to make it available for the handful of people willing to do the work. If a full theorem prover is sitting right below the high-level language you're currently working in, you don't have to bother with it most of the time, but you still have the option to do so when it makes sense. The metaprogramming can of course also be used directly in the dependently typed language, allowing compile-time manipulation of proofs, functions, and data. Verified proof tactics, macros, and higher-level embedded programming languages are all possible. This is the layer where absolutely essential proof automation tactics similar to Coq's `auto` or [Adam Chlipala's `crush`](http://adam.chlipala.net/cpdt/html/Cpdt.Intro.html), or fast counter-example searchers such as `quickcheck`, or [computational reflection systems](./posts/design-of-magmide.md#heavy-use-of-computational-reflection-to-improve-proof-performance) would be implemented. Importantly, the language will be self-hosting, so metaprogramming functions will benefit from the same bare metal performance and full verifiability. You can find rough notes about the current design thinking for the metaprogramming interface in [`posts/design-of-magmide.md`](./posts/design-of-magmide.md). ## Practical and ergonomic My experience using languages like Coq has been extremely painful, and the interface is "more knife than handle". I've been astounded how willing academics seem to be to use extremely clunky workflows and syntaxes just to avoid having to build better tools. To meet this description, this project will learn heavily from `cargo` and other excellent projects. **It should be possible to verify, interactively prove, and query Magmide code with a single tool.** The split Logic/Host architecture will likely make it easier to understand and use Magmide. It will also fully embrace ergonomic type inference, and use techniques such as those from ["Flux: Liquid Types for Rust"](https://arxiv.org/abs/2207.04034) to allow even many *proof* conditions to be inferred. ## Taught effectively **Working engineers are resource constrained and don't have years of free time to wade through arcane and disconnected academic papers.** Academics aren't incentivized to properly explain and expose their amazing work, and a massive amount of [research debt](https://distill.pub/2017/research-debt/) has accrued in many fields, including formal verification. To meet this description, this project will enshrine the following values in regard to teaching materials: - Speak to a person who wants to get something done and not a review committee evaluating academic merit. - Put concrete examples front and center. - Point the audience toward truly necessary prerequisites rather than assuming shared knowledge. - Prefer graspable human words to represent ideas, never use opaque and unsearchable non-ascii symbols, and only use symbolic notations when it's both truly useful and properly explained. - Prioritize the hard work of finding clear and distilled explanations. --- Read [`posts/design-of-magmide.md`](./posts/design-of-magmide.md) or [`posts/comparisons-with-other-projects.md`](./posts/comparisons-with-other-projects.md) to more deeply understand the intended design and how it's different than other projects. Building such a language is a massively ambitious goal. It might even be too ambitious! But we have to also consider the opposite: perhaps previous projects haven't been ambitious enough, and that's why formal verification is still niche! Software has been broken for too long, and we won't have truly solved the problem until it's at least *possible* for all software to be verified. # FAQ ## Is it technically possible to build a language like this? Yes! None of the technical details of this idea are untested or novel. Dependently typed proof languages, higher-order separation logic, query-based compilers, introspective metaprogramming, and abstract assembly languages are all ideas that have been proven in other contexts. Magmide would merely attempt to combine them into one unified and practical package. ## Is this language trying to replace Rust? No! My perfect outcome of this project would be for it to sit *underneath* Rust, acting as a new verified toolchain that Rust could "drop into". The concepts and api of Rust are awesome and widely loved, so Magmide would just try to give it a more solid foundation. ## If this is such a good idea why hasn't it happened yet? Mostly because this idea exists in an "incentive no man's land". Academics aren't incentivized to create something like this, because doing so is just "applied" research which tends not to be as prestigious. You don't get to write many groundbreaking papers by taking a bunch of existing ideas and putting them together nicely. Software engineers aren't incentivized to create something like this, because a programming language is a pure public good and there aren't any truly viable business models that can support it while still remaining open. Even amazing public good ideas like the [interplanetary filesystem](https://en.wikipedia.org/wiki/InterPlanetary_File_System) can be productized by applying the protocol to markets of networked computers, but a programming language can't really pull off that kind of maneuver. Although the software startup ecosystem does routinely build pure public goods such as databases and web frameworks, those projects tend to have an obvious and relatively short path to being useful in revenue-generating SaaS companies. The problems they solve are clear and visible enough that well-funded engineers can both recognize them and justify the time to fix them. In contrast the path to usefulness for a project like Magmide is absolutely not short, and despite promising immense benefits to both our industry and society as a whole, most engineers capable of building it can't clearly see those benefits behind the impenetrable fog of research debt. We only got Rust because Mozilla has been investing in dedicated research for a long time, and it still doesn't seem to have really financially paid off for them in the way you might hope. ## Will working engineers actually use it? Maybe! We can't force people or guarantee it will be successful, but we can learn a lot from how Rust has been able to successfully teach quite complex ideas to an huge and excited audience. I think Rust has succeeded by: - *Making big promises* in terms of how performant/robust/safe the final code can be. - *Delivering on those promises* by building something awesome. I hope that since the entire project will have verification in mind from the start it will be easier to ship something excellent and robust with less churn than usual. - *Respecting people's time* by making the teaching materials clear and distilled and the tooling simple and ergonomic. All of those things are easier said than done! Fully achieving those goals will require work from a huge community of contributors. ## Won't writing verified software be way more expensive? Do you actually think this is worth it? **Emphatically yes it is worth it.** As alluded to earlier, broken software is a massive drain on our society. Even if it were much more expensive to write verified software, it would still be worth it. Rust has already taught us that it's almost always worth it to [have the hangover first](https://www.youtube.com/watch?v=ylOpCXI2EMM&t=565s&ab_channel=Rust) rather than wastefully churn on a problem after you thought you could move on. Verification is obviously very difficult. Although I have some modest theories about ways to speed up/improve automatic theorem proving, and how to teach verification concepts in a more intuitive way that can thereby involve a larger body of engineers, we still can't avoid the fact that refining our abstractions and proving theorems is hard and will remain so. But we don't have to make verification completely easy and approachable to still get massive improvements. We only have to make proof labor more *available* and *reusable*. Since Magmide will be inherently metaprogrammable and integrate programming and proving, developments in one project can quickly disseminate through the entire language community. Research would be much less likely to remain trapped in the ivory tower, and could be usefully deployed in real software much more quickly. And of course, a big goal of the project is to make verification less expensive! Tooling, better education, better algorithms and abstractions can all decrease verification burden. If the project ever reaches maturity these kinds of improvements will likely be most of the continued effort for a long time. Besides, many projects already write [absolutely gobs of unit tests](https://softwareengineering.stackexchange.com/questions/156883/what-is-a-normal-functional-lines-of-code-to-test-lines-of-code-ratio), and a proof is literally *infinitely* better than a unit test. At this point I'm actually hopeful that proofs will *decrease* the cost of writing software. We'll see. ## Is it actually useful to prove code meets some specification if we still have to trust the specification? In a way yes this is true: when we prove an implementation meets some specification we're mostly just shifting uncertainty/trust from the implementation to the specification. This is part of why it's impossible for our systems to ever be completely perfect (whatever "perfect" means). However I assert that this shifting of trust from code to specifications (or put another way, from trusted code to trusted theory) is worth the effort and a huge improvement over the status quo for these reasons: - Specifications can refer to each other and be built upon, thereby revealing inconsistent assumptions and shaking out errors. Every time an incorrect specification in any way interfaces with a correct one then the incompatibility between them will be revealed at compile time. It's likely you've already experienced exactly this dynamic when you incorrectly define a *type* (type systems are just very simple proof systems!). If you mistakenly define a type field as an unsigned integer when it needs to be a signed integer, when you try to use the incorrect type in other code that expects a signed integer your mistake will be revealed. This won't always happen, but with deeper proof systems it has the opportunity to happen even more often than it happens in type systems. - Specifications can be much smaller and terser than implementations, and therefore easier to audit. When we audit a specification we only have to audit the type signatures of our theorems and functions, rather than all the code inside them. Implementations have to worry about performance and many internal details that don't need to be revealed, whereas specifications only have to make assertions about whatever visible behavior is desired. Specifications can be stated in the whatever naive, simple, pure functional form makes the assertion easy to understand, whereas implementations often need to use arcane tricks and confusingly evolving mutable structures to make the algorithm efficient. If the specification is larger than the implementation I would tend to suspect one or both of them could be structured more intelligently. ## Do you think this language will make all software perfectly secure? No! Although it's certainly [very exciting to see how truly secure verified software can be](https://www.quantamagazine.org/formal-verification-creates-hacker-proof-code-20160920/), there will always be a long tail of hacking risk. Not all code will be written in securable languages, not all engineers will have the diligence or the oversight to write secure code, people can make bad assumptions, and brilliant hackers might invent entirely new *types* of attack vectors that aren't considered by our safety specifications (although inventing new attack vectors is obviously way more difficult than just doing some web searches and running scripts, which is all a hacker has to do today). However *any* verified software is better than *none*, and right now it's basically impossible for a security-conscious team to even attempt to prove their code secure. Hopefully the "verification pyramid" referred to earlier will enable almost all software to quickly reuse secure foundations provided by someone else. And of course, social engineering and hardware tampering are never going away, no matter how perfect our software is. ## Is logically verifying code even useful if that code relies on possibly faulty software/hardware? This is nuanced, but the answer is still yes! First let's get something out of the way: software is *literally nothing more* than a mathematical/logical machine. It is one of the very few things in the world that can actually be perfect. Of course this perfection is in regard to an axiomatic model of a real machine rather than the true machine itself. But isn't it better to have an implementation that's provably correct according to a model rather than what we have now, an implementation that's obviously flawed according to a model? Formal verification is really just the next level of type checking, and type checking is still incredibly useful despite also only relating to a model. If you don't think a logical model can be accurate enough to model a real machine in sufficient detail, please check out these papers discussing [separation logic](http://www0.cs.ucl.ac.uk/staff/p.ohearn/papers/Marktoberdorf11LectureNotes.pdf), extremely high fidelity formalizations of the [x86](http://nickbenton.name/hlsl.pdf) and [arm](https://www.cl.cam.ac.uk/~mom22/arm-hoare-logic.pdf) instruction sets, and [Iris](https://people.mpi-sws.org/~dreyer/papers/iris-ground-up/paper.pdf). Academics have been busy doing amazing stuff, even if they haven't been sharing it very well. If you think we'll constantly be tripping over problems in incorrectly implemented operating systems or web browsers, well you're missing the whole point of this project. These systems provide environments for other software yes, but they're still just software themselves. Even if they aren't perfectly reliable *now*, the entire ambition of this project is to *make* them reliable. We would however need hardware axioms to model the abstractions provided by a concrete computer architecture, and this layer is trickier to be completely confident in. Hardware faults and ambient problems of all kinds can absolutely cause unavoidable data corruption. Hardware is intentionally designed with layers of error correction and redundancy to avoid propagating corruption, but it still gets through sometimes. There's one big reason to press on with formal verification nonetheless: the possibility of corruption or failure can be included in our axioms! Firmware and operating systems already include state consistency assertions and [error correction codes](https://en.wikipedia.org/wiki/Error_detection_and_correction), and it would be nice if those checks themselves could be verified. The entire purpose of trackable effects is to allow environmental assumptions to be as high fidelity and stringent as possible without requiring every piece of software to actually care about all that detail. This means the lowest levels of our verification pyramid can fully include the possibility of corruption and carefully prove it can only cause a certain amount of damage in a few well-understood places. Then the higher levels of the pyramid can build on top of that much sturdier foundation. Additionally the concept of [corruption panics](./posts/design-of-magmide.md#corruption-panics) would allow software to include consistency checks even in situations that are logically impossible, to account for situations where the hardware has failed. Yes it's true that we can only go so far with formal verification, so we should always remain humble and remember that real machines in the real world fail for lots of reasons we can't control. But we can go much much farther with formal verification than we can with testing alone! Proving correctness against a mere model with possible caveats is incalculably more robust than doing the same thing we've been doing for decades. ## Why can't you just teach people how to use existing proof languages like Coq? The short answer is that languages like Coq weren't designed with the intent of making formal verification mainstream, so they're all pretty mismatched to the task. If you want a deep answer to this question both for Coq and several other projects, check out [`posts/comparisons-with-other-projects.md`](./posts/comparisons-with-other-projects.md). This question is a lot like asking the Rust project creators "why not just write better tooling and teaching materials for C"? Because instead of making something *awesome* we'd have to drag around a bunch of frustrating design decisions. Sometimes it's worth it to start fresh. ## Isn't is undecidable to prove a program terminates or is correct? If I was claiming Magmide could somehow ignore the problem of [undecidability](https://en.wikipedia.org/wiki/Decidability_(logic)) (or the [halting problem](https://en.wikipedia.org/wiki/Halting_problem), or [Rice's theorem](https://en.wikipedia.org/wiki/Rice%27s_theorem), or [Godel's incompleteness theorems](https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems)) then this question would be a useful one. However I'm *not* claiming that, which means you just haven't understood Magmide and its goals. It's impossible to write an algorithm that can *automatically* and *without any guidance* determine whether *any arbitrary program* terminates/meets some non-trivial semantic condition. However it is possible to write algorithms that can do so *some* of the time. And it's *always* possible to use dependent type theory to check whether a proof object successfully proves some proposition. *Checking* proofs is decidable, it's only *constructing* proofs that's in general undecidable. Researchers routinely prove that *particular* programs terminate or have certain characteristics, and they often have to manually write proofs to do so. Nothing in any of these documents claims we can ignore proven truths of logic. Magmide is just trying to integrate proven concepts (proof assistants and bare metal compilers) into a nice package. I'm not an expert logician, and I'm happy to be corrected by more knowledgeable people. But if you're asking questions like this, you've simply misunderstood either Magmide or the referenced theorems. ## Isn't formal verification impractical in practice? Historically systems have been very impractical yes, with three commonly cited issues: - Extreme difficulty of composing proofs. - Overly long and burdensome correctness annotations. - Combinatorial explosion of proof terms or constraints, leading to unacceptable proof checking time. I'm not terribly worried about composability, since separation logic systems such as Iris have demonstrated how much improvement the right abstractions can give. And I'm betting design features such as [asserted types](./posts/design-of-magmide.md#builtin-asserted-types), [inferred annotations](https://arxiv.org/abs/2207.04034), and [inferred proof holes](./posts/design-of-magmide.md#inferred-proof-holes) would make composing verified functions much more ergonomic. Ergonomics and abstractions can be improved over time, especially for specific classes of problems. We shouldn't throw out the entire idea of verification just because previous systems have had poor ergonomics. I'm extremely excited about the already mentioned ["Flux: Liquid Types for Rust"](https://arxiv.org/abs/2207.04034) project, which demonstrated it's possible to ergonomically infer proof annotations. Essentially (mostly) all a programmer must do is add correctness conditions to *types* (just like [asserted types](./posts/design-of-magmide.md#builtin-asserted-types)) and (basically) all the other program annotations can be inferred. Flux then sends all those conditions to a solver and doesn't allow manual proofs for more complex conditions, but Magmide would allow manual proofs, meaning the correctness conditions could be arbitrarily interesting. As for questions like combinatorial explosion of verification conditions, it's absolutely true that all the computational work necessary to verify software can indeed be very expensive, especially if the proof system in question is fully automated and just generates a massive list of constraints to solve. A few techniques can help us improve the situation: - [Incremental compilation of proof terms](https://github.com/salsa-rs/salsa). - [Computational reflection](https://gmalecha.github.io/reflections/2017/speeding-up-proofs-with-computational-reflection). For many specific problem domains it's possible to write very targeted decidable algorithms to find proofs or at least discharge many trivial proof obligations (the Rust borrow checker is an example!). Since such algorithms are narrowly targeted at a specific domain, they can perform much better than a general purpose tactic or constraint solver. - Allowing manual/interactive proofs rather than requiring full automation. This may seem like a cop-out, and it certainly adds work for engineers, but if some theorem is simple to manually prove but would lead an automated system on a costly run through a massive search space, it's probably worth the time. Just like ergonomics, compiler performance can be improved over time. Type systems can potentially add a huge amount of usability pain and compilation cost, but if the right design tradeoffs are found then type systems are well worth the trouble. Proof systems are simply much more advanced type systems, and I'm willing to bet the combination of Iris and a few of the design ideas I've referenced can achieve a worthwhile set of tradeoffs. ## Do you really think non-experts can meaningfully contribute here? Aren't you ignoring the difficult problems that researchers still haven't solved? This question is a useful one to ask, but I ultimately think it's wrong-headed. I make this claim: **the most important bottleneck to the broader adoption and application of formal methods isn't unsolved research problems, but the "day one" problems of ergonomic usability and connected reusability.** Importantly, I only make this claim because Iris exists, which demonstrated the ability to verify extremely complex and realistic Rust code. Most of the software that's written every day isn't that complicated. Most of the correctness conditions people will actually care to prove will either relate to safety/security or to general robustness (not leaking memory, not throwing exceptions, not going into infinite loops/recursions), conditions that have been very rigorously explored by researchers. The research cutting edge is lightyears ahead of engineering practice, and we don't have to apply the full depth of theory to get huge payoffs in the general safety and stability of software. Researchers will continue to find solutions to difficult theoretical problems, which is great. But as long as their solutions only exist in difficult to reuse media such as Coq or pdf papers, those solutions will barely matter. Amazing theoretical progress hasn't truly fulfilled its purpose until it has *somehow* been applied to the real world. So instead of saying "we should wait for researchers to solve all these difficult problems", I propose we build a highly usable system *now* with the theory we already have. If such a system existed, even researchers would benefit, since they would have a place to contribute further breakthroughs that would give them more visibility and support and return contributions. Magmide just wants to give both industrial engineers and academic researchers a solid foundation, one they can share and build up together. ## Why build a system focused on engineers when even academics don't always use proof assistants? Shouldn't we try to build a system researchers will use first? No. If you create a tool that allows practical verification of real software systems, primarily intended for approachable use by engineers, you'll necessarily have created a theorem prover that's enjoyable and ergonomic to use, and that supports easy sharing and reuse of proof labor across an entire community. That design doesn't in any way preclude supporting the patterns that researchers like (using/supporting homotopy type theory, allowing concise notation using a flexible metaprogramming system, rendering proofs as latex/pdf/html/whatever documents). A highly metaprogrammable bare metal proof assistant would attract researchers, but a beautiful theorem prover without any special capability to reason about or compile bare metal code wouldn't attract engineers. Think about it: tons of researchers use python to analyze data or automate common tasks, or focus their research on the details of C or Rust or some specific instruction set architecture. Many fewer use Coq or do research about Coq. In general, at least in computing, researchers tend to follow industrial engineers. The verification use cases engineers care about are more specific and fully implied by those that researchers care about. If we nail the use cases engineers care about, we'll get the use cases researchers care about basically for free. ## Isn't most software too fuzzy or quickly evolving to make verification worth the effort? Yes, many systems don't really have a clear definition of "correct", but that doesn't mean *aspects* of the system aren't worth verifying, or that it wouldn't be worth building that system *using* verified tools. We don't have to be able to verify every facet of every program to make verification worth the effort, we just have to be able to prove enough useful things that we can't already prove with existing type systems. Refer to the concept of the [verification pyramid discussed above](https://github.com/magmide/magmide#fully-reusable). ## Why bother writing code and then verifying it when we could instead simply generate code from specifications? Generating code based on specifications is an extremely cool idea! [Some researchers have already made extremely interesting strides in that direction.](https://plv.csail.mit.edu/fiat/) It seems impossible to always generate code for *any* specification, since some specifications aren't true or are undecidable. I'm not even sure it would always be possible for even relatively mundane code (reach out to me if you know more about the related theory!) Regardless of the theoretical limits of the approach, deductive synthesis systems have to be built *from* something, and compile *to* something. That something ought to be a proof language capable of bare metal performance, so Magmide would be a perfect fit for creating deductive synthesis systems. ## How far are you? What remains to be done? Very early, and basically everything remains to be done! I've been playing with models of very simple assembly languages to get my arms around formalization of truly imperative execution. Especially interesting has been what it looks like to prove some specific assembly language program will always terminate, and to ergonomically discover paths in the control flow graph which require extra proof justification. I have some raw notes and thoughts about this in [`posts/toward-termination-vcgen.md`](./posts/toward-termination-vcgen.md). Basically I've been playing with the design for the foundational computational theory. In [`posts/design-of-magmide.md`](./posts/design-of-magmide.md) I outline my guess at the project's major milestones. Obviously a project as gigantic as this can only be achieved by inspiring a lot of hardworking people to come and make contributions, so each milestone will have to show exciting enough capability to make the next milestone happen. Read [this blog post discussing my journey to this project](https://blainehansen.me/post/my-path-to-magmide/) if you're interested in a more personal view. ## This is an exciting idea! How can I help? Just reach out! Since things are so early there are many questions to be answered, and I welcome any useful help. Feedback and encouragement are welcome, and you're free to reach out to me directly if you think you can contribute in some substantial way. If you would like to get up to speed with formal verification and Coq enough to contribute at this stage, you ought to read [Software Foundations](https://softwarefoundations.cis.upenn.edu/), [Certified Programming with Dependent Types](http://adam.chlipala.net/cpdt/html/Cpdt.Intro.html), [this introduction to separation logic](http://www0.cs.ucl.ac.uk/staff/p.ohearn/papers/Marktoberdorf11LectureNotes.pdf), and sections 1, 2, and 3 of the [Iris from the ground up](https://people.mpi-sws.org/~dreyer/papers/iris-ground-up/paper.pdf) paper. You might also find my unfinished [introduction to verification and logic in Magmide](./posts/intro-verification-logic-in-magmide.md) useful, even if it's still very rough. Here's a broad map of all the mad scribblings in this repo: - `theory` contains exploratory Coq code, much of which is unfinished. This is where I've been playing with designs for the foundational computational theory. - `src`, `plugins`, and `test_theory` contains Rust, Ocaml, and Coq code representing the current skeleton of the [initial bootstrapping toolchain](./posts/design-of-magmide.md#project-plan). - `posts` has a lot of speculative writing, mostly to help me nail down the goals and design of the project. - `notes` has papers on relevant topics and notes I've made purely for my own learning. - `notes.md` is a scratchpad for raw ideas, usually ripped right from my brain with very little editing. - `README.future.md` is speculative writing about a "by example" introduction to the language. I've been toying with different syntax ideas there, and have unsurprisingly found those decisions to be the most difficult and annoying :cry: Thank you! Hope to see you around! --- # What could we build with Magmide? A proof checker with builtin support for metaprogramming and verification of assembly languages would allow us to build any logically representable software system imaginable. Here are some rough ideas I think are uniquely empowered by the blend of capabilities that would be afforded by Magmide. Not all of these ideas are *only* possible with full verification, but I feel they would get much more tractable. ## Truly eternal software This is a general quality, one that could apply to any piece of software. With machine checked proofs, it's possible to write software *that never has to be rewritten or maintained*. Of course in practice we often want to add features or improve the interface or performance of a piece of software, and those kind of expected improvements can't be anticipated enough to prove them ahead of time. But if the intended function of a piece of software is completely understood and won't significantly evolve, it's possible to get it right *once and for all*. Places where this would be a good idea are places where it's hard to get to the software, such as in many embedded systems like firmware, IOT applications, software in spacecraft, etc. ## Safe foreign code execution without sandboxing If it's possible to prove a piece of code is well-behaved in arbitrary ways then it's possible to simply run foreign and untrusted code without any kind of sandboxing or resource limitations, as long as that foreign code provides a consistent proof object demonstrating it won't cause trouble. What kind of performance improvements and increased flexibility could we gain if layers like operating systems, hypervisors, or even internet browsers only had to type check foreign code to know it was safe to execute with arbitrary system access? Of course we still might deem this too large a risk, but it's interesting to imagine. ## Verified critical systems Many software applications are critical for safety of people and property. It would be nice if applications in aeronautics, medicine, industrial automation, cars, banking and finance, decentralized ledgers, and all the others were fully verified. ## Secure voting protocols It isn't good enough for voting machines to be provably secure, the voting system itself must be cryptographically transparent and auditable. The [ideal requirements](https://en.wikipedia.org/wiki/End-to-end_auditable_voting_systems) are extremely complex, and would be very difficult to get right without machine checked proofs. Voting is sufficiently high stakes that it's extremely important for a voting infrastructure to not simply be correct, but be *undeniably* correct. I imagine it will be much easier to assert the fairness and legitimacy of voting results if all the underlying code is much more than merely audited and tested. ## Universally applicable type systems Things like the [Underlay](https://research.protocol.ai/talks/the-underlay-a-distributed-public-knowledge-graph/) or the [Intercranial Abstraction System](https://research.protocol.ai/talks/the-inter-cranial-abstraction-system-icas/) get much more exciting in a world with a standardized proof checker syntax to describe binary type formats. If a piece of data can be annotated with its precise logical format, including things like endianness and layout semantics, then many more pieces of software can automatically interoperate. I'm particularly excited by the possibility of improving the universality of self-describing apis, ones that allow consumers to merely point at some endpoint and metaprogrammatically understand the protocol and type interface. ## Truly universal interoperability All computer programs in our world operate on bits, and those bits are commonly interpreted as the same few types of values (numbers, strings, booleans, lists, structures of those things, standardized media types). In a world where all common computation environments are formalized and programs can be verified to correctly model common logical types in any of those common computation environments, then correct interoperation between those environments can also be verified! It would be very exciting to know with deep rigorous certainty that a program can be compiled for a broad host of architectures and model the same logical behavior on all of them. ## Semver enforcing and truly secure package management Since so much more knowledge of a package's api can be had with proof checking and trackable effects, we can have distributed package management systems that can enforce semver protocols at a much greater granularity and ensure unwanted program effects don't accidentally (or maliciously!) sneak into our dependency graphs. ## Invariant protection without data hiding In many languages some idea of encapsulation or data hiding is supported by the language, to allow component authors to ensure outside components don't reach into data structures and break invariants. With proof checking available, it's possible to simply encode invariants directly alongside data, effectively making arbitrary invariants a part of the type system. When this is true data no longer has to be hidden at the type system level. We can still choose to make some data hidden from documentation, but doing so would simply be for clarity rather than necessity. Removing the need for data hiding allows us to reconsider almost all common software architectures, since most are simply trying to enforce consistency with extra separation. Correct composition can be easy and flexible, so we can architect systems for greatest performance or clarity and remove unnecessary walls. For example strict microservice architectures might lose much of their usefulness. ## Flattened async executor micro-kernel operating system The process model is a very good abstraction, but the main reason it's useful is because it creates hard boundaries around different programs to prevent them from corrupting each other's state. Related to the above point, what if we don't have to do that anymore? What if code from different sources could simply inhabit the same memory space without much intervention? The Rust community has made some very innovative strides with their asynchronous executor implementations, and I am one person who believes the "async task" paradigm is an extremely natural way to think about system concurrency and separation. What if an async task executor could simply be the entire operating system, doing nothing but managing task scheduling and type checking new code to ensure it will be well-behaved? In this paradigm, the abstractions offered by the operating system can be moved into a *library* instead of being offered at runtime, and can use arbitrary capability types to enforce permissions or other requirements. Might such a system be both much more performant and simpler to reason about? ## Metaprogrammable multi-persistence database Most databases are designed to run as an isolated service to ensure the persistence layer is always in a consistent state that can't accidentally be violated by user code. With proof invariants this isn't necessary, and databases can be implemented as mere libraries. Immutable update logs have proven their value, and with proof checking it would be much easier to correctly build "mutable seeming" materialized views based on update commands. Databases could more easily save multiple materialized views at different scales in different formats. ## More advanced memory ownership models Rust has inspired many engineers with the beautiful and powerful ideas of ownership and reference lifetimes, rooting out many tricky problems before they arise. However the model is too simple for many obviously correct scenarios, such as mutation of a value from multiple places within the same thread, or pointers in complex data structures that still only point to ownership ancestors or strict siblings such as is the case in doubly-linked lists. More advanced invariants and arbitrary proofs can solve this problem. ## Reactivity systems that are provably free from leaks, deadlocks, and cycles Reactive programming models have become ubiquitous in most user interface ecosystems, but in order to make sense they often rely on the tacit assumption that user code doesn't introduce resource leaks or deadlocks or infinite cycles between reactive tasks. Verification can step in here, and produce algorithms that enforce tree-like structures for arbitrary code. ================================================ FILE: iris-notes.md ================================================ > “number of steps of computation that the program may perform”. This intuition is not entirely correct, but it is close enough. VJAKδ is now a predicate over both a natural number k ∈ N and a closed value v. Intuitively, (k,v) ∈ VJAKδ means that no well-typed program using v at type A will “go wrong” in k steps (or less). what does it mean for something to hold for k steps? > iProp is obtained from a more general construction: uniform predicates over a unital resource algebra M, written UPred(M). The type UPred(M) consists of predicates over step-indices and resources (from M) which are down-closed with respect to the step-index and up-closed with respect to the resource: UPred(M) := {P ∈ Prop(N, M) | ∀(n, a) ∈ P. ∀m, b. m ≤ n ⇒ a ;included b ⇒ (m, b) ∈ P} so if some (n, a) is "proven", then so is any (m, b) where both m is `<=` (earlier than or same?) n and b is `>=` (includes or same) a so you can take a valid (n, a) and make it either closer in number of steps or involving a larger piece of resource algebra state? - how does step-indexing *actually* relate to program steps? - are the step indexes only ever `infinity` or `1`? ``` In the base case, when the argument is a value v, we have to prove the postcondition Q(v) (after potentially) updating the ghost state. Otherwise, if e is a proper expression, we get to assume the state interpretation SI(h) (explained below) and have to show two conditions: (1) the current expression e can make progress in the heap h where progress(e, h) := ∃e 0 , h0 . (e, h) ❀ (e 0 , h0 ) and (2) for any successor expression e 0 and heap h 0 , we have to show the weakest precondition and the state interpretation after an update to the ghost state and after a later. The updates in both cases makes sure that we can always update our ghost state when we prove a weakest precondition. These updates are instrumental for working with the state interpretation below and for verifying code which relies on auxiliary ghost state. The later in the second case ensures that the weakest precondition can be defined as a guarded fixpoint. Moreover, it ties program steps to laters in our program logic (i.e., in the rules LaterPureStep, LaterNew, LaterLoad, and LaterStore). In fact, this later in the definition of the weakest precondition is responsible for the intuition: “. P means P holds after the next step of computation”. More concretely, if one proves a weakest precondition wp e {v. Q(v)} under the assumption . P then, after the next step of computation, the goal becomes .wp e 0 {v. Q(v)}. We can then use the rule LaterMono to remove the later in front of wp e 0 {v. Q(v)} and in front of . P. ``` the prefix `TC` is "typeclass" and comes from stdpp. it seems they've redefined a bunch of the basic operators in coq (eq, and, or, forall, etc) as typeclasses? `bi` == bunched implications, which is just the logical ideas of separation logic (* operator as resource composition, -* like a "resource function" that can take resources and transform them, etc) `si` == step-indexed, still don't entirely get the intuition behind step indexed relations, but whatever `coPset` == set of positive binary numbers. `co` is for the idea of "cofiniteness"? a subset is `co`finite if it's `co`mplement is finite. it looks like `coPset`s are used as the "masks"? the sets that hold ghost variable/invariant names? `E` is generally used for masks `Canonical` is just a command for making some typeclass instance available to coq's type inference, so it can be found automatically `Structure` is the same as `Record`!!!! `lb` == lower bound `%I` means to resolve in `bi_scope` Leibniz equality is the kind where two things are equal if all propositions that are true for one are true for the other `|==>` is `bupd`, or basic update `P ==∗ Q` is `(P ⊢ |==> Q)`, so P entails you can get an updatable Q, using separation logic entailment confusingly it can also mean `(P -∗ |==> Q)` in bi_scope? ``` Class BUpd (PROP : Type) : Type := bupd : PROP → PROP. Notation "|==> Q" := (bupd Q) : bi_scope. Notation "P ==∗ Q" := (P ⊢ |==> Q) (only parsing) : stdpp_scope. Notation "P ==∗ Q" := (P -∗ |==> Q)%I : bi_scope. Class FUpd (PROP : Type) : Type := fupd : coPset → coPset → PROP → PROP. Notation "|={ E1 , E2 }=> Q" := (fupd E1 E2 Q) : bi_scope. Notation "P ={ E1 , E2 }=∗ Q" := (P -∗ |={E1,E2}=> Q)%I : bi_scope. Notation "P ={ E1 , E2 }=∗ Q" := (P -∗ |={E1,E2}=> Q) : stdpp_scope. Notation "|={ E }=> Q" := (fupd E E Q) : bi_scope. Notation "P ={ E }=∗ Q" := (P -∗ |={E}=> Q)%I : bi_scope. Notation "P ={ E }=∗ Q" := (P -∗ |={E}=> Q) : stdpp_scope. ``` In general the `▷=>^ n` syntax indicates a number of steps `n` accompanying the mask update? `wsat` is world satisfaction in the context of ofes `dist` means distance > The type `A -n> B` packages a function with a non-expansiveness proof > When an OFE structure on a function type is required but the domain is discrete, one can use the type `A -d> B`. This has the advantage of not bundling any proofs, i.e., this is notation for a plain Coq function type. > When writing `(P)%I`, notations in `P` are resolved in `bi_scope` so it looks like the suffix `I` means internal `■ (P)` means "plainly P", meaning P holds when no resources are available `Λ` is generally an instance of a `language` it seems `tp` is generally a thread pool? it seems `upd` is update and `bupd` is basic update and `fupd` is fancy update It seems the suffix `G` is used to mean "in global" the only purpose of "later" is to prevent the kinds of infinite loops that can make a logic invalid (able to prove False). it's used to define propositions like weakest preconditions that must somehow bake the idea of "the program takes a step" into their meaning ordered families of equivalences (ofe's) are just a "convenient" (if you can call them that) way of encoding "steps" into the system. ofe's make the equivalence of some pieces of data dependent on a step index, so pieces of data might be equivalent at some indexes but not others. but most of the time the step indexes don't matter! most actual *data types* aren't recursive or hold some concept of computational steps in them, so the "equivalences" hold for *all* step indexes! a "cmra" or "camera" is the fully general version of a resource algebra that actually uses the idea of step-indexed equality. just copying a chunk of `docs/resource_algebras.md`: > The type of Iris propositions `iProp Σ` is parameterized by a *global* list `Σ: gFunctors` of resource algebras that the proof may use. (Actually this list contains functors instead of resource algebras, but you only need to worry about that when dealing with higher-order ghost state -- see "Camera functors" below.) In our proofs, we always keep the `Σ` universally quantified to enable composition of proofs. Each proof just assumes that some particular resource algebras are contained in that global list. This is expressed via the `inG Σ R` typeclass, which roughly says that `R ∈ Σ` ("`R` is in the `G`lobal list of RAs `Σ` -- hence the `G`). iris program_logic: it seems contains files related to the instantiation of iris and weakest preconditions for the general "language" concept with exprs and vals etc. I don't think I care except to look for patterns and examples base_logic: is all the pay dirt in here? bi: contains files related to bunched implications logic? si_logic: contains files related to step-indexed logic? algebra: contains files related to resource algebras? So I'll have to define some `magmideG` typeclass and `magmideΣ` list of resource algebras and a `subG_magmideΣ` instance `inG` asserts some resource algebra is in a list `subG` asserts a list of resource algebras is contained in a list > The trailing `S` here is for "singleton" hmm ```coq Class magmideG Σ := { magmide_inG: inG Σ magmideR; magmide_some_other_library: some_other_libraryG Σ }. Local Existing Instances magmide_inG. Local Existing Instances magmide_some_other_library. ... other fields Definition magmideΣ: gFunctors := #[GFunctor magmideR; some_other_libraryΣ]. Instance subG_magmideΣ {Σ}: subG magmideΣ Σ → magmideG Σ. Proof. solve_inG. Qed. Section proof. Context `{!magmideG Σ, !otherthingsG Σ}. EndSection proof. ``` > The backtick (`` ` ``) is used to make anonymous assumptions and to automatically generalize the `Σ`. When adding assumptions with backtick, you should most of the time also add a `!` in front of every assumption. If you do not then Coq will also automatically generalize all indices of type-classes that you are assuming. This can easily lead to making more assumptions than you are aware of, and often it leads to duplicate assumptions which breaks type class resolutions. ================================================ FILE: justfile ================================================ # build: # dune build # wget --no-check-certificate -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add - # add-apt-repository 'deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-13 main' # sudo apt install llvm-13 libclang-common-13-dev lab: #!/usr/bin/env bash # lli-13 lab.ll cargo run lli-13 lab.bc echo $? test: cargo test dune runtest dev: cargo test test_lex -- --nocapture clean: #!/usr/bin/env bash pushd theory make clean rm -f *.glob rm -f *.vo rm -f *.vok rm -f *.vos rm -f .*.aux rm -f .*.d rm -f Makefile* rm -f .lia.cache rm -f *.ml* popd build: #!/usr/bin/env bash pushd theory make popd fullbuild: #!/usr/bin/env bash pushd theory coq_makefile -f _CoqProject *.v -o Makefile make clean make popd ================================================ FILE: lab.ll ================================================ ; https://stackoverflow.com/questions/41716079/llvm-how-do-i-write-ir-to-file-and-run-it/41833643 ; https://stackoverflow.com/questions/7773194/is-it-possible-to-use-llvm-assembly-directly ; https://ecksit.wordpress.com/2011/01/01/hello-world-in-llvm/ ; https://kripken.github.io/llvm.js/demo.html ; @str = internal constant [19 x i8] c"Hello LLVM-C world!" ; declare i32 @puts(i8*) define i32 @main() { doit: ; https://blog.yossarian.net/2020/09/19/LLVMs-getelementptr-by-example ; %0 = call i32 @puts(i8* getelementptr inbounds ([19 x i8], [19 x i8]* @str, i32 0, i32 0)) %0 = add i32 3, 4 %1 = add i32 %0, %0 ret i32 %1 } ================================================ FILE: mg_examples/main.mg ================================================ type Day; | Monday | Tuesday | Wednesday | Thursday | Friday | Saturday | Sunday proc next_weekday(d: Day): Day; match d; Day.Monday => Day.Tuesday Day.Tuesday => Day.Wednesday Day.Wednesday => Day.Thursday Day.Thursday => Day.Friday _ => Day.Monday proc same_day(d: Day): Day; d prop Eq(@T: type): [T, T]; (t: T): [t, t] thm example_next_weekday: Eq[next_weekday(Day.Saturday), Day.Monday]; Eq(Day.Monday) ================================================ FILE: notes/2019-popl-iron-final.md ================================================ pretty simple so far, just saying none of the concurrent separation logics enable tracking *obligations*, merely correctness in the sense of not *doing* something incorrect, rather than *incorrectly forgetting to do something necessary*. this is a problem whenever we're using persistent/duplicable/shareable invariants, which can be copied arbitrarily to be given to different threads. doing this is necessary in fork-style concurrency (vs "structured" concurrency in which the language syntax itself determines where invariants exist). since they're duplicable, they can be thrown away the main way they're going to solve this problem is with what they're calling "trackable resources" the first one is the "trackable points-to connective" `l ->_pi v`, where pi is a rational number describing what fraction of the heap we have control or knowledge of. `pi = 1` means we own the whole thing, and `pi < 1` means someone else has some control then they define Iron++, which defines "trackable invariants" (rather than resources), and Iron++ is linear rather than affine (it doesn't have the weakening rule, so you can't throw away resources). this means these invariants aren't duplicable, but instead have to be "split" getting into it, they define some rules, in which the `e_pi` proposition is like an empty heap, equivalent to the permission to allocate. emp-split: `e_pi1 ∗ e_pi2 <-> e_(pi1 + pi2)` pt-split: `(l -->_pi1 v) * (e_pi2) <-> (l -->_(pi1 + pi2) v)` since `e_pi` propositions allow us to demonstrate we've deallocated memory, we can prove a program doesn't leak memory by giving it a hoare triple of `{ e_pi } program { e_pi }` where pi is equal in pre and post, for any pi I got all I needed from this paper I think ================================================ FILE: notes/assembly-proofs.md ================================================ this paper is mostly just a reimplementation of vale in f*, but with a more efficient proof reflection style verification condition generator the generator is more efficient just because it's a polynomial time algorithm that checks all the easily decidable stuff and defers everything else to a solver. whatevs ================================================ FILE: notes/category-theory-for-programmers.md ================================================ ================================================ FILE: notes/coq-coq-correct.md ================================================ something I have to look at is how metaprogramming works in a bunch of these other languages, metacoq and f* metaprogramming > This paper proposes to switch from a trusted code base to a trusted theory base paradigm! okay I can't read this yet, I have to read metacoq ================================================ FILE: notes/coq-metacoq.md ================================================ okay reading this has actually been helpful I'm still a little hazy on all the typing rules of cic, I guess mostly that they seem to be not as complex as I would assume them to be. honestly the coq reference is almost certainly a better place to understand all of that. however now that I actually understand metacoq and how to use it, I intend to use it to play around with a simpler way of declaring everything, such as a `type A = ` oriented way of doing things ================================================ FILE: notes/indexing-foundational-proof-carrying-code.md ================================================ so far this paper is really simple, it's just saying what proof-carrying-code (PCC) is and why it's valuable. he's also saying it would be great for these systems to not assume a particular type-system, but instead just be rooted in mathematics/logic. VC generator: verification condition generator (akin to a tactic that examines code and infers hoare triples?) so the first 4 sections of this paper are just talking about how we can specify the operational semantics of a physical machine and instruction set, then define program state safety and program safety in terms of the step relation given by the operational semantics. pretty simple! especially interesting is the idea of a safe *program*, which depends on the program being written in a *position independent* manner (which I suppose would mean all instructions merely reference offsets from the program counter). see now in section 5 he's talking about *typed* intermediate representations, which is dumb! metaprogrammable recombination forever! he's also talking about the difference between syntactic and semantic type representation. I guess the core difference is that syntactic type representation is *opaque*, the syntax rules are basically assigned axiomatically. whereas semantic ones are rooted in actual logic, so all the transformation rules can be derived from the underlying meaning of the types. but now we're getting to "recursive contravariance?" and how it makes step-indexing necessary? I'm almost there. Instead of saying a type is a set of values, we say it is a set of pairs ``, where k is an approximation index and v is a value. The judgement `` ∈ τ means, "v approximately has type τ, and any program that runs for fewer than k instructions can't tell the difference."" The indices k allow the construction of a well founded recursion, even when modeling contravariant recursive types. So I guess the k-indexing is just a wrapper of some kind? I think contravariant recursion is just another way of saying it has to be strictly postive in the coq sense. an inductive constructor can't accept as an argument a function that itself takes the inductive type being defined as an argument, because this allows for infinite recursion and therefore unsoundness. ================================================ FILE: notes/indexing-indexed-model.md ================================================ this one is actually getting somewhere. it's basically the same paper as `indexing-foundational-proof-carrying-code` but actually gives some intuitions for what they're talking about with recursive types the important thing it seems is this mu operator ``` µF ≡ { | ∈ F^(k+1)(⊥)} µ(F) = λkλv.∀τ. ncomp(F, k + 1, ⊥, τ) ⇒ τ k v where ncomp(F, k, g, h) means informally that F^k (g) = h ncomp(f, n, x, y) can be defined as, ∀ g. (∀z. g(f, 0, z, z)) ⇒ (∀m, z1, z2 .m > 0 ⇒ g (f, m − 1, z1, z2 ) ⇒ g (f, m, z1, f(z2))) ⇒ g (f, n, x, y). ``` ================================================ FILE: notes/indexing-modal-model.md ================================================ Before getting into the real paper, I'm going to quickly try to gain some clue about what modal logic and kripke semantics are, why they're useful, and how they might relate to step-indexing. # https://plato.stanford.edu/entries/logic-modal/ In general, modal logic is a logic where the truth of a statement is "qualified", using some "mode" like "necessarily" or "possibly" There's a weak logic called `K` (after Saul Kripke) that includes ~, -> as usual, but also the `nec` operator for "necessarily". (written with the annoying box symbol □) `K` is just normal propositional logic with these rules added relating to the `nec` Necessitation Rule: If A is a theorem of K, then so is `nec(A)`. Distribution Axiom: `nec(A -> B) -> (nec(A) -> nec(B))`. Then there's the `may` operator (for "possibly" or "maybe", written with the annoying diamond symbol ◊). It can be defined from `nec` by letting `may(A) = ~nec(~A)`, or "not necessarily not A". This means `nec` and `may` mirror each other in the same way `forall` and `exists` do. Uh oh, there's a whole family of modal logics based on which axioms of "simplification" they include? They're saying which ones make sense depends on what area you're working in. I'm sure this will lead to fun situations in step-indexing. The important part! **Possible Worlds** Every proposition is given a truth value *in every possible world*, and different worlds might have different "truthiness". `v(p, w)` means that for some valuation `v`, propositional variable `p` is true in world `w`. ~ := `v(∼A, w) = True <-> v(A, w) = False` -> := `v(A -> B, w) = True <-> v(A, w) = False or v(B, w) = True` theorem 5 := `v(□A, w) = True <-> forall w': W, v(A, w') = True` ^^^^^^^^^ theorem 5 is important! it seems this is the thing that makes it all make sense. since `nec` and `may` are equivalent to "all" and "some" when thinking about possible worlds, theorem 5 implies that `may` is similar to `exists`, `◊A = ∼□∼A` `may` is true when the proposition is true in *some* worlds, but not necessarily all of them, or that we merely know that A isn't necessarily false *everywhere*. Ah yeah hold on, theorem 5 isn't always reasonable for every kind of modal logic. in temporal logic, where a "world" is really just an "instant" (hint, this is almost certainly what we're dealing with in step-indexing), `nec` really means that something will *continue* to be true into the future, but may not have been in the past. in these cases, we have to define some relation R to define "earlier than" theorem K := `v(□A, w) = True <-> forall w', (R(w, w') -> v(A, w')) = True` so essentially A is necessarily true in w if and only if forall worlds *that are later than w* A is still true so then a kripke frame `` is a pair of a set of worlds W and a relation R. I'm skipping over a bunch of stuff that doesn't seem relevant for getting to step-indexing. Okay bisimulation is a place where this is useful. labeled transition systems (LTSs) represent computation pathways between different machine states. An easily understood quote: ``` LTSs are generalizations of Kripke frames, consisting of a set W of states, and a collection of i-accessibility relations Ri, one for each computer process i. Intuitively, Ri(w, w') holds exactly when w' is a state that results from applying the process i to state w. ``` The last important thing I'll say: the properties (such as transitivity, or being a total preorder) of the *accessibility relation* R (it defines accessibility!) define what axioms are reasonable to use in some context. # moving onto the paper! https://www.irif.fr/~vouillon//smot/Proofs.html okay they're just talking about what they're trying to achieve, especially how we need recursive and quantified types (quantifed means that they may be generic or unknown, as is the case with something like `forall t: T`, where t is quantified) in order to represent tree structures in memory and other such things. types need to allow impredicativity, so types can refer to themselves they talk a little about the difference between syntactic and semantic interpretations. The way I choose to understand this distinction is that syntactic rules can only refer to themselves and can't derive value from other systems, whereas semantic ones are merely embedded in some larger logical system that itself can be used to extend the rules. This seems to point to an important distinction I've been missing: ``` We start from the idea of approximation which pervades all the semantic research in the area. If we type-check v : τ in order to guarantee safety of just the next k computation steps, we need only a k-approximation of the typing-judgment v : τ . ``` The important part is *next* k computation steps. It seems this implies that the type judgment maybe become false *after* k. This isn't how I was thinking about it, which was that the judgment *will become* true in k steps. The less-than relationship to k makes a lot more sense with this interpretation. This also seems important: ``` We express this idea here using a Kripke semantics whose possible-worlds accessibility relation R is well-founded: every path from a world w into the future must terminate. In this work, worlds characterize abstract computation states, giving an upper bound on the number of future computations steps and constraining the contents of memory. ``` I'm a little scared about the implications of this "every path must terminate" thing. I'm hoping that doesn't mean we can't prove things about possibly non-terminating programs (maybe we could define infinite divergence as a terminating "world"?). Nope! They specify in a later section of the paper that we can still use this idea to prove things about *any finite prefix* of any program. I'll write down some of their base rules to help me remember: w ||- v: T means v has type T in world w U |- T means every value u of type U in any world w also has type T in w this seems equivalent to saying U is a subtype of T. Then the modal operator "later"! `lat` quantifies over all worlds (all times) *strictly in the future* they point out that `nec` instead applies to *now* as well as the future. I guess this contradicts my intuition that the "less than k" steps thing is meaningful here More seemingly important stuff ``` Indeed, the combination of a well-founded R with a strict modal operator `lat` provides a clean induction principle to the logic, called the Löb rule, lat(T) |- T ----------- |- T ``` So if it is true that if a prop is True later then it is also true always, then it is always true always. It seems this just means the later is meaningless, *or that there's nothing in the prop that depends on the world*. > In this section we will interpret worlds as characterizing abstract properties of the current state of computation. In particular, in a system with mutable references, each world contains a memory typing In different types of machines, a "world" is a different thing (a lambda calculus with a store is a pair of an expression and that store, a von Neumann machine is the pair of registers (including program pointer) and memory) ``` Clearly, the same value v may or may not have type T depending on the world w, that is, depending on the data structures in memory that v points to. Accordingly, we call a pair (w, v) a configuration (abbreviated "config"): Config = W x V, and define a type T ∈ Type as a set of configurations. Then, (w, v) ∈ T and w |- v : T are two alternative notations expressing the same fact. ``` > We will show how our semantics connects the relation R between worlds and the relation >-> between states. I guess they're saying there's some sort of correspondence between the R relation showing how "worlds" are accessible in time from one-another and the small step relation `>->` that shows how computation states are accessible from one-another. This makes sense since worlds and states are the same thing. So a type is just a set of configurations, or a set of values pointing to something in some world. This is basically saying that a type is all values *who exist in a world* that makes the type assertions true. Yeah, they say "a type is just any set of configurations" basic stuff like the top/bottom types, "logical" intersection/union, and function types are pretty easy to describe then. I'll put the first few in my own words: top := {(w, v) | True} the top type describes all configs! so it is a subtype of all configs bot := {} the bot type is the empty set, so it describes no configs, so it is unrepresentable T /\ U := T intersection U type and set intersection are equivalent, since the intersection of type T and U is only the types that are described by both conditions T \/ U := T union U similar idea, we smush together the types which means any config describe by either of them is valid related, discriminated unions then are the union of types which have no intersection, or where the description of each type necessarily precludes the other U => T := {(w, v) | (w, v) ∈ U => (w, v) ∈ T } this is slightly more involved, but only because I'm not sure if he's talking about implication or functions. I'm going to guess implication, since there's no talk of substitution or anything like that all configs such that if the config is in U, it is also in T Now he gets into how quantification is represented in the type system. These are more interesting. importantly, in the below, A can be either Type, Loc, or Mem. forall x:A.T := global_intersection(T[a/x]) okay, first parsing it: forall x which is an A, then T is defined as the global intersection of all items a in A, for each which we've subsituted the a in the set which our variable x that basically means that forall is the intersection set of all configs (or locs or mems) where... I'm not sure I get it yet. the exists below is similar just with union, so I'll wait until later to understand what's going on. hopefully he gives an applied example. exists x:A.T := global_union(T[a/x]) Quantification over values in a world. pretty simple, !T := {(w, v) | forall v'. (w, v') in T} all values in the current world have type T ?T := {(w, v) | exists v'. (w, v') in T} some value in the current world has type T Then they brag how they can define types in terms of their primitives without using the underlying logic. T <=> U = T => U /\ U => T T iff U, pretty simple (this confirms my suspicion that => was meant to indicate implication, although implication is isomorphic with functions, so there's something there as well) teq(T, U) = !(T <=> U) basically type equality, since for all values in (the current world) the types are equivalent to each other the dependence on the current world is the only part I don't love.... world types (which teq(T, U) is one of) are types that only depend on the world, not the value (I'm guessing persistent types are ones that depend on neither) Okay Vector Values is where I got kind of stuck before, let's write things out as we go to keep it clear. ``` we have locations `l: Loc`, that index a mutable store m; storable values `u: SV` that are the range of m (contents of memory cells); and values `v: V`. We assume Loc subset SV (meaning locations are storable values, but there are more storable values than just locations) On a von Neumann machine, SV = Loc (so locations *do* in fact fully describe storable values) and v is a vector of locations (one could think of a register-bank) indexed by a natural number j. That is, if v is a value, then v(j) is a Loc. (meaning a "value" is a terrible name for what they're talking about! value is a register bank, so v(j. but they're using value in the config sense of a (w, v), or a world and a *value*. this means they're saying the world is the state of memory and the value is the state of the registers, at least on a von Neumann machine) is choosing a particular register to grab a Loc from. Magmide will make this clearer by just making all things byte arrays and lists of byte arrays) ``` This part is where is gets hairier: ``` In order to type locations, we choose an injective function (a function that is one-to-one) `.->` from storable values to values (ints to register banks), for instance `u-> := lambda j. u` This way the same set of types can be used for all kinds of values. ``` The "in order to type locations" is important. I'm hoping this will become more clear. I understand all the parts of that sentence, but not the purpose of the sentence. I think it becomes clearer with the "This way the same set of types can be used for all kinds of values." They're talking about *world/value/config* values in this context, so I guess this injective function is trying to produce some kind of equivalence between von Neumann machines and lambda calculus. This is even less clear ``` In lambda-calculus, `SV = V` is the usual set of values, so we have `Loc strictsubset SV` by syntactic inclusion, and we take `u-> := u` ``` again I understand the parts but not the sentence. perhaps they're saying that in lambda calculus the store can hold anything, and the "value" of a von Neumann machine is the current expression being reduced, so there isn't a need for this injective function? I'm still not sure what the injective function is for. Singletons and slots. I don't want to get stuck on this stuff. based on this definition of the single type `just u` (the single storable value (SV) u) just u := {(w, v) | v = u->} I'm going to choose to believe that the injective arrow function is just saying that the value (register bank) v *can possibly produce u*???? and then u: T = !(just u => T) : here means "has type" in the more traditional sense so for all values in the current world, if the value is u, then it has type T exists l: Loc. just l /\ w(l) Okay this makes more sense: The type `slot(j, T)` characterizes values v such that the jth slot has type T. slot(j, T) := {(w, v) | w ||- v(j): T} all configs such that in the current world the storable value at slot j has type T I think all this stuff was simpler than they made it seem, by this sentence: > To say that register 2 has the value 3 we write slot(2, just 3). Now on to the important stuff, ## Necessity and the modal operator "later" Given two types U and T, we write U |- T when the type U is a subset of the type T, meaning for every world w and value v, w ||- v: U implies w ||- v: T (if a value is a U, then it is also a T, so Us can be replaced with Ts, U is a subtype of T) We write |- T to mean top |- T (so we don't assume any (useful) types are subsets of T, only the top type, which is a subtype of all types) The accessibility relation R has to be transitive and well-founded, such as the less-than (<) relation. So R(w, w') means the world w' comes at a strictly later stage than the world w. From this we can define the later operator: later(T) := {(w, v) | forall w'. R(w, w') => (w', v) in T} so for all worlds strictly later than now (so w < w', or the step-index of w is less than w') so v has type later(T) when v has type T in all worlds strictly later than now (the world w) Some stuff can be proven about later, it's monotone (if U is a subtype of T, then later(U) is a subtype of later(T)) it distributes over intersection: `later(global_intersection(Ti)) = global_intersection(later(Ti))` now the necessity operator (the box), `nec` `nec` means now and later, and is defined simply: nec(T) = T /\ later(T) also monotone, forall T, nec(T) subtype of T if nec(U) subtype T, then nec(U) subtype nec(T) also distributes over intersection necessary types types that, once true in some world w, are true forever. necessary(T) = T subtype later(T) so if T is true then also later(T) or T is a subtype of later(T) or T can be used as later(T) this won't always be true, since the store evolves from one world to the next, possibly destroying some type forall T, necessary(nec(T)) since nec(T) simply contains lat(T) so we can grab it forall T, necessary(lat(T)) The lob rule since R is well-founded, this induction principle is true: ``` later(T) judges T ------------------ judges T ``` Recursive types I'm not going to go over this in detail. Basically, let's say we have a *type* operator F, which maps types to types. such an operator is contractive if A Kripke semantics of stores I think this sentence is what makes the later operator make sense: In this definition we write `later(m(l): T)`. There is some value u in memory at address l, and we guarantee to every future world that `u: T`. We don’t need to guarantee `u: T` in the current world because it takes one step just to dereference `m(l)`, and in that step we move to a future world. This use of the later operator rather than the nec operator is crucial in order to solve the cardinality issue. Indeed, for a configuration ((n, Ψ), v), only the configurations of index strictly less than n are then relevant in the type Ψ(l). So basically types can only refer to themselves because the assertions on memory locations only apply to future states. All types can only refer (at least in regards to memory) to worlds later than the current. this especially applies to reference types, since by necessity accessing the value requires a step of computation, so `ref T` just means that some location has type T *later*. Oh my god, they say in section 11 that a type T describes *the entire register bank*. it's the type of the whole machine! since reference types are attached to the locations stored in the "value" (the register bank), we can assert the state of memory just by the type of the register bank. we can type stack arguments by making assertions about the state of memory around the stack pointer. A minimal machine could get by with just a program counter and memory, since even the return address can be put in stack arguments in memory This paper hasn't heard of separation logic or something ha. They keep saying they have to specify that other registers aren't changed. no thanks. This paper still doesn't explain why *props* have to have step-indexing when they are self-referencing!! I get it all, at least at a high level, but I'm unsatisfied. maybe cpdt will help. ## cpdt because I said so (Universes) > A predicative system enforces the constraint that, when an object is defined using some sort of quantifier, none of the quantifiers may ever be instantiated with the object itself. an object can be passed itself as an argument. but what counts as "itself"? I guess Prop gets around this by not taking *itself*, but *instances* of itself okay all he really says is that since Prop is always eliminated at extraction, and therefore doesn't produce infinite regressions allowing infinite loops to prove anything, it doesn't matter if it's impredicative. so why can't iris use them directly!!!??? ================================================ FILE: notes/iris-from-the-ground-up.md ================================================ An affine logic seems to only mean that the logic includes the weakening rule: `P * Q -> P`, you can *throw away* knowledge/resources Resources algebras seem to be the important thing. A resource algebra is a tuple (M, V : M → Prop, |−| : M → M ? , (·) : M × M → M) rules: RA-associative: forall a, b, c. (a · b) · c = a · (b · c) it doesn't matter what order the composition operator is used in RA-commutative: forall a, b. a · b = b · a it doesn't matter what order the variables are composed in RA-core-composition-identity: forall a. |a|: M ⇒ |a| · a = a if the core of a value is in the type, then composing the core with the same value is the same as the original value RA-core-idempotent: forall a. |a|: M ⇒ ||a|| = |a| if the core of a value is in the type, then the core of the core is the same as the core (this also implies the core of the core composed with the original value is the same as the original value) RA-core-monotonic: forall a, b. |a|: M ∧ a << b ⇒ |b|: M ∧ |a| << |b| not sure yet M? := M union {False} M? basically is just the set of type invariants extended with contradiction M? · False <-> False · M? <-> M? composition with False is commutative and identical with M? a << b := exists c: M. b = a · c a is "less than", or "extended by" by some c exists that "fill the gap" between a and b in terms of composition a --> B := forall c?: M?. V(a · c?) -> exists b: B. V(b · c?) a --> b := a --> {b} a unital resource algebra (uRA) is a resource algebra M with an element ep satisfying these propositions: V(ep) ep is valid forall a: M. ep · a = a ep can be composed with anything without changing the original thing |ep| = ep the duplication of ep is the same as ep a frame-preserving update is an update from some resource a to some resource b, such that if a is compatible (according to the V function) with all c?: M?, then b is also compatible with all c? this essentially means that you can only update resources to a different state if you both already have valid resources and the updated state will be valid. the core function |−| is basically the *duplication* function. it can be partial when some variants of a type aren't duplicable The validity function V: M -> Prop basically defines what variants of the type are valid or acceptable the composition function · defines what happens when you combine resources from different threads, or maybe more correctly it's equivalent to the separating conjunction `*` from separation logic ghost state view shifts are *consuming*, to update `P ==>_ep Q` you have to update the state, or consume or destroy P. normal propositions `A -> B` are *constructive*, and wands a mask on a hoare triple is like a set or map keeping track of which invariants are in force. accessing an invariant removes that invariant's *namespace* from the mask. ================================================ FILE: notes/iris-lecture-notes.md ================================================ https://gitlab.mpi-sws.org/iris/examples/-/tree/master/theories/lecture_notes iris invariants let different threads read/write to the same locations, as long as they don't violate the invariant iris ghost state lets invariants evolve over time, and keep track of information that doesn't exist in the actual program # lambda,ref,conc > A configuration consists of a heap and a thread pool, and a thread pool is a mapping from thread identifiers (natural numbers) to expressions, i.e., a finite set of named threads. Note that reduction of configuations is nondeterministic: we may choose to reduce in any thread in the thread pool. This reflects that we are modelling a kind of preemptive concurrent system. > In the case of Iris the underlying language of “things” is simple type theory with a number of basic constants. These basic constants are given by the signature S. This signature concept is probably going to be important. > The types of Iris are built up from the following grammar, where T stands for additional base types which we will add later, Val and Exp are types of values and expressions in the language, and Prop is the type of Iris propositions. τ ::= T | Z | Val | Exp | Prop | 1 | τ + τ | τ × τ | τ → τ 1 is basically just shorthand for unit? I guess? > The judgments take the form Γ |-S t: τ and express when a term t has type τ in context Γ , given signature S. The variable context Γ assigns types to variables of the logic. It is a list of pairs of a variable x and a type τ such that all the variables are distinct. We write contexts in the usual way, e.g., x1: τ 1 , x2: τ 2 is a context. > The magic wand P −∗ Q is akin to the difference of resources in Q and those in P : it is the set of all those resources which when combined with any resource in P are in Q Then they go on for a long time discussing pretty obvious rules that I already understand (basic logic, separation logic, basic lambda calculus stuff). ================================================ FILE: notes/jung-thesis.md ================================================ << is an *inclusion relation*. a << b means that b is a "bigger resource" than a, or that we obtain b by composing a with some other resource ================================================ FILE: notes/known_types.md ================================================ ```coq Inductive typ: Type := | Unit: typ | Nat: typ | Bool: typ | Arrow: typ -> typ -> typ . Fixpoint typeDenote (t: typ): Set := match t with | Unit => unit | Nat => nat | Bool => bool | Arrow arg ret => typeDenote arg -> typeDenote ret end. (*Definition typctx := list type.*) Inductive exp: list typ -> typ -> Type := | Const: forall env newtyp (value: typeDenote newtyp), exp env newtyp | Var: forall env newtyp, member newtyp env -> exp env newtyp | App: forall env arg ret, exp env (Arrow arg ret) -> exp env arg -> exp env ret | Abs: forall env arg ret, exp (arg :: env) ret -> exp env (Arrow arg ret). Arguments Const [env]. (*Definition a: exp hlist Bool := Const HNil true.*) Fixpoint expDenote env t (e: exp env t): hlist typeDenote env -> typeDenote t := match e with | Const _ value => fun _ => tt | Var _ _ mem => fun s => hget s mem | App _ _ _ e1 e2 => fun s => (expDenote e1 s) (expDenote e2 s) | Abs _ _ _ e' => fun s => fun x => expDenote e' (HCons x s) end. (*Eval simpl in expDenote Const HNil.*) (* okay I feel like I want to have a `compile` function that takes terms and just reduces the knowns, typechecks them, and outputs a string representing the "compiled" program then a `run` function that reduces the knowns and typechecks the program, but then reduces all the terms and outputs the "stdout" of the program this is presupposing that you'll have some kind of effectful commands that append some string to the "stdout". that seems like the more natural way I would prefer to structure a language that I'll eventually be using to learn while making a real imperative language *) (*Require Import Coq.Strings.String. Require Import theorems.Maps. Inductive typ: Type := (*| Generic*) | Bool | Nat | Arrow (input output: typ) | UnionNil | UnionCons (arm_name: string) (arm_type: typ) (rest: typ) | TupleNil | TupleCons (left right: typ) (*| KnownType (type_value: trm)*) (*| KnownValue (value: trm)*) . Inductive Arm: Type := | arm (arm_name: string). Inductive trm: Type := | tru | fls | debug_bool (*| nat_const (n: nat)*) (*| nat_plus (left right: trm)*) (*| debug_nat*) | binding (decl_name: string) (after: trm) | usage (var_name: string) | test (conditional iftru iffls: trm) | fn (args_name: string) (output_type: typ) (body: trm) | call (target_fn args: trm) | union_nil | union_cons (arm_name: string) (arm_value: trm) (rest_type: typ) | union_match (tr: trm) (arms: list (string * trm)) | tuple_nil | tuple_cons (left right: trm) | tuple_access (tup: trm) (index: nat) . Fixpoint tuple_lookup (n: nat) (tr: trm): option trm := match tr with | tuple_cons t tr' => match n with | 0 => Some t | S n' => tuple_lookup n' tr' end | _ => None end . Fixpoint union_lookup (tr: trm) (arms: list (string, (string * trm))): option trm := match tr with | union_cons tr_arm_name tr_arm_value _ => match arms with | (arm_name, (arm_var, arm_body)) :: arms' => if eqb_string tr_arm_name arm_name then Some (substitute arm_var tr_arm_value arm_body) else union_lookup tr arms' | [] => None end | _ => None end . *) (*Require Import Coq.Strings.String. Require Import theorems.Maps. (*Notation memarr := (@list string).*) Inductive typ: Type := | Base: string -> typ | Arrow: typ -> typ -> typ | TupleNil: typ | TupleCons: typ -> typ -> typ. Inductive trm: Type := | var: string -> trm | call: trm -> trm -> trm | fn: string -> typ -> trm -> trm (* tuples *) | tuple_proj: trm -> nat -> trm | tuple_nil: trm | tuple_cons: trm -> trm -> trm. Inductive tuple_typ: typ -> Prop := | TTnil: tuple_typ TupleNil | TTcons: forall T1 T2, tuple_typ (TupleCons T1 T2). Inductive well_formed_typ: typ -> Prop := | wfBase: forall i, well_formed_typ (Base i) | wfArrow: forall T1 T2, well_formed_typ T1 -> well_formed_typ T2 -> well_formed_typ (Arrow T1 T2) | wfTupleNil: well_formed_typ TupleNil | wfTupleCons: forall T1 T2, well_formed_typ T1 -> well_formed_typ T2 -> tuple_typ T2 -> well_formed_typ (TupleCons T1 T2). Hint Constructors tuple_typ well_formed_typ. Inductive tuple_trm: trm -> Prop := | tuple_tuple_nil: tuple_trm tuple_nil | tuple_trm_tuple_cons: forall t1 t2, tuple_trm (tuple_cons t1 t2). Hint Constructors tuple_trm. (*Notation "x :: l" := (cons x l) (at level 60, right associativity).*) Notation "{ }" := tuple_nil. Notation "{ x ; .. ; y }" := (tuple_cons x .. (tuple_cons y tuple_nil) ..). Fixpoint subst (prev: string) (next: trm) (target: trm) : trm := match target with | var y => if eqb_string prev y then next else target | fn y T t1 => fn y T (if eqb_string prev y then t1 else (subst prev next t1)) | call t1 t2 => call (subst prev next t1) (subst prev next t2) | tuple_proj t1 i => tuple_proj (subst prev next t1) i | tuple_nil => tuple_nil | tuple_cons t1 tup => tuple_cons (subst prev next t1) (subst prev next tup) end. Notation "'[' prev ':=' next ']' target" := (subst prev next target) (at level 20). Inductive value: trm -> Prop := | v_fn: forall x T11 t12, value (fn x T11 t12) | v_tuple_nil: value tuple_nil | v_tuple_cons: forall v1 vtup, value v1 -> value vtup -> value (tuple_cons v1 vtup). Hint Constructors value. Fixpoint tuple_lookup (n: nat) (tr: trm): option trm := match tr with | tuple_cons t tr' => match n with | 0 => Some t | S n' => tuple_lookup n' tr' end | _ => None end. Open Scope string_scope. Notation a := (var "a"). Notation b := (var "b"). Notation c := (var "c"). Notation d := (var "d"). Notation e := (var "e"). Notation f := (var "f"). Notation g := (var "g"). Notation l := (var "l"). Notation A := (Base "A"). Notation B := (Base "B"). Notation k := (var "k"). Notation i1 := (var "i1"). Notation i2 := (var "i2"). Example test_tuple_lookup_nil_0: (tuple_lookup 0 {}) = None. Proof. reflexivity. Qed. Example test_tuple_lookup_nil_1: (tuple_lookup 1 {}) = None. Proof. reflexivity. Qed. Example test_tuple_lookup_cons_valid_0_a: (tuple_lookup 0 { a }) = Some a. Proof. reflexivity. Qed. Example test_tuple_lookup_cons_valid_0_a_b: (tuple_lookup 0 { a; b }) = Some a. Proof. reflexivity. Qed. Example test_tuple_lookup_cons_invalid: (tuple_lookup 3 { a; b; c }) = None. Proof. reflexivity. Qed. *) ``` ``` Add LoadPath "/home/blaine/lab/cpdtlib" as Cpdt. Set Implicit Arguments. Set Asymmetric Patterns. Require Import List Cpdt.CpdtTactics Cpdt.DepList theorems.Maps Coq.Strings.String. (*blaine, you need to write examples of what you'd like to accomplish in the near term*) (*some concrete examples of "metaprogramming" in some abstract language is all you need*) (*you don't have to prove almost anything about them, at least not at first, just get them working as expected and then prove things about them*) (*the term type you create *is* the meta datatype! syntactic macros are just functions that operate on the same objects as the compiler*) Inductive ty: Type := | Ty_Bool: ty | Ty_Arrow (domain: ty) (range: ty): ty. Inductive tm: Type := | tm_var (name: string): tm | tm_call (fn: tm) (arg: tm): tm | tm_fn (argname: string) (argty: ty) (body: tm): tm | tm_true: tm | tm_false: tm | tm_if (test: tm) (tbody: tm) (fbody: tm): tm. Declare Custom Entry stlc. Notation "<{ e }>" := e (e custom stlc at level 99). Notation "( x )" := x (in custom stlc, x at level 99). Notation "x" := x (in custom stlc at level 0, x constr at level 0). Notation "U -> T" := (Ty_Arrow U T) (in custom stlc at level 50, right associativity). Notation "x y" := (tm_call x y) (in custom stlc at level 1, left associativity). Notation "\ x : t , y" := (tm_fn x t y) ( in custom stlc at level 90, x at level 99, t custom stlc at level 99, y custom stlc at level 99, left associativity ). Coercion tm_var : string >-> tm. Notation "'Bool'" := Ty_Bool (in custom stlc at level 0). Notation "'if' x 'then' y 'else' z" := (tm_if x y z) ( in custom stlc at level 89, x custom stlc at level 99, y custom stlc at level 99, z custom stlc at level 99, left associativity ). Notation "'true'" := true (at level 1). Notation "'true'" := tm_true (in custom stlc at level 0). Notation "'false'" := false (at level 1). Notation "'false'" := tm_false (in custom stlc at level 0). Definition x: string := "x". Definition y: string := "y". Definition z: string := "z". Hint Unfold x: core. Hint Unfold y: core. Hint Unfold z: core. Notation idB := <{\x:Bool, x}>. Notation idBB := <{\x:Bool -> Bool, x}>. Inductive value: tm -> Prop := | v_fn: forall arg T body, value <{\arg:T, body}> | v_true: value <{true}> | v_false: value <{false}>. Hint Constructors value: core. Reserved Notation "'[' old ':=' new ']' target" (in custom stlc at level 20, old constr). Fixpoint subst (old: string) (new: tm) (target: tm): tm := match target with | <{true}> => <{true}> | <{false}> => <{false}> | tm_var varname => if string_dec old varname then new else target | <{\var:T, body}> => if string_dec old var then target else <{\var:T, [old:=new] body}> | <{fn arg}> => <{([old:=new] fn) ([old:=new] arg)}> | <{if test then tbody else fbody}> => <{if ([old:=new] test) then ([old:=new] tbody) else ([old:=new] fbody)}> end where "'[' old ':=' new ']' target" := (subst old new target) (in custom stlc). Hint Unfold subst: core. Check <{[x:=true] x}>. Compute <{[x:=true] x}>. Inductive substi (old: string) (new: tm): tm -> tm -> Prop := | s_true: substi old new <{true}> <{true}> | s_false: substi old new <{false}> <{false}> | s_var_matches: substi old new (tm_var old) new | s_var_not_matches: forall varname, let varitem := (tm_var varname) in old <> varname -> substi old new varitem varitem | s_fn_matches: forall T body, let fn := <{\old:T, body}> in substi old new fn fn | s_fn_not_matches: forall var T body newbody, old <> var -> substi old new body newbody -> substi old new <{\var:T, body}> <{\var:T, newbody}> | s_fn_call: forall fn newfn arg newarg, substi old new fn newfn -> substi old new arg newarg -> substi old new <{fn arg}> <{newfn newarg}> | s_if: forall test tbody fbody newtest newtbody newfbody, substi old new test newtest -> substi old new tbody newtbody -> substi old new fbody newfbody -> substi old new <{if test then tbody else fbody}> <{if newtest then newtbody else newfbody}> . Hint Constructors substi: core. (*Theorem substi_correct: forall old new before after, <{ [old:=new]before }> = after <-> substi old new before after. Proof. intros. split; generalize after. induction before; if_crush. induction 1; if_crush. Qed.*) Reserved Notation "t '-->' t'" (at level 40). Inductive step: tm -> tm -> Prop := | ST_AppAbs: forall x T2 t1 v2, value v2 -> <{(\x:T2, t1) v2}> --> <{ [x:=v2]t1 }> | ST_App1: forall t1 t1' t2, t1 --> t1' -> <{t1 t2}> --> <{t1' t2}> | ST_App2: forall v1 t2 t2', value v1 -> t2 --> t2' -> <{ v1 t2}> --> <{v1 t2'}> | ST_IfTrue: forall t1 t2, <{if true then t1 else t2}> --> t1 | ST_IfFalse: forall t1 t2, <{if false then t1 else t2}> --> t2 | ST_If: forall t1 t1' t2 t3, t1 --> t1' -> <{ if t1 then t2 else t3}> --> <{if t1' then t2 else t3}> where "t '-->' t'" := (step t t'). Definition relation (X: Type) := X -> X -> Prop. Inductive multi {X: Type} (R: relation X): relation X := | multi_refl: forall (x: X), multi R x x | multi_step: forall (x y z: X), R x y -> multi R y z -> multi R x z. Hint Constructors step: core. Notation multistep := (multi step). Notation "t1 '-->*' t2" := (multistep t1 t2) (at level 40). Tactic Notation "print_goal" := match goal with |- ?x => idtac x end. Tactic Notation "normalize" := repeat ( print_goal; eapply multi_step; [ (eauto 10; fail) | (instantiate; simpl)] ); apply multi_refl. Lemma step_example1': <{idBB idB}> -->* idB. Proof. normalize. Qed. Definition context := partial_map ty. Inductive typed: context -> tm -> ty -> Prop := | T_True: forall ctx, typed ctx <{true}> <{Bool}> | T_False: forall ctx, typed ctx <{false}> <{Bool}> | T_Var: forall ctx varname T, ctx varname = Some T -> typed ctx varname T | T_Abs: forall ctx var Tvar body Tbody, typed (update ctx var Tvar) body Tbody -> typed ctx <{\var:Tvar, body}> <{Tvar -> Tbody}> | T_App: forall ctx fn arg domain range, typed ctx fn <{domain -> range}> -> typed ctx arg domain -> typed ctx <{fn arg}> range | T_If: forall test tbody fbody T ctx, typed ctx test <{Bool}> -> typed ctx tbody T -> typed ctx fbody T -> typed ctx <{if test then tbody else fbody}> T . Hint Constructors typed: core. Example typing_example_1: typed empty <{\x:Bool, x}> <{Bool -> Bool}>. Proof. auto. Qed. Fixpoint types_equal (T1 T2: ty): {T1 = T2} + {T1 <> T2}. decide equality. Defined. Notation "x <- e1 -- e2" := (match e1 with | Some x => e2 | None => None end) (right associativity, at level 60). Fixpoint type_check (ctx: context) (t: tm): option ty := match t with | <{true}> => Some <{ Bool }> | <{false}> => Some <{ Bool }> | tm_var varname => ctx varname | <{\var:Tvar, body}> => Tbody <- type_check (update ctx var Tvar) body -- Some <{Tvar -> Tbody}> | <{fn arg}> => Tfn <- type_check ctx fn -- Targ <- type_check ctx arg -- match Tfn with | <{Tdomain -> Trange}> => if types_equal Tdomain Targ then Some Trange else None | _ => None end | <{if test then tbody else fbody}> => Ttest <- type_check ctx test -- Ttbody <- type_check ctx tbody -- Tfbody <- type_check ctx fbody -- match Ttest with | <{ Bool }> => if types_equal Ttbody Tfbody then Some Ttbody else None | _ => None end end. Hint Unfold type_check. Ltac solve_by_inverts n := match goal with | H : ?T |- _ => match type of T with Prop => solve [ inversion H; match n with S (S (?n')) => subst; solve_by_inverts (S n') end ] end end. Ltac solve_by_invert := solve_by_inverts 1. Ltac if_crush := crush; repeat match goal with | [ |- context[if ?X then _ else _] ] => destruct X end; crush. Theorem type_checking_complete: forall ctx t T, typed ctx t T -> type_check ctx t = Some T. Proof. intros. induction H; if_crush. Qed. Hint Resolve type_checking_complete: core. Theorem type_checking_sound: forall ctx t T, type_check ctx t = Some T -> typed ctx t T. Proof. intros ctx t. generalize dependent ctx. induction t; intros ctx T; inversion 1; crush. - rename t1 into fn, t2 into arg. remember (type_check ctx fn) as Fnchk. destruct Fnchk as [TFn|]; try solve_by_invert; destruct TFn as [|Tdomain Trange]; try solve_by_invert; remember (type_check ctx arg) as Argchk; destruct Argchk as [TArg|]; try solve_by_invert. destruct (types_equal Tdomain TArg) eqn: Hd; crush. apply T_App. - - intros. generalize dependent T. generalize dependent ctx. induction t; intros ctx T; inversion 1. - crush. - crush. induction t; intros crush. Qed. Hint Resolve type_checking_sound. Theorem type_checking_correct: forall ctx t T, type_check ctx t = Some T <-> typed ctx t T. Proof. crush. Qed. ``` You should probably write out this whole (almost) blog post informally before you really dig into the formal stuff. This is just such a huge undertaking, first understanding what you even precisely want to accomplish is a good idea. Think of it like writing the documentation before you write the code! You do that all the time since it helps clarify what's special and useful about the code, and what features it needs to have. So I guess this whole project has a few beliefs: - We can and should bring formally verified programming with dependent types to the mainstream. - We can and should make a bedrock language with a dependent type system that is defined in the smallest and most primitive constructs of machine computation, because all the code we actually write is intended for such systems. - We should design some set of "known" combinators to allow someone to write a compiler in bedrock that translates some set of terms of a language into bedrock, so that arbitrarily convenient and powerful languages can be implemented from these bedrock building blocks. By doing so we can have all languages be truly safe and also truly interoperable. Formalizing and implementing the algorithms for a type system in bedrock allows you to prove that all of your derived forms are valid in bedrock! Dependent types and the ability to prove arbitrary statements is *most* powerful at this lowest level of abstraction, since it allows us to build literally any language construct we can imagine, since the derived types people build can encapsulate on bytes and propositions, which are the most flexible constructs for machine computation. So far you've considered "generics" as something that exists in the "computable" set of terms, but that's not really correct a generic function is actually two function calls, the first of a "known" function that takes some function containing type variables and a type substitution mapping those type variables to concrete types (or to other type variables! which can allow you to partially apply generics, there should probably be two functions at least for now, one that expects all type variables to be resolved and returns a concrete function, and one that allows for partial application and returns a known function. both of these functions can resolve to either their intended type or a compilation error term) so you should probably have these inductives: concrete types (which include the types that encode type variables in a "computable" way. there's some thinking to do here, but I think this means that you can pass any concrete term to a known function as long as it meets some "known" criteria which for functions is assumed and but for other values simply means that they have to be constants) and concrete terms (basically just the base lambda calculus stuff), known types and known terms (which are the "inductive" step, since they can take both concrete things as well as other knowns, creating the unbounded but finite dag of compilation) all of this means that bedrock itself won't actually have "primitive syntactic" generics like other languages do, but syntactic generics will of course be possible by means of translation in any theoretical derived language. It is actually possible to have "dynamic" functions! By the time bedrock is done, *everything* will just be bytes, and *instructions* are just bytes! All you need in order to allow dynamic functions is to "include" the typechecker or compiler in your final "computable" binary! All we've done here is "move up" the known steps, since what is typically known and performed at compile time is still "dynamic" in the sense that actual machine computation is being performed, just like it will be at runtime! compile time is just a special case of runtime! Known types are simply all about how we're able to produce code. One of the first things we need is a "bedrock type". This is the actual If we implement this as a simply typed lambda calculus, then the "ordering" of everything is taken care of? It's also less interesting, but that's okay, at least for now. Really this first version to validate everything is basically just a simply typed lambda calculus but where there's some kind of "known" system that allows the functions to operate on types. You need to sit and draw out how different types relate to each other. Then you basically do all the work he does in SLTC. Define preservation and progress and all that. First you have "computable terms". These are basically just terms that have been reduced enough that they can actually be "run", whatever that means in the context you're talking about. In a "compiled" language that means something that's been reduced enough to be output as llvm and run. In these more theoretical contexts it's just reduced down to a subset of terms that have been deemed computable. The interesting part of the "computable term" definition is what terms it reveals as *not* being computable. These are basically all the "known" structures. Those known structures need to be reduced all the way to computable ones before they're ready to actually compute. But the *bodies* of the known structures *themselves* also need to be reduced as well! This produces a directed acyclic graph of "known" terms that need to be reduced in order all the way down to computable terms. Does this mean that the only "types" we actually *need* are computable ones? It certainly seems that way, since we can simply say that the only thing we need to "typecheck" is a computable term that we're about to compute. Having more "advanced" higher order types is merely useful for a more ergonomic version of the language that we can do a "higher order" typecheck on before even bothering to reduce any terms. Higher order typechecks probably also play right into a full proof-capable language, one where you can prove that your higher order functions will always reduce to things that will typecheck. For now it seems all this version needs is an initial "dag" check, if it even allows recursion that is. Does this mean that the typing relation is something like this? ```v Inductive ty : Type := | Bool: ty | Arrow: ty -> ty -> ty | Known: ty -> ty. I think this really is it! At least for formally defining it, all this "Known" type needs to do to work is to "reduce" in a different way. It yields an abstract description of the type or value or whatever rather than another term. Or rather the term it reduces to *is* the type. Is this true? I need to keep thinking. Inductive tm : Type := | var : string -> tm | call : tm -> tm -> tm | fn : string -> ty -> tm -> tm | tru : tm | fls : tm | test : tm -> tm -> tm -> tm. ``` maybe we define types not inherently, but as things that reduce from known terms? or maybe our typechecking function and relation aren't total, we can't (and don't want to bother to) typecheck terms that haven't reduced all the way to computable terms. the typechecking function should return `option` on all terms that aren't computable So let's say we had a language that had these types bool: typ; obvious, computable nat: typ; obvious, computable arrow: typ -> typ -> typ; obvious, computable typvalue: booltyp | nattyp | arrowtyp; hmmmm, this is computable since we need to compute based on it to progress and output something need union (variant) and tuple and unit known: (tm -> tm) -> typ?; not computable directly, but we can reduce it to being computable and these terms: tru: tm; obvious, computable fls: tm; obvious, computable n: nat -> tm; obvious, computable known While reading types and programming languages, something's occuring to me. The base "bedrock" language has to be fully strict and exact in the way it defines the calculable language, which can basically only consist of arrays of bytes and propositions on those arrays of bytes. However once we've done that, we can build all kinds of convenient language forms and theorems about them by simply defining them as meta-functions in that bedrock language. For example, in the strict "bedrock" sense, subtyping is basically never valid, since subtyping ignores the very concrete byte-level representation of the structures. But if we have a "meta-language" (which is just a "compiler" that itself is a program in bedrock that takes the terms of the meta-language and computes them to bedrock) then we can allow subtyping simply by saying that whenever we encounter an action that gives a subtype, we can compile that action into the actually valid bytes level action that will satisfy the propositions of bedrock. In this way we have a *provably correct* desugaring process. ================================================ FILE: notes/pony-reference-capabilities.md ================================================ http://jtfmumm.com/blog/2016/03/06/safely-sharing-data-pony-reference-capabilities/ `iso`: writeable/readable, only one reference exists (this one). can be used to read or write locally. can be converted to anything, including giving it up to pass to another actor `val`: readable, only immutable aliases exist, so can be shared for reading with anyone. `tag`: neither, the address of an actor, can be shared anywhere, but can't be read or written. `ref`: writeable/readable but only locally, an unknown number of mutable local aliases exist, so this is just like a typical alias. since we don't know how many aliases exist, we can only possibly share this thing if we somehow destroy those other aliases. `trn`: a local reference we can write/read, but can only create readable references from. this allows us to eventually convert this type to a `val`. `box`: readable locally, we don't know how many other people are looking at this thing the subtyping (or "can be substituted for") relation ``` --> ref -- / \ iso --> trn -- --> box --> tag \ / --> val -- ``` 1) A mutable reference capability denies neither read nor write permissions. This category includes `iso`, `ref`, and `trn`. 2) An immutable reference capability denies write permissions but not read permissions. This category includes `val` and `box`. 3) An opaque reference capability denies both read and write permissions. The only example is `tag`. https://tutorial.ponylang.io/reference-capabilities/reference-capabilities.html#isolated-data-may-be-complex ``` Isolated data may be complex An isolated piece of data may be a single byte. But it can also be a large data structure with multiple references between the various objects in that structure. What matters for the data to be isolated is that there is only a single reference to that structure as a whole. We talk about the isolation boundary of a data structure. For the structure to be isolated: There must only be a single reference outside the boundary that points to an object inside. There can be any number of references inside the boundary, but none of them must point to an object outside. Isolated, written iso. This is for references to isolated data structures. If you have an iso variable then you know that there are no other variables that can access that data. So you can change it however you like and give it to another actor. Value, written val. This is for references to immutable data structures. If you have a val variable then you know that no-one can change the data. So you can read it and share it with other actors. Reference, written ref. This is for references to mutable data structures that are not isolated, in other words, “normal” data. If you have a ref variable then you can read and write the data however you like and you can have multiple variables that can access the same data. But you can’t share it with other actors. Box. This is for references to data that is read-only to you. That data might be immutable and shared with other actors or there may be other variables using it in your actor that can change the data. Either way, the box variable can be used to safely read the data. This may sound a little pointless, but it allows you to write code that can work for both val and ref variables, as long as it doesn’t write to the object. Transition, written trn. This is used for data structures that you want to write to, while also holding read-only (box) variables for them. You can also convert the trn variable to a val variable later if you wish, which stops anyone from changing the data and allows it be shared with other actors. Tag. This is for references used only for identification. You cannot read or write data using a tag variable. But you can store and compare tags to check object identity and share tag variables with other actors. Note that if you have a variable referring to an actor then you can send messages to that actor regardless of what reference capability that variable has. ``` so reference capabilities have these qualities: readable/writeable *to you* readable/writeable *to others* writeable *locally* shareable *locally* shareable *globally* https://tutorial.ponylang.io/reference-capabilities/guarantees.html `iso`: others/local read/write unique `trn`: others/local write unique, others read unique `ref`: others read/write unique `val`: others/local immutable `box`: others immutable `tag`: opaque | | Deny global read/write | Deny global write | None denied | |-----------------------|--------------------------|-------------------|------------------| | Deny local read/write | `iso` (sendable) | | | | Deny local write | `trn` | `val` (sendable) | | | None denied | `ref` | `box` | `tag` (sendable) | | | (Mutable) | (Immutable) | (Opaque) | Sendable capabilities. If we want to send references to a different actor, we must make sure that the global and local aliases make the same guarantees. It’d be unsafe to send a trn to another actor, since we could possibly hold box references locally. Only iso, val, and tag have the same global and local restrictions – all of which are in the main diagonal of the matrix. ================================================ FILE: notes/tarjan/README.md ================================================ Tarjan and Kosaraju ------------------- # Main files ## Proofs of Tarjan strongly connected component algorithm (independent from each other) * `tarjan_rank.v` *(751 sloc)*: proof with rank * `tarjan_rank_bigmin.v` *(806 sloc)*: same proof but with a `\min_` instead of multiple inequalities on the output rank * `tarjan_num.v` *(1029 sloc)*: same proof as `tarjan_rank_bigmin.v` but with serial numbers instead of ranks * `tarjan_nocolor.v` *(548 sloc)*: new proof, with ranks and without colors, less fields in environement and less invariants, preconditions and postconditions. * `tarjan_nocolor_optim.v` *(560 sloc)*: same proof as `tarjan_nocolor.v`, but with the serial number field of the environement restored, and passing around stack extensions as sets. ## Proof of Kosaraju strongly connected component algorithm * `Kosaraju.v` *(679 sloc)*: proof of Kosaraju connected component algorithm ## Extra library files * `bigmin.v` *(137 sloc)*: extra library to deal with \min(i in A) F i * `extra.v` *(265 sloc)*: naive definitions of strongly connected components and various basic extentions of mathcomp libraries on paths and fintypes. # Authors: Cyril Cohen, Jean-Jacques Lévy and Laurent Théry ================================================ FILE: notes/tarjan/_CoqProject ================================================ -R . mathcomp.tarjan -arg -w -arg -notation-overridden tarjan_nocolors.v extra_nocolors.v ================================================ FILE: notes/tarjan/extra_nocolors.v ================================================ From mathcomp Require Import all_ssreflect. Set Implicit Arguments. Unset Strict Implicit. Unset Printing Implicit Defensive. Lemma ord_minn_le n (i j : 'I_n) : minn i j < n. Proof. by rewrite gtn_min ltn_ord. Qed. Definition ord_minn {n} (i j : 'I_n) := Ordinal (ord_minn_le i j). Section ord_min. Variable (n : nat). Notation T := (ord_max : 'I_n.+1). Notation min := (@ord_minn n.+1). Lemma minTo : left_id T min. Proof. by move=> i; apply/val_inj; rewrite /= (minn_idPr _) ?leq_ord. Qed. Lemma minoT : right_id T min. Proof. by move=> i; apply/val_inj; rewrite /= (minn_idPl _) ?leq_ord. Qed. Lemma minoA : associative min. Proof. by move=> ???; apply/val_inj/minnA. Qed. Lemma minoC : commutative min. Proof. by move=> ??; apply/val_inj/minnC. Qed. Canonical ord_minn_monoid := Monoid.Law minoA minTo minoT. Canonical ord_minn_comoid := Monoid.ComLaw minoC. End ord_min. Notation "\min_ ( i | P ) F" := (\big[ord_minn/ord_max]_(i | P%B) F%N) (at level 41, F at level 41, i at level 50, format "'[' \min_ ( i | P ) '/ ' F ']'") : nat_scope. Notation "\min_ i F" := (\big[ord_minn/ord_max]_i F%N) (at level 41, F at level 41, i at level 0, format "'[' \min_ i '/ ' F ']'") : nat_scope. Notation "\min_ ( i 'in' A | P ) F" := (\big[ord_minn/ord_max]_(i in A | P%B) F%N) (at level 41, F at level 41, i, A at level 50, format "'[' \min_ ( i 'in' A | P ) '/ ' F ']'") : nat_scope. Notation "\min_ ( i 'in' A ) F" := (\big[ord_minn/ord_max]_(i in A) F%N) (at level 41, F at level 41, i, A at level 50, format "'[' \min_ ( i 'in' A ) '/ ' F ']'") : nat_scope. Section extra_bigmin. Variables (n : nat) (I : finType). Implicit Type (F : I -> 'I_n.+1). Lemma geq_bigmin_cond (P : pred I) F i0 : P i0 -> F i0 >= \min_(i | P i) F i. Proof. by move=> Pi0; rewrite (bigD1 i0) //= geq_minl. Qed. Arguments geq_bigmin_cond [P F]. Lemma geq_bigmin F (i0 : I) : F i0 >= \min_i F i. Proof. exact: geq_bigmin_cond. Qed. Lemma bigmin_geqP (P : pred I) (m : 'I_n.+1) F : reflect (forall i, P i -> F i >= m) (\min_(i | P i) F i >= m). Proof. apply: (iffP idP) => leFm => [i Pi|]. by apply: leq_trans leFm _; apply: geq_bigmin_cond. by elim/big_ind: _; rewrite ?leq_ord // => m1 m2; rewrite leq_min => ->. Qed. Lemma bigmin_inf i0 (P : pred I) (m : 'I_n.+1) F : P i0 -> m >= F i0 -> m >= \min_(i | P i) F i. Proof. by move=> Pi0 le_m_Fi0; apply: leq_trans (geq_bigmin_cond i0 Pi0) _. Qed. Lemma bigmin_eq_arg i0 (P : pred I) F : P i0 -> \min_(i | P i) F i = F [arg min_(i < i0 | P i) F i]. Proof. move=> Pi0; case: arg_minP => //= i Pi minFi. by apply/val_inj/eqP; rewrite eqn_leq geq_bigmin_cond //=; apply/bigmin_geqP. Qed. Lemma eq_bigmin_cond (A : pred I) F : #|A| > 0 -> {i0 | i0 \in A & \min_(i in A) F i = F i0}. Proof. case: (pickP A) => [i0 Ai0 _ | ]; last by move/eq_card0->. by exists [arg min_(i < i0 in A) F i]; [case: arg_minP | apply: bigmin_eq_arg]. Qed. Lemma eq_bigmin F : #|I| > 0 -> {i0 : I | \min_i F i = F i0}. Proof. by case/(eq_bigmin_cond F) => x _ ->; exists x. Qed. Lemma bigmin_setU (A B : {set I}) F : \min_(i in (A :|: B)) F i = ord_minn (\min_(i in A) F i) (\min_(i in B) F i). Proof. have d : [disjoint A :\: B & B] by rewrite -setI_eq0 setIDAC setDIl setDv setI0. rewrite (eq_bigl [predU (A :\: B) & B]) ?bigU//=; last first. by move=> y; rewrite !inE; case: (_ \in _) (_ \in _) => [] []. symmetry; rewrite (big_setID B) /= [X in ord_minn X _]minoC -minoA. congr (ord_minn _ _); apply: val_inj; rewrite /= (minn_idPr _)//. by apply/bigmin_geqP=> i; rewrite inE => /andP[iA iB]; rewrite (bigmin_inf iB). Qed. End extra_bigmin. Arguments geq_bigmin_cond [n I P F]. Arguments geq_bigmin [n I F]. Arguments bigmin_geqP [n I P m F]. Arguments bigmin_inf [n I] i0 [P m F]. Arguments bigmin_eq_arg [n I] i0 [P F]. Section extra_fintype. Variable V : finType. Definition relto (a : pred V) (g : rel V) := [rel x y | (y \in a) && g x y]. Definition relfrom (a : pred V) (g : rel V) := [rel x y | (x \in a) && g x y]. Lemma connect_rev (g : rel V) : connect g =2 (fun x => connect (fun x => g^~ x) ^~ x). Proof. move=> x y; apply/connectP/connectP=> [] [p gp ->]; [exists (rev (belast x p))|exists (rev (belast y p))]; rewrite ?rev_path //; by case: (lastP p) => //= ??; rewrite belast_rcons rev_cons last_rcons. Qed. Lemma path_to a g z p : path (relto a g) z p = (path g z p) && (all a p). Proof. apply/(pathP z)/idP => [fgi|/andP[/pathP gi] /allP ga]; last first. by move=> i i_lt /=; rewrite gi ?andbT ?[_ \in _]ga // mem_nth. rewrite (appP (pathP z) idP) //=; last by move=> i /fgi /= /andP[_ ->]. by apply/(all_nthP z) => i /fgi /andP []. Qed. Lemma path_from a g z p : path (relfrom a g) z p = (path g z p) && (all a (belast z p)). Proof. by rewrite -rev_path path_to all_rev rev_path. Qed. Lemma connect_to (a : pred V) (g : rel V) x z : connect g x z -> exists y, [/\ (y \in a) ==> (x == y) && (x \in a), connect g x y & connect (relto a g) y z]. Proof. move=> /connectP [p gxp ->]. pose P := [pred i | let y := nth x (x :: p) i in [&& connect g x y & connect (relto a g) y (last x p)]]. have [] := @ex_minnP P. by exists (size p); rewrite /= nth_last (path_connect gxp) //= mem_last. move=> i /= /andP[g1 g2] i_min; exists (nth x (x :: p) i); split=> //. case: i => [|i] //= in g1 g2 i_min *; first by rewrite eqxx /= implybb. have i_lt : i < size p. by rewrite i_min // !nth_last /= (path_connect gxp) //= mem_last. have [<-/=|neq_xpi /=] := altP eqP; first by rewrite implybb. have := i_min i; rewrite ltnn => /contraNF /(_ isT) <-; apply/implyP=> axpi. rewrite (connect_trans _ g2) ?andbT //; last first. by rewrite connect1 //= [_ \in _]axpi /= (pathP x _). by rewrite (path_connect gxp) //= mem_nth //= ltnW. Qed. Lemma connect_from (a : pred V) (g : rel V) x z : connect g x z -> exists y, [/\ (y \in a) ==> (z == y) && (z \in a), connect (relfrom a g) x y & connect g y z]. Proof. rewrite connect_rev => cgxz; have [y [ayaz]]//= := connect_to a cgxz. by exists y; split; rewrite // connect_rev. Qed. Lemma connect1l (g : rel V) x z : connect g x z -> z != x -> exists2 y, g x y & connect g y z. Proof. move=> /connectP [[|y p] //= xyp ->]; first by rewrite eqxx. by move: xyp=> /andP[]; exists y => //; apply/connectP; exists p. Qed. Lemma connect1r (g : rel V) x z : connect g x z -> z != x -> exists2 y, connect g x y & g y z. Proof. move=> xz zNx; move: xz; rewrite connect_rev => /connect1l. by rewrite eq_sym => /(_ zNx) [y]; exists y; rewrite // connect_rev. Qed. Section connected. Variable (g : rel V). Definition connected := forall x y, connect g x y. Lemma cover1U (A : {set V}) P : cover (A |: P) = A :|: cover P. Proof. by apply/setP => x; rewrite /cover bigcup_setU big_set1. Qed. Lemma connectedU (A B : {set V}) : {in A &, connected} -> {in B &, connected} -> {in A & B, connected} -> {in B & A, connected} -> {in A :|: B &, connected}. Proof. move=> cA cB cAB cBA z t; rewrite !inE => /orP[zA|zB] /orP[tA|tB]; by[apply: cA|apply: cB|apply: cAB|apply: cBA]. Qed. End connected. Section Symconnect. Variable r : rel V. (* x is symconnected to y *) Definition symconnect x y := connect r x y && connect r y x. Lemma symconnect0 : reflexive symconnect. Proof. by move=> x; apply/andP. Qed. Lemma symconnect_sym : symmetric symconnect. Proof. by move=> x y; apply/andP/andP=> [] []. Qed. Lemma symconnect_trans : transitive symconnect. Proof. move=> x y z /andP[Cyx Cxy] /andP[Cxz Czx]. by rewrite /symconnect (connect_trans Cyx) ?(connect_trans Czx). Qed. Hint Resolve symconnect0 symconnect_sym symconnect_trans. Lemma symconnect_equiv : equivalence_rel symconnect. Proof. by apply/equivalence_relP; split; last apply/sym_left_transitive. Qed. (*************************************************) (* Connected components of the graph, abstractly *) (*************************************************) Definition sccs := equivalence_partition symconnect setT. Lemma sccs_partition : partition sccs setT. Proof. by apply: equivalence_partitionP => ?*; apply: symconnect_equiv. Qed. Definition cover_sccs := cover_partition sccs_partition. Lemma trivIset_sccs : trivIset sccs. Proof. by case/and3P: sccs_partition. Qed. Hint Resolve trivIset_sccs. Notation scc_of := (pblock sccs). Lemma mem_scc x y : x \in scc_of y = symconnect y x. Proof. by rewrite pblock_equivalence_partition // => ?*; apply: symconnect_equiv. Qed. Definition def_scc scc x := @def_pblock _ _ scc x trivIset_sccs. Definition is_subscc (A : {set V}) := A != set0 /\ {in A &, forall x y, connect r x y}. Lemma is_subscc_in_scc (A : {set V}) : is_subscc A -> exists2 scc, scc \in sccs & A \subset scc. Proof. move=> []; have [->|[x xA]] := set_0Vmem A; first by rewrite eqxx. move=> AN0 A_sub; exists (scc_of x); first by rewrite pblock_mem ?cover_sccs. by apply/subsetP => y yA; rewrite mem_scc /symconnect !A_sub. Qed. Lemma is_subscc1 x (A : {set V}) : x \in A -> (forall y, y \in A -> connect r x y /\ connect r y x) -> is_subscc A. Proof. move=> xA AP; split; first by apply: contraTneq xA => ->; rewrite inE. by move=> y z /AP [xy yx] /AP [xz zx]; rewrite (connect_trans yx). Qed. End Symconnect. Lemma setUD (B A C : {set V}) : B \subset A -> C \subset B -> (A :\: B) :|: (B :\: C) = (A :\: C). Proof. move=> subBA subCB; apply/setP=> x; rewrite !inE. have /implyP := subsetP subBA x; have /implyP := subsetP subCB x. by do !case: (_ \in _). Qed. Lemma setUDl (T : finType) (A B : {set T}) : A :|: B :\: A = A :|: B. Proof. by apply/setP=> x; rewrite !inE; do !case: (_ \in _). Qed. Lemma subset_cover (sccs sccs' : {set {set V}}) : sccs \subset sccs' -> cover sccs \subset cover sccs'. Proof. move=> /subsetP subsccs; apply/subsetP=> x /bigcupP [scc /subsccs]. by move=> scc' x_in; apply/bigcupP; exists scc. Qed. Lemma disjoint1s (A : pred V) (x : V) : [disjoint [set x] & A] = (x \notin A). Proof. apply/pred0P/idP=> [/(_ x)/=|]; first by rewrite inE eqxx /= => ->. by move=> xNA y; rewrite !inE; case: eqP => //= ->; apply/negbTE. Qed. Lemma disjoints1 (A : pred V) (x : V) : [disjoint A & [set x]] = (x \notin A). Proof. by rewrite disjoint_sym disjoint1s. Qed. End extra_fintype. ================================================ FILE: notes/tarjan/tarjan_nocolors.v ================================================ From mathcomp Require Import all_ssreflect. Require Import extra_nocolors. Set Implicit Arguments. Unset Strict Implicit. Unset Printing Implicit Defensive. Section tarjan. Variable (V : finType) (successor_seq : V -> seq V). Notation successors x := [set y in successor_seq x]. Notation infty := #|V|. (*************************************************************) (* Tarjan 72 algorithm, *) (* rewritten in a functional style with extra modifications *) (*************************************************************) Record env := Env {esccs : {set {set V}}; num: {ffun V -> nat}}. Definition visited e := [set x | num e x <= infty]. Notation sn e := #|visited e|. Definition stack e := [set x | num e x < sn e]. Definition visit x e := Env (esccs e) (finfun [eta num e with x |-> sn e]). Definition store scc e := Env (scc |: esccs e) [ffun x => if x \in scc then infty else num e x]. Definition dfs1 dfs x e := let: (n1, e1) as res := dfs (successors x) (visit x e) in if n1 < sn e then res else (infty, store (stack e1 :\: stack e) e1). Definition dfs dfs1 dfs (roots : {set V}) e := if [pick x in roots] isn't Some x then (infty, e) else let: (n1, e1) := if num e x <= infty then (num e x, e) else dfs1 x e in let: (n2, e2) := dfs (roots :\ x) e1 in (minn n1 n2, e2). Fixpoint rec k r e := if k is k.+1 then dfs (dfs1 (rec k)) (rec k) r e else (infty, e). Definition e0 := (Env set0 [ffun _ => infty.+1]). Definition tarjan := let: (_, e) := rec (infty * infty.+2) setT e0 in esccs e. (*****************) (* Abbreviations *) (*****************) Notation edge := (grel successor_seq). Notation gconnect := (connect edge). Notation gsymconnect := (symconnect edge). Notation gsccs := (sccs edge). Notation gscc_of := (pblock gsccs). Notation gconnected := (connected edge). Notation new_stack e1 e2 := (stack e2 :\: stack e1). Notation new_visited e1 e2 := (visited e2 :\: visited e1). Notation inord := (@inord infty). (*******************) (* next, and nexts *) (*******************) Section Nexts. Variable (D : {set V}). Definition nexts (A : {set V}) := \bigcup_(v in A) [set w in connect (relfrom (mem D) edge) v]. Lemma nexts0 : nexts set0 = set0. Proof. by rewrite /nexts big_set0. Qed. Lemma nexts1 x : nexts [set x] = x |: (if x \in D then nexts (successors x) else set0). Proof. apply/setP=> y; rewrite /nexts big_set1 !inE. have [->|neq_yx/=] := altP eqP; first by rewrite connect0. apply/idP/idP=> [/connect1l[]// z/=/andP[/= xD xz zy]|]. by rewrite xD; apply/bigcupP; exists z; rewrite !inE. case: ifPn; rewrite ?inE// => xD /bigcupP[z]; rewrite !inE. by move=> xz; apply/connect_trans/connect1; rewrite /= xD. Qed. Lemma nextsU A B : nexts (A :|: B) = nexts A :|: nexts B. Proof. exact: bigcup_setU. Qed. Lemma nextsS (A : {set V}) : A \subset nexts A. Proof. by apply/subsetP=> a aA; apply/bigcupP; exists a; rewrite ?inE. Qed. Lemma nextsT : nexts setT = setT. Proof. by apply/eqP; rewrite eqEsubset nextsS subsetT. Qed. Lemma nexts_id (A : {set V}) : nexts (nexts A) = nexts A. Proof. apply/eqP; rewrite eqEsubset nextsS andbT; apply/subsetP=> x. move=> /bigcupP[y /bigcupP[z zA]]; rewrite !inE => /connect_trans yto /yto zx. by apply/bigcupP; exists z; rewrite ?inE. Qed. Lemma in_nextsW A y : y \in nexts A -> exists2 x, x \in A & gconnect x y. Proof. move=>/bigcupP[x xA]; rewrite inE => xy; exists x => //. by apply: connect_sub xy => u v /andP[_ /connect1]. Qed. End Nexts. Lemma sub_nexts (D D' A B : {set V}) : D \subset D' -> A \subset B -> nexts D A \subset nexts D' B. Proof. move=> /subsetP subD /subsetP subAB; apply/subsetP => v /bigcupP[a /subAB aB]. rewrite !inE => av; apply/bigcupP; exists a; rewrite ?inE //=. by apply: connect_sub av => x y /andP[xD xy]; rewrite connect1//= subD. Qed. Lemma nextsUI A B C : nexts B A \subset A -> A :|: nexts (B :&: ~: A) C = A :|: nexts B C. Proof. move=> subA; apply/setP=> y; rewrite !inE; have [//|/= yNA] := boolP (y \in A). apply/idP/idP; first by apply: subsetP; rewrite sub_nexts// subsetIl. move=> /bigcupP[z zr zy]; apply/bigcupP; exists z; first by []. rewrite !inE; apply: contraTT isT => Nzy; move: zy; rewrite !inE. move=> /(connect_from (mem (~: A))) /= [t]. rewrite !inE => -[xtxy zt ty]; move: zt. rewrite (@eq_connect _ _ (relfrom (mem (B :&: ~: A)) edge)); last first. by move=> u v /=; rewrite !inE andbCA andbA. case: (altP eqP) xtxy => /= [<-|neq_yt]; first by rewrite (negPf Nzy). rewrite implybF negbK => tA zt; rewrite -(negPf yNA) (subsetP subA)//. by apply/bigcupP; exists t; rewrite // inE. Qed. Lemma nexts1_split (A : {set V}) x : x \in A -> nexts A [set x] = x |: nexts (A :\ x) (successors x). Proof. move=> xA; apply/setP=> y; apply/idP/idP; last first. rewrite nexts1 !inE xA; case: (_ == _); rewrite //=. by apply: subsetP; rewrite sub_nexts// subsetDl. move=> /bigcupP[z]; rewrite !inE => /eqP {z}->. move=> /connectP[p /shortenP[[_ _ _ /eqP->//|z q/=/andP[/andP[_ xz]]]]]. rewrite path_from => /andP[zq] /allP/= qA. move=> /and3P[xNzq _ _] _ ->; apply/orP; right. apply/bigcupP; exists z; rewrite !inE//. apply/connectP; exists q; rewrite // path_from zq/=. apply/allP=> t tq; rewrite !inE qA ?andbT//. by apply: contraNneq xNzq=> <-; apply: mem_belast tq. Qed. (*******************) (* Well formed env *) (*******************) Lemma num_le_infty e x : num e x <= infty = (x \in visited e). Proof. by rewrite inE. Qed. Lemma num_lt_sn e x : num e x < sn e = (x \in stack e). Proof. by rewrite inE. Qed. Lemma visited_visit e x : visited (visit x e) = x |: visited e. Proof. by apply/setP=> y; rewrite !inE ffunE/=; case: (altP eqP); rewrite ?max_card. Qed. Lemma sub_stack_visited e : stack e \subset visited e. Proof. by apply/subsetP => x; rewrite !inE => /ltnW /leq_trans ->//; rewrite max_card. Qed. Lemma sub_new_stack_visited e1 e2: new_stack e1 e2 \subset visited e2. Proof. by rewrite (subset_trans _ (sub_stack_visited _)) ?subsetDl. Qed. Section wfenv. Record wf_env e := WfEnv { sub_gsccs : esccs e \subset gsccs; num_lt_V_is_stack : forall x, num e x < infty -> num e x < sn e; num_sccs : forall x, (num e x == infty) = (x \in cover (esccs e)); le_connect : forall x y, num e x <= num e y < sn e -> gconnect x y; }. Variables (e : env) (e_wf : wf_env e). Lemma num_gt_V x : x \notin visited e -> num e x > infty. Proof. by rewrite inE -ltnNge. Qed. Lemma num_lt_V x : (num e x < infty) = (num e x < sn e). Proof. apply/idP/idP => [/num_lt_V_is_stack//|]; first exact. by move=> /leq_trans; apply; rewrite max_card. Qed. Lemma num_lt_card x (A : pred V) : visited e \subset A -> (num e x < #|A|) = (num e x < sn e). Proof. move=> subeA; apply/idP/idP => /leq_trans. by rewrite -num_lt_V; apply; rewrite max_card. by apply; rewrite subset_leq_card. Qed. Lemma visitedE : visited e = stack e :|: cover (esccs e). Proof. by apply/setP=> x; rewrite !inE leq_eqVlt -num_sccs// num_lt_V orbC. Qed. Lemma sub_sccs_visited : cover (esccs e) \subset visited e. Proof. by apply/subsetP => x; rewrite !inE -num_sccs// => /eqP->. Qed. Lemma stack_visit x : x \notin visited e -> stack (visit x e) = x |: stack e. Proof. move=> xNvisited; apply/setP=> y; rewrite !inE/= ffunE/= visited_visit. have [->|neq_yx]//= := altP eqP; first by rewrite cardsU1 xNvisited ltnS ?leqnn. by rewrite num_lt_card// subsetUr. Qed. End wfenv. Lemma wf_visit e x : wf_env e -> (forall y, num e y < sn e -> gconnect y x) -> x \notin visited e -> wf_env (visit x e). Proof. move=> e_wf x_connected xNvisited. constructor=> [|y|y|] //=; rewrite ?inE ?ffunE/=. - exact: sub_gsccs. - rewrite visited_visit cardsU1 xNvisited; case: ifPn => // _. by rewrite num_lt_V// ltnS => /ltnW. - have [->|] := altP (y =P x); last by rewrite num_sccs. rewrite -num_sccs// eq_sym !gtn_eqF ?num_gt_V//. by rewrite (@leq_trans #|x |: visited e|) ?max_card// cardsU1 xNvisited. move=> y z; rewrite !ffunE/=. have sub_visit : visited e \subset visited (visit x e). by apply/subsetP => ?; rewrite visited_visit !inE orbC => ->. have [{y}->|neq_yx] := altP eqP; have [{z}->|neq_zx]//= := altP eqP. + by rewrite num_lt_card//; case: ltngtP. + move=> /andP[/leq_ltn_trans lt/lt]. by rewrite num_lt_card//; apply: x_connected. + by rewrite num_lt_card//; apply: le_connect. Qed. Definition subenv e1 e2 := [&& esccs e1 \subset esccs e2, [forall x, (num e1 x <= infty) ==> (num e2 x == num e1 x)] & [forall x, (num e2 x < sn e1) ==> (num e1 x < sn e1)]]. Lemma sub_sccs e1 e2 : subenv e1 e2 -> esccs e1 \subset esccs e2. Proof. by move=> /and3P[]. Qed. Lemma sub_snum e1 e2 : subenv e1 e2 -> forall x, num e1 x <= infty -> num e2 x = num e1 x. Proof. by move=> /and3P[_ /forall_inP /(_ _ _) /eqP]. Qed. Lemma sub_vnum e1 e2 : subenv e1 e2 -> forall x, num e1 x < sn e1 -> num e2 x = num e1 x. Proof. move=> sube12 x /ltnW num_lt; rewrite (sub_snum sube12)//. by rewrite (leq_trans num_lt)// max_card. Qed. Lemma sub_num_lt e1 e2 : subenv e1 e2 -> forall x, (num e1 x < sn e1) = (num e2 x < sn e1). Proof. move=> /and3P[_ /forall_inP /(_ _ _)/eqP num_eq /forall_inP] num_lt x. have nume1_lt := num_lt x; apply/idP/idP => // {nume1_lt}nume1_lt. by rewrite num_eq ?inE// (leq_trans (ltnW nume1_lt))// max_card. Qed. Lemma sub_visited e1 e2 : subenv e1 e2 -> visited e1 \subset visited e2. Proof. move=> sube12; apply/subsetP=> x; rewrite !inE => x_visited1. by rewrite (sub_snum sube12)// inE. Qed. Lemma leq_sn e1 e2 : subenv e1 e2 -> sn e1 <= sn e2. Proof. by move=> sube12; rewrite subset_leq_card// sub_visited. Qed. Lemma sub_stack e1 e2 : subenv e1 e2 -> stack e1 \subset stack e2. Proof. move=> sube12; apply/subsetP=> x; rewrite !inE => x_stack. by rewrite (sub_vnum sube12)// (leq_trans x_stack)// leq_sn. Qed. Lemma new_stackE e1 e2 : subenv e1 e2 -> new_stack e1 e2 = [set x | sn e1 <= num e2 x < sn e2]. Proof. move=> sube12; apply/setP=> x; rewrite !inE. have [x_e2|] := ltnP (num e2 x) (sn e2); rewrite ?andbT ?andbF//. have [e1_after|e1_before] /= := leqP (sn e1) (num e1 x). by rewrite leqNgt -sub_num_lt// -leqNgt. by rewrite leqNgt -sub_num_lt// e1_before. Qed. Lemma new_visitedE e1 e2 : wf_env e1 -> wf_env e2 -> subenv e1 e2 -> (new_visited e1 e2) = (new_stack e1 e2) :|: cover (esccs e2) :\: cover (esccs e1). Proof. move=> e1_wf e2_wf sube12; rewrite !visitedE//; apply/setP=> x. rewrite !inE -!num_sccs -?num_lt_V//; do 2!case: ltngtP => //=. by rewrite num_lt_V// (sub_num_lt sube12)// => ->; rewrite ltnNge max_card. by move=> xe2 xe1; move: xe2; rewrite (sub_snum sube12)// ?xe1// ltnn. Qed. Lemma sub_new_stack_new_visited e1 e2 : subenv e1 e2 -> wf_env e1 -> wf_env e2 -> (new_stack e1 e2) \subset (new_visited e1 e2). Proof. by move=> e1wf e2wf sube12; rewrite (@new_visitedE e1 e2)// subsetUl. Qed. Lemma sub_refl e : subenv e e. Proof. by rewrite /subenv !subxx /=; apply/andP; split; apply/forall_inP. Qed. Hint Resolve sub_refl. Lemma sub_trans : transitive subenv. Proof. move=> e2 e1 e3 sub12 sub23; rewrite /subenv. rewrite (subset_trans (sub_sccs sub12))// ?sub_sccs//=. apply/andP; split; apply/forall_inP=> x xP. by rewrite (sub_snum sub23) ?(sub_snum sub12)//. have x2 : num e3 x < sn e2 by rewrite (leq_trans xP)// leq_sn. by rewrite (sub_num_lt sub12)// -(sub_vnum sub23)// (sub_num_lt sub23). Qed. Lemma sub_visit e x : x \notin visited e -> subenv e (visit x e). Proof. move=> xNvisited; rewrite /subenv subxx/=; apply/andP; split; last first. by apply/forall_inP => y; rewrite !ffunE/=; case: ifP; rewrite ?ltnn. apply/forall_inP => y y_in; rewrite !ffunE/=. by case: (altP (y =P x)) xNvisited => // <-; rewrite inE y_in. Qed. Lemma visited_store (A : {set V}) e : A \subset visited e -> visited (store A e) = visited e. Proof. move=> A_sub; apply/setP=> x; rewrite !inE/= ffunE. by case: ifPn => // /(subsetP A_sub); rewrite inE leqnn => ->. Qed. Lemma stack_store (A : {set V}) e : A \subset visited e -> stack (store A e) = stack e :\: A. Proof. move=> A_sub; apply/setP => x; rewrite !inE visited_store//= ffunE. by case: (x \in A); rewrite //= ltnNge max_card. Qed. (*********************) (* DFS specification *) (*********************) Definition outenv (roots : {set V}) (e e' : env) := [/\ {in new_stack e e' &, gconnected}, {in new_stack e e', forall x, exists2 y, y \in stack e & gconnect x y} & visited e' = visited e :|: nexts (~: visited e) roots ]. Variant dfs_spec_def (dfs : nat * env) (roots : {set V}) e : (nat * env) -> nat -> env -> Type := DfsSpec ne' (n : nat) e' of ne' = (n, e') & n = \min_(x in nexts (~: visited e) roots) inord (num e' x) & wf_env e' & subenv e e' & outenv roots e e' : dfs_spec_def dfs roots e ne' n e'. Notation dfs_spec ne' roots e := (dfs_spec_def ne' roots e ne' ne'.1 ne'.2). Definition dfs_correct dfs (roots : {set V}) e := wf_env e -> {in stack e & roots, gconnected} -> dfs_spec (dfs roots e) roots e. Definition dfs1_correct dfs1 x e := wf_env e -> x \notin visited e -> {in stack e & [set x], gconnected} -> dfs_spec (dfs1 x e) [set x] e. (*****************) (* Correctness *) (*****************) Lemma dfsP dfs1 dfsrec (roots : {set V}) e: (forall x, x \in roots -> dfs1_correct dfs1 x e) -> (forall x, x \in roots -> forall e1, subenv e e1 -> dfs_correct dfsrec (roots :\ x) e1) -> dfs_correct (dfs dfs1 dfsrec) roots e. Proof. rewrite /dfs => dfs1P dfsP e_wf roots_connected. case: pickP => /= [x x_roots|]; last first. move=> r0; have {r0}r_eq0 : roots = set0 by apply/setP=> x; rewrite inE. do ?constructor=> //=; rewrite ?setDv ?r_eq0 ?nexts0 ?sub0set ?eqxx ?setU0 ?big_set0 //=; by move=> ?; rewrite inE. have [numx_gt|numx_le]/= := ltnP; last first. have x_visited : x \in visited e by rewrite inE. case: dfsP => //= [u v ve|_ _ e1 ->-> e1_wf subee1 [new1c new1old visited1E]]. by rewrite inE => /andP[_ v_roots]; rewrite roots_connected. constructor => //=. rewrite -[in RHS](setD1K x_roots) nextsU nexts1 inE x_visited/= setU0. by rewrite bigmin_setU /= big_set1/= (@sub_snum e e1)// inordK//. constructor=> //=; rewrite -(setD1K x_roots) nextsU nexts1 inE x_visited/=. by rewrite setU0 setUCA setUA [x |: _](setUidPr _) ?sub1set. case: dfs1P => //=; first by rewrite inE -ltnNge. by move=> u v ue; rewrite inE => /eqP->; apply: roots_connected. move=> _ _ e1 -> -> e1_wf subee1 [new1c new1old visited1E]. case: dfsP => //= [u v ue1|_ _ e2 -> -> e2_wf sube12 [new2c new2old visited2E]]. rewrite inE => /andP[_ v_roots]. have [ue|uNe] := boolP (u \in stack e); first by rewrite roots_connected. have [|w we] := new1old u; first by rewrite inE ue1 uNe. by move=> /connect_trans->//; rewrite roots_connected//. have sube2 : subenv e e2 by exact: sub_trans sube12. have nexts_split : nexts (~: visited e) roots = nexts (~: visited e) [set x] :|: nexts (~: visited e1) (roots :\ x). rewrite -[in LHS](setD1K x_roots) nextsU visited1E. by rewrite setCU nextsUI// nexts_id. constructor => //=. rewrite (eq_bigr (inord \o num e2)). by rewrite -[LHS]/(val (ord_minn _ _)) -bigmin_setU /= -nexts_split. move=> y y_in; rewrite /= (@sub_snum e1 e2)// num_le_infty. by rewrite visited1E setUC inE y_in. constructor => /=. + rewrite -(@setUD _ (stack e1)) ?sub_stack//. apply: connectedU => // y z; last first. rewrite !new_stackE// ?inE => /andP[y_ge y_lt] /andP[z_ge z_lt]. rewrite (@le_connect e2) // z_lt (leq_trans _ z_ge)//. by rewrite (sub_vnum sube12)// ltnW. rewrite !new_stackE// ?inE => /andP[y_ge y_lt] /andP[z_ge z_lt]. have [|r] := new2old y; rewrite ?new_stackE ?inE ?y_ge//. move=> r_lt /connect_trans->//; have [rz|zr] := leqP (num e1 r) (num e1 z). by rewrite (@le_connect e1)// rz/=. by rewrite new1c ?new_stackE ?inE ?z_ge ?z_lt //= (leq_trans z_ge)// ltnW. + move=> y; rewrite ?new_stackE ?inE// => /andP[y_ge y_lt]. have [y_lt1|y_ge1] := ltnP (num e1 y) (sn e1). have [|r] := new1old y; last by exists r. by rewrite new_stackE ?inE// ?y_lt1 -(sub_vnum sube12) ?y_ge. have [|r r_lt1 yr] := new2old y; first by rewrite !inE -leqNgt y_ge1//. rewrite ?inE in r_lt1; have [r_lt|r_ge] := ltnP (num e r) (sn e). by exists r; rewrite ?inE. have [|r' r's rr'] := new1old r; first by rewrite ?inE -leqNgt r_ge r_lt1. by exists r'; rewrite // (connect_trans yr rr'). + by rewrite visited2E {1}visited1E nexts_split setUA. Qed. Lemma dfs1P dfs x e (A := successors x) : dfs_correct dfs A (visit x e) -> dfs1_correct (dfs1 dfs) x e. Proof. rewrite /dfs1 => dfsP e_wf xNvisited x_connected. have subexe: subenv e (visit x e) by exact: sub_visit. have numx : num e x > infty by apply: num_gt_V. have xNstack : x \notin stack e. by rewrite inE -leqNgt (leq_trans _ numx) ?leqW ?max_card. have xe_wf : wf_env (visit x e). by apply: wf_visit => // y y_lt; rewrite x_connected ?inE. have nexts1E : nexts (~: visited e) [set x] = x |: nexts (~: (x |: visited e)) A. by rewrite nexts1_split ?setDE ?setCU 1?setIC 1?inE. case: dfsP => //=. rewrite stack_visit// => u v; rewrite in_setU1=> /predU1P[->|ue]; rewrite inE => /(@connect1 _ edge)// /(connect_trans _)->//. by rewrite x_connected// set11. move=> _ _ e1 //= -> -> e1_wf subxe1 [newc new_old visited1E]. have sube1 : subenv e e1 by apply: sub_trans subxe1. have num1x : num e1 x = sn e. by rewrite (sub_snum subxe1)// ?inE ?ffunE/= ?eqxx// max_card. rewrite visited_visit in visited1E *. have lt_sn_sn1 : sn e < sn e1. by rewrite (leq_trans _ (leq_sn subxe1))// visited_visit cardsU1 xNvisited. have x_visited1 : x \in visited e1 by rewrite visited1E inE setU11. have x_stack : x \in stack e1. by rewrite (subsetP (sub_stack subxe1))//= stack_visit// setU11. have [min_after|min_before] := leqP; last first. constructor => //=. rewrite nexts1E bigmin_setU big_set1 /= inordK ?num1x ?ltnS ?max_card//. by rewrite (minn_idPr _)// ltnW. constructor=> //=; last by rewrite nexts1E setUCA setUA visited1E. move=> y z; have [-> _|neq_yx] := eqVneq y x. by rewrite new_stackE ?inE// -num1x; apply: le_connect. rewrite -(@setUD _ (stack (visit x e))) ?sub_stack//. rewrite [in X in _ :|: X]stack_visit// setDUl setDv setU0. rewrite [_ :\: stack e](setDidPl _) ?disjoint1s//. rewrite setUC !in_setU1 (negPf neq_yx)/=. move=> y_e1 /predU1P[->|]; last exact: newc y_e1. have [t] := new_old y y_e1; rewrite !inE => t_le /connect_trans->//. rewrite (@le_connect (visit x e))// andbC; move: t_le. by rewrite visited_visit !ffunE /= eqxx cardsU1 xNvisited add1n !ltnS leqnn. move=> y; have [v ve xv] : exists2 v, v \in stack e & gconnect x v. have [|v] := @eq_bigmin_cond _ _ (mem (nexts (~: (x |: visited e)) A)) (inord \o num e1). rewrite card_gt0; apply: contraTneq min_before => ->. by rewrite big_set0 -leqNgt max_card. rewrite !inE => v_in min_is_v; move: min_before; rewrite min_is_v/=. rewrite inordK; last by rewrite ltnS num_le_infty visited1E inE v_in orbT. rewrite -sub_num_lt// => v_lt; exists v; rewrite ?inE//. move: v_in => /in_nextsW[z]; rewrite inE => /(@connect1 _ edge). by apply: connect_trans. rewrite -(@setUD _ (stack (visit x e))) ?sub_stack//. rewrite [in X in _ :|: X]stack_visit// setDUl setDv setU0. rewrite [_ :\: stack e](setDidPl _) ?disjoint1s// setUC !in_setU1. move=> /predU1P[->|]; first by exists v. move=> /new_old[z]; rewrite stack_visit// in_setU1. move=> /predU1P[->|]; last by exists z. by move=> yx; exists v; rewrite // (connect_trans yx). have all_geq y : y \in nexts (~: visited e) [set x] -> (#|visited e| <= num e1 y) * (num e1 y <= infty). have := min_after; have sn_inord : sn e = inord (sn e). by rewrite inordK// ltnS max_card. rewrite {1}sn_inord; move/bigmin_geqP => /(_ y) y_ge. rewrite nexts1E !inE => /predU1P[->|yA]; rewrite ?num1x ?max_card ?leqnn//. rewrite sn_inord (leq_trans (y_ge _))// ?inordK//; by rewrite ?ltnS num_le_infty visited1E 2!inE yA orbT. constructor => //=. - rewrite big1// => y xy; rewrite ffunE new_stackE ?inE//=. have y_visited1 : num e1 y <= infty. by rewrite num_le_infty visited1E -setUA setUCA -nexts1E inE xy orbT. apply/val_inj=> /=; case: ifPn; rewrite ?inordK//. by rewrite all_geq//= -num_lt_V// -leqNgt; move: y_visited1; case: ltngtP. - constructor => //=; rewrite ?visited_store ?sub_new_stack_visited//. + rewrite subUset sub_gsccs// andbT sub1set. suff -> : new_stack e e1 = gscc_of x by rewrite pblock_mem ?cover_sccs. apply/setP=> y; rewrite mem_scc /symconnect. have [->|neq_yx] := eqVneq y x. by rewrite connect0 inE xNstack inE num1x lt_sn_sn1. apply/idP/andP=> [|[xy yx]]. move=> y_ee1; have y_xee1 : y \in new_stack (visit x e) e1. by rewrite inE stack_visit// in_setU1 (negPf neq_yx)/= -in_setD. split; last first. have [z] := new_old _ y_xee1. rewrite stack_visit// in_setU1 => /predU1P[->//|/x_connected]. by move=> /(_ _ (set11 x))/(connect_trans _) xz /xz. have: y \in new_visited (visit x e) e1. by apply: subsetP y_xee1; rewrite sub_new_stack_new_visited. rewrite inE visited1E in_setU visited_visit//; case: (y \in _) => //=. move=> /in_nextsW[z]; rewrite inE=> /(@connect1 _ edge). exact: connect_trans. have /(connect_from (mem (~: visited e))) [z []] := xy; rewrite inE. move=> eq_yz xz zy; have /all_geq [] : z \in nexts (~: visited e) [set x]. by apply/bigcupP; exists x; rewrite !inE. rewrite leqNgt -sub_num_lt// -num_lt_V// -leqNgt => zNstack. have zNcover e' : wf_env e' -> z \in cover (esccs e') -> x \in cover (esccs e'). move=> e'_wf /bigcupP[C] Ce zC; apply/bigcupP; exists C => //. have /def_scc: C \in gsccs by apply: subsetP Ce; apply: sub_gsccs. move=> /(_ _ zC)<-; rewrite mem_scc /= /symconnect (connect_trans zy)//=. by apply: connect_sub xz => ?? /andP[_ /connect1]. rewrite leq_eqVlt num_sccs// num_lt_V// => /orP[|z_stack]. move=> /zNcover; rewrite -num_sccs// num1x => /(_ _) /eqP eq_V. by rewrite eq_V// ltnNge max_card in lt_sn_sn1. have zNvisited : z \notin visited e. rewrite inE -ltnNge ltn_neqAle zNstack andbT/= eq_sym num_sccs//. by apply: contraTN isT => /(zNcover _ e_wf); rewrite -num_sccs// gtn_eqF. move: eq_yz; rewrite zNvisited /= => /andP[/eqP eq_yz _]. rewrite -eq_yz in zNstack z_stack. by rewrite !inE -num_lt_V// -leqNgt zNstack. + move=> v; rewrite ffunE/=; case: ifPn; rewrite ?ltnn// => vNe12. by rewrite num_lt_V// visited_store. + move=> v; rewrite ffunE /= cover1U [in RHS]inE. by case: ifPn; rewrite ?eqxx//= => vNe12; rewrite -num_sccs//. + move=> y z; rewrite !ffunE; case: ifPn => _. by move=> /andP[/leq_ltn_trans Vsmall/Vsmall]; rewrite ltnNge max_card. by case: ifPn => _; [by rewrite ltnNge max_card andbF|exact : le_connect]. - rewrite /subenv /= (subset_trans (sub_sccs sube1)) ?subsetUr//=. apply/andP; split; apply/forallP => v; apply/implyP; rewrite ffunE/= new_stackE// ?inE. move=> vs; rewrite (sub_snum sube1)// leqNgt -!num_lt_V// -leqNgt ifN//. by apply/negP => /andP[/leq_ltn_trans Vlt/Vlt]; rewrite ltnNge max_card. by case: ifPn; [move=> _; rewrite ltnNge max_card|rewrite -sub_num_lt]. - rewrite /outenv stack_store ?visited_store ?sub_new_stack_visited//. rewrite setDDr setDUl setDv set0D set0U setDIl !setDv setI0. split; do ?by move=> ?; rewrite inE. by rewrite visited1E -setUA setUCA -nexts1E. Qed. Theorem rec_terminates k (roots : {set V}) e : k >= #|~: visited e| * infty.+1 + #|roots| -> dfs_correct (rec k) roots e. Proof. move=> k_ge; elim: k => [|k IHk/=] in roots e k_ge *. move: k_ge; rewrite leqn0 addn_eq0 cards_eq0 => /andP[_ /eqP-> e_wf _]/=. constructor=> //=; rewrite /outenv ?nexts0 ?setDv ?big_set0// ?setU0. by split=> // ?; rewrite inE. apply: dfsP=> x x_roots; last first. move=> e1 subee1; apply: IHk; rewrite -ltnS (leq_trans _ k_ge)//. rewrite (cardsD1 x roots) x_roots add1n -addSnnS ltn_add2r ltnS. by rewrite leq_mul2r //= subset_leq_card// setCS sub_visited. move=> e_wf xNvisited; apply: dfs1P => //; apply: IHk. rewrite visited_visit setCU setIC -setDE -ltnS (leq_trans _ k_ge)//. rewrite (cardsD1 x (~: _)) inE xNvisited add1n mulSnr -addnA ltn_add2l. by rewrite ltn_addr// ltnS max_card. Qed. Lemma visited0 : visited e0 = set0. Proof. by apply/setP=> y; rewrite !inE ffunE ltnn. Qed. Lemma stack0 : stack e0 = set0. Proof. by apply/setP=> y; rewrite !inE ffunE ltnNge leqW ?max_card. Qed. Theorem tarjan_correct : tarjan = gsccs. Proof. rewrite /tarjan mulnSr; case: rec_terminates. - by rewrite visited0 setC0 cardsT. - constructor; rewrite /= ?sub0set// => x; rewrite !ffunE//. + by rewrite ltnNge leqW//. + by rewrite gtn_eqF// /cover big_set0 inE. + by move=> y; rewrite !ffunE//= andbC ltnNge leqW// ?max_card. - by move=> y; rewrite !inE !ffunE/= ltnNge leqW// max_card. move=> _ _ e -> _ e_wf _ [_]; rewrite stack0 setD0. have [stacke _|[x xe]] := set_0Vmem (stack e); last first. by move=> /(_ _ xe)[?]; rewrite inE. rewrite visited0 set0U setC0 nextsT => visitede. have numE x : num e x = infty. apply/eqP; have /setP/(_ x) := visitede. by rewrite visitedE// stacke set0U !inE -num_sccs. apply/eqP; rewrite eqEsubset sub_gsccs//=; apply/subsetP => _/imsetP[/=x _->]. have: x \in cover (esccs e) by rewrite -num_sccs ?numE//. move=> /bigcupP [C Csccs /(def_scc (subsetP (sub_gsccs e_wf) _ Csccs))] eqC. rewrite -eqC (_ : [set _ in _ | _] = gscc_of x)// in Csccs *. by apply/setP => y; rewrite !inE mem_scc /=. Qed. End tarjan. ================================================ FILE: notes/tarjan.md ================================================ so as we're going through the depth first search, we store: a stack of visited but not yet assigned vertices, pushed onto the stack in the order they are visited; a set of finalized components; the current serial number; and a "function" (map?) of vertices to serial numbers. two mutually recursive functions, they call then `dfs1` and `dsf`, but those are awful names, I'll renamed once I fully understand them - `dfs` takes a set of roots and an initial environment, returns a pair of an integer and the modified environment. if the roots is empty the integer is `infinity` (what they should have done is just used `option` or something here). Otherwise the returned integer is the minimum of the results of the calls to `dfs1` on non-visited vertices in r and of the serial numbers of the already visited ones. - `dfs1` the main function creates the initial environment with an empty stack, empty set of components, serial number 0, and an empty map assigning numbers to vertices ``` let rec dfs1 vertex e = let n0 = e.cur in let (n1, e1) = dfs (successors vertex) (add_stack_incr vertex e) in if n1 < n0 then (n1, e1) else let (s2, s3) = split x e1.stack in (+∞, {stack = s3; sccs = add (elements s2) e1.sccs; cur = e1.cur; num = set_infty s2 e1.num}) with dfs r e = if is_empty r then (+∞, e) else let x = choose r in let r’ = remove x r in let (n1, e1) = if e.num[x] != -1 then (e.num[x], e) else dfs1 x e in let (n2, e2) = dfs r’ e1 in (min n1 n2, e2) let tarjan () = let e = {stack = []; sccs = empty; cur = 0; num = const (-1)} in let (_, e’) = dfs vertices e in e’.sccs let add_stack_incr x e = let n = e.cur in {stack = x :: e.stack; sccs = e.sccs; cur = n+1; num = e.num[x ← n]} let rec set_infty s f = match s with | [] → f | x :: s’ → (set_infty s’ f)[x ← +∞] end let rec split x s = match s with | [] → ([], []) | y :: s’ → if x = y then ([x], s’) else let (s1’, s2) = split x s’ in (y :: s1’, s2) end ``` looks like I need to check out their better version https://www-sop.inria.fr/marelle/Tarjan/ https://math-comp.github.io/mcb/ Łukasz Czajka and Cezary Kaliszyk. Hammer for coq: Automation for dependent type theory. J. Autom. Reasoning, 61(1-4):423–453, 2018. ================================================ FILE: notes.md ================================================ modified orphan rule: traits can have crate *automatic derive implementations* that will "kick in" when a type the trait author hasn't explicitly defined an implementation for "requests" one. this means such trait authors could merely define manual implementations for the primitive types. this automatic implementation can be superseded by an explicit implementation in the type crate what about derive arguments for third-party crates? honestly easy to still solve using the newtype pattern https://people.mpi-sws.org/~dreyer/papers/sandboxing/paper.pdf https://people.mpi-sws.org/~beta/papers/unicoq.pdf https://www.sciencedirect.com/science/article/pii/S089054010300138X?via%3Dihub https://golem.ph.utexas.edu/category/2021/08/you_could_have_invented_de_bru.html https://proofassistants.stackexchange.com/questions/900/when-should-i-use-de-bruijn-levels-instead-of-indices https://www.sciencedirect.com/science/article/pii/0167642395000216 https://arxiv.org/pdf/1102.2405.pdf https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.156.1170 https://davidchristiansen.dk/tutorials/nbe/ when defining the "handy" or base equality proposition (`==`), why not make it an `or` over strict definitional equality (`===`) and an existence proof over a more complex setoid equality? or simply not include it as a base at all and let operator chaining allow people to use more complex custom equality concepts themselves? maybe we should pull apart "definitional" or "shallow" equality from "computable" equality? is that useful? https://inria.hal.science/hal-01094195/preview/CIC.pdf notes on `match` `match t as x` could probably just be rewritten with a `let` beforehand? or do like rust! `match let x: I[y1, ... yp] = t {}` the `in I y1 . . . yp` is basically a destructuring of the *type* of the match target. it allows you to use the *index* values of the type in the body. a better syntax that makes this more clear would be `match t: I y1 ... yp { ...match arms... }` every arm can have a different type, which means the *type* of the entire match *is itself another `match`!*, but one that returns a type and not a value which has a type strictly speaking the `return P` isn't necessary, but it allows you to use the `in whatever` type you've destructured in other places A match type checks mostly based on its `return` clause. to type check a match, you have to: - check all constructors are accounted for - check that each arm's type aligns with the `return` clause, including when the `return` is a function in which case you run the concrete pattern through that function this is how "absurdity" is possible, when the `return` clause gives a *different* type for the *actual* input data than it does for the concrete constructor arms! this can only happen when the input data is impossible to construct, because otherwise the constructor arms would definitely handle it since by construction they cover every possibility. ``` Definition do_inversion (x y: N): Prop := match x, y with S _, z => False | _,_ => True end. Definition le_inversion n (H: le (S n) z): False := match H in le x y return do_inversion x y with | lez x => True() | leS x y p => True() end. ``` there are also rules about which *sort* the match target and the `return` type are, and it *broadly* seems you just have to keep `Type` and `Prop` separate? that seems like a massive oversimplification, but... `fixpoint` definitions just have to make sense and syntactically terminate it seems that inductive types with indexed values are *usefully conceptually different than functions* syntactically they're similar, but it's almost like "primitive" functions that don't have a body, but instead are just "declared" to give a value of a certain type you could definitely frame these as something similar to a generic type, but there's even still a difference between that concept and an indexed type A universal asserted type constructor would really help basically you don't need `exists` or special case subset types like `vec` if you have a general purpose asserted type that's directly supported in the syntax you *don't need index values at all* for `Type` definitions if you have a universal asserted type system. this means you could do index values for `Prop` very differently, not treating them like functions that eventually return `Prop` but something more similar to generics ``` @(A: Type, R: (A, A) -> Prop) prop RT(A, A); // | RTrefl: @(x), RT x x | RTrefl(x): RT x x // | RTR: @(x, y), R x y -> RT x y // @() specifies names, types optional, () specifies types without names | RTR@(x, y)(R x y): RT x y // | RTtran: @(x, y, z), RT x y -> RT y z -> RT x z | RTtran@(x, y, z)(RT x y, RT y z): RT x z prop le(N, N); | lez: @(x) -> le(0, x) | leS: @(x, y), le x y -> le(S x, S y) prop le[N, N]; | lez(x): [0, x] | leS(x, y)&(le[x, y]): [S x, S y] or you could use <> instead of [] ``` to demonstrate that an imaginary type "models" a real type, you have to provide: - a function that can always convert (the separation logic representation of) the real type to the imaginary type - a function that can convert an imaginary type into a real type with a proof that this function is a mirror image of the above function https://www.cs.cmu.edu/~fp/papers/mfps89.pdf https://github.com/VictorTaelin/calculus-of-constructions https://github.com/coq/coq/wiki/TheoryBehindCoq https://softwarefoundations.cis.upenn.edu/lf-current/ProofObjects.html https://softwarefoundations.cis.upenn.edu/lf-current/Logic.html https://www.researchgate.net/figure/Sketch-of-type-checking-rules-in-Coq_fig17_221336389 https://www.williamjbowman.com/tmp/wjb-sized-coq.pdf https://www.labri.fr/perso/casteran/CoqArt/Tsinghua/C5.pdf https://hal.science/hal-02380196/document https://coq.inria.fr/refman/language/cic.html https://arxiv.org/pdf/2105.12077.pdf need to look at xcap paper and other references in the bedrock paper https://plv.csail.mit.edu/blog/iris-intro.html#iris-intro https://plv.csail.mit.edu/blog/alectryon.html#alectryon Verified hardware simulators are easy with magmide Engineers want tools that can give them stronger guarantees about safety robustness and performance, but that tool has to be tractably usable and respect their time This idea exists in incentive no man's land. Academics won't think about it or care about it because it merely applies existing work, so they'll trudge along in their tenure track and keep publishing post hoc verifications of existing systems. Engineers won't think about or care about it because it can't make money quickly or be made into a service or even very quickly be used to improve some service This is an idea that carries basically zero short term benefits, but incalculable long term ones, mainly in the way it could shift the culture of software and even mathematics and logic if successful. This project is hoping and gambling that it itself won't even be the truly exciting innovation, but some other project that builds upon it, and that wouldn't have happened otherwise. I'm merely hoping to be the pair of shoulders someone else stands on, and I hope the paradigm shift this project creates is merely assumed to be obvious, that they'll think we were insane to write programs and not prove them correct https://mattkimber.co.uk/avoiding-growth-by-accretion/ Most effects aren't really effects but environmental capabilities, although sometimes those capabilities come with effects Traits, shapes, and the next level of type inference Discriminated unions and procedural macros make dynamically typed languages pointless, and they've existed for eighty years. So what gives? What's better than a standard? An automatically checkable and enforceable standard https://project-oak.github.io/rust-verification-tools/2021/09/01/retrospective.html we have to go all the way. anything less than the capabilities given by a full proof checker proving theories on the literal environment abstractions isn't going to be good enough, will always have bugs and hard edges and cases that can't be done. but those full capabilties can *contain* other more "ad hoc" things like fuzzers, quickcheck libraries, test generators, etc. we must build upon a magmide! stop trying to make functional programming happen, it isn't going to happen ## project values - **Correctness**: this project should be a flexible toolkit capable of verifying and compiling any software for any architecture or environment. It should make it as easy as possible to model the abstractions presented by any hardware or host system with full and complete fidelity. - **Clarity**: this project should be accessible to as many people as possible, because it doesn't matter how powerful a tool is if no one can understand it. To guide us in this pursuit we have a few maxims: speak plainly and don't use jargon when simpler words could be just as precise; don't use a term unless you've given some path for the reader to understand it, if a topic has prerequisites point readers toward them; assume your reader is capable but busy; use fully descriptive words, not vague abbreviations and symbols. - **Practicality**: a tool must be usable, both in terms of the demands it makes and its design. This tool is intended to be used by busy people building real things with real stakes. - **Performance**: often those programs which absolutely must be fast are also those which absolutely must be correct. Infrastructural software is constantly depended on, and must perform well. These values inherently reinforce one another. As we gain more ability to guarantee correctness, we can make programs faster and solve more problems. As our tools become faster, they become more usable. Guaranteeing correctness saves others time and headache dealing with our bugs. As we improve clarity, more people gather to help improve the project, making it even better in every way. secondary values, simplicity before consistency before completeness. cultural values, code of conduct, we're accepting and open and humble. ``` In the spirit of Richard Gabriel, the Pony philosophy is neither "the-right-thing" nor "worse-is-better". It is "get-stuff-done". Correctness Incorrectness is simply not allowed. It's pointless to try to get stuff done if you can't guarantee the result is correct. Performance Runtime speed is more important than everything except correctness. If performance must be sacrificed for correctness, try to come up with a new way to do things. The faster the program can get stuff done, the better. This is more important than anything except a correct result. Simplicity Simplicity can be sacrificed for performance. It is more important for the interface to be simple than the implementation. The faster the programmer can get stuff done, the better. It's ok to make things a bit harder on the programmer to improve performance, but it's more important to make things easier on the programmer than it is to make things easier on the language/runtime. Consistency Consistency can be sacrificed for simplicity or performance. Don't let excessive consistency get in the way of getting stuff done. Completeness It's nice to cover as many things as possible, but completeness can be sacrificed for anything else. It's better to get some stuff done now than wait until everything can get done later. The "get-stuff-done" approach has the same attitude towards correctness and simplicity as "the-right-thing", but the same attitude towards consistency and completeness as "worse-is-better". It also adds performance as a new principle, treating it as the second most important thing (after correctness). https://www.ponylang.io/discover/#what-is-pony ``` Overall the difference between "the-right-thing" and "worse-is-better" can be understood as the difference between upfront and marginal costs. Doing something right the first time is an upfront cost, and once paid decreases marginal costs *forever*. The main problem in software, and the reason "worse-is-better" has been winning in an environment of growth-focused viral capitalism, was that it was basically impossible in practice to actually do something the right way! Since our languages haven't ever supported automatic verification we could only hope to weakly attempt to understand what correct even meant and then actually implement it. This meant the cost to chase the truly right thing was unacceptably uncertain. Magmide promises neither performance nor correctness nor consistency nor completeness, but instead promises the one thing that underlies all of those qualities: knowledge. Complete and total formal knowledge about the program you're writing. Magmide is simply a raw exposure of the basic elements of computing, in both the real sense of actual machine instructions and the ideal sense of formal logic. These basic elements can be combined in whatever way someone desires, even in the "worse-is-better" way! The main contribution of Magmide is that the tradeoffs one makes can be made *and flagged*. Nothing is done without knowledge. If you can prove it you can do it Nested environments! the tradeoffs made while designing the operating system can directly inform the proof obligations and effects of nested environments Possible Ways to Improve Automated Proof Checking checking assertions from the bottom up and in reverse instruction order, keeping track as we go of what assertions we're concerned with and only pulling along propositions with a known transformation path to those assertions. https://dl.acm.org/doi/abs/10.1145/3453483.3454084 https://ocamlpro.github.io/verification_for_dummies/ https://arxiv.org/abs/2110.01098 In most of these heap-enabled lambda calculi "allocation" just assumes an infinite heap and requires an owned points-to connective in order to read. In the real assembly language, you can always read, but doing so when you don't have any idea what the value you're reading just gets you a machine word of unknown shape, something like uninit or poison. How can I think about this in Magmide? Are there ever programs that intentionally read garbage? That's essentially always random input. Probably there's just a non-determinism token-effect you want. My hunch about why my approach is going to prove more robust then continuation-passing-style is because it doesn't seem cps can really directly understand programs as mere data, or would need special primitives to handle it, whereas in my approach it's given, which makes sense since again we're merely directly modeling what the machine actually does. yeah, lambda rust is amazing, but it's very tightly coupled to the way rust is implemented for host operating system programs. I don't think it's flexible enough to handle arbitrary machine/instruction definitions. it also can't handle irreducible control flow graphs, which absolutely could be created either with `goto` programs or by compiler optimizations that we want to be able to formally justify. with incremental verification it probably makes sense to allow possible data races (they don't result in a stuck state) but token flag them the interesting thing is that certain kinds of token problems, such as memory unsafety, data races, overflow, non-termination, actually invalidate the truth of triples! a program doesn't have enough certainty to guarantee *anything* if it isn't basically safe. Someone needs to do for formal verification what rust has been doing for systems programming Think about sections that are irrevocably exiting, such as sequential sections capped by an always exiting instruction either branch or jump always out or falling through to next, and you can prove that such sections and concatenations of them always in a well founded way exit the section and relate that to steps relation, and then all you need for sections that are self recursive is a well founded relation and a proof that for all steps that self recurse that only stay within the section that have a triple making progress then the section will always exit in a well founded way You can probably generalize this to whole programs if the steps relation is just parameterized by the section not the program Metaprogramatically embedded dsl https://www.youtube.com/watch?v=ybrQvs4x0Ps https://arxiv.org/abs/2007.00752 > In this work, we perform a large-scale empirical study to explore how software developers are using Unsafe Rust in real-world Rust libraries and applications. Our results indicate that software engineers use the keyword unsafe in less than 30% of Rust libraries, but more than half cannot be entirely statically checked by the Rust compiler because of Unsafe Rust hidden somewhere in a library's call chain. We conclude that although the use of the keyword unsafe is limited, the propagation of unsafeness offers a challenge to the claim of Rust as a memory-safe language. Furthermore, we recommend changes to the Rust compiler and to the central Rust repository's interface to help Rust software developers be aware of when their Rust code is unsafe. http://www.fstar-lang.org/tutorial/ ``` Lexicographic orderings F* also provides a convenience to enhance the well-founded ordering << to lexicographic combinations of <<. That is, given two lists of terms v₁, ..., vₙ and u₁, ..., uₙ, F* accepts that the following lexicographic ordering: v₁ << u₁ \/ (v₁ == u₁ /\ (v₂ << u₂ \/ (v₂ == u₂ /\ ( ... vₙ << uₙ)))) is also well-founded. In fact, it is possible to prove in F* that this ordering is well-founded, provided << is itself well-founded. Lexicographic ordering are common enough that F* provides special support to make it convenient to use them. In particular, the notation: %[v₁; v₂; ...; vₙ] << %[u₁; u₂; ...; uₙ] is shorthand for: v₁ << u₁ \/ (v₁ == u₁ /\ (v₂ << u₂ \/ (v₂ == u₂ /\ ( ... vₙ << uₙ)))) Let’s have a look at lexicographic orderings at work in proving that the classic ackermann function terminates on all inputs. let rec ackermann (m n:nat) : Tot nat (decreases %[m;n]) = if m=0 then n + 1 else if n = 0 then ackermann (m - 1) 1 else ackermann (m - 1) (ackermann m (n - 1)) The decreases %[m;n] syntax tells F* to use the lexicographic ordering on the pair of arguments m, n as the measure to prove this function terminating. Mutual recursion F* also supports mutual recursion and the same check of proving that a measure of the arguments decreases on each (mutually) recursive call applies. For example, one can write the following code to define a binary tree that stores an integer at each internal node—the keyword and allows defining several types that depend mutually on each other. To increment all the integers in the tree, we can write the mutually recursive functions, again using and to define incr_tree and incr_node to depend mutually on each other. F* is able to prove that these functions terminate, just by using the default measure as usual. type tree = | Terminal : tree | Internal : node -> tree and node = { left : tree; data : int; right : tree } let rec incr_tree (x:tree) : tree = match x with | Terminal -> Terminal | Internal node -> Internal (incr_node node) and incr_node (n:node) : node = { left = incr_tree n.left; data = n.data + 1; right = incr_tree n.right } Note Sometimes, a little trick with lexicographic orderings can help prove mutually recursive functions correct. We include it here as a tip, you can probably skip it on a first read. let rec foo (l:list int) : Tot int (decreases %[l;0]) = match l with | [] -> 0 | x :: xs -> bar xs and bar (l:list int) : Tot int (decreases %[l;1]) = foo l What’s happening here is that when foo l calls bar, the argument xs is legitimately a sub-term of l. However, bar l simply calls back foo l, without decreasing the argument. The reason this terminates, however, is that bar can freely call back foo, since foo will only ever call bar again with a smaller argument. You can convince F* of this by writing the decreases clauses shown, i.e., when bar calls foo, l doesn’t change, but the second component of the lexicographic rdering does decrease, i.e., 0 << 1. ``` ``` bool is_even(unsigned int n) { if (n == 0) return true; else return is_odd(n - 1); } bool is_odd(unsigned int n) { if (n == 0) return false; else return is_even(n - 1); } ``` https://iris-project.org/tutorial-pdfs/iris-lecture-notes.pdf https://gitlab.mpi-sws.org/iris/tutorial-popl21 https://gitlab.mpi-sws.org/iris/iris/-/blob/master/docs/heap_lang.md https://gitlab.mpi-sws.org/iris/iris/blob/master/docs/proof_mode.md I like the idea of having a `by` operator that can be used to justify passing a variable as some type with the accompanying proof script. so for example you could say `return x by crush`, or more complicated things such as `return x by (something; something)`. whatever level of automatic crushing should the system do by default? should there be a cheap crusher that's always used even without a `by`, and `by _` means "use a more expensive crusher"? or does no `by` mean to defer to a proof block? it makes sense to me for no `by` to imply simply deferring (trying to pass something as a type we can quickly verify it can't possibly be is just a type error), whereas `by _` means "use the crusher configured at this scope", and something like file/module/section/function/block level crushers can be configured a small and easy to use operator for embedding the proof language into the computational language would probably go a long way to making Magmide popular and easy to understand. it would probably be nice to have some shorthand for "extending" the proof value of functions and type aliases. something like `fn_name ;theorem` or something that implies adding the assumptions of the thing and the thing itself into the context of the proof, and adds the new proof for further use. look at koka lang what magmide can add is *unannotated* effects. polymorphic effects in a language like koka seem (at first glance) to require annotation, whereas in magmide they are simply implied by the `&` combination of assertions that is inherent to what a type is. a problem with effectual control flow is that we almost never actually *care* about control flow differences. effects in koka seem to me to be too obsessed with "purity" in the pedantic functional programming sense, rather than in the *logical correctness* sense. I don't terribly care if a subfunction causes yield effects or catches internal exceptions, I care about its performance and if it is correct or not. magmide is concerned with *correctness* effects, as in whether a function "poisons" the program with possible divergence or crashes or other issues. if a sub function does *potentially* dangerous things but internally proves them and it doesn't impact performance in a way I need to be aware of, then I don't care. well, it looks like they *largely* understand that. what I don't love though is how obsessed they are with effect handlers, to the extent they have `fun` and `val` variants **that are equivalent to just passing down a closure or value!** I guess it allows the effect giving functions to be used in more contexts than would be possible if they just required a function or value value capabilities seem cool, but in a world where we can verify everything, a global variable is in fact totally acceptable. hmmm here's my main takeaway from koka: I actually think it's pretty cool, but I think it's important to distinguish *control flow* effects from *correctness* effects. they have completely different purposes. in fact I'm hesitant to call what koka has effects at all, they're more like "contextual handlers" or something. maybe it's better just to call what *I'm* adding something else. Honestly it's pretty cool what koka had implemented. But I'm not as excited about it for async, because async code isn't really an effect the more I think about it. Async is a type level manifestation of a completely different mode of execution, in which execution is driven primarily by closures rather than by simple function execution. The program must be completely altered in terms of what data structures it produces and how they are processed Algebraic effects don't save us! Just be because the async effect can theoretically be composed with any other effect type doesn't mean that's actually a good choice. Async is all about recapturing and efficiently using io downtime to do more cpu work. A program simply must be structured differently in order to actually achieve that goal, and designating A function that actually awaits anything has now been effectively colored! It doesn't matter that other effects can exist alongside it, any calling function must either propogate the effect or handle it, which is exactly equivalent to how it works in rust The thing that bothers me about the red blue complaint is that it is just ignoring the reality that async programs have to be structured differently if you want to gain the performance benefits. Async functions merely prod engineers to make the right choices given that constraint They're of course free to do whatever they like, they can just block futures sequentially, or use the underlying blocking primitives, or use a language with transparent async, but they'll pick up the performance downsides in each case. But as they say you can choose to pick up one end of the stick but not the other I'm feeling more and more that other abstractions handle some of these specific cases better, at least from the perspective of how easy they are to reason about For example the fun and val versions of koka effects can be thought of as implicit arguments that can be separately passed in a different tuple of arguments. This is the same as giving a handler, but with stricter requirements about resumption which means we don't have to think about saving the stack. If some implicit arguments default to a global "effectful" function, then a call of that function with that default will itself have that effect Magmide could do algebraic effects but monomorphize all the specific instances, making them zero cost. All of this can be justified using branching instructions Functions could use a global "unsure" function equivalent to panic but that takes a symbol and a message and the default implicit value of this function is merely an instantiation of panic that ignores the symbol. Calling functions can provide something to replace the implicit panic and have it statically monomorphized The term "gradual verification" is useful to sell people on what's unique about this project. Magmide is tractable for the same reasons something like typescript or mypy is tractable. An exciting idea, of having the "language" be generic over a *machine*, which includes possibly none or many register (a bit array of known length) or memory location (also a bit array of known length, which accounts for architecture alignment) banks (a possibly infinite list), and a concrete instruction set. Then we can understand the "language" to just be a set of common tools and rules for describing machines. Some nice things follow from this: - "artifical" machines such as those supported by a runtime of some sort are easily described - machines can have multiple register and memory banks of different sizes, and dependent types could allow us to have different access rules or restrictions or semantics for them each. metaprogramming can "unbundle" these banks into simple names if that makes sense. - it becomes pretty simple to check if a machine is "abstract" or "concrete", by determining if all the sizes of register/memory banks are known or unknown (or possibly the correct thing is finite vs infinite?). with that information we can add alerts or something if the memory allocation function of an abstract machine isn't somehow fallible (in a concrete machine, failure to allocate is actually just a program failure! it has a more concrete meaning of having too much data of a specific kind. this concrete semantic failure in a concrete machine is what "bubbles up" to create an infinite but fallible allocation instruction in an abstract machine) I'm starting to think that what I'm really designing is more a *logic* for typed assembly languages. it's not *quite* like llvm precisely, because to really correctly compile to each individual instruction set, those instruction sets have to be fully specified! it seems I'm more moving toward a general logic with a *toolbox* of abstract instruction semantics, each of which can be tied concretely to actual applications. but the full instruction set of any architecture can be specified in full. it really does point toward having a few different "families" of programs: - embedded programs, in which the exact specifications are known up front - os programs? ones here the instruction set can be known but things like memory sizes aren't? - runtime programs, ones where some existing environment is already provided, often allowing looser assumptions probably what we want is a "general core" of instructions we assume every machine has some equivalent for, which we can build the more "higher level" languages on top of. then to write a "backend" someone would fully specify the instruction set and tie the real instructions to the general core ones, at least if they wanted to be able to support the higher level languages https://www.ralfj.de/blog/2020/12/14/provenance.html john regehr office hours - dependent type proof checker with purely logical `prop` and `set?` types - definition of bits and bit arrays that are given special treatment - definition of representation of logical types by bit arrays - prop of a "machine representable" type. since we can represent props as bit arrays, these can be represented - some syntactic metaprogramming commands, which can take basic syntactic structures like strings or tokens or identifiers and transform them into commands or other instructions - some semantic metaprogramming commands, which can operate on variables or identifiers or whatever to extract compile time information about them such as their type - abstract instructions that are able to operate on bit arrays (for now we take as given that these abstract instructions can be validly encoded as bit arrays with a known size, since llvm will actually do the work of translating them for now. in the future we'll absorb what llvm does by creating a system of concrete "hardware axioms" that represent the instruction set and memory layout etc of a real machine, and a mapping from the abstract instructions to these concrete ones. in the immediate future we'll also need "operating system" axioms, at least until there are operating systems built in bedrock that can simply be linked against) - formalization of instruction behaviors, especially control flow, locations, and allocation, and investigations into the well-foundedness of recursive locations --- Random theorizing about syntax: ``` fn debug value; match known(value).type; struct(fields) => for key in fields; print("#{key}: #{value[key]}") nat(n) => print(n) bool(b) => print(b) ``` --- basically this project will have a few large steps: first we'll define some really basic proof of concept of a theory of known types. this first version will basically just use the "computable terms are a subset of terms, and we only bother to typecheck terms once we've fully reduced them to computable terms". there are a million ways to go about doing this, so we'll just keep it really simple. we'll do this in a "simply typed lambda calculus" so it's easy to reason about. we'd probably want to demonstrate that this pattern can handle literally any meta-programming-like pattern, including: - generics - bounded generics - higher-kinded generics (demonstrate a monad type?) - macros of all kinds probably our definition of preservation and soundness etc would be a little more nuanced. we'd probably also require the assumption that the known functions reduced "correctly", something that would depend on the situation all computable types as simply a bit array with some predicate over that bit array. with this we can define n-ary unions, tuples, structs, the "intersection" type that simply "ands" together predicate of the two types then we can get more interesting by having "pre" typechecks. really what we would be doing there is just trying to allow people authoring higher order "known" functions to prove their functions correct, rather than simply relying on the "this known function will eventually reduce to some terms and *those* terms will be typechecked :shrug:". Basically we want these kinds of authors to have strong typing for their things as well, in a way that goes beyond just typechecking the actual "type value" structs that they happen to be manipulating we can think about it this way: in languages like rust, macros just input/output token streams. from a meta-programming perspective, that's like a program just operating on bytestreams at both ends. we want people to be able to type their known functions just as well as all the *actual* functions. what this can allow us to do is typecheck a program, and know *even before we've reduced certain known functions* that those known functions aren't being used appropriately in their context, and won't reduce to terms that will typecheck. in a language that's formally verified, we can then even potentially do the (very scary) potentially very performance enhancing task of *not actually bothering to typecheck the outputs of these known functions*. if we've verified the pre-conditions of the known function, and we have a proof that the known function will always output terms having some particular type, we can just take that type as a given after we've defined the semantics of types that consist *only* of bit arrays with a predicate, we can start actually defining the language semantics. the big things are n-ary unions and match statements, module paths and the dag, type definition syntax etc. but also the very interesting and potentially massive area of figuring out how we can prove that a loop (or family of co-recursive functions) will always terminate. since this language would have a rich proof system, doing that can actually be tractable and useful from the perspective of programmers. lexicographic ordering of stack arguments ["Proving termination"](http://www.fstar-lang.org/tutorial/). defining and proving correct a type inference algorithm then we have all the cool little ideas: - the "infecting" types of certain operations. we want infecters for code that potentially panics, diverges, accesses out of bounds memory (unsound), accesses uninitialized memory (unsafe?), or leaks any "primitive" resource (we could make this generic by having some kind of predicate that is "optional" but as a result of being optional infects the result type. so someone could write a library that has optional invariants about the caller needing to give back resources or something like that, and you can represent a program that doesn't maintain these invariants, but then your types will get infected. perhaps a more interesting way to do this is simply by understanding that any predicate over a type that *doesn't actually make any assertions about the type value's byte array* is like this?). it's probably also true that if we do this "infecting" correctly, we can notice programs where *it's certain* that some infected type consequence will happen, and we can warn programmers about it. - a "loop" command that's different than the "while" command, in the sense that the program doesn't ask for any proof that a "loop" will always terminate, since it's assumed that it might not. we can still maybe have some finer-grained check that simply asks if a loop construct has any code after it, and if it does there has to be *some* way of breaking out of the loop (other than the process being forcefully terminated, such as by receiving some control signal), or else that code is all dead. - with a tiny language that's so flexible, we can define and reason about a host of ergonomic sugars and optimizations. - all the little syntax things you like, such as the "block string", the different ways of calling and chaining functions, the idea of allowing syntax transforming known functions (or "keywords") and of allowing these kinds of functions to be attached as "members" of types for maximum ergonomics and allowing things like custom "question mark" effectful operators. - in our language we can define "stuckness" in a very different way, because even very bad things like panics or memory unsafe operations aren't *stuck*, they're just *infected*. this means that the entire range of valid machine code can be output by this language. this probably means the reasonable default return type of the `main` function of a program (the one that we will assume if they don't provide their own) should be `() | panic`, so we only assume in the common case that the program might be infected with the panic predicate but not any of the "unsoundness" ones. - "logical" vs normal computable types. logical types would basically only be for logic and verification, and not have any actual output artifacts, which means that all the values inhabiting logical types have to be known at compile time, and we can cheat about how efficient they are to make it more convenient to write proofs about them - wouldn't it be cool to connect proofs about this language to existing verification efforts around llvm? for co-recursive functions: we can create graphs of dependencies between functions, and we can group them together based on how strongly connected they are. for example here we mean that a and b both reference the other (and potentially themselves), so once we enter this group we might never leave (a - b) but if a and b point to some other function c, if c doesn't reference a or b (or any function that references a or b), then we'll never visit that group of a and b ever again, *but c might be co-recursive with some other family of functions*. however it's still useful in this situation to understand that we have in some important way *made progress in the recursion*. it seems that the important idea of a co-recursive family of functions is that from any of the functions you could go through some arbitrary set of steps to reach any of the other functions. if we unbundle both functions and the loops/conditionals into mere basic blocks like in llvm, then it's possible to do this graph analysis over the entire program in the same way. with some interesting new theories about what it means to make progress towards termination *in data* rather than *in control flow*, we can merge the two to understand and check if programs are certainly terminating. we can also unbundle the idea of "making progress in data" to "making progress in predicates", since our types are basically only defined as predicates over bit arrays. after all this, we really start to think about the proof checker, and how the proof aspect of the language interacts with the known functions. the simplest thing to notice is that theorems are just known functions that transform some instantiation of a logical type (so all the values of the logical type are known at compile time) to a different type. the more interesting thing to notice is that the same kind of really slick "tacticals" system that's included in coq can just be *fallible* functions that take props and try to produce proofs of them. this means that the "typecheck" function that the compiler actually uses when compiling code should be exposed to all functions (and therefore of course the known functions), and that it should return some kind of `Result` type. that way tacticals can just call it at will with the proofs they've been constructing, and return successfully if they find something the core typechecking algorithm is happy with. --- read introduction to separation logic the biggest way to make things more convenient for people is to have the *certified decision procedures* described by CPDT in the form of the type checking functions!!! that means that certain macros or subportions of the language that fit into some decidable type system can just have their type checking function proven and provided as the proof object! rather than have many layers of "typed" compilers each emitting the language of the one below it as described in the foundational proof carrying code paper, we simply have *one* very base low level language with arbitrarily powerful metaprogramming and proof abilities! we can create the higher level compilers as embedded constructs in the low level language. we're building *up* instead of *down*. https://www.cs.cmu.edu/afs/cs.cmu.edu/project/fox-19/member/jcr/www15818As2011/cs818A3-11.html (here now: 3.12 More about Annotated Specifications) https://www.cs.cmu.edu/afs/cs.cmu.edu/project/fox-19/member/jcr/www15818As2011/ch3.pdf https://en.wikipedia.org/wiki/Bunched_logic http://www.lsv.fr/~demri/OHearnPym99.pdf https://arxiv.org/pdf/1903.00982.pdf https://aaronweiss.us/pubs/popl19-src-oxide-slides.pdf the real genius of rust is education! people can understand separation logic and formal verification if we teach them well! a basic theory of binary-representable types would also of course be incredibly useful here. it seems that carbon could be specified completely by defining the simple `bit` type, and the basic tuple/record, union, and intersection combinators (it seems that intersection types can/should only be used between named records, and to add arbitrary logical propositions to types? it might make sense to only use intersection (as in `&`) for propositions, and have special `merge` and `concat` etc type transformer known functions to do the other kinds of operations people typically think of as being "intersection". then `&` is simple and well-defined and can be used to put any propositions together? it might also function nicely as the syntactic form for declaring propositions, instead of `must`, so `type Nat = int & >= 0`) logical propositions are so powerful that they could be the entire mode of specifying the base types! booleans are just a `byte` or whatever with props asserting that it can only hold certain values. traits are just props asserting that there exists an implementation in scope satisfying some shape. and of course arbitrary logical stuff can be done, including separation logic/ghost state type things. a reason to include the same kind of constructive inductive propositions is because it provides two ways of attacking a theory of "known" types that allows known functions to produce data structures representing these types is probably the most important first step. it seems you could prove that known types are general enough to provide the language with generics, all kinds of macros, and then dramatically expands the reach of usual static type systems by providing "type functions", which allow arbitrary derivations (you can easily do rust derived traits) and mapping, which allows for the kind of expressivity that typescript mapped and conditional types allows a general truth to remember about the goals of carbon is to remember what really made rust successful. it didn't shy away from complexity, and it didn't water down what people were capable of achieving, but it did find clean abstractions for complex things, and *especially* it did an amazing job **teaching** people how those concepts work. an amazing next generation language is equal parts good language/abstraction design and pedagogy. if you give people a huge amount of power to build incredible things, *and* you do an excellent job teaching them both how to use and why they should use it, then you've got an amazing project on your hands. also very important, and something that the academics have *miserably* failed to do (in addition to their language design and the teaching materials, both of which are usually absolutely dreadful), is building the *tooling* for the language. the tools (think `cargo`) and community infrastructure (think the crates system) are probably *more* important on average for the success of a language community than the language itself. people won't use even the most powerful language if it's an absolute godawful chore to accomplish even the smallest thing with it another thing the academics don't realize and do wrong (especially in coq) is just their conventions for naming things! in Coq basic commands like `Theorem` are both inconveniently capitalized and fully spelled out, but important variable names that could hint to us about the semantic content of a variable are given one letter names! that's completely backwards from a usability standpoint, since commands are something we see constantly, can look up in a single manual, and can have syntax highlighters give us context for; whereas variable names are specific to a project or a function/type/proof. shortening `Theorem` to `th` is perfectly acceptable, and lets us cut down on syntax in a reasonable place so we aren't tempted to do so in unreasonable places. `forall` could/should be shortened to something like `fa` or even a single character like `@`. `@(x: X, y: Y)` could be the "forall tuple", equivalent to `forall (x: X) (y: Y)` ## building a proof checker! https://cstheory.stackexchange.com/questions/5836/how-would-i-go-about-learning-the-underlying-theory-of-the-coq-proof-assistant https://www.irif.fr/~sozeau/research/publications/drafts/Coq_Coq_Correct.pdf https://github.com/coq/coq/tree/master/kernel you should almost certainly do everything you can to understand how coq works at a basic level, and read some of the very earliest papers on proof checkers/assistants to understand their actual machinery. hopefully the very basics are simple, and most of the work is defining theories etc on top. hopefully the footprint of the actual checker is tiny, and it's the standard libraries and proof tactics and such that really create most of the weight theory of known types carbon (and various projects in carbon) (when thinking about the compiler and checking refinement/dependent types, it probably makes sense to use an SMT solver for only the parts that you can't come up with a solid algorithm for, like the basic type checking, or to only fall back on it when some simple naive algorithm fails to either prove or disprove) https://www.cs.princeton.edu/~appel/papers/fpcc.pdf https://www.google.com/books/edition/Program_Logics_for_Certified_Compilers/ABkmAwAAQBAJ?hl=en&gbpv=1&printsec=frontcover https://www3.cs.stonybrook.edu/~bender/newpub/2015-BenderFiGi-SICOMP-topsort.pdf https://hal.inria.fr/hal-01094195/document https://coq.github.io/doc/V8.9.1/refman/language/cic.html https://ncatlab.org/nlab/show/calculus+of+constructions https://link.springer.com/content/pdf/10.1007%2F978-0-387-09680-3_24.pdf ???? https://softwarefoundations.cis.upenn.edu/lf-current/ProofObjects.html has a little portion about type-checking and the trusted base, reassuring "Given a type T, the type Πx : T, B will represent the type of dependent functions which given a term t : T computes a term of type B[t/x] corresponding to proofs of the logical proposition ∀x : T, B. Because types represent logical propositions, the language will contain empty types corresponding to unprovable propositions. Notations. We shall freely use the notation ∀x : A, B instead of Πx : A, B when B represents a proposition." theorems are just *dependently* typed functions! this means there's a nice "warning" when people construct propositions that don't use their universally quantified arguments, the propositions are vacuous or trivial and don't prove anything about the input. a big reason unit tests are actually more annoying and slower in development is the need for fixture data! coming up with either some set of examples, or some fixture dataset, or some model that can generate random data in the right shape is itself a large amount of work that doesn't necessarily complement the actual problem at hand. however proving theorems about your implementation is completely complementary, the proofs lock together with the implementation exactly, and you can prove your whole program correct without ever running it! once someone's skilled with the tool, that workflow is massively efficient, since they never have to leave the "code/typecheck" loop. also, proof automation is actually *much more general and easier* than automation of testing. with testing, you need to be able to generate arbitrarily specific models and have checking functions *that aren't the same as the unit under test*. doing that is a huge duplication of effort. probably ought to learn about effect systems as well an infecting proposition for a blocking operation in an async context is a good idea https://www.cs.cmu.edu/~rwh/papers/dtal/icfp01.pdf http://www.ats-lang.org/ looking at dml and xanadu might be good a very plausible reason that projects like dependently-typed-assembly-language and xanadu and ats haven't worked, is that smart separation logic wasn't there yet, and those languages didn't have powerful enough metaprogramming! in bedrock, the actual *language* can literally just be a powerful dependently typed assembly language along with the arbitrarily powerful meta-programming allowed by known types and some cool "keyword"-like primitives, but then the *programmer facing* language can have structs and functions and all the nice things we're used to but all defined with the meta-programming! the meta-programming is the thing that really allows us to package the hyper-powerful and granular dependent type system into a format that is still usable and can achieve mass adoption. in this way we can kinda call this language "machine scheme/lisp". a mistake everyone's been making when integrating dependant/refinement types into "practical" languages is requiring that only first order logic be used, and therefore that the constraints are *always* automatically solvable. We can still keep those easy forms around just by checking if they're applicable and then using them, but some people need/want more power and we should just give it to them! they'll be on their own to provide proofs, but that's fine! we're really making this tradeoff: would we rather have a bunch of languages that are easy to use but lack a bunch of power that makes us routinely create unsafe programs, or occasionally encounter a problem that's a huge pain in the ass to formalize correctness but we're still capable of doing so? I think we definitely want the second! And we can make abstractions to allow us to work in the first area for a subset of easily-shaped problems but still directly have "escape hatches" to the more powerful layer underneath. a full proof checker in a language gives us the exciting option to always include in our meta languages a direct escape hatch right down into the full language! as a future thing, the whole system can be generic over some set of "hardware axioms" (the memory locations and instructions that are intrinsic to an architecture), along with functions describing how to map the "universal" instructions and operations into the hardware instructions. an "incomplete" mapping could be provided, and compiling programs that included unmapped universal instructions would result in a compiler error this is interesting, he's making a lot of the same arguments I am https://media.ccc.de/v/34c3-9105-coming_soon_machine-checked_mathematical_proofs_in_everyday_software_and_hardware_development https://github.com/mit-plv/bedrock2 http://adam.chlipala.net/frap/frap_book.pdf ================================================ FILE: old/checker.rs ================================================ // use std::collections::{HashMap, HashSet}; // // a Ctx is a map of *mere Idents* (which for now will be strings but later will be intern ids) to CtxItems // // a "Path" isn't a thing held by a Ctx, a Path is the *result of chained accessors* // // when you build a Ctx, you only add the base Idents available at this point // // the Ctx for all the items in a module includes all the other items in the module, the base names of the siblings of the module, and "crate" and "super" // // within a block of statements, the Ctx inherits all the items from the parent scope (either a Module or another block of statements), and iteratively adds things to the Ctx as it goes // #[derive(Debug, Clone, PartialEq)] // struct Module { // name: String, // items: Vec, // } // // TODO at some point this will be some kind of id to an interned string // type Ident = String; // // a simple identifier can refer either to some global item with a path like a type or function (types and functions defined inside a block of statements are similar to this, but don't have a "path" in the strict sense since they aren't accessible from the outside) // // or a mere local variable // #[derive(Debug)] // struct Ctx { // scope: HashMap, // errors: Vec, // } // impl Ctx { // fn from(pairs: [(Ident, CtxItem); N]) -> Ctx { // Ctx { scope: HashMap::from(pairs), errors: Vec::new() } // } // // from_iter // fn checked_insert(&mut self, ident: Ident, ctx_item: CtxItem) { // if let Some(existing_item) = self.scope.insert(ident, ctx_item) { // self.add_error(format!("name {ident} has already been used")); // } // } // fn add_error(&mut self, error: String) { // self.errors.push(error); // } // } // #[derive(Debug)] // enum CtxItem { // Module(Module), // Prop(PropDefinition), // Theorem(TheoremDefinition), // Local(Term), // } // fn type_check_program(modules: Vec) -> CheckResult<()> { // // TODO add all the prelude uses! // // `use crate::std` for example // let crate_module = Module { name: "crate".into(), items: modules.into_iter().map(|m| ModuleItem::Module(m)) }; // // TODO this is where resolutions of all the imported crates would go! // let crate_ctx = Ctx::from([("crate", crate_module), ("super", crate_module)]); // type_check_module(crate_module.clone(), crate_ctx, &crate_module) // } // fn type_check_module(module: Module, parent_ctx: Ctx, crate_module: &Module) -> CheckResult<()> { // if module.name == "crate" || module.name == "super" || module.name == "self" { // errors.push(format!("can't name a module reserved word {module.name}")); // } // // TODO clone from parent? this means errors needs to be a global Arc or something, so every cloned Ctx points to the same thing // let mut ctx = parent_ctx.clone(); // // this first pass does nothing but build the ctx which checks for name collisions // for item in module.items { // name_pass_type_check_module_item(item, &parent_ctx, &mut ctx); // } // let ctx = ctx; // for item in module.items { // main_pass_type_check_module_item(item, &ctx, &crate_module, &module); // } // } // fn name_pass_type_check_module_item(module_item: ModuleItem, parent_ctx: &Ctx, ctx: &mut Ctx) { // match module_item { // Use(use_tree) => { // // when checking a use_tree, it can only refer to what's available *before* all the other items in this module are defined // // but it of course adds things to the ctx // type_check_use_tree_module_level(use_tree, &parent_ctx, &mut ctx); // }, // Module(child_module) => { // ctx.checked_insert(child_module.name, CtxItem::Module(child_module)) // }, // Prop(prop_definition) => { // ctx.checked_insert(prop_definition.name, CtxItem::Prop(prop_definition)); // }, // Theorem(theorem_definition) => { // ctx.checked_insert(theorem_definition.name, CtxItem::Theorem(theorem_definition)); // }, // Log(term) => { // // logging can't effect the Ctx, but it *can* refer to anything in the file so checking must be deferred // }, // } // } // fn main_pass_type_check_module_item(module_item: ModuleItem, ctx: &Ctx, crate_module: &Module, super_module: &Module) { // match module_item { // Use(_) => { /* nothing to do, already checked this */ }, // Module(child_module) => { // let child_ctx = HashMap::from([("crate": crate_module), ("super", super_module)]); // type_check_module(child_module, child_ctx, &crate_module); // }, // Prop(prop_definition) => { // type_check_prop_definition(prop_definition, &ctx) // }, // Theorem(theorem_definition) => { // type_check_theorem_definition(theorem_definition, &ctx) // }, // Log(term) => { // type_check_term(term, &ctx); // }, // } // } // fn type_check_statements(statements: Vec, parent_ctx: &Ctx, crate_module: &Module, super_module: &Module) -> CheckResult<()> { // let mut ctx = parent_ctx.clone(); // for statement in statements { // match statement { // _ => { /* nothing to do for these on this pass */ }, // InnerModuleItem(module_item) => { // name_pass_type_check_module_item(module_item, &parent_ctx, &ctx); // }, // } // } // for statement in statements { // match statement { // Let { name, term } => { // type_check_term(term, &ctx); // // TODO mark this term as invalid? // ctx.checked_insert(name, CtxItem::Local(term)); // }, // Bare(term) => { // // this must be a return // }, // // TODO this is problematic, since this ordering would imply inner module items can refer to lets? // InnerModuleItem(module_item) => { // main_pass_type_check_module_item(module_item, &ctx, &crate_module, &super_module); // }, // } // } // } // fn do_accessors(ctx: &Ctx, mut current_item: CtxItem, accessor_idents: Vec) -> CheckResult<(Option, CtxItem)> { // let mut current_item = current_item; // let mut current_ident = None; // for accessor_ident in accessor_idents { // // TODO handle super and crate // current_item = ctx.checked_access_path(current_item, accessor_ident)?; // current_ident = Some(accessor_ident); // } // (current_ident, current_item) // } // fn type_check_use_tree(use_tree: UseTree, parent_ctx: &Ctx, ctx: &mut Ctx) -> CheckResult<()> { // let (base_ident, rest_idents) = use_tree.path_idents; // let (current_ident, current_item) = do_accessors(&ctx, parent_ctx.checked_get(base_ident)?, rest_idents)?; // let current_ident = current_ident.unwrap_or(base_ident); // match &use_tree.cap { // None => { // ctx.checked_insert(current_ident, current_item); // }, // Some(cap) => match cap { // // TODO in case you want to not have two levels of nesting // // UseTreeCap::Empty => ctx.checked_insert(current_ident, current_item), // UseTreeCap::All => ctx.checked_insert_all(current_item), // UseTreeCap::AsName(as_name) => { // ctx.checked_insert(as_name, current_item); // }, // UseTreeCap::InnerTrees(inner_trees) => { // for inner_tree in inner_trees { // // TODO don't short circuit on each of these, the internal errors are good enough // let _ = type_check_use_tree(inner_tree, &parent_ctx, &mut ctx); // } // }, // }, // } // } // #[derive(Debug, Clone, PartialEq)] // enum TypeBody { // Unit, // // Tuple(Vec), // // Record(Vec), // // Union(Vec), // // AnonymousUnion(Vec) // } // // #[derive(Debug)] // // struct FieldDefinition { // // name: String, // // type: TypeReference, // // } // // #[derive(Debug)] // // struct VariantDefinition { // // name: String, // // type_body: TypeBody, // // } // #[derive(Debug)] // enum Pattern { // // for now only qualified *nominal* patterns are accepted? otherwise these constructor_names would be Option? // Unit { constructor_name: String }, // Compound { constructor_name: String, inner_patterns: Vec, is_record: bool }, // Union(Vec), // } // #[derive(Debug)] // struct NamedPattern { // name: String, // pattern: Option, // } // #[derive(Debug, Clone, PartialEq)] // enum ModuleItem { // Use(UseTree), // Module(Module), // Prop(PropDefinition), // Theorem(TheoremDefinition), // Log(Term), // // Model, // // Procedure, // } // impl ModuleItem { // fn give_name(&self) -> Option<&String> { // match self { // ModuleItem::Use(_) => None, // ModuleItem::Prop(PropDefinition { name, .. }) => Some(name), // ModuleItem::Theorem(TheoremDefinition { name, .. }) => Some(name), // } // } // } // #[derive(Debug, Clone, PartialEq)] // struct UseTree { // path_idents: (String, Vec), // cap: UseTreeCap, // } // #[derive(Debug, Clone, PartialEq)] // enum UseTreeCap { // Empty, // All, // AsName(String), // InnerTrees(Vec), // } // impl UseTree { // fn basic(base_path: &'static str) -> UseTree { // UseTree { base_path: base_path.into(), cap: UseTreeCap::Empty } // } // fn basic_as(base_path: &'static str, as_name: &'static str) -> UseTree { // UseTree { base_path: base_path.into(), cap: UseTreeCap::AsName(as_name.into()) } // } // } // impl UseTreeCap { // fn inners(static_inner_paths: [&'static str; N]) -> UseTreeCap { // let mut inner_paths = vec![]; // for static_inner_path in static_inner_paths { // inner_paths.push(UseTree::basic(static_inner_path)); // } // UseTreeCap::InnerTrees(inner_paths) // } // } // #[derive(Debug, Clone, PartialEq)] // struct PropDefinition { // name: String, // type_body: TypeBody // } // #[derive(Debug, Clone, PartialEq)] // struct TheoremDefinition { // name: String, // // parameters: Vec<(NamedPattern, Option)>, // return_type: Term, // statements: Vec, // } // #[derive(Debug, Clone, PartialEq)] // enum Statement { // Bare(Term), // Let { name: String, term: Term }, // // Return(Term), // InnerModuleItem(ModuleItem), // } // #[derive(Debug, Clone, PartialEq)] // enum Term { // Lone(String), // Chain(String, Vec), // Block { statements: Vec }, // Match { // discriminant: Term, // // discriminant_pattern: Pattern, // return_type: Term, // arms: Vec // }, // Func { parameters: Vec, return_type: Term, statements: Vec }, // } // #[derive(Debug)] // struct MatchArm { // pattern: Pattern, // statements: Vec, // } // #[derive(Debug)] // enum ChainItem { // Access(String), // Call { arguments: Vec }, // // IndexCall { arguments: Vec }, // // TODO yikes? using a complex term to return a function that's called freestanding? // FreeCall { target: Term, arguments: Vec }, // // tapping is only useful for debugging, and should be understood as provably not changing the current type // CatchCall { parameters: Either>, statements: Vec, is_tap: bool }, // ChainedMatch { return_type: Term, arms: Vec }, // } // fn type_check_module_item(item: ModuleItem) { // match item { // ModuleItem::Use(use_tree) => type_check_use_tree(use_tree), // ModuleItem::Prop(PropDefinition { name, type_body }) => { // // TODO check that the name hasn't already been used, or perhaps that's handled by earlier stages? // // maybe instead check if this definition has already been flagged, and skip checking if it has // // check that the definition only refers to things that exist and are valid // match type_body { // Unit => { /* nothing to check! perhaps warn to just use std library's "Trivial" prop though */ }, // // Tuple => , // // Record, // // Union, // } // }, // ModuleItem::Theorem(TheoremDefinition { name, return_type, statements }) => { // // TODO check name isn't already used? // // TODO check function doesn't have infinite recursion // // check that return type matches type implied by statements // match type_check_statements(statements) { // None => { // invalid_items.insert(make_path_absolute(name)); // }, // Some(inferred_type) => { // if !type_assignable(inferred_type, return_type) { // invalid_items.push(item); // errors.push(not_assignable_error(inferred_type, return_type)); // } // }, // } // }, // ModuleItem::Log(term) => { // // TODO make sure this can actually be performed but otherwise do nothing to the context // }, // } // } // fn type_check_statements(statements: Vec) -> TypeReference { // // TODO have to build this from existing module_ctx // let mut local_ctx = HashMap::new(); // let mut statements = statements.into_iter().peekable(); // while let Some(statement) = statements.next() { // match statement { // Statement::Let { name, term } => { // let inferred_type = type_check_term(term); // let existing_item = local_ctx.insert(name, inferred_type); // if existing_item.is_some() { // errors.push(format!("variable {name} is already defined")); // } // }, // Statement::InnerModuleItem(module_item) => { // // TODO add this module item to the running local_ctx // }, // // this is proof checker, which means there's no such thing as mutation or effects, // // which means leaving a term bare can only mean this should be the resolved value of this line of statements // Statement::Bare(term) => { // if statements.peek().is_some() { // errors.push(format!("unreachable code")); // } // return Some(type_check_term(term, local_ctx)); // }, // // TODO return is a control flow concept that could still be interesting and useful in an immutable language, since a `let` could have a block or match or if or "functional for" (a function that is being called with a block) where return captures control flow // // this means a return can effect the inferred type of a line of statements *above* this one // // "control flow" in this context is *actually* just desugaring to a version of the function where things like a match have been moved up a level // // let a = match something { one => return 1, two => do_something_else() }; a + 2 // same as // // match something { one => 1, two => let a = do_something_else(); a + 2 } // // Term::Return(term) // } // } // errors.push(format!("statements never resolve to a value, which doesn't make sense in a proof checker")); // None // } // fn type_check_term(term: Term, ctx: &Ctx) -> Term { // match term { // Term::Ident(ident) => { // // TODO why am I afraid this isn't correct or will recurse infinitely? // ctx.infer_term_type(ident) // }, // Term::Block { statements } => { // type_check_statements(statements) // }, // Term::Match { discriminant, return_type, arms } => { // let discriminant_type = unimplemented!(); // for arm in arms { // check_pattern_compatible(discriminant_type, arm.pattern); // // all the magic is hiding in check_assignable, which has to do reduction and things in complex cases // check_assignable(type_check_statements(arm.statements), return_type) // } // }, // Term::Chain(first, rest) => { // let mut chain_ctx = ctx.clone(); // let mut current_type = type_check_chain_root(first, &mut chain_ctx)?; // for chain_item in rest { // current_type = type_check_chain_item(chain_item, current_type, &mut chain_ctx)?; // } // current_type // }, // Term::Func { parameters, return_type, statements } => { // type_check_named_patterns(parameters, local_ctx); // check_assignable(type_check_statements(statements), return_type) // }, // } // } // fn type_check_named_patterns(named_patterns: Vec, pattern_names: &mut HashSet) { // for named_pattern in named_patterns { // pattern_names.insert(named_pattern.name)?.get_mad_if_exists(); // if let Some(pattern) = named_pattern.pattern { // type_check_pattern(pattern, pattern_names); // } // } // } // fn type_check_pattern(pattern: Pattern, pattern_names: &mut HashSet) { // match expr { // Pattern::Unit { constructor_name } => { // // TODO look up this constructor_name and see if it exists and is compatible with being a type // }, // Pattern::Compound { constructor_name, inner_patterns, is_record } => { // // TODO check constructor_name exists and matches with is_record // type_check_named_patterns(inner_patterns, pattern_names); // }, // Pattern::Union(patterns) => { // for pattern in patterns { // type_check_pattern(pattern, pattern_names); // } // }, // } // } // fn type_check_chain_root(chain_root: ChainRoot, chain_ctx: &mut HashMap) -> Term { // match chain_root { // ChainRoot::Path(path) => { // // TODO look up this path in the context // }, // ChainRoot::Call { path, arguments } => { // // TODO check the path exists, is callable, and its parameters match the arguments // }, // } // } // fn type_check_chain_item(chain_item: ChainItem, current_type: Term, chain_ctx: &mut HashMap) -> Term { // match chain_item { // ChainItem::FreeCall { path, arguments, is_bare } => { // if let Some(_) = current_type && is_bare { // errors.push(format!("used a bare call in the middle of a chain")); // return // } // }, // ChainItem::Access(accessor) => { // // TODO check this accessor exists on this type, give type of accessor // }, // ChainItem::AccessCall { accessor, arguments } => { // // TODO check the accessor exists, is callable, and its parameters match the arguments // }, // ChainItem::CatchCall { parameters, statements, is_tap } => { // match parameters { // Either::Left(parameter) => { // check_assignable(current_type, parameter) // // return if fail? // }, // Either::Right(parameters) => { // // TODO type check current_type is a spreadable thing that matches the parameters // }, // } // // TODO enrich the ctx with the parameters // let inferred_type = type_check_statements(statements); // // a tapping call is only good for debugging, and doesn't effect the type // if is_tap { current_type } else { inferred_type } // }, // } // } // fn type_check_use_tree(use_tree: UseTree) { // let UseTree { base_path, cap } = use_tree; // // check that base_path exists // if !module_ctx.has(base_path) { // errors.push(format!("{base_path} doesn't exist")); // } // match cap { // UseTreeCap::Empty => { /* nothing to check if final segment name has already been checked for validity */ } // UseTreeCap::All => { // // TODO check base_path refers to something with importable members // }, // UseTreeCap::AsName(as_name) => { // // TODO check that as_name hasn't already been used, or perhaps that's handled by earlier stages? // }, // UseTreeCap::InnerTrees(inner_trees) => { // for inner_tree in inner_trees { // type_check_use_tree(inner_tree); // } // }, // } // } // fn check_assignable(observed_type: Term, desired_type: Term) -> Term { // if !type_assignable(observed_type, desired_type) { // errors.push(format!("{observed_type} not assignable to {desired_type}")); // } // observed_type // } // // TODO this is a proof checker, which means types are just terms // // this is where we need to do canonicalization and reduction and check for equivalance // fn type_assignable(observed_type: Term, desired_type: Term) -> bool { // unimplemented!() // } // #[cfg(test)] // mod tests { // use super::*; // fn make_path(path: Path) -> Term { // Term::Lone(ChainRoot::Path("true")) // } // fn make_call(path: Path, arguments: Vec) -> Term { // Term::Lone(ChainRoot::Call { path: "invert", arguments: vec![make_true()] }) // } // fn make_trivial_prop() -> ModuleItem { // ModuleItem::Prop(PropDefinition { // name: "trivial".into(), // type_body: TypeBody::Unit, // }) // } // fn make_give_trivial_thm() -> ModuleItem { // ModuleItem::Theorem(TheoremDefinition { // name: "give_trivial".into(), // return_type: Term::Ident("trivial".into()), // statements: vec![ // Statement::Return(Term::Ident("trivial".into())), // ], // }) // } // #[test] // fn test_reduce_term(arg: Type) -> RetType { // // type bool = true | false // // use bool::* // // proc invert(b: bool): bool; match b; true; false, false; true // let program = Program { modules: vec![ // Module { name: "main".into(), items: vec![make_bool_type(), make_invert_proc()], child_modules: vec![] }, // ] }; // let term = make_call("invert", vec![make_path("true")]); // assert_eq!(reduce_term(term, local_ctx), make_path("crate::main::bool::false")); // let term = make_call("invert", vec![make_path("b")]); // // TODO enrich local_ctx with b: bool, so its some opaque thing // // match b; true; false, false; true // assert_eq!(reduce_term(term, local_ctx), make_match("b", vec![("true", "false"), ("false", "true")])); // // prop trivial // // thm trivial_proven: trivial = trivial // let term = make_path("trivial_proven"); // assert_eq!(reduce_term(term, local_ctx), make_path("crate::main::trivial")); // } // #[test] // fn test_type_check_trivial() { // let program = Program { modules: vec![ // Module { name: "main".into(), items: vec![make_trivial_prop(), make_give_trivial_thm()], child_modules: vec![] }, // ] }; // let mut errors = vec![]; // type_check_program(program, &mut errors); // assert_eq!(errors, vec![]); // } // #[test] // fn test_build_program_path_index() { // let trivial_prop = make_trivial_prop(); // let give_trivial_thm = make_give_trivial_thm(); // let program_path_index = build_program_path_index(Program { modules: vec![ // Module { name: "main".into(), items: vec![trivial_prop.clone(), give_trivial_thm.clone()], child_modules: vec![] }, // ] }); // let expected = HashMap::from([ // ("crate::main::trivial".into(), trivial_prop.clone()), // ("crate::main::give_trivial".into(), give_trivial_thm.clone()), // ]); // assert_eq!(program_path_index, expected); // let program_path_index = build_program_path_index(Program { modules: vec![ // Module { name: "main".into(), items: vec![trivial_prop.clone(), give_trivial_thm.clone()], child_modules: vec![ // Module { name: "main_child".into(), items: vec![give_trivial_thm.clone()], child_modules: vec![] }, // ] }, // Module { name: "side".into(), items: vec![trivial_prop.clone(), give_trivial_thm.clone()], child_modules: vec![ // Module { name: "side_child".into(), items: vec![give_trivial_thm.clone()], child_modules: vec![] }, // ] }, // ] }); // let expected = HashMap::from([ // ("crate::main::trivial".into(), trivial_prop.clone()), // ("crate::main::give_trivial".into(), give_trivial_thm.clone()), // ("crate::main::main_child::give_trivial".into(), give_trivial_thm.clone()), // ("crate::side::trivial".into(), trivial_prop.clone()), // ("crate::side::give_trivial".into(), give_trivial_thm.clone()), // ("crate::side::side_child::give_trivial".into(), give_trivial_thm.clone()), // ]); // assert_eq!(program_path_index, expected); // } // #[test] // fn test_build_module_ctx() { // let module_path = "crate"; // let side_use = ModuleItem::Use(UseTree { // // TODO "bare" references like this are assumed to be "relative", so at the same level as the current module // // TODO you could also do super and root // base_path: "side".into(), // cap: UseTreeCap::inners(["whatever", "other"]), // }); // let module = Module { name: "main".into(), items: vec![side_use, make_trivial_prop(), make_give_trivial_thm()], child_modules: vec![] }; // let expected = HashMap::from([ // ("trivial".into(), "crate::main::trivial".into()), // ("give_trivial".into(), "crate::main::give_trivial".into()), // ("whatever".into(), "crate::side::whatever".into()), // ("other".into(), "crate::side::other".into()), // ]); // assert_eq!(build_module_ctx(module_path, &module), expected); // let side_use = ModuleItem::Use(UseTree { // base_path: "crate::side::child".into(), // cap: UseTreeCap::InnerTrees(vec![ // UseTree::basic("whatever"), // UseTree::basic_as("other", "different"), // UseTree { base_path: "nested::thing".into(), cap: UseTreeCap::inners(["self", "a", "b"]) }, // ]), // }); // let module = Module { name: "main".into(), items: vec![side_use, make_trivial_prop(), make_give_trivial_thm()], child_modules: vec![] }; // let expected = HashMap::from([ // ("trivial".into(), "crate::main::trivial".into()), // ("give_trivial".into(), "crate::main::give_trivial".into()), // ("whatever".into(), "crate::side::child::whatever".into()), // ("different".into(), "crate::side::child::other".into()), // ("thing".into(), "crate::side::child::nested::thing".into()), // ("a".into(), "crate::side::child::nested::thing::a".into()), // ("b".into(), "crate::side::child::nested::thing::b".into()), // ]); // assert_eq!(build_module_ctx(module_path, &module), expected); // } // #[test] // fn test_make_path_absolute() { // for (current_absolute_path, possibly_relative_path, expected) in [ // ("crate", "crate::m", "crate::m"), // ("crate", "m", "crate::m"), // ("crate", "m::child", "crate::m::child"), // ("crate", "crate::m::child", "crate::m::child"), // ("crate::a::b::c", "crate::m", "crate::m"), // ("crate::a::b::c", "crate::m::child", "crate::m::child"), // ("crate::a::b::c", "m", "crate::a::b::c::m"), // ("crate::a::b::c", "m::child", "crate::a::b::c::m::child"), // ] { // assert_eq!(make_path_absolute(current_absolute_path, possibly_relative_path), expected); // } // } // #[test] // fn test_resolve_reference() { // let program_path_index = HashMap::from([ // ("crate::main::trivial".into(), make_trivial_prop()), // ("crate::main::give_trivial".into(), make_give_trivial_thm()), // ]); // let ctx = HashMap::from([]); // let current_path = "crate::main::"; // assert_eq!( // resolve_reference(ctx, current_path, "trivial"), // "crate::main::trivial" // ); // let ctx = HashMap::from([ // ("main"), // ]); // let current_path = "crate::side::"; // } // // #[test] // // fn test_resolve_term_type() { // // // in a scope with nothing "included" from higher scopes, identifiers resolve to the name of this scope // // assert_eq!( // // resolve_term_type("MyType", "some_module", {}), // // // if we refer to MyType while in some_module, and there aren't any references to that name, it must be local // // vec!["some_module", "MyType"], // // ); // // assert_eq!( // // resolve_term_type("MyType", "some_module", { "MyType": "other_module", "WhateverType": "yo_module" }), // // // if we refer to it while in some_module but something like a `use` introduced that name from another place, it's that place // // vec!["other_module", "MyType"], // // ); // // } // // #[test] // // fn trivial_type_and_fn() { // // // prop trivial // // // thm give_trivial: trivial; // // // return trivial // // let program = vec![ // // make_trivial_prop(), // // make_give_trivial_thm(), // // ]; // // assert!(type_check_program(program).is_some()); // // // assert the whole program type checks // // // assert querying give_trivial returns trivial, resolved // // } // // #[test] // // fn things() { // // // model bool; true | false // // // prop trival // // // prop impossible; | // // // model Thing; a: A, b: B, other: Z // // // @(P, Q); prop And(P, Q) // // // @(P, Q) // // // prop And; (P, Q) // // // @(P, Q); // // // prop And; (P, Q) // // // prop And; (@P, @Q) // // let and_type = TypeDefinition { // // name: "And".into(), // // kind: Kind::Prop, // // generics: vec![ // // Pattern{name: "P", type: None}, // // Pattern{name: "Q", type: None}, // // ], // // definition: TypeBody::Tuple(vec![bare("P"), bare("Q")]), // // }; // // } // } // // prop Or; Left(@P) | Right(@Q) // // Or.Left // // Or/Left // // using slash as the "namespace" separator gives a nice similarity to modules and the filesystem // // that might be a bad thing! although Rust also blends the two by using :: for both // // honestly I think just `.` is the best, it's just the "access namespace" operator // // accessing the namespace of a struct is similar to accessing the namespace of a module or a type // // // anonymous unions // // alias A = 'a' | 'A' | int // // fn do_a = |: :match; // // 'a' | 'A'; :do_something() // // int; :do_int_thing() // // `|` is for creating a function, `&` is for creating an assertion // // `|>` creates a function and catches, `|:` creates a function and chains // // TODO blaine what about return type annotation for anonymous functions? // // `&(n; , , )` creates an assertion and catches, `&( , , )` creates an assertion and chains (chains is the default) // // `&(n; )` creates an assertion and catches, `&` creates an assertion and chains (chains is the default) // // examples of the "forall" type // // @(a: A) -> Z // // @(a: A, b: B, inferred, d: D) -> Z // // examples of the "simple function" type // // (A) -> Z // // (A, B) -> Z // // there's no such thing as "terms" that only apply specifically for each of these, // // since there's *only* simple functions in the actual concrete language. // // "theorems" are functions which return things of kind Prop // // "algorithms" are functions which return things of kind Model, which may or may not have prop assertions on them // // "functions" are concrete and computational, have completely different rules // // TODO in general all of this path stuff should just reuse rustc functions, but I need something to play with for now // fn make_path_absolute(current_absolute_path: &str, possibly_relative_path: &str) -> String { // if possibly_relative_path.starts_with("crate") { // possibly_relative_path.to_owned() // } // // // TODO need to handle multiple "super" using a while loop? // // else if possibly_relative_path.startswith("super") { // // let trimmed_current_absolute_path = current_absolute_path.split("::").skip_last().unwrap(); // // // TODO can't just trim as many times as you want, have to count how many // // let reduced_relative_path = possibly_relative_path.trim_start_matches("super::"); // // let reduced_relative_path = reduced_relative_path.trim_start_matches("super"); // // format!("{trimmed_current_absolute_path}::{reduced_relative_path}", ) // // } // else { // format!("{current_absolute_path}::{possibly_relative_path}") // } // } // // fn make_dir_module(dirname: String, modules: Vec) -> Module { // // Module { name: dirname, items: modules.into_iter.map(|m| ModuleItem::Module(m)) } // // } // // TODO make a function that walks a directory and recursively calls make_dir_module ================================================ FILE: old/inductive_serde.v ================================================ (* it seems possible to define a function that given an AST representing an inductive type is able to produce a pair of functions that can serialize/deserialize values of that inductive type to/from binary arrays the cases that are especially interesting are: - any flat union (which should just be a single tag representing which variant the value is) - any type with exactly one unit variant and exactly one variant with only one recursive argument (which should basically have a tag/number at the beginning representing how many wrapped recursive constructors there are, and the rest of the array is any non-recursive wrapped data values) (the cases I have in mind here are natural numbers and lists) *) Add LoadPath "/home/blaine/lab/cpdtlib" as Cpdt. Set Implicit Arguments. Set Asymmetric Patterns. Require Import List String Cpdt.CpdtTactics. Import ListNotations. From stdpp Require Import base options stringmap. (*Definition yo: stringmap nat := {[ "hey" := 1; "dude" := 2 ]}. Example test_yo: (yo !!! "hey") = 1. Proof. reflexivity. Qed. Example test_yo': (yo !! "hey") = Some 1. Proof. reflexivity. Qed. Example test_yo'': (yo !!! "nope") = 0. Proof. reflexivity. Qed.*) Inductive bit: Type := b0 | b1. Notation bit_array := (list bit). Inductive TypeReference: Type := self_ref | other_ref (name: string). Inductive ConstructorNode := constructor_node { constructor_args: list TypeReference }. (* TODO polymorphic args *) Inductive InductiveType := inductive_node { constructors: stringmap ConstructorNode }. Inductive InductiveValue := inductive_value { value_type: string; value_constructor: string; value_args: list InductiveValue }. Notation InductiveContext := (stringmap InductiveType). (* do we have to modify/reduce ctx as we go down? *) Fixpoint ValueOfType (ctx: InductiveContext) (value: InductiveValue) (type: InductiveType) {struct value} : Prop := ctx !! value.(value_type) = Some type /\ exists constructor, type.(constructors) !! value.(value_constructor) = Some constructor /\ ValuesOfTypes ctx value.(value_args) constructor.(constructor_args) with ValuesOfTypes (ctx: InductiveContext) (values: list InductiveValue) (type_refs: list TypeReference) {struct values} : Prop := match values, type_refs with | [], [] => True | value :: values', type_ref :: type_refs' => let remainder_well_typed := (ValuesOfTypes ctx values' type_refs') in match type_ref with | self_ref => remainder_well_typed | other_ref other_name => exists other_type, ctx !! other_name = Some other_type /\ ValueOfType ctx value other_type /\ remainder_well_typed end | _, _ => False end . Inductive Result (T: Type) (E: Type): Type := | Ok (value: T) | Err (error: E). Arguments Ok {T} {E} _. Arguments Err {T} {E} _. Notation serializer T := (T -> bit_array). Notation deserializer T := (bit_array -> Result T string). Fixpoint produce_serde_functions (node: InductiveType) : Result (serializer, deserializer) string := . ================================================ FILE: old/machine.md ================================================ ``` map_disjoint_sym: ∀ (K: Type) (M: Type → Type) (H0: ∀ A: Type, Lookup K A (M A)) (A: Type), Symmetric (map_disjoint: relation (M A)) map_disjoint_weaken_l: ∀ (K: Type) (M: Type → Type) (H0: ∀ A: Type, Lookup K A (M A)) (A: Type) (m1 m1' m2: M A), m1' ##ₘ m2 → m1 ⊆ m1' → m1 ##ₘ m2 map_disjoint_weaken_r: ∀ (K: Type) (M: Type → Type) (H0: ∀ A: Type, Lookup K A (M A)) (A: Type) (m1 m2 m2': M A), m1 ##ₘ m2' → m2 ⊆ m2' → m1 ##ₘ m2 map_disjoint_weaken: ∀ (K: Type) (M: Type → Type) (H0: ∀ A: Type, Lookup K A (M A)) (A: Type) (m1 m1' m2 m2': M A), m1' ##ₘ m2' → m1 ⊆ m1' → m2 ⊆ m2' → m1 ##ₘ m2 map_disjoint_Some_r: ∀ (K: Type) (M: Type → Type) (H0: ∀ A: Type, Lookup K A (M A)) (A: Type) (m1 m2: M A) (i: K) (x: A), m1 ##ₘ m2 → m2 !! i = Some x → m1 !! i = None map_disjoint_Some_l: ∀ (K: Type) (M: Type → Type) (H0: ∀ A: Type, Lookup K A (M A)) (A: Type) (m1 m2: M A) (i: K) (x: A), m1 ##ₘ m2 → m1 !! i = Some x → m2 !! i = None map_disjoint_proper: ∀ (K: Type) (M: Type → Type) (H0: ∀ A: Type, Lookup K A (M A)) (A: Type) (H7: Equiv A), Proper (equiv ==> equiv ==> iff) map_disjoint map_disjoint_alt: ∀ (K: Type) (M: Type → Type) (H0: ∀ A: Type, Lookup K A (M A)) (A: Type) (m1 m2: M A), m1 ##ₘ m2 ↔ (∀ i: K, m1 !! i = None ∨ m2 !! i = None) map_disjoint_spec: ∀ (K: Type) (M: Type → Type) (H0: ∀ A: Type, Lookup K A (M A)) (A: Type) (m1 m2: M A), m1 ##ₘ m2 ↔ (∀ (i: K) (x y: A), m1 !! i = Some x → m2 !! i = Some y → False) map_disjoint_empty_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m: M A), m ##ₘ ∅ map_disjoint_empty_l: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m: M A), ∅ ##ₘ m map_disjoint_delete_l: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A) (i: K), m1 ##ₘ m2 → delete i m1 ##ₘ m2 map_disjoint_delete_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A) (i: K), m1 ##ₘ m2 → m1 ##ₘ delete i m2 map_union_comm: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A), m1 ##ₘ m2 → m1 ∪ m2 = m2 ∪ m1 map_disjoint_difference_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A), m1 ⊆ m2 → m1 ##ₘ m2 ∖ m1 map_disjoint_difference_l: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A), m1 ⊆ m2 → m2 ∖ m1 ##ₘ m1 map_union_subseteq_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A), m1 ##ₘ m2 → m2 ⊆ m1 ∪ m2 map_disjoint_union_l_2: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2 m3: M A), m1 ##ₘ m3 → m2 ##ₘ m3 → m1 ∪ m2 ##ₘ m3 map_disjoint_union_r_2: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2 m3: M A), m1 ##ₘ m2 → m1 ##ₘ m3 → m1 ##ₘ m2 ∪ m3 map_disjoint_fmap: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A B: Type) (f1 f2: A → B) (m1 m2: M A), m1 ##ₘ m2 ↔ f1 <$> m1 ##ₘ f2 <$> m2 map_disjoint_omap: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A B: Type) (f1 f2: A → option B) (m1 m2: M A), m1 ##ₘ m2 → omap f1 m1 ##ₘ omap f2 m2 map_disjoint_union_list_l_2: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (ms: list (M A)) (m: M A), Forall (λ m2: M A, m2 ##ₘ m) ms → ⋃ ms ##ₘ m map_disjoint_union_list_r_2: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (ms: list (M A)) (m: M A), Forall (λ m2: M A, m2 ##ₘ m) ms → m ##ₘ ⋃ ms map_disjoint_union_l: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2 m3: M A), m1 ∪ m2 ##ₘ m3 ↔ m1 ##ₘ m3 ∧ m2 ##ₘ m3 map_disjoint_union_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2 m3: M A), m1 ##ₘ m2 ∪ m3 ↔ m1 ##ₘ m2 ∧ m1 ##ₘ m3 map_disjoint_foldr_delete_l: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A) (is: list K), m1 ##ₘ m2 → foldr delete m1 is ##ₘ m2 map_disjoint_foldr_delete_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A) (is: list K), m1 ##ₘ m2 → m1 ##ₘ foldr delete m2 is map_disjoint_union_list_l: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (ms: list (M A)) (m: M A), ⋃ ms ##ₘ m ↔ Forall (λ m2: M A, m2 ##ₘ m) ms map_disjoint_union_list_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (ms: list (M A)) (m: M A), m ##ₘ ⋃ ms ↔ Forall (λ m2: M A, m2 ##ₘ m) ms map_Forall_union_1_2: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A) (P: K → A → Prop), m1 ##ₘ m2 → map_Forall P (m1 ∪ m2) → map_Forall P m2 map_union_subseteq_r': ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2 m3: M A), m2 ##ₘ m3 → m1 ⊆ m3 → m1 ⊆ m2 ∪ m3 map_disjoint_singleton_l_2: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m: M A) (i: K) (x: A), m !! i = None → {[i := x]} ##ₘ m map_disjoint_singleton_r_2: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m: M A) (i: K) (x: A), m !! i = None → m ##ₘ {[i := x]} map_union_cancel_l: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2 m3: M A), m1 ##ₘ m3 → m2 ##ₘ m3 → m3 ∪ m1 = m3 ∪ m2 → m1 = m2 map_union_cancel_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2 m3: M A), m1 ##ₘ m3 → m2 ##ₘ m3 → m1 ∪ m3 = m2 ∪ m3 → m1 = m2 map_disjoint_singleton_l: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m: M A) (i: K) (x: A), {[i := x]} ##ₘ m ↔ m !! i = None map_disjoint_singleton_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m: M A) (i: K) (x: A), m ##ₘ {[i := x]} ↔ m !! i = None map_seq_app_disjoint: ∀ (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup nat A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter nat A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList nat A (M A)) (EqDecision0: EqDecision nat), FinMap nat M → ∀ (A: Type) (start: nat) (xs1 xs2: list A), map_seq start xs1 ##ₘ map_seq (start + length xs1) xs2 map_union_mono_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2 m3: M A), m2 ##ₘ m3 → m1 ⊆ m2 → m1 ∪ m3 ⊆ m2 ∪ m3 map_disjoint_insert_l_2: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A) (i: K) (x: A), m2 !! i = None → m1 ##ₘ m2 → <[i:=x]> m1 ##ₘ m2 map_disjoint_insert_r_2: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A) (i: K) (x: A), m1 !! i = None → m1 ##ₘ m2 → m1 ##ₘ <[i:=x]> m2 map_omap_union: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A B: Type) (f: A → option B) (m1 m2: M A), m1 ##ₘ m2 → omap f (m1 ∪ m2) = omap f m1 ∪ omap f m2 map_Forall_union: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A) (P: K → A → Prop), m1 ##ₘ m2 → map_Forall P (m1 ∪ m2) ↔ map_Forall P m1 ∧ map_Forall P m2 lookup_union_Some_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A) (i: K) (x: A), m1 ##ₘ m2 → m2 !! i = Some x → (m1 ∪ m2) !! i = Some x map_union_reflecting_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2 m3: M A), m1 ##ₘ m3 → m2 ##ₘ m3 → m1 ∪ m3 ⊆ m2 ∪ m3 → m1 ⊆ m2 map_union_reflecting_l: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2 m3: M A), m3 ##ₘ m1 → m3 ##ₘ m2 → m3 ∪ m1 ⊆ m3 ∪ m2 → m1 ⊆ m2 map_disjoint_insert_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A) (i: K) (x: A), m1 ##ₘ <[i:=x]> m2 ↔ m1 !! i = None ∧ m1 ##ₘ m2 map_disjoint_insert_l: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A) (i: K) (x: A), <[i:=x]> m1 ##ₘ m2 ↔ m2 !! i = None ∧ m1 ##ₘ m2 map_size_disj_union: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A), m1 ##ₘ m2 → base.size (m1 ∪ m2) = base.size m1 + base.size m2 map_not_disjoint: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A), ¬ m1 ##ₘ m2 ↔ (∃ (i: K) (x1 x2: A), m1 !! i = Some x1 ∧ m2 !! i = Some x2) map_disjoint_list_to_map_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m: M A) (ixs: list (K * A)), m ##ₘ list_to_map ixs ↔ Forall (λ ix: K * A, m !! ix.1 = None) ixs map_disjoint_list_to_map_l: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m: M A) (ixs: list (K * A)), list_to_map ixs ##ₘ m ↔ Forall (λ ix: K * A, m !! ix.1 = None) ixs map_disjoint_dom_2: ∀ (K: Type) (M: Type → Type) (D: Type) (H: ∀ A: Type, Dom (M A) D) (H0: FMap M) (H1: ∀ A: Type, Lookup K A (M A)) (H2: ∀ A: Type, Empty (M A)) (H3: ∀ A: Type, PartialAlter K A (M A)) (H4: OMap M) (H5: Merge M) (H6: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K) (H7: ElemOf K D) (H8: Empty D) (H9: Singleton K D) (H10: Union D) (H11: Intersection D) (H12: Difference D), FinMapDom K M D → ∀ (A: Type) (m1 m2: M A), dom D m1 ## dom D m2 → m1 ##ₘ m2 map_disjoint_dom_1: ∀ (K: Type) (M: Type → Type) (D: Type) (H: ∀ A: Type, Dom (M A) D) (H0: FMap M) (H1: ∀ A: Type, Lookup K A (M A)) (H2: ∀ A: Type, Empty (M A)) (H3: ∀ A: Type, PartialAlter K A (M A)) (H4: OMap M) (H5: Merge M) (H6: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K) (H7: ElemOf K D) (H8: Empty D) (H9: Singleton K D) (H10: Union D) (H11: Intersection D) (H12: Difference D), FinMapDom K M D → ∀ (A: Type) (m1 m2: M A), m1 ##ₘ m2 → dom D m1 ## dom D m2 lookup_union_Some: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m1 m2: M A) (i: K) (x: A), m1 ##ₘ m2 → (m1 ∪ m2) !! i = Some x ↔ m1 !! i = Some x ∨ m2 !! i = Some x map_disjoint_dom: ∀ (K: Type) (M: Type → Type) (D: Type) (H: ∀ A: Type, Dom (M A) D) (H0: FMap M) (H1: ∀ A: Type, Lookup K A (M A)) (H2: ∀ A: Type, Empty (M A)) (H3: ∀ A: Type, PartialAlter K A (M A)) (H4: OMap M) (H5: Merge M) (H6: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K) (H7: ElemOf K D) (H8: Empty D) (H9: Singleton K D) (H10: Union D) (H11: Intersection D) (H12: Difference D), FinMapDom K M D → ∀ (A: Type) (m1 m2: M A), m1 ##ₘ m2 ↔ dom D m1 ## dom D m2 map_disjoint_filter: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (P: K * A → Prop) (H7: ∀ x: K * A, Decision (P x)) (m1 m2: M A), m1 ##ₘ m2 → filter P m1 ##ₘ filter P m2 map_disjoint_list_to_map_zip_l_2: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m: M A) (is: list K) (xs: list A), length is = length xs → Forall (λ i: K, m !! i = None) is → list_to_map (zip is xs) ##ₘ m map_disjoint_list_to_map_zip_r_2: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m: M A) (is: list K) (xs: list A), length is = length xs → Forall (λ i: K, m !! i = None) is → m ##ₘ list_to_map (zip is xs) map_disjoint_list_to_map_zip_l: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m: M A) (is: list K) (xs: list A), length is = length xs → list_to_map (zip is xs) ##ₘ m ↔ Forall (λ i: K, m !! i = None) is map_disjoint_list_to_map_zip_r: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (m: M A) (is: list K) (xs: list A), length is = length xs → m ##ₘ list_to_map (zip is xs) ↔ Forall (λ i: K, m !! i = None) is map_disjoint_filter_complement: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (P: K * A → Prop) (H7: ∀ x: K * A, Decision (P x)) (m: M A), filter P m ##ₘ filter (λ v: K * A, ¬ P v) m map_disjoint_kmap: ∀ (K1: Type) (M1: Type → Type) (H: FMap M1) (H0: ∀ A: Type, Lookup K1 A (M1 A)) (H1: ∀ A: Type, Empty (M1 A)) (H2: ∀ A: Type, PartialAlter K1 A (M1 A)) (H3: OMap M1) (H4: Merge M1) (H5: ∀ A: Type, FinMapToList K1 A (M1 A)) (EqDecision0: EqDecision K1), FinMap K1 M1 → ∀ (K2: Type) (M2: Type → Type) (H7: FMap M2) (H8: ∀ A: Type, Lookup K2 A (M2 A)) (H9: ∀ A: Type, Empty (M2 A)) (H10: ∀ A: Type, PartialAlter K2 A (M2 A)) (H11: OMap M2) (H12: Merge M2) (H13: ∀ A: Type, FinMapToList K2 A (M2 A)) (EqDecision1: EqDecision K2), FinMap K2 M2 → ∀ f: K1 → K2, Inj eq eq f → ∀ (A: Type) (m1 m2: M1 A), kmap f m1 ##ₘ kmap f m2 ↔ m1 ##ₘ m2 map_filter_union: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (P: K * A → Prop) (H7: ∀ x: K * A, Decision (P x)) (m1 m2: M A), m1 ##ₘ m2 → filter P (m1 ∪ m2) = filter P m1 ∪ filter P m2 dom_union_inv_L: ∀ (K: Type) (M: Type → Type) (D: Type) (H: ∀ A: Type, Dom (M A) D) (H0: FMap M) (H1: ∀ A: Type, Lookup K A (M A)) (H2: ∀ A: Type, Empty (M A)) (H3: ∀ A: Type, PartialAlter K A (M A)) (H4: OMap M) (H5: Merge M) (H6: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K) (H7: ElemOf K D) (H8: Empty D) (H9: Singleton K D) (H10: Union D) (H11: Intersection D) (H12: Difference D), FinMapDom K M D → LeibnizEquiv D → RelDecision elem_of → ∀ (A: Type) (m: M A) (X1 X2: D), X1 ## X2 → dom D m = X1 ∪ X2 → ∃ m1 m2: M A, m = m1 ∪ m2 ∧ m1 ##ₘ m2 ∧ dom D m1 = X1 ∧ dom D m2 = X2 map_cross_split: ∀ (K: Type) (M: Type → Type) (H: FMap M) (H0: ∀ A: Type, Lookup K A (M A)) (H1: ∀ A: Type, Empty (M A)) (H2: ∀ A: Type, PartialAlter K A (M A)) (H3: OMap M) (H4: Merge M) (H5: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K), FinMap K M → ∀ (A: Type) (ma mb mc md: M A), ma ##ₘ mb → mc ##ₘ md → ma ∪ mb = mc ∪ md → ∃ mac mad mbc mbd: M A, mac ##ₘ mad ∧ mbc ##ₘ mbd ∧ mac ##ₘ mbc ∧ mad ##ₘ mbd ∧ mac ∪ mad = ma ∧ mbc ∪ mbd = mb ∧ mac ∪ mbc = mc ∧ mad ∪ mbd = md dom_union_inv: ∀ (K: Type) (M: Type → Type) (D: Type) (H: ∀ A: Type, Dom (M A) D) (H0: FMap M) (H1: ∀ A: Type, Lookup K A (M A)) (H2: ∀ A: Type, Empty (M A)) (H3: ∀ A: Type, PartialAlter K A (M A)) (H4: OMap M) (H5: Merge M) (H6: ∀ A: Type, FinMapToList K A (M A)) (EqDecision0: EqDecision K) (H7: ElemOf K D) (H8: Empty D) (H9: Singleton K D) (H10: Union D) (H11: Intersection D) (H12: Difference D), FinMapDom K M D → RelDecision elem_of → ∀ (A: Type) (m: M A) (X1 X2: D), X1 ## X2 → dom D m ≡ X1 ∪ X2 → ∃ m1 m2: M A, m = m1 ∪ m2 ∧ m1 ##ₘ m2 ∧ dom D m1 ≡ X1 ∧ dom D m2 ≡ X2 H0: registers s1 ##ₘ registers s2 ∪ registers s3 map_union_assoc: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ A : Type, Assoc eq union map_union_idemp: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ A : Type, IdemP eq union H: program_counter s1 ##ₘ program_counter s2 ∪ program_counter s3 map_union_empty: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ A : Type, RightId eq ∅ union map_empty_union: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ A : Type, LeftId eq ∅ union map_union_subseteq_l: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A), m1 ⊆ m1 ∪ m2 map_subseteq_union: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A), m1 ⊆ m2 → m1 ∪ m2 = m2 map_union_comm: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A), m1 ##ₘ m2 → m1 ∪ m2 = m2 ∪ m1 map_union_subseteq_r: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A), m1 ##ₘ m2 → m2 ⊆ m1 ∪ m2 map_positive_l: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A), m1 ∪ m2 = ∅ → m1 = ∅ map_disjoint_union_l_2: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m1 ##ₘ m3 → m2 ##ₘ m3 → m1 ∪ m2 ##ₘ m3 map_disjoint_union_r_2: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m1 ##ₘ m2 → m1 ##ₘ m3 → m1 ##ₘ m2 ∪ m3 map_Forall_union_1_1: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (P : K → A → Prop), map_Forall P (m1 ∪ m2) → map_Forall P m1 map_union_subseteq_l': ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m1 ⊆ m2 → m1 ⊆ m2 ∪ m3 map_positive_l_alt: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A), m1 ≠ ∅ → m1 ∪ m2 ≠ ∅ map_disjoint_union_list_l_2: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (ms : list (M A)) (m : M A), Forall (λ m2 : M A, m2 ##ₘ m) ms → ⋃ ms ##ₘ m map_disjoint_union_list_r_2: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (ms : list (M A)) (m : M A), Forall (λ m2 : M A, m2 ##ₘ m) ms → m ##ₘ ⋃ ms map_disjoint_union_r: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m1 ##ₘ m2 ∪ m3 ↔ m1 ##ₘ m2 ∧ m1 ##ₘ m3 map_disjoint_union_l: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m1 ∪ m2 ##ₘ m3 ↔ m1 ##ₘ m3 ∧ m2 ##ₘ m3 map_disjoint_union_list_l: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (ms : list (M A)) (m : M A), ⋃ ms ##ₘ m ↔ Forall (λ m2 : M A, m2 ##ₘ m) ms map_disjoint_union_list_r: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (ms : list (M A)) (m : M A), m ##ₘ ⋃ ms ↔ Forall (λ m2 : M A, m2 ##ₘ m) ms map_difference_union: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A), m1 ⊆ m2 → m1 ∪ m2 ∖ m1 = m2 map_Forall_union_1_2: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (P : K → A → Prop), m1 ##ₘ m2 → map_Forall P (m1 ∪ m2) → map_Forall P m2 map_Forall_union_2: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (P : K → A → Prop), map_Forall P m1 → map_Forall P m2 → map_Forall P (m1 ∪ m2) map_union_mono_l: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m1 ⊆ m2 → m3 ∪ m1 ⊆ m3 ∪ m2 map_fmap_union: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A B : Type) (f : A → B) (m1 m2 : M A), f <$> m1 ∪ m2 = (f <$> m1) ∪ (f <$> m2) map_union_subseteq_r': ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m2 ##ₘ m3 → m1 ⊆ m3 → m1 ⊆ m2 ∪ m3 map_union_least: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m1 ⊆ m3 → m2 ⊆ m3 → m1 ∪ m2 ⊆ m3 map_union_cancel_l: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m1 ##ₘ m3 → m2 ##ₘ m3 → m3 ∪ m1 = m3 ∪ m2 → m1 = m2 map_union_cancel_r: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m1 ##ₘ m3 → m2 ##ₘ m3 → m1 ∪ m3 = m2 ∪ m3 → m1 = m2 lookup_union_l: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K), is_Some (m1 !! i) → (m1 ∪ m2) !! i = m1 !! i lookup_union_Some_l: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K) (x : A), m1 !! i = Some x → (m1 ∪ m2) !! i = Some x insert_union_singleton_l: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m : M A) (i : K) (x : A), <[i:=x]> m = {[i := x]} ∪ m map_union_mono_r: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m2 ##ₘ m3 → m1 ⊆ m2 → m1 ∪ m3 ⊆ m2 ∪ m3 lookup_union_r: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K), m1 !! i = None → (m1 ∪ m2) !! i = m2 !! i lookup_union_is_Some: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K), is_Some ((m1 ∪ m2) !! i) ↔ is_Some (m1 !! i) ∨ is_Some (m2 !! i) map_omap_union: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A B : Type) (f : A → option B) (m1 m2 : M A), m1 ##ₘ m2 → omap f (m1 ∪ m2) = omap f m1 ∪ omap f m2 insert_union_l: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K) (x : A), <[i:=x]> (m1 ∪ m2) = <[i:=x]> m1 ∪ m2 union_singleton_r: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m : M A) (i : K) (x y : A), m !! i = Some x → m ∪ {[i := y]} = m map_Forall_union: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (P : K → A → Prop), m1 ##ₘ m2 → map_Forall P (m1 ∪ m2) ↔ map_Forall P m1 ∧ map_Forall P m2 lookup_union_Some_r: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K) (x : A), m1 ##ₘ m2 → m2 !! i = Some x → (m1 ∪ m2) !! i = Some x map_union_reflecting_r: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m1 ##ₘ m3 → m2 ##ₘ m3 → m1 ∪ m3 ⊆ m2 ∪ m3 → m1 ⊆ m2 map_union_reflecting_l: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 m3 : M A), m3 ##ₘ m1 → m3 ##ₘ m2 → m3 ∪ m1 ⊆ m3 ∪ m2 → m1 ⊆ m2 lookup_union: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K), (m1 ∪ m2) !! i = union_with (λ x _ : A, Some x) (m1 !! i) (m2 !! i) delete_union: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K), delete i (m1 ∪ m2) = delete i m1 ∪ delete i m2 map_size_disj_union: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A), m1 ##ₘ m2 → base.size (m1 ∪ m2) = base.size m1 + base.size m2 union_proper: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (H7 : Equiv A), Proper (equiv ==> equiv ==> equiv) union lookup_union_Some_inv_r: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K) (x : A), (m1 ∪ m2) !! i = Some x → m1 !! i = None → m2 !! i = Some x lookup_union_Some_inv_l: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K) (x : A), (m1 ∪ m2) !! i = Some x → m2 !! i = None → m1 !! i = Some x lookup_union_None: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K), (m1 ∪ m2) !! i = None ↔ m1 !! i = None ∧ m2 !! i = None insert_union_singleton_r: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m : M A) (i : K) (x : A), m !! i = None → <[i:=x]> m = m ∪ {[i := x]} list_to_map_app: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (l1 l2 : list (K * A)), list_to_map (l1 ++ l2) = list_to_map l1 ∪ list_to_map l2 insert_union_r: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K) (x : A), m1 !! i = None → <[i:=x]> (m1 ∪ m2) = m1 ∪ <[i:=x]> m2 foldr_insert_union: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m : M A) (l : list (K * A)), foldr (λ p : K * A, <[p.1:=p.2]>) m l = list_to_map l ∪ m map_seq_app: ∀ (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup nat A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter nat A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList nat A (M A)) (EqDecision0 : EqDecision nat), FinMap nat M → ∀ (A : Type) (start : nat) (xs1 xs2 : list A), map_seq start (xs1 ++ xs2) = map_seq start xs1 ∪ map_seq (start + length xs1) xs2 lookup_union_Some: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K) (x : A), m1 ##ₘ m2 → (m1 ∪ m2) !! i = Some x ↔ m1 !! i = Some x ∨ m2 !! i = Some x union_delete_insert: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K) (x : A), m1 !! i = Some x → delete i m1 ∪ <[i:=x]> m2 = m1 ∪ m2 foldr_delete_union: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (is : list K), foldr delete (m1 ∪ m2) is = foldr delete m1 is ∪ foldr delete m2 is lookup_union_Some_raw: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K) (x : A), (m1 ∪ m2) !! i = Some x ↔ m1 !! i = Some x ∨ m1 !! i = None ∧ m2 !! i = Some x dom_union: ∀ (K : Type) (M : Type → Type) (D : Type) (H : ∀ A : Type, Dom (M A) D) (H0 : FMap M) (H1 : ∀ A : Type, Lookup K A (M A)) (H2 : ∀ A : Type, Empty (M A)) (H3 : ∀ A : Type, PartialAlter K A (M A)) (H4 : OMap M) (H5 : Merge M) (H6 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K) (H7 : ElemOf K D) (H8 : Empty D) (H9 : Singleton K D) (H10 : Union D) (H11 : Intersection D) (H12 : Difference D), FinMapDom K M D → ∀ (A : Type) (m1 m2 : M A), dom D (m1 ∪ m2) ≡ dom D m1 ∪ dom D m2 dom_union_L: ∀ (K : Type) (M : Type → Type) (D : Type) (H : ∀ A : Type, Dom (M A) D) (H0 : FMap M) (H1 : ∀ A : Type, Lookup K A (M A)) (H2 : ∀ A : Type, Empty (M A)) (H3 : ∀ A : Type, PartialAlter K A (M A)) (H4 : OMap M) (H5 : Merge M) (H6 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K) (H7 : ElemOf K D) (H8 : Empty D) (H9 : Singleton K D) (H10 : Union D) (H11 : Intersection D) (H12 : Difference D), FinMapDom K M D → LeibnizEquiv D → ∀ (A : Type) (m1 m2 : M A), dom D (m1 ∪ m2) = dom D m1 ∪ dom D m2 map_union_equiv_eq: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (H7 : Equiv A), Equivalence equiv → ∀ m1 m2a m2b : M A, m1 ≡ m2a ∪ m2b ↔ (∃ m2a' m2b' : M A, m1 = m2a' ∪ m2b' ∧ m2a' ≡ m2a ∧ m2b' ≡ m2b) union_insert_delete: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A) (i : K) (x : A), m1 !! i = None → m2 !! i = Some x → <[i:=x]> m1 ∪ delete i m2 = m1 ∪ m2 map_filter_union_complement: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (P : K * A → Prop) (H7 : ∀ x : K * A, Decision (P x)) (m : M A), filter P m ∪ filter (λ v : K * A, ¬ P v) m = m set_unfold_dom_union: ∀ (K : Type) (M : Type → Type) (D : Type) (H : ∀ A : Type, Dom (M A) D) (H0 : FMap M) (H1 : ∀ A : Type, Lookup K A (M A)) (H2 : ∀ A : Type, Empty (M A)) (H3 : ∀ A : Type, PartialAlter K A (M A)) (H4 : OMap M) (H5 : Merge M) (H6 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K) (H7 : ElemOf K D) (H8 : Empty D) (H9 : Singleton K D) (H10 : Union D) (H11 : Intersection D) (H12 : Difference D), FinMapDom K M D → ∀ (A : Type) (i : K) (m1 m2 : M A) (Q1 Q2 : Prop), SetUnfoldElemOf i (dom D m1) Q1 → SetUnfoldElemOf i (dom D m2) Q2 → SetUnfoldElemOf i (dom D (m1 ∪ m2)) (Q1 ∨ Q2) map_filter_union: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (P : K * A → Prop) (H7 : ∀ x : K * A, Decision (P x)) (m1 m2 : M A), m1 ##ₘ m2 → filter P (m1 ∪ m2) = filter P m1 ∪ filter P m2 kmap_union: ∀ (K1 : Type) (M1 : Type → Type) (H : FMap M1) (H0 : ∀ A : Type, Lookup K1 A (M1 A)) (H1 : ∀ A : Type, Empty (M1 A)) (H2 : ∀ A : Type, PartialAlter K1 A (M1 A)) (H3 : OMap M1) (H4 : Merge M1) (H5 : ∀ A : Type, FinMapToList K1 A (M1 A)) (EqDecision0 : EqDecision K1), FinMap K1 M1 → ∀ (K2 : Type) (M2 : Type → Type) (H7 : FMap M2) (H8 : ∀ A : Type, Lookup K2 A (M2 A)) (H9 : ∀ A : Type, Empty (M2 A)) (H10 : ∀ A : Type, PartialAlter K2 A (M2 A)) (H11 : OMap M2) (H12 : Merge M2) (H13 : ∀ A : Type, FinMapToList K2 A (M2 A)) (EqDecision1 : EqDecision K2), FinMap K2 M2 → ∀ f : K1 → K2, Inj eq eq f → ∀ (A : Type) (m1 m2 : M1 A), kmap f (m1 ∪ m2) = kmap f m1 ∪ kmap f m2 dom_union_inv_L: ∀ (K : Type) (M : Type → Type) (D : Type) (H : ∀ A : Type, Dom (M A) D) (H0 : FMap M) (H1 : ∀ A : Type, Lookup K A (M A)) (H2 : ∀ A : Type, Empty (M A)) (H3 : ∀ A : Type, PartialAlter K A (M A)) (H4 : OMap M) (H5 : Merge M) (H6 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K) (H7 : ElemOf K D) (H8 : Empty D) (H9 : Singleton K D) (H10 : Union D) (H11 : Intersection D) (H12 : Difference D), FinMapDom K M D → LeibnizEquiv D → RelDecision elem_of → ∀ (A : Type) (m : M A) (X1 X2 : D), X1 ## X2 → dom D m = X1 ∪ X2 → ∃ m1 m2 : M A, m = m1 ∪ m2 ∧ m1 ##ₘ m2 ∧ dom D m1 = X1 ∧ dom D m2 = X2 map_cross_split: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (ma mb mc md : M A), ma ##ₘ mb → mc ##ₘ md → ma ∪ mb = mc ∪ md → ∃ mac mad mbc mbd : M A, mac ##ₘ mad ∧ mbc ##ₘ mbd ∧ mac ##ₘ mbc ∧ mad ##ₘ mbd ∧ mac ∪ mad = ma ∧ mbc ∪ mbd = mb ∧ mac ∪ mbc = mc ∧ mad ∪ mbd = md dom_union_inv: ∀ (K : Type) (M : Type → Type) (D : Type) (H : ∀ A : Type, Dom (M A) D) (H0 : FMap M) (H1 : ∀ A : Type, Lookup K A (M A)) (H2 : ∀ A : Type, Empty (M A)) (H3 : ∀ A : Type, PartialAlter K A (M A)) (H4 : OMap M) (H5 : Merge M) (H6 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K) (H7 : ElemOf K D) (H8 : Empty D) (H9 : Singleton K D) (H10 : Union D) (H11 : Intersection D) (H12 : Difference D), FinMapDom K M D → RelDecision elem_of → ∀ (A : Type) (m : M A) (X1 X2 : D), X1 ## X2 → dom D m ≡ X1 ∪ X2 → ∃ m1 m2 : M A, m = m1 ∪ m2 ∧ m1 ##ₘ m2 ∧ dom D m1 ≡ X1 ∧ dom D m2 ≡ X2 map_intersection_filter: ∀ (K : Type) (M : Type → Type) (H : FMap M) (H0 : ∀ A : Type, Lookup K A (M A)) (H1 : ∀ A : Type, Empty (M A)) (H2 : ∀ A : Type, PartialAlter K A (M A)) (H3 : OMap M) (H4 : Merge M) (H5 : ∀ A : Type, FinMapToList K A (M A)) (EqDecision0 : EqDecision K), FinMap K M → ∀ (A : Type) (m1 m2 : M A), m1 ∩ m2 = filter (λ kx : K * A, is_Some (m1 !! kx.1) ∧ is_Some (m2 !! kx.1)) (m1 ∪ m2) ``` ================================================ FILE: old/machine.v ================================================ Add LoadPath "/home/blaine/lab/cpdtlib" as Cpdt. Set Implicit Arguments. Set Asymmetric Patterns. (*Require Import List String Cpdt.CpdtTactics Coq.Program.Wf.*) From Coq Require Import Morphisms RelationClasses Setoid. Require Import Cpdt.CpdtTactics. From stdpp Require Import base options fin gmap. (*Import ListNotations.*) Require Import theorems.utils. (*Inductive Bit: Type := B0 | B1.*) (*Notation BitWord word_size := (vec Bit word_size).*) (*https://coq.inria.fr/library/Coq.Bool.Bvector.html*) (*https://github.com/coq-community/bits*) (*https://github.com/mit-plv/bbv*) (*https://github.com/jasmin-lang/coqword*) (*https://coq.inria.fr/library/Coq.PArith.BinPosDef.html*) (*Notation MemoryBank word_size := (gmap (BitWord word_size) (BitWord word_size)).*) (*Notation MemoryBank size := (gmap (fin size) nat).*) (*Definition bank: RegisterBank 3 := {[ 2%fin := 1 ]}. Example test_bank: (bank !!! 2%fin) = 1. Proof. reflexivity. Qed. Definition empty_bank: RegisterBank 3 := empty. Example test_empty: (empty_bank !!! 2%fin) = 0. Proof. reflexivity. Qed.*) (*Notation RegisterBank word_size register_count := (gmap (fin register_count) (BitWord word_size)).*) Notation RegisterBank size := (gmap (fin size) nat). (*From stdpp Require Import natmap. Definition test__natmap_lookup_m: natmap nat := {[ 3 := 2; 0 := 2 ]}. Example test__natmap_lookup: test__natmap_lookup_m !! 3 = Some 2. Proof. reflexivity. Qed. Example test__vec_total_lookup: ([# 4; 2] !!! 0%fin) = 4. Proof. reflexivity. Qed.*) Module AbstractMachine. Parameter size: nat. Record MachineState := machine_state { program_counter: RegisterBank 1; registers: RegisterBank size }. (*Notation pc state := (state.(program_counter) !!! 0).*) Global Instance state_empty: Empty MachineState := machine_state empty empty. Global Instance state_union: Union MachineState := fun s1 s2 => (machine_state (union s1.(program_counter) s2.(program_counter)) (union s1.(registers) s2.(registers)) ). Theorem state_equality c1 c2 r1 r2: c1 = c2 /\ r1 = r2 <-> (machine_state c1 r1) = (machine_state c2 r2). Proof. naive_solver. Qed. Definition state_disjoint s1 s2 := map_disjoint s1.(program_counter) s2.(program_counter) /\ map_disjoint s1.(registers) s2.(registers). Global Instance state_disjoint_symmetric: Symmetric state_disjoint. Proof. intros ??; unfold state_disjoint; rewrite !map_disjoint_spec; naive_solver. Qed. Theorem state_union_commutative s1 s2: state_disjoint s1 s2 -> union s1 s2 = union s2 s1. Proof. intros [C R]; unfold union, state_union; rewrite (map_union_comm _ _ C); rewrite (map_union_comm _ _ R); reflexivity. Qed. Theorem state_disjoint_union_distributive s1 s2 s3: state_disjoint s1 (union s2 s3) <-> state_disjoint s1 s2 /\ state_disjoint s1 s3. Proof. split; unfold state_disjoint. - intros [?%map_disjoint_union_r ?%map_disjoint_union_r]; naive_solver. - intros [[] []]; split; apply map_disjoint_union_r_2; assumption. Qed. Theorem state_union_associative (s1 s2 s3: MachineState): union s1 (union s2 s3) = union (union s1 s2) s3. Proof. unfold union, state_union; simpl; apply state_equality; apply map_union_assoc. Qed. Theorem state_union_empty_l: forall state: MachineState, union empty state = state. Proof. unfold union, state_union; intros []; simpl; do 2 rewrite map_empty_union; reflexivity. Qed. Theorem state_union_empty_r: forall state: MachineState, union state empty = state. Proof. unfold union, state_union; intros []; simpl; do 2 rewrite map_union_empty; reflexivity. Qed. Theorem state_separate_counter_registers_disjoint: forall registers program_counter, state_disjoint (machine_state empty registers) (machine_state program_counter empty). Proof. intros; hnf; simpl; auto with map_disjoint. Qed. Theorem state_empty_disjoint: forall state, state_disjoint empty state. Proof. intros; hnf; simpl; auto with map_disjoint. Qed. Global Hint Extern 0 (state_disjoint _ _) => (split; assumption): core. Notation Assertion := (MachineState -> Prop) (only parsing). Declare Scope assertion_scope. Open Scope assertion_scope. Definition state_implies (H1 H2: Assertion): Prop := forall state, H1 state -> H2 state. Notation "H1 **> H2" := (state_implies H1 H2) (at level 55). Definition state_equivalent (H1 H2: Assertion): Prop := forall state, H1 state <-> H2 state. Notation "H1 <*> H2" := (state_equivalent H1 H2) (at level 60). Theorem state_implies_reflexive: forall H, H **> H. Proof. intros; hnf; trivial. Qed. Hint Resolve state_implies_reflexive: core. Theorem state_implies_transitive: forall H2 H1 H3, (H1 **> H2) -> (H2 **> H3) -> (H1 **> H3). Proof. intros ??? M1 M2 state H1state; auto. Qed. Hint Resolve state_implies_transitive: core. Theorem state_implies_antisymmetric: forall H1 H2, (H1 **> H2) -> (H2 **> H1) -> H1 <*> H2. Proof. intros ?? M1 M2; split; auto. Qed. Hint Resolve state_implies_antisymmetric: core. Theorem state_implies_transitive_r: forall H2 H1 H3, (H2 **> H3) -> (H1 **> H2) -> (H1 **> H3). Proof. intros ??? M1 M2; eapply state_implies_transitive; eauto. Qed. Hint Resolve state_implies_transitive_r: core. Theorem state_operation_commutative: forall (operation: Assertion -> Assertion -> Assertion), (forall H1 H2, operation H1 H2 **> operation H2 H1) -> (forall H1 H2, operation H1 H2 <*> operation H2 H1). Proof. intros; apply state_implies_antisymmetric; auto. Qed. Global Instance state_equivalent_reflexive: Reflexive state_equivalent. Proof. constructor; apply state_implies_reflexive. Qed. Global Instance state_equivalent_symmetric: Symmetric state_equivalent. Proof. hnf; intros ??[]; split; assumption. Qed. Global Instance state_equivalent_transitive: Transitive state_equivalent. Proof. hnf; intros ???[][]; split; eapply state_implies_transitive; eauto. Qed. Global Instance state_equivalence: Equivalence state_equivalent. Proof. constructor; auto using state_equivalent_reflexive, state_equivalent_symmetric,state_equivalent_transitive. Qed. Global Instance subrelation_state_equivalent_implies: subrelation state_equivalent state_implies. Proof. intros ??[]; assumption. Qed. Global Instance Proper_state_implies: Proper (state_equivalent ==> state_equivalent ==> flip impl) state_implies. Proof. intros ??[]??[]; hnf; eauto. Qed. Definition state_unknown: Assertion := fun state => state = empty. Notation "\[]" := (state_unknown) (at level 0): assertion_scope. Definition state_register (register: fin size) (value: nat): Assertion := fun state => state = machine_state empty (singletonM register value). Notation "'$' register '==' value" := (state_register register%fin value) (at level 32): assertion_scope. Definition state_counter (value: nat): Assertion := fun state => state = machine_state (singletonM 0%fin value) empty. Notation "'pc_at' value" := (state_counter value) (at level 32): assertion_scope. Definition state_star (H1 H2: Assertion): Assertion := fun state => exists s1 s2, H1 s1 /\ H2 s2 /\ state_disjoint s1 s2 /\ state = union s1 s2. Notation "H1 '\*' H2" := (state_star H1 H2) (at level 41, right associativity): assertion_scope. Definition state_exists (A: Type) (P: A -> Assertion): Assertion := fun state => exists a, P a state. Notation "'\exists' a1 .. an , H" := (state_exists (fun a1 => .. (state_exists (fun an => H)) ..)) ( at level 39, a1 binder, H at level 50, right associativity, format "'[' '\exists' '/ ' a1 .. an , '/ ' H ']'" ): assertion_scope. Definition state_forall (A: Type) (P: A -> Assertion): Assertion := fun state => forall a, P a state. Notation "'\forall' a1 .. an , H" := (state_forall (fun a1 => .. (state_forall (fun an => H)) ..)) ( at level 39, a1 binder, H at level 50, right associativity, format "'[' '\forall' '/ ' a1 .. an , '/ ' H ']'" ): assertion_scope. Definition state_pure (P: Prop): Assertion := \exists (p:P), \[]. Notation "\[ P ]" := (state_pure P) (at level 0, format "\[ P ]"): assertion_scope. Definition state_wand (H1 H2: Assertion): Assertion := \exists H0, H0 \* state_pure ((H1 \* H0) **> H2). Notation "H1 \-* H2" := (state_wand H1 H2) (at level 43, right associativity): assertion_scope. (* ### state_unknown *) Theorem state_unknown_intro: \[] empty. Proof. hnf; trivial. Qed. Theorem state_unknown_inversion: forall state, \[] state -> state = empty. Proof. intros; hnf; auto. Qed. (* ### state_star *) Theorem state_star_intro: forall (H1 H2: Assertion) s1 s2, H1 s1 -> H2 s2 -> state_disjoint s1 s2 -> (H1 \* H2) (union s1 s2). Proof. intros; exists s1, s2; auto. Qed. Theorem state_star_inversion: forall H1 H2 state, (H1 \* H2) state -> exists s1 s2, H1 s1 /\ H2 s2 /\ state_disjoint s1 s2 /\ state = union s1 s2. Proof. intros ??? A; hnf in A; eauto. Qed. Theorem state_star_commutative: forall H1 H2, H1 \* H2 <*> H2 \* H1. Proof. apply state_operation_commutative; unfold state_star; intros ?? state (s1 & s2 & ? & ? & [] & U); rewrite state_union_commutative in U; trivial; exists s2, s1; repeat split; auto. Qed. Theorem state_star_associative H1 H2 H3: (H1 \* H2) \* H3 <*> H1 \* (H2 \* H3). Proof. apply state_implies_antisymmetric. - intros state (state' & h3 & (h1 & h2 & ?&?&?&?)&?& D%symmetry &?); subst state'; exists h1, (union h2 h3); rewrite state_union_associative; assert (D' := D); apply state_disjoint_union_distributive in D' as [?%symmetry ?%symmetry]; repeat split; repeat apply state_star_intro; trivial; apply state_disjoint_union_distributive; split; trivial. - intros state (h1 & state' &?&(h2 & h3 &?&?&?&?)&D&?); subst state'; exists (union h1 h2), h3; rewrite <-state_union_associative; assert (D' := D); apply state_disjoint_union_distributive in D' as []; repeat split; repeat apply state_star_intro; trivial; symmetry; apply state_disjoint_union_distributive; split; symmetry; trivial. Qed. Theorem state_star_empty_l H: \[] \* H <*> H. Proof. apply state_implies_antisymmetric; hnf. - intros ? [? (? & ?%state_unknown_inversion & ? & ? & ?)]; subst; rewrite state_union_empty_l; assumption. - intros ?; exists empty, state; repeat split; simpl; try apply state_unknown_intro; try apply map_disjoint_empty_l; try rewrite state_union_empty_l; trivial. Qed. Theorem state_star_empty_r H: H \* \[] <*> H. Proof. rewrite state_star_commutative; apply state_star_empty_l. Qed. Theorem state_star_exists A (P: A -> Assertion) H: (\exists a, P a) \* H <*> \exists a, (P a \* H). Proof. apply state_implies_antisymmetric; intros state. - intros (s1 & s2 & (a &?)&?&?&?); exists a, s1, s2; auto. - intros (a & (s1 & s2 &?&?&?&?)); exists s1, s2; split; auto; exists a; trivial. Qed. Theorem state_implies_frame_l H2 H1 H1': H1 **> H1' -> (H1 \* H2) **> (H1' \* H2). Proof. intros ?? (s1 & s2 & (?&?&?&?)); exists s1, s2; auto. Qed. Theorem state_implies_frame_r H1 H2 H2': H2 **> H2' -> (H1 \* H2) **> (H1 \* H2'). Proof. intros ?? (s1 & s2 & (?&?&?&?)); exists s1, s2; auto. Qed. (*Theorem state_implies_frame_r' H1 H2 H2': H2 **> H2' -> (H1 \* H2) **> (H1 \* H2'). Proof. intros ?; do 2 rewrite (state_star_commutative H1); apply state_implies_frame_l; assumption. Qed.*) Theorem state_implies_frame H1 H1' H2 H2': H1 **> H1' -> H2 **> H2' -> (H1 \* H2) **> (H1' \* H2'). Proof. intros ??? (s1 & s2 & (?&?&?&?)); exists s1, s2; auto. Qed. Theorem state_implies_star_trans_l H1 H2 H3 H4: H1 **> H2 -> H2 \* H3 **> H4 -> H1 \* H3 **> H4. Proof. intros M1 M2 ? (s1 & s2 & (?&?&?&?)); apply M2; exists s1, s2; auto. Qed. Theorem state_implies_star_trans_r H1 H2 H3 H4: H1 **> H2 -> H3 \* H2 **> H4 -> H3 \* H1 **> H4. Proof. intros M1 M2 ? (s1 & s2 & (?&?&?&?)); apply M2; exists s1, s2; auto. Qed. (* ### state_pure *) Theorem state_pure_intro: forall P: Prop, P -> \[P] empty. Proof. intros ? P; exists P; hnf; auto. Qed. Theorem state_pure_inversion: forall P state, \[P] state -> P /\ state = empty. Proof. intros ?? A; hnf in A; naive_solver. Qed. Theorem state_star_pure_l P H state: (\[P] \* H) state <-> P /\ (H state). Proof. unfold state_pure, state_exists; split. - hnf. rewrite state_star_exists. - rewrite state_star_exists. rewrite* state_star_empty_l. iff (p&M) (p&M). { split~. } { exists~ p. } Qed. Theorem state_star_pure_r: forall P H h, (H \* \[P]) h = (H h /\ P). Proof. intros. rewrite hstar_comm. rewrite state_star_pure_l. apply* prop_ext. Qed. Theorem himpl_state_star_pure_r: forall P H H', P -> (H ==> H') -> H ==> (\[P] \* H'). Proof. introv HP W. intros h K. rewrite* state_star_pure_l. Qed. Theorem state_pure_inv_hempty: forall P h, \[P] h -> P /\ \[] h. Proof. introv M. rewrite <- state_star_pure_l. rewrite~ hstar_hempty_r. Qed. Theorem state_pure_intro_hempty: forall P h, \[] h -> P -> \[P] h. Proof. introv M N. rewrite <- (hstar_hempty_l \[P]). rewrite~ state_star_pure_r. Qed. Theorem himpl_hempty_state_pure: forall P, P -> \[] ==> \[P]. Proof. introv HP. intros h Hh. applys* state_pure_intro_hempty. Qed. Theorem himpl_state_star_pure_l: forall P H H', (P -> H ==> H') -> (\[P] \* H) ==> H'. Proof. introv W Hh. rewrite state_star_pure_l in Hh. applys* W. Qed. Theorem hempty_eq_state_pure_true : \[] = \[True]. Proof. applys himpl_antisym; intros h M. { applys* state_pure_intro_hempty. } { forwards*: state_pure_inv_hempty M. } Qed. Theorem hfalse_hstar_any: forall H, \[False] \* H = \[False]. Proof. intros. applys himpl_antisym; intros h; rewrite state_star_pure_l; intros M. { false*. } { lets: state_pure_inv_hempty M. false*. } Qed. (* ### state_register *) Theorem state_register_intro: forall register value, ($register == value) (machine_state empty {[ register := value ]}). Proof. intros; hnf; auto. Qed. Theorem state_register_inversion: forall register value state, ($register == value) state -> state = (machine_state empty {[ register := value ]}). Proof. intros ??? A; hnf in A; auto. Qed. Theorem state_star_register_same register v1 v2: ($register == v1) \* ($register == v2) **> \[False]. Proof. hnf; intros ? (s1 & s2 & ?%state_register_inversion & ?%state_register_inversion & [] & _); elimtype False; subst; simpl in *; eapply map_disjoint_spec; trivial; apply lookup_singleton. Qed. (* ### state_counter *) Theorem state_counter_intro: forall counter, (pc_at counter) (machine_state {[ 0%fin := counter ]} empty). Proof. intros; hnf; auto. Qed. Theorem state_counter_inversion: forall counter state, (pc_at counter) state -> state = (machine_state {[ 0%fin := counter ]} empty). Proof. intros ?? A; hnf in A; auto. Qed. Theorem state_star_counter v1 v2: (pc_at v1) \* (pc_at v2) **> \[False]. Proof. hnf; intros ? (s1 & s2 & ?%state_counter_inversion & ?%state_counter_inversion & [? _] & _); elimtype False; subst; simpl in *; eapply map_disjoint_spec; trivial; apply lookup_singleton. Qed. (* ### state_exists *) Theorem state_exists_intro: forall A (a: A) (P: A -> Assertion) state, P a state -> (\exists a, P a) state. Proof. intros; hnf; eauto. Qed. Theorem state_exists_inversion: forall X (P: X -> Assertion) state, (\exists x, P x) state -> exists x, P x state. Proof. intros ??? A; hnf in A; eauto. Qed. (* ### state_forall *) Theorem state_forall_intro: forall A (P: A -> Assertion) state, (forall a, P a state) -> (state_forall P) state. Proof. intros; hnf; assumption. Qed. Theorem state_forall_inversion: forall A (P: A -> Assertion) state, (state_forall P) state -> forall a, P a state. Proof. intros; hnf; trivial. Qed. (*Theorem state_star_forall H A (P: A -> Assertion): (state_forall P) \* H **> state_forall (P \* H). Proof. intros h M. destruct M as (h1&h2&M1&M2&D&U). intros x. exists~ h1 h2. Qed.*) Theorem state_implies_forall_r: forall A (P: A -> Assertion) H, (forall a, H **> P a) -> H **> (state_forall P). Proof. intros ??? M ???; apply M; assumption. Qed. Theorem state_implies_forall_l: forall A a (P: A -> Assertion) H, (P a **> H) -> (state_forall P) **> H. Proof. intros ???? M ??; apply M; trivial. Qed. Theorem state_forall_specialize: forall A a (P: A -> Assertion), (state_forall P) **> (P a). Proof. intros; apply (state_implies_forall_l a); auto. Qed. (* ### state_wand *) Theorem state_wand_equiv: forall H0 H1 H2, (H0 **> H1 \-* H2) <-> (H1 \* H0 **> H2). Proof. unfold state_wand. iff M. { rewrite state_star_comm. applys state_implies_state_star_trans_l (rm M). rewrite state_star_state_exists. applys state_implies_state_exists_l. intros H. rewrite (state_star_comm H). rewrite state_star_assoc. rewrite (state_star_comm H H1). applys~ state_implies_state_star_state_pure_l. } { applys state_implies_state_exists_r H0. rewrite <- (state_star_hempty_r H0) at 1. applys state_implies_frame_r. applys state_implies_hempty_state_pure M. } Qed. Theorem state_implies_state_wand_r: forall H1 H2 H3, H2 \* H1 **> H3 -> H1 **> (H2 \-* H3). Proof. introv M. rewrite~ state_wand_equiv. Qed. Theorem state_implies_state_wand_r_inv: forall H1 H2 H3, H1 **> (H2 \-* H3) -> H2 \* H1 **> H3. Proof. introv M. rewrite~ <- state_wand_equiv. Qed. Theorem state_wand_cancel: forall H1 H2, H1 \* (H1 \-* H2) **> H2. Proof. intros. applys state_implies_state_wand_r_inv. applys state_implies_refl. Qed. Arguments state_wand_cancel: clear implicits. Theorem state_implies_hempty_state_wand_same: forall H, \[] **> (H \-* H). Proof. intros. apply state_implies_state_wand_r. rewrite~ state_star_hempty_r. Qed. Theorem state_wand_hempty_l: forall H, (\[] \-* H) = H. Proof. intros. applys state_implies_antisym. { rewrite <- state_star_hempty_l at 1. applys state_wand_cancel. } { rewrite state_wand_equiv. rewrite~ state_star_hempty_l. } Qed. Theorem state_wand_state_pure_l: forall P H, P -> (\[P] \-* H) = H. Proof. introv HP. applys state_implies_antisym. { lets K: state_wand_cancel \[P] H. applys state_implies_trans K. applys* state_implies_state_star_state_pure_r. } { rewrite state_wand_equiv. applys* state_implies_state_star_state_pure_l. } Qed. Theorem state_wand_curry: forall H1 H2 H3, (H1 \* H2) \-* H3 **> H1 \-* (H2 \-* H3). Proof. intros. apply state_implies_state_wand_r. apply state_implies_state_wand_r. rewrite <- state_star_assoc. rewrite (state_star_comm H1 H2). applys state_wand_cancel. Qed. Theorem state_wand_uncurry: forall H1 H2 H3, H1 \-* (H2 \-* H3) **> (H1 \* H2) \-* H3. Proof. intros. rewrite state_wand_equiv. rewrite (state_star_comm H1 H2). rewrite state_star_assoc. applys state_implies_state_star_trans_r. { applys state_wand_cancel. } { applys state_wand_cancel. } Qed. Theorem state_wand_curry_eq: forall H1 H2 H3, (H1 \* H2) \-* H3 = H1 \-* (H2 \-* H3). Proof. intros. applys state_implies_antisym. { applys state_wand_curry. } { applys state_wand_uncurry. } Qed. Theorem state_wand_inv: forall h1 h2 H1 H2, (H1 \-* H2) h2 -> H1 h1 -> Fmap.disjoint h1 h2 -> H2 (h1 \u h2). Proof. introv M2 M1 D. unfolds state_wand. lets (H0&M3): state_exists_inv M2. lets (h0&h3&P1&P3&D'&U): state_star_inv M3. lets (P4&E3): state_pure_inv P3. subst h2 h3. rewrite union_empty_r in *. applys P4. applys* state_star_intro. Qed. End AbstractMachine. ================================================ FILE: old/main.md ================================================ ```v Theorem absurd_stuck instr: ~(stopping instr) -> forall program cur, (cur_instr cur program) = Some instr -> (forall next, ~(@step program cur next)) -> False. Proof. intros Hstopping ?? Hcur Hstuck; specialize (not_stopping_not_stuck Hstopping program cur Hcur) as [next]; specialize Hstuck with next; contradiction. Qed.s Theorem absurd_well_founded_minimal {T} (P: T -> T -> Prop) (least other: T): well_founded P -> P least other -> ~(P other least). Proof. intros. Qed. Section well_founded_compatibility. Variable A B: Type. Variable RA: A -> A -> Prop. Variable RB: B -> B -> Prop. Variable RB_well_founded: well_founded RB. Variable f: A -> B. Hypothesis H_compat_A: forall a1 a2: A, (RA a1 a2) -> (RB (f a1) (f a2)). Hypothesis H_compat_B: forall a1 a2: A, (RB (f a1) (f a2)) -> (RA a1 a2). https://github.com/charguer/tlc/blob/master/src/LibWf.v https://coq.inria.fr/library/Coq.Init.Wf.html Theorem yo: forall min other, RB (f min) (f other) -> Acc RA min. Proof. intros ?? HRB. apply H_compat_B in HRB. Hint Constructors Acc: core. Qed. Theorem well_founded_compat: well_founded RA. Proof. constructor. unfold well_founded in *. constructor. intros. rename y into a1; rename a into a2. specialize (H_compat a1 a2). remember (f a1) as b1; remember (f a2) as b2. destruct H_compat as [H_compat_A H_compat_B]. specialize (H_compat_A H) as ?. specialize (RB_well_founded b1) as Hb1. specialize (RB_well_founded b2) as Hb2. inversion Hb1. specialize (Acc_inv RB_well_founded). Qed. Check RB_well_founded. EndSection well_founded_compatibility. ``` ================================================ FILE: old/main.v ================================================ Add LoadPath "/home/blaine/lab/cpdtlib" as Cpdt. Set Implicit Arguments. Set Asymmetric Patterns. Require Import List String Cpdt.CpdtTactics Coq.Program.Wf. From stdpp Require Import base fin vector options. Import ListNotations. Require Import theorems.utils. Section Sized. Context {size: nat}. Notation register := (fin size). Record MachineState := machine_state { counter: nat; registers: (vec nat size); program_memory: list Instruction }. Inductive Operand: Type := | Literal (n: nat) | Register (r: register) . Definition eval_operand (cur: MachineState) (operand: Operand) : nat := match operand with | Literal n => n | Register r => (cur.(registers) !!! r) end . Inductive Instruction := | InstExit | InstMov (src: Operand) (dest: register) | InstAdd (val: Operand) (dest: register) (*| InstJump (to: nat)*) (*| InstBranchEq (a: Operand) (b: Operand) (to: nat)*) (*| InstBranchNeq (a: Operand) (b: Operand) (to: nat)*) (*| InstStore (src: Operand) (dest: Operand)*) (*| InstLoad (src: Operand) (dest: register)*) . Hint Constructors Instruction: core. Notation Within cur := (cur.(counter) < (length cur.(program_memory))) (only parsing). Notation cur_instr cur := (lookup cur.(counter) cur.(program_memory)) (only parsing). Notation get_instr cur := (@safe_lookup _ cur.(counter) cur.(program_memory) _) (only parsing). Notation get cur reg := (cur.(registers) !!! reg) (only parsing). Notation update cur dest val := (vinsert dest val cur.(registers)) (only parsing). Notation incr cur := (S cur.(counter)) (only parsing). Inductive Step: MachineState -> MachineState -> Prop := | Step_Mov: forall cur src dest, (cur_instr cur) = Some (InstMov src dest) -> Step program cur (machine_state (incr cur) (update cur dest (eval_operand cur src)) ) | Step_Add: forall cur val dest, (cur_instr cur) = Some (InstAdd val dest) -> Step program cur (machine_state (incr cur) (update cur dest ((eval_operand cur val) + (get cur dest))) ) (*| Step_Jump: forall cur to, (cur_instr cur program) = Some (InstJump to) -> Step program cur (machine_state to cur.(registers))*) (*| Step_BranchEq: forall cur a b to, (cur_instr cur program) = Some (InstBranchEq a b to) -> IF (a = b) then Step program cur (machine_state to cur.(registers)) else Step program cur (machine_state (incr cur) cur.(registers))*) (*| Step_BranchNeq: forall cur a b to, (cur_instr cur program) = Some (InstBranchNeq a b to) -> IF (a = b) then Step program cur (machine_state (incr cur) cur.(registers)) else Step program cur (machine_state to cur.(registers))*) . Hint Constructors Step: core. Theorem Step_always_Within program cur next: Step program cur next -> Within program cur. Proof. inversion 1; eauto using lookup_lt_Some. Qed. Inductive stopping: Instruction -> Prop := | stopping_Exit: stopping InstExit . Hint Constructors stopping: core. Definition is_stopping: forall instr, {stopping instr} + {~(stopping instr)}. refine (fun instr => match instr with | InstExit => Yes | _ => No end ); try constructor; inversion 1. Defined. Theorem stopping_stuck instr: stopping instr -> forall program cur next, (cur_instr cur program) = Some instr -> ~(Step program cur next). Proof. intros Hstopping ???? HStep; inversion Hstopping; inversion HStep; naive_solver. Qed. Theorem not_stopping_not_stuck instr: ~(stopping instr) -> forall program cur, (cur_instr cur program) = Some instr -> exists next, Step program cur next. Proof. destruct instr; try contradiction; eauto. Qed. Inductive branching: Instruction -> Prop := (*| branch_BranchEq: forall a b to, branching (InstBranchEq a b to)*) (*| branch_BranchNeq: forall a b to, branching (InstBranchNeq a b to)*) (*| branch_Jump: forall to, branching (InstJump to)*) . Hint Constructors branching: core. Definition is_branching: forall instr, {branching instr} + {~(branching instr)}. refine (fun instr => match instr with (*| InstBranchEq _ _ _ => Yes*) (*| InstBranchNeq _ _ _ => Yes*) (*| InstJump _ => Yes*) | _ => No end ); inversion 1. Defined. Inductive sequential: Instruction -> Prop := | sequential_Mov: forall src dest, sequential (InstMov src dest) | sequential_Add: forall val dest, sequential (InstAdd val dest) . Hint Constructors sequential: core. (*Definition is_sequential*) Theorem sequential_always_next instr: sequential instr -> forall (program: list Instruction) cur next, (cur_instr cur program) = Some instr -> Step program cur next -> counter next = S (counter cur). Proof. intros ????? HStep; destruct instr; inversion HStep; auto. Qed. Notation segment_sequential segment := (Forall sequential segment). Notation NextStep program instr cur next := ((cur_instr cur (program%list)) = Some instr -> Step (program%list) cur next) (only parsing). Definition execute_instruction: forall instr (cur: MachineState), ~stopping instr -> {next: MachineState | forall program, NextStep program instr cur next} . refine (fun instr cur => match instr with | InstMov src dest => fun _ => this (machine_state (incr cur) (update cur dest (eval_operand cur src)) ) | InstAdd val dest => fun _ => this (machine_state (incr cur) (update cur dest ((eval_operand cur val) + (get cur dest))) ) | _ => fun _ => impossible end ); destruct instr; try contradiction; auto. Defined. Inductive Steps (program: list Instruction) : list MachineState -> MachineState -> Prop := | Steps_start: forall start, Steps program cur [] | Steps_Step: forall start steps prev cur, Steps program start steps prev -> Step program prev cur -> Steps program start (steps ++ [prev]) cur . (* Inductive Trace: list MachineState -> Prop := | Trace_start: forall start, Trace [start] | Trace_step: forall past cur next, Trace (past ++ [cur]) -> Step cur next -> Trace (past ++ [cur] ++ [next]) . Theorem Trace_transitive: forall before mid after, Trace (before ++ [mid]) -> Trace ([mid] ++ after) -> Trace (before ++ [mid] ++ after). Proof. Qed. *) Theorem Steps_start_inversion program cur next: Steps program cur [] next -> Step program cur next. Proof. inversion 1; subst; trivial; apply app_eq_nil in H0 as [_ Hfalse]; discriminate Hfalse. Qed. Theorem Steps_connect_tail_last program first steps tail last: Steps program first (steps ++ [tail]) last -> Step program tail last. Proof. inversion 1; subst. - apply app_cons_not_nil in H0; contradiction. - apply app_inj_tail in H0 as []; subst; assumption. Qed. Theorem Steps_connect_first_head program first head steps last: Steps program first ([head] ++ steps) last -> Step program first head. Proof. inversion 1; subst. subst. intros H; induction H. - admit. - assumption. generalize dependent steps. induction steps. - inversion 1. subst. apply app_singleton in H0 as [[]|[]]; subst. + subst_injection H2; inversion H1; subst; auto; apply Steps_start_inversion; assumption. + discriminate H2. - intros. Qed. (*Theorem Steps_append program first steps1 meet steps2 last: Steps program first steps1 meet -> Steps program meet steps2 last -> Steps program first (steps1 ++ [meet] ++ steps2) last. Proof. intros. Qed.*) Theorem list_split_around_meet {T} items: forall n (Hlength: n < length items), exists meet, meet = safe_lookup steps Hlength /\ items = (take n items) ++ [use meet] ++ (drop (S n) items). Theorem Steps_split program first steps last: Steps program first steps last -> forall n (Hlength: n < length steps), exists meet, meet = safe_lookup steps Hlength /\ Steps program first (take n steps) (use meet) /\ Steps program (use meet) (drop (S n) steps) last. Proof. intros. Qed. Definition execute_eternal (program: list Instruction) (well_formed: WellFormed program) (start: MachineState) : forall previous cur, Within program cur -> . refine (cofix execute_eternal previous cur _ _ := let (instr, _) := (get_instr cur program) in if (is_stopping instr) then (previous ++ cur) else let (next, _) := (@execute_instruction instr cur _) in execute_eternal next _ ) Defined. Theorem Steps_deterministic: forall program start between1 last1 between2 last2, Steps program start between1 last1 -> Steps program start between2 last2 -> length between1 = length between2 -> last1 = last2 /\ between1 = between2. Proof. Qed. Definition stateprop := MachineState -> Prop. (* first the partial correctness version *) Definition triple (block: list Instruction) (H Q: stateprop) := forall prefix postfix first last between, H first -> Steps (prefix ++ block ++ postfix) first between last -> Q last. Definition triple (block: list Instruction) (H Q: stateprop) := forall prefix postfix first, H first -> Within program first -> exists between last, Steps (prefix ++ block ++ postfix) first between last /\ Q last. Definition exiting_triple (block: list Instruction). (* some concept I probably need is an idea of a Steps or Trace being "within" some program segment, as in all the machine states in that trace have program counters in the segment, so I can reason about "exiting" the segment, also theorems about concatenation of traces, so I can do things like "the beginning of this trace is all within this segment, but this concatened head state isn't, therefore we've exited the segment" *) (*Theorem Trace_to_Step: forall program start steps cur, Trace program (steps ++ [start]) (Some cur) -> Steps program start steps cur. Proof. Qed. Theorem Steps_to_Trace: forall program start steps cur, Steps program start steps cur -> Trace program (steps ++ [start]) (Some cur). Proof. Qed.*) (* things to prove using Trace: - if a trace is currently Trace_exit, then the program is stuck - `execute_unsafe_eternal` is approximated by the non-eternal version, and if it returns None the program isn't well-formed and there isn't a possible next step - `execute_eternal` is approximated by the non-eternal version, and if it returns None there doesn't exist a possible next step - a well_founded relation on the program step relation implies there exists a finite number of steps such that `n = (length states), Trace states None`. also execute_program perfectly defines execute_eternal in this situation *) (* I think what I want is this: - first just *local*, as in single instruction, versions of total state assertions (hoare triples) and framed state assertions (separation logic triples) - somehow tie those together with Trace? *) (*Notation stateprop := (MachineState -> Prop) (only parsing).*) (*hoare triples assert over total state, separation triples assert over the given state and all other states*) (*Definition execute_eternal program (well_formed: WellFormed program) : MachineState -> Step_stream program. refine (cofix execute_eternal cur => let (instr, _) = safe_lookup cur program in if (is_stopping instr) then else ) Defined. CoFixpoint execute_eternal program (H: WellFormed program): Step_stream program := do_Start H .*) Definition execute_program_unsafe (program: list Instruction) : nat -> MachineState -> option MachineState . refine (fix go Steps cur := match (cur_instr cur program) with | None => None | Some instr => if (is_stopping instr) then Some cur else match Steps with | 0 => None | S Steps' => let (next, _) := (@execute_instruction instr cur _) in go Steps' next end end ); assumption. Defined. Notation WellFormed program := (forall cur next, Step program cur next -> Within program next) (only parsing). Notation InstWellFormed len_program := (fun index instr => forall program cur next, len_program <= (length program) -> lookup (index%nat) program = Some instr -> cur.(counter) = (index%nat) -> Step program cur next -> Within program next ) (only parsing). Theorem Step_implies_instr program cur next: Step program cur next -> exists instr, (cur_instr cur program) = Some instr. Proof. intros []; eauto. Qed. Notation IndexPairsWellFormed program := (fun index_instr => InstWellFormed (length program) index_instr.1 index_instr.2) (only parsing). Theorem index_pairs_InstWellFormed_implies_WellFormed program: Forall (IndexPairsWellFormed program) (imap pair program) -> WellFormed program. Proof. intros H ?? HStep; rewrite Forall_lookup in H; specialize (Step_implies_instr HStep) as [instr]; specialize (H cur.(counter) (cur.(counter), instr)); eapply H; eauto; apply index_pairs_lookup_forward; assumption. Qed. Definition check_instruction_well_formed len_program: forall index_instr, partial (InstWellFormed len_program index_instr.1 index_instr.2) . refine (fun index_instr => if (is_stopping index_instr.2) then proven else if (lt_dec (S index_instr.1) len_program) then proven else unknown (*if (is_sequential instr)*) ); destruct index_instr as [index instr]; simpl in *; intros ???? Hsome Hcounter HStep; subst; try apply (stopping_stuck s Hsome) in HStep; destruct instr; inversion HStep; try contradiction; simpl in *; subst; lia. Defined. Definition execute_program_unknown_termination (program: list Instruction) (well_formed: WellFormed program) : nat -> forall cur, Within program cur -> option MachineState . refine (fix go steps cur _ := let (instr, _) := (get_instr cur program) in if (is_stopping instr) then Some cur else match steps with | 0 => None | S steps' => let (next, _) := (@execute_instruction instr cur _) in go steps' next _ end ); eauto. Defined. Section execute_program. Variable program: list Instruction. Variable well_formed: WellFormed program. Variable progression: MachineState -> MachineState -> Prop. Variable progression_wf: well_founded progression. Variable progress: forall cur next, Step program cur next -> progression next cur. Program Fixpoint execute_program cur (H: Within program cur) {wf progression cur} : MachineState := let (instr, _) := (get_instr cur program) in if (is_stopping instr) then cur else let (next, _) := (@execute_instruction instr cur _) in execute_program next _ . Solve All Obligations with eauto. End execute_program. End Sized. (*Arguments Literal {size} _. Arguments Register {size} _. Arguments execute_program_unsafe {size} _ _ _. Arguments InstMov {size} _ _. Arguments InstAdd {size} _ _. Arguments InstBranchEq {size} _ _ _. Arguments InstBranchNeq {size} _ _ _. Arguments InstExit {size}.*) Notation Within program cur := (cur.(counter) < (length program)) (only parsing). Notation WellFormed program := (forall cur next, Step program cur next -> Within program next) (only parsing). Notation InstWellFormed len_program := (fun index instr => forall program cur next, len_program <= (length program) -> lookup (index%nat) program = Some instr -> cur.(counter) = (index%nat) -> Step program cur next -> Within program next ) (only parsing). Notation IndexPairsWellFormed program := (fun index_instr => InstWellFormed (length program) index_instr.1 index_instr.2) (only parsing). Ltac program_well_formed := match goal with | |- WellFormed ?program => let program_type := type of program in match program_type with | list (@Instruction ?size) => apply index_pairs_InstWellFormed_implies_WellFormed; find_obligations__helper (IndexPairsWellFormed program) (@check_instruction_well_formed size (length program)) (imap pair program) end end. Module redundant_additions. Definition program: list (@Instruction 1) := [ InstMov (Literal 0) (0%fin); InstAdd (Literal 1) (0%fin); InstAdd (Literal 1) (0%fin); InstAdd (Literal 1) (0%fin); InstAdd (Literal 1) (0%fin); InstAdd (Literal 1) (0%fin); InstExit ]. Theorem well_formed: WellFormed program. Proof. program_well_formed. Qed. Theorem within: Within program (state 0 [#0]). Proof. simpl; lia. Qed. Example test: execute_program_unknown_termination well_formed (length program) (state 0 [#0]) within = Some (state 6 [#5]). Proof. reflexivity. Qed. End redundant_additions. Module redundant_doubling. Definition program: list (@Instruction 1) := [ InstMov (Literal 1) (0%fin); InstAdd (Register 0%fin) (0%fin); InstAdd (Register 0%fin) (0%fin); InstAdd (Register 0%fin) (0%fin); InstExit ]. Theorem well_formed: WellFormed program. Proof. program_well_formed. Qed. Theorem within: Within program (state 0 [#0]). Proof. simpl; lia. Qed. Example test: execute_program_unknown_termination well_formed (length program) (state 0 [#0]) within = Some (state 4 [#8]). Proof. reflexivity. Qed. End redundant_doubling. (*Notation val := Operand (only parsing). Notation expr := Instruction (only parsing). Notation of_val := InstExit (only parsing). Definition to_val (e: expr): option val := match e with | InstExit _ v => Some v | _ => None end. *) (* So the first program I'm interested in verifying is this one. I want to obviously verify it's safe and such, but also I want to be main: (this label is implicit) {{ True }} Mov 0 $r1 {{ $r1 = 0 }} loop: {{ exists n < 10, $r1 = n }} Add 1 $r1 {{ exists n <= 10, $r1 = n + 1}} BranchNeq $r1 10 loop done: {{ $r1 = 10 }} Exit *) (* (*CoInductive Trace (program: list Instruction) : list MachineState -> option MachineState -> Prop := | Trace_start: forall start, Within program start -> Trace program [] (Some start) | Trace_step: forall prev cur next, Trace program prev (Some cur) -> Step program cur next -> Trace program (cur :: prev) (Some next) | Trace_exit: forall prev cur, Trace program prev (Some cur) -> (cur_instr cur program) = Some InstExit -> Trace program (cur :: prev) None .*) *) Definition program_fizzbuzz := [ (* we accept the input n in register 1 *) (* we then increment register 2 *) "begin" (InstMov 0 "$2") "main" (InstAdd 1 "$2") (* prepare for test_3 by moving our increment into register 3 *) (InstMov "$2" "$3") "test_3" (* TODO if the number is already less than 3, what's the semantics here? *) (InstSub 3 "$3") (* if the current remainder is greater than or equal to 3, we have to keep iterating *) (InstBranchLt 3 "$3" "test_3") (* otherwise we continue *) (* if the remainder isn't 0, then we skip printing "Fizz" *) (InstBranchNeq "$2" "$3" "after_fizz") (InstPrint "fizz") "after_fizz" (* prepare for test_5 by moving our increment into register 3 *) (InstMov "$2" "$3") "test_5" (InstSub 5 "$3") (* if the current remainder is greater than or equal to 5, we have to keep iterating *) (InstBranchLt 5 "$3" "test_5") (* if the remainder isn't 0, then we skip printing "Buzz" *) (InstBranchNeq "$2" "$3" "after_buzz") (InstPrint "buzz") "after_buzz" (* if our increment is less than the input, rerun the loop *) (InstBranchLt "$2" "$1" "main") (* otherwise fall through to exit *) (* should we actually literally exit? *) (InstExit) ]. ================================================ FILE: old/parser_low.rs ================================================ // use self::{Instruction::*, Value::*}; // use anyhow::Error; // use inkwell::{context::Context, types::IntType, values::IntValue}; // use nom::{ // branch::alt, // bytes::complete::tag, // character::complete, // combinator::map, // multi::separated_list0, // sequence::{preceded, tuple}, // Finish, IResult, // }; // use ocaml_interop::{ // impl_conv_ocaml_variant, ocaml_export, OCaml, OCamlInt, OCamlList, OCamlRef, ToOCaml, // }; // use std::{collections::HashMap, fs::read, path::Path, str::from_utf8}; // #[derive(Clone, Copy, Debug, PartialEq)] // pub enum Value { // Const(i32), // Ref(i32), // } // #[derive(Clone, Copy, Debug, PartialEq)] // pub enum Instruction { // Return(Value), // Add(i32, Value, Value), // } // impl_conv_ocaml_variant! { // Value { // Const(v: OCamlInt), // Ref(r: OCamlInt), // } // } // impl_conv_ocaml_variant! { // Instruction { // Return(v: Value), // Add( result: OCamlInt, op1: Value, op2: Value ), // } // } // pub fn parse_file(filename: &str) -> Result, Error> { // parse(from_utf8(read(filename)?.as_slice())?) // } // fn parse(i: &str) -> Result, Error> { // // eg: // // %0 = 1 + 1 // // %1 = %0 + 1 // // return %1 // Ok(separated_list0(tag("\n"), instruction)(i) // .map_err(|err| err.to_owned()) // .finish() // .map(|x| x.1)?) // } // fn constant(i: &str) -> IResult<&str, i32> { // // eg. 42 // complete::i32(i) // } // fn reference(i: &str) -> IResult<&str, i32> { // // eg. %2 // preceded(tag("%"), complete::i32)(i) // } // fn value(i: &str) -> IResult<&str, Value> { // alt((map(constant, Const), map(reference, Ref)))(i) // } // fn add(i: &str) -> IResult<&str, Instruction> { // // eg. %1 = 3 + %0 // map( // tuple((reference, tag(" = "), value, tag(" + "), value)), // |(result, _, op1, _, op2)| Add(result, op1, op2), // )(i) // } // fn ret(i: &str) -> IResult<&str, Instruction> { // // eg. return 4 // map(preceded(tag("return "), value), Return)(i) // } // fn instruction(i: &str) -> IResult<&str, Instruction> { // alt((add, ret))(i) // } // pub fn emit_to_file>(instructions: &[Instruction], to: P) { // let context = Context::create(); // let module = context.create_module("lab"); // let builder = context.create_builder(); // let i32_type = context.i32_type(); // let fn_type = i32_type.fn_type(&[], false); // let function = module.add_function("main", fn_type, None); // let basic_block = context.append_basic_block(function, "doit"); // builder.position_at_end(basic_block); // let mut env = (HashMap::new(), i32_type); // fn val<'ctx>(env: &(HashMap>, IntType<'ctx>), v: Value) -> IntValue<'ctx> { // match v { // Const(i) => env.1.const_int(i as u64, false), // Ref(r) => *env.0.get(&r).unwrap(), // } // } // for instruction in instructions { // match instruction { // Return(v) => { // builder.build_return(Some(&val(&env, *v))); // break; // } // Add(result, op1, op2) => { // let a = val(&env, *op1); // let b = val(&env, *op2); // env.0.insert(*result, builder.build_int_add(a, b, "")); // } // } // } // module.write_bitcode_to_path(to.as_ref()); // } // pub fn parse_file_and_emit(filename: &str) -> Result, Error> { // let prog = parse_file(filename)?; // emit_to_file(&prog, Path::new(filename).with_extension("bc")); // Ok(prog) // } // ocaml_export! { // fn rust_parse(cr, expr: OCamlRef) -> OCaml, String>> { // let expr: String = expr.to_rust(&cr); // parse(expr.as_str()).map_err(|err| format!("{:#}", err)).to_ocaml(cr) // } // fn rust_parse_file(cr, filename: OCamlRef) -> OCaml, String>> { // let filename: String = filename.to_rust(&cr); // parse_file(filename.as_str()).map_err(|err| format!("{:#}", err)).to_ocaml(cr) // } // fn rust_parse_file_and_emit(cr, filename: OCamlRef) -> OCaml, String>> { // let filename: String = filename.to_rust(&cr); // parse_file_and_emit(filename.as_str()).map_err(|err| format!("{:#}", err)).to_ocaml(cr) // } // } // #[cfg(test)] // mod tests { // use crate::*; // use std::{fmt::Debug, fs}; // fn err_to_string(r: Result) -> Result { // r.map_err(|err| format!("{:?}", err)) // } // macro_rules! test_parse { // ($name:ident, $in:expr, $out:expr) => { // #[test] // fn $name() { // assert_eq!(err_to_string(parse($in)), $out); // } // }; // } // test_parse!(noop, "", Ok(vec![])); // test_parse!(four, "return 4", Ok(vec![Return(Const(4))])); // test_parse!( // plus, // "%0 = 2 + 3\nreturn %0", // Ok(vec![Add(0, Const(2), Const(3)), Return(Ref(0))]) // ); // test_parse!( // nest_plus, // "%0 = 2 + 3 // %1 = 1 + %0 // %2 = %1 + 4 // %3 = 0 + %2 // return %3", // Ok(vec![ // Add(0, Const(2), Const(3)), // Add(1, Const(1), Ref(0)), // Add(2, Ref(1), Const(4)), // Add(3, Const(0), Ref(2)), // Return(Ref(3)), // ]) // ); // #[test] // fn from_file() { // assert_eq!(err_to_string(fs::write("test.mg", b"return 0")), Ok(())); // let result = err_to_string(parse_file("test.mg")); // assert_eq!(err_to_string(fs::remove_file("test.mg")), Ok(())); // assert_eq!(result, Ok(vec![Return(Const(0))])); // } // } // // // use inkwell::OptimizationLevel; // // // use inkwell::builder::Builder; // // use inkwell::context::Context; // // // use inkwell::execution_engine::{ExecutionEngine, JitFunction}; // // // use inkwell::module::Module; // // use std::error::Error; // // // type SumFunc = unsafe extern "C" fn(u64, u64, u64) -> u64; // // // struct CodeGen<'ctx> { // // // context: &'ctx Context, // // // module: Module<'ctx>, // // // builder: Builder<'ctx>, // // // // execution_engine: ExecutionEngine<'ctx>, // // // } // // // impl<'ctx> CodeGen<'ctx> { // // // fn jit_compile_sum(&self) -> Option> { // // // let i32_type = self.context.i32_type(); // // // let fn_type = i32_type.fn_type(&[i32_type.into(), i32_type.into(), i32_type.into()], false); // // // let function = self.module.add_function("sum", fn_type, None); // // // let basic_block = self.context.append_basic_block(function, "entry"); // // // self.builder.position_at_end(basic_block); // // // let x = function.get_nth_param(0)?.into_int_value(); // // // let y = function.get_nth_param(1)?.into_int_value(); // // // let z = function.get_nth_param(2)?.into_int_value(); // // // let sum = self.builder.build_int_add(x, y, "sum"); // // // let sum = self.builder.build_int_add(sum, z, "sum"); // // // self.builder.build_return(Some(&sum)); // // // unsafe { self.execution_engine.get_function("sum").ok() } // // // } // // // } // // fn main() -> Result<(), Box> { // // let context = Context::create(); // // let module = context.create_module("lab"); // // let builder = context.create_builder(); // // // let execution_engine = module.create_jit_execution_engine(OptimizationLevel::None)?; // // // let codegen = CodeGen { // // // context: &context, // // // module, // // // builder: context.create_builder(), // // // // execution_engine, // // // }; // // let i32_type = context.i32_type(); // // let fn_type = i32_type.fn_type(&[], false); // // let function = module.add_function("main", fn_type, None); // // let basic_block = context.append_basic_block(function, "doit"); // // builder.position_at_end(basic_block); // // let sum = builder.build_int_add(i32_type.const_int(2, false), i32_type.const_int(4, false), "sum"); // // let sum = builder.build_int_add(sum, sum, "sum"); // // builder.build_return(Some(&sum)); // // module.write_bitcode_to_path(&std::path::Path::new("lab.bc")); // // // let sum = codegen.jit_compile_sum().ok_or("Unable to JIT compile `sum`")?; // // // let x = 1u64; // // // let y = 2u64; // // // let z = 3u64; // // // unsafe { // // // println!("{} + {} + {} = {}", x, y, z, sum.call(x, y, z)); // // // assert_eq!(sum.call(x, y, z), x + y + z); // // // } // // Ok(()) // // } ================================================ FILE: posts/approachable-language-design.md ================================================ we ought to only use nonword symbols to only indicate concepts of the *language*, things that can't be represented *within* the language, ones that apply to all types equally. that way we can get the broadest use from them for example, using `..` or `...` for exclusive and inclusive range operators is embedding a *trait-level concept* (ability for some type to generate a range from a beginning to an end, or to perform an operation from one type to another) directly into the language syntax (better to just create a trait functions like `1 :upuntil 5` or `1 :upthrough 5` that return iterators). In contrast, using the same operators to spread an indexed or a named object or especially an indexed or named *type* into another is more general. it's kind of an operator on *kinds* and is shorthand for a *metaprogramming* construct over all types rather than a programming construct over concrete types. always prefer to pass data explicitly and only proofs and types (which are really just types) implicitly the big problem with the convention of global constructors for datatype fields is that the names end up having to be mangled since they aren't scoped and could collide! it's terrible design to easily allow these unscoped operators/constructors One of the big ideas to create an approachable language is to really carefully design it to separate *hard prerequisite knowledge* from *defined knowledge*. Basically hard prerequisites are all the things someone absolutely must know in order to read or write the language, for example the syntax, primitive keywords and constructs, fundamental concepts, etc. These things can be much more terse and cryptic *because the learner has a known and prerequisite learning path to understand them* in the form of the basic language documentation. We know they'll be encountering them constantly, and we expect them to know them because we've provided such a paved path to do so, so it's okay if they're terse and efficient. The same doesn't go for things that can be defined *within* the language! Some random library shouldn't have the same power as the main language to introduce syntax or constructs, since we don't have any idea if someone will be able to track down that random library as the source of those constructs. The wall between what is primitively part of the language and what has been defined in user-land within the language should be crystal clear at all times. Another benefit of not allowing arbitrary custom symbolic syntax is that when a library author defines an operation, they are *forced* to come up with some intelligible name for it, which is a moment they can choose to use as an opportunity to make their code more clear. They of course can still choose a terrible name, but at least their poor choice will be more clearly exposed as a poor choice rather than being able to hide behind some "plausible" symbology. ================================================ FILE: posts/comparisons-with-other-projects.md ================================================ # Comparisons with other projects An important question to ask of any project is: "How is the project different than X?" or more probingly: "Why build this new project instead of just using X? Why can't X address your needs?" This page attempts to thorougly answer that question. Many comparisons with other projects aren't actually that interesting, since the projects don't even have the same goals, or the comparison project isn't "maxed out" in one of Magmide's design pillars [(logical/computational/expressive power)](./posts/design-of-magmide.md). Many of these projects are essentially attempts to allow users to verify code in a fully automated way. Although full automation is nice, I ultimately don't think it's productive to hide the underlying logical ideas from users, instead of just putting in the work to explain them properly. If a tool allows manual proofs alongside metaprogramming capabilities then it can still have full automation in many domains, whereas if a tool can only prove a certain subset of claims automatically then it's forever limited to that subset. - Rust/LLVM: Not maxed out in logical power, can't prove correctness. - [Liquid Haskell](https://liquid.kosmikus.org/01-intro.html): Not maxed out in logical power since it only has refinement types and not a full type theory. Not maxed out in computational power since Haskell doesn't easily allow bare metal operations. - [Ivy](http://microsoft.github.io/ivy/language.html): Only a first order logic, so not maxed out in logical power. However the idea of separating pure functions and imperative procedures was part of the inspiration for the Logic/Host separation. - [TLA+](https://en.wikipedia.org/wiki/TLA%2B): Not based on dependent type theory, so not maxed out in logical power. Not maxed out in computational power, since the language itself is only intended for specification writing rather than combined code/proofs. - Isabelle/HOL, ACL2, PVS, Twelf: Not maxed out in logical power, [missing either dependent types, higher order logic, or general `Prop` types](http://adam.chlipala.net/cpdt/html/Cpdt.Intro.html). - [Dafny](https://dafny-lang.github.io/dafny/): Not maxed out in computational power, since it only exposes a fairly high level imperative language. It seems like they've tried too hard to create an "easy mode" tool. - [Rodin tool/B-method](https://en.wikipedia.org/wiki/Rodin_tool): Only seems to be first order, so not maxed out in logical power. Also doesn't seem to use a bare metal language and separation logic to reason about real programs, which isn't surprising since separation logic was only recognizably invented in around 2002. - [Rudra](https://github.com/sslab-gatech/Rudra): Intended to only uncover common undefined behaviors, rather than to prove arbitrary properties. - [Prusti](https://github.com/viperproject/prusti-dev): Intended to only automatically check for absence of panics or overflows, or pre/post conditions rather than arbitrary properties. - [RustHorn](https://github.com/hopv/rust-horn): Intended to only automatically check pre/post conditions rather than arbitrary properties. - [KLEE](https://llvm.org/pubs/2008-12-OSDI-KLEE.html) and related tools: Intended to only generate reasonably high coverage tests rather than prove arbitrary properties. - [Property-based testing tools](https://www.lpalmieri.com/posts/an-introduction-to-property-based-testing-in-rust/): Intended to only test a random subset of values rather than all possible values. --- Then there are many academic projects which verify software at the same deep level as Magmide intends to, but don't have the intent to create a tool that can act as both the logical and computational foundation for all software. These research projects will be very useful to learn from, but again their goals aren't as directly focused on broad engineering practice as Magmide. - [Lambda Rust/RustBelt](https://www.ralfj.de/research/phd/thesis-screen.pdf): A formalization of a realistic subset of Rust, proofs of its soundness, and proofs of the soundness of many core Rust libraries that use `unsafe`. This project is what spurred the development of the Iris separation logic that will be extensively used in Magmide. RustBelt is obviously the direct ancestor of the Magmide project, and laid the formal foundations for it to be possible. However it only intended to verify Rust code "on the side", rather than creating a tool capable of *implementing* a verified version of Rust. I hope to deeply collaborate with the RustBelt authors! - [Vellvm](https://github.com/vellvm/vellvm): A formalization of LLVM in Coq. Doesn't intend to use this formalization to create a self-hosting/verifying proof assitant. Importantly, doesn't use a higher order separation logic such as Iris, so it likely can't be used directly in Magmide. However the project itself and its creators will be invaluable sources of knowledge. - [Vale](https://www.microsoft.com/en-us/research/wp-content/uploads/2017/08/Vale.pdf): A Dafny tool capable of verifying the correctness of annotated assembly language cryptographic routines. This project is extremely cool and similar to Magmide in the sense that it is capable of verifying arbitrary conditions of bare metal code. However, it is very narrowly focused on cryptographic applications, and has no intention of implementing a general purpose language. However the success of the project (and inspired work such as [this more efficient F* verification condition generator](https://www.andrew.cmu.edu/user/bparno/papers/vale-popl.pdf)) shows that something like Magmide is possible. - [Bedrock](http://adam.chlipala.net/papers/BedrockICFP13/BedrockICFP13.pdf): A project that honestly feels very similar to Magmide! Bedrock is especially concerned with metaprogramming and verification of low-level code. However the project has been closed and the research group has been working on a [much smaller successor project `bedrock2`](https://github.com/mit-plv/bedrock2), along with [many other more academic projects](https://github.com/mit-plv/). It's very unclear to me what relationship these projects have with the private and proprietary [Bedrock Systems](https://bedrocksystems.com), other than both being directly related to [Adam Chlipala](http://adam.chlipala.net). I strongly believe it's absolutely essential these verification systems are open source, not only controlled by corporations and governments, and are shared as broadly as possible, so even if Bedrock Systems was filling the exact same need as Magmide there would be a need for an open source version. All the same, the original bedrock is yet another project that is promising for Magmide, since it shows that verified *macros* are possible and tractable. My (wildly conjecturing) guess about why the original project was discontinued is because Iris came about, which seems reasonable since just in 2020 the research group [had a guest post about Iris on their blog](https://plv.csail.mit.edu/blog/iris-intro.html#iris-intro). It probably didn't make sense to pursue their previous direction if they could learn/use Iris instead. - [DeepSpec](https://deepspec.org/main): A project verifying a whole family of extant systems end-to-end, which happens to include Vellvm. This again is very similar to the Magmide project, but isn't at all focused on creating tools suitable for mainstream engineers, or building a *new* foundational language. Although I think this research is extremely valuable, I don't think it's going to create a lot of industry excitement for verification. - [Metamath Zero](https://github.com/digama0/mm0): A project intended to create a minimal and extremely efficient language for specifications and proofs. This project is very focused on simplicity of the proof language and the speed of the verifier, which aren't particular goals of the Magmide project. Magmide is more concerned with creating a foundational tool intended for mainstream use, so simplicity/speed of the verifier is desired but not essential. Instead of relying on a simple verifier implementation, Magmide is relying on Coq to bootstrap initial correctness, and speed of verification isn't a goal until after the project is bootstrapped [("make it work, make it right, make it fast")](https://wiki.c2.com/?MakeItWorkMakeItRightMakeItFast). However I'm excited to learn lessons from mm0 both during and after Magmide's bootstrapping! - [ATS](http://www.ats-lang.org/): ATS is an extremely advanced and interesting language, which seems to already be capable of building very robust and performant code. It has lots of conceptual overlap with Magmide, such as integrating linear types reminiscent of separation-logic, compiling to a bare metal language (C), and providing special syntax for refinement types. However there are a few important differences: - the design is extremely obtuse and the learning materials very academic (frankly I found it difficult even to navigate the docs enough to evaluate the merits of the design) - the language seems to require all correctness assertions be somehow integrated into the type signatures of functions, whereas in Magmide the intent is to both allow assertions in types (using asserted types such as `Int & != 0`) as well as assertions in separate theorems, which should allow users to add more and more assertions without hopelessly cluttering the actual implementation - the "proof threading" concept where proof objects are explicitly passed and returned is very painful, and Magmide intends to instead make all proofs either attached to data values through asserted types, or simply inferred based on which data values are passed (without requiring extra syntax to signal inference), or deferred to the proof section after the end of the function - the linear type system isn't as powerful as Iris, which means all the special use cases Iris specifically worked to support are likely not supported However my largest criticism of the project is its continued insistence on the pure functional paradigm. As I discuss at the end of this page, we shouldn't be trying to force the pure functional paradigm on our inherently imperative computational environments, but instead finding ways to ergonomically encode imperative concepts in the world of pure logic. Ultimately ATS is very interesting, and I hope the creators can share their insights and ideas with Magmide as it matures. --- Now to the really interesting comparisons, those with other higher order dependently typed proof assistants. I'll focus on Coq, Lean, and F*. Before I go on, I just want to make sure something's crystal clear: **These three projects are amazing, and have obviously involved a terrifying amount of work from a large number of shockingly intelligent and hardworking people.** I'm very confident I could never have come up with the core ideas behind any of them, or implemented anything like them if I wasn't slavishly copying from them. **Nothing I say below is meant to disparage the projects!** I'm not an academic, but I've done just barely enough work to understand this academic field to apply my self-taught non-academic viewpoint. And from my viewpoint I can see the possibilility of a truly beautiful and unified language that finally gets formal verification into the mainstream. Please tell me if I'm crazy or am missing something really obvious! I'm just saying what I see. With that out of the way.... Those three projects could all be used to *logically define* Magmide, and since they all *technically* have the capability to produce running code, they could rival the intended use cases of Magmide. However none of them quite fit. All three of them certainly feel very academic, and I'm not even sure exactly how hard the projects are *trying* to be approachable and achieve general adoption. I've already talked a lot about how academics do a pretty bad job explaining their work, often assuming shared knowledge rather than pointing readers toward prerequisites, using formal definitions and jargon instead of intuitive examples and metaphors, and not prioritizing ergonomic tools. But the *real* reason I think these languages haven't achieved general adoption is more nuanced: *I strongly believe all three of them are overly dogmatic about pure functional programming.* They mistakenly assume the functional programming language at their heart will *itself* be the thing people use to build software. Functional programming may have its devotees, but there's a reason it's much less adopted than imperative methods: *real computers aren't pure or functional!* In a real computer, *every* action is impure and effectful, since even the most basic operations like updating registers or memory are intrinsically mutations of global state. The main idea of functional programming is a falsehood, one that makes some problems easier to reason about, but at the cost of ignoring the real nature of the problem. That extreme level of abstraction isn't always intuitive or helpful, and most engineers trying to build high performance systems that take advantage of the real machine will never be willing to make that sacrifice. The Magmide design in contrast *splits up* Logic and Host into separate "dual" languages, each used in exactly the way it's most natural. Logic is the imaginary level where pure functions and mathematical algorithms and idealized models exist, which is the perfect realization of the goals of functional programming. Then those logical structures only exist at compile time to help reason about the messy and truly computational behavior of Host. Separation logic is what makes it possible to make robust safety and correctness assertions about imperative code, rather than simply outlawing mutation and side effects as is done in functional languages. And with a deeply powerful separation logic like Iris we can build things like trackable effects that are more practical, flexible, and ergonomic than other effect systems. This again brings to mind the possible comparison between Rust and C: "Why build Rust? Can't you do everything in C you could do in Rust?" Well, yes you could! But... do you really want to? It isn't *only* about whether something's possible, it's about whether it's natural and clear and ergonomic. Why mix together the pure logical code and the real computational code when doing so doesn't make things easier and isn't really true? We don't want abstraction mismatches in our foundational language! So in other words... ![stop trying to make functional programming happen, it's not going to happen](https://blainehansen.me/stop-trying-to-make-fp-happen.jpg) ... or at least, just use pure functional languages in contexts where their purity is actually correct. I obviously don't think functional languages should never be used to write programs. But we have to acknowledge the limitations of functional programming. With verified imperative foundations underneath us, it will be much easier to discover and implement whatever paradigms we find truly useful in whatever contexts they're useful, such as optimizations like ["functional but in-place"](https://www.microsoft.com/en-us/research/uploads/prod/2020/11/perceus-tr-v1.pdf). Let's dive into each of those three projects in detail: ## Coq Coq has made a lot of frustrating design decisions. - Metaprogramming is [technically possible in Coq](https://github.com/MetaCoq/metacoq), but it was grafted on many years into the project, and it feels like it. - The language is extremely cluttered and obviously ["designed by accretion"](https://stackoverflow.com/questions/56517779/what-is-the-difference-between-lemma-and-theorem-in-coq). - All the documentation and introductory books were clearly written by academics who have no interest in helping people with deadlines build something concrete. Compare the [Coq standard library file for `Equivalence`](https://coq.inria.fr/library/Coq.Classes.Equivalence.html) with the somewhat related [`Eq` trait in Rust](https://doc.rust-lang.org/std/cmp/trait.Eq.html). - The [Notation system](https://coq.inria.fr/refman/user-extensions/syntax-extensions.html) just begs for unclear and profoundly confusing custom syntax, and is itself extremely overengineered. - Using the tool [can be quite punishing](https://softwarefoundations.cis.upenn.edu/lf-current/Induction.html#lab50). - It's a pure functional language with a garbage collector, so it will never perform as well as a self-hosted bare metal compiler. - And let's be honest, the name "Coq" is just terrible. Another really important problem is that Coq can only produce runnable programs with the [extraction mechanism](https://softwarefoundations.cis.upenn.edu/lf-current/Extraction.html), which gives no guarantees about the extracted code doing the same thing as the original. Extraction [isn't itself verified](https://github.com/MetaCoq/metacoq/issues/163), so arbitrary bugs are possible during the process, and even if it were verified it would rely on the target language environment being correct. Although fully verified Magmide toolchains are very far away, the design is tailored specifically to that goal. Coq has existed *since 1989* and is still a very niche tool mostly only used by academics or former academics. Rust by comparison doesn't offer anywhere close to the correctness-proving power, has only been a mature language since 2015, but has achieved truly impressive adoption. The most damning accusation I can make against Coq is that it isn't even that broadly adopted *in academia*. Why aren't almost all papers in mathematics, logic, philosophy, economics, and computer science not verified in Coq? And yet approachable tools like python and julia and matlab are much more common? Coq is still powerful enough to be very useful though, which is why I've chosen it as Magmide's bootstrapping language. I'm working on [`posts/coq-for-engineers.md`](./posts/coq-for-engineers.md) to help get passionate contributors up the learning curve enough to be helpful, because I know I can't build this all myself, and I'm not sure how interested academics will be to help me :fearful: And of course I don't think it would be wise to just throw away all the awesome work done by the Coq project. At some point we could create a parser/converter to allow old Coq code to be read and used by Magmide. ## Lean Lean is very similar to Coq, and it seems its entire purpose is to be a more cleanly designed successor! However I'm somewhat frustrated that despite having an overall cleaner design, it isn't that substantially different. For example it still has a very ["baked in" custom notation metaprogramming system](https://leanprover.github.io/lean4/doc/syntax.html) rather than something more flexible. It also makes the mistake of overemphasizing pure functional programming, and muddies the theorem proving language with a bunch of effectfulness concepts and builtin computational types. It also uses the interpreted pure functional language itself as the metaprogramming language, which will hamper performance. Overall it seems the project is more interested in the needs of academic mathematicians, or at least that seems to be the group who's actually been adopting it. I am excited to see what new things they come up with though! ## F* F* is the project that's most frustratingly close to being capable of Magmide's goal, since it explictly supports extraction to various targets and is already being used in production-grade verification projects. But again, I don't think it's going to achieve general adoption, not just because of its academic tone, but because it also muddies the pure logical language with effectful computation concepts. Effectfulness is inherently an imperative computational concept. It seems very counterproductive to me to add primitive effects to a logical lambda calculus, since the whole point of a logical language is that it can be used to model any kind of effects for any kind of system. The logical language should be absolutely pure, because its job is just to do pure logic rather than computation. Most real functions will have *many* effects, and their authors only care in a small handful of circumstances, when those effects are unexpected/undesired. It's better to use automated checks/proofs to assert a function is *free from* certain effects rather than having to explicitly list effects. [SteelCore](https://www.fstar-lang.org/papers/steelcore/steelcore.pdf), the variant of separation logic used in various F* projects, also doesn't support the kind of recursive/impredicative ghost state and complex resource algebras that Iris does, or at least only supports them if they evolve monotonically. And unforunately, F* is also a frustrating name. It isn't unavoidably clear to everyone how to say it (asterisk? splat? bullet?) and searching for the name online is an annoying dance of trying combinations like `f-star`, `f star`, or `"f*"`. This is all very frustrating, because F* is *so close!* It's right on top of the right feature set, but the fact that it *hasn't* caught the attention of the engineering mainstream is likely the only evidence you need that it *won't*. Maybe it's just not done! Maybe it could flesh out and distill its documentation and add a few conveniences, but I just don't think that's going to be enough. The F* community seems very interested in solving the tricky balance between automation and manual proofs though, and they've done a lot of cool work relating to specification inference and automated verification condition generation, so I'll be watching them very closely to see what other handy ideas they come up with! --- So there you go. Maybe my problems with Coq and Lean and F* all seem like minor gripes to you. Maybe you're right! But again, the intention of this project is to build a proof language *for engineers*. Academics have so many little cultural quirks and invisible assumptions, and I rarely come across an academic project that doesn't *feel* like one. Magmide asks the question "what if we designed a proof language from scratch to take formal verification mainstream?" No other project has done that. ================================================ FILE: posts/coq-for-engineers.md ================================================ You'll probably have to chew on these big ideas over time, so I've tried my best to make them short and easy to read through quickly. That way it should be easy to come back and reread them as you need to. First a few chapters on basic ideas I just want to make sure we're on the same page about: - basic type system ideas, basically a tour of the Rust type system - pure functional languages and lambda calculi, what they are and why they're good for proving things - boolean logic (propositional calculus), using coq functions as truth tables - predicate logic (first-order logic). here I just talk about the ideas we want to be able to encode (`forall`, `exists`, predicates), and don't relate them to coq. now we're going to get into the meat of it, the stuff that's special about coq, so first I'll just introduce all the big ideas we're going to cover with short explanations so you know they're coming - dependent types, not really getting into how to use them yet, just showing that they exist. - type theory, just talking about the ideas, again not really bothering with anything concrete. - calculus of constructions, which is just a particular type theory that allows us to define inductive types. inductive types and forall-style function types are the only primitives in the type system. the computation rules do the rest and are pretty simple rules about unfolding function calls and stuff. this chapter is where we actually start doing real interesting coq examples. we only worry about `Type` level stuff and inductive types though. start with true/false, then implication, then how implication is the same as forall, then and/or - curry-howard, and talk of the prop universe and how propositions are types and proofs are values. also talking about interactive mode and how it works. - indexed types. basically allows you to make the type generic, but generic over *values* rather than types. this ability means that the constructor rules have to be more specific. https://coq.inria.fr/refman/language/core/conversion.html that's the entire big idea section! with that we're ready to just get into the language - reflexivity - rewriting - exists proofs - automation - destruction proofs - proving negations - induction over data structures - induction over propositions - inversion - coinduction (very short, mostly just refer to outside sources) then the more "reference by example" that will slowly be filled in # What is logic? *Logic is made up!* Logic isn't "real" in any strict sense of the word. In logic, a "theory" is just some completely arbitrary collection of *rules* for defining and manipulating imaginary structures, and academic logicians just study different kinds of theories and what they're capable of doing. Anyone in the world could make up any system of rules they wanted, and then call that system a "theory". However some sets of rules are more *useful* than others! The reason we bother with logic is because it helps us think through difficult problems we want to solve, so theories are more useful the more they actually "line up" with the real world. If we're able to come up with a set of clear and strict rules we can always follow to reliably make predictions about the real world and real problems, then that set of rules is helping us be more aligned with reality. It acts as a crutch we can use to deal with complex problems. In order to really "line up" with the real world, a logical theory must be "consistent" by never telling us things that aren't true. For example if our theories of counting and arithmetic said that `2 + 2 = 5`, then they wouldn't be very useful, because in the real world when we grab two objects in one hand, another two in our other hand, and then put them together and count them, we always get `4` rather than `5`. If you wanted to you could define a number system that made `2 + 2 = 5`! It just wouldn't be very useful. Of course if you ask an academic logician they could talk your ear off about consistency and and proof theory and incompleteness and all the other meta-theoretical stuff academic logicians care about. I'm obviously simplifying things here, but that's because I don't really think it helps us much to spin our wheels thinking about these big questions when thinkers over the centuries have already given us excellent practical answers. When you write computer programs you don't bother to worry about how the electrons are moving around in your machine's transistors or what subatomic fields are allowing them to do so, that's all just abstracted. Let's do the same thing here. # What are propositional/predicate/first order/higher order logic? Academic logicians have categorized different logical systems by how "powerful" they are. Basically https://en.wikipedia.org/wiki/First-order_logic - Propositional: basic ideas about true/false, and different truth table operations on those values. - Predicate/first-order logic: creating functions # What are dependent types? Think of a function that is supposed to take an integer and return a boolean representing whether or not that integer is equal to `1`. In Rust we might write that function like this: ```rust fn is_one(n: usize) -> bool { n == 1 } ``` The type of that function is `(usize) -> bool`, representing the fact that the function takes a single `usize` argument and returns a `bool`. But notice that the *type* `(usize) -> bool` can apply to *many* different functions, all of which do different things: ```rust fn is_one(n: usize) -> bool { n == 1 } fn is_zero(n: usize) -> bool { n == 0 } fn always_true(_: usize) -> bool { true } fn always_false(_: usize) -> bool { false } // ... many other possible functions ``` What if we want our type system to be able to *guarantee* that the boolean returned by our function did in fact perfectly correspond to the integer passed in being equal to `1`? Dependent types allow us to do that, because they are *types* that can reference *values*. Here's a Coq function that returns what's called a "subset type" doing exactly what we want (don't worry about understanding this code right now): ```v From stdpp Require Import tactics. Program Definition is_one n: {b | b = true <-> n = 1} := match n with | 1 => true | _ => false end . Solve All Obligations with naive_solver. ``` In Coq's dependent type system, any *value* used earlier in a type can be referenced in any type afterward. For example the type of `is_one` uses the `forall` keyword to introduce the name `n` that is referenced in the return type: `forall (n: nat), {b: bool | b = true <-> n = 1}`. Again, don't worry about the details yet, we'll go over that in detail later. # What is induction? # What is Type Theory? To understand type theory, it is actually helpful to understand what came *before* type theory: set theory. Type theory is just a way of defining what kinds of values can exist, and what operations can be performed on those values. That's really it! "Type theory" is actually a big umbrella term that contains a few *specific* type theories, but they all share one basic idea: every *term* in the logic has such as variables or functions has some *type* defining what operations are allowed for that term. also type theories define "rewriting rules", basically computation rules, about how some term can be "computed" or "evaluated" or "reduced" to transform it into a different term. Usually these rewriting rules are designed so that the terms only get "simpler", or get closer and closer to being pure values that can't be reduced any further. Academics call these irreducible terms "normal forms". types vs terms vs values https://en.wikipedia.org/wiki/Type_theory#Basic_concepts https://en.wikipedia.org/wiki/Inductive_type # What are pure functional languages? # What is the Lambda Calculus? # What is the Curry-Howard Correspondence? https://en.wikipedia.org/wiki/Intuitionistic_type_theory Types can be seen as propositions and terms as proofs. In this way of reading a typing, a function type {\displaystyle \alpha \rightarrow \beta }\alpha \rightarrow \beta is viewed as an implication, i.e. as the proposition, that {\displaystyle \beta }\beta follows from {\displaystyle \alpha }\alpha . https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspondence # What is the Calculus of Constructions? https://en.wikipedia.org/wiki/Calculus_of_constructions # What are Hoare Triples? Hoare Triples were invented by [Tony Hoare](), and are a very simple method for making logical assertions about the behavior of programs. Essentially, a "triple" is: - A piece of a program, or a *term* that we're making assertions about. Depending on the kind of language, it could be a function, a series of statements, an expression, etc. - A *precondition* that we assume to be true before the term is evaluated. - A *postcondition* that we claim will be true after the term is evaluated. We "assert" a triple by proving that if the precondition is true before you evaluate the term then the postcondition will always be true. This makes the precondition a *requirement* for anyone evaluating the term if they want the postcondition. The pre/post conditions are usually written in double brackets (`{{ some assertion }}`) and put before and after the term. Here's a really basic example: ``` {{ n = 0 }} while (n < 5) { n = n + 1 } {{ n = 5 }} ``` We can write triples that aren't true, we just won't be able to prove them! ``` {{ n = 0 }} while (n < 5) { n = n + 1 } {{ n = 6 }} // wrong! ``` There's lots of theory about properties of Hoare triples, but they get really interesting when combined with Separation Logic. # What is Separation Logic? When we're writing a real computer program, all the values we pass around "live" somehwere in the machine, either a cpu register or memory. When we pass values to other parts of the program, we need to somehow tell that other part where our values are, either by copying them to some different place or just giving a reference to where the value is stored. This means we don't really "give" the value, we just put it somewhere we know the other part will be looking for it. When we're writing a real computer program, all the values being passed around actually "live" somewhere in the machine, either in a register or memory. Let's make up ``` ``` This means they could be "destroyed" by writing different values into their spot. This is a very important quality of a program that has to be respected in order to get the program right, since carelessly writing values to different spots in the machine could destroy values other parts of the program are relying on. For a long time formal verification theories didn't do a good job of acknowledging this, which is why they typically only worked with pure functional systems where no value could be mutated. Doing so made it easy to pretend the real computational values were actually purely imaginary logical values, but this made it impractical to prove things about real high performance programs. Separation logic was invented as a solution to this problem. It's a system that makes it possible to encode "destructibility" or "ownership" into values, so they can finally reason about real locations in a machine. Let's go through how it works. If we have some assertion in a normal logic, such as `A & B`, we're allowed to "duplicate" parts of our assertion as long as doing so doesn't change its meaning. For example, if `A & B` is true, then so is `A & (A & B)` or `(A & B) & (A & B)`. However you can probably guess that this kind of duplication isn't actually consistent if the assertions are talking about values that live in some spot of the machine. Separation logic introduces a concept called the "separating conjunction", that basically claims *ownership* of some assertion, and requires us to "give up" an assertion if we want to share it with someone else. So we can work out examples, let's make up a notation to encode our assertions about memory locations: we'll decide that something like `[10] => 50` says that memory address `10` holds the value `50`. The all-important separating conjunction is almost always written with the `*` symbol, and is usually read aloud as "and separately". So `[1] => 1 * [2] => 2` would be read as "memory address `1` holds `1` *and separately* memory address `2` holds `2`". Here's the really important part: the separating conjunction is *defined* so that it isn't allowed to combine multiple assertions about a single location under any circumstances. For example `[1] => 1 * [1] => 1` isn't allowed, even though the two assertions say the same thing! This means that if we want to call a function and share knowledge of some memory location we've been writing to, we have to *give up* our assertion of that memory location while the function is working with it, and we can't just make a copy for ourselves. This is the killer feature of separation logic: it encodes the idea of destructible resources with some very simple rules. --- academic to engineer translation dictionary --- By the time we're done, you'll understand these ideas Dependent types, which allow types to reference values and so represent any logical claim about any value Curry Howard equivalence, which discovered that a computer program can literally represent a logical proof Type theory, the system that can act as a foundation for all logic and math, and is the thing that inspired programming in the first place Calculus of constructions, the particular kind of type theory used by coq Proof objects, the essential realization that proofs are just data, because logical claims are just types Separation logic, the variant of logic that deals with finite resources instead of duplicable assertions Basic type system ideas, they can skip if they know rust Boolean logic, they can skip if they know about truth tables and de morgans law Type theory by way of set theory, talking about the in operator and quantification and implication in my view, there are two major areas of learning you have to do to use Coq - The big picture theoretical ideas such as Type Theory, The Calculus of Constructions, The Curry-Howard Equivalence, and induction. Some of these are genuinely mind-bending and difficult to wrap your head around! - The actual *skill* of proving things, which is equal parts art and science. Most guides I've encountered discuss both in an intermingled style, and seem to order their examples based on how big or complicated they are versus how difficult they are to understand. Basically all of them encourage you to open the guide in an editor environment capable of stepping through the proof tactics so that you can see how the proofs work as you read along. In my experience trying to learn Coq, I routinely got hung up on the *concepts*, and found it difficult to really understand the proofs even if I could step through them. And similarly as I've tried to actually use Coq I've found it annoying to have to dig through a whole textbook to just find some tactic explanation or syntax example. For these reasons this guide goes a different direction. - The first part isn't really intended to be stepped through in an editor. It uses lots of examples, but it focuses on explaining the big ideas in a distilled and focused way. This part of the guide you can just read through and ponder. You'll likely have to sleep on many of the concepts to really get them. - The second part is more of a workshop in practical proving. It doesn't have any big ideas to share, but just talks about different proof strategies, goes through detailed examples, and goes over how many common tactics actually work. This is absolutely intended to be hands-on and done in an editor. - The third part is just a "by example" style reference, intended to be a resource you use while you're working on real projects. Coq is a *huge* language with tons of tactics and concepts, so the reference doesn't attempt to be truly complete, but we can work toward that goal together. This guide is for you if you're attracted to the idea of writing provably correct software. Even if you don't have any particular projects in mind that you think could be verified, learning these ideas will massively level up your understanding of logic and computer science, and will completely change how you think about software. > If a language doesn't change the way you think about programming it isn't worth learning somebody # basic type system concepts (which are secretly type theory ideas) # pure functional languages Coq is a pure functional language some basic examples of functions note how we're defining types with the `Inductive` keyword. don't worry about why `Inductive` makes sense for now, we'll talk about that a lot later. for reasons we'll talk about later we won't define any of we only have to know how a type will be represented as bits if we actually *run* code that uses that type. if the code is merely *type checked* then the types can remain truly imaginary haskell isn't *truly* pure and functional, it has little holes intentionally built into the language so that it can actually be used to run real computer programs. how it does that isn't relevant for us right now, so I won't go into it. but Coq *really is* absolutely pure, which is why *on its own* it can't really be used for real computation. the language itself has no way of interacting with an operating system, or printing to consoles, or a way to be defined in terms of bits and machine instructions. the real purpose of the language is only really to be type checked and not run but we can do these two things: - use a language interpreter to "run" the language. this is extremely slow and of course can't always happen if we're doing things with possibly infinite data or whatever - *extract* the language to a different one like ocaml or haskell. this is kinda gross in a way, since we're assuming that this process preserves all our correctness stuff and the target language can even do the things we want. but for many purposes it's perfectly acceptable. you may be surprised to find out that Coq only has three truly primitive aspects: - ability to define inductive types - ability to pattern match on inductive types - ability to define functions the type system only has two basic primitives: - functions - inductive types That's it! Everything else, all operators and even the definition of equality (`=`) are all defined within the language. in logical languages when we define types we're defining something "out of thin air". we're defining arbitrary rules that we think will be useful because they line up with something in the real world. that's not the case for types in real programming languages, since those types are just abstractions around different ways of packaging and interpreting digital bits. even numbers aren't really *real*, and when we count "things" we're really just thinking about billions of atoms that happen to be close enough together and stuck to each other enough that we can pick up one part of the "thing" and have the rest of the "thing" come along for the ride. everything is just atoms all the way down, but the concept of numbers is a useful abstraction we can apply to our world when we're thinking about the world in terms of these "things", these chunks of stuff that are bound together enough to act as a unit. # basic boolean logic # predicate logic? # The Curry-Howard correspondence, how code can prove things # set theory, and why it was superseded by type theory # inductive types and proof objects # coding with interactive tactics # the difference between "logical" types and "computational" types https://www.youtube.com/watch?v=56IIrBZy9Rc&ab_channel=BroadInstitute https://www.youtube.com/watch?v=5e7UdWzITyQ&ab_channel=BroadInstitute https://www.youtube.com/watch?v=OlkYNDRo2YE&ab_channel=BroadInstitute https://x80.org/collacoq/ https://github.com/jscert/jscert --- Is propositional logic required to reason about consistency? What underpins propositional logic? I've been reading around about consistency and various paradoxes discovered in different logical systems (such as [Russell's paradox](https://plato.stanford.edu/entries/russell-paradox/)), and I'm being trying to figure out what it really *means* for something to be inconsistent. All the resources I can find on the topic just appeal back to basic propositional calculus by saying something about it being possible to derive some variant of `false` in the logic, but for some reason that's unsatisfying to me. If we need some ideas of true/false/contradiction in order to even do *metatheory*, then does that make the propositional calculus some kind of axiomatic bedrock for all of logic? Where's the "bottom" of our logical systems? Do we have one? Do different metatheorists simply disagree or use different systems? To me it seems a true paradox like Russell's is worrying not because it's possible to derive a contradiction (is it possible to?), but because there are some situations *where we simply don't know what to do*. It seems these are merely problems of underspecification rather than inconsistency. For example here's a dumb little logic that introduces an inconsistency, but only because we're specifying it in such informal language: - There are constants `green`, `blue`. - There is an operation `yo` which can take two of the above constants and output another one of the above constants. - `yo(x, x)` outputs x (`yo(green, green) -> green`, `yo(blue, blue) -> blue`). - If `green` is input to `yo`, the result must be `green`. - If `blue` is the second argument to `yo`, the result must be `blue`. To me it doesn't make sense to say this logic is inconsistent, it just seems "poorly typed". If we were forced to actually encode our `yo` operation as a fully explicit list of input pairs along with output, there would be duplicates or gaps in the list depending on how we chose to ignore the problems in the logic. In coq we can't even get such a logic to type-check if we don't arbitrarily resolve the inconsistency: ```v Inductive constant: Type := green | blue. Definition yo c1 c2 := match c1, c2 with (* these rules make sense *) | green, green => green | blue, blue => blue | blue, green => green (* here we have to just resolve the problems by choosing which rule "wins" *) | green, blue => blue end. Notation valid_yo f := ( (forall t, f t t = t) /\ (forall t, f green t = green) /\ (forall t, f t green = green) /\ (forall t, f t blue = blue) ). Theorem valid_yo_impossible: forall yo, valid_yo yo -> False. Proof. intros ? (_ & green_left & _ & blue_right). specialize (green_left blue). specialize (blue_right green). rewrite green_left in blue_right. discriminate. Qed. ``` Could this potentially offer us an intuitive explanation of why strongly normalizing logics such as the calculus of constructions are consistent? Since their very structure demands operations to be fully explicit and complete and type-checkable it makes it impossible to even represent truly "inconsistent" terms? Is determinisic reduction the ================================================ FILE: posts/crossing-no-mans-land.md ================================================ # Crossing No Man's Land: figuring out Magmide's path to success These are just the important points I want to make. Should I extend this to a full blog post? Is this too terse? - The real problem with existing proof languages like Coq *isn't* that they aren't powerful enough to be useful. Because of their core ideas of dependent types, proof objects, and resulting theoretical tools like separation logic, they're already extremely powerful and could be used *today* to build practical things. - However they're so poorly designed and explained and exposed to practitioners that they might as well not exist. The real problem is the culture and incentives of academia. - This is frustrating, since these projects have *technically* occupied large areas of use-case space, thereby making it much more difficult for a project like Magmide to quickly deliver incremental concrete value. Any small milestone Magmide could reach would only provide functionality technically already provided by other projects. In order to provide concrete verification value, we'd have to get detoured working on things that don't bring the language closer to a useful threshold of functionality. - Most concrete verification problems, such as those in smart contracts or cryptography or even memory safe programming, mostly need *reusable theory libraries* defining correctness/safety conditions and algorithms to check for them, as well as a tool capable of applying those conditions to real implementation code. The Magmide project has nothing to do with domain specific theory libraries, but instead seeks to create a completely general tool that makes it uniquely possible for such libraries to be created, shared, and applied. It seeks to create a foundational ecosystem, giving others the tools and education to solve their own problems by leveraging [the verification pyramid](https://github.com/magmide/magmide#do-you-really-think-all-engineers-are-going-to-write-proofs-for-all-their-code). - This is again frustrating, since those kinds of specific use cases are the only ones not already occupied by other projects. This makes me unsure what incremental project milestones Magmide could use to propel itself toward completion. - I'm becoming more and more convinced the correct short term path is to first pursue the "Coq for programmers" project, an online book that clearly explains the core ideas of Coq, guides users through the rough edges of installing and working with the tool, gives a practical crash course in theorem proving, explains methods to parse and prove assertions about the contents of external files, and develops a handful of small but interesting case studies such as formalizing a simple smart contract language and verifying programs in it. - The "Coq for programmers" project would test this hypothesis: if we create documentation/libraries/examples for existing proof languages, such that particularly correctness-conscious teams can use them for small but important applications, that will generate *just enough* interest for a broader audience to see the massive usability flaws as obvious and dire and worth solving. Then hopefully the energy and resources necessary to implement a design like Magmide's (or some other possibly better design!) will inevitably show up. - An online book project can iterate very quickly, since it can be shared for feedback at almost every stage of completion. We can learn a ton about how programmers think about correctness, as well as how difficult it is to teach and understand the core ideas and heuristic skills of a proof assistant. By finding sharp edges and bad explanations and workarounds for them, we gain a much more solid map of all the problems we need to solve in Magmide. - The "Coq for programmers" project would live inside the [Magmide github organization](https://github.com/magmide), and would routinely point to Magmide whenever it was explaining something it had to admit was obtuse or difficult. It would be a [fast static site](https://nuxtjs.org/announcements/going-full-static/), with niceties such as [quick search functionality](https://docsearch.algolia.com/) such as on [tailwindcss.com](https://tailwindcss.com/). - The essential components of the "Coq for programmers" project: - Primer on prerequisite ideas: basic algebraic types, pure functional languages, boolean/predicate logic. - Core ideas section discussing: dependent types, type theory, proof objects/curry howard correspondence, indexed types, typeclasses as proof objects. - Course in interactive theorem proving discussing: rewriting, case analysis, induction, absurdity, automation, reflection, coinduction, setoid/morphism rewriting. - Hoare logic and separation logic. - Optional "nice to have" components, depending on scope: - Fast searchable tactic and tactic notation reference, heavily skewed toward examples rather than formal explanations like in the [Coq reference](https://coq.inria.fr/refman/coq-tacindex.html). - Explanation of Iris. - Explanation of core ideas of category theory. ================================================ FILE: posts/design-of-magmide.md ================================================ # Design of Magmide To achieve the goals of the Magmide project, we have to arrive at a system with these essential components: - The Logic language, a dependently typed lambda calculus of constructions. This is where "imaginary" types are defined and proofs are conducted. - The Host language, an imperative language that actually runs on real machines. If we have such a system, then *both* components (Logic and Host) can *formally reason about each other and themselves*, and can *run with bare-metal performance*. ``` represents and implements +------------+------------+ | | | | | | v | | Logic +---------> Host | ^ | | | | +-------------------------+ logically defines and verifies ``` These two components have a symbiotic relationship with one another: Logic is used to define and make assertions about Host, and Host computationally represents and implements both Logic and Host itself. The easiest way to understand this is to think of Logic as the type system of Host. Logic is "imaginary" and only exists at compile time, and constrains/defines the behavior of Host. Logic just happens to itself be a dependently typed functional programming language! This architecture makes it possible to max out all the important aspects of the language: - **Max out logical power** by making the full power of dependent type theory available to all components at all stages. Without this the design wouldn't be able to handle lots of interesting/useful/necessary problems, and couldn't be adopted by many teams. It wouldn't be able to act as a true *foundation* for verified computing. - **Max out computational power** by self-hosting in a bare metal language. If the language were interpreted or garbage collected then it would always perform worse than is strictly possible. - **Max out expressive power** by allowing deep metaprogramming capability. Metaprogramming is basically a cheat code for language design, since it gives a language access to an infinite range of possible features without having to explicitly support them. It's the single best primitive to add in terms of implementation overhead versus expressiveness. For a long time the goal of this project was to build a *new* Host language, something analogous to [LLVM](https://en.wikipedia.org/wiki/LLVM), so that all the formal reasoning could be made *foundational* (reaching all the way down to hardware), and so that extreme portability could be achieved. Those aims are absolutely still a part of the long-term vision of the project, but after discussions with [Juan Benet](https://www.linkedin.com/in/jbenetcs) and [Tej Chajed](https://www.chajed.io/) it was realized the project would be more realistic if it first sought to serve the needs of an existing language community. Rust is the obvious choice! With this realization the new project roadmap has these essential milestones: ### Build a proof assistant in Rust. A proof assistant is just a programming language with a type system powerful enough to represent pure logic, perhaps with some convenience features added on top to make it easier to practically use. This is what "Magmide" will actually be, a proof assistant written in Rust, designed from the beginning to be high-performing, highly modular and reusable from other projects, with support for and emphasis of Rust as the language of proof tactics and metaprogramming. So Rust will be to Magmide what [OCaml is to Coq](https://github.com/coq/coq). This means that Magmide will be the "Logic" language in the above diagram. ### Formalize Rust inside Magmide. A proof assistant is powerful enough to formally specify all the rules of any programming language less logically powerful than it, so just how [researchers have defined semantics for many languages in Coq](https://softwarefoundations.cis.upenn.edu/plf-current/Preface.html), so too could we define the semantics of Rust in Magmide. Doing this will *mostly* involve following in the lead of the various [RustBelt projects](https://plv.mpi-sws.org/rustbelt/) (and so would need to also translate Iris into Magmide). However it will be somewhat more difficult in our case, since we'll need to define the semantics more precisely in order to achieve the next milestone. ### Allow Magmide to formally certify Rust! If Magmide is implemented in Rust, and the formal semantics of Rust are transcribed in Magmide, then it's possible to ingest *real* Rust code and formally reason about it in Magmide! [Reflective proofs](http://adam.chlipala.net/cpdt/html/Reflection.html) are ones in which a *computable function* is used to certify some input data has some properties. To do this you need to be able to prove that certain properties of the *certifying function* demonstrate properties of the *input data*. This technique allows the proof assistant to merely run a function at proof-checking time rather than checking a possibly massive proof object. This kind of recursive self-analysis will be extremely powerful, especially to [hopefully dramatically improve the performance of many proof search/checking scenarios](https://gmalecha.github.io/reflections/2017/speeding-up-proofs-with-computational-reflection) (more discussion about performance below). To achieve recursive self-analysis we will: - Implement the "Host reflection" rule in Magmide. This means writing a special rule into the proof checker that accepts a proposition as proven if: given an *AST* of a Rust function and a normal Magmide proof that this AST is a "certifier", it compiles and successfully runs over whatever inputs are conditions of the proposition in question. - Implement the systems that can actually ingest real rust ASTs in whatever way is necessary for the Host reflection rule. - Design and implement whatever syntactic affordances make it clean and ergonomic to make proof assertions about real Rust. *Handwaving goes here!* This could happen in a multitude of different ways, and could be incrementally improved over time. - Figure out the [Trackable Effects](https://github.com/magmide/magmide#gradually-verifiable) system. This seems important to really make Rust fully formalizable, since effects are such a critical part of real system correctness. --- After the above milestones are achieved, we will have officially achieved the base goals of the Magmide project! From there we'd be able to incrementally improve performance and usability, and also take on further challenges: - Circle back to extending the formal foundations all the way to the hardware! This would involve verifying LLVM or building some new LLVM analogue, as well as the specific [architecture backends](https://en.wikipedia.org/wiki/LLVM#Backends). - Build a formally verified Rust compiler! If we can formally verify Rust code, then we can incrementally verify the compiler itself. - Formally verify the Magmide proof checker using a trusted theory base, much like was done in the [metacoq project](https://metacoq.github.io/). # Interlude on proof assistant performance One of the main reasons proof assistants and formal verification in general aren't mainstream is because existing proof assistants are *slow*. It isn't uncommon for large academic formalizations to take *many hours* to complete proof checking. This obviously can't scale to industrial codebases. This performance problem is largely a consequence of lack of priority. Almost all proof assistants were designed and built by academics for their own purposes, and industrial use is largely considered a bonus. Good work is certainly done to improve performance, but few projects concern themselves with that from the beginning. There is some low-hanging fruit to be had here, such as using [incremental compilation](https://blog.rust-lang.org/2016/09/08/incremental.html). Overall though, mere compiler design probably isn't the biggest problem. It seems pretty obvious that the *most* concerning problem is the poor performance of proof searching tactics. In languages such as Coq, proof tactics perform poorly for a few reasons: - The [Coq Ltac language is a separate interpreted language](https://coq.inria.fr/refman/proof-engine/ltac2.html#ltac2). Interpreted languages are slow! And such an ad-hoc language embedded into a larger system will never get the amount of performance attention it needs. Tactics don't actually have to be "correct" or "sound", they're just computations that *attempt* to find a proof term that seems like it will correctly proof-check to solve some goal. This means they can be written in any language we want. - Tactics often function by *searching* using various heuristics to find proof terms, and these searches by their very nature can sometimes take a long time! Magmide intends to attack these performance problems in these ways: - Using and emphasizing a high-performance language for proof tactics. Rust is an amazing language, and it seems like a perfect choice, especially since using Rust allows [binding to basically any other language](https://www.hobofan.com/rust-interop/). We could allow people to bring whatever tools they want to bear on proof search! - Proof *search* is often slower than proof *checking*. Once proof search has successfully found a correct proof term, it is wasteful and slow to run that search again instead of merely caching the proof term. Intelligently implemented incremental compilation can absolutely prevent many of these wasteful repeat proof searches. - Emphasizing [reflective proofs](http://adam.chlipala.net/cpdt/html/Reflection.html) for whatever situations allow it, as discussed above. I have a hunch many of the most common industrial use cases will be amenable to reflective proof. For example, just imagine if the Rust borrow checker was sufficiently verified to be a certifier! Taken all together these improvements make a convincing case for Magmide as a foundation for a fully verified software ecosystem, and the first proof assistant to go mainstream among industrial engineers. # Other notable design choices ## Corruption Panics Magmide will formalizes some kind of `panic` effect that can be used to mark programs, making it possible for ambitious projects to prove that they *cannot* panic. However realistic low-level software must contend with the possibility of *hardware* failure that has created data corruption. It should be possible to write code that asserts the maintenance of invariants despite possible hardware failure. Excitingly the need to check possible hardware failure doesn't have to mean we must tolerate ubiquitous `panic` trackable effects on all our code. If we introduce a separate idea of a *corruption* panic, an effect requiring a proof that, *assuming consistency of the hardware axioms*, the `panic` is impossible, we can write highly defensive software without giving up proof of panic freedom under normal hardware operation. ## Assumption Panics Similarly to corruption panics, it should be possible to prove some panics will only occur if some *logical* assumption isn't true. This is different than corruption panics since those deal with *hardware* assumptions. This is a reasonable thing to include because programs will sometimes want to take advantage of some conjectured theorem and aren't capable or don't have the resources to prove it true. If a program author is willing to risk the possibility of a panic if some conjecture isn't true then they should be able to do so, and have those panics signaled differently than other panics. ## No `Set` type `Set` is just `Type{0}`, so I personally don't see a reason to bother with `Set`. It makes learning more complex, and in the situations where someone might demand their objects to live at the lowest universe level (I can't come up with any convincing places where this is truly necessary, please reach out if you can think of one), they can simply use some syntax equivalent of `Type{0}`. ## Proof-irrelevant `Prop` type? I haven't had time to thoroughly read [these](https://tel.archives-ouvertes.fr/tel-03236271/document) [papers](https://dl.acm.org/doi/pdf/10.1145/3290316) about proof-irrelevant proposition universes and how their design is related to homotopy type theory. However from my early reading it seems as if `Prop` could simply be made proof-irrelevant along with some changes to the rules about pattern matching from `Prop` to `Type` universes, and the language would be more convenient and cleaner. Please reach out if you have knowledge about this topic you'd like to share! ## No distinction between inductive and coinductive types Every coinductive type could be written as an inductive type and vice-versa, and the real difference between the two only appears in `fix` and `cofix` functions. Some types wouldn't actually be useful in one or other of the settings (a truly infinite stream can't possibly be finitely constructed and so would never be useful in a normal recursive function), but occasionally we might appreciate types that can be reasoned about in both ways. So Magmide will only have one entrypoint for defining "inductive" types, and if a type could be compatible with use in either recursive or corecursive contexts then it can be used in either. It seems we could always infer whether a type is being used inductively or coinductively based on call context. If we can't, we should have a syntax that explicitly indicates corecursive use rather than splitting the type system. Please reach out if you have knowledge about this topic you'd like to share! ## Interactive tactics are just metaprogramming In Coq the tactics system and `Ltac` are "baked in", so writing proofs in a different tactic language requires a plugin. In Magmide the default tactic language will just be a metaprogrammatic entrypoint that's picked up automatically by the parser, so any user can create their own. ``` // `prf` (or whatever syntax we choose) is basically just a "first-class" macro thm four_is_even: even(4); prf; + add_two; + add_two; + zero // you could write your own! thm four_is_even: even(4); my_prf$ ++crushit ``` ## Incremental compilation as widely as possible [Incremental compilation](https://blog.rust-lang.org/2016/09/08/incremental.html) is a critical technique to ensure most compilation/type checking runs are reasonably fast. This is a very common technique in normal programming languages, but it [doesn't seem to have been implemented widely in proof assistants](https://proofassistants.stackexchange.com/questions/335/what-is-the-state-of-recompilation-avoidance-in-proof-assistants). In proof assistants that heavily use automated tactics one of the most expensive parts of proof checking is actually running the automated tactics to discover proofs, since those tactics often have to walk a very large search space before they successfully find the right proof terms. Although bare metal tactics/metaprogramming and the computational reflection discussed in the above section will mitigate some of this cost, it still makes sense to avoid rerunning tactics or rechecking proofs if none of their dependencies have changed. ## Builtin syntax for tuple-like and record-like types In Coq all types are just inductive types, and those that only have one constructor are essentially equivalent to tuple or record types in other languages. This means that *all* data accesses have to ultimately desugar to `match` statements. This cleanliness is fine and ought to remain that way in the kernel, but we don't have to make users deal with this distinction in their own code. Although Coq has somewhat supported these patterns with `Record` and primitive projections and other constructs, the implementation is cluttered and confusing. Here's a possible example of defining and using a few inductive types: ``` // nothing interesting here, just pointing out it can be done type MyUnit // unions are roughly the same, again no interesting differences type MyBool = | True | False // however for record-like types, there should only be as much syntactic difference with other types as is absolutely necessary type Person = name: string age: nat // the only syntax allowed to construct a record-like type let some_person = Person { name: "Alice", age: 12 } print some_person.name // we could still allow explicit partial application with a "hole" operator let unknown_youth = Person { age: 12, _ } let known_youth = unknown_youth { name: "Bob" } // tuple-like types are similar type Ip = (byte, byte, byte, byte) // only syntax allowed to construct tuple-like types let some_ip = Ip(127, 0, 0, 1) // zero indexed field accessors print some_ip.0 // partial application let unknown_ip = Ip(_, 0, 0, _) let known_ip = unknown_ip(127, 1) ``` ## Anonymous union types Often we find ourselves having to explicitly define boring "wrapper" union types that are only used in one place. It would be nice to have a syntax sugar for an anonymous union type that merely defines tuple-like variants holding the internal types. For example: ``` def my_weird_function(arg: bool | nat | str): str; match arg; bool(b); if b; "yes" \else; "no" nat(n); format_binary(nat) str(s); "string = #{s}" // values can be passed without being wrapped or converted? my_weird_function(true) my_weird_function(2) my_weird_function("hello") ``` ## No implicit type coercion Although type coercions can be very convenient, they make code harder to read and understand for those who didn't write it. Similarly to how Rust chose to make all type conversions explicit with casts or [the `From`/`To` traits](https://doc.rust-lang.org/std/convert/trait.From.html), Magmide would seek to do the same. This means Magmide will have a trait/typeclass system. We can however choose to make these conversions less verbose, perhaps choosing a short name such as `to` for the conversion function, or supporting conversions directly with some symbolic syntax (`.>`?). ## Inferred proof holes The common case of writing verified functions is to write the `Type` level operations out explicitly (programmers are often quite comfortable with this kind of thinking), and then in a separate interactive proof block after the function body "fill in the blanks" for any implied `Prop` operations. In general it's more natural to separate data operations from proof operations, and Magmide will make this mode of operation the well-supported default. Users can still choose to specify both `Type` and `Prop` operations explicitly. Or since `prf` is just a macro that constructs a term of some type, interactive tactics can be used to specify an entire term (as is possible in Coq), or *just a portion* of a term. ``` def my_function(arg: input_type): output_type; // I know I need to call this function with some known inputs... arg.inner_function( known_input, other_known_input, // ... but what should this be again? prf; // some tactics... ) ``` Since often we need to help a type-checking algorithm along at some points, an `assert` keyword can be used to generate a proof obligation making sure some hypothesis type is actually available at some point in a function. This would basically be a `Prop` level type cast that must be justified in the proof block after the function. ``` def my_function(arg: input_type): output_type; let value1 = arg.function(known_value) let value2 = arg.other(something) // I know something should be true about these values... assert SomeProp(value1, value2) // ... which makes the rest of my function easier some_function_requiring_SomeProp(value1, value2) prf; // tactics proving SomeProp(value1, value2) ``` ## Builtin "asserted types" Subset types are often a more natural way of thinking about data, and packaging assertions about data into the type of the data itself frees us from a few annoyances such as having to declare proof inputs as separate arguments to functions or at different levels of a definition. Although in a dependent type theory a subset type is absolutely a strictly different type than a normal constructed value, we can make life easier by providing syntax to define and quickly pull values in and out of subset types. I call these cheap representations of subset types "asserted types". ``` // using & is syntactically cheap type MyByte = nat & < 256 // multiple assertions type EligibleVoter = Person & .age >= 18 & .alive // with parentheses if we want to be clearer type EligibleVoter = Person & (.age >= 18) & .alive // using a list of predicates and a proof that all of them hold is more flexible than a single nested proposition type AssertedType (T: Type) (assertions: list (T -> Prop)) = forall (t: T), (t, ListForall assertions (|> assertion; assertion(t))) ``` We can provide universal conversion implementations to and from types and asserted versions of themselves. Pulling a value out of an asserted type is easy. Putting a value into an asserted type or converting between two seemingly incompatible asserted types would just generate a proof obligation. This same syntax makes sense to declare trait requirements on types as well: ``` def my_function(t: T): ... ``` Asserted types are simply a broader variant of [liquid types](https://goto.ucsd.edu/~rjhala/liquid/liquid_types.pdf), so it should be possible to infer annotations and invariants in many situations, as is done in ["Flux: Liquid Types for Rust"](https://arxiv.org/abs/2207.04034). ## Cargo-like tooling There's no reason to not make the tooling awesome! ## Metaprogramming instead of custom Notation Coq's [Notation system](https://coq.inria.fr/refman/user-extensions/syntax-extensions.html) is extremely convoluted. It essentially allows creating arbitrary custom parsers within Coq. While this may seem like a good thing, it's a bad thing. Reasoning about these custom parsing and scoping rules is extremely difficult, and easy to get wrong. It adds a huge amount of work to maintain the system in Coq, and learn the rules for users. It also makes it extremely easy to create custom symbolic notation that makes code much more difficult to learn and understand. Allowing custom symbolic notation is a bad design choice, since it blurs the line between the primitive notations defined by the language (which are reasonable to expect as prerequisite knowledge for all users) and custom notations. Although Coq makes it possible to query for notation definitions, this is again just more maintenance burden and complexity that still adds significant reading friction. Magmide's metaprogramming system won't allow unsignified custom symbolic notation, and will require all metaprogrammatic concepts to be syntactically scoped within known identifiers. Instead of defining an extremely complicated set of macro definition rules, metaprogramming in Magmide will give three very simple "syntactic entrypoints", and then just expose as much of the compiler query api as possible to allow for compile-time type introspection or other higher-level capabilities. Macros can either accept raw strings as input and parse them themselves or accept Magmide parsed token trees. This complete generality means that Magmide can support *any* parsing pattern for embedded languages. Someone could even define something just like Coq's notation system if they really want to, and their custom system would be cleanly cordoned off behind a clear `macro_name$` style signifier. By just leaning all the way into the power of metaprogramming, we can allow *any* feature without having to explicitly support it. To actually use macros you can do so inline, as a block, or using a "virtual" import that processes an entire file. ### Inline macros Inspired by Rust's explicit `!` macros and javascript template literals. Raw string version: ``` macro_name`inline raw string` ``` Syntax tree version: ``` macro_name$(some >magmide (symbols args)) ``` ### Block macros Uses explicit indentation to clearly indicate scope without requiring complex parsing rules. Raw string version uses a "stripped indentation" syntax inspired by [Scala multiline strings](https://docs.scala-lang.org/overviews/scala-book/two-notes-about-strings.html#multiline-strings), but using pure indentation instead of manual `|` characters. ``` // the |` syntax could be generally used to create multiline strings // with the base indentation whitespace automatically stripped let some_string = |` my random `string` with what''' ''' ever I want // placing the literal directly against a path expression // will call that expression as a raw string macro macro_name|` some raw string the base indentation will be stripped ``` Token tree version is like "custom keywords", with an "opening block" that takes two token trees for header and body, and possible continuation blocks. Here's an example of a "custom" if-else block being used. ``` $my_if some.conditional statement; the.body >> of my thing /my_else; some_symbol() ``` ### Import macros Allows entire files to be processed by a macro to fulfill an import command. you'll notice the syntax here is exactly the same as inline macros, but the language will detect their usage in an import statement and provide file contents and metadata automatically. ``` use function_name from macro_name`./some/file/path.extension` ``` ================================================ FILE: posts/intro-verification-logic-in-magmide.md ================================================ Hello! If you're reading this, you must be curious about how it could be possible to write truly *provably correct* programs, ones that you can have the same confidence in as proven theories of mathematics or logic. You likely want to learn how to write verified software yourself, and don't have time to wade through unnecessarily clunky academic jargon or stitch together knowledge scattered in dozens of obscure journal papers. You're in the right place! Magmide has been designed from the ground up to be usable and respect your time, to enable you to gain this incredibly powerful and revolutionary skill. I hope you're excited! I powerfully believe verified software will bring in the next era of computing, one in which we don't have to settle for broken, insecure, or unpredictable software. Here's the road ahead: First we'll take a glimpse at some Magmide programs, both toys and more useful ones, just to get an idea of what's possible and how the language feels. We'll take a surface level tour of Compute Magmide, the procedural portion of the language we'll use to actually write programs. Then we'll dive into Logic Magmide, the pure and functional part that is used to make and prove logical claims: - We'll talk about why it's necessary to use a pure and functional language at all (I promise I'm not a clojure fanboy or something). - How to code in Logic Magmide, what it feels like to write pure functional algorithms and how it's different than normal programming. - A short overview of formal logic, and some comparisons to normal programming. - Type Theory, The Calculus of Constructions, and the Curry-Howard Correspondence, the big important ideas that make it possible for a programming language to represent proofs. - How to actually make and prove logical claims (it's getting good!), along with some helpful rules of thumb. Now with a working knowledge of how to use Logic Magmide, we can use it to verify our real Compute Magmide programs! - Separation Logic, the logical method that helps us reason about ownership, sharing, mutation, and destruction of finite computational resources. - Writing proofs about Compute Magmide functions and data structures. - Logical modeling, or proving some kind of alignment between a pure functional structure and a real computational one. - Testing as a gradual path to proofs, using randomized conjecture-based testing. - Trackable effects, the system that allows you to prove your program is free from unexpected side-effects such as memory unsafety, infinite loops, panics, and just about anything else. And then finally all the deeper features that make Magmide truly powerful: - Metaprogramming in Magmide, the capabilities that allow you to write your own compile-time logic. - Some basic programming language and type-system theory. - A short overview of basic computer architecture, including assembly language concepts. - The lower assembly language layers of Compute Magmide, and how to "drop down" into them. - The abstract machine system, and how Magmide can be used to write and prove programs for any computational environment. - A deeper look at Iris and Resource Algebras, the complex higher-order Separation Logic that makes Trackable Effects and lots of other things possible. Throughout all of these sections, we'll do our best to not only help you understand all these concepts, but introduce you to the way academics talk about them. We'll do so in a no-nonsense way, but we think it's a good idea to make sure you can jump into the original research if you want and not have to relearn all the "formal" names for concepts you already understand. Let's get to it! ## Example Programs and a Tour of Compute Magmide ## Logic Magmide, How to Prove Things in a Programming Language ### Why pure and functional? There are quite a few pure and functional languages, such as [haskell]() and [clojure]() and [lisp]() and [racket]() and [elm](). What makes them different? Functional languages enforce two properties, with varying degress of strictness: - All data is immutable. There is no ability to mutate data structures, only create new structures based on them. Although most functional languages have some [cheating escape hatches]() for when it's *really* necessary to mutate something. - All functions are pure, meaning that if you pass the exact same inputs into them, you always receive the exact same inputs. This means you can't perform "impure" actions such as mutate a variable that wasn't passed into the function (remember, you can't mutate *anything*!), or create side effects such as reaching out to the surrounding system by doing things like file or network operations. Since programs that couldn't interface with the surrounding system *at all* would be completely worthless, functional languages have special runtime functions that allow you to interact with the system by passing them pure functions. But they all return some kind of "side effect monad", a concept we don't need to talk about here! Now, those two properties have some nice consequences. They mean that you can't accidentally change or destroy data some other part of the program was depending on, or get surprised about the complex ways different parts of your program interact with each other, or not realize some function was actually doing expensive network operations at a time you didn't expect. But the especially important consequence of purity and immutability is that a program is *simple*, at least from a logical perspective. Every function always outputs predictable results based only on inputs, no complex and difficult to reason about webs of global mutable state are possible, the language operates as if it were simply math or logic, where everything has a precise definition that can be formally reasoned about. There's just one big obvious problem: **all that purity and immutability is a lie!** When a computer is running, the *only thing* it's doing is mutating data. Your computer's memory and processor registers are all just mutable variables that are constantly being updated. Purity and immutability are useful abstractions, but they only go abstraction deep. Without mutation and impurity, computation can be nothing more than merely theoretical. **However,** this isn't actually a problem if we *are* just talking about something purely theoretical! We'll see in the coming sections how proof assistants like Magmide don't need to *run* programs to prove things, they just need to *type check* them, meaning it *doesn't matter* if the programs can't actually be run. This means that a pure and functional language is the perfect fit for a proof assistant. All that matters for a proof is that we're able to express theoretical ideas in a way that's clear and precise and can be formally reasoned about. We don't have to care about performance or data representation or any of the details of real computation. Soon we'll even see that type theory, the logical framework powerful enough to form the foundations of all of mathematics and computer science, is itself basically just a pure functional language! In a much later section we'll also discover that it *is* actually possible to prove things about imperative mutating code, and even that mutating code can be shown to perfectly correspond with purely theoretical code. This is one of the most important contributions of Magmide, that it integrates purely logical and realistically computable code and allows them to usefully interact. But before all that, we have to build up some foundations. ### Coding in Logic Magmide The thing that makes Logic Magmide and other proof assistant languages special is *dependent types*, but we can't really understand those yet. First let's just go over the basic features of Logic Magmide, the features it shares with basically every other functional language like haskell. First, we'll define a datatype, a discriminated union (called an `enum` in Rust) that's shaped just like our old friend `boolean`. This type lives in the `Ideal` sort and so is purely theoretical. TODO Pretty simple. Logic Magmide comes with a default boolean called `bool`, but we'll use our own for a second. Now let's define a function for `Boolean`. In normal imperative languages the body of a function is a series of *statements*, commands that mutate state as you go through the function. But in pure functional languages we can't mutate anything, so the body of a function is *only one expression*. Let's define the basic `negate`, `and`, `or`, and `xor` functions: TODO Some of these use `let` to give some expression a name that's used in subsequent lines, and the last expression is the final return value of the function. While this may seem at first to be a use of mutable state and against the rules, the way the evaluation rules of these languages are defined means these `let`s are technically a part of the final expression that are just evaluated first and replaced afterwards. Don't worry about it too much! Both Logic and Compute Magmide have an awesome trait system, and the `if` operator in both uses a trait called `Testable` that relates a type back to `bool`. Let's make our `Boolean` testable by implementing this trait, and then use `if` to redefine the functions: TODO We can also define types in the shape of a record (called `struct` in Rust): TODO Or tuples: TODO Or ["unit"](https://en.wikipedia.org/wiki/Unit_type) for types that can only have one possible value: TODO And of course the different *variants* (the academic term) of a discriminated union can be shaped like any of those types: TODO things like option and result and color and ipaddress etc We can also create an *empty* type, a type that's impossible to actually construct! TODO This is a discriminated union with *zero* variants, so if we try to choose some "constructor" to build this type, we can never actually find one and so will never be able to. You may wonder why we'd ever bother to define a type we can't actually construct, but I promise we'll discover a very powerful use for this type later. Before we move on it's a good idea to just notice a few ways these different varieties of types relate to each other: - Tuples and records aren't really that different, since a record is just a tuple with convenient syntax sugar names we can use to refer to the fields. But any record or tuple type could be refactored into the other shape and the program would do the exact same thing. TODO - The basic unit and record and tuple types are essentially also discriminated unions, they just only have one variant! Deep inside Magmide all types are actually represented that way, which is why an "empty" type is possible. TODO - The `true` and `false` in `Boolean` are both just the unit type, but they're given distinct names and *defined* as being different from each other by the discriminated union they live in. The same is true for `None` in `Option` and the colors in `Rgb` and `Color`. Now we get to the thing that makes `Ideal` types special, their ability to simply represent recursive types: TODO nat This type encodes natural numbers (or unsigned integers) in the weird recursive style of the [Peano axioms](https://en.wikipedia.org/wiki/Natural_number#Peano_axioms), where `0` is of course `zero`, `1` is `successor(zero)`, `2` is `successor(successor(zero))`, and so on. Remember, `successor` isn't a *function* that increments a value, it's a *type constructor* that *wraps* children values. Don't worry, you won't have to actually write them that way in practice, since the Magmide compiler will coerce normal numbers into `nat` when it makes sense to. You may wonder why we'd represent them this way? Wouldn't this be incredibly inefficient? Whatever happened to bytes? And you'd be right! In a real program this way of encoding numbers would be an absolute disaster. But the Peano encoding is perfect for proving properties of numbers since the definition is so simple, precise, and doesn't depend on any other types. Our real programs will never use this idealized representation, but it's extremely useful when we're proving things about bits and arrays and a whole lot more. We'll see exactly how when we finally get to proofs, so for now let's not worry about it and just write some functions for these numbers: TODO nat operations add, subtract, multiply, remainder divide, is_even Another extremely useful recursive type we'll use constantly is pure functional `List`, which is generic: TODO Basically every pure functional language uses the terms [`nil` and `cons`](https://en.wikipedia.org/wiki/Cons) when defining basic lists (`Cons` is short for "*cons*tructing memory objects"), so since they're so prevalent we've decided to stick with them here. `Nil` is just a "nothing" or empty list, and `Cons` pushes a value to the head of a child list, in basically the same way as a linked list. This means `[]` would be represented as `Nil`, `[1]` as `Cons{ item=1, rest=nil }`, `[1, 2]` as `Cons{ item=1, rest=Cons{ item=1, rest=Nil } }`, etc. Again you won't have to write them that way, the normal list syntax basically every language uses (`[1, 2, 3]`) will get coerced when it makes sense. Just like with `nat`, we won't almost ever actually represent lists this way in real programs, but this definition is perfect for proving things about any kind of ordered sequence of items. Here are a few functions for `list`: TODO Now that we're basically familiar with how to code in Logic Magmide, we can start understanding how to use it to prove things! ### A Crash Course in Logic If you ever took a discrete mathematics or formal logic class in school, you likely already know everything in this section. It isn't very complicated, but let's review quickly to make sure we're on the same page. A **proposition** is a claim that can be true or false (academics often use the symbols `⊤` and `⊥` for true and false). Some examples are: - `I am right-handed` - `It is nighttime` - `I have three cookies` We can assign these claims to a variable, to make it shorter to refer to them (`:=` means "is defined as"): - `P := I am right-handed` - `Q := It is nighttime` - `R := I have three cookies` Then we can combine these variables together into bigger propositions using [logical connectives](https://en.wikipedia.org/wiki/Logical_connective): - The `not` rule reverses or "negates" (the academic term) the truth value of a proposition. It's usually written with the symbol `¬`, but we'll use `~`. So if `A := ~P` then `A` is true when `P` isn't true. - The `and` rule requires two variables to both be true for the whole "conjunction" (the academic term) to be true. It's usually written with the `∧` symbol (think of it as being "strict" or "closed", since `and` is more "demanding" and points downward), but we'll use `&`. So if `A := P & Q` then `A` is true only when `P` and `Q` are both true. - The `or` rule requires only one of two variables to be true for the whole "disjunction" (the academic term) to be true. It's usually written with the `∨` symbol (think of it as a "loose" or "open" link, since `or` is less "demanding" and points upward), but we'll use `|`. So if `A := P | Q` then `A` is true when either `P` or `Q` are true. All these connectives can have a "truth table" written for them, which tells us if the overall expression is true based on the truth of the sub expressions. TODO truth tables for not, and, or The `implication` rule is especially important in the type theory we'll get into in later chapters. It's usually written with the `→` symbol or just `->`, and it's easy to think why it's shaped like an arrow: the truth value of the left variable "points to" the truth value of the right variable. So for example, if `A := P -> Q`, then `A` is true if whenever `P` is true `Q` is also true. It's also not an accident that `->` represents both implication and the type of functions (`str -> bool`), but we'll get to that later. TODO truth table Very importantly, notice how an implication is only false in one situation, when the left variable is true and the right is false. This means that if the left variable is *false* then the implication is always true, or if the right variable is *true* then the implication is always true. Basically you can think of implications like an assumption and some conclusion that should always follow from that conclusion: if the assumption isn't true, then it doesn't matter what the conclusion is! The `iff` or "if and only if" rule is easier to grasp. Written with `↔` or `<->` it basically requires the two truth values to be in sync with each other. If `A := P <-> Q` then if `P` is true or false then `Q` has to be the same thing. TODO truth table And of course, we can combine these connectives in arbitrarily nested structures! TODO truth table showing some compound structures Notice how the connectives can be restated in terms of each other? Like how `P <-> Q` is equivalent to `(P -> Q) & (Q -> P)`? Or `Q` is equivalent to `(P -> Q) & P`? There are lots of these equivalences, which all form basic *tautologies* (formulas that are always true) that can be used when proving things. TODO list of boolean rules This basic form of propositional logic is obviously somewhere at the heart of computing, from binary bits to boolean values. We're all familiar with operators like `!` and `&&` and `||` in common programming languages like rust and java and javascript, and they just represent these rules as computations on boolean values. But simple truth values and connectives aren't really enough to prove anything interesting. We don't just want to compute basic formulas from true or false values, we want to be able to prove facts about *things*, from numbers all the way to complex programs. With the simple rules we've been talking about, we can only stick arbitrary human sentences onto variables and then tell the computer if they're true or not. We need something more powerful! First we need **predicates**, which are just functions that accept inputs and return propositions about them. So if we can write a predicate in this general shape: `predicate_name(input1, input2, ...) := some proposition about inputs`, then these are all predicates: - `And(proposition1, proposition2) := proposition1 & proposition2` - `Equal(data1, data2) := data1 == data2` (`data1` is equal to `data2`) - `Even(number) := number is even` (I'm cheating here, since this is still just some arbitrary sentence. Let's ignore it for now 😊) We're also going to need these two ideas: - The "for all" rule (or *universal quantification*), saying that "for all values" some predicate is true when you input the values. Academics use the `∀` symbol for "for all", but I'll just write `forall`: - `forall number, Even(number) -> Even(number + 2)` (forall values `number`, if `number` is even then that implies `number + 2` is also even) - `forall data1 data2 data3, data1 == data2 & data2 == data3 -> data1 == data3` (forall values `data1` `data2` `data3`, if `data1` is equal to `data2` and `data2` is equal to `data3`, then that implies `data1` is equal to `data3`) - The "exists" rule, (or *existential quantification*), saying that "there exists" a value where some predicate is true when you input the values. Academics use the `∃` symbol for "there exists", but I'll just write `exists`: - `exists number, Even(number)` (there exists a `number` such that `number` is even) - `exists data1 data2, data1 == data2` (there exists a `data1` and `data2` such that they are equal to each other) The `forall` rule seems especially powerful! It would be extremely useful to prove that something is true about a potentially infinite "universe" of values. But how do we actually prove something like that in a programming language? We obviously can't just run all those infinite values through some function and test if returns true, especially since the whole point of using a purely theoretical programming language was that we don't actually have to run programs in order to prove things. The crucial trick is to represent our propositions and predicates as *types* instead of data! Let's see how it works. ### Type Theory, the Calculus of Constructions, and the Curry-Howard Correspondence A reasonable place to start is by figuring out how to represent the basic ideas of true and false in Logic Magmide. We might try just defining them as a discriminated union, our old friend `boolean`: TODO But if we walk much further down this path, we won't get anywhere. We'll end up having to actually compute and compare booleans in all our "proofs", since `true` and `false` are only different *values*, not different *types*. This is a good place to reveal something really important but fairly surprising: if you're a programmer, *then you already prove things whenever you program!* How is that true? Think about some random datatype such as `u64`. Any time you construct a value of `u64`, you're *proving* that some `u64` exists, or that the `u64` type is *inhabited* (the academic term). The mere act of providing a value of `u64` that actually typechecks as a `u64` very directly proves that it's possible to do so. Put another way, we can say that every concrete `u64` value provides *evidence* of `u64`, evidence that proves the "proposition" `u64`. It's a very different way of looking at what a datatype means, but it's true! The only problem with a proof of `u64` is that it isn't a very "interesting" or "useful" piece of evidence: but it's a piece of evidence nonetheless. In the same way, when you define a function, you're creating a *proof* that the input types of the function can somehow be transformed into the output type of the function. For example this function: ``` fn n_equals_zero n: u8; return n == 0 ``` has type `u8 -> bool`, so the function *proves* that if we're given a `u8` we can always produce a `bool`. In this way the `->` represents *both* the real computation that will happen *and* the implication operator `P -> Q`! The reason implication and functions are equivalent is exactly because datatypes and propositions are equivalent. Think of this example: - the implication `P -> Q` has been proven - so if `P` can be proven - then `Q` can also be proven TODO truth table To convert this into the language of types and programs, we just have to change "implication" to "function", "proven" to "constructed", and `P` and `Q` to some types: - the function `u8 -> bool` has been constructed - so if `u8` can be constructed - then `bool` can also be constructed Pretty cool huh! This simple idea is called the [Curry-Howard Correspondence](https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspondence), named after the researchers who discovered it. This is the main idea that allows programs to literally represent proofs. The only problem with `u8 -> bool` is that, yet again, it isn't a proof of anything very interesting! The type of this function doesn't actually enforce that the `bool` is even *related* to the `u8` we were given. All these other functions also have the type `u8 -> bool` and yet do completely different things! ``` fn always_true _: u8; return true fn always_false _: u8; return false fn n_equals_nine n: u8; return n == 9 ``` The simple type of `bool` only gives us information about one value, and can't encode *relationships between* values. But if we enhance our language with *dependent types*, we can start doing really interesting stuff. Let's start with a function whose *type* proves if its input is equal to 5. We've already introduced asserted types, so let's define our own type to represent that idea (this isn't the right way to do this, we'll improve it in a second). Let's also write a function that uses it. It's even possible to define types that *can't possibly* be constructed, such as an empty union: `type Empty; |`. When you try to actually create a value of `Empty`, you can't possibly do so, meaning that this type is impossible or "False". But what about this definition of `True`? TODO --- Representing propositions as datatypes and theorems as functions means that we don't have to *run* the code to "compute" the truth value of variables. We only have to *type check* the code to make sure the types are all consistent with each other. If the type of function asserts that it can change input propositions into some output proposition, and the body of the function successfully typechecks by demonstrating steps that do in fact break a piece of propositional data apart and transform it to create a piece of propositional data with a different type, then the very existence of that function proves that the input propositions all *imply* the output proposition. ### Proofs using Tactics ## Verified Compute Magmide ### Separation Logic https://cacm.acm.org/magazines/2019/2/234356-separation-logic/fulltext ### Separation Logic in Use ### Logical Modeling ### Testing and Conjecture Checking ### Trackable Effects --- ## Basics of Compute Magmide First let's get all the stuff that isn't unique about Magmide out of the way. Magmide has both a "Computational" language and a "Logical" language baked in. The Computational language is what we use to write programs that will actually run on some machine, and the Logical language is used to logically model data and prove things about programs. The Computational language is technically defined as a raw assembly language, but since programming in assembly is extremely tedious, the default way to write computational code is with `core`, a language a lot like Rust. ```magmide // single line comments use double-slash // comments can be indented so you can write them across as many lines as you like! `core` is whitespace sensitive, so indentation is used to structure the program // the main function is the entry point to your program fn main; // immutable variables are declared with let: let a = 1 // a = 2 <-- compiler error! // variables can be redeclared: let a = 2 // types are inferred, but you can provide them: let b: u8 = 255 // mutable variables are declared with mut: mut c = 2 c = 1 // and they can be redeclared: mut c = 3 let c = 2 // booleans let b: bool = true // standard signed and unsigned integers let unsigned_8_bits: u8 = 0 let signed_8_bits: i8 = 0 // floating point numbers let float_32_bits: f32 = 0.0 let float_64_bits: f64 = 0.0 // arrays // slices // tuples // core is literally just a "portable assembly language", // so it doesn't have growable lists or strings by default! // think of core in the same way as `no_std` in rust // we hope rust itself will someday be reimplemented and formally verified using magmide! // the type system is very similar to rust // you can declare type aliases: alias byte; u8 // you can declare new nominal types as structs: data User; id: usize age: u8 active: bool data Point; x: f64, y: f64 // or unit data Token data Signal // or tuples data Point; f64, f64 data Pair; i32 i32 // or discriminated unions data Event; | PageLoad | PageUnload | KeyPress; char | Paste; [char] | Click; x: i64, y: i64 // on which you can use the match command fn use_union event: Event; match event; PageLoad; PageUnload; Click x, y; _; () // the is operator is like "if let" in rust if event is KeyPress character; print character // and you can use it without destructuring? if event is Paste; print event.1 // and they can be generic data Pair T, U; T, U data NonEmpty T; first: T rest: [T] ``` But the entire point of Magmide is that you can prove your program is correct! How do we do that? The most common and simplest way we can make provable assertions about our programs is by making our types *asserted types*. If we want to guarantee that a piece of data will always meet some criteria, we can make assertions about it with the `&` operator. Then, any time we assign a value to that type, we have to fulfill a *proof obligation* that the value meets all the assertions of the type. More on proofs in a second. ``` // this type will just be represented as a normal usize // but we can't assign 0 to it alias NonZero; usize & > 0 // we can add as many assertions as we like // even using generic values in them alias Between min max; usize & >= min & <= max // the & operator essentially takes a single argument function, // so we can use the lambda operator alias Above min; usize & |> v; v > min // or we can use the "hole" _ to indicate the value in question alias Above min; usize & _ > min // this works for tuples and structs too // "^" can be used to refer to the parent datatype // so fields of a type can refer to other fields alias Range; i32, i32 & > ^.0 data Person; age: u8 // the value of is_adult // has to line up with .age >= 18 is_adult: bool & == ^.age >= 18 // this pattern of requiring a bool // to exactly line up with some assertion // is common enough to get an alias is_adult: bool.exact (^.age >= 18) ``` So how do we actually prove that our program actually follows these data assertions? Can the compiler figure it out by itself? In many simple situations it actually can! But it's literally impossible for it to do so in absolutely all situations (if it could, the compiler would be capable of solving any logical proof in the universe!). To really understand how this all works, we have to get into the Logical side of Magmide, and talk about `Ideal`, `Prop`, and The Calculus of Constructions. ## Logical Magmide First let's start with the `Ideal` type family. `Ideal` types are defined to represent *abstract*, *logical* data. They aren't intended to be encoded by real computers, and their only purpose is to help us define logical concepts and prove things about them. To go with the `Ideal` type family is a whole separate programming language, one that's *pure* and *functional*. Why pure and functional? Simply, pure and functional languages relate directly to mathematical type theory (mathematical type theory is nothing but a pure and functional language!). It's much easier to define abstract concepts and prove things about them in pure and functional settings than the messy imperative way real computers work. Otherwise we'd have to deal with distracting details about memory layout, bit representation, allocation, etc. The "programs" we write in this pure functional language aren't actually intended to be run! They just define abstract algorithms, so we only care about them for their type-checking behavior and not their real behavior. The type system of logical Magmide is shaped a lot like computational Magmide to make things convenient. But the big difference between types in logical Magmide and computational magmide is how they handle type recursion. ``` // in computational Magmide types must be representable in bits and have a finite and knowable size, // meaning they can't reference themselves without some kind of pointer indirection data NumLinkedList T; item: T next: *(NumLinkedList T)? // but in logical magmide there's no such restriction, since these types are abstract and imaginary idl List T; item: T next: List T ``` Logical Magmide only really needs three things to prove basically all of mathematics and therefore model computation and prove programs correct: - Inductive types, which live in one of two different type "sorts": - `Ideal`, the sort for "data" (even if it's abstract imaginary data). - `Prop`, the sort for "propositions", basically assertions about data. - Function types. --- ``` data equal_5 number; | yes & (number == 5) | no & (number != 5) fn is_equal_5 number: u8 -> equal_5 number; if number == 5; return yes else; return no ``` Pretty simple! The ability to *dependently* reference the input `number` in the output type makes this work. And in this case, because the assertions we're making are so simple, the compiler is able to prove they're consistent without any extra work from us. The compiler would complain if we tried to create a function that *wasn't* obviously consistent: ``` fn is_equal_5 number: u8 -> equal_5 number; if number == 6; return yes else; return no // compiler error! ``` But we don't have to define our own `equal_5` type, we can just use `bool`, which can already generically accept assertions. The same is also true for a few other standard library types like `Optional` and `Fallible` that are commonly used to assert something about data. ``` // bool can take a true assertion and a false assertion fn is_equal_5 number: u8 -> bool (number == 5) (number != 5); return number == 5 // we can also use the alias for this concept fn is_equal_5 number: u8 -> bool.exact (number == 5); return number == 5 ``` But something about the above assertions like `number == 5` and `number != 5` might bother you. If proofs are *just data*, then where do `==` and `!=` come from? Are they primitive in the language? Or are they just data as well? They are actually just data! But specifically, they're data that live in the `Prop` sort. `Prop` is the sort defined to hold logical assertions, and the rules about how it can be used make it suited for that task. Let's define a few of our own `Prop` types to get a feel for how it works. ``` // in the computational types, unit or `()` is the "zero size" type, // a type that holds no data alias UnitAlias; () data MyUnit // we can define a prop version as well! prop PropUnit // this type is "trivial", since it holds no information and can always be constructed // in the standard library this type is called "always", and in Coq it's called "True" alias AlwaysAlias; always // we already saw an example of a computational type that can't be constructed data Empty; | // and of course we can do the same in the prop sort prop PropEmpty; | // this type is impossible to construct, which might seem pointless at first, but we'll see how it can be extremely useful later // in the standard library it's called "never", and in Coq it's called "False" alias NeverAlias; never ``` Okay we have prop types representing either trivial propositions or impossible ones. Now let's define ones to encode the ideas of logical "or", "and", "not", "exists", "forall", and equality. ``` // logical "and", the proposition that asserts the truth of two child propositions, // is just a tuple! a tuple that holds the two child propositions as data elements // we have to present a proof of both propositions in order to prove their "conjunction", // which is the academic term for "and" prop MyAnd P: prop, Q: prop; P, Q // then we use this constructor just like any other def true_and_true: MyAnd always always = MyAnd always always // we could of course also structure it as a record, // but the names aren't really useful (which is why we invented tuples right?) prop MyAnd P: prop, Q: prop; left: P right: Q // logical "or", the proposition that asserts the truth of either one child proposition or another, // is just a union! // we only have to present a proof for one of the propositions in order to prove their "disjunction", // which is the academic term for "or" prop MyOr P: prop, Q: prop; | left; P | right; Q def true_or_true_left: MyOr always always = MyOr.left always def true_or_true_right: MyOr always always = MyOr.right always // logical "not" is a little more interesting. what's the best way to assert some proposition *isn't* true? should we say it's equal to "never"? that doesn't really make sense, since you'll see in a moment that "equality" is just an idea we're going to define ourselves. instead we just want to prove that this proposition behaves the same way as "false", in the way that it's impossible to actually construct a value of it. // The most elegant way is to say that if you *were* able to construct a value of this proposition, we would *also* be able to construct a value of "false"! So "not" is just a function that transforms some proposition value into "false". // notice that we don't need to create a new type for this, since MyNot is just a function alias MyNot P: prop; P -> False ``` Equality is interesting. Depending on exactly what you're doing, you could define what it means for things to be "equal": - Two byte-encoded values are equal if their bitwise `xor` is equal to 0. - Two values are equal if any function you pass them to will behave exactly the same with either one. - A value is only equal with exactly itself. In Logical magmide, since all values are "ideal" and not intended to actually ever exist, the simplest definition is actually that last one: a value is only equal with exactly itself. ``` prop MyEquality {T: Ideal} -- T, T; @t -> MyEquality t t ``` Logical "forall" is the most interesting one, since it's the only one that's actually defined as a primitive in the language. We've actually been using it already! You might be surprised to learn that the function arrow `->` is just the same as "forall"! It's just a looser version of it. In type theory, if we want to provide a proof that "forall objects in some set, some property holds", we just have to provide a *function* that takes as input one of those objects and returns a proof (which is just a piece of data) that it has that property. And of course it can take more than one input, any of which can be proof objects themselves. So how do you actually write a "forall" type? Since `forall` is such an important concept, its magmide syntax very concisely uses `@`. Here's how you would write the type for the `is_equal_5` function we wrote earlier: `@ number: u8 -> bool.exact number == 5`. I prefer to read this as: "given any `number` which is a `u8`, I'll give you a `bool` which exactly equals whether `number` is equal to 5". For functions that take multiple "forall" variables as inputs (the academic term for accepting a forall variable as input is to "universally quantify" the variable, since a forall assertion proves something universal), you use commas instead of arrows between them: `@ n1, n2 -> bool.exact n1 == n2`. Very importantly, in order for that function to *really* prove the assertion, it has to be infallible (it can't fail or crash on any input) and provably terminating (it can't infinitely loop on any input). It is allowed to require things about the input (for example a function can be written to only accept even integers rather than all integers), but it has to handle every value it makes an assertion about. ``` // since we're providing default assertions, // the normal form of bool only asserts the useless `always` type bool when_true = always, when_false = always; | true & when_true | false & when_false // you'll notice we don't bother with an assertion for T // since the user can just provide an asserted type themselves type Optional T, when_none = always; | Some; T | None & when_none // same thing here type Fallible T, E; | Ok; T | Err; E ``` Understanding indexed types In a language with normal generics, if there are multiple functions or constructors in the type that all use a generic variable then when those functions or constructors are actually used all the instances of those generic variables to have to be equal. You can get that exact same behavior in magmide, but you can *also* get more "dependent" versions of generic variables which are allowed to be different. This is useful in many situations, but it's best to start with two examples. normal polymorphic lists, to understand the normal situation, and how it would be annoying or inconsistent to allow different values of the generic variable. forcing them on the left side basically allows us to elide any mention of them and still keep the requirement of them aligning length indexed lists attestations of zero or one-ness even numbers in the case of the `even` predicate, the different constructors all provide evidence of the same proposition `even`, but they do so for different numbers. The key insight is in understanding *The Curry-Howard Correspondence* and the concept of *Proofs as Data*. These are a little mind-bending at first, but let's go through it. A good way to understand this is to see *normal* programming in a different light. Basically, any type that lives in the `Prop` sort is *by definition* a type that represents a truth value, a logical claim. Proofs are *all* defined by constructors for data, it's just data that lives in a special type sort, specifically the type sort `Prop`. First we define some type that defines data representing some logical claim, and then when we want to actually *prove* such a claim, we just have to *construct* a value of that type! It's important to notice though that this wouldn't be very useful if the type system of our language wasn't *dependent*, meaning that the type of one value can refer to any other separate value in scope. When we put propositions-as-data together with dependent types, we have a proof checker. This is the key insight. When we make any kind of logical claim, we have to define *out of nowhere* what the definition of that claim is. --- Theorem proving In normal programming, we usually follow a pattern of writing out code, then checking it, and then filling in the gaps, and we don't really need any kind of truly interactive system to help us as we go. But writing proofs, even though it's technically the exact same as just writing code, isn't as easy to do in that way. When we're trying to solve a proof it's difficult to keep in mind all the facts and variables we've assumed existed at that particular stage, and we often have to move forward step by step. This is why theorem proving is most often done with *interactive tactics*. Instead of writing out all or most of the code as we might for a purely computational function, we instead enter an interactive session with the compiler, where it shows us what we have available to us and what we have left to prove. In `magmide`, we do this using the `magmide tacticmode` command, or with a variety of editor plugins that use `tacticmode` under the hood. Say we want to prove something about numbers, maybe that addition of natural numbers is symmetric. First we write out our definition of that theorem: ``` thm addition_is_symmetric: @ (x, y): nat -> x == y <-> y == x; ``` then give the command `magmide tacticmode addition_is_symmetric` (or use the equivalent start command of our editor), and magmide will find that definition, parse and type check any definitions it depends on, and enter interactive tactic mode. It shows the *context*, or the variables this definition takes as inputs, and the *goal*, which is basically the type of the thing we're trying to prove. Remember, a theorem is a thing whose final output lives in the `Prop` sort, whether that be a piece of data that lives in `Prop` or a function that returns something in `Prop`. So when we "prove" a theorem we're really just constructing a piece of code! To really make clear that theorems are just code, let's actually write out a theorem manually! Or let's define a *computational* function using tactics. --- ## Abstract assembly language Here's an example of a Magmide program in the "core" assembly language that is general enough to compile to basically any architecture. Magmide itself ships with a few different layers of computational language: - Thinking about computation in general. - Thinking about a specific collection of generalizable instructions in an unknown machine. This means you're reasoning about specific calling conventions. - Thinking about blocks with "function arguments" - `asm_zero`: a truly representational abstract assembly language, used to define the "common core" of all supported architectures - `asm_one`: the next level of abstraction, with type definitions and sugars, llvm style functions along with call/ret instructions. must instantiate with a calling convention - `core`: a c-like language with normal functions, match/if/for/while/loop/piping structures, functions for malloc/free, but no ownership/lifetime stuff. must instantiate with a calling convention and definitions for malloc/free in the desired environment - `system_core`: same c-like language, but with assumptions of "system" calls for thread spawn/join, io, async, etc There can be a `call` instruction that takes a label or address and together with a `ret` instruction abstracts away the details of a calling convention. We assume it does whatever is necessary under the calling convention to set the return address and push arguments to wherever they go. https://cs61.seas.harvard.edu/site/2018/Asm1/ all labels and global static data are accessed relative to the current instruction pointer (or at least they should be to produce a safe position independent executable). so when assembling to machine code, the distance between an instruction accessing something and that thing is computed https://cs61.seas.harvard.edu/site/2018/Asm2/ A `push X` instruction pushes the variable X onto the stack, which changes the stack pointer (either up or down depending on the stack convention, and it will mostly do this the same amount since X must fit into a word??) and moves `X` into that location so `push` (`pop` is opposite) ``` add 8, %rsp // could be sub if stack grows downwards mov X, (%rsp) ``` ``` define i32 @add1(i32 %a, i32 %b) { entry: %tmp1 = add i32 %a, %b ret i32 %tmp1 } define i32 @add2(i32 %a, i32 %b) { entry: %tmp1 = icmp eq i32 %a, 0 br i1 %tmp1, label %done, label %recurse recurse: %tmp2 = sub i32 %a, 1 %tmp3 = add i32 %b, 1 %tmp4 = call i32 @add2(i32 %tmp2, i32 %tmp3) ret i32 %tmp4 done: ret i32 %b } ``` perhaps given all arguments then conclusion is a better syntax for implications, for readability --- types are algebraic because they follow algebra-like rules if False/never is 0 and True/always/unit is 1, then treating "tupling" like * and "unioning" like + has the same characteristics as with numbers anything tupled with never is equivalent to never anything unioned with never is equivalent to the original (since that arm of the union can't actually be constructed) anything tupled with always is equivalent to the original (the always adds no new information) anything unioned with always is just a thing with one more arm both tupling and unioning are "logically" commutative and associative (real code would have to access them differently, but that's the only difference, the information they hold is the same. it can be proven that any function taking them as input could always be transformed to a logically equivalent version) unioning is distributive over tupling (tupling something with a union is the same as tupling each arm of the union with that thing) (I think a partially commutative monoid with distributivity) ================================================ FILE: posts/iris-in-plain-terms.md ================================================ # A No Nonsense Introduction to the Iris Separation Logic The Iris Separation Logic is amazing, and I believe it's going to be the secret at the heart of the next generation of truly correct software. The only problem is that it isn't explained in a way *practicing software engineers* can understand. It's explained *extremely* well for programming language researchers (in fact unusually well), but it's still not quite good enough to cross the mainstream line: - There isn't a language engineers can actually *use* to play around with and gain an understanding of Iris, so it remains locked in papers. - The papers describing Iris gloss over math concepts like partial commutative monoids and extension orders and reflexivity. A full explanation of Iris would go over these things first or enough during. - The papers use difficult to parse and arbitrary symbology rather than helpful "graspable" names, so it's difficult to orient oneself while reading it. ================================================ FILE: posts/toward-termination-vcgen.md ================================================ we want to do this rather than simply think at a higher level of abstraction for two reasons: - we want the power and flexibility to prove termination of more exotic programs that can't be represented with simple functions (goto statements, algebraic effects, jumping to dynamic code, etc) - we need a foundation of reasoning *upon which to build those higher levels of abstraction!* the whole point of magmide is that we're extending our formal understanding as far down as we possibly can, so that we can combine upwards with confidence we haven't merely assumed something without cause. in order to prove an assembly language program terminates, we have to present two things: - some well-founded relation that we can somehow convert a machine state into (the relation could be directly over machine states) - a proof that the assembly step relation will, for our particular program, always make "progress" in the well-founded relation, by moving with each instruction into a machine state that is "less than" the previous one with regard to the relation A natural way to do this is by doing the following: - chop up our program into labeled blocks, much like the "basic blocks" of llvm (although we don't have to be as perfectly strict as those) - create a directed graph of the program with these labeled blocks as nodes, and possible jumps as edges - define a block that is intended to be the "starting" block of execution, and create an artificial "start" node representing any position which could in a well-formed way jump to our program, and create an edge from the start block to our real start block (an environment could choose to load our program and jump to some arbitrary instruction, but it makes sense to just require as a precondition of correct execution that a predefined starting instruction is used instead) (we likely want to simply declare some instruction as the defined starting point, and package this instruction together with the program to define the program as a broader "executable", or even just rearrange the program so that the desired starting instruction is always the very first one) - include an artificial "stop" node and create edges to it from all blocks that stop execution - find all the strongly connected components of the graph with an algorithm such as tarjan's. with the DAG of components, topographically order them according to their maximum distance from stop. this maximum distance number forms the first and highest index of our lexicographic ordering - now for each component we have to find a well-founded ordering. if each component is truly strongly connected (isn't just a non-recursive single node), then the programmer will likely have to provide a well-founded ordering, but we can go a little farther. in each component, we can do a similar exercise: - find the nodes that are actually jumped to from nodes outside the component, and create a new artificial "start" node that points to them - find the nodes that have jumps exiting the component, and create a new artificial "stop" node they point to - go through and find a maximum distance from stop for each node. this number won't necessarily create a topographical order, but it does create topographic *classes* we can use - go through each topographic class, and find out if the class itself has any strongly connected subdivisions. if there are any subdivisions within the class then we can create a topographic ordering within the class - every strongly connected component within a class needs to have a well-founded ordering provided by the programmer, and a proof that that ordering is decremented by the time execution leaves the class component *cleverly*, we only have to flag an edge for justification within the same distance class (that isn't a self-edge) if the distance class isn't itself strongly connected. also, it will *probably* be a better and more pleasant or correct experience if self-recursive nodes just get their own separate well-founded ordering and each individual recursive edge needs to be justified along that ordering. *MAYBE??* I guess if the work done by self-edges is related to the total progress of the whole component then they can just somehow incorporate the structures being progressed by self-edges into the structure being progressed by the component. what we're trying to do is fill in all the "obvious" portions. we want to only make them provide an *interesting* ordering that somehow relates to the semantics of their program, and then only make them justify steps in the program that really do truly need to respect that ordering. any tedious book-keeping we can do for them we should try to do things to watch out for: - "trapped jumps", any kind of jump that refers to the label of itself. it can be proven that if a program ever is in a state such that an instruction jumps to itself, the program will be permanently stuck on that instruction, and will never make any kind of useful progress in any well-founded relation. the machine state will never change, since even the program counter will remain the same lexicographic orderings have "higher priority" indices a program is a list of labeled sections we can go over that list and produce a directed graph of all instructions that go from one labeled section to another: - obviously branching instructions that go to a label count, even ones that go to the same labeled section since that's a recursive branch - any possibly sequential instructions at the *end* of a section go to the *next* section, so they also count from this graph, we can produce a list of strongly connected components, and the network of strongly connected components forms a DAG this DAG from the single starting instruction to all possible exit nodes (nodes that include an exit instruction) is well-founded, since we're decreasing the current maximum distance from an exit node. this forms the first and highest priority index in our total lexicographic order the case of non-recursive single-node components is trivial, since these aren't really strongly connected, and always first move sequentially through the section before always progressing along the DAG with this, we can prove termination if we're given a progress type/relation/function/proof for each component to narrow the instructions who need to be justified, we can look at each strongly connected component, and topographically order the nodes according to their maximum distance from an exit node (any node that exits the component) when they're ordered like this, we can imagine them as a DAG again by "severing" the "backwards" edges, ones that go toward a topographically lower node then we can supply a lexicographical ordering for this component by just push *their* decreasing type on the front of the same ordering we would produce for a *real* DAG. their supplied progress type will have the highest priority, since it represents the entire chunk of work the component is trying to do, and the rest of the ordering just handles all the boring book-keeping as we go along through this "severed" DAG. we give to them obligations that the "backwards" or recursive edges (or Steps) do in fact make progress. it will probably be necessary for sanity's sake to simply require a proof that the progress indicator gets decreased *sometime* before any backward edge or we need an even higher version of Steps, one that encodes program progression across section nodes rather than individual instructions. probably the final version requires us to prove that if a progression relation across section nodes is well-founded, then the underlying step progression is as well ```v forall (T: Type) (progress: T -> T -> Prop) (convert: MachineState -> T), well_founded progress forall cur next, Step cur next -> Within cur component -> Within next component -> progress (convert next) (convert cur) ``` so if we exit the segment, we've made progress within the segment we can just say we're making sequential progress? probably to prove a jump to dynamic code will terminate or just behave properly, we need to have the programmer provide a list of resumption locations in the current graph (which could be the exit location!) and prove the code they're jumping to will in fact only exit itself by going to those known places they also need to prove that somehow the unknown code has been itself checked for well-formedness and absence of unfulfilled proof obligations to justify jumping to it and still keeping clean trackable effects jumping to unchecked code violates *all* registered trackable effects, since the unchecked code could do literally anything it wants. ================================================ FILE: posts/what-is-magmide.md ================================================ ================================================ FILE: src/ast.rs ================================================ #[derive(Debug, Eq, PartialEq)] pub enum TypeBody { Unit, Union { branches: Vec }, // Tuple { }, // Record { fields: Vec<(String, String)> }, } // #[derive(Debug, Eq, PartialEq)] // pub enum Statement { // // Use(UseTree), // Let(LetStatement), // Debug(DebugStatement), // Named(ModuleItem), // } pub type Statement = Term; #[derive(Debug, Eq, PartialEq)] pub struct TypeDefinition { pub name: String, pub body: TypeBody, } #[derive(Debug, Eq, PartialEq)] pub struct ProcedureDefinition { pub name: String, pub parameters: Vec<(String, String)>, pub return_type: String, pub statements: Vec, } // #[derive(Debug)] // pub enum Statement { // Bare(Term), // Let(), // // Module // } // TODO // pub type Statement = Term; #[derive(Debug, Eq, PartialEq)] pub struct LetStatement { // pub pattern: Pattern, pub pattern: String, pub type_declaration: Option, pub term: Term, } #[derive(Debug, Eq, PartialEq)] pub struct DebugStatement { pub term: Term, } #[derive(Debug, Eq, PartialEq)] pub enum ModuleItem { Type(TypeDefinition), Procedure(ProcedureDefinition), Debug(DebugStatement), } #[derive(Debug, Eq, PartialEq)] pub enum Term { Lone(String), Chain(String, Vec), Match { discriminant: Box, arms: Vec, }, } #[derive(Debug, Eq, PartialEq)] pub struct MatchArm { // pattern: Pattern, pub pattern: Term, // statements: Vec, pub statement: Term, } #[derive(Debug, Eq, PartialEq)] pub enum ChainItem { Access(String), Call { arguments: Vec }, // // IndexCall { arguments: Vec }, // // TODO yikes? using a complex term to return a function that's called freestanding? // FreeCall { target: Term, arguments: Vec }, // // tapping is only useful for debugging, and should be understood as provably not changing the current type // CatchCall { parameters: Either>, statements: Vec, is_tap: bool }, // ChainedMatch { return_type: Term, arms: Vec }, } ================================================ FILE: src/checker.rs ================================================ // http://adam.chlipala.net/cpdt/html/Universes.html use std::collections::{HashMap}; use crate::ast::*; pub type Ident = String; // a simple identifier can refer either to some global item with a path like a type or function (types and functions defined inside a block of statements are similar to this, but don't have a "path" in the strict sense since they aren't accessible from the outside) // or a mere local variable #[derive(Debug, Clone)] pub struct Scope<'a> { scope: HashMap<&'a Ident, ScopeItem<'a>>, } // TODO should probably manually implement PartialEq in terms of pointer equality (std::ptr::eq) #[derive(Debug, PartialEq, Clone)] pub enum ScopeItem<'a> { // Module(Module), Type(&'a TypeDefinition), Procedure(&'a ProcedureDefinition), // Prop(PropDefinition), // Theorem(TheoremDefinition), // Local(Term), Data(Data), Placeholder, } trait DebugDisplay { fn debug_display(&self) -> String; } impl<'a> DebugDisplay for ScopeItem<'a> { fn debug_display(&self) -> String { match self { ScopeItem::Type(type_definition) => type_definition.name.clone(), ScopeItem::Procedure(procedure_definition) => procedure_definition.name.clone(), // ScopeItem::Prop(PropDefinition), // ScopeItem::Theorem(TheoremDefinition), // ScopeItem::Local(Term), ScopeItem::Data(data) => format!("{}.{}{}", data.type_path, data.name, data.body.debug_display()), ScopeItem::Placeholder => unimplemented!(), } } } #[derive(Debug, PartialEq, Clone)] pub struct Data { pub name: String, pub type_path: String, pub body: Body, } #[derive(Debug, PartialEq, Clone)] pub enum Body { NotConstructed, Unit, // Tuple, // Record, } impl DebugDisplay for Body { fn debug_display(&self) -> String { "".into() } } impl<'a> Scope<'a> { pub fn new() -> (Scope<'a>, Ctx) { (Scope{ scope: HashMap::new() }, Ctx{ errors: Vec::new(), debug_trace: Vec::new() }) } pub fn from(pairs: [(&'a Ident, ScopeItem<'a>); N]) -> (Scope<'a>, Ctx) { (Scope{ scope: HashMap::from(pairs) }, Ctx{ errors: Vec::new(), debug_trace: Vec::new() }) } pub fn checked_insert(&mut self, ctx: &mut Ctx, ident: &'a Ident, scope_item: ScopeItem<'a>) -> Option<()> { match self.scope.insert(ident, scope_item) { Some(_) => { ctx.add_error(format!("name {ident} has already been used")); None }, None => Some(()), } } pub fn type_check_module(&mut self, ctx: &mut Ctx, module_items: &'a Vec) { for module_item in module_items { self.name_pass_type_check_module_item(ctx, module_item); } for module_item in module_items { self.main_pass_type_check_module_item(ctx, module_item); } } pub fn name_pass_type_check_module_item(&mut self, ctx: &mut Ctx, module_item: &'a ModuleItem) { match module_item { ModuleItem::Type(type_definition) => { self.checked_insert(ctx, &type_definition.name, ScopeItem::Type(type_definition)); }, ModuleItem::Procedure(procedure_definition) => { self.checked_insert(ctx, &procedure_definition.name, ScopeItem::Procedure(procedure_definition)); }, ModuleItem::Debug(_) => {}, } } pub fn main_pass_type_check_module_item(&self, ctx: &mut Ctx, module_item: &ModuleItem) { match module_item { ModuleItem::Type(type_definition) => { self.type_check_type_definition(ctx, type_definition); }, ModuleItem::Procedure(procedure_definition) => { self.type_check_procedure_definition(ctx, procedure_definition); }, ModuleItem::Debug(debug_statement) => { self.type_check_debug_statement(ctx, debug_statement); }, } } pub fn type_check_type_definition(&self, _ctx: &mut Ctx, type_definition: &TypeDefinition) { match &type_definition.body { TypeBody::Unit => {}, TypeBody::Union { branches } => { for _branch in branches { // TODO these are just strings now, but there will be work to do soon } }, } } pub fn type_check_procedure_definition(&self, ctx: &mut Ctx, procedure_definition: &'a ProcedureDefinition) { let mut procedure_scope = self.clone(); procedure_scope.type_check_parameters(ctx, &procedure_definition.parameters); let statements_type = procedure_scope.type_check_statements(ctx, &procedure_definition.statements); let return_type = self.checked_get(ctx, &procedure_definition.return_type); match (statements_type, return_type) { (Some(statements_type), Some(return_type)) => { self.checked_assignable_to(ctx, &statements_type, return_type); }, _ => {}, } } pub fn type_check_parameters(&mut self, ctx: &mut Ctx, parameters: &'a Vec<(String, String)>) { for (param_name, param_type) in parameters { let param_type = self.checked_get(ctx, param_type).unwrap_or(&ScopeItem::Placeholder).clone(); self.checked_insert(ctx, param_name, param_type); } } pub fn type_check_statements(&mut self, ctx: &mut Ctx, statements: &Vec) -> Option> { let mut statements_type = None; for statement in statements { statements_type = self.type_check_term(ctx, statement); } statements_type } pub fn reduce_statements(&mut self, ctx: &mut Ctx, statements: &Vec) -> Option> { let mut current_item = None; for statement in statements { current_item = Some(self.reduce_term(ctx, statement)?); } current_item } pub fn checked_assignable_to(&self, ctx: &mut Ctx, proposed: &ScopeItem<'a>, demanded: &ScopeItem<'a>) { // TODO this will be more complicated, but for now it's just equality if *proposed == ScopeItem::Placeholder || *demanded == ScopeItem::Placeholder { return; } if proposed != demanded { ctx.add_error(format!("{} isn't assignable to {}", proposed.debug_display(), demanded.debug_display())); } } pub fn type_check_debug_statement(&self, ctx: &mut Ctx, debug_statement: &DebugStatement) -> Option<()> { self.type_check_term(ctx, &debug_statement.term)?; // TODO this probably actually deserves an unwrap/panic, reduction should always work after type checking // let item = self.reduce_term(ctx, &debug_statement.term)?; // ctx.debug_trace.push(format!("{:?}", item)); Some(()) } pub fn type_check_term(&self, ctx: &mut Ctx, term: &Term) -> Option> { match term { Term::Lone(ident) => { Some(self.checked_get(ctx, ident)?.clone()) }, Term::Chain(first, chain_items) => { let mut current_type = self.checked_get(ctx, first)?.clone(); for chain_item in chain_items { match chain_item { ChainItem::Access(path) => { current_type = self.type_check_access_path(ctx, ¤t_type, path)?; }, ChainItem::Call { arguments } => { let argument_types = arguments.iter().map(|arg| self.type_check_term(ctx, arg)).collect::>()?; current_type = self.type_check_call(ctx, ¤t_type, argument_types)?; }, } } Some(current_type) }, Term::Match { discriminant, arms } => { let discriminant_type = self.type_check_term(ctx, discriminant)?; // make sure discriminant_type is always assignable to the patterns, // and that all the arms have the same inferred type, which is difficult because it's more about universal assignability let mut arm_types = Vec::new(); for MatchArm { pattern, statement } in arms { self.type_check_pattern_matches(ctx, pattern, &discriminant_type); let arm_type = self.type_check_term(ctx, statement); arm_types.push(arm_type); } // TODO also have to make sure all branches are covered by the arms let arm_types: Vec<_> = arm_types.into_iter().collect::>()?; // if there aren't any arm_types then either arms were ill-typed or there aren't any arms, // which either means // - the arms don't cover the type and *that* will be an error // - the type is empty, in which case TODO what should we do here? there won't be any errors, because we won't check for assignability to anything, but we also won't have a type to return // honestly this entire function will just change in the future, since we'll probably accept some suggested inferred type or something // which means that matches over empty types will just be allowed to infer to whatever was inferred from the outside of this function let (first_arm_type, rest_arm_types) = arm_types.split_first()?; for rest_arm_type in rest_arm_types { self.checked_assignable_to(ctx, rest_arm_type, first_arm_type); } Some(first_arm_type.clone()) }, } } pub fn reduce_term(&self, ctx: &mut Ctx, term: &Term) -> Option> { match term { Term::Lone(ident) => { Some(self.checked_get(ctx, ident)?.clone()) }, Term::Chain(first, chain_items) => { let mut current_item = self.checked_get(ctx, first)?.clone(); for chain_item in chain_items { match chain_item { ChainItem::Access(path) => { current_item = self.checked_access_path(ctx, ¤t_item, path)?; }, ChainItem::Call { arguments } => { let arguments = arguments.iter().map(|arg| self.reduce_term(ctx, arg)).collect::>()?; current_item = self.checked_call(ctx, ¤t_item, arguments)?; }, } } Some(current_item) }, Term::Match { discriminant, arms } => { let discriminant = self.reduce_term(ctx, discriminant)?; for MatchArm { pattern, statement } in arms { if self.test_pattern_matches(ctx, pattern, &discriminant) { return self.reduce_term(ctx, statement); } } None }, } } pub fn type_check_pattern_matches(&self, ctx: &mut Ctx, pattern: &Term, discriminant_type: &ScopeItem) { // this pattern matching code will be one of the most complicated parts of the entire language // for now, we're just going to go through the arms and reduce everything and check equality match pattern { Term::Lone(s) if s == "_" => {}, pattern => { if let Some(pattern) = self.type_check_term(ctx, pattern) { if pattern != *discriminant_type { ctx.add_error(format!("{} isn't a match for {}", pattern.debug_display(), discriminant_type.debug_display())); } } }, } } pub fn test_pattern_matches(&self, ctx: &mut Ctx, pattern: &Term, discriminant: &ScopeItem) -> bool { // this pattern matching code will be one of the most complicated parts of the entire language // for now, we're just going to go through the arms and reduce everything and check equality match pattern { Term::Lone(s) if s == "_" => true, pattern => { if let Some(pattern) = self.reduce_term(ctx, pattern) { pattern == *discriminant } else { false } }, } } pub fn type_check_call(&self, ctx: &mut Ctx, item: &ScopeItem<'a>, argument_types: Vec>) -> Option> { match item { ScopeItem::Type(type_definition) => { // TODO this isn't accurate if the type is a tuple variant ctx.add_error(format!("type {} is a type, not a callable", type_definition.name)); None }, ScopeItem::Procedure(procedure_definition) => { // TODO this is one of those situations where incremental compilation will be helpful // in the future, we won't in any way recheck this procedure definition, // we'll just query for its already checked information and compare that against the arguments // that already checked information can potentially be poisoned/placeholder, so we'll just not bother let return_type = self.placeholder_get(&procedure_definition.return_type); let num_arguments = argument_types.len(); let num_params = procedure_definition.parameters.len(); if num_arguments != num_params { ctx.add_error(format!("{} takes {} parameters but this call gave {}", procedure_definition.name, num_params, num_arguments)); } else { for (arg_type, (_, param_type)) in std::iter::zip(argument_types.into_iter(), procedure_definition.parameters.iter()) { let param_type = self.placeholder_get(param_type); self.checked_assignable_to(ctx, &arg_type, param_type); } } Some(return_type.clone()) }, // TODO look up original type definition, and use to build a body of type tuple ScopeItem::Data(_data) => unimplemented!(), ScopeItem::Placeholder => None, } } pub fn checked_call(&self, ctx: &mut Ctx, item: &ScopeItem<'a>, arguments: Vec>) -> Option> { match item { ScopeItem::Type(_) => None, ScopeItem::Procedure(procedure_definition) => { let mut call_scope: Scope<'a> = self.clone(); for (arg, (param_name, _)) in std::iter::zip(arguments.into_iter(), procedure_definition.parameters.iter()) { call_scope.checked_insert(ctx, ¶m_name, arg)?; } call_scope.reduce_statements(ctx, &procedure_definition.statements) }, // TODO look up original type definition, and use to build a body of type tuple ScopeItem::Data(_data) => unimplemented!(), ScopeItem::Placeholder => None, } } pub fn type_check_access_path(&self, ctx: &mut Ctx, item: &ScopeItem<'a>, path: &Ident) -> Option> { match item { ScopeItem::Type(type_definition) => { // look through the type_definition to see if it has a constructor with this name match &type_definition.body { TypeBody::Unit => { ctx.add_error(format!("can't access property {path} on unit type {}", type_definition.name)); None }, TypeBody::Union{ branches } => { branches.iter().find(|b| b == &path)?; Some(ScopeItem::Type(type_definition)) }, } }, ScopeItem::Procedure(procedure_definition) => { ctx.add_error(format!("can't access property {path} on procedure {}", procedure_definition.name)); None }, ScopeItem::Data(_) => unimplemented!(), ScopeItem::Placeholder => None, } } pub fn checked_access_path(&self, _ctx: &mut Ctx, item: &ScopeItem<'a>, path: &Ident) -> Option> { match item { ScopeItem::Type(type_definition) => { match &type_definition.body { TypeBody::Unit => None, TypeBody::Union{ branches } => { let branch = branches.iter().find(|b| b == &path)?; // TODO body will get more complex as the nature of a branch gets more complex Some(ScopeItem::Data(Data{ type_path: type_definition.name.clone(), name: branch.clone(), body: Body::Unit })) }, } }, ScopeItem::Procedure(_) => None, ScopeItem::Placeholder => None, data => Some(data.clone()), } } pub fn placeholder_get<'s>(&'s self, ident: &Ident) -> &'s ScopeItem<'a> { self.scope.get(ident).unwrap_or(&ScopeItem::Placeholder) } pub fn checked_get<'s>(&'s self, ctx: &mut Ctx, ident: &Ident) -> Option<&'s ScopeItem<'a>> { match self.scope.get(ident) { None => { ctx.add_error(format!("{ident} can't be found")); None }, item => item, } } // pub fn checked_exists(arg: Type) -> RetType { // unimplemented!() // } } #[derive(Debug)] pub struct Ctx { pub errors: Vec, pub debug_trace: Vec, } impl Ctx { pub fn add_error(&mut self, error: String) { self.errors.push(error); } pub fn add_debug(&mut self, debug: String) { self.debug_trace.push(debug); } } #[cfg(test)] mod tests { use super::*; use crate::parser; #[test] fn basic_type_errors() { let i = r#" type Day; | Monday | Tuesday | Wednesday | Thursday | Friday | Saturday | Sunday type Bool; | True | False "#; let (remaining, module_items) = parser::parse_file_with_indentation(3, i).unwrap(); assert_eq!(remaining, ""); let (mut scope, mut ctx) = Scope::new(); scope.type_check_module(&mut ctx, &module_items); assert!(ctx.errors.is_empty()); for (input, expected_errors) in [ (r#" proc same_day(d: Nope): Day; d "#, vec!["Nope can't be found"]), (r#" proc same_day(d: Day): Nope; d "#, vec!["Nope can't be found"]), (r#" proc same_day(a: Day): Day; d "#, vec!["d can't be found"]), (r#" proc same_day(d: Day): Day; a "#, vec!["a can't be found"]), (r#" proc same_day(): Day; d "#, vec!["d can't be found"]), (r#" proc same_day(d: Day, d: Day): Day; d "#, vec!["name d has already been used"]), (r#" proc same_day(d: Day): Day; d proc same_day(d: Day): Day; d "#, vec!["name same_day has already been used"]), (r#" proc same_day(d: Day, b: Bool): Day; b "#, vec!["Bool isn't assignable to Day"]), (r#" proc same_day(d: Day, b: Bool): Bool; d "#, vec!["Day isn't assignable to Bool"]), (r#" proc same_day(d: Day, b: Bool): Bool; match d; Day.Monday => Day.Monday _ => Day.Monday "#, vec!["Day isn't assignable to Bool"]), (r#" proc same_day(d: Day, b: Bool): Day; match d; Bool.True => Day.Monday _ => Day.Monday "#, vec!["Bool isn't a match for Day"]), (r#" proc same_day(d: Day, b: Bool): Day; match b; Day.Monday => Day.Monday _ => Day.Tuesday "#, vec!["Day isn't a match for Bool"]), (r#" proc bool_negate(b: Bool): Day; match b; Bool.True => Bool.False Bool.False => Bool.True "#, vec!["Bool isn't assignable to Day"]), ] { let (remaining, module_items) = parser::parse_file_with_indentation(4, input).unwrap(); assert_eq!(remaining, ""); let mut loop_scope = scope.clone(); loop_scope.type_check_module(&mut ctx, &module_items); assert_eq!(ctx.errors, expected_errors); ctx.errors.clear(); } } #[test] fn foundations_day_of_week() { let i = r#" type Day; | Monday | Tuesday | Wednesday | Thursday | Friday | Saturday | Sunday proc next_weekday(d: Day): Day; match d; Day.Monday => Day.Tuesday Day.Tuesday => Day.Wednesday Day.Wednesday => Day.Thursday Day.Thursday => Day.Friday _ => Day.Monday proc same_day(d: Day): Day; d "#; let (remaining, module_items) = parser::parse_file_with_indentation(3, i).unwrap(); assert_eq!(remaining, ""); let (mut scope, mut ctx) = Scope::new(); scope.type_check_module(&mut ctx, &module_items); assert!(ctx.errors.is_empty()); assert!(ctx.debug_trace.is_empty()); let term = parser::parse_expression(0, "Day").unwrap().1; assert_eq!(scope.reduce_term(&mut ctx, &term).unwrap(), *scope.scope.get(&"Day".to_string()).unwrap()); assert!(ctx.errors.is_empty()); fn make_day(day: &str) -> ScopeItem { ScopeItem::Data(Data{ name: day.into(), type_path: "Day".into(), body: Body::Unit }) } for (input, expected) in [ ("Day.Monday", "Monday"), ("same_day(Day.Monday)", "Monday"), ("next_weekday(Day.Friday)", "Monday"), ("next_weekday(next_weekday(Day.Friday))", "Tuesday"), ] { let term = parser::parse_expression(0, input).unwrap().1; assert_eq!(scope.reduce_term(&mut ctx, &term).unwrap(), make_day(expected)); assert!(ctx.errors.is_empty()); } let i = r#" prop Eq(@T: type): [T, T]; (t: T): [t, t] thm example_next_weekday: Eq[next_weekday(Day.Saturday), Day.Monday]; Eq(Day.Monday) thm example_next_weekday_cleaner: next_weekday(Day.Saturday) :Eq Day.Monday; _ thm example_next_weekday_sugar: next_weekday(Day.Saturday) == Day.Monday; _ "#; let (remaining, proof_items) = parser::parse_file_with_indentation(3, i).unwrap(); assert_eq!(remaining, ""); scope.type_check_module(&mut ctx, &proof_items); assert!(ctx.errors.is_empty()); assert!(ctx.debug_trace.is_empty()); // let i = r#" // debug next_weekday(Day.Friday) // debug next_weekday(next_weekday(Day.Saturday)) // "#; // let (remaining, debug_items) = parser::parse_file_with_indentation(3, i).unwrap(); // assert_eq!(remaining, ""); // scope.type_check_module(&mut ctx, &debug_items); // assert!(ctx.errors.is_empty()); // assert_eq!(ctx.debug_trace, vec!["Day.Monday", "Day.Tuesday"]); } } ================================================ FILE: src/lib.rs ================================================ mod parser; mod checker; mod ast; ================================================ FILE: src/main.rs ================================================ // use magmide::ast; // use magmide::parser; // use magmide::Database; // use magmide::checker; fn main() { // let path = PathBuf::from(r"/"); // // starting from the original source file, we walk the imports (which for now don't exist) and add source files as we go (which probably requires intelligent separation to make sure we can do import tracking without an incremental database present) // // or not? things like type checking are incredibly query-oriented, so it probably doesn't make sense // let db = magmide::Database::default(); // let source_file = magmide::SourceFile::new(&db, "bad".into(), 0, path); // magmide::tracked_parse_file(&db, source_file); // let diagnostics = magmide::tracked_parse_file::accumulated::(&db, source_file); // eprintln!("{diagnostics:?}"); } ================================================ FILE: src/old.md ================================================ ```rust #[derive(Debug, Eq, PartialEq, Clone)] pub struct ModuleItemBlock { pub line: usize, pub body: String, pub kind: ModuleItemBlockKind, } #[derive(Debug, Eq, PartialEq, Clone)] pub enum ModuleItemBlockKind { Procedure{ name: String }, Type{ name: String }, Debug, Error, } pub fn parse_module_item_blocks(indentation: usize, i: &str) -> Vec { let mut module_item_blocks = Vec::new(); let mut current_block_body = Vec::new(); let mut current_block_line = 0; let mut current_block_kind = None; fn close_block( module_item_blocks: &mut Vec, current_block_body: &mut Vec<&str>, line: usize, kind: ModuleItemBlockKind, ) { let body = current_block_body.join("\n"); current_block_body.clear(); module_item_blocks.push(ModuleItemBlock{ line, body, kind }); } for (index, body_line) in i.lines().enumerate() { let line = index + 1; if get_tab_count(body_line) != indentation || body_line.len() == 0 { current_block_body.push(body_line); continue; } let trimmed_body_line = body_line.trim_start(); if trimmed_body_line.starts_with(';') { current_block_body.push(body_line); if current_block_kind.is_none() { current_block_kind = Some(ModuleItemBlockKind::Error); } continue; } use ModuleItemBlockKind::*; let kind = alt(( map(preceded(tag("proc "), parse_ident), |name| Procedure{ name: name.to_string() }), map(preceded(tag("type "), parse_ident), |name| Type{ name: name.to_string() }), value(Debug, tag("debug ")), ))(trimmed_body_line) .map(|(_, kind)| kind) .unwrap_or(ModuleItemBlockKind::Error); let previous_kind = std::mem::replace(&mut current_block_kind, Some(kind)); if let Some(previous_kind) = previous_kind { close_block(&mut module_item_blocks, &mut current_block_body, current_block_line, previous_kind); current_block_line = line; } current_block_body.push(body_line); } close_block(&mut module_item_blocks, &mut current_block_body, current_block_line, current_block_kind.unwrap_or(ModuleItemBlockKind::Error)); module_item_blocks } fn make_block(line: usize, kind: ModuleItemBlockKind, body: &str) -> ModuleItemBlock { ModuleItemBlock{ line, body: body.into(), kind } } #[test] fn test_parse_module_item_blocks() { let i = r#" proc hello what here ; actual body is:: arbitary nested things yo sup type hey; sup type yoyo; | whatev | thing and stuff and whatever bad alsobad | thing | hmm sdfjdk dkfjdk debug hey debug sup big things poppin and such proc same_day(d: Day): Day; asdf "#; assert_eq!(parse_module_item_blocks(3, i), [ make_block(0, ModuleItemBlockKind::Procedure{ name: "hello".into() }, "\n\t\t\tproc hello\n\t\t\t\twhat\n\t\t\t\there\n\t\t\t;\n\t\t\t\tactual body\n\t\t\t\tis::\n\t\t\t\t\tarbitary\n\t\t\t\t\t\tnested\n\t\t\t\t\tthings\n\t\t\t\tyo\n\n\t\t\t\t\tsup\n"), make_block(15, ModuleItemBlockKind::Type{ name: "hey".into() }, "\t\t\ttype hey; sup"), make_block(16, ModuleItemBlockKind::Type{ name: "yoyo".into() }, "\t\t\ttype yoyo;\n\t\t\t\t| whatev\n\t\t\t\t| thing\n\t\t\t\t\tand stuff\n\t\t\t\t\t\tand whatever\n"), make_block(22, ModuleItemBlockKind::Error, "\t\t\tbad"), make_block(23, ModuleItemBlockKind::Error, "\t\t\talsobad\n\t\t\t\t| thing\n\t\t\t\t| hmm\n\t\t\t\tsdfjdk\n\t\t\t\t\tdkfjdk\n"), make_block(29, ModuleItemBlockKind::Debug, "\t\t\tdebug hey"), make_block(30, ModuleItemBlockKind::Debug, "\t\t\tdebug sup\n\t\t\t\tbig\n\t\t\t\tthings\n\t\t\t\t\tpoppin\n\t\t\t\tand such\n"), make_block(36, ModuleItemBlockKind::Procedure{ name: "same_day".into() }, "\t\t\tproc same_day(d: Day): Day; asdf\n\n\t\t"), ]); } ``` ================================================ FILE: src/parser.rs ================================================ // https://matklad.github.io/2023/05/21/resilient-ll-parsing-tutorial.html // https://github.com/rust-analyzer/rowan // https://github.com/salsa-rs/salsa // https://github.com/rust-bakery/nom // https://tfpk.github.io/nominomicon/chapter_1.html // https://crates.io/crates/nom-peg use nom::{ bytes::complete::{tag, take_while, take_while1}, character::{complete::{tab, newline, char as c}}, branch::alt, combinator::{value, opt, map, all_consuming}, multi::{count, many0, many1, separated_list0, separated_list1}, sequence::{preceded, delimited, separated_pair, terminated, tuple}, // Finish, IResult, }; use crate::ast::*; pub type DiscardingResult = Result>>; fn is_underscore(chr: char) -> bool { chr == '_' } fn is_ident(chr: char) -> bool { chr.is_ascii_alphanumeric() || is_underscore(chr) } fn is_start_ident(chr: char) -> bool { chr.is_ascii_alphabetic() || is_underscore(chr) } pub fn parse_ident(i: &str) -> IResult<&str, String> { let (i, first) = take_while1(is_start_ident)(i)?; let (i, rest) = take_while(is_ident)(i)?; Ok((i, format!("{}{}", first, rest))) } pub fn parse_branch(_: usize, i: &str) -> IResult<&str, String> { let (i, b) = preceded(tag("| "), parse_ident)(i)?; Ok((i, b.into())) } fn parse_indents(indentation: usize, i: &str) -> IResult<&str, Vec> { count(tab, indentation)(i) } fn indents(i: &str, indentation: usize) -> DiscardingResult<&str> { Ok(parse_indents(indentation, i)?.0) } fn parse_newlines(i: &str) -> IResult<&str, Vec> { many0(newline)(i) } fn newlines(i: &str) -> DiscardingResult<&str> { Ok(parse_newlines(i)?.0) } fn indented_line(indentation: usize, i: &str, line_parser: fn(usize, &str) -> IResult<&str, T>) -> IResult<&str, T> { let i = indents(i, indentation)?; line_parser(indentation, i) } fn indented_block(indentation: usize, i: &str, line_parser: fn(usize, &str) -> IResult<&str, T>) -> IResult<&str, Vec> { let indentation = indentation + 1; let i = newlines(i)?; let (i, items) = separated_list1(many1(newline), |i| indented_line(indentation, i, line_parser))(i)?; Ok((i, items)) } pub fn parse_file(i: &str) -> IResult<&str, Vec> { parse_file_with_indentation(0, i) } pub fn parse_file_with_indentation(indentation: usize, i: &str) -> IResult<&str, Vec> { all_consuming( terminated( |i| parse_module_items_with_indentation(indentation, i), tuple((parse_newlines, |i| parse_indents(usize::max(indentation - 1, 0), i), parse_newlines)), ) )(i) } pub fn parse_module_items(i: &str) -> IResult<&str, Vec> { parse_module_items_with_indentation(0, i) } pub fn parse_module_items_with_indentation(indentation: usize, i: &str) -> IResult<&str, Vec> { separated_list1(many1(newline), |i| parse_module_item(indentation, i))(i) } pub fn parse_module_item(indentation: usize, i: &str) -> IResult<&str, ModuleItem> { let i = newlines(i)?; let i = indents(i, indentation)?; alt(( map(|i| parse_type_definition(indentation, i), ModuleItem::Type), map(|i| parse_procedure_definition(indentation, i), ModuleItem::Procedure), map(|i| parse_debug_statement(indentation, i), ModuleItem::Debug), ))(i) } pub fn parse_type_definition(indentation: usize, i: &str) -> IResult<&str, TypeDefinition> { let (i, name) = preceded(tag("type "), parse_ident)(i)?; let here_branch = |i| indented_block(indentation, i, parse_branch); let (i, branches) = opt(preceded(c(';'), here_branch))(i)?; let body = match branches { Some(branches) => TypeBody::Union{ branches }, None => TypeBody::Unit, }; Ok((i, TypeDefinition{ name: name.into(), body })) } pub fn parse_procedure_definition(indentation: usize, i: &str) -> IResult<&str, ProcedureDefinition> { let (i, name) = preceded(tag("proc "), parse_ident)(i)?; let (i, parameters) = delimited(c('('), parse_parameters, c(')'))(i)?; let (i, return_type) = preceded(tag(": "), parse_ident)(i)?; let here_statement = |i| indented_block(indentation, i, parse_statement); let (i, statements) = preceded(c(';'), here_statement)(i)?; Ok((i, ProcedureDefinition{ name: name.into(), parameters, return_type, statements })) } pub fn parse_debug_statement(indentation: usize, i: &str) -> IResult<&str, DebugStatement> { let (i, term) = preceded(tag("debug "), |i| parse_term(indentation, i))(i)?; Ok((i, DebugStatement{ term })) } // fn parse_parameters(indentation: usize) -> impl Fn(&str) -> IResult<&str, ModuleItem> { pub fn parse_parameters(i: &str) -> IResult<&str, Vec<(String, String)>> { separated_list0(tag(", "), parse_parameter)(i) } pub fn parse_parameter(i: &str) -> IResult<&str, (String, String)> { separated_pair(parse_ident, tag(": "), parse_ident)(i) } pub fn parse_statement(indentation: usize, i: &str) -> IResult<&str, Term> { parse_term(indentation, i) } pub fn parse_term(indentation: usize, i: &str) -> IResult<&str, Term> { alt(( |i| parse_match(indentation, i), |i| parse_expression(indentation, i), ))(i) } pub fn parse_match(indentation: usize, i: &str) -> IResult<&str, Term> { // TODO need to figure out how to use the full parse_term for the discriminant let (i, discriminant) = delimited(tag("match "), |i| parse_expression(indentation, i), c(';'))(i)?; let (i, arms) = indented_block(indentation, i, parse_match_arm)?; Ok((i, Term::Match{ discriminant: discriminant.into(), arms })) } pub fn parse_match_arm(indentation: usize, i: &str) -> IResult<&str, MatchArm> { let here_term = |i| parse_term(indentation, i); let (i, (pattern, statement)) = separated_pair(here_term, tag(" => "), here_term)(i)?; Ok((i, MatchArm{ pattern, statement })) } // pub fn parse_pattern(i: &str) -> IResult<&str, Pattern> { pub fn parse_pattern(i: &str) -> IResult<&str, String> { parse_ident(i) } pub fn parse_expression(indentation: usize, i: &str) -> IResult<&str, Term> { let (i, first) = parse_ident(i)?; let (i, rest) = many0(|i| parse_chain_item(indentation, i))(i)?; let term = if rest.len() == 0 { Term::Lone(first) } else { Term::Chain(first, rest) }; Ok((i, term)) } pub fn parse_chain_item(indentation: usize, i: &str) -> IResult<&str, ChainItem> { alt(( map(preceded(c('.'), parse_ident), ChainItem::Access), map( delimited(c('('), separated_list0(tag(", "), |i| parse_expression(indentation, i)), c(')')), |arguments| ChainItem::Call{ arguments }, ), ))(i) } fn get_tab_count(i: &str) -> usize { let mut tab_count = 0; let mut line_chars = i.chars(); while line_chars.next() == Some('\t') { tab_count = tab_count + 1; } tab_count } #[cfg(test)] mod tests { use super::*; fn make_lone(s: &str) -> Term { Term::Lone(s.into()) } fn make_day(day: &str) -> Term { Term::Chain("Day".into(), vec![ChainItem::Access(day.into())]) } #[test] fn test_parse_expression() { let i = "Day.Monday"; assert_eq!( parse_expression(0, i).unwrap().1, make_day("Monday"), ); let i = "fn(next, hello)"; assert_eq!( parse_expression(0, i).unwrap().1, Term::Chain("fn".into(), vec![ChainItem::Call{ arguments: vec![make_lone("next"), make_lone("hello")] }]), ); } #[test] fn test_parse_match() { let i = r#" match d; Day.Monday => Day.Tuesday Day.Tuesday => Day.Wednesday "#.trim(); assert_eq!( parse_match(3, i).unwrap().1, Term::Match{ discriminant: Box::new(Term::Lone("d".into())), arms: vec![ MatchArm{ pattern: make_day("Monday"), statement: make_day("Tuesday") }, MatchArm{ pattern: make_day("Tuesday"), statement: make_day("Wednesday") }, ] }, ); } #[test] fn test_parse_type_definition() { let i = r#" type Day; | Monday | Tuesday "#.trim(); assert_eq!( parse_type_definition(3, i).unwrap().1, TypeDefinition{ name: "Day".into(), body: TypeBody::Union{ branches: vec!["Monday".into(), "Tuesday".into()] }, }, ); } #[test] fn test_parse_procedure() { let i = r#" proc same_day(d: Day): Day; d "#.trim(); assert_eq!( parse_procedure_definition(3, i).unwrap().1, ProcedureDefinition{ name: "same_day".into(), parameters: vec![("d".into(), "Day".into())], return_type: "Day".into(), statements: vec![Term::Lone("d".into())], }, ); let i = r#" proc same_day(): Day; d "#.trim(); assert_eq!( parse_procedure_definition(3, i).unwrap().1, ProcedureDefinition{ name: "same_day".into(), parameters: vec![], return_type: "Day".into(), statements: vec![Term::Lone("d".into())], }, ); } #[test] fn test_parse_debug_statement() { let i = r#" debug next_weekday(next_weekday(d)) "#.trim(); assert_eq!( parse_debug_statement(3, i).unwrap().1, DebugStatement{ term: Term::Chain("next_weekday".into(), vec![ChainItem::Call{ arguments: vec![ Term::Chain("next_weekday".into(), vec![ChainItem::Call{ arguments: vec![make_lone("d")] }]), ]}]), }, ); } } // fn deindent_to_base(s: &str) -> String { // let s = s.strip_prefix("\n").unwrap_or(s); // let (s, base_tabs) = match is_a("\t")(s) { // Ok((s, tabs)) => (s, tabs.len()), // Err(_) => { return s.into() }, // }; // let mut final_s = String::new(); // for line in s.split("\n") { // // take base_tabs number of tabs, always expecting // } // s.into() // } ================================================ FILE: theory/_CoqProject ================================================ -R . theory -Q /home/blaine/lab/cpdtlib Cpdt ================================================ FILE: theory/list_assertions.v ================================================ Set Implicit Arguments. Set Asymmetric Patterns. Require Import List. Import ListNotations. From stdpp Require Import base options list tactics. (* All just collects unordered independent assertions. *) Section All. Inductive All: list Prop -> Prop := | All_empty: All [] | All_push: forall (P: Prop) (l: list Prop), P -> All l -> All (P :: l) . Hint Constructors All: core. Theorem All_pop (P: Prop) (l: list Prop): All (P :: l) -> P. Proof. inversion 1; auto. Qed. Theorem All_And (P: Prop) (l: list Prop): P /\ All l <-> All (P :: l). Proof. split. - intros [??]; auto. - inversion 1; auto. Qed. Theorem All_flatten_head (A B: Prop) (l: list Prop): All ((A /\ B) :: l) <-> All (A :: B :: l). Proof. split. - inversion 1; naive_solver. - inversion 1 as [|??? H']; inversion H'; auto. Qed. Theorem All_permutation (A B: list Prop): Permutation A B -> All A -> All B. Proof. intros Hpermutation HA; induction Hpermutation as []; try inversion HA; auto. inversion HA as [|??? Hl]; inversion Hl; auto. Qed. Global Instance Proper_Permutation_All: Proper (Permutation ==> flip impl) All. Proof. intros ??[]?; eauto using All_permutation, Permutation_sym. inversion H as [| ??? Hl']; inversion Hl'; auto. Qed. Theorem All_permutation_cleaner (A B: list Prop): Permutation A B -> All A -> All B. Proof. intros ?H; rewrite <-H; auto. Qed. Theorem All_In_middle (P: Prop) (before after: list Prop): All (before ++ [P] ++ after) -> P. Proof. intros H; assert (Hpermutation: Permutation (before ++ [P] ++ after) ([P] ++ (before ++ after))) by eauto using Permutation_app_swap_app; rewrite Hpermutation in H; assert (Hswap: ([P] ++ before ++ after) = (P :: (before ++ after))) by eauto; rewrite Hswap in H; eapply All_pop; eauto. Qed. Theorem All_In (P: Prop) (l: list Prop): In P l -> All l -> P. Proof. intros Hin Hall; apply in_split in Hin as [before [after]]; subst; rewrite cons_middle in Hall; eapply All_In_middle; eauto. Qed. End All. #[export] Hint Constructors All: core. (* Compatible links assertions that must have some notion of compatibility between all items. *) Section Compatible. Context {T: Type}. Hypothesis ItemsCompatible: T -> T -> Prop. Hypothesis ItemsCompatible_symmetric: forall t1 t2, ItemsCompatible t1 t2 -> ItemsCompatible t2 t1. Inductive Compatible: list T -> Prop := | Compatible_empty: Compatible [] | Compatible_push: forall t l, Forall (ItemsCompatible t) l -> Compatible (t :: l) . Hint Constructors Compatible: core. Theorem Compatible_symmetric t1 t2: Compatible [t1; t2] -> Compatible [t2; t1]. Proof using ItemsCompatible_symmetric. intros H; inversion H as [| ?? HF]; inversion HF; eauto. Qed. (*Theorem Compatible_swap t1 t2 l: Compatible (t1 :: t2 :: l) -> Compatible (t2 :: t1 :: l). Proof. intros H. inversion H as [| ???]. subst. - subst. constructor. Qed. Global Instance Proper_Permutation_Compatible: Proper (Permutation ==> flip impl) Compatible. Proof. intros ??[]?; try auto. - inversion H0; subst; simpl. induction H as []; try auto; subst; simpl. intros ??[]?; eauto using All_permutation, Permutation_sym. inversion H as [| ??? Hl']; inversion Hl'; auto. Qed. Theorem Compatible_Permutation l1 l2: Permutation l1 l2 -> Compatible l1 -> Compatible l2. Proof. intros HP HC. induction HC as [|? ? ? ?]. - apply Permutation_nil in HP; naive_solver. - inversion H. + subst. apply Permutation_singleton_l in HP. naive_solver. + subst. rewrite <-HP. induction HP as []; try inversion HC; auto. intros Hpermutation HA; induction Hpermutation as []; try inversion HA; auto. inversion HA as [|??? Hl]; inversion Hl; auto. Qed.*) End Compatible. #[export] Hint Constructors Compatible: core. Section test_Compatible_neq. Theorem Compatible_neq (a b c: nat): a <> b -> b <> c -> a <> c -> Compatible (fun a b => a <> b) [a; b; c]. Proof. auto. Qed. End test_Compatible_neq. (* Chain links assertions in a strict order, where each assertion is implied by the previous. *) Section Chain. Context {T: Type}. Hypothesis Link: T -> T -> Prop. Inductive Chain: list T -> Prop := | Chain_empty: forall start, Chain [start] | Chain_link: forall past cur next, Link cur next -> Chain (cur :: past) -> Chain (next :: (cur :: past)) . Hint Constructors Chain: core. Inductive Chained: T -> T -> Prop := | Chained_start_finish: forall start mid finish, Chain (finish :: (mid ++ [start])) -> Chained start finish . Theorem Chain_to_Chained l: Chain l -> Chained start finish. Proof. Qed. End Chain. ================================================ FILE: theory/main.v ================================================ Add LoadPath "/home/blaine/lab/cpdtlib" as Cpdt. Set Implicit Arguments. Set Asymmetric Patterns. From stdpp Require Import base options strings stringmap. Import ListNotations. (*Require Import theory.utils.*) Require Import String. Notation MachineState := (stringmap nat). (*Record MachineState: Type := machine_state { global_identifiers: (stringmap nat); local_identifiers: (stringmap nat); }.*) Inductive Operand: Type := | Literal (n: nat) | Identifier (s: string) . Definition evaluate_operand (operand: Operand) (state: MachineState): option nat := match operand with | Literal n => Some n (*| Identifier s => lookup s state.(local_identifiers)*) | Identifier s => lookup s state end . (*gen_heap_valid*) Inductive Instruction: Type := | Inst_Add (dest: string) (op1 op2: Operand) . Definition evaluate_instruction (instruction: Instruction) (state: MachineState) : option (string * nat) := match instruction with | Inst_Add dest op1 op2 => match (evaluate_operand op1 state), (evaluate_operand op2 state) with | (Some op1value), (Some op2value) => Some (dest, (op1value + op2value)) | _, _ => None end end . (*proofs that evaluate_instruction returns Some if wp is satisfied*) Definition step_instruction (instruction: Instruction) (state: MachineState) : option MachineState := match evaluate_instruction instruction state with | None => None (*| Some (dest, value) => Some (insert dest value state.(local_identifiers))*) | Some (dest, value) => Some (insert dest value state) end . (*proofs that step_instruction returns Some if wp is satisfied*) Fixpoint step_instructions (instructions: list Instruction) (state: MachineState) : option MachineState := match instructions with | instruction :: rest_instructions => match step_instruction instruction state with | None => None | Some next_state => step_instructions rest_instructions next_state end | [] => Some state end . (*proofs that step_instructions returns Some if "steps" wp is satisfied*) Inductive TerminatorInstruction: Type := | Branch (label: string) | BranchIf (condition: Operand) (if_label else_label: string) . Definition evaluate_terminator terminator state := match terminator with | Branch label => Some label | BranchIf condition if_label else_label => match evaluate_operand condition state with | None => None | Some condition_nat => Some (if Nat.eq_dec condition_nat 1 then if_label else else_label) end end . (*proofs that evaluate_terminator returns Some if terminator wp is satisfied*) Record BasicBlock: Type := { sequential: list Instruction; terminator: TerminatorInstruction; }. Record ExecutableProgram: Type := { starting_block: BasicBlock; program: stringmap BasicBlock; }. Record ProgramState: Type := { program: ExecutableProgram; current_label: string; current_index: nat; }. Definition step_block block state := match step_instructions block.(sequential) state with | None => None | Some stepped_state => match evaluate_terminator block.(terminator) stepped_state with | None => None | Some next_label => Some (next_label, stepped_state) end end . (*proofs that step_block returns Some if overall wp is satisfied*) From iris.base_logic.lib Require Import gen_heap. Class magmideGS Σ := MagmideGS { (*magmideGS_invGS : invGS_gen HasLc Σ;*) magmideGS_gen_heapGS :> gen_heapGS string nat Σ; (*magmideGS_inv_heapGS :> inv_heapGS string nat Σ;*) }. Section magmide. Context `{!magmideGS Σ}. Definition state_interpretation (machine_state: MachineState): iProp Σ := (*gen_heap_interp machine_state.(local_identifiers).*) gen_heap_interp machine_state. Notation "'%' var_name '==' value" := (mapsto (L:=string) (V:=nat) var_name (DfracOwn 1) value) (at level 20, format "'%' var_name '==' value"): bi_scope. Notation spec pre_condition program post_condition := ([pre_condition; program; post_condition]) (only parsing). Definition one_add_program := [ (Inst_Add "one_add" (Identifier "arg") (Literal 5)) ]. Open Scope bi_scope. Theorem test__one_add_program val: spec (%"arg"==val) one_add_program (%"one_add"==(val + 5)). Proof. Qed. Definition two_add_program := [ (Inst_Add "two_add" (Identifier "arg") (Literal 1)); (Inst_Add "two_add" (Identifier "arg") (Literal 1)) ]. Theorem test__two_add_program arg: {{ %arg==arg }} two_add_program {{ %two_add==arg + 2 }}. Proof. Qed. End magmide. From iris.algebra Require Export ofe. Global Instance MachineState_inhabited : Inhabited MachineState := populate {| local_identifiers := inhabitant |}. (*Canonical Structure MachineStateO := leibnizO MachineState.*) (*Notation Mask := coPset.*) Section Sized. Context {size: nat}. Notation register := (fin size). Inductive Operand: Type := | Literal (n: nat) | Register (r: register) . (* It seems pretty obvious I need to break instructions into "sequential" and "terminator" versions (kinda like llvm), and then have two layers of assertions, each with their own "later" concept: - a layer for reasoning purely about sequential "segments" of instructions (the body of a basic block) - a layer for reasoning about graphs of labeled blocks. this layer is basically where we can encode some concept of recursion, and where we use the lob induction iris provides to make it possible to reason about loops and such we have one layer of weakest preconditions (or just preconditions like in the low level code papers?) that moves sequential statements along then we have a "block" structure that contains a list of sequential instructions and a single terminator instruction then we have an "execute block" function that takes a machine state (that doesn't include an instruction pointer for now?) and transforms it according to the sequential segment and then returns the transformed state and an optional next label, with None meaning execution is finished *) Inductive Instruction := | Instruction_Exit | Instruction_Move (src: Operand) (dest: register) | Instruction_Add (val: Operand) (dest: register) | Instruction_Jump (to: nat) | Instruction_BranchEq (a: Operand) (b: Operand) (to: nat) | Instruction_BranchNeq (a: Operand) (b: Operand) (to: nat) (*| Instruction_Store (src: Operand) (dest: Operand)*) (*| Instruction_Load (src: Operand) (dest: register)*) . Hint Constructors Instruction: core. Record MachineState := machine_state { instruction_pointer: nat; (*registers: (vec nat size);*) registers: (gmap nat nat); program: list Instruction }. Notation current_instruction s := (lookup s.(instruction_pointer) s.(program)) (only parsing). Notation incr s := (S s.(instruction_pointer)) (only parsing). Notation get cur reg := (cur.(registers) !!! reg) (only parsing). Notation update s dest val := (vinsert dest val s.(registers)) (only parsing). Notation make_next cur next_instruction_pointer next_registers := (machine_state next_instruction_pointer next_registers cur.(program)) (only parsing). Definition eval_operand (cur: MachineState) (operand: Operand) : nat := match operand with | Literal n => n | Register r => (cur.(registers) !!! r) end . Inductive Step: MachineState -> MachineState -> Prop := | Step_Move: forall cur src dest, (current_instruction cur) = Some (Instruction_Move src dest) -> Step cur (make_next cur (incr cur) (update cur dest (eval_operand cur src)) ) (*$s==v ∗ $d==? ∗ ($s==v ∗ $d==? -∗ Q d v) wp Move $r $d {Q d v}*) | Step_Add: forall cur val dest, (current_instruction cur) = Some (Instruction_Add val dest) -> Step cur (make_next cur (incr cur) (update cur dest ((eval_operand cur val) + (get cur dest))) ) | Step_Jump: forall cur to, (current_instruction cur) = Some (Instruction_Jump to) -> Step cur (make_next cur to cur.(registers)) | Step_BranchEq_Yes: forall cur a b to, (current_instruction cur) = Some (Instruction_BranchEq a b to) -> a = b -> Step cur (make_next cur to cur.(registers)) | Step_BranchEq_No: forall cur a b to, (current_instruction cur) = Some (Instruction_BranchEq a b to) -> ~(a = b) -> Step cur (make_next cur (incr cur) cur.(registers)) | Step_BranchNeq_Yes: forall cur a b to, (current_instruction cur) = Some (Instruction_BranchNeq a b to) -> ~(a = b) -> Step cur (make_next cur to cur.(registers)) | Step_BranchNeq_No: forall cur a b to, (current_instruction cur) = Some (Instruction_BranchNeq a b to) -> a = b -> Step cur (make_next cur (incr cur) cur.(registers)) . Hint Constructors Step: core. (*theorems about individual instructions, such as that exit never takes a step*) (*use the Chain prop to think about traces*) (*theorem that any trace with Exit at the top never continues*) (*theorems lifting list operators for Trace*) Class magmideGS (Σ: gFunctors) := MagmideG { (*magmideGS_invGS :> invGS_gen HasNoLc Σ;*) (*magmideGS_gen_heapG :> gen_heapGS nat nat Σ;*) magmideGS_ghost_mapG :> ghost_mapGS nat nat Σ; }. Global Opaque magmide_invGS. Global Arguments MagmideG {Σ}. (*gen_heap_interp state.(heap)*) (*leads to*) (*ghost_map_auth (gen_heap_name hG) 1 state.(heap)*) (*leads to*) (*Record state : Type := { heap: gmap loc (option val); }.*) Definition state_interpretation state := gen_heap_interp state.(heap) (*∗ steps_auth step_cnt*). Definition wp_pre `{!magmideGS Σ} (wp: Mask -d> MachineState -d> (MachineState -d> iPropO Σ) -d> iPropO Σ) : Mask -d> MachineState -d> (MachineState -d> iPropO Σ) -d> iPropO Σ := fun mask cur Post => match current_instruction cur with | None => |={mask}=> Post cur | Some Instruction_Exit => |={mask}=> Post cur | Some instruction => state_interpretation cur -∗ |={mask,∅}=> ( CanStep cur ∗ (forall next, Step cur next -∗ |={∅,mask}=> ▷ ( state_interpretation next ∗ wp mask next Post )) ) end%I. Local Instance wp_pre_contractive `{!magmideGS Σ}: Contractive wp_pre. Proof. rewrite /wp_pre /= ⇒ n wp wp' Hwp E e1 Φ. do 25 (f_contractive || f_equiv). induction num_laters_per_step as [|k IH]; simpl. - repeat (f_contractive || f_equiv); apply Hwp. - by rewrite -IH. Qed. (*probably have to define our own Wp typeclass :( *) Class Wp := wp: Mask -> MachineState -> (MachineState -> iPropO Σ) -> iPropO Σ. Global Arguments wp {_ _ _ _ _} _ _ _%E _%I. Global Instance: Params (@wp) 8 := {}. Local Definition wp_def `{!magmideGS Σ}: Wp := fixpoint wp_pre. Local Definition wp_aux: seal (@wp_def). Proof. by eexists. Qed. Definition wp' := wp_aux.(unseal). Global Arguments wp' {hlc Λ Σ _}. Global Existing Instance wp'. Local Lemma wp_unseal `{!magmideGS Σ}: wp = @wp_def hlc Λ Σ _. Proof. rewrite -wp_aux.(seal_eq) //. Qed. End Sized. (* define the wp recursively, referring to a state interpretation (that includes a reference to Step?) and using later and update modalities to link step indexing to steps in the program then prove a bunch of consequences of the wp, such as a wp for each individual instruction or specialized versions of operators like * and -* then do things like prove the wp is non expansive? TCEq (to_val e) None -> Proper (pointwise_relation _ (dist_later n) ==> dist n) (wp (PROP:=iProp Σ) s E e). that it preserves equivalences? Proper (pointwise_relation _ (≡) ==> (≡)) (wp (PROP:=iProp Σ) s E e). then higher level concepts, such as being able to form wps for blocks of code instead of individual instructions I want to get to the point where I can: - verify a simple straight line program that does something like adding four numbers - verify a program with a branch implemented loop - add input and output signals similar to the kinds of simple signals in a chip like a 6502, and use them to verify something like a hello world program then is it time to start figuring out and building systems for trackable effects? *) ================================================ FILE: theory/playground.v ================================================ Inductive Trivial: Prop := | TrivialCreate. Definition Trivial_manual: Trivial := TrivialCreate. Definition Trivial_inductive_manual (P: Prop) (p: P) (evidence: Trivial): P := match evidence with | TrivialCreate => p end. Inductive Eq {T: Type}: T -> T -> Prop := | EqCreate: forall t, Eq t t. Definition Eq_inductive_manual (T: Type) (P: T -> T -> Prop) (f: forall t: T, P t t) (l r: T) (evidence: Eq l r) : P l r := match evidence in (Eq l r) return (P l r) with | EqCreate t => f t end. Inductive And (P Q: Prop): Prop := | AndCreate (Ph: P) (Pq: Q): And P Q. Definition And_inductive_manual (P Q R: Prop) (f: P -> Q -> R) (evidence: And P Q) : R := match evidence with | AndCreate _ _ p q => f p q end. Definition And_left_manual (P Q: Prop) (evidence: And P Q): P := match evidence with | AndCreate _ _ p q => p end. Definition And_right_manual (P Q: Prop) (evidence: And P Q): Q := match evidence with | AndCreate _ _ p q => q end. Inductive Or (P Q: Prop): Prop := | OrLeft: P -> Or P Q | OrRight: Q -> Or P Q. Definition Or_inductive_manual (P Q R: Prop) (f_left: P -> R) (f_right: Q -> R) (evidence: Or P Q) : R := match evidence with | OrLeft _ _ l => f_left l | OrRight _ _ r => f_right r end. Inductive Bool: Type := | BoolTrue | BoolFalse. Definition Bool_inductive_manual (P: Bool -> Prop) (P_BoolTrue: P BoolTrue) (P_BoolFalse: P BoolFalse) : forall (b: Bool), P b := fun b => match b with | BoolTrue => P_BoolTrue | BoolFalse => P_BoolFalse end. Definition Eq_manual: Eq BoolTrue BoolTrue := EqCreate BoolTrue. Fail Definition Eq_manual_bad: Eq BoolTrue BoolFalse := EqCreate BoolTrue. Fail Definition Eq_manual_bad: Eq BoolTrue BoolFalse := EqCreate BoolFalse. Fail Definition Eq_manual_bad: Eq BoolFalse BoolTrue := EqCreate BoolTrue. Fail Definition Eq_manual_bad: Eq BoolFalse BoolTrue := EqCreate BoolFalse. Inductive Impossible: Prop := . Definition BoolTrue_not_BoolFalse_manual: (Eq BoolTrue BoolFalse) -> Impossible := fun wrong_eq => match wrong_eq with | EqCreate t => match t with | BoolTrue => TrivialCreate | BoolFalse => TrivialCreate end end. Definition Not (P: Prop) := P -> Impossible. Definition not_impossible_manual: Not Impossible := fun impossible => match impossible with end. Definition ex_falso_quodlibet: forall (P: Prop), Impossible -> P := fun (P: Prop) (impossible: Impossible) => match impossible return P with end. Definition BoolTrue_not_BoolFalse_manual_full: Not (Eq BoolTrue BoolFalse) := fun wrong_eq => match wrong_eq in (Eq l r) return match l, r with | BoolTrue, BoolTrue => Trivial | BoolFalse, BoolFalse => Trivial | _, _ => Impossible end with | EqCreate t => match t with | BoolTrue => TrivialCreate | BoolFalse => TrivialCreate end end. Print BoolTrue_not_BoolFalse_manual_full. Inductive day: Type := | monday | tuesday | wednesday | thursday | friday | saturday | sunday. Definition next_weekday (d: day): day := match d with | monday => tuesday | tuesday => wednesday | wednesday => thursday | thursday => friday | friday => monday | saturday => monday | sunday => monday end. Compute (next_weekday friday). (* ==> monday : day *) Compute (next_weekday (next_weekday saturday)). (* ==> tuesday : day *) Definition test_next_weekday_manual: Eq (next_weekday saturday) monday := EqCreate monday. (* Counterexample by Thierry Coquand and Christine Paulin Translated into Coq by Vilhelm Sjöberg *) (* NotPos represents any positive, but not strictly positive, operator. *) (*Definition NotPos (a: Type) := (a -> Prop) -> Prop.*) (* If we were allowed to form the inductive type Inductive A: Type := createA: NotPos A -> A. then among other things, we would get the following. *) Axiom A: Type. (*Axiom createA: NotPos A -> A.*) Axiom createA: ((A -> Prop) -> Prop) -> A. (*Axiom matchA: A -> NotPos A.*) Axiom matchA: A -> ((A -> Prop) -> Prop). (*Axiom roundtrip_same: forall (a: NotPos A), matchA (createA a) = a.*) Axiom roundtrip_same: forall (a: ((A -> Prop) -> Prop)), matchA (createA a) = a. (* In particular, createA is an injection. *) Lemma createA_injective: forall p p', createA p = createA p' -> p = p'. Proof. intros. assert (matchA (createA p) = (matchA (createA p'))) as H1 by congruence. rewrite roundtrip_same in H1. rewrite roundtrip_same in H1. assumption. (*now repeat rewrite roundtrip_same in H1.*) Qed. (* However, ... *) (* Proposition: For any type A, there cannot be an injection from NotPos(A) to A. *) (* For any type T, there is an injection from T to (T->Prop), which is λt1.(λt2.t1=t2) . *) Definition type_to_prop {T:Type}: T -> (T -> Prop) := fun t1 t2 => t1 = t2. Lemma type_to_prop_injective: forall T (t t': T), type_to_prop t = type_to_prop t' -> t = t'. Proof. intros. assert (type_to_prop t t = type_to_prop t' t) as H1 by congruence. compute in H1. symmetry. rewrite <- H1. reflexivity. Qed. (* Hence, by composition, we get an injection prop_to_type from A->Prop to A. *) Definition prop_to_type: (A -> Prop) -> A := fun p => createA (type_to_prop p). Lemma prop_to_type_injective: forall p p', prop_to_type p = prop_to_type p' -> p = p'. Proof. unfold prop_to_type. intros. apply createA_injective in H. apply type_to_prop_injective in H. assumption. Qed. (* We are now back to the usual Cantor-Russel paradox. *) (* We can define *) Definition AProp: A -> Prop (*the user gives us some `a: A`*) := fun a => (*and then we ask them to prove that some prop exists which:*) (*- **is the same as the object `a` they just gave us** *) (*- doesn't hold true for `a` *) (*these things together ask them to give us a prop that isn't true of itself!*) exists (P: A -> Prop), prop_to_type P = a /\ ~P a. (*this shouldn't be possible!*) Definition a0: A := prop_to_type AProp. (* We have (AProp a0) iff ~(AProp a0) *) Lemma bad: (AProp a0) <-> ~(AProp a0). Proof. split. * unfold not. (*in this first case we've assumed aprop, but that's a problem because aprop *carries inside itself* a proof that there's some prop that isn't true of the thing we passed in, but also a proof that the thing we passed in is the same as the prop that isn't true!*) (*this is the core problem of non-strict positivity, that it allows an assumption we've made, an argument to our function, to already contain a proof of its own falsity*) (*this is definitely a kind of recursion, but it's *constructor* recursion rather than computational recursion *) intros [P [H1 H2]] H. unfold not in H2. unfold a0 in H1. apply prop_to_type_injective in H1. rewrite H1 in H2. auto. * (*this second case is the more straightforward one that assumes falsity already, so we just have to show that the proof of falsity actually applies to itself*) intros. exists AProp. split; try assumption. unfold a0. reflexivity. Qed. (* Hence a contradiction. *) Theorem worse: False. pose bad. tauto. Qed. Print worse. (*to construct yes, all you have to do is construct no*) (*and to construct no, you only have to construct a *function* that accepts (assumes) yes *) (*and you've done this in an environment where its possible to use the contradictory theorem to prove False once you've hit that level*) (*is that the core idea of non-strict positivity? that it allows you to create situations where in order to create a *thing* you merely have to create a function that accepts the thing? with a side trip to creating ~thing?*) Definition worse_manual: False := and_ind (fun (forward: AProp a0 -> ~AProp a0) (backward: ~AProp a0 -> AProp a0) => let aprop: AProp a0 := ( let not_aprop: ~AProp a0 := ( fun aprop: AProp a0 => let not_aprop: ~AProp a0 := forward aprop in let ohno: False := not_aprop aprop in False_ind False ohno ): ~AProp a0 in backward not_aprop ) in (fun aprop: AProp a0 => let not_aprop: ~AProp a0 := forward aprop in let ohno: False := not_aprop aprop in False_ind False ohno ) aprop ) bad. Theorem worse_full: False. pose bad. destruct bad as [forward backward]. clear i. assert (AProp a0) by (apply backward; unfold not; intros; apply forward; assumption). apply forward. assumption. assumption. Qed. ================================================ FILE: theory/utils.v ================================================ Add LoadPath "/home/blaine/lab/cpdtlib" as Cpdt. Set Implicit Arguments. Set Asymmetric Patterns. Require Import List Cpdt.CpdtTactics. From stdpp Require Import base fin vector options. Import ListNotations. Ltac solve_crush := try solve [crush]. Ltac solve_assumption := try solve [assumption]. Ltac subst_injection H := injection H; intros; subst; clear H. Notation impossible := (False_rect _ _). Notation this item := (exist _ item _). Notation use item := (proj1_sig item). Require Import theory.list_assertions. From Coq.Wellfounded Require Import Inclusion Inverse_Image. Section wf_transfer. Variable A B: Type. Variable RA: A -> A -> Prop. Variable RB: B -> B -> Prop. Variable f: A -> B. Theorem wf_transfer: (forall a1 a2, (RA a1 a2) -> (RB (f a1) (f a2))) -> well_founded RB -> well_founded RA. Proof. intros H?; apply (wf_incl _ _ (fun a1 a2 => (RB (f a1) (f a2))) H); apply wf_inverse_image; assumption. Qed. End wf_transfer. Section VectorIndexAssertions. Context {T: Type}. Context {size: nat}. Notation IndexPair := (prod (fin size) T) (only parsing). Definition index_disjoint (index: fin size) := fun (index_pair: IndexPair) => ((index_pair.1) <> index). Inductive UnionAssertion (vector: vec T size): list IndexPair -> Prop := | union_nil: UnionAssertion vector [] | union_cons: forall pairs index value, UnionAssertion vector pairs -> (vector !!! index) = value -> Forall (index_disjoint index) pairs -> UnionAssertion vector ((index, value) :: pairs) . (*Theorem UnionAssertion_no_duplicate_indices: forall vector index_pairs, UnionAssertion vector index_pairs -> NoDup (map (fun index_pair => index_pair.1) index_pairs). Proof. Qed.*) (* while this is fancy and everything, we almost certainly don't need it, since we really want a higher level opaque theorem stating that if two assertions point to the same location in the same vector then we can derive False *) End VectorIndexAssertions. Section convert_subset. Variable T: Type. Variable P Q: T -> Prop. Theorem convert_subset: {t | P t} -> (forall t: T, P t -> Q t) -> {t | Q t}. Proof. intros []; eauto. Qed. End convert_subset. Arguments convert_subset {T} {P} {Q} _ _. Notation convert item := (convert_subset item _). Notation Yes := (left _ _). Notation No := (right _ _). Notation Reduce x := (if x then Yes else No). Theorem append_single_cons {T: Type}: forall (t: T) l, t :: l = [t] ++ l. Proof. induction l; auto. Qed. Theorem valid_index_not_None {T} (l: list T) index: index < (length l) -> (lookup index l) <> None. Proof. intros ??%lookup_ge_None; lia. Qed. Theorem valid_index_Some {T} (l: list T) index: index < (length l) -> exists t, (lookup index l) = Some t. Proof. intros ?%(lookup_lt_is_Some_2 l index); unfold is_Some in *; assumption. Qed. (*lookup_lt_Some*) Definition safe_lookup {T} index (l: list T): index < (length l) -> {t | (lookup index l) = Some t} . intros ?%valid_index_not_None; destruct (lookup index l) eqn:Hlook; try contradiction; rewrite <- Hlook; apply (exist _ t Hlook). Defined. Theorem safe_lookup_In {T} index (l: list T) (H: index < length l): In (use (safe_lookup l H)) l. Proof. apply elem_of_list_In; destruct (safe_lookup l H); simpl; apply elem_of_list_lookup; exists index; assumption. Qed. Theorem Forall_safe_lookup {T} (P: T -> Prop) l: Forall P l <-> forall index (H: index < length l), P (use (safe_lookup l H)). Proof. split. - intros; destruct (safe_lookup l _); simpl; apply (Forall_lookup_1 P l index); assumption. - intros ?Hfunc; apply Forall_lookup; intros index item Hlookup; specialize (lookup_lt_Some l index item Hlookup) as Hindex; specialize (Hfunc index Hindex); destruct (safe_lookup l _) as [item' Hitem'] in Hfunc; simpl in Hfunc; rewrite Hlookup in Hitem'; subst_injection Hitem'; assumption. Qed. Definition closer_to target: nat -> nat -> Prop := fun next cur => (target - next) < (target - cur). (*Hint Unfold closer_to: core.*) Theorem closer_to_well_founded target: well_founded (closer_to target). Proof. apply (well_founded_lt_compat nat (fun a => target - a)); intros; assumption. Defined. Theorem closer_to_reverse: forall target cur next, (target - next) < (target - cur) -> cur < next. Proof. lia. Qed. Theorem closer_to_bounded_reverse: forall target cur next, cur < next -> cur < target -> (target - next) < (target - cur). Proof. lia. Qed. Definition closer_to_end {T} (arr: list T) := closer_to (length arr). Theorem closer_to_end_well_founded {T} (arr: list T): well_founded (closer_to_end arr). Proof. apply closer_to_well_founded. Qed. Theorem numeric_capped_incr_safe total begin cap index: total = begin + cap -> 0 < cap -> index < begin -> S index < total. Proof. lia. Qed. Theorem capped_incr_safe {T} (total begin cap: list T) index: total = begin ++ cap -> 0 < length cap -> index < length begin -> S index < length total. Proof. intros Htotal Hcap Hindex; assert (Hlen: length total = (length begin) + (length cap)) by solve [rewrite Htotal; apply app_length]; apply (numeric_capped_incr_safe Hlen Hcap Hindex). Qed. Theorem index_pairs_lookup_forward {A B}: forall (items: list A) (f: nat -> A -> B) item index, lookup index items = Some item -> lookup index (imap f items) = Some (f index item). Proof. induction items; intros ??[]?; try solve [apply (IHitems (compose f S)); assumption]; naive_solver. Qed. Theorem index_pairs_lookup_back {A B}: forall (items: list A) (f: nat -> A -> B) item index, (forall index i1 i2, f index i1 = f index i2 -> i1 = i2) -> lookup index (imap f items) = Some (f index item) -> lookup index items = Some item. Proof. induction items; intros ??[]Hf?; try solve [injection H; intros ?%Hf; naive_solver]; try solve [apply (IHitems (compose f S)); eauto]; naive_solver. Qed. Theorem index_pair_equality {A B} (a: A) (b1 b2: B): (a, b1) = (a, b2) -> b1 = b2. Proof. naive_solver. Qed. Inductive partial (P: Prop): Type := | Proven: P -> partial P (*| Falsified: ~P -> partial P*) | Unknown: partial P . Notation proven := (Proven _). Notation unknown := (Unknown _). Notation provenif test := (if test then proven else unknown). Section find_obligations. Context {T: Type}. Variable P: T -> Prop. Theorem forall_done_undone items done undone: Permutation items (done ++ undone) -> Forall P done -> Forall P undone -> Forall P items. Proof. intros Hpermutation??; assert (Happ: Forall P (done ++ undone)) by solve [apply Forall_app_2; assumption]; setoid_rewrite Hpermutation; assumption. Qed. Variable compute_partial: forall t: T, partial (P t). Definition split_by_maybe: forall items: list T, { pair | Permutation items (pair.1 ++ pair.2) /\ Forall P pair.1 }. refine (fix split_by_maybe items := match items with | [] => this ([], []) | item :: items' => let (pair, H) := split_by_maybe items' in match (compute_partial item) with | Proven _ => this ((item :: pair.1), pair.2) | Unknown => this (pair.1, (item :: pair.2)) end end ); intros; split; simpl in *; try destruct H; try solve [setoid_rewrite H; apply Permutation_middle]; auto. Defined. Definition find_obligations_function: forall items, { obligations | Forall P obligations -> Forall P items }. refine (fun items => let (pair, H) := split_by_maybe items in this pair.2 ); destruct H; apply (forall_done_undone H); assumption. Defined. Theorem verify__find_obligations_function: forall items found, found = find_obligations_function items -> Forall P (use found) -> Forall P items. Proof. intros ?[]; auto. Qed. End find_obligations. Ltac find_obligations__helper P compute_partial items := let found := eval compute in (find_obligations_function P compute_partial items) in let pf := eval compute in (proj2_sig found) in apply pf; apply Forall_fold_right; simpl; repeat split . Ltac find_obligations P compute_partial items := match goal with | |- Forall P items => find_obligations__helper P compute_partial items | |- forall item, In item items -> P item => apply Coq.Lists.List.Forall_forall; find_obligations__helper P compute_partial items | |- forall item, elem_of item items -> P item => apply Forall_forall; find_obligations__helper P compute_partial items | |- forall index def, index < length items -> P (nth index items def) => apply Coq.Lists.List.Forall_nth; find_obligations__helper P compute_partial items | |- forall index item, (lookup index items) = Some item -> P item => apply Forall_lookup; find_obligations__helper P compute_partial items | |- forall index, index < length items -> P (items !!! index) => apply Forall_lookup_total; find_obligations__helper P compute_partial items | |- forall index (H: index < length items), P (use (safe_lookup items H)) => apply Forall_safe_lookup; find_obligations__helper P compute_partial items end . Module test__find_obligations. Definition P n := (n < 4 \/ n < 6). Definition compute_partial: forall n, partial (P n). refine (fun n => if (lt_dec n 4) then proven else unknown); unfold P; lia. Defined. Definition items := [0; 1; 2; 4; 3; 2; 5]. Theorem find_obligations__Forall: Forall P items. Proof. find_obligations P compute_partial items; lia. Qed. Theorem find_obligations__forall_In: forall item, In item items -> P item. Proof. find_obligations P compute_partial items; lia. Qed. Theorem find_obligations__forall_elem_of: forall item, elem_of item items -> P item. Proof. find_obligations P compute_partial items; lia. Qed. Theorem find_obligations__forall_nth: forall index def, index < length items -> P (nth index items def). Proof. find_obligations P compute_partial items; lia. Qed. Theorem find_obligations__forall_lookup: forall index item, (lookup index items) = Some item -> P item. Proof. find_obligations P compute_partial items; lia. Qed. Theorem find_obligations__forall_lookup_total: forall index, index < length items -> P (items !!! index). Proof. find_obligations P compute_partial items; lia. Qed. Theorem find_obligations__forall_safe_lookup: forall index (H: index < length items), P (use (safe_lookup items H)). Proof. find_obligations P compute_partial items; lia. Qed. End test__find_obligations. (* Inductive Result (T: Type) (E: Type): Type := | Ok (value: T) | Err (error: E). Arguments Ok {T} {E} _. Arguments Err {T} {E} _. *)