Repository: elm-lang/projects
Branch: master
Commit: 0d0f52abb320
Files: 4
Total size: 24.3 KB
Directory structure:
gitextract_ux15czw1/
├── README.md
├── notes/
│ ├── on-general-purpose.md
│ └── on-sharing-types.md
└── roadmap.md
================================================
FILE CONTENTS
================================================
================================================
FILE: README.md
================================================
**Before you get started,** watch [this video](https://youtu.be/DSjbTC-hvqQ?t=14m5s) that outlines how collaboration works in the Elm community.
# Elm Projects
### For New Contributors
- [Data structure evaluation website](#data-structure-evaluation-website)
- [Profiling `virtual-dom`](#profiling-virtual-dom-with-v8-tools)
- [WebGL](#webgl)
### For Advanced Community Members
- [`elm-find`](#elm-find)
- [REPL](#in-browser-repl)
- [Type-directed autocomplete](#type-directed-autocomplete)
- [Monomorphizing](#explore-monomorphizing-compilers)
- [WebAssembly](#explore-webassembly)
Again, it is very important that you watch [this video](https://youtu.be/DSjbTC-hvqQ?t=14m5s) and read [these comments](roadmap.md) before diving into any of this stuff! Writing the code is definitely not the hard part on any of these!
* * *
## Data structure evaluation website
Over time, the community will develop various implementations of data structures with different performance characterists. In my experiences in other languages, picking “the right one” is very time consuming. So **the big goal here is to help people learn about these data structures and make the right choices for their application.**
Say we have a couple `Queue` and `PriorityQueue` implementations, all with different performance characteristics. It would be wonderful to have a website that:
1. Points people to great learning resources on immutable queues. What is a queue? What is a priority queue? Is it a tree structure? How does that work? Etc.
2. Lets people race the implementations for various scenarios. Ideally folks could specify the size and shape of their data on the website and race based on that. If person A has ~100 items with `Int` priorities and person B has ~10000 items with `String` priorities, I would expect them to make the different choices.
3. Maybe show graphs of performance vs. collection size so people can quickly get a feel for how data structures would work for their expected usage.
In the end, I think this would be **an interactive website that approximates a college-level data structures course, but probably more fun!** All the learning would be directed towards practical problems that folks actually have at the moment, and it will save them a bunch of time!
> Some curration will be necessary to make this great. For example, there would have to be some process for including packages in these numbers. For example, if there is a package that is just has worse performance in all cases, it does not seem like a good use of time for people perusing this website. Ultimately, I think this project would need some coordination with core community members, so please talk about your work on [discourse](https://discourse.elm-lang.org/) to get feedback as early as possible!
## Profiling `virtual-dom` with V8 tools
The `elm-lang/virtual-dom` library is [quite fast](https://elm-lang.org/news/blazing-fast-html-round-two). It may be possible to do better though!
So the goal here is to **get [elm/virtual-dom](https://github.com/elm/virtual-dom/) setup with [v8-natives](https://www.npmjs.com/package/v8-natives)**. This would allow us to:
1. Be sure that code is getting optimized (but not deoptimized!)
2. Know more about how much garbage is produced.
Perhaps certain functions need to be broken into smaller chunks? Perhaps values can be made more monomorphic? A great result would be writeups of what you observe when testing things by hand. If that is useful, it would be best to make these tests a reproducible part of the library. This way any potential changes could be informed by performance implications.
## WebGL
There is already a great foundation for WebGL with [elm-community/webgl](https://package.elm-lang.org/packages/elm-community/webgl/latest), allowing scenes [like this](https://twitter.com/unsoundscapes/status/817493065405435905).
From there folks are working on projects for [terrain generation](https://twitter.com/czaplic/status/819324109674815489) and [loading 3D meshes](https://twitter.com/czaplic/status/820055313386586112).
I think we can make it really fun and easy to create 3D graphics with Elm, and it is a very deep topic. I recommend exploring topics like [picking](http://away3d.com/tutorials/Introduction_to_Mouse_Picking), [entity-component systems](https://en.wikipedia.org/wiki/Entity%E2%80%93component%E2%80%93system), [fancy shaders](https://www.shadertoy.com/), etc. by digging into a project of your own and (1) seeing what you need and (2) sharing your observations in blog posts on [/r/elm](https://www.reddit.com/r/elm/).
# More Advanced Projects
There are a bunch of more speculative projects that it would be fun to look into.
These are probably best for people who are quite experienced with Elm. Maybe who have been using it for a year or more. Long enough to have a good sense of the design aesthetic of core Elm projects.
## `elm-find`
Create a command line tool named `elm-find` that is all about quickly finding definitions and usages.
```
$ elm-find Fake.thing
[]
```
```
$ elm-find toFullName
[
{
"definition": {
"isLocal": true,
"path": "src/Student.elm",
"module": "Student",
"region": {
"start": { "row": 22, "col": 0 }
"end": { "row": 34, "col": 0 }
}
},
"uses": []
}
]
```
```
$ elm-find String.reverse
[
{
"definition": {
"isLocal": false,
"package": "elm/core",
"module": "String"
},
"uses": [
{
"path": "src/Main.elm",
"module": "Main",
"region": {
"start": { "row": 22, "col": 14 }
"end": { "row": 22, "col": 20 }
}
}
]
}
]
```
The implementation should focus on being as fast as possible. Think `grep`. The goal would be for commands to finish in under 500ms on 100k line projects, but I think it's possible to do a lot better.
This could be used to make a bunch of helpful tools:
- A command line tool to do bulk renames in large projects.
- Editor integrations for "Jump to Definition" and bulk renames.
I would recommend implementing this by starting with the existing compiler and stripping out anything that is not needed. This would give you a good foundation for crawling files efficiently, making nice command line arguments, and parsing very quickly.
## In-browser REPL
Right now our REPL only runs on the command line. It would be amazing to have an in-browser REPL so we could actually see `Html` values and have a better UI for poking around expressions and values.
[The most successful effort](http://elmrepl.cuberoot.in/) runs `elm-repl` on a server. This is quite handy, but I think we should take things like [iPython notebooks](https://nbviewer.jupyter.org/) as inspiration for the end goal.
I expect the early prototype would be a REPL for a simple lambda calculus (not connected to Elm at all) so you could test out the core UI ideas. From there, you would get in contact with community members who work on core tools and figure out what the path would be to bring it in. For example, it would be really cool if the time-travel debugger let you easily go into the REPL for any frame!
## Type-Directed Autocomplete
If we see the following code:
```elm
longestName : List User -> Int
longestName users =
users
|> List.map .name
|> ...
```
In the `...` we know we want to get from `List String` to `Int` so we can suggest functions like `List.length`.
The best way to approach this problem is to try to implement the following function:
```elm
suggest : Dict String Type -> Type -> List Expr
suggest knownValues targetType =
...
type Type = Var String | Lambda Type Type | ...
-- simple representation of types
type Expr = Call String (List String)
-- in the simplest version, you suggest very minimal expressions
-- things like `List.foldl (+) 0`
```
By doing this, you decouple your work from other stuff. That means you can focus on *making suggestions*. If it goes well, integrating this with other things can be a separate project.
From there, perhaps we can rank suggestions based on which file or package they live in.
**Note:** Based on my experiences working on the Gmail team, I am very interested to see how well we can do with simple heuristics. Assuming quality is good, this approach has many benefits over machine learning. For example, you can easily special case thing, you can tweak heuristics without retraining, and the results may be more predictable for users. Point is, focus on the simplest plausible approach before diving into fancier techniques!
## Explore Monomorphizing Compilers
The [MLton](http://mlton.org/) project pioneered the “monomorphizing” compiler.
Normally functions like `map : (a -> b) -> List a -> List b` produce generic code that can work on *any* list, but if you happen to know that it is used as `map : (Int -> Int) -> List Int -> List Int` you could use a denser memory representation for the `List Int`, leading to quite serious performance improvements. Less indirection and less garbage generated!
This topic has a complex relationship to Elm because whole program optimization does not mix well with the kind of asset bundling you need to do in complex webapps.
**The ideal result is documentation.** What papers are relevant? What techniques are needed? Are there any Elm specific details or issues? This kind of thing can be shared as a blog post or on [the Elm Discourse](https://discourse.elm-lang.org/).
## Explore WebAssembly
[WebAssembly](https://webassembly.org/) will be maturing over the next few years. Without a garbage collector, it is not viable for languages like Elm. In the meantime, there are a few questions it would be good to answer:
- What are the facilities for representing UTF-8 strings? If you make it all from bit arrays and trees, we should do a “literature review” of how strings are represented in languages like JS, Java, Go, etc.
- How does WebAssembly code interact with the DOM? What does that mean for Elm’s virtual-dom library?
- How does WebAssembly interact with WebGL? What does that mean for Elm’s webgl library?
In all these cases, **the ideal result is documentation** that gets shared with [discourse](https://discourse.elm-lang.org/). Before making technical decisions and investments, certain big questions must be addressed. So it probably makes sense to do some prototyping, but the actual deliverable here is knowledge.
================================================
FILE: notes/on-general-purpose.md
================================================
# On “General-Purpose” Languages
Many people ask me “will Elm become a general-purpose language?” or “do you think Elm is a general-purpose language?” What are they talking about?
## What is a general-purpose language?
Well, the definition [from wikipedia](https://en.wikipedia.org/wiki/General-purpose_programming_language) says:
> a general-purpose programming language is a programming language designed to be used for writing software in a wide variety of application domains
So is this term useful for any real question? “Can I use Python for a wide variety of things?” Sure. But it is very crucial to know *which* things to make any technical decision.
It would be a mistake to say, “Oh, Python can be used for a wide variety of things. Great! I will use it for this embedded system.”
## Examples of “general-purpose” languages
[The list on wikipedia](https://en.wikipedia.org/wiki/General-purpose_programming_language) has quite a few entries. Let’s examine a few.
| Language | Strong For | Weak For |
|----------|-------------------|------------------|
| C | operating systems | web programming |
| Java | servers | emdedded systems |
| Python | scripting | iPhone apps |
Why is it that these “general-purpose” languages have such clear niches?
Language design choices like “how does memory management work?” inherently select certain target domains. For embedded systems, you want manual memory management! For a quick script, you do not!
Ecosystems also form around certain domains. Python may seem like an odd choice for scientific computing from a performance perspective, but it has a great ecosystem for that domain. Without the ecosystem, it would be silly to do this, and you can observe that this is true in Ruby and Perl and PHP.
## Relevance to Elm
When a young language says “X is a general-purpose language” I hear “I am not sure what X is for but I am making it anyway”. I think languages like Elm, Elixir, and Julia have had success because they focused on a particular domain and attempted to understand the *people* who work in those domains.
So why spend a bunch of time early in the life of a language to transition from “great in one domain” to “weak in three domains”? Many languages have tried this, and you probably have not heard of them because it is not a good opening strategy. Historically, languages branch out after establishing their niche. Clojure began for servers and branched out to web apps. Python began for scripting and branched out to scientific computing, education, etc.
I prefer to have a coherent strategy about expanding to other domains. Why expand to another domain? What strengths could Elm bring there? How much work would it be? Should work on web apps slow down for this? What is best for the long-term health of the project? Etc. I also think it is important to clearly inform people what the language is decent at *right now*. That way they do not waste their time on paths we already know that Python or Erlang or C can do better *right now*.
> If you want to experiment with Elm in other domains, please (1) let folks know on elm-dev as soon as possible and (2) label your work as experimental very clearly. This way it can be clearer to everyone how to use their time efficiently.
================================================
FILE: notes/on-sharing-types.md
================================================
# On Sharing Types
Imagine that we could use the same types in the front-end and backend-code. The crucial question becomes **How do we keep these types synchronized between server and clients?** Whenever any serialized type changes, we would need to force all users to refresh to the latest code.
## Challenges
Forced refreshes raise some difficult technical questions:
- What happens while the servers are transitioning? If one request is routed to a new server, but another to an old one, what happens? Does the user thrash back and forth? How can that be avoided? Maybe versions only can go “up” so there is a history of all releases? And if the request goes to an old endpoint it just fails?
- What happens when someone is writing something when this transition occurs? Does the refresh just happen and the work is lost? Do we save it somehow? The types do not match in the new code, so what format do we use?
Perhaps it is possible to resolve these challenges. That could be really cool. My point is only that “having types on both sides” alone does not magically resolves the challenges of coordinating client and server code.
You may be thinking that these problems seem hard, but how often does it really matter?!
## Frequency
Say you are releasing new code once a day. With a naive implementation, that means everyone gets forced refreshes every day as well.
Let’s try to be more clever though. Say we have a hash that represents all types that may be serialized between client and server. **But how do I know which types those are?** Every server endpoint would need to be statically known by the compiler. `/user/profile` returns a `Profile`, `/comment/INT` returns a `Comment`, etc. What design permits that?
Let’s try to be less clever. Say we have a hash that represents all types used in the whole program. That could reduce the number of forced refreshes, but with extreme caveats. Adding a new `Msg` or new field in your `Model` would force a refresh.
Again, the point here is that “just run Elm on node.js” or “just run Elm on BEAM VM” does not address the important and difficult questions that could make “sharing types” valuable in practice.
================================================
FILE: roadmap.md
================================================
# Roadmap FAQ
People want to know what is going to happen with Elm. Hopefully this document helps!
- [What is the timeline?](#what-is-the-timeline)
- [Can I use Elm on servers?](#can-i-use-elm-on-servers)
- [When will Elm compile to X?](#when-will-elm-compile-to-x)
- [Where is the localStorage package?](#where-is-the-localstorage-package)
If you are looking for ways to contribute, please check out [the suggestions](README.md) in this repo.
## What is the timeline?
**There is no timeline with exact dates.**
A lot of people think that (1) languages are just features that need implementing, (2) these features should be placed on a calendar, and (3) we should throw people at the features until they are done early. This is not how Elm has developed!
First, when it comes to language design, I think doing things *right* is far more valuable than doing things *right now*. It is easy to think short term and rush features, but languages live for decades, and there are no take-backs on mistakes.
Second, looking back, playfulness and inspiration have been crucial parts of Elm’s development. For example, the friendly error messages began with a project to provide JSON output to code editors. There was no huge demand for JSON output, but after spending a bit of time on the project, it became clear that the error messages could be dramatically improved. No one knew that was possible! That means no one was asking for it either! So by any reasonable prioritization scheme, this was a “pointless” project, but it turned into an extremely valuable part of Elm.
Point is, I think scheduling progress is kind of a silly idea. “That conceptual breakthrough is scheduled for September!” Listening and experimenting are a part of the process. It has been extremely valuable to Elm so far, and I expect that to continue to be true.
## Can I use Elm on servers?
**No, but it is an area of interest.**
Many users would really like to run Elm on servers in a direct way. "Take Elm code and have it run on servers." I think this naive path seems promising due to a couple common misconceptions about running the same language on the front-end and back-end:
* Folks want to share types. No more JSON! That *sounds* amazing, but what happens when a user keeps a tab open for a while? If you share types directly, changing types will break their code. Creating a serialization format that can evolve with your product is a very hard problem, [as elaborated upon here](notes/on-sharing-types.md), and simply “having types” does not solve it.
* People want server-side rendering of HTML. Running a server all in Elm does not really give you this. I mean, it can, but it is way more complicated than having a tool that can take Elm modules and spit out HTML. The simpler version can work with any existing server, which is why that is the approach being explored by various tools created by community members.
* Folks like to think languages are “general-purpose” and if the language is good, it should be good at *anything*. People say C++ is a general-purpose language, but it is not so nice for web apps. People say Python is a general purpose language, but it is not so good for operating systems. And you cannot just “make Python good at operating systems” by compiling it to C. More notes on “general-purpose” as a questionable concept [here](notes/on-general-purpose.md).
All that said, I think there could be something very promising in this area if it is done in a very particular way. It is something that I have been thinking about for many years now, and I am exploring some compilation techniques that might make a project in this general area worthwhile.
## When will Elm compile to X?
**Compiling through X probably does not accomplish what you hope.**
Many folks tell me “Elm should compile to X” where X is a thing they like. Here are people suggesting [Go](https://twitter.com/zvozin/status/847860742787223553), [Lua](https://groups.google.com/d/msg/elm-dev/Mi9j3nVD5NE/11akZGmNAgAJ) and [Erlang](https://groups.google.com/d/msg/elm-dev/Mi9j3nVD5NE/Pf1GXS2QAgAJ). But why not go through OCaml or C# or Java or Scala or F# or Haskell or ES6 or C++ or Rust or node.js?
**The hard part of supporting a domain is not the compiler. It is making a good ecosystem.** Python is nice for scientific computing because of things like NumPy and SciPy, not because of whatever backend. Elm is nice for front-end because of the ecosystem, like the HTML library and The Elm Architecture, not the particulars of code generation. **Just putting a typed functional language in a domain does not mean it will be fun and productive in practice!**
So say we choose to go through Erlang to make Elm a server language too. Great. Now the question is “how do you write a typed server with no side-effects that is (1) true to the spirit of Elm and (2) feels nicer than other server languages?” That's the hard question, and the answer may be that it cannot be done. So writing a compiler backend is like 5% of what it takes to make a project like this worthwhile. The ideal form of a project like this probably has its own bespoke backend, possibly going to assembly directly, but I think the harder work is the API design of figuring out "how does Elm interact with a database in a nice way?" and "how is security handled?"
Point is, I think hacking some code generator together would be extremely time consuming and risky, but not very rewarding. Community members have figured out how to run projects like `elm-optimize-level-2` and `elm-spa` that get at these considerations, but without embarking on a project that is intractable. I recommend trying something more like that, and looking through [these projects](README.md) for inspiration on ways to contribute to the Elm ecosystem through code.
## Where is the localStorage package?
**We ask that people use [ports](https://guide.elm-lang.org/interop/ports.html) for aspects of [the web platform](https://platform.html5.org/) that do not have Elm packages yet.**
First, many people think expanding “web platform” support is easy. “Just copy the JS API into Elm as tasks!” The whole point of Elm is to rethink common problems and try to do better. Figuring out how to do it “the right way” in a typed functional language with no side-effects takes time. For example, Elm existed for more than *two years* before we added HTML support. Imagine what people were saying to me then! But now, rather than having a fragmented ecosystem of competing HTML libraries of varying quality, we have one library that is great. For many Elm users, the fact that there are clear defaults that work well within the ecosystem is a huge draw.
Second, the general policy is to prioritize things that *cannot be done* over things that *could be done better*. Obviously it would be great if the whole Web Platform was available in Elm today, but anything that is missing appears to be possible with [ports](https://guide.elm-lang.org/interop/ports.html) or [custom elements](https://guide.elm-lang.org/interop/custom_elements.html).
Third, a great deal of work has actually gone into `localStorage` and IndexedDB already, but the results were not good enough to be released. If a library like this is released, it will need to be supported forever because we cannot just switch to a better API when we figure it out. Programmers may store important information on thousands of computers out in the world, and losing that data could hurt their business. So until we can offer a really solid approach here (one that is cross-browser, will grow with your application, and can upgrade browser data with new releases) we prefer to be clear that we do not offer this yet. I suspect that work on the prior two questions may reveal something worthwhile, but it is still quite uncertain.
Maybe these prioritization choices do not appeal to your sensibility or today’s tradeoffs do not seem to work for your scenario. That is fine! It is totally reasonable to circle back to Elm after Web Platform support has expanded and see if it works better for you then.
> **Note:** Many people wonder “why not have the community expand the web platform?” I have tried to address this question [here](https://groups.google.com/d/msg/elm-dev/1JW6wknkDIo/H9ZnS71BCAAJ) and [here](https://groups.google.com/d/msg/elm-dev/bAHD_8PbgKE/X-z67wTdCAAJ) and [here](https://discourse.elm-lang.org/t/native-code-in-0-19/826). There are other languages out there that make different choices on this question, and I recommend checking them out. They may suit you better!